diff --git a/.gitattributes b/.gitattributes index a0643702b3..e594c0e9b7 100644 --- a/.gitattributes +++ b/.gitattributes @@ -138,6 +138,12 @@ *.heic filter=lfs diff=lfs merge=lfs -text *.hif filter=lfs diff=lfs merge=lfs -text *.avif filter=lfs diff=lfs merge=lfs -text +*.hvc1 filter=lfs diff=lfs merge=lfs -text +*.hvcc filter=lfs diff=lfs merge=lfs -text +*.ivf filter=lfs diff=lfs merge=lfs -text +*.y4m filter=lfs diff=lfs merge=lfs -text +*.yuv filter=lfs diff=lfs merge=lfs -text +*.bit filter=lfs diff=lfs merge=lfs -text ############################################################################### # Handle ICC files by git lfs ############################################################################### diff --git a/HEIF_IMPLEMENTATION_PLAN.md b/HEIF_IMPLEMENTATION_PLAN.md new file mode 100644 index 0000000000..116bb50eb3 --- /dev/null +++ b/HEIF_IMPLEMENTATION_PLAN.md @@ -0,0 +1,1805 @@ +# AVIF and AV1 implementation plan + +## Goal + +Complete a production-quality, fully managed AV1 codec and its bounded AVIF/HEIF image container integration for ImageSharp. The finished work must decode and encode still images and bounded image sequences, preserve source precision, use ImageSharp memory ownership, and provide SIMD-first hot paths with one behaviorally identical scalar fallback. + +This plan is the authoritative delivery checklist. A source file, unit test, build, self-roundtrip, or local implementation is not completion evidence by itself. + +## Takeover audit: 2026-09-05 + +The earlier checked boxes and measurements below are historical checkpoint reports, not accepted conclusions about +the current encoder or complete decoder. The fresh production-path audit is still in progress. No benchmark has +been run during this investigation, and the complete 738-file upstream diff has not yet received a line-by-line audit. + +### Reference and worktree evidence + +- Live `git ls-remote` identifies official `https://aomedia.googlesource.com/aom` main as + `d565eec60f084421fa34fc0534b760c6452b6a6c`. The export at `D:\GitHub\ynse01\aom-d565eec6-source` + was compared with that revision's official archive: 1,522 files present, 18 byte-identical, 1,504 differing only + by CRLF versus LF, and zero remaining content differences. The archive is temporary and outside this repository. +- Live ImageSharp main and local `upstream/main` both resolve to `adb982081a7e89a824f873f7f286f517e04f80dd`. + The starting HEAD is `a7f0fca6b01d0d498862aba66e024942393c1614`; the index is empty. The initial worktree + contains 13 modified files and two untracked benchmark files. Preserve all existing work while correcting demonstrated defects. +- The optimized native build is `D:\GitHub\ynse01\aom-d565eec6-build-x64-release`: Ninja, MSVC x64, + Release `/O2 /Ob2 /DNDEBUG`, runtime CPU dispatch, decoder, encoder, and high-bit-depth support. + Generated `config/aom_config.h` enables SSE2, SSE4.1, AVX2, and AVX512. The cache's zero-valued HAVE entries + do not describe the generated configuration. The version header reports 3.15.0 but does not independently identify a commit. +- Commit `a7f0fca6b` already contains temporary native integration: + `tests/ImageSharp.Benchmarks/Codecs/Heif/Native/aom_benchmark.c`, its `CMakeLists.txt`, + `LibaomBenchmarkEncoder.cs`, and the referencing sequence benchmark. The current untracked decoder benchmark + and adapter wrapper are also temporary integration. Do not stage or commit them. Removal from existing commits + or deletion of local reference files requires a separate, concrete proposal; no history rewrite or deletion is authorized here. + Existing generated conformance fixtures also require classification before any cleanup proposal. + +### Confirmed implementation deviations + +Managed paths below are relative to the repository; reference paths are relative to the verified libaom export. +Line numbers describe the inspected starting tree, before subsequent corrections. + +| Class | Managed evidence | Official reference evidence | Finding | +| --- | --- | --- | --- | +| Missing functionality | `Av1FrameEncoder.cs:372-408`, under `src/ImageSharp/Formats/Heif/Av1/Pipeline` | `av1/encoder/encoder.c:641-646`; `av1/av1_cx_iface.c:287-288,1284-1286,1561-1562` | Sequence setup unconditionally disables CDEF, restoration, and intra-edge filtering. These are not equivalent to the reference's configured tool decisions. | +| Architectural deviation | `Av1FrameEncoder.cs:508-542,1551-1557` | `av1/encoder/encode_strategy.c:168-230,1664-1669` | Every frame is error resilient, refreshes all slots, disables frame-end CDF publication, and resets probabilities. The reference selects retained primary-reference state. | +| Missing functionality | `Av1IntraSuperblockEncoder.ModeDecision.cs:185-245`; `Av1IntraSuperblockEncoder.ReferenceModeDecision.cs:526-566` | `av1/encoder/partition_search.c:3320` onward; `av1/encoder/rdopt.c:6196-6236` | Inter frames retain a fixed 8x8 partition tree and search only LAST. Larger partitions and additional reference roles are not implemented by this path. | +| Architectural deviation | `Av1IntraSuperblockEncoder.ModeDecision.cs:597-609,662-672,824-834` | `av1/encoder/rdopt.c:111-142,6186-6236`; `av1/encoder/intra_mode_search.c:1291-1344` | Managed coding finishes intra search before inter evaluation. Reference inter search has its own ordered candidates, pruning state, bounds, and later intra evaluation. | +| Architectural deviation | `Av1IntraSuperblockEncoder.ReferenceModeDecision.cs:576-617` | `av1/encoder/rdopt.c:111-142` | Managed single-reference mode order is NEAREST, NEAR, GLOBAL, NEW. The reference default order is NEAREST, NEW, NEAR, GLOBAL across eligible references. | +| Architectural deviation | `Av1IntraSuperblockEncoder.ReferenceModeDecision.cs:1278-1414` | `av1/encoder/mcomp.c`; caller policy in `av1/encoder/motion_search_facade.c` | The managed radius is an effort-shifted value capped by its border; each scale visits eight offsets once. This is a simplified search controller whose full reference-policy reconciliation remains open. | +| Missing functionality | `Av1TransformBlockEncoder.cs:1047-1097` | `av1/encoder/encodemb.c:842-885` | Lossy transform coding ends at fast quantization. Reference coding selects quantization with trellis policy and can optimize coefficients before reconstruction. Native primitive arithmetic alone does not establish encoder parity. | +| Architectural deviation | `Av1IntraSuperblockEncoder.ModeDecision.cs:1581-1655` | `av1/encoder/intra_mode_search_utils.h:622-657`; `av1/encoder/intra_mode_search.c:467-491,1597-1619` | The 8x8 SATD screen imports only the 1.5-best threshold. The reference also maintains ranked candidates and quantizer/neighbor-dependent pruning, and owns the surrounding mode/transform decisions. Contrary to an initial audit hypothesis, this revision's `intra_model_rd` does return raw SATD. No modeled-RD-versus-SATD numerical defect is established. | +| Missing functionality | `Av1TransformBlockEncoder.cs:695-837` | `av1/common/reconintra.c:958-986,1204-1243` | Encoder directional prediction bypasses edge preparation and upsampling. Flipping the sequence flag alone would make encoder reconstruction disagree with its emitted syntax. | +| Verification gap | `Av1InverseTransformerFactory.cs:47-60,98-112`; `Av1InverseTransformTests.cs:402-532` | Sparse inverse dispatch in `av1/common/idct.c` and `av1/common/x86` | The uncommitted decoder path specializes DC-only DCT; other lossy EOB values still use full transforms. Its new tests compare against the managed full transform, not an independent native oracle. Complete sparse dispatch and SIMD reconciliation remain open. | + +Two reconstruction-input defects were established and corrected during this audit: + +- Rectangular intra transforms require width plus height samples on each extended edge. The starting encoder + prepared twice the width above and twice the height to the left + (`Av1IntraSuperblockEncoder.ModeDecision.cs:1466-1534,2583-2677`; + `Av1IntraSuperblockEncoder.ChromaModeDecision.cs:179-182,1153-1223`). + This could expose unprepared scratch samples to directional prediction. Reference + `av1/common/reconintra.c:1149-1184,1451-1488,1817-1820` copies the available adjacent edge and repeats its endpoint + through the width-plus-height extent. Luma and chroma now reuse the existing edge-preparation method, and tiled + candidate preparation follows the same extent without adding storage or an allocation. +- Trial and final geometry discarded the enclosing partition, and all encoder directional edge-availability calls + passed `None` (`Av1IntraSuperblockEncoder.ModeDecision.cs:397-418,858-921,1434-1464,2557-2581` in the starting tree). + Reference `av1/common/reconintra.c:158-192,343-379` selects different availability tables for mixed vertical + partitions; `av1/decoder/decodeframe.c:1392-1416` distinguishes split child nodes from mixed-partition leaves. + Both trial and final setup now retain the terminal partition in existing mode information, and luma, chroma, + and tiled prediction consume it. Split children retain their own implicit `None` leaf state. + +The isolated 8x8 Hadamard pruning hook has been removed from production mode selection. Its primitive and existing +component tests remain uncommitted in the worktree. Removing the unsupported hook does not complete the remaining +encoder controller or establish a quality or performance improvement. + +Disabled tools, limited search, and different decision order can also change reconstructed samples without +producing an invalid bitstream. They are separate from the reconstruction-input defects above. + +### Architecture and verification findings + +Reference-edge correction checkpoint `578ec34d9`, verified on 2026-09-05: + +- Final Release .NET 11 build after removing the screening hook: zero errors and zero reported warnings + on that incremental build. The preceding test compilation reported 1,009 existing warnings. +- Visual Studio VSTest 18.9, .NET 11 preview 7, serialized collections, one test thread, stop-on-failure: + 251/251 cases passed in `Av1EncoderFrameTests`, `Av1IntraSuperblockEncoderTests`, and `HeifEncoderTests`. + The final report is `D:\GitHub\ynse01\av1-takeover-20260905\no-screen-final.trx`. +- `RectangularIntraReferencesExtendTheLastAvailableSample` checks explicit reference-edge samples at + 8/10/12 bits for both rectangle orientations and available/unavailable extensions, including output sentinels. +- `ProductionMixedPartitionsPreserveReconstructionOrder` requires actual mixed partitions and compares the live + mapped encoder partition state and retained reconstruction with production decoding. Its two emitted 32x32 + monochrome streams also match the optimized libaom decoder: 2,048 luma samples, maximum error 0, + and zero samples exceeding one. +- The twelve freshly regenerated two-frame color streams cover 8/10/12-bit 4:2:0, 4:2:2, and 4:4:4: + managed and optimized native decoding agree on all 21,348 Y/U/V samples, maximum error 0, zero exceeding one. + Per-frame and per-plane counts are in `D:\GitHub\ynse01\av1-takeover-20260905\decoder-comparison.json`. +- These are bounded reconstruction and same-bitstream decoder checks. They do not prove separately encoded + output parity, complete encoder control flow, or decoder-wide conformance. No benchmark was run. + Temporary launch scripts, native comparison output, logs, and reports remain local and are excluded from commits. + +Tile-reader construction ownership correction, verified on 2026-09-05: + +- At checkpoint `330d4c4ea`, `Av1TileReader.cs:279-316` acquired frame syntax storage before constructing + above/left neighbor contexts. Failure in the above constructor bypassed cleanup; failure in the left + constructor returned only the above context. Both leaked the completed `FrameInfo` owner. This is a + demonstrated lifetime defect, independent of reconstruction quality and encoder search completeness. +- Reference `av1/common/alloccommon.c:370-408,411-452` keeps partial above-context allocations reachable + from common state; decoder destruction calls `av1_remove_common` (`av1/decoder/decoder.c:239`), which + frees those contexts (`alloccommon.c:501-506`). Native contexts are reused until dimensions outgrow them + (`av1/decoder/decodeframe.c:5125-5135`); the managed per-frame allocation lifetime remains a separate + architectural deviation. This correction only restores cleanup at the existing managed owning boundary. +- The new regression failed when the ninth allocator request was rejected: all eight successful frame-state + allocations had no matching return (`tile-ownership-red.trx`). Serialized VSTest stopped on that failure. + The existing first catch now also covers above-context construction, and the left-context catch returns + both preceding owners. No new buffer, owner, guard, or native dependency was introduced. +- Four standalone-reader cases cover monochrome/color and 64/128 superblocks, reject every allocator request + in turn, and require exactly one return per successful allocation. After the final edit, the focused tiling, + reference-store, reference-motion, and decoder conformance set passed **137/137** through serialized + Release .NET 11 Visual Studio VSTest in 1.8500 minutes (`tile-ownership-final.trx`). The final build had + zero warnings/errors; Roslynk reported zero compiler errors. Reports remain in the temporary directory + `D:\GitHub\ynse01\av1-takeover-20260905`, outside the repository. No benchmark was run; this does not + establish separate-encoder sample parity or complete decoder ownership/reference-lifetime equivalence. + +Sequence-construction ownership correction, verified subsequently on 2026-09-05: + +- At checkpoint `578ec34d9`, `Av1FrameEncoder.cs:1492-1560,1668-1713,1778-1824` allocated common state and + frame owners without unwinding partial construction. `Av1EncoderPictureBuffer.cs:87-164` and + `Av1SymbolEncoder.cs:270-274` had the same problem inside their multi-owner constructors. A failed constructor + never returns an instance to the caller's `using` statement. This is a demonstrated lifetime defect, independent + of encoder quality. Reference `av1/encoder/encoder.c:1480-1499` clears compressor state and invokes + `av1_remove_compressor` if construction fails. +- The new production-factory regression failed before the correction: rejecting the second allocation left the + first allocation unreturned. The first failure stopped VSTest as configured; evidence is + `D:\GitHub\ynse01\av1-takeover-20260905\allocation-failure-before.trx`. +- Common sequence state, both sample-width frame constructors, picture state, and symbol state now unwind + completed owners at their own construction boundaries. Picture context views borrow its two owners + (`Av1NeighborArrayUnit.cs:65-70,135-141`), so failed picture construction releases those owners directly. + Common cleanup avoids dispatching into derived frame disposal before derived construction begins. + Nullable disposal checks are restricted to owners that can be absent after an allocation failure; no new + owners, buffers, copies, or native dependencies were introduced. +- Six color/alpha cases at 8/10/12 bits reject every allocator request in turn and require exactly one return + for every earlier successful request. All six pass. The final affected frame, superblock, and public encoder + set passes 257/257 cases through serialized Release .NET 11 Visual Studio VSTest with stop-on-failure. + The report is `D:\GitHub\ynse01\av1-takeover-20260905\ownership-final.trx`. +- The final incremental Release .NET 11 build reports zero warnings and errors; Roslynk reports zero compiler + errors. The twelve regenerated color sequences and two mixed-partition streams were compared again with the + optimized native decoder: 23,396 samples, maximum error 0, zero samples exceeding one. This remains bounded + same-stream decoder/reconstruction evidence, not separate-encoder parity. No benchmark was run. + +- PNG resolves options before converted metadata and sanitizes incompatible output combinations + (`src/ImageSharp/Formats/Png/PngEncoderCore.cs:1632-1675`). TIFF follows the same precedence and converts + unsupported combinations (`src/ImageSharp/Formats/Tiff/TiffEncoderCore.cs:111-175,371-457`). + HEIF's generic pixel input must retain that conversion contract; source pixel type is not an eligibility gate. +- JPEG's closed `JpegColorConverter` owns traversal and calls semantic static operator arithmetic + (`src/ImageSharp/Formats/Jpeg/Components/ColorConverters/JpegColorConverter.Operator.cs:144-250`). + Shared prediction work must follow that family boundary and existing ownership APIs. +- Encoder block scratch already shares mode and inter storage by their non-overlapping lifetimes + (`Av1EncoderBlockWorkspace.cs:31-103`). The sequence constructor retains picture, coefficient, block, + entropy, conversion, and frame state (`Av1FrameEncoder.cs:1492-1646`). Exact sizing, failure unwinding, + alignment, reference parity, and allocation attribution still need the complete native comparison. +- Two decoders agreeing on one managed bitstream establishes only decoding agreement for that bitstream. + The benchmark's photographic setup checks that agreement; it does not compare separately encoded outputs. + Its native output uses ImageSharp color conversion, so it is not an independent RGB conversion oracle. +- Historical separate-encoder Y/U/V maxima of 40/30/53 fail the required one-component-unit limit. + Counts exceeding one were not supplied with those historical figures. Neither those figures nor the recorded + 1,363.09/71.73 ms timing pair is a new measurement of a subsequently edited tree. +- VSTest logs identify Visual Studio 18.9 x64 and `.NETCoreApp,Version=v11.0`. Future runs must deduplicate + child environment keys case-insensitively, disable collection parallelism, stop on failure, and run serially. + Absence of an observed dialog is not evidence that no popup occurred. + +Ordinary-intra skip investigation at checkpoint `c778217a9`: + +- Managed `Av1IntraSuperblockEncoder.ModeDecision.cs:622-667,758-829,1291-1396` scans retained empty + transforms, computes their rate again, and can replace ordinary intra coefficient syntax with block skip. + `Av1TileWriter.cs:2628-2648` explicitly describes this as a departure from current libaom. +- Verified reference `av1/encoder/rdopt.c:3516-3578` assigns ordinary intra rate including + `skip_txfm_cost[skip_ctx][0]` and sets `skip_txfm = 0` before separate IBC evaluation. + Its intra-in-inter-frame path does the same at `rdopt.c:5772-5801`. + Final coding enforces this at `av1/encoder/partition_search.c:2122`. + Reference `av1/common/blockd.h:372-374` counts IBC as inter for this decision. +- This is an encoder-policy deviation, not an invalid-bitstream claim. Suppressing empty transform symbols + changes block-skip and coefficient probability adaptation for subsequent blocks even when current pixels match. + Managed `Av1TileWriter.cs:825-852,1100-1143` consumes the selected flag in symbol order and publishes neighbors. +- The regression `PreservesIntraNonSkipForAllZeroTransforms` asserts the reference policy while retaining + all zero-EOB and precomputed-versus-live syntax checks. It failed on the fixed-DC traversal before correction: + mono intra returned `Skip = true` (`intra-skip-before.trx` in the local takeover report directory). + The first affected run then exposed a second path: `Av1IntraSuperblockEncoder.cs:214-285` independently + marked zero-coefficient blocks skipped, so its exact byte comparison with the corrected live path failed. + Both paths now preserve ordinary-intra non-skip, and their exact byte comparison is retained. + These changes enforce the demonstrated reference contract; no pixel tolerance was changed. +- Automatic approval review rejected a combined production patch and deletion of the old helper-specific + entropy test. A second review rejected their removal after verification as weakened coverage. + The unused helper and its test remain unchanged; no further removal was attempted. + With that helper and test still present, the corrected production, fixed-DC, and public encoder paths pass + 258/258 serialized Release .NET 11 VSTest cases (`intra-skip-r2.trx`). Decoded mode assertions cover ordinary + intra syntax in key/inter frames at 8/10/12 bits and all three color subsampling formats; repeated-frame + tests still require skipped inter blocks. The build has zero errors and 1,009 existing test-project warnings, + with none in the changed files; Roslynk reports zero compiler errors. + Optimized native decoding of freshly regenerated streams matches 23,396 samples: maximum error 0 and zero + samples exceeding one. This is bounded same-stream evidence, not separate-encoder parity. + Roslynk now finds only the old entropy test referencing the obsolete skip helper; no production caller remains. + No benchmark was run, and the full controller, coefficient optimization, and separate-encoder gates remain open. + +Coefficient optimization and evaluation-stage investigation: + +- `av1/encoder/encodemb.c:208-224,474-563,831-887` selects quantization and coefficient optimization + from segment and evaluation policy before publishing coefficient context and reconstruction. +- `av1/encoder/rdopt_utils.h:608-709` distinguishes default, mode, and winner evaluation: transform + pruning, default transform use, skip/DC prediction, distortion domain, coefficient optimization threshold, + and transform-size search differ by stage. Changing stage invalidates cached RD results. +- `av1/encoder/txb_rdopt.c:400-560` consumes plane/block RD scaling, transform/EOB/context costs, + quantized and original coefficients, dequantization and matrices. It can lower coefficients, move EOB, + and select an empty transform; it updates coefficients, EOB, entropy context, and rate together. + Importing this primitive without those callers and their state would leave the controller deviation unresolved. +- `aom/aomcx.h:215-221`, `av1/av1_cx_iface.c:781-782,1401-1409`, and + `av1/encoder/speed_features.c:2709-2776` show usage-specific CPU settings and feature initialization. + ImageSharp's 0-10 effort scale has not yet been reconciled with these policies. No new effort mapping is assumed. + +Further quantization, distortion, and final-packing source comparison on 2026-09-05: + +- `Av1ForwardQuantizer.cs:88-236` and `Av1ForwardQuantizer.Operator.cs:98-316` implement the no-matrix + fast quantizers. The rounding factor 64 agrees with `av1/encoder/av1_quantize.c:609-651` at sharpness zero; + the regular quantizer's factor 48 is not evidence of a fast-quantizer rounding defect. The managed path + does not implement the reference's sharpness adjustment, regular quantizer, or quantization-matrix policy. +- `Av1TransformBlockEncoder.cs:1175-1225` always ends lossy coefficient selection at fast quantization. + Reference `av1/encoder/tx_search.c:2064-2391` derives trellis eligibility from segment, evaluation stage, + normalized residual energy, and transformed SATD; chooses fast or regular quantization; measures coefficient + rate before reconstruction; selects transform-domain or pixel-domain distortion; retains the winning + coefficient buffer, EOB, type, and entropy context; then reconstructs intra neighbors. These are coupled + state transitions, not interchangeable standalone numerical primitives. +- `Av1IntraSuperblockEncoder.ModeDecision.cs:166-168` stores coded planar views. + `Av1EncoderFrame.cs:109-119` assigns those views the aligned dimensions. Consequently + `Av1IntraSuperblockEncoder.ReferenceModeDecision.cs:984-987` includes coded alignment in its model bounds. + An initial suspicion that this always violates reference clipping was rejected after following + `av1/encoder/encoder.h:4291-4309`: the reference also uses aligned dimensions when `do_border_pad` is false. + Its conditional true-frame policy is selected at `av1/encoder/encoder.c:4560-4568` and consumed by + `av1/encoder/rdopt_utils.h:361-401` and `av1/encoder/model_rd.h:256-303`. That policy is missing here. + The misleading visible-only comment was corrected; no arithmetic change or numerical-defect claim is justified + without reconciling that policy and its delta-Q/TPL prerequisites. High-bit-depth SSE rounding in the current + caller matches `av1/encoder/model_rd.h:70-108`. +- `Av1TileEncoder.cs:294-329` selects each block and writes its entropy data immediately. + `Av1TileWriter.cs:820-834,1070-1097` consumes the current palette and map, and + `Av1EncoderSuperblockWorkspace.cs:94-105` clears reusable decisions between superblocks. + Frame coefficients and EOB/type state survive in `Av1EncoderCoefficientBuffer.cs:32-59,117-150`, but these + alone cannot reproduce all final block syntax after frame-wide filter selection. +- Reference `av1/encoder/encoder.c:2787-2838,2891-2911,3787-3813` selects/applies deblocking, CDEF, + and restoration after reconstruction and before final bitstream packing. It preserves restoration boundary + rows before and after CDEF. Its palette path tokenizes selected maps during coding + (`av1/encoder/tokenize.c:182-225,264-278`) and packs those retained tokens later + (`av1/encoder/bitstream.c:1516-1535`). `TokenExtra` occupies one byte per palette sample; + `av1/encoder/tokenize.h:105-135` and `av1/encoder/encodeframe.c:1413-1430` size and retain that frame storage. + The managed single-pass lifetime is a controller/ownership deviation that must be resolved together with + delayed packing. Simply flipping CDEF/restoration flags or saving only probability state is insufficient. +- This comparison ran no benchmark or runtime test and establishes no new numerical or performance result. + +Further controller-state findings after `b2aee3036` on 2026-09-05: + +- Reference `av1/encoder/block.h:239-259`, `av1/encoder/rdopt.h:315-328`, and + `av1/encoder/partition_search.c:1555-1556` retain the winning reference-MV stack, weights, count, mode context, + and global vectors for final packing. `av1/encoder/bitstream.c:1062-1089,1133-1158,1251-1264` consumes that + retained state for inter-mode, DRL, MV, and IBC symbols. `Av1TileWriter.cs:928-975` currently rebuilds the stack + at the immediate-write boundary. Delaying the write requires preserving its decision-time state, not assuming + a rebuild against a completed frame grid is equivalent. +- Reference `av1/common/av1_common_int.h:1775-1856` derives final partition structure from the retained mode grid. + A second frame-sized partition-tree copy is therefore not required by the reference architecture. +- `Av1SymbolEncoder.cs:424-480,1183-1334` calculates candidate costs from live adaptive distributions. + Reference `av1/encoder/rd.c:82-130,602-668` fills distinct mode/coefficient cost tables, including marginal + coefficient costs needed by optimization. `av1/encoder/encodeframe_utils.c:1556-1692` updates mode, + coefficient, MV, and displacement-vector costs at separately configured superblock/row/tile frequencies; + `av1/encoder/speed_features.c:2385-2387` starts the inter cost policies at superblock frequency. + Updating CDFs after each selected block is not the same as refreshing every RD cost after that block. + This missing cost-state lifecycle is an architectural deviation. Its contribution to time or quality has + not been measured; no isolated cache or invented effort-dependent refresh policy has been introduced. +- Reference `av1/encoder/rd.c:687-853` initializes the complete cost state and combines control and speed-feature + update frequencies. `av1/av1_cx_iface.c:391-394,1342-1348` defaults controls to superblock updates and disables + MV-cost updates for all-intra configuration. These are configured policies, not a public-effort mapping. + `av1/encoder/encoder_alloc.h:58-88` omits the main MV-cost allocation for all-intra; displacement costs are + initialized only when needed (`rd.c:843-851`). Worker costs are separately owned when their update policy + requires independent state (`av1/encoder/ethread.c:1610-1641`). Managed sizing and lifetime must follow those + usage boundaries rather than adding every table to every frame or candidate. + +CDF symbol-cost floor correction, verified after `37541f7f3` on 2026-09-05: + +- `Av1ProbabilityCost.cs:47-52` previously passed each CDF interval directly to raw probability conversion. + Native `av1/encoder/cost.c:30-48` first floors symbol mass at `EC_MIN_PROB` (4, `aom_dsp/entcode.h:21`). + Raw conversion has a different domain and still permits 1 (`av1/encoder/cost.h:32-42`). Conflating the two + overcharged sufficiently rare symbols by up to 1,024 rate units, or two bits. +- CDF symbol conversion now applies the existing range-coder minimum before raw conversion. Gathered edge + partition costs (`Av1SymbolEncoder.cs:969-980,1003-1014`) use that same symbol boundary, matching native + `av1/encoder/partition_search.c:3419-3449`. Existing raw-probability behavior and tests remain unchanged. +- The new zero-mass middle-interval regression failed before correction: expected 6,656, actual 7,680 + (`symbol-floor-red.trx`). Cases at masses 0 through 4 and 8 exercise both ordinary CDF and gathered-symbol + conversion. They do not replace the missing RD-cost refresh lifecycle or coefficient optimizer. +- Final Release .NET 11 build: zero errors and 1,009 existing warnings; Roslynk: zero compiler errors. + Serialized Visual Studio VSTest passes 2,283/2,283 in 27.0448 seconds (`symbol-floor-final.trx`), covering + entropy, intra-superblock, encoder-frame, and public HEIF encoder tests. +- Optimized libaom decoding was repeated after the final edit for 23 palette, eight partition, and twelve color + sequence streams: 38,973 samples, maximum error 0, and zero samples exceeding one. Per-plane reports remain + in the temporary takeover directory. These are bounded same-bitstream checks, not separate-encoder parity, + a quality improvement claim, or performance acceptance. No benchmark was run. + +Frame/block RD and decoder filter follow-up after `aa2ecf690`: + +- The two base multiplier formulas and high-bit-depth normalization in `Av1RateDistortion.cs:122-158` match + native `av1/encoder/rd.c:371-444` for their represented key/ordinary-inter roles with default PSNR tuning. + They do not implement the surrounding frame-role, layer/boost, tuning, or block adjustments + (`rd.c:447-463,802-809`, `av1/encoder/partition_search.c:596-658`). In particular, ALLINTRA derives a + superblock modifier from the range of subblock variances (`partition_search.c:5721-5734`). + `Av1IntraSuperblockEncoder.ModeDecision.cs:172-176` uses only the segment-zero base quantizer and intra flag. + These missing policies remain architectural deviations; no isolated multiplier adjustment was introduced. +- The default `disable_trellis_quant = 3` (`av1/av1_cx_iface.c:291`) means + `NO_ESTIMATE_YRD_TRELLIS_OPT`, not final-pass-only optimization (`speed_features.c:2493-2513`). + `encodemb.h:157-162` and `tx_search.c:2084-2085,2216-2229` still allow optimization during transform + candidate evaluation under that default. Control value 2 selects final-pass-only behavior. A winner-only + optimizer would therefore leave the default production path incomplete. +- The complete optimizer traversal at `av1/encoder/txb_rdopt.c:18-560` and its cost/dequantization helpers + (`txb_rdopt_utils.h:39-204`) were followed through level lowering, EOB replacement, empty-transform selection, + signed DC handling, plane/precision/tuning scaling, and joint rate/EOB/context publication. It retains only + three nonzero positions for the EOB phase, then changes traversal after that phase's bound is exceeded. + That bound belongs to this complete traversal; it is not a general candidate-pruning threshold. + Managed `Av1SymbolEncoder.cs:1183-1334` evaluates an unchanged coefficient vector and + `Av1TransformBlockEncoder.cs:1175-1225` stops at fast quantization. Existing scratch can support parts of the + arithmetic, but cost-state policy, evaluation stages, mutation, and reconstruction must be integrated together. +- The caller policy also changes quantization, not only the decision to invoke trellis: + `tx_search.c:1964-2000,2147-2156,2216-2229` derives the MSE/SATD gates from the active evaluation stage, + switches between fast and regular quantization, then optimizes before distortion/reconstruction. + Threshold tables and default/mode/winner selection live in `speed_features.c:76-100` and `rd.h:353-381`. + Managed candidate reconstruction currently happens before `GetCoefficientCost` + (`Av1TransformBlockEncoder.cs:187-251`, `Av1IntraSuperblockEncoder.ModeDecision.cs:2494-2550`). + Adding optimization to that later cost call would reconstruct twice or leave candidate pixels stale. +- The optional residual border policy is selected in `encoder.c:4559-4568`: GOOD mode, objective delta-Q, + TPL enabled, no AQ/segmentation/ROI/QP sweep/ducky path, and sharpness other than three. + `encodemb.c:80-173` fills outside-visible residuals using a whole-block mean, per-axis mean, or zero, + depending on transform type. Thus border residual reuse across transform candidates is conditional. + This policy is not an unconditional replacement for coded-edge replication, and is not enabled for ALLINTRA. + No isolated border-padding rule was added; its configuration and candidate integration remain open. +- Transform pixel-error normalization and the modeled-rate skip comparison were also traced through + `Av1TransformBlockEncoder.cs:577-590`, `Av1RateDistortion.cs:259-307`, native + `av1/encoder/model_rd.h:70-106,162-199`, and `av1/encoder/tx_search.c:979-1051`. + No additional numerical defect was established in those formulas. The conditional border policy and + transform-domain/winner evaluation stages remain unresolved as recorded above. +- The complete `Av1CdefDecoder.cs` frame/unit traversal was compared with `av1/common/cdef.c:29-478`: + unfiltered top/left context, coded-edge sentinels, skipped-block lists, luma direction ownership, and chroma + reuse are present. The managed sequential traversal reads the still-unmodified bottom row directly; native + retains bottom lines for its worker-capable traversal. This inspection establishes no decoder-wide or SIMD + completeness claim. Decoder CDEF storage remains an operation-scoped owner, not native reusable worker state. + +Retained-state and cost-policy follow-up after `ef8b1a823`: + +- `Av1EncoderTransformBlockState.cs:12-43` uses four bytes for EOB and byte-sized transform type, leaving + one padding byte. Reference `av1/encoder/encodetxb.c:593-625,714-730` records the neighboring skip context + in bits 0-3 and DC-sign context in bits 4-5 beside each EOB. Its packer consumes those retained contexts + (`encodetxb.c:296-306,410-421`). The managed structure can represent that state without increasing its size, + but storing it alone would not implement deferred packing; no unused state was added. +- Reference palette-token allocation is conditional on non-statistics coding with screen-content tools allowed + (`av1/encoder/encodeframe.c:1413-1430`). `tokenize.h:105-135` reserves up to two full-resolution planes in + maximum-superblock-rounded storage. Tokens retain the selected context and color-order rank, including the + first raw index (`tokenize.c:174-225,264-278`, `bitstream.c:353-368`); keeping only palette colors is insufficient. +- `Av1EncoderPictureBuffer.Reset` clears the complete mode grid and packed state (`:344-363`), so it cannot + be reused as the boundary between analysis and packing. Selected prediction fields remain in the reusable + `Av1EncoderBlockStruct` workspace, while `Av1EncoderBlockModeInfo` retains only its smaller neighbor subset. + These lifetimes must be reconciled together with palette tokens, selected MV state, and coefficient contexts. +- Native cost defaults are explicit controls as well as speed features: `av1/av1_cx_iface.c:391-394,550-553` + differs between default and realtime configurations. `rd.c:724-758,824-851` combines controls with speed policy + and initializes frame costs; `encodeframe_utils.c:1629-1689` suppresses block refresh when CDF updates are + disabled. No new managed effort mapping or isolated cost-refresh threshold was introduced. + +Decoder coefficient-stage correction after `b305e6e89`, verified on 2026-09-05: + +- The source trace established an architectural deviation: `Av1SymbolDecoder.cs:1415-1459` published a + count prefix and scan-ordered quantized levels; `Av1TileReader.cs:1148-1157` packed those variable-length + groups. `Av1BlockDecoder.cs:126-153,1349-1399` then used another superblock-sized, all-plane workspace + to dequantize and reorder every transform during reconstruction. Both buffers were cleared separately. + The native entropy traversal dequantizes each signed level directly into its coefficient region + (`av1/decoder/decodetxb.c:116-165,279-312`); EOB belongs to separate metadata + (`av1/common/blockd.h:452-461`). Native region cursors advance by nominal transform area + (`av1/decoder/decodeframe.c:274-279`), independently of EOB. +- Parsing now publishes dequantized coefficients directly and records EOB in `Av1TransformInfo`. Each plane's + parser/reconstruction cursor advances by nominal transform area, including skipped transforms. Frame state + reserves 16 coefficient slots per 4x4 unit, with no count prefix. Reconstruction consumes that storage + directly; its second coefficient workspace and inverse-quantization pass are removed. These changes span + the production parser, transform descriptors, frame storage, and reconstruction caller, rather than adding + a disconnected native primitive. +- Quantization state moves to the parser. Mode syntax establishes delta-Q before `Residual` updates the + segment/plane values, matching `decodeframe.c:1172-1221`. Transform-local parameters preserve matrix + bypass, weighted-quantizer rounding, the 24-bit product mask, transform scaling before sign, and signed + precision clipping (`Av1InverseQuantizer.cs:92-135`, `decodetxb.c:52-58,298-312`). The entropy context still + uses the masked quantized magnitude and original DC sign (`Av1SymbolDecoder.cs:1424-1476`). No additional + allocator-owned buffer or native production dependency was added. +- The coefficient capacities decrease by 25.5 KiB for a 64x64 4:2:0 superblock configuration and 102 KiB for + 128x128 4:2:0: this combines removal of the second workspace with removal of count-prefix capacity. + These are source-derived coefficient-buffer sizes, excluding descriptor/object overhead, not measured + total memory or a timing improvement. No benchmark was run. +- Existing entropy tests now check the published dequantized raster values, including sparse and beyond-EOB + zeros, against fixed reference qindex-23 DC/AC values. Four added matrix/arithmetic cases use explicit + 8/10/12-bit reference values, matrix bypass for identity/one-dimensional transforms, lossless bypass, + asymmetric precision limits, product-mask wraparound, and sign-after-scaling rounding. The old matrix test + checked lengths only. These component cases do not establish complete signaled-matrix bitstream coverage. +- After the production edit, serialized Release .NET 11 Visual Studio VSTest passed **9,371/9,371** AV1 and + public HEIF encoder cases in 2.6933 minutes (`coefficient-stage-final.trx`). After adding the fixed-value + tests, a focused set passed **130/130** in 3.0217 seconds (`coefficient-stage-last-edit.trx`). Following final + whitespace cleanup, the checkpoint set passed **159/159** in 5.5189 seconds (`coefficient-stage-checkpoint.trx`). + The final build had zero errors and the existing 1,009 warnings; Roslynk reported zero compiler errors. + No production behavior changed after the broad run, and no retained reference samples were altered. +- Optimized current-reference redecoding matched the retained restoration and film-grain references across + **8,500,087** samples, maximum error **0**, zero exceeding one; those references also passed the managed + conformance tests. Twelve regenerated color sequences matched another **21,348** native samples exactly. + Reports are `restoration-comparison.json`, `film-grain-comparison.json`, and `decoder-comparison.json` in + `D:\GitHub\ynse01\av1-takeover-20260905`, outside the repository. This is bounded same-bitstream evidence, + not separately encoded output parity, complete decoder conformance, or a performance acceptance result. +- Remaining architecture differences are explicit: the managed reader still parses a complete superblock + before reconstruction (`Av1TileReader.ReadTile`, `Av1FrameDecoder.DecodePartition`) and clears its complete + coefficient regions before reuse. Native single-thread decoding interleaves parsing/reconstruction through + visitors (`decodeframe.c:935-958,2746-2765,2792-2801`), clears only through the maximum populated raster + position after inverse transform (`:154-164`), and separates parsing/reconstruction for row workers with + different buffer lifetimes (`:3244-3277`). Those traversal, clearing, and worker-lifetime differences remain + open; this checkpoint does not claim that changing coefficient representation completes them. + +Film-grain decoder source comparison after `ef8b1a823`: + +- The complete template generation, random state, autoregression, scaling interpolation, overlap traversal, + noise application, and native-sample load/store paths were compared with `av1/decoder/grain_synthesis.c`. + Managed `Av1FilmGrainDecoder.cs:874-1172,1204-1231` matches the represented rules in native `:429-629`; + all 2,048 Gaussian entries also match exactly. No new arithmetic defect was established in this comparison. +- Managed noise application processes chroma before luma (`Av1FilmGrainNoise.cs:109-174`), retaining ungrained + luma for chroma scaling. Native `grain_synthesis.c:685-745,803-862` uses that same ordering. The two-component + luma average is horizontal only; vertical chroma subsampling selects a row rather than averaging two rows. + Restricted identity-matrix chroma uses luma's upper endpoint, and high-depth lookup interpolates below entry 255. +- `Av1FilmGrainDecoder.cs:698-857` and native `grain_synthesis.c:1252-1376` exclude already-applied overlap + strips and retain right/bottom grain boundaries. The managed plane span retains existing padded storage + (`:208-216`), including addresses for empty interiors at clipped edges; no new guard or scratch plane was added. +- Grain presentation preserves references in `Av1Decoder.cs:891-909,945-1002`: refreshed frames receive a + separate presentation copy, and unreferenced shown frames can be grained in place. `CopyVisibleTo` also copies + active geometry (`Av1FrameBuffer.cs:248-258`). This source trace does not complete the decoder-wide lifetime audit. +- SIMD dispatch remains an open architecture/performance issue. `Av1FilmGrainNoise.cs:180-238,388-522,947-955` + uses AVX2 gather or a high-depth-only portable path with separate width overloads. `Av1FilmGrainOverlap.cs:142-175` + additionally gates 512-bit processing on `Vector.Count`. Neither gate has fresh end-to-end evidence here. + Comments claiming that scalar reads are slower, interpolation repays them, or `Vector` establishes processor + execution width were corrected. Runtime dispatch and arithmetic were not changed; no improvement is claimed. +- After the final comment edit, the Release .NET 11 build completed with zero errors and 1,009 existing warnings; + Roslynk reported zero compiler errors. Serialized Visual Studio VSTest passed 5/5 focused film-grain/reference + cases in 8.4512 seconds (`film-grain-audit.trx`), including existing hardware-fallback and constrained-allocation checks. +- Current optimized libaom regenerated seven still references and the two ten-frame official sequence references. + All 3,113,847 decoded samples match the retained references exactly: maximum error 0, zero samples exceeding one. + Per-frame/per-plane results are in temporary `film-grain-comparison.json`. Those references are also used by the + passing managed tests. This is bounded same-bitstream decoder evidence, not complete conformance or encoder parity. + No benchmark ran, and no fixture, native integration, or generated comparison output was added to the repository. + +RD accumulation correction after `4c7880d`: + +- Numerical defect: before this correction, `Av1IntraSuperblockEncoder.ModeDecision.cs:349-381` added + rounded child costs, `:727-731` added rounded luma/chroma costs, and `:1193-1212` added separately rounded + block syntax. Reference `av1/encoder/rd.h:32-34,208-233` keeps raw rate and distortion; rectangle and split + accumulation in `partition_search.c:3487-3504,4605-4607` rounds the combined rate. For multiplier 128, + two rates of 2 cost 1 jointly but cost 2 when rounded separately; two rates of 1 show the opposite error. +- `Av1RateDistortionStatistics.cs:9-61` now carries raw rate, distortion, and the comparison cost through + spatial, tiled, filter-intra, palette, CfL, IBC, and inter winners. Mode/transform comparisons retain their + existing strict tie rules and scalar bounds. Bounded-out split candidates retain an invalid sentinel; + only valid selected statistics are accumulated. Existing per-transform raw accumulation is preserved. +- Partition evaluation now combines those statistics in `Av1IntraSuperblockEncoder.ModeDecision.cs:336-390`; + ordinary block syntax and luma/chroma aggregation use the same retained raw inputs. This adds value state, + not a buffer or allocation. It does not implement native partition pruning, mode order, cost refresh, + quantization stages, reference control, or adaptive rate multipliers. No performance improvement is claimed. +- Seven focused regressions pass in Release .NET 11 through serialized Visual Studio VSTest + (`rd-statistics-focused.trx`, 2.6557 seconds). Four production block cases independently recount selected + syntax and pixel-domain SSE for monochrome/color and zero/nonzero residuals. Three fixed arithmetic cases + cover both rounding directions and 64-bit distortion. +- After the final C# edit, the Release .NET 11 build completed with zero errors and 1,009 existing warnings; + Roslynk reported zero compiler errors. Serialized Visual Studio VSTest passed 2,259 entropy, + intra-superblock, HEIF encoder, and reconstruction-conformance cases in 2.1328 minutes + (`rd-statistics-final.trx`), then 132 encoder-frame cases in 18.7250 seconds (`rd-statistics-frames.trx`). + The seven new cases are included in those totals. Tests encoding the existing effort policy only verify + current behavior; their success does not validate that policy against libaom. +- Optimized libaom decoding of the regenerated eight partition, 23 palette, and twelve color-sequence + streams matches all 38,973 samples exactly: maximum error 0, zero samples exceeding one. This is + same-bitstream reconstruction evidence. Separate-encoder acceptance remains unmet; no benchmark ran. + +Range-writer output-capacity correction, verified after `93aba785f`: + +- `Av1SymbolWriter.cs:213-214,339` before correction sliced a fixed initial allocation for finalization + and eight-byte flushes. Reference `aom_dsp/entenc.c:78-91,270-285` grows capacity when either needs + more room. The packet estimate in `Av1FrameEncoder.cs:544-563` does not replace that range-coder policy. + This is a demonstrated writer-capacity deviation; no production frame overflowing that estimate was established. +- The existing owner now grows at those two boundaries. Word flushes double the current tile capacity and add + eight bytes; finalization reserves its exact terminating length. Reallocation preserves finalized preceding + tiles and the current completed prefix, including bytes that can receive a backward carry. Pending bits stay + in the range state. The old owner is returned only after successful allocation/copy, and reset reuses capacity. + Existing frame aggregation remains; this does not complete native deferred-packing or worker ownership parity. +- Before correction, the zero-capacity consecutive-tile regression failed in `Normalize` with + `ArgumentOutOfRangeException` (`writer-growth-red.trx`); VSTest stopped on that first failure. + Five small initial capacities now preserve three consecutive tiles byte-for-byte against sufficient-capacity + encoding, with CDF adaptation enabled/disabled. Existing native carry assertions cover two additional + finalization-growth capacities. Allocation limits independently exercise failure at both growth boundaries, + and allocation identities verify every successful owner is returned exactly once. +- Final Release .NET 11 incremental build: zero errors and warnings; preceding test compilation: + 1,009 existing warnings. Roslynk reports zero compiler errors. Serialized Visual Studio VSTest passes + 2,292/2,292 entropy, intra-superblock, encoder-frame, and HEIF encoder cases in 26.9726 seconds + (`writer-growth-final.trx`), including the twelve focused writer cases. +- Current optimized libaom decoding of the regenerated 23 palette, eight partition, and twelve color-sequence + streams matches all 38,973 samples exactly: maximum error 0, zero samples exceeding one. + These are bounded same-bitstream checks, not separate-encoder parity or a timing/quality improvement. + No benchmark was run. All temporary native comparison output and test reports remain excluded from commits. + +Restoration processing-unit correction: + +- Before correction, `Av1LoopRestorationDecoder.cs:147-166,309-358` sized its bordered source, Wiener + intermediate, and eight-bit output bridge for a whole restoration-unit stripe, including an absorbed tail. + Only the self-guided branch split horizontally into processing units. Native + `av1/common/restoration.c:389-408,904-963,987-1054` dispatches both filters in 64-luma-sample processing + units with chroma subsampling applied. This is a traversal/sizing deviation, not a demonstrated pixel defect. +- Both branches now share that bounded traversal and scratch sizing. Source context crosses every chunk and + restoration-unit edge; replication remains restricted to the frame edge. Stripe-boundary rows retain their + deblocked provenance. Existing kernels accept the exact tail width, whereas native Wiener SIMD rounds its + final write into padded storage. Frame output ownership and the self-guided statistics boundaries are retained. +- For a luma plane at least 384 samples wide with a nominal 256-sample restoration unit, the combined ushort + scratch request calculated from the source falls from 155.47 KiB to 26.72 KiB at eight bits, and from + 107.47 KiB to 18.72 KiB at 10/12 bits. These figures exclude the destination plane and integer self-guided + scratch. They are allocation-formula results, not measured process memory or a timing improvement. +- Final verification passes nine serialized Release .NET 11 VSTest cases in 16.0008 seconds + (`restoration-grid-final-r2.trx`), including native-plane restoration, both filter + types, all three precisions, 4:2:0/4:2:2/4:4:4, super-resolution, and hardware fallbacks. No tests or expected + outputs were changed. The final incremental build reports zero errors and warnings; the preceding compilation + reported 1,009 existing warnings. Roslynk reports zero compiler errors. +- All six retained restoration references were independently regenerated with the current optimized native + decoder and match exactly: 5,386,240 Y/U/V samples, maximum error 0, zero samples exceeding one. + Per-plane results and payload sizes are in the temporary `restoration-comparison.json`. This verifies the + provenance of the exact references used by the tests; it does not establish separate-encoder parity. +- `Av1WienerFilter.cs:96-134,165-188` still computes one horizontal output using a vector dot product and + traverses vertical outputs scalarly. The optimized reference instead processes multiple outputs per vector + (`av1/common/x86/wiener_convolve_avx2.c`). That SIMD/traversal architecture and the eight-bit output bridge + remain open. No benchmark was run, and temporary native output and scripts remain excluded from commits. + +Palette coded-boundary correction, verified after `b2aee3036` on 2026-09-05: + +- The encoder clipped luma/chroma palette search to visible frame dimensions + (`Av1IntraSuperblockEncoder.PaletteModeDecision.cs:46-49`, + `Av1IntraSuperblockEncoder.ChromaPaletteModeDecision.cs:54-57` before correction), and + `Av1TileWriter.cs:1085-1086` omitted palette symbols outside those visible dimensions. This is a numerical + and syntax defect, separate from the conditional distortion-model border policy described above. +- Reference palette search (`av1/encoder/palette.c:555-597,787-804`), tokenization + (`av1/encoder/tokenize.c:229-241`), and decoding (`av1/decoder/detokenize.c:65-77`) all use + `av1/common/blockd.h:1517-1557`. Its distances come from coded mode-info dimensions + (`av1/common/av1_common_int.h:1358-1364`), not visible-pixel dimensions or the optional RD border policy. + The managed decoder already follows those coded distances (`Av1PartitionInfo.cs:190-194`, + `Av1TileReader.cs:2720-2744`). Search, rate evaluation, and writing now agree on that same extent. +- The existing 5x3 luma regression checked retained reconstruction without decoding its payload. Extending it + to decode every case exposed a truncated tile entropy stream (`palette-bounds-red.trx`). An earlier assertion + incorrectly read `FrameBuffer` after `Decode` had released the native planes; that test mistake was corrected + using `DecodeFrameBuffer` and is not codec-failure evidence (`palette-bounds-before.trx`). +- Existing palette/color/EOB/reconstruction assertions remain. Added cases cover clipped/transposed luma at + 10/12 bits and 4:4:4, 4:2:2, and 4:2:0 chroma, including one-pixel source axes. The same boundary correction + prevents those subsampled axes from creating zero-length palette input. No defensive rejection, new owner, + extra production buffer, or change to search effort thresholds was introduced. +- Final Release .NET 11 build: zero errors, 1,009 existing warnings. A preceding build stopped on a missing + blank line before a comment; it was corrected before verification. Roslynk reports zero compiler errors. + Serialized Visual Studio VSTest passes 190/190 cases in 18.4734 seconds: intra-superblock encoder, + HEIF encoder, and AV1 palette cases (`palette-bounds-final.trx`). +- Optimized official libaom decodes all 23 regenerated palette payloads with exact agreement against retained + encoder reconstruction: 985 visible Y/U/V samples, maximum error 0, zero samples exceeding one unit. + Per-plane results and output sizes are in `D:\GitHub\ynse01\av1-takeover-20260905\palette-comparison.json`. + All generated streams, raw planes, scripts, and native output remain temporary and excluded from commits. + This is same-bitstream reconstruction verification, not separate-encoder parity or performance acceptance. + +Explicit grid-sampling conversion correction, verified after `5f7bad3a6` on 2026-09-05: + +- `HeifEncoderCore.Sequence.cs:97-109` promoted incompatible odd-grid sampling only when the public option + was unspecified; `HeifEncoderCore.cs:1078-1081` then rejected the same dimensions for explicit 4:2:0/4:2:2. + The established TIFF conversion contract (`TiffEncoderCore.cs:371-444`) resolves unsupported option + combinations before encoding, including explicit choices. The existing HEIF fallback already demonstrates + that these source dimensions can be preserved through 4:4:4 conversion. +- The extended existing public grid regression first passed default sampling and then failed on explicit + 4:2:0 with that exact exception. VSTest stopped on the failure; see `grid-sampling-before.trx` in the local + takeover report directory. Option resolution now promotes explicit and default incompatible sampling alike + before matrix/profile selection. The redundant private grid guard was removed: its only caller obtains the + resolved settings first (`HeifEncoderCore.cs:1040-1062`), and multi-frame AV1 input follows the sequence path. +- All original descriptor, extent, edge-replication, hidden-item, reference-order, and exact-pixel assertions + remain. Four cases additionally inspect both emitted cell headers and preserve the source profile object. + Final serialized Release .NET 11 Visual Studio VSTest passes 67/67 HEIF encoder cases in 6.2836 seconds + (`D:\GitHub\ynse01\av1-takeover-20260905\grid-sampling-final.trx`). The final source build reports zero errors + and warnings; the preceding test compilation reports 1,009 existing warnings. Roslynk reports zero compiler errors. + No benchmark or separate-encoder measurement was run. Decoder rejection tests for invalid input grids remain unchanged. + +Identity-matrix option correction after `fff06700d`: + +- Unsupported restriction: `HeifEncoderCore.Sequence.cs:129-156` required BT.709/sRGB metadata for every + identity-matrix 4:4:4 encode and otherwise substituted BT.601. Reference + `av1/encoder/bitstream.c:2457-2494` permits other primaries/transfer descriptions with an explicit range + bit. The special BT.709/sRGB branch alone infers full range. Reference validation requires unsubsampled + identity planes (`av1/av1_cx_iface.c:921-928,1005-1013`), not that special color description. +- The production writer already has the correct syntax branches (`ObuWriter.cs:397-446`), and the shared + semantic identity operator already maps G/B/R with the luma range on every plane + (`HeifColorConverter.IdentityOperator.cs:17-139`). Option resolution now preserves valid identity + descriptions and their explicit range; only BT.709/sRGB identity normalizes limited to full range. + Existing conversion for incompatible sampling remains. No converter, buffer, or public API was added. +- Before correction the first public regression failed with expected Identity versus emitted BT.601 + (`identity-profile-red.trx`); VSTest stopped on that failure. After the final edit, Release .NET 11 + built with zero errors/warnings and Roslynk reported zero compiler errors. Serialized Visual Studio + VSTest passed 81/81 HEIF encoder cases in 6.4887 seconds (`identity-profile-final.trx`). +- Fourteen new cases cover BT.2020/PQ full/limited range at 8/10/12 bits, stills/sequences, and special + sRGB range inference. They inspect emitted syntax, container metadata, preserved source metadata, + and every decoded RGB component with a one-unit limit for range conversion. Optimized libaom decoding + matches all 4,032 independently calculated GBR samples exactly: maximum error 0, zero samples exceeding + one (`identity-comparison.json`, temporary and outside the repository). This is bounded conversion and + same-bitstream evidence, not separate-encoder parity or performance evidence. No benchmark ran. + +Color-conversion boundary correction after checkpoint `f7bd907d6`, verified on 2026-09-05: + +- `HeifEncoderCore.Sequence.cs:74-194` resolved output sampling and preserved reversible YCgCo matrix metadata, + including when default 4:2:0 or requested 4:2:2 was incompatible. The shared converter's established contract + rejects that combination at `HeifColorConversionParameters.cs:265-282`. The public save regression failed + on default sampling with that exact exception (`matrix-fallback-before.trx`). +- PNG and TIFF resolve incompatible options through conversion at `PngEncoderCore.cs:1640-1654` and + `TiffEncoderCore.cs:378-444`. HEIF now extends its existing identity-matrix fallback to incompatible + YCgCo-Re/Ro sampling, converts with BT.601, and writes the matching matrix metadata. Explicit 4:4:4 remains + eligible for the existing reversible operator. This changes encoder option resolution, not decoder acceptance. +- Seven public cases retain the original profile object and values, inspect decoded matrix/range metadata, + and require byte-identical output to the same packed pixels explicitly encoded with the fallback matrix. + The cases include the original identity fallback and YCgCo-Re/Ro with default, 4:2:0, and 4:2:2 sampling. +- A separate lifetime defect existed at `Av1FrameEncoder.cs:1404-1434`: conversion parameters were resolved + after renting row storage. The internal-factory regression confirmed that rejected conversion had already + made one allocator request (`conversion-allocation-before.trx`). Resolution now precedes storage allocation; + the regression requires both allocation and return logs to remain empty. No new guard or owner was added. +- Final Release .NET 11 build: zero reported warnings and errors on the incremental build. Roslynk reports + zero compiler errors. Serialized Visual Studio VSTest with stop-on-failure passes 265/265 affected cases + (`conversion-final.trx` in the local takeover report directory). + The regenerated color and partition streams again match optimized native decoding on all 23,396 samples, + maximum error 0 and zero exceeding one. No benchmark or separate-encoder parity comparison was run. + +Color/output source coverage and remaining limits: + +- `Av1YuvConverter.cs:24-135,175-274` dispatches byte/high-bit-depth, complete/cropped/scaled output, and alpha + through shared HEIF adapters. `HeifPlanarColorConverter.cs:39-320,329-870` was read through both traversals: + conversion owns reusable row scratch, interpolates chroma before matrix conversion, and uses existing + `PixelOperations` packing/unpacking or Rgb48/Rgba64 conversion. +- `HeifColorConverter.Operator.cs:164-431` uses closed semantic operators and descending SIMD widths. + These architecture observations are not proof that every H.273 operator or pixel format is numerically correct. + Independent color-conversion and complete SIMD coverage remain open. +- The reference codec interface exposes native planes and strides (`aom/aom_image.h:284-292`). + The temporary comparison adapter supplies I420 using the managed RGB conversion. Its output cannot independently + validate that RGB conversion, even when both codec decoders agree on native planes. + +Intra-reference frame-extent correction after checkpoint `182f39ae5`, verified on 2026-09-05: + +- Source comparison found that extension availability and extension length had been conflated. + `Av1IntraSuperblockEncoder.ChromaModeDecision.cs:1211-1240` and + `Av1IntraSuperblockEncoder.ModeDecision.cs:2393-2449` previously copied a complete adjacent extent whenever + its coding-order availability flag was true. Reference `av1/common/reconintra.c:1737-1742,1817-1820` + also clips the available count to the remaining coded frame extent, then repeats the final available + sample in the edge preparation at `reconintra.c:1149-1184`. +- The new 56x56 production partition case failed before the correction with `ArgumentOutOfRangeException` + at the top-right copy (`edge-extent-before.trx`: two existing 32x32 cases passed, then execution stopped + on the new failure). This is an implementation defect, not a search-performance hypothesis. + Bottom-edge reads also require the bound: `Buffer2DRegion{T}.cs:89-97` limits row width but resolves the + row index against the backing buffer, whose encoder border can contain samples outside the coded region. +- Both shared luma/chroma references and tiled candidate references now bound adjacent samples by the + plane's coded extent before endpoint repetition. The existing availability rules and allocation ownership + remain the governing contracts; no new guard, rejection policy, owner, or scratch buffer was introduced. +- The rectangular reference cases retain all twelve earlier checks and add six explicit clipped-extent + cases across 8, 10, and 12 bits and both orientations. They use a larger backing buffer with distinct + values beyond the coded region, fixed expected edge sequences, and destination sentinels. + The production mixed-partition test retains both 32x32 orientations and adds both 56x56 orientations. +- Final Release net11.0 build: zero errors and 1,009 existing warnings. Serialized Visual Studio VSTest: + **273/273 passed** in `edge-extent-final.trx` (20.3789 seconds), covering encoder frames, intra-superblocks, + HEIF encoder contracts, and the retained empty-transform cost-helper test. Roslynk reports zero compiler errors. +- Fresh optimized-reference decoding of four regenerated partition streams matches all 8,320 retained luma samples. + Twelve regenerated moving color streams match all 21,348 Y/U/V samples. Combined maximum error is **0** across + **29,668** samples, with **0** samples exceeding one. These are same-bitstream decoder/reconstruction comparisons; + they do not establish separate-encoder parity or performance. No benchmark was run. + +Intra-edge integration after checkpoint `2424ff9f9`, verified on 2026-09-05: + +- Reference `av1/av1_cx_iface.c:333,1561-1562` enables intra-edge filtering by default and propagates it to + sequence configuration (`av1/encoder/encoder.c:641-647`). `Av1FrameEncoder.cs:402` now enables that syntax. + CDEF and restoration remain disabled and unresolved. The starting-tree findings above remain historical evidence. +- Encoder `Av1TransformBlockEncoder.cs:739-800,875-937` now prepares directional edges before prediction. + Luma mode trials, selected-mode transform refinement, split luma transforms, tiled planes, and chroma candidates + propagate both the sequence flag and the neighboring smooth-mode class. Raw references remain separate from + candidate copies; filtering does not mutate references used by subsequent mode or transform trials. +- Neighbor selection at `Av1IntraSuperblockEncoder.ChromaModeDecision.cs:1035-1081` follows native + `av1/common/av1_common_int.h:1359-1415` for the luma units that own subsampled chroma neighbors and + `reconintra.c:958-986` for smooth-mode classification. Inter winners can retain a previous intra trial's UV field + (`Av1IntraSuperblockEncoder.ReferenceModeDecision.cs:925-933`), so that field is only meaningful for an intra neighbor. +- `Av1IntraEdgePreparation.cs:39-116` shares the complete corner/filter/upsampling order between encoder and decoder. + Native `reconintra.c:1132-1147,1204-1243,1512-1548` defines the missing-sole-edge early return and directional + preparation. Strength thresholds follow `reconintra.c:989-1026`; half-sample selection follows `reconintra.h:148-155`. + The shared code preserves a missing sole edge's constant value rather than interpolating its distinct corner. +- `Av1IntraEdgeFilter` and `Av1IntraEdgeUpsampler` have separate closed generic traversals and semantic readonly + operators, with descending 512/256/128-bit widths and scalar tails. Smoothing uses rounded nonnegative kernels; + upsampling uses signed [-1,9,9,-1] arithmetic, rounding, clipping, and linear interleaving. Native definitions are + `reconintra.c:1028-1082,1349-1381`. Inline comments explain endpoint padding, lane ordering, bounds, and scaling. +- Each candidate borrows existing transform scratch (`Av1EncoderBlockWorkspace.cs:143-144`) until prediction and + residual formation finish. Two 160-sample edges retain native prefix sizing; only required edges are copied. + Smoothing uses 132 samples including three endpoint padding positions. Upsampling needs exactly the native + 19 samples, including corner and endpoint extension; vector reads no longer require a larger padded window. + Decoder scratch is 4,548 short samples (about 8.88 KiB), replacing its previous 4,576-sample workspace. + No new owner or per-candidate allocation was added. This source-level sizing result is not a timing claim. +- Existing independent scalar kernel tests now cover all SIMD tiers, lengths around lane boundaries, extrema, + and exact scratch capacities. Eight added preparation cases distinguish smooth-neighbor thresholds and missing + sole edges in both orientations. Production tests assert the emitted sequence flag; mixed-partition tests retain + the four unfiltered cases and add four filtered cases without weakening partition or reconstruction assertions. +- Final Release net11.0 build: zero errors and 1,009 existing warnings. Roslynk: zero compiler errors. + Serialized Visual Studio VSTest passed **314/314** in `intra-edge-final.trx` (30.7293 seconds), including encoder + frames, intra-superblocks, transform-block contracts, predictor SIMD tiers, native decoder fixtures and fallbacks, + HEIF encoder contracts, and the retained empty-transform cost-helper test. +- Fresh optimized-reference decoding of eight regenerated partition streams matches all 16,640 retained luma samples. + Twelve regenerated moving color streams match all 21,348 Y/U/V samples. Combined maximum error is **0** across + **37,988** samples, with **0** exceeding one. These remain bounded same-bitstream reconstruction comparisons; + separate-encoder sample parity, complete decoder coverage, and end-to-end performance are still unverified. + No benchmark was run. Temporary scripts, native output, and reports remain outside the commit. + +### Required completion gates + +Frame-context investigation after checkpoint `1639550a0`: + +- `Av1SymbolEncoder.cs:355-369` resets tile CDFs to quantizer-band defaults. `Av1TileEncoder.cs:281-337` + does that between tiles and retains only their output offsets and lengths. Reference + `av1/encoder/encodeframe.c:1456` initializes each tile from the unchanged frame context, then + `av1/encoder/bitstream.c:4074-4079` signals the largest encoded tile as the update source. + `av1/encoder/encoder.c:4489-4495` copies that tile's CDFs, resets observation counters, and stores them + with the reconstructed reference frame. Encoder publication is missing in the managed path. +- Primary-reference selection is coupled to the reference-role and frame-layer controller + (`av1/encoder/encode_strategy.c:168-230`), not simply the last encoded frame. The managed encoder retains + only the preceding reconstruction and always writes global-motion models relative to identity + (`ObuWriter.cs:1206-1226`). Enabling primary-reference reuse requires reconciling those inherited models, + loop-filter deltas, segmentation, refresh slots, and entropy state together. No encoder frame-context flag + or reference policy was changed during this investigation. +- Existing `Av1FrameEntropyContexts.BeginFrame`, `Av1FrameEntropyContext.CopyFrom`, and `SnapshotTo` + already implement decoder base/working/published state and counter reset. They are existing reusable + contracts to consider when implementing encoder publication; a new ownership framework is not justified. + +Independent loop-filter delta entropy correction, verified on 2026-09-05: + +- The context-family comparison found a numerical decoder defect. Before correction, + `Av1SymbolDecoder.cs:921-924` always read `DeltaLoopFilterAbsolute`, including the per-channel loop at + `Av1TileReader.cs:3065-3067`. Native `av1/decoder/decodemv.c:749-765` instead selects independent + `delta_lf_multi_cdf[lf_id]` distributions for multi-delta syntax and the shared CDF otherwise. + Defaults are in `av1/common/entropymode.c:844-851`; counter reset is in `av1/common/entropy.c:166-169`. +- A new regression encodes independent channel histories using explicit reference defaults and signed magnitude + syntax from `av1/encoder/bitstream.c:323-353`. The original decoder failed on the third symbol, returning + **-1 instead of -2** (`delta-lf-before.trx`, stopped on the first failure). +- Four independent distributions now participate in prototype construction, deep copying, default restoration, + and frame snapshot counter reset. The existing tile loop passes its parsed multi-delta flag and channel index + to the symbol reader. No per-symbol owner, allocation, new guard, or rejection policy was introduced. +- Tests cover four color channels, two monochrome channels, shared-delta syntax, disabled CDF adaptation, + signed escape magnitudes, independent copies, counter reset, and unchanged shared-delta defaults. + Final Release net11.0 build: zero errors and 1,009 existing warnings; Roslynk: zero compiler errors. + Serialized Visual Studio VSTest passed **2,095/2,095** in `delta-lf-final.trx` (1.8771 minutes), covering + entropy tests, frame-context lifecycle tests, and the AV1 reconstruction conformance suite. +- Current libaom normal encoding sets `DEFAULT_DELTA_LF_MULTI` to zero (`av1/common/enums.h:73`, + `av1/encoder/encodeframe.c:2357`). Existing native output must not be assumed to exercise multi-delta syntax. + A complete independently authored multi-delta bitstream remains a verification gap. This correction does not + establish encoder parity or complete decoder correctness. No benchmark was run. + +Loop-filter level clipping correction after checkpoint `77f535828`, verified on 2026-09-05: + +- Following decoded delta values into deblocking found another numerical defect. At the preceding checkpoint, + `Av1LoopFilterDecoder.cs:373-390` clipped the reference adjustment before adding the mode adjustment. + Native `av1/common/av1_loopfilter.c:95-101,182-187` clips their combined result once, both for per-block + delta-LF and for the precomputed frame-level table. Opposite adjustments must be allowed to cancel before clipping. +- The production-frame regression failed before correction: base level 1, reference delta -63, and mode delta +63 + should retain level 1, but premature clipping produced level 63. An expected sample of 100 became 104 at the first + differing position (`loop-level-before.trx`); later edge samples also differed. VSTest stopped on that first failure. + The local runner then failed to print xUnit's Unicode arrows under cp1252. The saved report was inspected and the + runner's stdout encoding corrected; the failed test was not rerun before the implementation change. +- `Av1LoopFilterDecoder.GetFilterLevel` now adds both adjustments using the scale derived from the original level, + then clips once. Existing base/delta-LF and segmentation clipping remain in their normative order. + No allocation, ownership, or syntax policy changed. +- The existing production-frame test and its three expected outputs are retained. Four new cases exercise cancelling + deltas at both limits, with delta-LF present and absent, through `DecodeFrame` and an independent scalar filter. + Final Release net11.0 incremental build reported zero errors and zero warnings; the earlier test compilation + reported 1,009 existing warnings. Roslynk reports zero compiler errors. +- Serialized Visual Studio VSTest passed **11/11** in `loop-level-final.trx` (5.7519 seconds): deblocking scalar/SIMD + definitions, production-frame delta cases, and native deblocking, CDF-update, profile, and all-intra fixtures. + The earlier 2,095-case report applies to the preceding checkpoint. No benchmark or separate-encoder parity + measurement was run, and the complete multi-delta bitstream verification gap remains open. + +Motion-controller investigation continued after correction checkpoint `578ec34d9`: + +- Managed `Av1IntraSuperblockEncoder.ReferenceModeDecision.cs:1278-1493` uses the same normalized squared-error + plus complete mode/vector RD cost for integer and fractional candidates. The integer operators at + `Av1IntraSuperblockEncoder.Operator.cs:466-493,985-1014` compute squared error, not SAD or centered variance. +- Reference `av1/encoder/mcomp.c:72-130,184-238,314-384,644-664` separates full-pixel SAD cost, variance cost, + SAD-per-bit scaling, error-per-bit scaling, and their motion limits. Its full-pixel dispatcher + (`mcomp.c:1768-1903`) owns the configured diamond/hexagonal/pattern search and conditional mesh search, + including downsampled-SAD fallback. +- Reference `av1/encoder/motion_search_facade.c:150-324,347-488` derives the starting search step, considers + configured start candidates, retains a second full-pixel candidate, prunes repeated dynamic-reference searches, + and can refine and compare both candidates. Fractional search + (`mcomp.c:3266-3337`) has configured precision, iteration count, repeated-position tracking, and a second-level + check. The managed one-ring-per-scale controller does not implement that path. +- The next motion implementation must reconcile configuration, limits, start-candidate lifetime, search costs, + full-pixel traversal, fractional traversal, and winner publication together. Substituting SAD or variance alone, + increasing the radius, or adding isolated search points would not establish that contract. No benchmark or + motion-search implementation change has been made from this follow-up investigation. + +- Reference good-quality speed policy is layered rather than a radius lookup. The defaults at + `av1/encoder/speed_features.c:2353-2364` select NSTEP, full eighth-pixel precision, two subpixel iterations, + and eight-tap search. Good-quality overrides at `speed_features.c:1248-1250,1308-1312,1370-1409` change + iteration count, search range, full-pixel and fractional methods, second-candidate refinement, and mesh pruning. + Resolution-dependent speed-six overrides at `speed_features.c:1029-1076` also select block-size-dependent + search and reference-candidate pruning. These source observations do not make existing managed effort values + equivalent to native cpu-used values. +- Encoder intra-edge filtering requires the complete candidate prediction path. Reference + `av1/common/reconintra.c:958-986,1204-1243,1512-1548` derives neighbour-dependent strength, filters the corner + and required edges, then upsamples before directional prediction. The decoder already performs these stages + at `src/ImageSharp/Formats/Heif/Av1/Prediction/Av1PredictionDecoder.cs:915-966`. Encoder luma, chroma, and + tiled candidates must consume the same filtered-reference contract before the sequence flag can be enabled. + The decoder's chroma smooth-neighbour test at `Av1PredictionDecoder.cs:1820-1837` does not repeat the native + inter-block check, but `Av1TileReader.cs:2037-2042` resets every inter block's UV mode to DC. That owning + invariant prevents stale smooth modes; no redundant guard or numerical defect is justified here. +- Sparse inverse dispatch remains incomplete. Reference + `av1/common/x86/highbd_inv_txfm_avx2.c:4088-4180` derives separate horizontal and vertical nonzero extents + from EOB and chooses low-one, low-eight, low-sixteen, or full DCT/ADST axis kernels as applicable. + Managed `Av1InverseTransformerFactory.cs:47-60,98-112` only distinguishes DC-only DCT from the full lossy + transform. The DC arithmetic at `Av1Inverse2dTransformer.cs:44-57` retains separate axis scaling and rounding; + agreement with the managed full path still does not independently establish all native sparse cases. + +- [ ] Finish the full production-path and complete upstream-diff audit, including conversion, animation, + ownership, filters, decoder SIMD, and independent validity of the claimed tests. +- [ ] Reconcile frame configuration and encoder decision policy with the reference before isolated pruning changes. +- [ ] Implement missing tools and complete reference, partition, motion, transform, coefficient, and winner decisions. +- [ ] Compare separately encoded results from identical source samples with explicitly reconciled settings. + Report maximum absolute error and counts exceeding one for every decoded output component and every frame. + The acceptance limit is one component unit per sample; PSNR and average error cannot replace it. +- [ ] Verify each final relevant edit with focused serialized Release .NET 11 Visual Studio VSTest and independent + native production-output checks. Compilation and component tests do not close codec completeness. +- [ ] Run equivalent end-to-end benchmarks only after the relevant source comparison justifies the next change. + Retain output sizes, absolute times, per-sample errors, memory units, reference configuration, and limitations. +- [ ] Inspect the staged diff before every verified checkpoint commit and exclude all temporary native integration, + codec sources, binaries, build directories, and generated comparison artifacts. Do not push. + +## Source authority + +- AV1 codec syntax, tables, fixed-point arithmetic, prediction, transforms, entropy behavior, filters, encoder decisions, and lifecycle behavior must be ported and checked only against the current `main` branch of the official libaom checkout at the verified `D:\GitHub\ynse01\aom-d565eec6-source` export. +- Libaom is the sole external codec implementation source. Do not use HM, libheif, FFmpeg, GPAC, SVT-AV1, dav1d, libgav1, or any other codec implementation as an algorithm, arithmetic, output, or architecture reference. +- Existing ImageSharp and JPEG code is authoritative only for ImageSharp architecture, allocator ownership, SIMD dispatch, pixel conversion, and test API patterns. It is not an alternate AV1 algorithm source. +- Production code must not load, invoke, install, or fall back to a native codec. +- Existing independent container files may be used only as interoperability inputs. Native AV1 expected output must be generated by the current libaom `main` checkout, and no independent decoder output may substitute for it. + +Reference checkout evidence refreshed on 2026-09-05: + +- The current encoder source comparison uses the official libaom `main` revision `d565eec60f084421fa34fc0534b760c6452b6a6c`, exported at `D:\GitHub\ynse01\aom-d565eec6-source`. +- The earlier generic reference decoder was `D:\GitHub\ynse01\aom-d565eec6-build-generic2\aomdec.exe`; the fresh correction comparisons use the optimized x64 Release build recorded above. Its CMake cache identifies the current source export above, and it reports version 3.15.0. On 2026-09-05 it accepted both frames of each retained-reference sequence at efforts five, seven, eight, and nine. This establishes syntax acceptance for those four streams, not complete interpolation or codec conformance. + +## Status notation + +- [x] Recorded checkpoint: the associated dated report claims focused verification. Historical marks outside the takeover section have not been accepted by the fresh audit and do not establish current-tree completeness. +- [~] Locally implemented, checkpoint open: production source exists, but current-tree verification is missing or a known audit issue invalidates the checkpoint. +- [ ] Remaining: the production behavior is absent, incomplete, or has not reached its required implementation boundary. + +## Current source reconciliation + +Reconciled with the worktree on 2026-09-05. + +- [~] The bounded container reader, still-image path, sequence parser, AV1 decoder, color pipeline, presentation pipeline, and broad AV1 test suite exist locally. +- [x] The inter-frame decoder has verified checkpoints through inter deblocking decisions and + reference/mode deltas. +- [~] Loop filtering, CDEF, super-resolution, restoration, film grain, layered presentation, alpha composition, and color conversion exist locally. Shared-source cleanup changed the current tree, so final production-path verification is open. +- [~] AV1 writer primitives, forward transforms, symbol encoding, and tile-writing source are connected to the public encoder for bounded still-image and all-intra sequence AVIF color with optional auxiliary alpha output. +- [~] The public AV1 encoder has local single-image, grid, lossless sequence, and LAST_FRAME lossy sequence paths. Current-tree verification remains open. Additional references, compound prediction, remaining inter tools, orientation handling, and default format registration remain open. +- [x] Patented codec production code, registrations, tests, benchmarks, fixtures, reference outputs, and notices were manually deleted and committed by `78a74d448`. +- [x] Remaining task-created HM, HEVC, libheif, GPAC, Nokia, FFmpeg, Pillow HEIF, libavif-build, and libjpeg-build directories were traced to their creation commands in the recovered Codex session history and deleted on 2026-08-31. The user-provided repositories and all libaom-only source, build, and reference data were left untouched. +- [x] The PNG metadata-suppression fix and three HEIF/AV1 diagnostic-save call-site corrections passed the exact 34 net11.0 ARM CI cases and were committed with the single-reference checkpoint as `54bb6cbe59bd113058854a3ee31448cf61f462ca`. They are infrastructure evidence, not decoder or encoder completion evidence. +- [ ] The complete decoder and encoder release matrix is not complete. + +## Immediate execution queue + +Historical interpolation-search reports from before the takeover follow. Their timings and test counts apply only to the trees identified by those reports. The takeover audit and required completion gates above determine current work. + +- [~] The earlier elementary-stream decoder comparison included parsing, reconstruction, output allocation, RGB conversion, and disposal for both ImageSharp and optimized current-main libaom. Eight-bit Kodak and ten-bit Cosmos inputs match every `Rgb48` sample before timing. Initial warmed means are 15.130 versus 5.423 ms and 27.655 versus 10.246 ms respectively. These expose an open decoder gap; they are not container-load measurements. Reproduction and limitations are recorded in the benchmark README. +- [~] Uncommitted shared inverse reconstruction uses EOB to select a DC-only DCT path, preserving both axis roundings, rectangular normalization, input clamps, and final clipping through the existing semantic output operators. Vector512 output was added to that operator contract. Every transform size, signed boundary, padded separate/in-place destination, and supported sample precision is checked against the full transform. The post-change photographic encoder payload remained byte-identical; decoder RGB output remained exact against libaom. Short decoder measurements do not yet establish a statistically significant improvement. +- [ ] The isolated fixed-8x8 Hadamard screening hook was removed in correction checkpoint `578ec34d9`. The following is a historical experiment, not an active implementation or an accepted improvement. Existing tensor and transpose APIs vectorize the operation over frame-reused scratch; no per-candidate allocation or custom hardware-width operator was introduced. A five-warmup, ten-measurement repeat records 1,363.09 ms versus native 71.73 ms, about 34% faster than the initial managed baseline but still approximately 19x behind native. Output increased slightly to 11.577 KiB and aggregate native-plane PSNR declined from 37.124 to 37.069 dB. This tradeoff does not close the performance/compression gate. Larger-block screening, top-ranked pruning, transform bounds, winner refinement, filter decisions, and allocation attribution remain open. Evidence: `artifacts/BenchmarkDotNet/av1-screen-verified-short-20260905/20260905-140358`. +- [~] The earlier screening/DC/native-profile/moving-color subset passed 25 cases in each of three separately configured VSTest hardware tiers. Normal-path verification passes 84 screening/DC/superblock cases and 165 frame/public encoder cases. The photographic benchmark now additionally requires exact ImageSharp/libaom RGB agreement across all three dependent frames before timing. No expected image or golden output was changed. Release test/benchmark builds have zero errors and their existing 1,009/39 warning baselines. Evidence and the corrected process-local VSTest environment handling are recorded in `tests/ImageSharp.Benchmarks/Codecs/Heif/README.md` and `artifacts/TestResults/av1-screen-20260905`. + +- [~] The earlier reference benchmark measured RGB-to-OBU boundaries, including pixel conversion for every frame on both sides. A benchmark-only C adapter links the optimized current-main libaom build in-process and borrows the existing converted plane storage; production remains fully managed. Setup and file I/O are excluded equally. Native quantizer bounds are fixed to the managed base index, with independent speed settings and explicit size/quality reporting. Reproduction, native build provenance, lifetime documentation, and exact output hashes are in `tests/ImageSharp.Benchmarks/Codecs/Heif/README.md`. +- [ ] The corrected benchmark exposes a substantial remaining performance and compression gap. For three photographic 256x256 frames, ImageSharp effort seven takes 2,053.31 ms and writes 11.54 KiB at 37.124 dB aggregate native YUV PSNR; current-main libaom cpu-used six takes 70.67 ms and writes 8.27 KiB at 38.942 dB. Both include RGB conversion and both measured outputs decode to all three complete frames. This is fixed-base-quantizer evidence, not equal-quality evidence. The managed path records 9.38 MiB of managed allocations per operation, which still requires attribution; native memory is not measured by that counter. The Short-run evidence is `artifacts/BenchmarkDotNet/av1-sequence-rgb-fixed-q-short-20260905/20260905-131149`. Power-plan and CPU-query warnings remain documented. The new benchmark files build in net11.0 Release and have no Roslyn compiler/analyzer diagnostics. Do not close the interpolation-performance gate or advance to additional reference tools until the gap is addressed. +- [x] The requested in-progress tree was committed as `433afd1a9` before further encoder work. That commit is a checkpoint, not a claim of completed interpolation or codec delivery. +- [x] The subsequent partition/interpolation/lossless correction was committed as `7cf7fc4` after the 225-case affected encoder run and exact current-main native comparison. +- [x] Odd-sized color sequence verification exposed and corrected two further production defects. Empty inter luma transforms now retain the inferred DCT type before chroma inherits it; normalizing only during writing was too late. Region-major coefficient writing now rounds chroma end coordinates in 4x4 units, preserving the shared minimum chroma transform on sub-8x8 partitions instead of truncating it away. Both rules match current libaom's transform-type inference and `av1_write_intra_coeffs_mb` region bounds. The corrections add no allocation or sample copy. +- [x] All 12 moving-color sequence cases pass with SIMD enabled and disabled: 8/10/12-bit 4:2:0 at effort eight, and all three bit depths across 4:2:0/4:2:2/4:4:4 at effort nine. The 23x19 sources require actual inter motion and subsampled chroma phases. Current-main libaom decodes all 24 emitted frames with exact native Y/U/V equality. Evidence: `artifacts/TestResults/av1-interpolation-color-sequence-20260905/color-sequence-r3.trx`, `color-sequence-scalar-r3.trx`, and the adjacent raw plane outputs under `tests/Images/ActualOutput/Heif/Av1/SequenceEncoderPreservesNativeColorPlanesWithSubpixelMotion`. The affected public/frame/superblock set passes 237 cases; the expanded encoder/entropy set passes 2,524, with no failures or skips. Evidence: `artifacts/TestResults/av1-interpolation-partition-broad-20260905/color-broad-r3.trx` and `color-encoder-entropy-r3.trx`. The final Release build and Roslyn compiler/analyzer passes have no errors. This closes the identified subsampled inter syntax/reconstruction gaps, not the remaining end-to-end performance and complete codec release matrix. +- [x] Production interpolation verification now forces Smooth and Sharp at effort eight, both dual-filter axis orders at effort nine, and native 10/12-bit two-axis half-sample motion. The tests assert actual retained filter symbols, vectors, exact reconstruction, and no allocator rent during tile coding. The sequence fixture now uses the retained-reference decode contract; the still-image buffer-transfer API intentionally releases the reference map and cannot decode dependent samples in succession. +- [x] This verification exposed a live partition traversal defect: after search changed an earlier node's child count, a later node could consume an unrelated entry from the initial flat 8x8 skeleton. Unsearched intra partitions now derive their default from the current block size, preserving the geometry-driven traversal used by current libaom. Wide and tall lossless regressions cover clipped parents and superblock boundaries at efforts nine and ten. +- [x] The affected frame encoder, intra-superblock encoder, and public HEIF encoder set passes all 225 cases on the corrected tree with zero failures or skips. Evidence: `artifacts/TestResults/av1-interpolation-partition-broad-20260905/partition-broad-r3.trx`. This is the affected encoder surface, not the full codec release matrix. +- [x] Lossless tiled luma and chroma search no longer evaluate unsignaled angle deltas on 4x8/8x4 coding blocks; tiled chroma also respects the ordinary effort limits. Partition trials publish lossless chroma coefficient contexts at 4x4 transform granularity. The extended native RGB-plane regression exposed the chroma angle defect as a real lossless mismatch reproduced by libaom, and passes after the correction without changing expected samples. +- [x] All 20 focused partition/interpolation/lossless cases pass with hardware intrinsics enabled and all 20 pass with them disabled through serialized net11.0 Release Visual Studio VSTest with stop-on-failure. Current-main libaom `d565eec60f084421fa34fc0534b760c6452b6a6c` decodes all 20 emitted streams (28 frames) with exact native-plane equality against the retained reconstruction or lossless source planes. Evidence: `artifacts/TestResults/av1-interpolation-partition-interpolation-20260905/partition-interpolation-r3.trx`, `partition-interpolation-scalar-r3.trx`, and `artifacts/av1-partition-interpolation-net11-20260905-r3.log`. Roslyn compiler and analyzer passes report no errors or changed-file warnings. Subsampled inter-plane coverage, broader current-tree verification, and end-to-end performance remain open. +- [x] The 8x8 single-candidate SAD, four-candidate SAD, and variance paths now share closed-generic traversal in `Av1ResidualBuilder`. Its existing byte/ushort residual operators own scalar and SIMD arithmetic; the superblock operators no longer duplicate these row loops or hardware dispatch. The four-candidate path retains one source load/conversion per row, exact eight-sample loads preserve final-row bounds, and both variance moments retain native precision until the existing normalization boundary. Six known-result cases cover 8/10/12-bit signed extrema, distinct source/prediction rows and strides, unaligned starts, exact final-row lengths, and four-candidate output order. Together with nine existing intra-block-copy cases and the extended zero-allocation test, all 16 pass with hardware intrinsics enabled and all 16 pass with them disabled. All 58 public HEIF encoder cases also pass. Verification used serialized net11.0 Release Visual Studio VSTest with stop-on-failure; evidence is `artifacts/TestResults/av1-search-metrics-20260905/search-metrics-r2.trx`, `search-metrics-scalar-r2.trx`, and `search-metrics-encoder-r2.trx` in that directory. The exact Release build has zero errors and the existing 1,009 warnings; Roslyn reports no diagnostics in the changed files. This verifies the search-metric refactor, not the remaining interpolation conformance or end-to-end performance work. +- [~] Effort eight searches the common regular, smooth, and sharp interpolation families; efforts nine and ten enable independent vertical/horizontal filter selection. Lower efforts retain the fixed regular-filter path. +- [~] Filter ranking follows the curve-fit prediction-error model in the official `d565eec60f084421fa34fc0534b760c6452b6a6c` source, before full transform search. The existing rate-distortion type owns the static model tables and paired portable SIMD cubic evaluation. Visible-plane SSE uses the existing SIMD residual reduction, including native-bit-depth normalization and cropped edges. +- [~] Candidate and retained prediction views alternate within the existing inter workspace. At most one normalization copy per active plane retains the chosen predictor for transform search; no new pixel owner, coefficient owner, per-block rent, or reconstructed-frame copy is introduced. Zero-phase axes retain only the cheapest signaled filter instead of repeating equivalent prediction trials. +- [~] The selected filters, skip flags, segment, and primary reference fit the original seven-byte block-mode record and eight-byte macroblock record. Filter costing and writing use the live tile CDFs. Encoder and decoder share the context-combination mapping, and encoder search and writing share filter-symbol eligibility. +- [~] The earlier 67-case net11.0 Release set covered model curve samples and skip decisions, 8/10/12-bit quantizer normalization, exact entropy bytes and live adaptation, packed-field independence, tile-boundary contexts, syntax eligibility, and sequence-header signaling. Its 12 model cases also passed with hardware intrinsics disabled. The four flat retained-reference sequences were accepted by current libaom but did not force non-regular filters. The production, allocation, and exact native-plane evidence above extends that coverage; subsampled inter planes and end-to-end timing remain required before closing this checkpoint. +- [x] Focused runtime verification exposed an invalid low-effort inter-frame header: `force_integer_mv` was set while screen-content tools were disabled, making the writer omit the high-precision flag that a conforming reader expects. The frame encoder now retains the inferred false flag and controls integer-only search through the existing effort boundary. All four retained-reference cases assert the parsed precision, quantizer, and filter fields and decode both frames; current libaom accepts the same saved streams. +- [x] The preceding interpolation checkpoint's affected net11.0 Release run passed 2,494 cases with zero failures or skips through one serialized Visual Studio VSTest process with stop-on-failure enabled. It combined 2,269 entropy cases, 167 frame/superblock/transform/picture-storage cases, and 58 public HEIF encoder cases. That build had zero errors and the existing 1,009 test-project warnings, with none in the changed files. Evidence: `artifacts/TestResults/av1-interpolation-20260905/verified-encoder-entropy-r10.trx` and `artifacts/av1-interpolation-net11-release-20260905-r10.log`. This predates the search-metric refactor above and is not the complete current-tree decoder/encoder release matrix. +- [x] Allocation regressions now reflect the implemented lifetimes instead of the former layout: all seven trailing tile-state integers are proved contiguous within the picture's second allocator owner; coefficient level, context, and output owners are proved allocated at construction, reused across costing, writing, finalization, and frame resets, and returned exactly once. The two-owner picture and three-owner symbol-encoder limits are retained. +- [x] Grid regressions use valid 4:2:2/4:2:0 syntax and the inferred monochrome subsampling flags used by production configuration. Odd-dimension and undersized-cell failures assert their grid-specific messages, so malformed AV1 fixtures or an earlier configuration mismatch cannot satisfy those tests. Smaller right/bottom color and alpha cells, separate primary roots, lossless sequences, metadata, and public precision/sampling cases pass in the 58-case set above. + +Work must proceed in this order. Do not skip to a later item while an earlier checkpoint is open. + +### 1. Finish and verify the AV1-only cleanup + +- [x] Remove production types, registrations, constants, parser branches, properties, tests, benchmarks, fixtures, reference outputs, notices, and documentation for removed codec work. +- [x] Remove downloaded non-libaom reference source, tools, generated outputs, and local installations. +- [x] Retain the official current-main libaom checkout and libaom-only build artifacts required for AV1 verification. +- [x] Retain user-supplied AV1 fixtures and their recorded expected outputs. +- [x] Audit production source, tests, benchmarks, assets, project files, notices, and documentation for stale removed-code references. +- [x] The cleanup and cICP tree built in Release for net10.0 and net11.0 with restore disabled, build servers disabled, and one MSBuild node. +- [x] The exact 34 net11.0 ARM CI failures pass after the cICP correction, and the subsequent single-reference checkpoint set passes on net10.0 and net11.0. +- [x] Roslynk, scoped StyleCop, whitespace, and `git diff --check` accepted the cleanup and cICP checkpoint. +- [x] The cleanup and cICP evidence was recorded and committed with the single-reference checkpoint. + +Historical cleanup evidence from 2026-08-30, retained with its limitation: + +- Release source builds passed for net10.0 and net11.0 with zero warnings and zero errors. Both builds used `--no-restore`, `--disable-build-servers`, and one MSBuild node. +- The focused net10.0 HEIF decoder, encoder, metadata, sequence-parser, and AV1 reconstruction set passed 221 of 221 tests with zero failures and zero skips. It did not execute the net11.0 diagnostic-save path that later failed in CI. +- The Roslyn compiler and configured StyleCop analyzers accepted the changed production source. Roslynk's `open_solution` entry point was attempted separately but failed before returning a solution handle, so no Roslynk result is claimed. +- The tracked-source text and filename audit found no removed-code references outside the unchanged repository and shared-infrastructure `.gitattributes` patterns. A later history reconstruction found ignored task-created reference directories that this audit missed; those directories were deleted on 2026-08-31. +- `git diff --check` passed and neither `.gitattributes` file changed. + +Current cICP failure correction evidence from 2026-08-31: + +- The failure was not decoded HEIF metadata. `PngEncoderCore.WriteCicpChunk` ignored `PngChunkFilter.ExcludeAll`, so diagnostic PNG saves attempted to write a non-identity source matrix that PNG cannot represent. +- `PngEncoderCore` now honors the existing `SkipMetadata` contract for cICP, and the three affected HEIF/AV1 diagnostic saves explicitly use `PngEncoder { SkipMetadata = true }`. Actual comparisons and decoded-image metadata assertions remain unchanged. +- The direct embedded-ICC case and every row of the 12-case profile matrix passed: 13 of 13 net11.0 Release cases. +- The exact 34 cases reported by CI passed: 34 of 34 net11.0 Release cases, with zero failures and zero skips. +- Roslynk reported zero compiler errors after the fix, and `git diff --check` passed. +- The remaining four branch-introduced `GC.AllocateUninitializedArray` calls are removed from AV1 configuration, pixel-information, XMP, and Exif ownership boundaries. Each retained value still receives exactly one array and one copy because its source span belongs to pooled storage; no second materialization was introduced. The exact net11 Release rebuild remains at 1,005 warnings and zero errors, 166 focused configuration and metadata cases pass, and all 9,181 HEIF tests pass through direct VSTest. + +Recovered task-history evidence from 2026-08-31: + +- The primary session beginning on 2026-08-24 was reopened from task ID `01a03239-831b-7831-84e7-7f6947279ccb`: 96,777 records, 295 turn contexts, 211 compactions, 190 user messages, 1,920 assistant messages, and 13,671 tool calls. +- The continuation beginning on 2026-08-27 was reopened from task ID `01a04314-f1c6-7133-b1bc-5c74a94dd714`: 61,129 records at the audit point, 166 turn contexts, 96 compactions, 223 user messages, 1,113 assistant messages, and 8,942 tool calls. +- The restored first session records the user selecting official AOM/libaom as the AV1 source after the ImageSharp discussion was inspected. It does not authorize another codec implementation as an AV1 source and does not authorize importing a patented codec. +- The restored tool calls identify the exact creation commands for the non-libaom source, tool, and output directories removed on 2026-08-31. No directory was selected for deletion from its name alone. +- The recovered Git sequence establishes that `78a74d448` removed the patented codec implementation and `92fa7a8ca` merged the later upstream ImageSharp changes. The current branch and worktree, not an older summary, remain authoritative. + +### 2. Correct the single-reference inter-frame checkpoint + +The checkpoint is complete through `c4b4e4e0386328dea574a884b6fa36c360ad5a9b`. It replaces frame-sized palette maps with fixed decoder-session scratch, reconstructs each superblock before reusing that scratch, and passes the ownership, documentation, full AV1 test, and Release source-build gates on both target frameworks. + +- [x] Reconcile interpolation-filter syntax in `Av1TileReader` with current libaom `main`. + - Current libaom `av1_is_interp_needed` calls `is_nontrans_global_motion`, whose loop rejects only `TRANSLATION`. Identity GLOBALMV therefore omits switchable-filter symbols. + - Current `Av1TileReader` uses the same non-Translation classification. The existing Identity test leaves sentinel filter symbols unread, while the Translation test consumes them. + - No production change is required. The focused test describes only the syntax behavior it proves. +- [x] Reconcile both spatial single-reference extension loops in `Av1ReferenceMotionVectors` with current libaom `main`. + - Current libaom `setup_ref_mv_list` stops both loops at `MAX_MV_REF_CANDIDATES`, which is two. `MAX_REF_MV_STACK_SIZE`, which is eight, is the stack capacity used by the earlier direct and temporal candidate collection; it is not the stop condition for these two extension loops. + - Current `Av1ReferenceMotionVectors` uses the same two-entry stop condition and retains an eight-entry stack for earlier candidates and DRL selection. + - No production change is required. This remains spatial single-reference extension, not temporal extension. +- [x] Establish and enforce the contiguous frame-plane invariant used by `Av1FrameBuffer` and inter reconstruction. + - One ImageSharp allocator owner now contains the aligned Y, U, and V storage, matching libaom's frame-buffer ownership while non-owning `Buffer2D` views preserve ImageSharp's row API. Coded dimensions are aligned to eight samples, the luma stride is aligned to 32 samples, and chroma strides and heights are derived from that luma layout exactly once. A 4K eight-bit 4:2:0 frame owner occupies about 17.3 MiB. + - The single owner removes the previous three-rent constructor and its allocation-cleanup `try/catch`. `Av1FrameBuffer` rejects external geometry whose complete aligned frame reaches the contiguous `int.MaxValue` boundary before allocation, making every direct `DangerousGetSingleSpan` call an enforced owner invariant. + - `ConstructorRequestsContiguousPaddedPlanes` proves that a frame larger than the allocator's group capacity remains one group. `ConstructorUsesOneFrameOwnerForAllPaddedPlanes` proves exact one-rent Y/U/V ownership and exactly-once return. `ConstructorRejectsPaddedPlaneThatCannotBeContiguous` proves that an unrepresentable frame is rejected before allocation, and the high-bit-depth stride regression proves the 608-sample libaom layout for a three-pixel coded row. + - The complete HEIF/AV1 namespace passes 8,808 of 8,808 direct net11 VSTest cases in Release after the physical layout change. The production path performs no plane copy and no per-block, per-row, or per-scanline allocation. +- [x] Prove the real `Av1BlockDecoder.DecodeBlock` inter-reconstruction branch. + - Decode the progressive dependent-frame fixture through the complete public production path. + - Compare the final frame's native Y, Cb, and Cr planes exactly with current-main libaom output. + - Compare the final presented image through the established ImageSharp reference-image comparison API. + - Do not substitute an internal helper test, fake tile reader, non-zero assertion, custom pixel loop, or tolerant comparison. +- [x] Prove motion-field ownership and lifetime after the current reconstruction-timing change. + - Track initialization, retained-slot aliases, failure unwinding, presentation ownership, decoder-result ownership, and final disposal. + - Every allocator-owned object must be returned exactly once. +- [x] Correct stale documentation for the current worktree. + - Av1InterFrameModeInfoTests must describe the behavior it actually proves. + - Do not claim production reconstruction, constrained allocation, ownership, or reference-stack coverage unless the test executes that contract. + +Checkpoint gate: + +- [x] Default Identity-omission and Translation-consumption GLOBALMV syntax cases pass in the focused current-tree run. +- [x] Two-entry spatial single-reference extension passes; current-main source inspection confirms the separate eight-entry overall stack capacity and DRL access. +- [x] The exact dependent-frame native-plane comparison passes. +- [x] The established exact presentation comparison passes. +- [x] Normal, AVX-512-disabled, AVX-disabled, and scalar FeatureTestRunner configurations pass where supported. +- [x] Constrained allocation preserves the enforced single-group plane invariant without copying or per-block allocation. +- [x] Motion-field allocation tracking is balanced across success and failure on net10.0 and net11.0. +- [x] Release source builds pass for net10.0 and net11.0 with zero warnings and zero errors. +- [x] The complete AV1 namespace passes 8,732 of 8,732 tests on net10.0 and net11.0 with zero failures or skips. +- [x] Roslynk reports zero compiler errors; scoped analyzer inspection reports no diagnostics introduced by the current changes; `git diff --check` passes. +- [x] The completed checkpoint was committed as `54bb6cbe59bd113058854a3ee31448cf61f462ca` with author and committer `James Jackson-South `. +- [x] The palette-memory follow-up was committed as `c4b4e4e0386328dea574a884b6fa36c360ad5a9b` with author and committer `James Jackson-South `. + +Verified single-reference checkpoint evidence on 2026-08-31: + +- The current-main `aomdec` was rebuilt directly from `D:\GitHub\AOMediaCodec\aom` and identified itself as `3.15.0-13-g441c439b99`. +- Decoding the 72-byte progressive payload with `--all-layers`, one thread, and row multithreading disabled produced 2,178 YUV444 color samples. All samples in both layers match the first three planes of the stored YUV444-alpha reference exactly. +- `DecodeProgressiveSingleMatchesReference` executes the production decoder through FeatureTestRunner and compares the complete presented `Rgba32` image with `CompareToReferenceOutput(ImageComparer.Exact, provider)`. The redundant manual alpha loop was removed. +- `DecodeProgressiveSingleWithConstrainedAllocator` executes the same production reconstruction with a 1,024-byte allocator group capacity and verifies that every allocation is returned exactly once. +- `MotionFieldsFollowAliasesAndPresentationOwnership`, `MotionFieldAllocationFailureUnwindsTileReaderOwnership`, `DecodeProgressiveSingleTracksMotionFieldOwnership`, and the reference-store replacement, reset, and transfer tests cover initialization, aliases, presentation ownership, decoder-result ownership, failure unwinding, repeated disposal, and exactly-once final returns in the current worktree. +- The current worktree passes the four-case palette set, seven-case ownership set, and 29-case syntax, plane, and production reconstruction set on both target frameworks. The complete AV1 namespace passes 8,732 of 8,732 tests on net10.0 and net11.0 with zero failures or skips. +- Release source builds passed for net10.0 and net11.0 with zero warnings and zero errors. +- Roslynk reported zero compiler errors. The scoped changed-file analyzer inspection reported no StyleCop diagnostics attributable to this checkpoint; its only remaining match is the pre-existing xUnit cancellation warning in an unrelated `HeifDecoderTests` method. +- `git diff --check` passed, and neither `.gitattributes` file changed. + +Exact verification commands, run directly in the foreground from `D:\GitHub\ynse01\ImageSharp`: + +```powershell +$env:MSBUILDUSESERVER = '0' +$env:DOTNET_CLI_USE_MSBUILD_SERVER = '0' +$env:DOTNET_CLI_HOME = 'D:\GitHub\ynse01\ImageSharp\.dotnet' +$env:DOTNET_SKIP_FIRST_TIME_EXPERIENCE = '1' +$env:DOTNET_CLI_TELEMETRY_OPTOUT = '1' +$env:DOTNET_DbgEnableMiniDump = '0' +$env:COMPlus_DbgEnableMiniDump = '0' +$env:DOTNET_EnableCrashReport = '0' +$env:COMPlus_EnableCrashReport = '0' + +$heifCheckpointFilter = 'FullyQualifiedName~Av1InterFrameModeInfoTests.ReadInterFrameModeInfoReadsInterpolationFilters|FullyQualifiedName~Av1InterFrameModeInfoTests.IdentityGlobalMotionOmitsInterpolationFilters|FullyQualifiedName~Av1ReferenceMotionVectorsTests.BuildReversesOppositeDirectionExtensionCandidate|FullyQualifiedName~Av1FrameBufferTests|FullyQualifiedName~Av1ReferenceFrameStoreTests.MotionFieldsFollowAliasesAndPresentationOwnership|FullyQualifiedName~Av1ReferenceFrameStoreTests.MotionFieldAllocationFailureUnwindsTileReaderOwnership|FullyQualifiedName~Av1ReferenceFrameStoreTests.PartialReplacementPreservesSharedOwner|FullyQualifiedName~Av1ReferenceFrameStoreTests.FinalReplacementReleasesDisplacedOwner|FullyQualifiedName~Av1ReferenceFrameStoreTests.ResetReleasesUniqueOwnersAndClearsSlots|FullyQualifiedName~Av1ReferenceFrameStoreTests.TakeOutputTransfersPlanesAndReleasesOtherReferences|FullyQualifiedName~Av1ReconstructionConformanceTests.DecodeProgressiveSingleMatchesReference|FullyQualifiedName~Av1ReconstructionConformanceTests.DecodeProgressiveSingleWithConstrainedAllocator|FullyQualifiedName~Av1ReconstructionConformanceTests.DecodeProgressiveSingleTracksMotionFieldOwnership' + +dotnet build src\ImageSharp\ImageSharp.csproj -c Release -f net10.0 --no-restore --disable-build-servers -m:1 --no-incremental --nologo --verbosity:minimal +dotnet build src\ImageSharp\ImageSharp.csproj -c Release -f net11.0 --no-restore --disable-build-servers -m:1 --no-incremental --nologo --verbosity:minimal +dotnet test tests\ImageSharp.Tests\ImageSharp.Tests.csproj -c Release -f net10.0 --no-restore --disable-build-servers -m:1 --filter $heifCheckpointFilter --logger 'console;verbosity=minimal' +dotnet test tests\ImageSharp.Tests\ImageSharp.Tests.csproj -c Release -f net11.0 --no-restore --disable-build-servers -m:1 --filter $heifCheckpointFilter --logger 'console;verbosity=minimal' +``` + +```powershell +$aomVcVars = 'C:\Program Files (x86)\Microsoft Visual Studio\18\BuildTools\VC\Auxiliary\Build\vcvars64.bat' +$aomCmake = 'C:\Program Files (x86)\Microsoft Visual Studio\18\BuildTools\Common7\IDE\CommonExtensions\Microsoft\CMake\CMake\bin\cmake.exe' +$aomEnvironment = & cmd.exe /d /s /c "`"$aomVcVars`" >nul && set" +foreach ($aomEntry in $aomEnvironment) +{ + $aomParts = $aomEntry -split '=', 2 + if ($aomParts.Length -eq 2) + { + [Environment]::SetEnvironmentVariable($aomParts[0], $aomParts[1], 'Process') + } +} + +& $aomCmake --build artifacts\reference\aom-generic --target aomdec --config Release --parallel 1 +& 'artifacts\reference\aom-generic\aomdec.exe' --codec=av1 --rawvideo --all-layers --threads=1 --row-mt=0 --output='artifacts\reference\aom-generic\progressive-current-main-all-layers.yuv' 'tests\Images\Input\Heif\Av1\Conformance\libavif-progressive-draw-points-8b.bit' +``` + +### 3. Reverify downstream inter prediction in recorded order + +The single-reference syntax, buffer, reconstruction, and ownership foundation is verified by `54bb6cbe59bd113058854a3ee31448cf61f462ca`. Reverify the existing downstream implementations in this exact order, treating each as locally implemented but unverified until its current-main evidence is recorded. + +- [x] Compound reference selection, paired reference-MV derivation, and equal averaging. +- [x] Inter-intra prediction. +- [x] Distance-weighted compound prediction. +- [x] Wedge compound prediction. +- [x] Difference-weighted compound prediction. +- [x] OBMC. +- [x] Scaled-reference prediction. +- [x] Local warped prediction. +- [x] Non-translational global prediction. +- [x] Inter deblocking decisions and reference/mode deltas. + +Verified equal-average compound checkpoint evidence on 2026-08-31: + +- [x] Refreshed the clean official libaom `main` checkout and audited the observed revision + `441c439b9916474cac15d2822af47a9ad70674a8`. Reference selection and compound mode syntax match + `read_comp_reference_type` and `read_ref_frames` in `av1/decoder/decodemv.c`; contexts match + `av1/common/pred_common.c`; paired reference-MV construction and eight-entry extension match + `process_compound_ref_mv_candidate` and `setup_ref_mv_list` in `av1/common/mvref_common.c`. +- [x] Audited equal-average reconstruction against `av1/common/convolve.c` and + `av1/common/convolve.h`. Corrected the unscaled 10/12-bit translational path so both references + retain libaom's no-round compound intermediates until the sole final average and clipping step, + including the larger first-round shift required for 12-bit horizontal intermediates. +- [x] Added descending Vector512, Vector256, Vector128, and scalar high-bit-depth traversal to the + existing semantic compound-prediction operator families. No per-block, per-row, or per-scanline + allocation or copy was added. +- [x] Added FeatureTestRunner coverage for 10/12-bit copy, horizontal, vertical, and separable + subpixel prediction at widths 9, 17, 33, and 65, with an independent no-round bilinear oracle, + row-padding sentinels, and explicit scalar comparison. +- [x] Added a complete `Av1BlockDecoder.DecodeBlock` 10/12-bit half-sample regression whose expected + result comes from the scalar no-round pipeline. The selected vector differs by one sample from the + obsolete round-each-reference behavior, so the test proves the production branch selection. +- [x] Refreshed the official libaom `main` remote immediately before verification and decoded the + fixture's 5,465-byte AV1 `mdat` payload with current `aomdec`, one thread and row threading + disabled. All 19 frames decoded; the final 19,200 YUV444 samples have SHA-256 + `E79D2F49C260B1AC9B1B9BBBB2D611126AFD3B241DA389EB9E7BD4EA0ED42080` and match the retained native + reference with zero differing samples. +- [x] The real 19-frame production sequence requires decoded equal-average compound blocks, compares + the final native Y, U, and V planes exactly, compares final RGBA presentation through ImageSharp's + established reference-output API, and repeats the complete decode with a 1,024-byte constrained + tracked allocator and exactly-once return checks. +- [x] The focused Release checkpoint set passes 31/31 on net10.0 and 31/31 on net11.0, with zero + failures or skips. Scoped analyzer and whitespace verification pass for every changed C# file, + Roslynk reports zero compiler errors and no diagnostics in the changed files, and `git diff --check` + passes. `.gitattributes` is unchanged. +- [x] The completed checkpoint was committed as `4075a0844836e863a93cb2e2f3ca42d202c7df1b` + with author and committer `James Jackson-South `. + +Verified inter-intra checkpoint evidence on 2026-08-31: + +- [x] Audited syntax against current libaom `av1/decoder/decodemv.c` and + `av1/common/blockd.h`. ImageSharp applies the same sequence enable, skip-mode, block-size, and + single-reference gates, reads the same four-mode CDF, and reads wedge syntax only within libaom's + wedge-supported `BLOCK_8X8` through `BLOCK_32X32` range. +- [x] Audited reconstruction against `ii_weights1d`, `ii_size_scales`, + `build_smooth_interintra_mask`, and `combine_interintra` in current + `av1/common/reconinter.c`. The ImageSharp weights, plane-size scaling, smooth-mask direction, + complemented destination orientation, wedge sign, subsampling, and final 6-bit blend match. No + production change was required. +- [x] The mask tests cover all four inter-intra modes, complemented orientation, row-padding + sentinels, and the 32-wide curve. FeatureTestRunner covers byte and high-bit-depth selectable + blending under SIMD and scalar dispatch, and complete `Av1BlockDecoder.DecodeBlock` tests execute + smooth inter-intra reconstruction at 8, 10, and 12 bits. +- [x] Extracted the fixture's 5,327-byte AV1 `mdat` payload and decoded it with the refreshed current + libaom `aomdec`, using one thread with row threading disabled. All 19 frames decoded. The final + 19,200 YUV444 samples have SHA-256 + `E8B776C2751DC30CA838931A4B74535FC6E681179568A1278747A38CFF2E5BFA` and match the retained + native reference with zero differing samples. +- [x] The real production sequence requires both smooth and wedge inter-intra blocks, compares the + final native Y, Cb, and Cr planes exactly, and compares final RGBA presentation through + ImageSharp's established reference-output API. Its constrained 1,024-byte tracked-allocator run + proves motion-field allocation and exactly one return for every allocation. +- [x] The focused Release checkpoint set passes 50/50 on net10.0 and 50/50 on net11.0, with zero + failures or skips. Scoped analyzer and whitespace verification pass for both changed C# files. + Roslynk reports zero compiler errors and no diagnostics in the changed files, `git diff --check` + passes, and `.gitattributes` is unchanged. +- [x] The completed checkpoint was committed as `18b1c881271a3494489ca6f410ab140544902e2d` + with author and committer `James Jackson-South `. + +Verified distance-weighted compound checkpoint evidence on 2026-08-31: + +- [x] Audited reference-distance quantization against `quant_dist_weight` and + `quant_dist_lookup_table` in current libaom `av1/common/common_data.h`, and audited order-hint + distance selection and forward/backward reference assignment against + `av1_dist_wtd_comp_weight_assign` in `av1/common/reconinter.c`. +- [x] Audited reconstruction against current libaom `av1/common/convolve.c`. Corrected the production + 10/12-bit subpixel path, which incorrectly finalized its two no-round compound intermediates with an + equal average instead of the signaled distance weights. The fixed path applies libaom's 4-bit weighted + shift before bias removal, final rounding, and clipping. +- [x] Added descending Vector512, Vector256, Vector128, and scalar traversal to the existing semantic + distance-weighted intermediate predictor family. Unsigned widening preserves the biased 12-bit + intermediate range. No per-block, per-row, or per-scanline allocation or copy was added. +- [x] Added FeatureTestRunner coverage for every current-libaom distance-weight class in both reference + orders, and for 10/12-bit copy, horizontal, vertical, and separable subpixel prediction at widths 9, + 17, 33, and 65, with an independent no-round oracle and row-padding sentinels. +- [x] Added a complete `Av1BlockDecoder.DecodeBlock` 10/12-bit half-sample regression that selects the + 13:3 distance weights through real order hints. Its first reconstructed sample differs from the old + equal-average result, so the test proves the corrected production branch is executed. +- [x] Extracted the fixture's 5,372-byte AV1 `mdat` payload and decoded it with the refreshed current + libaom `aomdec`, using one thread with row threading disabled. All 19 frames decoded. The final 19,200 + YUV444 samples have SHA-256 + `E8CAA650F1571C5B9CACAF8C06E1DDF5F5D2ED35F65F1C34377076C573425899` and match the retained native + reference with zero differing samples. +- [x] The real 19-frame production sequence requires decoded distance-weighted compound blocks, compares + the final native Y, Cb, and Cr planes exactly, compares final RGBA presentation through ImageSharp's + established reference-output API, and repeats the complete decode with a 1,024-byte constrained + tracked allocator and exactly-once return checks. +- [x] The focused Release checkpoint set passes 44/44 on net10.0 and 44/44 on net11.0, with zero failures + or skips. Scoped analyzer and whitespace verification pass for every changed C# file. Roslynk reports + zero compiler errors and no diagnostics in the changed files, `git diff --check` passes, and + `.gitattributes` is unchanged. +- [x] The completed checkpoint was committed as `7e2de7a2c25852acc374b17936a1a644464f77f3` + with author and committer `James Jackson-South `. + +Verified wedge compound checkpoint evidence on 2026-08-31: + +- [x] Audited mask generation against current libaom `tools/gen_wedge_masks_data.py` and + `av1/common/reconinter.c`, including the master prototypes, direction transforms, block-size + codebooks, sign flips, offsets, and luma/chroma mask sampling. ImageSharp's generated masks match + those definitions; only stale “pinned” documentation required correction. +- [x] Audited reconstruction against current libaom `aom_dsp/blend_a64_mask.c`. The high-bit-depth + d16 path applies the Q6 mask to both no-round intermediates before bias removal, the sole final + rounding step, and clipping. +- [x] Corrected the production high-bit-depth intermediate eligibility gate, which admitted only + equal-average blocks and made the distance-weighted and wedge no-round finalizers unreachable. + Average, distance-weighted, and wedge subpixel blocks now retain both intermediates until their + signaled finalizer; difference-weighted blending remains excluded for its next ordered checkpoint. +- [x] Added high-bit-depth traversal to the existing semantic mask-blend predictor and readonly + operator family with descending Vector512, Vector256, Vector128, and scalar dispatch. Unsigned + widening preserves the biased 12-bit intermediate range. No per-block, per-row, or per-scanline + allocation or copy was added. +- [x] Extended FeatureTestRunner coverage with an independent Q6 mask oracle across 10/12-bit copy, + horizontal, vertical, and separable subpixel prediction, widths 9, 17, 33, and 65, all mask weights + from 0 through 64, and row-padding sentinels. A complete `Av1BlockDecoder.DecodeBlock` regression + verifies the current-libaom 8x8 wedge mask and the production no-round branch. +- [x] Extracted the fixture's 5,374-byte AV1 `mdat` payload and decoded it with refreshed current + libaom `aomdec`, using one thread with row threading disabled. All 19 frames decoded. The final + 19,200 YUV444 samples have SHA-256 + `E8CAA650F1571C5B9CACAF8C06E1DDF5F5D2ED35F65F1C34377076C573425899` and match the retained native + reference with zero differing samples. +- [x] The real 19-frame production sequence requires both wedge-mask orientations, compares final + native Y, Cb, and Cr planes exactly, compares final RGBA presentation through ImageSharp's + established reference-output API, and repeats the complete decode with a 1,024-byte constrained + tracked allocator and exactly-once return checks. +- [x] The focused Release checkpoint set passes 35/35 on net10.0 and 35/35 on net11.0, with zero + failures or skips. Scoped analyzer and whitespace verification pass for every changed C# file. + Roslynk reports zero compiler errors, `git diff --check` passes, and `.gitattributes` is unchanged. +- [x] The completed checkpoint was committed as `9883a24dc319e16b471f68be632d4f62f2c1cd5e` + with author and committer `James Jackson-South `. + +Verified difference-weighted compound checkpoint evidence on 2026-08-31: + +- [x] Audited syntax against current libaom `av1/decoder/decodemv.c`. ImageSharp applies the same + masked-compound enable and block-size gates, selects difference-weighted compound directly when wedge + is unavailable, and reads the same one-bit type-38 mask orientation. +- [x] Audited mask generation and reconstruction against current libaom `av1/common/reconinter.c` and + `aom_dsp/blend_a64_mask.c`. The d16 path rounds the absolute intermediate difference by the + convolution and bit-depth shift, scales it by 1/16, adds the type-38 base, clamps or inverts the mask, + and then blends the original no-round intermediates before final rounding and clipping. Chroma reuses + the luma-derived mask through rounded subsampling. +- [x] Corrected the production 10/12-bit subpixel eligibility gate, which previously rounded both + references before difference-mask construction and blending. Difference-weighted blocks now use the + existing semantic intermediate mask-builder and mask-blend predictor/operator families through the + sole final rounding step. No new operator family, per-block allocation, or copy was introduced. +- [x] Renamed the stale “pinned formula” test and extended FeatureTestRunner's independent oracle across + current-libaom regular and d16 mask arithmetic, both mask orientations, 8/10/12-bit samples, widths + that cross every Vector512, Vector256, Vector128, and scalar boundary, subpixel phases, and row-padding + sentinels. +- [x] Added a complete `Av1BlockDecoder.DecodeBlock` regression for 10/12-bit half-sample prediction + and both type-38 orientations. Its expected mask and reconstruction are calculated directly from the + current-libaom equations, independently of the production mask builder and finalizer. +- [x] Extracted the fixture's 5,358-byte AV1 `mdat` payload and decoded it with refreshed current + libaom `aomdec`, using one thread with row threading disabled. All 19 frames decoded. The final + 19,200 YUV444 samples have SHA-256 + `E8CAA650F1571C5B9CACAF8C06E1DDF5F5D2ED35F65F1C34377076C573425899` and match the retained native + reference with zero differing samples. +- [x] The real 19-frame production sequence requires both difference-mask orientations, compares final + native Y, Cb, and Cr planes exactly, compares final RGBA presentation through ImageSharp's established + reference-output API, and repeats the complete decode with a 1,024-byte constrained tracked allocator + and exactly-once return checks. +- [x] The focused Release checkpoint set passes 37/37 on net10.0 and 37/37 on net11.0, with zero + failures or skips. Scoped analyzer and whitespace verification pass for every changed C# file. + Roslynk reports zero compiler errors, `git diff --check` passes, and `.gitattributes` is unchanged. +- [x] The completed checkpoint was committed as `fb4c64474e1ced4067a42731384f3b5ad4212a2f` + with author and committer `James Jackson-South `. + +Verified OBMC checkpoint evidence on 2026-08-31: + +- [x] Audited motion-mode syntax against current libaom `av1/decoder/decodemv.c`, + `av1/common/blockd.h`, `av1/common/reconinter.c`, `av1/common/obmc.h`, and + `av1/common/reconinter_template.inc`. ImageSharp applies the same switchable-mode, skip, + single-reference, inter-intra, minimum-size, overlappable-neighbor, fixed-global-motion, scaled + reference, and projection-sample gates and reads the matching binary or three-way CDF. +- [x] Audited above and left neighbor traversal, 4x4 pairing, neighbor caps, chroma suppression, + prediction rectangles, interpolation filters, first-reference selection, mask tables, and blend + order against current libaom. The existing semantic mask-blend predictor remains the correct + SIMD-first traversal; no OBMC-specific operator family, allocation, or copy was introduced. +- [x] Corrected the unscaled neighbor far-edge UMV clamp. After converting libaom's neighbor-relative + motion-vector limits to an absolute source coordinate, the prediction extent cancels from the + right and bottom limits; the previous code counted it twice. +- [x] Extracted the fixture's 5,387-byte AV1 `mdat` payload at AVIF offset 1,065 and decoded it with + refreshed current libaom `aomdec`, using one thread with row threading disabled. All 19 frames + decoded. The final 19,200 YUV444 samples have SHA-256 + `E8CAA650F1571C5B9CACAF8C06E1DDF5F5D2ED35F65F1C34377076C573425899` and match the retained native + reference with zero differing samples. +- [x] The production sequence asserts decoded OBMC mode state, compares final native Y, Cb, and Cr + planes exactly, compares final RGBA presentation through ImageSharp's established reference-output + API under normal and scalar FeatureTestRunner dispatch, and repeats reconstruction with a 1,024-byte + constrained tracked allocator. Direct `DecodeBlock` tests cover above-then-left blending at + 8/10/12-bit and 4:2:0 and 4:2:2 chroma geometry. +- [x] Renamed the stale pinned-reference test and its established reference-output PNG together. The + PNG SHA-256 remains + `D2CB388C9092EF17C4F0382C0150DD30D6F9D0EE247FF45AB5D7D4D312CEB23C`; only its contract-derived + filename changed. +- [x] The focused Release checkpoint set passes 18/18 on net10.0 and 18/18 on net11.0, with zero + failures or skips. Scoped analyzer and whitespace verification pass for every changed C# file. + Roslynk reports zero compiler errors, `git diff --check` passes, and `.gitattributes` is unchanged. +- [x] The completed checkpoint was committed as `7e7e3cbe6438d63926b31d966795d2652e221939` + with author and committer `James Jackson-South `. + +Verified scaled-reference checkpoint evidence on 2026-08-31: + +- [x] Audited reference-size validation and variable-scale coordinates, filters, edge extension, convolution + rounding, and compound intermediates against current libaom `av1/common/scale.c`, + `av1/decoder/decodeframe.c`, and `av1/common/convolve.c`. The frame boundary accepts the same + half-to-sixteen-times dimension range and requires at least one compatible selected reference. +- [x] Corrected the production scaled-compound branch. It previously rounded each scaled reference into + native pixels before blending; current libaom retains both `CONV_BUF_TYPE` values with + `COMPOUND_ROUND1_BITS` equal to seven and performs one final rounding after the selected compound blend. +- [x] Kept native-pixel and compound output in the existing `Av1ScaledInterPredictor` traversal with + semantic `NativeOperator` and `CompoundOperator` output contracts. The closed generic traversal shares + variable-phase arithmetic across byte and ushort sources, dispatches Vector512, Vector256, Vector128, + then scalar, and adds no per-block allocation or copy. +- [x] Added independent FeatureTestRunner oracles for native and no-round compound output across 8, 10, + and 12 bits, variable phases, all interpolation families, reduced kernels, vector tails, and destination + padding. A complete `Av1BlockDecoder.DecodeBlock()` regression covers scaled compound prediction across + all, AVX-512-disabled, AVX-disabled, and scalar configurations and proves the vector differs from an + incorrectly early-rounded blend. +- [x] Decoded the 2,195-byte layered payload with refreshed current libaom `aomdec`, using one thread, + row threading disabled, all layers selected, and raw 8-bit output. The 40x40 YUV444 base and 80x80 + YUV444 dependent frames total 24,000 samples with SHA-256 + `DD219E41B52C6C9343A92CD0A2D451DF57B73B25F10124811675B4CB2F8D666F`; both match their retained + native references with zero differing samples. +- [x] The production tests compare both native frames exactly, compare selected-layer and final RGBA + presentation through ImageSharp's established reference-output API, and repeat both paths with a + 1,024-byte constrained tracked allocator whose allocations have balanced exactly-once returns. +- [x] Renamed the two stale pinned-reference tests and their contract-derived PNGs together. Their Git blob + identifiers remain unchanged, and their SHA-256 values remain + `DC4C6DBE6BD92C5FCE1E3E23700AFA603EF04ED02EDD336213EBBA1E3BD84BA0` and + `678C5E5D4650EA6F0C590302E7DB9E3C6608851BC577453DA4A6837BDB4D3AF3`. +- [x] The focused Release checkpoint set passes 10/10 on net10.0 and 10/10 on net11.0, with zero failures + or skips. Scoped analyzer and whitespace verification pass for every changed C# file. Roslynk reports + zero compiler errors, `git diff --check` passes, and `.gitattributes` is unchanged. +- [x] The completed checkpoint was committed as `658a9cd1b6e22806decbae923da8800bca03a09e` + with author and committer `James Jackson-South `. + +Verified local warped-prediction checkpoint evidence on 2026-08-31: + +- [x] Refreshed the clean official libaom `main` checkout and audited the observed revision + `441c439b9916474cac15d2822af47a9ad70674a8`. Motion-mode eligibility and CDF selection match + `read_motion_mode` in `av1/decoder/decodemv.c`; above, left, top-left, and top-right spatial projection + samples and threshold selection match `findSamples` and `selectSamples` in + `av1/common/mvref_common.c`; affine fitting, shear reduction, phase derivation, filters, rounding, + clipping, and invalid-model fallback match `av1/common/warped_motion.c` and + `av1/common/reconinter.c`. +- [x] Mechanically compared all 1,544 ImageSharp and independent-test warped-filter coefficients against + current libaom's `av1_warped_filter`; both comparisons have zero differences. The separate scalar test + transcription covers 8-, 10-, and 12-bit luma and subsampled-chroma coordinates, tail widths, destination + stride preservation, libaom's 12-bit round adjustment, and AVX-512, AVX, 128-bit, and scalar dispatch + through `FeatureTestRunner`. +- [x] Extracted the fixture's exact 2,310-byte AV1 `mdat` payload at AVIF offset 997. Its SHA-256 is + `644D04FE1D1A32BB7A3856AD7EB49CF1EFDE0AC845E55BEAC4170F72353F2391`. Current official + libaom decoded both 256x256 YUV444 frames with one thread, row threading disabled, and all layers enabled. + The complete Y4M SHA-256 is + `8FDC5D46014F5E5A7455A83643AB6F0DA66FC5A984E72A43F8C75BAD8271C299`; the final + frame's 196,608 native samples have SHA-256 + `47B2AB39BF3B9DA15C1EC59840F964DFDF227760947F6E1295FB38A84555F75C` and match the + retained native reference with zero differences. +- [x] The real two-frame fixture exercises `Av1BlockDecoder.DecodeBlock()`, requires decoded + `WARPED_CAUSAL` state and the expected multi-sample affine model, compares final native Y, U, and V + planes exactly, compares the retained final presentation through ImageSharp's established reference-output + API, and passes through intrinsic and scalar dispatch. The 1,024-byte constrained tracked-allocator path + passes with motion-field allocations present and balanced exactly-once returns. +- [x] Renamed the stale pinned-reference test and its contract-derived PNG together without changing the PNG + bytes. Its SHA-256 remains + `4490D62FB6679378E92CACA48427359091AD2106BE49FC1A3848F78BE03BEEB1`. +- [x] The focused Release checkpoint set passes 4/4 on net10.0 and 4/4 on net11.0, with zero failures or + skips. Scoped analyzer verification passes for both changed C# files. Roslynk reports zero compiler errors, + `git diff --check` passes, and `.gitattributes` is unchanged. +- [x] The completed checkpoint was committed as `27a522424fe7aaea25078e705d71a501da110727` + with author and committer `James Jackson-South `. + +Verified non-translational global-prediction checkpoint evidence on 2026-08-31: + +- [x] Audited global-motion syntax, coefficient decoding, previous-reference recentering, shear validation, + motion-vector projection, and warped-prediction eligibility against current official libaom `main` at the + observed revision `441c439b9916474cac15d2822af47a9ad70674a8`. The implementation matches + `read_global_motion_params`, `read_global_motion_model`, `gm_get_motion_vector`, `is_global_mv_block`, + and the WARP_PRED selection in `av1/common/reconinter.c`. +- [x] Corrected high-bit-depth compound warped/global prediction to retain both references in libaom's + unsigned no-round compound domain. Current `get_conv_params_no_round`, `av1_warp_plane`, and + `av1_highbd_warp_affine_c` require the 12-bit first-round adjustment while retaining a seven-bit second + round; native clipping now occurs only after the compound blend. +- [x] The independent scalar libaom transcription validates native and no-round compound output for byte, + 8-bit, 10-bit, and 12-bit sources, including tail widths and destination-stride preservation. All cases pass + through AVX-512, AVX, 128-bit, and scalar dispatch with `FeatureTestRunner`. Direct + `Av1BlockDecoder.DecodeBlock()` coverage validates `GLOBAL_GLOBALMV` compound reconstruction at all + supported bit depths. +- [x] Extracted the fixture's exact 38,475-byte AV1 `mdat` payload at AVIF offset 997. Its SHA-256 is + `6AC7EC9984B1FF5C00403D7E3858441E9CEE75128F7414101D06DEEE59A351D0`. Current + official libaom decoded both 256x256 YUV444 frames with one thread, row threading disabled, and all layers + enabled. The complete Y4M SHA-256 is + `84754DE0B9FABC4F3F8F344C848183EC17B625BFD87E4519C3D8AD7DEFD20F2C`; the final + frame's 196,608 native samples have SHA-256 + `FEC89E2DE7496980389806B194425042F3800C7BAA817249D1A51D44A2B37A8E` and match the + retained native reference with zero differences. +- [x] The real two-frame fixture exercises the production decoder, requires decoded non-translational global + motion, compares final native Y, U, and V planes exactly, compares the retained presentation through + ImageSharp's established reference-output API, and passes the constrained tracked-allocator path. +- [x] Renamed the stale pinned-reference test and its contract-derived PNG together without changing the PNG + bytes. Its SHA-256 remains + `F7D27ABF79450DFA311F72106FD1DA80997EABC0937F2F5578EF627119FF83B0`, and Git + attributes select the LFS filter and diff driver. +- [x] The focused Release checkpoint set passes 11/11 on net10.0 and 11/11 on net11.0, with zero failures or + skips. Scoped analyzer verification passes for all six changed C# files. Roslynk reports zero compiler + errors, `git diff --check` passes, and `.gitattributes` is unchanged. +- [x] The completed checkpoint was committed as `25295683d39a2336e9b98484c9fd54f33107ea66` + with author and committer `James Jackson-South `. + +Verified inter-deblocking checkpoint evidence on 2026-08-31: + +- [x] Audited frame-level loop-filter syntax and primary-reference inheritance against + `setup_loopfilter` in current `av1/decoder/decodeframe.c`; per-superblock delta-LF parsing and + prediction against `read_delta_q_params` in `av1/decoder/decodemv.c`; and default reference/mode + deltas against `av1/common/entropymode.c` at observed current-main revision + `441c439b9916474cac15d2822af47a9ad70674a8`. +- [x] Audited filter-level derivation, segmentation adjustment, reference scaling, global/non-global + mode classes, skipped-transform prediction-unit decisions, transform-edge selection, kernel length, + sharpness limits, and vertical-then-horizontal traversal against `get_filter_level`, + `set_lpf_parameters`, `av1_filter_block_plane_vert`, `av1_filter_block_plane_horz`, and + `av1_thread_loop_filter_rows`. No production arithmetic change was required. +- [x] Added direct production `Av1LoopFilterDecoder.DecodeFrame()` coverage using adjacent skipped + 16x8 inter blocks split into 8x8 transforms. An independent scalar oracle proves that internal + transform edges remain untouched and the prediction-unit edge uses current-libaom levels 17 for + LAST/GLOBALMV, 21 for LAST/NEWMV, and 22 for GOLDEN/GLOBALMV. Existing `FeatureTestRunner` + coverage continues to verify every filter width at 8, 10, and 12 bits under intrinsic and scalar + dispatch. +- [x] Current official libaom decoded the retained 20,750-byte 8-bit, 37,169-byte 10-bit, and + 23,769-byte 12-bit elementary streams with one thread, row threading disabled, raw output, and their + native output depths. The generated native files match the retained references byte for byte. Their + output SHA-256 values are + `8DDE2EEC742C39F0579C29AE84CBA0FE01522A9008ADCB2CFFCCEC0295D18141`, + `9A59DD92A0C579F942ACCA8281EBD0465DC848BE200A4D2FF57EAFF589445F6C`, and + `EF712BE32AF7CF0A95C5C41BDCC51AFC05A4AB7C047383F5F65EDAD2BB986712`. +- [x] Reused the already current-main scaled-reference sequence as the real inter checkpoint. It + requires an inter frame with reference/mode-delta processing enabled, nonzero chroma filter levels, + intra, inter, and skipped-inter blocks; compares both decoded native frames exactly; compares final + presentation through ImageSharp's established reference-output API; and passes constrained tracked + allocation with balanced returns. +- [x] Removed an obsolete SVT-AV1 design link from mode-map documentation. Current official libaom + remains the sole external codec implementation source. +- [x] The focused Release checkpoint set passes 6/6 on net10.0 and 6/6 on net11.0, with zero failures + or skips. +- [x] Scoped analyzer verification passes for all four changed C# files. Roslynk reports zero compiler + errors, `git diff --check` passes, and `.gitattributes` is unchanged. +- [x] The completed checkpoint was committed as `fcb502e4960cc7b8efb06b6f060e2c73a913a2bf` + with author and committer `James Jackson-South `. + +For every item: + +- [ ] Trace syntax and arithmetic to the current libaom `main` tree. +- [ ] Execute the real production decoder path. +- [ ] Compare native planes exactly. +- [ ] Compare presentation through the established reference-image API. +- [ ] Run constrained allocator and exactly-once ownership coverage. +- [ ] Run FeatureTestRunner for SIMD and scalar dispatch when the implementation has SIMD. +- [ ] Record focused Release evidence before marking the item verified. + +### 4. Close AV1 decoder coverage + +Previously verified algorithm checkpoints remain valuable evidence, but the final decoder gate requires a fresh current-tree run after the inter and cleanup corrections. + +- [x] Bounded OBU framing, sequence headers, frame headers, tile groups, alignment, and trailing-bit parsing have been re-audited and verified against current libaom `main`. +- [x] Partition traversal, mode information, segmentation, delta quantization, transform-size selection, coefficient decoding, inverse quantization, and inverse transforms have been re-audited and verified against current libaom `main`. +- [x] Intra prediction covers directional, DC, smooth, Paeth, chroma-from-luma, filter-intra, and palette families with the established operator architecture. +- [x] Intra-block copy has exact native reconstruction and feature-isolated SIMD evidence. +- [x] Lossless inverse transform, loop filtering, CDEF, super-resolution, restoration, and film grain have focused checkpoint evidence. +- [x] Retained references, CDF snapshots, segmentation maps, global motion, temporal motion fields, and dependent-frame lifecycle have been re-audited and verified against current libaom `main`. +- [x] The 12-case all-intra profile matrix covers every valid 8, 10, and 12-bit monochrome, 4:2:0, 4:2:2, and 4:4:4 combination. Dependent-frame coverage is recorded separately above. +- [x] The exact current-tree native-plane matrix passes through the production decoder on net10.0 and net11.0. The normal-dispatch and FeatureTestRunner fallback methods pass 2 of 2 focused tests on each target. +- [x] The exact current-tree presentation matrix passes 12 of 12 cases through ImageSharp's established reference-image API on net10.0 and net11.0. +- [x] Verify malformed/truncated data, frame IDs, reference slots, tile bounds, allocation limits, cancellation, and failure unwinding. +- [x] Verify still items and bounded sequences from file, memory, non-seekable, and short-read streams. +- [~] Verify ICC, CICP, alpha, grids, pixel aspect ratio, clean aperture, rotation, mirroring, metadata, and every presented sequence frame. Grid validation now requires the first cell to be at least 64 samples on both axes, enforces even output and cell dimensions along each subsampled AV1 chroma axis, and requires every cell to cover its row-major output region without exceeding the first cell's dimensions. This accepts the smaller right and bottom cells supported by the writer while also accepting uniform coded cells whose final row and column are cropped to the grid descriptor. Auxiliary alpha uses the same cropped overlap, so padded cells cannot write outside the final frame. Production regressions cover smaller right, bottom, and bottom-right color and alpha cells; Roslynk compiler and scoped analyzer diagnostics are clean, while runtime verification of the current tree remains pending. +- [x] Complete the public AVIF format/API review so registered capabilities match implemented behavior. +- [x] Remove or reject every valid in-scope AV1 syntax branch that remains silently ignored or unsupported. + +Verified negative-path and frame-identifier gate evidence on 2026-08-31: + +- [x] A two-frame lossless frame-identifier sequence was generated and decoded with the clean official + libaom `main` checkout at observed revision `441c439b9916474cac15d2822af47a9ad70674a8`. + Both decoded frames match the source Y, Cb, and Cr samples exactly. +- [x] `DecodeFrameIdentifiersMatchReference` executes the production decoder through FeatureTestRunner, + compares both native frames exactly, and proves the second frame is dependent with a changed current + frame identifier. The current-frame, reference-delta, stale-slot, and refreshed-slot identifier logic + was audited against the same current `main` source. +- [x] The focused negative-path set passes 46 of 46 cases on net10.0 and 46 of 46 on net11.0, with zero + failures or skips. It covers truncated palette entropy, malformed-following-OBU recovery, parser + lifecycle failure, overflowing and invalid tile bounds, reference-slot ownership and transfer, + constrained multi-group allocation, motion-field allocation failure unwinding, and frame identifiers. +- [x] The established paused-stream cancellation suite now includes AVIF. It verifies cancellation at + 0%, 30%, and 70% of both file and memory streams, plus pre-cancelled identification, on both targets. +- [x] The completed checkpoint was committed as + `7f0e08126b3354e8f1eb45886f0d572006ae27de` with author and committer + `James Jackson-South `. + +Verified bounded-OBU checkpoint evidence on 2026-08-31: + +- [x] Audited `av1/decoder/obu.c`, `av1/decoder/decodeframe.c`, `av1/common/obu_util.c`, + `av1/common/tile_common.c`, `aom/src/aom_integer.c`, and `aom_dsp/bitreader_buffer.c` in the + clean official libaom `main` checkout. Both `HEAD` and `origin/main` resolved to the observed + revision `441c439b9916474cac15d2822af47a9ad70674a8`; this is verification evidence, not a pin. +- [x] The bounded container scanner and production OBU reader now agree with current libaom on ignored + reserved header fields and the shared unsigned 32-bit LEB128 limit. +- [x] Sequence-header validation now rejects undefined level indices, initial display delays above ten, + frame identifiers above sixteen bits, zero timing units, the UVLC overflow sentinel, and invalid + identity-matrix profile or subsampling combinations at the owning syntax boundary. +- [x] Frame and tile parsing now rejects `show_existing_frame` in a combined `OBU_FRAME`, the all-slots + intra-only refresh mask, inner tile columns below current libaom's super-resolution-aware minimum, + overflowing or out-of-bounds tile sizes, and empty final tile payloads. +- [x] The still-image writer now emits the required zero tile-bound-presence bit for a multi-tile combined + `OBU_FRAME`, matching current libaom's single-tile-group encoder path. +- [x] `ObuFrameHeaderTests` and `ObuFrameLifecycleTests` cover the corrected syntax through the real + bounded parser. The focused parser set passes 50 of 50 cases on net10.0. +- [x] The final focused production set passes 55 of 55 cases on net10.0 and 55 of 55 on net11.0, with zero + failures or skips. It includes exact final-layer and selected-layer native planes, exact established + reference-image presentation, constrained allocator ownership, malformed-following-OBU recovery, and + FeatureTestRunner normal, AVX-512-disabled, AVX-disabled, and scalar execution. +- [x] A fresh direct foreground current-main `aomdec` run decoded both progressive layers with one thread + and row multithreading disabled. All 2,178 Y, U, and V samples match the retained YUV444-alpha reference; + the alpha plane is excluded from the AV1 native-plane comparison. +- [x] The current-libaom production reference test and its established PNG were renamed together. The PNG + bytes remain unchanged at SHA-256 + `0758C17DC36E38AEE9F4389A335C2BF332AB91E4C79D7B0B22994FDDD0FD1605`, both paths resolve to + `diff=lfs`, and `.gitattributes` was not edited. +- [x] Release source builds pass for net10.0 and net11.0 with zero warnings and zero errors. Roslynk reports + zero compiler errors, and scoped production and test analyzer verification reports no changes. +- [x] The completed checkpoint was committed as `243524c2c0b52a49d8d161fab806ab092cabe47c` with author + and committer `James Jackson-South `. + +Verified partition, mode, segmentation, quantization, and transform checkpoint evidence on 2026-08-31: + +- [x] Audited partition traversal and chroma representability against `read_partition` and the subsampled + plane-size rejection in current libaom `av1/decoder/decodeframe.c`; spatial segment-ID decoding and + corruption handling against `read_segment_id` in `av1/decoder/decodemv.c`; delta-Q syntax, resolution, + arithmetic, and clamping against `read_delta_qindex` and `read_delta_q_params` in the same file. +- [x] Audited selected and variable transform-size traversal against `read_tx_size`, `read_tx_size_vartx`, + and transform-block traversal in `av1/decoder/decodeframe.c`; coefficient syntax and arithmetic against + `av1_read_coeffs_txb` in `av1/decoder/decodetxb.c`; inverse quantization and transform application against + current `av1/decoder/decodeframe.c`, `av1/common/idct.c`, and the current libaom transform test oracle. + The observed clean `HEAD` and `origin/main` revision was + `441c439b9916474cac15d2822af47a9ad70674a8`; this is verification evidence, not a pin. +- [x] Partition decoding now rejects an invalid partition subsize and a block size that cannot represent the + current subsampled chroma plane. Spatial segmentation rejects decoded IDs above the active segment range. + Focused tests exercise both current-libaom corruption boundaries through the production tile reader. +- [x] Coefficient entropy decoding uses one allocator-owned maximum-size `Av1LevelBuffer` per tile reader. + Each transform resets and clears only its active padded geometry, so no transform creates an allocation. + Allocation tracking over all eight minimum- and maximum-quantizer frames proves exactly one coefficient + scratch allocation per frame and exactly-once return after decoder disposal. +- [x] Palette index maps use one allocator-backed 32 KiB decoder-session owner with non-owning 128x128 luma + and chroma views. Each parsed superblock is reconstructed before either view is reused, and each block + clears only its transient `Buffer2DRegion` after prediction. The fixed session cost replaces the former + full-frame maps without copies, fragmented memory groups, constructor rollback, or per-block allocations. + The one-rent ownership regression and native palette reconstruction pass on net11.0, the complete HEIF/AV1 + namespace passes 8,808 of 8,808 direct VSTest cases in Release, and the four-case palette set passes with + exact native and presentation output, truncated-entropy rejection, and balanced exactly-once disposal. +- [x] `Av1BlockModeInfo` is value storage, removing the managed object allocation formerly created for every + decoded coding block. Explicit `ModeInfoIndex` values preserve libaom's mode-info identity semantics at + prediction-unit loop-filter edges, and the frame map now uses integer offsets so more than 65,535 decoded + blocks cannot wrap its lookup identity. +- [x] Current official libaom reproduced the 39-frame all-intra reference and all four 8/10-bit minimum- and + maximum-quantizer references byte for byte. The production tests compare every native sample exactly, + cover every intra mode and seven selected transform types, execute SIMD and scalar paths through + `FeatureTestRunner`, and exercise the quantizer sequences under constrained tracked allocation. +- [x] Current official libaom decoded the 42-byte palette payload into the retained 1,089-byte YUV444 + reference at SHA-256 `E05F7C0DF06ECCF0E43869D1D7B03DAA1D635ACD26A766F8940899BE18D53251`. + The exact native test requires luma and chroma palette syntax. The established reference-output test uses + the unchanged presentation PNG at SHA-256 + `1148EBF6AA4B0F2D069D5E9B9605F6FB2A315E525F18016CDCAE23EFDD81DA84`, whose renamed path still + resolves to `diff=lfs`; `.gitattributes` was not edited. +- [x] The exact final AV1 namespace passes 8,732 of 8,732 cases on net10.0 and 8,732 of 8,732 cases on + net11.0, with zero failures or skips. Release source builds pass for net10.0 and net11.0 with zero warnings + and zero errors. Roslynk reports zero compiler errors, and scoped analyzer verification reports no changes. +- [x] The completed checkpoint was committed as `57a3f6668e39d0934e7b6b8d37a3dc2a5adc88f0` with author + and committer `James Jackson-South `. + +Verified retained-frame lifecycle checkpoint evidence on 2026-08-31: + +- [x] Audited primary-reference entropy selection, independent per-tile CDF starts, context-update-tile + publication, segmentation-map inheritance, reference-map refresh, and show-existing key-frame reset + against current libaom `av1/decoder/decodeframe.c`, `av1/decoder/decodemv.c`, + `av1/decoder/decoder.c`, and `av1/common/entropymode.c`. +- [x] Audited retained motion-vector cells, reference-side classification, projection source ordering, + projection limits, and reference-frame publication against `av1_copy_frame_mvs`, + `av1_calculate_ref_frame_side`, `motion_field_projection`, and `av1_setup_motion_field` in current + libaom. Same-role primary-reference global-motion inheritance remains covered by the exact current-main + global-warp fixture. The observed clean `HEAD` and `origin/main` revision was + `441c439b9916474cac15d2822af47a9ad70674a8`; this is verification evidence, not a pin. +- [x] Current official libaom decoded the retained `cdfupdate`, `mfmv`, `svc-L2T1`, `svc-L1T2`, and + `svc-L2T2` streams with one thread, row threading disabled, and eight-bit output depth. Their generated + Y4M files match the retained references byte for byte at SHA-256 + `4FBFF73FF0DE2D9084DAE557D1D4BD677B0486516525BF4D327D2D795D5A7779`, + `F7DB607694818C19E62FD9A27F53E1A3E2D00B72C39C0430C1B26399CC76777D`, + `7A427631ECBF144F435AA4612F1201415FB1A9BCF9A67BA010AEF830B0C3AB81`, + `4012DE2D4AFD095E7BB68EAE18B50B0674781BB4971CECABC0E5471E63373ED3`, and + `1ABB981CFF76BA9557DA437B258D8A95FCA755DED8E3949D857E8388AB1D6AE3`. +- [x] Existing allocation-tracking tests exercise initialization, retained-slot aliases, allocation-failure + unwinding, presentation ownership, decoder-result ownership, repeated disposal, and final exactly-once + return of reference frames, frame-owned motion fields, entropy snapshots, and segmentation maps. +- [x] The focused Release checkpoint set passes 54 of 54 cases on net10.0 and 54 of 54 cases on net11.0, + with zero failures or skips. It includes exact native CDF-update, motion-field, spatial-layer, + temporal-layer, spatial-temporal-layer, progressive dependent-frame, and global-warp production paths, + plus constrained allocator coverage. +- [x] Release source builds pass for net10.0 and net11.0 with zero warnings and zero errors. Roslynk + reports zero compiler errors, scoped analyzer verification reports no changes, `git diff --check` + passes, and `.gitattributes` is unchanged. + +Final decoder allocation, lifetime, precision, architecture, and test-validity audit evidence on 2026-09-01: + +- [x] Refreshed the official libaom remote and audited against observed `origin/main` + `976867526367f571a1c09b994066af8364aed781`. The intervening external-rate-controller commit does + not change `av1/decoder`, `av1/common`, `aom_dsp`, or the AV1 decoder build definition. +- [x] CDEF now uses one bounded 64x64-unit bordered source workspace, two preserved top-row slots per + plane, preserved left columns, and unit-local direction and variance storage. This replaces the + frame-wide source copy and frame-wide direction maps while retaining libaom's unit traversal and + cross-plane luma-direction lifetime. +- [x] Loop restoration now retains the required immutable source and separate destination, but stores the + full destination in native sample width. Eight-bit filtering narrows only bounded unit output after + clipping, while high-bit-depth filtering writes directly to the native `ushort` destination. +- [x] Reference-to-presentation copying now copies visible native rows only. Padding remains destination + owned, and the ownership tests mutate a copied visible sample rather than unrelated padding. +- [x] The remaining decoder allocations and copies are either bounded scratch or required ownership + boundaries. Frame planes enforce their contiguous single-span invariant before allocation; palette, + transform, film-grain, super-resolution, color-conversion, and alpha workspaces remain bounded and + allocator owned. No per-block managed allocation remains in reconstruction. +- [x] Block reconstruction now uses one exact-size signed-short owner for inverse quantization, inverse + transform, compound prediction, convolution, and chroma-from-luma scratch. Even-length slices provide + the integer workspaces without another rent. Monochrome reserves no chroma coefficients, and 4:2:0, + 4:2:2, and 4:4:4 reserve two symmetric chroma planes at their coded subsampling. This replaces three + constructor rents and their catch-all rollback path; exact allocation length, coefficient span length, + and exactly-once return pass for all four layouts, with 549 adjacent reconstruction tests passing direct + net11 VSTest in Release. +- [x] Valid unsupported tile-list syntax is rejected explicitly. Reserved and metadata OBUs are consumed + only after bounded framing and trailing-bit validation. Eight-, ten-, and twelve-bit reconstruction, + presentation, alpha, restoration, and film-grain paths retain native precision. +- [x] Predictor traversal remains split into semantic readonly operator families. The planar sample + adapter and transform-block context are value types, and Release construction sites use `default` + without null-forgiving suppression. +- [x] The net11.0 Release test project builds with zero errors. Roslynk reports zero compiler errors, + `git diff --check` passes, and `.gitattributes` is unchanged. +- [x] Visual Studio 18.9 VSTest ran the complete `Formats.Heif.Av1` namespace with collection + parallelism disabled and stop-on-failure enabled: 8,746 of 8,746 cases passed. The touched + `HeifDecoderTests` and `HeifSequenceParserTests` add 104 of 104 passing integration cases. + Focused CDEF, restoration, film-grain, copy-ownership, and reference-isolation runs also pass 15 of + 15 cases. The historical report recorded successful process exits; it did not independently establish absence of Windows application-error dialogs. + +Final decoder stream, presentation, and public-registration evidence on 2026-09-01: + +- [x] Real AV1 still-item and timed-sequence files decode identically from a file stream, memory stream, + non-seekable stream, and a seekable stream limited to three bytes per read. All eight stream rows pass + through public format detection and production decoding, comparing every presented frame exactly. +- [x] A two-frame production sequence applies a centered clean-aperture crop, counter-clockwise rotation, + mirroring, pixel-aspect-ratio metadata, and CICP metadata to every frame. The complete five-frame real + auxiliary-alpha sequence composes non-opaque alpha and retains timing, Exif, and XMP for every frame. +- [x] The fixed-header detector accepts both compact and extended-size leading file-type boxes. Default + configuration registers the implemented HEIF decoder and detector but no longer advertises the + incomplete HEIF encoder. +- [x] Visual Studio 18.9 VSTest, serialized with stop-on-failure enabled, passes the 12 of 12 new + stream/presentation/registration cases and the complete current `HeifDecoderTests` plus + `HeifSequenceParserTests` set with the registration contract: 115 of 115. The final explicit + no-encoder registration assertion passes 1 of 1 after its final edit. +- [x] The net11.0 Release test project builds with zero errors, Roslynk reports zero compiler errors, + `git diff --check` passes, and `.gitattributes` is unchanged. The historical report recorded normal VSTest exits and no surviving test host; that does not establish absence of Windows application-error dialogs. + +SIMD traversal consistency evidence on 2026-09-02: + +- [x] The shared `Numerics` vector-count helpers now cover same-lane spans and all fixed hardware widths. + AV1 decoder and current encoder hot paths use those helpers for complete-vector traversal instead of + repeating local modulo or last-vector calculations. Reverse-source indexing and algorithm-specific + partial-output groups remain explicit because they are not vector-count calculations. +- [x] Forward quantization, palette prediction, and scaled inter prediction construct width-specific SIMD + constants only when at least one vector batch will execute. Narrower dispatch tiers consume only the + remainder left by wider tiers before the scalar tail. +- [x] The net11.0 Release production assembly builds with zero warnings and zero errors. The test project + builds with zero errors while retaining the existing repository warning set. Roslynk reports zero + compiler errors, `git diff --check` passes, and `.gitattributes` is unchanged. + Foreground VSTest passes 63 of 63 focused quantizer, forward-transform, CDEF, restoration, palette, + intra, inter, film-grain, and super-resolution cases. + +Decoder exit gate: + +- [ ] Re-establish current-main native evidence for every supported AV1 tool through the complete production path. +- [ ] Re-establish presentation behavior against independent reference images at the correct output precision. +- [ ] Complete the decoder audit for managed execution, copies, per-block allocation, and segmented memory. +- [ ] Verify allocator ownership and exceptional-path disposal throughout the decoder. +- [ ] Record focused Release verification of the final tree; earlier checkpoint results do not close these gates. + +## AV1 encoder implementation + +Writer primitives are not an encoder. The public encoder remains incomplete until its complete decision and reconstruction paths follow the reference, every exposed option has production-path verification, and separately encoded output satisfies the one-unit per-sample acceptance limit. + +### 5. Define and enforce the encoder contract + +- [x] Use official libaom `main` at `d565eec60f084421fa34fc0534b760c6452b6a6c` as the encoder syntax, probability-model, transform, quantization, filtering, and bitstream reference. +- [x] Use the existing PNG, TIFF, and JPEG encoders as the ImageSharp architecture reference: generic `Image` input, encoder options taking precedence over converted format metadata and codec defaults, allocator-owned temporary storage, and deterministic disposal. +- [x] Treat source pixel type, source alpha representation, and decoded source bit depth as conversion inputs, never as output-eligibility checks. Do not pre-scan pixels before encoding. +- [x] Resolve output configuration once from explicit encoder options, converted `HeifMetadata`, and AV1 defaults in that order. Sanitize only combinations that cannot describe a legal requested output, and never write resolved values back to source metadata. +- [~] Finalize observable options for quality, effort, lossless mode, bit depth, chroma subsampling, alpha quality, metadata, and bounded sequences. Sequence repeat-count options override converted HEIF metadata like the existing animated encoders. Legacy JPEG treats the AV1-specific lossless and bit-depth options as inapplicable and continues with its native eight-bit encoding contract. AV1 sequences now follow the existing animated-image contract: the primary item reuses the first sync sample when the root is animated, while an excluded root is encoded once as the independent primary image and the sequence begins at frame index one. Final verification remains open. +- [x] Preserve high-bit-depth source precision through 16-bit RGB and native 10/12-bit component planes. +- [~] HEIF is registered through the default configuration module. Keep public AV1 capability claims limited to the paths covered by the encoder verification matrix until the remaining encoder work is complete. + +Encoder data-flow contract: + +1. Resolve immutable frame and sequence output settings before allocating codec state. +2. Convert each generic `ImageFrame` once through `PixelOperations` and the SIMD-first HEIF planar converter into native 8, 10, or 12-bit planes. Alpha is encoded as an auxiliary image when requested by the resolved output contract; it is not discarded through a source scan. +3. Reuse allocator-owned plane, row, block, transform, quantization, entropy, and reconstruction workspaces for the complete frame. No active path may allocate per row, block, transform, scanline, or SIMD tail. +4. Analyze and encode tiles directly from those planes, retaining reconstructed reference frames only for the bounded sequence lifetime. +5. Build OBU headers in bounded allocator-backed scratch and stream entropy-coded tile owners and container extents directly. Every ownership transfer is explicit, every owner is disposed exactly once, and no `ToArray` or file-sized copy crosses a layer boundary. +6. Iterate image frames using ImageSharp frame metadata and format-connecting metadata. Root-frame-only behavior is permitted only for an explicitly static output contract. + +Encoder verification contract: + +- Exercise source pixel formats independently from requested AV1 bit depth, chroma subsampling, alpha, and lossless/lossy mode. +- Run every SIMD operator through FeatureTestRunner at Vector512, Vector256, Vector128, and scalar tiers against an independent scalar oracle shaped from the same libaom revision. +- Cover discontiguous allocator buffers, constrained memory groups, cancellation, non-seekable output, multiple extents, auxiliary alpha, and bounded sequences. +- Validate produced AV1 payloads with current-main libaom and compare native planes before using ImageSharp self-decode as supplemental container coverage. + +### 6. Build the complete AV1 frame encoder + +- [~] SIMD-first RGB-to-native-plane conversion now feeds eight-bit and high-bit-depth bordered AV1 source frames directly, preserving ImageSharp's arbitrary packed-pixel input contract without an intermediate full-frame native-plane copy. +- [~] Auxiliary-alpha encoding now follows the same packed-pixel conversion boundary without scanning pixel contents or cloning the image. Source alpha is converted through ImageSharp's 16-bit pixel contract, deinterleaved with descending Vector512, Vector256, Vector128, and scalar traversal through the shared vector-count helpers, then scaled and rounded once by the existing native-sample writer directly into the final bordered monochrome source frame. One operation-wide allocator owner provides the packed and planar row views; there is no frame-sized alpha staging allocation or second owner. Exact 12-bit precision, physical border extension, the single 12-bytes-per-pixel row rent, and balanced return pass through the production converter. The complete 47-case frame-encoder set passes direct foreground net11 Release VSTest, and current-main `aomdec` at `a40ed1ea9e4ecc3df58a5bccb76623f2c94ae727` accepts the generated 8-, 10-, and 12-bit monochrome payloads. AVIF auxiliary item properties, references, and public activation remain open. +- [~] Forward transform families, transform workspace, and an allocation-free DC intra block boundary exist locally. For eight-bit and high-bit-depth samples, the composed boundary now follows current libaom's encoder order: predict into the reconstruction plane, subtract prediction from source, transform, quantize into separate qcoeff and dqcoeff storage, retain EOB and transform type, and inverse-transform only when EOB is nonzero so later blocks consume decoder-identical references. Prediction and subtraction retain their SIMD-first operators, independent source and reconstruction strides are preserved, and no frame-sized or per-block buffer is introduced. The block boundary consumes the real bordered encoder-plane regions and indexes their one-segment owner directly; this preserves physical row strides without a row copy and avoids the per-call enumerator allocation exposed by the initial array-only test. One reusable 61 KiB allocator owner supplies tightly packed residual, aligned transform-coefficient, dequantized-coefficient, and transform scratch spans across transform blocks; quantized coefficients write directly to the retained frame coefficient owner instead of being duplicated. A fixed 8x8 DC-intra superblock baseline now traverses the same recursive preorder and frame-edge pruning as the tile writer, gathers left references into that reusable block workspace, writes luma and chroma coefficient-owner slices in the writer's exact consumption order, and updates the caller-owned reconstruction planes for subsequent predictions. Stage-by-stage scalar-oracle, physical-border, retained-syntax, superblock-to-writer synchronization, high-bit-depth precision, and steady-state zero-allocation coverage passes 8 of 8 through direct net11 VSTest in Release. This is a legal fixed baseline, not complete partition or mode analysis. +- [~] The production tile writer walks raster superblocks, analyzes each immediately before entropy coding, reuses one decision workspace and one block workspace, and retains decoder-identical reconstructed references across each tile. Frames exceeding AV1's 4,096-sample tile-width or 4,096-by-2,304-sample tile-area limit now select the minimum uniform tile-column and tile-row logarithms used by current libaom. Every tile begins from the same normative frame probabilities, appends its independently finalized range-coded bytes to one bounded output allocation, and records only its offset and length in the picture-state owner. Closed byte and high-bit-depth operators feed the existing superblock boundary without runtime sample-type checks. Existing byte-exact and clipped-superblock tests cover traversal and coefficient indexing; a production 4,097-sample-wide lossless case crosses the first tile boundary and checks decoded pixels on both sides. Roslynk reports zero compiler errors; runtime and current-libaom verification of this multi-tile checkpoint remain pending. +- [~] A non-owning encoder-frame view now separates visible conversion regions from coded regions and performs complete left, top, right, bottom, and corner extension across each bordered plane. Current libaom uses 8-sample-aligned coded dimensions, a 32-sample-aligned luma stride with chroma stride derived from it, and a 64-pixel luma border for non-resized all-intra encoding. One operation-ready frame owner now rents the aligned Y, U, and V storage contiguously, exposes non-owning `Buffer2D` plane views, and returns the rent exactly once. A 4K 4:2:0 frame occupies about 13.0 MiB at 8-bit or 26.0 MiB at 10/12-bit; source and reconstruction therefore remain distinct frame owners rather than adding a full-frame copy. The corrected tests use this real ownership path and verify the exact 54 KiB 64x64 4:2:0 rent. The frame-encoder operation now instantiates matching source and reconstruction owners with ordinary `using` lifetimes and converts packed pixels directly into the source owner before extension. +- [~] Temporal delimiter, sequence header, frame header, combined-frame tile-group writing, uniform multi-tile layout, and reduced and non-reduced frame operations now exist locally. The remaining codec-tool and verification work is tracked below. +- [~] Implement superblock and partition analysis for every permitted block size and partition. Efforts zero through eight deliberately split every in-frame node to 8x8 blocks. Effort nine performs recursive live rate-distortion selection at complete 8x8 and 16x16 nodes, while effort ten extends the same search to complete 32x32, 64x64, and 128x128 nodes. Candidate order matches current libaom: `PARTITION_NONE`, `PARTITION_SPLIT`, `PARTITION_HORZ`, `PARTITION_VERT`, the four asymmetric partitions, then `PARTITION_HORZ_4` and `PARTITION_VERT_4`; the two 1-to-4 partitions are excluded at 128x128 as required by current libaom. Invalid chroma geometries are excluded before evaluation. Each candidate saves and restores the exact partition, coefficient, transform, and palette neighbor edges in one aligned block-workspace owner; trials neither allocate nor copy probability state. Recursive split trials publish each selected child's decoded mode, transform, coefficient, and palette contexts before evaluating its next sibling. Coefficient contexts are published per retained transform rather than broadcasting the first transform over an entire partition leaf. Large luma and chroma leaves are evaluated as bounded-64, raster-ordered transform tiles in the existing aligned workspace, and each winning plane is copied to retained storage once. Production picture state retains the compact 8x8 mode allocation below effort nine and explicitly selects 4x4 allocation granularity when sub-8x8 partitions are enabled. Effort-dependent pruning remains. +- [~] Implement intra mode search, palette, filter intra, chroma-from-luma, and intra-block copy decisions. Live luma search now covers all 13 zero-angle base modes and all six nonzero adjustments for each of the eight directional modes. Joint spatial chroma search covers the same 61 candidates, combines both chroma planes in one rate-distortion decision, and preserves the winning shared angle adjustment. Chroma-from-luma now searches the complete signed alpha alphabet from reconstructed luma and retains its joint U/V syntax. Filter-intra now searches all five predictors after ordinary luma modes. Palette entropy, retained state, production syntax, luma and paired chroma palette selection, screen-content activation, and joint intra-block-copy mode selection exist, but their full reference decision policy and separate-encoder parity remain unverified. +- [~] Implement inter mode search for bounded sequences, including reference selection and the decoder-supported inter tools. The sequence encoder retains the preceding reconstruction and, from effort six, searches a bounded full-pixel frame translation against that LAST_FRAME reference. Candidate discovery uses the existing SIMD-first squared-error kernels over a central analysis window, validates the winner over the complete coded luma plane, and charges its exact uncompressed-header bit count in the inter-frame rate-distortion domain. Pure translation is signaled as an identity-scale rotation/zoom model, matching current libaom's workaround for the AV1 translation-only axis defect. Each 8x8 inter-frame block first retains the complete intra candidate, then compares NEARESTMV, all three legal NEARMV dynamic-list entries, GLOBALMV, and all three legal NEWMV dynamic-list entries against it with live intra/inter, single-reference, mode, DRL, differential-vector, skip, transform, coefficient, and distortion costs. The initial NEWMV search retains the reference's cheap prediction-error stage, but every surviving mode now owns a complete transform, coefficient, skip, and distortion evaluation before mode selection; selected and candidate workspace views exchange ownership only on strict improvement. Inter trials remain in the existing shared workspace until they strictly beat the intra result, so losing trials require no backup buffer or copy. The tile writer emits the matching DRL path and normative context-selected `LAST_FRAME` reference tree instead of forcing every block through a segmentation feature. The selected DRL index reuses the filter-intra byte because those block syntax branches are mutually exclusive, preserving the existing packed state size. Packed short vectors reuse the existing picture-state owner. Effort six keeps a full-pixel fast path; effort seven refines each selected NEWMV through half- and quarter-pixel eight-tap prediction; effort eight adds the final eighth-pixel stage. The frame header advertises the matching precision, and the shared motion-vector entropy path emits fractional and high-precision symbols only when that precision permits them. Roslynk reports no compiler or scoped analyzer diagnostics, but runtime verification is pending. Additional retained reference pictures, compound prediction, and the remaining inter tools remain. +- [~] Current-libaom `av1_quantize_fp_no_qmatrix` arithmetic is implemented as a closed generic forward-quantizer family with Vector512, Vector256, Vector128, and scalar paths, raster-order output, coded 64-point coefficient limits, and scan-order EOB selection. High-bit-depth paths widen before multiplying instead of applying the eight-bit coefficient clamp. Lossless blocks use the AV1 4x4 Walsh-Hadamard transform, exact lossless quantization and dequantization, four-by-four-only transform syntax, and non-skipped residual coding. Transform search and coefficient optimization remain. +- [~] Implement real rate-distortion selection and make quality and effort change work, size, and output quality. The complete luma and joint chroma candidate sets, including chroma-from-luma, filter-intra, palette, and intra-block copy, now perform live rate-distortion selection. Public quality mapping and effort tiers through exhaustive uniform luma mode/transform search are implemented. Effort nine adds exact recursive 8x8 and 16x16 partition rate-distortion selection, and effort ten extends it through 128x128; effort-dependent pruning and the remaining sequence searches remain. +- [~] Frame effort now progressively expands the available current search: zero is DC-only, one adds every zero-angle spatial mode, two adds every legal directional adjustment, three refines the preliminary luma winner's transform type, four adds filter-intra and chroma-from-luma, and five adds adaptive palette and intra-block-copy analysis. Lower tiers do not signal unavailable sequence or frame tools, and tiers below five skip the whole-frame screen-content scan. Effort six enables `TX_MODE_SELECT` and compares the winning ordinary spatial or filter-intra luma mode as one 8x8 transform against four raster-ordered 4x4 transforms; each luma palette candidate owns that size comparison from effort six onward. Effort seven searches every legal 8x8 transform type inside every ordinary spatial candidate rather than refining only the preliminary winner. Effort eight also performs the 8x8-versus-four-4x4 comparison inside every ordinary spatial and filter-intra candidate, matching current libaom's per-candidate uniform-transform ownership. Effort nine additionally searches every legal partition at complete 8x8 and 16x16 nodes in current-libaom order, and effort ten extends that recursive search through 128x128. Prediction and residual construction run once per mode and are reused across its legal transform types. A 128x128 leaf evaluates four 64x64 luma transforms and as many as sixteen 32x32 transforms per 4:4:4 chroma plane, retaining sparse state at coefficient-area offsets. Residual emission follows AV1's bounded-region order, completing Y, U, and V for each 64x64 luma region before advancing. Every 4x4 transform searches all legal types with live coefficient contexts and reconstructed intra references. The search reuses the aligned block workspace, preserves only global improvements, and performs no per-block, per-partition, or per-transform rent. Non-skipped intra-block copy writes and costs the current-libaom unsplit variable-transform root; skipped intra-block copy emits no transform-partition symbol. Effort-dependent model and transform pruning remain. Decoder-visible production cases inspect the emitted restrictions and frame state and decode the produced streams, including real effort-nine streams selecting sub-8x8 and 8x16 rectangular blocks. The complete non-HEVC HEIF/AV1 namespace passes 9,077 of 9,077 through one foreground net11 Release VSTest run. The last independently built `aomdec`, from the then-current `a40ed1ea9e4ecc3df58a5bccb76623f2c94ae727` snapshot, accepts the previously generated effort-eight and effort-ten payloads as well as the existing palette and intra-block-copy payloads. The affected encoder, partition, and workspace surface passes 139 of 139 through one foreground net11 Release VSTest run. The net11 Release build and Roslynk compiler and analyzer passes report zero errors. +- [~] Encoder rate accounting converts the entropy writer's live inverse cumulative distributions into current-libaom fixed-point symbol costs without allocating or duplicating probability state. Read-only luma-mode, directional-delta, filter-intra, chroma-mode, block-skip, transform-size, transform-block-skip, and complete transform-coefficient queries share the exact distributions mutated by the subsequent entropy write. Complete coefficient costing follows current libaom's optimized shape: it returns immediately for an empty transform, uses the EOB-specific base-range context, fuses magnitude, sign, base-range, and Golomb accounting into one reverse traversal, and combines repeated full base-range chunks instead of replaying each emitted symbol. Tile-lifetime level and context scratch is reused, the one-coefficient path neither clears nor initializes the forward-neighbor level map, and steady-state queries allocate nothing. Transform-size writing and costing share one subdivision-depth calculation, while shared closed symbol operations keep the writer and cost mappings for transform skip, transform type, and EOB syntax identical without forcing the estimator through the writer's slower two-pass coefficient traversal. The current-libaom fixed-point RD combiner preserves 64-bit distortion and rounds the weighted 1/512-bit rate at the required boundary. Its key-frame multiplier follows libaom's squared DC-quantizer formula and exact 10/12-bit normalization. Live final-block selection evaluates all 61 legal 8x8 luma candidates: the 13 zero-angle base modes in current-libaom order, followed by six nonzero adjustments for each directional mode. Joint chroma selection evaluates the equivalent 61 spatial candidates, combines U and V distortion plus coefficient rate, and charges one live chroma-mode and shared-angle symbol over the actual subsampled 4x4, 4x8, or 8x8 geometry. Chroma-from-luma subsamples the reconstructed luma block once into fixed-stride Q3 stack scratch, subtracts the rounded mean, evaluates all 33 signed alpha values independently for each plane with complete transform RD, and combines the cached plane results across all 1,088 valid joint pairs with one live sign cost and the conditional U/V magnitude costs. This is the allocation-free equivalent of current libaom's exhaustive 33-value path: it requires 66 evaluation transforms rather than transforming every joint pair, preserves DC-before-CfL-before-spatial tie order, and fixes the implicit chroma transform to DCT-DCT. Filter-intra follows ordinary luma candidates, searches all five predictors in syntax order, and evaluates every legal transform while reusing one prepared prediction and source residual per filter mode. Every candidate includes its live mode, angle, filter mode, alpha, and coefficient rate plus normalized pixel-domain distortion. The corrected prepared reference edges retain the common-corner prefix and width-plus-height extent required by rectangular directional prediction. A shared encoder/decoder availability calculation selects reconstructed top-right and bottom-left extensions according to tile, frame, superblock, and block reconstruction order; unavailable extensions repeat the nearest coded endpoint. Missing top or left edges retain current libaom's perpendicular-sample and bit-depth-midpoint rules. Directional prediction applies the AV1 three-degree adjustment step and reuses transform workspace for zone-three transposition before the transform overwrites it, keeping candidate evaluation allocation-free. The winning luma and chroma signed adjustments are retained in the packed final-block state consumed by the tile writer. The tile writer invokes these reusable workspace-backed selectors after mapping current neighbors and immediately before writing each block, so later decisions see reconstructed samples, coefficient contexts, and CDF updates from every preceding block. Block skip is read only after the callback has combined every coded plane. Luma and chroma candidate scratch is partitioned from the encoder's single aligned reusable block workspace; transform-size search uses that owner for four retained 4x4 transform states, local coefficient contexts, and the compact trial reconstruction needed to preserve the best result. No candidate path rents a buffer per block or per transform. Only a newly winning candidate is copied into retained frame storage. Production fixtures force every luma base predictor, both extreme adjustments in all three directional zones, available top-right and bottom-left extensions, high-bit-depth adjustment propagation, exact signed luma and chroma angle-rate terms, joint U/V decisions, packed chroma state, and 4:2:0, 4:2:2, and 4:4:4 transform geometry. The CfL fixtures derive target chroma from a pilot production encode's actual reconstructed luma through an independent scalar Q3 oracle and prove exact positive/negative alpha syntax plus zero-residual DCT-DCT reconstruction for all three subsampling geometries at 8, 10, and 12 bits. The stable fixed-DC traversal comparison uses neutral samples for which both the baseline and live search select DC with non-skip coefficient syntax, instead of relying on textured content to happen to select the baseline mode. Luma palette selection now evaluates dominant-color and one-dimensional K-means candidates for every legal size, snaps near-cache colors with the reference threshold and tie order, removes duplicate snapped colors, extends boundary maps from active samples, and performs complete transform rate-distortion search. Ordinary DC and filter-intra candidates pay the palette-disabled symbol whenever screen-content syntax is enabled. The exact net11 Release rebuild reports 1,992 test-project warnings and zero errors, all 58 intra-superblock cases pass, all 8,935 AVIF cases pass, and all 230 HEIF cases pass. Remaining mode decision work includes transform-size coverage for filter-intra and palette, broader joint mode/transform refinement, and effort-dependent pruning. Ordinary intra blocks now remain non-skipped even when all transforms are empty; inter and intra-block-copy mode selection own their distinct skip-transform RD decisions. +- [~] The tile writer now publishes one packed coefficient context per covered 4x4 edge unit and derives luma/chroma skip plus DC-sign contexts from the complete transform edges using current-libaom units. Partition, transform, and coefficient neighbor state retains only the above and left context regions used by current libaom; the unused third top-left region, its granularity state, and its unused sentinel are removed. One picture owner packs segmentation, every tile's partition, luma, chroma, and transform edges, CDEF state, preceding quantizer, and encoded payload bounds into one clean byte allocation with typed non-owning views; together with the separately typed packed mode-information owner, the complete picture state uses two allocator rents rather than seven. Each encoded tile has independent neighbor and probability state while sharing the bounded output owner. Earlier aligned-length and balanced-return coverage exists; runtime allocation verification of the current multi-tile layout remains pending. +- [~] Encoder mode information now uses a frame-owned integer alias grid over a packed 8-byte value allocation, matching current libaom's `mi_grid_base` and `mi_alloc` relationship without a managed object or reference per 4x4 entry. The visible dimensions are aligned to eight luma samples, the grid stride and allocated row count are aligned to 32 mode-information units, and optional 8x8 allocation granularity reduces the value store in both dimensions exactly as current libaom does. One clean ImageSharp byte owner contains both independently typed regions, reducing libaom's two allocation lifetimes to one without a copy. At 4K, the 4x4 layout occupies about 6.0 MiB in total; the 8x8 layout occupies about 3.0 MiB. Exact geometry, clean allocation, typed lengths, aligned mapping, untouched row padding, and exactly-once return pass 4 of 4 direct net11 VSTest cases in Release. Every coded 4x4 cell covered by square, rectangular, or clipped edge blocks maps to its owning allocation entry before context-dependent symbols are written. Packed syntax, relative neighbor lookup, full block mapping, writer traversal, entropy, and OBU coverage pass 1,947 of 1,947 direct net11 VSTest cases in Release; complete mode decision still remains. +- [~] The superblock decision and palette-map workspace uses one reusable 40.3 KiB ImageSharp allocator owner. Its aligned 8.3 KiB decision region contains 1,024 explicitly packed 8-byte final-block entries and the 341 preorder partition bytes required by a complete 128x128-through-8x8 quadtree; its remaining 32 KiB contains the fixed 128x128 luma and chroma palette maps. This combines storage held separately by libaom; exact lifetime and size reconciliation remains part of the fresh allocation audit, and fewer owners alone does not establish an improvement. Palette colors have their own current-block value and are copied only to the picture edges that later blocks can reference, so enabling palette mode does not add 50 bytes to every final-block entry. Construction and the explicit per-superblock reset initialize every syntax field, including the nonzero sentinel that disables filter-intra prediction; pooled quantizer, prediction, partition, and current-palette bytes cannot leak into the next decision pass. Roslynk reports zero compiler errors for the current one-owner refactor; runtime allocation verification remains pending. +- [~] Finalized transform coefficients and packed EOB/type state now use raster-ordered, per-superblock plane segments matching current libaom's coefficient-pool geometry. One ImageSharp allocator owner replaces libaom's separate coefficient, EOB, and entropy-context allocations while preserving the full 1024 luma and 256-per-chroma 4x4 state capacity of a 128x128 4:2:0 superblock. The fixed 8x8 DC-intra traversal populates the owner's quantized coefficient and state slices while updating the caller-owned reconstruction plane directly, and a real tile-writer integration check proves that both sides consume identical luma and chroma areas. Complete mode decision still remains. +- [~] Tile partition writing now follows current libaom's recursive `write_modes_sb` preorder traversal and `update_ext_partition_context` edge updates directly. Bottom-edge blocks use the horizontal-alike partition CDF and right-edge blocks use the vertical-alike CDF; byte-exact regressions cover both paths after the previous calls were found reversed. Lossless chroma-from-luma availability now uses the subsampled plane block size shared with the decoder instead of the lossy 32x32 limit, preserving the correct UV-mode alphabet for each segment. The obsolete SVT-derived global geometry catalog and its unimplemented lookup are removed; transform geometry is derived in libaom's bounded 64x64 residual order, fixed intra transform-size symbols use the reference depth and neighbor contexts, and each derived transform size is persisted to the frame-owned mode information before the entropy snapshot and coefficient traversal consume it. Frame-edge and segmentation syntax use mode-information units, and 128x128 CDEF units use libaom's 0-to-3 indexing and first-block strength ownership. The focused transform-state regression passes 3 of 3 direct net11 VSTest cases in Release. Writer, entropy, and OBU coverage passes 1,957 of 1,957 direct net11 VSTest cases in Release, with 20 of 20 focused encoder and decoder chroma-from-luma cases. Partition and mode analysis still need to populate these retained decisions; variable inter-transform syntax remains part of later inter-frame support. +- [ ] Implement legal deblocking, CDEF, restoration, super-resolution, and film-grain signaling decisions. +- [~] The coefficient symbol encoder allocates its bounded level and raster-context workspaces with the encoder state and reuses them for every transform and sequence sample, matching current libaom's fixed-geometry compressor lifetime instead of renting scratch during the first coded frame. Its range coder matches current libaom's 64-bit coding window, bulk big-endian byte flush, and backward carry propagation while using one byte of allocator scratch per estimated output byte instead of the former 16-bit pre-carry storage. Production tiles finalize consecutively inside one encoder-owned bounded output allocation; the OBU writer consumes non-owning tile slices synchronously before the encoder is reset or disposed, so no payload owner transfer, second rent, or full-tile copy occurs. Exact-length test callers retain the copying overload. In-memory allocation coverage proves reset-to-offset and reset-to-zero reuse the same output owner; runtime allocation verification of the complete production multi-tile and sequence paths remains pending. +- [~] The planar conversion, DC intra prediction, residual construction, forward transform, and forward quantizer use descending SIMD dispatch: Vector512, Vector256, Vector128, then scalar. Residual construction matches current libaom's exact source-minus-prediction arithmetic for 8-bit and high-bit-depth planes, preserves independent row strides and unaligned starts, and writes directly into caller-owned signed-short storage without allocation. Candidate distortion reuses that residual workspace and widens signed 12-bit lanes before vector squaring, accumulating exact full-block SSE in 64-bit scalar storage. The composed block path delegates arithmetic to those closed operators and adds no allocation. Apply the same rule to every later hot-path family. +- [~] Residual tests verify misaligned planes, independent source, prediction, and destination strides, SIMD remainders, untouched padding, 8-bit, 10-bit, and 12-bit precision, every operator width independently of host acceleration, the scalar fallback, and zero per-transform allocations. +- [~] The unused coefficient-shape transform facade and its unimplemented N2, N4, and DC-only branches are removed. Finalized block encoding now follows the complete-transform path that current libaom uses before fast quantization; later rate-distortion search may add proven coefficient optimization without exposing inactive runtime throws. +- [~] Forward-quantizer FeatureTestRunner and zero-allocation tests compare every hardware tier with an independent scan-order scalar oracle shaped from current-main libaom. Both passed direct net11 VSTest in Release. +- [~] The combined-frame writer now completes the byte-counted uncompressed frame header before starting the optional multi-tile tile-group flag, matching current libaom's separate frame-header and tile-group writers. A non-uniform two-tile round trip verifies the explicit boundaries, both tile payloads, and complete stream consumption through direct net11 VSTest in Release. +- [~] The first internal frame-to-OBU operation encodes 8-, 10-, and 12-bit monochrome reduced still pictures through the production tile writer and production decoder. Coefficient context initialization now stores `min(abs(level), 127)`, matching current libaom; the previous signed clamp converted every negative transform coefficient to zero and selected invalid nonzero-map distributions. Signed dense and sparse entropy round trips, direct level-buffer saturation coverage, and eight constant/gradient frame cases pass 52 of 52 direct net11 VSTest cases in Release. Current-main `aomdec` accepts all eight emitted payloads. After winner-mode transform refinement, their decoded-frame MD5 values are `d09ea148582b9c93fa78e59426193bbc` (16x16 8-bit constant), `b83eedd5a84428f0120130253b30bdaa` (16x16 8-bit gradient), `f949f7422913e83dff07ee5e0a5087d3` (8x8 8-bit constant), `ae7233a94558978934469dcc4da764dd` (8x8 8-bit gradient), `09223b227f3abc3134d0a3ea15f70c0a` (8x8 10-bit constant), `539aab0e6e14bcaec271febfa8e25444` (8x8 10-bit gradient), `73117a8fc102e5d028f82444fc4d15ab` (8x8 12-bit constant), and `6936a2b62d7220dfb12f3763bb49965d` (8x8 12-bit gradient). This is an independently decodable baseline, not completion evidence for chroma, alpha, options, containers, or the public encoder. +- [x] The exact net11 Release rebuild completed at the established 1,005-warning repository baseline with zero errors. The complete HEIF/AV1 namespace passes 8,838 of 8,838 direct VSTest cases with zero failures or skips. Roslynk reports zero compiler errors and no diagnostics in the five changed C# files; `git diff --check` passes and `.gitattributes` is unchanged. +- [~] The same internal frame operation now produces 4:2:0, 4:2:2, and 4:4:4 payloads at 8, 10, and 12 bits. Twenty-one color cases cover constant and spatially varying input at aligned dimensions plus odd 13x11 visible dimensions for every chroma geometry. The production decoder consumes every payload, the decoded output retains non-neutral chroma, and current-main `aomdec` accepts all 29 monochrome and color outputs. After live spatial chroma mode selection, implicit chroma-transform correction, winner-mode luma-transform refinement, exhaustive chroma-from-luma alpha selection, and filter-intra search, the odd-dimension decoded-frame MD5 values are `9985f05790d2c9f5f28723ef86d5b89b` (4:2:0), `2ba2f1d0fcfef60394a5175553c7cb8b` (4:2:2), and `6aa7a2ed0dbf76ad2ec0c222585272d0` (4:4:4). This proves legal current-libaom payload syntax across native plane geometries; it does not yet prove target quality or native-plane equality with an independently encoded reference. +- [x] Spatial chroma candidates now use the implicit transform derived from the selected UV mode and active transform set, matching current libaom's `intra_mode_to_tx_type` and `av1_get_tx_type` behavior. The same shared derivation is consumed by the decoder, so coefficient scan order, entropy contexts, inverse reconstruction, and encoder rate estimates cannot drift between the two paths. The previous DCT-DCT candidate transform could produce syntactically accepted streams whose non-DC chroma coefficients were interpreted under a different implicit transform. Six production mode-decision cases retain nonzero U and V coefficients and assert the selected transform state across 4:2:0, 4:2:2, and 4:4:4; fifteen exact mapping cases cover every intra mode, reduced sets, and the 32x32 DCT-only fallback. The focused contract passes 21 of 21 direct net11 VSTest cases, the complete HEIF/AV1 namespace passes 8,947 of 8,947, the exact Release rebuild remains at 1,005 warnings and zero errors, and current-main `aomdec` accepts all 29 regenerated payloads. +- [~] Luma mode selection now evaluates each of its 61 mode-and-angle candidates with the mode-derived default transform used by current libaom's fast intra path. It then refines only the winning mode across all seven transform types permitted by the 8x8 intra set in transform-enum order. This removes the fixed DCT-DCT limitation while avoiding a 61-by-7 expansion; each trial includes live transform-type and coefficient rate, reconstructed pixel-domain distortion, and the existing aligned reusable block workspace. Eighteen exact-prediction production cases prove DCT-DCT wins equal-cost ties in reference order even when the first pass used a different default, while the 72x72 textured traversal proves a non-DCT transform with nonzero coefficients reaches retained syntax. Current-main `aomdec` accepts all 29 regenerated payloads. Special-mode transform-size coverage, full partition search, and broader effort-dependent joint mode/transform search remain. +- [x] Chroma-from-luma mode decision now reuses the decoder's SIMD-first 4:2:0, 4:2:2, and 4:4:4 reconstructed-luma preparation and prediction kernels for both byte and high-bit-depth encoder operators. The constant DC predictor for each chroma plane is computed once and its sample refills every alpha candidate, matching libaom's per-plane DC cache instead of rebuilding the same edge average 33 times. Each block uses 512 bytes of fixed stack scratch for the maximum 8-row predictor surface plus 792 bytes for complete U/V rate and distortion tables; no allocator owner, managed object, frame copy, or persistent buffer was added. Live probability costs exactly mirror current libaom's joint-sign ownership and conditional magnitude symbols. Nine production cases independently derive exact CfL targets from decoder-visible reconstructed luma at 8, 10, and 12 bits, and three entropy cases cover two nonzero signs plus each single-zero-plane form. The exact net11 Release rebuild remains at 1,005 warnings and zero errors, all 8,959 HEIF/AV1 tests pass, and current-main `aomdec` at `a40ed1ea9e4ecc3df58a5bccb76623f2c94ae727` accepts all 29 regenerated payloads. +- [x] Filter-intra mode decision now runs after ordinary luma modes in current-libaom order, evaluates all five recursive predictors, and refines each predictor across every legal 8x8 transform in transform-enum order. Strictly-better replacement preserves ordinary-mode and filter-mode tie order. Each filter prediction and its source residual are prepared once and reused across transform candidates, avoiding repeated recursive prediction while retaining SIMD-first predictor and subtraction operators. The stack cost is 192 bytes for eight-bit samples or 256 bytes for high-bit-depth samples; no allocator owner or managed buffer was added. Fifteen production cases force every filter mode at 8, 10, and 12 bits and prove retained filter syntax, zero-residual reconstruction, and the DCT-DCT equal-cost transform tie. The decoded-frame MD5 values selected by this checkpoint are `d7d68803763b95827483f14515281d3a` for the 8x8 10-bit gradient, `3f7e34d44c65d7797ad26b5cd4c35bf4` for the 8x8 12-bit gradient, and `9985f05790d2c9f5f28723ef86d5b89b`, `2ba2f1d0fcfef60394a5175553c7cb8b`, and `6aa7a2ed0dbf76ad2ec0c222585272d0` for the odd 4:2:0, 4:2:2, and 4:4:4 gradients. The exact net11 Release rebuild remains at 1,005 warnings and zero errors, 18 focused filter-intra, predictor-reference, syntax-cost, and allocation cases pass, all 8,974 HEIF/AV1 tests pass, and current-main `aomdec` at `a40ed1ea9e4ecc3df58a5bccb76623f2c94ae727` accepts all 29 regenerated payloads. +- [x] Empty-transform block skip now compares the complete live rate of the two decoder-identical syntax choices after luma and every coded chroma plane have been selected. Current libaom forces all-intra blocks to non-skip; this encoder retains that behavior for every non-empty block and for equal-cost empty blocks, but emits block skip when its adapted context cost is strictly lower than non-skip plus all empty-transform coefficient costs. Costing and writing share the same above-and-left skip-context calculation, and the coefficient estimator returns after the transform-block-skip symbol without reading coefficient storage. This adds no allocation, copy, or persistent state. A focused adapted-CDF regression proves both outcomes through the production decision helper, the two production all-zero fixtures still prove the default real block path, the exact net11 Release rebuild remains at 1,005 warnings and zero errors, all 8,975 HEIF/AV1 tests pass, and current-main `aomdec` at `a40ed1ea9e4ecc3df58a5bccb76623f2c94ae727` accepts all 29 regenerated payloads. +- [~] Palette entropy coding now mirrors current libaom's adaptive luma-mode, chroma-mode, palette-size, and spatial color-index distributions, together with its truncated-binary uniform code used by palette colors. The complete mutable palette probability graph is created once on first palette search or write, so the current palette-disabled frame path retains zero palette allocations. Three focused regressions cover every legal 2-through-8 color alphabet and every defined mode, size, and color-index context; all 1,928 entropy cases and all 8,978 HEIF/AV1 cases pass direct net11 Release VSTest. The exact Release rebuild remains at 1,005 warnings and zero errors. This checkpoint adds the exact entropy foundation only: palette candidate generation, retained color and index storage, mode decision, map tokenization, and production syntax remain incomplete, and no generated payload changed. +- [~] Luma and chroma palette-color coding now matches current libaom's neighbor-cache flags, sorted delta representation, wrapped V-plane deltas, strict delta-versus-raw V selection, and fixed-point color-rate model at 8, 10, and 12 bits. Encoder costing and emission use only fixed stack spans, including explicitly initialized cache-membership state, and steady-state color costing allocates zero managed bytes. The decoder consumes the same bounded color-syntax primitive after the tile reader derives its neighbor cache, removing duplicated color parsing without changing retained palette ownership. Nine focused syntax, exact palette decode, constrained-allocation, truncation, presentation, and allocation cases pass; all 1,933 entropy cases and all 8,983 HEIF/AV1 cases pass direct net11 Release VSTest. The exact Release rebuild remains at 1,005 warnings and zero errors. Retained encoder palette colors, neighbor caches, color-index maps, candidate generation, and production palette selection remain incomplete, and the compact 8-byte frame mode entries were not enlarged. +- [~] Palette color-index map coding now shares the exact current-libaom neighbor weights, stable color ordering, five context classes, first-index uniform code, and diagonal wavefront between encoder costing, encoder writing, and decoder parsing. The decoder's stack-allocated context scores are explicitly cleared before accumulation, removing an invalid dependency on uninitialized stack contents. Costing and writing use a closed generic operation while the shared driver owns traversal and context derivation, so the semantic operations remain independent of map layout and tail handling. The path adds no retained state or per-call managed allocation. Its allocation regression now runs one complete unmeasured hot-path window before measuring an independent 1,000-call steady-state window, so tiered-runtime transitions cannot make the full parallel suite report a one-time allocation as a recurring operation cost. Twelve focused map, exact palette decode, padding, trailing-bit, and allocation cases pass; all 1,941 entropy cases and all 8,991 HEIF/AV1 cases pass direct net11 Release VSTest. The exact Release rebuild remains at 1,005 warnings and zero errors. Production payloads remain unchanged because palette selection is still disabled; retained colors, neighbor caches, index-map storage, candidate generation, and production palette mode decision remain incomplete. +- [~] Retained encoder palette state and production palette writing now mirror current libaom's 50-byte palette-mode contents, separate luma and shared-chroma sizes, three eight-color planes, above-and-left sorted cache, 64-sample above-cache boundary, mode contexts, palette colors, color-index maps, and syntax order. The current block keeps one inline value in the reusable superblock workspace; only the 4x4-granularity top and left picture edges retain copies for later blocks. For a 3840x2160 tile these edges occupy about 73.4 KiB instead of about 6.2 MiB for a 50-byte palette value attached to every 8x8 mode allocation. Luma and chroma index maps occupy a fixed 32 KiB region of the single 40.3 KiB superblock-workspace owner. That owner is allocated with encoder state, matching libaom's compressor-state lifetime while removing libaom's separate palette allocation and cleanup path. The compact final-block decision region remains about 8.3 KiB. The writer caps map traversal to the coded plane count, writes maps before transform syntax, and publishes palette edges only after the current block has consumed preceding contexts. The previous eight focused size, alignment, ownership, cache-boundary, round-trip, map-consumption, and edge-publication cases passed with all 114 palette cases, all 1,942 entropy cases, and all 8,996 HEIF/AV1 cases through direct foreground net11 Release VSTest. The current one-owner refactor has zero Roslynk compiler errors; runtime verification remains pending. The current source reference is official libaom main at `d565eec60f084421fa34fc0534b760c6452b6a6c`. +- [~] Luma palette clustering now follows current libaom's one-dimensional search primitive exactly: equal-interval midpoint initialization, first-color tie order, rounded centroid means, deterministic empty-cluster replacement, the 50-iteration limit, and retention of the preceding state when distortion increases. Nearest-color assignment dispatches Vector512, Vector256, Vector128, then scalar through ImageSharp's shared vector-count helpers. Wider dispatch alone does not establish an end-to-end performance improvement. The primitive uses only bounded stack scratch and introduces no allocator rent, managed array, or per-row copy. Three independent tests cover exact centroid convergence, initialization order, 12-bit nearest-color distortion, destination bounds, and every hardware-intrinsic tier. The complete AVIF set passes 8,930 of 8,930 cases and the HEIF set passes 230 of 230 cases through direct foreground net11 Release VSTest. The exact net11 Release rebuild reports 1,050 solution warnings and zero errors, and Roslynk reports zero compiler errors. Candidate enumeration, palette-cache snapping, transform RD selection, and production activation remain in the open luma-palette checkpoint. +- [~] Live luma palette selection now follows current libaom's dominant-color and one-dimensional K-means candidate families, cache-bias threshold, sorted duplicate removal, active-edge map extension, and strict winner tie order. It evaluates both candidate families at every legal 2-through-8 size without the reference's speed-dependent pruning, then evaluates every legal transform. This is a controller deviation; evaluating more candidates does not establish improved quality, speed, or reference parity. Candidate storage remains bounded stack memory; the reusable maps come from the fixed encoder-lifetime superblock workspace, so palette search cannot introduce a first-use allocation. A production tile test proves that full 8x8 and clipped 5x3 blocks at 8 and 12 bits select exact colors and indices, extend the visible edges through coded padding, reconstruct every sample without coefficients, and emit a nonempty tile. The complete 57-case intra-superblock set, 8,931-case AVIF set, and 230-case HEIF set pass direct foreground net11 Release VSTest. The exact Release test-project build reports 1,992 baseline warnings and zero errors; Roslynk reports zero compiler errors and no touched-file analyzer warnings. Production frame activation remains gated until chroma palette mode and its rate accounting are complete. +- [~] Paired chroma palette clustering now preserves current libaom's squared two-component distance, first-centroid tie order, independently rounded U/V means, paired deterministic empty-cluster replacement, preceding-state retention on increased distortion, and 50-iteration limit. The source planes remain separate, with Vector512, Vector256, Vector128, then scalar dispatch through ImageSharp's shared vector-count helpers. An end-to-end improvement over the native implementation has not been established. Three independent tests cover exact paired convergence, midpoint initialization, 12-bit distance and index parity, untouched destination bounds, and every intrinsic tier. The exact Release test-project build reports 1,992 baseline warnings and zero errors; the focused three-case set, complete 8,934-case AVIF set, and complete 230-case HEIF set pass direct foreground net11 Release VSTest. Roslynk reports zero compiler errors and no touched-file analyzer warnings. Candidate integration and production activation remain in the open chroma-palette checkpoint. +- [~] Live paired chroma palette selection now follows current libaom's complete 2-through-8 color-size search, U-plane neighbor-cache snapping, stable U-ordered color pairs, shared U/V index map, implicit DCT-DCT transform, and strict rate-distortion winner replacement. It omits the reference's early header-cost pruning, keeps planar U/V source data separate, and reuses the existing prediction, residual, transform, quantization, and reconstruction operators. Omitting pruning is an unresolved decision-policy deviation, not an established improvement. The production tile regression proves both palette-mode probability branches, exact paired colors and indices, coefficient-free reconstruction, and nonempty syntax. The complete 58-case intra-superblock set, 8,935-case AVIF set, and 230-case HEIF set pass direct foreground net11 Release VSTest. The exact Release test-project build reports 1,992 baseline warnings and zero errors; Roslynk reports zero compiler errors and no touched-file analyzer warnings. Production frame activation remains the next checkpoint. +- [x] Production screen-content activation now matches current libaom's default good-quality detector: it scans only complete 16x16 luma blocks, normalizes palette samples to eight bits, admits 2-through-4-color blocks, and uses the reference's strict greater-than-ten-percent frame-area threshold. The same pass accumulates centered sums and squared sums at native precision, applies libaom's exact 10-bit and 12-bit variance rounding, and enables intra-block copy only when positive rounded per-pixel variance exceeds its strict one-twelfth frame-area threshold. A 256-bit stack bitset and fifth-color early exit replace libaom's larger per-block histogram without a second source scan or allocation. The adaptive sequence flag remains enabled and both frame flags are fixed before picture-state allocation. Focused regressions prove strict palette-threshold equality, high-bit-depth normalization, the exact variance rounding boundary, five-color rejection, emitted frame-header activation, actual production IBC selection, and production decode. The exact Release test-project build reports 1,992 baseline warnings and zero errors; all 9,242 non-HEVC HEIF/AV1 cases pass direct foreground net11 Release VSTest. Current-main `aomdec` at `a40ed1ea9e4ecc3df58a5bccb76623f2c94ae727` accepts all 31 payloads regenerated by the current test tree, including an actual IBC-coded 328x16 stream with decoded MD5 `677435e5af39c930af1178f91c34af6a`. Roslynk reports zero compiler errors and no touched-file analyzer warnings. +- [~] Intra-block-copy rate accounting now uses the live frame-local flag and displacement-vector distributions without copying or adapting either context during candidate measurement. Displacement-vector costing and writing share one closed symbol operation over the exact current-libaom joint, sign, magnitude-class, class-zero, and integer-offset syntax; final mode evaluation applies libaom's 120/128 displacement-rate weight with nearest-integer rounding. Independent fixed costs cover all four joint states, both signs, class zero, and large offset classes before adaptive writes, followed by an encoder/decoder round trip through the same sequence. Encoder and decoder reference-vector derivation now share the exact eight-candidate spatial scan, independent nearest and outer-region ranking, top-right partition geometry, clamping, and tile-relative fallback. Selected vectors use a naturally aligned pair of signed 16-bit components packed into the existing picture-state owner only when intra-block copy is permitted; a 3840x2160 frame retains 130,560 vectors in 510 KiB while leaving the compact 8-byte mode allocation unchanged. The tile writer derives the same reference and emits the retained vector without another allocation or copy. Coefficient costing and writing now select the inter transform sets and frame-local probability tables required by intra-block copy; independent tests verify every legal symbol against the exact default inter distribution and round-trip full and reduced sets from 4x4 through 32x32. Legal 8x8 hash discovery now indexes every visible source origin, including unaligned origins, in libaom's coarse-to-fine insertion order with the same 256-candidate bucket cap. A separable rolling hash fills one packed picture-lifetime workspace before reconstruction, then reuses that workspace for integer candidate links; exact wide or SIMD block comparison rejects hash collisions, and SIMD variance uses libaom's eight-bit normalization at 8, 10, and 12 bits. Power-of-two bucket arrays scale down with small images and stop at the reference's 16-bit limit, avoiding libaom's fixed six-size pointer table; the 3840x2160 search index occupies about 32.2 MiB and introduces no additional owner or frame copy. Above and left search rectangles, integer displacement legality, strict tie order, and live raw displacement rate follow current libaom. Motion-candidate ranking uses libaom's undiscounted probability cost and exact variance-domain error-per-bit scaling, separately from the later 120/128 final-mode discount. The allocation-free full-pixel core now follows current libaom's NSTEP search: it clamps the spatial reference to each legal region, traverses the fixed 15-stage radii and site order, skips equivalent centered 210-pixel stages, repeats progressively shorter paths, and compares their winners in the normalized variance domain. Paths above the speed-zero screen-content threshold continue through libaom's 256-pixel, one-pixel-step exhaustive mesh. Four adjacent byte or high-bit-depth candidates share each SIMD source load, strict row-major tie ordering is retained, and the final legal tail column remains searchable where libaom's current four-wide remainder loop omits it. Byte and high-bit-depth operators compute each 8x8 absolute difference with Vector128 before scalar fallback; high-bit-depth SAD remains in its native sample scale while its quantizer-derived rate multiplier uses libaom's normalized AC step. Production mode decision now derives the same spatial displacement reference used by the writer, deduplicates hash and full-pixel finalists in search order, and evaluates every surviving vector through complete luma and chroma transform RD. This differs from libaom's preliminary-error pruning by permitting a hash and pixel finalist from the same search region to compete using final syntax and reconstruction costs. It is an unresolved controller deviation, with no established quality or performance improvement. Prediction is prepared once per plane and vector, including integer or half-sample chroma phase, then reused across every legal inter transform without an allocator rent or frame copy. The joint comparison includes the live intra-block-copy flag, discounted displacement rate, skip flag, coefficient syntax, and normalized Y/U/V distortion; an empty transform alternative can win only when its complete skip cost is strictly lower, while conventional intra and earlier vectors retain tie precedence. Winning reconstruction, coefficients, transform state, DC modes, cleared palette/filter/CfL state, and displacement are copied once into the existing retained stores. Production regressions force the path at 8 and 12 bits and force 4:2:0 horizontal half-sample chroma with an unaligned reference. The former bulk local workspace occupied 2.75 KiB for byte samples or 3.375 KiB for high-bit-depth samples. Prediction, candidate, winning reconstruction, residual, and coefficient scratch now occupy one naturally aligned 3.125 KiB extension of the existing frame-reused block-workspace owner, matching libaom's reusable macroblock-scratch lifetime without adding an allocation; only the 128-byte reference, weight, and finalist arrays remain on the stack. The net11 Release solution build reports zero errors; all 2,082 focused transform, entropy, intra-block-copy, intra-superblock, and frame-encoder cases and all 9,242 non-HEVC HEIF/AV1 cases pass through direct foreground VSTest, with tiered compilation disabled only for the full allocation-sensitive suite. Adaptive production activation is complete, and the emitted frame flag remains authoritative for the complete frame rather than being invalidated after tile coding. +- [x] The expanded checkpoint exposed a pre-existing transform-block test that asserted uninitialized pooled padding was zero. The test now initializes the complete physical luma plane with a sentinel and proves the block operation leaves both adjacent padding samples unchanged. The exact net11 Release rebuild remains at 1,005 baseline warnings and zero errors, the focused allocator-order set passes 30 of 30 cases, and the complete HEIF/AV1 namespace passes 8,859 of 8,859 direct VSTest cases with zero failures or skips. +- [x] Combined-frame OBU output now counts the byte-aligned frame and tile-group headers, non-final tile-size fields, and owned tile payloads before emitting the OBU size. It retains only the small allocator-owned header scratch and writes each entropy-coded tile span directly from its detached owner, removing the second file-sized allocator rent and complete-payload copy. A 64 KiB regression proves exactly one sub-payload-sized byte rent with a balanced return and verifies the exact streamed tile tail; the existing two-tile round trip proves size-prefix and ordering parity. The focused writer and production-frame set passes 32 of 32 direct net11 VSTest cases, current-main `aomdec` accepts all 29 generated native-format payloads, and the complete HEIF/AV1 namespace passes 8,860 of 8,860 cases with zero failures or skips. +- [ ] The earlier fixed-block skip checkpoint was not equivalent to libaom's ordinary-intra policy. + Its all-zero-EOB conjunction produced valid streams, but ordinary intra retains non-skip syntax in the + reference. The takeover correction above aligns both fixed-DC traversal and live mode selection. + Earlier decoder acceptance, unchanged frame hashes, and one-byte size reductions did not establish + encoder-policy parity; the earlier 8,862-case result is historical evidence only. +- [x] Operation-wide allocation tracking now exercises a real 64x64 12-bit 4:4:4 frame through packed-pixel conversion, both native frame owners, picture and coefficient state, reusable block workspaces, entropy coding, OBU framing, and a non-seekable destination. It proves exactly one 60 KiB tile-output reservation from current libaom's all-intra 2.5x rule and balanced exactly-once returns for every tracked allocation before the operation completes. The focused ownership case passes 1 of 1 and the complete HEIF/AV1 namespace passes 8,863 of 8,863 direct net11 VSTest cases with zero failures or skips. +- [x] HEIF box offsets are now counted from the start of the encoded file instead of reading `Stream.Position`. This preserves ISO BMFF file-relative `iloc` offsets when the destination begins at a nonzero position and permits non-seekable output. Decoder item extents and image-sequence chunk offsets now resolve from that same file origin rather than the backing stream origin. Real legacy-JPEG HEIF round trips cover non-seekable output and a prefixed destination, while current-position AV1 decode covers both a still item and a five-frame sequence. All 96 encoder/decoder cases and all 38 sequence-parser cases pass direct net11 Release VSTest; the Release build remains at the established 1,005-warning baseline with zero errors. + +- [~] Current-libaom source comparison now drives uniform luma transform ownership at each effort boundary. Effort six retains the cheaper winner-only size decision for ordinary spatial and filter-intra modes, while every palette candidate already owns its size decision. Effort seven evaluates every legal 8x8 transform type for every ordinary spatial candidate. Effort eight and above make transform size part of every ordinary spatial and filter-intra candidate's rate-distortion result, so an 8x8-only preliminary comparison cannot discard the mode that wins with four 4x4 transforms. Prediction and subtraction are prepared once per mode and reused across transform types, matching the reference separation between prediction and transform search. Each 4x4 transform searches every legal type with coefficient contexts derived from retained transform edges and preceding trial blocks, while reconstructed top-right and bottom-left references follow production coding order. Palette prediction uses non-owning subregions of the retained color map, and filter-intra rebuilds each recursive prediction from reconstructed edges. The existing aligned block-workspace owner retains prediction, residual, coefficients, contexts, compact reconstruction, and four final states; no allocator rent, managed array, best-candidate re-transform, or full-block intermediate copy was added. Dense decision points now document scratch lifetime, enumeration tie order, global-winner publication, raster reconstruction dependencies, and the deliberate lower-effort shortcut. The packed encoder transform edges initialize to 64, matching libaom and the ImageSharp decoder before a coded neighbor publishes its size, and variable transform syntax remains gated to blocks larger than 4x4. The focused Release verification passes 13 of 13 cases across efforts zero through eight and ten, palette split selection, and transform-size selection. The complete non-HEVC HEIF/AV1 namespace passed 9,301 of 9,301 cases at that checkpoint. The `aomdec` built from the then-current `a40ed1ea9e4ecc3df58a5bccb76623f2c94ae727` snapshot accepts the generated effort-eight and effort-ten streams. Partition search and effort-dependent pruning remain. + +- [x] Intra-block-copy transform search now prepares motion compensation and subtraction once per plane, alternates the existing candidate and selected work buffers whenever a transform improves, and performs at most one final normalization copy into the caller-owned selected span. This matches current libaom's pointer-swap ownership without adding an allocation or a third reconstruction buffer. Inline documentation now records the scratch lifetime, strict transform tie order, skip-rate replacement, unsplit transform-root syntax, joint-plane winner retention, and final publication boundary. The focused Release encoder and intra-block-copy set passes 17 of 17 cases, the complete non-HEVC HEIF/AV1 namespace passes 9,301 of 9,301 cases with zero failures or skips, and current-main `aomdec` accepts the regenerated effort-five and effort-six intra-block-copy streams. + +- [x] Uniform four-by-four luma transform search now uses two compact reconstruction and coefficient views already available in the aligned mode-decision workspace. Legal transform trials write into the non-winning view and exchange span ownership only on strict rate-distortion improvement; the selected coefficients and strided reconstruction mosaic are published once after the type search so the next raster transform sees the required decoded edge. This removes reconstruction and coefficient copies on every improving transform without adding storage, changing tie order, or repeating a transform. All four representative palette, filter-intra, and effort-eight output hashes are unchanged, the focused Release set passes 13 of 13 cases, and current-main `aomdec` accepts every checked stream. + +- [x] Candidate distortion now follows the reference separation between immutable source residuals and reconstructed-pixel error. The forward-transform boundary accepts a read-only residual, so all transform types for one prediction reuse that block directly instead of copying it into transform scratch before every trial. The shared residual API now measures strided source-versus-reconstruction squared error with documented Vector512, Vector256, Vector128, and scalar traversal through ImageSharp's vector-count helpers, eliminating the former residual destination write and second reduction pass. The same path covers ordinary intra, filter intra, palette, chroma-from-luma, split transforms, and intra-block copy at 8, 10, and 12 bits without an allocation. Independent scalar, stride, tail, intrinsic-tier, and zero-allocation coverage passes with the 140-case focused encoder set; the complete non-HEVC HEIF/AV1 namespace passes 9,301 of 9,301. Representative palette, filter-intra, and effort-eight output hashes remain byte-identical, and current-main `aomdec` accepts every checked stream. + +- [x] Lossless still-image coding now follows current libaom's qindex-zero path without introducing a per-block allocation or a second frame buffer. The forward 4x4 Walsh-Hadamard transform and its transpose into the entropy pipeline's row-major coefficient order are allocation-free at Vector512, Vector256, Vector128, and scalar tiers; lossless quantization reconstructs the original transform coefficient exactly. The mode decision fixes lossless transforms to DCT-DCT syntax and four-by-four blocks, disables transform skip for nonzero residuals, and excludes the fixed-eight-by-eight intra-block-copy search that cannot represent the required lossless transform grid. The frame coefficient owner reserves the exact worst-case 2,048 transform states needed by both 128x128 4:4:4 chroma planes without adding an allocation. The color configuration derives its plane count from the monochrome flag, so high-bit-depth direct-frame writers and readers cannot retain contradictory mutable state. Public 8-bit, 10-bit, and 12-bit color and auxiliary-alpha round trips are pixel exact on their native sample lattices; direct 10-bit and 12-bit 4:4:4 frame round trips are also exact at native-plane precision. The Release build completes with zero errors, Roslynk reports zero compiler errors, and the complete non-HEVC HEIF/AV1 namespace passes 9,081 of 9,081 through one foreground VSTest run. An independently built generic `aomdec` from current official libaom `main` at `d565eec60f084421fa34fc0534b760c6452b6a6c` accepts all 59 current payloads, including the public color and auxiliary-alpha lossless streams at every supported precision. + +### 7. Write complete AVIF output + +- [~] The encoder-side AV1 codec configuration is now derived directly from the encoded sequence header and writes the fixed four-byte `av1C` record with empty `configOBUs`. The image payload retains the required sequence header, so the property introduces no sequence-header allocation, retention, or copy. Four production-header cases cover main, high, and professional profiles; 8-, 10-, and 12-bit precision; monochrome, 4:2:0, 4:2:2, and 4:4:4 sampling; exact fixed bytes; decoder reparsing; and header/property equivalence through direct net11 Release VSTest. Property-container emission and public AVIF activation remain open. +- [~] AV1 image properties now write `ispe`, `pixi`, `av1C`, `colr`, and `auxC` in current AVIF item order. Only `av1C` is essential; color and alpha items retain independent property sets and the registered alpha auxiliary type. The property container reacquires its span after nested expansion before patching `ipco`, removing the prior stale-buffer write, and selects compact or 15-bit `ipma` indices from the property count rather than the unrelated item count. A forced-growth color-plus-alpha case validates every property payload and association byte; a separate 43-item, 129-property case proves indices 127 through 129 and the extended essential bit. Both pass direct foreground net11 Release VSTest. Complete AVIF assembly remains open. +- [~] Explicit public AV1 encoding now writes a still-image AVIF with `avif` major brand, compatible `avif`, `mif1`, and `miaf` brands, one primary color item, an optional alpha auxiliary item, `auxl` from alpha to color, independent item properties, absolute version-one `iloc` extents, and a shared `mdat`. Quality uses current libaom's quantizer-to-qindex mapping with public quality 100 deliberately clamped from lossless qindex 0 to qindex 4. Effort controls the implemented search stages, and the resolved value is required explicitly by every internal frame, tile, and mode-decision operation rather than repeated as optional defaults. Encoder options take precedence over source metadata for 8-, 10-, and 12-bit monochrome, 4:2:0, 4:2:2, and 4:4:4 output. Alpha derives from the source pixel type without scanning pixels, and incompatible identity-matrix metadata is normalized without mutating the source image. +- [~] The production path writes color and alpha payloads sequentially through allocator-backed chunked storage, supports non-seekable and prefixed destinations, and does not materialize a complete file or payload copy. Uniform encoder-side `pixi` depth is written directly without allocating per-item channel-depth arrays; decoder-side non-uniform channel depths remain supported. The Release test project builds with zero errors, all 39 encoder cases pass, the complete non-HEVC HEIF namespace passes 9,277 of 9,277, and current official libaom accepts all 47 generated payloads. +- [x] Still-image AVIF metadata preservation now writes an unrestricted ICC `colr/prof` property before the independent `colr/nclx` property, Exif and XMP as separate `mdat` items, and one `cdsc` relationship from each metadata item to the primary color item. Exif stores the exact big-endian TIFF-header offset required by the HEIF item syntax; XMP uses the `mime` item type and `application/rdf+xml` content type. Existing ICC and XMP storage is read synchronously and copied once into final encoder storage rather than cloned into an intermediate array. `SkipMetadata` suppresses all three profile types while retaining the CICP values required to describe the encoded planes. The same option now reaches legacy JPEG payloads, whose encoder no longer writes application profiles or comments when metadata is disabled. +- [x] Exact container tests verify every emitted item declaration, name, MIME content type, `cdsc` relationship, Exif offset and payload, XMP payload, ICC/CICP property order, compact association byte, propertyless metadata exclusion, decoded profile value, and both `SkipMetadata` branches. The final HEIF encoder set passes 44 of 44 and the complete JPEG encoder set passes 257 of 257 through direct foreground net11 Release VSTest. The complete non-HEVC HEIF namespace passes 9,282 of 9,282 with no failure, crash, or detached test host, and current official libaom accepts all 47 current generated AV1 payloads. +- [x] A code-wide production HEIF/AV1 stack-storage audit, excluding HEVC, removed every block-sized, variable-length, or repeatedly nested scratch buffer. Spatial luma and chroma, filter-intra, chroma-from-luma, luma and chroma palette selection, and K-means iteration now use typed views over 642 signed-integer elements, about 2.51 KiB, at the start of the shared inter-prediction region. Those searches are sequential for one block, so the block-workspace owner does not grow and no rent, copy, or additional lifetime is introduced. CDEF directions, variances, and its 64-entry block list now append 1 KiB to the existing bounded operation owner instead of occupying hidden inline or explicit stack arrays. No remaining `stackalloc` depends on block dimensions, sample count, or runtime length; the largest remaining individual span is 128 bytes, and the remaining sites are fixed syntax, SIMD-lane, filter-tap, plane-metadata, or small candidate storage. The exact-owner test now proves the mode, palette, and reference-prediction views share one allocation. Roslynk reports zero compiler errors and no diagnostics in the changed files, the Release test-project build completes with the established 1,992 warnings and zero errors, 81 of 81 focused cases pass, and the complete non-HEVC HEIF/AV1 namespace passes 9,282 of 9,282 through one foreground net11 VSTest run. +- [~] Bounded public image-sequence output now emits an `avis` movie with version-one movie, track, and media headers; AV1 visual sample entries; exact run-length-compressed timing; per-sample sizes; 64-bit chunk offsets; and an explicit sync-sample table. The file type includes the required `miaf` compatibility brand, and every sequence now has the MIAF primary image item emitted by current libavif: normal sequences share the first sync-sample extent without another encode or copy, while separate-root sequences retain the still root as the primary image and begin timed samples at frame index one. The decoder allocates one final `Image`: the root is either the first timed sample or the separately decoded primary item, and each visible timed sample is decoded directly into a frame owned by that image. Exact quarter-turn presentation uses one frame-sized reusable pre-rotation buffer rather than a second image or a separately built frame collection. Color and optional auxiliary alpha use independently configured AV1 tracks linked by `auxl`. Lossless samples remain independently decodable key pictures and repeat the sequence header required for random access. Lossy continuation samples use LAST_FRAME inter prediction through the existing SIMD translational predictor; one track-scoped encoder session reuses its source allocation, packed-to-planar row storage and color converter, frame-sized coefficient storage, fixed-geometry picture and frame-header syntax state, tile/superblock/entropy cursors, block arithmetic workspace, complete probability graph, bounded tile-output owner, and OBU-header owner, and swaps two complete reconstruction buffers so the preceding decoded frame becomes the next reference without a plane copy. Sequence samples signal `still_picture=0` and use the complete non-reduced sequence and frame-header prefixes required for a multi-frame coded sequence. Frame payloads are written once into contiguous allocator-backed chunks per track. One compact managed table retains only offset, length, and duration for both tracks, and the bounded `moov` owner is patched once after its final size is known, so prefixed and non-seekable destinations require neither seeking nor a file-sized copy. The media timescale uses the exact representable least common multiple of animated frame-delay denominators and a documented microsecond fallback; zero delays become the smallest legal positive duration. Public lossless color-and-alpha round trips preserve all frames, distinct 24, 25, and 30 fps delays, finite or infinite repetition, ICC, Exif, and XMP metadata, while a separate case proves prefixed non-seekable output. Per-tile CDEF preset, preceding-quantizer state, and payload bounds now occupy one aligned region in the reusable allocator-owned picture buffer rather than separate managed arrays for every frame. The last verified net11 Release checkpoint completed with the established 1,005 warnings and zero errors, all 48 HEIF encoder cases passed, and the complete non-HEVC HEIF/AV1 namespace passed 9,317 of 9,317 through foreground VSTest. Current official libaom `main` at `d565eec60f084421fa34fc0534b760c6452b6a6c` accepted all 66 raw AV1 payloads regenerated by that suite. Verification of the current primary-item, separate-root, non-reduced sequence-header, retained-reference continuation, grid implementation, block-local motion search, conversion-row, probability, picture, frame-header, tile-cursor, tile-output, and OBU-header reuse, and multi-tile output is pending. Additional reference roles and compound prediction remain open. +- [~] Oversized still-image encoding now writes a derived AVIF grid when either source dimension exceeds the AV1 frame-header limit. Cells are encoded row-major from source rectangles without cropping to temporary images. Every coded cell uses the same at-most-65,536-sample extent for current-reader interoperability, with edge replication supplying the AVIF minimum 64-sample dimension and any final-row or final-column crop. Color and optional alpha grids use hidden AV1 items, ordered `dimg` references, one shared property set per plane, and independent descriptor payloads. The item-property length calculation counts every reused `ipma` association while retaining one `ipco` property definition. Grid dimensions that are odd along a requested subsampled axis promote the resolved color sampling to 4:4:4, for both explicit and default options. Exact descriptor, hidden-flag, reference-order, property-reuse, cell-padding, and public round-trip coverage is present, and Roslynk reports zero compiler errors. Runtime verification remains pending. +- [~] Write the correct AVIF file type, item information, locations, references, properties, AV1 configuration, dimensions, color, alpha, metadata, and media data. +- [~] Support single images, alpha auxiliary images, grids, multiple extents, and bounded image sequences in the final public scope. +- [x] Preserve ICC, Exif, and XMP according to encoder options. +- [~] Write CICP, range, chroma position, bit depth, and subsampling values that match the encoded planes. +- [x] Encode the current ImageSharp pixel lattice without inventing HEIF clean-aperture, rotation, or mirror properties. The HEIF decoder materializes those container transforms before returning an image, while ImageSharp encoders consistently preserve explicit Exif metadata without implicitly changing the pixels. +- [~] Stream output through allocator-backed chunked storage without file-sized copies or ToArray materialization. + +Encoder exit gate: + +- [ ] Current-main libaom accepts the payloads regenerated from the final tree. +- [ ] Reverify lossless output at public pixel and native-plane precision for 8-, 10-, and 12-bit output. +- [ ] Separately encoded lossy outputs differ by no more than one unit at every decoded output sample with reconciled settings; report maxima and counts exceeding one. +- [ ] Record equivalent end-to-end absolute timing, output size, quality, and allocation evidence after the source audit. +- [ ] 8, 10, and 12-bit monochrome, 4:2:0, 4:2:2, and 4:4:4 outputs pass. +- [ ] Alpha, grids, metadata, color profiles, transforms, and bounded sequences pass. +- [ ] ImageSharp decode of its own output is supplemental coverage only, never the sole oracle. +- [ ] Verify every exposed encoding combination through established ImageSharp conversion behavior. +- [ ] Focused Release and FeatureTestRunner verification passes with exact recorded evidence. + +## Architecture rules + +- Follow the JPEG color-converter operator architecture exactly. +- Each distinct prediction traversal owns a family-named predictor type. +- The family .Operator.cs file defines the nested static operator contract. +- Each semantic readonly struct belongs to that owner and implements concrete scalar, Vector128, Vector256, and Vector512 arithmetic for the shared traversal. +- Do not place a distinct predictor beneath a broad Av1IntraPredictor or Av1InterPredictor. +- Do not create semantic forwarding wrappers, top-level operator types, hardware-width-named operator types, CRTP contracts, or one file containing unrelated semantic operators. +- Forward transforms belong to Av1ForwardTransformer and its semantic operator files. +- Inverse axis transforms belong to Av1Inverse2dTransformer and its semantic operator files. +- Reconstruction output operators belong to Av1InverseTransformer. +- Shared lane primitives belong only in explicitly named Operations types. +- Dispatch from widest to narrowest supported SIMD width, then execute one scalar tail. +- Keep codec execution sequential. Do not add parallel execution inside the codec. +- Do not allocate per row, block, transform, scanline, or SIMD tail. +- Use ImageSharp allocators and pools. Do not use ToArray to cross an ownership boundary. +- On internal types, use public members when other types consume them; reserve private members for type-local behavior. +- Use established ImageSharp test data, allocator tracking, FeatureTestRunner, and reference-image comparison APIs. Do not build custom substitutes. +- Public XML documentation describes observable behavior only. +- Inline comments explain the current-libaom numerical rule, ownership boundary, edge extension, entropy ordering, or SIMD shape at technically complex points. +- Every multiline statement or declaration is followed by vertical whitespace. +- Do not edit .gitattributes directly. +- Do not install or download tools without explicit permission. + +## Final verification matrix + +- [ ] Release source build: net10.0, zero errors. +- [ ] Release source build: net11.0, zero errors. +- [ ] Scoped semantic inspection: zero compiler errors attributable to this work. +- [ ] Focused decoder syntax, reconstruction, ownership, presentation, and malformed-input tests. +- [ ] Focused encoder syntax, payload, container, precision, ownership, and option tests. +- [ ] FeatureTestRunner coverage for normal, narrower SIMD tiers, and scalar fallback. +- [ ] Constrained multi-group allocator coverage with balanced exactly-once returns. +- [ ] Exact native-plane comparisons against current-main libaom. +- [ ] Established final-presentation comparisons at the target pixel precision. +- [ ] Scoped StyleCop and vertical-whitespace inspection. +- [ ] No stale unsupported capability claims or removed-code references. +- [ ] No restore-source failures, background test hosts, detached processes, or crash-report popups. +- [ ] .gitattributes unchanged. +- [ ] git diff --check clean. +- [ ] Documentation records exact commands, counts, current-main reference revision evidence, and results. +- [ ] Commit only after the relevant checkpoint is genuinely complete. +- [ ] Do not push. diff --git a/ImageSharp.slnx b/ImageSharp.slnx index 2321e4ef42..3fec6841f7 100644 --- a/ImageSharp.slnx +++ b/ImageSharp.slnx @@ -114,6 +114,11 @@ + + + + + diff --git a/THIRD-PARTY-NOTICES.TXT b/THIRD-PARTY-NOTICES.TXT index eaf9f73f07..55dc937022 100644 --- a/THIRD-PARTY-NOTICES.TXT +++ b/THIRD-PARTY-NOTICES.TXT @@ -30,3 +30,145 @@ See the LICENSE file in the CoreFX project root for more information. https://github.com/dotnet/corefx/blob/17300169760c61a90cab8d913636c1058a30a8c1/LICENSE.TXT https://github.com/dotnet/corefx/blob/17300169760c61a90cab8d913636c1058a30a8c1/src/Common/src/CoreLib/System/IO/Stream.cs#L742 https://github.com/dotnet/corefx/blob/17300169760c61a90cab8d913636c1058a30a8c1/src/Common/src/CoreLib/System/IO/Stream.cs#L775 + + +License notice for the AOM AV1 Codec Library +----- + +Parts of the AV1 implementation are adapted from the AOM AV1 Codec Library. +https://aomedia.googlesource.com/aom/ + +Copyright (c) 2016, Alliance for Open Media. All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions +are met: + +1. Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + +2. Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in + the documentation and/or other materials provided with the + distribution. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS +FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE +COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, +INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, +BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; +LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER +CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN +ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE +POSSIBILITY OF SUCH DAMAGE. + +Alliance for Open Media Patent License 1.0 + +1. License Terms. + +1.1. Patent License. Subject to the terms and conditions of this License, each + Licensor, on behalf of itself and successors in interest and assigns, + grants Licensee a non-sublicensable, perpetual, worldwide, non-exclusive, + no-charge, royalty-free, irrevocable (except as expressly stated in this + License) patent license to its Necessary Claims to make, use, sell, offer + for sale, import or distribute any Implementation. + +1.2. Conditions. + +1.2.1. Availability. As a condition to the grant of rights to Licensee to make, + sell, offer for sale, import or distribute an Implementation under + Section 1.1, Licensee must make its Necessary Claims available under + this License, and must reproduce this License with any Implementation + as follows: + + a. For distribution in source code, by including this License in the + root directory of the source code with its Implementation. + + b. For distribution in any other form (including binary, object form, + and/or hardware description code (e.g., HDL, RTL, Gate Level Netlist, + GDSII, etc.)), by including this License in the documentation, legal + notices, and/or other written materials provided with the + Implementation. + +1.2.2. Additional Conditions. This license is directly from Licensor to + Licensee. Licensee acknowledges as a condition of benefiting from it + that no rights from Licensor are received from suppliers, distributors, + or otherwise in connection with this License. + +1.3. Defensive Termination. If any Licensee, its Affiliates, or its agents + initiates patent litigation or files, maintains, or voluntarily + participates in a lawsuit against another entity or any person asserting + that any Implementation infringes Necessary Claims, any patent licenses + granted under this License directly to the Licensee are immediately + terminated as of the date of the initiation of action unless 1) that suit + was in response to a corresponding suit regarding an Implementation first + brought against an initiating entity, or 2) that suit was brought to + enforce the terms of this License (including intervention in a third-party + action by a Licensee). + +1.4. Disclaimers. The Reference Implementation and Specification are provided + "AS IS" and without warranty. The entire risk as to implementing or + otherwise using the Reference Implementation or Specification is assumed + by the implementer and user. Licensor expressly disclaims any warranties + (express, implied, or otherwise), including implied warranties of + merchantability, non-infringement, fitness for a particular purpose, or + title, related to the material. IN NO EVENT WILL LICENSOR BE LIABLE TO + ANY OTHER PARTY FOR LOST PROFITS OR ANY FORM OF INDIRECT, SPECIAL, + INCIDENTAL, OR CONSEQUENTIAL DAMAGES OF ANY CHARACTER FROM ANY CAUSES OF + ACTION OF ANY KIND WITH RESPECT TO THIS LICENSE, WHETHER BASED ON BREACH + OF CONTRACT, TORT (INCLUDING NEGLIGENCE), OR OTHERWISE, AND WHETHER OR + NOT THE OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +2. Definitions. + +2.1. Affiliate. "Affiliate" means an entity that directly or indirectly + Controls, is Controlled by, or is under common Control of that party. + +2.2. Control. "Control" means direct or indirect control of more than 50% of + the voting power to elect directors of that corporation, or for any other + entity, the power to direct management of such entity. + +2.3. Decoder. "Decoder" means any decoder that conforms fully with all + non-optional portions of the Specification. + +2.4. Encoder. "Encoder" means any encoder that produces a bitstream that can + be decoded by a Decoder only to the extent it produces such a bitstream. + +2.5. Final Deliverable. "Final Deliverable" means the final version of a + deliverable approved by the Alliance for Open Media as a Final + Deliverable. + +2.6. Implementation. "Implementation" means any implementation, including the + Reference Implementation, that is an Encoder and/or a Decoder. An + Implementation also includes components of an Implementation only to the + extent they are used as part of an Implementation. + +2.7. License. "License" means this license. + +2.8. Licensee. "Licensee" means any person or entity who exercises patent + rights granted under this License. + +2.9. Licensor. "Licensor" means (i) any Licensee that makes, sells, offers + for sale, imports or distributes any Implementation, or (ii) a person + or entity that has a licensing obligation to the Implementation as a + result of its membership and/or participation in the Alliance for Open + Media working group that developed the Specification. + +2.10. Necessary Claims. "Necessary Claims" means all claims of patents or + patent applications, (a) that currently or at any time in the future, + are owned or controlled by the Licensor, and (b) (i) would be an + Essential Claim as defined by the W3C Policy as of February 5, 2004 + (https://www.w3.org/Consortium/Patent-Policy-20040205/#def-essential) + as if the Specification was a W3C Recommendation; or (ii) are infringed + by the Reference Implementation. + +2.11. Reference Implementation. "Reference Implementation" means an Encoder + and/or Decoder released by the Alliance for Open Media as a Final + Deliverable. + +2.12. Specification. "Specification" means the specification designated by + the Alliance for Open Media as a Final Deliverable for which this + License was issued. diff --git a/shared-infrastructure b/shared-infrastructure index 52fc1c3e87..a835a9d74e 160000 --- a/shared-infrastructure +++ b/shared-infrastructure @@ -1 +1 @@ -Subproject commit 52fc1c3e87a755b2b17ce77e8cd22ec4bea17bcb +Subproject commit a835a9d74e82b2d32b580a7902eb2699ebc47098 diff --git a/src/ImageSharp/Common/Helpers/DisposableDictionary.cs b/src/ImageSharp/Common/Helpers/DisposableDictionary.cs new file mode 100644 index 0000000000..d96d4387fd --- /dev/null +++ b/src/ImageSharp/Common/Helpers/DisposableDictionary.cs @@ -0,0 +1,53 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Common.Helpers; + +/// +/// Dictionary of objects, which is itself . +/// +/// The type of the key. +/// Tye type of value, needs to implement . +public sealed class DisposableDictionary : Dictionary, IDisposable + where TKey : notnull + where TValue : IDisposable +{ + private bool disposedValue; + + /// + public DisposableDictionary() + : base() + { + } + + /// + public DisposableDictionary(int capacity) + : base(capacity) + { + } + + /// + public void Dispose() + { + // Do not change this code. Put cleanup code in 'Dispose(bool disposing)' method + this.Dispose(disposing: true); + GC.SuppressFinalize(this); + } + + private void Dispose(bool disposing) + { + if (!this.disposedValue) + { + if (disposing) + { + foreach (KeyValuePair pair in this) + { + pair.Value?.Dispose(); + } + } + + this.Clear(); + this.disposedValue = true; + } + } +} diff --git a/src/ImageSharp/Common/Helpers/DisposableList.cs b/src/ImageSharp/Common/Helpers/DisposableList.cs new file mode 100644 index 0000000000..da71670c8d --- /dev/null +++ b/src/ImageSharp/Common/Helpers/DisposableList.cs @@ -0,0 +1,51 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Common.Helpers; + +/// +/// List of objects, which is itself . +/// +/// Tye type of value, needs to implement . +public sealed class DisposableList : List, IDisposable + where TValue : IDisposable +{ + private bool disposedValue; + + /// + public DisposableList() + : base() + { + } + + /// + public DisposableList(int capacity) + : base(capacity) + { + } + + /// + public void Dispose() + { + // Do not change this code. Put cleanup code in 'Dispose(bool disposing)' method + this.Dispose(disposing: true); + GC.SuppressFinalize(this); + } + + private void Dispose(bool disposing) + { + if (!this.disposedValue) + { + if (disposing) + { + foreach (TValue item in this) + { + item?.Dispose(); + } + } + + this.Clear(); + this.disposedValue = true; + } + } +} diff --git a/src/ImageSharp/Common/Helpers/Numerics.cs b/src/ImageSharp/Common/Helpers/Numerics.cs index e5a6b45493..b1c6f29f19 100644 --- a/src/ImageSharp/Common/Helpers/Numerics.cs +++ b/src/ImageSharp/Common/Helpers/Numerics.cs @@ -1024,6 +1024,46 @@ public static nuint Vector512Count(this ReadOnlySpan span) where TVector : struct => (uint)span.Length / (uint)Vector512.Count; + /// + /// Gets the count of vectors that safely fit into a span whose element type matches the vector lane type. + /// + /// The type of the span elements and vector lanes. + /// The given span. + /// Count of vectors that safely fit into the span. + public static nuint Vector128Count(this ReadOnlySpan span) + where TVector : struct + => (uint)span.Length / (uint)Vector128.Count; + + /// + /// Gets the count of vectors that safely fit into a span whose element type matches the vector lane type. + /// + /// The type of the span elements and vector lanes. + /// The given span. + /// Count of vectors that safely fit into the span. + public static nuint Vector256Count(this ReadOnlySpan span) + where TVector : struct + => (uint)span.Length / (uint)Vector256.Count; + + /// + /// Gets the count of vectors that safely fit into a span whose element type matches the vector lane type. + /// + /// The type of the span elements and vector lanes. + /// The given span. + /// Count of vectors that safely fit into the span. + public static nuint Vector512Count(this ReadOnlySpan span) + where TVector : struct + => (uint)span.Length / (uint)Vector512.Count; + + /// + /// Gets the count of vectors that safely fit into the given length. + /// + /// The type of the vector. + /// The given length. + /// Count of vectors that safely fit into the length. + public static nuint Vector128Count(int length) + where TVector : struct + => (uint)length / (uint)Vector128.Count; + /// /// Gets the count of vectors that safely fit into length. /// diff --git a/src/ImageSharp/Common/Helpers/SimdUtils.HwIntrinsics.cs b/src/ImageSharp/Common/Helpers/SimdUtils.HwIntrinsics.cs index 1740df5b59..c3b934b182 100644 --- a/src/ImageSharp/Common/Helpers/SimdUtils.HwIntrinsics.cs +++ b/src/ImageSharp/Common/Helpers/SimdUtils.HwIntrinsics.cs @@ -9,13 +9,12 @@ using System.Runtime.Intrinsics.Arm; using System.Runtime.Intrinsics.X86; using SixLabors.ImageSharp.Common.Helpers; -using SixLabors.ImageSharp.PixelFormats; namespace SixLabors.ImageSharp; internal static partial class SimdUtils { - public static class HwIntrinsics + public static partial class HwIntrinsics { #pragma warning disable SA1117 // Parameters should be on same line or separate lines #pragma warning disable SA1137 // Elements should have the same indentation @@ -25,31 +24,15 @@ public static class HwIntrinsics [MethodImpl(MethodImplOptions.AggressiveInlining)] public static Vector512 PermuteMaskDeinterleave16x32() => Vector512.Create(0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15); - [MethodImpl(MethodImplOptions.AggressiveInlining)] - public static Vector256 PermuteMaskEvenOdd8x32() => Vector256.Create(0, 0, 0, 0, 2, 0, 0, 0, 4, 0, 0, 0, 6, 0, 0, 0, 1, 0, 0, 0, 3, 0, 0, 0, 5, 0, 0, 0, 7, 0, 0, 0).AsUInt32(); - [MethodImpl(MethodImplOptions.AggressiveInlining)] public static Vector256 PermuteMaskSwitchInnerDWords8x32() => Vector256.Create(0, 0, 0, 0, 1, 0, 0, 0, 4, 0, 0, 0, 5, 0, 0, 0, 2, 0, 0, 0, 3, 0, 0, 0, 6, 0, 0, 0, 7, 0, 0, 0).AsUInt32(); - [MethodImpl(MethodImplOptions.AggressiveInlining)] - private static Vector256 MoveFirst24BytesToSeparateLanes() => Vector256.Create(0, 0, 0, 0, 1, 0, 0, 0, 2, 0, 0, 0, 6, 0, 0, 0, 3, 0, 0, 0, 4, 0, 0, 0, 5, 0, 0, 0, 7, 0, 0, 0).AsUInt32(); - - [MethodImpl(MethodImplOptions.AggressiveInlining)] - internal static Vector256 ExtractRgb() => Vector256.Create(0, 3, 6, 9, 1, 4, 7, 10, 2, 5, 8, 11, 0xFF, 0xFF, 0xFF, 0xFF, 0, 3, 6, 9, 1, 4, 7, 10, 2, 5, 8, 11, 0xFF, 0xFF, 0xFF, 0xFF); - [MethodImpl(MethodImplOptions.AggressiveInlining)] private static Vector128 ShuffleMaskPad4Nx16() => Vector128.Create(0, 1, 2, 0x80, 3, 4, 5, 0x80, 6, 7, 8, 0x80, 9, 10, 11, 0x80); [MethodImpl(MethodImplOptions.AggressiveInlining)] private static Vector128 ShuffleMaskSlice4Nx16() => Vector128.Create(0, 1, 2, 4, 5, 6, 8, 9, 10, 12, 13, 14, 0x80, 0x80, 0x80, 0x80); - [MethodImpl(MethodImplOptions.AggressiveInlining)] - private static Vector256 ShuffleMaskShiftAlpha() => Vector256.Create( - (byte)0, 1, 2, 4, 5, 6, 8, 9, 10, 12, 13, 14, 3, 7, 11, 15, - 0, 1, 2, 4, 5, 6, 8, 9, 10, 12, 13, 14, 3, 7, 11, 15); - - [MethodImpl(MethodImplOptions.AggressiveInlining)] - public static Vector256 PermuteMaskShiftAlpha8x32() => Vector256.Create(0u, 1, 2, 4, 5, 6, 3, 7); #pragma warning restore SA1137 // Elements should have the same indentation #pragma warning restore SA1117 // Parameters should be on same line or separate lines @@ -1038,171 +1021,5 @@ internal static void FloatToByteSaturate( } } } - - internal static void PackFromRgbPlanesAvx2Reduce( - ref ReadOnlySpan redChannel, - ref ReadOnlySpan greenChannel, - ref ReadOnlySpan blueChannel, - ref Span destination) - { - ref Vector256 rBase = ref Unsafe.As>(ref MemoryMarshal.GetReference(redChannel)); - ref Vector256 gBase = ref Unsafe.As>(ref MemoryMarshal.GetReference(greenChannel)); - ref Vector256 bBase = ref Unsafe.As>(ref MemoryMarshal.GetReference(blueChannel)); - ref byte dBase = ref Unsafe.As(ref MemoryMarshal.GetReference(destination)); - - nuint count = redChannel.Vector256Count(); - - Vector256 control1 = PermuteMaskEvenOdd8x32(); - - Vector256 control2 = PermuteMaskShiftAlpha8x32(); - Vector256 a = Vector256.Create((byte)255); - - Vector256 shuffleAlpha = ShuffleMaskShiftAlpha(); - - for (nuint i = 0; i < count; i++) - { - Vector256 r0 = Unsafe.Add(ref rBase, i); - Vector256 g0 = Unsafe.Add(ref gBase, i); - Vector256 b0 = Unsafe.Add(ref bBase, i); - - r0 = Avx2.PermuteVar8x32(r0.AsUInt32(), control1).AsByte(); - g0 = Avx2.PermuteVar8x32(g0.AsUInt32(), control1).AsByte(); - b0 = Avx2.PermuteVar8x32(b0.AsUInt32(), control1).AsByte(); - - Vector256 rg = Avx2.UnpackLow(r0, g0); - Vector256 b1 = Avx2.UnpackLow(b0, a); - - Vector256 rgb1 = Avx2.UnpackLow(rg.AsUInt16(), b1.AsUInt16()).AsByte(); - Vector256 rgb2 = Avx2.UnpackHigh(rg.AsUInt16(), b1.AsUInt16()).AsByte(); - - rg = Avx2.UnpackHigh(r0, g0); - b1 = Avx2.UnpackHigh(b0, a); - - Vector256 rgb3 = Avx2.UnpackLow(rg.AsUInt16(), b1.AsUInt16()).AsByte(); - Vector256 rgb4 = Avx2.UnpackHigh(rg.AsUInt16(), b1.AsUInt16()).AsByte(); - - rgb1 = Avx2.Shuffle(rgb1, shuffleAlpha); - rgb2 = Avx2.Shuffle(rgb2, shuffleAlpha); - rgb3 = Avx2.Shuffle(rgb3, shuffleAlpha); - rgb4 = Avx2.Shuffle(rgb4, shuffleAlpha); - - rgb1 = Avx2.PermuteVar8x32(rgb1.AsUInt32(), control2).AsByte(); - rgb2 = Avx2.PermuteVar8x32(rgb2.AsUInt32(), control2).AsByte(); - rgb3 = Avx2.PermuteVar8x32(rgb3.AsUInt32(), control2).AsByte(); - rgb4 = Avx2.PermuteVar8x32(rgb4.AsUInt32(), control2).AsByte(); - - ref byte d1 = ref Unsafe.Add(ref dBase, 24 * 4 * i); - ref byte d2 = ref Unsafe.Add(ref d1, 24); - ref byte d3 = ref Unsafe.Add(ref d2, 24); - ref byte d4 = ref Unsafe.Add(ref d3, 24); - - Unsafe.As>(ref d1) = rgb1; - Unsafe.As>(ref d2) = rgb2; - Unsafe.As>(ref d3) = rgb3; - Unsafe.As>(ref d4) = rgb4; - } - - int slice = (int)count * Vector256.Count; - redChannel = redChannel[slice..]; - greenChannel = greenChannel[slice..]; - blueChannel = blueChannel[slice..]; - destination = destination[slice..]; - } - - internal static void PackFromRgbPlanesAvx2Reduce( - ref ReadOnlySpan redChannel, - ref ReadOnlySpan greenChannel, - ref ReadOnlySpan blueChannel, - ref Span destination) - { - ref Vector256 rBase = ref Unsafe.As>(ref MemoryMarshal.GetReference(redChannel)); - ref Vector256 gBase = ref Unsafe.As>(ref MemoryMarshal.GetReference(greenChannel)); - ref Vector256 bBase = ref Unsafe.As>(ref MemoryMarshal.GetReference(blueChannel)); - ref Vector256 dBase = ref Unsafe.As>(ref MemoryMarshal.GetReference(destination)); - - nuint count = redChannel.Vector256Count(); - Vector256 control1 = PermuteMaskEvenOdd8x32(); - Vector256 a = Vector256.Create((byte)255); - - for (nuint i = 0; i < count; i++) - { - Vector256 r0 = Unsafe.Add(ref rBase, i); - Vector256 g0 = Unsafe.Add(ref gBase, i); - Vector256 b0 = Unsafe.Add(ref bBase, i); - - r0 = Avx2.PermuteVar8x32(r0.AsUInt32(), control1).AsByte(); - g0 = Avx2.PermuteVar8x32(g0.AsUInt32(), control1).AsByte(); - b0 = Avx2.PermuteVar8x32(b0.AsUInt32(), control1).AsByte(); - - Vector256 rg = Avx2.UnpackLow(r0, g0); - Vector256 b1 = Avx2.UnpackLow(b0, a); - - Vector256 rgb1 = Avx2.UnpackLow(rg.AsUInt16(), b1.AsUInt16()).AsByte(); - Vector256 rgb2 = Avx2.UnpackHigh(rg.AsUInt16(), b1.AsUInt16()).AsByte(); - - rg = Avx2.UnpackHigh(r0, g0); - b1 = Avx2.UnpackHigh(b0, a); - - Vector256 rgb3 = Avx2.UnpackLow(rg.AsUInt16(), b1.AsUInt16()).AsByte(); - Vector256 rgb4 = Avx2.UnpackHigh(rg.AsUInt16(), b1.AsUInt16()).AsByte(); - - ref Vector256 d0 = ref Unsafe.Add(ref dBase, i * 4); - d0 = rgb1; - Unsafe.Add(ref d0, 1) = rgb2; - Unsafe.Add(ref d0, 2) = rgb3; - Unsafe.Add(ref d0, 3) = rgb4; - } - - int slice = (int)count * Vector256.Count; - redChannel = redChannel[slice..]; - greenChannel = greenChannel[slice..]; - blueChannel = blueChannel[slice..]; - destination = destination[slice..]; - } - - internal static void UnpackToRgbPlanesAvx2Reduce( - ref Span redChannel, - ref Span greenChannel, - ref Span blueChannel, - ref ReadOnlySpan source) - { - ref Vector256 rgbByteSpan = ref Unsafe.As>(ref MemoryMarshal.GetReference(source)); - ref Vector256 destRRef = ref Unsafe.As>(ref MemoryMarshal.GetReference(redChannel)); - ref Vector256 destGRef = ref Unsafe.As>(ref MemoryMarshal.GetReference(greenChannel)); - ref Vector256 destBRef = ref Unsafe.As>(ref MemoryMarshal.GetReference(blueChannel)); - - Vector256 extractToLanesMask = MoveFirst24BytesToSeparateLanes(); - Vector256 extractRgbMask = ExtractRgb(); - Vector256 rgb, rg, bx; - Vector256 r, g, b; - - // Each iteration consumes 8 Rgb24 pixels (24 bytes) but starts with a 32-byte load, - // so we need 3 extra pixels of addressable slack beyond the vectorized chunk. - const int bytesPerRgbStride = 24; - nuint count = source.Length > 3 ? (uint)(source.Length - 3) / 8 : 0; - for (nuint i = 0; i < count; i++) - { - rgb = Avx2.PermuteVar8x32(Unsafe.AddByteOffset(ref rgbByteSpan, (uint)(bytesPerRgbStride * i)).AsUInt32(), extractToLanesMask).AsByte(); - - rgb = Avx2.Shuffle(rgb, extractRgbMask); - - rg = Avx2.UnpackLow(rgb, Vector256.Zero); - bx = Avx2.UnpackHigh(rgb, Vector256.Zero); - - r = Avx.ConvertToVector256Single(Avx2.UnpackLow(rg, Vector256.Zero).AsInt32()); - g = Avx.ConvertToVector256Single(Avx2.UnpackHigh(rg, Vector256.Zero).AsInt32()); - b = Avx.ConvertToVector256Single(Avx2.UnpackLow(bx, Vector256.Zero).AsInt32()); - - Unsafe.Add(ref destRRef, i) = r; - Unsafe.Add(ref destGRef, i) = g; - Unsafe.Add(ref destBRef, i) = b; - } - - int sliceCount = (int)(count * 8); - redChannel = redChannel[sliceCount..]; - greenChannel = greenChannel[sliceCount..]; - blueChannel = blueChannel[sliceCount..]; - source = source[sliceCount..]; - } } } diff --git a/src/ImageSharp/Common/Helpers/SimdUtils.Pack.cs b/src/ImageSharp/Common/Helpers/SimdUtils.Pack.cs index f471d0231b..71afd3a742 100644 --- a/src/ImageSharp/Common/Helpers/SimdUtils.Pack.cs +++ b/src/ImageSharp/Common/Helpers/SimdUtils.Pack.cs @@ -3,7 +3,9 @@ using System.Runtime.CompilerServices; using System.Runtime.InteropServices; +using System.Runtime.Intrinsics; using System.Runtime.Intrinsics.X86; +using SixLabors.ImageSharp.Common.Helpers; using SixLabors.ImageSharp.PixelFormats; namespace SixLabors.ImageSharp; @@ -19,11 +21,16 @@ internal static void PackFromRgbPlanes( { DebugGuard.IsTrue(greenChannel.Length == redChannel.Length, nameof(greenChannel), "Channels must be of same size!"); DebugGuard.IsTrue(blueChannel.Length == redChannel.Length, nameof(blueChannel), "Channels must be of same size!"); - DebugGuard.IsTrue(destination.Length > redChannel.Length + 2, nameof(destination), "'destination' must contain a padding of 3 elements!"); + DebugGuard.IsTrue(destination.Length >= redChannel.Length, nameof(destination), "'destination' span should not be shorter than the source channels!"); if (Avx2.IsSupported) { - HwIntrinsics.PackFromRgbPlanesAvx2Reduce(ref redChannel, ref greenChannel, ref blueChannel, ref destination); + HwIntrinsics.PackFromRgbPlanesReduce(ref redChannel, ref greenChannel, ref blueChannel, ref destination); + } + + if (Vector128.IsHardwareAccelerated) + { + PackFromRgbPlanesVector128Reduce(ref redChannel, ref greenChannel, ref blueChannel, ref destination); } else { @@ -42,11 +49,16 @@ internal static void PackFromRgbPlanes( { DebugGuard.IsTrue(greenChannel.Length == redChannel.Length, nameof(greenChannel), "Channels must be of same size!"); DebugGuard.IsTrue(blueChannel.Length == redChannel.Length, nameof(blueChannel), "Channels must be of same size!"); - DebugGuard.IsTrue(destination.Length > redChannel.Length, nameof(destination), "'destination' span should not be shorter than the source channels!"); + DebugGuard.IsTrue(destination.Length >= redChannel.Length, nameof(destination), "'destination' span should not be shorter than the source channels!"); if (Avx2.IsSupported) { - HwIntrinsics.PackFromRgbPlanesAvx2Reduce(ref redChannel, ref greenChannel, ref blueChannel, ref destination); + HwIntrinsics.PackFromRgbPlanesReduce(ref redChannel, ref greenChannel, ref blueChannel, ref destination); + } + + if (Vector128.IsHardwareAccelerated) + { + PackFromRgbPlanesVector128Reduce(ref redChannel, ref greenChannel, ref blueChannel, ref destination); } else { @@ -69,12 +81,143 @@ internal static void UnpackToRgbPlanes( if (Avx2.IsSupported) { - HwIntrinsics.UnpackToRgbPlanesAvx2Reduce(ref redChannel, ref greenChannel, ref blueChannel, ref source); + HwIntrinsics.UnpackToRgbPlanesReduce(ref redChannel, ref greenChannel, ref blueChannel, ref source); } UnpackToRgbPlanesScalar(redChannel, greenChannel, blueChannel, source); } + /// + /// Packs complete sixteen-pixel batches into exact-length storage using portable 128-bit SIMD. + /// + /// The red source span, advanced past the converted batches. + /// The green source span, advanced past the converted batches. + /// The blue source span, advanced past the converted batches. + /// The destination span, advanced past the converted batches. + private static void PackFromRgbPlanesVector128Reduce( + ref ReadOnlySpan redChannel, + ref ReadOnlySpan greenChannel, + ref ReadOnlySpan blueChannel, + ref Span destination) + { + ref byte redBase = ref MemoryMarshal.GetReference(redChannel); + ref byte greenBase = ref MemoryMarshal.GetReference(greenChannel); + ref byte blueBase = ref MemoryMarshal.GetReference(blueChannel); + ref byte destinationBase = ref Unsafe.As(ref MemoryMarshal.GetReference(destination)); + Vector128 opaqueAlpha = Vector128.Create(byte.MaxValue); + Vector128 removeAlpha = Vector128.Create((byte)0, 1, 2, 4, 5, 6, 8, 9, 10, 12, 13, 14, byte.MaxValue, byte.MaxValue, byte.MaxValue, byte.MaxValue); + nuint batchCount = (nuint)(uint)redChannel.Length / (uint)Vector128.Count; + + for (nuint i = 0; i < batchCount; i++) + { + nuint sourceOffset = i * (uint)Vector128.Count; + Vector128 red = Vector128.LoadUnsafe(ref redBase, sourceOffset); + Vector128 green = Vector128.LoadUnsafe(ref greenBase, sourceOffset); + Vector128 blue = Vector128.LoadUnsafe(ref blueBase, sourceOffset); + InterleaveRgbPlanes(red, green, blue, opaqueAlpha, out Vector128 rgba0, out Vector128 rgba1, out Vector128 rgba2, out Vector128 rgba3); + + // The native byte shuffle removes alpha from four pixels at a time. Each result owns twelve bytes, so + // exact stores avoid coupling the SIMD path to padding beyond the row or the next memory-group segment. + ref byte destination0 = ref Unsafe.Add(ref destinationBase, i * 48); + StoreRgb24Batch(Vector128.ShuffleNative(rgba0, removeAlpha), ref destination0); + StoreRgb24Batch(Vector128.ShuffleNative(rgba1, removeAlpha), ref Unsafe.Add(ref destination0, 12)); + StoreRgb24Batch(Vector128.ShuffleNative(rgba2, removeAlpha), ref Unsafe.Add(ref destination0, 24)); + StoreRgb24Batch(Vector128.ShuffleNative(rgba3, removeAlpha), ref Unsafe.Add(ref destination0, 36)); + } + + int convertedCount = (int)(batchCount * (uint)Vector128.Count); + redChannel = redChannel[convertedCount..]; + greenChannel = greenChannel[convertedCount..]; + blueChannel = blueChannel[convertedCount..]; + destination = destination[convertedCount..]; + } + + /// + /// Packs complete sixteen-pixel batches into exact-length storage using portable 128-bit SIMD. + /// + /// The red source span, advanced past the converted batches. + /// The green source span, advanced past the converted batches. + /// The blue source span, advanced past the converted batches. + /// The destination span, advanced past the converted batches. + private static void PackFromRgbPlanesVector128Reduce( + ref ReadOnlySpan redChannel, + ref ReadOnlySpan greenChannel, + ref ReadOnlySpan blueChannel, + ref Span destination) + { + ref byte redBase = ref MemoryMarshal.GetReference(redChannel); + ref byte greenBase = ref MemoryMarshal.GetReference(greenChannel); + ref byte blueBase = ref MemoryMarshal.GetReference(blueChannel); + ref Vector128 destinationBase = ref Unsafe.As>(ref MemoryMarshal.GetReference(destination)); + Vector128 opaqueAlpha = Vector128.Create(byte.MaxValue); + nuint batchCount = (nuint)(uint)redChannel.Length / (uint)Vector128.Count; + + for (nuint i = 0; i < batchCount; i++) + { + nuint sourceOffset = i * (uint)Vector128.Count; + Vector128 red = Vector128.LoadUnsafe(ref redBase, sourceOffset); + Vector128 green = Vector128.LoadUnsafe(ref greenBase, sourceOffset); + Vector128 blue = Vector128.LoadUnsafe(ref blueBase, sourceOffset); + InterleaveRgbPlanes(red, green, blue, opaqueAlpha, out Vector128 rgba0, out Vector128 rgba1, out Vector128 rgba2, out Vector128 rgba3); + + ref Vector128 destination0 = ref Unsafe.Add(ref destinationBase, i * 4); + destination0 = rgba0; + Unsafe.Add(ref destination0, 1) = rgba1; + Unsafe.Add(ref destination0, 2) = rgba2; + Unsafe.Add(ref destination0, 3) = rgba3; + } + + int convertedCount = (int)(batchCount * (uint)Vector128.Count); + redChannel = redChannel[convertedCount..]; + greenChannel = greenChannel[convertedCount..]; + blueChannel = blueChannel[convertedCount..]; + destination = destination[convertedCount..]; + } + + /// + /// Interleaves sixteen planar RGB samples into four groups of four opaque RGBA pixels. + /// + /// The red component lanes. + /// The green component lanes. + /// The blue component lanes. + /// The opaque alpha lanes. + /// The first four interleaved pixels. + /// The second four interleaved pixels. + /// The third four interleaved pixels. + /// The fourth four interleaved pixels. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static void InterleaveRgbPlanes( + Vector128 red, + Vector128 green, + Vector128 blue, + Vector128 alpha, + out Vector128 rgba0, + out Vector128 rgba1, + out Vector128 rgba2, + out Vector128 rgba3) + { + Vector128 redGreenLow = Vector128_.UnpackLow(red, green); + Vector128 redGreenHigh = Vector128_.UnpackHigh(red, green); + Vector128 blueAlphaLow = Vector128_.UnpackLow(blue, alpha); + Vector128 blueAlphaHigh = Vector128_.UnpackHigh(blue, alpha); + rgba0 = Vector128_.UnpackLow(redGreenLow.AsInt16(), blueAlphaLow.AsInt16()).AsByte(); + rgba1 = Vector128_.UnpackHigh(redGreenLow.AsInt16(), blueAlphaLow.AsInt16()).AsByte(); + rgba2 = Vector128_.UnpackLow(redGreenHigh.AsInt16(), blueAlphaHigh.AsInt16()).AsByte(); + rgba3 = Vector128_.UnpackHigh(redGreenHigh.AsInt16(), blueAlphaHigh.AsInt16()).AsByte(); + } + + /// + /// Stores the twelve packed RGB bytes in one shuffled SIMD value without writing its unused lanes. + /// + /// The packed RGB bytes in the first twelve lanes. + /// The first destination byte. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static void StoreRgb24Batch(Vector128 value, ref byte destination) + { + Unsafe.WriteUnaligned(ref destination, value.AsUInt64().ToScalar()); + Unsafe.WriteUnaligned(ref Unsafe.Add(ref destination, 8), value.AsUInt32().GetElement(2)); + } + private static void PackFromRgbPlanesScalarBatchedReduce( ref ReadOnlySpan redChannel, ref ReadOnlySpan greenChannel, @@ -86,8 +229,8 @@ private static void PackFromRgbPlanesScalarBatchedReduce( ref ByteTuple4 b = ref Unsafe.As(ref MemoryMarshal.GetReference(blueChannel)); ref Rgb24 rgb = ref MemoryMarshal.GetReference(destination); - nuint count = (uint)redChannel.Length / 4; - for (nuint i = 0; i < count; i++) + nuint batchCount = (uint)redChannel.Length / 4; + for (nuint i = 0; i < batchCount; i++) { ref Rgb24 d0 = ref Unsafe.Add(ref rgb, i * 4); ref Rgb24 d1 = ref Unsafe.Add(ref d0, 1); @@ -115,11 +258,11 @@ private static void PackFromRgbPlanesScalarBatchedReduce( d3.B = bb.V3; } - int finished = (int)(count * 4); - redChannel = redChannel[finished..]; - greenChannel = greenChannel[finished..]; - blueChannel = blueChannel[finished..]; - destination = destination[finished..]; + int convertedCount = (int)(batchCount * 4); + redChannel = redChannel[convertedCount..]; + greenChannel = greenChannel[convertedCount..]; + blueChannel = blueChannel[convertedCount..]; + destination = destination[convertedCount..]; } private static void PackFromRgbPlanesScalarBatchedReduce( @@ -133,9 +276,9 @@ private static void PackFromRgbPlanesScalarBatchedReduce( ref ByteTuple4 b = ref Unsafe.As(ref MemoryMarshal.GetReference(blueChannel)); ref Rgba32 rgb = ref MemoryMarshal.GetReference(destination); - nuint count = (uint)redChannel.Length / 4; + nuint batchCount = (uint)redChannel.Length / 4; destination.Fill(new Rgba32(0, 0, 0, 255)); - for (nuint i = 0; i < count; i++) + for (nuint i = 0; i < batchCount; i++) { ref Rgba32 d0 = ref Unsafe.Add(ref rgb, i * 4); ref Rgba32 d1 = ref Unsafe.Add(ref d0, 1); @@ -163,11 +306,11 @@ private static void PackFromRgbPlanesScalarBatchedReduce( d3.B = bb.V3; } - int finished = (int)(count * 4); - redChannel = redChannel[finished..]; - greenChannel = greenChannel[finished..]; - blueChannel = blueChannel[finished..]; - destination = destination[finished..]; + int convertedCount = (int)(batchCount * 4); + redChannel = redChannel[convertedCount..]; + greenChannel = greenChannel[convertedCount..]; + blueChannel = blueChannel[convertedCount..]; + destination = destination[convertedCount..]; } private static void PackFromRgbPlanesRemainder( @@ -181,7 +324,7 @@ private static void PackFromRgbPlanesRemainder( ref byte b = ref MemoryMarshal.GetReference(blueChannel); ref Rgb24 rgb = ref MemoryMarshal.GetReference(destination); - for (nuint i = 0; i < (uint)destination.Length; i++) + for (nuint i = 0; i < (uint)redChannel.Length; i++) { ref Rgb24 d = ref Unsafe.Add(ref rgb, i); d.R = Unsafe.Add(ref r, i); @@ -201,7 +344,7 @@ private static void PackFromRgbPlanesRemainder( ref byte b = ref MemoryMarshal.GetReference(blueChannel); ref Rgba32 rgba = ref MemoryMarshal.GetReference(destination); - for (nuint i = 0; i < (uint)destination.Length; i++) + for (nuint i = 0; i < (uint)redChannel.Length; i++) { ref Rgba32 d = ref Unsafe.Add(ref rgba, i); d.R = Unsafe.Add(ref r, i); @@ -234,4 +377,291 @@ private static void UnpackToRgbPlanesScalar( Unsafe.Add(ref b, i) = src.B; } } + + /// + /// Provides the hardware-intrinsic reducers used by the planar RGB packing pipeline. + /// + public static partial class HwIntrinsics + { + /// + /// Creates the AVX2 lane order used before interleaving planar RGB components. + /// + /// The source lane permutation. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector256 PermuteMaskEvenOdd8x32() => Vector256.Create(0u, 2, 4, 6, 1, 3, 5, 7); + + /// + /// Packs complete AVX2 batches into pixels and retains the unconverted remainder. + /// + /// The red source span. + /// The green source span. + /// The blue source span. + /// The destination pixel span. + internal static void PackFromRgbPlanesReduce( + ref ReadOnlySpan redChannel, + ref ReadOnlySpan greenChannel, + ref ReadOnlySpan blueChannel, + ref Span destination) + { + ref Vector256 redBase = ref Unsafe.As>(ref MemoryMarshal.GetReference(redChannel)); + ref Vector256 greenBase = ref Unsafe.As>(ref MemoryMarshal.GetReference(greenChannel)); + ref Vector256 blueBase = ref Unsafe.As>(ref MemoryMarshal.GetReference(blueChannel)); + ref byte destinationBase = ref Unsafe.As(ref MemoryMarshal.GetReference(destination)); + nuint batchCount = redChannel.Vector256Count(); + Vector256 sourceOrder = PermuteMaskEvenOdd8x32(); + Vector256 packedOrder = Vector256.Create(0u, 1, 2, 4, 5, 6, 3, 7); + Vector256 opaqueAlpha = Vector256.Create(byte.MaxValue); + Vector128 removeAlphaLower = Vector128.Create((byte)0, 1, 2, 4, 5, 6, 8, 9, 10, 12, 13, 14, 3, 7, 11, 15); + Vector128 removeAlphaUpper = Vector128.Create((byte)16, 17, 18, 20, 21, 22, 24, 25, 26, 28, 29, 30, 19, 23, 27, 31); + Vector256 removeAlpha = Vector256.Create(removeAlphaLower, removeAlphaUpper); + + bool hasWritablePadding = destination.Length >= redChannel.Length + 3; + nuint i = 0; + + // Non-final batches retain the original four overlapping wide stores. Splitting the final batch keeps + // the exact-row decision out of the hot loop and limits the narrower stores to the only bytes that can + // cross the destination boundary. + for (; i + 1 < batchCount; i++) + { + PackRgb24Batch( + Unsafe.Add(ref redBase, i), + Unsafe.Add(ref greenBase, i), + Unsafe.Add(ref blueBase, i), + opaqueAlpha, + sourceOrder, + packedOrder, + removeAlpha, + out Vector256 rgb0, + out Vector256 rgb1, + out Vector256 rgb2, + out Vector256 rgb3); + + ref byte destination0 = ref Unsafe.Add(ref destinationBase, 96 * i); + ref byte destination1 = ref Unsafe.Add(ref destination0, 24); + ref byte destination2 = ref Unsafe.Add(ref destination1, 24); + ref byte destination3 = ref Unsafe.Add(ref destination2, 24); + + Unsafe.As>(ref destination0) = rgb0; + Unsafe.As>(ref destination1) = rgb1; + Unsafe.As>(ref destination2) = rgb2; + Unsafe.As>(ref destination3) = rgb3; + } + + if (i < batchCount) + { + PackRgb24Batch( + Unsafe.Add(ref redBase, i), + Unsafe.Add(ref greenBase, i), + Unsafe.Add(ref blueBase, i), + opaqueAlpha, + sourceOrder, + packedOrder, + removeAlpha, + out Vector256 rgb0, + out Vector256 rgb1, + out Vector256 rgb2, + out Vector256 rgb3); + + ref byte destination0 = ref Unsafe.Add(ref destinationBase, 96 * i); + ref byte destination1 = ref Unsafe.Add(ref destination0, 24); + ref byte destination2 = ref Unsafe.Add(ref destination1, 24); + ref byte destination3 = ref Unsafe.Add(ref destination2, 24); + + Unsafe.As>(ref destination0) = rgb0; + Unsafe.As>(ref destination1) = rgb1; + Unsafe.As>(ref destination2) = rgb2; + + if (hasWritablePadding) + { + Unsafe.As>(ref destination3) = rgb3; + } + else + { + // The final compacted vector contains 24 RGB bytes followed by eight unused bytes. Exact stores + // retain all useful bytes without writing beyond an unpadded destination row. + Unsafe.As>(ref destination3) = rgb3.GetLower(); + Unsafe.As(ref Unsafe.Add(ref destination3, 16)) = rgb3.GetUpper().AsUInt64().ToScalar(); + } + } + + int convertedCount = (int)batchCount * Vector256.Count; + redChannel = redChannel[convertedCount..]; + greenChannel = greenChannel[convertedCount..]; + blueChannel = blueChannel[convertedCount..]; + destination = destination[convertedCount..]; + } + + /// + /// Packs complete AVX2 batches into pixels and retains the unconverted remainder. + /// + /// The red source span. + /// The green source span. + /// The blue source span. + /// The destination pixel span. + internal static void PackFromRgbPlanesReduce( + ref ReadOnlySpan redChannel, + ref ReadOnlySpan greenChannel, + ref ReadOnlySpan blueChannel, + ref Span destination) + { + ref Vector256 redBase = ref Unsafe.As>(ref MemoryMarshal.GetReference(redChannel)); + ref Vector256 greenBase = ref Unsafe.As>(ref MemoryMarshal.GetReference(greenChannel)); + ref Vector256 blueBase = ref Unsafe.As>(ref MemoryMarshal.GetReference(blueChannel)); + ref Vector256 destinationBase = ref Unsafe.As>(ref MemoryMarshal.GetReference(destination)); + nuint batchCount = redChannel.Vector256Count(); + Vector256 sourceOrder = PermuteMaskEvenOdd8x32(); + Vector256 opaqueAlpha = Vector256.Create(byte.MaxValue); + + for (nuint i = 0; i < batchCount; i++) + { + InterleaveRgbPlanes( + Unsafe.Add(ref redBase, i), + Unsafe.Add(ref greenBase, i), + Unsafe.Add(ref blueBase, i), + opaqueAlpha, + sourceOrder, + out Vector256 rgba0, + out Vector256 rgba1, + out Vector256 rgba2, + out Vector256 rgba3); + + ref Vector256 destination0 = ref Unsafe.Add(ref destinationBase, i * 4); + destination0 = rgba0; + Unsafe.Add(ref destination0, 1) = rgba1; + Unsafe.Add(ref destination0, 2) = rgba2; + Unsafe.Add(ref destination0, 3) = rgba3; + } + + int convertedCount = (int)batchCount * Vector256.Count; + redChannel = redChannel[convertedCount..]; + greenChannel = greenChannel[convertedCount..]; + blueChannel = blueChannel[convertedCount..]; + destination = destination[convertedCount..]; + } + + /// + /// Unpacks complete AVX2 batches from pixels and retains the unconverted remainder. + /// + /// The red destination span. + /// The green destination span. + /// The blue destination span. + /// The source pixel span. + internal static void UnpackToRgbPlanesReduce( + ref Span redChannel, + ref Span greenChannel, + ref Span blueChannel, + ref ReadOnlySpan source) + { + ref Vector256 sourceBase = ref Unsafe.As>(ref MemoryMarshal.GetReference(source)); + ref Vector256 redBase = ref Unsafe.As>(ref MemoryMarshal.GetReference(redChannel)); + ref Vector256 greenBase = ref Unsafe.As>(ref MemoryMarshal.GetReference(greenChannel)); + ref Vector256 blueBase = ref Unsafe.As>(ref MemoryMarshal.GetReference(blueChannel)); + Vector256 separateLanes = Vector256.Create(0u, 1, 2, 6, 3, 4, 5, 7); + Vector128 extractRgbLower = Vector128.Create((byte)0, 3, 6, 9, 1, 4, 7, 10, 2, 5, 8, 11, byte.MaxValue, byte.MaxValue, byte.MaxValue, byte.MaxValue); + Vector128 extractRgbUpper = Vector128.Create((byte)16, 19, 22, 25, 17, 20, 23, 26, 18, 21, 24, 27, byte.MaxValue, byte.MaxValue, byte.MaxValue, byte.MaxValue); + Vector256 extractRgb = Vector256.Create(extractRgbLower, extractRgbUpper); + + // Each iteration consumes eight Rgb24 pixels, or 24 bytes, but starts with a 32-byte load. Three extra + // source pixels must therefore remain addressable beyond every vectorized batch. + const int bytesPerBatch = 24; + nuint batchCount = source.Length > 3 ? (uint)(source.Length - 3) / 8 : 0; + + for (nuint i = 0; i < batchCount; i++) + { + Vector256 packed = Avx2.PermuteVar8x32(Unsafe.AddByteOffset(ref sourceBase, (uint)(bytesPerBatch * i)).AsUInt32(), separateLanes).AsByte(); + packed = Vector256.ShuffleNative(packed, extractRgb); + + Vector256 redGreen = Avx2.UnpackLow(packed, Vector256.Zero); + Vector256 blue = Avx2.UnpackHigh(packed, Vector256.Zero); + Vector256 red = Avx.ConvertToVector256Single(Avx2.UnpackLow(redGreen, Vector256.Zero).AsInt32()); + Vector256 green = Avx.ConvertToVector256Single(Avx2.UnpackHigh(redGreen, Vector256.Zero).AsInt32()); + Vector256 blueValues = Avx.ConvertToVector256Single(Avx2.UnpackLow(blue, Vector256.Zero).AsInt32()); + + Unsafe.Add(ref redBase, i) = red; + Unsafe.Add(ref greenBase, i) = green; + Unsafe.Add(ref blueBase, i) = blueValues; + } + + int convertedCount = (int)(batchCount * 8); + redChannel = redChannel[convertedCount..]; + greenChannel = greenChannel[convertedCount..]; + blueChannel = blueChannel[convertedCount..]; + source = source[convertedCount..]; + } + + /// + /// Interleaves and compacts one AVX2 batch into four groups of eight pixels. + /// + /// The red component lanes. + /// The green component lanes. + /// The blue component lanes. + /// The opaque alpha lanes used during interleaving. + /// The cross-lane source permutation. + /// The cross-lane packed RGB permutation. + /// The native byte-shuffle indices that compact RGBA to RGB. + /// The first eight packed pixels. + /// The second eight packed pixels. + /// The third eight packed pixels. + /// The fourth eight packed pixels. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static void PackRgb24Batch( + Vector256 red, + Vector256 green, + Vector256 blue, + Vector256 alpha, + Vector256 sourceOrder, + Vector256 packedOrder, + Vector256 removeAlpha, + out Vector256 rgb0, + out Vector256 rgb1, + out Vector256 rgb2, + out Vector256 rgb3) + { + InterleaveRgbPlanes(red, green, blue, alpha, sourceOrder, out Vector256 rgba0, out Vector256 rgba1, out Vector256 rgba2, out Vector256 rgba3); + + rgb0 = Avx2.PermuteVar8x32(Vector256.ShuffleNative(rgba0, removeAlpha).AsUInt32(), packedOrder).AsByte(); + rgb1 = Avx2.PermuteVar8x32(Vector256.ShuffleNative(rgba1, removeAlpha).AsUInt32(), packedOrder).AsByte(); + rgb2 = Avx2.PermuteVar8x32(Vector256.ShuffleNative(rgba2, removeAlpha).AsUInt32(), packedOrder).AsByte(); + rgb3 = Avx2.PermuteVar8x32(Vector256.ShuffleNative(rgba3, removeAlpha).AsUInt32(), packedOrder).AsByte(); + } + + /// + /// Interleaves 32 planar RGB samples into four groups of eight opaque RGBA pixels. + /// + /// The red component lanes. + /// The green component lanes. + /// The blue component lanes. + /// The opaque alpha lanes. + /// The cross-lane source permutation. + /// The first eight interleaved pixels. + /// The second eight interleaved pixels. + /// The third eight interleaved pixels. + /// The fourth eight interleaved pixels. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static void InterleaveRgbPlanes( + Vector256 red, + Vector256 green, + Vector256 blue, + Vector256 alpha, + Vector256 sourceOrder, + out Vector256 rgba0, + out Vector256 rgba1, + out Vector256 rgba2, + out Vector256 rgba3) + { + red = Avx2.PermuteVar8x32(red.AsUInt32(), sourceOrder).AsByte(); + green = Avx2.PermuteVar8x32(green.AsUInt32(), sourceOrder).AsByte(); + blue = Avx2.PermuteVar8x32(blue.AsUInt32(), sourceOrder).AsByte(); + + Vector256 redGreenLow = Avx2.UnpackLow(red, green); + Vector256 redGreenHigh = Avx2.UnpackHigh(red, green); + Vector256 blueAlphaLow = Avx2.UnpackLow(blue, alpha); + Vector256 blueAlphaHigh = Avx2.UnpackHigh(blue, alpha); + + rgba0 = Avx2.UnpackLow(redGreenLow.AsUInt16(), blueAlphaLow.AsUInt16()).AsByte(); + rgba1 = Avx2.UnpackHigh(redGreenLow.AsUInt16(), blueAlphaLow.AsUInt16()).AsByte(); + rgba2 = Avx2.UnpackLow(redGreenHigh.AsUInt16(), blueAlphaHigh.AsUInt16()).AsByte(); + rgba3 = Avx2.UnpackHigh(redGreenHigh.AsUInt16(), blueAlphaHigh.AsUInt16()).AsByte(); + } + } } diff --git a/src/ImageSharp/Common/InlineArray.cs b/src/ImageSharp/Common/InlineArray.cs index 700551a8f3..12c12bd365 100644 --- a/src/ImageSharp/Common/InlineArray.cs +++ b/src/ImageSharp/Common/InlineArray.cs @@ -8,6 +8,24 @@ namespace SixLabors.ImageSharp; +/// +/// Represents a safe, fixed sized buffer of 2 elements. +/// +[InlineArray(2)] +internal struct InlineArray2 +{ + private T t; +} + +/// +/// Represents a safe, fixed sized buffer of 3 elements. +/// +[InlineArray(3)] +internal struct InlineArray3 +{ + private T t; +} + /// /// Represents a safe, fixed sized buffer of 4 elements. /// @@ -17,6 +35,24 @@ internal struct InlineArray4 private T t; } +/// +/// Represents a safe, fixed sized buffer of 6 elements. +/// +[InlineArray(6)] +internal struct InlineArray6 +{ + private T t; +} + +/// +/// Represents a safe, fixed sized buffer of 7 elements. +/// +[InlineArray(7)] +internal struct InlineArray7 +{ + private T t; +} + /// /// Represents a safe, fixed sized buffer of 8 elements. /// @@ -26,6 +62,24 @@ internal struct InlineArray8 private T t; } +/// +/// Represents a safe, fixed sized buffer of 10 elements. +/// +[InlineArray(10)] +internal struct InlineArray10 +{ + private T t; +} + +/// +/// Represents a safe, fixed sized buffer of 12 elements. +/// +[InlineArray(12)] +internal struct InlineArray12 +{ + private T t; +} + /// /// Represents a safe, fixed sized buffer of 14 elements. /// @@ -62,6 +116,24 @@ internal struct InlineArray19 private T t; } +/// +/// Represents a safe, fixed sized buffer of 24 elements. +/// +[InlineArray(24)] +internal struct InlineArray24 +{ + private T t; +} + +/// +/// Represents a safe, fixed sized buffer of 25 elements. +/// +[InlineArray(25)] +internal struct InlineArray25 +{ + private T t; +} + /// /// Represents a safe, fixed sized buffer of 26 elements. /// @@ -80,6 +152,15 @@ internal struct InlineArray36 private T t; } +/// +/// Represents a safe, fixed sized buffer of 65 elements. +/// +[InlineArray(65)] +internal struct InlineArray65 +{ + private T t; +} + /// /// Represents a safe, fixed sized buffer of 256 elements. /// diff --git a/src/ImageSharp/Common/InlineArray.tt b/src/ImageSharp/Common/InlineArray.tt index d689b0469a..3869e42bb6 100644 --- a/src/ImageSharp/Common/InlineArray.tt +++ b/src/ImageSharp/Common/InlineArray.tt @@ -16,7 +16,7 @@ namespace SixLabors.ImageSharp; <#GenerateInlineArrays();#> <#+ -private static int[] Lengths = [4, 8, 14, 16, 18, 19, 26, 36, 256]; +private static int[] Lengths = [2, 3, 4, 6, 7, 8, 10, 12, 14, 16, 18, 19, 24, 25, 26, 36, 65, 256]; void GenerateInlineArrays() { diff --git a/src/ImageSharp/Configuration.cs b/src/ImageSharp/Configuration.cs index 94073dc099..47a54e7480 100644 --- a/src/ImageSharp/Configuration.cs +++ b/src/ImageSharp/Configuration.cs @@ -10,6 +10,7 @@ using SixLabors.ImageSharp.Formats.Cur; using SixLabors.ImageSharp.Formats.Exr; using SixLabors.ImageSharp.Formats.Gif; +using SixLabors.ImageSharp.Formats.Heif; using SixLabors.ImageSharp.Formats.Ico; using SixLabors.ImageSharp.Formats.Jpeg; using SixLabors.ImageSharp.Formats.Pbm; @@ -225,6 +226,7 @@ public void Configure(IImageFormatConfigurationModule configuration) /// . /// . /// . + /// . /// . /// /// The default configuration of . @@ -239,6 +241,7 @@ public void Configure(IImageFormatConfigurationModule configuration) new WebpConfigurationModule(), new ExrConfigurationModule(), new QoiConfigurationModule(), + new HeifConfigurationModule(), new IcoConfigurationModule(), new CurConfigurationModule(), new AniConfigurationModule()); diff --git a/src/ImageSharp/Formats/DecoderOptions.cs b/src/ImageSharp/Formats/DecoderOptions.cs index 916888af33..f1925913ce 100644 --- a/src/ImageSharp/Formats/DecoderOptions.cs +++ b/src/ImageSharp/Formats/DecoderOptions.cs @@ -78,7 +78,7 @@ internal bool TryGetIccProfileForColorConversion(IccProfile? profile, [NotNullWh return false; } - if (this.ColorProfileHandling == ColorProfileHandling.Preserve) + if (this.ColorProfileHandling != ColorProfileHandling.Convert) { return false; } diff --git a/src/ImageSharp/Formats/Heif/Av1/Av1BitDepth.cs b/src/ImageSharp/Formats/Heif/Av1/Av1BitDepth.cs new file mode 100644 index 0000000000..54ec00ca13 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Av1BitDepth.cs @@ -0,0 +1,25 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1; + +/// +/// Identifies the sample precision of an AV1 sequence. +/// +internal enum Av1BitDepth : int +{ + /// + /// Eight bits per sample. + /// + EightBit = 0, + + /// + /// Ten bits per sample. + /// + TenBit = 1, + + /// + /// Twelve bits per sample. + /// + TwelveBit = 2, +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Av1BitDepthExtensions.cs b/src/ImageSharp/Formats/Heif/Av1/Av1BitDepthExtensions.cs new file mode 100644 index 0000000000..06aef289a4 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Av1BitDepthExtensions.cs @@ -0,0 +1,19 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1; + +/// +/// Provides sample-precision conversions for AV1 bit-depth values. +/// +internal static class Av1BitDepthExtensions +{ + /// + /// Gets the number of bits represented by an AV1 bit-depth value. + /// + /// The AV1 bit-depth value. + /// Eight, ten, or twelve. + public static int GetBitCount(this Av1BitDepth bitDepth) => 8 + ((int)bitDepth << 1); +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Av1BitStreamReader.cs b/src/ImageSharp/Formats/Heif/Av1/Av1BitStreamReader.cs new file mode 100644 index 0000000000..89aaa68b63 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Av1BitStreamReader.cs @@ -0,0 +1,314 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1; + +/// +/// Reads AV1 fixed-width and variable-length syntax from a most-significant-bit-first byte span. +/// +internal ref struct Av1BitStreamReader +{ + /// + /// The complete encoded byte span. + /// + private readonly Span data; + + /// + /// Initializes a new instance of the struct. + /// + /// The encoded AV1 data. + public Av1BitStreamReader(Span data) => this.data = data; + + /// + /// Gets the zero-based position of the next bit to read. + /// + public int BitPosition { get; private set; } = 0; + + /// + /// Gets the number of bytes in the reader's buffer. + /// + public readonly int Length => this.data.Length; + + /// + /// Moves the next read position to the beginning of the buffer. + /// + public void Reset() => this.BitPosition = 0; + + /// + /// Advances the read position without interpreting the skipped bits. + /// + /// The number of bits to skip. + public void Skip(int bitCount) => this.BitPosition += bitCount; + + /// + /// Reads an unsigned fixed-width value in most-significant-bit-first order. + /// + /// The number of bits to read. + /// The decoded unsigned value. + public uint ReadLiteral(int bitCount) + { + DebugGuard.MustBeBetweenOrEqualTo(bitCount, 0, 32, nameof(bitCount)); + + uint literal = 0; + for (int bit = bitCount - 1; bit >= 0; bit--) + { + literal |= this.ReadBit() << bit; + } + + return literal; + } + + /// + /// Reads the next encoded bit. + /// + /// Zero or one. + public uint ReadBit() + { + int byteOffset = Av1Math.DivideBy8Floor(this.BitPosition); + byte shift = (byte)(7 - Av1Math.Modulus8(this.BitPosition)); + this.BitPosition++; + return (uint)((this.data[byteOffset] >> shift) & 0x01); + } + + /// + /// Reads the next encoded bit as a Boolean value. + /// + /// for one; otherwise, . + public bool ReadBoolean() => this.ReadLiteral(1) > 0; + + /// + /// Reads an AV1 little-endian base-128 value from a byte-aligned position. + /// + /// Receives the number of encoded bytes consumed. + /// The decoded unsigned value. + public ulong ReadLittleEndianBytes128(out int length) + { + DebugGuard.IsTrue((this.BitPosition & 0x07) == 0, $"Reading of Little Endian 128 value only allowed on byte alignment (offset {this.BitPosition})."); + + ulong value = 0; + length = 0; + for (int shift = 0; shift < 56; shift += 7) + { + uint leb128Byte = this.ReadLiteral(8); + value |= (leb128Byte & 0x7FUL) << shift; + length++; + if ((leb128Byte & 0x80U) == 0) + { + return value; + } + } + + // AV1 limits unsigned LEB128 fields to eight bytes. A continuation bit in the eighth byte does not describe + // another value byte; accepting it would move the following OBU header into the declared size field. + throw new InvalidImageContentException("The AV1 LEB128 value is not terminated within eight bytes."); + } + + /// + /// Reads the AV1 unsigned-variable-length code. + /// + /// The decoded unsigned value. + public uint ReadUnsignedVariableLength() + { + int leadingZerosCount = 0; + while (leadingZerosCount < 32) + { + uint bit = this.ReadLiteral(1); + if (bit == 1) + { + break; + } + + leadingZerosCount++; + } + + if (leadingZerosCount == 32) + { + return uint.MaxValue; + } + + if (leadingZerosCount != 0) + { + uint basis = (1U << leadingZerosCount) - 1U; + uint value = this.ReadLiteral(leadingZerosCount); + return basis + value; + } + + return 0; + } + + /// + /// Reads a value from an alphabet whose size is not a power of two. + /// + /// The number of symbols in the alphabet. + /// A decoded symbol in the range zero through minus one. + public uint ReadNonSymmetric(uint n) + { + if (n <= 1) + { + return 0; + } + + int w = (int)(Av1Math.FloorLog2(n) + 1); + uint m = (uint)((1 << w) - n); + uint v = this.ReadLiteral(w - 1); + if (v < m) + { + return v; + } + + return (v << 1) - m + this.ReadLiteral(1); + } + + /// + /// Reads a finite subexponential value recentered around a signed reference value. + /// + /// One greater than the maximum absolute value in the signed domain. + /// The bit width of the first subexponential group. + /// The signed reference value around which smaller codewords are concentrated. + /// A decoded value in the inclusive range from minus plus one through + /// minus one. + public int ReadSignedReferenceSubexponential(int valueMagnitude, int groupBitCount, int reference) + { + int shiftedReference = reference + valueMagnitude - 1; + int scaledValueCount = (valueMagnitude << 1) - 1; + return this.ReadReferenceSubexponential(scaledValueCount, groupBitCount, shiftedReference) - valueMagnitude + 1; + } + + /// + /// Reads a fixed-width two's-complement signed integer. + /// + /// The encoded bit width. + /// The sign-extended integer. + public int ReadSignedFromUnsigned(int n) + { + int signedValue; + uint value = this.ReadLiteral(n); + uint signMask = 1U << (n - 1); + if ((value & signMask) == signMask) + { + // The subtraction represents sign extension; widening first preserves the n=32 case. + signedValue = (int)((long)value - (signMask << 1)); + } + else + { + signedValue = (int)value; + } + + return signedValue; + } + + /// + /// Reads a byte-aligned unsigned integer whose least-significant byte is encoded first. + /// + /// The number of bytes to read. + /// The decoded unsigned integer. + public uint ReadLittleEndian(int n) + { + DebugGuard.IsTrue(Av1Math.Modulus8(this.BitPosition) == 0, "Reading of Little Endian value only allowed on byte alignment"); + + uint t = 0; + for (int i = 0; i < 8 * n; i += 8) + { + t += this.ReadLiteral(8) << i; + } + + return t; + } + + /// + /// Gets a byte-aligned tile payload for entropy decoding and advances past it. + /// + /// The tile payload length in bytes. + /// The tile payload span. + public Span GetSymbolReader(int tileDataSize) + => this.ReadBytes(tileDataSize); + + /// + /// Gets the next byte-aligned portion of the encoded data and advances past it. + /// + /// The number of bytes to read. + /// The requested bytes. + public Span ReadBytes(int byteCount) + { + DebugGuard.IsTrue(Av1Math.Modulus8(this.BitPosition) == 0, "Byte spans must start on a byte boundary."); + int byteOffset = Av1Math.DivideBy8Floor(this.BitPosition); + if ((uint)byteOffset > (uint)this.data.Length || (uint)byteCount > (uint)(this.data.Length - byteOffset)) + { + throw new InvalidImageContentException("The AV1 payload exceeds its declared data boundary."); + } + + Span payload = this.data.Slice(byteOffset, byteCount); + this.Skip(byteCount << 3); + return payload; + } + + /// + /// Reads a finite subexponential value and inverse-recenters it around an unsigned reference value. + /// + /// The number of values in the finite domain. + /// The bit width of the first subexponential group. + /// The reference value within the finite domain. + /// The decoded value in the range zero through minus one. + private int ReadReferenceSubexponential(int valueCount, int groupBitCount, int reference) + { + int value = this.ReadSubexponential(valueCount, groupBitCount); + + // Recentering enumerates values by increasing distance from the reference. References in the upper half use + // the mirrored domain so the shorter side of the finite range always participates in the alternating mapping. + if ((reference << 1) <= valueCount) + { + return InverseRecenter(reference, value); + } + + return valueCount - 1 - InverseRecenter(valueCount - 1 - reference, value); + } + + /// + /// Reads one value from a finite subexponential code. + /// + /// The number of values in the finite domain. + /// The bit width of the first subexponential group. + /// The decoded zero-based value. + private int ReadSubexponential(int valueCount, int groupBitCount) + { + int groupIndex = 0; + int groupStart = 0; + while (true) + { + // AV1 keeps the first two groups at width k and then doubles each following group. Once fewer than three + // groups remain, the non-symmetric code consumes the exact finite tail without introducing unused values. + int bitCount = groupIndex == 0 ? groupBitCount : groupBitCount + groupIndex - 1; + int groupSize = 1 << bitCount; + if (valueCount <= groupStart + (3 * groupSize)) + { + return (int)this.ReadNonSymmetric((uint)(valueCount - groupStart)) + groupStart; + } + + if (!this.ReadBoolean()) + { + return (int)this.ReadLiteral(bitCount) + groupStart; + } + + groupIndex++; + groupStart += groupSize; + } + } + + /// + /// Maps a nonnegative code value around a nonnegative reference value. + /// + /// The recentering reference. + /// The coded nonnegative value. + /// The inverse-recentered value. + private static int InverseRecenter(int reference, int value) + { + // Codes within twice the reference alternate above and below it: even values select the upper side and odd + // values select the lower side. Larger codes lie beyond the lower-side range and map directly to the tail. + if (value > (reference << 1)) + { + return value; + } + + return (value & 1) == 0 ? (value >> 1) + reference : reference - ((value + 1) >> 1); + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Av1BitStreamWriter.cs b/src/ImageSharp/Formats/Heif/Av1/Av1BitStreamWriter.cs new file mode 100644 index 0000000000..782986cfeb --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Av1BitStreamWriter.cs @@ -0,0 +1,309 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1; + +/// +/// Writes AV1 fixed-width and variable-length syntax to a caller-provided buffer. +/// +internal ref struct Av1BitStreamWriter +{ + /// + /// The number of bits in one output byte. + /// + private const int WordSize = 8; + + /// + /// The writable output buffer. + /// + private readonly Span span; + + /// + /// The partially assembled output byte. + /// + private byte buffer = 0; + + /// + /// Initializes a new instance of the struct. + /// + /// The preallocated output buffer. + public Av1BitStreamWriter(Span span) + { + this.span = span; + } + + /// + /// Gets the zero-based position of the next output bit. + /// + public int BitPosition { get; private set; } = 0; + + /// + /// Gets the current output capacity in bytes. + /// + public readonly int Capacity => this.span.Length; + + /// + /// Encodes an unsigned 32-bit value using little-endian base-128 bytes. + /// + /// The value to encode. + /// The destination receiving up to five bytes. + /// The number of bytes written. + public static int GetLittleEndianBytes128(uint value, Span span) + { + int length = 0; + do + { + byte encodedByte = (byte)(value & 0x7fU); + value >>= 7; + if (value != 0) + { + encodedByte |= 0x80; + } + + span[length++] = encodedByte; + } + while (value != 0); + + return length; + } + + /// + /// Advances the output position, emitting the current byte whenever the skip crosses a byte boundary. + /// + /// The number of bits to skip. + public void Skip(int bitCount) + { + this.BitPosition += bitCount; + while (this.BitPosition >= WordSize) + { + this.BitPosition -= WordSize; + this.WriteBuffer(); + } + } + + /// + /// Writes a partially assembled byte and resets the position for output-memory reuse. + /// + public void Flush() + { + if (Av1Math.Modulus8(this.BitPosition) != 0) + { + // Flush a partial byte also. + this.WriteBuffer(); + } + + this.BitPosition = 0; + } + + /// + /// Writes an unsigned fixed-width value in most-significant-bit-first order. + /// + /// The value to write. + /// The number of low-order bits to write. + public void WriteLiteral(uint value, int bitCount) + { + for (int bit = bitCount - 1; bit >= 0; bit--) + { + this.WriteBit((byte)((value >> bit) & 0x1)); + } + } + + /// + /// Writes one Boolean bit. + /// + /// The Boolean value. + public void WriteBoolean(bool value) + { + byte boolByte = value ? (byte)1 : (byte)0; + this.WriteBit(boolByte); + } + + /// + /// Writes a fixed-width signed integer in two's-complement form. + /// + /// The signed value. + /// The encoded bit width. + public void WriteSignedFromUnsigned(int signedValue, int n) + { + ulong value = (ulong)signedValue; + if (signedValue < 0) + { + value += 1UL << n; + } + + this.WriteLiteral((uint)value, n); + } + + /// + /// Writes an unsigned 32-bit value using little-endian base-128 bytes. + /// + /// The value to write. + public void WriteLittleEndianBytes128(uint value) + { + int wordPosition = this.BitPosition >> 3; + int bytesWritten = GetLittleEndianBytes128(value, this.span[wordPosition..]); + this.BitPosition += bytesWritten << 3; + } + + /// + /// Writes a value from an alphabet whose size is not a power of two. + /// + /// The symbol value. + /// The number of symbols in the alphabet. + public void WriteNonSymmetric(uint value, uint numberOfSymbols) + { + if (numberOfSymbols <= 1) + { + return; + } + + int w = (int)(Av1Math.FloorLog2(numberOfSymbols) + 1); + uint m = (uint)((1 << w) - numberOfSymbols); + if (value < m) + { + this.WriteLiteral(value, w - 1); + } + else + { + // libaom partitions the upper values into a shorter prefix followed by the low bit of the offset from m. + uint offset = value - m; + uint k = m + (offset >> 1); + this.WriteLiteral(k, w - 1); + this.WriteLiteral(offset & 1, 1); + } + } + + /// + /// Writes a finite subexponential value recentered around a signed reference value. + /// + /// The signed value to write. + /// One greater than the maximum absolute value in the signed domain. + /// The bit width of the first subexponential group. + /// The signed reference value around which smaller codewords are concentrated. + public void WriteSignedReferenceSubexponential(int value, int valueMagnitude, int groupBitCount, int reference) + { + int shiftedReference = reference + valueMagnitude - 1; + int shiftedValue = value + valueMagnitude - 1; + int scaledValueCount = (valueMagnitude << 1) - 1; + int recenteredValue = RecenterFiniteNonNegative(scaledValueCount, shiftedReference, shiftedValue); + + this.WriteSubexponential(recenteredValue, scaledValueCount, groupBitCount); + } + + /// + /// Writes one value using a finite sequence of exponentially growing code groups. + /// + private void WriteSubexponential(int value, int valueCount, int groupBitCount) + { + int groupIndex = 0; + int groupStart = 0; + while (true) + { + // The first two groups retain the initial width. Later groups grow one bit at a time until the + // finite tail is small enough for the exact non-symmetric alphabet. + int bitCount = groupIndex == 0 ? groupBitCount : groupBitCount + groupIndex - 1; + int groupSize = 1 << bitCount; + if (valueCount <= groupStart + (3 * groupSize)) + { + this.WriteNonSymmetric((uint)(value - groupStart), (uint)(valueCount - groupStart)); + + return; + } + + bool useLaterGroup = value >= groupStart + groupSize; + this.WriteBoolean(useLaterGroup); + if (!useLaterGroup) + { + this.WriteLiteral((uint)(value - groupStart), bitCount); + return; + } + + groupIndex++; + groupStart += groupSize; + } + } + + /// + /// Maps an unsigned value to increasing distance from a reference inside a finite domain. + /// + private static int RecenterFiniteNonNegative(int valueCount, int reference, int value) + { + if ((reference << 1) <= valueCount) + { + return RecenterNonNegative(reference, value); + } + + return RecenterNonNegative(valueCount - 1 - reference, valueCount - 1 - value); + } + + /// + /// Maps an unsigned value to alternating positions around a nonnegative reference. + /// + private static int RecenterNonNegative(int reference, int value) + { + if (value > (reference << 1)) + { + return value; + } + + return value >= reference + ? (value - reference) << 1 + : ((reference - value) << 1) - 1; + } + + /// + /// Appends one bit to the partially assembled output byte. + /// + /// Zero or one. + private void WriteBit(byte value) + { + int bit = this.BitPosition & 0x07; + this.buffer = (byte)(((value << (7 - bit)) & 0xff) | this.buffer); + if (bit == 7) + { + this.WriteBuffer(); + } + + this.BitPosition++; + } + + /// + /// Writes an unsigned integer with its least-significant byte first. + /// + /// The value to write. + /// The number of bytes to write. + public void WriteLittleEndian(uint value, int n) + { + DebugGuard.IsTrue(Av1Math.Modulus8(this.BitPosition) == 0, "Writing of Little Endian value only allowed on byte alignment"); + + uint t = value; + for (int i = 0; i < n; i++) + { + this.WriteLiteral(t & 0xff, 8); + t >>= 8; + } + } + + /// + /// Writes a byte-aligned entropy-coded tile payload. + /// + /// The tile payload. + public void WriteBlob(ReadOnlySpan tileData) + { + DebugGuard.IsTrue(Av1Math.Modulus8(this.BitPosition) == 0, "Writing of Tile Data only allowed on byte alignment"); + + int wordPosition = this.BitPosition >> 3; + tileData.CopyTo(this.span[wordPosition..]); + this.BitPosition += tileData.Length << 3; + } + + /// + /// Stores the current output byte. + /// + private void WriteBuffer() + { + int wordPosition = Av1Math.DivideBy8Floor(this.BitPosition); + this.span[wordPosition] = this.buffer; + this.buffer = 0; + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Av1BlockSize.cs b/src/ImageSharp/Formats/Heif/Av1/Av1BlockSize.cs new file mode 100644 index 0000000000..3c1808c9cf --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Av1BlockSize.cs @@ -0,0 +1,140 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1; + +/// +/// Identifies every luma block size defined by AV1. +/// +internal enum Av1BlockSize : byte +{ + /// + /// A block of samples, 4 samples wide and 4 samples high. + /// + Block4x4 = 0, + + /// + /// A block of samples, 4 samples wide and 8 samples high. + /// + Block4x8 = 1, + + /// + /// A block of samples, 8 samples wide and 4 samples high. + /// + Block8x4 = 2, + + /// + /// A block of samples, 8 samples wide and 8 samples high. + /// + Block8x8 = 3, + + /// + /// A block of samples, 8 samples wide and 16 samples high. + /// + Block8x16 = 4, + + /// + /// A block of samples, 16 samples wide and 8 samples high. + /// + Block16x8 = 5, + + /// + /// A block of samples, 16 samples wide and 16 samples high. + /// + Block16x16 = 6, + + /// + /// A block of samples, 16 samples wide and 32 samples high. + /// + Block16x32 = 7, + + /// + /// A block of samples, 32 samples wide and 16 samples high. + /// + Block32x16 = 8, + + /// + /// A block of samples, 32 samples wide and 32 samples high. + /// + Block32x32 = 9, + + /// + /// A block of samples, 32 samples wide and 64 samples high. + /// + Block32x64 = 10, + + /// + /// A block of samples, 64 samples wide and 32 samples high. + /// + Block64x32 = 11, + + /// + /// A block of samples, 64 samples wide and 64 samples high. + /// + Block64x64 = 12, + + /// + /// A block of samples, 64 samples wide and 128 samples high. + /// + Block64x128 = 13, + + /// + /// A block of samples, 128 samples wide and 64 samples high. + /// + Block128x64 = 14, + + /// + /// A block of samples, 128 samples wide and 128 samples high. + /// + Block128x128 = 15, + + /// + /// A block of samples, 4 samples wide and 16 samples high. + /// + Block4x16 = 16, + + /// + /// A block of samples, 16 samples wide and 4 samples high. + /// + Block16x4 = 17, + + /// + /// A block of samples, 8 samples wide and 32 samples high. + /// + Block8x32 = 18, + + /// + /// A block of samples, 32 samples wide and 8 samples high. + /// + Block32x8 = 19, + + /// + /// A block of samples, 16 samples wide and 64 samples high. + /// + Block16x64 = 20, + + /// + /// A block of samples, 64 samples wide and 16 samples high. + /// + Block64x16 = 21, + + /// + /// The number of concrete block-size values. + /// + AllSizes = 22, + + /// + /// The first extended rectangular block size following the primary size set. + /// + SizeS = Block4x16, + + /// + /// A sentinel representing an invalid block size. + /// + Invalid = 255, + + /// + /// The final value in the primary block-size set. + /// + Largest = SizeS - 1, +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Av1BlockSizeExtensions.cs b/src/ImageSharp/Formats/Heif/Av1/Av1BlockSizeExtensions.cs new file mode 100644 index 0000000000..75005182cb --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Av1BlockSizeExtensions.cs @@ -0,0 +1,245 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1; + +/// +/// Provides dimensions, chroma subsampling, and transform limits for AV1 block sizes. +/// +internal static class Av1BlockSizeExtensions +{ + /// + /// The width of each block size in units of four samples. + /// + private static readonly int[] SizeWide = [1, 1, 2, 2, 2, 4, 4, 4, 8, 8, 8, 16, 16, 16, 32, 32, 1, 4, 2, 8, 4, 16]; + + /// + /// The height of each block size in units of four samples. + /// + private static readonly int[] SizeHigh = [1, 2, 1, 2, 4, 2, 4, 8, 4, 8, 16, 8, 16, 32, 16, 32, 4, 1, 8, 2, 16, 4]; + + /// + /// Maps each luma block size and pair of chroma subsampling shifts to its residual-plane block size. + /// + private static readonly Av1BlockSize[][][] SubSampled = + [ + + // ss_x == 0 ss_x == 0 ss_x == 1 ss_x == 1 + // ss_y == 0 ss_y == 1 ss_y == 0 ss_y == 1 + [[Av1BlockSize.Block4x4, Av1BlockSize.Block4x4], [Av1BlockSize.Block4x4, Av1BlockSize.Block4x4]], + [[Av1BlockSize.Block4x8, Av1BlockSize.Block4x4], [Av1BlockSize.Invalid, Av1BlockSize.Block4x4]], + [[Av1BlockSize.Block8x4, Av1BlockSize.Invalid], [Av1BlockSize.Block4x4, Av1BlockSize.Block4x4]], + [[Av1BlockSize.Block8x8, Av1BlockSize.Block8x4], [Av1BlockSize.Block4x8, Av1BlockSize.Block4x4]], + [[Av1BlockSize.Block8x16, Av1BlockSize.Block8x8], [Av1BlockSize.Invalid, Av1BlockSize.Block4x8]], + [[Av1BlockSize.Block16x8, Av1BlockSize.Invalid], [Av1BlockSize.Block8x8, Av1BlockSize.Block8x4]], + [[Av1BlockSize.Block16x16, Av1BlockSize.Block16x8], [Av1BlockSize.Block8x16, Av1BlockSize.Block8x8]], + [[Av1BlockSize.Block16x32, Av1BlockSize.Block16x16], [Av1BlockSize.Invalid, Av1BlockSize.Block8x16]], + [[Av1BlockSize.Block32x16, Av1BlockSize.Invalid], [Av1BlockSize.Block16x16, Av1BlockSize.Block16x8]], + [[Av1BlockSize.Block32x32, Av1BlockSize.Block32x16], [Av1BlockSize.Block16x32, Av1BlockSize.Block16x16]], + [[Av1BlockSize.Block32x64, Av1BlockSize.Block32x32], [Av1BlockSize.Invalid, Av1BlockSize.Block16x32]], + [[Av1BlockSize.Block64x32, Av1BlockSize.Invalid], [Av1BlockSize.Block32x32, Av1BlockSize.Block32x16]], + [[Av1BlockSize.Block64x64, Av1BlockSize.Block64x32], [Av1BlockSize.Block32x64, Av1BlockSize.Block32x32]], + [[Av1BlockSize.Block64x128, Av1BlockSize.Block64x64], [Av1BlockSize.Invalid, Av1BlockSize.Block32x64]], + [[Av1BlockSize.Block128x64, Av1BlockSize.Invalid], [Av1BlockSize.Block64x64, Av1BlockSize.Block64x32]], + [[Av1BlockSize.Block128x128, Av1BlockSize.Block128x64], [Av1BlockSize.Block64x128, Av1BlockSize.Block64x64]], + [[Av1BlockSize.Block4x16, Av1BlockSize.Block4x8], [Av1BlockSize.Invalid, Av1BlockSize.Block4x8]], + [[Av1BlockSize.Block16x4, Av1BlockSize.Invalid], [Av1BlockSize.Block8x4, Av1BlockSize.Block8x4]], + [[Av1BlockSize.Block8x32, Av1BlockSize.Block8x16], [Av1BlockSize.Invalid, Av1BlockSize.Block4x16]], + [[Av1BlockSize.Block32x8, Av1BlockSize.Invalid], [Av1BlockSize.Block16x8, Av1BlockSize.Block16x4]], + [[Av1BlockSize.Block16x64, Av1BlockSize.Block16x32], [Av1BlockSize.Invalid, Av1BlockSize.Block8x32]], + [[Av1BlockSize.Block64x16, Av1BlockSize.Invalid], [Av1BlockSize.Block32x16, Av1BlockSize.Block32x8]] + ]; + + /// + /// Maps each block size to its largest permitted transform size. + /// + private static readonly Av1TransformSize[] MaxTransformSize = [ + Av1TransformSize.Size4x4, Av1TransformSize.Size4x8, Av1TransformSize.Size8x4, Av1TransformSize.Size8x8, + Av1TransformSize.Size8x16, Av1TransformSize.Size16x8, Av1TransformSize.Size16x16, Av1TransformSize.Size16x32, + Av1TransformSize.Size32x16, Av1TransformSize.Size32x32, Av1TransformSize.Size32x64, Av1TransformSize.Size64x32, + Av1TransformSize.Size64x64, Av1TransformSize.Size64x64, Av1TransformSize.Size64x64, Av1TransformSize.Size64x64, + Av1TransformSize.Size4x16, Av1TransformSize.Size16x4, Av1TransformSize.Size8x32, Av1TransformSize.Size32x8, + Av1TransformSize.Size16x64, Av1TransformSize.Size64x16 + ]; + + /// + /// Contains the base-two logarithm of the sample count for each block size. + /// + private static readonly int[] PelsLog2Count = + [4, 5, 5, 6, 7, 7, 8, 9, 9, 10, 11, 11, 12, 13, 13, 14, 6, 6, 8, 8, 10, 10]; + + /// + /// Maps geometry dimension logarithms to an AV1 block size using the mode-decision scan's transposed axis convention. + /// + private static readonly Av1BlockSize[][] HeightWidthToSize = [ + [Av1BlockSize.Block4x4, Av1BlockSize.Block4x8, Av1BlockSize.Block4x16, Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Invalid], + [Av1BlockSize.Block8x4, Av1BlockSize.Block8x8, Av1BlockSize.Block8x16, Av1BlockSize.Block8x32, Av1BlockSize.Invalid, Av1BlockSize.Invalid], + [Av1BlockSize.Block16x4, Av1BlockSize.Block16x8, Av1BlockSize.Block16x16, Av1BlockSize.Block16x32, Av1BlockSize.Block16x64, Av1BlockSize.Invalid], + [Av1BlockSize.Invalid, Av1BlockSize.Block32x8, Av1BlockSize.Block32x16, Av1BlockSize.Block32x32, Av1BlockSize.Block32x64, Av1BlockSize.Invalid], + [Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Block64x16, Av1BlockSize.Block64x32, Av1BlockSize.Block64x64, Av1BlockSize.Block64x128], + [Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Block128x64, Av1BlockSize.Block128x128] + ]; + + /// + /// Gets the block width in units of four samples. + /// + /// The block size. + /// The number of four-sample columns. + public static int Get4x4WideCount(this Av1BlockSize blockSize) => SizeWide[(int)blockSize]; + + /// + /// Gets the block height in units of four samples. + /// + /// The block size. + /// The number of four-sample rows. + public static int Get4x4HighCount(this Av1BlockSize blockSize) => SizeHigh[(int)blockSize]; + + /// + /// Gets the block size from mode-decision geometry dimension logarithms, where zero represents four samples. + /// + /// The base-two width logarithm minus two. + /// The base-two height logarithm minus two. + /// The matching block size, or for unsupported dimensions. + public static Av1BlockSize FromWidthAndHeight(uint widthLog2, uint heightLog2) + { + // Mode-decision geometry is ported with its source axis order, so its size lookup is indexed height first. + return HeightWidthToSize[heightLog2][widthLog2]; + } + + /// + /// Gets the block width in samples. + /// + /// The block size. + /// The block width in samples. + public static int GetWidth(this Av1BlockSize blockSize) + => Get4x4WideCount(blockSize) << 2; + + /// + /// Gets the block height in samples. + /// + /// The block size. + /// The block height in samples. + public static int GetHeight(this Av1BlockSize blockSize) + => Get4x4HighCount(blockSize) << 2; + + /// + /// Gets the base-two logarithm of the block width in units of four samples. + /// + /// The block size. + /// The base-two logarithm of the four-sample column count. + public static int Get4x4WidthLog2(this Av1BlockSize blockSize) + => Av1Math.Log2(Get4x4WideCount(blockSize)); + + /// + /// Gets the base-two logarithm of the block height in units of four samples. + /// + /// The block size. + /// The base-two logarithm of the four-sample row count. + public static int Get4x4HeightLog2(this Av1BlockSize blockSize) + => Av1Math.Log2(Get4x4HighCount(blockSize)); + + /// + /// Gets the entropy context group associated with the block size. + /// + /// The block size. + /// The zero-based size group in the inclusive range zero through three. + public static int GetSizeGroup(this Av1BlockSize blockSize) + { + // AV1 section 9.3 groups a block by its smaller dimension in 4x4 units and caps that logarithm at three. + // Deriving the value from the existing geometry tables exactly matches the reference decoder's size_group_lookup table. + return Math.Min(3, Math.Min(blockSize.Get4x4WidthLog2(), blockSize.Get4x4HeightLog2())); + } + + /// + /// Gets the residual-plane block size for Boolean chroma subsampling flags. + /// + /// The luma block size. + /// Indicates horizontal chroma subsampling. + /// Indicates vertical chroma subsampling. + /// The corresponding residual-plane block size. + public static Av1BlockSize GetSubsampled(this Av1BlockSize blockSize, bool subX, bool subY) + => GetSubsampled(blockSize, subX ? 1 : 0, subY ? 1 : 0); + + /// + /// Gets the residual-plane block size for chroma subsampling shifts. + /// + /// The luma block size. + /// The horizontal chroma subsampling shift. + /// The vertical chroma subsampling shift. + /// The corresponding residual-plane block size, or when unavailable. + public static Av1BlockSize GetSubsampled(this Av1BlockSize blockSize, int subX, int subY) + { + if (blockSize == Av1BlockSize.Invalid) + { + return Av1BlockSize.Invalid; + } + + return SubSampled[(int)blockSize][subX][subY]; + } + + /// + /// Determines whether a luma block permits chroma-from-luma prediction. + /// + /// The luma block size. + /// Indicates whether the block belongs to a lossless segment. + /// Indicates horizontal chroma subsampling. + /// Indicates vertical chroma subsampling. + /// when chroma-from-luma prediction is permitted; otherwise, . + public static bool AllowsChromaFromLuma( + this Av1BlockSize blockSize, + bool isLossless, + bool subX, + bool subY) + { + if (isLossless) + { + // Lossless coding fixes the transform to 4x4, so the subsampled chroma block must have the same dimensions. + return blockSize.GetSubsampled(subX, subY) == Av1BlockSize.Block4x4; + } + + return blockSize.GetWidth() <= 32 && blockSize.GetHeight() <= 32; + } + + /// + /// Gets the maximum chroma transform size after applying plane subsampling and AV1 chroma transform limits. + /// + /// The luma block size. + /// Indicates horizontal chroma subsampling. + /// Indicates vertical chroma subsampling. + /// The maximum chroma transform size, or when the plane block size is invalid. + public static Av1TransformSize GetMaxUvTransformSize(this Av1BlockSize blockSize, bool subX, bool subY) + { + Av1BlockSize planeBlockSize = blockSize.GetSubsampled(subX, subY); + Av1TransformSize uvTransformSize = Av1TransformSize.Invalid; + if (planeBlockSize < Av1BlockSize.AllSizes) + { + uvTransformSize = planeBlockSize.GetMaximumTransformSize(); + } + + return uvTransformSize switch + { + Av1TransformSize.Size64x64 or Av1TransformSize.Size64x32 or Av1TransformSize.Size32x64 => Av1TransformSize.Size32x32, + Av1TransformSize.Size64x16 => Av1TransformSize.Size32x16, + Av1TransformSize.Size16x64 => Av1TransformSize.Size16x32, + _ => uvTransformSize, + }; + } + + /// + /// Gets the largest square or rectangular transform size permitted for a block. + /// + /// The block size. + /// The maximum transform size. + public static Av1TransformSize GetMaximumTransformSize(this Av1BlockSize blockSize) + => MaxTransformSize[(int)blockSize]; + + /// + /// Gets the base-two logarithm of the block's sample count. + /// + /// The block size. + /// The base-two logarithm of width multiplied by height. + public static int GetPelsLog2Count(this Av1BlockSize blockSize) + => PelsLog2Count[(int)blockSize]; +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Av1CodecConfiguration.cs b/src/ImageSharp/Formats/Heif/Av1/Av1CodecConfiguration.cs new file mode 100644 index 0000000000..5e39a3f5d7 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Av1CodecConfiguration.cs @@ -0,0 +1,825 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Buffers.Binary; +using SixLabors.ImageSharp.ColorProfiles; +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1; + +/// +/// Contains the image-description fields stored in an AV1 codec-configuration item property. +/// +internal sealed class Av1CodecConfiguration +{ + /// + /// The number of bytes in the fixed AV1 codec-configuration record. + /// + public const int FixedHeaderSize = 4; + + /// + /// The optional sequence-header payload retained from the configuration open bitstream units. + /// + private readonly byte[] configSequenceHeader; + + /// + /// The sequence-header OBU extension byte, or -1 when its header has no extension. + /// + private readonly int configSequenceHeaderExtension; + + /// + /// The content light-level metadata carried by the configuration OBUs, or when absent. + /// + private readonly HeifContentLightLevel? configContentLightLevel; + + /// + /// The mastering-display color volume carried by the configuration OBUs, or when absent. + /// + private readonly HeifMasteringDisplayColorVolume? configMasteringDisplayColorVolume; + + /// + /// Initializes a new instance of the class from an AV1 codec-configuration + /// item-property payload. + /// + /// The configuration payload beginning with the marker and version fields. + /// The general options governing metadata validation. + public Av1CodecConfiguration(Span boxBuffer, DecoderOptions options) + { + if (boxBuffer.Length < FixedHeaderSize) + { + throw new InvalidImageContentException("The AV1 codec configuration is truncated."); + } + + Av1BitStreamReader reader = new(boxBuffer); + uint marker = reader.ReadLiteral(1); + uint version = reader.ReadLiteral(7); + if (marker != 1 || version != 1) + { + throw new InvalidImageContentException("The AV1 codec configuration has an invalid marker or version."); + } + + this.SequenceProfile = (byte)reader.ReadLiteral(3); + this.SequenceLevelIndex = (byte)reader.ReadLiteral(5); + this.SequenceTier = reader.ReadLiteral(1) == 1; + this.HighBitDepth = reader.ReadLiteral(1) == 1; + this.TwelveBit = reader.ReadLiteral(1) == 1; + this.IsMonochrome = reader.ReadLiteral(1) == 1; + this.ChromaSubsamplingX = reader.ReadLiteral(1) == 1; + this.ChromaSubsamplingY = reader.ReadLiteral(1) == 1; + this.ChromaSamplePosition = (byte)reader.ReadLiteral(2); + if (this.SequenceProfile > (byte)ObuSequenceProfile.Professional + || (this.TwelveBit && !this.HighBitDepth) + || this.ChromaSamplePosition == (byte)ObuChromoSamplePosition.Reserved) + { + throw new InvalidImageContentException("The AV1 codec configuration contains invalid image-description fields."); + } + + if (reader.ReadLiteral(3) != 0) + { + throw new InvalidImageContentException("The AV1 codec configuration has nonzero reserved bits."); + } + + bool hasInitialPresentationDelay = reader.ReadLiteral(1) == 1; + uint delayOrReserved = reader.ReadLiteral(4); + if (!hasInitialPresentationDelay && delayOrReserved != 0) + { + throw new InvalidImageContentException("The AV1 codec configuration has a nonzero reserved delay field."); + } + + // The delay syntax is consumed to validate the fixed record, but it describes sample presentation and has + // no meaning for the independently presented image item supported by this bounded container implementation. + ReadOnlySpan configObus = boxBuffer[FixedHeaderSize..]; + int sequenceHeaderCount = ScanObus( + configObus, + true, + true, + "AV1 codec configuration", + options, + out int configSequenceHeaderOffset, + out int configSequenceHeaderLength, + out this.configSequenceHeaderExtension, + out this.configContentLightLevel, + out this.configMasteringDisplayColorVolume); + + if (sequenceHeaderCount > 1) + { + throw new InvalidImageContentException("The AV1 codec configuration contains more than one sequence header OBU."); + } + + if (configSequenceHeaderOffset < 0) + { + this.configSequenceHeader = []; + } + else + { + // The property-reader span is pooled and reused. Retain only the sequence-header bytes required for + // item/sample equivalence instead of materializing every optional configuration OBU. + this.configSequenceHeader = new byte[configSequenceHeaderLength]; + configObus.Slice(configSequenceHeaderOffset, configSequenceHeaderLength).CopyTo(this.configSequenceHeader); + } + } + + /// + /// Initializes a new instance of the class from an encoded sequence header. + /// + /// The sequence header describing the encoded image item. + public Av1CodecConfiguration(ObuSequenceHeader sequenceHeader) + { + ObuOperatingPoint operatingPoint = sequenceHeader.OperatingPoint[0]; + ObuColorConfig colorConfig = sequenceHeader.ColorConfig; + + this.SequenceProfile = (byte)sequenceHeader.SequenceProfile; + this.SequenceLevelIndex = (byte)operatingPoint.SequenceLevelIndex; + this.SequenceTier = operatingPoint.SequenceTier != 0; + this.HighBitDepth = colorConfig.BitDepth is Av1BitDepth.TenBit or Av1BitDepth.TwelveBit; + this.TwelveBit = colorConfig.BitDepth == Av1BitDepth.TwelveBit; + this.IsMonochrome = colorConfig.IsMonochrome; + this.ChromaSubsamplingX = colorConfig.SubSamplingX; + this.ChromaSubsamplingY = colorConfig.SubSamplingY; + this.ChromaSamplePosition = (byte)colorConfig.ChromaSamplePosition; + this.configSequenceHeader = []; + this.configSequenceHeaderExtension = -1; + this.configContentLightLevel = null; + this.configMasteringDisplayColorVolume = null; + } + + /// + /// Gets the sequence profile declared for the coded image. + /// + public byte SequenceProfile { get; } + + /// + /// Gets the first operating point's sequence-level index. + /// + public byte SequenceLevelIndex { get; } + + /// + /// Gets a value indicating whether the first operating point uses the high tier. + /// + public bool SequenceTier { get; } + + /// + /// Gets a value indicating whether the coded image uses more than eight bits per sample. + /// + public bool HighBitDepth { get; } + + /// + /// Gets a value indicating whether the coded image uses twelve bits per sample. + /// + public bool TwelveBit { get; } + + /// + /// Gets the coded image sample precision in bits. + /// + public HeifBitDepth BitDepth => this.TwelveBit ? HeifBitDepth.Bit12 : this.HighBitDepth ? HeifBitDepth.Bit10 : HeifBitDepth.Bit8; + + /// + /// Gets a value indicating whether the coded image contains only a luma plane. + /// + public bool IsMonochrome { get; } + + /// + /// Gets a value indicating whether the coded image's chroma planes are horizontally subsampled. + /// + public bool ChromaSubsamplingX { get; } + + /// + /// Gets a value indicating whether the coded image's chroma planes are vertically subsampled. + /// + public bool ChromaSubsamplingY { get; } + + /// + /// Gets the position of vertically subsampled chroma samples relative to luma samples. + /// + public byte ChromaSamplePosition { get; } + + /// + /// Writes the fixed AV1 codec-configuration record without optional configuration OBUs. + /// + /// The destination receiving the four-byte record. + public void WriteFixedHeader(Span destination) + { + // The image item payload already begins with its required sequence header. Keeping configOBUs empty avoids + // retaining or copying the same OBU into the configuration property. + destination[0] = 0x81; + destination[1] = (byte)((this.SequenceProfile << 5) | this.SequenceLevelIndex); + destination[2] = (byte)( + (this.SequenceTier ? 1 << 7 : 0) + | (this.HighBitDepth ? 1 << 6 : 0) + | (this.TwelveBit ? 1 << 5 : 0) + | (this.IsMonochrome ? 1 << 4 : 0) + | (this.ChromaSubsamplingX ? 1 << 3 : 0) + | (this.ChromaSubsamplingY ? 1 << 2 : 0) + | this.ChromaSamplePosition); + destination[3] = 0; + } + + /// + /// Validates the AV1 image item OBU layout and metadata against its item properties and configuration record. + /// + /// The complete AV1 image item payload. + /// + /// The content light-level property associated with the image item, or when absent. + /// + /// + /// The mastering-display property associated with the image item, or when absent. + /// + /// The general options governing metadata validation. + /// + /// Receives the content light-level metadata carried by the combined configuration and item OBUs. + /// + /// + /// Receives the mastering-display metadata carried by the combined configuration and item OBUs. + /// + public void ValidateItemData( + ReadOnlySpan itemData, + HeifContentLightLevel? itemContentLightLevel, + HeifMasteringDisplayColorVolume? itemMasteringDisplayColorVolume, + DecoderOptions options, + out HeifContentLightLevel? contentLightLevel, + out HeifMasteringDisplayColorVolume? masteringDisplayColorVolume) + => this.ValidateData( + itemData, + true, + "AV1 image item", + itemContentLightLevel, + itemMasteringDisplayColorVolume, + options, + out contentLightLevel, + out masteringDisplayColorVolume); + + /// + /// Validates one AV1 track sample against its sync-sample declaration, sample-entry metadata, and configuration record. + /// + /// The complete AV1 sample payload. + /// Indicates that the sample is declared as a random-access point. + /// + /// The content light-level property associated with the sample entry, or when absent. + /// + /// + /// The mastering-display property associated with the sample entry, or when absent. + /// + /// The general options governing metadata validation. + /// + /// Receives the content light-level metadata carried by the combined configuration and sample OBUs. + /// + /// + /// Receives the mastering-display metadata carried by the combined configuration and sample OBUs. + /// + public void ValidateSampleData( + ReadOnlySpan sampleData, + bool isSyncSample, + HeifContentLightLevel? sampleContentLightLevel, + HeifMasteringDisplayColorVolume? sampleMasteringDisplayColorVolume, + DecoderOptions options, + out HeifContentLightLevel? contentLightLevel, + out HeifMasteringDisplayColorVolume? masteringDisplayColorVolume) + => this.ValidateData( + sampleData, + isSyncSample, + "AV1 track sample", + sampleContentLightLevel, + sampleMasteringDisplayColorVolume, + options, + out contentLightLevel, + out masteringDisplayColorVolume); + + /// + /// Validates one bounded AV1 payload while applying the item or track sequence-header requirement. + /// + /// The complete bounded AV1 payload. + /// Indicates that exactly one sequence header is required. + /// The source description used by invalid-content errors. + /// The content light-level property associated with the payload. + /// The mastering-display property associated with the payload. + /// The general options governing metadata validation. + /// Receives validated OBU content light-level metadata. + /// Receives validated OBU mastering-display metadata. + private void ValidateData( + ReadOnlySpan data, + bool sequenceHeaderRequired, + string sourceName, + HeifContentLightLevel? containerContentLightLevel, + HeifMasteringDisplayColorVolume? containerMasteringDisplayColorVolume, + DecoderOptions options, + out HeifContentLightLevel? contentLightLevel, + out HeifMasteringDisplayColorVolume? masteringDisplayColorVolume) + { + int sequenceHeaderCount = ScanObus( + data, + false, + false, + sourceName, + options, + out int dataSequenceHeaderOffset, + out int dataSequenceHeaderLength, + out int dataSequenceHeaderExtension, + out HeifContentLightLevel? dataObuContentLightLevel, + out HeifMasteringDisplayColorVolume? dataObuMasteringDisplayColorVolume); + + if (sequenceHeaderCount > 1 || (sequenceHeaderRequired && sequenceHeaderCount != 1)) + { + string requirement = sequenceHeaderRequired ? "exactly one" : "at most one"; + throw new InvalidImageContentException($"The {sourceName} contains {sequenceHeaderCount} sequence header OBUs instead of {requirement}."); + } + + if (this.configSequenceHeader.Length != 0 && dataSequenceHeaderOffset >= 0) + { + ReadOnlySpan dataSequenceHeader = data.Slice( + dataSequenceHeaderOffset, + dataSequenceHeaderLength); + + // Compare the extension and payload rather than the encoded OBU size. Configuration OBUs must carry a + // size field while a payload's final OBU may omit one, and different legal LEB128 widths do not alter + // the Sequence Header OBU being repeated. + if (this.configSequenceHeaderExtension != dataSequenceHeaderExtension + || !this.configSequenceHeader.AsSpan().SequenceEqual(dataSequenceHeader)) + { + throw new InvalidImageContentException( + $"The AV1 codec configuration sequence header does not match the {sourceName} sequence header."); + } + } + + contentLightLevel = null; + masteringDisplayColorVolume = null; + if (options.SkipMetadata) + { + return; + } + + try + { + ValidateContentLightLevel(this.configContentLightLevel, containerContentLightLevel, "AV1 codec configuration"); + ValidateContentLightLevel(dataObuContentLightLevel, containerContentLightLevel, sourceName); + ValidateMasteringDisplayColorVolume( + this.configMasteringDisplayColorVolume, + containerMasteringDisplayColorVolume, + "AV1 codec configuration"); + + ValidateMasteringDisplayColorVolume( + dataObuMasteringDisplayColorVolume, + containerMasteringDisplayColorVolume, + sourceName); + + if (this.configContentLightLevel is not null + && dataObuContentLightLevel is not null + && !ContentLightLevelsMatch(this.configContentLightLevel.Value, dataObuContentLightLevel.Value)) + { + throw new InvalidImageContentException( + $"The AV1 codec configuration and {sourceName} contain conflicting content light-level metadata."); + } + + if (this.configMasteringDisplayColorVolume is not null + && dataObuMasteringDisplayColorVolume is not null + && this.configMasteringDisplayColorVolume.Value != dataObuMasteringDisplayColorVolume.Value) + { + throw new InvalidImageContentException( + $"The AV1 codec configuration and {sourceName} contain conflicting mastering-display metadata."); + } + + // Configuration OBUs precede the payload OBUs, so a payload OBU supplies the effective value when both + // sequences repeat the same metadata type. + contentLightLevel = dataObuContentLightLevel ?? this.configContentLightLevel; + masteringDisplayColorVolume = dataObuMasteringDisplayColorVolume ?? this.configMasteringDisplayColorVolume; + } + catch (Exception ex) when (ImageDecoderCore.ShouldIgnoreAncillarySegmentError(options, ex)) + { + // Conflicting optional OBU metadata is discarded without weakening OBU framing or sequence-header checks. + } + } + + /// + /// Determines whether another item configuration describes the same coded-image sample layout. + /// + /// The configuration to compare. + /// when every fixed image-description field is equal. + public bool HasMatchingImageConfiguration(Av1CodecConfiguration other) + => this.SequenceProfile == other.SequenceProfile + && this.SequenceLevelIndex == other.SequenceLevelIndex + && this.SequenceTier == other.SequenceTier + && this.HighBitDepth == other.HighBitDepth + && this.TwelveBit == other.TwelveBit + && this.IsMonochrome == other.IsMonochrome + && this.ChromaSubsamplingX == other.ChromaSubsamplingX + && this.ChromaSubsamplingY == other.ChromaSubsamplingY + && this.ChromaSamplePosition == other.ChromaSamplePosition; + + /// + /// Validates the configuration fields against the sequence header that describes the coded image item. + /// + /// The decoded AV1 sequence header. + public void Validate(ObuSequenceHeader sequenceHeader) + { + ObuOperatingPoint operatingPoint = sequenceHeader.OperatingPoint[0]; + ObuColorConfig colorConfig = sequenceHeader.ColorConfig; + bool highBitDepth = colorConfig.BitDepth is Av1BitDepth.TenBit or Av1BitDepth.TwelveBit; + bool twelveBit = colorConfig.BitDepth == Av1BitDepth.TwelveBit; + + if (this.SequenceProfile != (byte)sequenceHeader.SequenceProfile + || this.SequenceLevelIndex != operatingPoint.SequenceLevelIndex + || this.SequenceTier != (operatingPoint.SequenceTier != 0) + || this.HighBitDepth != highBitDepth + || this.TwelveBit != twelveBit + || this.IsMonochrome != colorConfig.IsMonochrome + || this.ChromaSubsamplingX != colorConfig.SubSamplingX + || this.ChromaSubsamplingY != colorConfig.SubSamplingY + || this.ChromaSamplePosition != (byte)colorConfig.ChromaSamplePosition) + { + throw new InvalidImageContentException("The AV1 item configuration does not match its sequence header."); + } + } + + /// + /// Scans a low-overhead AV1 OBU sequence and locates its still-image description metadata. + /// + /// The complete bounded OBU sequence. + /// Indicates that every OBU must carry its registered payload-size field. + /// + /// Indicates that a sequence-header OBU, when present, must be the first OBU in the sequence. + /// + /// The source description used by invalid-content errors. + /// The general options governing metadata validation. + /// Receives the first sequence-header payload offset, or -1. + /// Receives the first sequence-header payload length. + /// Receives the first sequence-header extension byte, or -1. + /// + /// Receives the content light-level metadata carried by the sequence, or when absent. + /// + /// + /// Receives the mastering-display metadata carried by the sequence, or when absent. + /// + /// The number of sequence-header OBUs in the sequence. + private static int ScanObus( + ReadOnlySpan data, + bool requireSizeFields, + bool sequenceHeaderMustBeFirst, + string sourceName, + DecoderOptions options, + out int sequenceHeaderOffset, + out int sequenceHeaderLength, + out int sequenceHeaderExtension, + out HeifContentLightLevel? contentLightLevel, + out HeifMasteringDisplayColorVolume? masteringDisplayColorVolume) + { + sequenceHeaderOffset = -1; + sequenceHeaderLength = 0; + sequenceHeaderExtension = -1; + contentLightLevel = null; + masteringDisplayColorVolume = null; + int sequenceHeaderCount = 0; + int obuIndex = 0; + int offset = 0; + while (offset < data.Length) + { + byte header = data[offset++]; + if ((header & 0x80) != 0) + { + throw new InvalidImageContentException($"The {sourceName} contains an OBU with a set forbidden header bit."); + } + + // The reference decoder deliberately ignores obu_reserved_1bit. The bit does not alter the OBU boundary or + // decoded syntax, so the bounded container scan must not reject data that the production parser accepts. + ObuType type = (ObuType)((header >> 3) & 0x0F); + bool hasExtension = (header & 0x04) != 0; + bool hasSizeField = (header & 0x02) != 0; + int extension = -1; + if (hasExtension) + { + if (offset >= data.Length) + { + throw new InvalidImageContentException($"The {sourceName} contains a truncated OBU extension header."); + } + + // extension_header_reserved_3bits is also consumed but ignored by the reference decoder. + extension = data[offset++]; + } + + if (requireSizeFields && !hasSizeField) + { + throw new InvalidImageContentException($"The {sourceName} contains an OBU without its required payload-size field."); + } + + int payloadLength; + if (hasSizeField) + { + payloadLength = ReadObuPayloadLength(data, ref offset, sourceName); + } + else + { + // Low-overhead image item syntax permits only the final OBU to omit its size, in which case the + // remaining item bytes are that OBU's payload and cannot contain another independently parsed OBU. + payloadLength = data.Length - offset; + } + + if (payloadLength > data.Length - offset) + { + throw new InvalidImageContentException($"The {sourceName} contains an OBU payload that exceeds its data boundary."); + } + + if (type == ObuType.SequenceHeader) + { + if (sequenceHeaderMustBeFirst && obuIndex != 0) + { + throw new InvalidImageContentException($"The {sourceName} contains a sequence header OBU after another OBU."); + } + + sequenceHeaderCount++; + if (sequenceHeaderOffset < 0) + { + sequenceHeaderOffset = offset; + sequenceHeaderLength = payloadLength; + sequenceHeaderExtension = extension; + } + } + else if (type == ObuType.Metadata && !options.SkipMetadata) + { + try + { + ReadHdrMetadata( + data.Slice(offset, payloadLength), + sourceName, + out HeifContentLightLevel? obuContentLightLevel, + out HeifMasteringDisplayColorVolume? obuMasteringDisplayColorVolume); + + if (obuContentLightLevel is not null) + { + if (contentLightLevel is not null + && !ContentLightLevelsMatch(contentLightLevel.Value, obuContentLightLevel.Value)) + { + throw new InvalidImageContentException($"The {sourceName} contains conflicting content light-level metadata OBUs."); + } + + contentLightLevel = obuContentLightLevel; + } + + if (obuMasteringDisplayColorVolume is not null) + { + if (masteringDisplayColorVolume is not null + && masteringDisplayColorVolume.Value != obuMasteringDisplayColorVolume.Value) + { + throw new InvalidImageContentException($"The {sourceName} contains conflicting mastering-display metadata OBUs."); + } + + masteringDisplayColorVolume = obuMasteringDisplayColorVolume; + } + } + catch (Exception ex) when (ImageDecoderCore.ShouldIgnoreAncillarySegmentError(options, ex)) + { + // The OBU payload remains bounded by the image-data scan; only its invalid optional metadata is discarded. + } + } + + offset += payloadLength; + obuIndex++; + } + + return sequenceHeaderCount; + } + + /// + /// Reads a bounded AV1 little-endian base-128 OBU payload length. + /// + /// The complete bounded OBU sequence. + /// The current byte offset, advanced past the encoded length. + /// The source description used by invalid-content errors. + /// The payload length representable by the current item buffer. + private static int ReadObuPayloadLength(ReadOnlySpan data, ref int offset, string sourceName) + { + ulong value = ReadLeb128(data, ref offset, sourceName, "OBU payload length"); + if (value > int.MaxValue) + { + throw new InvalidImageContentException($"The {sourceName} contains an OBU payload too large to buffer."); + } + + return (int)value; + } + + /// + /// Reads still-image high-dynamic-range data from an AV1 metadata OBU payload. + /// + /// The bounded metadata OBU payload. + /// The source description used by invalid-content errors. + /// Receives decoded content light-level metadata when present. + /// Receives decoded mastering-display metadata when present. + public static void ReadHdrMetadata( + ReadOnlySpan payload, + string sourceName, + out HeifContentLightLevel? contentLightLevel, + out HeifMasteringDisplayColorVolume? masteringDisplayColorVolume) + { + contentLightLevel = null; + masteringDisplayColorVolume = null; + int offset = 0; + ulong metadataType = ReadLeb128(payload, ref offset, sourceName, "metadata type"); + if (metadataType != (ulong)ObuMetadataType.HdrCll + && metadataType != (ulong)ObuMetadataType.HdrMdcv) + { + return; + } + + int metadataLength = metadataType == (ulong)ObuMetadataType.HdrCll ? 4 : 24; + if (payload.Length - offset <= metadataLength) + { + throw new InvalidImageContentException($"The {sourceName} contains truncated HDR metadata or no trailing bits."); + } + + ReadOnlySpan metadataData = payload.Slice(offset, metadataLength); + ValidateByteAlignedMetadataTrailingBits(payload[(offset + metadataLength)..], sourceName); + + if (metadataType == (ulong)ObuMetadataType.HdrCll) + { + contentLightLevel = new HeifContentLightLevel( + BinaryPrimitives.ReadUInt16BigEndian(metadataData), + BinaryPrimitives.ReadUInt16BigEndian(metadataData[2..])); + + return; + } + + const float chromaticityScale = 1F / 65536F; + const double maximumLuminanceScale = 1D / 256D; + const double minimumLuminanceScale = 1D / 16384D; + + // AV1 stores the primaries in R, G, B order and uses codec-specific fixed-point units that differ from the + // ISOBMFF mdcv property. Decode both representations to the same observable ImageSharp color coordinates. + CieXyChromaticityCoordinates redPrimary = new( + BinaryPrimitives.ReadUInt16BigEndian(metadataData) * chromaticityScale, + BinaryPrimitives.ReadUInt16BigEndian(metadataData[2..]) * chromaticityScale); + + CieXyChromaticityCoordinates greenPrimary = new( + BinaryPrimitives.ReadUInt16BigEndian(metadataData[4..]) * chromaticityScale, + BinaryPrimitives.ReadUInt16BigEndian(metadataData[6..]) * chromaticityScale); + + CieXyChromaticityCoordinates bluePrimary = new( + BinaryPrimitives.ReadUInt16BigEndian(metadataData[8..]) * chromaticityScale, + BinaryPrimitives.ReadUInt16BigEndian(metadataData[10..]) * chromaticityScale); + + masteringDisplayColorVolume = new HeifMasteringDisplayColorVolume( + new RgbPrimariesChromaticityCoordinates(redPrimary, greenPrimary, bluePrimary), + new CieXyChromaticityCoordinates( + BinaryPrimitives.ReadUInt16BigEndian(metadataData[12..]) * chromaticityScale, + BinaryPrimitives.ReadUInt16BigEndian(metadataData[14..]) * chromaticityScale), + BinaryPrimitives.ReadUInt32BigEndian(metadataData[16..]) * maximumLuminanceScale, + BinaryPrimitives.ReadUInt32BigEndian(metadataData[20..]) * minimumLuminanceScale); + } + + /// + /// Validates the trailing bits of byte-aligned fixed-length AV1 metadata. + /// + /// The metadata payload bytes following its fixed fields. + /// The source description used by invalid-content errors. + private static void ValidateByteAlignedMetadataTrailingBits(ReadOnlySpan trailingData, string sourceName) + { + byte lastNonzeroByte = 0; + for (int i = trailingData.Length - 1; i >= 0; i--) + { + if (trailingData[i] != 0) + { + lastNonzeroByte = trailingData[i]; + break; + } + } + + // Both fixed HDR structures end on a byte boundary. Zero padding after the required 0x80 byte is accepted, + // so locate the last nonzero byte rather than assuming the OBU payload ends immediately after trailing_bits(). + if (lastNonzeroByte != 0x80) + { + throw new InvalidImageContentException($"The {sourceName} HDR metadata has invalid trailing bits."); + } + } + + /// + /// Reads a bounded AV1 little-endian base-128 value. + /// + /// The complete bounded byte sequence. + /// The current byte offset, advanced past the encoded value. + /// The source description used by invalid-content errors. + /// The value description used by invalid-content errors. + /// The decoded unsigned value. + private static ulong ReadLeb128( + ReadOnlySpan data, + ref int offset, + string sourceName, + string valueName) + { + ulong value = 0; + for (int byteIndex = 0; byteIndex < 8; byteIndex++) + { + if (offset >= data.Length) + { + throw new InvalidImageContentException($"The {sourceName} contains a truncated {valueName}."); + } + + byte current = data[offset++]; + value |= (ulong)(current & 0x7F) << (byteIndex * 7); + if ((current & 0x80) == 0) + { + if (value > uint.MaxValue) + { + throw new InvalidImageContentException($"The {sourceName} contains a {valueName} larger than the AV1 32-bit limit."); + } + + return value; + } + } + + throw new InvalidImageContentException($"The {sourceName} contains an unterminated {valueName}."); + } + + /// + /// Validates content light-level metadata against the corresponding image-item property when both are present. + /// + /// The value carried by an AV1 metadata OBU. + /// The value carried by the associated image-item property. + /// The OBU source description used by invalid-content errors. + private static void ValidateContentLightLevel( + HeifContentLightLevel? obuContentLightLevel, + HeifContentLightLevel? itemContentLightLevel, + string sourceName) + { + if (obuContentLightLevel is not null + && itemContentLightLevel is not null + && !ContentLightLevelsMatch(obuContentLightLevel.Value, itemContentLightLevel.Value)) + { + throw new InvalidImageContentException($"The {sourceName} content light-level metadata does not match the image-item property."); + } + } + + /// + /// Determines whether two content light-level descriptions carry the same observable values. + /// + /// The first content light-level description. + /// The second content light-level description. + /// when both light-level fields are equal. + private static bool ContentLightLevelsMatch(HeifContentLightLevel left, HeifContentLightLevel right) + { + return left.MaximumContentLightLevel == right.MaximumContentLightLevel + && left.MaximumPictureAverageLightLevel == right.MaximumPictureAverageLightLevel; + } + + /// + /// Validates mastering-display metadata against the corresponding image-item property when both are present. + /// + /// The value carried by an AV1 metadata OBU. + /// The value carried by the associated image-item property. + /// The OBU source description used by invalid-content errors. + private static void ValidateMasteringDisplayColorVolume( + HeifMasteringDisplayColorVolume? obuColorVolume, + HeifMasteringDisplayColorVolume? itemColorVolume, + string sourceName) + { + if (obuColorVolume is not null + && itemColorVolume is not null + && !MasteringDisplayColorVolumesMatch(obuColorVolume.Value, itemColorVolume.Value)) + { + throw new InvalidImageContentException($"The {sourceName} mastering-display metadata does not match the image-item property."); + } + } + + /// + /// Determines whether AV1 and ISOBMFF mastering-display values agree within their fixed-point precision. + /// + /// The mastering-display values decoded from the AV1 representation. + /// The mastering-display values decoded from the ISOBMFF representation. + /// when all decoded values agree within their combined quantization error. + private static bool MasteringDisplayColorVolumesMatch( + HeifMasteringDisplayColorVolume obuColorVolume, + HeifMasteringDisplayColorVolume itemColorVolume) + { + const float chromaticityTolerance = ((1F / 65536F) + (1F / 50000F)) / 2F; + const double maximumLuminanceTolerance = ((1D / 256D) + (1D / 10000D)) / 2D; + const double minimumLuminanceTolerance = ((1D / 16384D) + (1D / 10000D)) / 2D; + + return ChromaticitiesMatch(obuColorVolume.Primaries.R, itemColorVolume.Primaries.R, chromaticityTolerance) + && ChromaticitiesMatch(obuColorVolume.Primaries.G, itemColorVolume.Primaries.G, chromaticityTolerance) + && ChromaticitiesMatch(obuColorVolume.Primaries.B, itemColorVolume.Primaries.B, chromaticityTolerance) + && ChromaticitiesMatch(obuColorVolume.WhitePoint, itemColorVolume.WhitePoint, chromaticityTolerance) + && ValuesMatch(obuColorVolume.MaximumLuminance, itemColorVolume.MaximumLuminance, maximumLuminanceTolerance) + && ValuesMatch(obuColorVolume.MinimumLuminance, itemColorVolume.MinimumLuminance, minimumLuminanceTolerance); + } + + /// + /// Determines whether two chromaticity-coordinate pairs agree within the supplied fixed-point tolerance. + /// + /// The first chromaticity-coordinate pair. + /// The second chromaticity-coordinate pair. + /// The maximum permitted difference on either coordinate axis. + /// when both coordinate differences are within the tolerance. + private static bool ChromaticitiesMatch( + CieXyChromaticityCoordinates left, + CieXyChromaticityCoordinates right, + float tolerance) + { + return ValuesMatch(left.X, right.X, tolerance) + && ValuesMatch(left.Y, right.Y, tolerance); + } + + /// + /// Determines whether two decoded fixed-point values agree within the supplied tolerance. + /// + /// The first decoded value. + /// The second decoded value. + /// The maximum permitted absolute difference. + /// when the absolute difference does not exceed the tolerance. + private static bool ValuesMatch(double left, double right, double tolerance) + => Math.Abs(left - right) <= tolerance; +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Av1ColorFormat.cs b/src/ImageSharp/Formats/Heif/Av1/Av1ColorFormat.cs new file mode 100644 index 0000000000..1ebef2924e --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Av1ColorFormat.cs @@ -0,0 +1,30 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1; + +/// +/// Identifies the AV1 luma and chroma plane sampling layout. +/// +internal enum Av1ColorFormat +{ + /// + /// Monochrome luma samples without chroma planes. + /// + Yuv400, + + /// + /// Chroma samples subsampled by two horizontally and vertically. + /// + Yuv420, + + /// + /// Chroma samples subsampled by two horizontally. + /// + Yuv422, + + /// + /// Full-resolution luma and chroma samples. + /// + Yuv444, +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Av1Constants.cs b/src/ImageSharp/Formats/Heif/Av1/Av1Constants.cs new file mode 100644 index 0000000000..b168122c3c --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Av1Constants.cs @@ -0,0 +1,353 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1; + +/// +/// Defines shared AV1 syntax, geometry, entropy, and transform limits. +/// +internal static class Av1Constants +{ + /// + /// The highest sequence profile defined by AV1. + /// + public const ObuSequenceProfile MaxSequenceProfile = ObuSequenceProfile.Professional; + + /// + /// The number of bits used for an operating-point level index. + /// + public const int LevelBits = 5; + + /// + /// The number of bits used for the zero-based operating-point count. + /// + public const int OperatingPointCountBits = 5; + + /// + /// The number of bits used for an operating-point layer-selection mask. + /// + public const int OperatingPointIdcBits = 12; + + /// + /// The number of bits used for a frame type. + /// + public const int FrameTypeBits = 2; + + /// + /// The first sequence-level index that carries an explicit tier bit. + /// + public const int SequenceTierMinimumLevelIndex = 8; + + /// + /// The number of bits used to select a frame from the eight-slot reference map. + /// + public const int ReferenceFrameIndexBits = 3; + + /// + /// The sequence-header value that lets each frame choose whether to use screen-content tools. + /// + public const int SelectScreenContentTools = 2; + + /// + /// The sequence-header value that lets each applicable frame choose whether to require integer motion vectors. + /// + public const int SelectIntegerMotionVector = 2; + + /// + /// The maximum number of operating points declared by one AV1 sequence header. + /// + public const int MaxOperatingPointCount = 32; + + /// + /// The maximum number of spatial layers identified by an AV1 OBU extension header. + /// + public const int MaxSpatialLayerCount = 4; + + /// + /// The number of bits used to signal a super-resolution denominator offset. + /// + public const int SuperResolutionScaleBits = 3; + + /// + /// The fixed numerator of the AV1 super-resolution scaling ratio. + /// + public const int ScaleNumerator = 8; + + /// + /// The number of reference frames that can be used for inter prediction. + /// + public const int ReferencesPerFrame = 7; + + /// + /// The largest frame width or height representable by the 16-bit AV1 dimension syntax. + /// + public const int MaxFrameDimension = 1 << 16; + + /// + /// The maximum area of a tile in units of luma samples. + /// + public const int MaxTileArea = 4096 * 2304; + + /// + /// The maximum width of a tile in units of luma samples. + /// + public const int MaxTileWidth = 4096; + + /// + /// The maximum number of tile columns. + /// + public const int MaxTileColumnCount = 64; + + /// + /// The maximum number of tile rows. + /// + public const int MaxTileRowCount = 64; + + /// + /// The number of 64x64 CDEF filter units in a 128x128 superblock. + /// + public const int CdefUnitsPerSuperblock = 4; + + /// + /// The number of frames that can be stored for future reference. + /// + public const int ReferenceFrameCount = 8; + + /// + /// The primary-reference-frame value indicating that no primary reference is selected. + /// + public const uint PrimaryReferenceFrameNone = 7; + + /// + /// The number of bits used to signal a primary reference frame. + /// + public const int PrimaryReferenceBits = 3; + + /// + /// The number of segments allowed in a segmentation map. + /// + public const int MaxSegmentCount = 8; + + /// + /// The smallest signaled denominator for an active super-resolution ratio. + /// + public const int SuperResolutionScaleDenominatorMinimum = 9; + + /// + /// The base-two logarithm of the maximum superblock size in luma samples. + /// + public const int MaxSuperBlockSizeLog2 = 7; + + /// + /// The base-two logarithm of the smallest mode-info block size in luma samples. + /// + public const int ModeInfoSizeLog2 = 2; + + /// + /// The maximum quantizer index. + /// + public const int MaxQ = 255; + + /// + /// The number of segmentation features. + /// + public const int SegmentationLevelMax = 8; + + /// + /// The maximum loop-restoration tile size in samples. + /// + public const int RestorationMaxTileSize = 256; + + /// + /// The number of independent Wiener filter coefficients per direction. + /// + public const int WienerCoefficientCount = 3; + + /// + /// The number of luma and chroma frame loop-filter levels. + /// + public const int FrameLoopFilterCount = 4; + + /// + /// The first quantizer-delta magnitude encoded through the escape path. + /// + public const int DeltaQuantizerSmall = 3; + + /// + /// The first loop-filter-delta magnitude encoded through the escape path. + /// + public const int DeltaLoopFilterSmall = 3; + + /// + /// The maximum loop-filter strength. + /// + public const int MaxLoopFilter = 63; + + /// + /// The maximum directional-prediction angle-delta magnitude. + /// + public const int MaxAngleDelta = 3; + + /// + /// The maximum number of color planes. + /// + public const int MaxPlanes = 3; + + /// + /// The number of reference-frame types, including the intra type. + /// + public const int TotalReferencesPerFrame = 8; + + /// + /// The maximum palette size. + /// + public const int PaletteMaxSize = 8; + + /// + /// The number of transform-size probability categories. + /// + public const int MaxTransformCategories = 4; + + /// + /// The number of cumulative coefficient-level magnitude contexts. + /// + public const int CoefficientContextCount = 6; + + /// + /// The number of coefficient magnitudes represented by base symbols before base-range coding. + /// + public const int BaseLevelsCount = 2; + + /// + /// The maximum coefficient magnitude increment represented by base-range symbols. + /// + public const int CoefficientBaseRange = 12; + + /// + /// The maximum transform dimension in samples. + /// + public const int MaxTransformSize = 1 << 6; + + /// + /// The maximum transform dimension in units of four samples. + /// + public const int MaxTransformSizeUnit = MaxTransformSize >> 2; + + /// + /// The number of low-order bits reserved for a cumulative coefficient-level context. + /// + public const int CoefficientContextBitCount = 3; + + /// + /// The mask selecting the cumulative coefficient-level magnitude bits. + /// + public const int CoefficientContextMask = (1 << CoefficientContextBitCount) - 1; + + /// + /// The base-two logarithm of the horizontal coefficient-context padding. + /// + public const int TransformPadHorizontalLog2 = 2; + + /// + /// The horizontal coefficient-context padding in elements. + /// + public const int TransformPadHorizontal = 1 << TransformPadHorizontalLog2; + + /// + /// The total vertical coefficient-context padding in rows. + /// + public const int TransformPadVertical = 6; + + /// + /// The trailing coefficient-context padding in elements. + /// + public const int TransformPadEnd = 16; + + /// + /// The maximum padded two-dimensional coefficient-context allocation size. + /// + public const int TransformPad2d = ((MaxTransformSize + TransformPadHorizontal) * (MaxTransformSize + TransformPadVertical)) + TransformPadEnd; + + /// + /// The coefficient-context padding above a transform. + /// + public const int TransformPadTop = 2; + + /// + /// The coefficient-context padding below a transform. + /// + public const int TransformPadBottom = 4; + + /// + /// The largest symbol in a coefficient base-range distribution. + /// + public const int BaseRangeSizeMinus1 = 3; + + /// + /// The largest coefficient magnitude represented before Golomb coding. + /// + public const int MaxBaseRange = 15; + + /// + /// The base-two logarithm of the chroma-from-luma alpha alphabet size. + /// + public const int ChromaFromLumaAlphabetSizeLog2 = 4; + + /// + /// The number of quantization-matrix levels. + /// + public const int QuantificationMatrixLevelCount = 1 << 4; + + /// + /// The fixed-point precision of each quantization-matrix element. + /// + public const int QuantizationMatrixElementBitCount = 5; + + /// + /// The directional intra-prediction angle increment in degrees. + /// + public const int AngleStep = 3; + + /// + /// The maximum number of stages in a one-dimensional transform function. + /// + public const int MaxTransformStageNumber = 12; + + /// + /// The number of partition contexts per block-size logarithm. + /// + public const int PartitionProbabilitySet = 4; + + /// + /// The number of square transform-size contexts that can signal extended transforms. + /// + public const int ExtendedTransformCount = 4; + + /// + /// The highest variable-transform depth index. + /// + public const int MaxVarTransform = 2; + + /// + /// Number of items in the enumeration. + /// + public const int PlaneTypeCount = 2; + + /// + /// Gets the number of payload bits used by each segmentation feature. + /// + public static ReadOnlySpan SegmentationFeatureBits => [8, 6, 6, 6, 6, 3, 0, 0]; + + /// + /// Gets values indicating whether each segmentation feature is signed. + /// + public static ReadOnlySpan SegmentationFeatureSigned => [1, 1, 1, 1, 1, 0, 0, 0]; + + /// + /// Gets the maximum magnitude or value permitted for each segmentation feature. + /// + public static ReadOnlySpan SegmentationFeatureMax => [MaxQ, MaxLoopFilter, MaxLoopFilter, MaxLoopFilter, MaxLoopFilter, 7, 0, 0]; +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Av1Decoder.cs b/src/ImageSharp/Formats/Heif/Av1/Av1Decoder.cs new file mode 100644 index 0000000000..152df4509f --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Av1Decoder.cs @@ -0,0 +1,1077 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Buffers; +using SixLabors.ImageSharp.Formats.Heif.Av1.Color; +using SixLabors.ImageSharp.Formats.Heif.Av1.Entropy; +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline; +using SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.FilmGrain; +using SixLabors.ImageSharp.Formats.Heif.Av1.ReferenceFrames; +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; +using SixLabors.ImageSharp.Memory; +using SixLabors.ImageSharp.Metadata; +using SixLabors.ImageSharp.Metadata.Profiles.Cicp; +using SixLabors.ImageSharp.PixelFormats; +using SixLabors.ImageSharp.PixelFormats.Utils; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1; + +/// +/// Decodes bounded AV1 image payloads and image-sequence samples into ImageSharp frames. +/// +internal sealed class Av1Decoder : IAv1TileReader, IDisposable +{ + /// + /// The open-bitstream-unit parser for the current image item. + /// + private readonly ObuReader obuReader; + + /// + /// The configuration used for decoded image and scratch-memory allocation. + /// + private readonly Configuration configuration; + + /// + /// Reusable luma palette indices for the coding blocks in one superblock. + /// + private readonly Buffer2D lumaPaletteColorIndexMap; + + /// + /// Reusable chroma palette indices for the coding blocks in one superblock. + /// + private readonly Buffer2D chromaPaletteColorIndexMap; + + /// + /// The shared backing owner for both reusable palette maps. + /// + private readonly IMemoryOwner paletteColorIndexMapOwner; + + /// + /// The reconstructed references and selected presentation output owned by the current bounded decode session. + /// + private readonly Av1ReferenceFrameStore referenceFrames = new(); + + /// + /// The frame-base, tile-working, and published entropy contexts created for the first coded frame and then reused + /// for this bounded decoder session. + /// + private Av1FrameEntropyContexts? entropyContexts; + + /// + /// The coded sequence governing the active reference map and reusable entropy session. + /// + private ObuSequenceHeader? entropySequenceHeader; + + /// + /// The item codec configuration validated before reconstructing a completed frame. + /// + private Av1CodecConfiguration? codecConfiguration; + + /// + /// The container color description applied before reconstructing a completed frame. + /// + private CicpProfile? containerColorProfile; + + /// + /// The sequence header already validated for the current bounded payload. + /// + private ObuSequenceHeader? validatedSequenceHeader; + + /// + /// The complete parser, sample buffer, and reconstruction state for the frame currently being decoded. + /// + private FrameDecodeState? frameDecodeState; + + /// + /// Initializes a new instance of the class. + /// + /// The configuration used for image and scratch-memory allocation. + public Av1Decoder(Configuration configuration) + : this(configuration, 0) + { + } + + /// + /// Initializes a new instance of the class for one selected AV1 operating point. + /// + /// The configuration used for image and scratch-memory allocation. + /// The zero-based sequence-header operating-point index to decode. + public Av1Decoder(Configuration configuration, byte operatingPointIndex) + { + this.configuration = configuration; + this.obuReader = new(operatingPointIndex, this.referenceFrames); + + // Sequential tile decoding needs only the palette indices belonging to the current superblock. One fixed + // owner keeps both maximum-superblock maps reusable across the bounded session without fragmented group rents. + int paletteMapLength = 1 << Av1Constants.MaxSuperBlockSizeLog2; + int paletteMapArea = paletteMapLength * paletteMapLength; + this.paletteColorIndexMapOwner = configuration.MemoryAllocator.Allocate(2 * paletteMapArea); + Memory paletteMaps = this.paletteColorIndexMapOwner.Memory; + this.lumaPaletteColorIndexMap = Buffer2D.WrapMemory( + paletteMaps[..paletteMapArea], + paletteMapLength, + paletteMapLength); + + this.chromaPaletteColorIndexMap = Buffer2D.WrapMemory( + paletteMaps[paletteMapArea..], + paletteMapLength, + paletteMapLength); + } + + /// + /// Gets the final retained shown-frame header, or before a shown frame completes. + /// + public ObuFrameHeader? FrameHeader { get; private set; } + + /// + /// Gets the sequence header governing the final retained shown frame, or before one completes. + /// + public ObuSequenceHeader? SequenceHeader { get; private set; } + + /// + /// Gets tile and superblock state for the most recently reconstructed frame, or when no + /// frame was reconstructed or the output selected an existing reference without new tile syntax. + /// + public Av1FrameInfo? FrameInfo { get; private set; } + + /// + /// Gets the inter-prediction features selected by every coded frame completed in the most recently decoded payload. + /// + public Av1InterPredictionFeatures DecodedInterPredictionFeatures { get; private set; } + + /// + /// Gets the native planes of the current retained shown frame, or before one completes. + /// + public Av1FrameBuffer? FrameBuffer => this.referenceFrames.OutputFrame?.FrameBuffer; + + /// + /// Decodes a bounded AV1 image payload and presents its final shown frame. + /// + /// The destination pixel type. + /// The complete AV1 elementary-stream payload. + /// + /// The container color description that supplies unspecified sequence-header color information. + /// + /// + /// The item-associated AV1 codec configuration validated against the coded sequence header. + /// + /// The optional byte boundaries of a layered AV1 image item. + /// The requested item presentation size, or an empty size for the coded dimensions. + /// The decoded image. + public Image Decode( + Span buffer, + CicpProfile? containerColorProfile = null, + Av1CodecConfiguration? codecConfiguration = null, + Av1LayeredImageIndex? layeredImageIndex = null, + Size presentationSize = default) + where TPixel : unmanaged, IPixel + { + ImageFrame frame = this.DecodeFrame( + buffer, + containerColorProfile, + codecConfiguration, + out CicpProfile effectiveColorProfile, + layeredImageIndex, + presentationSize); + + ImageMetadata metadata = new() + { + CicpProfile = effectiveColorProfile + }; + + HeifContentLightLevel? contentLightLevel = this.obuReader.ContentLightLevel; + HeifMasteringDisplayColorVolume? masteringDisplayColorVolume = this.obuReader.MasteringDisplayColorVolume; + if (contentLightLevel is not null || masteringDisplayColorVolume is not null) + { + HeifMetadata heifMetadata = metadata.GetHeifMetadata(); + heifMetadata.ContentLightLevel = contentLightLevel; + heifMetadata.MasteringDisplayColorVolume = masteringDisplayColorVolume; + } + + try + { + return new Image(this.configuration, metadata, [frame]); + } + catch + { + // Ownership transfers only after the image constructor accepts the decoded frame. + frame.Dispose(); + throw; + } + } + + /// + /// Decodes an AV1 elementary-stream payload into one independently owned ImageSharp frame. + /// + /// The destination pixel type. + /// The complete AV1 elementary-stream payload. + /// + /// The container color description that supplies unspecified sequence-header color information. + /// + /// + /// The AV1 codec configuration validated against the coded sequence header. + /// + /// Receives the effective CICP description used for conversion. + /// The optional byte boundaries of a layered AV1 image item. + /// The requested item presentation size, or an empty size for the coded dimensions. + /// The decoded frame. Ownership transfers to the caller. + public ImageFrame DecodeFrame( + Span buffer, + CicpProfile? containerColorProfile, + Av1CodecConfiguration? codecConfiguration, + out CicpProfile effectiveColorProfile, + Av1LayeredImageIndex? layeredImageIndex = null, + Size presentationSize = default) + where TPixel : unmanaged, IPixel + { + using Av1FrameBuffer frameBuffer = this.DecodeFrameBuffer( + buffer, + containerColorProfile, + codecConfiguration, + out effectiveColorProfile, + out ObuFrameHeader frameHeader, + layeredImageIndex); + + return this.ConvertToFrame(frameBuffer, frameHeader, effectiveColorProfile, presentationSize); + } + + /// + /// Decodes the next visible sample in a bounded AV1 image sequence while retaining its reference state. + /// + /// The destination pixel type. + /// The complete AV1 sample payload. + /// The container color description. + /// The AV1 sample-entry configuration. + /// The independently owned decoded frame. + public ImageFrame DecodeSequenceFrame( + Span buffer, + CicpProfile? containerColorProfile, + Av1CodecConfiguration? codecConfiguration) + where TPixel : unmanaged, IPixel + { + CicpProfile effectiveColorProfile = this.DecodePayload( + buffer, + containerColorProfile, + codecConfiguration, + null, + requireShownFrame: true); + + Av1ReferenceFrame outputFrame = this.referenceFrames.ResolveOutput(); + return this.ConvertToFrame(outputFrame.FrameBuffer, outputFrame.FrameHeader, effectiveColorProfile); + } + + /// + /// Decodes the next visible sample in a bounded AV1 image sequence directly into a caller-owned frame. + /// + /// The destination pixel type. + /// The complete AV1 sample payload. + /// The container color description. + /// The AV1 sample-entry configuration. + /// The coded dimensions declared by the visual sample entry. + /// The clean-aperture region mapped to the complete destination frame. + /// The caller-owned packed-pixel frame receiving the presented sample. + public void DecodeSequenceFrame( + Span buffer, + CicpProfile? containerColorProfile, + Av1CodecConfiguration? codecConfiguration, + Size expectedCodedSize, + Rectangle sourceRectangle, + ImageFrame destination) + where TPixel : unmanaged, IPixel + { + CicpProfile effectiveColorProfile = this.DecodePayload( + buffer, + containerColorProfile, + codecConfiguration, + null, + requireShownFrame: true); + + Av1ReferenceFrame outputFrame = this.referenceFrames.ResolveOutput(); + Size codedSize = new( + outputFrame.FrameHeader.FrameSize.SuperResolutionUpscaledWidth, + outputFrame.FrameHeader.FrameSize.FrameHeight); + + if (codedSize != expectedCodedSize) + { + throw new InvalidImageContentException( + "The decoded image-sequence sample dimensions do not match its visual sample entry."); + } + + Av1YuvConverter.ConvertRegionToRgb( + this.configuration, + outputFrame.FrameBuffer, + sourceRectangle, + destination); + + destination.Metadata.CicpProfile = effectiveColorProfile.DeepClone(); + } + + /// + /// Decodes one non-presented AV1 image-sequence sample while retaining its reference state. + /// + /// The complete AV1 sample payload. + /// The container color description. + /// The AV1 sample-entry configuration. + public void DecodeSequenceReference( + Span buffer, + CicpProfile? containerColorProfile, + Av1CodecConfiguration? codecConfiguration) + => _ = this.DecodePayload( + buffer, + containerColorProfile, + codecConfiguration, + null, + requireShownFrame: false); + + /// + /// Decodes the next visible monochrome AV1 sequence sample and composes it into a color frame. + /// + /// The destination color pixel type. + /// The complete AV1 sample payload. + /// The container color description. + /// The AV1 sample-entry configuration. + /// The required coded dimensions. + /// The clean-aperture luma region mapped to the destination. + /// The packed color frame receiving alpha values. + /// The complete presented size of the auxiliary image. + /// The destination region receiving the alpha image. + /// Whether stored color samples must be converted to unassociated alpha. + public void DecodeSequenceAlpha( + Span buffer, + CicpProfile? containerColorProfile, + Av1CodecConfiguration? codecConfiguration, + Size expectedCodedSize, + Rectangle sourceRectangle, + ImageFrame destination, + Size outputSize, + Rectangle destinationRectangle, + bool premultiplied) + where TPixel : unmanaged, IPixel + { + _ = this.DecodePayload( + buffer, + containerColorProfile, + codecConfiguration, + null, + requireShownFrame: true); + + Av1ReferenceFrame outputFrame = this.referenceFrames.ResolveOutput(); + this.ComposeAlpha( + outputFrame.FrameBuffer, + expectedCodedSize, + sourceRectangle, + destination, + outputSize, + destinationRectangle, + premultiplied); + } + + /// + /// Converts native AV1 planes into one independently owned packed-pixel frame. + /// + /// The destination pixel type. + /// The decoded native planes. + /// The completed header describing the decoded native planes. + /// The effective CICP description. + /// The requested item presentation size, or an empty size for the coded dimensions. + /// The independently owned packed-pixel frame. + private ImageFrame ConvertToFrame( + Av1FrameBuffer frameBuffer, + ObuFrameHeader frameHeader, + CicpProfile effectiveColorProfile, + Size presentationSize = default) + where TPixel : unmanaged, IPixel + { + ImageFrame? resultFrame = null; + try + { + Size codedSize = new( + frameHeader.FrameSize.SuperResolutionUpscaledWidth, + frameHeader.FrameSize.FrameHeight); + + // A selected lower spatial layer can only be scaled upward to the image item's ispe extent here. + // Other item-size corrections keep using the shared packed-pixel presentation path after decoding. + Size outputSize = presentationSize.Width >= codedSize.Width && presentationSize.Height >= codedSize.Height + ? presentationSize + : codedSize; + + resultFrame = new ImageFrame( + this.configuration, + outputSize.Width, + outputSize.Height); + + Av1YuvConverter.ConvertToRgb(this.configuration, frameBuffer, resultFrame); + resultFrame.Metadata.CicpProfile = effectiveColorProfile.DeepClone(); + return resultFrame; + } + catch + { + resultFrame?.Dispose(); + throw; + } + } + + /// + /// Decodes an AV1 elementary-stream payload and composes its luma plane directly into a packed color frame. + /// + /// The destination color pixel type. + /// The complete AV1 elementary-stream payload. + /// + /// The container color description that supplies unspecified sequence-header color information. + /// + /// The AV1 codec configuration validated against the coded sequence header. + /// The required coded dimensions, or an empty size when the item extent may differ. + /// The packed color frame receiving alpha values. + /// The complete presented size of the auxiliary image or grid tile. + /// The destination region receiving the top-left portion of the presented alpha image. + /// Whether stored color samples must be converted to unassociated alpha. + /// The optional byte boundaries of a layered AV1 image item. + public void DecodeAlpha( + Span buffer, + CicpProfile? containerColorProfile, + Av1CodecConfiguration? codecConfiguration, + Size expectedCodedSize, + ImageFrame destination, + Size outputSize, + Rectangle destinationRectangle, + bool premultiplied, + Av1LayeredImageIndex? layeredImageIndex = null) + where TPixel : unmanaged, IPixel + { + using Av1FrameBuffer frameBuffer = this.DecodeFrameBuffer( + buffer, + containerColorProfile, + codecConfiguration, + out _, + layeredImageIndex); + + this.ComposeAlpha( + frameBuffer, + expectedCodedSize, + destination, + outputSize, + destinationRectangle, + premultiplied); + } + + /// + /// Composes one decoded monochrome plane into a packed color frame. + /// + /// The destination color pixel type. + /// The decoded monochrome planes. + /// The required coded dimensions. + /// The packed color frame receiving alpha values. + /// The complete presented size of the auxiliary image. + /// The destination region receiving the alpha image. + /// Whether stored color samples must be converted to unassociated alpha. + private void ComposeAlpha( + Av1FrameBuffer frameBuffer, + Size expectedCodedSize, + ImageFrame destination, + Size outputSize, + Rectangle destinationRectangle, + bool premultiplied) + where TPixel : unmanaged, IPixel + => this.ComposeAlpha( + frameBuffer, + expectedCodedSize, + new Rectangle(0, 0, frameBuffer.Width, frameBuffer.Height), + destination, + outputSize, + destinationRectangle, + premultiplied); + + /// + /// Composes one decoded monochrome region into a packed color frame. + /// + /// The destination color pixel type. + /// The decoded monochrome planes. + /// The required coded dimensions. + /// The luma region mapped to the destination rectangle. + /// The packed color frame receiving alpha values. + /// The complete presented size of the auxiliary image. + /// The destination region receiving the alpha image. + /// Whether stored color samples must be converted to unassociated alpha. + private void ComposeAlpha( + Av1FrameBuffer frameBuffer, + Size expectedCodedSize, + Rectangle sourceRectangle, + ImageFrame destination, + Size outputSize, + Rectangle destinationRectangle, + bool premultiplied) + where TPixel : unmanaged, IPixel + { + if (expectedCodedSize != default && (frameBuffer.Width != expectedCodedSize.Width || frameBuffer.Height != expectedCodedSize.Height)) + { + throw new InvalidImageContentException("The decoded alpha sample dimensions do not match its visual sample entry."); + } + + if (frameBuffer.ColorFormat != Av1ColorFormat.Yuv400) + { + // AVIF auxiliary alpha is the luma plane of an AV1 monochrome image. Accepting chroma-bearing payloads + // would silently reinterpret a color image and contradict the Sequence Header mono_chrome requirement. + throw new InvalidImageContentException("An AV1 auxiliary alpha image must be encoded as monochrome."); + } + + Av1YuvConverter.ComposeAlpha( + this.configuration, + frameBuffer, + sourceRectangle, + destination, + outputSize, + destinationRectangle, + premultiplied); + } + + /// + /// Parses every coded frame in an AV1 payload and returns the final shown frame's native component planes. + /// + /// The complete AV1 elementary-stream payload. + /// + /// The container color description that supplies unspecified sequence-header color information. + /// + /// The AV1 codec configuration validated against the coded sequence header. + /// Receives the effective CICP description associated with the native planes. + /// The optional byte boundaries of a layered AV1 image item. + /// The reconstructed native frame buffer. Ownership transfers to the caller. + public Av1FrameBuffer DecodeFrameBuffer( + Span buffer, + CicpProfile? containerColorProfile, + Av1CodecConfiguration? codecConfiguration, + out CicpProfile effectiveColorProfile, + Av1LayeredImageIndex? layeredImageIndex = null) + => this.DecodeFrameBuffer( + buffer, + containerColorProfile, + codecConfiguration, + out effectiveColorProfile, + out _, + layeredImageIndex); + + /// + /// Parses every coded frame in an AV1 payload and returns the final shown frame's native planes and header. + /// + private Av1FrameBuffer DecodeFrameBuffer( + Span buffer, + CicpProfile? containerColorProfile, + Av1CodecConfiguration? codecConfiguration, + out CicpProfile effectiveColorProfile, + out ObuFrameHeader frameHeader, + Av1LayeredImageIndex? layeredImageIndex) + { + effectiveColorProfile = this.DecodePayload( + buffer, + containerColorProfile, + codecConfiguration, + layeredImageIndex, + requireShownFrame: true); + + using Av1ReferenceFrame outputFrame = this.referenceFrames.TakeOutput(); + frameHeader = outputFrame.FrameHeader; + return outputFrame.TakeFrameBuffer(); + } + + /// + /// Parses one bounded payload into the retained decoder session. + /// + /// The complete AV1 payload. + /// The container color description. + /// The AV1 codec configuration. + /// The optional layer byte boundaries. + /// Whether the payload must produce a shown frame. + /// The effective CICP description. + private CicpProfile DecodePayload( + Span buffer, + CicpProfile? containerColorProfile, + Av1CodecConfiguration? codecConfiguration, + Av1LayeredImageIndex? layeredImageIndex, + bool requireShownFrame) + { + this.codecConfiguration = codecConfiguration; + this.containerColorProfile = containerColorProfile; + this.validatedSequenceHeader = null; + this.obuReader.ResetMetadata(); + this.SequenceHeader = null; + this.FrameHeader = null; + this.DecodedInterPredictionFeatures = Av1InterPredictionFeatures.None; + + // Full tile syntax describes only frames reconstructed by this payload. Reference slots already own the compact + // state needed by later frames, so release the previous payload's reconstruction graph before parsing the next. + this.FrameInfo?.ReleaseOwner(); + this.FrameInfo = null; + + try + { + if (layeredImageIndex is null) + { + Av1BitStreamReader reader = new(buffer); + this.obuReader.ReadAll(ref reader, buffer.Length, this, false); + } + else + { + int layerOffset = 0; + for (int layer = 0; layer < Av1Constants.MaxSpatialLayerCount - 1 && layerOffset < buffer.Length; layer++) + { + uint declaredLayerSize = layer switch + { + 0 => layeredImageIndex.Value.FirstLayerSize, + 1 => layeredImageIndex.Value.SecondLayerSize, + _ => layeredImageIndex.Value.ThirdLayerSize + }; + + if (declaredLayerSize == 0) + { + break; + } + + int layerSize = (int)declaredLayerSize; + Av1BitStreamReader layerReader = new(buffer.Slice(layerOffset, layerSize)); + this.obuReader.ReadAll(ref layerReader, layerSize, this, false); + layerOffset += layerSize; + } + + if (layerOffset < buffer.Length) + { + Span finalLayer = buffer[layerOffset..]; + Av1BitStreamReader finalLayerReader = new(finalLayer); + this.obuReader.ReadAll(ref finalLayerReader, finalLayer.Length, this, false); + } + } + + ObuSequenceHeader sequenceHeader = this.obuReader.SequenceHeader + ?? throw new InvalidImageContentException("The AV1 payload contains no sequence header."); + + if (requireShownFrame) + { + _ = this.referenceFrames.ResolveOutput(); + } + + // Preserve the effective CICP description used for conversion, including container values that legally + // supplied unspecified bitstream fields. This also exposes bitstream-only color metadata to callers. + ObuColorConfig effectiveColorConfig = sequenceHeader.ColorConfig; + return new CicpProfile( + (byte)effectiveColorConfig.ColorPrimaries, + (byte)effectiveColorConfig.TransferCharacteristics, + (byte)effectiveColorConfig.MatrixCoefficients, + effectiveColorConfig.ColorRange); + } + catch + { + // A failed frame may own pooled neighbor contexts while earlier layers own reconstructed references and + // published CDF snapshots. None can be reused after a non-transactional frame transition has failed. + this.frameDecodeState?.Dispose(); + this.frameDecodeState = null; + this.obuReader.Reset(); + this.entropyContexts?.Reset(); + this.entropySequenceHeader = null; + this.SequenceHeader = null; + this.FrameHeader = null; + this.DecodedInterPredictionFeatures = Av1InterPredictionFeatures.None; + this.FrameInfo?.ReleaseOwner(); + this.FrameInfo = null; + throw; + } + finally + { + // Validation inputs belong to this bounded decode call. Completed native buffers retain no references to + // either description, so releasing them here prevents a reused decoder from observing stale item state. + this.codecConfiguration = null; + this.containerColorProfile = null; + this.validatedSequenceHeader = null; + } + } + + /// + /// Validates the active sequence against its container declarations before reconstruction begins. + /// + /// The active sequence header. + private void ValidateSequence(ObuSequenceHeader sequenceHeader) + { + if (ReferenceEquals(this.validatedSequenceHeader, sequenceHeader)) + { + return; + } + + Av1FrameBuffer.ValidateDimensions( + sequenceHeader, + sequenceHeader.ColorConfig.GetColorFormat(), + false); + + this.codecConfiguration?.Validate(sequenceHeader); + CicpProfile? colorProfile = this.containerColorProfile; + + if (colorProfile is not null) + { + ObuColorConfig colorConfig = sequenceHeader.ColorConfig; + ObuColorPrimaries containerColorPrimaries = (ObuColorPrimaries)colorProfile.ColorPrimaries; + ObuTransferCharacteristics containerTransferCharacteristics = (ObuTransferCharacteristics)colorProfile.TransferCharacteristics; + ObuMatrixCoefficients containerMatrixCoefficients = (ObuMatrixCoefficients)colorProfile.MatrixCoefficients; + + // AV1-ISOBMFF permits nclx to supply only bitstream fields explicitly coded as unspecified. A different + // specified value is a conformance error rather than a container-level color override. + if (colorConfig.ColorPrimaries == ObuColorPrimaries.Unspecified) + { + colorConfig.ColorPrimaries = containerColorPrimaries; + } + else if (colorConfig.ColorPrimaries != containerColorPrimaries) + { + throw new InvalidImageContentException("The HEIF CICP color primaries do not match the AV1 sequence header."); + } + + if (colorConfig.TransferCharacteristics == ObuTransferCharacteristics.Unspecified) + { + colorConfig.TransferCharacteristics = containerTransferCharacteristics; + } + else if (colorConfig.TransferCharacteristics != containerTransferCharacteristics) + { + throw new InvalidImageContentException("The HEIF CICP transfer characteristics do not match the AV1 sequence header."); + } + + if (colorConfig.MatrixCoefficients == ObuMatrixCoefficients.Unspecified) + { + colorConfig.MatrixCoefficients = containerMatrixCoefficients; + } + else if (colorConfig.MatrixCoefficients != containerMatrixCoefficients) + { + throw new InvalidImageContentException("The HEIF CICP matrix coefficients do not match the AV1 sequence header."); + } + + if (colorConfig.ColorRange != colorProfile.FullRange) + { + throw new InvalidImageContentException("The HEIF CICP color range does not match the AV1 sequence header."); + } + } + + // The same sequence header governs subsequent layered frames until another header OBU replaces it. + this.validatedSequenceHeader = sequenceHeader; + } + + /// + /// Parses one entropy-coded tile payload into the current frame state. + /// + /// The entropy-coded tile payload. + /// The raster-order tile index. + public void ReadTile(Span tileData, int tileNum) + { + FrameDecodeState frameDecodeState; + if (this.frameDecodeState is null) + { + ObuSequenceHeader sequenceHeader = this.obuReader.CurrentSequenceHeader; + ObuFrameHeader frameHeader = this.obuReader.CurrentFrameHeader; + this.ValidateSequence(sequenceHeader); + + if (!ReferenceEquals(this.entropySequenceHeader, sequenceHeader)) + { + if (this.entropySequenceHeader is not null) + { + // A coded-sequence boundary invalidates both sample references and their retained CDF snapshots. + // Returned snapshot graphs stay decoder-local and can be overwritten for the new sequence. + this.referenceFrames.Reset(); + this.entropyContexts?.Reset(); + } + + this.entropySequenceHeader = sequenceHeader; + } + + Av1FrameEntropyContext? primaryReferenceContext = null; + byte? primaryReferenceSlot = frameHeader.PrimaryReferenceSlot; + if (primaryReferenceSlot is not null) + { + // The uncompressed-header parser validates slot occupancy. Entropy ownership is checked here because + // only the reconstructed frame owner knows whether that slot retained a completed CDF snapshot. + Av1ReferenceFrame? primaryReference = this.referenceFrames.Resolve(primaryReferenceSlot.Value); + if (primaryReference is null || primaryReference.EntropyContext is null) + { + throw new InvalidImageContentException("The AV1 primary reference has no retained entropy context."); + } + + primaryReferenceContext = primaryReference.EntropyContext; + } + + // Every tile group in a frame contributes to the same mode-info and coefficient state. + Av1FrameEntropyContexts entropyContexts = + this.entropyContexts ??= new(frameHeader.QuantizationParameters.BaseQIndex); + + Av1TileReader? tileReader = null; + Av1FrameBuffer? frameBuffer = null; + + // Presentation-only samples contain no new tile syntax, so they keep the most recently reconstructed + // frame state. Release that state only when a new reconstruction begins to avoid overlapping two graphs. + this.FrameInfo?.ReleaseOwner(); + this.FrameInfo = null; + + try + { + tileReader = new Av1TileReader( + this.configuration, + sequenceHeader, + frameHeader, + entropyContexts, + primaryReferenceContext, + this.referenceFrames, + this.lumaPaletteColorIndexMap, + this.chromaPaletteColorIndexMap); + + frameBuffer = new Av1FrameBuffer( + this.configuration, + sequenceHeader, + sequenceHeader.ColorConfig.GetColorFormat(), + false, + frameHeader.FrameSize.SuperResolutionUpscaledWidth, + frameHeader.FrameSize.FrameHeight) + { + Width = frameHeader.FrameSize.FrameWidth, + Height = frameHeader.FrameSize.FrameHeight + }; + + Av1FrameDecoder frameDecoder = new( + sequenceHeader, + frameHeader, + tileReader.FrameInfo, + frameBuffer, + this.referenceFrames, + new Av1TileReader.PaletteColorIndexMaps( + this.lumaPaletteColorIndexMap, + this.chromaPaletteColorIndexMap)); + + tileReader.FrameDecoder = frameDecoder; + frameDecodeState = new(tileReader, frameBuffer, frameDecoder); + this.frameDecodeState = frameDecodeState; + } + catch + { + frameBuffer?.Dispose(); + tileReader?.Dispose(); + throw; + } + } + else + { + frameDecodeState = this.frameDecodeState.Value; + } + + frameDecodeState.TileReader.ReadTile(tileData, tileNum); + } + + /// + /// Reconstructs a frame after all of its tile payloads have been parsed. + /// + public void CompleteFrame() + { + ObuSequenceHeader sequenceHeader = this.obuReader.SequenceHeader + ?? throw new InvalidImageContentException("An AV1 frame cannot complete before its sequence header."); + + ObuFrameHeader frameHeader = this.obuReader.FrameHeader + ?? throw new InvalidImageContentException("An AV1 frame cannot complete before its frame header."); + + Av1FrameBuffer? frameBuffer = null; + Av1FrameDecoder? frameDecoder = null; + Av1TileReader? tileReader = null; + Av1FrameBuffer? presentationBuffer = null; + + try + { + this.ValidateSequence(sequenceHeader); + + if (frameHeader.ShowExistingFrame) + { + Av1ReferenceFrame existingFrame = this.referenceFrames.ShowExisting((int)frameHeader.FrameToShowMapIdx); + ObuFrameHeader existingFrameHeader = existingFrame.FrameHeader; + + if (existingFrameHeader.FrameType == ObuFrameType.KeyFrame) + { + // Both the decoder working context and the context retained by the newly aliased key frame reset + // frame. Later primary-reference selection must therefore observe normative defaults. + existingFrame.ResetEntropyContext(); + this.entropyContexts?.Reset(); + } + + if (existingFrameHeader.FilmGrainParameters.ApplyGrain) + { + presentationBuffer = new Av1FrameBuffer( + this.configuration, + sequenceHeader, + existingFrame.FrameBuffer.ColorFormat, + false, + existingFrame.FrameBuffer.MaxWidth, + existingFrame.FrameBuffer.MaxHeight); + + // Retained reference samples remain ungrained. Existing-frame presentation receives its own + // allocator-owned copy only when the inherited film-grain parameters actually modify the output. + existingFrame.FrameBuffer.CopyVisibleTo(presentationBuffer); + Av1FilmGrainDecoder filmGrainDecoder = new(sequenceHeader, existingFrameHeader, presentationBuffer); + filmGrainDecoder.DecodeFrame(); + + Av1ReferenceFrame presentationFrame = new(presentationBuffer, existingFrameHeader); + presentationBuffer = null; + this.referenceFrames.CommitOutput(presentationFrame); + } + + this.SequenceHeader = sequenceHeader; + this.FrameHeader = existingFrameHeader; + return; + } + + FrameDecodeState? activeFrameDecodeState = this.frameDecodeState; + if (activeFrameDecodeState is null) + { + throw new InvalidImageContentException("The AV1 frame completed without tile syntax."); + } + + this.frameDecodeState = null; + FrameDecodeState activeFrame = activeFrameDecodeState.Value; + frameBuffer = activeFrame.FrameBuffer; + frameDecoder = activeFrame.FrameDecoder; + tileReader = activeFrame.TileReader; + + Av1FrameInfo frameInfo = tileReader.FrameInfo; + Av1FrameBuffer reconstructedFrameBuffer = frameBuffer; + frameDecoder.CompleteFrame(); + + bool retainsReference = (frameHeader.RefreshFrameFlags & byte.MaxValue) != 0; + if (retainsReference) + { + // Motion compensation may address any clamped position inside the decoder border. Extending once after + // all in-loop filters lets every later block use the full padded span without per-prediction edge copies. + Av1ReferenceFrameBorder.Extend(reconstructedFrameBuffer); + + // Detach only the state libaom retains on RefCntBuffer before any later ownership transfer can fail. + // The full reconstruction graph remains local to the current result and expires independently. + frameInfo.PrepareReferenceState(); + } + + bool needsSeparatePresentation = frameHeader.ShowFrame && frameHeader.FilmGrainParameters.ApplyGrain && retainsReference; + if (needsSeparatePresentation) + { + presentationBuffer = new Av1FrameBuffer( + this.configuration, + sequenceHeader, + reconstructedFrameBuffer.ColorFormat, + false, + reconstructedFrameBuffer.MaxWidth, + reconstructedFrameBuffer.MaxHeight); + + // Film grain must never contaminate a decoded reference. A shown frame that is also refreshed therefore + // receives one allocator-owned presentation copy; frames with no reference role are grained in place. + reconstructedFrameBuffer.CopyVisibleTo(presentationBuffer); + } + + Av1FrameBuffer grainTarget = presentationBuffer ?? reconstructedFrameBuffer; + if (frameHeader.ShowFrame && frameHeader.FilmGrainParameters.ApplyGrain) + { + Av1FilmGrainDecoder filmGrainDecoder = new(sequenceHeader, frameHeader, grainTarget); + filmGrainDecoder.DecodeFrame(); + } + + Av1ReferenceFrame referenceFrame; + if (retainsReference) + { + Av1FrameEntropyContexts entropyContexts = tileReader.EntropyContexts; + + Av1FrameEntropyContext entropySnapshot = entropyContexts.RentPublishedSnapshot(); + try + { + referenceFrame = new(reconstructedFrameBuffer, frameHeader, frameInfo, entropySnapshot, entropyContexts); + } + catch + { + // The snapshot rent precedes the reference owner. Return it if object construction cannot accept it. + entropyContexts.ReturnSnapshot(entropySnapshot); + throw; + } + } + else + { + // Presentation-only frames can never become primary references, so they own no unused CDF graph. + referenceFrame = new(reconstructedFrameBuffer, frameHeader); + } + + frameBuffer = null; + + if (!this.referenceFrames.Commit(frameHeader.RefreshFrameFlags, referenceFrame, frameHeader.ShowFrame && !needsSeparatePresentation)) + { + referenceFrame.Dispose(); + } + + if (presentationBuffer is not null) + { + Av1ReferenceFrame presentationFrame = new(presentationBuffer, frameHeader); + presentationBuffer = null; + this.referenceFrames.CommitOutput(presentationFrame); + } + + this.SequenceHeader = sequenceHeader; + this.FrameHeader = frameHeader; + this.DecodedInterPredictionFeatures |= frameInfo.InterPredictionFeatures; + + // Hidden frames can contain the inter syntax needed to validate a sequence. Retain only the latest full + // reconstruction state until the next bounded decode; reference-map entries keep their compact state. + frameInfo.AddOwner(); + this.FrameInfo?.ReleaseOwner(); + this.FrameInfo = frameInfo; + } + finally + { + // A non-shown frame or failed reconstruction never escapes this callback. The tile reader releases the + // reconstruction lease; a retained frame keeps only its compact reference state after neighbor contexts + // and the remaining frame-sized syntax are returned. + frameDecoder?.Dispose(); + presentationBuffer?.Dispose(); + frameBuffer?.Dispose(); + tileReader?.Dispose(); + this.frameDecodeState?.Dispose(); + this.frameDecodeState = null; + } + } + + /// + /// Releases the current tile parser, reference map, and retained presentation output. + /// + public void Dispose() + { + this.frameDecodeState?.Dispose(); + this.frameDecodeState = null; + this.referenceFrames.Dispose(); + this.FrameInfo?.ReleaseOwner(); + this.FrameInfo = null; + this.lumaPaletteColorIndexMap.Dispose(); + this.chromaPaletteColorIndexMap.Dispose(); + this.paletteColorIndexMapOwner.Dispose(); + } + + /// + /// Carries the active frame resources as one valid state so no partially initialized combination can be observed. + /// + private readonly struct FrameDecodeState( + Av1TileReader tileReader, + Av1FrameBuffer frameBuffer, + Av1FrameDecoder frameDecoder) : IDisposable + { + /// + /// Gets the tile parser shared by all tile groups in the frame. + /// + public Av1TileReader TileReader { get; } = tileReader; + + /// + /// Gets the destination sample buffer reconstructed by the frame pipeline. + /// + public Av1FrameBuffer FrameBuffer { get; } = frameBuffer; + + /// + /// Gets the reconstruction pipeline for the frame. + /// + public Av1FrameDecoder FrameDecoder { get; } = frameDecoder; + + /// + /// Releases every resource when ownership has not transferred to a completed frame. + /// + public void Dispose() + { + this.FrameDecoder.Dispose(); + this.FrameBuffer.Dispose(); + this.TileReader.Dispose(); + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Av1FrameBuffer.cs b/src/ImageSharp/Formats/Heif/Av1/Av1FrameBuffer.cs new file mode 100644 index 0000000000..1a8c494db7 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Av1FrameBuffer.cs @@ -0,0 +1,649 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Buffers; +using System.Runtime.CompilerServices; +using System.Runtime.InteropServices; +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Memory; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1; + +/// +/// Owns the padded luma and chroma sample planes for one decoded AV1 frame. +/// +/// The unmanaged storage-element type used by the plane allocations. +internal sealed class Av1FrameBuffer : IDisposable + where T : unmanaged +{ + /// + /// The number of luma border samples reserved for prediction and in-loop filtering. + /// + // Scaled prediction may start 284 luma samples outside a retained frame and then consume three preceding filter + // taps. The normative 288-sample border keeps that entire source window directly addressable without block copies. + public const int DecoderPaddingValue = 288; + + /// + /// The number of elements occupied by one logical sample. + /// + private readonly int storageElementsPerSample; + + /// + /// The complete plane ownership state, or after disposal. + /// + private FramePlanes? planes; + + /// + /// Initializes a new instance of the class at sequence-maximum dimensions. + /// + /// The configuration providing the plane allocator. + /// The sequence header defining maximum dimensions, bit depth, and chroma layout. + /// The maximum color format to allocate for a non-monochrome sequence. + /// Indicates whether reconstruction uses native 16-bit sample storage. + /// The padded frame planes cannot be represented as contiguous allocations. + public Av1FrameBuffer(Configuration configuration, ObuSequenceHeader sequenceHeader, Av1ColorFormat maxColorFormat, bool is16BitPipeline) + : this( + configuration, + sequenceHeader, + maxColorFormat, + is16BitPipeline, + sequenceHeader.MaxFrameWidth, + sequenceHeader.MaxFrameHeight) + { + } + + /// + /// Initializes a new instance of the class for one active frame allocation. + /// + /// The configuration providing the plane allocator. + /// The sequence header defining bit depth and chroma layout. + /// The color format to allocate for a non-monochrome sequence. + /// Indicates whether reconstruction uses native 16-bit sample storage. + /// The padded plane's active luma width before decoder borders. + /// The padded plane's active luma height before decoder borders. + /// The padded frame planes cannot be represented as contiguous allocations. + public Av1FrameBuffer( + Configuration configuration, + ObuSequenceHeader sequenceHeader, + Av1ColorFormat maxColorFormat, + bool is16BitPipeline, + int allocationWidth, + int allocationHeight) + { + ValidateDimensions(sequenceHeader, maxColorFormat, is16BitPipeline); + + this.MemoryAllocator = configuration.MemoryAllocator; + Av1ColorFormat colorFormat = sequenceHeader.ColorConfig.IsMonochrome ? Av1ColorFormat.Yuv400 : maxColorFormat; + this.MaxWidth = allocationWidth; + this.MaxHeight = allocationHeight; + this.BitDepth = sequenceHeader.ColorConfig.BitDepth; + this.ColorConfig = sequenceHeader.ColorConfig; + this.BytesPerSample = this.BitDepth > Av1BitDepth.EightBit || is16BitPipeline ? 2 : 1; + this.storageElementsPerSample = Math.Max( + (this.BytesPerSample + Unsafe.SizeOf() - 1) / Unsafe.SizeOf(), + 1); + + this.ColorFormat = colorFormat; + this.Is16BitPipeline = is16BitPipeline; + this.StartPosition = new Point(DecoderPaddingValue, DecoderPaddingValue); + + this.Width = this.MaxWidth; + this.Height = this.MaxHeight; + this.OriginX = DecoderPaddingValue; + this.OriginY = DecoderPaddingValue; + + FrameBufferLayout layout = CreateFrameBufferLayout( + allocationWidth, + allocationHeight, + colorFormat, + this.storageElementsPerSample); + + // Libaom stores Y, U, and V in one aligned frame allocation. Non-owning Buffer2D views retain ImageSharp's + // row API without introducing separate plane rents or constructor rollback paths. + IMemoryOwner owner = configuration.MemoryAllocator.Allocate(layout.StorageLength); + Memory storage = owner.Memory; + Buffer2D luma = Buffer2D.WrapMemory( + storage.Slice(0, layout.LumaElementCount), + layout.LumaStorageWidth, + layout.LumaHeight); + + ChromaPlanes? chroma = null; + if (!sequenceHeader.ColorConfig.IsMonochrome) + { + Buffer2D chromaBlue = Buffer2D.WrapMemory( + storage.Slice(layout.ChromaBlueOffset, layout.ChromaElementCount), + layout.ChromaStorageWidth, + layout.ChromaHeight); + + Buffer2D chromaRed = Buffer2D.WrapMemory( + storage.Slice(layout.ChromaRedOffset, layout.ChromaElementCount), + layout.ChromaStorageWidth, + layout.ChromaHeight); + + chroma = new ChromaPlanes(chromaBlue, chromaRed); + } + + this.planes = new(owner, luma, chroma); + } + + /// + /// Gets the padded luma-coordinate origin of the visible frame. + /// + public Point StartPosition { get; private set; } + + /// + /// Gets the Y luma buffer. + /// + public Buffer2D? BufferY => this.planes?.Luma; + + /// + /// Gets the U chroma buffer. + /// + public Buffer2D? BufferCb => this.planes?.Chroma?.Blue; + + /// + /// Gets the V chroma buffer. + /// + public Buffer2D? BufferCr => this.planes?.Chroma?.Red; + + /// + /// Gets or sets the horizontal padding distance. + /// + public int OriginX { get; set; } + + /// + /// Gets or sets the vertical padding distance. + /// + public int OriginY { get; set; } + + /// + /// Gets or sets the luma picture width, excluding padding. + /// + public int Width { get; set; } + + /// + /// Gets or sets the luma picture height, excluding padding. + /// + public int Height { get; set; } + + /// + /// Gets or sets the maximum luma picture width. + /// + public int MaxWidth { get; set; } + + /// + /// Gets or sets the pixel bit depth. + /// + public Av1BitDepth BitDepth { get; set; } + + /// + /// Gets the number of bytes used to store each reconstructed sample. + /// + public int BytesPerSample { get; } + + /// + /// Gets the color configuration signaled by the AV1 sequence header. + /// + public ObuColorConfig ColorConfig { get; } + + /// + /// Gets or sets the luma and chroma plane sampling layout. + /// + public Av1ColorFormat ColorFormat { get; set; } + + /// + /// Gets or sets the maximum luma picture height. + /// + public int MaxHeight { get; set; } + + /// + /// Gets a value indicating whether reconstruction uses native 16-bit samples. + /// + public bool Is16BitPipeline { get; } + + /// + /// Gets the allocator used for frame-owned and frame-scoped working buffers. + /// + public MemoryAllocator MemoryAllocator { get; } + + /// + /// Validates that the maximum sequence planes fit the decoder's contiguous ownership contract. + /// + /// The sequence header defining maximum dimensions, bit depth, and chroma layout. + /// The maximum color format required by the sequence. + /// Indicates whether reconstruction uses native 16-bit sample storage. + public static void ValidateDimensions( + ObuSequenceHeader sequenceHeader, + Av1ColorFormat maxColorFormat, + bool is16BitPipeline) + { + int bytesPerSample = sequenceHeader.ColorConfig.BitDepth > Av1BitDepth.EightBit || is16BitPipeline ? 2 : 1; + int storageElementsPerSample = Math.Max( + (bytesPerSample + Unsafe.SizeOf() - 1) / Unsafe.SizeOf(), + 1); + + Av1ColorFormat colorFormat = sequenceHeader.ColorConfig.IsMonochrome ? Av1ColorFormat.Yuv400 : maxColorFormat; + _ = CreateFrameBufferLayout( + sequenceHeader.MaxFrameWidth, + sequenceHeader.MaxFrameHeight, + colorFormat, + storageElementsPerSample); + } + + /// + /// Gets the padded storage allocation for one component plane. + /// + /// The requested component plane. + /// The requested plane allocation. + public Buffer2D GetPlaneBuffer(Av1Plane plane) + { + this.GetPlaneLayout(plane, 0, 0, out Buffer2D buffer, out _, out _, out _, out _); + return buffer; + } + + /// + /// Copies the visible sample planes and active picture geometry to another compatible frame buffer. + /// + /// The frame buffer receiving the copied reconstruction. + public void CopyVisibleTo(Av1FrameBuffer destination) + { + destination.StartPosition = this.StartPosition; + destination.OriginX = this.OriginX; + destination.OriginY = this.OriginY; + destination.Width = this.Width; + destination.Height = this.Height; + destination.MaxWidth = this.MaxWidth; + destination.MaxHeight = this.MaxHeight; + destination.BitDepth = this.BitDepth; + destination.ColorFormat = this.ColorFormat; + + int planeCount = this.ColorFormat == Av1ColorFormat.Yuv400 ? 1 : 3; + for (int planeIndex = 0; planeIndex < planeCount; planeIndex++) + { + Av1Plane plane = (Av1Plane)planeIndex; + int subX = plane != Av1Plane.Y && this.ColorFormat is Av1ColorFormat.Yuv420 or Av1ColorFormat.Yuv422 ? 1 : 0; + int subY = plane != Av1Plane.Y && this.ColorFormat == Av1ColorFormat.Yuv420 ? 1 : 0; + + this.GetPlaneLayout( + plane, + subX, + subY, + out Buffer2D sourceBuffer, + out int sourceOriginX, + out int sourceOriginY, + out int width, + out int height); + + destination.GetPlaneLayout( + plane, + subX, + subY, + out Buffer2D destinationBuffer, + out int destinationOriginX, + out int destinationOriginY, + out _, + out _); + + int storageWidth = width * this.storageElementsPerSample; + int sourceStorageX = sourceOriginX * this.storageElementsPerSample; + int destinationStorageX = destinationOriginX * this.storageElementsPerSample; + + // A film-grain presentation owns only the active picture. Grain synthesis creates its odd-edge + // extension before reading it, so copying reference borders or unused sequence-sized storage is waste. + for (int row = 0; row < height; row++) + { + sourceBuffer.DangerousGetRowSpan(sourceOriginY + row) + .Slice(sourceStorageX, storageWidth) + .CopyTo(destinationBuffer.DangerousGetRowSpan(destinationOriginY + row).Slice(destinationStorageX, storageWidth)); + } + } + } + + /// + /// Releases the owned luma and chroma plane allocations. + /// + public void Dispose() + { + FramePlanes? ownedPlanes = this.planes; + this.planes = null; + if (ownedPlanes is null) + { + return; + } + + FramePlanes activePlanes = ownedPlanes.Value; + activePlanes.Luma.Dispose(); + ChromaPlanes? chroma = activePlanes.Chroma; + if (chroma is not null) + { + chroma.Value.Blue.Dispose(); + chroma.Value.Red.Dispose(); + } + + activePlanes.Owner.Dispose(); + } + + /// + /// Gets a storage-element span beginning one logical row before a block. + /// + /// The luma or chroma plane. + /// The block origin in plane samples. + /// The horizontal chroma subsampling shift. + /// The vertical chroma subsampling shift. + /// Receives the logical samples between adjacent rows. + /// The span beginning one logical row before the block. + public Span DeriveBlockPointer(Av1Plane plane, Point locationInPixels, int subX, int subY, out int stride) + { + this.GetPlaneLayout( + plane, + subX, + subY, + out Buffer2D buffer, + out int originX, + out int originY, + out _, + out _); + + int elementStride = buffer.Width; + stride = elementStride / this.storageElementsPerSample; + int blockOffset = (((originY + locationInPixels.Y) * stride) + originX + locationInPixels.X) * + this.storageElementsPerSample; + + // Intra prediction addresses above neighbors relative to the destination span, so index zero is the previous row. + blockOffset -= elementStride; + Guard.MustBeGreaterThanOrEqualTo(blockOffset, 0, nameof(blockOffset)); + + return buffer.DangerousGetSingleSpan()[blockOffset..]; + } + + /// + /// Gets a native 16-bit sample span beginning one logical row before a block. + /// + /// The luma or chroma plane. + /// The block origin in plane samples. + /// The horizontal chroma subsampling shift. + /// The vertical chroma subsampling shift. + /// Receives the logical samples between adjacent rows. + /// The 16-bit span beginning one logical row before the block. + public Span DeriveBlockPointer16(Av1Plane plane, Point locationInPixels, int subX, int subY, out int stride) + { + this.GetPlaneLayout( + plane, + subX, + subY, + out Buffer2D buffer, + out int originX, + out int originY, + out _, + out _); + + stride = buffer.Width / this.storageElementsPerSample; + int blockOffset = ((originY + locationInPixels.Y - 1) * stride) + originX + locationInPixels.X; + Guard.MustBeGreaterThanOrEqualTo(blockOffset, 0, nameof(blockOffset)); + + // High-bit-depth reconstruction uses native 16-bit samples in the byte-backed frame planes. + return MemoryMarshal.Cast(buffer.DangerousGetSingleSpan())[blockOffset..]; + } + + /// + /// Gets the visible sample region for one plane. + /// + /// The luma or chroma plane. + /// The horizontal chroma subsampling shift. + /// The vertical chroma subsampling shift. + /// The plane region excluding decoder padding. + public Buffer2DRegion DeriveBlockPointer(Av1Plane plane, int subX, int subY) + { + this.GetPlaneLayout( + plane, + subX, + subY, + out Buffer2D buffer, + out int originX, + out int originY, + out int width, + out int height); + + Rectangle region = new( + originX * this.storageElementsPerSample, + originY, + width * this.storageElementsPerSample, + height); + + return new Buffer2DRegion(buffer, region); + } + + /// + /// Gets one visible row of native 16-bit samples from a plane. + /// + /// The luma or chroma plane. + /// The zero-based visible row index. + /// The horizontal chroma subsampling shift. + /// The vertical chroma subsampling shift. + /// The visible row without decoder padding. + public Span GetHighBitDepthRowSpan(Av1Plane plane, int row, int subX, int subY) + { + this.GetPlaneLayout( + plane, + subX, + subY, + out Buffer2D buffer, + out int originX, + out int originY, + out int width, + out _); + + Span samples = MemoryMarshal.Cast(buffer.DangerousGetRowSpan(originY + row)); + return samples.Slice(originX, width); + } + + /// + /// Gets the complete padded storage allocation for one plane. + /// + /// The luma or chroma plane. + /// The horizontal chroma subsampling shift. + /// The vertical chroma subsampling shift. + /// Receives the number of logical samples between adjacent rows. + /// Receives the visible plane origin within the padded allocation. + /// The complete plane allocation, including decoder padding. + public Span GetPaddedPlaneSpan(Av1Plane plane, int subX, int subY, out int stride, out Point origin) + { + this.GetPlaneLayout( + plane, + subX, + subY, + out Buffer2D buffer, + out int originX, + out int originY, + out _, + out _); + + stride = buffer.Width / this.storageElementsPerSample; + origin = new(originX, originY); + return buffer.DangerousGetSingleSpan(); + } + + /// + /// Gets the complete padded storage allocation for one native 16-bit plane. + /// + /// The luma or chroma plane. + /// The horizontal chroma subsampling shift. + /// The vertical chroma subsampling shift. + /// Receives the number of logical samples between adjacent rows. + /// Receives the visible plane origin within the padded allocation. + /// The complete plane allocation, including decoder padding. + public Span GetPaddedPlaneSpan16(Av1Plane plane, int subX, int subY, out int stride, out Point origin) + => MemoryMarshal.Cast(this.GetPaddedPlaneSpan(plane, subX, subY, out stride, out origin)); + + /// + /// Resolves a plane allocation and its visible padded layout. + /// + /// The luma or chroma plane. + /// The horizontal chroma subsampling shift. + /// The vertical chroma subsampling shift. + /// Receives the selected plane allocation. + /// Receives the horizontal visible origin in plane samples. + /// Receives the vertical visible origin in plane samples. + /// Receives the visible plane width. + /// Receives the visible plane height. + private void GetPlaneLayout( + Av1Plane plane, + int subX, + int subY, + out Buffer2D buffer, + out int originX, + out int originY, + out int width, + out int height) + { + FramePlanes? ownedPlanes = this.planes; + ObjectDisposedException.ThrowIf(ownedPlanes is null, this); + + FramePlanes activePlanes = ownedPlanes.Value; + switch (plane) + { + case Av1Plane.Y: + buffer = activePlanes.Luma; + originX = this.OriginX; + originY = this.OriginY; + width = this.Width; + height = this.Height; + break; + case Av1Plane.U: + buffer = activePlanes.Chroma?.Blue + ?? throw new InvalidOperationException("A monochrome AV1 frame has no blue-difference plane."); + + originX = this.OriginX >> subX; + originY = this.OriginY >> subY; + width = Av1Math.DivideLog2Ceiling(this.Width, subX); + height = Av1Math.DivideLog2Ceiling(this.Height, subY); + break; + case Av1Plane.V: + default: + buffer = activePlanes.Chroma?.Red + ?? throw new InvalidOperationException("A monochrome AV1 frame has no red-difference plane."); + + originX = this.OriginX >> subX; + originY = this.OriginY >> subY; + width = Av1Math.DivideLog2Ceiling(this.Width, subX); + height = Av1Math.DivideLog2Ceiling(this.Height, subY); + break; + } + } + + /// + /// Calculates the aligned physical plane layout retained by one frame owner. + /// + private static FrameBufferLayout CreateFrameBufferLayout( + int width, + int height, + Av1ColorFormat colorFormat, + int storageElementsPerSample) + { + long alignedWidth = (width + 7L) & ~7L; + long alignedHeight = (height + 7L) & ~7L; + long lumaStride = (alignedWidth + (2L * DecoderPaddingValue) + 31L) & ~31L; + long lumaHeight = alignedHeight + (2L * DecoderPaddingValue); + int subsamplingX = colorFormat is Av1ColorFormat.Yuv420 or Av1ColorFormat.Yuv422 ? 1 : 0; + int subsamplingY = colorFormat == Av1ColorFormat.Yuv420 ? 1 : 0; + long chromaStride = colorFormat == Av1ColorFormat.Yuv400 ? 0 : lumaStride >> subsamplingX; + long chromaHeight = colorFormat == Av1ColorFormat.Yuv400 + ? 0 + : (alignedHeight >> subsamplingY) + (2L * (DecoderPaddingValue >> subsamplingY)); + + long lumaStorageWidth = lumaStride * storageElementsPerSample; + long chromaStorageWidth = chromaStride * storageElementsPerSample; + long lumaElementCount = lumaStorageWidth * lumaHeight; + long chromaElementCount = chromaStorageWidth * chromaHeight; + long planeAlignment = Math.Max(32 / Unsafe.SizeOf(), 1); + long chromaBlueOffset = ((lumaElementCount + planeAlignment - 1) / planeAlignment) * planeAlignment; + long chromaRedOffset = ((chromaBlueOffset + chromaElementCount + planeAlignment - 1) / planeAlignment) * planeAlignment; + long storageLength = colorFormat == Av1ColorFormat.Yuv400 + ? lumaElementCount + : chromaRedOffset + chromaElementCount; + + if (storageLength >= int.MaxValue) + { + // Reconstruction operators require one contiguous owner so every padded row remains directly addressable. + throw new InvalidImageContentException("The AV1 frame dimensions exceed the contiguous decoder frame limit."); + } + + return new FrameBufferLayout( + (int)lumaStorageWidth, + (int)lumaHeight, + (int)lumaElementCount, + (int)chromaStorageWidth, + (int)chromaHeight, + (int)chromaElementCount, + (int)chromaBlueOffset, + (int)chromaRedOffset, + (int)storageLength); + } + + /// + /// Carries the one frame owner, mandatory luma view, and optional complete chroma pair as one state. + /// + private readonly struct FramePlanes(IMemoryOwner owner, Buffer2D luma, ChromaPlanes? chroma) + { + /// + /// Gets the complete frame allocation. + /// + public IMemoryOwner Owner { get; } = owner; + + /// + /// Gets the padded luma plane. + /// + public Buffer2D Luma { get; } = luma; + + /// + /// Gets the padded chroma planes when the frame contains chroma. + /// + public ChromaPlanes? Chroma { get; } = chroma; + } + + /// + /// Describes the physical storage slices used by the component-plane views. + /// + private readonly struct FrameBufferLayout( + int lumaStorageWidth, + int lumaHeight, + int lumaElementCount, + int chromaStorageWidth, + int chromaHeight, + int chromaElementCount, + int chromaBlueOffset, + int chromaRedOffset, + int storageLength) + { + public int LumaStorageWidth { get; } = lumaStorageWidth; + + public int LumaHeight { get; } = lumaHeight; + + public int LumaElementCount { get; } = lumaElementCount; + + public int ChromaStorageWidth { get; } = chromaStorageWidth; + + public int ChromaHeight { get; } = chromaHeight; + + public int ChromaElementCount { get; } = chromaElementCount; + + public int ChromaBlueOffset { get; } = chromaBlueOffset; + + public int ChromaRedOffset { get; } = chromaRedOffset; + + public int StorageLength { get; } = storageLength; + } + + private readonly struct ChromaPlanes(Buffer2D blue, Buffer2D red) + { + /// + /// Gets the padded blue-difference plane. + /// + public Buffer2D Blue { get; } = blue; + + /// + /// Gets the padded red-difference plane. + /// + public Buffer2D Red { get; } = red; + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Av1LayerSelector.cs b/src/ImageSharp/Formats/Heif/Av1/Av1LayerSelector.cs new file mode 100644 index 0000000000..27cec2dcaf --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Av1LayerSelector.cs @@ -0,0 +1,21 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1; + +/// +/// Identifies the AV1 spatial layer selected by an AVIF image item. +/// +/// The spatial-layer identifier, or for progressive or final-layer decoding. +internal readonly struct Av1LayerSelector(ushort layerId) +{ + /// + /// The layer identifier that selects progressive exposure or the final layer rather than one specific spatial layer. + /// + public const ushort AllLayers = ushort.MaxValue; + + /// + /// Gets the spatial-layer identifier, or when no individual layer is selected. + /// + public ushort LayerId { get; } = layerId; +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Av1LayeredImageIndex.cs b/src/ImageSharp/Formats/Heif/Av1/Av1LayeredImageIndex.cs new file mode 100644 index 0000000000..fd19f0444a --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Av1LayeredImageIndex.cs @@ -0,0 +1,97 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1; + +/// +/// Describes the explicit payload sizes that delimit the first three layers of a layered AV1 image item. +/// +/// The first layer size in bytes. +/// The second layer size in bytes. +/// The third layer size in bytes. +internal readonly struct Av1LayeredImageIndex(uint firstLayerSize, uint secondLayerSize, uint thirdLayerSize) +{ + /// + /// Gets the first layer size in bytes. + /// + public uint FirstLayerSize { get; } = firstLayerSize; + + /// + /// Gets the second layer size in bytes. + /// + public uint SecondLayerSize { get; } = secondLayerSize; + + /// + /// Gets the third layer size in bytes. + /// + public uint ThirdLayerSize { get; } = thirdLayerSize; + + /// + /// Gets the number of item bytes needed to decode the selected spatial layer. + /// + /// The complete logical image-item payload size. + /// The requested spatial layer, or to decode the final layer. + /// The cumulative payload size through the selected layer, or the complete item size for final-layer decoding. + public int GetPayloadLength(int itemSize, Av1LayerSelector? selector) + { + int selectedLayer = selector is null || selector.Value.LayerId == Av1LayerSelector.AllLayers + ? -1 + : selector.Value.LayerId; + + uint remainingSize = (uint)itemSize; + uint selectedPayloadSize = 0; + int layerCount = 0; + for (int layer = 0; layer < Av1Constants.MaxSpatialLayerCount - 1; layer++) + { + uint layerSize = layer switch + { + 0 => this.FirstLayerSize, + 1 => this.SecondLayerSize, + _ => this.ThirdLayerSize + }; + + layerCount++; + if (layerSize == 0) + { + if (selectedLayer < 0 || selectedLayer == layer) + { + selectedPayloadSize += remainingSize; + } + + remainingSize = 0; + break; + } + + if (layerSize >= remainingSize) + { + // Every explicit layer must leave at least one byte for the final implicit layer. A zero entry instead + // identifies the current layer as final and consumes the complete remainder. + throw new InvalidImageContentException($"AV1 layered-image layer {layer} does not fit within the item payload."); + } + + if (selectedLayer < 0 || layer <= selectedLayer) + { + selectedPayloadSize += layerSize; + } + + remainingSize -= layerSize; + } + + if (remainingSize != 0) + { + if (selectedLayer < 0 || selectedLayer == layerCount) + { + selectedPayloadSize += remainingSize; + } + + layerCount++; + } + + if (selectedLayer >= layerCount) + { + throw new InvalidImageContentException($"AV1 layer selector requests layer {selectedLayer}, but the item contains {layerCount} layers."); + } + + return selectedLayer < 0 ? itemSize : (int)selectedPayloadSize; + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Av1Math.cs b/src/ImageSharp/Formats/Heif/Av1/Av1Math.cs new file mode 100644 index 0000000000..9a5702d86c --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Av1Math.cs @@ -0,0 +1,309 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1; + +/// +/// Provides the integer arithmetic primitives used by AV1 syntax and reconstruction. +/// +internal static class Av1Math +{ + /// + /// Gets the zero-based position of the most significant set bit. + /// + /// A nonzero unsigned value. + /// The most significant set-bit position. + public static int MostSignificantBit(uint value) + { + int log = 0; + int i; + + Guard.IsTrue(value != 0, nameof(value), "Must have at least one bit set."); + + for (i = 4; i >= 0; --i) + { + int shift = 1 << i; + uint x = value >> shift; + if (x != 0) + { + value = x; + log += shift; + } + } + + return log; + } + + /// + /// Gets the integer base-two logarithm of a positive value. + /// + /// The value. + /// The zero-based position of the most significant set bit. + public static int Log2(int n) + { + int result = 0; + while ((n >>= 1) > 0) + { + result++; + } + + return result; + } + + /// + /// Gets the integer base-two logarithm of an unsigned 32-bit value. + /// + /// The value. + /// The zero-based position of the most significant set bit. + public static uint Log2_32(uint x) + { + uint log = 0; + int i; + for (i = 4; i >= 0; --i) + { + uint shift = 1u << i; + uint n = x >> (int)shift; + if (n != 0) + { + x = n; + log += shift; + } + } + + return log; + } + + /// + /// Gets the greatest integer less than or equal to the base-two logarithm of a nonzero value. + /// + /// The nonzero value. + /// The floor of the base-two logarithm. + public static uint FloorLog2(uint value) + { + uint s = 0; + while (value != 0U) + { + value >>= 1; + s++; + } + + return s - 1; + } + + /// + /// Gets the least integer greater than or equal to the base-two logarithm of a value. + /// + /// The value. + /// The ceiling of the base-two logarithm, or zero for values below two. + public static uint CeilLog2(uint value) + { + if (value < 2) + { + return 0; + } + + uint i = 1; + uint p = 2; + while (p < value) + { + i++; + p <<= 1; + } + + return i; + } + + /// + /// Clips an unsigned sample to the range represented by a bit depth. + /// + /// The sample value. + /// The number of sample bits. + /// The clipped sample. + public static uint Clip1(uint value, int bitDepth) => + Clip3(0, (1U << bitDepth) - 1, value); + + /// + /// Clips an unsigned value to an inclusive range. + /// + /// The inclusive lower bound. + /// The inclusive upper bound. + /// The value to clip. + /// The clipped value. + public static uint Clip3(uint min, uint max, uint value) => Math.Max(min, Math.Min(max, value)); + + /// + /// Clips a signed value to an inclusive range. + /// + /// The inclusive lower bound. + /// The inclusive upper bound. + /// The value to clip. + /// The clipped value. + public static int Clip3(int min, int max, int value) => Math.Max(min, Math.Min(max, value)); + + /// + /// Divides an unsigned value by a power of two with nearest-integer rounding. + /// + /// The value. + /// The base-two divisor exponent. + /// The rounded quotient. + public static uint Round2(uint value, int n) + { + if (n == 0) + { + return value; + } + + return (uint)((value + (1 << (n - 1))) >> n); + } + + /// + /// Divides the absolute magnitude of a signed value by a power of two with nearest-integer rounding. + /// + /// The signed value. + /// The base-two divisor exponent. + /// The rounded nonnegative magnitude. + public static int Round2(int value, int n) + { + if (value < 0) + { + value = -value; + } + + return (int)Round2((uint)value, n); + } + + /// + /// Aligns a value upward to a multiple of a power of two. + /// + /// The value to align. + /// The base-two alignment exponent. + /// The aligned value. + public static int AlignPowerOf2(int value, int n) + { + int mask = (1 << n) - 1; + return (value + mask) & ~mask; + } + + /// + /// Divides a value by a power of two with nearest-integer rounding. + /// + /// The value. + /// The base-two divisor exponent. + /// The rounded quotient. + public static int RoundPowerOf2(int value, int n) => (value + ((1 << n) >> 1)) >> n; + + /// + /// Clamps a signed integer to an inclusive range. + /// + /// The value to clamp. + /// The inclusive lower bound. + /// The inclusive upper bound. + /// The clamped value. + public static int Clamp(int value, int low, int high) + => Math.Max(low, Math.Min(high, value)); + + /// + /// Clamps a signed long integer to an inclusive range. + /// + /// The value to clamp. + /// The inclusive lower bound. + /// The inclusive upper bound. + /// The clamped value. + public static long Clamp(long value, long low, long high) + => Math.Max(low, Math.Min(high, value)); + + /// + /// Divides a value by a power of two with floor rounding. + /// + /// The value. + /// The base-two divisor exponent. + /// The floor-rounded quotient. + public static int DivideLog2Floor(int value, int n) + => value >> n; + + /// + /// Divides a nonnegative value by a power of two with ceiling rounding. + /// + /// The value. + /// The base-two divisor exponent. + /// The ceiling-rounded quotient. + public static int DivideLog2Ceiling(int value, int n) + => (value + (1 << n) - 1) >> n; + + /// + /// Divides a value by a power of two with nearest-integer rounding. + /// + /// The value. + /// The base-two divisor exponent. + /// The rounded quotient. + public static int DivideRound(int value, int bitCount) + => (value + (1 << (bitCount - 1))) >> bitCount; + + /// + /// Gets the nonnegative remainder after division by eight. + /// + /// The value. + /// The low three bits of the value. + public static int Modulus8(int value) => value & 0x07; + + /// + /// Divides a value by eight with floor rounding. + /// + /// The value. + /// The floor-rounded quotient. + public static int DivideBy8Floor(int value) => value >> 3; + + /// + /// Divides a signed value by a power of two with symmetric nearest-integer rounding. + /// + /// The signed value. + /// The base-two divisor exponent. + /// The signed rounded quotient. + public static int RoundPowerOf2Signed(int value, int n) + => (value < 0) ? -RoundPowerOf2(-value, n) : RoundPowerOf2(value, n); + + /// + /// Right-shifts a long intermediate with nearest-integer rounding. + /// + /// The value. + /// The positive shift count. + /// The rounded signed result. + public static int RoundShift(long value, int bit) + { + DebugGuard.MustBeGreaterThanOrEqualTo(bit, 1, nameof(bit)); + return (int)((value + (1L << (bit - 1))) >> bit); + } + + /// + /// Evaluates logical implication from one Boolean condition to another. + /// + /// The antecedent. + /// The consequent. + /// only when is true and is false. + public static bool Implies(bool a, bool b) => !a || b; + + /// + /// Gets one bit from an integer value. + /// + /// The value. + /// The zero-based bit position. + /// Zero or one. + public static int GetBit(int value, int n) + => (value & (1 << n)) >> n; + + /// + /// Sets one bit in an integer value. + /// + /// The value to update. + /// The zero-based bit position. + public static void SetBit(ref int endOfBlockExtra, int n) + => endOfBlockExtra |= 1 << n; + + /// + /// Gets the absolute difference between two integers. + /// + /// The first value. + /// The second value. + /// The nonnegative absolute difference. + public static int AbsoluteDifference(int a, int b) => (a > b) ? a - b : b - a; +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Av1OperatingPointSelector.cs b/src/ImageSharp/Formats/Heif/Av1/Av1OperatingPointSelector.cs new file mode 100644 index 0000000000..4d48e01187 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Av1OperatingPointSelector.cs @@ -0,0 +1,16 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1; + +/// +/// Identifies the AV1 sequence-header operating point selected by an AVIF image item. +/// +/// The zero-based operating-point index. +internal readonly struct Av1OperatingPointSelector(byte index) +{ + /// + /// Gets the zero-based operating-point index. + /// + public byte Index { get; } = index; +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Av1PartitionType.cs b/src/ImageSharp/Formats/Heif/Av1/Av1PartitionType.cs new file mode 100644 index 0000000000..8941f191eb --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Av1PartitionType.cs @@ -0,0 +1,153 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1; + +/// +/// Identifies the subdivision pattern applied to an AV1 coding block. +/// +internal enum Av1PartitionType +{ + /// + /// Not partitioned any further. + /// + /// + /// + /// *** + /// * * + /// *** + /// + /// + None = 0, + + /// + /// Horizontally split in 2 partitions. + /// + /// + /// + /// *** + /// * * + /// *** + /// * * + /// *** + /// + /// + Horizontal = 1, + + /// + /// Vertically split in 2 partitions. + /// + /// + /// + /// ***** + /// * * * + /// ***** + /// + /// + Vertical = 2, + + /// + /// 4 equally sized partitions. + /// + /// + /// + /// ***** + /// * * * + /// ***** + /// * * * + /// ***** + /// + /// + Split = 3, + + /// + /// Horizontal split and the top partition is split again. + /// + /// + /// + /// ***** + /// * * * + /// ***** + /// * * + /// ***** + /// + /// + HorizontalA = 4, + + /// + /// Horizontal split and the bottom partition is split again. + /// + /// + /// + /// ***** + /// * * + /// ***** + /// * * * + /// ***** + /// + /// + HorizontalB = 5, + + /// + /// Vertical split and the left partition is split again. + /// + /// + /// + /// ***** + /// * * * + /// *** * + /// * * * + /// ***** + /// + /// + VerticalA = 6, + + /// + /// Vertical split and the right partition is split again. + /// + /// + /// + /// ***** + /// * * * + /// * *** + /// * * * + /// ***** + /// + /// + VerticalB = 7, + + /// + /// 4:1 horizontal partition. + /// + /// + /// + /// *** + /// * * + /// *** + /// * * + /// *** + /// * * + /// *** + /// * * + /// *** + /// + /// + Horizontal4 = 8, + + /// + /// 4:1 vertical partition. + /// + /// + /// + /// ********* + /// * * * * * + /// ********* + /// + /// + Vertical4 = 9, + + /// + /// Invalid value. + /// + Invalid = 255 +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Av1PartitionTypeExtensions.cs b/src/ImageSharp/Formats/Heif/Av1/Av1PartitionTypeExtensions.cs new file mode 100644 index 0000000000..546e6e94ea --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Av1PartitionTypeExtensions.cs @@ -0,0 +1,116 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1; + +/// +/// Provides child-block geometry for AV1 partition types. +/// +internal static class Av1PartitionTypeExtensions +{ + /// + /// Maps each partition type and parent block size to the size of its component blocks. + /// + private static readonly Av1BlockSize[][] PartitionSubSize = [ + [ + Av1BlockSize.Block4x4, + Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Block8x8, + Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Block16x16, + Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Block32x32, + Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Block64x64, + Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Block128x128, + Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Invalid, + Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Invalid + ], [ + Av1BlockSize.Invalid, + Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Block8x4, + Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Block16x8, + Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Block32x16, + Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Block64x32, + Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Block128x64, + Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Invalid, + Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Invalid + ], [ + Av1BlockSize.Invalid, + Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Block4x8, + Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Block8x16, + Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Block16x32, + Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Block32x64, + Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Block64x128, + Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Invalid, + Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Invalid + ], [ + Av1BlockSize.Block4x4, + Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Block4x4, + Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Block8x8, + Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Block16x16, + Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Block32x32, + Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Block64x64, + Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Invalid, + Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Invalid + ], [ + Av1BlockSize.Invalid, + Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Block8x4, + Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Block16x8, + Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Block32x16, + Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Block64x32, + Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Block128x64, + Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Invalid, + Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Invalid + ], [ + Av1BlockSize.Invalid, + Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Block8x4, + Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Block16x8, + Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Block32x16, + Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Block64x32, + Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Block128x64, + Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Invalid, + Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Invalid + ], [ + Av1BlockSize.Invalid, + Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Block4x8, + Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Block8x16, + Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Block16x32, + Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Block32x64, + Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Block64x128, + Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Invalid, + Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Invalid + ], [ + Av1BlockSize.Invalid, + Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Block4x8, + Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Block8x16, + Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Block16x32, + Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Block32x64, + Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Block64x128, + Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Invalid, + Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Invalid + ], [ + Av1BlockSize.Invalid, + Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Invalid, + Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Block16x4, + Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Block32x8, + Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Block64x16, + Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Invalid, + Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Invalid, + Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Invalid, + ], [ + Av1BlockSize.Invalid, + Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Invalid, + Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Block4x16, + Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Block8x32, + Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Block16x64, + Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Invalid, + Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Invalid, + Av1BlockSize.Invalid, Av1BlockSize.Invalid, Av1BlockSize.Invalid, + ] + ]; + + /// + /// Gets the component block size produced by a partition operation. + /// + /// The partition operation. + /// The parent block size. + /// The component block size, or when the partition is not permitted. + public static Av1BlockSize GetBlockSubSize(this Av1PartitionType partition, Av1BlockSize blockSize) + => PartitionSubSize[(int)partition][(int)blockSize]; +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Av1Plane.cs b/src/ImageSharp/Formats/Heif/Av1/Av1Plane.cs new file mode 100644 index 0000000000..fed90934a0 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Av1Plane.cs @@ -0,0 +1,25 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1; + +/// +/// Identifies an AV1 luma or chroma sample plane. +/// +internal enum Av1Plane : int +{ + /// + /// The luma plane. + /// + Y = 0, + + /// + /// The first chroma plane. + /// + U = 1, + + /// + /// The second chroma plane. + /// + V = 2, +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Color/Av1PlanarSampleBuffer.cs b/src/ImageSharp/Formats/Heif/Av1/Color/Av1PlanarSampleBuffer.cs new file mode 100644 index 0000000000..6e7bdb6275 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Color/Av1PlanarSampleBuffer.cs @@ -0,0 +1,133 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.InteropServices; +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Formats.Heif.Components; +using SixLabors.ImageSharp.Memory; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Color; + +/// +/// Adapts reconstructed AV1 planes to the shared HEIF planar color pipeline. +/// +/// The native unsigned sample storage type. +internal readonly struct Av1PlanarSampleBuffer : IHeifPlanarSampleBuffer + where TSample : unmanaged +{ + /// + /// The reconstructed AV1 frame containing the component planes. + /// + private readonly Av1FrameBuffer frameBuffer; + + /// + /// The visible luma plane in byte-backed storage. + /// + private readonly Buffer2DRegion luma; + + /// + /// The visible blue-difference plane in byte-backed storage. + /// + private readonly Buffer2DRegion chromaBlue; + + /// + /// The visible red-difference plane in byte-backed storage. + /// + private readonly Buffer2DRegion chromaRed; + + /// + /// Initializes a new instance of the struct. + /// + /// The reconstructed AV1 frame. + public Av1PlanarSampleBuffer(Av1FrameBuffer frameBuffer) + { + this.frameBuffer = frameBuffer; + this.luma = frameBuffer.DeriveBlockPointer(Av1Plane.Y, 0, 0); + this.chromaBlue = this.IsMonochrome + ? default + : frameBuffer.DeriveBlockPointer(Av1Plane.U, this.ChromaSubsamplingX, this.ChromaSubsamplingY); + + this.chromaRed = this.IsMonochrome + ? default + : frameBuffer.DeriveBlockPointer(Av1Plane.V, this.ChromaSubsamplingX, this.ChromaSubsamplingY); + } + + /// + public readonly int Width => this.frameBuffer.Width; + + /// + public readonly int Height => this.frameBuffer.Height; + + /// + public readonly int LumaBitDepth => this.frameBuffer.BitDepth.GetBitCount(); + + /// + public readonly int ChromaBitDepth => this.frameBuffer.BitDepth.GetBitCount(); + + /// + public readonly bool IsMonochrome => this.frameBuffer.ColorFormat == Av1ColorFormat.Yuv400; + + /// + public readonly int ChromaSubsamplingX => this.frameBuffer.ColorConfig.SubSamplingX ? 1 : 0; + + /// + public readonly int ChromaSubsamplingY => this.frameBuffer.ColorConfig.SubSamplingY ? 1 : 0; + + /// + public readonly int ChromaPositionX + { + get + { + if (this.ChromaSubsamplingX == 0) + { + return 0; + } + + // AV1 4:2:2 chroma is centered horizontally. For 4:2:0, CSP_UNKNOWN is centered while the two + // explicitly positioned layouts are co-sited with the left luma sample. + bool isCentered = this.ChromaSubsamplingY == 0 + || this.frameBuffer.ColorConfig.ChromaSamplePosition == ObuChromoSamplePosition.Unknown; + + return isCentered ? 1 : 0; + } + } + + /// + public readonly int ChromaPositionY + => this.ChromaSubsamplingY != 0 && this.frameBuffer.ColorConfig.ChromaSamplePosition != ObuChromoSamplePosition.Colocated ? 1 : 0; + + /// + public Span GetLumaRowSpan(int row) + { + if (typeof(TSample) == typeof(byte)) + { + return MemoryMarshal.Cast(this.luma.DangerousGetRowSpan(row)); + } + + return MemoryMarshal.Cast(this.frameBuffer.GetHighBitDepthRowSpan(Av1Plane.Y, row, 0, 0)); + } + + /// + public Span GetChromaBlueRowSpan(int row) + { + if (typeof(TSample) == typeof(byte)) + { + return MemoryMarshal.Cast(this.chromaBlue.DangerousGetRowSpan(row)); + } + + return MemoryMarshal.Cast( + this.frameBuffer.GetHighBitDepthRowSpan(Av1Plane.U, row, this.ChromaSubsamplingX, this.ChromaSubsamplingY)); + } + + /// + public Span GetChromaRedRowSpan(int row) + { + if (typeof(TSample) == typeof(byte)) + { + return MemoryMarshal.Cast(this.chromaRed.DangerousGetRowSpan(row)); + } + + return MemoryMarshal.Cast( + this.frameBuffer.GetHighBitDepthRowSpan(Av1Plane.V, row, this.ChromaSubsamplingX, this.ChromaSubsamplingY)); + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Color/Av1PresentationSampleBuffer.cs b/src/ImageSharp/Formats/Heif/Av1/Color/Av1PresentationSampleBuffer.cs new file mode 100644 index 0000000000..64b1904836 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Color/Av1PresentationSampleBuffer.cs @@ -0,0 +1,780 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.InteropServices; +using System.Runtime.Intrinsics; +using SixLabors.ImageSharp.Common.Helpers; +using SixLabors.ImageSharp.Formats.Heif.Components; +using SixLabors.ImageSharp.Memory; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Color; + +/// +/// Owns AV1 component planes scaled to an image item's presentation extent. +/// +/// The native unsigned sample storage type. +/// The reconstructed AV1 plane adapter. +internal sealed class Av1PresentationSampleBuffer : IDisposable + where TSample : unmanaged + where TBuffer : struct, IHeifPlanarSampleBuffer +{ + /// + /// The allocator that owns the presentation planes and row workspace. + /// + private readonly MemoryAllocator memoryAllocator; + + /// + /// The complete set of owned presentation planes, or after disposal. + /// + private PresentationPlanes? planes; + + /// + /// Initializes a new instance of the class. + /// + /// The configuration providing allocator-owned plane storage. + /// The unscaled reconstructed AV1 planes. + /// The presented luma width. + /// The presented luma height. + public Av1PresentationSampleBuffer(Configuration configuration, TBuffer source, int width, int height) + { + this.memoryAllocator = configuration.MemoryAllocator; + this.Width = width; + this.Height = height; + this.LumaBitDepth = source.LumaBitDepth; + this.ChromaBitDepth = source.ChromaBitDepth; + this.IsMonochrome = source.IsMonochrome; + this.ChromaSubsamplingX = source.ChromaSubsamplingX; + this.ChromaSubsamplingY = source.ChromaSubsamplingY; + this.ChromaPositionX = source.ChromaPositionX; + this.ChromaPositionY = source.ChromaPositionY; + + int sourceChromaWidth = DivideCeiling(source.Width, 1 << source.ChromaSubsamplingX); + int sourceChromaHeight = DivideCeiling(source.Height, 1 << source.ChromaSubsamplingY); + int destinationChromaWidth = DivideCeiling(width, 1 << source.ChromaSubsamplingX); + int destinationChromaHeight = DivideCeiling(height, 1 << source.ChromaSubsamplingY); + + Buffer2D? luma = null; + Buffer2D? chromaBlue = null; + Buffer2D? chromaRed = null; + try + { + luma = this.memoryAllocator.Allocate2D(width, height); + this.ScalePlane(source, Av1Plane.Y, source.Width, source.Height, luma); + + ChromaPlanes? chroma = null; + + if (!source.IsMonochrome) + { + chromaBlue = this.memoryAllocator.Allocate2D(destinationChromaWidth, destinationChromaHeight); + this.ScalePlane(source, Av1Plane.U, sourceChromaWidth, sourceChromaHeight, chromaBlue); + + chromaRed = this.memoryAllocator.Allocate2D(destinationChromaWidth, destinationChromaHeight); + this.ScalePlane(source, Av1Plane.V, sourceChromaWidth, sourceChromaHeight, chromaRed); + chroma = new ChromaPlanes(chromaBlue, chromaRed); + } + + // Publish ownership only after every required plane has been allocated and initialized. + this.planes = new PresentationPlanes(luma, chroma); + } + catch + { + luma?.Dispose(); + chromaBlue?.Dispose(); + chromaRed?.Dispose(); + throw; + } + } + + /// + /// Gets the presented luma width. + /// + public int Width { get; } + + /// + /// Gets the presented luma height. + /// + public int Height { get; } + + /// + /// Gets the luma sample bit depth. + /// + public int LumaBitDepth { get; } + + /// + /// Gets the chroma sample bit depth. + /// + public int ChromaBitDepth { get; } + + /// + /// Gets a value indicating whether only luma is present. + /// + public bool IsMonochrome { get; } + + /// + /// Gets the horizontal chroma-subsampling shift. + /// + public int ChromaSubsamplingX { get; } + + /// + /// Gets the vertical chroma-subsampling shift. + /// + public int ChromaSubsamplingY { get; } + + /// + /// Gets the horizontal chroma position in half-luma-sample units. + /// + public int ChromaPositionX { get; } + + /// + /// Gets the vertical chroma position in half-luma-sample units. + /// + public int ChromaPositionY { get; } + + /// + /// Gets a borrowed adapter over the scaled planes. + /// + public Av1PresentationSampleBufferView View => new(this); + + /// + /// Releases the scaled planes. + /// + public void Dispose() + { + PresentationPlanes? planes = this.planes; + this.planes = null; + + if (planes is null) + { + return; + } + + planes.Value.Luma.Dispose(); + ChromaPlanes? chroma = planes.Value.Chroma; + if (chroma is not null) + { + chroma.Value.Blue.Dispose(); + chroma.Value.Red.Dispose(); + } + } + + /// + /// Gets one scaled component row. + /// + /// The requested component plane. + /// The zero-based plane row. + /// The visible samples in the requested row. + public Span GetRowSpan(Av1Plane plane, int row) + { + PresentationPlanes planes = this.planes + ?? throw new ObjectDisposedException(nameof(Av1PresentationSampleBuffer)); + + Buffer2D buffer = plane switch + { + Av1Plane.Y => planes.Luma, + Av1Plane.U => planes.Chroma?.Blue + ?? throw new InvalidOperationException("The AV1 presentation buffer has no blue-difference plane."), + _ => planes.Chroma?.Red + ?? throw new InvalidOperationException("The AV1 presentation buffer has no red-difference plane.") + }; + + return buffer.DangerousGetRowSpan(row); + } + + /// + /// Scales one component plane with the native integer filter used by pinned libavif's libyuv backend. + /// + /// The reconstructed component planes. + /// The component plane to scale. + /// The source plane width. + /// The source plane height. + /// The scaled destination plane. + private void ScalePlane( + TBuffer source, + Av1Plane plane, + int sourceWidth, + int sourceHeight, + Buffer2D destination) + { + int destinationWidth = destination.Width; + int destinationHeight = destination.Height; + if (sourceWidth == destinationWidth && sourceHeight == destinationHeight) + { + for (int y = 0; y < sourceHeight; y++) + { + GetSourceRow(source, plane, y)[..sourceWidth].CopyTo(destination.DangerousGetRowSpan(y)); + } + + return; + } + + bool doublesWidth = (destinationWidth + 1) / 2 == sourceWidth; + bool doublesHeight = (destinationHeight + 1) / 2 == sourceHeight; + if (doublesWidth && doublesHeight) + { + ScaleUp2(source, plane, sourceWidth, sourceHeight, destination); + return; + } + + if (doublesWidth && sourceHeight == destinationHeight) + { + for (int y = 0; y < sourceHeight; y++) + { + ScaleRowUp2Linear( + GetSourceRow(source, plane, y)[..sourceWidth], + destination.DangerousGetRowSpan(y)); + } + + return; + } + + if (sourceHeight == destinationHeight) + { + int rowHorizontalStep = sourceWidth > 1 && destinationWidth > 1 + ? FixedDivideOne(sourceWidth, destinationWidth) + : 0; + + for (int y = 0; y < sourceHeight; y++) + { + ScaleHorizontal( + GetSourceRow(source, plane, y)[..sourceWidth], + destination.DangerousGetRowSpan(y), + rowHorizontalStep); + } + + return; + } + + // Layer selection presents a lower spatial layer at the full item extent, so both dimensions are monotonic. + // The general libyuv path maps destination centers in 16.16 fixed point and retains only two horizontally + // filtered rows. This avoids a second full-plane intermediate and remains group-safe under small allocators. + using Buffer2D horizontalRows = this.memoryAllocator.Allocate2D(destinationWidth, 2); + int horizontalStep = sourceWidth > 1 && destinationWidth > 1 + ? FixedDivideOne(sourceWidth, destinationWidth) + : 0; + + int verticalStep = sourceHeight > 1 && destinationHeight > 1 + ? FixedDivideOne(sourceHeight, destinationHeight) + : 0; + + int sourcePositionY = 0; + int firstSourceRow = -1; + int secondSourceRow = -1; + int firstSlot = 0; + int secondSlot = 1; + for (int y = 0; y < destinationHeight; y++) + { + int sourceRow = sourcePositionY >> 16; + int nextSourceRow = Math.Min(sourceRow + 1, sourceHeight - 1); + if (sourceRow == secondSourceRow) + { + (firstSourceRow, secondSourceRow) = (secondSourceRow, firstSourceRow); + (firstSlot, secondSlot) = (secondSlot, firstSlot); + } + + if (firstSourceRow != sourceRow) + { + ScaleHorizontal( + GetSourceRow(source, plane, sourceRow)[..sourceWidth], + horizontalRows.DangerousGetRowSpan(firstSlot), + horizontalStep); + + firstSourceRow = sourceRow; + } + + if (secondSourceRow != nextSourceRow) + { + ScaleHorizontal( + GetSourceRow(source, plane, nextSourceRow)[..sourceWidth], + horizontalRows.DangerousGetRowSpan(secondSlot), + horizontalStep); + + secondSourceRow = nextSourceRow; + } + + int verticalFraction = (sourcePositionY >> 8) & 255; + InterpolateRows( + horizontalRows.DangerousGetRowSpan(firstSlot), + horizontalRows.DangerousGetRowSpan(secondSlot), + destination.DangerousGetRowSpan(y), + verticalFraction); + + sourcePositionY += verticalStep; + } + } + + /// + /// Applies libyuv's edge-aware two-times bilinear kernel to one complete plane. + /// + /// The reconstructed component planes. + /// The component plane to scale. + /// The source plane width. + /// The source plane height. + /// The scaled destination plane. + private static void ScaleUp2( + TBuffer source, + Av1Plane plane, + int sourceWidth, + int sourceHeight, + Buffer2D destination) + { + Span firstSource = GetSourceRow(source, plane, 0)[..sourceWidth]; + Span firstDestination = destination.DangerousGetRowSpan(0); + ScaleRowUp2Bilinear(firstSource, firstSource, firstDestination, firstDestination); + + int destinationRow = 1; + for (int y = 0; y < sourceHeight - 1; y++) + { + ScaleRowUp2Bilinear( + GetSourceRow(source, plane, y)[..sourceWidth], + GetSourceRow(source, plane, y + 1)[..sourceWidth], + destination.DangerousGetRowSpan(destinationRow), + destination.DangerousGetRowSpan(destinationRow + 1)); + + destinationRow += 2; + } + + if ((destination.Height & 1) == 0) + { + Span lastSource = GetSourceRow(source, plane, sourceHeight - 1)[..sourceWidth]; + Span lastDestination = destination.DangerousGetRowSpan(destination.Height - 1); + ScaleRowUp2Bilinear(lastSource, lastSource, lastDestination, lastDestination); + } + } + + /// + /// Gets one visible source row without boxing the codec adapter. + /// + /// The reconstructed component planes. + /// The requested component plane. + /// The zero-based plane row. + /// The source row. + private static Span GetSourceRow(TBuffer source, Av1Plane plane, int row) + => plane switch + { + Av1Plane.Y => source.GetLumaRowSpan(row), + Av1Plane.U => source.GetChromaBlueRowSpan(row), + _ => source.GetChromaRedRowSpan(row) + }; + + /// + /// Applies the edge-aware two-times bilinear row kernel. + /// + /// The upper source row. + /// The lower source row. + /// The upper destination row. + /// The lower destination row. + private static void ScaleRowUp2Bilinear( + ReadOnlySpan topSource, + ReadOnlySpan bottomSource, + Span topDestination, + Span bottomDestination) + { + if (typeof(TSample) == typeof(byte)) + { + ScaleRowUp2BilinearByte( + MemoryMarshal.Cast(topSource), + MemoryMarshal.Cast(bottomSource), + MemoryMarshal.Cast(topDestination), + MemoryMarshal.Cast(bottomDestination)); + + return; + } + + ScaleRowUp2BilinearUInt16( + MemoryMarshal.Cast(topSource), + MemoryMarshal.Cast(bottomSource), + MemoryMarshal.Cast(topDestination), + MemoryMarshal.Cast(bottomDestination)); + } + + /// + /// Applies the byte two-times bilinear row kernel through portable 128-bit lanes and a scalar tail. + /// + /// The upper source row. + /// The lower source row. + /// The upper destination row. + /// The lower destination row. + private static void ScaleRowUp2BilinearByte( + ReadOnlySpan topSource, + ReadOnlySpan bottomSource, + Span topDestination, + Span bottomDestination) + { + int lastSource = topSource.Length - 1; + topDestination[0] = (byte)(((3 * topSource[0]) + bottomSource[0] + 2) >> 2); + bottomDestination[0] = (byte)((topSource[0] + (3 * bottomSource[0]) + 2) >> 2); + + int x = 0; + if (Vector128.IsHardwareAccelerated) + { + ref byte topSourceBase = ref MemoryMarshal.GetReference(topSource); + ref byte bottomSourceBase = ref MemoryMarshal.GetReference(bottomSource); + ref byte topDestinationBase = ref MemoryMarshal.GetReference(topDestination); + ref byte bottomDestinationBase = ref MemoryMarshal.GetReference(bottomDestination); + for (; x + 8 <= lastSource; x += 8) + { + Vector128 top0 = LoadEightBytes(ref topSourceBase, x); + Vector128 top1 = LoadEightBytes(ref topSourceBase, x + 1); + Vector128 bottom0 = LoadEightBytes(ref bottomSourceBase, x); + Vector128 bottom1 = LoadEightBytes(ref bottomSourceBase, x + 1); + CalculateBilinearPairs( + top0, + top1, + bottom0, + bottom1, + out Vector128 upperEven, + out Vector128 upperOdd, + out Vector128 lowerEven, + out Vector128 lowerOdd); + + StoreInterleavedBytes(upperEven, upperOdd, ref topDestinationBase, 1 + (2 * x)); + StoreInterleavedBytes(lowerEven, lowerOdd, ref bottomDestinationBase, 1 + (2 * x)); + } + } + + for (; x < lastSource; x++) + { + int top0 = topSource[x]; + int top1 = topSource[x + 1]; + int bottom0 = bottomSource[x]; + int bottom1 = bottomSource[x + 1]; + int destination = 1 + (2 * x); + topDestination[destination] = (byte)(((9 * top0) + (3 * top1) + (3 * bottom0) + bottom1 + 8) >> 4); + topDestination[destination + 1] = (byte)(((3 * top0) + (9 * top1) + bottom0 + (3 * bottom1) + 8) >> 4); + bottomDestination[destination] = (byte)(((3 * top0) + top1 + (9 * bottom0) + (3 * bottom1) + 8) >> 4); + bottomDestination[destination + 1] = (byte)((top0 + (3 * top1) + (3 * bottom0) + (9 * bottom1) + 8) >> 4); + } + + int lastDestination = topDestination.Length - 1; + topDestination[lastDestination] = (byte)(((3 * topSource[lastSource]) + bottomSource[lastSource] + 2) >> 2); + bottomDestination[lastDestination] = (byte)((topSource[lastSource] + (3 * bottomSource[lastSource]) + 2) >> 2); + } + + /// + /// Applies the unsigned 16-bit two-times bilinear row kernel through portable 128-bit lanes and a scalar tail. + /// + /// The upper source row. + /// The lower source row. + /// The upper destination row. + /// The lower destination row. + private static void ScaleRowUp2BilinearUInt16( + ReadOnlySpan topSource, + ReadOnlySpan bottomSource, + Span topDestination, + Span bottomDestination) + { + int lastSource = topSource.Length - 1; + topDestination[0] = (ushort)(((3 * topSource[0]) + bottomSource[0] + 2) >> 2); + bottomDestination[0] = (ushort)((topSource[0] + (3 * bottomSource[0]) + 2) >> 2); + + int x = 0; + if (Vector128.IsHardwareAccelerated) + { + ref ushort topSourceBase = ref MemoryMarshal.GetReference(topSource); + ref ushort bottomSourceBase = ref MemoryMarshal.GetReference(bottomSource); + ref ushort topDestinationBase = ref MemoryMarshal.GetReference(topDestination); + ref ushort bottomDestinationBase = ref MemoryMarshal.GetReference(bottomDestination); + nuint vectorCount = topSource[..lastSource].Vector128Count(); + + for (; vectorCount > 0; vectorCount--, x += Vector128.Count) + { + Vector128 top0 = Vector128.LoadUnsafe(ref topSourceBase, (nuint)x); + Vector128 top1 = Vector128.LoadUnsafe(ref topSourceBase, (nuint)(x + 1)); + Vector128 bottom0 = Vector128.LoadUnsafe(ref bottomSourceBase, (nuint)x); + Vector128 bottom1 = Vector128.LoadUnsafe(ref bottomSourceBase, (nuint)(x + 1)); + CalculateBilinearPairs( + top0, + top1, + bottom0, + bottom1, + out Vector128 upperEven, + out Vector128 upperOdd, + out Vector128 lowerEven, + out Vector128 lowerOdd); + + StoreInterleavedUInt16(upperEven, upperOdd, ref topDestinationBase, 1 + (2 * x)); + StoreInterleavedUInt16(lowerEven, lowerOdd, ref bottomDestinationBase, 1 + (2 * x)); + } + } + + for (; x < lastSource; x++) + { + int top0 = topSource[x]; + int top1 = topSource[x + 1]; + int bottom0 = bottomSource[x]; + int bottom1 = bottomSource[x + 1]; + int destination = 1 + (2 * x); + topDestination[destination] = (ushort)(((9 * top0) + (3 * top1) + (3 * bottom0) + bottom1 + 8) >> 4); + topDestination[destination + 1] = (ushort)(((3 * top0) + (9 * top1) + bottom0 + (3 * bottom1) + 8) >> 4); + bottomDestination[destination] = (ushort)(((3 * top0) + top1 + (9 * bottom0) + (3 * bottom1) + 8) >> 4); + bottomDestination[destination + 1] = (ushort)((top0 + (3 * top1) + (3 * bottom0) + (9 * bottom1) + 8) >> 4); + } + + int lastDestination = topDestination.Length - 1; + topDestination[lastDestination] = (ushort)(((3 * topSource[lastSource]) + bottomSource[lastSource] + 2) >> 2); + bottomDestination[lastDestination] = (ushort)((topSource[lastSource] + (3 * bottomSource[lastSource]) + 2) >> 2); + } + + /// + /// Calculates the four interleaved bilinear products for eight source positions. + /// + /// The upper-left samples. + /// The upper-right samples. + /// The lower-left samples. + /// The lower-right samples. + /// Receives the upper left-biased samples. + /// Receives the upper right-biased samples. + /// Receives the lower left-biased samples. + /// Receives the lower right-biased samples. + private static void CalculateBilinearPairs( + Vector128 top0, + Vector128 top1, + Vector128 bottom0, + Vector128 bottom1, + out Vector128 upperEven, + out Vector128 upperOdd, + out Vector128 lowerEven, + out Vector128 lowerOdd) + { + Vector128 rounding = Vector128.Create((ushort)8); + + // The largest twelve-bit weighted sum is 16 * 4095 + 8, which remains within unsigned 16-bit lanes. + // Keeping eight independent source positions per vector therefore avoids widening and preserves libyuv's + // exact add-before-shift rounding for both byte and high-bit-depth presentation planes. + upperEven = (((top0 << 3) + top0) + ((top1 << 1) + top1) + ((bottom0 << 1) + bottom0) + bottom1 + rounding) >> 4; + upperOdd = (((top0 << 1) + top0) + ((top1 << 3) + top1) + bottom0 + ((bottom1 << 1) + bottom1) + rounding) >> 4; + lowerEven = (((top0 << 1) + top0) + top1 + ((bottom0 << 3) + bottom0) + ((bottom1 << 1) + bottom1) + rounding) >> 4; + lowerOdd = (top0 + ((top1 << 1) + top1) + ((bottom0 << 1) + bottom0) + ((bottom1 << 3) + bottom1) + rounding) >> 4; + } + + /// + /// Loads eight byte samples as unsigned 16-bit lanes. + /// + /// The first source byte. + /// The byte offset. + /// The widened samples. + private static Vector128 LoadEightBytes(ref byte source, int offset) + { + ulong packed = Unsafe.ReadUnaligned(ref Unsafe.Add(ref source, offset)); + return Vector128.WidenLower(Vector128.CreateScalarUnsafe(packed).AsByte()); + } + + /// + /// Interleaves and stores eight pairs of byte results. + /// + /// The left-biased results. + /// The right-biased results. + /// The first destination byte. + /// The destination byte offset. + private static void StoreInterleavedBytes( + Vector128 even, + Vector128 odd, + ref byte destination, + int offset) + { + Vector128 lower = Vector128_.UnpackLow(even.AsInt16(), odd.AsInt16()).AsUInt16(); + Vector128 upper = Vector128_.UnpackHigh(even.AsInt16(), odd.AsInt16()).AsUInt16(); + Vector128.Narrow(lower, upper).StoreUnsafe(ref destination, (nuint)offset); + } + + /// + /// Interleaves and stores eight pairs of unsigned 16-bit results. + /// + /// The left-biased results. + /// The right-biased results. + /// The first destination sample. + /// The destination sample offset. + private static void StoreInterleavedUInt16( + Vector128 even, + Vector128 odd, + ref ushort destination, + int offset) + { + Vector128_.UnpackLow(even.AsInt16(), odd.AsInt16()).AsUInt16().StoreUnsafe(ref destination, (nuint)offset); + Vector128_.UnpackHigh(even.AsInt16(), odd.AsInt16()).AsUInt16().StoreUnsafe( + ref destination, + (nuint)(offset + Vector128.Count)); + } + + /// + /// Applies libyuv's edge-aware horizontal two-times linear kernel. + /// + /// The source row. + /// The destination row. + private static void ScaleRowUp2Linear(ReadOnlySpan source, Span destination) + { + if (typeof(TSample) == typeof(byte)) + { + ScaleRowUp2LinearByte( + MemoryMarshal.Cast(source), + MemoryMarshal.Cast(destination)); + + return; + } + + ScaleRowUp2LinearUInt16( + MemoryMarshal.Cast(source), + MemoryMarshal.Cast(destination)); + } + + /// + /// Applies the byte horizontal two-times linear kernel. + /// + /// The source row. + /// The destination row. + private static void ScaleRowUp2LinearByte(ReadOnlySpan source, Span destination) + { + destination[0] = source[0]; + for (int x = 0; x < source.Length - 1; x++) + { + int destinationX = 1 + (2 * x); + destination[destinationX] = (byte)(((3 * source[x]) + source[x + 1] + 2) >> 2); + destination[destinationX + 1] = (byte)((source[x] + (3 * source[x + 1]) + 2) >> 2); + } + + destination[^1] = source[^1]; + } + + /// + /// Applies the unsigned 16-bit horizontal two-times linear kernel. + /// + /// The source row. + /// The destination row. + private static void ScaleRowUp2LinearUInt16(ReadOnlySpan source, Span destination) + { + destination[0] = source[0]; + for (int x = 0; x < source.Length - 1; x++) + { + int destinationX = 1 + (2 * x); + destination[destinationX] = (ushort)(((3 * source[x]) + source[x + 1] + 2) >> 2); + destination[destinationX + 1] = (ushort)((source[x] + (3 * source[x + 1]) + 2) >> 2); + } + + destination[^1] = source[^1]; + } + + /// + /// Horizontally maps one source row with libyuv's 16.16 fixed-point bilinear positions. + /// + /// The source row. + /// The destination row. + /// The 16.16 source-position increment. + private static void ScaleHorizontal(ReadOnlySpan source, Span destination, int step) + { + if (source.Length == destination.Length) + { + source.CopyTo(destination); + return; + } + + if (source.Length == 1) + { + destination.Fill(source[0]); + return; + } + + int sourcePosition = 0; + if (typeof(TSample) == typeof(byte)) + { + ReadOnlySpan sourceBytes = MemoryMarshal.Cast(source); + Span destinationBytes = MemoryMarshal.Cast(destination); + for (int x = 0; x < destinationBytes.Length; x++) + { + int sourceX = sourcePosition >> 16; + int fraction = (sourcePosition & 0xFFFF) >> 9; + int left = sourceBytes[sourceX]; + int right = sourceBytes[sourceX + 1]; + destinationBytes[x] = (byte)(left + (((fraction * (right - left)) + 0x40) >> 7)); + sourcePosition += step; + } + + return; + } + + ReadOnlySpan sourceWords = MemoryMarshal.Cast(source); + Span destinationWords = MemoryMarshal.Cast(destination); + for (int x = 0; x < destinationWords.Length; x++) + { + int sourceX = sourcePosition >> 16; + int fraction = sourcePosition & 0xFFFF; + int left = sourceWords[sourceX]; + int right = sourceWords[sourceX + 1]; + destinationWords[x] = (ushort)(left + ((((long)fraction * (right - left)) + 0x8000) >> 16)); + sourcePosition += step; + } + } + + /// + /// Vertically interpolates two horizontally scaled rows. + /// + /// The upper row. + /// The lower row. + /// The destination row. + /// The lower-row weight with a denominator of 256. + private static void InterpolateRows( + ReadOnlySpan top, + ReadOnlySpan bottom, + Span destination, + int bottomWeight) + { + if (bottomWeight == 0) + { + top.CopyTo(destination); + return; + } + + int topWeight = 256 - bottomWeight; + if (typeof(TSample) == typeof(byte)) + { + ReadOnlySpan topBytes = MemoryMarshal.Cast(top); + ReadOnlySpan bottomBytes = MemoryMarshal.Cast(bottom); + Span destinationBytes = MemoryMarshal.Cast(destination); + for (int x = 0; x < destinationBytes.Length; x++) + { + destinationBytes[x] = (byte)(((topBytes[x] * topWeight) + (bottomBytes[x] * bottomWeight) + 128) >> 8); + } + + return; + } + + ReadOnlySpan topWords = MemoryMarshal.Cast(top); + ReadOnlySpan bottomWords = MemoryMarshal.Cast(bottom); + Span destinationWords = MemoryMarshal.Cast(destination); + for (int x = 0; x < destinationWords.Length; x++) + { + destinationWords[x] = (ushort)(((topWords[x] * topWeight) + (bottomWords[x] * bottomWeight) + 128) >> 8); + } + } + + /// + /// Divides two decremented lengths into libyuv's 16.16 endpoint-preserving step. + /// + /// The source length. + /// The destination length. + /// The 16.16 source-position increment. + private static int FixedDivideOne(int sourceLength, int destinationLength) + => (int)((((long)sourceLength << 16) - 0x00010001) / (destinationLength - 1)); + + /// + /// Divides a positive value by a positive divisor with ceiling rounding. + /// + /// The value to divide. + /// The positive divisor. + /// The ceiling-rounded quotient. + private static int DivideCeiling(int value, int divisor) => (value + divisor - 1) / divisor; + + private readonly struct ChromaPlanes(Buffer2D blue, Buffer2D red) + { + public Buffer2D Blue { get; } = blue; + + public Buffer2D Red { get; } = red; + } + + private readonly struct PresentationPlanes(Buffer2D luma, ChromaPlanes? chroma) + { + public Buffer2D Luma { get; } = luma; + + public ChromaPlanes? Chroma { get; } = chroma; + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Color/Av1PresentationSampleBufferView.cs b/src/ImageSharp/Formats/Heif/Av1/Color/Av1PresentationSampleBufferView.cs new file mode 100644 index 0000000000..d7f09520f7 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Color/Av1PresentationSampleBufferView.cs @@ -0,0 +1,63 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Components; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Color; + +/// +/// Exposes borrowed rows from an owned AV1 presentation buffer to the shared HEIF color converter. +/// +/// The native unsigned sample storage type. +/// The reconstructed AV1 plane adapter owned by the presentation buffer. +internal readonly struct Av1PresentationSampleBufferView : IHeifPlanarSampleBuffer + where TSample : unmanaged + where TBuffer : struct, IHeifPlanarSampleBuffer +{ + /// + /// The owner that keeps all exposed rows alive. + /// + private readonly Av1PresentationSampleBuffer owner; + + /// + /// Initializes a new instance of the struct. + /// + /// The scaled plane owner. + public Av1PresentationSampleBufferView(Av1PresentationSampleBuffer owner) => this.owner = owner; + + /// + public int Width => this.owner.Width; + + /// + public int Height => this.owner.Height; + + /// + public int LumaBitDepth => this.owner.LumaBitDepth; + + /// + public int ChromaBitDepth => this.owner.ChromaBitDepth; + + /// + public bool IsMonochrome => this.owner.IsMonochrome; + + /// + public int ChromaSubsamplingX => this.owner.ChromaSubsamplingX; + + /// + public int ChromaSubsamplingY => this.owner.ChromaSubsamplingY; + + /// + public int ChromaPositionX => this.owner.ChromaPositionX; + + /// + public int ChromaPositionY => this.owner.ChromaPositionY; + + /// + public Span GetLumaRowSpan(int row) => this.owner.GetRowSpan(Av1Plane.Y, row); + + /// + public Span GetChromaBlueRowSpan(int row) => this.owner.GetRowSpan(Av1Plane.U, row); + + /// + public Span GetChromaRedRowSpan(int row) => this.owner.GetRowSpan(Av1Plane.V, row); +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Color/Av1YuvConverter.cs b/src/ImageSharp/Formats/Heif/Av1/Color/Av1YuvConverter.cs new file mode 100644 index 0000000000..5fe0b0cfc4 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Color/Av1YuvConverter.cs @@ -0,0 +1,275 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Formats.Heif.Components; +using SixLabors.ImageSharp.Formats.Heif.Components.Alpha; +using SixLabors.ImageSharp.Metadata.Profiles.Cicp; +using SixLabors.ImageSharp.PixelFormats; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Color; + +/// +/// Adapts AV1 color signaling and reconstructed planes to the shared HEIF color pipeline. +/// +internal static class Av1YuvConverter +{ + /// + /// Converts the reconstructed component planes to packed pixels. + /// + /// The destination pixel type. + /// The configuration used for allocation and pixel conversion. + /// The reconstructed AV1 frame. + /// The destination image frame. + public static void ConvertToRgb(Configuration configuration, Av1FrameBuffer frameBuffer, ImageFrame image) + where TPixel : unmanaged, IPixel + { + HeifColorConversionParameters parameters = GetConversionParameters(frameBuffer.ColorConfig, out HeifColorConversionMode mode); + if (frameBuffer.BitDepth == Av1BitDepth.EightBit) + { + Av1PlanarSampleBuffer buffer = new(frameBuffer); + if (buffer.Width != image.Width || buffer.Height != image.Height) + { + // AVIF spatial-layer selection scales native YUV planes before color conversion. The retained + // reconstruction remains untouched because later dependent layers can still reference its coded + // dimensions, while this short-lived owner contains only the presented sample grid. + using Av1PresentationSampleBuffer> presentationBuffer = new( + configuration, + buffer, + image.Width, + image.Height); + + HeifPlanarColorConverter.ConvertToRgb< + TPixel, + Av1PresentationSampleBufferView>, + byte, + HeifByteSampleConverter>( + configuration, + presentationBuffer.View, + image, + in parameters, + mode); + + return; + } + + HeifPlanarColorConverter.ConvertToRgb, byte, HeifByteSampleConverter>( + configuration, + buffer, + image, + in parameters, + mode); + + return; + } + + Av1PlanarSampleBuffer highBitDepthBuffer = new(frameBuffer); + if (highBitDepthBuffer.Width != image.Width || highBitDepthBuffer.Height != image.Height) + { + using Av1PresentationSampleBuffer> presentationBuffer = new( + configuration, + highBitDepthBuffer, + image.Width, + image.Height); + + HeifPlanarColorConverter.ConvertToRgb< + TPixel, + Av1PresentationSampleBufferView>>( + configuration, + presentationBuffer.View, + image, + in parameters, + mode); + + return; + } + + HeifPlanarColorConverter.ConvertToRgb>( + configuration, + highBitDepthBuffer, + image, + in parameters, + mode); + } + + /// + /// Converts a rectangular region of reconstructed component planes directly to packed pixels. + /// + /// The destination pixel type. + /// The configuration used for allocation and pixel conversion. + /// The reconstructed AV1 frame. + /// The luma-sample region mapped to the complete destination frame. + /// The destination image frame. + public static void ConvertRegionToRgb( + Configuration configuration, + Av1FrameBuffer frameBuffer, + Rectangle sourceRectangle, + ImageFrame image) + where TPixel : unmanaged, IPixel + { + HeifColorConversionParameters parameters = GetConversionParameters(frameBuffer.ColorConfig, out HeifColorConversionMode mode); + if (frameBuffer.BitDepth == Av1BitDepth.EightBit) + { + Av1PlanarSampleBuffer buffer = new(frameBuffer); + HeifPlanarColorConverter.ConvertToRgb, byte, HeifByteSampleConverter>( + configuration, + buffer, + image, + in parameters, + mode, + sourceRectangle.X, + sourceRectangle.Y); + + return; + } + + Av1PlanarSampleBuffer highBitDepthBuffer = new(frameBuffer); + HeifPlanarColorConverter.ConvertToRgb>( + configuration, + highBitDepthBuffer, + image, + in parameters, + mode, + sourceRectangle.X, + sourceRectangle.Y); + } + + /// + /// Composes the reconstructed luma plane into a packed color frame as auxiliary alpha. + /// + /// The destination color pixel type. + /// The configuration used for allocation and pixel conversion. + /// The reconstructed AV1 frame containing the alpha luma plane. + /// The packed color frame receiving alpha values. + /// The complete presented size of the auxiliary image or grid tile. + /// The destination region receiving the top-left portion of the presented alpha image. + /// Whether stored color samples must be converted to unassociated alpha. + public static void ComposeAlpha( + Configuration configuration, + Av1FrameBuffer frameBuffer, + ImageFrame destination, + Size outputSize, + Rectangle destinationRectangle, + bool premultiplied) + where TPixel : unmanaged, IPixel + => ComposeAlpha( + configuration, + frameBuffer, + new Rectangle(0, 0, frameBuffer.Width, frameBuffer.Height), + destination, + outputSize, + destinationRectangle, + premultiplied); + + /// + /// Composes a rectangular reconstructed luma region into a packed color frame as auxiliary alpha. + /// + /// The destination color pixel type. + /// The configuration used for allocation and pixel conversion. + /// The reconstructed AV1 frame containing the alpha luma plane. + /// The luma-sample region mapped to the destination rectangle. + /// The packed color frame receiving alpha values. + /// The complete presented size of the auxiliary image or grid tile. + /// The destination region receiving the presented alpha image. + /// Whether stored color samples must be converted to unassociated alpha. + public static void ComposeAlpha( + Configuration configuration, + Av1FrameBuffer frameBuffer, + Rectangle sourceRectangle, + ImageFrame destination, + Size outputSize, + Rectangle destinationRectangle, + bool premultiplied) + where TPixel : unmanaged, IPixel + { + HeifColorConversionParameters parameters = GetConversionParameters(frameBuffer.ColorConfig, out _); + if (frameBuffer.BitDepth == Av1BitDepth.EightBit) + { + Av1PlanarSampleBuffer buffer = new(frameBuffer); + HeifPlanarAlphaCompositor.Compose, byte, HeifByteSampleConverter>( + configuration, + buffer, + destination, + in parameters, + sourceRectangle, + outputSize, + destinationRectangle, + premultiplied); + + return; + } + + Av1PlanarSampleBuffer highBitDepthBuffer = new(frameBuffer); + HeifPlanarAlphaCompositor.Compose, ushort, HeifUShortSampleConverter>( + configuration, + highBitDepthBuffer, + destination, + in parameters, + sourceRectangle, + outputSize, + destinationRectangle, + premultiplied); + } + + /// + /// Converts packed pixels to the configured monochrome or component planes used by the AV1 encoder. + /// + /// The source pixel type. + /// The configuration used for allocation and pixel conversion. + /// The source image frame. + /// The destination AV1 frame. + public static void ConvertFromRgb(Configuration configuration, ImageFrame image, Av1FrameBuffer frameBuffer) + where TPixel : unmanaged, IPixel + { + HeifColorConversionParameters parameters = GetConversionParameters(frameBuffer.ColorConfig, out HeifColorConversionMode mode); + if (frameBuffer.BitDepth == Av1BitDepth.EightBit) + { + Av1PlanarSampleBuffer buffer = new(frameBuffer); + HeifPlanarColorConverter.ConvertFromRgb, byte, HeifByteSampleConverter>( + configuration, + image, + buffer, + in parameters, + mode); + + return; + } + + Av1PlanarSampleBuffer highBitDepthBuffer = new(frameBuffer); + HeifPlanarColorConverter.ConvertFromRgb, ushort, HeifUShortSampleConverter>( + configuration, + image, + highBitDepthBuffer, + in parameters, + mode); + } + + /// + /// Resolves the H.273 conversion mode, matrix coefficients, and sample range for a frame. + /// + /// The signaled AV1 color configuration. + /// The resolved conversion mode. + /// The resolved conversion parameters. + public static HeifColorConversionParameters GetConversionParameters( + ObuColorConfig colorConfig, + out HeifColorConversionMode mode) + { + if (colorConfig.ChromaSamplePosition == ObuChromoSamplePosition.Reserved) + { + throw new InvalidImageContentException("The reserved AV1 chroma sample position is invalid."); + } + + bool isMonochrome = colorConfig.IsMonochrome; + + return HeifColorConversionParameters.Create( + (CicpColorPrimaries)(byte)colorConfig.ColorPrimaries, + (CicpTransferCharacteristics)(byte)colorConfig.TransferCharacteristics, + (CicpMatrixCoefficients)(byte)colorConfig.MatrixCoefficients, + colorConfig.ColorRange, + colorConfig.BitDepth.GetBitCount(), + colorConfig.BitDepth.GetBitCount(), + isMonochrome, + colorConfig.GetColorFormat() == Av1ColorFormat.Yuv444, + out mode); + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Entropy/Av1DefaultDistributions.cs b/src/ImageSharp/Formats/Heif/Av1/Entropy/Av1DefaultDistributions.cs new file mode 100644 index 0000000000..f70c722fcd --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Entropy/Av1DefaultDistributions.cs @@ -0,0 +1,2800 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Entropy; + +/// +/// Provides the normative initial AV1 cumulative distributions used before tile-local adaptation. +/// +internal static class Av1DefaultDistributions +{ + /// + /// Gets the inter-frame intra luma-mode distributions indexed by block-size group. + /// + public static Av1Distribution[] FrameYMode => + [ + new(22801, 23489, 24293, 24756, 25601, 26123, 26606, 27418, 27945, 29228, 29685, 30349), + new(18673, 19845, 22631, 23318, 23950, 24649, 25527, 27364, 28152, 29701, 29984, 30852), + new(19770, 20979, 23396, 23939, 24241, 24654, 25136, 27073, 27830, 29360, 29730, 30659), + new(20155, 21301, 22838, 23178, 23261, 23533, 23703, 24804, 25352, 26575, 27016, 28049) + ]; + + /// + /// Gets the distributions that select intra or inter prediction from the available spatial neighbors. + /// + public static Av1Distribution[] IntraInter => [new(806), new(16662), new(20186), new(26538)]; + + /// + /// Gets the distributions that select a newly decoded motion vector before the remaining single-reference modes. + /// + public static Av1Distribution[] NewMv => [new(24035), new(16630), new(15339), new(8386), new(12222), new(4676)]; + + /// + /// Gets the distributions that select global motion before the spatial reference-motion-vector modes. + /// + public static Av1Distribution[] ZeroMv => [new(2175), new(1054)]; + + /// + /// Gets the distributions that select the nearest or near spatial reference motion vector. + /// + public static Av1Distribution[] RefMv => [new(23974), new(24188), new(17848), new(28622), new(24312), new(19923)]; + + /// + /// Gets the distributions that advance through the dynamic reference-motion-vector candidate list. + /// + public static Av1Distribution[] Drl => [new(13104), new(24560), new(18945)]; + + /// + /// Gets the single-reference selection distributions indexed by spatial context and tree decision. + /// + public static Av1Distribution[][] SingleReference => + [ + [new(4897), new(1555), new(4236), new(8650), new(904), new(1444)], + [new(16973), new(16751), new(19647), new(24773), new(11014), new(15087)], + [new(29744), new(30279), new(31194), new(31895), new(26875), new(30304)], + ]; + + /// + /// Gets the distributions that select single-reference or compound-reference prediction for a block. + /// + public static Av1Distribution[] CompInter => [new(26828), new(24035), new(12031), new(10640), new(2901)]; + + /// + /// Gets the distributions that select unidirectional or bidirectional compound reference pairs. + /// + public static Av1Distribution[] CompoundReferenceType => [new(1198), new(2070), new(9166), new(7499), new(22475)]; + + /// + /// Gets the unidirectional compound-reference selection distributions indexed by spatial context and tree decision. + /// + public static Av1Distribution[][] UnidirectionalCompoundReference => + [ + [new(5284), new(3865), new(3128)], + [new(23152), new(14173), new(15270)], + [new(31774), new(25120), new(26710)], + ]; + + /// + /// Gets the bidirectional compound forward-reference distributions indexed by spatial context and tree decision. + /// + public static Av1Distribution[][] CompoundReference => + [ + [new(4946), new(9468), new(1503)], + [new(19891), new(22441), new(15160)], + [new(30731), new(31059), new(27544)], + ]; + + /// + /// Gets the bidirectional compound backward-reference distributions indexed by spatial context and tree decision. + /// + public static Av1Distribution[][] CompoundBackwardReference => + [ + [new(2235), new(1423)], + [new(17182), new(15175)], + [new(30606), new(30489)], + ]; + + /// + /// Gets the eight-symbol compound inter-mode distributions indexed by derived mode context. + /// + public static Av1Distribution[] InterCompoundMode => + [ + new(7760, 13823, 15808, 17641, 19156, 20666, 26891), + new(10730, 19452, 21145, 22749, 24039, 25131, 28724), + new(10664, 20221, 21588, 22906, 24295, 25387, 28436), + new(13298, 16984, 20471, 24182, 25067, 25736, 26422), + new(18904, 23325, 25242, 27432, 27898, 28258, 30758), + new(10725, 17454, 20124, 22820, 24195, 25168, 26046), + new(17125, 24273, 25814, 27492, 28214, 28704, 30592), + new(13046, 23214, 24505, 25942, 27435, 28442, 29330), + ]; + + /// + /// Gets the inter-intra prediction flag distributions indexed by block-size group. + /// + public static Av1Distribution[] InterIntra => [new(16384), new(26887), new(27597), new(30237)]; + + /// + /// Gets the four-symbol inter-intra mode distributions indexed by block-size group. + /// + public static Av1Distribution[] InterIntraMode => + [ + new(8192, 16384, 24576), + new(1875, 11082, 27332), + new(2473, 9996, 26388), + new(4238, 11537, 25926), + ]; + + /// + /// Gets the inter-intra wedge-selection distributions indexed by block size. + /// + public static Av1Distribution[] WedgeInterIntra => + [ + new(16384), new(16384), new(16384), new(20036), new(24957), new(26704), + new(27530), new(29564), new(29444), new(26872), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384), + ]; + + /// + /// Gets the wedge-or-difference masked-compound distributions indexed by block size. + /// + public static Av1Distribution[] CompoundType => + [ + new(16384), new(16384), new(16384), new(23431), new(13171), new(11470), + new(9770), new(9100), new(8233), new(6172), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384), new(16384), new(16384), + new(11820), new(7701), new(16384), new(16384), + ]; + + /// + /// Gets the sixteen-symbol wedge-index distributions indexed by block size. + /// + public static Av1Distribution[] WedgeIndex => + [ + CreateUniformWedgeIndexDistribution(), + CreateUniformWedgeIndexDistribution(), + CreateUniformWedgeIndexDistribution(), + new(2438, 4440, 6599, 8663, 11005, 12874, 15751, 18094, 20359, 22362, 24127, 25702, 27752, 29450, 31171), + new(806, 3266, 6005, 6738, 7218, 7367, 7771, 14588, 16323, 17367, 18452, 19422, 22839, 26127, 29629), + new(2779, 3738, 4683, 7213, 7775, 8017, 8655, 14357, 17939, 21332, 24520, 27470, 29456, 30529, 31656), + new(1684, 3625, 5675, 7108, 9302, 11274, 14429, 17144, 19163, 20961, 22884, 24471, 26719, 28714, 30877), + new(1142, 3491, 6277, 7314, 8089, 8355, 9023, 13624, 15369, 16730, 18114, 19313, 22521, 26012, 29550), + new(2742, 4195, 5727, 8035, 8980, 9336, 10146, 14124, 17270, 20533, 23434, 25972, 27944, 29570, 31416), + new(1727, 3948, 6101, 7796, 9841, 12344, 15766, 18944, 20638, 22038, 23963, 25311, 26988, 28766, 31012), + CreateUniformWedgeIndexDistribution(), + CreateUniformWedgeIndexDistribution(), + CreateUniformWedgeIndexDistribution(), + CreateUniformWedgeIndexDistribution(), + CreateUniformWedgeIndexDistribution(), + CreateUniformWedgeIndexDistribution(), + CreateUniformWedgeIndexDistribution(), + CreateUniformWedgeIndexDistribution(), + new(154, 987, 1925, 2051, 2088, 2111, 2151, 23033, 23703, 24284, 24985, 25684, 27259, 28883, 30911), + new(1135, 1322, 1493, 2635, 2696, 2737, 2770, 21016, 22935, 25057, 27251, 29173, 30089, 30960, 31933), + CreateUniformWedgeIndexDistribution(), + CreateUniformWedgeIndexDistribution(), + ]; + + /// + /// Gets the average-or-distance-weighted compound distributions indexed by derived context. + /// + public static Av1Distribution[] CompoundIndex => + [new(18244), new(12865), new(7053), new(13259), new(9334), new(4644)]; + + /// + /// Gets the unmasked-or-masked compound-group distributions indexed by derived context. + /// + public static Av1Distribution[] CompoundGroupIndex => + [new(26607), new(22891), new(18840), new(24594), new(19934), new(22674)]; + + /// + /// Gets the Simple Translation, OBMC, or Warped motion-mode distributions indexed by block size. + /// + public static Av1Distribution[] MotionMode => + [ + new(10923, 21845), + new(10923, 21845), + new(10923, 21845), + new(7651, 24760), + new(4738, 24765), + new(5391, 25528), + new(19419, 26810), + new(5123, 23606), + new(11606, 24308), + new(26260, 29116), + new(20360, 28062), + new(21679, 26830), + new(29516, 30701), + new(28898, 30397), + new(30878, 31335), + new(32507, 32558), + new(10923, 21845), + new(10923, 21845), + new(28799, 31390), + new(26431, 30774), + new(28973, 31594), + new(29742, 31203), + ]; + + /// + /// Gets the Simple Translation or OBMC motion-mode distributions indexed by block size. + /// + public static Av1Distribution[] Obmc => + [ + new(16384), + new(16384), + new(16384), + new(10437), + new(9371), + new(9301), + new(17432), + new(14423), + new(15142), + new(25817), + new(22823), + new(22083), + new(30128), + new(31014), + new(31560), + new(32638), + new(16384), + new(16384), + new(23664), + new(20901), + new(24008), + new(26879), + ]; + + /// + /// Gets the switchable interpolation-filter distributions indexed by reference type, direction, and neighbor state. + /// + public static Av1Distribution[] SwitchableInterpolation => + [ + new(31935, 32720), + new(5568, 32719), + new(422, 2938), + new(28244, 32608), + new(31206, 31953), + new(4862, 32121), + new(770, 1152), + new(20889, 25637), + new(31910, 32724), + new(4120, 32712), + new(305, 2247), + new(27403, 32636), + new(31022, 32009), + new(2963, 32093), + new(601, 943), + new(14969, 21398), + ]; + + /// + /// Gets the key-frame luma-mode distributions indexed by the above and left intra-mode contexts. + /// + public static Av1Distribution[][] FilterYMode => + [ + [ + new(15588, 17027, 19338, 20218, 20682, 21110, 21825, 23244, 24189, 28165, 29093, 30466), + new(12016, 18066, 19516, 20303, 20719, 21444, 21888, 23032, 24434, 28658, 30172, 31409), + new(10052, 10771, 22296, 22788, 23055, 23239, 24133, 25620, 26160, 29336, 29929, 31567), + new(14091, 15406, 16442, 18808, 19136, 19546, 19998, 22096, 24746, 29585, 30958, 32462), + new(12122, 13265, 15603, 16501, 18609, 20033, 22391, 25583, 26437, 30261, 31073, 32475) + ], [ + new(10023, 19585, 20848, 21440, 21832, 22760, 23089, 24023, 25381, 29014, 30482, 31436), + new(5983, 24099, 24560, 24886, 25066, 25795, 25913, 26423, 27610, 29905, 31276, 31794), + new(7444, 12781, 20177, 20728, 21077, 21607, 22170, 23405, 24469, 27915, 29090, 30492), + new(8537, 14689, 15432, 17087, 17408, 18172, 18408, 19825, 24649, 29153, 31096, 32210), + new(7543, 14231, 15496, 16195, 17905, 20717, 21984, 24516, 26001, 29675, 30981, 31994) + ], [ + new(12613, 13591, 21383, 22004, 22312, 22577, 23401, 25055, 25729, 29538, 30305, 32077), + new(9687, 13470, 18506, 19230, 19604, 20147, 20695, 22062, 23219, 27743, 29211, 30907), + new(6183, 6505, 26024, 26252, 26366, 26434, 27082, 28354, 28555, 30467, 30794, 32086), + new(10718, 11734, 14954, 17224, 17565, 17924, 18561, 21523, 23878, 28975, 30287, 32252), + new(9194, 9858, 16501, 17263, 18424, 19171, 21563, 25961, 26561, 30072, 30737, 32463) + ], [ + new(12602, 14399, 15488, 18381, 18778, 19315, 19724, 21419, 25060, 29696, 30917, 32409), + new(8203, 13821, 14524, 17105, 17439, 18131, 18404, 19468, 25225, 29485, 31158, 32342), + new(8451, 9731, 15004, 17643, 18012, 18425, 19070, 21538, 24605, 29118, 30078, 32018), + new(7714, 9048, 9516, 16667, 16817, 16994, 17153, 18767, 26743, 30389, 31536, 32528), + new(8843, 10280, 11496, 15317, 16652, 17943, 19108, 22718, 25769, 29953, 30983, 32485) + ], [ + new(12578, 13671, 15979, 16834, 19075, 20913, 22989, 25449, 26219, 30214, 31150, 32477), + new(9563, 13626, 15080, 15892, 17756, 20863, 22207, 24236, 25380, 29653, 31143, 32277), + new(8356, 8901, 17616, 18256, 19350, 20106, 22598, 25947, 26466, 29900, 30523, 32261), + new(10835, 11815, 13124, 16042, 17018, 18039, 18947, 22753, 24615, 29489, 30883, 32482), + new(7618, 8288, 9859, 10509, 15386, 18657, 22903, 28776, 29180, 31355, 31802, 32593) + ] + ]; + + /// + /// Gets the chroma intra-mode distributions indexed first by chroma-from-luma availability and then by luma mode. + /// + public static Av1Distribution[][] UvMode => + [ + [ + new(22631, 24152, 25378, 25661, 25986, 26520, 27055, 27923, 28244, 30059, 30941, 31961), + new(9513, 26881, 26973, 27046, 27118, 27664, 27739, 27824, 28359, 29505, 29800, 31796), + new(9845, 9915, 28663, 28704, 28757, 28780, 29198, 29822, 29854, 30764, 31777, 32029), + new(13639, 13897, 14171, 25331, 25606, 25727, 25953, 27148, 28577, 30612, 31355, 32493), + new(9764, 9835, 9930, 9954, 25386, 27053, 27958, 28148, 28243, 31101, 31744, 32363), + new(11825, 13589, 13677, 13720, 15048, 29213, 29301, 29458, 29711, 31161, 31441, 32550), + new(14175, 14399, 16608, 16821, 17718, 17775, 28551, 30200, 30245, 31837, 32342, 32667), + new(12885, 13038, 14978, 15590, 15673, 15748, 16176, 29128, 29267, 30643, 31961, 32461), + new(12026, 13661, 13874, 15305, 15490, 15726, 15995, 16273, 28443, 30388, 30767, 32416), + new(19052, 19840, 20579, 20916, 21150, 21467, 21885, 22719, 23174, 28861, 30379, 32175), + new(18627, 19649, 20974, 21219, 21492, 21816, 22199, 23119, 23527, 27053, 31397, 32148), + new(17026, 19004, 19997, 20339, 20586, 21103, 21349, 21907, 22482, 25896, 26541, 31819), + new(12124, 13759, 14959, 14992, 15007, 15051, 15078, 15166, 15255, 15753, 16039, 16606) + ], [ + new(10407, 11208, 12900, 13181, 13823, 14175, 14899, 15656, 15986, 20086, 20995, 22455, 24212), + new(4532, 19780, 20057, 20215, 20428, 21071, 21199, 21451, 22099, 24228, 24693, 27032, 29472), + new(5273, 5379, 20177, 20270, 20385, 20439, 20949, 21695, 21774, 23138, 24256, 24703, 26679), + new(6740, 7167, 7662, 14152, 14536, 14785, 15034, 16741, 18371, 21520, 22206, 23389, 24182), + new(4987, 5368, 5928, 6068, 19114, 20315, 21857, 22253, 22411, 24911, 25380, 26027, 26376), + new(5370, 6889, 7247, 7393, 9498, 21114, 21402, 21753, 21981, 24780, 25386, 26517, 27176), + new(4816, 4961, 7204, 7326, 8765, 8930, 20169, 20682, 20803, 23188, 23763, 24455, 24940), + new(6608, 6740, 8529, 9049, 9257, 9356, 9735, 18827, 19059, 22336, 23204, 23964, 24793), + new(5998, 7419, 7781, 8933, 9255, 9549, 9753, 10417, 18898, 22494, 23139, 24764, 25989), + new(10660, 11298, 12550, 12957, 13322, 13624, 14040, 15004, 15534, 20714, 21789, 23443, 24861), + new(10522, 11530, 12552, 12963, 13378, 13779, 14245, 15235, 15902, 20102, 22696, 23774, 25838), + new(10099, 10691, 12639, 13049, 13386, 13665, 14125, 15163, 15636, 19676, 20474, 23519, 25208), + new(3144, 5087, 7382, 7504, 7593, 7690, 7801, 8064, 8232, 9248, 9875, 10521, 29048) + ] + ]; + + /// + /// Gets the directional intra-prediction angle-delta distributions indexed by directional mode. + /// + public static Av1Distribution[] AngleDelta => + [ + new(2180, 5032, 7567, 22776, 26989, 30217), + new(2301, 5608, 8801, 23487, 26974, 30330), + new(3780, 11018, 13699, 19354, 23083, 31286), + new(4581, 11226, 15147, 17138, 21834, 28397), + new(1737, 10927, 14509, 19588, 22745, 28823), + new(2664, 10176, 12485, 17650, 21600, 30495), + new(2240, 11096, 15453, 20341, 22561, 28917), + new(3605, 10428, 12459, 17676, 21244, 30655) + ]; + + /// + /// Gets the binary distribution that signals intra-block copy. + /// + public static Av1Distribution IntraBlockCopy => new(30531); + + /// + /// Gets the distribution selecting none, Wiener, or self-guided filtering for a switchable restoration unit. + /// + public static Av1Distribution SwitchableRestoration => new(9413, 22581); + + /// + /// Gets the distribution selecting whether a Wiener restoration unit is filtered. + /// + public static Av1Distribution WienerRestoration => new(11570); + + /// + /// Gets the distribution selecting whether a self-guided restoration unit is filtered. + /// + public static Av1Distribution SgrProjectionRestoration => new(16855); + + /// + /// Gets the luma palette-mode distributions indexed by block-size and neighboring-palette contexts. + /// + public static Av1Distribution[][] PaletteYMode => + [ + [new(31676), new(3419), new(1261)], + [new(31912), new(2859), new(980)], + [new(31823), new(3400), new(781)], + [new(32030), new(3561), new(904)], + [new(32309), new(7337), new(1462)], + [new(32265), new(4015), new(1521)], + [new(32450), new(7946), new(129)] + ]; + + /// + /// Gets the chroma palette-mode distributions indexed by whether luma uses a palette. + /// + public static Av1Distribution[] PaletteUvMode => [new(32461), new(21488)]; + + /// + /// Gets the luma palette-size distributions indexed by block-size context. + /// + public static Av1Distribution[] PaletteYSize => + [ + new(7952, 13000, 18149, 21478, 25527, 29241), + new(7139, 11421, 16195, 19544, 23666, 28073), + new(7788, 12741, 17325, 20500, 24315, 28530), + new(8271, 14064, 18246, 21564, 25071, 28533), + new(12725, 19180, 21863, 24839, 27535, 30120), + new(9711, 14888, 16923, 21052, 25661, 27875), + new(14940, 20797, 21678, 24186, 27033, 28999) + ]; + + /// + /// Gets the chroma palette-size distributions indexed by block-size context. + /// + public static Av1Distribution[] PaletteUvSize => + [ + new(8713, 19979, 27128, 29609, 31331, 32272), + new(5839, 15573, 23581, 26947, 29848, 31700), + new(4426, 11260, 17999, 21483, 25863, 29430), + new(3228, 9464, 14993, 18089, 22523, 27420), + new(3768, 8886, 13091, 17852, 22495, 27207), + new(2464, 8451, 12861, 21632, 25525, 28555), + new(1269, 5435, 10433, 18963, 21700, 25865) + ]; + + /// + /// Gets the luma palette color-index distributions indexed by palette size minus two and spatial context. + /// + public static Av1Distribution[][] PaletteYColorIndex => + [ + [new(28710), new(16384), new(10553), new(27036), new(31603)], + [new(27877, 30490), new(11532, 25697), new(6544, 30234), new(23018, 28072), new(31915, 32385)], + [new(25572, 28046, 30045), new(9478, 21590, 27256), new(7248, 26837, 29824), new(19167, 24486, 28349), new(31400, 31825, 32250)], + [new(24779, 26955, 28576, 30282), new(8669, 20364, 24073, 28093), new(4255, 27565, 29377, 31067), new(19864, 23674, 26716, 29530), new(31646, 31893, 32147, 32426)], + [new(23132, 25407, 26970, 28435, 30073), new(7443, 17242, 20717, 24762, 27982), new(6300, 24862, 26944, 28784, 30671), new(18916, 22895, 25267, 27435, 29652), new(31270, 31550, 31808, 32059, 32353)], + [new(23105, 25199, 26464, 27684, 28931, 30318), new(6950, 15447, 18952, 22681, 25567, 28563), new(7560, 23474, 25490, 27203, 28921, 30708), new(18544, 22373, 24457, 26195, 28119, 30045), new(31198, 31451, 31670, 31882, 32123, 32391)], + [new(21689, 23883, 25163, 26352, 27506, 28827, 30195), new(6892, 15385, 17840, 21606, 24287, 26753, 29204), new(5651, 23182, 25042, 26518, 27982, 29392, 30900), new(19349, 22578, 24418, 25994, 27524, 29031, 30448), new(31028, 31270, 31504, 31705, 31927, 32153, 32392)] + ]; + + /// + /// Gets the chroma palette color-index distributions indexed by palette size minus two and spatial context. + /// + public static Av1Distribution[][] PaletteUvColorIndex => + [ + [new(29089), new(16384), new(8713), new(29257), new(31610)], + [new(25257, 29145), new(12287, 27293), new(7033, 27960), new(20145, 25405), new(30608, 31639)], + [new(24210, 27175, 29903), new(9888, 22386, 27214), new(5901, 26053, 29293), new(18318, 22152, 28333), new(30459, 31136, 31926)], + [new(22980, 25479, 27781, 29986), new(8413, 21408, 24859, 28874), new(2257, 29449, 30594, 31598), new(19189, 21202, 25915, 28620), new(31844, 32044, 32281, 32518)], + [new(22217, 24567, 26637, 28683, 30548), new(7307, 16406, 19636, 24632, 28424), new(4441, 25064, 26879, 28942, 30919), new(17210, 20528, 23319, 26750, 29582), new(30674, 30953, 31396, 31735, 32207)], + [new(21239, 23168, 25044, 26962, 28705, 30506), new(6545, 15012, 18004, 21817, 25503, 28701), new(3448, 26295, 27437, 28704, 30126, 31442), new(15889, 18323, 21704, 24698, 26976, 29690), new(30988, 31204, 31479, 31734, 31983, 32325)], + [new(21442, 23288, 24758, 26246, 27649, 28980, 30563), new(5863, 14933, 17552, 20668, 23683, 26411, 29273), new(3415, 25810, 26877, 27990, 29223, 30394, 31618), new(17965, 20084, 22232, 23974, 26274, 28402, 30390), new(31190, 31329, 31516, 31679, 31825, 32026, 32322)] + ]; + + /// + /// Gets the partition-type distributions indexed by block-size and neighboring split context. + /// + public static Av1Distribution[] PartitionTypes => + [ + new(19132, 25510, 30392), + new(13928, 19855, 28540), + new(12522, 23679, 28629), + new(9896, 18783, 25853), + new(15597, 20929, 24571, 26706, 27664, 28821, 29601, 30571, 31902), + new(7925, 11043, 16785, 22470, 23971, 25043, 26651, 28701, 29834), + new(5414, 13269, 15111, 20488, 22360, 24500, 25537, 26336, 32117), + new(2662, 6362, 8614, 20860, 23053, 24778, 26436, 27829, 31171), + new(18462, 20920, 23124, 27647, 28227, 29049, 29519, 30178, 31544), + new(7689, 9060, 12056, 24992, 25660, 26182, 26951, 28041, 29052), + new(6015, 9009, 10062, 24544, 25409, 26545, 27071, 27526, 32047), + new(1394, 2208, 2796, 28614, 29061, 29466, 29840, 30185, 31899), + new(20137, 21547, 23078, 29566, 29837, 30261, 30524, 30892, 31724), + new(6732, 7490, 9497, 27944, 28250, 28515, 28969, 29630, 30104), + new(5945, 7663, 8348, 28683, 29117, 29749, 30064, 30298, 32238), + new(870, 1212, 1487, 31198, 31394, 31574, 31743, 31881, 32332), + new(27899, 28219, 28529, 32484, 32539, 32619, 32639), + new(6607, 6990, 8268, 32060, 32219, 32338, 32371), + new(5429, 6676, 7122, 32027, 32227, 32531, 32582), + new(711, 966, 1172, 32448, 32538, 32617, 32664) + ]; + + /// + /// Gets the transform-skip distributions indexed by the neighboring skip context. + /// + public static Av1Distribution[] Skip => [new(31671), new(16515), new(4576)]; + + /// + /// Gets the skip-mode distributions indexed by the neighboring skip-mode context. + /// + public static Av1Distribution[] SkipMode => [new(32621), new(20708), new(8127)]; + + /// + /// Gets the distribution for an absolute loop-filter delta magnitude. + /// + public static Av1Distribution DeltaLoopFilterAbsolute => new(28160, 32120, 32677); + + /// + /// Gets independent loop-filter delta distributions for vertical luma, horizontal luma, U, and V. + /// + public static Av1Distribution[] DeltaLoopFilterMultiAbsolute => + [ + new(28160, 32120, 32677), + new(28160, 32120, 32677), + new(28160, 32120, 32677), + new(28160, 32120, 32677) + ]; + + /// + /// Gets the distribution for an absolute quantizer delta magnitude. + /// + public static Av1Distribution DeltaQuantizerAbsolute => new(28160, 32120, 32677); + + /// + /// Gets the spatially predicted segment-identifier distributions indexed by neighboring segment context. + /// + public static Av1Distribution[] SegmentId => + [ + new(5622, 7893, 16093, 18233, 27809, 28373, 32533), + new(14274, 18230, 22557, 24935, 29980, 30851, 32344), + new(27527, 28487, 28723, 28890, 32397, 32647, 32679), + ]; + + /// + /// Gets the temporal segment-map prediction distributions indexed by the predicted state of the above and left blocks. + /// + /// + /// AV1 initializes each binary context to 16384, the equiprobable midpoint of its Q15 probability domain. + /// + public static Av1Distribution[] SegmentIdPredicted => [new(16384), new(16384), new(16384)]; + + /// + /// Gets the key-frame luma intra-mode distributions indexed by the above and left mode contexts. + /// + public static Av1Distribution[][] KeyFrameYMode => + [ + [ + new(15588, 17027, 19338, 20218, 20682, 21110, 21825, 23244, 24189, 28165, 29093, 30466), + new(12016, 18066, 19516, 20303, 20719, 21444, 21888, 23032, 24434, 28658, 30172, 31409), + new(10052, 10771, 22296, 22788, 23055, 23239, 24133, 25620, 26160, 29336, 29929, 31567), + new(14091, 15406, 16442, 18808, 19136, 19546, 19998, 22096, 24746, 29585, 30958, 32462), + new(12122, 13265, 15603, 16501, 18609, 20033, 22391, 25583, 26437, 30261, 31073, 32475), + ], [ + new(10023, 19585, 20848, 21440, 21832, 22760, 23089, 24023, 25381, 29014, 30482, 31436), + new(5983, 24099, 24560, 24886, 25066, 25795, 25913, 26423, 27610, 29905, 31276, 31794), + new(7444, 12781, 20177, 20728, 21077, 21607, 22170, 23405, 24469, 27915, 29090, 30492), + new(8537, 14689, 15432, 17087, 17408, 18172, 18408, 19825, 24649, 29153, 31096, 32210), + new(7543, 14231, 15496, 16195, 17905, 20717, 21984, 24516, 26001, 29675, 30981, 31994) + ], [ + new(12613, 13591, 21383, 22004, 22312, 22577, 23401, 25055, 25729, 29538, 30305, 32077), + new(9687, 13470, 18506, 19230, 19604, 20147, 20695, 22062, 23219, 27743, 29211, 30907), + new(6183, 6505, 26024, 26252, 26366, 26434, 27082, 28354, 28555, 30467, 30794, 32086), + new(10718, 11734, 14954, 17224, 17565, 17924, 18561, 21523, 23878, 28975, 30287, 32252), + new(9194, 9858, 16501, 17263, 18424, 19171, 21563, 25961, 26561, 30072, 30737, 32463) + ], [ + new(12602, 14399, 15488, 18381, 18778, 19315, 19724, 21419, 25060, 29696, 30917, 32409), + new(8203, 13821, 14524, 17105, 17439, 18131, 18404, 19468, 25225, 29485, 31158, 32342), + new(8451, 9731, 15004, 17643, 18012, 18425, 19070, 21538, 24605, 29118, 30078, 32018), + new(7714, 9048, 9516, 16667, 16817, 16994, 17153, 18767, 26743, 30389, 31536, 32528), + new(8843, 10280, 11496, 15317, 16652, 17943, 19108, 22718, 25769, 29953, 30983, 32485) + ], [ + new(12578, 13671, 15979, 16834, 19075, 20913, 22989, 25449, 26219, 30214, 31150, 32477), + new(9563, 13626, 15080, 15892, 17756, 20863, 22207, 24236, 25380, 29653, 31143, 32277), + new(8356, 8901, 17616, 18256, 19350, 20106, 22598, 25947, 26466, 29900, 30523, 32261), + new(10835, 11815, 13124, 16042, 17018, 18039, 18947, 22753, 24615, 29489, 30883, 32482), + new(7618, 8288, 9859, 10509, 15386, 18657, 22903, 28776, 29180, 31355, 31802, 32593) + ] + ]; + + /// + /// Gets the distribution selecting a filter-intra prediction mode. + /// + public static Av1Distribution FilterIntraMode => new(8949, 12776, 17211, 29558); + + /// + /// Gets the binary filter-intra enable distributions indexed by block size. + /// + public static Av1Distribution[] FilterIntra => + [ + new(4621), new(6743), new(5893), new(7866), new(12551), new(9394), + new(12408), new(14301), new(12756), new(22343), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384), new(12770), new(10368), + new(20229), new(18101), new(16384), new(16384) + ]; + + /// + /// Gets the transform-size distributions indexed by maximum transform category and neighboring transform-size context. + /// + public static Av1Distribution[][] TransformSize => + [ + [new(19968), new(19968), new(24320)], + [new(12272, 30172), new(12272, 30172), new(18677, 30848)], + [new(12986, 15180), new(12986, 15180), new(24302, 25602)], + [new(5782, 11475), new(5782, 11475), new(16803, 22759)], + ]; + + /// + /// Gets the binary variable-transform partition distributions indexed by transform geometry and neighbors. + /// + public static Av1Distribution[] TransformPartition => + [ + new(28581), new(23846), new(20847), new(24315), new(18196), new(12133), new(18791), + new(10887), new(11005), new(27179), new(20004), new(11281), new(26549), new(19308), + new(14224), new(28015), new(21546), new(14400), new(28165), new(22401), new(16088) + ]; + + /// + /// Gets the end-of-block token distributions for transforms containing at most 16 coefficients, indexed by + /// quantizer, plane, and transform-class contexts. + /// + private static Av1Distribution[][][] EndOfBlockFlagMulti16 { get; } = + [ + [ + [new(840, 1039, 1980, 4895), new(370, 671, 1883, 4471)], + [new(3247, 4950, 9688, 14563), new(1904, 3354, 7763, 14647)] + ], + [ + [new(2125, 2551, 5165, 8946), new(513, 765, 1859, 6339)], + [new(7637, 9498, 14259, 19108), new(2497, 4096, 8866, 16993)] + ], + [ + [new(4016, 4897, 8881, 14968), new(716, 1105, 2646, 10056)], + [new(11139, 13270, 18241, 23566), new(3192, 5032, 10297, 19755)] + ], + [ + [new(6708, 8958, 14746, 22133), new(1222, 2074, 4783, 15410)], + [new(19575, 21766, 26044, 29709), new(7297, 10767, 19273, 28194)] + ] + ]; + + /// + /// Gets the end-of-block token distributions for transforms containing at most 32 coefficients, indexed by + /// quantizer, plane, and transform-class contexts. + /// + private static Av1Distribution[][][] EndOfBlockFlagMulti32 { get; } = + [ + [ + [new(400, 520, 977, 2102, 6542), new(210, 405, 1315, 3326, 7537)], + [new(2636, 4273, 7588, 11794, 20401), new(1786, 3179, 6902, 11357, 19054)] + ], + [ + [new(989, 1249, 2019, 4151, 10785), new(313, 441, 1099, 2917, 8562)], + [new(8394, 10352, 13932, 18855, 26014), new(2578, 4124, 8181, 13670, 24234)] + ], + [ + [new(2515, 3003, 4452, 8162, 16041), new(574, 821, 1836, 5089, 13128)], + [new(13468, 16303, 20361, 25105, 29281), new(3542, 5502, 10415, 16760, 25644)] + ], + [ + [new(4617, 5709, 8446, 13584, 23135), new(1156, 1702, 3675, 9274, 20539)], + [new(22086, 24282, 27010, 29770, 31743), new(7699, 10897, 20891, 26926, 31628)] + ] + ]; + + /// + /// Gets the end-of-block token distributions for transforms containing at most 64 coefficients, indexed by + /// quantizer, plane, and transform-class contexts. + /// + private static Av1Distribution[][][] EndOfBlockFlagMulti64 { get; } = + [ + [ + [new(329, 498, 1101, 1784, 3265, 7758), new(335, 730, 1459, 5494, 8755, 12997)], + [new(3505, 5304, 10086, 13814, 17684, 23370), new(1563, 2700, 4876, 10911, 14706, 22480)], + ], + [ + [new(1260, 1446, 2253, 3712, 6652, 13369), new(401, 605, 1029, 2563, 5845, 12626)], + [new(8609, 10612, 14624, 18714, 22614, 29024), new(1923, 3127, 5867, 9703, 14277, 27100)] + ], + [ + [new(2374, 2772, 4583, 7276, 12288, 19706), new(497, 810, 1315, 3000, 7004, 15641)], + [new(15050, 17126, 21410, 24886, 28156, 30726), new(4034, 6290, 10235, 14982, 21214, 28491)] + ], + [ + [new(6307, 7541, 12060, 16358, 22553, 27865), new(1289, 2320, 3971, 7926, 14153, 24291)], + [new(24212, 25708, 28268, 30035, 31307, 32049), new(8726, 12378, 19409, 26450, 30038, 32462)] + ] + ]; + + /// + /// Gets the end-of-block token distributions for transforms containing at most 128 coefficients, indexed by + /// quantizer, plane, and transform-class contexts. + /// + private static Av1Distribution[][][] EndOfBlockFlagMulti128 { get; } = + [ + [ + [new(219, 482, 1140, 2091, 3680, 6028, 12586), new(371, 699, 1254, 4830, 9479, 12562, 17497)], + [new(5245, 7456, 12880, 15852, 20033, 23932, 27608), new(2054, 3472, 5869, 14232, 18242, 20590, 26752)] + ], + [ + [new(685, 933, 1488, 2714, 4766, 8562, 19254), new(217, 352, 618, 2303, 5261, 9969, 17472)], + [new(8045, 11200, 15497, 19595, 23948, 27408, 30938), new(2310, 4160, 7471, 14997, 17931, 20768, 30240)] + ], + [ + [new(1366, 1738, 2527, 5016, 9355, 15797, 24643), new(354, 558, 944, 2760, 7287, 14037, 21779)], + [new(13627, 16246, 20173, 24429, 27948, 30415, 31863), new(6275, 9889, 14769, 23164, 27988, 30493, 32272)] + ], + [ + [new(3472, 4885, 7489, 12481, 18517, 24536, 29635), new(886, 1731, 3271, 8469, 15569, 22126, 28383)], + [new(24313, 26062, 28385, 30107, 31217, 31898, 32345), new(9165, 13282, 21150, 30286, 31894, 32571, 32712)] + ] + ]; + + /// + /// Gets the end-of-block token distributions for transforms containing at most 256 coefficients, indexed by + /// quantizer, plane, and transform-class contexts. + /// + private static Av1Distribution[][][] EndOfBlockFlagMulti256 { get; } = + [ + [ + [ + new(310, 584, 1887, 3589, 6168, 8611, 11352, 15652), + new(998, 1850, 2998, 5604, 17341, 19888, 22899, 25583), + ], + [ + new(2520, 3240, 5952, 8870, 12577, 17558, 19954, 24168), + new(2203, 4130, 7435, 10739, 20652, 23681, 25609, 27261) + ], + ], + [ + [ + new(1448, 2109, 4151, 6263, 9329, 13260, 17944, 23300), + new(399, 1019, 1749, 3038, 10444, 15546, 22739, 27294) + ], + [ + new(6402, 8148, 12623, 15072, 18728, 22847, 26447, 29377), + new(1674, 3252, 5734, 10159, 22397, 23802, 24821, 30940) + ] + ], + [ + [ + new(3089, 3920, 6038, 9460, 14266, 19881, 25766, 29176), + new(1084, 2358, 3488, 5122, 11483, 18103, 26023, 29799) + ], + [ + new(11514, 13794, 17480, 20754, 24361, 27378, 29492, 31277), + new(6571, 9610, 15516, 21826, 29092, 30829, 31842, 32708) + ] + ], + [ + [ + new(5348, 7113, 11820, 15924, 22106, 26777, 30334, 31757), + new(2453, 4474, 6307, 8777, 16474, 22975, 29000, 31547) + ], + [ + new(23110, 24597, 27140, 28894, 30167, 30927, 31392, 32094), + new(9998, 17661, 25178, 28097, 31308, 32038, 32403, 32695) + ] + ] + ]; + + /// + /// Gets the end-of-block token distributions for transforms containing at most 512 coefficients, indexed by + /// quantizer, plane, and transform-class contexts. + /// + private static Av1Distribution[][][] EndOfBlockFlagMulti512 { get; } = + [ + [ + [ + new(641, 983, 3707, 5430, 10234, 14958, 18788, 23412, 26061), + new(3277, 6554, 9830, 13107, 16384, 19661, 22938, 26214, 29491) + ], + [ + new(5095, 6446, 9996, 13354, 16017, 17986, 20919, 26129, 29140), + new(3277, 6554, 9830, 13107, 16384, 19661, 22938, 26214, 29491) + ] + ], + [ + [ + new(1230, 2278, 5035, 7776, 11871, 15346, 19590, 24584, 28749), + new(3277, 6554, 9830, 13107, 16384, 19661, 22938, 26214, 29491) + ], + [ + new(7265, 9979, 15819, 19250, 21780, 23846, 26478, 28396, 31811), + new(3277, 6554, 9830, 13107, 16384, 19661, 22938, 26214, 29491) + ] + ], + [ + [ + new(2624, 3936, 6480, 9686, 13979, 17726, 23267, 28410, 31078), + new(3277, 6554, 9830, 13107, 16384, 19661, 22938, 26214, 29491) + ], + [ + new(12015, 14769, 19588, 22052, 24222, 25812, 27300, 29219, 32114), + new(3277, 6554, 9830, 13107, 16384, 19661, 22938, 26214, 29491) + ] + ], + [ + [ + new(5927, 7809, 10923, 14597, 19439, 24135, 28456, 31142, 32060), + new(3277, 6554, 9830, 13107, 16384, 19661, 22938, 26214, 29491) + ], + [ + new(21093, 23043, 25742, 27658, 29097, 29716, 30073, 30820, 31956), + new(3277, 6554, 9830, 13107, 16384, 19661, 22938, 26214, 29491) + ] + ] + ]; + + /// + /// Gets the end-of-block token distributions for transforms containing at most 1024 coded coefficients, indexed by + /// quantizer, plane, and transform-class contexts. + /// + private static Av1Distribution[][][] EndOfBlockFlagMulti1024 { get; } = + [ + [ + [ + new(393, 421, 751, 1623, 3160, 6352, 13345, 18047, 22571, 25830), + new(2979, 5958, 8937, 11916, 14895, 17873, 20852, 23831, 26810, 29789) + ], + [ + new(1865, 1988, 2930, 4242, 10533, 16538, 21354, 27255, 28546, 31784), + new(2979, 5958, 8937, 11916, 14895, 17873, 20852, 23831, 26810, 29789) + ] + ], + [ + [ + new(696, 948, 3145, 5702, 9706, 13217, 17851, 21856, 25692, 28034), + new(2979, 5958, 8937, 11916, 14895, 17873, 20852, 23831, 26810, 29789) + ], + [ + new(2672, 3591, 9330, 17084, 22725, 24284, 26527, 28027, 28377, 30876), + new(2979, 5958, 8937, 11916, 14895, 17873, 20852, 23831, 26810, 29789) + ] + ], + [ + [ + new(2784, 3831, 7041, 10521, 14847, 18844, 23155, 26682, 29229, 31045), + new(2979, 5958, 8937, 11916, 14895, 17873, 20852, 23831, 26810, 29789) + ], + [ + new(9577, 12466, 17739, 20750, 22061, 23215, 24601, 25483, 25843, 32056), + new(2979, 5958, 8937, 11916, 14895, 17873, 20852, 23831, 26810, 29789) + ] + ], + [ + [ + new(6698, 8334, 11961, 15762, 20186, 23862, 27434, 29326, 31082, 32050), + new(2979, 5958, 8937, 11916, 14895, 17873, 20852, 23831, 26810, 29789) + ], + [ + new(20569, 22426, 25569, 26859, 28053, 28913, 29486, 29724, 29807, 32570), + new(2979, 5958, 8937, 11916, 14895, 17873, 20852, 23831, 26810, 29789) + ] + ] + ]; + + /// + /// Gets the coefficient base-range distributions indexed by quantizer, transform-size, plane, and base-range + /// contexts. + /// + private static Av1Distribution[][][][] CoefficientsBaseRange { get; } = + [ + [ + [ + [ + new(14298, 20718, 24174), new(12536, 19601, 23789), new(8712, 15051, 19503), + new(6170, 11327, 15434), new(4742, 8926, 12538), new(3803, 7317, 10546), + new(1696, 3317, 4871), new(14392, 19951, 22756), new(15978, 23218, 26818), + new(12187, 19474, 23889), new(9176, 15640, 20259), new(7068, 12655, 17028), + new(5656, 10442, 14472), new(2580, 4992, 7244), new(12136, 18049, 21426), + new(13784, 20721, 24481), new(10836, 17621, 21900), new(8372, 14444, 18847), + new(6523, 11779, 16000), new(5337, 9898, 13760), new(3034, 5860, 8462) + ], + [ + new(15967, 22905, 26286), new(13534, 20654, 24579), new(9504, 16092, 20535), + new(6975, 12568, 16903), new(5364, 10091, 14020), new(4357, 8370, 11857), + new(2506, 4934, 7218), new(23032, 28815, 30936), new(19540, 26704, 29719), + new(15158, 22969, 27097), new(11408, 18865, 23650), new(8885, 15448, 20250), + new(7108, 12853, 17416), new(4231, 8041, 11480), new(19823, 26490, 29156), + new(18890, 25929, 28932), new(15660, 23491, 27433), new(12147, 19776, 24488), + new(9728, 16774, 21649), new(7919, 14277, 19066), new(5440, 10170, 14185) + ] + ], + [ + [ + new(14406, 20862, 24414), new(11824, 18907, 23109), new(8257, 14393, 18803), + new(5860, 10747, 14778), new(4475, 8486, 11984), new(3606, 6954, 10043), + new(1736, 3410, 5048), new(14430, 20046, 22882), new(15593, 22899, 26709), + new(12102, 19368, 23811), new(9059, 15584, 20262), new(6999, 12603, 17048), + new(5684, 10497, 14553), new(2822, 5438, 7862), new(15785, 21585, 24359), + new(18347, 25229, 28266), new(14974, 22487, 26389), new(11423, 18681, 23271), + new(8863, 15350, 20008), new(7153, 12852, 17278), new(3707, 7036, 9982) + ], + [ + new(15460, 21696, 25469), new(12170, 19249, 23191), new(8723, 15027, 19332), + new(6428, 11704, 15874), new(4922, 9292, 13052), new(4139, 7695, 11010), + new(2291, 4508, 6598), new(19856, 26920, 29828), new(17923, 25289, 28792), + new(14278, 21968, 26297), new(10910, 18136, 22950), new(8423, 14815, 19627), + new(6771, 12283, 16774), new(4074, 7750, 11081), new(19852, 26074, 28672), + new(19371, 26110, 28989), new(16265, 23873, 27663), new(12758, 20378, 24952), + new(10095, 17098, 21961), new(8250, 14628, 19451), new(5205, 9745, 13622) + ] + ], + [ + [ + new(10563, 16233, 19763), new(9794, 16022, 19804), new(6750, 11945, 15759), + new(4963, 9186, 12752), new(3845, 7435, 10627), new(3051, 6085, 8834), + new(1311, 2596, 3830), new(11246, 16404, 19689), new(12315, 18911, 22731), + new(10557, 17095, 21289), new(8136, 14006, 18249), new(6348, 11474, 15565), + new(5196, 9655, 13400), new(2349, 4526, 6587), new(13337, 18730, 21569), + new(19306, 26071, 28882), new(15952, 23540, 27254), new(12409, 19934, 24430), + new(9760, 16706, 21389), new(8004, 14220, 18818), new(4138, 7794, 10961) + ], + [ + new(10870, 16684, 20949), new(9664, 15230, 18680), new(6886, 12109, 15408), + new(4825, 8900, 12305), new(3630, 7162, 10314), new(3036, 6429, 9387), + new(1671, 3296, 4940), new(13819, 19159, 23026), new(11984, 19108, 23120), + new(10690, 17210, 21663), new(7984, 14154, 18333), new(6868, 12294, 16124), + new(5274, 8994, 12868), new(2988, 5771, 8424), new(19736, 26647, 29141), + new(18933, 26070, 28984), new(15779, 23048, 27200), new(12638, 20061, 24532), + new(10692, 17545, 22220), new(9217, 15251, 20054), new(5078, 9284, 12594) + ] + ], + [ + [ + new(2331, 3662, 5244), new(2891, 4771, 6145), new(4598, 7623, 9729), + new(3520, 6845, 9199), new(3417, 6119, 9324), new(2601, 5412, 7385), + new(600, 1173, 1744), new(7672, 13286, 17469), new(4232, 7792, 10793), + new(2915, 5317, 7397), new(2318, 4356, 6152), new(2127, 4000, 5554), + new(1850, 3478, 5275), new(977, 1933, 2843), new(18280, 24387, 27989), + new(15852, 22671, 26185), new(13845, 20951, 24789), new(11055, 17966, 22129), + new(9138, 15422, 19801), new(7454, 13145, 17456), new(3370, 6393, 9013) + ], + [ + new(5842, 9229, 10838), new(2313, 3491, 4276), new(2998, 6104, 7496), + new(2420, 7447, 9868), new(3034, 8495, 10923), new(4076, 8937, 10975), + new(1086, 2370, 3299), new(9714, 17254, 20444), new(8543, 13698, 17123), + new(4918, 9007, 11910), new(4129, 7532, 10553), new(2364, 5533, 8058), + new(1834, 3546, 5563), new(1473, 2908, 4133), new(15405, 21193, 25619), + new(15691, 21952, 26561), new(12962, 19194, 24165), new(10272, 17855, 22129), + new(8588, 15270, 20718), new(8682, 14669, 19500), new(4870, 9636, 13205) + ] + ], + [ + [ + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576) + ], + [ + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576) + ] + ] + ], + [ + [ + [ + new(14995, 21341, 24749), new(13158, 20289, 24601), new(8941, 15326, 19876), + new(6297, 11541, 15807), new(4817, 9029, 12776), new(3731, 7273, 10627), + new(1847, 3617, 5354), new(14472, 19659, 22343), new(16806, 24162, 27533), + new(12900, 20404, 24713), new(9411, 16112, 20797), new(7056, 12697, 17148), + new(5544, 10339, 14460), new(2954, 5704, 8319), new(12464, 18071, 21354), + new(15482, 22528, 26034), new(12070, 19269, 23624), new(8953, 15406, 20106), + new(7027, 12730, 17220), new(5887, 10913, 15140), new(3793, 7278, 10447) + ], + [ + new(15571, 22232, 25749), new(14506, 21575, 25374), new(10189, 17089, 21569), + new(7316, 13301, 17915), new(5783, 10912, 15190), new(4760, 9155, 13088), + new(2993, 5966, 8774), new(23424, 28903, 30778), new(20775, 27666, 30290), + new(16474, 24410, 28299), new(12471, 20180, 24987), new(9410, 16487, 21439), + new(7536, 13614, 18529), new(5048, 9586, 13549), new(21090, 27290, 29756), + new(20796, 27402, 30026), new(17819, 25485, 28969), new(13860, 21909, 26462), + new(11002, 18494, 23529), new(8953, 15929, 20897), new(6448, 11918, 16454) + ] + ], + [ + [ + new(15999, 22208, 25449), new(13050, 19988, 24122), new(8594, 14864, 19378), + new(6033, 11079, 15238), new(4554, 8683, 12347), new(3672, 7139, 10337), + new(1900, 3771, 5576), new(15788, 21340, 23949), new(16825, 24235, 27758), + new(12873, 20402, 24810), new(9590, 16363, 21094), new(7352, 13209, 17733), + new(5960, 10989, 15184), new(3232, 6234, 9007), new(15761, 20716, 23224), + new(19318, 25989, 28759), new(15529, 23094, 26929), new(11662, 18989, 23641), + new(8955, 15568, 20366), new(7281, 13106, 17708), new(4248, 8059, 11440) + ], + [ + new(14899, 21217, 24503), new(13519, 20283, 24047), new(9429, 15966, 20365), + new(6700, 12355, 16652), new(5088, 9704, 13716), new(4243, 8154, 11731), + new(2702, 5364, 7861), new(22745, 28388, 30454), new(20235, 27146, 29922), + new(15896, 23715, 27637), new(11840, 19350, 24131), new(9122, 15932, 20880), + new(7488, 13581, 18362), new(5114, 9568, 13370), new(20845, 26553, 28932), + new(20981, 27372, 29884), new(17781, 25335, 28785), new(13760, 21708, 26297), + new(10975, 18415, 23365), new(9045, 15789, 20686), new(6130, 11199, 15423) + ] + ], + [ + [ + new(13549, 19724, 23158), new(11844, 18382, 22246), new(7919, 13619, 17773), + new(5486, 10143, 13946), new(4166, 7983, 11324), new(3364, 6506, 9427), + new(1598, 3160, 4674), new(15281, 20979, 23781), new(14939, 22119, 25952), + new(11363, 18407, 22812), new(8609, 14857, 19370), new(6737, 12184, 16480), + new(5506, 10263, 14262), new(2990, 5786, 8380), new(20249, 25253, 27417), + new(21070, 27518, 30001), new(16854, 24469, 28074), new(12864, 20486, 25000), + new(9962, 16978, 21778), new(8074, 14338, 19048), new(4494, 8479, 11906) + ], + [ + new(13960, 19617, 22829), new(11150, 17341, 21228), new(7150, 12964, 17190), + new(5331, 10002, 13867), new(4167, 7744, 11057), new(3480, 6629, 9646), + new(1883, 3784, 5686), new(18752, 25660, 28912), new(16968, 24586, 28030), + new(13520, 21055, 25313), new(10453, 17626, 22280), new(8386, 14505, 19116), + new(6742, 12595, 17008), new(4273, 8140, 11499), new(22120, 27827, 30233), + new(20563, 27358, 29895), new(17076, 24644, 28153), new(13362, 20942, 25309), + new(10794, 17965, 22695), new(9014, 15652, 20319), new(5708, 10512, 14497) + ] + ], + [ + [ + new(5705, 10930, 15725), new(7946, 12765, 16115), new(6801, 12123, 16226), + new(5462, 10135, 14200), new(4189, 8011, 11507), new(3191, 6229, 9408), + new(1057, 2137, 3212), new(10018, 17067, 21491), new(7380, 12582, 16453), + new(6068, 10845, 14339), new(5098, 9198, 12555), new(4312, 8010, 11119), + new(3700, 6966, 9781), new(1693, 3326, 4887), new(18757, 24930, 27774), + new(17648, 24596, 27817), new(14707, 22052, 26026), new(11720, 18852, 23292), + new(9357, 15952, 20525), new(7810, 13753, 18210), new(3879, 7333, 10328) + ], + [ + new(8278, 13242, 15922), new(10547, 15867, 18919), new(9106, 15842, 20609), + new(6833, 13007, 17218), new(4811, 9712, 13923), new(3985, 7352, 11128), + new(1688, 3458, 5262), new(12951, 21861, 26510), new(9788, 16044, 20276), + new(6309, 11244, 14870), new(5183, 9349, 12566), new(4389, 8229, 11492), + new(3633, 6945, 10620), new(3600, 6847, 9907), new(21748, 28137, 30255), + new(19436, 26581, 29560), new(16359, 24201, 27953), new(13961, 21693, 25871), + new(11544, 18686, 23322), new(9372, 16462, 20952), new(6138, 11210, 15390) + ] + ], + [ + [ + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576) + ], + [ + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576) + ] + ] + ], + [ + [ + [ + new(16138, 22223, 25509), new(15347, 22430, 26332), new(9614, 16736, 21332), + new(6600, 12275, 16907), new(4811, 9424, 13547), new(3748, 7809, 11420), + new(2254, 4587, 6890), new(15196, 20284, 23177), new(18317, 25469, 28451), + new(13918, 21651, 25842), new(10052, 17150, 21995), new(7499, 13630, 18587), + new(6158, 11417, 16003), new(4014, 7785, 11252), new(15048, 21067, 24384), + new(18202, 25346, 28553), new(14302, 22019, 26356), new(10839, 18139, 23166), + new(8715, 15744, 20806), new(7536, 13576, 18544), new(5413, 10335, 14498) + ], + [ + new(17394, 24501, 27895), new(15889, 23420, 27185), new(11561, 19133, 23870), + new(8285, 14812, 19844), new(6496, 12043, 16550), new(4771, 9574, 13677), + new(3603, 6830, 10144), new(21656, 27704, 30200), new(21324, 27915, 30511), + new(17327, 25336, 28997), new(13417, 21381, 26033), new(10132, 17425, 22338), + new(8580, 15016, 19633), new(5694, 11477, 16411), new(24116, 29780, 31450), + new(23853, 29695, 31591), new(20085, 27614, 30428), new(15326, 24335, 28575), + new(11814, 19472, 24810), new(10221, 18611, 24767), new(7689, 14558, 20321) + ] + ], + [ + [ + new(16214, 22380, 25770), new(14213, 21304, 25295), new(9213, 15823, 20455), + new(6395, 11758, 16139), new(4779, 9187, 13066), new(3821, 7501, 10953), + new(2293, 4567, 6795), new(15859, 21283, 23820), new(18404, 25602, 28726), + new(14325, 21980, 26206), new(10669, 17937, 22720), new(8297, 14642, 19447), + new(6746, 12389, 16893), new(4324, 8251, 11770), new(16532, 21631, 24475), + new(20667, 27150, 29668), new(16728, 24510, 28175), new(12861, 20645, 25332), + new(10076, 17361, 22417), new(8395, 14940, 19963), new(5731, 10683, 14912) + ], + [ + new(14433, 21155, 24938), new(14658, 21716, 25545), new(9923, 16824, 21557), + new(6982, 13052, 17721), new(5419, 10503, 15050), new(4852, 9162, 13014), + new(3271, 6395, 9630), new(22210, 27833, 30109), new(20750, 27368, 29821), + new(16894, 24828, 28573), new(13247, 21276, 25757), new(10038, 17265, 22563), + new(8587, 14947, 20327), new(5645, 11371, 15252), new(22027, 27526, 29714), + new(23098, 29146, 31221), new(19886, 27341, 30272), new(15609, 23747, 28046), + new(11993, 20065, 24939), new(9637, 18267, 23671), new(7625, 13801, 19144) + ] + ], + [ + [ + new(14438, 20798, 24089), new(12621, 19203, 23097), new(8177, 14125, 18402), + new(5674, 10501, 14456), new(4236, 8239, 11733), new(3447, 6750, 9806), + new(1986, 3950, 5864), new(16208, 22099, 24930), new(16537, 24025, 27585), + new(12780, 20381, 24867), new(9767, 16612, 21416), new(7686, 13738, 18398), + new(6333, 11614, 15964), new(3941, 7571, 10836), new(22819, 27422, 29202), + new(22224, 28514, 30721), new(17660, 25433, 28913), new(13574, 21482, 26002), + new(10629, 17977, 22938), new(8612, 15298, 20265), new(5607, 10491, 14596) + ], + [ + new(13569, 19800, 23206), new(13128, 19924, 23869), new(8329, 14841, 19403), + new(6130, 10976, 15057), new(4682, 8839, 12518), new(3656, 7409, 10588), + new(2577, 5099, 7412), new(22427, 28684, 30585), new(20913, 27750, 30139), + new(15840, 24109, 27834), new(12308, 20029, 24569), new(10216, 16785, 21458), + new(8309, 14203, 19113), new(6043, 11168, 15307), new(23166, 28901, 30998), + new(21899, 28405, 30751), new(18413, 26091, 29443), new(15233, 23114, 27352), + new(12683, 20472, 25288), new(10702, 18259, 23409), new(8125, 14464, 19226) + ] + ], + [ + [ + new(9040, 14786, 18360), new(9979, 15718, 19415), new(7913, 13918, 18311), + new(5859, 10889, 15184), new(4593, 8677, 12510), new(3820, 7396, 10791), + new(1730, 3471, 5192), new(11803, 18365, 22709), new(11419, 18058, 22225), + new(9418, 15774, 20243), new(7539, 13325, 17657), new(6233, 11317, 15384), + new(5137, 9656, 13545), new(2977, 5774, 8349), new(21207, 27246, 29640), + new(19547, 26578, 29497), new(16169, 23871, 27690), new(12820, 20458, 25018), + new(10224, 17332, 22214), new(8526, 15048, 19884), new(5037, 9410, 13118) + ], + [ + new(12339, 17329, 20140), new(13505, 19895, 23225), new(9847, 16944, 21564), + new(7280, 13256, 18348), new(4712, 10009, 14454), new(4361, 7914, 12477), + new(2870, 5628, 7995), new(20061, 25504, 28526), new(15235, 22878, 26145), + new(12985, 19958, 24155), new(9782, 16641, 21403), new(9456, 16360, 20760), + new(6855, 12940, 18557), new(5661, 10564, 15002), new(25656, 30602, 31894), + new(22570, 29107, 31092), new(18917, 26423, 29541), new(15940, 23649, 27754), + new(12803, 20581, 25219), new(11082, 18695, 23376), new(7939, 14373, 19005) + ] + ], + [ + [ + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576) + ], + [ + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576) + ] + ] + ], + [ + [ + [ + new(18315, 24289, 27551), new(16854, 24068, 27835), new(10140, 17927, 23173), + new(6722, 12982, 18267), new(4661, 9826, 14706), new(3832, 8165, 12294), + new(2795, 6098, 9245), new(17145, 23326, 26672), new(20733, 27680, 30308), + new(16032, 24461, 28546), new(11653, 20093, 25081), new(9290, 16429, 22086), + new(7796, 14598, 19982), new(6502, 12378, 17441), new(21681, 27732, 30320), + new(22389, 29044, 31261), new(19027, 26731, 30087), new(14739, 23755, 28624), + new(11358, 20778, 25511), new(10995, 18073, 24190), new(9162, 14990, 20617) + ], + [ + new(21425, 27952, 30388), new(18062, 25838, 29034), new(11956, 19881, 24808), + new(7718, 15000, 20980), new(5702, 11254, 16143), new(4898, 9088, 16864), + new(3679, 6776, 11907), new(23294, 30160, 31663), new(24397, 29896, 31836), + new(19245, 27128, 30593), new(13202, 19825, 26404), new(11578, 19297, 23957), + new(8073, 13297, 21370), new(5461, 10923, 19745), new(27367, 30521, 31934), + new(24904, 30671, 31940), new(23075, 28460, 31299), new(14400, 23658, 30417), + new(13885, 23882, 28325), new(14746, 22938, 27853), new(5461, 16384, 27307) + ] + ], + [ + [ + new(18274, 24813, 27890), new(15537, 23149, 27003), new(9449, 16740, 21827), + new(6700, 12498, 17261), new(4988, 9866, 14198), new(4236, 8147, 11902), + new(2867, 5860, 8654), new(17124, 23171, 26101), new(20396, 27477, 30148), + new(16573, 24629, 28492), new(12749, 20846, 25674), new(10233, 17878, 22818), + new(8525, 15332, 20363), new(6283, 11632, 16255), new(20466, 26511, 29286), + new(23059, 29174, 31191), new(19481, 27263, 30241), new(15458, 23631, 28137), + new(12416, 20608, 25693), new(10261, 18011, 23261), new(8016, 14655, 19666) + ], + [ + new(17616, 24586, 28112), new(15809, 23299, 27155), new(10767, 18890, 23793), + new(7727, 14255, 18865), new(6129, 11926, 16882), new(4482, 9704, 14861), + new(3277, 7452, 11522), new(22956, 28551, 30730), new(22724, 28937, 30961), + new(18467, 26324, 29580), new(13234, 20713, 25649), new(11181, 17592, 22481), + new(8291, 18358, 24576), new(7568, 11881, 14984), new(24948, 29001, 31147), + new(25674, 30619, 32151), new(20841, 26793, 29603), new(14669, 24356, 28666), + new(11334, 23593, 28219), new(8922, 14762, 22873), new(8301, 13544, 20535) + ] + ], + [ + [ + new(17113, 23733, 27081), new(14139, 21406, 25452), new(8552, 15002, 19776), + new(5871, 11120, 15378), new(4455, 8616, 12253), new(3469, 6910, 10386), + new(2255, 4553, 6782), new(18224, 24376, 27053), new(19290, 26710, 29614), + new(14936, 22991, 27184), new(11238, 18951, 23762), new(8786, 15617, 20588), + new(7317, 13228, 18003), new(5101, 9512, 13493), new(22639, 28222, 30210), + new(23216, 29331, 31307), new(19075, 26762, 29895), new(15014, 23113, 27457), + new(11938, 19857, 24752), new(9942, 17280, 22282), new(7167, 13144, 17752) + ], + [ + new(15820, 22738, 26488), new(13530, 20885, 25216), new(8395, 15530, 20452), + new(6574, 12321, 16380), new(5353, 10419, 14568), new(4613, 8446, 12381), + new(3440, 7158, 9903), new(24247, 29051, 31224), new(22118, 28058, 30369), + new(16498, 24768, 28389), new(12920, 21175, 26137), new(10730, 18619, 25352), + new(10187, 16279, 22791), new(9310, 14631, 22127), new(24970, 30558, 32057), + new(24801, 29942, 31698), new(22432, 28453, 30855), new(19054, 25680, 29580), + new(14392, 23036, 28109), new(12495, 20947, 26650), new(12442, 20326, 26214) + ] + ], + [ + [ + new(12162, 18785, 22648), new(12749, 19697, 23806), new(8580, 15297, 20346), + new(6169, 11749, 16543), new(4836, 9391, 13448), new(3821, 7711, 11613), + new(2228, 4601, 7070), new(16319, 24725, 28280), new(15698, 23277, 27168), + new(12726, 20368, 25047), new(9912, 17015, 21976), new(7888, 14220, 19179), + new(6777, 12284, 17018), new(4492, 8590, 12252), new(23249, 28904, 30947), + new(21050, 27908, 30512), new(17440, 25340, 28949), new(14059, 22018, 26541), + new(11288, 18903, 23898), new(9411, 16342, 21428), new(6278, 11588, 15944) + ], + [ + new(13981, 20067, 23226), new(16922, 23580, 26783), new(11005, 19039, 24487), + new(7389, 14218, 19798), new(5598, 11505, 17206), new(6090, 11213, 15659), + new(3820, 7371, 10119), new(21082, 26925, 29675), new(21262, 28627, 31128), + new(18392, 26454, 30437), new(14870, 22910, 27096), new(12620, 19484, 24908), + new(9290, 16553, 22802), new(6668, 14288, 20004), new(27704, 31055, 31949), + new(24709, 29978, 31788), new(21668, 29264, 31657), new(18295, 26968, 30074), + new(16399, 24422, 29313), new(14347, 23026, 28104), new(12370, 19806, 24477) + ] + ], + [ + [ + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576) + ], + [ + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576) + ] + ] + ] + ]; + + /// + /// Gets the coefficient base-level distributions indexed by quantizer, transform-size, plane, and nonzero-map + /// contexts. + /// + private static Av1Distribution[][][][] CoefficientsBase { get; } = + [ + [ + [ + [ + new(4034, 8930, 12727), new(18082, 29741, 31877), new(12596, 26124, 30493), + new(9446, 21118, 27005), new(6308, 15141, 21279), new(2463, 6357, 9783), + new(20667, 30546, 31929), new(13043, 26123, 30134), new(8151, 18757, 24778), + new(5255, 12839, 18632), new(2820, 7206, 11161), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(15736, 27553, 30604), new(11210, 23794, 28787), new(5947, 13874, 19701), + new(4215, 9323, 13891), new(2833, 6462, 10059), new(19605, 30393, 31582), + new(13523, 26252, 30248), new(8446, 18622, 24512), new(3818, 10343, 15974), + new(1481, 4117, 6796), new(22649, 31302, 32190), new(14829, 27127, 30449), + new(8313, 17702, 23304), new(3022, 8301, 12786), new(1536, 4412, 7184), + new(22354, 29774, 31372), new(14723, 25472, 29214), new(6673, 13745, 18662), + new(2068, 5766, 9322), new(8192, 16384, 24576), new(8192, 16384, 24576) + ], + [ + new(6302, 16444, 21761), new(23040, 31538, 32475), new(15196, 28452, 31496), + new(10020, 22946, 28514), new(6533, 16862, 23501), new(3538, 9816, 15076), + new(24444, 31875, 32525), new(15881, 28924, 31635), new(9922, 22873, 28466), + new(6527, 16966, 23691), new(4114, 11303, 17220), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(20201, 30770, 32209), new(14754, 28071, 31258), new(8378, 20186, 26517), + new(5916, 15299, 21978), new(4268, 11583, 17901), new(24361, 32025, 32581), + new(18673, 30105, 31943), new(10196, 22244, 27576), new(5495, 14349, 20417), + new(2676, 7415, 11498), new(24678, 31958, 32585), new(18629, 29906, 31831), + new(9364, 20724, 26315), new(4641, 12318, 18094), new(2758, 7387, 11579), + new(25433, 31842, 32469), new(18795, 29289, 31411), new(7644, 17584, 23592), + new(3408, 9014, 15047), new(8192, 16384, 24576), new(8192, 16384, 24576) + ] + ], + [ + [ + new(4536, 10072, 14001), new(25459, 31416, 32206), new(16605, 28048, 30818), + new(11008, 22857, 27719), new(6915, 16268, 22315), new(2625, 6812, 10537), + new(24257, 31788, 32499), new(16880, 29454, 31879), new(11958, 25054, 29778), + new(7916, 18718, 25084), new(3383, 8777, 13446), new(22720, 31603, 32393), + new(14960, 28125, 31335), new(9731, 22210, 27928), new(6304, 15832, 22277), + new(2910, 7818, 12166), new(20375, 30627, 32131), new(13904, 27284, 30887), + new(9368, 21558, 27144), new(5937, 14966, 21119), new(2667, 7225, 11319), + new(23970, 31470, 32378), new(17173, 29734, 32018), new(12795, 25441, 29965), + new(8981, 19680, 25893), new(4728, 11372, 16902), new(24287, 31797, 32439), + new(16703, 29145, 31696), new(10833, 23554, 28725), new(6468, 16566, 23057), + new(2415, 6562, 10278), new(26610, 32395, 32659), new(18590, 30498, 32117), + new(12420, 25756, 29950), new(7639, 18746, 24710), new(3001, 8086, 12347), + new(25076, 32064, 32580), new(17946, 30128, 32028), new(12024, 24985, 29378), + new(7517, 18390, 24304), new(3243, 8781, 13331), new(8192, 16384, 24576) + ], + [ + new(6037, 16771, 21957), new(24774, 31704, 32426), new(16830, 28589, 31056), + new(10602, 22828, 27760), new(6733, 16829, 23071), new(3250, 8914, 13556), + new(25582, 32220, 32668), new(18659, 30342, 32223), new(12546, 26149, 30515), + new(8420, 20451, 26801), new(4636, 12420, 18344), new(27581, 32362, 32639), + new(18987, 30083, 31978), new(11327, 24248, 29084), new(7264, 17719, 24120), + new(3995, 10768, 16169), new(25893, 31831, 32487), new(16577, 28587, 31379), + new(10189, 22748, 28182), new(6832, 17094, 23556), new(3708, 10110, 15334), + new(25904, 32282, 32656), new(19721, 30792, 32276), new(12819, 26243, 30411), + new(8572, 20614, 26891), new(5364, 14059, 20467), new(26580, 32438, 32677), + new(20852, 31225, 32340), new(12435, 25700, 29967), new(8691, 20825, 26976), + new(4446, 12209, 17269), new(27350, 32429, 32696), new(21372, 30977, 32272), + new(12673, 25270, 29853), new(9208, 20925, 26640), new(5018, 13351, 18732), + new(27351, 32479, 32713), new(21398, 31209, 32387), new(12162, 25047, 29842), + new(7896, 18691, 25319), new(4670, 12882, 18881), new(8192, 16384, 24576) + ] + ], + [ + [ + new(5487, 10460, 13708), new(21597, 28303, 30674), new(11037, 21953, 26476), + new(8147, 17962, 22952), new(5242, 13061, 18532), new(1889, 5208, 8182), + new(26774, 32133, 32590), new(17844, 29564, 31767), new(11690, 24438, 29171), + new(7542, 18215, 24459), new(2993, 8050, 12319), new(28023, 32328, 32591), + new(18651, 30126, 31954), new(12164, 25146, 29589), new(7762, 18530, 24771), + new(3492, 9183, 13920), new(27591, 32008, 32491), new(17149, 28853, 31510), + new(11485, 24003, 28860), new(7697, 18086, 24210), new(3075, 7999, 12218), + new(28268, 32482, 32654), new(19631, 31051, 32404), new(13860, 27260, 31020), + new(9605, 21613, 27594), new(4876, 12162, 17908), new(27248, 32316, 32576), + new(18955, 30457, 32075), new(11824, 23997, 28795), new(7346, 18196, 24647), + new(3403, 9247, 14111), new(29711, 32655, 32735), new(21169, 31394, 32417), + new(13487, 27198, 30957), new(8828, 21683, 27614), new(4270, 11451, 17038), + new(28708, 32578, 32731), new(20120, 31241, 32482), new(13692, 27550, 31321), + new(9418, 22514, 28439), new(4999, 13283, 19462), new(8192, 16384, 24576) + ], + [ + new(5673, 14302, 19711), new(26251, 30701, 31834), new(12782, 23783, 27803), + new(9127, 20657, 25808), new(6368, 16208, 21462), new(2465, 7177, 10822), + new(29961, 32563, 32719), new(18318, 29891, 31949), new(11361, 24514, 29357), + new(7900, 19603, 25607), new(4002, 10590, 15546), new(29637, 32310, 32595), + new(18296, 29913, 31809), new(10144, 21515, 26871), new(5358, 14322, 20394), + new(3067, 8362, 13346), new(28652, 32470, 32676), new(17538, 30771, 32209), + new(13924, 26882, 30494), new(10496, 22837, 27869), new(7236, 16396, 21621), + new(30743, 32687, 32746), new(23006, 31676, 32489), new(14494, 27828, 31120), + new(10174, 22801, 28352), new(6242, 15281, 21043), new(25817, 32243, 32720), + new(18618, 31367, 32325), new(13997, 28318, 31878), new(12255, 26534, 31383), + new(9561, 21588, 28450), new(28188, 32635, 32724), new(22060, 32365, 32728), + new(18102, 30690, 32528), new(14196, 28864, 31999), new(12262, 25792, 30865), + new(24176, 32109, 32628), new(18280, 29681, 31963), new(10205, 23703, 29664), + new(7889, 20025, 27676), new(6060, 16743, 23970), new(8192, 16384, 24576) + ] + ], + [ + [ + new(5141, 7096, 8260), new(27186, 29022, 29789), new(6668, 12568, 15682), + new(2172, 6181, 8638), new(1126, 3379, 4531), new(443, 1361, 2254), + new(26083, 31153, 32436), new(13486, 24603, 28483), new(6508, 14840, 19910), + new(3386, 8800, 13286), new(1530, 4322, 7054), new(29639, 32080, 32548), + new(15897, 27552, 30290), new(8588, 20047, 25383), new(4889, 13339, 19269), + new(2240, 6871, 10498), new(28165, 32197, 32517), new(20735, 30427, 31568), + new(14325, 24671, 27692), new(5119, 12554, 17805), new(1810, 5441, 8261), + new(31212, 32724, 32748), new(23352, 31766, 32545), new(14669, 27570, 31059), + new(8492, 20894, 27272), new(3644, 10194, 15204), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576) + ], + [ + new(2461, 7013, 9371), new(24749, 29600, 30986), new(9466, 19037, 22417), + new(3584, 9280, 14400), new(1505, 3929, 5433), new(677, 1500, 2736), + new(23987, 30702, 32117), new(13554, 24571, 29263), new(6211, 14556, 21155), + new(3135, 10972, 15625), new(2435, 7127, 11427), new(31300, 32532, 32550), + new(14757, 30365, 31954), new(4405, 11612, 18553), new(580, 4132, 7322), + new(1695, 10169, 14124), new(30008, 32282, 32591), new(19244, 30108, 31748), + new(11180, 24158, 29555), new(5650, 14972, 19209), new(2114, 5109, 8456), + new(31856, 32716, 32748), new(23012, 31664, 32572), new(13694, 26656, 30636), + new(8142, 19508, 26093), new(4253, 10955, 16724), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576) + ] + ], + [ + [ + new(601, 983, 1311), new(18725, 23406, 28087), new(5461, 8192, 10923), + new(3781, 15124, 21425), new(2587, 7761, 12072), new(106, 458, 810), + new(22282, 29710, 31894), new(8508, 20926, 25984), new(3726, 12713, 18083), + new(1620, 7112, 10893), new(729, 2236, 3495), new(30163, 32474, 32684), + new(18304, 30464, 32000), new(11443, 26526, 29647), new(6007, 15292, 21299), + new(2234, 6703, 8937), new(30954, 32177, 32571), new(17363, 29562, 31076), + new(9686, 22464, 27410), new(8192, 16384, 21390), new(1755, 8046, 11264), + new(31168, 32734, 32748), new(22486, 31441, 32471), new(12833, 25627, 29738), + new(6980, 17379, 23122), new(3111, 8887, 13479), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576) + ], + [ + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576) + ] + ] + ], + [ + [ + [ + new(6041, 11854, 15927), new(20326, 30905, 32251), new(14164, 26831, 30725), + new(9760, 20647, 26585), new(6416, 14953, 21219), new(2966, 7151, 10891), + new(23567, 31374, 32254), new(14978, 27416, 30946), new(9434, 20225, 26254), + new(6658, 14558, 20535), new(3916, 8677, 12989), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(18088, 29545, 31587), new(13062, 25843, 30073), new(8940, 16827, 22251), + new(7654, 13220, 17973), new(5733, 10316, 14456), new(22879, 31388, 32114), + new(15215, 27993, 30955), new(9397, 19445, 24978), new(3442, 9813, 15344), + new(1368, 3936, 6532), new(25494, 32033, 32406), new(16772, 27963, 30718), + new(9419, 18165, 23260), new(2677, 7501, 11797), new(1516, 4344, 7170), + new(26556, 31454, 32101), new(17128, 27035, 30108), new(8324, 15344, 20249), + new(1903, 5696, 9469), new(8192, 16384, 24576), new(8192, 16384, 24576) + ], + [ + new(8455, 19003, 24368), new(23563, 32021, 32604), new(16237, 29446, 31935), + new(10724, 23999, 29358), new(6725, 17528, 24416), new(3927, 10927, 16825), + new(26313, 32288, 32634), new(17430, 30095, 32095), new(11116, 24606, 29679), + new(7195, 18384, 25269), new(4726, 12852, 19315), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(22822, 31648, 32483), new(16724, 29633, 31929), new(10261, 23033, 28725), + new(7029, 17840, 24528), new(4867, 13886, 21502), new(25298, 31892, 32491), + new(17809, 29330, 31512), new(9668, 21329, 26579), new(4774, 12956, 18976), + new(2322, 7030, 11540), new(25472, 31920, 32543), new(17957, 29387, 31632), + new(9196, 20593, 26400), new(4680, 12705, 19202), new(2917, 8456, 13436), + new(26471, 32059, 32574), new(18458, 29783, 31909), new(8400, 19464, 25956), + new(3812, 10973, 17206), new(8192, 16384, 24576), new(8192, 16384, 24576) + ] + ], + [ + [ + new(6779, 13743, 17678), new(24806, 31797, 32457), new(17616, 29047, 31372), + new(11063, 23175, 28003), new(6521, 16110, 22324), new(2764, 7504, 11654), + new(25266, 32367, 32637), new(19054, 30553, 32175), new(12139, 25212, 29807), + new(7311, 18162, 24704), new(3397, 9164, 14074), new(25988, 32208, 32522), + new(16253, 28912, 31526), new(9151, 21387, 27372), new(5688, 14915, 21496), + new(2717, 7627, 12004), new(23144, 31855, 32443), new(16070, 28491, 31325), + new(8702, 20467, 26517), new(5243, 13956, 20367), new(2621, 7335, 11567), + new(26636, 32340, 32630), new(19990, 31050, 32341), new(13243, 26105, 30315), + new(8588, 19521, 25918), new(4717, 11585, 17304), new(25844, 32292, 32582), + new(19090, 30635, 32097), new(11963, 24546, 28939), new(6218, 16087, 22354), + new(2340, 6608, 10426), new(28046, 32576, 32694), new(21178, 31313, 32296), + new(13486, 26184, 29870), new(7149, 17871, 23723), new(2833, 7958, 12259), + new(27710, 32528, 32686), new(20674, 31076, 32268), new(12413, 24955, 29243), + new(6676, 16927, 23097), new(2966, 8333, 12919), new(8192, 16384, 24576) + ], + [ + new(8639, 19339, 24429), new(24404, 31837, 32525), new(16997, 29425, 31784), + new(11253, 24234, 29149), new(6751, 17394, 24028), new(3490, 9830, 15191), + new(26283, 32471, 32714), new(19599, 31168, 32442), new(13146, 26954, 30893), + new(8214, 20588, 26890), new(4699, 13081, 19300), new(28212, 32458, 32669), + new(18594, 30316, 32100), new(11219, 24408, 29234), new(6865, 17656, 24149), + new(3678, 10362, 16006), new(25825, 32136, 32616), new(17313, 29853, 32021), + new(11197, 24471, 29472), new(6947, 17781, 24405), new(3768, 10660, 16261), + new(27352, 32500, 32706), new(20850, 31468, 32469), new(14021, 27707, 31133), + new(8964, 21748, 27838), new(5437, 14665, 21187), new(26304, 32492, 32698), + new(20409, 31380, 32385), new(13682, 27222, 30632), new(8974, 21236, 26685), + new(4234, 11665, 16934), new(26273, 32357, 32711), new(20672, 31242, 32441), + new(14172, 27254, 30902), new(9870, 21898, 27275), new(5164, 13506, 19270), + new(26725, 32459, 32728), new(20991, 31442, 32527), new(13071, 26434, 30811), + new(8184, 20090, 26742), new(4803, 13255, 19895), new(8192, 16384, 24576) + ] + ], + [ + [ + new(7555, 14942, 18501), new(24410, 31178, 32287), new(14394, 26738, 30253), + new(8413, 19554, 25195), new(4766, 12924, 18785), new(2029, 5806, 9207), + new(26776, 32364, 32663), new(18732, 29967, 31931), new(11005, 23786, 28852), + new(6466, 16909, 23510), new(3044, 8638, 13419), new(29208, 32582, 32704), + new(20068, 30857, 32208), new(12003, 25085, 29595), new(6947, 17750, 24189), + new(3245, 9103, 14007), new(27359, 32465, 32669), new(19421, 30614, 32174), + new(11915, 25010, 29579), new(6950, 17676, 24074), new(3007, 8473, 13096), + new(29002, 32676, 32735), new(22102, 31849, 32576), new(14408, 28009, 31405), + new(9027, 21679, 27931), new(4694, 12678, 18748), new(28216, 32528, 32682), + new(20849, 31264, 32318), new(12756, 25815, 29751), new(7565, 18801, 24923), + new(3509, 9533, 14477), new(30133, 32687, 32739), new(23063, 31910, 32515), + new(14588, 28051, 31132), new(9085, 21649, 27457), new(4261, 11654, 17264), + new(29518, 32691, 32748), new(22451, 31959, 32613), new(14864, 28722, 31700), + new(9695, 22964, 28716), new(4932, 13358, 19502), new(8192, 16384, 24576) + ], + [ + new(6465, 16958, 21688), new(25199, 31514, 32360), new(14774, 27149, 30607), + new(9257, 21438, 26972), new(5723, 15183, 21882), new(3150, 8879, 13731), + new(26989, 32262, 32682), new(17396, 29937, 32085), new(11387, 24901, 29784), + new(7289, 18821, 25548), new(3734, 10577, 16086), new(29728, 32501, 32695), + new(17431, 29701, 31903), new(9921, 22826, 28300), new(5896, 15434, 22068), + new(3430, 9646, 14757), new(28614, 32511, 32705), new(19364, 30638, 32263), + new(13129, 26254, 30402), new(8754, 20484, 26440), new(4378, 11607, 17110), + new(30292, 32671, 32744), new(21780, 31603, 32501), new(14314, 27829, 31291), + new(9611, 22327, 28263), new(4890, 13087, 19065), new(25862, 32567, 32733), + new(20794, 32050, 32567), new(17243, 30625, 32254), new(13283, 27628, 31474), + new(9669, 22532, 28918), new(27435, 32697, 32748), new(24922, 32390, 32714), + new(21449, 31504, 32536), new(16392, 29729, 31832), new(11692, 24884, 29076), + new(24193, 32290, 32735), new(18909, 31104, 32563), new(12236, 26841, 31403), + new(8171, 21840, 29082), new(7224, 17280, 25275), new(8192, 16384, 24576) + ] + ], + [ + [ + new(3078, 6839, 9890), new(13837, 20450, 24479), new(5914, 14222, 19328), + new(3866, 10267, 14762), new(2612, 7208, 11042), new(1067, 2991, 4776), + new(25817, 31646, 32529), new(13708, 26338, 30385), new(7328, 18585, 24870), + new(4691, 13080, 19276), new(1825, 5253, 8352), new(29386, 32315, 32624), + new(17160, 29001, 31360), new(9602, 21862, 27396), new(5915, 15772, 22148), + new(2786, 7779, 12047), new(29246, 32450, 32663), new(18696, 29929, 31818), + new(10510, 23369, 28560), new(6229, 16499, 23125), new(2608, 7448, 11705), + new(30753, 32710, 32748), new(21638, 31487, 32503), new(12937, 26854, 30870), + new(8182, 20596, 26970), new(3637, 10269, 15497), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576) + ], + [ + new(5244, 12150, 16906), new(20486, 26858, 29701), new(7756, 18317, 23735), + new(3452, 9256, 13146), new(2020, 5206, 8229), new(1801, 4993, 7903), + new(27051, 31858, 32531), new(15988, 27531, 30619), new(9188, 21484, 26719), + new(6273, 17186, 23800), new(3108, 9355, 14764), new(31076, 32520, 32680), + new(18119, 30037, 31850), new(10244, 22969, 27472), new(4692, 14077, 19273), + new(3694, 11677, 17556), new(30060, 32581, 32720), new(21011, 30775, 32120), + new(11931, 24820, 29289), new(7119, 17662, 24356), new(3833, 10706, 16304), + new(31954, 32731, 32748), new(23913, 31724, 32489), new(15520, 28060, 31286), + new(11517, 23008, 28571), new(6193, 14508, 20629), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576) + ] + ], + [ + [ + new(1035, 2807, 4156), new(13162, 18138, 20939), new(2696, 6633, 8755), + new(1373, 4161, 6853), new(1099, 2746, 4716), new(340, 1021, 1599), + new(22826, 30419, 32135), new(10395, 21762, 26942), new(4726, 12407, 17361), + new(2447, 7080, 10593), new(1227, 3717, 6011), new(28156, 31424, 31934), + new(16915, 27754, 30373), new(9148, 20990, 26431), new(5950, 15515, 21148), + new(2492, 7327, 11526), new(30602, 32477, 32670), new(20026, 29955, 31568), + new(11220, 23628, 28105), new(6652, 17019, 22973), new(3064, 8536, 13043), + new(31769, 32724, 32748), new(22230, 30887, 32373), new(12234, 25079, 29731), + new(7326, 18816, 25353), new(3933, 10907, 16616), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576) + ], + [ + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576) + ] + ] + ], + [ + [ + [ + new(8896, 16227, 20630), new(23629, 31782, 32527), new(15173, 27755, 31321), + new(10158, 21233, 27382), new(6420, 14857, 21558), new(3269, 8155, 12646), + new(24835, 32009, 32496), new(16509, 28421, 31579), new(10957, 21514, 27418), + new(7881, 15930, 22096), new(5388, 10960, 15918), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(20745, 30773, 32093), new(15200, 27221, 30861), new(13032, 20873, 25667), + new(12285, 18663, 23494), new(11563, 17481, 21489), new(26260, 31982, 32320), + new(15397, 28083, 31100), new(9742, 19217, 24824), new(3261, 9629, 15362), + new(1480, 4322, 7499), new(27599, 32256, 32460), new(16857, 27659, 30774), + new(9551, 18290, 23748), new(3052, 8933, 14103), new(2021, 5910, 9787), + new(29005, 32015, 32392), new(17677, 27694, 30863), new(9204, 17356, 23219), + new(2403, 7516, 12814), new(8192, 16384, 24576), new(8192, 16384, 24576) + ], + [ + new(10808, 22056, 26896), new(25739, 32313, 32676), new(17288, 30203, 32221), + new(11359, 24878, 29896), new(6949, 17767, 24893), new(4287, 11796, 18071), + new(27880, 32521, 32705), new(19038, 31004, 32414), new(12564, 26345, 30768), + new(8269, 19947, 26779), new(5674, 14657, 21674), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(25742, 32319, 32671), new(19557, 31164, 32454), new(13381, 26381, 30755), + new(10101, 21466, 26722), new(9209, 19650, 26825), new(27107, 31917, 32432), + new(18056, 28893, 31203), new(10200, 21434, 26764), new(4660, 12913, 19502), + new(2368, 6930, 12504), new(26960, 32158, 32613), new(18628, 30005, 32031), + new(10233, 22442, 28232), new(5471, 14630, 21516), new(3235, 10767, 17109), + new(27696, 32440, 32692), new(20032, 31167, 32438), new(8700, 21341, 28442), + new(5662, 14831, 21795), new(8192, 16384, 24576), new(8192, 16384, 24576) + ] + ], + [ + [ + new(9704, 17294, 21132), new(26762, 32278, 32633), new(18382, 29620, 31819), + new(10891, 23475, 28723), new(6358, 16583, 23309), new(3248, 9118, 14141), + new(27204, 32573, 32699), new(19818, 30824, 32329), new(11772, 25120, 30041), + new(6995, 18033, 25039), new(3752, 10442, 16098), new(27222, 32256, 32559), + new(15356, 28399, 31475), new(8821, 20635, 27057), new(5511, 14404, 21239), + new(2935, 8222, 13051), new(24875, 32120, 32529), new(15233, 28265, 31445), + new(8605, 20570, 26932), new(5431, 14413, 21196), new(2994, 8341, 13223), + new(28201, 32604, 32700), new(21041, 31446, 32456), new(13221, 26213, 30475), + new(8255, 19385, 26037), new(4930, 12585, 18830), new(28768, 32448, 32627), + new(19705, 30561, 32021), new(11572, 23589, 28220), new(5532, 15034, 21446), + new(2460, 7150, 11456), new(29874, 32619, 32699), new(21621, 31071, 32201), + new(12511, 24747, 28992), new(6281, 16395, 22748), new(3246, 9278, 14497), + new(29715, 32625, 32712), new(20958, 31011, 32283), new(11233, 23671, 28806), + new(6012, 16128, 22868), new(3427, 9851, 15414), new(8192, 16384, 24576) + ], + [ + new(11016, 22111, 26794), new(25946, 32357, 32677), new(17890, 30452, 32252), + new(11678, 25142, 29816), new(6720, 17534, 24584), new(4230, 11665, 17820), + new(28400, 32623, 32747), new(21164, 31668, 32575), new(13572, 27388, 31182), + new(8234, 20750, 27358), new(5065, 14055, 20897), new(28981, 32547, 32705), + new(18681, 30543, 32239), new(10919, 24075, 29286), new(6431, 17199, 24077), + new(3819, 10464, 16618), new(26870, 32467, 32693), new(19041, 30831, 32347), + new(11794, 25211, 30016), new(6888, 18019, 24970), new(4370, 12363, 18992), + new(29578, 32670, 32744), new(23159, 32007, 32613), new(15315, 28669, 31676), + new(9298, 22607, 28782), new(6144, 15913, 22968), new(28110, 32499, 32669), + new(21574, 30937, 32015), new(12759, 24818, 28727), new(6545, 16761, 23042), + new(3649, 10597, 16833), new(28163, 32552, 32728), new(22101, 31469, 32464), + new(13160, 25472, 30143), new(7303, 18684, 25468), new(5241, 13975, 20955), + new(28400, 32631, 32744), new(22104, 31793, 32603), new(13557, 26571, 30846), + new(7749, 19861, 26675), new(4873, 14030, 21234), new(8192, 16384, 24576) + ] + ], + [ + [ + new(9800, 17635, 21073), new(26153, 31885, 32527), new(15038, 27852, 31006), + new(8718, 20564, 26486), new(5128, 14076, 20514), new(2636, 7566, 11925), + new(27551, 32504, 32701), new(18310, 30054, 32100), new(10211, 23420, 29082), + new(6222, 16876, 23916), new(3462, 9954, 15498), new(29991, 32633, 32721), + new(19883, 30751, 32201), new(11141, 24184, 29285), new(6420, 16940, 23774), + new(3392, 9753, 15118), new(28465, 32616, 32712), new(19850, 30702, 32244), + new(10983, 24024, 29223), new(6294, 16770, 23582), new(3244, 9283, 14509), + new(30023, 32717, 32748), new(22940, 32032, 32626), new(14282, 27928, 31473), + new(8562, 21327, 27914), new(4846, 13393, 19919), new(29981, 32590, 32695), + new(20465, 30963, 32166), new(11479, 23579, 28195), new(5916, 15648, 22073), + new(3031, 8605, 13398), new(31146, 32691, 32739), new(23106, 31724, 32444), + new(13783, 26738, 30439), new(7852, 19468, 25807), new(3860, 11124, 16853), + new(31014, 32724, 32748), new(23629, 32109, 32628), new(14747, 28115, 31403), + new(8545, 21242, 27478), new(4574, 12781, 19067), new(8192, 16384, 24576) + ], + [ + new(9185, 19694, 24688), new(26081, 31985, 32621), new(16015, 29000, 31787), + new(10542, 23690, 29206), new(6732, 17945, 24677), new(3916, 11039, 16722), + new(28224, 32566, 32744), new(19100, 31138, 32485), new(12528, 26620, 30879), + new(7741, 20277, 26885), new(4566, 12845, 18990), new(29933, 32593, 32718), + new(17670, 30333, 32155), new(10385, 23600, 28909), new(6243, 16236, 22407), + new(3976, 10389, 16017), new(28377, 32561, 32738), new(19366, 31175, 32482), + new(13327, 27175, 31094), new(8258, 20769, 27143), new(4703, 13198, 19527), + new(31086, 32706, 32748), new(22853, 31902, 32583), new(14759, 28186, 31419), + new(9284, 22382, 28348), new(5585, 15192, 21868), new(28291, 32652, 32746), + new(19849, 32107, 32571), new(14834, 26818, 29214), new(10306, 22594, 28672), + new(6615, 17384, 23384), new(28947, 32604, 32745), new(25625, 32289, 32646), + new(18758, 28672, 31403), new(10017, 23430, 28523), new(6862, 15269, 22131), + new(23933, 32509, 32739), new(19927, 31495, 32631), new(11903, 26023, 30621), + new(7026, 20094, 27252), new(5998, 18106, 24437), new(8192, 16384, 24576) + ] + ], + [ + [ + new(4456, 11274, 15533), new(21219, 29079, 31616), new(11173, 23774, 28567), + new(7282, 18293, 24263), new(4890, 13286, 19115), new(1890, 5508, 8659), + new(26651, 32136, 32647), new(14630, 28254, 31455), new(8716, 21287, 27395), + new(5615, 15331, 22008), new(2675, 7700, 12150), new(29954, 32526, 32690), + new(16126, 28982, 31633), new(9030, 21361, 27352), new(5411, 14793, 21271), + new(2943, 8422, 13163), new(29539, 32601, 32730), new(18125, 30385, 32201), + new(10422, 24090, 29468), new(6468, 17487, 24438), new(2970, 8653, 13531), + new(30912, 32715, 32748), new(20666, 31373, 32497), new(12509, 26640, 30917), + new(8058, 20629, 27290), new(4231, 12006, 18052), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576) + ], + [ + new(10202, 20633, 25484), new(27336, 31445, 32352), new(12420, 24384, 28552), + new(7648, 18115, 23856), new(5662, 14341, 19902), new(3611, 10328, 15390), + new(30945, 32616, 32736), new(18682, 30505, 32253), new(11513, 25336, 30203), + new(7449, 19452, 26148), new(4482, 13051, 18886), new(32022, 32690, 32747), + new(18578, 30501, 32146), new(11249, 23368, 28631), new(5645, 16958, 22158), + new(5009, 11444, 16637), new(31357, 32710, 32748), new(21552, 31494, 32504), + new(13891, 27677, 31340), new(9051, 22098, 28172), new(5190, 13377, 19486), + new(32364, 32740, 32748), new(24839, 31907, 32551), new(17160, 28779, 31696), + new(12452, 24137, 29602), new(6165, 15389, 22477), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576) + ] + ], + [ + [ + new(2575, 7281, 11077), new(14002, 20866, 25402), new(6343, 15056, 19658), + new(4474, 11858, 17041), new(2865, 8299, 12534), new(1344, 3949, 6391), + new(24720, 31239, 32459), new(12585, 25356, 29968), new(7181, 18246, 24444), + new(5025, 13667, 19885), new(2521, 7304, 11605), new(29908, 32252, 32584), + new(17421, 29156, 31575), new(9889, 22188, 27782), new(5878, 15647, 22123), + new(2814, 8665, 13323), new(30183, 32568, 32713), new(18528, 30195, 32049), + new(10982, 24606, 29657), new(6957, 18165, 25231), new(3508, 10118, 15468), + new(31761, 32736, 32748), new(21041, 31328, 32546), new(12568, 26732, 31166), + new(8052, 20720, 27733), new(4336, 12192, 18396), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576) + ], + [ + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576) + ] + ] + ], + [ + [ + [ + new(7062, 16472, 22319), new(24538, 32261, 32674), new(13675, 28041, 31779), + new(8590, 20674, 27631), new(5685, 14675, 22013), new(3655, 9898, 15731), + new(26493, 32418, 32658), new(16376, 29342, 32090), new(10594, 22649, 28970), + new(8176, 17170, 24303), new(5605, 12694, 19139), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(23888, 31902, 32542), new(18612, 29687, 31987), new(16245, 24852, 29249), + new(15765, 22608, 27559), new(19895, 24699, 27510), new(28401, 32212, 32457), + new(15274, 27825, 30980), new(9364, 18128, 24332), new(2283, 8193, 15082), + new(1228, 3972, 7881), new(29455, 32469, 32620), new(17981, 28245, 31388), + new(10921, 20098, 26240), new(3743, 11829, 18657), new(2374, 9593, 15715), + new(31068, 32466, 32635), new(20321, 29572, 31971), new(10771, 20255, 27119), + new(2795, 10410, 17361), new(8192, 16384, 24576), new(8192, 16384, 24576) + ], + [ + new(9320, 22102, 27840), new(27057, 32464, 32724), new(16331, 30268, 32309), + new(10319, 23935, 29720), new(6189, 16448, 24106), new(3589, 10884, 18808), + new(29026, 32624, 32748), new(19226, 31507, 32587), new(12692, 26921, 31203), + new(7049, 19532, 27635), new(7727, 15669, 23252), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(28056, 32625, 32748), new(22383, 32075, 32669), new(15417, 27098, 31749), + new(18127, 26493, 27190), new(5461, 16384, 21845), new(27982, 32091, 32584), + new(19045, 29868, 31972), new(10397, 22266, 27932), new(5990, 13697, 21500), + new(1792, 6912, 15104), new(28198, 32501, 32718), new(21534, 31521, 32569), + new(11109, 25217, 30017), new(5671, 15124, 26151), new(4681, 14043, 18725), + new(28688, 32580, 32741), new(22576, 32079, 32661), new(10627, 22141, 28340), + new(9362, 14043, 28087), new(8192, 16384, 24576), new(8192, 16384, 24576) + ] + ], + [ + [ + new(7754, 16948, 22142), new(25670, 32330, 32691), new(15663, 29225, 31994), + new(9878, 23288, 29158), new(6419, 17088, 24336), new(3859, 11003, 17039), + new(27562, 32595, 32725), new(17575, 30588, 32399), new(10819, 24838, 30309), + new(7124, 18686, 25916), new(4479, 12688, 19340), new(28385, 32476, 32673), + new(15306, 29005, 31938), new(8937, 21615, 28322), new(5982, 15603, 22786), + new(3620, 10267, 16136), new(27280, 32464, 32667), new(15607, 29160, 32004), + new(9091, 22135, 28740), new(6232, 16632, 24020), new(4047, 11377, 17672), + new(29220, 32630, 32718), new(19650, 31220, 32462), new(13050, 26312, 30827), + new(9228, 20870, 27468), new(6146, 15149, 21971), new(30169, 32481, 32623), + new(17212, 29311, 31554), new(9911, 21311, 26882), new(4487, 13314, 20372), + new(2570, 7772, 12889), new(30924, 32613, 32708), new(19490, 30206, 32107), + new(11232, 23998, 29276), new(6769, 17955, 25035), new(4398, 12623, 19214), + new(30609, 32627, 32722), new(19370, 30582, 32287), new(10457, 23619, 29409), + new(6443, 17637, 24834), new(4645, 13236, 20106), new(8192, 16384, 24576) + ], + [ + new(8626, 20271, 26216), new(26707, 32406, 32711), new(16999, 30329, 32286), + new(11445, 25123, 30286), new(6411, 18828, 25601), new(6801, 12458, 20248), + new(29918, 32682, 32748), new(20649, 31739, 32618), new(12879, 27773, 31581), + new(7896, 21751, 28244), new(5260, 14870, 23698), new(29252, 32593, 32731), + new(17072, 30460, 32294), new(10653, 24143, 29365), new(6536, 17490, 23983), + new(4929, 13170, 20085), new(28137, 32518, 32715), new(18171, 30784, 32407), + new(11437, 25436, 30459), new(7252, 18534, 26176), new(4126, 13353, 20978), + new(31162, 32726, 32748), new(23017, 32222, 32701), new(15629, 29233, 32046), + new(9387, 22621, 29480), new(6922, 17616, 25010), new(28838, 32265, 32614), + new(19701, 30206, 31920), new(11214, 22410, 27933), new(5320, 14177, 23034), + new(5049, 12881, 17827), new(27484, 32471, 32734), new(21076, 31526, 32561), + new(12707, 26303, 31211), new(8169, 21722, 28219), new(6045, 19406, 27042), + new(27753, 32572, 32745), new(20832, 31878, 32653), new(13250, 27356, 31674), + new(7718, 21508, 29858), new(7209, 18350, 25559), new(8192, 16384, 24576) + ] + ], + [ + [ + new(7876, 16901, 21741), new(24001, 31898, 32625), new(14529, 27959, 31451), + new(8273, 20818, 27258), new(5278, 14673, 21510), new(2983, 8843, 14039), + new(28016, 32574, 32732), new(17471, 30306, 32301), new(10224, 24063, 29728), + new(6602, 17954, 25052), new(4002, 11585, 17759), new(30190, 32634, 32739), + new(17497, 30282, 32270), new(10229, 23729, 29538), new(6344, 17211, 24440), + new(3849, 11189, 17108), new(28570, 32583, 32726), new(17521, 30161, 32238), + new(10153, 23565, 29378), new(6455, 17341, 24443), new(3907, 11042, 17024), + new(30689, 32715, 32748), new(21546, 31840, 32610), new(13547, 27581, 31459), + new(8912, 21757, 28309), new(5548, 15080, 22046), new(30783, 32540, 32685), + new(17540, 29528, 31668), new(10160, 21468, 26783), new(4724, 13393, 20054), + new(2702, 8174, 13102), new(31648, 32686, 32742), new(20954, 31094, 32337), + new(12420, 25698, 30179), new(7304, 19320, 26248), new(4366, 12261, 18864), + new(31581, 32723, 32748), new(21373, 31586, 32525), new(12744, 26625, 30885), + new(7431, 20322, 26950), new(4692, 13323, 20111), new(8192, 16384, 24576) + ], + [ + new(7833, 18369, 24095), new(26650, 32273, 32702), new(16371, 29961, 32191), + new(11055, 24082, 29629), new(6892, 18644, 25400), new(5006, 13057, 19240), + new(29834, 32666, 32748), new(19577, 31335, 32570), new(12253, 26509, 31122), + new(7991, 20772, 27711), new(5677, 15910, 23059), new(30109, 32532, 32720), + new(16747, 30166, 32252), new(10134, 23542, 29184), new(5791, 16176, 23556), + new(4362, 10414, 17284), new(29492, 32626, 32748), new(19894, 31402, 32525), + new(12942, 27071, 30869), new(8346, 21216, 27405), new(6572, 17087, 23859), + new(32035, 32735, 32748), new(22957, 31838, 32618), new(14724, 28572, 31772), + new(10364, 23999, 29553), new(7004, 18433, 25655), new(27528, 32277, 32681), + new(16959, 31171, 32096), new(10486, 23593, 27962), new(8192, 16384, 23211), + new(8937, 17873, 20852), new(27715, 32002, 32615), new(15073, 29491, 31676), + new(11264, 24576, 28672), new(2341, 18725, 23406), new(7282, 18204, 25486), + new(28547, 32213, 32657), new(20788, 29773, 32239), new(6780, 21469, 30508), + new(5958, 14895, 23831), new(16384, 21845, 27307), new(8192, 16384, 24576) + ] + ], + [ + [ + new(5992, 14304, 19765), new(22612, 31238, 32456), new(13456, 27162, 31087), + new(8001, 20062, 26504), new(5168, 14105, 20764), new(2632, 7771, 12385), + new(27034, 32344, 32709), new(15850, 29415, 31997), new(9494, 22776, 28841), + new(6151, 16830, 23969), new(3461, 10039, 15722), new(30134, 32569, 32731), + new(15638, 29422, 31945), new(9150, 21865, 28218), new(5647, 15719, 22676), + new(3402, 9772, 15477), new(28530, 32586, 32735), new(17139, 30298, 32292), + new(10200, 24039, 29685), new(6419, 17674, 24786), new(3544, 10225, 15824), + new(31333, 32726, 32748), new(20618, 31487, 32544), new(12901, 27217, 31232), + new(8624, 21734, 28171), new(5104, 14191, 20748), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576) + ], + [ + new(11206, 21090, 26561), new(28759, 32279, 32671), new(14171, 27952, 31569), + new(9743, 22907, 29141), new(6871, 17886, 24868), new(4960, 13152, 19315), + new(31077, 32661, 32748), new(19400, 31195, 32515), new(12752, 26858, 31040), + new(8370, 22098, 28591), new(5457, 15373, 22298), new(31697, 32706, 32748), + new(17860, 30657, 32333), new(12510, 24812, 29261), new(6180, 19124, 24722), + new(5041, 13548, 17959), new(31552, 32716, 32748), new(21908, 31769, 32623), + new(14470, 28201, 31565), new(9493, 22982, 28608), new(6858, 17240, 24137), + new(32543, 32752, 32756), new(24286, 32097, 32666), new(15958, 29217, 32024), + new(10207, 24234, 29958), new(6929, 18305, 25652), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576) + ] + ], + [ + [ + new(4137, 10847, 15682), new(17824, 27001, 30058), new(10204, 22796, 28291), + new(6076, 15935, 22125), new(3852, 10937, 16816), new(2252, 6324, 10131), + new(25840, 32016, 32662), new(15109, 28268, 31531), new(9385, 22231, 28340), + new(6082, 16672, 23479), new(3318, 9427, 14681), new(30594, 32574, 32718), + new(16836, 29552, 31859), new(9556, 22542, 28356), new(6305, 16725, 23540), + new(3376, 9895, 15184), new(29383, 32617, 32745), new(18891, 30809, 32401), + new(11688, 25942, 30687), new(7468, 19469, 26651), new(3909, 11358, 17012), + new(31564, 32736, 32748), new(20906, 31611, 32600), new(13191, 27621, 31537), + new(8768, 22029, 28676), new(5079, 14109, 20906), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576) + ], + [ + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576), + new(8192, 16384, 24576), new(8192, 16384, 24576), new(8192, 16384, 24576) + ] + ] + ] + ]; + + /// + /// Gets the final-nonzero coefficient base-level distributions indexed by quantizer, transform-size, plane, and + /// end-of-block contexts. + /// + private static Av1Distribution[][][][] BaseEndOfBlock { get; } = + [ + [ + [ + [new(17837, 29055), new(29600, 31446), new(30844, 31878), new(24926, 28948)], + [new(21365, 30026), new(30512, 32423), new(31658, 32621), new(29630, 31881)] + ], + [ + [new(5717, 26477), new(30491, 31703), new(31550, 32158), new(29648, 31491)], + [new(12608, 27820), new(30680, 32225), new(30809, 32335), new(31299, 32423)] + ], + [ + [new(1786, 12612), new(30663, 31625), new(32339, 32468), new(31148, 31833)], + [new(18857, 23865), new(31428, 32428), new(31744, 32373), new(31775, 32526)] + ], + [ + [new(1787, 2532), new(30832, 31662), new(31824, 32682), new(32133, 32569)], + [new(13751, 22235), new(32089, 32409), new(27084, 27920), new(29291, 32594)] + ], + [ + [new(1725, 3449), new(31102, 31935), new(32457, 32613), new(32412, 32649)], + [new(10923, 21845), new(10923, 21845), new(10923, 21845), new(10923, 21845)] + ] + ], + [ + [ + [new(17560, 29888), new(29671, 31549), new(31007, 32056), new(27286, 30006)], + [new(26594, 31212), new(31208, 32582), new(31835, 32637), new(30595, 32206)] + ], + [ + [new(15239, 29932), new(31315, 32095), new(32130, 32434), new(30864, 31996)], + [new(26279, 30968), new(31142, 32495), new(31713, 32540), new(31929, 32594)] + ], + [ + [new(2644, 25198), new(32038, 32451), new(32639, 32695), new(32166, 32518)], + [new(17187, 27668), new(31714, 32550), new(32283, 32678), new(31930, 32563)] + ], + [ + [new(1044, 2257), new(30755, 31923), new(32208, 32693), new(32244, 32615)], + [new(21317, 26207), new(29133, 30868), new(29311, 31231), new(29657, 31087)] + ], + [ + [new(478, 1834), new(31005, 31987), new(32317, 32724), new(30865, 32648)], + [new(10923, 21845), new(10923, 21845), new(10923, 21845), new(10923, 21845)] + ] + ], + [ + [ + [new(20092, 30774), new(30695, 32020), new(31131, 32103), new(28666, 30870)], + [new(27258, 31095), new(31804, 32623), new(31763, 32528), new(31438, 32506)] + ], + [ + [new(18049, 30489), new(31706, 32286), new(32163, 32473), new(31550, 32184)], + [new(27116, 30842), new(31971, 32598), new(32088, 32576), new(32067, 32664)] + ], + [ + [new(12854, 29093), new(32272, 32558), new(32667, 32729), new(32306, 32585)], + [new(25476, 30366), new(32169, 32687), new(32479, 32689), new(31673, 32634)] + ], + [ + [new(2809, 19301), new(32205, 32622), new(32338, 32730), new(31786, 32616)], + [new(22737, 29105), new(30810, 32362), new(30014, 32627), new(30528, 32574)] + ], + [ + [new(935, 3382), new(30789, 31909), new(32466, 32756), new(30860, 32513)], + [new(10923, 21845), new(10923, 21845), new(10923, 21845), new(10923, 21845)] + ] + ], + [ + [ + [new(22497, 31198), new(31715, 32495), new(31606, 32337), new(30388, 31990)], + [new(27877, 31584), new(32170, 32728), new(32155, 32688), new(32219, 32702)] + ], + [ + [new(21457, 31043), new(31951, 32483), new(32153, 32562), new(31473, 32215)], + [new(27558, 31151), new(32020, 32640), new(32097, 32575), new(32242, 32719)] + ], + [ + [new(19980, 30591), new(32219, 32597), new(32581, 32706), new(31803, 32287)], + [new(26473, 30507), new(32431, 32723), new(32196, 32611), new(31588, 32528)] + ], + [ + [new(24647, 30463), new(32412, 32695), new(32468, 32720), new(31269, 32523)], + [new(28482, 31505), new(32152, 32701), new(31732, 32598), new(31767, 32712)] + ], + [ + [new(12358, 24977), new(31331, 32385), new(32634, 32756), new(30411, 32548)], + [new(10923, 21845), new(10923, 21845), new(10923, 21845), new(10923, 21845)] + ] + ] + ]; + + /// + /// Gets the DC coefficient-sign distributions indexed by quantizer, plane, and neighboring sign contexts. + /// + private static Av1Distribution[][][] DcSign { get; } = + [ + [ + [new(128 * 125), new(128 * 102), new(128 * 147)], + [new(128 * 119), new(128 * 101), new(128 * 135)] + ], + [ + [new(128 * 125), new(128 * 102), new(128 * 147)], + [new(128 * 119), new(128 * 101), new(128 * 135)] + ], + [ + [new(128 * 125), new(128 * 102), new(128 * 147)], + [new(128 * 119), new(128 * 101), new(128 * 135)] + ], + [ + [new(128 * 125), new(128 * 102), new(128 * 147)], + [new(128 * 119), new(128 * 101), new(128 * 135)] + ], + ]; + + /// + /// Gets the transform-block skip distributions indexed by quantizer, transform-size, and neighboring skip contexts. + /// + private static Av1Distribution[][][] TransformBlockSkip { get; } = + [ + [ + [ + new(31849), new(5892), new(12112), new(21935), new(20289), new(27473), new(32487), + new(7654), new(19473), new(29984), new(9961), new(30242), new(32117) + ], + [ + new(31548), new(1549), new(10130), new(16656), new(18591), new(26308), new(32537), + new(5403), new(18096), new(30003), new(16384), new(16384), new(16384) + ], + [ + new(29957), new(5391), new(18039), new(23566), new(22431), new(25822), new(32197), + new(3778), new(15336), new(28981), new(16384), new(16384), new(16384) + ], + [ + new(17920), new(1818), new(7282), new(25273), new(10923), new(31554), new(32624), + new(1366), new(15628), new(30462), new(146), new(5132), new(31657) + ], + [ + new(6308), new(117), new(1638), new(2161), new(16384), new(10923), new(30247), + new(16384), new(16384), new(16384), new(16384), new(16384), new(16384) + ] + ], + [ + [ + new(30371), new(7570), new(13155), new(20751), new(20969), new(27067), new(32013), + new(5495), new(17942), new(28280), new(16384), new(16384), new(16384) + ], + [ + new(31782), new(1836), new(10689), new(17604), new(21622), new(27518), new(32399), + new(4419), new(16294), new(28345), new(16384), new(16384), new(16384) + ], + [ + new(31901), new(10311), new(18047), new(24806), new(23288), new(27914), new(32296), + new(4215), new(15756), new(28341), new(16384), new(16384), new(16384) + ], + [ + new(26726), new(1045), new(11703), new(20590), new(18554), new(25970), new(31938), + new(5583), new(21313), new(29390), new(641), new(22265), new(31452) + ], + [ + new(26584), new(188), new(8847), new(24519), new(22938), new(30583), new(32608), + new(16384), new(16384), new(16384), new(16384), new(16384), new(16384) + ] + ], + [ + [ + new(29614), new(9068), new(12924), new(19538), new(17737), new(24619), new(30642), + new(4119), new(16026), new(25657), new(16384), new(16384), new(16384) + ], + [ + new(31957), new(3230), new(11153), new(18123), new(20143), new(26536), new(31986), + new(3050), new(14603), new(25155), new(16384), new(16384), new(16384) + ], + [ + new(32363), new(10692), new(19090), new(24357), new(24442), new(28312), new(32169), + new(3648), new(15690), new(26815), new(16384), new(16384), new(16384) + ], + [ + new(30669), new(3832), new(11663), new(18889), new(19782), new(23313), new(31330), + new(5124), new(18719), new(28468), new(3082), new(20982), new(29443) + ], + [ + new(28573), new(3183), new(17802), new(25977), new(26677), new(27832), new(32387), + new(16384), new(16384), new(16384), new(16384), new(16384), new(16384) + ] + ], + [ + [ + new(26887), new(6729), new(10361), new(17442), new(15045), new(22478), new(29072), + new(2713), new(11861), new(20773), new(16384), new(16384), new(16384) + ], + [ + new(31903), new(2044), new(7528), new(14618), new(16182), new(24168), new(31037), + new(2786), new(11194), new(20155), new(16384), new(16384), new(16384) + ], + [ + new(32510), new(8430), new(17318), new(24154), new(23674), new(28789), new(32139), + new(3440), new(13117), new(22702), new(16384), new(16384), new(16384) + ], + [ + new(31671), new(2056), new(11746), new(16852), new(18635), new(24715), new(31484), + new(4656), new(16074), new(24704), new(1806), new(14645), new(25336) + ], + [ + new(31539), new(8433), new(20576), new(27904), new(27852), new(30026), new(32441), + new(16384), new(16384), new(16384), new(16384), new(16384), new(16384) + ] + ] + ]; + + /// + /// Gets the end-of-block extra-bit distributions indexed by quantizer, transform-size, plane, and padded token + /// contexts. + /// + private static Av1Distribution[][][][] EndOfBlockExtra { get; } = + [ + [ + [ + [ + new(16384), new(16384), new(16384), new(16961), new(17223), new(7621), + new(16384), new(16384), new(16384), new(16384), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384) + ], + [ + new(16384), new(16384), new(16384), new(19069), new(22525), new(13377), + new(16384), new(16384), new(16384), new(16384), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384) + ] + ], + [ + [ + new(16384), new(16384), new(16384), new(20401), new(17025), new(12845), + new(12873), new(14094), new(16384), new(16384), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384) + ], + [ + new(16384), new(16384), new(16384), new(20681), new(20701), new(15250), + new(15017), new(14928), new(16384), new(16384), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384) + ] + ], + [ + [ + new(16384), new(16384), new(16384), new(23905), new(17194), new(16170), + new(17695), new(13826), new(15810), new(12036), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384) + ], + [ + new(16384), new(16384), new(16384), new(23959), new(20799), new(19021), + new(16203), new(17886), new(14144), new(12010), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384) + ] + ], + [ + [ + new(16384), new(16384), new(16384), new(27399), new(16327), new(18071), + new(19584), new(20721), new(18432), new(19560), new(10150), new(8805), + new(16384), new(16384), new(16384), new(16384), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384) + ], + [ + new(16384), new(16384), new(16384), new(24932), new(20833), new(12027), + new(16670), new(19914), new(15106), new(17662), new(13783), new(28756), + new(16384), new(16384), new(16384), new(16384), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384) + ] + ], + [ + [ + new(16384), new(16384), new(16384), new(23406), new(21845), new(18432), + new(16384), new(17096), new(12561), new(17320), new(22395), new(21370), + new(16384), new(16384), new(16384), new(16384), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384) + ], + [ + new(16384), new(16384), new(16384), new(16384), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384) + ] + ] + ], + [ + [ + [ + new(16384), new(16384), new(16384), new(17471), new(20223), new(11357), + new(16384), new(16384), new(16384), new(16384), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384) + ], + [ + new(16384), new(16384), new(16384), new(20335), new(21667), new(14818), + new(16384), new(16384), new(16384), new(16384), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384) + ] + ], + [ + [ + new(16384), new(16384), new(16384), new(20430), new(20662), new(15367), + new(16970), new(14657), new(16384), new(16384), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384) + ], + [ + new(16384), new(16384), new(16384), new(22117), new(22028), new(18650), + new(16042), new(15885), new(16384), new(16384), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384) + ] + ], + [ + [ + new(16384), new(16384), new(16384), new(22409), new(21012), new(15650), + new(17395), new(15469), new(20205), new(19511), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384) + ], + [ + new(16384), new(16384), new(16384), new(24220), new(22480), new(17737), + new(18916), new(19268), new(18412), new(18844), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384) + ] + ], + [ + [ + new(16384), new(16384), new(16384), new(25991), new(20314), new(17731), + new(19678), new(18649), new(17307), new(21798), new(17549), new(15630), + new(16384), new(16384), new(16384), new(16384), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384) + ], + [ + new(16384), new(16384), new(16384), new(26585), new(21469), new(20432), + new(17735), new(19280), new(15235), new(20297), new(22471), new(28997), + new(16384), new(16384), new(16384), new(16384), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384) + ] + ], + [ + [ + new(16384), new(16384), new(16384), new(26605), new(11304), new(16726), + new(16560), new(20866), new(23524), new(19878), new(13469), new(23084), + new(16384), new(16384), new(16384), new(16384), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384) + ], + [ + new(16384), new(16384), new(16384), new(16384), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384) + ] + ] + ], + [ + [ + [ + new(16384), new(16384), new(16384), new(18983), new(20512), new(14885), + new(16384), new(16384), new(16384), new(16384), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384) + ], + [ + new(16384), new(16384), new(16384), new(20090), new(19444), new(17286), + new(16384), new(16384), new(16384), new(16384), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384) + ] + ], + [ + [ + new(16384), new(16384), new(16384), new(19139), new(21487), new(18959), + new(20910), new(19089), new(16384), new(16384), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384) + ], + [ + new(16384), new(16384), new(16384), new(20536), new(20664), new(20625), + new(19123), new(14862), new(16384), new(16384), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384) + ] + ], + [ + [ + new(16384), new(16384), new(16384), new(19833), new(21502), new(17485), + new(20267), new(18353), new(23329), new(21478), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384) + ], + [ + new(16384), new(16384), new(16384), new(22041), new(23434), new(20001), + new(20554), new(20951), new(20145), new(15562), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384) + ] + ], + [ + [ + new(16384), new(16384), new(16384), new(23312), new(21607), new(16526), + new(18957), new(18034), new(18934), new(24247), new(16921), new(17080), + new(16384), new(16384), new(16384), new(16384), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384) + ], + [ + new(16384), new(16384), new(16384), new(26579), new(24910), new(18637), + new(19800), new(20388), new(9887), new(15642), new(30198), new(24721), + new(16384), new(16384), new(16384), new(16384), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384) + ] + ], + [ + [ + new(16384), new(16384), new(16384), new(26998), new(16737), new(17838), + new(18922), new(19515), new(18636), new(17333), new(15776), new(22658), + new(16384), new(16384), new(16384), new(16384), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384) + ], + [ + new(16384), new(16384), new(16384), new(16384), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384) + ] + ] + ], + [ + [ + [ + new(16384), new(16384), new(16384), new(20177), new(20789), new(20262), + new(16384), new(16384), new(16384), new(16384), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384) + ], + [ + new(16384), new(16384), new(16384), new(21416), new(20855), new(23410), + new(16384), new(16384), new(16384), new(16384), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384) + ] + ], + [ + [ + new(16384), new(16384), new(16384), new(20238), new(21057), new(19159), + new(22337), new(20159), new(16384), new(16384), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384) + ], + [ + new(16384), new(16384), new(16384), new(20125), new(20559), new(21707), + new(22296), new(17333), new(16384), new(16384), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384) + ] + ], + [ + [ + new(16384), new(16384), new(16384), new(19941), new(20527), new(21470), + new(22487), new(19558), new(22354), new(20331), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384) + ], + [ + new(16384), new(16384), new(16384), new(22752), new(25006), new(22075), + new(21576), new(17740), new(21690), new(19211), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384) + ] + ], + [ + [ + new(16384), new(16384), new(16384), new(21442), new(22358), new(18503), + new(20291), new(19945), new(21294), new(21178), new(19400), new(10556), + new(16384), new(16384), new(16384), new(16384), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384) + ], + [ + new(16384), new(16384), new(16384), new(24648), new(24949), new(20708), + new(23905), new(20501), new(9558), new(9423), new(30365), new(19253), + new(16384), new(16384), new(16384), new(16384), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384) + ] + ], + [ + [ + new(16384), new(16384), new(16384), new(26064), new(22098), new(19613), + new(20525), new(17595), new(16618), new(20497), new(18989), new(15513), + new(16384), new(16384), new(16384), new(16384), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384) + ], + [ + new(16384), new(16384), new(16384), new(16384), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384), new(16384), new(16384), + new(16384), new(16384), new(16384), new(16384) + ] + ] + ] + ]; + + /// + /// Gets the joint chroma-from-luma sign distribution for the U and V alpha values. + /// + public static Av1Distribution ChromaFromLumaSign => new(1418, 2123, 13340, 18405, 26972, 28343, 32294); + + /// + /// Gets the chroma-from-luma alpha-magnitude distributions indexed by joint-sign context. + /// + public static Av1Distribution[] ChromaFromLumaAlpha => + [ + new(7637, 20719, 31401, 32481, 32657, 32688, 32692, 32696, 32700, 32704, 32708, 32712, 32716, 32720, 32724), + new(14365, 23603, 28135, 31168, 32167, 32395, 32487, 32573, 32620, 32647, 32668, 32672, 32676, 32680, 32684), + new(11532, 22380, 28445, 31360, 32349, 32523, 32584, 32649, 32673, 32677, 32681, 32685, 32689, 32693, 32697), + new(26990, 31402, 32282, 32571, 32692, 32696, 32700, 32704, 32708, 32712, 32716, 32720, 32724, 32728, 32732), + new(17248, 26058, 28904, 30608, 31305, 31877, 32126, 32321, 32394, 32464, 32516, 32560, 32576, 32593, 32622), + new(14738, 21678, 25779, 27901, 29024, 30302, 30980, 31843, 32144, 32413, 32520, 32594, 32622, 32656, 32660) + ]; + + /// + /// Gets the inter transform-type distributions indexed by transform set and square transform-size group. + /// + public static Av1Distribution[][] InterExtendedTransform => + [ + [new(0), new(0), new(0), new(0)], + [ + new(4458, 5560, 7695, 9709, 13330, 14789, 17537, 20266, 21504, 22848, 23934, 25474, 27727, 28915, 30631), + new(1645, 2573, 4778, 5711, 7807, 8622, 10522, 15357, 17674, 20408, 22517, 25010, 27116, 28856, 30749), + new(2048, 4096, 6144, 8192, 10240, 12288, 14336, 16384, 18432, 20480, 22528, 24576, 26624, 28672, 30720), + new(2048, 4096, 6144, 8192, 10240, 12288, 14336, 16384, 18432, 20480, 22528, 24576, 26624, 28672, 30720) + ], + [ + new(2731, 5461, 8192, 10923, 13653, 16384, 19115, 21845, 24576, 27307, 30037), + new(2731, 5461, 8192, 10923, 13653, 16384, 19115, 21845, 24576, 27307, 30037), + new(770, 2421, 5225, 12907, 15819, 18927, 21561, 24089, 26595, 28526, 30529), + new(2731, 5461, 8192, 10923, 13653, 16384, 19115, 21845, 24576, 27307, 30037) + ], + [new(16384), new(4167), new(1998), new(748)] + ]; + + /// + /// Gets the intra transform-type distributions indexed by transform set, transform-size group, and intra prediction mode. + /// + public static Av1Distribution[][][] IntraExtendedTransform => + [ + [ + [ + new(0), + new(0), + new(0), + new(0), + new(0), + new(0), + new(0), + new(0), + new(0), + new(0), + new(0), + new(0), + new(0) + ], + [ + new(0), + new(0), + new(0), + new(0), + new(0), + new(0), + new(0), + new(0), + new(0), + new(0), + new(0), + new(0), + new(0) + ], + [ + new(0), + new(0), + new(0), + new(0), + new(0), + new(0), + new(0), + new(0), + new(0), + new(0), + new(0), + new(0), + new(0) + ], + [ + new(0), + new(0), + new(0), + new(0), + new(0), + new(0), + new(0), + new(0), + new(0), + new(0), + new(0), + new(0), + new(0) + ], + ], + [ + [ + new(1535, 8035, 9461, 12751, 23467, 27825), + new(564, 3335, 9709, 10870, 18143, 28094), + new(672, 3247, 3676, 11982, 19415, 23127), + new(5279, 13885, 15487, 18044, 23527, 30252), + new(4423, 6074, 7985, 10416, 25693, 29298), + new(1486, 4241, 9460, 10662, 16456, 27694), + new(439, 2838, 3522, 6737, 18058, 23754), + new(1190, 4233, 4855, 11670, 20281, 24377), + new(1045, 4312, 8647, 10159, 18644, 29335), + new(202, 3734, 4747, 7298, 17127, 24016), + new(447, 4312, 6819, 8884, 16010, 23858), + new(277, 4369, 5255, 8905, 16465, 22271), + new(3409, 5436, 10599, 15599, 19687, 24040) + ], + [ + new(1870, 13742, 14530, 16498, 23770, 27698), + new(326, 8796, 14632, 15079, 19272, 27486), + new(484, 7576, 7712, 14443, 19159, 22591), + new(1126, 15340, 15895, 17023, 20896, 30279), + new(655, 4854, 5249, 5913, 22099, 27138), + new(1299, 6458, 8885, 9290, 14851, 25497), + new(311, 5295, 5552, 6885, 16107, 22672), + new(883, 8059, 8270, 11258, 17289, 21549), + new(741, 7580, 9318, 10345, 16688, 29046), + new(110, 7406, 7915, 9195, 16041, 23329), + new(363, 7974, 9357, 10673, 15629, 24474), + new(153, 7647, 8112, 9936, 15307, 19996), + new(3511, 6332, 11165, 15335, 19323, 23594) + ], + [ + new(4681, 9362, 14043, 18725, 23406, 28087), + new(4681, 9362, 14043, 18725, 23406, 28087), + new(4681, 9362, 14043, 18725, 23406, 28087), + new(4681, 9362, 14043, 18725, 23406, 28087), + new(4681, 9362, 14043, 18725, 23406, 28087), + new(4681, 9362, 14043, 18725, 23406, 28087), + new(4681, 9362, 14043, 18725, 23406, 28087), + new(4681, 9362, 14043, 18725, 23406, 28087), + new(4681, 9362, 14043, 18725, 23406, 28087), + new(4681, 9362, 14043, 18725, 23406, 28087), + new(4681, 9362, 14043, 18725, 23406, 28087), + new(4681, 9362, 14043, 18725, 23406, 28087), + new(4681, 9362, 14043, 18725, 23406, 28087) + ], + [ + new(4681, 9362, 14043, 18725, 23406, 28087), + new(4681, 9362, 14043, 18725, 23406, 28087), + new(4681, 9362, 14043, 18725, 23406, 28087), + new(4681, 9362, 14043, 18725, 23406, 28087), + new(4681, 9362, 14043, 18725, 23406, 28087), + new(4681, 9362, 14043, 18725, 23406, 28087), + new(4681, 9362, 14043, 18725, 23406, 28087), + new(4681, 9362, 14043, 18725, 23406, 28087), + new(4681, 9362, 14043, 18725, 23406, 28087), + new(4681, 9362, 14043, 18725, 23406, 28087), + new(4681, 9362, 14043, 18725, 23406, 28087), + new(4681, 9362, 14043, 18725, 23406, 28087), + new(4681, 9362, 14043, 18725, 23406, 28087), + ], + ], + [ + [ + new(6554, 13107, 19661, 26214), + new(6554, 13107, 19661, 26214), + new(6554, 13107, 19661, 26214), + new(6554, 13107, 19661, 26214), + new(6554, 13107, 19661, 26214), + new(6554, 13107, 19661, 26214), + new(6554, 13107, 19661, 26214), + new(6554, 13107, 19661, 26214), + new(6554, 13107, 19661, 26214), + new(6554, 13107, 19661, 26214), + new(6554, 13107, 19661, 26214), + new(6554, 13107, 19661, 26214), + new(6554, 13107, 19661, 26214) + ], + [ + new(6554, 13107, 19661, 26214), + new(6554, 13107, 19661, 26214), + new(6554, 13107, 19661, 26214), + new(6554, 13107, 19661, 26214), + new(6554, 13107, 19661, 26214), + new(6554, 13107, 19661, 26214), + new(6554, 13107, 19661, 26214), + new(6554, 13107, 19661, 26214), + new(6554, 13107, 19661, 26214), + new(6554, 13107, 19661, 26214), + new(6554, 13107, 19661, 26214), + new(6554, 13107, 19661, 26214), + new(6554, 13107, 19661, 26214) + ], + [ + new(1127, 12814, 22772, 27483), + new(145, 6761, 11980, 26667), + new(362, 5887, 11678, 16725), + new(385, 15213, 18587, 30693), + new(25, 2914, 23134, 27903), + new(60, 4470, 11749, 23991), + new(37, 3332, 14511, 21448), + new(157, 6320, 13036, 17439), + new(119, 6719, 12906, 29396), + new(47, 5537, 12576, 21499), + new(269, 6076, 11258, 23115), + new(83, 5615, 12001, 17228), + new(1968, 5556, 12023, 18547) + ], + [ + new(6554, 13107, 19661, 26214), + new(6554, 13107, 19661, 26214), + new(6554, 13107, 19661, 26214), + new(6554, 13107, 19661, 26214), + new(6554, 13107, 19661, 26214), + new(6554, 13107, 19661, 26214), + new(6554, 13107, 19661, 26214), + new(6554, 13107, 19661, 26214), + new(6554, 13107, 19661, 26214), + new(6554, 13107, 19661, 26214), + new(6554, 13107, 19661, 26214), + new(6554, 13107, 19661, 26214), + new(6554, 13107, 19661, 26214) + ] + ], + ]; + + /// + /// Gets the complete set of end-of-block token distributions for a base quantizer. + /// + /// The frame base quantizer index. + /// The distributions indexed by coefficient-count category, plane, and transform-class contexts. + public static Av1Distribution[][][] GetEndOfBlockFlag(int baseQIndex) + { + // AV1 initializes coefficient models from one of four quantizer bands, then adapts each tile's copy. + int qContext = GetQContext(baseQIndex); + return + [ + Av1Distribution.CreateCopy(EndOfBlockFlagMulti16[qContext]), + Av1Distribution.CreateCopy(EndOfBlockFlagMulti32[qContext]), + Av1Distribution.CreateCopy(EndOfBlockFlagMulti64[qContext]), + Av1Distribution.CreateCopy(EndOfBlockFlagMulti128[qContext]), + Av1Distribution.CreateCopy(EndOfBlockFlagMulti256[qContext]), + Av1Distribution.CreateCopy(EndOfBlockFlagMulti512[qContext]), + Av1Distribution.CreateCopy(EndOfBlockFlagMulti1024[qContext]), + ]; + } + + /// + /// Gets the coefficient base-range distributions for a base quantizer. + /// + /// The frame base quantizer index. + /// The distributions indexed by transform-size, plane, and base-range contexts. + public static Av1Distribution[][][] GetCoefficientsBaseRange(int baseQIndex) + => Av1Distribution.CreateCopy(CoefficientsBaseRange[GetQContext(baseQIndex)]); + + /// + /// Gets the coefficient base-level distributions for a base quantizer. + /// + /// The frame base quantizer index. + /// The distributions indexed by transform-size, plane, and nonzero-map contexts. + public static Av1Distribution[][][] GetCoefficientsBase(int baseQIndex) + => Av1Distribution.CreateCopy(CoefficientsBase[GetQContext(baseQIndex)]); + + /// + /// Gets the final-nonzero coefficient base-level distributions for a base quantizer. + /// + /// The frame base quantizer index. + /// The distributions indexed by transform-size, plane, and end-of-block contexts. + public static Av1Distribution[][][] GetBaseEndOfBlock(int baseQIndex) + => Av1Distribution.CreateCopy(BaseEndOfBlock[GetQContext(baseQIndex)]); + + /// + /// Gets the DC sign distributions for a base quantizer. + /// + /// The frame base quantizer index. + /// The distributions indexed by plane and neighboring sign context. + public static Av1Distribution[][] GetDcSign(int baseQIndex) + => Av1Distribution.CreateCopy(DcSign[GetQContext(baseQIndex)]); + + /// + /// Gets the transform-block skip distributions for a base quantizer. + /// + /// The frame base quantizer index. + /// The distributions indexed by transform-size and neighboring skip context. + public static Av1Distribution[][] GetTransformBlockSkip(int baseQIndex) + => Av1Distribution.CreateCopy(TransformBlockSkip[GetQContext(baseQIndex)]); + + /// + /// Gets the end-of-block extra-bit distributions for a base quantizer. + /// + /// The frame base quantizer index. + /// The distributions indexed by transform-size, plane, and end-of-block token context. + public static Av1Distribution[][][] GetEndOfBlockExtra(int baseQIndex) + => Av1Distribution.CreateCopy(EndOfBlockExtra[GetQContext(baseQIndex)]); + + /// + /// Creates the uniform sixteen-symbol distribution used by block sizes that cannot signal a wedge index. + /// + private static Av1Distribution CreateUniformWedgeIndexDistribution() + => new(2048, 4096, 6144, 8192, 10240, 12288, 14336, 16384, 18432, 20480, 22528, 24576, 26624, 28672, 30720); + + /// + /// Maps a base quantizer index to one of the four AV1 coefficient-probability initialization bands. + /// + /// The base quantizer index. + /// The zero-based quantizer context. + private static int GetQContext(int q) + { + if (q <= 20) + { + return 0; + } + + if (q <= 60) + { + return 1; + } + + if (q <= 120) + { + return 2; + } + + return 3; + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Entropy/Av1Distribution.cs b/src/ImageSharp/Formats/Heif/Av1/Entropy/Av1Distribution.cs new file mode 100644 index 0000000000..d962e5a605 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Entropy/Av1Distribution.cs @@ -0,0 +1,423 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Entropy; + +/// +/// Stores and adapts an AV1 inverse cumulative distribution used by the range coder. +/// +internal sealed class Av1Distribution +{ + /// + /// The exclusive upper bound of the Q15 probability domain. + /// + public const int ProbabilityTop = 1 << ProbabilityBitCount; + + /// + /// The minimum sub-range reserved for each symbol during range coding. + /// + public const int ProbabilityMinimum = 4; + + /// + /// The shift that converts stored Q15 cumulative values to the range-coder precision. + /// + public const int CdfShift = 15 - ProbabilityBitCount; + + /// + /// The precision reduction applied before multiplying a cumulative value by the coding range. + /// + public const int ProbabilityShift = 6; + + /// + /// The number of fractional bits in a stored cumulative probability. + /// + private const int ProbabilityBitCount = 15; + + /// + /// The inverse cumulative thresholds followed by the required zero sentinel. + /// + private InlineArray16 probabilities; + + /// + /// The symbol-count contribution to the adaptive update rate. + /// + private readonly int speed; + + /// + /// The capped number of observations already incorporated into this distribution. + /// + private int updateCount; + + /// + /// Initializes a new instance of the class for a binary alphabet. + /// + /// The cumulative threshold following symbol zero. + public Av1Distribution(uint p0) + : this([p0, 0], 1) + { + } + + /// + /// Initializes a new instance of the class for a three-symbol alphabet. + /// + /// The cumulative threshold following symbol zero. + /// The cumulative threshold following symbol one. + public Av1Distribution(uint p0, uint p1) + : this([p0, p1, 0], 1) + { + } + + /// + /// Initializes a new instance of the class for a four-symbol alphabet. + /// + /// The cumulative threshold following symbol zero. + /// The cumulative threshold following symbol one. + /// The cumulative threshold following symbol two. + public Av1Distribution(uint p0, uint p1, uint p2) + : this([p0, p1, p2, 0], 2) + { + } + + /// + /// Initializes a new instance of the class for a five-symbol alphabet. + /// + /// The cumulative threshold following symbol zero. + /// The cumulative threshold following symbol one. + /// The cumulative threshold following symbol two. + /// The cumulative threshold following symbol three. + public Av1Distribution(uint p0, uint p1, uint p2, uint p3) + : this([p0, p1, p2, p3, 0], 2) + { + } + + /// + /// Initializes a new instance of the class for a six-symbol alphabet. + /// + /// The cumulative threshold following symbol zero. + /// The cumulative threshold following symbol one. + /// The cumulative threshold following symbol two. + /// The cumulative threshold following symbol three. + /// The cumulative threshold following symbol four. + public Av1Distribution(uint p0, uint p1, uint p2, uint p3, uint p4) + : this([p0, p1, p2, p3, p4, 0], 2) + { + } + + /// + /// Initializes a new instance of the class for a seven-symbol alphabet. + /// + /// The cumulative threshold following symbol zero. + /// The cumulative threshold following symbol one. + /// The cumulative threshold following symbol two. + /// The cumulative threshold following symbol three. + /// The cumulative threshold following symbol four. + /// The cumulative threshold following symbol five. + public Av1Distribution(uint p0, uint p1, uint p2, uint p3, uint p4, uint p5) + : this([p0, p1, p2, p3, p4, p5, 0], 2) + { + } + + /// + /// Initializes a new instance of the class for an eight-symbol alphabet. + /// + /// The cumulative threshold following symbol zero. + /// The cumulative threshold following symbol one. + /// The cumulative threshold following symbol two. + /// The cumulative threshold following symbol three. + /// The cumulative threshold following symbol four. + /// The cumulative threshold following symbol five. + /// The cumulative threshold following symbol six. + public Av1Distribution(uint p0, uint p1, uint p2, uint p3, uint p4, uint p5, uint p6) + : this([p0, p1, p2, p3, p4, p5, p6, 0], 2) + { + } + + /// + /// Initializes a new instance of the class for a nine-symbol alphabet. + /// + /// The cumulative threshold following symbol zero. + /// The cumulative threshold following symbol one. + /// The cumulative threshold following symbol two. + /// The cumulative threshold following symbol three. + /// The cumulative threshold following symbol four. + /// The cumulative threshold following symbol five. + /// The cumulative threshold following symbol six. + /// The cumulative threshold following symbol seven. + public Av1Distribution(uint p0, uint p1, uint p2, uint p3, uint p4, uint p5, uint p6, uint p7) + : this([p0, p1, p2, p3, p4, p5, p6, p7, 0], 2) + { + } + + /// + /// Initializes a new instance of the class for a ten-symbol alphabet. + /// + /// The cumulative threshold following symbol zero. + /// The cumulative threshold following symbol one. + /// The cumulative threshold following symbol two. + /// The cumulative threshold following symbol three. + /// The cumulative threshold following symbol four. + /// The cumulative threshold following symbol five. + /// The cumulative threshold following symbol six. + /// The cumulative threshold following symbol seven. + /// The cumulative threshold following symbol eight. + public Av1Distribution(uint p0, uint p1, uint p2, uint p3, uint p4, uint p5, uint p6, uint p7, uint p8) + : this([p0, p1, p2, p3, p4, p5, p6, p7, p8, 0], 2) + { + } + + /// + /// Initializes a new instance of the class for an eleven-symbol alphabet. + /// + /// The cumulative threshold following symbol zero. + /// The cumulative threshold following symbol one. + /// The cumulative threshold following symbol two. + /// The cumulative threshold following symbol three. + /// The cumulative threshold following symbol four. + /// The cumulative threshold following symbol five. + /// The cumulative threshold following symbol six. + /// The cumulative threshold following symbol seven. + /// The cumulative threshold following symbol eight. + /// The cumulative threshold following symbol nine. + public Av1Distribution(uint p0, uint p1, uint p2, uint p3, uint p4, uint p5, uint p6, uint p7, uint p8, uint p9) + : this([p0, p1, p2, p3, p4, p5, p6, p7, p8, p9, 0], 2) + { + } + + /// + /// Initializes a new instance of the class for a twelve-symbol alphabet. + /// + /// The cumulative threshold following symbol zero. + /// The cumulative threshold following symbol one. + /// The cumulative threshold following symbol two. + /// The cumulative threshold following symbol three. + /// The cumulative threshold following symbol four. + /// The cumulative threshold following symbol five. + /// The cumulative threshold following symbol six. + /// The cumulative threshold following symbol seven. + /// The cumulative threshold following symbol eight. + /// The cumulative threshold following symbol nine. + /// The cumulative threshold following symbol ten. + public Av1Distribution(uint p0, uint p1, uint p2, uint p3, uint p4, uint p5, uint p6, uint p7, uint p8, uint p9, uint p10) + : this([p0, p1, p2, p3, p4, p5, p6, p7, p8, p9, p10, 0], 2) + { + } + + /// + /// Initializes a new instance of the class for a thirteen-symbol alphabet. + /// + /// The cumulative threshold following symbol zero. + /// The cumulative threshold following symbol one. + /// The cumulative threshold following symbol two. + /// The cumulative threshold following symbol three. + /// The cumulative threshold following symbol four. + /// The cumulative threshold following symbol five. + /// The cumulative threshold following symbol six. + /// The cumulative threshold following symbol seven. + /// The cumulative threshold following symbol eight. + /// The cumulative threshold following symbol nine. + /// The cumulative threshold following symbol ten. + /// The cumulative threshold following symbol eleven. + public Av1Distribution(uint p0, uint p1, uint p2, uint p3, uint p4, uint p5, uint p6, uint p7, uint p8, uint p9, uint p10, uint p11) + : this([p0, p1, p2, p3, p4, p5, p6, p7, p8, p9, p10, p11, 0], 2) + { + } + + /// + /// Initializes a new instance of the class for a fourteen-symbol alphabet. + /// + /// The cumulative threshold following symbol zero. + /// The cumulative threshold following symbol one. + /// The cumulative threshold following symbol two. + /// The cumulative threshold following symbol three. + /// The cumulative threshold following symbol four. + /// The cumulative threshold following symbol five. + /// The cumulative threshold following symbol six. + /// The cumulative threshold following symbol seven. + /// The cumulative threshold following symbol eight. + /// The cumulative threshold following symbol nine. + /// The cumulative threshold following symbol ten. + /// The cumulative threshold following symbol eleven. + /// The cumulative threshold following symbol twelve. + public Av1Distribution(uint p0, uint p1, uint p2, uint p3, uint p4, uint p5, uint p6, uint p7, uint p8, uint p9, uint p10, uint p11, uint p12) + : this([p0, p1, p2, p3, p4, p5, p6, p7, p8, p9, p10, p11, p12, 0], 2) + { + } + + /// + /// Initializes a new instance of the class for a sixteen-symbol alphabet. + /// + /// The cumulative threshold following symbol zero. + /// The cumulative threshold following symbol one. + /// The cumulative threshold following symbol two. + /// The cumulative threshold following symbol three. + /// The cumulative threshold following symbol four. + /// The cumulative threshold following symbol five. + /// The cumulative threshold following symbol six. + /// The cumulative threshold following symbol seven. + /// The cumulative threshold following symbol eight. + /// The cumulative threshold following symbol nine. + /// The cumulative threshold following symbol ten. + /// The cumulative threshold following symbol eleven. + /// The cumulative threshold following symbol twelve. + /// The cumulative threshold following symbol thirteen. + /// The cumulative threshold following symbol fourteen. + public Av1Distribution(uint p0, uint p1, uint p2, uint p3, uint p4, uint p5, uint p6, uint p7, uint p8, uint p9, uint p10, uint p11, uint p12, uint p13, uint p14) + : this([p0, p1, p2, p3, p4, p5, p6, p7, p8, p9, p10, p11, p12, p13, p14, 0], 2) + { + } + + /// + /// Initializes a new instance of the class from forward cumulative thresholds. + /// + /// The forward Q15 thresholds followed by a zero sentinel slot. + /// The symbol-count contribution to the update rate. + private Av1Distribution(ReadOnlySpan props, int speed) + { + Span probabilities = this.probabilities; + + // AV1 range coding consumes inverse cumulative thresholds. The defaults are written in the more readable + // forward form, so convert every real threshold while leaving the final zero sentinel untouched. + for (int i = 0; i < props.Length - 1; i++) + { + probabilities[i] = ProbabilityTop - props[i]; + } + + this.NumberOfSymbols = props.Length; + this.speed = speed; + } + + /// + /// Initializes a new instance of the class with the same probability and adaptation state as another distribution. + /// + /// The distribution state to copy. + private Av1Distribution(Av1Distribution source) + { + ReadOnlySpan sourceProbabilities = source.probabilities; + Span probabilities = this.probabilities; + sourceProbabilities[..source.NumberOfSymbols].CopyTo(probabilities); + + // The adaptation rate depends on both the alphabet size and prior update count, so copying only the + // thresholds would make the cloned frame context diverge after its next symbol. + this.speed = source.speed; + this.updateCount = source.updateCount; + this.NumberOfSymbols = source.NumberOfSymbols; + } + + /// + /// Gets the number of symbols represented by the distribution. + /// + public int NumberOfSymbols { get; } + + /// + /// Gets an inverse cumulative threshold by symbol index. + /// + /// The zero-based threshold index. + /// The Q15 inverse cumulative threshold. + public uint this[int index] => this.probabilities[index]; + + /// + /// Creates an independently adaptable copy of a distribution. + /// + /// A distribution initialized with the same probabilities and update count. + public Av1Distribution CreateCopy() => new(this); + + /// + /// Replaces the probability and adaptation state with the state of another distribution having the same alphabet. + /// + /// The distribution state to copy. + public void CopyFrom(Av1Distribution source) + { + // Entropy contexts are created from the same fixed default table shape. Copy only mutable state so resetting a + // working tile never allocates or replaces the distribution objects referenced by the symbol decoder. + ReadOnlySpan sourceProbabilities = source.probabilities; + Span probabilities = this.probabilities; + sourceProbabilities[..source.NumberOfSymbols].CopyTo(probabilities); + this.updateCount = source.updateCount; + } + + /// + /// Resets the observation count that controls the adaptive update rate without changing probability thresholds. + /// + public void ResetUpdateCount() => this.updateCount = 0; + + /// + /// Creates independently adaptable copies of a distribution array. + /// + /// The distributions to copy. + /// An array with the same shape and distribution state. + public static Av1Distribution[] CreateCopy(Av1Distribution[] source) + { + Av1Distribution[] result = new Av1Distribution[source.Length]; + for (int i = 0; i < source.Length; i++) + { + result[i] = source[i].CreateCopy(); + } + + return result; + } + + /// + /// Creates independently adaptable copies of a two-dimensional jagged distribution array. + /// + /// The distributions to copy. + /// An array with the same shape and distribution state. + public static Av1Distribution[][] CreateCopy(Av1Distribution[][] source) + { + Av1Distribution[][] result = new Av1Distribution[source.Length][]; + for (int i = 0; i < source.Length; i++) + { + result[i] = CreateCopy(source[i]); + } + + return result; + } + + /// + /// Creates independently adaptable copies of a three-dimensional jagged distribution array. + /// + /// The distributions to copy. + /// An array with the same shape and distribution state. + public static Av1Distribution[][][] CreateCopy(Av1Distribution[][][] source) + { + Av1Distribution[][][] result = new Av1Distribution[source.Length][][]; + for (int i = 0; i < source.Length; i++) + { + result[i] = CreateCopy(source[i]); + } + + return result; + } + + /// + /// Adapts the cumulative thresholds after coding one symbol. + /// + /// The zero-based symbol that was coded. + public void Update(int value) + { + // AV1 slows adaptation after 16 and 32 observations. The symbol-count term is precomputed by each overload + // because every distribution has a fixed alphabet size. + int rate15 = this.updateCount > 15 ? 1 : 0; + int rate31 = this.updateCount > 31 ? 1 : 0; + int rate = 3 + rate15 + rate31 + this.speed; + int tmp = ProbabilityTop; + + // Switching tmp to zero at the observed symbol moves the thresholds on either side toward the sample while + // preserving their inverse-cumulative ordering in one pass. + for (int i = 0; i < this.NumberOfSymbols - 1; i++) + { + tmp = i == value ? 0 : tmp; + uint p = this.probabilities[i]; + if (tmp < p) + { + this.probabilities[i] -= (ushort)((p - tmp) >> rate); + } + else + { + this.probabilities[i] += (ushort)((tmp - p) >> rate); + } + } + + int rate32 = this.updateCount < 32 ? 1 : 0; + this.updateCount += rate32; + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Entropy/Av1FrameEntropyContext.cs b/src/ImageSharp/Formats/Heif/Av1/Entropy/Av1FrameEntropyContext.cs new file mode 100644 index 0000000000..c1b0d82e6c --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Entropy/Av1FrameEntropyContext.cs @@ -0,0 +1,766 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Entropy; + +/// +/// Owns the adaptive AV1 distributions currently implemented by the frame and tile syntax decoders. +/// +/// +/// One frame context supplies the initial state copied into every tile context. Each tile adapts an independent working +/// copy, and only the tile selected by context_update_tile_id supplies the completed frame snapshot. +/// +internal sealed class Av1FrameEntropyContext +{ + /// + /// The inclusive upper bound of the first AV1 coefficient-probability quantizer band. + /// + private const int FirstQuantizerBandMaximum = 20; + + /// + /// The inclusive upper bound of the second AV1 coefficient-probability quantizer band. + /// + private const int SecondQuantizerBandMaximum = 60; + + /// + /// The inclusive upper bound of the third AV1 coefficient-probability quantizer band. + /// + private const int ThirdQuantizerBandMaximum = 120; + + /// + /// The immutable normative contexts used to restore reusable frame state without rebuilding distribution graphs. + /// + private static readonly Av1FrameEntropyContext[] DefaultPrototypes = + [ + new((byte)0), + new((byte)1), + new((byte)2), + new((byte)3) + ]; + + /// + /// Initializes a new instance of the class from the normative default + /// distributions selected by a frame quantizer index. + /// + /// The frame base quantizer index selecting coefficient distribution defaults. + public Av1FrameEntropyContext(int qIndex) + : this(DefaultPrototypes[GetQContext(qIndex)]) + { + } + + /// + /// Initializes a new instance of the class as an immutable normative prototype. + /// + /// The zero-based coefficient-probability quantizer band. + private Av1FrameEntropyContext(byte qContext) + { + int qIndex = qContext switch + { + 0 => 0, + 1 => FirstQuantizerBandMaximum + 1, + 2 => SecondQuantizerBandMaximum + 1, + _ => ThirdQuantizerBandMaximum + 1 + }; + + // Every default-distribution accessor constructs independently mutable state. Retaining those returned + // graphs directly confines generated-table construction to the four process-wide quantizer-band prototypes. + this.IntraBlockCopy = Av1DefaultDistributions.IntraBlockCopy; + + // Normal motion vectors and intra-block-copy displacement vectors start from identical defaults, but AV1 + // adapts NMVC and NDVC independently. Distinct object graphs preserve that separation for the prototype too. + this.MotionVector = new(); + this.DisplacementVector = new(); + this.SwitchableRestoration = Av1DefaultDistributions.SwitchableRestoration; + this.WienerRestoration = Av1DefaultDistributions.WienerRestoration; + this.SgrProjectionRestoration = Av1DefaultDistributions.SgrProjectionRestoration; + this.PaletteYMode = Av1DefaultDistributions.PaletteYMode; + this.PaletteUvMode = Av1DefaultDistributions.PaletteUvMode; + this.PaletteYSize = Av1DefaultDistributions.PaletteYSize; + this.PaletteUvSize = Av1DefaultDistributions.PaletteUvSize; + this.PaletteYColorIndex = Av1DefaultDistributions.PaletteYColorIndex; + this.PaletteUvColorIndex = Av1DefaultDistributions.PaletteUvColorIndex; + this.PartitionTypes = Av1DefaultDistributions.PartitionTypes; + this.FrameYMode = Av1DefaultDistributions.FrameYMode; + this.KeyFrameYMode = Av1DefaultDistributions.KeyFrameYMode; + this.IntraInter = Av1DefaultDistributions.IntraInter; + this.NewMv = Av1DefaultDistributions.NewMv; + this.ZeroMv = Av1DefaultDistributions.ZeroMv; + this.RefMv = Av1DefaultDistributions.RefMv; + this.Drl = Av1DefaultDistributions.Drl; + this.SingleReference = Av1DefaultDistributions.SingleReference; + this.CompInter = Av1DefaultDistributions.CompInter; + this.CompoundReferenceType = Av1DefaultDistributions.CompoundReferenceType; + this.UnidirectionalCompoundReference = Av1DefaultDistributions.UnidirectionalCompoundReference; + this.CompoundReference = Av1DefaultDistributions.CompoundReference; + this.CompoundBackwardReference = Av1DefaultDistributions.CompoundBackwardReference; + this.InterCompoundMode = Av1DefaultDistributions.InterCompoundMode; + this.InterIntra = Av1DefaultDistributions.InterIntra; + this.InterIntraMode = Av1DefaultDistributions.InterIntraMode; + this.WedgeInterIntra = Av1DefaultDistributions.WedgeInterIntra; + this.CompoundType = Av1DefaultDistributions.CompoundType; + this.WedgeIndex = Av1DefaultDistributions.WedgeIndex; + this.CompoundIndex = Av1DefaultDistributions.CompoundIndex; + this.CompoundGroupIndex = Av1DefaultDistributions.CompoundGroupIndex; + this.MotionMode = Av1DefaultDistributions.MotionMode; + this.Obmc = Av1DefaultDistributions.Obmc; + this.SwitchableInterpolation = Av1DefaultDistributions.SwitchableInterpolation; + this.UvMode = Av1DefaultDistributions.UvMode; + this.Skip = Av1DefaultDistributions.Skip; + this.SkipMode = Av1DefaultDistributions.SkipMode; + this.DeltaLoopFilterAbsolute = Av1DefaultDistributions.DeltaLoopFilterAbsolute; + this.DeltaLoopFilterMultiAbsolute = Av1DefaultDistributions.DeltaLoopFilterMultiAbsolute; + this.DeltaQuantizerAbsolute = Av1DefaultDistributions.DeltaQuantizerAbsolute; + this.SegmentId = Av1DefaultDistributions.SegmentId; + this.SegmentIdPredicted = Av1DefaultDistributions.SegmentIdPredicted; + this.AngleDelta = Av1DefaultDistributions.AngleDelta; + this.FilterIntraMode = Av1DefaultDistributions.FilterIntraMode; + this.FilterIntra = Av1DefaultDistributions.FilterIntra; + this.TransformSize = Av1DefaultDistributions.TransformSize; + this.TransformPartition = Av1DefaultDistributions.TransformPartition; + this.ChromaFromLumaSign = Av1DefaultDistributions.ChromaFromLumaSign; + this.ChromaFromLumaAlpha = Av1DefaultDistributions.ChromaFromLumaAlpha; + this.IntraExtendedTransform = Av1DefaultDistributions.IntraExtendedTransform; + this.InterExtendedTransform = Av1DefaultDistributions.InterExtendedTransform; + + // Coefficient defaults use one of four quantizer bands. Their array shapes remain fixed, so later tile resets + // copy only thresholds and update counts into this context's already allocated distribution graph. + this.EndOfBlockFlag = Av1DefaultDistributions.GetEndOfBlockFlag(qIndex); + this.CoefficientsBase = Av1DefaultDistributions.GetCoefficientsBase(qIndex); + this.BaseEndOfBlock = Av1DefaultDistributions.GetBaseEndOfBlock(qIndex); + this.DcSign = Av1DefaultDistributions.GetDcSign(qIndex); + this.CoefficientsBaseRange = Av1DefaultDistributions.GetCoefficientsBaseRange(qIndex); + this.TransformBlockSkip = Av1DefaultDistributions.GetTransformBlockSkip(qIndex); + this.EndOfBlockExtra = Av1DefaultDistributions.GetEndOfBlockExtra(qIndex); + } + + /// + /// Initializes a new instance of the class with an independently adaptable copy of a prototype. + /// + /// The prototype or retained context whose state is copied. + private Av1FrameEntropyContext(Av1FrameEntropyContext source) + { + // Session and retained-frame contexts need one mutable graph, not four generated quantizer-band graphs whose + // unused bands are immediately discarded. Deep-copy the already selected prototype shape exactly once. + this.IntraBlockCopy = source.IntraBlockCopy.CreateCopy(); + this.MotionVector = new(); + this.MotionVector.CopyFrom(source.MotionVector); + this.DisplacementVector = new(); + this.DisplacementVector.CopyFrom(source.DisplacementVector); + this.SwitchableRestoration = source.SwitchableRestoration.CreateCopy(); + this.WienerRestoration = source.WienerRestoration.CreateCopy(); + this.SgrProjectionRestoration = source.SgrProjectionRestoration.CreateCopy(); + this.PaletteYMode = Av1Distribution.CreateCopy(source.PaletteYMode); + this.PaletteUvMode = Av1Distribution.CreateCopy(source.PaletteUvMode); + this.PaletteYSize = Av1Distribution.CreateCopy(source.PaletteYSize); + this.PaletteUvSize = Av1Distribution.CreateCopy(source.PaletteUvSize); + this.PaletteYColorIndex = Av1Distribution.CreateCopy(source.PaletteYColorIndex); + this.PaletteUvColorIndex = Av1Distribution.CreateCopy(source.PaletteUvColorIndex); + this.PartitionTypes = Av1Distribution.CreateCopy(source.PartitionTypes); + this.FrameYMode = Av1Distribution.CreateCopy(source.FrameYMode); + this.KeyFrameYMode = Av1Distribution.CreateCopy(source.KeyFrameYMode); + this.IntraInter = Av1Distribution.CreateCopy(source.IntraInter); + this.NewMv = Av1Distribution.CreateCopy(source.NewMv); + this.ZeroMv = Av1Distribution.CreateCopy(source.ZeroMv); + this.RefMv = Av1Distribution.CreateCopy(source.RefMv); + this.Drl = Av1Distribution.CreateCopy(source.Drl); + this.SingleReference = Av1Distribution.CreateCopy(source.SingleReference); + this.CompInter = Av1Distribution.CreateCopy(source.CompInter); + this.CompoundReferenceType = Av1Distribution.CreateCopy(source.CompoundReferenceType); + this.UnidirectionalCompoundReference = Av1Distribution.CreateCopy(source.UnidirectionalCompoundReference); + this.CompoundReference = Av1Distribution.CreateCopy(source.CompoundReference); + this.CompoundBackwardReference = Av1Distribution.CreateCopy(source.CompoundBackwardReference); + this.InterCompoundMode = Av1Distribution.CreateCopy(source.InterCompoundMode); + this.InterIntra = Av1Distribution.CreateCopy(source.InterIntra); + this.InterIntraMode = Av1Distribution.CreateCopy(source.InterIntraMode); + this.WedgeInterIntra = Av1Distribution.CreateCopy(source.WedgeInterIntra); + this.CompoundType = Av1Distribution.CreateCopy(source.CompoundType); + this.WedgeIndex = Av1Distribution.CreateCopy(source.WedgeIndex); + this.CompoundIndex = Av1Distribution.CreateCopy(source.CompoundIndex); + this.CompoundGroupIndex = Av1Distribution.CreateCopy(source.CompoundGroupIndex); + this.MotionMode = Av1Distribution.CreateCopy(source.MotionMode); + this.Obmc = Av1Distribution.CreateCopy(source.Obmc); + this.SwitchableInterpolation = Av1Distribution.CreateCopy(source.SwitchableInterpolation); + this.UvMode = Av1Distribution.CreateCopy(source.UvMode); + this.Skip = Av1Distribution.CreateCopy(source.Skip); + this.SkipMode = Av1Distribution.CreateCopy(source.SkipMode); + this.DeltaLoopFilterAbsolute = source.DeltaLoopFilterAbsolute.CreateCopy(); + this.DeltaLoopFilterMultiAbsolute = Av1Distribution.CreateCopy(source.DeltaLoopFilterMultiAbsolute); + this.DeltaQuantizerAbsolute = source.DeltaQuantizerAbsolute.CreateCopy(); + this.SegmentId = Av1Distribution.CreateCopy(source.SegmentId); + this.SegmentIdPredicted = Av1Distribution.CreateCopy(source.SegmentIdPredicted); + this.AngleDelta = Av1Distribution.CreateCopy(source.AngleDelta); + this.FilterIntraMode = source.FilterIntraMode.CreateCopy(); + this.FilterIntra = Av1Distribution.CreateCopy(source.FilterIntra); + this.TransformSize = Av1Distribution.CreateCopy(source.TransformSize); + this.TransformPartition = Av1Distribution.CreateCopy(source.TransformPartition); + this.EndOfBlockFlag = Av1Distribution.CreateCopy(source.EndOfBlockFlag); + this.CoefficientsBase = Av1Distribution.CreateCopy(source.CoefficientsBase); + this.BaseEndOfBlock = Av1Distribution.CreateCopy(source.BaseEndOfBlock); + this.DcSign = Av1Distribution.CreateCopy(source.DcSign); + this.CoefficientsBaseRange = Av1Distribution.CreateCopy(source.CoefficientsBaseRange); + this.TransformBlockSkip = Av1Distribution.CreateCopy(source.TransformBlockSkip); + this.EndOfBlockExtra = Av1Distribution.CreateCopy(source.EndOfBlockExtra); + this.ChromaFromLumaSign = source.ChromaFromLumaSign.CreateCopy(); + this.ChromaFromLumaAlpha = Av1Distribution.CreateCopy(source.ChromaFromLumaAlpha); + this.IntraExtendedTransform = Av1Distribution.CreateCopy(source.IntraExtendedTransform); + this.InterExtendedTransform = Av1Distribution.CreateCopy(source.InterExtendedTransform); + } + + /// + /// Gets the intra-block-copy distribution. + /// + public Av1Distribution IntraBlockCopy { get; } + + /// + /// Gets the entropy context used by normal inter-prediction motion vectors. + /// + public Av1MotionVectorContext MotionVector { get; } + + /// + /// Gets the integer displacement-vector context used by intra-block copy. + /// + public Av1MotionVectorContext DisplacementVector { get; } + + /// + /// Gets the switchable loop-restoration distribution. + /// + public Av1Distribution SwitchableRestoration { get; } + + /// + /// Gets the Wiener loop-restoration distribution. + /// + public Av1Distribution WienerRestoration { get; } + + /// + /// Gets the self-guided loop-restoration distribution. + /// + public Av1Distribution SgrProjectionRestoration { get; } + + /// + /// Gets the luma palette-mode distributions. + /// + public Av1Distribution[][] PaletteYMode { get; } + + /// + /// Gets the chroma palette-mode distributions. + /// + public Av1Distribution[] PaletteUvMode { get; } + + /// + /// Gets the luma palette-size distributions. + /// + public Av1Distribution[] PaletteYSize { get; } + + /// + /// Gets the chroma palette-size distributions. + /// + public Av1Distribution[] PaletteUvSize { get; } + + /// + /// Gets the luma palette color-index distributions. + /// + public Av1Distribution[][] PaletteYColorIndex { get; } + + /// + /// Gets the chroma palette color-index distributions. + /// + public Av1Distribution[][] PaletteUvColorIndex { get; } + + /// + /// Gets the partition-type distributions. + /// + public Av1Distribution[] PartitionTypes { get; } + + /// + /// Gets the inter-frame intra luma-mode distributions indexed by the normative block-size group. + /// + public Av1Distribution[] FrameYMode { get; } + + /// + /// Gets the key-frame luma-mode distributions. + /// + public Av1Distribution[][] KeyFrameYMode { get; } + + /// + /// Gets the distributions that select intra or inter prediction from the available spatial neighbors. + /// + public Av1Distribution[] IntraInter { get; } + + /// + /// Gets the distributions that select a newly decoded motion vector before the remaining single-reference modes. + /// + public Av1Distribution[] NewMv { get; } + + /// + /// Gets the distributions that select global motion before the spatial reference-motion-vector modes. + /// + public Av1Distribution[] ZeroMv { get; } + + /// + /// Gets the distributions that select the nearest or near spatial reference motion vector. + /// + public Av1Distribution[] RefMv { get; } + + /// + /// Gets the distributions that advance through the dynamic reference-motion-vector candidate list. + /// + public Av1Distribution[] Drl { get; } + + /// + /// Gets the single-reference selection distributions indexed by spatial context and tree decision. + /// + public Av1Distribution[][] SingleReference { get; } + + /// + /// Gets the distributions that select single-reference or compound-reference prediction for a block. + /// + public Av1Distribution[] CompInter { get; } + + /// + /// Gets the distributions that select unidirectional or bidirectional compound reference pairs. + /// + public Av1Distribution[] CompoundReferenceType { get; } + + /// + /// Gets the unidirectional compound-reference selection distributions. + /// + public Av1Distribution[][] UnidirectionalCompoundReference { get; } + + /// + /// Gets the bidirectional compound forward-reference selection distributions. + /// + public Av1Distribution[][] CompoundReference { get; } + + /// + /// Gets the bidirectional compound backward-reference selection distributions. + /// + public Av1Distribution[][] CompoundBackwardReference { get; } + + /// + /// Gets the compound inter-mode distributions indexed by derived mode context. + /// + public Av1Distribution[] InterCompoundMode { get; } + + /// + /// Gets the inter-intra prediction flag distributions indexed by block-size group. + /// + public Av1Distribution[] InterIntra { get; } + + /// + /// Gets the inter-intra mode distributions indexed by block-size group. + /// + public Av1Distribution[] InterIntraMode { get; } + + /// + /// Gets the inter-intra wedge-selection distributions indexed by block size. + /// + public Av1Distribution[] WedgeInterIntra { get; } + + /// + /// Gets the masked-compound type distributions indexed by block size. + /// + public Av1Distribution[] CompoundType { get; } + + /// + /// Gets the wedge-index distributions indexed by block size. + /// + public Av1Distribution[] WedgeIndex { get; } + + /// + /// Gets the average-or-distance-weighted compound distributions indexed by derived context. + /// + public Av1Distribution[] CompoundIndex { get; } + + /// + /// Gets the unmasked-or-masked compound-group distributions indexed by derived context. + /// + public Av1Distribution[] CompoundGroupIndex { get; } + + /// + /// Gets the three-way motion-mode distributions indexed by block size. + /// + public Av1Distribution[] MotionMode { get; } + + /// + /// Gets the binary Simple Translation or OBMC distributions indexed by block size. + /// + public Av1Distribution[] Obmc { get; } + + /// + /// Gets the switchable interpolation-filter distributions. + /// + public Av1Distribution[] SwitchableInterpolation { get; } + + /// + /// Gets the chroma intra-mode distributions. + /// + public Av1Distribution[][] UvMode { get; } + + /// + /// Gets the transform-skip distributions. + /// + public Av1Distribution[] Skip { get; } + + /// + /// Gets the skip-mode distributions. + /// + public Av1Distribution[] SkipMode { get; } + + /// + /// Gets the absolute loop-filter delta distribution. + /// + public Av1Distribution DeltaLoopFilterAbsolute { get; } + + /// + /// Gets independent delta distributions for vertical luma, horizontal luma, U, and V loop filters. + /// + public Av1Distribution[] DeltaLoopFilterMultiAbsolute { get; } + + /// + /// Gets the absolute quantizer delta distribution. + /// + public Av1Distribution DeltaQuantizerAbsolute { get; } + + /// + /// Gets the spatial segment-identifier distributions. + /// + public Av1Distribution[] SegmentId { get; } + + /// + /// Gets the temporal segment-map prediction distributions. + /// + public Av1Distribution[] SegmentIdPredicted { get; } + + /// + /// Gets the directional angle-delta distributions. + /// + public Av1Distribution[] AngleDelta { get; } + + /// + /// Gets the filter-intra mode distribution. + /// + public Av1Distribution FilterIntraMode { get; } + + /// + /// Gets the filter-intra enable distributions. + /// + public Av1Distribution[] FilterIntra { get; } + + /// + /// Gets the transform-size distributions. + /// + public Av1Distribution[][] TransformSize { get; } + + /// + /// Gets the variable-transform partition distributions. + /// + public Av1Distribution[] TransformPartition { get; } + + /// + /// Gets the end-of-block token distributions selected for the frame base quantizer. + /// + public Av1Distribution[][][] EndOfBlockFlag { get; } + + /// + /// Gets the coefficient base-level distributions selected for the frame base quantizer. + /// + public Av1Distribution[][][] CoefficientsBase { get; } + + /// + /// Gets the final-nonzero coefficient distributions selected for the frame base quantizer. + /// + public Av1Distribution[][][] BaseEndOfBlock { get; } + + /// + /// Gets the DC sign distributions selected for the frame base quantizer. + /// + public Av1Distribution[][] DcSign { get; } + + /// + /// Gets the coefficient base-range distributions selected for the frame base quantizer. + /// + public Av1Distribution[][][] CoefficientsBaseRange { get; } + + /// + /// Gets the transform-block skip distributions selected for the frame base quantizer. + /// + public Av1Distribution[][] TransformBlockSkip { get; } + + /// + /// Gets the end-of-block extra-bit distributions selected for the frame base quantizer. + /// + public Av1Distribution[][][] EndOfBlockExtra { get; } + + /// + /// Gets the joint chroma-from-luma sign distribution. + /// + public Av1Distribution ChromaFromLumaSign { get; } + + /// + /// Gets the chroma-from-luma alpha-magnitude distributions. + /// + public Av1Distribution[] ChromaFromLumaAlpha { get; } + + /// + /// Gets the intra transform-type distributions. + /// + public Av1Distribution[][][] IntraExtendedTransform { get; } + + /// + /// Gets the inter transform-type distributions. + /// + public Av1Distribution[][] InterExtendedTransform { get; } + + /// + /// Restores the normative frame defaults selected by a base quantizer index. + /// + /// The frame base quantizer index selecting coefficient distribution defaults. + public void ResetToDefaults(int qIndex) + { + int qContext = GetQContext(qIndex); + + // The prototypes are never exposed to a range reader or writer. Copying their state lets each codec session + // reuse its mutable object graphs even when successive frames select different coefficient-model bands. + this.CopyFrom(DefaultPrototypes[qContext]); + } + + /// + /// Maps a frame base quantizer to its normative coefficient-probability initialization band. + /// + /// The frame base quantizer index. + /// The zero-based quantizer-band index. + private static int GetQContext(int qIndex) + => qIndex switch + { + <= FirstQuantizerBandMaximum => 0, + <= SecondQuantizerBandMaximum => 1, + <= ThirdQuantizerBandMaximum => 2, + _ => 3 + }; + + /// + /// Replaces every probability threshold and adaptation count with state copied from another frame context. + /// + /// The frame context state to copy. + public void CopyFrom(Av1FrameEntropyContext source) + { + this.IntraBlockCopy.CopyFrom(source.IntraBlockCopy); + this.MotionVector.CopyFrom(source.MotionVector); + this.DisplacementVector.CopyFrom(source.DisplacementVector); + this.SwitchableRestoration.CopyFrom(source.SwitchableRestoration); + this.WienerRestoration.CopyFrom(source.WienerRestoration); + this.SgrProjectionRestoration.CopyFrom(source.SgrProjectionRestoration); + CopyState(source.PaletteYMode, this.PaletteYMode); + CopyState(source.PaletteUvMode, this.PaletteUvMode); + CopyState(source.PaletteYSize, this.PaletteYSize); + CopyState(source.PaletteUvSize, this.PaletteUvSize); + CopyState(source.PaletteYColorIndex, this.PaletteYColorIndex); + CopyState(source.PaletteUvColorIndex, this.PaletteUvColorIndex); + CopyState(source.PartitionTypes, this.PartitionTypes); + CopyState(source.FrameYMode, this.FrameYMode); + CopyState(source.KeyFrameYMode, this.KeyFrameYMode); + CopyState(source.IntraInter, this.IntraInter); + CopyState(source.NewMv, this.NewMv); + CopyState(source.ZeroMv, this.ZeroMv); + CopyState(source.RefMv, this.RefMv); + CopyState(source.Drl, this.Drl); + CopyState(source.SingleReference, this.SingleReference); + CopyState(source.CompInter, this.CompInter); + CopyState(source.CompoundReferenceType, this.CompoundReferenceType); + CopyState(source.UnidirectionalCompoundReference, this.UnidirectionalCompoundReference); + CopyState(source.CompoundReference, this.CompoundReference); + CopyState(source.CompoundBackwardReference, this.CompoundBackwardReference); + CopyState(source.InterCompoundMode, this.InterCompoundMode); + CopyState(source.InterIntra, this.InterIntra); + CopyState(source.InterIntraMode, this.InterIntraMode); + CopyState(source.WedgeInterIntra, this.WedgeInterIntra); + CopyState(source.CompoundType, this.CompoundType); + CopyState(source.WedgeIndex, this.WedgeIndex); + CopyState(source.CompoundIndex, this.CompoundIndex); + CopyState(source.CompoundGroupIndex, this.CompoundGroupIndex); + CopyState(source.MotionMode, this.MotionMode); + CopyState(source.Obmc, this.Obmc); + CopyState(source.SwitchableInterpolation, this.SwitchableInterpolation); + CopyState(source.UvMode, this.UvMode); + CopyState(source.Skip, this.Skip); + CopyState(source.SkipMode, this.SkipMode); + this.DeltaLoopFilterAbsolute.CopyFrom(source.DeltaLoopFilterAbsolute); + CopyState(source.DeltaLoopFilterMultiAbsolute, this.DeltaLoopFilterMultiAbsolute); + this.DeltaQuantizerAbsolute.CopyFrom(source.DeltaQuantizerAbsolute); + CopyState(source.SegmentId, this.SegmentId); + CopyState(source.SegmentIdPredicted, this.SegmentIdPredicted); + CopyState(source.AngleDelta, this.AngleDelta); + this.FilterIntraMode.CopyFrom(source.FilterIntraMode); + CopyState(source.FilterIntra, this.FilterIntra); + CopyState(source.TransformSize, this.TransformSize); + CopyState(source.TransformPartition, this.TransformPartition); + CopyState(source.EndOfBlockFlag, this.EndOfBlockFlag); + CopyState(source.CoefficientsBase, this.CoefficientsBase); + CopyState(source.BaseEndOfBlock, this.BaseEndOfBlock); + CopyState(source.DcSign, this.DcSign); + CopyState(source.CoefficientsBaseRange, this.CoefficientsBaseRange); + CopyState(source.TransformBlockSkip, this.TransformBlockSkip); + CopyState(source.EndOfBlockExtra, this.EndOfBlockExtra); + this.ChromaFromLumaSign.CopyFrom(source.ChromaFromLumaSign); + CopyState(source.ChromaFromLumaAlpha, this.ChromaFromLumaAlpha); + CopyState(source.IntraExtendedTransform, this.IntraExtendedTransform); + CopyState(source.InterExtendedTransform, this.InterExtendedTransform); + } + + /// + /// Copies this tile-adapted context into a destination used as completed frame state. + /// + /// The independently owned frame context that receives the snapshot. + /// + /// AV1 resets CDF observation counters after publishing the context-update tile. The copied thresholds remain + /// adapted, while the next frame starts its update-rate history from zero. + /// + public void SnapshotTo(Av1FrameEntropyContext destination) + { + destination.CopyFrom(this); + destination.ResetUpdateCounts(); + } + + /// + /// Resets the observation count of every distribution without changing its probability thresholds. + /// + private void ResetUpdateCounts() + { + this.IntraBlockCopy.ResetUpdateCount(); + this.MotionVector.ResetUpdateCounts(); + this.DisplacementVector.ResetUpdateCounts(); + this.SwitchableRestoration.ResetUpdateCount(); + this.WienerRestoration.ResetUpdateCount(); + this.SgrProjectionRestoration.ResetUpdateCount(); + ResetUpdateCounts(this.PaletteYMode); + ResetUpdateCounts(this.PaletteUvMode); + ResetUpdateCounts(this.PaletteYSize); + ResetUpdateCounts(this.PaletteUvSize); + ResetUpdateCounts(this.PaletteYColorIndex); + ResetUpdateCounts(this.PaletteUvColorIndex); + ResetUpdateCounts(this.PartitionTypes); + ResetUpdateCounts(this.FrameYMode); + ResetUpdateCounts(this.KeyFrameYMode); + ResetUpdateCounts(this.IntraInter); + ResetUpdateCounts(this.NewMv); + ResetUpdateCounts(this.ZeroMv); + ResetUpdateCounts(this.RefMv); + ResetUpdateCounts(this.Drl); + ResetUpdateCounts(this.SingleReference); + ResetUpdateCounts(this.CompInter); + ResetUpdateCounts(this.CompoundReferenceType); + ResetUpdateCounts(this.UnidirectionalCompoundReference); + ResetUpdateCounts(this.CompoundReference); + ResetUpdateCounts(this.CompoundBackwardReference); + ResetUpdateCounts(this.InterCompoundMode); + ResetUpdateCounts(this.InterIntra); + ResetUpdateCounts(this.InterIntraMode); + ResetUpdateCounts(this.WedgeInterIntra); + ResetUpdateCounts(this.CompoundType); + ResetUpdateCounts(this.WedgeIndex); + ResetUpdateCounts(this.CompoundIndex); + ResetUpdateCounts(this.CompoundGroupIndex); + ResetUpdateCounts(this.MotionMode); + ResetUpdateCounts(this.Obmc); + ResetUpdateCounts(this.SwitchableInterpolation); + ResetUpdateCounts(this.UvMode); + ResetUpdateCounts(this.Skip); + ResetUpdateCounts(this.SkipMode); + this.DeltaLoopFilterAbsolute.ResetUpdateCount(); + ResetUpdateCounts(this.DeltaLoopFilterMultiAbsolute); + this.DeltaQuantizerAbsolute.ResetUpdateCount(); + ResetUpdateCounts(this.SegmentId); + ResetUpdateCounts(this.SegmentIdPredicted); + ResetUpdateCounts(this.AngleDelta); + this.FilterIntraMode.ResetUpdateCount(); + ResetUpdateCounts(this.FilterIntra); + ResetUpdateCounts(this.TransformSize); + ResetUpdateCounts(this.TransformPartition); + ResetUpdateCounts(this.EndOfBlockFlag); + ResetUpdateCounts(this.CoefficientsBase); + ResetUpdateCounts(this.BaseEndOfBlock); + ResetUpdateCounts(this.DcSign); + ResetUpdateCounts(this.CoefficientsBaseRange); + ResetUpdateCounts(this.TransformBlockSkip); + ResetUpdateCounts(this.EndOfBlockExtra); + this.ChromaFromLumaSign.ResetUpdateCount(); + ResetUpdateCounts(this.ChromaFromLumaAlpha); + ResetUpdateCounts(this.IntraExtendedTransform); + ResetUpdateCounts(this.InterExtendedTransform); + } + + /// + /// Copies one distribution row into an existing row with the same default-table shape. + /// + /// The source distribution row. + /// The destination distribution row. + private static void CopyState(Av1Distribution[] source, Av1Distribution[] destination) + { + for (int index = 0; index < source.Length; index++) + { + destination[index].CopyFrom(source[index]); + } + } + + /// + /// Copies a two-dimensional distribution table into an existing table with the same default-table shape. + /// + /// The source distribution table. + /// The destination distribution table. + private static void CopyState(Av1Distribution[][] source, Av1Distribution[][] destination) + { + for (int index = 0; index < source.Length; index++) + { + CopyState(source[index], destination[index]); + } + } + + /// + /// Copies a three-dimensional distribution table into an existing table with the same default-table shape. + /// + /// The source distribution table. + /// The destination distribution table. + private static void CopyState(Av1Distribution[][][] source, Av1Distribution[][][] destination) + { + for (int index = 0; index < source.Length; index++) + { + CopyState(source[index], destination[index]); + } + } + + /// + /// Resets observation counts in one distribution row. + /// + /// The distribution row to reset. + private static void ResetUpdateCounts(Av1Distribution[] distributions) + { + for (int index = 0; index < distributions.Length; index++) + { + distributions[index].ResetUpdateCount(); + } + } + + /// + /// Resets observation counts in a two-dimensional distribution table. + /// + /// The distribution table to reset. + private static void ResetUpdateCounts(Av1Distribution[][] distributions) + { + for (int index = 0; index < distributions.Length; index++) + { + ResetUpdateCounts(distributions[index]); + } + } + + /// + /// Resets observation counts in a three-dimensional distribution table. + /// + /// The distribution table to reset. + private static void ResetUpdateCounts(Av1Distribution[][][] distributions) + { + for (int index = 0; index < distributions.Length; index++) + { + ResetUpdateCounts(distributions[index]); + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Entropy/Av1FrameEntropyContexts.cs b/src/ImageSharp/Formats/Heif/Av1/Entropy/Av1FrameEntropyContexts.cs new file mode 100644 index 0000000000..4394aa06ea --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Entropy/Av1FrameEntropyContexts.cs @@ -0,0 +1,140 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Entropy; + +/// +/// Owns the reusable frame-base, tile-working, and published AV1 entropy contexts for one decoder session. +/// +internal sealed class Av1FrameEntropyContexts +{ + /// + /// The maximum number of live reference-map and presentation owners plus the newly reconstructed frame awaiting + /// commit. + /// + private const int MaximumSnapshotCount = Av1Constants.ReferenceFrameCount + 2; + + /// + /// Session-local returned snapshot graphs available for later refreshed frames. + /// + private InlineArray10 returnedSnapshots; + + /// + /// The base quantizer index used to initialize a newly required snapshot graph. + /// + private int currentQIndex; + + /// + /// Initializes a new instance of the class. + /// + /// The initial frame base quantizer index. + public Av1FrameEntropyContexts(int qIndex) + { + this.Base = new(qIndex); + this.Working = new(qIndex); + this.Published = new(qIndex); + this.currentQIndex = qIndex; + } + + /// + /// Gets the unchanged frame context from which each independently decoded tile starts. + /// + public Av1FrameEntropyContext Base { get; } + + /// + /// Gets the tile-local context reused sequentially for each tile in the current frame. + /// + public Av1FrameEntropyContext Working { get; } + + /// + /// Gets the completed frame context selected by the signaled context-update tile, or the unchanged frame-base + /// context when frame-end updates are disabled. + /// + public Av1FrameEntropyContext Published { get; } + + /// + /// Initializes frame entropy state from either a retained primary reference or normative quantizer-band defaults. + /// + /// The frame base quantizer index selecting coefficient distribution defaults. + /// + /// The retained primary-reference context, or when the frame selects normative defaults. + /// + public void BeginFrame(int qIndex, Av1FrameEntropyContext? primaryReferenceContext) + { + this.currentQIndex = qIndex; + if (primaryReferenceContext is null) + { + this.Base.ResetToDefaults(qIndex); + } + else + { + // A retained context is independent from the working and published graphs. Copying it here preserves the + // reference owner's snapshot while the current frame adapts its own tile-local state. + this.Base.CopyFrom(primaryReferenceContext); + } + + // The context-update tile can precede later tiles. Published therefore cannot alias Working: a later tile + // must be free to overwrite Working while the selected completed-frame state remains available to the owner. + this.Base.SnapshotTo(this.Published); + } + + /// + /// Clears active frame entropy state when a new coded sequence invalidates the complete reference map. + /// + public void Reset() + { + this.currentQIndex = 0; + this.Base.ResetToDefaults(this.currentQIndex); + this.Base.SnapshotTo(this.Working); + this.Base.SnapshotTo(this.Published); + + // Returned graphs contain no live reference state and remain private to this decoder. Retaining them here + // allows the next sequence to reuse peak reference ownership without a static cross-decode pool. + } + + /// + /// Rents an independently owned, reset-counter snapshot of the completed frame entropy context. + /// + /// The snapshot that must later be returned through . + public Av1FrameEntropyContext RentPublishedSnapshot() + { + Av1FrameEntropyContext? snapshot = null; + for (int snapshotIndex = 0; snapshotIndex < MaximumSnapshotCount; snapshotIndex++) + { + Av1FrameEntropyContext? returnedSnapshot = this.returnedSnapshots[snapshotIndex]; + if (returnedSnapshot is null) + { + continue; + } + + this.returnedSnapshots[snapshotIndex] = null; + snapshot = returnedSnapshot; + break; + } + + // Eight slots can own distinct frames while the selected output owns a ninth frame no longer present in + // the map. Rent one further graph before commit releases the owner displaced by the completed frame. + snapshot ??= new(this.currentQIndex); + + this.Published.SnapshotTo(snapshot); + return snapshot; + } + + /// + /// Returns a retained-frame entropy snapshot to this decoder session for later reuse. + /// + /// The snapshot whose reference-frame ownership has ended. + public void ReturnSnapshot(Av1FrameEntropyContext snapshot) + { + // The fixed capacity covers eight distinct slot owners, one detached presentation owner, and the replacement + // frame rented before commit. Av1ReferenceFrame returns each graph exactly once, so one slot is always free. + for (int snapshotIndex = 0; snapshotIndex < MaximumSnapshotCount; snapshotIndex++) + { + if (this.returnedSnapshots[snapshotIndex] is null) + { + this.returnedSnapshots[snapshotIndex] = snapshot; + return; + } + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Entropy/Av1MotionVectorContext.cs b/src/ImageSharp/Formats/Heif/Av1/Entropy/Av1MotionVectorContext.cs new file mode 100644 index 0000000000..5a28d06735 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Entropy/Av1MotionVectorContext.cs @@ -0,0 +1,414 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1.Motion; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Entropy; + +/// +/// Owns one independently adaptive AV1 motion-vector entropy context. +/// +/// +/// Normal inter-prediction vectors and intra-block-copy displacement vectors use identical initial distributions, but +/// each syntax domain owns a separate instance so observations from one domain cannot adapt the other. +/// +internal sealed class Av1MotionVectorContext +{ + /// + /// The number of magnitude classes defined by AV1. + /// + private const int MagnitudeClassCount = 11; + + /// + /// The number of integer magnitude bits coded directly for class zero. + /// + private const int ClassZeroBitCount = 1; + + /// + /// The number of integer magnitude offsets represented by class zero. + /// + private const int ClassZeroSize = 1 << ClassZeroBitCount; + + /// + /// Defines one non-mutating cost or mutating write operation over shared motion-vector syntax. + /// + public interface IMotionVectorSymbolOperation + { + /// + /// Processes one entropy-coded symbol. + /// + /// The tile range encoder. + /// The zero-based symbol. + /// The live symbol distribution. + /// The symbol cost in 1/512-bit units, or zero when writing. + public static abstract int ProcessSymbol(Av1SymbolWriter writer, int symbol, Av1Distribution distribution); + } + + /// + /// Gets the distribution selecting which vector components are nonzero. + /// + public Av1Distribution Joint { get; } = new(4096, 11264, 19328); + + /// + /// Gets the adaptive distributions for the vertical vector component. + /// + public Component Vertical { get; } = new(); + + /// + /// Gets the adaptive distributions for the horizontal vector component. + /// + public Component Horizontal { get; } = new(); + + /// + /// Replaces every motion-vector distribution with state copied from another context. + /// + /// The motion-vector context state to copy. + public void CopyFrom(Av1MotionVectorContext source) + { + this.Joint.CopyFrom(source.Joint); + this.Vertical.CopyFrom(source.Vertical); + this.Horizontal.CopyFrom(source.Horizontal); + } + + /// + /// Resets every observation count used to adapt the motion-vector distributions. + /// + public void ResetUpdateCounts() + { + this.Joint.ResetUpdateCount(); + this.Vertical.ResetUpdateCounts(); + this.Horizontal.ResetUpdateCounts(); + } + + /// + /// Reads a motion-vector delta relative to a spatially derived reference. + /// + /// The tile range decoder. + /// The reference motion vector. + /// The fractional precision allowed by the current frame. + /// The decoded motion vector in one-eighth-sample units. + public Av1MotionVector Read(ref Av1SymbolReader reader, Av1MotionVector reference, Av1MotionVectorPrecision precision) + { + int jointType = reader.ReadSymbol(this.Joint); + + // Joint values 1 and 3 carry a horizontal delta; values 2 and 3 carry a vertical delta. Reading only the + // signaled components preserves the normative entropy-symbol order and leaves zero components unadapted. + int row = jointType >= 2 ? this.Vertical.Read(ref reader, precision) : 0; + int column = (jointType & 1) != 0 ? this.Horizontal.Read(ref reader, precision) : 0; + + return reference + new Av1MotionVector(row, column); + } + + /// + /// Writes a motion vector relative to a spatially derived reference. + /// + /// The tile range encoder. + /// The displacement vector to encode. + /// The spatially derived reference vector. + /// The fractional precision selected by the frame header. + public void Write(Av1SymbolWriter writer, Av1MotionVector value, Av1MotionVector reference, Av1MotionVectorPrecision precision) + => _ = this.Process(writer, value, reference, precision); + + /// + /// Measures a motion-vector delta against the live distributions without changing them. + /// + /// The tile range encoder associated with the live context. + /// The motion vector to measure. + /// The spatially derived reference vector. + /// The fractional precision selected by the frame header. + /// The syntax cost in 1/512-bit units. + public int GetCost(Av1SymbolWriter writer, Av1MotionVector value, Av1MotionVector reference, Av1MotionVectorPrecision precision) + => this.Process(writer, value, reference, precision); + + /// + /// Processes one complete motion-vector delta through a closed symbol operation. + /// + private int Process(Av1SymbolWriter writer, Av1MotionVector value, Av1MotionVector reference, Av1MotionVectorPrecision precision) + where TOperation : struct, IMotionVectorSymbolOperation + { + int row = value.Row - reference.Row; + int column = value.Column - reference.Column; + + // Bit zero signals a horizontal delta and bit one signals a vertical delta, producing the four normative + // zero/horizontal/vertical/both joint symbols without a lookup. + int jointType = (row != 0 ? 2 : 0) | (column != 0 ? 1 : 0); + + int rate = TOperation.ProcessSymbol(writer, jointType, this.Joint); + if (row != 0) + { + rate += this.Vertical.Process(writer, row, precision); + } + + if (column != 0) + { + rate += this.Horizontal.Process(writer, column, precision); + } + + return rate; + } + + /// + /// Emits motion-vector syntax and reports no estimated rate. + /// + private readonly struct MotionVectorWriteOperation : IMotionVectorSymbolOperation + { + /// + public static int ProcessSymbol(Av1SymbolWriter writer, int symbol, Av1Distribution distribution) + { + writer.WriteSymbol(symbol, distribution); + return 0; + } + } + + /// + /// Measures motion-vector syntax against the live distributions without changing them. + /// + private readonly struct MotionVectorCostOperation : IMotionVectorSymbolOperation + { + /// + public static int ProcessSymbol(Av1SymbolWriter writer, int symbol, Av1Distribution distribution) + => Av1ProbabilityCost.GetSymbolCost(distribution, symbol); + } + + /// + /// Owns the adaptive magnitude distributions for one motion-vector component. + /// + public sealed class Component + { + /// + /// Gets the distribution selecting the magnitude class of a nonzero component. + /// + public Av1Distribution MagnitudeClass { get; } = new(28672, 30976, 31858, 32320, 32551, 32656, 32740, 32757, 32762, 32767); + + /// + /// Gets the fractional distributions selected by the two class-zero integer offsets. + /// + public Av1Distribution[] ClassZeroFractional { get; } = + [ + new(16384, 24576, 26624), + new(12288, 21248, 24128) + ]; + + /// + /// Gets the fractional distribution used by nonzero magnitude classes. + /// + public Av1Distribution Fractional { get; } = new(8192, 17408, 21248); + + /// + /// Gets the distribution selecting the sign of a nonzero component. + /// + public Av1Distribution Sign { get; } = new(16384); + + /// + /// Gets the eighth-sample distribution used by class-zero magnitudes. + /// + public Av1Distribution ClassZeroHighPrecision { get; } = new(20480); + + /// + /// Gets the eighth-sample distribution used by nonzero magnitude classes. + /// + public Av1Distribution HighPrecision { get; } = new(16384); + + /// + /// Gets the distribution selecting either of the two class-zero integer magnitude offsets. + /// + public Av1Distribution ClassZero { get; } = new(27648); + + /// + /// Gets the binary distributions that reconstruct larger integer magnitude offsets from least to most significant bit. + /// + public Av1Distribution[] OffsetBits { get; } = + [ + new(17408), new(17920), new(18944), new(20480), new(22528), + new(24576), new(28672), new(29952), new(29952), new(30720) + ]; + + /// + /// Replaces every component distribution with state copied from another component. + /// + /// The component state to copy. + public void CopyFrom(Component source) + { + this.MagnitudeClass.CopyFrom(source.MagnitudeClass); + + for (int offset = 0; offset < this.ClassZeroFractional.Length; offset++) + { + this.ClassZeroFractional[offset].CopyFrom(source.ClassZeroFractional[offset]); + } + + this.Fractional.CopyFrom(source.Fractional); + this.Sign.CopyFrom(source.Sign); + this.ClassZeroHighPrecision.CopyFrom(source.ClassZeroHighPrecision); + this.HighPrecision.CopyFrom(source.HighPrecision); + this.ClassZero.CopyFrom(source.ClassZero); + + for (int bit = 0; bit < this.OffsetBits.Length; bit++) + { + this.OffsetBits[bit].CopyFrom(source.OffsetBits[bit]); + } + } + + /// + /// Resets every observation count used to adapt one component's distributions. + /// + public void ResetUpdateCounts() + { + this.MagnitudeClass.ResetUpdateCount(); + + for (int offset = 0; offset < this.ClassZeroFractional.Length; offset++) + { + this.ClassZeroFractional[offset].ResetUpdateCount(); + } + + this.Fractional.ResetUpdateCount(); + this.Sign.ResetUpdateCount(); + this.ClassZeroHighPrecision.ResetUpdateCount(); + this.HighPrecision.ResetUpdateCount(); + this.ClassZero.ResetUpdateCount(); + + for (int bit = 0; bit < this.OffsetBits.Length; bit++) + { + this.OffsetBits[bit].ResetUpdateCount(); + } + } + + /// + /// Reads one signed motion-vector component at the requested precision. + /// + /// The tile range decoder. + /// The fractional precision allowed by the current frame. + /// The signed component in one-eighth-sample units. + public int Read(ref Av1SymbolReader reader, Av1MotionVectorPrecision precision) + { + bool isNegative = reader.ReadSymbol(this.Sign) != 0; + int magnitudeClass = reader.ReadSymbol(this.MagnitudeClass); + bool isClassZero = magnitudeClass == 0; + int integerOffset; + int magnitudeBase; + + if (isClassZero) + { + integerOffset = reader.ReadSymbol(this.ClassZero); + magnitudeBase = 0; + } + else + { + int bitCount = magnitudeClass + ClassZeroBitCount - 1; + integerOffset = 0; + + for (int bit = 0; bit < bitCount; bit++) + { + // AV1 transmits the integer offset least-significant bit first, with an independently adapting + // distribution for every bit position. + integerOffset |= reader.ReadSymbol(this.OffsetBits[bit]) << bit; + } + + // Class one uses a base of two whole samples, or sixteen eighth-sample units, and every later class doubles + // that base. CLASS0_SIZE shifted by class + 2 expresses the same scale directly in eighth-sample units. + magnitudeBase = ClassZeroSize << (magnitudeClass + 2); + } + + int fractional; + int highPrecision; + + if (precision != Av1MotionVectorPrecision.Integer) + { + // Class-zero magnitudes select one of two fractional CDFs using the already decoded integer offset; + // larger classes share one fractional CDF because their expanded integer range supplies the context. + Av1Distribution fractionalDistribution = isClassZero ? this.ClassZeroFractional[integerOffset] : this.Fractional; + fractional = reader.ReadSymbol(fractionalDistribution); + + // Quarter-sample motion omits the eighth-sample symbol. The normative implicit one, combined with the + // final increment below, constrains the result to even one-eighth-sample units. + highPrecision = precision == Av1MotionVectorPrecision.EighthSample + ? reader.ReadSymbol(isClassZero ? this.ClassZeroHighPrecision : this.HighPrecision) + : 1; + } + else + { + // Integer motion omits both fractional symbols. The implicit maximum values make the low three bits + // all one before the final increment, constraining the result to whole-sample multiples of eight. + fractional = 3; + highPrecision = 1; + } + + // The entropy syntax represents magnitude minus one. Integer offset occupies bits three and above, + // fractional occupies bits one and two, and high precision occupies bit zero, all in one-eighth-sample units. + int magnitude = magnitudeBase + ((integerOffset << 3) | (fractional << 1) | highPrecision) + 1; + + return isNegative ? -magnitude : magnitude; + } + + /// + /// Writes one signed motion-vector component. + /// + /// The tile range encoder. + /// The nonzero component in one-eighth-sample units. + /// The fractional precision selected by the frame header. + public void Write(Av1SymbolWriter writer, int value, Av1MotionVectorPrecision precision) + => _ = this.Process(writer, value, precision); + + /// + /// Processes one nonzero signed component through the shared motion-vector symbol operation. + /// + public int Process(Av1SymbolWriter writer, int value, Av1MotionVectorPrecision precision) + where TOperation : struct, IMotionVectorSymbolOperation + { + int magnitude = Math.Abs(value); + int precisionMask = precision == Av1MotionVectorPrecision.Integer + ? 7 + : precision == Av1MotionVectorPrecision.QuarterSample ? 1 : 0; + + DebugGuard.IsTrue( + magnitude > 0 && (magnitude & precisionMask) == 0, + "Motion-vector components must match the frame precision."); + + // The coded value is magnitude minus one. Its whole-sample portion selects the doubling class, + // while the remainder carries integer offset, fractional phase, and the high-precision bit. + int codedMagnitude = magnitude - 1; + uint classValue = (uint)(codedMagnitude >> 3); + int magnitudeClass = classValue == 0 ? 0 : Av1Math.MostSignificantBit(classValue); + DebugGuard.MustBeLessThan(magnitudeClass, MagnitudeClassCount, nameof(magnitudeClass)); + int magnitudeBase = magnitudeClass == 0 ? 0 : ClassZeroSize << (magnitudeClass + 2); + int offset = codedMagnitude - magnitudeBase; + int integerOffset = offset >> 3; + int fractional = (offset >> 1) & 3; + int highPrecision = offset & 1; + int rate = TOperation.ProcessSymbol(writer, value < 0 ? 1 : 0, this.Sign); + rate += TOperation.ProcessSymbol(writer, magnitudeClass, this.MagnitudeClass); + + if (magnitudeClass == 0) + { + rate += TOperation.ProcessSymbol(writer, integerOffset, this.ClassZero); + } + else + { + for (int bit = 0; bit < magnitudeClass; bit++) + { + // Integer offsets are transmitted least-significant bit first through independent models. + rate += TOperation.ProcessSymbol(writer, (integerOffset >> bit) & 1, this.OffsetBits[bit]); + } + } + + if (precision != Av1MotionVectorPrecision.Integer) + { + Av1Distribution fractionalDistribution = magnitudeClass == 0 + ? this.ClassZeroFractional[integerOffset] + : this.Fractional; + + rate += TOperation.ProcessSymbol(writer, fractional, fractionalDistribution); + } + + if (precision == Av1MotionVectorPrecision.EighthSample) + { + Av1Distribution highPrecisionDistribution = magnitudeClass == 0 + ? this.ClassZeroHighPrecision + : this.HighPrecision; + + rate += TOperation.ProcessSymbol(writer, highPrecision, highPrecisionDistribution); + } + + return rate; + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Entropy/Av1NzMap.cs b/src/ImageSharp/Formats/Heif/Av1/Entropy/Av1NzMap.cs new file mode 100644 index 0000000000..0a56ec06de --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Entropy/Av1NzMap.cs @@ -0,0 +1,501 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Entropy; + +/// +/// Derives the AV1 nonzero-coefficient probability context from neighboring coefficient levels and transform geometry. +/// +internal static class Av1NzMap +{ + /// + /// The first one-dimensional nonzero-map context, immediately after the 26 two-dimensional contexts. + /// + private const int NzMapContext0 = 26; + + /// + /// The second one-dimensional position band. + /// + private const int NzMapContext5 = NzMapContext0 + 5; + + /// + /// The final one-dimensional position band. + /// + private const int NzMapContext10 = NzMapContext0 + 10; + + /// + /// Maps a horizontal or vertical coefficient coordinate to its one-dimensional context offset. + /// + private static readonly int[] NzMapContextOffset1d = [ + NzMapContext0, NzMapContext5, NzMapContext10, NzMapContext10, NzMapContext10, NzMapContext10, NzMapContext10, + NzMapContext10, NzMapContext10, NzMapContext10, NzMapContext10, NzMapContext10, NzMapContext10, NzMapContext10, + NzMapContext10, NzMapContext10, NzMapContext10, NzMapContext10, NzMapContext10, NzMapContext10, NzMapContext10, + NzMapContext10, NzMapContext10, NzMapContext10, NzMapContext10, NzMapContext10, NzMapContext10, NzMapContext10, + NzMapContext10, NzMapContext10, NzMapContext10, NzMapContext10, + ]; + + /// + /// The row-major positional context offsets for a 4x4 two-dimensional transform. + /// + private static readonly int[] NzMapContextOffset4x4 = [ + 0, 1, 6, 6, + 1, 6, 6, 21, + 6, 6, 21, 21, + 6, 21, 21, 21]; + + /// + /// The row-major positional context offsets for an 8x8 two-dimensional transform. + /// + private static readonly int[] NzMapContextOffset8x8 = [ + 0, 1, 6, 6, 21, 21, 21, 21, + 1, 6, 6, 21, 21, 21, 21, 21, + 6, 6, 21, 21, 21, 21, 21, 21, + 6, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, + ]; + + /// + /// The row-major positional context offsets for a 16x16 two-dimensional transform. + /// + private static readonly int[] NzMapContextOffset16x16 = [ + 0, 1, 6, 6, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 1, 6, 6, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 6, 6, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 6, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + ]; + + /// + /// The row-major positional context offsets for a 32x32 two-dimensional transform. + /// + private static readonly int[] NzMapContextOffset32x32 = [ + 0, 1, 6, 6, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 1, 6, 6, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 6, 6, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 6, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + ]; + + /// + /// The row-major positional context offsets for an 8x4 two-dimensional transform. + /// + private static readonly int[] NzMapContextOffset8x4 = [ + 0, 16, 6, 6, 21, 21, 21, 21, + 16, 16, 6, 21, 21, 21, 21, 21, + 16, 16, 21, 21, 21, 21, 21, 21, + 16, 16, 21, 21, 21, 21, 21, 21, + ]; + + /// + /// The row-major positional context offsets for a 16x8 two-dimensional transform. + /// + private static readonly int[] NzMapContextOffset16x8 = [ + 0, 16, 6, 6, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 16, 16, 6, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 16, 16, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 16, 16, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 16, 16, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 16, 16, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 16, 16, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 16, 16, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + ]; + + /// + /// The row-major positional context offsets for a 16x32 two-dimensional transform. + /// + private static readonly int[] NzMapContextOffset16x32 = [ + 0, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, + 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, + 6, 6, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 6, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + ]; + + /// + /// The row-major positional context offsets for a 32x16 two-dimensional transform. + /// + private static readonly int[] NzMapContextOffset32x16 = [ + 0, 16, 6, 6, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 16, 16, 6, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 16, 16, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 16, 16, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 16, 16, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 16, 16, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 16, 16, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 16, 16, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 16, 16, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 16, 16, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 16, 16, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 16, 16, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 16, 16, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 16, 16, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 16, 16, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 16, 16, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + ]; + + /// + /// The row-major positional context offsets for the coded low-frequency region of a 32x64 transform. + /// + private static readonly int[] NzMapContextOffset32x64 = [ + 0, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, + 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, + 6, 6, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 6, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + ]; + + /// + /// The row-major positional context offsets for the coded low-frequency region of a 64x32 transform. + /// + private static readonly int[] NzMapContextOffset64x32 = [ + 0, 16, 6, 6, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 16, 16, 6, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 16, 16, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 16, 16, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 16, 16, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 16, 16, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 16, 16, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 16, 16, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 16, 16, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 16, 16, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 16, 16, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 16, 16, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 16, 16, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 16, 16, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 16, 16, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 16, 16, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 16, 16, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 16, 16, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 16, 16, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 16, 16, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 16, 16, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 16, 16, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 16, 16, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 16, 16, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 16, 16, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 16, 16, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 16, 16, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 16, 16, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 16, 16, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 16, 16, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 16, 16, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 16, 16, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + ]; + + /// + /// The row-major positional context offsets for a 4x16 two-dimensional transform. + /// + private static readonly int[] NzMapContextOffset4x16 = [ + 0, 11, 11, 11, + 11, 11, 11, 11, + 6, 6, 21, 21, + 6, 21, 21, 21, + 21, 21, 21, 21, + 21, 21, 21, 21, + 21, 21, 21, 21, + 21, 21, 21, 21, + 21, 21, 21, 21, + 21, 21, 21, 21, + 21, 21, 21, 21, + 21, 21, 21, 21, + 21, 21, 21, 21, + 21, 21, 21, 21, + 21, 21, 21, 21, + 21, 21, 21, 21, + ]; + + /// + /// The row-major positional context offsets for a 16x4 two-dimensional transform. + /// + private static readonly int[] NzMapContextOffset16x4 = [ + 0, 16, 6, 6, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 16, 16, 6, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 16, 16, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 16, 16, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + ]; + + /// + /// The row-major positional context offsets for an 8x32 two-dimensional transform. + /// + private static readonly int[] NzMapContextOffset8x32 = [ + 0, 11, 11, 11, 11, 11, 11, 11, + 11, 11, 11, 11, 11, 11, 11, 11, + 6, 6, 21, 21, 21, 21, 21, 21, + 6, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, + 21, 21, 21, 21, 21, 21, 21, 21, + ]; + + /// + /// The row-major positional context offsets for a 32x8 two-dimensional transform. + /// + private static readonly int[] NzMapContextOffset32x8 = [ + 0, 16, 6, 6, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 16, 16, 6, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 16, 16, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 16, 16, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 16, 16, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 16, 16, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 16, 16, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + 16, 16, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, + ]; + + /// + /// Maps each AV1 transform-size value to its row-major two-dimensional positional context table. + /// + private static readonly int[][] NzMapContextOffset = [ + + // Several rectangular transforms reuse a prefix of the next larger table. Every reused prefix has the same + // row-major stride as the adjusted coefficient plane, which avoids duplicating identical context offsets. + NzMapContextOffset4x4, // TX_4x4 + NzMapContextOffset8x8, // TX_8x8 + NzMapContextOffset16x16, // TX_16x16 + NzMapContextOffset32x32, // TX_32x32 + NzMapContextOffset32x32, // TX_64x64 + NzMapContextOffset4x16, // TX_4x8 + NzMapContextOffset8x4, // TX_8x4 + NzMapContextOffset8x32, // TX_8x16 + NzMapContextOffset16x8, // TX_16x8 + NzMapContextOffset16x32, // TX_16x32 + NzMapContextOffset32x16, // TX_32x16 + NzMapContextOffset32x64, // TX_32x64 + NzMapContextOffset64x32, // TX_64x32 + NzMapContextOffset4x16, // TX_4x16 + NzMapContextOffset16x4, // TX_16x4 + NzMapContextOffset8x32, // TX_8x32 + NzMapContextOffset32x8, // TX_32x8 + NzMapContextOffset16x32, // TX_16x64 + NzMapContextOffset64x32, // TX_64x16 + ]; + + /// + /// Sums the clipped magnitudes of the transform-class-specific forward coefficient neighbors. + /// + /// The padded absolute-coefficient level plane. + /// The coefficient position. + /// The transform direction class selecting the neighbor pattern. + /// The summed neighbor magnitude used to select a nonzero-map context. + public static int GetNzMagnitude(Av1LevelBuffer levels, Point position, Av1TransformClass transformClass) + { + int mag; + Span row0 = levels.GetRow(position.Y)[position.X..]; + Span row1 = levels.GetRow(position.Y + 1)[position.X..]; + Span row2 = levels.GetRow(position.Y + 2)[position.X..]; + + // Large levels must not dominate probability selection; AV1 contributes at most three from each neighbor. + mag = ClipMax3(row0[1]); // { 0, 1 } + mag += ClipMax3(row1[0]); // { 1, 0 } + + switch (transformClass) + { + case Av1TransformClass.Class2D: + mag += ClipMax3(row1[1]); // { 1, 1 } + mag += ClipMax3(row0[2]); // { 0, 2 } + mag += ClipMax3(row2[0]); // { 2, 0 } + break; + + case Av1TransformClass.ClassVertical: + Span row3 = levels.GetRow(position.Y + 3)[position.X..]; + Span row4 = levels.GetRow(position.Y + 4)[position.X..]; + mag += ClipMax3(row2[0]); // { 2, 0 } + mag += ClipMax3(row3[0]); // { 3, 0 } + mag += ClipMax3(row4[0]); // { 4, 0 } + break; + case Av1TransformClass.ClassHorizontal: + mag += ClipMax3(row0[2]); // { 0, 2 } + mag += ClipMax3(row0[3]); // { 0, 3 } + mag += ClipMax3(row0[4]); // { 0, 4 } + break; + } + + return mag; + } + + /// + /// Combines a neighboring-level statistic with the coefficient's transform-class-specific position band. + /// + /// The clipped sum of the applicable forward-neighbor magnitudes. + /// The coefficient position. + /// The coded transform size selecting the positional table. + /// The transform direction class. + /// The nonzero-map probability context. + public static int GetNzMapContextFromStats(int stats, Point position, Av1TransformSize transformSize, Av1TransformClass transformClass) + { + // The DC coefficient has a dedicated 2D context independent of neighboring levels. + if (transformClass == Av1TransformClass.Class2D && position.X == 0 && position.Y == 0) + { + return 0; + } + + // Rounding the neighbor sum before clipping produces the five AV1 magnitude bands 0 through 4. + int ctx = (stats + 1) >> 1; + ctx = Math.Min(ctx, 4); + switch (transformClass) + { + case Av1TransformClass.Class2D: + // The tables preserve AV1's distinct early-row and early-column bands for rectangular transforms. + return ctx + GetNzMapContext(transformSize, position); + case Av1TransformClass.ClassHorizontal: + return ctx + NzMapContextOffset1d[position.X]; + case Av1TransformClass.ClassVertical: + return ctx + NzMapContextOffset1d[position.Y]; + default: + break; + } + + return 0; + } + + /// + /// Gets the two-dimensional positional context offset for a coefficient position. + /// + /// The coded transform size. + /// The coefficient position. + /// The positional context offset. + public static int GetNzMapContext(Av1TransformSize transformSize, Point position) + { + // AV1 codes only the low-frequency 32-sample region of a 64-point transform dimension. The table still + // uses the signaled shape to select tall or wide bands, but its row-major stride follows the coded region. + int codedWidth = transformSize.GetAdjusted().GetWidth(); + return GetNzMapContext(transformSize, position.X + (position.Y * codedWidth)); + } + + /// + /// Gets the two-dimensional positional context offset for a row-major coefficient index. + /// + /// The coded transform size. + /// The row-major coefficient index. + /// The positional context offset. + public static int GetNzMapContext(Av1TransformSize transformSize, int position) => NzMapContextOffset[(int)transformSize][position]; + + /// + /// Clips a coefficient magnitude to the maximum contribution allowed per neighbor. + /// + /// The coefficient magnitude. + /// The magnitude limited to three. + private static int ClipMax3(int value) => Math.Min(value, 3); +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Entropy/Av1ProbabilityCost.cs b/src/ImageSharp/Formats/Heif/Av1/Entropy/Av1ProbabilityCost.cs new file mode 100644 index 0000000000..d9ff3958b1 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Entropy/Av1ProbabilityCost.cs @@ -0,0 +1,84 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Numerics; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Entropy; + +/// +/// Converts AV1 symbol probabilities into fixed-point encoder rate costs. +/// +internal static class Av1ProbabilityCost +{ + /// + /// The number of fractional bits in an encoder rate cost. + /// + public const int CostShift = 9; + + /// + /// Gets the probability costs for normalized eight-bit probabilities from 128 through 255. + /// + private static ReadOnlySpan ProbabilityCosts => + [ + 512, 506, 501, 495, 489, 484, 478, 473, 467, 462, 456, 451, 446, 441, 435, + 430, 425, 420, 415, 410, 405, 400, 395, 390, 385, 380, 375, 371, 366, 361, + 356, 352, 347, 343, 338, 333, 329, 324, 320, 316, 311, 307, 302, 298, 294, + 289, 285, 281, 277, 273, 268, 264, 260, 256, 252, 248, 244, 240, 236, 232, + 228, 224, 220, 216, 212, 209, 205, 201, 197, 194, 190, 186, 182, 179, 175, + 171, 168, 164, 161, 157, 153, 150, 146, 143, 139, 136, 132, 129, 125, 122, + 119, 115, 112, 109, 105, 102, 99, 95, 92, 89, 86, 82, 79, 76, 73, 70, + 66, 63, 60, 57, 54, 51, 48, 45, 42, 38, 35, 32, 29, 26, 23, 20, 18, 15, + 12, 9, 6, 3 + ]; + + /// + /// Gets the fixed-point cost of writing the requested number of equiprobable bits. + /// + /// The number of bits. + /// The rate cost in 1/512-bit units. + public static int GetLiteralCost(int bitCount) => bitCount << CostShift; + + /// + /// Gets the fixed-point cost of coding one symbol from an inverse cumulative distribution. + /// + /// The distribution used by the entropy writer. + /// The zero-based symbol. + /// The rate cost in 1/512-bit units. + public static int GetSymbolCost(Av1Distribution distribution, int symbol) + { + int inverseLower = symbol == 0 ? Av1Distribution.ProbabilityTop : (int)distribution[symbol - 1]; + int inverseUpper = (int)distribution[symbol]; + return GetSymbolCost(inverseLower - inverseUpper); + } + + /// + /// Gets the fixed-point cost of an entropy-coded symbol with a Q15 probability. + /// + /// The Q15 probability numerator. + /// The rate cost in 1/512-bit units. + public static int GetSymbolCost(int probability) + { + // The range coder reserves a minimum interval even when CDF adaptation collapses a symbol's mass. + // RD costs use that same floor; the raw probability conversion below retains its separate numerical domain. + return GetProbabilityCost(Math.Max(probability, Av1Distribution.ProbabilityMinimum)); + } + + /// + /// Gets the fixed-point cost of a Q15 probability. + /// + /// The Q15 probability numerator. + /// The rate cost in 1/512-bit units. + public static int GetProbabilityCost(int probability) + { + probability = Math.Clamp(probability, 1, Av1Distribution.ProbabilityTop - 1); + int shift = 14 - BitOperations.Log2((uint)probability); + + // Normalization puts every probability in the upper half of an eight-bit range. The lookup therefore + // covers one binary order of magnitude, while the shift contributes the exact number of whole bits. + int normalizedProbability = (((probability << shift) * 256) + (Av1Distribution.ProbabilityTop >> 1)) + / Av1Distribution.ProbabilityTop; + normalizedProbability = Math.Min(normalizedProbability, 255); + + return ProbabilityCosts[normalizedProbability - 128] + (shift << CostShift); + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Entropy/Av1RateDistortion.cs b/src/ImageSharp/Formats/Heif/Av1/Entropy/Av1RateDistortion.cs new file mode 100644 index 0000000000..fee03ebaa7 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Entropy/Av1RateDistortion.cs @@ -0,0 +1,319 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.Intrinsics; +using SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.Quantizers; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Entropy; + +/// +/// Combines fixed-point AV1 rate and distortion values for encoder decisions. +/// +internal static class Av1RateDistortion +{ + /// + /// Each fitted curve contains 65 equally spaced samples, including the cubic interpolation endpoints. + /// + private const int ModelCurveLength = 65; + + /// + /// Gets the rate-curve category for each AV1 block geometry. + /// + private static ReadOnlySpan ModelRateCategories => [0, 0, 0, 1, 1, 1, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 1, 1, 2, 2, 3, 3]; + + /// + /// Gets the four block-size rate curves in fixed-point bit-cost units per sample. + /// + private static ReadOnlySpan ModelRateCurves => + [ + 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, + 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, + 0.000000, 118.257702, 120.210658, 121.434853, 122.100487, + 122.377758, 122.436865, 72.290102, 96.974289, 101.652727, + 126.830141, 140.417377, 157.644879, 184.315291, 215.823873, + 262.300169, 335.919859, 420.624173, 519.185032, 619.854243, + 726.053595, 827.663369, 933.127475, 1037.988755, 1138.839609, + 1233.342933, 1333.508064, 1428.760126, 1533.396364, 1616.952052, + 1744.539319, 1803.413586, 1951.466618, 1994.227838, 2086.031680, + 2148.635443, 2239.068450, 2222.590637, 2338.859809, 2402.929011, + 2418.727875, 2435.342670, 2471.159469, 2523.187446, 2591.183827, + 2674.905840, 2774.110714, 2888.555675, 3017.997952, 3162.194773, + 3320.903365, 3493.880956, 3680.884773, 3881.672045, 4096.000000, + 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, + 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, + 0.000000, 13.087244, 15.919735, 25.930313, 24.412411, + 28.567417, 29.924194, 30.857010, 32.742979, 36.382570, + 39.210386, 42.265690, 47.378572, 57.014850, 82.740067, + 137.346562, 219.968084, 316.781856, 415.643773, 516.706538, + 614.914364, 714.303763, 815.512135, 911.210485, 1008.501528, + 1109.787854, 1213.772279, 1322.922561, 1414.752579, 1510.505641, + 1615.741888, 1697.989032, 1780.123933, 1847.453790, 1913.742309, + 1960.828122, 2047.500168, 2085.454095, 2129.230668, 2158.171824, + 2182.231724, 2217.684864, 2269.589211, 2337.264824, 2420.618694, + 2519.557814, 2633.989178, 2763.819779, 2908.956609, 3069.306660, + 3244.776927, 3435.274401, 3640.706076, 3860.978945, 4096.000000, + 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, + 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, + 0.000000, 4.656893, 5.123633, 5.594132, 6.162376, + 6.918433, 7.768444, 8.739415, 10.105862, 11.477328, + 13.236604, 15.421030, 19.093623, 25.801871, 46.724612, + 98.841054, 181.113466, 272.586364, 359.499769, 445.546343, + 525.944439, 605.188743, 681.793483, 756.668359, 838.486885, + 926.950356, 1015.482542, 1113.353926, 1204.897193, 1288.871992, + 1373.464145, 1455.746628, 1527.796460, 1588.475066, 1658.144771, + 1710.302500, 1807.563351, 1863.197608, 1927.281616, 1964.450872, + 2022.719898, 2100.041145, 2185.205712, 2280.993936, 2387.616216, + 2505.282950, 2634.204540, 2774.591385, 2926.653884, 3090.602436, + 3266.647443, 3454.999303, 3655.868416, 3869.465182, 4096.000000, + 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, + 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, + 0.000000, 0.337370, 0.391916, 0.468839, 0.566334, + 0.762564, 1.069225, 1.384361, 1.787581, 2.293948, + 3.251909, 4.412991, 8.050068, 11.606073, 27.668092, + 65.227758, 128.463938, 202.097653, 262.715851, 312.464873, + 355.601398, 400.609054, 447.201352, 495.761568, 552.871938, + 619.067625, 691.984883, 773.753288, 860.628503, 946.262808, + 1019.805896, 1106.061360, 1178.422145, 1244.852258, 1302.173987, + 1399.650266, 1548.092912, 1545.928652, 1670.817500, 1694.523823, + 1779.195362, 1882.155494, 1990.662097, 2108.325181, 2235.456119, + 2372.366287, 2519.367059, 2676.769812, 2844.885918, 3024.026754, + 3214.503695, 3416.628115, 3630.711389, 3857.064892, 4096.000000, + ]; + + /// + /// Gets the low- and high-error distortion curves in sixteenth-sample-error units. + /// + private static ReadOnlySpan ModelDistortionCurves => + [ + 16.000000, 15.962891, 15.925174, 15.886888, 15.848074, + 15.808770, 15.769015, 15.728850, 15.688313, 15.647445, + 15.606284, 15.564870, 15.525918, 15.483820, 15.373330, + 15.126844, 14.637442, 14.184387, 13.560070, 12.880717, + 12.165995, 11.378144, 10.438769, 9.130790, 7.487633, + 5.688649, 4.267515, 3.196300, 2.434201, 1.834064, + 1.369920, 1.035921, 0.775279, 0.574895, 0.427232, + 0.314123, 0.233236, 0.171440, 0.128188, 0.092762, + 0.067569, 0.049324, 0.036330, 0.027008, 0.019853, + 0.015539, 0.011093, 0.008733, 0.007624, 0.008105, + 0.005427, 0.004065, 0.003427, 0.002848, 0.002328, + 0.001865, 0.001457, 0.001103, 0.000801, 0.000550, + 0.000348, 0.000193, 0.000085, 0.000021, 0.000000, + 16.000000, 15.996116, 15.984769, 15.966413, 15.941505, + 15.910501, 15.873856, 15.832026, 15.785466, 15.734633, + 15.679981, 15.621967, 15.560961, 15.460157, 15.288367, + 15.052462, 14.466922, 13.921212, 13.073692, 12.222005, + 11.237799, 9.985848, 8.898823, 7.423519, 5.995325, + 4.773152, 3.744032, 2.938217, 2.294526, 1.762412, + 1.327145, 1.020728, 0.765535, 0.570548, 0.425833, + 0.313825, 0.232959, 0.171324, 0.128174, 0.092750, + 0.067558, 0.049319, 0.036330, 0.027008, 0.019853, + 0.015539, 0.011093, 0.008733, 0.007624, 0.008105, + 0.005427, 0.004065, 0.003427, 0.002848, 0.002328, + 0.001865, 0.001457, 0.001103, 0.000801, 0.000550, + 0.000348, 0.000193, 0.000085, 0.000021, -0.000000, + ]; + + /// + /// Gets the key-frame rate multiplier for an AV1 quantizer and sample precision. + /// + /// The segment quantizer index. + /// The coded sample bit depth. + /// The rate multiplier. + public static int GetKeyFrameRateMultiplier(int qIndex, Av1BitDepth bitDepth) + { + int quantizer = Av1QuantizationLookup.GetDcQuant(qIndex, 0, bitDepth); + + // Key frames use a quantizer-dependent weight over the squared DC step. High-bit-depth + // distortion is normalized back to the eight-bit domain, so its rate multiplier follows it. + long multiplier = (long)((quantizer * (long)quantizer) * (3.3 + (0.0015 * quantizer))); + int shift = (bitDepth.GetBitCount() - 8) * 2; + if (shift > 0) + { + multiplier = (multiplier + (1L << (shift - 1))) >> shift; + } + + return (int)Math.Max(multiplier, 1); + } + + /// + /// Gets the inter-frame rate multiplier for an AV1 quantizer and sample precision. + /// + /// The segment quantizer index. + /// The coded sample bit depth. + /// The rate multiplier. + public static int GetInterFrameRateMultiplier(int qIndex, Av1BitDepth bitDepth) + { + int quantizer = Av1QuantizationLookup.GetDcQuant(qIndex, 0, bitDepth); + + // Ordinary inter frames use a slightly lower rate weight than key frames, preserving more residual detail. + // Distortion remains normalized to the eight-bit domain before it is combined with this value. + long multiplier = (long)((quantizer * (long)quantizer) * (3.2 + (0.0015 * quantizer))); + int shift = (bitDepth.GetBitCount() - 8) * 2; + if (shift > 0) + { + multiplier = (multiplier + (1L << (shift - 1))) >> shift; + } + + return (int)Math.Max(multiplier, 1); + } + + /// + /// Gets a rate-distortion cost using the encoder probability-cost precision. + /// + /// The rate weight selected by the encoder quality model. + /// The syntax rate in 1/512-bit units. + /// The sample-domain distortion. + /// The rounded weighted rate plus distortion. + public static long GetCost(int rateMultiplier, int rate, long distortion) + { + long weightedRate = (long)rate * rateMultiplier; + long roundedRate = (weightedRate + (1 << (Av1ProbabilityCost.CostShift - 1))) >> Av1ProbabilityCost.CostShift; + return roundedRate + (distortion << 7); + } + + /// + /// Gets the variance-domain cost of a full-pixel motion candidate. + /// + /// The rate weight selected by the encoder quality model. + /// The motion-vector syntax rate in 1/512-bit units. + /// The normalized sample variance. + /// The variance plus the motion-vector error cost. + public static int GetMotionSearchCost(int rateMultiplier, int motionVectorRate, int variance) + { + const int RateMultiplierShift = 6; + const int MotionErrorShift = 14; + int errorPerBit = Math.Max(rateMultiplier >> RateMultiplierShift, 1); + + // Motion search compares pixel variance directly, so the syntax term is reduced to the same + // error domain instead of using the final mode-decision distortion scale. + long weightedRate = (long)motionVectorRate * errorPerBit; + int motionError = (int)((weightedRate + (1 << (MotionErrorShift - 1))) >> MotionErrorShift); + return variance + motionError; + } + + /// + /// Gets the sum-of-absolute-differences rate scale for a frame quantizer. + /// + /// The segment quantizer index. + /// The coded sample bit depth. + /// The multiplier that converts motion-vector rate into the absolute-difference domain. + public static int GetMotionSearchSadPerBit(int qIndex, Av1BitDepth bitDepth) + { + int quantizerDivisor = 1 << (bitDepth.GetBitCount() - 6); + double quantizer = Av1QuantizationLookup.GetAcQuant(qIndex, 0, bitDepth) / (double)quantizerDivisor; + return (int)((0.0418 * quantizer) + 2.4107); + } + + /// + /// Gets the sum-of-absolute-differences cost of a full-pixel motion candidate. + /// + /// The quantizer-derived motion-rate scale. + /// The motion-vector syntax rate in 1/512-bit units. + /// The unnormalized sample-domain absolute difference. + /// The absolute difference plus the motion-vector search cost. + public static int GetMotionSearchSadCost(int sadPerBit, int motionVectorRate, int sumOfAbsoluteDifferences) + { + const int MotionRateShift = 9; + + // Full-pixel traversal uses absolute differences, so its quantizer-derived rate scale is deliberately + // distinct from the variance-domain error-per-bit scale used to compare the resulting search paths. + long weightedRate = (long)motionVectorRate * sadPerBit; + int motionError = (int)((weightedRate + (1 << (MotionRateShift - 1))) >> MotionRateShift); + return sumOfAbsoluteDifferences + motionError; + } + + /// + /// Estimates residual rate and distortion from prediction error without running transforms or quantization. + /// + /// The plane block geometry selecting the fitted rate curve. + /// The visible prediction error normalized to eight-bit precision. + /// The number of visible samples contributing to the error. + /// The plane AC dequantization step at native sample precision. + /// The native sample precision. + /// The block's rate-distortion multiplier. + /// The estimated residual rate in 1/512-bit units. + /// The estimated residual distortion in sixteenth-sample-error units. + public static void ModelPredictionError( + Av1BlockSize blockSize, + long squaredError, + int sampleCount, + int acQuantizer, + Av1BitDepth bitDepth, + int rateMultiplier, + out int rate, + out long distortion) + { + if (squaredError == 0) + { + rate = 0; + distortion = 0; + return; + } + + const double CurveStart = -15.5; + const double CurveStep = 0.5; + const double EndpointMargin = 1E-6; + const double HighErrorThreshold = 16; + const int DistortionScaleShift = 4; + + // Transform dequantizers are scaled by eight. Normalize both their precision and the prediction error + // before taking the logarithmic feature, so the same fitted curves serve eight-, ten-, and twelve-bit input. + int quantizerStep = Math.Max(acQuantizer >> (bitDepth.GetBitCount() - 5), 1); + double normalizedError = (double)squaredError / sampleCount; + double feature = Math.Log2(normalizedError / ((double)quantizerStep * quantizerStep)); + double lastCurvePosition = CurveStart + ((ModelCurveLength - 1) * CurveStep); + feature = Math.Clamp(feature, CurveStart + CurveStep + EndpointMargin, lastCurvePosition - CurveStep - EndpointMargin); + double position = (feature - CurveStart) / CurveStep; + int index = (int)position; + double fraction = position - index; + int rateCategory = ModelRateCategories[(int)blockSize]; + int distortionCategory = normalizedError > HighErrorThreshold ? 1 : 0; + ReadOnlySpan ratePoints = ModelRateCurves.Slice((rateCategory * ModelCurveLength) + index - 1, 4); + ReadOnlySpan distortionPoints = ModelDistortionCurves.Slice((distortionCategory * ModelCurveLength) + index - 1, 4); + double rateEstimate; + double distortionEstimate; + if (Vector128.IsHardwareAccelerated) + { + // The two lanes evaluate rate and distortion together. Keep the cubic polynomial's operation order, + // including its separate multiplies and adds, so vector and scalar rounding agree at decision boundaries. + Vector128 p0 = Vector128.Create(ratePoints[0], distortionPoints[0]); + Vector128 p1 = Vector128.Create(ratePoints[1], distortionPoints[1]); + Vector128 p2 = Vector128.Create(ratePoints[2], distortionPoints[2]); + Vector128 p3 = Vector128.Create(ratePoints[3], distortionPoints[3]); + Vector128 x = Vector128.Create(fraction); + Vector128 cubic = (Vector128.Create(3.0) * (p1 - p2)) + p3 - p0; + Vector128 quadratic = (Vector128.Create(2.0) * p0) - (Vector128.Create(5.0) * p1) + (Vector128.Create(4.0) * p2) - p3; + Vector128 result = p1 + (Vector128.Create(0.5) * x * (p2 - p0 + (x * (quadratic + (x * cubic))))); + + rateEstimate = result.GetElement(0); + distortionEstimate = result.GetElement(1); + } + else + { + rateEstimate = InterpolateModelCurve(ratePoints, fraction); + distortionEstimate = InterpolateModelCurve(distortionPoints, fraction); + } + + rate = (int)(Math.Max(0, rateEstimate * sampleCount) + 0.5); + distortion = (long)(Math.Max(0, (distortionEstimate * normalizedError) * sampleCount) + 0.5); + long skipDistortion = squaredError << DistortionScaleShift; + + // A modeled coded residual is useful only if it beats leaving the prediction unchanged. Preserve the + // reference model's zero-rate rule instead of returning an artificially low distortion for a skipped block. + if (rate == 0 || GetCost(rateMultiplier, rate, distortion) >= GetCost(rateMultiplier, 0, skipDistortion)) + { + rate = 0; + distortion = skipDistortion; + } + } + + /// + /// Evaluates one fitted curve's cubic segment without fusing arithmetic operations. + /// + private static double InterpolateModelCurve(ReadOnlySpan points, double fraction) + { + double cubic = (3.0 * (points[1] - points[2])) + points[3] - points[0]; + double quadratic = (2.0 * points[0]) - (5.0 * points[1]) + (4.0 * points[2]) - points[3]; + return points[1] + (0.5 * fraction * (points[2] - points[0] + (fraction * (quadratic + (fraction * cubic))))); + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Entropy/Av1RateDistortionStatistics.cs b/src/ImageSharp/Formats/Heif/Av1/Entropy/Av1RateDistortionStatistics.cs new file mode 100644 index 0000000000..0ec646f05d --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Entropy/Av1RateDistortionStatistics.cs @@ -0,0 +1,62 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Entropy; + +/// +/// Holds the rate, distortion, and rounded cost of an encoder candidate. +/// +internal struct Av1RateDistortionStatistics +{ + /// + /// Initializes a new instance of the struct. + /// + /// The rate multiplier for the current block. + /// The estimated syntax rate in 1/512-bit units. + /// The candidate distortion. + public Av1RateDistortionStatistics(int rateMultiplier, int rate, long distortion) + { + this.Rate = rate; + this.Distortion = distortion; + this.Cost = Av1RateDistortion.GetCost(rateMultiplier, rate, distortion); + } + + /// + /// Gets the sentinel for a candidate that cannot win a cost comparison. + /// + public static Av1RateDistortionStatistics Invalid => new() + { + Rate = int.MaxValue, + Distortion = long.MaxValue, + Cost = long.MaxValue + }; + + /// + /// Gets the estimated syntax rate in 1/512-bit units. + /// + public int Rate { get; private set; } + + /// + /// Gets the candidate distortion. + /// + public long Distortion { get; private set; } + + /// + /// Gets the rounded rate-distortion cost. + /// + public long Cost { get; private set; } + + /// + /// Adds a valid candidate's rate and distortion and updates the combined cost. + /// + /// The rate multiplier for the combined candidate. + /// The valid candidate to add. + public void Add(int rateMultiplier, in Av1RateDistortionStatistics other) + { + // Round the combined rate only once. Adding the already rounded child costs can change + // partition and inter/intra decisions even when both children have the same reconstruction. + this.Rate += other.Rate; + this.Distortion += other.Distortion; + this.Cost = Av1RateDistortion.GetCost(rateMultiplier, this.Rate, this.Distortion); + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Entropy/Av1SymbolContextHelper.cs b/src/ImageSharp/Formats/Heif/Av1/Entropy/Av1SymbolContextHelper.cs new file mode 100644 index 0000000000..d7d56d3249 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Entropy/Av1SymbolContextHelper.cs @@ -0,0 +1,1454 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Entropy; + +/// +/// Derives AV1 entropy contexts and syntax mappings shared by symbol readers and writers. +/// +internal static class Av1SymbolContextHelper +{ + /// + /// The bit offset of the global-motion decision context in a packed inter-mode context. + /// + private const int GlobalMvContextOffset = 3; + + /// + /// The bit offset of the spatial reference-motion-vector context in a packed inter-mode context. + /// + private const int RefMvContextOffset = 4; + + /// + /// The low-three-bit mask containing the new-motion-vector context. + /// + private const int NewMvContextMask = (1 << GlobalMvContextOffset) - 1; + + /// + /// The mask selecting the two-value global-motion context from its single packed bit. + /// + private const int ZeroMvContextMask = (1 << (RefMvContextOffset - GlobalMvContextOffset)) - 1; + + /// + /// The high-nibble mask containing the spatial reference-motion-vector context. + /// + private const int RefMvContextMask = (1 << (8 - RefMvContextOffset)) - 1; + + /// + /// The weight at which AV1 classifies a reference-motion-vector candidate as a strong spatial match. + /// + private const int ReferenceCategoryLevel = 640; + + /// + /// The number of interpolation filters selectable by per-block switchable syntax. + /// + public const int SwitchableInterpolationFilterCount = 3; + + /// + /// The number of transform types represented by each flattened transform-set row. + /// + private const int TransformTypeCount = 16; + + /// + /// The number of AV1 transform sets. + /// + private const int TransformSetCount = 6; + + /// + /// Gets the mapping from each transform set and transform type to its coded symbol index. + /// + private static ReadOnlySpan ExtendedTransformIndices => + [ + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, // DCT only + 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, // Inter set 3 + 1, 3, 4, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, // Intra set 2 + 1, 5, 6, 4, 0, 0, 0, 0, 0, 0, 2, 3, 0, 0, 0, 0, // Intra set 1 + 3, 4, 5, 8, 6, 7, 9, 10, 11, 0, 1, 2, 0, 0, 0, 0, // Inter set 2 + 7, 8, 9, 12, 10, 11, 13, 14, 15, 0, 1, 2, 3, 4, 5, 6 // All 16, inter set 1 + ]; + + /// + /// Gets the mapping from transform-set types to their intra and inter transform-type distribution indices. + /// + private static ReadOnlySpan ExtendedTransformSetToIndex => + [ + 0, -1, 2, 1, -1, -1, + 0, 3, -1, -1, 2, 1 + ]; + + /// + /// Gets the mapping from coded transform-type symbols to transform types for each transform set. + /// + private static ReadOnlySpan ExtendedTransformTypes => + [ + + // DCT only. Unused positions retain DCT-DCT so each set occupies one fixed 16-entry row. + Av1TransformType.DctDct, Av1TransformType.DctDct, Av1TransformType.DctDct, Av1TransformType.DctDct, + Av1TransformType.DctDct, Av1TransformType.DctDct, Av1TransformType.DctDct, Av1TransformType.DctDct, + Av1TransformType.DctDct, Av1TransformType.DctDct, Av1TransformType.DctDct, Av1TransformType.DctDct, + Av1TransformType.DctDct, Av1TransformType.DctDct, Av1TransformType.DctDct, Av1TransformType.DctDct, + + // Inter set 3. + Av1TransformType.Identity, Av1TransformType.DctDct, Av1TransformType.DctDct, Av1TransformType.DctDct, + Av1TransformType.DctDct, Av1TransformType.DctDct, Av1TransformType.DctDct, Av1TransformType.DctDct, + Av1TransformType.DctDct, Av1TransformType.DctDct, Av1TransformType.DctDct, Av1TransformType.DctDct, + Av1TransformType.DctDct, Av1TransformType.DctDct, Av1TransformType.DctDct, Av1TransformType.DctDct, + + // Intra set 2. + Av1TransformType.Identity, Av1TransformType.DctDct, Av1TransformType.AdstAdst, Av1TransformType.AdstDct, + Av1TransformType.DctAdst, Av1TransformType.DctDct, Av1TransformType.DctDct, Av1TransformType.DctDct, + Av1TransformType.DctDct, Av1TransformType.DctDct, Av1TransformType.DctDct, Av1TransformType.DctDct, + Av1TransformType.DctDct, Av1TransformType.DctDct, Av1TransformType.DctDct, Av1TransformType.DctDct, + + // Intra set 1. + Av1TransformType.Identity, Av1TransformType.DctDct, Av1TransformType.VerticalDct, Av1TransformType.HorizontalDct, + Av1TransformType.AdstAdst, Av1TransformType.AdstDct, Av1TransformType.DctAdst, Av1TransformType.DctDct, + Av1TransformType.DctDct, Av1TransformType.DctDct, Av1TransformType.DctDct, Av1TransformType.DctDct, + Av1TransformType.DctDct, Av1TransformType.DctDct, Av1TransformType.DctDct, Av1TransformType.DctDct, + + // Inter set 2. + Av1TransformType.Identity, Av1TransformType.VerticalDct, Av1TransformType.HorizontalDct, Av1TransformType.DctDct, + Av1TransformType.AdstDct, Av1TransformType.DctAdst, Av1TransformType.FlipAdstDct, Av1TransformType.DctFlipAdst, + Av1TransformType.AdstAdst, Av1TransformType.FlipAdstFlipAdst, Av1TransformType.AdstFlipAdst, Av1TransformType.FlipAdstAdst, + Av1TransformType.DctDct, Av1TransformType.DctDct, Av1TransformType.DctDct, Av1TransformType.DctDct, + + // All 16, inter set 1. + Av1TransformType.Identity, Av1TransformType.VerticalDct, Av1TransformType.HorizontalDct, Av1TransformType.VerticalAdst, + Av1TransformType.HorizontalAdst, Av1TransformType.VerticalFlipAdst, Av1TransformType.HorizontalFlipAdst, Av1TransformType.DctDct, + Av1TransformType.AdstDct, Av1TransformType.DctAdst, Av1TransformType.FlipAdstDct, Av1TransformType.DctFlipAdst, + Av1TransformType.AdstAdst, Av1TransformType.FlipAdstFlipAdst, Av1TransformType.AdstFlipAdst, Av1TransformType.FlipAdstAdst + ]; + + /// + /// Gets the number of coded symbols in each transform set. + /// + private static ReadOnlySpan ExtendedTransformTypeCounts => + [ + 1, 2, 5, 7, 12, 16 + ]; + + /// + /// Gets the number of extra offset bits associated with each end-of-block token. + /// + public static ReadOnlySpan EndOfBlockOffsetBits => [0, 0, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9]; + + /// + /// Gets the first coefficient position represented by each end-of-block token. + /// + public static ReadOnlySpan EndOfBlockGroupStart => [0, 1, 2, 3, 5, 9, 17, 33, 65, 129, 257, 513]; + + /// + /// Gets the mapping from end-of-block positions below 33 directly to their token. + /// + private static ReadOnlySpan EndOfBlockToPositionSmall => + [ + 0, 1, 2, // 0-2 + 3, 3, // 3-4 + 4, 4, 4, 4, // 5-8 + 5, 5, 5, 5, 5, 5, 5, 5, // 9-16 + 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6 // 17-32 + ]; + + /// + /// Gets the mapping from groups of 32 larger end-of-block positions to their token. + /// + private static ReadOnlySpan EndOfBlockToPositionLarge => + [ + 6, // place holder + 7, // 33-64 + 8, + 8, // 65-128 + 9, + 9, + 9, + 9, // 129-256 + 10, + 10, + 10, + 10, + 10, + 10, + 10, + 10, // 257-512 + 11 // 513- + ]; + + /// + /// Gets the mapping from clipped top and left coefficient-level classes to the transform-block skip context. + /// + private static ReadOnlySpan TransformBlockSkipContexts => + [ + 1, 2, 2, 2, 3, + 2, 4, 4, 4, 5, + 2, 4, 4, 4, 5, + 2, 4, 4, 4, 5, + 3, 5, 5, 5, 6 + ]; + + /// + /// Gets the context that selects unmasked or masked compound prediction. + /// + /// The above block, or at a tile boundary. + /// The left block, or at a tile boundary. + /// The context in the inclusive range zero through five. + public static int GetCompoundGroupIndexContext(Av1BlockModeInfo? above, Av1BlockModeInfo? left) + { + int aboveContext = 0; + if (above is not null) + { + Av1BlockModeInfo aboveModeInfo = above.Value; + aboveContext = aboveModeInfo.ReferenceFrames[1] > Av1ReferenceFrameType.Intra + ? aboveModeInfo.CompoundGroupIndex ? 1 : 0 + : aboveModeInfo.ReferenceFrames[0] == Av1ReferenceFrameType.Alternate ? 3 : 0; + } + + int leftContext = 0; + if (left is not null) + { + Av1BlockModeInfo leftModeInfo = left.Value; + leftContext = leftModeInfo.ReferenceFrames[1] > Av1ReferenceFrameType.Intra + ? leftModeInfo.CompoundGroupIndex ? 1 : 0 + : leftModeInfo.ReferenceFrames[0] == Av1ReferenceFrameType.Alternate ? 3 : 0; + } + + return Math.Min(5, aboveContext + leftContext); + } + + /// + /// Gets the context that selects equal or distance-weighted compound prediction. + /// + /// The active order-hint modulo domain. + /// The current frame and retained reference order hints. + /// The compound block whose references are selected. + /// The above block, or at a tile boundary. + /// The left block, or at a tile boundary. + /// The context in the inclusive range zero through five. + public static int GetCompoundIndexContext( + ObuOrderHintInfo orderHintInfo, + ObuFrameHeader frameHeader, + Av1BlockModeInfo modeInfo, + Av1BlockModeInfo? above, + Av1BlockModeInfo? left) + { + ReadOnlySpan referenceFrameIndices = frameHeader.GetReferenceFrameIndices(); + ReadOnlySpan referenceOrderHints = frameHeader.GetReferenceOrderHints(); + int primaryCanonicalIndex = (int)modeInfo.ReferenceFrames[0] - (int)Av1ReferenceFrameType.Last; + int secondaryCanonicalIndex = (int)modeInfo.ReferenceFrames[1] - (int)Av1ReferenceFrameType.Last; + uint primaryOrderHint = referenceOrderHints[(int)referenceFrameIndices[primaryCanonicalIndex]]; + uint secondaryOrderHint = referenceOrderHints[(int)referenceFrameIndices[secondaryCanonicalIndex]]; + int forwardDistance = Math.Abs(orderHintInfo.GetRelativeDistance(secondaryOrderHint, frameHeader.OrderHint)); + int backwardDistance = Math.Abs(orderHintInfo.GetRelativeDistance(frameHeader.OrderHint, primaryOrderHint)); + + int aboveContext = 0; + if (above is not null) + { + Av1BlockModeInfo aboveModeInfo = above.Value; + aboveContext = aboveModeInfo.ReferenceFrames[1] > Av1ReferenceFrameType.Intra + ? aboveModeInfo.CompoundIndex ? 1 : 0 + : aboveModeInfo.ReferenceFrames[0] == Av1ReferenceFrameType.Alternate ? 1 : 0; + } + + int leftContext = 0; + if (left is not null) + { + Av1BlockModeInfo leftModeInfo = left.Value; + leftContext = leftModeInfo.ReferenceFrames[1] > Av1ReferenceFrameType.Intra + ? leftModeInfo.CompoundIndex ? 1 : 0 + : leftModeInfo.ReferenceFrames[0] == Av1ReferenceFrameType.Alternate ? 1 : 0; + } + + return aboveContext + leftContext + (forwardDistance == backwardDistance ? 3 : 0); + } + + /// + /// Reduces a rectangular transform size to the square context used by transform-size distributions. + /// + /// The coded transform size. + /// The square transform-size context. + public static Av1TransformSize GetTransformSizeContext(Av1TransformSize originalSize) + => (Av1TransformSize)(((int)originalSize.GetSquareSize() + (int)originalSize.GetSquareUpSize() + 1) >> 1); + + /// + /// Derives the luma transform-block skip context from the neighboring coefficient levels. + /// + /// The union of the packed coefficient contexts above the transform. + /// The union of the packed coefficient contexts to the left of the transform. + /// The transform-block skip context. + public static int GetTransformBlockSkipContext(int top, int left) + { + int topClass = Math.Min(top, 4); + int leftClass = Math.Min(left, 4); + + // AV1 groups each edge into zero, low, or high coefficient-level classes. Retaining the reference decoder's complete table + // lets the reader and writer share one compile-time mapping without an encoder-side jagged-array allocation. + return TransformBlockSkipContexts[(topClass * 5) + leftClass]; + } + + /// + /// Derives the variable-transform partition context from the current node and its adjacent transform edges. + /// + /// The transform width retained immediately above the current node. + /// The transform height retained immediately left of the current node. + /// The containing coding-block size. + /// The transform size represented by the current partition node. + /// The variable-transform partition context. + public static int GetTransformPartitionContext( + byte aboveTransformWidth, + byte leftTransformHeight, + Av1BlockSize blockSize, + Av1TransformSize transformSize) + { + if (transformSize <= Av1TransformSize.Size4x4) + { + return 0; + } + + int above = aboveTransformWidth < transformSize.GetWidth() ? 1 : 0; + int left = leftTransformHeight < transformSize.GetHeight() ? 1 : 0; + int maximumDimension = Math.Max(blockSize.GetWidth(), blockSize.GetHeight()); + Av1TransformSize maximumSquareTransform = maximumDimension switch + { + >= 64 => Av1TransformSize.Size64x64, + >= 32 => Av1TransformSize.Size32x32, + >= 16 => Av1TransformSize.Size16x16, + _ => Av1TransformSize.Size8x8 + }; + + int category = (transformSize.GetSquareUpSize() != maximumSquareTransform && + maximumSquareTransform > Av1TransformSize.Size8x8 ? 1 : 0) + + ((((int)Av1TransformSize.SquareSizes - 1) - (int)maximumSquareTransform) * 2); + + return (category * 3) + above + left; + } + + /// + /// Reconstructs an end-of-block coefficient position from its token and extra offset. + /// + /// The decoded end-of-block token. + /// The decoded offset within the token group. + /// The one-based end-of-block coefficient position. + public static int RecordEndOfBlockPosition(int endOfBlockPoint, int endOfBlockExtra) + { + int endOfBlock = EndOfBlockGroupStart[endOfBlockPoint]; + if (endOfBlock > 2) + { + endOfBlock += endOfBlockExtra; + } + + return endOfBlock; + } + + /// + /// Derives the lower-level context for the final nonzero coefficient from an index expressed as a two-dimensional coordinate. + /// + /// The padded coefficient-level buffer. + /// The coordinate whose row-major index identifies the coefficient's scan position. + /// The end-of-block lower-level context. + public static int GetLowerLevelContextEndOfBlock(Av1LevelBuffer levels, Point position) + => GetLowerLevelContextEndOfBlock(levels, position.X + (position.Y * levels.Size.Width)); + + /// + /// Derives the lower-level context for the final nonzero coefficient from its scan-order index. + /// + /// The padded coefficient-level buffer. + /// The zero-based coefficient index in scan order. + /// The end-of-block lower-level context. + public static int GetLowerLevelContextEndOfBlock(Av1LevelBuffer levels, int scanIndex) + { + if (scanIndex == 0) + { + return 0; + } + + int total = levels.Size.Height * levels.Size.Width; + if (scanIndex <= total >> 3) + { + return 1; + } + + if (scanIndex <= total >> 2) + { + return 2; + } + + return 3; + } + + /// + /// Derives a two-dimensional lower-level context from five forward coefficient neighbors. + /// + /// The padded coefficient-level buffer. + /// The coefficient position in raster order. + /// The transform size selecting the positional context offset. + /// The lower-level context. + public static int GetLowerLevelsContext2d(Av1LevelBuffer levelBuffer, Point position, Av1TransformSize transformSize) + { + DebugGuard.MustBeGreaterThan(position.X + position.Y, 0, nameof(position)); + int mag; + Span row0 = levelBuffer.GetRow(position.Y)[position.X..]; + Span row1 = levelBuffer.GetRow(position.Y + 1)[position.X..]; + Span row2 = levelBuffer.GetRow(position.Y + 2)[position.X..]; + mag = Math.Min((int)row0[1], 3); // { 0, 1 } + mag += Math.Min((int)row1[0], 3); // { 1, 0 } + mag += Math.Min((int)row1[1], 3); // { 1, 1 } + mag += Math.Min((int)row0[2], 3); // { 0, 2 } + mag += Math.Min((int)row2[0], 3); // { 2, 0 } + + int ctx = Math.Min((mag + 1) >> 1, 4); + return ctx + Av1NzMap.GetNzMapContext(transformSize, position); + } + + /// + /// Section 8.3.2 in the spec, under coeff_br. Optimized for end of block based + /// on the fact that {0, 1}, {1, 0}, {1, 1}, {0, 2} and {2, 0} will all be 0 in + /// the end of block case. + /// + /// The final nonzero coefficient position. + /// The transform direction class. + /// The base-range context. + public static int GetBaseRangeContextEndOfBlock(Point pos, Av1TransformClass transformClass) + { + if (pos.X == 0 && pos.Y == 0) + { + return 0; + } + + if ((transformClass == Av1TransformClass.Class2D && pos.Y < 2 && pos.X < 2) || + (transformClass == Av1TransformClass.ClassHorizontal && pos.X == 0) || + (transformClass == Av1TransformClass.ClassVertical && pos.Y == 0)) + { + return 7; + } + + return 14; + } + + /// + /// Derives a base-range context from the transform-class-specific forward neighbors. + /// + /// Spec section 8.2.3, under 'coeff_br'. + /// The padded coefficient-level buffer. + /// The coefficient position in raster order. + /// The transform direction class. + /// The base-range context. + public static int GetBaseRangeContext(Av1LevelBuffer levels, Point position, Av1TransformClass transformClass) + { + Span row0 = levels.GetRow(position.Y); + Span row1 = levels.GetRow(position.Y + 1); + int mag = row0[position.X + 1]; + mag += row1[position.X]; + switch (transformClass) + { + case Av1TransformClass.Class2D: + mag += row1[position.X + 1]; + mag = Math.Min((mag + 1) >> 1, 6); + if ((position.X + position.Y) == 0) + { + return mag; + } + + if (position.Y < 2 && position.X < 2) + { + return mag + 7; + } + + break; + case Av1TransformClass.ClassHorizontal: + mag += row0[position.X + 2]; + mag = Math.Min((mag + 1) >> 1, 6); + if ((position.X + position.Y) == 0) + { + return mag; + } + + if (position.X == 0) + { + return mag + 7; + } + + break; + case Av1TransformClass.ClassVertical: + mag += levels.GetRow(position.Y + 2)[position.X]; + mag = Math.Min((mag + 1) >> 1, 6); + if ((position.X + position.Y) == 0) + { + return mag; + } + + if (position.Y == 0) + { + return mag + 7; + } + + break; + default: + break; + } + + return mag + 14; + } + + /// + /// Derives the two-dimensional base-range context from right, below, and below-right levels. + /// + /// The padded coefficient-level buffer. + /// The coefficient position in raster order. + /// The two-dimensional base-range context. + public static int GetBaseRangeContext2d(Av1LevelBuffer levels, Point position) + { + DebugGuard.MustBeGreaterThan(position.X + position.Y, 0, nameof(position)); + Span row0 = levels.GetRow(position.Y); + Span row1 = levels.GetRow(position.Y + 1); + + // The final magnitude context is clipped to six, so clipping every source level to the AV1 base-range limit + // first cannot change the result. + int mag = + row0[position.X + 1] + // {0, 1} + row1[position.X] + // {1, 0} + row1[position.X + 1]; // {1, 1} + mag = Math.Min((mag + 1) >> 1, 6); + if ((position.Y | position.X) < 2) + { + return mag + 7; + } + + return mag + 14; + } + + /// + /// Derives a lower-level context from the transform-class-specific nonzero-map magnitude. + /// + /// The padded coefficient-level buffer. + /// The coefficient position in raster order. + /// The coded transform size. + /// The transform direction class. + /// The lower-level coefficient context. + public static int GetLowerLevelsContext(Av1LevelBuffer levels, Point position, Av1TransformSize transformSize, Av1TransformClass transformClass) + { + int stats = Av1NzMap.GetNzMagnitude(levels, position, transformClass); + return Av1NzMap.GetNzMapContextFromStats(stats, position, transformSize, transformClass); + } + + /// + /// Selects the intra transform set permitted for a transform size and reduced-set flag. + /// + /// The coded transform size. + /// Indicates whether the frame restricts transform choices. + /// The permitted transform set. + public static Av1TransformSetType GetExtendedTransformSetType(Av1TransformSize transformSize, bool useReducedSet) + => GetExtendedTransformSetType(transformSize, false, useReducedSet); + + /// + /// Selects the transform set permitted for a transform size, prediction class, and reduced-set flag. + /// + /// The coded transform size. + /// Indicates whether the block uses inter prediction. + /// Indicates whether the frame restricts transform choices. + /// The permitted transform set. + public static Av1TransformSetType GetExtendedTransformSetType(Av1TransformSize transformSize, bool isInter, bool useReducedSet) + { + Av1TransformSize squareUpSize = transformSize.GetSquareUpSize(); + + if (squareUpSize > Av1TransformSize.Size32x32) + { + return Av1TransformSetType.DctOnly; + } + + if (squareUpSize == Av1TransformSize.Size32x32) + { + return isInter ? Av1TransformSetType.InterSet3 : Av1TransformSetType.DctOnly; + } + + if (useReducedSet) + { + return isInter ? Av1TransformSetType.InterSet3 : Av1TransformSetType.IntraSet2; + } + + Av1TransformSize squareSize = transformSize.GetSquareSize(); + if (isInter) + { + return squareSize == Av1TransformSize.Size16x16 ? Av1TransformSetType.InterSet2 : Av1TransformSetType.InterSet1; + } + + return squareSize == Av1TransformSize.Size16x16 + ? Av1TransformSetType.IntraSet2 + : Av1TransformSetType.IntraSet1; + } + + /// + /// Gets the mode-derived default intra transform after applying the active transform-set restriction. + /// + /// The intra prediction mode. + /// The coded transform size. + /// Indicates whether the frame restricts transform choices. + /// The permitted default transform for the mode. + public static Av1TransformType GetDefaultIntraTransformType( + Av1PredictionMode mode, + Av1TransformSize transformSize, + bool useReducedSet) + { + Av1TransformType transformType = mode.ToTransformType(); + Av1TransformSetType transformSetType = GetExtendedTransformSetType(transformSize, useReducedSet); + + // A mode-derived transform falls back to DCT-DCT when its transform set omits that type. + return transformType.IsExtendedSetUsed(transformSetType) ? transformType : Av1TransformType.DctDct; + } + + /// + /// Derives the nonzero-map context for one coefficient preceding the final nonzero coefficient. + /// + /// The padded coefficient-level buffer. + /// The coefficient position in raster order. + /// The coded transform size. + /// The transform direction class. + /// The nonzero-map context. + public static sbyte GetNzMapContext( + Av1LevelBuffer levels, + Point position, + Av1TransformSize transformSize, + Av1TransformClass transformClass) + { + int stats = Av1NzMap.GetNzMagnitude(levels, position, transformClass); + return (sbyte)Av1NzMap.GetNzMapContextFromStats(stats, position, transformSize, transformClass); + } + + /// + /// Populates nonzero-map contexts for every coefficient preceding the end-of-block position. + /// + /// The padded coefficient-level buffer. + /// The coefficient scan order. + /// The one-based end-of-block position. + /// The coded transform size. + /// The transform direction class. + /// The raster-indexed destination contexts. + public static void GetNzMapContexts( + Av1LevelBuffer levels, + ReadOnlySpan scan, + ushort eob, + Av1TransformSize transformSize, + Av1TransformClass transformClass, + Span coefficientContexts) + { + for (int i = 0; i < eob; ++i) + { + int pos = scan[i]; + Point position = levels.GetPosition(pos); + + // The final coefficient context is based on its scan position, while all preceding contexts use the + // coefficient's raster position and already-decoded forward neighbors. + coefficientContexts[pos] = i == eob - 1 + ? (sbyte)GetLowerLevelContextEndOfBlock(levels, i) + : GetNzMapContext(levels, position, transformSize, transformClass); + } + } + + /// + /// Gets the coded symbol index for a transform type in a transform set. + /// + /// The transform set. + /// The transform type. + /// The coded symbol index. + public static int GetExtendedTransformIndex(Av1TransformSetType setType, Av1TransformType transformType) + => ExtendedTransformIndices[((int)setType * TransformTypeCount) + (int)transformType]; + + /// + /// Gets the transform type represented by a coded symbol in a transform set. + /// + /// The transform set. + /// The coded symbol index. + /// The represented transform type. + public static Av1TransformType GetExtendedTransformType(Av1TransformSetType setType, int symbol) + => ExtendedTransformTypes[((int)setType * TransformTypeCount) + symbol]; + + /// + /// Gets the number of transform types in a transform set. + /// + /// The transform set. + /// The number of permitted transform types. + public static int GetExtendedTransformTypeCount(Av1TransformSetType setType) => ExtendedTransformTypeCounts[(int)setType]; + + /// + /// Gets the entropy-distribution index for an intra transform set. + /// + /// The transform set. + /// The distribution index, or -1 for an inter-only set. + public static int GetExtendedTransformSet(Av1TransformSetType setType) + => GetExtendedTransformSet(setType, false); + + /// + /// Gets the entropy-distribution index for a transform set and prediction class. + /// + /// The transform set. + /// Indicates whether the block uses inter prediction. + /// The distribution index, or -1 when the set is unavailable for the prediction class. + public static int GetExtendedTransformSet(Av1TransformSetType setType, bool isInter) + => ExtendedTransformSetToIndex[((isInter ? 1 : 0) * TransformSetCount) + (int)setType]; + + /// + /// Packs the sign of the DC coefficient into a cumulative-level context value. + /// + /// The cumulative-level context to update. + /// The signed DC coefficient. + public static void SetDcSign(ref int culLevel, int dcValue) + { + if (dcValue < 0) + { + culLevel |= 1 << Av1Constants.CoefficientContextBitCount; + } + else if (dcValue > 0) + { + culLevel += 2 << Av1Constants.CoefficientContextBitCount; + } + } + + /// + /// Packs the magnitude class and DC sign retained by neighboring transform blocks. + /// + /// The raster-ordered quantized coefficients. + /// The transform dimensions. + /// The transform type selecting scan order. + /// The one-based final nonzero scan position. + /// The packed coefficient context, or zero for an empty transform. + public static byte GetCoefficientContext( + ReadOnlySpan coefficients, + Av1TransformSize transformSize, + Av1TransformType transformType, + ushort endOfBlock) + { + if (endOfBlock == 0) + { + return 0; + } + + ReadOnlySpan scan = Av1ScanOrderConstants.GetScanOrder(transformSize, transformType).Scan; + int culLevel = 0; + for (int scanIndex = 0; scanIndex < endOfBlock; scanIndex++) + { + culLevel += Math.Abs(coefficients[scan[scanIndex]]); + } + + culLevel = Math.Min(Av1Constants.CoefficientContextMask, culLevel); + SetDcSign(ref culLevel, coefficients[0]); + return (byte)culLevel; + } + + /// + /// Converts a one-based end-of-block position to its token and group offset. + /// + /// The one-based end-of-block position. + /// Receives the offset within the selected token group. + /// The end-of-block token. + public static short GetEndOfBlockPosition(ushort endOfBlock, out int extra) + { + short t; + if (endOfBlock < 33) + { + t = EndOfBlockToPositionSmall[endOfBlock]; + } + else + { + int e = Math.Min((endOfBlock - 1) >> 5, 16); + t = EndOfBlockToPositionLarge[e]; + } + + extra = endOfBlock - EndOfBlockGroupStart[t]; + return t; + } + + /// + /// Gets the decoded segment identifier at one spatial-neighbor position. + /// + /// The row-major decoded segment map. + /// The mode-info row. + /// The mode-info column. + /// The segment identifier stored at the requested position. + public static int GetSegmentId(int[][] segmentIds, int rowIndex, int columnIndex) + => segmentIds[rowIndex][columnIndex]; + + /// + /// Gets the intra/inter prediction context from the immediately above and left blocks. + /// + /// The above block, or at a tile boundary. + /// The left block, or at a tile boundary. + /// The context in the inclusive range zero through three. + public static int GetIntraInterContext(Av1BlockModeInfo? above, Av1BlockModeInfo? left) + { + if (above is not null && left is not null) + { + Av1BlockModeInfo aboveModeInfo = above.Value; + Av1BlockModeInfo leftModeInfo = left.Value; + bool aboveIsIntra = aboveModeInfo.ReferenceFrames[0] <= Av1ReferenceFrameType.Intra; + bool leftIsIntra = leftModeInfo.ReferenceFrames[0] <= Av1ReferenceFrameType.Intra; + + // AV1 reserves context three for two intra neighbors, context one for a mixed pair, and context zero for + // two inter neighbors. These values directly index intra_inter_cdf and are not probability ranks. + if (aboveIsIntra && leftIsIntra) + { + return 3; + } + + return aboveIsIntra || leftIsIntra ? 1 : 0; + } + + // A single intra neighbor uses context two. A single inter neighbor and a block with no neighbors both use + // context zero, matching the unavailable-neighbor behavior in the reference decoder's av1_get_intra_inter_context. + if (above is not null) + { + return above.Value.ReferenceFrames[0] <= Av1ReferenceFrameType.Intra ? 2 : 0; + } + + if (left is not null) + { + return left.Value.ReferenceFrames[0] <= Av1ReferenceFrameType.Intra ? 2 : 0; + } + + return 0; + } + + /// + /// Gets the block reference-mode context from the immediately above and left blocks. + /// + /// The above block, or at a tile boundary. + /// The left block, or at a tile boundary. + /// The context in the inclusive range zero through four. + public static int GetReferenceModeContext(Av1BlockModeInfo? above, Av1BlockModeInfo? left) + { + // The reference decoder first classifies whether each neighbor uses a second inter reference. Single neighbors then contribute + // their forward/backward direction, while intra neighbors take the same branch as a non-forward reference. + if (above is not null && left is not null) + { + Av1BlockModeInfo aboveModeInfo = above.Value; + Av1BlockModeInfo leftModeInfo = left.Value; + bool aboveIsCompound = aboveModeInfo.ReferenceFrames[1] > Av1ReferenceFrameType.Intra; + bool leftIsCompound = leftModeInfo.ReferenceFrames[1] > Av1ReferenceFrameType.Intra; + + if (!aboveIsCompound && !leftIsCompound) + { + bool aboveIsBackward = aboveModeInfo.ReferenceFrames[0] >= Av1ReferenceFrameType.Backward; + bool leftIsBackward = leftModeInfo.ReferenceFrames[0] >= Av1ReferenceFrameType.Backward; + + return aboveIsBackward == leftIsBackward ? 0 : 1; + } + + if (!aboveIsCompound) + { + bool aboveIsBackward = aboveModeInfo.ReferenceFrames[0] >= Av1ReferenceFrameType.Backward; + bool aboveIsIntra = aboveModeInfo.ReferenceFrames[0] <= Av1ReferenceFrameType.Intra; + + return 2 + (aboveIsBackward || aboveIsIntra ? 1 : 0); + } + + if (!leftIsCompound) + { + bool leftIsBackward = leftModeInfo.ReferenceFrames[0] >= Av1ReferenceFrameType.Backward; + bool leftIsIntra = leftModeInfo.ReferenceFrames[0] <= Av1ReferenceFrameType.Intra; + + return 2 + (leftIsBackward || leftIsIntra ? 1 : 0); + } + + return 4; + } + + Av1BlockModeInfo? neighbor = above ?? left; + + if (neighbor is not null) + { + Av1BlockModeInfo neighborModeInfo = neighbor.Value; + bool isCompound = neighborModeInfo.ReferenceFrames[1] > Av1ReferenceFrameType.Intra; + + if (isCompound) + { + return 3; + } + + return neighborModeInfo.ReferenceFrames[0] >= Av1ReferenceFrameType.Backward ? 1 : 0; + } + + // With no spatial votes, AV1 uses the neutral single-versus-compound context rather than context zero. + return 1; + } + + /// + /// Gets the context that selects a unidirectional or bidirectional compound reference pair. + /// + /// The above block, or at a tile boundary. + /// The left block, or at a tile boundary. + /// The context in the inclusive range zero through four. + public static int GetCompoundReferenceTypeContext(Av1BlockModeInfo? above, Av1BlockModeInfo? left) + { + if (above is not null && left is not null) + { + Av1BlockModeInfo aboveModeInfo = above.Value; + Av1BlockModeInfo leftModeInfo = left.Value; + bool aboveIntra = !IsInterBlock(aboveModeInfo); + bool leftIntra = !IsInterBlock(leftModeInfo); + if (aboveIntra && leftIntra) + { + return 2; + } + + if (aboveIntra || leftIntra) + { + Av1BlockModeInfo inter = aboveIntra ? leftModeInfo : aboveModeInfo; + return HasCompoundReference(inter) ? 1 + (2 * (HasUnidirectionalCompoundReferences(inter) ? 1 : 0)) : 2; + } + + bool aboveSingle = !HasCompoundReference(aboveModeInfo); + bool leftSingle = !HasCompoundReference(leftModeInfo); + Av1ReferenceFrameType abovePrimary = aboveModeInfo.ReferenceFrames[0]; + Av1ReferenceFrameType leftPrimary = leftModeInfo.ReferenceFrames[0]; + if (aboveSingle && leftSingle) + { + return 1 + (2 * (IsBackwardReference(abovePrimary) == IsBackwardReference(leftPrimary) ? 1 : 0)); + } + + if (aboveSingle || leftSingle) + { + Av1BlockModeInfo compound = aboveSingle ? leftModeInfo : aboveModeInfo; + if (!HasUnidirectionalCompoundReferences(compound)) + { + return 1; + } + + return 3 + (IsBackwardReference(abovePrimary) == IsBackwardReference(leftPrimary) ? 1 : 0); + } + + bool aboveUnidirectional = HasUnidirectionalCompoundReferences(aboveModeInfo); + bool leftUnidirectional = HasUnidirectionalCompoundReferences(leftModeInfo); + if (!aboveUnidirectional && !leftUnidirectional) + { + return 0; + } + + if (!aboveUnidirectional || !leftUnidirectional) + { + return 2; + } + + return 3 + ((abovePrimary == Av1ReferenceFrameType.Backward) == (leftPrimary == Av1ReferenceFrameType.Backward) ? 1 : 0); + } + + Av1BlockModeInfo? edge = above ?? left; + if (edge is null) + { + return 2; + } + + Av1BlockModeInfo edgeModeInfo = edge.Value; + if (!IsInterBlock(edgeModeInfo) || !HasCompoundReference(edgeModeInfo)) + { + return 2; + } + + return HasUnidirectionalCompoundReferences(edgeModeInfo) ? 4 : 0; + } + + /// + /// Gets the switchable interpolation-filter context for one prediction direction. + /// + /// The current inter block. + /// The above block, or at a tile boundary. + /// The left block, or at a tile boundary. + /// Zero for the vertical filter or one for the horizontal filter. + /// The context in the inclusive range zero through fifteen. + public static int GetSwitchableInterpolationContext( + Av1BlockModeInfo modeInfo, + Av1BlockModeInfo? above, + Av1BlockModeInfo? left, + int direction) + { + ReadOnlySpan referenceFrames = modeInfo.ReferenceFrames; + Av1ReferenceFrameType primaryReference = referenceFrames[0]; + bool isCompound = referenceFrames[1] > Av1ReferenceFrameType.Intra; + + int leftFilter = GetReferenceInterpolationFilterContext(left, primaryReference, direction); + int aboveFilter = GetReferenceInterpolationFilterContext(above, primaryReference, direction); + + return GetSwitchableInterpolationContext(aboveFilter, leftFilter, isCompound, direction); + } + + /// + /// Gets the switchable interpolation-filter context from the encoder's packed single-reference neighbors. + /// + /// The current inter block. + /// The current block's available spatial neighbors. + /// Zero for the vertical filter or one for the horizontal filter. + /// The single-reference context for the selected direction. + public static int GetSwitchableInterpolationContext( + Av1EncoderBlockModeInfo modeInfo, + Av1MacroBlockD macroBlock, + int direction) + { + int aboveFilter = SwitchableInterpolationFilterCount; + int leftFilter = SwitchableInterpolationFilterCount; + if (macroBlock.IsUpAvailable) + { + Av1EncoderBlockModeInfo above = macroBlock.GetRelativeModeInfo(-macroBlock.ModeInfoStride).Block; + if (above.ReferenceFrame == modeInfo.ReferenceFrame) + { + aboveFilter = (int)(direction == 0 ? above.VerticalInterpolationFilter : above.HorizontalInterpolationFilter); + } + } + + if (macroBlock.IsLeftAvailable) + { + Av1EncoderBlockModeInfo left = macroBlock.GetRelativeModeInfo(-1).Block; + if (left.ReferenceFrame == modeInfo.ReferenceFrame) + { + leftFilter = (int)(direction == 0 ? left.VerticalInterpolationFilter : left.HorizontalInterpolationFilter); + } + } + + return GetSwitchableInterpolationContext(aboveFilter, leftFilter, isCompound: false, direction); + } + + /// + /// Combines the two neighboring filter states into the shared encoder and decoder context layout. + /// + private static int GetSwitchableInterpolationContext(int aboveFilter, int leftFilter, bool isCompound, int direction) + { + const int filterContextCount = SwitchableInterpolationFilterCount + 1; + const int horizontalContextOffset = filterContextCount * 2; + + // The sixteen rows are single vertical, compound vertical, single horizontal, then compound horizontal, + // with four neighbor states in each group. Both storage representations must use this same mapping. + int context = (isCompound ? filterContextCount : 0) + (direction * horizontalContextOffset); + + if (leftFilter == aboveFilter) + { + return context + leftFilter; + } + + // The fourth neighbor state means no matching primary reference or disagreement between contributing + // neighbors. It is not the Bilinear filter, which is absent from the switchable alphabet. + if (leftFilter == SwitchableInterpolationFilterCount) + { + return context + aboveFilter; + } + + if (aboveFilter == SwitchableInterpolationFilterCount) + { + return context + leftFilter; + } + + return context + SwitchableInterpolationFilterCount; + } + + /// + /// Gets the new-motion-vector decision context from a packed single-reference inter-mode context. + /// + /// The packed mode context produced by reference-motion-vector candidate analysis. + /// For a valid packed mode context, the context in the inclusive range zero through five. + public static int GetNewMvContext(int modeContext) => modeContext & NewMvContextMask; + + /// + /// Gets the global-motion decision context from a packed single-reference inter-mode context. + /// + /// The packed mode context produced by reference-motion-vector candidate analysis. + /// The context in the inclusive range zero through one. + public static int GetZeroMvContext(int modeContext) => (modeContext >> GlobalMvContextOffset) & ZeroMvContextMask; + + /// + /// Gets the spatial reference-motion-vector decision context from a packed single-reference inter-mode context. + /// + /// The packed mode context produced by reference-motion-vector candidate analysis. + /// For a valid packed mode context, the context in the inclusive range zero through five. + public static int GetRefMvContext(int modeContext) => (modeContext >> RefMvContextOffset) & RefMvContextMask; + + /// + /// Maps the packed paired-reference candidate context to one of the eight compound inter-mode distributions. + /// + /// The packed mode context produced by paired reference-motion-vector analysis. + /// The compound inter-mode context in the inclusive range zero through seven. + public static int GetCompoundModeContext(int modeContext) + { + ReadOnlySpan contextMap = + [ + 0, 1, 1, 1, 1, + 1, 2, 3, 4, 4, + 4, 4, 5, 6, 7, + ]; + + int newMvContext = Math.Min(GetNewMvContext(modeContext), 4); + int referenceContextGroup = GetRefMvContext(modeContext) >> 1; + return contextMap[(referenceContextGroup * 5) + newMvContext]; + } + + /// + /// Gets the dynamic reference-list context for two adjacent motion-vector candidates. + /// + /// The candidate weights in dynamic reference-list order. + /// The zero-based index of the first candidate in the pair. + /// The context in the inclusive range zero through two. + public static int GetDrlContext(ReadOnlySpan referenceWeights, int referenceIndex) + { + int currentWeight = referenceWeights[referenceIndex]; + int nextWeight = referenceWeights[referenceIndex + 1]; + + // Candidate weights at or above the reference-category threshold carry a strong spatial match. The four + // normative pairings use context zero for strong/strong and weak/strong, one for strong/weak, and two for weak/weak. + if (currentWeight >= ReferenceCategoryLevel && nextWeight >= ReferenceCategoryLevel) + { + return 0; + } + + if (currentWeight >= ReferenceCategoryLevel && nextWeight < ReferenceCategoryLevel) + { + return 1; + } + + return currentWeight < ReferenceCategoryLevel && nextWeight < ReferenceCategoryLevel ? 2 : 0; + } + + /// + /// Counts the reference-frame labels used by the immediately above and left inter blocks. + /// + /// The above block, or at a tile boundary. + /// The left block, or at a tile boundary. + /// The eight-entry reference-count destination indexed by . + public static void CollectNeighborReferenceCounts(Av1BlockModeInfo? above, Av1BlockModeInfo? left, Span referenceCounts) + { + // The caller reuses fixed inline storage across blocks. Clearing all eight entries matches the reference decoder's + // av1_collect_neighbors_ref_counts and prevents an unavailable neighbor from retaining an earlier block's vote. + referenceCounts.Clear(); + + if (above is not null) + { + AddNeighborReferenceCounts(above.Value, referenceCounts); + } + + if (left is not null) + { + AddNeighborReferenceCounts(left.Value, referenceCounts); + } + } + + /// + /// Gets the context that selects a backward instead of forward single reference. + /// + /// The neighboring reference counts indexed by . + /// The context in the inclusive range zero through two. + public static int GetSingleReferenceBackwardContext(ReadOnlySpan referenceCounts) + { + int forwardCount = referenceCounts[(int)Av1ReferenceFrameType.Last] + + referenceCounts[(int)Av1ReferenceFrameType.Last2] + + referenceCounts[(int)Av1ReferenceFrameType.Last3] + + referenceCounts[(int)Av1ReferenceFrameType.Golden]; + + int backwardCount = referenceCounts[(int)Av1ReferenceFrameType.Backward] + + referenceCounts[(int)Av1ReferenceFrameType.Alternate2] + + referenceCounts[(int)Av1ReferenceFrameType.Alternate]; + + return GetBinaryReferenceContext(forwardCount, backwardCount); + } + + /// + /// Gets the context that selects Alternate instead of Backward or Alternate2. + /// + /// The neighboring reference counts indexed by . + /// The context in the inclusive range zero through two. + public static int GetSingleReferenceAlternateContext(ReadOnlySpan referenceCounts) + { + int backwardOrAlternate2Count = referenceCounts[(int)Av1ReferenceFrameType.Backward] + + referenceCounts[(int)Av1ReferenceFrameType.Alternate2]; + + int alternateCount = referenceCounts[(int)Av1ReferenceFrameType.Alternate]; + + return GetBinaryReferenceContext(backwardOrAlternate2Count, alternateCount); + } + + /// + /// Gets the context that selects Last3 or Golden instead of Last or Last2. + /// + /// The neighboring reference counts indexed by . + /// The context in the inclusive range zero through two. + public static int GetSingleReferenceLast3OrGoldenContext(ReadOnlySpan referenceCounts) + { + int lastOrLast2Count = referenceCounts[(int)Av1ReferenceFrameType.Last] + + referenceCounts[(int)Av1ReferenceFrameType.Last2]; + + int last3OrGoldenCount = referenceCounts[(int)Av1ReferenceFrameType.Last3] + + referenceCounts[(int)Av1ReferenceFrameType.Golden]; + + return GetBinaryReferenceContext(lastOrLast2Count, last3OrGoldenCount); + } + + /// + /// Gets the context that selects Last2 instead of Last. + /// + /// The neighboring reference counts indexed by . + /// The context in the inclusive range zero through two. + public static int GetSingleReferenceLast2Context(ReadOnlySpan referenceCounts) + { + int lastCount = referenceCounts[(int)Av1ReferenceFrameType.Last]; + int last2Count = referenceCounts[(int)Av1ReferenceFrameType.Last2]; + + return GetBinaryReferenceContext(lastCount, last2Count); + } + + /// + /// Gets the context that selects Golden instead of Last3. + /// + /// The neighboring reference counts indexed by . + /// The context in the inclusive range zero through two. + public static int GetSingleReferenceGoldenContext(ReadOnlySpan referenceCounts) + { + int last3Count = referenceCounts[(int)Av1ReferenceFrameType.Last3]; + int goldenCount = referenceCounts[(int)Av1ReferenceFrameType.Golden]; + + return GetBinaryReferenceContext(last3Count, goldenCount); + } + + /// + /// Gets the context that selects Alternate2 instead of Backward. + /// + /// The neighboring reference counts indexed by . + /// The context in the inclusive range zero through two. + public static int GetSingleReferenceAlternate2Context(ReadOnlySpan referenceCounts) + { + int backwardCount = referenceCounts[(int)Av1ReferenceFrameType.Backward]; + int alternate2Count = referenceCounts[(int)Av1ReferenceFrameType.Alternate2]; + + return GetBinaryReferenceContext(backwardCount, alternate2Count); + } + + /// + /// Gets the first unidirectional compound-reference decision context. + /// + public static int GetUnidirectionalCompoundBackwardContext(ReadOnlySpan referenceCounts) + => GetSingleReferenceBackwardContext(referenceCounts); + + /// + /// Gets the context that selects Last3 or Golden instead of Last2 for a forward unidirectional pair. + /// + public static int GetUnidirectionalCompoundLast3OrGoldenContext(ReadOnlySpan referenceCounts) + { + int last2Count = referenceCounts[(int)Av1ReferenceFrameType.Last2]; + int last3OrGoldenCount = referenceCounts[(int)Av1ReferenceFrameType.Last3] + + referenceCounts[(int)Av1ReferenceFrameType.Golden]; + + return GetBinaryReferenceContext(last2Count, last3OrGoldenCount); + } + + /// + /// Gets the context that selects Golden instead of Last3 for a forward unidirectional pair. + /// + public static int GetUnidirectionalCompoundGoldenContext(ReadOnlySpan referenceCounts) + => GetSingleReferenceGoldenContext(referenceCounts); + + /// + /// Gets the context that selects Last3 or Golden instead of Last or Last2 for a bidirectional pair. + /// + public static int GetCompoundForwardLast3OrGoldenContext(ReadOnlySpan referenceCounts) + => GetSingleReferenceLast3OrGoldenContext(referenceCounts); + + /// + /// Gets the context that selects Last2 instead of Last for a bidirectional pair. + /// + public static int GetCompoundForwardLast2Context(ReadOnlySpan referenceCounts) + => GetSingleReferenceLast2Context(referenceCounts); + + /// + /// Gets the context that selects Golden instead of Last3 for a bidirectional pair. + /// + public static int GetCompoundForwardGoldenContext(ReadOnlySpan referenceCounts) + => GetSingleReferenceGoldenContext(referenceCounts); + + /// + /// Gets the context that selects Alternate instead of Backward or Alternate2 for a bidirectional pair. + /// + public static int GetCompoundBackwardAlternateContext(ReadOnlySpan referenceCounts) + => GetSingleReferenceAlternateContext(referenceCounts); + + /// + /// Gets the context that selects Alternate2 instead of Backward for a bidirectional pair. + /// + public static int GetCompoundBackwardAlternate2Context(ReadOnlySpan referenceCounts) + => GetSingleReferenceAlternate2Context(referenceCounts); + + /// + /// Gets the temporal segment-prediction context from the immediately above and left blocks. + /// + /// The above block, or at a tile boundary. + /// The left block, or at a tile boundary. + /// The context in the inclusive range zero through two. + public static int GetSegmentIdPredictedContext(Av1BlockModeInfo? aboveModeInfo, Av1BlockModeInfo? leftModeInfo) + { + int abovePredicted = aboveModeInfo is not null && aboveModeInfo.Value.SegmentIdPredicted ? 1 : 0; + int leftPredicted = leftModeInfo is not null && leftModeInfo.Value.SegmentIdPredicted ? 1 : 0; + return abovePredicted + leftPredicted; + } + + /// + /// Gets the minimum encoded segment identifier across a block's clipped mode-info coverage. + /// + /// The encoder frame geometry. + /// The row-major encoder segment map. + /// The block size. + /// The starting position in mode-info units. + /// The minimum segment identifier in the covered region. + public static int GetSegmentId(Av1EncoderCommon encoderCommon, ReadOnlySpan segmentIds, Av1BlockSize blockSize, Point modeInfoPosition) + { + int modeInfoOffset = (modeInfoPosition.Y * encoderCommon.ModeInfoColumnCount) + modeInfoPosition.X; + int blockWidth = blockSize.Get4x4WideCount(); + int blockHeight = blockSize.Get4x4HighCount(); + int columnCount = Math.Min(encoderCommon.ModeInfoColumnCount - modeInfoPosition.X, blockWidth); + int rowCount = Math.Min(encoderCommon.ModeInfoRowCount - modeInfoPosition.Y, blockHeight); + int segmentId = Av1Constants.MaxSegmentCount; + + for (int y = 0; y < rowCount; ++y) + { + int offset = modeInfoOffset + (y * encoderCommon.ModeInfoColumnCount); + for (int x = 0; x < columnCount; ++x) + { + segmentId = Math.Min(segmentId, segmentIds[offset + x]); + } + } + + Guard.IsTrue(segmentId is >= 0 and < Av1Constants.MaxSegmentCount, nameof(segmentId), "Segment ID needs to be in proper range."); + return segmentId; + } + + /// + /// Reconstructs a segment identifier coded as an alternating distance from its spatial predictor. + /// + /// The coded nonnegative distance symbol. + /// The predicted segment identifier. + /// The exclusive upper bound of the segment identifier range. + /// The reconstructed segment identifier. + public static int NegativeDeinterleave(int diff, int reference, int max) + { + if (reference == 0) + { + return diff; + } + + if (reference >= max - 1) + { + return max - diff - 1; + } + + if (2 * reference < max) + { + if (diff <= 2 * reference) + { + if ((diff & 1) > 0) + { + return reference + ((diff + 1) >> 1); + } + else + { + return reference - (diff >> 1); + } + } + + return diff; + } + else + { + if (diff <= 2 * (max - reference - 1)) + { + if ((diff & 1) > 0) + { + return reference + ((diff + 1) >> 1); + } + else + { + return reference - (diff >> 1); + } + } + + return max - (diff + 1); + } + } + + /// + /// Adds one decoded inter neighbor's primary and optional secondary reference votes. + /// + /// The decoded neighboring block. + /// The reference counts updated in place. + private static void AddNeighborReferenceCounts(Av1BlockModeInfo modeInfo, Span referenceCounts) + { + ReadOnlySpan referenceFrames = modeInfo.ReferenceFrames; + + if (referenceFrames[0] <= Av1ReferenceFrameType.Intra) + { + return; + } + + referenceCounts[(int)referenceFrames[0]]++; + + // A current block may use one reference, but the conditioning neighbors may be compound blocks. The reference decoder counts + // both labels so later single-reference decisions remain bit-exact when compound support is enabled. + if (referenceFrames[1] > Av1ReferenceFrameType.Intra) + { + referenceCounts[(int)referenceFrames[1]]++; + } + } + + /// + /// Converts neighboring votes for a binary reference-tree decision to its three-state AV1 context. + /// + /// The votes for the branch represented by symbol zero. + /// The votes for the branch represented by symbol one. + /// One for tied votes, zero when symbol one has more votes, or two when symbol zero has more votes. + private static int GetBinaryReferenceContext(int zeroSymbolCount, int oneSymbolCount) + => zeroSymbolCount == oneSymbolCount ? 1 : zeroSymbolCount < oneSymbolCount ? 0 : 2; + + /// + /// Determines whether a decoded block uses an inter reference. + /// + private static bool IsInterBlock(Av1BlockModeInfo modeInfo) + => modeInfo.ReferenceFrames[0] >= Av1ReferenceFrameType.Last; + + /// + /// Determines whether a decoded block has a second inter reference. + /// + private static bool HasCompoundReference(Av1BlockModeInfo modeInfo) + => modeInfo.ReferenceFrames[1] > Av1ReferenceFrameType.Intra; + + /// + /// Determines whether both compound references point in the same display-order direction. + /// + private static bool HasUnidirectionalCompoundReferences(Av1BlockModeInfo modeInfo) + => IsBackwardReference(modeInfo.ReferenceFrames[0]) == IsBackwardReference(modeInfo.ReferenceFrames[1]); + + /// + /// Determines whether a retained reference belongs to the backward group. + /// + private static bool IsBackwardReference(Av1ReferenceFrameType referenceFrame) + => referenceFrame >= Av1ReferenceFrameType.Backward; + + /// + /// Gets one neighbor's interpolation-filter contribution for the requested reference and direction. + /// + /// The decoded neighboring block, or when unavailable. + /// The current block's primary reference. + /// Zero for the vertical filter or one for the horizontal filter. + /// The selected filter index, or three when the neighbor does not contribute. + private static int GetReferenceInterpolationFilterContext( + Av1BlockModeInfo? modeInfo, + Av1ReferenceFrameType referenceFrame, + int direction) + { + if (modeInfo is null) + { + return SwitchableInterpolationFilterCount; + } + + Av1BlockModeInfo neighborModeInfo = modeInfo.Value; + ReadOnlySpan referenceFrames = neighborModeInfo.ReferenceFrames; + + // A compound neighbor contributes when either of its references matches the current primary reference. + if (referenceFrames[0] != referenceFrame && referenceFrames[1] != referenceFrame) + { + return SwitchableInterpolationFilterCount; + } + + return (int)neighborModeInfo.InterpolationFilters[direction]; + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Entropy/Av1SymbolDecoder.cs b/src/ImageSharp/Formats/Heif/Av1/Entropy/Av1SymbolDecoder.cs new file mode 100644 index 0000000000..eb4223e3a3 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Entropy/Av1SymbolDecoder.cs @@ -0,0 +1,1773 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1.Motion; +using SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.Quantizers; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.ChromaFromLuma; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; +using SixLabors.ImageSharp.Memory; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Entropy; + +/// +/// Decodes tile syntax elements and transform coefficients from an AV1 entropy-coded bitstream. +/// +internal ref struct Av1SymbolDecoder +{ + /// + /// The independently adaptable distribution graph for the current tile. + /// + private readonly Av1FrameEntropyContext context; + + /// + /// The range decoder over the current tile payload. + /// + private Av1SymbolReader reader; + + /// + /// Initializes a new instance of the struct for one AV1 tile. + /// + /// The configuration providing temporary memory. + /// The entropy-coded tile payload. + /// The frame base quantizer index. + /// A value indicating whether decoded symbols adapt their tile distributions. + public Av1SymbolDecoder(Configuration configuration, Span tileData, int qIndex, bool updateCdf = true) + : this(configuration, tileData, new Av1FrameEntropyContext(qIndex), updateCdf) + { + } + + /// + /// Initializes a new instance of the struct over a caller-owned tile entropy + /// context. + /// + /// The configuration providing temporary memory. + /// The entropy-coded tile payload. + /// The independently adaptable context initialized for this tile. + /// A value indicating whether decoded symbols adapt their tile distributions. + public Av1SymbolDecoder( + Configuration configuration, + Span tileData, + Av1FrameEntropyContext context, + bool updateCdf) + { + // The context owner controls reset and publication. Holding one reference here keeps the range decoder small + // and prevents a second set of aliases from becoming a competing source of entropy state. + this.context = context; + this.reader = new Av1SymbolReader(tileData, updateCdf); + } + + /// + /// Gets the reduced neighbor context for each intra prediction mode used by key-frame luma modes. + /// + private static ReadOnlySpan IntraModeContext => [0, 1, 2, 3, 4, 4, 4, 4, 3, 0, 1, 2, 0]; + + /// + /// Validates that range decoding remained within the bounded tile payload and consumed the required trailing-one bit. + /// + public void ValidateTrailingBits() + => this.reader.ValidateTrailingBits(); + + /// + /// Reads a fixed-width CDEF strength index. + /// + /// The number of bits signaled for the strength index. + /// The decoded CDEF strength index. + public int ReadCdfStrength(int bitCount) + { + ref Av1SymbolReader r = ref this.reader; + return r.ReadLiteral(bitCount); + } + + /// + /// Reads an unsigned fixed-width literal from the tile entropy stream. + /// + /// The number of literal bits to read. + /// The decoded literal. + public int ReadLiteral(int bitCount) + { + ref Av1SymbolReader r = ref this.reader; + return r.ReadLiteral(bitCount); + } + + /// + /// Reads a uniformly coded value from a non-power-of-two alphabet. + /// + /// The number of possible values. + /// A value in the range from zero through minus one. + public int ReadUniform(int valueCount) + { + ref Av1SymbolReader r = ref this.reader; + int bitCount = Av1Math.Log2(valueCount) + 1; + int threshold = (1 << bitCount) - valueCount; + int value = r.ReadLiteral(bitCount - 1); + if (value < threshold) + { + // The short prefix covers the lower values; only the remaining prefixes consume a final bit. + return value; + } + + return (value << 1) - threshold + r.ReadLiteral(1); + } + + /// + /// Reads sorted luma palette colors using selections from the neighboring color cache. + /// + /// The sorted unique colors inherited from eligible neighbors. + /// The number of palette colors. + /// The number of bits in each color sample. + /// The destination palette colors in prediction-index order. + public void ReadPaletteYColors( + scoped ReadOnlySpan colorCache, + int paletteSize, + int bitDepth, + scoped Span colors) + { + Span cachedColors = stackalloc ushort[Av1Constants.PaletteMaxSize]; + int colorIndex = 0; + for (int i = 0; i < colorCache.Length && colorIndex < paletteSize; i++) + { + if (this.ReadLiteral(1) != 0) + { + cachedColors[colorIndex++] = colorCache[i]; + } + } + + if (colorIndex == paletteSize) + { + cachedColors[..paletteSize].CopyTo(colors); + return; + } + + int cachedColorCount = colorIndex; + colors[colorIndex++] = (ushort)this.ReadLiteral(bitDepth); + if (colorIndex < paletteSize) + { + int bits = bitDepth - 3 + this.ReadLiteral(2); + int maximumColor = (1 << bitDepth) - 1; + int range = maximumColor - colors[colorIndex - 1]; + for (; colorIndex < paletteSize; colorIndex++) + { + int delta = this.ReadLiteral(bits) + 1; + colors[colorIndex] = (ushort)Av1Math.Clip3(0, maximumColor, colors[colorIndex - 1] + delta); + range -= colors[colorIndex] - colors[colorIndex - 1]; + bits = Math.Min(bits, (int)Av1Math.CeilLog2((uint)range)); + } + } + + MergePaletteColors(colors, cachedColors, paletteSize, cachedColorCount); + } + + /// + /// Reads shared chroma palette colors using cached U values and raw or delta-coded V values. + /// + /// The sorted unique U colors inherited from eligible neighbors. + /// The number of palette colors. + /// The number of bits in each color sample. + /// The destination U palette colors. + /// The destination V palette colors paired with . + public void ReadPaletteUvColors( + scoped ReadOnlySpan colorCache, + int paletteSize, + int bitDepth, + scoped Span uColors, + scoped Span vColors) + { + Span cachedColors = stackalloc ushort[Av1Constants.PaletteMaxSize]; + int colorIndex = 0; + for (int i = 0; i < colorCache.Length && colorIndex < paletteSize; i++) + { + if (this.ReadLiteral(1) != 0) + { + cachedColors[colorIndex++] = colorCache[i]; + } + } + + if (colorIndex < paletteSize) + { + int cachedColorCount = colorIndex; + uColors[colorIndex++] = (ushort)this.ReadLiteral(bitDepth); + if (colorIndex < paletteSize) + { + int bits = bitDepth - 3 + this.ReadLiteral(2); + int maximumColor = (1 << bitDepth) - 1; + int range = (1 << bitDepth) - uColors[colorIndex - 1]; + for (; colorIndex < paletteSize; colorIndex++) + { + int delta = this.ReadLiteral(bits); + uColors[colorIndex] = (ushort)Av1Math.Clip3(0, maximumColor, uColors[colorIndex - 1] + delta); + range -= uColors[colorIndex] - uColors[colorIndex - 1]; + bits = Math.Min(bits, (int)Av1Math.CeilLog2((uint)range)); + } + } + + MergePaletteColors(uColors, cachedColors, paletteSize, cachedColorCount); + } + else + { + cachedColors[..paletteSize].CopyTo(uColors); + } + + if (this.ReadLiteral(1) != 0) + { + // V deltas wrap in the unsigned sample domain so complementary chroma colors remain compact. + int bits = bitDepth - 4 + this.ReadLiteral(2); + int sampleRange = 1 << bitDepth; + vColors[0] = (ushort)this.ReadLiteral(bitDepth); + for (int i = 1; i < paletteSize; i++) + { + int delta = this.ReadLiteral(bits); + if (delta != 0 && this.ReadLiteral(1) != 0) + { + delta = -delta; + } + + int value = vColors[i - 1] + delta; + if (value < 0) + { + value += sampleRange; + } + + if (value >= sampleRange) + { + value -= sampleRange; + } + + vColors[i] = (ushort)value; + } + } + else + { + for (int i = 0; i < paletteSize; i++) + { + vColors[i] = (ushort)this.ReadLiteral(bitDepth); + } + } + } + + /// + /// Reads a palette color-index map in AV1 diagonal wavefront order. + /// + /// The number of colors in the palette. + /// The luma or chroma plane class. + /// The number of coded map rows. + /// The number of coded map columns. + /// The row-addressable destination map. + public void ReadPaletteColorMap( + int paletteSize, + Av1PlaneType planeType, + int rows, + int columns, + Buffer2DRegion colorIndexMap) + { + colorIndexMap.DangerousGetRowSpan(0)[0] = (byte)this.ReadUniform(paletteSize); + Span colorOrder = stackalloc byte[Av1Constants.PaletteMaxSize]; + for (int diagonal = 1; diagonal < rows + columns - 1; diagonal++) + { + int firstColumn = Math.Min(diagonal, columns - 1); + int lastColumn = Math.Max(0, diagonal - rows + 1); + for (int column = firstColumn; column >= lastColumn; column--) + { + int row = diagonal - column; + int colorContext = Av1PaletteColorMap.GetContext( + colorIndexMap, + row, + column, + paletteSize, + colorIndex: -1, + colorOrder, + out _); + + int colorOrderIndex = this.ReadPaletteColorIndex(paletteSize, colorContext, planeType); + colorIndexMap.DangerousGetRowSpan(row)[column] = colorOrder[colorOrderIndex]; + } + } + } + + /// + /// Reads a finite subexponential value recentered around a preceding value. + /// + /// The number of values in the coded domain. + /// The initial subexponential group-size exponent. + /// The preceding value expressed in the zero-based coded domain. + /// The decoded zero-based value. + public int ReadReferenceSubexponential(int valueCount, int k, int reference) + { + int value = this.ReadSubexponential(valueCount, k); + if ((reference << 1) <= valueCount) + { + return InverseRecenter(reference, value); + } + + return valueCount - 1 - InverseRecenter(valueCount - 1 - reference, value); + } + + /// + /// Reads the filter type selected for a switchable loop-restoration unit. + /// + /// The decoded unit filter type. + public Av1RestorationFilterType ReadSwitchableRestorationType() + { + ref Av1SymbolReader r = ref this.reader; + return (Av1RestorationFilterType)r.ReadSymbol(this.context.SwitchableRestoration); + } + + /// + /// Reads whether a Wiener loop-restoration unit applies its filter. + /// + /// when Wiener filtering is selected; otherwise, . + public bool ReadWienerRestoration() + { + ref Av1SymbolReader r = ref this.reader; + return r.ReadSymbol(this.context.WienerRestoration) != 0; + } + + /// + /// Reads whether a self-guided loop-restoration unit applies its filter. + /// + /// when self-guided filtering is selected; otherwise, . + public bool ReadSgrProjectionRestoration() + { + ref Av1SymbolReader r = ref this.reader; + return r.ReadSymbol(this.context.SgrProjectionRestoration) != 0; + } + + /// + /// Merges selected cached colors with the sorted transmitted colors. + /// + /// The transmitted colors beginning at and the merged output. + /// The selected cached colors in ascending order. + /// The total palette size. + /// The number of selected cached colors. + private static void MergePaletteColors( + Span colors, + ReadOnlySpan cachedColors, + int paletteSize, + int cachedColorCount) + { + if (cachedColorCount == 0) + { + return; + } + + int cacheIndex = 0; + int transmittedIndex = cachedColorCount; + for (int i = 0; i < paletteSize; i++) + { + if (cacheIndex < cachedColorCount && + (transmittedIndex >= paletteSize || cachedColors[cacheIndex] <= colors[transmittedIndex])) + { + colors[i] = cachedColors[cacheIndex++]; + } + else + { + colors[i] = colors[transmittedIndex++]; + } + } + } + + /// + /// Reads a finite subexponential code from the tile entropy stream. + /// + /// The number of values in the coded domain. + /// The initial subexponential group-size exponent. + /// The decoded zero-based value. + private int ReadSubexponential(int valueCount, int k) + { + int group = 0; + int groupStart = 0; + while (true) + { + int bitCount = group == 0 ? k : k + group - 1; + int groupSize = 1 << bitCount; + if (valueCount <= groupStart + (3 * groupSize)) + { + // The final group absorbs the remaining alphabet through truncated-binary coding + // once fewer than three full subexponential groups remain. + return this.ReadUniform(valueCount - groupStart) + groupStart; + } + + if (this.ReadLiteral(1) == 0) + { + return this.ReadLiteral(bitCount) + groupStart; + } + + group++; + groupStart += groupSize; + } + } + + /// + /// Maps a non-negative recentered code back around its reference value. + /// + /// The center of the coded value order. + /// The recentered non-negative value. + /// The value in its original non-negative domain. + private static int InverseRecenter(int reference, int value) + { + if (value > (reference << 1)) + { + return value; + } + + // Even and odd codes alternate above and below the reference so nearby values receive + // the shortest finite-subexponential representations. + return (value & 1) == 0 + ? (value >> 1) + reference + : reference - ((value + 1) >> 1); + } + + /// + /// Reads whether the current luma block uses palette prediction. + /// + /// The block-area context in the range from zero through six. + /// The number of available above and left luma neighbors that use palettes. + /// when luma palette prediction is selected; otherwise, . + public bool ReadPaletteYMode(int blockSizeContext, int neighborContext) + { + ref Av1SymbolReader r = ref this.reader; + return r.ReadSymbol(this.context.PaletteYMode[blockSizeContext][neighborContext]) != 0; + } + + /// + /// Reads whether the current chroma block uses palette prediction. + /// + /// A value indicating whether the current block uses a luma palette. + /// when chroma palette prediction is selected; otherwise, . + public bool ReadPaletteUvMode(bool hasLumaPalette) + { + ref Av1SymbolReader r = ref this.reader; + return r.ReadSymbol(this.context.PaletteUvMode[hasLumaPalette ? 1 : 0]) != 0; + } + + /// + /// Reads a luma or chroma palette size. + /// + /// The block-area context in the range from zero through six. + /// The luma or chroma plane class. + /// The palette size in the range from two through eight. + public int ReadPaletteSize(int blockSizeContext, Av1PlaneType planeType) + { + ref Av1SymbolReader r = ref this.reader; + Av1Distribution distribution = planeType == Av1PlaneType.Y + ? this.context.PaletteYSize[blockSizeContext] + : this.context.PaletteUvSize[blockSizeContext]; + + return r.ReadSymbol(distribution) + 2; + } + + /// + /// Reads a palette color-order index from the selected spatial context. + /// + /// The number of colors in the palette. + /// The color-index context derived from decoded neighboring indices. + /// The luma or chroma plane class. + /// The decoded index into the context-specific color order. + public int ReadPaletteColorIndex(int paletteSize, int colorContext, Av1PlaneType planeType) + { + ref Av1SymbolReader r = ref this.reader; + Av1Distribution distribution = planeType == Av1PlaneType.Y + ? this.context.PaletteYColorIndex[paletteSize - 2][colorContext] + : this.context.PaletteUvColorIndex[paletteSize - 2][colorContext]; + + return r.ReadSymbol(distribution); + } + + /// + /// Reads the frame-local intra-block-copy flag. + /// + /// when intra-block copy is selected. + public bool ReadUseIntraBlockCopy() + { + ref Av1SymbolReader r = ref this.reader; + return r.ReadSymbol(this.context.IntraBlockCopy) > 0; + } + + /// + /// Reads an integer intra-block-copy displacement vector relative to a spatial reference. + /// + /// The spatially derived reference vector. + /// The decoded displacement vector in one-eighth-sample units. + public Av1MotionVector ReadDisplacementVector(Av1MotionVector reference) + => this.context.DisplacementVector.Read(ref this.reader, reference, Av1MotionVectorPrecision.Integer); + + /// + /// Reads a normal inter-prediction motion vector relative to a selected reference candidate. + /// + /// The selected reference motion vector. + /// The fractional precision allowed by the current frame. + /// The decoded motion vector in one-eighth-sample units. + public Av1MotionVector ReadMotionVector(Av1MotionVector reference, Av1MotionVectorPrecision precision) + => this.context.MotionVector.Read(ref this.reader, reference, precision); + + /// + /// Reads a complete block partition type from the selected partition context. + /// + /// The partition probability context. + /// The decoded partition type. + public Av1PartitionType ReadPartitionType(int context) + { + ref Av1SymbolReader r = ref this.reader; + return (Av1PartitionType)r.ReadSymbol(this.context.PartitionTypes[context]); + } + + /// + /// Reads the binary split-versus-horizontal decision used at a clipped bottom tile boundary. + /// + /// The current block size. + /// The partition probability context. + /// or . + public Av1PartitionType ReadSplitOrHorizontal(Av1BlockSize blockSize, int context) + { + uint frequency = GetSplitOrHorizontalFrequency(this.context.PartitionTypes, blockSize, context); + ref Av1SymbolReader r = ref this.reader; + return r.ReadBoolean(frequency) ? Av1PartitionType.Split : Av1PartitionType.Horizontal; + } + + /// + /// Reads the binary split-versus-vertical decision used at a clipped right tile boundary. + /// + /// The current block size. + /// The partition probability context. + /// or . + public Av1PartitionType ReadSplitOrVertical(Av1BlockSize blockSize, int context) + { + uint frequency = GetSplitOrVerticalFrequency(this.context.PartitionTypes, blockSize, context); + ref Av1SymbolReader r = ref this.reader; + return r.ReadBoolean(frequency) ? Av1PartitionType.Split : Av1PartitionType.Vertical; + } + + /// + /// Reads a key-frame luma prediction mode using the available above and left modes. + /// + /// The above block mode, or at the frame boundary. + /// The left block mode, or at the frame boundary. + /// The decoded luma prediction mode. + public Av1PredictionMode ReadYMode(Av1BlockModeInfo? aboveModeInfo, Av1BlockModeInfo? leftModeInfo) + { + ref Av1SymbolReader r = ref this.reader; + Av1PredictionMode aboveMode = Av1PredictionMode.DC; + if (aboveModeInfo is not null) + { + aboveMode = aboveModeInfo.Value.YMode; + } + + Av1PredictionMode leftMode = Av1PredictionMode.DC; + if (leftModeInfo is not null) + { + leftMode = leftModeInfo.Value.YMode; + } + + int aboveContext = IntraModeContext[(int)aboveMode]; + int leftContext = IntraModeContext[(int)leftMode]; + return (Av1PredictionMode)r.ReadSymbol(this.context.KeyFrameYMode[aboveContext][leftContext]); + } + + /// + /// Reads an intra luma prediction mode for a block coded inside an inter frame. + /// + /// The decoded block size that selects the luma-mode distribution. + /// The decoded intra luma prediction mode. + public Av1PredictionMode ReadInterFrameYMode(Av1BlockSize blockSize) + { + int sizeGroup = blockSize.GetSizeGroup(); + ref Av1SymbolReader r = ref this.reader; + return (Av1PredictionMode)r.ReadSymbol(this.context.FrameYMode[sizeGroup]); + } + + /// + /// Reads whether a single-reference inter block uses inter-intra prediction. + /// + /// The decoded block size that selects the inter-intra flag distribution. + /// when an intra predictor is blended with the inter predictor. + public bool ReadIsInterIntra(Av1BlockSize blockSize) + { + int sizeGroup = blockSize.GetSizeGroup(); + ref Av1SymbolReader r = ref this.reader; + return r.ReadSymbol(this.context.InterIntra[sizeGroup]) != 0; + } + + /// + /// Reads the intra mode blended into a selected inter-intra block. + /// + /// The decoded block size that selects the mode distribution. + /// The selected inter-intra mode. + public Av1InterIntraMode ReadInterIntraMode(Av1BlockSize blockSize) + { + int sizeGroup = blockSize.GetSizeGroup(); + ref Av1SymbolReader r = ref this.reader; + return (Av1InterIntraMode)r.ReadSymbol(this.context.InterIntraMode[sizeGroup]); + } + + /// + /// Reads whether a selected inter-intra block uses a wedge mask. + /// + /// The decoded block size that selects the wedge flag distribution. + /// when wedge blending is selected. + public bool ReadUseInterIntraWedge(Av1BlockSize blockSize) + { + ref Av1SymbolReader r = ref this.reader; + return r.ReadSymbol(this.context.WedgeInterIntra[(int)blockSize]) != 0; + } + + /// + /// Reads a wedge-mask index. + /// + /// The decoded block size that selects the wedge-index distribution. + /// The wedge index in the inclusive range zero through fifteen. + public byte ReadWedgeIndex(Av1BlockSize blockSize) + { + ref Av1SymbolReader r = ref this.reader; + return (byte)r.ReadSymbol(this.context.WedgeIndex[(int)blockSize]); + } + + /// + /// Reads whether a compound block uses the masked-compound mode group. + /// + /// The derived neighboring compound-group context. + /// for masked compound prediction. + public bool ReadCompoundGroupIndex(int context) + { + ref Av1SymbolReader r = ref this.reader; + return r.ReadSymbol(this.context.CompoundGroupIndex[context]) != 0; + } + + /// + /// Reads whether an unmasked compound block uses equal averaging. + /// + /// The derived neighboring compound-index context. + /// for equal averaging; otherwise, distance-weighted blending. + public bool ReadCompoundIndex(int context) + { + ref Av1SymbolReader r = ref this.reader; + return r.ReadSymbol(this.context.CompoundIndex[context]) != 0; + } + + /// + /// Reads the selected masked-compound type for a wedge-capable block. + /// + /// The decoded block size that selects the masked-compound distribution. + /// The selected wedge or difference-weighted compound type. + public Av1CompoundType ReadMaskedCompoundType(Av1BlockSize blockSize) + { + ref Av1SymbolReader r = ref this.reader; + return (Av1CompoundType)((int)Av1CompoundType.Wedge + r.ReadSymbol(this.context.CompoundType[(int)blockSize])); + } + + /// + /// Reads the motion model selected for an eligible single-reference inter block. + /// + /// The decoded block size that selects the motion-mode distribution. + /// + /// A value indicating whether the block may select Warped in addition to Simple Translation and OBMC. + /// + /// The decoded motion mode. + public Av1MotionMode ReadMotionMode(Av1BlockSize blockSize, bool allowWarpedMotion) + { + ref Av1SymbolReader r = ref this.reader; + + // AV1 uses a separate binary CDF when Warped is ineligible; reading the first two leaves from the three-way + // CDF would use different probabilities and desynchronize the range decoder even when Simple is selected. + return allowWarpedMotion + ? (Av1MotionMode)r.ReadSymbol(this.context.MotionMode[(int)blockSize]) + : (Av1MotionMode)r.ReadSymbol(this.context.Obmc[(int)blockSize]); + } + + /// + /// Reads whether an inter-frame block uses inter prediction. + /// + /// The spatial intra/inter context in the inclusive range zero through three. + /// when the block uses inter prediction; otherwise, . + public bool ReadIsInter(int context) + { + ref Av1SymbolReader r = ref this.reader; + return r.ReadSymbol(this.context.IntraInter[context]) != 0; + } + + /// + /// Reads whether an inter block uses compound-reference instead of single-reference prediction. + /// + /// The spatial block reference-mode context in the inclusive range zero through four. + /// for compound-reference prediction; otherwise, . + public bool ReadIsCompoundReference(int context) + { + ref Av1SymbolReader r = ref this.reader; + + return r.ReadSymbol(this.context.CompInter[context]) != 0; + } + + /// + /// Reads whether a compound block uses one forward and one backward reference. + /// + /// The compound reference-type context in the inclusive range zero through four. + /// for a bidirectional pair; otherwise, . + public bool ReadCompoundReferenceIsBidirectional(int context) + { + ref Av1SymbolReader r = ref this.reader; + + return r.ReadSymbol(this.context.CompoundReferenceType[context]) != 0; + } + + /// + /// Reads one decision from the unidirectional compound-reference tree. + /// + /// The neighboring reference-vote context. + /// The zero-based tree decision in the inclusive range zero through two. + /// The decoded binary decision. + public bool ReadUnidirectionalCompoundReference(int context, int decision) + { + ref Av1SymbolReader r = ref this.reader; + + return r.ReadSymbol(this.context.UnidirectionalCompoundReference[context][decision]) != 0; + } + + /// + /// Reads one decision from the bidirectional compound forward-reference tree. + /// + /// The neighboring reference-vote context. + /// The zero-based tree decision in the inclusive range zero through two. + /// The decoded binary decision. + public bool ReadCompoundForwardReference(int context, int decision) + { + ref Av1SymbolReader r = ref this.reader; + + return r.ReadSymbol(this.context.CompoundReference[context][decision]) != 0; + } + + /// + /// Reads one decision from the bidirectional compound backward-reference tree. + /// + /// The neighboring reference-vote context. + /// The zero-based tree decision in the inclusive range zero through one. + /// The decoded binary decision. + public bool ReadCompoundBackwardReference(int context, int decision) + { + ref Av1SymbolReader r = ref this.reader; + + return r.ReadSymbol(this.context.CompoundBackwardReference[context][decision]) != 0; + } + + /// + /// Reads one per-block interpolation filter selected by a switchable frame. + /// + /// The reference, direction, and neighbor filter context. + /// The selected Regular, Smooth, or Sharp interpolation filter. + public Av1InterpolationFilter ReadSwitchableInterpolationFilter(int context) + { + ref Av1SymbolReader r = ref this.reader; + + return (Av1InterpolationFilter)r.ReadSymbol(this.context.SwitchableInterpolation[context]); + } + + /// + /// Reads the prediction mode for a single-reference inter block. + /// + /// The packed mode context produced by reference-motion-vector candidate analysis. + /// The selected new, global, nearest, or near motion-vector mode. + public Av1PredictionMode ReadInterMode(int modeContext) + { + ref Av1SymbolReader r = ref this.reader; + int newMvContext = Av1SymbolContextHelper.GetNewMvContext(modeContext); + + // AV1 assigns symbol zero to the NEWMV leaf and symbol one to the rest of the tree. Returning at the leaf is + // required both for the selected mode and to avoid consuming the unrelated lower decisions. + if (r.ReadSymbol(this.context.NewMv[newMvContext]) == 0) + { + return Av1PredictionMode.NewMotionVector; + } + + int zeroMvContext = Av1SymbolContextHelper.GetZeroMvContext(modeContext); + if (r.ReadSymbol(this.context.ZeroMv[zeroMvContext]) == 0) + { + return Av1PredictionMode.GlobalMotionVector; + } + + // The final zero symbol selects the nearest spatial candidate; one selects the near candidate and may be + // followed by dynamic-reference-list syntax when more than one near candidate is available. + int refMvContext = Av1SymbolContextHelper.GetRefMvContext(modeContext); + return r.ReadSymbol(this.context.RefMv[refMvContext]) == 0 + ? Av1PredictionMode.NearestMotionVector + : Av1PredictionMode.NearMotionVector; + } + + /// + /// Reads the prediction mode for a compound-reference inter block. + /// + /// The packed mode context produced by paired reference-motion-vector analysis. + /// The selected compound motion-vector mode. + public Av1PredictionMode ReadInterCompoundMode(int modeContext) + { + ref Av1SymbolReader r = ref this.reader; + int context = Av1SymbolContextHelper.GetCompoundModeContext(modeContext); + int mode = r.ReadSymbol(this.context.InterCompoundMode[context]); + + return (Av1PredictionMode)((int)Av1PredictionMode.NearestNearestMotionVector + mode); + } + + /// + /// Reads one dynamic reference-list decision for adjacent motion-vector candidates. + /// + /// The candidate-weight context in the inclusive range zero through two. + /// + /// when selection advances past the current candidate; otherwise, . + /// + public bool ReadDrl(int context) + { + ref Av1SymbolReader r = ref this.reader; + return r.ReadSymbol(this.context.Drl[context]) != 0; + } + + /// + /// Reads whether a single-reference block selects the backward-reference group. + /// + /// The neighboring forward-versus-backward vote context. + /// for a backward reference; otherwise, . + public bool ReadSingleReferenceIsBackward(int context) + => this.ReadSingleReferenceDecision(context, decision: 0); + + /// + /// Reads whether a backward single-reference block selects Alternate. + /// + /// The neighboring Backward-or-Alternate2-versus-Alternate vote context. + /// for Alternate; otherwise, . + public bool ReadSingleReferenceIsAlternate(int context) + => this.ReadSingleReferenceDecision(context, decision: 1); + + /// + /// Reads whether a forward single-reference block selects the Last3-or-Golden group. + /// + /// The neighboring near-forward-versus-far-forward vote context. + /// for Last3 or Golden; otherwise, . + public bool ReadSingleReferenceIsLast3OrGolden(int context) + => this.ReadSingleReferenceDecision(context, decision: 2); + + /// + /// Reads whether a near-forward single-reference block selects Last2. + /// + /// The neighboring Last-versus-Last2 vote context. + /// for Last2; otherwise, . + public bool ReadSingleReferenceIsLast2(int context) + => this.ReadSingleReferenceDecision(context, decision: 3); + + /// + /// Reads whether a far-forward single-reference block selects Golden. + /// + /// The neighboring Last3-versus-Golden vote context. + /// for Golden; otherwise, . + public bool ReadSingleReferenceIsGolden(int context) + => this.ReadSingleReferenceDecision(context, decision: 4); + + /// + /// Reads whether a non-Alternate backward single-reference block selects Alternate2. + /// + /// The neighboring Backward-versus-Alternate2 vote context. + /// for Alternate2; otherwise, . + public bool ReadSingleReferenceIsAlternate2(int context) + => this.ReadSingleReferenceDecision(context, decision: 5); + + /// + /// Reads one binary decision from the single-reference selection tree. + /// + /// The neighboring reference-vote context. + /// The zero-based tree decision matching one single_ref_cdf column. + /// when the decision selects symbol one; otherwise, . + private bool ReadSingleReferenceDecision(int context, int decision) + { + ref Av1SymbolReader r = ref this.reader; + + return r.ReadSymbol(this.context.SingleReference[context][decision]) != 0; + } + + /// + /// Reads a chroma intra prediction mode conditioned on the luma mode and chroma-from-luma availability. + /// + /// The decoded luma prediction mode. + /// Indicates whether chroma-from-luma is valid for the block. + /// The decoded chroma prediction mode. + public Av1ChromaPredictionMode ReadIntraModeUv(Av1PredictionMode mode, bool chromaFromLumaAllowed) + { + int chromaForLumaIndex = chromaFromLumaAllowed ? 1 : 0; + ref Av1SymbolReader r = ref this.reader; + return (Av1ChromaPredictionMode)r.ReadSymbol(this.context.UvMode[chromaForLumaIndex][(int)mode]); + } + + /// + /// Reads the transform-skip flag from a neighboring skip context. + /// + /// The neighboring skip context. + /// when the block contains no coded transform coefficients. + public bool ReadSkip(int ctx) + { + ref Av1SymbolReader r = ref this.reader; + return r.ReadSymbol(this.context.Skip[ctx]) > 0; + } + + /// + /// Reads the compound-reference skip-mode flag. + /// + /// The neighboring skip-mode context. + /// when skip mode is selected. + public bool ReadSkipMode(int context) + { + ref Av1SymbolReader r = ref this.reader; + return r.ReadSymbol(this.context.SkipMode[context]) > 0; + } + + /// + /// Reads a signed loop-filter delta value. + /// + /// Whether the frame signals independent deltas for each loop-filter channel. + /// The loop-filter channel, used only when is true. + /// The decoded loop-filter delta. + public int ReadDeltaLoopFilter(bool isMulti, int channel) + { + ref Av1SymbolReader r = ref this.reader; + + // Multi-delta syntax adapts one CDF per filter channel. Sharing the scalar-delta CDF would let + // an earlier channel change the range intervals used to decode the next channel in the same block. + Av1Distribution distribution = isMulti ? this.context.DeltaLoopFilterMultiAbsolute[channel] : this.context.DeltaLoopFilterAbsolute; + int deltaLoopFilterAbsolute = r.ReadSymbol(distribution); + if (deltaLoopFilterAbsolute == Av1Constants.DeltaLoopFilterSmall) + { + int deltaLoopFilterRemainingBits = r.ReadLiteral(3) + 1; + int deltaLoopFilterAbsoluteBitCount = r.ReadLiteral(deltaLoopFilterRemainingBits); + deltaLoopFilterAbsolute = deltaLoopFilterAbsoluteBitCount + (1 << deltaLoopFilterRemainingBits) + 1; + } + + bool deltaLoopFilterSign = true; + if (deltaLoopFilterAbsolute != 0) + { + deltaLoopFilterSign = r.ReadLiteral(1) > 0; + } + + return deltaLoopFilterSign ? -deltaLoopFilterAbsolute : deltaLoopFilterAbsolute; + } + + /// + /// Reads a signed quantizer-index delta value. + /// + /// The decoded quantizer-index delta. + public int ReadDeltaQuantizerIndex() + { + ref Av1SymbolReader r = ref this.reader; + int deltaQuantizerAbsolute = r.ReadSymbol(this.context.DeltaQuantizerAbsolute); + if (deltaQuantizerAbsolute == Av1Constants.DeltaQuantizerSmall) + { + int deltaQuantizerRemainingBits = r.ReadLiteral(3) + 1; + int deltaQuantizerAbsoluteBase = r.ReadLiteral(deltaQuantizerRemainingBits); + deltaQuantizerAbsolute = deltaQuantizerAbsoluteBase + (1 << deltaQuantizerRemainingBits) + 1; + } + + bool deltaQuantizerSignBit = true; + if (deltaQuantizerAbsolute != 0) + { + deltaQuantizerSignBit = r.ReadLiteral(1) > 0; + } + + return deltaQuantizerSignBit ? -deltaQuantizerAbsolute : deltaQuantizerAbsolute; + } + + /// + /// Reads a spatially predicted segment identifier. + /// + /// The context derived from neighboring segment identifiers. + /// The decoded segment identifier. + public int ReadSegmentId(int context) + { + ref Av1SymbolReader r = ref this.reader; + return r.ReadSymbol(this.context.SegmentId[context]); + } + + /// + /// Reads whether the current segment identifier is predicted from the retained primary-frame map. + /// + /// The sum of the above and left blocks' temporal-prediction flags. + /// when the retained map supplies the segment identifier. + public bool ReadSegmentIdPredicted(int context) + { + ref Av1SymbolReader r = ref this.reader; + return r.ReadSymbol(this.context.SegmentIdPredicted[context]) > 0; + } + + /// + /// Reads the unsigned directional angle-delta symbol for a prediction mode. + /// + /// The directional prediction mode. + /// The symbol in the range zero through twice the maximum signed angle delta. + public int ReadAngleDelta(Av1PredictionMode mode) + { + ref Av1SymbolReader r = ref this.reader; + return r.ReadSymbol(this.context.AngleDelta[(int)mode - 1]); + } + + /// + /// Reads the filter-intra enable flag and, when enabled, its prediction mode. + /// + /// The block size selecting the enable distribution. + /// The selected mode, or when filter-intra is disabled. + public Av1FilterIntraMode ReadFilterUltraMode(Av1BlockSize blockSize) + { + ref Av1SymbolReader r = ref this.reader; + Av1FilterIntraMode filterIntraMode = Av1FilterIntraMode.AllFilterIntraModes; + bool useFilterIntra = r.ReadSymbol(this.context.FilterIntra[(int)blockSize]) > 0; + if (useFilterIntra) + { + filterIntraMode = (Av1FilterIntraMode)r.ReadSymbol(this.context.FilterIntraMode); + } + + return filterIntraMode; + } + + /// + /// Reads a transform subdivision depth and resolves it to a transform size. + /// + /// The block size defining the maximum transform. + /// The neighboring transform-size context. + /// The decoded transform size. + public Av1TransformSize ReadTransformSize(Av1BlockSize blockSize, int context) + { + ref Av1SymbolReader r = ref this.reader; + Av1TransformSize maxTransformSize = blockSize.GetMaximumTransformSize(); + int depth = 0; + while (maxTransformSize != Av1TransformSize.Size4x4) + { + depth++; + maxTransformSize = maxTransformSize.GetSubSize(); + DebugGuard.MustBeLessThan(depth, 10, nameof(depth)); + } + + DebugGuard.MustBeLessThanOrEqualTo(depth, Av1Constants.MaxTransformCategories, nameof(depth)); + int category = depth - 1; + int value = r.ReadSymbol(this.context.TransformSize[category][context]); + Av1TransformSize transformSize = blockSize.GetMaximumTransformSize(); + for (int d = 0; d < value; ++d) + { + transformSize = transformSize.GetSubSize(); + } + + return transformSize; + } + + /// + /// Reads whether an inter transform node is subdivided. + /// + /// The variable-transform partition context. + /// when the transform node is split. + public bool ReadTransformPartition(int context) + { + ref Av1SymbolReader r = ref this.reader; + return r.ReadSymbol(this.context.TransformPartition[context]) > 0; + } + + /// + /// Reads a transform type from the transform set permitted for the block. + /// + /// The coded transform size. + /// Indicates whether the frame restricts transform choices. + /// Indicates whether the block uses inter prediction. + /// Indicates whether filter-intra prediction selected the intra direction. + /// Indicates whether the active segment uses lossless transforms. + /// The filter-intra mode when enabled. + /// The ordinary intra prediction mode. + /// The decoded transform type, or DCT-DCT when no transform type is signaled. + public Av1TransformType ReadTransformType( + Av1TransformSize transformSize, + bool useReducedTransformSet, + bool isInter, + bool useFilterIntra, + bool isLossless, + Av1FilterIntraMode filterIntraMode, + Av1PredictionMode intraDirection) + { + Av1TransformType transformType = Av1TransformType.DctDct; + + // A lossless segment selects DCT-DCT and carries no transform-type symbol. + if (isLossless) + { + return transformType; + } + + Av1TransformSetType transformSetType = Av1SymbolContextHelper.GetExtendedTransformSetType(transformSize, isInter, useReducedTransformSet); + if (transformSetType > Av1TransformSetType.DctOnly) + { + int extendedSet = Av1SymbolContextHelper.GetExtendedTransformSet(transformSetType, isInter); + Av1TransformSize squareTransformSize = transformSize.GetSquareSize(); + ref Av1SymbolReader r = ref this.reader; + int symbol; + if (isInter) + { + symbol = r.ReadSymbol(this.context.InterExtendedTransform[extendedSet][(int)squareTransformSize]); + } + else + { + Av1PredictionMode intraMode = useFilterIntra + ? filterIntraMode.ToIntraDirection() + : intraDirection; + + symbol = r.ReadSymbol(this.context.IntraExtendedTransform[extendedSet][(int)squareTransformSize][(int)intraMode]); + } + + transformType = Av1SymbolContextHelper.GetExtendedTransformType(transformSetType, symbol); + } + + return transformType; + } + + /// + /// Reads whether a transform block has no coded coefficients. + /// + /// The square transform-size probability context. + /// The context derived from neighboring coefficient blocks. + /// when the transform block is empty. + public bool ReadTransformBlockSkip(Av1TransformSize transformSizeContext, int skipContext) + { + ref Av1SymbolReader r = ref this.reader; + return r.ReadSymbol(this.context.TransformBlockSkip[(int)transformSizeContext][skipContext]) > 0; + } + + /// + /// Reads the joint U/V sign symbol for chroma-from-luma alpha values. + /// + /// The joint sign symbol. + public int ReadChromFromLumaSign() + { + ref Av1SymbolReader r = ref this.reader; + return r.ReadSymbol(this.context.ChromaFromLumaSign); + } + + /// + /// Reads the U-plane chroma-from-luma alpha-magnitude symbol. + /// + /// The one-based joint U/V sign symbol. + /// The U-plane alpha-magnitude symbol. + public int ReadChromaFromLumaAlphaU(int jointSignPlus1) + { + ref Av1SymbolReader r = ref this.reader; + int context = Av1ChromaFromLumaMath.ContextU(jointSignPlus1 - 1); + return r.ReadSymbol(this.context.ChromaFromLumaAlpha[context]); + } + + /// + /// Reads the V-plane chroma-from-luma alpha-magnitude symbol. + /// + /// The one-based joint U/V sign symbol. + /// The V-plane alpha-magnitude symbol. + public int ReadChromaFromLumaAlphaV(int jointSignPlus1) + { + ref Av1SymbolReader r = ref this.reader; + int context = Av1ChromaFromLumaMath.ContextV(jointSignPlus1 - 1); + return r.ReadSymbol(this.context.ChromaFromLumaAlpha[context]); + } + + /// + /// Decodes one transform block's coefficient syntax and updates its neighboring entropy contexts. + /// + /// The current block prediction and segment modes. + /// The transform-block offset within the coding block in four-sample units. + /// The above coefficient contexts for the current plane. + /// The left coefficient contexts for the current plane. + /// The first tile-relative above context covered by the transform. + /// The first superblock-row-relative left context covered by the transform. + /// The zero-based Y, U, or V plane index. + /// The available plane width in four-sample units. + /// The available plane height in four-sample units. + /// The neighboring skip and DC sign contexts. + /// The signaled transform size. + /// Indicates whether the active segment is lossless. + /// Indicates whether the frame restricts transform choices. + /// The luma transform type shared by inter-predicted chroma. + /// The transform descriptor updated with the decoded type and end-of-block position. + /// The signed distance from the mode block to the right frame edge. + /// The signed distance from the mode block to the bottom frame edge. + /// Reusable padded coefficient-context storage owned by the tile reader. + /// The zero-initialized destination receiving dequantized raster coefficients. + /// The quantizer containing the active segment and superblock delta-Q values. + /// The one-based end-of-block position, or zero for an empty transform block. + public int ReadCoefficients( + Av1BlockModeInfo modeInfo, + Point blockPosition, + Span aboveContexts, + Span leftContexts, + int aboveOffset, + int leftOffset, + int plane, + int blocksWide, + int blocksHigh, + Av1TransformBlockContext transformBlockContext, + Av1TransformSize transformSize, + bool isLossless, + bool useReducedTransformSet, + Av1TransformType lumaTransformType, + ref Av1TransformInfo transformInfo, + int modeBlocksToRightEdge, + int modeBlocksToBottomEdge, + Av1LevelBuffer levels, + Span coefficientBuffer, + Av1InverseQuantizer inverseQuantizer) + { + Av1TransformSize adjustedTransformSize = transformSize.GetAdjusted(); + int width = adjustedTransformSize.GetWidth(); + int height = adjustedTransformSize.GetHeight(); + Av1TransformSize transformSizeContext = Av1SymbolContextHelper.GetTransformSizeContext(transformSize); + Av1PlaneType planeType = (Av1PlaneType)Math.Min(plane, 1); + int culLevel = 0; + + // AV1 omits high-frequency coefficients beyond 32 samples on every 64-point transform dimension. Reusing + // tile-owned storage avoids an allocator round trip for every transform block. + levels.Reset(new Size(width, height)); + + bool allZero = this.ReadTransformBlockSkip(transformSizeContext, transformBlockContext.SkipContext); + int endOfBlock; + if (allZero) + { + transformInfo.EndOfBlock = 0; + if (plane == 0) + { + transformInfo.Type = Av1TransformType.DctDct; + } + + UpdateCoefficientContext(aboveContexts, leftContexts, blocksWide, blocksHigh, transformSize, blockPosition, aboveOffset, leftOffset, culLevel, modeBlocksToRightEdge, modeBlocksToBottomEdge); + return 0; + } + + bool usesInterTransformSet = modeInfo.ReferenceFrames[0] >= Av1ReferenceFrameType.Last || modeInfo.UseIntraBlockCopy; + if (plane == (int)Av1Plane.Y) + { + // Transform-set selection follows the prediction class. Intra-block copy uses inter residual syntax even + // though its reference is the current frame; ordinary inter blocks are identified by their retained ref. + transformInfo.Type = this.ReadTransformType( + transformSize, + useReducedTransformSet, + usesInterTransformSet, + modeInfo.UseFilterIntra, + isLossless, + modeInfo.FilterIntraMode, + modeInfo.YMode); + } + + transformInfo.Type = ComputeTransformType( + planeType, + modeInfo, + isLossless, + transformSize, + lumaTransformType, + in transformInfo, + useReducedTransformSet, + usesInterTransformSet); + Av1TransformClass transformClass = transformInfo.Type.ToClass(); + Av1ScanOrder scanOrder = Av1ScanOrderConstants.GetScanOrder(transformSize, transformInfo.Type); + ReadOnlySpan scan = scanOrder.Scan; + + endOfBlock = this.ReadEndOfBlockPosition(transformSize, transformClass, transformSizeContext, planeType); + if (endOfBlock > 1) + { + levels.Clear(); + } + + this.ReadCoefficientsEndOfBlock(transformClass, endOfBlock, scan, levels, transformSizeContext, planeType); + if (endOfBlock > 1) + { + if (transformClass == Av1TransformClass.Class2D) + { + this.ReadCoefficientsReverse2d(transformSize, 1, endOfBlock - 1 - 1, scan, levels, transformSizeContext, planeType); + this.ReadCoefficientsReverse(transformSize, transformClass, 0, 0, scan, levels, transformSizeContext, planeType); + } + else + { + this.ReadCoefficientsReverse(transformSize, transformClass, 0, endOfBlock - 1 - 1, scan, levels, transformSizeContext, planeType); + } + } + + DebugGuard.MustBeGreaterThan(scan.Length, 0, nameof(scan)); + Av1InverseQuantizer.TransformParameters quantization = new( + inverseQuantizer, modeInfo, transformInfo.Type, transformSize, (Av1Plane)plane); + + culLevel = this.ReadCoefficientsSign( + coefficientBuffer, endOfBlock, scan, levels, transformBlockContext.DcSignContext, planeType, quantization); + + UpdateCoefficientContext(aboveContexts, leftContexts, blocksWide, blocksHigh, transformSize, blockPosition, aboveOffset, leftOffset, culLevel, modeBlocksToRightEdge, modeBlocksToBottomEdge); + + transformInfo.EndOfBlock = (ushort)endOfBlock; + return endOfBlock; + } + + /// + /// Reads an end-of-block token and its literal suffix. + /// + /// The signaled transform size selecting the token alphabet. + /// The transform class selecting the two-dimensional or one-dimensional model. + /// The square transform-size probability context. + /// The luma or chroma plane category. + /// The one-based end-of-block coefficient position. + public int ReadEndOfBlockPosition(Av1TransformSize transformSize, Av1TransformClass transformClass, Av1TransformSize transformSizeContext, Av1PlaneType planeType) + { + ref Av1SymbolReader r = ref this.reader; + int endOfBlockExtra = 0; + int endOfBlockPoint = this.ReadEndOfBlockFlag(planeType, transformClass, transformSize); + int endOfBlockShift = Av1SymbolContextHelper.EndOfBlockOffsetBits[endOfBlockPoint]; + if (endOfBlockShift > 0) + { + // The local table retains placeholders for the first three tokens, unlike the reference decoder's compact table, + // so the decoded token is also the distribution index. + int endOfBlockContext = endOfBlockPoint; + bool bit = this.ReadEndOfBlockExtra(transformSizeContext, planeType, endOfBlockContext); + if (bit) + { + Av1Math.SetBit(ref endOfBlockExtra, endOfBlockShift - 1); + } + + for (int j = 1; j < endOfBlockShift; j++) + { + if (r.ReadLiteral(1) != 0) + { + Av1Math.SetBit(ref endOfBlockExtra, endOfBlockShift - 1 - j); + } + } + } + + return Av1SymbolContextHelper.RecordEndOfBlockPosition(endOfBlockPoint, endOfBlockExtra); + } + + /// + /// Decodes the mandatory nonzero coefficient at the end-of-block scan position. + /// + /// The transform direction class. + /// The one-based end-of-block position. + /// The transform's scan-to-raster mapping. + /// The padded absolute-coefficient level plane to update. + /// The square transform-size probability context. + /// The luma or chroma plane category. + public void ReadCoefficientsEndOfBlock(Av1TransformClass transformClass, int endOfBlock, ReadOnlySpan scan, Av1LevelBuffer levels, Av1TransformSize transformSizeContext, Av1PlaneType planeType) + { + int i = endOfBlock - 1; + Point position = levels.GetPosition(scan[i]); + int coefficientContext = Av1SymbolContextHelper.GetLowerLevelContextEndOfBlock(levels, i); + int level = this.ReadBaseEndOfBlock(transformSizeContext, planeType, coefficientContext) + 1; + if (level > Av1Constants.BaseLevelsCount) + { + int baseRangeContext = Av1SymbolContextHelper.GetBaseRangeContextEndOfBlock(position, transformClass); + this.ReadCoefficientsBaseRangeLoop(transformSizeContext, planeType, baseRangeContext, ref level); + } + + levels.GetRow(position)[position.X] = (byte)level; + } + + /// + /// Decodes a reverse scan range using the specialized two-dimensional coefficient contexts. + /// + /// The signaled transform size. + /// The inclusive lowest scan index. + /// The inclusive highest scan index. + /// The transform's scan-to-raster mapping. + /// The padded absolute-coefficient level plane to update. + /// The square transform-size probability context. + /// The luma or chroma plane category. + public void ReadCoefficientsReverse2d(Av1TransformSize transformSize, int startScanIndex, int endScanIndex, ReadOnlySpan scan, Av1LevelBuffer levels, Av1TransformSize transformSizeContext, Av1PlaneType planeType) + { + for (int c = endScanIndex; c >= startScanIndex; --c) + { + Point position = levels.GetPosition(scan[c]); + int coefficientContext = Av1SymbolContextHelper.GetLowerLevelsContext2d(levels, position, transformSize); + int level = this.ReadCoefficientsBase(transformSizeContext, planeType, coefficientContext); + if (level > Av1Constants.BaseLevelsCount) + { + int baseRangeContext = Av1SymbolContextHelper.GetBaseRangeContext2d(levels, position); + this.ReadCoefficientsBaseRangeLoop(transformSizeContext, planeType, baseRangeContext, ref level); + } + + levels.GetRow(position)[position.X] = (byte)level; + } + } + + /// + /// Decodes a reverse scan range using transform-class-specific coefficient contexts. + /// + /// The signaled transform size. + /// The transform direction class. + /// The inclusive lowest scan index. + /// The inclusive highest scan index. + /// The transform's scan-to-raster mapping. + /// The padded absolute-coefficient level plane to update. + /// The square transform-size probability context. + /// The luma or chroma plane category. + public void ReadCoefficientsReverse(Av1TransformSize transformSize, Av1TransformClass transformClass, int startScanIndex, int endScanIndex, ReadOnlySpan scan, Av1LevelBuffer levels, Av1TransformSize transformSizeContext, Av1PlaneType planeType) + { + for (int c = endScanIndex; c >= startScanIndex; --c) + { + int pos = scan[c]; + Point position = levels.GetPosition(pos); + int coefficientContext = Av1SymbolContextHelper.GetLowerLevelsContext(levels, position, transformSize, transformClass); + int level = this.ReadCoefficientsBase(transformSizeContext, planeType, coefficientContext); + if (level > Av1Constants.BaseLevelsCount) + { + int baseRangeContext = Av1SymbolContextHelper.GetBaseRangeContext(levels, position, transformClass); + this.ReadCoefficientsBaseRangeLoop(transformSizeContext, planeType, baseRangeContext, ref level); + } + + levels.GetRow(position)[position.X] = (byte)level; + } + } + + /// + /// Reads coefficient signs and Golomb extensions, then writes dequantized raster coefficients. + /// + /// The zero-initialized destination receiving dequantized coefficients. + /// The one-based end-of-block position and coefficient count. + /// The transform's scan-to-raster mapping. + /// The decoded absolute-coefficient level plane. + /// The neighboring DC sign context. + /// The luma or chroma plane category. + /// The segment, plane, matrix, scale, and clipping parameters for this transform. + /// The packed coefficient context used by adjacent transform blocks. + private int ReadCoefficientsSign( + Span coefficientBuffer, + int endOfBlock, + ReadOnlySpan scan, + Av1LevelBuffer levels, + int dcSignContext, + Av1PlaneType planeType, + Av1InverseQuantizer.TransformParameters quantization) + { + ref Av1SymbolReader r = ref this.reader; + int culLevel = 0; + int dcValue = 0; + for (int c = 0; c < endOfBlock; c++) + { + int sign = 0; + int pos = scan[c]; + Point position = levels.GetPosition(pos); + int level = levels[position]; + if (level != 0) + { + if (c == 0) + { + sign = this.ReadDcSign(planeType, dcSignContext); + } + else + { + sign = r.ReadLiteral(1); + } + + if (level >= Av1Constants.CoefficientBaseRange + Av1Constants.BaseLevelsCount + 1) + { + level += this.ReadGolomb(); + } + + if (c == 0) + { + dcValue = sign != 0 ? -level : level; + } + + level &= 0xfffff; + culLevel += level; + + // The entropy context uses the masked quantized magnitude, while reconstruction consumes the + // dequantized raster coefficient. Write it directly into the current superblock's zeroed region. + coefficientBuffer[pos] = quantization.Dequantize(level, pos, sign != 0); + } + } + + culLevel = Math.Min(Av1Constants.CoefficientContextMask, culLevel); + Av1SymbolContextHelper.SetDcSign(ref culLevel, dcValue); + + return culLevel; + } + + /// + /// Reads the end-of-block token for a transform coefficient-count category. + /// + /// The luma or chroma plane category. + /// The transform direction class. + /// The signaled transform size. + /// The one-based end-of-block token. + private int ReadEndOfBlockFlag(Av1PlaneType planeType, Av1TransformClass transformClass, Av1TransformSize transformSize) + { + int endOfBlockContext = transformClass == Av1TransformClass.Class2D ? 0 : 1; + int endOfBlockMultiSize = transformSize.GetLog2Minus4(); + ref Av1SymbolReader r = ref this.reader; + return r.ReadSymbol(this.context.EndOfBlockFlag[endOfBlockMultiSize][(int)planeType][endOfBlockContext]) + 1; + } + + /// + /// Reads the most significant context-coded bit of an end-of-block suffix. + /// + /// The square transform-size probability context. + /// The luma or chroma plane category. + /// The token-aligned extra-bit context in the padded local table. + /// The decoded suffix bit. + private bool ReadEndOfBlockExtra(Av1TransformSize transformSizeContext, Av1PlaneType planeType, int endOfBlockContext) + { + ref Av1SymbolReader r = ref this.reader; + return r.ReadSymbol(this.context.EndOfBlockExtra[(int)transformSizeContext][(int)planeType][endOfBlockContext]) > 0; + } + + /// + /// Reads the sign of a nonzero DC coefficient. + /// + /// The luma or chroma plane category. + /// The neighboring DC sign context. + /// Zero for positive or one for negative. + private int ReadDcSign(Av1PlaneType planeType, int dcSignContext) + { + ref Av1SymbolReader r = ref this.reader; + return r.ReadSymbol(this.context.DcSign[(int)planeType][dcSignContext]); + } + + /// + /// Reads the base-level symbol for the final nonzero coefficient. + /// + /// The square transform-size probability context. + /// The luma or chroma plane category. + /// The end-of-block coefficient context. + /// The zero-based base-level symbol. + private int ReadBaseEndOfBlock(Av1TransformSize transformSizeContext, Av1PlaneType planeType, int coefficientContext) + { + ref Av1SymbolReader r = ref this.reader; + return r.ReadSymbol(this.context.BaseEndOfBlock[(int)transformSizeContext][(int)planeType][coefficientContext]); + } + + /// + /// Reads the base-level symbol for a coefficient preceding end-of-block. + /// + /// The square transform-size probability context. + /// The luma or chroma plane category. + /// The nonzero-map coefficient context. + /// The decoded base-level symbol. + private int ReadCoefficientsBase(Av1TransformSize transformSizeContext, Av1PlaneType planeType, int coefficientContext) + { + ref Av1SymbolReader r = ref this.reader; + return r.ReadSymbol(this.context.CoefficientsBase[(int)transformSizeContext][(int)planeType][coefficientContext]); + } + + /// + /// Accumulates coefficient base-range symbols until the terminal symbol or AV1 range limit is reached. + /// + /// The square transform-size probability context. + /// The luma or chroma plane category. + /// The coefficient base-range context. + /// The coefficient level to increment. + private void ReadCoefficientsBaseRangeLoop(Av1TransformSize transformSizeContext, Av1PlaneType planeType, int baseRangeContext, ref int level) + { + ref Av1SymbolReader r = ref this.reader; + Av1TransformSize limitedTransformSizeContext = (Av1TransformSize)Math.Min((int)transformSizeContext, (int)Av1TransformSize.Size32x32); + Av1Distribution distribution = this.context.CoefficientsBaseRange[(int)limitedTransformSizeContext][(int)planeType][baseRangeContext]; + for (int idx = 0; idx < Av1Constants.CoefficientBaseRange; idx += Av1Constants.BaseRangeSizeMinus1) + { + int coefficientBaseRange = r.ReadSymbol(distribution); + level += coefficientBaseRange; + if (coefficientBaseRange < Av1Constants.BaseRangeSizeMinus1) + { + break; + } + } + } + + /// + /// Reads the unsigned exponential-Golomb suffix used for coefficient levels beyond the base range. + /// + /// The decoded nonnegative suffix value. + /// The unary prefix exceeds the AV1 coefficient limit. + public int ReadGolomb() + { + ref Av1SymbolReader r = ref this.reader; + int x = 1; + int length = 0; + int i = 0; + + while (i == 0) + { + i = r.ReadLiteral(1); + ++length; + if (length > 20) + { + throw new InvalidImageContentException("The AV1 coefficient Golomb code exceeds its 20-bit limit."); + } + } + + for (i = 0; i < length - 1; ++i) + { + x <<= 1; + x += r.ReadLiteral(1); + } + + return x - 1; + } + + /// + /// Stores a transform block's packed coefficient context into the above and left neighbor arrays. + /// + /// The above contexts for the current plane. + /// The left contexts for the current plane. + /// The available plane width in four-sample units. + /// The available plane height in four-sample units. + /// The signaled transform size. + /// The transform-block offset within the coding block in four-sample units. + /// The first tile-relative above context covered by the transform. + /// The first superblock-row-relative left context covered by the transform. + /// The packed coefficient magnitude and DC sign context. + /// The signed distance from the mode block to the right frame edge. + /// The signed distance from the mode block to the bottom frame edge. + private static void UpdateCoefficientContext( + Span aboveContexts, + Span leftContexts, + int blocksWide, + int blocksHigh, + Av1TransformSize transformSize, + Point blockPosition, + int aboveOffset, + int leftOffset, + int culLevel, + int modeBlockToRightEdge, + int modeBlockToBottomEdge) + { + int transformSizeWide = transformSize.Get4x4WideCount(); + int transformSizeHigh = transformSize.Get4x4HighCount(); + + if (modeBlockToRightEdge < 0) + { + int aboveContextCount = Math.Min(transformSizeWide, blocksWide - blockPosition.X); + aboveContexts.Slice(aboveOffset, aboveContextCount).Fill(culLevel); + aboveContexts.Slice(aboveOffset + aboveContextCount, transformSizeWide - aboveContextCount).Clear(); + } + else + { + aboveContexts.Slice(aboveOffset, transformSizeWide).Fill(culLevel); + } + + if (modeBlockToBottomEdge < 0) + { + int leftContextCount = Math.Min(transformSizeHigh, blocksHigh - blockPosition.Y); + leftContexts.Slice(leftOffset, leftContextCount).Fill(culLevel); + leftContexts.Slice(leftOffset + leftContextCount, transformSizeHigh - leftContextCount).Clear(); + } + else + { + leftContexts.Slice(leftOffset, transformSizeHigh).Fill(culLevel); + } + } + + /// + /// Resolves the transform type permitted for a plane after lossless, size, prediction, and transform-set restrictions. + /// + /// The luma or chroma plane category. + /// The current block prediction modes. + /// Indicates whether the active segment is lossless. + /// The signaled transform size. + /// The luma transform type shared by inter-predicted chroma. + /// The transform descriptor containing the signaled luma type. + /// Indicates whether the frame restricts transform choices. + /// Indicates whether prediction selects the inter transform set. + /// The transform type valid for the current plane. + private static Av1TransformType ComputeTransformType( + Av1PlaneType planeType, + Av1BlockModeInfo modeInfo, + bool isLossless, + Av1TransformSize transformSize, + Av1TransformType lumaTransformType, + in Av1TransformInfo transformInfo, + bool useReducedTransformSet, + bool usesInterTransformSet) + { + Av1TransformType transformType = Av1TransformType.DctDct; + if (isLossless || transformSize.GetSquareUpSize() > Av1TransformSize.Size32x32) + { + transformType = Av1TransformType.DctDct; + } + else + { + if (planeType == Av1PlaneType.Y) + { + transformType = transformInfo.Type; + } + else if (usesInterTransformSet) + { + // Inter prediction, including intra-block copy, shares the luma transform type with chroma at the + // corresponding luma-grid position rather than deriving an implicit type from the chroma mode. + transformType = lumaTransformType; + } + else + { + // Chroma has its own intra mode, so its implicit transform must be derived independently of luma. + transformType = Av1SymbolContextHelper.GetDefaultIntraTransformType( + modeInfo.UvMode.ToLumaMode(), + transformSize, + useReducedTransformSet); + } + } + + Av1TransformSetType transformSetType = Av1SymbolContextHelper.GetExtendedTransformSetType( + transformSize, + usesInterTransformSet, + useReducedTransformSet); + + if (!transformType.IsExtendedSetUsed(transformSetType)) + { + transformType = Av1TransformType.DctDct; + } + + return transformType; + } + + /// + /// Collapses a full partition distribution into the split-versus-horizontal boundary decision. + /// + /// The full partition distributions. + /// The current block size. + /// The partition probability context. + /// The Q15 probability of the split outcome. + public static uint GetSplitOrHorizontalFrequency(Av1Distribution[] inputs, Av1BlockSize blockSize, int context) + { + Av1Distribution input = inputs[context]; + + // At the bottom edge, AV1 gathers every vertical-like partition mass into the split branch of the + // temporary binary CDF. Reading the frequency directly avoids allocating an adaptive distribution. + uint frequency = GetElementProbability(input, Av1PartitionType.Vertical); + frequency += GetElementProbability(input, Av1PartitionType.Split); + frequency += GetElementProbability(input, Av1PartitionType.HorizontalA); + frequency += GetElementProbability(input, Av1PartitionType.VerticalA); + frequency += GetElementProbability(input, Av1PartitionType.VerticalB); + if (blockSize != Av1BlockSize.Block128x128) + { + frequency += GetElementProbability(input, Av1PartitionType.Vertical4); + } + + return frequency; + } + + /// + /// Collapses a full partition distribution into the split-versus-vertical boundary decision. + /// + /// The full partition distributions. + /// The current block size. + /// The partition probability context. + /// The Q15 probability of the split outcome. + public static uint GetSplitOrVerticalFrequency(Av1Distribution[] inputs, Av1BlockSize blockSize, int context) + { + Av1Distribution input = inputs[context]; + + // At the right edge, AV1 gathers every horizontal-like partition mass into the split branch of the + // temporary binary CDF. Reading the frequency directly avoids allocating an adaptive distribution. + uint frequency = GetElementProbability(input, Av1PartitionType.Horizontal); + frequency += GetElementProbability(input, Av1PartitionType.Split); + frequency += GetElementProbability(input, Av1PartitionType.HorizontalA); + frequency += GetElementProbability(input, Av1PartitionType.HorizontalB); + frequency += GetElementProbability(input, Av1PartitionType.VerticalA); + if (blockSize != Av1BlockSize.Block128x128) + { + frequency += GetElementProbability(input, Av1PartitionType.Horizontal4); + } + + return frequency; + } + + /// + /// Gets one symbol's probability mass from adjacent inverse-CDF thresholds. + /// + /// The inverse cumulative distribution. + /// The partition symbol. + /// The symbol's probability mass. + private static uint GetElementProbability(Av1Distribution probability, Av1PartitionType element) + => probability[(int)element - 1] - probability[(int)element]; +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Entropy/Av1SymbolEncoder.cs b/src/ImageSharp/Formats/Heif/Av1/Entropy/Av1SymbolEncoder.cs new file mode 100644 index 0000000000..1720c5898e --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Entropy/Av1SymbolEncoder.cs @@ -0,0 +1,2427 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Buffers; +using SixLabors.ImageSharp.Formats.Heif.Av1; +using SixLabors.ImageSharp.Formats.Heif.Av1.Motion; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.ChromaFromLuma; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; +using SixLabors.ImageSharp.Memory; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Entropy; + +/// +/// Encodes AV1 tile syntax elements and transform coefficients with tile-local adaptive distributions. +/// +internal sealed class Av1SymbolEncoder : IDisposable +{ + /// + /// The largest coefficient-context plane required after AV1 removes the uncoded half of 64-point transforms. + /// + private const int MaximumCoefficientContextCount = (Av1Constants.MaxTransformSize / 2) * (Av1Constants.MaxTransformSize / 2); + + /// + /// Owns every mutable tile distribution and restores normative defaults without rebuilding the object graph. + /// + private readonly Av1FrameEntropyContext entropyContext; + + /// + /// The tile-adaptive intra-block-copy distribution. + /// + private readonly Av1Distribution tileIntraBlockCopy; + + /// + /// The tile-adaptive integer displacement-vector context. + /// + private readonly Av1MotionVectorContext displacementVector; + + /// + /// The tile-adaptive normal inter motion-vector context. + /// + private readonly Av1MotionVectorContext motionVector; + + /// + /// The tile-adaptive NEWMV branch distributions. + /// + private readonly Av1Distribution[] newMotionVector; + + /// + /// The tile-adaptive GLOBALMV branch distributions. + /// + private readonly Av1Distribution[] zeroMotionVector; + + /// + /// The tile-adaptive NEARESTMV branch distributions. + /// + private readonly Av1Distribution[] referenceMotionVector; + + /// + /// The tile-adaptive dynamic-reference-list distributions. + /// + private readonly Av1Distribution[] dynamicReferenceList; + + /// + /// The tile-adaptive partition-type distributions. + /// + private readonly Av1Distribution[] tilePartitionTypes; + + /// + /// The tile-adaptive key-frame luma-mode distributions. + /// + private readonly Av1Distribution[][] keyFrameYMode; + + /// + /// The tile-adaptive inter-frame intra luma-mode distributions. + /// + private readonly Av1Distribution[] frameYMode; + + /// + /// The tile-adaptive intra-versus-inter distributions. + /// + private readonly Av1Distribution[] intraInter; + + /// + /// The tile-adaptive single-reference branch distributions. + /// + private readonly Av1Distribution[][] singleReference; + + /// + /// The tile-adaptive chroma intra-mode distributions. + /// + private readonly Av1Distribution[][] uvMode; + + /// + /// The tile-adaptive transform-block skip distributions selected for the frame base quantizer. + /// + private readonly Av1Distribution[][] transformBlockSkip; + + /// + /// The tile-adaptive end-of-block token distributions selected for the frame base quantizer. + /// + private readonly Av1Distribution[][][] endOfBlockFlag; + + /// + /// The tile-adaptive coefficient base-range distributions selected for the frame base quantizer. + /// + private readonly Av1Distribution[][][] coefficientsBaseRange; + + /// + /// The tile-adaptive coefficient base-level distributions selected for the frame base quantizer. + /// + private readonly Av1Distribution[][][] coefficientsBase; + + /// + /// The tile-adaptive final-nonzero coefficient distributions selected for the frame base quantizer. + /// + private readonly Av1Distribution[][][] coefficientsBaseEndOfBlock; + + /// + /// The tile-adaptive filter-intra enable distributions. + /// + private readonly Av1Distribution[] filterIntra; + + /// + /// The tile-adaptive filter-intra mode distribution. + /// + private readonly Av1Distribution filterIntraMode; + + /// + /// The tile-adaptive absolute quantizer delta distribution. + /// + private readonly Av1Distribution deltaQuantizerAbsolute; + + /// + /// The tile-adaptive DC sign distributions selected for the frame base quantizer. + /// + private readonly Av1Distribution[][] dcSign; + + /// + /// The tile-adaptive end-of-block extra-bit distributions selected for the frame base quantizer. + /// + private readonly Av1Distribution[][][] endOfBlockExtra; + + /// + /// The tile-adaptive intra transform-type distributions. + /// + private readonly Av1Distribution[][][] intraExtendedTransform; + + /// + /// The tile-adaptive inter transform-type distributions used by intra-block copy. + /// + private readonly Av1Distribution[][] interExtendedTransform; + + /// + /// The tile-adaptive fixed transform-size distributions. + /// + private readonly Av1Distribution[][] transformSize; + + /// + /// The tile-adaptive variable-transform partition distributions. + /// + private readonly Av1Distribution[] transformPartition; + + /// + /// The tile-adaptive spatial segment-identifier distributions. + /// + private readonly Av1Distribution[] segmentId; + + /// + /// The tile-adaptive directional angle-delta distributions. + /// + private readonly Av1Distribution[] angleDelta; + + /// + /// The tile-adaptive transform-skip distributions. + /// + private readonly Av1Distribution[] skip; + + /// + /// The tile-adaptive skip-mode distributions. + /// + private readonly Av1Distribution[] skipMode; + + /// + /// The tile-adaptive joint chroma-from-luma sign distribution. + /// + private readonly Av1Distribution chromaFromLumaSign; + + /// + /// The tile-adaptive chroma-from-luma alpha-magnitude distributions. + /// + private readonly Av1Distribution[] chromaFromLumaAlpha; + + /// + /// Indicates whether the range writer has been disposed. + /// + private bool isDisposed; + + /// + /// The reusable padded coefficient levels used to derive entropy contexts. + /// + private readonly Av1LevelBuffer levels; + + /// + /// The reusable raster-order coefficient contexts for one transform. + /// + private readonly IMemoryOwner coefficientContexts; + + /// + /// The range writer producing the current tile payload. + /// + private Av1SymbolWriter writer; + + /// + /// The frame base quantizer used to select coefficient probability models. + /// + private readonly int baseQIndex; + + /// + /// Initializes a new instance of the class with reusable tile state. + /// + /// The configuration providing output and temporary memory. + /// The initial output capacity in bytes. + /// The frame base quantizer index. + /// A value indicating whether encoded symbols adapt their tile distributions. + public Av1SymbolEncoder(Configuration configuration, int bufferLength, int qIndex, bool updateCdf) + { + this.entropyContext = new Av1FrameEntropyContext(qIndex); + + // Encoder and decoder now share the same mutable context shape. Every field aliases that single graph so + // sequence samples can restore normative defaults without replacing any distribution or array. + this.tileIntraBlockCopy = this.entropyContext.IntraBlockCopy; + this.motionVector = this.entropyContext.MotionVector; + this.displacementVector = this.entropyContext.DisplacementVector; + this.tilePartitionTypes = this.entropyContext.PartitionTypes; + this.keyFrameYMode = this.entropyContext.KeyFrameYMode; + this.frameYMode = this.entropyContext.FrameYMode; + this.intraInter = this.entropyContext.IntraInter; + this.singleReference = this.entropyContext.SingleReference; + this.newMotionVector = this.entropyContext.NewMv; + this.zeroMotionVector = this.entropyContext.ZeroMv; + this.referenceMotionVector = this.entropyContext.RefMv; + this.dynamicReferenceList = this.entropyContext.Drl; + this.uvMode = this.entropyContext.UvMode; + this.filterIntra = this.entropyContext.FilterIntra; + this.filterIntraMode = this.entropyContext.FilterIntraMode; + this.deltaQuantizerAbsolute = this.entropyContext.DeltaQuantizerAbsolute; + this.intraExtendedTransform = this.entropyContext.IntraExtendedTransform; + this.interExtendedTransform = this.entropyContext.InterExtendedTransform; + this.transformSize = this.entropyContext.TransformSize; + this.transformPartition = this.entropyContext.TransformPartition; + this.segmentId = this.entropyContext.SegmentId; + this.angleDelta = this.entropyContext.AngleDelta; + this.skip = this.entropyContext.Skip; + this.skipMode = this.entropyContext.SkipMode; + this.chromaFromLumaSign = this.entropyContext.ChromaFromLumaSign; + this.chromaFromLumaAlpha = this.entropyContext.ChromaFromLumaAlpha; + this.transformBlockSkip = this.entropyContext.TransformBlockSkip; + this.endOfBlockFlag = this.entropyContext.EndOfBlockFlag; + this.coefficientsBaseRange = this.entropyContext.CoefficientsBaseRange; + this.coefficientsBase = this.entropyContext.CoefficientsBase; + this.coefficientsBaseEndOfBlock = this.entropyContext.BaseEndOfBlock; + this.dcSign = this.entropyContext.DcSign; + this.endOfBlockExtra = this.entropyContext.EndOfBlockExtra; + + // Transform dimensions are bounded by the AV1 coefficient-coding rules, so the complete entropy scratch + // is known with the tile output capacity and remains valid for every transform in every sequence sample. + this.levels = new Av1LevelBuffer(configuration); + try + { + this.coefficientContexts = + configuration.MemoryAllocator.Allocate(MaximumCoefficientContextCount); + + this.writer = new(configuration, bufferLength, updateCdf); + this.baseQIndex = qIndex; + } + catch + { + // The level buffer is already owned here; a later allocation failure cannot be unwound by the caller. + this.coefficientContexts?.Dispose(); + this.levels.Dispose(); + throw; + } + } + + /// + /// Defines how shared coefficient-syntax helpers handle one adaptive symbol or literal bit field. + /// + private interface ICoefficientSymbolOperation + { + /// + /// Handles one symbol from an adaptive distribution. + /// + /// The tile range writer. + /// The zero-based symbol. + /// The symbol distribution. + /// The symbol's rate contribution. + public static abstract int ProcessSymbol( + ref Av1SymbolWriter writer, + int symbol, + Av1Distribution distribution); + + /// + /// Handles one most-significant-bit-first literal field. + /// + /// The tile range writer. + /// The low-order literal bits. + /// The number of bits. + /// The literal's rate contribution. + public static abstract int ProcessLiteral( + ref Av1SymbolWriter writer, + uint value, + int bitCount); + } + + /// + /// Defines how the shared palette-map traversal handles its uniform first index and adaptive remaining indices. + /// + private interface IPaletteColorMapOperation + { + /// + /// Handles the first uniformly coded palette index. + /// + /// The tile symbol encoder. + /// The number of colors in the palette. + /// The first palette index. + /// The index's rate contribution. + public static abstract int ProcessFirstIndex( + Av1SymbolEncoder encoder, + int paletteSize, + int colorIndex); + + /// + /// Handles one context-adaptive palette color-order index. + /// + /// The tile symbol encoder. + /// The number of colors in the palette. + /// The luma or chroma plane class. + /// The spatial color-index context. + /// The index in the context-specific color order. + /// The index's rate contribution. + public static abstract int ProcessColorIndex( + Av1SymbolEncoder encoder, + int paletteSize, + Av1PlaneType planeType, + int colorContext, + int colorOrderIndex); + } + + /// + /// Restores the initial tile distributions and range coder while retaining their complete object graph and buffers. + /// + public void Reset() + { + this.entropyContext.ResetToDefaults(this.baseQIndex); + this.writer.Reset(); + } + + /// + /// Restores the initial tile distributions and begins the next tile at an offset in the retained output buffer. + /// + /// The first output byte available to the next tile. + public void Reset(int outputOffset) + { + this.entropyContext.ResetToDefaults(this.baseQIndex); + this.writer.Reset(outputOffset); + } + + /// + /// Writes an unsigned fixed-width literal to the tile entropy stream. + /// + /// The low-order literal bits. + /// The number of bits to write. + public void WriteLiteral(uint value, int bitCount) + { + ref Av1SymbolWriter w = ref this.writer; + w.WriteLiteral(value, bitCount); + } + + /// + /// Writes a uniformly coded value from a non-power-of-two alphabet. + /// + /// The number of possible values. + /// The value in the range from zero through minus one. + public void WriteUniform(int valueCount, int value) + { + ref Av1SymbolWriter w = ref this.writer; + int bitCount = Av1Math.Log2(valueCount) + 1; + int threshold = (1 << bitCount) - valueCount; + if (value < threshold) + { + // The lower values use the short prefix; every remaining value carries one final disambiguating bit. + w.WriteLiteral((uint)value, bitCount - 1); + return; + } + + int offset = value - threshold; + w.WriteLiteral((uint)(threshold + (offset >> 1)), bitCount - 1); + w.WriteLiteral((uint)(offset & 1), 1); + } + + /// + /// Gets the fixed-point rate of a uniformly coded value. + /// + /// The number of possible values. + /// The value in the range from zero through minus one. + /// The rate cost in 1/512-bit units. + public static int GetUniformCost(int valueCount, int value) + { + int bitCount = Av1Math.Log2(valueCount) + 1; + int threshold = (1 << bitCount) - valueCount; + return Av1ProbabilityCost.GetLiteralCost(value < threshold ? bitCount - 1 : bitCount); + } + + /// + /// Gets the current fixed-point cost of the luma palette-mode flag. + /// + /// Indicates whether the block uses luma palette prediction. + /// The block-area context in the range from zero through six. + /// The number of available above and left luma neighbors that use palettes. + /// The rate cost in 1/512-bit units. + public int GetPaletteYModeCost(bool usePalette, int blockSizeContext, int neighborContext) + { + return Av1ProbabilityCost.GetSymbolCost( + this.entropyContext.PaletteYMode[blockSizeContext][neighborContext], + usePalette ? 1 : 0); + } + + /// + /// Writes the luma palette-mode flag. + /// + /// Indicates whether the block uses luma palette prediction. + /// The block-area context in the range from zero through six. + /// The number of available above and left luma neighbors that use palettes. + public void WritePaletteYMode(bool usePalette, int blockSizeContext, int neighborContext) + { + ref Av1SymbolWriter w = ref this.writer; + w.WriteSymbol(usePalette, this.entropyContext.PaletteYMode[blockSizeContext][neighborContext]); + } + + /// + /// Gets the current fixed-point cost of the chroma palette-mode flag. + /// + /// Indicates whether the block uses chroma palette prediction. + /// Indicates whether the current block uses a luma palette. + /// The rate cost in 1/512-bit units. + public int GetPaletteUvModeCost(bool usePalette, bool hasLumaPalette) + { + return Av1ProbabilityCost.GetSymbolCost( + this.entropyContext.PaletteUvMode[hasLumaPalette ? 1 : 0], + usePalette ? 1 : 0); + } + + /// + /// Writes the chroma palette-mode flag. + /// + /// Indicates whether the block uses chroma palette prediction. + /// Indicates whether the current block uses a luma palette. + public void WritePaletteUvMode(bool usePalette, bool hasLumaPalette) + { + ref Av1SymbolWriter w = ref this.writer; + w.WriteSymbol(usePalette, this.entropyContext.PaletteUvMode[hasLumaPalette ? 1 : 0]); + } + + /// + /// Gets the current fixed-point cost of a palette-size symbol. + /// + /// The palette size in the range from two through eight. + /// The block-area context in the range from zero through six. + /// The luma or chroma plane class. + /// The rate cost in 1/512-bit units. + public int GetPaletteSizeCost(int paletteSize, int blockSizeContext, Av1PlaneType planeType) + { + Av1Distribution distribution = planeType == Av1PlaneType.Y + ? this.entropyContext.PaletteYSize[blockSizeContext] + : this.entropyContext.PaletteUvSize[blockSizeContext]; + + return Av1ProbabilityCost.GetSymbolCost(distribution, paletteSize - 2); + } + + /// + /// Writes a palette-size symbol. + /// + /// The palette size in the range from two through eight. + /// The block-area context in the range from zero through six. + /// The luma or chroma plane class. + public void WritePaletteSize(int paletteSize, int blockSizeContext, Av1PlaneType planeType) + { + ref Av1SymbolWriter w = ref this.writer; + Av1Distribution distribution = planeType == Av1PlaneType.Y + ? this.entropyContext.PaletteYSize[blockSizeContext] + : this.entropyContext.PaletteUvSize[blockSizeContext]; + + w.WriteSymbol(paletteSize - 2, distribution); + } + + /// + /// Gets the current fixed-point cost of a palette color-order index. + /// + /// The index in the context-specific palette color order. + /// The number of colors in the palette. + /// The color-index context derived from preceding spatial indices. + /// The luma or chroma plane class. + /// The rate cost in 1/512-bit units. + public int GetPaletteColorIndexCost( + int colorOrderIndex, + int paletteSize, + int colorContext, + Av1PlaneType planeType) + { + Av1Distribution distribution = planeType == Av1PlaneType.Y + ? this.entropyContext.PaletteYColorIndex[paletteSize - 2][colorContext] + : this.entropyContext.PaletteUvColorIndex[paletteSize - 2][colorContext]; + + return Av1ProbabilityCost.GetSymbolCost(distribution, colorOrderIndex); + } + + /// + /// Writes a palette color-order index. + /// + /// The index in the context-specific palette color order. + /// The number of colors in the palette. + /// The color-index context derived from preceding spatial indices. + /// The luma or chroma plane class. + public void WritePaletteColorIndex( + int colorOrderIndex, + int paletteSize, + int colorContext, + Av1PlaneType planeType) + { + ref Av1SymbolWriter w = ref this.writer; + Av1Distribution distribution = planeType == Av1PlaneType.Y + ? this.entropyContext.PaletteYColorIndex[paletteSize - 2][colorContext] + : this.entropyContext.PaletteUvColorIndex[paletteSize - 2][colorContext]; + + w.WriteSymbol(colorOrderIndex, distribution); + } + + /// + /// Gets the fixed-point rate of the luma palette colors. + /// + /// The sorted unique colors inherited from eligible neighbors. + /// The sorted luma palette colors. + /// The number of bits in each color sample. + /// The rate cost in 1/512-bit units. + public static int GetPaletteYColorCost( + ReadOnlySpan colorCache, + ReadOnlySpan colors, + int bitDepth) + { + Span cacheColorFound = stackalloc byte[Av1Constants.PaletteMaxSize * 2]; + Span uncachedColors = stackalloc ushort[Av1Constants.PaletteMaxSize]; + int uncachedColorCount = IndexColorCache( + colorCache, + colors, + cacheColorFound, + uncachedColors); + + // Palette RD modeling charges every available cache flag even though emission can stop once all colors match. + int bitCount = colorCache.Length + + GetDeltaEncodedColorBitCount(uncachedColors[..uncachedColorCount], bitDepth, minimumDelta: 1); + + return Av1ProbabilityCost.GetLiteralCost(bitCount); + } + + /// + /// Writes the luma palette colors using neighboring cache selections followed by sorted deltas. + /// + /// The sorted unique colors inherited from eligible neighbors. + /// The sorted luma palette colors. + /// The number of bits in each color sample. + public void WritePaletteYColors( + ReadOnlySpan colorCache, + ReadOnlySpan colors, + int bitDepth) + { + Span cacheColorFound = stackalloc byte[Av1Constants.PaletteMaxSize * 2]; + Span uncachedColors = stackalloc ushort[Av1Constants.PaletteMaxSize]; + int uncachedColorCount = IndexColorCache( + colorCache, + colors, + cacheColorFound, + uncachedColors); + + int cachedColorCount = 0; + for (int i = 0; i < colorCache.Length && cachedColorCount < colors.Length; i++) + { + byte found = cacheColorFound[i]; + this.WriteLiteral(found, 1); + cachedColorCount += found; + } + + this.WriteDeltaEncodedColors(uncachedColors[..uncachedColorCount], bitDepth, minimumDelta: 1); + } + + /// + /// Gets the fixed-point rate of the shared chroma palette colors. + /// + /// The sorted unique U colors inherited from eligible neighbors. + /// The sorted U palette colors. + /// The V palette colors paired with . + /// The number of bits in each color sample. + /// The rate cost in 1/512-bit units. + public static int GetPaletteUvColorCost( + ReadOnlySpan colorCache, + ReadOnlySpan uColors, + ReadOnlySpan vColors, + int bitDepth) + { + Span cacheColorFound = stackalloc byte[Av1Constants.PaletteMaxSize * 2]; + Span uncachedColors = stackalloc ushort[Av1Constants.PaletteMaxSize]; + int uncachedColorCount = IndexColorCache( + colorCache, + uColors, + cacheColorFound, + uncachedColors); + + // Palette RD modeling charges every available cache flag even though emission can stop once all colors match. + int bitCount = colorCache.Length + + GetDeltaEncodedColorBitCount(uncachedColors[..uncachedColorCount], bitDepth, minimumDelta: 0); + + int deltaBits = GetPaletteVDeltaBitCount(vColors, bitDepth, out int zeroCount, out int minimumBits); + int deltaBitCount = 2 + bitDepth + ((deltaBits + 1) * (vColors.Length - 1)) - zeroCount; + int rawBitCount = bitDepth * vColors.Length; + bitCount += 1 + Math.Min(deltaBitCount, rawBitCount); + return Av1ProbabilityCost.GetLiteralCost(bitCount); + } + + /// + /// Writes the shared chroma palette colors using cached U values and the cheaper V representation. + /// + /// The sorted unique U colors inherited from eligible neighbors. + /// The sorted U palette colors. + /// The V palette colors paired with . + /// The number of bits in each color sample. + public void WritePaletteUvColors( + ReadOnlySpan colorCache, + ReadOnlySpan uColors, + ReadOnlySpan vColors, + int bitDepth) + { + Span cacheColorFound = stackalloc byte[Av1Constants.PaletteMaxSize * 2]; + Span uncachedColors = stackalloc ushort[Av1Constants.PaletteMaxSize]; + int uncachedColorCount = IndexColorCache( + colorCache, + uColors, + cacheColorFound, + uncachedColors); + + int cachedColorCount = 0; + for (int i = 0; i < colorCache.Length && cachedColorCount < uColors.Length; i++) + { + byte found = cacheColorFound[i]; + this.WriteLiteral(found, 1); + cachedColorCount += found; + } + + this.WriteDeltaEncodedColors(uncachedColors[..uncachedColorCount], bitDepth, minimumDelta: 0); + + int deltaBits = GetPaletteVDeltaBitCount(vColors, bitDepth, out int zeroCount, out int minimumBits); + int deltaBitCount = 2 + bitDepth + ((deltaBits + 1) * (vColors.Length - 1)) - zeroCount; + int rawBitCount = bitDepth * vColors.Length; + bool useDelta = deltaBitCount < rawBitCount; + this.WriteLiteral(useDelta ? 1u : 0u, 1); + if (!useDelta) + { + for (int i = 0; i < vColors.Length; i++) + { + this.WriteLiteral(vColors[i], bitDepth); + } + + return; + } + + this.WriteLiteral((uint)(deltaBits - minimumBits), 2); + this.WriteLiteral(vColors[0], bitDepth); + int sampleRange = 1 << bitDepth; + for (int i = 1; i < vColors.Length; i++) + { + int signedDelta = vColors[i] - vColors[i - 1]; + int delta = Math.Abs(signedDelta); + + // Chroma wraps in its unsigned sample domain, so signal whichever circular direction has less magnitude. + if (delta <= sampleRange - delta) + { + this.WriteLiteral((uint)delta, deltaBits); + if (delta != 0) + { + this.WriteLiteral(signedDelta < 0 ? 1u : 0u, 1); + } + } + else + { + this.WriteLiteral((uint)(sampleRange - delta), deltaBits); + this.WriteLiteral(signedDelta < 0 ? 0u : 1u, 1); + } + } + } + + /// + /// Gets the current fixed-point rate of a complete palette color-index map. + /// + /// The number of colors in the palette. + /// The luma or chroma plane class. + /// The number of coded map rows. + /// The number of coded map columns. + /// The complete row-addressable color-index map. + /// The rate cost in 1/512-bit units. + public int GetPaletteColorMapCost( + int paletteSize, + Av1PlaneType planeType, + int rows, + int columns, + Buffer2DRegion colorIndexMap) + => this.ProcessPaletteColorMap( + paletteSize, + planeType, + rows, + columns, + colorIndexMap); + + /// + /// Writes a complete palette color-index map in AV1 diagonal wavefront order. + /// + /// The number of colors in the palette. + /// The luma or chroma plane class. + /// The number of coded map rows. + /// The number of coded map columns. + /// The complete row-addressable color-index map. + public void WritePaletteColorMap( + int paletteSize, + Av1PlaneType planeType, + int rows, + int columns, + Buffer2DRegion colorIndexMap) + { + _ = this.ProcessPaletteColorMap( + paletteSize, + planeType, + rows, + columns, + colorIndexMap); + } + + /// + /// Writes the frame-local intra-block-copy flag. + /// + /// Indicates whether intra-block copy is selected. + public void WriteUseIntraBlockCopy(bool value) + { + ref Av1SymbolWriter w = ref this.writer; + w.WriteSymbol(value, this.tileIntraBlockCopy); + } + + /// + /// Measures the frame-local intra-block-copy flag against the live distribution. + /// + /// Indicates whether intra-block copy is selected. + /// The syntax cost in 1/512-bit units. + public int GetUseIntraBlockCopyCost(bool value) + => Av1ProbabilityCost.GetSymbolCost(this.tileIntraBlockCopy, value ? 1 : 0); + + /// + /// Writes an integer intra-block-copy displacement vector relative to a spatial reference. + /// + /// The displacement vector to encode. + /// The spatially derived reference vector. + public void WriteDisplacementVector(Av1MotionVector value, Av1MotionVector reference) + => this.displacementVector.Write(this.writer, value, reference, Av1MotionVectorPrecision.Integer); + + /// + /// Measures an integer intra-block-copy displacement vector against the live distributions. + /// + /// The displacement vector to measure. + /// The spatially derived reference vector. + /// The discounted syntax cost in 1/512-bit units. + public int GetDisplacementVectorCost(Av1MotionVector value, Av1MotionVector reference) + { + const int DisplacementVectorCostWeight = 120; + const int WeightShift = 7; + int rate = this.displacementVector.GetCost( + this.writer, + value, + reference, + Av1MotionVectorPrecision.Integer); + + // Displacement syntax uses a 120/128 discount during mode search; adding half the divisor rounds to nearest. + return ((rate * DisplacementVectorCostWeight) + (1 << (WeightShift - 1))) >> WeightShift; + } + + /// + /// Measures an integer intra-block-copy displacement vector for variance-domain motion search. + /// + /// The displacement vector to measure. + /// The spatially derived reference vector. + /// The syntax cost in 1/512-bit units. + public int GetDisplacementVectorSearchCost(Av1MotionVector value, Av1MotionVector reference) + => this.displacementVector.GetCost( + this.writer, + value, + reference, + Av1MotionVectorPrecision.Integer); + + /// + /// Measures one switchable interpolation filter against its live tile distribution. + /// + /// The regular, smooth, or sharp filter. + /// The spatial filter context for the selected direction. + /// The syntax cost in 1/512-bit units. + public int GetSwitchableInterpolationFilterCost(Av1InterpolationFilter filter, int context) + => Av1ProbabilityCost.GetSymbolCost(this.entropyContext.SwitchableInterpolation[context], (int)filter); + + /// + /// Writes one switchable interpolation filter and updates its live tile distribution. + /// + /// The regular, smooth, or sharp filter. + /// The spatial filter context for the selected direction. + public void WriteSwitchableInterpolationFilter(Av1InterpolationFilter filter, int context) + => this.writer.WriteSymbol((int)filter, this.entropyContext.SwitchableInterpolation[context]); + + /// + /// Measures a single-reference inter mode against the live branch distributions. + /// + /// The new, global, nearest, or near motion-vector mode. + /// The packed context derived from the reference-vector stack. + /// The syntax cost in 1/512-bit units. + public int GetInterModeCost(Av1PredictionMode mode, int modeContext) + { + bool isNotNew = mode != Av1PredictionMode.NewMotionVector; + int rate = Av1ProbabilityCost.GetSymbolCost( + this.newMotionVector[Av1SymbolContextHelper.GetNewMvContext(modeContext)], + isNotNew ? 1 : 0); + + if (!isNotNew) + { + return rate; + } + + bool isNotGlobal = mode != Av1PredictionMode.GlobalMotionVector; + rate += Av1ProbabilityCost.GetSymbolCost( + this.zeroMotionVector[Av1SymbolContextHelper.GetZeroMvContext(modeContext)], + isNotGlobal ? 1 : 0); + + if (!isNotGlobal) + { + return rate; + } + + return rate + Av1ProbabilityCost.GetSymbolCost( + this.referenceMotionVector[Av1SymbolContextHelper.GetRefMvContext(modeContext)], + mode == Av1PredictionMode.NearMotionVector ? 1 : 0); + } + + /// + /// Writes a single-reference inter mode through the NEWMV, GLOBALMV, and NEARESTMV branch tree. + /// + /// The new, global, nearest, or near motion-vector mode. + /// The packed context derived from the reference-vector stack. + public void WriteInterMode(Av1PredictionMode mode, int modeContext) + { + ref Av1SymbolWriter w = ref this.writer; + bool isNotNew = mode != Av1PredictionMode.NewMotionVector; + w.WriteSymbol(isNotNew, this.newMotionVector[Av1SymbolContextHelper.GetNewMvContext(modeContext)]); + if (!isNotNew) + { + return; + } + + bool isNotGlobal = mode != Av1PredictionMode.GlobalMotionVector; + w.WriteSymbol(isNotGlobal, this.zeroMotionVector[Av1SymbolContextHelper.GetZeroMvContext(modeContext)]); + if (!isNotGlobal) + { + return; + } + + w.WriteSymbol( + mode == Av1PredictionMode.NearMotionVector, + this.referenceMotionVector[Av1SymbolContextHelper.GetRefMvContext(modeContext)]); + } + + /// + /// Measures one dynamic-reference-list advance decision. + /// + /// Whether selection advances to the next candidate. + /// The candidate-weight context. + /// The syntax cost in 1/512-bit units. + public int GetDynamicReferenceListCost(bool advance, int context) + => Av1ProbabilityCost.GetSymbolCost(this.dynamicReferenceList[context], advance ? 1 : 0); + + /// + /// Writes one dynamic-reference-list advance decision. + /// + /// Whether selection advances to the next candidate. + /// The candidate-weight context. + public void WriteDynamicReferenceList(bool advance, int context) + { + ref Av1SymbolWriter w = ref this.writer; + w.WriteSymbol(advance, this.dynamicReferenceList[context]); + } + + /// + /// Measures an inter motion vector relative to its selected stack reference. + /// + /// The selected motion vector. + /// The differential reference from the candidate stack. + /// The fractional precision selected by the frame header. + /// The syntax cost in 1/512-bit units. + public int GetMotionVectorCost( + Av1MotionVector value, + Av1MotionVector reference, + Av1MotionVectorPrecision precision) + => this.motionVector.GetCost(this.writer, value, reference, precision); + + /// + /// Writes an inter motion vector relative to its selected stack reference. + /// + /// The selected motion vector. + /// The differential reference from the candidate stack. + /// The fractional precision selected by the frame header. + public void WriteMotionVector( + Av1MotionVector value, + Av1MotionVector reference, + Av1MotionVectorPrecision precision) + => this.motionVector.Write(this.writer, value, reference, precision); + + /// + /// Gets the current fixed-point cost of a complete block partition symbol. + /// + /// The partition type to measure. + /// The partition probability context. + /// The rate cost in 1/512-bit units. + public int GetPartitionTypeCost(Av1PartitionType partitionType, int context) + => Av1ProbabilityCost.GetSymbolCost(this.tilePartitionTypes[context], (int)partitionType); + + /// + /// Writes a complete block partition type using the selected partition context. + /// + /// The partition type to encode. + /// The partition probability context. + public void WritePartitionType(Av1PartitionType partitionType, int context) + { + ref Av1SymbolWriter w = ref this.writer; + w.WriteSymbol((int)partitionType, this.tilePartitionTypes[context]); + } + + /// + /// Writes the split-versus-horizontal boundary decision for a block clipped at the bottom tile edge. + /// + /// The split or horizontal partition outcome. + /// The current block size. + /// The partition probability context. + public void WriteSplitOrHorizontal(Av1PartitionType partitionType, Av1BlockSize blockSize, int context) + { + uint frequency = Av1SymbolDecoder.GetSplitOrHorizontalFrequency(this.tilePartitionTypes, blockSize, context); + bool value = partitionType == Av1PartitionType.Split; + ref Av1SymbolWriter w = ref this.writer; + w.WriteBoolean(value, frequency); + } + + /// + /// Gets the current fixed-point cost of the split-versus-horizontal boundary decision. + /// + /// The split or horizontal partition outcome. + /// The current block size. + /// The partition probability context. + /// The rate cost in 1/512-bit units. + public int GetSplitOrHorizontalCost(Av1PartitionType partitionType, Av1BlockSize blockSize, int context) + { + int frequency = (int)Av1SymbolDecoder.GetSplitOrHorizontalFrequency( + this.tilePartitionTypes, + blockSize, + context); + + return Av1ProbabilityCost.GetSymbolCost( + partitionType == Av1PartitionType.Split + ? frequency + : Av1Distribution.ProbabilityTop - frequency); + } + + /// + /// Writes the split-versus-vertical boundary decision for a block clipped at the right tile edge. + /// + /// The split or vertical partition outcome. + /// The current block size. + /// The partition probability context. + public void WriteSplitOrVertical(Av1PartitionType partitionType, Av1BlockSize blockSize, int context) + { + uint frequency = Av1SymbolDecoder.GetSplitOrVerticalFrequency(this.tilePartitionTypes, blockSize, context); + bool value = partitionType == Av1PartitionType.Split; + ref Av1SymbolWriter w = ref this.writer; + w.WriteBoolean(value, frequency); + } + + /// + /// Gets the current fixed-point cost of the split-versus-vertical boundary decision. + /// + /// The split or vertical partition outcome. + /// The current block size. + /// The partition probability context. + /// The rate cost in 1/512-bit units. + public int GetSplitOrVerticalCost(Av1PartitionType partitionType, Av1BlockSize blockSize, int context) + { + int frequency = (int)Av1SymbolDecoder.GetSplitOrVerticalFrequency( + this.tilePartitionTypes, + blockSize, + context); + + return Av1ProbabilityCost.GetSymbolCost( + partitionType == Av1PartitionType.Split + ? frequency + : Av1Distribution.ProbabilityTop - frequency); + } + + /// + /// Encodes one transform block's coefficient syntax using scan-order probability contexts. + /// + /// The signaled transform size. + /// The transform type selecting the scan and context class. + /// The block's intra prediction mode. + /// The raster-ordered signed coefficient levels. + /// The luma or chroma component category. + /// The neighboring skip and DC sign contexts. + /// The one-based final nonzero scan position, or zero for an empty block. + /// Indicates whether the frame restricts transform choices. + /// The selected filter-intra mode, or the disabled sentinel. + /// Indicates whether inter rather than intra transform probabilities apply. + /// The packed coefficient context used by adjacent transform blocks. + public int WriteCoefficients( + Av1TransformSize transformSize, + Av1TransformType transformType, + Av1PredictionMode intraDirection, + ReadOnlySpan coefficientBuffer, + Av1ComponentType componentType, + Av1TransformBlockContext transformBlockContext, + ushort endOfBlock, + bool useReducedTransformSet, + Av1FilterIntraMode filterIntraMode, + bool usesInterTransformSet) + { + Av1TransformSize transformSizeContext = Av1SymbolContextHelper.GetTransformSizeContext(transformSize); + + DebugGuard.MustBeLessThan((int)transformSizeContext, (int)Av1TransformSize.AllSizes, nameof(transformSizeContext)); + + _ = this.ProcessTransformBlockSkip( + endOfBlock == 0, + transformSizeContext, + transformBlockContext.SkipContext); + + if (endOfBlock == 0) + { + return 0; + } + + Av1TransformSize adjustedTransformSize = transformSize.GetAdjusted(); + int width = adjustedTransformSize.GetWidth(); + int height = adjustedTransformSize.GetHeight(); + Av1TransformClass transformClass = transformType.ToClass(); + ReadOnlySpan scan = Av1ScanOrderConstants.GetScanOrder(transformSize, transformType).Scan; + Av1LevelBuffer levels = this.PrepareCoefficientScratch( + width, + height, + clearLevels: true, + out Span coefficientContexts); + + levels.Initialize(coefficientBuffer); + if (componentType == Av1ComponentType.Luminance) + { + _ = this.ProcessTransformType( + transformType, + transformSize, + usesInterTransformSet, + useReducedTransformSet, + this.baseQIndex, + filterIntraMode, + intraDirection); + } + + _ = this.ProcessEndOfBlockPosition( + endOfBlock, + componentType, + transformClass, + transformSize, + transformSizeContext); + + Av1SymbolContextHelper.GetNzMapContexts(levels, scan, endOfBlock, transformSize, transformClass, coefficientContexts); + int limitedTransformSizeContext = Math.Min((int)transformSizeContext, (int)Av1TransformSize.Size32x32); + ref Av1SymbolWriter w = ref this.writer; + for (int c = endOfBlock - 1; c >= 0; --c) + { + short pos = scan[c]; + int value = coefficientBuffer[pos]; + short coefficientContext = coefficientContexts[pos]; + Point position = levels.GetPosition(pos); + int level = Math.Abs(value); + + if (c == endOfBlock - 1) + { + w.WriteSymbol( + Math.Min(level, 3) - 1, + this.coefficientsBaseEndOfBlock[(int)transformSizeContext][(int)componentType][coefficientContext]); + } + else + { + w.WriteSymbol( + Math.Min(level, 3), + this.coefficientsBase[(int)transformSizeContext][(int)componentType][coefficientContext]); + } + + if (level > Av1Constants.BaseLevelsCount) + { + // Base-range symbols extend levels above the two base levels in fixed-size chunks. + int baseRange = level - 1 - Av1Constants.BaseLevelsCount; + int baseRangeContext = Av1SymbolContextHelper.GetBaseRangeContext(levels, position, transformClass); + for (int idx = 0; idx < Av1Constants.CoefficientBaseRange; idx += Av1Constants.BaseRangeSizeMinus1) + { + int symbol = Math.Min(baseRange - idx, Av1Constants.BaseRangeSizeMinus1); + w.WriteSymbol( + symbol, + this.coefficientsBaseRange[limitedTransformSizeContext][(int)componentType][baseRangeContext]); + + if (symbol < Av1Constants.BaseRangeSizeMinus1) + { + break; + } + } + } + } + + // Signs follow every magnitude so the DC sign can use its neighboring context and AC signs remain literals. + int culLevel = 0; + for (int c = 0; c < endOfBlock; ++c) + { + short pos = scan[c]; + int value = coefficientBuffer[pos]; + int level = Math.Abs(value); + culLevel += level; + + uint sign = value < 0 ? 1u : 0u; + if (level > 0) + { + if (c == 0) + { + w.WriteSymbol( + (int)sign, + this.dcSign[(int)componentType][transformBlockContext.DcSignContext]); + } + else + { + w.WriteLiteral(sign, 1); + } + + if (level > (Av1Constants.CoefficientBaseRange + Av1Constants.BaseLevelsCount)) + { + this.WriteGolomb( + level - Av1Constants.CoefficientBaseRange - 1 - Av1Constants.BaseLevelsCount); + } + } + } + + culLevel = Math.Min(Av1Constants.CoefficientContextMask, culLevel); + + // The DC sign is packed above the magnitude bits so adjacent blocks can derive both contexts from one value. + Av1SymbolContextHelper.SetDcSign(ref culLevel, coefficientBuffer[0]); + return culLevel; + } + + /// + /// Gets the current fixed-point rate cost of one transform block's complete coefficient syntax. + /// + /// The signaled transform size. + /// The transform type selecting the scan and context class. + /// The block's intra prediction mode. + /// The raster-ordered signed coefficient levels. + /// The luma or chroma component category. + /// The neighboring skip and DC sign contexts. + /// The one-based final nonzero scan position, or zero for an empty block. + /// Indicates whether the frame restricts transform choices. + /// The selected filter-intra mode, or the disabled sentinel. + /// Indicates whether inter rather than intra transform probabilities apply. + /// The rate cost in 1/512-bit units. + public int GetCoefficientCost( + Av1TransformSize transformSize, + Av1TransformType transformType, + Av1PredictionMode intraDirection, + ReadOnlySpan coefficientBuffer, + Av1ComponentType componentType, + Av1TransformBlockContext transformBlockContext, + ushort endOfBlock, + bool useReducedTransformSet, + Av1FilterIntraMode filterIntraMode, + bool usesInterTransformSet) + { + Av1TransformSize transformSizeContext = Av1SymbolContextHelper.GetTransformSizeContext(transformSize); + + DebugGuard.MustBeLessThan((int)transformSizeContext, (int)Av1TransformSize.AllSizes, nameof(transformSizeContext)); + + int rate = this.ProcessTransformBlockSkip( + endOfBlock == 0, + transformSizeContext, + transformBlockContext.SkipContext); + + if (endOfBlock == 0) + { + return rate; + } + + Av1TransformSize adjustedTransformSize = transformSize.GetAdjusted(); + int width = adjustedTransformSize.GetWidth(); + int height = adjustedTransformSize.GetHeight(); + Av1TransformClass transformClass = transformType.ToClass(); + ReadOnlySpan scan = Av1ScanOrderConstants.GetScanOrder(transformSize, transformType).Scan; + bool needsLevelMap = endOfBlock > 1; + Av1LevelBuffer levels = this.PrepareCoefficientScratch( + width, + height, + needsLevelMap, + out Span coefficientContexts); + + // The final coefficient uses scan-position contexts only. Earlier coefficients need the complete + // forward-neighbor level map, so a one-coefficient candidate avoids initializing that plane. + if (needsLevelMap) + { + levels.Initialize(coefficientBuffer); + } + + if (componentType == Av1ComponentType.Luminance) + { + rate += this.GetTransformTypeCost( + transformType, + transformSize, + useReducedTransformSet, + this.baseQIndex, + filterIntraMode, + intraDirection, + usesInterTransformSet); + } + + rate += this.ProcessEndOfBlockPosition( + endOfBlock, + componentType, + transformClass, + transformSize, + transformSizeContext); + + Av1SymbolContextHelper.GetNzMapContexts(levels, scan, endOfBlock, transformSize, transformClass, coefficientContexts); + int limitedTransformSizeContext = Math.Min((int)transformSizeContext, (int)Av1TransformSize.Size32x32); + int c = endOfBlock - 1; + int pos = scan[c]; + int value = coefficientBuffer[pos]; + int level = Math.Abs(value); + int coefficientContext = coefficientContexts[pos]; + rate += Av1ProbabilityCost.GetSymbolCost( + this.coefficientsBaseEndOfBlock[(int)transformSizeContext][(int)componentType][coefficientContext], + Math.Min(level, 3) - 1); + + if (level > Av1Constants.BaseLevelsCount) + { + int baseRangeContext = Av1SymbolContextHelper.GetBaseRangeContextEndOfBlock( + levels.GetPosition(pos), + transformClass); + + rate += GetBaseRangeCost( + level, + this.coefficientsBaseRange[limitedTransformSizeContext][(int)componentType][baseRangeContext]); + } + + if (c == 0) + { + return rate + Av1ProbabilityCost.GetSymbolCost( + this.dcSign[(int)componentType][transformBlockContext.DcSignContext], + value < 0 ? 1 : 0); + } + + rate += Av1ProbabilityCost.GetLiteralCost(1); + for (c = endOfBlock - 2; c >= 1; --c) + { + pos = scan[c]; + value = coefficientBuffer[pos]; + level = Math.Abs(value); + coefficientContext = coefficientContexts[pos]; + rate += Av1ProbabilityCost.GetSymbolCost( + this.coefficientsBase[(int)transformSizeContext][(int)componentType][coefficientContext], + Math.Min(level, 3)); + + if (level == 0) + { + continue; + } + + rate += Av1ProbabilityCost.GetLiteralCost(1); + if (level > Av1Constants.BaseLevelsCount) + { + int baseRangeContext = Av1SymbolContextHelper.GetBaseRangeContext( + levels, + levels.GetPosition(pos), + transformClass); + + rate += GetBaseRangeCost( + level, + this.coefficientsBaseRange[limitedTransformSizeContext][(int)componentType][baseRangeContext]); + } + } + + pos = scan[0]; + value = coefficientBuffer[pos]; + level = Math.Abs(value); + coefficientContext = coefficientContexts[pos]; + rate += Av1ProbabilityCost.GetSymbolCost( + this.coefficientsBase[(int)transformSizeContext][(int)componentType][coefficientContext], + Math.Min(level, 3)); + + if (level > 0) + { + rate += Av1ProbabilityCost.GetSymbolCost( + this.dcSign[(int)componentType][transformBlockContext.DcSignContext], + value < 0 ? 1 : 0); + + if (level > Av1Constants.BaseLevelsCount) + { + int baseRangeContext = Av1SymbolContextHelper.GetBaseRangeContext( + levels, + levels.GetPosition(pos), + transformClass); + + rate += GetBaseRangeCost( + level, + this.coefficientsBaseRange[limitedTransformSizeContext][(int)componentType][baseRangeContext]); + } + } + + return rate; + } + + private Av1LevelBuffer PrepareCoefficientScratch( + int width, + int height, + bool clearLevels, + out Span coefficientContexts) + { + // AV1 omits high-frequency coefficients beyond 32 samples on every 64-point transform dimension. The tile + // creates maximum-sized workspaces once, then changes only the active views for subsequent transform blocks. + this.levels.Reset(new Size(width, height), clearLevels); + coefficientContexts = this.coefficientContexts.Memory.Span[..(width * height)]; + return this.levels; + } + + /// + /// Writes an end-of-block token and its context-coded and literal suffix bits. + /// + /// The one-based final nonzero scan position. + /// The luma or chroma component category. + /// The transform direction class. + /// The signaled transform size selecting the token alphabet. + /// The square transform-size probability context. + public void WriteEndOfBlockPosition(ushort endOfBlock, Av1ComponentType componentType, Av1TransformClass transformClass, Av1TransformSize transformSize, Av1TransformSize transformSizeContext) + { + _ = this.ProcessEndOfBlockPosition( + endOfBlock, + componentType, + transformClass, + transformSize, + transformSizeContext); + } + + private int ProcessEndOfBlockPosition( + ushort endOfBlock, + Av1ComponentType componentType, + Av1TransformClass transformClass, + Av1TransformSize transformSize, + Av1TransformSize transformSizeContext) + where TOperation : struct, ICoefficientSymbolOperation + { + short endOfBlockPosition = Av1SymbolContextHelper.GetEndOfBlockPosition(endOfBlock, out int eobExtra); + int rate = this.ProcessEndOfBlockFlag( + componentType, + transformClass, + transformSize, + endOfBlockPosition); + + int eobOffsetBitCount = Av1SymbolContextHelper.EndOfBlockOffsetBits[endOfBlockPosition]; + if (eobOffsetBitCount > 0) + { + ref Av1SymbolWriter w = ref this.writer; + int eobShift = eobOffsetBitCount - 1; + int bit = Av1Math.GetBit(eobExtra, eobShift); + + // The local table retains placeholders for the first three tokens, unlike the reference decoder's compact table, + // so the encoded token is also the distribution index. + int endOfBlockContext = endOfBlockPosition; + rate += TOperation.ProcessSymbol( + ref w, + bit, + this.endOfBlockExtra[(int)transformSizeContext][(int)componentType][endOfBlockContext]); + + // The context-coded high bit has already been consumed. The literal writer emits the remaining + // low-order suffix most-significant-bit first, preserving the AV1 syntax with one traversal call. + rate += TOperation.ProcessLiteral(ref w, (uint)eobExtra, eobOffsetBitCount - 1); + } + + return rate; + } + + /// + /// Gets the current fixed-point cost of the transform-block skip flag. + /// + /// Indicates whether the transform block is empty. + /// The square transform-size probability context. + /// The context derived from neighboring coefficient blocks. + /// The rate cost in 1/512-bit units. + public int GetTransformBlockSkipCost(bool skip, Av1TransformSize transformSizeContext, int skipContext) + => this.ProcessTransformBlockSkip(skip, transformSizeContext, skipContext); + + /// + /// Writes whether a transform block has no coded coefficients. + /// + /// Indicates whether the transform block is empty. + /// The square transform-size probability context. + /// The context derived from neighboring coefficient blocks. + public void WriteTransformBlockSkip(bool skip, Av1TransformSize transformSizeContext, int skipContext) + { + _ = this.ProcessTransformBlockSkip(skip, transformSizeContext, skipContext); + } + + private int ProcessTransformBlockSkip( + bool skip, + Av1TransformSize transformSizeContext, + int skipContext) + where TOperation : struct, ICoefficientSymbolOperation + { + ref Av1SymbolWriter w = ref this.writer; + return TOperation.ProcessSymbol( + ref w, + skip ? 1 : 0, + this.transformBlockSkip[(int)transformSizeContext][skipContext]); + } + + /// + /// Gets the current fixed-point cost of a transform-size subdivision depth. + /// + /// The block size defining the maximum transform. + /// The selected transform size. + /// The neighboring transform-size context. + /// The rate cost in 1/512-bit units. + public int GetTransformSizeCost(Av1BlockSize blockSize, Av1TransformSize transformSize, int context) + { + int selectedDepth = GetTransformSizeDepth(blockSize, transformSize, out int categoryDepth); + return Av1ProbabilityCost.GetSymbolCost(this.transformSize[categoryDepth - 1][context], selectedDepth); + } + + /// + /// Writes the selected transform size as its subdivision depth from the block maximum. + /// + /// The block size defining the maximum transform. + /// The selected transform size. + /// The neighboring transform-size context. + public void WriteTransformSize(Av1BlockSize blockSize, Av1TransformSize transformSize, int context) + { + int selectedDepth = GetTransformSizeDepth(blockSize, transformSize, out int categoryDepth); + ref Av1SymbolWriter w = ref this.writer; + w.WriteSymbol(selectedDepth, this.transformSize[categoryDepth - 1][context]); + } + + /// + /// Gets the current fixed-point cost of one variable-transform partition decision. + /// + /// Indicates whether the current transform node is split. + /// The neighboring variable-transform context. + /// The rate cost in 1/512-bit units. + public int GetTransformPartitionCost(bool split, int context) + => Av1ProbabilityCost.GetSymbolCost(this.transformPartition[context], split ? 1 : 0); + + /// + /// Writes one variable-transform partition decision. + /// + /// Indicates whether the current transform node is split. + /// The neighboring variable-transform context. + public void WriteTransformPartition(bool split, int context) + { + ref Av1SymbolWriter w = ref this.writer; + w.WriteSymbol(split ? 1 : 0, this.transformPartition[context]); + } + + private static int GetTransformSizeDepth( + Av1BlockSize blockSize, + Av1TransformSize transformSize, + out int categoryDepth) + { + Av1TransformSize maximumTransformSize = blockSize.GetMaximumTransformSize(); + Av1TransformSize currentTransformSize = maximumTransformSize; + categoryDepth = 0; + while (currentTransformSize != Av1TransformSize.Size4x4) + { + categoryDepth++; + currentTransformSize = currentTransformSize.GetSubSize(); + } + + int selectedDepth = 0; + currentTransformSize = maximumTransformSize; + while (currentTransformSize != transformSize && selectedDepth < Av1Constants.MaxVarTransform) + { + selectedDepth++; + currentTransformSize = currentTransformSize.GetSubSize(); + } + + DebugGuard.IsTrue(currentTransformSize == transformSize, nameof(transformSize)); + return selectedDepth; + } + + /// + /// Finalizes the range-coded tile payload and returns an owned exact-length copy. + /// + /// The memory owner containing the encoded tile bytes. + public IMemoryOwner Exit() + => this.writer.Exit(); + + /// + /// Finalizes the range-coded tile payload and exposes its encoded prefix without copying. + /// + /// The number of encoded bytes in the returned memory. + /// The encoded prefix, valid until this encoder is reset or disposed. + public ReadOnlyMemory Exit(out int length) + => this.writer.Exit(out length); + + /// + /// Exposes a prefix containing every consecutively encoded tile without copying their bytes. + /// + /// The number of bytes in the prefix. + /// The encoded prefix, valid until this encoder is reset or disposed. + public ReadOnlyMemory GetOutput(int length) + => this.writer.GetOutput(length); + + /// + /// Releases the range-coder output buffer and coefficient scratch memory. + /// + public void Dispose() + { + if (!this.isDisposed) + { + this.coefficientContexts.Dispose(); + this.levels.Dispose(); + this.writer.Dispose(); + this.isDisposed = true; + } + } + + /// + /// Writes the unsigned exponential-Golomb suffix used for coefficient levels beyond the base range. + /// + /// The nonnegative suffix value. + public void WriteGolomb(int level) + { + uint x = (uint)level + 1u; + int length = GetGolombBitLength(level); + ref Av1SymbolWriter w = ref this.writer; + w.WriteLiteral(0u, length - 1); + w.WriteLiteral(x, length); + } + + private static int GetBaseRangeCost(int level, Av1Distribution distribution) + { + int baseRange = Math.Min( + level - 1 - Av1Constants.BaseLevelsCount, + Av1Constants.CoefficientBaseRange); + + int fullChunkCount = baseRange / Av1Constants.BaseRangeSizeMinus1; + int rate = 0; + if (fullChunkCount > 0) + { + rate = fullChunkCount * Av1ProbabilityCost.GetSymbolCost( + distribution, + Av1Constants.BaseRangeSizeMinus1); + } + + // A partial range ends with its remainder symbol. Reaching the complete base range consumes four + // maximum symbols and has no terminating remainder before the Golomb escape. + if (baseRange < Av1Constants.CoefficientBaseRange) + { + int remainder = baseRange - (fullChunkCount * Av1Constants.BaseRangeSizeMinus1); + rate += Av1ProbabilityCost.GetSymbolCost(distribution, remainder); + } + + if (level > (Av1Constants.CoefficientBaseRange + Av1Constants.BaseLevelsCount)) + { + int golombValue = level - Av1Constants.CoefficientBaseRange - 1 - Av1Constants.BaseLevelsCount; + int length = GetGolombBitLength(golombValue); + rate += Av1ProbabilityCost.GetLiteralCost((2 * length) - 1); + } + + return rate; + } + + private static int GetGolombBitLength(int level) => (int)Av1Math.Log2_32((uint)level + 1u) + 1; + + /// + /// Writes the end-of-block token for a transform coefficient-count category. + /// + /// The luma or chroma component category. + /// The transform direction class. + /// The signaled transform size. + /// The one-based end-of-block token. + private int ProcessEndOfBlockFlag( + Av1ComponentType componentType, + Av1TransformClass transformClass, + Av1TransformSize transformSize, + int endOfBlockPosition) + where TOperation : struct, ICoefficientSymbolOperation + { + int endOfBlockMultiSize = transformSize.GetLog2Minus4(); + int endOfBlockContext = transformClass == Av1TransformClass.Class2D ? 0 : 1; + ref Av1SymbolWriter w = ref this.writer; + return TOperation.ProcessSymbol( + ref w, + endOfBlockPosition - 1, + this.endOfBlockFlag[endOfBlockMultiSize][(int)componentType][endOfBlockContext]); + } + + /// + /// Gets the current fixed-point rate cost of a transform type when the permitted transform set contains multiple choices. + /// + /// The transform type to cost. + /// The signaled transform size. + /// Indicates whether the frame restricts transform choices. + /// The active base quantizer index. + /// The filter-intra mode when enabled. + /// The ordinary intra prediction mode. + /// Indicates whether inter rather than intra transform probabilities apply. + /// The rate cost in 1/512-bit units. + public int GetTransformTypeCost( + Av1TransformType transformType, + Av1TransformSize transformSize, + bool useReducedTransformSet, + int baseQIndex, + Av1FilterIntraMode filterIntraMode, + Av1PredictionMode intraDirection, + bool usesInterTransformSet) + => this.ProcessTransformType( + transformType, + transformSize, + usesInterTransformSet, + useReducedTransformSet, + baseQIndex, + filterIntraMode, + intraDirection); + + /// + /// Writes a transform type when the permitted transform set contains multiple choices. + /// + /// The transform type to encode. + /// The signaled transform size. + /// Indicates whether the frame restricts transform choices. + /// The active base quantizer index. + /// The filter-intra mode when enabled. + /// The ordinary intra prediction mode. + /// Indicates whether inter rather than intra transform probabilities apply. + public void WriteTransformType( + Av1TransformType transformType, + Av1TransformSize transformSize, + bool useReducedTransformSet, + int baseQIndex, + Av1FilterIntraMode filterIntraMode, + Av1PredictionMode intraDirection, + bool usesInterTransformSet) + { + _ = this.ProcessTransformType( + transformType, + transformSize, + usesInterTransformSet, + useReducedTransformSet, + baseQIndex, + filterIntraMode, + intraDirection); + } + + private int ProcessTransformType( + Av1TransformType transformType, + Av1TransformSize transformSize, + bool usesInterTransformSet, + bool useReducedTransformSet, + int baseQIndex, + Av1FilterIntraMode filterIntraMode, + Av1PredictionMode intraDirection) + where TOperation : struct, ICoefficientSymbolOperation + { + Av1TransformSetType transformSetType = Av1SymbolContextHelper.GetExtendedTransformSetType( + transformSize, + usesInterTransformSet, + useReducedTransformSet); + + if (Av1SymbolContextHelper.GetExtendedTransformTypeCount(transformSetType) > 1 && baseQIndex > 0) + { + Av1TransformSize squareTransformSize = transformSize.GetSquareSize(); + DebugGuard.MustBeLessThanOrEqualTo((int)squareTransformSize, Av1Constants.ExtendedTransformCount, nameof(squareTransformSize)); + + int extendedSet = Av1SymbolContextHelper.GetExtendedTransformSet(transformSetType, usesInterTransformSet); + + // Set zero contains only DCT-DCT, which was excluded by the multiple-choice condition above. + DebugGuard.MustBeGreaterThan(extendedSet, 0, nameof(extendedSet)); + + int transformIndex = Av1SymbolContextHelper.GetExtendedTransformIndex(transformSetType, transformType); + ref Av1SymbolWriter w = ref this.writer; + if (usesInterTransformSet) + { + // Inter transforms are conditioned only by the transform set and square size. + return TOperation.ProcessSymbol( + ref w, + transformIndex, + this.interExtendedTransform[extendedSet][(int)squareTransformSize]); + } + + Av1PredictionMode intraDirectionContext; + if (filterIntraMode != Av1FilterIntraMode.AllFilterIntraModes) + { + intraDirectionContext = filterIntraMode.ToIntraDirection(); + } + else + { + intraDirectionContext = intraDirection; + } + + DebugGuard.MustBeLessThan((int)intraDirectionContext, 13, nameof(intraDirectionContext)); + DebugGuard.MustBeLessThan((int)squareTransformSize, 4, nameof(squareTransformSize)); + return TOperation.ProcessSymbol( + ref w, + transformIndex, + this.intraExtendedTransform[extendedSet][(int)squareTransformSize][(int)intraDirectionContext]); + } + + return 0; + } + + /// + /// Writes a spatially predicted segment identifier. + /// + /// The segment identifier. + /// The context derived from neighboring segment identifiers. + public void WriteSegmentId(int segmentId, int context) + { + ref Av1SymbolWriter w = ref this.writer; + w.WriteSymbol(segmentId, this.segmentId[context]); + } + + /// + /// Gets the current fixed-point cost of the transform-skip flag. + /// + /// Indicates whether the block contains no coded transform coefficients. + /// The neighboring skip context. + /// The rate cost in 1/512-bit units. + public int GetSkipCost(bool skip, int context) + => Av1ProbabilityCost.GetSymbolCost(this.skip[context], skip ? 1 : 0); + + /// + /// Writes the transform-skip flag from a neighboring skip context. + /// + /// Indicates whether the block contains no coded transform coefficients. + /// The neighboring skip context. + public void WriteSkip(bool skip, int context) + { + ref Av1SymbolWriter w = ref this.writer; + w.WriteSymbol(skip, this.skip[context]); + } + + /// + /// Writes the compound-reference skip-mode flag. + /// + /// Indicates whether skip mode is selected. + /// The neighboring skip-mode context. + public void WriteSkipMode(bool skip, int context) + { + ref Av1SymbolWriter w = ref this.writer; + w.WriteSymbol(skip, this.skipMode[context]); + } + + /// + /// Gets the current fixed-point cost of the filter-intra enable flag and selected mode. + /// + /// The selected filter-intra mode, or the disabled sentinel. + /// The block size selecting the enable distribution. + /// The rate cost in 1/512-bit units. + public int GetFilterIntraModeCost(Av1FilterIntraMode filterIntraMode, Av1BlockSize blockSize) + { + bool useFilter = filterIntraMode != Av1FilterIntraMode.AllFilterIntraModes; + int cost = Av1ProbabilityCost.GetSymbolCost(this.filterIntra[(int)blockSize], useFilter ? 1 : 0); + if (useFilter) + { + cost += Av1ProbabilityCost.GetSymbolCost(this.filterIntraMode, (int)filterIntraMode); + } + + return cost; + } + + /// + /// Writes the filter-intra enable flag and, when enabled, its prediction mode. + /// + /// The selected filter-intra mode, or the disabled sentinel. + /// The block size selecting the enable distribution. + public void WriteFilterIntraMode(Av1FilterIntraMode filterIntraMode, Av1BlockSize blockSize) + { + ref Av1SymbolWriter w = ref this.writer; + bool useFilter = filterIntraMode != Av1FilterIntraMode.AllFilterIntraModes; + w.WriteSymbol(useFilter, this.filterIntra[(int)blockSize]); + if (useFilter) + { + w.WriteSymbol((int)filterIntraMode, this.filterIntraMode); + } + } + + /// + /// Writes a signed quantizer-index delta value. + /// + /// The signed quantizer-index delta. + public void WriteDeltaQuantizerIndex(int deltaQindex) + { + ref Av1SymbolWriter w = ref this.writer; + bool sign = deltaQindex < 0; + int abs = Math.Abs(deltaQindex); + bool isSmallValue = abs < Av1Constants.DeltaQuantizerSmall; + + w.WriteSymbol(Math.Min(abs, Av1Constants.DeltaQuantizerSmall), this.deltaQuantizerAbsolute); + + if (!isSmallValue) + { + // Escape magnitudes encode their bit width first, followed by the offset within that width's range. + int remainingBitCount = Av1Math.MostSignificantBit((uint)(abs - 1)); + int threshold = (1 << remainingBitCount) + 1; + w.WriteLiteral((uint)(remainingBitCount - 1), 3); + w.WriteLiteral((uint)(abs - threshold), remainingBitCount); + } + + if (abs > 0) + { + w.WriteLiteral(sign); + } + } + + /// + /// Gets the current fixed-point cost of a key-frame luma prediction mode. + /// + /// The luma prediction mode. + /// The reduced above-mode context. + /// The reduced left-mode context. + /// The rate cost in 1/512-bit units. + public int GetLumaModeCost(Av1PredictionMode lumaMode, byte topContext, byte leftContext) + => Av1ProbabilityCost.GetSymbolCost( + this.keyFrameYMode[topContext][leftContext], + (int)lumaMode); + + /// + /// Writes a key-frame luma prediction mode using the above and left mode contexts. + /// + /// The luma prediction mode. + /// The reduced above-mode context. + /// The reduced left-mode context. + public void WriteLumaMode(Av1PredictionMode lumaMode, byte topContext, byte leftContext) + { + ref Av1SymbolWriter w = ref this.writer; + w.WriteSymbol((int)lumaMode, this.keyFrameYMode[topContext][leftContext]); + } + + /// + /// Gets the cost of an intra luma mode coded inside an inter frame. + /// + /// The intra luma mode. + /// The coding block size selecting the size group. + /// The syntax cost in 1/512-bit units. + public int GetInterFrameLumaModeCost(Av1PredictionMode lumaMode, Av1BlockSize blockSize) + => Av1ProbabilityCost.GetSymbolCost(this.frameYMode[blockSize.GetSizeGroup()], (int)lumaMode); + + /// + /// Writes an intra luma mode coded inside an inter frame. + /// + /// The intra luma mode. + /// The coding block size selecting the size group. + public void WriteInterFrameLumaMode(Av1PredictionMode lumaMode, Av1BlockSize blockSize) + { + ref Av1SymbolWriter w = ref this.writer; + w.WriteSymbol((int)lumaMode, this.frameYMode[blockSize.GetSizeGroup()]); + } + + /// + /// Gets the cost of the prediction-domain decision for an inter-frame block. + /// + /// Whether the block uses a retained reference frame. + /// The neighboring prediction-domain context. + /// The syntax cost in 1/512-bit units. + public int GetIsInterCost(bool isInter, int context) + => Av1ProbabilityCost.GetSymbolCost(this.intraInter[context], isInter ? 1 : 0); + + /// + /// Writes the prediction-domain decision for an inter-frame block. + /// + /// Whether the block uses a retained reference frame. + /// The neighboring prediction-domain context. + public void WriteIsInter(bool isInter, int context) + { + ref Av1SymbolWriter w = ref this.writer; + w.WriteSymbol(isInter, this.intraInter[context]); + } + + /// + /// Gets the cost of selecting one reference from the single-reference branch tree. + /// + /// The selected reference-frame label. + /// The neighboring reference counts indexed by reference-frame label. + /// The syntax cost in 1/512-bit units. + public int GetSingleReferenceCost( + Av1ReferenceFrameType referenceFrame, + ReadOnlySpan referenceCounts) + { + bool isBackward = referenceFrame >= Av1ReferenceFrameType.Backward; + int context = Av1SymbolContextHelper.GetSingleReferenceBackwardContext(referenceCounts); + int rate = Av1ProbabilityCost.GetSymbolCost(this.singleReference[context][0], isBackward ? 1 : 0); + if (isBackward) + { + bool isAlternate = referenceFrame == Av1ReferenceFrameType.Alternate; + context = Av1SymbolContextHelper.GetSingleReferenceAlternateContext(referenceCounts); + rate += Av1ProbabilityCost.GetSymbolCost(this.singleReference[context][1], isAlternate ? 1 : 0); + if (isAlternate) + { + return rate; + } + + context = Av1SymbolContextHelper.GetSingleReferenceAlternate2Context(referenceCounts); + return rate + Av1ProbabilityCost.GetSymbolCost( + this.singleReference[context][5], + referenceFrame == Av1ReferenceFrameType.Alternate2 ? 1 : 0); + } + + bool isLast3OrGolden = referenceFrame is Av1ReferenceFrameType.Last3 or Av1ReferenceFrameType.Golden; + context = Av1SymbolContextHelper.GetSingleReferenceLast3OrGoldenContext(referenceCounts); + rate += Av1ProbabilityCost.GetSymbolCost(this.singleReference[context][2], isLast3OrGolden ? 1 : 0); + if (isLast3OrGolden) + { + context = Av1SymbolContextHelper.GetSingleReferenceGoldenContext(referenceCounts); + return rate + Av1ProbabilityCost.GetSymbolCost( + this.singleReference[context][4], + referenceFrame == Av1ReferenceFrameType.Golden ? 1 : 0); + } + + context = Av1SymbolContextHelper.GetSingleReferenceLast2Context(referenceCounts); + return rate + Av1ProbabilityCost.GetSymbolCost( + this.singleReference[context][3], + referenceFrame == Av1ReferenceFrameType.Last2 ? 1 : 0); + } + + /// + /// Writes one reference through the single-reference branch tree. + /// + /// The selected reference-frame label. + /// The neighboring reference counts indexed by reference-frame label. + public void WriteSingleReference( + Av1ReferenceFrameType referenceFrame, + ReadOnlySpan referenceCounts) + { + ref Av1SymbolWriter w = ref this.writer; + bool isBackward = referenceFrame >= Av1ReferenceFrameType.Backward; + int context = Av1SymbolContextHelper.GetSingleReferenceBackwardContext(referenceCounts); + w.WriteSymbol(isBackward, this.singleReference[context][0]); + if (isBackward) + { + bool isAlternate = referenceFrame == Av1ReferenceFrameType.Alternate; + context = Av1SymbolContextHelper.GetSingleReferenceAlternateContext(referenceCounts); + w.WriteSymbol(isAlternate, this.singleReference[context][1]); + if (isAlternate) + { + return; + } + + context = Av1SymbolContextHelper.GetSingleReferenceAlternate2Context(referenceCounts); + w.WriteSymbol( + referenceFrame == Av1ReferenceFrameType.Alternate2, + this.singleReference[context][5]); + + return; + } + + bool isLast3OrGolden = referenceFrame is Av1ReferenceFrameType.Last3 or Av1ReferenceFrameType.Golden; + context = Av1SymbolContextHelper.GetSingleReferenceLast3OrGoldenContext(referenceCounts); + w.WriteSymbol(isLast3OrGolden, this.singleReference[context][2]); + if (isLast3OrGolden) + { + context = Av1SymbolContextHelper.GetSingleReferenceGoldenContext(referenceCounts); + w.WriteSymbol( + referenceFrame == Av1ReferenceFrameType.Golden, + this.singleReference[context][4]); + + return; + } + + context = Av1SymbolContextHelper.GetSingleReferenceLast2Context(referenceCounts); + w.WriteSymbol( + referenceFrame == Av1ReferenceFrameType.Last2, + this.singleReference[context][3]); + } + + /// + /// Gets the current fixed-point cost of a directional angle-delta symbol. + /// + /// The signed angle delta offset by . + /// The directional prediction mode selecting the distribution. + /// The rate cost in 1/512-bit units. + public int GetAngleDeltaCost(int angleDelta, Av1PredictionMode context) + => Av1ProbabilityCost.GetSymbolCost( + this.angleDelta[context - Av1PredictionMode.Vertical], + angleDelta); + + /// + /// Writes an unsigned directional angle-delta symbol. + /// + /// The signed angle delta offset by . + /// The directional prediction mode selecting the distribution. + public void WriteAngleDelta(int angleDelta, Av1PredictionMode context) + { + ref Av1SymbolWriter w = ref this.writer; + w.WriteSymbol(angleDelta, this.angleDelta[context - Av1PredictionMode.Vertical]); + } + + /// + /// Writes a fixed-width CDEF strength index. + /// + /// The CDEF strength index. + /// The number of signaled bits. + public void WriteCdefStrength(int cdefStrength, int bitCount) + { + ref Av1SymbolWriter w = ref this.writer; + w.WriteLiteral((uint)cdefStrength, bitCount); + } + + /// + /// Gets the current fixed-point cost of a chroma intra prediction mode. + /// + /// The chroma prediction mode. + /// Indicates whether chroma-from-luma is valid for the block. + /// The block's luma prediction mode. + /// The rate cost in 1/512-bit units. + public int GetChromaModeCost(Av1ChromaPredictionMode chromaMode, bool isChromaFromLumaAllowed, Av1PredictionMode lumaMode) + { + int cflAllowed = isChromaFromLumaAllowed ? 1 : 0; + return Av1ProbabilityCost.GetSymbolCost(this.uvMode[cflAllowed][(int)lumaMode], (int)chromaMode); + } + + /// + /// Gets the current fixed-point cost of joint chroma-from-luma alpha syntax. + /// + /// The packed U/V alpha-magnitude indices. + /// The joint U/V sign symbol. + /// The rate cost in 1/512-bit units. + public int GetChromaFromLumaCost(int chromaFromLumaIndex, int joinedSign) + { + int cost = Av1ProbabilityCost.GetSymbolCost(this.chromaFromLumaSign, joinedSign); + int signU = Av1ChromaFromLumaMath.SignU(joinedSign); + if (signU != Av1ChromaFromLumaMath.SignZero) + { + int contextU = Av1ChromaFromLumaMath.ContextU(joinedSign); + int indexU = Av1ChromaFromLumaMath.IndexU(chromaFromLumaIndex); + cost += Av1ProbabilityCost.GetSymbolCost(this.chromaFromLumaAlpha[contextU], indexU); + } + + int signV = Av1ChromaFromLumaMath.SignV(joinedSign); + if (signV != Av1ChromaFromLumaMath.SignZero) + { + int contextV = Av1ChromaFromLumaMath.ContextV(joinedSign); + int indexV = Av1ChromaFromLumaMath.IndexV(chromaFromLumaIndex); + cost += Av1ProbabilityCost.GetSymbolCost(this.chromaFromLumaAlpha[contextV], indexV); + } + + return cost; + } + + /// + /// Writes a chroma intra prediction mode conditioned on the luma mode and chroma-from-luma availability. + /// + /// The chroma prediction mode. + /// Indicates whether chroma-from-luma is valid for the block. + /// The block's luma prediction mode. + public void WriteChromaMode(Av1ChromaPredictionMode chromaMode, bool isChromaFromLumaAllowed, Av1PredictionMode lumaMode) + { + ref Av1SymbolWriter w = ref this.writer; + int cflAllowed = isChromaFromLumaAllowed ? 1 : 0; + w.WriteSymbol((int)chromaMode, this.uvMode[cflAllowed][(int)lumaMode]); + } + + /// + /// Writes the joint chroma-from-luma signs and the magnitude index for each nonzero plane. + /// + /// The packed U/V alpha-magnitude indices. + /// The joint U/V sign symbol. + public void WriteChromaFromLumaAlphas(int chromaFromLumaIndex, int joinedSign) + { + ref Av1SymbolWriter w = ref this.writer; + w.WriteSymbol(joinedSign, this.chromaFromLumaSign); + + // Magnitudes are only signaled for nonzero signs; the shared helper keeps encoder and decoder mappings exact. + int signU = Av1ChromaFromLumaMath.SignU(joinedSign); + if (signU != Av1ChromaFromLumaMath.SignZero) + { + int contextU = Av1ChromaFromLumaMath.ContextU(joinedSign); + int indexU = Av1ChromaFromLumaMath.IndexU(chromaFromLumaIndex); + w.WriteSymbol(indexU, this.chromaFromLumaAlpha[contextU]); + } + + int signV = Av1ChromaFromLumaMath.SignV(joinedSign); + if (signV != Av1ChromaFromLumaMath.SignZero) + { + int contextV = Av1ChromaFromLumaMath.ContextV(joinedSign); + int indexV = Av1ChromaFromLumaMath.IndexV(chromaFromLumaIndex); + w.WriteSymbol(indexV, this.chromaFromLumaAlpha[contextV]); + } + } + + /// + /// Traverses a palette color-index map once for either live rate costing or entropy emission. + /// + /// The closed map-symbol operation. + /// The number of colors in the palette. + /// The luma or chroma plane class. + /// The number of coded map rows. + /// The number of coded map columns. + /// The complete row-addressable color-index map. + /// The rate cost in 1/512-bit units, or zero while writing. + private int ProcessPaletteColorMap( + int paletteSize, + Av1PlaneType planeType, + int rows, + int columns, + Buffer2DRegion colorIndexMap) + where TOperation : struct, IPaletteColorMapOperation + { + int colorIndex = colorIndexMap.DangerousGetRowSpan(0)[0]; + int cost = TOperation.ProcessFirstIndex(this, paletteSize, colorIndex); + Span colorOrder = stackalloc byte[Av1Constants.PaletteMaxSize]; + for (int diagonal = 1; diagonal < rows + columns - 1; diagonal++) + { + int firstColumn = Math.Min(diagonal, columns - 1); + int lastColumn = Math.Max(0, diagonal - rows + 1); + for (int column = firstColumn; column >= lastColumn; column--) + { + int row = diagonal - column; + colorIndex = colorIndexMap.DangerousGetRowSpan(row)[column]; + int colorContext = Av1PaletteColorMap.GetContext( + colorIndexMap, + row, + column, + paletteSize, + colorIndex, + colorOrder, + out int colorOrderIndex); + + cost += TOperation.ProcessColorIndex( + this, + paletteSize, + planeType, + colorContext, + colorOrderIndex); + } + } + + return cost; + } + + /// + /// Separates palette colors selected from the neighbor cache from colors that require literal coding. + /// + /// The sorted unique neighbor colors. + /// The sorted palette colors. + /// The cache-selection flags. + /// The destination for colors absent from the cache. + /// The number of uncached colors. + private static int IndexColorCache( + ReadOnlySpan colorCache, + ReadOnlySpan colors, + Span cacheColorFound, + Span uncachedColors) + { + cacheColorFound[..colorCache.Length].Clear(); + Span inCache = stackalloc byte[Av1Constants.PaletteMaxSize]; + inCache.Clear(); + + // Cache-order flags drive the bitstream while palette-order flags preserve the sorted uncached output. + int cachedColorCount = 0; + for (int cacheIndex = 0; cacheIndex < colorCache.Length && cachedColorCount < colors.Length; cacheIndex++) + { + for (int colorIndex = 0; colorIndex < colors.Length; colorIndex++) + { + if (colors[colorIndex] == colorCache[cacheIndex]) + { + inCache[colorIndex] = 1; + cacheColorFound[cacheIndex] = 1; + cachedColorCount++; + break; + } + } + } + + int uncachedColorCount = 0; + for (int colorIndex = 0; colorIndex < colors.Length; colorIndex++) + { + if (inCache[colorIndex] == 0) + { + uncachedColors[uncachedColorCount++] = colors[colorIndex]; + } + } + + return uncachedColorCount; + } + + /// + /// Gets the literal length of an ascending palette-color sequence. + /// + /// The sorted colors. + /// The number of bits in each color sample. + /// The minimum representable difference between adjacent colors. + /// The literal length in bits. + private static int GetDeltaEncodedColorBitCount( + ReadOnlySpan colors, + int bitDepth, + int minimumDelta) + { + if (colors.IsEmpty) + { + return 0; + } + + int bitCount = bitDepth; + if (colors.Length == 1) + { + return bitCount; + } + + int maximumDelta = 0; + for (int i = 1; i < colors.Length; i++) + { + maximumDelta = Math.Max(maximumDelta, colors[i] - colors[i - 1]); + } + + int minimumBits = bitDepth - 3; + int bits = Math.Max( + (int)Av1Math.CeilLog2((uint)(maximumDelta + 1 - minimumDelta)), + minimumBits); + + int range = (1 << bitDepth) - colors[0] - minimumDelta; + bitCount += 2; + for (int i = 1; i < colors.Length; i++) + { + int delta = colors[i] - colors[i - 1]; + bitCount += bits; + range -= delta; + bits = Math.Min(bits, (int)Av1Math.CeilLog2((uint)range)); + } + + return bitCount; + } + + /// + /// Writes an ascending palette-color sequence as one literal followed by bounded deltas. + /// + /// The sorted colors. + /// The number of bits in each color sample. + /// The minimum representable difference between adjacent colors. + private void WriteDeltaEncodedColors( + ReadOnlySpan colors, + int bitDepth, + int minimumDelta) + { + if (colors.IsEmpty) + { + return; + } + + this.WriteLiteral(colors[0], bitDepth); + if (colors.Length == 1) + { + return; + } + + int maximumDelta = 0; + for (int i = 1; i < colors.Length; i++) + { + maximumDelta = Math.Max(maximumDelta, colors[i] - colors[i - 1]); + } + + int minimumBits = bitDepth - 3; + int bits = Math.Max( + (int)Av1Math.CeilLog2((uint)(maximumDelta + 1 - minimumDelta)), + minimumBits); + + this.WriteLiteral((uint)(bits - minimumBits), 2); + int range = (1 << bitDepth) - colors[0] - minimumDelta; + for (int i = 1; i < colors.Length; i++) + { + int delta = colors[i] - colors[i - 1]; + this.WriteLiteral((uint)(delta - minimumDelta), bits); + range -= delta; + bits = Math.Min(bits, (int)Av1Math.CeilLog2((uint)range)); + } + } + + /// + /// Gets the bit width required by wrapped V-plane palette deltas. + /// + /// The V-plane colors in U-palette order. + /// The number of bits in each color sample. + /// The number of deltas that omit a sign bit. + /// The minimum permitted delta width. + /// The delta width in bits. + private static int GetPaletteVDeltaBitCount( + ReadOnlySpan colors, + int bitDepth, + out int zeroCount, + out int minimumBits) + { + int sampleRange = 1 << bitDepth; + int maximumDelta = 0; + zeroCount = 0; + minimumBits = bitDepth - 4; + for (int i = 1; i < colors.Length; i++) + { + int delta = Math.Abs(colors[i] - colors[i - 1]); + int wrappedDelta = Math.Min(delta, sampleRange - delta); + maximumDelta = Math.Max(maximumDelta, wrappedDelta); + if (wrappedDelta == 0) + { + zeroCount++; + } + } + + return Math.Max((int)Av1Math.CeilLog2((uint)(maximumDelta + 1)), minimumBits); + } + + /// + /// Emits coefficient syntax and reports no estimated rate. + /// + private readonly struct CoefficientWriteOperation : ICoefficientSymbolOperation + { + public static int ProcessSymbol( + ref Av1SymbolWriter writer, + int symbol, + Av1Distribution distribution) + { + writer.WriteSymbol(symbol, distribution); + return 0; + } + + public static int ProcessLiteral( + ref Av1SymbolWriter writer, + uint value, + int bitCount) + { + writer.WriteLiteral(value, bitCount); + return 0; + } + } + + /// + /// Measures coefficient syntax against the live tile distributions without changing them. + /// + private readonly struct CoefficientCostOperation : ICoefficientSymbolOperation + { + public static int ProcessSymbol( + ref Av1SymbolWriter writer, + int symbol, + Av1Distribution distribution) + => Av1ProbabilityCost.GetSymbolCost(distribution, symbol); + + public static int ProcessLiteral( + ref Av1SymbolWriter writer, + uint value, + int bitCount) + => Av1ProbabilityCost.GetLiteralCost(bitCount); + } + + /// + /// Emits palette-map syntax and reports no estimated rate. + /// + private readonly struct PaletteColorMapWriteOperation : IPaletteColorMapOperation + { + public static int ProcessFirstIndex( + Av1SymbolEncoder encoder, + int paletteSize, + int colorIndex) + { + encoder.WriteUniform(paletteSize, colorIndex); + return 0; + } + + public static int ProcessColorIndex( + Av1SymbolEncoder encoder, + int paletteSize, + Av1PlaneType planeType, + int colorContext, + int colorOrderIndex) + { + encoder.WritePaletteColorIndex( + colorOrderIndex, + paletteSize, + colorContext, + planeType); + + return 0; + } + } + + /// + /// Measures palette-map syntax against the live tile distributions without changing them. + /// + private readonly struct PaletteColorMapCostOperation : IPaletteColorMapOperation + { + public static int ProcessFirstIndex( + Av1SymbolEncoder encoder, + int paletteSize, + int colorIndex) + => GetUniformCost(paletteSize, colorIndex); + + public static int ProcessColorIndex( + Av1SymbolEncoder encoder, + int paletteSize, + Av1PlaneType planeType, + int colorContext, + int colorOrderIndex) + => encoder.GetPaletteColorIndexCost( + colorOrderIndex, + paletteSize, + colorContext, + planeType); + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Entropy/Av1SymbolReader.cs b/src/ImageSharp/Formats/Heif/Av1/Entropy/Av1SymbolReader.cs new file mode 100644 index 0000000000..336cf703a9 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Entropy/Av1SymbolReader.cs @@ -0,0 +1,290 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Entropy; + +/// +/// Reads AV1 literals and adaptively coded symbols from one bounded entropy-coded byte span. +/// +internal ref struct Av1SymbolReader +{ + /// + /// The number of bits in the range-decoder code-value window. + /// + private const int DecoderWindowsSize = 32; + + /// + /// The synthetic count used after the bounded input has been exhausted and zero padding begins. + /// + private const int LotsOfBits = 0x4000; + + /// + /// The bounded entropy-coded bytes available to this reader. + /// + private readonly Span buffer; + + /// + /// Indicates whether decoded symbols adapt their distributions. + /// + private readonly bool updateCdf; + + /// + /// The next byte position to load into the code-value window. + /// + private int position; + + /// + /// The difference between the upper end of the current range and the coded value, minus one. + /// + /// + /// The decoder compares the upper 16 bits. Renormalization shifts consumed bits out and refills the lower portion + /// from so the comparison remains aligned with . + /// + private uint difference; + + /// + /// The number of code values in the current normalized interval. + /// + private uint range; + + /// + /// The number of buffered bits below the 16-bit comparison window. + /// + private int count; + + /// + /// The adjustment that preserves the logical consumed-bit count after the reader enters implicit zero padding. + /// + private int tellOffset; + + /// + /// Initializes a new instance of the struct over one entropy-coded span. + /// + /// The bounded entropy-coded bytes. + /// A value indicating whether decoded symbols adapt their distributions. + public Av1SymbolReader(Span span, bool updateCdf = true) + { + this.buffer = span; + this.updateCdf = updateCdf; + this.position = 0; + this.difference = (1U << (DecoderWindowsSize - 1)) - 1; + this.range = 0x8000; + this.count = -15; + this.tellOffset = 10 - (DecoderWindowsSize - 8); + this.Refill(); + } + + /// + /// Validates that range decoding remained within the bounded tile payload and ended at the required trailing-one bit. + /// + public void ValidateTrailingBits() + { + int consumedBitCount = this.GetConsumedBitCount(); + int consumedByteCount = (consumedBitCount + 7) >> 3; + if (consumedByteCount > this.buffer.Length) + { + throw new InvalidImageContentException("The AV1 tile entropy stream is truncated."); + } + + // The final consumed byte must contain one trailing-one bit at the range decoder's exact stopping position, + // followed only by zero bits. This is the same bounded-stream check performed after the reference decoder decodes a tile. + int trailingOneBit = 128 >> ((consumedBitCount - 1) & 7); + int trailingBitMask = (trailingOneBit << 1) - 1; + if ((this.buffer[consumedByteCount - 1] & trailingBitMask) != trailingOneBit) + { + throw new InvalidImageContentException("The AV1 tile entropy stream has invalid trailing bits."); + } + + for (int i = consumedByteCount; i < this.buffer.Length; i++) + { + if (this.buffer[i] != 0) + { + throw new InvalidImageContentException("The AV1 tile entropy stream has nonzero padding bytes."); + } + } + } + + /// + /// Reads one symbol and adapts its distribution when CDF updates are enabled. + /// + /// The inverse cumulative distribution for the symbol alphabet. + /// The decoded zero-based symbol. + public int ReadSymbol(Av1Distribution distribution) + { + int value = this.DecodeIntegerQ15(distribution); + + // disable_cdf_update freezes every tile distribution while leaving range decoding unchanged. + if (this.updateCdf) + { + distribution.Update(value); + } + + return value; + } + + /// + /// Reads one non-adaptive binary symbol using the supplied Q15 probability for . + /// + /// The probability that the symbol is , scaled by 32768. + /// The decoded binary symbol. + public bool ReadBoolean(uint frequency) => this.DecodeBoolQ15(frequency); + + /// + /// Reads an unsigned literal in most-significant-bit-first order. + /// + /// The number of literal bits to read. + /// The decoded literal. + public int ReadLiteral(int bitCount) + { + const uint prob = (0x7FFFFFU - (128 << 15) + 128) >> 8; + int literal = 0; + for (int bit = bitCount - 1; bit >= 0; bit--) + { + if (this.DecodeBoolQ15(prob)) + { + literal |= 1 << bit; + } + } + + return literal; + } + + /// + /// Decode a single binary value. + /// + /// The probability that the bit is one, scaled by 32768. + /// The decoded binary value. + private bool DecodeBoolQ15(uint frequency) + { + uint dif; + uint vw; + uint range; + uint newRange; + uint v; + bool ret; + + dif = this.difference; + range = this.range; + + // Reserve a minimum interval for both outcomes after reducing the Q15 frequency to the range-coder + // multiplication precision. This is the same rounding model used by Av1SymbolWriter. + v = ((range >> 8) * (frequency >> Av1Distribution.ProbabilityShift)) >> (7 - Av1Distribution.ProbabilityShift); + v += Av1Distribution.ProbabilityMinimum; + vw = v << (DecoderWindowsSize - 16); + ret = true; + newRange = v; + if (dif >= vw) + { + newRange = range - v; + dif -= vw; + ret = false; + } + + this.Normalize(dif, newRange); + return ret; + } + + /// + /// Decodes a symbol given an inverse cumulative distribution function(CDF) table in Q15. + /// + /// + /// CDF_PROB_TOP minus the CDF, such that symbol s falls in the range + /// [s > 0 ? (CDF_PROB_TOP - icdf[s - 1]) : 0, CDF_PROB_TOP - icdf[s]). + /// The values must be monotonically non - increasing, and icdf[nsyms - 1] must be 0. + /// + /// The decoded symbol. + private int DecodeIntegerQ15(Av1Distribution distribution) + { + uint c; + uint u; + uint v; + int ret; + + uint dif = this.difference; + uint r = this.range; + int n = distribution.NumberOfSymbols - 1; + + DebugGuard.MustBeLessThan(dif >> (DecoderWindowsSize - 16), r, nameof(r)); + DebugGuard.IsTrue(distribution[n] == 0, "Last value in probability array needs to be zero."); + DebugGuard.MustBeGreaterThanOrEqualTo(r, 32768U, nameof(r)); + DebugGuard.MustBeGreaterThanOrEqualTo(7 - Av1Distribution.ProbabilityShift - Av1Distribution.CdfShift, 0, nameof(Av1Distribution.CdfShift)); + c = dif >> (DecoderWindowsSize - 16); + v = r; + ret = -1; + do + { + u = v; + v = ((r >> 8) * (distribution[++ret] >> Av1Distribution.ProbabilityShift)) >> (7 - Av1Distribution.ProbabilityShift - Av1Distribution.CdfShift); + v += (uint)(Av1Distribution.ProbabilityMinimum * (n - ret)); + } + while (c < v); + + DebugGuard.MustBeLessThan(v, u, nameof(v)); + DebugGuard.MustBeLessThanOrEqualTo(u, r, nameof(u)); + r = u - v; + dif -= v << (DecoderWindowsSize - 16); + this.Normalize(dif, r); + return ret; + } + + /// + /// Takes updated dif and range values, renormalizes them so that + /// has value between 32768 and 65536 (reading more bytes from the stream into dif if + /// necessary), and stores them back in the decoder context. + /// + /// The updated code-value difference. + /// The updated coding interval width. + private void Normalize(uint dif, uint rng) + { + // Shifting by the leading-zero count restores the interval to [32768, 65536) and consumes the same number of + // code-value bits. Adding one before the shift preserves the decoder's difference-minus-one representation. + int d = 15 - Av1Math.MostSignificantBit(rng); + this.count -= d; + this.difference = ((dif + 1) << d) - 1; + this.range = rng << d; + if (this.count < 0) + { + this.Refill(); + } + } + + /// + /// Loads whole bytes into the lower portion of the code-value window after renormalization. + /// + private void Refill() + { + uint dif = this.difference; + int cnt = this.count; + int position = this.position; + int end = this.buffer.Length; + int s = DecoderWindowsSize - 9 - (cnt + 15); + for (; s >= 0 && position < end; s -= 8, position++) + { + // XOR inserts a source byte into the difference-minus-one representation. Advancing both the byte + // position and buffered-bit count leaves the logical number of consumed bits unchanged. + DebugGuard.MustBeLessThan(s, DecoderWindowsSize - 8, nameof(s)); + dif ^= (uint)this.buffer[position] << s; + cnt += 8; + } + + if (position >= end) + { + // AV1 range decoding permits the final interval to consume implicit zero padding. A large count models + // that padding without repeatedly attempting to refill it. Preserve the previous count in tellOffset so + // the logical position continues past the bounded source and truncated payloads remain detectable. + this.tellOffset += LotsOfBits - cnt; + cnt = LotsOfBits; + } + + this.difference = dif; + this.count = cnt; + this.position = position; + } + + /// + /// Gets the number of entropy bits consumed from the bounded tile payload, including the initial range-coder bit. + /// + /// The logical consumed-bit count. + private readonly int GetConsumedBitCount() + => (this.position * 8) - this.count + this.tellOffset; +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Entropy/Av1SymbolWriter.cs b/src/ImageSharp/Formats/Heif/Av1/Entropy/Av1SymbolWriter.cs new file mode 100644 index 0000000000..6b26d84d2e --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Entropy/Av1SymbolWriter.cs @@ -0,0 +1,419 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Buffers; +using System.Buffers.Binary; +using SixLabors.ImageSharp.Memory; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Entropy; + +/// +/// Writes AV1 literals and adaptively coded symbols to a range-coded byte sequence. +/// +internal sealed class Av1SymbolWriter : IDisposable +{ + /// + /// The normalized range before the first symbol narrows the coding interval. + /// + private const uint InitialRange = 0x8000U; + + /// + /// The initial bit count that crosses the first byte-and-carry flush boundary after one output byte. + /// + private const int InitialCount = -9; + + /// + /// The lower endpoint of the current coding interval. + /// + private ulong low; + + /// + /// The width of the current normalized coding interval. + /// + private uint rng = InitialRange; + + /// + /// The number of accumulated bits relative to the next byte-and-carry flush boundary. + /// + /// + /// The initial value of -9 crosses zero after one output byte and its carry bit have accumulated. + /// + private int cnt = InitialCount; + + /// + /// The configuration that supplies output allocation. + /// + private readonly Configuration configuration; + + /// + /// The owner of the output buffer shared by consecutively encoded tiles. + /// + private IMemoryOwner bufferOwner; + + /// + /// The complete requested output allocation, including every consecutively encoded tile. + /// + private Memory outputBuffer; + + /// + /// The requested output range, excluding any excess capacity returned by a pooling allocator. + /// + private Memory buffer; + + /// + /// Indicates whether encoded symbols adapt their distributions. + /// + private readonly bool updateCdf; + + /// + /// The next output byte position. + /// + private int position; + + /// + /// Initializes a new instance of the class. + /// + /// The configuration that supplies output allocation. + /// The initial output capacity in bytes. + /// A value indicating whether encoded symbols adapt their distributions. + public Av1SymbolWriter(Configuration configuration, int bufferLength, bool updateCdf) + { + this.configuration = configuration; + this.bufferOwner = configuration.MemoryAllocator.Allocate(bufferLength); + this.outputBuffer = this.bufferOwner.Memory[..bufferLength]; + this.buffer = this.outputBuffer; + this.updateCdf = updateCdf; + } + + /// + /// Restores the initial range-coder state and begins a new output sequence. + /// + public void Reset() => this.Reset(0); + + /// + /// Restores the initial range-coder state and begins writing at an offset in the retained output allocation. + /// + /// The first byte available to the next range-coded tile. + public void Reset(int outputOffset) + { + this.buffer = this.outputBuffer[outputOffset..]; + this.low = 0; + this.rng = InitialRange; + this.cnt = InitialCount; + this.position = 0; + } + + /// + /// Releases the tile output buffer. + /// + public void Dispose() => this.bufferOwner.Dispose(); + + /// + /// Writes one binary symbol and adapts its distribution when CDF updates are enabled. + /// + /// The binary symbol. + /// The inverse cumulative distribution for the binary alphabet. + public void WriteSymbol(bool symbol, Av1Distribution distribution) + => this.WriteSymbol(symbol ? 1 : 0, distribution); + + /// + /// Writes one symbol and adapts its distribution when CDF updates are enabled. + /// + /// The zero-based symbol. + /// The inverse cumulative distribution for the symbol alphabet. + public void WriteSymbol(int symbol, Av1Distribution distribution) + { + DebugGuard.MustBeGreaterThanOrEqualTo(symbol, 0, nameof(symbol)); + DebugGuard.MustBeLessThan(symbol, distribution.NumberOfSymbols, nameof(symbol)); + DebugGuard.IsTrue(distribution[distribution.NumberOfSymbols - 1] == 0, "Last entry in Probabilities table needs to be zero."); + + this.EncodeIntegerQ15(symbol, distribution); + + // disable_cdf_update freezes every tile distribution while leaving range encoding unchanged. + if (this.updateCdf) + { + distribution.Update(symbol); + } + } + + /// + /// Writes one non-adaptive binary symbol using the supplied Q15 probability for . + /// + /// The binary symbol. + /// The probability that the symbol is , scaled by 32768. + public void WriteBoolean(bool value, uint frequency) => this.EncodeBoolQ15(value, frequency); + + /// + /// Writes one equiprobable literal bit. + /// + /// The literal bit. + public void WriteLiteral(bool value) => this.WriteLiteral(value ? 1u : 0u, 1); + + /// + /// Writes the requested low-order bits in most-significant-bit-first order. + /// + /// The unsigned literal value. + /// The number of low-order bits to write. + public void WriteLiteral(uint value, int bitCount) + { + const uint p = 0x4000U; // (0x7FFFFFU - (128 << 15) + 128) >> 8; + for (int bit = bitCount - 1; bit >= 0; bit--) + { + bool bitValue = ((value >> bit) & 0x1) > 0; + this.EncodeBoolQ15(bitValue, p); + } + } + + /// + /// Terminates the range-coded sequence and propagates pending carries into an owned byte buffer. + /// + /// An owner containing the shortest byte sequence that preserves every encoded symbol. + public IMemoryOwner Exit() + { + int length = this.FinalizeRange(); + IMemoryOwner output = this.configuration.MemoryAllocator.Allocate(length); + this.buffer.Span[..length].CopyTo(output.Memory.Span); + + return output; + } + + /// + /// Finalizes the range-coded sequence and exposes its encoded prefix without copying. + /// + /// The number of encoded bytes in the returned memory. + /// The encoded prefix, valid until this writer is reset or disposed. + public ReadOnlyMemory Exit(out int length) + { + length = this.FinalizeRange(); + return this.buffer[..length]; + } + + /// + /// Exposes a prefix containing consecutively encoded tiles without copying their bytes. + /// + /// The number of bytes in the prefix. + /// The encoded prefix, valid until this writer is reset or disposed. + public ReadOnlyMemory GetOutput(int length) => this.outputBuffer[..length]; + + /// + /// Terminates the range-coded sequence in the current output allocation. + /// + /// The number of encoded bytes in the allocation. + private int FinalizeRange() + { + // Round the low endpoint into the current interval so the emitted prefix selects every symbol encoded so far, + // regardless of the bits that follow it. + ulong l = this.low; + int c = this.cnt; + int pos = this.position; + int s = 10; + ulong m = 0x3FFFU; + ulong e = ((l + m) & ~m) | (m + 1); + s += c; + int pendingByteCount = Math.Max((s + 7) >> 3, 0); + if (pos + pendingByteCount > this.buffer.Length) + { + // Finalization needs only the terminating bytes; ordinary word flushes reserve their own headroom. + this.ResizeBuffer(pos + pendingByteCount); + } + + Span buffer = this.buffer.Span[..(pos + pendingByteCount)]; + if (s > 0) + { + ulong n = (1UL << (c + 16)) - 1; + do + { + ushort value = (ushort)(e >> (c + 16)); + buffer[pos] = (byte)value; + if ((value & 0x100) != 0) + { + PropagateCarryBackward(buffer, pos - 1); + } + + pos++; + e &= n; + s -= 8; + c -= 8; + n >>= 8; + } + while (s > 0); + } + + return pos; + } + + /// + /// Encode a single binary value. + /// + /// The value to encode. + /// The probability that the value is true, scaled by 32768. + private void EncodeBoolQ15(bool val, uint frequency) + { + ulong l; + uint r; + uint v; + DebugGuard.MustBeGreaterThan(frequency, 0U, nameof(frequency)); + DebugGuard.MustBeLessThanOrEqualTo(frequency, 32768U, nameof(frequency)); + l = this.low; + r = this.rng; + DebugGuard.MustBeGreaterThanOrEqualTo(r, 32768U, nameof(r)); + + // Reduce the Q15 frequency to the range-coder multiplication precision and retain a nonzero interval for + // both outcomes. Av1SymbolReader applies the identical rounding model. + v = ((r >> 8) * (frequency >> Av1Distribution.ProbabilityShift)) >> (7 - Av1Distribution.ProbabilityShift); + v += Av1Distribution.ProbabilityMinimum; + if (val) + { + l += r - v; + r = v; + } + else + { + r -= v; + } + + this.Normalize(l, r); + } + + /// + /// Encodes a symbol given an inverse cumulative distribution function(CDF) table in Q15. + /// + /// The value to encode. + /// + /// CDF_PROB_TOP minus the CDF, such that symbol s falls in the range + /// [s > 0 ? (CDF_PROB_TOP - icdf[s - 1]) : 0, CDF_PROB_TOP - icdf[s]). + /// The values must be monotonically non - increasing, and icdf[nsyms - 1] must be 0. + /// + private void EncodeIntegerQ15(int symbol, Av1Distribution distribution) + => this.EncodeIntegerQ15(symbol > 0 ? distribution[symbol - 1] : Av1Distribution.ProbabilityTop, distribution[symbol], symbol, distribution.NumberOfSymbols); + + /// + /// Narrows the coding interval to one symbol's inverse-cumulative bounds. + /// + /// The inverse cumulative threshold preceding the symbol. + /// The inverse cumulative threshold following the symbol. + /// The zero-based symbol. + /// The size of the symbol alphabet. + private void EncodeIntegerQ15(uint lowFrequency, uint highFrequency, int symbol, int numberOfSymbols) + { + const int totalShift = 7 - Av1Distribution.ProbabilityShift - Av1Distribution.CdfShift; + ulong l = this.low; + uint r = this.rng; + DebugGuard.MustBeLessThanOrEqualTo(32768U, r, nameof(r)); + DebugGuard.MustBeLessThanOrEqualTo(highFrequency, lowFrequency, nameof(highFrequency)); + DebugGuard.MustBeLessThanOrEqualTo(lowFrequency, 32768U, nameof(lowFrequency)); + DebugGuard.MustBeGreaterThanOrEqualTo(totalShift, 0, nameof(totalShift)); + int n = numberOfSymbols - 1; + if (lowFrequency < Av1Distribution.ProbabilityTop) + { + uint u; + uint v; + u = (uint)((((r >> 8) * (lowFrequency >> Av1Distribution.ProbabilityShift)) >> totalShift) + + (Av1Distribution.ProbabilityMinimum * (n - (symbol - 1)))); + v = (uint)((((r >> 8) * (highFrequency >> Av1Distribution.ProbabilityShift)) >> totalShift) + + (Av1Distribution.ProbabilityMinimum * (n - symbol))); + l += r - u; + r = u - v; + } + else + { + r -= (uint)((((r >> 8) * (highFrequency >> Av1Distribution.ProbabilityShift)) >> totalShift) + + (Av1Distribution.ProbabilityMinimum * (n - symbol))); + } + + this.Normalize(l, r); + } + + /// + /// Takes updated low and range values, renormalizes them so that + /// lies between 32768 and 65536 (flushing bytes from low to the pre-carry buffer if necessary), + /// and stores them back in the encoder context. + /// + /// The new value of . + /// The new value of . + private void Normalize(ulong low, uint rng) + { + int c = this.cnt; + DebugGuard.MustBeLessThanOrEqualTo(rng, 65535U, nameof(rng)); + int d = 15 - Av1Math.MostSignificantBit(rng); + int s = c + d; + + // Keeping 16 bits free for the next symbol allows the 64-bit coding window to flush up to eight completed + // bytes together while preserving one carry bit. + if (s >= 40) + { + if (this.position + sizeof(ulong) > this.buffer.Length) + { + // A word store touches eight bytes even when fewer become logical output. Double the current + // tile capacity and add one word, matching the range coder's amortized growth from an empty buffer. + this.ResizeBuffer(checked((2 * this.buffer.Length) + sizeof(ulong))); + } + + Span buffer = this.buffer.Span[..(this.position + sizeof(ulong))]; + int readyByteCount = (s >> 3) + 1; + c += 24 - (readyByteCount << 3); + ulong output = low >> c; + low &= (1UL << c) - 1; + ulong carryMask = 1UL << (readyByteCount << 3); + bool hasCarry = (output & carryMask) != 0; + output &= carryMask - 1; + + // Writing one big-endian word avoids a byte-at-a-time hot loop. Only readyByteCount bytes become part + // of the logical output; the following bytes are overwritten by the next flush. + BinaryPrimitives.WriteUInt64BigEndian( + buffer.Slice(this.position, sizeof(ulong)), + output << ((sizeof(ulong) - readyByteCount) << 3)); + + if (hasCarry) + { + PropagateCarryBackward(buffer, this.position - 1); + } + + this.position += readyByteCount; + s = c + d - 24; + } + + this.low = low << d; + this.rng = rng << d; + this.cnt = s; + } + + /// + /// Replaces the output owner while retaining finalized tiles and the current tile's completed bytes. + /// + /// The required capacity starting at the current tile's output offset. + private void ResizeBuffer(int tileCapacity) + { + int outputOffset = this.outputBuffer.Length - this.buffer.Length; + int capacity = checked(outputOffset + tileCapacity); + IMemoryOwner replacement = this.configuration.MemoryAllocator.Allocate(capacity); + Memory replacementBuffer = replacement.Memory[..capacity]; + + // Previous tile bytes remain part of the frame payload. The current tile's completed prefix also carries + // backward into earlier bytes, so preserve that prefix before returning the old owner. Pending bits stay + // in low/cnt and need no copy. If allocation fails, the original owner remains available for disposal. + this.outputBuffer.Span[..(outputOffset + this.position)].CopyTo(replacementBuffer.Span); + this.bufferOwner.Dispose(); + this.bufferOwner = replacement; + this.outputBuffer = replacementBuffer; + this.buffer = replacementBuffer[outputOffset..]; + } + + /// + /// Adds a carry to the completed output prefix. + /// + /// The accumulated output bytes. + /// The final completed byte. + private static void PropagateCarryBackward(Span buffer, int offset) + { + int carry; + do + { + int sum = buffer[offset] + 1; + buffer[offset] = (byte)sum; + carry = sum >> 8; + offset--; + } + while (carry != 0); + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/IAv1TileReader.cs b/src/ImageSharp/Formats/Heif/Av1/IAv1TileReader.cs new file mode 100644 index 0000000000..7168cdaa1f --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/IAv1TileReader.cs @@ -0,0 +1,22 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1; + +/// +/// Defines tile-payload consumption and completion for one coded AV1 frame. +/// +internal interface IAv1TileReader +{ + /// + /// Reads one entropy-coded tile payload into the current frame state. + /// + /// The bounded bitstream bytes belonging to the tile. + /// The zero-based tile index in raster order. + void ReadTile(Span tileData, int tileNum); + + /// + /// Completes the current coded frame after all tile payloads have been read and releases frame-scoped resources. + /// + void CompleteFrame(); +} diff --git a/src/ImageSharp/Formats/Heif/Av1/IAv1TileWriter.cs b/src/ImageSharp/Formats/Heif/Av1/IAv1TileWriter.cs new file mode 100644 index 0000000000..ade1acffa3 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/IAv1TileWriter.cs @@ -0,0 +1,19 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1; + +/// +/// Interface for writing of image tiles. +/// +internal interface IAv1TileWriter +{ + /// + /// Gets the encoded bytes for a single tile. + /// + /// The index of the encoded tile. + /// + /// The bytes of encoded data in the bitstream dedicated to this tile. + /// + ReadOnlySpan GetTileData(int tileNum); +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Motion/Av1GlobalMotionParameters.cs b/src/ImageSharp/Formats/Heif/Av1/Motion/Av1GlobalMotionParameters.cs new file mode 100644 index 0000000000..5e9d7f6030 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Motion/Av1GlobalMotionParameters.cs @@ -0,0 +1,530 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Numerics; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Motion; + +/// +/// Stores one AV1 global-motion model in the codec's fixed-point affine matrix domain. +/// +internal struct Av1GlobalMotionParameters +{ + /// + /// The number of fractional bits carried by every stored matrix parameter. + /// + public const int ModelPrecisionBits = 16; + + /// + /// The fixed-point representation of one in the global-motion matrix domain. + /// + public const int ModelScale = 1 << ModelPrecisionBits; + + /// + /// The initial finite-subexponential group width used by every global-motion parameter. + /// + public const int SubexponentialGroupBitCount = 3; + + /// + /// The finite signed-domain size parameter for coded affine coefficients. + /// + public const int AlphaValueMagnitude = (1 << 12) + 1; + + /// + /// The number of fractional bits carried by coded affine coefficients. + /// + public const int AlphaPrecisionBits = 15; + + /// + /// The precision increase from a coded affine coefficient to the stored matrix. + /// + public const int AlphaPrecisionDifference = ModelPrecisionBits - AlphaPrecisionBits; + + /// + /// The scale factor that restores a coded affine coefficient to the stored matrix precision. + /// + public const int AlphaDecodeFactor = 1 << AlphaPrecisionDifference; + + /// + /// The signed magnitude bit count of a general affine model's translation components. + /// + public const int AbsoluteTranslationBits = 12; + + /// + /// The signed magnitude bit count of a translation-only model before precision adjustment. + /// + public const int AbsoluteTranslationOnlyBits = 9; + + /// + /// The number of fractional bits carried by general affine translation components. + /// + public const int TranslationPrecisionBits = 6; + + /// + /// The number of fractional bits carried by translation-only components. + /// + public const int TranslationOnlyPrecisionBits = 3; + + /// + /// The number of low-order bits removed from the derived shear parameters. + /// + private const int ShearParameterReductionBits = 6; + + /// + /// The number of fractional bits carried by entries in . + /// + private const int ReciprocalPrecisionBits = 14; + + /// + /// The number of divisor-fraction bits used to index . + /// + private const int ReciprocalIndexBits = 8; + + /// + /// The largest difference between a retained neighbor motion vector and the current block motion vector. + /// + private const int LocalProjectionMotionVectorLimit = 256; + + /// + /// The maximum magnitude of a non-diagonal affine coefficient relative to the identity matrix. + /// + private const int NonDiagonalAffineClamp = 1 << (ModelPrecisionBits - 3); + + /// + /// The exclusive upper magnitude of either translation coefficient. + /// + private const int TranslationClamp = 128 << ModelPrecisionBits; + + /// + /// The six parameters ordered as horizontal translation, vertical translation, and the four affine coefficients. + /// + private InlineArray6 matrix; + + /// + /// Gets an identity global-motion model. + /// + public static Av1GlobalMotionParameters Identity + { + get + { + Av1GlobalMotionParameters result = default; + result.matrix[2] = ModelScale; + result.matrix[5] = ModelScale; + return result; + } + } + + /// + /// Gets or sets the geometric model represented by the matrix parameters. + /// + public Av1GlobalMotionType Type { get; set; } + + /// + /// Gets the reduced horizontal scale delta used by warped prediction. + /// + public short Alpha { get; private set; } + + /// + /// Gets the reduced horizontal shear used by warped prediction. + /// + public short Beta { get; private set; } + + /// + /// Gets the reduced vertical shear used by warped prediction. + /// + public short Gamma { get; private set; } + + /// + /// Gets the reduced vertical scale delta used by warped prediction. + /// + public short Delta { get; private set; } + + /// + /// Gets a value indicating whether the affine model violates AV1's permitted shear bounds. + /// + public bool IsInvalid { get; private set; } + + /// + /// Gets the fixed-point reciprocal lookup used by AV1's affine shear derivation. + /// + private static ReadOnlySpan ReciprocalTable => + [ + 16384, 16320, 16257, 16194, 16132, 16070, 16009, 15948, 15888, 15828, 15768, + 15709, 15650, 15592, 15534, 15477, 15420, 15364, 15308, 15252, 15197, 15142, + 15087, 15033, 14980, 14926, 14873, 14821, 14769, 14717, 14665, 14614, 14564, + 14513, 14463, 14413, 14364, 14315, 14266, 14218, 14170, 14122, 14075, 14028, + 13981, 13935, 13888, 13843, 13797, 13752, 13707, 13662, 13618, 13574, 13530, + 13487, 13443, 13400, 13358, 13315, 13273, 13231, 13190, 13148, 13107, 13066, + 13026, 12985, 12945, 12906, 12866, 12827, 12788, 12749, 12710, 12672, 12633, + 12596, 12558, 12520, 12483, 12446, 12409, 12373, 12336, 12300, 12264, 12228, + 12193, 12157, 12122, 12087, 12053, 12018, 11984, 11950, 11916, 11882, 11848, + 11815, 11782, 11749, 11716, 11683, 11651, 11619, 11586, 11555, 11523, 11491, + 11460, 11429, 11398, 11367, 11336, 11305, 11275, 11245, 11215, 11185, 11155, + 11125, 11096, 11067, 11038, 11009, 10980, 10951, 10923, 10894, 10866, 10838, + 10810, 10782, 10755, 10727, 10700, 10673, 10645, 10618, 10592, 10565, 10538, + 10512, 10486, 10460, 10434, 10408, 10382, 10356, 10331, 10305, 10280, 10255, + 10230, 10205, 10180, 10156, 10131, 10107, 10082, 10058, 10034, 10010, 9986, + 9963, 9939, 9916, 9892, 9869, 9846, 9823, 9800, 9777, 9754, 9732, 9709, 9687, + 9664, 9642, 9620, 9598, 9576, 9554, 9533, 9511, 9489, 9468, 9447, 9425, 9404, + 9383, 9362, 9341, 9321, 9300, 9279, 9259, 9239, 9218, 9198, 9178, 9158, 9138, + 9118, 9098, 9079, 9059, 9039, 9020, 9001, 8981, 8962, 8943, 8924, 8905, 8886, + 8867, 8849, 8830, 8812, 8793, 8775, 8756, 8738, 8720, 8702, 8684, 8666, 8648, + 8630, 8613, 8595, 8577, 8560, 8542, 8525, 8508, 8490, 8473, 8456, 8439, 8422, + 8405, 8389, 8372, 8355, 8339, 8322, 8306, 8289, 8273, 8257, 8240, 8224, 8208, + 8192, + ]; + + /// + /// Gets or sets a matrix parameter in AV1 affine-transform order. + /// + /// The zero-based matrix parameter index. + /// The fixed-point matrix parameter. + public int this[int index] + { + get => this.matrix[index]; + set => this.matrix[index] = value; + } + + /// + /// Gets the translational motion vector represented by this model at the center of a coding block. + /// + /// + /// A value indicating whether motion vectors may retain one-eighth-sample precision. + /// + /// The coding block size. + /// The block origin in 4x4 mode-information units. + /// + /// A value indicating whether the result is rounded to an integer-sample displacement. + /// + /// The global motion vector in one-eighth-sample units. + public readonly Av1MotionVector GetMotionVector( + bool allowHighPrecisionMotionVector, + Av1BlockSize blockSize, + Point modeInfoPosition, + bool forceIntegerMotionVector) + { + if (this.Type == Av1GlobalMotionType.Identity) + { + return default; + } + + int row; + int column; + if (this.Type == Av1GlobalMotionType.Translation) + { + // AV1 accidentally assigns the horizontal translation parameter to the row component and the vertical + // parameter to the column component. Decoders preserve that published bitstream behavior for conformance. + row = this.matrix[0] >> (ModelPrecisionBits - 3); + column = this.matrix[1] >> (ModelPrecisionBits - 3); + } + else + { + int blockCenterX = (modeInfoPosition.X << Av1Constants.ModeInfoSizeLog2) + (blockSize.GetWidth() >> 1) - 1; + int blockCenterY = (modeInfoPosition.Y << Av1Constants.ModeInfoSizeLog2) + (blockSize.GetHeight() >> 1) - 1; + int horizontal = ((this.matrix[2] - ModelScale) * blockCenterX) + + (this.matrix[3] * blockCenterY) + + this.matrix[0]; + + int vertical = (this.matrix[4] * blockCenterX) + + ((this.matrix[5] - ModelScale) * blockCenterY) + + this.matrix[1]; + + int precisionBits = allowHighPrecisionMotionVector ? ModelPrecisionBits - 3 : ModelPrecisionBits - 2; + column = Av1Math.RoundPowerOf2Signed(horizontal, precisionBits); + row = Av1Math.RoundPowerOf2Signed(vertical, precisionBits); + if (!allowHighPrecisionMotionVector) + { + column *= 2; + row *= 2; + } + } + + return new Av1MotionVector(row, column).LowerPrecision( + allowHighPrecision: allowHighPrecisionMotionVector, + forceInteger: forceIntegerMotionVector); + } + + /// + /// Derives the local affine model for a warped inter block from its spatial neighbor samples. + /// + /// The neighbor-center positions relative to the current block in one-eighth-sample units. + /// The corresponding positions in the selected reference frame. + /// The current coding block size. + /// The current block motion vector in one-eighth-sample units. + /// The current block origin in 4x4 mode-information units. + /// The derived affine model, marked invalid when AV1's projection or shear constraints cannot be satisfied. + public static Av1GlobalMotionParameters DeriveLocalProjection( + ReadOnlySpan sourcePoints, + ReadOnlySpan referencePoints, + Av1BlockSize blockSize, + Av1MotionVector motionVector, + Point modeInfoPosition) + { + Av1GlobalMotionParameters result = Identity; + result.Type = Av1GlobalMotionType.Affine; + + int blockWidth = blockSize.GetWidth(); + int blockHeight = blockSize.GetHeight(); + int sampleThreshold = Math.Clamp(Math.Max(blockWidth, blockHeight), 16, 112); + bool hasSelectedSample = sourcePoints.Length == 1; + if (sourcePoints.Length > 1) + { + for (int index = 0; index < sourcePoints.Length; index++) + { + int difference = Math.Abs(referencePoints[index].X - sourcePoints[index].X - motionVector.Column) + + Math.Abs(referencePoints[index].Y - sourcePoints[index].Y - motionVector.Row); + + hasSelectedSample |= difference <= sampleThreshold; + } + } + + int sourceCenterX = ((blockWidth >> 1) - 1) << 3; + int sourceCenterY = ((blockHeight >> 1) - 1) << 3; + int referenceCenterX = sourceCenterX + motionVector.Column; + int referenceCenterY = sourceCenterY + motionVector.Row; + int a00 = 0; + int a01 = 0; + int a11 = 0; + int bx0 = 0; + int bx1 = 0; + int by0 = 0; + int by1 = 0; + + for (int index = 0; index < sourcePoints.Length; index++) + { + int motionVectorDifference = Math.Abs(referencePoints[index].X - sourcePoints[index].X - motionVector.Column) + + Math.Abs(referencePoints[index].Y - sourcePoints[index].Y - motionVector.Row); + + // av1_selectSamples retains the original first sample when every candidate exceeds the threshold. Keeping + // that rule here is important because the selected Warped syntax still requires a deterministic model. + if (sourcePoints.Length > 1 && motionVectorDifference > sampleThreshold && (hasSelectedSample || index != 0)) + { + continue; + } + + int sourceX = sourcePoints[index].X - sourceCenterX; + int sourceY = sourcePoints[index].Y - sourceCenterY; + int referenceX = referencePoints[index].X - referenceCenterX; + int referenceY = referencePoints[index].Y - referenceCenterY; + if (Math.Abs(sourceX - referenceX) >= LocalProjectionMotionVectorLimit || + Math.Abs(sourceY - referenceY) >= LocalProjectionMotionVectorLimit) + { + continue; + } + + // These biased products are the normative reduced-precision P'P, P'q, and P'r matrices. Computing them + // directly preserves the reference decoder's integer least-squares rounding instead of introducing floating-point drift. + a00 += LeastSquaresSquare(sourceX); + a01 += LeastSquaresProduct1(sourceX, sourceY); + a11 += LeastSquaresSquare(sourceY); + bx0 += LeastSquaresProduct2(sourceX, referenceX); + bx1 += LeastSquaresProduct1(sourceY, referenceX); + by0 += LeastSquaresProduct1(sourceX, referenceY); + by1 += LeastSquaresProduct2(sourceY, referenceY); + } + + long determinant = ((long)a00 * a11) - ((long)a01 * a01); + if (determinant == 0) + { + result.IsInvalid = true; + return result; + } + + int inverseDeterminant = ResolveDivisor((ulong)Math.Abs(determinant), out int determinantShift) * + (determinant < 0 ? -1 : 1); + + determinantShift -= ModelPrecisionBits; + if (determinantShift < 0) + { + inverseDeterminant <<= -determinantShift; + determinantShift = 0; + } + + long projectionX0 = ((long)a11 * bx0) - ((long)a01 * bx1); + long projectionX1 = -((long)a01 * bx0) + ((long)a00 * bx1); + long projectionY0 = ((long)a11 * by0) - ((long)a01 * by1); + long projectionY1 = -((long)a01 * by0) + ((long)a00 * by1); + + result.matrix[2] = ResolveProjectionCoefficient( + projectionX0, + inverseDeterminant, + determinantShift, + ModelScale - NonDiagonalAffineClamp + 1, + ModelScale + NonDiagonalAffineClamp - 1); + + result.matrix[3] = ResolveProjectionCoefficient( + projectionX1, + inverseDeterminant, + determinantShift, + -NonDiagonalAffineClamp + 1, + NonDiagonalAffineClamp - 1); + + result.matrix[4] = ResolveProjectionCoefficient( + projectionY0, + inverseDeterminant, + determinantShift, + -NonDiagonalAffineClamp + 1, + NonDiagonalAffineClamp - 1); + + result.matrix[5] = ResolveProjectionCoefficient( + projectionY1, + inverseDeterminant, + determinantShift, + ModelScale - NonDiagonalAffineClamp + 1, + ModelScale + NonDiagonalAffineClamp - 1); + + int absoluteCenterX = (modeInfoPosition.X << Av1Constants.ModeInfoSizeLog2) + (blockWidth >> 1) - 1; + int absoluteCenterY = (modeInfoPosition.Y << Av1Constants.ModeInfoSizeLog2) + (blockHeight >> 1) - 1; + int horizontalTranslation = (motionVector.Column << (ModelPrecisionBits - 3)) - + (absoluteCenterX * (result.matrix[2] - ModelScale)) - + (absoluteCenterY * result.matrix[3]); + + int verticalTranslation = (motionVector.Row << (ModelPrecisionBits - 3)) - + (absoluteCenterX * result.matrix[4]) - + (absoluteCenterY * (result.matrix[5] - ModelScale)); + + result.matrix[0] = Math.Clamp(horizontalTranslation, -TranslationClamp, TranslationClamp - 1); + result.matrix[1] = Math.Clamp(verticalTranslation, -TranslationClamp, TranslationClamp - 1); + result.UpdateShearParameters(); + return result; + } + + /// + /// Derives the reduced shear parameters and records whether the complete affine model is valid. + /// + public void UpdateShearParameters() + { + Span values = this.matrix; + this.Alpha = 0; + this.Beta = 0; + this.Gamma = 0; + this.Delta = 0; + + if (values[2] <= 0) + { + this.IsInvalid = true; + return; + } + + this.Alpha = (short)Math.Clamp(values[2] - ModelScale, short.MinValue, short.MaxValue); + this.Beta = (short)Math.Clamp(values[3], short.MinValue, short.MaxValue); + + // AV1 derives gamma and delta by multiplying with a fixed-precision reciprocal of the horizontal scale. + // The reciprocal lookup is normative; integer division would produce different warped sample positions. + int reciprocal = ResolveDivisor((uint)values[2], out int reciprocalShift); + long scaledVerticalCoefficient = (long)values[4] * ModelScale * reciprocal; + this.Gamma = (short)Math.Clamp(RoundPowerOf2Signed(scaledVerticalCoefficient, reciprocalShift), short.MinValue, short.MaxValue); + + long scaledCrossCoefficient = (long)values[3] * values[4] * reciprocal; + long verticalScaleDelta = values[5] - RoundPowerOf2Signed(scaledCrossCoefficient, reciprocalShift) - ModelScale; + this.Delta = (short)Math.Clamp(verticalScaleDelta, short.MinValue, short.MaxValue); + + // Warped filtering addresses a coarser parameter grid than the stored affine matrix. Symmetric rounding is + // required here so negative shear values are quantized identically to their positive counterparts. + this.Alpha = ReduceShearParameter(this.Alpha); + this.Beta = ReduceShearParameter(this.Beta); + this.Gamma = ReduceShearParameter(this.Gamma); + this.Delta = ReduceShearParameter(this.Delta); + + // These weighted L1 bounds are the AV1 validity test for the two shear axes. Equality is invalid because the + // warped-filter footprint would no longer remain inside the permitted affine sampling envelope. + this.IsInvalid = + ((4 * Math.Abs((int)this.Alpha)) + (7 * Math.Abs((int)this.Beta)) >= ModelScale) || + ((4 * Math.Abs((int)this.Gamma)) + (4 * Math.Abs((int)this.Delta)) >= ModelScale); + } + + /// + /// Quantizes one signed shear parameter to AV1's warped-filter precision. + /// + /// The full-precision shear parameter. + /// The reduced shear parameter. + private static short ReduceShearParameter(short value) + => (short)(RoundPowerOf2Signed(value, ShearParameterReductionBits) * (1 << ShearParameterReductionBits)); + + /// + /// Resolves a positive divisor into AV1's fixed-point reciprocal representation. + /// + /// The positive divisor. + /// Receives the reciprocal's binary scale. + /// The fixed-point reciprocal multiplier. + private static int ResolveDivisor(uint divisor, out int shift) + { + // Normalize the divisor around its highest set bit, then quantize the remaining fraction to the normative + // eight-bit table index. Adding the table's fourteen fractional bits yields the scale used by the caller's + // rounded multiply instead of a platform-dependent integer division. + shift = BitOperations.Log2(divisor); + int remainder = (int)(divisor - (1U << shift)); + int reciprocalIndex = shift > ReciprocalIndexBits + ? RoundPowerOf2(remainder, shift - ReciprocalIndexBits) + : remainder << (ReciprocalIndexBits - shift); + + shift += ReciprocalPrecisionBits; + return ReciprocalTable[reciprocalIndex]; + } + + /// + /// Resolves a positive 64-bit divisor into AV1's fixed-point reciprocal representation. + /// + /// The positive divisor. + /// Receives the reciprocal's binary scale. + /// The fixed-point reciprocal multiplier. + private static int ResolveDivisor(ulong divisor, out int shift) + { + shift = BitOperations.Log2(divisor); + ulong remainder = divisor - (1UL << shift); + int reciprocalIndex = shift > ReciprocalIndexBits + ? (int)((remainder + (1UL << (shift - ReciprocalIndexBits - 1))) >> (shift - ReciprocalIndexBits)) + : (int)(remainder << (ReciprocalIndexBits - shift)); + + shift += ReciprocalPrecisionBits; + return ReciprocalTable[reciprocalIndex]; + } + + /// + /// Resolves one adjugate numerator into a clamped affine matrix coefficient. + /// + private static int ResolveProjectionCoefficient(long numerator, int inverseDeterminant, int shift, int minimum, int maximum) + { + long product = numerator * inverseDeterminant; + long value = shift > 0 ? RoundPowerOf2Signed(product, shift) : product << -shift; + return (int)Math.Clamp(value, minimum, maximum); + } + + /// + /// Computes one reduced-precision diagonal element of the local projection matrix. + /// + private static int LeastSquaresSquare(int value) + => ((value * value * 4) + (value * 32) + 128) >> 4; + + /// + /// Computes one reduced-precision off-diagonal product of the local projection matrix. + /// + private static int LeastSquaresProduct1(int first, int second) + => ((first * second * 4) + ((first + second) * 16) + 64) >> 4; + + /// + /// Computes one reduced-precision source-to-reference product of the local projection matrix. + /// + private static int LeastSquaresProduct2(int first, int second) + => ((first * second * 4) + ((first + second) * 16) + 128) >> 4; + + /// + /// Divides a nonnegative integer by a power of two with nearest-integer rounding. + /// + /// The nonnegative value. + /// The base-two divisor exponent. + /// The rounded quotient. + private static int RoundPowerOf2(int value, int bitCount) + => (value + ((1 << bitCount) >> 1)) >> bitCount; + + /// + /// Divides a signed integer by a power of two with symmetric nearest-integer rounding. + /// + /// The signed value. + /// The base-two divisor exponent. + /// The rounded quotient. + private static long RoundPowerOf2Signed(long value, int bitCount) + => value < 0 + ? -(((-value) + ((1L << bitCount) >> 1)) >> bitCount) + : (value + ((1L << bitCount) >> 1)) >> bitCount; +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Motion/Av1GlobalMotionType.cs b/src/ImageSharp/Formats/Heif/Av1/Motion/Av1GlobalMotionType.cs new file mode 100644 index 0000000000..59cb14d7fe --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Motion/Av1GlobalMotionType.cs @@ -0,0 +1,30 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Motion; + +/// +/// Identifies the geometric model carried by AV1 global-motion parameters. +/// +internal enum Av1GlobalMotionType : byte +{ + /// + /// No geometric displacement is applied. + /// + Identity = 0, + + /// + /// Horizontal and vertical translation are applied. + /// + Translation = 1, + + /// + /// Translation, rotation, and uniform zoom are applied. + /// + RotationZoom = 2, + + /// + /// A general six-parameter affine transformation is applied. + /// + Affine = 3 +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Motion/Av1IntraBlockCopy.cs b/src/ImageSharp/Formats/Heif/Av1/Motion/Av1IntraBlockCopy.cs new file mode 100644 index 0000000000..d2db7a8d10 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Motion/Av1IntraBlockCopy.cs @@ -0,0 +1,626 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Motion; + +/// +/// Derives and validates AV1 intra-block-copy displacement vectors. +/// +internal static class Av1IntraBlockCopy +{ + /// + /// The number of surrounding mode-information rows and columns searched for reference vectors. + /// + private const int ReferenceSearchDistance = 3; + + /// + /// The weight separating immediately adjacent candidates from the outer search area. + /// + private const int NearestCandidateWeight = 640; + + /// + /// The number of 64-sample blocks that an intra-block-copy source must precede the active block. + /// + private const int Delay64 = 4; + + /// + /// Finds the spatial reference used to differentially decode an intra-block-copy displacement vector. + /// + /// The current block geometry and decoded neighbors. + /// The active tile boundaries. + /// The superblock width in 4x4 mode-information units. + /// Reusable storage for up to eight unique reference vectors. + /// Reusable storage for the corresponding spatial weights. + /// The nearest nonzero spatial candidate, or the normative tile-relative fallback. + public static Av1MotionVector FindReference( + ref Av1PartitionInfo partitionInfo, + Av1TileInfo tileInfo, + int superblockModeInfoSize, + Span candidates, + Span weights) + { + ReferenceContext context = new(ref partitionInfo, superblockModeInfoSize); + return FindReference(ref context, tileInfo, superblockModeInfoSize, candidates, weights); + } + + /// + /// Finds the spatial reference used to differentially encode an intra-block-copy displacement vector. + /// + /// The encoded frame's mapped mode and displacement state. + /// The current block's frame edges and tile availability. + /// The current block origin in 4x4 mode-information units. + /// The current block size. + /// The partition type that produced the block. + /// Reusable storage for up to eight unique reference vectors. + /// Reusable storage for the corresponding spatial weights. + /// The nearest nonzero spatial candidate, or the normative tile-relative fallback. + public static Av1MotionVector FindReference( + Av1PictureControlSet picture, + Av1MacroBlockD macroBlock, + Point modeInfoPosition, + Av1BlockSize blockSize, + Av1PartitionType partitionType, + Span candidates, + Span weights) + { + int superblockModeInfoSize = picture.Sequence.SequenceHeader.SuperblockModeInfoSize; + ReferenceContext context = new( + picture, + macroBlock, + modeInfoPosition, + blockSize, + partitionType, + superblockModeInfoSize); + + return FindReference( + ref context, + macroBlock.Tile, + superblockModeInfoSize, + candidates, + weights); + } + + /// + /// Ranks the shared decoder or encoder reference context without allocating candidate state. + /// + private static Av1MotionVector FindReference( + ref ReferenceContext context, + Av1TileInfo tileInfo, + int superblockModeInfoSize, + Span candidates, + Span weights) + { + Av1BlockSize blockSize = context.BlockSize; + int width = blockSize.Get4x4WideCount(); + int height = blockSize.Get4x4HighCount(); + int row = context.RowIndex; + int column = context.ColumnIndex; + int rowAdjustment = height < 2 && (row & 1) != 0 ? 1 : 0; + int columnAdjustment = width < 2 && (column & 1) != 0 ? 1 : 0; + int maximumRowOffset = 0; + int maximumColumnOffset = 0; + + if (context.AvailableAbove) + { + maximumRowOffset = height < 2 ? -4 + rowAdjustment : -(ReferenceSearchDistance << 1) + rowAdjustment; + maximumRowOffset = Math.Clamp(maximumRowOffset, tileInfo.ModeInfoRowStart - row, tileInfo.ModeInfoRowEnd - row - 1); + } + + if (context.AvailableLeft) + { + maximumColumnOffset = width < 2 ? -4 + columnAdjustment : -(ReferenceSearchDistance << 1) + columnAdjustment; + maximumColumnOffset = Math.Clamp(maximumColumnOffset, tileInfo.ModeInfoColumnStart - column, tileInfo.ModeInfoColumnEnd - column - 1); + } + + int candidateCount = 0; + int processedRows = 0; + int processedColumns = 0; + if (Math.Abs(maximumRowOffset) >= 1) + { + ScanRow(ref context, -1, maximumRowOffset, candidates, weights, ref candidateCount, ref processedRows); + } + + if (Math.Abs(maximumColumnOffset) >= 1) + { + ScanColumn(ref context, -1, maximumColumnOffset, candidates, weights, ref candidateCount, ref processedColumns); + } + + if (context.HasTopRight) + { + AddBlock(ref context, -1, width, tileInfo, candidates, weights, ref candidateCount); + } + + int nearestCandidateCount = candidateCount; + for (int index = 0; index < nearestCandidateCount; index++) + { + weights[index] += NearestCandidateWeight; + } + + // The top-left sample begins the outer search region. Sorting the adjacent and outer regions independently + // preserves the reference decoder's nearest/near ordering while still accumulating repeated vectors across both regions. + AddBlock(ref context, -1, -1, tileInfo, candidates, weights, ref candidateCount); + for (int index = 2; index <= ReferenceSearchDistance; index++) + { + int rowOffset = -(index << 1) + 1 + rowAdjustment; + int columnOffset = -(index << 1) + 1 + columnAdjustment; + if (Math.Abs(rowOffset) <= Math.Abs(maximumRowOffset) && Math.Abs(rowOffset) > processedRows) + { + ScanRow(ref context, rowOffset, maximumRowOffset, candidates, weights, ref candidateCount, ref processedRows); + } + + if (Math.Abs(columnOffset) <= Math.Abs(maximumColumnOffset) && Math.Abs(columnOffset) > processedColumns) + { + ScanColumn(ref context, columnOffset, maximumColumnOffset, candidates, weights, ref candidateCount, ref processedColumns); + } + } + + SortByWeight(candidates, weights, 0, nearestCandidateCount); + SortByWeight(candidates, weights, nearestCandidateCount, candidateCount); + + // The reference decoder clamps the ranked stack before selecting nearest and near. The displacement entropy syntax is + // differential, so using an unclamped spatial candidate changes every following component even though the + // final decoded displacement is validated separately against the stricter intra-block-copy source limits. + for (int index = 0; index < candidateCount; index++) + { + candidates[index] = candidates[index].ClampReference( + blockSize.GetWidth(), + blockSize.GetHeight(), + context.ModeBlockToLeftEdge, + context.ModeBlockToRightEdge, + context.ModeBlockToTopEdge, + context.ModeBlockToBottomEdge); + } + + Av1MotionVector reference = candidateCount > 0 ? candidates[0] : default; + if (reference.IsZero && candidateCount > 1) + { + reference = candidates[1]; + } + + if (!reference.IsZero) + { + return reference; + } + + const int modeInfoSampleSize = 1 << Av1Constants.ModeInfoSizeLog2; + const int eighthSampleScale = 8; + int fallbackRow = -modeInfoSampleSize * superblockModeInfoSize * eighthSampleScale; + int fallbackColumn = fallbackRow - (Delay64 * 64 * eighthSampleScale); + + return (row - superblockModeInfoSize) < tileInfo.ModeInfoRowStart + ? new Av1MotionVector(0, fallbackColumn) + : new Av1MotionVector(fallbackRow, 0); + } + + /// + /// Determines whether a decoded displacement vector references an earlier reconstructable block inside the tile. + /// + /// The decoded displacement vector in one-eighth-sample units. + /// The current block geometry. + /// The active tile boundaries. + /// The sequence-level superblock and chroma configuration. + /// when the complete source block is a permitted reference; otherwise, . + public static bool IsValid(Av1MotionVector vector, ref Av1PartitionInfo partitionInfo, Av1TileInfo tileInfo, ObuSequenceHeader sequenceHeader) + => IsValid( + vector, + new Point(partitionInfo.ColumnIndex, partitionInfo.RowIndex), + partitionInfo.ModeInfo.BlockSize, + partitionInfo.IsChroma, + tileInfo, + sequenceHeader); + + /// + /// Determines whether an encoder displacement vector references an earlier reconstructable block inside the tile. + /// + /// The displacement vector in one-eighth-sample units. + /// The current block origin in 4x4 mode-information units. + /// The current block size. + /// Indicates whether chroma subsampling constraints apply. + /// The active tile boundaries. + /// The sequence-level superblock and chroma configuration. + /// when the complete source block is a permitted reference; otherwise, . + public static bool IsValid( + Av1MotionVector vector, + Point modeInfoPosition, + Av1BlockSize blockSize, + bool isChroma, + Av1TileInfo tileInfo, + ObuSequenceHeader sequenceHeader) + { + const int eighthSampleScale = 8; + const int modeInfoSampleSize = 1 << Av1Constants.ModeInfoSizeLog2; + if ((vector.Row & (eighthSampleScale - 1)) != 0 || (vector.Column & (eighthSampleScale - 1)) != 0 || + vector.Row <= -(1 << 14) || vector.Row >= (1 << 14) || vector.Column <= -(1 << 14) || vector.Column >= (1 << 14)) + { + return false; + } + + int row = modeInfoPosition.Y; + int column = modeInfoPosition.X; + int blockWidth = blockSize.GetWidth(); + int blockHeight = blockSize.GetHeight(); + int sourceTop = (row * modeInfoSampleSize * eighthSampleScale) + vector.Row; + int sourceLeft = (column * modeInfoSampleSize * eighthSampleScale) + vector.Column; + int sourceBottom = (((row * modeInfoSampleSize) + blockHeight) * eighthSampleScale) + vector.Row; + int sourceRight = (((column * modeInfoSampleSize) + blockWidth) * eighthSampleScale) + vector.Column; + int tileTop = tileInfo.ModeInfoRowStart * modeInfoSampleSize * eighthSampleScale; + int tileLeft = tileInfo.ModeInfoColumnStart * modeInfoSampleSize * eighthSampleScale; + int tileBottom = tileInfo.ModeInfoRowEnd * modeInfoSampleSize * eighthSampleScale; + int tileRight = tileInfo.ModeInfoColumnEnd * modeInfoSampleSize * eighthSampleScale; + if (sourceTop < tileTop || sourceLeft < tileLeft || sourceBottom > tileBottom || sourceRight > tileRight) + { + return false; + } + + ObuColorConfig colorConfig = sequenceHeader.ColorConfig; + if (isChroma && colorConfig.PlaneCount > 1) + { + // A sub-8x8 luma block can map to a chroma block whose rounded origin lies one additional luma unit + // inside the tile. These checks prevent that chroma reference from crossing the tile boundary. + if (blockWidth < 8 && colorConfig.SubSamplingX && sourceLeft < tileLeft + (modeInfoSampleSize * eighthSampleScale)) + { + return false; + } + + if (blockHeight < 8 && colorConfig.SubSamplingY && sourceTop < tileTop + (modeInfoSampleSize * eighthSampleScale)) + { + return false; + } + } + + int superblockModeInfoSize = sequenceHeader.SuperblockModeInfoSize; + int superblockSize = superblockModeInfoSize * modeInfoSampleSize; + int superblockModeInfoSizeLog2 = sequenceHeader.SuperblockSizeLog2 - Av1Constants.ModeInfoSizeLog2; + int activeSuperblockRow = row >> superblockModeInfoSizeLog2; + int active64Column = (column * modeInfoSampleSize) >> 6; + int sourceSuperblockRow = ((sourceBottom >> 3) - 1) / superblockSize; + int source64Column = ((sourceRight >> 3) - 1) >> 6; + int tile64ColumnCount = ((tileInfo.ModeInfoColumnEnd - tileInfo.ModeInfoColumnStart - 1) >> 4) + 1; + int active64 = (activeSuperblockRow * tile64ColumnCount) + active64Column; + int source64 = (sourceSuperblockRow * tile64ColumnCount) + source64Column; + if (source64 >= active64 - Delay64) + { + return false; + } + + // The wavefront boundary reserves four completed 64-sample columns and advances farther right for every + // completed source row. A 128x128 superblock adds one column to account for its two 64-sample halves. + int gradient = 1 + Delay64 + (superblockSize > 64 ? 1 : 0); + int wavefrontOffset = gradient * (activeSuperblockRow - sourceSuperblockRow); + return sourceSuperblockRow <= activeSuperblockRow && source64Column < active64Column - Delay64 + wavefrontOffset; + } + + /// + /// Scans a mode-information row using AV1's block-size-dependent steps and weights. + /// + private static void ScanRow( + ref ReferenceContext context, + int rowOffset, + int maximumRowOffset, + Span candidates, + Span weights, + ref int candidateCount, + ref int processedRows) + { + int width = context.BlockSize.Get4x4WideCount(); + int end = Math.Min(context.GetMaxBlockWide(), 16); + int columnOffset = 0; + if (Math.Abs(rowOffset) > 1) + { + columnOffset = 1; + if ((context.ColumnIndex & 1) != 0 && width < 2) + { + columnOffset--; + } + } + + // Blocks below 64 samples use the finer two-mode-info-unit scan step. + bool useFourUnitStep = width >= 16; + for (int index = 0; index < end;) + { + ReferenceBlock candidate = context.GetModeInfoAt( + new Point(context.ColumnIndex + columnOffset + index, context.RowIndex + rowOffset)); + + int candidateWidth = candidate.BlockSize.Get4x4WideCount(); + int length = Math.Min(width, candidateWidth); + if (useFourUnitStep) + { + length = Math.Max(4, length); + } + else if (Math.Abs(rowOffset) > 1) + { + length = Math.Max(2, length); + } + + int weight = 2; + if (width >= 2 && width <= candidateWidth) + { + int increment = Math.Min(-maximumRowOffset + rowOffset + 1, candidate.BlockSize.Get4x4HighCount()); + weight = Math.Max(weight, increment); + processedRows = increment - rowOffset - 1; + } + + AddCandidate(candidate, length * weight, candidates, weights, ref candidateCount); + index += length; + } + } + + /// + /// Scans a mode-information column using AV1's block-size-dependent steps and weights. + /// + private static void ScanColumn( + ref ReferenceContext context, + int columnOffset, + int maximumColumnOffset, + Span candidates, + Span weights, + ref int candidateCount, + ref int processedColumns) + { + int height = context.BlockSize.Get4x4HighCount(); + int end = Math.Min(context.GetMaxBlockHigh(), 16); + int rowOffset = 0; + if (Math.Abs(columnOffset) > 1) + { + rowOffset = 1; + if ((context.RowIndex & 1) != 0 && height < 2) + { + rowOffset--; + } + } + + // Blocks below 64 samples use the finer two-mode-info-unit scan step. + bool useFourUnitStep = height >= 16; + for (int index = 0; index < end;) + { + ReferenceBlock candidate = context.GetModeInfoAt( + new Point(context.ColumnIndex + columnOffset, context.RowIndex + rowOffset + index)); + + int candidateHeight = candidate.BlockSize.Get4x4HighCount(); + int length = Math.Min(height, candidateHeight); + if (useFourUnitStep) + { + length = Math.Max(4, length); + } + else if (Math.Abs(columnOffset) > 1) + { + length = Math.Max(2, length); + } + + int weight = 2; + if (height >= 2 && height <= candidateHeight) + { + int increment = Math.Min(-maximumColumnOffset + columnOffset + 1, candidate.BlockSize.Get4x4WideCount()); + weight = Math.Max(weight, increment); + processedColumns = increment - columnOffset - 1; + } + + AddCandidate(candidate, length * weight, candidates, weights, ref candidateCount); + index += length; + } + } + + /// + /// Adds the intra-block-copy vector at one tile-relative search position. + /// + private static void AddBlock( + ref ReferenceContext context, + int rowOffset, + int columnOffset, + Av1TileInfo tileInfo, + Span candidates, + Span weights, + ref int candidateCount) + { + int row = context.RowIndex + rowOffset; + int column = context.ColumnIndex + columnOffset; + if (row < tileInfo.ModeInfoRowStart || row >= tileInfo.ModeInfoRowEnd || + column < tileInfo.ModeInfoColumnStart || column >= tileInfo.ModeInfoColumnEnd) + { + return; + } + + ReferenceBlock candidate = context.GetModeInfoAt(new Point(column, row)); + AddCandidate(candidate, 4, candidates, weights, ref candidateCount); + } + + /// + /// Accumulates one unique intra-block-copy candidate and its spatial weight. + /// + private static void AddCandidate( + ReferenceBlock candidate, + int weight, + Span candidates, + Span weights, + ref int candidateCount) + { + if (!candidate.UseIntraBlockCopy) + { + return; + } + + Av1MotionVector vector = candidate.DisplacementVector; + int index = 0; + for (; index < candidateCount; index++) + { + if (candidates[index] == vector) + { + weights[index] += weight; + return; + } + } + + if (candidateCount < candidates.Length) + { + candidates[candidateCount] = vector; + weights[candidateCount] = weight; + candidateCount++; + } + } + + /// + /// Sorts one candidate region by descending accumulated weight. + /// + private static void SortByWeight(Span candidates, Span weights, int start, int end) + { + int length = end; + while (length > start) + { + int lastSwap = start; + for (int index = start + 1; index < length; index++) + { + if (weights[index - 1] < weights[index]) + { + Av1MotionVector candidate = candidates[index - 1]; + candidates[index - 1] = candidates[index]; + candidates[index] = candidate; + + int weight = weights[index - 1]; + weights[index - 1] = weights[index]; + weights[index] = weight; + lastSwap = index; + } + } + + length = lastSwap; + } + } + + /// + /// Provides one allocation-free view over decoder or encoder mode-information storage. + /// + private readonly struct ReferenceContext + { + private readonly Av1SuperblockInfo decodedSuperblock; + private readonly Av1PictureControlSet? encodedPicture; + + public ReferenceContext(ref Av1PartitionInfo partitionInfo, int superblockModeInfoSize) + { + this.decodedSuperblock = partitionInfo.SuperblockInfo; + this.encodedPicture = null; + this.BlockSize = partitionInfo.ModeInfo.BlockSize; + this.RowIndex = partitionInfo.RowIndex; + this.ColumnIndex = partitionInfo.ColumnIndex; + this.AvailableAbove = partitionInfo.AvailableAbove; + this.AvailableLeft = partitionInfo.AvailableLeft; + this.ModeBlockToLeftEdge = partitionInfo.ModeBlockToLeftEdge; + this.ModeBlockToRightEdge = partitionInfo.ModeBlockToRightEdge; + this.ModeBlockToTopEdge = partitionInfo.ModeBlockToTopEdge; + this.ModeBlockToBottomEdge = partitionInfo.ModeBlockToBottomEdge; + this.HasTopRight = partitionInfo.HasTopRight(superblockModeInfoSize); + } + + public ReferenceContext( + Av1PictureControlSet picture, + Av1MacroBlockD macroBlock, + Point modeInfoPosition, + Av1BlockSize blockSize, + Av1PartitionType partitionType, + int superblockModeInfoSize) + { + this.decodedSuperblock = default; + this.encodedPicture = picture; + this.BlockSize = blockSize; + this.RowIndex = modeInfoPosition.Y; + this.ColumnIndex = modeInfoPosition.X; + this.AvailableAbove = macroBlock.IsUpAvailable; + this.AvailableLeft = macroBlock.IsLeftAvailable; + this.ModeBlockToLeftEdge = macroBlock.ToLeftEdge; + this.ModeBlockToRightEdge = macroBlock.ToRightEdge; + this.ModeBlockToTopEdge = macroBlock.ToTopEdge; + this.ModeBlockToBottomEdge = macroBlock.ToBottomEdge; + this.HasTopRight = Av1PartitionInfo.HasTopRight( + blockSize, + partitionType, + modeInfoPosition.Y, + modeInfoPosition.X, + superblockModeInfoSize); + } + + public Av1BlockSize BlockSize { get; } + + public int RowIndex { get; } + + public int ColumnIndex { get; } + + public bool AvailableAbove { get; } + + public bool AvailableLeft { get; } + + public int ModeBlockToLeftEdge { get; } + + public int ModeBlockToRightEdge { get; } + + public int ModeBlockToTopEdge { get; } + + public int ModeBlockToBottomEdge { get; } + + public bool HasTopRight { get; } + + public int GetMaxBlockWide() + { + int width = this.BlockSize.GetWidth(); + if (this.ModeBlockToRightEdge < 0) + { + width += this.ModeBlockToRightEdge >> 3; + } + + return width >> Av1Constants.ModeInfoSizeLog2; + } + + public int GetMaxBlockHigh() + { + int height = this.BlockSize.GetHeight(); + if (this.ModeBlockToBottomEdge < 0) + { + height += this.ModeBlockToBottomEdge >> 3; + } + + return height >> Av1Constants.ModeInfoSizeLog2; + } + + public ReferenceBlock GetModeInfoAt(Point position) + { + Av1PictureControlSet? picture = this.encodedPicture; + if (picture is not null) + { + Av1MacroBlockModeInfo encodedModeInfo = picture.GetFromModeInfoGrid(position); + return new ReferenceBlock( + encodedModeInfo.Block.BlockSize, + encodedModeInfo.Block.UseIntraBlockCopy, + picture.GetDisplacementVector(position)); + } + + Av1BlockModeInfo decodedModeInfo = this.decodedSuperblock.GetModeInfoAt(position); + return new ReferenceBlock( + decodedModeInfo.BlockSize, + decodedModeInfo.UseIntraBlockCopy, + decodedModeInfo.DisplacementVector); + } + } + + /// + /// Carries the three neighboring mode fields consumed by displacement-reference ranking. + /// + private readonly struct ReferenceBlock + { + public ReferenceBlock( + Av1BlockSize blockSize, + bool useIntraBlockCopy, + Av1MotionVector displacementVector) + { + this.BlockSize = blockSize; + this.UseIntraBlockCopy = useIntraBlockCopy; + this.DisplacementVector = displacementVector; + } + + public Av1BlockSize BlockSize { get; } + + public bool UseIntraBlockCopy { get; } + + public Av1MotionVector DisplacementVector { get; } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Motion/Av1IntraBlockCopySearchIndex.cs b/src/ImageSharp/Formats/Heif/Av1/Motion/Av1IntraBlockCopySearchIndex.cs new file mode 100644 index 0000000000..f92b4f92fb --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Motion/Av1IntraBlockCopySearchIndex.cs @@ -0,0 +1,1083 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.InteropServices; +using SixLabors.ImageSharp.Formats.Heif.Av1.Entropy; +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; +using SixLabors.ImageSharp.Memory; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Motion; + +/// +/// Indexes visible 8x8 luma blocks for intra-block-copy motion search. +/// +internal readonly struct Av1IntraBlockCopySearchIndex +{ + private const int BlockSize = 8; + private const int MaximumBucketCount = 1 << 16; + private const int MaximumCandidatesPerBucket = 256; + private const int MaximumFullPixelSearchOffset = (1 << 10) - 1; + private const int MinimumFullPixelMotionVector = -(1 << 11) + 1; + private const int MaximumFullPixelMotionVector = (1 << 11) - 1; + private const int ExhaustiveSearchRange = 256; + private const int ExhaustiveSearchThreshold = 1 << 12; + private const int ExhaustiveSearchBatchSize = 4; + private const uint HorizontalHashMultiplier = 257; + private const uint VerticalHashMultiplier = 65599; + private static readonly uint HorizontalLeadingWeight = GetLeadingWeight(HorizontalHashMultiplier); + private static readonly uint VerticalLeadingWeight = GetLeadingWeight(VerticalHashMultiplier); + private readonly Memory storage; + private readonly int hashLinkLength; + private readonly int bucketCount; + private readonly int headOffset; + private readonly int tailOffset; + private readonly int countOffset; + + /// + /// Initializes a new instance of the struct over picture-lifetime storage. + /// + /// The packed hash-link and bucket storage. + /// The visible luma width. + /// The visible luma height. + public Av1IntraBlockCopySearchIndex(Memory storage, int width, int height) + { + this.OriginWidth = Math.Max(0, width - BlockSize + 1); + this.OriginHeight = Math.Max(0, height - BlockSize + 1); + this.hashLinkLength = this.OriginWidth == 0 || this.OriginHeight == 0 + ? 0 + : checked(this.OriginWidth * height); + + this.bucketCount = GetBucketCount(this.OriginWidth, this.OriginHeight); + this.headOffset = checked(this.hashLinkLength * sizeof(int)); + this.tailOffset = checked(this.headOffset + (this.bucketCount * sizeof(int))); + this.countOffset = checked(this.tailOffset + (this.bucketCount * sizeof(int))); + this.storage = storage; + } + + /// + /// Defines sample-width-specific search arithmetic for the closed generic encoder path. + /// + /// The native unsigned sample storage type. + internal interface ISearchOperation + where TSample : unmanaged + { + /// + /// Converts one native sample into the unsigned hash domain. + /// + /// The sample to convert. + /// The unsigned sample value. + public static abstract uint GetHashSample(TSample sample); + + /// + /// Compares two complete 8x8 blocks. + /// + /// The plane containing both blocks. + /// The first block origin. + /// The second block origin. + /// when every sample is equal. + public static abstract bool BlocksEqual(Buffer2DRegion plane, Point first, Point second); + + /// + /// Gets the sum of absolute differences between the source block and reconstructed predictor. + /// + /// The coded source plane. + /// The source block origin. + /// The reconstructed luma plane. + /// The predictor block origin. + /// The unnormalized absolute difference over the complete 8x8 block. + public static abstract int GetSumOfAbsoluteDifferences( + Buffer2DRegion source, + Point sourceOrigin, + Buffer2DRegion reconstruction, + Point predictionOrigin); + + /// + /// Gets four sums of absolute differences for horizontally adjacent reconstructed predictors. + /// + /// The coded source plane. + /// The source block origin. + /// The reconstructed luma plane. + /// The first of four horizontally adjacent predictor origins. + /// Storage receiving the four unnormalized absolute differences. + public static abstract void GetFourSumsOfAbsoluteDifferences( + Buffer2DRegion source, + Point sourceOrigin, + Buffer2DRegion reconstruction, + Point firstPredictionOrigin, + Span sums); + + /// + /// Gets the normalized 8x8 variance between a source block and reconstructed predictor. + /// + /// The coded source plane. + /// The source block origin. + /// The reconstructed luma plane. + /// The predictor block origin. + /// The coded sample precision. + /// The variance in the eight-bit distortion domain. + public static abstract int GetVariance( + Buffer2DRegion source, + Point sourceOrigin, + Buffer2DRegion reconstruction, + Point predictionOrigin, + Av1BitDepth bitDepth); + } + + /// + /// Gets the visible horizontal origin count represented by the index. + /// + public int OriginWidth { get; } + + /// + /// Gets the visible vertical origin count represented by the index. + /// + public int OriginHeight { get; } + + /// + /// Gets the expanding NSTEP radii from the final one-pixel refinement through the largest search step. + /// + private static ReadOnlySpan SearchRadii => [1, 2, 3, 5, 8, 12, 18, 27, 41, 62, 93, 140, 210, 210, 210]; + + /// + /// Gets the tangential radius paired with each NSTEP primary radius. + /// + private static ReadOnlySpan SearchTangentialRadii => [1, 2, 3, 5, 3, 4, 7, 11, 16, 25, 38, 57, 86, 86, 86]; + + /// + /// Gets the packed storage length required for a visible frame. + /// + /// The visible luma width. + /// The visible luma height. + /// The required byte length. + public static int GetStorageLength(int width, int height) + { + int originWidth = Math.Max(0, width - BlockSize + 1); + int originHeight = Math.Max(0, height - BlockSize + 1); + if (originWidth == 0 || originHeight == 0) + { + return 0; + } + + int hashLinkLength = checked(originWidth * height); + int bucketCount = GetBucketCount(originWidth, originHeight); + return checked( + (hashLinkLength * sizeof(int)) + + (bucketCount * sizeof(int) * 2) + + (bucketCount * sizeof(ushort))); + } + + /// + /// Builds the complete visible-frame hash index into its picture-lifetime storage. + /// + /// The native unsigned sample storage type. + /// The closed sample operation. + /// The coded source luma plane. + public void Initialize(Buffer2DRegion source) + where TSample : unmanaged + where TOperation : struct, ISearchOperation + { + if (this.hashLinkLength == 0) + { + return; + } + + Span hashesAndLinks = this.GetHashesAndLinks(); + Span heads = this.GetHeads(); + Span tails = this.GetTails(); + Span counts = this.GetCounts(); + heads.Clear(); + tails.Clear(); + counts.Clear(); + + for (int row = 0; row < source.Height; row++) + { + ReadOnlySpan sourceRow = source.DangerousGetRowSpan(row); + int hashRowOffset = row * this.OriginWidth; + uint hash = 0; + for (int column = 0; column < BlockSize; column++) + { + hash = unchecked((hash * HorizontalHashMultiplier) + TOperation.GetHashSample(sourceRow[column])); + } + + hashesAndLinks[hashRowOffset] = (int)hash; + for (int column = 1; column < this.OriginWidth; column++) + { + uint previous = TOperation.GetHashSample(sourceRow[column - 1]); + uint next = TOperation.GetHashSample(sourceRow[column + BlockSize - 1]); + hash = unchecked(((hash - (previous * HorizontalLeadingWeight)) * HorizontalHashMultiplier) + next); + hashesAndLinks[hashRowOffset + column] = (int)hash; + } + } + + for (int column = 0; column < this.OriginWidth; column++) + { + uint hash = 0; + for (int row = 0; row < BlockSize; row++) + { + hash = unchecked((hash * VerticalHashMultiplier) + (uint)hashesAndLinks[(row * this.OriginWidth) + column]); + } + + for (int row = 0; row < this.OriginHeight; row++) + { + int position = (row * this.OriginWidth) + column; + uint previous = (uint)hashesAndLinks[position]; + hashesAndLinks[position] = (int)hash; + if (row + 1 < this.OriginHeight) + { + uint next = (uint)hashesAndLinks[((row + BlockSize) * this.OriginWidth) + column]; + hash = unchecked(((hash - (previous * VerticalLeadingWeight)) * VerticalHashMultiplier) + next); + } + } + } + + // Coarse-to-fine insertion disperses the first 256 identical blocks across the image instead of + // retaining one dense cluster. Links occupy the hash workspace after every hash has been derived. + int step = BlockSize; + int columnOffset = 0; + int rowOffset = 0; + while (step > 1) + { + for (int column = columnOffset; column < this.OriginWidth; column += step) + { + for (int row = rowOffset; row < this.OriginHeight; row += step) + { + int position = (row * this.OriginWidth) + column; + int bucket = hashesAndLinks[position] & (this.bucketCount - 1); + if (counts[bucket] < MaximumCandidatesPerBucket) + { + int encodedPosition = position + 1; + hashesAndLinks[position] = 0; + if (heads[bucket] == 0) + { + heads[bucket] = encodedPosition; + } + else + { + hashesAndLinks[tails[bucket] - 1] = encodedPosition; + } + + tails[bucket] = encodedPosition; + counts[bucket]++; + } + } + } + + if (columnOffset == 0 && rowOffset == 0) + { + columnOffset = step / 2; + } + else if (columnOffset == step / 2 && rowOffset == 0) + { + columnOffset = 0; + rowOffset = step / 2; + } + else if (columnOffset == 0 && rowOffset == step / 2) + { + columnOffset = step / 2; + } + else + { + step /= 2; + columnOffset = step / 2; + rowOffset = 0; + } + } + } + + /// + /// Finds the best exact-source hash candidate in the reference above and left search regions. + /// + /// The native unsigned sample storage type. + /// The closed sample operation. + /// The coded source luma plane. + /// The coded reconstructed luma plane. + /// The current 8x8 block origin. + /// The active tile boundaries. + /// The sequence geometry and sample precision. + /// The live tile entropy model used for displacement rate. + /// The spatial displacement-vector reference. + /// The active rate-distortion multiplier. + /// Storage receiving the above candidate followed by the left candidate. + /// The number of candidates written. + public int FindCandidates( + Buffer2DRegion source, + Buffer2DRegion reconstruction, + Point blockOrigin, + Av1TileInfo tile, + ObuSequenceHeader sequenceHeader, + Av1SymbolEncoder writer, + Av1MotionVector reference, + int rateMultiplier, + Span candidates) + where TSample : unmanaged + where TOperation : struct, ISearchOperation + { + if (this.hashLinkLength == 0) + { + return 0; + } + + const int ModeInfoSampleSize = 1 << Av1Constants.ModeInfoSizeLog2; + int tileLeft = tile.ModeInfoColumnStart * ModeInfoSampleSize; + int tileTop = tile.ModeInfoRowStart * ModeInfoSampleSize; + int tileRight = tile.ModeInfoColumnEnd * ModeInfoSampleSize; + int tileBottom = tile.ModeInfoRowEnd * ModeInfoSampleSize; + int superblockSize = sequenceHeader.SuperblockSize.GetWidth(); + int superblockLeft = (blockOrigin.X / superblockSize) * superblockSize; + int superblockTop = (blockOrigin.Y / superblockSize) * superblockSize; + int candidateCount = 0; + + if (this.TryFindCandidate( + source, + reconstruction, + blockOrigin, + tile, + sequenceHeader, + writer, + reference, + rateMultiplier, + tileLeft, + tileTop, + tileRight - BlockSize, + superblockTop - BlockSize, + out Av1MotionVector above)) + { + candidates[candidateCount++] = above; + } + + if (this.TryFindCandidate( + source, + reconstruction, + blockOrigin, + tile, + sequenceHeader, + writer, + reference, + rateMultiplier, + tileLeft, + tileTop, + superblockLeft - BlockSize, + Math.Min(superblockTop + superblockSize, tileBottom) - BlockSize, + out Av1MotionVector left)) + { + candidates[candidateCount++] = left; + } + + return candidateCount; + } + + /// + /// Finds the best full-pixel NSTEP candidate in the reference above and left search regions. + /// + /// The native unsigned sample storage type. + /// The closed sample operation. + /// The coded source luma plane. + /// The coded reconstructed luma plane. + /// The current 8x8 block origin. + /// The active tile boundaries. + /// The sequence geometry and sample precision. + /// The live tile entropy model used for displacement rate. + /// The spatial displacement-vector reference. + /// The effective segment quantizer index. + /// The active rate-distortion multiplier. + /// Storage receiving the above candidate followed by the left candidate. + /// The number of candidates written. + public int FindPixelCandidates( + Buffer2DRegion source, + Buffer2DRegion reconstruction, + Point blockOrigin, + Av1TileInfo tile, + ObuSequenceHeader sequenceHeader, + Av1SymbolEncoder writer, + Av1MotionVector reference, + int qIndex, + int rateMultiplier, + Span candidates) + where TSample : unmanaged + where TOperation : struct, ISearchOperation + { + if (this.OriginWidth == 0 || this.OriginHeight == 0) + { + return 0; + } + + const int ModeInfoSampleSize = 1 << Av1Constants.ModeInfoSizeLog2; + int tileLeft = tile.ModeInfoColumnStart * ModeInfoSampleSize; + int tileTop = tile.ModeInfoRowStart * ModeInfoSampleSize; + int tileRight = Math.Min((tile.ModeInfoColumnEnd * ModeInfoSampleSize) - BlockSize, this.OriginWidth - 1); + int tileBottom = Math.Min((tile.ModeInfoRowEnd * ModeInfoSampleSize) - BlockSize, this.OriginHeight - 1); + int superblockSize = sequenceHeader.SuperblockSize.GetWidth(); + int superblockLeft = (blockOrigin.X / superblockSize) * superblockSize; + int superblockTop = (blockOrigin.Y / superblockSize) * superblockSize; + int searchStepParameter = GetSearchStepParameter(Math.Max(this.OriginWidth + BlockSize - 1, this.OriginHeight + BlockSize - 1)); + int sadPerBit = Av1RateDistortion.GetMotionSearchSadPerBit(qIndex, sequenceHeader.ColorConfig.BitDepth); + int candidateCount = 0; + + if (TryFindPixelCandidate( + source, + reconstruction, + blockOrigin, + tile, + sequenceHeader, + writer, + reference, + rateMultiplier, + sadPerBit, + searchStepParameter, + tileLeft, + tileTop, + tileRight, + Math.Min(superblockTop - BlockSize, tileBottom), + out Av1MotionVector above)) + { + candidates[candidateCount++] = above; + } + + if (TryFindPixelCandidate( + source, + reconstruction, + blockOrigin, + tile, + sequenceHeader, + writer, + reference, + rateMultiplier, + sadPerBit, + searchStepParameter, + tileLeft, + tileTop, + Math.Min(superblockLeft - BlockSize, tileRight), + Math.Min(superblockTop + superblockSize - BlockSize, tileBottom), + out Av1MotionVector left)) + { + candidates[candidateCount++] = left; + } + + return candidateCount; + } + + private static uint GetLeadingWeight(uint multiplier) + { + uint result = 1; + for (int i = 1; i < BlockSize; i++) + { + result = unchecked(result * multiplier); + } + + return result; + } + + private static int GetBucketCount(int originWidth, int originHeight) + { + int originCount = checked(originWidth * originHeight); + if (originCount == 0) + { + return 0; + } + + // One power-of-two bucket per possible origin avoids libaom's fixed multi-megabyte pointer table + // on small images while retaining its 16-bit upper bound and constant-time mask lookup. + return originCount >= MaximumBucketCount + ? MaximumBucketCount + : 1 << (int)Av1Math.CeilLog2((uint)originCount); + } + + private static uint GetBlockHash(Buffer2DRegion source, Point origin) + where TSample : unmanaged + where TOperation : struct, ISearchOperation + { + uint blockHash = 0; + for (int row = 0; row < BlockSize; row++) + { + ReadOnlySpan sourceRow = source.DangerousGetRowSpan(origin.Y + row); + uint rowHash = 0; + for (int column = 0; column < BlockSize; column++) + { + rowHash = unchecked( + (rowHash * HorizontalHashMultiplier) + + TOperation.GetHashSample(sourceRow[origin.X + column])); + } + + blockHash = unchecked((blockHash * VerticalHashMultiplier) + rowHash); + } + + return blockHash; + } + + private bool TryFindCandidate( + Buffer2DRegion source, + Buffer2DRegion reconstruction, + Point blockOrigin, + Av1TileInfo tile, + ObuSequenceHeader sequenceHeader, + Av1SymbolEncoder writer, + Av1MotionVector reference, + int rateMultiplier, + int minimumColumn, + int minimumRow, + int maximumColumn, + int maximumRow, + out Av1MotionVector bestVector) + where TSample : unmanaged + where TOperation : struct, ISearchOperation + { + bestVector = default; + if (maximumColumn < minimumColumn || maximumRow < minimumRow) + { + return false; + } + + uint blockHash = GetBlockHash(source, blockOrigin); + int bucket = (int)(blockHash & (this.bucketCount - 1)); + Span hashesAndLinks = this.GetHashesAndLinks(); + int encodedPosition = this.GetHeads()[bucket]; + int bestCost = int.MaxValue; + bool found = false; + Point modeInfoPosition = new( + blockOrigin.X >> Av1Constants.ModeInfoSizeLog2, + blockOrigin.Y >> Av1Constants.ModeInfoSizeLog2); + + while (encodedPosition != 0) + { + int position = encodedPosition - 1; + int row = position / this.OriginWidth; + int column = position - (row * this.OriginWidth); + Point candidateOrigin = new(column, row); + encodedPosition = hashesAndLinks[position]; + if (column < minimumColumn || column > maximumColumn || row < minimumRow || row > maximumRow || + !TOperation.BlocksEqual(source, blockOrigin, candidateOrigin)) + { + continue; + } + + Av1MotionVector vector = new( + (row - blockOrigin.Y) * 8, + (column - blockOrigin.X) * 8); + + if (!Av1IntraBlockCopy.IsValid( + vector, + modeInfoPosition, + Av1BlockSize.Block8x8, + isChroma: false, + tile, + sequenceHeader)) + { + continue; + } + + int variance = TOperation.GetVariance( + source, + blockOrigin, + reconstruction, + candidateOrigin, + sequenceHeader.ColorConfig.BitDepth); + + int rate = writer.GetDisplacementVectorSearchCost(vector, reference); + int cost = Av1RateDistortion.GetMotionSearchCost(rateMultiplier, rate, variance); + if (cost < bestCost) + { + bestCost = cost; + bestVector = vector; + found = true; + } + } + + return found; + } + + private static bool TryFindPixelCandidate( + Buffer2DRegion source, + Buffer2DRegion reconstruction, + Point blockOrigin, + Av1TileInfo tile, + ObuSequenceHeader sequenceHeader, + Av1SymbolEncoder writer, + Av1MotionVector reference, + int rateMultiplier, + int sadPerBit, + int searchStepParameter, + int minimumColumn, + int minimumRow, + int maximumColumn, + int maximumRow, + out Av1MotionVector bestVector) + where TSample : unmanaged + where TOperation : struct, ISearchOperation + { + bestVector = default; + if (maximumColumn < minimumColumn || maximumRow < minimumRow) + { + return false; + } + + int referenceColumn = reference.Column >> 3; + int referenceRow = reference.Row >> 3; + int minimumColumnOffset = Math.Max( + Math.Max(minimumColumn - blockOrigin.X, referenceColumn - MaximumFullPixelSearchOffset), + MinimumFullPixelMotionVector); + + int maximumColumnOffset = Math.Min( + Math.Min(maximumColumn - blockOrigin.X, referenceColumn + MaximumFullPixelSearchOffset), + MaximumFullPixelMotionVector); + + int minimumRowOffset = Math.Max( + Math.Max(minimumRow - blockOrigin.Y, referenceRow - MaximumFullPixelSearchOffset), + MinimumFullPixelMotionVector); + + int maximumRowOffset = Math.Min( + Math.Min(maximumRow - blockOrigin.Y, referenceRow + MaximumFullPixelSearchOffset), + MaximumFullPixelMotionVector); + + if (maximumColumnOffset < minimumColumnOffset || maximumRowOffset < minimumRowOffset) + { + return false; + } + + Point start = new( + Av1Math.Clamp(referenceColumn, minimumColumnOffset, maximumColumnOffset), + Av1Math.Clamp(referenceRow, minimumRowOffset, maximumRowOffset)); + + Point best = FindBestPixelCandidate( + source, + reconstruction, + blockOrigin, + writer, + reference, + sequenceHeader.ColorConfig.BitDepth, + rateMultiplier, + sadPerBit, + searchStepParameter, + minimumColumnOffset, + minimumRowOffset, + maximumColumnOffset, + maximumRowOffset, + start); + + bestVector = new(best.Y * 8, best.X * 8); + Point modeInfoPosition = new( + blockOrigin.X >> Av1Constants.ModeInfoSizeLog2, + blockOrigin.Y >> Av1Constants.ModeInfoSizeLog2); + + return Av1IntraBlockCopy.IsValid( + bestVector, + modeInfoPosition, + Av1BlockSize.Block8x8, + isChroma: false, + tile, + sequenceHeader); + } + + private static Point FindBestPixelCandidate( + Buffer2DRegion source, + Buffer2DRegion reconstruction, + Point blockOrigin, + Av1SymbolEncoder writer, + Av1MotionVector reference, + Av1BitDepth bitDepth, + int rateMultiplier, + int sadPerBit, + int searchStepParameter, + int minimumColumnOffset, + int minimumRowOffset, + int maximumColumnOffset, + int maximumRowOffset, + Point start) + where TSample : unmanaged + where TOperation : struct, ISearchOperation + { + SearchDiamond( + source, + reconstruction, + blockOrigin, + writer, + reference, + sadPerBit, + searchStepParameter, + minimumColumnOffset, + minimumRowOffset, + maximumColumnOffset, + maximumRowOffset, + start, + out Point best, + out int centerSteps); + + int bestCost = GetVarianceCost( + source, + reconstruction, + blockOrigin, + writer, + reference, + bitDepth, + rateMultiplier, + best); + + int furtherSteps = SearchRadii.Length - 1 - searchStepParameter; + int shortenedBy = centerSteps; + while (shortenedBy < furtherSteps) + { + shortenedBy++; + SearchDiamond( + source, + reconstruction, + blockOrigin, + writer, + reference, + sadPerBit, + searchStepParameter + shortenedBy, + minimumColumnOffset, + minimumRowOffset, + maximumColumnOffset, + maximumRowOffset, + start, + out Point candidate, + out int additionalCenterSteps); + + int candidateCost = GetVarianceCost( + source, + reconstruction, + blockOrigin, + writer, + reference, + bitDepth, + rateMultiplier, + candidate); + + if (candidateCost < bestCost) + { + bestCost = candidateCost; + best = candidate; + } + + shortenedBy += additionalCenterSteps; + } + + // Intra-block copy is a screen-content tool. Scaling the encoder's 1 << 20 full-search threshold + // by the 8x8 block area yields this normalized variance-domain trigger. + if (bestCost > ExhaustiveSearchThreshold) + { + Point candidate = SearchExhaustiveMesh( + source, + reconstruction, + blockOrigin, + writer, + reference, + sadPerBit, + minimumColumnOffset, + minimumRowOffset, + maximumColumnOffset, + maximumRowOffset, + best); + + int candidateCost = GetVarianceCost( + source, + reconstruction, + blockOrigin, + writer, + reference, + bitDepth, + rateMultiplier, + candidate); + + if (candidateCost < bestCost) + { + best = candidate; + } + } + + return best; + } + + private static void SearchDiamond( + Buffer2DRegion source, + Buffer2DRegion reconstruction, + Point blockOrigin, + Av1SymbolEncoder writer, + Av1MotionVector reference, + int sadPerBit, + int searchStepParameter, + int minimumColumnOffset, + int minimumRowOffset, + int maximumColumnOffset, + int maximumRowOffset, + Point start, + out Point best, + out int centerSteps) + where TSample : unmanaged + where TOperation : struct, ISearchOperation + { + best = start; + centerSteps = 0; + bool movedFromStart = false; + int bestCost = GetSadCost( + source, + reconstruction, + blockOrigin, + writer, + reference, + sadPerBit, + best); + + for (int stage = SearchRadii.Length - 1 - searchStepParameter; stage >= 0; stage--) + { + int radius = SearchRadii[stage]; + int tangentialRadius = SearchTangentialRadii[stage]; + int searchSiteCount = radius <= 5 ? 8 : 12; + int bestSite = 0; + for (int site = 1; site <= searchSiteCount; site++) + { + Point delta = GetSearchOffset(site, radius, tangentialRadius); + Point candidate = new(best.X + delta.X, best.Y + delta.Y); + if (candidate.X < minimumColumnOffset || candidate.X > maximumColumnOffset || + candidate.Y < minimumRowOffset || candidate.Y > maximumRowOffset) + { + continue; + } + + Point predictionOrigin = new(blockOrigin.X + candidate.X, blockOrigin.Y + candidate.Y); + int sumOfAbsoluteDifferences = TOperation.GetSumOfAbsoluteDifferences( + source, + blockOrigin, + reconstruction, + predictionOrigin); + + // Motion-vector cost is nonnegative, so a raw absolute difference that already reaches the + // best combined cost cannot win and does not need an entropy-rate lookup. + if (sumOfAbsoluteDifferences >= bestCost) + { + continue; + } + + Av1MotionVector vector = new(candidate.Y * 8, candidate.X * 8); + int rate = writer.GetDisplacementVectorSearchCost(vector, reference); + int candidateCost = Av1RateDistortion.GetMotionSearchSadCost( + sadPerBit, + rate, + sumOfAbsoluteDifferences); + + if (candidateCost < bestCost) + { + bestCost = candidateCost; + bestSite = site; + } + } + + if (bestSite != 0) + { + Point delta = GetSearchOffset(bestSite, radius, tangentialRadius); + best = new(best.X + delta.X, best.Y + delta.Y); + movedFromStart = true; + } + + if (!movedFromStart) + { + centerSteps++; + } + + // The three largest NSTEP stages intentionally share one radius. When a stage remains centered, + // consume the equivalent duplicates exactly once instead of repeating the same candidate positions. + while (bestSite == 0 && stage > 2 && SearchRadii[stage - 1] == SearchRadii[stage]) + { + centerSteps++; + stage--; + } + } + } + + private static Point SearchExhaustiveMesh( + Buffer2DRegion source, + Buffer2DRegion reconstruction, + Point blockOrigin, + Av1SymbolEncoder writer, + Av1MotionVector reference, + int sadPerBit, + int minimumColumnOffset, + int minimumRowOffset, + int maximumColumnOffset, + int maximumRowOffset, + Point start) + where TSample : unmanaged + where TOperation : struct, ISearchOperation + { + Point best = start; + int bestCost = GetSadCost( + source, + reconstruction, + blockOrigin, + writer, + reference, + sadPerBit, + start); + + int startColumn = Math.Max(-ExhaustiveSearchRange, minimumColumnOffset - start.X); + int endColumn = Math.Min(ExhaustiveSearchRange, maximumColumnOffset - start.X); + int startRow = Math.Max(-ExhaustiveSearchRange, minimumRowOffset - start.Y); + int endRow = Math.Min(ExhaustiveSearchRange, maximumRowOffset - start.Y); + Span sumsOfAbsoluteDifferences = stackalloc int[ExhaustiveSearchBatchSize]; + for (int row = startRow; row <= endRow; row++) + { + int column = startColumn; + for (; column <= endColumn - (ExhaustiveSearchBatchSize - 1); column += ExhaustiveSearchBatchSize) + { + Point firstCandidate = new(start.X + column, start.Y + row); + Point firstPredictionOrigin = new( + blockOrigin.X + firstCandidate.X, + blockOrigin.Y + firstCandidate.Y); + + // Four adjacent candidates share the source load and row traversal, matching the batch width + // used by the native full-resolution pass without allocating temporary candidate buffers. + TOperation.GetFourSumsOfAbsoluteDifferences( + source, + blockOrigin, + reconstruction, + firstPredictionOrigin, + sumsOfAbsoluteDifferences); + + for (int i = 0; i < ExhaustiveSearchBatchSize; i++) + { + int sumOfAbsoluteDifferences = sumsOfAbsoluteDifferences[i]; + if (sumOfAbsoluteDifferences >= bestCost) + { + continue; + } + + Point candidate = new(firstCandidate.X + i, firstCandidate.Y); + Av1MotionVector vector = new(candidate.Y * 8, candidate.X * 8); + int rate = writer.GetDisplacementVectorSearchCost(vector, reference); + int candidateCost = Av1RateDistortion.GetMotionSearchSadCost( + sadPerBit, + rate, + sumOfAbsoluteDifferences); + + // Strict replacement preserves the first row-major candidate when costs tie. + if (candidateCost < bestCost) + { + bestCost = candidateCost; + best = candidate; + } + } + } + + // The SIMD batch width is only a traversal optimization; every legal tail column remains searchable. + for (; column <= endColumn; column++) + { + Point candidate = new(start.X + column, start.Y + row); + Point predictionOrigin = new(blockOrigin.X + candidate.X, blockOrigin.Y + candidate.Y); + int sumOfAbsoluteDifferences = TOperation.GetSumOfAbsoluteDifferences( + source, + blockOrigin, + reconstruction, + predictionOrigin); + + if (sumOfAbsoluteDifferences >= bestCost) + { + continue; + } + + Av1MotionVector vector = new(candidate.Y * 8, candidate.X * 8); + int rate = writer.GetDisplacementVectorSearchCost(vector, reference); + int candidateCost = Av1RateDistortion.GetMotionSearchSadCost( + sadPerBit, + rate, + sumOfAbsoluteDifferences); + + if (candidateCost < bestCost) + { + bestCost = candidateCost; + best = candidate; + } + } + } + + return best; + } + + private static int GetSadCost( + Buffer2DRegion source, + Buffer2DRegion reconstruction, + Point blockOrigin, + Av1SymbolEncoder writer, + Av1MotionVector reference, + int sadPerBit, + Point candidate) + where TSample : unmanaged + where TOperation : struct, ISearchOperation + { + Point predictionOrigin = new(blockOrigin.X + candidate.X, blockOrigin.Y + candidate.Y); + int sumOfAbsoluteDifferences = TOperation.GetSumOfAbsoluteDifferences( + source, + blockOrigin, + reconstruction, + predictionOrigin); + + Av1MotionVector vector = new(candidate.Y * 8, candidate.X * 8); + int rate = writer.GetDisplacementVectorSearchCost(vector, reference); + return Av1RateDistortion.GetMotionSearchSadCost(sadPerBit, rate, sumOfAbsoluteDifferences); + } + + private static int GetVarianceCost( + Buffer2DRegion source, + Buffer2DRegion reconstruction, + Point blockOrigin, + Av1SymbolEncoder writer, + Av1MotionVector reference, + Av1BitDepth bitDepth, + int rateMultiplier, + Point candidate) + where TSample : unmanaged + where TOperation : struct, ISearchOperation + { + Point predictionOrigin = new(blockOrigin.X + candidate.X, blockOrigin.Y + candidate.Y); + int variance = TOperation.GetVariance( + source, + blockOrigin, + reconstruction, + predictionOrigin, + bitDepth); + + Av1MotionVector vector = new(candidate.Y * 8, candidate.X * 8); + int rate = writer.GetDisplacementVectorSearchCost(vector, reference); + return Av1RateDistortion.GetMotionSearchCost(rateMultiplier, rate, variance); + } + + private static int GetSearchStepParameter(int frameSize) + { + int size = Math.Max(frameSize, 16); + int searchStepParameter = 0; + while (((long)size << searchStepParameter) < MaximumFullPixelSearchOffset) + { + searchStepParameter++; + } + + return Math.Min(searchStepParameter, 9); + } + + private static Point GetSearchOffset(int site, int radius, int tangentialRadius) + => site switch + { + 1 => new Point(0, -radius), + 2 => new Point(0, radius), + 3 => new Point(-radius, 0), + 4 => new Point(radius, 0), + 5 => new Point(-tangentialRadius, -radius), + 6 => new Point(tangentialRadius, radius), + 7 => new Point(radius, -tangentialRadius), + 8 => new Point(-radius, tangentialRadius), + 9 => new Point(tangentialRadius, -radius), + 10 => new Point(-tangentialRadius, radius), + 11 => new Point(radius, tangentialRadius), + _ => new Point(-radius, -tangentialRadius) + }; + + private Span GetHashesAndLinks() + => MemoryMarshal.Cast(this.storage.Span[..this.headOffset]); + + private Span GetHeads() + => MemoryMarshal.Cast(this.storage.Span.Slice(this.headOffset, this.bucketCount * sizeof(int))); + + private Span GetTails() + => MemoryMarshal.Cast(this.storage.Span.Slice(this.tailOffset, this.bucketCount * sizeof(int))); + + private Span GetCounts() + => MemoryMarshal.Cast(this.storage.Span.Slice(this.countOffset, this.bucketCount * sizeof(ushort))); +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Motion/Av1MotionVariationCandidates.cs b/src/ImageSharp/Formats/Heif/Av1/Motion/Av1MotionVariationCandidates.cs new file mode 100644 index 0000000000..aa89625234 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Motion/Av1MotionVariationCandidates.cs @@ -0,0 +1,283 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Motion; + +/// +/// Derives the neighboring-block state and fixed-capacity projection samples used to select an AV1 motion mode. +/// +internal sealed class Av1MotionVariationCandidates +{ + /// + /// The maximum number of neighboring motion samples retained for a local warped-motion projection. + /// + private const int ProjectionSampleCapacity = 8; + + /// + /// The largest neighbor step used by overlapping motion compensation, measured in 4x4 mode-information units. + /// + private const int MaximumNeighborStep = 16; + + /// + /// The number of fractional bits in an AV1 motion vector and warped-motion sample position. + /// + private const int MotionVectorSubpixelBits = 3; + + /// + /// Stores sample positions relative to the current block origin in one-eighth-sample units. + /// + private InlineArray8 sourcePoints; + + /// + /// Stores the corresponding reference-frame positions in one-eighth-sample units. + /// + private InlineArray8 referencePoints; + + /// + /// Gets the number of valid entries in and . + /// + public int Count { get; private set; } + + /// + /// Gets a value indicating whether an inter-coded block overlaps the current block's above or left edge. + /// + public bool HasOverlappableNeighbor { get; private set; } + + /// + /// Gets the retained current-frame sample positions in one-eighth-sample units relative to the current block. + /// + public ReadOnlySpan SourcePoints => this.sourcePoints[..this.Count]; + + /// + /// Gets the retained reference-frame sample positions in one-eighth-sample units relative to the current block. + /// + public ReadOnlySpan ReferencePoints => this.referencePoints[..this.Count]; + + /// + /// Derives the spatial state used to select Simple Translation, OBMC, or Warped motion for one inter block. + /// + /// The current block geometry and frame-wide decoded mode map. + /// The active tile boundaries. + /// The sequence-level superblock geometry. + /// The current frame dimensions. + /// The current block's primary canonical reference. + public void Build( + ref Av1PartitionInfo partitionInfo, + Av1TileInfo tileInfo, + ObuSequenceHeader sequenceHeader, + ObuFrameHeader frameHeader, + Av1ReferenceFrameType referenceFrame) + { + this.Count = 0; + this.CollectProjectionSamples(ref partitionInfo, tileInfo, sequenceHeader, frameHeader, referenceFrame); + this.HasOverlappableNeighbor = FindOverlappableNeighbor(ref partitionInfo, frameHeader); + } + + /// + /// Collects the at most eight spatial samples permitted by AV1's local warped-motion model. + /// + /// The current block geometry and frame-wide decoded mode map. + /// The active tile boundaries. + /// The sequence-level superblock geometry. + /// The current frame dimensions. + /// The current block's primary canonical reference. + private void CollectProjectionSamples( + ref Av1PartitionInfo partitionInfo, + Av1TileInfo tileInfo, + ObuSequenceHeader sequenceHeader, + ObuFrameHeader frameHeader, + Av1ReferenceFrameType referenceFrame) + { + Av1BlockSize blockSize = partitionInfo.ModeInfo.BlockSize; + int width = blockSize.Get4x4WideCount(); + int height = blockSize.Get4x4HighCount(); + int row = partitionInfo.RowIndex; + int column = partitionInfo.ColumnIndex; + bool includeTopLeft = true; + bool includeTopRight = true; + + if (partitionInfo.AvailableAbove) + { + Av1BlockModeInfo candidate = partitionInfo.SuperblockInfo.GetModeInfoAt(new Point(column, row - 1)); + int candidateWidth = candidate.BlockSize.Get4x4WideCount(); + if (width <= candidateWidth) + { + // A wider above block can also cover the diagonal search positions. The signed alignment offset + // prevents those positions from contributing the same block a second time. + int columnOffset = -column % candidateWidth; + includeTopLeft = columnOffset >= 0; + includeTopRight = columnOffset + candidateWidth <= width; + this.AddProjectionSample(candidate, referenceFrame, 0, -1, columnOffset, 1); + } + else + { + int end = Math.Min(width, frameHeader.ModeInfoColumnCount - column); + for (int index = 0; index < end && this.Count < ProjectionSampleCapacity; index += candidateWidth) + { + candidate = partitionInfo.SuperblockInfo.GetModeInfoAt(new Point(column + index, row - 1)); + candidateWidth = candidate.BlockSize.Get4x4WideCount(); + this.AddProjectionSample(candidate, referenceFrame, 0, -1, index, 1); + } + } + } + + if (partitionInfo.AvailableLeft && this.Count < ProjectionSampleCapacity) + { + Av1BlockModeInfo candidate = partitionInfo.SuperblockInfo.GetModeInfoAt(new Point(column - 1, row)); + int candidateHeight = candidate.BlockSize.Get4x4HighCount(); + if (height <= candidateHeight) + { + // The same alignment rule suppresses a duplicate top-left sample when one tall left block covers it. + int rowOffset = -row % candidateHeight; + includeTopLeft &= rowOffset >= 0; + this.AddProjectionSample(candidate, referenceFrame, rowOffset, 1, 0, -1); + } + else + { + int end = Math.Min(height, frameHeader.ModeInfoRowCount - row); + for (int index = 0; index < end && this.Count < ProjectionSampleCapacity; index += candidateHeight) + { + candidate = partitionInfo.SuperblockInfo.GetModeInfoAt(new Point(column - 1, row + index)); + candidateHeight = candidate.BlockSize.Get4x4HighCount(); + this.AddProjectionSample(candidate, referenceFrame, index, 1, 0, -1); + } + } + } + + if (includeTopLeft && partitionInfo.AvailableAbove && partitionInfo.AvailableLeft && this.Count < ProjectionSampleCapacity) + { + Av1BlockModeInfo candidate = partitionInfo.SuperblockInfo.GetModeInfoAt(new Point(column - 1, row - 1)); + this.AddProjectionSample(candidate, referenceFrame, 0, -1, 0, -1); + } + + int topRightRow = row - 1; + int topRightColumn = column + width; + bool topRightInsideTile = + topRightRow >= tileInfo.ModeInfoRowStart && + topRightRow < tileInfo.ModeInfoRowEnd && + topRightColumn >= tileInfo.ModeInfoColumnStart && + topRightColumn < tileInfo.ModeInfoColumnEnd; + + if (includeTopRight && + this.Count < ProjectionSampleCapacity && + partitionInfo.HasTopRight(sequenceHeader.SuperblockModeInfoSize) && + topRightInsideTile) + { + Av1BlockModeInfo candidate = partitionInfo.SuperblockInfo.GetModeInfoAt(new Point(topRightColumn, topRightRow)); + this.AddProjectionSample(candidate, referenceFrame, 0, -1, width, 1); + } + } + + /// + /// Determines whether an inter-coded neighbor covers either complete prediction edge of the current block. + /// + /// The current block geometry and frame-wide decoded mode map. + /// The current frame dimensions. + /// when an above or left inter block can contribute overlapping prediction. + private static bool FindOverlappableNeighbor(ref Av1PartitionInfo partitionInfo, ObuFrameHeader frameHeader) + { + Av1BlockSize blockSize = partitionInfo.ModeInfo.BlockSize; + int width = blockSize.Get4x4WideCount(); + int height = blockSize.Get4x4HighCount(); + int row = partitionInfo.RowIndex; + int column = partitionInfo.ColumnIndex; + + if (partitionInfo.AvailableAbove) + { + int endColumn = Math.Min(column + width, frameHeader.ModeInfoColumnCount); + for (int aboveColumn = column; aboveColumn < endColumn;) + { + Av1BlockModeInfo candidate = partitionInfo.SuperblockInfo.GetModeInfoAt(new Point(aboveColumn, row - 1)); + int step = Math.Min(candidate.BlockSize.Get4x4WideCount(), MaximumNeighborStep); + if (step == 1) + { + // AV1 treats a 4-sample-wide neighbor as one half of an 8-sample pair and reads the mode record + // attached to the pair's second cell before advancing across both cells. + aboveColumn &= ~1; + candidate = partitionInfo.SuperblockInfo.GetModeInfoAt(new Point(aboveColumn + 1, row - 1)); + step = 2; + } + + if (IsOverlappable(candidate)) + { + return true; + } + + aboveColumn += step; + } + } + + if (partitionInfo.AvailableLeft) + { + int endRow = Math.Min(row + height, frameHeader.ModeInfoRowCount); + for (int leftRow = row; leftRow < endRow;) + { + Av1BlockModeInfo candidate = partitionInfo.SuperblockInfo.GetModeInfoAt(new Point(column - 1, leftRow)); + int step = Math.Min(candidate.BlockSize.Get4x4HighCount(), MaximumNeighborStep); + if (step == 1) + { + // The vertical scan applies the corresponding 4-sample-high pairing rule. + leftRow &= ~1; + candidate = partitionInfo.SuperblockInfo.GetModeInfoAt(new Point(column - 1, leftRow + 1)); + step = 2; + } + + if (IsOverlappable(candidate)) + { + return true; + } + + leftRow += step; + } + } + + return false; + } + + /// + /// Appends one neighboring single-reference sample when it uses the current block's primary reference. + /// + /// The neighboring decoded block. + /// The current block's primary canonical reference. + /// The neighbor center row offset in 4x4 mode-information units. + /// The direction from the current block toward the neighbor on the vertical axis. + /// The neighbor center column offset in 4x4 mode-information units. + /// The direction from the current block toward the neighbor on the horizontal axis. + private void AddProjectionSample( + Av1BlockModeInfo candidate, + Av1ReferenceFrameType referenceFrame, + int rowOffset, + int rowSign, + int columnOffset, + int columnSign) + { + Span candidateReferences = candidate.ReferenceFrames; + if (candidateReferences[0] != referenceFrame || candidateReferences[1] != Av1ReferenceFrameType.None) + { + return; + } + + const int modeInfoSampleSize = 1 << Av1Constants.ModeInfoSizeLog2; + int sourceX = (columnOffset * modeInfoSampleSize) + (columnSign * (candidate.BlockSize.GetWidth() >> 1)) - 1; + int sourceY = (rowOffset * modeInfoSampleSize) + (rowSign * (candidate.BlockSize.GetHeight() >> 1)) - 1; + Point sourcePoint = new(sourceX << MotionVectorSubpixelBits, sourceY << MotionVectorSubpixelBits); + Av1MotionVector motionVector = candidate.MotionVectors[0]; + + // Neighbor centers and motion vectors share Q3 precision. Adding them directly produces the corresponding + // reference position without rounding away the fractional displacement needed by the projection solver. + this.sourcePoints[this.Count] = sourcePoint; + this.referencePoints[this.Count] = new Point(sourcePoint.X + motionVector.Column, sourcePoint.Y + motionVector.Row); + this.Count++; + } + + /// + /// Determines whether a decoded neighbor can participate in overlapping motion compensation. + /// + /// The neighboring decoded block. + /// for inter prediction or intra-block copy; otherwise, . + private static bool IsOverlappable(Av1BlockModeInfo candidate) + => candidate.UseIntraBlockCopy || candidate.ReferenceFrames[0] > Av1ReferenceFrameType.Intra; +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Motion/Av1MotionVector.cs b/src/ImageSharp/Formats/Heif/Av1/Motion/Av1MotionVector.cs new file mode 100644 index 0000000000..2854d1562f --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Motion/Av1MotionVector.cs @@ -0,0 +1,222 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Motion; + +/// +/// Represents an AV1 motion or displacement vector in one-eighth-sample units. +/// +internal readonly struct Av1MotionVector : IEquatable +{ + /// + /// The greatest absolute temporal distance used by AV1 motion-vector projection. + /// + public const int MaximumTemporalDistance = 31; + + /// + /// The number of fractional bits used by AV1 motion-vector components. + /// + public const int SubpixelBits = 3; + + /// + /// The number of motion-vector units in one full pixel. + /// + public const int SubpixelScale = 1 << SubpixelBits; + + /// + /// The reserved lower endpoint of the signed AV1 motion-vector domain. + /// + private const int LowerBound = -16384; + + /// + /// The exclusive upper endpoint of the signed AV1 motion-vector domain. + /// + private const int UpperBound = 16384; + + /// + /// The additional sixteen-sample border admitted while deriving spatial reference candidates, in one-eighth-sample units. + /// + private const int ReferenceBorder = 16 << 3; + + /// + /// Initializes a new instance of the struct. + /// + /// The signed vertical displacement in one-eighth-sample units. + /// The signed horizontal displacement in one-eighth-sample units. + public Av1MotionVector(int row, int column) + { + this.Row = row; + this.Column = column; + } + + /// + /// Gets the signed vertical displacement in one-eighth-sample units. + /// + public int Row { get; } + + /// + /// Gets the signed horizontal displacement in one-eighth-sample units. + /// + public int Column { get; } + + /// + /// Gets a value indicating whether both displacement components are zero. + /// + public bool IsZero => this.Row == 0 && this.Column == 0; + + /// + /// Gets a value indicating whether both components lie strictly between the two reserved AV1 endpoints. + /// + public bool IsValid => + this.Row > LowerBound && + this.Row < UpperBound && + this.Column > LowerBound && + this.Column < UpperBound; + + /// + /// Gets the reciprocal table used by AV1 temporal projection in fourteen-bit fixed-point precision. + /// + private static ReadOnlySpan ProjectionDivisors => + [0, 16384, 8192, 5461, 4096, 3276, 2730, 2340, 2048, 1820, 1638, 1489, 1365, 1260, 1170, 1092, + 1024, 963, 910, 862, 819, 780, 744, 712, 682, 655, 630, 606, 585, 564, 546, 528]; + + /// + /// Adds a component delta to this vector. + /// + /// The reference vector. + /// The decoded component delta. + /// The component-wise sum. + public static Av1MotionVector operator +(Av1MotionVector value, Av1MotionVector delta) + => new(value.Row + delta.Row, value.Column + delta.Column); + + /// + /// Determines whether two vectors have equal components. + /// + /// The first vector. + /// The second vector. + /// when both components are equal; otherwise, . + public static bool operator ==(Av1MotionVector left, Av1MotionVector right) => left.Equals(right); + + /// + /// Determines whether two vectors have different components. + /// + /// The first vector. + /// The second vector. + /// when either component differs; otherwise, . + public static bool operator !=(Av1MotionVector left, Av1MotionVector right) => !left.Equals(right); + + /// + /// Reduces this vector to the motion-vector precision selected by the current frame. + /// + /// + /// A value indicating whether one-eighth-sample precision may be retained. + /// + /// + /// A value indicating whether both components must be rounded to integer-sample precision. + /// + /// The precision-reduced vector. + public Av1MotionVector LowerPrecision(bool allowHighPrecision, bool forceInteger) + { + if (forceInteger) + { + return new(RoundToIntegerPrecision(this.Row), RoundToIntegerPrecision(this.Column)); + } + + if (allowHighPrecision) + { + return this; + } + + // Low precision removes the one-eighth-sample bit. Odd components move toward zero rather than rounding to + // the nearest even value, which is the normative lower_mv_precision behavior used by spatial and temporal MVs. + int row = (this.Row & 1) != 0 ? this.Row + (this.Row > 0 ? -1 : 1) : this.Row; + int column = (this.Column & 1) != 0 ? this.Column + (this.Column > 0 ? -1 : 1) : this.Column; + return new(row, column); + } + + /// + /// Clamps this vector to the spatial reference-candidate limits for a coding block. + /// + /// The coding-block width in luma samples. + /// The coding-block height in luma samples. + /// The signed distance to the left frame edge in one-eighth-sample units. + /// The signed distance to the right frame edge in one-eighth-sample units. + /// The signed distance to the top frame edge in one-eighth-sample units. + /// The signed distance to the bottom frame edge in one-eighth-sample units. + /// The vector clamped to the permitted spatial reference-candidate range. + public Av1MotionVector ClampReference( + int blockWidth, + int blockHeight, + int blockToLeftEdge, + int blockToRightEdge, + int blockToTopEdge, + int blockToBottomEdge) + { + int blockWidthSubpixel = blockWidth << 3; + int blockHeightSubpixel = blockHeight << 3; + + // Candidate derivation permits the complete block extent plus sixteen further luma samples beyond each + // visible frame edge. These are stack limits, not the tighter UMV limits applied later while sampling pixels. + int minimumColumn = blockToLeftEdge - blockWidthSubpixel - ReferenceBorder; + int maximumColumn = blockToRightEdge + blockWidthSubpixel + ReferenceBorder; + int minimumRow = blockToTopEdge - blockHeightSubpixel - ReferenceBorder; + int maximumRow = blockToBottomEdge + blockHeightSubpixel + ReferenceBorder; + return new( + Av1Math.Clip3(minimumRow, maximumRow, this.Row), + Av1Math.Clip3(minimumColumn, maximumColumn, this.Column)); + } + + /// + /// Projects this vector across a ratio of temporal frame distances. + /// + /// The signed source-to-target frame distance. + /// The positive source-to-reference frame distance. + /// The projected vector clamped inside the AV1 motion-vector domain. + public Av1MotionVector ProjectTemporal(int numerator, int denominator) + { + denominator = Math.Min(denominator, MaximumTemporalDistance); + numerator = Av1Math.Clip3(-MaximumTemporalDistance, MaximumTemporalDistance, numerator); + + // The reciprocal table represents 1 / denominator in Q14. Signed power-of-two rounding preserves symmetry + // for negative components, and AV1 excludes the two reserved endpoints from projected motion vectors. + // Motion-field retention limits each source component to 4095, keeping the complete Q14 product inside Int32. + int row = Av1Math.RoundPowerOf2Signed(this.Row * numerator * ProjectionDivisors[denominator], 14); + int column = Av1Math.RoundPowerOf2Signed(this.Column * numerator * ProjectionDivisors[denominator], 14); + row = Av1Math.Clip3(LowerBound + 1, UpperBound - 1, row); + column = Av1Math.Clip3(LowerBound + 1, UpperBound - 1, column); + return new(row, column); + } + + /// + /// Determines whether this vector has the same components as another vector. + /// + /// The vector to compare. + /// when both components are equal; otherwise, . + public bool Equals(Av1MotionVector other) => this.Row == other.Row && this.Column == other.Column; + + /// + public override bool Equals(object? obj) => obj is Av1MotionVector other && this.Equals(other); + + /// + public override int GetHashCode() => HashCode.Combine(this.Row, this.Column); + + /// + /// Rounds one component to the nearest integer-sample displacement. + /// + /// The component in one-eighth-sample units. + /// The integer-precision component in one-eighth-sample units. + private static int RoundToIntegerPrecision(int value) + { + int remainder = value % 8; + value -= remainder; + + // Exactly half an integer sample has magnitude four. AV1 leaves that truncated base unchanged, so both + // positive and negative half ties move toward zero; only larger remainders advance to the adjacent sample. + if (Math.Abs(remainder) > 4) + { + value += remainder > 0 ? 8 : -8; + } + + return value; + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Motion/Av1MotionVectorPrecision.cs b/src/ImageSharp/Formats/Heif/Av1/Motion/Av1MotionVectorPrecision.cs new file mode 100644 index 0000000000..79b89bb6a7 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Motion/Av1MotionVectorPrecision.cs @@ -0,0 +1,25 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Motion; + +/// +/// Identifies the fractional precision used to decode an AV1 motion-vector delta. +/// +internal enum Av1MotionVectorPrecision : sbyte +{ + /// + /// Restricts components to whole-sample increments. + /// + Integer = -1, + + /// + /// Allows components in quarter-sample increments. + /// + QuarterSample, + + /// + /// Allows components in eighth-sample increments. + /// + EighthSample +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Motion/Av1ReferenceMotionVectors.cs b/src/ImageSharp/Formats/Heif/Av1/Motion/Av1ReferenceMotionVectors.cs new file mode 100644 index 0000000000..758d3563e8 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Motion/Av1ReferenceMotionVectors.cs @@ -0,0 +1,1598 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Diagnostics.CodeAnalysis; +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Motion; + +/// +/// Derives the weighted AV1 reference-motion-vector candidates for one inter block. +/// +internal struct Av1ReferenceMotionVectors +{ + /// + /// The number of surrounding mode-information rows and columns examined by the spatial search. + /// + private const int ReferenceSearchDistance = 3; + + /// + /// The weight separating immediately adjacent candidates from temporal and outer spatial candidates. + /// + private const int NearestCandidateWeight = 640; + + /// + /// The maximum number of distinct candidates retained by AV1. + /// + private const int CandidateCapacity = 8; + + /// + /// The width and height of the outer spatial and temporal search boundary in 4x4 mode-information units. + /// + private const int MaximumSearchBlockSize = 16; + + /// + /// The packed mode-context bit containing temporal availability relative to global motion. + /// + private const int GlobalMotionContextBit = 1 << 3; + + /// + /// The bit offset of the reference-motion-vector context in the packed mode context. + /// + private const int ReferenceMotionVectorContextOffset = 4; + + /// + /// Stores the unique candidates in their normative weighted order. + /// + private InlineArray8 candidates; + + /// + /// Stores the secondary vector of each compound candidate. + /// + private InlineArray8 compoundCandidates; + + /// + /// Stores the accumulated spatial or temporal weight corresponding to each candidate. + /// + private InlineArray8 weights; + + /// + /// Stores the nearest and near references after applying AV1 fallback and precision rules. + /// + private InlineArray2 references; + + /// + /// Stores the nearest and near secondary references for a compound block. + /// + private InlineArray2 compoundReferences; + + /// + /// Gets the number of valid entries in and . + /// + public int Count { get; private set; } + + /// + /// Gets the packed entropy context derived from adjacent, outer, and temporal candidates. + /// + public int ModeContext { get; private set; } + + /// + /// Gets the derived candidates in normative nearest-region then outer-region order. + /// + [UnscopedRef] + public ReadOnlySpan Candidates => this.candidates[..this.Count]; + + /// + /// Gets the secondary vectors corresponding to for a compound block. + /// + [UnscopedRef] + public ReadOnlySpan CompoundCandidates => this.compoundCandidates[..this.Count]; + + /// + /// Gets the accumulated weight corresponding to each entry in . + /// + [UnscopedRef] + public ReadOnlySpan Weights => this.weights[..this.Count]; + + /// + /// Gets the nearest reference, or the current block's global-motion vector when no candidate exists. + /// + public Av1MotionVector Nearest => this.references[0]; + + /// + /// Derives all single- or compound-reference motion-vector candidates for the current block. + /// + /// The current block geometry and decoded spatial neighbors. + /// The active tile boundaries. + /// The frame-wide spatial map and projected temporal motion field. + /// The sequence-level superblock and order-hint configuration. + /// The frame-level global-motion and motion-vector precision configuration. + /// The primary canonical inter reference selected for the current block. + /// The secondary compound reference, or . + public void Build( + ref Av1PartitionInfo partitionInfo, + Av1TileInfo tileInfo, + Av1FrameInfo frameInfo, + ObuSequenceHeader sequenceHeader, + ObuFrameHeader frameHeader, + Av1ReferenceFrameType referenceFrame, + Av1ReferenceFrameType secondaryReferenceFrame = Av1ReferenceFrameType.None) + { + ReferenceContext context = new(ref partitionInfo, sequenceHeader.SuperblockModeInfoSize, frameInfo); + this.Build( + in context, + tileInfo, + sequenceHeader, + frameHeader, + referenceFrame, + secondaryReferenceFrame); + } + + /// + /// Derives the spatial single-reference motion-vector candidates for an encoder block. + /// + /// The encoder picture state containing previously coded neighbors. + /// The current block geometry and tile availability. + /// The current block origin in 4x4 mode-information units. + /// The current coding-block size. + /// The partition that produced the current block. + /// The sequence-level superblock configuration. + /// The frame-level global-motion and motion-vector precision configuration. + /// The canonical inter reference selected for the current block. + public void Build( + Av1PictureControlSet picture, + Av1MacroBlockD macroBlock, + Point modeInfoPosition, + Av1BlockSize blockSize, + Av1PartitionType partitionType, + ObuSequenceHeader sequenceHeader, + ObuFrameHeader frameHeader, + Av1ReferenceFrameType referenceFrame) + { + ReferenceContext context = new( + picture, + macroBlock, + modeInfoPosition, + blockSize, + partitionType, + sequenceHeader.SuperblockModeInfoSize); + + this.Build( + in context, + macroBlock.Tile, + sequenceHeader, + frameHeader, + referenceFrame, + Av1ReferenceFrameType.None); + } + + private void Build( + in ReferenceContext context, + Av1TileInfo tileInfo, + ObuSequenceHeader sequenceHeader, + ObuFrameHeader frameHeader, + Av1ReferenceFrameType referenceFrame, + Av1ReferenceFrameType secondaryReferenceFrame) + { + Av1BlockSize blockSize = context.BlockSize; + int width = blockSize.Get4x4WideCount(); + int height = blockSize.Get4x4HighCount(); + int row = context.RowIndex; + int column = context.ColumnIndex; + int rowAdjustment = height < 2 && (row & 1) != 0 ? 1 : 0; + int columnAdjustment = width < 2 && (column & 1) != 0 ? 1 : 0; + int maximumRowOffset = 0; + int maximumColumnOffset = 0; + + this.Count = 0; + this.ModeContext = 0; + + if (context.AvailableAbove) + { + maximumRowOffset = height < 2 ? -4 + rowAdjustment : -(ReferenceSearchDistance << 1) + rowAdjustment; + maximumRowOffset = Math.Clamp(maximumRowOffset, tileInfo.ModeInfoRowStart - row, tileInfo.ModeInfoRowEnd - row - 1); + } + + if (context.AvailableLeft) + { + maximumColumnOffset = width < 2 ? -4 + columnAdjustment : -(ReferenceSearchDistance << 1) + columnAdjustment; + maximumColumnOffset = Math.Clamp(maximumColumnOffset, tileInfo.ModeInfoColumnStart - column, tileInfo.ModeInfoColumnEnd - column - 1); + } + + Av1GlobalMotionParameters globalMotion = frameHeader.GetGlobalMotionParameters()[(int)referenceFrame - 1]; + Av1MotionVector globalMotionVector = globalMotion.GetMotionVector( + frameHeader.AllowHighPrecisionMotionVector, + blockSize, + new Point(column, row), + frameHeader.ForceIntegerMotionVector); + + Av1GlobalMotionParameters secondaryGlobalMotion = default; + Av1MotionVector secondaryGlobalMotionVector = default; + if (secondaryReferenceFrame > Av1ReferenceFrameType.Intra) + { + secondaryGlobalMotion = frameHeader.GetGlobalMotionParameters()[(int)secondaryReferenceFrame - 1]; + secondaryGlobalMotionVector = secondaryGlobalMotion.GetMotionVector( + frameHeader.AllowHighPrecisionMotionVector, + blockSize, + new Point(column, row), + frameHeader.ForceIntegerMotionVector); + } + + int processedRows = 0; + int processedColumns = 0; + int rowMatchCount = 0; + int columnMatchCount = 0; + int newMotionVectorCount = 0; + + // Immediate above and left scans form a distinct high-priority region. Their direction-level match counts, + // rather than their number of unique vectors, drive the packed inter-mode entropy context. + if (Math.Abs(maximumRowOffset) >= 1) + { + this.ScanRow( + in context, + referenceFrame, + secondaryReferenceFrame, + in globalMotion, + in secondaryGlobalMotion, + globalMotionVector, + secondaryGlobalMotionVector, + -1, + maximumRowOffset, + ref rowMatchCount, + ref newMotionVectorCount, + ref processedRows); + } + + if (Math.Abs(maximumColumnOffset) >= 1) + { + this.ScanColumn( + in context, + referenceFrame, + secondaryReferenceFrame, + in globalMotion, + in secondaryGlobalMotion, + globalMotionVector, + secondaryGlobalMotionVector, + -1, + maximumColumnOffset, + ref columnMatchCount, + ref newMotionVectorCount, + ref processedColumns); + } + + if (context.HasTopRight) + { + this.AddSpatialBlock( + in context, + tileInfo, + referenceFrame, + secondaryReferenceFrame, + in globalMotion, + in secondaryGlobalMotion, + globalMotionVector, + secondaryGlobalMotionVector, + -1, + width, + ref rowMatchCount, + ref newMotionVectorCount); + } + + int nearestMatch = (rowMatchCount > 0 ? 1 : 0) + (columnMatchCount > 0 ? 1 : 0); + int nearestCandidateCount = this.Count; + for (int index = 0; index < nearestCandidateCount; index++) + { + this.weights[index] += NearestCandidateWeight; + } + + if (frameHeader.UseReferenceFrameMotionVectors) + { + this.AddTemporalCandidates( + in context, + tileInfo, + sequenceHeader.OrderHintInfo, + frameHeader, + referenceFrame, + secondaryReferenceFrame, + globalMotionVector, + secondaryGlobalMotionVector); + } + + int ignoredNewMotionVectorCount = 0; + + // The top-left block begins the lower-priority outer region. Candidate deduplication still spans both + // regions, while the two independent stable sorts below preserve the normative nearest-before-outer order. + this.AddSpatialBlock( + in context, + tileInfo, + referenceFrame, + secondaryReferenceFrame, + in globalMotion, + in secondaryGlobalMotion, + globalMotionVector, + secondaryGlobalMotionVector, + -1, + -1, + ref rowMatchCount, + ref ignoredNewMotionVectorCount); + + for (int index = 2; index <= ReferenceSearchDistance; index++) + { + int rowOffset = -(index << 1) + 1 + rowAdjustment; + int columnOffset = -(index << 1) + 1 + columnAdjustment; + if (Math.Abs(rowOffset) <= Math.Abs(maximumRowOffset) && Math.Abs(rowOffset) > processedRows) + { + this.ScanRow( + in context, + referenceFrame, + secondaryReferenceFrame, + in globalMotion, + in secondaryGlobalMotion, + globalMotionVector, + secondaryGlobalMotionVector, + rowOffset, + maximumRowOffset, + ref rowMatchCount, + ref ignoredNewMotionVectorCount, + ref processedRows); + } + + if (Math.Abs(columnOffset) <= Math.Abs(maximumColumnOffset) && Math.Abs(columnOffset) > processedColumns) + { + this.ScanColumn( + in context, + referenceFrame, + secondaryReferenceFrame, + in globalMotion, + in secondaryGlobalMotion, + globalMotionVector, + secondaryGlobalMotionVector, + columnOffset, + maximumColumnOffset, + ref columnMatchCount, + ref ignoredNewMotionVectorCount, + ref processedColumns); + } + } + + int referenceMatchCount = (rowMatchCount > 0 ? 1 : 0) + (columnMatchCount > 0 ? 1 : 0); + this.ModeContext |= nearestMatch switch + { + 0 => (referenceMatchCount >= 1 ? 1 : 0) | + (referenceMatchCount == 1 ? 1 << ReferenceMotionVectorContextOffset : + referenceMatchCount >= 2 ? 2 << ReferenceMotionVectorContextOffset : 0), + 1 => (newMotionVectorCount > 0 ? 2 : 3) | + (referenceMatchCount == 1 ? 3 << ReferenceMotionVectorContextOffset : + referenceMatchCount >= 2 ? 4 << ReferenceMotionVectorContextOffset : 0), + _ => (newMotionVectorCount >= 1 ? 4 : 5) | (5 << ReferenceMotionVectorContextOffset), + }; + + this.SortByWeight(0, nearestCandidateCount); + this.SortByWeight(nearestCandidateCount, this.Count); + + int frameWidth = frameHeader.ModeInfoColumnCount; + int frameHeight = frameHeader.ModeInfoRowCount; + int modeInfoWidth = Math.Min(Math.Min(MaximumSearchBlockSize, width), frameWidth - column); + int modeInfoHeight = Math.Min(Math.Min(MaximumSearchBlockSize, height), frameHeight - row); + int extensionLength = Math.Min(modeInfoWidth, modeInfoHeight); + + if (secondaryReferenceFrame > Av1ReferenceFrameType.Intra) + { + if (this.Count < 2) + { + this.ExtendCompoundStack( + in context, + referenceFrame, + secondaryReferenceFrame, + globalMotionVector, + secondaryGlobalMotionVector, + Math.Abs(maximumRowOffset) >= 1, + Math.Abs(maximumColumnOffset) >= 1, + extensionLength); + } + } + else + { + // This fallback supplies only the nearest and near pair when direct and projected scans leave gaps. + // Differing reference sign biases are reversed before either candidate enters that pair. + for (int index = 0; Math.Abs(maximumRowOffset) >= 1 && index < extensionLength && this.Count < 2;) + { + ReferenceBlock candidate = context.GetModeInfoAt(new Point(column + index, row - 1)); + this.AddExtensionCandidate(candidate, in context, referenceFrame); + index += candidate.BlockSize.Get4x4WideCount(); + } + + for (int index = 0; Math.Abs(maximumColumnOffset) >= 1 && index < extensionLength && this.Count < 2;) + { + ReferenceBlock candidate = context.GetModeInfoAt(new Point(column - 1, row + index)); + this.AddExtensionCandidate(candidate, in context, referenceFrame); + index += candidate.BlockSize.Get4x4HighCount(); + } + } + + for (int index = 0; index < this.Count; index++) + { + this.candidates[index] = this.candidates[index].ClampReference( + blockSize.GetWidth(), + blockSize.GetHeight(), + context.ModeBlockToLeftEdge, + context.ModeBlockToRightEdge, + context.ModeBlockToTopEdge, + context.ModeBlockToBottomEdge); + + if (secondaryReferenceFrame > Av1ReferenceFrameType.Intra) + { + this.compoundCandidates[index] = this.compoundCandidates[index].ClampReference( + blockSize.GetWidth(), + blockSize.GetHeight(), + context.ModeBlockToLeftEdge, + context.ModeBlockToRightEdge, + context.ModeBlockToTopEdge, + context.ModeBlockToBottomEdge); + } + } + + // The two-element reference list is separate from the full DRL stack. Missing entries use global motion, + // and both entries undergo the same precision reduction as the reference decoder's av1_find_best_ref_mvs output. + this.references[0] = (this.Count > 0 ? this.candidates[0] : globalMotionVector).LowerPrecision( + frameHeader.AllowHighPrecisionMotionVector, + frameHeader.ForceIntegerMotionVector); + + this.references[1] = (this.Count > 1 ? this.candidates[1] : globalMotionVector).LowerPrecision( + frameHeader.AllowHighPrecisionMotionVector, + frameHeader.ForceIntegerMotionVector); + + if (secondaryReferenceFrame > Av1ReferenceFrameType.Intra) + { + this.compoundReferences[0] = this.compoundCandidates[0].LowerPrecision( + frameHeader.AllowHighPrecisionMotionVector, + frameHeader.ForceIntegerMotionVector); + + this.compoundReferences[1] = this.compoundCandidates[1].LowerPrecision( + frameHeader.AllowHighPrecisionMotionVector, + frameHeader.ForceIntegerMotionVector); + } + } + + /// + /// Gets the near reference selected by a decoded dynamic-reference-list index. + /// + /// The decoded zero-based dynamic-reference-list index. + /// The selected near motion vector. + public Av1MotionVector GetNearReference(int referenceMotionVectorIndex) + => referenceMotionVectorIndex == 0 ? this.references[1] : this.candidates[referenceMotionVectorIndex + 1]; + + /// + /// Gets the differential reference used to decode a new motion vector. + /// + /// The decoded zero-based dynamic-reference-list index. + /// The selected stack candidate, or the nearest fallback when the stack contains one or no entries. + public Av1MotionVector GetNewReference(int referenceMotionVectorIndex) + => this.Count > 1 ? this.candidates[referenceMotionVectorIndex] : this.references[0]; + + /// + /// Gets the nearest reference for one member of a compound pair. + /// + /// Zero for the primary reference or one for the secondary reference. + /// The precision-reduced nearest vector. + public Av1MotionVector GetCompoundNearestReference(int referenceIndex) + => referenceIndex == 0 ? this.references[0] : this.compoundReferences[0]; + + /// + /// Gets the near reference for one member of a compound pair. + /// + /// The decoded zero-based dynamic-reference-list index. + /// Zero for the primary reference or one for the secondary reference. + /// The precision-reduced near vector. + public Av1MotionVector GetCompoundNearReference(int referenceMotionVectorIndex, int referenceIndex) + { + if (referenceMotionVectorIndex == 0) + { + return referenceIndex == 0 ? this.references[1] : this.compoundReferences[1]; + } + + int candidateIndex = referenceMotionVectorIndex + 1; + return referenceIndex == 0 ? this.candidates[candidateIndex] : this.compoundCandidates[candidateIndex]; + } + + /// + /// Gets the differential reference for one member of a compound pair. + /// + /// The selected stack index. + /// Zero for the primary reference or one for the secondary reference. + /// The selected raw stack vector. + public Av1MotionVector GetCompoundNewReference(int referenceMotionVectorIndex, int referenceIndex) + => referenceIndex == 0 + ? this.candidates[referenceMotionVectorIndex] + : this.compoundCandidates[referenceMotionVectorIndex]; + + /// + /// Scans one spatial row using AV1's block-size-dependent steps and weights. + /// + /// The current block geometry and frame-wide mode map. + /// The canonical inter reference selected for the current block. + /// The secondary compound reference, or . + /// The selected reference's global-motion model. + /// The secondary reference's global-motion model. + /// The selected reference's global-motion vector at the current block. + /// The secondary reference's global-motion vector at the current block. + /// The signed row offset from the current block in 4x4 units. + /// The farthest permitted row offset inside the tile. + /// Accumulates matching reference labels found in this scan direction. + /// Accumulates matching neighbors whose inter mode contains a new vector. + /// Receives the spatial depth covered by block-height weighting. + private void ScanRow( + in ReferenceContext context, + Av1ReferenceFrameType referenceFrame, + Av1ReferenceFrameType secondaryReferenceFrame, + in Av1GlobalMotionParameters globalMotion, + in Av1GlobalMotionParameters secondaryGlobalMotion, + Av1MotionVector globalMotionVector, + Av1MotionVector secondaryGlobalMotionVector, + int rowOffset, + int maximumRowOffset, + ref int referenceMatchCount, + ref int newMotionVectorCount, + ref int processedRows) + { + int width = context.BlockSize.Get4x4WideCount(); + int end = Math.Min(context.GetMaxBlockWide(), MaximumSearchBlockSize); + int columnOffset = 0; + if (Math.Abs(rowOffset) > 1) + { + columnOffset = 1; + if ((context.ColumnIndex & 1) != 0 && width < 2) + { + columnOffset--; + } + } + + // The scan advances by at least one 16x16 mode-info region only when the current block reaches 64 pixels + // on this axis. Smaller blocks must visit narrow neighbors individually so none of their candidates vanish. + bool useFourUnitStep = width >= MaximumSearchBlockSize; + for (int index = 0; index < end;) + { + ReferenceBlock candidate = context.GetModeInfoAt( + new Point(context.ColumnIndex + columnOffset + index, context.RowIndex + rowOffset)); + + int candidateWidth = candidate.BlockSize.Get4x4WideCount(); + int length = Math.Min(width, candidateWidth); + if (useFourUnitStep) + { + length = Math.Max(4, length); + } + else if (Math.Abs(rowOffset) > 1) + { + length = Math.Max(2, length); + } + + int weight = 2; + if (width >= 2 && width <= candidateWidth) + { + int increment = Math.Min(-maximumRowOffset + rowOffset + 1, candidate.BlockSize.Get4x4HighCount()); + weight = Math.Max(weight, increment); + processedRows = increment - rowOffset - 1; + } + + this.AddCandidate( + candidate, + referenceFrame, + secondaryReferenceFrame, + in globalMotion, + in secondaryGlobalMotion, + globalMotionVector, + secondaryGlobalMotionVector, + length * weight, + ref referenceMatchCount, + ref newMotionVectorCount); + + index += length; + } + } + + /// + /// Scans one spatial column using AV1's block-size-dependent steps and weights. + /// + /// The current block geometry and frame-wide mode map. + /// The canonical inter reference selected for the current block. + /// The secondary compound reference, or . + /// The selected reference's global-motion model. + /// The secondary reference's global-motion model. + /// The selected reference's global-motion vector at the current block. + /// The secondary reference's global-motion vector at the current block. + /// The signed column offset from the current block in 4x4 units. + /// The farthest permitted column offset inside the tile. + /// Accumulates matching reference labels found in this scan direction. + /// Accumulates matching neighbors whose inter mode contains a new vector. + /// Receives the spatial depth covered by block-width weighting. + private void ScanColumn( + in ReferenceContext context, + Av1ReferenceFrameType referenceFrame, + Av1ReferenceFrameType secondaryReferenceFrame, + in Av1GlobalMotionParameters globalMotion, + in Av1GlobalMotionParameters secondaryGlobalMotion, + Av1MotionVector globalMotionVector, + Av1MotionVector secondaryGlobalMotionVector, + int columnOffset, + int maximumColumnOffset, + ref int referenceMatchCount, + ref int newMotionVectorCount, + ref int processedColumns) + { + int height = context.BlockSize.Get4x4HighCount(); + int end = Math.Min(context.GetMaxBlockHigh(), MaximumSearchBlockSize); + int rowOffset = 0; + if (Math.Abs(columnOffset) > 1) + { + rowOffset = 1; + if ((context.RowIndex & 1) != 0 && height < 2) + { + rowOffset--; + } + } + + // The scan advances by at least one 16x16 mode-info region only when the current block reaches 64 pixels + // on this axis. Smaller blocks must visit narrow neighbors individually so none of their candidates vanish. + bool useFourUnitStep = height >= MaximumSearchBlockSize; + for (int index = 0; index < end;) + { + ReferenceBlock candidate = context.GetModeInfoAt( + new Point(context.ColumnIndex + columnOffset, context.RowIndex + rowOffset + index)); + + int candidateHeight = candidate.BlockSize.Get4x4HighCount(); + int length = Math.Min(height, candidateHeight); + if (useFourUnitStep) + { + length = Math.Max(4, length); + } + else if (Math.Abs(columnOffset) > 1) + { + length = Math.Max(2, length); + } + + int weight = 2; + if (height >= 2 && height <= candidateHeight) + { + int increment = Math.Min(-maximumColumnOffset + columnOffset + 1, candidate.BlockSize.Get4x4WideCount()); + weight = Math.Max(weight, increment); + processedColumns = increment - columnOffset - 1; + } + + this.AddCandidate( + candidate, + referenceFrame, + secondaryReferenceFrame, + in globalMotion, + in secondaryGlobalMotion, + globalMotionVector, + secondaryGlobalMotionVector, + length * weight, + ref referenceMatchCount, + ref newMotionVectorCount); + + index += length; + } + } + + /// + /// Adds the candidate at one tile-relative spatial search position. + /// + /// The current block geometry and frame-wide mode map. + /// The active tile boundaries. + /// The canonical inter reference selected for the current block. + /// The secondary compound reference, or . + /// The selected reference's global-motion model. + /// The secondary reference's global-motion model. + /// The selected reference's global-motion vector at the current block. + /// The secondary reference's global-motion vector at the current block. + /// The signed row offset from the current block in 4x4 units. + /// The signed column offset from the current block in 4x4 units. + /// Accumulates matching reference labels at the search position. + /// Accumulates matching neighbors whose inter mode contains a new vector. + private void AddSpatialBlock( + in ReferenceContext context, + Av1TileInfo tileInfo, + Av1ReferenceFrameType referenceFrame, + Av1ReferenceFrameType secondaryReferenceFrame, + in Av1GlobalMotionParameters globalMotion, + in Av1GlobalMotionParameters secondaryGlobalMotion, + Av1MotionVector globalMotionVector, + Av1MotionVector secondaryGlobalMotionVector, + int rowOffset, + int columnOffset, + ref int referenceMatchCount, + ref int newMotionVectorCount) + { + int row = context.RowIndex + rowOffset; + int column = context.ColumnIndex + columnOffset; + if (row < tileInfo.ModeInfoRowStart || row >= tileInfo.ModeInfoRowEnd || + column < tileInfo.ModeInfoColumnStart || column >= tileInfo.ModeInfoColumnEnd) + { + return; + } + + ReferenceBlock candidate = context.GetModeInfoAt(new Point(column, row)); + this.AddCandidate( + candidate, + referenceFrame, + secondaryReferenceFrame, + in globalMotion, + in secondaryGlobalMotion, + globalMotionVector, + secondaryGlobalMotionVector, + 4, + ref referenceMatchCount, + ref newMotionVectorCount); + } + + /// + /// Accumulates matching references from one decoded inter block. + /// + /// The decoded neighboring block. + /// The canonical inter reference selected for the current block. + /// The secondary compound reference, or . + /// The selected reference's global-motion model. + /// The secondary reference's global-motion model. + /// The selected reference's global-motion vector at the current block. + /// The secondary reference's global-motion vector at the current block. + /// The spatial weight contributed by each matching reference. + /// Accumulates matching reference labels in the active scan direction. + /// Accumulates matching neighbors whose inter mode contains a new vector. + private void AddCandidate( + ReferenceBlock candidate, + Av1ReferenceFrameType referenceFrame, + Av1ReferenceFrameType secondaryReferenceFrame, + in Av1GlobalMotionParameters globalMotion, + in Av1GlobalMotionParameters secondaryGlobalMotion, + Av1MotionVector globalMotionVector, + Av1MotionVector secondaryGlobalMotionVector, + int weight, + ref int referenceMatchCount, + ref int newMotionVectorCount) + { + if (candidate.GetReferenceFrame(0) <= Av1ReferenceFrameType.Intra) + { + return; + } + + if (secondaryReferenceFrame > Av1ReferenceFrameType.Intra) + { + if (candidate.GetReferenceFrame(0) != referenceFrame || + candidate.GetReferenceFrame(1) != secondaryReferenceFrame) + { + return; + } + + bool usePrimaryGlobalMotion = + candidate.YMode == Av1PredictionMode.GlobalGlobalMotionVector && + globalMotion.Type > Av1GlobalMotionType.Translation && + Math.Min(candidate.BlockSize.GetWidth(), candidate.BlockSize.GetHeight()) >= 8; + + bool useSecondaryGlobalMotion = + candidate.YMode == Av1PredictionMode.GlobalGlobalMotionVector && + secondaryGlobalMotion.Type > Av1GlobalMotionType.Translation && + Math.Min(candidate.BlockSize.GetWidth(), candidate.BlockSize.GetHeight()) >= 8; + + Av1MotionVector primaryMotionVector = usePrimaryGlobalMotion + ? globalMotionVector + : candidate.GetMotionVector(0); + + Av1MotionVector secondaryMotionVector = useSecondaryGlobalMotion + ? secondaryGlobalMotionVector + : candidate.GetMotionVector(1); + + this.AddUnique(primaryMotionVector, secondaryMotionVector, weight); + if (UsesNewMotionVector(candidate.YMode)) + { + newMotionVectorCount++; + } + + referenceMatchCount++; + return; + } + + for (int referenceIndex = 0; referenceIndex < 2; referenceIndex++) + { + if (candidate.GetReferenceFrame(referenceIndex) != referenceFrame) + { + continue; + } + + // A non-translational global block has no independent translational candidate at the neighbor. AV1 + // therefore evaluates the selected reference's global model at the current block and contributes that + // vector, but only for blocks large enough to use affine global prediction. + bool useGlobalMotion = + (candidate.YMode is Av1PredictionMode.GlobalMotionVector or Av1PredictionMode.GlobalGlobalMotionVector) && + globalMotion.Type > Av1GlobalMotionType.Translation && + Math.Min(candidate.BlockSize.GetWidth(), candidate.BlockSize.GetHeight()) >= 8; + + Av1MotionVector motionVector = useGlobalMotion + ? globalMotionVector + : candidate.GetMotionVector(referenceIndex); + + this.AddUnique(motionVector, weight); + + // Every matching reference in a neighbor carrying a NEW component contributes to the adjacent NEWMV + // context even when its vector deduplicates against an earlier stack entry. + if (UsesNewMotionVector(candidate.YMode)) + { + newMotionVectorCount++; + } + + referenceMatchCount++; + } + } + + /// + /// Adds projected temporal candidates over the current block and its permitted extension positions. + /// + /// The current block geometry and decoder temporal state. + /// The active tile boundaries. + /// The sequence modulo order-hint configuration. + /// The frame-level motion-vector precision configuration. + /// The canonical inter reference selected for the current block. + /// The secondary compound reference, or . + /// The selected reference's global-motion vector at the current block. + /// The secondary reference's global-motion vector at the current block. + private void AddTemporalCandidates( + in ReferenceContext context, + Av1TileInfo tileInfo, + ObuOrderHintInfo orderHintInfo, + ObuFrameHeader frameHeader, + Av1ReferenceFrameType referenceFrame, + Av1ReferenceFrameType secondaryReferenceFrame, + Av1MotionVector globalMotionVector, + Av1MotionVector secondaryGlobalMotionVector) + { + int width = context.BlockSize.Get4x4WideCount(); + int height = context.BlockSize.Get4x4HighCount(); + int verticalOffset = Math.Max(2, height); + int horizontalOffset = Math.Max(2, width); + int blockRowEnd = Math.Min(height, MaximumSearchBlockSize); + int blockColumnEnd = Math.Min(width, MaximumSearchBlockSize); + int rowStep = height >= MaximumSearchBlockSize ? 4 : 2; + int columnStep = width >= MaximumSearchBlockSize ? 4 : 2; + bool firstSampleAvailable = false; + + for (int blockRow = 0; blockRow < blockRowEnd; blockRow += rowStep) + { + for (int blockColumn = 0; blockColumn < blockColumnEnd; blockColumn += columnStep) + { + bool available = this.AddTemporalCandidate( + in context, + tileInfo, + orderHintInfo, + frameHeader, + referenceFrame, + secondaryReferenceFrame, + globalMotionVector, + secondaryGlobalMotionVector, + blockRow, + blockColumn); + + if (blockRow == 0 && blockColumn == 0) + { + firstSampleAvailable = available; + } + } + } + + if (!firstSampleAvailable) + { + this.ModeContext |= GlobalMotionContextBit; + } + + bool allowExtension = height >= 2 && height < MaximumSearchBlockSize && width >= 2 && width < MaximumSearchBlockSize; + if (!allowExtension) + { + return; + } + + // These three positions extend the temporal search below-left, below-right, and above-right. The 64x64 + // boundary test is normative even when the sequence uses 128x128 superblocks. + this.AddTemporalExtension( + in context, + tileInfo, + orderHintInfo, + frameHeader, + referenceFrame, + secondaryReferenceFrame, + globalMotionVector, + secondaryGlobalMotionVector, + verticalOffset, + -2); + + this.AddTemporalExtension( + in context, + tileInfo, + orderHintInfo, + frameHeader, + referenceFrame, + secondaryReferenceFrame, + globalMotionVector, + secondaryGlobalMotionVector, + verticalOffset, + horizontalOffset); + + this.AddTemporalExtension( + in context, + tileInfo, + orderHintInfo, + frameHeader, + referenceFrame, + secondaryReferenceFrame, + globalMotionVector, + secondaryGlobalMotionVector, + verticalOffset - 2, + horizontalOffset); + } + + /// + /// Adds one optional temporal extension candidate after applying the normative 64x64 boundary rule. + /// + /// The current block geometry and decoder temporal state. + /// The active tile boundaries. + /// The sequence modulo order-hint configuration. + /// The frame-level motion-vector precision configuration. + /// The canonical inter reference selected for the current block. + /// The secondary compound reference, or . + /// The selected reference's global-motion vector at the current block. + /// The secondary reference's global-motion vector at the current block. + /// The temporal sample row relative to the current block in 4x4 units. + /// The temporal sample column relative to the current block in 4x4 units. + private void AddTemporalExtension( + in ReferenceContext context, + Av1TileInfo tileInfo, + ObuOrderHintInfo orderHintInfo, + ObuFrameHeader frameHeader, + Av1ReferenceFrameType referenceFrame, + Av1ReferenceFrameType secondaryReferenceFrame, + Av1MotionVector globalMotionVector, + Av1MotionVector secondaryGlobalMotionVector, + int blockRow, + int blockColumn) + { + int rowWithinBlock64 = context.RowIndex & (MaximumSearchBlockSize - 1); + int columnWithinBlock64 = context.ColumnIndex & (MaximumSearchBlockSize - 1); + if (rowWithinBlock64 + blockRow < 0 || rowWithinBlock64 + blockRow >= MaximumSearchBlockSize || + columnWithinBlock64 + blockColumn < 0 || columnWithinBlock64 + blockColumn >= MaximumSearchBlockSize) + { + return; + } + + _ = this.AddTemporalCandidate( + in context, + tileInfo, + orderHintInfo, + frameHeader, + referenceFrame, + secondaryReferenceFrame, + globalMotionVector, + secondaryGlobalMotionVector, + blockRow, + blockColumn); + } + + /// + /// Projects and accumulates one temporal motion-field sample. + /// + /// The current block geometry and decoder temporal state. + /// The active tile boundaries. + /// The sequence modulo order-hint configuration. + /// The frame-level motion-vector precision configuration. + /// The canonical inter reference selected for the current block. + /// The secondary compound reference, or . + /// The selected reference's global-motion vector at the current block. + /// The secondary reference's global-motion vector at the current block. + /// The temporal sample row relative to the current block in 4x4 units. + /// The temporal sample column relative to the current block in 4x4 units. + /// when the projected motion field covers the requested position. + private bool AddTemporalCandidate( + in ReferenceContext context, + Av1TileInfo tileInfo, + ObuOrderHintInfo orderHintInfo, + ObuFrameHeader frameHeader, + Av1ReferenceFrameType referenceFrame, + Av1ReferenceFrameType secondaryReferenceFrame, + Av1MotionVector globalMotionVector, + Av1MotionVector secondaryGlobalMotionVector, + int blockRow, + int blockColumn) + { + int rowOffset = (context.RowIndex & 1) != 0 ? blockRow : blockRow + 1; + int columnOffset = (context.ColumnIndex & 1) != 0 ? blockColumn : blockColumn + 1; + int row = context.RowIndex + rowOffset; + int column = context.ColumnIndex + columnOffset; + if (row < tileInfo.ModeInfoRowStart || row >= tileInfo.ModeInfoRowEnd || + column < tileInfo.ModeInfoColumnStart || column >= tileInfo.ModeInfoColumnEnd) + { + return false; + } + + if (!context.TryGetProjectedTemporalMotionVector( + row, + column, + referenceFrame, + orderHintInfo, + frameHeader.AllowHighPrecisionMotionVector, + frameHeader.ForceIntegerMotionVector, + out Av1MotionVector motionVector)) + { + return false; + } + + Av1MotionVector secondaryMotionVector = default; + if (secondaryReferenceFrame > Av1ReferenceFrameType.Intra && + !context.TryGetProjectedTemporalMotionVector( + row, + column, + secondaryReferenceFrame, + orderHintInfo, + frameHeader.AllowHighPrecisionMotionVector, + frameHeader.ForceIntegerMotionVector, + out secondaryMotionVector)) + { + return false; + } + + if (blockRow == 0 && blockColumn == 0 && + (Math.Abs(motionVector.Row - globalMotionVector.Row) >= 16 || + Math.Abs(motionVector.Column - globalMotionVector.Column) >= 16 || + (secondaryReferenceFrame > Av1ReferenceFrameType.Intra && + (Math.Abs(secondaryMotionVector.Row - secondaryGlobalMotionVector.Row) >= 16 || + Math.Abs(secondaryMotionVector.Column - secondaryGlobalMotionVector.Column) >= 16)))) + { + // The packed global-motion context records whether the first temporal sample is absent or differs from + // global motion by at least two full samples in either one-eighth-sample component. + this.ModeContext |= GlobalMotionContextBit; + } + + if (secondaryReferenceFrame > Av1ReferenceFrameType.Intra) + { + this.AddUnique(motionVector, secondaryMotionVector, 2); + } + else + { + this.AddUnique(motionVector, 2); + } + + return true; + } + + /// + /// Extends a short stack with inter vectors from a neighboring block, correcting their temporal direction. + /// + /// The decoded neighboring block. + /// The current block geometry and decoder reference classification. + /// The canonical inter reference selected for the current block. + private void AddExtensionCandidate( + ReferenceBlock candidate, + in ReferenceContext context, + Av1ReferenceFrameType referenceFrame) + { + bool targetSignBias = context.IsReferenceSignBiased(referenceFrame); + + for (int referenceIndex = 0; referenceIndex < 2; referenceIndex++) + { + Av1ReferenceFrameType candidateReference = candidate.GetReferenceFrame(referenceIndex); + if (candidateReference <= Av1ReferenceFrameType.Intra) + { + continue; + } + + Av1MotionVector motionVector = candidate.GetMotionVector(referenceIndex); + if (context.IsReferenceSignBiased(candidateReference) != targetSignBias) + { + motionVector = new Av1MotionVector(-motionVector.Row, -motionVector.Column); + } + + int candidateIndex; + for (candidateIndex = 0; candidateIndex < this.Count; candidateIndex++) + { + if (this.candidates[candidateIndex] == motionVector) + { + break; + } + } + + if (candidateIndex == this.Count && this.Count < CandidateCapacity) + { + // AV1's outer spatial extension only initializes a new stack entry. Unlike the weighted nearest and + // temporal scans, finding an existing vector here must not change its previously accumulated rank. + this.candidates[this.Count] = motionVector; + this.weights[this.Count] = 2; + this.Count++; + } + } + } + + /// + /// Extends a short compound stack from the immediate above and left blocks. + /// + private void ExtendCompoundStack( + in ReferenceContext context, + Av1ReferenceFrameType referenceFrame, + Av1ReferenceFrameType secondaryReferenceFrame, + Av1MotionVector globalMotionVector, + Av1MotionVector secondaryGlobalMotionVector, + bool hasAbove, + bool hasLeft, + int extensionLength) + { + InlineArray2 primaryExact = default; + InlineArray2 secondaryExact = default; + InlineArray2 primaryDifferent = default; + InlineArray2 secondaryDifferent = default; + int primaryExactCount = 0; + int secondaryExactCount = 0; + int primaryDifferentCount = 0; + int secondaryDifferentCount = 0; + int row = context.RowIndex; + int column = context.ColumnIndex; + + for (int index = 0; hasAbove && index < extensionLength;) + { + ReferenceBlock candidate = context.GetModeInfoAt(new Point(column + index, row - 1)); + CollectCompoundExtensionCandidate( + candidate, + in context, + referenceFrame, + secondaryReferenceFrame, + ref primaryExact, + ref primaryExactCount, + ref secondaryExact, + ref secondaryExactCount, + ref primaryDifferent, + ref primaryDifferentCount, + ref secondaryDifferent, + ref secondaryDifferentCount); + + index += candidate.BlockSize.Get4x4WideCount(); + } + + for (int index = 0; hasLeft && index < extensionLength;) + { + ReferenceBlock candidate = context.GetModeInfoAt(new Point(column - 1, row + index)); + CollectCompoundExtensionCandidate( + candidate, + in context, + referenceFrame, + secondaryReferenceFrame, + ref primaryExact, + ref primaryExactCount, + ref secondaryExact, + ref secondaryExactCount, + ref primaryDifferent, + ref primaryDifferentCount, + ref secondaryDifferent, + ref secondaryDifferentCount); + + index += candidate.BlockSize.Get4x4HighCount(); + } + + InlineArray2 primaryList = BuildCompoundExtensionList( + in primaryExact, + primaryExactCount, + in primaryDifferent, + primaryDifferentCount, + globalMotionVector); + + InlineArray2 secondaryList = BuildCompoundExtensionList( + in secondaryExact, + secondaryExactCount, + in secondaryDifferent, + secondaryDifferentCount, + secondaryGlobalMotionVector); + + if (this.Count == 1) + { + int listIndex = primaryList[0] == this.candidates[0] && secondaryList[0] == this.compoundCandidates[0] ? 1 : 0; + this.candidates[1] = primaryList[listIndex]; + this.compoundCandidates[1] = secondaryList[listIndex]; + this.weights[1] = 2; + this.Count = 2; + return; + } + + // The fallback list is positional rather than a weighted candidate scan. Preserve both entries even when + // they are equal so the derived DRL indices retain the same meaning. + for (int index = 0; index < 2; index++) + { + this.candidates[index] = primaryList[index]; + this.compoundCandidates[index] = secondaryList[index]; + this.weights[index] = 2; + } + + this.Count = 2; + } + + /// + /// Collects exact-reference and temporal-direction-corrected fallback vectors from one neighboring block. + /// + private static void CollectCompoundExtensionCandidate( + ReferenceBlock candidate, + in ReferenceContext context, + Av1ReferenceFrameType referenceFrame, + Av1ReferenceFrameType secondaryReferenceFrame, + ref InlineArray2 primaryExact, + ref int primaryExactCount, + ref InlineArray2 secondaryExact, + ref int secondaryExactCount, + ref InlineArray2 primaryDifferent, + ref int primaryDifferentCount, + ref InlineArray2 secondaryDifferent, + ref int secondaryDifferentCount) + { + for (int candidateIndex = 0; candidateIndex < 2; candidateIndex++) + { + Av1ReferenceFrameType candidateReference = candidate.GetReferenceFrame(candidateIndex); + Av1MotionVector candidateMotionVector = candidate.GetMotionVector(candidateIndex); + + for (int targetIndex = 0; targetIndex < 2; targetIndex++) + { + Av1ReferenceFrameType targetReference = targetIndex == 0 ? referenceFrame : secondaryReferenceFrame; + if (candidateReference == targetReference) + { + ref int exactCount = ref (targetIndex == 0 ? ref primaryExactCount : ref secondaryExactCount); + if (exactCount < 2) + { + if (targetIndex == 0) + { + primaryExact[exactCount] = candidateMotionVector; + } + else + { + secondaryExact[exactCount] = candidateMotionVector; + } + + exactCount++; + continue; + } + } + + if (candidateReference <= Av1ReferenceFrameType.Intra) + { + continue; + } + + ref int differentCount = ref (targetIndex == 0 ? ref primaryDifferentCount : ref secondaryDifferentCount); + if (differentCount >= 2) + { + continue; + } + + Av1MotionVector differentMotionVector = candidateMotionVector; + if (context.IsReferenceSignBiased(candidateReference) != context.IsReferenceSignBiased(targetReference)) + { + differentMotionVector = new Av1MotionVector(-differentMotionVector.Row, -differentMotionVector.Column); + } + + if (targetIndex == 0) + { + primaryDifferent[differentCount] = differentMotionVector; + } + else + { + secondaryDifferent[differentCount] = differentMotionVector; + } + + differentCount++; + } + } + } + + /// + /// Builds the two positional fallback entries for one member of a compound reference pair. + /// + private static InlineArray2 BuildCompoundExtensionList( + in InlineArray2 exact, + int exactCount, + in InlineArray2 different, + int differentCount, + Av1MotionVector globalMotionVector) + { + InlineArray2 result = default; + int resultCount = 0; + + for (int index = 0; index < exactCount && resultCount < 2; index++) + { + result[resultCount++] = exact[index]; + } + + for (int index = 0; index < differentCount && resultCount < 2; index++) + { + result[resultCount++] = different[index]; + } + + while (resultCount < 2) + { + result[resultCount++] = globalMotionVector; + } + + return result; + } + + /// + /// Adds a unique candidate or accumulates the weight of an existing candidate. + /// + /// The candidate vector in one-eighth-sample units. + /// The spatial or temporal weight contributed by this occurrence. + private void AddUnique(Av1MotionVector motionVector, int weight) + { + for (int index = 0; index < this.Count; index++) + { + if (this.candidates[index] == motionVector) + { + this.weights[index] += (ushort)weight; + return; + } + } + + if (this.Count < CandidateCapacity) + { + this.candidates[this.Count] = motionVector; + this.weights[this.Count] = (ushort)weight; + this.Count++; + } + } + + /// + /// Adds a unique compound candidate or accumulates the weight of an existing vector pair. + /// + private void AddUnique(Av1MotionVector motionVector, Av1MotionVector compoundMotionVector, int weight) + { + for (int index = 0; index < this.Count; index++) + { + if (this.candidates[index] == motionVector && this.compoundCandidates[index] == compoundMotionVector) + { + this.weights[index] += (ushort)weight; + return; + } + } + + if (this.Count < CandidateCapacity) + { + this.candidates[this.Count] = motionVector; + this.compoundCandidates[this.Count] = compoundMotionVector; + this.weights[this.Count] = (ushort)weight; + this.Count++; + } + } + + /// + /// Sorts one candidate region by descending accumulated weight while retaining scan order for equal weights. + /// + /// The inclusive first candidate index in the region. + /// The exclusive end candidate index in the region. + private void SortByWeight(int start, int end) + { + int length = end; + while (length > start) + { + int lastSwap = start; + for (int index = start + 1; index < length; index++) + { + if (this.weights[index - 1] < this.weights[index]) + { + Av1MotionVector candidate = this.candidates[index - 1]; + this.candidates[index - 1] = this.candidates[index]; + this.candidates[index] = candidate; + + Av1MotionVector compoundCandidate = this.compoundCandidates[index - 1]; + this.compoundCandidates[index - 1] = this.compoundCandidates[index]; + this.compoundCandidates[index] = compoundCandidate; + + ushort weight = this.weights[index - 1]; + this.weights[index - 1] = this.weights[index]; + this.weights[index] = weight; + lastSwap = index; + } + } + + length = lastSwap; + } + } + + /// + /// Determines whether an inter mode decodes at least one new motion-vector component. + /// + private static bool UsesNewMotionVector(Av1PredictionMode mode) + => mode is Av1PredictionMode.NewMotionVector or + Av1PredictionMode.NewNewMotionVector or + Av1PredictionMode.NearestNewMotionVector or + Av1PredictionMode.NewNearestMotionVector or + Av1PredictionMode.NearNewMotionVector or + Av1PredictionMode.NewNearMotionVector; + + /// + /// Provides one allocation-free view over decoder or encoder mode-information storage. + /// + private readonly struct ReferenceContext + { + private readonly Av1SuperblockInfo decodedSuperblock; + private readonly Av1PictureControlSet? encodedPicture; + private readonly Av1FrameInfo? decodedFrame; + + public ReferenceContext(ref Av1PartitionInfo partitionInfo, int superblockModeInfoSize, Av1FrameInfo frameInfo) + { + this.decodedSuperblock = partitionInfo.SuperblockInfo; + this.encodedPicture = null; + this.decodedFrame = frameInfo; + this.BlockSize = partitionInfo.ModeInfo.BlockSize; + this.RowIndex = partitionInfo.RowIndex; + this.ColumnIndex = partitionInfo.ColumnIndex; + this.AvailableAbove = partitionInfo.AvailableAbove; + this.AvailableLeft = partitionInfo.AvailableLeft; + this.ModeBlockToLeftEdge = partitionInfo.ModeBlockToLeftEdge; + this.ModeBlockToRightEdge = partitionInfo.ModeBlockToRightEdge; + this.ModeBlockToTopEdge = partitionInfo.ModeBlockToTopEdge; + this.ModeBlockToBottomEdge = partitionInfo.ModeBlockToBottomEdge; + this.HasTopRight = partitionInfo.HasTopRight(superblockModeInfoSize); + } + + public ReferenceContext( + Av1PictureControlSet picture, + Av1MacroBlockD macroBlock, + Point modeInfoPosition, + Av1BlockSize blockSize, + Av1PartitionType partitionType, + int superblockModeInfoSize) + { + this.decodedSuperblock = default; + this.encodedPicture = picture; + this.decodedFrame = null; + this.BlockSize = blockSize; + this.RowIndex = modeInfoPosition.Y; + this.ColumnIndex = modeInfoPosition.X; + this.AvailableAbove = macroBlock.IsUpAvailable; + this.AvailableLeft = macroBlock.IsLeftAvailable; + this.ModeBlockToLeftEdge = macroBlock.ToLeftEdge; + this.ModeBlockToRightEdge = macroBlock.ToRightEdge; + this.ModeBlockToTopEdge = macroBlock.ToTopEdge; + this.ModeBlockToBottomEdge = macroBlock.ToBottomEdge; + this.HasTopRight = Av1PartitionInfo.HasTopRight( + blockSize, + partitionType, + modeInfoPosition.Y, + modeInfoPosition.X, + superblockModeInfoSize); + } + + public Av1BlockSize BlockSize { get; } + + public int RowIndex { get; } + + public int ColumnIndex { get; } + + public bool AvailableAbove { get; } + + public bool AvailableLeft { get; } + + public int ModeBlockToLeftEdge { get; } + + public int ModeBlockToRightEdge { get; } + + public int ModeBlockToTopEdge { get; } + + public int ModeBlockToBottomEdge { get; } + + public bool HasTopRight { get; } + + public int GetMaxBlockWide() + { + int width = this.BlockSize.GetWidth(); + if (this.ModeBlockToRightEdge < 0) + { + width += this.ModeBlockToRightEdge >> 3; + } + + return width >> Av1Constants.ModeInfoSizeLog2; + } + + public int GetMaxBlockHigh() + { + int height = this.BlockSize.GetHeight(); + if (this.ModeBlockToBottomEdge < 0) + { + height += this.ModeBlockToBottomEdge >> 3; + } + + return height >> Av1Constants.ModeInfoSizeLog2; + } + + public ReferenceBlock GetModeInfoAt(Point position) + { + Av1PictureControlSet? picture = this.encodedPicture; + if (picture is not null) + { + Av1MacroBlockModeInfo encodedModeInfo = picture.GetFromModeInfoGrid(position); + return new ReferenceBlock( + encodedModeInfo.Block.BlockSize, + encodedModeInfo.Block.Mode, + encodedModeInfo.Block.ReferenceFrame, + Av1ReferenceFrameType.None, + picture.GetDisplacementVector(position), + default); + } + + Av1BlockModeInfo decodedModeInfo = this.decodedSuperblock.GetModeInfoAt(position); + return new ReferenceBlock( + decodedModeInfo.BlockSize, + decodedModeInfo.YMode, + decodedModeInfo.ReferenceFrames[0], + decodedModeInfo.ReferenceFrames[1], + decodedModeInfo.MotionVectors[0], + decodedModeInfo.MotionVectors[1]); + } + + public bool IsReferenceSignBiased(Av1ReferenceFrameType referenceFrame) + { + Av1FrameInfo? frameInfo = this.decodedFrame; + return frameInfo is not null && frameInfo.IsReferenceSignBiased(referenceFrame); + } + + public bool TryGetProjectedTemporalMotionVector( + int row, + int column, + Av1ReferenceFrameType referenceFrame, + ObuOrderHintInfo orderHintInfo, + bool allowHighPrecisionMotionVector, + bool forceIntegerMotionVector, + out Av1MotionVector motionVector) + { + Av1FrameInfo? frameInfo = this.decodedFrame; + if (frameInfo is null) + { + motionVector = default; + return false; + } + + return frameInfo.TryGetProjectedTemporalMotionVector( + row, + column, + referenceFrame, + orderHintInfo, + allowHighPrecisionMotionVector, + forceIntegerMotionVector, + out motionVector); + } + } + + /// + /// Carries the neighboring mode fields consumed by reference-vector ranking. + /// + private readonly struct ReferenceBlock + { + private readonly Av1ReferenceFrameType primaryReferenceFrame; + private readonly Av1ReferenceFrameType secondaryReferenceFrame; + private readonly Av1MotionVector primaryMotionVector; + private readonly Av1MotionVector secondaryMotionVector; + + public ReferenceBlock( + Av1BlockSize blockSize, + Av1PredictionMode mode, + Av1ReferenceFrameType primaryReferenceFrame, + Av1ReferenceFrameType secondaryReferenceFrame, + Av1MotionVector primaryMotionVector, + Av1MotionVector secondaryMotionVector) + { + this.BlockSize = blockSize; + this.YMode = mode; + this.primaryReferenceFrame = primaryReferenceFrame; + this.secondaryReferenceFrame = secondaryReferenceFrame; + this.primaryMotionVector = primaryMotionVector; + this.secondaryMotionVector = secondaryMotionVector; + } + + public Av1BlockSize BlockSize { get; } + + public Av1PredictionMode YMode { get; } + + public Av1ReferenceFrameType GetReferenceFrame(int index) + => index == 0 ? this.primaryReferenceFrame : this.secondaryReferenceFrame; + + public Av1MotionVector GetMotionVector(int index) + => index == 0 ? this.primaryMotionVector : this.secondaryMotionVector; + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuChromoSamplePosition.cs b/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuChromoSamplePosition.cs new file mode 100644 index 0000000000..9405e671df --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuChromoSamplePosition.cs @@ -0,0 +1,30 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; + +/// +/// Identifies the position of chroma samples relative to luma samples. +/// +internal enum ObuChromoSamplePosition : byte +{ + /// + /// Unknown. + /// + Unknown = 0, + + /// + /// The chroma sample is horizontally co-located with the top-left luma sample and lies between two luma rows. + /// + Vertical = 1, + + /// + /// The chroma sample is co-located with the top-left luma sample. + /// + Colocated = 2, + + /// + /// Reserved and invalid for AV1 content. + /// + Reserved = 3, +} diff --git a/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuColorConfig.cs b/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuColorConfig.cs new file mode 100644 index 0000000000..4cac5ffdc8 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuColorConfig.cs @@ -0,0 +1,101 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; + +/// +/// Contains the color configuration signaled by an AV1 sequence header. +/// +internal sealed class ObuColorConfig +{ + /// + /// Gets or sets a value indicating whether color-description syntax is present. + /// + public bool IsColorDescriptionPresent { get; set; } + + /// + /// Gets the number of color channels in this image. Can have the value 1 or 3. + /// + public int PlaneCount => this.IsMonochrome ? 1 : Av1Constants.MaxPlanes; + + /// + /// Gets or sets a value indicating whether the image has a single greyscale plane, will have + /// color planes otherwise. + /// + public bool IsMonochrome { get; set; } + + /// + /// Gets or sets the color-primary chromaticities. + /// + public ObuColorPrimaries ColorPrimaries { get; set; } + + /// + /// Gets or sets the transfer characteristics. + /// + public ObuTransferCharacteristics TransferCharacteristics { get; set; } + + /// + /// Gets or sets the matrix coefficients used to derive luma and chroma components. + /// + public ObuMatrixCoefficients MatrixCoefficients { get; set; } + + /// + /// Gets or sets a value indicating whether samples use the full numeric range. + /// + public bool ColorRange { get; set; } + + /// + /// Gets or sets a value indicating whether chroma is subsampled horizontally. + /// + public bool SubSamplingX { get; set; } + + /// + /// Gets or sets a value indicating whether chroma is subsampled vertically. + /// + public bool SubSamplingY { get; set; } + + /// + /// Gets or sets a value indicating whether the U and V planes use separate quantizer deltas. + /// + public bool HasSeparateUvDelta { get; set; } + + /// + /// Gets or sets the chroma sample position for vertically subsampled images. + /// + public ObuChromoSamplePosition ChromaSamplePosition { get; set; } + + /// + /// Gets or sets the encoded sample bit depth. + /// + public Av1BitDepth BitDepth { get; set; } + + /// + /// Gets the color format represented by the monochrome and chroma-subsampling flags. + /// + /// The corresponding AV1 color format. + public Av1ColorFormat GetColorFormat() + { + if (this.IsMonochrome) + { + // AV1 sets both subsampling flags for monochrome sequences even though no chroma planes exist. The + // mono_chrome syntax therefore owns the plane layout and must take precedence over those derived flags. + return Av1ColorFormat.Yuv400; + } + + Av1ColorFormat format = Av1ColorFormat.Yuv400; + if (this.SubSamplingX && this.SubSamplingY) + { + format = Av1ColorFormat.Yuv420; + } + else if (this.SubSamplingX & !this.SubSamplingY) + { + format = Av1ColorFormat.Yuv422; + } + else if (!this.SubSamplingX && !this.SubSamplingY) + { + format = Av1ColorFormat.Yuv444; + } + + return format; + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuColorPrimaries.cs b/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuColorPrimaries.cs new file mode 100644 index 0000000000..2bc013e4a2 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuColorPrimaries.cs @@ -0,0 +1,75 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; + +/// +/// Identifies the CICP color-primary chromaticities signaled by an AV1 sequence. +/// +internal enum ObuColorPrimaries +{ + /// + /// The reserved zero value. + /// + Reserved = 0, + + /// + /// ITU-R BT.709 primaries. + /// + Bt709 = 1, + + /// + /// Unspecified primaries. + /// + Unspecified = 2, + + /// + /// ITU-R BT.470 System M primaries. + /// + Bt470M = 4, + + /// + /// ITU-R BT.470 System B and G primaries. + /// + Bt470BG = 5, + + /// + /// ITU-R BT.601 primaries. + /// + Bt601 = 6, + + /// + /// SMPTE 240M primaries. + /// + Smpte240 = 7, + + /// + /// Generic film primaries. + /// + GenericFilm = 8, + + /// + /// ITU-R BT.2020 and BT.2100 primaries. + /// + Bt2020 = 9, + + /// + /// SMPTE ST 428 CIE XYZ primaries. + /// + Xyz = 10, + + /// + /// SMPTE RP 431-2 primaries. + /// + Smpte431 = 11, + + /// + /// SMPTE EG 432-1 primaries. + /// + Smpte432 = 12, + + /// + /// EBU Tech. 3213-E primaries. + /// + Ebu3213 = 22, +} diff --git a/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuConstraintDirectionalEnhancementFilterParameters.cs b/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuConstraintDirectionalEnhancementFilterParameters.cs new file mode 100644 index 0000000000..a8c7e8d9df --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuConstraintDirectionalEnhancementFilterParameters.cs @@ -0,0 +1,40 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; + +/// +/// Contains the constrained directional enhancement filter parameters for an AV1 frame. +/// +internal sealed class ObuConstraintDirectionalEnhancementFilterParameters +{ + /// + /// Stores the fixed sixteen luma filter strengths without a per-frame array allocation. + /// + private InlineArray16 yStrength; + + /// + /// Stores the fixed sixteen chroma filter strengths without a per-frame array allocation. + /// + private InlineArray16 uvStrength; + + /// + /// Gets or sets the number of bits used to select a filter-strength entry. + /// + public int BitCount { get; set; } + + /// + /// Gets or sets the filter damping value. + /// + public int Damping { get; set; } = 3; + + /// + /// Gets the primary and secondary luma strengths for each filter entry. + /// + public Span YStrength => this.yStrength; + + /// + /// Gets the primary and secondary chroma strengths for each filter entry. + /// + public Span UvStrength => this.uvStrength; +} diff --git a/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuDecoderModelInfo.cs b/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuDecoderModelInfo.cs new file mode 100644 index 0000000000..723db40461 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuDecoderModelInfo.cs @@ -0,0 +1,32 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1; + +/// +/// Contains the field widths and decoding-clock units used by the AV1 decoder model. +/// +internal sealed class ObuDecoderModelInfo +{ + /// + /// Gets or sets BufferDelayLength. Specifies the length of the decoder_buffer_delay and the encoder_buffer_delay + /// syntax elements, in bits. + /// + public uint BufferDelayLength { get; set; } + + /// + /// Gets or sets NumUnitsInDecodingTick. This is the number of time units of a decoding clock operating at the frequency time_scale Hz + /// that corresponds to one increment of a clock tick counter. + /// + public uint NumUnitsInDecodingTick { get; set; } + + /// + /// Gets or sets BufferRemovalTimeLength. Specifies the length of the buffer_removal_time syntax element, in bits. + /// + public uint BufferRemovalTimeLength { get; set; } + + /// + /// Gets or sets the FramePresentationTimeLength. Specifies the length of the frame_presentation_time syntax element, in bits. + /// + public uint FramePresentationTimeLength { get; set; } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuDeltaParameters.cs b/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuDeltaParameters.cs new file mode 100644 index 0000000000..c35cbc11d1 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuDeltaParameters.cs @@ -0,0 +1,25 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; + +/// +/// Contains the delta-quantizer or delta-loop-filter signaling parameters for an AV1 frame. +/// +internal sealed class ObuDeltaParameters +{ + /// + /// Gets or sets a value indicating whether per-block delta values are present. + /// + public bool IsPresent { get; set; } + + /// + /// Gets or sets the delta-value multiplier, which is one, two, four, or eight. + /// + public int Resolution { get; set; } + + /// + /// Gets or sets a value indicating whether separate loop-filter deltas are signaled for multiple filter targets. + /// + public bool IsMulti { get; set; } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuFilmGrainParameters.cs b/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuFilmGrainParameters.cs new file mode 100644 index 0000000000..8838c28ddf --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuFilmGrainParameters.cs @@ -0,0 +1,251 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; + +/// +/// Contains the parameters used to synthesize AV1 film grain for a decoded frame. +/// +internal sealed class ObuFilmGrainParameters +{ + /// + /// Stores the luma scaling-point coordinates without a per-frame array allocation. + /// + private InlineArray14 pointYValue; + + /// + /// Stores the luma scaling-point values without a per-frame array allocation. + /// + private InlineArray14 pointYScaling; + + /// + /// Stores the blue-difference scaling-point coordinates without a per-frame array allocation. + /// + private InlineArray10 pointCbValue; + + /// + /// Stores the blue-difference scaling-point values without a per-frame array allocation. + /// + private InlineArray10 pointCbScaling; + + /// + /// Stores the red-difference scaling-point coordinates without a per-frame array allocation. + /// + private InlineArray10 pointCrValue; + + /// + /// Stores the red-difference scaling-point values without a per-frame array allocation. + /// + private InlineArray10 pointCrScaling; + + /// + /// Stores the luma autoregressive coefficients without a per-frame array allocation. + /// + private InlineArray24 arCoeffsYPlus128; + + /// + /// Stores the blue-difference autoregressive coefficients without a per-frame array allocation. + /// + private InlineArray25 arCoeffsCbPlus128; + + /// + /// Stores the red-difference autoregressive coefficients without a per-frame array allocation. + /// + private InlineArray25 arCoeffsCrPlus128; + + /// + /// Gets or sets a value indicating whether film grain is applied to the displayed frame. + /// + public bool ApplyGrain { get; set; } + + /// + /// Gets or sets the 16-bit seed that initializes pseudo-random film-grain synthesis for this frame. + /// + public uint GrainSeed { get; set; } + + /// + /// Gets or sets a value indicating whether this frame signals a complete parameter set instead of inheriting one. + /// + public bool UpdateGrain { get; set; } + + /// + /// Gets or sets the physical reference-map index from which this frame inherited its film-grain parameters. + /// The index must match one of the frame's seven selected inter-reference slots. + /// + public uint FilmGrainParamsRefIdx { get; set; } + + /// + /// Gets or sets the number of active luma scaling points in the inclusive range zero through fourteen. + /// + public uint NumYPoints { get; set; } + + /// + /// Gets the fourteen-entry storage for the luma scaling-point coordinates. + /// + /// + /// Only the first entries are active. Coordinates use the eight-bit scale and must be + /// strictly increasing; 10-bit and 12-bit sample values are divided by four and sixteen respectively. + /// + public Span PointYValue => this.pointYValue; + + /// + /// Gets the fourteen-entry storage for the luma scaling-point output values. + /// + /// Only the first entries are active. + public Span PointYScaling => this.pointYScaling; + + /// + /// Gets or sets a value indicating whether both chroma scaling functions are derived from luma samples. + /// + public bool ChromaScalingFromLuma { get; set; } + + /// + /// Gets or sets the number of active blue-difference scaling points in the inclusive range zero through ten. + /// + public uint NumCbPoints { get; set; } + + /// + /// Gets or sets the number of active red-difference scaling points in the inclusive range zero through ten. + /// + public uint NumCrPoints { get; set; } + + /// + /// Gets the ten-entry storage for the blue-difference scaling-point coordinates. + /// + /// Only the first entries are active, and active coordinates must be strictly increasing. + public Span PointCbValue => this.pointCbValue; + + /// + /// Gets the ten-entry storage for the blue-difference scaling-point output values. + /// + /// Only the first entries are active. + public Span PointCbScaling => this.pointCbScaling; + + /// + /// Gets the ten-entry storage for the red-difference scaling-point coordinates. + /// + /// Only the first entries are active, and active coordinates must be strictly increasing. + public Span PointCrValue => this.pointCrValue; + + /// + /// Gets the ten-entry storage for the red-difference scaling-point output values. + /// + /// Only the first entries are active. + public Span PointCrScaling => this.pointCrScaling; + + /// + /// Gets or sets the scaling-function shift minus eight. Values from zero through three select an effective shift + /// from eight through eleven for every luma and chroma scaling value. + /// + public uint GrainScalingMinus8 { get; set; } + + /// + /// Gets or sets the autoregressive neighborhood lag in the inclusive range zero through three. + /// + public uint ArCoeffLag { get; set; } + + /// + /// Gets the twenty-four-entry storage for biased luma autoregressive coefficients. + /// + /// The active entry count is 2 * ArCoeffLag * (ArCoeffLag + 1). + public Span ArCoeffsYPlus128 => this.arCoeffsYPlus128; + + /// + /// Gets the twenty-five-entry storage for biased blue-difference autoregressive coefficients. + /// + /// The active entry count includes one additional luma coefficient when luma scaling points are present. + public Span ArCoeffsCbPlus128 => this.arCoeffsCbPlus128; + + /// + /// Gets the twenty-five-entry storage for biased red-difference autoregressive coefficients. + /// + /// The active entry count includes one additional luma coefficient when luma scaling points are present. + public Span ArCoeffsCrPlus128 => this.arCoeffsCrPlus128; + + /// + /// Gets or sets the autoregressive coefficient shift minus six in the inclusive range zero through three. + /// + public uint ArCoeffShiftMinus6 { get; set; } + + /// + /// Gets or sets the right shift applied to generated Gaussian grain samples in the inclusive range zero through three. + /// + public uint GrainScaleShift { get; set; } + + /// + /// Gets or sets the 8-bit blue-difference sample multiplier used to derive the chroma scaling index. + /// + public uint CbMult { get; set; } + + /// + /// Gets or sets the 8-bit average-luma multiplier used to derive the blue-difference scaling index. + /// + public uint CbLumaMult { get; set; } + + /// + /// Gets or sets the 9-bit offset used to derive the blue-difference scaling index. + /// + public uint CbOffset { get; set; } + + /// + /// Gets or sets the 8-bit red-difference sample multiplier used to derive the chroma scaling index. + /// + public uint CrMult { get; set; } + + /// + /// Gets or sets the 8-bit average-luma multiplier used to derive the red-difference scaling index. + /// + public uint CrLumaMult { get; set; } + + /// + /// Gets or sets the 9-bit offset used to derive the red-difference scaling index. + /// + public uint CrOffset { get; set; } + + /// + /// Gets or sets a value indicating whether neighboring film-grain blocks are blended across their boundaries. + /// + public bool OverlapFlag { get; set; } + + /// + /// Gets or sets a value indicating whether grained samples are clipped to the restricted range instead of the full range. + /// + public bool ClipToRestrictedRange { get; set; } + + /// + /// Replaces the complete film-grain parameter set with values retained by a reference frame. + /// + /// The retained reference-frame parameters. + public void CopyFrom(ObuFilmGrainParameters source) + { + this.ApplyGrain = source.ApplyGrain; + this.GrainSeed = source.GrainSeed; + this.UpdateGrain = source.UpdateGrain; + this.FilmGrainParamsRefIdx = source.FilmGrainParamsRefIdx; + this.NumYPoints = source.NumYPoints; + this.pointYValue = source.pointYValue; + this.pointYScaling = source.pointYScaling; + this.ChromaScalingFromLuma = source.ChromaScalingFromLuma; + this.NumCbPoints = source.NumCbPoints; + this.NumCrPoints = source.NumCrPoints; + this.pointCbValue = source.pointCbValue; + this.pointCbScaling = source.pointCbScaling; + this.pointCrValue = source.pointCrValue; + this.pointCrScaling = source.pointCrScaling; + this.GrainScalingMinus8 = source.GrainScalingMinus8; + this.ArCoeffLag = source.ArCoeffLag; + this.arCoeffsYPlus128 = source.arCoeffsYPlus128; + this.arCoeffsCbPlus128 = source.arCoeffsCbPlus128; + this.arCoeffsCrPlus128 = source.arCoeffsCrPlus128; + this.ArCoeffShiftMinus6 = source.ArCoeffShiftMinus6; + this.GrainScaleShift = source.GrainScaleShift; + this.CbMult = source.CbMult; + this.CbLumaMult = source.CbLumaMult; + this.CbOffset = source.CbOffset; + this.CrMult = source.CrMult; + this.CrLumaMult = source.CrLumaMult; + this.CrOffset = source.CrOffset; + this.OverlapFlag = source.OverlapFlag; + this.ClipToRestrictedRange = source.ClipToRestrictedRange; + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuFrameHeader.cs b/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuFrameHeader.cs new file mode 100644 index 0000000000..b18e367858 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuFrameHeader.cs @@ -0,0 +1,352 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1.Motion; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; + +/// +/// Stores the decoded syntax and derived state for one AV1 frame header. +/// +internal sealed class ObuFrameHeader +{ + /// + /// Stores the validity state of the eight reference-frame slots without a per-header array allocation. + /// + private InlineArray8 referenceValid; + + /// + /// Stores the multi-bit order hint associated with each of the eight reference-frame slots. + /// + private InlineArray8 referenceOrderHint; + + /// + /// Stores the frame identifier associated with each of the eight reference-map slots. + /// + private InlineArray8 referenceFrameId; + + /// + /// Stores the reference-map slot selected for each of the seven inter reference types. + /// + private InlineArray8 referenceFrameIndex; + + /// + /// Stores the global-motion model associated with each of the seven inter reference types. + /// + private InlineArray7 globalMotionParameters; + + /// + /// Stores the lossless-coding flag for each of the eight segments without a per-header array allocation. + /// + private InlineArray8 losslessArray; + + /// + /// Gets or sets the temporal-layer identifier carried by the primary frame-header OBU. + /// + public int TemporalId { get; set; } + + /// + /// Gets or sets the spatial-layer identifier carried by the primary frame-header OBU. + /// + public int SpatialId { get; set; } + + /// + /// Gets or sets a value indicating whether motion vectors use integer-sample precision. + /// + public bool ForceIntegerMotionVector { get; set; } + + /// + /// Gets or sets a value indicating whether intra-block copy is permitted. + /// + public bool AllowIntraBlockCopy { get; set; } + + /// + /// Gets or sets a value indicating whether reference-frame motion vectors may be used. + /// + public bool UseReferenceFrameMotionVectors { get; set; } + + /// + /// Gets or sets a value indicating whether motion vectors may use high precision. + /// + public bool AllowHighPrecisionMotionVector { get; set; } + + /// + /// Gets the component precision selected by the integer and high-precision frame flags. + /// + public Av1MotionVectorPrecision MotionVectorPrecision => this.ForceIntegerMotionVector + ? Av1MotionVectorPrecision.Integer + : this.AllowHighPrecisionMotionVector + ? Av1MotionVectorPrecision.EighthSample + : Av1MotionVectorPrecision.QuarterSample; + + /// + /// Gets or sets the frame-level interpolation filter used for inter prediction. + /// + public Av1InterpolationFilter InterpolationFilter { get; set; } + + /// + /// Gets or sets a value indicating whether inter blocks may select a non-translational motion mode. + /// + public bool IsMotionModeSwitchable { get; set; } + + /// + /// Gets or sets the decoded tile layout. + /// + public ObuTileGroupHeader TilesInfo { get; set; } = new ObuTileGroupHeader(); + + /// + /// Gets or sets a value indicating whether every segment uses lossless coding. + /// + public bool CodedLossless { get; set; } + + /// + /// Gets the mutable lossless-coding flags for each segment. + /// + public Span LosslessArray => this.losslessArray; + + /// + /// Gets or sets the frame quantization parameters. + /// + public ObuQuantizationParameters QuantizationParameters { get; set; } = new ObuQuantizationParameters(); + + /// + /// Gets or sets the frame segmentation parameters. + /// + public ObuSegmentationParameters SegmentationParameters { get; set; } = new ObuSegmentationParameters(); + + /// + /// Gets or sets a value indicating whether coding is lossless and no super-resolution scaling is applied. + /// + public bool AllLossless { get; set; } + + /// + /// Gets or sets a value indicating whether warped motion is permitted. + /// + public bool AllowWarpedMotion { get; set; } + + /// + /// Gets or sets the permitted reference prediction mode. + /// + public ObuReferenceMode ReferenceMode { get; set; } + + /// + /// Gets or sets the film-grain synthesis parameters. + /// + public ObuFilmGrainParameters FilmGrainParameters { get; set; } = new ObuFilmGrainParameters(); + + /// + /// Gets or sets a value indicating whether the reduced transform set is used. + /// + public bool UseReducedTransformSet { get; set; } + + /// + /// Gets or sets the loop-filter parameters. + /// + public ObuLoopFilterParameters LoopFilterParameters { get; set; } = new ObuLoopFilterParameters(); + + /// + /// Gets or sets the loop-restoration parameters. + /// + public ObuLoopRestorationParameters LoopRestorationParameters { get; set; } = new ObuLoopRestorationParameters(); + + /// + /// Gets or sets the constrained directional enhancement filter parameters. + /// + public ObuConstraintDirectionalEnhancementFilterParameters CdefParameters { get; set; } = new ObuConstraintDirectionalEnhancementFilterParameters(); + + /// + /// Gets or sets the number of mode-information columns in one stored row. + /// + public int ModeInfoStride { get; set; } + + /// + /// Gets or sets a value indicating whether the frame-end probability update is disabled. + /// + public bool DisableFrameEndUpdateCdf { get; set; } + + /// + /// Gets or sets the skip-mode parameters. + /// + public ObuSkipModeParameters SkipModeParameters { get; set; } = new ObuSkipModeParameters(); + + /// + /// Gets or sets the transform-size selection mode. + /// + public Av1TransformMode TransformMode { get; set; } + + /// + /// Gets or sets the loop-filter delta parameters. + /// + public ObuDeltaParameters DeltaLoopFilterParameters { get; set; } = new ObuDeltaParameters(); + + /// + /// Gets or sets the quantizer delta parameters. + /// + public ObuDeltaParameters DeltaQParameters { get; set; } = new ObuDeltaParameters(); + + /// + /// Gets a value indicating whether the frame uses intra prediction only. + /// + public bool IsIntra => this.FrameType is ObuFrameType.IntraOnlyFrame or ObuFrameType.KeyFrame; + + /// + /// Gets or sets the decoded and rendered frame dimensions. + /// + public ObuFrameSize FrameSize { get; set; } = new ObuFrameSize(); + + /// + /// Gets or sets the frame width in mode-information units. + /// + public int ModeInfoColumnCount { get; set; } + + /// + /// Gets or sets the frame height in mode-information units. + /// + public int ModeInfoRowCount { get; set; } + + /// + /// Gets or sets a value indicating whether an existing reference frame is displayed without decoding a new frame. + /// + public bool ShowExistingFrame { get; set; } + + /// + /// Gets or sets the coded frame type. + /// + public ObuFrameType FrameType { get; set; } + + /// + /// Gets or sets a value indicating whether the decoded frame is immediately displayed. + /// + public bool ShowFrame { get; set; } + + /// + /// Gets or sets a value indicating whether the frame may be displayed by a later header. + /// + public bool ShowableFrame { get; set; } + + /// + /// Gets or sets the reference map index selected when showing an existing frame. + /// + public uint FrameToShowMapIdx { get; set; } + + /// + /// Gets or sets the display frame identifier. + /// + public uint DisplayFrameId { get; set; } + + /// + /// Gets or sets a value indicating whether the frame can be decoded without state from earlier frames. + /// + public bool ErrorResilientMode { get; set; } + + /// + /// Gets or sets a value indicating whether screen-content coding tools are permitted. + /// + public bool AllowScreenContentTools { get; set; } + + /// + /// Gets or sets a value indicating whether probability adaptation is disabled for this frame. + /// + public bool DisableCdfUpdate { get; set; } + + /// + /// Gets or sets the identifier of the current frame. + /// + public uint CurrentFrameId { get; set; } + + /// + /// Gets or sets the frame order hint. + /// + public uint OrderHint { get; set; } + + /// + /// Gets or sets the zero-based inter-reference type that supplies the initial frame context, or the no-reference + /// sentinel. + /// + public uint PrimaryReferenceFrame { get; set; } = Av1Constants.PrimaryReferenceFrameNone; + + /// + /// Gets or sets the resolved reference-map slot supplying the initial frame context, or when + /// the frame uses the default context. + /// + public byte? PrimaryReferenceSlot { get; set; } + + /// + /// Gets or sets the bit mask of reference slots refreshed by this frame. + /// + public uint RefreshFrameFlags { get; set; } + + /// + /// Gets or sets the presentation time signaled by temporal point information. + /// + public uint FramePresentationTime { get; set; } + + /// + /// Gets the validity state of each reference-frame slot. + /// + /// The mutable eight-entry reference-validity table. + public Span GetReferenceValidity() => this.referenceValid; + + /// + /// Gets the multi-bit order hint associated with each reference-frame slot. + /// + /// The mutable eight-entry reference-order-hint table. + public Span GetReferenceOrderHints() => this.referenceOrderHint; + + /// + /// Gets the frame identifier associated with each reference-map slot. + /// + /// The mutable eight-entry reference-frame-identifier table. + public Span GetReferenceFrameIds() => this.referenceFrameId; + + /// + /// Gets the reference-map slot selected for each inter reference type. + /// + /// The mutable seven-entry inter-reference-map table. + public Span GetReferenceFrameIndices() => this.referenceFrameIndex[..Av1Constants.ReferencesPerFrame]; + + /// + /// Gets the global-motion model associated with each canonical inter reference type from LAST through ALTREF. + /// + /// The mutable seven-entry global-motion parameter table. + public Span GetGlobalMotionParameters() => this.globalMotionParameters; + + /// + /// Invalidates retained reference slots whose frame identifiers fall outside the permitted backwards window. + /// + /// The number of bits in the modulo frame-identifier domain. + /// The number of bits used to signal reference-frame identifier deltas. + public void MarkReferenceFrames(int frameIdLength, int deltaFrameIdLength) + { + uint referenceWindow = 1U << deltaFrameIdLength; + uint frameIdModulus = 1U << frameIdLength; + Span referenceFrameIds = this.GetReferenceFrameIds(); + Span referenceValidity = this.GetReferenceValidity(); + + for (int slot = 0; slot < Av1Constants.ReferenceFrameCount; slot++) + { + uint referenceFrameId = referenceFrameIds[slot]; + if (this.CurrentFrameId > referenceWindow) + { + // Without wraparound, a retained identifier is valid only in the closed interval ending at the + // current identifier and extending referenceWindow values backwards. + if (referenceFrameId > this.CurrentFrameId || referenceFrameId < this.CurrentFrameId - referenceWindow) + { + referenceValidity[slot] = false; + } + } + else + { + // When the backwards window crosses zero, valid identifiers occupy both ends of the modulo domain. + // Only the open interval between the current identifier and the wrapped lower bound is invalid. + uint wrappedLowerBound = frameIdModulus + this.CurrentFrameId - referenceWindow; + if (referenceFrameId > this.CurrentFrameId && referenceFrameId < wrappedLowerBound) + { + referenceValidity[slot] = false; + } + } + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuFrameReferenceState.cs b/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuFrameReferenceState.cs new file mode 100644 index 0000000000..82f4ed2cca --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuFrameReferenceState.cs @@ -0,0 +1,101 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; + +/// +/// Stores the uncompressed-header reference state retained by one AV1 OBU reader session. +/// +/// +/// This state describes the eight reference-map slots but does not own reconstructed sample buffers. Pixel ownership +/// remains with the decoder's reference-frame store and is committed before this syntax state is completed. CDF, +/// segmentation, loop-filter, motion, and layer metadata remain on that retained frame owner; the current header's +/// resolved primary-reference slot selects the shared owner instead of duplicating those values here. +/// +internal struct ObuFrameReferenceState +{ + /// + /// Stores whether each of the eight reference-map slots can be selected by a later frame. + /// + private InlineArray8 referenceValidity; + + /// + /// Stores the frame identifier associated with each of the eight reference-map slots. + /// + private InlineArray8 referenceFrameIds; + + /// + /// Stores the order hint associated with each of the eight reference-map slots. + /// + private InlineArray8 referenceOrderHints; + + /// + /// Gets a value indicating whether a completed frame identifier is available for the next header. + /// + public bool HasCurrentFrameId { get; private set; } + + /// + /// Gets the frame identifier of the most recently completed frame. + /// + public uint CurrentFrameId { get; private set; } + + /// + /// Copies the completed reference-map state into a newly created frame header. + /// + /// The frame header that will parse and derive state from the retained map. + public void InitializeFrameHeader(ObuFrameHeader frameHeader) + { + ReadOnlySpan referenceValidity = this.referenceValidity; + ReadOnlySpan referenceFrameIds = this.referenceFrameIds; + ReadOnlySpan referenceOrderHints = this.referenceOrderHints; + + // Only the eight retained-slot tables cross a frame boundary. The seven inter-reference roles are signaled or + // derived afresh for each frame, and the primary context source is resolved from that per-frame mapping. + referenceValidity.CopyTo(frameHeader.GetReferenceValidity()); + referenceFrameIds.CopyTo(frameHeader.GetReferenceFrameIds()); + referenceOrderHints.CopyTo(frameHeader.GetReferenceOrderHints()); + } + + /// + /// Publishes the reference-map transition produced by a successfully completed frame. + /// + /// The completed frame header whose refresh mask selects the replaced slots. + /// + /// A value indicating whether the sequence carries modulo frame identifiers. + /// + public void CompleteFrame(ObuFrameHeader frameHeader, bool frameIdNumbersPresent) + { + Span referenceValidity = frameHeader.GetReferenceValidity(); + Span referenceFrameIds = frameHeader.GetReferenceFrameIds(); + Span referenceOrderHints = frameHeader.GetReferenceOrderHints(); + + // Refresh is published only at this successful completion boundary. Updating the completed header first keeps + // the same object retained by the reconstructed frame owner synchronized with the next parser-session snapshot. + for (int slot = 0; slot < Av1Constants.ReferenceFrameCount; slot++) + { + if ((frameHeader.RefreshFrameFlags & (1U << slot)) != 0) + { + referenceValidity[slot] = true; + referenceFrameIds[slot] = frameHeader.CurrentFrameId; + referenceOrderHints[slot] = frameHeader.OrderHint; + } + } + + referenceValidity.CopyTo(this.referenceValidity); + referenceFrameIds.CopyTo(this.referenceFrameIds); + referenceOrderHints.CopyTo(this.referenceOrderHints); + + if (frameIdNumbersPresent) + { + // the reference decoder keeps one current_frame_id in decoder-session state. The following header snapshots this value as + // its previous identifier before consuming its own current_frame_id syntax. + this.CurrentFrameId = frameHeader.CurrentFrameId; + this.HasCurrentFrameId = true; + } + } + + /// + /// Clears the completed frame identifier and every retained reference-map slot. + /// + public void Reset() => this = default; +} diff --git a/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuFrameSize.cs b/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuFrameSize.cs new file mode 100644 index 0000000000..b5b1cab437 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuFrameSize.cs @@ -0,0 +1,40 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; + +/// +/// Contains the coded, upscaled, and rendered dimensions of an AV1 frame. +/// +internal sealed class ObuFrameSize +{ + /// + /// Gets or sets the coded frame width. + /// + public int FrameWidth { get; set; } + + /// + /// Gets or sets the coded frame height. + /// + public int FrameHeight { get; set; } + + /// + /// Gets or sets the denominator used by AV1 super-resolution scaling. + /// + public int SuperResolutionDenominator { get; set; } + + /// + /// Gets or sets the frame width after super-resolution upscaling. + /// + public int SuperResolutionUpscaledWidth { get; set; } + + /// + /// Gets or sets the intended display width. + /// + public int RenderWidth { get; set; } + + /// + /// Gets or sets the intended display height. + /// + public int RenderHeight { get; set; } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuFrameType.cs b/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuFrameType.cs new file mode 100644 index 0000000000..c06d954548 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuFrameType.cs @@ -0,0 +1,30 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; + +/// +/// Identifies the AV1 frame type signaled by a frame header. +/// +internal enum ObuFrameType +{ + /// + /// A key frame that is decoded without reference to another frame. + /// + KeyFrame = 0, + + /// + /// An inter frame that can refer to previously decoded frames. + /// + InterFrame = 1, + + /// + /// An intra-only frame that does not refresh all reference slots. + /// + IntraOnlyFrame = 2, + + /// + /// A switch frame that permits switching between coded sequences. + /// + SwitchFrame = 3, +} diff --git a/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuHeader.cs b/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuHeader.cs new file mode 100644 index 0000000000..ec2d3e4f30 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuHeader.cs @@ -0,0 +1,45 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; + +/// +/// Contains the parsed header and payload size of an AV1 open bitstream unit. +/// +internal sealed class ObuHeader +{ + /// + /// Gets or sets the number of bytes occupied by the fixed OBU header and optional extension. + /// + public int Size { get; set; } + + /// + /// Gets or sets the OBU payload type. + /// + public ObuType Type { get; set; } + + /// + /// Gets or sets a value indicating whether the OBU carries an explicit payload-size field. + /// + public bool HasSize { get; set; } + + /// + /// Gets or sets a value indicating whether the OBU carries temporal and spatial identifiers. + /// + public bool HasExtension { get; set; } + + /// + /// Gets or sets the temporal-layer identifier. + /// + public int TemporalId { get; set; } + + /// + /// Gets or sets the spatial-layer identifier. + /// + public int SpatialId { get; set; } + + /// + /// Gets or sets the OBU payload size, in bytes. + /// + public int PayloadSize { get; set; } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuLoopFilterParameters.cs b/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuLoopFilterParameters.cs new file mode 100644 index 0000000000..0f0c3f07e3 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuLoopFilterParameters.cs @@ -0,0 +1,77 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; + +/// +/// Contains the AV1 deblocking-loop-filter parameters for a frame. +/// +internal sealed class ObuLoopFilterParameters +{ + /// + /// Stores the horizontal and vertical luma filter levels. + /// + private InlineArray4 filterLevel; + + /// + /// Stores the fixed reference-frame delta table. + /// + private InlineArray8 referenceDeltas; + + /// + /// Stores the fixed prediction-mode delta table. + /// + private InlineArray4 modeDeltas; + + /// + /// Initializes a new instance of the class with the AV1 default reference and mode deltas. + /// + public ObuLoopFilterParameters() + { + // AV1 indexes this table from Intra through Alternate. Golden is -1; Backward remains 0. + this.referenceDeltas[0] = 1; + this.referenceDeltas[4] = -1; + this.referenceDeltas[6] = -1; + this.referenceDeltas[7] = -1; + } + + /// + /// Gets the horizontal and vertical luma filter levels. + /// + public Span FilterLevel => this.filterLevel[..2]; + + /// + /// Gets or sets the U-plane filter level. + /// + public int FilterLevelU { get; set; } + + /// + /// Gets or sets the V-plane filter level. + /// + public int FilterLevelV { get; set; } + + /// + /// Gets or sets the filter sharpness level. + /// + public int SharpnessLevel { get; set; } + + /// + /// Gets or sets a value indicating whether reference-frame and mode deltas are enabled. + /// + public bool ReferenceDeltaModeEnabled { get; set; } + + /// + /// Gets or sets a value indicating whether reference-frame and mode deltas are updated by this frame. + /// + public bool ReferenceDeltaModeUpdate { get; set; } + + /// + /// Gets the filter-level deltas for the AV1 reference-frame categories. + /// + public Span ReferenceDeltas => this.referenceDeltas; + + /// + /// Gets the filter-level deltas for the AV1 prediction modes. + /// + public Span ModeDeltas => this.modeDeltas[..2]; +} diff --git a/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuLoopRestorationItem.cs b/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuLoopRestorationItem.cs new file mode 100644 index 0000000000..06585758fa --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuLoopRestorationItem.cs @@ -0,0 +1,20 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; + +/// +/// Contains the loop-restoration type and unit size for one color plane. +/// +internal struct ObuLoopRestorationItem +{ + /// + /// Gets or sets the restoration-unit size, in samples. + /// + public int Size { get; set; } + + /// + /// Gets or sets the restoration filter type. + /// + public ObuRestorationType Type { get; set; } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuLoopRestorationParameters.cs b/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuLoopRestorationParameters.cs new file mode 100644 index 0000000000..7c1ec6013a --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuLoopRestorationParameters.cs @@ -0,0 +1,40 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; + +/// +/// Contains the AV1 loop-restoration configuration for a frame. +/// +internal sealed class ObuLoopRestorationParameters +{ + /// + /// Stores the fixed three plane configurations without an outer array or per-plane object allocation. + /// + private InlineArray4 items; + + /// + /// Gets or sets a value indicating whether any plane uses loop restoration. + /// + public bool UsesLoopRestoration { get; set; } + + /// + /// Gets or sets a value indicating whether either chroma plane uses loop restoration. + /// + public bool UsesChromaLoopRestoration { get; set; } + + /// + /// Gets the loop-restoration configuration for each plane. + /// + public Span Items => this.items[..3]; + + /// + /// Gets or sets the luma restoration-unit size shift. + /// + public int UnitShift { get; set; } + + /// + /// Gets or sets the chroma restoration-unit size shift relative to luma. + /// + public int UVShift { get; set; } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuMatrixCoefficients.cs b/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuMatrixCoefficients.cs new file mode 100644 index 0000000000..5024a9e9c7 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuMatrixCoefficients.cs @@ -0,0 +1,95 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; + +/// +/// Identifies the CICP matrix coefficients used to derive luma and chroma components. +/// +internal enum ObuMatrixCoefficients +{ + /// + /// The identity matrix used for GBR component ordering. + /// + Identity = 0, + + /// + /// ITU-R BT.709 coefficients. + /// + Bt709 = 1, + + /// + /// Unspecified coefficients. + /// + Unspecified = 2, + + /// + /// United States FCC 73.628 coefficients. + /// + Fcc = 4, + + /// + /// ITU-R BT.470 System B and G coefficients. + /// + Bt470BG = 5, + + /// + /// ITU-R BT.601 coefficients. + /// + Bt601 = 6, + + /// + /// SMPTE 240M coefficients. + /// + Smpte240 = 7, + + /// + /// SMPTE YCgCo coefficients. + /// + SmpteYCgCo = 8, + + /// + /// ITU-R BT.2020 non-constant-luminance coefficients. + /// + Bt2020NonConstantLuminance = 9, + + /// + /// ITU-R BT.2020 constant-luminance coefficients. + /// + Bt2020ConstantLuminance = 10, + + /// + /// SMPTE ST 2085 YDzDx coefficients. + /// + Smpte2085 = 11, + + /// + /// Chromaticity-derived non-constant-luminance coefficients. + /// + ChromaticityDerivedNonConstantLuminance = 12, + + /// + /// Chromaticity-derived constant-luminance coefficients. + /// + ChromaticityDerivedConstantLuminance = 13, + + /// + /// ITU-R BT.2100 ICtCp coefficients. + /// + Bt2100ICtCp = 14, + + /// + /// IPT-C2 coefficients. + /// + IptC2 = 15, + + /// + /// Reversible YCgCo coefficients with two additional encoded bits. + /// + YCgCoRe = 16, + + /// + /// Reversible YCgCo coefficients with one additional encoded bit. + /// + YCgCoRo = 17, +} diff --git a/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuMetadataType.cs b/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuMetadataType.cs new file mode 100644 index 0000000000..1021c11bae --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuMetadataType.cs @@ -0,0 +1,40 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; + +/// +/// Identifies the payload carried by an AV1 metadata OBU. +/// +internal enum ObuMetadataType +{ + /// + /// The reserved zero value. + /// + Reserved = 0, + + /// + /// Content light-level metadata. + /// + HdrCll = 1, + + /// + /// Mastering-display color-volume metadata. + /// + HdrMdcv = 2, + + /// + /// Scalability-structure metadata. + /// + Scalability = 3, + + /// + /// ITU-T T.35 terminal-provider metadata. + /// + ItutT35 = 4, + + /// + /// Timecode metadata. + /// + Timecode = 5, +} diff --git a/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuOperatingPoint.cs b/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuOperatingPoint.cs new file mode 100644 index 0000000000..ced173c3fb --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuOperatingPoint.cs @@ -0,0 +1,61 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; + +/// +/// Contains the sequence-level constraints for an AV1 operating point. +/// +internal sealed class ObuOperatingPoint +{ + /// + /// Gets or sets the operating-point index. + /// + public int OperatorIndex { get; set; } + + /// + /// Gets or sets the AV1 sequence-level index. + /// + public int SequenceLevelIndex { get; set; } + + /// + /// Gets or sets the sequence tier. + /// + public int SequenceTier { get; set; } + + /// + /// Gets or sets a value indicating whether decoder-model timing is present for this operating point. + /// + public bool IsDecoderModelInfoPresent { get; set; } + + /// + /// Gets or sets the decoder-buffer delay measured in decoding ticks. + /// + public uint DecoderBufferDelay { get; set; } + + /// + /// Gets or sets the encoder-buffer delay measured in decoding ticks. + /// + public uint EncoderBufferDelay { get; set; } + + /// + /// Gets or sets a value indicating whether the operating point uses the low-delay decoding model. + /// + public bool LowDelayMode { get; set; } + + /// + /// Gets or sets a value indicating whether an initial display delay is present for this operating point. + /// + public bool IsInitialDisplayDelayPresent { get; set; } + + /// + /// Gets or sets the initial display delay, in decoded frames. + /// + public uint InitialDisplayDelay { get; set; } + + /// + /// Gets or sets the bitmask selecting temporal and spatial layers for the operating point. + /// A value of zero selects the complete coded sequence. + /// + public uint Idc { get; set; } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuOrderHintInfo.cs b/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuOrderHintInfo.cs new file mode 100644 index 0000000000..ba4c193ebf --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuOrderHintInfo.cs @@ -0,0 +1,53 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; + +/// +/// Contains the sequence-level order-hint and dependent prediction-tool settings. +/// +internal sealed class ObuOrderHintInfo +{ + /// + /// Gets or sets a value indicating whether order hints are enabled. + /// + public bool EnableOrderHint { get; set; } + + /// + /// Gets or sets a value indicating whether joint compound prediction is enabled. + /// + public bool EnableJointCompound { get; set; } + + /// + /// Gets or sets a value indicating whether reference-frame motion vectors are enabled. + /// + public bool EnableReferenceFrameMotionVectors { get; set; } + + /// + /// Gets or sets the number of bits used to encode order hints. + /// + public int OrderHintBits { get; set; } + + /// + /// Computes the signed distance between two order hints in the sequence's modulo order-hint domain. + /// + /// The first order hint. + /// The order hint subtracted from . + /// + /// The shortest signed modulo distance, or zero when order hints are disabled for the sequence. + /// + public int GetRelativeDistance(uint first, uint second) + { + if (!this.EnableOrderHint) + { + return 0; + } + + int difference = (int)first - (int)second; + int signBit = 1 << (this.OrderHintBits - 1); + + // Folding around the sign bit maps the unsigned difference to [-2^(bits - 1), 2^(bits - 1)), including + // the wraparound between the highest encoded order hint and zero. + return (difference & (signBit - 1)) - (difference & signBit); + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuQuantizationParameters.cs b/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuQuantizationParameters.cs new file mode 100644 index 0000000000..92f685d466 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuQuantizationParameters.cs @@ -0,0 +1,65 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; + +/// +/// Contains the AV1 quantization parameters for a frame. +/// +internal sealed class ObuQuantizationParameters +{ + /// + /// Stores the effective quantizer index for each of the eight segments without a per-header array allocation. + /// + private InlineArray8 qIndex; + + /// + /// Stores the three plane DC quantizer-index deltas without a per-frame array allocation. + /// + private InlineArray4 deltaQDc; + + /// + /// Stores the three plane AC quantizer-index deltas without a per-frame array allocation. + /// + private InlineArray4 deltaQAc; + + /// + /// Stores the three plane quantization-matrix levels without a per-frame array allocation. + /// + private InlineArray4 qMatrix; + + /// + /// Gets or sets the base quantizer index. + /// + public int BaseQIndex { get; set; } + + /// + /// Gets the mutable effective quantizer indices for each segment. + /// + public Span QIndex => this.qIndex; + + /// + /// Gets or sets a value indicating whether quantization matrices are enabled. + /// + public bool IsUsingQMatrix { get; set; } + + /// + /// Gets the DC quantizer-index deltas for the Y, U, and V planes. + /// + public Span DeltaQDc => this.deltaQDc[..3]; + + /// + /// Gets the AC quantizer-index deltas for the Y, U, and V planes. + /// + public Span DeltaQAc => this.deltaQAc[..3]; + + /// + /// Gets the quantization-matrix level for the Y, U, and V planes. + /// + public Span QMatrix => this.qMatrix[..3]; + + /// + /// Gets or sets a value indicating whether the U and V planes use separate quantizer deltas. + /// + public bool HasSeparateUvDelta { get; set; } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuReader.cs b/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuReader.cs new file mode 100644 index 0000000000..de876efbaa --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuReader.cs @@ -0,0 +1,3081 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1.Motion; +using SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.Quantizers; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; +using SixLabors.ImageSharp.Formats.Heif.Av1.ReferenceFrames; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; + +/// +/// Parses AV1 open bitstream units and supplies decoded tile payloads to an AV1 tile reader. +/// +internal sealed class ObuReader +{ + /// + /// The zero-based sequence-header operating-point index selected by the container. + /// + private readonly byte operatingPointIndex; + + /// + /// The reconstructed frames retained by the owning decoder for inter-frame syntax and prediction. + /// + private readonly Av1ReferenceFrameStore? referenceFrames; + + /// + /// The completed frame-identifier, validity, and order-hint state retained across frame headers in this session. + /// + private ObuFrameReferenceState frameReferenceState; + + /// + /// The temporal- and spatial-layer mask for the selected operating point. + /// + private uint currentOperatingPointIdc; + + /// + /// Initializes a new instance of the class using operating-point index zero without a + /// reconstructed reference map. + /// + public ObuReader() + : this(0) + { + } + + /// + /// Initializes a new instance of the class for one selected AV1 operating point without a + /// reconstructed reference map. + /// + /// The zero-based sequence-header operating-point index to decode. + public ObuReader(byte operatingPointIndex) + => this.operatingPointIndex = operatingPointIndex; + + /// + /// Initializes a new instance of the class for one selected AV1 operating point and + /// retained reference map. + /// + /// The zero-based sequence-header operating-point index to decode. + /// The reconstructed reference frames retained by the owning decoder. + public ObuReader(byte operatingPointIndex, Av1ReferenceFrameStore referenceFrames) + { + this.operatingPointIndex = operatingPointIndex; + this.referenceFrames = referenceFrames; + } + + /// + /// Supplies a tile reader while preserving distinct fixed-reader and factory call contracts. + /// + private interface ITileReaderProvider + { + /// + /// Gets the tile reader for the current frame. + /// + /// The tile reader. + IAv1TileReader Get(); + } + + /// + /// Gets or sets the most recently parsed sequence header. + /// + public ObuSequenceHeader? SequenceHeader { get; set; } + + /// + /// Gets or sets the frame header associated with the current coded frame. + /// + public ObuFrameHeader? FrameHeader { get; set; } + + /// + /// Gets content light-level metadata parsed from the current AV1 session. + /// + public HeifContentLightLevel? ContentLightLevel { get; private set; } + + /// + /// Gets mastering-display color-volume metadata parsed from the current AV1 session. + /// + public HeifMasteringDisplayColorVolume? MasteringDisplayColorVolume { get; private set; } + + /// + /// Gets the sequence header established before frame-dependent syntax is read. + /// + public ObuSequenceHeader CurrentSequenceHeader => + this.SequenceHeader + ?? throw new InvalidImageContentException("AV1 frame syntax appeared before a sequence header."); + + /// + /// Gets the frame header established before frame-dependent syntax is read. + /// + public ObuFrameHeader CurrentFrameHeader => + this.FrameHeader + ?? throw new InvalidImageContentException("AV1 tile syntax appeared before a frame header."); + + /// + /// Clears metadata retained from the preceding bounded AV1 image payload. + /// + public void ResetMetadata() + { + this.ContentLightLevel = null; + this.MasteringDisplayColorVolume = null; + } + + /// + /// Parses every open bitstream unit in one bounded AV1 payload. + /// + /// The reader positioned at the first OBU. + /// The number of bytes available for the bounded payload. + /// Creates one tile reader when the first tile payload of each coded frame is encountered. + /// A value indicating whether each OBU is prefixed by an Annex B length field. + public void ReadAll(ref Av1BitStreamReader reader, int dataSize, Func creator, bool isAnnexB = false) + { + ArgumentNullException.ThrowIfNull(creator); + + this.ReadAll(ref reader, dataSize, new TileReaderFactoryProvider(creator), isAnnexB); + } + + /// + /// Parses every open bitstream unit in one bounded AV1 payload using one existing tile reader. + /// + /// The reader positioned at the first OBU. + /// The number of bytes available for the bounded payload. + /// The tile reader used for each coded frame in the payload. + /// A value indicating whether each OBU is prefixed by an Annex B length field. + public void ReadAll(ref Av1BitStreamReader reader, int dataSize, IAv1TileReader tileReader, bool isAnnexB = false) + { + ArgumentNullException.ThrowIfNull(tileReader); + + this.ReadAll(ref reader, dataSize, new FixedTileReaderProvider(tileReader), isAnnexB); + } + + /// + /// Parses every open bitstream unit in one bounded AV1 payload. + /// + /// The reader positioned at the first OBU. + /// The number of bytes available for the bounded payload. + /// The non-allocating tile-reader source used by this payload. + /// Provides one tile reader for each coded frame. + /// A value indicating whether each OBU is prefixed by an Annex B length field. + private void ReadAll( + ref Av1BitStreamReader reader, + int dataSize, + TTileReaderProvider tileReaderProvider, + bool isAnnexB) + where TTileReaderProvider : struct, ITileReaderProvider + { + bool completed = false; + + try + { + int availableByteCount = reader.Length - Av1Math.DivideBy8Floor(reader.BitPosition); + if ((reader.BitPosition & 0x7) != 0 || (uint)dataSize > (uint)availableByteCount) + { + throw new InvalidImageContentException("The AV1 OBU data boundary is invalid."); + } + + bool seenFrameHeader = false; + int nextTileStart = 0; + Span primaryFrameHeaderPayload = default; + IAv1TileReader? activeDecoder = null; + + while (dataSize > 0) + { + int annexObuSize = 0; + if (isAnnexB) + { + ReadObuSize(ref reader, out annexObuSize, out int annexLengthSize); + if (annexLengthSize > dataSize || annexObuSize < 1) + { + throw new InvalidImageContentException("The Annex B AV1 OBU length is invalid."); + } + + dataSize -= annexLengthSize; + if (annexObuSize > dataSize) + { + throw new InvalidImageContentException("The Annex B AV1 OBU exceeds its temporal-unit boundary."); + } + } + else if (dataSize < 1) + { + throw new InvalidImageContentException("The AV1 OBU header is truncated."); + } + + int obuStartBitPosition = reader.BitPosition; + ObuHeader header = ReadObuHeaderSize(ref reader, out _); + int headerAndLengthSize = (reader.BitPosition - obuStartBitPosition) >> 3; + int boundedObuSize = isAnnexB ? annexObuSize : dataSize; + if (headerAndLengthSize > boundedObuSize) + { + throw new InvalidImageContentException("The AV1 OBU header exceeds its declared boundary."); + } + + // AV1-ISOBMFF permits the final low-overhead OBU to omit its size field. In that form the remaining + // sample bytes are the payload, which also makes this OBU final because no following boundary exists. + int payloadSize = header.HasSize ? header.PayloadSize : boundedObuSize - headerAndLengthSize; + if ((uint)payloadSize > (uint)(boundedObuSize - headerAndLengthSize)) + { + throw new InvalidImageContentException("The AV1 OBU payload exceeds its declared boundary."); + } + + int completeObuSize = headerAndLengthSize + payloadSize; + if (isAnnexB && completeObuSize != annexObuSize) + { + throw new InvalidImageContentException("The nested and Annex B AV1 OBU lengths do not match."); + } + + dataSize -= isAnnexB ? annexObuSize : completeObuSize; + header.PayloadSize = payloadSize; + + // A dedicated payload reader prevents malformed syntax from consuming the following OBU. The parent + // advances once here, so ignored metadata, padding, and reserved OBUs are skipped without copying. + Span obuPayload = reader.ReadBytes(payloadSize); + + // AV1 operating_point_idc uses bits 0-7 for temporal IDs and bits 8-11 for spatial IDs. the reference decoder + // requires both selected bits for an extended OBU, while an all-zero mask and unextended OBUs apply + // universally. Sequence headers establish the mask and temporal delimiters define framing, so neither + // can be filtered even when their extension identifies a layer outside the selected operating point. + bool isOperatingPointIndependent = header.Type is ObuType.SequenceHeader or ObuType.TemporalDelimiter; + bool isInCurrentOperatingPoint = this.currentOperatingPointIdc == 0 + || !header.HasExtension + || (((this.currentOperatingPointIdc >> header.TemporalId) & 1U) != 0 + && ((this.currentOperatingPointIdc >> (header.SpatialId + 8)) & 1U) != 0); + + if (!isOperatingPointIndependent && !isInCurrentOperatingPoint) + { + continue; + } + + Av1BitStreamReader payloadReader = new(obuPayload); + IAv1TileReader? decoderToComplete = null; + int decodedPayloadSize; + + switch (header.Type) + { + case ObuType.SequenceHeader: + if (seenFrameHeader) + { + throw new InvalidImageContentException("An AV1 sequence header interrupts an incomplete coded frame."); + } + + this.SequenceHeader = new(); + ReadSequenceHeader(ref payloadReader, this.SequenceHeader); + if (this.operatingPointIndex >= this.SequenceHeader.OperatingPoint.Length) + { + throw new InvalidImageContentException( + $"The AV1 operating-point selector requests index {this.operatingPointIndex}, " + + $"but the sequence header declares {this.SequenceHeader.OperatingPoint.Length} operating points."); + } + + this.currentOperatingPointIdc = this.SequenceHeader.OperatingPoint[this.operatingPointIndex].Idc; + + // A sequence header starts a new reference domain. Clear both the syntax snapshot and decoded + // owners only after the complete header and selected operating point have been accepted, so a + // later inter header cannot pair an empty parser map with samples retained from the old sequence. + this.frameReferenceState.Reset(); + this.referenceFrames?.Reset(); + decodedPayloadSize = Av1Math.DivideBy8Floor(payloadReader.BitPosition); + break; + case ObuType.FrameHeader: + if (this.SequenceHeader is null) + { + throw new InvalidImageContentException("An AV1 frame header appears before its sequence header."); + } + + if (seenFrameHeader) + { + throw new InvalidImageContentException("An AV1 frame contains more than one primary frame header."); + } + + seenFrameHeader = true; + ObuFrameHeader primaryFrameHeader = new() + { + TemporalId = header.TemporalId, + SpatialId = header.SpatialId + }; + + this.frameReferenceState.InitializeFrameHeader(primaryFrameHeader); + this.FrameHeader = primaryFrameHeader; + this.ReadFrameHeader(ref payloadReader, header, trailingBit: true); + decodedPayloadSize = Av1Math.DivideBy8Floor(payloadReader.BitPosition); + primaryFrameHeaderPayload = obuPayload[..decodedPayloadSize]; + + if (primaryFrameHeader.ShowExistingFrame) + { + // This header completes by selecting retained samples; no tile group belongs to it. + activeDecoder ??= tileReaderProvider.Get(); + decoderToComplete = activeDecoder; + } + + break; + case ObuType.RedundantFrameHeader: + if (!seenFrameHeader) + { + throw new InvalidImageContentException("A redundant AV1 frame header appears before its primary frame header."); + } + + if (primaryFrameHeaderPayload.Length > obuPayload.Length + || !obuPayload[..primaryFrameHeaderPayload.Length].SequenceEqual(primaryFrameHeaderPayload)) + { + throw new InvalidImageContentException("The redundant AV1 frame header does not match its primary header."); + } + + // The primary header already owns the decoded frame state. Matching its encoded bytes avoids + // parsing the same adaptive frame-header syntax twice. + decodedPayloadSize = primaryFrameHeaderPayload.Length; + break; + case ObuType.Frame: + if (this.SequenceHeader is null) + { + throw new InvalidImageContentException("An AV1 frame appears before its sequence header."); + } + + if (seenFrameHeader) + { + throw new InvalidImageContentException("A combined AV1 frame OBU follows a separate frame header."); + } + + seenFrameHeader = true; + ObuFrameHeader combinedFrameHeader = new() + { + TemporalId = header.TemporalId, + SpatialId = header.SpatialId + }; + + this.frameReferenceState.InitializeFrameHeader(combinedFrameHeader); + this.FrameHeader = combinedFrameHeader; + this.ReadFrameHeader(ref payloadReader, header, trailingBit: false); + primaryFrameHeaderPayload = obuPayload[..Av1Math.DivideBy8Floor(payloadReader.BitPosition)]; + + if (combinedFrameHeader.ShowExistingFrame) + { + // The reference decoder permits show_existing_frame only in a standalone frame-header OBU. A + // combined frame OBU is required to continue with a tile group and therefore cannot use + // the header-only retained-frame presentation form. + throw new InvalidImageContentException("A combined AV1 frame OBU cannot display an existing frame."); + } + + goto TILE_GROUP; + case ObuType.TileGroup: + TILE_GROUP: + if (!seenFrameHeader) + { + throw new InvalidImageContentException("An AV1 tile group appears before its frame header."); + } + + activeDecoder ??= tileReaderProvider.Get(); + + // A combined frame OBU reaches this label after its frame-header portion has + // been consumed, leaving the same tile-group syntax as a standalone tile OBU. + this.ReadTileGroup(ref payloadReader, activeDecoder, header, ref nextTileStart, out bool frameDecodingFinished); + if (frameDecodingFinished) + { + decoderToComplete = activeDecoder; + } + + decodedPayloadSize = Av1Math.DivideBy8Floor(payloadReader.BitPosition); + break; + case ObuType.TemporalDelimiter: + if (seenFrameHeader) + { + throw new InvalidImageContentException("An AV1 temporal delimiter interrupts an incomplete coded frame."); + } + + // AV1 section 5.6 defines no delimiter syntax. The common post-switch validation still permits + // zero bytes between the empty syntax and the declared payload boundary, matching the reference decoder. + decodedPayloadSize = 0; + break; + case ObuType.Metadata: + decodedPayloadSize = this.ReadMetadata(obuPayload); + break; + case ObuType.TileList: + // Tile-list OBUs require AV1 large-scale tile mode, which this decoder does not implement. + // Rejecting the syntax avoids silently returning a partial reconstruction. + throw new InvalidImageContentException("AV1 tile-list OBUs are not supported."); + case ObuType.Padding: + int lastNonzeroIndex = obuPayload.Length - 1; + while (lastNonzeroIndex >= 0 && obuPayload[lastNonzeroIndex] == 0) + { + lastNonzeroIndex--; + } + + // AV1 padding contains only its trailing one bit and optional zero bytes. A header-only + // padding OBU is also valid, so the empty payload bypasses this final-byte check. + if (lastNonzeroIndex >= 0 && obuPayload[lastNonzeroIndex] != 0x80) + { + throw new InvalidImageContentException("The AV1 padding OBU has invalid trailing bits."); + } + + if (obuPayload.Length > 0 && lastNonzeroIndex < 0) + { + throw new InvalidImageContentException("The AV1 padding OBU is missing its trailing one bit."); + } + + decodedPayloadSize = payloadSize; + break; + default: + // Reserved OBUs do not contribute to this still-image reconstruction pass. Their declared payload has + // already been skipped by the parent reader. The reference decoder rejects a nonempty unrecognized + // payload that contains only zeros because it has no trailing one bit. + if (payloadSize > 0) + { + int ignoredLastNonzeroIndex = payloadSize - 1; + while (ignoredLastNonzeroIndex >= 0 && obuPayload[ignoredLastNonzeroIndex] == 0) + { + ignoredLastNonzeroIndex--; + } + + if (ignoredLastNonzeroIndex < 0) + { + throw new InvalidImageContentException("The ignored AV1 OBU is missing its trailing one bit."); + } + } + + decodedPayloadSize = payloadSize; + break; + } + + // Parsed syntax may be followed only by zero bytes within its declared OBU payload. Ignored metadata + // and reserved OBUs set decodedPayloadSize to the full payload because their syntax is not consumed here. + for (int i = decodedPayloadSize; i < obuPayload.Length; i++) + { + if (obuPayload[i] != 0) + { + throw new InvalidImageContentException("The AV1 OBU contains nonzero data after its decoded syntax."); + } + } + + if (decoderToComplete is not null) + { + // Complete reconstruction and reference-buffer ownership before publishing the matching syntax + // state. Any decoder failure leaves the preceding session snapshot intact for deterministic cleanup. + decoderToComplete.CompleteFrame(); + this.frameReferenceState.CompleteFrame( + this.CurrentFrameHeader, + this.CurrentSequenceHeader.IsFrameIdNumbersPresent); + + activeDecoder = null; + seenFrameHeader = false; + nextTileStart = 0; + primaryFrameHeaderPayload = default; + } + } + + if (seenFrameHeader || activeDecoder is not null) + { + throw new InvalidImageContentException("The AV1 payload ends before the current coded frame is complete."); + } + + completed = true; + } + catch (IndexOutOfRangeException exception) + { + throw new InvalidImageContentException("The AV1 OBU syntax exceeds its payload boundary.", exception); + } + catch (ArgumentOutOfRangeException exception) + { + throw new InvalidImageContentException("The AV1 OBU syntax exceeds its payload boundary.", exception); + } + finally + { + if (!completed) + { + // A bounded payload can commit earlier layers before a later OBU fails. Those transitions cannot be + // rolled back after displaced owners have been released, so invalidate the complete decoder session. + this.Reset(); + } + } + } + + /// + /// Clears all parser, tile-reader, syntax-reference, and reconstructed-reference state owned by this session. + /// + public void Reset() + { + this.SequenceHeader = null; + this.FrameHeader = null; + this.ResetMetadata(); + this.currentOperatingPointIdc = 0; + this.frameReferenceState.Reset(); + this.referenceFrames?.Reset(); + } + + /// + /// Reads the fixed OBU header and optional extension fields. + /// + /// The reader positioned at an OBU header. + /// The parsed OBU header. + private static ObuHeader ReadObuHeader(ref Av1BitStreamReader reader) + { + ObuHeader header = new(); + if (reader.ReadBoolean()) + { + throw new ImageFormatException("Forbidden bit in header should be unset."); + } + + header.Size = 1; + header.Type = (ObuType)reader.ReadLiteral(4); + header.HasExtension = reader.ReadBoolean(); + header.HasSize = reader.ReadBoolean(); + + // The reference decoder consumes obu_reserved_1bit without rejecting its value. Reserved fields do not change the + // decoded syntax, so accepting either value preserves forward-compatible framing while the forbidden bit + // remains a hard error above. + _ = reader.ReadBoolean(); + + if (header.HasExtension) + { + header.Size++; + header.TemporalId = (int)reader.ReadLiteral(3); + header.SpatialId = (int)reader.ReadLiteral(2); + + // The reference decoder likewise consumes extension_header_reserved_3bits without interpreting their value. + _ = reader.ReadLiteral(3); + } + else + { + header.SpatialId = 0; + header.TemporalId = 0; + } + + return header; + } + + /// + /// Reads an OBU size encoded as an unsigned little-endian base-128 value. + /// + /// The reader positioned at the size value. + /// The decoded OBU size. + /// The number of bytes occupied by the encoded size. + private static void ReadObuSize(ref Av1BitStreamReader reader, out int obuSize, out int lengthSize) + { + ulong rawSize = reader.ReadLittleEndianBytes128(out lengthSize); + if (rawSize > int.MaxValue) + { + throw new InvalidImageContentException("The AV1 OBU size exceeds the supported image payload limit."); + } + + obuSize = (int)rawSize; + } + + /// + /// Reads an OBU header followed by its optional payload-size field. + /// + /// The reader positioned at an OBU header. + /// The number of bytes occupied by the payload-size field. + /// The parsed OBU header and payload size. + private static ObuHeader ReadObuHeaderSize(ref Av1BitStreamReader reader, out int lengthSize) + { + ObuHeader header = ReadObuHeader(ref reader); + lengthSize = 0; + if (header.HasSize) + { + ReadObuSize(ref reader, out int payloadSize, out lengthSize); + header.PayloadSize = payloadSize; + } + + return header; + } + + /// + /// Reads and validates the trailing one bit followed by zero padding. + /// + /// The reader positioned at the trailing bits. + /// Consumes a byte, if already byte aligned before the check. + private static void ReadTrailingBits(ref Av1BitStreamReader reader) + { + int bitsBeforeAlignment = 8 - (reader.BitPosition & 0x7); + uint trailing = reader.ReadLiteral(bitsBeforeAlignment); + if (trailing != (1U << (bitsBeforeAlignment - 1))) + { + throw new ImageFormatException("Trailing bits not properly formatted."); + } + } + + /// + /// Consumes zero padding until the reader reaches a byte boundary. + /// + /// The reader to align. + private static void AlignToByteBoundary(ref Av1BitStreamReader reader) + { + int alignmentStartPosition = reader.BitPosition; + while ((reader.BitPosition & 0x7) > 0) + { + int paddingBitPosition = reader.BitPosition; + if (reader.ReadBoolean()) + { + string message = + $"Incorrect byte alignment padding bit at offset {paddingBitPosition}; alignment started at offset {alignmentStartPosition}."; + + throw new ImageFormatException(message); + } + } + } + + /// + /// Reads and validates one AV1 metadata OBU payload. + /// + /// The bounded metadata payload. + /// The number of payload bytes occupied by decoded syntax. + private int ReadMetadata(Span payload) + { + int metadataOffset = 0; + ulong metadataTypeValue = ReadMetadataType(payload, ref metadataOffset); + Span metadataPayload = payload[metadataOffset..]; + + if (metadataTypeValue == (ulong)ObuMetadataType.Reserved + || metadataTypeValue > (ulong)ObuMetadataType.Timecode) + { + // Reserved and private metadata have no syntax the decoder can interpret. libaom still requires their + // opaque payload, including its trailing bit, to contain at least one nonzero byte. + if (FindLastNonzeroByteIndex(metadataPayload) < 0) + { + throw new InvalidImageContentException("The AV1 metadata OBU is missing its trailing one bit."); + } + + return payload.Length; + } + + ObuMetadataType metadataType = (ObuMetadataType)metadataTypeValue; + if (metadataType is ObuMetadataType.HdrCll or ObuMetadataType.HdrMdcv) + { + Av1CodecConfiguration.ReadHdrMetadata( + payload, + "AV1 metadata OBU", + out HeifContentLightLevel? contentLightLevel, + out HeifMasteringDisplayColorVolume? masteringDisplayColorVolume); + + this.ContentLightLevel = contentLightLevel ?? this.ContentLightLevel; + this.MasteringDisplayColorVolume = masteringDisplayColorVolume ?? this.MasteringDisplayColorVolume; + return payload.Length; + } + + if (metadataType == ObuMetadataType.ItutT35) + { + ValidateItutT35Metadata(metadataPayload); + return payload.Length; + } + + Av1BitStreamReader metadataReader = new(metadataPayload); + if (metadataType == ObuMetadataType.Scalability) + { + ReadScalabilityMetadata(ref metadataReader); + } + else + { + ReadTimecodeMetadata(ref metadataReader); + } + + ReadTrailingBits(ref metadataReader); + return metadataOffset + Av1Math.DivideBy8Floor(metadataReader.BitPosition); + } + + /// + /// Reads the unsigned little-endian base-128 metadata type. + /// + /// The bounded metadata payload. + /// The byte offset, advanced past the metadata type. + /// The decoded metadata type. + private static ulong ReadMetadataType(ReadOnlySpan payload, ref int offset) + { + ulong value = 0; + for (int i = 0; i < 8; i++) + { + if ((uint)offset >= (uint)payload.Length) + { + throw new InvalidImageContentException("The AV1 metadata type is truncated."); + } + + byte current = payload[offset++]; + value |= (ulong)(current & 0x7F) << (i * 7); + if ((current & 0x80) == 0) + { + return value; + } + } + + throw new InvalidImageContentException("The AV1 metadata type exceeds the permitted LEB128 length."); + } + + /// + /// Validates byte-aligned ITU-T T.35 metadata syntax and trailing bits. + /// + /// The metadata bytes following the metadata type. + private static void ValidateItutT35Metadata(ReadOnlySpan payload) + { + if (payload.IsEmpty) + { + throw new InvalidImageContentException("The AV1 ITU-T T.35 country code is missing."); + } + + int countryCodeSize = payload[0] == 0xFF ? 2 : 1; + if (payload.Length < countryCodeSize) + { + throw new InvalidImageContentException("The AV1 ITU-T T.35 country-code extension byte is missing."); + } + + int lastNonzeroIndex = FindLastNonzeroByteIndex(payload); + if (lastNonzeroIndex < countryCodeSize || payload[lastNonzeroIndex] != 0x80) + { + throw new InvalidImageContentException("The AV1 ITU-T T.35 metadata has invalid trailing bits."); + } + } + + /// + /// Reads scalability metadata syntax so its bounded payload and trailing bits can be validated. + /// + /// The metadata payload reader. + private static void ReadScalabilityMetadata(ref Av1BitStreamReader reader) + { + const uint scalabilityStructureMode = 14; + uint scalabilityMode = reader.ReadLiteral(8); + if (scalabilityMode != scalabilityStructureMode) + { + return; + } + + int spatialLayerCount = (int)reader.ReadLiteral(2) + 1; + bool hasSpatialLayerDimensions = reader.ReadBoolean(); + bool hasSpatialLayerDescriptions = reader.ReadBoolean(); + bool hasTemporalGroupDescriptions = reader.ReadBoolean(); + + // AV1 reserves these three bits and requires decoders to consume and ignore them. + _ = reader.ReadLiteral(3); + + if (hasSpatialLayerDimensions) + { + for (int i = 0; i < spatialLayerCount; i++) + { + _ = reader.ReadLiteral(16); + _ = reader.ReadLiteral(16); + } + } + + if (hasSpatialLayerDescriptions) + { + for (int i = 0; i < spatialLayerCount; i++) + { + _ = reader.ReadLiteral(8); + } + } + + if (hasTemporalGroupDescriptions) + { + int temporalGroupSize = (int)reader.ReadLiteral(8); + for (int i = 0; i < temporalGroupSize; i++) + { + _ = reader.ReadLiteral(3); + _ = reader.ReadBoolean(); + _ = reader.ReadBoolean(); + int referenceCount = (int)reader.ReadLiteral(3); + + for (int j = 0; j < referenceCount; j++) + { + _ = reader.ReadLiteral(8); + } + } + } + } + + /// + /// Reads timecode metadata syntax so its bounded payload and trailing bits can be validated. + /// + /// The metadata payload reader. + private static void ReadTimecodeMetadata(ref Av1BitStreamReader reader) + { + _ = reader.ReadLiteral(5); + bool hasFullTimestamp = reader.ReadBoolean(); + _ = reader.ReadBoolean(); + _ = reader.ReadBoolean(); + _ = reader.ReadLiteral(9); + + if (hasFullTimestamp) + { + _ = reader.ReadLiteral(6); + _ = reader.ReadLiteral(6); + _ = reader.ReadLiteral(5); + } + else if (reader.ReadBoolean()) + { + _ = reader.ReadLiteral(6); + if (reader.ReadBoolean()) + { + _ = reader.ReadLiteral(6); + if (reader.ReadBoolean()) + { + _ = reader.ReadLiteral(5); + } + } + } + + int timeOffsetLength = (int)reader.ReadLiteral(5); + if (timeOffsetLength > 0) + { + _ = reader.ReadLiteral(timeOffsetLength); + } + } + + /// + /// Finds the final nonzero byte in one bounded payload. + /// + /// The payload to inspect. + /// The final nonzero byte index, or -1 when every byte is zero. + private static int FindLastNonzeroByteIndex(ReadOnlySpan payload) + { + for (int i = payload.Length - 1; i >= 0; i--) + { + if (payload[i] != 0) + { + return i; + } + } + + return -1; + } + + /// + /// Computes the mode-information dimensions and stride for the current frame. + /// + /// The sequence header defining the maximum frame geometry and superblock size. + private void ComputeImageSize(ObuSequenceHeader sequenceHeader) + { + ObuFrameHeader frameHeader = this.CurrentFrameHeader; + frameHeader.ModeInfoColumnCount = 2 * ((frameHeader.FrameSize.FrameWidth + 7) >> 3); + frameHeader.ModeInfoRowCount = 2 * ((frameHeader.FrameSize.FrameHeight + 7) >> 3); + frameHeader.ModeInfoStride = Av1Math.AlignPowerOf2(sequenceHeader.MaxFrameWidth, Av1Constants.MaxSuperBlockSizeLog2) >> Av1Constants.ModeInfoSizeLog2; + } + + /// + /// Reads an AV1 sequence-header OBU payload. + /// + /// The reader positioned at the sequence-header payload. + /// The sequence header to populate. + public static void ReadSequenceHeader(ref Av1BitStreamReader reader, ObuSequenceHeader sequenceHeader) + { + sequenceHeader.SequenceProfile = (ObuSequenceProfile)reader.ReadLiteral(3); + if (sequenceHeader.SequenceProfile > Av1Constants.MaxSequenceProfile) + { + throw new ImageFormatException("Unknown sequence profile."); + } + + sequenceHeader.IsStillPicture = reader.ReadBoolean(); + sequenceHeader.IsReducedStillPictureHeader = reader.ReadBoolean(); + if (!sequenceHeader.IsStillPicture && sequenceHeader.IsReducedStillPictureHeader) + { + // The reduced header omits state required by a multi-frame sequence, so AV1 permits it only when the + // sequence is explicitly declared to contain a single still picture. + throw new InvalidImageContentException("An AV1 reduced still-picture header requires the still-picture flag."); + } + + if (sequenceHeader.IsReducedStillPictureHeader) + { + sequenceHeader.TimingInfo = null; + sequenceHeader.DecoderModelInfoPresentFlag = false; + sequenceHeader.InitialDisplayDelayPresentFlag = false; + ObuOperatingPoint operatingPoint = new(); + sequenceHeader.OperatingPoint[0] = operatingPoint; + operatingPoint.OperatorIndex = 0; + operatingPoint.SequenceLevelIndex = (int)reader.ReadLiteral(Av1Constants.LevelBits); + if (!IsValidSequenceLevel(sequenceHeader.OperatingPoint[0].SequenceLevelIndex)) + { + throw new ImageFormatException("Invalid sequence level."); + } + + operatingPoint.SequenceTier = 0; + operatingPoint.IsDecoderModelInfoPresent = false; + operatingPoint.IsInitialDisplayDelayPresent = false; + } + else + { + sequenceHeader.TimingInfoPresentFlag = reader.ReadBoolean(); + if (sequenceHeader.TimingInfoPresentFlag) + { + ReadTimingInfo(ref reader, sequenceHeader); + sequenceHeader.DecoderModelInfoPresentFlag = reader.ReadBoolean(); + if (sequenceHeader.DecoderModelInfoPresentFlag) + { + ReadDecoderModelInfo(ref reader, sequenceHeader); + } + else + { + sequenceHeader.DecoderModelInfoPresentFlag = false; + } + } + + sequenceHeader.InitialDisplayDelayPresentFlag = reader.ReadBoolean(); + int operatingPointsCnt = (int)reader.ReadLiteral(Av1Constants.OperatingPointCountBits) + 1; + if (sequenceHeader.OperatingPoint.Length != operatingPointsCnt) + { + sequenceHeader.OperatingPoint = new ObuOperatingPoint[operatingPointsCnt]; + } + + for (int i = 0; i < operatingPointsCnt; i++) + { + sequenceHeader.OperatingPoint[i] = new ObuOperatingPoint + { + Idc = reader.ReadLiteral(Av1Constants.OperatingPointIdcBits), + SequenceLevelIndex = (int)reader.ReadLiteral(Av1Constants.LevelBits) + }; + if (!IsValidSequenceLevel(sequenceHeader.OperatingPoint[i].SequenceLevelIndex)) + { + throw new InvalidImageContentException("The AV1 sequence header contains an undefined sequence-level index."); + } + + if (sequenceHeader.OperatingPoint[i].SequenceLevelIndex >= Av1Constants.SequenceTierMinimumLevelIndex) + { + sequenceHeader.OperatingPoint[i].SequenceTier = (int)reader.ReadLiteral(1); + } + else + { + sequenceHeader.OperatingPoint[i].SequenceTier = 0; + } + + if (sequenceHeader.DecoderModelInfoPresentFlag) + { + sequenceHeader.OperatingPoint[i].IsDecoderModelInfoPresent = reader.ReadBoolean(); + if (sequenceHeader.OperatingPoint[i].IsDecoderModelInfoPresent) + { + // Retain the scheduling values so the parsed sequence header can be written again without + // losing decoder-model state that is independent from pixel reconstruction. + ObuDecoderModelInfo decoderModelInfo = sequenceHeader.GetDecoderModelInfo(); + ReadOperatingParametersInfo( + ref reader, + (int)decoderModelInfo.BufferDelayLength, + sequenceHeader.OperatingPoint[i]); + } + } + else + { + sequenceHeader.OperatingPoint[i].IsDecoderModelInfoPresent = false; + } + + if (sequenceHeader.InitialDisplayDelayPresentFlag) + { + sequenceHeader.OperatingPoint[i].IsInitialDisplayDelayPresent = reader.ReadBoolean(); + if (sequenceHeader.OperatingPoint[i].IsInitialDisplayDelayPresent) + { + sequenceHeader.OperatingPoint[i].InitialDisplayDelay = reader.ReadLiteral(4) + 1; + if (sequenceHeader.OperatingPoint[i].InitialDisplayDelay > 10) + { + throw new InvalidImageContentException("The AV1 initial display delay exceeds ten decoded frames."); + } + } + } + } + } + + // The operating-point selector is supplied by the bounded item or sequence decoder. Every operating point is + // still parsed above because its timing syntax precedes the shared coded-image dimensions. + sequenceHeader.FrameWidthBits = (int)reader.ReadLiteral(4) + 1; + sequenceHeader.FrameHeightBits = (int)reader.ReadLiteral(4) + 1; + sequenceHeader.MaxFrameWidth = (int)reader.ReadLiteral(sequenceHeader.FrameWidthBits) + 1; + sequenceHeader.MaxFrameHeight = (int)reader.ReadLiteral(sequenceHeader.FrameHeightBits) + 1; + if (sequenceHeader.IsReducedStillPictureHeader) + { + sequenceHeader.IsFrameIdNumbersPresent = false; + } + else + { + sequenceHeader.IsFrameIdNumbersPresent = reader.ReadBoolean(); + } + + if (sequenceHeader.IsFrameIdNumbersPresent) + { + sequenceHeader.DeltaFrameIdLength = (int)reader.ReadLiteral(4) + 2; + sequenceHeader.AdditionalFrameIdLength = reader.ReadLiteral(3) + 1; + sequenceHeader.FrameIdLength = sequenceHeader.DeltaFrameIdLength + (int)sequenceHeader.AdditionalFrameIdLength; + if (sequenceHeader.FrameIdLength > 16) + { + throw new InvalidImageContentException("The AV1 frame identifier length exceeds sixteen bits."); + } + } + + sequenceHeader.Use128x128Superblock = reader.ReadBoolean(); + sequenceHeader.EnableFilterIntra = reader.ReadBoolean(); + sequenceHeader.EnableIntraEdgeFilter = reader.ReadBoolean(); + + if (sequenceHeader.IsReducedStillPictureHeader) + { + sequenceHeader.EnableInterIntraCompound = false; + sequenceHeader.EnableMaskedCompound = false; + sequenceHeader.EnableWarpedMotion = false; + sequenceHeader.EnableDualFilter = false; + sequenceHeader.OrderHintInfo.EnableJointCompound = false; + sequenceHeader.OrderHintInfo.EnableReferenceFrameMotionVectors = false; + sequenceHeader.ForceScreenContentTools = Av1Constants.SelectScreenContentTools; + sequenceHeader.ForceIntegerMotionVector = Av1Constants.SelectIntegerMotionVector; + sequenceHeader.OrderHintInfo.OrderHintBits = 0; + } + else + { + sequenceHeader.EnableInterIntraCompound = reader.ReadBoolean(); + sequenceHeader.EnableMaskedCompound = reader.ReadBoolean(); + sequenceHeader.EnableWarpedMotion = reader.ReadBoolean(); + sequenceHeader.EnableDualFilter = reader.ReadBoolean(); + sequenceHeader.EnableOrderHint = reader.ReadBoolean(); + if (sequenceHeader.EnableOrderHint) + { + sequenceHeader.OrderHintInfo.EnableJointCompound = reader.ReadBoolean(); + sequenceHeader.OrderHintInfo.EnableReferenceFrameMotionVectors = reader.ReadBoolean(); + } + else + { + sequenceHeader.OrderHintInfo.EnableJointCompound = false; + sequenceHeader.OrderHintInfo.EnableReferenceFrameMotionVectors = false; + } + + bool seqChooseScreenContentTools = reader.ReadBoolean(); + if (seqChooseScreenContentTools) + { + sequenceHeader.ForceScreenContentTools = Av1Constants.SelectScreenContentTools; + } + else + { + sequenceHeader.ForceScreenContentTools = (int)reader.ReadLiteral(1); + } + + if (sequenceHeader.ForceScreenContentTools > 0) + { + bool seqChooseIntegerMv = reader.ReadBoolean(); + if (seqChooseIntegerMv) + { + sequenceHeader.ForceIntegerMotionVector = Av1Constants.SelectIntegerMotionVector; + } + else + { + sequenceHeader.ForceIntegerMotionVector = (int)reader.ReadLiteral(1); + } + } + else + { + sequenceHeader.ForceIntegerMotionVector = Av1Constants.SelectIntegerMotionVector; + } + + if (sequenceHeader.EnableOrderHint) + { + sequenceHeader.OrderHintInfo.OrderHintBits = (int)reader.ReadLiteral(3) + 1; + } + else + { + sequenceHeader.OrderHintInfo.OrderHintBits = 0; + } + } + + sequenceHeader.EnableSuperResolution = reader.ReadBoolean(); + sequenceHeader.EnableCdef = reader.ReadBoolean(); + sequenceHeader.EnableRestoration = reader.ReadBoolean(); + sequenceHeader.ColorConfig = ReadColorConfig(ref reader, sequenceHeader); + sequenceHeader.AreFilmGrainingParametersPresent = reader.ReadBoolean(); + ReadTrailingBits(ref reader); + } + + /// + /// Reads the sequence color configuration. + /// + /// The reader positioned at the color-configuration syntax. + /// The sequence header that determines the permitted color formats. + /// The parsed color configuration. + private static ObuColorConfig ReadColorConfig(ref Av1BitStreamReader reader, ObuSequenceHeader sequenceHeader) + { + ObuColorConfig colorConfig = new(); + ReadBitDepth(ref reader, colorConfig, sequenceHeader); + colorConfig.IsMonochrome = false; + if (sequenceHeader.SequenceProfile != ObuSequenceProfile.High) + { + colorConfig.IsMonochrome = reader.ReadBoolean(); + } + + colorConfig.IsColorDescriptionPresent = reader.ReadBoolean(); + colorConfig.ColorPrimaries = ObuColorPrimaries.Unspecified; + colorConfig.TransferCharacteristics = ObuTransferCharacteristics.Unspecified; + colorConfig.MatrixCoefficients = ObuMatrixCoefficients.Unspecified; + if (colorConfig.IsColorDescriptionPresent) + { + colorConfig.ColorPrimaries = (ObuColorPrimaries)reader.ReadLiteral(8); + colorConfig.TransferCharacteristics = (ObuTransferCharacteristics)reader.ReadLiteral(8); + colorConfig.MatrixCoefficients = (ObuMatrixCoefficients)reader.ReadLiteral(8); + } + + colorConfig.ColorRange = false; + colorConfig.SubSamplingX = false; + colorConfig.SubSamplingY = false; + colorConfig.ChromaSamplePosition = ObuChromoSamplePosition.Unknown; + colorConfig.HasSeparateUvDelta = false; + if (colorConfig.IsMonochrome) + { + colorConfig.ColorRange = reader.ReadBoolean(); + colorConfig.SubSamplingX = true; + colorConfig.SubSamplingY = true; + return colorConfig; + } + else if ( + colorConfig.ColorPrimaries == ObuColorPrimaries.Bt709 && + colorConfig.TransferCharacteristics == ObuTransferCharacteristics.Srgb && + colorConfig.MatrixCoefficients == ObuMatrixCoefficients.Identity) + { + if (sequenceHeader.SequenceProfile != ObuSequenceProfile.High + && !(sequenceHeader.SequenceProfile == ObuSequenceProfile.Professional + && colorConfig.BitDepth == Av1BitDepth.TwelveBit)) + { + throw new InvalidImageContentException("The AV1 sRGB identity-matrix color configuration is incompatible with its sequence profile."); + } + + // AV1 defines this RGB identity-matrix combination as full-range 4:4:4 and omits + // the range and subsampling syntax that other color combinations carry. + colorConfig.ColorRange = true; + colorConfig.SubSamplingX = false; + colorConfig.SubSamplingY = false; + } + else + { + colorConfig.ColorRange = reader.ReadBoolean(); + switch (sequenceHeader.SequenceProfile) + { + case ObuSequenceProfile.Main: + colorConfig.SubSamplingX = true; + colorConfig.SubSamplingY = true; + break; + case ObuSequenceProfile.High: + colorConfig.SubSamplingX = false; + colorConfig.SubSamplingY = false; + break; + case ObuSequenceProfile.Professional: + default: + if (colorConfig.BitDepth == Av1BitDepth.TwelveBit) + { + colorConfig.SubSamplingX = reader.ReadBoolean(); + if (colorConfig.SubSamplingX) + { + colorConfig.SubSamplingY = reader.ReadBoolean(); + } + } + else + { + colorConfig.SubSamplingX = true; + colorConfig.SubSamplingY = false; + } + + break; + } + + if (colorConfig.MatrixCoefficients == ObuMatrixCoefficients.Identity + && (colorConfig.SubSamplingX || colorConfig.SubSamplingY)) + { + throw new InvalidImageContentException("The AV1 identity matrix requires 4:4:4 color sampling."); + } + + if (colorConfig.SubSamplingX && colorConfig.SubSamplingY) + { + colorConfig.ChromaSamplePosition = (ObuChromoSamplePosition)reader.ReadLiteral(2); + } + } + + colorConfig.HasSeparateUvDelta = reader.ReadBoolean(); + return colorConfig; + } + + /// + /// Reads the decoder-model field widths and decoding-clock units. + /// + /// The reader positioned at the decoder-model syntax. + /// The sequence header that receives the decoder-model information. + private static void ReadDecoderModelInfo(ref Av1BitStreamReader reader, ObuSequenceHeader sequenceHeader) => sequenceHeader.DecoderModelInfo = new ObuDecoderModelInfo + { + BufferDelayLength = reader.ReadLiteral(5) + 1, + NumUnitsInDecodingTick = reader.ReadLiteral(32), + BufferRemovalTimeLength = reader.ReadLiteral(5) + 1, + FramePresentationTimeLength = reader.ReadLiteral(5) + 1 + }; + + /// + /// Reads the decoder-model parameters for one operating point. + /// + /// The reader positioned at the operating-point parameters. + /// The bit width of each encoded buffer delay. + /// The operating point that receives the decoded parameters. + private static void ReadOperatingParametersInfo( + ref Av1BitStreamReader reader, + int bufferDelayLength, + ObuOperatingPoint operatingPoint) + { + operatingPoint.DecoderBufferDelay = reader.ReadLiteral(bufferDelayLength); + operatingPoint.EncoderBufferDelay = reader.ReadLiteral(bufferDelayLength); + operatingPoint.LowDelayMode = reader.ReadBoolean(); + } + + /// + /// Reads the sequence timing information. + /// + /// The reader positioned at the timing-information syntax. + /// The sequence header that receives the timing information. + private static void ReadTimingInfo(ref Av1BitStreamReader reader, ObuSequenceHeader sequenceHeader) + { + uint numUnitsInDisplayTick = reader.ReadLiteral(32); + uint timeScale = reader.ReadLiteral(32); + if (numUnitsInDisplayTick == 0 || timeScale == 0) + { + throw new InvalidImageContentException("The AV1 timing tick and time scale must both be nonzero."); + } + + sequenceHeader.TimingInfo = new ObuTimingInfo + { + NumUnitsInDisplayTick = numUnitsInDisplayTick, + TimeScale = timeScale, + EqualPictureInterval = reader.ReadBoolean() + }; + + if (sequenceHeader.TimingInfo.EqualPictureInterval) + { + uint numTicksPerPictureMinusOne = reader.ReadUnsignedVariableLength(); + if (numTicksPerPictureMinusOne == uint.MaxValue) + { + throw new InvalidImageContentException("The AV1 ticks-per-picture value exceeds its permitted range."); + } + + sequenceHeader.TimingInfo.NumTicksPerPicture = numTicksPerPictureMinusOne + 1; + } + } + + /// + /// Reads the bit depth permitted by the selected sequence profile. + /// + /// The reader positioned at the high-bit-depth flag. + /// The color configuration that receives the bit depth. + /// The sequence header containing the selected profile. + private static void ReadBitDepth(ref Av1BitStreamReader reader, ObuColorConfig colorConfig, ObuSequenceHeader sequenceHeader) + { + bool hasHighBitDepth = reader.ReadBoolean(); + if (sequenceHeader.SequenceProfile == ObuSequenceProfile.Professional && hasHighBitDepth) + { + colorConfig.BitDepth = reader.ReadBoolean() ? Av1BitDepth.TwelveBit : Av1BitDepth.TenBit; + } + else if (sequenceHeader.SequenceProfile <= ObuSequenceProfile.Professional) + { + colorConfig.BitDepth = hasHighBitDepth ? Av1BitDepth.TenBit : Av1BitDepth.EightBit; + } + else + { + colorConfig.BitDepth = Av1BitDepth.EightBit; + } + } + + /// + /// Reads the super-resolution parameters and derives the coded frame width. + /// + /// The reader positioned at the super-resolution syntax. + private void ReadSuperResolutionParameters(ref Av1BitStreamReader reader) + { + ObuSequenceHeader sequenceHeader = this.CurrentSequenceHeader; + ObuFrameHeader frameHeader = this.CurrentFrameHeader; + bool useSuperResolution = false; + + if (sequenceHeader.EnableSuperResolution) + { + useSuperResolution = reader.ReadBoolean(); + } + + if (useSuperResolution) + { + frameHeader.FrameSize.SuperResolutionDenominator = + (int)reader.ReadLiteral(Av1Constants.SuperResolutionScaleBits) + Av1Constants.SuperResolutionScaleDenominatorMinimum; + } + else + { + frameHeader.FrameSize.SuperResolutionDenominator = Av1Constants.ScaleNumerator; + } + + frameHeader.FrameSize.SuperResolutionUpscaledWidth = frameHeader.FrameSize.FrameWidth; + + // AV1 signals the upscaled width first. Tile and block decoding use the nearest-integer coded width obtained + // from the fixed scale numerator and signaled denominator. + frameHeader.FrameSize.FrameWidth = + ((frameHeader.FrameSize.SuperResolutionUpscaledWidth * Av1Constants.ScaleNumerator) + + (frameHeader.FrameSize.SuperResolutionDenominator / 2)) / + frameHeader.FrameSize.SuperResolutionDenominator; + + if (frameHeader.FrameSize.SuperResolutionDenominator != Av1Constants.ScaleNumerator) + { + // Appendix A requires an active super-resolution coded width of at least 16 samples, + // except when the signaled upscaled image itself is narrower than that minimum. + int minimumWidth = Math.Min(16, frameHeader.FrameSize.SuperResolutionUpscaledWidth); + frameHeader.FrameSize.FrameWidth = Math.Max(minimumWidth, frameHeader.FrameSize.FrameWidth); + } + } + + /// + /// Reads the optional render dimensions for the current frame. + /// + /// The reader positioned at the render-size syntax. + private void ReadRenderSize(ref Av1BitStreamReader reader) + { + ObuFrameHeader frameHeader = this.CurrentFrameHeader; + bool renderSizeAndFrameSizeDifferent = reader.ReadBoolean(); + + if (renderSizeAndFrameSizeDifferent) + { + // render_width_minus_1 and render_height_minus_1 are fixed 16-bit fields, independent of the sequence's + // coded-dimension bit widths. + frameHeader.FrameSize.RenderWidth = (int)reader.ReadLiteral(16) + 1; + frameHeader.FrameSize.RenderHeight = (int)reader.ReadLiteral(16) + 1; + } + else + { + frameHeader.FrameSize.RenderWidth = frameHeader.FrameSize.SuperResolutionUpscaledWidth; + frameHeader.FrameSize.RenderHeight = frameHeader.FrameSize.FrameHeight; + } + } + + /// + /// Reads or derives the current frame dimensions. + /// + /// The reader positioned at the frame-size syntax. + /// A value indicating whether dimensions are signaled instead of inherited from the sequence maximum. + private void ReadFrameSize(ref Av1BitStreamReader reader, bool frameSizeOverrideFlag) + { + ObuSequenceHeader sequenceHeader = this.CurrentSequenceHeader; + ObuFrameHeader frameHeader = this.CurrentFrameHeader; + + if (frameSizeOverrideFlag) + { + frameHeader.FrameSize.FrameWidth = (int)reader.ReadLiteral(sequenceHeader.FrameWidthBits) + 1; + frameHeader.FrameSize.FrameHeight = (int)reader.ReadLiteral(sequenceHeader.FrameHeightBits) + 1; + + // Section 5.9.7 signals frame dimensions using the sequence maxima's bit widths, but the resulting values + // remain constrained by those maxima. Rejecting the oversized result here prevents later buffer geometry + // from accepting a value that the sequence header does not permit. + if (frameHeader.FrameSize.FrameWidth > sequenceHeader.MaxFrameWidth || + frameHeader.FrameSize.FrameHeight > sequenceHeader.MaxFrameHeight) + { + throw new InvalidImageContentException("AV1 frame dimensions exceed the sequence maximum dimensions."); + } + } + else + { + frameHeader.FrameSize.FrameWidth = sequenceHeader.MaxFrameWidth; + frameHeader.FrameSize.FrameHeight = sequenceHeader.MaxFrameHeight; + } + + this.ReadSuperResolutionParameters(ref reader); + this.ComputeImageSize(sequenceHeader); + } + + /// + /// Reads or inherits inter-frame dimensions using the seven selected reference roles. + /// + /// The reader positioned at the frame-size-with-references syntax. + /// The retained reconstructed frames selected by the current reference mapping. + private void ReadFrameSizeWithReferences(ref Av1BitStreamReader reader, Av1ReferenceFrameStore referenceFrames) + { + ObuSequenceHeader sequenceHeader = this.CurrentSequenceHeader; + ObuFrameHeader frameHeader = this.CurrentFrameHeader; + ObuFrameSize frameSize = frameHeader.FrameSize; + Span referenceFrameIndices = frameHeader.GetReferenceFrameIndices(); + bool foundReference = false; + + // frame_size_with_refs carries one found_ref bit per selected role only until the first one is set. A set bit + // terminates this syntax immediately; no flags for the remaining roles are present in the bitstream. + for (int reference = 0; reference < Av1Constants.ReferencesPerFrame; reference++) + { + if (!reader.ReadBoolean()) + { + continue; + } + + Av1ReferenceFrame referenceFrame = referenceFrames.ResolveRequired((int)referenceFrameIndices[reference]); + ObuFrameSize referenceSize = referenceFrame.FrameHeader.FrameSize; + + // AV1 5.9.7 inherits the reference buffer's visible post-super-resolution dimensions, corresponding to + // the reference decoder's y_crop_width and y_crop_height, plus its render rectangle. The current frame then signals its own + // super-resolution denominator, so the reference's coded width and denominator are not copied. + frameSize.FrameWidth = referenceFrame.FrameBuffer.Width; + frameSize.FrameHeight = referenceFrame.FrameBuffer.Height; + frameSize.RenderWidth = referenceSize.RenderWidth; + frameSize.RenderHeight = referenceSize.RenderHeight; + this.ReadSuperResolutionParameters(ref reader); + this.ComputeImageSize(sequenceHeader); + foundReference = true; + break; + } + + if (!foundReference) + { + // When no reference supplies dimensions, frame_size_with_refs carries the ordinary explicit frame size, + // current super-resolution syntax, and render-size syntax in that order. + this.ReadFrameSize(ref reader, true); + this.ReadRenderSize(ref reader); + } + + bool hasCompatibleReferenceSize = false; + ObuColorConfig colorConfig = sequenceHeader.ColorConfig; + + for (int reference = 0; reference < Av1Constants.ReferencesPerFrame; reference++) + { + Av1ReferenceFrame referenceFrame = referenceFrames.ResolveRequired((int)referenceFrameIndices[reference]); + int referenceWidth = referenceFrame.FrameBuffer.Width; + int referenceHeight = referenceFrame.FrameBuffer.Height; + + // AV1 6.8.6 permits a reference dimension from one half through sixteen times the current coded + // dimension. setup_frame_size_with_refs requires at least one of the seven selected roles to satisfy both + // axes before the frame may proceed. + hasCompatibleReferenceSize |= + (2 * frameSize.FrameWidth) >= referenceWidth && + (2 * frameSize.FrameHeight) >= referenceHeight && + frameSize.FrameWidth <= (16 * referenceWidth) && + frameSize.FrameHeight <= (16 * referenceHeight); + + ObuColorConfig referenceColorConfig = referenceFrame.FrameBuffer.ColorConfig; + + // Every selected reference participates in the same prediction sample domain. Mixing bit depth or chroma + // subsampling would change sample interpretation and is prohibited even when that role is not selected by + // any block in the current frame. + if (referenceFrame.FrameBuffer.BitDepth != colorConfig.BitDepth || + referenceColorConfig.SubSamplingX != colorConfig.SubSamplingX || + referenceColorConfig.SubSamplingY != colorConfig.SubSamplingY) + { + throw new InvalidImageContentException("An AV1 inter frame selects a reference with an incompatible color format."); + } + } + + if (!hasCompatibleReferenceSize) + { + throw new InvalidImageContentException("An AV1 inter frame has no reference with compatible dimensions."); + } + } + + /// + /// Reads the frame-level interpolation-filter selection. + /// + /// The reader positioned at the interpolation-filter syntax. + /// The fixed filter family or the per-block switchable selection. + private static Av1InterpolationFilter ReadFrameInterpolationFilter(ref Av1BitStreamReader reader) + { + // A leading one omits the two-bit fixed-family field and delegates the choice to each inter block. Otherwise, + // the literal values map directly to regular, smooth, sharp, and bilinear as defined by AV1 6.10.2. + return reader.ReadBoolean() + ? Av1InterpolationFilter.Switchable + : (Av1InterpolationFilter)reader.ReadLiteral(2); + } + + /// + /// Reads the tile layout and derives tile boundaries in mode-information units. + /// + /// The reader positioned at the tile-information syntax. + /// The sequence header defining superblock geometry. + /// The frame header defining the current frame geometry. + /// The parsed tile layout. + private static ObuTileGroupHeader ReadTileInfo(ref Av1BitStreamReader reader, ObuSequenceHeader sequenceHeader, ObuFrameHeader frameHeader) + { + ObuTileGroupHeader tileInfo = new(); + int superblockColumnCount; + int superblockRowCount; + int superblockSizeLog2 = sequenceHeader.SuperblockSizeLog2; + int superblockShift = superblockSizeLog2 - Av1Constants.ModeInfoSizeLog2; + superblockColumnCount = (frameHeader.ModeInfoColumnCount + sequenceHeader.SuperblockModeInfoSize - 1) >> superblockShift; + superblockRowCount = (frameHeader.ModeInfoRowCount + sequenceHeader.SuperblockModeInfoSize - 1) >> superblockShift; + + int maxTileAreaOfSuperBlock = Av1Constants.MaxTileArea >> (superblockSizeLog2 << 1); + + // The bitstream constrains tile dimensions in superblocks, while the decoder stores + // boundaries in mode-information units for direct use during block traversal. + tileInfo.MaxTileWidthSuperblock = Av1Constants.MaxTileWidth >> superblockSizeLog2; + tileInfo.MaxTileHeightSuperblock = (Av1Constants.MaxTileArea / Av1Constants.MaxTileWidth) >> superblockSizeLog2; + tileInfo.MinLog2TileColumnCount = TileLog2(tileInfo.MaxTileWidthSuperblock, superblockColumnCount); + tileInfo.MaxLog2TileColumnCount = (int)Av1Math.CeilLog2((uint)Math.Min(superblockColumnCount, Av1Constants.MaxTileColumnCount)); + tileInfo.MaxLog2TileRowCount = (int)Av1Math.CeilLog2((uint)Math.Min(superblockRowCount, Av1Constants.MaxTileRowCount)); + tileInfo.MinLog2TileCount = Math.Max(tileInfo.MinLog2TileColumnCount, TileLog2(maxTileAreaOfSuperBlock, superblockColumnCount * superblockRowCount)); + tileInfo.HasUniformTileSpacing = reader.ReadBoolean(); + + // Boundary storage is bounded by AV1's active tile limits. Sequence-sized arrays would retain + // thousands of unused entries on wide frames even though AV1 permits at most 64 rows or columns. + if (tileInfo.HasUniformTileSpacing) + { + tileInfo.TileColumnCountLog2 = tileInfo.MinLog2TileColumnCount; + while (tileInfo.TileColumnCountLog2 < tileInfo.MaxLog2TileColumnCount) + { + if (reader.ReadBoolean()) + { + tileInfo.TileColumnCountLog2++; + } + else + { + break; + } + } + + int tileWidthSuperblock = Av1Math.DivideLog2Ceiling(superblockColumnCount, tileInfo.TileColumnCountLog2); + DebugGuard.MustBeLessThanOrEqualTo(tileWidthSuperblock, tileInfo.MaxTileWidthSuperblock, nameof(tileWidthSuperblock)); + int i = 0; + for (int startSuperblock = 0; startSuperblock < superblockColumnCount; startSuperblock += tileWidthSuperblock) + { + tileInfo.TileColumnStartModeInfo[i] = startSuperblock << superblockShift; + i++; + } + + tileInfo.TileColumnStartModeInfo[i] = frameHeader.ModeInfoColumnCount; + tileInfo.TileColumnCount = i; + + tileInfo.MinLog2TileRowCount = Math.Max(tileInfo.MinLog2TileCount - tileInfo.TileColumnCountLog2, 0); + tileInfo.TileRowCountLog2 = tileInfo.MinLog2TileRowCount; + while (tileInfo.TileRowCountLog2 < tileInfo.MaxLog2TileRowCount) + { + if (reader.ReadBoolean()) + { + tileInfo.TileRowCountLog2++; + } + else + { + break; + } + } + + int tileHeightSuperblock = Av1Math.DivideLog2Ceiling(superblockRowCount, tileInfo.TileRowCountLog2); + DebugGuard.MustBeLessThanOrEqualTo(tileHeightSuperblock, tileInfo.MaxTileHeightSuperblock, nameof(tileHeightSuperblock)); + i = 0; + for (int startSuperblock = 0; startSuperblock < superblockRowCount; startSuperblock += tileHeightSuperblock) + { + tileInfo.TileRowStartModeInfo[i] = startSuperblock << superblockShift; + i++; + } + + tileInfo.TileRowStartModeInfo[i] = frameHeader.ModeInfoRowCount; + tileInfo.TileRowCount = i; + } + else + { + uint widestTileSuperBlock = 0U; + int startSuperBlock = 0; + int i = 0; + for (; startSuperBlock < superblockColumnCount; i++) + { + if (i == Av1Constants.MaxTileColumnCount) + { + throw new InvalidImageContentException("The AV1 frame exceeds the maximum tile-column count."); + } + + tileInfo.TileColumnStartModeInfo[i] = startSuperBlock << superblockShift; + uint maxWidth = (uint)Math.Min(superblockColumnCount - startSuperBlock, tileInfo.MaxTileWidthSuperblock); + uint widthInSuperBlocks = reader.ReadNonSymmetric(maxWidth) + 1; + widestTileSuperBlock = Math.Max(widthInSuperBlocks, widestTileSuperBlock); + startSuperBlock += (int)widthInSuperBlocks; + } + + if (startSuperBlock != superblockColumnCount) + { + throw new ImageFormatException("Super block tiles width does not add up to total width."); + } + + tileInfo.TileColumnStartModeInfo[i] = frameHeader.ModeInfoColumnCount; + tileInfo.TileColumnCount = i; + tileInfo.TileColumnCountLog2 = TileLog2(1, tileInfo.TileColumnCount); + if (tileInfo.MinLog2TileCount > 0) + { + maxTileAreaOfSuperBlock = (superblockRowCount * superblockColumnCount) >> (tileInfo.MinLog2TileCount + 1); + } + else + { + maxTileAreaOfSuperBlock = superblockRowCount * superblockColumnCount; + } + + DebugGuard.MustBeGreaterThan(widestTileSuperBlock, 0U, nameof(widestTileSuperBlock)); + tileInfo.MaxTileHeightSuperblock = Math.Max(maxTileAreaOfSuperBlock / (int)widestTileSuperBlock, 1); + + startSuperBlock = 0; + for (i = 0; startSuperBlock < superblockRowCount; i++) + { + if (i == Av1Constants.MaxTileRowCount) + { + throw new InvalidImageContentException("The AV1 frame exceeds the maximum tile-row count."); + } + + tileInfo.TileRowStartModeInfo[i] = startSuperBlock << superblockShift; + uint maxHeight = (uint)Math.Min(superblockRowCount - startSuperBlock, tileInfo.MaxTileHeightSuperblock); + uint heightInSuperBlocks = reader.ReadNonSymmetric(maxHeight) + 1; + startSuperBlock += (int)heightInSuperBlocks; + } + + if (startSuperBlock != superblockRowCount) + { + throw new ImageFormatException("Super block tiles height does not add up to total height."); + } + + tileInfo.TileRowStartModeInfo[i] = frameHeader.ModeInfoRowCount; + tileInfo.TileRowCount = i; + tileInfo.TileRowCountLog2 = TileLog2(1, tileInfo.TileRowCount); + } + + if (tileInfo.TileColumnCount > Av1Constants.MaxTileColumnCount || tileInfo.TileRowCount > Av1Constants.MaxTileRowCount) + { + throw new ImageFormatException("Tile width or height too big."); + } + + if (tileInfo.TileColumnCount > 1) + { + int minimumInnerTileWidth = 64 << (frameHeader.FrameSize.FrameWidth != frameHeader.FrameSize.SuperResolutionUpscaledWidth ? 1 : 0); + for (int column = 0; column < tileInfo.TileColumnCount - 1; column++) + { + int tileWidth = (tileInfo.TileColumnStartModeInfo[column + 1] - tileInfo.TileColumnStartModeInfo[column]) + << Av1Constants.ModeInfoSizeLog2; + + // The reference decoder excludes the rightmost column from this conformance check because it receives the + // remainder of the coded width. Every inner column must be at least 64 pixels, doubled when the frame + // is super-resolution scaled. + if (tileWidth < minimumInnerTileWidth) + { + throw new InvalidImageContentException("The AV1 frame contains an inner tile column narrower than the permitted minimum."); + } + } + } + + if (tileInfo.TileColumnCountLog2 > 0 || tileInfo.TileRowCountLog2 > 0) + { + tileInfo.ContextUpdateTileId = reader.ReadLiteral(tileInfo.TileRowCountLog2 + tileInfo.TileColumnCountLog2); + tileInfo.TileSizeBytes = (int)reader.ReadLiteral(2) + 1; + } + else + { + tileInfo.ContextUpdateTileId = 0; + } + + if (tileInfo.ContextUpdateTileId >= (tileInfo.TileColumnCount * tileInfo.TileRowCount)) + { + throw new ImageFormatException("Context update Tile ID too large."); + } + + return tileInfo; + } + + /// + /// Reads the uncompressed syntax for one coded frame in a bounded AV1 image item or image sequence. + /// + /// The reader positioned at the uncompressed frame header. + /// The OBU header identifying the frame's temporal and spatial layers. + private void ReadUncompressedFrameHeader(ref Av1BitStreamReader reader, ObuHeader header) + { + ObuSequenceHeader sequenceHeader = this.CurrentSequenceHeader; + ObuFrameHeader frameHeader = this.CurrentFrameHeader; + Av1ReferenceFrame? primaryReference = null; + bool frameSizeOverrideFlag = false; + int idLength = sequenceHeader.FrameIdLength; + + if (sequenceHeader.IsReducedStillPictureHeader) + { + frameHeader.ShowExistingFrame = false; + frameHeader.FrameType = ObuFrameType.KeyFrame; + frameHeader.ShowFrame = true; + frameHeader.ShowableFrame = false; + frameHeader.ErrorResilientMode = true; + } + else + { + frameHeader.ShowExistingFrame = reader.ReadBoolean(); + if (frameHeader.ShowExistingFrame) + { + if (sequenceHeader.IsStillPicture) + { + throw new InvalidImageContentException("An AV1 still picture cannot display a previously decoded frame."); + } + + frameHeader.FrameToShowMapIdx = reader.ReadLiteral(Av1Constants.ReferenceFrameIndexBits); + + if (sequenceHeader.DecoderModelInfoPresentFlag && sequenceHeader.TimingInfo?.EqualPictureInterval == false) + { + // 5.9.31. Temporal point info syntax. + ObuDecoderModelInfo decoderModelInfo = sequenceHeader.GetDecoderModelInfo(); + frameHeader.FramePresentationTime = reader.ReadLiteral((int)decoderModelInfo.FramePresentationTimeLength); + } + + if (sequenceHeader.IsFrameIdNumbersPresent) + { + frameHeader.DisplayFrameId = reader.ReadLiteral(idLength); + } + + Av1ReferenceFrameStore? retainedReferenceFrames = this.referenceFrames; + if (retainedReferenceFrames is null) + { + throw new InvalidOperationException("AV1 existing-frame presentation requires a reconstructed reference map."); + } + + int existingFrameSlot = (int)frameHeader.FrameToShowMapIdx; + Av1ReferenceFrame? existingFrame = retainedReferenceFrames.Resolve(existingFrameSlot); + if (existingFrame is null) + { + throw new InvalidImageContentException("The AV1 existing-frame header selects an unoccupied reference-map slot."); + } + + if (!existingFrame.FrameHeader.ShowableFrame) + { + throw new InvalidImageContentException("The AV1 existing-frame header selects a frame that is not showable."); + } + + if (sequenceHeader.IsFrameIdNumbersPresent && + (!frameHeader.GetReferenceValidity()[existingFrameSlot] || + frameHeader.DisplayFrameId != frameHeader.GetReferenceFrameIds()[existingFrameSlot])) + { + throw new InvalidImageContentException("The AV1 existing-frame header has a mismatched display frame identifier."); + } + + ObuFrameHeader existingFrameHeader = existingFrame.FrameHeader; + frameHeader.FrameType = existingFrameHeader.FrameType; + frameHeader.ShowFrame = true; + frameHeader.ShowableFrame = existingFrameHeader.ShowableFrame; + frameHeader.OrderHint = existingFrameHeader.OrderHint; + frameHeader.FrameSize = existingFrameHeader.FrameSize; + frameHeader.FilmGrainParameters = existingFrameHeader.FilmGrainParameters; + + if (existingFrameHeader.FrameType == ObuFrameType.KeyFrame) + { + frameHeader.RefreshFrameFlags = byte.MaxValue; + frameHeader.CurrentFrameId = frameHeader.GetReferenceFrameIds()[existingFrameSlot]; + } + else + { + frameHeader.RefreshFrameFlags = 0; + if (this.frameReferenceState.HasCurrentFrameId) + { + // Non-key existing-frame presentation does not consume or replace decoder current_frame_id. + frameHeader.CurrentFrameId = this.frameReferenceState.CurrentFrameId; + } + } + + return; + } + + frameHeader.FrameType = (ObuFrameType)reader.ReadLiteral(Av1Constants.FrameTypeBits); + frameHeader.ShowFrame = reader.ReadBoolean(); + if (sequenceHeader.IsStillPicture && (frameHeader.FrameType != ObuFrameType.KeyFrame || !frameHeader.ShowFrame)) + { + throw new InvalidImageContentException("An AV1 still picture must be encoded as a shown key frame."); + } + + if (frameHeader.ShowFrame && sequenceHeader.DecoderModelInfoPresentFlag && sequenceHeader.TimingInfo?.EqualPictureInterval == false) + { + // 5.9.31. Temporal point info syntax. + ObuDecoderModelInfo decoderModelInfo = sequenceHeader.GetDecoderModelInfo(); + frameHeader.FramePresentationTime = reader.ReadLiteral((int)decoderModelInfo.FramePresentationTimeLength); + } + + if (frameHeader.ShowFrame) + { + frameHeader.ShowableFrame = frameHeader.FrameType != ObuFrameType.KeyFrame; + } + else + { + frameHeader.ShowableFrame = reader.ReadBoolean(); + } + + if (frameHeader.FrameType == ObuFrameType.SwitchFrame || (frameHeader.FrameType == ObuFrameType.KeyFrame && frameHeader.ShowFrame)) + { + frameHeader.ErrorResilientMode = true; + } + else + { + frameHeader.ErrorResilientMode = reader.ReadBoolean(); + } + } + + if (frameHeader.FrameType == ObuFrameType.KeyFrame && frameHeader.ShowFrame) + { + frameHeader.GetReferenceValidity().Clear(); + } + + frameHeader.DisableCdfUpdate = reader.ReadBoolean(); + if (sequenceHeader.ForceScreenContentTools == 2) + { + frameHeader.AllowScreenContentTools = reader.ReadBoolean(); + } + else + { + frameHeader.AllowScreenContentTools = sequenceHeader.ForceScreenContentTools != 0; + } + + if (frameHeader.AllowScreenContentTools) + { + if (sequenceHeader.ForceIntegerMotionVector == 2) + { + frameHeader.ForceIntegerMotionVector = reader.ReadBoolean(); + } + else + { + frameHeader.ForceIntegerMotionVector = sequenceHeader.ForceIntegerMotionVector != 0; + } + } + else + { + frameHeader.ForceIntegerMotionVector = false; + } + + if (frameHeader.IsIntra) + { + frameHeader.ForceIntegerMotionVector = true; + } + + bool havePreviousFrameId = this.frameReferenceState.HasCurrentFrameId && + !(frameHeader.FrameType == ObuFrameType.KeyFrame && frameHeader.ShowFrame); + + uint previousFrameId = this.frameReferenceState.CurrentFrameId; + + if (sequenceHeader.IsFrameIdNumbersPresent) + { + frameHeader.CurrentFrameId = reader.ReadLiteral(idLength); + if (havePreviousFrameId) + { + uint frameIdModulus = 1U << idLength; + uint diffFrameId = frameHeader.CurrentFrameId > previousFrameId + ? frameHeader.CurrentFrameId - previousFrameId + : frameIdModulus + frameHeader.CurrentFrameId - previousFrameId; + + if (frameHeader.CurrentFrameId == previousFrameId || diffFrameId >= 1U << (idLength - 1)) + { + throw new ImageFormatException("Current frame ID cannot be same as previous Frame ID"); + } + } + + frameHeader.MarkReferenceFrames(idLength, sequenceHeader.DeltaFrameIdLength); + } + else + { + frameHeader.CurrentFrameId = 0; + } + + if (frameHeader.FrameType == ObuFrameType.SwitchFrame) + { + frameSizeOverrideFlag = true; + } + else if (sequenceHeader.IsReducedStillPictureHeader) + { + frameSizeOverrideFlag = false; + } + else + { + frameSizeOverrideFlag = reader.ReadBoolean(); + } + + frameHeader.OrderHint = reader.ReadLiteral(sequenceHeader.OrderHintInfo.OrderHintBits); + + if (frameHeader.IsIntra || frameHeader.ErrorResilientMode) + { + frameHeader.PrimaryReferenceFrame = Av1Constants.PrimaryReferenceFrameNone; + } + else + { + frameHeader.PrimaryReferenceFrame = reader.ReadLiteral(Av1Constants.PrimaryReferenceBits); + } + + if (sequenceHeader.DecoderModelInfoPresentFlag) + { + bool bufferRemovalTimePresent = reader.ReadBoolean(); + if (bufferRemovalTimePresent) + { + ObuDecoderModelInfo decoderModelInfo = sequenceHeader.GetDecoderModelInfo(); + int bufferRemovalTimeLength = (int)decoderModelInfo.BufferRemovalTimeLength; + foreach (ObuOperatingPoint operatingPoint in sequenceHeader.OperatingPoint) + { + // A layer-specific OBU carries one removal time only for operating points which select both + // of its layer IDs; the value affects scheduling, so consume it without retaining video state. + bool appliesToLayer = operatingPoint.Idc == 0 || + (((operatingPoint.Idc >> header.TemporalId) & 1U) != 0 && + ((operatingPoint.Idc >> (header.SpatialId + 8)) & 1U) != 0); + + if (operatingPoint.IsDecoderModelInfoPresent && appliesToLayer) + { + _ = reader.ReadLiteral(bufferRemovalTimeLength); + } + } + } + } + + frameHeader.AllowHighPrecisionMotionVector = false; + frameHeader.UseReferenceFrameMotionVectors = false; + frameHeader.AllowIntraBlockCopy = false; + if (frameHeader.FrameType == ObuFrameType.SwitchFrame || (frameHeader.FrameType == ObuFrameType.KeyFrame && frameHeader.ShowFrame)) + { + frameHeader.RefreshFrameFlags = 0xFFU; + } + else + { + frameHeader.RefreshFrameFlags = reader.ReadLiteral(8); + } + + if (frameHeader.FrameType == ObuFrameType.IntraOnlyFrame) + { + if (frameHeader.RefreshFrameFlags == byte.MaxValue) + { + throw new InvalidImageContentException("An AV1 intra-only frame cannot refresh every reference-map slot."); + } + } + + if (!frameHeader.IsIntra || (frameHeader.RefreshFrameFlags != 0xFFU)) + { + if (frameHeader.ErrorResilientMode && sequenceHeader.OrderHintInfo.EnableOrderHint) + { + Span referenceOrderHints = frameHeader.GetReferenceOrderHints(); + Span referenceValidity = frameHeader.GetReferenceValidity(); + for (int i = 0; i < Av1Constants.ReferenceFrameCount; i++) + { + uint referenceOrderHint = reader.ReadLiteral(sequenceHeader.OrderHintInfo.OrderHintBits); + if (referenceOrderHint != referenceOrderHints[i]) + { + referenceValidity[i] = false; + } + } + } + } + + if (frameHeader.IsIntra) + { + this.ReadFrameSize(ref reader, frameSizeOverrideFlag); + this.ReadRenderSize(ref reader); + if (frameHeader.AllowScreenContentTools && frameHeader.FrameSize.RenderWidth != 0) + { + if (frameHeader.FrameSize.FrameWidth == frameHeader.FrameSize.SuperResolutionUpscaledWidth) + { + frameHeader.AllowIntraBlockCopy = reader.ReadBoolean(); + } + } + } + else + { + Av1ReferenceFrameStore? retainedReferenceFrames = this.referenceFrames; + + if (retainedReferenceFrames is null) + { + // Inter-frame size syntax reads dimensions from reconstructed references. Header-only parser users do + // not own those samples, while the production decoder establishes this dependency in its constructor. + throw new InvalidOperationException("AV1 inter-frame parsing requires a reconstructed reference map."); + } + + ReadReferenceFrameIndices(ref reader, sequenceHeader, frameHeader, retainedReferenceFrames); + + if (frameHeader.PrimaryReferenceSlot.HasValue) + { + // Reference-index parsing validates the resolved slot before publishing it on the header. Retaining + // the owner here keeps every inherited frame state tied to the same normative primary reference. + primaryReference = retainedReferenceFrames.ResolveRequired(frameHeader.PrimaryReferenceSlot.Value); + } + + if (!frameHeader.ErrorResilientMode && frameSizeOverrideFlag) + { + this.ReadFrameSizeWithReferences(ref reader, retainedReferenceFrames); + } + else + { + this.ReadFrameSize(ref reader, frameSizeOverrideFlag); + this.ReadRenderSize(ref reader); + } + + if (!frameHeader.ForceIntegerMotionVector) + { + frameHeader.AllowHighPrecisionMotionVector = reader.ReadBoolean(); + } + + frameHeader.InterpolationFilter = ReadFrameInterpolationFilter(ref reader); + frameHeader.IsMotionModeSwitchable = reader.ReadBoolean(); + } + + bool mightAllowReferenceFrameMotionVectors = + !frameHeader.ErrorResilientMode && + sequenceHeader.OrderHintInfo.EnableReferenceFrameMotionVectors && + sequenceHeader.OrderHintInfo.EnableOrderHint && + !frameHeader.IsIntra; + + if (mightAllowReferenceFrameMotionVectors) + { + // AV1 5.9.2 carries this flag only when temporal order hints and the sequence-level reference-MV tool are + // both available. All other frame classes derive false without consuming a bit. + frameHeader.UseReferenceFrameMotionVectors = reader.ReadBoolean(); + } + + // SetupFrameBufferReferences(sequenceHeader, frameHeader); + // CheckAddTemporalMotionVectorBuffer(sequenceHeader, frameHeader); + + // Sign bias is derived from retained reference order hints when frame motion state is initialized. It is not + // mutable uncompressed-header state and therefore is not duplicated here. + if (sequenceHeader.IsReducedStillPictureHeader || frameHeader.DisableCdfUpdate) + { + frameHeader.DisableFrameEndUpdateCdf = true; + } + else + { + frameHeader.DisableFrameEndUpdateCdf = reader.ReadBoolean(); + } + + if (primaryReference is not null) + { + // When update flags omit new values, loop-filter deltas inherit from the primary frame. Copying the two + // fixed tables before parsing lets the existing header object retain unchanged entries without aliases. + primaryReference.FrameHeader.LoopFilterParameters.ReferenceDeltas.CopyTo(frameHeader.LoopFilterParameters.ReferenceDeltas); + primaryReference.FrameHeader.LoopFilterParameters.ModeDeltas.CopyTo(frameHeader.LoopFilterParameters.ModeDeltas); + } + + // Entropy defaults depend on base_q_idx, which follows tile information in the header. Av1TileReader therefore + // loads either the retained primary snapshot or the selected quantizer-band defaults at the first tile boundary. + + // Reference-map refresh remains transactional until reconstruction completes; parsing only records the + // validated refresh flags and selected slots on the frame header. + frameHeader.TilesInfo = ReadTileInfo(ref reader, sequenceHeader, frameHeader); + ReadQuantizationParameters(ref reader, sequenceHeader, frameHeader); + ReadSegmentationParameters(ref reader, frameHeader, primaryReference?.FrameHeader.SegmentationParameters); + ReadFrameDeltaQParameters(ref reader, frameHeader); + ReadFrameDeltaLoopFilterParameters(ref reader, frameHeader); + + // SetupSegmentationDequantization(); + // The primary frame retains its decoded segment map in Av1FrameInfo. Inter block parsing copies or predicts + // segment identifiers from that map according to update_map instead of duplicating it in the frame header. + Av1QuantizationLookup.UpdateFrameQuantizationState(frameHeader); + + if (frameHeader.CodedLossless) + { + DebugGuard.IsFalse(frameHeader.DeltaQParameters.IsPresent, nameof(frameHeader.DeltaQParameters.IsPresent), "No Delta Q parameters are allowed for lossless frame."); + } + + this.ReadLoopFilterParameters(ref reader, sequenceHeader); + ReadCdefParameters(ref reader, sequenceHeader, frameHeader); + ReadLoopRestorationParameters(ref reader, sequenceHeader, frameHeader); + ReadTransformMode(ref reader, frameHeader); + + frameHeader.ReferenceMode = ReadFrameReferenceMode(ref reader, frameHeader); + ReadSkipModeParameters(ref reader, sequenceHeader, frameHeader); + if (frameHeader.IsIntra || frameHeader.ErrorResilientMode || !sequenceHeader.EnableWarpedMotion) + { + frameHeader.AllowWarpedMotion = false; + } + else + { + frameHeader.AllowWarpedMotion = reader.ReadBoolean(); + } + + frameHeader.UseReducedTransformSet = reader.ReadBoolean(); + this.ReadGlobalMotionParameters(ref reader, frameHeader); + this.ReadFilmGrainFilterParameters(ref reader, sequenceHeader, frameHeader); + } + + /// + /// Reads or derives the seven reference-map slots used by an inter frame and resolves its primary context source. + /// + /// The reader positioned at the inter-reference signaling syntax. + /// The sequence header defining frame-ID and order-hint domains. + /// The frame header that receives the seven-entry reference mapping. + /// The retained reconstructed frames backing the eight reference-map slots. + private static void ReadReferenceFrameIndices( + ref Av1BitStreamReader reader, + ObuSequenceHeader sequenceHeader, + ObuFrameHeader frameHeader, + Av1ReferenceFrameStore referenceFrames) + { + Span referenceFrameIndices = frameHeader.GetReferenceFrameIndices(); + Span referenceValidity = frameHeader.GetReferenceValidity(); + bool usesShortSignaling = sequenceHeader.OrderHintInfo.EnableOrderHint && reader.ReadBoolean(); + + if (usesShortSignaling) + { + uint lastFrameIndex = reader.ReadLiteral(Av1Constants.ReferenceFrameIndexBits); + uint goldenFrameIndex = reader.ReadLiteral(Av1Constants.ReferenceFrameIndexBits); + InlineArray8 slotOccupancyStorage = default; + Span slotOccupancy = slotOccupancyStorage; + + referenceFrames.FillOccupancy(slotOccupancy); + + // Short signaling transmits only LAST and GOLDEN. The remaining five roles are a normative derivation from + // the persisted slot order hints and physical slot occupancy, not frame-ID validity or a decoder heuristic. + Av1ReferenceFrameDerivation.DeriveShortSignaledReferences( + frameHeader.OrderHint, + sequenceHeader.OrderHintInfo.OrderHintBits, + lastFrameIndex, + goldenFrameIndex, + frameHeader.GetReferenceOrderHints(), + slotOccupancy, + referenceFrameIndices); + } + + Span referenceFrameIds = frameHeader.GetReferenceFrameIds(); + uint frameIdModulus = 1U << sequenceHeader.FrameIdLength; + + for (int reference = 0; reference < Av1Constants.ReferencesPerFrame; reference++) + { + uint slot = referenceFrameIndices[reference]; + if (!usesShortSignaling) + { + slot = reader.ReadLiteral(Av1Constants.ReferenceFrameIndexBits); + referenceFrameIndices[reference] = slot; + } + + // Slot occupancy and frame-ID validity are independent normative states. Short signaling derives roles + // from every occupied slot before this per-role validity check, matching av1_set_frame_refs followed by + // the reference decoder's valid_for_referencing check. + if (referenceFrames.Resolve((int)slot) is null) + { + throw new InvalidImageContentException("An AV1 inter frame selects an unoccupied reference-map slot."); + } + + if (!referenceValidity[(int)slot]) + { + throw new InvalidImageContentException("An AV1 inter frame selects a reference that is not valid for referencing."); + } + + if (sequenceHeader.IsFrameIdNumbersPresent) + { + uint deltaFrameId = reader.ReadLiteral(sequenceHeader.DeltaFrameIdLength) + 1U; + uint expectedFrameId = (frameHeader.CurrentFrameId + frameIdModulus - deltaFrameId) % frameIdModulus; + + if (referenceFrameIds[(int)slot] != expectedFrameId) + { + throw new InvalidImageContentException("An AV1 inter reference does not match its signaled frame identifier."); + } + } + } + + if (frameHeader.PrimaryReferenceFrame != Av1Constants.PrimaryReferenceFrameNone) + { + // primary_ref_frame indexes the seven inter-reference roles, not the eight-slot retained map. Resolve it + // once so entropy, segmentation, loop-filter, and motion state all select the same retained owner later. + frameHeader.PrimaryReferenceSlot = (byte)referenceFrameIndices[(int)frameHeader.PrimaryReferenceFrame]; + } + } + + /// + /// Reads an AV1 frame header and removes its byte length from the remaining OBU payload size. + /// + /// The reader positioned at the frame-header payload. + /// The OBU header whose remaining payload size is updated. + /// A value indicating whether trailing-bit syntax follows the frame header. + public void ReadFrameHeader(ref Av1BitStreamReader reader, ObuHeader header, bool trailingBit) + { + int startBitPosition = reader.BitPosition; + this.ReadUncompressedFrameHeader(ref reader, header); + if (trailingBit) + { + ReadTrailingBits(ref reader); + } + + AlignToByteBoundary(ref reader); + + int endPosition = reader.BitPosition; + int headerBytes = (endPosition - startBitPosition) / 8; + header.PayloadSize -= headerBytes; + } + + /// + /// Reads a tile-group header and passes each contained tile payload to the tile reader. + /// + /// The reader positioned at the tile-group payload. + /// The tile reader that decodes each tile payload. + /// The OBU header containing the remaining tile-group payload size. + /// The zero-based tile index that must begin this group and receives the next expected index. + /// Receives whether this group completes the frame's ordered tile coverage. + private void ReadTileGroup( + ref Av1BitStreamReader reader, + IAv1TileReader decoder, + ObuHeader header, + ref int nextTileStart, + out bool isLastTileGroup) + { + ObuSequenceHeader sequenceHeader = this.CurrentSequenceHeader; + ObuFrameHeader frameHeader = this.CurrentFrameHeader; + ObuTileGroupHeader tileInfo = frameHeader.TilesInfo; + int tileCount = tileInfo.TileColumnCount * tileInfo.TileRowCount; + int startBitPosition = reader.BitPosition; + bool tileStartAndEndPresentFlag = false; + if (tileCount > 1) + { + tileStartAndEndPresentFlag = reader.ReadBoolean(); + } + + if (header.Type == ObuType.Frame && tileStartAndEndPresentFlag) + { + throw new InvalidImageContentException("A combined AV1 frame OBU cannot signal explicit tile-group bounds."); + } + + int tileGroupStart = 0; + int tileGroupEnd = tileCount - 1; + if (tileCount != 1 && tileStartAndEndPresentFlag) + { + int tileBits = tileInfo.TileColumnCountLog2 + tileInfo.TileRowCountLog2; + tileGroupStart = (int)reader.ReadLiteral(tileBits); + tileGroupEnd = (int)reader.ReadLiteral(tileBits); + } + + if (tileGroupStart != nextTileStart || tileGroupStart > tileGroupEnd || tileGroupEnd >= tileCount) + { + throw new InvalidImageContentException("The AV1 tile groups do not provide complete ordered frame coverage."); + } + + AlignToByteBoundary(ref reader); + int endBitPosition = reader.BitPosition; + int headerBytes = (endBitPosition - startBitPosition) / 8; + header.PayloadSize -= headerBytes; + + bool noIbc = !frameHeader.AllowIntraBlockCopy; + bool doLoopFilter = noIbc && (frameHeader.LoopFilterParameters.FilterLevel[0] != 0 || frameHeader.LoopFilterParameters.FilterLevel[1] != 0); + bool doCdef = noIbc && (!frameHeader.CodedLossless && + (frameHeader.CdefParameters.BitCount != 0 || + frameHeader.CdefParameters.YStrength[0] != 0 || + frameHeader.CdefParameters.UvStrength[0] != 0)); + bool doLoopRestoration = noIbc && + (frameHeader.LoopRestorationParameters.Items[(int)Av1Plane.Y].Type != ObuRestorationType.None || + frameHeader.LoopRestorationParameters.Items[(int)Av1Plane.U].Type != ObuRestorationType.None || + frameHeader.LoopRestorationParameters.Items[(int)Av1Plane.V].Type != ObuRestorationType.None); + + // All tile sizes except the final size are explicitly stored as size-minus-one. The + // last tile consumes the bytes that remain in the OBU payload. + for (int tileNum = tileGroupStart; tileNum <= tileGroupEnd; tileNum++) + { + bool isLastTile = tileNum == tileGroupEnd; + int tileDataSize; + if (!isLastTile) + { + if (header.PayloadSize <= tileInfo.TileSizeBytes) + { + throw new InvalidImageContentException("The AV1 tile group ends before its declared tile-size field and payload."); + } + + uint tileDataSizeMinusOne = reader.ReadLittleEndian(tileInfo.TileSizeBytes); + header.PayloadSize -= tileInfo.TileSizeBytes; + + // Compare in the encoded unsigned domain before adding one. A four-byte 0xFFFFFFFF field would + // otherwise wrap to a zero-length signed tile and shift the following tile boundary. + if (tileDataSizeMinusOne >= (uint)header.PayloadSize) + { + throw new InvalidImageContentException("The AV1 tile size exceeds the remaining tile-group payload."); + } + + tileDataSize = (int)tileDataSizeMinusOne + 1; + header.PayloadSize -= tileDataSize; + } + else + { + tileDataSize = header.PayloadSize; + header.PayloadSize = 0; + if (tileDataSize <= 0) + { + throw new InvalidImageContentException("The AV1 tile group contains an empty tile payload."); + } + } + + Span tileData = reader.GetSymbolReader(tileDataSize); + decoder.ReadTile(tileData, tileNum); + } + + nextTileStart = tileGroupEnd + 1; + isLastTileGroup = nextTileStart == tileCount; + + if (tileGroupEnd != tileCount - 1) + { + return; + } + } + + /// + /// Reads an optional signed quantizer-index delta. + /// + /// The reader positioned at a delta-quantizer field. + /// The decoded delta, or zero when the field is absent. + private static int ReadDeltaQ(ref Av1BitStreamReader reader) + { + int deltaQ = 0; + if (reader.ReadBoolean()) + { + deltaQ = reader.ReadSignedFromUnsigned(7); + } + + return deltaQ; + } + + /// + /// Reads the frame-level delta-quantizer configuration. + /// + /// The reader positioned at the delta-quantizer parameters. + /// The frame header that receives the parameters. + private static void ReadFrameDeltaQParameters(ref Av1BitStreamReader reader, ObuFrameHeader frameHeader) + { + frameHeader.DeltaQParameters.Resolution = 1; + frameHeader.DeltaQParameters.IsPresent = false; + if (frameHeader.QuantizationParameters.BaseQIndex > 0) + { + frameHeader.DeltaQParameters.IsPresent = reader.ReadBoolean(); + } + + if (frameHeader.DeltaQParameters.IsPresent) + { + frameHeader.DeltaQParameters.Resolution = 1 << (int)reader.ReadLiteral(2); + } + } + + /// + /// Reads the frame-level delta-loop-filter configuration. + /// + /// The reader positioned at the delta-loop-filter parameters. + /// The frame header that receives the parameters. + private static void ReadFrameDeltaLoopFilterParameters(ref Av1BitStreamReader reader, ObuFrameHeader frameHeader) + { + frameHeader.DeltaLoopFilterParameters.IsPresent = false; + frameHeader.DeltaLoopFilterParameters.Resolution = 1; + frameHeader.DeltaLoopFilterParameters.IsMulti = false; + if (frameHeader.DeltaQParameters.IsPresent) + { + if (!frameHeader.AllowIntraBlockCopy) + { + frameHeader.DeltaLoopFilterParameters.IsPresent = reader.ReadBoolean(); + } + + if (frameHeader.DeltaLoopFilterParameters.IsPresent) + { + frameHeader.DeltaLoopFilterParameters.Resolution = 1 << (int)reader.ReadLiteral(2); + frameHeader.DeltaLoopFilterParameters.IsMulti = reader.ReadBoolean(); + } + } + } + + /// + /// Reads the base index, plane deltas, and optional quantization matrices for a frame. + /// + /// The reader positioned at the quantization parameters. + /// The sequence header defining the active color planes. + /// The frame header that receives the quantization parameters. + private static void ReadQuantizationParameters(ref Av1BitStreamReader reader, ObuSequenceHeader sequenceHeader, ObuFrameHeader frameHeader) + { + ObuQuantizationParameters quantParams = frameHeader.QuantizationParameters; + ObuColorConfig colorInfo = sequenceHeader.ColorConfig; + quantParams.BaseQIndex = (int)reader.ReadLiteral(8); + quantParams.DeltaQDc[(int)Av1Plane.Y] = ReadDeltaQ(ref reader); + quantParams.DeltaQAc[(int)Av1Plane.Y] = 0; + if (colorInfo.PlaneCount > 1) + { + quantParams.HasSeparateUvDelta = false; + if (colorInfo.HasSeparateUvDelta) + { + quantParams.HasSeparateUvDelta = reader.ReadBoolean(); + } + + quantParams.DeltaQDc[(int)Av1Plane.U] = ReadDeltaQ(ref reader); + quantParams.DeltaQAc[(int)Av1Plane.U] = ReadDeltaQ(ref reader); + if (quantParams.HasSeparateUvDelta) + { + quantParams.DeltaQDc[(int)Av1Plane.V] = ReadDeltaQ(ref reader); + quantParams.DeltaQAc[(int)Av1Plane.V] = ReadDeltaQ(ref reader); + } + else + { + quantParams.DeltaQDc[(int)Av1Plane.V] = quantParams.DeltaQDc[(int)Av1Plane.U]; + quantParams.DeltaQAc[(int)Av1Plane.V] = quantParams.DeltaQAc[(int)Av1Plane.U]; + } + } + else + { + quantParams.DeltaQDc[(int)Av1Plane.U] = 0; + quantParams.DeltaQAc[(int)Av1Plane.U] = 0; + quantParams.DeltaQDc[(int)Av1Plane.V] = 0; + quantParams.DeltaQAc[(int)Av1Plane.V] = 0; + } + + quantParams.IsUsingQMatrix = reader.ReadBoolean(); + if (quantParams.IsUsingQMatrix) + { + quantParams.QMatrix[(int)Av1Plane.Y] = (int)reader.ReadLiteral(4); + quantParams.QMatrix[(int)Av1Plane.U] = (int)reader.ReadLiteral(4); + if (!colorInfo.HasSeparateUvDelta) + { + quantParams.QMatrix[(int)Av1Plane.V] = quantParams.QMatrix[(int)Av1Plane.U]; + } + else + { + quantParams.QMatrix[(int)Av1Plane.V] = (int)reader.ReadLiteral(4); + } + } + else + { + quantParams.QMatrix[(int)Av1Plane.Y] = 0; + quantParams.QMatrix[(int)Av1Plane.U] = 0; + quantParams.QMatrix[(int)Av1Plane.V] = 0; + } + } + + /// + /// Reads the segmentation map controls and per-segment feature values. + /// + /// The reader positioned at the segmentation parameters. + /// The frame header that receives the segmentation state. + /// + /// The primary-reference feature state, or when the frame has no primary reference. + /// + private static void ReadSegmentationParameters( + ref Av1BitStreamReader reader, + ObuFrameHeader frameHeader, + ObuSegmentationParameters? primaryParameters) + { + frameHeader.SegmentationParameters.Enabled = reader.ReadBoolean(); + + if (frameHeader.SegmentationParameters.Enabled) + { + if (frameHeader.PrimaryReferenceFrame == Av1Constants.PrimaryReferenceFrameNone) + { + frameHeader.SegmentationParameters.SegmentationUpdateMap = 1; + frameHeader.SegmentationParameters.SegmentationTemporalUpdate = 0; + frameHeader.SegmentationParameters.SegmentationUpdateData = 1; + } + else + { + frameHeader.SegmentationParameters.SegmentationUpdateMap = reader.ReadBoolean() ? 1 : 0; + if (frameHeader.SegmentationParameters.SegmentationUpdateMap == 1) + { + frameHeader.SegmentationParameters.SegmentationTemporalUpdate = reader.ReadBoolean() ? 1 : 0; + } + + frameHeader.SegmentationParameters.SegmentationUpdateData = reader.ReadBoolean() ? 1 : 0; + } + + if (frameHeader.SegmentationParameters.SegmentationUpdateData == 1) + { + for (int i = 0; i < Av1Constants.MaxSegmentCount; i++) + { + for (int j = 0; j < Av1Constants.SegmentationLevelMax; j++) + { + int featureValue = 0; + bool featureEnabled = reader.ReadBoolean(); + frameHeader.SegmentationParameters.SetFeatureEnabled(i, j, featureEnabled); + int clippedValue = 0; + if (featureEnabled) + { + int bitsToRead = Av1Constants.SegmentationFeatureBits[j]; + int limit = Av1Constants.SegmentationFeatureMax[j]; + if (Av1Constants.SegmentationFeatureSigned[j] == 1) + { + featureValue = reader.ReadSignedFromUnsigned(1 + bitsToRead); + clippedValue = Av1Math.Clip3(-limit, limit, featureValue); + } + else + { + featureValue = (int)reader.ReadLiteral(bitsToRead); + clippedValue = featureValue; + } + } + + frameHeader.SegmentationParameters.SetFeatureData(i, j, clippedValue); + } + } + } + else + { + // update_data equal to zero preserves the complete feature mask and values from the primary frame. + // The current header owns its arrays, so later reference replacement cannot mutate inherited state. + if (primaryParameters is null) + { + throw new InvalidImageContentException("AV1 segmentation cannot inherit data without a primary reference."); + } + + frameHeader.SegmentationParameters.CopyFeaturesFrom(primaryParameters); + } + } + else + { + for (int i = 0; i < Av1Constants.MaxSegmentCount; i++) + { + for (int j = 0; j < Av1Constants.SegmentationLevelMax; j++) + { + frameHeader.SegmentationParameters.SetFeatureEnabled(i, j, false); + frameHeader.SegmentationParameters.SetFeatureData(i, j, 0); + } + } + } + + frameHeader.SegmentationParameters.SegmentIdPrecedesSkip = false; + frameHeader.SegmentationParameters.LastActiveSegmentId = 0; + for (int i = 0; i < Av1Constants.MaxSegmentCount; i++) + { + for (int j = 0; j < Av1Constants.SegmentationLevelMax; j++) + { + if (frameHeader.SegmentationParameters.IsFeatureActive(i, (ObuSegmentationLevelFeature)j)) + { + frameHeader.SegmentationParameters.LastActiveSegmentId = i; + if (j >= (int)ObuSegmentationLevelFeature.ReferenceFrame) + { + frameHeader.SegmentationParameters.SegmentIdPrecedesSkip = true; + } + } + } + } + } + + /// + /// Reads the deblocking-loop-filter levels and optional reference and mode deltas. + /// + /// The reader positioned at the loop-filter parameters. + /// The sequence header defining the active color planes. + private void ReadLoopFilterParameters(ref Av1BitStreamReader reader, ObuSequenceHeader sequenceHeader) + { + ObuFrameHeader frameHeader = this.CurrentFrameHeader; + if (frameHeader.CodedLossless || frameHeader.AllowIntraBlockCopy) + { + return; + } + + frameHeader.LoopFilterParameters.FilterLevel[0] = (int)reader.ReadLiteral(6); + frameHeader.LoopFilterParameters.FilterLevel[1] = (int)reader.ReadLiteral(6); + + if (sequenceHeader.ColorConfig.PlaneCount > 1) + { + if (frameHeader.LoopFilterParameters.FilterLevel[0] > 0 || frameHeader.LoopFilterParameters.FilterLevel[1] > 0) + { + frameHeader.LoopFilterParameters.FilterLevelU = (int)reader.ReadLiteral(6); + frameHeader.LoopFilterParameters.FilterLevelV = (int)reader.ReadLiteral(6); + } + } + + frameHeader.LoopFilterParameters.SharpnessLevel = (int)reader.ReadLiteral(3); + frameHeader.LoopFilterParameters.ReferenceDeltaModeEnabled = reader.ReadBoolean(); + if (frameHeader.LoopFilterParameters.ReferenceDeltaModeEnabled) + { + frameHeader.LoopFilterParameters.ReferenceDeltaModeUpdate = reader.ReadBoolean(); + if (frameHeader.LoopFilterParameters.ReferenceDeltaModeUpdate) + { + for (int i = 0; i < Av1Constants.TotalReferencesPerFrame; i++) + { + if (reader.ReadBoolean()) + { + frameHeader.LoopFilterParameters.ReferenceDeltas[i] = reader.ReadSignedFromUnsigned(7); + } + } + + for (int i = 0; i < 2; i++) + { + if (reader.ReadBoolean()) + { + frameHeader.LoopFilterParameters.ModeDeltas[i] = reader.ReadSignedFromUnsigned(7); + } + } + } + } + } + + /// + /// Reads or derives the transform-size selection mode. + /// + /// The reader positioned at the transform-mode flag. + /// The frame header that receives the transform mode. + private static void ReadTransformMode(ref Av1BitStreamReader reader, ObuFrameHeader frameHeader) + { + if (frameHeader.CodedLossless) + { + frameHeader.TransformMode = Av1TransformMode.Only4x4; + } + else + { + if (reader.ReadBoolean()) + { + frameHeader.TransformMode = Av1TransformMode.Select; + } + else + { + frameHeader.TransformMode = Av1TransformMode.Largest; + } + } + } + + /// + /// Reads the loop-restoration type and restoration-unit size for each plane. + /// + /// The reader positioned at the loop-restoration parameters. + /// The sequence header defining restoration availability and color planes. + /// The frame header that receives the restoration parameters. + private static void ReadLoopRestorationParameters(ref Av1BitStreamReader reader, ObuSequenceHeader sequenceHeader, ObuFrameHeader frameHeader) + { + if (frameHeader.CodedLossless || frameHeader.AllowIntraBlockCopy || !sequenceHeader.EnableRestoration) + { + return; + } + + frameHeader.LoopRestorationParameters.UsesLoopRestoration = false; + frameHeader.LoopRestorationParameters.UsesChromaLoopRestoration = false; + int planesCount = sequenceHeader.ColorConfig.PlaneCount; + for (int i = 0; i < planesCount; i++) + { + // The AV1 frame syntax orders its two restoration bits as none, switchable, + // Wiener, and self-guided projection, matching ObuRestorationType values. + frameHeader.LoopRestorationParameters.Items[i].Type = (ObuRestorationType)reader.ReadLiteral(2); + + if (frameHeader.LoopRestorationParameters.Items[i].Type != ObuRestorationType.None) + { + frameHeader.LoopRestorationParameters.UsesLoopRestoration = true; + if (i > 0) + { + frameHeader.LoopRestorationParameters.UsesChromaLoopRestoration = true; + } + } + } + + if (frameHeader.LoopRestorationParameters.UsesLoopRestoration) + { + frameHeader.LoopRestorationParameters.UnitShift = (int)reader.ReadLiteral(1); + if (sequenceHeader.Use128x128Superblock) + { + frameHeader.LoopRestorationParameters.UnitShift++; + } + else if (frameHeader.LoopRestorationParameters.UnitShift != 0) + { + // A 64x64-superblock frame signals the extra size bit only after selecting a + // restoration unit larger than 64 samples with the first size bit. + frameHeader.LoopRestorationParameters.UnitShift += (int)reader.ReadLiteral(1); + } + + frameHeader.LoopRestorationParameters.Items[0].Size = Av1Constants.RestorationMaxTileSize >> (2 - frameHeader.LoopRestorationParameters.UnitShift); + frameHeader.LoopRestorationParameters.UVShift = 0; + if (sequenceHeader.ColorConfig.SubSamplingX && sequenceHeader.ColorConfig.SubSamplingY && frameHeader.LoopRestorationParameters.UsesChromaLoopRestoration) + { + frameHeader.LoopRestorationParameters.UVShift = (int)reader.ReadLiteral(1); + } + + frameHeader.LoopRestorationParameters.Items[1].Size = frameHeader.LoopRestorationParameters.Items[0].Size >> frameHeader.LoopRestorationParameters.UVShift; + frameHeader.LoopRestorationParameters.Items[2].Size = frameHeader.LoopRestorationParameters.Items[0].Size >> frameHeader.LoopRestorationParameters.UVShift; + } + } + + /// + /// Reads constrained directional enhancement filter strengths for the active planes. + /// + /// The reader positioned at the CDEF parameters. + /// The sequence header defining CDEF availability and color planes. + /// The frame header that receives the CDEF parameters. + private static void ReadCdefParameters(ref Av1BitStreamReader reader, ObuSequenceHeader sequenceHeader, ObuFrameHeader frameHeader) + { + ObuConstraintDirectionalEnhancementFilterParameters cdefInfo = frameHeader.CdefParameters; + bool multiPlane = sequenceHeader.ColorConfig.PlaneCount > 1; + if (frameHeader.CodedLossless || frameHeader.AllowIntraBlockCopy || !sequenceHeader.EnableCdef) + { + cdefInfo.BitCount = 0; + cdefInfo.YStrength[0] = 0; + cdefInfo.YStrength[4] = 0; + cdefInfo.UvStrength[0] = 0; + cdefInfo.UvStrength[4] = 0; + cdefInfo.Damping = 0; + return; + } + + cdefInfo.Damping = (int)reader.ReadLiteral(2) + 3; + cdefInfo.BitCount = (int)reader.ReadLiteral(2); + for (int i = 0; i < (1 << frameHeader.CdefParameters.BitCount); i++) + { + cdefInfo.YStrength[i] = (int)reader.ReadLiteral(6); + + if (multiPlane) + { + cdefInfo.UvStrength[i] = (int)reader.ReadLiteral(6); + } + } + } + + /// + /// Reads global-motion parameters when permitted by the frame type. + /// + /// The reader positioned at the global-motion parameters. + /// The current frame header. + private void ReadGlobalMotionParameters(ref Av1BitStreamReader reader, ObuFrameHeader frameHeader) + { + Span parameters = frameHeader.GetGlobalMotionParameters(); + parameters.Fill(Av1GlobalMotionParameters.Identity); + + if (frameHeader.IsIntra) + { + return; + } + + ObuFrameHeader? primaryReferenceHeader = null; + byte? primaryReferenceSlot = frameHeader.PrimaryReferenceSlot; + if (primaryReferenceSlot is not null) + { + // primary_ref_frame identifies the preceding frame whose same seven canonical reference roles supply the + // recentering values. Reference-slot validation has already completed before this syntax is reached. + Av1ReferenceFrameStore referenceFrames = this.referenceFrames + ?? throw new InvalidImageContentException("AV1 global motion requires a reconstructed reference map."); + + primaryReferenceHeader = referenceFrames.ResolveRequired(primaryReferenceSlot.Value).FrameHeader; + } + + for (int referenceIndex = 0; referenceIndex < Av1Constants.ReferencesPerFrame; referenceIndex++) + { + Av1GlobalMotionParameters referenceParameters = primaryReferenceHeader is null + ? Av1GlobalMotionParameters.Identity + : primaryReferenceHeader.GetGlobalMotionParameters()[referenceIndex]; + + ReadGlobalMotionModel( + ref reader, + ref parameters[referenceIndex], + referenceParameters, + frameHeader.AllowHighPrecisionMotionVector); + } + } + + /// + /// Reads one global-motion model relative to the corresponding model retained by the primary reference frame. + /// + /// The reader positioned at the model type and parameter syntax. + /// The destination global-motion model. + /// The same-role model retained by the primary reference frame. + /// + /// A value indicating whether translation-only parameters retain their high-precision bit. + /// + private static void ReadGlobalMotionModel( + ref Av1BitStreamReader reader, + ref Av1GlobalMotionParameters parameters, + Av1GlobalMotionParameters referenceParameters, + bool allowHighPrecisionMotionVector) + { + Av1GlobalMotionType type = Av1GlobalMotionType.Identity; + if (reader.ReadBoolean()) + { + if (reader.ReadBoolean()) + { + type = Av1GlobalMotionType.RotationZoom; + } + else + { + type = reader.ReadBoolean() ? Av1GlobalMotionType.Translation : Av1GlobalMotionType.Affine; + } + } + + parameters = Av1GlobalMotionParameters.Identity; + parameters.Type = type; + if (type >= Av1GlobalMotionType.RotationZoom) + { + // Diagonal terms are coded as a delta from the identity scale, whereas off-diagonal terms are centered + // directly around zero. Both are restored to the common sixteen-bit matrix precision after decoding. + int referenceHorizontalScale = + (referenceParameters[2] >> Av1GlobalMotionParameters.AlphaPrecisionDifference) - + (1 << Av1GlobalMotionParameters.AlphaPrecisionBits); + + parameters[2] = + (reader.ReadSignedReferenceSubexponential( + Av1GlobalMotionParameters.AlphaValueMagnitude, + Av1GlobalMotionParameters.SubexponentialGroupBitCount, + referenceHorizontalScale) * + Av1GlobalMotionParameters.AlphaDecodeFactor) + + Av1GlobalMotionParameters.ModelScale; + + parameters[3] = reader.ReadSignedReferenceSubexponential( + Av1GlobalMotionParameters.AlphaValueMagnitude, + Av1GlobalMotionParameters.SubexponentialGroupBitCount, + referenceParameters[3] >> Av1GlobalMotionParameters.AlphaPrecisionDifference) * + Av1GlobalMotionParameters.AlphaDecodeFactor; + } + + if (type >= Av1GlobalMotionType.Affine) + { + int referenceVerticalScale = + (referenceParameters[5] >> Av1GlobalMotionParameters.AlphaPrecisionDifference) - + (1 << Av1GlobalMotionParameters.AlphaPrecisionBits); + + parameters[4] = reader.ReadSignedReferenceSubexponential( + Av1GlobalMotionParameters.AlphaValueMagnitude, + Av1GlobalMotionParameters.SubexponentialGroupBitCount, + referenceParameters[4] >> Av1GlobalMotionParameters.AlphaPrecisionDifference) * + Av1GlobalMotionParameters.AlphaDecodeFactor; + + parameters[5] = + (reader.ReadSignedReferenceSubexponential( + Av1GlobalMotionParameters.AlphaValueMagnitude, + Av1GlobalMotionParameters.SubexponentialGroupBitCount, + referenceVerticalScale) * + Av1GlobalMotionParameters.AlphaDecodeFactor) + + Av1GlobalMotionParameters.ModelScale; + } + else + { + // Rotation-zoom constrains the second matrix row to the perpendicular vector of the first row. Identity + // and translation models retain the same derived identity coefficients. + parameters[4] = -parameters[3]; + parameters[5] = parameters[2]; + } + + if (type >= Av1GlobalMotionType.Translation) + { + // Translation-only models use a wider coordinate domain than affine models. When high-precision motion is + // disabled, AV1 removes one coded bit and adds one reconstruction shift so the physical displacement grid + // remains in quarter-sample units. Affine translation retains the fixed model-to-translation precision gap. + int precisionAdjustment = type == Av1GlobalMotionType.Translation && !allowHighPrecisionMotionVector ? 1 : 0; + int translationBits = type == Av1GlobalMotionType.Translation + ? Av1GlobalMotionParameters.AbsoluteTranslationOnlyBits - precisionAdjustment + : Av1GlobalMotionParameters.AbsoluteTranslationBits; + + int translationPrecisionDifference = type == Av1GlobalMotionType.Translation + ? Av1GlobalMotionParameters.ModelPrecisionBits - + Av1GlobalMotionParameters.TranslationOnlyPrecisionBits + + precisionAdjustment + : Av1GlobalMotionParameters.ModelPrecisionBits - + Av1GlobalMotionParameters.TranslationPrecisionBits; + + int translationDecodeFactor = 1 << translationPrecisionDifference; + int translationValueMagnitude = (1 << translationBits) + 1; + parameters[0] = reader.ReadSignedReferenceSubexponential( + translationValueMagnitude, + Av1GlobalMotionParameters.SubexponentialGroupBitCount, + referenceParameters[0] >> translationPrecisionDifference) * translationDecodeFactor; + + parameters[1] = reader.ReadSignedReferenceSubexponential( + translationValueMagnitude, + Av1GlobalMotionParameters.SubexponentialGroupBitCount, + referenceParameters[1] >> translationPrecisionDifference) * translationDecodeFactor; + } + + // Invalid shear does not invalidate the frame header. AV1 retains the decoded model and marks it unavailable + // to warped prediction, which is why validity is stored with the parameters instead of throwing here. + parameters.UpdateShearParameters(); + } + + /// + /// Reads or derives the reference prediction mode. + /// + /// The reader positioned at the reference-mode flag. + /// The current frame header. + /// The frame reference mode. + private static ObuReferenceMode ReadFrameReferenceMode(ref Av1BitStreamReader reader, ObuFrameHeader frameHeader) + { + if (frameHeader.IsIntra) + { + return ObuReferenceMode.SingleReference; + } + + return (ObuReferenceMode)reader.ReadLiteral(1); + } + + /// + /// Reads skip-mode enablement when the frame is eligible to use it. + /// + /// The reader positioned at the skip-mode syntax. + /// The sequence header controlling order hints. + /// The frame header that receives the skip-mode state. + private static void ReadSkipModeParameters(ref Av1BitStreamReader reader, ObuSequenceHeader sequenceHeader, ObuFrameHeader frameHeader) + { + ObuSkipModeParameters parameters = frameHeader.SkipModeParameters; + parameters.Derive(sequenceHeader.OrderHintInfo, frameHeader); + parameters.SkipModeFlag = parameters.SkipModeAllowed && reader.ReadBoolean(); + } + + /// + /// Reads film-grain synthesis parameters for the current displayed frame. + /// + /// The reader positioned at the film-grain parameters. + /// The sequence header defining film-grain availability and color sampling. + /// The current frame header. + private void ReadFilmGrainFilterParameters(ref Av1BitStreamReader reader, ObuSequenceHeader sequenceHeader, ObuFrameHeader frameHeader) + { + ObuFilmGrainParameters grainParams = frameHeader.FilmGrainParameters; + if (!sequenceHeader.AreFilmGrainingParametersPresent || (!frameHeader.ShowFrame && !frameHeader.ShowableFrame)) + { + return; + } + + grainParams.ApplyGrain = reader.ReadBoolean(); + if (!grainParams.ApplyGrain) + { + return; + } + + grainParams.GrainSeed = reader.ReadLiteral(16); + + if (frameHeader.FrameType == ObuFrameType.InterFrame) + { + grainParams.UpdateGrain = reader.ReadBoolean(); + } + else + { + // Only inter frames can inherit parameters from a reference frame. Intra frames always carry a complete + // parameter set when grain is enabled. + grainParams.UpdateGrain = true; + } + + if (!grainParams.UpdateGrain) + { + grainParams.FilmGrainParamsRefIdx = reader.ReadLiteral(3); + Span referenceFrameIndices = frameHeader.GetReferenceFrameIndices(); + bool isSelectedReference = false; + for (int reference = 0; reference < Av1Constants.ReferencesPerFrame; reference++) + { + isSelectedReference |= referenceFrameIndices[reference] == grainParams.FilmGrainParamsRefIdx; + } + + Av1ReferenceFrame? referenceFrame = isSelectedReference + ? this.referenceFrames?.Resolve((int)grainParams.FilmGrainParamsRefIdx) + : null; + + if (referenceFrame is null) + { + throw new InvalidImageContentException("The AV1 film-grain reference does not provide retained parameters."); + } + + uint grainSeed = grainParams.GrainSeed; + uint referenceIndex = grainParams.FilmGrainParamsRefIdx; + + // AV1 inherits the complete parameter set but always uses the new frame's independently signaled seed. + // Fixed inline buffers make this a value copy rather than nine small array allocations. + grainParams.CopyFrom(referenceFrame.FrameHeader.FilmGrainParameters); + grainParams.GrainSeed = grainSeed; + grainParams.FilmGrainParamsRefIdx = referenceIndex; + return; + } + + grainParams.NumYPoints = reader.ReadLiteral(4); + if (grainParams.NumYPoints > 14) + { + throw new InvalidImageContentException("The AV1 film-grain luma scaling function exceeds fourteen points."); + } + + for (int i = 0; i < grainParams.NumYPoints; i++) + { + grainParams.PointYValue[i] = (byte)reader.ReadLiteral(8); + if (i > 0 && grainParams.PointYValue[i] <= grainParams.PointYValue[i - 1]) + { + throw new InvalidImageContentException("The AV1 film-grain luma scaling coordinates are not strictly increasing."); + } + + grainParams.PointYScaling[i] = (byte)reader.ReadLiteral(8); + } + + if (sequenceHeader.ColorConfig.IsMonochrome) + { + grainParams.ChromaScalingFromLuma = false; + } + else + { + grainParams.ChromaScalingFromLuma = reader.ReadBoolean(); + } + + if (sequenceHeader.ColorConfig.IsMonochrome || + grainParams.ChromaScalingFromLuma || + (sequenceHeader.ColorConfig.SubSamplingX && sequenceHeader.ColorConfig.SubSamplingY && grainParams.NumYPoints == 0)) + { + grainParams.NumCbPoints = 0; + grainParams.NumCrPoints = 0; + } + else + { + grainParams.NumCbPoints = reader.ReadLiteral(4); + if (grainParams.NumCbPoints > 10) + { + throw new InvalidImageContentException("The AV1 film-grain blue-difference scaling function exceeds ten points."); + } + + for (int i = 0; i < grainParams.NumCbPoints; i++) + { + grainParams.PointCbValue[i] = (byte)reader.ReadLiteral(8); + if (i > 0 && grainParams.PointCbValue[i] <= grainParams.PointCbValue[i - 1]) + { + throw new InvalidImageContentException("The AV1 film-grain blue-difference scaling coordinates are not strictly increasing."); + } + + grainParams.PointCbScaling[i] = (byte)reader.ReadLiteral(8); + } + + grainParams.NumCrPoints = reader.ReadLiteral(4); + if (grainParams.NumCrPoints > 10) + { + throw new InvalidImageContentException("The AV1 film-grain red-difference scaling function exceeds ten points."); + } + + for (int i = 0; i < grainParams.NumCrPoints; i++) + { + grainParams.PointCrValue[i] = (byte)reader.ReadLiteral(8); + if (i > 0 && grainParams.PointCrValue[i] <= grainParams.PointCrValue[i - 1]) + { + throw new InvalidImageContentException("The AV1 film-grain red-difference scaling coordinates are not strictly increasing."); + } + + grainParams.PointCrScaling[i] = (byte)reader.ReadLiteral(8); + } + + if (sequenceHeader.ColorConfig.SubSamplingX && + sequenceHeader.ColorConfig.SubSamplingY && + (grainParams.NumCbPoints == 0) != (grainParams.NumCrPoints == 0)) + { + throw new InvalidImageContentException("AV1 4:2:0 film grain must apply to both chroma planes or neither."); + } + } + + grainParams.GrainScalingMinus8 = reader.ReadLiteral(2); + grainParams.ArCoeffLag = reader.ReadLiteral(2); + uint numPosLuma = 2 * grainParams.ArCoeffLag * (grainParams.ArCoeffLag + 1); + + uint numPosChroma = 0; + if (grainParams.NumYPoints != 0) + { + numPosChroma = numPosLuma + 1; + for (int i = 0; i < numPosLuma; i++) + { + grainParams.ArCoeffsYPlus128[i] = (byte)reader.ReadLiteral(8); + } + } + else + { + numPosChroma = numPosLuma; + } + + if (grainParams.ChromaScalingFromLuma || grainParams.NumCbPoints != 0) + { + for (int i = 0; i < numPosChroma; i++) + { + grainParams.ArCoeffsCbPlus128[i] = (byte)reader.ReadLiteral(8); + } + } + + if (grainParams.ChromaScalingFromLuma || grainParams.NumCrPoints != 0) + { + for (int i = 0; i < numPosChroma; i++) + { + grainParams.ArCoeffsCrPlus128[i] = (byte)reader.ReadLiteral(8); + } + } + + grainParams.ArCoeffShiftMinus6 = reader.ReadLiteral(2); + grainParams.GrainScaleShift = reader.ReadLiteral(2); + if (grainParams.NumCbPoints != 0) + { + grainParams.CbMult = reader.ReadLiteral(8); + grainParams.CbLumaMult = reader.ReadLiteral(8); + grainParams.CbOffset = reader.ReadLiteral(9); + } + + if (grainParams.NumCrPoints != 0) + { + grainParams.CrMult = reader.ReadLiteral(8); + grainParams.CrLumaMult = reader.ReadLiteral(8); + grainParams.CrOffset = reader.ReadLiteral(9); + } + + grainParams.OverlapFlag = reader.ReadBoolean(); + grainParams.ClipToRestrictedRange = reader.ReadBoolean(); + } + + /// + /// Determines whether a sequence-level index is assigned by the AV1 specification. + /// + /// The sequence-level index. + /// for assigned indices; otherwise, . + private static bool IsValidSequenceLevel(int sequenceLevelIndex) + => sequenceLevelIndex is < 24 or 31; + + /// + /// Returns the smallest shift for which shifted left reaches . + /// + /// The initial block count. + /// The minimum shifted value. + /// The required base-2 shift. + public static int TileLog2(int blockSize, int target) + { + int k; + for (k = 0; (blockSize << k) < target; k++) + { + } + + return k; + } + + // The generic providers keep the fixed-reader and factory contracts distinct without allocating + // a closure for fixed-reader payloads or admitting an invalid pair of nullable arguments. + private readonly struct TileReaderFactoryProvider(Func creator) : ITileReaderProvider + { + private readonly Func creator = creator; + + public IAv1TileReader Get() => this.creator(); + } + + private readonly struct FixedTileReaderProvider(IAv1TileReader tileReader) : ITileReaderProvider + { + private readonly IAv1TileReader tileReader = tileReader; + + public IAv1TileReader Get() => this.tileReader; + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuReferenceMode.cs b/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuReferenceMode.cs new file mode 100644 index 0000000000..f61336cccd --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuReferenceMode.cs @@ -0,0 +1,20 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; + +/// +/// Identifies whether inter blocks use only single references or can select compound references. +/// +internal enum ObuReferenceMode +{ + /// + /// Only single-reference prediction is permitted. + /// + SingleReference = 0, + + /// + /// Each eligible block selects single- or compound-reference prediction. + /// + ReferenceModeSelect = 1, +} diff --git a/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuRestorationType.cs b/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuRestorationType.cs new file mode 100644 index 0000000000..ffed1502ef --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuRestorationType.cs @@ -0,0 +1,30 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; + +/// +/// Identifies the loop-restoration mode encoded by the two-bit frame-restoration syntax. +/// +internal enum ObuRestorationType : uint +{ + /// + /// Loop restoration is disabled. + /// + None = 0, + + /// + /// Each restoration unit selects its filter type. + /// + Switchable = 1, + + /// + /// Separable Wiener filtering is used. + /// + Wiener = 2, + + /// + /// Self-guided restoration projection is used. + /// + SgrProj = 3, +} diff --git a/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuSegmentationLevelFeature.cs b/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuSegmentationLevelFeature.cs new file mode 100644 index 0000000000..26175de29d --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuSegmentationLevelFeature.cs @@ -0,0 +1,50 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; + +/// +/// Identifies a feature that AV1 can configure independently for each segment. +/// +internal enum ObuSegmentationLevelFeature +{ + /// + /// Adjusts the segment quantizer index. + /// + AlternativeQuantizer, + + /// + /// Adjusts the vertical luma loop-filter level. + /// + AlternativeLoopFilterYVertical, + + /// + /// Adjusts the horizontal luma loop-filter level. + /// + AlternativeLoopFilterYHorizontal, + + /// + /// Adjusts the U-plane loop-filter level. + /// + AlternativeLoopFilterU, + + /// + /// Adjusts the V-plane loop-filter level. + /// + AlternativeLoopFilterV, + + /// + /// Selects a reference frame for the segment. + /// + ReferenceFrame, + + /// + /// Marks every block in the segment as skipped. + /// + Skip, + + /// + /// Uses the global motion vector for the segment. + /// + GlobalMotionVector, +} diff --git a/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuSegmentationParameters.cs b/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuSegmentationParameters.cs new file mode 100644 index 0000000000..eda7e99e2d --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuSegmentationParameters.cs @@ -0,0 +1,113 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; + +/// +/// Contains the AV1 segmentation state and per-segment feature values for a frame. +/// +internal sealed class ObuSegmentationParameters +{ + /// + /// Stores the feature-enable flags for each segment and feature without per-frame array allocations. + /// + private InlineArray8> featureEnabled; + + /// + /// Stores the feature values for each segment and feature without per-frame array allocations. + /// + private InlineArray8> featureData; + + /// + /// Stores the effective quantization-matrix level for each plane and segment without jagged arrays. + /// + private InlineArray4> qmLevel; + + /// + /// Gets the mutable effective quantization-matrix levels for the Y, U, and V planes. + /// + public Span> QMLevel => this.qmLevel[..Av1Constants.MaxPlanes]; + + /// + /// Gets or sets a value indicating whether segmentation is enabled for the frame. + /// + public bool Enabled { get; set; } + + /// + /// Gets or sets a value indicating whether segment identifiers are decoded before skip-mode decisions. + /// + public bool SegmentIdPrecedesSkip { get; set; } + + /// + /// Gets or sets the greatest segment identifier that has at least one active feature. + /// + public int LastActiveSegmentId { get; set; } + + /// + /// Gets or sets a value indicating whether the segmentation map is updated while decoding the frame. + /// + public int SegmentationUpdateMap { get; set; } + + /// + /// Gets or sets a value indicating whether segmentation-map updates are coded relative to the existing map. + /// + public int SegmentationTemporalUpdate { get; set; } + + /// + /// Gets or sets a value indicating whether the frame supplies new per-segment feature data. + /// + public int SegmentationUpdateData { get; set; } + + /// + /// Determines whether a feature is active for a segment. + /// + /// The segment identifier. + /// The feature to inspect. + /// when the feature is active; otherwise, . + public bool IsFeatureActive(int segmentId, ObuSegmentationLevelFeature feature) + => this.featureEnabled[segmentId][(int)feature]; + + /// + /// Gets a feature value for a segment. + /// + /// The segment identifier. + /// The feature identifier. + /// The stored feature value. + public int GetFeatureData(int segmentId, int featureId) => this.featureData[segmentId][featureId]; + + /// + /// Sets whether a feature is active for a segment. + /// + /// The segment identifier. + /// The feature identifier. + /// Whether the feature is active. + public void SetFeatureEnabled(int segmentId, int featureId, bool enabled) + => this.featureEnabled[segmentId][featureId] = enabled; + + /// + /// Sets a feature value for a segment. + /// + /// The segment identifier. + /// The feature identifier. + /// The feature value. + public void SetFeatureData(int segmentId, int featureId, int value) + => this.featureData[segmentId][featureId] = value; + + /// + /// Replaces every feature enable flag and value with state from a primary reference frame. + /// + /// The primary-reference segmentation state. + public void CopyFeaturesFrom(ObuSegmentationParameters source) + { + // AV1 inherits feature data but not the current frame's enabled or update flags. Both dimensions are fixed by + // the bitstream syntax, and copying values into this header prevents retained frames from sharing mutable state. + for (int segment = 0; segment < Av1Constants.MaxSegmentCount; segment++) + { + for (int feature = 0; feature < Av1Constants.SegmentationLevelMax; feature++) + { + this.featureEnabled[segment][feature] = source.featureEnabled[segment][feature]; + this.featureData[segment][feature] = source.featureData[segment][feature]; + } + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuSequenceHeader.cs b/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuSequenceHeader.cs new file mode 100644 index 0000000000..f694837976 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuSequenceHeader.cs @@ -0,0 +1,237 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; + +/// +/// Stores AV1 sequence capabilities, dimensions, timing, and color configuration. +/// +internal sealed class ObuSequenceHeader +{ + /// + /// Backs while its dependent geometry is updated. + /// + private bool use128x128Superblock; + + /// + /// Initializes a new instance of the class with the normative 64x64 superblock geometry. + /// + public ObuSequenceHeader() => this.Use128x128Superblock = false; + + /// + /// Gets or sets a value indicating whether filter-intra prediction is enabled. + /// + public bool EnableFilterIntra { get; set; } + + /// + /// Gets or sets a value indicating whether constrained directional enhancement filtering is enabled. + /// + public bool EnableCdef { get; set; } + + /// + /// Gets or sets a value indicating whether the sequence contains one still picture. + /// + public bool IsStillPicture { get; set; } + + /// + /// Gets or sets a value indicating whether the reduced still-picture header syntax is used. + /// + public bool IsReducedStillPictureHeader { get; set; } + + /// + /// Gets or sets the sequence profile. + /// + public ObuSequenceProfile SequenceProfile { get; set; } + + /// + /// Gets or sets the signaled operating points. + /// + public ObuOperatingPoint[] OperatingPoint { get; set; } = new ObuOperatingPoint[1]; + + /// + /// Gets or sets the decoder-buffer model information when present. + /// + public ObuDecoderModelInfo? DecoderModelInfo { get; set; } + + /// + /// Gets or sets a value indicating whether operating points may carry initial display delays. + /// + public bool InitialDisplayDelayPresentFlag { get; set; } + + /// + /// Gets or sets a value indicating whether decoder-model information is present. + /// + public bool DecoderModelInfoPresentFlag { get; set; } + + /// + /// Gets or sets a value indicating whether sequence timing information is present. + /// + public bool TimingInfoPresentFlag { get; set; } + + /// + /// Gets or sets the sequence timing information when present. + /// + public ObuTimingInfo? TimingInfo { get; set; } + + /// + /// Gets or sets a value indicating whether frame identifiers are signaled. + /// + public bool IsFrameIdNumbersPresent { get; set; } + + /// + /// Gets or sets the number of bits used to signal frame widths minus one. + /// + public int FrameWidthBits { get; set; } + + /// + /// Gets or sets the number of bits used to signal frame heights minus one. + /// + public int FrameHeightBits { get; set; } + + /// + /// Gets or sets the maximum decoded frame width in pixels. + /// + public int MaxFrameWidth { get; set; } + + /// + /// Gets or sets the maximum decoded frame height in pixels. + /// + public int MaxFrameHeight { get; set; } + + /// + /// Gets or sets a value indicating whether 128x128 superblocks are used. + /// + public bool Use128x128Superblock + { + get => this.use128x128Superblock; + set + { + this.use128x128Superblock = value; + + // Superblock selection controls every downstream partition-grid unit, so update the derived geometry atomically. + this.SuperblockSize = value ? Av1BlockSize.Block128x128 : Av1BlockSize.Block64x64; + this.SuperblockSizeLog2 = value ? 7 : 6; + this.SuperblockModeInfoSize = 1 << (this.SuperblockSizeLog2 - Av1Constants.ModeInfoSizeLog2); + } + } + + /// + /// Gets the selected superblock dimensions. + /// + public Av1BlockSize SuperblockSize { get; private set; } + + /// + /// Gets the superblock width and height in mode-information units. + /// + public int SuperblockModeInfoSize { get; private set; } + + /// + /// Gets the base-two logarithm of the superblock size in pixels. + /// + public int SuperblockSizeLog2 { get; private set; } + + /// + /// Gets or sets a value indicating whether intra edge filtering is enabled. + /// + public bool EnableIntraEdgeFilter { get; set; } + + /// + /// Gets or sets the order-hint capabilities. + /// + public ObuOrderHintInfo OrderHintInfo { get; set; } = new ObuOrderHintInfo(); + + /// + /// Gets or sets a value indicating whether order hints are enabled. + /// + public bool EnableOrderHint + { + get => this.OrderHintInfo.EnableOrderHint; + set + { + // Order-hint availability is consumed through OrderHintInfo by frame parsing, so + // keep the sequence-facing flag and dependent tool state synchronized. + this.OrderHintInfo.EnableOrderHint = value; + } + } + + /// + /// Gets or sets a value indicating whether inter-intra compound prediction is enabled. + /// + public bool EnableInterIntraCompound { get; set; } + + /// + /// Gets or sets a value indicating whether masked compound prediction is enabled. + /// + public bool EnableMaskedCompound { get; set; } + + /// + /// Gets or sets a value indicating whether warped motion is enabled. + /// + public bool EnableWarpedMotion { get; set; } + + /// + /// Gets or sets a value indicating whether independent horizontal and vertical interpolation filters are enabled. + /// + public bool EnableDualFilter { get; set; } + + /// + /// Gets or sets the sequence-level integer-motion-vector selection. + /// + public int ForceIntegerMotionVector { get; set; } + + /// + /// Gets or sets the sequence-level screen-content-tools selection. + /// + public int ForceScreenContentTools { get; set; } + + /// + /// Gets or sets a value indicating whether frame super-resolution is enabled. + /// + public bool EnableSuperResolution { get; set; } + + /// + /// Gets or sets a value indicating whether loop restoration is enabled. + /// + public bool EnableRestoration { get; set; } + + /// + /// Gets or sets the decoded color configuration. + /// + public ObuColorConfig ColorConfig { get; set; } = new ObuColorConfig(); + + /// + /// Gets or sets a value indicating whether frame headers may carry film-grain parameters. + /// + public bool AreFilmGrainingParametersPresent { get; set; } + + /// + /// Gets or sets the total number of bits in a frame identifier. + /// + public int FrameIdLength { get; set; } + + /// + /// Gets or sets the number of bits in a delta frame identifier. + /// + public int DeltaFrameIdLength { get; set; } + + /// + /// Gets or sets the additional frame-identifier bit count signaled by the sequence header. + /// + public uint AdditionalFrameIdLength { get; set; } + + /// + /// Gets the timing information required by syntax whose presence flag is set. + /// + /// The sequence timing information. + public ObuTimingInfo GetTimingInfo() => + this.TimingInfo + ?? throw new InvalidOperationException("The AV1 sequence has no timing information."); + + /// + /// Gets the decoder-buffer model information required by syntax whose presence flag is set. + /// + /// The decoder-buffer model information. + public ObuDecoderModelInfo GetDecoderModelInfo() => + this.DecoderModelInfo + ?? throw new InvalidOperationException("The AV1 sequence has no decoder-model information."); +} diff --git a/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuSequenceProfile.cs b/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuSequenceProfile.cs new file mode 100644 index 0000000000..2bd5af9b3e --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuSequenceProfile.cs @@ -0,0 +1,25 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; + +/// +/// Identifies the AV1 sequence profile and its permitted bit-depth and chroma formats. +/// +internal enum ObuSequenceProfile : uint +{ + /// + /// The Main profile. + /// + Main = 0, + + /// + /// The High profile. + /// + High = 1, + + /// + /// The Professional profile. + /// + Professional = 2, +} diff --git a/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuSkipModeParameters.cs b/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuSkipModeParameters.cs new file mode 100644 index 0000000000..e5c9766d2b --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuSkipModeParameters.cs @@ -0,0 +1,109 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; + +/// +/// Contains the availability and enabled state of AV1 skip mode for a frame. +/// +internal sealed class ObuSkipModeParameters +{ + /// + /// Gets a value indicating whether the frame is permitted to use skip mode. + /// + public bool SkipModeAllowed { get; private set; } + + /// + /// Gets or sets a value indicating whether skip mode is enabled for the frame. + /// + public bool SkipModeFlag { get; set; } + + /// + /// Gets the first canonical inter-reference type selected for skip-mode blocks. + /// + public Av1ReferenceFrameType FirstReferenceFrame { get; private set; } = Av1ReferenceFrameType.None; + + /// + /// Gets the second canonical inter-reference type selected for skip-mode blocks. + /// + public Av1ReferenceFrameType SecondReferenceFrame { get; private set; } = Av1ReferenceFrameType.None; + + /// + /// Derives skip-mode availability and its reference pair from the current frame's retained-reference mapping. + /// + /// The sequence-level order-hint configuration. + /// The current frame header and its seven canonical inter-reference mappings. + public void Derive(ObuOrderHintInfo orderHintInfo, ObuFrameHeader frameHeader) + { + this.SkipModeAllowed = false; + this.FirstReferenceFrame = Av1ReferenceFrameType.None; + this.SecondReferenceFrame = Av1ReferenceFrameType.None; + + if (!orderHintInfo.EnableOrderHint || frameHeader.IsIntra || frameHeader.ReferenceMode == ObuReferenceMode.SingleReference) + { + return; + } + + ReadOnlySpan referenceFrameIndices = frameHeader.GetReferenceFrameIndices(); + ReadOnlySpan referenceOrderHints = frameHeader.GetReferenceOrderHints(); + int nearestForwardOrderHint = -1; + int nearestBackwardOrderHint = int.MaxValue; + int nearestForwardReferenceIndex = -1; + int nearestBackwardReferenceIndex = -1; + + // The seven entries are canonical roles, while each value selects one physical reference-map slot. Compare + // the selected slot's order hint so duplicate roles retain the same deterministic ordering as the reference decoder. + for (int referenceIndex = 0; referenceIndex < Av1Constants.ReferencesPerFrame; referenceIndex++) + { + uint referenceOrderHint = referenceOrderHints[(int)referenceFrameIndices[referenceIndex]]; + int distanceFromCurrent = orderHintInfo.GetRelativeDistance(referenceOrderHint, frameHeader.OrderHint); + + if (distanceFromCurrent < 0 && + (nearestForwardOrderHint == -1 || orderHintInfo.GetRelativeDistance(referenceOrderHint, (uint)nearestForwardOrderHint) > 0)) + { + // Among past frames, the greatest relative order is the closest frame before the current one. + nearestForwardOrderHint = (int)referenceOrderHint; + nearestForwardReferenceIndex = referenceIndex; + } + else if (distanceFromCurrent > 0 && + (nearestBackwardOrderHint == int.MaxValue || orderHintInfo.GetRelativeDistance(referenceOrderHint, (uint)nearestBackwardOrderHint) < 0)) + { + // Among future frames, the smallest relative order is the closest frame after the current one. + nearestBackwardOrderHint = (int)referenceOrderHint; + nearestBackwardReferenceIndex = referenceIndex; + } + } + + if (nearestForwardReferenceIndex >= 0 && nearestBackwardReferenceIndex < 0) + { + nearestBackwardOrderHint = -1; + + // A forward-only sequence pairs the nearest past frame with the closest distinct frame preceding it. + for (int referenceIndex = 0; referenceIndex < Av1Constants.ReferencesPerFrame; referenceIndex++) + { + uint referenceOrderHint = referenceOrderHints[(int)referenceFrameIndices[referenceIndex]]; + bool precedesNearestForward = orderHintInfo.GetRelativeDistance(referenceOrderHint, (uint)nearestForwardOrderHint) < 0; + + if (precedesNearestForward && + (nearestBackwardOrderHint == -1 || orderHintInfo.GetRelativeDistance(referenceOrderHint, (uint)nearestBackwardOrderHint) > 0)) + { + nearestBackwardOrderHint = (int)referenceOrderHint; + nearestBackwardReferenceIndex = referenceIndex; + } + } + } + + if (nearestForwardReferenceIndex < 0 || nearestBackwardReferenceIndex < 0) + { + return; + } + + int firstReferenceIndex = Math.Min(nearestForwardReferenceIndex, nearestBackwardReferenceIndex); + int secondReferenceIndex = Math.Max(nearestForwardReferenceIndex, nearestBackwardReferenceIndex); + this.FirstReferenceFrame = (Av1ReferenceFrameType)(firstReferenceIndex + (int)Av1ReferenceFrameType.Last); + this.SecondReferenceFrame = (Av1ReferenceFrameType)(secondReferenceIndex + (int)Av1ReferenceFrameType.Last); + this.SkipModeAllowed = true; + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuTileGroupHeader.cs b/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuTileGroupHeader.cs new file mode 100644 index 0000000000..99238b13fb --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuTileGroupHeader.cs @@ -0,0 +1,93 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; + +/// +/// Contains the tile layout derived from an AV1 frame header. +/// +internal sealed class ObuTileGroupHeader +{ + private InlineArray65 tileColumnStartModeInfo; + private InlineArray65 tileRowStartModeInfo; + + /// + /// Gets or sets the maximum tile width, in superblocks. + /// + public int MaxTileWidthSuperblock { get; set; } + + /// + /// Gets or sets the maximum tile height, in superblocks. + /// + public int MaxTileHeightSuperblock { get; set; } + + /// + /// Gets or sets the minimum base-2 logarithm of the tile-column count. + /// + public int MinLog2TileColumnCount { get; set; } + + /// + /// Gets or sets the maximum base-2 logarithm of the tile-column count. + /// + public int MaxLog2TileColumnCount { get; set; } + + /// + /// Gets or sets the maximum base-2 logarithm of the tile-row count. + /// + public int MaxLog2TileRowCount { get; set; } + + /// + /// Gets or sets the minimum base-2 logarithm of the total tile count. + /// + public int MinLog2TileCount { get; set; } + + /// + /// Gets or sets a value indicating whether tile columns and rows use uniform spacing. + /// + public bool HasUniformTileSpacing { get; set; } + + /// + /// Gets or sets the base-2 logarithm of the tile-column count. + /// + public int TileColumnCountLog2 { get; set; } + + /// + /// Gets or sets the number of tile columns. + /// + public int TileColumnCount { get; set; } + + /// + /// Gets the fixed-capacity starting superblock column storage for each tile column. + /// + public Span TileColumnStartModeInfo => this.tileColumnStartModeInfo; + + /// + /// Gets or sets the minimum base-2 logarithm of the tile-row count. + /// + public int MinLog2TileRowCount { get; set; } + + /// + /// Gets or sets the base-2 logarithm of the tile-row count. + /// + public int TileRowCountLog2 { get; set; } + + /// + /// Gets the fixed-capacity starting superblock row storage for each tile row. + /// + public Span TileRowStartModeInfo => this.tileRowStartModeInfo; + + /// + /// Gets or sets the number of tile rows. + /// + public int TileRowCount { get; set; } + + /// + /// Gets or sets the tile whose entropy context is retained after frame decoding. + /// + public uint ContextUpdateTileId { get; set; } + + /// + /// Gets or sets the number of bytes used to signal each tile size. + /// + public int TileSizeBytes { get; set; } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuTimingInfo.cs b/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuTimingInfo.cs new file mode 100644 index 0000000000..8cbb49487e --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuTimingInfo.cs @@ -0,0 +1,36 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; + +/// +/// Contains the timing syntax signaled by an AV1 sequence header. +/// +internal sealed class ObuTimingInfo +{ + /// + /// Gets or sets NumUnitsInDisplayTick. NumUnitsInDisplayTick is the number of time units of a clock operating at the frequency TimeScale Hz that + /// corresponds to one increment of a clock tick counter. A display clock tick, in seconds, is equal to + /// NumUnitsInDisplayTick divided by TimeScale. + /// + public uint NumUnitsInDisplayTick { get; set; } + + /// + /// Gets or sets TimeScale. TimeScale is the number of time units that pass in one second. + /// It is a requirement of bitstream conformance that TimeScale is greater than 0. + /// + public uint TimeScale { get; set; } + + /// + /// Gets or sets a value indicating whether that pictures should be displayed according to their output order with the + /// number of ticks between two consecutive pictures (without dropping frames) specified by NumTicksPerPicture. + /// EqualPictureInterval equal to false indicates that the interval between two consecutive pictures is not specified. + /// + public bool EqualPictureInterval { get; set; } + + /// + /// Gets or sets NumTicksPerPicture. NumTicksPerPicture specifies the number of clock ticks corresponding to output time between two + /// consecutive pictures in the output order. + /// + public uint NumTicksPerPicture { get; set; } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuTransferCharacteristics.cs b/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuTransferCharacteristics.cs new file mode 100644 index 0000000000..739f1a72c3 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuTransferCharacteristics.cs @@ -0,0 +1,95 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; + +/// +/// Identifies the CICP transfer characteristics signaled by an AV1 sequence. +/// +internal enum ObuTransferCharacteristics +{ + /// + /// ITU-R BT.709 transfer characteristics. + /// + Bt709 = 1, + + /// + /// Unspecified transfer characteristics. + /// + Unspecified = 2, + + /// + /// ITU-R BT.470 System M transfer characteristics. + /// + Bt470M = 4, + + /// + /// ITU-R BT.470 System B and G transfer characteristics. + /// + Bt470BG = 5, + + /// + /// ITU-R BT.601 transfer characteristics. + /// + Bt601 = 6, + + /// + /// SMPTE 240M transfer characteristics. + /// + Smpte240 = 7, + + /// + /// Linear light. + /// + Linear = 8, + + /// + /// Logarithmic transfer with a 100:1 range. + /// + Log100 = 9, + + /// + /// Logarithmic transfer with a 100 times square-root-of-ten to one range. + /// + Log100Sqrt10 = 10, + + /// + /// IEC 61966-2-4 transfer characteristics. + /// + Iec61966 = 11, + + /// + /// ITU-R BT.1361 transfer characteristics. + /// + Bt1361 = 12, + + /// + /// IEC 61966-2-1 sRGB or sYCC transfer characteristics. + /// + Srgb = 13, + + /// + /// ITU-R BT.2020 transfer characteristics for 10-bit systems. + /// + Bt202010Bit = 14, + + /// + /// ITU-R BT.2020 transfer characteristics for 12-bit systems. + /// + Bt202012Bit = 15, + + /// + /// SMPTE ST 2084 perceptual-quantizer transfer characteristics. + /// + Smpte2084 = 16, + + /// + /// SMPTE ST 428 transfer characteristics. + /// + Smpte428 = 17, + + /// + /// ITU-R BT.2100 hybrid-log-gamma transfer characteristics. + /// + Hlg = 18, +} diff --git a/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuType.cs b/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuType.cs new file mode 100644 index 0000000000..404837c0da --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuType.cs @@ -0,0 +1,60 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; + +/// +/// Identifies an AV1 open bitstream unit payload. +/// +internal enum ObuType +{ + /// + /// The reserved zero value. + /// + None = 0, + + /// + /// A sequence header. + /// + SequenceHeader = 1, + + /// + /// A temporal delimiter. + /// + TemporalDelimiter = 2, + + /// + /// A frame header without tile data. + /// + FrameHeader = 3, + + /// + /// One or more encoded tiles. + /// + TileGroup = 4, + + /// + /// Metadata associated with the coded sequence. + /// + Metadata = 5, + + /// + /// A frame header followed by tile data. + /// + Frame = 6, + + /// + /// A repeated copy of the current frame header. + /// + RedundantFrameHeader = 7, + + /// + /// A list of tiles for large-scale tile decoding. + /// + TileList = 8, + + /// + /// Padding bytes. + /// + Padding = 15, +} diff --git a/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuWriter.cs b/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuWriter.cs new file mode 100644 index 0000000000..7d06c4e3d2 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuWriter.cs @@ -0,0 +1,1477 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Buffers; +using System.Buffers.Binary; +using SixLabors.ImageSharp.Formats.Heif.Av1.Motion; +using SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.Quantizers; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; +using SixLabors.ImageSharp.Memory; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; + +/// +/// Writes the AV1 open bitstream units for one coded frame. +/// +internal sealed class ObuWriter : IDisposable +{ + // Sequence and uncompressed-frame syntax have fixed field and array limits. A 512-byte owner covers their + // maximum supported representation without retaining any entropy-coded tile bytes in the header scratch. + private const int MaximumHeaderLength = 512; + + private readonly IMemoryOwner headerOwner; + + /// + /// Initializes a new instance of the class. + /// + /// The configuration providing reusable header memory. + public ObuWriter(Configuration configuration) + => this.headerOwner = configuration.MemoryAllocator.Allocate(MaximumHeaderLength); + + /// + /// Writes a temporal delimiter, sequence header, and coded frame for the first sample in a sequence. + /// + /// The non-boxed tile source type. + /// The destination stream. + /// The sequence header. + /// The frame header. + /// The encoded tile source. + public void WriteSequenceFrame( + Stream stream, + ObuSequenceHeader sequenceHeader, + ObuFrameHeader frameHeader, + TTileWriter tileWriter) + where TTileWriter : IAv1TileWriter + { + Span headerBuffer = this.headerOwner.Memory.Span[..MaximumHeaderLength]; + Av1BitStreamWriter writer = new(headerBuffer); + WriteObuHeaderAndSize(stream, ObuType.TemporalDelimiter, []); + WriteSequenceHeader(ref writer, sequenceHeader); + int bytesWritten = (writer.BitPosition + 7) >> 3; + writer.Flush(); + WriteObuHeaderAndSize(stream, ObuType.SequenceHeader, headerBuffer[..bytesWritten]); + WriteFrameObu(stream, sequenceHeader, frameHeader, tileWriter, headerBuffer, ref writer); + } + + /// + /// Writes an empty temporal-delimiter OBU. + /// + /// The destination stream. + public static void WriteTemporalDelimiter(Stream stream) + => WriteObuHeaderAndSize(stream, ObuType.TemporalDelimiter, []); + + /// + /// Writes a temporal delimiter followed by one sequence-header OBU. + /// + /// The configuration used to allocate temporary encoding memory. + /// The destination stream. + /// The sequence header. + public static void WriteSequenceHeader(Configuration configuration, Stream stream, ObuSequenceHeader sequenceHeader) + { + using IMemoryOwner headerOwner = configuration.MemoryAllocator.Allocate(MaximumHeaderLength); + Span headerBuffer = headerOwner.Memory.Span[..MaximumHeaderLength]; + Av1BitStreamWriter writer = new(headerBuffer); + WriteObuHeaderAndSize(stream, ObuType.TemporalDelimiter, []); + WriteSequenceHeader(ref writer, sequenceHeader); + int bytesWritten = (writer.BitPosition + 7) >> 3; + writer.Flush(); + WriteObuHeaderAndSize(stream, ObuType.SequenceHeader, headerBuffer[..bytesWritten]); + } + + /// + /// Writes a temporal delimiter and coded frame that continues an established sequence. + /// + /// The non-boxed tile source type. + /// The destination stream. + /// The sequence header established by an earlier sample. + /// The frame header. + /// The encoded tile source. + public void WriteFrame( + Stream stream, + ObuSequenceHeader sequenceHeader, + ObuFrameHeader frameHeader, + TTileWriter tileWriter) + where TTileWriter : IAv1TileWriter + { + Span headerBuffer = this.headerOwner.Memory.Span[..MaximumHeaderLength]; + Av1BitStreamWriter writer = new(headerBuffer); + WriteObuHeaderAndSize(stream, ObuType.TemporalDelimiter, []); + WriteFrameObu(stream, sequenceHeader, frameHeader, tileWriter, headerBuffer, ref writer); + } + + /// + public void Dispose() => this.headerOwner.Dispose(); + + /// + /// Writes the combined frame OBU header followed by each retained tile payload. + /// + /// The non-boxed tile source type. + /// The destination stream. + /// The sequence header governing frame syntax. + /// The uncompressed frame header. + /// The encoded tile source. + /// The reusable OBU header scratch. + /// The bit writer over . + private static void WriteFrameObu( + Stream stream, + ObuSequenceHeader sequenceHeader, + ObuFrameHeader frameHeader, + TTileWriter tileWriter, + Span headerBuffer, + ref Av1BitStreamWriter writer) + where TTileWriter : IAv1TileWriter + { + WriteFrameHeader(ref writer, sequenceHeader, frameHeader); + ObuTileGroupHeader tileInfo = frameHeader.TilesInfo; + WriteTileGroupHeader(ref writer, tileInfo); + + int frameHeaderBytes = (writer.BitPosition + 7) >> 3; + writer.Flush(); + + int tileCount = tileInfo.TileColumnCount * tileInfo.TileRowCount; + uint framePayloadSize = (uint)(frameHeaderBytes + ((tileCount - 1) * tileInfo.TileSizeBytes)); + for (int tileNum = 0; tileNum < tileCount; tileNum++) + { + framePayloadSize += (uint)tileWriter.GetTileData(tileNum).Length; + } + + WriteObuHeaderAndSize(stream, ObuType.Frame, framePayloadSize); + stream.Write(headerBuffer[..frameHeaderBytes]); + WriteTileData(stream, tileInfo, tileWriter); + } + + /// + /// Creates a byte-aligned OBU header with an explicit payload-size field and no extension. + /// + /// The OBU payload type. + /// The encoded OBU header byte. + private static byte WriteObuHeader(ObuType type) + { + // The only set fields are the four-bit type and the has-size flag; forbidden, + // extension, and reserved bits remain zero. + return (byte)(((byte)type << 3) | 0x02); + } + + /// + /// Writes a complete byte-aligned OBU with a little-endian base-128 payload size. + /// + /// The destination stream. + /// The OBU payload type. + /// The complete OBU payload. + private static void WriteObuHeaderAndSize(Stream stream, ObuType type, ReadOnlySpan payload) + { + WriteObuHeaderAndSize(stream, type, (uint)payload.Length); + stream.Write(payload); + } + + /// + /// Writes a byte-aligned OBU header and its little-endian base-128 payload size. + /// + /// The destination stream. + /// The OBU payload type. + /// The number of payload bytes that follow the header. + private static void WriteObuHeaderAndSize(Stream stream, ObuType type, uint payloadSize) + { + stream.WriteByte(WriteObuHeader(type)); + + // A 32-bit OBU payload length requires at most five base-128 bytes. + Span lengthBytes = stackalloc byte[5]; + int lengthLength = Av1BitStreamWriter.GetLittleEndianBytes128(payloadSize, lengthBytes); + stream.Write(lengthBytes, 0, lengthLength); + } + + /// + /// Writes a trailing one bit followed by enough zero bits to reach a byte boundary. + /// + /// The bit writer receiving the trailing bits. + /// Writes an additional byte when the writer is already byte aligned. + private static void WriteTrailingBits(ref Av1BitStreamWriter writer) + { + int bitsBeforeAlignment = 8 - (writer.BitPosition & 0x7); + writer.WriteLiteral(1U << (bitsBeforeAlignment - 1), bitsBeforeAlignment); + } + + /// + /// Writes zero padding until the output reaches a byte boundary. + /// + /// The bit writer to align. + private static void AlignToByteBoundary(ref Av1BitStreamWriter writer) + { + while ((writer.BitPosition & 0x7) > 0) + { + writer.WriteBoolean(false); + } + } + + /// + /// Writes an AV1 sequence header. + /// + /// The bit writer receiving the sequence header. + /// The sequence header to encode. + private static void WriteSequenceHeader(ref Av1BitStreamWriter writer, ObuSequenceHeader sequenceHeader) + { + writer.WriteLiteral((uint)sequenceHeader.SequenceProfile, 3); + writer.WriteBoolean(sequenceHeader.IsStillPicture); + writer.WriteBoolean(sequenceHeader.IsReducedStillPictureHeader); + if (sequenceHeader.IsReducedStillPictureHeader) + { + writer.WriteLiteral((uint)sequenceHeader.OperatingPoint[0].SequenceLevelIndex, Av1Constants.LevelBits); + } + else + { + writer.WriteBoolean(sequenceHeader.TimingInfoPresentFlag); + if (sequenceHeader.TimingInfoPresentFlag) + { + WriteTimingInfo(ref writer, sequenceHeader.GetTimingInfo()); + writer.WriteBoolean(sequenceHeader.DecoderModelInfoPresentFlag); + if (sequenceHeader.DecoderModelInfoPresentFlag) + { + WriteDecoderModelInfo(ref writer, sequenceHeader.GetDecoderModelInfo()); + } + } + + writer.WriteBoolean(sequenceHeader.InitialDisplayDelayPresentFlag); + writer.WriteLiteral( + (uint)(sequenceHeader.OperatingPoint.Length - 1), + Av1Constants.OperatingPointCountBits); + + foreach (ObuOperatingPoint operatingPoint in sequenceHeader.OperatingPoint) + { + writer.WriteLiteral(operatingPoint.Idc, Av1Constants.OperatingPointIdcBits); + writer.WriteLiteral((uint)operatingPoint.SequenceLevelIndex, Av1Constants.LevelBits); + if (operatingPoint.SequenceLevelIndex >= Av1Constants.SequenceTierMinimumLevelIndex) + { + writer.WriteBoolean(operatingPoint.SequenceTier != 0); + } + + if (sequenceHeader.DecoderModelInfoPresentFlag) + { + writer.WriteBoolean(operatingPoint.IsDecoderModelInfoPresent); + if (operatingPoint.IsDecoderModelInfoPresent) + { + WriteOperatingParametersInfo( + ref writer, + sequenceHeader.GetDecoderModelInfo(), + operatingPoint); + } + } + + if (sequenceHeader.InitialDisplayDelayPresentFlag) + { + writer.WriteBoolean(operatingPoint.IsInitialDisplayDelayPresent); + if (operatingPoint.IsInitialDisplayDelayPresent) + { + writer.WriteLiteral(operatingPoint.InitialDisplayDelay - 1, 4); + } + } + } + } + + // The maximum dimensions determine the fixed-width fields used by every frame in the sequence. + writer.WriteLiteral((uint)sequenceHeader.FrameWidthBits - 1, 4); + writer.WriteLiteral((uint)sequenceHeader.FrameHeightBits - 1, 4); + writer.WriteLiteral((uint)sequenceHeader.MaxFrameWidth - 1, sequenceHeader.FrameWidthBits); + writer.WriteLiteral((uint)sequenceHeader.MaxFrameHeight - 1, sequenceHeader.FrameHeightBits); + if (!sequenceHeader.IsReducedStillPictureHeader) + { + writer.WriteBoolean(sequenceHeader.IsFrameIdNumbersPresent); + if (sequenceHeader.IsFrameIdNumbersPresent) + { + writer.WriteLiteral((uint)sequenceHeader.DeltaFrameIdLength - 2, 4); + writer.WriteLiteral(sequenceHeader.AdditionalFrameIdLength - 1, 3); + } + } + + writer.WriteBoolean(sequenceHeader.Use128x128Superblock); + writer.WriteBoolean(sequenceHeader.EnableFilterIntra); + writer.WriteBoolean(sequenceHeader.EnableIntraEdgeFilter); + if (!sequenceHeader.IsReducedStillPictureHeader) + { + writer.WriteBoolean(sequenceHeader.EnableInterIntraCompound); + writer.WriteBoolean(sequenceHeader.EnableMaskedCompound); + writer.WriteBoolean(sequenceHeader.EnableWarpedMotion); + writer.WriteBoolean(sequenceHeader.EnableDualFilter); + writer.WriteBoolean(sequenceHeader.EnableOrderHint); + if (sequenceHeader.EnableOrderHint) + { + writer.WriteBoolean(sequenceHeader.OrderHintInfo.EnableJointCompound); + writer.WriteBoolean(sequenceHeader.OrderHintInfo.EnableReferenceFrameMotionVectors); + } + + bool chooseScreenContentTools = sequenceHeader.ForceScreenContentTools == Av1Constants.SelectScreenContentTools; + writer.WriteBoolean(chooseScreenContentTools); + if (!chooseScreenContentTools) + { + writer.WriteBoolean(sequenceHeader.ForceScreenContentTools != 0); + } + + if (sequenceHeader.ForceScreenContentTools > 0) + { + bool chooseIntegerMotionVector = sequenceHeader.ForceIntegerMotionVector == Av1Constants.SelectIntegerMotionVector; + writer.WriteBoolean(chooseIntegerMotionVector); + if (!chooseIntegerMotionVector) + { + writer.WriteBoolean(sequenceHeader.ForceIntegerMotionVector != 0); + } + } + + if (sequenceHeader.EnableOrderHint) + { + writer.WriteLiteral((uint)sequenceHeader.OrderHintInfo.OrderHintBits - 1, 3); + } + } + + writer.WriteBoolean(sequenceHeader.EnableSuperResolution); + writer.WriteBoolean(sequenceHeader.EnableCdef); + writer.WriteBoolean(sequenceHeader.EnableRestoration); + WriteColorConfig(ref writer, sequenceHeader); + writer.WriteBoolean(sequenceHeader.AreFilmGrainingParametersPresent); + WriteTrailingBits(ref writer); + } + + /// + /// Writes sequence timing in the fixed-width and unsigned-variable-length forms required by AV1. + /// + /// The bit writer receiving the timing information. + /// The timing values to encode. + private static void WriteTimingInfo(ref Av1BitStreamWriter writer, ObuTimingInfo timingInfo) + { + writer.WriteLiteral(timingInfo.NumUnitsInDisplayTick, 32); + writer.WriteLiteral(timingInfo.TimeScale, 32); + writer.WriteBoolean(timingInfo.EqualPictureInterval); + if (timingInfo.EqualPictureInterval) + { + WriteUnsignedVariableLength(ref writer, timingInfo.NumTicksPerPicture - 1); + } + } + + /// + /// Writes decoder-buffer field widths and decoding-clock units. + /// + /// The bit writer receiving the decoder-model information. + /// The decoder-model values to encode. + private static void WriteDecoderModelInfo(ref Av1BitStreamWriter writer, ObuDecoderModelInfo decoderModelInfo) + { + writer.WriteLiteral(decoderModelInfo.BufferDelayLength - 1, 5); + writer.WriteLiteral(decoderModelInfo.NumUnitsInDecodingTick, 32); + writer.WriteLiteral(decoderModelInfo.BufferRemovalTimeLength - 1, 5); + writer.WriteLiteral(decoderModelInfo.FramePresentationTimeLength - 1, 5); + } + + /// + /// Writes the decoder-model parameters for one operating point. + /// + /// The bit writer receiving the operating-point parameters. + /// The decoder model defining the delay field width. + /// The operating-point values to encode. + private static void WriteOperatingParametersInfo( + ref Av1BitStreamWriter writer, + ObuDecoderModelInfo decoderModelInfo, + ObuOperatingPoint operatingPoint) + { + int bufferDelayLength = (int)decoderModelInfo.BufferDelayLength; + writer.WriteLiteral(operatingPoint.DecoderBufferDelay, bufferDelayLength); + writer.WriteLiteral(operatingPoint.EncoderBufferDelay, bufferDelayLength); + writer.WriteBoolean(operatingPoint.LowDelayMode); + } + + /// + /// Writes an AV1 unsigned variable-length value. + /// + /// The bit writer receiving the value. + /// The value to encode. + private static void WriteUnsignedVariableLength(ref Av1BitStreamWriter writer, uint value) + { + uint encodedValue = value + 1; + int leadingZeroCount = Av1Math.MostSignificantBit(encodedValue); + writer.WriteLiteral(0, leadingZeroCount); + writer.WriteLiteral(encodedValue, leadingZeroCount + 1); + } + + /// + /// Writes the sequence color configuration. + /// + /// The bit writer receiving the color configuration. + /// The sequence header containing the color configuration. + private static void WriteColorConfig(ref Av1BitStreamWriter writer, ObuSequenceHeader sequenceHeader) + { + ObuColorConfig colorConfig = sequenceHeader.ColorConfig; + WriteBitDepth(ref writer, colorConfig, sequenceHeader); + if (sequenceHeader.SequenceProfile != ObuSequenceProfile.High) + { + writer.WriteBoolean(colorConfig.IsMonochrome); + } + + writer.WriteBoolean(colorConfig.IsColorDescriptionPresent); + if (colorConfig.IsColorDescriptionPresent) + { + writer.WriteLiteral((uint)colorConfig.ColorPrimaries, 8); + writer.WriteLiteral((uint)colorConfig.TransferCharacteristics, 8); + writer.WriteLiteral((uint)colorConfig.MatrixCoefficients, 8); + } + + if (colorConfig.IsMonochrome) + { + writer.WriteBoolean(colorConfig.ColorRange); + return; + } + else if ( + colorConfig.ColorPrimaries == ObuColorPrimaries.Bt709 && + colorConfig.TransferCharacteristics == ObuTransferCharacteristics.Srgb && + colorConfig.MatrixCoefficients == ObuMatrixCoefficients.Identity) + { + // AV1 fixes this RGB identity-matrix combination to full-range 4:4:4 and omits + // the range and subsampling fields used by YUV configurations. + } + else + { + writer.WriteBoolean(colorConfig.ColorRange); + if (sequenceHeader.SequenceProfile == ObuSequenceProfile.Professional && colorConfig.BitDepth == Av1BitDepth.TwelveBit) + { + writer.WriteBoolean(colorConfig.SubSamplingX); + if (colorConfig.SubSamplingX) + { + writer.WriteBoolean(colorConfig.SubSamplingY); + } + } + + if (colorConfig.SubSamplingX && colorConfig.SubSamplingY) + { + writer.WriteLiteral((uint)colorConfig.ChromaSamplePosition, 2); + } + } + + writer.WriteBoolean(colorConfig.HasSeparateUvDelta); + } + + /// + /// Writes the profile-dependent bit-depth flags. + /// + /// The bit writer receiving the flags. + /// The color configuration containing the bit depth. + /// The sequence header containing the selected profile. + private static void WriteBitDepth(ref Av1BitStreamWriter writer, ObuColorConfig colorConfig, ObuSequenceHeader sequenceHeader) + { + bool hasHighBitDepth = colorConfig.BitDepth > Av1BitDepth.EightBit; + writer.WriteBoolean(hasHighBitDepth); + if (sequenceHeader.SequenceProfile == ObuSequenceProfile.Professional && hasHighBitDepth) + { + writer.WriteBoolean(colorConfig.BitDepth == Av1BitDepth.TwelveBit); + } + } + + /// + /// Writes the super-resolution enablement and scale denominator. + /// + /// The bit writer receiving the super-resolution syntax. + /// The sequence header controlling super-resolution availability. + /// The frame header containing the scale denominator. + private static void WriteSuperResolutionParameters(ref Av1BitStreamWriter writer, ObuSequenceHeader sequenceHeader, ObuFrameHeader frameHeader) + { + bool useSuperResolution = sequenceHeader.EnableSuperResolution && + frameHeader.FrameSize.SuperResolutionDenominator != Av1Constants.ScaleNumerator; + + if (sequenceHeader.EnableSuperResolution) + { + writer.WriteBoolean(useSuperResolution); + } + + if (useSuperResolution) + { + writer.WriteLiteral((uint)frameHeader.FrameSize.SuperResolutionDenominator - Av1Constants.SuperResolutionScaleDenominatorMinimum, Av1Constants.SuperResolutionScaleBits); + } + } + + /// + /// Writes the render-size override when display dimensions differ from the upscaled frame. + /// + /// The bit writer receiving the render-size syntax. + /// The frame header containing coded and render dimensions. + private static void WriteRenderSize(ref Av1BitStreamWriter writer, ObuFrameHeader frameHeader) + { + bool renderSizeAndFrameSizeDifferent = + frameHeader.FrameSize.RenderWidth != frameHeader.FrameSize.SuperResolutionUpscaledWidth || + frameHeader.FrameSize.RenderHeight != frameHeader.FrameSize.FrameHeight; + + writer.WriteBoolean(renderSizeAndFrameSizeDifferent); + if (renderSizeAndFrameSizeDifferent) + { + writer.WriteLiteral((uint)frameHeader.FrameSize.RenderWidth - 1, 16); + writer.WriteLiteral((uint)frameHeader.FrameSize.RenderHeight - 1, 16); + } + } + + /// + /// Writes an optional frame-size override followed by super-resolution syntax. + /// + /// The bit writer receiving the frame size. + /// The sequence header defining dimension field widths. + /// The frame header containing the dimensions. + /// A value indicating whether explicit dimensions are written. + private static void WriteFrameSize(ref Av1BitStreamWriter writer, ObuSequenceHeader sequenceHeader, ObuFrameHeader frameHeader, bool frameSizeOverrideFlag) + { + if (frameSizeOverrideFlag) + { + writer.WriteLiteral((uint)frameHeader.FrameSize.FrameWidth - 1, sequenceHeader.FrameWidthBits); + writer.WriteLiteral((uint)frameHeader.FrameSize.FrameHeight - 1, sequenceHeader.FrameHeightBits); + } + + WriteSuperResolutionParameters(ref writer, sequenceHeader, frameHeader); + } + + /// + /// Writes the frame tile layout. + /// + /// The bit writer receiving the tile information. + /// The sequence header defining superblock geometry. + /// The frame header containing tile boundaries. + private static void WriteTileInfo(ref Av1BitStreamWriter writer, ObuSequenceHeader sequenceHeader, ObuFrameHeader frameHeader) + { + ObuTileGroupHeader tileInfo = frameHeader.TilesInfo; + int superblockColumnCount; + int superblockRowCount; + int superblockSizeLog2 = sequenceHeader.SuperblockSizeLog2; + int superblockShift = superblockSizeLog2 - Av1Constants.ModeInfoSizeLog2; + superblockColumnCount = (frameHeader.ModeInfoColumnCount + sequenceHeader.SuperblockModeInfoSize - 1) >> superblockShift; + superblockRowCount = (frameHeader.ModeInfoRowCount + sequenceHeader.SuperblockModeInfoSize - 1) >> superblockShift; + int superBlockSize = superblockShift + 2; + int maxTileAreaOfSuperBlock = Av1Constants.MaxTileArea >> (2 * superBlockSize); + + tileInfo.MaxTileWidthSuperblock = Av1Constants.MaxTileWidth >> superBlockSize; + tileInfo.MaxTileHeightSuperblock = (Av1Constants.MaxTileArea / Av1Constants.MaxTileWidth) >> superBlockSize; + tileInfo.MinLog2TileColumnCount = ObuReader.TileLog2(tileInfo.MaxTileWidthSuperblock, superblockColumnCount); + tileInfo.MaxLog2TileColumnCount = ObuReader.TileLog2(1, Math.Min(superblockColumnCount, Av1Constants.MaxTileColumnCount)); + tileInfo.MaxLog2TileRowCount = ObuReader.TileLog2(1, Math.Min(superblockRowCount, Av1Constants.MaxTileRowCount)); + tileInfo.MinLog2TileCount = Math.Max(tileInfo.MinLog2TileColumnCount, ObuReader.TileLog2(maxTileAreaOfSuperBlock, superblockColumnCount * superblockRowCount)); + + int log2TileColumnCount = ObuReader.TileLog2(1, tileInfo.TileColumnCount); + int log2TileRowCount = ObuReader.TileLog2(1, tileInfo.TileRowCount); + tileInfo.TileColumnCountLog2 = log2TileColumnCount; + tileInfo.TileRowCountLog2 = log2TileRowCount; + + writer.WriteBoolean(tileInfo.HasUniformTileSpacing); + if (tileInfo.HasUniformTileSpacing) + { + // Uniform spaced tiles with power-of-two number of rows and columns + // tile columns + int ones = log2TileColumnCount - tileInfo.MinLog2TileColumnCount; + while (ones-- > 0) + { + writer.WriteBoolean(true); + } + + if (log2TileColumnCount < tileInfo.MaxLog2TileColumnCount) + { + writer.WriteBoolean(false); + } + + // rows + tileInfo.MinLog2TileRowCount = Math.Max(tileInfo.MinLog2TileCount - log2TileColumnCount, 0); + ones = log2TileRowCount - tileInfo.MinLog2TileRowCount; + while (ones-- > 0) + { + writer.WriteBoolean(true); + } + + if (log2TileRowCount < tileInfo.MaxLog2TileRowCount) + { + writer.WriteBoolean(false); + } + } + else + { + int startSuperBlock = 0; + for (int i = 0; i < tileInfo.TileColumnCount; i++) + { + int endSuperBlock = i == tileInfo.TileColumnCount - 1 + ? superblockColumnCount + : tileInfo.TileColumnStartModeInfo[i + 1] >> superblockShift; + + // The stored terminal boundary is clipped to the visible mode-info width. libaom retains the exact + // superblock endpoint, so the final tile uses the derived frame-wide superblock count instead. + uint widthInSuperBlocks = (uint)(endSuperBlock - startSuperBlock); + uint maxWidth = (uint)Math.Min(superblockColumnCount - startSuperBlock, tileInfo.MaxTileWidthSuperblock); + writer.WriteNonSymmetric(widthInSuperBlocks - 1, maxWidth); + startSuperBlock = endSuperBlock; + } + + if (startSuperBlock != superblockColumnCount) + { + throw new ImageFormatException("Super block tiles width does not add up to total width."); + } + + startSuperBlock = 0; + for (int i = 0; i < tileInfo.TileRowCount; i++) + { + int endSuperBlock = i == tileInfo.TileRowCount - 1 + ? superblockRowCount + : tileInfo.TileRowStartModeInfo[i + 1] >> superblockShift; + + // As with columns, the final visible mode-info boundary may end inside its containing superblock. + uint heightInSuperBlocks = (uint)(endSuperBlock - startSuperBlock); + uint maxHeight = (uint)Math.Min(superblockRowCount - startSuperBlock, tileInfo.MaxTileHeightSuperblock); + writer.WriteNonSymmetric(heightInSuperBlocks - 1, maxHeight); + startSuperBlock = endSuperBlock; + } + + if (startSuperBlock != superblockRowCount) + { + throw new ImageFormatException("Super block tiles height does not add up to total height."); + } + } + + if (tileInfo.TileColumnCountLog2 > 0 || tileInfo.TileRowCountLog2 > 0) + { + writer.WriteLiteral(tileInfo.ContextUpdateTileId, tileInfo.TileRowCountLog2 + tileInfo.TileColumnCountLog2); + writer.WriteLiteral((uint)tileInfo.TileSizeBytes - 1, 2); + } + + frameHeader.TilesInfo = tileInfo; + } + + /// + /// Writes the uncompressed header for an AV1 frame. + /// + /// The bit writer receiving the uncompressed frame header. + /// The sequence header controlling available coding tools. + /// The frame header to encode. + private static void WriteUncompressedFrameHeader(ref Av1BitStreamWriter writer, ObuSequenceHeader sequenceHeader, ObuFrameHeader frameHeader) + { + bool frameSizeOverrideFlag = false; + if (!sequenceHeader.IsReducedStillPictureHeader) + { + writer.WriteBoolean(frameHeader.ShowExistingFrame); + if (frameHeader.ShowExistingFrame) + { + writer.WriteLiteral(frameHeader.FrameToShowMapIdx, Av1Constants.ReferenceFrameIndexBits); + if (sequenceHeader.DecoderModelInfoPresentFlag && !sequenceHeader.GetTimingInfo().EqualPictureInterval) + { + writer.WriteLiteral( + frameHeader.FramePresentationTime, + (int)sequenceHeader.GetDecoderModelInfo().FramePresentationTimeLength); + } + + if (sequenceHeader.IsFrameIdNumbersPresent) + { + writer.WriteLiteral(frameHeader.DisplayFrameId, sequenceHeader.FrameIdLength); + } + + return; + } + + writer.WriteLiteral((uint)frameHeader.FrameType, Av1Constants.FrameTypeBits); + writer.WriteBoolean(frameHeader.ShowFrame); + if (frameHeader.ShowFrame && + sequenceHeader.DecoderModelInfoPresentFlag && + !sequenceHeader.GetTimingInfo().EqualPictureInterval) + { + writer.WriteLiteral( + frameHeader.FramePresentationTime, + (int)sequenceHeader.GetDecoderModelInfo().FramePresentationTimeLength); + } + + if (!frameHeader.ShowFrame) + { + writer.WriteBoolean(frameHeader.ShowableFrame); + } + + if (frameHeader.FrameType != ObuFrameType.SwitchFrame && + (frameHeader.FrameType != ObuFrameType.KeyFrame || !frameHeader.ShowFrame)) + { + writer.WriteBoolean(frameHeader.ErrorResilientMode); + } + } + + writer.WriteBoolean(frameHeader.DisableCdfUpdate); + if (sequenceHeader.ForceScreenContentTools == Av1Constants.SelectScreenContentTools) + { + writer.WriteBoolean(frameHeader.AllowScreenContentTools); + } + else + { + // Guard.IsTrue(frameHeader.AllowScreenContentTools == sequenceHeader.ForceScreenContentTools); + } + + if (frameHeader.AllowScreenContentTools) + { + if (sequenceHeader.ForceIntegerMotionVector == Av1Constants.SelectIntegerMotionVector) + { + writer.WriteBoolean(frameHeader.ForceIntegerMotionVector); + } + else + { + // Guard.IsTrue(frameHeader.ForceIntegerMotionVector == sequenceHeader.ForceIntegerMotionVector, nameof(frameHeader.ForceIntegerMotionVector), "Frame and sequence must be in sync"); + } + } + + if (!sequenceHeader.IsReducedStillPictureHeader) + { + if (sequenceHeader.IsFrameIdNumbersPresent) + { + writer.WriteLiteral(frameHeader.CurrentFrameId, sequenceHeader.FrameIdLength); + } + + frameSizeOverrideFlag = frameHeader.FrameType == ObuFrameType.SwitchFrame || + frameHeader.FrameSize.SuperResolutionUpscaledWidth != sequenceHeader.MaxFrameWidth || + frameHeader.FrameSize.FrameHeight != sequenceHeader.MaxFrameHeight; + + if (frameHeader.FrameType != ObuFrameType.SwitchFrame) + { + writer.WriteBoolean(frameSizeOverrideFlag); + } + + writer.WriteLiteral(frameHeader.OrderHint, sequenceHeader.OrderHintInfo.OrderHintBits); + if (!frameHeader.ErrorResilientMode && !frameHeader.IsIntra) + { + writer.WriteLiteral(frameHeader.PrimaryReferenceFrame, Av1Constants.PrimaryReferenceBits); + } + } + + if (sequenceHeader.DecoderModelInfoPresentFlag) + { + // Image-sequence timing is carried by the container track, so encoded samples do not signal decoder-buffer removal times. + writer.WriteBoolean(false); + } + + if ((frameHeader.FrameType == ObuFrameType.KeyFrame && !frameHeader.ShowFrame) || + frameHeader.FrameType is ObuFrameType.InterFrame or ObuFrameType.IntraOnlyFrame) + { + writer.WriteLiteral(frameHeader.RefreshFrameFlags, Av1Constants.ReferenceFrameCount); + } + + if (frameHeader.FrameType == ObuFrameType.KeyFrame) + { + if (!frameHeader.ShowFrame) + { + throw new NotImplementedException("No support for hidden frames."); + } + } + else if (frameHeader.FrameType == ObuFrameType.IntraOnlyFrame) + { + throw new NotImplementedException("No IntraOnly frames supported."); + } + + if (frameHeader.FrameType == ObuFrameType.KeyFrame) + { + WriteFrameSize(ref writer, sequenceHeader, frameHeader, frameSizeOverrideFlag); + WriteRenderSize(ref writer, frameHeader); + if (frameHeader.AllowScreenContentTools) + { + writer.WriteBoolean(frameHeader.AllowIntraBlockCopy); + } + } + else if (frameHeader.FrameType == ObuFrameType.IntraOnlyFrame) + { + WriteFrameSize(ref writer, sequenceHeader, frameHeader, frameSizeOverrideFlag); + WriteRenderSize(ref writer, frameHeader); + if (frameHeader.AllowScreenContentTools) + { + writer.WriteBoolean(frameHeader.AllowIntraBlockCopy); + } + } + else + { + WriteReferenceFrameIndices(ref writer, sequenceHeader, frameHeader); + WriteFrameSize(ref writer, sequenceHeader, frameHeader, frameSizeOverrideFlag); + WriteRenderSize(ref writer, frameHeader); + if (!frameHeader.ForceIntegerMotionVector) + { + writer.WriteBoolean(frameHeader.AllowHighPrecisionMotionVector); + } + + WriteFrameInterpolationFilter(ref writer, frameHeader.InterpolationFilter); + writer.WriteBoolean(frameHeader.IsMotionModeSwitchable); + } + + bool mightAllowReferenceFrameMotionVectors = + !frameHeader.ErrorResilientMode && + sequenceHeader.OrderHintInfo.EnableReferenceFrameMotionVectors && + sequenceHeader.EnableOrderHint && + !frameHeader.IsIntra; + + if (mightAllowReferenceFrameMotionVectors) + { + writer.WriteBoolean(frameHeader.UseReferenceFrameMotionVectors); + } + + if (!sequenceHeader.IsReducedStillPictureHeader && !frameHeader.DisableCdfUpdate) + { + writer.WriteBoolean(frameHeader.DisableFrameEndUpdateCdf); + } + + WriteTileInfo(ref writer, sequenceHeader, frameHeader); + WriteQuantizationParameters(ref writer, sequenceHeader, frameHeader); + WriteSegmentationParameters(ref writer, frameHeader); + Av1QuantizationLookup.UpdateFrameQuantizationState(frameHeader); + + if (frameHeader.QuantizationParameters.BaseQIndex > 0) + { + writer.WriteBoolean(frameHeader.DeltaQParameters.IsPresent); + if (frameHeader.DeltaQParameters.IsPresent) + { + writer.WriteLiteral((uint)Av1Math.MostSignificantBit((uint)frameHeader.DeltaQParameters.Resolution), 2); + + if (frameHeader.AllowIntraBlockCopy) + { + Guard.IsFalse( + frameHeader.DeltaLoopFilterParameters.IsPresent, + nameof(frameHeader.DeltaLoopFilterParameters.IsPresent), + "Allow INTRA block copy required Loop Filter."); + } + else + { + writer.WriteBoolean(frameHeader.DeltaLoopFilterParameters.IsPresent); + } + + if (frameHeader.DeltaLoopFilterParameters.IsPresent) + { + writer.WriteLiteral((uint)Av1Math.MostSignificantBit((uint)frameHeader.DeltaLoopFilterParameters.Resolution), 2); + writer.WriteBoolean(frameHeader.DeltaLoopFilterParameters.IsMulti); + } + } + } + + if (!frameHeader.AllLossless) + { + if (!frameHeader.CodedLossless) + { + WriteLoopFilterParameters(ref writer, sequenceHeader, frameHeader); + if (sequenceHeader.EnableCdef) + { + WriteCdefParameters(ref writer, sequenceHeader, frameHeader); + } + } + + if (sequenceHeader.EnableRestoration) + { + WriteLoopRestorationParameters(ref writer, sequenceHeader, frameHeader); + } + } + + WriteTransformMode(ref writer, frameHeader); + + WriteFrameReferenceMode(ref writer, frameHeader); + WriteSkipModeParameters(ref writer, frameHeader); + if (!frameHeader.IsIntra && !frameHeader.ErrorResilientMode && sequenceHeader.EnableWarpedMotion) + { + writer.WriteBoolean(frameHeader.AllowWarpedMotion); + } + + writer.WriteBoolean(frameHeader.UseReducedTransformSet); + + WriteGlobalMotionParameters(ref writer, frameHeader); + WriteFilmGrainFilterParameters(ref writer, sequenceHeader, frameHeader); + } + + /// + /// Writes the seven reference-map slots selected by an inter frame. + /// + /// The bit writer receiving the reference indices. + /// The sequence header defining frame-ID and order-hint syntax. + /// The frame header containing the selected reference slots. + private static void WriteReferenceFrameIndices( + ref Av1BitStreamWriter writer, + ObuSequenceHeader sequenceHeader, + ObuFrameHeader frameHeader) + { + // Long signaling is deterministic and permits every reference role to select the same retained slot. + if (sequenceHeader.EnableOrderHint) + { + writer.WriteBoolean(false); + } + + Span referenceFrameIndices = frameHeader.GetReferenceFrameIndices(); + Span referenceFrameIds = frameHeader.GetReferenceFrameIds(); + uint frameIdModulus = 1U << sequenceHeader.FrameIdLength; + for (int reference = 0; reference < Av1Constants.ReferencesPerFrame; reference++) + { + uint slot = referenceFrameIndices[reference]; + writer.WriteLiteral(slot, Av1Constants.ReferenceFrameIndexBits); + if (sequenceHeader.IsFrameIdNumbersPresent) + { + uint deltaFrameId = (frameHeader.CurrentFrameId + frameIdModulus - referenceFrameIds[(int)slot]) % frameIdModulus; + writer.WriteLiteral(deltaFrameId - 1, sequenceHeader.DeltaFrameIdLength); + } + } + } + + /// + /// Writes the frame-level interpolation-filter selection. + /// + /// The bit writer receiving the filter selection. + /// The fixed filter family or per-block selection. + private static void WriteFrameInterpolationFilter(ref Av1BitStreamWriter writer, Av1InterpolationFilter filter) + { + bool isSwitchable = filter == Av1InterpolationFilter.Switchable; + writer.WriteBoolean(isSwitchable); + if (!isSwitchable) + { + writer.WriteLiteral((uint)filter, 2); + } + } + + /// + /// Writes the frame-header portion of a combined frame OBU. + /// + /// The bit writer receiving the frame header. + /// The sequence header controlling available coding tools. + /// The frame header to encode. + private static void WriteFrameHeader(ref Av1BitStreamWriter writer, ObuSequenceHeader sequenceHeader, ObuFrameHeader frameHeader) + { + WriteUncompressedFrameHeader(ref writer, sequenceHeader, frameHeader); + } + + /// + /// Writes the byte-aligned tile-group header for a combined frame OBU. + /// + /// The bit writer receiving the tile group. + /// The frame tile layout. + private static void WriteTileGroupHeader(ref Av1BitStreamWriter writer, ObuTileGroupHeader tileInfo) + { + int tileCount = tileInfo.TileColumnCount * tileInfo.TileRowCount; + + // libaom starts the tile-group header at the next byte after the uncompressed frame header. This + // boundary is required before the optional flag because the flag belongs to tile_group_obu syntax. + AlignToByteBoundary(ref writer); + + if (tileCount > 1) + { + // A combined frame always carries the complete raster tile group. The zero bit selects those implicit + // full-frame bounds instead of adding explicit start and end tile indices. + writer.WriteBoolean(false); + } + + AlignToByteBoundary(ref writer); + } + + /// + /// Writes the size-prefixed tile payloads in raster order. + /// + /// The non-boxed tile source type. + /// The destination stream receiving tile data. + /// The frame tile layout and tile-size field width. + /// The writer that produces each tile payload. + private static void WriteTileData( + Stream stream, + ObuTileGroupHeader tileInfo, + TTileWriter tileWriter) + where TTileWriter : IAv1TileWriter + { + int tileCount = tileInfo.TileColumnCount * tileInfo.TileRowCount; + Span tileSizeBuffer = stackalloc byte[sizeof(uint)]; + + for (int tileNum = 0; tileNum < tileCount; tileNum++) + { + ReadOnlySpan tileData = tileWriter.GetTileData(tileNum); + if (tileNum != tileCount - 1 && tileCount > 1) + { + // AV1 stores each non-final tile size minus one with the least-significant byte first. + BinaryPrimitives.WriteUInt32LittleEndian(tileSizeBuffer, (uint)tileData.Length - 1U); + stream.Write(tileSizeBuffer[..tileInfo.TileSizeBytes]); + } + + stream.Write(tileData); + } + } + + /// + /// Writes an optional signed quantizer-index delta. + /// + /// The bit writer receiving the delta. + /// The quantizer-index delta. + private static void WriteDeltaQ(ref Av1BitStreamWriter writer, int deltaQ) + { + bool isCoded = deltaQ != 0; + writer.WriteBoolean(isCoded); + if (isCoded) + { + writer.WriteSignedFromUnsigned(deltaQ, 7); + } + } + + /// + /// Writes the base index, plane deltas, and optional quantization matrices for a frame. + /// + /// The bit writer receiving the quantization parameters. + /// The sequence header defining active color planes. + /// The frame header containing the quantization parameters. + private static void WriteQuantizationParameters(ref Av1BitStreamWriter writer, ObuSequenceHeader sequenceHeader, ObuFrameHeader frameHeader) + { + ObuQuantizationParameters quantParams = frameHeader.QuantizationParameters; + writer.WriteLiteral((uint)quantParams.BaseQIndex, 8); + WriteDeltaQ(ref writer, quantParams.DeltaQDc[(int)Av1Plane.Y]); + if (sequenceHeader.ColorConfig.PlaneCount > 1) + { + if (sequenceHeader.ColorConfig.HasSeparateUvDelta) + { + writer.WriteBoolean(quantParams.HasSeparateUvDelta); + } + + WriteDeltaQ(ref writer, quantParams.DeltaQDc[(int)Av1Plane.U]); + WriteDeltaQ(ref writer, quantParams.DeltaQAc[(int)Av1Plane.U]); + if (quantParams.HasSeparateUvDelta) + { + WriteDeltaQ(ref writer, quantParams.DeltaQDc[(int)Av1Plane.V]); + WriteDeltaQ(ref writer, quantParams.DeltaQAc[(int)Av1Plane.V]); + } + } + + writer.WriteBoolean(quantParams.IsUsingQMatrix); + if (quantParams.IsUsingQMatrix) + { + writer.WriteLiteral((uint)quantParams.QMatrix[(int)Av1Plane.Y], 4); + writer.WriteLiteral((uint)quantParams.QMatrix[(int)Av1Plane.U], 4); + if (sequenceHeader.ColorConfig.HasSeparateUvDelta) + { + writer.WriteLiteral((uint)quantParams.QMatrix[(int)Av1Plane.V], 4); + } + } + } + + /// + /// Writes segmentation feature data for one coded frame. + /// + /// The bit writer receiving the segmentation parameters. + /// The frame header containing segmentation feature data. + private static void WriteSegmentationParameters(ref Av1BitStreamWriter writer, ObuFrameHeader frameHeader) + { + ObuSegmentationParameters segmentation = frameHeader.SegmentationParameters; + writer.WriteBoolean(segmentation.Enabled); + if (!segmentation.Enabled) + { + return; + } + + // A frame with no primary reference starts a new segmentation domain. AV1 therefore infers + // update-map and update-data as enabled and carries the complete feature state directly. + for (int segmentId = 0; segmentId < Av1Constants.MaxSegmentCount; segmentId++) + { + for (int featureId = 0; featureId < Av1Constants.SegmentationLevelMax; featureId++) + { + bool enabled = segmentation.IsFeatureActive(segmentId, (ObuSegmentationLevelFeature)featureId); + writer.WriteBoolean(enabled); + if (!enabled) + { + continue; + } + + int bitCount = Av1Constants.SegmentationFeatureBits[featureId]; + int value = segmentation.GetFeatureData(segmentId, featureId); + if (Av1Constants.SegmentationFeatureSigned[featureId] == 1) + { + writer.WriteSignedFromUnsigned(value, bitCount + 1); + } + else + { + writer.WriteLiteral((uint)value, bitCount); + } + } + } + } + + /// + /// Writes the deblocking-loop-filter levels and optional reference and mode deltas. + /// + /// The bit writer receiving the loop-filter parameters. + /// The sequence header defining active color planes. + /// The frame header containing the loop-filter parameters. + private static void WriteLoopFilterParameters(ref Av1BitStreamWriter writer, ObuSequenceHeader sequenceHeader, ObuFrameHeader frameHeader) + { + if (frameHeader.CodedLossless || frameHeader.AllowIntraBlockCopy) + { + return; + } + + writer.WriteLiteral((uint)frameHeader.LoopFilterParameters.FilterLevel[0], 6); + writer.WriteLiteral((uint)frameHeader.LoopFilterParameters.FilterLevel[1], 6); + if (sequenceHeader.ColorConfig.PlaneCount > 1) + { + if (frameHeader.LoopFilterParameters.FilterLevel[0] > 0 || frameHeader.LoopFilterParameters.FilterLevel[1] > 0) + { + writer.WriteLiteral((uint)frameHeader.LoopFilterParameters.FilterLevelU, 6); + writer.WriteLiteral((uint)frameHeader.LoopFilterParameters.FilterLevelV, 6); + } + } + + writer.WriteLiteral((uint)frameHeader.LoopFilterParameters.SharpnessLevel, 3); + writer.WriteBoolean(frameHeader.LoopFilterParameters.ReferenceDeltaModeEnabled); + if (frameHeader.LoopFilterParameters.ReferenceDeltaModeEnabled) + { + writer.WriteBoolean(frameHeader.LoopFilterParameters.ReferenceDeltaModeUpdate); + if (frameHeader.LoopFilterParameters.ReferenceDeltaModeUpdate) + { + // An independent still frame can emit every current delta as an update. This is + // slightly larger than comparing against retained state but requires no video + // reference-frame state and produces the same observable filter parameters. + for (int i = 0; i < Av1Constants.TotalReferencesPerFrame; i++) + { + writer.WriteBoolean(true); + writer.WriteSignedFromUnsigned(frameHeader.LoopFilterParameters.ReferenceDeltas[i], 7); + } + + for (int i = 0; i < 2; i++) + { + writer.WriteBoolean(true); + writer.WriteSignedFromUnsigned(frameHeader.LoopFilterParameters.ModeDeltas[i], 7); + } + } + } + } + + /// + /// Writes the transform-size selection mode when the frame is not lossless. + /// + /// The bit writer receiving the transform-mode flag. + /// The frame header containing the transform mode. + private static void WriteTransformMode(ref Av1BitStreamWriter writer, ObuFrameHeader frameHeader) + { + if (!frameHeader.CodedLossless) + { + writer.WriteBoolean(frameHeader.TransformMode == Av1TransformMode.Select); + } + } + + /// + /// Writes the loop-restoration type and restoration-unit size for each plane. + /// + /// The bit writer receiving the loop-restoration parameters. + /// The sequence header defining restoration availability and color planes. + /// The frame header containing restoration parameters. + private static void WriteLoopRestorationParameters(ref Av1BitStreamWriter writer, ObuSequenceHeader sequenceHeader, ObuFrameHeader frameHeader) + { + if (frameHeader.CodedLossless || frameHeader.AllowIntraBlockCopy || !sequenceHeader.EnableRestoration) + { + return; + } + + int planesCount = sequenceHeader.ColorConfig.PlaneCount; + for (int i = 0; i < planesCount; i++) + { + writer.WriteLiteral((uint)frameHeader.LoopRestorationParameters.Items[i].Type, 2); + } + + if (frameHeader.LoopRestorationParameters.UsesLoopRestoration) + { + uint unitShift = (uint)frameHeader.LoopRestorationParameters.UnitShift; + if (sequenceHeader.Use128x128Superblock) + { + writer.WriteLiteral(unitShift - 1, 1); + } + else + { + writer.WriteBoolean(unitShift > 0); + if (unitShift > 0) + { + writer.WriteLiteral(unitShift - 1, 1); + } + } + + if (sequenceHeader.ColorConfig.SubSamplingX && sequenceHeader.ColorConfig.SubSamplingY && frameHeader.LoopRestorationParameters.UsesChromaLoopRestoration) + { + writer.WriteLiteral((uint)frameHeader.LoopRestorationParameters.UVShift, 1); + } + } + } + + /// + /// Writes constrained directional enhancement filter strengths for the active planes. + /// + /// The bit writer receiving the CDEF parameters. + /// The sequence header defining CDEF availability and color planes. + /// The frame header containing CDEF strengths. + private static void WriteCdefParameters(ref Av1BitStreamWriter writer, ObuSequenceHeader sequenceHeader, ObuFrameHeader frameHeader) + { + if (frameHeader.CodedLossless || frameHeader.AllowIntraBlockCopy || !sequenceHeader.EnableCdef) + { + return; + } + + ObuConstraintDirectionalEnhancementFilterParameters cdef = frameHeader.CdefParameters; + writer.WriteLiteral((uint)cdef.Damping - 3, 2); + writer.WriteLiteral((uint)cdef.BitCount, 2); + int strengthCount = 1 << cdef.BitCount; + bool hasChroma = sequenceHeader.ColorConfig.PlaneCount > 1; + for (int i = 0; i < strengthCount; i++) + { + writer.WriteLiteral((uint)cdef.YStrength[i], 6); + if (hasChroma) + { + writer.WriteLiteral((uint)cdef.UvStrength[i], 6); + } + } + } + + /// + /// Writes global-motion parameters when permitted by the frame type. + /// + /// The bit writer positioned at the global-motion syntax. + /// The current frame header. + private static void WriteGlobalMotionParameters(ref Av1BitStreamWriter writer, ObuFrameHeader frameHeader) + { + if (frameHeader.IsIntra) + { + return; + } + + ReadOnlySpan parameters = frameHeader.GetGlobalMotionParameters(); + Av1GlobalMotionParameters referenceParameters = Av1GlobalMotionParameters.Identity; + for (int reference = 0; reference < Av1Constants.ReferencesPerFrame; reference++) + { + WriteGlobalMotionModel( + ref writer, + parameters[reference], + referenceParameters, + frameHeader.AllowHighPrecisionMotionVector); + } + } + + /// + /// Writes one global-motion model relative to the same-role model in the primary reference frame. + /// + private static void WriteGlobalMotionModel( + ref Av1BitStreamWriter writer, + Av1GlobalMotionParameters parameters, + Av1GlobalMotionParameters referenceParameters, + bool allowHighPrecisionMotionVector) + { + Av1GlobalMotionType type = parameters.Type; + writer.WriteBoolean(type != Av1GlobalMotionType.Identity); + if (type != Av1GlobalMotionType.Identity) + { + writer.WriteBoolean(type == Av1GlobalMotionType.RotationZoom); + if (type != Av1GlobalMotionType.RotationZoom) + { + writer.WriteBoolean(type == Av1GlobalMotionType.Translation); + } + } + + if (type >= Av1GlobalMotionType.RotationZoom) + { + int horizontalScale = + (parameters[2] >> Av1GlobalMotionParameters.AlphaPrecisionDifference) - + (1 << Av1GlobalMotionParameters.AlphaPrecisionBits); + + int referenceHorizontalScale = + (referenceParameters[2] >> Av1GlobalMotionParameters.AlphaPrecisionDifference) - + (1 << Av1GlobalMotionParameters.AlphaPrecisionBits); + + writer.WriteSignedReferenceSubexponential( + horizontalScale, + Av1GlobalMotionParameters.AlphaValueMagnitude, + Av1GlobalMotionParameters.SubexponentialGroupBitCount, + referenceHorizontalScale); + + writer.WriteSignedReferenceSubexponential( + parameters[3] >> Av1GlobalMotionParameters.AlphaPrecisionDifference, + Av1GlobalMotionParameters.AlphaValueMagnitude, + Av1GlobalMotionParameters.SubexponentialGroupBitCount, + referenceParameters[3] >> Av1GlobalMotionParameters.AlphaPrecisionDifference); + } + + if (type >= Av1GlobalMotionType.Affine) + { + int verticalScale = + (parameters[5] >> Av1GlobalMotionParameters.AlphaPrecisionDifference) - + (1 << Av1GlobalMotionParameters.AlphaPrecisionBits); + + int referenceVerticalScale = + (referenceParameters[5] >> Av1GlobalMotionParameters.AlphaPrecisionDifference) - + (1 << Av1GlobalMotionParameters.AlphaPrecisionBits); + + writer.WriteSignedReferenceSubexponential( + parameters[4] >> Av1GlobalMotionParameters.AlphaPrecisionDifference, + Av1GlobalMotionParameters.AlphaValueMagnitude, + Av1GlobalMotionParameters.SubexponentialGroupBitCount, + referenceParameters[4] >> Av1GlobalMotionParameters.AlphaPrecisionDifference); + + writer.WriteSignedReferenceSubexponential( + verticalScale, + Av1GlobalMotionParameters.AlphaValueMagnitude, + Av1GlobalMotionParameters.SubexponentialGroupBitCount, + referenceVerticalScale); + } + + if (type >= Av1GlobalMotionType.Translation) + { + int precisionAdjustment = + type == Av1GlobalMotionType.Translation && !allowHighPrecisionMotionVector ? 1 : 0; + + int translationBits = type == Av1GlobalMotionType.Translation + ? Av1GlobalMotionParameters.AbsoluteTranslationOnlyBits - precisionAdjustment + : Av1GlobalMotionParameters.AbsoluteTranslationBits; + + int translationPrecisionDifference = type == Av1GlobalMotionType.Translation + ? Av1GlobalMotionParameters.ModelPrecisionBits - + Av1GlobalMotionParameters.TranslationOnlyPrecisionBits + + precisionAdjustment + : Av1GlobalMotionParameters.ModelPrecisionBits - + Av1GlobalMotionParameters.TranslationPrecisionBits; + + int translationValueMagnitude = (1 << translationBits) + 1; + writer.WriteSignedReferenceSubexponential( + parameters[0] >> translationPrecisionDifference, + translationValueMagnitude, + Av1GlobalMotionParameters.SubexponentialGroupBitCount, + referenceParameters[0] >> translationPrecisionDifference); + + writer.WriteSignedReferenceSubexponential( + parameters[1] >> translationPrecisionDifference, + translationValueMagnitude, + Av1GlobalMotionParameters.SubexponentialGroupBitCount, + referenceParameters[1] >> translationPrecisionDifference); + } + } + + /// + /// Gets the exact number of uncompressed-header bits required by one global-motion model. + /// + /// The model to measure. + /// Whether translation may retain one-eighth-sample precision. + /// The encoded model length in bits. + internal static int GetGlobalMotionModelBitCount( + Av1GlobalMotionParameters parameters, + bool allowHighPrecisionMotionVector) + { + InlineArray16 storage = default; + Span buffer = storage; + Av1BitStreamWriter writer = new(buffer); + WriteGlobalMotionModel( + ref writer, + parameters, + Av1GlobalMotionParameters.Identity, + allowHighPrecisionMotionVector); + + return writer.BitPosition; + } + + /// + /// Writes reference-mode selection when permitted by the frame type. + /// + /// The bit writer positioned at the reference-mode syntax. + /// The current frame header. + private static void WriteFrameReferenceMode(ref Av1BitStreamWriter writer, ObuFrameHeader frameHeader) + { + if (frameHeader.IsIntra) + { + return; + } + + writer.WriteBoolean(frameHeader.ReferenceMode == ObuReferenceMode.ReferenceModeSelect); + } + + /// + /// Writes the skip-mode flag when skip mode is available. + /// + /// The bit writer receiving the skip-mode flag. + /// The frame header containing skip-mode state. + private static void WriteSkipModeParameters(ref Av1BitStreamWriter writer, ObuFrameHeader frameHeader) + { + if (frameHeader.SkipModeParameters.SkipModeAllowed) + { + writer.WriteBoolean(frameHeader.SkipModeParameters.SkipModeFlag); + } + } + + /// + /// Writes film-grain synthesis parameters for a displayed frame. + /// + /// The bit writer receiving the film-grain parameters. + /// The sequence header defining film-grain availability and color sampling. + /// The frame header containing film-grain parameters. + private static void WriteFilmGrainFilterParameters(ref Av1BitStreamWriter writer, ObuSequenceHeader sequenceHeader, ObuFrameHeader frameHeader) + { + ObuFilmGrainParameters grainParams = frameHeader.FilmGrainParameters; + if (!sequenceHeader.AreFilmGrainingParametersPresent || (!frameHeader.ShowFrame && !frameHeader.ShowableFrame)) + { + return; + } + + writer.WriteBoolean(grainParams.ApplyGrain); + if (!grainParams.ApplyGrain) + { + return; + } + + writer.WriteLiteral(grainParams.GrainSeed, 16); + writer.WriteLiteral(grainParams.NumYPoints, 4); + for (int i = 0; i < grainParams.NumYPoints; i++) + { + writer.WriteLiteral(grainParams.PointYValue[i], 8); + writer.WriteLiteral(grainParams.PointYScaling[i], 8); + } + + if (!sequenceHeader.ColorConfig.IsMonochrome) + { + writer.WriteBoolean(grainParams.ChromaScalingFromLuma); + } + + if (!sequenceHeader.ColorConfig.IsMonochrome && + !grainParams.ChromaScalingFromLuma && + (!sequenceHeader.ColorConfig.SubSamplingX || !sequenceHeader.ColorConfig.SubSamplingY || grainParams.NumYPoints != 0)) + { + writer.WriteLiteral(grainParams.NumCbPoints, 4); + for (int i = 0; i < grainParams.NumCbPoints; i++) + { + writer.WriteLiteral(grainParams.PointCbValue[i], 8); + writer.WriteLiteral(grainParams.PointCbScaling[i], 8); + } + + writer.WriteLiteral(grainParams.NumCrPoints, 4); + for (int i = 0; i < grainParams.NumCrPoints; i++) + { + writer.WriteLiteral(grainParams.PointCrValue[i], 8); + writer.WriteLiteral(grainParams.PointCrScaling[i], 8); + } + } + + writer.WriteLiteral(grainParams.GrainScalingMinus8, 2); + writer.WriteLiteral(grainParams.ArCoeffLag, 2); + uint numPosLuma = 2 * grainParams.ArCoeffLag * (grainParams.ArCoeffLag + 1); + + uint numPosChroma = numPosLuma; + if (grainParams.NumYPoints != 0) + { + numPosChroma++; + for (int i = 0; i < numPosLuma; i++) + { + writer.WriteLiteral(grainParams.ArCoeffsYPlus128[i], 8); + } + } + + if (grainParams.ChromaScalingFromLuma || grainParams.NumCbPoints != 0) + { + for (int i = 0; i < numPosChroma; i++) + { + writer.WriteLiteral(grainParams.ArCoeffsCbPlus128[i], 8); + } + } + + if (grainParams.ChromaScalingFromLuma || grainParams.NumCrPoints != 0) + { + for (int i = 0; i < numPosChroma; i++) + { + writer.WriteLiteral(grainParams.ArCoeffsCrPlus128[i], 8); + } + } + + writer.WriteLiteral(grainParams.ArCoeffShiftMinus6, 2); + writer.WriteLiteral(grainParams.GrainScaleShift, 2); + if (grainParams.NumCbPoints != 0) + { + writer.WriteLiteral(grainParams.CbMult, 8); + writer.WriteLiteral(grainParams.CbLumaMult, 8); + writer.WriteLiteral(grainParams.CbOffset, 9); + } + + if (grainParams.NumCrPoints != 0) + { + writer.WriteLiteral(grainParams.CrMult, 8); + writer.WriteLiteral(grainParams.CrLumaMult, 8); + writer.WriteLiteral(grainParams.CrOffset, 9); + } + + writer.WriteBoolean(grainParams.OverlapFlag); + writer.WriteBoolean(grainParams.ClipToRestrictedRange); + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1EncoderBlockWorkspace.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1EncoderBlockWorkspace.cs new file mode 100644 index 0000000000..62e982c752 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1EncoderBlockWorkspace.cs @@ -0,0 +1,225 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Buffers; +using System.Runtime.InteropServices; +using SixLabors.ImageSharp.Formats.Heif.Av1.Motion; +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; +using SixLabors.ImageSharp.Memory; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline; + +/// +/// Owns the reusable spatial, transform, and reconstruction storage for AV1 block encoding. +/// +internal sealed class Av1EncoderBlockWorkspace : IDisposable +{ + /// + /// The maximum number of spatial residual samples in one AV1 transform block. + /// + public const int MaximumResidualCount = Av1Constants.MaxTransformSize * Av1Constants.MaxTransformSize; + + /// + /// The maximum number of coded coefficients after AV1 removes the uncoded half of 64-point axes. + /// + public const int MaximumCoefficientCount = (Av1Constants.MaxTransformSize / 2) * (Av1Constants.MaxTransformSize / 2); + + /// + /// The complete workspace length in signed-integer storage elements. + /// + public const int StorageLength = + ResidualStorageLength + + MaximumCoefficientCount + + MaximumCoefficientCount + + Av1TransformWorkspace.MaximumLength + + SharedModeDecisionStorageLength + + PartitionContextStorageLength; + + private const int ResidualStorageLength = MaximumResidualCount / 2; + private const int TransformCoefficientOffset = ResidualStorageLength; + private const int DequantizedCoefficientOffset = TransformCoefficientOffset + MaximumCoefficientCount; + private const int TransformWorkspaceOffset = DequantizedCoefficientOffset + MaximumCoefficientCount; + private const int InterPredictionSampleStorageOffset = TransformWorkspaceOffset + Av1TransformWorkspace.MaximumLength; + private const int InterPredictionSampleStorageLength = + Av1EncoderInterPredictionWorkspace.SampleBufferCount * + Av1EncoderInterPredictionWorkspace.MaximumSampleCount * + sizeof(ushort) / + sizeof(int); + + private const int InterPredictionResidualStorageOffset = + InterPredictionSampleStorageOffset + InterPredictionSampleStorageLength; + + private const int InterPredictionResidualStorageLength = + Av1EncoderInterPredictionWorkspace.MaximumSampleCount * + sizeof(short) / + sizeof(int); + + private const int InterPredictionScratchStorageOffset = + InterPredictionResidualStorageOffset + InterPredictionResidualStorageLength; + + private const int InterPredictionScratchStorageLength = + Av1EncoderInterPredictionWorkspace.PredictionScratchCount * + sizeof(short) / + sizeof(int); + + private const int InterPredictionCoefficientStorageOffset = + InterPredictionScratchStorageOffset + InterPredictionScratchStorageLength; + + private const int InterPredictionCoefficientStorageLength = + Av1EncoderInterPredictionWorkspace.CoefficientBufferCount * + Av1EncoderInterPredictionWorkspace.MaximumSampleCount; + + private const int InterPredictionStorageLength = + InterPredictionSampleStorageLength + + InterPredictionResidualStorageLength + + InterPredictionScratchStorageLength + + InterPredictionCoefficientStorageLength; + + private const int ModeDecisionStorageLength = Av1EncoderModeDecisionWorkspace.StorageLength; + private const int SharedModeDecisionStorageLength = ModeDecisionStorageLength > InterPredictionStorageLength + ? ModeDecisionStorageLength + : InterPredictionStorageLength; + + private const int PartitionContextStorageOffset = + InterPredictionSampleStorageOffset + SharedModeDecisionStorageLength; + + private const int MaximumPartitionEdgeUnitCount = + 2 * (1 << (Av1Constants.MaxSuperBlockSizeLog2 - Av1Constants.ModeInfoSizeLog2)); + + private const int PartitionContextBytesPerEdgeUnit = + Av1PartitionContext.StorageSize + (4 * sizeof(byte)) + Av1EncoderPaletteInfo.StorageSize; + + private const int PartitionContextSlotByteLength = + MaximumPartitionEdgeUnitCount * PartitionContextBytesPerEdgeUnit; + + private const int PartitionContextSlotLength = + PartitionContextSlotByteLength / sizeof(int); + + private const int PartitionTrialLevelCount = + Av1Constants.MaxSuperBlockSizeLog2 - 3 + 1; + + private const int PartitionContextStorageLength = + PartitionContextSlotLength * PartitionTrialLevelCount; + + /// + /// Owns the complete reusable block workspace in 32-bit elements so every transform region is naturally aligned. + /// + private readonly IMemoryOwner owner; + + /// + /// Reuses the fixed-capacity reference-vector stack for every inter block in the frame. + /// + private Av1ReferenceMotionVectors referenceMotionVectors; + + /// + /// Initializes a new instance of the class. + /// + /// The configuration providing the encoder allocator. + public Av1EncoderBlockWorkspace(Configuration configuration) + => this.owner = configuration.MemoryAllocator.Allocate(StorageLength); + + /// + /// Gets the maximum-size spatial residual workspace as a compact 16-bit view of the aligned owner. + /// + public Span Residual + => MemoryMarshal.Cast(this.owner.Memory.Span[..ResidualStorageLength]); + + /// + /// Gets the maximum-size forward-transform coefficient workspace. + /// + public Span TransformCoefficients + => this.owner.Memory.Span.Slice(TransformCoefficientOffset, MaximumCoefficientCount); + + /// + /// Gets the maximum-size dequantized reconstruction coefficient workspace. + /// + public Span DequantizedCoefficients + => this.owner.Memory.Span.Slice(DequantizedCoefficientOffset, MaximumCoefficientCount); + + /// + /// Gets the reusable two-dimensional transform workspace. + /// + public Span TransformWorkspace + => this.owner.Memory.Span.Slice(TransformWorkspaceOffset, Av1TransformWorkspace.MaximumLength); + + /// + /// Gets the reusable reference-vector stack used by inter mode decision and syntax writing. + /// + public ref Av1ReferenceMotionVectors ReferenceMotionVectors => ref this.referenceMotionVectors; + + /// + /// Gets the disjoint edge snapshot used to restore one square partition-search level. + /// + /// The square partition node being evaluated. + /// The maximum-size byte view reserved for that node depth. + public Span GetPartitionContextStorage(Av1BlockSize blockSize) + { + int blockSizeLog2 = Av1Math.Log2(blockSize.GetWidth()); + int slotIndex = Av1Constants.MaxSuperBlockSizeLog2 - blockSizeLog2; + Span storage = this.owner.Memory.Span.Slice( + PartitionContextStorageOffset + (slotIndex * PartitionContextSlotLength), + PartitionContextSlotLength); + + return MemoryMarshal.AsBytes(storage); + } + + /// + /// Gets the reusable storage used while comparing spatial, chroma-from-luma, filter-intra, and palette candidates. + /// + /// The native sample type selected by the encoder pipeline. + /// The typed mode-decision workspace. + public Av1EncoderModeDecisionWorkspace GetModeDecisionWorkspace() + where TSample : unmanaged + { + // Conventional intra search finishes before reference prediction begins for the same block. + // Both phases can therefore reuse this aligned region without extending the owner or preserving stale scratch. + Span storage = this.owner.Memory.Span.Slice( + InterPredictionSampleStorageOffset, + SharedModeDecisionStorageLength); + + return new(storage[..Av1EncoderModeDecisionWorkspace.StorageLength]); + } + + /// + /// Gets the reusable storage used while comparing single-reference or intra-block-copy candidates. + /// + /// The native sample type selected by the encoder pipeline. + /// The typed inter-prediction workspace. + public Av1EncoderInterPredictionWorkspace GetInterPredictionWorkspace() + where TSample : unmanaged + { + Span storage = this.owner.Memory.Span; + Span sampleStorage = MemoryMarshal + .Cast(storage.Slice(InterPredictionSampleStorageOffset, InterPredictionSampleStorageLength)); + + sampleStorage = sampleStorage[ + ..(Av1EncoderInterPredictionWorkspace.SampleBufferCount * + Av1EncoderInterPredictionWorkspace.MaximumSampleCount)]; + + Span residualStorage = MemoryMarshal + .Cast(storage.Slice(InterPredictionResidualStorageOffset, InterPredictionResidualStorageLength)); + + residualStorage = residualStorage[..Av1EncoderInterPredictionWorkspace.MaximumSampleCount]; + + Span predictionScratch = MemoryMarshal + .Cast(storage.Slice(InterPredictionScratchStorageOffset, InterPredictionScratchStorageLength)); + + predictionScratch = predictionScratch[..Av1EncoderInterPredictionWorkspace.PredictionScratchCount]; + + Span coefficientStorage = storage.Slice( + InterPredictionCoefficientStorageOffset, + InterPredictionCoefficientStorageLength); + + return new Av1EncoderInterPredictionWorkspace( + sampleStorage, + residualStorage, + predictionScratch, + coefficientStorage); + } + + /// + /// Releases the reusable block workspace. + /// + public void Dispose() => this.owner.Dispose(); +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1EncoderFrame.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1EncoderFrame.cs new file mode 100644 index 0000000000..3f63afff57 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1EncoderFrame.cs @@ -0,0 +1,395 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Components; +using SixLabors.ImageSharp.Memory; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline; + +/// +/// Provides non-owning visible and coded views over operation-scoped AV1 component planes. +/// +/// The native unsigned sample storage type. +internal readonly struct Av1EncoderFrame + where TSample : unmanaged +{ + /// + /// The base-two alignment exponent applied to coded frame dimensions. + /// + private const int CodedDimensionAlignmentLog2 = 3; + + /// + /// The base-two alignment exponent applied to the physical luma row stride. + /// + private const int LumaStrideAlignmentLog2 = 5; + + /// + /// The physical luma border required by non-resized all-intra encoding. + /// + public const int LumaBorder = 64; + + /// + /// Initializes a new instance of the struct for a monochrome frame. + /// + /// The coded luma region inside the bordered plane owned by the encode operation. + /// The visible luma width. + /// The visible luma height. + /// The native component precision. + public Av1EncoderFrame(Buffer2DRegion luma, int width, int height, int bitDepth) + : this(luma, default, default, width, height, bitDepth, Av1ColorFormat.Yuv400, 0, 0) + { + } + + /// + /// Initializes a new instance of the struct for a color frame. + /// + /// The coded luma region inside the bordered plane owned by the encode operation. + /// The coded blue-difference region inside its bordered plane. + /// The coded red-difference region inside its bordered plane. + /// The visible luma width. + /// The visible luma height. + /// The native component precision. + /// The native luma and chroma sampling layout. + /// The horizontal chroma position in half-luma-sample units. + /// The vertical chroma position in half-luma-sample units. + public Av1EncoderFrame( + Buffer2DRegion luma, + Buffer2DRegion chromaBlue, + Buffer2DRegion chromaRed, + int width, + int height, + int bitDepth, + Av1ColorFormat colorFormat, + int chromaPositionX, + int chromaPositionY) + { + this.Width = width; + this.Height = height; + this.LumaBitDepth = bitDepth; + this.ChromaBitDepth = bitDepth; + this.IsMonochrome = colorFormat == Av1ColorFormat.Yuv400; + this.ChromaSubsamplingX = colorFormat is Av1ColorFormat.Yuv420 or Av1ColorFormat.Yuv422 ? 1 : 0; + this.ChromaSubsamplingY = colorFormat == Av1ColorFormat.Yuv420 ? 1 : 0; + this.ChromaPositionX = chromaPositionX; + this.ChromaPositionY = chromaPositionY; + this.CodedWidth = luma.Width; + this.CodedHeight = luma.Height; + int visibleChromaWidth = (width + this.ChromaSubsamplingX) >> this.ChromaSubsamplingX; + int visibleChromaHeight = (height + this.ChromaSubsamplingY) >> this.ChromaSubsamplingY; + Buffer2DRegion visibleChromaBlue = this.IsMonochrome + ? default + : chromaBlue.GetSubRegion(0, 0, visibleChromaWidth, visibleChromaHeight); + + Buffer2DRegion visibleChromaRed = this.IsMonochrome + ? default + : chromaRed.GetSubRegion(0, 0, visibleChromaWidth, visibleChromaHeight); + + this.View = new PlanarView( + luma.GetSubRegion(0, 0, width, height), + visibleChromaBlue, + visibleChromaRed, + width, + height, + bitDepth, + colorFormat, + chromaPositionX, + chromaPositionY); + + this.CodedView = new PlanarView( + luma, + chromaBlue, + chromaRed, + this.CodedWidth, + this.CodedHeight, + bitDepth, + colorFormat, + chromaPositionX, + chromaPositionY); + } + + /// + /// Gets the writable component-plane view used by closed generic conversion and coding operations. + /// + public PlanarView View { get; } + + /// + /// Gets the writable coded component planes used by block coding and reconstruction. + /// + public PlanarView CodedView { get; } + + /// + /// Gets the visible luma width. + /// + public int Width { get; } + + /// + /// Gets the visible luma height. + /// + public int Height { get; } + + /// + /// Gets the luma width rounded up to the fixed coding-block boundary. + /// + public int CodedWidth { get; } + + /// + /// Gets the luma height rounded up to the fixed coding-block boundary. + /// + public int CodedHeight { get; } + + /// + /// Gets the native luma sample precision. + /// + public int LumaBitDepth { get; } + + /// + /// Gets the native chroma sample precision. + /// + public int ChromaBitDepth { get; } + + /// + /// Gets a value indicating whether the frame contains only luma samples. + /// + public bool IsMonochrome { get; } + + /// + /// Gets the horizontal chroma subsampling shift. + /// + public int ChromaSubsamplingX { get; } + + /// + /// Gets the vertical chroma subsampling shift. + /// + public int ChromaSubsamplingY { get; } + + /// + /// Gets the horizontal chroma position in half-luma-sample units. + /// + public int ChromaPositionX { get; } + + /// + /// Gets the vertical chroma position in half-luma-sample units. + /// + public int ChromaPositionY { get; } + + /// + /// Calculates the coded luma dimensions used by the fixed all-intra frame layout. + /// + /// The visible luma width. + /// The visible luma height. + /// The visible dimensions rounded up to the coding alignment. + public static Size GetCodedSize(int width, int height) + => new( + Av1Math.AlignPowerOf2(width, CodedDimensionAlignmentLog2), + Av1Math.AlignPowerOf2(height, CodedDimensionAlignmentLog2)); + + /// + /// Calculates the physical dimensions required for an all-intra component plane. + /// + /// The visible luma width. + /// The visible luma height. + /// The plane's horizontal subsampling shift. + /// The plane's vertical subsampling shift. + /// The physical plane dimensions, including its complete border and row padding. + public static Size GetPlaneBufferSize(int width, int height, int subsamplingX, int subsamplingY) + { + Size codedSize = GetCodedSize(width, height); + + // libaom aligns the complete luma row before deriving a subsampled plane's stride. + // Aligning chroma independently would produce a different physical layout for narrow or odd-sized frames. + int lumaStride = Av1Math.AlignPowerOf2(codedSize.Width + (2 * LumaBorder), LumaStrideAlignmentLog2); + int planeStride = lumaStride >> subsamplingX; + int planeBorderHeight = LumaBorder >> subsamplingY; + return new Size(planeStride, (codedSize.Height >> subsamplingY) + (2 * planeBorderHeight)); + } + + /// + /// Extends the visible edge samples through the coded padding. + /// + public void ExtendBorders() + => this.CodedView.ExtendBorders(this.Width, this.Height); + + /// + /// Replicates the visible edge samples through a plane's complete physical border. + /// + private static void ExtendPlane(Buffer2DRegion plane, int visibleWidth, int visibleHeight) + { + Buffer2D buffer = plane.Buffer; + Rectangle bounds = plane.Bounds; + for (int y = 0; y < visibleHeight; y++) + { + Span row = buffer.DangerousGetRowSpan(bounds.Y + y); + + // libaom fills both physical borders and the right-hand coded alignment from the nearest visible sample. + row[..bounds.X].Fill(row[bounds.X]); + row[(bounds.X + visibleWidth)..].Fill(row[bounds.X + visibleWidth - 1]); + } + + // Horizontal extension runs first so copying the first and last visible rows also initializes both corners. + ReadOnlySpan firstVisibleRow = buffer.DangerousGetRowSpan(bounds.Y); + for (int y = 0; y < bounds.Y; y++) + { + firstVisibleRow.CopyTo(buffer.DangerousGetRowSpan(y)); + } + + ReadOnlySpan finalVisibleRow = buffer.DangerousGetRowSpan(bounds.Y + visibleHeight - 1); + for (int y = bounds.Y + visibleHeight; y < buffer.Height; y++) + { + finalVisibleRow.CopyTo(buffer.DangerousGetRowSpan(y)); + } + } + + /// + /// Provides a non-owning component-plane view for generic hot-path operations. + /// + internal readonly struct PlanarView : IHeifPlanarSampleBuffer + { + /// + /// The writable luma plane. + /// + private readonly Buffer2DRegion luma; + + /// + /// The writable blue-difference plane, or the default region for monochrome frames. + /// + private readonly Buffer2DRegion chromaBlue; + + /// + /// The writable red-difference plane, or the default region for monochrome frames. + /// + private readonly Buffer2DRegion chromaRed; + + /// + /// Initializes a new instance of the struct. + /// + public PlanarView( + Buffer2DRegion luma, + Buffer2DRegion chromaBlue, + Buffer2DRegion chromaRed, + int width, + int height, + int bitDepth, + Av1ColorFormat colorFormat, + int chromaPositionX, + int chromaPositionY) + { + this.luma = luma; + this.chromaBlue = chromaBlue; + this.chromaRed = chromaRed; + this.Width = width; + this.Height = height; + this.LumaBitDepth = bitDepth; + this.ChromaBitDepth = bitDepth; + this.IsMonochrome = colorFormat == Av1ColorFormat.Yuv400; + this.ChromaSubsamplingX = colorFormat is Av1ColorFormat.Yuv420 or Av1ColorFormat.Yuv422 ? 1 : 0; + this.ChromaSubsamplingY = colorFormat == Av1ColorFormat.Yuv420 ? 1 : 0; + this.ChromaPositionX = chromaPositionX; + this.ChromaPositionY = chromaPositionY; + } + + /// + public int Width { get; } + + /// + public int Height { get; } + + /// + public int LumaBitDepth { get; } + + /// + public int ChromaBitDepth { get; } + + /// + public bool IsMonochrome { get; } + + /// + public int ChromaSubsamplingX { get; } + + /// + public int ChromaSubsamplingY { get; } + + /// + public int ChromaPositionX { get; } + + /// + public int ChromaPositionY { get; } + + /// + /// Replicates the visible component edges through the coded padding. + /// + /// The visible luma width. + /// The visible luma height. + public void ExtendBorders(int visibleWidth, int visibleHeight) + { + ExtendPlane(this.luma, visibleWidth, visibleHeight); + + if (!this.IsMonochrome) + { + int visibleChromaWidth = (visibleWidth + this.ChromaSubsamplingX) >> this.ChromaSubsamplingX; + int visibleChromaHeight = (visibleHeight + this.ChromaSubsamplingY) >> this.ChromaSubsamplingY; + ExtendPlane(this.chromaBlue, visibleChromaWidth, visibleChromaHeight); + ExtendPlane(this.chromaRed, visibleChromaWidth, visibleChromaHeight); + } + } + + /// + /// Gets a writable coded component plane. + /// + /// The requested component plane. + /// The complete coded plane region. + public Buffer2DRegion GetPlane(Av1Plane plane) + => plane switch + { + Av1Plane.Y => this.luma, + Av1Plane.U => this.chromaBlue, + _ => this.chromaRed + }; + + /// + /// Gets a top-left view whose visible dimensions can be smaller than the backing coded planes. + /// + /// The visible luma width. + /// The visible luma height. + /// The requested non-owning planar view. + public PlanarView GetSubView(int width, int height) + { + Av1ColorFormat colorFormat = this.IsMonochrome + ? Av1ColorFormat.Yuv400 + : this.ChromaSubsamplingX == 0 + ? Av1ColorFormat.Yuv444 + : this.ChromaSubsamplingY == 0 + ? Av1ColorFormat.Yuv422 + : Av1ColorFormat.Yuv420; + + int chromaWidth = (width + this.ChromaSubsamplingX) >> this.ChromaSubsamplingX; + int chromaHeight = (height + this.ChromaSubsamplingY) >> this.ChromaSubsamplingY; + Buffer2DRegion blue = this.IsMonochrome + ? default + : this.chromaBlue.GetSubRegion(0, 0, chromaWidth, chromaHeight); + + Buffer2DRegion red = this.IsMonochrome + ? default + : this.chromaRed.GetSubRegion(0, 0, chromaWidth, chromaHeight); + + return new PlanarView( + this.luma.GetSubRegion(0, 0, width, height), + blue, + red, + width, + height, + this.LumaBitDepth, + colorFormat, + this.ChromaPositionX, + this.ChromaPositionY); + } + + /// + public Span GetLumaRowSpan(int row) => this.luma.DangerousGetRowSpan(row); + + /// + public Span GetChromaBlueRowSpan(int row) => this.chromaBlue.DangerousGetRowSpan(row); + + /// + public Span GetChromaRedRowSpan(int row) => this.chromaRed.DangerousGetRowSpan(row); + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1EncoderFrameBuffer.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1EncoderFrameBuffer.cs new file mode 100644 index 0000000000..1742b94ed0 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1EncoderFrameBuffer.cs @@ -0,0 +1,170 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Buffers; +using System.Runtime.CompilerServices; +using SixLabors.ImageSharp.Memory; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline; + +/// +/// Owns the aligned luma and chroma planes used by one AV1 encoder frame. +/// +/// The native unsigned sample storage type. +internal sealed class Av1EncoderFrameBuffer : IDisposable + where TSample : unmanaged +{ + /// + /// The byte boundary used by libaom for SIMD-accessible component planes. + /// + private const int PlaneAlignmentBytes = 32; + + /// + /// The complete frame owner, or after disposal. + /// + private IMemoryOwner? owner; + + /// + /// Initializes a new instance of the class. + /// + /// The configuration providing the frame allocator. + /// The visible luma width. + /// The visible luma height. + /// The native component precision. + /// The native luma and chroma sampling layout. + /// The horizontal chroma position in half-luma-sample units. + /// The vertical chroma position in half-luma-sample units. + public Av1EncoderFrameBuffer( + Configuration configuration, + int width, + int height, + int bitDepth, + Av1ColorFormat colorFormat, + int chromaPositionX, + int chromaPositionY) + { + int subsamplingX = colorFormat is Av1ColorFormat.Yuv420 or Av1ColorFormat.Yuv422 ? 1 : 0; + int subsamplingY = colorFormat == Av1ColorFormat.Yuv420 ? 1 : 0; + Size codedSize = Av1EncoderFrame.GetCodedSize(width, height); + Size lumaSize = Av1EncoderFrame.GetPlaneBufferSize(width, height, 0, 0); + int lumaElementCount = checked(lumaSize.Width * lumaSize.Height); + Size chromaSize = colorFormat == Av1ColorFormat.Yuv400 + ? Size.Empty + : Av1EncoderFrame.GetPlaneBufferSize(width, height, subsamplingX, subsamplingY); + + int chromaElementCount = checked(chromaSize.Width * chromaSize.Height); + int planeAlignment = Math.Max(PlaneAlignmentBytes / Unsafe.SizeOf(), 1); + int chromaBlueOffset = Align(lumaElementCount, planeAlignment); + int chromaRedOffset = Align(checked(chromaBlueOffset + chromaElementCount), planeAlignment); + int storageLength = colorFormat == Av1ColorFormat.Yuv400 + ? lumaElementCount + : checked(chromaRedOffset + chromaElementCount); + + // Libaom keeps the three component planes in one 32-byte-aligned frame allocation. The non-owning + // Buffer2D views preserve ImageSharp's row API without introducing separate plane rents or copies. + IMemoryOwner owner = configuration.MemoryAllocator.Allocate(storageLength); + Memory storage = owner.Memory; + Buffer2D luma = Buffer2D.WrapMemory( + storage[..lumaElementCount], + lumaSize.Width, + lumaSize.Height); + + this.Luma = luma; + + Buffer2DRegion lumaRegion = luma.GetRegion( + Av1EncoderFrame.LumaBorder, + Av1EncoderFrame.LumaBorder, + codedSize.Width, + codedSize.Height); + + Buffer2DRegion chromaBlueRegion = default; + Buffer2DRegion chromaRedRegion = default; + if (colorFormat != Av1ColorFormat.Yuv400) + { + Buffer2D chromaBlue = Buffer2D.WrapMemory( + storage.Slice(chromaBlueOffset, chromaElementCount), + chromaSize.Width, + chromaSize.Height); + + Buffer2D chromaRed = Buffer2D.WrapMemory( + storage.Slice(chromaRedOffset, chromaElementCount), + chromaSize.Width, + chromaSize.Height); + + this.ChromaBlue = chromaBlue; + this.ChromaRed = chromaRed; + + int chromaBorderX = Av1EncoderFrame.LumaBorder >> subsamplingX; + int chromaBorderY = Av1EncoderFrame.LumaBorder >> subsamplingY; + int codedChromaWidth = codedSize.Width >> subsamplingX; + int codedChromaHeight = codedSize.Height >> subsamplingY; + chromaBlueRegion = chromaBlue.GetRegion( + chromaBorderX, + chromaBorderY, + codedChromaWidth, + codedChromaHeight); + + chromaRedRegion = chromaRed.GetRegion( + chromaBorderX, + chromaBorderY, + codedChromaWidth, + codedChromaHeight); + } + + this.owner = owner; + this.Frame = new( + lumaRegion, + chromaBlueRegion, + chromaRedRegion, + width, + height, + bitDepth, + colorFormat, + chromaPositionX, + chromaPositionY); + } + + /// + /// Gets the non-owning coded frame view. + /// + public Av1EncoderFrame Frame { get; } + + /// + /// Gets the complete padded luma plane. + /// + public Buffer2D Luma { get; } + + /// + /// Gets the complete padded blue-difference chroma plane. + /// + public Buffer2D? ChromaBlue { get; } + + /// + /// Gets the complete padded red-difference chroma plane. + /// + public Buffer2D? ChromaRed { get; } + + /// + /// Releases the complete frame allocation. + /// + public void Dispose() + { + IMemoryOwner? ownedMemory = this.owner; + this.owner = null; + if (ownedMemory is null) + { + return; + } + + this.Luma.Dispose(); + this.ChromaBlue?.Dispose(); + this.ChromaRed?.Dispose(); + ownedMemory.Dispose(); + } + + /// + /// Aligns an element offset to the next component-plane boundary. + /// + private static int Align(int value, int alignment) + => checked((value + alignment - 1) & -alignment); +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1EncoderInterPredictionWorkspace.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1EncoderInterPredictionWorkspace.cs new file mode 100644 index 0000000000..9d26114870 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1EncoderInterPredictionWorkspace.cs @@ -0,0 +1,166 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline; + +/// +/// Provides disjoint reusable buffers for single-reference and intra-block-copy mode decisions. +/// +/// The native sample type selected by the encoder pipeline. +internal readonly ref struct Av1EncoderInterPredictionWorkspace + where TSample : unmanaged +{ + /// + /// The width and height of the fixed prediction block handled by the current inter search. + /// + private const int MaximumBlockDimension = 8; + + /// + /// The number of samples in the fixed prediction block. + /// + public const int MaximumSampleCount = MaximumBlockDimension * MaximumBlockDimension; + + /// + /// The signed intermediate capacity needed when both translational interpolation axes are filtered. + /// + public const int PredictionScratchCount = + Av1TranslationalInterPredictor.MinimumScratchStride * + (MaximumBlockDimension + Av1TranslationalInterPredictor.MaximumExtraRows); + + /// + /// The number of sample buffers retained by one mode decision. + /// + public const int SampleBufferCount = 10; + + /// + /// The number of coefficient buffers retained by one mode decision. + /// + public const int CoefficientBufferCount = 7; + + private readonly Span samples; + private readonly Span residual; + private readonly Span predictionScratch; + private readonly Span coefficients; + + /// + /// Initializes a new instance of the struct. + /// + /// The sample storage. + /// The residual storage shared by sequential plane evaluations. + /// The intermediate storage used by two-dimensional interpolation. + /// The coefficient storage. + public Av1EncoderInterPredictionWorkspace( + Span samples, + Span residual, + Span predictionScratch, + Span coefficients) + { + this.samples = samples; + this.residual = residual; + this.predictionScratch = predictionScratch; + this.coefficients = coefficients; + } + + /// + /// Gets the selected luma reconstruction. + /// + public Span SelectedLumaReconstruction => this.GetSamples(0); + + /// + /// Gets the selected blue-difference chroma reconstruction. + /// + public Span SelectedBlueReconstruction => this.GetSamples(1); + + /// + /// Gets the selected red-difference chroma reconstruction. + /// + public Span SelectedRedReconstruction => this.GetSamples(2); + + /// + /// Gets the current luma prediction. + /// + public Span LumaPrediction => this.GetSamples(3); + + /// + /// Gets the current blue-difference chroma prediction. + /// + public Span BluePrediction => this.GetSamples(4); + + /// + /// Gets the current red-difference chroma prediction. + /// + public Span RedPrediction => this.GetSamples(5); + + /// + /// Gets the current luma candidate reconstruction. + /// + public Span LumaCandidateReconstruction => this.GetSamples(6); + + /// + /// Gets the current blue-difference chroma candidate reconstruction. + /// + public Span BlueCandidateReconstruction => this.GetSamples(7); + + /// + /// Gets the current red-difference chroma candidate reconstruction. + /// + public Span RedCandidateReconstruction => this.GetSamples(8); + + /// + /// Gets the reconstruction scratch overwritten by each transform trial. + /// + public Span TransformReconstruction => this.GetSamples(9); + + /// + /// Gets the residual scratch shared by sequential plane evaluations. + /// + public Span Residual => this.residual; + + /// + /// Gets the intermediate scratch used when both translational interpolation axes are filtered. + /// + public Span PredictionScratch => this.predictionScratch; + + /// + /// Gets the selected luma coefficients. + /// + public Span SelectedLumaCoefficients => this.GetCoefficients(0); + + /// + /// Gets the selected blue-difference chroma coefficients. + /// + public Span SelectedBlueCoefficients => this.GetCoefficients(1); + + /// + /// Gets the selected red-difference chroma coefficients. + /// + public Span SelectedRedCoefficients => this.GetCoefficients(2); + + /// + /// Gets the current luma candidate coefficients. + /// + public Span LumaCandidateCoefficients => this.GetCoefficients(3); + + /// + /// Gets the current blue-difference chroma candidate coefficients. + /// + public Span BlueCandidateCoefficients => this.GetCoefficients(4); + + /// + /// Gets the current red-difference chroma candidate coefficients. + /// + public Span RedCandidateCoefficients => this.GetCoefficients(5); + + /// + /// Gets the coefficient scratch overwritten by each transform trial. + /// + public Span TransformCoefficients => this.GetCoefficients(6); + + private Span GetSamples(int index) + => this.samples.Slice(index * MaximumSampleCount, MaximumSampleCount); + + private Span GetCoefficients(int index) + => this.coefficients.Slice(index * MaximumSampleCount, MaximumSampleCount); +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1EncoderModeDecisionWorkspace.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1EncoderModeDecisionWorkspace.cs new file mode 100644 index 0000000000..e3079621f2 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1EncoderModeDecisionWorkspace.cs @@ -0,0 +1,337 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.InteropServices; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.ChromaFromLuma; +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline; + +/// +/// Provides typed views over the reusable storage shared by mutually exclusive AV1 mode searches. +/// +/// The native sample type selected by the encoder pipeline. +internal readonly ref struct Av1EncoderModeDecisionWorkspace + where TSample : unmanaged +{ + /// + /// The largest coding-block dimension evaluated directly by the current partition search. + /// + public const int MaximumBlockDimension = 128; + + /// + /// The maximum number of samples in one directly evaluated coding block. + /// + public const int MaximumSampleCount = MaximumBlockDimension * MaximumBlockDimension; + + /// + /// The maximum number of samples in one AV1 transform. + /// + public const int MaximumTransformSampleCount = + Av1Constants.MaxTransformSize * Av1Constants.MaxTransformSize; + + /// + /// The number of 4x4 transform blocks covering one 8x8 coding block. + /// + public const int CandidateTransformBlockCount = 4; + + /// + /// The maximum number of transform states needed while evaluating both chroma planes of one 128x128 block. + /// + public const int MaximumCandidateTransformBlockCount = + 2 * MaximumSampleCount / MinimumTransformSampleCount; + + /// + /// The required workspace length in signed-integer storage elements. + /// + public const int StorageLength = TransientStorageOffset + Av1EncoderPaletteWorkspace.StorageLength; + + private const int ReferenceBufferLength = (2 * Av1Constants.MaxTransformSize) + 1; + private const int ReferenceBufferCount = 4; + private const int MinimumTransformSampleCount = 1 << (2 * Av1Constants.ModeInfoSizeLog2); + private const int ReferenceStorageLength = ReferenceBufferCount * ReferenceBufferLength * sizeof(ushort) / sizeof(int); + private const int CandidateSampleStorageOffset = ReferenceStorageLength; + private const int CandidateSampleStorageLength = 2 * MaximumSampleCount * sizeof(ushort) / sizeof(int); + private const int CandidateCoefficientStorageOffset = CandidateSampleStorageOffset + CandidateSampleStorageLength; + private const int CandidateCoefficientStorageLength = 2 * MaximumSampleCount; + private const int CandidateTransformBlockStorageOffset = CandidateCoefficientStorageOffset + CandidateCoefficientStorageLength; + private const int CandidateTransformBlockStorageLength = MaximumCandidateTransformBlockCount; + private const int TransformContextStorageOffset = CandidateTransformBlockStorageOffset + CandidateTransformBlockStorageLength; + private const int TransformContextStorageLength = + 4 * (MaximumBlockDimension >> Av1Constants.ModeInfoSizeLog2) * sizeof(byte) / sizeof(int); + + private const int TransientStorageOffset = TransformContextStorageOffset + TransformContextStorageLength; + private const int ChromaFromLumaSampleCount = Av1ChromaFromLumaContext.BufferLength; + + private const int ChromaFromLumaSampleStorageLength = ChromaFromLumaSampleCount * sizeof(short) / sizeof(int); + private const int ChromaFromLumaBlueRateOffset = ChromaFromLumaSampleStorageLength; + private const int ChromaFromLumaRedRateOffset = ChromaFromLumaBlueRateOffset + Av1ChromaFromLumaMath.AlphaCandidateCount; + private const int ChromaFromLumaBlueDistortionOffset = + ChromaFromLumaRedRateOffset + Av1ChromaFromLumaMath.AlphaCandidateCount; + + private const int ChromaFromLumaDistortionStorageLength = + Av1ChromaFromLumaMath.AlphaCandidateCount * sizeof(long) / sizeof(int); + + private const int ChromaFromLumaRedDistortionOffset = + ChromaFromLumaBlueDistortionOffset + ChromaFromLumaDistortionStorageLength; + + private readonly Span storage; + + /// + /// Initializes a new instance of the struct. + /// + /// The reusable aligned decision storage. + public Av1EncoderModeDecisionWorkspace(Span storage) => this.storage = storage; + + /// + /// Gets the temporary prediction span shared by mutually exclusive mode searches. + /// + public Span Prediction + => MemoryMarshal.Cast(this.storage[TransientStorageOffset..])[..MaximumTransformSampleCount]; + + /// + /// Gets the temporary residual span shared by mutually exclusive mode searches. + /// + public Span Residual + => MemoryMarshal.Cast( + this.storage.Slice( + TransientStorageOffset + (MaximumTransformSampleCount * sizeof(ushort) / sizeof(int)), + MaximumTransformSampleCount * sizeof(short) / sizeof(int))); + + /// + /// Gets the fixed-stride subsampled luma values used by chroma-from-luma mode search. + /// + public Span ChromaFromLumaSamples + => MemoryMarshal.Cast( + this.storage.Slice(TransientStorageOffset, ChromaFromLumaSampleStorageLength)); + + /// + /// Gets the palette-search view over transient storage that is no longer needed after spatial and CfL search. + /// + public Av1EncoderPaletteWorkspace Palette + => new(this.storage[TransientStorageOffset..]); + + /// + /// Gets the transform states retained while evaluating a multi-transform candidate. + /// + public Span CandidateTransformBlocks + => MemoryMarshal.Cast( + this.storage.Slice(CandidateTransformBlockStorageOffset, CandidateTransformBlockStorageLength)); + + /// + /// Gets coefficient-context edges shared by mutually exclusive luma and chroma transform trials. + /// + public Span TransformContexts + => MemoryMarshal.AsBytes(this.storage.Slice(TransformContextStorageOffset, TransformContextStorageLength)); + + /// + /// Gets one reference edge including its common-corner prefix. + /// + /// The zero-based edge index. + /// The fixed reference-edge span. + public Span GetReferenceSamples(int index) + => MemoryMarshal.Cast(this.storage[..ReferenceStorageLength]) + .Slice(index * ReferenceBufferLength, ReferenceBufferLength); + + /// + /// Gets one candidate reconstruction plane. + /// + /// The zero-based plane index. + /// The maximum-size candidate reconstruction span. + public Span GetCandidateReconstruction(int index) + => MemoryMarshal.Cast( + this.storage.Slice(CandidateSampleStorageOffset, CandidateSampleStorageLength)) + .Slice(index * MaximumSampleCount, MaximumSampleCount); + + /// + /// Gets one candidate coefficient plane. + /// + /// The zero-based plane index. + /// The maximum-size candidate coefficient span. + public Span GetCandidateCoefficients(int index) + => this.storage + .Slice(CandidateCoefficientStorageOffset, CandidateCoefficientStorageLength) + .Slice(index * MaximumSampleCount, MaximumSampleCount); + + /// + /// Gets one plane's chroma-from-luma coefficient-rate table. + /// + /// The zero-based chroma plane index. + /// The rate table for every signed alpha candidate. + public Span GetChromaFromLumaRates(int planeIndex) + => this.storage.Slice( + TransientStorageOffset + ChromaFromLumaBlueRateOffset + + (planeIndex * Av1ChromaFromLumaMath.AlphaCandidateCount), + Av1ChromaFromLumaMath.AlphaCandidateCount); + + /// + /// Gets one plane's chroma-from-luma distortion table. + /// + /// The zero-based chroma plane index. + /// The distortion table for every signed alpha candidate. + public Span GetChromaFromLumaDistortions(int planeIndex) + => MemoryMarshal.Cast( + this.storage.Slice( + TransientStorageOffset + ChromaFromLumaBlueDistortionOffset + + (planeIndex * ChromaFromLumaDistortionStorageLength), + ChromaFromLumaDistortionStorageLength)); +} + +/// +/// Provides typed luma and chroma palette-search buffers over reusable mode-decision storage. +/// +/// The native sample type selected by the encoder pipeline. +internal readonly ref struct Av1EncoderPaletteWorkspace + where TSample : unmanaged +{ + /// + /// The required workspace length in signed-integer storage elements. + /// + public const int StorageLength = ColorCacheOffset + ColorCacheStorageLength; + + private const int MaximumBlockDimension = 64; + private const int MaximumSampleCount = MaximumBlockDimension * MaximumBlockDimension; + private const int PlaneShortStorageLength = MaximumSampleCount * sizeof(short) / sizeof(int); + private const int PlaneSampleStorageLength = MaximumSampleCount * sizeof(ushort) / sizeof(int); + private const int PlaneByteStorageLength = MaximumSampleCount / sizeof(int); + private const int PaletteColorStorageLength = Av1Constants.PaletteMaxSize * sizeof(ushort) / sizeof(int); + private const int FirstSampleOffset = 0; + private const int SecondSampleOffset = FirstSampleOffset + PlaneShortStorageLength; + private const int FirstUniqueColorOffset = SecondSampleOffset + PlaneShortStorageLength; + private const int SecondUniqueColorOffset = FirstUniqueColorOffset + PlaneShortStorageLength; + private const int FirstPredictionOffset = SecondUniqueColorOffset + PlaneShortStorageLength; + private const int SecondPredictionOffset = FirstPredictionOffset + PlaneSampleStorageLength; + private const int FirstResidualOffset = SecondPredictionOffset + PlaneSampleStorageLength; + private const int SecondResidualOffset = FirstResidualOffset + PlaneShortStorageLength; + private const int RetainedIndexOffset = SecondResidualOffset + PlaneShortStorageLength; + private const int IndexOffset = RetainedIndexOffset + PlaneByteStorageLength; + private const int FirstCentroidOffset = IndexOffset + PlaneByteStorageLength; + private const int SecondCentroidOffset = FirstCentroidOffset + PaletteColorStorageLength; + private const int FirstPaletteColorOffset = SecondCentroidOffset + PaletteColorStorageLength; + private const int SecondPaletteColorOffset = FirstPaletteColorOffset + PaletteColorStorageLength; + private const int FirstAlternateCentroidOffset = SecondPaletteColorOffset + PaletteColorStorageLength; + private const int SecondAlternateCentroidOffset = FirstAlternateCentroidOffset + PaletteColorStorageLength; + private const int AlternateIndexOffset = SecondAlternateCentroidOffset + PaletteColorStorageLength; + private const int ColorCountOffset = AlternateIndexOffset + PlaneByteStorageLength; + private const int DominantOrderOffset = ColorCountOffset + MaximumSampleCount; + private const int ColorCacheOffset = DominantOrderOffset + PlaneByteStorageLength; + private const int ColorCacheStorageLength = 2 * Av1Constants.PaletteMaxSize * sizeof(ushort) / sizeof(int); + + private readonly Span storage; + + /// + /// Initializes a new instance of the struct. + /// + /// The reusable aligned palette storage. + public Av1EncoderPaletteWorkspace(Span storage) => this.storage = storage; + + /// + /// Gets the retained winning color-index map. + /// + public Span RetainedIndices + => MemoryMarshal.AsBytes(this.storage.Slice(RetainedIndexOffset, PlaneByteStorageLength)); + + /// + /// Gets the current color-index map. + /// + public Span Indices + => MemoryMarshal.AsBytes(this.storage.Slice(IndexOffset, PlaneByteStorageLength)); + + /// + /// Gets the alternate K-means color-index map. + /// + public Span AlternateIndices + => MemoryMarshal.AsBytes(this.storage.Slice(AlternateIndexOffset, PlaneByteStorageLength)); + + /// + /// Gets the luma occurrence count for every unique color. + /// + public Span LumaColorCounts + => this.storage.Slice(ColorCountOffset, MaximumSampleCount); + + /// + /// Gets the luma unique-color ordering by descending occurrence count. + /// + public Span LumaDominantOrder + => MemoryMarshal.AsBytes(this.storage.Slice(DominantOrderOffset, PlaneByteStorageLength)); + + /// + /// Gets the sorted neighboring palette colors available to the current block. + /// + public Span ColorCache + => MemoryMarshal.Cast(this.storage.Slice(ColorCacheOffset, ColorCacheStorageLength)); + + /// + /// Gets one plane's active palette samples. + /// + /// The zero-based plane index. + /// The maximum-size sample span. + public Span GetSamples(int planeIndex) + => MemoryMarshal.Cast( + this.storage.Slice(FirstSampleOffset + (planeIndex * PlaneShortStorageLength), PlaneShortStorageLength)); + + /// + /// Gets one plane's unique palette colors. + /// + /// The zero-based plane index. + /// The maximum-size unique-color span. + public Span GetUniqueColors(int planeIndex) + => MemoryMarshal.Cast( + this.storage.Slice( + FirstUniqueColorOffset + (planeIndex * PlaneShortStorageLength), + PlaneShortStorageLength)); + + /// + /// Gets one plane's palette prediction. + /// + /// The zero-based plane index. + /// The maximum-size prediction span. + public Span GetPrediction(int planeIndex) + => MemoryMarshal.Cast( + this.storage.Slice( + FirstPredictionOffset + (planeIndex * PlaneSampleStorageLength), + PlaneSampleStorageLength))[..MaximumSampleCount]; + + /// + /// Gets one plane's palette residual. + /// + /// The zero-based plane index. + /// The maximum-size residual span. + public Span GetResidual(int planeIndex) + => MemoryMarshal.Cast( + this.storage.Slice(FirstResidualOffset + (planeIndex * PlaneShortStorageLength), PlaneShortStorageLength)); + + /// + /// Gets one plane's current palette centroids. + /// + /// The zero-based plane index. + /// The maximum-size centroid span. + public Span GetCentroids(int planeIndex) + => MemoryMarshal.Cast( + this.storage.Slice( + FirstCentroidOffset + (planeIndex * PaletteColorStorageLength), + PaletteColorStorageLength)); + + /// + /// Gets one plane's coded palette colors. + /// + /// The zero-based plane index. + /// The maximum-size coded-color span. + public Span GetPaletteColors(int planeIndex) + => MemoryMarshal.Cast( + this.storage.Slice( + FirstPaletteColorOffset + (planeIndex * PaletteColorStorageLength), + PaletteColorStorageLength)); + + /// + /// Gets one plane's alternate K-means centroids. + /// + /// The zero-based plane index. + /// The maximum-size alternate-centroid span. + public Span GetAlternateCentroids(int planeIndex) + => MemoryMarshal.Cast( + this.storage.Slice( + FirstAlternateCentroidOffset + (planeIndex * PaletteColorStorageLength), + PaletteColorStorageLength)); +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1FrameDecoder.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1FrameDecoder.cs new file mode 100644 index 0000000000..d814b1fe61 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1FrameDecoder.cs @@ -0,0 +1,189 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.Cdef; +using SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.LoopFilter; +using SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.LoopRestoration; +using SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.SuperResolution; +using SixLabors.ImageSharp.Formats.Heif.Av1.ReferenceFrames; +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline; + +/// +/// Reconstructs the coded blocks of one AV1 image frame into planar sample buffers. +/// +internal sealed class Av1FrameDecoder : IAv1FrameDecoder, IDisposable +{ + /// + /// The sequence-level superblock and color configuration. + /// + private readonly ObuSequenceHeader sequenceHeader; + + /// + /// The frame-level tile, quantization, and reconstruction configuration. + /// + private readonly ObuFrameHeader frameHeader; + + /// + /// The parsed superblock and block-mode information for the frame. + /// + private readonly Av1FrameInfo frameInfo; + + /// + /// The destination planar sample buffers for reconstructed pixels. + /// + private readonly Av1FrameBuffer frameBuffer; + + /// + /// The retained reconstructed frames addressable by inter prediction. + /// + private readonly Av1ReferenceFrameStore referenceFrames; + + /// + /// The transform-size map populated during reconstruction and consumed by deblocking. + /// + private readonly Av1LoopFilterContext loopFilterContext; + + /// + /// The block reconstruction stage that applies prediction and inverse transforms. + /// + private readonly Av1BlockDecoder blockDecoder; + + /// + /// Initializes a new instance of the class. + /// + /// The parsed AV1 sequence header. + /// The parsed AV1 frame header. + /// The parsed superblock and block-mode information. + /// The destination planar sample buffers. + /// The retained reconstructed frames selected by inter blocks. + /// The complete decoder-session palette map state. + public Av1FrameDecoder( + ObuSequenceHeader sequenceHeader, + ObuFrameHeader frameHeader, + Av1FrameInfo frameInfo, + Av1FrameBuffer frameBuffer, + Av1ReferenceFrameStore referenceFrames, + Av1TileReader.PaletteColorIndexMaps? paletteColorIndexMaps = null) + { + this.sequenceHeader = sequenceHeader; + this.frameHeader = frameHeader; + this.frameInfo = frameInfo; + this.frameBuffer = frameBuffer; + this.referenceFrames = referenceFrames; + this.loopFilterContext = new(frameBuffer.MemoryAllocator, sequenceHeader, frameHeader); + try + { + this.blockDecoder = new( + this.sequenceHeader, + this.frameHeader, + this.frameBuffer, + this.loopFilterContext, + this.referenceFrames, + paletteColorIndexMaps); + } + catch + { + this.loopFilterContext.Dispose(); + throw; + } + } + + /// + /// Releases the pooled block-reconstruction workspaces owned by this decoder. + /// + public void Dispose() + { + this.blockDecoder.Dispose(); + this.loopFilterContext.Dispose(); + } + + /// + /// Applies the in-loop frame stages after every superblock has been reconstructed. + /// + public void CompleteFrame() + { + bool doLoopRestoration = this.frameHeader.LoopRestorationParameters.UsesLoopRestoration; + + Av1LoopFilterDecoder loopFilterDecoder = new( + this.sequenceHeader, + this.frameHeader, + this.frameInfo, + this.frameBuffer, + this.loopFilterContext); + + loopFilterDecoder.DecodeFrame(); + + using Av1LoopRestorationBoundary? restorationBoundary = doLoopRestoration + ? new(this.sequenceHeader, this.frameHeader, this.frameBuffer) + : null; + + if (restorationBoundary is not null) + { + restorationBoundary.SaveDeblockedRows(); + } + + Av1CdefDecoder cdefDecoder = new(this.sequenceHeader, this.frameHeader, this.frameInfo, this.frameBuffer); + cdefDecoder.DecodeFrame(); + + Av1SuperResolutionDecoder superResolutionDecoder = new(this.sequenceHeader, this.frameHeader, this.frameBuffer); + superResolutionDecoder.DecodeFrame(); + + if (restorationBoundary is not null) + { + restorationBoundary.SaveFrameEdgeRows(); + Av1LoopRestorationDecoder loopRestorationDecoder = new( + this.sequenceHeader, + this.frameHeader, + this.frameInfo, + this.frameBuffer, + restorationBoundary); + + loopRestorationDecoder.DecodeFrame(); + } + + // Film grain is deliberately excluded here because this buffer is the normative post-restoration reference. + // The owning decoder applies grain only to the presentation buffer after reference ownership is established. + } + + /// + /// Reconstructs one superblock from its parsed block state and dequantized coefficients. + /// + /// The superblock's top-left position in 4x4 mode-info units. + /// The decoded syntax and block modes for the superblock. + /// The tile that contains the superblock. + public void DecodeSuperblock(Point modeInfoPosition, Av1SuperblockInfo superblockInfo, Av1TileInfo tileInfo) + { + this.blockDecoder.UpdateSuperblock(superblockInfo); + this.DecodePartition(modeInfoPosition, superblockInfo, tileInfo); + } + + /// + /// Reconstructs each decoded block in a superblock partition. + /// + /// The superblock's frame-relative origin in 4x4 mode-info units. + /// The superblock whose block modes are traversed. + /// The tile boundary information used by intra prediction. + /// Traverses the depth-first block order produced by tile parsing. + private void DecodePartition(Point modeInfoPosition, Av1SuperblockInfo superblockInfo, Av1TileInfo tileInfo) + { + foreach (ref Av1BlockModeInfo modeInfo in superblockInfo.GetModeInfos()) + { + Point subPosition = modeInfo.PositionInSuperblock; + Av1BlockSize subSize = modeInfo.BlockSize; + Point globalPosition = new(modeInfoPosition.X, modeInfoPosition.Y); + + // Block positions are stored relative to the superblock; prediction and reconstruction require frame-relative mode-info coordinates. + globalPosition.Offset(subPosition); + this.blockDecoder.DecodeBlock(modeInfo, globalPosition, subSize, superblockInfo, tileInfo); + + // Palette maps are decoder-session scratch. Retained mode information must not keep views after the block + // has consumed them because the next superblock reuses the same storage. + modeInfo.SetPaletteColorIndexMap(Av1PlaneType.Y, default); + modeInfo.SetPaletteColorIndexMap(Av1PlaneType.Uv, default); + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1FrameEncoder.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1FrameEncoder.cs new file mode 100644 index 0000000000..69ef8c6f70 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1FrameEncoder.cs @@ -0,0 +1,1922 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Buffers; +using System.Runtime.CompilerServices; +using System.Runtime.InteropServices; +using SixLabors.ImageSharp.Formats.Heif.Av1.Color; +using SixLabors.ImageSharp.Formats.Heif.Av1.Entropy; +using SixLabors.ImageSharp.Formats.Heif.Av1.Motion; +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.Quantizers; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; +using SixLabors.ImageSharp.Formats.Heif.Components; +using SixLabors.ImageSharp.Formats.Heif.Components.Alpha; +using SixLabors.ImageSharp.Memory; +using SixLabors.ImageSharp.PixelFormats; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline; + +/// +/// Performs operation-scoped AV1 frame encoding. +/// +internal static class Av1FrameEncoder +{ + /// + /// The base-two exponent used to align each frame dimension for output sizing. Rounding to 32 samples accounts + /// for partial edge storage before the raw-plane size and all-intra expansion factor are calculated. + /// + private const int OutputAlignmentLog2 = 5; + + /// + /// The lower bound, in bytes, for the bounded compressed-frame buffer. The raw-size ratio is too small for tiny + /// images to provide useful coder headroom, so the reference allocation retains an 8 KiB floor. + /// + private const int MinimumCompressedFrameBufferLength = 8 * 1024; + + /// + /// The numerator of the all-intra output-capacity ratio. Together with the denominator, this reserves 2.5 times + /// the aligned uncompressed plane size because incompressible input can produce more output than its raw size. + /// + private const int AllIntraBufferScaleNumerator = 5; + + /// + /// The denominator of the all-intra output-capacity ratio, completing the reference encoder's 5:2 sizing rule. + /// + private const int AllIntraBufferScaleDenominator = 2; + + /// + /// The sequence-level value that leaves the operating point unconstrained for decoder capability signaling. + /// + private const int UnconstrainedSequenceLevelIndex = 31; + + /// + /// The highest public effort value, which enables 128x128 superblocks for sufficiently large images. + /// + private const int MaximumEffort = 10; + + /// + /// The native component precision used by the byte pipeline. + /// + private const int ByteSampleBitDepth = 8; + + /// + /// The centered chroma position expressed in AV1 half-luma-sample units. + /// + private const int CenteredChromaSamplePosition = 1; + + /// + /// The first effort tier that searches frame-level translation between sequence samples. + /// + private const int MinimumGlobalMotionSearchEffort = 6; + + /// + /// The first effort tier that compares the three interpolation families for reference-frame prediction. + /// + private const int MinimumSwitchableInterpolationEffort = 8; + + /// + /// The first effort tier that searches independent vertical and horizontal interpolation families. + /// + private const int MinimumDualInterpolationEffort = 9; + + /// + /// The smallest full-pixel radius searched when frame-level motion analysis is enabled. + /// + private const int MinimumGlobalMotionSearchRadius = 4; + + /// + /// The largest dimension of the central luma window used during candidate discovery. + /// + private const int MaximumGlobalMotionAnalysisDimension = 512; + + /// + /// The number of cardinal and diagonal candidates evaluated at each motion-search step. + /// + private const int GlobalMotionSearchDirectionCount = 8; + + private enum FrameEncodingKind + { + StillColor, + StillAlpha + } + + /// + /// Defines the sample-specific SIMD squared-error operation used by frame-level motion search. + /// + /// The component sample type. + private interface IGlobalMotionSearchOperator + where TSample : unmanaged + { + /// + /// Calculates squared error between two equally sized strided sample regions. + /// + /// The first sample of the source region. + /// The source distance, in samples, between adjacent rows. + /// The first sample of the prediction region. + /// The prediction distance, in samples, between adjacent rows. + /// The number of samples compared in each row. + /// The number of rows compared. + /// The sum of squared component differences. + public static abstract long SumSquaredError( + ReadOnlySpan source, + int sourceStride, + ReadOnlySpan prediction, + int predictionStride, + int width, + int height); + } + + /// + /// Encodes one reduced-still-picture AV1 frame into a low-overhead OBU stream. + /// + /// The packed source pixel type. + /// The configuration providing every operation-scoped allocation. + /// The packed source frame. + /// The destination receiving the complete AV1 item payload. + /// The resolved native color and precision configuration. + /// The frame quantizer index. + /// The mode-search effort in the inclusive range zero through ten. + /// The sequence header describing the encoded payload. + public static ObuSequenceHeader Encode( + Configuration configuration, + ImageFrame image, + Stream stream, + ObuColorConfig colorConfig, + int qIndex, + int effort) + where TPixel : unmanaged, IPixel + { + Rectangle sourceRectangle = new(0, 0, image.Width, image.Height); + return Encode( + configuration, + image, + sourceRectangle, + sourceRectangle.Size, + stream, + colorConfig, + qIndex, + effort, + FrameEncodingKind.StillColor); + } + + /// + /// Encodes one grid cell as a reduced-still-picture AV1 frame in a low-overhead OBU stream. + /// + /// The packed source pixel type. + /// The configuration providing every operation-scoped allocation. + /// The packed source frame. + /// The source region copied into the top-left of the encoded cell. + /// The encoded cell dimensions, including any required edge padding. + /// The destination receiving the complete AV1 item payload. + /// The resolved native color and precision configuration. + /// The frame quantizer index. + /// The mode-search effort in the inclusive range zero through ten. + /// The sequence header describing the encoded payload. + public static ObuSequenceHeader EncodeGridCell( + Configuration configuration, + ImageFrame image, + Rectangle sourceRectangle, + Size cellSize, + Stream stream, + ObuColorConfig colorConfig, + int qIndex, + int effort) + where TPixel : unmanaged, IPixel + => Encode( + configuration, + image, + sourceRectangle, + cellSize, + stream, + colorConfig, + qIndex, + effort, + FrameEncodingKind.StillColor); + + /// + /// Encodes one packed alpha channel as a reduced-still-picture monochrome AV1 frame. + /// + /// The packed source pixel type. + /// The configuration providing every operation-scoped allocation. + /// The packed source frame. + /// The destination receiving the complete AV1 item payload. + /// The resolved monochrome precision configuration. + /// The frame quantizer index. + /// The mode-search effort in the inclusive range zero through ten. + /// The sequence header describing the encoded payload. + public static ObuSequenceHeader EncodeAlpha( + Configuration configuration, + ImageFrame image, + Stream stream, + ObuColorConfig colorConfig, + int qIndex, + int effort) + where TPixel : unmanaged, IPixel + { + Rectangle sourceRectangle = new(0, 0, image.Width, image.Height); + return Encode( + configuration, + image, + sourceRectangle, + sourceRectangle.Size, + stream, + colorConfig, + qIndex, + effort, + FrameEncodingKind.StillAlpha); + } + + /// + /// Encodes one packed alpha grid cell as a reduced-still-picture monochrome AV1 frame. + /// + /// The packed source pixel type. + /// The configuration providing every operation-scoped allocation. + /// The packed source frame. + /// The source region copied into the top-left of the encoded cell. + /// The encoded cell dimensions, including any required edge padding. + /// The destination receiving the complete AV1 item payload. + /// The resolved monochrome precision configuration. + /// The frame quantizer index. + /// The mode-search effort in the inclusive range zero through ten. + /// The sequence header describing the encoded payload. + public static ObuSequenceHeader EncodeAlphaGridCell( + Configuration configuration, + ImageFrame image, + Rectangle sourceRectangle, + Size cellSize, + Stream stream, + ObuColorConfig colorConfig, + int qIndex, + int effort) + where TPixel : unmanaged, IPixel + => Encode( + configuration, + image, + sourceRectangle, + cellSize, + stream, + colorConfig, + qIndex, + effort, + FrameEncodingKind.StillAlpha); + + /// + /// Creates an encoder that retains reconstructed color frames for prediction by later samples in the sequence. + /// + public static SequenceEncoder CreateColorSequenceEncoder( + Configuration configuration, + int width, + int height, + ObuColorConfig colorConfig, + int qIndex, + int effort) + => CreateSequenceEncoder(configuration, width, height, colorConfig, qIndex, effort, false); + + /// + /// Creates an encoder that retains reconstructed alpha frames for prediction by later samples in the sequence. + /// + public static SequenceEncoder CreateAlphaSequenceEncoder( + Configuration configuration, + int width, + int height, + ObuColorConfig colorConfig, + int qIndex, + int effort) + => CreateSequenceEncoder(configuration, width, height, colorConfig, qIndex, effort, true); + + private static ObuSequenceHeader Encode( + Configuration configuration, + ImageFrame image, + Rectangle sourceRectangle, + Size frameSize, + Stream stream, + ObuColorConfig colorConfig, + int qIndex, + int effort, + FrameEncodingKind encodingKind) + where TPixel : unmanaged, IPixel + { + int width = frameSize.Width; + int height = frameSize.Height; + bool encodeAlpha = encodingKind == FrameEncodingKind.StillAlpha; + Av1ColorFormat colorFormat = colorConfig.GetColorFormat(); + ObuSequenceHeader sequenceHeader = CreateSequenceHeader( + width, + height, + colorConfig, + effort, + true); + + ObuFrameHeader frameHeader = CreateFrameHeader( + width, + height, + qIndex, + effort, + ObuFrameType.KeyFrame); + + int tileBufferLength = GetTileBufferLength(width, height, colorConfig); + if (colorConfig.BitDepth == Av1BitDepth.EightBit) + { + EncodeByte( + configuration, + image, + sourceRectangle, + frameSize, + stream, + sequenceHeader, + frameHeader, + colorFormat, + tileBufferLength, + effort, + encodeAlpha); + } + else + { + EncodeHighBitDepth( + configuration, + image, + sourceRectangle, + frameSize, + stream, + sequenceHeader, + frameHeader, + colorFormat, + tileBufferLength, + effort, + encodeAlpha); + } + + return sequenceHeader; + } + + private static SequenceEncoder CreateSequenceEncoder( + Configuration configuration, + int width, + int height, + ObuColorConfig colorConfig, + int qIndex, + int effort, + bool encodeAlpha) + { + if (colorConfig.BitDepth == Av1BitDepth.EightBit) + { + return new ByteSequenceEncoder(configuration, width, height, colorConfig, qIndex, effort, encodeAlpha); + } + + return new HighBitDepthSequenceEncoder(configuration, width, height, colorConfig, qIndex, effort, encodeAlpha); + } + + private static ObuSequenceHeader CreateSequenceHeader( + int width, + int height, + ObuColorConfig colorConfig, + int effort, + bool isStillPicture) + { + Av1ColorFormat colorFormat = colorConfig.GetColorFormat(); + ObuSequenceProfile sequenceProfile = colorConfig.BitDepth == Av1BitDepth.TwelveBit || + colorFormat == Av1ColorFormat.Yuv422 + ? ObuSequenceProfile.Professional + : colorFormat == Av1ColorFormat.Yuv444 + ? ObuSequenceProfile.High + : ObuSequenceProfile.Main; + + return new ObuSequenceHeader + { + IsStillPicture = isStillPicture, + IsReducedStillPictureHeader = isStillPicture, + SequenceProfile = sequenceProfile, + OperatingPoint = [new ObuOperatingPoint { SequenceLevelIndex = UnconstrainedSequenceLevelIndex }], + FrameWidthBits = width > 1 ? Av1Math.MostSignificantBit((uint)(width - 1)) + 1 : 1, + FrameHeightBits = height > 1 ? Av1Math.MostSignificantBit((uint)(height - 1)) + 1 : 1, + MaxFrameWidth = width, + MaxFrameHeight = height, + Use128x128Superblock = Uses128x128Superblock(width, height, effort), + ForceScreenContentTools = Av1Constants.SelectScreenContentTools, + ForceIntegerMotionVector = Av1Constants.SelectIntegerMotionVector, + EnableFilterIntra = effort >= 4, + EnableDualFilter = !isStillPicture && effort >= MinimumDualInterpolationEffort, + EnableIntraEdgeFilter = true, + EnableSuperResolution = false, + EnableCdef = false, + EnableRestoration = false, + ColorConfig = colorConfig + }; + } + + private static bool Uses128x128Superblock(int width, int height, int effort) + => effort == MaximumEffort && + width >= Av1BlockSize.Block128x128.GetWidth() && + height >= Av1BlockSize.Block128x128.GetHeight(); + + private static ObuTileGroupHeader CreateTileGroupHeader( + int width, + int height, + int modeInfoColumnCount, + int modeInfoRowCount, + int effort) + { + int superblockSizeLog2 = Uses128x128Superblock(width, height, effort) + ? Av1Constants.MaxSuperBlockSizeLog2 + : Av1Constants.MaxSuperBlockSizeLog2 - 1; + + int superblockShift = superblockSizeLog2 - Av1Constants.ModeInfoSizeLog2; + int superblockColumns = Av1Math.DivideLog2Ceiling(modeInfoColumnCount, superblockShift); + int superblockRows = Av1Math.DivideLog2Ceiling(modeInfoRowCount, superblockShift); + int maximumTileWidth = Av1Constants.MaxTileWidth >> superblockSizeLog2; + int maximumTileArea = Av1Constants.MaxTileArea >> (2 * superblockSizeLog2); + int tileColumnCountLog2 = ObuReader.TileLog2(maximumTileWidth, superblockColumns); + int minimumTileCountLog2 = Math.Max( + tileColumnCountLog2, + ObuReader.TileLog2(maximumTileArea, superblockColumns * superblockRows)); + + int tileRowCountLog2 = minimumTileCountLog2 - tileColumnCountLog2; + int tileWidthSuperblocks = Av1Math.DivideLog2Ceiling(superblockColumns, tileColumnCountLog2); + int tileHeightSuperblocks = Av1Math.DivideLog2Ceiling(superblockRows, tileRowCountLog2); + ObuTileGroupHeader tiles = new() + { + HasUniformTileSpacing = true, + TileColumnCountLog2 = tileColumnCountLog2, + TileRowCountLog2 = tileRowCountLog2, + TileSizeBytes = sizeof(uint) + }; + + // Uniform tile boundaries are derived in superblock units. The terminal entries retain the exact + // visible mode-info dimensions so clipped right and bottom superblocks end at the frame boundary. + int tileColumn = 0; + for (int startSuperblock = 0; startSuperblock < superblockColumns; startSuperblock += tileWidthSuperblocks) + { + tiles.TileColumnStartModeInfo[tileColumn++] = startSuperblock << superblockShift; + } + + tiles.TileColumnStartModeInfo[tileColumn] = modeInfoColumnCount; + tiles.TileColumnCount = tileColumn; + + int tileRow = 0; + for (int startSuperblock = 0; startSuperblock < superblockRows; startSuperblock += tileHeightSuperblocks) + { + tiles.TileRowStartModeInfo[tileRow++] = startSuperblock << superblockShift; + } + + tiles.TileRowStartModeInfo[tileRow] = modeInfoRowCount; + tiles.TileRowCount = tileRow; + return tiles; + } + + private static ObuFrameHeader CreateFrameHeader( + int width, + int height, + int qIndex, + int effort, + ObuFrameType frameType) + { + int modeInfoColumnCount = 2 * ((width + 7) >> 3); + int modeInfoRowCount = 2 * ((height + 7) >> 3); + ObuTileGroupHeader tiles = CreateTileGroupHeader( + width, + height, + modeInfoColumnCount, + modeInfoRowCount, + effort); + + ObuFrameHeader frameHeader = new() + { + ModeInfoColumnCount = modeInfoColumnCount, + ModeInfoRowCount = modeInfoRowCount, + TilesInfo = tiles, + FrameSize = new ObuFrameSize + { + FrameWidth = width, + FrameHeight = height, + SuperResolutionDenominator = Av1Constants.ScaleNumerator, + SuperResolutionUpscaledWidth = width, + RenderWidth = width, + RenderHeight = height + } + }; + + ConfigureFrameHeader(frameHeader, qIndex, effort, frameType); + return frameHeader; + } + + /// + /// Restores every frame-varying encoder field while retaining the fixed geometry and syntax object graph. + /// + private static void ConfigureFrameHeader( + ObuFrameHeader frameHeader, + int qIndex, + int effort, + ObuFrameType frameType) + { + frameHeader.FrameType = frameType; + frameHeader.ShowFrame = true; + frameHeader.ErrorResilientMode = true; + frameHeader.RefreshFrameFlags = byte.MaxValue; + frameHeader.DisableFrameEndUpdateCdf = true; + frameHeader.ReferenceMode = ObuReferenceMode.SingleReference; + frameHeader.InterpolationFilter = Av1InterpolationFilter.Regular; + frameHeader.IsMotionModeSwitchable = false; + frameHeader.TransformMode = qIndex == 0 + ? Av1TransformMode.Only4x4 + : effort >= 6 ? Av1TransformMode.Select : Av1TransformMode.Largest; + + frameHeader.AllowScreenContentTools = false; + frameHeader.AllowIntraBlockCopy = false; + frameHeader.ForceIntegerMotionVector = false; + frameHeader.AllowHighPrecisionMotionVector = false; + if (frameType == ObuFrameType.InterFrame) + { + // Disabling screen-content tools makes force_integer_mv implicitly false. Lower-effort searches + // still stop at full pixels, but their vectors use the normal fractional-motion syntax. + frameHeader.AllowHighPrecisionMotionVector = effort >= 8; + frameHeader.InterpolationFilter = effort >= MinimumSwitchableInterpolationEffort + ? Av1InterpolationFilter.Switchable + : Av1InterpolationFilter.Regular; + } + + frameHeader.QuantizationParameters.BaseQIndex = qIndex; + Av1QuantizationLookup.UpdateFrameQuantizationState(frameHeader); + } + + private static int GetTileBufferLength(int width, int height, ObuColorConfig colorConfig) + { + // Libaom reserves 2.5 times the 32-sample-aligned native input for an all-intra output packet. + // Counting the active planes directly retains that headroom without charging monochrome for unused chroma. + // This is an initial estimate: the range writer grows if encoded syntax exceeds its remaining capacity. + int alignedWidth = Av1Math.AlignPowerOf2(width, OutputAlignmentLog2); + int alignedHeight = Av1Math.AlignPowerOf2(height, OutputAlignmentLog2); + int subsamplingX = colorConfig.SubSamplingX ? 1 : 0; + int subsamplingY = colorConfig.SubSamplingY ? 1 : 0; + long sampleCount = (long)alignedWidth * alignedHeight; + if (!colorConfig.IsMonochrome) + { + sampleCount += 2L * (alignedWidth >> subsamplingX) * (alignedHeight >> subsamplingY); + } + + int sampleSize = colorConfig.BitDepth == Av1BitDepth.EightBit ? 1 : 2; + long scaledInputLength = (sampleCount * sampleSize * AllIntraBufferScaleNumerator) + / AllIntraBufferScaleDenominator; + + return checked((int)Math.Max(MinimumCompressedFrameBufferLength, scaledInputLength)); + } + + /// + /// Converts packed pixels directly into an eight-bit bordered AV1 source frame. + /// + /// The packed source pixel type. + /// The configuration used for row-buffer allocation and pixel conversion. + /// The packed source frame. + /// The operation-owned AV1 source planes. + /// The color configuration written to the AV1 sequence header. + public static void PrepareSource( + Configuration configuration, + ImageFrame image, + Av1EncoderFrame source, + ObuColorConfig colorConfig) + where TPixel : unmanaged, IPixel + { + Rectangle sourceRectangle = new(0, 0, image.Width, image.Height); + PrepareSource(configuration, image, sourceRectangle, source, colorConfig, false); + } + + /// + /// Converts packed pixels directly into a high-bit-depth bordered AV1 source frame. + /// + /// The packed source pixel type. + /// The configuration used for row-buffer allocation and pixel conversion. + /// The packed source frame. + /// The operation-owned AV1 source planes. + /// The color configuration written to the AV1 sequence header. + public static void PrepareSource( + Configuration configuration, + ImageFrame image, + Av1EncoderFrame source, + ObuColorConfig colorConfig) + where TPixel : unmanaged, IPixel + { + Rectangle sourceRectangle = new(0, 0, image.Width, image.Height); + PrepareSource(configuration, image, sourceRectangle, source, colorConfig, false); + } + + private static void EncodeByte( + Configuration configuration, + ImageFrame image, + Rectangle sourceRectangle, + Size frameSize, + Stream stream, + ObuSequenceHeader sequenceHeader, + ObuFrameHeader frameHeader, + Av1ColorFormat colorFormat, + int tileBufferLength, + int effort, + bool encodeAlpha) + where TPixel : unmanaged, IPixel + { + using Av1EncoderFrameBuffer source = new( + configuration, + frameSize.Width, + frameSize.Height, + ByteSampleBitDepth, + colorFormat, + chromaPositionX: CenteredChromaSamplePosition, + chromaPositionY: CenteredChromaSamplePosition); + + using Av1EncoderFrameBuffer reconstruction = new( + configuration, + frameSize.Width, + frameSize.Height, + ByteSampleBitDepth, + colorFormat, + chromaPositionX: CenteredChromaSamplePosition, + chromaPositionY: CenteredChromaSamplePosition); + + using Av1EncoderCoefficientBuffer coefficients = new( + configuration, + sequenceHeader, + frameSize.Width, + frameSize.Height); + + using Av1EncoderSuperblockWorkspace superblockWorkspace = new(configuration); + + Av1EncoderTileWorkspace tileWorkspace = new(frameHeader, superblockWorkspace); + using Av1EncoderBlockWorkspace blockWorkspace = new(configuration); + using Av1SymbolEncoder symbolEncoder = new( + configuration, + tileBufferLength, + frameHeader.QuantizationParameters.BaseQIndex, + updateCdf: !frameHeader.DisableCdfUpdate); + + using ObuWriter obuWriter = new(configuration); + + PrepareFrame( + configuration, + image, + sourceRectangle, + source.Frame, + reconstruction.Frame, + sequenceHeader, + frameHeader, + effort, + encodeAlpha); + + using Av1EncoderPictureBuffer picture = new( + configuration, + sequenceHeader, + frameHeader, + source.Frame.Width, + source.Frame.Height, + disallow4x4AllFrames: !frameHeader.CodedLossless && effort < 9); + + Encode( + obuWriter, + stream, + sequenceHeader, + frameHeader, + picture.Picture, + source, + reconstruction, + reconstruction, + coefficients, + tileWorkspace, + blockWorkspace, + symbolEncoder, + effort, + true); + } + + private static void EncodeHighBitDepth( + Configuration configuration, + ImageFrame image, + Rectangle sourceRectangle, + Size frameSize, + Stream stream, + ObuSequenceHeader sequenceHeader, + ObuFrameHeader frameHeader, + Av1ColorFormat colorFormat, + int tileBufferLength, + int effort, + bool encodeAlpha) + where TPixel : unmanaged, IPixel + { + int bitDepth = sequenceHeader.ColorConfig.BitDepth.GetBitCount(); + using Av1EncoderFrameBuffer source = new( + configuration, + frameSize.Width, + frameSize.Height, + bitDepth, + colorFormat, + chromaPositionX: CenteredChromaSamplePosition, + chromaPositionY: CenteredChromaSamplePosition); + + using Av1EncoderFrameBuffer reconstruction = new( + configuration, + frameSize.Width, + frameSize.Height, + bitDepth, + colorFormat, + chromaPositionX: CenteredChromaSamplePosition, + chromaPositionY: CenteredChromaSamplePosition); + + using Av1EncoderCoefficientBuffer coefficients = new( + configuration, + sequenceHeader, + frameSize.Width, + frameSize.Height); + + using Av1EncoderSuperblockWorkspace superblockWorkspace = new(configuration); + + Av1EncoderTileWorkspace tileWorkspace = new(frameHeader, superblockWorkspace); + using Av1EncoderBlockWorkspace blockWorkspace = new(configuration); + using Av1SymbolEncoder symbolEncoder = new( + configuration, + tileBufferLength, + frameHeader.QuantizationParameters.BaseQIndex, + updateCdf: !frameHeader.DisableCdfUpdate); + + using ObuWriter obuWriter = new(configuration); + + PrepareFrame( + configuration, + image, + sourceRectangle, + source.Frame, + reconstruction.Frame, + sequenceHeader, + frameHeader, + effort, + encodeAlpha); + + using Av1EncoderPictureBuffer picture = new( + configuration, + sequenceHeader, + frameHeader, + source.Frame.Width, + source.Frame.Height, + disallow4x4AllFrames: !frameHeader.CodedLossless && effort < 9); + + Encode( + obuWriter, + stream, + sequenceHeader, + frameHeader, + picture.Picture, + source, + reconstruction, + reconstruction, + coefficients, + tileWorkspace, + blockWorkspace, + symbolEncoder, + effort, + true); + } + + /// + /// Converts one source frame and resolves every content-dependent coding tool before picture-state allocation. + /// + private static void PrepareFrame( + Configuration configuration, + ImageFrame image, + Rectangle sourceRectangle, + Av1EncoderFrame source, + Av1EncoderFrame reference, + ObuSequenceHeader sequenceHeader, + ObuFrameHeader frameHeader, + int effort, + bool encodeAlpha) + where TPixel : unmanaged, IPixel + { + PrepareSource( + configuration, + image, + sourceRectangle, + source, + sequenceHeader.ColorConfig, + encodeAlpha); + + ConfigureFrameTools( + source, + reference, + sequenceHeader, + frameHeader, + effort); + } + + /// + /// Converts one sequence sample through its retained row workspace before resolving frame coding tools. + /// + private static void PrepareFrame( + Configuration configuration, + ImageFrame image, + Rectangle sourceRectangle, + Av1EncoderFrame source, + Av1EncoderFrame reference, + ObuSequenceHeader sequenceHeader, + ObuFrameHeader frameHeader, + int effort, + Av1EncoderConversionWorkspace conversionWorkspace) + where TPixel : unmanaged, IPixel + { + PrepareSource( + configuration, + image, + sourceRectangle, + source, + conversionWorkspace); + + ConfigureFrameTools( + source, + reference, + sequenceHeader, + frameHeader, + effort); + } + + /// + /// Resolves the eight-bit frame tools whose syntax depends on the converted source samples. + /// + private static void ConfigureFrameTools( + Av1EncoderFrame source, + Av1EncoderFrame reference, + ObuSequenceHeader sequenceHeader, + ObuFrameHeader frameHeader, + int effort) + { + ConfigureGlobalMotion( + source, + reference, + frameHeader, + sequenceHeader.ColorConfig.BitDepth, + effort); + + bool allowScreenContentTools = false; + bool allowIntraBlockCopy = false; + if (effort >= 5) + { + // Lower effort levels never search palette or intra-block-copy modes, so scanning the complete + // luma plane cannot affect their bitstream decisions. + Av1ScreenContentDetector.Detect( + source, + out allowScreenContentTools, + out allowIntraBlockCopy); + } + + frameHeader.AllowScreenContentTools = allowScreenContentTools; + + // The current intra-block-copy search owns one 8x8 transform. Lossless coding requires reversible + // 4x4 transforms, so palette remains available while this incompatible candidate is omitted. + frameHeader.AllowIntraBlockCopy = + frameHeader.IsIntra && + !frameHeader.CodedLossless && + allowIntraBlockCopy; + } + + /// + /// Converts one high-bit-depth source frame and resolves every content-dependent coding tool before picture-state allocation. + /// + private static void PrepareFrame( + Configuration configuration, + ImageFrame image, + Rectangle sourceRectangle, + Av1EncoderFrame source, + Av1EncoderFrame reference, + ObuSequenceHeader sequenceHeader, + ObuFrameHeader frameHeader, + int effort, + bool encodeAlpha) + where TPixel : unmanaged, IPixel + { + PrepareSource( + configuration, + image, + sourceRectangle, + source, + sequenceHeader.ColorConfig, + encodeAlpha); + + ConfigureFrameTools( + source, + reference, + sequenceHeader, + frameHeader, + effort); + } + + /// + /// Converts one high-bit-depth sequence sample through retained row storage before resolving frame coding tools. + /// + private static void PrepareFrame( + Configuration configuration, + ImageFrame image, + Rectangle sourceRectangle, + Av1EncoderFrame source, + Av1EncoderFrame reference, + ObuSequenceHeader sequenceHeader, + ObuFrameHeader frameHeader, + int effort, + Av1EncoderConversionWorkspace conversionWorkspace) + where TPixel : unmanaged, IPixel + { + PrepareSource( + configuration, + image, + sourceRectangle, + source, + conversionWorkspace); + + ConfigureFrameTools( + source, + reference, + sequenceHeader, + frameHeader, + effort); + } + + /// + /// Resolves the high-bit-depth frame tools whose syntax depends on the converted source samples. + /// + private static void ConfigureFrameTools( + Av1EncoderFrame source, + Av1EncoderFrame reference, + ObuSequenceHeader sequenceHeader, + ObuFrameHeader frameHeader, + int effort) + { + ConfigureGlobalMotion( + source, + reference, + frameHeader, + sequenceHeader.ColorConfig.BitDepth, + effort); + + bool allowScreenContentTools = false; + bool allowIntraBlockCopy = false; + if (effort >= 5) + { + // Lower effort levels never search palette or intra-block-copy modes, so scanning the complete + // luma plane cannot affect their bitstream decisions. + Av1ScreenContentDetector.Detect( + source, + out allowScreenContentTools, + out allowIntraBlockCopy); + } + + frameHeader.AllowScreenContentTools = allowScreenContentTools; + + // The current intra-block-copy search owns one 8x8 transform. Lossless coding requires reversible + // 4x4 transforms, so palette remains available while this incompatible candidate is omitted. + frameHeader.AllowIntraBlockCopy = + frameHeader.IsIntra && + !frameHeader.CodedLossless && + allowIntraBlockCopy; + } + + private static void Encode( + ObuWriter obuWriter, + Stream stream, + ObuSequenceHeader sequenceHeader, + ObuFrameHeader frameHeader, + Av1PictureControlSet picture, + Av1EncoderFrameBuffer source, + Av1EncoderFrameBuffer reference, + Av1EncoderFrameBuffer reconstruction, + Av1EncoderCoefficientBuffer coefficients, + Av1EncoderTileWorkspace tileWorkspace, + Av1EncoderBlockWorkspace blockWorkspace, + Av1SymbolEncoder symbolEncoder, + int effort, + bool writeSequenceHeader) + { + Av1TileEncoder tileWriter = new( + symbolEncoder, + source.Frame, + reference.Frame, + reconstruction.Frame, + picture, + coefficients, + tileWorkspace, + blockWorkspace, + effort); + + if (writeSequenceHeader) + { + obuWriter.WriteSequenceFrame(stream, sequenceHeader, frameHeader, tileWriter); + } + else + { + obuWriter.WriteFrame(stream, sequenceHeader, frameHeader, tileWriter); + } + } + + private static void Encode( + ObuWriter obuWriter, + Stream stream, + ObuSequenceHeader sequenceHeader, + ObuFrameHeader frameHeader, + Av1PictureControlSet picture, + Av1EncoderFrameBuffer source, + Av1EncoderFrameBuffer reference, + Av1EncoderFrameBuffer reconstruction, + Av1EncoderCoefficientBuffer coefficients, + Av1EncoderTileWorkspace tileWorkspace, + Av1EncoderBlockWorkspace blockWorkspace, + Av1SymbolEncoder symbolEncoder, + int effort, + bool writeSequenceHeader) + { + Av1TileEncoder tileWriter = new( + symbolEncoder, + source.Frame, + reference.Frame, + reconstruction.Frame, + picture, + coefficients, + tileWorkspace, + blockWorkspace, + effort); + + if (writeSequenceHeader) + { + obuWriter.WriteSequenceFrame(stream, sequenceHeader, frameHeader, tileWriter); + } + else + { + obuWriter.WriteFrame(stream, sequenceHeader, frameHeader, tileWriter); + } + } + + /// + /// Converts packed pixels into native component planes and initializes every coded and physical edge sample. + /// + private static void PrepareSource( + Configuration configuration, + ImageFrame image, + Rectangle sourceRectangle, + Av1EncoderFrame source, + ObuColorConfig colorConfig, + bool encodeAlpha) + where TPixel : unmanaged, IPixel + where TSample : unmanaged + where TStorer : struct, IHeifSampleConverter + { + Av1EncoderFrame.PlanarView destination = source.CodedView.GetSubView( + sourceRectangle.Width, + sourceRectangle.Height); + + if (encodeAlpha) + { + HeifPlanarAlphaEncoder.Convert< + TPixel, + Av1EncoderFrame.PlanarView, + TSample, + TStorer>( + configuration, + image, + sourceRectangle, + destination); + } + else + { + HeifColorConversionParameters parameters = Av1YuvConverter.GetConversionParameters( + colorConfig, + out HeifColorConversionMode mode); + + // Conversion writes into the final bordered analysis planes. Later coding stages consume the native + // source without a second full-frame copy from an intermediate component buffer. + HeifPlanarColorConverter.ConvertFromRgb< + TPixel, + Av1EncoderFrame.PlanarView, + TSample, + TStorer>( + configuration, + image, + sourceRectangle, + destination, + in parameters, + mode); + } + + // A short grid edge may occupy only the top-left of its AV1 frame. Replicating that edge initializes + // both the remaining coded cell and the physical prediction border without another image or plane copy. + source.CodedView.ExtendBorders(sourceRectangle.Width, sourceRectangle.Height); + } + + /// + /// Converts one sequence sample with track-owned row storage and initializes every coded and physical edge. + /// + private static void PrepareSource( + Configuration configuration, + ImageFrame image, + Rectangle sourceRectangle, + Av1EncoderFrame source, + Av1EncoderConversionWorkspace conversionWorkspace) + where TPixel : unmanaged, IPixel + where TSample : unmanaged + where TStorer : struct, IHeifSampleConverter + { + Av1EncoderFrame.PlanarView destination = source.CodedView.GetSubView( + sourceRectangle.Width, + sourceRectangle.Height); + + conversionWorkspace.Convert< + TPixel, + Av1EncoderFrame.PlanarView, + TSample, + TStorer>( + configuration, + image, + sourceRectangle, + destination); + + // Sequence geometry is fixed, but grid-edge cells can still expose less source data than their coded + // extent. The same edge replication completes both coded padding and the physical prediction border. + source.CodedView.ExtendBorders(sourceRectangle.Width, sourceRectangle.Height); + } + + /// + /// Selects a bounded whole-frame translation model for an inter frame. + /// + private static void ConfigureGlobalMotion( + Av1EncoderFrame source, + Av1EncoderFrame reference, + ObuFrameHeader frameHeader, + Av1BitDepth bitDepth, + int effort) + where TSample : unmanaged + where TOperator : struct, IGlobalMotionSearchOperator + { + Span models = frameHeader.GetGlobalMotionParameters(); + models.Fill(Av1GlobalMotionParameters.Identity); + if (frameHeader.IsIntra || effort < MinimumGlobalMotionSearchEffort) + { + return; + } + + Buffer2DRegion sourceLuma = source.CodedView.GetPlane(Av1Plane.Y); + Buffer2DRegion referenceLuma = reference.CodedView.GetPlane(Av1Plane.Y); + int analysisWidth = Math.Min(source.CodedWidth, MaximumGlobalMotionAnalysisDimension); + int analysisHeight = Math.Min(source.CodedHeight, MaximumGlobalMotionAnalysisDimension); + Point analysisOrigin = new( + (source.CodedWidth - analysisWidth) >> 1, + (source.CodedHeight - analysisHeight) >> 1); + + int effortShift = effort - MinimumGlobalMotionSearchEffort; + int searchRadius = Math.Min( + MinimumGlobalMotionSearchRadius << effortShift, + Av1EncoderFrame.LumaBorder); + + Point bestOffset = default; + long bestAnalysisError = GetGlobalMotionSquaredError( + sourceLuma, + referenceLuma, + analysisOrigin, + analysisWidth, + analysisHeight, + bestOffset); + + for (int step = searchRadius; step > 0; step >>= 1) + { + Point stageBestOffset = bestOffset; + long stageBestError = bestAnalysisError; + for (int directionIndex = 0; directionIndex < GlobalMotionSearchDirectionCount; directionIndex++) + { + Point direction = GetGlobalMotionSearchDirection(directionIndex); + Point candidateOffset = new( + bestOffset.X + (direction.X * step), + bestOffset.Y + (direction.Y * step)); + + if (Math.Abs(candidateOffset.X) > searchRadius || + Math.Abs(candidateOffset.Y) > searchRadius) + { + continue; + } + + long directionError = GetGlobalMotionSquaredError( + sourceLuma, + referenceLuma, + analysisOrigin, + analysisWidth, + analysisHeight, + candidateOffset); + + // Strict replacement preserves identity and the earlier reference search order on ties. + if (directionError < stageBestError) + { + stageBestError = directionError; + stageBestOffset = candidateOffset; + } + } + + bestOffset = stageBestOffset; + bestAnalysisError = stageBestError; + } + + if (bestOffset == default) + { + return; + } + + Point frameOrigin = default; + long identityError = GetGlobalMotionSquaredError( + sourceLuma, + referenceLuma, + frameOrigin, + source.CodedWidth, + source.CodedHeight, + frameOrigin); + + long candidateError = GetGlobalMotionSquaredError( + sourceLuma, + referenceLuma, + frameOrigin, + source.CodedWidth, + source.CodedHeight, + bestOffset); + + Av1GlobalMotionParameters candidate = Av1GlobalMotionParameters.Identity; + + // Pure translation is stored as identity-scale rotation/zoom because the translation-only AV1 model + // has a published row/column assignment defect. The resulting block vector remains exact. + candidate.Type = Av1GlobalMotionType.RotationZoom; + candidate[0] = bestOffset.X * Av1GlobalMotionParameters.ModelScale; + candidate[1] = bestOffset.Y * Av1GlobalMotionParameters.ModelScale; + candidate.UpdateShearParameters(); + + int rateMultiplier = Av1RateDistortion.GetInterFrameRateMultiplier( + frameHeader.QuantizationParameters.BaseQIndex, + bitDepth); + + int identityRate = + ObuWriter.GetGlobalMotionModelBitCount( + Av1GlobalMotionParameters.Identity, + frameHeader.AllowHighPrecisionMotionVector) << + Av1ProbabilityCost.CostShift; + + int candidateRate = + ObuWriter.GetGlobalMotionModelBitCount( + candidate, + frameHeader.AllowHighPrecisionMotionVector) << + Av1ProbabilityCost.CostShift; + + long identityCost = Av1RateDistortion.GetCost( + rateMultiplier, + identityRate, + NormalizeGlobalMotionSquaredError(identityError, bitDepth)); + + long candidateCost = Av1RateDistortion.GetCost( + rateMultiplier, + candidateRate, + NormalizeGlobalMotionSquaredError(candidateError, bitDepth)); + + if (candidateCost < identityCost) + { + models[0] = candidate; + } + } + + /// + /// Calculates squared error for one translated luma candidate using the physical reference border. + /// + private static long GetGlobalMotionSquaredError( + Buffer2DRegion source, + Buffer2DRegion reference, + Point sourceOrigin, + int width, + int height, + Point referenceOffset) + where TSample : unmanaged + where TOperator : struct, IGlobalMotionSearchOperator + { + Rectangle sourceBounds = source.Bounds; + Rectangle referenceBounds = reference.Bounds; + int sourceIndex = + ((sourceBounds.Y + sourceOrigin.Y) * source.Stride) + + sourceBounds.X + + sourceOrigin.X; + + int referenceIndex = + ((referenceBounds.Y + sourceOrigin.Y + referenceOffset.Y) * reference.Stride) + + referenceBounds.X + + sourceOrigin.X + + referenceOffset.X; + + return TOperator.SumSquaredError( + source.Buffer.DangerousGetSingleSpan()[sourceIndex..], + source.Stride, + reference.Buffer.DangerousGetSingleSpan()[referenceIndex..], + reference.Stride, + width, + height); + } + + /// + /// Gets one cardinal or diagonal direction in the reference encoder's search order. + /// + private static Point GetGlobalMotionSearchDirection(int index) + => index switch + { + 0 => new Point(0, -1), + 1 => new Point(0, 1), + 2 => new Point(-1, 0), + 3 => new Point(1, 0), + 4 => new Point(-1, -1), + 5 => new Point(1, 1), + 6 => new Point(1, -1), + _ => new Point(-1, 1) + }; + + /// + /// Normalizes high-bit-depth frame error to the eight-bit distortion domain. + /// + private static long NormalizeGlobalMotionSquaredError(long error, Av1BitDepth bitDepth) + { + int shift = (bitDepth.GetBitCount() - ByteSampleBitDepth) * 2; + return shift == 0 ? error : (error + (1L << (shift - 1))) >> shift; + } + + /// + /// Routes byte samples through the SIMD-first shared residual operation. + /// + private readonly struct ByteGlobalMotionSearchOperator : IGlobalMotionSearchOperator + { + /// + public static long SumSquaredError( + ReadOnlySpan source, + int sourceStride, + ReadOnlySpan prediction, + int predictionStride, + int width, + int height) + => Av1ResidualBuilder.SumSquaredError( + source, + sourceStride, + prediction, + predictionStride, + width, + height); + } + + /// + /// Routes high-bit-depth samples through the SIMD-first shared residual operation. + /// + private readonly struct UInt16GlobalMotionSearchOperator : IGlobalMotionSearchOperator + { + /// + public static long SumSquaredError( + ReadOnlySpan source, + int sourceStride, + ReadOnlySpan prediction, + int predictionStride, + int width, + int height) + => Av1ResidualBuilder.SumSquaredError( + source, + sourceStride, + prediction, + predictionStride, + width, + height); + } + + /// + /// Owns the fixed-size packed and planar row storage reused by every sample in one sequence track. + /// + internal sealed class Av1EncoderConversionWorkspace : IDisposable + { + private readonly IMemoryOwner storageOwner; + private readonly Memory componentMemory; + private readonly Memory packedMemory; + private readonly HeifColorConversionParameters parameters; + private readonly HeifColorConverterBase colorConverter; + private readonly bool encodeAlpha; + private readonly int packedPixelCount; + + /// + /// Initializes a new instance of the class. + /// + /// The configuration providing reusable row storage. + /// The fixed sequence width. + /// The native component layout. + /// Whether the workspace converts the auxiliary alpha track. + /// Whether color conversion requires a packed row. + public Av1EncoderConversionWorkspace( + Configuration configuration, + int width, + ObuColorConfig colorConfig, + bool encodeAlpha, + bool usesHighBitDepth) + { + // Resolve conversion before renting storage: a rejected color description must not strand an owner + // in a constructor that never returns to the sequence encoder's disposal boundary. + this.parameters = Av1YuvConverter.GetConversionParameters(colorConfig, out HeifColorConversionMode mode); + this.colorConverter = HeifColorConverterBase.Create(mode, in this.parameters, colorConfig.IsMonochrome); + int subsamplingY = colorConfig.SubSamplingY ? 1 : 0; + int componentLength = encodeAlpha + ? HeifPlanarAlphaEncoder.GetRowStorageLength(width) + : HeifPlanarColorConverter.GetRgbToYuvComponentBufferLength( + width, + colorConfig.IsMonochrome, + subsamplingY); + + int packedByteLength = usesHighBitDepth && !encodeAlpha + ? width * Unsafe.SizeOf() + : 0; + + int packedFloatLength = (int)Numerics.DivideCeil((uint)packedByteLength, sizeof(float)); + this.storageOwner = configuration.MemoryAllocator.Allocate( + componentLength + packedFloatLength); + + Memory storage = this.storageOwner.Memory; + this.componentMemory = storage[..componentLength]; + this.packedMemory = storage.Slice(componentLength, packedFloatLength); + this.encodeAlpha = encodeAlpha; + this.packedPixelCount = usesHighBitDepth && !encodeAlpha ? width : 0; + } + + /// + /// Converts one packed frame directly into its final native component planes. + /// + /// The packed source pixel type. + /// The native destination-plane adapter. + /// The native sample storage type. + /// The SIMD narrowing and storage operation. + /// The configuration used for packed-pixel conversion. + /// The source image frame. + /// The source region mapped to the destination planes. + /// The destination component planes. + public void Convert( + Configuration configuration, + ImageFrame image, + Rectangle sourceRectangle, + TBuffer buffer) + where TPixel : unmanaged, IPixel + where TBuffer : struct, IHeifPlanarSampleBuffer + where TSample : unmanaged + where TStorer : struct, IHeifSampleConverter + { + if (this.encodeAlpha) + { + HeifPlanarAlphaEncoder.Convert( + configuration, + image, + sourceRectangle, + buffer, + this.componentMemory.Span); + + return; + } + + Span packed = MemoryMarshal.Cast( + this.packedMemory.Span)[..this.packedPixelCount]; + + HeifPlanarColorConverter.ConvertFromRgb( + configuration, + image, + sourceRectangle, + buffer, + in this.parameters, + this.colorConverter, + packed, + this.componentMemory.Span); + } + + /// + public void Dispose() => this.storageOwner.Dispose(); + } + + /// + /// Retains the reconstructed reference state shared by the samples of one AV1 sequence track. + /// + internal abstract class SequenceEncoder : IDisposable + { + protected SequenceEncoder( + Configuration configuration, + int width, + int height, + ObuColorConfig colorConfig, + int qIndex, + int effort, + bool encodeAlpha, + bool usesHighBitDepth) + { + this.Configuration = configuration; + this.SequenceHeader = CreateSequenceHeader(width, height, colorConfig, effort, false); + this.QIndex = qIndex; + this.Effort = effort; + this.TileBufferLength = GetTileBufferLength(width, height, colorConfig); + this.EncodeAlpha = encodeAlpha; + this.FrameHeader = CreateFrameHeader( + width, + height, + qIndex, + effort, + ObuFrameType.KeyFrame); + + this.ConversionWorkspace = new Av1EncoderConversionWorkspace( + configuration, + width, + colorConfig, + encodeAlpha, + usesHighBitDepth); + + try + { + bool allocateScreenContentState = effort >= 5; + bool allocateIntraBlockCopySearch = + allocateScreenContentState && + !this.FrameHeader.CodedLossless; + + // Sequence geometry and maximum tool capacity are fixed before the first sample. Reusing this owner + // avoids renting the complete mode grid and optional screen-content index for every frame. + this.PictureBuffer = new Av1EncoderPictureBuffer( + configuration, + this.SequenceHeader, + this.FrameHeader, + width, + height, + disallow4x4AllFrames: !this.FrameHeader.CodedLossless && effort < 9, + allocateScreenContentState: allocateScreenContentState, + allocateMotionVectorState: true, + allocateIntraBlockCopySearch: allocateIntraBlockCopySearch); + + this.Coefficients = new Av1EncoderCoefficientBuffer( + configuration, + this.SequenceHeader, + width, + height); + + this.SuperblockWorkspace = new Av1EncoderSuperblockWorkspace(configuration); + + this.TileWorkspace = new Av1EncoderTileWorkspace(this.FrameHeader, this.SuperblockWorkspace); + this.BlockWorkspace = new Av1EncoderBlockWorkspace(configuration); + + // Tile probabilities adapt within a sample, while error-resilient frame headers prohibit carrying + // those updates into the next sample. The retained encoder is therefore reset before each frame. + this.SymbolEncoder = new Av1SymbolEncoder( + configuration, + this.TileBufferLength, + qIndex, + updateCdf: true); + + this.ObuWriter = new ObuWriter(configuration); + } + catch + { + // The caller receives no encoder when construction fails. Release only completed common owners; + // derived frame construction has not started and must not be reached through virtual disposal. + this.DisposeResources(); + throw; + } + } + + /// + /// Gets the sequence header shared by every sample written by this encoder. + /// + public ObuSequenceHeader SequenceHeader { get; } + + protected Configuration Configuration { get; } + + protected int QIndex { get; } + + protected int Effort { get; } + + protected int TileBufferLength { get; } + + protected bool EncodeAlpha { get; } + + /// + /// Gets the packed and planar row storage reused by every sample in the track. + /// + protected Av1EncoderConversionWorkspace ConversionWorkspace { get; } + + /// + /// Gets the fixed-geometry picture state reused by every sample in the track. + /// + protected Av1EncoderPictureBuffer PictureBuffer { get; } + + /// + /// Gets the frame header and nested syntax state reused by every sample in the track. + /// + protected ObuFrameHeader FrameHeader { get; } + + /// + /// Gets the frame-sized coefficient storage reused by every sample in the track. + /// + protected Av1EncoderCoefficientBuffer Coefficients { get; } + + /// + /// Gets the superblock decision workspace reused serially across the track. + /// + protected Av1EncoderSuperblockWorkspace SuperblockWorkspace { get; } + + /// + /// Gets the tile, superblock, and entropy cursor graph reused serially across the track. + /// + protected Av1EncoderTileWorkspace TileWorkspace { get; } + + /// + /// Gets the block arithmetic workspace reused serially across the track. + /// + protected Av1EncoderBlockWorkspace BlockWorkspace { get; } + + /// + /// Gets the tile probability graph and bounded output buffer reused by every sample in the track. + /// + protected Av1SymbolEncoder SymbolEncoder { get; } + + /// + /// Gets the reusable OBU header writer for the track. + /// + protected ObuWriter ObuWriter { get; } + + /// + /// Encodes an independently decodable sample with the sequence header required for random access. + /// + public void EncodeKeyFrame(ImageFrame image, Stream stream) + where TPixel : unmanaged, IPixel + => this.EncodeFrame(image, stream, ObuFrameType.KeyFrame, true); + + /// + /// Encodes a continuation sample predicted from the preceding reconstructed frame. + /// + public void EncodeInterFrame(ImageFrame image, Stream stream) + where TPixel : unmanaged, IPixel + => this.EncodeFrame(image, stream, ObuFrameType.InterFrame, false); + + /// + public void Dispose() + { + this.DisposeFrames(); + this.DisposeResources(); + } + + /// + /// Releases the sample-type-specific frame buffers retained by the track encoder. + /// + protected abstract void DisposeFrames(); + + protected abstract void EncodeFrame( + ImageFrame image, + Stream stream, + ObuFrameType frameType, + bool writeSequenceHeader) + where TPixel : unmanaged, IPixel; + + private void DisposeResources() + { + // Construction can stop between any two allocations. Successful instances have every owner; + // failed constructors retain only the prefix completed before the allocator rejected a request. + this.ObuWriter?.Dispose(); + this.SymbolEncoder?.Dispose(); + this.BlockWorkspace?.Dispose(); + this.SuperblockWorkspace?.Dispose(); + this.Coefficients?.Dispose(); + this.PictureBuffer?.Dispose(); + this.ConversionWorkspace.Dispose(); + } + } + + private sealed class ByteSequenceEncoder : SequenceEncoder + { + private readonly Av1EncoderFrameBuffer source; + private Av1EncoderFrameBuffer reference; + private Av1EncoderFrameBuffer reconstruction; + + public ByteSequenceEncoder( + Configuration configuration, + int width, + int height, + ObuColorConfig colorConfig, + int qIndex, + int effort, + bool encodeAlpha) + : base( + configuration, + width, + height, + colorConfig, + qIndex, + effort, + encodeAlpha, + usesHighBitDepth: false) + { + try + { + Av1ColorFormat colorFormat = colorConfig.GetColorFormat(); + this.source = new( + configuration, + width, + height, + ByteSampleBitDepth, + colorFormat, + CenteredChromaSamplePosition, + CenteredChromaSamplePosition); + + this.reference = new( + configuration, + width, + height, + ByteSampleBitDepth, + colorFormat, + CenteredChromaSamplePosition, + CenteredChromaSamplePosition); + + this.reconstruction = new( + configuration, + width, + height, + ByteSampleBitDepth, + colorFormat, + CenteredChromaSamplePosition, + CenteredChromaSamplePosition); + } + catch + { + // The common state already exists, and any preceding frame allocations also need returning. + this.Dispose(); + throw; + } + } + + protected override void DisposeFrames() + { + // A derived constructor can fail before all three frame owners exist. + this.reconstruction?.Dispose(); + this.reference?.Dispose(); + this.source?.Dispose(); + } + + protected override void EncodeFrame( + ImageFrame image, + Stream stream, + ObuFrameType frameType, + bool writeSequenceHeader) + { + ObuFrameHeader frameHeader = this.FrameHeader; + ConfigureFrameHeader( + frameHeader, + this.QIndex, + this.Effort, + frameType); + + Rectangle sourceRectangle = new(0, 0, image.Width, image.Height); + this.SymbolEncoder.Reset(); + PrepareFrame( + this.Configuration, + image, + sourceRectangle, + this.source.Frame, + this.reference.Frame, + this.SequenceHeader, + frameHeader, + this.Effort, + this.ConversionWorkspace); + + this.PictureBuffer.Reset(frameHeader); + Encode( + this.ObuWriter, + stream, + this.SequenceHeader, + frameHeader, + this.PictureBuffer.Picture, + this.source, + this.reference, + this.reconstruction, + this.Coefficients, + this.TileWorkspace, + this.BlockWorkspace, + this.SymbolEncoder, + this.Effort, + writeSequenceHeader); + + this.reconstruction.Frame.ExtendBorders(); + + // The just-reconstructed frame becomes LAST_FRAME for the next sample without copying any plane. + (this.reference, this.reconstruction) = (this.reconstruction, this.reference); + } + } + + private sealed class HighBitDepthSequenceEncoder : SequenceEncoder + { + private readonly Av1EncoderFrameBuffer source; + private Av1EncoderFrameBuffer reference; + private Av1EncoderFrameBuffer reconstruction; + + public HighBitDepthSequenceEncoder( + Configuration configuration, + int width, + int height, + ObuColorConfig colorConfig, + int qIndex, + int effort, + bool encodeAlpha) + : base( + configuration, + width, + height, + colorConfig, + qIndex, + effort, + encodeAlpha, + usesHighBitDepth: true) + { + try + { + int bitDepth = colorConfig.BitDepth.GetBitCount(); + Av1ColorFormat colorFormat = colorConfig.GetColorFormat(); + this.source = new( + configuration, + width, + height, + bitDepth, + colorFormat, + CenteredChromaSamplePosition, + CenteredChromaSamplePosition); + + this.reference = new( + configuration, + width, + height, + bitDepth, + colorFormat, + CenteredChromaSamplePosition, + CenteredChromaSamplePosition); + + this.reconstruction = new( + configuration, + width, + height, + bitDepth, + colorFormat, + CenteredChromaSamplePosition, + CenteredChromaSamplePosition); + } + catch + { + // The common state already exists, and any preceding frame allocations also need returning. + this.Dispose(); + throw; + } + } + + protected override void DisposeFrames() + { + // A derived constructor can fail before all three frame owners exist. + this.reconstruction?.Dispose(); + this.reference?.Dispose(); + this.source?.Dispose(); + } + + protected override void EncodeFrame( + ImageFrame image, + Stream stream, + ObuFrameType frameType, + bool writeSequenceHeader) + { + ObuFrameHeader frameHeader = this.FrameHeader; + ConfigureFrameHeader( + frameHeader, + this.QIndex, + this.Effort, + frameType); + + Rectangle sourceRectangle = new(0, 0, image.Width, image.Height); + this.SymbolEncoder.Reset(); + PrepareFrame( + this.Configuration, + image, + sourceRectangle, + this.source.Frame, + this.reference.Frame, + this.SequenceHeader, + frameHeader, + this.Effort, + this.ConversionWorkspace); + + this.PictureBuffer.Reset(frameHeader); + Encode( + this.ObuWriter, + stream, + this.SequenceHeader, + frameHeader, + this.PictureBuffer.Picture, + this.source, + this.reference, + this.reconstruction, + this.Coefficients, + this.TileWorkspace, + this.BlockWorkspace, + this.SymbolEncoder, + this.Effort, + writeSequenceHeader); + + this.reconstruction.Frame.ExtendBorders(); + + // Swapping the frame owners preserves the complete reconstructed reference, including extended borders. + (this.reference, this.reconstruction) = (this.reconstruction, this.reference); + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1IntraSuperblockEncoder.ChromaModeDecision.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1IntraSuperblockEncoder.ChromaModeDecision.cs new file mode 100644 index 0000000000..ae7283eced --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1IntraSuperblockEncoder.ChromaModeDecision.cs @@ -0,0 +1,1312 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1.Entropy; +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.ChromaFromLuma; +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; +using SixLabors.ImageSharp.Memory; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline; + +/// +/// Provides live-probability chroma mode decisions for intra encoding. +/// +internal static partial class Av1IntraSuperblockEncoder +{ + /// + /// Gets the spatial chroma modes in the order used by the reference encoder. + /// + private static ReadOnlySpan ChromaModeSearchOrder => + [ + Av1ChromaPredictionMode.DC, + Av1ChromaPredictionMode.Horizontal, + Av1ChromaPredictionMode.Vertical, + Av1ChromaPredictionMode.Smooth, + Av1ChromaPredictionMode.Paeth, + Av1ChromaPredictionMode.SmoothVertical, + Av1ChromaPredictionMode.SmoothHorizontal, + Av1ChromaPredictionMode.Directional135Degrees, + Av1ChromaPredictionMode.Directional203Degrees, + Av1ChromaPredictionMode.Directional157Degrees, + Av1ChromaPredictionMode.Directional67Degrees, + Av1ChromaPredictionMode.Directional113Degrees, + Av1ChromaPredictionMode.Directional45Degrees + ]; + + internal partial struct ModeDecision + where TSample : unmanaged + where TOperator : struct, IBlockEncodingOperator + { + private Av1ChromaPredictionMode SelectChromaMode( + Av1SymbolEncoder writer, + Av1MacroBlockD macroBlock, + Av1MacroBlockModeInfo modeInfo, + Point lumaOrigin, + Point chromaOrigin, + Av1BlockSize blockSize, + ushort tileIndex, + Av1PredictionMode lumaMode, + Av1TransformSize transformSize, + Span retainedBlueCoefficients, + Span retainedRedCoefficients, + Span retainedBlueStates, + Span retainedRedStates, + ref Av1EncoderPaletteInfo paletteInfo, + out int selectedAngleDelta, + out byte selectedChromaFromLumaIndex, + out sbyte selectedChromaFromLumaSigns, + out Av1RateDistortionStatistics selectedStatistics) + { + Av1EncoderModeDecisionWorkspace workspace = + this.blockWorkspace.GetModeDecisionWorkspace(); + + ObuColorConfig colorConfig = this.picture.Sequence.SequenceHeader.ColorConfig; + int subsamplingX = colorConfig.SubSamplingX ? 1 : 0; + int subsamplingY = colorConfig.SubSamplingY ? 1 : 0; + int width = transformSize.GetWidth(); + int height = transformSize.GetHeight(); + int sampleCount = transformSize.GetSize2d(); + Av1BlockSize chromaBlockSize = blockSize.GetSubsampled( + colorConfig.SubSamplingX, + colorConfig.SubSamplingY); + + if (chromaBlockSize.GetWidth() > width || chromaBlockSize.GetHeight() > height) + { + return this.SelectTiledChromaMode( + writer, + macroBlock, + modeInfo, + lumaOrigin, + chromaOrigin, + blockSize, + chromaBlockSize, + tileIndex, + lumaMode, + transformSize, + retainedBlueCoefficients, + retainedRedCoefficients, + retainedBlueStates, + retainedRedStates, + paletteInfo, + out selectedAngleDelta, + out selectedChromaFromLumaIndex, + out selectedChromaFromLumaSigns, + out selectedStatistics); + } + + int modeInfoRow = lumaOrigin.Y >> Av1Constants.ModeInfoSizeLog2; + int modeInfoColumn = lumaOrigin.X >> Av1Constants.ModeInfoSizeLog2; + bool hasLeft = macroBlock.IsLeftAvailable; + bool hasAbove = macroBlock.IsUpAvailable; + + // Subsampled chroma belongs to the bottom-right luma unit in its shared 8x8 region. Its external + // references therefore begin before that region, not immediately beside the owning 4x4 luma block. + if (subsamplingX != 0 && blockSize.Get4x4WideCount() < Av1BlockSize.Block8x8.Get4x4WideCount()) + { + hasLeft = modeInfoColumn - 1 > macroBlock.Tile.ModeInfoColumnStart; + } + + if (subsamplingY != 0 && blockSize.Get4x4HighCount() < Av1BlockSize.Block8x8.Get4x4HighCount()) + { + hasAbove = modeInfoRow - 1 > macroBlock.Tile.ModeInfoRowStart; + } + + bool rightAvailable = modeInfoColumn + (transformSize.Get4x4WideCount() << subsamplingX) < macroBlock.Tile.ModeInfoColumnEnd; + bool bottomAvailable = modeInfoRow + (transformSize.Get4x4HighCount() << subsamplingY) < macroBlock.Tile.ModeInfoRowEnd; + Av1PartitionType partitionType = modeInfo.Block.PartitionType; + bool hasTopRight = Av1IntraReferenceAvailability.HasTopRight( + this.picture.Sequence.SequenceHeader.SuperblockSize, + blockSize, + modeInfoRow, + modeInfoColumn, + hasAbove, + rightAvailable, + partitionType, + transformSize, + 0, + 0, + subsamplingX, + subsamplingY); + + bool hasBottomLeft = Av1IntraReferenceAvailability.HasBottomLeft( + this.picture.Sequence.SequenceHeader.SuperblockSize, + blockSize, + modeInfoRow, + modeInfoColumn, + bottomAvailable, + hasLeft, + partitionType, + transformSize, + 0, + 0, + subsamplingX, + subsamplingY); + + Buffer2DRegion blueSource = this.source.GetPlane(Av1Plane.U); + Buffer2DRegion redSource = this.source.GetPlane(Av1Plane.V); + Buffer2DRegion blueReconstruction = this.reconstruction.GetPlane(Av1Plane.U); + Buffer2DRegion redReconstruction = this.reconstruction.GetPlane(Av1Plane.V); + Span blueAboveStorage = workspace.GetReferenceSamples(0); + Span blueLeftStorage = workspace.GetReferenceSamples(1); + Span redAboveStorage = workspace.GetReferenceSamples(2); + Span redLeftStorage = workspace.GetReferenceSamples(3); + PrepareReferenceSamples( + blueReconstruction, + chromaOrigin, + width, + height, + hasLeft, + hasAbove, + hasTopRight, + hasBottomLeft, + this.bitDepth, + blueAboveStorage, + blueLeftStorage); + + PrepareReferenceSamples( + redReconstruction, + chromaOrigin, + width, + height, + hasLeft, + hasAbove, + hasTopRight, + hasBottomLeft, + this.bitDepth, + redAboveStorage, + redLeftStorage); + + ReadOnlySpan blueAbove = blueAboveStorage.Slice(1, width + height); + ReadOnlySpan blueLeft = blueLeftStorage.Slice(1, width + height); + ReadOnlySpan redAbove = redAboveStorage.Slice(1, width + height); + ReadOnlySpan redLeft = redLeftStorage.Slice(1, width + height); + Av1TransformBlockContext blueContext = Av1TileWriter.GetTransformBlockContexts( + Av1ComponentType.Chroma, + this.picture.CbDcSignLevelCoefficientNeighbors[tileIndex], + chromaOrigin, + chromaBlockSize, + transformSize); + + Av1TransformBlockContext redContext = Av1TileWriter.GetTransformBlockContexts( + Av1ComponentType.Chroma, + this.picture.CrDcSignLevelCoefficientNeighbors[tileIndex], + chromaOrigin, + chromaBlockSize, + transformSize); + + Span candidateBlueReconstruction = workspace.GetCandidateReconstruction(0); + Span candidateRedReconstruction = workspace.GetCandidateReconstruction(1); + Span candidateBlueCoefficients = workspace.GetCandidateCoefficients(0); + Span candidateRedCoefficients = workspace.GetCandidateCoefficients(1); + Av1RateDistortionStatistics bestStatistics = Av1RateDistortionStatistics.Invalid; + Av1ChromaPredictionMode bestMode = Av1ChromaPredictionMode.DC; + selectedAngleDelta = 0; + selectedChromaFromLumaIndex = 0; + selectedChromaFromLumaSigns = 0; + int baseModeCount = ChromaModeSearchOrder.Length; + int deltaCount = AngleDeltaSearchOrder.Length; + int directionalModeCount = (int)Av1ChromaPredictionMode.Directional67Degrees - (int)Av1ChromaPredictionMode.Vertical + 1; + + // Effort zero evaluates DC only, effort one adds every zero-angle mode, and higher levels add all directional adjustments. + int candidateCount = this.effort switch + { + 0 => 1, + 1 => baseModeCount, + _ when blockSize >= Av1BlockSize.Block8x8 => baseModeCount + (directionalModeCount * deltaCount), + _ => baseModeCount + }; + + bool hasLumaPalette = paletteInfo.PaletteSizes[0] != 0; + int paletteDisabledCost = Av1TileWriter.IsPaletteAllowed( + this.picture.Parent.FrameHeader.AllowScreenContentTools, + blockSize) + ? writer.GetPaletteUvModeCost(false, hasLumaPalette) + : 0; + + // Spatial base modes precede the six nonzero adjustments for each directional mode. + // Chroma-from-luma remains a separate search because it consumes reconstructed luma AC state. + for (int candidateIndex = 0; candidateIndex < candidateCount; candidateIndex++) + { + Av1ChromaPredictionMode chromaMode; + int angleDelta; + if (candidateIndex < baseModeCount) + { + chromaMode = ChromaModeSearchOrder[candidateIndex]; + angleDelta = 0; + } + else + { + int adjustedIndex = candidateIndex - baseModeCount; + chromaMode = (Av1ChromaPredictionMode)((int)Av1ChromaPredictionMode.Vertical + (adjustedIndex / deltaCount)); + angleDelta = AngleDeltaSearchOrder[adjustedIndex % deltaCount]; + } + + Av1EncoderTransformBlockState candidateBlueState = default; + Av1EncoderTransformBlockState candidateRedState = default; + + // A chroma mode and angle are shared by U and V, so neither plane can replace the + // retained result independently. Their complete rate and distortion compete jointly. + Av1RateDistortionStatistics candidateStatistics = this.GetChromaCandidateCost( + writer, + modeInfo, + lumaMode, + chromaMode, + angleDelta, + blockSize, + chromaOrigin, + transformSize, + blueSource, + redSource, + blueAbove, + blueLeft, + redAbove, + redLeft, + hasLeft, + hasAbove, + this.UseSmoothIntraEdges(macroBlock, lumaOrigin, blockSize, Av1Plane.U), + blueContext, + redContext, + paletteDisabledCost, + candidateBlueReconstruction[..sampleCount], + candidateRedReconstruction[..sampleCount], + candidateBlueCoefficients[..sampleCount], + candidateRedCoefficients[..sampleCount], + ref candidateBlueState, + ref candidateRedState); + + if (candidateStatistics.Cost < bestStatistics.Cost) + { + CopyCandidate( + candidateBlueReconstruction, + candidateBlueCoefficients, + blueReconstruction, + chromaOrigin, + retainedBlueCoefficients, + transformSize, + candidateBlueState, + ref retainedBlueStates[0]); + + CopyCandidate( + candidateRedReconstruction, + candidateRedCoefficients, + redReconstruction, + chromaOrigin, + retainedRedCoefficients, + transformSize, + candidateRedState, + ref retainedRedStates[0]); + + bestStatistics = candidateStatistics; + bestMode = chromaMode; + selectedAngleDelta = angleDelta; + } + } + + bool chromaFromLumaAllowed = blockSize.AllowsChromaFromLuma( + this.picture.Parent.FrameHeader.LosslessArray[modeInfo.Block.SegmentId], + colorConfig.SubSamplingX, + colorConfig.SubSamplingY); + + if (this.effort >= 4 && chromaFromLumaAllowed) + { + Span lumaQ3 = workspace.ChromaFromLumaSamples; + Point chromaLumaOrigin = new( + chromaOrigin.X << subsamplingX, + chromaOrigin.Y << subsamplingY); + + // CfL consumes the complete luma region represented by this chroma block, which starts before + // the bottom-right ownership point for subsampled 4x4 luma leaves. + TOperator.PrepareChromaFromLuma( + this.reconstruction.GetPlane(Av1Plane.Y), + chromaLumaOrigin, + lumaQ3, + transformSize, + colorConfig.SubSamplingX, + colorConfig.SubSamplingY); + + // Every alpha candidate uses the same constant DC predictor, so compute each plane once and + // refill the candidate block from its sample instead of rebuilding the identical edge average. + TOperator.PrepareChromaFromLumaDc( + candidateBlueReconstruction[..sampleCount], + blueAbove, + blueLeft, + hasLeft, + hasAbove, + transformSize, + this.bitDepth); + + TSample blueDc = candidateBlueReconstruction[0]; + TOperator.PrepareChromaFromLumaDc( + candidateRedReconstruction[..sampleCount], + redAbove, + redLeft, + hasLeft, + hasAbove, + transformSize, + this.bitDepth); + + TSample redDc = candidateRedReconstruction[0]; + Span blueRates = workspace.GetChromaFromLumaRates(0); + Span redRates = workspace.GetChromaFromLumaRates(1); + Span blueDistortions = workspace.GetChromaFromLumaDistortions(0); + Span redDistortions = workspace.GetChromaFromLumaDistortions(1); + + // Each plane has only 33 signed alpha values. Caching those complete transform results reduces + // the joint search from 1089 transform pairs to 66 transforms plus inexpensive rate combinations. + for (int alphaCandidateIndex = 0; alphaCandidateIndex < Av1ChromaFromLumaMath.AlphaCandidateCount; alphaCandidateIndex++) + { + int alphaQ3 = Av1ChromaFromLumaMath.CandidateIndexToAlpha(alphaCandidateIndex); + Av1EncoderTransformBlockState candidateBlueState = default; + blueDistortions[alphaCandidateIndex] = this.GetChromaFromLumaPlaneCost( + writer, + lumaMode, + Av1Plane.U, + chromaOrigin, + transformSize, + blueSource, + blueDc, + blueContext, + lumaQ3, + alphaQ3, + candidateBlueReconstruction[..sampleCount], + candidateBlueCoefficients[..sampleCount], + ref candidateBlueState, + out blueRates[alphaCandidateIndex]); + + Av1EncoderTransformBlockState candidateRedState = default; + redDistortions[alphaCandidateIndex] = this.GetChromaFromLumaPlaneCost( + writer, + lumaMode, + Av1Plane.V, + chromaOrigin, + transformSize, + redSource, + redDc, + redContext, + lumaQ3, + alphaQ3, + candidateRedReconstruction[..sampleCount], + candidateRedCoefficients[..sampleCount], + ref candidateRedState, + out redRates[alphaCandidateIndex]); + } + + int chromaFromLumaModeRate = Av1TileWriter.GetChromaModeCost( + writer, + this.picture.Parent.FrameHeader, + colorConfig, + modeInfo, + blockSize, + lumaMode, + Av1ChromaPredictionMode.ChromaFromLuma, + 0); + + bool chromaFromLumaSelected = false; + int selectedBlueCandidateIndex = 0; + int selectedRedCandidateIndex = 0; + for (int blueCandidateIndex = 0; blueCandidateIndex < Av1ChromaFromLumaMath.AlphaCandidateCount; blueCandidateIndex++) + { + int alphaU = Av1ChromaFromLumaMath.CandidateIndexToAlpha(blueCandidateIndex); + int signU = Av1ChromaFromLumaMath.AlphaToSign(alphaU); + int indexU = Av1ChromaFromLumaMath.AlphaToMagnitudeIndex(alphaU); + for (int redCandidateIndex = 0; redCandidateIndex < Av1ChromaFromLumaMath.AlphaCandidateCount; redCandidateIndex++) + { + int alphaV = Av1ChromaFromLumaMath.CandidateIndexToAlpha(redCandidateIndex); + int signV = Av1ChromaFromLumaMath.AlphaToSign(alphaV); + if (signU == Av1ChromaFromLumaMath.SignZero && signV == Av1ChromaFromLumaMath.SignZero) + { + continue; + } + + int indexV = Av1ChromaFromLumaMath.AlphaToMagnitudeIndex(alphaV); + int jointSign = Av1ChromaFromLumaMath.JointSign(signU, signV); + int packedIndex = Av1ChromaFromLumaMath.PackIndices(indexU, indexV); + int rate = chromaFromLumaModeRate + + blueRates[blueCandidateIndex] + + redRates[redCandidateIndex] + + writer.GetChromaFromLumaCost(packedIndex, jointSign); + + long distortion = blueDistortions[blueCandidateIndex] + redDistortions[redCandidateIndex]; + Av1RateDistortionStatistics candidateStatistics = new(this.rateMultiplier, rate, distortion); + bool winsSearchOrderTie = candidateStatistics.Cost == bestStatistics.Cost + && !chromaFromLumaSelected + && bestMode != Av1ChromaPredictionMode.DC; + + // CfL follows DC and precedes every other chroma mode in the reference search order. + if (candidateStatistics.Cost < bestStatistics.Cost || winsSearchOrderTie) + { + bestStatistics = candidateStatistics; + bestMode = Av1ChromaPredictionMode.ChromaFromLuma; + selectedAngleDelta = 0; + selectedBlueCandidateIndex = blueCandidateIndex; + selectedRedCandidateIndex = redCandidateIndex; + selectedChromaFromLumaIndex = (byte)packedIndex; + selectedChromaFromLumaSigns = (sbyte)jointSign; + chromaFromLumaSelected = true; + } + } + } + + if (chromaFromLumaSelected) + { + // The alpha tables retain only rate and distortion. Regenerate the two selected planes + // once here instead of copying reconstruction and coefficient blocks for all 66 trials. + Av1EncoderTransformBlockState candidateBlueState = default; + _ = this.GetChromaFromLumaPlaneCost( + writer, + lumaMode, + Av1Plane.U, + chromaOrigin, + transformSize, + blueSource, + blueDc, + blueContext, + lumaQ3, + Av1ChromaFromLumaMath.CandidateIndexToAlpha(selectedBlueCandidateIndex), + candidateBlueReconstruction[..sampleCount], + candidateBlueCoefficients[..sampleCount], + ref candidateBlueState, + out _); + + CopyCandidate( + candidateBlueReconstruction, + candidateBlueCoefficients, + blueReconstruction, + chromaOrigin, + retainedBlueCoefficients, + transformSize, + candidateBlueState, + ref retainedBlueStates[0]); + + Av1EncoderTransformBlockState candidateRedState = default; + _ = this.GetChromaFromLumaPlaneCost( + writer, + lumaMode, + Av1Plane.V, + chromaOrigin, + transformSize, + redSource, + redDc, + redContext, + lumaQ3, + Av1ChromaFromLumaMath.CandidateIndexToAlpha(selectedRedCandidateIndex), + candidateRedReconstruction[..sampleCount], + candidateRedCoefficients[..sampleCount], + ref candidateRedState, + out _); + + CopyCandidate( + candidateRedReconstruction, + candidateRedCoefficients, + redReconstruction, + chromaOrigin, + retainedRedCoefficients, + transformSize, + candidateRedState, + ref retainedRedStates[0]); + } + } + + if (this.effort >= 5 && + blockSize == Av1BlockSize.Block8x8 && + this.picture.Parent.FrameHeader.AllowScreenContentTools && + this.SelectChromaPalette( + writer, + macroBlock, + modeInfo, + lumaOrigin, + chromaOrigin, + tileIndex, + lumaMode, + transformSize, + blueContext, + redContext, + candidateBlueReconstruction[..sampleCount], + candidateRedReconstruction[..sampleCount], + candidateBlueCoefficients[..sampleCount], + candidateRedCoefficients[..sampleCount], + retainedBlueCoefficients, + retainedRedCoefficients, + ref retainedBlueStates[0], + ref retainedRedStates[0], + ref bestStatistics, + ref paletteInfo)) + { + bestMode = Av1ChromaPredictionMode.DC; + selectedAngleDelta = 0; + selectedChromaFromLumaIndex = 0; + selectedChromaFromLumaSigns = 0; + } + + selectedStatistics = bestStatistics; + return bestMode; + } + + private Av1ChromaPredictionMode SelectTiledChromaMode( + Av1SymbolEncoder writer, + Av1MacroBlockD macroBlock, + Av1MacroBlockModeInfo modeInfo, + Point lumaOrigin, + Point chromaOrigin, + Av1BlockSize blockSize, + Av1BlockSize chromaBlockSize, + ushort tileIndex, + Av1PredictionMode lumaMode, + Av1TransformSize transformSize, + Span retainedBlueCoefficients, + Span retainedRedCoefficients, + Span retainedBlueStates, + Span retainedRedStates, + Av1EncoderPaletteInfo paletteInfo, + out int selectedAngleDelta, + out byte selectedChromaFromLumaIndex, + out sbyte selectedChromaFromLumaSigns, + out Av1RateDistortionStatistics selectedStatistics) + { + Av1EncoderModeDecisionWorkspace workspace = + this.blockWorkspace.GetModeDecisionWorkspace(); + + ObuColorConfig colorConfig = this.picture.Sequence.SequenceHeader.ColorConfig; + int subsamplingX = colorConfig.SubSamplingX ? 1 : 0; + int subsamplingY = colorConfig.SubSamplingY ? 1 : 0; + int blockWidth = chromaBlockSize.GetWidth(); + int blockHeight = chromaBlockSize.GetHeight(); + int blockSampleCount = blockWidth * blockHeight; + int transformWidth = transformSize.GetWidth(); + int transformHeight = transformSize.GetHeight(); + int transformColumnCount = blockWidth / transformWidth; + int transformRowCount = blockHeight / transformHeight; + int transformBlockCount = transformColumnCount * transformRowCount; + Av1BlockSize maximumUnitBlockSize = + Av1BlockSize.Block64x64.GetSubsampled(colorConfig.SubSamplingX, colorConfig.SubSamplingY); + + Span candidateBlueReconstruction = + workspace.GetCandidateReconstruction(0)[..blockSampleCount]; + + Span candidateRedReconstruction = + workspace.GetCandidateReconstruction(1)[..blockSampleCount]; + + Span candidateBlueCoefficients = + workspace.GetCandidateCoefficients(0)[..blockSampleCount]; + + Span candidateRedCoefficients = + workspace.GetCandidateCoefficients(1)[..blockSampleCount]; + + Span candidateStates = workspace.CandidateTransformBlocks; + Span candidateBlueStates = + candidateStates[..transformBlockCount]; + + Span candidateRedStates = + candidateStates.Slice(transformBlockCount, transformBlockCount); + + int contextWidth = chromaBlockSize.Get4x4WideCount(); + int contextHeight = chromaBlockSize.Get4x4HighCount(); + Span contexts = workspace.TransformContexts; + Span blueTopContexts = contexts[..contextWidth]; + Span blueLeftContexts = contexts.Slice(contextWidth, contextHeight); + Span redTopContexts = contexts.Slice(contextWidth + contextHeight, contextWidth); + Span redLeftContexts = contexts.Slice( + (2 * contextWidth) + contextHeight, + contextHeight); + + Av1NeighborArrayUnit blueNeighbors = + this.picture.CbDcSignLevelCoefficientNeighbors[tileIndex]; + + Av1NeighborArrayUnit redNeighbors = + this.picture.CrDcSignLevelCoefficientNeighbors[tileIndex]; + + int blueTopIndex = blueNeighbors.GetTopIndex(chromaOrigin); + int blueLeftIndex = blueNeighbors.GetLeftIndex(chromaOrigin); + int redTopIndex = redNeighbors.GetTopIndex(chromaOrigin); + int redLeftIndex = redNeighbors.GetLeftIndex(chromaOrigin); + Buffer2DRegion blueSource = this.source.GetPlane(Av1Plane.U); + Buffer2DRegion redSource = this.source.GetPlane(Av1Plane.V); + Buffer2DRegion blueReconstruction = this.reconstruction.GetPlane(Av1Plane.U); + Buffer2DRegion redReconstruction = this.reconstruction.GetPlane(Av1Plane.V); + bool hasLumaPalette = paletteInfo.PaletteSizes[0] != 0; + int paletteDisabledCost = Av1TileWriter.IsPaletteAllowed( + this.picture.Parent.FrameHeader.AllowScreenContentTools, + blockSize) + ? writer.GetPaletteUvModeCost(false, hasLumaPalette) + : 0; + + int baseModeCount = ChromaModeSearchOrder.Length; + int deltaCount = AngleDeltaSearchOrder.Length; + int directionalModeCount = + (int)Av1ChromaPredictionMode.Directional67Degrees - + (int)Av1ChromaPredictionMode.Vertical + + 1; + + // Multiple transforms do not expand the block's prediction syntax. Preserve the ordinary + // effort tiers and exclude angle adjustments for 4x8/8x4 blocks, where no delta is signaled. + int candidateCount = this.effort switch + { + 0 => 1, + 1 => baseModeCount, + _ when blockSize >= Av1BlockSize.Block8x8 => baseModeCount + (directionalModeCount * deltaCount), + _ => baseModeCount + }; + + Av1RateDistortionStatistics bestStatistics = Av1RateDistortionStatistics.Invalid; + Av1ChromaPredictionMode bestMode = Av1ChromaPredictionMode.DC; + selectedAngleDelta = 0; + selectedChromaFromLumaIndex = 0; + selectedChromaFromLumaSigns = 0; + + // A large chroma block is predicted and transformed in the same raster order used by the tile + // writer. Each completed transform supplies both reconstructed edges and coefficient contexts. + for (int candidateIndex = 0; candidateIndex < candidateCount; candidateIndex++) + { + Av1ChromaPredictionMode chromaMode; + int angleDelta; + if (candidateIndex < baseModeCount) + { + chromaMode = ChromaModeSearchOrder[candidateIndex]; + angleDelta = 0; + } + else + { + int adjustedIndex = candidateIndex - baseModeCount; + chromaMode = (Av1ChromaPredictionMode)( + (int)Av1ChromaPredictionMode.Vertical + (adjustedIndex / deltaCount)); + + angleDelta = AngleDeltaSearchOrder[adjustedIndex % deltaCount]; + } + + blueNeighbors.Top.Slice(blueTopIndex, contextWidth).CopyTo(blueTopContexts); + blueNeighbors.Left.Slice(blueLeftIndex, contextHeight).CopyTo(blueLeftContexts); + redNeighbors.Top.Slice(redTopIndex, contextWidth).CopyTo(redTopContexts); + redNeighbors.Left.Slice(redLeftIndex, contextHeight).CopyTo(redLeftContexts); + Av1PredictionMode predictionMode = chromaMode.ToLumaMode(); + long distortion = this.GetTiledPlaneCost( + writer, + macroBlock, + lumaOrigin, + chromaOrigin, + blockSize, + chromaBlockSize, + transformSize, + maximumUnitBlockSize, + subsamplingX, + subsamplingY, + lumaMode, + predictionMode, + angleDelta, + Av1Plane.U, + blueSource, + blueReconstruction, + candidateBlueReconstruction, + candidateBlueCoefficients, + candidateBlueStates, + blueTopContexts, + blueLeftContexts, + out int blueRate); + + distortion += this.GetTiledPlaneCost( + writer, + macroBlock, + lumaOrigin, + chromaOrigin, + blockSize, + chromaBlockSize, + transformSize, + maximumUnitBlockSize, + subsamplingX, + subsamplingY, + lumaMode, + predictionMode, + angleDelta, + Av1Plane.V, + redSource, + redReconstruction, + candidateRedReconstruction, + candidateRedCoefficients, + candidateRedStates, + redTopContexts, + redLeftContexts, + out int redRate); + + int rate = Av1TileWriter.GetChromaModeCost( + writer, + this.picture.Parent.FrameHeader, + colorConfig, + modeInfo, + blockSize, + lumaMode, + chromaMode, + angleDelta); + + rate += blueRate + redRate; + if (chromaMode == Av1ChromaPredictionMode.DC) + { + rate += paletteDisabledCost; + } + + Av1RateDistortionStatistics candidateStatistics = new(this.rateMultiplier, rate, distortion); + if (candidateStatistics.Cost < bestStatistics.Cost) + { + CopyTiledCandidate( + candidateBlueReconstruction, + candidateBlueCoefficients, + candidateBlueStates, + blueReconstruction, + chromaOrigin, + blockWidth, + blockHeight, + transformSize, + retainedBlueCoefficients, + retainedBlueStates); + + CopyTiledCandidate( + candidateRedReconstruction, + candidateRedCoefficients, + candidateRedStates, + redReconstruction, + chromaOrigin, + blockWidth, + blockHeight, + transformSize, + retainedRedCoefficients, + retainedRedStates); + + bestStatistics = candidateStatistics; + bestMode = chromaMode; + selectedAngleDelta = angleDelta; + } + } + + selectedStatistics = bestStatistics; + return bestMode; + } + + private long GetTiledPlaneCost( + Av1SymbolEncoder writer, + Av1MacroBlockD macroBlock, + Point lumaOrigin, + Point chromaOrigin, + Av1BlockSize blockSize, + Av1BlockSize chromaBlockSize, + Av1TransformSize transformSize, + Av1BlockSize maximumUnitBlockSize, + int subsamplingX, + int subsamplingY, + Av1PredictionMode lumaMode, + Av1PredictionMode predictionMode, + int angleDelta, + Av1Plane plane, + Buffer2DRegion source, + Buffer2DRegion reconstruction, + Span candidateReconstruction, + Span candidateCoefficients, + Span candidateStates, + Span topContexts, + Span leftContexts, + out int rate) + { + Av1EncoderModeDecisionWorkspace workspace = + this.blockWorkspace.GetModeDecisionWorkspace(); + + int blockWidth = chromaBlockSize.GetWidth(); + int blockHeight = chromaBlockSize.GetHeight(); + int transformWidth = transformSize.GetWidth(); + int transformHeight = transformSize.GetHeight(); + int transformSampleCount = transformSize.GetSize2d(); + int transformWidth4x4 = transformSize.Get4x4WideCount(); + int transformHeight4x4 = transformSize.Get4x4HighCount(); + int maximumUnitWidth = Math.Min(maximumUnitBlockSize.GetWidth(), blockWidth); + int maximumUnitHeight = Math.Min(maximumUnitBlockSize.GetHeight(), blockHeight); + Av1TransformType transformType = this.picture.Parent.FrameHeader.CodedLossless + ? Av1TransformType.DctDct + : Av1SymbolContextHelper.GetDefaultIntraTransformType( + predictionMode, + transformSize, + this.picture.Parent.FrameHeader.UseReducedTransformSet); + + Av1ComponentType componentType = plane == Av1Plane.Y + ? Av1ComponentType.Luminance + : Av1ComponentType.Chroma; + + Span prediction = workspace.Prediction[..transformSampleCount]; + Span residual = workspace.Residual[..transformSampleCount]; + Span aboveStorage = workspace.GetReferenceSamples(0); + Span leftStorage = workspace.GetReferenceSamples(1); + int coefficientOffset = 0; + int transformIndex = 0; + long distortion = 0; + rate = 0; + + // Residual syntax completes each bounded 64x64 luma region, scaled for chroma, before + // moving to the next region. Candidate coefficients and states must retain that exact order. + for (int regionRow = 0; regionRow < blockHeight; regionRow += maximumUnitHeight) + { + int unitBottom = Math.Min(regionRow + maximumUnitHeight, blockHeight); + for (int regionColumn = 0; regionColumn < blockWidth; regionColumn += maximumUnitWidth) + { + int unitRight = Math.Min(regionColumn + maximumUnitWidth, blockWidth); + for (int rowOffset = regionRow; rowOffset < unitBottom; rowOffset += transformHeight) + { + int transformRow = rowOffset / transformHeight; + for (int columnOffset = regionColumn; columnOffset < unitRight; columnOffset += transformWidth) + { + int transformColumn = columnOffset / transformWidth; + int reconstructionOffset = (rowOffset * blockWidth) + columnOffset; + Point transformOrigin = chromaOrigin + new Size(columnOffset, rowOffset); + this.PrepareTransformReferenceSamples( + reconstruction, + lumaOrigin, + chromaOrigin, + blockSize, + macroBlock, + transformRow, + transformColumn, + blockWidth, + transformSize, + subsamplingX, + subsamplingY, + candidateReconstruction, + aboveStorage, + leftStorage, + out bool hasLeft, + out bool hasAbove); + + TOperator.PrepareIntra( + this.blockWorkspace, + source, + transformOrigin, + prediction, + aboveStorage.Slice(1, transformWidth + transformHeight), + leftStorage.Slice(1, transformWidth + transformHeight), + hasLeft, + hasAbove, + predictionMode, + angleDelta, + this.picture.Sequence.SequenceHeader.EnableIntraEdgeFilter, + this.UseSmoothIntraEdges(macroBlock, lumaOrigin, blockSize, plane), + residual, + transformSize, + this.bitDepth); + + Av1TransformBlockContext blockContext = Av1TileWriter.GetTransformBlockContexts( + componentType, + topContexts.Slice(transformColumn * transformWidth4x4, transformWidth4x4), + leftContexts.Slice(transformRow * transformHeight4x4, transformHeight4x4), + chromaBlockSize, + transformSize); + + Span transformCoefficients = candidateCoefficients.Slice( + coefficientOffset, + transformSampleCount); + + ref Av1EncoderTransformBlockState state = ref candidateStates[transformIndex++]; + distortion += TOperator.EncodePredictionCandidate( + this.blockWorkspace, + source, + transformOrigin, + prediction, + residual, + candidateReconstruction[reconstructionOffset..], + blockWidth, + transformCoefficients, + transformSize, + transformType, + plane, + this.quantization.QIndex[0], + this.quantization.DeltaQDc[(int)plane], + this.quantization.DeltaQAc[(int)plane], + this.bitDepth, + ref state); + + rate += writer.GetCoefficientCost( + transformSize, + transformType, + lumaMode, + transformCoefficients, + componentType, + blockContext, + state.EndOfBlock, + this.picture.Parent.FrameHeader.UseReducedTransformSet, + Av1FilterIntraMode.AllFilterIntraModes, + usesInterTransformSet: false); + + byte coefficientContext = Av1SymbolContextHelper.GetCoefficientContext( + transformCoefficients, + transformSize, + transformType, + state.EndOfBlock); + + topContexts + .Slice(transformColumn * transformWidth4x4, transformWidth4x4) + .Fill(coefficientContext); + + leftContexts + .Slice(transformRow * transformHeight4x4, transformHeight4x4) + .Fill(coefficientContext); + + coefficientOffset += transformSampleCount; + } + } + } + } + + return distortion; + } + + private static void CopyTiledCandidate( + ReadOnlySpan candidateReconstruction, + ReadOnlySpan candidateCoefficients, + ReadOnlySpan candidateStates, + Buffer2DRegion reconstruction, + Point blockOrigin, + int blockWidth, + int blockHeight, + Av1TransformSize transformSize, + Span retainedCoefficients, + Span retainedStates) + { + int blockSampleCount = blockWidth * blockHeight; + int transformStateStride = + transformSize.GetSize2d() / + Av1EncoderCoefficientBuffer.TransformBlockUnitCoefficientCount; + + candidateCoefficients[..blockSampleCount].CopyTo(retainedCoefficients); + for (int transformIndex = 0; transformIndex < candidateStates.Length; transformIndex++) + { + retainedStates[transformIndex * transformStateStride] = candidateStates[transformIndex]; + } + + for (int row = 0; row < blockHeight; row++) + { + candidateReconstruction.Slice(row * blockWidth, blockWidth) + .CopyTo(reconstruction.DangerousGetRowSpan(blockOrigin.Y + row).Slice(blockOrigin.X, blockWidth)); + } + } + + private long GetChromaFromLumaPlaneCost( + Av1SymbolEncoder writer, + Av1PredictionMode lumaMode, + Av1Plane plane, + Point chromaOrigin, + Av1TransformSize transformSize, + Buffer2DRegion source, + TSample dc, + Av1TransformBlockContext context, + ReadOnlySpan lumaQ3, + int alphaQ3, + Span reconstruction, + Span coefficients, + ref Av1EncoderTransformBlockState state, + out int rate) + { + long distortion = TOperator.EncodeChromaFromLumaCandidate( + this.blockWorkspace, + source, + chromaOrigin, + reconstruction, + dc, + lumaQ3, + alphaQ3, + coefficients, + transformSize, + plane, + this.quantization.QIndex[0], + this.quantization.DeltaQDc[(int)plane], + this.quantization.DeltaQAc[(int)plane], + this.bitDepth, + ref state); + + rate = writer.GetCoefficientCost( + transformSize, + Av1TransformType.DctDct, + lumaMode, + coefficients, + Av1ComponentType.Chroma, + context, + state.EndOfBlock, + this.picture.Parent.FrameHeader.UseReducedTransformSet, + Av1FilterIntraMode.AllFilterIntraModes, + usesInterTransformSet: false); + + return distortion; + } + + /// + /// Derives the directional edge-filter class from the relevant neighboring coding blocks. + /// + private bool UseSmoothIntraEdges(Av1MacroBlockD macroBlock, Point lumaOrigin, Av1BlockSize blockSize, Av1Plane plane) + { + ObuColorConfig colorConfig = this.picture.Sequence.SequenceHeader.ColorConfig; + int subX = plane == Av1Plane.Y ? 0 : colorConfig.SubSamplingX ? 1 : 0; + int subY = plane == Av1Plane.Y ? 0 : colorConfig.SubSamplingY ? 1 : 0; + int row = lumaOrigin.Y >> Av1Constants.ModeInfoSizeLog2; + int column = lumaOrigin.X >> Av1Constants.ModeInfoSizeLog2; + bool hasAbove = macroBlock.IsUpAvailable; + bool hasLeft = macroBlock.IsLeftAvailable; + if (subX != 0 && blockSize.Get4x4WideCount() < 2) + { + hasLeft = column - 1 > macroBlock.Tile.ModeInfoColumnStart; + } + + if (subY != 0 && blockSize.Get4x4HighCount() < 2) + { + hasAbove = row - 1 > macroBlock.Tile.ModeInfoRowStart; + } + + // Chroma may cover several luma units. Its neighbors are the bottom-right luma units in the + // adjacent chroma regions, measured from the top-left unit covered by the current chroma block. + int baseOffset = -((row & subY) * macroBlock.ModeInfoStride) - (column & subX); + if (hasAbove && IsSmoothIntraNeighbor( + macroBlock.GetRelativeModeInfo(baseOffset - macroBlock.ModeInfoStride + subX).Block, plane)) + { + return true; + } + + return hasLeft && IsSmoothIntraNeighbor( + macroBlock.GetRelativeModeInfo(baseOffset + (subY * macroBlock.ModeInfoStride) - 1).Block, plane); + } + + /// + /// Determines whether a neighboring block supplies the smooth edge-filter class. + /// + private static bool IsSmoothIntraNeighbor(Av1EncoderBlockModeInfo modeInfo, Av1Plane plane) + { + if (plane == Av1Plane.Y) + { + return modeInfo.Mode is Av1PredictionMode.Smooth or Av1PredictionMode.SmoothVertical or Av1PredictionMode.SmoothHorizontal; + } + + // An inter winner can retain the preceding intra trial's UV field. That field has no inter + // meaning, so only an ordinary intra neighbor can select chroma smooth-edge thresholds. + return !modeInfo.UseIntraBlockCopy && modeInfo.Mode < Av1PredictionMode.InterModeStart + && modeInfo.UvMode is Av1ChromaPredictionMode.Smooth or Av1ChromaPredictionMode.SmoothVertical or Av1ChromaPredictionMode.SmoothHorizontal; + } + + private Av1RateDistortionStatistics GetChromaCandidateCost( + Av1SymbolEncoder writer, + Av1MacroBlockModeInfo modeInfo, + Av1PredictionMode lumaMode, + Av1ChromaPredictionMode chromaMode, + int angleDelta, + Av1BlockSize blockSize, + Point chromaOrigin, + Av1TransformSize transformSize, + Buffer2DRegion blueSource, + Buffer2DRegion redSource, + ReadOnlySpan blueAbove, + ReadOnlySpan blueLeft, + ReadOnlySpan redAbove, + ReadOnlySpan redLeft, + bool hasLeft, + bool hasAbove, + bool smoothIntraEdges, + Av1TransformBlockContext blueContext, + Av1TransformBlockContext redContext, + int paletteDisabledCost, + Span candidateBlueReconstruction, + Span candidateRedReconstruction, + Span candidateBlueCoefficients, + Span candidateRedCoefficients, + ref Av1EncoderTransformBlockState candidateBlueState, + ref Av1EncoderTransformBlockState candidateRedState) + { + Av1PredictionMode predictionMode = chromaMode.ToLumaMode(); + + // Intra chroma derives one transform type from the shared UV prediction mode. The type is not + // signaled independently for either chroma plane, so U and V must use the same legal fallback. + Av1TransformType transformType = this.picture.Parent.FrameHeader.CodedLossless + ? Av1TransformType.DctDct + : Av1SymbolContextHelper.GetDefaultIntraTransformType( + predictionMode, + transformSize, + this.picture.Parent.FrameHeader.UseReducedTransformSet); + + long distortion = TOperator.EncodeCandidate( + this.blockWorkspace, + blueSource, + chromaOrigin, + candidateBlueReconstruction, + blueAbove, + blueLeft, + hasLeft, + hasAbove, + predictionMode, + angleDelta, + this.picture.Sequence.SequenceHeader.EnableIntraEdgeFilter, + smoothIntraEdges, + candidateBlueCoefficients, + transformSize, + transformType, + Av1Plane.U, + this.quantization.QIndex[0], + this.quantization.DeltaQDc[(int)Av1Plane.U], + this.quantization.DeltaQAc[(int)Av1Plane.U], + this.bitDepth, + ref candidateBlueState); + + distortion += TOperator.EncodeCandidate( + this.blockWorkspace, + redSource, + chromaOrigin, + candidateRedReconstruction, + redAbove, + redLeft, + hasLeft, + hasAbove, + predictionMode, + angleDelta, + this.picture.Sequence.SequenceHeader.EnableIntraEdgeFilter, + smoothIntraEdges, + candidateRedCoefficients, + transformSize, + transformType, + Av1Plane.V, + this.quantization.QIndex[0], + this.quantization.DeltaQDc[(int)Av1Plane.V], + this.quantization.DeltaQAc[(int)Av1Plane.V], + this.bitDepth, + ref candidateRedState); + + // The mode and angle are written once for the UV pair; coefficient syntax remains independent + // because each plane has its own EOB, scan values, and neighboring coefficient context. + int rate = Av1TileWriter.GetChromaModeCost( + writer, + this.picture.Parent.FrameHeader, + this.picture.Sequence.SequenceHeader.ColorConfig, + modeInfo, + blockSize, + lumaMode, + chromaMode, + angleDelta); + + if (chromaMode == Av1ChromaPredictionMode.DC) + { + rate += paletteDisabledCost; + } + + rate += writer.GetCoefficientCost( + transformSize, + transformType, + lumaMode, + candidateBlueCoefficients, + Av1ComponentType.Chroma, + blueContext, + candidateBlueState.EndOfBlock, + this.picture.Parent.FrameHeader.UseReducedTransformSet, + Av1FilterIntraMode.AllFilterIntraModes, + usesInterTransformSet: false); + + rate += writer.GetCoefficientCost( + transformSize, + transformType, + lumaMode, + candidateRedCoefficients, + Av1ComponentType.Chroma, + redContext, + candidateRedState.EndOfBlock, + this.picture.Parent.FrameHeader.UseReducedTransformSet, + Av1FilterIntraMode.AllFilterIntraModes, + usesInterTransformSet: false); + + return new(this.rateMultiplier, rate, distortion); + } + + /// + /// Prepares the shared corner and extended top and left edges for intra prediction. + /// + /// The previously reconstructed plane. + /// The prediction block's origin in plane samples. + /// The transform width in samples. + /// The transform height in samples. + /// Whether the left edge is available. + /// Whether the top edge is available. + /// Whether the adjacent top-right block is reconstructed. + /// Whether the adjacent bottom-left block is reconstructed. + /// The sample precision used for unavailable edges. + /// The corner followed by at least width plus height top-edge samples. + /// The corner followed by at least width plus height left-edge samples. + public static void PrepareReferenceSamples( + Buffer2DRegion reconstructionPlane, + Point blockOrigin, + int width, + int height, + bool hasLeft, + bool hasAbove, + bool hasTopRight, + bool hasBottomLeft, + Av1BitDepth bitDepth, + Span aboveStorage, + Span leftStorage) + { + // A directional ray can reach width + height - 1 on either edge, including on rectangles. + // Only one adjacent block supplies extension samples; the rest repeat its final sample. + Span above = aboveStorage.Slice(1, width + height); + Span left = leftStorage.Slice(1, width + height); + if (hasAbove) + { + reconstructionPlane.DangerousGetRowSpan(blockOrigin.Y - 1).Slice(blockOrigin.X, width).CopyTo(above[..width]); + } + + if (hasLeft) + { + for (int row = 0; row < height; row++) + { + left[row] = reconstructionPlane.DangerousGetRowSpan(blockOrigin.Y + row)[blockOrigin.X - 1]; + } + } + + int midpoint = 128 << (bitDepth.GetBitCount() - 8); + if (!hasAbove) + { + above[..width].Fill(hasLeft ? left[0] : TOperator.CreateSample(midpoint - 1)); + } + + if (!hasLeft) + { + left[..height].Fill(hasAbove ? above[0] : TOperator.CreateSample(midpoint + 1)); + } + + // Availability describes coding order, not the number of samples left at the frame boundary. + // A partially present adjacent block contributes only its coded samples before endpoint repetition. + int topRightCount = hasTopRight + ? Math.Min(Math.Min(width, height), reconstructionPlane.Width - blockOrigin.X - width) + : 0; + + if (hasTopRight) + { + reconstructionPlane.DangerousGetRowSpan(blockOrigin.Y - 1) + .Slice(blockOrigin.X + width, topRightCount) + .CopyTo(above[width..]); + } + + int topCount = width + topRightCount; + above[topCount..].Fill(above[topCount - 1]); + + int bottomLeftCount = hasBottomLeft + ? Math.Min(Math.Min(height, width), reconstructionPlane.Height - blockOrigin.Y - height) + : 0; + + for (int row = height; row < height + bottomLeftCount; row++) + { + left[row] = reconstructionPlane.DangerousGetRowSpan(blockOrigin.Y + row)[blockOrigin.X - 1]; + } + + int leftCount = height + bottomLeftCount; + left[leftCount..].Fill(left[leftCount - 1]); + + // Zone-two projection and Paeth address the common corner immediately before both edges. + // Missing edges derive it from the closest coded sample or the bit-depth midpoint. + TSample corner = hasAbove && hasLeft + ? reconstructionPlane.DangerousGetRowSpan(blockOrigin.Y - 1)[blockOrigin.X - 1] + : hasAbove + ? above[0] + : hasLeft + ? left[0] + : TOperator.CreateSample(midpoint); + + aboveStorage[0] = corner; + leftStorage[0] = corner; + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1IntraSuperblockEncoder.ChromaPaletteModeDecision.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1IntraSuperblockEncoder.ChromaPaletteModeDecision.cs new file mode 100644 index 0000000000..8ffdbe2cf2 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1IntraSuperblockEncoder.ChromaPaletteModeDecision.cs @@ -0,0 +1,392 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1.Entropy; +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; +using SixLabors.ImageSharp.Memory; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline; + +/// +/// Provides paired chroma palette mode decisions for intra encoding. +/// +internal static partial class Av1IntraSuperblockEncoder +{ + internal partial struct ModeDecision + where TSample : unmanaged + where TOperator : struct, IBlockEncodingOperator + { + private bool SelectChromaPalette( + Av1SymbolEncoder writer, + Av1MacroBlockD macroBlock, + Av1MacroBlockModeInfo modeInfo, + Point lumaOrigin, + Point chromaOrigin, + ushort tileIndex, + Av1PredictionMode lumaMode, + Av1TransformSize transformSize, + Av1TransformBlockContext blueContext, + Av1TransformBlockContext redContext, + Span candidateBlueReconstruction, + Span candidateRedReconstruction, + Span candidateBlueCoefficients, + Span candidateRedCoefficients, + Span retainedBlueCoefficients, + Span retainedRedCoefficients, + ref Av1EncoderTransformBlockState retainedBlueState, + ref Av1EncoderTransformBlockState retainedRedState, + ref Av1RateDistortionStatistics bestStatistics, + ref Av1EncoderPaletteInfo paletteInfo) + { + const Av1BlockSize BlockSize = Av1BlockSize.Block8x8; + const int LumaBlockLength = 8; + Av1EncoderPaletteWorkspace workspace = + this.blockWorkspace.GetModeDecisionWorkspace().Palette; + + ObuColorConfig colorConfig = this.picture.Sequence.SequenceHeader.ColorConfig; + int subsamplingX = colorConfig.SubSamplingX ? 1 : 0; + int subsamplingY = colorConfig.SubSamplingY ? 1 : 0; + int width = transformSize.GetWidth(); + int height = transformSize.GetHeight(); + + // Clip against the coded mode-info boundary before subsampling, as the decoder does. Visible odd + // dimensions still have complete coded chroma samples; truncating them here can leave an empty palette input. + int rows = (LumaBlockLength + (Math.Min(0, macroBlock.ToBottomEdge) >> 3)) >> subsamplingY; + int columns = (LumaBlockLength + (Math.Min(0, macroBlock.ToRightEdge) >> 3)) >> subsamplingX; + int activeSampleCount = rows * columns; + Span blueSamples = workspace.GetSamples(0)[..activeSampleCount]; + Span redSamples = workspace.GetSamples(1)[..activeSampleCount]; + Buffer2DRegion blueSource = this.source.GetPlane(Av1Plane.U); + Buffer2DRegion redSource = this.source.GetPlane(Av1Plane.V); + TOperator.CopyPaletteSamples(blueSource, chromaOrigin, rows, columns, blueSamples); + TOperator.CopyPaletteSamples(redSource, chromaOrigin, rows, columns, redSamples); + + Span uniqueBlueColors = workspace.GetUniqueColors(0); + Span uniqueRedColors = workspace.GetUniqueColors(1); + int uniqueBlueColorCount = 0; + int uniqueRedColorCount = 0; + short blueMinimum = blueSamples[0]; + short blueMaximum = blueSamples[0]; + short redMinimum = redSamples[0]; + short redMaximum = redSamples[0]; + for (int sampleIndex = 0; sampleIndex < activeSampleCount; sampleIndex++) + { + short blueSample = blueSamples[sampleIndex]; + short redSample = redSamples[sampleIndex]; + if (!uniqueBlueColors[..uniqueBlueColorCount].Contains(blueSample)) + { + uniqueBlueColors[uniqueBlueColorCount++] = blueSample; + } + + if (!uniqueRedColors[..uniqueRedColorCount].Contains(redSample)) + { + uniqueRedColors[uniqueRedColorCount++] = redSample; + } + + blueMinimum = Math.Min(blueMinimum, blueSample); + blueMaximum = Math.Max(blueMaximum, blueSample); + redMinimum = Math.Min(redMinimum, redSample); + redMaximum = Math.Max(redMaximum, redSample); + } + + int maximumColorCount = Math.Max(uniqueBlueColorCount, uniqueRedColorCount); + if (maximumColorCount < 2) + { + return false; + } + + int maximumPaletteSize = Math.Min(maximumColorCount, Av1Constants.PaletteMaxSize); + Av1NeighborArrayUnit paletteContexts = this.picture.PaletteContexts[tileIndex]; + Span colorCache = workspace.ColorCache; + int colorCacheSize = Av1TileWriter.GetPaletteCache( + paletteContexts, + macroBlock, + lumaOrigin, + Av1Plane.U, + colorCache); + + colorCache = colorCache[..colorCacheSize]; + int blockSizeContext = Av1TileWriter.GetPaletteBlockSizeContext(BlockSize); + bool hasLumaPalette = paletteInfo.PaletteSizes[0] != 0; + Buffer2DRegion colorIndexMap = this.superblock.Workspace + .GetPaletteMaps() + .GetMap(Av1PlaneType.Uv, width, height); + + Span retainedColorIndexMap = workspace.RetainedIndices; + Span blueCentroids = workspace.GetCentroids(0); + Span redCentroids = workspace.GetCentroids(1); + Span colorIndices = workspace.Indices; + Span bluePrediction = workspace.GetPrediction(0); + Span redPrediction = workspace.GetPrediction(1); + Span blueResidual = workspace.GetResidual(0); + Span redResidual = workspace.GetResidual(1); + Span bluePaletteColorStorage = workspace.GetPaletteColors(0); + Span redPaletteColorStorage = workspace.GetPaletteColors(1); + int sampleCount = transformSize.GetSize2d(); + int cacheThreshold = 4 << (this.bitDepth.GetBitCount() - 8); + bool paletteSelected = false; + + // Chroma uses one paired K-means family; exhaustive size search avoids early header-cost pruning. + for (int paletteSize = 2; paletteSize <= maximumPaletteSize; paletteSize++) + { + Span candidateBlueCentroids = blueCentroids[..paletteSize]; + Span candidateRedCentroids = redCentroids[..paletteSize]; + Av1PaletteKMeans2D.InitializeCentroids( + blueMinimum, + blueMaximum, + redMinimum, + redMaximum, + candidateBlueCentroids, + candidateRedCentroids); + + Av1PaletteKMeans2D.Cluster( + blueSamples, + redSamples, + candidateBlueCentroids, + candidateRedCentroids, + colorIndices[..activeSampleCount], + workspace.GetAlternateCentroids(0), + workspace.GetAlternateCentroids(1), + workspace.AlternateIndices); + + // Only U participates in the neighbor-color cache. Snap bounded U deltas before sorting, + // while preserving each U/V centroid as one paired palette entry. + for (int colorIndex = 0; colorIndex < paletteSize && !colorCache.IsEmpty; colorIndex++) + { + int minimumDifference = Math.Abs(candidateBlueCentroids[colorIndex] - colorCache[0]); + int nearestCacheIndex = 0; + for (int cacheIndex = 1; cacheIndex < colorCache.Length; cacheIndex++) + { + int difference = Math.Abs(candidateBlueCentroids[colorIndex] - colorCache[cacheIndex]); + if (difference < minimumDifference) + { + minimumDifference = difference; + nearestCacheIndex = cacheIndex; + } + } + + if (minimumDifference <= cacheThreshold) + { + candidateBlueCentroids[colorIndex] = (short)colorCache[nearestCacheIndex]; + } + } + + // U is the coded ordering key, so each swap carries its paired V color with it. + for (int colorIndex = 0; colorIndex < paletteSize - 1; colorIndex++) + { + int minimumIndex = colorIndex; + for (int candidateIndex = colorIndex + 1; candidateIndex < paletteSize; candidateIndex++) + { + if (candidateBlueCentroids[candidateIndex] < candidateBlueCentroids[minimumIndex]) + { + minimumIndex = candidateIndex; + } + } + + if (minimumIndex != colorIndex) + { + (candidateBlueCentroids[colorIndex], candidateBlueCentroids[minimumIndex]) = + (candidateBlueCentroids[minimumIndex], candidateBlueCentroids[colorIndex]); + + (candidateRedCentroids[colorIndex], candidateRedCentroids[minimumIndex]) = + (candidateRedCentroids[minimumIndex], candidateRedCentroids[colorIndex]); + } + } + + Av1PaletteKMeans2D.AssignIndices( + blueSamples, + redSamples, + candidateBlueCentroids, + candidateRedCentroids, + colorIndices); + + for (int row = 0; row < rows; row++) + { + Span mapRow = colorIndexMap.DangerousGetRowSpan(row)[..width]; + colorIndices.Slice(row * columns, columns).CopyTo(mapRow); + mapRow[columns..].Fill(mapRow[columns - 1]); + } + + // The shared U/V map covers the complete declared chroma block even at visible frame edges. + for (int row = rows; row < height; row++) + { + colorIndexMap.DangerousGetRowSpan(rows - 1)[..width] + .CopyTo(colorIndexMap.DangerousGetRowSpan(row)); + } + + Span bluePaletteColors = bluePaletteColorStorage[..paletteSize]; + Span redPaletteColors = redPaletteColorStorage[..paletteSize]; + for (int colorIndex = 0; colorIndex < paletteSize; colorIndex++) + { + bluePaletteColors[colorIndex] = (ushort)candidateBlueCentroids[colorIndex]; + redPaletteColors[colorIndex] = (ushort)candidateRedCentroids[colorIndex]; + } + + // U and V share one color-index map but reconstruct through their own palette values and + // residuals. Both preparations remain valid until the next palette-size candidate. + TOperator.PreparePalette( + blueSource, + chromaOrigin, + bluePaletteColors, + colorIndexMap, + bluePrediction[..sampleCount], + blueResidual[..sampleCount], + transformSize); + + TOperator.PreparePalette( + redSource, + chromaOrigin, + redPaletteColors, + colorIndexMap, + redPrediction[..sampleCount], + redResidual[..sampleCount], + transformSize); + + // Intra chroma derives one transform type from the shared UV mode. Palette uses UV DC, so + // both planes use DCT while retaining independent coefficient contexts and end positions. + Av1EncoderTransformBlockState candidateBlueState = default; + long distortion = TOperator.EncodePredictionCandidate( + this.blockWorkspace, + blueSource, + chromaOrigin, + bluePrediction, + blueResidual, + candidateBlueReconstruction, + transformSize.GetWidth(), + candidateBlueCoefficients, + transformSize, + Av1TransformType.DctDct, + Av1Plane.U, + this.quantization.QIndex[0], + this.quantization.DeltaQDc[(int)Av1Plane.U], + this.quantization.DeltaQAc[(int)Av1Plane.U], + this.bitDepth, + ref candidateBlueState); + + Av1EncoderTransformBlockState candidateRedState = default; + distortion += TOperator.EncodePredictionCandidate( + this.blockWorkspace, + redSource, + chromaOrigin, + redPrediction, + redResidual, + candidateRedReconstruction, + transformSize.GetWidth(), + candidateRedCoefficients, + transformSize, + Av1TransformType.DctDct, + Av1Plane.V, + this.quantization.QIndex[0], + this.quantization.DeltaQDc[(int)Av1Plane.V], + this.quantization.DeltaQAc[(int)Av1Plane.V], + this.bitDepth, + ref candidateRedState); + + int rate = Av1TileWriter.GetChromaModeCost( + writer, + this.picture.Parent.FrameHeader, + colorConfig, + modeInfo, + BlockSize, + lumaMode, + Av1ChromaPredictionMode.DC, + 0); + + rate += writer.GetPaletteUvModeCost(true, hasLumaPalette); + rate += writer.GetPaletteSizeCost(paletteSize, blockSizeContext, Av1PlaneType.Uv); + rate += Av1SymbolEncoder.GetPaletteUvColorCost( + colorCache, + bluePaletteColors, + redPaletteColors, + this.bitDepth.GetBitCount()); + + rate += writer.GetPaletteColorMapCost( + paletteSize, + Av1PlaneType.Uv, + rows, + columns, + colorIndexMap); + + rate += writer.GetCoefficientCost( + transformSize, + Av1TransformType.DctDct, + lumaMode, + candidateBlueCoefficients, + Av1ComponentType.Chroma, + blueContext, + candidateBlueState.EndOfBlock, + this.picture.Parent.FrameHeader.UseReducedTransformSet, + Av1FilterIntraMode.AllFilterIntraModes, + usesInterTransformSet: false); + + // Mode, palette, and color-map syntax is shared by the pair; coefficient syntax and + // distortion remain per plane before the joint chroma rate-distortion comparison. + rate += writer.GetCoefficientCost( + transformSize, + Av1TransformType.DctDct, + lumaMode, + candidateRedCoefficients, + Av1ComponentType.Chroma, + redContext, + candidateRedState.EndOfBlock, + this.picture.Parent.FrameHeader.UseReducedTransformSet, + Av1FilterIntraMode.AllFilterIntraModes, + usesInterTransformSet: false); + + Av1RateDistortionStatistics candidateStatistics = new(this.rateMultiplier, rate, distortion); + if (candidateStatistics.Cost < bestStatistics.Cost) + { + // Every following palette size overwrites the shared maps and candidate spans, so a + // global improvement must retain reconstruction, coefficients, colors, and indices together. + Buffer2DRegion blueReconstruction = this.reconstruction.GetPlane(Av1Plane.U); + Buffer2DRegion redReconstruction = this.reconstruction.GetPlane(Av1Plane.V); + CopyCandidate( + candidateBlueReconstruction, + candidateBlueCoefficients, + blueReconstruction, + chromaOrigin, + retainedBlueCoefficients, + transformSize, + candidateBlueState, + ref retainedBlueState); + + CopyCandidate( + candidateRedReconstruction, + candidateRedCoefficients, + redReconstruction, + chromaOrigin, + retainedRedCoefficients, + transformSize, + candidateRedState, + ref retainedRedState); + + for (int row = 0; row < height; row++) + { + colorIndexMap.DangerousGetRowSpan(row)[..width] + .CopyTo(retainedColorIndexMap[(row * width)..]); + } + + paletteInfo.PaletteSizes[1] = (byte)paletteSize; + paletteInfo.SetColors(Av1Plane.U, bluePaletteColors); + paletteInfo.SetColors(Av1Plane.V, redPaletteColors); + bestStatistics = candidateStatistics; + paletteSelected = true; + } + } + + if (paletteSelected) + { + for (int row = 0; row < height; row++) + { + retainedColorIndexMap.Slice(row * width, width) + .CopyTo(colorIndexMap.DangerousGetRowSpan(row)); + } + } + + return paletteSelected; + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1IntraSuperblockEncoder.ModeDecision.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1IntraSuperblockEncoder.ModeDecision.cs new file mode 100644 index 0000000000..05e148eb17 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1IntraSuperblockEncoder.ModeDecision.cs @@ -0,0 +1,2668 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.InteropServices; +using SixLabors.ImageSharp.Formats.Heif.Av1.Entropy; +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; +using SixLabors.ImageSharp.Memory; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline; + +/// +/// Provides live-probability final-block mode decisions for intra encoding. +/// +internal static partial class Av1IntraSuperblockEncoder +{ + /// + /// Gets the zero-angle luma modes in the order used by the reference encoder. + /// + private static ReadOnlySpan LumaModeSearchOrder => + [ + Av1PredictionMode.DC, + Av1PredictionMode.Horizontal, + Av1PredictionMode.Vertical, + Av1PredictionMode.Smooth, + Av1PredictionMode.Paeth, + Av1PredictionMode.SmoothVertical, + Av1PredictionMode.SmoothHorizontal, + Av1PredictionMode.Directional135Degrees, + Av1PredictionMode.Directional203Degrees, + Av1PredictionMode.Directional157Degrees, + Av1PredictionMode.Directional67Degrees, + Av1PredictionMode.Directional113Degrees, + Av1PredictionMode.Directional45Degrees + ]; + + /// + /// Gets the nonzero directional adjustments in the exhaustive order used by the reference encoder. + /// + private static ReadOnlySpan AngleDeltaSearchOrder => [-3, -2, -1, 1, 2, 3]; + + /// + /// Gets partition candidates in the evaluation order used by the reference encoder. + /// + private static ReadOnlySpan PartitionSearchOrder => + [ + Av1PartitionType.None, + Av1PartitionType.Split, + Av1PartitionType.Horizontal, + Av1PartitionType.Vertical, + Av1PartitionType.HorizontalA, + Av1PartitionType.HorizontalB, + Av1PartitionType.VerticalA, + Av1PartitionType.VerticalB, + Av1PartitionType.Horizontal4, + Av1PartitionType.Vertical4 + ]; + + /// + /// Builds the fixed 8x8 partition skeleton consumed by interleaved mode decision and tile writing. + /// + /// The frame coding and mode-information state. + /// The reusable partition and final-block decisions. + /// The absolute luma-sample origin of the superblock. + public static void Prepare( + Av1PictureControlSet picture, + Av1Superblock superblock, + Point superblockOrigin) + { + superblock.Workspace.Reset(); + int partitionIndex = 0; + PreparePartitionTree( + picture, + superblock, + superblockOrigin, + picture.Sequence.SequenceHeader.SuperblockSize, + ref partitionIndex); + } + + private static void PreparePartitionTree( + Av1PictureControlSet picture, + Av1Superblock superblock, + Point blockOrigin, + Av1BlockSize blockSize, + ref int partitionIndex) + { + Av1EncoderCommon common = picture.Parent.Common; + Point modeInfoPosition = blockOrigin >> Av1Constants.ModeInfoSizeLog2; + if (modeInfoPosition.Y >= common.ModeInfoRowCount || modeInfoPosition.X >= common.ModeInfoColumnCount) + { + return; + } + + if (blockSize == Av1BlockSize.Block8x8) + { + superblock.CodingUnitPartitionTypes[partitionIndex++] = (byte)Av1PartitionType.None; + ref Av1MacroBlockModeInfo modeInfo = ref picture.GetMacroBlockModeInfo(modeInfoPosition); + modeInfo.Block = new Av1EncoderBlockModeInfo + { + BlockSize = Av1BlockSize.Block8x8, + PartitionType = Av1PartitionType.None + }; + + return; + } + + superblock.CodingUnitPartitionTypes[partitionIndex++] = (byte)Av1PartitionType.Split; + Av1BlockSize subSize = Av1PartitionType.Split.GetBlockSubSize(blockSize); + int halfBlockSize = blockSize.GetWidth() >> 1; + + // The same preorder drives partition symbols, block decisions, and coefficient offsets. + PreparePartitionTree(picture, superblock, blockOrigin, subSize, ref partitionIndex); + PreparePartitionTree(picture, superblock, blockOrigin + new Size(halfBlockSize, 0), subSize, ref partitionIndex); + PreparePartitionTree(picture, superblock, blockOrigin + new Size(0, halfBlockSize), subSize, ref partitionIndex); + PreparePartitionTree(picture, superblock, blockOrigin + new Size(halfBlockSize, halfBlockSize), subSize, ref partitionIndex); + } + + /// + /// Produces one final block at a time against the tile state immediately preceding its syntax. + /// + /// The native unsigned sample storage type. + /// The type-specific block encoding operations. + internal partial struct ModeDecision : Av1TileWriter.IBlockEncodingHandler + where TSample : unmanaged + where TOperator : struct, IBlockEncodingOperator + { + private readonly Av1EncoderFrame.PlanarView source; + private readonly Av1EncoderFrame.PlanarView reference; + private readonly Av1EncoderFrame.PlanarView reconstruction; + private readonly Av1PictureControlSet picture; + private readonly Av1Superblock superblock; + private readonly Av1EncoderCoefficientBuffer coefficientBuffer; + private readonly Av1EncoderBlockWorkspace blockWorkspace; + private readonly ObuQuantizationParameters quantization; + private readonly Av1BitDepth bitDepth; + private readonly int rateMultiplier; + private readonly int effort; + private int codedAreaLuma; + private int codedAreaChroma; + + /// + /// Initializes a new instance of the struct. + /// + /// The coded source frame. + /// The reconstructed inter reference, or the current reconstruction for an intra frame. + /// The reconstructed frame updated by winning candidates. + /// The frame coding and mode-information state. + /// The current superblock. + /// The frame-owned quantized coefficient and transform state. + /// The reusable block arithmetic workspace. + /// The mode-search effort in the inclusive range zero through ten. + public ModeDecision( + Av1EncoderFrame source, + Av1EncoderFrame reference, + Av1EncoderFrame reconstruction, + Av1PictureControlSet picture, + Av1Superblock superblock, + Av1EncoderCoefficientBuffer coefficientBuffer, + Av1EncoderBlockWorkspace blockWorkspace, + int effort) + { + this.source = source.CodedView; + this.reference = reference.CodedView; + this.reconstruction = reconstruction.CodedView; + this.picture = picture; + this.superblock = superblock; + this.coefficientBuffer = coefficientBuffer; + this.blockWorkspace = blockWorkspace; + this.quantization = picture.Parent.FrameHeader.QuantizationParameters; + this.bitDepth = picture.Sequence.SequenceHeader.ColorConfig.BitDepth; + this.rateMultiplier = picture.Parent.FrameHeader.IsIntra + ? Av1RateDistortion.GetKeyFrameRateMultiplier(this.quantization.QIndex[0], this.bitDepth) + : Av1RateDistortion.GetInterFrameRateMultiplier(this.quantization.QIndex[0], this.bitDepth); + + this.effort = effort; + this.codedAreaLuma = 0; + this.codedAreaChroma = 0; + this.SelectedBlockStatistics = default; + } + + /// + /// Gets the statistics of the most recently encoded block. + /// + public Av1RateDistortionStatistics SelectedBlockStatistics { get; private set; } + + /// + public Av1PartitionType SelectPartition( + Av1SymbolEncoder writer, + Av1MacroBlockD macroBlock, + Point blockOrigin, + ushort tileIndex, + Av1BlockSize blockSize, + Av1PartitionType preparedPartition) + { + if (!this.picture.Parent.FrameHeader.IsIntra) + { + // The first inter implementation retains the prepared 8x8 tree so every prediction and residual + // transform fits the single reusable block workspace while larger inter partitions remain unsearched. + return preparedPartition; + } + + // Live decisions change the number of nodes visited before this position. The original flat + // skeleton's index no longer identifies this block, so derive its default from current geometry. + // Otherwise an earlier unsplit 16x16 can make a later 32x32 consume an old 8x8 NONE entry. + preparedPartition = blockSize == Av1BlockSize.Block8x8 ? Av1PartitionType.None : Av1PartitionType.Split; + + bool searchPartition = blockSize is Av1BlockSize.Block8x8 or Av1BlockSize.Block16x16 || + (this.effort == 10 && + blockSize is Av1BlockSize.Block32x32 or Av1BlockSize.Block64x64 or Av1BlockSize.Block128x128); + + if (!searchPartition) + { + return preparedPartition; + } + + Point modeInfoPosition = blockOrigin >> Av1Constants.ModeInfoSizeLog2; + bool hasRows = + modeInfoPosition.Y + blockSize.Get4x4HighCount() <= this.picture.Parent.Common.ModeInfoRowCount; + + bool hasColumns = + modeInfoPosition.X + blockSize.Get4x4WideCount() <= this.picture.Parent.Common.ModeInfoColumnCount; + + if (!hasRows || !hasColumns) + { + // Coded dimensions are aligned to eight samples, so an incomplete searched node must retain + // the prepared split tree rather than evaluating a block that extends beyond source storage. + this.PreparePartitionGeometry(blockOrigin, blockSize, preparedPartition); + return preparedPartition; + } + + if (this.effort < 9) + { + return preparedPartition; + } + + Av1PartitionType selectedPartition = this.SelectBestPartition( + writer, + macroBlock, + blockOrigin, + tileIndex, + blockSize); + + // Trial reconstruction and mode entries need no copy-back. The selected branch is evaluated again + // in raster order, overwriting each trial-local value before a later selected leaf can consume it. + this.PreparePartitionGeometry(blockOrigin, blockSize, selectedPartition); + return selectedPartition; + } + + private Av1PartitionType SelectBestPartition( + Av1SymbolEncoder writer, + Av1MacroBlockD macroBlock, + Point blockOrigin, + ushort tileIndex, + Av1BlockSize blockSize) + { + int savedLumaArea = this.codedAreaLuma; + int savedChromaArea = this.codedAreaChroma; + this.SavePartitionTrialContexts(blockOrigin, tileIndex, blockSize); + Av1RateDistortionStatistics bestStatistics = Av1RateDistortionStatistics.Invalid; + Av1PartitionType selectedPartition = Av1PartitionType.None; + ReadOnlySpan searchOrder = PartitionSearchOrder; + int candidateCount = blockSize == Av1BlockSize.Block8x8 ? 4 : searchOrder.Length; + for (int candidateIndex = 0; candidateIndex < candidateCount; candidateIndex++) + { + Av1PartitionType partitionType = searchOrder[candidateIndex]; + if (!this.IsPartitionCandidateAllowed(blockSize, partitionType)) + { + continue; + } + + Av1RateDistortionStatistics candidateStatistics = this.EvaluatePartitionCandidate( + writer, + macroBlock, + blockOrigin, + tileIndex, + blockSize, + partitionType, + publishFinalContexts: false); + + if (candidateStatistics.Cost < bestStatistics.Cost) + { + bestStatistics = candidateStatistics; + selectedPartition = partitionType; + } + + this.ResetPartitionTrial( + blockOrigin, + tileIndex, + blockSize, + savedLumaArea, + savedChromaArea); + } + + return selectedPartition; + } + + private Av1RateDistortionStatistics EvaluateSelectedPartitionTree( + Av1SymbolEncoder writer, + Av1MacroBlockD macroBlock, + Point blockOrigin, + ushort tileIndex, + Av1BlockSize blockSize, + bool publishContexts) + { + Av1PartitionType selectedPartition = this.SelectBestPartition( + writer, + macroBlock, + blockOrigin, + tileIndex, + blockSize); + + Av1RateDistortionStatistics statistics = this.EvaluatePartitionCandidate( + writer, + macroBlock, + blockOrigin, + tileIndex, + blockSize, + selectedPartition, + publishContexts); + + if (publishContexts) + { + Av1TileWriter.UpdatePartitionContexts( + this.picture.PartitionContexts[tileIndex], + blockOrigin, + selectedPartition.GetBlockSubSize(blockSize), + blockSize, + selectedPartition); + } + + return statistics; + } + + private Av1RateDistortionStatistics EvaluatePartitionCandidate( + Av1SymbolEncoder writer, + Av1MacroBlockD macroBlock, + Point blockOrigin, + ushort tileIndex, + Av1BlockSize blockSize, + Av1PartitionType partitionType, + bool publishFinalContexts) + { + int rate = Av1TileWriter.GetPartitionCost( + this.picture, + writer, + blockSize, + partitionType, + blockOrigin, + this.picture.PartitionContexts[tileIndex]); + + Av1RateDistortionStatistics statistics = new(this.rateMultiplier, rate, 0); + int leafCount = GetPartitionLeafCount(partitionType); + + // Child reconstruction and syntax contexts become input to the next child. Publishing only + // the required leaves reproduces libaom's raster dry run without writing entropy symbols. + for (int leafIndex = 0; leafIndex < leafCount; leafIndex++) + { + GetPartitionLeafGeometry( + blockOrigin, + blockSize, + partitionType, + leafIndex, + out Point leafOrigin, + out Av1BlockSize leafSize); + + bool publishContexts = leafIndex < leafCount - 1 || publishFinalContexts; + Av1RateDistortionStatistics childStatistics = partitionType == Av1PartitionType.Split && blockSize > Av1BlockSize.Block8x8 + ? this.EvaluateSelectedPartitionTree( + writer, + macroBlock, + leafOrigin, + tileIndex, + leafSize, + publishContexts) + : this.EvaluatePartitionLeaf( + writer, + macroBlock, + leafOrigin, + tileIndex, + leafSize, + partitionType == Av1PartitionType.Split ? Av1PartitionType.None : partitionType, + publishContexts); + + statistics.Add(this.rateMultiplier, in childStatistics); + } + + return statistics; + } + + private void ResetPartitionTrial( + Point blockOrigin, + ushort tileIndex, + Av1BlockSize blockSize, + int savedLumaArea, + int savedChromaArea) + { + this.codedAreaLuma = savedLumaArea; + this.codedAreaChroma = savedChromaArea; + this.RestorePartitionTrialContexts(blockOrigin, tileIndex, blockSize); + } + + private void PreparePartitionGeometry( + Point blockOrigin, + Av1BlockSize blockSize, + Av1PartitionType partitionType) + { + int leafCount = GetPartitionLeafCount(partitionType); + for (int leafIndex = 0; leafIndex < leafCount; leafIndex++) + { + GetPartitionLeafGeometry( + blockOrigin, + blockSize, + partitionType, + leafIndex, + out Point leafOrigin, + out Av1BlockSize leafSize); + + if (this.IsBlockOriginInsideFrame(leafOrigin)) + { + // Mixed vertical partitions reconstruct square leaves in a different order. + // Retain the parent decision so prediction uses the same edge availability as the decoder. + // Split children own another partition node; a terminal 4x4 child implicitly owns NONE. + this.SetBlockGeometry( + leafOrigin, + leafSize, + partitionType == Av1PartitionType.Split ? Av1PartitionType.None : partitionType); + } + } + } + + private bool IsPartitionCandidateAllowed( + Av1BlockSize blockSize, + Av1PartitionType partitionType) + { + if (partitionType.GetBlockSubSize(blockSize) == Av1BlockSize.Invalid) + { + return false; + } + + if (blockSize == Av1BlockSize.Block128x128 && + partitionType is Av1PartitionType.Horizontal4 or Av1PartitionType.Vertical4) + { + // AV1 excludes 128x32 and 32x128 leaves from the 128x128 partition alphabet. + return false; + } + + if (this.source.IsMonochrome) + { + return true; + } + + ObuColorConfig colorConfig = this.picture.Sequence.SequenceHeader.ColorConfig; + int leafCount = GetPartitionLeafCount(partitionType); + for (int leafIndex = 0; leafIndex < leafCount; leafIndex++) + { + GetPartitionLeafGeometry( + Point.Empty, + blockSize, + partitionType, + leafIndex, + out _, + out Av1BlockSize leafSize); + + if (leafSize.GetSubsampled(colorConfig.SubSamplingX, colorConfig.SubSamplingY) == + Av1BlockSize.Invalid) + { + return false; + } + } + + return true; + } + + private bool IsBlockOriginInsideFrame(Point blockOrigin) + { + Point modeInfoPosition = blockOrigin >> Av1Constants.ModeInfoSizeLog2; + return modeInfoPosition.Y < this.picture.Parent.Common.ModeInfoRowCount && + modeInfoPosition.X < this.picture.Parent.Common.ModeInfoColumnCount; + } + + private static int GetPartitionLeafCount(Av1PartitionType partitionType) + => partitionType switch + { + Av1PartitionType.None => 1, + Av1PartitionType.Horizontal or Av1PartitionType.Vertical => 2, + Av1PartitionType.HorizontalA or + Av1PartitionType.HorizontalB or + Av1PartitionType.VerticalA or + Av1PartitionType.VerticalB => 3, + _ => 4 + }; + + private static void GetPartitionLeafGeometry( + Point blockOrigin, + Av1BlockSize blockSize, + Av1PartitionType partitionType, + int leafIndex, + out Point leafOrigin, + out Av1BlockSize leafSize) + { + int halfWidth = blockSize.GetWidth() >> 1; + int halfHeight = blockSize.GetHeight() >> 1; + Av1BlockSize rectangularSize = partitionType.GetBlockSubSize(blockSize); + Av1BlockSize splitSize = Av1PartitionType.Split.GetBlockSubSize(blockSize); + switch (partitionType) + { + case Av1PartitionType.Horizontal: + leafOrigin = blockOrigin + new Size(0, leafIndex * halfHeight); + leafSize = rectangularSize; + return; + case Av1PartitionType.Vertical: + leafOrigin = blockOrigin + new Size(leafIndex * halfWidth, 0); + leafSize = rectangularSize; + return; + case Av1PartitionType.Split: + leafOrigin = blockOrigin + new Size( + (leafIndex & 1) * halfWidth, + (leafIndex >> 1) * halfHeight); + + leafSize = splitSize; + return; + case Av1PartitionType.HorizontalA: + leafOrigin = leafIndex < 2 + ? blockOrigin + new Size(leafIndex * halfWidth, 0) + : blockOrigin + new Size(0, halfHeight); + + leafSize = leafIndex < 2 ? splitSize : rectangularSize; + return; + case Av1PartitionType.HorizontalB: + leafOrigin = leafIndex == 0 + ? blockOrigin + : blockOrigin + new Size((leafIndex - 1) * halfWidth, halfHeight); + + leafSize = leafIndex == 0 ? rectangularSize : splitSize; + return; + case Av1PartitionType.VerticalA: + leafOrigin = leafIndex < 2 + ? blockOrigin + new Size(0, leafIndex * halfHeight) + : blockOrigin + new Size(halfWidth, 0); + + leafSize = leafIndex < 2 ? splitSize : rectangularSize; + return; + case Av1PartitionType.VerticalB: + leafOrigin = leafIndex == 0 + ? blockOrigin + : blockOrigin + new Size(halfWidth, (leafIndex - 1) * halfHeight); + + leafSize = leafIndex == 0 ? rectangularSize : splitSize; + return; + case Av1PartitionType.Horizontal4: + leafOrigin = blockOrigin + new Size(0, leafIndex * (blockSize.GetHeight() >> 2)); + leafSize = rectangularSize; + return; + case Av1PartitionType.Vertical4: + leafOrigin = blockOrigin + new Size(leafIndex * (blockSize.GetWidth() >> 2), 0); + leafSize = rectangularSize; + return; + default: + leafOrigin = blockOrigin; + leafSize = blockSize; + return; + } + } + + /// + public void EncodeBlock( + Av1SymbolEncoder writer, + Av1MacroBlockD macroBlock, + Point blockOrigin, + ushort tileIndex, + ref Av1MacroBlockModeInfo modeInfo, + ref Av1EncoderBlockStruct block, + ref Av1EncoderPaletteInfo paletteInfo) + { + Av1BlockSize blockSize = modeInfo.Block.BlockSize; + Av1PartitionType partitionType = modeInfo.Block.PartitionType; + Av1TransformSize maximumLumaTransformSize = this.picture.Parent.FrameHeader.CodedLossless + ? Av1TransformSize.Size4x4 + : blockSize.GetMaximumTransformSize(); + + int qIndex = this.quantization.QIndex[0]; + modeInfo.Block = new Av1EncoderBlockModeInfo + { + BlockSize = blockSize, + PartitionType = partitionType, + SegmentId = 0, + TransformSize = maximumLumaTransformSize, + Mode = Av1PredictionMode.DC, + UvMode = Av1ChromaPredictionMode.DC + }; + + modeInfo.CdefStrength = 0; + Point modeInfoPosition = blockOrigin >> Av1Constants.ModeInfoSizeLog2; + block.HasChroma = !this.source.IsMonochrome && + Av1TileReader.HasChroma(this.picture.Sequence.SequenceHeader, modeInfoPosition, blockSize); + + block.QuantizationIndex = qIndex; + block.SegmentId = 0; + + Span lumaCoefficients = this.coefficientBuffer.GetPlaneSpan(this.superblock.Index, Av1Plane.Y); + Span lumaTransformBlocks = + this.coefficientBuffer.GetTransformBlockSpan(this.superblock.Index, Av1Plane.Y); + + int lumaTransformIndex = this.codedAreaLuma / + Av1EncoderCoefficientBuffer.TransformBlockUnitCoefficientCount; + + Span retainedLumaStates = lumaTransformBlocks[lumaTransformIndex..]; + modeInfo.Block.Mode = this.SelectLumaMode( + writer, + macroBlock, + blockOrigin, + blockSize, + tileIndex, + lumaCoefficients[this.codedAreaLuma..], + retainedLumaStates, + ref paletteInfo, + out int lumaAngleDelta, + out Av1FilterIntraMode filterIntraMode, + out Av1TransformSize lumaTransformSize, + out Av1RateDistortionStatistics lumaStatistics); + + block.PredictionUnit.AngleDelta[(int)Av1PlaneType.Y] = (sbyte)lumaAngleDelta; + block.FilterIntraMode = filterIntraMode; + modeInfo.Block.TransformSize = lumaTransformSize; + + // Ordinary intra keeps the block non-skipped, including when all transforms are empty. Its RD cost + // includes those transform symbols and the non-skip flag; only inter or IBC winners can replace this state. + if (this.source.IsMonochrome) + { + bool allowIntraBlockCopy = blockSize == Av1BlockSize.Block8x8 && + this.picture.Parent.FrameHeader.AllowIntraBlockCopy; + + Av1RateDistortionStatistics regularStatistics = this.GetRegularBlockCost( + writer, + macroBlock, + lumaStatistics, + allowIntraBlockCopy); + + if (!this.picture.Parent.FrameHeader.IsIntra) + { + this.SelectedBlockStatistics = this.SelectInterPrediction( + writer, + macroBlock, + blockOrigin, + tileIndex, + regularStatistics, + ref modeInfo, + ref block, + ref paletteInfo); + } + else + { + this.SelectedBlockStatistics = allowIntraBlockCopy + ? this.SelectIntraBlockCopy( + writer, + macroBlock, + blockOrigin, + tileIndex, + regularStatistics, + ref modeInfo, + ref block, + ref paletteInfo) + : regularStatistics; + } + + this.codedAreaLuma += blockSize.GetWidth() * blockSize.GetHeight(); + return; + } + + ObuColorConfig colorConfig = this.picture.Sequence.SequenceHeader.ColorConfig; + int subsamplingX = colorConfig.SubSamplingX ? 1 : 0; + int subsamplingY = colorConfig.SubSamplingY ? 1 : 0; + Point chromaOrigin = Av1TileWriter.GetChromaBlockOrigin( + blockOrigin, + subsamplingX, + subsamplingY); + + Av1TransformSize chromaTransformSize = this.picture.Parent.FrameHeader.CodedLossless + ? Av1TransformSize.Size4x4 + : blockSize.GetMaxUvTransformSize( + colorConfig.SubSamplingX, + colorConfig.SubSamplingY); + + Av1BlockSize chromaBlockSize = blockSize.GetSubsampled( + colorConfig.SubSamplingX, + colorConfig.SubSamplingY); + + Span blueCoefficients = this.coefficientBuffer.GetPlaneSpan(this.superblock.Index, Av1Plane.U); + Span redCoefficients = this.coefficientBuffer.GetPlaneSpan(this.superblock.Index, Av1Plane.V); + Span blueTransformBlocks = + this.coefficientBuffer.GetTransformBlockSpan(this.superblock.Index, Av1Plane.U); + Span redTransformBlocks = + this.coefficientBuffer.GetTransformBlockSpan(this.superblock.Index, Av1Plane.V); + + int chromaTransformIndex = this.codedAreaChroma / + Av1EncoderCoefficientBuffer.TransformBlockUnitCoefficientCount; + + Span retainedBlueStates = blueTransformBlocks[chromaTransformIndex..]; + Span retainedRedStates = redTransformBlocks[chromaTransformIndex..]; + Av1RateDistortionStatistics chromaStatistics = default; + if (block.HasChroma) + { + modeInfo.Block.UvMode = this.SelectChromaMode( + writer, + macroBlock, + modeInfo, + blockOrigin, + chromaOrigin, + blockSize, + tileIndex, + modeInfo.Block.Mode, + chromaTransformSize, + blueCoefficients[this.codedAreaChroma..], + redCoefficients[this.codedAreaChroma..], + retainedBlueStates, + retainedRedStates, + ref paletteInfo, + out int chromaAngleDelta, + out byte chromaFromLumaIndex, + out sbyte chromaFromLumaSigns, + out chromaStatistics); + + block.PredictionUnit.AngleDelta[(int)Av1PlaneType.Uv] = (sbyte)chromaAngleDelta; + block.PredictionUnit.ChromaFromLumaIndex = chromaFromLumaIndex; + block.PredictionUnit.ChromaFromLumaSigns = chromaFromLumaSigns; + } + + bool allowColorIntraBlockCopy = blockSize == Av1BlockSize.Block8x8 && + this.picture.Parent.FrameHeader.AllowIntraBlockCopy; + + lumaStatistics.Add(this.rateMultiplier, in chromaStatistics); + Av1RateDistortionStatistics regularColorStatistics = this.GetRegularBlockCost( + writer, + macroBlock, + lumaStatistics, + allowColorIntraBlockCopy); + + if (!this.picture.Parent.FrameHeader.IsIntra) + { + this.SelectedBlockStatistics = this.SelectInterPrediction( + writer, + macroBlock, + blockOrigin, + tileIndex, + regularColorStatistics, + ref modeInfo, + ref block, + ref paletteInfo); + } + else + { + this.SelectedBlockStatistics = allowColorIntraBlockCopy + ? this.SelectIntraBlockCopy( + writer, + macroBlock, + blockOrigin, + tileIndex, + regularColorStatistics, + ref modeInfo, + ref block, + ref paletteInfo) + : regularColorStatistics; + } + + this.codedAreaLuma += blockSize.GetWidth() * blockSize.GetHeight(); + if (block.HasChroma) + { + this.codedAreaChroma += chromaBlockSize.GetWidth() * chromaBlockSize.GetHeight(); + } + } + + private Av1RateDistortionStatistics EvaluatePartitionLeaf( + Av1SymbolEncoder writer, + Av1MacroBlockD macroBlock, + Point blockOrigin, + ushort tileIndex, + Av1BlockSize blockSize, + Av1PartitionType partitionType, + bool publishContexts) + { + // Trial leaves must use the same reconstruction order as final leaves of this partition. + this.SetBlockGeometry(blockOrigin, blockSize, partitionType); + Point modeInfoPosition = blockOrigin >> Av1Constants.ModeInfoSizeLog2; + Av1TileWriter.SetModeInfoRowAndColumn( + this.picture, + macroBlock, + macroBlock.Tile, + modeInfoPosition, + blockSize, + this.picture.Parent.Common.ModeInfoStride, + this.picture.Parent.Common.ModeInfoRowCount, + this.picture.Parent.Common.ModeInfoColumnCount); + + ref Av1MacroBlockModeInfo modeInfo = ref this.picture.GetMacroBlockModeInfo(modeInfoPosition); + Av1EncoderBlockStruct block = default; + Av1EncoderPaletteInfo paletteInfo = default; + int lumaArea = this.codedAreaLuma; + int chromaArea = this.codedAreaChroma; + this.EncodeBlock( + writer, + macroBlock, + blockOrigin, + tileIndex, + ref modeInfo, + ref block, + ref paletteInfo); + + if (publishContexts) + { + this.PublishPartitionLeafContexts( + blockOrigin, + tileIndex, + lumaArea, + chromaArea, + modeInfo, + block, + paletteInfo); + } + + return this.SelectedBlockStatistics; + } + + private void SetBlockGeometry( + Point blockOrigin, + Av1BlockSize blockSize, + Av1PartitionType partitionType) + { + Point modeInfoPosition = blockOrigin >> Av1Constants.ModeInfoSizeLog2; + ref Av1MacroBlockModeInfo modeInfo = ref this.picture.GetMacroBlockModeInfo(modeInfoPosition); + modeInfo.Block = new Av1EncoderBlockModeInfo + { + BlockSize = blockSize, + PartitionType = partitionType + }; + + this.picture.MapModeInfoBlock(modeInfoPosition, blockSize); + } + + private void PublishPartitionLeafContexts( + Point blockOrigin, + ushort tileIndex, + int lumaArea, + int chromaArea, + Av1MacroBlockModeInfo modeInfo, + Av1EncoderBlockStruct block, + Av1EncoderPaletteInfo paletteInfo) + { + Av1BlockSize blockSize = modeInfo.Block.BlockSize; + Av1TransformSize transformSize = modeInfo.Block.TransformSize; + Size blockDimensions = new(blockSize.GetWidth(), blockSize.GetHeight()); + Av1NeighborArrayUnit transformContexts = this.picture.TransformFunctionContexts[tileIndex]; + transformContexts.UnitModeWrite( + (byte)transformSize.GetWidth(), + blockOrigin, + blockDimensions, + Av1NeighborArrayUnit.UnitMask.Top); + + transformContexts.UnitModeWrite( + (byte)transformSize.GetHeight(), + blockOrigin, + blockDimensions, + Av1NeighborArrayUnit.UnitMask.Left); + + Span lumaStates = + this.coefficientBuffer.GetTransformBlockSpan(this.superblock.Index, Av1Plane.Y); + + Span lumaCoefficients = this.coefficientBuffer.GetPlaneSpan(this.superblock.Index, Av1Plane.Y); + PublishCoefficientContexts( + this.picture.LuminanceDcSignLevelCoefficientNeighbors[tileIndex], + blockOrigin, + blockSize, + transformSize, + Av1BlockSize.Block64x64, + lumaCoefficients[lumaArea..], + lumaStates[(lumaArea / Av1EncoderCoefficientBuffer.TransformBlockUnitCoefficientCount)..]); + + if (this.picture.Parent.FrameHeader.AllowScreenContentTools) + { + const Av1NeighborArrayUnit.UnitMask PaletteContextMask = + Av1NeighborArrayUnit.UnitMask.Top | + Av1NeighborArrayUnit.UnitMask.Left; + + this.picture.PaletteContexts[tileIndex].UnitModeWrite( + paletteInfo, + blockOrigin, + blockDimensions, + PaletteContextMask); + } + + if (!block.HasChroma) + { + return; + } + + ObuColorConfig colorConfig = this.picture.Sequence.SequenceHeader.ColorConfig; + int subsamplingX = colorConfig.SubSamplingX ? 1 : 0; + int subsamplingY = colorConfig.SubSamplingY ? 1 : 0; + Point chromaOrigin = Av1TileWriter.GetChromaBlockOrigin( + blockOrigin, + subsamplingX, + subsamplingY); + + Av1BlockSize chromaBlockSize = blockSize.GetSubsampled( + colorConfig.SubSamplingX, + colorConfig.SubSamplingY); + + // Lossless residuals retain one state per 4x4 transform, including chroma. Publish those exact + // edges during partition trials so a later sibling sees the contexts that final writing will use. + Av1TransformSize chromaTransformSize = this.picture.Parent.FrameHeader.CodedLossless + ? Av1TransformSize.Size4x4 + : blockSize.GetMaxUvTransformSize(colorConfig.SubSamplingX, colorConfig.SubSamplingY); + + Av1BlockSize maximumChromaUnitBlockSize = + Av1BlockSize.Block64x64.GetSubsampled(colorConfig.SubSamplingX, colorConfig.SubSamplingY); + + int chromaStateIndex = + chromaArea / Av1EncoderCoefficientBuffer.TransformBlockUnitCoefficientCount; + + Span blueStates = + this.coefficientBuffer.GetTransformBlockSpan(this.superblock.Index, Av1Plane.U); + + Span redStates = + this.coefficientBuffer.GetTransformBlockSpan(this.superblock.Index, Av1Plane.V); + + Span blueCoefficients = this.coefficientBuffer.GetPlaneSpan(this.superblock.Index, Av1Plane.U); + Span redCoefficients = this.coefficientBuffer.GetPlaneSpan(this.superblock.Index, Av1Plane.V); + PublishCoefficientContexts( + this.picture.CbDcSignLevelCoefficientNeighbors[tileIndex], + chromaOrigin, + chromaBlockSize, + chromaTransformSize, + maximumChromaUnitBlockSize, + blueCoefficients[chromaArea..], + blueStates[chromaStateIndex..]); + + PublishCoefficientContexts( + this.picture.CrDcSignLevelCoefficientNeighbors[tileIndex], + chromaOrigin, + chromaBlockSize, + chromaTransformSize, + maximumChromaUnitBlockSize, + redCoefficients[chromaArea..], + redStates[chromaStateIndex..]); + } + + private static void PublishCoefficientContexts( + Av1NeighborArrayUnit neighbors, + Point blockOrigin, + Av1BlockSize blockSize, + Av1TransformSize transformSize, + Av1BlockSize maximumUnitBlockSize, + ReadOnlySpan coefficients, + ReadOnlySpan states) + { + const Av1NeighborArrayUnit.UnitMask EdgeMask = + Av1NeighborArrayUnit.UnitMask.Top | + Av1NeighborArrayUnit.UnitMask.Left; + + int blockWidth = blockSize.GetWidth(); + int blockHeight = blockSize.GetHeight(); + int transformWidth = transformSize.GetWidth(); + int transformHeight = transformSize.GetHeight(); + int transformSampleCount = transformSize.GetSize2d(); + int maximumUnitWidth = Math.Min(maximumUnitBlockSize.GetWidth(), blockWidth); + int maximumUnitHeight = Math.Min(maximumUnitBlockSize.GetHeight(), blockHeight); + int transformStateOffset = 0; + int coefficientOffset = 0; + int transformStateStride = + transformSampleCount / Av1EncoderCoefficientBuffer.TransformBlockUnitCoefficientCount; + + // Coefficients retain AV1's bounded-region order rather than unrestricted row-major order. + // Publishing the same sequence pairs every state with the transform that produced it. + for (int regionRow = 0; regionRow < blockHeight; regionRow += maximumUnitHeight) + { + int unitBottom = Math.Min(regionRow + maximumUnitHeight, blockHeight); + for (int regionColumn = 0; regionColumn < blockWidth; regionColumn += maximumUnitWidth) + { + int unitRight = Math.Min(regionColumn + maximumUnitWidth, blockWidth); + for (int row = regionRow; row < unitBottom; row += transformHeight) + { + for (int column = regionColumn; column < unitRight; column += transformWidth) + { + Av1EncoderTransformBlockState state = states[transformStateOffset]; + byte context = Av1SymbolContextHelper.GetCoefficientContext( + coefficients[coefficientOffset..], + transformSize, + state.TransformType, + state.EndOfBlock); + + neighbors.UnitModeWrite( + context, + blockOrigin + new Size(column, row), + new Size(transformWidth, transformHeight), + EdgeMask); + + coefficientOffset += transformSampleCount; + transformStateOffset += transformStateStride; + } + } + } + } + } + + private void SavePartitionTrialContexts( + Point blockOrigin, + ushort tileIndex, + Av1BlockSize blockSize) + { + Span storage = this.blockWorkspace.GetPartitionContextStorage(blockSize); + int offset = 0; + SaveNeighborEdges( + this.picture.PartitionContexts[tileIndex], + blockOrigin, + blockSize.Get4x4WideCount(), + blockSize.Get4x4HighCount(), + storage, + ref offset); + + SaveNeighborEdges( + this.picture.LuminanceDcSignLevelCoefficientNeighbors[tileIndex], + blockOrigin, + blockSize.Get4x4WideCount(), + blockSize.Get4x4HighCount(), + storage, + ref offset); + + SaveNeighborEdges( + this.picture.TransformFunctionContexts[tileIndex], + blockOrigin, + blockSize.Get4x4WideCount(), + blockSize.Get4x4HighCount(), + storage, + ref offset); + + if (this.picture.Parent.FrameHeader.AllowScreenContentTools) + { + SaveNeighborEdges( + this.picture.PaletteContexts[tileIndex], + blockOrigin, + blockSize.Get4x4WideCount(), + blockSize.Get4x4HighCount(), + storage, + ref offset); + } + + if (this.source.IsMonochrome) + { + return; + } + + ObuColorConfig colorConfig = this.picture.Sequence.SequenceHeader.ColorConfig; + int subsamplingX = colorConfig.SubSamplingX ? 1 : 0; + int subsamplingY = colorConfig.SubSamplingY ? 1 : 0; + Point chromaOrigin = Av1TileWriter.GetChromaBlockOrigin( + blockOrigin, + subsamplingX, + subsamplingY); + + Av1BlockSize chromaBlockSize = blockSize.GetSubsampled( + colorConfig.SubSamplingX, + colorConfig.SubSamplingY); + + SaveNeighborEdges( + this.picture.CbDcSignLevelCoefficientNeighbors[tileIndex], + chromaOrigin, + chromaBlockSize.Get4x4WideCount(), + chromaBlockSize.Get4x4HighCount(), + storage, + ref offset); + + SaveNeighborEdges( + this.picture.CrDcSignLevelCoefficientNeighbors[tileIndex], + chromaOrigin, + chromaBlockSize.Get4x4WideCount(), + chromaBlockSize.Get4x4HighCount(), + storage, + ref offset); + } + + private void RestorePartitionTrialContexts( + Point blockOrigin, + ushort tileIndex, + Av1BlockSize blockSize) + { + ReadOnlySpan storage = this.blockWorkspace.GetPartitionContextStorage(blockSize); + int offset = 0; + RestoreNeighborEdges( + this.picture.PartitionContexts[tileIndex], + blockOrigin, + blockSize.Get4x4WideCount(), + blockSize.Get4x4HighCount(), + storage, + ref offset); + + RestoreNeighborEdges( + this.picture.LuminanceDcSignLevelCoefficientNeighbors[tileIndex], + blockOrigin, + blockSize.Get4x4WideCount(), + blockSize.Get4x4HighCount(), + storage, + ref offset); + + RestoreNeighborEdges( + this.picture.TransformFunctionContexts[tileIndex], + blockOrigin, + blockSize.Get4x4WideCount(), + blockSize.Get4x4HighCount(), + storage, + ref offset); + + if (this.picture.Parent.FrameHeader.AllowScreenContentTools) + { + RestoreNeighborEdges( + this.picture.PaletteContexts[tileIndex], + blockOrigin, + blockSize.Get4x4WideCount(), + blockSize.Get4x4HighCount(), + storage, + ref offset); + } + + if (this.source.IsMonochrome) + { + return; + } + + ObuColorConfig colorConfig = this.picture.Sequence.SequenceHeader.ColorConfig; + int subsamplingX = colorConfig.SubSamplingX ? 1 : 0; + int subsamplingY = colorConfig.SubSamplingY ? 1 : 0; + Point chromaOrigin = Av1TileWriter.GetChromaBlockOrigin( + blockOrigin, + subsamplingX, + subsamplingY); + + Av1BlockSize chromaBlockSize = blockSize.GetSubsampled( + colorConfig.SubSamplingX, + colorConfig.SubSamplingY); + + RestoreNeighborEdges( + this.picture.CbDcSignLevelCoefficientNeighbors[tileIndex], + chromaOrigin, + chromaBlockSize.Get4x4WideCount(), + chromaBlockSize.Get4x4HighCount(), + storage, + ref offset); + + RestoreNeighborEdges( + this.picture.CrDcSignLevelCoefficientNeighbors[tileIndex], + chromaOrigin, + chromaBlockSize.Get4x4WideCount(), + chromaBlockSize.Get4x4HighCount(), + storage, + ref offset); + } + + private static void SaveNeighborEdges( + Av1NeighborArrayUnit neighbors, + Point blockOrigin, + int width, + int height, + Span storage, + ref int offset) + where T : struct + { + Span top = MemoryMarshal.AsBytes( + neighbors.Top.Slice(neighbors.GetTopIndex(blockOrigin), width)); + + top.CopyTo(storage[offset..]); + offset += top.Length; + Span left = MemoryMarshal.AsBytes( + neighbors.Left.Slice(neighbors.GetLeftIndex(blockOrigin), height)); + + left.CopyTo(storage[offset..]); + offset += left.Length; + } + + private static void RestoreNeighborEdges( + Av1NeighborArrayUnit neighbors, + Point blockOrigin, + int width, + int height, + ReadOnlySpan storage, + ref int offset) + where T : struct + { + Span top = MemoryMarshal.AsBytes( + neighbors.Top.Slice(neighbors.GetTopIndex(blockOrigin), width)); + + storage.Slice(offset, top.Length).CopyTo(top); + offset += top.Length; + Span left = MemoryMarshal.AsBytes( + neighbors.Left.Slice(neighbors.GetLeftIndex(blockOrigin), height)); + + storage.Slice(offset, left.Length).CopyTo(left); + offset += left.Length; + } + + private Av1RateDistortionStatistics GetRegularBlockCost( + Av1SymbolEncoder writer, + Av1MacroBlockD macroBlock, + Av1RateDistortionStatistics modeStatistics, + bool allowIntraBlockCopy) + { + int rateAdjustment = writer.GetSkipCost(false, Av1TileWriter.GetSkipContext(macroBlock)); + if (!this.picture.Parent.FrameHeader.IsIntra) + { + int intraInterContext = Av1TileWriter.GetIntraInterContext(macroBlock); + rateAdjustment += writer.GetIsInterCost(false, intraInterContext); + } + + if (allowIntraBlockCopy) + { + rateAdjustment += writer.GetUseIntraBlockCopyCost(false); + } + + return new(this.rateMultiplier, modeStatistics.Rate + rateAdjustment, modeStatistics.Distortion); + } + + private Av1PredictionMode SelectLumaMode( + Av1SymbolEncoder writer, + Av1MacroBlockD macroBlock, + Point blockOrigin, + Av1BlockSize blockSize, + ushort tileIndex, + Span retainedCoefficients, + Span retainedStates, + ref Av1EncoderPaletteInfo paletteInfo, + out int selectedAngleDelta, + out Av1FilterIntraMode selectedFilterIntraMode, + out Av1TransformSize selectedTransformSize, + out Av1RateDistortionStatistics selectedStatistics) + { + bool codedLossless = this.picture.Parent.FrameHeader.CodedLossless; + Av1TransformSize transformSize = codedLossless + ? Av1TransformSize.Size4x4 + : blockSize.GetMaximumTransformSize(); + + int blockWidth = blockSize.GetWidth(); + int blockHeight = blockSize.GetHeight(); + Av1EncoderModeDecisionWorkspace workspace = + this.blockWorkspace.GetModeDecisionWorkspace(); + + Buffer2DRegion sourcePlane = this.source.GetPlane(Av1Plane.Y); + Buffer2DRegion reconstructionPlane = this.reconstruction.GetPlane(Av1Plane.Y); + if (blockWidth > transformSize.GetWidth() || blockHeight > transformSize.GetHeight()) + { + return this.SelectTiledLumaMode( + writer, + macroBlock, + blockOrigin, + blockSize, + tileIndex, + transformSize, + retainedCoefficients, + retainedStates, + out selectedAngleDelta, + out selectedFilterIntraMode, + out selectedTransformSize, + out selectedStatistics); + } + + bool hasLeft = macroBlock.IsLeftAvailable; + bool hasAbove = macroBlock.IsUpAvailable; + int modeInfoRow = blockOrigin.Y >> Av1Constants.ModeInfoSizeLog2; + int modeInfoColumn = blockOrigin.X >> Av1Constants.ModeInfoSizeLog2; + bool rightAvailable = modeInfoColumn + transformSize.Get4x4WideCount() < macroBlock.Tile.ModeInfoColumnEnd; + bool bottomAvailable = modeInfoRow + transformSize.Get4x4HighCount() < macroBlock.Tile.ModeInfoRowEnd; + Av1PartitionType partitionType = macroBlock.GetRelativeModeInfo(0).Block.PartitionType; + bool hasTopRight = Av1IntraReferenceAvailability.HasTopRight( + this.picture.Sequence.SequenceHeader.SuperblockSize, + blockSize, + modeInfoRow, + modeInfoColumn, + hasAbove, + rightAvailable, + partitionType, + transformSize, + 0, + 0, + 0, + 0); + + bool hasBottomLeft = Av1IntraReferenceAvailability.HasBottomLeft( + this.picture.Sequence.SequenceHeader.SuperblockSize, + blockSize, + modeInfoRow, + modeInfoColumn, + bottomAvailable, + hasLeft, + partitionType, + transformSize, + 0, + 0, + 0, + 0); + + Span aboveStorage = workspace.GetReferenceSamples(0); + Span leftStorage = workspace.GetReferenceSamples(1); + PrepareReferenceSamples( + reconstructionPlane, + blockOrigin, + blockWidth, + blockHeight, + hasLeft, + hasAbove, + hasTopRight, + hasBottomLeft, + this.bitDepth, + aboveStorage, + leftStorage); + + ReadOnlySpan above = aboveStorage.Slice(1, blockWidth + blockHeight); + ReadOnlySpan left = leftStorage.Slice(1, blockWidth + blockHeight); + + Av1TransformBlockContext blockContext = Av1TileWriter.GetTransformBlockContexts( + Av1ComponentType.Luminance, + this.picture.LuminanceDcSignLevelCoefficientNeighbors[tileIndex], + blockOrigin, + blockSize, + transformSize); + + int transformSizeContext = Av1TileWriter.GetTransformSizeContext( + this.picture.TransformFunctionContexts[tileIndex], + macroBlock, + blockOrigin, + blockSize); + + int largestTransformRate = this.picture.Parent.FrameHeader.TransformMode == Av1TransformMode.Select + && blockSize > Av1BlockSize.Block4x4 + ? writer.GetTransformSizeCost(blockSize, transformSize, transformSizeContext) + : 0; + + int paletteDisabledCost = 0; + if (Av1TileWriter.IsPaletteAllowed( + this.picture.Parent.FrameHeader.AllowScreenContentTools, + blockSize)) + { + Av1NeighborArrayUnit paletteContexts = this.picture.PaletteContexts[tileIndex]; + int blockSizeContext = Av1TileWriter.GetPaletteBlockSizeContext(blockSize); + int neighborContext = Av1TileWriter.GetPaletteYModeContext( + paletteContexts, + macroBlock, + blockOrigin); + + paletteDisabledCost = writer.GetPaletteYModeCost( + false, + blockSizeContext, + neighborContext); + } + + Span candidateReconstruction = workspace.GetCandidateReconstruction(0); + Span candidateCoefficients = workspace.GetCandidateCoefficients(0); + Span prediction = workspace.Prediction; + Span residual = workspace.Residual; + Av1RateDistortionStatistics bestStatistics = Av1RateDistortionStatistics.Invalid; + Av1PredictionMode bestMode = Av1PredictionMode.DC; + selectedAngleDelta = 0; + selectedFilterIntraMode = Av1FilterIntraMode.AllFilterIntraModes; + selectedTransformSize = transformSize; + int baseModeCount = LumaModeSearchOrder.Length; + int deltaCount = AngleDeltaSearchOrder.Length; + int directionalModeCount = (int)Av1PredictionMode.Directional67Degrees - (int)Av1PredictionMode.Vertical + 1; + + // Effort zero evaluates DC only, effort one adds every zero-angle mode, and higher levels add all directional adjustments. + int candidateCount = this.effort switch + { + 0 => 1, + 1 => baseModeCount, + _ when blockSize >= Av1BlockSize.Block8x8 => baseModeCount + (directionalModeCount * deltaCount), + _ => baseModeCount + }; + + bool useReducedTransformSet = this.picture.Parent.FrameHeader.UseReducedTransformSet; + Av1TransformSetType transformSetType = Av1SymbolContextHelper.GetExtendedTransformSetType( + transformSize, + useReducedTransformSet); + + // Transform type and transform size are separate search axes. Splitting their effort thresholds + // gives callers a useful intermediate tier without changing the fast default path. + bool searchEveryTransformType = !codedLossless && this.effort >= 7; + bool searchEveryTransformSize = !codedLossless && + this.effort >= 8 && + transformSize == Av1TransformSize.Size8x8; + + // Zero-angle modes precede groups of six nonzero adjustments for each directional mode. + // A single index preserves that tie-breaking order without duplicating candidate evaluation. + for (int candidateIndex = 0; candidateIndex < candidateCount; candidateIndex++) + { + Av1PredictionMode mode; + int angleDelta; + if (candidateIndex < baseModeCount) + { + mode = LumaModeSearchOrder[candidateIndex]; + angleDelta = 0; + } + else + { + int adjustedIndex = candidateIndex - baseModeCount; + mode = (Av1PredictionMode)((int)Av1PredictionMode.Vertical + (adjustedIndex / deltaCount)); + angleDelta = AngleDeltaSearchOrder[adjustedIndex % deltaCount]; + } + + // Prediction and subtraction do not depend on transform type. Preparing them once keeps + // exhaustive transform search from repeating the same pixel traversal for every candidate. + TOperator.PrepareIntra( + this.blockWorkspace, + sourcePlane, + blockOrigin, + prediction, + above, + left, + hasLeft, + hasAbove, + mode, + angleDelta, + this.picture.Sequence.SequenceHeader.EnableIntraEdgeFilter, + this.UseSmoothIntraEdges(macroBlock, blockOrigin, blockSize, Av1Plane.Y), + residual, + transformSize, + this.bitDepth); + + // Transform types are visited in AV1 enumeration order. A strict cost comparison below keeps + // the first legal type on ties, while lower efforts visit only the mode-derived default. + Av1TransformType firstTransformType = codedLossless + ? Av1TransformType.DctDct + : searchEveryTransformType + ? Av1TransformType.DctDct + : Av1SymbolContextHelper.GetDefaultIntraTransformType( + mode, + transformSize, + useReducedTransformSet); + + Av1TransformType transformTypeLimit = searchEveryTransformType + ? Av1TransformType.AllTransformTypes + : (Av1TransformType)((int)firstTransformType + 1); + + for (Av1TransformType transformType = firstTransformType; + transformType < transformTypeLimit; + transformType++) + { + if (!transformType.IsExtendedSetUsed(transformSetType)) + { + continue; + } + + Av1EncoderTransformBlockState candidateState = default; + Av1RateDistortionStatistics candidateStatistics = this.GetLumaCandidateCost( + writer, + macroBlock, + sourcePlane, + blockOrigin, + prediction, + residual, + mode, + angleDelta, + blockSize, + transformSize, + transformType, + blockContext, + paletteDisabledCost, + largestTransformRate, + candidateReconstruction, + candidateCoefficients, + ref candidateState); + + if (candidateStatistics.Cost < bestStatistics.Cost) + { + // The shared candidate spans are overwritten by the next transform. Copy only a + // global improvement into final block storage so no per-mode retained buffer is needed. + CopyCandidate( + candidateReconstruction, + candidateCoefficients, + reconstructionPlane, + blockOrigin, + retainedCoefficients, + transformSize, + candidateState, + ref retainedStates[0]); + + bestStatistics = candidateStatistics; + bestMode = mode; + selectedAngleDelta = angleDelta; + selectedTransformSize = transformSize; + } + } + + // At exhaustive effort, transform size belongs to this mode's RD result. Evaluate it + // before advancing so an 8x8-only preliminary result cannot discard a better split mode. + if (searchEveryTransformSize && + this.picture.Parent.FrameHeader.TransformMode == Av1TransformMode.Select) + { + Av1RateDistortionStatistics splitStatistics = this.GetSplitLumaCandidateCost( + writer, + macroBlock, + sourcePlane, + reconstructionPlane, + blockOrigin, + tileIndex, + mode, + angleDelta, + Av1FilterIntraMode.AllFilterIntraModes, + 0, + ReadOnlySpan.Empty, + 0, + paletteDisabledCost, + transformSizeContext, + bestStatistics.Cost, + candidateReconstruction, + candidateCoefficients, + workspace.CandidateTransformBlocks); + + if (splitStatistics.Cost < bestStatistics.Cost) + { + CopySplitCandidate( + candidateReconstruction, + candidateCoefficients, + workspace.CandidateTransformBlocks, + reconstructionPlane, + blockOrigin, + retainedCoefficients, + retainedStates); + + bestStatistics = splitStatistics; + bestMode = mode; + selectedAngleDelta = angleDelta; + selectedTransformSize = Av1TransformSize.Size4x4; + } + } + } + + // Midrange effort refines the preliminary mode only. Higher effort already searched every + // mode-transform pair above, so repeating the winning mode would add no candidates. + Av1RateDistortionStatistics bestTransformStatistics = bestStatistics; + if (!codedLossless && this.effort >= 3 && !searchEveryTransformType) + { + // The shared spans now contain the last mode visited above, so rebuild the preliminary + // winner once before refining its transform types. + TOperator.PrepareIntra( + this.blockWorkspace, + sourcePlane, + blockOrigin, + prediction, + above, + left, + hasLeft, + hasAbove, + bestMode, + selectedAngleDelta, + this.picture.Sequence.SequenceHeader.EnableIntraEdgeFilter, + this.UseSmoothIntraEdges(macroBlock, blockOrigin, blockSize, Av1Plane.Y), + residual, + transformSize, + this.bitDepth); + + for (Av1TransformType transformType = Av1TransformType.DctDct; + transformType < Av1TransformType.AllTransformTypes; + transformType++) + { + if (!transformType.IsExtendedSetUsed(transformSetType)) + { + continue; + } + + Av1EncoderTransformBlockState candidateState = default; + Av1RateDistortionStatistics candidateStatistics = this.GetLumaCandidateCost( + writer, + macroBlock, + sourcePlane, + blockOrigin, + prediction, + residual, + bestMode, + selectedAngleDelta, + blockSize, + transformSize, + transformType, + blockContext, + paletteDisabledCost, + largestTransformRate, + candidateReconstruction, + candidateCoefficients, + ref candidateState); + + if (candidateStatistics.Cost < bestTransformStatistics.Cost) + { + CopyCandidate( + candidateReconstruction, + candidateCoefficients, + reconstructionPlane, + blockOrigin, + retainedCoefficients, + transformSize, + candidateState, + ref retainedStates[0]); + + bestTransformStatistics = candidateStatistics; + } + } + } + + if (this.effort >= 4 && + Av1TileWriter.IsFilterIntraAllowedBlockSize( + this.picture.Sequence.SequenceHeader.EnableFilterIntra, + blockSize)) + { + // Each recursive filter prediction and its source residual are independent of transform type. + // Prepare them once per filter mode so all legal transforms reuse the same samples. + for (Av1FilterIntraMode filterIntraMode = Av1FilterIntraMode.DC; + filterIntraMode < Av1FilterIntraMode.AllFilterIntraModes; + filterIntraMode++) + { + TOperator.PrepareFilterIntra( + this.blockWorkspace, + sourcePlane, + blockOrigin, + prediction, + above, + left, + residual, + filterIntraMode, + transformSize, + this.bitDepth); + + Av1TransformType filterTransformTypeLimit = codedLossless + ? (Av1TransformType)((int)Av1TransformType.DctDct + 1) + : Av1TransformType.AllTransformTypes; + + for (Av1TransformType transformType = Av1TransformType.DctDct; + transformType < filterTransformTypeLimit; + transformType++) + { + if (!transformType.IsExtendedSetUsed(transformSetType)) + { + continue; + } + + Av1EncoderTransformBlockState candidateState = default; + Av1RateDistortionStatistics candidateStatistics = this.GetFilterIntraCandidateCost( + writer, + macroBlock, + sourcePlane, + blockOrigin, + prediction, + residual, + filterIntraMode, + blockSize, + transformSize, + transformType, + blockContext, + paletteDisabledCost, + largestTransformRate, + candidateReconstruction, + candidateCoefficients, + ref candidateState); + + if (candidateStatistics.Cost < bestTransformStatistics.Cost) + { + CopyCandidate( + candidateReconstruction, + candidateCoefficients, + reconstructionPlane, + blockOrigin, + retainedCoefficients, + transformSize, + candidateState, + ref retainedStates[0]); + + bestTransformStatistics = candidateStatistics; + bestMode = Av1PredictionMode.DC; + selectedAngleDelta = 0; + selectedFilterIntraMode = filterIntraMode; + selectedTransformSize = transformSize; + } + } + + // Filter-intra mode and transform size form one candidate for RD comparison, just as + // ordinary spatial mode and transform size do in the exhaustive search above. + if (searchEveryTransformSize && + this.picture.Parent.FrameHeader.TransformMode == Av1TransformMode.Select) + { + Av1RateDistortionStatistics splitStatistics = this.GetSplitLumaCandidateCost( + writer, + macroBlock, + sourcePlane, + reconstructionPlane, + blockOrigin, + tileIndex, + Av1PredictionMode.DC, + 0, + filterIntraMode, + 0, + ReadOnlySpan.Empty, + 0, + paletteDisabledCost, + transformSizeContext, + bestTransformStatistics.Cost, + candidateReconstruction, + candidateCoefficients, + workspace.CandidateTransformBlocks); + + if (splitStatistics.Cost < bestTransformStatistics.Cost) + { + CopySplitCandidate( + candidateReconstruction, + candidateCoefficients, + workspace.CandidateTransformBlocks, + reconstructionPlane, + blockOrigin, + retainedCoefficients, + retainedStates); + + bestTransformStatistics = splitStatistics; + bestMode = Av1PredictionMode.DC; + selectedAngleDelta = 0; + selectedFilterIntraMode = filterIntraMode; + selectedTransformSize = Av1TransformSize.Size4x4; + } + } + } + } + + if (this.effort >= 5 && + blockSize == Av1BlockSize.Block8x8 && + this.picture.Parent.FrameHeader.AllowScreenContentTools && + this.SelectLumaPalette( + writer, + macroBlock, + sourcePlane, + reconstructionPlane, + blockOrigin, + tileIndex, + transformSetType, + blockContext, + largestTransformRate, + transformSizeContext, + candidateReconstruction, + candidateCoefficients, + retainedCoefficients, + retainedStates, + ref bestTransformStatistics, + ref paletteInfo, + ref selectedTransformSize)) + { + bestMode = Av1PredictionMode.DC; + selectedAngleDelta = 0; + selectedFilterIntraMode = Av1FilterIntraMode.AllFilterIntraModes; + } + + // Efforts six and seven save work by testing transform size only for the global non-palette + // winner. Effort eight and above already tested both sizes inside every candidate. + if (this.effort >= 6 && + transformSize == Av1TransformSize.Size8x8 && + !searchEveryTransformSize && + this.picture.Parent.FrameHeader.TransformMode == Av1TransformMode.Select && + paletteInfo.PaletteSizes[0] == 0) + { + Av1RateDistortionStatistics splitStatistics = this.GetSplitLumaCandidateCost( + writer, + macroBlock, + sourcePlane, + reconstructionPlane, + blockOrigin, + tileIndex, + bestMode, + selectedAngleDelta, + selectedFilterIntraMode, + 0, + ReadOnlySpan.Empty, + 0, + paletteDisabledCost, + transformSizeContext, + bestTransformStatistics.Cost, + candidateReconstruction, + candidateCoefficients, + workspace.CandidateTransformBlocks); + + if (splitStatistics.Cost < bestTransformStatistics.Cost) + { + CopySplitCandidate( + candidateReconstruction, + candidateCoefficients, + workspace.CandidateTransformBlocks, + reconstructionPlane, + blockOrigin, + retainedCoefficients, + retainedStates); + + bestTransformStatistics = splitStatistics; + selectedTransformSize = Av1TransformSize.Size4x4; + } + } + + selectedStatistics = bestTransformStatistics; + return bestMode; + } + + private Av1PredictionMode SelectTiledLumaMode( + Av1SymbolEncoder writer, + Av1MacroBlockD macroBlock, + Point blockOrigin, + Av1BlockSize blockSize, + ushort tileIndex, + Av1TransformSize transformSize, + Span retainedCoefficients, + Span retainedStates, + out int selectedAngleDelta, + out Av1FilterIntraMode selectedFilterIntraMode, + out Av1TransformSize selectedTransformSize, + out Av1RateDistortionStatistics selectedStatistics) + { + Av1EncoderModeDecisionWorkspace workspace = + this.blockWorkspace.GetModeDecisionWorkspace(); + + int blockWidth = blockSize.GetWidth(); + int blockHeight = blockSize.GetHeight(); + int blockSampleCount = blockWidth * blockHeight; + int transformBlockCount = blockSampleCount / transformSize.GetSize2d(); + Span candidateReconstruction = + workspace.GetCandidateReconstruction(0)[..blockSampleCount]; + + Span candidateCoefficients = + workspace.GetCandidateCoefficients(0)[..blockSampleCount]; + + Span candidateStates = + workspace.CandidateTransformBlocks[..transformBlockCount]; + + int contextWidth = blockSize.Get4x4WideCount(); + int contextHeight = blockSize.Get4x4HighCount(); + Span contexts = workspace.TransformContexts; + Span topContexts = contexts[..contextWidth]; + Span leftContexts = contexts.Slice(contextWidth, contextHeight); + Av1NeighborArrayUnit coefficientNeighbors = + this.picture.LuminanceDcSignLevelCoefficientNeighbors[tileIndex]; + + int topIndex = coefficientNeighbors.GetTopIndex(blockOrigin); + int leftIndex = coefficientNeighbors.GetLeftIndex(blockOrigin); + int transformSizeContext = Av1TileWriter.GetTransformSizeContext( + this.picture.TransformFunctionContexts[tileIndex], + macroBlock, + blockOrigin, + blockSize); + + int transformSizeRate = this.picture.Parent.FrameHeader.TransformMode == Av1TransformMode.Select + ? writer.GetTransformSizeCost(blockSize, transformSize, transformSizeContext) + : 0; + + int paletteDisabledCost = Av1TileWriter.IsPaletteAllowed( + this.picture.Parent.FrameHeader.AllowScreenContentTools, + blockSize) + ? writer.GetPaletteYModeCost( + false, + Av1TileWriter.GetPaletteBlockSizeContext(blockSize), + Av1TileWriter.GetPaletteYModeContext( + this.picture.PaletteContexts[tileIndex], + macroBlock, + blockOrigin)) + : 0; + + int baseModeCount = LumaModeSearchOrder.Length; + int deltaCount = AngleDeltaSearchOrder.Length; + int directionalModeCount = + (int)Av1PredictionMode.Directional67Degrees - (int)Av1PredictionMode.Vertical + 1; + + // A lossless 4x8 or 8x4 block has multiple 4x4 transforms but carries no angle-delta symbol. + // Its predictor must therefore use the unadjusted direction, just as the decoder does. + int candidateCount = this.effort switch + { + 0 => 1, + 1 => baseModeCount, + _ when blockSize >= Av1BlockSize.Block8x8 => baseModeCount + (directionalModeCount * deltaCount), + _ => baseModeCount + }; + + Buffer2DRegion sourcePlane = this.source.GetPlane(Av1Plane.Y); + Buffer2DRegion reconstructionPlane = this.reconstruction.GetPlane(Av1Plane.Y); + Av1RateDistortionStatistics bestStatistics = Av1RateDistortionStatistics.Invalid; + Av1PredictionMode bestMode = Av1PredictionMode.DC; + selectedAngleDelta = 0; + selectedFilterIntraMode = Av1FilterIntraMode.AllFilterIntraModes; + selectedTransformSize = transformSize; + + // Each candidate starts from the live block-edge contexts. Transform updates remain local until + // that candidate wins, so later modes never inherit state from an earlier trial. + for (int candidateIndex = 0; candidateIndex < candidateCount; candidateIndex++) + { + Av1PredictionMode mode; + int angleDelta; + if (candidateIndex < baseModeCount) + { + mode = LumaModeSearchOrder[candidateIndex]; + angleDelta = 0; + } + else + { + int adjustedIndex = candidateIndex - baseModeCount; + mode = (Av1PredictionMode)( + (int)Av1PredictionMode.Vertical + (adjustedIndex / deltaCount)); + + angleDelta = AngleDeltaSearchOrder[adjustedIndex % deltaCount]; + } + + coefficientNeighbors.Top.Slice(topIndex, contextWidth).CopyTo(topContexts); + coefficientNeighbors.Left.Slice(leftIndex, contextHeight).CopyTo(leftContexts); + long distortion = this.GetTiledPlaneCost( + writer, + macroBlock, + blockOrigin, + blockOrigin, + blockSize, + blockSize, + transformSize, + Av1BlockSize.Block64x64, + 0, + 0, + mode, + mode, + angleDelta, + Av1Plane.Y, + sourcePlane, + reconstructionPlane, + candidateReconstruction, + candidateCoefficients, + candidateStates, + topContexts, + leftContexts, + out int coefficientRate); + + int rate = Av1TileWriter.GetLumaModeCost( + writer, + macroBlock, + blockSize, + mode, + angleDelta, + this.picture.Parent.FrameHeader.IsIntra); + + rate += transformSizeRate + coefficientRate; + if (mode == Av1PredictionMode.DC) + { + rate += paletteDisabledCost; + if (Av1TileWriter.IsFilterIntraAllowedBlockSize( + this.picture.Sequence.SequenceHeader.EnableFilterIntra, + blockSize)) + { + rate += writer.GetFilterIntraModeCost( + Av1FilterIntraMode.AllFilterIntraModes, + blockSize); + } + } + + Av1RateDistortionStatistics candidateStatistics = new(this.rateMultiplier, rate, distortion); + if (candidateStatistics.Cost < bestStatistics.Cost) + { + CopyTiledCandidate( + candidateReconstruction, + candidateCoefficients, + candidateStates, + reconstructionPlane, + blockOrigin, + blockWidth, + blockHeight, + transformSize, + retainedCoefficients, + retainedStates); + + bestStatistics = candidateStatistics; + bestMode = mode; + selectedAngleDelta = angleDelta; + } + } + + selectedStatistics = bestStatistics; + return bestMode; + } + + private Av1RateDistortionStatistics GetSplitLumaCandidateCost( + Av1SymbolEncoder writer, + Av1MacroBlockD macroBlock, + Buffer2DRegion sourcePlane, + Buffer2DRegion reconstructionPlane, + Point blockOrigin, + ushort tileIndex, + Av1PredictionMode mode, + int angleDelta, + Av1FilterIntraMode filterIntraMode, + int paletteSize, + scoped ReadOnlySpan paletteColors, + int paletteHeaderRate, + int paletteDisabledCost, + int transformSizeContext, + long costLimit, + Span candidateReconstruction, + Span candidateCoefficients, + Span candidateTransformBlocks) + { + const Av1BlockSize BlockSize = Av1BlockSize.Block8x8; + const Av1TransformSize TransformSize = Av1TransformSize.Size4x4; + const int BlockWidth = 8; + const int TransformWidth = 4; + const int TransformSampleCount = TransformWidth * TransformWidth; + Av1EncoderModeDecisionWorkspace workspace = + this.blockWorkspace.GetModeDecisionWorkspace(); + + // One transient 64-sample plane holds the 4x4 prediction followed by two reconstruction buffers. + // The disjoint views stay live together and need no additional owner or allocator rent. + Span transformSamples = workspace.GetCandidateReconstruction(1); + Span prediction = transformSamples[..TransformSampleCount]; + Span candidateTransformReconstruction = transformSamples.Slice( + TransformSampleCount, + TransformSampleCount); + + Span bestTransformReconstruction = transformSamples.Slice( + TransformSampleCount * 2, + TransformSampleCount); + + Span transformCoefficientStorage = workspace.GetCandidateCoefficients(1); + Span candidateTransformCoefficients = transformCoefficientStorage[..TransformSampleCount]; + Span bestTransformCoefficients = transformCoefficientStorage.Slice( + TransformSampleCount, + TransformSampleCount); + + Span residual = workspace.Residual[..TransformSampleCount]; + Span contexts = workspace.TransformContexts; + Span topContexts = contexts[..2]; + Span leftContexts = contexts[2..4]; + Av1NeighborArrayUnit coefficientNeighbors = + this.picture.LuminanceDcSignLevelCoefficientNeighbors[tileIndex]; + + int topIndex = coefficientNeighbors.GetTopIndex(blockOrigin); + int leftIndex = coefficientNeighbors.GetLeftIndex(blockOrigin); + coefficientNeighbors.Top.Slice(topIndex, 2).CopyTo(topContexts); + coefficientNeighbors.Left.Slice(leftIndex, 2).CopyTo(leftContexts); + bool useReducedTransformSet = this.picture.Parent.FrameHeader.UseReducedTransformSet; + Av1TransformSetType transformSetType = Av1SymbolContextHelper.GetExtendedTransformSetType( + TransformSize, + useReducedTransformSet); + + // Prediction-mode and transform-size symbols belong to the 8x8 coding block, while each + // 4x4 transform contributes its own coefficient rate below. + int rate = writer.GetTransformSizeCost(BlockSize, TransformSize, transformSizeContext); + if (paletteSize > 0) + { + rate += paletteHeaderRate; + } + else + { + rate += Av1TileWriter.GetLumaModeCost( + writer, + macroBlock, + BlockSize, + mode, + angleDelta, + this.picture.Parent.FrameHeader.IsIntra); + + if (mode == Av1PredictionMode.DC) + { + rate += paletteDisabledCost; + if (this.picture.Sequence.SequenceHeader.EnableFilterIntra) + { + rate += writer.GetFilterIntraModeCost( + filterIntraMode, + BlockSize); + } + } + } + + Buffer2DRegion colorIndexMap = default; + if (paletteSize > 0) + { + colorIndexMap = this.superblock.Workspace + .GetPaletteMaps() + .GetMap(Av1PlaneType.Y, BlockWidth, BlockWidth); + } + + long distortion = 0; + + // Raster order is observable here: each retained 4x4 reconstruction supplies reference + // samples and coefficient context to transforms that follow it in the same coding block. + for (int transformRow = 0; transformRow < 2; transformRow++) + { + for (int transformColumn = 0; transformColumn < 2; transformColumn++) + { + int transformIndex = (transformRow * 2) + transformColumn; + int reconstructionOffset = + (transformRow * TransformWidth * BlockWidth) + (transformColumn * TransformWidth); + + Point transformOrigin = blockOrigin + new Size( + transformColumn * TransformWidth, + transformRow * TransformWidth); + + if (paletteSize > 0) + { + // Palette prediction is block-local. A view over the retained map avoids copying indices or + // preparing reconstructed neighbor edges that this prediction mode cannot consume. + TOperator.PreparePalette( + sourcePlane, + transformOrigin, + paletteColors, + colorIndexMap.GetSubRegion( + new Rectangle( + transformColumn * TransformWidth, + transformRow * TransformWidth, + TransformWidth, + TransformWidth)), + prediction, + residual, + TransformSize); + } + else + { + Span aboveStorage = workspace.GetReferenceSamples(0); + Span leftStorage = workspace.GetReferenceSamples(1); + this.PrepareTransformReferenceSamples( + reconstructionPlane, + blockOrigin, + blockOrigin, + BlockSize, + macroBlock, + transformRow, + transformColumn, + BlockWidth, + TransformSize, + 0, + 0, + candidateReconstruction, + aboveStorage, + leftStorage, + out bool hasLeft, + out bool hasAbove); + + if (filterIntraMode == Av1FilterIntraMode.AllFilterIntraModes) + { + TOperator.PrepareIntra( + this.blockWorkspace, + sourcePlane, + transformOrigin, + prediction, + aboveStorage.Slice(1, TransformWidth * 2), + leftStorage.Slice(1, TransformWidth * 2), + hasLeft, + hasAbove, + mode, + angleDelta, + this.picture.Sequence.SequenceHeader.EnableIntraEdgeFilter, + this.UseSmoothIntraEdges(macroBlock, blockOrigin, BlockSize, Av1Plane.Y), + residual, + TransformSize, + this.bitDepth); + } + else + { + // Filter-intra prediction is recursive within each transform unit, so rebuild it from + // the reconstructed edges established by the preceding 4x4 candidate. + TOperator.PrepareFilterIntra( + this.blockWorkspace, + sourcePlane, + transformOrigin, + prediction, + aboveStorage.Slice(1, TransformWidth * 2), + leftStorage.Slice(1, TransformWidth * 2), + residual, + filterIntraMode, + TransformSize, + this.bitDepth); + } + } + + Av1TransformBlockContext blockContext = Av1TileWriter.GetTransformBlockContexts( + Av1ComponentType.Luminance, + topContexts.Slice(transformColumn, 1), + leftContexts.Slice(transformRow, 1), + BlockSize, + TransformSize); + + long bestTransformCost = long.MaxValue; + Av1TransformType bestTransformType = Av1TransformType.DctDct; + int bestTransformRate = 0; + long bestTransformDistortion = 0; + Av1EncoderTransformBlockState bestTransformState = default; + Span retainedTransformCoefficients = candidateCoefficients.Slice( + transformIndex * TransformSampleCount, + TransformSampleCount); + + // Each transform writes into the compact buffer that does not hold the current best. + // Swapping spans on improvement keeps the winner without copying it inside the search loop. + for (Av1TransformType transformType = Av1TransformType.DctDct; + transformType < Av1TransformType.AllTransformTypes; + transformType++) + { + if (!transformType.IsExtendedSetUsed(transformSetType)) + { + continue; + } + + Av1EncoderTransformBlockState candidateState = default; + long candidateDistortion = TOperator.EncodePredictionCandidate( + this.blockWorkspace, + sourcePlane, + transformOrigin, + prediction, + residual, + candidateTransformReconstruction, + TransformWidth, + candidateTransformCoefficients, + TransformSize, + transformType, + Av1Plane.Y, + this.quantization.QIndex[0], + this.quantization.DeltaQDc[(int)Av1Plane.Y], + this.quantization.DeltaQAc[(int)Av1Plane.Y], + this.bitDepth, + ref candidateState); + + int candidateRate = writer.GetCoefficientCost( + TransformSize, + transformType, + mode, + candidateTransformCoefficients, + Av1ComponentType.Luminance, + blockContext, + candidateState.EndOfBlock, + useReducedTransformSet, + filterIntraMode, + usesInterTransformSet: false); + + long candidateCost = Av1RateDistortion.GetCost( + this.rateMultiplier, + candidateRate, + candidateDistortion); + + if (candidateCost < bestTransformCost) + { + Span previousBestReconstruction = bestTransformReconstruction; + bestTransformReconstruction = candidateTransformReconstruction; + candidateTransformReconstruction = previousBestReconstruction; + + Span previousBestCoefficients = bestTransformCoefficients; + bestTransformCoefficients = candidateTransformCoefficients; + candidateTransformCoefficients = previousBestCoefficients; + + bestTransformCost = candidateCost; + bestTransformType = transformType; + bestTransformRate = candidateRate; + bestTransformDistortion = candidateDistortion; + bestTransformState = candidateState; + } + } + + // The next 4x4 prediction consumes this reconstruction from the block mosaic. Publish the + // final winner once, after transform search, along with its entropy-context coefficients. + bestTransformCoefficients.CopyTo(retainedTransformCoefficients); + for (int row = 0; row < TransformWidth; row++) + { + bestTransformReconstruction.Slice(row * TransformWidth, TransformWidth) + .CopyTo( + candidateReconstruction.Slice( + reconstructionOffset + (row * BlockWidth), + TransformWidth)); + } + + rate += bestTransformRate; + distortion += bestTransformDistortion; + candidateTransformBlocks[transformIndex] = bestTransformState; + byte coefficientContext = Av1SymbolContextHelper.GetCoefficientContext( + retainedTransformCoefficients, + TransformSize, + bestTransformType, + bestTransformState.EndOfBlock); + + topContexts[transformColumn] = coefficientContext; + leftContexts[transformRow] = coefficientContext; + + // Every remaining transform can only add nonnegative rate and distortion. + if (Av1RateDistortion.GetCost(this.rateMultiplier, rate, distortion) >= costLimit) + { + return Av1RateDistortionStatistics.Invalid; + } + } + } + + return new(this.rateMultiplier, rate, distortion); + } + + private void PrepareTransformReferenceSamples( + Buffer2DRegion reconstructionPlane, + Point lumaBlockOrigin, + Point planeBlockOrigin, + Av1BlockSize blockSize, + Av1MacroBlockD macroBlock, + int transformRow, + int transformColumn, + int planeBlockWidth, + Av1TransformSize transformSize, + int subsamplingX, + int subsamplingY, + ReadOnlySpan candidateReconstruction, + Span aboveStorage, + Span leftStorage, + out bool hasLeft, + out bool hasAbove) + { + int transformWidth = transformSize.GetWidth(); + int transformHeight = transformSize.GetHeight(); + int rowOffset = transformRow * transformHeight; + int columnOffset = transformColumn * transformWidth; + int modeInfoRow = lumaBlockOrigin.Y >> Av1Constants.ModeInfoSizeLog2; + int modeInfoColumn = lumaBlockOrigin.X >> Av1Constants.ModeInfoSizeLog2; + + // Internal top and left edges come from the candidate mosaic built in raster order. Edges outside + // the candidate continue to read committed reconstruction, keeping unsuccessful trials isolated. + hasAbove = transformRow > 0 || macroBlock.IsUpAvailable; + hasLeft = transformColumn > 0 || macroBlock.IsLeftAvailable; + int transformRow4x4 = rowOffset >> Av1Constants.ModeInfoSizeLog2; + int transformColumn4x4 = columnOffset >> Av1Constants.ModeInfoSizeLog2; + bool rightAvailable = + modeInfoColumn + + ((transformColumn4x4 + transformSize.Get4x4WideCount()) << subsamplingX) < + macroBlock.Tile.ModeInfoColumnEnd; + + bool bottomAvailable = + modeInfoRow + + ((transformRow4x4 + transformSize.Get4x4HighCount()) << subsamplingY) < + macroBlock.Tile.ModeInfoRowEnd; + + Av1PartitionType partitionType = macroBlock.GetRelativeModeInfo(0).Block.PartitionType; + bool hasTopRight = Av1IntraReferenceAvailability.HasTopRight( + this.picture.Sequence.SequenceHeader.SuperblockSize, + blockSize, + modeInfoRow, + modeInfoColumn, + hasAbove, + rightAvailable, + partitionType, + transformSize, + transformRow4x4, + transformColumn4x4, + subsamplingX, + subsamplingY); + + bool hasBottomLeft = Av1IntraReferenceAvailability.HasBottomLeft( + this.picture.Sequence.SequenceHeader.SuperblockSize, + blockSize, + modeInfoRow, + modeInfoColumn, + bottomAvailable, + hasLeft, + partitionType, + transformSize, + transformRow4x4, + transformColumn4x4, + subsamplingX, + subsamplingY); + + // Rectangular transforms project as far as width + height - 1 on either edge. + // The existing reference storage already holds this maximum; no additional scratch is needed. + Span above = aboveStorage.Slice(1, transformWidth + transformHeight); + Span left = leftStorage.Slice(1, transformWidth + transformHeight); + if (hasAbove) + { + if (transformRow > 0) + { + candidateReconstruction + .Slice(((rowOffset - 1) * planeBlockWidth) + columnOffset, transformWidth) + .CopyTo(above); + } + else + { + reconstructionPlane.DangerousGetRowSpan(planeBlockOrigin.Y - 1) + .Slice(planeBlockOrigin.X + columnOffset, transformWidth) + .CopyTo(above); + } + } + + if (hasLeft) + { + if (transformColumn > 0) + { + for (int row = 0; row < transformHeight; row++) + { + left[row] = candidateReconstruction[ + ((rowOffset + row) * planeBlockWidth) + columnOffset - 1]; + } + } + else + { + for (int row = 0; row < transformHeight; row++) + { + left[row] = reconstructionPlane + .DangerousGetRowSpan(planeBlockOrigin.Y + rowOffset + row)[planeBlockOrigin.X - 1]; + } + } + } + + int midpoint = 128 << (this.bitDepth.GetBitCount() - 8); + if (!hasAbove) + { + above[..transformWidth].Fill(hasLeft ? left[0] : TOperator.CreateSample(midpoint - 1)); + } + + if (!hasLeft) + { + left[..transformHeight].Fill(hasAbove ? above[0] : TOperator.CreateSample(midpoint + 1)); + } + + // Candidate mosaics share the committed frame's coded extent. Padding beyond that extent is + // never a reference sample, even when coding order makes the adjacent block available. + int topRightCount = hasTopRight + ? Math.Min( + Math.Min(transformWidth, transformHeight), + reconstructionPlane.Width - planeBlockOrigin.X - columnOffset - transformWidth) + : 0; + + if (hasTopRight) + { + if (transformRow > 0) + { + candidateReconstruction + .Slice( + ((rowOffset - 1) * planeBlockWidth) + columnOffset + transformWidth, + topRightCount) + .CopyTo(above[transformWidth..]); + } + else + { + reconstructionPlane.DangerousGetRowSpan(planeBlockOrigin.Y - 1) + .Slice(planeBlockOrigin.X + columnOffset + transformWidth, topRightCount) + .CopyTo(above[transformWidth..]); + } + } + + int topCount = transformWidth + topRightCount; + above[topCount..].Fill(above[topCount - 1]); + + int bottomLeftCount = hasBottomLeft + ? Math.Min( + Math.Min(transformHeight, transformWidth), + reconstructionPlane.Height - planeBlockOrigin.Y - rowOffset - transformHeight) + : 0; + + if (hasBottomLeft) + { + if (transformColumn > 0) + { + for (int row = transformHeight; row < transformHeight + bottomLeftCount; row++) + { + left[row] = candidateReconstruction[ + ((rowOffset + row) * planeBlockWidth) + columnOffset - 1]; + } + } + else + { + for (int row = transformHeight; row < transformHeight + bottomLeftCount; row++) + { + left[row] = reconstructionPlane + .DangerousGetRowSpan(planeBlockOrigin.Y + rowOffset + row)[planeBlockOrigin.X - 1]; + } + } + } + + int leftCount = transformHeight + bottomLeftCount; + left[leftCount..].Fill(left[leftCount - 1]); + + // Only an interior transform corner belongs to decision scratch. Boundary corners continue + // to read the already reconstructed neighboring block so candidate trials remain isolated. + TSample corner = hasAbove && hasLeft + ? transformRow > 0 && transformColumn > 0 + ? candidateReconstruction[((rowOffset - 1) * planeBlockWidth) + columnOffset - 1] + : reconstructionPlane.DangerousGetRowSpan(planeBlockOrigin.Y + rowOffset - 1)[ + planeBlockOrigin.X + columnOffset - 1] + : hasAbove + ? above[0] + : hasLeft + ? left[0] + : TOperator.CreateSample(midpoint); + + aboveStorage[0] = corner; + leftStorage[0] = corner; + } + + private Av1RateDistortionStatistics GetLumaCandidateCost( + Av1SymbolEncoder writer, + Av1MacroBlockD macroBlock, + Buffer2DRegion sourcePlane, + Point blockOrigin, + ReadOnlySpan prediction, + ReadOnlySpan residual, + Av1PredictionMode mode, + int angleDelta, + Av1BlockSize blockSize, + Av1TransformSize transformSize, + Av1TransformType transformType, + Av1TransformBlockContext blockContext, + int paletteDisabledCost, + int transformSizeRate, + Span candidateReconstruction, + Span candidateCoefficients, + ref Av1EncoderTransformBlockState candidateState) + { + // Prediction and subtraction were prepared by the owning mode loop. This stage performs only + // transform, quantization, reconstruction, and distortion for the requested transform type. + long distortion = TOperator.EncodePredictionCandidate( + this.blockWorkspace, + sourcePlane, + blockOrigin, + prediction, + residual, + candidateReconstruction, + transformSize.GetWidth(), + candidateCoefficients, + transformSize, + transformType, + Av1Plane.Y, + this.quantization.QIndex[0], + this.quantization.DeltaQDc[(int)Av1Plane.Y], + this.quantization.DeltaQAc[(int)Av1Plane.Y], + this.bitDepth, + ref candidateState); + + // Charge every block-level choice that distinguishes this spatial candidate before adding + // coefficient syntax derived from the live neighboring-transform context. + int rate = Av1TileWriter.GetLumaModeCost( + writer, + macroBlock, + blockSize, + mode, + angleDelta, + this.picture.Parent.FrameHeader.IsIntra); + + rate += transformSizeRate; + if (mode == Av1PredictionMode.DC) + { + rate += paletteDisabledCost; + } + + if (mode == Av1PredictionMode.DC && + Av1TileWriter.IsFilterIntraAllowedBlockSize( + this.picture.Sequence.SequenceHeader.EnableFilterIntra, + blockSize)) + { + rate += writer.GetFilterIntraModeCost(Av1FilterIntraMode.AllFilterIntraModes, blockSize); + } + + rate += writer.GetCoefficientCost( + transformSize, + transformType, + mode, + candidateCoefficients, + Av1ComponentType.Luminance, + blockContext, + candidateState.EndOfBlock, + this.picture.Parent.FrameHeader.UseReducedTransformSet, + Av1FilterIntraMode.AllFilterIntraModes, + usesInterTransformSet: false); + + return new(this.rateMultiplier, rate, distortion); + } + + private Av1RateDistortionStatistics GetFilterIntraCandidateCost( + Av1SymbolEncoder writer, + Av1MacroBlockD macroBlock, + Buffer2DRegion sourcePlane, + Point blockOrigin, + ReadOnlySpan prediction, + ReadOnlySpan residual, + Av1FilterIntraMode filterIntraMode, + Av1BlockSize blockSize, + Av1TransformSize transformSize, + Av1TransformType transformType, + Av1TransformBlockContext blockContext, + int paletteDisabledCost, + int transformSizeRate, + Span candidateReconstruction, + Span candidateCoefficients, + ref Av1EncoderTransformBlockState candidateState) + { + long distortion = TOperator.EncodePredictionCandidate( + this.blockWorkspace, + sourcePlane, + blockOrigin, + prediction, + residual, + candidateReconstruction, + transformSize.GetWidth(), + candidateCoefficients, + transformSize, + transformType, + Av1Plane.Y, + this.quantization.QIndex[0], + this.quantization.DeltaQDc[(int)Av1Plane.Y], + this.quantization.DeltaQAc[(int)Av1Plane.Y], + this.bitDepth, + ref candidateState); + + int rate = Av1TileWriter.GetLumaModeCost( + writer, + macroBlock, + blockSize, + Av1PredictionMode.DC, + 0, + this.picture.Parent.FrameHeader.IsIntra); + + rate += transformSizeRate; + rate += paletteDisabledCost; + rate += writer.GetFilterIntraModeCost(filterIntraMode, blockSize); + rate += writer.GetCoefficientCost( + transformSize, + transformType, + Av1PredictionMode.DC, + candidateCoefficients, + Av1ComponentType.Luminance, + blockContext, + candidateState.EndOfBlock, + this.picture.Parent.FrameHeader.UseReducedTransformSet, + filterIntraMode, + usesInterTransformSet: false); + + return new(this.rateMultiplier, rate, distortion); + } + + private static void CopyCandidate( + ReadOnlySpan candidateReconstruction, + ReadOnlySpan candidateCoefficients, + Buffer2DRegion reconstructionPlane, + Point blockOrigin, + Span retainedCoefficients, + Av1TransformSize transformSize, + Av1EncoderTransformBlockState candidateState, + ref Av1EncoderTransformBlockState retainedState) + { + int width = transformSize.GetWidth(); + int height = transformSize.GetHeight(); + candidateCoefficients[..transformSize.GetSize2d()].CopyTo(retainedCoefficients); + for (int row = 0; row < height; row++) + { + candidateReconstruction.Slice(row * width, width) + .CopyTo(reconstructionPlane.DangerousGetRowSpan(blockOrigin.Y + row).Slice(blockOrigin.X, width)); + } + + retainedState = candidateState; + } + + private static void CopySplitCandidate( + ReadOnlySpan candidateReconstruction, + ReadOnlySpan candidateCoefficients, + ReadOnlySpan candidateTransformBlocks, + Buffer2DRegion reconstructionPlane, + Point blockOrigin, + Span retainedCoefficients, + Span retainedTransformBlocks) + { + const int BlockWidth = 8; + const int SampleCount = BlockWidth * BlockWidth; + candidateCoefficients[..SampleCount].CopyTo(retainedCoefficients); + candidateTransformBlocks[..Av1EncoderModeDecisionWorkspace.CandidateTransformBlockCount] + .CopyTo(retainedTransformBlocks); + + for (int row = 0; row < BlockWidth; row++) + { + candidateReconstruction.Slice(row * BlockWidth, BlockWidth) + .CopyTo( + reconstructionPlane + .DangerousGetRowSpan(blockOrigin.Y + row) + .Slice(blockOrigin.X, BlockWidth)); + } + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1IntraSuperblockEncoder.Operator.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1IntraSuperblockEncoder.Operator.cs new file mode 100644 index 0000000000..7bf726c86a --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1IntraSuperblockEncoder.Operator.cs @@ -0,0 +1,1492 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.InteropServices; +using System.Runtime.Intrinsics; +using SixLabors.ImageSharp.Formats.Heif.Av1.Motion; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.ChromaFromLuma; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.IntraBlockCopy; +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; +using SixLabors.ImageSharp.Memory; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline; + +/// +/// Defines the sample-storage operations used by fixed intra superblock traversal. +/// +internal static partial class Av1IntraSuperblockEncoder +{ + /// + /// The width and height of the fixed block currently used by inter motion search. + /// + private const int InterSearchBlockDimension = 8; + + /// + /// Defines type-specific block encoding without coupling traversal to sample storage width. + /// + /// The native unsigned sample storage type. + internal interface IBlockEncodingOperator : Av1IntraBlockCopySearchIndex.ISearchOperation + where TSample : unmanaged + { + /// + /// Gets temporary contiguous storage for left reference samples. + /// + /// The reusable residual workspace. + /// The number of reference samples. + /// The writable reference span. + public static abstract Span GetLeftReference(Span residual, int length); + + /// + /// Converts a valid sample value to the native plane storage type. + /// + /// The sample value. + /// The converted sample. + public static abstract TSample CreateSample(int value); + + /// + /// Copies active palette-search samples into contiguous signed storage. + /// + /// The coded source plane. + /// The block origin in plane samples. + /// The active row count. + /// The active column count. + /// The contiguous sample destination. + public static abstract void CopyPaletteSamples( + Buffer2DRegion source, + Point blockOrigin, + int rows, + int columns, + Span samples); + + /// + /// Builds palette prediction and the matching source residual for transform search. + /// + /// The coded source plane. + /// The block origin in plane samples. + /// The palette colors in index order. + /// The complete padded color-index map. + /// The contiguous prediction destination. + /// The contiguous source-minus-prediction destination. + /// The prediction dimensions. + public static abstract void PreparePalette( + Buffer2DRegion source, + Point blockOrigin, + ReadOnlySpan paletteColors, + Buffer2DRegion colorIndexMap, + Span prediction, + Span residual, + Av1TransformSize transformSize); + + /// + /// Builds the zero-mean Q3 luma surface shared by chroma-from-luma candidates. + /// + /// The coded reconstructed luma plane. + /// The luma block origin in plane samples. + /// The fixed-stride Q3 predictor workspace. + /// The chroma transform dimensions. + /// Whether luma is subsampled horizontally for chroma. + /// Whether luma is subsampled vertically for chroma. + public static abstract void PrepareChromaFromLuma( + Buffer2DRegion reconstruction, + Point blockOrigin, + Span lumaQ3, + Av1TransformSize transformSize, + bool subsamplingX, + bool subsamplingY); + + /// + /// Computes the DC predictor shared by every chroma-from-luma alpha candidate. + /// + /// The contiguous candidate reconstruction. + /// The top reference samples. + /// The left reference samples. + /// Whether the left reference is available. + /// Whether the top reference is available. + /// The chroma transform dimensions. + /// The coded sample bit depth. + public static abstract void PrepareChromaFromLumaDc( + Span reconstruction, + ReadOnlySpan above, + ReadOnlySpan left, + bool hasLeft, + bool hasAbove, + Av1TransformSize transformSize, + Av1BitDepth bitDepth); + + /// + /// Encodes and reconstructs one DC intra transform block. + /// + /// The reusable block workspace. + /// The coded source plane. + /// The coded reconstruction plane. + /// The transform-block origin in plane samples. + /// The top reference samples. + /// The left reference samples. + /// Whether the left reference is available. + /// Whether the top reference is available. + /// The retained entropy-coding coefficients. + /// The transform dimensions. + /// The effective segment quantizer index. + /// The plane DC quantizer adjustment. + /// The plane AC quantizer adjustment. + /// The component plane containing the block. + /// The coded sample bit depth. + /// The retained transform state. + public static abstract void Encode( + Av1EncoderBlockWorkspace workspace, + Buffer2DRegion source, + Buffer2DRegion reconstruction, + Point blockOrigin, + ReadOnlySpan above, + ReadOnlySpan left, + bool hasLeft, + bool hasAbove, + Span quantizedCoefficients, + Av1TransformSize transformSize, + int qIndex, + int dcDeltaQ, + int acDeltaQ, + Av1Plane plane, + Av1BitDepth bitDepth, + ref Av1EncoderTransformBlockState state); + + /// + /// Encodes one intra candidate into contiguous decision scratch. + /// + /// The reusable block workspace. + /// The coded source plane. + /// The transform-block origin in plane samples. + /// The contiguous candidate reconstruction. + /// The top reference samples, with prefix storage for the shared corner. + /// The left reference samples. + /// Whether the left reference is available. + /// Whether the top reference is available. + /// The intra prediction mode. + /// The signed directional-angle adjustment. + /// Whether sequence syntax enables directional edge filtering. + /// Whether a relevant neighboring block uses smooth prediction. + /// The candidate entropy-coding coefficients. + /// The transform dimensions. + /// The compound transform applied to the residual. + /// The component plane containing the block. + /// The effective segment quantizer index. + /// The plane DC quantizer adjustment. + /// The plane AC quantizer adjustment. + /// The coded sample bit depth. + /// The candidate transform state. + /// The normalized pixel-domain distortion in AV1 transform units. + public static abstract long EncodeCandidate( + Av1EncoderBlockWorkspace workspace, + Buffer2DRegion source, + Point blockOrigin, + Span reconstruction, + ReadOnlySpan above, + ReadOnlySpan left, + bool hasLeft, + bool hasAbove, + Av1PredictionMode mode, + int angleDelta, + bool enableIntraEdgeFilter, + bool smoothIntraEdges, + Span quantizedCoefficients, + Av1TransformSize transformSize, + Av1TransformType transformType, + Av1Plane plane, + int qIndex, + int dcDeltaQ, + int acDeltaQ, + Av1BitDepth bitDepth, + ref Av1EncoderTransformBlockState state); + + /// + /// Builds one spatial intra prediction and its source residual for reuse across transform candidates. + /// + /// The reusable block workspace. + /// The coded source plane. + /// The transform-block origin in plane samples. + /// The contiguous prediction destination. + /// The top reference samples, with prefix storage for the shared corner. + /// The left reference samples. + /// Whether the left reference is available. + /// Whether the top reference is available. + /// The intra prediction mode. + /// The signed directional-angle adjustment. + /// Whether sequence syntax enables directional edge filtering. + /// Whether a relevant neighboring block uses smooth prediction. + /// The contiguous source-minus-prediction destination. + /// The prediction dimensions. + /// The coded sample bit depth. + public static abstract void PrepareIntra( + Av1EncoderBlockWorkspace workspace, + Buffer2DRegion source, + Point blockOrigin, + Span prediction, + ReadOnlySpan above, + ReadOnlySpan left, + bool hasLeft, + bool hasAbove, + Av1PredictionMode mode, + int angleDelta, + bool enableIntraEdgeFilter, + bool smoothIntraEdges, + Span residual, + Av1TransformSize transformSize, + Av1BitDepth bitDepth); + + /// + /// Builds one filter-intra prediction for reuse across transform candidates. + /// + /// The reusable block workspace. + /// The coded source plane. + /// The transform-block origin in plane samples. + /// The contiguous prediction destination. + /// The top reference samples, with prefix storage for the shared corner. + /// The left reference samples. + /// The contiguous source-minus-prediction destination. + /// The selected filter-intra mode. + /// The prediction dimensions. + /// The coded sample bit depth. + public static abstract void PrepareFilterIntra( + Av1EncoderBlockWorkspace workspace, + Buffer2DRegion source, + Point blockOrigin, + Span prediction, + ReadOnlySpan above, + ReadOnlySpan left, + Span residual, + Av1FilterIntraMode filterIntraMode, + Av1TransformSize transformSize, + Av1BitDepth bitDepth); + + /// + /// Builds an intra-block-copy prediction and the matching source residual. + /// + /// The coded source plane. + /// The destination block origin in plane samples. + /// The reconstructed plane containing the reference samples. + /// The integer reference origin preceding any half-sample phase. + /// Indicates whether the horizontal source phase is one half-sample. + /// Indicates whether the vertical source phase is one half-sample. + /// The contiguous prediction destination. + /// The contiguous source-minus-prediction destination. + /// The prediction dimensions. + public static abstract void PrepareIntraBlockCopyPrediction( + Buffer2DRegion source, + Point blockOrigin, + Buffer2DRegion reconstruction, + Point predictionOrigin, + bool halfX, + bool halfY, + Span prediction, + Span residual, + Av1TransformSize transformSize); + + /// + /// Subtracts a retained prediction from its source without rebuilding the inter predictor. + /// + /// The source plane. + /// The block origin in plane samples. + /// The tightly packed prediction samples. + /// The destination signed residual samples. + /// The plane block geometry. + public static abstract void SubtractPrediction( + Buffer2DRegion source, + Point blockOrigin, + ReadOnlySpan prediction, + Span residual, + Av1TransformSize transformSize); + + /// + /// Builds a translational prediction from a retained reference frame and the matching source residual. + /// + /// The coded source plane. + /// The destination block origin in plane samples. + /// The padded retained reference plane. + /// The integer reference origin preceding the subpixel phase. + /// The horizontal interpolation filter. + /// The vertical interpolation filter. + /// The horizontal phase in one-sixteenth-sample units. + /// The vertical phase in one-sixteenth-sample units. + /// The contiguous prediction destination. + /// The contiguous source-minus-prediction destination. + /// The intermediate storage used by two-dimensional filtering. + /// The prediction dimensions. + /// The coded sample bit depth. + public static abstract void PrepareTranslationalInterPrediction( + Buffer2DRegion source, + Point blockOrigin, + Buffer2DRegion reference, + Point predictionOrigin, + Av1InterpolationFilter horizontalFilter, + Av1InterpolationFilter verticalFilter, + int horizontalPhase, + int verticalPhase, + Span prediction, + Span residual, + Span predictionScratch, + Av1TransformSize transformSize, + Av1BitDepth bitDepth); + + /// + /// Measures an 8x8 full-pixel reference candidate through the bordered plane storage. + /// + /// The coded source plane. + /// The source block origin in visible-plane coordinates. + /// The padded retained reference plane. + /// The candidate origin, which may lie inside the physical border. + /// The coded sample precision. + /// The squared error normalized to the eight-bit distortion domain. + public static abstract long GetInterPredictionError( + Buffer2DRegion source, + Point sourceOrigin, + Buffer2DRegion reference, + Point predictionOrigin, + Av1BitDepth bitDepth); + + /// + /// Encodes one prepared prediction with the selected transform into decision scratch. + /// + /// The reusable block workspace. + /// The coded source plane. + /// The transform-block origin in plane samples. + /// The contiguous prediction samples. + /// The contiguous source-minus-prediction samples. + /// The candidate reconstruction. + /// The number of reconstruction samples between rows. + /// The candidate entropy-coding coefficients. + /// The transform dimensions. + /// The compound transform applied to the residual. + /// The component plane containing the block. + /// The effective segment quantizer index. + /// The plane DC quantizer adjustment. + /// The plane AC quantizer adjustment. + /// The coded sample bit depth. + /// The candidate transform state. + /// The normalized pixel-domain distortion in AV1 transform units. + public static abstract long EncodePredictionCandidate( + Av1EncoderBlockWorkspace workspace, + Buffer2DRegion source, + Point blockOrigin, + ReadOnlySpan prediction, + ReadOnlySpan residual, + Span reconstruction, + int reconstructionStride, + Span quantizedCoefficients, + Av1TransformSize transformSize, + Av1TransformType transformType, + Av1Plane plane, + int qIndex, + int dcDeltaQ, + int acDeltaQ, + Av1BitDepth bitDepth, + ref Av1EncoderTransformBlockState state); + + /// + /// Encodes one chroma-from-luma candidate into contiguous decision scratch. + /// + /// The reusable block workspace. + /// The coded source plane. + /// The transform-block origin in plane samples. + /// The contiguous candidate reconstruction. + /// The cached DC predictor sample shared by every alpha. + /// The zero-mean reconstructed-luma predictor surface. + /// The signed chroma-from-luma multiplier. + /// The candidate entropy-coding coefficients. + /// The transform dimensions. + /// The component plane containing the block. + /// The effective segment quantizer index. + /// The plane DC quantizer adjustment. + /// The plane AC quantizer adjustment. + /// The coded sample bit depth. + /// The candidate transform state. + /// The normalized pixel-domain distortion in AV1 transform units. + public static abstract long EncodeChromaFromLumaCandidate( + Av1EncoderBlockWorkspace workspace, + Buffer2DRegion source, + Point blockOrigin, + Span reconstruction, + TSample dc, + ReadOnlySpan lumaQ3, + int alphaQ3, + Span quantizedCoefficients, + Av1TransformSize transformSize, + Av1Plane plane, + int qIndex, + int dcDeltaQ, + int acDeltaQ, + Av1BitDepth bitDepth, + ref Av1EncoderTransformBlockState state); + } + + private static int GetNormalizedVariance(int sum, int sumOfSquares, Av1BitDepth bitDepth) + { + int coefficientShift = bitDepth.GetBitCount() - 8; + if (coefficientShift > 0) + { + // Normalize both moments before subtracting them so high-bit-depth motion search uses the + // same eight-bit distortion scale as the encoder's other rate-distortion comparisons. + int squareShift = coefficientShift * 2; + sumOfSquares = (sumOfSquares + (1 << (squareShift - 1))) >> squareShift; + sum = (sum + (1 << (coefficientShift - 1))) >> coefficientShift; + } + + long variance = sumOfSquares - (((long)sum * sum) / 64); + return (int)Math.Max(variance, 0); + } + + /// + /// Encodes blocks stored as eight-bit samples. + /// + internal readonly struct ByteOperator : IBlockEncodingOperator + { + /// + public static Span GetLeftReference(Span residual, int length) + => MemoryMarshal.AsBytes(residual)[..length]; + + /// + public static byte CreateSample(int value) => (byte)value; + + /// + public static uint GetHashSample(byte sample) => sample; + + /// + public static bool BlocksEqual(Buffer2DRegion plane, Point first, Point second) + { + for (int row = 0; row < 8; row++) + { + ReadOnlySpan firstRow = plane.DangerousGetRowSpan(first.Y + row)[first.X..]; + ReadOnlySpan secondRow = plane.DangerousGetRowSpan(second.Y + row)[second.X..]; + + // Compare the complete row as byte lanes so collision rejection remains independent of native endianness. + if (Vector64.Create(firstRow) != Vector64.Create(secondRow)) + { + return false; + } + } + + return true; + } + + /// + public static long GetInterPredictionError( + Buffer2DRegion source, + Point sourceOrigin, + Buffer2DRegion reference, + Point predictionOrigin, + Av1BitDepth bitDepth) + { + Rectangle sourceBounds = source.Bounds; + Rectangle referenceBounds = reference.Bounds; + int sourceIndex = + ((sourceBounds.Y + sourceOrigin.Y) * source.Stride) + + sourceBounds.X + + sourceOrigin.X; + + int referenceIndex = + ((referenceBounds.Y + predictionOrigin.Y) * reference.Stride) + + referenceBounds.X + + predictionOrigin.X; + + // The shared residual kernel selects the widest available vector width and handles the scalar tail. + return Av1ResidualBuilder.SumSquaredError( + source.Buffer.DangerousGetSingleSpan()[sourceIndex..], + source.Stride, + reference.Buffer.DangerousGetSingleSpan()[referenceIndex..], + reference.Stride, + InterSearchBlockDimension, + InterSearchBlockDimension); + } + + /// + public static int GetSumOfAbsoluteDifferences( + Buffer2DRegion source, + Point sourceOrigin, + Buffer2DRegion reconstruction, + Point predictionOrigin) + => Av1ResidualBuilder.SumAbsoluteDifferences8x8( + Av1TransformBlockEncoder.GetPlaneSpan(source, sourceOrigin), + source.Stride, + Av1TransformBlockEncoder.GetPlaneSpan(reconstruction, predictionOrigin), + reconstruction.Stride); + + /// + public static void GetFourSumsOfAbsoluteDifferences( + Buffer2DRegion source, + Point sourceOrigin, + Buffer2DRegion reconstruction, + Point firstPredictionOrigin, + Span sums) + => Av1ResidualBuilder.SumFourAbsoluteDifferences8x8( + Av1TransformBlockEncoder.GetPlaneSpan(source, sourceOrigin), + source.Stride, + Av1TransformBlockEncoder.GetPlaneSpan(reconstruction, firstPredictionOrigin), + reconstruction.Stride, + sums); + + /// + public static int GetVariance( + Buffer2DRegion source, + Point sourceOrigin, + Buffer2DRegion reconstruction, + Point predictionOrigin, + Av1BitDepth bitDepth) + { + Av1ResidualBuilder.GetMoments8x8( + Av1TransformBlockEncoder.GetPlaneSpan(source, sourceOrigin), + source.Stride, + Av1TransformBlockEncoder.GetPlaneSpan(reconstruction, predictionOrigin), + reconstruction.Stride, + out int sum, + out int sumOfSquares); + + return GetNormalizedVariance(sum, sumOfSquares, bitDepth); + } + + /// + public static void CopyPaletteSamples( + Buffer2DRegion source, + Point blockOrigin, + int rows, + int columns, + Span samples) + { + int sampleOffset = 0; + for (int row = 0; row < rows; row++) + { + ReadOnlySpan sourceRow = source + .DangerousGetRowSpan(blockOrigin.Y + row) + .Slice(blockOrigin.X, columns); + + // A complete row widens in one vector; clipped edge rows retain scalar bounds. + if (columns == 8 && Vector128.IsHardwareAccelerated) + { + Vector128.WidenLower(Vector128.Create(Vector64.Create(sourceRow), Vector64.Zero)).AsInt16().CopyTo(samples[sampleOffset..]); + + sampleOffset += columns; + continue; + } + + for (int column = 0; column < sourceRow.Length; column++) + { + samples[sampleOffset++] = sourceRow[column]; + } + } + } + + /// + public static void PreparePalette( + Buffer2DRegion source, + Point blockOrigin, + ReadOnlySpan paletteColors, + Buffer2DRegion colorIndexMap, + Span prediction, + Span residual, + Av1TransformSize transformSize) + { + int width = transformSize.GetWidth(); + int height = transformSize.GetHeight(); + Av1PalettePredictor.Predict( + paletteColors, + colorIndexMap, + prediction, + width, + width, + height); + + Av1ResidualBuilder.Subtract( + Av1TransformBlockEncoder.GetPlaneSpan(source, blockOrigin), + source.Stride, + prediction, + width, + residual, + width, + width, + height); + } + + /// + public static void PrepareChromaFromLuma( + Buffer2DRegion reconstruction, + Point blockOrigin, + Span lumaQ3, + Av1TransformSize transformSize, + bool subsamplingX, + bool subsamplingY) + => Av1ChromaFromLumaContext.PrepareBlock( + Av1TransformBlockEncoder.GetPlaneSpan(reconstruction, blockOrigin), + reconstruction.Stride, + lumaQ3, + transformSize, + subsamplingX, + subsamplingY); + + /// + public static void PrepareChromaFromLumaDc( + Span reconstruction, + ReadOnlySpan above, + ReadOnlySpan left, + bool hasLeft, + bool hasAbove, + Av1TransformSize transformSize, + Av1BitDepth bitDepth) + { + int width = transformSize.GetWidth(); + Av1DcIntraPredictor.Predict( + hasLeft, + hasAbove, + reconstruction, + width, + above, + left, + width, + transformSize.GetHeight()); + } + + /// + public static void Encode( + Av1EncoderBlockWorkspace workspace, + Buffer2DRegion source, + Buffer2DRegion reconstruction, + Point blockOrigin, + ReadOnlySpan above, + ReadOnlySpan left, + bool hasLeft, + bool hasAbove, + Span quantizedCoefficients, + Av1TransformSize transformSize, + int qIndex, + int dcDeltaQ, + int acDeltaQ, + Av1Plane plane, + Av1BitDepth bitDepth, + ref Av1EncoderTransformBlockState state) + => Av1TransformBlockEncoder.EncodeIntraDcLossy( + workspace, + source, + reconstruction, + blockOrigin, + above, + left, + hasLeft, + hasAbove, + quantizedCoefficients, + transformSize, + Av1TransformType.DctDct, + qIndex, + dcDeltaQ, + acDeltaQ, + plane, + ref state); + + /// + public static long EncodeCandidate( + Av1EncoderBlockWorkspace workspace, + Buffer2DRegion source, + Point blockOrigin, + Span reconstruction, + ReadOnlySpan above, + ReadOnlySpan left, + bool hasLeft, + bool hasAbove, + Av1PredictionMode mode, + int angleDelta, + bool enableIntraEdgeFilter, + bool smoothIntraEdges, + Span quantizedCoefficients, + Av1TransformSize transformSize, + Av1TransformType transformType, + Av1Plane plane, + int qIndex, + int dcDeltaQ, + int acDeltaQ, + Av1BitDepth bitDepth, + ref Av1EncoderTransformBlockState state) + => Av1TransformBlockEncoder.EncodeIntraLossyCandidate( + workspace, + source, + blockOrigin, + reconstruction, + above, + left, + hasLeft, + hasAbove, + mode, + angleDelta, + enableIntraEdgeFilter, + smoothIntraEdges, + quantizedCoefficients, + transformSize, + transformType, + qIndex, + dcDeltaQ, + acDeltaQ, + plane, + ref state); + + /// + public static void PrepareIntra( + Av1EncoderBlockWorkspace workspace, + Buffer2DRegion source, + Point blockOrigin, + Span prediction, + ReadOnlySpan above, + ReadOnlySpan left, + bool hasLeft, + bool hasAbove, + Av1PredictionMode mode, + int angleDelta, + bool enableIntraEdgeFilter, + bool smoothIntraEdges, + Span residual, + Av1TransformSize transformSize, + Av1BitDepth bitDepth) + => Av1TransformBlockEncoder.PrepareIntraPrediction( + workspace, + Av1TransformBlockEncoder.GetPlaneSpan(source, blockOrigin), + source.Stride, + prediction, + transformSize.GetWidth(), + above, + left, + hasLeft, + hasAbove, + mode, + angleDelta, + enableIntraEdgeFilter, + smoothIntraEdges, + residual, + transformSize); + + /// + public static void PrepareFilterIntra( + Av1EncoderBlockWorkspace workspace, + Buffer2DRegion source, + Point blockOrigin, + Span prediction, + ReadOnlySpan above, + ReadOnlySpan left, + Span residual, + Av1FilterIntraMode filterIntraMode, + Av1TransformSize transformSize, + Av1BitDepth bitDepth) + { + int width = transformSize.GetWidth(); + int height = transformSize.GetHeight(); + + // Prediction finishes before transform search, so its temporary rows can borrow the transform workspace. + Span filterScratch = MemoryMarshal.AsBytes(workspace.TransformWorkspace).Slice( + 0, + Av1FilterIntraPredictorBase.ScratchLength); + + Av1FilterIntraPredictorBase.GetPredictor(filterIntraMode) + .Predict(prediction, width, above, left, width, height, filterScratch); + + Av1ResidualBuilder.Subtract( + Av1TransformBlockEncoder.GetPlaneSpan(source, blockOrigin), + source.Stride, + prediction, + width, + residual, + width, + width, + height); + } + + /// + public static void PrepareIntraBlockCopyPrediction( + Buffer2DRegion source, + Point blockOrigin, + Buffer2DRegion reconstruction, + Point predictionOrigin, + bool halfX, + bool halfY, + Span prediction, + Span residual, + Av1TransformSize transformSize) + { + int width = transformSize.GetWidth(); + int height = transformSize.GetHeight(); + Av1IntraBlockCopyPredictor.Predict( + Av1TransformBlockEncoder.GetPlaneSpan(reconstruction, predictionOrigin), + reconstruction.Stride, + prediction, + width, + width, + height, + halfX, + halfY); + + Av1ResidualBuilder.Subtract( + Av1TransformBlockEncoder.GetPlaneSpan(source, blockOrigin), + source.Stride, + prediction, + width, + residual, + width, + width, + height); + } + + /// + public static void SubtractPrediction( + Buffer2DRegion source, + Point blockOrigin, + ReadOnlySpan prediction, + Span residual, + Av1TransformSize transformSize) + => Av1ResidualBuilder.Subtract( + Av1TransformBlockEncoder.GetPlaneSpan(source, blockOrigin), + source.Stride, + prediction, + transformSize.GetWidth(), + residual, + transformSize.GetWidth(), + transformSize.GetWidth(), + transformSize.GetHeight()); + + /// + public static void PrepareTranslationalInterPrediction( + Buffer2DRegion source, + Point blockOrigin, + Buffer2DRegion reference, + Point predictionOrigin, + Av1InterpolationFilter horizontalFilter, + Av1InterpolationFilter verticalFilter, + int horizontalPhase, + int verticalPhase, + Span prediction, + Span residual, + Span predictionScratch, + Av1TransformSize transformSize, + Av1BitDepth bitDepth) + { + int width = transformSize.GetWidth(); + int height = transformSize.GetHeight(); + Rectangle referenceBounds = reference.Bounds; + int referenceOrigin = + ((referenceBounds.Y + predictionOrigin.Y) * reference.Stride) + + referenceBounds.X + + predictionOrigin.X; + + Av1TranslationalInterPredictor.Predict( + reference.Buffer.DangerousGetSingleSpan(), + reference.Stride, + referenceOrigin, + prediction, + width, + width, + height, + horizontalFilter, + verticalFilter, + horizontalPhase, + verticalPhase, + predictionScratch); + + SubtractPrediction(source, blockOrigin, prediction, residual, transformSize); + } + + /// + public static long EncodePredictionCandidate( + Av1EncoderBlockWorkspace workspace, + Buffer2DRegion source, + Point blockOrigin, + ReadOnlySpan prediction, + ReadOnlySpan residual, + Span reconstruction, + int reconstructionStride, + Span quantizedCoefficients, + Av1TransformSize transformSize, + Av1TransformType transformType, + Av1Plane plane, + int qIndex, + int dcDeltaQ, + int acDeltaQ, + Av1BitDepth bitDepth, + ref Av1EncoderTransformBlockState state) + => Av1TransformBlockEncoder.EncodePredictionLossyCandidate( + workspace, + source, + blockOrigin, + prediction, + residual, + reconstruction, + reconstructionStride, + quantizedCoefficients, + transformSize, + transformType, + qIndex, + dcDeltaQ, + acDeltaQ, + plane, + ref state); + + /// + public static long EncodeChromaFromLumaCandidate( + Av1EncoderBlockWorkspace workspace, + Buffer2DRegion source, + Point blockOrigin, + Span reconstruction, + byte dc, + ReadOnlySpan lumaQ3, + int alphaQ3, + Span quantizedCoefficients, + Av1TransformSize transformSize, + Av1Plane plane, + int qIndex, + int dcDeltaQ, + int acDeltaQ, + Av1BitDepth bitDepth, + ref Av1EncoderTransformBlockState state) + => Av1TransformBlockEncoder.EncodeChromaFromLumaLossyCandidate( + workspace, + source, + blockOrigin, + reconstruction, + dc, + lumaQ3, + alphaQ3, + quantizedCoefficients, + transformSize, + qIndex, + dcDeltaQ, + acDeltaQ, + plane, + ref state); + } + + /// + /// Encodes blocks stored as high-bit-depth samples. + /// + internal readonly struct UInt16Operator : IBlockEncodingOperator + { + /// + public static Span GetLeftReference(Span residual, int length) + => MemoryMarshal.Cast(residual)[..length]; + + /// + public static ushort CreateSample(int value) => (ushort)value; + + /// + public static uint GetHashSample(ushort sample) => sample; + + /// + public static bool BlocksEqual(Buffer2DRegion plane, Point first, Point second) + { + for (int row = 0; row < 8; row++) + { + ReadOnlySpan firstRow = plane.DangerousGetRowSpan(first.Y + row)[first.X..]; + ReadOnlySpan secondRow = plane.DangerousGetRowSpan(second.Y + row)[second.X..]; + if (Vector128.IsHardwareAccelerated) + { + Vector128 firstSamples = Vector128.LoadUnsafe(ref MemoryMarshal.GetReference(firstRow)); + Vector128 secondSamples = Vector128.LoadUnsafe(ref MemoryMarshal.GetReference(secondRow)); + if (!Vector128.EqualsAll(firstSamples, secondSamples)) + { + return false; + } + } + else if (!firstRow[..8].SequenceEqual(secondRow[..8])) + { + return false; + } + } + + return true; + } + + /// + public static long GetInterPredictionError( + Buffer2DRegion source, + Point sourceOrigin, + Buffer2DRegion reference, + Point predictionOrigin, + Av1BitDepth bitDepth) + { + Rectangle sourceBounds = source.Bounds; + Rectangle referenceBounds = reference.Bounds; + int sourceIndex = + ((sourceBounds.Y + sourceOrigin.Y) * source.Stride) + + sourceBounds.X + + sourceOrigin.X; + + int referenceIndex = + ((referenceBounds.Y + predictionOrigin.Y) * reference.Stride) + + referenceBounds.X + + predictionOrigin.X; + + long error = Av1ResidualBuilder.SumSquaredError( + source.Buffer.DangerousGetSingleSpan()[sourceIndex..], + source.Stride, + reference.Buffer.DangerousGetSingleSpan()[referenceIndex..], + reference.Stride, + InterSearchBlockDimension, + InterSearchBlockDimension); + + int shift = (bitDepth.GetBitCount() - 8) * 2; + return shift == 0 ? error : (error + (1L << (shift - 1))) >> shift; + } + + /// + public static int GetSumOfAbsoluteDifferences( + Buffer2DRegion source, + Point sourceOrigin, + Buffer2DRegion reconstruction, + Point predictionOrigin) + => Av1ResidualBuilder.SumAbsoluteDifferences8x8( + Av1TransformBlockEncoder.GetPlaneSpan(source, sourceOrigin), + source.Stride, + Av1TransformBlockEncoder.GetPlaneSpan(reconstruction, predictionOrigin), + reconstruction.Stride); + + /// + public static void GetFourSumsOfAbsoluteDifferences( + Buffer2DRegion source, + Point sourceOrigin, + Buffer2DRegion reconstruction, + Point firstPredictionOrigin, + Span sums) + => Av1ResidualBuilder.SumFourAbsoluteDifferences8x8( + Av1TransformBlockEncoder.GetPlaneSpan(source, sourceOrigin), + source.Stride, + Av1TransformBlockEncoder.GetPlaneSpan(reconstruction, firstPredictionOrigin), + reconstruction.Stride, + sums); + + /// + public static int GetVariance( + Buffer2DRegion source, + Point sourceOrigin, + Buffer2DRegion reconstruction, + Point predictionOrigin, + Av1BitDepth bitDepth) + { + Av1ResidualBuilder.GetMoments8x8( + Av1TransformBlockEncoder.GetPlaneSpan(source, sourceOrigin), + source.Stride, + Av1TransformBlockEncoder.GetPlaneSpan(reconstruction, predictionOrigin), + reconstruction.Stride, + out int sum, + out int sumOfSquares); + + return GetNormalizedVariance(sum, sumOfSquares, bitDepth); + } + + /// + public static void CopyPaletteSamples( + Buffer2DRegion source, + Point blockOrigin, + int rows, + int columns, + Span samples) + { + int sampleOffset = 0; + for (int row = 0; row < rows; row++) + { + ReadOnlySpan sourceRow = source + .DangerousGetRowSpan(blockOrigin.Y + row) + .Slice(blockOrigin.X, columns); + + MemoryMarshal.Cast(sourceRow).CopyTo(samples[sampleOffset..]); + sampleOffset += sourceRow.Length; + } + } + + /// + public static void PreparePalette( + Buffer2DRegion source, + Point blockOrigin, + ReadOnlySpan paletteColors, + Buffer2DRegion colorIndexMap, + Span prediction, + Span residual, + Av1TransformSize transformSize) + { + int width = transformSize.GetWidth(); + int height = transformSize.GetHeight(); + Av1PalettePredictor.Predict( + paletteColors, + colorIndexMap, + MemoryMarshal.Cast(prediction), + width, + width, + height); + + Av1ResidualBuilder.Subtract( + Av1TransformBlockEncoder.GetPlaneSpan(source, blockOrigin), + source.Stride, + prediction, + width, + residual, + width, + width, + height); + } + + /// + public static void PrepareChromaFromLuma( + Buffer2DRegion reconstruction, + Point blockOrigin, + Span lumaQ3, + Av1TransformSize transformSize, + bool subsamplingX, + bool subsamplingY) + => Av1ChromaFromLumaContext.PrepareBlock( + MemoryMarshal.Cast(Av1TransformBlockEncoder.GetPlaneSpan(reconstruction, blockOrigin)), + reconstruction.Stride, + lumaQ3, + transformSize, + subsamplingX, + subsamplingY); + + /// + public static void PrepareChromaFromLumaDc( + Span reconstruction, + ReadOnlySpan above, + ReadOnlySpan left, + bool hasLeft, + bool hasAbove, + Av1TransformSize transformSize, + Av1BitDepth bitDepth) + { + int width = transformSize.GetWidth(); + Av1DcIntraPredictor.Predict( + hasLeft, + hasAbove, + MemoryMarshal.Cast(reconstruction), + width, + MemoryMarshal.Cast(above), + MemoryMarshal.Cast(left), + width, + transformSize.GetHeight(), + bitDepth.GetBitCount()); + } + + /// + public static void Encode( + Av1EncoderBlockWorkspace workspace, + Buffer2DRegion source, + Buffer2DRegion reconstruction, + Point blockOrigin, + ReadOnlySpan above, + ReadOnlySpan left, + bool hasLeft, + bool hasAbove, + Span quantizedCoefficients, + Av1TransformSize transformSize, + int qIndex, + int dcDeltaQ, + int acDeltaQ, + Av1Plane plane, + Av1BitDepth bitDepth, + ref Av1EncoderTransformBlockState state) + => Av1TransformBlockEncoder.EncodeIntraDcLossy( + workspace, + source, + reconstruction, + blockOrigin, + above, + left, + hasLeft, + hasAbove, + quantizedCoefficients, + transformSize, + Av1TransformType.DctDct, + qIndex, + dcDeltaQ, + acDeltaQ, + plane, + bitDepth, + ref state); + + /// + public static long EncodeCandidate( + Av1EncoderBlockWorkspace workspace, + Buffer2DRegion source, + Point blockOrigin, + Span reconstruction, + ReadOnlySpan above, + ReadOnlySpan left, + bool hasLeft, + bool hasAbove, + Av1PredictionMode mode, + int angleDelta, + bool enableIntraEdgeFilter, + bool smoothIntraEdges, + Span quantizedCoefficients, + Av1TransformSize transformSize, + Av1TransformType transformType, + Av1Plane plane, + int qIndex, + int dcDeltaQ, + int acDeltaQ, + Av1BitDepth bitDepth, + ref Av1EncoderTransformBlockState state) + => Av1TransformBlockEncoder.EncodeIntraLossyCandidate( + workspace, + source, + blockOrigin, + reconstruction, + above, + left, + hasLeft, + hasAbove, + mode, + angleDelta, + enableIntraEdgeFilter, + smoothIntraEdges, + quantizedCoefficients, + transformSize, + transformType, + qIndex, + dcDeltaQ, + acDeltaQ, + plane, + bitDepth, + ref state); + + /// + public static void PrepareIntra( + Av1EncoderBlockWorkspace workspace, + Buffer2DRegion source, + Point blockOrigin, + Span prediction, + ReadOnlySpan above, + ReadOnlySpan left, + bool hasLeft, + bool hasAbove, + Av1PredictionMode mode, + int angleDelta, + bool enableIntraEdgeFilter, + bool smoothIntraEdges, + Span residual, + Av1TransformSize transformSize, + Av1BitDepth bitDepth) + => Av1TransformBlockEncoder.PrepareIntraPrediction( + workspace, + Av1TransformBlockEncoder.GetPlaneSpan(source, blockOrigin), + source.Stride, + prediction, + transformSize.GetWidth(), + above, + left, + hasLeft, + hasAbove, + mode, + angleDelta, + enableIntraEdgeFilter, + smoothIntraEdges, + residual, + transformSize, + bitDepth); + + /// + public static void PrepareFilterIntra( + Av1EncoderBlockWorkspace workspace, + Buffer2DRegion source, + Point blockOrigin, + Span prediction, + ReadOnlySpan above, + ReadOnlySpan left, + Span residual, + Av1FilterIntraMode filterIntraMode, + Av1TransformSize transformSize, + Av1BitDepth bitDepth) + { + int width = transformSize.GetWidth(); + int height = transformSize.GetHeight(); + + // Prediction finishes before transform search, so its temporary rows can borrow the transform workspace. + Span filterScratch = MemoryMarshal.Cast(workspace.TransformWorkspace).Slice( + 0, + Av1FilterIntraPredictorBase.ScratchLength); + + Av1FilterIntraPredictorBase.GetPredictor(filterIntraMode) + .Predict( + MemoryMarshal.Cast(prediction), + width, + MemoryMarshal.Cast(above), + MemoryMarshal.Cast(left), + width, + height, + bitDepth.GetBitCount(), + filterScratch); + + Av1ResidualBuilder.Subtract( + Av1TransformBlockEncoder.GetPlaneSpan(source, blockOrigin), + source.Stride, + prediction, + width, + residual, + width, + width, + height); + } + + /// + public static void PrepareIntraBlockCopyPrediction( + Buffer2DRegion source, + Point blockOrigin, + Buffer2DRegion reconstruction, + Point predictionOrigin, + bool halfX, + bool halfY, + Span prediction, + Span residual, + Av1TransformSize transformSize) + { + int width = transformSize.GetWidth(); + int height = transformSize.GetHeight(); + Av1IntraBlockCopyPredictor.Predict( + MemoryMarshal.Cast(Av1TransformBlockEncoder.GetPlaneSpan(reconstruction, predictionOrigin)), + reconstruction.Stride, + MemoryMarshal.Cast(prediction), + width, + width, + height, + halfX, + halfY); + + Av1ResidualBuilder.Subtract( + Av1TransformBlockEncoder.GetPlaneSpan(source, blockOrigin), + source.Stride, + prediction, + width, + residual, + width, + width, + height); + } + + /// + public static void SubtractPrediction( + Buffer2DRegion source, + Point blockOrigin, + ReadOnlySpan prediction, + Span residual, + Av1TransformSize transformSize) + => Av1ResidualBuilder.Subtract( + Av1TransformBlockEncoder.GetPlaneSpan(source, blockOrigin), + source.Stride, + prediction, + transformSize.GetWidth(), + residual, + transformSize.GetWidth(), + transformSize.GetWidth(), + transformSize.GetHeight()); + + /// + public static void PrepareTranslationalInterPrediction( + Buffer2DRegion source, + Point blockOrigin, + Buffer2DRegion reference, + Point predictionOrigin, + Av1InterpolationFilter horizontalFilter, + Av1InterpolationFilter verticalFilter, + int horizontalPhase, + int verticalPhase, + Span prediction, + Span residual, + Span predictionScratch, + Av1TransformSize transformSize, + Av1BitDepth bitDepth) + { + int width = transformSize.GetWidth(); + int height = transformSize.GetHeight(); + Rectangle referenceBounds = reference.Bounds; + int referenceOrigin = + ((referenceBounds.Y + predictionOrigin.Y) * reference.Stride) + + referenceBounds.X + + predictionOrigin.X; + + Av1TranslationalInterPredictor.Predict( + reference.Buffer.DangerousGetSingleSpan(), + reference.Stride, + referenceOrigin, + prediction, + width, + width, + height, + horizontalFilter, + verticalFilter, + horizontalPhase, + verticalPhase, + bitDepth.GetBitCount(), + predictionScratch); + + SubtractPrediction(source, blockOrigin, prediction, residual, transformSize); + } + + /// + public static long EncodePredictionCandidate( + Av1EncoderBlockWorkspace workspace, + Buffer2DRegion source, + Point blockOrigin, + ReadOnlySpan prediction, + ReadOnlySpan residual, + Span reconstruction, + int reconstructionStride, + Span quantizedCoefficients, + Av1TransformSize transformSize, + Av1TransformType transformType, + Av1Plane plane, + int qIndex, + int dcDeltaQ, + int acDeltaQ, + Av1BitDepth bitDepth, + ref Av1EncoderTransformBlockState state) + => Av1TransformBlockEncoder.EncodePredictionLossyCandidate( + workspace, + source, + blockOrigin, + prediction, + residual, + reconstruction, + reconstructionStride, + quantizedCoefficients, + transformSize, + transformType, + qIndex, + dcDeltaQ, + acDeltaQ, + plane, + bitDepth, + ref state); + + /// + public static long EncodeChromaFromLumaCandidate( + Av1EncoderBlockWorkspace workspace, + Buffer2DRegion source, + Point blockOrigin, + Span reconstruction, + ushort dc, + ReadOnlySpan lumaQ3, + int alphaQ3, + Span quantizedCoefficients, + Av1TransformSize transformSize, + Av1Plane plane, + int qIndex, + int dcDeltaQ, + int acDeltaQ, + Av1BitDepth bitDepth, + ref Av1EncoderTransformBlockState state) + => Av1TransformBlockEncoder.EncodeChromaFromLumaLossyCandidate( + workspace, + source, + blockOrigin, + reconstruction, + dc, + lumaQ3, + alphaQ3, + quantizedCoefficients, + transformSize, + qIndex, + dcDeltaQ, + acDeltaQ, + plane, + bitDepth, + ref state); + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1IntraSuperblockEncoder.PaletteModeDecision.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1IntraSuperblockEncoder.PaletteModeDecision.cs new file mode 100644 index 0000000000..756042f0b3 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1IntraSuperblockEncoder.PaletteModeDecision.cs @@ -0,0 +1,512 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1.Entropy; +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; +using SixLabors.ImageSharp.Memory; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline; + +/// +/// Provides luma palette mode decisions for intra encoding. +/// +internal static partial class Av1IntraSuperblockEncoder +{ + internal partial struct ModeDecision + where TSample : unmanaged + where TOperator : struct, IBlockEncodingOperator + { + private bool SelectLumaPalette( + Av1SymbolEncoder writer, + Av1MacroBlockD macroBlock, + Buffer2DRegion sourcePlane, + Buffer2DRegion reconstructionPlane, + Point blockOrigin, + ushort tileIndex, + Av1TransformSetType transformSetType, + Av1TransformBlockContext blockContext, + int transformSizeRate, + int transformSizeContext, + Span candidateReconstruction, + Span candidateCoefficients, + Span retainedCoefficients, + Span retainedStates, + ref Av1RateDistortionStatistics bestStatistics, + ref Av1EncoderPaletteInfo paletteInfo, + ref Av1TransformSize selectedTransformSize) + { + const Av1BlockSize BlockSize = Av1BlockSize.Block8x8; + const int BlockLength = 8; + Av1EncoderPaletteWorkspace workspace = + this.blockWorkspace.GetModeDecisionWorkspace().Palette; + + // Palette samples and symbols extend to the coded mode-info boundary, including replicated source + // alignment. The signed edge distances use eighth-sample units and exclude only blocks beyond that boundary. + int rows = BlockLength + (Math.Min(0, macroBlock.ToBottomEdge) >> 3); + int columns = BlockLength + (Math.Min(0, macroBlock.ToRightEdge) >> 3); + int sampleCount = rows * columns; + Span samples = workspace.GetSamples(0)[..sampleCount]; + TOperator.CopyPaletteSamples(sourcePlane, blockOrigin, rows, columns, samples); + + Span uniqueColors = workspace.GetUniqueColors(0); + Span colorCounts = workspace.LumaColorCounts; + int uniqueColorCount = 0; + short minimum = samples[0]; + short maximum = samples[0]; + + // An 8x8 block has at most 64 distinct samples, so a compact workspace histogram avoids a + // dictionary allocation while collecting both frequency seeds and range endpoints. + foreach (short sample in samples) + { + int colorIndex = uniqueColors[..uniqueColorCount].IndexOf(sample); + if (colorIndex >= 0) + { + colorCounts[colorIndex]++; + } + else + { + uniqueColors[uniqueColorCount] = sample; + colorCounts[uniqueColorCount] = 1; + uniqueColorCount++; + } + + minimum = Math.Min(minimum, sample); + maximum = Math.Max(maximum, sample); + } + + if (uniqueColorCount < 2) + { + return false; + } + + int maximumPaletteSize = Math.Min(uniqueColorCount, Av1Constants.PaletteMaxSize); + Span dominantOrder = workspace.LumaDominantOrder; + for (int index = 0; index < uniqueColorCount; index++) + { + dominantOrder[index] = (byte)index; + } + + // Count order chooses the colors that explain most samples first; sample value resolves equal counts. + for (int index = 1; index < uniqueColorCount; index++) + { + byte current = dominantOrder[index]; + int destination = index; + while (destination > 0) + { + byte preceding = dominantOrder[destination - 1]; + bool precedes = colorCounts[current] > colorCounts[preceding] || + (colorCounts[current] == colorCounts[preceding] && uniqueColors[current] < uniqueColors[preceding]); + + if (!precedes) + { + break; + } + + dominantOrder[destination] = preceding; + destination--; + } + + dominantOrder[destination] = current; + } + + Av1NeighborArrayUnit paletteContexts = this.picture.PaletteContexts[tileIndex]; + int blockSizeContext = Av1TileWriter.GetPaletteBlockSizeContext(BlockSize); + int neighborContext = Av1TileWriter.GetPaletteYModeContext(paletteContexts, macroBlock, blockOrigin); + Span colorCache = workspace.ColorCache; + int colorCacheSize = Av1TileWriter.GetPaletteCache( + paletteContexts, + macroBlock, + blockOrigin, + Av1Plane.Y, + colorCache); + + Buffer2DRegion colorIndexMap = this.superblock.Workspace + .GetPaletteMaps() + .GetMap(Av1PlaneType.Y, BlockLength, BlockLength); + + Span retainedColorIndexMap = workspace.RetainedIndices; + Span centroids = workspace.GetCentroids(0); + bool paletteSelected = false; + + // Evaluate both frequency-seeded and range-seeded palette families for every legal size. + // Exhaustive ascending size order avoids speed-dependent pruning and gives smaller palettes + // deterministic precedence when complete rate-distortion costs tie. + for (int paletteSize = 2; paletteSize <= maximumPaletteSize; paletteSize++) + { + for (int index = 0; index < paletteSize; index++) + { + centroids[index] = uniqueColors[dominantOrder[index]]; + } + + this.EvaluateLumaPaletteCandidate( + writer, + macroBlock, + blockOrigin, + tileIndex, + transformSetType, + blockContext, + transformSizeRate, + transformSizeContext, + samples, + rows, + columns, + colorCache[..colorCacheSize], + blockSizeContext, + neighborContext, + centroids[..paletteSize], + colorIndexMap, + candidateReconstruction, + candidateCoefficients, + retainedCoefficients, + retainedStates, + retainedColorIndexMap, + reconstructionPlane, + ref bestStatistics, + ref paletteInfo, + ref selectedTransformSize, + ref paletteSelected); + } + + if (uniqueColorCount == 2) + { + centroids[0] = minimum; + centroids[1] = maximum; + this.EvaluateLumaPaletteCandidate( + writer, + macroBlock, + blockOrigin, + tileIndex, + transformSetType, + blockContext, + transformSizeRate, + transformSizeContext, + samples, + rows, + columns, + colorCache[..colorCacheSize], + blockSizeContext, + neighborContext, + centroids[..2], + colorIndexMap, + candidateReconstruction, + candidateCoefficients, + retainedCoefficients, + retainedStates, + retainedColorIndexMap, + reconstructionPlane, + ref bestStatistics, + ref paletteInfo, + ref selectedTransformSize, + ref paletteSelected); + } + else + { + Span clusterIndices = workspace.Indices[..sampleCount]; + for (int paletteSize = 2; paletteSize <= maximumPaletteSize; paletteSize++) + { + Span candidateCentroids = centroids[..paletteSize]; + Av1PaletteKMeans.InitializeCentroids(minimum, maximum, candidateCentroids); + Av1PaletteKMeans.Cluster( + samples, + candidateCentroids, + clusterIndices, + workspace.GetAlternateCentroids(0), + workspace.AlternateIndices); + + this.EvaluateLumaPaletteCandidate( + writer, + macroBlock, + blockOrigin, + tileIndex, + transformSetType, + blockContext, + transformSizeRate, + transformSizeContext, + samples, + rows, + columns, + colorCache[..colorCacheSize], + blockSizeContext, + neighborContext, + candidateCentroids, + colorIndexMap, + candidateReconstruction, + candidateCoefficients, + retainedCoefficients, + retainedStates, + retainedColorIndexMap, + reconstructionPlane, + ref bestStatistics, + ref paletteInfo, + ref selectedTransformSize, + ref paletteSelected); + } + } + + if (paletteSelected) + { + for (int row = 0; row < BlockLength; row++) + { + retainedColorIndexMap.Slice(row * BlockLength, BlockLength) + .CopyTo(colorIndexMap.DangerousGetRowSpan(row)); + } + } + + return paletteSelected; + } + + private void EvaluateLumaPaletteCandidate( + Av1SymbolEncoder writer, + Av1MacroBlockD macroBlock, + Point blockOrigin, + ushort tileIndex, + Av1TransformSetType transformSetType, + Av1TransformBlockContext blockContext, + int transformSizeRate, + int transformSizeContext, + ReadOnlySpan samples, + int rows, + int columns, + ReadOnlySpan colorCache, + int blockSizeContext, + int neighborContext, + Span centroids, + Buffer2DRegion colorIndexMap, + Span candidateReconstruction, + Span candidateCoefficients, + Span retainedCoefficients, + Span retainedStates, + Span retainedColorIndexMap, + Buffer2DRegion reconstructionPlane, + ref Av1RateDistortionStatistics bestStatistics, + ref Av1EncoderPaletteInfo paletteInfo, + ref Av1TransformSize selectedTransformSize, + ref bool paletteSelected) + { + const Av1BlockSize BlockSize = Av1BlockSize.Block8x8; + const Av1TransformSize TransformSize = Av1TransformSize.Size8x8; + const int BlockLength = 8; + Av1EncoderModeDecisionWorkspace modeDecisionWorkspace = + this.blockWorkspace.GetModeDecisionWorkspace(); + + Av1EncoderPaletteWorkspace workspace = modeDecisionWorkspace.Palette; + + int bitDepth = this.bitDepth.GetBitCount(); + int cacheThreshold = 4 << (bitDepth - 8); + + // Nearby colors snap to a coded-neighbor cache entry when the quantization error is bounded. + // Snapping can merge centroids, so sorting and compaction below establish the final coded palette. + for (int colorIndex = 0; colorIndex < centroids.Length && !colorCache.IsEmpty; colorIndex++) + { + int minimumDifference = Math.Abs(centroids[colorIndex] - colorCache[0]); + int nearestCacheIndex = 0; + for (int cacheIndex = 1; cacheIndex < colorCache.Length; cacheIndex++) + { + int difference = Math.Abs(centroids[colorIndex] - colorCache[cacheIndex]); + if (difference < minimumDifference) + { + minimumDifference = difference; + nearestCacheIndex = cacheIndex; + } + } + + if (minimumDifference <= cacheThreshold) + { + centroids[colorIndex] = (short)colorCache[nearestCacheIndex]; + } + } + + centroids.Sort(); + int paletteSize = 1; + for (int colorIndex = 1; colorIndex < centroids.Length; colorIndex++) + { + if (centroids[colorIndex] != centroids[colorIndex - 1]) + { + centroids[paletteSize++] = centroids[colorIndex]; + } + } + + if (paletteSize < 2) + { + return; + } + + ReadOnlySpan paletteCentroids = centroids[..paletteSize]; + Span paletteColors = workspace.GetPaletteColors(0)[..paletteSize]; + for (int colorIndex = 0; colorIndex < paletteSize; colorIndex++) + { + paletteColors[colorIndex] = (ushort)paletteCentroids[colorIndex]; + } + + Span colorIndices = workspace.Indices; + Av1PaletteKMeans.AssignIndices(samples, paletteCentroids, colorIndices); + for (int row = 0; row < rows; row++) + { + Span mapRow = colorIndexMap.DangerousGetRowSpan(row)[..BlockLength]; + colorIndices.Slice(row * columns, columns).CopyTo(mapRow); + mapRow[columns..].Fill(mapRow[columns - 1]); + } + + // Padding repeats the last active edge so transform prediction matches coded-frame edge extension. + for (int row = rows; row < BlockLength; row++) + { + colorIndexMap.DangerousGetRowSpan(rows - 1)[..BlockLength] + .CopyTo(colorIndexMap.DangerousGetRowSpan(row)); + } + + Span prediction = workspace.GetPrediction(0); + Span residual = workspace.GetResidual(0); + TOperator.PreparePalette( + this.source.GetPlane(Av1Plane.Y), + blockOrigin, + paletteColors, + colorIndexMap, + prediction, + residual, + TransformSize); + + int rate = Av1TileWriter.GetLumaModeCost( + writer, + macroBlock, + BlockSize, + Av1PredictionMode.DC, + 0, + this.picture.Parent.FrameHeader.IsIntra); + + rate += writer.GetPaletteYModeCost(true, blockSizeContext, neighborContext); + rate += writer.GetPaletteSizeCost(paletteSize, blockSizeContext, Av1PlaneType.Y); + rate += Av1SymbolEncoder.GetPaletteYColorCost(colorCache, paletteColors, bitDepth); + rate += writer.GetPaletteColorMapCost( + paletteSize, + Av1PlaneType.Y, + rows, + columns, + colorIndexMap); + + // Palette prediction and subtraction are invariant for this color map. Reuse them across legal + // transform types, whose enumeration order also supplies deterministic tie precedence. + for (Av1TransformType transformType = Av1TransformType.DctDct; + transformType < Av1TransformType.AllTransformTypes; + transformType++) + { + if (!transformType.IsExtendedSetUsed(transformSetType)) + { + continue; + } + + Av1EncoderTransformBlockState candidateState = default; + long distortion = TOperator.EncodePredictionCandidate( + this.blockWorkspace, + this.source.GetPlane(Av1Plane.Y), + blockOrigin, + prediction, + residual, + candidateReconstruction, + TransformSize.GetWidth(), + candidateCoefficients, + TransformSize, + transformType, + Av1Plane.Y, + this.quantization.QIndex[0], + this.quantization.DeltaQDc[(int)Av1Plane.Y], + this.quantization.DeltaQAc[(int)Av1Plane.Y], + this.bitDepth, + ref candidateState); + + int candidateRate = rate + transformSizeRate; + candidateRate += writer.GetCoefficientCost( + TransformSize, + transformType, + Av1PredictionMode.DC, + candidateCoefficients, + Av1ComponentType.Luminance, + blockContext, + candidateState.EndOfBlock, + this.picture.Parent.FrameHeader.UseReducedTransformSet, + Av1FilterIntraMode.AllFilterIntraModes, + usesInterTransformSet: false); + + Av1RateDistortionStatistics candidateStatistics = new(this.rateMultiplier, candidateRate, distortion); + if (candidateStatistics.Cost < bestStatistics.Cost) + { + // Later palette sizes reuse every candidate span and the shared color map. Publish the + // complete palette state only when this candidate improves the global luma decision. + CopyCandidate( + candidateReconstruction, + candidateCoefficients, + reconstructionPlane, + blockOrigin, + retainedCoefficients, + TransformSize, + candidateState, + ref retainedStates[0]); + + for (int row = 0; row < BlockLength; row++) + { + colorIndexMap.DangerousGetRowSpan(row)[..BlockLength] + .CopyTo(retainedColorIndexMap[(row * BlockLength)..]); + } + + paletteInfo.PaletteSizes[0] = (byte)paletteSize; + paletteInfo.SetColors(Av1Plane.Y, paletteColors); + selectedTransformSize = TransformSize; + bestStatistics = candidateStatistics; + paletteSelected = true; + } + } + + if (this.effort >= 6 && + this.picture.Parent.FrameHeader.TransformMode == Av1TransformMode.Select) + { + // Transform size is part of each palette candidate's RD result. Searching it here preserves + // candidates whose 4x4 residual partition wins even when their 8x8 result does not. + Av1RateDistortionStatistics splitStatistics = this.GetSplitLumaCandidateCost( + writer, + macroBlock, + this.source.GetPlane(Av1Plane.Y), + reconstructionPlane, + blockOrigin, + tileIndex, + Av1PredictionMode.DC, + 0, + Av1FilterIntraMode.AllFilterIntraModes, + paletteSize, + paletteColors, + rate, + 0, + transformSizeContext, + bestStatistics.Cost, + candidateReconstruction, + candidateCoefficients, + modeDecisionWorkspace.CandidateTransformBlocks); + + if (splitStatistics.Cost < bestStatistics.Cost) + { + CopySplitCandidate( + candidateReconstruction, + candidateCoefficients, + modeDecisionWorkspace.CandidateTransformBlocks, + reconstructionPlane, + blockOrigin, + retainedCoefficients, + retainedStates); + + for (int row = 0; row < BlockLength; row++) + { + colorIndexMap.DangerousGetRowSpan(row)[..BlockLength] + .CopyTo(retainedColorIndexMap[(row * BlockLength)..]); + } + + paletteInfo.PaletteSizes[0] = (byte)paletteSize; + paletteInfo.SetColors(Av1Plane.Y, paletteColors); + selectedTransformSize = Av1TransformSize.Size4x4; + bestStatistics = splitStatistics; + paletteSelected = true; + } + } + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1IntraSuperblockEncoder.ReferenceModeDecision.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1IntraSuperblockEncoder.ReferenceModeDecision.cs new file mode 100644 index 0000000000..229b809730 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1IntraSuperblockEncoder.ReferenceModeDecision.cs @@ -0,0 +1,1759 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1.Entropy; +using SixLabors.ImageSharp.Formats.Heif.Av1.Motion; +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.Quantizers; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; +using SixLabors.ImageSharp.Memory; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline; + +/// +/// Provides full rate-distortion selection for reference-frame and intra-block-copy candidates. +/// +internal static partial class Av1IntraSuperblockEncoder +{ + /// + /// The first effort tier that searches a block-local motion vector. + /// + private const int MinimumInterMotionSearchEffort = 6; + + /// + /// The smallest full-pixel radius used by block-local inter search. + /// + private const int MinimumInterMotionSearchRadius = 4; + + /// + /// The first effort tier that refines full-pixel motion to quarter-pixel precision. + /// + private const int MinimumSubpixelMotionSearchEffort = 7; + + /// + /// The first effort tier that adds the final eighth-pixel refinement step. + /// + private const int MinimumHighPrecisionMotionSearchEffort = 8; + + /// + /// The physical border reserved on each side for fractional eight-tap filtering. + /// + private const int FractionalInterpolationBorder = 4; + + /// + /// The number of cardinal and diagonal candidates examined at each search step. + /// + private const int InterMotionSearchDirectionCount = 8; + + /// + /// One nearest, three near, one global, and three new-motion candidates. + /// + private const int MaximumInterModeCandidateCount = 8; + + internal partial struct ModeDecision + where TSample : unmanaged + where TOperator : struct, IBlockEncodingOperator + { + private Av1RateDistortionStatistics SelectIntraBlockCopy( + Av1SymbolEncoder writer, + Av1MacroBlockD macroBlock, + Point blockOrigin, + ushort tileIndex, + Av1RateDistortionStatistics regularStatistics, + ref Av1MacroBlockModeInfo modeInfo, + ref Av1EncoderBlockStruct block, + ref Av1EncoderPaletteInfo paletteInfo) + { + const Av1BlockSize BlockSize = Av1BlockSize.Block8x8; + const Av1TransformSize LumaTransformSize = Av1TransformSize.Size8x8; + Buffer2DRegion lumaSource = this.source.GetPlane(Av1Plane.Y); + Buffer2DRegion lumaReconstruction = this.reconstruction.GetPlane(Av1Plane.Y); + Point modeInfoPosition = new( + blockOrigin.X >> Av1Constants.ModeInfoSizeLog2, + blockOrigin.Y >> Av1Constants.ModeInfoSizeLog2); + + Span referenceCandidates = stackalloc Av1MotionVector[8]; + Span referenceWeights = stackalloc int[8]; + Av1MotionVector reference = Av1IntraBlockCopy.FindReference( + this.picture, + macroBlock, + modeInfoPosition, + BlockSize, + Av1PartitionType.None, + referenceCandidates, + referenceWeights); + + Span candidates = stackalloc Av1MotionVector[4]; + Av1IntraBlockCopySearchIndex search = this.picture.IntraBlockCopySearch; + int candidateCount = search.FindCandidates( + lumaSource, + lumaReconstruction, + blockOrigin, + macroBlock.Tile, + this.picture.Sequence.SequenceHeader, + writer, + reference, + this.rateMultiplier, + candidates); + + candidateCount += search.FindPixelCandidates( + lumaSource, + lumaReconstruction, + blockOrigin, + macroBlock.Tile, + this.picture.Sequence.SequenceHeader, + writer, + reference, + this.quantization.QIndex[0], + this.rateMultiplier, + candidates[candidateCount..]); + + // Hash and full-pixel searches can converge on the same vector. Preserve the first search-order + // occurrence so repeated vectors do not pay for duplicate transform searches or alter ties. + int uniqueCandidateCount = 0; + for (int candidateIndex = 0; candidateIndex < candidateCount; candidateIndex++) + { + Av1MotionVector candidate = candidates[candidateIndex]; + bool duplicate = false; + for (int uniqueIndex = 0; uniqueIndex < uniqueCandidateCount; uniqueIndex++) + { + if (candidate == candidates[uniqueIndex]) + { + duplicate = true; + break; + } + } + + if (!duplicate) + { + candidates[uniqueCandidateCount++] = candidate; + } + } + + if (uniqueCandidateCount == 0) + { + return regularStatistics; + } + + int skipContext = Av1TileWriter.GetSkipContext(macroBlock); + Av1RateDistortionStatistics bestStatistics = regularStatistics; + bool hasSelectedCandidate = false; + bool selectedSkip = false; + Av1MotionVector selectedVector = default; + Av1EncoderTransformBlockState selectedLumaState = default; + Av1EncoderTransformBlockState selectedBlueState = default; + Av1EncoderTransformBlockState selectedRedState = default; + Av1EncoderInterPredictionWorkspace workspace = + this.blockWorkspace.GetInterPredictionWorkspace(); + + Av1TransformBlockContext lumaContext = Av1TileWriter.GetTransformBlockContexts( + Av1ComponentType.Luminance, + this.picture.LuminanceDcSignLevelCoefficientNeighbors[tileIndex], + blockOrigin, + BlockSize, + LumaTransformSize); + + // A coded IBC residual uses the unsplit transform root at this fixed block size. A skipped block + // omits both the transform-partition bit and coefficient syntax, so this rate is added only below. + int transformPartitionRate = 0; + if (this.picture.Parent.FrameHeader.TransformMode == Av1TransformMode.Select) + { + Av1NeighborArrayUnit transformContexts = this.picture.TransformFunctionContexts[tileIndex]; + int topIndex = transformContexts.GetTopIndex(blockOrigin); + int leftIndex = transformContexts.GetLeftIndex(blockOrigin); + int transformPartitionContext = Av1SymbolContextHelper.GetTransformPartitionContext( + transformContexts.Top[topIndex], + transformContexts.Left[leftIndex], + BlockSize, + LumaTransformSize); + + transformPartitionRate = writer.GetTransformPartitionCost( + false, + transformPartitionContext); + } + + ObuColorConfig colorConfig = this.picture.Sequence.SequenceHeader.ColorConfig; + int subsamplingX = colorConfig.SubSamplingX ? 1 : 0; + int subsamplingY = colorConfig.SubSamplingY ? 1 : 0; + Point chromaOrigin = new(blockOrigin.X >> subsamplingX, blockOrigin.Y >> subsamplingY); + Av1TransformSize chromaTransformSize = BlockSize.GetMaxUvTransformSize( + colorConfig.SubSamplingX, + colorConfig.SubSamplingY); + + Av1TransformBlockContext blueContext = default; + Av1TransformBlockContext redContext = default; + if (!this.source.IsMonochrome) + { + Av1BlockSize chromaBlockSize = BlockSize.GetSubsampled( + colorConfig.SubSamplingX, + colorConfig.SubSamplingY); + + blueContext = Av1TileWriter.GetTransformBlockContexts( + Av1ComponentType.Chroma, + this.picture.CbDcSignLevelCoefficientNeighbors[tileIndex], + chromaOrigin, + chromaBlockSize, + chromaTransformSize); + + redContext = Av1TileWriter.GetTransformBlockContexts( + Av1ComponentType.Chroma, + this.picture.CrDcSignLevelCoefficientNeighbors[tileIndex], + chromaOrigin, + chromaBlockSize, + chromaTransformSize); + } + + // Per-vector plane results reuse candidate scratch. Separate selected spans retain only a new + // global winner, allowing the complete search to finish before committed reconstruction changes. + for (int candidateIndex = 0; candidateIndex < uniqueCandidateCount; candidateIndex++) + { + Av1MotionVector candidate = candidates[candidateIndex]; + this.EvaluateInterPlane( + writer, + candidate, + Av1Plane.Y, + Av1ComponentType.Luminance, + Av1PredictionMode.DC, + usePreparedPrediction: false, + Av1InterpolationFilter.Bilinear, + Av1InterpolationFilter.Bilinear, + lumaReconstruction, + blockOrigin, + 0, + 0, + LumaTransformSize, + Av1TransformType.AllTransformTypes, + lumaContext, + workspace.LumaPrediction, + workspace.Residual, + workspace.TransformReconstruction, + workspace.TransformCoefficients, + workspace.LumaCandidateReconstruction, + workspace.LumaCandidateCoefficients, + out Av1EncoderTransformBlockState lumaCandidateState, + out int lumaRate, + out long lumaDistortion, + out bool hasEmptyLuma, + out Av1EncoderTransformBlockState emptyLumaState, + out long emptyLumaDistortion); + + int blueRate = 0; + int redRate = 0; + long blueDistortion = 0; + long redDistortion = 0; + long emptyBlueDistortion = 0; + long emptyRedDistortion = 0; + bool hasEmptyBlue = true; + bool hasEmptyRed = true; + Av1EncoderTransformBlockState blueCandidateState = default; + Av1EncoderTransformBlockState redCandidateState = default; + Av1EncoderTransformBlockState emptyBlueState = default; + Av1EncoderTransformBlockState emptyRedState = default; + if (!this.source.IsMonochrome) + { + Av1TransformType chromaTransformType = lumaCandidateState.TransformType; + Av1TransformSetType chromaTransformSet = Av1SymbolContextHelper.GetExtendedTransformSetType( + chromaTransformSize, + isInter: true, + this.picture.Parent.FrameHeader.UseReducedTransformSet); + + // Inter prediction does not signal an independent chroma transform type. Chroma reuses the + // selected luma type when that type belongs to its transform set and otherwise falls back to DCT. + if (!chromaTransformType.IsExtendedSetUsed(chromaTransformSet)) + { + chromaTransformType = Av1TransformType.DctDct; + } + + this.EvaluateInterPlane( + writer, + candidate, + Av1Plane.U, + Av1ComponentType.Chroma, + Av1PredictionMode.DC, + usePreparedPrediction: false, + Av1InterpolationFilter.Bilinear, + Av1InterpolationFilter.Bilinear, + this.reconstruction.GetPlane(Av1Plane.U), + blockOrigin, + subsamplingX, + subsamplingY, + chromaTransformSize, + chromaTransformType, + blueContext, + workspace.BluePrediction, + workspace.Residual, + workspace.TransformReconstruction, + workspace.TransformCoefficients, + workspace.BlueCandidateReconstruction, + workspace.BlueCandidateCoefficients, + out blueCandidateState, + out blueRate, + out blueDistortion, + out hasEmptyBlue, + out emptyBlueState, + out emptyBlueDistortion); + + this.EvaluateInterPlane( + writer, + candidate, + Av1Plane.V, + Av1ComponentType.Chroma, + Av1PredictionMode.DC, + usePreparedPrediction: false, + Av1InterpolationFilter.Bilinear, + Av1InterpolationFilter.Bilinear, + this.reconstruction.GetPlane(Av1Plane.V), + blockOrigin, + subsamplingX, + subsamplingY, + chromaTransformSize, + chromaTransformType, + redContext, + workspace.RedPrediction, + workspace.Residual, + workspace.TransformReconstruction, + workspace.TransformCoefficients, + workspace.RedCandidateReconstruction, + workspace.RedCandidateCoefficients, + out redCandidateState, + out redRate, + out redDistortion, + out hasEmptyRed, + out emptyRedState, + out emptyRedDistortion); + } + + int displacementRate = writer.GetDisplacementVectorCost(candidate, reference); + int candidateRate = writer.GetUseIntraBlockCopyCost(true) + + displacementRate + + writer.GetSkipCost(false, skipContext) + + transformPartitionRate + + lumaRate + + blueRate + + redRate; + + long candidateDistortion = lumaDistortion + blueDistortion + redDistortion; + Av1RateDistortionStatistics candidateStatistics = new(this.rateMultiplier, candidateRate, candidateDistortion); + bool candidateSkip = false; + + // The skip alternative is available only when every coded plane has an empty transform. Its + // distortion comes from prediction alone and its rate excludes the transform tree and coefficients. + if (hasEmptyLuma && hasEmptyBlue && hasEmptyRed) + { + int skipRate = writer.GetUseIntraBlockCopyCost(true) + + displacementRate + + writer.GetSkipCost(true, skipContext); + + long skipDistortion = emptyLumaDistortion + emptyBlueDistortion + emptyRedDistortion; + Av1RateDistortionStatistics skipStatistics = new(this.rateMultiplier, skipRate, skipDistortion); + if (skipStatistics.Cost < candidateStatistics.Cost) + { + candidateStatistics = skipStatistics; + candidateSkip = true; + } + } + + // Conventional intra and earlier IBC vectors retain strict search-order precedence on equal RD. + if (candidateStatistics.Cost >= bestStatistics.Cost) + { + continue; + } + + bestStatistics = candidateStatistics; + hasSelectedCandidate = true; + selectedSkip = candidateSkip; + selectedVector = candidate; + if (candidateSkip) + { + workspace.LumaPrediction.CopyTo(workspace.SelectedLumaReconstruction); + workspace.SelectedLumaCoefficients.Clear(); + selectedLumaState = emptyLumaState; + if (!this.source.IsMonochrome) + { + int chromaSampleCount = chromaTransformSize.GetSize2d(); + workspace.BluePrediction[..chromaSampleCount].CopyTo(workspace.SelectedBlueReconstruction); + workspace.RedPrediction[..chromaSampleCount].CopyTo(workspace.SelectedRedReconstruction); + workspace.SelectedBlueCoefficients[..chromaSampleCount].Clear(); + workspace.SelectedRedCoefficients[..chromaSampleCount].Clear(); + selectedBlueState = emptyBlueState; + selectedRedState = emptyRedState; + } + } + else + { + workspace.LumaCandidateReconstruction.CopyTo(workspace.SelectedLumaReconstruction); + workspace.LumaCandidateCoefficients.CopyTo(workspace.SelectedLumaCoefficients); + selectedLumaState = lumaCandidateState; + if (!this.source.IsMonochrome) + { + int chromaSampleCount = chromaTransformSize.GetSize2d(); + workspace.BlueCandidateReconstruction[..chromaSampleCount] + .CopyTo(workspace.SelectedBlueReconstruction); + + workspace.RedCandidateReconstruction[..chromaSampleCount] + .CopyTo(workspace.SelectedRedReconstruction); + + workspace.BlueCandidateCoefficients[..chromaSampleCount] + .CopyTo(workspace.SelectedBlueCoefficients); + + workspace.RedCandidateCoefficients[..chromaSampleCount] + .CopyTo(workspace.SelectedRedCoefficients); + + selectedBlueState = blueCandidateState; + selectedRedState = redCandidateState; + } + } + } + + if (!hasSelectedCandidate) + { + return bestStatistics; + } + + // Only the winning vector is now visible to later coding blocks. This single publication keeps + // rejected motion vectors from contaminating intra references or entropy contexts. + Span retainedLumaCoefficients = this.coefficientBuffer.GetPlaneSpan(this.superblock.Index, Av1Plane.Y); + Span retainedLumaTransformBlocks = + this.coefficientBuffer.GetTransformBlockSpan(this.superblock.Index, Av1Plane.Y); + + int lumaTransformIndex = this.codedAreaLuma / + Av1EncoderCoefficientBuffer.TransformBlockUnitCoefficientCount; + + ref Av1EncoderTransformBlockState retainedLumaState = ref retainedLumaTransformBlocks[lumaTransformIndex]; + CopyCandidate( + workspace.SelectedLumaReconstruction, + workspace.SelectedLumaCoefficients, + lumaReconstruction, + blockOrigin, + retainedLumaCoefficients[this.codedAreaLuma..], + LumaTransformSize, + selectedLumaState, + ref retainedLumaState); + + if (!this.source.IsMonochrome) + { + Span retainedBlueCoefficients = this.coefficientBuffer.GetPlaneSpan(this.superblock.Index, Av1Plane.U); + Span retainedRedCoefficients = this.coefficientBuffer.GetPlaneSpan(this.superblock.Index, Av1Plane.V); + Span retainedBlueTransformBlocks = + this.coefficientBuffer.GetTransformBlockSpan(this.superblock.Index, Av1Plane.U); + + Span retainedRedTransformBlocks = + this.coefficientBuffer.GetTransformBlockSpan(this.superblock.Index, Av1Plane.V); + + int chromaTransformIndex = this.codedAreaChroma / + Av1EncoderCoefficientBuffer.TransformBlockUnitCoefficientCount; + + ref Av1EncoderTransformBlockState retainedBlueState = ref retainedBlueTransformBlocks[chromaTransformIndex]; + ref Av1EncoderTransformBlockState retainedRedState = ref retainedRedTransformBlocks[chromaTransformIndex]; + CopyCandidate( + workspace.SelectedBlueReconstruction, + workspace.SelectedBlueCoefficients, + this.reconstruction.GetPlane(Av1Plane.U), + chromaOrigin, + retainedBlueCoefficients[this.codedAreaChroma..], + chromaTransformSize, + selectedBlueState, + ref retainedBlueState); + + CopyCandidate( + workspace.SelectedRedReconstruction, + workspace.SelectedRedCoefficients, + this.reconstruction.GetPlane(Av1Plane.V), + chromaOrigin, + retainedRedCoefficients[this.codedAreaChroma..], + chromaTransformSize, + selectedRedState, + ref retainedRedState); + } + + modeInfo.Block.Mode = Av1PredictionMode.DC; + modeInfo.Block.UvMode = Av1ChromaPredictionMode.DC; + modeInfo.Block.TransformSize = LumaTransformSize; + modeInfo.Block.Skip = selectedSkip; + modeInfo.Block.UseIntraBlockCopy = true; + block.FilterIntraMode = Av1FilterIntraMode.AllFilterIntraModes; + block.PredictionUnit.AngleDelta[(int)Av1PlaneType.Y] = 0; + block.PredictionUnit.AngleDelta[(int)Av1PlaneType.Uv] = 0; + block.PredictionUnit.ChromaFromLumaIndex = 0; + block.PredictionUnit.ChromaFromLumaSigns = 0; + paletteInfo = default; + this.picture.SetDisplacementVector(modeInfoPosition, selectedVector); + return bestStatistics; + } + + /// + /// Compares the retained intra result with an inter candidate without disturbing the intra result on loss. + /// + private Av1RateDistortionStatistics SelectInterPrediction( + Av1SymbolEncoder writer, + Av1MacroBlockD macroBlock, + Point blockOrigin, + ushort tileIndex, + Av1RateDistortionStatistics regularStatistics, + ref Av1MacroBlockModeInfo modeInfo, + ref Av1EncoderBlockStruct block, + ref Av1EncoderPaletteInfo paletteInfo) + { + Av1MacroBlockModeInfo interModeInfo = modeInfo; + Av1EncoderBlockStruct interBlock = block; + Av1EncoderPaletteInfo interPaletteInfo = default; + Av1RateDistortionStatistics selectedStatistics = this.SelectInterBlock( + writer, + macroBlock, + blockOrigin, + tileIndex, + regularStatistics, + ref interModeInfo, + ref interBlock, + ref interPaletteInfo); + + if (selectedStatistics.Cost < regularStatistics.Cost) + { + modeInfo = interModeInfo; + block = interBlock; + paletteInfo = interPaletteInfo; + } + + return selectedStatistics; + } + + /// + /// Evaluates the supported LAST_FRAME modes and publishes only a strict improvement over the intra result. + /// + private Av1RateDistortionStatistics SelectInterBlock( + Av1SymbolEncoder writer, + Av1MacroBlockD macroBlock, + Point blockOrigin, + ushort tileIndex, + Av1RateDistortionStatistics regularStatistics, + ref Av1MacroBlockModeInfo modeInfo, + ref Av1EncoderBlockStruct block, + ref Av1EncoderPaletteInfo paletteInfo) + { + const Av1BlockSize BlockSize = Av1BlockSize.Block8x8; + const Av1TransformSize LumaTransformSize = Av1TransformSize.Size8x8; + const int LastReferenceIndex = 0; + DebugGuard.IsTrue(modeInfo.Block.BlockSize == BlockSize, "Inter prediction currently uses the prepared 8x8 partition tree."); + + modeInfo.Block.ReferenceFrame = Av1ReferenceFrameType.Last; + modeInfo.Block.UvMode = Av1ChromaPredictionMode.DC; + modeInfo.Block.TransformSize = LumaTransformSize; + modeInfo.Block.UseIntraBlockCopy = false; + block.FilterIntraMode = Av1FilterIntraMode.AllFilterIntraModes; + block.PredictionUnit.AngleDelta[(int)Av1PlaneType.Y] = 0; + block.PredictionUnit.AngleDelta[(int)Av1PlaneType.Uv] = 0; + block.PredictionUnit.ChromaFromLumaIndex = 0; + block.PredictionUnit.ChromaFromLumaSigns = 0; + paletteInfo = default; + + Av1EncoderInterPredictionWorkspace workspace = + this.blockWorkspace.GetInterPredictionWorkspace(); + + ObuFrameHeader frameHeader = this.picture.Parent.FrameHeader; + Point modeInfoPosition = blockOrigin >> Av1Constants.ModeInfoSizeLog2; + ref Av1ReferenceMotionVectors referenceMotionVectors = ref this.blockWorkspace.ReferenceMotionVectors; + referenceMotionVectors.Build( + this.picture, + macroBlock, + modeInfoPosition, + BlockSize, + modeInfo.Block.PartitionType, + this.picture.Sequence.SequenceHeader, + frameHeader, + Av1ReferenceFrameType.Last); + + Av1MotionVector globalMotion = frameHeader + .GetGlobalMotionParameters()[LastReferenceIndex] + .GetMotionVector( + frameHeader.AllowHighPrecisionMotionVector, + BlockSize, + modeInfoPosition, + frameHeader.ForceIntegerMotionVector); + + Span candidateVectors = stackalloc Av1MotionVector[MaximumInterModeCandidateCount]; + Span candidateModes = stackalloc Av1PredictionMode[MaximumInterModeCandidateCount]; + Span candidateReferenceIndices = stackalloc byte[MaximumInterModeCandidateCount]; + int candidateCount = 0; + if (this.effort >= MinimumInterMotionSearchEffort) + { + // Predictor-stack modes precede global and new motion so strict ties retain the reference order. + candidateVectors[candidateCount] = referenceMotionVectors.Nearest; + candidateModes[candidateCount] = Av1PredictionMode.NearestMotionVector; + candidateReferenceIndices[candidateCount++] = 0; + + int maximumNearIndex = Math.Min(2, Math.Max(0, referenceMotionVectors.Count - 2)); + for (int referenceIndex = 0; referenceIndex <= maximumNearIndex; referenceIndex++) + { + candidateVectors[candidateCount] = referenceMotionVectors.GetNearReference(referenceIndex); + candidateModes[candidateCount] = Av1PredictionMode.NearMotionVector; + candidateReferenceIndices[candidateCount++] = (byte)referenceIndex; + } + } + + candidateVectors[candidateCount] = globalMotion; + candidateModes[candidateCount] = Av1PredictionMode.GlobalMotionVector; + candidateReferenceIndices[candidateCount++] = 0; + if (this.effort >= MinimumInterMotionSearchEffort) + { + int maximumNewIndex = Math.Min(2, Math.Max(0, referenceMotionVectors.Count - 1)); + for (int referenceIndex = 0; referenceIndex <= maximumNewIndex; referenceIndex++) + { + Av1MotionVector newReference = referenceMotionVectors.GetNewReference(referenceIndex); + Av1MotionVector searched = this.FindInterMotionVector( + writer, + blockOrigin, + newReference, + referenceIndex); + + // Equal prediction vectors can carry different DRL and mode costs. Preserve each syntax choice + // as an independent candidate instead of deduplicating solely by reconstructed pixels. + candidateVectors[candidateCount] = searched; + candidateModes[candidateCount] = Av1PredictionMode.NewMotionVector; + candidateReferenceIndices[candidateCount++] = (byte)referenceIndex; + } + } + + Span selectedLumaReconstruction = workspace.SelectedLumaReconstruction; + Span candidateLumaReconstruction = workspace.LumaCandidateReconstruction; + Span selectedBlueReconstruction = workspace.SelectedBlueReconstruction; + Span candidateBlueReconstruction = workspace.BlueCandidateReconstruction; + Span selectedRedReconstruction = workspace.SelectedRedReconstruction; + Span candidateRedReconstruction = workspace.RedCandidateReconstruction; + Span selectedLumaCoefficients = workspace.SelectedLumaCoefficients; + Span candidateLumaCoefficients = workspace.LumaCandidateCoefficients; + Span selectedBlueCoefficients = workspace.SelectedBlueCoefficients; + Span candidateBlueCoefficients = workspace.BlueCandidateCoefficients; + Span selectedRedCoefficients = workspace.SelectedRedCoefficients; + Span candidateRedCoefficients = workspace.RedCandidateCoefficients; + int skipContext = Av1TileWriter.GetSkipContext(macroBlock); + Span referenceCounts = stackalloc byte[Av1Constants.ReferenceFrameCount]; + Av1TileWriter.CollectNeighborReferenceCounts(macroBlock, referenceCounts); + int commonPredictionRate = writer.GetIsInterCost( + isInter: true, + Av1TileWriter.GetIntraInterContext(macroBlock)) + + writer.GetSingleReferenceCost(Av1ReferenceFrameType.Last, referenceCounts); + + int transformPartitionRate = 0; + if (frameHeader.TransformMode == Av1TransformMode.Select) + { + Av1NeighborArrayUnit transformContexts = this.picture.TransformFunctionContexts[tileIndex]; + int topIndex = transformContexts.GetTopIndex(blockOrigin); + int leftIndex = transformContexts.GetLeftIndex(blockOrigin); + int transformPartitionContext = Av1SymbolContextHelper.GetTransformPartitionContext( + transformContexts.Top[topIndex], + transformContexts.Left[leftIndex], + BlockSize, + LumaTransformSize); + + transformPartitionRate = writer.GetTransformPartitionCost(false, transformPartitionContext); + } + + Av1RateDistortionStatistics selectedStatistics = regularStatistics; + Av1MotionVector selectedVector = default; + Av1PredictionMode selectedMode = default; + int selectedReferenceIndex = 0; + bool selectedSkip = false; + Av1EncoderTransformBlockState selectedLumaState = default; + Av1EncoderTransformBlockState selectedBlueState = default; + Av1EncoderTransformBlockState selectedRedState = default; + bool hasInterWinner = false; + + ObuSequenceHeader sequenceHeader = this.picture.Sequence.SequenceHeader; + bool isSwitchable = frameHeader.InterpolationFilter == Av1InterpolationFilter.Switchable; + bool isDualFilter = sequenceHeader.EnableDualFilter; + Av1InterpolationFilter defaultFilter = isSwitchable ? Av1InterpolationFilter.Regular : frameHeader.InterpolationFilter; + Av1InterpolationFilter selectedVerticalFilter = defaultFilter; + Av1InterpolationFilter selectedHorizontalFilter = defaultFilter; + const int FilterCount = Av1SymbolContextHelper.SwitchableInterpolationFilterCount; + Span verticalFilterRates = stackalloc int[FilterCount]; + Span horizontalFilterRates = stackalloc int[FilterCount]; + int cheapestVerticalFilter = 0; + int cheapestHorizontalFilter = 0; + if (isSwitchable) + { + int verticalContext = Av1SymbolContextHelper.GetSwitchableInterpolationContext(modeInfo.Block, macroBlock, direction: 0); + int horizontalContext = Av1SymbolContextHelper.GetSwitchableInterpolationContext(modeInfo.Block, macroBlock, direction: 1); + for (int filterIndex = 0; filterIndex < FilterCount; filterIndex++) + { + Av1InterpolationFilter filter = (Av1InterpolationFilter)filterIndex; + verticalFilterRates[filterIndex] = writer.GetSwitchableInterpolationFilterCost(filter, verticalContext); + horizontalFilterRates[filterIndex] = isDualFilter ? writer.GetSwitchableInterpolationFilterCost(filter, horizontalContext) : 0; + if (verticalFilterRates[filterIndex] < verticalFilterRates[cheapestVerticalFilter]) + { + cheapestVerticalFilter = filterIndex; + } + + if (horizontalFilterRates[filterIndex] < horizontalFilterRates[cheapestHorizontalFilter]) + { + cheapestHorizontalFilter = filterIndex; + } + } + } + + // An integer luma displacement can still land between chroma samples. Test the finest active plane's + // phase before collapsing filter choices; otherwise odd translations would skip real chroma differences. + int horizontalFractionMask = (Av1MotionVector.SubpixelScale << (block.HasChroma && sequenceHeader.ColorConfig.SubSamplingX ? 1 : 0)) - 1; + int verticalFractionMask = (Av1MotionVector.SubpixelScale << (block.HasChroma && sequenceHeader.ColorConfig.SubSamplingY ? 1 : 0)) - 1; + + // Rank interpolation families with prediction-error modeling before running a full transform search. + // The selected inter reconstruction remains untouched while two existing prediction views alternate. + for (int candidateIndex = 0; candidateIndex < candidateCount; candidateIndex++) + { + modeInfo.Block.Mode = candidateModes[candidateIndex]; + bool writesFilters = Av1TileWriter.UsesSwitchableInterpolation(frameHeader, modeInfo.Block); + Av1InterpolationFilter verticalFilter = defaultFilter; + Av1InterpolationFilter horizontalFilter = defaultFilter; + int filterRate = 0; + if (writesFilters) + { + bool hasHorizontalPhase = (candidateVectors[candidateIndex].Column & horizontalFractionMask) != 0; + bool hasVerticalPhase = (candidateVectors[candidateIndex].Row & verticalFractionMask) != 0; + int filterPairCount = isDualFilter ? FilterCount * FilterCount : FilterCount; + long bestModelCost = long.MaxValue; + bool bestPredictionUsesWorkspace = true; + Span bestLumaPrediction = workspace.LumaPrediction; + Span trialLumaPrediction = candidateLumaReconstruction; + Span bestBluePrediction = workspace.BluePrediction; + Span trialBluePrediction = candidateBlueReconstruction; + Span bestRedPrediction = workspace.RedPrediction; + Span trialRedPrediction = candidateRedReconstruction; + for (int filterPairIndex = 0; filterPairIndex < filterPairCount; filterPairIndex++) + { + int verticalFilterIndex = isDualFilter ? filterPairIndex / FilterCount : filterPairIndex; + int horizontalFilterIndex = isDualFilter ? filterPairIndex % FilterCount : filterPairIndex; + + // At zero phase every filter produces identical samples. Retain only the cheapest signaled + // choice for that axis, or for the common filter when neither axis has a fractional phase. + bool redundantFilter = isDualFilter + ? (!hasVerticalPhase && verticalFilterIndex != cheapestVerticalFilter) || + (!hasHorizontalPhase && horizontalFilterIndex != cheapestHorizontalFilter) + : !hasVerticalPhase && !hasHorizontalPhase && verticalFilterIndex != cheapestVerticalFilter; + + if (redundantFilter) + { + continue; + } + + Av1InterpolationFilter trialVerticalFilter = (Av1InterpolationFilter)verticalFilterIndex; + Av1InterpolationFilter trialHorizontalFilter = (Av1InterpolationFilter)horizontalFilterIndex; + int trialFilterRate = verticalFilterRates[verticalFilterIndex] + horizontalFilterRates[horizontalFilterIndex]; + long modelCost = this.GetInterFilterModelCost( + candidateVectors[candidateIndex], + blockOrigin, + block.HasChroma, + trialHorizontalFilter, + trialVerticalFilter, + trialFilterRate, + trialLumaPrediction, + trialBluePrediction, + trialRedPrediction); + + if (modelCost >= bestModelCost) + { + continue; + } + + bestModelCost = modelCost; + verticalFilter = trialVerticalFilter; + horizontalFilter = trialHorizontalFilter; + filterRate = trialFilterRate; + Span previousLumaPrediction = bestLumaPrediction; + bestLumaPrediction = trialLumaPrediction; + trialLumaPrediction = previousLumaPrediction; + Span previousBluePrediction = bestBluePrediction; + bestBluePrediction = trialBluePrediction; + trialBluePrediction = previousBluePrediction; + Span previousRedPrediction = bestRedPrediction; + bestRedPrediction = trialRedPrediction; + trialRedPrediction = previousRedPrediction; + bestPredictionUsesWorkspace = !bestPredictionUsesWorkspace; + } + + // Keep the winner in the prediction views before transforms reuse candidate reconstruction. + // This requires at most one copy per plane and never rebuilds the chosen interpolation. + if (!bestPredictionUsesWorkspace) + { + bestLumaPrediction.CopyTo(workspace.LumaPrediction); + if (block.HasChroma) + { + bestBluePrediction.CopyTo(workspace.BluePrediction); + bestRedPrediction.CopyTo(workspace.RedPrediction); + } + } + } + + Av1RateDistortionStatistics candidateStatistics = this.EvaluateInterCandidate( + writer, + blockOrigin, + tileIndex, + block.HasChroma, + commonPredictionRate + filterRate, + skipContext, + transformPartitionRate, + candidateVectors[candidateIndex], + candidateModes[candidateIndex], + writesFilters, + horizontalFilter, + verticalFilter, + candidateReferenceIndices[candidateIndex], + in referenceMotionVectors, + candidateLumaReconstruction, + candidateLumaCoefficients, + candidateBlueReconstruction, + candidateBlueCoefficients, + candidateRedReconstruction, + candidateRedCoefficients, + out bool candidateSkip, + out Av1EncoderTransformBlockState candidateLumaState, + out Av1EncoderTransformBlockState candidateBlueState, + out Av1EncoderTransformBlockState candidateRedState); + + // Strict replacement preserves predictor-stack, global, then new-motion order on equal RD cost. + if (candidateStatistics.Cost >= selectedStatistics.Cost) + { + continue; + } + + Span previousLumaReconstruction = selectedLumaReconstruction; + selectedLumaReconstruction = candidateLumaReconstruction; + candidateLumaReconstruction = previousLumaReconstruction; + + Span previousBlueReconstruction = selectedBlueReconstruction; + selectedBlueReconstruction = candidateBlueReconstruction; + candidateBlueReconstruction = previousBlueReconstruction; + + Span previousRedReconstruction = selectedRedReconstruction; + selectedRedReconstruction = candidateRedReconstruction; + candidateRedReconstruction = previousRedReconstruction; + + Span previousLumaCoefficients = selectedLumaCoefficients; + selectedLumaCoefficients = candidateLumaCoefficients; + candidateLumaCoefficients = previousLumaCoefficients; + + Span previousBlueCoefficients = selectedBlueCoefficients; + selectedBlueCoefficients = candidateBlueCoefficients; + candidateBlueCoefficients = previousBlueCoefficients; + + Span previousRedCoefficients = selectedRedCoefficients; + selectedRedCoefficients = candidateRedCoefficients; + candidateRedCoefficients = previousRedCoefficients; + + selectedStatistics = candidateStatistics; + selectedVector = candidateVectors[candidateIndex]; + selectedMode = candidateModes[candidateIndex]; + selectedHorizontalFilter = horizontalFilter; + selectedVerticalFilter = verticalFilter; + selectedReferenceIndex = candidateReferenceIndices[candidateIndex]; + selectedSkip = candidateSkip; + selectedLumaState = candidateLumaState; + selectedBlueState = candidateBlueState; + selectedRedState = candidateRedState; + hasInterWinner = true; + } + + // Inter trials never overwrite retained picture state. The complete intra result remains authoritative + // when no inter candidate strictly improves its rate-distortion cost. + if (!hasInterWinner) + { + return regularStatistics; + } + + Span retainedLumaCoefficients = this.coefficientBuffer.GetPlaneSpan(this.superblock.Index, Av1Plane.Y); + Span retainedLumaTransformBlocks = + this.coefficientBuffer.GetTransformBlockSpan(this.superblock.Index, Av1Plane.Y); + + int lumaTransformIndex = this.codedAreaLuma / + Av1EncoderCoefficientBuffer.TransformBlockUnitCoefficientCount; + + CopyCandidate( + selectedLumaReconstruction, + selectedLumaCoefficients, + this.reconstruction.GetPlane(Av1Plane.Y), + blockOrigin, + retainedLumaCoefficients[this.codedAreaLuma..], + LumaTransformSize, + selectedLumaState, + ref retainedLumaTransformBlocks[lumaTransformIndex]); + + if (block.HasChroma) + { + ObuColorConfig colorConfig = this.picture.Sequence.SequenceHeader.ColorConfig; + int subsamplingX = colorConfig.SubSamplingX ? 1 : 0; + int subsamplingY = colorConfig.SubSamplingY ? 1 : 0; + Point chromaOrigin = Av1TileWriter.GetChromaBlockOrigin( + blockOrigin, + subsamplingX, + subsamplingY); + + Av1TransformSize chromaTransformSize = BlockSize.GetMaxUvTransformSize( + colorConfig.SubSamplingX, + colorConfig.SubSamplingY); + + Span retainedBlueCoefficients = this.coefficientBuffer.GetPlaneSpan(this.superblock.Index, Av1Plane.U); + Span retainedRedCoefficients = this.coefficientBuffer.GetPlaneSpan(this.superblock.Index, Av1Plane.V); + Span retainedBlueTransformBlocks = + this.coefficientBuffer.GetTransformBlockSpan(this.superblock.Index, Av1Plane.U); + + Span retainedRedTransformBlocks = + this.coefficientBuffer.GetTransformBlockSpan(this.superblock.Index, Av1Plane.V); + + int chromaTransformIndex = this.codedAreaChroma / + Av1EncoderCoefficientBuffer.TransformBlockUnitCoefficientCount; + + CopyCandidate( + selectedBlueReconstruction, + selectedBlueCoefficients, + this.reconstruction.GetPlane(Av1Plane.U), + chromaOrigin, + retainedBlueCoefficients[this.codedAreaChroma..], + chromaTransformSize, + selectedBlueState, + ref retainedBlueTransformBlocks[chromaTransformIndex]); + + CopyCandidate( + selectedRedReconstruction, + selectedRedCoefficients, + this.reconstruction.GetPlane(Av1Plane.V), + chromaOrigin, + retainedRedCoefficients[this.codedAreaChroma..], + chromaTransformSize, + selectedRedState, + ref retainedRedTransformBlocks[chromaTransformIndex]); + } + + modeInfo.Block.Mode = selectedMode; + modeInfo.Block.Skip = selectedSkip; + modeInfo.Block.VerticalInterpolationFilter = selectedVerticalFilter; + modeInfo.Block.HorizontalInterpolationFilter = selectedHorizontalFilter; + block.ReferenceMotionVectorIndex = selectedReferenceIndex; + this.picture.SetDisplacementVector(modeInfoPosition, selectedVector); + return selectedStatistics; + } + + /// + /// Ranks a filter pair from visible prediction error using the reference curve model, without transforming samples. + /// + private long GetInterFilterModelCost( + Av1MotionVector vector, + Point blockOrigin, + bool hasChroma, + Av1InterpolationFilter horizontalFilter, + Av1InterpolationFilter verticalFilter, + int filterRate, + Span lumaPrediction, + Span bluePrediction, + Span redPrediction) + { + const Av1BlockSize BlockSize = Av1BlockSize.Block8x8; + const int InterpolationPrecisionBits = 4; + const int InterpolationPhaseMask = (1 << InterpolationPrecisionBits) - 1; + Av1EncoderInterPredictionWorkspace workspace = this.blockWorkspace.GetInterPredictionWorkspace(); + int planeCount = hasChroma ? 3 : 1; + int rate = filterRate; + long distortion = 0; + for (int planeIndex = 0; planeIndex < planeCount; planeIndex++) + { + Av1Plane plane = (Av1Plane)planeIndex; + int subsamplingX = plane == Av1Plane.Y ? 0 : this.source.ChromaSubsamplingX; + int subsamplingY = plane == Av1Plane.Y ? 0 : this.source.ChromaSubsamplingY; + Av1BlockSize planeBlockSize = BlockSize.GetSubsampled(subsamplingX != 0, subsamplingY != 0); + Av1TransformSize transformSize = BlockSize.GetMaxUvTransformSize(subsamplingX != 0, subsamplingY != 0); + int width = transformSize.GetWidth(); + int height = transformSize.GetHeight(); + Point planeOrigin = new(blockOrigin.X >> subsamplingX, blockOrigin.Y >> subsamplingY); + int sourceColumn = (planeOrigin.X << InterpolationPrecisionBits) + (vector.Column << (1 - subsamplingX)); + int sourceRow = (planeOrigin.Y << InterpolationPrecisionBits) + (vector.Row << (1 - subsamplingY)); + Point predictionOrigin = new(sourceColumn >> InterpolationPrecisionBits, sourceRow >> InterpolationPrecisionBits); + Span prediction = plane == Av1Plane.Y ? lumaPrediction : plane == Av1Plane.U ? bluePrediction : redPrediction; + Span residual = workspace.Residual[..(width * height)]; + TOperator.PrepareTranslationalInterPrediction( + this.source.GetPlane(plane), + planeOrigin, + this.reference.GetPlane(plane), + predictionOrigin, + horizontalFilter, + verticalFilter, + sourceColumn & InterpolationPhaseMask, + sourceRow & InterpolationPhaseMask, + prediction, + residual, + workspace.PredictionScratch, + transformSize, + this.bitDepth); + + int visibleWidth = Math.Min(width, ((this.source.Width + subsamplingX) >> subsamplingX) - planeOrigin.X); + int visibleHeight = Math.Min(height, ((this.source.Height + subsamplingY) >> subsamplingY) - planeOrigin.Y); + long squaredError = 0; + + // This view includes coded alignment samples, matching libaom when do_border_pad is false. + // Its conditional border-padding policy is not implemented here; these are not visible-frame bounds. + // Full blocks use one SIMD reduction; only a partial right edge needs row-sized reductions. + if (visibleWidth == width) + { + squaredError = Av1ResidualBuilder.SumSquares(residual[..(width * visibleHeight)]); + } + else + { + for (int row = 0; row < visibleHeight; row++) + { + squaredError += Av1ResidualBuilder.SumSquares(residual.Slice(row * width, visibleWidth)); + } + } + + int normalizationShift = (this.bitDepth.GetBitCount() - 8) * 2; + if (normalizationShift != 0) + { + squaredError = (squaredError + (1L << (normalizationShift - 1))) >> normalizationShift; + } + + int acQuantizer = Av1QuantizationLookup.GetAcQuant( + this.quantization.QIndex[0], + this.quantization.DeltaQAc[planeIndex], + this.bitDepth); + + Av1RateDistortion.ModelPredictionError( + planeBlockSize, + squaredError, + visibleWidth * visibleHeight, + acQuantizer, + this.bitDepth, + this.rateMultiplier, + out int planeRate, + out long planeDistortion); + + rate += planeRate; + distortion += planeDistortion; + } + + return Av1RateDistortion.GetCost(this.rateMultiplier, rate, distortion); + } + + /// + /// Evaluates one inter mode through prediction, transform, coefficient, skip, and distortion selection. + /// + private Av1RateDistortionStatistics EvaluateInterCandidate( + Av1SymbolEncoder writer, + Point blockOrigin, + ushort tileIndex, + bool hasChroma, + int commonPredictionRate, + int skipContext, + int transformPartitionRate, + Av1MotionVector vector, + Av1PredictionMode mode, + bool usePreparedPrediction, + Av1InterpolationFilter horizontalFilter, + Av1InterpolationFilter verticalFilter, + int referenceMotionVectorIndex, + in Av1ReferenceMotionVectors referenceMotionVectors, + Span lumaReconstruction, + Span lumaCoefficients, + Span blueReconstruction, + Span blueCoefficients, + Span redReconstruction, + Span redCoefficients, + out bool skip, + out Av1EncoderTransformBlockState lumaState, + out Av1EncoderTransformBlockState blueState, + out Av1EncoderTransformBlockState redState) + { + const Av1BlockSize BlockSize = Av1BlockSize.Block8x8; + const Av1TransformSize LumaTransformSize = Av1TransformSize.Size8x8; + Av1EncoderInterPredictionWorkspace workspace = + this.blockWorkspace.GetInterPredictionWorkspace(); + + Av1TransformBlockContext lumaContext = Av1TileWriter.GetTransformBlockContexts( + Av1ComponentType.Luminance, + this.picture.LuminanceDcSignLevelCoefficientNeighbors[tileIndex], + blockOrigin, + BlockSize, + LumaTransformSize); + + this.EvaluateInterPlane( + writer, + vector, + Av1Plane.Y, + Av1ComponentType.Luminance, + mode, + usePreparedPrediction, + horizontalFilter, + verticalFilter, + this.reference.GetPlane(Av1Plane.Y), + blockOrigin, + 0, + 0, + LumaTransformSize, + Av1TransformType.AllTransformTypes, + lumaContext, + workspace.LumaPrediction, + workspace.Residual, + workspace.TransformReconstruction, + workspace.TransformCoefficients, + lumaReconstruction, + lumaCoefficients, + out lumaState, + out int lumaRate, + out long lumaDistortion, + out bool hasEmptyLuma, + out Av1EncoderTransformBlockState emptyLumaState, + out long emptyLumaDistortion); + + // Empty luma transforms signal no transform type. Chroma inherits the decoder's inferred DCT + // type, not the last searched luma type, so normalize before evaluating either chroma plane. + if (lumaState.EndOfBlock == 0) + { + lumaState.TransformType = Av1TransformType.DctDct; + } + + ObuColorConfig colorConfig = this.picture.Sequence.SequenceHeader.ColorConfig; + int subsamplingX = colorConfig.SubSamplingX ? 1 : 0; + int subsamplingY = colorConfig.SubSamplingY ? 1 : 0; + Point chromaOrigin = Av1TileWriter.GetChromaBlockOrigin( + blockOrigin, + subsamplingX, + subsamplingY); + + Av1TransformSize chromaTransformSize = BlockSize.GetMaxUvTransformSize( + colorConfig.SubSamplingX, + colorConfig.SubSamplingY); + + int blueRate = 0; + int redRate = 0; + long blueDistortion = 0; + long redDistortion = 0; + long emptyBlueDistortion = 0; + long emptyRedDistortion = 0; + bool hasEmptyBlue = true; + bool hasEmptyRed = true; + blueState = default; + redState = default; + Av1EncoderTransformBlockState emptyBlueState = default; + Av1EncoderTransformBlockState emptyRedState = default; + if (hasChroma) + { + Av1BlockSize chromaBlockSize = BlockSize.GetSubsampled( + colorConfig.SubSamplingX, + colorConfig.SubSamplingY); + + Av1TransformBlockContext blueContext = Av1TileWriter.GetTransformBlockContexts( + Av1ComponentType.Chroma, + this.picture.CbDcSignLevelCoefficientNeighbors[tileIndex], + chromaOrigin, + chromaBlockSize, + chromaTransformSize); + + Av1TransformBlockContext redContext = Av1TileWriter.GetTransformBlockContexts( + Av1ComponentType.Chroma, + this.picture.CrDcSignLevelCoefficientNeighbors[tileIndex], + chromaOrigin, + chromaBlockSize, + chromaTransformSize); + + Av1TransformType chromaTransformType = lumaState.TransformType; + Av1TransformSetType chromaTransformSet = Av1SymbolContextHelper.GetExtendedTransformSetType( + chromaTransformSize, + isInter: true, + this.picture.Parent.FrameHeader.UseReducedTransformSet); + + if (!chromaTransformType.IsExtendedSetUsed(chromaTransformSet)) + { + chromaTransformType = Av1TransformType.DctDct; + } + + this.EvaluateInterPlane( + writer, + vector, + Av1Plane.U, + Av1ComponentType.Chroma, + mode, + usePreparedPrediction, + horizontalFilter, + verticalFilter, + this.reference.GetPlane(Av1Plane.U), + blockOrigin, + subsamplingX, + subsamplingY, + chromaTransformSize, + chromaTransformType, + blueContext, + workspace.BluePrediction, + workspace.Residual, + workspace.TransformReconstruction, + workspace.TransformCoefficients, + blueReconstruction, + blueCoefficients, + out blueState, + out blueRate, + out blueDistortion, + out hasEmptyBlue, + out emptyBlueState, + out emptyBlueDistortion); + + this.EvaluateInterPlane( + writer, + vector, + Av1Plane.V, + Av1ComponentType.Chroma, + mode, + usePreparedPrediction, + horizontalFilter, + verticalFilter, + this.reference.GetPlane(Av1Plane.V), + blockOrigin, + subsamplingX, + subsamplingY, + chromaTransformSize, + chromaTransformType, + redContext, + workspace.RedPrediction, + workspace.Residual, + workspace.TransformReconstruction, + workspace.TransformCoefficients, + redReconstruction, + redCoefficients, + out redState, + out redRate, + out redDistortion, + out hasEmptyRed, + out emptyRedState, + out emptyRedDistortion); + } + + int predictionRate = commonPredictionRate + + this.GetInterModeRate( + writer, + mode, + vector, + referenceMotionVectorIndex, + in referenceMotionVectors); + + int codedRate = predictionRate + + writer.GetSkipCost(false, skipContext) + + transformPartitionRate + + lumaRate + + blueRate + + redRate; + + long codedDistortion = lumaDistortion + blueDistortion + redDistortion; + Av1RateDistortionStatistics selectedStatistics = new(this.rateMultiplier, codedRate, codedDistortion); + skip = false; + if (hasEmptyLuma && hasEmptyBlue && hasEmptyRed) + { + int skipRate = predictionRate + writer.GetSkipCost(true, skipContext); + long skipDistortion = emptyLumaDistortion + emptyBlueDistortion + emptyRedDistortion; + Av1RateDistortionStatistics skipStatistics = new(this.rateMultiplier, skipRate, skipDistortion); + if (skipStatistics.Cost < selectedStatistics.Cost) + { + selectedStatistics = skipStatistics; + skip = true; + workspace.LumaPrediction[..LumaTransformSize.GetSize2d()].CopyTo(lumaReconstruction); + lumaCoefficients[..LumaTransformSize.GetSize2d()].Clear(); + lumaState = emptyLumaState; + if (hasChroma) + { + int chromaSampleCount = chromaTransformSize.GetSize2d(); + workspace.BluePrediction[..chromaSampleCount].CopyTo(blueReconstruction); + workspace.RedPrediction[..chromaSampleCount].CopyTo(redReconstruction); + blueCoefficients[..chromaSampleCount].Clear(); + redCoefficients[..chromaSampleCount].Clear(); + blueState = emptyBlueState; + redState = emptyRedState; + } + } + } + + return selectedStatistics; + } + + /// + /// Searches a bounded full-pixel neighborhood around the spatial reference vector. + /// + /// The live tile entropy model used to measure vector syntax. + /// The current 8x8 luma origin. + /// The differential reference from the spatial candidate stack. + /// The selected dynamic-reference-list entry. + /// The lowest-cost full-pixel vector found by the effort-scaled search. + private Av1MotionVector FindInterMotionVector( + Av1SymbolEncoder writer, + Point blockOrigin, + Av1MotionVector referenceVector, + int referenceMotionVectorIndex) + { + int effortShift = this.effort - MinimumInterMotionSearchEffort; + int searchRadius = Math.Min( + MinimumInterMotionSearchRadius << effortShift, + Av1EncoderFrame.LumaBorder); + + int referenceColumn = referenceVector.Column >> Av1MotionVector.SubpixelBits; + int referenceRow = referenceVector.Row >> Av1MotionVector.SubpixelBits; + int minimumColumn = Math.Max(-Av1EncoderFrame.LumaBorder, referenceColumn - searchRadius); + int maximumColumn = Math.Min(Av1EncoderFrame.LumaBorder, referenceColumn + searchRadius); + int minimumRow = Math.Max(-Av1EncoderFrame.LumaBorder, referenceRow - searchRadius); + int maximumRow = Math.Min(Av1EncoderFrame.LumaBorder, referenceRow + searchRadius); + Point best = new( + Av1Math.Clamp(referenceColumn, minimumColumn, maximumColumn), + Av1Math.Clamp(referenceRow, minimumRow, maximumRow)); + + ref Av1ReferenceMotionVectors referenceMotionVectors = ref this.blockWorkspace.ReferenceMotionVectors; + Av1MotionVector bestVector = new( + best.Y * Av1MotionVector.SubpixelScale, + best.X * Av1MotionVector.SubpixelScale); + + long bestCost = this.GetInterMotionCandidateCost( + writer, + blockOrigin, + bestVector, + Av1PredictionMode.NewMotionVector, + referenceMotionVectorIndex, + in referenceMotionVectors); + + for (int step = searchRadius; step > 0; step >>= 1) + { + Point stageBest = best; + long stageBestCost = bestCost; + for (int directionIndex = 0; directionIndex < InterMotionSearchDirectionCount; directionIndex++) + { + Point direction = GetInterMotionSearchDirection(directionIndex); + Point candidate = new( + best.X + (direction.X * step), + best.Y + (direction.Y * step)); + + if (candidate.X < minimumColumn || candidate.X > maximumColumn || + candidate.Y < minimumRow || candidate.Y > maximumRow) + { + continue; + } + + Av1MotionVector candidateVector = new( + candidate.Y * Av1MotionVector.SubpixelScale, + candidate.X * Av1MotionVector.SubpixelScale); + + long candidateCost = this.GetInterMotionCandidateCost( + writer, + blockOrigin, + candidateVector, + Av1PredictionMode.NewMotionVector, + referenceMotionVectorIndex, + in referenceMotionVectors); + + // Strict replacement preserves the earlier reference-centered search position on ties. + if (candidateCost < stageBestCost) + { + stageBestCost = candidateCost; + stageBest = candidate; + } + } + + best = stageBest; + bestCost = stageBestCost; + } + + bestVector = new( + best.Y * Av1MotionVector.SubpixelScale, + best.X * Av1MotionVector.SubpixelScale); + + if (this.effort < MinimumSubpixelMotionSearchEffort) + { + return bestVector; + } + + int minimumSubpixel = (-Av1EncoderFrame.LumaBorder + FractionalInterpolationBorder) * + Av1MotionVector.SubpixelScale; + + int maximumSubpixel = (Av1EncoderFrame.LumaBorder - FractionalInterpolationBorder) * + Av1MotionVector.SubpixelScale; + + if (bestVector.Column < minimumSubpixel || bestVector.Column > maximumSubpixel || + bestVector.Row < minimumSubpixel || bestVector.Row > maximumSubpixel) + { + return bestVector; + } + + int finalStep = this.effort >= MinimumHighPrecisionMotionSearchEffort ? 1 : 2; + for (int step = Av1MotionVector.SubpixelScale >> 1; step >= finalStep; step >>= 1) + { + Av1MotionVector stageBest = bestVector; + long stageBestCost = bestCost; + for (int directionIndex = 0; directionIndex < InterMotionSearchDirectionCount; directionIndex++) + { + Point direction = GetInterMotionSearchDirection(directionIndex); + Av1MotionVector candidate = new( + bestVector.Row + (direction.Y * step), + bestVector.Column + (direction.X * step)); + + if (candidate.Column < minimumSubpixel || candidate.Column > maximumSubpixel || + candidate.Row < minimumSubpixel || candidate.Row > maximumSubpixel) + { + continue; + } + + long candidateCost = this.GetInterMotionCandidateCost( + writer, + blockOrigin, + candidate, + Av1PredictionMode.NewMotionVector, + referenceMotionVectorIndex, + in referenceMotionVectors); + + // Each precision stage remains centered on its incoming winner; strict replacement keeps + // the integer or coarser fractional vector when an interpolated candidate only ties it. + if (candidateCost < stageBestCost) + { + stageBestCost = candidateCost; + stageBest = candidate; + } + } + + bestVector = stageBest; + bestCost = stageBestCost; + } + + return bestVector; + } + + /// + /// Combines normalized prediction error with the exact mode and vector syntax rate. + /// + /// The live tile entropy model. + /// The current 8x8 luma origin. + /// The candidate motion vector. + /// The candidate single-reference inter mode. + /// The selected dynamic-reference-list entry. + /// The current spatial candidate stack. + /// The rate-distortion search cost. + private long GetInterMotionCandidateCost( + Av1SymbolEncoder writer, + Point blockOrigin, + Av1MotionVector vector, + Av1PredictionMode mode, + int referenceMotionVectorIndex, + in Av1ReferenceMotionVectors referenceMotionVectors) + { + long predictionError; + if (((vector.Row | vector.Column) & (Av1MotionVector.SubpixelScale - 1)) == 0) + { + Point predictionOrigin = new( + blockOrigin.X + (vector.Column >> Av1MotionVector.SubpixelBits), + blockOrigin.Y + (vector.Row >> Av1MotionVector.SubpixelBits)); + + predictionError = TOperator.GetInterPredictionError( + this.source.GetPlane(Av1Plane.Y), + blockOrigin, + this.reference.GetPlane(Av1Plane.Y), + predictionOrigin, + this.bitDepth); + } + else + { + const Av1TransformSize SearchTransformSize = Av1TransformSize.Size8x8; + Av1EncoderInterPredictionWorkspace workspace = + this.blockWorkspace.GetInterPredictionWorkspace(); + + int sourceColumnQ4 = (blockOrigin.X << 4) + (vector.Column << 1); + int sourceRowQ4 = (blockOrigin.Y << 4) + (vector.Row << 1); + Point predictionOrigin = new(sourceColumnQ4 >> 4, sourceRowQ4 >> 4); + ObuFrameHeader frameHeader = this.picture.Parent.FrameHeader; + + // Fractional candidates must pass through the same interpolation and residual kernels used by + // final reconstruction; comparing only their integer origins would choose the wrong phase. + TOperator.PrepareTranslationalInterPrediction( + this.source.GetPlane(Av1Plane.Y), + blockOrigin, + this.reference.GetPlane(Av1Plane.Y), + predictionOrigin, + frameHeader.InterpolationFilter == Av1InterpolationFilter.Switchable ? Av1InterpolationFilter.Regular : frameHeader.InterpolationFilter, + frameHeader.InterpolationFilter == Av1InterpolationFilter.Switchable ? Av1InterpolationFilter.Regular : frameHeader.InterpolationFilter, + sourceColumnQ4 & 15, + sourceRowQ4 & 15, + workspace.LumaPrediction, + workspace.Residual, + workspace.PredictionScratch, + SearchTransformSize, + this.bitDepth); + + predictionError = Av1ResidualBuilder.SumSquares(workspace.Residual); + int normalizationShift = (this.bitDepth.GetBitCount() - 8) * 2; + if (normalizationShift != 0) + { + predictionError = (predictionError + (1L << (normalizationShift - 1))) >> + normalizationShift; + } + } + + int rate = this.GetInterModeRate( + writer, + mode, + vector, + referenceMotionVectorIndex, + in referenceMotionVectors); + + return Av1RateDistortion.GetCost(this.rateMultiplier, rate, predictionError); + } + + /// + /// Measures the complete mode, dynamic-reference-list, and differential-vector syntax for one candidate. + /// + /// The live tile entropy model. + /// The candidate single-reference inter mode. + /// The candidate motion vector. + /// The selected dynamic-reference-list entry. + /// The current spatial candidate stack. + /// The syntax rate in 1/512-bit units. + private int GetInterModeRate( + Av1SymbolEncoder writer, + Av1PredictionMode mode, + Av1MotionVector vector, + int referenceMotionVectorIndex, + in Av1ReferenceMotionVectors referenceMotionVectors) + { + int rate = writer.GetInterModeCost(mode, referenceMotionVectors.ModeContext); + if (mode == Av1PredictionMode.NearMotionVector) + { + for (int index = 1; index < 3 && referenceMotionVectors.Count > index + 1; index++) + { + bool advance = referenceMotionVectorIndex >= index; + int context = Av1SymbolContextHelper.GetDrlContext(referenceMotionVectors.Weights, index); + rate += writer.GetDynamicReferenceListCost(advance, context); + if (!advance) + { + break; + } + } + + return rate; + } + + if (mode != Av1PredictionMode.NewMotionVector) + { + return rate; + } + + for (int index = 0; index < 2 && referenceMotionVectors.Count > index + 1; index++) + { + bool advance = referenceMotionVectorIndex > index; + int context = Av1SymbolContextHelper.GetDrlContext(referenceMotionVectors.Weights, index); + rate += writer.GetDynamicReferenceListCost(advance, context); + if (!advance) + { + break; + } + } + + Av1MotionVector reference = referenceMotionVectors.GetNewReference(referenceMotionVectorIndex); + return rate + writer.GetMotionVectorCost( + vector, + reference, + this.picture.Parent.FrameHeader.MotionVectorPrecision); + } + + /// + /// Gets one cardinal or diagonal search direction in stable reference order. + /// + /// The zero-based direction index. + /// The unit full-pixel direction. + private static Point GetInterMotionSearchDirection(int index) + => index switch + { + 0 => new Point(0, -1), + 1 => new Point(0, 1), + 2 => new Point(-1, 0), + 3 => new Point(1, 0), + 4 => new Point(-1, -1), + 5 => new Point(1, 1), + 6 => new Point(1, -1), + _ => new Point(-1, 1) + }; + + /// + /// Builds one plane prediction and selects its transform without repeating interpolation for each transform type. + /// + private void EvaluateInterPlane( + Av1SymbolEncoder writer, + Av1MotionVector vector, + Av1Plane plane, + Av1ComponentType componentType, + Av1PredictionMode predictionMode, + bool usePreparedPrediction, + Av1InterpolationFilter horizontalFilter, + Av1InterpolationFilter verticalFilter, + Buffer2DRegion referencePlane, + Point lumaOrigin, + int subsamplingX, + int subsamplingY, + Av1TransformSize transformSize, + Av1TransformType transformTypeSelection, + Av1TransformBlockContext blockContext, + Span prediction, + Span residual, + Span transformReconstruction, + Span transformCoefficients, + Span selectedReconstruction, + Span selectedCoefficients, + out Av1EncoderTransformBlockState selectedState, + out int selectedRate, + out long selectedDistortion, + out bool hasEmptyTransform, + out Av1EncoderTransformBlockState emptyState, + out long emptyDistortion) + { + Point planeOrigin = new(lumaOrigin.X >> subsamplingX, lumaOrigin.Y >> subsamplingY); + int sourceColumnQ4 = (planeOrigin.X << 4) + (vector.Column << (1 - subsamplingX)); + int sourceRowQ4 = (planeOrigin.Y << 4) + (vector.Row << (1 - subsamplingY)); + Point predictionOrigin = new(sourceColumnQ4 >> 4, sourceRowQ4 >> 4); + int sampleCount = transformSize.GetSize2d(); + Buffer2DRegion sourcePlane = this.source.GetPlane(plane); + if (usePreparedPrediction) + { + TOperator.SubtractPrediction(sourcePlane, planeOrigin, prediction[..sampleCount], residual[..sampleCount], transformSize); + } + else if (predictionMode >= Av1PredictionMode.InterModeStart) + { + Span predictionScratch = this.blockWorkspace + .GetInterPredictionWorkspace() + .PredictionScratch; + + // Reference-frame modes use the complete interpolation pipeline even when the current zero-phase + // global vector reduces to a SIMD copy. Later fractional vectors therefore share decoder arithmetic. + TOperator.PrepareTranslationalInterPrediction( + sourcePlane, + planeOrigin, + referencePlane, + predictionOrigin, + horizontalFilter, + verticalFilter, + sourceColumnQ4 & 15, + sourceRowQ4 & 15, + prediction[..sampleCount], + residual[..sampleCount], + predictionScratch, + transformSize, + this.picture.Sequence.SequenceHeader.ColorConfig.BitDepth); + } + else + { + // Intra-block copy has its own bilinear half-sample rules and reads the current reconstruction. + TOperator.PrepareIntraBlockCopyPrediction( + sourcePlane, + planeOrigin, + referencePlane, + predictionOrigin, + (sourceColumnQ4 & 15) != 0, + (sourceRowQ4 & 15) != 0, + prediction[..sampleCount], + residual[..sampleCount], + transformSize); + } + + // Motion compensation and subtraction do not depend on transform type. Keep them outside the + // transform loop so exhaustive luma search traverses the source and reference blocks only once. + Av1TransformSetType transformSetType = Av1SymbolContextHelper.GetExtendedTransformSetType( + transformSize, + isInter: true, + this.picture.Parent.FrameHeader.UseReducedTransformSet); + + Av1TransformType firstTransformType = transformTypeSelection == Av1TransformType.AllTransformTypes + ? Av1TransformType.DctDct + : transformTypeSelection; + Av1TransformType transformTypeLimit = transformTypeSelection == Av1TransformType.AllTransformTypes + ? Av1TransformType.AllTransformTypes + : (Av1TransformType)((int)transformTypeSelection + 1); + + long bestCost = long.MaxValue; + selectedState = default; + selectedRate = 0; + selectedDistortion = 0; + hasEmptyTransform = false; + emptyState = default; + emptyDistortion = 0; + + // The candidate and best spans alternate ownership whenever a transform improves the result. + // This mirrors the reference's buffer-pointer swap and replaces a copy on every improvement + // with at most one normalization copy after the transform search. + Span candidateReconstruction = transformReconstruction[..sampleCount]; + Span candidateCoefficients = transformCoefficients[..sampleCount]; + Span bestReconstruction = selectedReconstruction[..sampleCount]; + Span bestCoefficients = selectedCoefficients[..sampleCount]; + bool bestUsesSelectedStorage = true; + for (Av1TransformType transformType = firstTransformType; + transformType < transformTypeLimit; + transformType++) + { + if (!transformType.IsExtendedSetUsed(transformSetType)) + { + continue; + } + + Av1EncoderTransformBlockState candidateState = default; + long candidateDistortion = TOperator.EncodePredictionCandidate( + this.blockWorkspace, + sourcePlane, + planeOrigin, + prediction[..sampleCount], + residual[..sampleCount], + candidateReconstruction, + transformSize.GetWidth(), + candidateCoefficients, + transformSize, + transformType, + plane, + this.quantization.QIndex[0], + this.quantization.DeltaQDc[(int)plane], + this.quantization.DeltaQAc[(int)plane], + this.bitDepth, + ref candidateState); + + int candidateRate = writer.GetCoefficientCost( + transformSize, + transformType, + predictionMode, + candidateCoefficients, + componentType, + blockContext, + candidateState.EndOfBlock, + this.picture.Parent.FrameHeader.UseReducedTransformSet, + Av1FilterIntraMode.AllFilterIntraModes, + usesInterTransformSet: true); + + long candidateCost = Av1RateDistortion.GetCost( + this.rateMultiplier, + candidateRate, + candidateDistortion); + + if (candidateCost < bestCost) + { + Span previousBestReconstruction = bestReconstruction; + bestReconstruction = candidateReconstruction; + candidateReconstruction = previousBestReconstruction; + + Span previousBestCoefficients = bestCoefficients; + bestCoefficients = candidateCoefficients; + candidateCoefficients = previousBestCoefficients; + bestUsesSelectedStorage = !bestUsesSelectedStorage; + bestCost = candidateCost; + selectedState = candidateState; + selectedRate = candidateRate; + selectedDistortion = candidateDistortion; + } + + if (candidateState.EndOfBlock == 0 && + (!hasEmptyTransform || candidateDistortion < emptyDistortion)) + { + hasEmptyTransform = true; + emptyState = candidateState; + emptyDistortion = candidateDistortion; + } + } + + // Callers retain the designated selected spans after this scratch workspace is reused by the + // next plane or motion vector, so normalize only when the final best result occupies scratch. + if (!bestUsesSelectedStorage) + { + bestReconstruction.CopyTo(selectedReconstruction); + bestCoefficients.CopyTo(selectedCoefficients); + } + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1IntraSuperblockEncoder.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1IntraSuperblockEncoder.cs new file mode 100644 index 0000000000..e815a89ce5 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1IntraSuperblockEncoder.cs @@ -0,0 +1,302 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; +using SixLabors.ImageSharp.Memory; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline; + +/// +/// Builds fixed-partition intra decisions and reconstructed samples for one AV1 superblock. +/// +internal static partial class Av1IntraSuperblockEncoder +{ + /// + /// Encodes one superblock stored as eight-bit samples. + /// + /// The coded source frame. + /// The reconstructed frame updated by the block transforms. + /// The frame coding and mode-information state. + /// The reusable partition and final-block decisions. + /// The frame-owned quantized coefficient and transform state. + /// The reusable block arithmetic workspace. + public static void Encode( + Av1EncoderFrame source, + Av1EncoderFrame reconstruction, + Av1PictureControlSet picture, + Av1Superblock superblock, + Av1EncoderCoefficientBuffer coefficientBuffer, + Av1EncoderBlockWorkspace blockWorkspace) + => Encode(source, reconstruction, picture, superblock, coefficientBuffer, blockWorkspace); + + /// + /// Encodes one superblock stored as high-bit-depth samples. + /// + /// The coded source frame. + /// The reconstructed frame updated by the block transforms. + /// The frame coding and mode-information state. + /// The reusable partition and final-block decisions. + /// The frame-owned quantized coefficient and transform state. + /// The reusable block arithmetic workspace. + public static void Encode( + Av1EncoderFrame source, + Av1EncoderFrame reconstruction, + Av1PictureControlSet picture, + Av1Superblock superblock, + Av1EncoderCoefficientBuffer coefficientBuffer, + Av1EncoderBlockWorkspace blockWorkspace) + => Encode(source, reconstruction, picture, superblock, coefficientBuffer, blockWorkspace); + + private static void Encode( + Av1EncoderFrame source, + Av1EncoderFrame reconstruction, + Av1PictureControlSet picture, + Av1Superblock superblock, + Av1EncoderCoefficientBuffer coefficientBuffer, + Av1EncoderBlockWorkspace blockWorkspace) + where TSample : unmanaged + where TOperator : struct, IBlockEncodingOperator + { + int superblockSize = picture.Sequence.SequenceHeader.SuperblockSize.GetWidth(); + Point superblockOrigin = new( + (superblock.Index % coefficientBuffer.SuperblockColumnCount) * superblockSize, + (superblock.Index / coefficientBuffer.SuperblockColumnCount) * superblockSize); + + superblock.Workspace.Reset(); + Traversal traversal = new( + source.CodedView, + reconstruction.CodedView, + picture, + superblock, + coefficientBuffer, + blockWorkspace); + + traversal.EncodePartitionTree(superblockOrigin, picture.Sequence.SequenceHeader.SuperblockSize); + } + + private static void EncodePlaneBlock( + Av1EncoderFrame.PlanarView source, + Av1EncoderFrame.PlanarView reconstruction, + Av1EncoderBlockWorkspace blockWorkspace, + ObuQuantizationParameters quantization, + Av1BitDepth bitDepth, + Av1Plane plane, + Point blockOrigin, + Av1TransformSize transformSize, + Span coefficients, + ref Av1EncoderTransformBlockState state) + where TSample : unmanaged + where TOperator : struct, IBlockEncodingOperator + { + Buffer2DRegion sourcePlane = source.GetPlane(plane); + Buffer2DRegion reconstructionPlane = reconstruction.GetPlane(plane); + int width = transformSize.GetWidth(); + int height = transformSize.GetHeight(); + bool hasLeft = blockOrigin.X > 0; + bool hasAbove = blockOrigin.Y > 0; + ReadOnlySpan above = hasAbove + ? reconstructionPlane.DangerousGetRowSpan(blockOrigin.Y - 1).Slice(blockOrigin.X, width) + : []; + + Span left = TOperator.GetLeftReference(blockWorkspace.Residual, height); + if (hasLeft) + { + for (int row = 0; row < height; row++) + { + left[row] = reconstructionPlane.DangerousGetRowSpan(blockOrigin.Y + row)[blockOrigin.X - 1]; + } + } + + // Prediction consumes every gathered reference before residual construction reuses the same workspace bytes. + TOperator.Encode( + blockWorkspace, + sourcePlane, + reconstructionPlane, + blockOrigin, + above, + left, + hasLeft, + hasAbove, + coefficients, + transformSize, + quantization.QIndex[0], + quantization.DeltaQDc[(int)plane], + quantization.DeltaQAc[(int)plane], + plane, + bitDepth, + ref state); + } + + /// + /// Retains the stack-only state shared by recursive partition and final-block traversal. + /// + /// The native unsigned sample storage type. + /// The type-specific block encoding operations. + private ref struct Traversal + where TSample : unmanaged + where TOperator : struct, IBlockEncodingOperator + { + private readonly Av1EncoderFrame.PlanarView source; + private readonly Av1EncoderFrame.PlanarView reconstruction; + private readonly Av1PictureControlSet picture; + private readonly Av1Superblock superblock; + private readonly Av1EncoderBlockWorkspace blockWorkspace; + private readonly ObuQuantizationParameters quantization; + private readonly Av1BitDepth bitDepth; + private readonly Span lumaCoefficients; + private readonly Span blueCoefficients; + private readonly Span redCoefficients; + private readonly Span lumaTransformBlocks; + private readonly Span blueTransformBlocks; + private readonly Span redTransformBlocks; + private int partitionIndex; + private int finalBlockIndex; + private int codedAreaLuma; + private int codedAreaChroma; + + public Traversal( + Av1EncoderFrame.PlanarView source, + Av1EncoderFrame.PlanarView reconstruction, + Av1PictureControlSet picture, + Av1Superblock superblock, + Av1EncoderCoefficientBuffer coefficientBuffer, + Av1EncoderBlockWorkspace blockWorkspace) + { + this.source = source; + this.reconstruction = reconstruction; + this.picture = picture; + this.superblock = superblock; + this.blockWorkspace = blockWorkspace; + this.quantization = picture.Parent.FrameHeader.QuantizationParameters; + this.bitDepth = picture.Sequence.SequenceHeader.ColorConfig.BitDepth; + this.lumaCoefficients = coefficientBuffer.GetPlaneSpan(superblock.Index, Av1Plane.Y); + this.blueCoefficients = coefficientBuffer.GetPlaneSpan(superblock.Index, Av1Plane.U); + this.redCoefficients = coefficientBuffer.GetPlaneSpan(superblock.Index, Av1Plane.V); + this.lumaTransformBlocks = coefficientBuffer.GetTransformBlockSpan(superblock.Index, Av1Plane.Y); + this.blueTransformBlocks = coefficientBuffer.GetTransformBlockSpan(superblock.Index, Av1Plane.U); + this.redTransformBlocks = coefficientBuffer.GetTransformBlockSpan(superblock.Index, Av1Plane.V); + this.partitionIndex = 0; + this.finalBlockIndex = 0; + this.codedAreaLuma = 0; + this.codedAreaChroma = 0; + } + + public void EncodePartitionTree(Point blockOrigin, Av1BlockSize blockSize) + { + Av1EncoderCommon common = this.picture.Parent.Common; + Point modeInfoPosition = blockOrigin >> Av1Constants.ModeInfoSizeLog2; + if (modeInfoPosition.Y >= common.ModeInfoRowCount || modeInfoPosition.X >= common.ModeInfoColumnCount) + { + return; + } + + if (blockSize == Av1BlockSize.Block8x8) + { + this.superblock.CodingUnitPartitionTypes[this.partitionIndex++] = (byte)Av1PartitionType.None; + this.EncodeFinalBlock(blockOrigin, modeInfoPosition); + return; + } + + this.superblock.CodingUnitPartitionTypes[this.partitionIndex++] = (byte)Av1PartitionType.Split; + Av1BlockSize subSize = Av1PartitionType.Split.GetBlockSubSize(blockSize); + int halfBlockSize = blockSize.GetWidth() >> 1; + + // The preorder and out-of-frame pruning match tile writing, so one reusable decision workspace is sufficient. + this.EncodePartitionTree(blockOrigin, subSize); + this.EncodePartitionTree(blockOrigin + new Size(halfBlockSize, 0), subSize); + this.EncodePartitionTree(blockOrigin + new Size(0, halfBlockSize), subSize); + this.EncodePartitionTree(blockOrigin + new Size(halfBlockSize, halfBlockSize), subSize); + } + + private void EncodeFinalBlock(Point blockOrigin, Point modeInfoPosition) + { + const Av1BlockSize BlockSize = Av1BlockSize.Block8x8; + const Av1TransformSize LumaTransformSize = Av1TransformSize.Size8x8; + int qIndex = this.quantization.QIndex[0]; + ref Av1MacroBlockModeInfo modeInfo = ref this.picture.GetMacroBlockModeInfo(modeInfoPosition); + modeInfo.Block = new Av1EncoderBlockModeInfo + { + BlockSize = BlockSize, + PartitionType = Av1PartitionType.None, + SegmentId = 0, + TransformSize = LumaTransformSize, + Mode = Av1PredictionMode.DC, + UvMode = Av1ChromaPredictionMode.DC + }; + + modeInfo.CdefStrength = 0; + ref Av1EncoderBlockStruct block = ref this.superblock.FinalBlocks[this.finalBlockIndex++]; + block.HasChroma = !this.source.IsMonochrome; + block.QuantizationIndex = qIndex; + block.SegmentId = 0; + + int lumaTransformIndex = this.codedAreaLuma / + Av1EncoderCoefficientBuffer.TransformBlockUnitCoefficientCount; + + ref Av1EncoderTransformBlockState lumaState = ref this.lumaTransformBlocks[lumaTransformIndex]; + this.EncodePlaneBlock( + Av1Plane.Y, + blockOrigin, + LumaTransformSize, + this.lumaCoefficients[this.codedAreaLuma..], + ref lumaState); + + this.codedAreaLuma += LumaTransformSize.GetSize2d(); + if (this.source.IsMonochrome) + { + return; + } + + ObuColorConfig colorConfig = this.picture.Sequence.SequenceHeader.ColorConfig; + int subsamplingX = colorConfig.SubSamplingX ? 1 : 0; + int subsamplingY = colorConfig.SubSamplingY ? 1 : 0; + Point chromaOrigin = new(blockOrigin.X >> subsamplingX, blockOrigin.Y >> subsamplingY); + Av1TransformSize chromaTransformSize = BlockSize.GetMaxUvTransformSize( + colorConfig.SubSamplingX, + colorConfig.SubSamplingY); + + int chromaTransformIndex = this.codedAreaChroma / + Av1EncoderCoefficientBuffer.TransformBlockUnitCoefficientCount; + + ref Av1EncoderTransformBlockState blueState = ref this.blueTransformBlocks[chromaTransformIndex]; + ref Av1EncoderTransformBlockState redState = ref this.redTransformBlocks[chromaTransformIndex]; + this.EncodePlaneBlock( + Av1Plane.U, + chromaOrigin, + chromaTransformSize, + this.blueCoefficients[this.codedAreaChroma..], + ref blueState); + + this.EncodePlaneBlock( + Av1Plane.V, + chromaOrigin, + chromaTransformSize, + this.redCoefficients[this.codedAreaChroma..], + ref redState); + + // Ordinary intra retains non-skip syntax even for empty transforms, matching live mode selection. + this.codedAreaChroma += chromaTransformSize.GetSize2d(); + } + + private void EncodePlaneBlock( + Av1Plane plane, + Point blockOrigin, + Av1TransformSize transformSize, + Span coefficients, + ref Av1EncoderTransformBlockState state) + => Av1IntraSuperblockEncoder.EncodePlaneBlock( + this.source, + this.reconstruction, + this.blockWorkspace, + this.quantization, + this.bitDepth, + plane, + blockOrigin, + transformSize, + coefficients, + ref state); + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1PaletteKMeans.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1PaletteKMeans.cs new file mode 100644 index 0000000000..c6d97384a9 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1PaletteKMeans.cs @@ -0,0 +1,246 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.InteropServices; +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline; + +/// +/// Assigns luma samples to AV1 palette colors and refines one-dimensional palette centroids. +/// +internal static class Av1PaletteKMeans +{ + /// + /// The iteration limit used by the reference encoder for palette clustering. + /// + public const int MaximumIterations = 50; + + /// + /// Assigns every sample to its nearest palette color. + /// + /// The active block samples. + /// The candidate palette colors. + /// The destination palette indices. + /// The sum of squared sample-to-centroid distances. + public static long AssignIndices( + ReadOnlySpan samples, + ReadOnlySpan centroids, + Span indices) + { + Span distanceScratch = stackalloc short[Vector512.Count]; + Span indexScratch = stackalloc short[Vector512.Count]; + ref short sampleBase = ref MemoryMarshal.GetReference(samples); + int offset = 0; + long distortion = 0; + + if (Vector512.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector512Count(samples); + for (; vectorCount > 0; vectorCount--, offset += Vector512.Count) + { + Vector512 sample = Vector512.LoadUnsafe(ref sampleBase, (nuint)offset); + Vector512 bestDistance = Vector512.Abs(sample - Vector512.Create(centroids[0])); + Vector512 bestIndex = Vector512.Zero; + for (int centroidIndex = 1; centroidIndex < centroids.Length; centroidIndex++) + { + Vector512 distance = Vector512.Abs(sample - Vector512.Create(centroids[centroidIndex])); + Vector512 replace = Vector512.LessThan(distance, bestDistance); + bestDistance = Vector512.ConditionalSelect(replace, distance, bestDistance); + bestIndex = Vector512.ConditionalSelect(replace, Vector512.Create((short)centroidIndex), bestIndex); + } + + // Strict comparison preserves the first centroid on ties, matching scalar AV1 palette selection. + bestDistance.CopyTo(distanceScratch); + bestIndex.CopyTo(indexScratch); + for (int lane = 0; lane < Vector512.Count; lane++) + { + indices[offset + lane] = (byte)indexScratch[lane]; + long distance = distanceScratch[lane]; + distortion += distance * distance; + } + } + } + + if (Vector256.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector256Count(samples[offset..]); + for (; vectorCount > 0; vectorCount--, offset += Vector256.Count) + { + Vector256 sample = Vector256.LoadUnsafe(ref sampleBase, (nuint)offset); + Vector256 bestDistance = Vector256.Abs(sample - Vector256.Create(centroids[0])); + Vector256 bestIndex = Vector256.Zero; + for (int centroidIndex = 1; centroidIndex < centroids.Length; centroidIndex++) + { + Vector256 distance = Vector256.Abs(sample - Vector256.Create(centroids[centroidIndex])); + Vector256 replace = Vector256.LessThan(distance, bestDistance); + bestDistance = Vector256.ConditionalSelect(replace, distance, bestDistance); + bestIndex = Vector256.ConditionalSelect(replace, Vector256.Create((short)centroidIndex), bestIndex); + } + + bestDistance.CopyTo(distanceScratch); + bestIndex.CopyTo(indexScratch); + for (int lane = 0; lane < Vector256.Count; lane++) + { + indices[offset + lane] = (byte)indexScratch[lane]; + long distance = distanceScratch[lane]; + distortion += distance * distance; + } + } + } + + if (Vector128.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector128Count(samples[offset..]); + for (; vectorCount > 0; vectorCount--, offset += Vector128.Count) + { + Vector128 sample = Vector128.LoadUnsafe(ref sampleBase, (nuint)offset); + Vector128 bestDistance = Vector128.Abs(sample - Vector128.Create(centroids[0])); + Vector128 bestIndex = Vector128.Zero; + for (int centroidIndex = 1; centroidIndex < centroids.Length; centroidIndex++) + { + Vector128 distance = Vector128.Abs(sample - Vector128.Create(centroids[centroidIndex])); + Vector128 replace = Vector128.LessThan(distance, bestDistance); + bestDistance = Vector128.ConditionalSelect(replace, distance, bestDistance); + bestIndex = Vector128.ConditionalSelect(replace, Vector128.Create((short)centroidIndex), bestIndex); + } + + bestDistance.CopyTo(distanceScratch); + bestIndex.CopyTo(indexScratch); + for (int lane = 0; lane < Vector128.Count; lane++) + { + indices[offset + lane] = (byte)indexScratch[lane]; + long distance = distanceScratch[lane]; + distortion += distance * distance; + } + } + } + + for (; offset < samples.Length; offset++) + { + int bestDistance = Math.Abs(samples[offset] - centroids[0]); + int bestIndex = 0; + for (int centroidIndex = 1; centroidIndex < centroids.Length; centroidIndex++) + { + int distance = Math.Abs(samples[offset] - centroids[centroidIndex]); + if (distance < bestDistance) + { + bestDistance = distance; + bestIndex = centroidIndex; + } + } + + indices[offset] = (byte)bestIndex; + distortion += (long)bestDistance * bestDistance; + } + + return distortion; + } + + /// + /// Refines initialized palette colors through the reference encoder's deterministic clustering sequence. + /// + /// The active block samples. + /// The initialized colors, replaced with the best refined colors. + /// The palette indices belonging to the retained colors. + /// Reusable storage for the next centroid iteration. + /// Reusable storage for the next index iteration. + /// The retained sum of squared distances. + public static long Cluster( + ReadOnlySpan samples, + Span centroids, + Span indices, + Span alternateCentroids, + Span alternateIndices) + { + alternateCentroids = alternateCentroids[..centroids.Length]; + alternateIndices = alternateIndices[..samples.Length]; + long distortion = AssignIndices(samples, centroids, indices); + bool currentIsAlternate = false; + + for (int iteration = 0; iteration < MaximumIterations; iteration++) + { + ReadOnlySpan currentCentroids = currentIsAlternate ? alternateCentroids : centroids; + ReadOnlySpan currentIndices = currentIsAlternate ? alternateIndices : indices; + Span nextCentroids = currentIsAlternate ? centroids : alternateCentroids; + Span nextIndices = currentIsAlternate ? indices : alternateIndices; + CalculateCentroids(samples, currentIndices, nextCentroids); + if (nextCentroids.SequenceEqual(currentCentroids)) + { + break; + } + + long nextDistortion = AssignIndices(samples, nextCentroids, nextIndices); + if (nextDistortion > distortion) + { + break; + } + + distortion = nextDistortion; + currentIsAlternate = !currentIsAlternate; + } + + if (currentIsAlternate) + { + alternateCentroids.CopyTo(centroids); + alternateIndices.CopyTo(indices); + } + + return distortion; + } + + /// + /// Places initial colors at the midpoint of equal intervals spanning the sample range. + /// + /// The smallest sample value. + /// The largest sample value. + /// The palette colors to initialize. + public static void InitializeCentroids(short minimum, short maximum, Span centroids) + { + int range = maximum - minimum; + for (int index = 0; index < centroids.Length; index++) + { + centroids[index] = (short)(minimum + (((2 * index) + 1) * range / centroids.Length / 2)); + } + } + + /// + /// Recalculates each centroid from its assigned samples. + /// + private static void CalculateCentroids( + ReadOnlySpan samples, + ReadOnlySpan indices, + Span centroids) + { + Span counts = stackalloc int[Av1Constants.PaletteMaxSize]; + Span sums = stackalloc int[Av1Constants.PaletteMaxSize]; + counts = counts[..centroids.Length]; + sums = sums[..centroids.Length]; + counts.Clear(); + sums.Clear(); + for (int index = 0; index < samples.Length; index++) + { + int centroidIndex = indices[index]; + counts[centroidIndex]++; + sums[centroidIndex] += samples[index]; + } + + uint randomState = (uint)samples[0]; + for (int centroidIndex = 0; centroidIndex < centroids.Length; centroidIndex++) + { + int count = counts[centroidIndex]; + if (count == 0) + { + // Empty clusters use the same seeded sequence on every platform so palette choices remain reproducible. + randomState = unchecked((randomState * 1103515245U) + 12345U); + uint random = (randomState / 65536U) % 32768U; + centroids[centroidIndex] = samples[(int)(random % (uint)samples.Length)]; + } + else + { + centroids[centroidIndex] = (short)((sums[centroidIndex] + (count / 2)) / count); + } + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1PaletteKMeans2D.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1PaletteKMeans2D.cs new file mode 100644 index 0000000000..84832d03a0 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1PaletteKMeans2D.cs @@ -0,0 +1,362 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.InteropServices; +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline; + +/// +/// Assigns paired chroma samples to AV1 palette colors and refines two-dimensional palette centroids. +/// +internal static class Av1PaletteKMeans2D +{ + /// + /// Assigns every chroma pair to its nearest palette color. + /// + /// The active first-plane samples. + /// The active second-plane samples. + /// The first-plane palette colors. + /// The second-plane palette colors. + /// The destination palette indices. + /// The sum of squared two-plane sample-to-centroid distances. + public static long AssignIndices( + ReadOnlySpan firstSamples, + ReadOnlySpan secondSamples, + ReadOnlySpan firstCentroids, + ReadOnlySpan secondCentroids, + Span indices) + { + Span distanceScratch = stackalloc int[Vector512.Count]; + Span indexScratch = stackalloc int[Vector512.Count]; + ref short firstSampleBase = ref MemoryMarshal.GetReference(firstSamples); + ref short secondSampleBase = ref MemoryMarshal.GetReference(secondSamples); + int offset = 0; + long distortion = 0; + + if (Vector512.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector512Count(firstSamples); + for (; vectorCount > 0; vectorCount--, offset += Vector512.Count) + { + Vector512 firstSample = Vector512.LoadUnsafe(ref firstSampleBase, (nuint)offset); + Vector512 secondSample = Vector512.LoadUnsafe(ref secondSampleBase, (nuint)offset); + Vector512 firstDifference = firstSample - Vector512.Create(firstCentroids[0]); + Vector512 secondDifference = secondSample - Vector512.Create(secondCentroids[0]); + (Vector512 firstLower, Vector512 firstUpper) = Vector512.Widen(firstDifference); + (Vector512 secondLower, Vector512 secondUpper) = Vector512.Widen(secondDifference); + Vector512 bestDistanceLower = (firstLower * firstLower) + (secondLower * secondLower); + Vector512 bestDistanceUpper = (firstUpper * firstUpper) + (secondUpper * secondUpper); + Vector512 bestIndexLower = Vector512.Zero; + Vector512 bestIndexUpper = Vector512.Zero; + for (int centroidIndex = 1; centroidIndex < firstCentroids.Length; centroidIndex++) + { + firstDifference = firstSample - Vector512.Create(firstCentroids[centroidIndex]); + secondDifference = secondSample - Vector512.Create(secondCentroids[centroidIndex]); + (firstLower, firstUpper) = Vector512.Widen(firstDifference); + (secondLower, secondUpper) = Vector512.Widen(secondDifference); + Vector512 distanceLower = (firstLower * firstLower) + (secondLower * secondLower); + Vector512 distanceUpper = (firstUpper * firstUpper) + (secondUpper * secondUpper); + Vector512 replaceLower = Vector512.LessThan(distanceLower, bestDistanceLower); + Vector512 replaceUpper = Vector512.LessThan(distanceUpper, bestDistanceUpper); + bestDistanceLower = Vector512.ConditionalSelect(replaceLower, distanceLower, bestDistanceLower); + bestDistanceUpper = Vector512.ConditionalSelect(replaceUpper, distanceUpper, bestDistanceUpper); + bestIndexLower = Vector512.ConditionalSelect( + replaceLower, + Vector512.Create(centroidIndex), + bestIndexLower); + + bestIndexUpper = Vector512.ConditionalSelect( + replaceUpper, + Vector512.Create(centroidIndex), + bestIndexUpper); + } + + bestDistanceLower.CopyTo(distanceScratch); + bestDistanceUpper.CopyTo(distanceScratch[Vector512.Count..]); + bestIndexLower.CopyTo(indexScratch); + bestIndexUpper.CopyTo(indexScratch[Vector512.Count..]); + for (int lane = 0; lane < Vector512.Count; lane++) + { + indices[offset + lane] = (byte)indexScratch[lane]; + distortion += distanceScratch[lane]; + } + } + } + + if (Vector256.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector256Count(firstSamples[offset..]); + for (; vectorCount > 0; vectorCount--, offset += Vector256.Count) + { + Vector256 firstSample = Vector256.LoadUnsafe(ref firstSampleBase, (nuint)offset); + Vector256 secondSample = Vector256.LoadUnsafe(ref secondSampleBase, (nuint)offset); + Vector256 firstDifference = firstSample - Vector256.Create(firstCentroids[0]); + Vector256 secondDifference = secondSample - Vector256.Create(secondCentroids[0]); + (Vector256 firstLower, Vector256 firstUpper) = Vector256.Widen(firstDifference); + (Vector256 secondLower, Vector256 secondUpper) = Vector256.Widen(secondDifference); + Vector256 bestDistanceLower = (firstLower * firstLower) + (secondLower * secondLower); + Vector256 bestDistanceUpper = (firstUpper * firstUpper) + (secondUpper * secondUpper); + Vector256 bestIndexLower = Vector256.Zero; + Vector256 bestIndexUpper = Vector256.Zero; + for (int centroidIndex = 1; centroidIndex < firstCentroids.Length; centroidIndex++) + { + firstDifference = firstSample - Vector256.Create(firstCentroids[centroidIndex]); + secondDifference = secondSample - Vector256.Create(secondCentroids[centroidIndex]); + (firstLower, firstUpper) = Vector256.Widen(firstDifference); + (secondLower, secondUpper) = Vector256.Widen(secondDifference); + Vector256 distanceLower = (firstLower * firstLower) + (secondLower * secondLower); + Vector256 distanceUpper = (firstUpper * firstUpper) + (secondUpper * secondUpper); + Vector256 replaceLower = Vector256.LessThan(distanceLower, bestDistanceLower); + Vector256 replaceUpper = Vector256.LessThan(distanceUpper, bestDistanceUpper); + bestDistanceLower = Vector256.ConditionalSelect(replaceLower, distanceLower, bestDistanceLower); + bestDistanceUpper = Vector256.ConditionalSelect(replaceUpper, distanceUpper, bestDistanceUpper); + bestIndexLower = Vector256.ConditionalSelect( + replaceLower, + Vector256.Create(centroidIndex), + bestIndexLower); + + bestIndexUpper = Vector256.ConditionalSelect( + replaceUpper, + Vector256.Create(centroidIndex), + bestIndexUpper); + } + + bestDistanceLower.CopyTo(distanceScratch); + bestDistanceUpper.CopyTo(distanceScratch[Vector256.Count..]); + bestIndexLower.CopyTo(indexScratch); + bestIndexUpper.CopyTo(indexScratch[Vector256.Count..]); + for (int lane = 0; lane < Vector256.Count; lane++) + { + indices[offset + lane] = (byte)indexScratch[lane]; + distortion += distanceScratch[lane]; + } + } + } + + if (Vector128.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector128Count(firstSamples[offset..]); + for (; vectorCount > 0; vectorCount--, offset += Vector128.Count) + { + Vector128 firstSample = Vector128.LoadUnsafe(ref firstSampleBase, (nuint)offset); + Vector128 secondSample = Vector128.LoadUnsafe(ref secondSampleBase, (nuint)offset); + Vector128 firstDifference = firstSample - Vector128.Create(firstCentroids[0]); + Vector128 secondDifference = secondSample - Vector128.Create(secondCentroids[0]); + (Vector128 firstLower, Vector128 firstUpper) = Vector128.Widen(firstDifference); + (Vector128 secondLower, Vector128 secondUpper) = Vector128.Widen(secondDifference); + Vector128 bestDistanceLower = (firstLower * firstLower) + (secondLower * secondLower); + Vector128 bestDistanceUpper = (firstUpper * firstUpper) + (secondUpper * secondUpper); + Vector128 bestIndexLower = Vector128.Zero; + Vector128 bestIndexUpper = Vector128.Zero; + for (int centroidIndex = 1; centroidIndex < firstCentroids.Length; centroidIndex++) + { + firstDifference = firstSample - Vector128.Create(firstCentroids[centroidIndex]); + secondDifference = secondSample - Vector128.Create(secondCentroids[centroidIndex]); + (firstLower, firstUpper) = Vector128.Widen(firstDifference); + (secondLower, secondUpper) = Vector128.Widen(secondDifference); + Vector128 distanceLower = (firstLower * firstLower) + (secondLower * secondLower); + Vector128 distanceUpper = (firstUpper * firstUpper) + (secondUpper * secondUpper); + Vector128 replaceLower = Vector128.LessThan(distanceLower, bestDistanceLower); + Vector128 replaceUpper = Vector128.LessThan(distanceUpper, bestDistanceUpper); + bestDistanceLower = Vector128.ConditionalSelect(replaceLower, distanceLower, bestDistanceLower); + bestDistanceUpper = Vector128.ConditionalSelect(replaceUpper, distanceUpper, bestDistanceUpper); + bestIndexLower = Vector128.ConditionalSelect( + replaceLower, + Vector128.Create(centroidIndex), + bestIndexLower); + + bestIndexUpper = Vector128.ConditionalSelect( + replaceUpper, + Vector128.Create(centroidIndex), + bestIndexUpper); + } + + bestDistanceLower.CopyTo(distanceScratch); + bestDistanceUpper.CopyTo(distanceScratch[Vector128.Count..]); + bestIndexLower.CopyTo(indexScratch); + bestIndexUpper.CopyTo(indexScratch[Vector128.Count..]); + for (int lane = 0; lane < Vector128.Count; lane++) + { + indices[offset + lane] = (byte)indexScratch[lane]; + distortion += distanceScratch[lane]; + } + } + } + + for (; offset < firstSamples.Length; offset++) + { + int firstDifference = firstSamples[offset] - firstCentroids[0]; + int secondDifference = secondSamples[offset] - secondCentroids[0]; + int bestDistance = (firstDifference * firstDifference) + (secondDifference * secondDifference); + int bestIndex = 0; + for (int centroidIndex = 1; centroidIndex < firstCentroids.Length; centroidIndex++) + { + firstDifference = firstSamples[offset] - firstCentroids[centroidIndex]; + secondDifference = secondSamples[offset] - secondCentroids[centroidIndex]; + int distance = (firstDifference * firstDifference) + (secondDifference * secondDifference); + if (distance < bestDistance) + { + bestDistance = distance; + bestIndex = centroidIndex; + } + } + + indices[offset] = (byte)bestIndex; + distortion += bestDistance; + } + + return distortion; + } + + /// + /// Refines initialized paired colors through the reference encoder's deterministic clustering sequence. + /// + /// The active first-plane samples. + /// The active second-plane samples. + /// The initialized first-plane colors. + /// The initialized second-plane colors. + /// The palette indices belonging to the retained colors. + /// Reusable storage for the next first-plane centroid iteration. + /// Reusable storage for the next second-plane centroid iteration. + /// Reusable storage for the next index iteration. + /// The retained sum of squared two-plane distances. + public static long Cluster( + ReadOnlySpan firstSamples, + ReadOnlySpan secondSamples, + Span firstCentroids, + Span secondCentroids, + Span indices, + Span alternateFirstCentroids, + Span alternateSecondCentroids, + Span alternateIndices) + { + alternateFirstCentroids = alternateFirstCentroids[..firstCentroids.Length]; + alternateSecondCentroids = alternateSecondCentroids[..secondCentroids.Length]; + alternateIndices = alternateIndices[..firstSamples.Length]; + long distortion = AssignIndices( + firstSamples, + secondSamples, + firstCentroids, + secondCentroids, + indices); + + bool currentIsAlternate = false; + for (int iteration = 0; iteration < Av1PaletteKMeans.MaximumIterations; iteration++) + { + ReadOnlySpan currentFirstCentroids = currentIsAlternate + ? alternateFirstCentroids + : firstCentroids; + + ReadOnlySpan currentSecondCentroids = currentIsAlternate + ? alternateSecondCentroids + : secondCentroids; + + ReadOnlySpan currentIndices = currentIsAlternate ? alternateIndices : indices; + Span nextFirstCentroids = currentIsAlternate ? firstCentroids : alternateFirstCentroids; + Span nextSecondCentroids = currentIsAlternate ? secondCentroids : alternateSecondCentroids; + Span nextIndices = currentIsAlternate ? indices : alternateIndices; + CalculateCentroids( + firstSamples, + secondSamples, + currentIndices, + nextFirstCentroids, + nextSecondCentroids); + + if (nextFirstCentroids.SequenceEqual(currentFirstCentroids) && + nextSecondCentroids.SequenceEqual(currentSecondCentroids)) + { + break; + } + + long nextDistortion = AssignIndices( + firstSamples, + secondSamples, + nextFirstCentroids, + nextSecondCentroids, + nextIndices); + + if (nextDistortion > distortion) + { + break; + } + + distortion = nextDistortion; + currentIsAlternate = !currentIsAlternate; + } + + if (currentIsAlternate) + { + alternateFirstCentroids.CopyTo(firstCentroids); + alternateSecondCentroids.CopyTo(secondCentroids); + alternateIndices.CopyTo(indices); + } + + return distortion; + } + + /// + /// Places paired initial colors at the midpoints of equal intervals spanning each plane's sample range. + /// + public static void InitializeCentroids( + short firstMinimum, + short firstMaximum, + short secondMinimum, + short secondMaximum, + Span firstCentroids, + Span secondCentroids) + { + int firstRange = firstMaximum - firstMinimum; + int secondRange = secondMaximum - secondMinimum; + for (int index = 0; index < firstCentroids.Length; index++) + { + firstCentroids[index] = (short)(firstMinimum + (((2 * index) + 1) * firstRange / firstCentroids.Length / 2)); + secondCentroids[index] = (short)(secondMinimum + (((2 * index) + 1) * secondRange / secondCentroids.Length / 2)); + } + } + + private static void CalculateCentroids( + ReadOnlySpan firstSamples, + ReadOnlySpan secondSamples, + ReadOnlySpan indices, + Span firstCentroids, + Span secondCentroids) + { + Span counts = stackalloc int[Av1Constants.PaletteMaxSize]; + Span firstSums = stackalloc int[Av1Constants.PaletteMaxSize]; + Span secondSums = stackalloc int[Av1Constants.PaletteMaxSize]; + counts = counts[..firstCentroids.Length]; + firstSums = firstSums[..firstCentroids.Length]; + secondSums = secondSums[..secondCentroids.Length]; + counts.Clear(); + firstSums.Clear(); + secondSums.Clear(); + for (int index = 0; index < firstSamples.Length; index++) + { + int centroidIndex = indices[index]; + counts[centroidIndex]++; + firstSums[centroidIndex] += firstSamples[index]; + secondSums[centroidIndex] += secondSamples[index]; + } + + uint randomState = (uint)firstSamples[0]; + for (int centroidIndex = 0; centroidIndex < firstCentroids.Length; centroidIndex++) + { + int count = counts[centroidIndex]; + if (count == 0) + { + // Empty paired clusters copy both components from the same deterministically selected sample. + randomState = unchecked((randomState * 1103515245U) + 12345U); + uint random = (randomState / 65536U) % 32768U; + int sampleIndex = (int)(random % (uint)firstSamples.Length); + firstCentroids[centroidIndex] = firstSamples[sampleIndex]; + secondCentroids[centroidIndex] = secondSamples[sampleIndex]; + } + else + { + firstCentroids[centroidIndex] = (short)((firstSums[centroidIndex] + (count / 2)) / count); + secondCentroids[centroidIndex] = (short)((secondSums[centroidIndex] + (count / 2)) / count); + } + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1ResidualBuilder.Operator.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1ResidualBuilder.Operator.cs new file mode 100644 index 0000000000..c418d163f9 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1ResidualBuilder.Operator.cs @@ -0,0 +1,276 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline; + +/// +/// Defines the sample-width-specific arithmetic used by . +/// +internal static partial class Av1ResidualBuilder +{ + /// + /// Defines one AV1 source-minus-prediction operation across hardware widths. + /// + /// The source and prediction sample type. + internal interface IResidualOperator + where TSample : unmanaged + { + /// + /// Subtracts eight or sixteen source and prediction samples. + /// + /// The source samples. + /// The prediction samples. + /// The upper residual lanes when the inputs contain 8-bit samples. + /// The lower residual lanes. + public static abstract Vector128 Subtract(Vector128 source, Vector128 prediction, out Vector128 upper); + + /// + /// Subtracts sixteen or thirty-two source and prediction samples. + /// + /// The source samples. + /// The prediction samples. + /// The upper residual lanes when the inputs contain 8-bit samples. + /// The lower residual lanes. + public static abstract Vector256 Subtract(Vector256 source, Vector256 prediction, out Vector256 upper); + + /// + /// Subtracts thirty-two or sixty-four source and prediction samples. + /// + /// The source samples. + /// The prediction samples. + /// The upper residual lanes when the inputs contain 8-bit samples. + /// The lower residual lanes. + public static abstract Vector512 Subtract(Vector512 source, Vector512 prediction, out Vector512 upper); + + /// + /// Subtracts one source and prediction sample. + /// + /// The source sample. + /// The prediction sample. + /// The signed residual. + public static abstract short Subtract(TSample source, TSample prediction); + + /// + /// Measures one scalar absolute sample difference. + /// + /// The source sample. + /// The prediction sample. + /// The sum of absolute differences. + public static abstract int SumAbsoluteDifferences(TSample source, TSample prediction); + + /// + /// Measures eight absolute sample differences; byte inputs occupy only the lower eight lanes. + /// + /// The eight source samples. + /// The eight prediction samples. + /// The sum of absolute differences. + public static abstract int SumAbsoluteDifferences(Vector128 source, Vector128 prediction); + + /// + /// Measures four eight-sample predictions, returning their costs in candidate order. + /// Byte inputs occupy only the lower eight lanes. + /// + /// The eight source samples. + /// The eight samples for candidate 0. + /// The eight samples for candidate 1. + /// The eight samples for candidate 2. + /// The eight samples for candidate 3. + /// Four absolute-difference sums in increasing candidate order. + public static abstract Vector128 SumFourAbsoluteDifferences( + Vector128 source, + Vector128 prediction0, + Vector128 prediction1, + Vector128 prediction2, + Vector128 prediction3); + + /// + /// Calculates one squared difference and returns its signed difference for the first moment. + /// + /// The source sample. + /// The prediction sample. + /// The signed sum of source-minus-prediction differences. + /// The sum of squared differences. + public static abstract int SumSquaredDifferences(TSample source, TSample prediction, out int sum); + + /// + /// Calculates eight squared differences and their signed sum; byte inputs occupy only the lower eight lanes. + /// + /// The eight source samples. + /// The eight prediction samples. + /// The signed sum of source-minus-prediction differences. + /// The sum of squared differences. + public static abstract int SumSquaredDifferences(Vector128 source, Vector128 prediction, out int sum); + } + + /// + /// Widens 8-bit samples before subtraction so every residual is represented without precision loss. + /// + internal readonly struct ByteOperator : IResidualOperator + { + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static int SumAbsoluteDifferences(byte source, byte prediction) => Math.Abs(Subtract(source, prediction)); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static int SumAbsoluteDifferences(Vector128 source, Vector128 prediction) + { + // Widen byte samples before subtraction; twelve-bit word samples already fit signed-short lanes. + // Eight absolute residuals sum to at most 32760, so the signed-short horizontal sum remains exact. + Vector128 difference = Vector128.WidenLower(source).AsInt16() - Vector128.WidenLower(prediction).AsInt16(); + return Vector128.Sum(Vector128.Abs(difference)); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector128 SumFourAbsoluteDifferences( + Vector128 source, + Vector128 prediction0, + Vector128 prediction1, + Vector128 prediction2, + Vector128 prediction3) + { + // Reuse the source conversion across all four candidates. Each reduction contributes one independent + // 32-bit lane, allowing the traversal to accumulate all eight rows without extracting candidate costs. + Vector128 sourceSamples = Vector128.WidenLower(source).AsInt16(); + return Vector128.Create( + (int)Vector128.Sum(Vector128.Abs(sourceSamples - Vector128.WidenLower(prediction0).AsInt16())), + (int)Vector128.Sum(Vector128.Abs(sourceSamples - Vector128.WidenLower(prediction1).AsInt16())), + (int)Vector128.Sum(Vector128.Abs(sourceSamples - Vector128.WidenLower(prediction2).AsInt16())), + (int)Vector128.Sum(Vector128.Abs(sourceSamples - Vector128.WidenLower(prediction3).AsInt16()))); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static int SumSquaredDifferences(byte source, byte prediction, out int sum) + { + sum = Subtract(source, prediction); + return sum * sum; + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static int SumSquaredDifferences(Vector128 source, Vector128 prediction, out int sum) + { + Vector128 difference = Vector128.WidenLower(source).AsInt16() - Vector128.WidenLower(prediction).AsInt16(); + sum = Vector128.Sum(difference); + + // The shared square reduction widens to int before multiplying. All eight twelve-bit squares + // fit in the returned int; no short multiplication or premature bit-depth rounding is permitted. + return (int)SumSquares(difference); + } + + /// + public static Vector128 Subtract(Vector128 source, Vector128 prediction, out Vector128 upper) + { + Vector128 lower = Vector128.WidenLower(source).AsInt16() - Vector128.WidenLower(prediction).AsInt16(); + upper = Vector128.WidenUpper(source).AsInt16() - Vector128.WidenUpper(prediction).AsInt16(); + return lower; + } + + /// + public static Vector256 Subtract(Vector256 source, Vector256 prediction, out Vector256 upper) + { + Vector256 lower = Vector256.WidenLower(source).AsInt16() - Vector256.WidenLower(prediction).AsInt16(); + upper = Vector256.WidenUpper(source).AsInt16() - Vector256.WidenUpper(prediction).AsInt16(); + return lower; + } + + /// + public static Vector512 Subtract(Vector512 source, Vector512 prediction, out Vector512 upper) + { + Vector512 lower = Vector512.WidenLower(source).AsInt16() - Vector512.WidenLower(prediction).AsInt16(); + upper = Vector512.WidenUpper(source).AsInt16() - Vector512.WidenUpper(prediction).AsInt16(); + return lower; + } + + /// + public static short Subtract(byte source, byte prediction) => (short)(source - prediction); + } + + /// + /// Subtracts high-bit-depth samples directly because AV1's 10-bit and 12-bit ranges fit signed-short lanes. + /// + internal readonly struct UInt16Operator : IResidualOperator + { + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static int SumAbsoluteDifferences(ushort source, ushort prediction) => Math.Abs(Subtract(source, prediction)); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static int SumAbsoluteDifferences(Vector128 source, Vector128 prediction) + { + // Widen byte samples before subtraction; twelve-bit word samples already fit signed-short lanes. + // Eight absolute residuals sum to at most 32760, so the signed-short horizontal sum remains exact. + Vector128 difference = source.AsInt16() - prediction.AsInt16(); + return Vector128.Sum(Vector128.Abs(difference)); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector128 SumFourAbsoluteDifferences( + Vector128 source, + Vector128 prediction0, + Vector128 prediction1, + Vector128 prediction2, + Vector128 prediction3) + { + // Reuse the source conversion across all four candidates. Each reduction contributes one independent + // 32-bit lane, allowing the traversal to accumulate all eight rows without extracting candidate costs. + Vector128 sourceSamples = source.AsInt16(); + return Vector128.Create( + (int)Vector128.Sum(Vector128.Abs(sourceSamples - prediction0.AsInt16())), + (int)Vector128.Sum(Vector128.Abs(sourceSamples - prediction1.AsInt16())), + (int)Vector128.Sum(Vector128.Abs(sourceSamples - prediction2.AsInt16())), + (int)Vector128.Sum(Vector128.Abs(sourceSamples - prediction3.AsInt16()))); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static int SumSquaredDifferences(ushort source, ushort prediction, out int sum) + { + sum = Subtract(source, prediction); + return sum * sum; + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static int SumSquaredDifferences(Vector128 source, Vector128 prediction, out int sum) + { + Vector128 difference = source.AsInt16() - prediction.AsInt16(); + sum = Vector128.Sum(difference); + + // The shared square reduction widens to int before multiplying. All eight twelve-bit squares + // fit in the returned int; no short multiplication or premature bit-depth rounding is permitted. + return (int)SumSquares(difference); + } + + /// + public static Vector128 Subtract(Vector128 source, Vector128 prediction, out Vector128 upper) + { + upper = default; + return source.AsInt16() - prediction.AsInt16(); + } + + /// + public static Vector256 Subtract(Vector256 source, Vector256 prediction, out Vector256 upper) + { + upper = default; + return source.AsInt16() - prediction.AsInt16(); + } + + /// + public static Vector512 Subtract(Vector512 source, Vector512 prediction, out Vector512 upper) + { + upper = default; + return source.AsInt16() - prediction.AsInt16(); + } + + /// + public static short Subtract(ushort source, ushort prediction) => (short)(source - prediction); + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1ResidualBuilder.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1ResidualBuilder.cs new file mode 100644 index 0000000000..3a3db74803 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1ResidualBuilder.cs @@ -0,0 +1,584 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.InteropServices; +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline; + +/// +/// Builds signed AV1 residual planes and measures sample-domain error. +/// +internal static partial class Av1ResidualBuilder +{ + /// + /// The width and height of the encoder's fixed motion-search block, in samples. + /// + private const int SearchBlockDimension = 8; + + /// + /// Calculates the sum of absolute differences for an 8x8 block. + /// + /// The source samples starting at the block origin. + /// The source row stride, in samples. + /// The prediction samples starting at the block origin. + /// The prediction row stride, in samples. + /// The sum of absolute sample differences. + public static int SumAbsoluteDifferences8x8(ReadOnlySpan source, int sourceStride, ReadOnlySpan prediction, int predictionStride) + => SumAbsoluteDifferences8x8(source, sourceStride, prediction, predictionStride); + + /// + /// Measures four horizontally adjacent 8x8 predictions against one source block. + /// + /// The source samples starting at the block origin. + /// The source row stride, in samples. + /// The first prediction, with three additional samples available at the right of each row. + /// The prediction row stride, in samples. + /// The four results in increasing horizontal-offset order. + public static void SumFourAbsoluteDifferences8x8( + ReadOnlySpan source, int sourceStride, ReadOnlySpan prediction, int predictionStride, Span sums) + => SumFourAbsoluteDifferences8x8(source, sourceStride, prediction, predictionStride, sums); + + /// + /// Calculates the signed sum and squared sum of differences for an 8x8 block. + /// + /// The source samples starting at the block origin. + /// The source row stride, in samples. + /// The prediction samples starting at the block origin. + /// The prediction row stride, in samples. + /// The signed sum of sample differences. + /// The sum of squared sample differences. + public static void GetMoments8x8( + ReadOnlySpan source, int sourceStride, ReadOnlySpan prediction, int predictionStride, out int sum, out int sumOfSquares) + => GetMoments8x8(source, sourceStride, prediction, predictionStride, out sum, out sumOfSquares); + + /// + /// Calculates the sum of absolute differences for an 8x8 block. + /// + /// The source samples starting at the block origin. + /// The source row stride, in samples. + /// The prediction samples starting at the block origin. + /// The prediction row stride, in samples. + /// The sum of absolute sample differences. + public static int SumAbsoluteDifferences8x8(ReadOnlySpan source, int sourceStride, ReadOnlySpan prediction, int predictionStride) + => SumAbsoluteDifferences8x8(source, sourceStride, prediction, predictionStride); + + /// + /// Measures four horizontally adjacent 8x8 predictions against one source block. + /// + /// The source samples starting at the block origin. + /// The source row stride, in samples. + /// The first prediction, with three additional samples available at the right of each row. + /// The prediction row stride, in samples. + /// The four results in increasing horizontal-offset order. + public static void SumFourAbsoluteDifferences8x8( + ReadOnlySpan source, int sourceStride, ReadOnlySpan prediction, int predictionStride, Span sums) + => SumFourAbsoluteDifferences8x8(source, sourceStride, prediction, predictionStride, sums); + + /// + /// Calculates the signed sum and squared sum of differences for an 8x8 block. + /// + /// The source samples starting at the block origin. + /// The source row stride, in samples. + /// The prediction samples starting at the block origin. + /// The prediction row stride, in samples. + /// The signed sum of sample differences. + /// The sum of squared sample differences. + public static void GetMoments8x8( + ReadOnlySpan source, int sourceStride, ReadOnlySpan prediction, int predictionStride, out int sum, out int sumOfSquares) + => GetMoments8x8(source, sourceStride, prediction, predictionStride, out sum, out sumOfSquares); + + /// + /// Traverses an 8x8 block while its closed operator calculates scalar or eight-sample row costs. + /// + private static int SumAbsoluteDifferences8x8( + ReadOnlySpan source, int sourceStride, ReadOnlySpan prediction, int predictionStride) + where TSample : unmanaged + where TOperator : struct, IResidualOperator + { + int sum = 0; + if (Vector128.IsHardwareAccelerated) + { + // Eight widened AV1 samples exactly fill 128 bits. Wider loads would cross the row boundary; + // byte storage is loaded as eight bytes and ushort storage as eight native-order words. + for (int row = 0; row < SearchBlockDimension; row++) + { + Vector128 sourceRow = LoadSearchRow(source[(row * sourceStride)..]); + Vector128 predictionRow = LoadSearchRow(prediction[(row * predictionStride)..]); + sum += TOperator.SumAbsoluteDifferences(sourceRow, predictionRow); + } + } + else + { + for (int row = 0; row < SearchBlockDimension; row++) + { + ReadOnlySpan sourceRow = source.Slice(row * sourceStride, SearchBlockDimension); + ReadOnlySpan predictionRow = prediction.Slice(row * predictionStride, SearchBlockDimension); + for (int column = 0; column < SearchBlockDimension; column++) + { + sum += TOperator.SumAbsoluteDifferences(sourceRow[column], predictionRow[column]); + } + } + } + + return sum; + } + + /// + /// Traverses four adjacent candidates together, retaining one source load per row or scalar sample. + /// + private static void SumFourAbsoluteDifferences8x8( + ReadOnlySpan source, int sourceStride, ReadOnlySpan prediction, int predictionStride, Span sums) + where TSample : unmanaged + where TOperator : struct, IResidualOperator + { + if (Vector128.IsHardwareAccelerated) + { + Vector128 totals = Vector128.Zero; + for (int row = 0; row < SearchBlockDimension; row++) + { + ReadOnlySpan predictionRow = prediction[(row * predictionStride)..]; + Vector128 sourceRow = LoadSearchRow(source[(row * sourceStride)..]); + + // The four prediction windows overlap, but each candidate owns one result lane. The operator + // widens the source only once and reuses it for all four independent absolute-difference sums. + totals += TOperator.SumFourAbsoluteDifferences( + sourceRow, + LoadSearchRow(predictionRow), + LoadSearchRow(predictionRow[1..]), + LoadSearchRow(predictionRow[2..]), + LoadSearchRow(predictionRow[3..])); + } + + totals.CopyTo(sums); + } + else + { + int sum0 = 0; + int sum1 = 0; + int sum2 = 0; + int sum3 = 0; + for (int row = 0; row < SearchBlockDimension; row++) + { + ReadOnlySpan sourceRow = source.Slice(row * sourceStride, SearchBlockDimension); + ReadOnlySpan predictionRow = prediction.Slice(row * predictionStride, SearchBlockDimension + 3); + for (int column = 0; column < SearchBlockDimension; column++) + { + TSample sample = sourceRow[column]; + sum0 += TOperator.SumAbsoluteDifferences(sample, predictionRow[column]); + sum1 += TOperator.SumAbsoluteDifferences(sample, predictionRow[column + 1]); + sum2 += TOperator.SumAbsoluteDifferences(sample, predictionRow[column + 2]); + sum3 += TOperator.SumAbsoluteDifferences(sample, predictionRow[column + 3]); + } + } + + sums[0] = sum0; + sums[1] = sum1; + sums[2] = sum2; + sums[3] = sum3; + } + } + + /// + /// Accumulates both residual moments in one traversal without materializing a residual buffer. + /// + private static void GetMoments8x8( + ReadOnlySpan source, int sourceStride, ReadOnlySpan prediction, int predictionStride, out int sum, out int sumOfSquares) + where TSample : unmanaged + where TOperator : struct, IResidualOperator + { + sum = 0; + sumOfSquares = 0; + + // Even 64 maximum twelve-bit residual squares fit in a signed int. Preserve the unnormalized + // moments here; the caller applies the frame's precision-dependent rounding before deriving variance. + if (Vector128.IsHardwareAccelerated) + { + for (int row = 0; row < SearchBlockDimension; row++) + { + Vector128 sourceRow = LoadSearchRow(source[(row * sourceStride)..]); + Vector128 predictionRow = LoadSearchRow(prediction[(row * predictionStride)..]); + sumOfSquares += TOperator.SumSquaredDifferences(sourceRow, predictionRow, out int rowSum); + sum += rowSum; + } + } + else + { + for (int row = 0; row < SearchBlockDimension; row++) + { + ReadOnlySpan sourceRow = source.Slice(row * sourceStride, SearchBlockDimension); + ReadOnlySpan predictionRow = prediction.Slice(row * predictionStride, SearchBlockDimension); + for (int column = 0; column < SearchBlockDimension; column++) + { + sumOfSquares += TOperator.SumSquaredDifferences(sourceRow[column], predictionRow[column], out int difference); + sum += difference; + } + } + } + } + + /// + /// Loads exactly eight native-order samples; byte rows occupy the lower half of the returned vector. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector128 LoadSearchRow(ReadOnlySpan source) + where TSample : unmanaged + { + // The closed byte/ushort instantiation removes this storage-width choice. The eight-byte load + // never consumes padding or a following row; the operator widens only its eight populated lanes. + return Vector128.Count == SearchBlockDimension + ? Vector128.Create(source) + : Vector128.Create(Vector64.Create(source), Vector64.Zero); + } + + /// + /// Subtracts an 8-bit prediction plane from its source plane. + /// + /// The source samples. + /// The source row stride. + /// The prediction samples. + /// The prediction row stride. + /// The destination residual samples. + /// The residual row stride. + /// The number of samples per row. + /// The number of rows. + public static void Subtract( + ReadOnlySpan source, + int sourceStride, + ReadOnlySpan prediction, + int predictionStride, + Span residual, + int residualStride, + int width, + int height) + => Subtract(source, sourceStride, prediction, predictionStride, residual, residualStride, width, height); + + /// + /// Subtracts a high-bit-depth prediction plane from its source plane. + /// + /// The source samples. + /// The source row stride. + /// The prediction samples. + /// The prediction row stride. + /// The destination residual samples. + /// The residual row stride. + /// The number of samples per row. + /// The number of rows. + public static void Subtract( + ReadOnlySpan source, + int sourceStride, + ReadOnlySpan prediction, + int predictionStride, + Span residual, + int residualStride, + int width, + int height) + => Subtract(source, sourceStride, prediction, predictionStride, residual, residualStride, width, height); + + /// + /// Calculates the exact squared error between strided 8-bit sample planes. + /// + /// The source samples. + /// The source row stride. + /// The prediction or reconstruction samples. + /// The prediction row stride. + /// The number of samples per row. + /// The number of rows. + /// The sum of squared sample differences. + public static long SumSquaredError( + ReadOnlySpan source, + int sourceStride, + ReadOnlySpan prediction, + int predictionStride, + int width, + int height) + => SumSquaredError(source, sourceStride, prediction, predictionStride, width, height); + + /// + /// Calculates the exact squared error between strided high-bit-depth sample planes. + /// + /// The source samples. + /// The source row stride. + /// The prediction or reconstruction samples. + /// The prediction row stride. + /// The number of samples per row. + /// The number of rows. + /// The sum of squared sample differences. + public static long SumSquaredError( + ReadOnlySpan source, + int sourceStride, + ReadOnlySpan prediction, + int predictionStride, + int width, + int height) + => SumSquaredError(source, sourceStride, prediction, predictionStride, width, height); + + /// + /// Sums the squares of a contiguous signed residual block. + /// + /// The residual samples. + /// The exact sum of squared sample differences. + public static long SumSquares(ReadOnlySpan residual) + { + ref short residualBase = ref MemoryMarshal.GetReference(residual); + long sum = 0; + int offset = 0; + + // Each short lane widens before multiplication, preserving the full 12-bit residual square. + // The accumulated scalar is 64-bit because a complete encoder block can exceed 32-bit range. + if (Vector512.IsHardwareAccelerated) + { + nuint vectorCount = residual.Vector512Count(); + + for (; vectorCount > 0; vectorCount--, offset += Vector512.Count) + { + Vector512 values = Unsafe.As>(ref Unsafe.Add(ref residualBase, offset)); + sum += SumSquares(values); + } + } + + if (Vector256.IsHardwareAccelerated) + { + nuint vectorCount = residual[offset..].Vector256Count(); + + for (; vectorCount > 0; vectorCount--, offset += Vector256.Count) + { + Vector256 values = Unsafe.As>(ref Unsafe.Add(ref residualBase, offset)); + sum += SumSquares(values); + } + } + + if (Vector128.IsHardwareAccelerated) + { + nuint vectorCount = residual[offset..].Vector128Count(); + + for (; vectorCount > 0; vectorCount--, offset += Vector128.Count) + { + Vector128 values = Unsafe.As>(ref Unsafe.Add(ref residualBase, offset)); + sum += SumSquares(values); + } + } + + for (; offset < residual.Length; offset++) + { + int value = Unsafe.Add(ref residualBase, offset); + sum += value * value; + } + + return sum; + } + + private static long SumSquaredError( + ReadOnlySpan source, + int sourceStride, + ReadOnlySpan prediction, + int predictionStride, + int width, + int height) + where TSample : unmanaged + where TOperator : struct, IResidualOperator + { + long sum = 0; + for (int y = 0; y < height; y++) + { + ReadOnlySpan sourceRow = source.Slice(y * sourceStride, width); + ReadOnlySpan predictionRow = prediction.Slice(y * predictionStride, width); + ref TSample sourceBase = ref MemoryMarshal.GetReference(sourceRow); + ref TSample predictionBase = ref MemoryMarshal.GetReference(predictionRow); + int x = 0; + + // Descending hardware widths consume every complete vector before the scalar tail. Byte + // subtraction produces two widened residual vectors; high-bit-depth subtraction produces one. + if (Vector512.IsHardwareAccelerated) + { + nuint vectorCount = sourceRow.Vector512Count(); + + for (; vectorCount > 0; vectorCount--, x += Vector512.Count) + { + Vector512 sourceVector = + Unsafe.As>(ref Unsafe.Add(ref sourceBase, x)); + + Vector512 predictionVector = + Unsafe.As>(ref Unsafe.Add(ref predictionBase, x)); + + Vector512 lower = TOperator.Subtract(sourceVector, predictionVector, out Vector512 upper); + sum += SumSquares(lower); + if (Vector512.Count != Vector512.Count) + { + sum += SumSquares(upper); + } + } + } + + if (Vector256.IsHardwareAccelerated) + { + nuint vectorCount = sourceRow[x..].Vector256Count(); + + for (; vectorCount > 0; vectorCount--, x += Vector256.Count) + { + Vector256 sourceVector = + Unsafe.As>(ref Unsafe.Add(ref sourceBase, x)); + + Vector256 predictionVector = + Unsafe.As>(ref Unsafe.Add(ref predictionBase, x)); + + Vector256 lower = TOperator.Subtract(sourceVector, predictionVector, out Vector256 upper); + sum += SumSquares(lower); + if (Vector256.Count != Vector256.Count) + { + sum += SumSquares(upper); + } + } + } + + if (Vector128.IsHardwareAccelerated) + { + nuint vectorCount = sourceRow[x..].Vector128Count(); + + for (; vectorCount > 0; vectorCount--, x += Vector128.Count) + { + Vector128 sourceVector = + Unsafe.As>(ref Unsafe.Add(ref sourceBase, x)); + + Vector128 predictionVector = + Unsafe.As>(ref Unsafe.Add(ref predictionBase, x)); + + Vector128 lower = TOperator.Subtract(sourceVector, predictionVector, out Vector128 upper); + sum += SumSquares(lower); + if (Vector128.Count != Vector128.Count) + { + sum += SumSquares(upper); + } + } + } + + for (; x < width; x++) + { + int difference = TOperator.Subtract( + Unsafe.Add(ref sourceBase, x), + Unsafe.Add(ref predictionBase, x)); + + sum += difference * difference; + } + } + + return sum; + } + + private static void Subtract( + ReadOnlySpan source, + int sourceStride, + ReadOnlySpan prediction, + int predictionStride, + Span residual, + int residualStride, + int width, + int height) + where TSample : unmanaged + where TOperator : struct, IResidualOperator + { + for (int y = 0; y < height; y++) + { + ReadOnlySpan sourceRow = source.Slice(y * sourceStride, width); + ReadOnlySpan predictionRow = prediction.Slice(y * predictionStride, width); + Span residualRow = residual.Slice(y * residualStride, width); + + ref TSample sourceBase = ref MemoryMarshal.GetReference(sourceRow); + ref TSample predictionBase = ref MemoryMarshal.GetReference(predictionRow); + ref short residualBase = ref MemoryMarshal.GetReference(residualRow); + int x = 0; + + // Each narrower tier resumes at the shared sample offset, preserving SIMD execution for the widest + // possible remainder while leaving only a sub-vector tail for scalar subtraction. + if (Vector512.IsHardwareAccelerated) + { + nuint vectorCount = sourceRow.Vector512Count(); + + for (; vectorCount > 0; vectorCount--, x += Vector512.Count) + { + Vector512 sourceVector = Unsafe.As>(ref Unsafe.Add(ref sourceBase, x)); + Vector512 predictionVector = Unsafe.As>(ref Unsafe.Add(ref predictionBase, x)); + Vector512 lower = TOperator.Subtract(sourceVector, predictionVector, out Vector512 upper); + + Unsafe.As>(ref Unsafe.Add(ref residualBase, x)) = lower; + + // Byte vectors widen into two signed-short vectors; high-bit-depth vectors retain one lane per sample. + if (Vector512.Count != Vector512.Count) + { + Unsafe.As>(ref Unsafe.Add(ref residualBase, x + Vector512.Count)) = upper; + } + } + } + + if (Vector256.IsHardwareAccelerated) + { + nuint vectorCount = sourceRow[x..].Vector256Count(); + + for (; vectorCount > 0; vectorCount--, x += Vector256.Count) + { + Vector256 sourceVector = Unsafe.As>(ref Unsafe.Add(ref sourceBase, x)); + Vector256 predictionVector = Unsafe.As>(ref Unsafe.Add(ref predictionBase, x)); + Vector256 lower = TOperator.Subtract(sourceVector, predictionVector, out Vector256 upper); + + Unsafe.As>(ref Unsafe.Add(ref residualBase, x)) = lower; + + if (Vector256.Count != Vector256.Count) + { + Unsafe.As>(ref Unsafe.Add(ref residualBase, x + Vector256.Count)) = upper; + } + } + } + + if (Vector128.IsHardwareAccelerated) + { + nuint vectorCount = sourceRow[x..].Vector128Count(); + + for (; vectorCount > 0; vectorCount--, x += Vector128.Count) + { + Vector128 sourceVector = Unsafe.As>(ref Unsafe.Add(ref sourceBase, x)); + Vector128 predictionVector = Unsafe.As>(ref Unsafe.Add(ref predictionBase, x)); + Vector128 lower = TOperator.Subtract(sourceVector, predictionVector, out Vector128 upper); + + Unsafe.As>(ref Unsafe.Add(ref residualBase, x)) = lower; + + if (Vector128.Count != Vector128.Count) + { + Unsafe.As>(ref Unsafe.Add(ref residualBase, x + Vector128.Count)) = upper; + } + } + } + + for (; x < width; x++) + { + Unsafe.Add(ref residualBase, x) = TOperator.Subtract( + Unsafe.Add(ref sourceBase, x), + Unsafe.Add(ref predictionBase, x)); + } + } + } + + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static long SumSquares(Vector128 values) + { + Vector128 lower = Vector128.WidenLower(values); + Vector128 upper = Vector128.WidenUpper(values); + return (long)Vector128.Sum(lower * lower) + Vector128.Sum(upper * upper); + } + + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static long SumSquares(Vector256 values) + { + Vector256 lower = Vector256.WidenLower(values); + Vector256 upper = Vector256.WidenUpper(values); + return (long)Vector256.Sum(lower * lower) + Vector256.Sum(upper * upper); + } + + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static long SumSquares(Vector512 values) + { + Vector512 lower = Vector512.WidenLower(values); + Vector512 upper = Vector512.WidenUpper(values); + return (long)Vector512.Sum(lower * lower) + Vector512.Sum(upper * upper); + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1ScreenContentDetector.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1ScreenContentDetector.cs new file mode 100644 index 0000000000..52683b8dca --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1ScreenContentDetector.cs @@ -0,0 +1,182 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline; + +/// +/// Detects frames that can benefit from AV1 screen-content coding tools. +/// +internal static class Av1ScreenContentDetector +{ + private const int DetectionBlockLength = 16; + private const int DetectionBlockArea = DetectionBlockLength * DetectionBlockLength; + private const int MaximumPaletteColorCount = 4; + + /// + /// Converts native source samples to the eight-bit domain used by screen-content detection. + /// + /// The native sample storage type. + private interface ISampleOperator + where TSample : unmanaged + { + /// + /// Converts one native sample to an integer without changing its precision. + /// + /// The source sample. + /// The native sample value. + public static abstract int ToInt32(TSample value); + } + + /// + /// Detects palette-friendly content in an eight-bit source frame. + /// + /// The converted source frame. + /// when palette tools should be enabled; otherwise, . + public static bool IsPaletteLikely(Av1EncoderFrame source) + { + Detect(source, out bool allowScreenContentTools, out _); + return allowScreenContentTools; + } + + /// + /// Detects palette-friendly content in a high-bit-depth source frame. + /// + /// The converted source frame. + /// when palette tools should be enabled; otherwise, . + public static bool IsPaletteLikely(Av1EncoderFrame source) + { + Detect(source, out bool allowScreenContentTools, out _); + return allowScreenContentTools; + } + + /// + /// Detects palette and intra-block-copy content in an eight-bit source frame. + /// + /// The converted source frame. + /// Receives whether palette syntax should be enabled. + /// Receives whether intra-block copy should be enabled. + public static void Detect( + Av1EncoderFrame source, + out bool allowScreenContentTools, + out bool allowIntraBlockCopy) + => Detect(source, out allowScreenContentTools, out allowIntraBlockCopy); + + /// + /// Detects palette and intra-block-copy content in a high-bit-depth source frame. + /// + /// The converted source frame. + /// Receives whether palette syntax should be enabled. + /// Receives whether intra-block copy should be enabled. + public static void Detect( + Av1EncoderFrame source, + out bool allowScreenContentTools, + out bool allowIntraBlockCopy) + => Detect(source, out allowScreenContentTools, out allowIntraBlockCopy); + + private static void Detect( + Av1EncoderFrame source, + out bool allowScreenContentTools, + out bool allowIntraBlockCopy) + where TSample : unmanaged + where TOperator : struct, ISampleOperator + { + Av1EncoderFrame.PlanarView view = source.View; + int width = source.Width; + int height = source.Height; + long frameArea = (long)width * height; + int bitDepthShift = source.LumaBitDepth - 8; + int paletteBlockCount = 0; + int intraBlockCopyBlockCount = 0; + Span seenColors = stackalloc ulong[4]; + allowScreenContentTools = false; + allowIntraBlockCopy = false; + + // Complete 16x16 blocks and the strict frame-area threshold preserve the reference detector's decision. + for (int blockRow = 0; blockRow + DetectionBlockLength <= height; blockRow += DetectionBlockLength) + { + for (int blockColumn = 0; blockColumn + DetectionBlockLength <= width; blockColumn += DetectionBlockLength) + { + seenColors.Clear(); + int colorCount = 0; + long sum = 0; + long sumOfSquares = 0; + for (int row = 0; row < DetectionBlockLength && colorCount <= MaximumPaletteColorCount; row++) + { + ReadOnlySpan samples = view + .GetLumaRowSpan(blockRow + row) + .Slice(blockColumn, DetectionBlockLength); + + // Histogram updates depend on each sample value, so a compact scalar bitset avoids gather/scatter overhead. + for (int column = 0; column < samples.Length; column++) + { + int nativeValue = TOperator.ToInt32(samples[column]); + int value = nativeValue >> bitDepthShift; + int wordIndex = value >> 6; + ulong mask = 1UL << (value & 63); + ref ulong word = ref seenColors[wordIndex]; + int centeredValue = nativeValue - (128 << bitDepthShift); + sum += centeredValue; + sumOfSquares += (long)centeredValue * centeredValue; + if ((word & mask) == 0) + { + word |= mask; + colorCount++; + if (colorCount > MaximumPaletteColorCount) + { + break; + } + } + } + } + + if (colorCount > 1 && colorCount <= MaximumPaletteColorCount) + { + paletteBlockCount++; + long normalizedSum = sum; + long normalizedSumOfSquares = sumOfSquares; + if (bitDepthShift != 0) + { + normalizedSum = RoundPowerOfTwo(sum, bitDepthShift); + normalizedSumOfSquares = RoundPowerOfTwo(sumOfSquares, bitDepthShift * 2); + } + + long variance = normalizedSumOfSquares - ((normalizedSum * normalizedSum) >> 8); + if (variance >= DetectionBlockArea / 2) + { + intraBlockCopyBlockCount++; + } + + allowScreenContentTools = (long)paletteBlockCount * DetectionBlockArea * 10 > frameArea; + allowIntraBlockCopy = allowScreenContentTools && + (long)intraBlockCopyBlockCount * DetectionBlockArea * 12 > frameArea; + + if (allowIntraBlockCopy) + { + return; + } + } + } + } + } + + private static long RoundPowerOfTwo(long value, int shift) + => (value + (1L << (shift - 1))) >> shift; + + /// + /// Preserves native eight-bit samples. + /// + private readonly struct ByteSampleOperator : ISampleOperator + { + /// + public static int ToInt32(byte value) => value; + } + + /// + /// Normalizes high-bit-depth samples to eight-bit precision. + /// + private readonly struct UShortSampleOperator : ISampleOperator + { + /// + public static int ToInt32(ushort value) => value; + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1TileEncoder.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1TileEncoder.cs new file mode 100644 index 0000000000..e2cda37de0 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1TileEncoder.cs @@ -0,0 +1,383 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1.Entropy; +using SixLabors.ImageSharp.Formats.Heif.Av1.Motion; +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline; + +/// +/// Encodes one range-coded AV1 tile payload. +/// +internal readonly struct Av1TileEncoder : IAv1TileWriter +{ + private readonly ReadOnlyMemory tileData; + private readonly Av1PictureControlSet picture; + + /// + /// Initializes a new instance of the struct for eight-bit samples. + /// + /// The symbol encoder that retains tile output through the enclosing frame write. + /// The coded source frame. + /// The reconstructed frame updated during encoding. + /// The frame coding and mode-information state. + /// The frame-owned quantized coefficient and transform state. + /// The reusable partition and final-block decision workspace. + /// The reusable block arithmetic workspace. + /// The mode-search effort in the inclusive range zero through ten. + public Av1TileEncoder( + Av1SymbolEncoder writer, + Av1EncoderFrame source, + Av1EncoderFrame reconstruction, + Av1PictureControlSet picture, + Av1EncoderCoefficientBuffer coefficientBuffer, + Av1EncoderSuperblockWorkspace superblockWorkspace, + Av1EncoderBlockWorkspace blockWorkspace, + int effort) + { + this.picture = picture; + this.tileData = Encode( + writer, + source, + reconstruction, + reconstruction, + picture, + coefficientBuffer, + new Av1EncoderTileWorkspace(picture.Parent.FrameHeader, superblockWorkspace), + blockWorkspace, + effort); + } + + /// + /// Initializes a new instance of the struct for an eight-bit inter frame. + /// + /// The symbol encoder that retains tile output through the enclosing frame write. + /// The coded source frame. + /// The reconstructed reference frame. + /// The reconstructed frame updated during encoding. + /// The frame coding and mode-information state. + /// The frame-owned quantized coefficient and transform state. + /// The reusable partition and final-block decision workspace. + /// The reusable block arithmetic workspace. + /// The mode-search effort in the inclusive range zero through ten. + public Av1TileEncoder( + Av1SymbolEncoder writer, + Av1EncoderFrame source, + Av1EncoderFrame reference, + Av1EncoderFrame reconstruction, + Av1PictureControlSet picture, + Av1EncoderCoefficientBuffer coefficientBuffer, + Av1EncoderSuperblockWorkspace superblockWorkspace, + Av1EncoderBlockWorkspace blockWorkspace, + int effort) + { + this.picture = picture; + this.tileData = Encode( + writer, + source, + reference, + reconstruction, + picture, + coefficientBuffer, + new Av1EncoderTileWorkspace(picture.Parent.FrameHeader, superblockWorkspace), + blockWorkspace, + effort); + } + + /// + /// Initializes a new instance of the struct for an eight-bit inter frame. + /// + /// The symbol encoder that retains tile output through the enclosing frame write. + /// The coded source frame. + /// The reconstructed reference frame. + /// The reconstructed frame updated during encoding. + /// The frame coding and mode-information state. + /// The frame-owned quantized coefficient and transform state. + /// The retained tile, superblock, and entropy cursor graph. + /// The reusable block arithmetic workspace. + /// The mode-search effort in the inclusive range zero through ten. + public Av1TileEncoder( + Av1SymbolEncoder writer, + Av1EncoderFrame source, + Av1EncoderFrame reference, + Av1EncoderFrame reconstruction, + Av1PictureControlSet picture, + Av1EncoderCoefficientBuffer coefficientBuffer, + Av1EncoderTileWorkspace tileWorkspace, + Av1EncoderBlockWorkspace blockWorkspace, + int effort) + { + this.picture = picture; + this.tileData = Encode( + writer, + source, + reference, + reconstruction, + picture, + coefficientBuffer, + tileWorkspace, + blockWorkspace, + effort); + } + + /// + /// Initializes a new instance of the struct for high-bit-depth samples. + /// + /// The symbol encoder that retains tile output through the enclosing frame write. + /// The coded source frame. + /// The reconstructed frame updated during encoding. + /// The frame coding and mode-information state. + /// The frame-owned quantized coefficient and transform state. + /// The reusable partition and final-block decision workspace. + /// The reusable block arithmetic workspace. + /// The mode-search effort in the inclusive range zero through ten. + public Av1TileEncoder( + Av1SymbolEncoder writer, + Av1EncoderFrame source, + Av1EncoderFrame reconstruction, + Av1PictureControlSet picture, + Av1EncoderCoefficientBuffer coefficientBuffer, + Av1EncoderSuperblockWorkspace superblockWorkspace, + Av1EncoderBlockWorkspace blockWorkspace, + int effort) + { + this.picture = picture; + this.tileData = Encode( + writer, + source, + reconstruction, + reconstruction, + picture, + coefficientBuffer, + new Av1EncoderTileWorkspace(picture.Parent.FrameHeader, superblockWorkspace), + blockWorkspace, + effort); + } + + /// + /// Initializes a new instance of the struct for a high-bit-depth inter frame. + /// + /// The symbol encoder that retains tile output through the enclosing frame write. + /// The coded source frame. + /// The reconstructed reference frame. + /// The reconstructed frame updated during encoding. + /// The frame coding and mode-information state. + /// The frame-owned quantized coefficient and transform state. + /// The reusable partition and final-block decision workspace. + /// The reusable block arithmetic workspace. + /// The mode-search effort in the inclusive range zero through ten. + public Av1TileEncoder( + Av1SymbolEncoder writer, + Av1EncoderFrame source, + Av1EncoderFrame reference, + Av1EncoderFrame reconstruction, + Av1PictureControlSet picture, + Av1EncoderCoefficientBuffer coefficientBuffer, + Av1EncoderSuperblockWorkspace superblockWorkspace, + Av1EncoderBlockWorkspace blockWorkspace, + int effort) + { + this.picture = picture; + this.tileData = Encode( + writer, + source, + reference, + reconstruction, + picture, + coefficientBuffer, + new Av1EncoderTileWorkspace(picture.Parent.FrameHeader, superblockWorkspace), + blockWorkspace, + effort); + } + + /// + /// Initializes a new instance of the struct for a high-bit-depth inter frame. + /// + /// The symbol encoder that retains tile output through the enclosing frame write. + /// The coded source frame. + /// The reconstructed reference frame. + /// The reconstructed frame updated during encoding. + /// The frame coding and mode-information state. + /// The frame-owned quantized coefficient and transform state. + /// The retained tile, superblock, and entropy cursor graph. + /// The reusable block arithmetic workspace. + /// The mode-search effort in the inclusive range zero through ten. + public Av1TileEncoder( + Av1SymbolEncoder writer, + Av1EncoderFrame source, + Av1EncoderFrame reference, + Av1EncoderFrame reconstruction, + Av1PictureControlSet picture, + Av1EncoderCoefficientBuffer coefficientBuffer, + Av1EncoderTileWorkspace tileWorkspace, + Av1EncoderBlockWorkspace blockWorkspace, + int effort) + { + this.picture = picture; + this.tileData = Encode( + writer, + source, + reference, + reconstruction, + picture, + coefficientBuffer, + tileWorkspace, + blockWorkspace, + effort); + } + + /// + public ReadOnlySpan GetTileData(int tileNum) + { + int offset = this.picture.TileDataOffsets.Span[tileNum]; + int length = this.picture.TileDataLengths.Span[tileNum]; + return this.tileData.Span.Slice(offset, length); + } + + private static ReadOnlyMemory Encode( + Av1SymbolEncoder writer, + Av1EncoderFrame source, + Av1EncoderFrame reference, + Av1EncoderFrame reconstruction, + Av1PictureControlSet picture, + Av1EncoderCoefficientBuffer coefficientBuffer, + Av1EncoderTileWorkspace tileWorkspace, + Av1EncoderBlockWorkspace blockWorkspace, + int effort) + where TSample : unmanaged + where TOperator : struct, Av1IntraSuperblockEncoder.IBlockEncodingOperator + { + ObuFrameHeader frameHeader = picture.Parent.FrameHeader; + ObuSequenceHeader sequenceHeader = picture.Sequence.SequenceHeader; + Av1TileInfo tile = tileWorkspace.Tile; + Av1Superblock superblock = tileWorkspace.Superblock; + Av1TileWriter.Av1EntropyCodingContext entropyContext = tileWorkspace.EntropyContext; + + int superblockModeInfoSize = sequenceHeader.SuperblockModeInfoSize; + int superblockShift = sequenceHeader.SuperblockSizeLog2 - Av1Constants.ModeInfoSizeLog2; + ObuTileGroupHeader tileLayout = frameHeader.TilesInfo; + Span tileDataOffsets = picture.TileDataOffsets.Span; + Span tileDataLengths = picture.TileDataLengths.Span; + if (frameHeader.AllowIntraBlockCopy) + { + // Hash the visible source once before reconstruction begins so candidate discovery never depends + // on coding order and the workspace can be reused as compact bucket links afterward. + picture.IntraBlockCopySearch.Initialize( + source.View.GetPlane(Av1Plane.Y)); + } + + int tileIndex = 0; + int tileDataEnd = 0; + for (int tileRow = 0; tileRow < tileLayout.TileRowCount; tileRow++) + { + tile.SetTileRow(tileLayout, frameHeader.ModeInfoRowCount, tileRow); + for (int tileColumn = 0; tileColumn < tileLayout.TileColumnCount; tileColumn++) + { + tile.SetTileColumn(tileLayout, frameHeader.ModeInfoColumnCount, tileColumn); + if (tileIndex > 0) + { + // Every tile begins from the same frame probabilities, while its bytes follow the preceding + // tile in the retained output allocation. + writer.Reset(tileDataEnd); + } + + Point firstModeInfoPosition = new(tile.ModeInfoColumnStart, tile.ModeInfoRowStart); + entropyContext.MacroBlockModeInfo = picture.GetMacroBlockModeInfo(firstModeInfoPosition); + for (int modeInfoRow = tile.ModeInfoRowStart; + modeInfoRow < tile.ModeInfoRowEnd; + modeInfoRow += superblockModeInfoSize) + { + for (int modeInfoColumn = tile.ModeInfoColumnStart; + modeInfoColumn < tile.ModeInfoColumnEnd; + modeInfoColumn += superblockModeInfoSize) + { + int superblockRow = modeInfoRow >> superblockShift; + int superblockColumn = modeInfoColumn >> superblockShift; + superblock.Index = (superblockRow * coefficientBuffer.SuperblockColumnCount) + superblockColumn; + entropyContext.SuperblockOrigin = new Point( + modeInfoColumn << Av1Constants.ModeInfoSizeLog2, + modeInfoRow << Av1Constants.ModeInfoSizeLog2); + + Av1IntraSuperblockEncoder.Prepare( + picture, + superblock, + entropyContext.SuperblockOrigin); + + Av1IntraSuperblockEncoder.ModeDecision blockEncoder = new( + source, + reference, + reconstruction, + picture, + superblock, + coefficientBuffer, + blockWorkspace, + effort); + + Av1TileWriter.WriteSuperblock( + picture, + entropyContext, + writer, + superblock, + coefficientBuffer, + (ushort)tileIndex, + ref blockEncoder); + } + } + + _ = writer.Exit(out int tileDataLength); + tileDataOffsets[tileIndex] = tileDataEnd; + tileDataLengths[tileIndex] = tileDataLength; + tileDataEnd += tileDataLength; + tileIndex++; + } + } + + return writer.GetOutput(tileDataEnd); + } +} + +/// +/// Retains the mutable tile, superblock, and entropy cursor graph reused by serial frame encoding. +/// +internal readonly struct Av1EncoderTileWorkspace +{ + /// + /// Initializes a new instance of the struct. + /// + /// The fixed-geometry frame header defining tile boundaries. + /// The retained superblock decision storage. + public Av1EncoderTileWorkspace( + ObuFrameHeader frameHeader, + Av1EncoderSuperblockWorkspace superblockWorkspace) + { + this.Tile = new Av1TileInfo(0, 0, frameHeader); + this.Superblock = new Av1Superblock + { + Workspace = superblockWorkspace, + TileInfo = this.Tile + }; + + this.EntropyContext = new Av1TileWriter.Av1EntropyCodingContext + { + MacroBlock = new Av1MacroBlockD { Tile = this.Tile }, + MacroBlockModeInfo = default + }; + } + + /// + /// Gets the mutable tile boundaries selected during raster traversal. + /// + public Av1TileInfo Tile { get; } + + /// + /// Gets the mutable superblock cursor connected to the retained decision workspace. + /// + public Av1Superblock Superblock { get; } + + /// + /// Gets the mutable entropy cursor shared by successive superblocks. + /// + public Av1TileWriter.Av1EntropyCodingContext EntropyContext { get; } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1TransformBlockEncoder.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1TransformBlockEncoder.cs new file mode 100644 index 0000000000..cf295115e2 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1TransformBlockEncoder.cs @@ -0,0 +1,1238 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.InteropServices; +using SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.Quantizers; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.ChromaFromLuma; +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; +using SixLabors.ImageSharp.Memory; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline; + +/// +/// Predicts, transforms, quantizes, and reconstructs finalized AV1 blocks. +/// +internal static class Av1TransformBlockEncoder +{ + /// + /// Encodes and reconstructs one eight-bit lossy DC intra block in contiguous encoder planes. + /// + /// The reusable residual, coefficient, and transform storage. + /// The coded source plane. + /// The coded reconstruction plane. + /// The block origin in plane samples. + /// The contiguous top reference samples. + /// The contiguous left reference samples. + /// Whether the left reference is available. + /// Whether the top reference is available. + /// The retained entropy-coding coefficients. + /// The selected transform dimensions. + /// The selected compound transform type. + /// The segment quantizer index. + /// The plane DC quantizer adjustment. + /// The plane AC quantizer adjustment. + /// The component plane containing the block. + /// The retained transform type and end-of-block syntax. + public static void EncodeIntraDcLossy( + Av1EncoderBlockWorkspace workspace, + Buffer2DRegion source, + Buffer2DRegion reconstruction, + Point blockOrigin, + ReadOnlySpan above, + ReadOnlySpan left, + bool hasLeft, + bool hasAbove, + Span quantizedCoefficients, + Av1TransformSize transformSize, + Av1TransformType transformType, + int qIndex, + int dcDeltaQ, + int acDeltaQ, + Av1Plane plane, + ref Av1EncoderTransformBlockState state) + { + ReadOnlySpan sourceSamples = GetPlaneSpan(source, blockOrigin); + Span reconstructionSamples = GetPlaneSpan(reconstruction, blockOrigin); + + EncodeIntraLossyContiguous( + workspace, + sourceSamples, + source.Stride, + reconstructionSamples, + reconstruction.Stride, + above, + left, + hasLeft, + hasAbove, + Av1PredictionMode.DC, + 0, + false, + false, + quantizedCoefficients, + transformSize, + transformType, + qIndex, + dcDeltaQ, + acDeltaQ, + plane, + ref state); + } + + /// + /// Encodes one eight-bit intra candidate into contiguous decision scratch. + /// + /// The reusable residual, coefficient, and transform storage. + /// The coded source plane. + /// The block origin in plane samples. + /// The contiguous candidate reconstruction. + /// The contiguous top reference samples, with prefix storage for the shared corner. + /// The contiguous left reference samples. + /// Whether the left reference is available. + /// Whether the top reference is available. + /// The intra prediction mode. + /// The signed directional-angle adjustment. + /// Whether sequence syntax enables directional edge filtering. + /// Whether a relevant neighboring block uses smooth prediction. + /// The candidate entropy-coding coefficients. + /// The selected transform dimensions. + /// The selected compound transform type. + /// The segment quantizer index. + /// The plane DC quantizer adjustment. + /// The plane AC quantizer adjustment. + /// The component plane containing the block. + /// The candidate transform type and end-of-block syntax. + /// The normalized pixel-domain distortion in AV1 transform units. + public static long EncodeIntraLossyCandidate( + Av1EncoderBlockWorkspace workspace, + Buffer2DRegion source, + Point blockOrigin, + Span reconstruction, + ReadOnlySpan above, + ReadOnlySpan left, + bool hasLeft, + bool hasAbove, + Av1PredictionMode mode, + int angleDelta, + bool enableIntraEdgeFilter, + bool smoothIntraEdges, + Span quantizedCoefficients, + Av1TransformSize transformSize, + Av1TransformType transformType, + int qIndex, + int dcDeltaQ, + int acDeltaQ, + Av1Plane plane, + ref Av1EncoderTransformBlockState state) + { + int width = transformSize.GetWidth(); + int height = transformSize.GetHeight(); + ReadOnlySpan sourceSamples = GetPlaneSpan(source, blockOrigin); + + EncodeIntraLossyContiguous( + workspace, + sourceSamples, + source.Stride, + reconstruction, + width, + above, + left, + hasLeft, + hasAbove, + mode, + angleDelta, + enableIntraEdgeFilter, + smoothIntraEdges, + quantizedCoefficients, + transformSize, + transformType, + qIndex, + dcDeltaQ, + acDeltaQ, + plane, + ref state); + + long distortion = Av1ResidualBuilder.SumSquaredError( + sourceSamples, + source.Stride, + reconstruction, + width, + width, + height); + + return distortion << 4; + } + + /// + /// Encodes one eight-bit candidate from a cached prediction and source residual. + /// + /// The reusable residual, coefficient, and transform storage. + /// The coded source plane. + /// The block origin in plane samples. + /// The contiguous prediction samples. + /// The contiguous source-minus-prediction samples. + /// The candidate reconstruction. + /// The number of reconstruction samples between rows. + /// The candidate entropy-coding coefficients. + /// The selected transform dimensions. + /// The selected compound transform type. + /// The segment quantizer index. + /// The plane DC quantizer adjustment. + /// The plane AC quantizer adjustment. + /// The component plane containing the block. + /// The candidate transform type and end-of-block syntax. + /// The normalized pixel-domain distortion in AV1 transform units. + public static long EncodePredictionLossyCandidate( + Av1EncoderBlockWorkspace workspace, + Buffer2DRegion source, + Point blockOrigin, + ReadOnlySpan prediction, + ReadOnlySpan residual, + Span reconstruction, + int reconstructionStride, + Span quantizedCoefficients, + Av1TransformSize transformSize, + Av1TransformType transformType, + int qIndex, + int dcDeltaQ, + int acDeltaQ, + Av1Plane plane, + ref Av1EncoderTransformBlockState state) + { + int width = transformSize.GetWidth(); + int height = transformSize.GetHeight(); + int sampleCount = transformSize.GetSize2d(); + ReadOnlySpan sourceSamples = GetPlaneSpan(source, blockOrigin); + + // Each transform trial overwrites reconstruction but consumes the prepared residual read-only. + // Row copies preserve a larger candidate surface without materializing a second compact block. + for (int row = 0; row < height; row++) + { + prediction.Slice(row * width, width).CopyTo(reconstruction.Slice(row * reconstructionStride, width)); + } + + EncodeLossy( + workspace, + residual[..sampleCount], + quantizedCoefficients, + transformSize, + transformType, + qIndex, + dcDeltaQ, + acDeltaQ, + Av1BitDepth.EightBit, + ref state); + + if (state.EndOfBlock > 0) + { + Av1InverseTransformer.Reconstruct8Bit( + workspace.DequantizedCoefficients, + reconstruction, + reconstructionStride, + transformSize, + state.TransformType, + (int)plane, + state.EndOfBlock, + qIndex == 0, + workspace.TransformWorkspace); + } + + long distortion = Av1ResidualBuilder.SumSquaredError( + sourceSamples, + source.Stride, + reconstruction, + reconstructionStride, + width, + height); + + return distortion << 4; + } + + /// + /// Encodes one eight-bit chroma-from-luma candidate into contiguous decision scratch. + /// + /// The reusable residual, coefficient, and transform storage. + /// The coded source plane. + /// The block origin in plane samples. + /// The contiguous candidate reconstruction. + /// The cached DC predictor sample shared by every alpha. + /// The zero-mean reconstructed-luma predictor surface. + /// The signed chroma-from-luma multiplier. + /// The candidate entropy-coding coefficients. + /// The selected chroma transform dimensions. + /// The segment quantizer index. + /// The plane DC quantizer adjustment. + /// The plane AC quantizer adjustment. + /// The component plane containing the block. + /// The candidate transform type and end-of-block syntax. + /// The normalized pixel-domain distortion in AV1 transform units. + public static long EncodeChromaFromLumaLossyCandidate( + Av1EncoderBlockWorkspace workspace, + Buffer2DRegion source, + Point blockOrigin, + Span reconstruction, + byte dc, + ReadOnlySpan lumaQ3, + int alphaQ3, + Span quantizedCoefficients, + Av1TransformSize transformSize, + int qIndex, + int dcDeltaQ, + int acDeltaQ, + Av1Plane plane, + ref Av1EncoderTransformBlockState state) + { + int width = transformSize.GetWidth(); + int height = transformSize.GetHeight(); + ReadOnlySpan sourceSamples = GetPlaneSpan(source, blockOrigin); + reconstruction[..transformSize.GetSize2d()].Fill(dc); + + // CfL adds its scaled reconstructed-luma AC contribution to the cached DC predictor before residual coding. + Av1ChromaFromLumaPredictor.Predict(lumaQ3, reconstruction, width, alphaQ3, width, height); + Av1ResidualBuilder.Subtract( + sourceSamples, + source.Stride, + reconstruction, + width, + workspace.Residual, + width, + width, + height); + + EncodeLossy( + workspace, + quantizedCoefficients, + transformSize, + Av1TransformType.DctDct, + qIndex, + dcDeltaQ, + acDeltaQ, + Av1BitDepth.EightBit, + ref state); + + if (state.EndOfBlock > 0) + { + Av1InverseTransformer.Reconstruct8Bit( + workspace.DequantizedCoefficients, + reconstruction, + width, + transformSize, + state.TransformType, + (int)plane, + state.EndOfBlock, + qIndex == 0, + workspace.TransformWorkspace); + } + + // Final distortion is measured against the samples a decoder reconstructs, not the unquantized predictor. + long distortion = Av1ResidualBuilder.SumSquaredError( + sourceSamples, + source.Stride, + reconstruction, + width, + width, + height); + + return distortion << 4; + } + + /// + /// Encodes and reconstructs one high-bit-depth lossy DC intra block in contiguous encoder planes. + /// + /// The reusable residual, coefficient, and transform storage. + /// The coded source plane. + /// The coded reconstruction plane. + /// The block origin in plane samples. + /// The contiguous top reference samples. + /// The contiguous left reference samples. + /// Whether the left reference is available. + /// Whether the top reference is available. + /// The retained entropy-coding coefficients. + /// The selected transform dimensions. + /// The selected compound transform type. + /// The segment quantizer index. + /// The plane DC quantizer adjustment. + /// The plane AC quantizer adjustment. + /// The component plane containing the block. + /// The coded sample bit depth. + /// The retained transform type and end-of-block syntax. + public static void EncodeIntraDcLossy( + Av1EncoderBlockWorkspace workspace, + Buffer2DRegion source, + Buffer2DRegion reconstruction, + Point blockOrigin, + ReadOnlySpan above, + ReadOnlySpan left, + bool hasLeft, + bool hasAbove, + Span quantizedCoefficients, + Av1TransformSize transformSize, + Av1TransformType transformType, + int qIndex, + int dcDeltaQ, + int acDeltaQ, + Av1Plane plane, + Av1BitDepth bitDepth, + ref Av1EncoderTransformBlockState state) + { + ReadOnlySpan sourceSamples = GetPlaneSpan(source, blockOrigin); + Span reconstructionSamples = GetPlaneSpan(reconstruction, blockOrigin); + + EncodeIntraLossyContiguous( + workspace, + sourceSamples, + source.Stride, + reconstructionSamples, + reconstruction.Stride, + above, + left, + hasLeft, + hasAbove, + Av1PredictionMode.DC, + 0, + false, + false, + quantizedCoefficients, + transformSize, + transformType, + qIndex, + dcDeltaQ, + acDeltaQ, + plane, + bitDepth, + ref state); + } + + /// + /// Encodes one high-bit-depth intra candidate into contiguous decision scratch. + /// + /// The reusable residual, coefficient, and transform storage. + /// The coded source plane. + /// The block origin in plane samples. + /// The contiguous candidate reconstruction. + /// The contiguous top reference samples, with prefix storage for the shared corner. + /// The contiguous left reference samples. + /// Whether the left reference is available. + /// Whether the top reference is available. + /// The intra prediction mode. + /// The signed directional-angle adjustment. + /// Whether sequence syntax enables directional edge filtering. + /// Whether a relevant neighboring block uses smooth prediction. + /// The candidate entropy-coding coefficients. + /// The selected transform dimensions. + /// The selected compound transform type. + /// The segment quantizer index. + /// The plane DC quantizer adjustment. + /// The plane AC quantizer adjustment. + /// The component plane containing the block. + /// The coded sample bit depth. + /// The candidate transform type and end-of-block syntax. + /// The normalized pixel-domain distortion in AV1 transform units. + public static long EncodeIntraLossyCandidate( + Av1EncoderBlockWorkspace workspace, + Buffer2DRegion source, + Point blockOrigin, + Span reconstruction, + ReadOnlySpan above, + ReadOnlySpan left, + bool hasLeft, + bool hasAbove, + Av1PredictionMode mode, + int angleDelta, + bool enableIntraEdgeFilter, + bool smoothIntraEdges, + Span quantizedCoefficients, + Av1TransformSize transformSize, + Av1TransformType transformType, + int qIndex, + int dcDeltaQ, + int acDeltaQ, + Av1Plane plane, + Av1BitDepth bitDepth, + ref Av1EncoderTransformBlockState state) + { + int width = transformSize.GetWidth(); + int height = transformSize.GetHeight(); + ReadOnlySpan sourceSamples = GetPlaneSpan(source, blockOrigin); + + EncodeIntraLossyContiguous( + workspace, + sourceSamples, + source.Stride, + reconstruction, + width, + above, + left, + hasLeft, + hasAbove, + mode, + angleDelta, + enableIntraEdgeFilter, + smoothIntraEdges, + quantizedCoefficients, + transformSize, + transformType, + qIndex, + dcDeltaQ, + acDeltaQ, + plane, + bitDepth, + ref state); + + long distortion = Av1ResidualBuilder.SumSquaredError( + sourceSamples, + source.Stride, + reconstruction, + width, + width, + height); + + int shift = (bitDepth.GetBitCount() - 8) * 2; + long normalizedDistortion = shift == 0 + ? distortion + : (distortion + (1L << (shift - 1))) >> shift; + + return normalizedDistortion << 4; + } + + /// + /// Encodes one high-bit-depth candidate from a cached prediction and source residual. + /// + /// The reusable residual, coefficient, and transform storage. + /// The coded source plane. + /// The block origin in plane samples. + /// The contiguous prediction samples. + /// The contiguous source-minus-prediction samples. + /// The candidate reconstruction. + /// The number of reconstruction samples between rows. + /// The candidate entropy-coding coefficients. + /// The selected transform dimensions. + /// The selected compound transform type. + /// The segment quantizer index. + /// The plane DC quantizer adjustment. + /// The plane AC quantizer adjustment. + /// The component plane containing the block. + /// The coded sample bit depth. + /// The candidate transform type and end-of-block syntax. + /// The normalized pixel-domain distortion in AV1 transform units. + public static long EncodePredictionLossyCandidate( + Av1EncoderBlockWorkspace workspace, + Buffer2DRegion source, + Point blockOrigin, + ReadOnlySpan prediction, + ReadOnlySpan residual, + Span reconstruction, + int reconstructionStride, + Span quantizedCoefficients, + Av1TransformSize transformSize, + Av1TransformType transformType, + int qIndex, + int dcDeltaQ, + int acDeltaQ, + Av1Plane plane, + Av1BitDepth bitDepth, + ref Av1EncoderTransformBlockState state) + { + int width = transformSize.GetWidth(); + int height = transformSize.GetHeight(); + int sampleCount = transformSize.GetSize2d(); + ReadOnlySpan sourceSamples = GetPlaneSpan(source, blockOrigin); + + // Each transform trial overwrites reconstruction but consumes the prepared residual read-only. + // Row copies preserve a larger candidate surface without materializing a second compact block. + for (int row = 0; row < height; row++) + { + prediction.Slice(row * width, width).CopyTo(reconstruction.Slice(row * reconstructionStride, width)); + } + + EncodeLossy( + workspace, + residual[..sampleCount], + quantizedCoefficients, + transformSize, + transformType, + qIndex, + dcDeltaQ, + acDeltaQ, + bitDepth, + ref state); + + if (state.EndOfBlock > 0) + { + Av1InverseTransformer.ReconstructHighBitDepth( + workspace.DequantizedCoefficients, + MemoryMarshal.Cast(reconstruction), + reconstructionStride, + transformSize, + state.TransformType, + (int)plane, + state.EndOfBlock, + qIndex == 0, + bitDepth, + workspace.TransformWorkspace); + } + + long distortion = Av1ResidualBuilder.SumSquaredError( + sourceSamples, + source.Stride, + reconstruction, + reconstructionStride, + width, + height); + + int shift = (bitDepth.GetBitCount() - 8) * 2; + long normalizedDistortion = shift == 0 + ? distortion + : (distortion + (1L << (shift - 1))) >> shift; + + return normalizedDistortion << 4; + } + + /// + /// Encodes one high-bit-depth chroma-from-luma candidate into contiguous decision scratch. + /// + /// The reusable residual, coefficient, and transform storage. + /// The coded source plane. + /// The block origin in plane samples. + /// The contiguous candidate reconstruction. + /// The cached DC predictor sample shared by every alpha. + /// The zero-mean reconstructed-luma predictor surface. + /// The signed chroma-from-luma multiplier. + /// The candidate entropy-coding coefficients. + /// The selected chroma transform dimensions. + /// The segment quantizer index. + /// The plane DC quantizer adjustment. + /// The plane AC quantizer adjustment. + /// The component plane containing the block. + /// The coded sample bit depth. + /// The candidate transform type and end-of-block syntax. + /// The normalized pixel-domain distortion in AV1 transform units. + public static long EncodeChromaFromLumaLossyCandidate( + Av1EncoderBlockWorkspace workspace, + Buffer2DRegion source, + Point blockOrigin, + Span reconstruction, + ushort dc, + ReadOnlySpan lumaQ3, + int alphaQ3, + Span quantizedCoefficients, + Av1TransformSize transformSize, + int qIndex, + int dcDeltaQ, + int acDeltaQ, + Av1Plane plane, + Av1BitDepth bitDepth, + ref Av1EncoderTransformBlockState state) + { + int width = transformSize.GetWidth(); + int height = transformSize.GetHeight(); + ReadOnlySpan sourceSamples = GetPlaneSpan(source, blockOrigin); + Span signedReconstruction = MemoryMarshal.Cast(reconstruction); + reconstruction[..transformSize.GetSize2d()].Fill(dc); + + Av1ChromaFromLumaPredictor.Predict( + lumaQ3, + signedReconstruction, + width, + alphaQ3, + bitDepth.GetBitCount(), + width, + height); + + Av1ResidualBuilder.Subtract( + sourceSamples, + source.Stride, + reconstruction, + width, + workspace.Residual, + width, + width, + height); + + EncodeLossy( + workspace, + quantizedCoefficients, + transformSize, + Av1TransformType.DctDct, + qIndex, + dcDeltaQ, + acDeltaQ, + bitDepth, + ref state); + + if (state.EndOfBlock > 0) + { + Av1InverseTransformer.ReconstructHighBitDepth( + workspace.DequantizedCoefficients, + signedReconstruction, + width, + transformSize, + state.TransformType, + (int)plane, + state.EndOfBlock, + qIndex == 0, + bitDepth, + workspace.TransformWorkspace); + } + + long distortion = Av1ResidualBuilder.SumSquaredError( + sourceSamples, + source.Stride, + reconstruction, + width, + width, + height); + + int shift = (bitDepth.GetBitCount() - 8) * 2; + long normalizedDistortion = shift == 0 + ? distortion + : (distortion + (1L << (shift - 1))) >> shift; + + return normalizedDistortion << 4; + } + + /// + /// Builds an eight-bit intra prediction and its compact source residual. + /// + /// The reusable prediction scratch. + /// The source samples. + /// The number of source samples between rows. + /// The prediction destination. + /// The number of prediction samples between rows. + /// The contiguous top reference samples, with prefix storage for the shared corner. + /// The contiguous left reference samples. + /// Whether the left reference is available. + /// Whether the top reference is available. + /// The intra prediction mode. + /// The signed directional-angle adjustment. + /// Whether sequence syntax enables directional edge filtering. + /// Whether a relevant neighboring block uses smooth prediction. + /// The compact source-minus-prediction destination. + /// The prediction dimensions. + public static void PrepareIntraPrediction( + Av1EncoderBlockWorkspace workspace, + ReadOnlySpan source, + int sourceStride, + Span prediction, + int predictionStride, + ReadOnlySpan above, + ReadOnlySpan left, + bool hasLeft, + bool hasAbove, + Av1PredictionMode mode, + int angleDelta, + bool enableIntraEdgeFilter, + bool smoothIntraEdges, + Span residual, + Av1TransformSize transformSize) + { + int width = transformSize.GetWidth(); + int height = transformSize.GetHeight(); + + // Prediction remains in the specialized SIMD-first kernels. This boundary only shares the prepared + // samples and residual across transform trials that differ in transform size or type. + if (mode == Av1PredictionMode.DC) + { + Av1DcIntraPredictor.Predict(hasLeft, hasAbove, prediction, predictionStride, above, left, width, height); + } + else if (mode.IsDirectional()) + { + int angle = mode.ToAngle() + (angleDelta * Av1Constants.AngleStep); + Span scratch = MemoryMarshal.AsBytes(workspace.TransformWorkspace); + int predictionLength = width * height; + Span directionalScratch = scratch[..predictionLength]; + bool upsampleAbove = false; + bool upsampleLeft = false; + if (enableIntraEdgeFilter) + { + // Mode trials share raw references. Prepare private edge copies after the directional scratch; + // this entire transform workspace is reusable once prediction and residual formation finish. + int edgeLength = Av1IntraEdgePreparation.ReferenceBufferLength; + int prefixLength = Av1IntraEdgePreparation.ReferencePrefixLength; + Span aboveStorage = scratch.Slice(predictionLength, edgeLength); + Span leftStorage = scratch.Slice(predictionLength + edgeLength, edgeLength); + aboveStorage.Fill(127); + leftStorage.Fill(129); + if (angle < 180) + { + above.CopyTo(aboveStorage[prefixLength..]); + aboveStorage[prefixLength - 1] = Unsafe.Subtract(ref MemoryMarshal.GetReference(above), 1); + } + + if (angle > 90) + { + left.CopyTo(leftStorage[prefixLength..]); + leftStorage[prefixLength - 1] = Unsafe.Subtract(ref MemoryMarshal.GetReference(left), 1); + } + + Span filteredAbove = aboveStorage[prefixLength..]; + Span filteredLeft = leftStorage[prefixLength..]; + Av1IntraEdgePreparation.Prepare( + filteredAbove, + filteredLeft, + width, + height, + angle, + hasAbove ? width : 0, + hasLeft ? height : 0, + smoothIntraEdges, + 8, + scratch.Slice(predictionLength + (2 * edgeLength), Av1IntraEdgeFilter.ScratchLength), + out upsampleAbove, + out upsampleLeft); + + above = filteredAbove; + left = filteredLeft; + } + + Av1DirectionalIntraPredictor.Predict( + prediction, + predictionStride, + transformSize, + above, + left, + upsampleAbove, + upsampleLeft, + angle, + directionalScratch); + } + else + { + Av1NonDirectionalIntraPredictorBase.GetPredictor(mode) + .Predict(prediction, predictionStride, above, left, width, height); + } + + Av1ResidualBuilder.Subtract( + source, + sourceStride, + prediction, + predictionStride, + residual, + width, + width, + height); + } + + /// + /// Builds a high-bit-depth intra prediction and its compact source residual. + /// + /// The reusable prediction scratch. + /// The source samples. + /// The number of source samples between rows. + /// The prediction destination. + /// The number of prediction samples between rows. + /// The contiguous top reference samples, with prefix storage for the shared corner. + /// The contiguous left reference samples. + /// Whether the left reference is available. + /// Whether the top reference is available. + /// The intra prediction mode. + /// The signed directional-angle adjustment. + /// Whether sequence syntax enables directional edge filtering. + /// Whether a relevant neighboring block uses smooth prediction. + /// The compact source-minus-prediction destination. + /// The prediction dimensions. + /// The coded sample bit depth. + public static void PrepareIntraPrediction( + Av1EncoderBlockWorkspace workspace, + ReadOnlySpan source, + int sourceStride, + Span prediction, + int predictionStride, + ReadOnlySpan above, + ReadOnlySpan left, + bool hasLeft, + bool hasAbove, + Av1PredictionMode mode, + int angleDelta, + bool enableIntraEdgeFilter, + bool smoothIntraEdges, + Span residual, + Av1TransformSize transformSize, + Av1BitDepth bitDepth) + { + int width = transformSize.GetWidth(); + int height = transformSize.GetHeight(); + + // Valid high-bit-depth samples remain below the sign bit, so the predictor kernels can share + // the unsigned frame storage with their signed transform-domain implementation. + Span signedPrediction = MemoryMarshal.Cast(prediction); + ReadOnlySpan signedAbove = MemoryMarshal.Cast(above); + ReadOnlySpan signedLeft = MemoryMarshal.Cast(left); + if (mode == Av1PredictionMode.DC) + { + Av1DcIntraPredictor.Predict( + hasLeft, + hasAbove, + signedPrediction, + predictionStride, + signedAbove, + signedLeft, + width, + height, + bitDepth.GetBitCount()); + } + else if (mode.IsDirectional()) + { + int angle = mode.ToAngle() + (angleDelta * Av1Constants.AngleStep); + Span scratch = MemoryMarshal.Cast(workspace.TransformWorkspace); + int predictionLength = width * height; + Span directionalScratch = scratch[..predictionLength]; + bool upsampleAbove = false; + bool upsampleLeft = false; + if (enableIntraEdgeFilter) + { + // Mode trials share raw references. Prepare private edge copies after the directional scratch; + // this entire transform workspace is reusable once prediction and residual formation finish. + int edgeLength = Av1IntraEdgePreparation.ReferenceBufferLength; + int prefixLength = Av1IntraEdgePreparation.ReferencePrefixLength; + Span aboveStorage = scratch.Slice(predictionLength, edgeLength); + Span leftStorage = scratch.Slice(predictionLength + edgeLength, edgeLength); + int midpoint = 128 << (bitDepth.GetBitCount() - 8); + aboveStorage.Fill((short)(midpoint - 1)); + leftStorage.Fill((short)(midpoint + 1)); + if (angle < 180) + { + signedAbove.CopyTo(aboveStorage[prefixLength..]); + aboveStorage[prefixLength - 1] = Unsafe.Subtract(ref MemoryMarshal.GetReference(signedAbove), 1); + } + + if (angle > 90) + { + signedLeft.CopyTo(leftStorage[prefixLength..]); + leftStorage[prefixLength - 1] = Unsafe.Subtract(ref MemoryMarshal.GetReference(signedLeft), 1); + } + + Span filteredAbove = aboveStorage[prefixLength..]; + Span filteredLeft = leftStorage[prefixLength..]; + Av1IntraEdgePreparation.Prepare( + filteredAbove, + filteredLeft, + width, + height, + angle, + hasAbove ? width : 0, + hasLeft ? height : 0, + smoothIntraEdges, + bitDepth.GetBitCount(), + scratch.Slice(predictionLength + (2 * edgeLength), Av1IntraEdgeFilter.ScratchLength), + out upsampleAbove, + out upsampleLeft); + + signedAbove = filteredAbove; + signedLeft = filteredLeft; + } + + Av1DirectionalIntraPredictor.Predict( + signedPrediction, + predictionStride, + transformSize, + signedAbove, + signedLeft, + upsampleAbove, + upsampleLeft, + angle, + directionalScratch); + } + else + { + Av1NonDirectionalIntraPredictorBase.GetPredictor(mode) + .Predict(signedPrediction, predictionStride, signedAbove, signedLeft, width, height); + } + + Av1ResidualBuilder.Subtract( + source, + sourceStride, + prediction, + predictionStride, + residual, + width, + width, + height); + } + + /// + /// Encodes and reconstructs one eight-bit lossy intra block. + /// + /// The reusable residual, coefficient, and transform storage. + /// The source samples. + /// The number of source samples between rows. + /// The reconstructed frame samples and prediction destination. + /// The number of reconstruction samples between rows. + /// The contiguous top reference samples, with prefix storage for the shared corner. + /// The contiguous left reference samples. + /// Whether the left reference is available. + /// Whether the top reference is available. + /// The intra prediction mode. + /// The signed directional-angle adjustment. + /// Whether sequence syntax enables directional edge filtering. + /// Whether a relevant neighboring block uses smooth prediction. + /// The retained entropy-coding coefficients. + /// The selected transform dimensions. + /// The selected compound transform type. + /// The segment quantizer index. + /// The plane DC quantizer adjustment. + /// The plane AC quantizer adjustment. + /// The component plane containing the block. + /// The retained transform type and end-of-block syntax. + private static void EncodeIntraLossyContiguous( + Av1EncoderBlockWorkspace workspace, + ReadOnlySpan source, + int sourceStride, + Span reconstruction, + int reconstructionStride, + ReadOnlySpan above, + ReadOnlySpan left, + bool hasLeft, + bool hasAbove, + Av1PredictionMode mode, + int angleDelta, + bool enableIntraEdgeFilter, + bool smoothIntraEdges, + Span quantizedCoefficients, + Av1TransformSize transformSize, + Av1TransformType transformType, + int qIndex, + int dcDeltaQ, + int acDeltaQ, + Av1Plane plane, + ref Av1EncoderTransformBlockState state) + { + PrepareIntraPrediction( + workspace, + source, + sourceStride, + reconstruction, + reconstructionStride, + above, + left, + hasLeft, + hasAbove, + mode, + angleDelta, + enableIntraEdgeFilter, + smoothIntraEdges, + workspace.Residual, + transformSize); + + EncodeLossy( + workspace, + quantizedCoefficients, + transformSize, + transformType, + qIndex, + dcDeltaQ, + acDeltaQ, + Av1BitDepth.EightBit, + ref state); + + if (state.EndOfBlock > 0) + { + // Reconstructing the quantized result makes later predictions use exactly the samples a decoder will reproduce. + Av1InverseTransformer.Reconstruct8Bit( + workspace.DequantizedCoefficients, + reconstruction, + reconstructionStride, + transformSize, + state.TransformType, + (int)plane, + state.EndOfBlock, + qIndex == 0, + workspace.TransformWorkspace); + } + } + + /// + /// Encodes and reconstructs one high-bit-depth lossy intra block. + /// + /// The reusable residual, coefficient, and transform storage. + /// The source samples. + /// The number of source samples between rows. + /// The reconstructed frame samples and prediction destination. + /// The number of reconstruction samples between rows. + /// The contiguous top reference samples, with prefix storage for the shared corner. + /// The contiguous left reference samples. + /// Whether the left reference is available. + /// Whether the top reference is available. + /// The intra prediction mode. + /// The signed directional-angle adjustment. + /// Whether sequence syntax enables directional edge filtering. + /// Whether a relevant neighboring block uses smooth prediction. + /// The retained entropy-coding coefficients. + /// The selected transform dimensions. + /// The selected compound transform type. + /// The segment quantizer index. + /// The plane DC quantizer adjustment. + /// The plane AC quantizer adjustment. + /// The component plane containing the block. + /// The coded sample bit depth. + /// The retained transform type and end-of-block syntax. + private static void EncodeIntraLossyContiguous( + Av1EncoderBlockWorkspace workspace, + ReadOnlySpan source, + int sourceStride, + Span reconstruction, + int reconstructionStride, + ReadOnlySpan above, + ReadOnlySpan left, + bool hasLeft, + bool hasAbove, + Av1PredictionMode mode, + int angleDelta, + bool enableIntraEdgeFilter, + bool smoothIntraEdges, + Span quantizedCoefficients, + Av1TransformSize transformSize, + Av1TransformType transformType, + int qIndex, + int dcDeltaQ, + int acDeltaQ, + Av1Plane plane, + Av1BitDepth bitDepth, + ref Av1EncoderTransformBlockState state) + { + PrepareIntraPrediction( + workspace, + source, + sourceStride, + reconstruction, + reconstructionStride, + above, + left, + hasLeft, + hasAbove, + mode, + angleDelta, + enableIntraEdgeFilter, + smoothIntraEdges, + workspace.Residual, + transformSize, + bitDepth); + + EncodeLossy( + workspace, + quantizedCoefficients, + transformSize, + transformType, + qIndex, + dcDeltaQ, + acDeltaQ, + bitDepth, + ref state); + + if (state.EndOfBlock > 0) + { + // Reconstructing the quantized result makes later predictions use exactly the samples a decoder will reproduce. + Av1InverseTransformer.ReconstructHighBitDepth( + workspace.DequantizedCoefficients, + MemoryMarshal.Cast(reconstruction), + reconstructionStride, + transformSize, + state.TransformType, + (int)plane, + state.EndOfBlock, + qIndex == 0, + bitDepth, + workspace.TransformWorkspace); + } + } + + /// + /// Applies a lossy forward transform and quantization to one residual block. + /// + /// The reusable residual, coefficient, and transform storage. + /// The retained entropy-coding coefficients. + /// The selected transform dimensions. + /// The selected compound transform type. + /// The segment quantizer index. + /// The plane DC quantizer adjustment. + /// The plane AC quantizer adjustment. + /// The coded sample bit depth. + /// The retained transform type and end-of-block syntax. + public static void EncodeLossy( + Av1EncoderBlockWorkspace workspace, + Span quantizedCoefficients, + Av1TransformSize transformSize, + Av1TransformType transformType, + int qIndex, + int dcDeltaQ, + int acDeltaQ, + Av1BitDepth bitDepth, + ref Av1EncoderTransformBlockState state) + => EncodeLossy( + workspace, + workspace.Residual, + quantizedCoefficients, + transformSize, + transformType, + qIndex, + dcDeltaQ, + acDeltaQ, + bitDepth, + ref state); + + private static void EncodeLossy( + Av1EncoderBlockWorkspace workspace, + ReadOnlySpan residual, + Span quantizedCoefficients, + Av1TransformSize transformSize, + Av1TransformType transformType, + int qIndex, + int dcDeltaQ, + int acDeltaQ, + Av1BitDepth bitDepth, + ref Av1EncoderTransformBlockState state) + { + int coefficientCount = transformSize.GetAdjusted().GetSize2d(); + Span transformed = workspace.TransformCoefficients[..coefficientCount]; + Span quantized = quantizedCoefficients[..coefficientCount]; + Span dequantized = workspace.DequantizedCoefficients[..coefficientCount]; + + if (qIndex == 0) + { + // A coded-lossless frame fixes every transform block at 4x4 and uses the reversible transform. Its + // quantizer removes only the transform's fixed scale, leaving reconstruction coefficients unchanged. + Av1ForwardTransformer.TransformLossless4x4(residual, transformed, (uint)transformSize.GetWidth()); + state.EndOfBlock = Av1ForwardQuantizer.QuantizeLossless(transformed, quantized, dequantized, bitDepth); + state.TransformType = Av1TransformType.DctDct; + return; + } + + // The forward transform reads the prepared residual without changing it, so every type candidate can + // reuse one source-minus-prediction block. Separate coefficient spans preserve each later representation. + Av1ForwardTransformer.Transform2d( + residual, + transformed, + (uint)transformSize.GetWidth(), + transformType, + transformSize, + bitDepth.GetBitCount(), + workspace.TransformWorkspace); + + state.EndOfBlock = Av1ForwardQuantizer.QuantizeLossy( + transformed, + quantized, + dequantized, + transformSize, + transformType, + qIndex, + dcDeltaQ, + acDeltaQ, + bitDepth); + + state.TransformType = transformType; + } + + public static Span GetPlaneSpan(Buffer2DRegion plane, Point blockOrigin) + where TSample : unmanaged + { + int offset = + ((plane.Bounds.Y + blockOrigin.Y) * plane.Stride) + + plane.Bounds.X + + blockOrigin.X; + + // Encoder planes wrap one contiguous frame owner, so direct segment access retains physical strides without an enumerator or row copy. + return plane.Buffer.FastMemoryGroup[0].Span[offset..]; + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/Cdef/Av1CdefDecoder.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Cdef/Av1CdefDecoder.cs new file mode 100644 index 0000000000..d5938bed6b --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Cdef/Av1CdefDecoder.cs @@ -0,0 +1,728 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Buffers; +using System.Runtime.CompilerServices; +using System.Runtime.InteropServices; +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; +using SixLabors.ImageSharp.Memory; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.Cdef; + +/// +/// Applies AV1 constrained directional enhancement filtering to a reconstructed still-image frame. +/// +internal sealed class Av1CdefDecoder +{ + /// + /// The width and height of a CDEF unit in 4x4 luma mode-information units. + /// + private const int CdefUnitModeInfoSize = 16; + + /// + /// The number of unavailable samples reserved on each source-plane edge. + /// + private const int SourceBorder = 2; + + /// + /// The maximum number of non-skipped 8x8 luma blocks in one 64x64 CDEF unit. + /// + private const int MaximumBlocksPerUnit = 8 * 8; + + /// + /// The maximum width or height of one CDEF unit in plane samples. + /// + private const int MaximumUnitPlaneSize = CdefUnitModeInfoSize << Av1Constants.ModeInfoSizeLog2; + + /// + /// The stride of the reusable bordered CDEF source unit. + /// + private const int SourceStride = MaximumUnitPlaneSize + (SourceBorder * 2); + + /// + /// The sample count of the reusable bordered CDEF source unit. + /// + private const int SourceBufferLength = SourceStride * (MaximumUnitPlaneSize + (SourceBorder * 2)); + + /// + /// The sequence-level superblock, bit-depth, and color configuration. + /// + private readonly ObuSequenceHeader sequenceHeader; + + /// + /// The frame dimensions and CDEF strength table. + /// + private readonly ObuFrameHeader frameHeader; + + /// + /// The decoded block skip state and CDEF-unit strength selections. + /// + private readonly Av1FrameInfo frameInfo; + + /// + /// The reconstructed plane samples modified by CDEF. + /// + private readonly Av1FrameBuffer frameBuffer; + + /// + /// Initializes a new instance of the class. + /// + /// The sequence header defining CDEF availability and the color layout. + /// The frame header defining dimensions and CDEF strengths. + /// The decoded block skip state and per-unit strength selections. + /// The deblocked frame samples to filter. + public Av1CdefDecoder( + ObuSequenceHeader sequenceHeader, + ObuFrameHeader frameHeader, + Av1FrameInfo frameInfo, + Av1FrameBuffer frameBuffer) + { + this.sequenceHeader = sequenceHeader; + this.frameHeader = frameHeader; + this.frameInfo = frameInfo; + this.frameBuffer = frameBuffer; + } + + /// + /// Filters every enabled color plane using directions derived from the deblocked luma plane. + /// + public void DecodeFrame() + { + if (!this.sequenceHeader.EnableCdef || this.frameHeader.CodedLossless || this.frameHeader.AllowIntraBlockCopy) + { + return; + } + + ObuConstraintDirectionalEnhancementFilterParameters parameters = this.frameHeader.CdefParameters; + int strengthCount = 1 << parameters.BitCount; + bool hasNonZeroStrength = false; + for (int i = 0; i < strengthCount; i++) + { + if (parameters.YStrength[i] != 0 || + (this.sequenceHeader.ColorConfig.PlaneCount > 1 && parameters.UvStrength[i] != 0)) + { + hasNonZeroStrength = true; + break; + } + } + + if (!hasNonZeroStrength) + { + return; + } + + ObuColorConfig colorConfig = this.sequenceHeader.ColorConfig; + int planeCount = colorConfig.PlaneCount; + Span subsamplingX = stackalloc int[3]; + Span subsamplingY = stackalloc int[3]; + Span planeWidths = stackalloc int[3]; + Span lineBufferOffsets = stackalloc int[3]; + Span columnBufferOffsets = stackalloc int[3]; + Span columnBufferLengths = stackalloc int[3]; + int lineBufferLength = 0; + int columnBufferLength = 0; + + for (int planeIndex = 0; planeIndex < planeCount; planeIndex++) + { + Av1Plane plane = (Av1Plane)planeIndex; + int planeSubsamplingX = plane != Av1Plane.Y && colorConfig.SubSamplingX ? 1 : 0; + int planeSubsamplingY = plane != Av1Plane.Y && colorConfig.SubSamplingY ? 1 : 0; + int planeWidth = this.frameHeader.ModeInfoColumnCount << (Av1Constants.ModeInfoSizeLog2 - planeSubsamplingX); + int maximumUnitHeight = MaximumUnitPlaneSize >> planeSubsamplingY; + + subsamplingX[planeIndex] = planeSubsamplingX; + subsamplingY[planeIndex] = planeSubsamplingY; + planeWidths[planeIndex] = planeWidth; + lineBufferOffsets[planeIndex] = lineBufferLength; + columnBufferOffsets[planeIndex] = columnBufferLength; + columnBufferLengths[planeIndex] = (maximumUnitHeight + (SourceBorder * 2)) * SourceBorder; + lineBufferLength += planeWidth * SourceBorder * 2; + columnBufferLength += columnBufferLengths[planeIndex]; + } + + int directionStorageLength = MaximumBlocksPerUnit * sizeof(int) / sizeof(ushort); + int blockStorageLength = MaximumBlocksPerUnit * Unsafe.SizeOf() / sizeof(ushort); + int unitStorageOffset = SourceBufferLength + lineBufferLength + columnBufferLength; + int scratchLength = unitStorageOffset + (directionStorageLength * 2) + blockStorageLength; + MemoryAllocator allocator = this.frameBuffer.MemoryAllocator; + using IMemoryOwner scratchOwner = allocator.Allocate(scratchLength); + Span scratch = scratchOwner.Memory.Span[..scratchLength]; + Span source = scratch[..SourceBufferLength]; + Span lineBuffer = scratch.Slice(SourceBufferLength, lineBufferLength); + Span columnBuffer = scratch.Slice(SourceBufferLength + lineBufferLength, columnBufferLength); + + // Every preceding plane region has an even ushort length, so the appended unit state remains 32-bit aligned. + // Directions, variances, and block coordinates share the owner because they are reused one unit at a time. + Span directions = MemoryMarshal.Cast( + scratch.Slice(unitStorageOffset, directionStorageLength)); + + Span variances = MemoryMarshal.Cast( + scratch.Slice(unitStorageOffset + directionStorageLength, directionStorageLength)); + + Span blocks = MemoryMarshal.Cast( + scratch.Slice(unitStorageOffset + (directionStorageLength * 2), blockStorageLength)); + + Span cdefLeft = stackalloc bool[3]; + int unitColumnCount = (this.frameHeader.ModeInfoColumnCount + CdefUnitModeInfoSize - 1) / CdefUnitModeInfoSize; + int unitRowCount = (this.frameHeader.ModeInfoRowCount + CdefUnitModeInfoSize - 1) / CdefUnitModeInfoSize; + + // libaom traverses one 64x64 unit at a time so chroma consumes the luma directions before + // the fixed direction arrays are reused. This also bounds direction storage to 64 entries. + for (int unitRow = 0; unitRow < unitRowCount; unitRow++) + { + cdefLeft.Clear(); + int unitModeInfoRow = unitRow * CdefUnitModeInfoSize; + + // Preserve the final two unfiltered rows before this unit row is modified. The alternating + // slots keep the previous row available while the next row's top border is captured. + if (unitRow < unitRowCount - 1) + { + for (int planeIndex = 0; planeIndex < planeCount; planeIndex++) + { + Av1Plane plane = (Av1Plane)planeIndex; + int planeSubsamplingX = subsamplingX[planeIndex]; + int planeSubsamplingY = subsamplingY[planeIndex]; + int planeWidth = planeWidths[planeIndex]; + int nextPlaneRow = ((unitModeInfoRow + CdefUnitModeInfoSize) << Av1Constants.ModeInfoSizeLog2) >> planeSubsamplingY; + int lineSlotOffset = lineBufferOffsets[planeIndex] + + ((unitRow & 1) * SourceBorder * planeWidth); + + this.GetPlaneDestination( + plane, + planeSubsamplingX, + planeSubsamplingY, + out Span lowBitDepthDestination, + out Span highBitDepthDestination, + out int destinationStride); + + this.CopyFrameRectangle( + lowBitDepthDestination, + highBitDepthDestination, + destinationStride + ((nextPlaneRow - SourceBorder) * destinationStride), + destinationStride, + lineBuffer, + lineSlotOffset, + planeWidth, + planeWidth, + SourceBorder); + } + } + + for (int unitColumn = 0; unitColumn < unitColumnCount; unitColumn++) + { + int unitModeInfoColumn = unitColumn * CdefUnitModeInfoSize; + int strengthIndex = this.GetStrengthIndex(unitModeInfoColumn, unitModeInfoRow); + if (strengthIndex < 0) + { + cdefLeft.Clear(); + continue; + } + + int yStrength = parameters.YStrength[strengthIndex]; + int uvStrength = parameters.UvStrength[strengthIndex]; + bool unitNeedsDirections = yStrength != 0 || (planeCount > 1 && uvStrength != 0); + if (!unitNeedsDirections) + { + cdefLeft.Clear(); + continue; + } + + int unitModeInfoRowEnd = Math.Min(unitModeInfoRow + CdefUnitModeInfoSize, this.frameHeader.ModeInfoRowCount); + int unitModeInfoColumnEnd = Math.Min(unitModeInfoColumn + CdefUnitModeInfoSize, this.frameHeader.ModeInfoColumnCount); + int blockCount = 0; + + for (int blockModeInfoRow = unitModeInfoRow; blockModeInfoRow < unitModeInfoRowEnd; blockModeInfoRow += 2) + { + for (int blockModeInfoColumn = unitModeInfoColumn; blockModeInfoColumn < unitModeInfoColumnEnd; blockModeInfoColumn += 2) + { + if (this.IsBlockSkipped(blockModeInfoColumn, blockModeInfoRow)) + { + continue; + } + + blocks[blockCount++] = new CdefBlock(blockModeInfoColumn, blockModeInfoRow); + } + } + + if (blockCount == 0) + { + cdefLeft.Clear(); + continue; + } + + // Luma is always prepared first when either plane type needs CDEF because it owns + // the direction search. Chroma then reuses those per-unit results without a frame map. + for (int planeIndex = 0; planeIndex < planeCount; planeIndex++) + { + if (planeIndex != (int)Av1Plane.Y && uvStrength == 0) + { + cdefLeft[planeIndex] = false; + continue; + } + + int planeWidth = planeWidths[planeIndex]; + int currentLineSlotOffset = lineBufferOffsets[planeIndex] + + (((unitRow - 1) & 1) * SourceBorder * planeWidth); + ReadOnlySpan topLineBuffer = unitRow == 0 + ? default + : lineBuffer.Slice(currentLineSlotOffset, SourceBorder * planeWidth); + Span planeColumnBuffer = columnBuffer.Slice( + columnBufferOffsets[planeIndex], + columnBufferLengths[planeIndex]); + + this.FilterPlane( + (Av1Plane)planeIndex, + subsamplingX[planeIndex], + subsamplingY[planeIndex], + unitModeInfoColumn, + unitModeInfoRow, + blocks[..blockCount], + directions, + variances, + yStrength, + uvStrength, + source, + topLineBuffer, + planeColumnBuffer, + cdefLeft[planeIndex]); + + cdefLeft[planeIndex] = true; + } + } + } + } + + /// + /// Filters one color plane in a CDEF unit from a bounded immutable source snapshot. + /// + /// The color plane to filter. + /// The horizontal chroma subsampling shift. + /// The vertical chroma subsampling shift. + /// The unit's frame-relative column in 4x4 luma units. + /// The unit's frame-relative row in 4x4 luma units. + /// The unit's non-skipped 8x8 luma blocks. + /// The unit-local luma directions in block-list order. + /// The unit-local luma directional variances in block-list order. + /// The coded luma strength. + /// The coded chroma strength. + /// The reusable bordered source-unit buffer. + /// The two preserved unfiltered rows immediately above this unit row. + /// The preserved unfiltered columns immediately left of this unit. + /// Whether the preceding unit overwrote samples needed by this unit. + private void FilterPlane( + Av1Plane plane, + int subsamplingX, + int subsamplingY, + int unitModeInfoColumn, + int unitModeInfoRow, + ReadOnlySpan blocks, + Span directions, + Span variances, + int yStrength, + int uvStrength, + Span source, + ReadOnlySpan topLineBuffer, + Span columnBuffer, + bool leftPrepared) + { + int planeWidth = this.frameHeader.ModeInfoColumnCount << (Av1Constants.ModeInfoSizeLog2 - subsamplingX); + int planeHeight = this.frameHeader.ModeInfoRowCount << (Av1Constants.ModeInfoSizeLog2 - subsamplingY); + int planeColumn = (unitModeInfoColumn << Av1Constants.ModeInfoSizeLog2) >> subsamplingX; + int planeRow = (unitModeInfoRow << Av1Constants.ModeInfoSizeLog2) >> subsamplingY; + int unitWidth = Math.Min(MaximumUnitPlaneSize >> subsamplingX, planeWidth - planeColumn); + int unitHeight = Math.Min(MaximumUnitPlaneSize >> subsamplingY, planeHeight - planeRow); + bool hasLeft = planeColumn > 0; + bool hasRight = planeColumn + unitWidth < planeWidth; + bool hasTop = planeRow > 0; + bool hasBottom = planeRow + unitHeight < planeHeight; + int leftSampleCount = hasLeft ? SourceBorder : 0; + int rightSampleCount = hasRight ? SourceBorder : 0; + int copyColumn = planeColumn - leftSampleCount; + int copyWidth = leftSampleCount + unitWidth + rightSampleCount; + int sourceColumn = SourceBorder - leftSampleCount; + + // CDEF output must never become input to a later unit. libaom therefore reconstructs a + // bordered unit from saved top/left samples and still-unmodified frame samples. Filling first + // also gives every unavailable frame-edge tap the normative CDEF_VERY_LARGE sentinel. + source.Fill(Av1CdefFilter.VeryLarge); + + this.GetPlaneDestination( + plane, + subsamplingX, + subsamplingY, + out Span lowBitDepthDestination, + out Span highBitDepthDestination, + out int destinationStride); + + if (hasTop) + { + Av1CdefFilter.CopyPlane( + topLineBuffer, + copyColumn, + planeWidth, + source, + sourceColumn, + SourceStride, + copyWidth, + SourceBorder); + } + + this.CopyFrameRectangle( + lowBitDepthDestination, + highBitDepthDestination, + destinationStride + (planeRow * destinationStride) + copyColumn, + destinationStride, + source, + (SourceBorder * SourceStride) + sourceColumn, + SourceStride, + copyWidth, + unitHeight); + + if (hasBottom) + { + this.CopyFrameRectangle( + lowBitDepthDestination, + highBitDepthDestination, + destinationStride + ((planeRow + unitHeight) * destinationStride) + copyColumn, + destinationStride, + source, + ((SourceBorder + unitHeight) * SourceStride) + sourceColumn, + SourceStride, + copyWidth, + SourceBorder); + } + + int preservedHeight = SourceBorder + unitHeight + (hasBottom ? SourceBorder : 0); + if (leftPrepared) + { + Av1CdefFilter.CopyPlane( + columnBuffer, + 0, + SourceBorder, + source, + 0, + SourceStride, + SourceBorder, + preservedHeight); + } + + // Save the final unfiltered columns before this unit writes its destination. The next unit + // restores them over the frame samples that this unit has already replaced. + Av1CdefFilter.CopyPlane( + source, + unitWidth, + SourceStride, + columnBuffer, + 0, + SourceBorder, + SourceBorder, + preservedHeight); + + ObuConstraintDirectionalEnhancementFilterParameters parameters = this.frameHeader.CdefParameters; + int coefficientShift = Math.Max(this.frameBuffer.BitDepth.GetBitCount() - 8, 0); + int blockWidth = 8 >> subsamplingX; + int blockHeight = 8 >> subsamplingY; + int codedStrength = plane == Av1Plane.Y ? yStrength : uvStrength; + int primaryStrength = (codedStrength / 4) << coefficientShift; + int secondaryStrength = codedStrength % 4; + + // The two-bit secondary field leaves value three unused and represents strength four instead. + secondaryStrength += secondaryStrength == 3 ? 1 : 0; + secondaryStrength <<= coefficientShift; + int damping = parameters.Damping + coefficientShift - (plane == Av1Plane.Y ? 0 : 1); + + if (plane == Av1Plane.Y) + { + int blockIndex = 0; + + // The reference decoder analyzes two listed 8x8 blocks together. The per-unit fixed list preserves that traversal + // without allocating a managed block list or repeating four skip-map lookups during filtering. + for (; blockIndex < blocks.Length - 1; blockIndex += 2) + { + CdefBlock firstBlock = blocks[blockIndex]; + CdefBlock secondBlock = blocks[blockIndex + 1]; + + Av1CdefFilter.FindDirections( + source, + firstBlock.GetSourceOffset(SourceStride, SourceBorder, unitModeInfoColumn, unitModeInfoRow, 0, 0), + secondBlock.GetSourceOffset(SourceStride, SourceBorder, unitModeInfoColumn, unitModeInfoRow, 0, 0), + SourceStride, + coefficientShift, + out directions[blockIndex], + out variances[blockIndex], + out directions[blockIndex + 1], + out variances[blockIndex + 1]); + } + + if (blockIndex < blocks.Length) + { + CdefBlock block = blocks[blockIndex]; + + directions[blockIndex] = Av1CdefFilter.FindDirection( + source, + block.GetSourceOffset(SourceStride, SourceBorder, unitModeInfoColumn, unitModeInfoRow, 0, 0), + SourceStride, + coefficientShift, + out variances[blockIndex]); + } + } + + if (codedStrength == 0) + { + return; + } + + for (int blockIndex = 0; blockIndex < blocks.Length; blockIndex++) + { + CdefBlock block = blocks[blockIndex]; + int filteredPrimaryStrength = plane == Av1Plane.Y + ? Av1CdefFilter.AdjustStrength(primaryStrength, variances[blockIndex]) + : primaryStrength; + + if (filteredPrimaryStrength == 0 && secondaryStrength == 0) + { + continue; + } + + // Secondary-only filtering uses direction zero; otherwise chroma remaps the + // luma direction into its asymmetrically subsampled sample grid when required. + int direction = primaryStrength != 0 + ? Av1CdefFilter.ConvertDirection(directions[blockIndex], subsamplingX, subsamplingY) + : 0; + int blockPlaneColumn = (block.ModeInfoColumn << Av1Constants.ModeInfoSizeLog2) >> subsamplingX; + int blockPlaneRow = (block.ModeInfoRow << Av1Constants.ModeInfoSizeLog2) >> subsamplingY; + int blockSourceOffset = block.GetSourceOffset( + SourceStride, + SourceBorder, + unitModeInfoColumn, + unitModeInfoRow, + subsamplingX, + subsamplingY); + + int blockDestinationOffset = destinationStride + (blockPlaneRow * destinationStride) + blockPlaneColumn; + + if (this.frameBuffer.BytesPerSample == 2) + { + Av1CdefFilter.FilterBlock( + source, + blockSourceOffset, + SourceStride, + highBitDepthDestination, + blockDestinationOffset, + destinationStride, + filteredPrimaryStrength, + secondaryStrength, + direction, + damping, + damping, + coefficientShift, + blockWidth, + blockHeight); + } + else + { + Av1CdefFilter.FilterBlock( + source, + blockSourceOffset, + SourceStride, + lowBitDepthDestination, + blockDestinationOffset, + destinationStride, + filteredPrimaryStrength, + secondaryStrength, + direction, + damping, + damping, + coefficientShift, + blockWidth, + blockHeight); + } + } + } + + /// + /// Gets the byte or native 16-bit destination span for one frame plane. + /// + /// The color plane. + /// The horizontal chroma subsampling shift. + /// The vertical chroma subsampling shift. + /// Receives the byte destination for an eight-bit frame. + /// Receives the native destination for a high-bit-depth frame. + /// Receives the number of samples between adjacent rows. + private void GetPlaneDestination( + Av1Plane plane, + int subsamplingX, + int subsamplingY, + out Span lowBitDepthDestination, + out Span highBitDepthDestination, + out int destinationStride) + { + lowBitDepthDestination = default; + highBitDepthDestination = default; + + if (this.frameBuffer.BytesPerSample == 2) + { + Span signedDestination = this.frameBuffer.DeriveBlockPointer16( + plane, + Point.Empty, + subsamplingX, + subsamplingY, + out destinationStride); + + highBitDepthDestination = MemoryMarshal.Cast(signedDestination); + } + else + { + lowBitDepthDestination = this.frameBuffer.DeriveBlockPointer( + plane, + Point.Empty, + subsamplingX, + subsamplingY, + out destinationStride); + } + } + + /// + /// Copies one frame rectangle into 16-bit CDEF working storage. + /// + /// The byte source for an eight-bit frame. + /// The native source for a high-bit-depth frame. + /// The offset of the rectangle's top-left source sample. + /// The number of samples between adjacent source rows. + /// The 16-bit working destination. + /// The offset of the rectangle's top-left destination sample. + /// The number of samples between adjacent destination rows. + /// The rectangle width in samples. + /// The rectangle height in samples. + private void CopyFrameRectangle( + ReadOnlySpan lowBitDepthSource, + ReadOnlySpan highBitDepthSource, + int sourceOffset, + int sourceStride, + Span destination, + int destinationOffset, + int destinationStride, + int width, + int height) + { + if (this.frameBuffer.BytesPerSample == 2) + { + Av1CdefFilter.CopyPlane( + highBitDepthSource, + sourceOffset, + sourceStride, + destination, + destinationOffset, + destinationStride, + width, + height); + } + else + { + Av1CdefFilter.CopyPlane( + lowBitDepthSource, + sourceOffset, + sourceStride, + destination, + destinationOffset, + destinationStride, + width, + height); + } + } + + /// + /// Gets the strength-table selection assigned to a 64x64 CDEF unit. + /// + /// The unit's frame-relative column in 4x4 luma units. + /// The unit's frame-relative row in 4x4 luma units. + /// The strength-table index, or minus one when every block in the unit is skipped. + private int GetStrengthIndex(int modeInfoColumn, int modeInfoRow) + { + int superblockModeInfoSize = this.frameInfo.SuperblockModeInfoSize; + Point superblockPosition = new( + modeInfoColumn / superblockModeInfoSize, + modeInfoRow / superblockModeInfoSize); + + int unitColumn = (modeInfoColumn % superblockModeInfoSize) / CdefUnitModeInfoSize; + int unitRow = (modeInfoRow % superblockModeInfoSize) / CdefUnitModeInfoSize; + + // A 128x128 superblock stores four raster-ordered 64x64 selections; the same + // expression naturally resolves to index zero for a 64x64 superblock. + int unitIndex = unitColumn + (unitRow << 1); + return this.frameInfo.GetCdefStrength(superblockPosition)[unitIndex]; + } + + /// + /// Determines whether every 4x4 mode-information block covered by an 8x8 CDEF block is skipped. + /// + /// The block's frame-relative column in 4x4 luma units. + /// The block's frame-relative row in 4x4 luma units. + /// when the complete 8x8 block is skipped; otherwise, . + private bool IsBlockSkipped(int modeInfoColumn, int modeInfoRow) + { + for (int row = 0; row < 2; row++) + { + for (int column = 0; column < 2; column++) + { + if (!this.frameInfo.GetModeInfoAt(new Point(modeInfoColumn + column, modeInfoRow + row)).Skip) + { + return false; + } + } + } + + return true; + } + + /// + /// Identifies one 8x8 luma block by its frame-relative mode-information coordinates. + /// + private readonly struct CdefBlock + { + /// + /// Initializes a new instance of the struct. + /// + /// The frame-relative column in 4x4 luma units. + /// The frame-relative row in 4x4 luma units. + public CdefBlock(int modeInfoColumn, int modeInfoRow) + { + this.ModeInfoColumn = modeInfoColumn; + this.ModeInfoRow = modeInfoRow; + } + + /// + /// Gets the frame-relative column in 4x4 luma units. + /// + public int ModeInfoColumn { get; } + + /// + /// Gets the frame-relative row in 4x4 luma units. + /// + public int ModeInfoRow { get; } + + /// + /// Gets the block offset in a bordered CDEF source unit. + /// + /// The number of samples between adjacent source rows. + /// The number of unavailable samples surrounding the source. + /// The unit's frame-relative column in 4x4 luma units. + /// The unit's frame-relative row in 4x4 luma units. + /// The horizontal chroma subsampling shift. + /// The vertical chroma subsampling shift. + /// The source-unit offset. + public int GetSourceOffset( + int sourceStride, + int sourceBorder, + int unitModeInfoColumn, + int unitModeInfoRow, + int subsamplingX, + int subsamplingY) + { + int planeColumn = ((this.ModeInfoColumn - unitModeInfoColumn) << Av1Constants.ModeInfoSizeLog2) >> subsamplingX; + int planeRow = ((this.ModeInfoRow - unitModeInfoRow) << Av1Constants.ModeInfoSizeLog2) >> subsamplingY; + return ((planeRow + sourceBorder) * sourceStride) + planeColumn + sourceBorder; + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/Cdef/Av1CdefFilter.ByteOutputOperator.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Cdef/Av1CdefFilter.ByteOutputOperator.cs new file mode 100644 index 0000000000..3066c68410 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Cdef/Av1CdefFilter.ByteOutputOperator.cs @@ -0,0 +1,34 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.Cdef; + +internal static partial class Av1CdefFilter +{ + /// + /// Writes filtered samples to eight-bit plane storage. + /// + private readonly struct ByteOutputOperator : IOutputOperator + { + /// + public static void StoreVector(ref byte destination, int offset, Vector128 value, int count) + { + Vector64 packed = Vector128.Narrow(value.AsUInt16(), Vector128.Zero).GetLower(); + ref byte output = ref Unsafe.Add(ref destination, offset); + if (count == 8) + { + packed.StoreUnsafe(ref output); + } + else + { + Unsafe.WriteUnaligned(ref output, packed.AsUInt32().ToScalar()); + } + } + + /// + public static void StoreScalar(ref byte destination, int offset, int value) => Unsafe.Add(ref destination, offset) = (byte)value; + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/Cdef/Av1CdefFilter.CopyFilterOperator.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Cdef/Av1CdefFilter.CopyFilterOperator.cs new file mode 100644 index 0000000000..8a28ad4180 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Cdef/Av1CdefFilter.CopyFilterOperator.cs @@ -0,0 +1,19 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.Cdef; + +internal static partial class Av1CdefFilter +{ + /// + /// Disables both tap groups so the source block is copied unchanged. + /// + private readonly struct CopyFilterOperator : IFilterOperator + { + /// + public static bool EnablePrimary => false; + + /// + public static bool EnableSecondary => false; + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/Cdef/Av1CdefFilter.Operator.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Cdef/Av1CdefFilter.Operator.cs new file mode 100644 index 0000000000..df9e5b0606 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Cdef/Av1CdefFilter.Operator.cs @@ -0,0 +1,50 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.Cdef; + +internal static partial class Av1CdefFilter +{ + /// + /// Defines storage-specific writes for one filtered row. + /// + /// The destination sample storage type. + private interface IOutputOperator + where TSample : unmanaged + { + /// + /// Stores four or eight filtered samples from the low vector lanes. + /// + /// The first element in the destination plane. + /// The offset of the first sample to write. + /// The filtered samples in the low lanes. + /// The number of valid lanes. + public static abstract void StoreVector(ref TSample destination, int offset, Vector128 value, int count); + + /// + /// Stores one filtered sample. + /// + /// The first element in the destination plane. + /// The offset of the sample to write. + /// The filtered sample. + public static abstract void StoreScalar(ref TSample destination, int offset, int value); + } + + /// + /// Defines which groups of directional taps participate in one closed filter kernel. + /// + private interface IFilterOperator + { + /// + /// Gets a value indicating whether the primary directional taps are enabled. + /// + public static abstract bool EnablePrimary { get; } + + /// + /// Gets a value indicating whether the secondary off-axis taps are enabled. + /// + public static abstract bool EnableSecondary { get; } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/Cdef/Av1CdefFilter.PrimaryAndSecondaryFilterOperator.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Cdef/Av1CdefFilter.PrimaryAndSecondaryFilterOperator.cs new file mode 100644 index 0000000000..2bfc98a7d8 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Cdef/Av1CdefFilter.PrimaryAndSecondaryFilterOperator.cs @@ -0,0 +1,19 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.Cdef; + +internal static partial class Av1CdefFilter +{ + /// + /// Enables both directional tap groups and their combined clipping rule. + /// + private readonly struct PrimaryAndSecondaryFilterOperator : IFilterOperator + { + /// + public static bool EnablePrimary => true; + + /// + public static bool EnableSecondary => true; + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/Cdef/Av1CdefFilter.PrimaryFilterOperator.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Cdef/Av1CdefFilter.PrimaryFilterOperator.cs new file mode 100644 index 0000000000..091835f8eb --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Cdef/Av1CdefFilter.PrimaryFilterOperator.cs @@ -0,0 +1,19 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.Cdef; + +internal static partial class Av1CdefFilter +{ + /// + /// Enables only the primary directional taps. + /// + private readonly struct PrimaryFilterOperator : IFilterOperator + { + /// + public static bool EnablePrimary => true; + + /// + public static bool EnableSecondary => false; + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/Cdef/Av1CdefFilter.SecondaryFilterOperator.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Cdef/Av1CdefFilter.SecondaryFilterOperator.cs new file mode 100644 index 0000000000..0a1521c02a --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Cdef/Av1CdefFilter.SecondaryFilterOperator.cs @@ -0,0 +1,19 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.Cdef; + +internal static partial class Av1CdefFilter +{ + /// + /// Enables only the secondary off-axis taps. + /// + private readonly struct SecondaryFilterOperator : IFilterOperator + { + /// + public static bool EnablePrimary => false; + + /// + public static bool EnableSecondary => true; + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/Cdef/Av1CdefFilter.UInt16OutputOperator.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Cdef/Av1CdefFilter.UInt16OutputOperator.cs new file mode 100644 index 0000000000..df80b20316 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Cdef/Av1CdefFilter.UInt16OutputOperator.cs @@ -0,0 +1,34 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.Cdef; + +internal static partial class Av1CdefFilter +{ + /// + /// Writes filtered samples to 16-bit plane storage. + /// + private readonly struct UInt16OutputOperator : IOutputOperator + { + /// + public static void StoreVector(ref ushort destination, int offset, Vector128 value, int count) + { + ref ushort output = ref Unsafe.Add(ref destination, offset); + if (count == 8) + { + value.AsUInt16().StoreUnsafe(ref output); + } + else + { + ref byte outputBytes = ref Unsafe.As(ref output); + Unsafe.WriteUnaligned(ref outputBytes, value.AsUInt64().GetLower().ToScalar()); + } + } + + /// + public static void StoreScalar(ref ushort destination, int offset, int value) => Unsafe.Add(ref destination, offset) = (ushort)value; + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/Cdef/Av1CdefFilter.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Cdef/Av1CdefFilter.cs new file mode 100644 index 0000000000..2eabaac131 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Cdef/Av1CdefFilter.cs @@ -0,0 +1,1695 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.InteropServices; +using System.Runtime.Intrinsics; +using System.Runtime.Intrinsics.X86; +using SixLabors.ImageSharp.Common.Helpers; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.Cdef; + +/// +/// Applies AV1 constrained directional enhancement filtering and derives the dominant direction of reconstructed blocks. +/// +/// +/// Filtering uses signed 16-bit working samples. The 256-bit kernel packs either two complete eight-sample rows or +/// four complete four-sample rows, keeping directional offsets within 128-bit lanes. Direction analysis instead packs +/// one eight-by-eight block per 128-bit lane so AVX2 can evaluate two independent blocks together. Scalar kernels retain +/// the same constrain, clipping, and tie-breaking rules for unsupported hardware and partial edge blocks. +/// +internal static partial class Av1CdefFilter +{ + /// + /// The sample value used in the bordered source plane for neighbors outside the coded frame. + /// + public const ushort VeryLarge = 0x4000; + + /// + /// Copies an eight-bit sample rectangle into the 16-bit CDEF working plane. + /// + /// The source sample plane. + /// The offset of the rectangle's top-left source sample. + /// The number of samples between adjacent source rows. + /// The destination CDEF working plane. + /// The offset of the rectangle's top-left destination sample. + /// The number of samples between adjacent destination rows. + /// The rectangle width in samples. + /// The even rectangle height in samples. + public static void CopyPlane( + ReadOnlySpan source, + int sourceOffset, + int sourceStride, + Span destination, + int destinationOffset, + int destinationStride, + int width, + int height) + { + ref byte sourceBase = ref MemoryMarshal.GetReference(source); + ref ushort destinationBase = ref MemoryMarshal.GetReference(destination); + + if (Avx2.IsSupported) + { + // AV1 plane dimensions are multiples of four, so the CDEF copy has an even row count. Processing two rows + // together follows the reference decoder's AVX2 scheduling while each conversion widens sixteen unsigned samples exactly. + for (int row = 0; row < height; row += 2) + { + int firstSourceRow = sourceOffset + (row * sourceStride); + int secondSourceRow = firstSourceRow + sourceStride; + int firstDestinationRow = destinationOffset + (row * destinationStride); + int secondDestinationRow = firstDestinationRow + destinationStride; + int column = 0; + nuint vectorCount = Numerics.Vector128Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector128.Count) + { + Vector128 first = Vector128.LoadUnsafe(ref sourceBase, (nuint)(firstSourceRow + column)); + Vector128 second = Vector128.LoadUnsafe(ref sourceBase, (nuint)(secondSourceRow + column)); + Avx2.ConvertToVector256Int16(first).AsUInt16().StoreUnsafe(ref destinationBase, (nuint)(firstDestinationRow + column)); + Avx2.ConvertToVector256Int16(second).AsUInt16().StoreUnsafe(ref destinationBase, (nuint)(secondDestinationRow + column)); + } + + CopyRemainingSamples(ref sourceBase, firstSourceRow, ref destinationBase, firstDestinationRow, column, width); + CopyRemainingSamples(ref sourceBase, secondSourceRow, ref destinationBase, secondDestinationRow, column, width); + } + + return; + } + + for (int row = 0; row < height; row++) + { + int sourceRow = sourceOffset + (row * sourceStride); + int destinationRow = destinationOffset + (row * destinationStride); + CopyRemainingSamples(ref sourceBase, sourceRow, ref destinationBase, destinationRow, 0, width); + } + } + + /// + /// Copies a 16-bit sample rectangle into the CDEF working plane. + /// + /// The source sample plane. + /// The offset of the rectangle's top-left source sample. + /// The number of samples between adjacent source rows. + /// The destination CDEF working plane. + /// The offset of the rectangle's top-left destination sample. + /// The number of samples between adjacent destination rows. + /// The rectangle width in samples. + /// The rectangle height in samples. + public static void CopyPlane( + ReadOnlySpan source, + int sourceOffset, + int sourceStride, + Span destination, + int destinationOffset, + int destinationStride, + int width, + int height) + { + for (int row = 0; row < height; row++) + { + ReadOnlySpan sourceRow = source.Slice(sourceOffset + (row * sourceStride), width); + + sourceRow.CopyTo(destination.Slice(destinationOffset + (row * destinationStride), width)); + } + } + + /// + /// Finds the dominant direction of an 8x8 luma block and its directional variance. + /// + /// The bordered, deblocked source plane. + /// The offset of the block's top-left sample. + /// The number of samples between adjacent source rows. + /// The number of bits above the eight-bit analysis precision. + /// Receives the variance difference between the selected and orthogonal directions. + /// The zero-based AV1 direction index. + public static int FindDirection( + ReadOnlySpan source, + int sourceOffset, + int sourceStride, + int coefficientShift, + out int variance) + { + if (Vector128.IsHardwareAccelerated) + { + return FindDirectionVector(source, sourceOffset, sourceStride, coefficientShift, out variance); + } + + return FindDirectionScalar(source, sourceOffset, sourceStride, coefficientShift, out variance); + } + + /// + /// Finds the dominant directions and directional variances of two independent 8x8 luma blocks. + /// + /// The bordered, deblocked source plane containing both blocks. + /// The offset of the first block's top-left sample. + /// The offset of the second block's top-left sample. + /// The number of samples between adjacent source rows. + /// The number of bits above the eight-bit analysis precision. + /// Receives the first block's zero-based AV1 direction index. + /// Receives the first block's directional variance. + /// Receives the second block's zero-based AV1 direction index. + /// Receives the second block's directional variance. + public static void FindDirections( + ReadOnlySpan source, + int firstSourceOffset, + int secondSourceOffset, + int sourceStride, + int coefficientShift, + out int firstDirection, + out int firstVariance, + out int secondDirection, + out int secondVariance) + { + if (Avx2.IsSupported) + { + FindDirectionsVector( + source, + firstSourceOffset, + secondSourceOffset, + sourceStride, + coefficientShift, + out firstDirection, + out firstVariance, + out secondDirection, + out secondVariance); + + return; + } + + firstDirection = FindDirection(source, firstSourceOffset, sourceStride, coefficientShift, out firstVariance); + secondDirection = FindDirection(source, secondSourceOffset, sourceStride, coefficientShift, out secondVariance); + } + + /// + /// Adjusts a luma primary strength according to the directional variance of its 8x8 block. + /// + /// The bit-depth-scaled primary strength. + /// The directional variance returned by . + /// The variance-adjusted primary strength. + public static int AdjustStrength(int strength, int variance) + { + int varianceClass = variance >> 6; + int adjustment = varianceClass != 0 ? Math.Min(Av1Math.MostSignificantBit((uint)varianceClass), 12) : 0; + return variance != 0 ? ((strength * (4 + adjustment)) + 8) >> 4 : 0; + } + + /// + /// Converts a luma direction to the matching chroma direction for asymmetric subsampling. + /// + /// The zero-based luma direction index. + /// The horizontal chroma subsampling shift. + /// The vertical chroma subsampling shift. + /// The direction index in the chroma sample grid. + public static int ConvertDirection(int direction, int subsamplingX, int subsamplingY) + { + if (subsamplingX == subsamplingY) + { + return direction; + } + + return subsamplingX != 0 + ? direction switch + { + 0 => 7, + 1 => 0, + 2 => 2, + 3 => 4, + 4 => 5, + _ => 6 + } + : direction switch + { + 0 => 1, + 1 or 2 or 3 => 2, + 4 => 3, + 5 => 4, + 6 => 6, + _ => 0 + }; + } + + /// + /// Filters one luma or chroma block into eight-bit sample storage. + /// + /// The bordered, deblocked source plane. + /// The offset of the block's top-left source sample. + /// The number of samples between adjacent source rows. + /// The unbordered filtered destination plane. + /// The offset of the block's top-left destination sample. + /// The number of samples between adjacent destination rows. + /// The bit-depth-scaled primary strength. + /// The bit-depth-scaled secondary strength. + /// The zero-based AV1 direction index. + /// The damping value applied to primary taps. + /// The damping value applied to secondary taps. + /// The number of bits above eight-bit sample precision. + /// The block width in plane samples. + /// The block height in plane samples. + public static void FilterBlock( + ReadOnlySpan source, + int sourceOffset, + int sourceStride, + Span destination, + int destinationOffset, + int destinationStride, + int primaryStrength, + int secondaryStrength, + int direction, + int primaryDamping, + int secondaryDamping, + int coefficientShift, + int blockWidth, + int blockHeight) + => FilterBlock( + source, + sourceOffset, + sourceStride, + destination, + destinationOffset, + destinationStride, + primaryStrength, + secondaryStrength, + direction, + primaryDamping, + secondaryDamping, + coefficientShift, + blockWidth, + blockHeight); + + /// + /// Filters one luma or chroma block into 16-bit sample storage. + /// + /// The bordered, deblocked source plane. + /// The offset of the block's top-left source sample. + /// The number of samples between adjacent source rows. + /// The unbordered filtered destination plane. + /// The offset of the block's top-left destination sample. + /// The number of samples between adjacent destination rows. + /// The bit-depth-scaled primary strength. + /// The bit-depth-scaled secondary strength. + /// The zero-based AV1 direction index. + /// The damping value applied to primary taps. + /// The damping value applied to secondary taps. + /// The number of bits above eight-bit sample precision. + /// The block width in plane samples. + /// The block height in plane samples. + public static void FilterBlock( + ReadOnlySpan source, + int sourceOffset, + int sourceStride, + Span destination, + int destinationOffset, + int destinationStride, + int primaryStrength, + int secondaryStrength, + int direction, + int primaryDamping, + int secondaryDamping, + int coefficientShift, + int blockWidth, + int blockHeight) + => FilterBlock( + source, + sourceOffset, + sourceStride, + destination, + destinationOffset, + destinationStride, + primaryStrength, + secondaryStrength, + direction, + primaryDamping, + secondaryDamping, + coefficientShift, + blockWidth, + blockHeight); + + /// + /// Selects the packed CDEF kernel or its scalar fallback through one closed output operator. + /// + /// The destination sample storage type. + /// The storage-specific output operator. + /// The bordered, deblocked source plane. + /// The offset of the block's top-left source sample. + /// The number of samples between adjacent source rows. + /// The unbordered filtered destination plane. + /// The offset of the block's top-left destination sample. + /// The number of samples between adjacent destination rows. + /// The bit-depth-scaled primary strength. + /// The bit-depth-scaled secondary strength. + /// The zero-based AV1 direction index. + /// The damping value applied to primary taps. + /// The damping value applied to secondary taps. + /// The number of bits above eight-bit sample precision. + /// The block width in plane samples. + /// The block height in plane samples. + private static void FilterBlock( + ReadOnlySpan source, + int sourceOffset, + int sourceStride, + Span destination, + int destinationOffset, + int destinationStride, + int primaryStrength, + int secondaryStrength, + int direction, + int primaryDamping, + int secondaryDamping, + int coefficientShift, + int blockWidth, + int blockHeight) + where TSample : unmanaged + where TOutputOperator : struct, IOutputOperator + { + ref ushort sourceBase = ref MemoryMarshal.GetReference(source); + ref TSample destinationBase = ref MemoryMarshal.GetReference(destination); + + // the reference decoder selects one of four closed kernels from the two strength flags. The semantic operator makes the same + // choice once per block so the JIT removes primary/secondary mode branches from every row and tap. + if (primaryStrength != 0) + { + if (secondaryStrength != 0) + { + FilterBlock( + ref sourceBase, + sourceOffset, + sourceStride, + ref destinationBase, + destinationOffset, + destinationStride, + primaryStrength, + secondaryStrength, + direction, + primaryDamping, + secondaryDamping, + coefficientShift, + blockWidth, + blockHeight); + } + else + { + FilterBlock( + ref sourceBase, + sourceOffset, + sourceStride, + ref destinationBase, + destinationOffset, + destinationStride, + primaryStrength, + secondaryStrength, + direction, + primaryDamping, + secondaryDamping, + coefficientShift, + blockWidth, + blockHeight); + } + + return; + } + + if (secondaryStrength != 0) + { + FilterBlock( + ref sourceBase, + sourceOffset, + sourceStride, + ref destinationBase, + destinationOffset, + destinationStride, + primaryStrength, + secondaryStrength, + direction, + primaryDamping, + secondaryDamping, + coefficientShift, + blockWidth, + blockHeight); + + return; + } + + FilterBlock( + ref sourceBase, + sourceOffset, + sourceStride, + ref destinationBase, + destinationOffset, + destinationStride, + primaryStrength, + secondaryStrength, + direction, + primaryDamping, + secondaryDamping, + coefficientShift, + blockWidth, + blockHeight); + } + + /// + /// Selects the packed or scalar implementation of one closed filter kernel. + /// + /// The destination sample storage type. + /// The storage-specific output operator. + /// The enabled directional-tap operator. + /// The first element in the bordered source plane. + /// The offset of the block's top-left source sample. + /// The number of samples between adjacent source rows. + /// The first element in the destination plane. + /// The offset of the block's top-left destination sample. + /// The number of samples between adjacent destination rows. + /// The bit-depth-scaled primary strength. + /// The bit-depth-scaled secondary strength. + /// The zero-based AV1 direction index. + /// The damping value applied to primary taps. + /// The damping value applied to secondary taps. + /// The number of bits above eight-bit sample precision. + /// The block width in plane samples. + /// The block height in plane samples. + private static void FilterBlock( + ref ushort source, + int sourceOffset, + int sourceStride, + ref TSample destination, + int destinationOffset, + int destinationStride, + int primaryStrength, + int secondaryStrength, + int direction, + int primaryDamping, + int secondaryDamping, + int coefficientShift, + int blockWidth, + int blockHeight) + where TSample : unmanaged + where TOutputOperator : struct, IOutputOperator + where TFilterOperator : struct, IFilterOperator + { + if (Avx2.IsSupported) + { + FilterBlockWideVector( + ref source, + sourceOffset, + sourceStride, + ref destination, + destinationOffset, + destinationStride, + primaryStrength, + secondaryStrength, + direction, + primaryDamping, + secondaryDamping, + coefficientShift, + blockWidth, + blockHeight); + + return; + } + + if (Vector128.IsHardwareAccelerated) + { + FilterBlockVector( + ref source, + sourceOffset, + sourceStride, + ref destination, + destinationOffset, + destinationStride, + primaryStrength, + secondaryStrength, + direction, + primaryDamping, + secondaryDamping, + coefficientShift, + blockWidth, + blockHeight); + + return; + } + + FilterBlockScalar( + ref source, + sourceOffset, + sourceStride, + ref destination, + destinationOffset, + destinationStride, + primaryStrength, + secondaryStrength, + direction, + primaryDamping, + secondaryDamping, + coefficientShift, + blockWidth, + blockHeight); + } + + /// + /// Applies one packed CDEF kernel to two 8-wide rows or four 4-wide rows at a time. + /// + /// The destination sample storage type. + /// The storage-specific output operator. + /// The enabled directional-tap operator. + /// The first element in the bordered source plane. + /// The offset of the block's top-left source sample. + /// The number of samples between adjacent source rows. + /// The first element in the destination plane. + /// The offset of the block's top-left destination sample. + /// The number of samples between adjacent destination rows. + /// The bit-depth-scaled primary strength. + /// The bit-depth-scaled secondary strength. + /// The zero-based AV1 direction index. + /// The damping value applied to primary taps. + /// The damping value applied to secondary taps. + /// The number of bits above eight-bit sample precision. + /// The block width in plane samples. + /// The block height in plane samples. + private static void FilterBlockWideVector( + ref ushort source, + int sourceOffset, + int sourceStride, + ref TSample destination, + int destinationOffset, + int destinationStride, + int primaryStrength, + int secondaryStrength, + int direction, + int primaryDamping, + int secondaryDamping, + int coefficientShift, + int blockWidth, + int blockHeight) + where TSample : unmanaged + where TOutputOperator : struct, IOutputOperator + where TFilterOperator : struct, IFilterOperator + { + bool clippingRequired = TFilterOperator.EnablePrimary && TFilterOperator.EnableSecondary; + int primaryDampingShift = TFilterOperator.EnablePrimary ? Math.Max(0, primaryDamping - Av1Math.MostSignificantBit((uint)primaryStrength)) : 0; + int secondaryDampingShift = TFilterOperator.EnableSecondary ? Math.Max(0, secondaryDamping - Av1Math.MostSignificantBit((uint)secondaryStrength)) : 0; + int primaryTapSet = (primaryStrength >> coefficientShift) & 1; + int primaryNearOffset = TFilterOperator.EnablePrimary ? GetDirectionOffset(direction, 0, sourceStride) : 0; + int primaryFarOffset = TFilterOperator.EnablePrimary ? GetDirectionOffset(direction, 1, sourceStride) : 0; + int secondaryNearOffset0 = TFilterOperator.EnableSecondary ? GetDirectionOffset((direction + 2) & 7, 0, sourceStride) : 0; + int secondaryFarOffset0 = TFilterOperator.EnableSecondary ? GetDirectionOffset((direction + 2) & 7, 1, sourceStride) : 0; + int secondaryNearOffset1 = TFilterOperator.EnableSecondary ? GetDirectionOffset((direction + 6) & 7, 0, sourceStride) : 0; + int secondaryFarOffset1 = TFilterOperator.EnableSecondary ? GetDirectionOffset((direction + 6) & 7, 1, sourceStride) : 0; + Vector256 primaryNearWeight = Vector256.Create((short)(primaryTapSet == 0 ? 4 : 3)); + Vector256 primaryFarWeight = Vector256.Create((short)(primaryTapSet == 0 ? 2 : 3)); + Vector256 secondaryNearWeight = Vector256.Create((short)2); + Vector256 secondaryFarWeight = Vector256.Create((short)1); + Vector256 sentinel = Vector256.Create((short)VeryLarge); + Vector256 zero = Vector256.Zero; + Vector256 rounding = Vector256.Create((short)8); + Vector256 one = Vector256.Create((short)1); + int rowsPerBatch = blockWidth == 8 ? 2 : 4; + + // The 256-bit lane layout follows the reference decoder: two complete 8-wide rows, or four complete 4-wide rows. Directional + // offsets therefore remain ordinary source offsets, while all constrain, weight, clip, and round operations + // advance several output rows together without crossing a row boundary inside any 128-bit lane. + for (int row = 0; row < blockHeight; row += rowsPerBatch) + { + int sourceIndex = sourceOffset + (row * sourceStride); + Vector256 sample = LoadRows(ref source, sourceIndex, sourceStride, blockWidth); + Vector256 sum = zero; + Vector256 minimum = sample; + Vector256 maximum = sample; + + for (int tap = 0; tap < 2; tap++) + { + if (TFilterOperator.EnablePrimary) + { + int offset = tap == 0 ? primaryNearOffset : primaryFarOffset; + Vector256 neighbor0 = LoadRows(ref source, sourceIndex + offset, sourceStride, blockWidth); + Vector256 neighbor1 = LoadRows(ref source, sourceIndex - offset, sourceStride, blockWidth); + Vector256 constrained = Constrain(neighbor0, sample, primaryStrength, primaryDampingShift) + + Constrain(neighbor1, sample, primaryStrength, primaryDampingShift); + + sum += constrained * (tap == 0 ? primaryNearWeight : primaryFarWeight); + if (clippingRequired) + { + minimum = Vector256.Min(minimum, Vector256.Min(neighbor0, neighbor1)); + maximum = MaximumIgnoringSentinel(maximum, neighbor0, sentinel); + maximum = MaximumIgnoringSentinel(maximum, neighbor1, sentinel); + } + } + + if (TFilterOperator.EnableSecondary) + { + int offset0 = tap == 0 ? secondaryNearOffset0 : secondaryFarOffset0; + int offset1 = tap == 0 ? secondaryNearOffset1 : secondaryFarOffset1; + Vector256 neighbor0 = LoadRows(ref source, sourceIndex + offset0, sourceStride, blockWidth); + Vector256 neighbor1 = LoadRows(ref source, sourceIndex - offset0, sourceStride, blockWidth); + Vector256 neighbor2 = LoadRows(ref source, sourceIndex + offset1, sourceStride, blockWidth); + Vector256 neighbor3 = LoadRows(ref source, sourceIndex - offset1, sourceStride, blockWidth); + Vector256 constrained = Constrain(neighbor0, sample, secondaryStrength, secondaryDampingShift) + + Constrain(neighbor1, sample, secondaryStrength, secondaryDampingShift) + + Constrain(neighbor2, sample, secondaryStrength, secondaryDampingShift) + + Constrain(neighbor3, sample, secondaryStrength, secondaryDampingShift); + + sum += constrained * (tap == 0 ? secondaryNearWeight : secondaryFarWeight); + if (clippingRequired) + { + minimum = Vector256.Min(minimum, Vector256.Min(Vector256.Min(neighbor0, neighbor1), Vector256.Min(neighbor2, neighbor3))); + maximum = MaximumIgnoringSentinel(maximum, neighbor0, sentinel); + maximum = MaximumIgnoringSentinel(maximum, neighbor1, sentinel); + maximum = MaximumIgnoringSentinel(maximum, neighbor2, sentinel); + maximum = MaximumIgnoringSentinel(maximum, neighbor3, sentinel); + } + } + } + + Vector256 correction = (sum >> 15) & one; + Vector256 filtered = sample + ((sum + rounding - correction) >> 4); + if (clippingRequired) + { + filtered = Vector256.Min(Vector256.Max(filtered, minimum), maximum); + } + + StoreRows( + ref destination, + destinationOffset + (row * destinationStride), + destinationStride, + filtered, + blockWidth); + } + } + + /// + /// Finds the dominant direction with eight signed 16-bit lanes representing one complete source row. + /// + /// The bordered, deblocked source plane. + /// The offset of the block's top-left sample. + /// The number of samples between adjacent source rows. + /// The number of bits above the eight-bit analysis precision. + /// Receives the variance difference between the selected and orthogonal directions. + /// The zero-based AV1 direction index. + private static int FindDirectionVector( + ReadOnlySpan source, + int sourceOffset, + int sourceStride, + int coefficientShift, + out int variance) + { + ref ushort sourceBase = ref MemoryMarshal.GetReference(source); + InlineArray8> lines = default; + Vector128 analysisBias = Vector128.Create((short)128); + + for (int row = 0; row < 8; row++) + { + Vector128 samples = Vector128.LoadUnsafe(ref sourceBase, (nuint)(sourceOffset + (row * sourceStride))); + + // AV1 direction selection is defined in an eight-bit domain. The signed bias keeps every accumulated line sum + // within Int16 while preserving identical direction and variance results for 8-, 10-, and 12-bit samples. + lines[row] = (samples >> coefficientShift).AsInt16() - analysisBias; + } + + // Vector lanes are written in memory order. These are the low-to-high forms of the reference decoder's set-style constants. + Vector128 foldWeights0 = Vector128.Create(840, 420, 280, 210); + Vector128 foldWeights1 = Vector128.Create(168, 140, 120, 105); + Vector128 diagonalWeights0 = Vector128.Create(0, 0, 420, 210); + Vector128 diagonalWeights1 = Vector128.Create(140, 105, 105, 105); + InlineArray8 costs = default; + ref int costBase = ref costs[0]; + + // The first pass evaluates directions 4..7. Rotating the block counter-clockwise lets the identical arithmetic + // evaluate directions 0..3, exactly matching the reference decoder's portable vector implementation. + ComputeDirectionCosts(ref lines, foldWeights0, foldWeights1, diagonalWeights0, diagonalWeights1).StoreUnsafe(ref costBase, 4); + ReverseTranspose(ref lines); + ComputeDirectionCosts(ref lines, foldWeights0, foldWeights1, diagonalWeights0, diagonalWeights1).StoreUnsafe(ref costBase); + + return SelectDirection(ref costs, out variance); + } + + /// + /// Finds two dominant directions with each 128-bit lane of a 256-bit vector representing one independent block. + /// + /// The bordered, deblocked source plane containing both blocks. + /// The offset of the first block's top-left sample. + /// The offset of the second block's top-left sample. + /// The number of samples between adjacent source rows. + /// The number of bits above the eight-bit analysis precision. + /// Receives the first block's zero-based AV1 direction index. + /// Receives the first block's directional variance. + /// Receives the second block's zero-based AV1 direction index. + /// Receives the second block's directional variance. + private static void FindDirectionsVector( + ReadOnlySpan source, + int firstSourceOffset, + int secondSourceOffset, + int sourceStride, + int coefficientShift, + out int firstDirection, + out int firstVariance, + out int secondDirection, + out int secondVariance) + { + ref ushort sourceBase = ref MemoryMarshal.GetReference(source); + InlineArray8> lines = default; + Vector256 analysisBias = Vector256.Create((short)128); + + for (int row = 0; row < 8; row++) + { + Vector128 first = Vector128.LoadUnsafe(ref sourceBase, (nuint)(firstSourceOffset + (row * sourceStride))); + Vector128 second = Vector128.LoadUnsafe(ref sourceBase, (nuint)(secondSourceOffset + (row * sourceStride))); + + // Keeping one block in each 128-bit lane is critical: every byte shift, unpack, multiply, + // and transpose remains lane-local while one AVX2 instruction advances both direction searches. + lines[row] = (Vector256.Create(first, second) >> coefficientShift).AsInt16() - analysisBias; + } + + Vector128 foldWeights0 = Vector128.Create(840, 420, 280, 210); + Vector128 foldWeights1 = Vector128.Create(168, 140, 120, 105); + Vector128 diagonalWeights0 = Vector128.Create(0, 0, 420, 210); + Vector128 diagonalWeights1 = Vector128.Create(140, 105, 105, 105); + Vector256 packedFoldWeights0 = Vector256.Create(foldWeights0, foldWeights0); + Vector256 packedFoldWeights1 = Vector256.Create(foldWeights1, foldWeights1); + Vector256 packedDiagonalWeights0 = Vector256.Create(diagonalWeights0, diagonalWeights0); + Vector256 packedDiagonalWeights1 = Vector256.Create(diagonalWeights1, diagonalWeights1); + InlineArray8 firstCosts = default; + InlineArray8 secondCosts = default; + ref int firstCostBase = ref firstCosts[0]; + ref int secondCostBase = ref secondCosts[0]; + + Vector256 direction47 = ComputeDirectionCosts( + ref lines, + packedFoldWeights0, + packedFoldWeights1, + packedDiagonalWeights0, + packedDiagonalWeights1); + + direction47.GetLower().StoreUnsafe(ref firstCostBase, 4); + direction47.GetUpper().StoreUnsafe(ref secondCostBase, 4); + ReverseTranspose(ref lines); + Vector256 direction03 = ComputeDirectionCosts( + ref lines, + packedFoldWeights0, + packedFoldWeights1, + packedDiagonalWeights0, + packedDiagonalWeights1); + + direction03.GetLower().StoreUnsafe(ref firstCostBase); + direction03.GetUpper().StoreUnsafe(ref secondCostBase); + firstDirection = SelectDirection(ref firstCosts, out firstVariance); + secondDirection = SelectDirection(ref secondCosts, out secondVariance); + } + + /// + /// Computes four adjacent AV1 direction costs from eight packed source rows. + /// + /// The eight signed, biased source rows. + /// The line-length weights for the first four folded pairs. + /// The line-length weights for the second four folded pairs. + /// The first line-length weights for the shallow diagonal directions. + /// The second line-length weights for the shallow diagonal directions. + /// The four costs ordered by increasing direction within the current orientation. + private static Vector128 ComputeDirectionCosts( + ref InlineArray8> lines, + Vector128 foldWeights0, + Vector128 foldWeights1, + Vector128 diagonalWeights0, + Vector128 diagonalWeights1) + { + // Byte-lane shifts move whole Int16 samples while inserting zeroes. Each shifted row therefore lands in the + // vector lanes for one geometric line without gathers or a per-block partial-sum buffer. + Vector128 partial4A = Vector128_.ShiftLeftBytesInVector(lines[0].AsByte(), 14).AsInt16(); + Vector128 partial4B = Vector128_.ShiftRightBytesInVector(lines[0].AsByte(), 2).AsInt16(); + partial4A += Vector128_.ShiftLeftBytesInVector(lines[1].AsByte(), 12).AsInt16(); + partial4B += Vector128_.ShiftRightBytesInVector(lines[1].AsByte(), 4).AsInt16(); + Vector128 pair = lines[0] + lines[1]; + Vector128 partial5A = Vector128_.ShiftLeftBytesInVector(pair.AsByte(), 10).AsInt16(); + Vector128 partial5B = Vector128_.ShiftRightBytesInVector(pair.AsByte(), 6).AsInt16(); + Vector128 partial7A = Vector128_.ShiftLeftBytesInVector(pair.AsByte(), 4).AsInt16(); + Vector128 partial7B = Vector128_.ShiftRightBytesInVector(pair.AsByte(), 12).AsInt16(); + Vector128 partial6 = pair; + + partial4A += Vector128_.ShiftLeftBytesInVector(lines[2].AsByte(), 10).AsInt16(); + partial4B += Vector128_.ShiftRightBytesInVector(lines[2].AsByte(), 6).AsInt16(); + partial4A += Vector128_.ShiftLeftBytesInVector(lines[3].AsByte(), 8).AsInt16(); + partial4B += Vector128_.ShiftRightBytesInVector(lines[3].AsByte(), 8).AsInt16(); + pair = lines[2] + lines[3]; + partial5A += Vector128_.ShiftLeftBytesInVector(pair.AsByte(), 8).AsInt16(); + partial5B += Vector128_.ShiftRightBytesInVector(pair.AsByte(), 8).AsInt16(); + partial7A += Vector128_.ShiftLeftBytesInVector(pair.AsByte(), 6).AsInt16(); + partial7B += Vector128_.ShiftRightBytesInVector(pair.AsByte(), 10).AsInt16(); + partial6 += pair; + + partial4A += Vector128_.ShiftLeftBytesInVector(lines[4].AsByte(), 6).AsInt16(); + partial4B += Vector128_.ShiftRightBytesInVector(lines[4].AsByte(), 10).AsInt16(); + partial4A += Vector128_.ShiftLeftBytesInVector(lines[5].AsByte(), 4).AsInt16(); + partial4B += Vector128_.ShiftRightBytesInVector(lines[5].AsByte(), 12).AsInt16(); + pair = lines[4] + lines[5]; + partial5A += Vector128_.ShiftLeftBytesInVector(pair.AsByte(), 6).AsInt16(); + partial5B += Vector128_.ShiftRightBytesInVector(pair.AsByte(), 10).AsInt16(); + partial7A += Vector128_.ShiftLeftBytesInVector(pair.AsByte(), 8).AsInt16(); + partial7B += Vector128_.ShiftRightBytesInVector(pair.AsByte(), 8).AsInt16(); + partial6 += pair; + + partial4A += Vector128_.ShiftLeftBytesInVector(lines[6].AsByte(), 2).AsInt16(); + partial4B += Vector128_.ShiftRightBytesInVector(lines[6].AsByte(), 14).AsInt16(); + partial4A += lines[7]; + pair = lines[6] + lines[7]; + partial5A += Vector128_.ShiftLeftBytesInVector(pair.AsByte(), 4).AsInt16(); + partial5B += Vector128_.ShiftRightBytesInVector(pair.AsByte(), 12).AsInt16(); + partial7A += Vector128_.ShiftLeftBytesInVector(pair.AsByte(), 10).AsInt16(); + partial7B += Vector128_.ShiftRightBytesInVector(pair.AsByte(), 6).AsInt16(); + partial6 += pair; + + Vector128 partial4Cost = FoldDirectionPartials(partial4A, partial4B, foldWeights0, foldWeights1); + Vector128 partial5Cost = FoldDirectionPartials(partial5A, partial5B, diagonalWeights0, diagonalWeights1); + Vector128 partial7Cost = FoldDirectionPartials(partial7A, partial7B, diagonalWeights0, diagonalWeights1); + Vector128 partial6Cost = Vector128_.MultiplyAddAdjacent(partial6, partial6) * Vector128.Create(105); + return HorizontalSumFour(partial4Cost, partial5Cost, partial6Cost, partial7Cost); + } + + /// + /// Computes four adjacent AV1 direction costs for two blocks packed into independent 128-bit lanes. + /// + /// The eight signed, biased source rows for both blocks. + /// The line-length weights for the first four folded pairs. + /// The line-length weights for the second four folded pairs. + /// The first line-length weights for the shallow diagonal directions. + /// The second line-length weights for the shallow diagonal directions. + /// The four costs per block ordered by increasing direction within the current orientation. + private static Vector256 ComputeDirectionCosts( + ref InlineArray8> lines, + Vector256 foldWeights0, + Vector256 foldWeights1, + Vector256 diagonalWeights0, + Vector256 diagonalWeights1) + { + Vector256 partial4A = Avx2.ShiftLeftLogical128BitLane(lines[0].AsByte(), 14).AsInt16(); + Vector256 partial4B = Avx2.ShiftRightLogical128BitLane(lines[0].AsByte(), 2).AsInt16(); + partial4A += Avx2.ShiftLeftLogical128BitLane(lines[1].AsByte(), 12).AsInt16(); + partial4B += Avx2.ShiftRightLogical128BitLane(lines[1].AsByte(), 4).AsInt16(); + Vector256 pair = lines[0] + lines[1]; + Vector256 partial5A = Avx2.ShiftLeftLogical128BitLane(pair.AsByte(), 10).AsInt16(); + Vector256 partial5B = Avx2.ShiftRightLogical128BitLane(pair.AsByte(), 6).AsInt16(); + Vector256 partial7A = Avx2.ShiftLeftLogical128BitLane(pair.AsByte(), 4).AsInt16(); + Vector256 partial7B = Avx2.ShiftRightLogical128BitLane(pair.AsByte(), 12).AsInt16(); + Vector256 partial6 = pair; + + partial4A += Avx2.ShiftLeftLogical128BitLane(lines[2].AsByte(), 10).AsInt16(); + partial4B += Avx2.ShiftRightLogical128BitLane(lines[2].AsByte(), 6).AsInt16(); + partial4A += Avx2.ShiftLeftLogical128BitLane(lines[3].AsByte(), 8).AsInt16(); + partial4B += Avx2.ShiftRightLogical128BitLane(lines[3].AsByte(), 8).AsInt16(); + pair = lines[2] + lines[3]; + partial5A += Avx2.ShiftLeftLogical128BitLane(pair.AsByte(), 8).AsInt16(); + partial5B += Avx2.ShiftRightLogical128BitLane(pair.AsByte(), 8).AsInt16(); + partial7A += Avx2.ShiftLeftLogical128BitLane(pair.AsByte(), 6).AsInt16(); + partial7B += Avx2.ShiftRightLogical128BitLane(pair.AsByte(), 10).AsInt16(); + partial6 += pair; + + partial4A += Avx2.ShiftLeftLogical128BitLane(lines[4].AsByte(), 6).AsInt16(); + partial4B += Avx2.ShiftRightLogical128BitLane(lines[4].AsByte(), 10).AsInt16(); + partial4A += Avx2.ShiftLeftLogical128BitLane(lines[5].AsByte(), 4).AsInt16(); + partial4B += Avx2.ShiftRightLogical128BitLane(lines[5].AsByte(), 12).AsInt16(); + pair = lines[4] + lines[5]; + partial5A += Avx2.ShiftLeftLogical128BitLane(pair.AsByte(), 6).AsInt16(); + partial5B += Avx2.ShiftRightLogical128BitLane(pair.AsByte(), 10).AsInt16(); + partial7A += Avx2.ShiftLeftLogical128BitLane(pair.AsByte(), 8).AsInt16(); + partial7B += Avx2.ShiftRightLogical128BitLane(pair.AsByte(), 8).AsInt16(); + partial6 += pair; + + partial4A += Avx2.ShiftLeftLogical128BitLane(lines[6].AsByte(), 2).AsInt16(); + partial4B += Avx2.ShiftRightLogical128BitLane(lines[6].AsByte(), 14).AsInt16(); + partial4A += lines[7]; + pair = lines[6] + lines[7]; + partial5A += Avx2.ShiftLeftLogical128BitLane(pair.AsByte(), 4).AsInt16(); + partial5B += Avx2.ShiftRightLogical128BitLane(pair.AsByte(), 12).AsInt16(); + partial7A += Avx2.ShiftLeftLogical128BitLane(pair.AsByte(), 10).AsInt16(); + partial7B += Avx2.ShiftRightLogical128BitLane(pair.AsByte(), 6).AsInt16(); + partial6 += pair; + + Vector256 partial4Cost = FoldDirectionPartials(partial4A, partial4B, foldWeights0, foldWeights1); + Vector256 partial5Cost = FoldDirectionPartials(partial5A, partial5B, diagonalWeights0, diagonalWeights1); + Vector256 partial7Cost = FoldDirectionPartials(partial7A, partial7B, diagonalWeights0, diagonalWeights1); + Vector256 partial6Cost = Avx2.MultiplyAddAdjacent(partial6, partial6) * Vector256.Create(105); + return HorizontalSumFour(partial4Cost, partial5Cost, partial6Cost, partial7Cost); + } + + /// + /// Squares, weights, and combines the two halves of one set of directional line sums. + /// + /// The first eight line sums. + /// The remaining seven line sums followed by zero. + /// The first four line-length weights. + /// The second four line-length weights. + /// Four packed weighted partial costs. + private static Vector128 FoldDirectionPartials( + Vector128 partialA, + Vector128 partialB, + Vector128 weights0, + Vector128 weights1) + { + // Reversal aligns equally long lines. Interleaving then gives MultiplyAddAdjacent the [x,y] pairs whose + // squared magnitudes share one line-length weight, including the unpaired centre line with an inserted zero. + partialB = Vector128.ShuffleNative(partialB, Vector128.Create((short)6, 5, 4, 3, 2, 1, 0, 7)); + Vector128 originalA = partialA; + partialA = Vector128_.UnpackLow(partialA, partialB); + partialB = Vector128_.UnpackHigh(originalA, partialB); + Vector128 lower = Vector128_.MultiplyAddAdjacent(partialA, partialA) * weights0; + Vector128 upper = Vector128_.MultiplyAddAdjacent(partialB, partialB) * weights1; + return lower + upper; + } + + /// + /// Squares, weights, and combines directional line sums for two independent packed blocks. + /// + /// The first eight line sums in each 128-bit lane. + /// The remaining seven line sums followed by zero in each 128-bit lane. + /// The first four line-length weights in each 128-bit lane. + /// The second four line-length weights in each 128-bit lane. + /// Four packed weighted partial costs per block. + private static Vector256 FoldDirectionPartials( + Vector256 partialA, + Vector256 partialB, + Vector256 weights0, + Vector256 weights1) + { + Vector128 laneShuffle = Vector128.Create((byte)12, 13, 10, 11, 8, 9, 6, 7, 4, 5, 2, 3, 0, 1, 14, 15); + partialB = Avx2.Shuffle(partialB.AsByte(), Vector256.Create(laneShuffle, laneShuffle)).AsInt16(); + Vector256 originalA = partialA; + partialA = Avx2.UnpackLow(partialA, partialB); + partialB = Avx2.UnpackHigh(originalA, partialB); + Vector256 lower = Avx2.MultiplyAddAdjacent(partialA, partialA) * weights0; + Vector256 upper = Avx2.MultiplyAddAdjacent(partialB, partialB) * weights1; + return lower + upper; + } + + /// + /// Horizontally reduces four cost vectors into four direction costs. + /// + /// The first direction's four partial costs. + /// The second direction's four partial costs. + /// The third direction's four partial costs. + /// The fourth direction's four partial costs. + /// The four horizontally reduced costs. + private static Vector128 HorizontalSumFour( + Vector128 cost0, + Vector128 cost1, + Vector128 cost2, + Vector128 cost3) + { + Vector128 pair01Lower = Vector128_.UnpackLow(cost0, cost1); + Vector128 pair23Lower = Vector128_.UnpackLow(cost2, cost3); + Vector128 pair01Upper = Vector128_.UnpackHigh(cost0, cost1); + Vector128 pair23Upper = Vector128_.UnpackHigh(cost2, cost3); + Vector128 quad0 = Vector128_.UnpackLow(pair01Lower.AsInt64(), pair23Lower.AsInt64()).AsInt32(); + Vector128 quad1 = Vector128_.UnpackHigh(pair01Lower.AsInt64(), pair23Lower.AsInt64()).AsInt32(); + Vector128 quad2 = Vector128_.UnpackLow(pair01Upper.AsInt64(), pair23Upper.AsInt64()).AsInt32(); + Vector128 quad3 = Vector128_.UnpackHigh(pair01Upper.AsInt64(), pair23Upper.AsInt64()).AsInt32(); + return (quad0 + quad1) + (quad2 + quad3); + } + + /// + /// Horizontally reduces four cost vectors independently within both 128-bit lanes. + /// + /// The first direction's four partial costs per block. + /// The second direction's four partial costs per block. + /// The third direction's four partial costs per block. + /// The fourth direction's four partial costs per block. + /// The four horizontally reduced costs per block. + private static Vector256 HorizontalSumFour( + Vector256 cost0, + Vector256 cost1, + Vector256 cost2, + Vector256 cost3) + { + Vector256 pair01Lower = Avx2.UnpackLow(cost0, cost1); + Vector256 pair23Lower = Avx2.UnpackLow(cost2, cost3); + Vector256 pair01Upper = Avx2.UnpackHigh(cost0, cost1); + Vector256 pair23Upper = Avx2.UnpackHigh(cost2, cost3); + Vector256 quad0 = Avx2.UnpackLow(pair01Lower.AsInt64(), pair23Lower.AsInt64()).AsInt32(); + Vector256 quad1 = Avx2.UnpackHigh(pair01Lower.AsInt64(), pair23Lower.AsInt64()).AsInt32(); + Vector256 quad2 = Avx2.UnpackLow(pair01Upper.AsInt64(), pair23Upper.AsInt64()).AsInt32(); + Vector256 quad3 = Avx2.UnpackHigh(pair01Upper.AsInt64(), pair23Upper.AsInt64()).AsInt32(); + return (quad0 + quad1) + (quad2 + quad3); + } + + /// + /// Rotates an 8x8 packed sample block counter-clockwise by transposing and reversing its rows. + /// + /// The source rows, replaced by the rotated rows. + private static void ReverseTranspose(ref InlineArray8> lines) + { + Vector128 pair01Lower = Vector128_.UnpackLow(lines[0], lines[1]); + Vector128 pair23Lower = Vector128_.UnpackLow(lines[2], lines[3]); + Vector128 pair01Upper = Vector128_.UnpackHigh(lines[0], lines[1]); + Vector128 pair23Upper = Vector128_.UnpackHigh(lines[2], lines[3]); + Vector128 pair45Lower = Vector128_.UnpackLow(lines[4], lines[5]); + Vector128 pair67Lower = Vector128_.UnpackLow(lines[6], lines[7]); + Vector128 pair45Upper = Vector128_.UnpackHigh(lines[4], lines[5]); + Vector128 pair67Upper = Vector128_.UnpackHigh(lines[6], lines[7]); + Vector128 quad03Lower = Vector128_.UnpackLow(pair01Lower.AsInt32(), pair23Lower.AsInt32()); + Vector128 quad47Lower = Vector128_.UnpackLow(pair45Lower.AsInt32(), pair67Lower.AsInt32()); + Vector128 quad03Middle = Vector128_.UnpackHigh(pair01Lower.AsInt32(), pair23Lower.AsInt32()); + Vector128 quad47Middle = Vector128_.UnpackHigh(pair45Lower.AsInt32(), pair67Lower.AsInt32()); + Vector128 quad03Upper = Vector128_.UnpackLow(pair01Upper.AsInt32(), pair23Upper.AsInt32()); + Vector128 quad47Upper = Vector128_.UnpackLow(pair45Upper.AsInt32(), pair67Upper.AsInt32()); + Vector128 quad03Highest = Vector128_.UnpackHigh(pair01Upper.AsInt32(), pair23Upper.AsInt32()); + Vector128 quad47Highest = Vector128_.UnpackHigh(pair45Upper.AsInt32(), pair67Upper.AsInt32()); + + // Writing in reverse order turns the normal transpose into the counter-clockwise rotation required to reuse + // the same four-direction cost kernel for the orthogonal half of the search. + lines[7] = Vector128_.UnpackLow(quad03Lower.AsInt64(), quad47Lower.AsInt64()).AsInt16(); + lines[6] = Vector128_.UnpackHigh(quad03Lower.AsInt64(), quad47Lower.AsInt64()).AsInt16(); + lines[5] = Vector128_.UnpackLow(quad03Middle.AsInt64(), quad47Middle.AsInt64()).AsInt16(); + lines[4] = Vector128_.UnpackHigh(quad03Middle.AsInt64(), quad47Middle.AsInt64()).AsInt16(); + lines[3] = Vector128_.UnpackLow(quad03Upper.AsInt64(), quad47Upper.AsInt64()).AsInt16(); + lines[2] = Vector128_.UnpackHigh(quad03Upper.AsInt64(), quad47Upper.AsInt64()).AsInt16(); + lines[1] = Vector128_.UnpackLow(quad03Highest.AsInt64(), quad47Highest.AsInt64()).AsInt16(); + lines[0] = Vector128_.UnpackHigh(quad03Highest.AsInt64(), quad47Highest.AsInt64()).AsInt16(); + } + + /// + /// Rotates two packed 8x8 sample blocks counter-clockwise within their independent 128-bit lanes. + /// + /// The source rows for both blocks, replaced by the rotated rows. + private static void ReverseTranspose(ref InlineArray8> lines) + { + Vector256 pair01Lower = Avx2.UnpackLow(lines[0], lines[1]); + Vector256 pair23Lower = Avx2.UnpackLow(lines[2], lines[3]); + Vector256 pair01Upper = Avx2.UnpackHigh(lines[0], lines[1]); + Vector256 pair23Upper = Avx2.UnpackHigh(lines[2], lines[3]); + Vector256 pair45Lower = Avx2.UnpackLow(lines[4], lines[5]); + Vector256 pair67Lower = Avx2.UnpackLow(lines[6], lines[7]); + Vector256 pair45Upper = Avx2.UnpackHigh(lines[4], lines[5]); + Vector256 pair67Upper = Avx2.UnpackHigh(lines[6], lines[7]); + Vector256 quad03Lower = Avx2.UnpackLow(pair01Lower.AsInt32(), pair23Lower.AsInt32()); + Vector256 quad47Lower = Avx2.UnpackLow(pair45Lower.AsInt32(), pair67Lower.AsInt32()); + Vector256 quad03Middle = Avx2.UnpackHigh(pair01Lower.AsInt32(), pair23Lower.AsInt32()); + Vector256 quad47Middle = Avx2.UnpackHigh(pair45Lower.AsInt32(), pair67Lower.AsInt32()); + Vector256 quad03Upper = Avx2.UnpackLow(pair01Upper.AsInt32(), pair23Upper.AsInt32()); + Vector256 quad47Upper = Avx2.UnpackLow(pair45Upper.AsInt32(), pair67Upper.AsInt32()); + Vector256 quad03Highest = Avx2.UnpackHigh(pair01Upper.AsInt32(), pair23Upper.AsInt32()); + Vector256 quad47Highest = Avx2.UnpackHigh(pair45Upper.AsInt32(), pair67Upper.AsInt32()); + + lines[7] = Avx2.UnpackLow(quad03Lower.AsInt64(), quad47Lower.AsInt64()).AsInt16(); + lines[6] = Avx2.UnpackHigh(quad03Lower.AsInt64(), quad47Lower.AsInt64()).AsInt16(); + lines[5] = Avx2.UnpackLow(quad03Middle.AsInt64(), quad47Middle.AsInt64()).AsInt16(); + lines[4] = Avx2.UnpackHigh(quad03Middle.AsInt64(), quad47Middle.AsInt64()).AsInt16(); + lines[3] = Avx2.UnpackLow(quad03Upper.AsInt64(), quad47Upper.AsInt64()).AsInt16(); + lines[2] = Avx2.UnpackHigh(quad03Upper.AsInt64(), quad47Upper.AsInt64()).AsInt16(); + lines[1] = Avx2.UnpackLow(quad03Highest.AsInt64(), quad47Highest.AsInt64()).AsInt16(); + lines[0] = Avx2.UnpackHigh(quad03Highest.AsInt64(), quad47Highest.AsInt64()).AsInt16(); + } + + /// + /// Finds the dominant direction without vector instructions or a frame-block partial-sum buffer. + /// + /// The bordered, deblocked source plane. + /// The offset of the block's top-left sample. + /// The number of samples between adjacent source rows. + /// The number of bits above the eight-bit analysis precision. + /// Receives the variance difference between the selected and orthogonal directions. + /// The zero-based AV1 direction index. + private static int FindDirectionScalar( + ReadOnlySpan source, + int sourceOffset, + int sourceStride, + int coefficientShift, + out int variance) + { + InlineArray8 costs = default; + InlineArray16 partials = default; + Span lineSums = partials; + + // The fallback reuses one 15-line accumulator for each direction. Re-reading the 8x8 block is preferable to + // reserving and clearing the old 120-element partial table on every block when SIMD is explicitly disabled. + for (int direction = 0; direction < 8; direction++) + { + lineSums.Clear(); + for (int row = 0; row < 8; row++) + { + for (int column = 0; column < 8; column++) + { + int value = (source[sourceOffset + (row * sourceStride) + column] >> coefficientShift) - 128; + lineSums[GetDirectionLine(direction, row, column)] += value; + } + } + + costs[direction] = CalculateDirectionCost(direction, lineSums); + } + + return SelectDirection(ref costs, out variance); + } + + /// + /// Selects the first maximum direction cost and derives its orthogonal variance difference. + /// + /// The eight direction costs in ascending direction order. + /// Receives the scaled difference from the selected direction's orthogonal cost. + /// The zero-based AV1 direction index. + private static int SelectDirection(ref InlineArray8 costs, out int variance) + { + int bestCost = 0; + int bestDirection = 0; + for (int direction = 0; direction < 8; direction++) + { + if (costs[direction] > bestCost) + { + bestCost = costs[direction]; + bestDirection = direction; + } + } + + // All directions omit the same sum-of-squares term, so the scaled difference from the orthogonal cost is the + // directional variance consumed by the luma strength adjustment. + variance = (bestCost - costs[(bestDirection + 4) & 7]) >> 10; + return bestDirection; + } + + /// + /// Maps one source coordinate to its line in the requested AV1 direction. + /// + /// The zero-based AV1 direction index. + /// The source row within the 8x8 block. + /// The source column within the 8x8 block. + /// The zero-based line index. + private static int GetDirectionLine(int direction, int row, int column) => direction switch + { + 0 => row + column, + 1 => row + (column >> 1), + 2 => row, + 3 => 3 + row - (column >> 1), + 4 => 7 + row - column, + 5 => 3 - (row >> 1) + column, + 6 => column, + _ => (row >> 1) + column + }; + + /// + /// Computes one weighted direction cost from its accumulated line sums. + /// + /// The zero-based AV1 direction index. + /// The accumulated line sums. + /// The weighted direction cost. + private static int CalculateDirectionCost(int direction, ReadOnlySpan lineSums) + { + int cost = 0; + if (direction is 2 or 6) + { + for (int line = 0; line < 8; line++) + { + cost += lineSums[line] * lineSums[line]; + } + + return cost * 105; + } + + if ((direction & 1) == 0) + { + for (int line = 0; line < 7; line++) + { + int mirroredLine = 14 - line; + cost += ((lineSums[line] * lineSums[line]) + (lineSums[mirroredLine] * lineSums[mirroredLine])) * GetDivisionMultiplier(line + 1); + } + + return cost + (lineSums[7] * lineSums[7] * 105); + } + + for (int line = 3; line < 8; line++) + { + cost += lineSums[line] * lineSums[line]; + } + + cost *= 105; + for (int line = 0; line < 3; line++) + { + int mirroredLine = 10 - line; + cost += ((lineSums[line] * lineSums[line]) + (lineSums[mirroredLine] * lineSums[mirroredLine])) * GetDivisionMultiplier((line * 2) + 2); + } + + return cost; + } + + /// + /// Gets the common multiple used to compare lines of different lengths without division. + /// + /// The number of samples contributing to the line. + /// The multiplier equal to 840 divided by the line length. + private static int GetDivisionMultiplier(int lineLength) => lineLength switch + { + 1 => 840, + 2 => 420, + 3 => 280, + 4 => 210, + 5 => 168, + 6 => 140, + 7 => 120, + _ => 105 + }; + + /// + /// Applies one packed CDEF kernel with each signed 16-bit lane representing a column in the current output row. + /// + /// The destination sample storage type. + /// The storage-specific output operator. + /// The enabled directional-tap operator. + /// The first element in the bordered source plane. + /// The offset of the block's top-left source sample. + /// The number of samples between adjacent source rows. + /// The first element in the destination plane. + /// The offset of the block's top-left destination sample. + /// The number of samples between adjacent destination rows. + /// The bit-depth-scaled primary strength. + /// The bit-depth-scaled secondary strength. + /// The zero-based AV1 direction index. + /// The damping value applied to primary taps. + /// The damping value applied to secondary taps. + /// The number of bits above eight-bit sample precision. + /// The block width in plane samples. + /// The block height in plane samples. + private static void FilterBlockVector( + ref ushort source, + int sourceOffset, + int sourceStride, + ref TSample destination, + int destinationOffset, + int destinationStride, + int primaryStrength, + int secondaryStrength, + int direction, + int primaryDamping, + int secondaryDamping, + int coefficientShift, + int blockWidth, + int blockHeight) + where TSample : unmanaged + where TOutputOperator : struct, IOutputOperator + where TFilterOperator : struct, IFilterOperator + { + bool clippingRequired = TFilterOperator.EnablePrimary && TFilterOperator.EnableSecondary; + int primaryDampingShift = TFilterOperator.EnablePrimary ? Math.Max(0, primaryDamping - Av1Math.MostSignificantBit((uint)primaryStrength)) : 0; + int secondaryDampingShift = TFilterOperator.EnableSecondary ? Math.Max(0, secondaryDamping - Av1Math.MostSignificantBit((uint)secondaryStrength)) : 0; + int primaryTapSet = (primaryStrength >> coefficientShift) & 1; + int primaryNearOffset = TFilterOperator.EnablePrimary ? GetDirectionOffset(direction, 0, sourceStride) : 0; + int primaryFarOffset = TFilterOperator.EnablePrimary ? GetDirectionOffset(direction, 1, sourceStride) : 0; + int secondaryNearOffset0 = TFilterOperator.EnableSecondary ? GetDirectionOffset((direction + 2) & 7, 0, sourceStride) : 0; + int secondaryFarOffset0 = TFilterOperator.EnableSecondary ? GetDirectionOffset((direction + 2) & 7, 1, sourceStride) : 0; + int secondaryNearOffset1 = TFilterOperator.EnableSecondary ? GetDirectionOffset((direction + 6) & 7, 0, sourceStride) : 0; + int secondaryFarOffset1 = TFilterOperator.EnableSecondary ? GetDirectionOffset((direction + 6) & 7, 1, sourceStride) : 0; + Vector128 primaryNearWeight = Vector128.Create((short)(primaryTapSet == 0 ? 4 : 3)); + Vector128 primaryFarWeight = Vector128.Create((short)(primaryTapSet == 0 ? 2 : 3)); + Vector128 secondaryNearWeight = Vector128.Create((short)2); + Vector128 secondaryFarWeight = Vector128.Create((short)1); + Vector128 sentinel = Vector128.Create((short)VeryLarge); + Vector128 zero = Vector128.Zero; + Vector128 rounding = Vector128.Create((short)8); + Vector128 one = Vector128.Create((short)1); + + for (int row = 0; row < blockHeight; row++) + { + int sourceIndex = sourceOffset + (row * sourceStride); + Vector128 sample = LoadSamples(ref source, sourceIndex, blockWidth); + Vector128 sum = zero; + Vector128 minimum = sample; + Vector128 maximum = sample; + + for (int tap = 0; tap < 2; tap++) + { + if (TFilterOperator.EnablePrimary) + { + int offset = tap == 0 ? primaryNearOffset : primaryFarOffset; + Vector128 neighbor0 = LoadSamples(ref source, sourceIndex + offset, blockWidth); + Vector128 neighbor1 = LoadSamples(ref source, sourceIndex - offset, blockWidth); + Vector128 constrained = Constrain(neighbor0, sample, primaryStrength, primaryDampingShift) + + Constrain(neighbor1, sample, primaryStrength, primaryDampingShift); + + sum += constrained * (tap == 0 ? primaryNearWeight : primaryFarWeight); + if (clippingRequired) + { + minimum = Vector128.Min(minimum, Vector128.Min(neighbor0, neighbor1)); + maximum = MaximumIgnoringSentinel(maximum, neighbor0, sentinel); + maximum = MaximumIgnoringSentinel(maximum, neighbor1, sentinel); + } + } + + if (TFilterOperator.EnableSecondary) + { + int offset0 = tap == 0 ? secondaryNearOffset0 : secondaryFarOffset0; + int offset1 = tap == 0 ? secondaryNearOffset1 : secondaryFarOffset1; + Vector128 neighbor0 = LoadSamples(ref source, sourceIndex + offset0, blockWidth); + Vector128 neighbor1 = LoadSamples(ref source, sourceIndex - offset0, blockWidth); + Vector128 neighbor2 = LoadSamples(ref source, sourceIndex + offset1, blockWidth); + Vector128 neighbor3 = LoadSamples(ref source, sourceIndex - offset1, blockWidth); + Vector128 constrained = Constrain(neighbor0, sample, secondaryStrength, secondaryDampingShift) + + Constrain(neighbor1, sample, secondaryStrength, secondaryDampingShift) + + Constrain(neighbor2, sample, secondaryStrength, secondaryDampingShift) + + Constrain(neighbor3, sample, secondaryStrength, secondaryDampingShift); + + sum += constrained * (tap == 0 ? secondaryNearWeight : secondaryFarWeight); + if (clippingRequired) + { + minimum = Vector128.Min(minimum, Vector128.Min(Vector128.Min(neighbor0, neighbor1), Vector128.Min(neighbor2, neighbor3))); + maximum = MaximumIgnoringSentinel(maximum, neighbor0, sentinel); + maximum = MaximumIgnoringSentinel(maximum, neighbor1, sentinel); + maximum = MaximumIgnoringSentinel(maximum, neighbor2, sentinel); + maximum = MaximumIgnoringSentinel(maximum, neighbor3, sentinel); + } + } + } + + // The sign lane contributes the one-unit correction required by AV1's asymmetric rounding for negative sums. + Vector128 correction = (sum >> 15) & one; + Vector128 filtered = sample + ((sum + rounding - correction) >> 4); + if (clippingRequired) + { + filtered = Vector128.Min(Vector128.Max(filtered, minimum), maximum); + } + + TOutputOperator.StoreVector(ref destination, destinationOffset + (row * destinationStride), filtered, blockWidth); + } + } + + /// + /// Applies the scalar CDEF fallback to one block. + /// + /// The destination sample storage type. + /// The storage-specific output operator. + /// The enabled directional-tap operator. + /// The first element in the bordered source plane. + /// The offset of the block's top-left source sample. + /// The number of samples between adjacent source rows. + /// The first element in the destination plane. + /// The offset of the block's top-left destination sample. + /// The number of samples between adjacent destination rows. + /// The bit-depth-scaled primary strength. + /// The bit-depth-scaled secondary strength. + /// The zero-based AV1 direction index. + /// The damping value applied to primary taps. + /// The damping value applied to secondary taps. + /// The number of bits above eight-bit sample precision. + /// The block width in plane samples. + /// The block height in plane samples. + private static void FilterBlockScalar( + ref ushort source, + int sourceOffset, + int sourceStride, + ref TSample destination, + int destinationOffset, + int destinationStride, + int primaryStrength, + int secondaryStrength, + int direction, + int primaryDamping, + int secondaryDamping, + int coefficientShift, + int blockWidth, + int blockHeight) + where TSample : unmanaged + where TOutputOperator : struct, IOutputOperator + where TFilterOperator : struct, IFilterOperator + { + bool clippingRequired = TFilterOperator.EnablePrimary && TFilterOperator.EnableSecondary; + int primaryDampingShift = TFilterOperator.EnablePrimary ? Math.Max(0, primaryDamping - Av1Math.MostSignificantBit((uint)primaryStrength)) : 0; + int secondaryDampingShift = TFilterOperator.EnableSecondary ? Math.Max(0, secondaryDamping - Av1Math.MostSignificantBit((uint)secondaryStrength)) : 0; + int primaryTapSet = (primaryStrength >> coefficientShift) & 1; + int primaryNearOffset = TFilterOperator.EnablePrimary ? GetDirectionOffset(direction, 0, sourceStride) : 0; + int primaryFarOffset = TFilterOperator.EnablePrimary ? GetDirectionOffset(direction, 1, sourceStride) : 0; + int secondaryNearOffset0 = TFilterOperator.EnableSecondary ? GetDirectionOffset((direction + 2) & 7, 0, sourceStride) : 0; + int secondaryFarOffset0 = TFilterOperator.EnableSecondary ? GetDirectionOffset((direction + 2) & 7, 1, sourceStride) : 0; + int secondaryNearOffset1 = TFilterOperator.EnableSecondary ? GetDirectionOffset((direction + 6) & 7, 0, sourceStride) : 0; + int secondaryFarOffset1 = TFilterOperator.EnableSecondary ? GetDirectionOffset((direction + 6) & 7, 1, sourceStride) : 0; + + for (int row = 0; row < blockHeight; row++) + { + for (int column = 0; column < blockWidth; column++) + { + int sourceIndex = sourceOffset + (row * sourceStride) + column; + int sample = Unsafe.Add(ref source, sourceIndex); + int sum = 0; + int minimum = sample; + int maximum = sample; + + for (int tap = 0; tap < 2; tap++) + { + if (TFilterOperator.EnablePrimary) + { + int offset = tap == 0 ? primaryNearOffset : primaryFarOffset; + int neighbor0 = Unsafe.Add(ref source, sourceIndex + offset); + int neighbor1 = Unsafe.Add(ref source, sourceIndex - offset); + int weight = primaryTapSet == 0 ? (tap == 0 ? 4 : 2) : 3; + sum += weight * Constrain(neighbor0 - sample, primaryStrength, primaryDampingShift); + sum += weight * Constrain(neighbor1 - sample, primaryStrength, primaryDampingShift); + + if (clippingRequired) + { + maximum = neighbor0 != VeryLarge ? Math.Max(maximum, neighbor0) : maximum; + maximum = neighbor1 != VeryLarge ? Math.Max(maximum, neighbor1) : maximum; + minimum = Math.Min(minimum, Math.Min(neighbor0, neighbor1)); + } + } + + if (TFilterOperator.EnableSecondary) + { + int offset0 = tap == 0 ? secondaryNearOffset0 : secondaryFarOffset0; + int offset1 = tap == 0 ? secondaryNearOffset1 : secondaryFarOffset1; + int neighbor0 = Unsafe.Add(ref source, sourceIndex + offset0); + int neighbor1 = Unsafe.Add(ref source, sourceIndex - offset0); + int neighbor2 = Unsafe.Add(ref source, sourceIndex + offset1); + int neighbor3 = Unsafe.Add(ref source, sourceIndex - offset1); + int weight = tap == 0 ? 2 : 1; + sum += weight * Constrain(neighbor0 - sample, secondaryStrength, secondaryDampingShift); + sum += weight * Constrain(neighbor1 - sample, secondaryStrength, secondaryDampingShift); + sum += weight * Constrain(neighbor2 - sample, secondaryStrength, secondaryDampingShift); + sum += weight * Constrain(neighbor3 - sample, secondaryStrength, secondaryDampingShift); + + if (clippingRequired) + { + maximum = neighbor0 != VeryLarge ? Math.Max(maximum, neighbor0) : maximum; + maximum = neighbor1 != VeryLarge ? Math.Max(maximum, neighbor1) : maximum; + maximum = neighbor2 != VeryLarge ? Math.Max(maximum, neighbor2) : maximum; + maximum = neighbor3 != VeryLarge ? Math.Max(maximum, neighbor3) : maximum; + minimum = Math.Min(minimum, Math.Min(Math.Min(neighbor0, neighbor1), Math.Min(neighbor2, neighbor3))); + } + } + } + + int filtered = sample + ((8 + sum - (sum < 0 ? 1 : 0)) >> 4); + TOutputOperator.StoreScalar( + ref destination, + destinationOffset + (row * destinationStride) + column, + clippingRequired ? Av1Math.Clip3(minimum, maximum, filtered) : filtered); + } + } + } + + /// + /// Loads four or eight contiguous source samples into signed 16-bit lanes. + /// + /// The first element in the bordered source plane. + /// The offset of the first sample to load. + /// The number of valid samples. + /// The samples in the low vector lanes. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector128 LoadSamples(ref ushort source, int offset, int count) + { + if (count == 8) + { + return Vector128.LoadUnsafe(ref source, (nuint)offset).AsInt16(); + } + + // Four-wide chroma blocks load exactly 64 bits so the final block never reads beyond its two-sample sentinel border. + ref byte sourceBytes = ref Unsafe.As(ref Unsafe.Add(ref source, offset)); + return Vector128.Create(Unsafe.ReadUnaligned(ref sourceBytes), 0UL).AsInt16(); + } + + /// + /// Widens the remaining samples in one eight-bit source row into the CDEF working plane. + /// + /// The first element in the source plane. + /// The offset of the first source sample in the row. + /// The first element in the destination plane. + /// The offset of the first destination sample in the row. + /// The first column not already processed by a wider vector path. + /// The rectangle width in samples. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static void CopyRemainingSamples( + ref byte source, + int sourceRow, + ref ushort destination, + int destinationRow, + int column, + int width) + { + if (Vector128.IsHardwareAccelerated) + { + for (; column <= width - Vector64.Count; column += Vector64.Count) + { + ref byte sourceBytes = ref Unsafe.Add(ref source, sourceRow + column); + ulong packed = Unsafe.ReadUnaligned(ref sourceBytes); + Vector128 widened = Vector128.WidenLower(Vector128.CreateScalarUnsafe(packed).AsByte()); + widened.StoreUnsafe(ref destination, (nuint)(destinationRow + column)); + } + } + + for (; column < width; column++) + { + Unsafe.Add(ref destination, destinationRow + column) = Unsafe.Add(ref source, sourceRow + column); + } + } + + /// + /// Loads two 8-wide rows or four 4-wide rows into the independent row groups of one 256-bit vector. + /// + /// The first element in the bordered source plane. + /// The offset of the first sample to load. + /// The number of samples between adjacent rows. + /// The number of valid samples in each row. + /// The packed source rows. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector256 LoadRows(ref ushort source, int offset, int stride, int width) + { + if (width == 8) + { + Vector128 firstRow = Vector128.LoadUnsafe(ref source, (nuint)offset).AsInt16(); + Vector128 secondRow = Vector128.LoadUnsafe(ref source, (nuint)(offset + stride)).AsInt16(); + return Vector256.Create(firstRow, secondRow); + } + + // Four-row batches use one 64-bit load per row. Pairing two rows in each 128-bit lane preserves the exact row + // boundaries required by the reference decoder's lane-local shifts while avoiding reads beyond the frame sentinel border. + Vector64 row0 = LoadSamples(ref source, offset, 4).GetLower(); + Vector64 row1 = LoadSamples(ref source, offset + stride, 4).GetLower(); + Vector64 row2 = LoadSamples(ref source, offset + (2 * stride), 4).GetLower(); + Vector64 row3 = LoadSamples(ref source, offset + (3 * stride), 4).GetLower(); + return Vector256.Create(Vector128.Create(row0, row1), Vector128.Create(row2, row3)); + } + + /// + /// Stores two 8-wide rows or four 4-wide rows from one packed result vector. + /// + /// The destination sample storage type. + /// The storage-specific output operator. + /// The first element in the destination plane. + /// The offset of the first row to write. + /// The number of samples between adjacent destination rows. + /// The packed filtered rows. + /// The number of valid samples in each row. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static void StoreRows( + ref TSample destination, + int offset, + int stride, + Vector256 value, + int width) + where TSample : unmanaged + where TOutputOperator : struct, IOutputOperator + { + Vector128 lower = value.GetLower(); + Vector128 upper = value.GetUpper(); + if (width == 8) + { + TOutputOperator.StoreVector(ref destination, offset, lower, 8); + TOutputOperator.StoreVector(ref destination, offset + stride, upper, 8); + return; + } + + TOutputOperator.StoreVector(ref destination, offset, lower, 4); + TOutputOperator.StoreVector(ref destination, offset + stride, Vector128.Create(lower.GetUpper(), Vector64.Zero), 4); + TOutputOperator.StoreVector(ref destination, offset + (2 * stride), upper, 4); + TOutputOperator.StoreVector(ref destination, offset + (3 * stride), Vector128.Create(upper.GetUpper(), Vector64.Zero), 4); + } + + /// + /// Limits packed neighbor differences according to one filter strength and its pre-adjusted damping shift. + /// + /// The neighboring samples. + /// The current samples. + /// The bit-depth-scaled filter strength. + /// The damping shift after accounting for the threshold magnitude. + /// The signed constrained differences. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector128 Constrain(Vector128 neighbor, Vector128 sample, int threshold, int dampingShift) + { + Vector128 difference = neighbor - sample; + Vector128 sign = difference >> 15; + Vector128 magnitude = Vector128.Abs(difference).AsUInt16(); + Vector128 remaining = Vector128.SubtractSaturate(Vector128.Create((ushort)threshold), magnitude >> dampingShift); + Vector128 constrained = Vector128.Min(magnitude, remaining).AsInt16(); + + // Adding the all-bits sign before XOR reproduces sign(value) * magnitude without a branch or a multiply. + return (constrained + sign) ^ sign; + } + + /// + /// Limits packed neighbor differences for several independent output rows. + /// + /// The neighboring samples. + /// The current samples. + /// The bit-depth-scaled filter strength. + /// The damping shift after accounting for the threshold magnitude. + /// The signed constrained differences. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector256 Constrain(Vector256 neighbor, Vector256 sample, int threshold, int dampingShift) + { + Vector256 difference = neighbor - sample; + Vector256 sign = difference >> 15; + Vector256 magnitude = Vector256.Abs(difference).AsUInt16(); + Vector256 remaining = Vector256.SubtractSaturate(Vector256.Create((ushort)threshold), magnitude >> dampingShift); + Vector256 constrained = Vector256.Min(magnitude, remaining).AsInt16(); + return (constrained + sign) ^ sign; + } + + /// + /// Updates a packed maximum while treating unavailable-neighbor sentinels as zero. + /// + /// The current per-lane maximum. + /// The candidate neighboring samples. + /// The unavailable-neighbor sentinel in every lane. + /// The updated per-lane maximum. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector128 MaximumIgnoringSentinel(Vector128 maximum, Vector128 candidate, Vector128 sentinel) + { + Vector128 available = Vector128.ConditionalSelect(Vector128.Equals(candidate, sentinel), Vector128.Zero, candidate); + return Vector128.Max(maximum, available); + } + + /// + /// Updates packed row maxima while treating unavailable-neighbor sentinels as zero. + /// + /// The current per-lane maximum. + /// The candidate neighboring samples. + /// The unavailable-neighbor sentinel in every lane. + /// The updated per-lane maximum. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector256 MaximumIgnoringSentinel(Vector256 maximum, Vector256 candidate, Vector256 sentinel) + { + Vector256 available = Vector256.ConditionalSelect(Vector256.Equals(candidate, sentinel), Vector256.Zero, candidate); + return Vector256.Max(maximum, available); + } + + /// + /// Limits one scalar neighbor difference according to a filter strength and damping value. + /// + /// The signed difference from the current sample. + /// The bit-depth-scaled filter strength. + /// The damping shift after accounting for the threshold magnitude. + /// The signed constrained difference. + private static int Constrain(int difference, int threshold, int dampingShift) + { + int magnitude = Math.Abs(difference); + int constrained = Av1Math.Clip3(0, magnitude, threshold - (magnitude >> dampingShift)); + return difference < 0 ? -constrained : constrained; + } + + /// + /// Converts a direction and tap number to a signed plane-buffer offset. + /// + /// The zero-based AV1 direction index. + /// The zero-based distance index. + /// The number of samples between adjacent rows. + /// The signed sample offset. + private static int GetDirectionOffset(int direction, int tap, int stride) => direction switch + { + 0 => tap == 0 ? -stride + 1 : (-2 * stride) + 2, + 1 => tap == 0 ? 1 : -stride + 2, + 2 => tap == 0 ? 1 : 2, + 3 => tap == 0 ? 1 : stride + 2, + 4 => tap == 0 ? stride + 1 : (2 * stride) + 2, + 5 => tap == 0 ? stride : (2 * stride) + 1, + 6 => tap == 0 ? stride : 2 * stride, + _ => tap == 0 ? stride : (2 * stride) - 1 + }; +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/FilmGrain/Av1FilmGrainDecoder.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/FilmGrain/Av1FilmGrainDecoder.cs new file mode 100644 index 0000000000..6bcacdf967 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/FilmGrain/Av1FilmGrainDecoder.cs @@ -0,0 +1,1232 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Buffers; +using System.Runtime.CompilerServices; +using System.Runtime.InteropServices; +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Memory; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.FilmGrain; + +/// +/// Synthesizes the film-grain signal carried by one displayed AV1 image frame. +/// +/// +/// Grain is applied after loop restoration to presentation samples and is not part of reference reconstruction. +/// +internal sealed class Av1FilmGrainDecoder +{ + /// + /// The luma width and height of one independently selected grain block. + /// + private const int LumaSubblockSize = 32; + + /// + /// The maximum autoregressive lag reserved around generated grain templates. + /// + private const int AutoregressivePadding = 3; + + /// + /// The template padding that precedes the autoregressive stabilization region. + /// + private const int TemplatePadding = 3; + + /// + /// The number of random bits used to index the normative Gaussian sequence. + /// + private const int GaussianIndexBits = 11; + + /// + /// The sequence-level bit-depth and chroma-sampling configuration. + /// + private readonly ObuSequenceHeader sequenceHeader; + + /// + /// The frame-level film-grain synthesis parameters. + /// + private readonly ObuFrameHeader frameHeader; + + /// + /// The restored planar samples that receive the synthesized grain. + /// + private readonly Av1FrameBuffer frameBuffer; + + /// + /// Initializes a new instance of the class. + /// + /// The sequence header defining bit depth and chroma sampling. + /// The frame header containing the complete grain parameters. + /// The restored frame samples to which grain is added. + public Av1FilmGrainDecoder( + ObuSequenceHeader sequenceHeader, + ObuFrameHeader frameHeader, + Av1FrameBuffer frameBuffer) + { + this.sequenceHeader = sequenceHeader; + this.frameHeader = frameHeader; + this.frameBuffer = frameBuffer; + } + + /// + /// Adds the signaled film grain to the visible luma and chroma samples. + /// + public void DecodeFrame() + { + ObuFilmGrainParameters parameters = this.frameHeader.FilmGrainParameters; + + // Film grain is a presentation process. A frame which does not signal it must retain the restored samples + // byte-for-byte, so the decoder does not allocate templates or touch the padded frame planes in this case. + if (!parameters.ApplyGrain) + { + return; + } + + ObuColorConfig colorConfig = this.sequenceHeader.ColorConfig; + bool isMonochrome = colorConfig.IsMonochrome; + int subsamplingX = !isMonochrome && colorConfig.SubSamplingX ? 1 : 0; + int subsamplingY = !isMonochrome && colorConfig.SubSamplingY ? 1 : 0; + int visibleWidth = this.frameBuffer.Width; + int visibleHeight = this.frameBuffer.Height; + + // Grain blocks are traversed in half-resolution luma coordinates and expanded in 2x2 sample groups. + // Replicating an odd final row or column makes that traversal complete without changing the visible extent. + int alignedWidth = Av1Math.AlignPowerOf2(visibleWidth, 1); + int alignedHeight = Av1Math.AlignPowerOf2(visibleHeight, 1); + + Buffer2D lumaBuffer = this.frameBuffer.GetPlaneBuffer(Av1Plane.Y); + + // Frame planes are allocated as bytes even for high-bit-depth pictures. Convert their byte strides to + // native sample strides once so every later offset is expressed consistently in samples. + int lumaStride = lumaBuffer.Width / this.frameBuffer.BytesPerSample; + int chromaStride = isMonochrome + ? 0 + : this.frameBuffer.GetPlaneBuffer(Av1Plane.U).Width / this.frameBuffer.BytesPerSample; + + // Closing ApplyGrain over byte or ushort keeps synthesis in the frame buffer's native representation. + // This avoids an intermediate converted image while allowing the JIT to remove the sample-type branches. + if (this.frameBuffer.BytesPerSample == 2) + { + Span luma = GetPlaneSamples( + lumaBuffer, + this.frameBuffer.OriginX, + this.frameBuffer.OriginY); + + Span cb = isMonochrome + ? Span.Empty + : GetPlaneSamples( + this.frameBuffer.GetPlaneBuffer(Av1Plane.U), + this.frameBuffer.OriginX >> subsamplingX, + this.frameBuffer.OriginY >> subsamplingY); + + Span cr = isMonochrome + ? Span.Empty + : GetPlaneSamples( + this.frameBuffer.GetPlaneBuffer(Av1Plane.V), + this.frameBuffer.OriginX >> subsamplingX, + this.frameBuffer.OriginY >> subsamplingY); + + ExtendPlane(luma, lumaStride, visibleWidth, visibleHeight, alignedWidth, alignedHeight); + if (!isMonochrome) + { + int visibleChromaWidth = Av1Math.DivideLog2Ceiling(visibleWidth, subsamplingX); + int visibleChromaHeight = Av1Math.DivideLog2Ceiling(visibleHeight, subsamplingY); + int alignedChromaWidth = alignedWidth >> subsamplingX; + int alignedChromaHeight = alignedHeight >> subsamplingY; + ExtendPlane(cb, chromaStride, visibleChromaWidth, visibleChromaHeight, alignedChromaWidth, alignedChromaHeight); + ExtendPlane(cr, chromaStride, visibleChromaWidth, visibleChromaHeight, alignedChromaWidth, alignedChromaHeight); + } + + this.ApplyGrain( + parameters, + luma, + cb, + cr, + alignedWidth, + alignedHeight, + lumaStride, + chromaStride, + subsamplingX, + subsamplingY, + isMonochrome); + } + else + { + Span luma = GetPlaneSamples( + lumaBuffer, + this.frameBuffer.OriginX, + this.frameBuffer.OriginY); + + Span cb = isMonochrome + ? Span.Empty + : GetPlaneSamples( + this.frameBuffer.GetPlaneBuffer(Av1Plane.U), + this.frameBuffer.OriginX >> subsamplingX, + this.frameBuffer.OriginY >> subsamplingY); + + Span cr = isMonochrome + ? Span.Empty + : GetPlaneSamples( + this.frameBuffer.GetPlaneBuffer(Av1Plane.V), + this.frameBuffer.OriginX >> subsamplingX, + this.frameBuffer.OriginY >> subsamplingY); + + ExtendPlane(luma, lumaStride, visibleWidth, visibleHeight, alignedWidth, alignedHeight); + if (!isMonochrome) + { + int visibleChromaWidth = Av1Math.DivideLog2Ceiling(visibleWidth, subsamplingX); + int visibleChromaHeight = Av1Math.DivideLog2Ceiling(visibleHeight, subsamplingY); + int alignedChromaWidth = alignedWidth >> subsamplingX; + int alignedChromaHeight = alignedHeight >> subsamplingY; + ExtendPlane(cb, chromaStride, visibleChromaWidth, visibleChromaHeight, alignedChromaWidth, alignedChromaHeight); + ExtendPlane(cr, chromaStride, visibleChromaWidth, visibleChromaHeight, alignedChromaWidth, alignedChromaHeight); + } + + this.ApplyGrain( + parameters, + luma, + cb, + cr, + alignedWidth, + alignedHeight, + lumaStride, + chromaStride, + subsamplingX, + subsamplingY, + isMonochrome); + } + } + + /// + /// Gets a plane span beginning at its first visible sample. + /// + /// The native eight-bit or high-bit-depth sample type. + /// The padded plane allocation. + /// The horizontal visible origin in samples. + /// The vertical visible origin in rows. + /// The sample span beginning at the visible origin and retaining the padded row stride. + private static Span GetPlaneSamples(Buffer2D buffer, int originX, int originY) + where TSample : unmanaged + { + Span samples = MemoryMarshal.Cast(buffer.DangerousGetSingleSpan()); + int stride = buffer.Width / Unsafe.SizeOf(); + + // The returned span intentionally retains the allocation beyond the visible rectangle. Film-grain overlap + // and odd-dimension extension use the frame buffer's existing right and bottom padding through this stride. + return samples[((originY * stride) + originX)..]; + } + + /// + /// Replicates the last visible row and column when film-grain block traversal requires even dimensions. + /// + /// The native eight-bit or high-bit-depth sample type. + /// The plane span beginning at its visible origin. + /// The number of samples between adjacent rows. + /// The visible plane width. + /// The visible plane height. + /// The even width consumed by synthesis. + /// The even height consumed by synthesis. + private static void ExtendPlane( + Span plane, + int stride, + int visibleWidth, + int visibleHeight, + int alignedWidth, + int alignedHeight) + where TSample : unmanaged + { + if (visibleWidth != alignedWidth) + { + // The synthetic column is consumed only as the partner of the final visible sample in a 2x2 group. + for (int row = 0; row < visibleHeight; row++) + { + int rowOffset = row * stride; + plane[rowOffset + visibleWidth] = plane[rowOffset + visibleWidth - 1]; + } + } + + if (visibleHeight != alignedHeight) + { + // Span.CopyTo uses the runtime's optimized bulk-copy path and preserves the already replicated edge column. + plane.Slice((visibleHeight - 1) * stride, alignedWidth) + .CopyTo(plane.Slice(visibleHeight * stride, alignedWidth)); + } + } + + /// + /// Generates reusable grain templates and adds selected blocks to one restored frame. + /// + /// The native eight-bit or high-bit-depth sample type. + /// The complete self-contained frame grain parameters. + /// The luma plane beginning at its visible origin. + /// The first chroma plane, or an empty span for monochrome input. + /// The second chroma plane, or an empty span for monochrome input. + /// The even luma width processed by synthesis. + /// The even luma height processed by synthesis. + /// The luma row stride in samples. + /// The chroma row stride in samples. + /// The horizontal chroma subsampling shift. + /// The vertical chroma subsampling shift. + /// Whether the frame has no chroma planes. + private void ApplyGrain( + ObuFilmGrainParameters parameters, + Span luma, + Span cb, + Span cr, + int width, + int height, + int lumaStride, + int chromaStride, + int subsamplingX, + int subsamplingY, + bool isMonochrome) + where TSample : unmanaged + { + // A template contains a selectable 64x64 luma region, the maximum three-sample autoregressive history, + // and the fixed margins required by the block-offset process. Chroma dimensions contract with sampling. + int chromaSubblockHeight = LumaSubblockSize >> subsamplingY; + int chromaSubblockWidth = LumaSubblockSize >> subsamplingX; + int lumaBlockHeight = TemplatePadding + (2 * AutoregressivePadding) + (2 * LumaSubblockSize); + int lumaBlockWidth = TemplatePadding + (2 * AutoregressivePadding) + (2 * LumaSubblockSize) + + (2 * AutoregressivePadding) + TemplatePadding; + + int chromaBlockHeight = TemplatePadding + ((2 >> subsamplingY) * AutoregressivePadding) + + (2 * chromaSubblockHeight); + + int chromaBlockWidth = TemplatePadding + ((2 >> subsamplingX) * AutoregressivePadding) + + (2 * chromaSubblockWidth) + ((2 >> subsamplingX) * AutoregressivePadding) + TemplatePadding; + + int lumaGrainLength = lumaBlockHeight * lumaBlockWidth; + int chromaGrainLength = isMonochrome ? 0 : chromaBlockHeight * chromaBlockWidth; + int scalingLength = isMonochrome ? 256 : 768; + + // Overlap keeps the outgoing two luma rows/columns, or their subsampled chroma equivalents, until the + // adjacent block is selected. Frames without overlap do not reserve these line and column workspaces. + int lumaLineLength = parameters.OverlapFlag ? lumaStride * 2 : 0; + int chromaLineLength = parameters.OverlapFlag && !isMonochrome + ? chromaStride * (2 >> subsamplingY) + : 0; + + int lumaColumnLength = parameters.OverlapFlag ? (LumaSubblockSize + 2) * 2 : 0; + int chromaColumnLength = parameters.OverlapFlag && !isMonochrome + ? (chromaSubblockHeight + (2 >> subsamplingY)) * (2 >> subsamplingX) + : 0; + + int scratchLength = scalingLength + lumaGrainLength + (2 * chromaGrainLength) + + lumaLineLength + (2 * chromaLineLength) + lumaColumnLength + (2 * chromaColumnLength); + + // All frame-lifetime film-grain state shares one allocator-backed owner. The slices below are disjoint, + // and their logical ordering mirrors lookup tables, templates, horizontal boundaries, then vertical boundaries. + using IMemoryOwner scratchOwner = this.frameBuffer.MemoryAllocator.Allocate(scratchLength); + Span scratch = scratchOwner.GetSpan()[..scratchLength]; + int scratchOffset = 0; + + Span scalingY = scratch.Slice(scratchOffset, 256); + scratchOffset += 256; + Span scalingCb = isMonochrome ? Span.Empty : scratch.Slice(scratchOffset, 256); + scratchOffset += scalingCb.Length; + Span scalingCr = isMonochrome ? Span.Empty : scratch.Slice(scratchOffset, 256); + scratchOffset += scalingCr.Length; + Span lumaGrain = scratch.Slice(scratchOffset, lumaGrainLength); + scratchOffset += lumaGrain.Length; + Span cbGrain = isMonochrome ? Span.Empty : scratch.Slice(scratchOffset, chromaGrainLength); + scratchOffset += cbGrain.Length; + Span crGrain = isMonochrome ? Span.Empty : scratch.Slice(scratchOffset, chromaGrainLength); + scratchOffset += crGrain.Length; + Span yLineBuffer = parameters.OverlapFlag ? scratch.Slice(scratchOffset, lumaLineLength) : Span.Empty; + scratchOffset += yLineBuffer.Length; + Span cbLineBuffer = parameters.OverlapFlag && !isMonochrome + ? scratch.Slice(scratchOffset, chromaLineLength) + : Span.Empty; + + scratchOffset += cbLineBuffer.Length; + Span crLineBuffer = parameters.OverlapFlag && !isMonochrome + ? scratch.Slice(scratchOffset, chromaLineLength) + : Span.Empty; + + scratchOffset += crLineBuffer.Length; + Span yColumnBuffer = parameters.OverlapFlag + ? scratch.Slice(scratchOffset, lumaColumnLength) + : Span.Empty; + + scratchOffset += yColumnBuffer.Length; + Span cbColumnBuffer = parameters.OverlapFlag && !isMonochrome + ? scratch.Slice(scratchOffset, chromaColumnLength) + : Span.Empty; + + scratchOffset += cbColumnBuffer.Length; + Span crColumnBuffer = parameters.OverlapFlag && !isMonochrome + ? scratch.Slice(scratchOffset, chromaColumnLength) + : Span.Empty; + + // the reference decoder zero-initializes the lookup structure before expanding control points. This matters + // when chroma scaling is inherited from an empty luma scaling function. + scalingY.Clear(); + scalingCb.Clear(); + scalingCr.Clear(); + + int bitDepth = this.sequenceHeader.ColorConfig.BitDepth.GetBitCount(); + ushort randomRegister = (ushort)parameters.GrainSeed; + + // Luma consumes the seed's initial pseudo-random sequence. Chroma generation subsequently reinitializes + // the same register with plane-specific row identities so its two templates remain deterministic and distinct. + GenerateLumaGrain( + parameters, + ref randomRegister, + lumaGrain, + lumaBlockHeight, + lumaBlockWidth, + lumaBlockWidth, + bitDepth); + + if (!isMonochrome) + { + GenerateChromaGrain( + parameters, + ref randomRegister, + lumaGrain, + cbGrain, + crGrain, + lumaBlockWidth, + chromaBlockHeight, + chromaBlockWidth, + chromaBlockWidth, + subsamplingX, + subsamplingY, + bitDepth); + } + + // Signaled points describe piecewise-linear functions over the eight-bit domain. High-bit-depth samples + // interpolate between these 256 entries later, rather than allocating larger per-depth lookup tables. + InitializeScalingFunction(parameters.PointYValue, parameters.PointYScaling, (int)parameters.NumYPoints, scalingY); + if (!isMonochrome) + { + if (parameters.ChromaScalingFromLuma) + { + scalingY.CopyTo(scalingCb); + scalingY.CopyTo(scalingCr); + } + else + { + InitializeScalingFunction( + parameters.PointCbValue, + parameters.PointCbScaling, + (int)parameters.NumCbPoints, + scalingCb); + + InitializeScalingFunction( + parameters.PointCrValue, + parameters.PointCrScaling, + (int)parameters.NumCrPoints, + scalingCr); + } + } + + int grainMinimum = -(1 << (bitDepth - 1)); + int grainMaximum = (1 << (bitDepth - 1)) - 1; + bool isIdentityMatrix = this.sequenceHeader.ColorConfig.MatrixCoefficients == ObuMatrixCoefficients.Identity; + + // Coordinates are halved because each iteration owns one 32x32 luma block but all frame offsets are even. + // Keeping the loop in this domain also makes one-unit boundary adjustments represent two luma samples. + for (int halfY = 0; halfY < height / 2; halfY += LumaSubblockSize >> 1) + { + // Block rows restart from a seed mixed with their luma row number. This makes a block's selection + // independent of decoder traversal outside its row while remaining reproducible from the bitstream. + InitializeRandomGenerator(ref randomRegister, halfY << 1, (ushort)parameters.GrainSeed); + + for (int halfX = 0; halfX < width / 2; halfX += LumaSubblockSize >> 1) + { + // The high and low nibbles choose an even luma offset inside the reusable 64x64 template region. + // Chroma offsets use the corresponding subsampled position so all planes share the same selection. + int randomOffset = GetRandomNumber(ref randomRegister, 8); + int offsetX = (randomOffset >> 4) & 15; + int offsetY = randomOffset & 15; + int lumaOffsetY = TemplatePadding + (2 * AutoregressivePadding) + (offsetY << 1); + int lumaOffsetX = TemplatePadding + (2 * AutoregressivePadding) + (offsetX << 1); + int chromaOffsetY = TemplatePadding + ((2 >> subsamplingY) * AutoregressivePadding) + + (offsetY * (2 >> subsamplingY)); + + int chromaOffsetX = TemplatePadding + ((2 >> subsamplingX) * AutoregressivePadding) + + (offsetX * (2 >> subsamplingX)); + + if (parameters.OverlapFlag && halfX != 0) + { + // Blend the incoming template columns with the outgoing columns saved by the block on the left. + // Writing back to the column buffers produces the exact grain region applied at this boundary. + Av1FilmGrainOverlap.Vertical( + yColumnBuffer, + 2, + lumaGrain[((lumaOffsetY * lumaBlockWidth) + lumaOffsetX)..], + lumaBlockWidth, + yColumnBuffer, + 2, + 2, + Math.Min(LumaSubblockSize + 2, height - (halfY << 1)), + grainMinimum, + grainMaximum); + + if (!isMonochrome) + { + int chromaOverlapWidth = 2 >> subsamplingX; + int chromaOverlapHeight = Math.Min( + chromaSubblockHeight + (2 >> subsamplingY), + (height - (halfY << 1)) >> subsamplingY); + + Av1FilmGrainOverlap.Vertical( + cbColumnBuffer, + chromaOverlapWidth, + cbGrain[((chromaOffsetY * chromaBlockWidth) + chromaOffsetX)..], + chromaBlockWidth, + cbColumnBuffer, + chromaOverlapWidth, + chromaOverlapWidth, + chromaOverlapHeight, + grainMinimum, + grainMaximum); + + Av1FilmGrainOverlap.Vertical( + crColumnBuffer, + chromaOverlapWidth, + crGrain[((chromaOffsetY * chromaBlockWidth) + chromaOffsetX)..], + chromaBlockWidth, + crColumnBuffer, + chromaOverlapWidth, + chromaOverlapWidth, + chromaOverlapHeight, + grainMinimum, + grainMaximum); + } + + int rowAdjustment = halfY != 0 ? 1 : 0; + + // The top overlap row, when present, is owned by the horizontal-boundary pass below. Skip it here + // so the corner and vertical boundary are each added to the decoded samples exactly once. + int destinationLumaOffset = (((halfY + rowAdjustment) << 1) * lumaStride) + (halfX << 1); + int destinationChromaOffset = (((halfY + rowAdjustment) << (1 - subsamplingY)) * chromaStride) + + (halfX << (1 - subsamplingX)); + + int columnGrainOffset = rowAdjustment * (2 - subsamplingY) * (2 - subsamplingX); + Span destinationCb = isMonochrome + ? Span.Empty + : cb[destinationChromaOffset..]; + + Span destinationCr = isMonochrome + ? Span.Empty + : cr[destinationChromaOffset..]; + + Span columnCbGrain = isMonochrome + ? Span.Empty + : cbColumnBuffer[columnGrainOffset..]; + + Span columnCrGrain = isMonochrome + ? Span.Empty + : crColumnBuffer[columnGrainOffset..]; + + Av1FilmGrainNoise.Apply( + parameters, + scalingY, + scalingCb, + scalingCr, + luma[destinationLumaOffset..], + destinationCb, + destinationCr, + lumaStride, + chromaStride, + yColumnBuffer[(rowAdjustment * 4)..], + columnCbGrain, + columnCrGrain, + 2, + 2 - subsamplingX, + Math.Min(LumaSubblockSize >> 1, (height / 2) - halfY) - rowAdjustment, + 1, + bitDepth, + subsamplingX, + subsamplingY, + isMonochrome, + isIdentityMatrix); + } + + if (parameters.OverlapFlag && halfY != 0) + { + if (halfX != 0) + { + // At an interior corner, first combine the saved top boundary with the already blended left + // boundary. The resulting corner is then part of the horizontal boundary applied below. + Av1FilmGrainOverlap.Horizontal( + yLineBuffer[(halfX << 1)..], + lumaStride, + yColumnBuffer, + 2, + yLineBuffer[(halfX << 1)..], + lumaStride, + 2, + 2, + grainMinimum, + grainMaximum); + + if (!isMonochrome) + { + int chromaOverlapWidth = 2 >> subsamplingX; + int chromaOverlapHeight = 2 >> subsamplingY; + int chromaLineOffset = halfX * chromaOverlapWidth; + Av1FilmGrainOverlap.Horizontal( + cbLineBuffer[chromaLineOffset..], + chromaStride, + cbColumnBuffer, + chromaOverlapWidth, + cbLineBuffer[chromaLineOffset..], + chromaStride, + chromaOverlapWidth, + chromaOverlapHeight, + grainMinimum, + grainMaximum); + + Av1FilmGrainOverlap.Horizontal( + crLineBuffer[chromaLineOffset..], + chromaStride, + crColumnBuffer, + chromaOverlapWidth, + crLineBuffer[chromaLineOffset..], + chromaStride, + chromaOverlapWidth, + chromaOverlapHeight, + grainMinimum, + grainMaximum); + } + } + + int overlappedColumn = halfX != 0 ? halfX + 1 : 0; + int templateColumnAdjustment = halfX != 0 ? 2 : 0; + + // The horizontal boundary excludes the two luma columns already emitted by vertical overlap. + // The same adjustment contracts to one column for horizontally subsampled chroma. + int horizontalWidth = Math.Min( + LumaSubblockSize - templateColumnAdjustment, + width - (overlappedColumn << 1)); + + Av1FilmGrainOverlap.Horizontal( + yLineBuffer[(overlappedColumn << 1)..], + lumaStride, + lumaGrain[((lumaOffsetY * lumaBlockWidth) + lumaOffsetX + templateColumnAdjustment)..], + lumaBlockWidth, + yLineBuffer[(overlappedColumn << 1)..], + lumaStride, + horizontalWidth, + 2, + grainMinimum, + grainMaximum); + + if (!isMonochrome) + { + int chromaColumnAdjustment = halfX != 0 ? 2 >> subsamplingX : 0; + int chromaDestinationOffset = overlappedColumn << (1 - subsamplingX); + int chromaWidth = Math.Min( + chromaSubblockWidth - chromaColumnAdjustment, + (width - (overlappedColumn << 1)) >> subsamplingX); + + Av1FilmGrainOverlap.Horizontal( + cbLineBuffer[chromaDestinationOffset..], + chromaStride, + cbGrain[((chromaOffsetY * chromaBlockWidth) + chromaOffsetX + chromaColumnAdjustment)..], + chromaBlockWidth, + cbLineBuffer[chromaDestinationOffset..], + chromaStride, + chromaWidth, + 2 >> subsamplingY, + grainMinimum, + grainMaximum); + + Av1FilmGrainOverlap.Horizontal( + crLineBuffer[chromaDestinationOffset..], + chromaStride, + crGrain[((chromaOffsetY * chromaBlockWidth) + chromaOffsetX + chromaColumnAdjustment)..], + chromaBlockWidth, + crLineBuffer[chromaDestinationOffset..], + chromaStride, + chromaWidth, + 2 >> subsamplingY, + grainMinimum, + grainMaximum); + } + + int boundaryLumaOffset = ((halfY << 1) * lumaStride) + (halfX << 1); + int boundaryChromaOffset = ((halfY << (1 - subsamplingY)) * chromaStride) + + (halfX << (1 - subsamplingX)); + + Span boundaryCb = isMonochrome + ? Span.Empty + : cb[boundaryChromaOffset..]; + + Span boundaryCr = isMonochrome + ? Span.Empty + : cr[boundaryChromaOffset..]; + + Span boundaryCbGrain = isMonochrome + ? Span.Empty + : cbLineBuffer[(halfX << (1 - subsamplingX))..]; + + Span boundaryCrGrain = isMonochrome + ? Span.Empty + : crLineBuffer[(halfX << (1 - subsamplingX))..]; + + // Apply the completed top boundary as a one-unit half-height strip, which is two luma rows and + // one or two chroma rows depending on vertical subsampling. + Av1FilmGrainNoise.Apply( + parameters, + scalingY, + scalingCb, + scalingCr, + luma[boundaryLumaOffset..], + boundaryCb, + boundaryCr, + lumaStride, + chromaStride, + yLineBuffer[(halfX << 1)..], + boundaryCbGrain, + boundaryCrGrain, + lumaStride, + chromaStride, + 1, + Math.Min(LumaSubblockSize >> 1, (width / 2) - halfX), + bitDepth, + subsamplingX, + subsamplingY, + isMonochrome, + isIdentityMatrix); + } + + int interiorRowAdjustment = parameters.OverlapFlag && halfY != 0 ? 1 : 0; + int interiorColumnAdjustment = parameters.OverlapFlag && halfX != 0 ? 1 : 0; + + // Move both the destination and template origins past boundary strips already applied above. This + // leaves a disjoint interior rectangle, including clipped partial blocks at the right and bottom edges. + int lumaGrainOffset = ((lumaOffsetY + (interiorRowAdjustment << 1)) * lumaBlockWidth) + + lumaOffsetX + (interiorColumnAdjustment << 1); + + int chromaGrainOffset = ((chromaOffsetY + + (interiorRowAdjustment << (1 - subsamplingY))) * chromaBlockWidth) + + chromaOffsetX + (interiorColumnAdjustment << (1 - subsamplingX)); + + int interiorLumaOffset = (((halfY + interiorRowAdjustment) << 1) * lumaStride) + + ((halfX + interiorColumnAdjustment) << 1); + + int interiorChromaOffset = (((halfY + interiorRowAdjustment) << (1 - subsamplingY)) * chromaStride) + + ((halfX + interiorColumnAdjustment) << (1 - subsamplingX)); + + Span interiorCb = isMonochrome ? Span.Empty : cb[interiorChromaOffset..]; + Span interiorCr = isMonochrome ? Span.Empty : cr[interiorChromaOffset..]; + Span interiorCbGrain = isMonochrome ? Span.Empty : cbGrain[chromaGrainOffset..]; + Span interiorCrGrain = isMonochrome ? Span.Empty : crGrain[chromaGrainOffset..]; + + Av1FilmGrainNoise.Apply( + parameters, + scalingY, + scalingCb, + scalingCr, + luma[interiorLumaOffset..], + interiorCb, + interiorCr, + lumaStride, + chromaStride, + lumaGrain[lumaGrainOffset..], + interiorCbGrain, + interiorCrGrain, + lumaBlockWidth, + chromaBlockWidth, + Math.Min(LumaSubblockSize >> 1, (height / 2) - halfY) - interiorRowAdjustment, + Math.Min(LumaSubblockSize >> 1, (width / 2) - halfX) - interiorColumnAdjustment, + bitDepth, + subsamplingX, + subsamplingY, + isMonochrome, + isIdentityMatrix); + + if (parameters.OverlapFlag) + { + if (halfX != 0) + { + // Preserve the completed corner in the line buffers before the column buffers are overwritten. + // It becomes the top input for the block at this column position on the next block row. + CopyArea( + yColumnBuffer[(LumaSubblockSize << 1)..], + 2, + yLineBuffer[(halfX << 1)..], + lumaStride, + 2, + 2); + + if (!isMonochrome) + { + int chromaOverlapWidth = 2 >> subsamplingX; + int chromaOverlapHeight = 2 >> subsamplingY; + int sourceOffset = chromaSubblockHeight << (1 - subsamplingX); + int destinationOffset = halfX << (1 - subsamplingX); + CopyArea( + cbColumnBuffer[sourceOffset..], + chromaOverlapWidth, + cbLineBuffer[destinationOffset..], + chromaStride, + chromaOverlapWidth, + chromaOverlapHeight); + + CopyArea( + crColumnBuffer[sourceOffset..], + chromaOverlapWidth, + crLineBuffer[destinationOffset..], + chromaStride, + chromaOverlapWidth, + chromaOverlapHeight); + } + } + + int lineDestinationColumn = halfX != 0 ? halfX + 1 : 0; + int lineTemplateAdjustment = halfX != 0 ? 2 : 0; + + // Save the template's bottom boundary for the block directly below. Columns already represented + // by the corner are skipped so the line buffer remains one contiguous frame-width boundary. + int lineWidth = Math.Min(LumaSubblockSize, width - (halfX << 1)) - lineTemplateAdjustment; + CopyArea( + lumaGrain[(((lumaOffsetY + LumaSubblockSize) * lumaBlockWidth) + + lumaOffsetX + lineTemplateAdjustment)..], + lumaBlockWidth, + yLineBuffer[(lineDestinationColumn << 1)..], + lumaStride, + lineWidth, + 2); + + if (!isMonochrome) + { + int chromaLineAdjustment = halfX != 0 ? 2 >> subsamplingX : 0; + int chromaLineWidth = Math.Min( + chromaSubblockWidth, + (width - (halfX << 1)) >> subsamplingX) - chromaLineAdjustment; + + int chromaLineDestination = lineDestinationColumn << (1 - subsamplingX); + CopyArea( + cbGrain[(((chromaOffsetY + chromaSubblockHeight) * chromaBlockWidth) + + chromaOffsetX + chromaLineAdjustment)..], + chromaBlockWidth, + cbLineBuffer[chromaLineDestination..], + chromaStride, + chromaLineWidth, + 2 >> subsamplingY); + + CopyArea( + crGrain[(((chromaOffsetY + chromaSubblockHeight) * chromaBlockWidth) + + chromaOffsetX + chromaLineAdjustment)..], + chromaBlockWidth, + crLineBuffer[chromaLineDestination..], + chromaStride, + chromaLineWidth, + 2 >> subsamplingY); + } + + // Finally retain the template's right boundary for the next block in this row. The extra two rows + // extend beyond the nominal block so a later corner blend has both horizontal overlap rows available. + CopyArea( + lumaGrain[((lumaOffsetY * lumaBlockWidth) + lumaOffsetX + LumaSubblockSize)..], + lumaBlockWidth, + yColumnBuffer, + 2, + 2, + Math.Min(LumaSubblockSize + 2, height - (halfY << 1))); + + if (!isMonochrome) + { + int chromaOverlapWidth = 2 >> subsamplingX; + int chromaOverlapHeight = Math.Min( + chromaSubblockHeight + (2 >> subsamplingY), + (height - (halfY << 1)) >> subsamplingY); + + CopyArea( + cbGrain[((chromaOffsetY * chromaBlockWidth) + + chromaOffsetX + chromaSubblockWidth)..], + chromaBlockWidth, + cbColumnBuffer, + chromaOverlapWidth, + chromaOverlapWidth, + chromaOverlapHeight); + + CopyArea( + crGrain[((chromaOffsetY * chromaBlockWidth) + + chromaOffsetX + chromaSubblockWidth)..], + chromaBlockWidth, + crColumnBuffer, + chromaOverlapWidth, + chromaOverlapWidth, + chromaOverlapHeight); + } + } + } + } + } + + /// + /// Generates the luma grain template and applies its autoregressive filter. + /// + /// The frame grain parameters. + /// The pseudo-random register advanced while filling the template. + /// The luma grain template. + /// The template height. + /// The template width. + /// The template row stride. + /// The decoded sample bit depth. + private static void GenerateLumaGrain( + ObuFilmGrainParameters parameters, + ref ushort randomRegister, + Span grain, + int height, + int width, + int stride, + int bitDepth) + { + if (parameters.NumYPoints == 0) + { + // Without a luma scaling function no luma grain is ever applied. A zero template is still required when + // chroma autoregression is present because its optional luma predictor must then contribute zero. + grain.Clear(); + return; + } + + // The fixed Gaussian table has 12-bit amplitude. GrainScaleShift and the decoded bit depth reduce it to + // the signed working range before the causal autoregressive filter changes its spatial correlation. + int gaussianShift = 12 - bitDepth + (int)parameters.GrainScaleShift; + int gaussianRounding = (1 << gaussianShift) >> 1; + ReadOnlySpan gaussian = Av1FilmGrainGaussianSequence.Samples; + for (int row = 0; row < height; row++) + { + for (int column = 0; column < width; column++) + { + int randomIndex = GetRandomNumber(ref randomRegister, GaussianIndexBits); + grain[(row * stride) + column] = (gaussian[randomIndex] + gaussianRounding) >> gaussianShift; + } + } + + int lag = (int)parameters.ArCoeffLag; + int coefficientShift = (int)parameters.ArCoeffShiftMinus6 + 6; + int roundingOffset = 1 << (coefficientShift - 1); + int grainMinimum = -(1 << (bitDepth - 1)); + int grainMaximum = (1 << (bitDepth - 1)) - 1; + ReadOnlySpan coefficients = parameters.ArCoeffsYPlus128; + + // TemplatePadding leaves every lag-one through lag-three predecessor addressable without a boundary branch. + // Raster order guarantees that all rows above and all samples to the left have already been filtered. + for (int row = TemplatePadding; row < height; row++) + { + for (int column = TemplatePadding; column < width - TemplatePadding; column++) + { + int weightedSum = 0; + int coefficientIndex = 0; + + // The AV1 coefficient order walks the complete rows above the current sample before + // the already generated samples to its left on the current row. + for (int relativeRow = -lag; relativeRow < 0; relativeRow++) + { + for (int relativeColumn = -lag; relativeColumn <= lag; relativeColumn++) + { + weightedSum += ((int)coefficients[coefficientIndex++] - 128) * + grain[((row + relativeRow) * stride) + column + relativeColumn]; + } + } + + for (int relativeColumn = -lag; relativeColumn < 0; relativeColumn++) + { + weightedSum += ((int)coefficients[coefficientIndex++] - 128) * + grain[(row * stride) + column + relativeColumn]; + } + + int grainIndex = (row * stride) + column; + grain[grainIndex] = Av1Math.Clamp( + grain[grainIndex] + ((weightedSum + roundingOffset) >> coefficientShift), + grainMinimum, + grainMaximum); + } + } + } + + /// + /// Generates both chroma grain templates and applies their autoregressive filters. + /// + /// The frame grain parameters. + /// The pseudo-random register used for chroma template generation. + /// The completed luma grain template. + /// The first chroma grain template. + /// The second chroma grain template. + /// The luma grain-template row stride. + /// The chroma template height. + /// The chroma template width. + /// The chroma template row stride. + /// The horizontal chroma subsampling shift. + /// The vertical chroma subsampling shift. + /// The decoded sample bit depth. + private static void GenerateChromaGrain( + ObuFilmGrainParameters parameters, + ref ushort randomRegister, + ReadOnlySpan lumaGrain, + Span cbGrain, + Span crGrain, + int lumaStride, + int height, + int width, + int stride, + int subsamplingX, + int subsamplingY, + int bitDepth) + { + bool applyCb = parameters.NumCbPoints != 0 || parameters.ChromaScalingFromLuma; + bool applyCr = parameters.NumCrPoints != 0 || parameters.ChromaScalingFromLuma; + int gaussianShift = 12 - bitDepth + (int)parameters.GrainScaleShift; + int gaussianRounding = (1 << gaussianShift) >> 1; + ReadOnlySpan gaussian = Av1FilmGrainGaussianSequence.Samples; + if (applyCb) + { + // The fixed luma-line identities seven and eleven decorrelate the two chroma pseudo-random sequences + // from each other and from the luma template while retaining deterministic generation from GrainSeed. + InitializeRandomGenerator(ref randomRegister, 7 << 5, (ushort)parameters.GrainSeed); + FillGaussianGrain(ref randomRegister, cbGrain, height, width, stride, gaussian, gaussianShift, gaussianRounding); + } + else + { + cbGrain.Clear(); + } + + if (applyCr) + { + InitializeRandomGenerator(ref randomRegister, 11 << 5, (ushort)parameters.GrainSeed); + FillGaussianGrain(ref randomRegister, crGrain, height, width, stride, gaussian, gaussianShift, gaussianRounding); + } + else + { + crGrain.Clear(); + } + + int lag = (int)parameters.ArCoeffLag; + int coefficientShift = (int)parameters.ArCoeffShiftMinus6 + 6; + int roundingOffset = 1 << (coefficientShift - 1); + int grainMinimum = -(1 << (bitDepth - 1)); + int grainMaximum = (1 << (bitDepth - 1)) - 1; + ReadOnlySpan cbCoefficients = parameters.ArCoeffsCbPlus128; + ReadOnlySpan crCoefficients = parameters.ArCoeffsCrPlus128; + + // Cb and Cr share the same causal predecessor walk, so both accumulators advance one coefficient index + // together. A disabled plane stays zero but does not alter the coefficient ordering of the enabled plane. + for (int row = TemplatePadding; row < height; row++) + { + for (int column = TemplatePadding; column < width - TemplatePadding; column++) + { + int weightedCb = 0; + int weightedCr = 0; + int coefficientIndex = 0; + for (int relativeRow = -lag; relativeRow < 0; relativeRow++) + { + for (int relativeColumn = -lag; relativeColumn <= lag; relativeColumn++) + { + int grainIndex = ((row + relativeRow) * stride) + column + relativeColumn; + if (applyCb) + { + weightedCb += ((int)cbCoefficients[coefficientIndex] - 128) * cbGrain[grainIndex]; + } + + if (applyCr) + { + weightedCr += ((int)crCoefficients[coefficientIndex] - 128) * crGrain[grainIndex]; + } + + coefficientIndex++; + } + } + + for (int relativeColumn = -lag; relativeColumn < 0; relativeColumn++) + { + int grainIndex = (row * stride) + column + relativeColumn; + if (applyCb) + { + weightedCb += ((int)cbCoefficients[coefficientIndex] - 128) * cbGrain[grainIndex]; + } + + if (applyCr) + { + weightedCr += ((int)crCoefficients[coefficientIndex] - 128) * crGrain[grainIndex]; + } + + coefficientIndex++; + } + + if (parameters.NumYPoints != 0) + { + // Chroma has one additional autoregressive predictor when luma grain exists. Average the luma + // template footprint represented by this chroma sample before applying that final coefficient. + int lumaRow = ((row - TemplatePadding) << subsamplingY) + TemplatePadding; + int lumaColumn = ((column - TemplatePadding) << subsamplingX) + TemplatePadding; + int averageLuma = 0; + for (int relativeRow = 0; relativeRow <= subsamplingY; relativeRow++) + { + for (int relativeColumn = 0; relativeColumn <= subsamplingX; relativeColumn++) + { + averageLuma += lumaGrain[((lumaRow + relativeRow) * lumaStride) + + lumaColumn + relativeColumn]; + } + } + + int averagingShift = subsamplingX + subsamplingY; + averageLuma = (averageLuma + ((1 << averagingShift) >> 1)) >> averagingShift; + if (applyCb) + { + weightedCb += ((int)cbCoefficients[coefficientIndex] - 128) * averageLuma; + } + + if (applyCr) + { + weightedCr += ((int)crCoefficients[coefficientIndex] - 128) * averageLuma; + } + } + + int currentIndex = (row * stride) + column; + if (applyCb) + { + cbGrain[currentIndex] = Av1Math.Clamp( + cbGrain[currentIndex] + ((weightedCb + roundingOffset) >> coefficientShift), + grainMinimum, + grainMaximum); + } + + if (applyCr) + { + crGrain[currentIndex] = Av1Math.Clamp( + crGrain[currentIndex] + ((weightedCr + roundingOffset) >> coefficientShift), + grainMinimum, + grainMaximum); + } + } + } + } + + /// + /// Fills one chroma template from the normative Gaussian sequence. + /// + /// The pseudo-random register advanced for each sample. + /// The destination grain template. + /// The template height. + /// The template width. + /// The template row stride. + /// The normative Gaussian sample sequence. + /// The bit-depth-dependent Gaussian scaling shift. + /// The Gaussian scaling rounding offset. + private static void FillGaussianGrain( + ref ushort randomRegister, + Span grain, + int height, + int width, + int stride, + ReadOnlySpan gaussian, + int gaussianShift, + int gaussianRounding) + { + // Eleven pseudo-random bits address all 2,048 Gaussian entries with no modulo operation or distribution skew. + for (int row = 0; row < height; row++) + { + for (int column = 0; column < width; column++) + { + int randomIndex = GetRandomNumber(ref randomRegister, GaussianIndexBits); + grain[(row * stride) + column] = (gaussian[randomIndex] + gaussianRounding) >> gaussianShift; + } + } + } + + /// + /// Expands signaled control points into the 256-entry film-grain scaling lookup table. + /// + /// The strictly increasing input coordinates. + /// The scaling value at each input coordinate. + /// The number of populated control points. + /// The destination scaling lookup table. + private static void InitializeScalingFunction( + ReadOnlySpan pointValues, + ReadOnlySpan pointScalings, + int pointCount, + Span lookup) + { + if (pointCount == 0) + { + return; + } + + // Values outside the first and last control points extend their nearest endpoint rather than extrapolating. + lookup[..(int)pointValues[0]].Fill((int)pointScalings[0]); + for (int point = 0; point < pointCount - 1; point++) + { + int deltaY = (int)pointScalings[point + 1] - (int)pointScalings[point]; + int deltaX = (int)pointValues[point + 1] - (int)pointValues[point]; + + // A rounded Q16 reciprocal performs the piecewise-linear interpolation using integer arithmetic. The + // 32768 bias below rounds each reconstructed scaling value when it returns to integer precision. + long delta = deltaY * ((65536 + (deltaX >> 1)) / deltaX); + for (int x = 0; x < deltaX; x++) + { + lookup[(int)pointValues[point] + x] = (int)pointScalings[point] + + (int)(((x * delta) + 32768) >> 16); + } + } + + lookup[(int)pointValues[pointCount - 1]..].Fill((int)pointScalings[pointCount - 1]); + } + + /// + /// Copies a rectangular grain region while preserving independent source and destination strides. + /// + /// The source region. + /// The source row stride. + /// The destination region. + /// The destination row stride. + /// The number of values copied per row. + /// The number of rows copied. + private static void CopyArea( + ReadOnlySpan source, + int sourceStride, + Span destination, + int destinationStride, + int width, + int height) + { + for (int row = 0; row < height; row++) + { + source.Slice(row * sourceStride, width) + .CopyTo(destination.Slice(row * destinationStride, width)); + } + } + + /// + /// Initializes the film-grain linear-feedback shift register for one luma block row. + /// + /// The register to initialize. + /// The zero-based luma row represented by the block row. + /// The frame grain seed. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static void InitializeRandomGenerator(ref ushort randomRegister, int lumaLine, ushort seed) + { + randomRegister = seed; + int lumaBlock = lumaLine >> 5; + + // The two affine mixes inject the block-row identity into both bytes of the 16-bit LFSR state. + randomRegister ^= (ushort)((((lumaBlock * 37) + 178) & 255) << 8); + randomRegister ^= (ushort)(((lumaBlock * 173) + 105) & 255); + } + + /// + /// Advances the film-grain linear-feedback shift register and returns its high-order bits. + /// + /// The register to advance. + /// The number of result bits. + /// A value in the range zero through 2^bitCount - 1. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static int GetRandomNumber(ref ushort randomRegister, int bitCount) + { + // AV1 uses taps 0, 1, 3, and 12 of the current register. Shifting right exposes the requested high bits + // after feedback has entered bit 15, matching both Gaussian indexing and block-offset selection. + int feedback = (randomRegister ^ (randomRegister >> 1) ^ (randomRegister >> 3) ^ + (randomRegister >> 12)) & 1; + + randomRegister = (ushort)((randomRegister >> 1) | (feedback << 15)); + return (randomRegister >> (16 - bitCount)) & ((1 << bitCount) - 1); + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/FilmGrain/Av1FilmGrainGaussianSequence.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/FilmGrain/Av1FilmGrainGaussianSequence.cs new file mode 100644 index 0000000000..509d0b78f4 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/FilmGrain/Av1FilmGrainGaussianSequence.cs @@ -0,0 +1,145 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.FilmGrain; + +/// +/// Provides the normative 2,048-sample Gaussian sequence used by AV1 film-grain synthesis. +/// +internal static class Av1FilmGrainGaussianSequence +{ + /// + /// Gets the signed 12-bit Gaussian samples in their normative index order. + /// + public static ReadOnlySpan Samples => + [ + 56, 568, -180, 172, 124, -84, 172, -64, -900, 24, 820, 224, 1248, 996, 272, -8, + -916, -388, -732, -104, -188, 800, 112, -652, -320, -376, 140, -252, 492, -168, 44, -788, + 588, -584, 500, -228, 12, 680, 272, -476, 972, -100, 652, 368, 432, -196, -720, -192, + 1000, -332, 652, -136, -552, -604, -4, 192, -220, -136, 1000, -52, 372, -96, -624, 124, + -24, 396, 540, -12, -104, 640, 464, 244, -208, -84, 368, -528, -740, 248, -968, -848, + 608, 376, -60, -292, -40, -156, 252, -292, 248, 224, -280, 400, -244, 244, -60, 76, + -80, 212, 532, 340, 128, -36, 824, -352, -60, -264, -96, -612, 416, -704, 220, -204, + 640, -160, 1220, -408, 900, 336, 20, -336, -96, -792, 304, 48, -28, -1232, -1172, -448, + 104, -292, -520, 244, 60, -948, 0, -708, 268, 108, 356, -548, 488, -344, -136, 488, + -196, -224, 656, -236, -1128, 60, 4, 140, 276, -676, -376, 168, -108, 464, 8, 564, + 64, 240, 308, -300, -400, -456, -136, 56, 120, -408, -116, 436, 504, -232, 328, 844, + -164, -84, 784, -168, 232, -224, 348, -376, 128, 568, 96, -1244, -288, 276, 848, 832, + -360, 656, 464, -384, -332, -356, 728, -388, 160, -192, 468, 296, 224, 140, -776, -100, + 280, 4, 196, 44, -36, -648, 932, 16, 1428, 28, 528, 808, 772, 20, 268, 88, + -332, -284, 124, -384, -448, 208, -228, -1044, -328, 660, 380, -148, -300, 588, 240, 540, + 28, 136, -88, -436, 256, 296, -1000, 1400, 0, -48, 1056, -136, 264, -528, -1108, 632, + -484, -592, -344, 796, 124, -668, -768, 388, 1296, -232, -188, -200, -288, -4, 308, 100, + -168, 256, -500, 204, -508, 648, -136, 372, -272, -120, -1004, -552, -548, -384, 548, -296, + 428, -108, -8, -912, -324, -224, -88, -112, -220, -100, 996, -796, 548, 360, -216, 180, + 428, -200, -212, 148, 96, 148, 284, 216, -412, -320, 120, -300, -384, -604, -572, -332, + -8, -180, -176, 696, 116, -88, 628, 76, 44, -516, 240, -208, -40, 100, -592, 344, + -308, -452, -228, 20, 916, -1752, -136, -340, -804, 140, 40, 512, 340, 248, 184, -492, + 896, -156, 932, -628, 328, -688, -448, -616, -752, -100, 560, -1020, 180, -800, -64, 76, + 576, 1068, 396, 660, 552, -108, -28, 320, -628, 312, -92, -92, -472, 268, 16, 560, + 516, -672, -52, 492, -100, 260, 384, 284, 292, 304, -148, 88, -152, 1012, 1064, -228, + 164, -376, -684, 592, -392, 156, 196, -524, -64, -884, 160, -176, 636, 648, 404, -396, + -436, 864, 424, -728, 988, -604, 904, -592, 296, -224, 536, -176, -920, 436, -48, 1176, + -884, 416, -776, -824, -884, 524, -548, -564, -68, -164, -96, 692, 364, -692, -1012, -68, + 260, -480, 876, -1116, 452, -332, -352, 892, -1088, 1220, -676, 12, -292, 244, 496, 372, + -32, 280, 200, 112, -440, -96, 24, -644, -184, 56, -432, 224, -980, 272, -260, 144, + -436, 420, 356, 364, -528, 76, 172, -744, -368, 404, -752, -416, 684, -688, 72, 540, + 416, 92, 444, 480, -72, -1416, 164, -1172, -68, 24, 424, 264, 1040, 128, -912, -524, + -356, 64, 876, -12, 4, -88, 532, 272, -524, 320, 276, -508, 940, 24, -400, -120, + 756, 60, 236, -412, 100, 376, -484, 400, -100, -740, -108, -260, 328, -268, 224, -200, + -416, 184, -604, -564, -20, 296, 60, 892, -888, 60, 164, 68, -760, 216, -296, 904, + -336, -28, 404, -356, -568, -208, -1480, -512, 296, 328, -360, -164, -1560, -776, 1156, -428, + 164, -504, -112, 120, -216, -148, -264, 308, 32, 64, -72, 72, 116, 176, -64, -272, + 460, -536, -784, -280, 348, 108, -752, -132, 524, -540, -776, 116, -296, -1196, -288, -560, + 1040, -472, 116, -848, -1116, 116, 636, 696, 284, -176, 1016, 204, -864, -648, -248, 356, + 972, -584, -204, 264, 880, 528, -24, -184, 116, 448, -144, 828, 524, 212, -212, 52, + 12, 200, 268, -488, -404, -880, 824, -672, -40, 908, -248, 500, 716, -576, 492, -576, + 16, 720, -108, 384, 124, 344, 280, 576, -500, 252, 104, -308, 196, -188, -8, 1268, + 296, 1032, -1196, 436, 316, 372, -432, -200, -660, 704, -224, 596, -132, 268, 32, -452, + 884, 104, -1008, 424, -1348, -280, 4, -1168, 368, 476, 696, 300, -8, 24, 180, -592, + -196, 388, 304, 500, 724, -160, 244, -84, 272, -256, -420, 320, 208, -144, -156, 156, + 364, 452, 28, 540, 316, 220, -644, -248, 464, 72, 360, 32, -388, 496, -680, -48, + 208, -116, -408, 60, -604, -392, 548, -840, 784, -460, 656, -544, -388, -264, 908, -800, + -628, -612, -568, 572, -220, 164, 288, -16, -308, 308, -112, -636, -760, 280, -668, 432, + 364, 240, -196, 604, 340, 384, 196, 592, -44, -500, 432, -580, -132, 636, -76, 392, + 4, -412, 540, 508, 328, -356, -36, 16, -220, -64, -248, -60, 24, -192, 368, 1040, + 92, -24, -1044, -32, 40, 104, 148, 192, -136, -520, 56, -816, -224, 732, 392, 356, + 212, -80, -424, -1008, -324, 588, -1496, 576, 460, -816, -848, 56, -580, -92, -1372, -112, + -496, 200, 364, 52, -140, 48, -48, -60, 84, 72, 40, 132, -356, -268, -104, -284, + -404, 732, -520, 164, -304, -540, 120, 328, -76, -460, 756, 388, 588, 236, -436, -72, + -176, -404, -316, -148, 716, -604, 404, -72, -88, -888, -68, 944, 88, -220, -344, 960, + 472, 460, -232, 704, 120, 832, -228, 692, -508, 132, -476, 844, -748, -364, -44, 1116, + -1104, -1056, 76, 428, 552, -692, 60, 356, 96, -384, -188, -612, -576, 736, 508, 892, + 352, -1132, 504, -24, -352, 324, 332, -600, -312, 292, 508, -144, -8, 484, 48, 284, + -260, -240, 256, -100, -292, -204, -44, 472, -204, 908, -188, -1000, -256, 92, 1164, -392, + 564, 356, 652, -28, -884, 256, 484, -192, 760, -176, 376, -524, -452, -436, 860, -736, + 212, 124, 504, -476, 468, 76, -472, 552, -692, -944, -620, 740, -240, 400, 132, 20, + 192, -196, 264, -668, -1012, -60, 296, -316, -828, 76, -156, 284, -768, -448, -832, 148, + 248, 652, 616, 1236, 288, -328, -400, -124, 588, 220, 520, -696, 1032, 768, -740, -92, + -272, 296, 448, -464, 412, -200, 392, 440, -200, 264, -152, -260, 320, 1032, 216, 320, + -8, -64, 156, -1016, 1084, 1172, 536, 484, -432, 132, 372, -52, -256, 84, 116, -352, + 48, 116, 304, -384, 412, 924, -300, 528, 628, 180, 648, 44, -980, -220, 1320, 48, + 332, 748, 524, -268, -720, 540, -276, 564, -344, -208, -196, 436, 896, 88, -392, 132, + 80, -964, -288, 568, 56, -48, -456, 888, 8, 552, -156, -292, 948, 288, 128, -716, + -292, 1192, -152, 876, 352, -600, -260, -812, -468, -28, -120, -32, -44, 1284, 496, 192, + 464, 312, -76, -516, -380, -456, -1012, -48, 308, -156, 36, 492, -156, -808, 188, 1652, + 68, -120, -116, 316, 160, -140, 352, 808, -416, 592, 316, -480, 56, 528, -204, -568, + 372, -232, 752, -344, 744, -4, 324, -416, -600, 768, 268, -248, -88, -132, -420, -432, + 80, -288, 404, -316, -1216, -588, 520, -108, 92, -320, 368, -480, -216, -92, 1688, -300, + 180, 1020, -176, 820, -68, -228, -260, 436, -904, 20, 40, -508, 440, -736, 312, 332, + 204, 760, -372, 728, 96, -20, -632, -520, -560, 336, 1076, -64, -532, 776, 584, 192, + 396, -728, -520, 276, -188, 80, -52, -612, -252, -48, 648, 212, -688, 228, -52, -260, + 428, -412, -272, -404, 180, 816, -796, 48, 152, 484, -88, -216, 988, 696, 188, -528, + 648, -116, -180, 316, 476, 12, -564, 96, 476, -252, -364, -376, -392, 556, -256, -576, + 260, -352, 120, -16, -136, -260, -492, 72, 556, 660, 580, 616, 772, 436, 424, -32, + -324, -1268, 416, -324, -80, 920, 160, 228, 724, 32, -516, 64, 384, 68, -128, 136, + 240, 248, -204, -68, 252, -932, -120, -480, -628, -84, 192, 852, -404, -288, -132, 204, + 100, 168, -68, -196, -868, 460, 1080, 380, -80, 244, 0, 484, -888, 64, 184, 352, + 600, 460, 164, 604, -196, 320, -64, 588, -184, 228, 12, 372, 48, -848, -344, 224, + 208, -200, 484, 128, -20, 272, -468, -840, 384, 256, -720, -520, -464, -580, 112, -120, + 644, -356, -208, -608, -528, 704, 560, -424, 392, 828, 40, 84, 200, -152, 0, -144, + 584, 280, -120, 80, -556, -972, -196, -472, 724, 80, 168, -32, 88, 160, -688, 0, + 160, 356, 372, -776, 740, -128, 676, -248, -480, 4, -364, 96, 544, 232, -1032, 956, + 236, 356, 20, -40, 300, 24, -676, -596, 132, 1120, -104, 532, -1096, 568, 648, 444, + 508, 380, 188, -376, -604, 1488, 424, 24, 756, -220, -192, 716, 120, 920, 688, 168, + 44, -460, 568, 284, 1144, 1160, 600, 424, 888, 656, -356, -320, 220, 316, -176, -724, + -188, -816, -628, -348, -228, -380, 1012, -452, -660, 736, 928, 404, -696, -72, -268, -892, + 128, 184, -344, -780, 360, 336, 400, 344, 428, 548, -112, 136, -228, -216, -820, -516, + 340, 92, -136, 116, -300, 376, -244, 100, -316, -520, -284, -12, 824, 164, -548, -180, + -128, 116, -924, -828, 268, -368, -580, 620, 192, 160, 0, -1676, 1068, 424, -56, -360, + 468, -156, 720, 288, -528, 556, -364, 548, -148, 504, 316, 152, -648, -620, -684, -24, + -376, -384, -108, -920, -1032, 768, 180, -264, -508, -1268, -260, -60, 300, -240, 988, 724, + -376, -576, -212, -736, 556, 192, 1092, -620, -880, 376, -56, -4, -216, -32, 836, 268, + 396, 1332, 864, -600, 100, 56, -412, -92, 356, 180, 884, -468, -436, 292, -388, -804, + -704, -840, 368, -348, 140, -724, 1536, 940, 372, 112, -372, 436, -480, 1136, 296, -32, + -228, 132, -48, -220, 868, -1016, -60, -1044, -464, 328, 916, 244, 12, -736, -296, 360, + 468, -376, -108, -92, 788, 368, -56, 544, 400, -672, -420, 728, 16, 320, 44, -284, + -380, -796, 488, 132, 204, -596, -372, 88, -152, -908, -636, -572, -624, -116, -692, -200, + -56, 276, -88, 484, -324, 948, 864, 1000, -456, -184, -276, 292, -296, 156, 676, 320, + 160, 908, -84, -1236, -288, -116, 260, -372, -644, 732, -756, -96, 84, 344, -520, 348, + -688, 240, -84, 216, -1044, -136, -676, -396, -1500, 960, -40, 176, 168, 1516, 420, -504, + -344, -364, -360, 1216, -940, -380, -212, 252, -660, -708, 484, -444, -152, 928, -120, 1112, + 476, -260, 560, -148, -344, 108, -196, 228, -288, 504, 560, -328, -88, 288, -1008, 460, + -228, 468, -836, -196, 76, 388, 232, 412, -1168, -716, -644, 756, -172, -356, -504, 116, + 432, 528, 48, 476, -168, -608, 448, 160, -532, -272, 28, -676, -12, 828, 980, 456, + 520, 104, -104, 256, -344, -4, -28, -368, -52, -524, -572, -556, -200, 768, 1124, -208, + -512, 176, 232, 248, -148, -888, 604, -600, -304, 804, -156, -212, 488, -192, -804, -256, + 368, -360, -916, -328, 228, -240, -448, -472, 856, -556, -364, 572, -12, -156, -368, -340, + 432, 252, -752, -152, 288, 268, -580, -848, -592, 108, -76, 244, 312, -716, 592, -80, + 436, 360, 4, -248, 160, 516, 584, 732, 44, -468, -280, -292, -156, -588, 28, 308, + 912, 24, 124, 156, 180, -252, 944, -924, -772, -520, -428, -624, 300, -212, -1144, 32, + -724, 800, -1128, -212, -1288, -848, 180, -416, 440, 192, -576, -792, -76, -1080, 80, -532, + -352, -132, 380, -820, 148, 1112, 128, 164, 456, 700, -924, 144, -668, -384, 648, -832, + 508, 552, -52, -100, -656, 208, -568, 748, -88, 680, 232, 300, 192, -408, -1012, -152, + -252, -268, 272, -876, -664, -648, -332, -136, 16, 12, 1152, -28, 332, -536, 320, -672, + -460, -316, 532, -260, 228, -40, 1052, -816, 180, 88, -496, -556, -672, -368, 428, 92, + 356, 404, -408, 252, 196, -176, -556, 792, 268, 32, 372, 40, 96, -332, 328, 120, + 372, -900, -40, 472, -264, -592, 952, 128, 656, 112, 664, -232, 420, 4, -344, -464, + 556, 244, -416, -32, 252, 0, -412, 188, -696, 508, -476, 324, -1096, 656, -312, 560, + 264, -136, 304, 160, -64, -580, 248, 336, -720, 560, -348, -288, -276, -196, -500, 852, + -544, -236, -1128, -992, -776, 116, 56, 52, 860, 884, 212, -12, 168, 1020, 512, -552, + 924, -148, 716, 188, 164, -340, -520, -184, 880, -152, -680, -208, -1156, -300, -528, -472, + 364, 100, -744, -1056, -32, 540, 280, 144, -676, -32, -232, -280, -224, 96, 568, -76, + 172, 148, 148, 104, 32, -296, -32, 788, -80, 32, -16, 280, 288, 944, 428, -484, + ]; +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/FilmGrain/Av1FilmGrainNoise.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/FilmGrain/Av1FilmGrainNoise.cs new file mode 100644 index 0000000000..6293db935e --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/FilmGrain/Av1FilmGrainNoise.cs @@ -0,0 +1,1007 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.InteropServices; +using System.Runtime.Intrinsics; +using System.Runtime.Intrinsics.X86; +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.FilmGrain; + +/// +/// Applies selected AV1 grain blocks to restored luma and chroma samples. +/// +/// +/// SIMD lanes follow consecutive samples within one plane row. Grain values and native samples are widened to signed +/// 32-bit lanes before the scaling-table lookup and fixed-point addition, then clipped and narrowed only once. AVX2 +/// uses indexed gathers for the 256-entry scaling table. The current portable 128-bit dispatch handles high-bit-depth +/// interpolation with scalar table reads; all remaining columns use the same scalar equation. +/// +internal static class Av1FilmGrainNoise +{ + /// + /// The lower restricted-range luma value at eight-bit precision. + /// + private const int RestrictedLumaMinimum = 16; + + /// + /// The upper restricted-range luma value at eight-bit precision. + /// + private const int RestrictedLumaMaximum = 235; + + /// + /// The lower restricted-range chroma value at eight-bit precision. + /// + private const int RestrictedChromaMinimum = 16; + + /// + /// The upper restricted-range chroma value at eight-bit precision. + /// + private const int RestrictedChromaMaximum = 240; + + /// + /// Adds a selected grain rectangle to its corresponding restored samples. + /// + /// The native sample type. + /// The complete frame grain parameters. + /// The luma scaling lookup table. + /// The first chroma scaling lookup table. + /// The second chroma scaling lookup table. + /// The restored luma rectangle. + /// The restored first chroma rectangle. + /// The restored second chroma rectangle. + /// The luma row stride in samples. + /// The chroma row stride in samples. + /// The selected luma grain rectangle. + /// The selected first chroma grain rectangle. + /// The selected second chroma grain rectangle. + /// The luma grain row stride. + /// The chroma grain row stride. + /// Half the luma rectangle height. + /// Half the luma rectangle width. + /// The decoded sample bit depth. + /// The horizontal chroma subsampling shift. + /// The vertical chroma subsampling shift. + /// Whether the frame has no chroma planes. + /// Whether every plane uses the luma restricted range. + public static void Apply( + ObuFilmGrainParameters parameters, + ReadOnlySpan scalingY, + ReadOnlySpan scalingCb, + ReadOnlySpan scalingCr, + Span luma, + Span cb, + Span cr, + int lumaStride, + int chromaStride, + ReadOnlySpan lumaGrain, + ReadOnlySpan cbGrain, + ReadOnlySpan crGrain, + int lumaGrainStride, + int chromaGrainStride, + int halfLumaHeight, + int halfLumaWidth, + int bitDepth, + int subsamplingX, + int subsamplingY, + bool isMonochrome, + bool isIdentityMatrix) + where TSample : unmanaged + { + int scalingShift = (int)parameters.GrainScalingMinus8 + 8; + int roundingOffset = 1 << (scalingShift - 1); + int depthScale = 1 << (bitDepth - 8); + int sampleMaximum = (256 * depthScale) - 1; + int lumaMinimum = 0; + int lumaMaximum = sampleMaximum; + int chromaMinimum = 0; + int chromaMaximum = sampleMaximum; + if (parameters.ClipToRestrictedRange) + { + // Restricted-range endpoints are signaled at eight-bit precision and scale exactly at higher depths. + lumaMinimum = RestrictedLumaMinimum * depthScale; + lumaMaximum = RestrictedLumaMaximum * depthScale; + chromaMinimum = (isIdentityMatrix ? RestrictedLumaMinimum : RestrictedChromaMinimum) * depthScale; + chromaMaximum = (isIdentityMatrix ? RestrictedLumaMaximum : RestrictedChromaMaximum) * depthScale; + } + + if (!isMonochrome) + { + int cbMultiplier = (int)parameters.CbMult - 128; + int cbLumaMultiplier = (int)parameters.CbLumaMult - 128; + int cbOffset = ((int)parameters.CbOffset * depthScale) - (256 * depthScale); + int crMultiplier = (int)parameters.CrMult - 128; + int crLumaMultiplier = (int)parameters.CrLumaMult - 128; + int crOffset = ((int)parameters.CrOffset * depthScale) - (256 * depthScale); + if (parameters.ChromaScalingFromLuma) + { + // Unity in the Q6 luma-multiplier domain selects luma directly and removes the chroma contribution. + cbMultiplier = 0; + cbLumaMultiplier = 64; + cbOffset = 0; + crMultiplier = 0; + crLumaMultiplier = 64; + crOffset = 0; + } + + ApplyChroma( + scalingCb, + scalingCr, + luma, + cb, + cr, + lumaStride, + chromaStride, + cbGrain, + crGrain, + chromaGrainStride, + halfLumaHeight << (1 - subsamplingY), + halfLumaWidth << (1 - subsamplingX), + bitDepth, + subsamplingX, + subsamplingY, + parameters.NumCbPoints != 0 || parameters.ChromaScalingFromLuma, + parameters.NumCrPoints != 0 || parameters.ChromaScalingFromLuma, + cbMultiplier, + cbLumaMultiplier, + cbOffset, + crMultiplier, + crLumaMultiplier, + crOffset, + roundingOffset, + scalingShift, + sampleMaximum, + chromaMinimum, + chromaMaximum); + } + + if (parameters.NumYPoints != 0) + { + ApplyLuma( + scalingY, + luma, + lumaStride, + lumaGrain, + lumaGrainStride, + halfLumaHeight << 1, + halfLumaWidth << 1, + bitDepth, + roundingOffset, + scalingShift, + lumaMinimum, + lumaMaximum); + } + } + + /// + /// Selects the widest available luma traversal. + /// + private static void ApplyLuma( + ReadOnlySpan scaling, + Span samples, + int sampleStride, + ReadOnlySpan grain, + int grainStride, + int height, + int width, + int bitDepth, + int roundingOffset, + int scalingShift, + int minimum, + int maximum) + where TSample : unmanaged + { + // Scaling is an indexed lookup, so AVX2 is selected by gather support rather than generic preferred vector + // width. The portable path is deliberately limited by CanVectorizeWithoutGather for the same reason. + if (Avx2.IsSupported) + { + ApplyLuma( + scaling, + samples, + sampleStride, + grain, + grainStride, + height, + width, + bitDepth, + roundingOffset, + scalingShift, + minimum, + maximum, + Vector256.Zero); + + return; + } + + if (CanVectorizeWithoutGather(bitDepth)) + { + ApplyLuma( + scaling, + samples, + sampleStride, + grain, + grainStride, + height, + width, + bitDepth, + roundingOffset, + scalingShift, + minimum, + maximum, + Vector128.Zero); + + return; + } + + ApplyLumaScalar(scaling, samples, sampleStride, grain, grainStride, height, width, bitDepth, roundingOffset, scalingShift, minimum, maximum); + } + + /// + /// Applies luma grain eight samples at a time. + /// + private static void ApplyLuma( + ReadOnlySpan scaling, + Span samples, + int sampleStride, + ReadOnlySpan grain, + int grainStride, + int height, + int width, + int bitDepth, + int roundingOffset, + int scalingShift, + int minimum, + int maximum, + Vector256 vector) + where TSample : unmanaged + { + ref TSample sampleBase = ref MemoryMarshal.GetReference(samples); + ref int grainBase = ref MemoryMarshal.GetReference(grain); + for (int row = 0; row < height; row++) + { + int sampleRowOffset = row * sampleStride; + int grainRowOffset = row * grainStride; + int column = 0; + int vectorEnd = (int)(Numerics.Vector256Count(width) * (nuint)Vector256.Count); + + // Samples, grain, and scale indices share the same lane coordinate. No permutation is required between + // the lookup, fixed-point multiply, clipping, and native-sample store. + for (; column < vectorEnd; column += Vector256.Count) + { + ref TSample destination = ref Unsafe.Add(ref sampleBase, sampleRowOffset + column); + Vector256 source = Av1FilmGrainSampleOperations.Load8(ref destination); + Vector256 grainValues = Vector256.LoadUnsafe(ref grainBase, (nuint)(grainRowOffset + column)); + Vector256 result = AddNoise(source, grainValues, scaling, bitDepth, roundingOffset, scalingShift, minimum, maximum); + Av1FilmGrainSampleOperations.Store8(ref destination, result); + } + + ApplyLumaScalar( + scaling, + samples.Slice(sampleRowOffset + column), + sampleStride, + grain.Slice(grainRowOffset + column), + grainStride, + 1, + width - column, + bitDepth, + roundingOffset, + scalingShift, + minimum, + maximum); + } + } + + /// + /// Applies luma grain four samples at a time. + /// + private static void ApplyLuma( + ReadOnlySpan scaling, + Span samples, + int sampleStride, + ReadOnlySpan grain, + int grainStride, + int height, + int width, + int bitDepth, + int roundingOffset, + int scalingShift, + int minimum, + int maximum, + Vector128 vector) + where TSample : unmanaged + { + ref TSample sampleBase = ref MemoryMarshal.GetReference(samples); + ref int grainBase = ref MemoryMarshal.GetReference(grain); + for (int row = 0; row < height; row++) + { + int sampleRowOffset = row * sampleStride; + int grainRowOffset = row * grainStride; + int column = 0; + int vectorEnd = (int)(Numerics.Vector128Count(width) * (nuint)Vector128.Count); + + // Four scalar table reads assemble the scale vector; the rest of the normative grain equation remains + // lane-wise, including interpolation for 10- and 12-bit coordinates. + for (; column < vectorEnd; column += Vector128.Count) + { + ref TSample destination = ref Unsafe.Add(ref sampleBase, sampleRowOffset + column); + Vector128 source = Av1FilmGrainSampleOperations.Load4(ref destination); + Vector128 grainValues = Vector128.LoadUnsafe(ref grainBase, (nuint)(grainRowOffset + column)); + Vector128 result = AddNoise(source, grainValues, scaling, bitDepth, roundingOffset, scalingShift, minimum, maximum); + Av1FilmGrainSampleOperations.Store4(ref destination, result); + } + + ApplyLumaScalar( + scaling, + samples.Slice(sampleRowOffset + column), + sampleStride, + grain.Slice(grainRowOffset + column), + grainStride, + 1, + width - column, + bitDepth, + roundingOffset, + scalingShift, + minimum, + maximum); + } + } + + /// + /// Applies the luma scalar remainder or complete scalar fallback. + /// + private static void ApplyLumaScalar( + ReadOnlySpan scaling, + Span samples, + int sampleStride, + ReadOnlySpan grain, + int grainStride, + int height, + int width, + int bitDepth, + int roundingOffset, + int scalingShift, + int minimum, + int maximum) + where TSample : unmanaged + { + ref TSample sampleBase = ref MemoryMarshal.GetReference(samples); + ref int grainBase = ref MemoryMarshal.GetReference(grain); + for (int row = 0; row < height; row++) + { + int sampleRowOffset = row * sampleStride; + int grainRowOffset = row * grainStride; + for (int column = 0; column < width; column++) + { + ref TSample destination = ref Unsafe.Add(ref sampleBase, sampleRowOffset + column); + int source = Av1FilmGrainSampleOperations.Load(ref destination); + int scale = ScaleLookup(scaling, source, bitDepth); + int value = source + (((scale * Unsafe.Add(ref grainBase, grainRowOffset + column)) + roundingOffset) >> scalingShift); + Av1FilmGrainSampleOperations.Store(ref destination, Av1Math.Clamp(value, minimum, maximum)); + } + } + } + + /// + /// Selects the widest available chroma traversal. + /// + private static void ApplyChroma( + ReadOnlySpan scalingCb, + ReadOnlySpan scalingCr, + Span luma, + Span cb, + Span cr, + int lumaStride, + int chromaStride, + ReadOnlySpan cbGrain, + ReadOnlySpan crGrain, + int grainStride, + int height, + int width, + int bitDepth, + int subsamplingX, + int subsamplingY, + bool applyCb, + bool applyCr, + int cbMultiplier, + int cbLumaMultiplier, + int cbOffset, + int crMultiplier, + int crLumaMultiplier, + int crOffset, + int roundingOffset, + int scalingShift, + int sampleMaximum, + int minimum, + int maximum) + where TSample : unmanaged + { + // Chroma shares luma's dispatch: AVX2 gathers scaling values, while the portable vector path is currently + // enabled only for high-bit-depth interpolation. This policy does not establish which path is faster. + if (Avx2.IsSupported) + { + ApplyChroma( + scalingCb, + scalingCr, + luma, + cb, + cr, + lumaStride, + chromaStride, + cbGrain, + crGrain, + grainStride, + height, + width, + bitDepth, + subsamplingX, + subsamplingY, + applyCb, + applyCr, + cbMultiplier, + cbLumaMultiplier, + cbOffset, + crMultiplier, + crLumaMultiplier, + crOffset, + roundingOffset, + scalingShift, + sampleMaximum, + minimum, + maximum, + Vector256.Zero); + + return; + } + + if (CanVectorizeWithoutGather(bitDepth)) + { + ApplyChroma( + scalingCb, + scalingCr, + luma, + cb, + cr, + lumaStride, + chromaStride, + cbGrain, + crGrain, + grainStride, + height, + width, + bitDepth, + subsamplingX, + subsamplingY, + applyCb, + applyCr, + cbMultiplier, + cbLumaMultiplier, + cbOffset, + crMultiplier, + crLumaMultiplier, + crOffset, + roundingOffset, + scalingShift, + sampleMaximum, + minimum, + maximum, + Vector128.Zero); + + return; + } + + ApplyChromaScalar( + scalingCb, + scalingCr, + luma, + cb, + cr, + lumaStride, + chromaStride, + cbGrain, + crGrain, + grainStride, + height, + width, + bitDepth, + subsamplingX, + subsamplingY, + applyCb, + applyCr, + cbMultiplier, + cbLumaMultiplier, + cbOffset, + crMultiplier, + crLumaMultiplier, + crOffset, + roundingOffset, + scalingShift, + sampleMaximum, + minimum, + maximum); + } + + /// + /// Applies chroma grain eight samples at a time. + /// + private static void ApplyChroma( + ReadOnlySpan scalingCb, + ReadOnlySpan scalingCr, + Span luma, + Span cb, + Span cr, + int lumaStride, + int chromaStride, + ReadOnlySpan cbGrain, + ReadOnlySpan crGrain, + int grainStride, + int height, + int width, + int bitDepth, + int subsamplingX, + int subsamplingY, + bool applyCb, + bool applyCr, + int cbMultiplier, + int cbLumaMultiplier, + int cbOffset, + int crMultiplier, + int crLumaMultiplier, + int crOffset, + int roundingOffset, + int scalingShift, + int sampleMaximum, + int minimum, + int maximum, + Vector256 vector) + where TSample : unmanaged + { + ref TSample lumaBase = ref MemoryMarshal.GetReference(luma); + ref TSample cbBase = ref MemoryMarshal.GetReference(cb); + ref TSample crBase = ref MemoryMarshal.GetReference(cr); + ref int cbGrainBase = ref MemoryMarshal.GetReference(cbGrain); + ref int crGrainBase = ref MemoryMarshal.GetReference(crGrain); + Vector256 zero = Vector256.Zero; + Vector256 maximumIndex = Vector256.Create(sampleMaximum); + for (int row = 0; row < height; row++) + { + ref TSample lumaRow = ref Unsafe.Add(ref lumaBase, (row << subsamplingY) * lumaStride); + int chromaRowOffset = row * chromaStride; + int grainRowOffset = row * grainStride; + int column = 0; + int vectorEnd = (int)(Numerics.Vector256Count(width) * (nuint)Vector256.Count); + + // Each lane represents one chroma coordinate and its corresponding reconstructed-luma coordinate. The Q6 + // luma/chroma blend is clamped to a legal sample code before it becomes a scaling-table index. + for (; column < vectorEnd; column += Vector256.Count) + { + ref TSample lumaSource = ref Unsafe.Add(ref lumaRow, column << subsamplingX); + Vector256 averageLuma = Av1FilmGrainSampleOperations.LoadChromaLuma8(ref lumaSource, subsamplingX); + if (applyCb) + { + ref TSample destination = ref Unsafe.Add(ref cbBase, chromaRowOffset + column); + Vector256 source = Av1FilmGrainSampleOperations.Load8(ref destination); + Vector256 scalingIndex = ((averageLuma * cbLumaMultiplier) + (source * cbMultiplier)) >> 6; + scalingIndex = Vector256.Min(Vector256.Max(scalingIndex + Vector256.Create(cbOffset), zero), maximumIndex); + Vector256 grainValues = Vector256.LoadUnsafe(ref cbGrainBase, (nuint)(grainRowOffset + column)); + Vector256 result = AddNoise( + source, + grainValues, + scalingCb, + scalingIndex, + bitDepth, + roundingOffset, + scalingShift, + minimum, + maximum); + + Av1FilmGrainSampleOperations.Store8(ref destination, result); + } + + if (applyCr) + { + ref TSample destination = ref Unsafe.Add(ref crBase, chromaRowOffset + column); + Vector256 source = Av1FilmGrainSampleOperations.Load8(ref destination); + Vector256 scalingIndex = ((averageLuma * crLumaMultiplier) + (source * crMultiplier)) >> 6; + scalingIndex = Vector256.Min(Vector256.Max(scalingIndex + Vector256.Create(crOffset), zero), maximumIndex); + Vector256 grainValues = Vector256.LoadUnsafe(ref crGrainBase, (nuint)(grainRowOffset + column)); + Vector256 result = AddNoise( + source, + grainValues, + scalingCr, + scalingIndex, + bitDepth, + roundingOffset, + scalingShift, + minimum, + maximum); + + Av1FilmGrainSampleOperations.Store8(ref destination, result); + } + } + + ApplyChromaScalar( + scalingCb, + scalingCr, + luma.Slice(((row << subsamplingY) * lumaStride) + (column << subsamplingX)), + cb.Slice(chromaRowOffset + column), + cr.Slice(chromaRowOffset + column), + lumaStride, + chromaStride, + cbGrain.Slice(grainRowOffset + column), + crGrain.Slice(grainRowOffset + column), + grainStride, + 1, + width - column, + bitDepth, + subsamplingX, + subsamplingY, + applyCb, + applyCr, + cbMultiplier, + cbLumaMultiplier, + cbOffset, + crMultiplier, + crLumaMultiplier, + crOffset, + roundingOffset, + scalingShift, + sampleMaximum, + minimum, + maximum); + } + } + + /// + /// Applies chroma grain four samples at a time. + /// + private static void ApplyChroma( + ReadOnlySpan scalingCb, + ReadOnlySpan scalingCr, + Span luma, + Span cb, + Span cr, + int lumaStride, + int chromaStride, + ReadOnlySpan cbGrain, + ReadOnlySpan crGrain, + int grainStride, + int height, + int width, + int bitDepth, + int subsamplingX, + int subsamplingY, + bool applyCb, + bool applyCr, + int cbMultiplier, + int cbLumaMultiplier, + int cbOffset, + int crMultiplier, + int crLumaMultiplier, + int crOffset, + int roundingOffset, + int scalingShift, + int sampleMaximum, + int minimum, + int maximum, + Vector128 vector) + where TSample : unmanaged + { + ref TSample lumaBase = ref MemoryMarshal.GetReference(luma); + ref TSample cbBase = ref MemoryMarshal.GetReference(cb); + ref TSample crBase = ref MemoryMarshal.GetReference(cr); + ref int cbGrainBase = ref MemoryMarshal.GetReference(cbGrain); + ref int crGrainBase = ref MemoryMarshal.GetReference(crGrain); + Vector128 zero = Vector128.Zero; + Vector128 maximumIndex = Vector128.Create(sampleMaximum); + for (int row = 0; row < height; row++) + { + ref TSample lumaRow = ref Unsafe.Add(ref lumaBase, (row << subsamplingY) * lumaStride); + int chromaRowOffset = row * chromaStride; + int grainRowOffset = row * grainStride; + int column = 0; + int vectorEnd = (int)(Numerics.Vector128Count(width) * (nuint)Vector128.Count); + + // The four-lane path preserves the same coordinate alignment and Q6 scaling-index arithmetic. Only the + // table read changes from a hardware gather to four scalar reads assembled into a vector. + for (; column < vectorEnd; column += Vector128.Count) + { + ref TSample lumaSource = ref Unsafe.Add(ref lumaRow, column << subsamplingX); + Vector128 averageLuma = Av1FilmGrainSampleOperations.LoadChromaLuma4(ref lumaSource, subsamplingX); + if (applyCb) + { + ref TSample destination = ref Unsafe.Add(ref cbBase, chromaRowOffset + column); + Vector128 source = Av1FilmGrainSampleOperations.Load4(ref destination); + Vector128 scalingIndex = ((averageLuma * cbLumaMultiplier) + (source * cbMultiplier)) >> 6; + scalingIndex = Vector128.Min(Vector128.Max(scalingIndex + Vector128.Create(cbOffset), zero), maximumIndex); + Vector128 grainValues = Vector128.LoadUnsafe(ref cbGrainBase, (nuint)(grainRowOffset + column)); + Vector128 result = AddNoise( + source, + grainValues, + scalingCb, + scalingIndex, + bitDepth, + roundingOffset, + scalingShift, + minimum, + maximum); + + Av1FilmGrainSampleOperations.Store4(ref destination, result); + } + + if (applyCr) + { + ref TSample destination = ref Unsafe.Add(ref crBase, chromaRowOffset + column); + Vector128 source = Av1FilmGrainSampleOperations.Load4(ref destination); + Vector128 scalingIndex = ((averageLuma * crLumaMultiplier) + (source * crMultiplier)) >> 6; + scalingIndex = Vector128.Min(Vector128.Max(scalingIndex + Vector128.Create(crOffset), zero), maximumIndex); + Vector128 grainValues = Vector128.LoadUnsafe(ref crGrainBase, (nuint)(grainRowOffset + column)); + Vector128 result = AddNoise( + source, + grainValues, + scalingCr, + scalingIndex, + bitDepth, + roundingOffset, + scalingShift, + minimum, + maximum); + + Av1FilmGrainSampleOperations.Store4(ref destination, result); + } + } + + ApplyChromaScalar( + scalingCb, + scalingCr, + luma.Slice(((row << subsamplingY) * lumaStride) + (column << subsamplingX)), + cb.Slice(chromaRowOffset + column), + cr.Slice(chromaRowOffset + column), + lumaStride, + chromaStride, + cbGrain.Slice(grainRowOffset + column), + crGrain.Slice(grainRowOffset + column), + grainStride, + 1, + width - column, + bitDepth, + subsamplingX, + subsamplingY, + applyCb, + applyCr, + cbMultiplier, + cbLumaMultiplier, + cbOffset, + crMultiplier, + crLumaMultiplier, + crOffset, + roundingOffset, + scalingShift, + sampleMaximum, + minimum, + maximum); + } + } + + /// + /// Applies the chroma scalar remainder or complete scalar fallback. + /// + private static void ApplyChromaScalar( + ReadOnlySpan scalingCb, + ReadOnlySpan scalingCr, + Span luma, + Span cb, + Span cr, + int lumaStride, + int chromaStride, + ReadOnlySpan cbGrain, + ReadOnlySpan crGrain, + int grainStride, + int height, + int width, + int bitDepth, + int subsamplingX, + int subsamplingY, + bool applyCb, + bool applyCr, + int cbMultiplier, + int cbLumaMultiplier, + int cbOffset, + int crMultiplier, + int crLumaMultiplier, + int crOffset, + int roundingOffset, + int scalingShift, + int sampleMaximum, + int minimum, + int maximum) + where TSample : unmanaged + { + ref TSample lumaBase = ref MemoryMarshal.GetReference(luma); + ref TSample cbBase = ref MemoryMarshal.GetReference(cb); + ref TSample crBase = ref MemoryMarshal.GetReference(cr); + ref int cbGrainBase = ref MemoryMarshal.GetReference(cbGrain); + ref int crGrainBase = ref MemoryMarshal.GetReference(crGrain); + for (int row = 0; row < height; row++) + { + int lumaRowOffset = (row << subsamplingY) * lumaStride; + int chromaRowOffset = row * chromaStride; + int grainRowOffset = row * grainStride; + for (int column = 0; column < width; column++) + { + int lumaOffset = lumaRowOffset + (column << subsamplingX); + int averageLuma = Av1FilmGrainSampleOperations.Load(ref Unsafe.Add(ref lumaBase, lumaOffset)); + if (subsamplingX != 0) + { + averageLuma = (averageLuma + Av1FilmGrainSampleOperations.Load(ref Unsafe.Add(ref lumaBase, lumaOffset + 1)) + 1) >> 1; + } + + int chromaOffset = chromaRowOffset + column; + int grainOffset = grainRowOffset + column; + if (applyCb) + { + ref TSample destination = ref Unsafe.Add(ref cbBase, chromaOffset); + int source = Av1FilmGrainSampleOperations.Load(ref destination); + int scalingIndex = Av1Math.Clamp( + (((averageLuma * cbLumaMultiplier) + (source * cbMultiplier)) >> 6) + cbOffset, + 0, + sampleMaximum); + + int scale = ScaleLookup(scalingCb, scalingIndex, bitDepth); + int value = source + (((scale * Unsafe.Add(ref cbGrainBase, grainOffset)) + roundingOffset) >> scalingShift); + Av1FilmGrainSampleOperations.Store(ref destination, Av1Math.Clamp(value, minimum, maximum)); + } + + if (applyCr) + { + ref TSample destination = ref Unsafe.Add(ref crBase, chromaOffset); + int source = Av1FilmGrainSampleOperations.Load(ref destination); + int scalingIndex = Av1Math.Clamp( + (((averageLuma * crLumaMultiplier) + (source * crMultiplier)) >> 6) + crOffset, + 0, + sampleMaximum); + + int scale = ScaleLookup(scalingCr, scalingIndex, bitDepth); + int value = source + (((scale * Unsafe.Add(ref crGrainBase, grainOffset)) + roundingOffset) >> scalingShift); + Av1FilmGrainSampleOperations.Store(ref destination, Av1Math.Clamp(value, minimum, maximum)); + } + } + } + } + + /// + /// Adds scaled grain to eight source samples and clips the result. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector256 AddNoise( + Vector256 source, + Vector256 grain, + ReadOnlySpan scaling, + int bitDepth, + int roundingOffset, + int scalingShift, + int minimum, + int maximum) + => AddNoise(source, grain, scaling, source, bitDepth, roundingOffset, scalingShift, minimum, maximum); + + /// + /// Adds scaled grain to eight source samples using independent scaling coordinates. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector256 AddNoise( + Vector256 source, + Vector256 grain, + ReadOnlySpan scaling, + Vector256 scalingIndex, + int bitDepth, + int roundingOffset, + int scalingShift, + int minimum, + int maximum) + { + Vector256 scale = ScaleLookup(scaling, scalingIndex, bitDepth); + Vector256 result = source + (((scale * grain) + Vector256.Create(roundingOffset)) >> scalingShift); + return Vector256.Min(Vector256.Max(result, Vector256.Create(minimum)), Vector256.Create(maximum)); + } + + /// + /// Adds scaled grain to four source samples and clips the result. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector128 AddNoise( + Vector128 source, + Vector128 grain, + ReadOnlySpan scaling, + int bitDepth, + int roundingOffset, + int scalingShift, + int minimum, + int maximum) + => AddNoise(source, grain, scaling, source, bitDepth, roundingOffset, scalingShift, minimum, maximum); + + /// + /// Adds scaled grain to four source samples using independent scaling coordinates. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector128 AddNoise( + Vector128 source, + Vector128 grain, + ReadOnlySpan scaling, + Vector128 scalingIndex, + int bitDepth, + int roundingOffset, + int scalingShift, + int minimum, + int maximum) + { + Vector128 scale = ScaleLookup(scaling, scalingIndex, bitDepth); + Vector128 result = source + (((scale * grain) + Vector128.Create(roundingOffset)) >> scalingShift); + return Vector128.Min(Vector128.Max(result, Vector128.Create(minimum)), Vector128.Create(maximum)); + } + + /// + /// Determines whether the current dispatch enables the portable vector traversal. + /// + /// The decoded sample bit depth. + /// Whether to use the portable vector traversal. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static bool CanVectorizeWithoutGather(int bitDepth) + { + // This path assembles four scale values from scalar lookups. High-depth samples additionally interpolate + // between table entries. The current depth gate requires end-to-end evidence before it can be justified + // as a performance policy; it does not express a numerical requirement of film-grain synthesis. + return bitDepth > 8 && Vector128.IsHardwareAccelerated; + } + + /// + /// Gathers eight scaling values and interpolates high-bit-depth coordinates. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static unsafe Vector256 ScaleLookup(ReadOnlySpan scaling, Vector256 index, int bitDepth) + { + int depthShift = bitDepth - 8; + Vector256 tableIndex = index >> depthShift; + fixed (int* table = scaling) + { + Vector256 current = Avx2.GatherVector256(table, tableIndex, sizeof(int)); + if (depthShift == 0) + { + return current; + } + + // Clamping the following index extends entry 255 across the final interpolation interval. + Vector256 nextIndex = Vector256.Min(tableIndex + Vector256.One, Vector256.Create(255)); + Vector256 next = Avx2.GatherVector256(table, nextIndex, sizeof(int)); + Vector256 fraction = index & Vector256.Create((1 << depthShift) - 1); + return current + ((((next - current) * fraction) + Vector256.Create(1 << (depthShift - 1))) >> depthShift); + } + } + + /// + /// Reads four scaling values and interpolates high-bit-depth coordinates. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector128 ScaleLookup(ReadOnlySpan scaling, Vector128 index, int bitDepth) + => Vector128.Create( + ScaleLookup(scaling, index.GetElement(0), bitDepth), + ScaleLookup(scaling, index.GetElement(1), bitDepth), + ScaleLookup(scaling, index.GetElement(2), bitDepth), + ScaleLookup(scaling, index.GetElement(3), bitDepth)); + + /// + /// Reads one scaling value, interpolating between eight-bit entries when required. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static int ScaleLookup(ReadOnlySpan scaling, int index, int bitDepth) + { + int depthShift = bitDepth - 8; + int tableIndex = index >> depthShift; + if (depthShift == 0 || tableIndex == 255) + { + return scaling[tableIndex]; + } + + int fraction = index & ((1 << depthShift) - 1); + return scaling[tableIndex] + ((((scaling[tableIndex + 1] - scaling[tableIndex]) * fraction) + (1 << (depthShift - 1))) >> depthShift); + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/FilmGrain/Av1FilmGrainOverlap.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/FilmGrain/Av1FilmGrainOverlap.cs new file mode 100644 index 0000000000..a23bc17227 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/FilmGrain/Av1FilmGrainOverlap.cs @@ -0,0 +1,318 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Numerics; +using System.Runtime.CompilerServices; +using System.Runtime.InteropServices; +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.FilmGrain; + +/// +/// Blends grain samples across adjacent synthesis blocks. +/// +/// +/// Vertical boundaries contain only one or two strided columns and use the fixed scalar kernels. Horizontal boundaries +/// are contiguous and progress through the enabled vector widths before the +/// scalar tail. Every lane applies the same Q5 overlap weights, rounding offset, and signed grain clamp. +/// +internal static class Av1FilmGrainOverlap +{ + /// + /// Blends the two grain columns on a vertical block boundary. + /// + /// The saved grain columns from the block on the left. + /// The saved-column row stride. + /// The grain columns selected for the block on the right. + /// The right-block row stride. + /// The overlap destination. + /// The destination row stride. + /// The one- or two-sample overlap width. + /// The overlap height. + /// The minimum grain value. + /// The maximum grain value. + public static void Vertical( + ReadOnlySpan left, + int leftStride, + ReadOnlySpan right, + int rightStride, + Span destination, + int destinationStride, + int width, + int height, + int minimum, + int maximum) + { + // Each row contributes one or two strided samples. This traversal handles those columns directly; + // the horizontal traversal below groups contiguous samples into vector lanes. + if (width == 1) + { + for (int row = 0; row < height; row++) + { + int leftOffset = row * leftStride; + int rightOffset = row * rightStride; + int destinationOffset = row * destinationStride; + + // A subsampled one-column boundary uses the dedicated 23:22 overlap weights. + destination[destinationOffset] = Av1Math.Clamp( + ((left[leftOffset] * 23) + (right[rightOffset] * 22) + 16) >> 5, + minimum, + maximum); + } + + return; + } + + for (int row = 0; row < height; row++) + { + int leftOffset = row * leftStride; + int rightOffset = row * rightStride; + int destinationOffset = row * destinationStride; + + // The two-column kernel biases the outer samples toward their originating block and crosses the 27:17 + // weights for the inner samples. These fixed weights are part of AV1 grain synthesis. + destination[destinationOffset] = Av1Math.Clamp( + ((left[leftOffset] * 27) + (right[rightOffset] * 17) + 16) >> 5, + minimum, + maximum); + + destination[destinationOffset + 1] = Av1Math.Clamp( + ((left[leftOffset + 1] * 17) + (right[rightOffset + 1] * 27) + 16) >> 5, + minimum, + maximum); + } + } + + /// + /// Blends the one or two grain rows on a horizontal block boundary. + /// + /// The saved grain rows from the block above. + /// The saved-row stride. + /// The grain rows selected for the block below. + /// The lower-block row stride. + /// The overlap destination. + /// The destination row stride. + /// The overlap width. + /// The one- or two-sample overlap height. + /// The minimum grain value. + /// The maximum grain value. + public static void Horizontal( + ReadOnlySpan top, + int topStride, + ReadOnlySpan bottom, + int bottomStride, + Span destination, + int destinationStride, + int width, + int height, + int minimum, + int maximum) + { + if (height == 1) + { + // Vertically subsampled chroma collapses the overlap to the single-row 23:22 kernel. + BlendRow(top, bottom, destination, width, 23, 22, minimum, maximum); + return; + } + + // Luma and full-height chroma use the crossed two-row 27:17 overlap kernel. + BlendRow(top, bottom, destination, width, 27, 17, minimum, maximum); + BlendRow( + top[topStride..], + bottom[bottomStride..], + destination[destinationStride..], + width, + 17, + 27, + minimum, + maximum); + } + + /// + /// Blends one contiguous overlap row using the widest useful vector width and a scalar remainder. + /// + /// The samples from the preceding block. + /// The samples from the following block. + /// The blended samples. + /// The number of samples to blend. + /// The preceding-block weight. + /// The following-block weight. + /// The minimum grain value. + /// The maximum grain value. + private static void BlendRow( + ReadOnlySpan left, + ReadOnlySpan right, + Span destination, + int width, + int leftWeight, + int rightWeight, + int minimum, + int maximum) + { + int column = 0; + + // The current 512-bit gate also requires Vector to expose sixteen int lanes. That is a dispatch choice, + // not evidence of the processor's execution width or of faster overlap processing. + if (Vector512.IsHardwareAccelerated && Vector.Count == Vector512.Count) + { + column = Blend(left, right, destination, width, column, leftWeight, rightWeight, minimum, maximum, Vector512.Zero); + } + + if (Vector256.IsHardwareAccelerated) + { + column = Blend(left, right, destination, width, column, leftWeight, rightWeight, minimum, maximum, Vector256.Zero); + } + + if (Vector128.IsHardwareAccelerated) + { + column = Blend(left, right, destination, width, column, leftWeight, rightWeight, minimum, maximum, Vector128.Zero); + } + + for (; column < width; column++) + { + int value = ((left[column] * leftWeight) + (right[column] * rightWeight) + 16) >> 5; + destination[column] = Av1Math.Clamp(value, minimum, maximum); + } + } + + /// + /// Blends complete 512-bit groups from one overlap row. + /// + /// The samples from the preceding block. + /// The samples from the following block. + /// The blended samples. + /// The number of samples to blend. + /// The first unprocessed sample. + /// The preceding-block weight. + /// The following-block weight. + /// The minimum grain value. + /// The maximum grain value. + /// The overload-selection value. + /// The first sample not processed by this vector width. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static int Blend( + ReadOnlySpan left, + ReadOnlySpan right, + Span destination, + int width, + int column, + int leftWeight, + int rightWeight, + int minimum, + int maximum, + Vector512 vector) + { + ref int leftBase = ref MemoryMarshal.GetReference(left); + ref int rightBase = ref MemoryMarshal.GetReference(right); + ref int destinationBase = ref MemoryMarshal.GetReference(destination); + Vector512 leftWeights = Vector512.Create(leftWeight); + Vector512 rightWeights = Vector512.Create(rightWeight); + Vector512 rounding = Vector512.Create(16); + Vector512 minima = Vector512.Create(minimum); + Vector512 maxima = Vector512.Create(maximum); + nuint vectorCount = Numerics.Vector512Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector512.Count) + { + Vector512 leftValues = Vector512.LoadUnsafe(ref leftBase, (nuint)column); + Vector512 rightValues = Vector512.LoadUnsafe(ref rightBase, (nuint)column); + Vector512 result = ((leftValues * leftWeights) + (rightValues * rightWeights) + rounding) >> 5; + Vector512.Min(Vector512.Max(result, minima), maxima).StoreUnsafe(ref destinationBase, (nuint)column); + } + + return column; + } + + /// + /// Blends complete 256-bit groups from one overlap row. + /// + /// The samples from the preceding block. + /// The samples from the following block. + /// The blended samples. + /// The number of samples to blend. + /// The first unprocessed sample. + /// The preceding-block weight. + /// The following-block weight. + /// The minimum grain value. + /// The maximum grain value. + /// The overload-selection value. + /// The first sample not processed by this vector width. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static int Blend( + ReadOnlySpan left, + ReadOnlySpan right, + Span destination, + int width, + int column, + int leftWeight, + int rightWeight, + int minimum, + int maximum, + Vector256 vector) + { + ref int leftBase = ref MemoryMarshal.GetReference(left); + ref int rightBase = ref MemoryMarshal.GetReference(right); + ref int destinationBase = ref MemoryMarshal.GetReference(destination); + Vector256 leftWeights = Vector256.Create(leftWeight); + Vector256 rightWeights = Vector256.Create(rightWeight); + Vector256 rounding = Vector256.Create(16); + Vector256 minima = Vector256.Create(minimum); + Vector256 maxima = Vector256.Create(maximum); + nuint vectorCount = Numerics.Vector256Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector256.Count) + { + Vector256 leftValues = Vector256.LoadUnsafe(ref leftBase, (nuint)column); + Vector256 rightValues = Vector256.LoadUnsafe(ref rightBase, (nuint)column); + Vector256 result = ((leftValues * leftWeights) + (rightValues * rightWeights) + rounding) >> 5; + Vector256.Min(Vector256.Max(result, minima), maxima).StoreUnsafe(ref destinationBase, (nuint)column); + } + + return column; + } + + /// + /// Blends complete 128-bit groups from one overlap row. + /// + /// The samples from the preceding block. + /// The samples from the following block. + /// The blended samples. + /// The number of samples to blend. + /// The first unprocessed sample. + /// The preceding-block weight. + /// The following-block weight. + /// The minimum grain value. + /// The maximum grain value. + /// The overload-selection value. + /// The first sample not processed by this vector width. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static int Blend( + ReadOnlySpan left, + ReadOnlySpan right, + Span destination, + int width, + int column, + int leftWeight, + int rightWeight, + int minimum, + int maximum, + Vector128 vector) + { + ref int leftBase = ref MemoryMarshal.GetReference(left); + ref int rightBase = ref MemoryMarshal.GetReference(right); + ref int destinationBase = ref MemoryMarshal.GetReference(destination); + Vector128 leftWeights = Vector128.Create(leftWeight); + Vector128 rightWeights = Vector128.Create(rightWeight); + Vector128 rounding = Vector128.Create(16); + Vector128 minima = Vector128.Create(minimum); + Vector128 maxima = Vector128.Create(maximum); + nuint vectorCount = Numerics.Vector128Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector128.Count) + { + Vector128 leftValues = Vector128.LoadUnsafe(ref leftBase, (nuint)column); + Vector128 rightValues = Vector128.LoadUnsafe(ref rightBase, (nuint)column); + Vector128 result = ((leftValues * leftWeights) + (rightValues * rightWeights) + rounding) >> 5; + Vector128.Min(Vector128.Max(result, minima), maxima).StoreUnsafe(ref destinationBase, (nuint)column); + } + + return column; + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/FilmGrain/Av1FilmGrainSampleOperations.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/FilmGrain/Av1FilmGrainSampleOperations.cs new file mode 100644 index 0000000000..eb855a7101 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/FilmGrain/Av1FilmGrainSampleOperations.cs @@ -0,0 +1,201 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.Intrinsics; +using System.Runtime.Intrinsics.X86; +using SixLabors.ImageSharp.Common.Helpers; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.FilmGrain; + +/// +/// Loads and stores native AV1 samples for the film-grain arithmetic pipeline. +/// +/// The native sample type. +/// +/// The decoder closes this operator over for eight-bit planes and for +/// high-bit-depth planes. The JIT removes the unused type branch, so widening and narrowing remain branch-free in the +/// row loops. All arithmetic uses signed 32-bit lanes; decoded samples are nonnegative and at most twelve bits, making +/// the intermediate signed 16-bit views safe wherever pairwise operations require them. +/// +internal readonly struct Av1FilmGrainSampleOperations + where TSample : unmanaged +{ + /// + /// Loads eight consecutive samples into 32-bit lanes. + /// + /// The first source sample. + /// The widened samples. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector256 Load8(ref TSample source) + { + if (typeof(TSample) == typeof(byte)) + { + // Reading exactly eight bytes avoids touching the following row when the visible width has no padding. + ref byte sourceBytes = ref Unsafe.As(ref source); + ulong packed = Unsafe.ReadUnaligned(ref sourceBytes); + return Avx2.ConvertToVector256Int32(Vector128.CreateScalarUnsafe(packed).AsByte()); + } + + ref ushort sourceValues = ref Unsafe.As(ref source); + return Avx2.ConvertToVector256Int32(Vector128.LoadUnsafe(ref sourceValues)); + } + + /// + /// Loads four consecutive samples into 32-bit lanes. + /// + /// The first source sample. + /// The widened samples. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector128 Load4(ref TSample source) + { + if (typeof(TSample) == typeof(byte)) + { + // The source occupies only the low four byte lanes; two lower-half widens preserve their original order. + ref byte sourceBytes = ref Unsafe.As(ref source); + uint packedBytes = Unsafe.ReadUnaligned(ref sourceBytes); + Vector128 widened = Vector128.WidenLower(Vector128.CreateScalarUnsafe(packedBytes).AsByte()); + return Vector128.WidenLower(widened).AsInt32(); + } + + ref ushort sourceValues = ref Unsafe.As(ref source); + ulong packedValues = Unsafe.ReadUnaligned(ref Unsafe.As(ref sourceValues)); + Vector64 packedSamples = Vector64.CreateScalarUnsafe(packedValues).AsUInt16(); + return Vector128.WidenLower(Vector128.Create(packedSamples, Vector64.Zero)).AsInt32(); + } + + /// + /// Loads the luma coordinates corresponding to eight chroma samples. + /// + /// The first luma sample. + /// The horizontal chroma subsampling shift. + /// The luma values used by chroma scaling. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector256 LoadChromaLuma8(ref TSample source, int subsamplingX) + { + if (subsamplingX == 0) + { + return Load8(ref source); + } + + Vector256 lumaPairs; + if (typeof(TSample) == typeof(byte)) + { + ref byte sourceBytes = ref Unsafe.As(ref source); + Vector128 packed = Vector128.LoadUnsafe(ref sourceBytes); + lumaPairs = Vector256.Create(Vector128.WidenLower(packed).AsInt16(), Vector128.WidenUpper(packed).AsInt16()); + } + else + { + ref ushort sourceValues = ref Unsafe.As(ref source); + lumaPairs = Vector256.LoadUnsafe(ref sourceValues).AsInt16(); + } + + // Horizontal 4:2:x chroma uses the rounded mean of each adjacent luma pair. + return (Avx2.MultiplyAddAdjacent(lumaPairs, Vector256.Create((short)1)) + Vector256.One) >> 1; + } + + /// + /// Loads the luma coordinates corresponding to four chroma samples. + /// + /// The first luma sample. + /// The horizontal chroma subsampling shift. + /// The luma values used by chroma scaling. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector128 LoadChromaLuma4(ref TSample source, int subsamplingX) + { + if (subsamplingX == 0) + { + return Load4(ref source); + } + + Vector128 lumaPairs; + if (typeof(TSample) == typeof(byte)) + { + ref byte sourceBytes = ref Unsafe.As(ref source); + ulong packed = Unsafe.ReadUnaligned(ref sourceBytes); + lumaPairs = Vector128.WidenLower(Vector128.CreateScalarUnsafe(packed).AsByte()).AsInt16(); + } + else + { + ref ushort sourceValues = ref Unsafe.As(ref source); + lumaPairs = Vector128.LoadUnsafe(ref sourceValues).AsInt16(); + } + + // Multiply-add with unity coefficients collapses four adjacent pairs without scalar deinterleaving. + return (Vector128_.MultiplyAddAdjacent(lumaPairs, Vector128.Create((short)1)) + Vector128.One) >> 1; + } + + /// + /// Stores eight already clipped 32-bit samples in their native representation. + /// + /// The first destination sample. + /// The samples to store. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void Store8(ref TSample destination, Vector256 values) + { + // AddNoise has already clipped every lane to the native sample range. Unsigned narrowing is therefore exact + // and serves only to repack lane width; it does not supply saturation or alter out-of-range values. + Vector128 packed = Vector128.Narrow(values.GetLower().AsUInt32(), values.GetUpper().AsUInt32()); + if (typeof(TSample) == typeof(byte)) + { + Vector64 bytes = Vector128.Narrow(packed, Vector128.Zero).GetLower(); + bytes.StoreUnsafe(ref Unsafe.As(ref destination)); + } + else + { + packed.StoreUnsafe(ref Unsafe.As(ref destination)); + } + } + + /// + /// Stores four already clipped 32-bit samples in their native representation. + /// + /// The first destination sample. + /// The samples to store. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void Store4(ref TSample destination, Vector128 values) + { + // Only four destination samples are valid. Narrow through the low half, then write four bytes or four UInt16 + // values so the store cannot overwrite the next row or an adjacent plane allocation. + Vector64 packed = Vector128.Narrow(values.AsUInt32(), Vector128.Zero).GetLower(); + if (typeof(TSample) == typeof(byte)) + { + Vector64 bytes = Vector128.Narrow(Vector128.Create(packed, Vector64.Zero), Vector128.Zero).GetLower(); + Unsafe.WriteUnaligned(ref Unsafe.As(ref destination), bytes.AsUInt32().GetElement(0)); + } + else + { + packed.StoreUnsafe(ref Unsafe.As(ref destination)); + } + } + + /// + /// Reads one native sample as an integer. + /// + /// The source sample. + /// The unsigned sample value. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static int Load(ref TSample source) + => typeof(TSample) == typeof(byte) + ? Unsafe.As(ref source) + : Unsafe.As(ref source); + + /// + /// Stores one already clipped sample in its native representation. + /// + /// The destination sample. + /// The sample value. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void Store(ref TSample destination, int value) + { + if (typeof(TSample) == typeof(byte)) + { + Unsafe.As(ref destination) = (byte)value; + } + else + { + Unsafe.As(ref destination) = (ushort)value; + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/IAv1FrameDecoder.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/IAv1FrameDecoder.cs new file mode 100644 index 0000000000..975b6bbe2d --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/IAv1FrameDecoder.cs @@ -0,0 +1,20 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline; + +/// +/// Defines reconstruction of decoded AV1 superblocks within one coded image frame. +/// +internal interface IAv1FrameDecoder +{ + /// + /// Reconstructs one decoded superblock into the current frame buffer. + /// + /// The superblock's top-left position in 4x4 mode-info units. + /// The decoded syntax and block modes for the superblock. + /// The tile that contains the superblock. + void DecodeSuperblock(Point modeInfoPosition, Av1SuperblockInfo superblockInfo, Av1TileInfo tileInfo); +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/LoopFilter/Av1DeblockingFilter.HorizontalByteEdgeOperator.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/LoopFilter/Av1DeblockingFilter.HorizontalByteEdgeOperator.cs new file mode 100644 index 0000000000..8871d36a45 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/LoopFilter/Av1DeblockingFilter.HorizontalByteEdgeOperator.cs @@ -0,0 +1,45 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.LoopFilter; + +internal static partial class Av1DeblockingFilter +{ + /// + /// Accesses four columns across a horizontal edge in eight-bit storage. + /// + private readonly struct HorizontalByteEdgeOperator : IEdgeOperator + { + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector128 LoadVector(ref byte samples, int q0Offset, int stride, int distance) + { + ref byte source = ref Unsafe.Add(ref samples, q0Offset + (distance * stride)); + uint packed = Unsafe.ReadUnaligned(ref source); + Vector128 widened = Vector128.WidenLower(Vector128.CreateScalarUnsafe(packed).AsByte()); + return Vector128.WidenLower(widened).AsInt32(); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void StoreVector(ref byte samples, int q0Offset, int stride, int distance, Vector128 value) + { + Vector128 narrowed16 = Vector128.Narrow(value.AsUInt32(), Vector128.Zero); + Vector128 narrowed8 = Vector128.Narrow(narrowed16, Vector128.Zero); + Unsafe.WriteUnaligned(ref Unsafe.Add(ref samples, q0Offset + (distance * stride)), narrowed8.AsUInt32().ToScalar()); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static int LoadScalar(ref byte samples, int q0Offset, int stride, int distance, int index) + => Unsafe.Add(ref samples, q0Offset + (distance * stride) + index); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void StoreScalar(ref byte samples, int q0Offset, int stride, int distance, int index, int value) + => Unsafe.Add(ref samples, q0Offset + (distance * stride) + index) = (byte)value; + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/LoopFilter/Av1DeblockingFilter.HorizontalUInt16EdgeOperator.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/LoopFilter/Av1DeblockingFilter.HorizontalUInt16EdgeOperator.cs new file mode 100644 index 0000000000..115b3bae55 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/LoopFilter/Av1DeblockingFilter.HorizontalUInt16EdgeOperator.cs @@ -0,0 +1,44 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.LoopFilter; + +internal static partial class Av1DeblockingFilter +{ + /// + /// Accesses four columns across a horizontal edge in 16-bit storage. + /// + private readonly struct HorizontalUInt16EdgeOperator : IEdgeOperator + { + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector128 LoadVector(ref ushort samples, int q0Offset, int stride, int distance) + { + ref ushort source = ref Unsafe.Add(ref samples, q0Offset + (distance * stride)); + ulong packed = Unsafe.ReadUnaligned(ref Unsafe.As(ref source)); + return Vector128.WidenLower(Vector128.CreateScalarUnsafe(packed).AsUInt16()).AsInt32(); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void StoreVector(ref ushort samples, int q0Offset, int stride, int distance, Vector128 value) + { + Vector64 narrowed = Vector128.Narrow(value, Vector128.Zero).AsUInt16().GetLower(); + ref byte destination = ref Unsafe.As(ref Unsafe.Add(ref samples, q0Offset + (distance * stride))); + Unsafe.WriteUnaligned(ref destination, narrowed.AsUInt64().ToScalar()); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static int LoadScalar(ref ushort samples, int q0Offset, int stride, int distance, int index) + => Unsafe.Add(ref samples, q0Offset + (distance * stride) + index); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void StoreScalar(ref ushort samples, int q0Offset, int stride, int distance, int index, int value) + => Unsafe.Add(ref samples, q0Offset + (distance * stride) + index) = (ushort)value; + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/LoopFilter/Av1DeblockingFilter.Operator.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/LoopFilter/Av1DeblockingFilter.Operator.cs new file mode 100644 index 0000000000..18f949b049 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/LoopFilter/Av1DeblockingFilter.Operator.cs @@ -0,0 +1,59 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.LoopFilter; + +internal static partial class Av1DeblockingFilter +{ + /// + /// Defines orientation- and storage-specific access to the four samples running along one edge segment. + /// + /// The reconstructed sample storage type. + private interface IEdgeOperator + where TSample : unmanaged + { + /// + /// Loads four samples at one signed distance across the edge. + /// + /// The first element in the plane storage. + /// The offset of the first Q-side sample. + /// The number of samples between adjacent rows. + /// The signed sample distance from Q0. + /// The widened samples ordered along the edge. + public static abstract Vector128 LoadVector(ref TSample samples, int q0Offset, int stride, int distance); + + /// + /// Stores four samples at one signed distance across the edge. + /// + /// The first element in the plane storage. + /// The offset of the first Q-side sample. + /// The number of samples between adjacent rows. + /// The signed sample distance from Q0. + /// The widened samples ordered along the edge. + public static abstract void StoreVector(ref TSample samples, int q0Offset, int stride, int distance, Vector128 value); + + /// + /// Loads one sample at a signed distance across and an offset along the edge. + /// + /// The first element in the plane storage. + /// The offset of the first Q-side sample. + /// The number of samples between adjacent rows. + /// The signed sample distance from Q0. + /// The sample offset along the edge. + /// The selected sample. + public static abstract int LoadScalar(ref TSample samples, int q0Offset, int stride, int distance, int index); + + /// + /// Stores one sample at a signed distance across and an offset along the edge. + /// + /// The first element in the plane storage. + /// The offset of the first Q-side sample. + /// The number of samples between adjacent rows. + /// The signed sample distance from Q0. + /// The sample offset along the edge. + /// The filtered sample. + public static abstract void StoreScalar(ref TSample samples, int q0Offset, int stride, int distance, int index, int value); + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/LoopFilter/Av1DeblockingFilter.VerticalByteEdgeOperator.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/LoopFilter/Av1DeblockingFilter.VerticalByteEdgeOperator.cs new file mode 100644 index 0000000000..cd491a755e --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/LoopFilter/Av1DeblockingFilter.VerticalByteEdgeOperator.cs @@ -0,0 +1,45 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.LoopFilter; + +internal static partial class Av1DeblockingFilter +{ + /// + /// Accesses four rows across a vertical edge in eight-bit storage. + /// + private readonly struct VerticalByteEdgeOperator : IEdgeOperator + { + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector128 LoadVector(ref byte samples, int q0Offset, int stride, int distance) + => Vector128.Create( + (int)Unsafe.Add(ref samples, q0Offset + distance), + Unsafe.Add(ref samples, q0Offset + stride + distance), + Unsafe.Add(ref samples, q0Offset + (2 * stride) + distance), + Unsafe.Add(ref samples, q0Offset + (3 * stride) + distance)); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void StoreVector(ref byte samples, int q0Offset, int stride, int distance, Vector128 value) + { + Unsafe.Add(ref samples, q0Offset + distance) = (byte)value.GetElement(0); + Unsafe.Add(ref samples, q0Offset + stride + distance) = (byte)value.GetElement(1); + Unsafe.Add(ref samples, q0Offset + (2 * stride) + distance) = (byte)value.GetElement(2); + Unsafe.Add(ref samples, q0Offset + (3 * stride) + distance) = (byte)value.GetElement(3); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static int LoadScalar(ref byte samples, int q0Offset, int stride, int distance, int index) + => Unsafe.Add(ref samples, q0Offset + (index * stride) + distance); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void StoreScalar(ref byte samples, int q0Offset, int stride, int distance, int index, int value) + => Unsafe.Add(ref samples, q0Offset + (index * stride) + distance) = (byte)value; + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/LoopFilter/Av1DeblockingFilter.VerticalUInt16EdgeOperator.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/LoopFilter/Av1DeblockingFilter.VerticalUInt16EdgeOperator.cs new file mode 100644 index 0000000000..707c4226c5 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/LoopFilter/Av1DeblockingFilter.VerticalUInt16EdgeOperator.cs @@ -0,0 +1,45 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.LoopFilter; + +internal static partial class Av1DeblockingFilter +{ + /// + /// Accesses four rows across a vertical edge in 16-bit storage. + /// + private readonly struct VerticalUInt16EdgeOperator : IEdgeOperator + { + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector128 LoadVector(ref ushort samples, int q0Offset, int stride, int distance) + => Vector128.Create( + (int)Unsafe.Add(ref samples, q0Offset + distance), + Unsafe.Add(ref samples, q0Offset + stride + distance), + Unsafe.Add(ref samples, q0Offset + (2 * stride) + distance), + Unsafe.Add(ref samples, q0Offset + (3 * stride) + distance)); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void StoreVector(ref ushort samples, int q0Offset, int stride, int distance, Vector128 value) + { + Unsafe.Add(ref samples, q0Offset + distance) = (ushort)value.GetElement(0); + Unsafe.Add(ref samples, q0Offset + stride + distance) = (ushort)value.GetElement(1); + Unsafe.Add(ref samples, q0Offset + (2 * stride) + distance) = (ushort)value.GetElement(2); + Unsafe.Add(ref samples, q0Offset + (3 * stride) + distance) = (ushort)value.GetElement(3); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static int LoadScalar(ref ushort samples, int q0Offset, int stride, int distance, int index) + => Unsafe.Add(ref samples, q0Offset + (index * stride) + distance); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void StoreScalar(ref ushort samples, int q0Offset, int stride, int distance, int index, int value) + => Unsafe.Add(ref samples, q0Offset + (index * stride) + distance) = (ushort)value; + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/LoopFilter/Av1DeblockingFilter.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/LoopFilter/Av1DeblockingFilter.cs new file mode 100644 index 0000000000..e9d36757b1 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/LoopFilter/Av1DeblockingFilter.cs @@ -0,0 +1,896 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.InteropServices; +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.LoopFilter; + +/// +/// Applies the AV1 deblocking kernels to four-sample edge segments. +/// +/// +/// Each 32-bit vector lane represents one sample position along the edge. The orientation-specific operators gather or +/// load the corresponding p6..p0,q0..q6 neighborhood, after which masks, flatness tests, and filter equations remain +/// lane-wise. Conditional selection preserves unfiltered lanes while allowing four adjacent edge positions to share +/// one kernel invocation. +/// +internal static partial class Av1DeblockingFilter +{ + /// + /// Filters four rows crossing one vertical boundary in eight-bit storage. + /// + /// The plane storage containing the edge and its neighboring samples. + /// The offset of the first Q-side sample. + /// The number of samples between adjacent rows. + /// The signaled 4-, 6-, 8-, or 14-tap filter length. + /// The threshold for adjacent samples on either side of the edge. + /// The threshold for the discontinuity across the edge. + /// The threshold that selects the narrow high-variance adjustment. + public static void FilterVertical( + Span samples, + int q0Offset, + int stride, + int filterLength, + int limit, + int boundaryLimit, + int highEdgeVarianceThreshold) + => Filter(samples, q0Offset, stride, filterLength, limit, boundaryLimit, highEdgeVarianceThreshold, 8); + + /// + /// Filters four columns crossing one horizontal boundary in eight-bit storage. + /// + /// The plane storage containing the edge and its neighboring samples. + /// The offset of the first Q-side sample. + /// The number of samples between adjacent rows. + /// The signaled 4-, 6-, 8-, or 14-tap filter length. + /// The threshold for adjacent samples on either side of the edge. + /// The threshold for the discontinuity across the edge. + /// The threshold that selects the narrow high-variance adjustment. + public static void FilterHorizontal( + Span samples, + int q0Offset, + int stride, + int filterLength, + int limit, + int boundaryLimit, + int highEdgeVarianceThreshold) + => Filter(samples, q0Offset, stride, filterLength, limit, boundaryLimit, highEdgeVarianceThreshold, 8); + + /// + /// Filters four rows crossing one vertical boundary in 16-bit storage. + /// + /// The plane storage containing the edge and its neighboring samples. + /// The offset of the first Q-side sample. + /// The number of samples between adjacent rows. + /// The signaled 4-, 6-, 8-, or 14-tap filter length. + /// The eight-bit-domain threshold for adjacent samples on either side of the edge. + /// The eight-bit-domain threshold for the discontinuity across the edge. + /// The eight-bit-domain threshold that selects the narrow high-variance adjustment. + /// The sample bit depth. + public static void FilterVertical( + Span samples, + int q0Offset, + int stride, + int filterLength, + int limit, + int boundaryLimit, + int highEdgeVarianceThreshold, + int bitDepth) + => Filter(samples, q0Offset, stride, filterLength, limit, boundaryLimit, highEdgeVarianceThreshold, bitDepth); + + /// + /// Filters four columns crossing one horizontal boundary in 16-bit storage. + /// + /// The plane storage containing the edge and its neighboring samples. + /// The offset of the first Q-side sample. + /// The number of samples between adjacent rows. + /// The signaled 4-, 6-, 8-, or 14-tap filter length. + /// The eight-bit-domain threshold for adjacent samples on either side of the edge. + /// The eight-bit-domain threshold for the discontinuity across the edge. + /// The eight-bit-domain threshold that selects the narrow high-variance adjustment. + /// The sample bit depth. + public static void FilterHorizontal( + Span samples, + int q0Offset, + int stride, + int filterLength, + int limit, + int boundaryLimit, + int highEdgeVarianceThreshold, + int bitDepth) + => Filter(samples, q0Offset, stride, filterLength, limit, boundaryLimit, highEdgeVarianceThreshold, bitDepth); + + /// + /// Selects the packed or scalar kernel through one closed edge-access operator. + /// + /// The reconstructed sample storage type. + /// The orientation- and storage-specific edge operator. + /// The plane storage containing the edge and its neighboring samples. + /// The offset of the first Q-side sample. + /// The number of samples between adjacent rows. + /// The signaled filter length. + /// The eight-bit-domain adjacent-sample threshold. + /// The eight-bit-domain edge-discontinuity threshold. + /// The eight-bit-domain high-edge-variance threshold. + /// The sample bit depth. + private static void Filter( + Span samples, + int q0Offset, + int stride, + int filterLength, + int limit, + int boundaryLimit, + int highEdgeVarianceThreshold, + int bitDepth) + where TSample : unmanaged + where TEdgeOperator : struct, IEdgeOperator + { + ref TSample sampleBase = ref MemoryMarshal.GetReference(samples); + if (Vector128.IsHardwareAccelerated) + { + FilterVector(ref sampleBase, q0Offset, stride, filterLength, limit, boundaryLimit, highEdgeVarianceThreshold, bitDepth); + return; + } + + FilterScalar(ref sampleBase, q0Offset, stride, filterLength, limit, boundaryLimit, highEdgeVarianceThreshold, bitDepth); + } + + /// + /// Applies one packed AV1 kernel with each 32-bit lane representing one row or column along the edge. + /// + /// The reconstructed sample storage type. + /// The orientation- and storage-specific edge operator. + /// The first element in the plane storage. + /// The offset of the first Q-side sample. + /// The number of samples between adjacent rows. + /// The signaled filter length. + /// The eight-bit-domain adjacent-sample threshold. + /// The eight-bit-domain edge-discontinuity threshold. + /// The eight-bit-domain high-edge-variance threshold. + /// The sample bit depth. + private static void FilterVector( + ref TSample samples, + int q0Offset, + int stride, + int filterLength, + int limit, + int boundaryLimit, + int highEdgeVarianceThreshold, + int bitDepth) + where TSample : unmanaged + where TEdgeOperator : struct, IEdgeOperator + { + int radius = GetFilterRadius(filterLength); + InlineArray14> window = default; + + // AV1 names the samples p6..p0,q0..q6. Loading them into that exact order lets the packed equations below + // follow the normative scalar formulas without lane shuffles or an intermediate per-edge sample buffer. + for (int distance = 1; distance <= radius; distance++) + { + window[7 - distance] = TEdgeOperator.LoadVector(ref samples, q0Offset, stride, -distance); + window[6 + distance] = TEdgeOperator.LoadVector(ref samples, q0Offset, stride, distance - 1); + } + + FilterSamples(ref window, filterLength, limit, boundaryLimit, highEdgeVarianceThreshold, bitDepth); + + int modifiedRadius = GetModifiedRadius(filterLength); + for (int distance = 1; distance <= modifiedRadius; distance++) + { + TEdgeOperator.StoreVector(ref samples, q0Offset, stride, -distance, window[7 - distance]); + TEdgeOperator.StoreVector(ref samples, q0Offset, stride, distance - 1, window[6 + distance]); + } + } + + /// + /// Applies the scalar fallback to each of the four samples along an edge. + /// + /// The reconstructed sample storage type. + /// The orientation- and storage-specific edge operator. + /// The first element in the plane storage. + /// The offset of the first Q-side sample. + /// The number of samples between adjacent rows. + /// The signaled filter length. + /// The eight-bit-domain adjacent-sample threshold. + /// The eight-bit-domain edge-discontinuity threshold. + /// The eight-bit-domain high-edge-variance threshold. + /// The sample bit depth. + private static void FilterScalar( + ref TSample samples, + int q0Offset, + int stride, + int filterLength, + int limit, + int boundaryLimit, + int highEdgeVarianceThreshold, + int bitDepth) + where TSample : unmanaged + where TEdgeOperator : struct, IEdgeOperator + { + int radius = GetFilterRadius(filterLength); + int modifiedRadius = GetModifiedRadius(filterLength); + InlineArray14 window = default; + Span sampleWindow = window; + + for (int index = 0; index < 4; index++) + { + // The fixed p6..q6 window is reused for all four lanes. Every value consumed by the selected kernel is + // overwritten before filtering, so the fallback requires neither per-lane clearing nor stack allocation. + for (int distance = 1; distance <= radius; distance++) + { + sampleWindow[7 - distance] = TEdgeOperator.LoadScalar(ref samples, q0Offset, stride, -distance, index); + sampleWindow[6 + distance] = TEdgeOperator.LoadScalar(ref samples, q0Offset, stride, distance - 1, index); + } + + FilterSamples(sampleWindow, filterLength, limit, boundaryLimit, highEdgeVarianceThreshold, bitDepth); + + for (int distance = 1; distance <= modifiedRadius; distance++) + { + TEdgeOperator.StoreScalar(ref samples, q0Offset, stride, -distance, index, sampleWindow[7 - distance]); + TEdgeOperator.StoreScalar(ref samples, q0Offset, stride, distance - 1, index, sampleWindow[6 + distance]); + } + } + } + + /// + /// Gets the number of samples read from each side of an edge for a filter length. + /// + /// The AV1 filter length. + /// The sample radius on either side of the edge. + private static int GetFilterRadius(int filterLength) => filterLength switch + { + 4 => 2, + 6 => 3, + 8 => 4, + 14 => 7, + _ => 0 + }; + + /// + /// Gets the number of samples that a filter can modify on each side of an edge. + /// + /// The AV1 filter length. + /// The modified sample radius on either side of the edge. + private static int GetModifiedRadius(int filterLength) => filterLength switch + { + 4 or 6 => 2, + 8 => 3, + 14 => 6, + _ => 0 + }; + + /// + /// Selects and applies the packed filter arithmetic for four edge lanes. + /// + /// The packed p6..p0,q0..q6 sample window. + /// The signaled filter length. + /// The eight-bit-domain adjacent-sample threshold. + /// The eight-bit-domain edge-discontinuity threshold. + /// The eight-bit-domain high-edge-variance threshold. + /// The sample bit depth. + private static void FilterSamples( + ref InlineArray14> samples, + int filterLength, + int limit, + int boundaryLimit, + int highEdgeVarianceThreshold, + int bitDepth) + { + int thresholdScale = 1 << (bitDepth - 8); + Vector128 limitVector = Vector128.Create(limit * thresholdScale); + Vector128 boundaryLimitVector = Vector128.Create(boundaryLimit * thresholdScale); + Vector128 varianceThresholdVector = Vector128.Create(highEdgeVarianceThreshold * thresholdScale); + Vector128 flatnessThresholdVector = Vector128.Create(thresholdScale); + + switch (filterLength) + { + case 4: + Filter4(ref samples, IsFilter2Enabled(ref samples, limitVector, boundaryLimitVector), varianceThresholdVector, bitDepth); + break; + case 6: + Filter6( + ref samples, + IsChromaFilterEnabled(ref samples, limitVector, boundaryLimitVector), + IsChromaFlat(ref samples, flatnessThresholdVector), + varianceThresholdVector, + bitDepth); + break; + case 8: + Filter8( + ref samples, + IsFilterEnabled(ref samples, limitVector, boundaryLimitVector), + IsFlat(ref samples, flatnessThresholdVector), + varianceThresholdVector, + bitDepth); + break; + case 14: + Filter14( + ref samples, + IsFilterEnabled(ref samples, limitVector, boundaryLimitVector), + IsFlat(ref samples, flatnessThresholdVector), + IsOuterFlat(ref samples, flatnessThresholdVector), + varianceThresholdVector, + bitDepth); + break; + } + } + + /// + /// Selects and applies the scalar filter arithmetic for one edge lane. + /// + /// The scalar p6..p0,q0..q6 sample window. + /// The signaled filter length. + /// The eight-bit-domain adjacent-sample threshold. + /// The eight-bit-domain edge-discontinuity threshold. + /// The eight-bit-domain high-edge-variance threshold. + /// The sample bit depth. + private static void FilterSamples( + Span samples, + int filterLength, + int limit, + int boundaryLimit, + int highEdgeVarianceThreshold, + int bitDepth) + { + int thresholdScale = 1 << (bitDepth - 8); + int scaledLimit = limit * thresholdScale; + int scaledBoundaryLimit = boundaryLimit * thresholdScale; + + switch (filterLength) + { + case 4: + Filter4(samples, IsFilter2Enabled(samples, scaledLimit, scaledBoundaryLimit), highEdgeVarianceThreshold * thresholdScale, bitDepth); + break; + case 6: + Filter6( + samples, + IsChromaFilterEnabled(samples, scaledLimit, scaledBoundaryLimit), + IsChromaFlat(samples, thresholdScale), + highEdgeVarianceThreshold * thresholdScale, + bitDepth); + break; + case 8: + Filter8( + samples, + IsFilterEnabled(samples, scaledLimit, scaledBoundaryLimit), + IsFlat(samples, thresholdScale), + highEdgeVarianceThreshold * thresholdScale, + bitDepth); + break; + case 14: + Filter14( + samples, + IsFilterEnabled(samples, scaledLimit, scaledBoundaryLimit), + IsFlat(samples, thresholdScale), + IsOuterFlat(samples, thresholdScale), + highEdgeVarianceThreshold * thresholdScale, + bitDepth); + break; + } + } + + /// + /// Determines which packed lanes satisfy the four-tap AV1 filter mask. + /// + /// The packed p6..p0,q0..q6 sample window. + /// The scaled adjacent-sample threshold. + /// The scaled edge-discontinuity threshold. + /// A mask containing all bits set in each enabled lane. + private static Vector128 IsFilter2Enabled(ref InlineArray14> samples, Vector128 limit, Vector128 boundaryLimit) + => Vector128.LessThanOrEqual(Vector128.Abs(samples[5] - samples[6]), limit) + & Vector128.LessThanOrEqual(Vector128.Abs(samples[8] - samples[7]), limit) + & Vector128.LessThanOrEqual((Vector128.Abs(samples[6] - samples[7]) * 2) + (Vector128.Abs(samples[5] - samples[8]) >> 1), boundaryLimit); + + /// + /// Determines which packed lanes satisfy the eight- and fourteen-tap AV1 filter mask. + /// + /// The packed p6..p0,q0..q6 sample window. + /// The scaled adjacent-sample threshold. + /// The scaled edge-discontinuity threshold. + /// A mask containing all bits set in each enabled lane. + private static Vector128 IsFilterEnabled(ref InlineArray14> samples, Vector128 limit, Vector128 boundaryLimit) + => Vector128.LessThanOrEqual(Vector128.Abs(samples[3] - samples[4]), limit) + & Vector128.LessThanOrEqual(Vector128.Abs(samples[4] - samples[5]), limit) + & Vector128.LessThanOrEqual(Vector128.Abs(samples[5] - samples[6]), limit) + & Vector128.LessThanOrEqual(Vector128.Abs(samples[8] - samples[7]), limit) + & Vector128.LessThanOrEqual(Vector128.Abs(samples[9] - samples[8]), limit) + & Vector128.LessThanOrEqual(Vector128.Abs(samples[10] - samples[9]), limit) + & Vector128.LessThanOrEqual((Vector128.Abs(samples[6] - samples[7]) * 2) + (Vector128.Abs(samples[5] - samples[8]) >> 1), boundaryLimit); + + /// + /// Determines which packed lanes satisfy the six-tap chroma filter mask. + /// + /// The packed p6..p0,q0..q6 sample window. + /// The scaled adjacent-sample threshold. + /// The scaled edge-discontinuity threshold. + /// A mask containing all bits set in each enabled lane. + private static Vector128 IsChromaFilterEnabled(ref InlineArray14> samples, Vector128 limit, Vector128 boundaryLimit) + => Vector128.LessThanOrEqual(Vector128.Abs(samples[4] - samples[5]), limit) + & Vector128.LessThanOrEqual(Vector128.Abs(samples[5] - samples[6]), limit) + & Vector128.LessThanOrEqual(Vector128.Abs(samples[8] - samples[7]), limit) + & Vector128.LessThanOrEqual(Vector128.Abs(samples[9] - samples[8]), limit) + & Vector128.LessThanOrEqual((Vector128.Abs(samples[6] - samples[7]) * 2) + (Vector128.Abs(samples[5] - samples[8]) >> 1), boundaryLimit); + + /// + /// Determines which packed lanes satisfy the inner flatness mask. + /// + /// The packed p6..p0,q0..q6 sample window. + /// The scaled flatness threshold. + /// A mask containing all bits set in each flat lane. + private static Vector128 IsFlat(ref InlineArray14> samples, Vector128 threshold) + => Vector128.LessThanOrEqual(Vector128.Abs(samples[5] - samples[6]), threshold) + & Vector128.LessThanOrEqual(Vector128.Abs(samples[8] - samples[7]), threshold) + & Vector128.LessThanOrEqual(Vector128.Abs(samples[4] - samples[6]), threshold) + & Vector128.LessThanOrEqual(Vector128.Abs(samples[9] - samples[7]), threshold) + & Vector128.LessThanOrEqual(Vector128.Abs(samples[3] - samples[6]), threshold) + & Vector128.LessThanOrEqual(Vector128.Abs(samples[10] - samples[7]), threshold); + + /// + /// Determines which packed lanes satisfy the chroma flatness mask. + /// + /// The packed p6..p0,q0..q6 sample window. + /// The scaled flatness threshold. + /// A mask containing all bits set in each flat lane. + private static Vector128 IsChromaFlat(ref InlineArray14> samples, Vector128 threshold) + => Vector128.LessThanOrEqual(Vector128.Abs(samples[5] - samples[6]), threshold) + & Vector128.LessThanOrEqual(Vector128.Abs(samples[8] - samples[7]), threshold) + & Vector128.LessThanOrEqual(Vector128.Abs(samples[4] - samples[6]), threshold) + & Vector128.LessThanOrEqual(Vector128.Abs(samples[9] - samples[7]), threshold); + + /// + /// Determines which packed lanes satisfy the outer fourteen-tap flatness mask. + /// + /// The packed p6..p0,q0..q6 sample window. + /// The scaled flatness threshold. + /// A mask containing all bits set in each flat lane. + private static Vector128 IsOuterFlat(ref InlineArray14> samples, Vector128 threshold) + => Vector128.LessThanOrEqual(Vector128.Abs(samples[1] - samples[6]), threshold) + & Vector128.LessThanOrEqual(Vector128.Abs(samples[11] - samples[7]), threshold) + & Vector128.LessThanOrEqual(Vector128.Abs(samples[2] - samples[6]), threshold) + & Vector128.LessThanOrEqual(Vector128.Abs(samples[12] - samples[7]), threshold) + & Vector128.LessThanOrEqual(Vector128.Abs(samples[0] - samples[6]), threshold) + & Vector128.LessThanOrEqual(Vector128.Abs(samples[13] - samples[7]), threshold); + + /// + /// Applies the packed narrow signed-saturating AV1 edge adjustment. + /// + /// The packed p6..p0,q0..q6 sample window. + /// The per-lane filter-enable mask. + /// The scaled high-edge-variance threshold. + /// The sample bit depth. + private static void Filter4( + ref InlineArray14> samples, + Vector128 filterEnabled, + Vector128 highEdgeVarianceThreshold, + int bitDepth) + { + Vector128 p1 = samples[5]; + Vector128 p0 = samples[6]; + Vector128 q0 = samples[7]; + Vector128 q1 = samples[8]; + Vector128 highEdgeVariance = Vector128.GreaterThan(Vector128.Abs(p1 - p0), highEdgeVarianceThreshold) + | Vector128.GreaterThan(Vector128.Abs(q1 - q0), highEdgeVarianceThreshold); + + int signedOffset = 128 << (bitDepth - 8); + Vector128 minimum = Vector128.Create(-signedOffset); + Vector128 maximum = Vector128.Create(signedOffset - 1); + Vector128 offset = Vector128.Create(signedOffset); + Vector128 signedP1 = p1 - offset; + Vector128 signedP0 = p0 - offset; + Vector128 signedQ0 = q0 - offset; + Vector128 signedQ1 = q1 - offset; + + // the reference decoder performs every delta operation in the signed sample domain. Saturating only the final samples is + // not equivalent because the intermediate delta can clip before the asymmetric +4/+3 rounding is applied. + Vector128 filter = Vector128.ConditionalSelect(highEdgeVariance, Vector128.Clamp(signedP1 - signedQ1, minimum, maximum), Vector128.Zero); + filter = Vector128.Clamp(filter + (3 * (signedQ0 - signedP0)), minimum, maximum) & filterEnabled; + + Vector128 filter1 = Vector128.Clamp(filter + Vector128.Create(4), minimum, maximum) >> 3; + Vector128 filter2 = Vector128.Clamp(filter + Vector128.Create(3), minimum, maximum) >> 3; + samples[7] = Vector128.Clamp(signedQ0 - filter1, minimum, maximum) + offset; + samples[6] = Vector128.Clamp(signedP0 + filter2, minimum, maximum) + offset; + + Vector128 outerFilter = ((filter1 + Vector128.One) >> 1) & ~highEdgeVariance; + samples[8] = Vector128.Clamp(signedQ1 - outerFilter, minimum, maximum) + offset; + samples[5] = Vector128.Clamp(signedP1 + outerFilter, minimum, maximum) + offset; + } + + /// + /// Applies the packed six-tap chroma filter or its four-tap fallback. + /// + /// The packed p6..p0,q0..q6 sample window. + /// The per-lane filter-enable mask. + /// The per-lane inner-flatness mask. + /// The scaled high-edge-variance threshold. + /// The sample bit depth. + private static void Filter6( + ref InlineArray14> samples, + Vector128 filterEnabled, + Vector128 flat, + Vector128 highEdgeVarianceThreshold, + int bitDepth) + { + Vector128 p2 = samples[4]; + Vector128 p1 = samples[5]; + Vector128 p0 = samples[6]; + Vector128 q0 = samples[7]; + Vector128 q1 = samples[8]; + Vector128 q2 = samples[9]; + Vector128 wideFilter = filterEnabled & flat; + + Filter4(ref samples, filterEnabled, highEdgeVarianceThreshold, bitDepth); + if (Vector128.EqualsAll(wideFilter, Vector128.Zero)) + { + return; + } + + Vector128 filteredP1 = RoundPowerOfTwo((3 * p2) + (2 * p1) + (2 * p0) + q0, 3); + Vector128 filteredP0 = RoundPowerOfTwo(p2 + (2 * p1) + (2 * p0) + (2 * q0) + q1, 3); + Vector128 filteredQ0 = RoundPowerOfTwo(p1 + (2 * p0) + (2 * q0) + (2 * q1) + q2, 3); + Vector128 filteredQ1 = RoundPowerOfTwo(p0 + (2 * q0) + (2 * q1) + (3 * q2), 3); + samples[5] = Vector128.ConditionalSelect(wideFilter, filteredP1, samples[5]); + samples[6] = Vector128.ConditionalSelect(wideFilter, filteredP0, samples[6]); + samples[7] = Vector128.ConditionalSelect(wideFilter, filteredQ0, samples[7]); + samples[8] = Vector128.ConditionalSelect(wideFilter, filteredQ1, samples[8]); + } + + /// + /// Applies the packed eight-tap luma filter or its four-tap fallback. + /// + /// The packed p6..p0,q0..q6 sample window. + /// The per-lane filter-enable mask. + /// The per-lane inner-flatness mask. + /// The scaled high-edge-variance threshold. + /// The sample bit depth. + private static void Filter8( + ref InlineArray14> samples, + Vector128 filterEnabled, + Vector128 flat, + Vector128 highEdgeVarianceThreshold, + int bitDepth) + { + Vector128 p3 = samples[3]; + Vector128 p2 = samples[4]; + Vector128 p1 = samples[5]; + Vector128 p0 = samples[6]; + Vector128 q0 = samples[7]; + Vector128 q1 = samples[8]; + Vector128 q2 = samples[9]; + Vector128 q3 = samples[10]; + Vector128 wideFilter = filterEnabled & flat; + + Filter4(ref samples, filterEnabled, highEdgeVarianceThreshold, bitDepth); + if (Vector128.EqualsAll(wideFilter, Vector128.Zero)) + { + return; + } + + Vector128 filteredP2 = RoundPowerOfTwo((3 * p3) + (2 * p2) + p1 + p0 + q0, 3); + Vector128 filteredP1 = RoundPowerOfTwo((2 * p3) + p2 + (2 * p1) + p0 + q0 + q1, 3); + Vector128 filteredP0 = RoundPowerOfTwo(p3 + p2 + p1 + (2 * p0) + q0 + q1 + q2, 3); + Vector128 filteredQ0 = RoundPowerOfTwo(p2 + p1 + p0 + (2 * q0) + q1 + q2 + q3, 3); + Vector128 filteredQ1 = RoundPowerOfTwo(p1 + p0 + q0 + (2 * q1) + q2 + (2 * q3), 3); + Vector128 filteredQ2 = RoundPowerOfTwo(p0 + q0 + q1 + (2 * q2) + (3 * q3), 3); + samples[4] = Vector128.ConditionalSelect(wideFilter, filteredP2, samples[4]); + samples[5] = Vector128.ConditionalSelect(wideFilter, filteredP1, samples[5]); + samples[6] = Vector128.ConditionalSelect(wideFilter, filteredP0, samples[6]); + samples[7] = Vector128.ConditionalSelect(wideFilter, filteredQ0, samples[7]); + samples[8] = Vector128.ConditionalSelect(wideFilter, filteredQ1, samples[8]); + samples[9] = Vector128.ConditionalSelect(wideFilter, filteredQ2, samples[9]); + } + + /// + /// Applies the packed fourteen-tap luma filter or its eight- and four-tap fallbacks. + /// + /// The packed p6..p0,q0..q6 sample window. + /// The per-lane filter-enable mask. + /// The per-lane inner-flatness mask. + /// The per-lane outer-flatness mask. + /// The scaled high-edge-variance threshold. + /// The sample bit depth. + private static void Filter14( + ref InlineArray14> samples, + Vector128 filterEnabled, + Vector128 flat, + Vector128 outerFlat, + Vector128 highEdgeVarianceThreshold, + int bitDepth) + { + Vector128 p6 = samples[0]; + Vector128 p5 = samples[1]; + Vector128 p4 = samples[2]; + Vector128 p3 = samples[3]; + Vector128 p2 = samples[4]; + Vector128 p1 = samples[5]; + Vector128 p0 = samples[6]; + Vector128 q0 = samples[7]; + Vector128 q1 = samples[8]; + Vector128 q2 = samples[9]; + Vector128 q3 = samples[10]; + Vector128 q4 = samples[11]; + Vector128 q5 = samples[12]; + Vector128 q6 = samples[13]; + Vector128 wideFilter = filterEnabled & flat & outerFlat; + + // The eight-tap routine first produces the normative fallback. Lanes satisfying the outer flatness mask are + // then replaced with the wider results, matching the reference decoder's mask blend without evaluating lanes independently. + Filter8(ref samples, filterEnabled, flat, highEdgeVarianceThreshold, bitDepth); + if (Vector128.EqualsAll(wideFilter, Vector128.Zero)) + { + return; + } + + Vector128 filteredP5 = RoundPowerOfTwo((7 * p6) + (2 * p5) + (2 * p4) + p3 + p2 + p1 + p0 + q0, 4); + Vector128 filteredP4 = RoundPowerOfTwo((5 * p6) + (2 * p5) + (2 * p4) + (2 * p3) + p2 + p1 + p0 + q0 + q1, 4); + Vector128 filteredP3 = RoundPowerOfTwo((4 * p6) + p5 + (2 * p4) + (2 * p3) + (2 * p2) + p1 + p0 + q0 + q1 + q2, 4); + Vector128 filteredP2 = RoundPowerOfTwo((3 * p6) + p5 + p4 + (2 * p3) + (2 * p2) + (2 * p1) + p0 + q0 + q1 + q2 + q3, 4); + Vector128 filteredP1 = RoundPowerOfTwo((2 * p6) + p5 + p4 + p3 + (2 * p2) + (2 * p1) + (2 * p0) + q0 + q1 + q2 + q3 + q4, 4); + Vector128 filteredP0 = RoundPowerOfTwo(p6 + p5 + p4 + p3 + p2 + (2 * p1) + (2 * p0) + (2 * q0) + q1 + q2 + q3 + q4 + q5, 4); + Vector128 filteredQ0 = RoundPowerOfTwo(p5 + p4 + p3 + p2 + p1 + (2 * p0) + (2 * q0) + (2 * q1) + q2 + q3 + q4 + q5 + q6, 4); + Vector128 filteredQ1 = RoundPowerOfTwo(p4 + p3 + p2 + p1 + p0 + (2 * q0) + (2 * q1) + (2 * q2) + q3 + q4 + q5 + (2 * q6), 4); + Vector128 filteredQ2 = RoundPowerOfTwo(p3 + p2 + p1 + p0 + q0 + (2 * q1) + (2 * q2) + (2 * q3) + q4 + q5 + (3 * q6), 4); + Vector128 filteredQ3 = RoundPowerOfTwo(p2 + p1 + p0 + q0 + q1 + (2 * q2) + (2 * q3) + (2 * q4) + q5 + (4 * q6), 4); + Vector128 filteredQ4 = RoundPowerOfTwo(p1 + p0 + q0 + q1 + q2 + (2 * q3) + (2 * q4) + (2 * q5) + (5 * q6), 4); + Vector128 filteredQ5 = RoundPowerOfTwo(p0 + q0 + q1 + q2 + q3 + (2 * q4) + (2 * q5) + (7 * q6), 4); + + samples[1] = Vector128.ConditionalSelect(wideFilter, filteredP5, samples[1]); + samples[2] = Vector128.ConditionalSelect(wideFilter, filteredP4, samples[2]); + samples[3] = Vector128.ConditionalSelect(wideFilter, filteredP3, samples[3]); + samples[4] = Vector128.ConditionalSelect(wideFilter, filteredP2, samples[4]); + samples[5] = Vector128.ConditionalSelect(wideFilter, filteredP1, samples[5]); + samples[6] = Vector128.ConditionalSelect(wideFilter, filteredP0, samples[6]); + samples[7] = Vector128.ConditionalSelect(wideFilter, filteredQ0, samples[7]); + samples[8] = Vector128.ConditionalSelect(wideFilter, filteredQ1, samples[8]); + samples[9] = Vector128.ConditionalSelect(wideFilter, filteredQ2, samples[9]); + samples[10] = Vector128.ConditionalSelect(wideFilter, filteredQ3, samples[10]); + samples[11] = Vector128.ConditionalSelect(wideFilter, filteredQ4, samples[11]); + samples[12] = Vector128.ConditionalSelect(wideFilter, filteredQ5, samples[12]); + } + + /// + /// Determines whether a scalar lane satisfies the four-tap AV1 filter mask. + /// + /// The scalar p6..p0,q0..q6 sample window. + /// The scaled adjacent-sample threshold. + /// The scaled edge-discontinuity threshold. + /// when the edge satisfies the filter mask. + private static bool IsFilter2Enabled(ReadOnlySpan samples, int limit, int boundaryLimit) + => Math.Abs(samples[5] - samples[6]) <= limit + && Math.Abs(samples[8] - samples[7]) <= limit + && ((2 * Math.Abs(samples[6] - samples[7])) + (Math.Abs(samples[5] - samples[8]) / 2)) <= boundaryLimit; + + /// + /// Determines whether a scalar lane satisfies the eight- and fourteen-tap AV1 filter mask. + /// + /// The scalar p6..p0,q0..q6 sample window. + /// The scaled adjacent-sample threshold. + /// The scaled edge-discontinuity threshold. + /// when the edge satisfies the filter mask. + private static bool IsFilterEnabled(ReadOnlySpan samples, int limit, int boundaryLimit) + => Math.Abs(samples[3] - samples[4]) <= limit + && Math.Abs(samples[4] - samples[5]) <= limit + && Math.Abs(samples[5] - samples[6]) <= limit + && Math.Abs(samples[8] - samples[7]) <= limit + && Math.Abs(samples[9] - samples[8]) <= limit + && Math.Abs(samples[10] - samples[9]) <= limit + && ((2 * Math.Abs(samples[6] - samples[7])) + (Math.Abs(samples[5] - samples[8]) / 2)) <= boundaryLimit; + + /// + /// Determines whether a scalar lane satisfies the six-tap chroma filter mask. + /// + /// The scalar p6..p0,q0..q6 sample window. + /// The scaled adjacent-sample threshold. + /// The scaled edge-discontinuity threshold. + /// when the edge satisfies the filter mask. + private static bool IsChromaFilterEnabled(ReadOnlySpan samples, int limit, int boundaryLimit) + => Math.Abs(samples[4] - samples[5]) <= limit + && Math.Abs(samples[5] - samples[6]) <= limit + && Math.Abs(samples[8] - samples[7]) <= limit + && Math.Abs(samples[9] - samples[8]) <= limit + && ((2 * Math.Abs(samples[6] - samples[7])) + (Math.Abs(samples[5] - samples[8]) / 2)) <= boundaryLimit; + + /// + /// Determines whether a scalar lane satisfies the inner flatness mask. + /// + /// The scalar p6..p0,q0..q6 sample window. + /// The scaled flatness threshold. + /// when the edge satisfies the flatness mask. + private static bool IsFlat(ReadOnlySpan samples, int threshold) + => Math.Abs(samples[5] - samples[6]) <= threshold + && Math.Abs(samples[8] - samples[7]) <= threshold + && Math.Abs(samples[4] - samples[6]) <= threshold + && Math.Abs(samples[9] - samples[7]) <= threshold + && Math.Abs(samples[3] - samples[6]) <= threshold + && Math.Abs(samples[10] - samples[7]) <= threshold; + + /// + /// Determines whether a scalar lane satisfies the chroma flatness mask. + /// + /// The scalar p6..p0,q0..q6 sample window. + /// The scaled flatness threshold. + /// when the edge satisfies the flatness mask. + private static bool IsChromaFlat(ReadOnlySpan samples, int threshold) + => Math.Abs(samples[5] - samples[6]) <= threshold + && Math.Abs(samples[8] - samples[7]) <= threshold + && Math.Abs(samples[4] - samples[6]) <= threshold + && Math.Abs(samples[9] - samples[7]) <= threshold; + + /// + /// Determines whether a scalar lane satisfies the outer fourteen-tap flatness mask. + /// + /// The scalar p6..p0,q0..q6 sample window. + /// The scaled flatness threshold. + /// when the edge satisfies the flatness mask. + private static bool IsOuterFlat(ReadOnlySpan samples, int threshold) + => Math.Abs(samples[1] - samples[6]) <= threshold + && Math.Abs(samples[11] - samples[7]) <= threshold + && Math.Abs(samples[2] - samples[6]) <= threshold + && Math.Abs(samples[12] - samples[7]) <= threshold + && Math.Abs(samples[0] - samples[6]) <= threshold + && Math.Abs(samples[13] - samples[7]) <= threshold; + + /// + /// Applies the scalar narrow signed-saturating AV1 edge adjustment. + /// + /// The scalar p6..p0,q0..q6 sample window. + /// Whether the edge satisfies the filter mask. + /// The scaled high-edge-variance threshold. + /// The sample bit depth. + private static void Filter4(Span samples, bool filterEnabled, int highEdgeVarianceThreshold, int bitDepth) + { + if (!filterEnabled) + { + return; + } + + int signedOffset = 128 << (bitDepth - 8); + int signedMinimum = -signedOffset; + int signedMaximum = signedOffset - 1; + int p1 = samples[5] - signedOffset; + int p0 = samples[6] - signedOffset; + int q0 = samples[7] - signedOffset; + int q1 = samples[8] - signedOffset; + bool highEdgeVariance = Math.Abs(samples[5] - samples[6]) > highEdgeVarianceThreshold + || Math.Abs(samples[8] - samples[7]) > highEdgeVarianceThreshold; + + int filter = highEdgeVariance ? Math.Clamp(p1 - q1, signedMinimum, signedMaximum) : 0; + filter = Math.Clamp(filter + (3 * (q0 - p0)), signedMinimum, signedMaximum); + + int filter1 = Math.Clamp(filter + 4, signedMinimum, signedMaximum) >> 3; + int filter2 = Math.Clamp(filter + 3, signedMinimum, signedMaximum) >> 3; + samples[7] = Math.Clamp(q0 - filter1, signedMinimum, signedMaximum) + signedOffset; + samples[6] = Math.Clamp(p0 + filter2, signedMinimum, signedMaximum) + signedOffset; + + int outerFilter = highEdgeVariance ? 0 : (filter1 + 1) >> 1; + samples[8] = Math.Clamp(q1 - outerFilter, signedMinimum, signedMaximum) + signedOffset; + samples[5] = Math.Clamp(p1 + outerFilter, signedMinimum, signedMaximum) + signedOffset; + } + + /// + /// Applies the scalar six-tap chroma filter or its four-tap fallback. + /// + /// The scalar p6..p0,q0..q6 sample window. + /// Whether the edge satisfies the filter mask. + /// Whether the edge satisfies the inner-flatness mask. + /// The scaled high-edge-variance threshold. + /// The sample bit depth. + private static void Filter6(Span samples, bool filterEnabled, bool flat, int highEdgeVarianceThreshold, int bitDepth) + { + if (filterEnabled && flat) + { + int p2 = samples[4]; + int p1 = samples[5]; + int p0 = samples[6]; + int q0 = samples[7]; + int q1 = samples[8]; + int q2 = samples[9]; + + samples[5] = RoundPowerOfTwo((3 * p2) + (2 * p1) + (2 * p0) + q0, 3); + samples[6] = RoundPowerOfTwo(p2 + (2 * p1) + (2 * p0) + (2 * q0) + q1, 3); + samples[7] = RoundPowerOfTwo(p1 + (2 * p0) + (2 * q0) + (2 * q1) + q2, 3); + samples[8] = RoundPowerOfTwo(p0 + (2 * q0) + (2 * q1) + (3 * q2), 3); + return; + } + + Filter4(samples, filterEnabled, highEdgeVarianceThreshold, bitDepth); + } + + /// + /// Applies the scalar eight-tap luma filter or its four-tap fallback. + /// + /// The scalar p6..p0,q0..q6 sample window. + /// Whether the edge satisfies the filter mask. + /// Whether the edge satisfies the inner-flatness mask. + /// The scaled high-edge-variance threshold. + /// The sample bit depth. + private static void Filter8(Span samples, bool filterEnabled, bool flat, int highEdgeVarianceThreshold, int bitDepth) + { + if (filterEnabled && flat) + { + int p3 = samples[3]; + int p2 = samples[4]; + int p1 = samples[5]; + int p0 = samples[6]; + int q0 = samples[7]; + int q1 = samples[8]; + int q2 = samples[9]; + int q3 = samples[10]; + + samples[4] = RoundPowerOfTwo((3 * p3) + (2 * p2) + p1 + p0 + q0, 3); + samples[5] = RoundPowerOfTwo((2 * p3) + p2 + (2 * p1) + p0 + q0 + q1, 3); + samples[6] = RoundPowerOfTwo(p3 + p2 + p1 + (2 * p0) + q0 + q1 + q2, 3); + samples[7] = RoundPowerOfTwo(p2 + p1 + p0 + (2 * q0) + q1 + q2 + q3, 3); + samples[8] = RoundPowerOfTwo(p1 + p0 + q0 + (2 * q1) + q2 + (2 * q3), 3); + samples[9] = RoundPowerOfTwo(p0 + q0 + q1 + (2 * q2) + (3 * q3), 3); + return; + } + + Filter4(samples, filterEnabled, highEdgeVarianceThreshold, bitDepth); + } + + /// + /// Applies the scalar fourteen-tap luma filter or its eight- and four-tap fallbacks. + /// + /// The scalar p6..p0,q0..q6 sample window. + /// Whether the edge satisfies the filter mask. + /// Whether the edge satisfies the inner-flatness mask. + /// Whether the edge satisfies the outer-flatness mask. + /// The scaled high-edge-variance threshold. + /// The sample bit depth. + private static void Filter14(Span samples, bool filterEnabled, bool flat, bool outerFlat, int highEdgeVarianceThreshold, int bitDepth) + { + if (filterEnabled && flat && outerFlat) + { + int p6 = samples[0]; + int p5 = samples[1]; + int p4 = samples[2]; + int p3 = samples[3]; + int p2 = samples[4]; + int p1 = samples[5]; + int p0 = samples[6]; + int q0 = samples[7]; + int q1 = samples[8]; + int q2 = samples[9]; + int q3 = samples[10]; + int q4 = samples[11]; + int q5 = samples[12]; + int q6 = samples[13]; + + // The repeated endpoints extend the normative 13-tap window without reading beyond p6 and q6. + samples[1] = RoundPowerOfTwo((7 * p6) + (2 * p5) + (2 * p4) + p3 + p2 + p1 + p0 + q0, 4); + samples[2] = RoundPowerOfTwo((5 * p6) + (2 * p5) + (2 * p4) + (2 * p3) + p2 + p1 + p0 + q0 + q1, 4); + samples[3] = RoundPowerOfTwo((4 * p6) + p5 + (2 * p4) + (2 * p3) + (2 * p2) + p1 + p0 + q0 + q1 + q2, 4); + samples[4] = RoundPowerOfTwo((3 * p6) + p5 + p4 + (2 * p3) + (2 * p2) + (2 * p1) + p0 + q0 + q1 + q2 + q3, 4); + samples[5] = RoundPowerOfTwo((2 * p6) + p5 + p4 + p3 + (2 * p2) + (2 * p1) + (2 * p0) + q0 + q1 + q2 + q3 + q4, 4); + samples[6] = RoundPowerOfTwo(p6 + p5 + p4 + p3 + p2 + (2 * p1) + (2 * p0) + (2 * q0) + q1 + q2 + q3 + q4 + q5, 4); + samples[7] = RoundPowerOfTwo(p5 + p4 + p3 + p2 + p1 + (2 * p0) + (2 * q0) + (2 * q1) + q2 + q3 + q4 + q5 + q6, 4); + samples[8] = RoundPowerOfTwo(p4 + p3 + p2 + p1 + p0 + (2 * q0) + (2 * q1) + (2 * q2) + q3 + q4 + q5 + (2 * q6), 4); + samples[9] = RoundPowerOfTwo(p3 + p2 + p1 + p0 + q0 + (2 * q1) + (2 * q2) + (2 * q3) + q4 + q5 + (3 * q6), 4); + samples[10] = RoundPowerOfTwo(p2 + p1 + p0 + q0 + q1 + (2 * q2) + (2 * q3) + (2 * q4) + q5 + (4 * q6), 4); + samples[11] = RoundPowerOfTwo(p1 + p0 + q0 + q1 + q2 + (2 * q3) + (2 * q4) + (2 * q5) + (5 * q6), 4); + samples[12] = RoundPowerOfTwo(p0 + q0 + q1 + q2 + q3 + (2 * q4) + (2 * q5) + (7 * q6), 4); + return; + } + + Filter8(samples, filterEnabled, flat, highEdgeVarianceThreshold, bitDepth); + } + + /// + /// Rounds a packed integer while dividing by a power of two. + /// + /// The packed integers to round. + /// The base-two divisor exponent. + /// The rounded packed quotients. + private static Vector128 RoundPowerOfTwo(Vector128 value, int bitCount) + => (value + Vector128.Create(1 << (bitCount - 1))) >> bitCount; + + /// + /// Rounds a scalar integer while dividing by a power of two. + /// + /// The integer to round. + /// The base-two divisor exponent. + /// The rounded quotient. + private static int RoundPowerOfTwo(int value, int bitCount) + => (value + (1 << (bitCount - 1))) >> bitCount; +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/LoopFilter/Av1LoopFilterContext.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/LoopFilter/Av1LoopFilterContext.cs new file mode 100644 index 0000000000..0f5afc2b52 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/LoopFilter/Av1LoopFilterContext.cs @@ -0,0 +1,167 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; +using SixLabors.ImageSharp.Memory; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.LoopFilter; + +/// +/// Stores the transform-size map consumed by the AV1 deblocking loop filter. +/// +internal sealed class Av1LoopFilterContext : IDisposable +{ + /// + /// Stores luma transform sizes at plane-relative 4x4 granularity. + /// + private readonly MemoryGroup transformSizesY; + + /// + /// The active luma transform-map dimensions in plane-relative 4x4 units. + /// + private readonly Size transformSizesYSize; + + /// + /// Stores shared-chroma transform sizes at plane-relative 4x4 granularity. + /// + private readonly MemoryGroup? transformSizesUv; + + /// + /// The active shared-chroma transform-map dimensions in plane-relative 4x4 units. + /// + private readonly Size transformSizesUvSize; + + /// + /// Initializes a new instance of the class. + /// + /// The allocator that owns the frame-sized transform maps. + /// The sequence header defining superblock and chroma geometry. + /// The frame header defining active coded dimensions. + public Av1LoopFilterContext( + MemoryAllocator memoryAllocator, + ObuSequenceHeader sequenceHeader, + ObuFrameHeader frameHeader) + { + int modeInfoWidth = frameHeader.ModeInfoColumnCount; + int modeInfoHeight = frameHeader.ModeInfoRowCount; + MemoryGroup? transformSizesY = null; + MemoryGroup? transformSizesUv = null; + this.transformSizesUvSize = default; + + try + { + long lumaLength = (long)modeInfoWidth * modeInfoHeight; + transformSizesY = memoryAllocator.AllocateGroup( + lumaLength, + 1, + AllocationOptions.Clean); + + if (!sequenceHeader.ColorConfig.IsMonochrome) + { + int subX = sequenceHeader.ColorConfig.SubSamplingX ? 1 : 0; + int subY = sequenceHeader.ColorConfig.SubSamplingY ? 1 : 0; + int chromaWidth = Av1Math.DivideLog2Ceiling(modeInfoWidth, subX); + int chromaHeight = Av1Math.DivideLog2Ceiling(modeInfoHeight, subY); + long chromaLength = (long)chromaWidth * chromaHeight; + transformSizesUv = memoryAllocator.AllocateGroup( + chromaLength, + 1, + AllocationOptions.Clean); + this.transformSizesUvSize = new Size(chromaWidth, chromaHeight); + } + + this.transformSizesY = transformSizesY; + this.transformSizesYSize = new Size(modeInfoWidth, modeInfoHeight); + this.transformSizesUv = transformSizesUv; + } + catch + { + transformSizesUv?.Dispose(); + transformSizesY?.Dispose(); + throw; + } + } + + /// + /// Stores a transform size across every 4x4 position covered by one transform block. + /// + /// The luma or chroma plane. + /// The transform origin in plane-relative 4x4 units. + /// The transform size. + public void SetTransformSize(Av1Plane plane, Point position, Av1TransformSize transformSize) + { + int planeType = Math.Min((int)plane, (int)Av1PlaneType.Uv); + MemoryGroup transformSizeMap; + Size transformSizeMapSize; + if (planeType == (int)Av1PlaneType.Y) + { + transformSizeMap = this.transformSizesY; + transformSizeMapSize = this.transformSizesYSize; + } + else + { + transformSizeMap = this.transformSizesUv + ?? throw new InvalidOperationException("A monochrome AV1 frame has no chroma transform-size map."); + + transformSizeMapSize = this.transformSizesUvSize; + } + + int width = Math.Min(transformSize.Get4x4WideCount(), transformSizeMapSize.Width - position.X); + int height = Math.Min(transformSize.Get4x4HighCount(), transformSizeMapSize.Height - position.Y); + + // libaom clips transform coverage to the active plane mi dimensions at frame edges. Each logical row may cross + // allocator segments, so fill only the current segment before continuing at the same logical map offset. + for (int y = 0; y < height; y++) + { + long offset = ((long)(position.Y + y) * transformSizeMapSize.Width) + position.X; + int remaining = width; + while (remaining > 0) + { + Span destination = transformSizeMap.GetRemainingSliceOfBuffer(offset); + int count = Math.Min(remaining, destination.Length); + destination[..count].Fill(transformSize); + offset += count; + remaining -= count; + } + } + } + + /// + /// Gets the transform size covering a plane-relative 4x4 position. + /// + /// The luma or chroma plane. + /// The position in plane-relative 4x4 units. + /// The transform size covering the position. + public Av1TransformSize GetTransformSize(Av1Plane plane, Point position) + { + int planeType = Math.Min((int)plane, (int)Av1PlaneType.Uv); + MemoryGroup transformSizeMap; + int width; + if (planeType == (int)Av1PlaneType.Y) + { + transformSizeMap = this.transformSizesY; + width = this.transformSizesYSize.Width; + } + else + { + transformSizeMap = this.transformSizesUv + ?? throw new InvalidOperationException("A monochrome AV1 frame has no chroma transform-size map."); + + width = this.transformSizesUvSize.Width; + } + + long offset = ((long)position.Y * width) + position.X; + return transformSizeMap.GetRemainingSliceOfBuffer(offset)[0]; + } + + /// + /// Returns the allocator-owned transform-size maps. + /// + public void Dispose() + { + this.transformSizesUv?.Dispose(); + this.transformSizesY.Dispose(); + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/LoopFilter/Av1LoopFilterDecoder.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/LoopFilter/Av1LoopFilterDecoder.cs new file mode 100644 index 0000000000..47e7208e5d --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/LoopFilter/Av1LoopFilterDecoder.cs @@ -0,0 +1,394 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.InteropServices; +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.LoopFilter; + +/// +/// Applies the AV1 in-loop deblocking stage to a reconstructed still-image frame. +/// +internal sealed class Av1LoopFilterDecoder +{ + /// + /// The sequence-level superblock and color configuration. + /// + private readonly ObuSequenceHeader sequenceHeader; + + /// + /// The frame dimensions, segmentation state, and loop-filter parameters. + /// + private readonly ObuFrameHeader frameHeader; + + /// + /// The decoded mode and superblock delta information. + /// + private readonly Av1FrameInfo frameInfo; + + /// + /// The reconstructed plane samples modified by deblocking. + /// + private readonly Av1FrameBuffer frameBuffer; + + /// + /// The per-plane transform-size map populated during reconstruction. + /// + private readonly Av1LoopFilterContext loopFilterContext; + + /// + /// Initializes a new instance of the class. + /// + /// The sequence header defining superblock size and color layout. + /// The frame header defining dimensions and filter parameters. + /// The decoded block-mode and superblock delta information. + /// The reconstructed frame samples to filter. + /// The transform-size map populated during reconstruction. + public Av1LoopFilterDecoder( + ObuSequenceHeader sequenceHeader, + ObuFrameHeader frameHeader, + Av1FrameInfo frameInfo, + Av1FrameBuffer frameBuffer, + Av1LoopFilterContext loopFilterContext) + { + this.sequenceHeader = sequenceHeader; + this.frameHeader = frameHeader; + this.frameInfo = frameInfo; + this.frameBuffer = frameBuffer; + this.loopFilterContext = loopFilterContext; + } + + /// + /// Filters every enabled plane in maximum-superblock row bands. + /// + public void DecodeFrame() + { + ObuLoopFilterParameters filterParameters = this.frameHeader.LoopFilterParameters; + if (filterParameters.FilterLevel[0] == 0 && filterParameters.FilterLevel[1] == 0) + { + return; + } + + ObuColorConfig colorConfig = this.sequenceHeader.ColorConfig; + int modeInfoRowsPerBand = 1 << (Av1Constants.MaxSuperBlockSizeLog2 - Av1Constants.ModeInfoSizeLog2); + + // the reference decoder's loop_filter_rows processes one MAX_MIB_SIZE band at a time so that a completed band can be + // presented before the remainder of the frame. The ordering is observable because the vertical and + // horizontal passes modify intersecting sample neighborhoods in place. + for (int rowStart = 0; rowStart < this.frameHeader.ModeInfoRowCount; rowStart += modeInfoRowsPerBand) + { + int rowEnd = Math.Min(rowStart + modeInfoRowsPerBand, this.frameHeader.ModeInfoRowCount); + + for (int planeIndex = 0; planeIndex < colorConfig.PlaneCount; planeIndex++) + { + Av1Plane plane = (Av1Plane)planeIndex; + int planeFilterLevel = plane switch + { + Av1Plane.U => filterParameters.FilterLevelU, + Av1Plane.V => filterParameters.FilterLevelV, + _ => Math.Max(filterParameters.FilterLevel[0], filterParameters.FilterLevel[1]) + }; + + if (planeFilterLevel == 0) + { + continue; + } + + int subX = plane != Av1Plane.Y && colorConfig.SubSamplingX ? 1 : 0; + int subY = plane != Av1Plane.Y && colorConfig.SubSamplingY ? 1 : 0; + Span lowBitDepthSamples = default; + Span highBitDepthSamples = default; + int stride; + + if (this.frameBuffer.BytesPerSample == 2) + { + Span signedSamples = this.frameBuffer.DeriveBlockPointer16(plane, Point.Empty, subX, subY, out stride); + highBitDepthSamples = MemoryMarshal.Cast(signedSamples); + } + else + { + lowBitDepthSamples = this.frameBuffer.DeriveBlockPointer(plane, Point.Empty, subX, subY, out stride); + } + + this.FilterPlane(plane, rowStart, rowEnd, subX, subY, stride, lowBitDepthSamples, highBitDepthSamples); + } + } + } + + /// + /// Filters one plane band in the AV1 vertical-then-horizontal boundary order. + /// + /// The color plane to filter. + /// The inclusive band origin in luma 4x4 units. + /// The exclusive band limit in luma 4x4 units. + /// The horizontal chroma subsampling shift. + /// The vertical chroma subsampling shift. + /// The plane stride in logical samples. + /// The low-bit-depth plane storage, when active. + /// The high-bit-depth plane storage, when active. + private void FilterPlane( + Av1Plane plane, + int rowStart, + int rowEnd, + int subX, + int subY, + int stride, + Span lowBitDepthSamples, + Span highBitDepthSamples) + { + int rowStep = 1 << subY; + int columnStep = 1 << subX; + + // The vertical pass advances across each row because successive vertical edges do not share modified samples. + // Chroma rows retain luma-grid coordinates and therefore advance by two mode-info units when subsampled. + for (int row = rowStart; row < rowEnd; row += rowStep) + { + for (int column = 0; column < this.frameHeader.ModeInfoColumnCount; column += columnStep) + { + this.FilterEdge( + plane, + 0, + row, + column, + subX, + subY, + stride, + lowBitDepthSamples, + highBitDepthSamples); + } + } + + // Horizontal edges are visited down each column. This ordering is observable because adjacent horizontal + // filters can modify samples that a later edge reads, so it must match the reference decoder's av1_filter_block_plane_horz. + for (int column = 0; column < this.frameHeader.ModeInfoColumnCount; column += columnStep) + { + for (int row = rowStart; row < rowEnd; row += rowStep) + { + this.FilterEdge( + plane, + 1, + row, + column, + subX, + subY, + stride, + lowBitDepthSamples, + highBitDepthSamples); + } + } + } + + /// + /// Derives and applies the filter for one 4x4 luma-grid boundary. + /// + /// The color plane to filter. + /// Zero for a vertical boundary; one for a horizontal boundary. + /// The boundary row in luma 4x4 units. + /// The boundary column in luma 4x4 units. + /// The horizontal chroma subsampling shift. + /// The vertical chroma subsampling shift. + /// The plane stride in logical samples. + /// The low-bit-depth plane storage, when active. + /// The high-bit-depth plane storage, when active. + private void FilterEdge( + Av1Plane plane, + int pass, + int row, + int column, + int subX, + int subY, + int stride, + Span lowBitDepthSamples, + Span highBitDepthSamples) + { + int x = column << Av1Constants.ModeInfoSizeLog2; + int y = row << Av1Constants.ModeInfoSizeLog2; + bool verticalBoundary = pass == 0; + if (x >= this.frameHeader.FrameSize.FrameWidth || + y >= this.frameHeader.FrameSize.FrameHeight || + (verticalBoundary ? x == 0 : y == 0)) + { + return; + } + + int adjustedRow = row | subY; + int adjustedColumn = column | subX; + int previousRow = adjustedRow - (verticalBoundary ? 0 : 1 << subY); + int previousColumn = adjustedColumn - (verticalBoundary ? 1 << subX : 0); + Point modeInfoPosition = new(adjustedColumn, adjustedRow); + Point previousModeInfoPosition = new(previousColumn, previousRow); + Point planeTransformPosition = new(adjustedColumn >> subX, adjustedRow >> subY); + Point previousPlaneTransformPosition = new(previousColumn >> subX, previousRow >> subY); + Av1BlockModeInfo modeInfo = this.frameInfo.GetModeInfoAt(modeInfoPosition); + Av1BlockModeInfo previousModeInfo = this.frameInfo.GetModeInfoAt(previousModeInfoPosition); + Av1TransformSize transformSize = this.loopFilterContext.GetTransformSize(plane, planeTransformPosition); + Av1TransformSize previousTransformSize = this.loopFilterContext.GetTransformSize(plane, previousPlaneTransformPosition); + int planeX = x >> subX; + int planeY = y >> subY; + bool isTransformEdge = verticalBoundary + ? planeX % transformSize.GetWidth() == 0 + : planeY % transformSize.GetHeight() == 0; + + // A skipped intra block still has reconstructed prediction samples and is not a skipped inter transform. + // The skip predicate therefore applies only when the corresponding primary reference is inter. + bool currentSkippedTransform = modeInfo.Skip && modeInfo.ReferenceFrames[0] > Av1ReferenceFrameType.Intra; + bool previousSkippedTransform = previousModeInfo.Skip && previousModeInfo.ReferenceFrames[0] > Av1ReferenceFrameType.Intra; + + // Every covered 4x4 position carries the owning block's storage index. Comparing those indices is the value-type + // equivalent of the reference decoder's current-versus-previous MB_MODE_INFO pointer comparison at a prediction-unit boundary. + bool isBlockEdge = modeInfo.ModeInfoIndex != previousModeInfo.ModeInfoIndex; + bool applyFilter = isTransformEdge && (isBlockEdge || !currentSkippedTransform || !previousSkippedTransform); + if (!applyFilter) + { + return; + } + + int currentLevel = this.GetFilterLevel(modeInfo, modeInfoPosition, plane, pass); + int filterLevel = currentLevel != 0 + ? currentLevel + : this.GetFilterLevel(previousModeInfo, previousModeInfoPosition, plane, pass); + + if (filterLevel == 0) + { + return; + } + + int baseFilterSize = verticalBoundary + ? Math.Min(transformSize.GetWidth(), previousTransformSize.GetWidth()) + : Math.Min(transformSize.GetHeight(), previousTransformSize.GetHeight()); + + int maximumFilterSize = plane == Av1Plane.Y ? 16 : 8; + int filterSize = Math.Min(maximumFilterSize, baseFilterSize); + int kernelLength = plane == Av1Plane.Y + ? filterSize switch + { + 4 => 4, + 8 => 8, + _ => 14 + } + : filterSize == 4 ? 4 : 6; + + int sharpness = this.frameHeader.LoopFilterParameters.SharpnessLevel; + int shift = sharpness > 4 ? 2 : sharpness > 0 ? 1 : 0; + int limit = sharpness > 0 + ? Av1Math.Clip3(1, 9 - sharpness, filterLevel >> shift) + : Math.Max(1, filterLevel >> shift); + + int boundaryLimit = (2 * (filterLevel + 2)) + limit; + int highEdgeVarianceThreshold = filterLevel >> 4; + int q0Offset = stride + (planeY * stride) + planeX; + + if (this.frameBuffer.BytesPerSample == 2) + { + if (verticalBoundary) + { + Av1DeblockingFilter.FilterVertical( + highBitDepthSamples, + q0Offset, + stride, + kernelLength, + limit, + boundaryLimit, + highEdgeVarianceThreshold, + this.frameBuffer.BitDepth.GetBitCount()); + } + else + { + Av1DeblockingFilter.FilterHorizontal( + highBitDepthSamples, + q0Offset, + stride, + kernelLength, + limit, + boundaryLimit, + highEdgeVarianceThreshold, + this.frameBuffer.BitDepth.GetBitCount()); + } + } + else if (verticalBoundary) + { + Av1DeblockingFilter.FilterVertical( + lowBitDepthSamples, + q0Offset, + stride, + kernelLength, + limit, + boundaryLimit, + highEdgeVarianceThreshold); + } + else + { + Av1DeblockingFilter.FilterHorizontal( + lowBitDepthSamples, + q0Offset, + stride, + kernelLength, + limit, + boundaryLimit, + highEdgeVarianceThreshold); + } + } + + /// + /// Derives the adaptive filter level for one block, plane, and boundary direction. + /// + /// The decoded mode and segment information. + /// The frame-relative position in luma 4x4 units. + /// The color plane. + /// Zero for a vertical boundary; one for a horizontal boundary. + /// The filter level in the AV1 zero-to-63 domain. + private int GetFilterLevel(Av1BlockModeInfo modeInfo, Point modeInfoPosition, Av1Plane plane, int pass) + { + int filterIndex = plane == Av1Plane.Y ? pass : (int)plane + 1; + ObuLoopFilterParameters parameters = this.frameHeader.LoopFilterParameters; + int baseLevel = filterIndex switch + { + 0 => parameters.FilterLevel[0], + 1 => parameters.FilterLevel[1], + 2 => parameters.FilterLevelU, + _ => parameters.FilterLevelV + }; + + int superblockShift = this.sequenceHeader.SuperblockSizeLog2 - Av1Constants.ModeInfoSizeLog2; + Point superblockPosition = new(modeInfoPosition.X >> superblockShift, modeInfoPosition.Y >> superblockShift); + Span deltaLoopFilter = this.frameInfo.GetSuperblock(superblockPosition).SuperblockDeltaLoopFilter; + int delta = this.frameHeader.DeltaLoopFilterParameters.IsMulti ? deltaLoopFilter[filterIndex] : deltaLoopFilter[0]; + int level = Av1Math.Clip3(0, Av1Constants.MaxLoopFilter, baseLevel + delta); + ObuSegmentationLevelFeature feature = (ObuSegmentationLevelFeature)((int)ObuSegmentationLevelFeature.AlternativeLoopFilterYVertical + filterIndex); + ObuSegmentationParameters segmentation = this.frameHeader.SegmentationParameters; + + if (segmentation.IsFeatureActive(modeInfo.SegmentId, feature)) + { + level = Av1Math.Clip3( + 0, + Av1Constants.MaxLoopFilter, + level + segmentation.GetFeatureData(modeInfo.SegmentId, (int)feature)); + } + + if (parameters.ReferenceDeltaModeEnabled) + { + int referenceScale = 1 << (level >> 5); + Av1ReferenceFrameType referenceFrame = modeInfo.ReferenceFrames[0]; + level += parameters.ReferenceDeltas[(int)referenceFrame] * referenceScale; + + if (referenceFrame > Av1ReferenceFrameType.Intra) + { + // AV1's second mode-delta class contains every inter mode except the two global-motion modes. + // Keeping this classification next to the level arithmetic mirrors the reference decoder's mode_lf_lut lookup. + int modeDeltaIndex = modeInfo.YMode is Av1PredictionMode.GlobalMotionVector or + Av1PredictionMode.GlobalGlobalMotionVector ? 0 : 1; + + level += parameters.ModeDeltas[modeDeltaIndex] * referenceScale; + } + + // Reference and mode adjustments use the same scale and may cancel beyond either limit. + // Clipping the intermediate reference sum would discard part of that cancellation. + level = Av1Math.Clip3(0, Av1Constants.MaxLoopFilter, level); + } + + return level; + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/LoopRestoration/Av1LoopRestorationBoundary.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/LoopRestoration/Av1LoopRestorationBoundary.cs new file mode 100644 index 0000000000..37843510e8 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/LoopRestoration/Av1LoopRestorationBoundary.cs @@ -0,0 +1,445 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Buffers; +using System.Runtime.InteropServices; +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.SuperResolution; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.LoopRestoration; + +/// +/// Preserves the deblocked and frame-edge rows required by AV1 striped loop restoration. +/// +internal sealed class Av1LoopRestorationBoundary : IDisposable +{ + /// + /// The height of a complete restoration processing stripe in luma samples. + /// + public const int ProcessingStripeSize = 64; + + /// + /// The upward offset of the restoration stripe grid from the restoration-unit grid. + /// + public const int ProcessingStripeOffset = 8; + + /// + /// The number of distinct deblocked rows preserved above and below a stripe. + /// + private const int ContextRowCount = 2; + + /// + /// The sequence-level bit-depth and plane-layout configuration. + /// + private readonly ObuSequenceHeader sequenceHeader; + + /// + /// The coded, reconstructed, and upscaled frame dimensions. + /// + private readonly ObuFrameHeader frameHeader; + + /// + /// The reconstructed sample planes read at each pipeline boundary. + /// + private readonly Av1FrameBuffer frameBuffer; + + /// + /// The two preserved rows above every processing stripe, stored by plane. + /// + private InlineArray4?> rowsAbove; + + /// + /// The two preserved rows below every processing stripe, stored by plane. + /// + private InlineArray4?> rowsBelow; + + /// + /// The upscaled sample width stored for each plane boundary row. + /// + private InlineArray4 planeWidths; + + /// + /// The number of processing stripes represented for each plane. + /// + private InlineArray4 stripeCounts; + + /// + /// Initializes a new instance of the class. + /// + /// The sequence header defining bit depth and chroma subsampling. + /// The frame header defining coded and upscaled dimensions. + /// The reconstructed frame sampled before and after CDEF. + public Av1LoopRestorationBoundary( + ObuSequenceHeader sequenceHeader, + ObuFrameHeader frameHeader, + Av1FrameBuffer frameBuffer) + { + this.sequenceHeader = sequenceHeader; + this.frameHeader = frameHeader; + this.frameBuffer = frameBuffer; + + try + { + ObuColorConfig colorConfig = sequenceHeader.ColorConfig; + for (int planeIndex = 0; planeIndex < colorConfig.PlaneCount; planeIndex++) + { + if (frameHeader.LoopRestorationParameters.Items[planeIndex].Type == ObuRestorationType.None) + { + continue; + } + + Av1Plane plane = (Av1Plane)planeIndex; + int subsamplingX = plane != Av1Plane.Y && colorConfig.SubSamplingX ? 1 : 0; + int subsamplingY = plane != Av1Plane.Y && colorConfig.SubSamplingY ? 1 : 0; + int planeWidth = Av1Math.DivideLog2Ceiling( + frameHeader.FrameSize.SuperResolutionUpscaledWidth, + subsamplingX); + + int planeHeight = Av1Math.DivideLog2Ceiling(frameHeader.FrameSize.FrameHeight, subsamplingY); + int stripeHeight = ProcessingStripeSize >> subsamplingY; + int stripeOffset = ProcessingStripeOffset >> subsamplingY; + + // The stripe grid begins eight luma rows above the restoration-unit grid. Including + // that offset in the ceiling retains the short final stripe when one is present. + int stripeCount = (planeHeight + stripeOffset + stripeHeight - 1) / stripeHeight; + int storageLength = stripeCount * ContextRowCount * planeWidth; + this.planeWidths[planeIndex] = planeWidth; + this.stripeCounts[planeIndex] = stripeCount; + this.rowsAbove[planeIndex] = frameBuffer.MemoryAllocator.Allocate(storageLength); + this.rowsBelow[planeIndex] = frameBuffer.MemoryAllocator.Allocate(storageLength); + } + } + catch + { + // The object is not available to its caller when construction fails, so release any + // plane owners acquired before the allocator reported the failure. + this.Dispose(); + throw; + } + } + + /// + /// Releases the preserved plane-boundary storage. + /// + public void Dispose() + { + for (int plane = 0; plane < Av1Constants.MaxPlanes; plane++) + { + this.rowsAbove[plane]?.Dispose(); + this.rowsAbove[plane] = null; + this.rowsBelow[plane]?.Dispose(); + this.rowsBelow[plane] = null; + } + } + + /// + /// Preserves deblocked rows at every internal restoration-stripe boundary. + /// + public void SaveDeblockedRows() + { + ObuColorConfig colorConfig = this.sequenceHeader.ColorConfig; + ObuFrameSize frameSize = this.frameHeader.FrameSize; + bool usesSuperResolution = frameSize.FrameWidth != frameSize.SuperResolutionUpscaledWidth; + for (int planeIndex = 0; planeIndex < colorConfig.PlaneCount; planeIndex++) + { + if (this.stripeCounts[planeIndex] == 0) + { + continue; + } + + Av1Plane plane = (Av1Plane)planeIndex; + int subsamplingX = plane != Av1Plane.Y && colorConfig.SubSamplingX ? 1 : 0; + int subsamplingY = plane != Av1Plane.Y && colorConfig.SubSamplingY ? 1 : 0; + int codedWidth = Av1Math.DivideLog2Ceiling(frameSize.FrameWidth, subsamplingX); + int upscaledWidth = this.planeWidths[planeIndex]; + int reconstructedWidth = this.frameHeader.ModeInfoColumnCount + << (Av1Constants.ModeInfoSizeLog2 - subsamplingX); + + // Super-resolution phase uses the coded width, while its filter taps can consume the + // complete mode-info-aligned reconstruction at the right edge. + int planeHeight = Av1Math.DivideLog2Ceiling(frameSize.FrameHeight, subsamplingY); + int stripeHeight = ProcessingStripeSize >> subsamplingY; + int stripeOffset = ProcessingStripeOffset >> subsamplingY; + int sourceBorder = usesSuperResolution ? Av1SuperResolutionFilter.SourceBorder : 0; + + Span lowBitDepthPlane = default; + Span highBitDepthPlane = default; + int sourceStride; + if (this.frameBuffer.BytesPerSample == 2) + { + Span signedPlane = this.frameBuffer.DeriveBlockPointer16( + plane, + new Point(-sourceBorder, 0), + subsamplingX, + subsamplingY, + out sourceStride); + + highBitDepthPlane = MemoryMarshal.Cast(signedPlane); + } + else + { + lowBitDepthPlane = this.frameBuffer.DeriveBlockPointer( + plane, + new Point(-sourceBorder, 0), + subsamplingX, + subsamplingY, + out sourceStride); + } + + int step = usesSuperResolution + ? Av1SuperResolutionFilter.GetConvolveStep(codedWidth, upscaledWidth) + : 0; + + int initialSubpixel = usesSuperResolution + ? Av1SuperResolutionFilter.GetInitialSubpixel(codedWidth, upscaledWidth, step) + : 0; + + for (int stripe = 0; stripe < this.stripeCounts[planeIndex]; stripe++) + { + int stripeStart = Math.Max(0, (stripe * stripeHeight) - stripeOffset); + int stripeEnd = Math.Min(((stripe + 1) * stripeHeight) - stripeOffset, planeHeight); + if (stripe > 0) + { + // Internal top context is the two deblocked rows immediately preceding the + // stripe; restoration later expands the first row to fill its three-row border. + this.SaveDeblockedRow( + lowBitDepthPlane, + highBitDepthPlane, + sourceStride, + stripeStart - ContextRowCount, + reconstructedWidth, + step, + initialSubpixel, + sourceBorder, + this.GetBoundaryRow(this.rowsAbove, planeIndex, stripe, 0)); + + this.SaveDeblockedRow( + lowBitDepthPlane, + highBitDepthPlane, + sourceStride, + stripeStart - 1, + reconstructedWidth, + step, + initialSubpixel, + sourceBorder, + this.GetBoundaryRow(this.rowsAbove, planeIndex, stripe, 1)); + } + + if (stripeEnd < planeHeight) + { + // Internal bottom context begins at the exclusive stripe end. A one-row tail + // duplicates its final sample row, matching AV1 crop-edge clamping. + this.SaveDeblockedRow( + lowBitDepthPlane, + highBitDepthPlane, + sourceStride, + stripeEnd, + reconstructedWidth, + step, + initialSubpixel, + sourceBorder, + this.GetBoundaryRow(this.rowsBelow, planeIndex, stripe, 0)); + + this.SaveDeblockedRow( + lowBitDepthPlane, + highBitDepthPlane, + sourceStride, + Math.Min(stripeEnd + 1, planeHeight - 1), + reconstructedWidth, + step, + initialSubpixel, + sourceBorder, + this.GetBoundaryRow(this.rowsBelow, planeIndex, stripe, 1)); + } + } + } + } + + /// + /// Preserves the post-CDEF and post-super-resolution rows at the top and bottom of each active plane. + /// + public void SaveFrameEdgeRows() + { + ObuColorConfig colorConfig = this.sequenceHeader.ColorConfig; + ObuFrameSize frameSize = this.frameHeader.FrameSize; + for (int planeIndex = 0; planeIndex < colorConfig.PlaneCount; planeIndex++) + { + int stripeCount = this.stripeCounts[planeIndex]; + if (stripeCount == 0) + { + continue; + } + + Av1Plane plane = (Av1Plane)planeIndex; + int subsamplingX = plane != Av1Plane.Y && colorConfig.SubSamplingX ? 1 : 0; + int subsamplingY = plane != Av1Plane.Y && colorConfig.SubSamplingY ? 1 : 0; + int planeHeight = Av1Math.DivideLog2Ceiling(frameSize.FrameHeight, subsamplingY); + Span lowBitDepthPlane = default; + Span highBitDepthPlane = default; + int sourceStride; + if (this.frameBuffer.BytesPerSample == 2) + { + Span signedPlane = this.frameBuffer.DeriveBlockPointer16( + plane, + Point.Empty, + subsamplingX, + subsamplingY, + out sourceStride); + + highBitDepthPlane = MemoryMarshal.Cast(signedPlane); + } + else + { + lowBitDepthPlane = this.frameBuffer.DeriveBlockPointer( + plane, + Point.Empty, + subsamplingX, + subsamplingY, + out sourceStride); + } + + Span topRow0 = this.GetBoundaryRow(this.rowsAbove, planeIndex, 0, 0); + Span topRow1 = this.GetBoundaryRow(this.rowsAbove, planeIndex, 0, 1); + CopyFrameRow(lowBitDepthPlane, highBitDepthPlane, sourceStride, 0, 0, topRow0); + + // Frame boundaries use post-CDEF/post-super-resolution samples and replicate the outer row. + topRow0.CopyTo(topRow1); + + int lastStripe = stripeCount - 1; + Span bottomRow0 = this.GetBoundaryRow(this.rowsBelow, planeIndex, lastStripe, 0); + Span bottomRow1 = this.GetBoundaryRow(this.rowsBelow, planeIndex, lastStripe, 1); + CopyFrameRow(lowBitDepthPlane, highBitDepthPlane, sourceStride, planeHeight - 1, 0, bottomRow0); + bottomRow0.CopyTo(bottomRow1); + } + } + + /// + /// Gets one preserved row above a restoration processing stripe. + /// + /// The zero-based color-plane index. + /// The frame-relative processing-stripe index. + /// The first or second preserved context row. + /// The preserved upscaled row. + public ReadOnlySpan GetRowAbove(int plane, int stripe, int contextRow) + => this.GetBoundaryRow(this.rowsAbove, plane, stripe, contextRow); + + /// + /// Gets one preserved row below a restoration processing stripe. + /// + /// The zero-based color-plane index. + /// The frame-relative processing-stripe index. + /// The first or second preserved context row. + /// The preserved upscaled row. + public ReadOnlySpan GetRowBelow(int plane, int stripe, int contextRow) + => this.GetBoundaryRow(this.rowsBelow, plane, stripe, contextRow); + + /// + /// Preserves one deblocked row, applying normative super-resolution when the frame is scaled. + /// + /// The byte-backed plane when the frame uses eight-bit samples. + /// The native 16-bit plane when the frame uses high-bit-depth samples. + /// The number of samples between reconstructed rows. + /// The reconstructed row to preserve. + /// The mode-info-aligned width supplying super-resolution taps. + /// The fixed-point super-resolution source-position increment, or zero when unscaled. + /// The first super-resolution source position, or zero when unscaled. + /// The decoder-padding samples preceding the visible source row. + /// The preserved upscaled boundary row. + private void SaveDeblockedRow( + Span lowBitDepthPlane, + Span highBitDepthPlane, + int sourceStride, + int row, + int reconstructedWidth, + int step, + int initialSubpixel, + int sourceBorder, + Span destination) + { + if (sourceBorder == 0) + { + CopyFrameRow(lowBitDepthPlane, highBitDepthPlane, sourceStride, row, 0, destination); + return; + } + + // Boundary rows must follow the same continuous phase and aligned right-edge behavior as + // the full-frame super-resolution stage or restoration would see different stripe context. + int sourceOffset = sourceStride + (row * sourceStride); + if (!highBitDepthPlane.IsEmpty) + { + Span source = highBitDepthPlane.Slice(sourceOffset, reconstructedWidth + (sourceBorder * 2)); + Span reconstructedSamples = source.Slice(sourceBorder, reconstructedWidth); + + source[..sourceBorder].Fill(reconstructedSamples[0]); + source[(sourceBorder + reconstructedWidth)..].Fill(reconstructedSamples[^1]); + Av1SuperResolutionFilter.UpscaleRow( + source, + destination, + step, + initialSubpixel, + this.frameBuffer.BitDepth.GetBitCount()); + + return; + } + + Span lowBitDepthSource = lowBitDepthPlane.Slice(sourceOffset, reconstructedWidth + (sourceBorder * 2)); + Span lowBitDepthReconstructedSamples = lowBitDepthSource.Slice(sourceBorder, reconstructedWidth); + + lowBitDepthSource[..sourceBorder].Fill(lowBitDepthReconstructedSamples[0]); + lowBitDepthSource[(sourceBorder + reconstructedWidth)..].Fill(lowBitDepthReconstructedSamples[^1]); + Av1SuperResolutionFilter.UpscaleRow(lowBitDepthSource, destination, step, initialSubpixel); + } + + /// + /// Copies one reconstructed sample row into 16-bit working storage. + /// + /// The byte-backed plane when the frame uses eight-bit samples. + /// The native 16-bit plane when the frame uses high-bit-depth samples. + /// The number of samples between reconstructed rows. + /// The zero-based visible row index. + /// The decoder-padding samples preceding the visible source row. + /// The destination row whose length determines the copied width. + private static void CopyFrameRow( + ReadOnlySpan lowBitDepthPlane, + ReadOnlySpan highBitDepthPlane, + int sourceStride, + int row, + int sourceBorder, + Span destination) + { + int sourceOffset = sourceStride + (row * sourceStride) + sourceBorder; + if (!highBitDepthPlane.IsEmpty) + { + highBitDepthPlane.Slice(sourceOffset, destination.Length).CopyTo(destination); + return; + } + + ReadOnlySpan source = lowBitDepthPlane.Slice(sourceOffset, destination.Length); + for (int column = 0; column < destination.Length; column++) + { + destination[column] = source[column]; + } + } + + /// + /// Gets writable storage for one plane-relative boundary row. + /// + /// The above- or below-stripe storage for every plane. + /// The zero-based color-plane index. + /// The frame-relative processing-stripe index. + /// The first or second preserved context row. + /// The selected boundary-row span. + private Span GetBoundaryRow( + ReadOnlySpan?> storage, + int plane, + int stripe, + int contextRow) + { + int width = this.planeWidths[plane]; + int offset = ((stripe * ContextRowCount) + contextRow) * width; + IMemoryOwner owner = storage[plane] + ?? throw new InvalidOperationException("The selected AV1 plane has no loop-restoration boundary storage."); + + return owner.Memory.Span.Slice(offset, width); + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/LoopRestoration/Av1LoopRestorationDecoder.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/LoopRestoration/Av1LoopRestorationDecoder.cs new file mode 100644 index 0000000000..06cb665bad --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/LoopRestoration/Av1LoopRestorationDecoder.cs @@ -0,0 +1,635 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Buffers; +using System.Runtime.InteropServices; +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; +using SixLabors.ImageSharp.Memory; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.LoopRestoration; + +/// +/// Applies decoded AV1 loop-restoration units to a reconstructed still-image frame. +/// +internal sealed class Av1LoopRestorationDecoder +{ + /// + /// The number of source rows and columns required around each filtered processing stripe. + /// + private const int FilterBorder = 3; + + /// + /// The additional zero-coefficient tap read by the padded Wiener convolution kernel. + /// + private const int WienerPadding = 1; + + /// + /// The sequence-level bit-depth and plane-layout configuration. + /// + private readonly ObuSequenceHeader sequenceHeader; + + /// + /// The frame-level restoration-unit and dimension configuration. + /// + private readonly ObuFrameHeader frameHeader; + + /// + /// The decoded restoration filter and coefficient selections. + /// + private readonly Av1FrameInfo frameInfo; + + /// + /// The reconstructed sample planes updated with restored output. + /// + private readonly Av1FrameBuffer frameBuffer; + + /// + /// The preserved deblocked rows used at restoration-stripe boundaries. + /// + private readonly Av1LoopRestorationBoundary boundary; + + /// + /// Initializes a new instance of the class. + /// + /// The sequence header defining bit depth and chroma subsampling. + /// The frame header defining restoration-unit sizes and frame dimensions. + /// The decoded restoration-unit selections and coefficients. + /// The CDEF-filtered and upscaled frame samples. + /// The deblocked context preserved around restoration stripes. + public Av1LoopRestorationDecoder( + ObuSequenceHeader sequenceHeader, + ObuFrameHeader frameHeader, + Av1FrameInfo frameInfo, + Av1FrameBuffer frameBuffer, + Av1LoopRestorationBoundary boundary) + { + this.sequenceHeader = sequenceHeader; + this.frameHeader = frameHeader; + this.frameInfo = frameInfo; + this.frameBuffer = frameBuffer; + this.boundary = boundary; + } + + /// + /// Restores every active color plane from an immutable post-super-resolution source snapshot. + /// + public void DecodeFrame() + { + ObuColorConfig colorConfig = this.sequenceHeader.ColorConfig; + for (int planeIndex = 0; planeIndex < colorConfig.PlaneCount; planeIndex++) + { + ObuLoopRestorationItem item = this.frameHeader.LoopRestorationParameters.Items[planeIndex]; + if (item.Type == ObuRestorationType.None) + { + continue; + } + + Av1Plane plane = (Av1Plane)planeIndex; + int subsamplingX = plane != Av1Plane.Y && colorConfig.SubSamplingX ? 1 : 0; + int subsamplingY = plane != Av1Plane.Y && colorConfig.SubSamplingY ? 1 : 0; + this.DecodePlane(plane, subsamplingX, subsamplingY, item.Size); + } + } + + /// + /// Restores one color plane in raster-ordered restoration units. + /// + /// The luma or chroma plane. + /// The horizontal chroma subsampling shift. + /// The vertical chroma subsampling shift. + /// The nominal restoration-unit width and height in plane samples. + private void DecodePlane(Av1Plane plane, int subsamplingX, int subsamplingY, int unitSize) + { + int planeIndex = (int)plane; + ObuFrameSize frameSize = this.frameHeader.FrameSize; + int planeWidth = Av1Math.DivideLog2Ceiling(frameSize.SuperResolutionUpscaledWidth, subsamplingX); + int planeHeight = Av1Math.DivideLog2Ceiling(frameSize.FrameHeight, subsamplingY); + int planeLength = planeWidth * planeHeight; + MemoryAllocator allocator = this.frameBuffer.MemoryAllocator; + ReadOnlySpan lowBitDepthSource = default; + ReadOnlySpan highBitDepthSource = default; + int sourceStride; + if (this.frameBuffer.BytesPerSample == 2) + { + Span signedPlane = this.frameBuffer.DeriveBlockPointer16( + plane, + Point.Empty, + subsamplingX, + subsamplingY, + out sourceStride); + + highBitDepthSource = MemoryMarshal.Cast(signedPlane); + } + else + { + lowBitDepthSource = this.frameBuffer.DeriveBlockPointer( + plane, + Point.Empty, + subsamplingX, + subsamplingY, + out sourceStride); + } + + int destinationStorageLength = planeLength * this.frameBuffer.BytesPerSample; + using IMemoryOwner destinationOwner = allocator.Allocate(destinationStorageLength); + Span destinationStorage = destinationOwner.Memory.Span[..destinationStorageLength]; + Span lowBitDepthDestination = this.frameBuffer.BytesPerSample == 1 + ? destinationStorage + : default; + Span highBitDepthDestination = this.frameBuffer.BytesPerSample == 2 + ? MemoryMarshal.Cast(destinationStorage) + : default; + + // Restoration units overlap in their filter context but not in their output. Reading the reconstructed + // plane directly while writing a native-width destination matches libaom's frame/rst_frame ownership. + + // The last restoration unit can absorb a remainder, but both filters process at most 64 luma samples + // per axis before reusing scratch. Unit dimensions therefore do not determine the filter workspace. + int extendedUnitSize = (unitSize * 3) / 2; + int processingUnitWidth = Av1LoopRestorationBoundary.ProcessingStripeSize >> subsamplingX; + int maximumBlockWidth = Math.Min(processingUnitWidth, planeWidth); + int maximumStripeHeight = Av1LoopRestorationBoundary.ProcessingStripeSize >> subsamplingY; + int borderedStride = maximumBlockWidth + (FilterBorder * 2) + WienerPadding; + int borderedLength = borderedStride * (maximumStripeHeight + (FilterBorder * 2) + WienerPadding); + int wienerScratchLength = Av1WienerFilter.GetScratchLength(maximumBlockWidth, maximumStripeHeight); + int filterOutputLength = this.frameBuffer.BytesPerSample == 1 + ? maximumBlockWidth * maximumStripeHeight + : 0; + + int ushortScratchLength = borderedLength + wienerScratchLength + filterOutputLength; + using IMemoryOwner ushortScratchOwner = allocator.Allocate(ushortScratchLength); + Span ushortScratch = ushortScratchOwner.Memory.Span[..ushortScratchLength]; + Span borderedSource = ushortScratch[..borderedLength]; + Span wienerScratch = ushortScratch.Slice(borderedLength, wienerScratchLength); + Span filterOutput = ushortScratch[(borderedLength + wienerScratchLength)..]; + int selfGuidedScratchLength = Av1SelfGuidedFilter.GetScratchLength(maximumBlockWidth, maximumStripeHeight); + using IMemoryOwner selfGuidedScratchOwner = allocator.Allocate(selfGuidedScratchLength); + Span selfGuidedScratch = selfGuidedScratchOwner.Memory.Span[..selfGuidedScratchLength]; + + int unitRowCount = this.frameInfo.GetLoopRestorationUnitRowCount(planeIndex); + int unitColumnCount = this.frameInfo.GetLoopRestorationUnitColumnCount(planeIndex); + int unitY = 0; + for (int unitRow = 0; unitRow < unitRowCount; unitRow++) + { + int remainingHeight = planeHeight - unitY; + int unadjustedUnitHeight = remainingHeight < extendedUnitSize ? remainingHeight : unitSize; + int verticalStart = unitY; + int verticalEnd = unitY + unadjustedUnitHeight; + int verticalOffset = Av1LoopRestorationBoundary.ProcessingStripeOffset >> subsamplingY; + + // Unit ownership is signaled on the unshifted grid, but filtering rows follow the + // processing-stripe grid positioned eight luma samples above it. + verticalStart = Math.Max(0, verticalStart - verticalOffset); + if (verticalEnd < planeHeight) + { + verticalEnd -= verticalOffset; + } + + int unitX = 0; + for (int unitColumn = 0; unitColumn < unitColumnCount; unitColumn++) + { + int remainingWidth = planeWidth - unitX; + int unitWidth = remainingWidth < extendedUnitSize ? remainingWidth : unitSize; + Av1LoopRestorationUnit unit = this.frameInfo.GetLoopRestorationUnit( + planeIndex, + unitRow, + unitColumn); + + this.FilterUnit( + planeIndex, + subsamplingX, + lowBitDepthSource, + highBitDepthSource, + sourceStride, + lowBitDepthDestination, + highBitDepthDestination, + planeWidth, + planeHeight, + unitX, + unitX + unitWidth, + verticalStart, + verticalEnd, + unit, + borderedSource, + wienerScratch, + filterOutput, + selfGuidedScratch); + + unitX += unitWidth; + } + + unitY += unadjustedUnitHeight; + } + + this.CopyWorkingBufferToPlane( + plane, + subsamplingX, + subsamplingY, + planeWidth, + planeHeight, + lowBitDepthDestination, + highBitDepthDestination); + } + + /// + /// Copies or filters one restoration unit without reading already restored destination samples. + /// + /// The zero-based color-plane index. + /// The horizontal chroma subsampling shift. + /// The immutable byte post-super-resolution plane, when present. + /// The immutable 16-bit post-super-resolution plane, when present. + /// The number of samples between source rows. + /// The byte restoration destination, when present. + /// The native 16-bit restoration destination, when present. + /// The visible plane width. + /// The visible plane height. + /// The unit's first plane column. + /// The exclusive unit column limit. + /// The unit's stripe-aligned first plane row. + /// The exclusive unit row limit. + /// The decoded unit filter and coefficients. + /// Reusable storage for one bordered processing stripe. + /// Reusable Wiener intermediate storage. + /// Reusable native-precision filter output for an eight-bit frame. + /// Reusable self-guided intermediate storage. + private void FilterUnit( + int plane, + int subsamplingX, + ReadOnlySpan lowBitDepthSource, + ReadOnlySpan highBitDepthSource, + int sourceStride, + Span lowBitDepthDestination, + Span highBitDepthDestination, + int planeWidth, + int planeHeight, + int horizontalStart, + int horizontalEnd, + int verticalStart, + int verticalEnd, + Av1LoopRestorationUnit unit, + Span borderedSource, + Span wienerScratch, + Span filterOutput, + Span selfGuidedScratch) + { + int unitWidth = horizontalEnd - horizontalStart; + if (unit.FilterType == Av1RestorationFilterType.None) + { + // Every output sample still belongs to exactly one unit, including units that select + // RESTORE_NONE, so copy the immutable source rectangle into the destination plane. + CopyRectangle( + lowBitDepthSource, + highBitDepthSource, + sourceStride, + lowBitDepthDestination, + highBitDepthDestination, + planeWidth, + horizontalStart, + unitWidth, + verticalStart, + verticalEnd); + + return; + } + + int subsamplingY = plane == (int)Av1Plane.Y || !this.sequenceHeader.ColorConfig.SubSamplingY ? 0 : 1; + int fullStripeHeight = Av1LoopRestorationBoundary.ProcessingStripeSize >> subsamplingY; + int stripeOffset = Av1LoopRestorationBoundary.ProcessingStripeOffset >> subsamplingY; + int unitHeight = verticalEnd - verticalStart; + for (int unitRow = 0; unitRow < unitHeight;) + { + int stripeStart = verticalStart + unitRow; + int frameStripe = (stripeStart + stripeOffset) / fullStripeHeight; + int nominalStripeHeight = fullStripeHeight - (frameStripe == 0 ? stripeOffset : 0); + int stripeHeight = Math.Min(nominalStripeHeight, verticalEnd - stripeStart); + + // The first frame stripe is shortened by the upward offset; subsequent stripes remain + // 64 luma samples high, with the current unit limiting only the final iteration. + int processingUnitWidth = Av1LoopRestorationBoundary.ProcessingStripeSize >> subsamplingX; + + // Both filters consume bounded processing units. Their context still comes from the full plane, + // so a chunk boundary never becomes a replicated edge or reads an already restored sample. + for (int unitColumn = 0; unitColumn < unitWidth; unitColumn += processingUnitWidth) + { + int blockWidth = Math.Min(processingUnitWidth, unitWidth - unitColumn); + int blockStart = horizontalStart + unitColumn; + int borderedStride = blockWidth + (FilterBorder * 2) + WienerPadding; + int sourceLength = borderedStride * (stripeHeight + (FilterBorder * 2) + WienerPadding); + Span filterSource = borderedSource[..sourceLength]; + this.PopulateBorderedSource( + plane, + frameStripe, + lowBitDepthSource, + highBitDepthSource, + sourceStride, + planeWidth, + planeHeight, + blockStart, + blockWidth, + stripeStart, + stripeHeight, + borderedStride, + filterSource); + + int destinationOffset = (stripeStart * planeWidth) + blockStart; + Span filterDestination = highBitDepthDestination.IsEmpty + ? filterOutput[..(blockWidth * stripeHeight)] + : highBitDepthDestination[destinationOffset..]; + + int filterDestinationStride = highBitDepthDestination.IsEmpty ? blockWidth : planeWidth; + + // Native Wiener kernels round the final chunk's write width up for SIMD. This kernel accepts + // the exact tail width, retaining all seven-tap context without writing beyond the plane. + if (unit.FilterType == Av1RestorationFilterType.Wiener) + { + int scratchLength = Av1WienerFilter.GetScratchLength(blockWidth, stripeHeight); + Av1WienerFilter.FilterStripe( + filterSource, + borderedStride, + filterDestination, + filterDestinationStride, + blockWidth, + stripeHeight, + this.frameBuffer.BitDepth.GetBitCount(), + unit.WienerHorizontal, + unit.WienerVertical, + wienerScratch[..scratchLength]); + } + else + { + int scratchLength = Av1SelfGuidedFilter.GetScratchLength(blockWidth, stripeHeight); + Av1SelfGuidedFilter.FilterBlock( + filterSource, + borderedStride, + filterDestination, + filterDestinationStride, + blockWidth, + stripeHeight, + this.frameBuffer.BitDepth.GetBitCount(), + unit.SgrParameterSet, + unit.SgrProjectionCoefficients, + selfGuidedScratch[..scratchLength]); + } + + if (highBitDepthDestination.IsEmpty) + { + CopyFilterOutput( + filterDestination, + filterDestinationStride, + lowBitDepthDestination, + destinationOffset, + planeWidth, + blockWidth, + stripeHeight); + } + } + + unitRow += stripeHeight; + } + } + + /// + /// Builds one filter source rectangle with normative horizontal and stripe-boundary extension. + /// + /// The zero-based color-plane index. + /// The frame-relative processing-stripe index. + /// The immutable byte post-super-resolution plane, when present. + /// The immutable 16-bit post-super-resolution plane, when present. + /// The number of samples between source rows. + /// The visible plane width. + /// The visible plane height. + /// The first filtered plane column. + /// The number of filtered columns. + /// The first filtered plane row. + /// The number of filtered rows. + /// The number of samples between bordered destination rows. + /// The bordered filter source rectangle. + private void PopulateBorderedSource( + int plane, + int frameStripe, + ReadOnlySpan lowBitDepthSource, + ReadOnlySpan highBitDepthSource, + int sourceStride, + int planeWidth, + int planeHeight, + int blockStart, + int blockWidth, + int stripeStart, + int stripeHeight, + int destinationStride, + Span destination) + { + int stripeEnd = stripeStart + stripeHeight; + int sourceRowCount = stripeHeight + (FilterBorder * 2) + WienerPadding; + for (int destinationRow = 0; destinationRow < sourceRowCount; destinationRow++) + { + int sourceY = stripeStart + destinationRow - FilterBorder; + ReadOnlySpan lowBitDepthSourceRow = default; + ReadOnlySpan highBitDepthSourceRow; + if (sourceY < 0) + { + highBitDepthSourceRow = this.boundary.GetRowAbove(plane, frameStripe, 0); + } + else if (sourceY < stripeStart) + { + // Two preserved deblocked rows expand to three filter rows as [0, 0, 1]. + int contextRow = Math.Min(Math.Max(destinationRow - 1, 0), 1); + highBitDepthSourceRow = this.boundary.GetRowAbove(plane, frameStripe, contextRow); + } + else if (sourceY >= planeHeight) + { + highBitDepthSourceRow = this.boundary.GetRowBelow(plane, frameStripe, 0); + } + else if (sourceY >= stripeEnd) + { + // The bottom expansion is [0, 1, 1]; the padded Wiener zero tap also reads row 1. + int contextRow = Math.Min(sourceY - stripeEnd, 1); + highBitDepthSourceRow = this.boundary.GetRowBelow(plane, frameStripe, contextRow); + } + else if (!highBitDepthSource.IsEmpty) + { + int sourceOffset = sourceStride + (sourceY * sourceStride); + highBitDepthSourceRow = highBitDepthSource.Slice(sourceOffset, planeWidth); + } + else + { + int sourceOffset = sourceStride + (sourceY * sourceStride); + lowBitDepthSourceRow = lowBitDepthSource.Slice(sourceOffset, planeWidth); + highBitDepthSourceRow = default; + } + + Span destinationRowSpan = destination.Slice(destinationRow * destinationStride, destinationStride); + int sourceX = blockStart - FilterBorder; + int leftExtension = Math.Max(-sourceX, 0); + + // Horizontal context crosses restoration-unit and self-guided processing-unit edges. + // Replication occurs only at the visible frame boundary. + if (leftExtension > 0) + { + ushort firstSample = highBitDepthSourceRow.IsEmpty + ? lowBitDepthSourceRow[0] + : highBitDepthSourceRow[0]; + + destinationRowSpan[..leftExtension].Fill(firstSample); + } + + int copiedStart = Math.Max(sourceX, 0); + int copiedEnd = Math.Min(sourceX + destinationStride, planeWidth); + int copiedLength = copiedEnd - copiedStart; + Span copiedDestination = destinationRowSpan.Slice(leftExtension, copiedLength); + if (!highBitDepthSourceRow.IsEmpty) + { + highBitDepthSourceRow.Slice(copiedStart, copiedLength).CopyTo(copiedDestination); + } + else + { + ReadOnlySpan copiedSource = lowBitDepthSourceRow.Slice(copiedStart, copiedLength); + for (int column = 0; column < copiedLength; column++) + { + copiedDestination[column] = copiedSource[column]; + } + } + + int populatedLength = leftExtension + copiedLength; + if (populatedLength < destinationStride) + { + ushort lastSample = highBitDepthSourceRow.IsEmpty + ? lowBitDepthSourceRow[^1] + : highBitDepthSourceRow[^1]; + + destinationRowSpan[populatedLength..].Fill(lastSample); + } + } + } + + /// + /// Copies an unfiltered restoration-unit rectangle between plane working buffers. + /// + /// The immutable byte source plane, when present. + /// The immutable 16-bit source plane, when present. + /// The number of samples between source rows. + /// The byte destination, when present. + /// The native 16-bit destination, when present. + /// The number of samples between plane rows. + /// The first copied column. + /// The number of copied columns. + /// The first copied row. + /// The exclusive copied row limit. + private static void CopyRectangle( + ReadOnlySpan lowBitDepthSource, + ReadOnlySpan highBitDepthSource, + int sourceStride, + Span lowBitDepthDestination, + Span highBitDepthDestination, + int planeWidth, + int horizontalStart, + int width, + int verticalStart, + int verticalEnd) + { + for (int row = verticalStart; row < verticalEnd; row++) + { + int sourceOffset = sourceStride + (row * sourceStride) + horizontalStart; + int destinationOffset = (row * planeWidth) + horizontalStart; + if (!highBitDepthDestination.IsEmpty) + { + highBitDepthSource.Slice(sourceOffset, width).CopyTo(highBitDepthDestination.Slice(destinationOffset, width)); + } + else + { + lowBitDepthSource.Slice(sourceOffset, width).CopyTo(lowBitDepthDestination.Slice(destinationOffset, width)); + } + } + } + + /// + /// Narrows one bounded restoration-filter output into its eight-bit frame destination. + /// + /// The 16-bit filter output. + /// The number of samples between source rows. + /// The byte restoration destination. + /// The offset of the first destination sample. + /// The number of samples between destination rows. + /// The copied width in samples. + /// The copied height in samples. + private static void CopyFilterOutput( + ReadOnlySpan source, + int sourceStride, + Span destination, + int destinationOffset, + int destinationStride, + int width, + int height) + { + for (int row = 0; row < height; row++) + { + ReadOnlySpan sourceRow = source.Slice(row * sourceStride, width); + Span destinationRow = destination.Slice(destinationOffset + (row * destinationStride), width); + + for (int column = 0; column < width; column++) + { + // The 8-bit restoration filters clip every result to the bit-depth range before this exact narrowing. + destinationRow[column] = (byte)sourceRow[column]; + } + } + } + + /// + /// Copies one native-width restoration buffer back to its reconstructed plane. + /// + /// The luma or chroma plane. + /// The horizontal chroma subsampling shift. + /// The vertical chroma subsampling shift. + /// The visible plane width. + /// The visible plane height. + /// The row-major byte restoration buffer, when present. + /// The row-major native restoration buffer, when present. + private void CopyWorkingBufferToPlane( + Av1Plane plane, + int subsamplingX, + int subsamplingY, + int planeWidth, + int planeHeight, + ReadOnlySpan lowBitDepthSource, + ReadOnlySpan highBitDepthSource) + { + Span lowBitDepthPlane = default; + Span highBitDepthPlane = default; + int destinationStride; + if (this.frameBuffer.BytesPerSample == 2) + { + Span signedPlane = this.frameBuffer.DeriveBlockPointer16( + plane, + Point.Empty, + subsamplingX, + subsamplingY, + out destinationStride); + + highBitDepthPlane = MemoryMarshal.Cast(signedPlane); + } + else + { + lowBitDepthPlane = this.frameBuffer.DeriveBlockPointer( + plane, + Point.Empty, + subsamplingX, + subsamplingY, + out destinationStride); + } + + for (int row = 0; row < planeHeight; row++) + { + // The destination view has the same preceding prediction row as the source view. + int frameOffset = destinationStride + (row * destinationStride); + if (!highBitDepthPlane.IsEmpty) + { + highBitDepthSource.Slice(row * planeWidth, planeWidth).CopyTo(highBitDepthPlane[frameOffset..]); + } + else + { + lowBitDepthSource.Slice(row * planeWidth, planeWidth).CopyTo(lowBitDepthPlane[frameOffset..]); + } + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/LoopRestoration/Av1SelfGuidedFilter.Operations.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/LoopRestoration/Av1SelfGuidedFilter.Operations.cs new file mode 100644 index 0000000000..b269bf9915 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/LoopRestoration/Av1SelfGuidedFilter.Operations.cs @@ -0,0 +1,1431 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.InteropServices; +using System.Runtime.Intrinsics; +using System.Runtime.Intrinsics.X86; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.LoopRestoration; + +/// +/// Provides the 128- and 256-bit self-guided restoration pipelines. Consecutive lanes represent neighboring output +/// columns throughout integral-image construction, coefficient generation, filtering, and projection. Each vector +/// loop passes its final horizontal prefix to the scalar tail, preserving one continuous summed-area row without +/// recomputing already processed samples. +/// +internal static partial class Av1SelfGuidedFilter +{ + /// + /// Applies self-guided restoration with the 256-bit traversal. + /// + /// The bordered processing-unit source rectangle. + /// The number of samples between source rows. + /// The destination processing-unit rectangle. + /// The number of samples between destination rows. + /// The processing-unit width in samples. + /// The processing-unit height in samples. + /// The encoded sample bit depth. + /// The decoded self-guided parameter-set index. + /// The two transmitted projection coefficients. + /// The caller-owned work storage. + /// The overload-selection value. + private static void FilterBlock( + ReadOnlySpan source, + int sourceStride, + Span destination, + int destinationStride, + int width, + int height, + int bitDepth, + int parameterSetIndex, + ReadOnlySpan projectionCoefficients, + Span scratch, + Vector256 vector) + { + // The caller-owned span contains two visible filtered planes followed by four identically strided coefficient + // planes. Keeping these regions disjoint allows projection to read either radius after the integral buffers + // have been reused as immutable inputs, without per-unit allocation or copying. + int filteredLength = width * height; + int bufferLength = GetBufferLength(width, height); + int bufferStride = GetBufferStride(width); + Span filtered0 = scratch[..filteredLength]; + Span filtered1 = scratch.Slice(filteredLength, filteredLength); + Span blendFactors = scratch.Slice(filteredLength * 2, bufferLength); + Span localMeans = scratch.Slice((filteredLength * 2) + bufferLength, bufferLength); + Span squareIntegral = scratch.Slice((filteredLength * 2) + (bufferLength * 2), bufferLength); + Span sumIntegral = scratch.Slice((filteredLength * 2) + (bufferLength * 3), bufferLength); + + BuildIntegralImages(source, sourceStride, width + (Border * 2), height + (Border * 2), bufferStride, squareIntegral, sumIntegral, vector); + + int parameterOffset = parameterSetIndex * 2; + ReadOnlySpan radii = ParameterRadii.Slice(parameterOffset, 2); + ReadOnlySpan scales = ParameterScales.Slice(parameterOffset, 2); + if (radii[0] > 0) + { + CalculateIntermediateCoefficients( + width, + height, + bitDepth, + radii[0], + scales[0], + skipAlternateRows: true, + bufferStride, + squareIntegral, + sumIntegral, + blendFactors, + localMeans, + vector); + + CalculateRadiusTwoFilter(source, sourceStride, width, height, bufferStride, blendFactors, localMeans, filtered0, vector); + } + + if (radii[1] > 0) + { + CalculateIntermediateCoefficients( + width, + height, + bitDepth, + radii[1], + scales[1], + skipAlternateRows: false, + bufferStride, + squareIntegral, + sumIntegral, + blendFactors, + localMeans, + vector); + + CalculateRadiusOneFilter(source, sourceStride, width, height, bufferStride, blendFactors, localMeans, filtered1, vector); + } + + DecodeProjectionCoefficients(radii, projectionCoefficients, out int projection0, out int projection1); + Project( + source, + sourceStride, + destination, + destinationStride, + width, + height, + bitDepth, + radii, + projection0, + projection1, + filtered0, + filtered1, + vector); + } + + /// + /// Applies self-guided restoration with the cross-platform 128-bit traversal. + /// + /// The bordered processing-unit source rectangle. + /// The number of samples between source rows. + /// The destination processing-unit rectangle. + /// The number of samples between destination rows. + /// The processing-unit width in samples. + /// The processing-unit height in samples. + /// The encoded sample bit depth. + /// The decoded self-guided parameter-set index. + /// The two transmitted projection coefficients. + /// The caller-owned work storage. + /// The overload-selection value. + private static void FilterBlock( + ReadOnlySpan source, + int sourceStride, + Span destination, + int destinationStride, + int width, + int height, + int bitDepth, + int parameterSetIndex, + ReadOnlySpan projectionCoefficients, + Span scratch, + Vector128 vector) + { + // Use the same scratch partition as the 256-bit path. Vector width changes only the number of adjacent columns + // advanced by each stage; all offsets and fixed-point representations remain identical. + int filteredLength = width * height; + int bufferLength = GetBufferLength(width, height); + int bufferStride = GetBufferStride(width); + Span filtered0 = scratch[..filteredLength]; + Span filtered1 = scratch.Slice(filteredLength, filteredLength); + Span blendFactors = scratch.Slice(filteredLength * 2, bufferLength); + Span localMeans = scratch.Slice((filteredLength * 2) + bufferLength, bufferLength); + Span squareIntegral = scratch.Slice((filteredLength * 2) + (bufferLength * 2), bufferLength); + Span sumIntegral = scratch.Slice((filteredLength * 2) + (bufferLength * 3), bufferLength); + + BuildIntegralImages(source, sourceStride, width + (Border * 2), height + (Border * 2), bufferStride, squareIntegral, sumIntegral, vector); + + int parameterOffset = parameterSetIndex * 2; + ReadOnlySpan radii = ParameterRadii.Slice(parameterOffset, 2); + ReadOnlySpan scales = ParameterScales.Slice(parameterOffset, 2); + if (radii[0] > 0) + { + CalculateIntermediateCoefficients( + width, + height, + bitDepth, + radii[0], + scales[0], + skipAlternateRows: true, + bufferStride, + squareIntegral, + sumIntegral, + blendFactors, + localMeans, + vector); + + CalculateRadiusTwoFilter(source, sourceStride, width, height, bufferStride, blendFactors, localMeans, filtered0, vector); + } + + if (radii[1] > 0) + { + CalculateIntermediateCoefficients( + width, + height, + bitDepth, + radii[1], + scales[1], + skipAlternateRows: false, + bufferStride, + squareIntegral, + sumIntegral, + blendFactors, + localMeans, + vector); + + CalculateRadiusOneFilter(source, sourceStride, width, height, bufferStride, blendFactors, localMeans, filtered1, vector); + } + + DecodeProjectionCoefficients(radii, projectionCoefficients, out int projection0, out int projection1); + Project( + source, + sourceStride, + destination, + destinationStride, + width, + height, + bitDepth, + radii, + projection0, + projection1, + filtered0, + filtered1, + vector); + } + + /// + /// Builds the summed-area tables consumed by the 256-bit coefficient stage. + /// + /// The complete bordered source rectangle. + /// The number of samples between source rows. + /// The bordered source width. + /// The bordered source height. + /// The padded work-buffer row stride. + /// The destination integral image of squared samples. + /// The destination integral image of samples. + /// The overload-selection value. + private static void BuildIntegralImages( + ReadOnlySpan source, + int sourceStride, + int width, + int height, + int bufferStride, + Span squareIntegral, + Span sumIntegral, + Vector256 vector) + { + squareIntegral[..(width + 1)].Clear(); + sumIntegral[..(width + 1)].Clear(); + ref ushort sourceBase = ref MemoryMarshal.GetReference(source); + ref int squareBase = ref MemoryMarshal.GetReference(squareIntegral); + ref int sumBase = ref MemoryMarshal.GetReference(sumIntegral); + + for (int row = 0; row < height; row++) + { + int sourceRowOffset = row * sourceStride; + int previousRowOffset = row * bufferStride; + int currentRowOffset = previousRowOffset + bufferStride; + squareIntegral[currentRowOffset] = 0; + sumIntegral[currentRowOffset] = 0; + Vector256 squareCarry = Vector256.Zero; + Vector256 sumCarry = Vector256.Zero; + int column = 0; + nuint vectorCount = Numerics.Vector256Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector256.Count) + { + // Eight packed 16-bit samples become eight 32-bit lanes. The prefix scans mirror + // the reference decoder's scan_32, and the replicated carry joins consecutive vector batches. + Vector128 packed = Vector128.LoadUnsafe(ref sourceBase, (nuint)(sourceRowOffset + column)); + Vector256 samples = Vector256.WidenLower(Vector256.Create(packed, Vector128.Zero)).AsInt32(); + Vector256 squares = samples * samples; + Vector256 scannedSums = Scan(samples); + Vector256 scannedSquares = Scan(squares); + Vector256 sumsAbove = Vector256.LoadUnsafe(ref sumBase, (nuint)(previousRowOffset + column + 1)); + Vector256 squaresAbove = Vector256.LoadUnsafe(ref squareBase, (nuint)(previousRowOffset + column + 1)); + Vector256 rowSums = scannedSums + sumsAbove + sumCarry; + Vector256 rowSquares = scannedSquares + squaresAbove + squareCarry; + rowSums.StoreUnsafe(ref sumBase, (nuint)(currentRowOffset + column + 1)); + rowSquares.StoreUnsafe(ref squareBase, (nuint)(currentRowOffset + column + 1)); + + sumCarry = Vector256.Create(rowSums.GetElement(Vector256.Count - 1) - sumsAbove.GetElement(Vector256.Count - 1)); + squareCarry = Vector256.Create(rowSquares.GetElement(Vector256.Count - 1) - squaresAbove.GetElement(Vector256.Count - 1)); + } + + int runningSum = sumCarry.GetElement(0); + int runningSquareSum = squareCarry.GetElement(0); + for (; column < width; column++) + { + int sample = source[sourceRowOffset + column]; + runningSum += sample; + runningSquareSum += sample * sample; + sumIntegral[currentRowOffset + column + 1] = sumIntegral[previousRowOffset + column + 1] + runningSum; + squareIntegral[currentRowOffset + column + 1] = squareIntegral[previousRowOffset + column + 1] + runningSquareSum; + } + } + } + + /// + /// Builds the summed-area tables consumed by the cross-platform coefficient stage. + /// + /// The complete bordered source rectangle. + /// The number of samples between source rows. + /// The bordered source width. + /// The bordered source height. + /// The padded work-buffer row stride. + /// The destination integral image of squared samples. + /// The destination integral image of samples. + /// The overload-selection value. + private static void BuildIntegralImages( + ReadOnlySpan source, + int sourceStride, + int width, + int height, + int bufferStride, + Span squareIntegral, + Span sumIntegral, + Vector128 vector) + { + squareIntegral[..(width + 1)].Clear(); + sumIntegral[..(width + 1)].Clear(); + ref ushort sourceBase = ref MemoryMarshal.GetReference(source); + ref int squareBase = ref MemoryMarshal.GetReference(squareIntegral); + ref int sumBase = ref MemoryMarshal.GetReference(sumIntegral); + + for (int row = 0; row < height; row++) + { + int sourceRowOffset = row * sourceStride; + int previousRowOffset = row * bufferStride; + int currentRowOffset = previousRowOffset + bufferStride; + squareIntegral[currentRowOffset] = 0; + sumIntegral[currentRowOffset] = 0; + Vector128 squareCarry = Vector128.Zero; + Vector128 sumCarry = Vector128.Zero; + int column = 0; + nuint vectorCount = Numerics.Vector128Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector128.Count) + { + // Loading through Vector64 avoids reading beyond the four samples owned by this + // batch. Widening is normalized by the runtime for both x86 and Arm64 targets. + ref ushort sourceReference = ref Unsafe.Add(ref sourceBase, sourceRowOffset + column); + Vector64 packed = Unsafe.As>(ref sourceReference); + Vector128 samples = Vector128.WidenLower(Vector128.Create(packed, Vector64.Zero)).AsInt32(); + Vector128 squares = samples * samples; + Vector128 scannedSums = Scan(samples); + Vector128 scannedSquares = Scan(squares); + Vector128 sumsAbove = Vector128.LoadUnsafe(ref sumBase, (nuint)(previousRowOffset + column + 1)); + Vector128 squaresAbove = Vector128.LoadUnsafe(ref squareBase, (nuint)(previousRowOffset + column + 1)); + Vector128 rowSums = scannedSums + sumsAbove + sumCarry; + Vector128 rowSquares = scannedSquares + squaresAbove + squareCarry; + rowSums.StoreUnsafe(ref sumBase, (nuint)(currentRowOffset + column + 1)); + rowSquares.StoreUnsafe(ref squareBase, (nuint)(currentRowOffset + column + 1)); + + sumCarry = Vector128.Create(rowSums.GetElement(Vector128.Count - 1) - sumsAbove.GetElement(Vector128.Count - 1)); + squareCarry = Vector128.Create(rowSquares.GetElement(Vector128.Count - 1) - squaresAbove.GetElement(Vector128.Count - 1)); + } + + int runningSum = sumCarry.GetElement(0); + int runningSquareSum = squareCarry.GetElement(0); + for (; column < width; column++) + { + int sample = source[sourceRowOffset + column]; + runningSum += sample; + runningSquareSum += sample * sample; + sumIntegral[currentRowOffset + column + 1] = sumIntegral[previousRowOffset + column + 1] + runningSum; + squareIntegral[currentRowOffset + column + 1] = squareIntegral[previousRowOffset + column + 1] + runningSquareSum; + } + } + } + + /// + /// Computes inclusive prefix sums for eight 32-bit lanes. + /// + /// The independent input values. + /// The inclusive prefix sum in each lane. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector256 Scan(Vector256 values) + { + if (Avx2.IsSupported) + { + // AVX2 lane shifts provide the shortest x86 dependency chain. After scanning each 128-bit half, the lower + // half total is broadcast into the upper half so the result remains one continuous eight-lane prefix. + Vector256 avx2Scan = values + Avx2.ShiftLeftLogical128BitLane(values.AsByte(), sizeof(int)).AsInt32(); + avx2Scan += Avx2.ShiftLeftLogical128BitLane(avx2Scan.AsByte(), sizeof(int) * 2).AsInt32(); + Vector256 lowerTotal = Vector256.Create(Vector128.Zero, Vector128.Create(avx2Scan.GetElement(3))); + return avx2Scan + lowerTotal; + } + + // Each shuffle shifts the preceding partial sums by one, two, and four lanes. The portable shuffle is required + // because indices outside the vector produce zero; ShuffleNative is allowed to wrap those indices on some ISAs. + Vector256 scan = values + Vector256.Shuffle(values, Vector256.Create(8, 0, 1, 2, 3, 4, 5, 6)); + scan += Vector256.Shuffle(scan, Vector256.Create(8, 8, 0, 1, 2, 3, 4, 5)); + return scan + Vector256.Shuffle(scan, Vector256.Create(8, 8, 8, 8, 0, 1, 2, 3)); + } + + /// + /// Computes inclusive prefix sums for four 32-bit lanes. + /// + /// The independent input values. + /// The inclusive prefix sum in each lane. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector128 Scan(Vector128 values) + { + // The portable shuffle is required here because its out-of-range indices produce zero. + // ShuffleNative may mask those indices and wrap them back into the input on some ISAs. + Vector128 scan = values + Vector128.Shuffle(values, Vector128.Create(4, 0, 1, 2)); + return scan + Vector128.Shuffle(scan, Vector128.Create(4, 4, 0, 1)); + } + + /// + /// Calculates the coefficient grid in eight-sample SIMD batches. + /// + /// The processing-unit width in samples. + /// The processing-unit height in samples. + /// The encoded sample bit depth. + /// The square-window radius. + /// The variance scale for the selected parameter set. + /// Whether only alternate coefficient rows are required. + /// The padded work-buffer row stride. + /// The integral image of squared samples. + /// The integral image of samples. + /// The destination local blend factors. + /// The destination scaled local means. + /// The overload-selection value. + private static void CalculateIntermediateCoefficients( + int width, + int height, + int bitDepth, + int radius, + int scale, + bool skipAlternateRows, + int bufferStride, + ReadOnlySpan squareIntegral, + ReadOnlySpan sumIntegral, + Span blendFactors, + Span localMeans, + Vector256 vector) + { + int windowDiameter = (radius * 2) + 1; + int windowArea = windowDiameter * windowDiameter; + int reciprocal = OneByX[windowArea - 1]; + int rowStep = skipAlternateRows ? 2 : 1; + int bufferOrigin = (Border + 1) * (bufferStride + 1); + Vector256 windowAreaVector = Vector256.Create(windowArea); + Vector256 scaleVector = Vector256.Create((uint)scale); + Vector256 reciprocalVector = Vector256.Create(reciprocal); + Vector256 varianceRounding = Vector256.Create(1U << (ScaleBits - 1)); + Vector256 meanRounding = Vector256.Create(1U << (ReciprocalBits - 1)); + Vector256 maximumTableIndex = Vector256.Create(255U); + Vector256 selfGuidedScale = Vector256.Create(SelfGuidedScale); + ref int blendBase = ref MemoryMarshal.GetReference(blendFactors); + ref int meanBase = ref MemoryMarshal.GetReference(localMeans); + + for (int row = -1; row < height + 1; row += rowStep) + { + int column = -1; + int remaining = width + 2; + for (; remaining >= Vector256.Count; column += Vector256.Count, remaining -= Vector256.Count) + { + Vector256 sums = BoxSum(sumIntegral, bufferOrigin + (row * bufferStride) + column, bufferStride, radius, vector); + Vector256 squareSums = BoxSum(squareIntegral, bufferOrigin + (row * bufferStride) + column, bufferStride, radius, vector); + Vector256 variance = CalculateVariance(sums, squareSums, bitDepth, windowAreaVector, vector); + + // The fixed-point product is intentionally unsigned. Its legal range can set the + // sign bit even though the normative value remains a non-negative 32-bit integer. + Vector256 tableIndices = Vector256.Min( + Vector256.ShiftRightLogical((variance * scaleVector) + varianceRounding, ScaleBits), + maximumTableIndex); + + Vector256 factors = LookupBlendFactors(tableIndices); + Vector256 meanProducts = ((selfGuidedScale - factors) * reciprocalVector * sums).AsUInt32(); + Vector256 means = Vector256.ShiftRightLogical(meanProducts + meanRounding, ReciprocalBits).AsInt32(); + int coefficientOffset = bufferOrigin + (row * bufferStride) + column; + factors.StoreUnsafe(ref blendBase, (nuint)coefficientOffset); + means.StoreUnsafe(ref meanBase, (nuint)coefficientOffset); + } + + for (; remaining > 0; column++, remaining--) + { + CalculateIntermediateCoefficient( + squareIntegral, + sumIntegral, + bufferOrigin + (row * bufferStride) + column, + bufferStride, + bitDepth, + radius, + windowArea, + scale, + reciprocal, + out int blendFactor, + out int localMean); + + int coefficientOffset = bufferOrigin + (row * bufferStride) + column; + blendFactors[coefficientOffset] = blendFactor; + localMeans[coefficientOffset] = localMean; + } + } + } + + /// + /// Calculates the coefficient grid in four-sample cross-platform batches. + /// + /// The processing-unit width in samples. + /// The processing-unit height in samples. + /// The encoded sample bit depth. + /// The square-window radius. + /// The variance scale for the selected parameter set. + /// Whether only alternate coefficient rows are required. + /// The padded work-buffer row stride. + /// The integral image of squared samples. + /// The integral image of samples. + /// The destination local blend factors. + /// The destination scaled local means. + /// The overload-selection value. + private static void CalculateIntermediateCoefficients( + int width, + int height, + int bitDepth, + int radius, + int scale, + bool skipAlternateRows, + int bufferStride, + ReadOnlySpan squareIntegral, + ReadOnlySpan sumIntegral, + Span blendFactors, + Span localMeans, + Vector128 vector) + { + int windowDiameter = (radius * 2) + 1; + int windowArea = windowDiameter * windowDiameter; + int reciprocal = OneByX[windowArea - 1]; + int rowStep = skipAlternateRows ? 2 : 1; + int bufferOrigin = (Border + 1) * (bufferStride + 1); + Vector128 windowAreaVector = Vector128.Create(windowArea); + Vector128 scaleVector = Vector128.Create((uint)scale); + Vector128 reciprocalVector = Vector128.Create(reciprocal); + Vector128 varianceRounding = Vector128.Create(1U << (ScaleBits - 1)); + Vector128 meanRounding = Vector128.Create(1U << (ReciprocalBits - 1)); + Vector128 maximumTableIndex = Vector128.Create(255U); + Vector128 selfGuidedScale = Vector128.Create(SelfGuidedScale); + ref int blendBase = ref MemoryMarshal.GetReference(blendFactors); + ref int meanBase = ref MemoryMarshal.GetReference(localMeans); + + for (int row = -1; row < height + 1; row += rowStep) + { + int column = -1; + int remaining = width + 2; + for (; remaining >= Vector128.Count; column += Vector128.Count, remaining -= Vector128.Count) + { + Vector128 sums = BoxSum(sumIntegral, bufferOrigin + (row * bufferStride) + column, bufferStride, radius, vector); + Vector128 squareSums = BoxSum(squareIntegral, bufferOrigin + (row * bufferStride) + column, bufferStride, radius, vector); + Vector128 variance = CalculateVariance(sums, squareSums, bitDepth, windowAreaVector, vector); + Vector128 tableIndices = Vector128.Min( + Vector128.ShiftRightLogical((variance * scaleVector) + varianceRounding, ScaleBits), + maximumTableIndex); + + Vector128 factors = LookupBlendFactors(tableIndices); + Vector128 meanProducts = ((selfGuidedScale - factors) * reciprocalVector * sums).AsUInt32(); + Vector128 means = Vector128.ShiftRightLogical(meanProducts + meanRounding, ReciprocalBits).AsInt32(); + int coefficientOffset = bufferOrigin + (row * bufferStride) + column; + factors.StoreUnsafe(ref blendBase, (nuint)coefficientOffset); + means.StoreUnsafe(ref meanBase, (nuint)coefficientOffset); + } + + for (; remaining > 0; column++, remaining--) + { + CalculateIntermediateCoefficient( + squareIntegral, + sumIntegral, + bufferOrigin + (row * bufferStride) + column, + bufferStride, + bitDepth, + radius, + windowArea, + scale, + reciprocal, + out int blendFactor, + out int localMean); + + int coefficientOffset = bufferOrigin + (row * bufferStride) + column; + blendFactors[coefficientOffset] = blendFactor; + localMeans[coefficientOffset] = localMean; + } + } + } + + /// + /// Calculates eight adjacent box sums from one integral image. + /// + /// The source integral image. + /// The integral-image offset corresponding to the first box center. + /// The integral-image row stride. + /// The square-box radius. + /// The overload-selection value. + /// The eight adjacent box sums. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector256 BoxSum(ReadOnlySpan integral, int centerOffset, int stride, int radius, Vector256 vector) + { + ref int integralBase = ref MemoryMarshal.GetReference(integral); + int upperOffset = centerOffset - ((radius + 1) * stride); + int lowerOffset = centerOffset + (radius * stride); + Vector256 topLeft = Vector256.LoadUnsafe(ref integralBase, (nuint)(upperOffset - radius - 1)); + Vector256 topRight = Vector256.LoadUnsafe(ref integralBase, (nuint)(upperOffset + radius)); + Vector256 bottomLeft = Vector256.LoadUnsafe(ref integralBase, (nuint)(lowerOffset - radius - 1)); + Vector256 bottomRight = Vector256.LoadUnsafe(ref integralBase, (nuint)(lowerOffset + radius)); + return (bottomRight - bottomLeft) - (topRight - topLeft); + } + + /// + /// Calculates four adjacent box sums from one integral image. + /// + /// The source integral image. + /// The integral-image offset corresponding to the first box center. + /// The integral-image row stride. + /// The square-box radius. + /// The overload-selection value. + /// The four adjacent box sums. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector128 BoxSum(ReadOnlySpan integral, int centerOffset, int stride, int radius, Vector128 vector) + { + ref int integralBase = ref MemoryMarshal.GetReference(integral); + int upperOffset = centerOffset - ((radius + 1) * stride); + int lowerOffset = centerOffset + (radius * stride); + Vector128 topLeft = Vector128.LoadUnsafe(ref integralBase, (nuint)(upperOffset - radius - 1)); + Vector128 topRight = Vector128.LoadUnsafe(ref integralBase, (nuint)(upperOffset + radius)); + Vector128 bottomLeft = Vector128.LoadUnsafe(ref integralBase, (nuint)(lowerOffset - radius - 1)); + Vector128 bottomRight = Vector128.LoadUnsafe(ref integralBase, (nuint)(lowerOffset + radius)); + return (bottomRight - bottomLeft) - (topRight - topLeft); + } + + /// + /// Converts eight window sums into the bounded variance measure defined by AV1. + /// + /// The sample sums. + /// The squared-sample sums. + /// The encoded sample bit depth. + /// The replicated square-window area. + /// The overload-selection value. + /// The non-negative variance measure. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector256 CalculateVariance( + Vector256 sums, + Vector256 squareSums, + int bitDepth, + Vector256 windowArea, + Vector256 vector) + { + if (bitDepth > 8) + { + int depthShift = bitDepth - 8; + squareSums = Vector256.ShiftRightLogical( + squareSums.AsUInt32() + Vector256.Create(1U << ((depthShift * 2) - 1)), depthShift * 2).AsInt32(); + + sums = Vector256.ShiftRightLogical(sums.AsUInt32() + Vector256.Create(1U << (depthShift - 1)), depthShift).AsInt32(); + } + + Vector256 squareOfSums = sums * sums; + Vector256 scaledSquareSums = squareSums * windowArea; + + // Rounding high-bit-depth inputs can put the squared mean one step above + // the mean square. AV1 saturates that artifact before applying the scale. + return Vector256.Max(scaledSquareSums, squareOfSums).AsUInt32() - squareOfSums.AsUInt32(); + } + + /// + /// Converts four window sums into the bounded variance measure defined by AV1. + /// + /// The sample sums. + /// The squared-sample sums. + /// The encoded sample bit depth. + /// The replicated square-window area. + /// The overload-selection value. + /// The non-negative variance measure. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector128 CalculateVariance( + Vector128 sums, + Vector128 squareSums, + int bitDepth, + Vector128 windowArea, + Vector128 vector) + { + if (bitDepth > 8) + { + int depthShift = bitDepth - 8; + squareSums = Vector128.ShiftRightLogical( + squareSums.AsUInt32() + Vector128.Create(1U << ((depthShift * 2) - 1)), depthShift * 2).AsInt32(); + + sums = Vector128.ShiftRightLogical(sums.AsUInt32() + Vector128.Create(1U << (depthShift - 1)), depthShift).AsInt32(); + } + + Vector128 squareOfSums = sums * sums; + Vector128 scaledSquareSums = squareSums * windowArea; + return Vector128.Max(scaledSquareSums, squareOfSums).AsUInt32() - squareOfSums.AsUInt32(); + } + + /// + /// Maps eight bounded variance indices to their normative blend factors. + /// + /// The table indices. + /// The gathered blend factors. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static unsafe Vector256 LookupBlendFactors(Vector256 indices) + { + ReadOnlySpan table = XByXPlusOne; + + if (Avx2.IsSupported) + { + // Variance normalization bounds every index to the 256-entry table. AVX2 gather keeps all eight independent + // column lookups in the vector pipeline instead of materializing an intermediate scalar scale buffer. + fixed (int* tablePointer = table) + { + return Avx2.GatherVector256(tablePointer, indices.AsInt32(), sizeof(int)); + } + } + + // Vector256 has no portable indexed-load operation. Constructing the result from eight bounded reads retains + // the 256-bit coefficient pipeline on other implementations without allocating or adding another row pass. + return Vector256.Create( + table[(int)indices.GetElement(0)], + table[(int)indices.GetElement(1)], + table[(int)indices.GetElement(2)], + table[(int)indices.GetElement(3)], + table[(int)indices.GetElement(4)], + table[(int)indices.GetElement(5)], + table[(int)indices.GetElement(6)], + table[(int)indices.GetElement(7)]); + } + + /// + /// Maps four bounded variance indices to their normative blend factors. + /// + /// The table indices. + /// The gathered blend factors. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector128 LookupBlendFactors(Vector128 indices) + { + // Portable 128-bit APIs do not provide indexed loads. Four bounded scalar reads are assembled directly into + // the result vector, avoiding both an allocation and a second pass over the coefficient row. + ReadOnlySpan table = XByXPlusOne; + return Vector128.Create( + table[(int)indices.GetElement(0)], + table[(int)indices.GetElement(1)], + table[(int)indices.GetElement(2)], + table[(int)indices.GetElement(3)]); + } + + /// + /// Calculates one coefficient pair for a vector remainder. + /// + /// The integral image of squared samples. + /// The integral image of samples. + /// The integral-image offset corresponding to the box center. + /// The integral-image row stride. + /// The encoded sample bit depth. + /// The square-window radius. + /// The square-window area. + /// The variance scale for the selected parameter set. + /// The fixed-point reciprocal of the window area. + /// The calculated local blend factor. + /// The calculated scaled local mean. + private static void CalculateIntermediateCoefficient( + ReadOnlySpan squareIntegral, + ReadOnlySpan sumIntegral, + int centerOffset, + int stride, + int bitDepth, + int radius, + int windowArea, + int scale, + int reciprocal, + out int blendFactor, + out int localMean) + { + int sum = BoxSum(sumIntegral, centerOffset, stride, radius); + int squareSum = BoxSum(squareIntegral, centerOffset, stride, radius); + int normalizedSquareSum = RoundPowerOfTwo(squareSum, 2 * (bitDepth - 8)); + int normalizedSum = RoundPowerOfTwo(sum, bitDepth - 8); + uint squareOfSum = (uint)normalizedSum * (uint)normalizedSum; + uint scaledSquareSum = (uint)normalizedSquareSum * (uint)windowArea; + uint variance = scaledSquareSum < squareOfSum ? 0 : scaledSquareSum - squareOfSum; + uint varianceIndex = RoundPowerOfTwo(variance * (uint)scale, ScaleBits); + blendFactor = XByXPlusOne[(int)Math.Min(varianceIndex, 255U)]; + uint meanProduct = (uint)(SelfGuidedScale - blendFactor) * (uint)reciprocal * (uint)sum; + localMean = (int)RoundPowerOfTwo(meanProduct, ReciprocalBits); + } + + /// + /// Calculates one square-window sum from an integral image. + /// + /// The source integral image. + /// The integral-image offset corresponding to the box center. + /// The integral-image row stride. + /// The square-box radius. + /// The square-window sum. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static int BoxSum(ReadOnlySpan integral, int centerOffset, int stride, int radius) + { + int upperOffset = centerOffset - ((radius + 1) * stride); + int lowerOffset = centerOffset + (radius * stride); + int top = integral[upperOffset + radius] - integral[upperOffset - radius - 1]; + int bottom = integral[lowerOffset + radius] - integral[lowerOffset - radius - 1]; + return bottom - top; + } + + /// + /// Produces the radius-two filtered values in eight-sample SIMD batches. + /// + /// The bordered processing-unit source rectangle. + /// The number of samples between source rows. + /// The processing-unit width in samples. + /// The processing-unit height in samples. + /// The padded coefficient-buffer row stride. + /// The local sample blend factors. + /// The scaled local means. + /// The destination fixed-point filtered values. + /// The overload-selection value. + private static void CalculateRadiusTwoFilter( + ReadOnlySpan source, + int sourceStride, + int width, + int height, + int bufferStride, + ReadOnlySpan blendFactors, + ReadOnlySpan localMeans, + Span filtered, + Vector256 vector) + { + int bufferOrigin = (Border + 1) * (bufferStride + 1); + ref ushort sourceBase = ref MemoryMarshal.GetReference(source); + ref int filteredBase = ref MemoryMarshal.GetReference(filtered); + + for (int row = 0; row < height; row++) + { + int sourceRowOffset = ((row + Border) * sourceStride) + Border; + int filteredRowOffset = row * width; + int coefficientRowOffset = bufferOrigin + (row * bufferStride); + int roundingBits = SelfGuidedBits + ((row & 1) == 0 ? 5 : 4) - RestorationBits; + Vector256 rounding = Vector256.Create(1 << (roundingBits - 1)); + int column = 0; + int vectorEnd = (int)(Numerics.Vector256Count(width) * (nuint)Vector256.Count); + + // Filtered signals use Q4 precision. Projection applies the signaled Q7 weights to their difference from + // the unfiltered Q4 sample, then performs the combined Q11 rounding shift once before clipping. + for (; column < vectorEnd; column += Vector256.Count) + { + Vector256 factors = CrossSum(blendFactors, coefficientRowOffset + column, bufferStride, row, vector); + Vector256 means = CrossSum(localMeans, coefficientRowOffset + column, bufferStride, row, vector); + Vector128 packed = Vector128.LoadUnsafe(ref sourceBase, (nuint)(sourceRowOffset + column)); + Vector256 samples = Vector256.WidenLower(Vector256.Create(packed, Vector128.Zero)).AsInt32(); + Vector256 values = Vector256.ShiftRightArithmetic((factors * samples) + means + rounding, roundingBits); + values.StoreUnsafe(ref filteredBase, (nuint)(filteredRowOffset + column)); + } + + for (; column < width; column++) + { + int factors = CrossSum(blendFactors, coefficientRowOffset + column, bufferStride, row); + int means = CrossSum(localMeans, coefficientRowOffset + column, bufferStride, row); + int value = (factors * source[sourceRowOffset + column]) + means; + filtered[filteredRowOffset + column] = RoundPowerOfTwo(value, roundingBits); + } + } + } + + /// + /// Produces the radius-two filtered values in four-sample cross-platform batches. + /// + /// The bordered processing-unit source rectangle. + /// The number of samples between source rows. + /// The processing-unit width in samples. + /// The processing-unit height in samples. + /// The padded coefficient-buffer row stride. + /// The local sample blend factors. + /// The scaled local means. + /// The destination fixed-point filtered values. + /// The overload-selection value. + private static void CalculateRadiusTwoFilter( + ReadOnlySpan source, + int sourceStride, + int width, + int height, + int bufferStride, + ReadOnlySpan blendFactors, + ReadOnlySpan localMeans, + Span filtered, + Vector128 vector) + { + int bufferOrigin = (Border + 1) * (bufferStride + 1); + ref ushort sourceBase = ref MemoryMarshal.GetReference(source); + ref int filteredBase = ref MemoryMarshal.GetReference(filtered); + + for (int row = 0; row < height; row++) + { + int sourceRowOffset = ((row + Border) * sourceStride) + Border; + int filteredRowOffset = row * width; + int coefficientRowOffset = bufferOrigin + (row * bufferStride); + int roundingBits = SelfGuidedBits + ((row & 1) == 0 ? 5 : 4) - RestorationBits; + Vector128 rounding = Vector128.Create(1 << (roundingBits - 1)); + int column = 0; + int vectorEnd = (int)(Numerics.Vector128Count(width) * (nuint)Vector128.Count); + + // The 128-bit path uses the same Q4/Q7 projection equation. The four-sample load and store are deliberately + // 64 bits wide so a tightly strided destination row never requires writable padding. + for (; column < vectorEnd; column += Vector128.Count) + { + Vector128 factors = CrossSum(blendFactors, coefficientRowOffset + column, bufferStride, row, vector); + Vector128 means = CrossSum(localMeans, coefficientRowOffset + column, bufferStride, row, vector); + ref ushort sourceReference = ref Unsafe.Add(ref sourceBase, sourceRowOffset + column); + Vector64 packed = Unsafe.As>(ref sourceReference); + Vector128 samples = Vector128.WidenLower(Vector128.Create(packed, Vector64.Zero)).AsInt32(); + Vector128 values = Vector128.ShiftRightArithmetic((factors * samples) + means + rounding, roundingBits); + values.StoreUnsafe(ref filteredBase, (nuint)(filteredRowOffset + column)); + } + + for (; column < width; column++) + { + int factors = CrossSum(blendFactors, coefficientRowOffset + column, bufferStride, row); + int means = CrossSum(localMeans, coefficientRowOffset + column, bufferStride, row); + int value = (factors * source[sourceRowOffset + column]) + means; + filtered[filteredRowOffset + column] = RoundPowerOfTwo(value, roundingBits); + } + } + } + + /// + /// Produces the radius-one filtered values in eight-sample SIMD batches. + /// + /// The bordered processing-unit source rectangle. + /// The number of samples between source rows. + /// The processing-unit width in samples. + /// The processing-unit height in samples. + /// The padded coefficient-buffer row stride. + /// The local sample blend factors. + /// The scaled local means. + /// The destination fixed-point filtered values. + /// The overload-selection value. + private static void CalculateRadiusOneFilter( + ReadOnlySpan source, + int sourceStride, + int width, + int height, + int bufferStride, + ReadOnlySpan blendFactors, + ReadOnlySpan localMeans, + Span filtered, + Vector256 vector) + { + int bufferOrigin = (Border + 1) * (bufferStride + 1); + int roundingBits = SelfGuidedBits + 5 - RestorationBits; + Vector256 rounding = Vector256.Create(1 << (roundingBits - 1)); + ref ushort sourceBase = ref MemoryMarshal.GetReference(source); + ref int filteredBase = ref MemoryMarshal.GetReference(filtered); + + for (int row = 0; row < height; row++) + { + int sourceRowOffset = ((row + Border) * sourceStride) + Border; + int filteredRowOffset = row * width; + int coefficientRowOffset = bufferOrigin + (row * bufferStride); + int column = 0; + nuint vectorCount = Numerics.Vector256Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector256.Count) + { + Vector256 factors = CrossSum(blendFactors, coefficientRowOffset + column, bufferStride, vector); + Vector256 means = CrossSum(localMeans, coefficientRowOffset + column, bufferStride, vector); + Vector128 packed = Vector128.LoadUnsafe(ref sourceBase, (nuint)(sourceRowOffset + column)); + Vector256 samples = Vector256.WidenLower(Vector256.Create(packed, Vector128.Zero)).AsInt32(); + Vector256 values = Vector256.ShiftRightArithmetic((factors * samples) + means + rounding, roundingBits); + values.StoreUnsafe(ref filteredBase, (nuint)(filteredRowOffset + column)); + } + + for (; column < width; column++) + { + int factors = CrossSum(blendFactors, coefficientRowOffset + column, bufferStride); + int means = CrossSum(localMeans, coefficientRowOffset + column, bufferStride); + int value = (factors * source[sourceRowOffset + column]) + means; + filtered[filteredRowOffset + column] = RoundPowerOfTwo(value, roundingBits); + } + } + } + + /// + /// Produces the radius-one filtered values in four-sample cross-platform batches. + /// + /// The bordered processing-unit source rectangle. + /// The number of samples between source rows. + /// The processing-unit width in samples. + /// The processing-unit height in samples. + /// The padded coefficient-buffer row stride. + /// The local sample blend factors. + /// The scaled local means. + /// The destination fixed-point filtered values. + /// The overload-selection value. + private static void CalculateRadiusOneFilter( + ReadOnlySpan source, + int sourceStride, + int width, + int height, + int bufferStride, + ReadOnlySpan blendFactors, + ReadOnlySpan localMeans, + Span filtered, + Vector128 vector) + { + int bufferOrigin = (Border + 1) * (bufferStride + 1); + int roundingBits = SelfGuidedBits + 5 - RestorationBits; + Vector128 rounding = Vector128.Create(1 << (roundingBits - 1)); + ref ushort sourceBase = ref MemoryMarshal.GetReference(source); + ref int filteredBase = ref MemoryMarshal.GetReference(filtered); + + for (int row = 0; row < height; row++) + { + int sourceRowOffset = ((row + Border) * sourceStride) + Border; + int filteredRowOffset = row * width; + int coefficientRowOffset = bufferOrigin + (row * bufferStride); + int column = 0; + nuint vectorCount = Numerics.Vector128Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector128.Count) + { + Vector128 factors = CrossSum(blendFactors, coefficientRowOffset + column, bufferStride, vector); + Vector128 means = CrossSum(localMeans, coefficientRowOffset + column, bufferStride, vector); + ref ushort sourceReference = ref Unsafe.Add(ref sourceBase, sourceRowOffset + column); + Vector64 packed = Unsafe.As>(ref sourceReference); + Vector128 samples = Vector128.WidenLower(Vector128.Create(packed, Vector64.Zero)).AsInt32(); + Vector128 values = Vector128.ShiftRightArithmetic((factors * samples) + means + rounding, roundingBits); + values.StoreUnsafe(ref filteredBase, (nuint)(filteredRowOffset + column)); + } + + for (; column < width; column++) + { + int factors = CrossSum(blendFactors, coefficientRowOffset + column, bufferStride); + int means = CrossSum(localMeans, coefficientRowOffset + column, bufferStride); + int value = (factors * source[sourceRowOffset + column]) + means; + filtered[filteredRowOffset + column] = RoundPowerOfTwo(value, roundingBits); + } + } + } + + /// + /// Calculates eight radius-one weighted cross sums. + /// + /// The coefficient buffer. + /// The first center coefficient. + /// The coefficient-buffer row stride. + /// The overload-selection value. + /// The weighted cross sums. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector256 CrossSum(ReadOnlySpan buffer, int offset, int stride, Vector256 vector) + { + ref int bufferBase = ref MemoryMarshal.GetReference(buffer); + Vector256 topLeft = Vector256.LoadUnsafe(ref bufferBase, (nuint)(offset - stride - 1)); + Vector256 top = Vector256.LoadUnsafe(ref bufferBase, (nuint)(offset - stride)); + Vector256 topRight = Vector256.LoadUnsafe(ref bufferBase, (nuint)(offset - stride + 1)); + Vector256 left = Vector256.LoadUnsafe(ref bufferBase, (nuint)(offset - 1)); + Vector256 center = Vector256.LoadUnsafe(ref bufferBase, (nuint)offset); + Vector256 right = Vector256.LoadUnsafe(ref bufferBase, (nuint)(offset + 1)); + Vector256 bottomLeft = Vector256.LoadUnsafe(ref bufferBase, (nuint)(offset + stride - 1)); + Vector256 bottom = Vector256.LoadUnsafe(ref bufferBase, (nuint)(offset + stride)); + Vector256 bottomRight = Vector256.LoadUnsafe(ref bufferBase, (nuint)(offset + stride + 1)); + Vector256 corners = topLeft + topRight + bottomLeft + bottomRight; + Vector256 remainder = left + top + center + right + bottom; + return Vector256.ShiftLeft(corners + remainder, 2) - corners; + } + + /// + /// Calculates four radius-one weighted cross sums. + /// + /// The coefficient buffer. + /// The first center coefficient. + /// The coefficient-buffer row stride. + /// The overload-selection value. + /// The weighted cross sums. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector128 CrossSum(ReadOnlySpan buffer, int offset, int stride, Vector128 vector) + { + ref int bufferBase = ref MemoryMarshal.GetReference(buffer); + Vector128 topLeft = Vector128.LoadUnsafe(ref bufferBase, (nuint)(offset - stride - 1)); + Vector128 top = Vector128.LoadUnsafe(ref bufferBase, (nuint)(offset - stride)); + Vector128 topRight = Vector128.LoadUnsafe(ref bufferBase, (nuint)(offset - stride + 1)); + Vector128 left = Vector128.LoadUnsafe(ref bufferBase, (nuint)(offset - 1)); + Vector128 center = Vector128.LoadUnsafe(ref bufferBase, (nuint)offset); + Vector128 right = Vector128.LoadUnsafe(ref bufferBase, (nuint)(offset + 1)); + Vector128 bottomLeft = Vector128.LoadUnsafe(ref bufferBase, (nuint)(offset + stride - 1)); + Vector128 bottom = Vector128.LoadUnsafe(ref bufferBase, (nuint)(offset + stride)); + Vector128 bottomRight = Vector128.LoadUnsafe(ref bufferBase, (nuint)(offset + stride + 1)); + Vector128 corners = topLeft + topRight + bottomLeft + bottomRight; + Vector128 remainder = left + top + center + right + bottom; + return Vector128.ShiftLeft(corners + remainder, 2) - corners; + } + + /// + /// Calculates eight radius-two weighted cross sums from the required coefficient rows. + /// + /// The coefficient buffer. + /// The first center coefficient. + /// The coefficient-buffer row stride. + /// The destination row selecting the even or odd kernel. + /// The overload-selection value. + /// The weighted cross sums. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector256 CrossSum(ReadOnlySpan buffer, int offset, int stride, int row, Vector256 vector) + { + ref int bufferBase = ref MemoryMarshal.GetReference(buffer); + if ((row & 1) != 0) + { + Vector256 left = Vector256.LoadUnsafe(ref bufferBase, (nuint)(offset - 1)); + Vector256 center = Vector256.LoadUnsafe(ref bufferBase, (nuint)offset); + Vector256 right = Vector256.LoadUnsafe(ref bufferBase, (nuint)(offset + 1)); + Vector256 combined = left + center + right; + return Vector256.ShiftLeft(combined, 2) + combined + center; + } + + Vector256 topLeft = Vector256.LoadUnsafe(ref bufferBase, (nuint)(offset - stride - 1)); + Vector256 top = Vector256.LoadUnsafe(ref bufferBase, (nuint)(offset - stride)); + Vector256 topRight = Vector256.LoadUnsafe(ref bufferBase, (nuint)(offset - stride + 1)); + Vector256 bottomLeft = Vector256.LoadUnsafe(ref bufferBase, (nuint)(offset + stride - 1)); + Vector256 bottom = Vector256.LoadUnsafe(ref bufferBase, (nuint)(offset + stride)); + Vector256 bottomRight = Vector256.LoadUnsafe(ref bufferBase, (nuint)(offset + stride + 1)); + Vector256 centers = top + bottom; + Vector256 combinedRows = topLeft + topRight + bottomLeft + bottomRight + centers; + return Vector256.ShiftLeft(combinedRows, 2) + combinedRows + centers; + } + + /// + /// Calculates four radius-two weighted cross sums from the required coefficient rows. + /// + /// The coefficient buffer. + /// The first center coefficient. + /// The coefficient-buffer row stride. + /// The destination row selecting the even or odd kernel. + /// The overload-selection value. + /// The weighted cross sums. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector128 CrossSum(ReadOnlySpan buffer, int offset, int stride, int row, Vector128 vector) + { + ref int bufferBase = ref MemoryMarshal.GetReference(buffer); + if ((row & 1) != 0) + { + Vector128 left = Vector128.LoadUnsafe(ref bufferBase, (nuint)(offset - 1)); + Vector128 center = Vector128.LoadUnsafe(ref bufferBase, (nuint)offset); + Vector128 right = Vector128.LoadUnsafe(ref bufferBase, (nuint)(offset + 1)); + Vector128 combined = left + center + right; + return Vector128.ShiftLeft(combined, 2) + combined + center; + } + + Vector128 topLeft = Vector128.LoadUnsafe(ref bufferBase, (nuint)(offset - stride - 1)); + Vector128 top = Vector128.LoadUnsafe(ref bufferBase, (nuint)(offset - stride)); + Vector128 topRight = Vector128.LoadUnsafe(ref bufferBase, (nuint)(offset - stride + 1)); + Vector128 bottomLeft = Vector128.LoadUnsafe(ref bufferBase, (nuint)(offset + stride - 1)); + Vector128 bottom = Vector128.LoadUnsafe(ref bufferBase, (nuint)(offset + stride)); + Vector128 bottomRight = Vector128.LoadUnsafe(ref bufferBase, (nuint)(offset + stride + 1)); + Vector128 centers = top + bottom; + Vector128 combinedRows = topLeft + topRight + bottomLeft + bottomRight + centers; + return Vector128.ShiftLeft(combinedRows, 2) + combinedRows + centers; + } + + /// + /// Calculates one scalar radius-one weighted cross sum. + /// + /// The coefficient buffer. + /// The center coefficient. + /// The coefficient-buffer row stride. + /// The weighted cross sum. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static int CrossSum(ReadOnlySpan buffer, int offset, int stride) + { + int corners = buffer[offset - stride - 1] + buffer[offset - stride + 1] + buffer[offset + stride - 1] + buffer[offset + stride + 1]; + int remainder = buffer[offset - 1] + buffer[offset - stride] + buffer[offset] + buffer[offset + 1] + buffer[offset + stride]; + return ((corners + remainder) << 2) - corners; + } + + /// + /// Calculates one scalar radius-two weighted cross sum. + /// + /// The coefficient buffer. + /// The center coefficient. + /// The coefficient-buffer row stride. + /// The destination row selecting the even or odd kernel. + /// The weighted cross sum. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static int CrossSum(ReadOnlySpan buffer, int offset, int stride, int row) + { + if ((row & 1) != 0) + { + int center = buffer[offset]; + int combined = buffer[offset - 1] + center + buffer[offset + 1]; + return (combined << 2) + combined + center; + } + + int centers = buffer[offset - stride] + buffer[offset + stride]; + int combinedRows = buffer[offset - stride - 1] + buffer[offset - stride + 1] + + buffer[offset + stride - 1] + buffer[offset + stride + 1] + centers; + + return (combinedRows << 2) + combinedRows + centers; + } + + /// + /// Decodes the transmitted projection coefficients for the active radius pair. + /// + /// The two selected filter radii. + /// The two transmitted projection coefficients. + /// The first decoded projection coefficient. + /// The second decoded projection coefficient. + private static void DecodeProjectionCoefficients(ReadOnlySpan radii, ReadOnlySpan transmitted, out int projection0, out int projection1) + { + if (radii[0] == 0) + { + projection0 = 0; + projection1 = (1 << ProjectionBits) - transmitted[1]; + } + else if (radii[1] == 0) + { + projection0 = transmitted[0]; + projection1 = 0; + } + else + { + projection0 = transmitted[0]; + projection1 = (1 << ProjectionBits) - projection0 - transmitted[1]; + } + } + + /// + /// Projects the two restored signals in eight-sample SIMD batches. + /// + /// The bordered processing-unit source rectangle. + /// The number of samples between source rows. + /// The destination processing-unit rectangle. + /// The number of samples between destination rows. + /// The processing-unit width in samples. + /// The processing-unit height in samples. + /// The encoded sample bit depth. + /// The two selected filter radii. + /// The first decoded projection coefficient. + /// The second decoded projection coefficient. + /// The first fixed-point restored signal. + /// The second fixed-point restored signal. + /// The overload-selection value. + private static void Project( + ReadOnlySpan source, + int sourceStride, + Span destination, + int destinationStride, + int width, + int height, + int bitDepth, + ReadOnlySpan radii, + int projection0, + int projection1, + ReadOnlySpan filtered0, + ReadOnlySpan filtered1, + Vector256 vector) + { + const int projectionShift = ProjectionBits + RestorationBits; + Vector256 projection0Vector = Vector256.Create(projection0); + Vector256 projection1Vector = Vector256.Create(projection1); + Vector256 rounding = Vector256.Create(1 << (projectionShift - 1)); + Vector256 maximumSample = Vector256.Create((1 << bitDepth) - 1); + ref ushort sourceBase = ref MemoryMarshal.GetReference(source); + ref ushort destinationBase = ref MemoryMarshal.GetReference(destination); + ref int filtered0Base = ref MemoryMarshal.GetReference(filtered0); + ref int filtered1Base = ref MemoryMarshal.GetReference(filtered1); + + for (int row = 0; row < height; row++) + { + int sourceRowOffset = ((row + Border) * sourceStride) + Border; + int destinationRowOffset = row * destinationStride; + int filteredRowOffset = row * width; + int column = 0; + nuint vectorCount = Numerics.Vector256Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector256.Count) + { + Vector128 packed = Vector128.LoadUnsafe(ref sourceBase, (nuint)(sourceRowOffset + column)); + Vector256 samples = Vector256.WidenLower(Vector256.Create(packed, Vector128.Zero)).AsInt32(); + Vector256 unfiltered = Vector256.ShiftLeft(samples, RestorationBits); + Vector256 projected = Vector256.ShiftLeft(unfiltered, ProjectionBits); + if (radii[0] > 0) + { + Vector256 restored = Vector256.LoadUnsafe(ref filtered0Base, (nuint)(filteredRowOffset + column)); + projected += projection0Vector * (restored - unfiltered); + } + + if (radii[1] > 0) + { + Vector256 restored = Vector256.LoadUnsafe(ref filtered1Base, (nuint)(filteredRowOffset + column)); + projected += projection1Vector * (restored - unfiltered); + } + + Vector256 result = Vector256.ShiftRightArithmetic(projected + rounding, projectionShift); + result = Vector256.Min(Vector256.Max(result, Vector256.Zero), maximumSample); + + // Narrowing the result with a zero upper vector places the eight ordered samples in the lower 128 bits, + // which can be stored directly without an ISA-specific lane permutation. + Vector128 narrowed = Vector256.Narrow(result.AsUInt32(), Vector256.Zero).GetLower(); + narrowed.StoreUnsafe(ref destinationBase, (nuint)(destinationRowOffset + column)); + } + + for (; column < width; column++) + { + int filteredOffset = filteredRowOffset + column; + int sample = source[sourceRowOffset + column]; + int restored0 = radii[0] > 0 ? filtered0[filteredOffset] : 0; + int restored1 = radii[1] > 0 ? filtered1[filteredOffset] : 0; + destination[destinationRowOffset + column] = ProjectSample( + sample, + bitDepth, + radii, + projection0, + projection1, + restored0, + restored1); + } + } + } + + /// + /// Projects the two restored signals in four-sample cross-platform batches. + /// + /// The bordered processing-unit source rectangle. + /// The number of samples between source rows. + /// The destination processing-unit rectangle. + /// The number of samples between destination rows. + /// The processing-unit width in samples. + /// The processing-unit height in samples. + /// The encoded sample bit depth. + /// The two selected filter radii. + /// The first decoded projection coefficient. + /// The second decoded projection coefficient. + /// The first fixed-point restored signal. + /// The second fixed-point restored signal. + /// The overload-selection value. + private static void Project( + ReadOnlySpan source, + int sourceStride, + Span destination, + int destinationStride, + int width, + int height, + int bitDepth, + ReadOnlySpan radii, + int projection0, + int projection1, + ReadOnlySpan filtered0, + ReadOnlySpan filtered1, + Vector128 vector) + { + const int projectionShift = ProjectionBits + RestorationBits; + Vector128 projection0Vector = Vector128.Create(projection0); + Vector128 projection1Vector = Vector128.Create(projection1); + Vector128 rounding = Vector128.Create(1 << (projectionShift - 1)); + Vector128 maximumSample = Vector128.Create((1 << bitDepth) - 1); + ref ushort sourceBase = ref MemoryMarshal.GetReference(source); + ref ushort destinationBase = ref MemoryMarshal.GetReference(destination); + ref int filtered0Base = ref MemoryMarshal.GetReference(filtered0); + ref int filtered1Base = ref MemoryMarshal.GetReference(filtered1); + + for (int row = 0; row < height; row++) + { + int sourceRowOffset = ((row + Border) * sourceStride) + Border; + int destinationRowOffset = row * destinationStride; + int filteredRowOffset = row * width; + int column = 0; + nuint vectorCount = Numerics.Vector128Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector128.Count) + { + ref ushort sourceReference = ref Unsafe.Add(ref sourceBase, sourceRowOffset + column); + Vector64 packed = Unsafe.As>(ref sourceReference); + Vector128 samples = Vector128.WidenLower(Vector128.Create(packed, Vector64.Zero)).AsInt32(); + Vector128 unfiltered = Vector128.ShiftLeft(samples, RestorationBits); + Vector128 projected = Vector128.ShiftLeft(unfiltered, ProjectionBits); + if (radii[0] > 0) + { + Vector128 restored = Vector128.LoadUnsafe(ref filtered0Base, (nuint)(filteredRowOffset + column)); + projected += projection0Vector * (restored - unfiltered); + } + + if (radii[1] > 0) + { + Vector128 restored = Vector128.LoadUnsafe(ref filtered1Base, (nuint)(filteredRowOffset + column)); + projected += projection1Vector * (restored - unfiltered); + } + + Vector128 result = Vector128.ShiftRightArithmetic(projected + rounding, projectionShift); + result = Vector128.Min(Vector128.Max(result, Vector128.Zero), maximumSample); + Vector64 narrowed = Vector128.Narrow(result.AsUInt32(), Vector128.Zero).GetLower(); + ref ushort destinationReference = ref Unsafe.Add(ref destinationBase, destinationRowOffset + column); + Unsafe.As>(ref destinationReference) = narrowed; + } + + for (; column < width; column++) + { + int filteredOffset = filteredRowOffset + column; + int sample = source[sourceRowOffset + column]; + int restored0 = radii[0] > 0 ? filtered0[filteredOffset] : 0; + int restored1 = radii[1] > 0 ? filtered1[filteredOffset] : 0; + destination[destinationRowOffset + column] = ProjectSample( + sample, + bitDepth, + radii, + projection0, + projection1, + restored0, + restored1); + } + } + } + + /// + /// Projects one scalar remainder sample from the active restored signals. + /// + /// The unfiltered source sample. + /// The encoded sample bit depth. + /// The two selected filter radii. + /// The first decoded projection coefficient. + /// The second decoded projection coefficient. + /// The first fixed-point restored value. + /// The second fixed-point restored value. + /// The clipped projected sample. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static ushort ProjectSample( + int sample, + int bitDepth, + ReadOnlySpan radii, + int projection0, + int projection1, + int filtered0, + int filtered1) + { + int unfiltered = sample << RestorationBits; + int projected = unfiltered << ProjectionBits; + if (radii[0] > 0) + { + projected += projection0 * (filtered0 - unfiltered); + } + + if (radii[1] > 0) + { + projected += projection1 * (filtered1 - unfiltered); + } + + return (ushort)Av1Math.Clip3(0, (1 << bitDepth) - 1, RoundPowerOfTwo(projected, ProjectionBits + RestorationBits)); + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/LoopRestoration/Av1SelfGuidedFilter.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/LoopRestoration/Av1SelfGuidedFilter.cs new file mode 100644 index 0000000000..bb76c805de --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/LoopRestoration/Av1SelfGuidedFilter.cs @@ -0,0 +1,560 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.LoopRestoration; + +/// +/// Applies the normative AV1 self-guided restoration filter and projection. +/// +/// +/// The accelerated paths process adjacent output columns in SIMD lanes and use the caller-provided scratch span for +/// filtered samples, local coefficients, and padded integral images. The 256-bit and 128-bit paths use portable +/// vector operations, with AVX2 selected locally for the prefix scan and variance-to-blend lookup when available. +/// The scalar path uses the same fixed-point units and scratch partition. +/// +internal static partial class Av1SelfGuidedFilter +{ + /// + /// The number of source samples required on every side of a filtered processing unit. + /// + private const int Border = 3; + + /// + /// The number of fractional bits retained by each self-guided filter result. + /// + private const int RestorationBits = 4; + + /// + /// The number of fractional bits used by the decoded projection coefficients. + /// + private const int ProjectionBits = 7; + + /// + /// The number of fractional bits used by the local self-guided blend factors. + /// + private const int SelfGuidedBits = 8; + + /// + /// The number of fractional bits used by the self-guided scale table. + /// + private const int ScaleBits = 20; + + /// + /// The number of fractional bits used by reciprocal window-area values. + /// + private const int ReciprocalBits = 12; + + /// + /// The base-two exponent used to align work-buffer rows for every supported vector width. + /// + private const int BufferAlignmentLog2 = 4; + + /// + /// The extra columns separating integral-image rows to avoid adverse cache aliasing. + /// + private const int BufferPadding = 16; + + /// + /// The complete fixed-point self-guided blend range. + /// + private const int SelfGuidedScale = 1 << SelfGuidedBits; + + /// + /// Gets the radii selected by each of the sixteen self-guided parameter sets. + /// + private static ReadOnlySpan ParameterRadii => + [ + 2, 1, 2, 1, 2, 1, 2, 1, + 2, 1, 2, 1, 2, 1, 2, 1, + 2, 1, 2, 1, 0, 1, 0, 1, + 0, 1, 0, 1, 2, 0, 2, 0, + ]; + + /// + /// Gets the variance scales selected by each of the sixteen self-guided parameter sets. + /// + private static ReadOnlySpan ParameterScales => + [ + 140, 3236, 112, 2158, 93, 1618, 80, 1438, + 70, 1295, 58, 1177, 47, 1079, 37, 996, + 30, 925, 25, 863, -1, 2589, -1, 1618, + -1, 1177, -1, 925, 56, -1, 22, -1, + ]; + + /// + /// Gets the table mapping a bounded variance measure to its fixed-point local sample blend factor. + /// + private static ReadOnlySpan XByXPlusOne => + [ + 1, 128, 171, 192, 205, 213, 219, 224, 228, 230, 233, 235, 236, 238, 239, + 240, 241, 242, 243, 243, 244, 244, 245, 245, 246, 246, 247, 247, 247, 247, + 248, 248, 248, 248, 249, 249, 249, 249, 249, 250, 250, 250, 250, 250, 250, + 250, 251, 251, 251, 251, 251, 251, 251, 251, 251, 251, 252, 252, 252, 252, + 252, 252, 252, 252, 252, 252, 252, 252, 252, 252, 252, 252, 252, 253, 253, + 253, 253, 253, 253, 253, 253, 253, 253, 253, 253, 253, 253, 253, 253, 253, + 253, 253, 253, 253, 253, 253, 253, 253, 253, 253, 253, 253, 254, 254, 254, + 254, 254, 254, 254, 254, 254, 254, 254, 254, 254, 254, 254, 254, 254, 254, + 254, 254, 254, 254, 254, 254, 254, 254, 254, 254, 254, 254, 254, 254, 254, + 254, 254, 254, 254, 254, 254, 254, 254, 254, 254, 254, 254, 254, 254, 254, + 254, 254, 254, 254, 254, 254, 254, 254, 254, 254, 254, 254, 254, 254, 254, + 254, 254, 254, 254, 254, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, + 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, + 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, + 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, + 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, + 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, + 256, + ]; + + /// + /// Gets fixed-point reciprocals for every supported square-window area. + /// + private static ReadOnlySpan OneByX => + [ + 4096, 2048, 1365, 1024, 819, 683, 585, 512, 455, 410, 372, 341, 315, + 293, 273, 256, 241, 228, 216, 205, 195, 186, 178, 171, 164, + ]; + + /// + /// Gets the number of integer samples required to filter one processing unit. + /// + /// The destination processing-unit width. + /// The destination processing-unit height. + /// The required scratch-span length. + public static int GetScratchLength(int width, int height) + { + int filteredLength = width * height; + int bufferLength = GetBufferLength(width, height); + return (filteredLength * 2) + (bufferLength * 4); + } + + /// + /// Filters one processing unit from a source rectangle containing the required three-sample borders. + /// + /// The source rectangle beginning three samples above and left of the processing unit. + /// The number of samples between source rows. + /// The destination span beginning at the restored processing-unit origin. + /// The number of samples between destination rows. + /// The processing-unit width in plane samples. + /// The processing-unit height in plane samples. + /// The encoded sample bit depth. + /// The decoded self-guided parameter-set index. + /// The two transmitted projection coefficients. + /// Integer storage sized according to . + public static void FilterBlock( + ReadOnlySpan source, + int sourceStride, + Span destination, + int destinationStride, + int width, + int height, + int bitDepth, + int parameterSetIndex, + ReadOnlySpan projectionCoefficients, + Span scratch) + { + // The closed vector overloads share the same scratch layout and fixed-point equations. Dispatch is based on + // portable vector width; ISA-specific acceleration is confined to the individual operation that requires it. + if (Vector256.IsHardwareAccelerated) + { + FilterBlock( + source, + sourceStride, + destination, + destinationStride, + width, + height, + bitDepth, + parameterSetIndex, + projectionCoefficients, + scratch, + Vector256.Zero); + + return; + } + + if (Vector128.IsHardwareAccelerated) + { + FilterBlock( + source, + sourceStride, + destination, + destinationStride, + width, + height, + bitDepth, + parameterSetIndex, + projectionCoefficients, + scratch, + Vector128.Zero); + + return; + } + + int filteredLength = width * height; + Span filtered0 = scratch[..filteredLength]; + Span filtered1 = scratch.Slice(filteredLength, filteredLength); + int coefficientLength = GetCoefficientBufferLength(width, height); + Span blendFactors = scratch.Slice(filteredLength * 2, coefficientLength); + Span localMeans = scratch.Slice((filteredLength * 2) + coefficientLength, coefficientLength); + + int parameterOffset = parameterSetIndex * 2; + ReadOnlySpan radii = ParameterRadii.Slice(parameterOffset, 2); + ReadOnlySpan scales = ParameterScales.Slice(parameterOffset, 2); + if (radii[0] > 0) + { + CalculateIntermediateCoefficients( + source, + sourceStride, + width, + height, + bitDepth, + radii[0], + scales[0], + skipAlternateRows: true, + blendFactors, + localMeans); + + CalculateRadiusTwoFilter(source, sourceStride, width, height, blendFactors, localMeans, filtered0); + } + + if (radii[1] > 0) + { + CalculateIntermediateCoefficients( + source, + sourceStride, + width, + height, + bitDepth, + radii[1], + scales[1], + skipAlternateRows: false, + blendFactors, + localMeans); + + CalculateRadiusOneFilter(source, sourceStride, width, height, blendFactors, localMeans, filtered1); + } + + int projection0; + int projection1; + if (radii[0] == 0) + { + projection0 = 0; + projection1 = (1 << ProjectionBits) - projectionCoefficients[1]; + } + else if (radii[1] == 0) + { + projection0 = projectionCoefficients[0]; + projection1 = 0; + } + else + { + projection0 = projectionCoefficients[0]; + projection1 = (1 << ProjectionBits) - projection0 - projectionCoefficients[1]; + } + + int maximumSample = (1 << bitDepth) - 1; + for (int row = 0; row < height; row++) + { + int sourceRowOffset = (row + Border) * sourceStride; + int destinationRowOffset = row * destinationStride; + int filteredRowOffset = row * width; + for (int column = 0; column < width; column++) + { + int filteredOffset = filteredRowOffset + column; + int unfiltered = source[sourceRowOffset + column + Border] << RestorationBits; + int projected = unfiltered << ProjectionBits; + if (radii[0] > 0) + { + projected += projection0 * (filtered0[filteredOffset] - unfiltered); + } + + if (radii[1] > 0) + { + projected += projection1 * (filtered1[filteredOffset] - unfiltered); + } + + destination[destinationRowOffset + column] = + (ushort)Av1Math.Clip3( + 0, + maximumSample, + RoundPowerOfTwo(projected, ProjectionBits + RestorationBits)); + } + } + } + + /// + /// Calculates the local blend factor and mean for the requested filter radius. + /// + /// The bordered processing-unit source rectangle. + /// The number of samples between source rows. + /// The processing-unit width in samples. + /// The processing-unit height in samples. + /// The encoded sample bit depth. + /// The square-window radius. + /// The variance scale for the selected parameter set and radius. + /// Whether only the rows consumed by the radius-two filter are calculated. + /// The destination buffer for local sample blend factors. + /// The destination buffer for scaled local means. + private static void CalculateIntermediateCoefficients( + ReadOnlySpan source, + int sourceStride, + int width, + int height, + int bitDepth, + int radius, + int scale, + bool skipAlternateRows, + Span blendFactors, + Span localMeans) + { + int bufferStride = width + 2; + int bufferOrigin = bufferStride + 1; + int windowDiameter = (radius * 2) + 1; + int windowArea = windowDiameter * windowDiameter; + int rowStep = skipAlternateRows ? 2 : 1; + ReadOnlySpan xByXPlusOne = XByXPlusOne; + ReadOnlySpan oneByX = OneByX; + + for (int row = -1; row < height + 1; row += rowStep) + { + int centerY = row + Border; + int centerX = Border - 1; + int sum = 0; + int squareSum = 0; + for (int windowY = centerY - radius; windowY <= centerY + radius; windowY++) + { + int sourceRowOffset = windowY * sourceStride; + for (int windowX = centerX - radius; windowX <= centerX + radius; windowX++) + { + int sample = source[sourceRowOffset + windowX]; + sum += sample; + squareSum += sample * sample; + } + } + + for (int column = -1; column < width + 1; column++) + { + if (column > -1) + { + int departingX = centerX - radius; + int arrivingX = centerX + radius + 1; + for (int windowY = centerY - radius; windowY <= centerY + radius; windowY++) + { + int sourceRowOffset = windowY * sourceStride; + int departingSample = source[sourceRowOffset + departingX]; + int arrivingSample = source[sourceRowOffset + arrivingX]; + sum += arrivingSample - departingSample; + squareSum += (arrivingSample * arrivingSample) - (departingSample * departingSample); + } + + centerX++; + } + + int normalizedSquareSum = RoundPowerOfTwo(squareSum, 2 * (bitDepth - 8)); + int normalizedSum = RoundPowerOfTwo(sum, bitDepth - 8); + uint squareOfSum = (uint)normalizedSum * (uint)normalizedSum; + uint scaledSquareSum = (uint)normalizedSquareSum * (uint)windowArea; + + // High-bit-depth normalization can round a nearly flat window's squared mean + // above its mean square. AV1 saturates that rounding artefact to zero variance. + uint variance = scaledSquareSum < squareOfSum ? 0 : scaledSquareSum - squareOfSum; + uint varianceIndex = RoundPowerOfTwo(variance * (uint)scale, ScaleBits); + int coefficientOffset = bufferOrigin + (row * bufferStride) + column; + int blendFactor = xByXPlusOne[(int)Math.Min(varianceIndex, 255U)]; + blendFactors[coefficientOffset] = blendFactor; + + // The zero-variance table entry is deliberately one rather than zero. This keeps + // the complementary factor below 256 and the scaled mean inside its proven range. + uint meanProduct = + (uint)(SelfGuidedScale - blendFactor) * (uint)sum * oneByX[windowArea - 1]; + + localMeans[coefficientOffset] = (int)RoundPowerOfTwo(meanProduct, ReciprocalBits); + } + } + } + + /// + /// Produces the radius-two filtered values from alternate coefficient rows. + /// + /// The bordered processing-unit source rectangle. + /// The number of samples between source rows. + /// The processing-unit width in samples. + /// The processing-unit height in samples. + /// The local sample blend factors. + /// The scaled local means. + /// The destination fixed-point filtered values. + private static void CalculateRadiusTwoFilter( + ReadOnlySpan source, + int sourceStride, + int width, + int height, + ReadOnlySpan blendFactors, + ReadOnlySpan localMeans, + Span filtered) + { + int bufferStride = width + 2; + int bufferOrigin = bufferStride + 1; + for (int row = 0; row < height; row++) + { + int sourceRowOffset = (row + Border) * sourceStride; + int filteredRowOffset = row * width; + for (int column = 0; column < width; column++) + { + int coefficientOffset = bufferOrigin + (row * bufferStride) + column; + int blendFactor; + int localMean; + int roundingBits; + if ((row & 1) == 0) + { + blendFactor = + (6 * (blendFactors[coefficientOffset - bufferStride] + blendFactors[coefficientOffset + bufferStride])) + + (5 * ( + blendFactors[coefficientOffset - bufferStride - 1] + + blendFactors[coefficientOffset - bufferStride + 1] + + blendFactors[coefficientOffset + bufferStride - 1] + + blendFactors[coefficientOffset + bufferStride + 1])); + + localMean = + (6 * (localMeans[coefficientOffset - bufferStride] + localMeans[coefficientOffset + bufferStride])) + + (5 * ( + localMeans[coefficientOffset - bufferStride - 1] + + localMeans[coefficientOffset - bufferStride + 1] + + localMeans[coefficientOffset + bufferStride - 1] + + localMeans[coefficientOffset + bufferStride + 1])); + + roundingBits = SelfGuidedBits + 5 - RestorationBits; + } + else + { + blendFactor = + (6 * blendFactors[coefficientOffset]) + + (5 * (blendFactors[coefficientOffset - 1] + blendFactors[coefficientOffset + 1])); + + localMean = + (6 * localMeans[coefficientOffset]) + + (5 * (localMeans[coefficientOffset - 1] + localMeans[coefficientOffset + 1])); + + roundingBits = SelfGuidedBits + 4 - RestorationBits; + } + + int value = + (blendFactor * source[sourceRowOffset + column + Border]) + localMean; + + filtered[filteredRowOffset + column] = RoundPowerOfTwo(value, roundingBits); + } + } + } + + /// + /// Produces the radius-one filtered values from the complete coefficient grid. + /// + /// The bordered processing-unit source rectangle. + /// The number of samples between source rows. + /// The processing-unit width in samples. + /// The processing-unit height in samples. + /// The local sample blend factors. + /// The scaled local means. + /// The destination fixed-point filtered values. + private static void CalculateRadiusOneFilter( + ReadOnlySpan source, + int sourceStride, + int width, + int height, + ReadOnlySpan blendFactors, + ReadOnlySpan localMeans, + Span filtered) + { + int bufferStride = width + 2; + int bufferOrigin = bufferStride + 1; + int roundingBits = SelfGuidedBits + 5 - RestorationBits; + for (int row = 0; row < height; row++) + { + int sourceRowOffset = (row + Border) * sourceStride; + int filteredRowOffset = row * width; + for (int column = 0; column < width; column++) + { + int coefficientOffset = bufferOrigin + (row * bufferStride) + column; + int blendFactor = + (4 * ( + blendFactors[coefficientOffset] + + blendFactors[coefficientOffset - 1] + + blendFactors[coefficientOffset + 1] + + blendFactors[coefficientOffset - bufferStride] + + blendFactors[coefficientOffset + bufferStride])) + + (3 * ( + blendFactors[coefficientOffset - bufferStride - 1] + + blendFactors[coefficientOffset - bufferStride + 1] + + blendFactors[coefficientOffset + bufferStride - 1] + + blendFactors[coefficientOffset + bufferStride + 1])); + + int localMean = + (4 * ( + localMeans[coefficientOffset] + + localMeans[coefficientOffset - 1] + + localMeans[coefficientOffset + 1] + + localMeans[coefficientOffset - bufferStride] + + localMeans[coefficientOffset + bufferStride])) + + (3 * ( + localMeans[coefficientOffset - bufferStride - 1] + + localMeans[coefficientOffset - bufferStride + 1] + + localMeans[coefficientOffset + bufferStride - 1] + + localMeans[coefficientOffset + bufferStride + 1])); + + int value = + (blendFactor * source[sourceRowOffset + column + Border]) + localMean; + + filtered[filteredRowOffset + column] = RoundPowerOfTwo(value, roundingBits); + } + } + } + + /// + /// Gets the number of entries in one bordered intermediate-coefficient buffer. + /// + /// The processing-unit width. + /// The processing-unit height. + /// The number of required integer entries. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static int GetCoefficientBufferLength(int width, int height) => (width + 2) * (height + 2); + + /// + /// Gets the padded row stride shared by coefficient and integral-image buffers. + /// + /// The filtered processing-unit width. + /// The aligned number of integers reserved for each work-buffer row. + private static int GetBufferStride(int width) + => Av1Math.AlignPowerOf2(width + (Border * 2) + BufferPadding, BufferAlignmentLog2); + + /// + /// Gets the number of integers reserved for one padded work buffer. + /// + /// The filtered processing-unit width. + /// The filtered processing-unit height. + /// The required work-buffer length. + private static int GetBufferLength(int width, int height) + => GetBufferStride(width) * (height + (Border * 2) + 1); + + /// + /// Rounds a signed fixed-point value to the requested lower precision. + /// + /// The signed fixed-point value. + /// The number of low bits to discard. + /// The rounded signed value. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static int RoundPowerOfTwo(int value, int bitCount) + => bitCount == 0 ? value : (value + (1 << (bitCount - 1))) >> bitCount; + + /// + /// Rounds an unsigned fixed-point value to the requested lower precision. + /// + /// The unsigned fixed-point value. + /// The number of low bits to discard. + /// The rounded unsigned value. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static uint RoundPowerOfTwo(uint value, int bitCount) + => bitCount == 0 ? value : (value + (1U << (bitCount - 1))) >> bitCount; +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/LoopRestoration/Av1WienerFilter.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/LoopRestoration/Av1WienerFilter.cs new file mode 100644 index 0000000000..b00f5be9b8 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/LoopRestoration/Av1WienerFilter.cs @@ -0,0 +1,199 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.InteropServices; +using System.Runtime.Intrinsics; +using SixLabors.ImageSharp.Common.Helpers; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.LoopRestoration; + +/// +/// Applies the normative separable Wiener filter used by AV1 loop restoration. +/// +/// +/// The horizontal pass consumes one contiguous eight-tap window at a time. SIMD pairwise multiply-add widens adjacent +/// unsigned-sample and signed-coefficient products into four 32-bit partial sums, which are reduced before normative +/// rounding and clipping. The resulting caller-owned intermediate plane supplies contiguous columns to the vertical +/// pass without allocating per stripe. +/// +internal static class Av1WienerFilter +{ + /// + /// The number of coefficients in the padded Wiener convolution kernel. + /// + private const int FilterTapCount = 8; + + /// + /// The number of independent symmetric coefficients transmitted for each filter direction. + /// + private const int TransmittedCoefficientCount = 3; + + /// + /// The number of fractional bits in the Wiener filter coefficients. + /// + private const int FilterBits = 7; + + /// + /// The default first-pass rounding shift. + /// + private const int InitialHorizontalRoundBits = 3; + + /// + /// The number of intermediate rows required beyond the destination stripe height. + /// + private const int IntermediateRowExtension = FilterTapCount - 1; + + /// + /// Gets the number of intermediate samples required to filter a stripe. + /// + /// The destination stripe width. + /// The destination stripe height. + /// The required scratch-span length. + public static int GetScratchLength(int width, int height) => width * (height + IntermediateRowExtension); + + /// + /// Filters one restoration stripe from a source rectangle containing the required three-sample borders. + /// + /// The source rectangle beginning three samples above and left of the destination stripe. + /// The number of samples between source rows. + /// The destination span beginning at the restored stripe origin. + /// The number of samples between destination rows. + /// The stripe width in plane samples. + /// The stripe height in plane samples. + /// The encoded sample bit depth. + /// The three transmitted horizontal coefficients. + /// The three transmitted vertical coefficients. + /// Intermediate sample storage sized according to . + public static void FilterStripe( + ReadOnlySpan source, + int sourceStride, + Span destination, + int destinationStride, + int width, + int height, + int bitDepth, + ReadOnlySpan horizontalCoefficients, + ReadOnlySpan verticalCoefficients, + Span scratch) + { + Span horizontalFilter = stackalloc short[FilterTapCount]; + Span verticalFilter = stackalloc short[FilterTapCount]; + PopulateFilter(horizontalCoefficients, horizontalFilter); + PopulateFilter(verticalCoefficients, verticalFilter); + + int horizontalRoundBits = InitialHorizontalRoundBits; + int intermediateBitCount = bitDepth + FilterBits - horizontalRoundBits + 2; + if (intermediateBitCount > 16) + { + // Twelve-bit input would otherwise exceed the unsigned 16-bit intermediate used by + // the normative two-pass convolution, so AV1 transfers those excess bits to pass two. + horizontalRoundBits += intermediateBitCount - 16; + } + + int verticalRoundBits = (FilterBits * 2) - horizontalRoundBits; + int intermediateMaximum = (1 << (bitDepth + 1 + FilterBits - horizontalRoundBits)) - 1; + int intermediateHeight = height + IntermediateRowExtension; + int horizontalBias = 1 << (bitDepth + FilterBits - 1); + for (int row = 0; row < intermediateHeight; row++) + { + int sourceRowOffset = row * sourceStride; + int intermediateRowOffset = row * width; + for (int column = 0; column < width; column++) + { + int sourceOffset = sourceRowOffset + column; + int sum = DotProduct(source, sourceOffset, horizontalFilter); + + // The transmitted center coefficient excludes its implicit 128 contribution. + // Adding the unfiltered center sample here reconstructs the complete kernel. + sum += (source[sourceOffset + TransmittedCoefficientCount] << FilterBits) + horizontalBias; + int value = RoundPowerOfTwo(sum, horizontalRoundBits); + scratch[intermediateRowOffset + column] = (ushort)Av1Math.Clip3(0, intermediateMaximum, value); + } + } + + int maximumSample = (1 << bitDepth) - 1; + int verticalBias = 1 << (bitDepth + verticalRoundBits - 1); + for (int row = 0; row < height; row++) + { + int destinationRowOffset = row * destinationStride; + for (int column = 0; column < width; column++) + { + int sum = 0; + for (int tap = 0; tap < FilterTapCount; tap++) + { + sum += scratch[((row + tap) * width) + column] * verticalFilter[tap]; + } + + int center = scratch[((row + TransmittedCoefficientCount) * width) + column]; + sum += (center << FilterBits) - verticalBias; + destination[destinationRowOffset + column] = + (ushort)Av1Math.Clip3(0, maximumSample, RoundPowerOfTwo(sum, verticalRoundBits)); + } + } + } + + /// + /// Expands three transmitted symmetric coefficients into the padded eight-tap convolution kernel. + /// + /// The transmitted outer-to-inner coefficients. + /// The destination eight-tap kernel. + private static void PopulateFilter(ReadOnlySpan coefficients, Span filter) + { + int outer = coefficients[0]; + int middle = coefficients[1]; + int inner = coefficients[2]; + filter[0] = (short)outer; + filter[1] = (short)middle; + filter[2] = (short)inner; + filter[3] = (short)(-2 * (outer + middle + inner)); + filter[4] = (short)inner; + filter[5] = (short)middle; + filter[6] = (short)outer; + + // the reference decoder stores a seven-tap Wiener kernel in the shared eight-tap interpolation shape. + filter[7] = 0; + } + + /// + /// Computes one signed eight-tap horizontal filter product. + /// + /// The source rectangle containing the requested samples. + /// The first source sample consumed by the filter. + /// The eight signed filter coefficients. + /// The unrounded signed filter sum. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static int DotProduct(ReadOnlySpan source, int sourceOffset, ReadOnlySpan filter) + { + if (Vector128.IsHardwareAccelerated) + { + ref ushort sourceReference = ref MemoryMarshal.GetReference(source); + ref short filterReference = ref MemoryMarshal.GetReference(filter); + Vector128 samples = Vector128.LoadUnsafe(ref sourceReference, (nuint)sourceOffset).AsInt16(); + Vector128 coefficients = Vector128.LoadUnsafe(ref filterReference); + Vector128 pairSums = Vector128_.MultiplyAddAdjacent(samples, coefficients); + + // The shared helper provides the architecture-specific adjacent products; reducing its + // four 32-bit lanes scalarly avoids an additional platform-specific shuffle sequence. + return pairSums.GetElement(0) + pairSums.GetElement(1) + pairSums.GetElement(2) + pairSums.GetElement(3); + } + + int sum = 0; + for (int tap = 0; tap < FilterTapCount; tap++) + { + sum += source[sourceOffset + tap] * filter[tap]; + } + + return sum; + } + + /// + /// Rounds a signed fixed-point value to the requested lower precision. + /// + /// The signed fixed-point value. + /// The number of low bits to discard. + /// The rounded signed value. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static int RoundPowerOfTwo(int value, int bitCount) + => (value + (1 << (bitCount - 1))) >> bitCount; +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/Quantizers/Av1DeQuantizationContext.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Quantizers/Av1DeQuantizationContext.cs new file mode 100644 index 0000000000..5ecf17ad7b --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Quantizers/Av1DeQuantizationContext.cs @@ -0,0 +1,82 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.Quantizers; + +/// +/// Stores the AV1 DC and AC dequantization values for every segment and color plane in a frame. +/// +internal sealed class Av1DeQuantizationContext +{ + /// + /// The DC dequantization values indexed by segment and then plane. + /// + private InlineArray8> dcContent; + + /// + /// The AC dequantization values indexed by segment and then plane. + /// + private InlineArray8> acContent; + + /// + /// Initializes a new instance of the class from the frame's base quantizer, + /// segment adjustments, plane deltas, and coded bit depth. + /// + /// The sequence header that supplies the coded bit depth. + /// The frame header that supplies segmentation and quantization parameters. + public Av1DeQuantizationContext(ObuSequenceHeader sequenceHeader, ObuFrameHeader frameHeader) + { + Av1BitDepth bitDepth = sequenceHeader.ColorConfig.BitDepth; + for (int segmentId = 0; segmentId < Av1Constants.MaxSegmentCount; segmentId++) + { + int qindex = Av1QuantizationLookup.GetQIndex(frameHeader.SegmentationParameters, segmentId, frameHeader.QuantizationParameters.BaseQIndex); + + for (int plane = 0; plane < Av1Constants.MaxPlanes; plane++) + { + int dcDeltaQ = frameHeader.QuantizationParameters.DeltaQDc[plane]; + int acDeltaQ = frameHeader.QuantizationParameters.DeltaQAc[plane]; + + this.dcContent[segmentId][plane] = Av1QuantizationLookup.GetDcQuant(qindex, dcDeltaQ, bitDepth); + this.acContent[segmentId][plane] = Av1QuantizationLookup.GetAcQuant(qindex, acDeltaQ, bitDepth); + } + } + } + + /// + /// Gets the DC dequantization value for a segment and color plane. + /// + /// The zero-based AV1 segment identifier. + /// The color plane. + /// The DC dequantization value. + public short GetDc(int segmentId, Av1Plane plane) + => this.dcContent[segmentId][(int)plane]; + + /// + /// Gets the AC dequantization value for a segment and color plane. + /// + /// The zero-based AV1 segment identifier. + /// The color plane. + /// The AC dequantization value. + public short GetAc(int segmentId, Av1Plane plane) + => this.acContent[segmentId][(int)plane]; + + /// + /// Sets the AC dequantization value for a segment and color plane. + /// + /// The zero-based AV1 segment identifier. + /// The color plane. + /// The AC dequantization value. + public void SetAc(int segmentId, Av1Plane plane, short value) + => this.acContent[segmentId][(int)plane] = value; + + /// + /// Sets the DC dequantization value for a segment and color plane. + /// + /// The zero-based AV1 segment identifier. + /// The color plane. + /// The DC dequantization value. + public void SetDc(int segmentId, Av1Plane plane, short value) + => this.dcContent[segmentId][(int)plane] = value; +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/Quantizers/Av1ForwardQuantizer.Operator.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Quantizers/Av1ForwardQuantizer.Operator.cs new file mode 100644 index 0000000000..98fc6e96cd --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Quantizers/Av1ForwardQuantizer.Operator.cs @@ -0,0 +1,375 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.Quantizers; + +/// +/// Defines the arithmetic operators used by . +/// +internal static partial class Av1ForwardQuantizer +{ + /// + /// Defines one AV1 forward-quantization arithmetic contract across hardware widths. + /// + internal interface IForwardQuantizationOperator + { + /// + /// Quantizes four raster-order transform coefficients. + /// + /// The signed transform coefficients. + /// The positive rounding constant after transform-size scaling. + /// The Q16 reciprocal quantizer. + /// The Q3 reconstruction quantizer. + /// The transform-size quantization scale. + /// The signed reconstruction coefficients. + /// The signed entropy-coding coefficients. + public static abstract Vector128 Quantize( + Vector128 coefficients, + Vector128 rounding, + Vector128 quantizer, + Vector128 dequantizer, + int logScale, + out Vector128 dequantizedCoefficients); + + /// + /// Quantizes eight raster-order transform coefficients. + /// + /// The signed transform coefficients. + /// The positive rounding constant after transform-size scaling. + /// The Q16 reciprocal quantizer. + /// The Q3 reconstruction quantizer. + /// The transform-size quantization scale. + /// The signed reconstruction coefficients. + /// The signed entropy-coding coefficients. + public static abstract Vector256 Quantize( + Vector256 coefficients, + Vector256 rounding, + Vector256 quantizer, + Vector256 dequantizer, + int logScale, + out Vector256 dequantizedCoefficients); + + /// + /// Quantizes sixteen raster-order transform coefficients. + /// + /// The signed transform coefficients. + /// The positive rounding constant after transform-size scaling. + /// The Q16 reciprocal quantizer. + /// The Q3 reconstruction quantizer. + /// The transform-size quantization scale. + /// The signed reconstruction coefficients. + /// The signed entropy-coding coefficients. + public static abstract Vector512 Quantize( + Vector512 coefficients, + Vector512 rounding, + Vector512 quantizer, + Vector512 dequantizer, + int logScale, + out Vector512 dequantizedCoefficients); + + /// + /// Quantizes one transform coefficient. + /// + /// The signed transform coefficient. + /// The positive rounding constant after transform-size scaling. + /// The Q16 reciprocal quantizer. + /// The Q3 reconstruction quantizer. + /// The transform-size quantization scale. + /// The signed reconstruction coefficient. + /// The signed entropy-coding coefficient. + public static abstract int Quantize( + int coefficient, + int rounding, + int quantizer, + int dequantizer, + int logScale, + out int dequantizedCoefficient); + } + + /// + /// Implements libaom's fast no-matrix quantizer for lossy transform blocks. + /// + /// + /// Every SIMD overload preserves the scalar operation order: magnitude threshold, saturating round, Q16 reciprocal + /// multiply, transform-size shift, dequantization, and sign restoration. Each lane owns one raster-order coefficient. + /// + internal readonly struct FastQuantizationOperator : IForwardQuantizationOperator + { + /// + public static Vector128 Quantize( + Vector128 coefficients, + Vector128 rounding, + Vector128 quantizer, + Vector128 dequantizer, + int logScale, + out Vector128 dequantizedCoefficients) + { + Vector128 coefficientSign = Vector128.ShiftRightArithmetic(coefficients, 31); + Vector128 magnitude = Vector128.Abs(coefficients); + + // libaom retains equality at the dequantizer threshold. Reversing the comparison and complementing its mask + // expresses scaledMagnitude >= dequantizer with the vector operations available for every supported ISA. + Vector128 thresholdMask = ~Vector128.GreaterThan(dequantizer, magnitude << (1 + logScale)); + + // Clamp the rounded magnitude to 32,767 so the following Q16 product remains within a signed 32-bit lane. + Vector128 rounded = Vector128.Min(magnitude + rounding, Vector128.Create((int)short.MaxValue)); + Vector128 quantizedMagnitude = ((rounded * quantizer) >> (16 - logScale)) & thresholdMask; + + // XOR followed by subtraction restores the original sign without a lane-wise branch. + Vector128 quantized = (quantizedMagnitude ^ coefficientSign) - coefficientSign; + Vector128 dequantizedMagnitude = (quantizedMagnitude * dequantizer) >> logScale; + dequantizedCoefficients = (dequantizedMagnitude ^ coefficientSign) - coefficientSign; + return quantized; + } + + /// + public static Vector256 Quantize( + Vector256 coefficients, + Vector256 rounding, + Vector256 quantizer, + Vector256 dequantizer, + int logScale, + out Vector256 dequantizedCoefficients) + { + Vector256 coefficientSign = Vector256.ShiftRightArithmetic(coefficients, 31); + Vector256 magnitude = Vector256.Abs(coefficients); + + // Preserve threshold equality by complementing dequantizer > scaledMagnitude. + Vector256 thresholdMask = ~Vector256.GreaterThan(dequantizer, magnitude << (1 + logScale)); + + // Clamp the rounded magnitude to 32,767 so the following Q16 product remains within a signed 32-bit lane. + Vector256 rounded = Vector256.Min(magnitude + rounding, Vector256.Create((int)short.MaxValue)); + Vector256 quantizedMagnitude = ((rounded * quantizer) >> (16 - logScale)) & thresholdMask; + + // Apply the input sign to both coded and reconstructed magnitudes without branching. + Vector256 quantized = (quantizedMagnitude ^ coefficientSign) - coefficientSign; + Vector256 dequantizedMagnitude = (quantizedMagnitude * dequantizer) >> logScale; + dequantizedCoefficients = (dequantizedMagnitude ^ coefficientSign) - coefficientSign; + return quantized; + } + + /// + public static Vector512 Quantize( + Vector512 coefficients, + Vector512 rounding, + Vector512 quantizer, + Vector512 dequantizer, + int logScale, + out Vector512 dequantizedCoefficients) + { + Vector512 coefficientSign = Vector512.ShiftRightArithmetic(coefficients, 31); + Vector512 magnitude = Vector512.Abs(coefficients); + + // Preserve threshold equality by complementing dequantizer > scaledMagnitude. + Vector512 thresholdMask = ~Vector512.GreaterThan(dequantizer, magnitude << (1 + logScale)); + + // Clamp the rounded magnitude to 32,767 so the following Q16 product remains within a signed 32-bit lane. + Vector512 rounded = Vector512.Min(magnitude + rounding, Vector512.Create((int)short.MaxValue)); + Vector512 quantizedMagnitude = ((rounded * quantizer) >> (16 - logScale)) & thresholdMask; + + // Apply the input sign to both coded and reconstructed magnitudes without branching. + Vector512 quantized = (quantizedMagnitude ^ coefficientSign) - coefficientSign; + Vector512 dequantizedMagnitude = (quantizedMagnitude * dequantizer) >> logScale; + dequantizedCoefficients = (dequantizedMagnitude ^ coefficientSign) - coefficientSign; + return quantized; + } + + /// + public static int Quantize( + int coefficient, + int rounding, + int quantizer, + int dequantizer, + int logScale, + out int dequantizedCoefficient) + { + int coefficientSign = coefficient >> 31; + int magnitude = (coefficient ^ coefficientSign) - coefficientSign; + int quantizedMagnitude = 0; + + // The scalar tail keeps the same threshold, clamp, and fixed-point operation order as every vector lane. + if (((long)magnitude << (1 + logScale)) >= dequantizer) + { + int rounded = Math.Min(magnitude + rounding, short.MaxValue); + quantizedMagnitude = (rounded * quantizer) >> (16 - logScale); + } + + int quantized = (quantizedMagnitude ^ coefficientSign) - coefficientSign; + int dequantizedMagnitude = (quantizedMagnitude * dequantizer) >> logScale; + dequantizedCoefficient = (dequantizedMagnitude ^ coefficientSign) - coefficientSign; + return quantized; + } + } + + /// + /// Implements fast no-matrix quantization without truncating high-bit-depth transform magnitudes. + /// + /// + /// The reciprocal product is widened to 64 bits because 10-bit and 12-bit transforms can exceed the signed + /// 16-bit range. Each SIMD overload preserves the scalar fixed-point operation order in every lane. + /// + internal readonly struct HighBitDepthFastQuantizationOperator : IForwardQuantizationOperator + { + /// + public static Vector128 Quantize( + Vector128 coefficients, + Vector128 rounding, + Vector128 quantizer, + Vector128 dequantizer, + int logScale, + out Vector128 dequantizedCoefficients) + { + Vector128 coefficientSign = Vector128.ShiftRightArithmetic(coefficients, 31); + Vector128 magnitude = Vector128.Abs(coefficients); + Vector128 thresholdMask = ~Vector128.GreaterThan(dequantizer, magnitude << (1 + logScale)); + Vector128 rounded = magnitude + rounding; + + // Widen before multiplying so transform magnitudes above 32,767 retain their full precision. + Vector128 lowerProduct = Vector128.WidenLower(rounded) * Vector128.WidenLower(quantizer); + Vector128 upperProduct = Vector128.WidenUpper(rounded) * Vector128.WidenUpper(quantizer); + Vector128 quantizedMagnitude = + Vector128.Narrow(lowerProduct >> (16 - logScale), upperProduct >> (16 - logScale)) & thresholdMask; + + Vector128 quantized = (quantizedMagnitude ^ coefficientSign) - coefficientSign; + Vector128 dequantizedMagnitude = (quantizedMagnitude * dequantizer) >> logScale; + dequantizedCoefficients = (dequantizedMagnitude ^ coefficientSign) - coefficientSign; + return quantized; + } + + /// + public static Vector256 Quantize( + Vector256 coefficients, + Vector256 rounding, + Vector256 quantizer, + Vector256 dequantizer, + int logScale, + out Vector256 dequantizedCoefficients) + { + Vector256 coefficientSign = Vector256.ShiftRightArithmetic(coefficients, 31); + Vector256 magnitude = Vector256.Abs(coefficients); + Vector256 thresholdMask = ~Vector256.GreaterThan(dequantizer, magnitude << (1 + logScale)); + Vector256 rounded = magnitude + rounding; + + // Widen before multiplying so transform magnitudes above 32,767 retain their full precision. + Vector256 lowerProduct = Vector256.WidenLower(rounded) * Vector256.WidenLower(quantizer); + Vector256 upperProduct = Vector256.WidenUpper(rounded) * Vector256.WidenUpper(quantizer); + Vector256 quantizedMagnitude = + Vector256.Narrow(lowerProduct >> (16 - logScale), upperProduct >> (16 - logScale)) & thresholdMask; + + Vector256 quantized = (quantizedMagnitude ^ coefficientSign) - coefficientSign; + Vector256 dequantizedMagnitude = (quantizedMagnitude * dequantizer) >> logScale; + dequantizedCoefficients = (dequantizedMagnitude ^ coefficientSign) - coefficientSign; + return quantized; + } + + /// + public static Vector512 Quantize( + Vector512 coefficients, + Vector512 rounding, + Vector512 quantizer, + Vector512 dequantizer, + int logScale, + out Vector512 dequantizedCoefficients) + { + Vector512 coefficientSign = Vector512.ShiftRightArithmetic(coefficients, 31); + Vector512 magnitude = Vector512.Abs(coefficients); + Vector512 thresholdMask = ~Vector512.GreaterThan(dequantizer, magnitude << (1 + logScale)); + Vector512 rounded = magnitude + rounding; + + // Widen before multiplying so transform magnitudes above 32,767 retain their full precision. + Vector512 lowerProduct = Vector512.WidenLower(rounded) * Vector512.WidenLower(quantizer); + Vector512 upperProduct = Vector512.WidenUpper(rounded) * Vector512.WidenUpper(quantizer); + Vector512 quantizedMagnitude = + Vector512.Narrow(lowerProduct >> (16 - logScale), upperProduct >> (16 - logScale)) & thresholdMask; + + Vector512 quantized = (quantizedMagnitude ^ coefficientSign) - coefficientSign; + Vector512 dequantizedMagnitude = (quantizedMagnitude * dequantizer) >> logScale; + dequantizedCoefficients = (dequantizedMagnitude ^ coefficientSign) - coefficientSign; + return quantized; + } + + /// + public static int Quantize( + int coefficient, + int rounding, + int quantizer, + int dequantizer, + int logScale, + out int dequantizedCoefficient) + { + int coefficientSign = coefficient >> 31; + int magnitude = (coefficient ^ coefficientSign) - coefficientSign; + int quantizedMagnitude = 0; + + if (((long)magnitude << (1 + logScale)) >= dequantizer) + { + quantizedMagnitude = (int)((((long)magnitude + rounding) * quantizer) >> (16 - logScale)); + } + + int quantized = (quantizedMagnitude ^ coefficientSign) - coefficientSign; + int dequantizedMagnitude = (quantizedMagnitude * dequantizer) >> logScale; + dequantizedCoefficient = (dequantizedMagnitude ^ coefficientSign) - coefficientSign; + return quantized; + } + } + + /// + /// Removes the reversible transform's fixed scale for entropy coding while retaining exact reconstruction coefficients. + /// + internal readonly struct LosslessQuantizationOperator : IForwardQuantizationOperator + { + /// + public static Vector128 Quantize( + Vector128 coefficients, + Vector128 rounding, + Vector128 quantizer, + Vector128 dequantizer, + int logScale, + out Vector128 dequantizedCoefficients) + { + dequantizedCoefficients = coefficients; + return coefficients >> 2; + } + + /// + public static Vector256 Quantize( + Vector256 coefficients, + Vector256 rounding, + Vector256 quantizer, + Vector256 dequantizer, + int logScale, + out Vector256 dequantizedCoefficients) + { + dequantizedCoefficients = coefficients; + return coefficients >> 2; + } + + /// + public static Vector512 Quantize( + Vector512 coefficients, + Vector512 rounding, + Vector512 quantizer, + Vector512 dequantizer, + int logScale, + out Vector512 dequantizedCoefficients) + { + dequantizedCoefficients = coefficients; + return coefficients >> 2; + } + + /// + public static int Quantize( + int coefficient, + int rounding, + int quantizer, + int dequantizer, + int logScale, + out int dequantizedCoefficient) + { + dequantizedCoefficient = coefficient; + return coefficient >> 2; + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/Quantizers/Av1ForwardQuantizer.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Quantizers/Av1ForwardQuantizer.cs new file mode 100644 index 0000000000..76341f56c6 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Quantizers/Av1ForwardQuantizer.cs @@ -0,0 +1,244 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.InteropServices; +using System.Runtime.Intrinsics; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.Quantizers; + +/// +/// Applies AV1 forward quantization to raster-order transform coefficients. +/// +internal static partial class Av1ForwardQuantizer +{ + /// + /// Quantizes one lossy transform block with libaom's fast no-matrix arithmetic. + /// + /// The raster-order forward-transform coefficients. + /// The raster-order entropy-coding coefficients. + /// The raster-order reconstruction coefficients. + /// The transform-block dimensions. + /// The compound transform type. + /// The segment quantizer index. + /// The plane DC quantizer adjustment. + /// The plane AC quantizer adjustment. + /// The coded sample bit depth. + /// The one-based end position in coefficient scan order. + public static ushort QuantizeLossy( + ReadOnlySpan coefficients, + Span quantizedCoefficients, + Span dequantizedCoefficients, + Av1TransformSize transformSize, + Av1TransformType transformType, + int qIndex, + int dcDeltaQ, + int acDeltaQ, + Av1BitDepth bitDepth) + => bitDepth == Av1BitDepth.EightBit + ? Quantize( + coefficients, + quantizedCoefficients, + dequantizedCoefficients, + transformSize, + transformType, + qIndex, + dcDeltaQ, + acDeltaQ, + bitDepth) + : Quantize( + coefficients, + quantizedCoefficients, + dequantizedCoefficients, + transformSize, + transformType, + qIndex, + dcDeltaQ, + acDeltaQ, + bitDepth); + + /// + /// Quantizes one reversible four-by-four transform without changing its reconstruction coefficients. + /// + /// The raster-order reversible-transform coefficients. + /// The raster-order entropy-coding coefficients. + /// The raster-order reconstruction coefficients. + /// The coded sample bit depth. + /// The one-based end position in coefficient scan order. + public static ushort QuantizeLossless( + ReadOnlySpan coefficients, + Span quantizedCoefficients, + Span dequantizedCoefficients, + Av1BitDepth bitDepth) + => Quantize( + coefficients, + quantizedCoefficients, + dequantizedCoefficients, + Av1TransformSize.Size4x4, + Av1TransformType.DctDct, + 0, + 0, + 0, + bitDepth); + + /// + /// Applies a closed generic quantization operator across the widest available hardware widths. + /// + private static ushort Quantize( + ReadOnlySpan coefficients, + Span quantizedCoefficients, + Span dequantizedCoefficients, + Av1TransformSize transformSize, + Av1TransformType transformType, + int qIndex, + int dcDeltaQ, + int acDeltaQ, + Av1BitDepth bitDepth) + where TOperator : struct, IForwardQuantizationOperator + { + int coefficientCount = transformSize.GetAdjusted().GetSize2d(); + int logScale = transformSize.GetScale(); + int dcDequantizer = Av1QuantizationLookup.GetDcQuant(qIndex, dcDeltaQ, bitDepth); + int acDequantizer = Av1QuantizationLookup.GetAcQuant(qIndex, acDeltaQ, bitDepth); + int dcQuantizer = (1 << 16) / dcDequantizer; + int acQuantizer = (1 << 16) / acDequantizer; + int dcRounding = RoundPowerOfTwo((64 * dcDequantizer) >> 7, logScale); + int acRounding = RoundPowerOfTwo((64 * acDequantizer) >> 7, logScale); + + ref int sourceBase = ref MemoryMarshal.GetReference(coefficients); + ref int quantizedBase = ref MemoryMarshal.GetReference(quantizedCoefficients); + ref int dequantizedBase = ref MemoryMarshal.GetReference(dequantizedCoefficients); + + // Raster coefficient zero is the only DC coefficient, so it is encoded once with the plane's DC constants + // before the AC-only SIMD traversal begins. + Unsafe.Add(ref quantizedBase, 0) = TOperator.Quantize( + Unsafe.Add(ref sourceBase, 0), + dcRounding, + dcQuantizer, + dcDequantizer, + logScale, + out Unsafe.Add(ref dequantizedBase, 0)); + + int i = 1; + + // Raster traversal keeps loads and stores contiguous. Each narrower tier resumes at the shared offset left by + // the previous tier, retaining vector execution for the widest possible remainder without overlapping lanes. + if (Vector512.IsHardwareAccelerated) + { + nuint vectorCount = coefficients[i..coefficientCount].Vector512Count(); + + if (vectorCount > 0) + { + // Width-specific constants are created only when at least one complete vector remains. + Vector512 rounding = Vector512.Create(acRounding); + Vector512 quantizer = Vector512.Create(acQuantizer); + Vector512 dequantizer = Vector512.Create(acDequantizer); + + for (; vectorCount > 0; vectorCount--, i += Vector512.Count) + { + Vector512 source = Unsafe.As>(ref Unsafe.Add(ref sourceBase, i)); + Vector512 quantized = TOperator.Quantize( + source, + rounding, + quantizer, + dequantizer, + logScale, + out Vector512 dequantized); + + Unsafe.As>(ref Unsafe.Add(ref quantizedBase, i)) = quantized; + Unsafe.As>(ref Unsafe.Add(ref dequantizedBase, i)) = dequantized; + } + } + } + + if (Vector256.IsHardwareAccelerated) + { + nuint vectorCount = coefficients[i..coefficientCount].Vector256Count(); + + if (vectorCount > 0) + { + // The shared offset exposes only the remainder left by wider lanes, so no coefficient is revisited. + Vector256 rounding = Vector256.Create(acRounding); + Vector256 quantizer = Vector256.Create(acQuantizer); + Vector256 dequantizer = Vector256.Create(acDequantizer); + + for (; vectorCount > 0; vectorCount--, i += Vector256.Count) + { + Vector256 source = Unsafe.As>(ref Unsafe.Add(ref sourceBase, i)); + Vector256 quantized = TOperator.Quantize( + source, + rounding, + quantizer, + dequantizer, + logScale, + out Vector256 dequantized); + + Unsafe.As>(ref Unsafe.Add(ref quantizedBase, i)) = quantized; + Unsafe.As>(ref Unsafe.Add(ref dequantizedBase, i)) = dequantized; + } + } + } + + if (Vector128.IsHardwareAccelerated) + { + nuint vectorCount = coefficients[i..coefficientCount].Vector128Count(); + + if (vectorCount > 0) + { + // The final SIMD tier consumes complete four-lane groups and leaves fewer than four coefficients. + Vector128 rounding = Vector128.Create(acRounding); + Vector128 quantizer = Vector128.Create(acQuantizer); + Vector128 dequantizer = Vector128.Create(acDequantizer); + + for (; vectorCount > 0; vectorCount--, i += Vector128.Count) + { + Vector128 source = Unsafe.As>(ref Unsafe.Add(ref sourceBase, i)); + Vector128 quantized = TOperator.Quantize( + source, + rounding, + quantizer, + dequantizer, + logScale, + out Vector128 dequantized); + + Unsafe.As>(ref Unsafe.Add(ref quantizedBase, i)) = quantized; + Unsafe.As>(ref Unsafe.Add(ref dequantizedBase, i)) = dequantized; + } + } + } + + // On SIMD-capable systems this loop receives only the final zero-to-three AC coefficients. + for (; i < coefficientCount; i++) + { + Unsafe.Add(ref quantizedBase, i) = TOperator.Quantize( + Unsafe.Add(ref sourceBase, i), + acRounding, + acQuantizer, + acDequantizer, + logScale, + out Unsafe.Add(ref dequantizedBase, i)); + } + + ReadOnlySpan scan = Av1ScanOrderConstants.GetScanOrder(transformSize, transformType).Scan; + + // Quantized coefficients remain in raster order for reconstruction and entropy coding. A reverse scan finds + // the final nonzero position without another buffer, and normally exits on its first iteration at high quality. + for (int scanIndex = coefficientCount - 1; scanIndex >= 0; scanIndex--) + { + if (Unsafe.Add(ref quantizedBase, scan[scanIndex]) != 0) + { + return (ushort)(scanIndex + 1); + } + } + + return 0; + } + + /// + /// Applies libaom's positive round-power-of-two operation to one quantizer constant. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static int RoundPowerOfTwo(int value, int shift) + => shift == 0 ? value : (value + (1 << (shift - 1))) >> shift; +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/Quantizers/Av1InverseQuantizationLookup.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Quantizers/Av1InverseQuantizationLookup.cs new file mode 100644 index 0000000000..b5390bc9d0 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Quantizers/Av1InverseQuantizationLookup.cs @@ -0,0 +1,6823 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.Quantizers; + +/// +/// Provides the normative AV1 inverse quantization matrices for each matrix level, plane class, and transform size. +/// +internal static class Av1InverseQuantizationLookup +{ + /// + /// Maps each AV1 transform size to its stored matrix index; sizes with a 64-pixel dimension reuse the adjusted 32-pixel matrix. + /// + private static readonly byte[] TransformMatrixIndices = + [ + 0, 1, 2, 3, 3, 4, 5, 6, 7, 8, 9, 3, 3, 10, 11, 12, 13, 8, 9 + ]; + + /// + /// The inverse matrix weights indexed by matrix level, plane class, adjusted transform-size index, and raster coefficient. + /// Luma and chroma have separate matrices; U and V share the chroma set. Size-specific matrices are subsampled from the + /// normative 32x32 and 16x16 bases, and matrix levels represent different frequency-domain weighting strengths. + /// + private static readonly int[][][][] InverseWeightTable = + [ + [ + [ + /* Luma */ + /* Size 4x4 */ + [32, 43, 73, 97, 43, 67, 94, 110, 73, 94, 137, 150, 97, 110, 150, 200], + /* Size 8x8 */ + [32, 32, 38, 51, 68, 84, 95, 109, 32, 35, 40, 49, 63, 76, 89, 102, 38, + 40, 54, 65, 78, 91, 98, 106, 51, 49, 65, 82, 97, 111, 113, 121, 68, 63, + 78, 97, 117, 134, 138, 142, 84, 76, 91, 111, 134, 152, 159, 168, 95, 89, + 98, 113, 138, 159, 183, 199, 109, 102, 106, 121, 142, 168, 199, 220], + /* Size 16x16 */ + [32, 31, 31, 34, 36, 44, 48, 59, 65, 80, 83, 91, 97, 104, 111, 119, 31, + 32, 32, 33, 34, 41, 44, 54, 59, 72, 75, 83, 90, 97, 104, 112, 31, 32, + 33, 35, 36, 42, 45, 54, 59, 71, 74, 81, 86, 93, 100, 107, 34, 33, 35, + 39, 42, 47, 51, 58, 63, 74, 76, 81, 84, 90, 97, 105, 36, 34, 36, 42, 48, + 54, 57, 64, 68, 79, 81, 88, 91, 96, 102, 105, 44, 41, 42, 47, 54, 63, + 67, 75, 79, 90, 92, 95, 100, 102, 109, 112, 48, 44, 45, 51, 57, 67, 71, + 80, 85, 96, 99, 107, 108, 111, 117, 120, 59, 54, 54, 58, 64, 75, 80, 92, + 98, 110, 113, 115, 116, 122, 125, 130, 65, 59, 59, 63, 68, 79, 85, 98, + 105, 118, 121, 127, 130, 134, 135, 140, 80, 72, 71, 74, 79, 90, 96, 110, + 118, 134, 137, 140, 143, 144, 146, 152, 83, 75, 74, 76, 81, 92, 99, 113, + 121, 137, 140, 151, 152, 155, 158, 165, 91, 83, 81, 81, 88, 95, 107, + 115, 127, 140, 151, 159, 166, 169, 173, 179, 97, 90, 86, 84, 91, 100, + 108, 116, 130, 143, 152, 166, 174, 182, 189, 193, 104, 97, 93, 90, 96, + 102, 111, 122, 134, 144, 155, 169, 182, 191, 200, 210, 111, 104, 100, + 97, 102, 109, 117, 125, 135, 146, 158, 173, 189, 200, 210, 220, 119, + 112, 107, 105, 105, 112, 120, 130, 140, 152, 165, 179, 193, 210, 220, + 231], + /* Size 32x32 */ + [32, 31, 31, 31, 31, 32, 34, 35, 36, 39, 44, 46, 48, 54, 59, 62, 65, 71, + 80, 81, 83, 88, 91, 94, 97, 101, 104, 107, 111, 115, 119, 123, 31, 32, + 32, 32, 32, 32, 34, 34, 35, 38, 42, 44, 46, 51, 56, 59, 62, 68, 76, 77, + 78, 84, 86, 89, 92, 95, 99, 102, 105, 109, 113, 116, 31, 32, 32, 32, 32, + 32, 33, 34, 34, 37, 41, 42, 44, 49, 54, 56, 59, 65, 72, 73, 75, 80, 83, + 86, 90, 93, 97, 101, 104, 108, 112, 116, 31, 32, 32, 32, 33, 33, 34, 35, + 35, 38, 41, 43, 45, 49, 54, 56, 59, 64, 72, 73, 74, 79, 82, 85, 88, 91, + 94, 97, 101, 104, 107, 111, 31, 32, 32, 33, 33, 34, 35, 36, 36, 39, 42, + 44, 45, 50, 54, 56, 59, 64, 71, 72, 74, 78, 81, 84, 86, 89, 93, 96, 100, + 104, 107, 111, 32, 32, 32, 33, 34, 35, 37, 37, 38, 40, 42, 44, 46, 49, + 53, 55, 58, 63, 69, 70, 72, 76, 79, 82, 85, 89, 93, 96, 99, 102, 106, + 109, 34, 34, 33, 34, 35, 37, 39, 41, 42, 45, 47, 49, 51, 54, 58, 60, 63, + 68, 74, 75, 76, 80, 81, 82, 84, 87, 90, 93, 97, 101, 105, 110, 35, 34, + 34, 35, 36, 37, 41, 43, 45, 47, 50, 52, 53, 57, 61, 63, 65, 70, 76, 77, + 79, 82, 84, 86, 89, 91, 92, 93, 96, 100, 103, 107, 36, 35, 34, 35, 36, + 38, 42, 45, 48, 50, 54, 55, 57, 60, 64, 66, 68, 73, 79, 80, 81, 85, 88, + 90, 91, 93, 96, 99, 102, 103, 105, 107, 39, 38, 37, 38, 39, 40, 45, 47, + 50, 54, 58, 59, 61, 65, 69, 71, 73, 78, 84, 85, 86, 91, 92, 92, 95, 98, + 100, 101, 103, 106, 110, 114, 44, 42, 41, 41, 42, 42, 47, 50, 54, 58, + 63, 65, 67, 71, 75, 77, 79, 84, 90, 91, 92, 95, 95, 97, 100, 101, 102, + 105, 109, 111, 112, 114, 46, 44, 42, 43, 44, 44, 49, 52, 55, 59, 65, 67, + 69, 74, 78, 80, 82, 87, 93, 94, 95, 98, 100, 103, 102, 105, 108, 110, + 111, 113, 117, 121, 48, 46, 44, 45, 45, 46, 51, 53, 57, 61, 67, 69, 71, + 76, 80, 83, 85, 90, 96, 97, 99, 103, 107, 105, 108, 111, 111, 113, 117, + 119, 120, 122, 54, 51, 49, 49, 50, 49, 54, 57, 60, 65, 71, 74, 76, 82, + 87, 89, 92, 97, 104, 105, 106, 111, 110, 111, 114, 113, 116, 120, 120, + 121, 125, 130, 59, 56, 54, 54, 54, 53, 58, 61, 64, 69, 75, 78, 80, 87, + 92, 95, 98, 103, 110, 111, 113, 115, 115, 119, 116, 120, 122, 122, 125, + 129, 130, 130, 62, 59, 56, 56, 56, 55, 60, 63, 66, 71, 77, 80, 83, 89, + 95, 98, 101, 107, 114, 115, 117, 119, 123, 121, 125, 126, 125, 129, 131, + 131, 135, 140, 65, 62, 59, 59, 59, 58, 63, 65, 68, 73, 79, 82, 85, 92, + 98, 101, 105, 111, 118, 119, 121, 126, 127, 128, 130, 130, 134, 133, + 135, 140, 140, 140, 71, 68, 65, 64, 64, 63, 68, 70, 73, 78, 84, 87, 90, + 97, 103, 107, 111, 117, 125, 126, 128, 134, 132, 136, 133, 138, 137, + 140, 143, 142, 145, 150, 80, 76, 72, 72, 71, 69, 74, 76, 79, 84, 90, 93, + 96, 104, 110, 114, 118, 125, 134, 135, 137, 139, 140, 139, 143, 142, + 144, 146, 146, 151, 152, 151, 81, 77, 73, 73, 72, 70, 75, 77, 80, 85, + 91, 94, 97, 105, 111, 115, 119, 126, 135, 137, 138, 144, 147, 146, 148, + 149, 151, 150, 156, 155, 157, 163, 83, 78, 75, 74, 74, 72, 76, 79, 81, + 86, 92, 95, 99, 106, 113, 117, 121, 128, 137, 138, 140, 147, 151, 156, + 152, 157, 155, 161, 158, 162, 165, 164, 88, 84, 80, 79, 78, 76, 80, 82, + 85, 91, 95, 98, 103, 111, 115, 119, 126, 134, 139, 144, 147, 152, 154, + 158, 163, 159, 165, 163, 168, 168, 169, 176, 91, 86, 83, 82, 81, 79, 81, + 84, 88, 92, 95, 100, 107, 110, 115, 123, 127, 132, 140, 147, 151, 154, + 159, 161, 166, 171, 169, 173, 173, 176, 179, 177, 94, 89, 86, 85, 84, + 82, 82, 86, 90, 92, 97, 103, 105, 111, 119, 121, 128, 136, 139, 146, + 156, 158, 161, 166, 168, 174, 179, 178, 180, 183, 183, 190, 97, 92, 90, + 88, 86, 85, 84, 89, 91, 95, 100, 102, 108, 114, 116, 125, 130, 133, 143, + 148, 152, 163, 166, 168, 174, 176, 182, 187, 189, 188, 193, 191, 101, + 95, 93, 91, 89, 89, 87, 91, 93, 98, 101, 105, 111, 113, 120, 126, 130, + 138, 142, 149, 157, 159, 171, 174, 176, 183, 184, 191, 195, 199, 197, + 204, 104, 99, 97, 94, 93, 93, 90, 92, 96, 100, 102, 108, 111, 116, 122, + 125, 134, 137, 144, 151, 155, 165, 169, 179, 182, 184, 191, 193, 200, + 204, 210, 206, 107, 102, 101, 97, 96, 96, 93, 93, 99, 101, 105, 110, + 113, 120, 122, 129, 133, 140, 146, 150, 161, 163, 173, 178, 187, 191, + 193, 200, 202, 210, 214, 222, 111, 105, 104, 101, 100, 99, 97, 96, 102, + 103, 109, 111, 117, 120, 125, 131, 135, 143, 146, 156, 158, 168, 173, + 180, 189, 195, 200, 202, 210, 212, 220, 224, 115, 109, 108, 104, 104, + 102, 101, 100, 103, 106, 111, 113, 119, 121, 129, 131, 140, 142, 151, + 155, 162, 168, 176, 183, 188, 199, 204, 210, 212, 220, 222, 230, 119, + 113, 112, 107, 107, 106, 105, 103, 105, 110, 112, 117, 120, 125, 130, + 135, 140, 145, 152, 157, 165, 169, 179, 183, 193, 197, 210, 214, 220, + 222, 231, 232, 123, 116, 116, 111, 111, 109, 110, 107, 107, 114, 114, + 121, 122, 130, 130, 140, 140, 150, 151, 163, 164, 176, 177, 190, 191, + 204, 206, 222, 224, 230, 232, 242], + /* Size 4x8 */ + [32, 42, 75, 91, 33, 42, 69, 86, 37, 58, 84, 91, 49, 71, 103, 110, 65, + 84, 125, 128, 80, 97, 142, 152, 91, 100, 145, 178, 104, 112, 146, 190], + /* Size 8x4 */ + [32, 33, 37, 49, 65, 80, 91, 104, 42, 42, 58, 71, 84, 97, 100, 112, 75, + 69, 84, 103, 125, 142, 145, 146, 91, 86, 91, 110, 128, 152, 178, 190], + /* Size 8x16 */ + [32, 32, 36, 53, 65, 87, 93, 99, 31, 33, 34, 49, 59, 78, 86, 93, 32, 34, + 36, 50, 59, 77, 82, 89, 34, 37, 42, 54, 63, 79, 80, 88, 36, 38, 48, 60, + 68, 84, 86, 90, 44, 43, 53, 71, 79, 95, 94, 97, 48, 46, 56, 76, 85, 102, + 105, 105, 58, 54, 63, 87, 98, 116, 112, 115, 65, 58, 68, 92, 105, 124, + 122, 124, 79, 70, 79, 104, 118, 141, 135, 135, 82, 72, 81, 106, 121, + 144, 149, 146, 91, 80, 88, 106, 130, 148, 162, 159, 97, 86, 94, 107, + 128, 157, 167, 171, 103, 93, 98, 114, 131, 150, 174, 186, 110, 100, 101, + 117, 138, 161, 183, 193, 118, 107, 105, 118, 136, 157, 182, 203], + /* Size 16x8 */ + [32, 31, 32, 34, 36, 44, 48, 58, 65, 79, 82, 91, 97, 103, 110, 118, 32, + 33, 34, 37, 38, 43, 46, 54, 58, 70, 72, 80, 86, 93, 100, 107, 36, 34, + 36, 42, 48, 53, 56, 63, 68, 79, 81, 88, 94, 98, 101, 105, 53, 49, 50, + 54, 60, 71, 76, 87, 92, 104, 106, 106, 107, 114, 117, 118, 65, 59, 59, + 63, 68, 79, 85, 98, 105, 118, 121, 130, 128, 131, 138, 136, 87, 78, 77, + 79, 84, 95, 102, 116, 124, 141, 144, 148, 157, 150, 161, 157, 93, 86, + 82, 80, 86, 94, 105, 112, 122, 135, 149, 162, 167, 174, 183, 182, 99, + 93, 89, 88, 90, 97, 105, 115, 124, 135, 146, 159, 171, 186, 193, 203], + /* Size 16x32 */ + [32, 31, 32, 34, 36, 44, 53, 59, 65, 79, 87, 90, 93, 96, 99, 102, 31, 32, + 32, 34, 35, 42, 51, 56, 62, 75, 82, 85, 88, 91, 94, 97, 31, 32, 33, 33, + 34, 41, 49, 54, 59, 72, 78, 82, 86, 90, 93, 97, 31, 32, 33, 34, 35, 41, + 49, 54, 59, 71, 78, 81, 84, 87, 90, 93, 32, 32, 34, 35, 36, 42, 50, 54, + 59, 71, 77, 80, 82, 86, 89, 93, 32, 33, 35, 37, 38, 42, 49, 53, 58, 69, + 75, 78, 82, 86, 89, 92, 34, 34, 37, 39, 42, 48, 54, 58, 63, 73, 79, 78, + 80, 83, 88, 92, 35, 34, 37, 41, 45, 50, 57, 61, 65, 76, 82, 83, 84, 84, + 87, 90, 36, 34, 38, 43, 48, 54, 60, 64, 68, 78, 84, 87, 86, 89, 90, 90, + 39, 37, 40, 45, 50, 58, 65, 69, 73, 84, 89, 89, 91, 91, 93, 96, 44, 41, + 43, 48, 53, 63, 71, 75, 79, 90, 95, 93, 94, 95, 97, 97, 46, 43, 44, 49, + 55, 65, 73, 78, 82, 93, 98, 100, 98, 100, 99, 103, 48, 45, 46, 51, 56, + 67, 76, 80, 85, 96, 102, 102, 105, 102, 105, 104, 53, 49, 50, 54, 60, + 71, 82, 87, 92, 103, 109, 107, 107, 110, 107, 111, 58, 54, 54, 58, 63, + 75, 87, 92, 98, 110, 116, 115, 112, 111, 115, 112, 61, 57, 56, 60, 66, + 77, 89, 95, 101, 114, 120, 118, 119, 118, 116, 120, 65, 60, 58, 63, 68, + 79, 92, 98, 105, 118, 124, 123, 122, 123, 124, 121, 71, 65, 63, 68, 73, + 84, 97, 103, 111, 125, 132, 132, 130, 128, 127, 130, 79, 72, 70, 74, 79, + 90, 104, 110, 118, 133, 141, 136, 135, 135, 135, 131, 81, 74, 71, 75, + 80, 91, 105, 112, 119, 135, 142, 140, 140, 138, 139, 142, 82, 75, 72, + 76, 81, 92, 106, 113, 121, 136, 144, 151, 149, 149, 146, 143, 88, 80, + 77, 80, 85, 97, 108, 115, 126, 142, 149, 153, 153, 152, 152, 154, 91, + 83, 80, 81, 88, 100, 106, 114, 130, 142, 148, 155, 162, 160, 159, 155, + 94, 85, 83, 82, 91, 100, 105, 118, 131, 137, 153, 160, 165, 167, 166, + 168, 97, 88, 86, 85, 94, 100, 107, 123, 128, 140, 157, 161, 167, 173, + 171, 169, 100, 91, 89, 87, 97, 100, 111, 121, 127, 145, 152, 164, 173, + 178, 182, 181, 103, 94, 93, 90, 98, 101, 114, 120, 131, 144, 150, 170, + 174, 180, 186, 183, 107, 97, 96, 93, 100, 104, 117, 119, 136, 142, 155, + 168, 177, 187, 191, 198, 110, 101, 100, 97, 101, 108, 117, 123, 138, + 141, 161, 165, 183, 188, 193, 200, 114, 104, 104, 100, 103, 112, 117, + 127, 137, 146, 159, 167, 185, 190, 201, 206, 118, 108, 107, 103, 105, + 115, 118, 131, 136, 151, 157, 172, 182, 197, 203, 208, 122, 111, 111, + 107, 107, 119, 119, 136, 136, 156, 156, 178, 179, 203, 204, 217], + /* Size 32x16 */ + [32, 31, 31, 31, 32, 32, 34, 35, 36, 39, 44, 46, 48, 53, 58, 61, 65, 71, + 79, 81, 82, 88, 91, 94, 97, 100, 103, 107, 110, 114, 118, 122, 31, 32, + 32, 32, 32, 33, 34, 34, 34, 37, 41, 43, 45, 49, 54, 57, 60, 65, 72, 74, + 75, 80, 83, 85, 88, 91, 94, 97, 101, 104, 108, 111, 32, 32, 33, 33, 34, + 35, 37, 37, 38, 40, 43, 44, 46, 50, 54, 56, 58, 63, 70, 71, 72, 77, 80, + 83, 86, 89, 93, 96, 100, 104, 107, 111, 34, 34, 33, 34, 35, 37, 39, 41, + 43, 45, 48, 49, 51, 54, 58, 60, 63, 68, 74, 75, 76, 80, 81, 82, 85, 87, + 90, 93, 97, 100, 103, 107, 36, 35, 34, 35, 36, 38, 42, 45, 48, 50, 53, + 55, 56, 60, 63, 66, 68, 73, 79, 80, 81, 85, 88, 91, 94, 97, 98, 100, + 101, 103, 105, 107, 44, 42, 41, 41, 42, 42, 48, 50, 54, 58, 63, 65, 67, + 71, 75, 77, 79, 84, 90, 91, 92, 97, 100, 100, 100, 100, 101, 104, 108, + 112, 115, 119, 53, 51, 49, 49, 50, 49, 54, 57, 60, 65, 71, 73, 76, 82, + 87, 89, 92, 97, 104, 105, 106, 108, 106, 105, 107, 111, 114, 117, 117, + 117, 118, 119, 59, 56, 54, 54, 54, 53, 58, 61, 64, 69, 75, 78, 80, 87, + 92, 95, 98, 103, 110, 112, 113, 115, 114, 118, 123, 121, 120, 119, 123, + 127, 131, 136, 65, 62, 59, 59, 59, 58, 63, 65, 68, 73, 79, 82, 85, 92, + 98, 101, 105, 111, 118, 119, 121, 126, 130, 131, 128, 127, 131, 136, + 138, 137, 136, 136, 79, 75, 72, 71, 71, 69, 73, 76, 78, 84, 90, 93, 96, + 103, 110, 114, 118, 125, 133, 135, 136, 142, 142, 137, 140, 145, 144, + 142, 141, 146, 151, 156, 87, 82, 78, 78, 77, 75, 79, 82, 84, 89, 95, 98, + 102, 109, 116, 120, 124, 132, 141, 142, 144, 149, 148, 153, 157, 152, + 150, 155, 161, 159, 157, 156, 90, 85, 82, 81, 80, 78, 78, 83, 87, 89, + 93, 100, 102, 107, 115, 118, 123, 132, 136, 140, 151, 153, 155, 160, + 161, 164, 170, 168, 165, 167, 172, 178, 93, 88, 86, 84, 82, 82, 80, 84, + 86, 91, 94, 98, 105, 107, 112, 119, 122, 130, 135, 140, 149, 153, 162, + 165, 167, 173, 174, 177, 183, 185, 182, 179, 96, 91, 90, 87, 86, 86, 83, + 84, 89, 91, 95, 100, 102, 110, 111, 118, 123, 128, 135, 138, 149, 152, + 160, 167, 173, 178, 180, 187, 188, 190, 197, 203, 99, 94, 93, 90, 89, + 89, 88, 87, 90, 93, 97, 99, 105, 107, 115, 116, 124, 127, 135, 139, 146, + 152, 159, 166, 171, 182, 186, 191, 193, 201, 203, 204, 102, 97, 97, 93, + 93, 92, 92, 90, 90, 96, 97, 103, 104, 111, 112, 120, 121, 130, 131, 142, + 143, 154, 155, 168, 169, 181, 183, 198, 200, 206, 208, 217], + /* Size 4x16 */ + [31, 44, 79, 96, 32, 41, 72, 90, 32, 42, 71, 86, 34, 48, 73, 83, 34, 54, + 78, 89, 41, 63, 90, 95, 45, 67, 96, 102, 54, 75, 110, 111, 60, 79, 118, + 123, 72, 90, 133, 135, 75, 92, 136, 149, 83, 100, 142, 160, 88, 100, + 140, 173, 94, 101, 144, 180, 101, 108, 141, 188, 108, 115, 151, 197], + /* Size 16x4 */ + [31, 32, 32, 34, 34, 41, 45, 54, 60, 72, 75, 83, 88, 94, 101, 108, 44, + 41, 42, 48, 54, 63, 67, 75, 79, 90, 92, 100, 100, 101, 108, 115, 79, 72, + 71, 73, 78, 90, 96, 110, 118, 133, 136, 142, 140, 144, 141, 151, 96, 90, + 86, 83, 89, 95, 102, 111, 123, 135, 149, 160, 173, 180, 188, 197], + /* Size 8x32 */ + [32, 32, 36, 53, 65, 87, 93, 99, 31, 32, 35, 51, 62, 82, 88, 94, 31, 33, + 34, 49, 59, 78, 86, 93, 31, 33, 35, 49, 59, 78, 84, 90, 32, 34, 36, 50, + 59, 77, 82, 89, 32, 35, 38, 49, 58, 75, 82, 89, 34, 37, 42, 54, 63, 79, + 80, 88, 35, 37, 45, 57, 65, 82, 84, 87, 36, 38, 48, 60, 68, 84, 86, 90, + 39, 40, 50, 65, 73, 89, 91, 93, 44, 43, 53, 71, 79, 95, 94, 97, 46, 44, + 55, 73, 82, 98, 98, 99, 48, 46, 56, 76, 85, 102, 105, 105, 53, 50, 60, + 82, 92, 109, 107, 107, 58, 54, 63, 87, 98, 116, 112, 115, 61, 56, 66, + 89, 101, 120, 119, 116, 65, 58, 68, 92, 105, 124, 122, 124, 71, 63, 73, + 97, 111, 132, 130, 127, 79, 70, 79, 104, 118, 141, 135, 135, 81, 71, 80, + 105, 119, 142, 140, 139, 82, 72, 81, 106, 121, 144, 149, 146, 88, 77, + 85, 108, 126, 149, 153, 152, 91, 80, 88, 106, 130, 148, 162, 159, 94, + 83, 91, 105, 131, 153, 165, 166, 97, 86, 94, 107, 128, 157, 167, 171, + 100, 89, 97, 111, 127, 152, 173, 182, 103, 93, 98, 114, 131, 150, 174, + 186, 107, 96, 100, 117, 136, 155, 177, 191, 110, 100, 101, 117, 138, + 161, 183, 193, 114, 104, 103, 117, 137, 159, 185, 201, 118, 107, 105, + 118, 136, 157, 182, 203, 122, 111, 107, 119, 136, 156, 179, 204], + /* Size 32x8 */ + [32, 31, 31, 31, 32, 32, 34, 35, 36, 39, 44, 46, 48, 53, 58, 61, 65, 71, + 79, 81, 82, 88, 91, 94, 97, 100, 103, 107, 110, 114, 118, 122, 32, 32, + 33, 33, 34, 35, 37, 37, 38, 40, 43, 44, 46, 50, 54, 56, 58, 63, 70, 71, + 72, 77, 80, 83, 86, 89, 93, 96, 100, 104, 107, 111, 36, 35, 34, 35, 36, + 38, 42, 45, 48, 50, 53, 55, 56, 60, 63, 66, 68, 73, 79, 80, 81, 85, 88, + 91, 94, 97, 98, 100, 101, 103, 105, 107, 53, 51, 49, 49, 50, 49, 54, 57, + 60, 65, 71, 73, 76, 82, 87, 89, 92, 97, 104, 105, 106, 108, 106, 105, + 107, 111, 114, 117, 117, 117, 118, 119, 65, 62, 59, 59, 59, 58, 63, 65, + 68, 73, 79, 82, 85, 92, 98, 101, 105, 111, 118, 119, 121, 126, 130, 131, + 128, 127, 131, 136, 138, 137, 136, 136, 87, 82, 78, 78, 77, 75, 79, 82, + 84, 89, 95, 98, 102, 109, 116, 120, 124, 132, 141, 142, 144, 149, 148, + 153, 157, 152, 150, 155, 161, 159, 157, 156, 93, 88, 86, 84, 82, 82, 80, + 84, 86, 91, 94, 98, 105, 107, 112, 119, 122, 130, 135, 140, 149, 153, + 162, 165, 167, 173, 174, 177, 183, 185, 182, 179, 99, 94, 93, 90, 89, + 89, 88, 87, 90, 93, 97, 99, 105, 107, 115, 116, 124, 127, 135, 139, 146, + 152, 159, 166, 171, 182, 186, 191, 193, 201, 203, 204] + ], + [ /* Chroma */ + /* Size 4x4 */ + [35, 46, 57, 66, 46, 60, 69, 71, 57, 69, 90, 90, 66, 71, 90, 109], + /* Size 8x8 */ + [31, 38, 47, 50, 57, 63, 67, 71, 38, 47, 46, 47, 52, 57, 62, 67, 47, 46, + 54, 57, 61, 66, 67, 68, 50, 47, 57, 66, 72, 77, 75, 75, 57, 52, 61, 72, + 82, 88, 86, 84, 63, 57, 66, 77, 88, 96, 95, 95, 67, 62, 67, 75, 86, 95, + 104, 107, 71, 67, 68, 75, 84, 95, 107, 113], + /* Size 16x16 */ + [32, 30, 33, 41, 49, 49, 50, 54, 57, 63, 65, 68, 70, 72, 74, 76, 30, 32, + 35, 42, 46, 45, 46, 49, 52, 57, 58, 62, 64, 67, 70, 72, 33, 35, 39, 45, + 47, 45, 46, 49, 51, 56, 57, 60, 62, 64, 66, 69, 41, 42, 45, 48, 50, 49, + 50, 52, 53, 57, 58, 59, 60, 61, 64, 67, 49, 46, 47, 50, 53, 53, 54, 55, + 56, 60, 61, 64, 64, 65, 66, 66, 49, 45, 45, 49, 53, 58, 60, 62, 63, 67, + 68, 67, 69, 68, 70, 70, 50, 46, 46, 50, 54, 60, 61, 65, 67, 71, 71, 74, + 73, 73, 74, 74, 54, 49, 49, 52, 55, 62, 65, 71, 73, 78, 79, 78, 77, 78, + 78, 78, 57, 52, 51, 53, 56, 63, 67, 73, 76, 82, 83, 84, 84, 84, 82, 83, + 63, 57, 56, 57, 60, 67, 71, 78, 82, 89, 90, 90, 89, 88, 87, 88, 65, 58, + 57, 58, 61, 68, 71, 79, 83, 90, 91, 94, 93, 93, 92, 93, 68, 62, 60, 59, + 64, 67, 74, 78, 84, 90, 94, 98, 99, 98, 98, 98, 70, 64, 62, 60, 64, 69, + 73, 77, 84, 89, 93, 99, 102, 103, 104, 104, 72, 67, 64, 61, 65, 68, 73, + 78, 84, 88, 93, 98, 103, 106, 108, 109, 74, 70, 66, 64, 66, 70, 74, 78, + 82, 87, 92, 98, 104, 108, 111, 112, 76, 72, 69, 67, 66, 70, 74, 78, 83, + 88, 93, 98, 104, 109, 112, 116], + /* Size 32x32 */ + [32, 31, 30, 32, 33, 36, 41, 45, 49, 48, 49, 50, 50, 52, 54, 56, 57, 60, + 63, 64, 65, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 78, 31, 31, 31, 33, + 34, 38, 42, 45, 47, 47, 47, 47, 48, 50, 52, 53, 54, 57, 60, 61, 61, 63, + 64, 65, 66, 67, 68, 69, 70, 71, 72, 74, 30, 31, 32, 33, 35, 40, 42, 44, + 46, 45, 45, 45, 46, 47, 49, 51, 52, 54, 57, 58, 58, 61, 62, 63, 64, 66, + 67, 68, 70, 71, 72, 74, 32, 33, 33, 35, 37, 41, 43, 45, 47, 46, 45, 46, + 46, 47, 49, 50, 51, 54, 57, 57, 58, 60, 61, 62, 63, 64, 65, 66, 67, 68, + 69, 70, 33, 34, 35, 37, 39, 43, 45, 46, 47, 46, 45, 46, 46, 47, 49, 50, + 51, 53, 56, 57, 57, 59, 60, 61, 62, 63, 64, 65, 66, 68, 69, 70, 36, 38, + 40, 41, 43, 47, 47, 47, 48, 46, 45, 46, 46, 47, 48, 49, 50, 52, 54, 55, + 55, 57, 58, 59, 61, 62, 64, 65, 66, 67, 68, 69, 41, 42, 42, 43, 45, 47, + 48, 49, 50, 49, 49, 49, 50, 50, 52, 52, 53, 55, 57, 58, 58, 60, 59, 59, + 60, 61, 61, 63, 64, 66, 67, 69, 45, 45, 44, 45, 46, 47, 49, 50, 51, 51, + 51, 51, 52, 52, 53, 54, 55, 57, 59, 59, 60, 61, 61, 62, 63, 63, 63, 63, + 63, 64, 65, 66, 49, 47, 46, 47, 47, 48, 50, 51, 53, 53, 53, 54, 54, 54, + 55, 56, 56, 58, 60, 61, 61, 63, 64, 64, 64, 64, 65, 66, 66, 66, 66, 66, + 48, 47, 45, 46, 46, 46, 49, 51, 53, 54, 55, 56, 56, 57, 58, 59, 60, 61, + 63, 64, 64, 66, 66, 65, 66, 67, 67, 67, 67, 68, 69, 70, 49, 47, 45, 45, + 45, 45, 49, 51, 53, 55, 58, 59, 60, 61, 62, 63, 63, 65, 67, 67, 68, 69, + 67, 68, 69, 68, 68, 69, 70, 70, 70, 70, 50, 47, 45, 46, 46, 46, 49, 51, + 54, 56, 59, 60, 60, 62, 64, 64, 65, 67, 69, 69, 70, 70, 71, 71, 70, 70, + 71, 71, 71, 71, 72, 74, 50, 48, 46, 46, 46, 46, 50, 52, 54, 56, 60, 60, + 61, 63, 65, 66, 67, 68, 71, 71, 71, 73, 74, 72, 73, 74, 73, 73, 74, 74, + 74, 74, 52, 50, 47, 47, 47, 47, 50, 52, 54, 57, 61, 62, 63, 66, 68, 69, + 70, 72, 75, 75, 75, 77, 75, 75, 76, 75, 75, 76, 75, 75, 76, 77, 54, 52, + 49, 49, 49, 48, 52, 53, 55, 58, 62, 64, 65, 68, 71, 72, 73, 75, 78, 78, + 79, 79, 78, 79, 77, 78, 78, 77, 78, 79, 78, 78, 56, 53, 51, 50, 50, 49, + 52, 54, 56, 59, 63, 64, 66, 69, 72, 73, 75, 77, 80, 80, 81, 81, 82, 80, + 81, 81, 79, 81, 80, 79, 81, 82, 57, 54, 52, 51, 51, 50, 53, 55, 56, 60, + 63, 65, 67, 70, 73, 75, 76, 79, 82, 82, 83, 85, 84, 83, 84, 83, 84, 82, + 82, 84, 83, 82, 60, 57, 54, 54, 53, 52, 55, 57, 58, 61, 65, 67, 68, 72, + 75, 77, 79, 82, 85, 85, 86, 88, 86, 87, 85, 86, 85, 85, 86, 84, 85, 86, + 63, 60, 57, 57, 56, 54, 57, 59, 60, 63, 67, 69, 71, 75, 78, 80, 82, 85, + 89, 89, 90, 90, 90, 89, 89, 88, 88, 88, 87, 88, 88, 87, 64, 61, 58, 57, + 57, 55, 58, 59, 61, 64, 67, 69, 71, 75, 78, 80, 82, 85, 89, 90, 91, 92, + 93, 92, 92, 91, 91, 90, 91, 90, 90, 92, 65, 61, 58, 58, 57, 55, 58, 60, + 61, 64, 68, 70, 71, 75, 79, 81, 83, 86, 90, 91, 91, 94, 94, 96, 93, 94, + 93, 94, 92, 93, 93, 92, 67, 63, 61, 60, 59, 57, 60, 61, 63, 66, 69, 70, + 73, 77, 79, 81, 85, 88, 90, 92, 94, 96, 96, 97, 98, 95, 97, 95, 96, 95, + 95, 96, 68, 64, 62, 61, 60, 58, 59, 61, 64, 66, 67, 71, 74, 75, 78, 82, + 84, 86, 90, 93, 94, 96, 98, 98, 99, 100, 98, 99, 98, 98, 98, 97, 69, 65, + 63, 62, 61, 59, 59, 62, 64, 65, 68, 71, 72, 75, 79, 80, 83, 87, 89, 92, + 96, 97, 98, 100, 100, 101, 102, 101, 101, 101, 100, 102, 70, 66, 64, 63, + 62, 61, 60, 63, 64, 66, 69, 70, 73, 76, 77, 81, 84, 85, 89, 92, 93, 98, + 99, 100, 102, 102, 103, 104, 104, 103, 104, 102, 71, 67, 66, 64, 63, 62, + 61, 63, 64, 67, 68, 70, 74, 75, 78, 81, 83, 86, 88, 91, 94, 95, 100, + 101, 102, 104, 104, 105, 106, 107, 105, 107, 72, 68, 67, 65, 64, 64, 61, + 63, 65, 67, 68, 71, 73, 75, 78, 79, 84, 85, 88, 91, 93, 97, 98, 102, + 103, 104, 106, 106, 108, 108, 109, 107, 73, 69, 68, 66, 65, 65, 63, 63, + 66, 67, 69, 71, 73, 76, 77, 81, 82, 85, 88, 90, 94, 95, 99, 101, 104, + 105, 106, 109, 108, 110, 111, 112, 74, 70, 70, 67, 66, 66, 64, 63, 66, + 67, 70, 71, 74, 75, 78, 80, 82, 86, 87, 91, 92, 96, 98, 101, 104, 106, + 108, 108, 111, 111, 112, 113, 75, 71, 71, 68, 68, 67, 66, 64, 66, 68, + 70, 71, 74, 75, 79, 79, 84, 84, 88, 90, 93, 95, 98, 101, 103, 107, 108, + 110, 111, 113, 113, 115, 76, 72, 72, 69, 69, 68, 67, 65, 66, 69, 70, 72, + 74, 76, 78, 81, 83, 85, 88, 90, 93, 95, 98, 100, 104, 105, 109, 111, + 112, 113, 116, 115, 78, 74, 74, 70, 70, 69, 69, 66, 66, 70, 70, 74, 74, + 77, 78, 82, 82, 86, 87, 92, 92, 96, 97, 102, 102, 107, 107, 112, 113, + 115, 115, 118], + /* Size 4x8 */ + [31, 47, 60, 66, 40, 45, 54, 61, 46, 56, 64, 64, 48, 61, 75, 73, 54, 65, + 85, 82, 61, 69, 92, 92, 64, 68, 90, 102, 68, 71, 87, 105], + /* Size 8x4 */ + [31, 40, 46, 48, 54, 61, 64, 68, 47, 45, 56, 61, 65, 69, 68, 71, 60, 54, + 64, 75, 85, 92, 90, 87, 66, 61, 64, 73, 82, 92, 102, 105], + /* Size 8x16 */ + [32, 37, 48, 52, 57, 66, 68, 71, 30, 40, 46, 48, 52, 60, 63, 66, 33, 43, + 47, 47, 51, 59, 60, 63, 42, 47, 50, 50, 53, 60, 59, 62, 49, 48, 53, 54, + 57, 62, 62, 62, 49, 46, 53, 61, 64, 69, 66, 66, 50, 46, 54, 64, 67, 73, + 72, 70, 54, 49, 55, 68, 73, 80, 76, 75, 57, 50, 56, 70, 76, 84, 80, 79, + 63, 55, 60, 75, 82, 92, 87, 84, 64, 56, 61, 75, 83, 93, 93, 89, 68, 59, + 64, 74, 86, 94, 98, 94, 70, 62, 66, 73, 83, 96, 99, 98, 72, 64, 66, 75, + 83, 92, 101, 104, 74, 67, 66, 74, 84, 94, 103, 106, 76, 69, 67, 73, 82, + 91, 101, 109], + /* Size 16x8 */ + [32, 30, 33, 42, 49, 49, 50, 54, 57, 63, 64, 68, 70, 72, 74, 76, 37, 40, + 43, 47, 48, 46, 46, 49, 50, 55, 56, 59, 62, 64, 67, 69, 48, 46, 47, 50, + 53, 53, 54, 55, 56, 60, 61, 64, 66, 66, 66, 67, 52, 48, 47, 50, 54, 61, + 64, 68, 70, 75, 75, 74, 73, 75, 74, 73, 57, 52, 51, 53, 57, 64, 67, 73, + 76, 82, 83, 86, 83, 83, 84, 82, 66, 60, 59, 60, 62, 69, 73, 80, 84, 92, + 93, 94, 96, 92, 94, 91, 68, 63, 60, 59, 62, 66, 72, 76, 80, 87, 93, 98, + 99, 101, 103, 101, 71, 66, 63, 62, 62, 66, 70, 75, 79, 84, 89, 94, 98, + 104, 106, 109], + /* Size 16x32 */ + [32, 31, 37, 42, 48, 49, 52, 54, 57, 63, 66, 67, 68, 69, 71, 72, 31, 31, + 38, 42, 47, 47, 50, 52, 54, 60, 63, 64, 65, 66, 67, 68, 30, 32, 40, 42, + 46, 45, 48, 50, 52, 57, 60, 62, 63, 65, 66, 68, 32, 34, 41, 44, 46, 45, + 48, 49, 51, 57, 59, 61, 62, 63, 64, 65, 33, 36, 43, 45, 47, 46, 47, 49, + 51, 56, 59, 60, 60, 62, 63, 65, 37, 40, 47, 47, 47, 45, 47, 48, 50, 54, + 57, 58, 60, 61, 62, 63, 42, 43, 47, 48, 50, 49, 50, 52, 53, 57, 60, 58, + 59, 60, 62, 63, 45, 44, 47, 49, 51, 51, 52, 54, 55, 59, 61, 61, 61, 60, + 61, 61, 49, 46, 48, 50, 53, 53, 54, 55, 57, 60, 62, 63, 62, 63, 62, 62, + 48, 46, 47, 50, 53, 56, 57, 59, 60, 64, 66, 65, 65, 64, 64, 65, 49, 45, + 46, 49, 53, 58, 61, 62, 64, 67, 69, 67, 66, 66, 66, 65, 49, 46, 46, 49, + 53, 59, 62, 64, 65, 69, 71, 70, 68, 68, 67, 68, 50, 46, 46, 50, 54, 59, + 64, 65, 67, 71, 73, 72, 72, 70, 70, 69, 52, 48, 47, 50, 54, 61, 66, 68, + 71, 75, 77, 74, 73, 73, 71, 72, 54, 50, 49, 52, 55, 62, 68, 71, 73, 78, + 80, 78, 76, 74, 75, 73, 55, 51, 49, 52, 56, 63, 69, 72, 75, 80, 82, 80, + 79, 78, 76, 77, 57, 52, 50, 53, 56, 64, 70, 73, 76, 82, 84, 82, 80, 80, + 79, 77, 60, 54, 52, 55, 58, 65, 72, 75, 79, 85, 88, 86, 84, 82, 81, 81, + 63, 57, 55, 58, 60, 67, 75, 78, 82, 89, 92, 88, 87, 85, 84, 81, 64, 58, + 55, 58, 61, 68, 75, 78, 82, 89, 92, 90, 89, 87, 86, 86, 64, 59, 56, 58, + 61, 68, 75, 79, 83, 90, 93, 95, 93, 91, 89, 87, 67, 61, 58, 60, 63, 69, + 76, 79, 85, 92, 95, 96, 94, 92, 91, 91, 68, 62, 59, 60, 64, 71, 74, 78, + 86, 91, 94, 96, 98, 96, 94, 91, 69, 62, 60, 60, 65, 70, 72, 79, 85, 88, + 95, 98, 99, 98, 97, 96, 70, 63, 62, 60, 66, 69, 73, 81, 83, 89, 96, 97, + 99, 101, 98, 97, 71, 64, 63, 61, 67, 68, 74, 79, 82, 90, 93, 98, 102, + 102, 102, 101, 72, 65, 64, 62, 66, 68, 75, 78, 83, 89, 92, 100, 101, + 103, 104, 102, 73, 66, 65, 63, 66, 69, 75, 76, 84, 87, 93, 98, 102, 105, + 106, 107, 74, 67, 67, 64, 66, 70, 74, 77, 84, 86, 94, 96, 103, 105, 106, + 107, 75, 68, 68, 65, 66, 71, 74, 78, 83, 87, 93, 96, 103, 105, 109, 109, + 76, 69, 69, 66, 67, 72, 73, 80, 82, 88, 91, 97, 101, 107, 109, 110, 77, + 70, 70, 67, 67, 73, 73, 81, 81, 90, 90, 99, 99, 108, 108, 113], + /* Size 32x16 */ + [32, 31, 30, 32, 33, 37, 42, 45, 49, 48, 49, 49, 50, 52, 54, 55, 57, 60, + 63, 64, 64, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 31, 31, 32, 34, + 36, 40, 43, 44, 46, 46, 45, 46, 46, 48, 50, 51, 52, 54, 57, 58, 59, 61, + 62, 62, 63, 64, 65, 66, 67, 68, 69, 70, 37, 38, 40, 41, 43, 47, 47, 47, + 48, 47, 46, 46, 46, 47, 49, 49, 50, 52, 55, 55, 56, 58, 59, 60, 62, 63, + 64, 65, 67, 68, 69, 70, 42, 42, 42, 44, 45, 47, 48, 49, 50, 50, 49, 49, + 50, 50, 52, 52, 53, 55, 58, 58, 58, 60, 60, 60, 60, 61, 62, 63, 64, 65, + 66, 67, 48, 47, 46, 46, 47, 47, 50, 51, 53, 53, 53, 53, 54, 54, 55, 56, + 56, 58, 60, 61, 61, 63, 64, 65, 66, 67, 66, 66, 66, 66, 67, 67, 49, 47, + 45, 45, 46, 45, 49, 51, 53, 56, 58, 59, 59, 61, 62, 63, 64, 65, 67, 68, + 68, 69, 71, 70, 69, 68, 68, 69, 70, 71, 72, 73, 52, 50, 48, 48, 47, 47, + 50, 52, 54, 57, 61, 62, 64, 66, 68, 69, 70, 72, 75, 75, 75, 76, 74, 72, + 73, 74, 75, 75, 74, 74, 73, 73, 54, 52, 50, 49, 49, 48, 52, 54, 55, 59, + 62, 64, 65, 68, 71, 72, 73, 75, 78, 78, 79, 79, 78, 79, 81, 79, 78, 76, + 77, 78, 80, 81, 57, 54, 52, 51, 51, 50, 53, 55, 57, 60, 64, 65, 67, 71, + 73, 75, 76, 79, 82, 82, 83, 85, 86, 85, 83, 82, 83, 84, 84, 83, 82, 81, + 63, 60, 57, 57, 56, 54, 57, 59, 60, 64, 67, 69, 71, 75, 78, 80, 82, 85, + 89, 89, 90, 92, 91, 88, 89, 90, 89, 87, 86, 87, 88, 90, 66, 63, 60, 59, + 59, 57, 60, 61, 62, 66, 69, 71, 73, 77, 80, 82, 84, 88, 92, 92, 93, 95, + 94, 95, 96, 93, 92, 93, 94, 93, 91, 90, 67, 64, 62, 61, 60, 58, 58, 61, + 63, 65, 67, 70, 72, 74, 78, 80, 82, 86, 88, 90, 95, 96, 96, 98, 97, 98, + 100, 98, 96, 96, 97, 99, 68, 65, 63, 62, 60, 60, 59, 61, 62, 65, 66, 68, + 72, 73, 76, 79, 80, 84, 87, 89, 93, 94, 98, 99, 99, 102, 101, 102, 103, + 103, 101, 99, 69, 66, 65, 63, 62, 61, 60, 60, 63, 64, 66, 68, 70, 73, + 74, 78, 80, 82, 85, 87, 91, 92, 96, 98, 101, 102, 103, 105, 105, 105, + 107, 108, 71, 67, 66, 64, 63, 62, 62, 61, 62, 64, 66, 67, 70, 71, 75, + 76, 79, 81, 84, 86, 89, 91, 94, 97, 98, 102, 104, 106, 106, 109, 109, + 108, 72, 68, 68, 65, 65, 63, 63, 61, 62, 65, 65, 68, 69, 72, 73, 77, 77, + 81, 81, 86, 87, 91, 91, 96, 97, 101, 102, 107, 107, 109, 110, 113], + /* Size 4x16 */ + [31, 49, 63, 69, 32, 45, 57, 65, 36, 46, 56, 62, 43, 49, 57, 60, 46, 53, + 60, 63, 45, 58, 67, 66, 46, 59, 71, 70, 50, 62, 78, 74, 52, 64, 82, 80, + 57, 67, 89, 85, 59, 68, 90, 91, 62, 71, 91, 96, 63, 69, 89, 101, 65, 68, + 89, 103, 67, 70, 86, 105, 69, 72, 88, 107], + /* Size 16x4 */ + [31, 32, 36, 43, 46, 45, 46, 50, 52, 57, 59, 62, 63, 65, 67, 69, 49, 45, + 46, 49, 53, 58, 59, 62, 64, 67, 68, 71, 69, 68, 70, 72, 63, 57, 56, 57, + 60, 67, 71, 78, 82, 89, 90, 91, 89, 89, 86, 88, 69, 65, 62, 60, 63, 66, + 70, 74, 80, 85, 91, 96, 101, 103, 105, 107], + /* Size 8x32 */ + [32, 37, 48, 52, 57, 66, 68, 71, 31, 38, 47, 50, 54, 63, 65, 67, 30, 40, + 46, 48, 52, 60, 63, 66, 32, 41, 46, 48, 51, 59, 62, 64, 33, 43, 47, 47, + 51, 59, 60, 63, 37, 47, 47, 47, 50, 57, 60, 62, 42, 47, 50, 50, 53, 60, + 59, 62, 45, 47, 51, 52, 55, 61, 61, 61, 49, 48, 53, 54, 57, 62, 62, 62, + 48, 47, 53, 57, 60, 66, 65, 64, 49, 46, 53, 61, 64, 69, 66, 66, 49, 46, + 53, 62, 65, 71, 68, 67, 50, 46, 54, 64, 67, 73, 72, 70, 52, 47, 54, 66, + 71, 77, 73, 71, 54, 49, 55, 68, 73, 80, 76, 75, 55, 49, 56, 69, 75, 82, + 79, 76, 57, 50, 56, 70, 76, 84, 80, 79, 60, 52, 58, 72, 79, 88, 84, 81, + 63, 55, 60, 75, 82, 92, 87, 84, 64, 55, 61, 75, 82, 92, 89, 86, 64, 56, + 61, 75, 83, 93, 93, 89, 67, 58, 63, 76, 85, 95, 94, 91, 68, 59, 64, 74, + 86, 94, 98, 94, 69, 60, 65, 72, 85, 95, 99, 97, 70, 62, 66, 73, 83, 96, + 99, 98, 71, 63, 67, 74, 82, 93, 102, 102, 72, 64, 66, 75, 83, 92, 101, + 104, 73, 65, 66, 75, 84, 93, 102, 106, 74, 67, 66, 74, 84, 94, 103, 106, + 75, 68, 66, 74, 83, 93, 103, 109, 76, 69, 67, 73, 82, 91, 101, 109, 77, + 70, 67, 73, 81, 90, 99, 108], + /* Size 32x8 */ + [32, 31, 30, 32, 33, 37, 42, 45, 49, 48, 49, 49, 50, 52, 54, 55, 57, 60, + 63, 64, 64, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 37, 38, 40, 41, + 43, 47, 47, 47, 48, 47, 46, 46, 46, 47, 49, 49, 50, 52, 55, 55, 56, 58, + 59, 60, 62, 63, 64, 65, 67, 68, 69, 70, 48, 47, 46, 46, 47, 47, 50, 51, + 53, 53, 53, 53, 54, 54, 55, 56, 56, 58, 60, 61, 61, 63, 64, 65, 66, 67, + 66, 66, 66, 66, 67, 67, 52, 50, 48, 48, 47, 47, 50, 52, 54, 57, 61, 62, + 64, 66, 68, 69, 70, 72, 75, 75, 75, 76, 74, 72, 73, 74, 75, 75, 74, 74, + 73, 73, 57, 54, 52, 51, 51, 50, 53, 55, 57, 60, 64, 65, 67, 71, 73, 75, + 76, 79, 82, 82, 83, 85, 86, 85, 83, 82, 83, 84, 84, 83, 82, 81, 66, 63, + 60, 59, 59, 57, 60, 61, 62, 66, 69, 71, 73, 77, 80, 82, 84, 88, 92, 92, + 93, 95, 94, 95, 96, 93, 92, 93, 94, 93, 91, 90, 68, 65, 63, 62, 60, 60, + 59, 61, 62, 65, 66, 68, 72, 73, 76, 79, 80, 84, 87, 89, 93, 94, 98, 99, + 99, 102, 101, 102, 103, 103, 101, 99, 71, 67, 66, 64, 63, 62, 62, 61, + 62, 64, 66, 67, 70, 71, 75, 76, 79, 81, 84, 86, 89, 91, 94, 97, 98, 102, + 104, 106, 106, 109, 109, 108] + ] + ], + [ + [ /* Luma */ + /* Size 4x4 */ + [32, 41, 69, 92, 41, 63, 88, 103, 69, 88, 127, 140, 92, 103, 140, 184], + /* Size 8x8 */ + [32, 32, 37, 47, 62, 78, 90, 102, 32, 35, 39, 46, 58, 72, 84, 96, 37, 39, + 51, 60, 71, 84, 93, 100, 47, 46, 60, 73, 87, 100, 106, 113, 62, 58, 71, + 87, 105, 121, 129, 132, 78, 72, 84, 100, 121, 140, 148, 155, 90, 84, 93, + 106, 129, 148, 169, 183, 102, 96, 100, 113, 132, 155, 183, 201], + /* Size 16x16 */ + [32, 31, 31, 32, 36, 39, 47, 54, 61, 71, 80, 86, 92, 98, 104, 111, 31, + 32, 32, 33, 34, 37, 44, 50, 56, 65, 73, 79, 85, 91, 98, 105, 31, 32, 33, + 34, 36, 39, 45, 50, 56, 64, 71, 77, 82, 88, 94, 100, 32, 33, 34, 36, 40, + 42, 47, 51, 57, 65, 71, 76, 80, 85, 91, 98, 36, 34, 36, 40, 48, 50, 56, + 60, 65, 73, 79, 84, 86, 90, 95, 98, 39, 37, 39, 42, 50, 54, 60, 65, 70, + 78, 84, 89, 95, 96, 102, 105, 47, 44, 45, 47, 56, 60, 69, 75, 81, 89, + 95, 100, 102, 104, 109, 112, 54, 50, 50, 51, 60, 65, 75, 82, 89, 97, + 104, 109, 110, 114, 117, 121, 61, 56, 56, 57, 65, 70, 81, 89, 97, 106, + 113, 119, 122, 126, 125, 130, 71, 65, 64, 65, 73, 78, 89, 97, 106, 117, + 125, 131, 134, 134, 136, 141, 80, 73, 71, 71, 79, 84, 95, 104, 113, 125, + 134, 140, 142, 145, 146, 152, 86, 79, 77, 76, 84, 89, 100, 109, 119, + 131, 140, 147, 154, 157, 160, 165, 92, 85, 82, 80, 86, 95, 102, 110, + 122, 134, 142, 154, 162, 168, 174, 178, 98, 91, 88, 85, 90, 96, 104, + 114, 126, 134, 145, 157, 168, 176, 184, 193, 104, 98, 94, 91, 95, 102, + 109, 117, 125, 136, 146, 160, 174, 184, 193, 201, 111, 105, 100, 98, 98, + 105, 112, 121, 130, 141, 152, 165, 178, 193, 201, 210], + /* Size 32x32 */ + [32, 31, 31, 31, 31, 32, 32, 34, 36, 38, 39, 44, 47, 49, 54, 59, 61, 65, + 71, 76, 80, 83, 86, 89, 92, 95, 98, 101, 104, 108, 111, 114, 31, 32, 32, + 32, 32, 32, 33, 34, 35, 37, 38, 42, 45, 47, 51, 56, 58, 62, 68, 72, 76, + 78, 82, 85, 88, 90, 93, 96, 99, 102, 105, 109, 31, 32, 32, 32, 32, 32, + 33, 33, 34, 36, 37, 41, 44, 46, 50, 54, 56, 60, 65, 70, 73, 76, 79, 82, + 85, 88, 91, 95, 98, 101, 105, 109, 31, 32, 32, 32, 32, 33, 33, 34, 35, + 36, 38, 41, 44, 45, 49, 54, 56, 59, 65, 69, 72, 75, 78, 81, 84, 86, 89, + 92, 95, 98, 101, 104, 31, 32, 32, 32, 33, 34, 34, 35, 36, 38, 39, 42, + 45, 46, 50, 54, 56, 59, 64, 68, 71, 74, 77, 79, 82, 85, 88, 91, 94, 97, + 100, 104, 32, 32, 32, 33, 34, 35, 36, 37, 38, 39, 40, 42, 45, 46, 49, + 53, 55, 58, 63, 66, 69, 72, 74, 78, 81, 84, 87, 90, 93, 96, 99, 102, 32, + 33, 33, 33, 34, 36, 36, 38, 40, 41, 42, 44, 47, 48, 51, 55, 57, 60, 65, + 68, 71, 73, 76, 78, 80, 82, 85, 88, 91, 95, 98, 102, 34, 34, 33, 34, 35, + 37, 38, 39, 42, 44, 45, 47, 50, 51, 54, 58, 60, 63, 68, 71, 74, 76, 79, + 82, 85, 86, 87, 88, 90, 93, 96, 99, 36, 35, 34, 35, 36, 38, 40, 42, 48, + 50, 50, 54, 56, 57, 60, 64, 65, 68, 73, 76, 79, 81, 84, 86, 86, 88, 90, + 93, 95, 97, 98, 100, 38, 37, 36, 36, 38, 39, 41, 44, 50, 51, 52, 56, 58, + 60, 63, 67, 68, 71, 76, 79, 82, 84, 87, 87, 90, 93, 94, 95, 96, 100, + 103, 106, 39, 38, 37, 38, 39, 40, 42, 45, 50, 52, 54, 58, 60, 62, 65, + 69, 70, 73, 78, 81, 84, 86, 89, 92, 95, 95, 96, 99, 102, 104, 105, 106, + 44, 42, 41, 41, 42, 42, 44, 47, 54, 56, 58, 63, 66, 68, 71, 75, 77, 79, + 84, 88, 90, 92, 95, 97, 97, 99, 102, 103, 103, 106, 109, 113, 47, 45, + 44, 44, 45, 45, 47, 50, 56, 58, 60, 66, 69, 71, 75, 79, 81, 84, 89, 92, + 95, 97, 100, 100, 102, 105, 104, 106, 109, 111, 112, 113, 49, 47, 46, + 45, 46, 46, 48, 51, 57, 60, 62, 68, 71, 73, 77, 81, 83, 87, 92, 95, 98, + 100, 103, 105, 107, 106, 109, 112, 112, 113, 117, 120, 54, 51, 50, 49, + 50, 49, 51, 54, 60, 63, 65, 71, 75, 77, 82, 87, 89, 92, 97, 101, 104, + 106, 109, 112, 110, 113, 114, 114, 117, 121, 121, 121, 59, 56, 54, 54, + 54, 53, 55, 58, 64, 67, 69, 75, 79, 81, 87, 92, 94, 98, 103, 107, 110, + 113, 116, 114, 117, 118, 117, 121, 122, 122, 125, 129, 61, 58, 56, 56, + 56, 55, 57, 60, 65, 68, 70, 77, 81, 83, 89, 94, 97, 101, 106, 110, 113, + 116, 119, 120, 122, 121, 126, 124, 125, 130, 130, 130, 65, 62, 60, 59, + 59, 58, 60, 63, 68, 71, 73, 79, 84, 87, 92, 98, 101, 105, 111, 115, 118, + 121, 124, 128, 125, 129, 128, 131, 133, 132, 135, 139, 71, 68, 65, 65, + 64, 63, 65, 68, 73, 76, 78, 84, 89, 92, 97, 103, 106, 111, 117, 122, + 125, 128, 131, 131, 134, 132, 134, 136, 136, 140, 141, 140, 76, 72, 70, + 69, 68, 66, 68, 71, 76, 79, 81, 88, 92, 95, 101, 107, 110, 115, 122, + 127, 130, 133, 136, 136, 138, 139, 141, 140, 145, 143, 146, 151, 80, 76, + 73, 72, 71, 69, 71, 74, 79, 82, 84, 90, 95, 98, 104, 110, 113, 118, 125, + 130, 134, 137, 140, 146, 142, 146, 145, 149, 146, 150, 152, 151, 83, 78, + 76, 75, 74, 72, 73, 76, 81, 84, 86, 92, 97, 100, 106, 113, 116, 121, + 128, 133, 137, 140, 144, 147, 152, 148, 154, 151, 156, 155, 156, 162, + 86, 82, 79, 78, 77, 74, 76, 79, 84, 87, 89, 95, 100, 103, 109, 116, 119, + 124, 131, 136, 140, 144, 147, 150, 154, 159, 157, 160, 160, 162, 165, + 162, 89, 85, 82, 81, 79, 78, 78, 82, 86, 87, 92, 97, 100, 105, 112, 114, + 120, 128, 131, 136, 146, 147, 150, 155, 156, 161, 166, 165, 167, 169, + 169, 175, 92, 88, 85, 84, 82, 81, 80, 85, 86, 90, 95, 97, 102, 107, 110, + 117, 122, 125, 134, 138, 142, 152, 154, 156, 162, 163, 168, 173, 174, + 174, 178, 176, 95, 90, 88, 86, 85, 84, 82, 86, 88, 93, 95, 99, 105, 106, + 113, 118, 121, 129, 132, 139, 146, 148, 159, 161, 163, 169, 170, 176, + 180, 183, 181, 187, 98, 93, 91, 89, 88, 87, 85, 87, 90, 94, 96, 102, + 104, 109, 114, 117, 126, 128, 134, 141, 145, 154, 157, 166, 168, 170, + 176, 178, 184, 188, 193, 188, 101, 96, 95, 92, 91, 90, 88, 88, 93, 95, + 99, 103, 106, 112, 114, 121, 124, 131, 136, 140, 149, 151, 160, 165, + 173, 176, 178, 184, 186, 192, 196, 203, 104, 99, 98, 95, 94, 93, 91, 90, + 95, 96, 102, 103, 109, 112, 117, 122, 125, 133, 136, 145, 146, 156, 160, + 167, 174, 180, 184, 186, 193, 194, 201, 204, 108, 102, 101, 98, 97, 96, + 95, 93, 97, 100, 104, 106, 111, 113, 121, 122, 130, 132, 140, 143, 150, + 155, 162, 169, 174, 183, 188, 192, 194, 201, 202, 210, 111, 105, 105, + 101, 100, 99, 98, 96, 98, 103, 105, 109, 112, 117, 121, 125, 130, 135, + 141, 146, 152, 156, 165, 169, 178, 181, 193, 196, 201, 202, 210, 211, + 114, 109, 109, 104, 104, 102, 102, 99, 100, 106, 106, 113, 113, 120, + 121, 129, 130, 139, 140, 151, 151, 162, 162, 175, 176, 187, 188, 203, + 204, 210, 211, 219], + /* Size 4x8 */ + [32, 42, 69, 88, 33, 42, 64, 83, 36, 56, 77, 88, 46, 67, 93, 105, 60, 79, + 112, 122, 75, 92, 130, 144, 86, 95, 136, 167, 98, 105, 136, 177], + /* Size 8x4 */ + [32, 33, 36, 46, 60, 75, 86, 98, 42, 42, 56, 67, 79, 92, 95, 105, 69, 64, + 77, 93, 112, 130, 136, 136, 88, 83, 88, 105, 122, 144, 167, 177], + /* Size 8x16 */ + [32, 32, 36, 47, 65, 79, 90, 96, 31, 32, 35, 44, 60, 72, 84, 90, 32, 34, + 36, 45, 59, 71, 80, 87, 32, 35, 40, 47, 60, 71, 78, 85, 36, 37, 48, 56, + 68, 78, 83, 87, 39, 40, 50, 60, 73, 84, 91, 94, 47, 45, 56, 69, 84, 95, + 101, 101, 53, 50, 60, 75, 92, 103, 108, 110, 61, 56, 65, 81, 100, 113, + 116, 118, 71, 64, 73, 89, 111, 125, 129, 129, 79, 70, 79, 95, 118, 133, + 142, 138, 86, 76, 84, 100, 124, 140, 153, 150, 92, 82, 89, 101, 121, + 148, 157, 161, 98, 88, 93, 108, 124, 141, 163, 174, 104, 94, 95, 110, + 129, 151, 171, 181, 110, 100, 98, 111, 127, 147, 169, 188], + /* Size 16x8 */ + [32, 31, 32, 32, 36, 39, 47, 53, 61, 71, 79, 86, 92, 98, 104, 110, 32, + 32, 34, 35, 37, 40, 45, 50, 56, 64, 70, 76, 82, 88, 94, 100, 36, 35, 36, + 40, 48, 50, 56, 60, 65, 73, 79, 84, 89, 93, 95, 98, 47, 44, 45, 47, 56, + 60, 69, 75, 81, 89, 95, 100, 101, 108, 110, 111, 65, 60, 59, 60, 68, 73, + 84, 92, 100, 111, 118, 124, 121, 124, 129, 127, 79, 72, 71, 71, 78, 84, + 95, 103, 113, 125, 133, 140, 148, 141, 151, 147, 90, 84, 80, 78, 83, 91, + 101, 108, 116, 129, 142, 153, 157, 163, 171, 169, 96, 90, 87, 85, 87, + 94, 101, 110, 118, 129, 138, 150, 161, 174, 181, 188], + /* Size 16x32 */ + [32, 31, 32, 32, 36, 44, 47, 53, 65, 73, 79, 87, 90, 93, 96, 99, 31, 32, + 32, 33, 35, 42, 45, 51, 62, 69, 75, 83, 86, 88, 91, 94, 31, 32, 32, 33, + 35, 41, 44, 49, 60, 67, 72, 80, 84, 87, 90, 94, 31, 32, 33, 33, 35, 41, + 44, 49, 59, 66, 71, 79, 82, 84, 87, 90, 32, 32, 34, 34, 36, 42, 45, 50, + 59, 65, 71, 78, 80, 83, 87, 90, 32, 33, 35, 36, 38, 42, 45, 49, 58, 64, + 69, 76, 80, 83, 86, 88, 32, 33, 35, 36, 40, 44, 47, 51, 60, 66, 71, 76, + 78, 81, 85, 89, 34, 34, 36, 38, 42, 48, 50, 54, 63, 69, 73, 80, 82, 81, + 84, 86, 36, 34, 37, 40, 48, 54, 56, 60, 68, 74, 78, 84, 83, 86, 87, 87, + 38, 36, 39, 41, 49, 56, 58, 63, 71, 77, 81, 86, 88, 88, 90, 93, 39, 37, + 40, 42, 50, 58, 60, 65, 73, 79, 84, 90, 91, 92, 94, 93, 44, 41, 42, 45, + 53, 63, 66, 71, 79, 85, 90, 96, 94, 96, 96, 99, 47, 44, 45, 47, 56, 66, + 69, 75, 84, 90, 95, 99, 101, 98, 101, 99, 49, 46, 47, 48, 57, 67, 71, + 77, 86, 93, 97, 103, 103, 105, 102, 106, 53, 49, 50, 51, 60, 71, 75, 82, + 92, 99, 103, 111, 108, 107, 110, 107, 58, 54, 54, 55, 63, 75, 79, 87, + 98, 105, 110, 114, 114, 113, 111, 115, 61, 56, 56, 57, 65, 77, 81, 89, + 100, 107, 113, 118, 116, 117, 118, 116, 65, 60, 59, 60, 68, 79, 84, 92, + 105, 112, 118, 126, 124, 122, 121, 124, 71, 65, 64, 65, 73, 84, 89, 97, + 111, 119, 125, 130, 129, 129, 129, 125, 76, 69, 68, 69, 76, 88, 92, 101, + 115, 123, 130, 134, 134, 131, 132, 135, 79, 72, 70, 71, 79, 90, 95, 104, + 118, 127, 133, 143, 142, 141, 138, 136, 82, 75, 73, 74, 81, 92, 97, 106, + 121, 130, 136, 146, 145, 144, 144, 145, 86, 78, 76, 77, 84, 95, 100, + 109, 124, 133, 140, 147, 153, 151, 150, 146, 89, 81, 79, 78, 87, 95, 99, + 112, 124, 130, 145, 152, 156, 157, 156, 158, 92, 84, 82, 80, 89, 95, + 101, 116, 121, 132, 148, 151, 157, 163, 161, 159, 95, 86, 85, 83, 92, + 95, 105, 114, 120, 136, 143, 155, 163, 167, 171, 170, 98, 89, 88, 85, + 93, 95, 108, 113, 124, 136, 141, 160, 163, 169, 174, 171, 101, 92, 91, + 88, 94, 98, 110, 112, 128, 133, 146, 158, 166, 175, 179, 185, 104, 95, + 94, 91, 95, 101, 110, 115, 129, 132, 151, 154, 171, 175, 181, 186, 107, + 98, 97, 94, 96, 105, 110, 119, 128, 136, 149, 156, 173, 177, 188, 192, + 110, 101, 100, 97, 98, 108, 111, 123, 127, 141, 147, 161, 169, 183, 188, + 193, 114, 104, 104, 100, 100, 111, 111, 126, 127, 145, 145, 166, 166, + 189, 190, 201], + /* Size 32x16 */ + [32, 31, 31, 31, 32, 32, 32, 34, 36, 38, 39, 44, 47, 49, 53, 58, 61, 65, + 71, 76, 79, 82, 86, 89, 92, 95, 98, 101, 104, 107, 110, 114, 31, 32, 32, + 32, 32, 33, 33, 34, 34, 36, 37, 41, 44, 46, 49, 54, 56, 60, 65, 69, 72, + 75, 78, 81, 84, 86, 89, 92, 95, 98, 101, 104, 32, 32, 32, 33, 34, 35, + 35, 36, 37, 39, 40, 42, 45, 47, 50, 54, 56, 59, 64, 68, 70, 73, 76, 79, + 82, 85, 88, 91, 94, 97, 100, 104, 32, 33, 33, 33, 34, 36, 36, 38, 40, + 41, 42, 45, 47, 48, 51, 55, 57, 60, 65, 69, 71, 74, 77, 78, 80, 83, 85, + 88, 91, 94, 97, 100, 36, 35, 35, 35, 36, 38, 40, 42, 48, 49, 50, 53, 56, + 57, 60, 63, 65, 68, 73, 76, 79, 81, 84, 87, 89, 92, 93, 94, 95, 96, 98, + 100, 44, 42, 41, 41, 42, 42, 44, 48, 54, 56, 58, 63, 66, 67, 71, 75, 77, + 79, 84, 88, 90, 92, 95, 95, 95, 95, 95, 98, 101, 105, 108, 111, 47, 45, + 44, 44, 45, 45, 47, 50, 56, 58, 60, 66, 69, 71, 75, 79, 81, 84, 89, 92, + 95, 97, 100, 99, 101, 105, 108, 110, 110, 110, 111, 111, 53, 51, 49, 49, + 50, 49, 51, 54, 60, 63, 65, 71, 75, 77, 82, 87, 89, 92, 97, 101, 104, + 106, 109, 112, 116, 114, 113, 112, 115, 119, 123, 126, 65, 62, 60, 59, + 59, 58, 60, 63, 68, 71, 73, 79, 84, 86, 92, 98, 100, 105, 111, 115, 118, + 121, 124, 124, 121, 120, 124, 128, 129, 128, 127, 127, 73, 69, 67, 66, + 65, 64, 66, 69, 74, 77, 79, 85, 90, 93, 99, 105, 107, 112, 119, 123, + 127, 130, 133, 130, 132, 136, 136, 133, 132, 136, 141, 145, 79, 75, 72, + 71, 71, 69, 71, 73, 78, 81, 84, 90, 95, 97, 103, 110, 113, 118, 125, + 130, 133, 136, 140, 145, 148, 143, 141, 146, 151, 149, 147, 145, 87, 83, + 80, 79, 78, 76, 76, 80, 84, 86, 90, 96, 99, 103, 111, 114, 118, 126, + 130, 134, 143, 146, 147, 152, 151, 155, 160, 158, 154, 156, 161, 166, + 90, 86, 84, 82, 80, 80, 78, 82, 83, 88, 91, 94, 101, 103, 108, 114, 116, + 124, 129, 134, 142, 145, 153, 156, 157, 163, 163, 166, 171, 173, 169, + 166, 93, 88, 87, 84, 83, 83, 81, 81, 86, 88, 92, 96, 98, 105, 107, 113, + 117, 122, 129, 131, 141, 144, 151, 157, 163, 167, 169, 175, 175, 177, + 183, 189, 96, 91, 90, 87, 87, 86, 85, 84, 87, 90, 94, 96, 101, 102, 110, + 111, 118, 121, 129, 132, 138, 144, 150, 156, 161, 171, 174, 179, 181, + 188, 188, 190, 99, 94, 94, 90, 90, 88, 89, 86, 87, 93, 93, 99, 99, 106, + 107, 115, 116, 124, 125, 135, 136, 145, 146, 158, 159, 170, 171, 185, + 186, 192, 193, 201], + /* Size 4x16 */ + [31, 44, 73, 93, 32, 41, 67, 87, 32, 42, 65, 83, 33, 44, 66, 81, 34, 54, + 74, 86, 37, 58, 79, 92, 44, 66, 90, 98, 49, 71, 99, 107, 56, 77, 107, + 117, 65, 84, 119, 129, 72, 90, 127, 141, 78, 95, 133, 151, 84, 95, 132, + 163, 89, 95, 136, 169, 95, 101, 132, 175, 101, 108, 141, 183], + /* Size 16x4 */ + [31, 32, 32, 33, 34, 37, 44, 49, 56, 65, 72, 78, 84, 89, 95, 101, 44, 41, + 42, 44, 54, 58, 66, 71, 77, 84, 90, 95, 95, 95, 101, 108, 73, 67, 65, + 66, 74, 79, 90, 99, 107, 119, 127, 133, 132, 136, 132, 141, 93, 87, 83, + 81, 86, 92, 98, 107, 117, 129, 141, 151, 163, 169, 175, 183], + /* Size 8x32 */ + [32, 32, 36, 47, 65, 79, 90, 96, 31, 32, 35, 45, 62, 75, 86, 91, 31, 32, + 35, 44, 60, 72, 84, 90, 31, 33, 35, 44, 59, 71, 82, 87, 32, 34, 36, 45, + 59, 71, 80, 87, 32, 35, 38, 45, 58, 69, 80, 86, 32, 35, 40, 47, 60, 71, + 78, 85, 34, 36, 42, 50, 63, 73, 82, 84, 36, 37, 48, 56, 68, 78, 83, 87, + 38, 39, 49, 58, 71, 81, 88, 90, 39, 40, 50, 60, 73, 84, 91, 94, 44, 42, + 53, 66, 79, 90, 94, 96, 47, 45, 56, 69, 84, 95, 101, 101, 49, 47, 57, + 71, 86, 97, 103, 102, 53, 50, 60, 75, 92, 103, 108, 110, 58, 54, 63, 79, + 98, 110, 114, 111, 61, 56, 65, 81, 100, 113, 116, 118, 65, 59, 68, 84, + 105, 118, 124, 121, 71, 64, 73, 89, 111, 125, 129, 129, 76, 68, 76, 92, + 115, 130, 134, 132, 79, 70, 79, 95, 118, 133, 142, 138, 82, 73, 81, 97, + 121, 136, 145, 144, 86, 76, 84, 100, 124, 140, 153, 150, 89, 79, 87, 99, + 124, 145, 156, 156, 92, 82, 89, 101, 121, 148, 157, 161, 95, 85, 92, + 105, 120, 143, 163, 171, 98, 88, 93, 108, 124, 141, 163, 174, 101, 91, + 94, 110, 128, 146, 166, 179, 104, 94, 95, 110, 129, 151, 171, 181, 107, + 97, 96, 110, 128, 149, 173, 188, 110, 100, 98, 111, 127, 147, 169, 188, + 114, 104, 100, 111, 127, 145, 166, 190], + /* Size 32x8 */ + [32, 31, 31, 31, 32, 32, 32, 34, 36, 38, 39, 44, 47, 49, 53, 58, 61, 65, + 71, 76, 79, 82, 86, 89, 92, 95, 98, 101, 104, 107, 110, 114, 32, 32, 32, + 33, 34, 35, 35, 36, 37, 39, 40, 42, 45, 47, 50, 54, 56, 59, 64, 68, 70, + 73, 76, 79, 82, 85, 88, 91, 94, 97, 100, 104, 36, 35, 35, 35, 36, 38, + 40, 42, 48, 49, 50, 53, 56, 57, 60, 63, 65, 68, 73, 76, 79, 81, 84, 87, + 89, 92, 93, 94, 95, 96, 98, 100, 47, 45, 44, 44, 45, 45, 47, 50, 56, 58, + 60, 66, 69, 71, 75, 79, 81, 84, 89, 92, 95, 97, 100, 99, 101, 105, 108, + 110, 110, 110, 111, 111, 65, 62, 60, 59, 59, 58, 60, 63, 68, 71, 73, 79, + 84, 86, 92, 98, 100, 105, 111, 115, 118, 121, 124, 124, 121, 120, 124, + 128, 129, 128, 127, 127, 79, 75, 72, 71, 71, 69, 71, 73, 78, 81, 84, 90, + 95, 97, 103, 110, 113, 118, 125, 130, 133, 136, 140, 145, 148, 143, 141, + 146, 151, 149, 147, 145, 90, 86, 84, 82, 80, 80, 78, 82, 83, 88, 91, 94, + 101, 103, 108, 114, 116, 124, 129, 134, 142, 145, 153, 156, 157, 163, + 163, 166, 171, 173, 169, 166, 96, 91, 90, 87, 87, 86, 85, 84, 87, 90, + 94, 96, 101, 102, 110, 111, 118, 121, 129, 132, 138, 144, 150, 156, 161, + 171, 174, 179, 181, 188, 188, 190] + ], + [ /* Chroma */ + /* Size 4x4 */ + [33, 45, 56, 64, 45, 58, 66, 69, 56, 66, 86, 87, 64, 69, 87, 105], + /* Size 8x8 */ + [31, 38, 47, 48, 54, 61, 66, 69, 38, 47, 47, 46, 50, 55, 61, 65, 47, 47, + 53, 55, 58, 63, 65, 66, 48, 46, 55, 62, 67, 72, 73, 73, 54, 50, 58, 67, + 76, 83, 84, 82, 61, 55, 63, 72, 83, 91, 92, 92, 66, 61, 65, 73, 84, 92, + 101, 103, 69, 65, 66, 73, 82, 92, 103, 109], + /* Size 16x16 */ + [32, 30, 33, 38, 49, 48, 50, 52, 55, 60, 63, 66, 68, 70, 72, 74, 30, 31, + 35, 41, 46, 46, 46, 48, 51, 55, 58, 60, 63, 65, 68, 70, 33, 35, 39, 44, + 47, 46, 46, 47, 50, 53, 56, 58, 60, 62, 65, 67, 38, 41, 44, 47, 49, 48, + 47, 48, 50, 53, 55, 58, 58, 60, 62, 65, 49, 46, 47, 49, 53, 53, 54, 54, + 56, 58, 60, 62, 62, 63, 64, 64, 48, 46, 46, 48, 53, 54, 56, 57, 59, 61, + 63, 65, 67, 66, 68, 68, 50, 46, 46, 47, 54, 56, 61, 63, 65, 68, 70, 72, + 71, 71, 72, 72, 52, 48, 47, 48, 54, 57, 63, 66, 69, 72, 75, 76, 75, 76, + 76, 76, 55, 51, 50, 50, 56, 59, 65, 69, 73, 77, 79, 81, 81, 81, 80, 80, + 60, 55, 53, 53, 58, 61, 68, 72, 77, 82, 85, 87, 87, 85, 84, 85, 63, 58, + 56, 55, 60, 63, 70, 75, 79, 85, 89, 91, 91, 90, 89, 90, 66, 60, 58, 58, + 62, 65, 72, 76, 81, 87, 91, 94, 96, 95, 95, 95, 68, 63, 60, 58, 62, 67, + 71, 75, 81, 87, 91, 96, 99, 100, 100, 100, 70, 65, 62, 60, 63, 66, 71, + 76, 81, 85, 90, 95, 100, 103, 104, 105, 72, 68, 65, 62, 64, 68, 72, 76, + 80, 84, 89, 95, 100, 104, 107, 108, 74, 70, 67, 65, 64, 68, 72, 76, 80, + 85, 90, 95, 100, 105, 108, 111], + /* Size 32x32 */ + [32, 31, 30, 31, 33, 36, 38, 41, 49, 49, 48, 49, 50, 51, 52, 54, 55, 57, + 60, 62, 63, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 31, 31, 31, 32, + 34, 38, 40, 42, 47, 47, 47, 47, 48, 48, 50, 52, 53, 54, 57, 59, 60, 61, + 63, 64, 65, 66, 67, 67, 68, 69, 70, 71, 30, 31, 31, 32, 35, 39, 41, 42, + 46, 46, 46, 45, 46, 47, 48, 50, 51, 52, 55, 57, 58, 59, 60, 62, 63, 64, + 65, 67, 68, 69, 70, 71, 31, 32, 32, 33, 36, 40, 41, 43, 46, 46, 45, 45, + 46, 46, 47, 49, 50, 51, 54, 56, 57, 58, 59, 61, 62, 63, 63, 64, 65, 66, + 67, 68, 33, 34, 35, 36, 39, 43, 44, 45, 47, 46, 46, 45, 46, 47, 47, 49, + 50, 51, 53, 55, 56, 57, 58, 59, 60, 61, 62, 63, 65, 66, 67, 68, 36, 38, + 39, 40, 43, 47, 47, 47, 48, 47, 46, 45, 46, 46, 47, 48, 49, 50, 52, 53, + 54, 55, 56, 58, 59, 61, 62, 63, 64, 65, 66, 66, 38, 40, 41, 41, 44, 47, + 47, 48, 49, 48, 48, 47, 47, 47, 48, 49, 50, 51, 53, 54, 55, 56, 58, 58, + 58, 59, 60, 61, 62, 64, 65, 66, 41, 42, 42, 43, 45, 47, 48, 48, 50, 50, + 49, 49, 50, 50, 50, 52, 52, 53, 55, 56, 57, 58, 59, 60, 61, 61, 61, 61, + 62, 63, 63, 64, 49, 47, 46, 46, 47, 48, 49, 50, 53, 53, 53, 53, 54, 54, + 54, 55, 56, 56, 58, 59, 60, 61, 62, 63, 62, 62, 63, 64, 64, 64, 64, 64, + 49, 47, 46, 46, 46, 47, 48, 50, 53, 53, 54, 55, 55, 55, 56, 57, 58, 58, + 60, 61, 62, 63, 64, 64, 64, 65, 65, 65, 65, 66, 67, 68, 48, 47, 46, 45, + 46, 46, 48, 49, 53, 54, 54, 55, 56, 56, 57, 58, 59, 60, 61, 63, 63, 64, + 65, 66, 67, 66, 66, 67, 68, 68, 68, 68, 49, 47, 45, 45, 45, 45, 47, 49, + 53, 55, 55, 58, 59, 60, 61, 62, 63, 63, 65, 66, 67, 68, 69, 69, 68, 68, + 69, 69, 69, 69, 70, 71, 50, 48, 46, 46, 46, 46, 47, 50, 54, 55, 56, 59, + 61, 61, 63, 64, 65, 66, 68, 69, 70, 71, 72, 71, 71, 72, 71, 71, 72, 72, + 72, 71, 51, 48, 47, 46, 47, 46, 47, 50, 54, 55, 56, 60, 61, 62, 64, 66, + 66, 67, 69, 70, 71, 72, 73, 73, 74, 73, 73, 74, 73, 73, 74, 75, 52, 50, + 48, 47, 47, 47, 48, 50, 54, 56, 57, 61, 63, 64, 66, 68, 69, 70, 72, 74, + 75, 75, 76, 77, 75, 76, 76, 75, 76, 77, 76, 75, 54, 52, 50, 49, 49, 48, + 49, 52, 55, 57, 58, 62, 64, 66, 68, 71, 72, 73, 75, 77, 78, 79, 80, 78, + 79, 78, 77, 78, 78, 77, 78, 79, 55, 53, 51, 50, 50, 49, 50, 52, 56, 58, + 59, 63, 65, 66, 69, 72, 73, 74, 77, 78, 79, 80, 81, 81, 81, 80, 81, 80, + 80, 81, 80, 79, 57, 54, 52, 51, 51, 50, 51, 53, 56, 58, 60, 63, 66, 67, + 70, 73, 74, 76, 79, 80, 82, 83, 84, 85, 83, 84, 83, 83, 83, 82, 82, 83, + 60, 57, 55, 54, 53, 52, 53, 55, 58, 60, 61, 65, 68, 69, 72, 75, 77, 79, + 82, 84, 85, 86, 87, 86, 87, 85, 85, 85, 84, 86, 85, 84, 62, 59, 57, 56, + 55, 53, 54, 56, 59, 61, 63, 66, 69, 70, 74, 77, 78, 80, 84, 86, 87, 88, + 90, 89, 89, 88, 88, 87, 88, 87, 87, 88, 63, 60, 58, 57, 56, 54, 55, 57, + 60, 62, 63, 67, 70, 71, 75, 78, 79, 82, 85, 87, 89, 90, 91, 93, 91, 91, + 90, 91, 89, 90, 90, 89, 65, 61, 59, 58, 57, 55, 56, 58, 61, 63, 64, 68, + 71, 72, 75, 79, 80, 83, 86, 88, 90, 91, 93, 94, 95, 92, 94, 92, 93, 92, + 91, 93, 66, 63, 60, 59, 58, 56, 58, 59, 62, 64, 65, 69, 72, 73, 76, 80, + 81, 84, 87, 90, 91, 93, 94, 95, 96, 97, 95, 95, 95, 95, 95, 93, 67, 64, + 62, 61, 59, 58, 58, 60, 63, 64, 66, 69, 71, 73, 77, 78, 81, 85, 86, 89, + 93, 94, 95, 97, 97, 98, 99, 97, 97, 97, 96, 98, 68, 65, 63, 62, 60, 59, + 58, 61, 62, 64, 67, 68, 71, 74, 75, 79, 81, 83, 87, 89, 91, 95, 96, 97, + 99, 98, 100, 100, 100, 99, 100, 98, 69, 66, 64, 63, 61, 61, 59, 61, 62, + 65, 66, 68, 72, 73, 76, 78, 80, 84, 85, 88, 91, 92, 97, 98, 98, 101, + 100, 102, 102, 103, 101, 102, 70, 67, 65, 63, 62, 62, 60, 61, 63, 65, + 66, 69, 71, 73, 76, 77, 81, 83, 85, 88, 90, 94, 95, 99, 100, 100, 103, + 102, 104, 104, 105, 103, 71, 67, 67, 64, 63, 63, 61, 61, 64, 65, 67, 69, + 71, 74, 75, 78, 80, 83, 85, 87, 91, 92, 95, 97, 100, 102, 102, 105, 104, + 106, 106, 108, 72, 68, 68, 65, 65, 64, 62, 62, 64, 65, 68, 69, 72, 73, + 76, 78, 80, 83, 84, 88, 89, 93, 95, 97, 100, 102, 104, 104, 107, 106, + 108, 108, 73, 69, 69, 66, 66, 65, 64, 63, 64, 66, 68, 69, 72, 73, 77, + 77, 81, 82, 86, 87, 90, 92, 95, 97, 99, 103, 104, 106, 106, 109, 108, + 110, 74, 70, 70, 67, 67, 66, 65, 63, 64, 67, 68, 70, 72, 74, 76, 78, 80, + 82, 85, 87, 90, 91, 95, 96, 100, 101, 105, 106, 108, 108, 111, 110, 75, + 71, 71, 68, 68, 66, 66, 64, 64, 68, 68, 71, 71, 75, 75, 79, 79, 83, 84, + 88, 89, 93, 93, 98, 98, 102, 103, 108, 108, 110, 110, 113], + /* Size 4x8 */ + [31, 47, 57, 65, 40, 45, 52, 61, 46, 55, 61, 63, 47, 60, 70, 72, 52, 64, + 79, 81, 59, 68, 87, 90, 63, 66, 88, 99, 66, 69, 85, 102], + /* Size 8x4 */ + [31, 40, 46, 47, 52, 59, 63, 66, 47, 45, 55, 60, 64, 68, 66, 69, 57, 52, + 61, 70, 79, 87, 88, 85, 65, 61, 63, 72, 81, 90, 99, 102], + /* Size 8x16 */ + [32, 35, 48, 50, 57, 63, 68, 70, 30, 38, 46, 46, 52, 58, 63, 65, 33, 41, + 47, 46, 51, 56, 60, 63, 39, 46, 48, 47, 51, 55, 58, 61, 49, 48, 53, 54, + 57, 60, 61, 61, 48, 46, 53, 56, 60, 64, 65, 65, 50, 46, 54, 61, 66, 70, + 71, 69, 52, 47, 54, 63, 71, 75, 75, 74, 55, 49, 56, 65, 74, 79, 79, 78, + 60, 53, 58, 68, 79, 85, 85, 82, 63, 55, 60, 70, 82, 89, 91, 87, 66, 58, + 62, 72, 84, 91, 95, 91, 68, 60, 64, 71, 81, 94, 97, 96, 70, 62, 65, 73, + 81, 89, 98, 101, 72, 65, 65, 72, 82, 92, 100, 103, 74, 67, 65, 71, 79, + 89, 98, 105], + /* Size 16x8 */ + [32, 30, 33, 39, 49, 48, 50, 52, 55, 60, 63, 66, 68, 70, 72, 74, 35, 38, + 41, 46, 48, 46, 46, 47, 49, 53, 55, 58, 60, 62, 65, 67, 48, 46, 47, 48, + 53, 53, 54, 54, 56, 58, 60, 62, 64, 65, 65, 65, 50, 46, 46, 47, 54, 56, + 61, 63, 65, 68, 70, 72, 71, 73, 72, 71, 57, 52, 51, 51, 57, 60, 66, 71, + 74, 79, 82, 84, 81, 81, 82, 79, 63, 58, 56, 55, 60, 64, 70, 75, 79, 85, + 89, 91, 94, 89, 92, 89, 68, 63, 60, 58, 61, 65, 71, 75, 79, 85, 91, 95, + 97, 98, 100, 98, 70, 65, 63, 61, 61, 65, 69, 74, 78, 82, 87, 91, 96, + 101, 103, 105], + /* Size 16x32 */ + [32, 31, 35, 38, 48, 49, 50, 52, 57, 61, 63, 67, 68, 69, 70, 71, 31, 31, + 37, 40, 47, 47, 48, 50, 54, 57, 60, 63, 64, 65, 66, 67, 30, 32, 38, 40, + 46, 45, 46, 48, 52, 55, 58, 61, 63, 64, 65, 67, 31, 33, 38, 41, 46, 45, + 46, 48, 52, 55, 57, 60, 61, 62, 63, 64, 33, 36, 41, 44, 47, 46, 46, 47, + 51, 54, 56, 59, 60, 61, 63, 64, 37, 40, 45, 47, 47, 45, 46, 47, 50, 52, + 54, 57, 59, 61, 62, 62, 39, 41, 46, 47, 48, 47, 47, 48, 51, 54, 55, 57, + 58, 59, 61, 62, 42, 43, 46, 48, 50, 49, 50, 50, 53, 56, 57, 60, 60, 59, + 60, 60, 49, 46, 48, 49, 53, 53, 54, 54, 57, 59, 60, 63, 61, 62, 61, 61, + 48, 46, 47, 48, 53, 55, 55, 56, 58, 61, 62, 64, 64, 63, 63, 64, 48, 46, + 46, 48, 53, 56, 56, 57, 60, 62, 64, 66, 65, 65, 65, 64, 49, 45, 45, 47, + 53, 58, 59, 61, 64, 66, 67, 69, 67, 67, 66, 67, 50, 46, 46, 48, 54, 59, + 61, 63, 66, 68, 70, 71, 71, 68, 69, 67, 51, 47, 47, 48, 54, 60, 61, 64, + 68, 70, 71, 73, 72, 72, 70, 71, 52, 48, 47, 48, 54, 61, 63, 66, 71, 73, + 75, 77, 75, 73, 74, 71, 54, 50, 49, 50, 55, 62, 65, 68, 73, 76, 78, 79, + 78, 76, 74, 75, 55, 51, 49, 50, 56, 63, 65, 69, 74, 77, 79, 81, 79, 78, + 78, 75, 57, 52, 50, 51, 56, 64, 66, 70, 76, 79, 82, 85, 83, 81, 79, 79, + 60, 54, 53, 53, 58, 65, 68, 72, 79, 82, 85, 87, 85, 84, 82, 80, 62, 56, + 54, 55, 60, 66, 69, 74, 81, 84, 87, 88, 87, 85, 84, 84, 63, 57, 55, 56, + 60, 67, 70, 75, 82, 86, 89, 92, 91, 89, 87, 84, 64, 59, 56, 57, 61, 68, + 71, 75, 83, 87, 90, 93, 92, 90, 89, 89, 66, 60, 58, 58, 62, 69, 72, 76, + 84, 88, 91, 94, 95, 93, 91, 89, 67, 61, 59, 58, 63, 68, 71, 78, 83, 86, + 93, 96, 96, 96, 94, 94, 68, 62, 60, 59, 64, 67, 71, 79, 81, 86, 94, 95, + 97, 98, 96, 94, 69, 63, 61, 60, 65, 66, 72, 77, 80, 88, 91, 96, 99, 99, + 100, 98, 70, 64, 62, 60, 65, 66, 73, 76, 81, 87, 89, 97, 98, 100, 101, + 99, 71, 65, 64, 61, 65, 67, 73, 74, 82, 85, 90, 95, 99, 102, 103, 104, + 72, 65, 65, 62, 65, 68, 72, 75, 82, 83, 92, 93, 100, 102, 103, 104, 73, + 66, 66, 63, 65, 69, 72, 76, 81, 85, 90, 93, 100, 102, 105, 106, 74, 67, + 67, 64, 65, 70, 71, 77, 79, 86, 89, 94, 98, 103, 105, 106, 75, 68, 68, + 65, 65, 71, 71, 78, 78, 87, 87, 96, 96, 105, 105, 109], + /* Size 32x16 */ + [32, 31, 30, 31, 33, 37, 39, 42, 49, 48, 48, 49, 50, 51, 52, 54, 55, 57, + 60, 62, 63, 64, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 31, 31, 32, 33, + 36, 40, 41, 43, 46, 46, 46, 45, 46, 47, 48, 50, 51, 52, 54, 56, 57, 59, + 60, 61, 62, 63, 64, 65, 65, 66, 67, 68, 35, 37, 38, 38, 41, 45, 46, 46, + 48, 47, 46, 45, 46, 47, 47, 49, 49, 50, 53, 54, 55, 56, 58, 59, 60, 61, + 62, 64, 65, 66, 67, 68, 38, 40, 40, 41, 44, 47, 47, 48, 49, 48, 48, 47, + 48, 48, 48, 50, 50, 51, 53, 55, 56, 57, 58, 58, 59, 60, 60, 61, 62, 63, + 64, 65, 48, 47, 46, 46, 47, 47, 48, 50, 53, 53, 53, 53, 54, 54, 54, 55, + 56, 56, 58, 60, 60, 61, 62, 63, 64, 65, 65, 65, 65, 65, 65, 65, 49, 47, + 45, 45, 46, 45, 47, 49, 53, 55, 56, 58, 59, 60, 61, 62, 63, 64, 65, 66, + 67, 68, 69, 68, 67, 66, 66, 67, 68, 69, 70, 71, 50, 48, 46, 46, 46, 46, + 47, 50, 54, 55, 56, 59, 61, 61, 63, 65, 65, 66, 68, 69, 70, 71, 72, 71, + 71, 72, 73, 73, 72, 72, 71, 71, 52, 50, 48, 48, 47, 47, 48, 50, 54, 56, + 57, 61, 63, 64, 66, 68, 69, 70, 72, 74, 75, 75, 76, 78, 79, 77, 76, 74, + 75, 76, 77, 78, 57, 54, 52, 52, 51, 50, 51, 53, 57, 58, 60, 64, 66, 68, + 71, 73, 74, 76, 79, 81, 82, 83, 84, 83, 81, 80, 81, 82, 82, 81, 79, 78, + 61, 57, 55, 55, 54, 52, 54, 56, 59, 61, 62, 66, 68, 70, 73, 76, 77, 79, + 82, 84, 86, 87, 88, 86, 86, 88, 87, 85, 83, 85, 86, 87, 63, 60, 58, 57, + 56, 54, 55, 57, 60, 62, 64, 67, 70, 71, 75, 78, 79, 82, 85, 87, 89, 90, + 91, 93, 94, 91, 89, 90, 92, 90, 89, 87, 67, 63, 61, 60, 59, 57, 57, 60, + 63, 64, 66, 69, 71, 73, 77, 79, 81, 85, 87, 88, 92, 93, 94, 96, 95, 96, + 97, 95, 93, 93, 94, 96, 68, 64, 63, 61, 60, 59, 58, 60, 61, 64, 65, 67, + 71, 72, 75, 78, 79, 83, 85, 87, 91, 92, 95, 96, 97, 99, 98, 99, 100, + 100, 98, 96, 69, 65, 64, 62, 61, 61, 59, 59, 62, 63, 65, 67, 68, 72, 73, + 76, 78, 81, 84, 85, 89, 90, 93, 96, 98, 99, 100, 102, 102, 102, 103, + 105, 70, 66, 65, 63, 63, 62, 61, 60, 61, 63, 65, 66, 69, 70, 74, 74, 78, + 79, 82, 84, 87, 89, 91, 94, 96, 100, 101, 103, 103, 105, 105, 105, 71, + 67, 67, 64, 64, 62, 62, 60, 61, 64, 64, 67, 67, 71, 71, 75, 75, 79, 80, + 84, 84, 89, 89, 94, 94, 98, 99, 104, 104, 106, 106, 109], + /* Size 4x16 */ + [31, 49, 61, 69, 32, 45, 55, 64, 36, 46, 54, 61, 41, 47, 54, 59, 46, 53, + 59, 62, 46, 56, 62, 65, 46, 59, 68, 68, 48, 61, 73, 73, 51, 63, 77, 78, + 54, 65, 82, 84, 57, 67, 86, 89, 60, 69, 88, 93, 62, 67, 86, 98, 64, 66, + 87, 100, 65, 68, 83, 102, 67, 70, 86, 103], + /* Size 16x4 */ + [31, 32, 36, 41, 46, 46, 46, 48, 51, 54, 57, 60, 62, 64, 65, 67, 49, 45, + 46, 47, 53, 56, 59, 61, 63, 65, 67, 69, 67, 66, 68, 70, 61, 55, 54, 54, + 59, 62, 68, 73, 77, 82, 86, 88, 86, 87, 83, 86, 69, 64, 61, 59, 62, 65, + 68, 73, 78, 84, 89, 93, 98, 100, 102, 103], + /* Size 8x32 */ + [32, 35, 48, 50, 57, 63, 68, 70, 31, 37, 47, 48, 54, 60, 64, 66, 30, 38, + 46, 46, 52, 58, 63, 65, 31, 38, 46, 46, 52, 57, 61, 63, 33, 41, 47, 46, + 51, 56, 60, 63, 37, 45, 47, 46, 50, 54, 59, 62, 39, 46, 48, 47, 51, 55, + 58, 61, 42, 46, 50, 50, 53, 57, 60, 60, 49, 48, 53, 54, 57, 60, 61, 61, + 48, 47, 53, 55, 58, 62, 64, 63, 48, 46, 53, 56, 60, 64, 65, 65, 49, 45, + 53, 59, 64, 67, 67, 66, 50, 46, 54, 61, 66, 70, 71, 69, 51, 47, 54, 61, + 68, 71, 72, 70, 52, 47, 54, 63, 71, 75, 75, 74, 54, 49, 55, 65, 73, 78, + 78, 74, 55, 49, 56, 65, 74, 79, 79, 78, 57, 50, 56, 66, 76, 82, 83, 79, + 60, 53, 58, 68, 79, 85, 85, 82, 62, 54, 60, 69, 81, 87, 87, 84, 63, 55, + 60, 70, 82, 89, 91, 87, 64, 56, 61, 71, 83, 90, 92, 89, 66, 58, 62, 72, + 84, 91, 95, 91, 67, 59, 63, 71, 83, 93, 96, 94, 68, 60, 64, 71, 81, 94, + 97, 96, 69, 61, 65, 72, 80, 91, 99, 100, 70, 62, 65, 73, 81, 89, 98, + 101, 71, 64, 65, 73, 82, 90, 99, 103, 72, 65, 65, 72, 82, 92, 100, 103, + 73, 66, 65, 72, 81, 90, 100, 105, 74, 67, 65, 71, 79, 89, 98, 105, 75, + 68, 65, 71, 78, 87, 96, 105], + /* Size 32x8 */ + [32, 31, 30, 31, 33, 37, 39, 42, 49, 48, 48, 49, 50, 51, 52, 54, 55, 57, + 60, 62, 63, 64, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 35, 37, 38, 38, + 41, 45, 46, 46, 48, 47, 46, 45, 46, 47, 47, 49, 49, 50, 53, 54, 55, 56, + 58, 59, 60, 61, 62, 64, 65, 66, 67, 68, 48, 47, 46, 46, 47, 47, 48, 50, + 53, 53, 53, 53, 54, 54, 54, 55, 56, 56, 58, 60, 60, 61, 62, 63, 64, 65, + 65, 65, 65, 65, 65, 65, 50, 48, 46, 46, 46, 46, 47, 50, 54, 55, 56, 59, + 61, 61, 63, 65, 65, 66, 68, 69, 70, 71, 72, 71, 71, 72, 73, 73, 72, 72, + 71, 71, 57, 54, 52, 52, 51, 50, 51, 53, 57, 58, 60, 64, 66, 68, 71, 73, + 74, 76, 79, 81, 82, 83, 84, 83, 81, 80, 81, 82, 82, 81, 79, 78, 63, 60, + 58, 57, 56, 54, 55, 57, 60, 62, 64, 67, 70, 71, 75, 78, 79, 82, 85, 87, + 89, 90, 91, 93, 94, 91, 89, 90, 92, 90, 89, 87, 68, 64, 63, 61, 60, 59, + 58, 60, 61, 64, 65, 67, 71, 72, 75, 78, 79, 83, 85, 87, 91, 92, 95, 96, + 97, 99, 98, 99, 100, 100, 98, 96, 70, 66, 65, 63, 63, 62, 61, 60, 61, + 63, 65, 66, 69, 70, 74, 74, 78, 79, 82, 84, 87, 89, 91, 94, 96, 100, + 101, 103, 103, 105, 105, 105] + ] + ], + [ + [ /* Luma */ + /* Size 4x4 */ + [32, 38, 63, 86, 38, 56, 78, 97, 63, 78, 113, 130, 86, 97, 130, 169], + /* Size 8x8 */ + [32, 32, 35, 46, 57, 76, 85, 96, 32, 34, 37, 45, 54, 70, 79, 90, 35, 37, + 48, 56, 64, 79, 87, 93, 46, 45, 56, 70, 80, 96, 100, 105, 57, 54, 64, + 80, 93, 111, 121, 122, 76, 70, 79, 96, 111, 134, 138, 144, 85, 79, 87, + 100, 121, 138, 156, 168, 96, 90, 93, 105, 122, 144, 168, 184], + /* Size 16x16 */ + [32, 31, 31, 32, 34, 39, 44, 49, 58, 65, 71, 81, 87, 93, 98, 104, 31, 32, + 32, 32, 34, 38, 41, 46, 54, 60, 66, 75, 81, 86, 92, 98, 31, 32, 33, 34, + 36, 39, 42, 46, 53, 59, 64, 73, 78, 83, 88, 94, 32, 32, 34, 35, 37, 40, + 42, 46, 52, 58, 63, 71, 75, 80, 86, 92, 34, 34, 36, 37, 42, 47, 50, 53, + 59, 65, 70, 77, 82, 85, 89, 92, 39, 38, 39, 40, 47, 54, 58, 62, 68, 73, + 78, 85, 90, 90, 96, 98, 44, 41, 42, 42, 50, 58, 63, 68, 74, 79, 84, 91, + 96, 98, 102, 104, 49, 46, 46, 46, 53, 62, 68, 73, 81, 87, 92, 99, 103, + 107, 109, 112, 58, 54, 53, 52, 59, 68, 74, 81, 90, 97, 102, 110, 114, + 118, 117, 121, 65, 60, 59, 58, 65, 73, 79, 87, 97, 105, 111, 120, 125, + 125, 126, 130, 71, 66, 64, 63, 70, 78, 84, 92, 102, 111, 117, 127, 133, + 134, 136, 141, 81, 75, 73, 71, 77, 85, 91, 99, 110, 120, 127, 137, 143, + 145, 148, 152, 87, 81, 78, 75, 82, 90, 96, 103, 114, 125, 133, 143, 150, + 156, 160, 163, 93, 86, 83, 80, 85, 90, 98, 107, 118, 125, 134, 145, 156, + 163, 169, 177, 98, 92, 88, 86, 89, 96, 102, 109, 117, 126, 136, 148, + 160, 169, 176, 184, 104, 98, 94, 92, 92, 98, 104, 112, 121, 130, 141, + 152, 163, 177, 184, 191], + /* Size 32x32 */ + [32, 31, 31, 31, 31, 32, 32, 34, 34, 36, 39, 41, 44, 48, 49, 54, 58, 59, + 65, 69, 71, 80, 81, 83, 87, 90, 93, 95, 98, 101, 104, 107, 31, 32, 32, + 32, 32, 32, 32, 34, 34, 35, 38, 39, 42, 46, 47, 51, 55, 57, 62, 66, 68, + 76, 77, 78, 83, 85, 88, 90, 93, 96, 99, 101, 31, 32, 32, 32, 32, 32, 32, + 33, 34, 34, 38, 39, 41, 45, 46, 50, 54, 55, 60, 64, 66, 73, 75, 76, 81, + 83, 86, 89, 92, 95, 98, 101, 31, 32, 32, 32, 32, 32, 32, 33, 34, 34, 37, + 38, 41, 44, 45, 49, 53, 54, 59, 63, 65, 72, 74, 75, 79, 81, 84, 86, 89, + 91, 94, 97, 31, 32, 32, 32, 33, 33, 34, 35, 36, 36, 39, 40, 42, 45, 46, + 50, 53, 54, 59, 63, 64, 71, 73, 74, 78, 80, 83, 85, 88, 91, 94, 97, 32, + 32, 32, 32, 33, 34, 34, 36, 36, 37, 40, 40, 42, 45, 46, 49, 53, 54, 58, + 62, 63, 70, 72, 73, 77, 79, 82, 85, 87, 90, 92, 95, 32, 32, 32, 32, 34, + 34, 35, 37, 37, 38, 40, 41, 42, 45, 46, 49, 52, 54, 58, 61, 63, 69, 71, + 72, 75, 78, 80, 83, 86, 89, 92, 95, 34, 34, 33, 33, 35, 36, 37, 39, 41, + 42, 45, 46, 47, 50, 51, 54, 57, 59, 63, 66, 68, 74, 75, 76, 80, 81, 82, + 83, 85, 87, 90, 93, 34, 34, 34, 34, 36, 36, 37, 41, 42, 45, 47, 48, 50, + 53, 53, 56, 59, 61, 65, 68, 70, 76, 77, 78, 82, 83, 85, 88, 89, 90, 92, + 93, 36, 35, 34, 34, 36, 37, 38, 42, 45, 48, 50, 51, 54, 56, 57, 60, 63, + 64, 68, 71, 73, 79, 80, 81, 85, 87, 89, 89, 90, 93, 96, 99, 39, 38, 38, + 37, 39, 40, 40, 45, 47, 50, 54, 55, 58, 61, 62, 65, 68, 69, 73, 76, 78, + 84, 85, 86, 90, 89, 90, 93, 96, 97, 98, 99, 41, 39, 39, 38, 40, 40, 41, + 46, 48, 51, 55, 56, 59, 62, 63, 67, 70, 71, 75, 78, 80, 86, 87, 88, 91, + 93, 96, 97, 97, 99, 102, 105, 44, 42, 41, 41, 42, 42, 42, 47, 50, 54, + 58, 59, 63, 66, 68, 71, 74, 75, 79, 83, 84, 90, 91, 92, 96, 98, 98, 99, + 102, 104, 104, 105, 48, 46, 45, 44, 45, 45, 45, 50, 53, 56, 61, 62, 66, + 70, 71, 76, 79, 80, 85, 88, 90, 96, 97, 98, 101, 100, 102, 105, 105, + 105, 109, 112, 49, 47, 46, 45, 46, 46, 46, 51, 53, 57, 62, 63, 68, 71, + 73, 77, 81, 82, 87, 90, 92, 98, 99, 100, 103, 106, 107, 106, 109, 112, + 112, 112, 54, 51, 50, 49, 50, 49, 49, 54, 56, 60, 65, 67, 71, 76, 77, + 82, 86, 87, 92, 96, 97, 104, 105, 106, 110, 110, 109, 113, 114, 113, + 116, 120, 58, 55, 54, 53, 53, 53, 52, 57, 59, 63, 68, 70, 74, 79, 81, + 86, 90, 91, 97, 100, 102, 109, 110, 111, 114, 114, 118, 116, 117, 121, + 121, 120, 59, 57, 55, 54, 54, 54, 54, 59, 61, 64, 69, 71, 75, 80, 82, + 87, 91, 93, 99, 102, 104, 111, 112, 113, 117, 121, 120, 122, 124, 122, + 125, 129, 65, 62, 60, 59, 59, 58, 58, 63, 65, 68, 73, 75, 79, 85, 87, + 92, 97, 99, 105, 109, 111, 118, 120, 121, 125, 124, 125, 127, 126, 130, + 130, 129, 69, 66, 64, 63, 63, 62, 61, 66, 68, 71, 76, 78, 83, 88, 90, + 96, 100, 102, 109, 113, 115, 123, 125, 126, 129, 130, 131, 130, 134, + 133, 135, 139, 71, 68, 66, 65, 64, 63, 63, 68, 70, 73, 78, 80, 84, 90, + 92, 97, 102, 104, 111, 115, 117, 125, 127, 128, 133, 136, 134, 139, 136, + 139, 141, 140, 80, 76, 73, 72, 71, 70, 69, 74, 76, 79, 84, 86, 90, 96, + 98, 104, 109, 111, 118, 123, 125, 134, 136, 137, 142, 138, 143, 140, + 144, 144, 144, 149, 81, 77, 75, 74, 73, 72, 71, 75, 77, 80, 85, 87, 91, + 97, 99, 105, 110, 112, 120, 125, 127, 136, 137, 139, 143, 148, 145, 148, + 148, 150, 152, 149, 83, 78, 76, 75, 74, 73, 72, 76, 78, 81, 86, 88, 92, + 98, 100, 106, 111, 113, 121, 126, 128, 137, 139, 140, 145, 149, 153, + 153, 154, 155, 155, 161, 87, 83, 81, 79, 78, 77, 75, 80, 82, 85, 90, 91, + 96, 101, 103, 110, 114, 117, 125, 129, 133, 142, 143, 145, 150, 151, + 156, 159, 160, 160, 163, 161, 90, 85, 83, 81, 80, 79, 78, 81, 83, 87, + 89, 93, 98, 100, 106, 110, 114, 121, 124, 130, 136, 138, 148, 149, 151, + 156, 157, 162, 166, 168, 166, 172, 93, 88, 86, 84, 83, 82, 80, 82, 85, + 89, 90, 96, 98, 102, 107, 109, 118, 120, 125, 131, 134, 143, 145, 153, + 156, 157, 163, 164, 169, 172, 177, 172, 95, 90, 89, 86, 85, 85, 83, 83, + 88, 89, 93, 97, 99, 105, 106, 113, 116, 122, 127, 130, 139, 140, 148, + 153, 159, 162, 164, 169, 170, 176, 179, 185, 98, 93, 92, 89, 88, 87, 86, + 85, 89, 90, 96, 97, 102, 105, 109, 114, 117, 124, 126, 134, 136, 144, + 148, 154, 160, 166, 169, 170, 176, 177, 184, 186, 101, 96, 95, 91, 91, + 90, 89, 87, 90, 93, 97, 99, 104, 105, 112, 113, 121, 122, 130, 133, 139, + 144, 150, 155, 160, 168, 172, 176, 177, 184, 185, 191, 104, 99, 98, 94, + 94, 92, 92, 90, 92, 96, 98, 102, 104, 109, 112, 116, 121, 125, 130, 135, + 141, 144, 152, 155, 163, 166, 177, 179, 184, 185, 191, 192, 107, 101, + 101, 97, 97, 95, 95, 93, 93, 99, 99, 105, 105, 112, 112, 120, 120, 129, + 129, 139, 140, 149, 149, 161, 161, 172, 172, 185, 186, 191, 192, 199], + /* Size 4x8 */ + [32, 38, 62, 86, 32, 40, 58, 80, 34, 51, 68, 85, 44, 61, 85, 101, 54, 69, + 98, 117, 72, 84, 118, 136, 82, 89, 129, 157, 92, 98, 127, 165], + /* Size 8x4 */ + [32, 32, 34, 44, 54, 72, 82, 92, 38, 40, 51, 61, 69, 84, 89, 98, 62, 58, + 68, 85, 98, 118, 129, 127, 86, 80, 85, 101, 117, 136, 157, 165], + /* Size 8x16 */ + [32, 32, 36, 44, 58, 79, 88, 93, 31, 32, 35, 41, 54, 73, 81, 88, 32, 33, + 36, 42, 53, 71, 78, 84, 32, 34, 38, 42, 52, 69, 76, 82, 34, 36, 44, 50, + 59, 75, 81, 84, 39, 39, 50, 58, 68, 84, 88, 90, 44, 42, 53, 63, 74, 90, + 97, 97, 49, 46, 57, 67, 81, 97, 104, 105, 57, 53, 63, 74, 90, 108, 111, + 113, 65, 59, 68, 79, 97, 118, 123, 122, 71, 64, 73, 84, 102, 125, 135, + 131, 81, 72, 80, 91, 110, 135, 145, 141, 87, 77, 85, 96, 114, 140, 148, + 151, 92, 83, 88, 102, 117, 133, 153, 163, 98, 88, 89, 103, 121, 141, + 160, 169, 103, 94, 92, 103, 119, 137, 158, 175], + /* Size 16x8 */ + [32, 31, 32, 32, 34, 39, 44, 49, 57, 65, 71, 81, 87, 92, 98, 103, 32, 32, + 33, 34, 36, 39, 42, 46, 53, 59, 64, 72, 77, 83, 88, 94, 36, 35, 36, 38, + 44, 50, 53, 57, 63, 68, 73, 80, 85, 88, 89, 92, 44, 41, 42, 42, 50, 58, + 63, 67, 74, 79, 84, 91, 96, 102, 103, 103, 58, 54, 53, 52, 59, 68, 74, + 81, 90, 97, 102, 110, 114, 117, 121, 119, 79, 73, 71, 69, 75, 84, 90, + 97, 108, 118, 125, 135, 140, 133, 141, 137, 88, 81, 78, 76, 81, 88, 97, + 104, 111, 123, 135, 145, 148, 153, 160, 158, 93, 88, 84, 82, 84, 90, 97, + 105, 113, 122, 131, 141, 151, 163, 169, 175], + /* Size 16x32 */ + [32, 31, 32, 32, 36, 39, 44, 53, 58, 65, 79, 81, 88, 90, 93, 96, 31, 32, + 32, 32, 35, 38, 42, 51, 55, 62, 75, 77, 83, 86, 88, 91, 31, 32, 32, 32, + 35, 38, 41, 50, 54, 60, 73, 75, 81, 84, 88, 91, 31, 32, 32, 33, 34, 37, + 41, 49, 53, 59, 72, 74, 79, 82, 84, 87, 32, 32, 33, 34, 36, 39, 42, 50, + 53, 59, 71, 72, 78, 81, 84, 87, 32, 32, 34, 34, 37, 40, 42, 49, 53, 58, + 70, 71, 77, 80, 83, 85, 32, 33, 34, 35, 38, 40, 42, 49, 52, 58, 69, 70, + 76, 78, 82, 86, 34, 34, 35, 37, 42, 45, 48, 54, 57, 63, 73, 75, 79, 79, + 81, 83, 34, 34, 36, 37, 44, 47, 50, 56, 59, 65, 75, 77, 81, 83, 84, 84, + 36, 34, 37, 38, 48, 51, 54, 60, 63, 68, 78, 80, 85, 85, 86, 89, 39, 37, + 39, 40, 50, 54, 58, 65, 68, 73, 84, 85, 88, 89, 90, 89, 40, 38, 40, 41, + 51, 55, 59, 67, 70, 75, 85, 87, 91, 92, 92, 95, 44, 41, 42, 43, 53, 58, + 63, 71, 74, 79, 90, 91, 97, 94, 97, 95, 47, 44, 45, 46, 56, 61, 66, 75, + 79, 85, 95, 97, 99, 101, 98, 102, 49, 46, 46, 47, 57, 62, 67, 77, 81, + 86, 97, 99, 104, 102, 105, 102, 53, 49, 50, 50, 60, 65, 71, 82, 86, 92, + 103, 105, 109, 108, 106, 110, 57, 53, 53, 53, 63, 68, 74, 86, 90, 97, + 108, 110, 111, 112, 113, 110, 59, 54, 54, 54, 64, 69, 75, 87, 91, 98, + 111, 112, 119, 117, 115, 118, 65, 60, 59, 58, 68, 73, 79, 92, 97, 105, + 118, 119, 123, 123, 122, 119, 69, 63, 62, 62, 71, 76, 83, 96, 100, 109, + 122, 124, 127, 125, 125, 128, 71, 65, 64, 63, 73, 78, 84, 97, 102, 111, + 125, 127, 135, 134, 131, 129, 79, 72, 71, 70, 79, 84, 90, 104, 109, 118, + 133, 135, 137, 136, 136, 137, 81, 74, 72, 71, 80, 85, 91, 105, 110, 120, + 135, 137, 145, 143, 141, 138, 82, 75, 73, 72, 81, 86, 92, 106, 111, 121, + 136, 139, 147, 148, 147, 149, 87, 79, 77, 76, 85, 90, 96, 110, 114, 125, + 140, 143, 148, 154, 151, 149, 90, 82, 80, 78, 87, 89, 99, 108, 113, 129, + 135, 146, 153, 157, 160, 159, 92, 84, 83, 81, 88, 90, 102, 106, 117, + 128, 133, 150, 153, 158, 163, 160, 95, 87, 85, 83, 88, 92, 103, 105, + 120, 125, 137, 148, 155, 164, 168, 173, 98, 89, 88, 85, 89, 95, 103, + 108, 121, 124, 141, 144, 160, 164, 169, 174, 100, 92, 91, 88, 90, 98, + 103, 111, 120, 127, 139, 146, 161, 165, 175, 179, 103, 94, 94, 90, 92, + 101, 103, 114, 119, 131, 137, 150, 158, 170, 175, 180, 106, 97, 97, 93, + 93, 104, 104, 118, 118, 135, 135, 154, 155, 175, 176, 187], + /* Size 32x16 */ + [32, 31, 31, 31, 32, 32, 32, 34, 34, 36, 39, 40, 44, 47, 49, 53, 57, 59, + 65, 69, 71, 79, 81, 82, 87, 90, 92, 95, 98, 100, 103, 106, 31, 32, 32, + 32, 32, 32, 33, 34, 34, 34, 37, 38, 41, 44, 46, 49, 53, 54, 60, 63, 65, + 72, 74, 75, 79, 82, 84, 87, 89, 92, 94, 97, 32, 32, 32, 32, 33, 34, 34, + 35, 36, 37, 39, 40, 42, 45, 46, 50, 53, 54, 59, 62, 64, 71, 72, 73, 77, + 80, 83, 85, 88, 91, 94, 97, 32, 32, 32, 33, 34, 34, 35, 37, 37, 38, 40, + 41, 43, 46, 47, 50, 53, 54, 58, 62, 63, 70, 71, 72, 76, 78, 81, 83, 85, + 88, 90, 93, 36, 35, 35, 34, 36, 37, 38, 42, 44, 48, 50, 51, 53, 56, 57, + 60, 63, 64, 68, 71, 73, 79, 80, 81, 85, 87, 88, 88, 89, 90, 92, 93, 39, + 38, 38, 37, 39, 40, 40, 45, 47, 51, 54, 55, 58, 61, 62, 65, 68, 69, 73, + 76, 78, 84, 85, 86, 90, 89, 90, 92, 95, 98, 101, 104, 44, 42, 41, 41, + 42, 42, 42, 48, 50, 54, 58, 59, 63, 66, 67, 71, 74, 75, 79, 83, 84, 90, + 91, 92, 96, 99, 102, 103, 103, 103, 103, 104, 53, 51, 50, 49, 50, 49, + 49, 54, 56, 60, 65, 67, 71, 75, 77, 82, 86, 87, 92, 96, 97, 104, 105, + 106, 110, 108, 106, 105, 108, 111, 114, 118, 58, 55, 54, 53, 53, 53, 52, + 57, 59, 63, 68, 70, 74, 79, 81, 86, 90, 91, 97, 100, 102, 109, 110, 111, + 114, 113, 117, 120, 121, 120, 119, 118, 65, 62, 60, 59, 59, 58, 58, 63, + 65, 68, 73, 75, 79, 85, 86, 92, 97, 98, 105, 109, 111, 118, 120, 121, + 125, 129, 128, 125, 124, 127, 131, 135, 79, 75, 73, 72, 71, 70, 69, 73, + 75, 78, 84, 85, 90, 95, 97, 103, 108, 111, 118, 122, 125, 133, 135, 136, + 140, 135, 133, 137, 141, 139, 137, 135, 81, 77, 75, 74, 72, 71, 70, 75, + 77, 80, 85, 87, 91, 97, 99, 105, 110, 112, 119, 124, 127, 135, 137, 139, + 143, 146, 150, 148, 144, 146, 150, 154, 88, 83, 81, 79, 78, 77, 76, 79, + 81, 85, 88, 91, 97, 99, 104, 109, 111, 119, 123, 127, 135, 137, 145, + 147, 148, 153, 153, 155, 160, 161, 158, 155, 90, 86, 84, 82, 81, 80, 78, + 79, 83, 85, 89, 92, 94, 101, 102, 108, 112, 117, 123, 125, 134, 136, + 143, 148, 154, 157, 158, 164, 164, 165, 170, 175, 93, 88, 88, 84, 84, + 83, 82, 81, 84, 86, 90, 92, 97, 98, 105, 106, 113, 115, 122, 125, 131, + 136, 141, 147, 151, 160, 163, 168, 169, 175, 175, 176, 96, 91, 91, 87, + 87, 85, 86, 83, 84, 89, 89, 95, 95, 102, 102, 110, 110, 118, 119, 128, + 129, 137, 138, 149, 149, 159, 160, 173, 174, 179, 180, 187], + /* Size 4x16 */ + [31, 39, 65, 90, 32, 38, 60, 84, 32, 39, 59, 81, 33, 40, 58, 78, 34, 47, + 65, 83, 37, 54, 73, 89, 41, 58, 79, 94, 46, 62, 86, 102, 53, 68, 97, + 112, 60, 73, 105, 123, 65, 78, 111, 134, 74, 85, 120, 143, 79, 90, 125, + 154, 84, 90, 128, 158, 89, 95, 124, 164, 94, 101, 131, 170], + /* Size 16x4 */ + [31, 32, 32, 33, 34, 37, 41, 46, 53, 60, 65, 74, 79, 84, 89, 94, 39, 38, + 39, 40, 47, 54, 58, 62, 68, 73, 78, 85, 90, 90, 95, 101, 65, 60, 59, 58, + 65, 73, 79, 86, 97, 105, 111, 120, 125, 128, 124, 131, 90, 84, 81, 78, + 83, 89, 94, 102, 112, 123, 134, 143, 154, 158, 164, 170], + /* Size 8x32 */ + [32, 32, 36, 44, 58, 79, 88, 93, 31, 32, 35, 42, 55, 75, 83, 88, 31, 32, + 35, 41, 54, 73, 81, 88, 31, 32, 34, 41, 53, 72, 79, 84, 32, 33, 36, 42, + 53, 71, 78, 84, 32, 34, 37, 42, 53, 70, 77, 83, 32, 34, 38, 42, 52, 69, + 76, 82, 34, 35, 42, 48, 57, 73, 79, 81, 34, 36, 44, 50, 59, 75, 81, 84, + 36, 37, 48, 54, 63, 78, 85, 86, 39, 39, 50, 58, 68, 84, 88, 90, 40, 40, + 51, 59, 70, 85, 91, 92, 44, 42, 53, 63, 74, 90, 97, 97, 47, 45, 56, 66, + 79, 95, 99, 98, 49, 46, 57, 67, 81, 97, 104, 105, 53, 50, 60, 71, 86, + 103, 109, 106, 57, 53, 63, 74, 90, 108, 111, 113, 59, 54, 64, 75, 91, + 111, 119, 115, 65, 59, 68, 79, 97, 118, 123, 122, 69, 62, 71, 83, 100, + 122, 127, 125, 71, 64, 73, 84, 102, 125, 135, 131, 79, 71, 79, 90, 109, + 133, 137, 136, 81, 72, 80, 91, 110, 135, 145, 141, 82, 73, 81, 92, 111, + 136, 147, 147, 87, 77, 85, 96, 114, 140, 148, 151, 90, 80, 87, 99, 113, + 135, 153, 160, 92, 83, 88, 102, 117, 133, 153, 163, 95, 85, 88, 103, + 120, 137, 155, 168, 98, 88, 89, 103, 121, 141, 160, 169, 100, 91, 90, + 103, 120, 139, 161, 175, 103, 94, 92, 103, 119, 137, 158, 175, 106, 97, + 93, 104, 118, 135, 155, 176], + /* Size 32x8 */ + [32, 31, 31, 31, 32, 32, 32, 34, 34, 36, 39, 40, 44, 47, 49, 53, 57, 59, + 65, 69, 71, 79, 81, 82, 87, 90, 92, 95, 98, 100, 103, 106, 32, 32, 32, + 32, 33, 34, 34, 35, 36, 37, 39, 40, 42, 45, 46, 50, 53, 54, 59, 62, 64, + 71, 72, 73, 77, 80, 83, 85, 88, 91, 94, 97, 36, 35, 35, 34, 36, 37, 38, + 42, 44, 48, 50, 51, 53, 56, 57, 60, 63, 64, 68, 71, 73, 79, 80, 81, 85, + 87, 88, 88, 89, 90, 92, 93, 44, 42, 41, 41, 42, 42, 42, 48, 50, 54, 58, + 59, 63, 66, 67, 71, 74, 75, 79, 83, 84, 90, 91, 92, 96, 99, 102, 103, + 103, 103, 103, 104, 58, 55, 54, 53, 53, 53, 52, 57, 59, 63, 68, 70, 74, + 79, 81, 86, 90, 91, 97, 100, 102, 109, 110, 111, 114, 113, 117, 120, + 121, 120, 119, 118, 79, 75, 73, 72, 71, 70, 69, 73, 75, 78, 84, 85, 90, + 95, 97, 103, 108, 111, 118, 122, 125, 133, 135, 136, 140, 135, 133, 137, + 141, 139, 137, 135, 88, 83, 81, 79, 78, 77, 76, 79, 81, 85, 88, 91, 97, + 99, 104, 109, 111, 119, 123, 127, 135, 137, 145, 147, 148, 153, 153, + 155, 160, 161, 158, 155, 93, 88, 88, 84, 84, 83, 82, 81, 84, 86, 90, 92, + 97, 98, 105, 106, 113, 115, 122, 125, 131, 136, 141, 147, 151, 160, 163, + 168, 169, 175, 175, 176] + ], + [ /* Chroma */ + /* Size 4x4 */ + [32, 45, 53, 63, 45, 55, 62, 67, 53, 62, 80, 84, 63, 67, 84, 101], + /* Size 8x8 */ + [31, 36, 47, 48, 52, 60, 64, 67, 36, 43, 47, 46, 49, 55, 59, 63, 47, 47, + 53, 54, 55, 60, 63, 64, 48, 46, 54, 61, 65, 70, 71, 71, 52, 49, 55, 65, + 71, 78, 81, 79, 60, 55, 60, 70, 78, 89, 89, 89, 64, 59, 63, 71, 81, 89, + 97, 99, 67, 63, 64, 71, 79, 89, 99, 104], + /* Size 16x16 */ + [32, 30, 33, 36, 44, 48, 49, 51, 54, 57, 60, 64, 67, 68, 70, 72, 30, 31, + 35, 39, 44, 46, 46, 47, 50, 53, 55, 59, 61, 64, 66, 68, 33, 35, 39, 43, + 46, 46, 45, 47, 49, 51, 53, 57, 59, 61, 63, 65, 36, 39, 43, 47, 47, 46, + 45, 46, 48, 50, 52, 55, 57, 58, 61, 63, 44, 44, 46, 47, 50, 51, 51, 51, + 53, 54, 56, 59, 61, 61, 63, 62, 48, 46, 46, 46, 51, 54, 55, 56, 58, 60, + 61, 64, 65, 64, 66, 66, 49, 46, 45, 45, 51, 55, 58, 60, 62, 63, 65, 68, + 69, 69, 69, 69, 51, 47, 47, 46, 51, 56, 60, 62, 65, 67, 69, 72, 73, 74, + 73, 73, 54, 50, 49, 48, 53, 58, 62, 65, 70, 73, 75, 78, 79, 79, 77, 77, + 57, 53, 51, 50, 54, 60, 63, 67, 73, 76, 79, 82, 84, 83, 82, 82, 60, 55, + 53, 52, 56, 61, 65, 69, 75, 79, 82, 86, 88, 87, 86, 87, 64, 59, 57, 55, + 59, 64, 68, 72, 78, 82, 86, 90, 93, 92, 91, 92, 67, 61, 59, 57, 61, 65, + 69, 73, 79, 84, 88, 93, 95, 96, 96, 96, 68, 64, 61, 58, 61, 64, 69, 74, + 79, 83, 87, 92, 96, 99, 100, 101, 70, 66, 63, 61, 63, 66, 69, 73, 77, + 82, 86, 91, 96, 100, 103, 104, 72, 68, 65, 63, 62, 66, 69, 73, 77, 82, + 87, 92, 96, 101, 104, 106], + /* Size 32x32 */ + [32, 31, 30, 30, 33, 35, 36, 41, 44, 49, 48, 48, 49, 50, 51, 52, 54, 55, + 57, 59, 60, 63, 64, 65, 67, 68, 68, 69, 70, 71, 72, 73, 31, 31, 31, 31, + 34, 36, 38, 42, 44, 47, 47, 47, 47, 48, 48, 50, 51, 52, 54, 56, 57, 60, + 61, 61, 63, 64, 65, 66, 67, 67, 68, 69, 30, 31, 31, 31, 35, 37, 39, 42, + 44, 47, 46, 46, 46, 47, 47, 48, 50, 51, 53, 54, 55, 58, 59, 60, 61, 63, + 64, 65, 66, 67, 68, 69, 30, 31, 31, 32, 35, 37, 40, 42, 44, 46, 45, 45, + 45, 46, 46, 47, 49, 50, 52, 53, 54, 57, 58, 58, 60, 61, 62, 63, 63, 64, + 65, 66, 33, 34, 35, 35, 39, 41, 43, 45, 46, 47, 46, 46, 45, 46, 47, 47, + 49, 49, 51, 53, 53, 56, 57, 57, 59, 60, 61, 62, 63, 64, 65, 66, 35, 36, + 37, 37, 41, 43, 45, 46, 46, 47, 46, 46, 45, 46, 46, 47, 48, 49, 50, 52, + 53, 55, 56, 56, 58, 59, 60, 61, 62, 63, 64, 64, 36, 38, 39, 40, 43, 45, + 47, 47, 47, 48, 46, 46, 45, 46, 46, 47, 48, 48, 50, 51, 52, 54, 55, 55, + 57, 58, 58, 59, 61, 62, 63, 64, 41, 42, 42, 42, 45, 46, 47, 48, 49, 50, + 49, 49, 49, 50, 50, 50, 51, 52, 53, 54, 55, 57, 58, 58, 60, 60, 59, 59, + 60, 61, 61, 62, 44, 44, 44, 44, 46, 46, 47, 49, 50, 51, 51, 51, 51, 51, + 51, 52, 53, 53, 54, 56, 56, 59, 59, 59, 61, 61, 61, 62, 63, 62, 62, 62, + 49, 47, 47, 46, 47, 47, 48, 50, 51, 53, 53, 53, 53, 54, 54, 54, 55, 55, + 56, 58, 58, 60, 61, 61, 63, 63, 64, 63, 63, 64, 65, 66, 48, 47, 46, 45, + 46, 46, 46, 49, 51, 53, 54, 54, 55, 56, 56, 57, 58, 59, 60, 61, 61, 63, + 64, 64, 65, 65, 64, 65, 66, 66, 66, 66, 48, 47, 46, 45, 46, 46, 46, 49, + 51, 53, 54, 55, 56, 57, 57, 58, 59, 60, 61, 62, 63, 65, 65, 65, 66, 67, + 68, 67, 67, 67, 68, 69, 49, 47, 46, 45, 45, 45, 45, 49, 51, 53, 55, 56, + 58, 59, 60, 61, 62, 62, 63, 65, 65, 67, 68, 68, 69, 70, 69, 69, 69, 70, + 69, 69, 50, 48, 47, 46, 46, 46, 46, 50, 51, 54, 56, 57, 59, 61, 62, 63, + 64, 65, 66, 68, 68, 70, 71, 71, 72, 71, 71, 72, 71, 71, 71, 72, 51, 48, + 47, 46, 47, 46, 46, 50, 51, 54, 56, 57, 60, 62, 62, 64, 65, 66, 67, 69, + 69, 71, 72, 72, 73, 74, 74, 72, 73, 74, 73, 73, 52, 50, 48, 47, 47, 47, + 47, 50, 52, 54, 57, 58, 61, 63, 64, 66, 68, 68, 70, 72, 72, 75, 75, 75, + 77, 76, 75, 76, 76, 74, 75, 76, 54, 51, 50, 49, 49, 48, 48, 51, 53, 55, + 58, 59, 62, 64, 65, 68, 70, 70, 73, 74, 75, 77, 78, 78, 79, 78, 79, 78, + 77, 78, 77, 77, 55, 52, 51, 50, 49, 49, 48, 52, 53, 55, 59, 60, 62, 65, + 66, 68, 70, 71, 73, 75, 76, 78, 79, 79, 80, 81, 80, 80, 81, 79, 79, 81, + 57, 54, 53, 52, 51, 50, 50, 53, 54, 56, 60, 61, 63, 66, 67, 70, 73, 73, + 76, 78, 79, 82, 82, 83, 84, 83, 83, 83, 82, 83, 82, 81, 59, 56, 54, 53, + 53, 52, 51, 54, 56, 58, 61, 62, 65, 68, 69, 72, 74, 75, 78, 80, 81, 84, + 85, 85, 86, 86, 86, 84, 85, 84, 84, 85, 60, 57, 55, 54, 53, 53, 52, 55, + 56, 58, 61, 63, 65, 68, 69, 72, 75, 76, 79, 81, 82, 85, 86, 86, 88, 88, + 87, 88, 86, 87, 87, 85, 63, 60, 58, 57, 56, 55, 54, 57, 59, 60, 63, 65, + 67, 70, 71, 75, 77, 78, 82, 84, 85, 89, 89, 90, 92, 89, 91, 89, 90, 89, + 88, 89, 64, 61, 59, 58, 57, 56, 55, 58, 59, 61, 64, 65, 68, 71, 72, 75, + 78, 79, 82, 85, 86, 89, 90, 91, 93, 94, 92, 92, 91, 91, 92, 90, 65, 61, + 60, 58, 57, 56, 55, 58, 59, 61, 64, 65, 68, 71, 72, 75, 78, 79, 83, 85, + 86, 90, 91, 91, 93, 94, 95, 94, 94, 94, 93, 94, 67, 63, 61, 60, 59, 58, + 57, 60, 61, 63, 65, 66, 69, 72, 73, 77, 79, 80, 84, 86, 88, 92, 93, 93, + 95, 95, 96, 97, 96, 95, 96, 94, 68, 64, 63, 61, 60, 59, 58, 60, 61, 63, + 65, 67, 70, 71, 74, 76, 78, 81, 83, 86, 88, 89, 94, 94, 95, 97, 97, 98, + 99, 99, 97, 99, 68, 65, 64, 62, 61, 60, 58, 59, 61, 64, 64, 68, 69, 71, + 74, 75, 79, 80, 83, 86, 87, 91, 92, 95, 96, 97, 99, 99, 100, 100, 101, + 99, 69, 66, 65, 63, 62, 61, 59, 59, 62, 63, 65, 67, 69, 72, 72, 76, 78, + 80, 83, 84, 88, 89, 92, 94, 97, 98, 99, 101, 100, 102, 102, 104, 70, 67, + 66, 63, 63, 62, 61, 60, 63, 63, 66, 67, 69, 71, 73, 76, 77, 81, 82, 85, + 86, 90, 91, 94, 96, 99, 100, 100, 103, 102, 104, 104, 71, 67, 67, 64, + 64, 63, 62, 61, 62, 64, 66, 67, 70, 71, 74, 74, 78, 79, 83, 84, 87, 89, + 91, 94, 95, 99, 100, 102, 102, 104, 104, 106, 72, 68, 68, 65, 65, 64, + 63, 61, 62, 65, 66, 68, 69, 71, 73, 75, 77, 79, 82, 84, 87, 88, 92, 93, + 96, 97, 101, 102, 104, 104, 106, 106, 73, 69, 69, 66, 66, 64, 64, 62, + 62, 66, 66, 69, 69, 72, 73, 76, 77, 81, 81, 85, 85, 89, 90, 94, 94, 99, + 99, 104, 104, 106, 106, 108], + /* Size 4x8 */ + [31, 47, 54, 64, 38, 46, 50, 60, 46, 53, 57, 62, 46, 56, 66, 71, 50, 59, + 74, 79, 57, 64, 82, 88, 61, 65, 85, 97, 65, 67, 82, 99], + /* Size 8x4 */ + [31, 38, 46, 46, 50, 57, 61, 65, 47, 46, 53, 56, 59, 64, 65, 67, 54, 50, + 57, 66, 74, 82, 85, 82, 64, 60, 62, 71, 79, 88, 97, 99], + /* Size 8x16 */ + [32, 34, 48, 49, 54, 63, 67, 69, 31, 36, 46, 46, 50, 58, 62, 65, 33, 40, + 47, 46, 49, 56, 59, 62, 37, 44, 47, 45, 48, 54, 57, 60, 44, 46, 51, 51, + 53, 59, 60, 61, 48, 46, 53, 56, 58, 64, 64, 64, 49, 45, 53, 58, 62, 67, + 70, 68, 51, 47, 54, 60, 65, 71, 73, 72, 54, 49, 55, 62, 70, 77, 77, 76, + 57, 51, 56, 64, 73, 82, 83, 81, 60, 53, 58, 65, 75, 85, 89, 85, 64, 57, + 61, 68, 78, 89, 93, 89, 66, 59, 63, 69, 79, 91, 94, 93, 68, 61, 63, 71, + 79, 87, 96, 98, 70, 63, 63, 70, 80, 89, 97, 100, 72, 65, 63, 69, 77, 86, + 95, 102], + /* Size 16x8 */ + [32, 31, 33, 37, 44, 48, 49, 51, 54, 57, 60, 64, 66, 68, 70, 72, 34, 36, + 40, 44, 46, 46, 45, 47, 49, 51, 53, 57, 59, 61, 63, 65, 48, 46, 47, 47, + 51, 53, 53, 54, 55, 56, 58, 61, 63, 63, 63, 63, 49, 46, 46, 45, 51, 56, + 58, 60, 62, 64, 65, 68, 69, 71, 70, 69, 54, 50, 49, 48, 53, 58, 62, 65, + 70, 73, 75, 78, 79, 79, 80, 77, 63, 58, 56, 54, 59, 64, 67, 71, 77, 82, + 85, 89, 91, 87, 89, 86, 67, 62, 59, 57, 60, 64, 70, 73, 77, 83, 89, 93, + 94, 96, 97, 95, 69, 65, 62, 60, 61, 64, 68, 72, 76, 81, 85, 89, 93, 98, + 100, 102], + /* Size 16x32 */ + [32, 31, 34, 37, 48, 48, 49, 52, 54, 57, 63, 64, 67, 68, 69, 69, 31, 31, + 35, 38, 47, 47, 47, 50, 51, 54, 60, 61, 63, 64, 65, 66, 31, 32, 36, 39, + 46, 46, 46, 48, 50, 53, 58, 59, 62, 63, 65, 66, 30, 32, 36, 40, 46, 45, + 45, 48, 49, 52, 57, 58, 60, 61, 62, 63, 33, 36, 40, 43, 47, 46, 46, 47, + 49, 51, 56, 57, 59, 60, 62, 63, 35, 38, 42, 45, 47, 46, 45, 47, 48, 50, + 55, 56, 58, 60, 61, 61, 37, 40, 44, 47, 47, 46, 45, 47, 48, 50, 54, 55, + 57, 58, 60, 61, 42, 43, 45, 47, 50, 50, 49, 50, 51, 53, 57, 58, 59, 58, + 59, 59, 44, 44, 46, 47, 51, 51, 51, 52, 53, 54, 59, 59, 60, 61, 61, 60, + 49, 46, 47, 48, 53, 53, 53, 54, 55, 57, 60, 61, 63, 62, 62, 63, 48, 46, + 46, 47, 53, 54, 56, 57, 58, 60, 64, 64, 64, 64, 64, 63, 48, 45, 46, 46, + 53, 55, 56, 58, 59, 61, 65, 65, 66, 66, 65, 66, 49, 45, 45, 46, 53, 56, + 58, 61, 62, 64, 67, 68, 70, 67, 68, 66, 50, 46, 46, 46, 54, 56, 59, 63, + 65, 66, 70, 71, 70, 71, 68, 70, 51, 47, 47, 47, 54, 57, 60, 64, 65, 68, + 71, 72, 73, 71, 72, 70, 52, 48, 47, 47, 54, 57, 61, 66, 68, 71, 75, 75, + 76, 75, 73, 73, 54, 49, 49, 48, 55, 58, 62, 68, 70, 73, 77, 78, 77, 77, + 76, 74, 54, 50, 49, 49, 55, 59, 62, 68, 70, 74, 78, 79, 81, 79, 77, 78, + 57, 52, 51, 50, 56, 60, 64, 70, 73, 76, 82, 82, 83, 82, 81, 78, 59, 54, + 52, 52, 58, 61, 65, 72, 74, 78, 84, 85, 85, 83, 82, 82, 60, 54, 53, 52, + 58, 62, 65, 72, 75, 79, 85, 86, 89, 87, 85, 82, 63, 57, 56, 55, 60, 64, + 67, 75, 77, 82, 89, 90, 90, 88, 87, 86, 64, 58, 57, 55, 61, 64, 68, 75, + 78, 82, 89, 90, 93, 91, 89, 87, 64, 59, 57, 56, 61, 65, 68, 75, 78, 83, + 90, 91, 94, 93, 92, 91, 66, 60, 59, 57, 63, 66, 69, 77, 79, 84, 91, 93, + 94, 95, 93, 91, 67, 61, 60, 58, 63, 65, 70, 75, 78, 85, 88, 93, 96, 97, + 97, 95, 68, 62, 61, 59, 63, 64, 71, 74, 79, 84, 87, 94, 96, 97, 98, 96, + 69, 63, 62, 60, 63, 65, 71, 72, 80, 82, 88, 93, 96, 99, 100, 101, 70, + 64, 63, 60, 63, 66, 70, 73, 80, 81, 89, 90, 97, 99, 100, 101, 71, 65, + 64, 61, 63, 67, 70, 74, 78, 82, 88, 90, 97, 99, 102, 103, 72, 65, 65, + 62, 63, 68, 69, 75, 77, 83, 86, 92, 95, 100, 102, 103, 73, 66, 66, 63, + 63, 69, 69, 76, 76, 84, 84, 93, 93, 101, 101, 105], + /* Size 32x16 */ + [32, 31, 31, 30, 33, 35, 37, 42, 44, 49, 48, 48, 49, 50, 51, 52, 54, 54, + 57, 59, 60, 63, 64, 64, 66, 67, 68, 69, 70, 71, 72, 73, 31, 31, 32, 32, + 36, 38, 40, 43, 44, 46, 46, 45, 45, 46, 47, 48, 49, 50, 52, 54, 54, 57, + 58, 59, 60, 61, 62, 63, 64, 65, 65, 66, 34, 35, 36, 36, 40, 42, 44, 45, + 46, 47, 46, 46, 45, 46, 47, 47, 49, 49, 51, 52, 53, 56, 57, 57, 59, 60, + 61, 62, 63, 64, 65, 66, 37, 38, 39, 40, 43, 45, 47, 47, 47, 48, 47, 46, + 46, 46, 47, 47, 48, 49, 50, 52, 52, 55, 55, 56, 57, 58, 59, 60, 60, 61, + 62, 63, 48, 47, 46, 46, 47, 47, 47, 50, 51, 53, 53, 53, 53, 54, 54, 54, + 55, 55, 56, 58, 58, 60, 61, 61, 63, 63, 63, 63, 63, 63, 63, 63, 48, 47, + 46, 45, 46, 46, 46, 50, 51, 53, 54, 55, 56, 56, 57, 57, 58, 59, 60, 61, + 62, 64, 64, 65, 66, 65, 64, 65, 66, 67, 68, 69, 49, 47, 46, 45, 46, 45, + 45, 49, 51, 53, 56, 56, 58, 59, 60, 61, 62, 62, 64, 65, 65, 67, 68, 68, + 69, 70, 71, 71, 70, 70, 69, 69, 52, 50, 48, 48, 47, 47, 47, 50, 52, 54, + 57, 58, 61, 63, 64, 66, 68, 68, 70, 72, 72, 75, 75, 75, 77, 75, 74, 72, + 73, 74, 75, 76, 54, 51, 50, 49, 49, 48, 48, 51, 53, 55, 58, 59, 62, 65, + 65, 68, 70, 70, 73, 74, 75, 77, 78, 78, 79, 78, 79, 80, 80, 78, 77, 76, + 57, 54, 53, 52, 51, 50, 50, 53, 54, 57, 60, 61, 64, 66, 68, 71, 73, 74, + 76, 78, 79, 82, 82, 83, 84, 85, 84, 82, 81, 82, 83, 84, 63, 60, 58, 57, + 56, 55, 54, 57, 59, 60, 64, 65, 67, 70, 71, 75, 77, 78, 82, 84, 85, 89, + 89, 90, 91, 88, 87, 88, 89, 88, 86, 84, 64, 61, 59, 58, 57, 56, 55, 58, + 59, 61, 64, 65, 68, 71, 72, 75, 78, 79, 82, 85, 86, 90, 90, 91, 93, 93, + 94, 93, 90, 90, 92, 93, 67, 63, 62, 60, 59, 58, 57, 59, 60, 63, 64, 66, + 70, 70, 73, 76, 77, 81, 83, 85, 89, 90, 93, 94, 94, 96, 96, 96, 97, 97, + 95, 93, 68, 64, 63, 61, 60, 60, 58, 58, 61, 62, 64, 66, 67, 71, 71, 75, + 77, 79, 82, 83, 87, 88, 91, 93, 95, 97, 97, 99, 99, 99, 100, 101, 69, + 65, 65, 62, 62, 61, 60, 59, 61, 62, 64, 65, 68, 68, 72, 73, 76, 77, 81, + 82, 85, 87, 89, 92, 93, 97, 98, 100, 100, 102, 102, 101, 69, 66, 66, 63, + 63, 61, 61, 59, 60, 63, 63, 66, 66, 70, 70, 73, 74, 78, 78, 82, 82, 86, + 87, 91, 91, 95, 96, 101, 101, 103, 103, 105], + /* Size 4x16 */ + [31, 48, 57, 68, 32, 46, 53, 63, 36, 46, 51, 60, 40, 46, 50, 58, 44, 51, + 54, 61, 46, 54, 60, 64, 45, 56, 64, 67, 47, 57, 68, 71, 49, 58, 73, 77, + 52, 60, 76, 82, 54, 62, 79, 87, 58, 64, 82, 91, 60, 66, 84, 95, 62, 64, + 84, 97, 64, 66, 81, 99, 65, 68, 83, 100], + /* Size 16x4 */ + [31, 32, 36, 40, 44, 46, 45, 47, 49, 52, 54, 58, 60, 62, 64, 65, 48, 46, + 46, 46, 51, 54, 56, 57, 58, 60, 62, 64, 66, 64, 66, 68, 57, 53, 51, 50, + 54, 60, 64, 68, 73, 76, 79, 82, 84, 84, 81, 83, 68, 63, 60, 58, 61, 64, + 67, 71, 77, 82, 87, 91, 95, 97, 99, 100], + /* Size 8x32 */ + [32, 34, 48, 49, 54, 63, 67, 69, 31, 35, 47, 47, 51, 60, 63, 65, 31, 36, + 46, 46, 50, 58, 62, 65, 30, 36, 46, 45, 49, 57, 60, 62, 33, 40, 47, 46, + 49, 56, 59, 62, 35, 42, 47, 45, 48, 55, 58, 61, 37, 44, 47, 45, 48, 54, + 57, 60, 42, 45, 50, 49, 51, 57, 59, 59, 44, 46, 51, 51, 53, 59, 60, 61, + 49, 47, 53, 53, 55, 60, 63, 62, 48, 46, 53, 56, 58, 64, 64, 64, 48, 46, + 53, 56, 59, 65, 66, 65, 49, 45, 53, 58, 62, 67, 70, 68, 50, 46, 54, 59, + 65, 70, 70, 68, 51, 47, 54, 60, 65, 71, 73, 72, 52, 47, 54, 61, 68, 75, + 76, 73, 54, 49, 55, 62, 70, 77, 77, 76, 54, 49, 55, 62, 70, 78, 81, 77, + 57, 51, 56, 64, 73, 82, 83, 81, 59, 52, 58, 65, 74, 84, 85, 82, 60, 53, + 58, 65, 75, 85, 89, 85, 63, 56, 60, 67, 77, 89, 90, 87, 64, 57, 61, 68, + 78, 89, 93, 89, 64, 57, 61, 68, 78, 90, 94, 92, 66, 59, 63, 69, 79, 91, + 94, 93, 67, 60, 63, 70, 78, 88, 96, 97, 68, 61, 63, 71, 79, 87, 96, 98, + 69, 62, 63, 71, 80, 88, 96, 100, 70, 63, 63, 70, 80, 89, 97, 100, 71, + 64, 63, 70, 78, 88, 97, 102, 72, 65, 63, 69, 77, 86, 95, 102, 73, 66, + 63, 69, 76, 84, 93, 101], + /* Size 32x8 */ + [32, 31, 31, 30, 33, 35, 37, 42, 44, 49, 48, 48, 49, 50, 51, 52, 54, 54, + 57, 59, 60, 63, 64, 64, 66, 67, 68, 69, 70, 71, 72, 73, 34, 35, 36, 36, + 40, 42, 44, 45, 46, 47, 46, 46, 45, 46, 47, 47, 49, 49, 51, 52, 53, 56, + 57, 57, 59, 60, 61, 62, 63, 64, 65, 66, 48, 47, 46, 46, 47, 47, 47, 50, + 51, 53, 53, 53, 53, 54, 54, 54, 55, 55, 56, 58, 58, 60, 61, 61, 63, 63, + 63, 63, 63, 63, 63, 63, 49, 47, 46, 45, 46, 45, 45, 49, 51, 53, 56, 56, + 58, 59, 60, 61, 62, 62, 64, 65, 65, 67, 68, 68, 69, 70, 71, 71, 70, 70, + 69, 69, 54, 51, 50, 49, 49, 48, 48, 51, 53, 55, 58, 59, 62, 65, 65, 68, + 70, 70, 73, 74, 75, 77, 78, 78, 79, 78, 79, 80, 80, 78, 77, 76, 63, 60, + 58, 57, 56, 55, 54, 57, 59, 60, 64, 65, 67, 70, 71, 75, 77, 78, 82, 84, + 85, 89, 89, 90, 91, 88, 87, 88, 89, 88, 86, 84, 67, 63, 62, 60, 59, 58, + 57, 59, 60, 63, 64, 66, 70, 70, 73, 76, 77, 81, 83, 85, 89, 90, 93, 94, + 94, 96, 96, 96, 97, 97, 95, 93, 69, 65, 65, 62, 62, 61, 60, 59, 61, 62, + 64, 65, 68, 68, 72, 73, 76, 77, 81, 82, 85, 87, 89, 92, 93, 97, 98, 100, + 100, 102, 102, 101] + ] + ], + [ + [ /* Luma */ + /* Size 4x4 */ + [32, 37, 58, 81, 37, 54, 72, 91, 58, 72, 102, 121, 81, 91, 121, 156], + /* Size 8x8 */ + [32, 32, 35, 42, 53, 68, 78, 90, 32, 33, 36, 42, 51, 64, 74, 84, 35, 36, + 46, 52, 60, 72, 80, 87, 42, 42, 52, 63, 73, 84, 92, 98, 53, 51, 60, 73, + 86, 100, 109, 114, 68, 64, 72, 84, 100, 117, 128, 133, 78, 74, 80, 92, + 109, 128, 140, 155, 90, 84, 87, 98, 114, 133, 155, 168], + /* Size 16x16 */ + [32, 31, 31, 32, 34, 36, 41, 47, 54, 59, 65, 74, 82, 87, 92, 97, 31, 32, + 32, 32, 34, 35, 39, 45, 50, 55, 61, 69, 76, 81, 87, 92, 31, 32, 33, 33, + 35, 36, 40, 44, 49, 54, 59, 67, 73, 78, 83, 88, 32, 32, 33, 35, 37, 38, + 41, 45, 49, 53, 58, 65, 71, 75, 80, 86, 34, 34, 35, 37, 39, 42, 46, 50, + 54, 58, 63, 70, 76, 80, 84, 85, 36, 35, 36, 38, 42, 48, 52, 56, 60, 64, + 68, 75, 80, 85, 90, 91, 41, 39, 40, 41, 46, 52, 57, 62, 67, 71, 75, 83, + 88, 92, 95, 97, 47, 45, 44, 45, 50, 56, 62, 69, 75, 79, 84, 91, 97, 100, + 102, 104, 54, 50, 49, 49, 54, 60, 67, 75, 82, 87, 92, 100, 106, 110, + 109, 112, 59, 55, 54, 53, 58, 64, 71, 79, 87, 92, 98, 106, 112, 117, + 117, 121, 65, 61, 59, 58, 63, 68, 75, 84, 92, 98, 105, 114, 120, 125, + 126, 130, 74, 69, 67, 65, 70, 75, 83, 91, 100, 106, 114, 123, 131, 135, + 137, 140, 82, 76, 73, 71, 76, 80, 88, 97, 106, 112, 120, 131, 139, 144, + 148, 150, 87, 81, 78, 75, 80, 85, 92, 100, 110, 117, 125, 135, 144, 150, + 155, 162, 92, 87, 83, 80, 84, 90, 95, 102, 109, 117, 126, 137, 148, 155, + 162, 168, 97, 92, 88, 86, 85, 91, 97, 104, 112, 121, 130, 140, 150, 162, + 168, 174], + /* Size 32x32 */ + [32, 31, 31, 31, 31, 31, 32, 32, 34, 35, 36, 39, 41, 44, 47, 48, 54, 56, + 59, 64, 65, 71, 74, 80, 82, 83, 87, 90, 92, 95, 97, 100, 31, 32, 32, 32, + 32, 32, 32, 33, 34, 35, 35, 38, 40, 42, 45, 46, 51, 53, 56, 61, 62, 68, + 71, 76, 78, 78, 83, 85, 88, 90, 92, 95, 31, 32, 32, 32, 32, 32, 32, 33, + 34, 34, 35, 38, 39, 42, 45, 45, 50, 52, 55, 60, 61, 67, 69, 74, 76, 77, + 81, 84, 87, 89, 92, 95, 31, 32, 32, 32, 32, 32, 32, 33, 33, 34, 34, 37, + 38, 41, 44, 44, 49, 51, 54, 58, 59, 65, 68, 72, 74, 75, 79, 81, 84, 86, + 88, 90, 31, 32, 32, 32, 33, 33, 33, 34, 35, 36, 36, 39, 40, 42, 44, 45, + 49, 51, 54, 58, 59, 64, 67, 71, 73, 74, 78, 80, 83, 85, 88, 90, 31, 32, + 32, 32, 33, 33, 34, 34, 35, 36, 36, 39, 40, 42, 45, 45, 50, 51, 54, 58, + 59, 64, 67, 71, 73, 74, 78, 80, 82, 84, 86, 89, 32, 32, 32, 32, 33, 34, + 35, 36, 37, 38, 38, 40, 41, 42, 45, 46, 49, 51, 53, 57, 58, 63, 65, 69, + 71, 72, 75, 78, 80, 83, 86, 89, 32, 33, 33, 33, 34, 34, 36, 36, 38, 39, + 40, 42, 43, 44, 47, 47, 51, 53, 55, 59, 60, 65, 67, 71, 73, 73, 77, 78, + 80, 82, 84, 86, 34, 34, 34, 33, 35, 35, 37, 38, 39, 42, 42, 45, 46, 47, + 50, 51, 54, 56, 58, 62, 63, 68, 70, 74, 76, 76, 80, 82, 84, 85, 85, 86, + 35, 35, 34, 34, 36, 36, 38, 39, 42, 46, 47, 49, 50, 52, 55, 55, 59, 60, + 62, 66, 67, 72, 74, 78, 79, 80, 83, 84, 85, 87, 90, 92, 36, 35, 35, 34, + 36, 36, 38, 40, 42, 47, 48, 50, 52, 54, 56, 57, 60, 61, 64, 67, 68, 73, + 75, 79, 80, 81, 85, 87, 90, 91, 91, 92, 39, 38, 38, 37, 39, 39, 40, 42, + 45, 49, 50, 54, 55, 58, 60, 61, 65, 66, 69, 72, 73, 78, 80, 84, 86, 86, + 90, 91, 91, 92, 95, 97, 41, 40, 39, 38, 40, 40, 41, 43, 46, 50, 52, 55, + 57, 60, 62, 63, 67, 69, 71, 75, 75, 80, 83, 86, 88, 89, 92, 93, 95, 97, + 97, 98, 44, 42, 42, 41, 42, 42, 42, 44, 47, 52, 54, 58, 60, 63, 66, 67, + 71, 73, 75, 79, 79, 84, 86, 90, 92, 92, 96, 98, 98, 98, 101, 104, 47, + 45, 45, 44, 44, 45, 45, 47, 50, 55, 56, 60, 62, 66, 69, 70, 75, 77, 79, + 83, 84, 89, 91, 95, 97, 97, 100, 99, 102, 105, 104, 104, 48, 46, 45, 44, + 45, 45, 46, 47, 51, 55, 57, 61, 63, 67, 70, 71, 76, 78, 80, 84, 85, 90, + 93, 96, 98, 99, 102, 106, 106, 105, 108, 111, 54, 51, 50, 49, 49, 50, + 49, 51, 54, 59, 60, 65, 67, 71, 75, 76, 82, 84, 87, 91, 92, 97, 100, + 104, 106, 106, 110, 108, 109, 112, 112, 111, 56, 53, 52, 51, 51, 51, 51, + 53, 56, 60, 61, 66, 69, 73, 77, 78, 84, 86, 89, 93, 94, 100, 102, 106, + 108, 109, 112, 113, 115, 114, 116, 119, 59, 56, 55, 54, 54, 54, 53, 55, + 58, 62, 64, 69, 71, 75, 79, 80, 87, 89, 92, 97, 98, 103, 106, 110, 112, + 113, 117, 118, 117, 121, 121, 119, 64, 61, 60, 58, 58, 58, 57, 59, 62, + 66, 67, 72, 75, 79, 83, 84, 91, 93, 97, 102, 103, 109, 112, 116, 118, + 119, 122, 121, 125, 123, 125, 128, 65, 62, 61, 59, 59, 59, 58, 60, 63, + 67, 68, 73, 75, 79, 84, 85, 92, 94, 98, 103, 105, 111, 114, 118, 120, + 121, 125, 129, 126, 129, 130, 129, 71, 68, 67, 65, 64, 64, 63, 65, 68, + 72, 73, 78, 80, 84, 89, 90, 97, 100, 103, 109, 111, 117, 120, 125, 127, + 128, 133, 130, 134, 133, 133, 137, 74, 71, 69, 68, 67, 67, 65, 67, 70, + 74, 75, 80, 83, 86, 91, 93, 100, 102, 106, 112, 114, 120, 123, 128, 131, + 131, 135, 137, 137, 138, 140, 137, 80, 76, 74, 72, 71, 71, 69, 71, 74, + 78, 79, 84, 86, 90, 95, 96, 104, 106, 110, 116, 118, 125, 128, 134, 136, + 137, 142, 141, 142, 143, 143, 147, 82, 78, 76, 74, 73, 73, 71, 73, 76, + 79, 80, 86, 88, 92, 97, 98, 106, 108, 112, 118, 120, 127, 131, 136, 139, + 139, 144, 147, 148, 147, 150, 148, 83, 78, 77, 75, 74, 74, 72, 73, 76, + 80, 81, 86, 89, 92, 97, 99, 106, 109, 113, 119, 121, 128, 131, 137, 139, + 140, 145, 150, 152, 155, 152, 157, 87, 83, 81, 79, 78, 78, 75, 77, 80, + 83, 85, 90, 92, 96, 100, 102, 110, 112, 117, 122, 125, 133, 135, 142, + 144, 145, 150, 151, 155, 158, 162, 158, 90, 85, 84, 81, 80, 80, 78, 78, + 82, 84, 87, 91, 93, 98, 99, 106, 108, 113, 118, 121, 129, 130, 137, 141, + 147, 150, 151, 156, 156, 161, 164, 169, 92, 88, 87, 84, 83, 82, 80, 80, + 84, 85, 90, 91, 95, 98, 102, 106, 109, 115, 117, 125, 126, 134, 137, + 142, 148, 152, 155, 156, 162, 162, 168, 170, 95, 90, 89, 86, 85, 84, 83, + 82, 85, 87, 91, 92, 97, 98, 105, 105, 112, 114, 121, 123, 129, 133, 138, + 143, 147, 155, 158, 161, 162, 168, 168, 174, 97, 92, 92, 88, 88, 86, 86, + 84, 85, 90, 91, 95, 97, 101, 104, 108, 112, 116, 121, 125, 130, 133, + 140, 143, 150, 152, 162, 164, 168, 168, 174, 175, 100, 95, 95, 90, 90, + 89, 89, 86, 86, 92, 92, 97, 98, 104, 104, 111, 111, 119, 119, 128, 129, + 137, 137, 147, 148, 157, 158, 169, 170, 174, 175, 181], + /* Size 4x8 */ + [32, 35, 59, 83, 32, 36, 57, 78, 34, 47, 65, 82, 41, 53, 78, 97, 51, 61, + 92, 111, 65, 73, 108, 129, 75, 81, 117, 148, 86, 92, 119, 154], + /* Size 8x4 */ + [32, 32, 34, 41, 51, 65, 75, 86, 35, 36, 47, 53, 61, 73, 81, 92, 59, 57, + 65, 78, 92, 108, 117, 119, 83, 78, 82, 97, 111, 129, 148, 154], + /* Size 8x16 */ + [32, 31, 35, 44, 53, 65, 82, 90, 31, 32, 34, 41, 50, 61, 76, 85, 31, 33, + 35, 42, 49, 59, 73, 81, 32, 34, 37, 42, 49, 58, 71, 79, 34, 35, 41, 48, + 54, 63, 76, 81, 36, 36, 46, 54, 60, 68, 80, 87, 41, 40, 49, 60, 67, 76, + 88, 93, 47, 44, 53, 66, 75, 84, 97, 101, 53, 50, 57, 71, 82, 92, 106, + 108, 58, 54, 61, 75, 87, 98, 112, 116, 65, 59, 66, 79, 92, 105, 120, + 124, 74, 67, 73, 86, 100, 113, 131, 134, 82, 73, 79, 92, 105, 120, 139, + 142, 87, 78, 83, 96, 110, 125, 144, 153, 92, 83, 84, 97, 114, 132, 150, + 157, 97, 88, 86, 97, 111, 128, 147, 163], + /* Size 16x8 */ + [32, 31, 31, 32, 34, 36, 41, 47, 53, 58, 65, 74, 82, 87, 92, 97, 31, 32, + 33, 34, 35, 36, 40, 44, 50, 54, 59, 67, 73, 78, 83, 88, 35, 34, 35, 37, + 41, 46, 49, 53, 57, 61, 66, 73, 79, 83, 84, 86, 44, 41, 42, 42, 48, 54, + 60, 66, 71, 75, 79, 86, 92, 96, 97, 97, 53, 50, 49, 49, 54, 60, 67, 75, + 82, 87, 92, 100, 105, 110, 114, 111, 65, 61, 59, 58, 63, 68, 76, 84, 92, + 98, 105, 113, 120, 125, 132, 128, 82, 76, 73, 71, 76, 80, 88, 97, 106, + 112, 120, 131, 139, 144, 150, 147, 90, 85, 81, 79, 81, 87, 93, 101, 108, + 116, 124, 134, 142, 153, 157, 163], + /* Size 16x32 */ + [32, 31, 31, 32, 35, 36, 44, 47, 53, 62, 65, 79, 82, 88, 90, 93, 31, 32, + 32, 32, 35, 35, 42, 45, 51, 59, 62, 75, 78, 83, 86, 88, 31, 32, 32, 32, + 34, 35, 41, 45, 50, 58, 61, 74, 76, 82, 85, 88, 31, 32, 32, 33, 34, 34, + 41, 44, 49, 57, 59, 72, 74, 79, 82, 84, 31, 32, 33, 34, 35, 36, 42, 44, + 49, 57, 59, 71, 73, 79, 81, 84, 32, 32, 33, 34, 36, 36, 42, 45, 50, 57, + 59, 71, 73, 78, 80, 82, 32, 33, 34, 35, 37, 38, 42, 45, 49, 56, 58, 69, + 71, 76, 79, 83, 32, 33, 34, 36, 39, 40, 44, 47, 51, 58, 60, 71, 73, 76, + 78, 80, 34, 34, 35, 37, 41, 42, 48, 50, 54, 61, 63, 73, 76, 81, 81, 80, + 35, 34, 36, 38, 45, 47, 52, 55, 59, 65, 67, 77, 79, 82, 83, 86, 36, 34, + 36, 38, 46, 48, 54, 56, 60, 66, 68, 78, 80, 85, 87, 86, 39, 37, 39, 40, + 48, 50, 58, 60, 65, 71, 73, 84, 86, 89, 88, 91, 41, 39, 40, 41, 49, 51, + 60, 62, 67, 74, 76, 86, 88, 91, 93, 91, 44, 41, 42, 43, 51, 53, 63, 66, + 71, 78, 79, 90, 92, 97, 94, 97, 47, 44, 44, 45, 53, 56, 66, 69, 75, 82, + 84, 95, 97, 98, 101, 98, 48, 45, 45, 46, 54, 56, 67, 70, 76, 83, 85, 96, + 98, 104, 101, 105, 53, 49, 50, 50, 57, 60, 71, 75, 82, 90, 92, 103, 106, + 107, 108, 105, 55, 51, 51, 51, 59, 61, 72, 77, 84, 92, 94, 106, 108, + 111, 110, 112, 58, 54, 54, 54, 61, 63, 75, 79, 87, 95, 98, 110, 112, + 117, 116, 113, 63, 58, 58, 57, 65, 67, 78, 83, 91, 100, 103, 116, 118, + 119, 119, 121, 65, 60, 59, 58, 66, 68, 79, 84, 92, 102, 105, 118, 120, + 127, 124, 122, 71, 65, 64, 63, 71, 73, 84, 89, 97, 108, 111, 125, 127, + 129, 129, 130, 74, 68, 67, 66, 73, 75, 86, 91, 100, 110, 113, 128, 131, + 135, 134, 130, 79, 72, 71, 70, 77, 79, 90, 95, 104, 115, 118, 133, 136, + 140, 139, 140, 82, 75, 73, 72, 79, 81, 92, 97, 105, 117, 120, 136, 139, + 145, 142, 140, 82, 75, 74, 72, 79, 81, 92, 97, 106, 117, 121, 136, 139, + 148, 150, 149, 87, 79, 78, 76, 83, 85, 96, 100, 110, 120, 125, 141, 144, + 148, 153, 150, 89, 82, 81, 78, 83, 87, 97, 99, 113, 118, 128, 139, 145, + 153, 157, 161, 92, 84, 83, 80, 84, 89, 97, 101, 114, 116, 132, 135, 150, + 153, 157, 162, 94, 86, 85, 82, 85, 92, 97, 104, 112, 119, 130, 136, 151, + 154, 163, 166, 97, 88, 88, 85, 86, 94, 97, 107, 111, 123, 128, 140, 147, + 159, 163, 167, 99, 91, 91, 87, 87, 97, 97, 110, 110, 126, 126, 144, 144, + 163, 163, 173], + /* Size 32x16 */ + [32, 31, 31, 31, 31, 32, 32, 32, 34, 35, 36, 39, 41, 44, 47, 48, 53, 55, + 58, 63, 65, 71, 74, 79, 82, 82, 87, 89, 92, 94, 97, 99, 31, 32, 32, 32, + 32, 32, 33, 33, 34, 34, 34, 37, 39, 41, 44, 45, 49, 51, 54, 58, 60, 65, + 68, 72, 75, 75, 79, 82, 84, 86, 88, 91, 31, 32, 32, 32, 33, 33, 34, 34, + 35, 36, 36, 39, 40, 42, 44, 45, 50, 51, 54, 58, 59, 64, 67, 71, 73, 74, + 78, 81, 83, 85, 88, 91, 32, 32, 32, 33, 34, 34, 35, 36, 37, 38, 38, 40, + 41, 43, 45, 46, 50, 51, 54, 57, 58, 63, 66, 70, 72, 72, 76, 78, 80, 82, + 85, 87, 35, 35, 34, 34, 35, 36, 37, 39, 41, 45, 46, 48, 49, 51, 53, 54, + 57, 59, 61, 65, 66, 71, 73, 77, 79, 79, 83, 83, 84, 85, 86, 87, 36, 35, + 35, 34, 36, 36, 38, 40, 42, 47, 48, 50, 51, 53, 56, 56, 60, 61, 63, 67, + 68, 73, 75, 79, 81, 81, 85, 87, 89, 92, 94, 97, 44, 42, 41, 41, 42, 42, + 42, 44, 48, 52, 54, 58, 60, 63, 66, 67, 71, 72, 75, 78, 79, 84, 86, 90, + 92, 92, 96, 97, 97, 97, 97, 97, 47, 45, 45, 44, 44, 45, 45, 47, 50, 55, + 56, 60, 62, 66, 69, 70, 75, 77, 79, 83, 84, 89, 91, 95, 97, 97, 100, 99, + 101, 104, 107, 110, 53, 51, 50, 49, 49, 50, 49, 51, 54, 59, 60, 65, 67, + 71, 75, 76, 82, 84, 87, 91, 92, 97, 100, 104, 105, 106, 110, 113, 114, + 112, 111, 110, 62, 59, 58, 57, 57, 57, 56, 58, 61, 65, 66, 71, 74, 78, + 82, 83, 90, 92, 95, 100, 102, 108, 110, 115, 117, 117, 120, 118, 116, + 119, 123, 126, 65, 62, 61, 59, 59, 59, 58, 60, 63, 67, 68, 73, 76, 79, + 84, 85, 92, 94, 98, 103, 105, 111, 113, 118, 120, 121, 125, 128, 132, + 130, 128, 126, 79, 75, 74, 72, 71, 71, 69, 71, 73, 77, 78, 84, 86, 90, + 95, 96, 103, 106, 110, 116, 118, 125, 128, 133, 136, 136, 141, 139, 135, + 136, 140, 144, 82, 78, 76, 74, 73, 73, 71, 73, 76, 79, 80, 86, 88, 92, + 97, 98, 106, 108, 112, 118, 120, 127, 131, 136, 139, 139, 144, 145, 150, + 151, 147, 144, 88, 83, 82, 79, 79, 78, 76, 76, 81, 82, 85, 89, 91, 97, + 98, 104, 107, 111, 117, 119, 127, 129, 135, 140, 145, 148, 148, 153, + 153, 154, 159, 163, 90, 86, 85, 82, 81, 80, 79, 78, 81, 83, 87, 88, 93, + 94, 101, 101, 108, 110, 116, 119, 124, 129, 134, 139, 142, 150, 153, + 157, 157, 163, 163, 163, 93, 88, 88, 84, 84, 82, 83, 80, 80, 86, 86, 91, + 91, 97, 98, 105, 105, 112, 113, 121, 122, 130, 130, 140, 140, 149, 150, + 161, 162, 166, 167, 173], + /* Size 4x16 */ + [31, 36, 62, 88, 32, 35, 58, 82, 32, 36, 57, 79, 33, 38, 56, 76, 34, 42, + 61, 81, 34, 48, 66, 85, 39, 51, 74, 91, 44, 56, 82, 98, 49, 60, 90, 107, + 54, 63, 95, 117, 60, 68, 102, 127, 68, 75, 110, 135, 75, 81, 117, 145, + 79, 85, 120, 148, 84, 89, 116, 153, 88, 94, 123, 159], + /* Size 16x4 */ + [31, 32, 32, 33, 34, 34, 39, 44, 49, 54, 60, 68, 75, 79, 84, 88, 36, 35, + 36, 38, 42, 48, 51, 56, 60, 63, 68, 75, 81, 85, 89, 94, 62, 58, 57, 56, + 61, 66, 74, 82, 90, 95, 102, 110, 117, 120, 116, 123, 88, 82, 79, 76, + 81, 85, 91, 98, 107, 117, 127, 135, 145, 148, 153, 159], + /* Size 8x32 */ + [32, 31, 35, 44, 53, 65, 82, 90, 31, 32, 35, 42, 51, 62, 78, 86, 31, 32, + 34, 41, 50, 61, 76, 85, 31, 32, 34, 41, 49, 59, 74, 82, 31, 33, 35, 42, + 49, 59, 73, 81, 32, 33, 36, 42, 50, 59, 73, 80, 32, 34, 37, 42, 49, 58, + 71, 79, 32, 34, 39, 44, 51, 60, 73, 78, 34, 35, 41, 48, 54, 63, 76, 81, + 35, 36, 45, 52, 59, 67, 79, 83, 36, 36, 46, 54, 60, 68, 80, 87, 39, 39, + 48, 58, 65, 73, 86, 88, 41, 40, 49, 60, 67, 76, 88, 93, 44, 42, 51, 63, + 71, 79, 92, 94, 47, 44, 53, 66, 75, 84, 97, 101, 48, 45, 54, 67, 76, 85, + 98, 101, 53, 50, 57, 71, 82, 92, 106, 108, 55, 51, 59, 72, 84, 94, 108, + 110, 58, 54, 61, 75, 87, 98, 112, 116, 63, 58, 65, 78, 91, 103, 118, + 119, 65, 59, 66, 79, 92, 105, 120, 124, 71, 64, 71, 84, 97, 111, 127, + 129, 74, 67, 73, 86, 100, 113, 131, 134, 79, 71, 77, 90, 104, 118, 136, + 139, 82, 73, 79, 92, 105, 120, 139, 142, 82, 74, 79, 92, 106, 121, 139, + 150, 87, 78, 83, 96, 110, 125, 144, 153, 89, 81, 83, 97, 113, 128, 145, + 157, 92, 83, 84, 97, 114, 132, 150, 157, 94, 85, 85, 97, 112, 130, 151, + 163, 97, 88, 86, 97, 111, 128, 147, 163, 99, 91, 87, 97, 110, 126, 144, + 163], + /* Size 32x8 */ + [32, 31, 31, 31, 31, 32, 32, 32, 34, 35, 36, 39, 41, 44, 47, 48, 53, 55, + 58, 63, 65, 71, 74, 79, 82, 82, 87, 89, 92, 94, 97, 99, 31, 32, 32, 32, + 33, 33, 34, 34, 35, 36, 36, 39, 40, 42, 44, 45, 50, 51, 54, 58, 59, 64, + 67, 71, 73, 74, 78, 81, 83, 85, 88, 91, 35, 35, 34, 34, 35, 36, 37, 39, + 41, 45, 46, 48, 49, 51, 53, 54, 57, 59, 61, 65, 66, 71, 73, 77, 79, 79, + 83, 83, 84, 85, 86, 87, 44, 42, 41, 41, 42, 42, 42, 44, 48, 52, 54, 58, + 60, 63, 66, 67, 71, 72, 75, 78, 79, 84, 86, 90, 92, 92, 96, 97, 97, 97, + 97, 97, 53, 51, 50, 49, 49, 50, 49, 51, 54, 59, 60, 65, 67, 71, 75, 76, + 82, 84, 87, 91, 92, 97, 100, 104, 105, 106, 110, 113, 114, 112, 111, + 110, 65, 62, 61, 59, 59, 59, 58, 60, 63, 67, 68, 73, 76, 79, 84, 85, 92, + 94, 98, 103, 105, 111, 113, 118, 120, 121, 125, 128, 132, 130, 128, 126, + 82, 78, 76, 74, 73, 73, 71, 73, 76, 79, 80, 86, 88, 92, 97, 98, 106, + 108, 112, 118, 120, 127, 131, 136, 139, 139, 144, 145, 150, 151, 147, + 144, 90, 86, 85, 82, 81, 80, 79, 78, 81, 83, 87, 88, 93, 94, 101, 101, + 108, 110, 116, 119, 124, 129, 134, 139, 142, 150, 153, 157, 157, 163, + 163, 163] + ], + [ /* Chroma */ + /* Size 4x4 */ + [32, 45, 51, 61, 45, 54, 59, 65, 51, 59, 75, 81, 61, 65, 81, 97], + /* Size 8x8 */ + [31, 34, 46, 47, 50, 57, 61, 65, 34, 39, 47, 45, 48, 53, 57, 61, 46, 47, + 52, 52, 54, 58, 61, 62, 47, 45, 52, 58, 62, 65, 68, 68, 50, 48, 54, 62, + 68, 73, 77, 76, 57, 53, 58, 65, 73, 82, 86, 86, 61, 57, 61, 68, 77, 86, + 91, 95, 65, 61, 62, 68, 76, 86, 95, 100], + /* Size 16x16 */ + [32, 31, 33, 36, 41, 49, 49, 50, 52, 54, 57, 61, 64, 67, 68, 70, 31, 31, + 34, 39, 42, 47, 46, 47, 49, 51, 53, 57, 60, 62, 64, 66, 33, 34, 37, 42, + 44, 47, 46, 46, 47, 49, 51, 55, 57, 59, 61, 63, 36, 39, 42, 47, 47, 48, + 46, 46, 47, 48, 50, 53, 55, 57, 59, 61, 41, 42, 44, 47, 48, 50, 49, 50, + 50, 52, 53, 56, 58, 60, 61, 60, 49, 47, 47, 48, 50, 53, 53, 54, 54, 55, + 56, 59, 61, 63, 64, 64, 49, 46, 46, 46, 49, 53, 55, 57, 59, 60, 61, 64, + 66, 67, 67, 67, 50, 47, 46, 46, 50, 54, 57, 61, 63, 64, 66, 69, 70, 72, + 71, 71, 52, 49, 47, 47, 50, 54, 59, 63, 66, 68, 70, 73, 75, 77, 75, 75, + 54, 51, 49, 48, 52, 55, 60, 64, 68, 71, 73, 76, 79, 80, 79, 79, 57, 53, + 51, 50, 53, 56, 61, 66, 70, 73, 76, 80, 82, 84, 83, 84, 61, 57, 55, 53, + 56, 59, 64, 69, 73, 76, 80, 84, 87, 89, 88, 88, 64, 60, 57, 55, 58, 61, + 66, 70, 75, 79, 82, 87, 91, 93, 93, 93, 67, 62, 59, 57, 60, 63, 67, 72, + 77, 80, 84, 89, 93, 95, 96, 97, 68, 64, 61, 59, 61, 64, 67, 71, 75, 79, + 83, 88, 93, 96, 99, 100, 70, 66, 63, 61, 60, 64, 67, 71, 75, 79, 84, 88, + 93, 97, 100, 102], + /* Size 32x32 */ + [32, 31, 31, 30, 33, 33, 36, 38, 41, 47, 49, 48, 49, 49, 50, 50, 52, 53, + 54, 56, 57, 60, 61, 63, 64, 65, 67, 67, 68, 69, 70, 71, 31, 31, 31, 31, + 34, 34, 38, 40, 42, 46, 47, 47, 47, 47, 48, 48, 50, 50, 52, 54, 54, 57, + 58, 60, 61, 61, 63, 64, 65, 65, 66, 67, 31, 31, 31, 31, 34, 35, 39, 40, + 42, 46, 47, 46, 46, 46, 47, 47, 49, 50, 51, 53, 53, 56, 57, 59, 60, 60, + 62, 63, 64, 65, 66, 67, 30, 31, 31, 32, 34, 35, 40, 41, 42, 45, 46, 45, + 45, 45, 46, 46, 47, 48, 49, 51, 52, 54, 55, 57, 58, 58, 60, 61, 62, 62, + 63, 64, 33, 34, 34, 34, 37, 38, 42, 43, 44, 46, 47, 46, 46, 45, 46, 46, + 47, 48, 49, 51, 51, 53, 55, 56, 57, 57, 59, 60, 61, 62, 63, 64, 33, 34, + 35, 35, 38, 39, 43, 44, 45, 47, 47, 46, 46, 45, 46, 46, 47, 48, 49, 51, + 51, 53, 54, 56, 57, 57, 59, 60, 60, 61, 62, 62, 36, 38, 39, 40, 42, 43, + 47, 47, 47, 47, 48, 46, 46, 45, 46, 46, 47, 47, 48, 49, 50, 52, 53, 54, + 55, 55, 57, 58, 59, 60, 61, 62, 38, 40, 40, 41, 43, 44, 47, 47, 48, 48, + 49, 48, 47, 47, 47, 47, 48, 49, 49, 51, 51, 53, 54, 55, 56, 56, 58, 58, + 58, 59, 60, 60, 41, 42, 42, 42, 44, 45, 47, 48, 48, 50, 50, 49, 49, 49, + 50, 50, 50, 51, 52, 53, 53, 55, 56, 57, 58, 58, 60, 61, 61, 61, 60, 60, + 47, 46, 46, 45, 46, 47, 47, 48, 50, 52, 52, 52, 52, 52, 53, 53, 53, 54, + 55, 55, 56, 58, 58, 60, 60, 61, 62, 61, 61, 62, 63, 64, 49, 47, 47, 46, + 47, 47, 48, 49, 50, 52, 53, 53, 53, 53, 54, 54, 54, 55, 55, 56, 56, 58, + 59, 60, 61, 61, 63, 63, 64, 64, 64, 64, 48, 47, 46, 45, 46, 46, 46, 48, + 49, 52, 53, 54, 55, 55, 56, 56, 57, 58, 58, 59, 60, 61, 62, 63, 64, 64, + 66, 65, 65, 65, 66, 67, 49, 47, 46, 45, 46, 46, 46, 47, 49, 52, 53, 55, + 55, 57, 57, 58, 59, 59, 60, 61, 61, 63, 64, 65, 66, 66, 67, 67, 67, 68, + 67, 67, 49, 47, 46, 45, 45, 45, 45, 47, 49, 52, 53, 55, 57, 58, 59, 60, + 61, 62, 62, 63, 63, 65, 66, 67, 68, 68, 69, 70, 69, 68, 69, 70, 50, 48, + 47, 46, 46, 46, 46, 47, 50, 53, 54, 56, 57, 59, 61, 61, 63, 64, 64, 66, + 66, 68, 69, 70, 70, 71, 72, 70, 71, 72, 71, 70, 50, 48, 47, 46, 46, 46, + 46, 47, 50, 53, 54, 56, 58, 60, 61, 61, 63, 64, 65, 66, 67, 68, 69, 71, + 71, 71, 73, 74, 73, 72, 73, 74, 52, 50, 49, 47, 47, 47, 47, 48, 50, 53, + 54, 57, 59, 61, 63, 63, 66, 67, 68, 70, 70, 72, 73, 75, 75, 75, 77, 75, + 75, 76, 75, 74, 53, 50, 50, 48, 48, 48, 47, 49, 51, 54, 55, 58, 59, 62, + 64, 64, 67, 68, 69, 71, 71, 73, 74, 76, 77, 77, 78, 78, 78, 76, 77, 78, + 54, 52, 51, 49, 49, 49, 48, 49, 52, 55, 55, 58, 60, 62, 64, 65, 68, 69, + 71, 73, 73, 75, 76, 78, 79, 79, 80, 80, 79, 80, 79, 78, 56, 54, 53, 51, + 51, 51, 49, 51, 53, 55, 56, 59, 61, 63, 66, 66, 70, 71, 73, 75, 76, 78, + 79, 81, 82, 82, 83, 81, 83, 81, 81, 82, 57, 54, 53, 52, 51, 51, 50, 51, + 53, 56, 56, 60, 61, 63, 66, 67, 70, 71, 73, 76, 76, 79, 80, 82, 82, 83, + 84, 85, 83, 84, 84, 82, 60, 57, 56, 54, 53, 53, 52, 53, 55, 58, 58, 61, + 63, 65, 68, 68, 72, 73, 75, 78, 79, 82, 83, 85, 86, 86, 88, 86, 87, 86, + 85, 86, 61, 58, 57, 55, 55, 54, 53, 54, 56, 58, 59, 62, 64, 66, 69, 69, + 73, 74, 76, 79, 80, 83, 84, 86, 87, 88, 89, 89, 88, 88, 88, 86, 63, 60, + 59, 57, 56, 56, 54, 55, 57, 60, 60, 63, 65, 67, 70, 71, 75, 76, 78, 81, + 82, 85, 86, 89, 90, 90, 92, 91, 91, 90, 89, 91, 64, 61, 60, 58, 57, 57, + 55, 56, 58, 60, 61, 64, 66, 68, 70, 71, 75, 77, 79, 82, 82, 86, 87, 90, + 91, 91, 93, 93, 93, 92, 93, 91, 65, 61, 60, 58, 57, 57, 55, 56, 58, 61, + 61, 64, 66, 68, 71, 71, 75, 77, 79, 82, 83, 86, 88, 90, 91, 91, 93, 94, + 95, 95, 93, 95, 67, 63, 62, 60, 59, 59, 57, 58, 60, 62, 63, 66, 67, 69, + 72, 73, 77, 78, 80, 83, 84, 88, 89, 92, 93, 93, 95, 95, 96, 96, 97, 95, + 67, 64, 63, 61, 60, 60, 58, 58, 61, 61, 63, 65, 67, 70, 70, 74, 75, 78, + 80, 81, 85, 86, 89, 91, 93, 94, 95, 97, 97, 98, 98, 100, 68, 65, 64, 62, + 61, 60, 59, 58, 61, 61, 64, 65, 67, 69, 71, 73, 75, 78, 79, 83, 83, 87, + 88, 91, 93, 95, 96, 97, 99, 98, 100, 100, 69, 65, 65, 62, 62, 61, 60, + 59, 61, 62, 64, 65, 68, 68, 72, 72, 76, 76, 80, 81, 84, 86, 88, 90, 92, + 95, 96, 98, 98, 100, 100, 101, 70, 66, 66, 63, 63, 62, 61, 60, 60, 63, + 64, 66, 67, 69, 71, 73, 75, 77, 79, 81, 84, 85, 88, 89, 93, 93, 97, 98, + 100, 100, 102, 101, 71, 67, 67, 64, 64, 62, 62, 60, 60, 64, 64, 67, 67, + 70, 70, 74, 74, 78, 78, 82, 82, 86, 86, 91, 91, 95, 95, 100, 100, 101, + 101, 104], + /* Size 4x8 */ + [31, 47, 53, 63, 36, 47, 50, 59, 46, 52, 55, 61, 45, 53, 63, 70, 49, 55, + 71, 77, 54, 58, 77, 86, 59, 61, 81, 94, 63, 65, 80, 95], + /* Size 8x4 */ + [31, 36, 46, 45, 49, 54, 59, 63, 47, 47, 52, 53, 55, 58, 61, 65, 53, 50, + 55, 63, 71, 77, 81, 80, 63, 59, 61, 70, 77, 86, 94, 95], + /* Size 8x16 */ + [32, 33, 45, 49, 52, 57, 64, 68, 31, 34, 45, 46, 49, 53, 60, 64, 33, 37, + 46, 45, 47, 51, 57, 61, 37, 43, 47, 45, 47, 50, 55, 59, 42, 44, 49, 49, + 50, 53, 58, 60, 49, 47, 52, 53, 54, 57, 61, 63, 48, 46, 51, 57, 59, 61, + 66, 67, 50, 46, 52, 59, 63, 66, 71, 71, 52, 47, 53, 61, 66, 71, 75, 74, + 54, 49, 54, 62, 68, 73, 79, 79, 57, 51, 55, 64, 70, 76, 83, 83, 61, 55, + 58, 66, 73, 80, 87, 87, 64, 57, 60, 68, 75, 83, 91, 91, 66, 59, 61, 69, + 77, 84, 93, 95, 68, 61, 61, 68, 77, 86, 94, 97, 70, 63, 61, 67, 75, 83, + 92, 98], + /* Size 16x8 */ + [32, 31, 33, 37, 42, 49, 48, 50, 52, 54, 57, 61, 64, 66, 68, 70, 33, 34, + 37, 43, 44, 47, 46, 46, 47, 49, 51, 55, 57, 59, 61, 63, 45, 45, 46, 47, + 49, 52, 51, 52, 53, 54, 55, 58, 60, 61, 61, 61, 49, 46, 45, 45, 49, 53, + 57, 59, 61, 62, 64, 66, 68, 69, 68, 67, 52, 49, 47, 47, 50, 54, 59, 63, + 66, 68, 70, 73, 75, 77, 77, 75, 57, 53, 51, 50, 53, 57, 61, 66, 71, 73, + 76, 80, 83, 84, 86, 83, 64, 60, 57, 55, 58, 61, 66, 71, 75, 79, 83, 87, + 91, 93, 94, 92, 68, 64, 61, 59, 60, 63, 67, 71, 74, 79, 83, 87, 91, 95, + 97, 98], + /* Size 16x32 */ + [32, 31, 33, 37, 45, 48, 49, 50, 52, 56, 57, 63, 64, 67, 68, 68, 31, 31, + 34, 38, 45, 47, 47, 48, 50, 53, 54, 60, 61, 63, 64, 65, 31, 32, 34, 39, + 45, 46, 46, 47, 49, 52, 53, 59, 60, 62, 64, 65, 30, 32, 35, 40, 44, 46, + 45, 46, 48, 51, 52, 57, 58, 60, 61, 62, 33, 35, 37, 42, 46, 47, 45, 46, + 47, 50, 51, 56, 57, 60, 61, 62, 33, 36, 38, 43, 46, 47, 46, 46, 47, 50, + 51, 56, 57, 59, 60, 60, 37, 40, 43, 47, 47, 47, 45, 46, 47, 49, 50, 54, + 55, 57, 59, 61, 39, 41, 43, 47, 48, 48, 47, 47, 48, 50, 51, 55, 56, 57, + 58, 59, 42, 43, 44, 47, 49, 50, 49, 50, 50, 53, 53, 57, 58, 60, 60, 59, + 47, 46, 46, 48, 51, 52, 53, 53, 53, 55, 56, 60, 61, 61, 61, 62, 49, 46, + 47, 48, 52, 53, 53, 54, 54, 56, 57, 60, 61, 63, 63, 62, 48, 46, 46, 47, + 51, 53, 56, 56, 57, 59, 60, 64, 64, 65, 64, 65, 48, 45, 46, 46, 51, 53, + 57, 57, 59, 61, 61, 65, 66, 66, 67, 65, 49, 45, 45, 46, 51, 53, 58, 59, + 61, 63, 64, 67, 68, 70, 67, 68, 50, 46, 46, 46, 52, 54, 59, 61, 63, 65, + 66, 70, 71, 70, 71, 68, 50, 46, 46, 46, 52, 54, 59, 61, 64, 66, 67, 71, + 71, 73, 71, 72, 52, 48, 47, 47, 53, 54, 61, 63, 66, 70, 71, 75, 75, 75, + 74, 72, 53, 49, 48, 48, 53, 55, 61, 64, 67, 71, 72, 76, 77, 77, 75, 76, + 54, 50, 49, 49, 54, 55, 62, 65, 68, 72, 73, 78, 79, 80, 79, 76, 56, 51, + 51, 50, 55, 56, 63, 66, 70, 74, 76, 81, 82, 81, 80, 80, 57, 52, 51, 50, + 55, 56, 64, 66, 70, 75, 76, 82, 83, 85, 83, 80, 60, 54, 54, 52, 57, 58, + 65, 68, 72, 77, 79, 85, 86, 86, 85, 84, 61, 56, 55, 53, 58, 59, 66, 69, + 73, 79, 80, 86, 87, 89, 87, 84, 63, 57, 56, 55, 59, 60, 67, 70, 75, 80, + 82, 89, 90, 91, 89, 89, 64, 58, 57, 56, 60, 61, 68, 71, 75, 81, 83, 90, + 91, 93, 91, 89, 64, 59, 58, 56, 60, 61, 68, 71, 75, 81, 83, 90, 91, 94, + 94, 93, 66, 60, 59, 57, 61, 63, 69, 72, 77, 82, 84, 92, 93, 94, 95, 93, + 67, 61, 60, 58, 61, 63, 69, 70, 78, 80, 85, 90, 93, 96, 97, 97, 68, 62, + 61, 59, 61, 64, 68, 71, 77, 79, 86, 88, 94, 96, 97, 98, 69, 63, 62, 59, + 61, 65, 68, 72, 76, 80, 85, 88, 94, 95, 99, 99, 70, 63, 63, 60, 61, 66, + 67, 73, 75, 81, 83, 89, 92, 97, 98, 99, 70, 64, 64, 61, 61, 67, 67, 74, + 74, 82, 82, 90, 90, 98, 98, 102], + /* Size 32x16 */ + [32, 31, 31, 30, 33, 33, 37, 39, 42, 47, 49, 48, 48, 49, 50, 50, 52, 53, + 54, 56, 57, 60, 61, 63, 64, 64, 66, 67, 68, 69, 70, 70, 31, 31, 32, 32, + 35, 36, 40, 41, 43, 46, 46, 46, 45, 45, 46, 46, 48, 49, 50, 51, 52, 54, + 56, 57, 58, 59, 60, 61, 62, 63, 63, 64, 33, 34, 34, 35, 37, 38, 43, 43, + 44, 46, 47, 46, 46, 45, 46, 46, 47, 48, 49, 51, 51, 54, 55, 56, 57, 58, + 59, 60, 61, 62, 63, 64, 37, 38, 39, 40, 42, 43, 47, 47, 47, 48, 48, 47, + 46, 46, 46, 46, 47, 48, 49, 50, 50, 52, 53, 55, 56, 56, 57, 58, 59, 59, + 60, 61, 45, 45, 45, 44, 46, 46, 47, 48, 49, 51, 52, 51, 51, 51, 52, 52, + 53, 53, 54, 55, 55, 57, 58, 59, 60, 60, 61, 61, 61, 61, 61, 61, 48, 47, + 46, 46, 47, 47, 47, 48, 50, 52, 53, 53, 53, 53, 54, 54, 54, 55, 55, 56, + 56, 58, 59, 60, 61, 61, 63, 63, 64, 65, 66, 67, 49, 47, 46, 45, 45, 46, + 45, 47, 49, 53, 53, 56, 57, 58, 59, 59, 61, 61, 62, 63, 64, 65, 66, 67, + 68, 68, 69, 69, 68, 68, 67, 67, 50, 48, 47, 46, 46, 46, 46, 47, 50, 53, + 54, 56, 57, 59, 61, 61, 63, 64, 65, 66, 66, 68, 69, 70, 71, 71, 72, 70, + 71, 72, 73, 74, 52, 50, 49, 48, 47, 47, 47, 48, 50, 53, 54, 57, 59, 61, + 63, 64, 66, 67, 68, 70, 70, 72, 73, 75, 75, 75, 77, 78, 77, 76, 75, 74, + 56, 53, 52, 51, 50, 50, 49, 50, 53, 55, 56, 59, 61, 63, 65, 66, 70, 71, + 72, 74, 75, 77, 79, 80, 81, 81, 82, 80, 79, 80, 81, 82, 57, 54, 53, 52, + 51, 51, 50, 51, 53, 56, 57, 60, 61, 64, 66, 67, 71, 72, 73, 76, 76, 79, + 80, 82, 83, 83, 84, 85, 86, 85, 83, 82, 63, 60, 59, 57, 56, 56, 54, 55, + 57, 60, 60, 64, 65, 67, 70, 71, 75, 76, 78, 81, 82, 85, 86, 89, 90, 90, + 92, 90, 88, 88, 89, 90, 64, 61, 60, 58, 57, 57, 55, 56, 58, 61, 61, 64, + 66, 68, 71, 71, 75, 77, 79, 82, 83, 86, 87, 90, 91, 91, 93, 93, 94, 94, + 92, 90, 67, 63, 62, 60, 60, 59, 57, 57, 60, 61, 63, 65, 66, 70, 70, 73, + 75, 77, 80, 81, 85, 86, 89, 91, 93, 94, 94, 96, 96, 95, 97, 98, 68, 64, + 64, 61, 61, 60, 59, 58, 60, 61, 63, 64, 67, 67, 71, 71, 74, 75, 79, 80, + 83, 85, 87, 89, 91, 94, 95, 97, 97, 99, 98, 98, 68, 65, 65, 62, 62, 60, + 61, 59, 59, 62, 62, 65, 65, 68, 68, 72, 72, 76, 76, 80, 80, 84, 84, 89, + 89, 93, 93, 97, 98, 99, 99, 102], + /* Size 4x16 */ + [31, 48, 56, 67, 32, 46, 52, 62, 35, 47, 50, 60, 40, 47, 49, 57, 43, 50, + 53, 60, 46, 53, 56, 63, 45, 53, 61, 66, 46, 54, 65, 70, 48, 54, 70, 75, + 50, 55, 72, 80, 52, 56, 75, 85, 56, 59, 79, 89, 58, 61, 81, 93, 60, 63, + 82, 94, 62, 64, 79, 96, 63, 66, 81, 97], + /* Size 16x4 */ + [31, 32, 35, 40, 43, 46, 45, 46, 48, 50, 52, 56, 58, 60, 62, 63, 48, 46, + 47, 47, 50, 53, 53, 54, 54, 55, 56, 59, 61, 63, 64, 66, 56, 52, 50, 49, + 53, 56, 61, 65, 70, 72, 75, 79, 81, 82, 79, 81, 67, 62, 60, 57, 60, 63, + 66, 70, 75, 80, 85, 89, 93, 94, 96, 97], + /* Size 8x32 */ + [32, 33, 45, 49, 52, 57, 64, 68, 31, 34, 45, 47, 50, 54, 61, 64, 31, 34, + 45, 46, 49, 53, 60, 64, 30, 35, 44, 45, 48, 52, 58, 61, 33, 37, 46, 45, + 47, 51, 57, 61, 33, 38, 46, 46, 47, 51, 57, 60, 37, 43, 47, 45, 47, 50, + 55, 59, 39, 43, 48, 47, 48, 51, 56, 58, 42, 44, 49, 49, 50, 53, 58, 60, + 47, 46, 51, 53, 53, 56, 61, 61, 49, 47, 52, 53, 54, 57, 61, 63, 48, 46, + 51, 56, 57, 60, 64, 64, 48, 46, 51, 57, 59, 61, 66, 67, 49, 45, 51, 58, + 61, 64, 68, 67, 50, 46, 52, 59, 63, 66, 71, 71, 50, 46, 52, 59, 64, 67, + 71, 71, 52, 47, 53, 61, 66, 71, 75, 74, 53, 48, 53, 61, 67, 72, 77, 75, + 54, 49, 54, 62, 68, 73, 79, 79, 56, 51, 55, 63, 70, 76, 82, 80, 57, 51, + 55, 64, 70, 76, 83, 83, 60, 54, 57, 65, 72, 79, 86, 85, 61, 55, 58, 66, + 73, 80, 87, 87, 63, 56, 59, 67, 75, 82, 90, 89, 64, 57, 60, 68, 75, 83, + 91, 91, 64, 58, 60, 68, 75, 83, 91, 94, 66, 59, 61, 69, 77, 84, 93, 95, + 67, 60, 61, 69, 78, 85, 93, 97, 68, 61, 61, 68, 77, 86, 94, 97, 69, 62, + 61, 68, 76, 85, 94, 99, 70, 63, 61, 67, 75, 83, 92, 98, 70, 64, 61, 67, + 74, 82, 90, 98], + /* Size 32x8 */ + [32, 31, 31, 30, 33, 33, 37, 39, 42, 47, 49, 48, 48, 49, 50, 50, 52, 53, + 54, 56, 57, 60, 61, 63, 64, 64, 66, 67, 68, 69, 70, 70, 33, 34, 34, 35, + 37, 38, 43, 43, 44, 46, 47, 46, 46, 45, 46, 46, 47, 48, 49, 51, 51, 54, + 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 45, 45, 45, 44, 46, 46, 47, 48, + 49, 51, 52, 51, 51, 51, 52, 52, 53, 53, 54, 55, 55, 57, 58, 59, 60, 60, + 61, 61, 61, 61, 61, 61, 49, 47, 46, 45, 45, 46, 45, 47, 49, 53, 53, 56, + 57, 58, 59, 59, 61, 61, 62, 63, 64, 65, 66, 67, 68, 68, 69, 69, 68, 68, + 67, 67, 52, 50, 49, 48, 47, 47, 47, 48, 50, 53, 54, 57, 59, 61, 63, 64, + 66, 67, 68, 70, 70, 72, 73, 75, 75, 75, 77, 78, 77, 76, 75, 74, 57, 54, + 53, 52, 51, 51, 50, 51, 53, 56, 57, 60, 61, 64, 66, 67, 71, 72, 73, 76, + 76, 79, 80, 82, 83, 83, 84, 85, 86, 85, 83, 82, 64, 61, 60, 58, 57, 57, + 55, 56, 58, 61, 61, 64, 66, 68, 71, 71, 75, 77, 79, 82, 83, 86, 87, 90, + 91, 91, 93, 93, 94, 94, 92, 90, 68, 64, 64, 61, 61, 60, 59, 58, 60, 61, + 63, 64, 67, 67, 71, 71, 74, 75, 79, 80, 83, 85, 87, 89, 91, 94, 95, 97, + 97, 99, 98, 98] + ] + ], + [ + [ /* Luma */ + /* Size 4x4 */ + [32, 34, 53, 75, 34, 49, 64, 81, 53, 64, 91, 112, 75, 81, 112, 140], + /* Size 8x8 */ + [32, 32, 34, 39, 50, 62, 76, 84, 32, 33, 35, 40, 48, 59, 71, 79, 34, 35, + 39, 46, 53, 63, 74, 81, 39, 40, 46, 56, 65, 75, 86, 92, 50, 48, 53, 65, + 78, 90, 101, 106, 62, 59, 63, 75, 90, 105, 118, 123, 76, 71, 74, 86, + 101, 118, 134, 142, 84, 79, 81, 92, 106, 123, 142, 153], + /* Size 16x16 */ + [32, 31, 31, 32, 33, 36, 39, 44, 48, 54, 59, 66, 74, 81, 86, 91, 31, 32, + 32, 32, 33, 35, 38, 42, 46, 51, 56, 63, 70, 77, 81, 86, 31, 32, 32, 33, + 34, 35, 38, 41, 45, 49, 54, 60, 67, 73, 77, 82, 32, 32, 33, 34, 36, 37, + 40, 42, 45, 49, 53, 59, 66, 71, 75, 80, 33, 33, 34, 36, 38, 42, 44, 46, + 50, 53, 57, 63, 69, 74, 78, 80, 36, 35, 35, 37, 42, 48, 50, 54, 57, 60, + 64, 69, 75, 80, 84, 85, 39, 38, 38, 40, 44, 50, 54, 58, 61, 65, 69, 74, + 80, 85, 89, 91, 44, 42, 41, 42, 46, 54, 58, 63, 67, 71, 75, 80, 86, 91, + 95, 97, 48, 46, 45, 45, 50, 57, 61, 67, 71, 76, 80, 86, 93, 98, 101, + 104, 54, 51, 49, 49, 53, 60, 65, 71, 76, 82, 87, 93, 100, 105, 109, 112, + 59, 56, 54, 53, 57, 64, 69, 75, 80, 87, 92, 99, 106, 112, 116, 120, 66, + 63, 60, 59, 63, 69, 74, 80, 86, 93, 99, 107, 115, 121, 125, 129, 74, 70, + 67, 66, 69, 75, 80, 86, 93, 100, 106, 115, 123, 130, 135, 138, 81, 77, + 73, 71, 74, 80, 85, 91, 98, 105, 112, 121, 130, 137, 142, 148, 86, 81, + 77, 75, 78, 84, 89, 95, 101, 109, 116, 125, 135, 142, 147, 153, 91, 86, + 82, 80, 80, 85, 91, 97, 104, 112, 120, 129, 138, 148, 153, 159], + /* Size 32x32 */ + [32, 31, 31, 31, 31, 31, 32, 32, 33, 34, 36, 36, 39, 41, 44, 46, 48, 52, + 54, 58, 59, 65, 66, 71, 74, 80, 81, 83, 86, 89, 91, 93, 31, 32, 32, 32, + 32, 32, 32, 32, 33, 34, 35, 35, 38, 39, 42, 44, 46, 50, 51, 56, 56, 62, + 63, 68, 71, 76, 77, 78, 82, 84, 86, 88, 31, 32, 32, 32, 32, 32, 32, 32, + 33, 34, 35, 35, 38, 39, 42, 44, 46, 49, 51, 55, 56, 61, 63, 67, 70, 75, + 77, 78, 81, 84, 86, 88, 31, 32, 32, 32, 32, 32, 32, 32, 33, 33, 34, 34, + 37, 38, 41, 42, 44, 48, 49, 53, 54, 59, 60, 65, 68, 72, 74, 75, 78, 80, + 82, 84, 31, 32, 32, 32, 32, 33, 33, 33, 34, 34, 35, 35, 38, 39, 41, 43, + 45, 48, 49, 53, 54, 59, 60, 65, 67, 72, 73, 74, 77, 80, 82, 84, 31, 32, + 32, 32, 33, 33, 33, 34, 35, 35, 36, 36, 39, 40, 42, 44, 45, 48, 50, 53, + 54, 59, 60, 64, 67, 71, 73, 74, 77, 79, 81, 83, 32, 32, 32, 32, 33, 33, + 34, 35, 36, 36, 37, 38, 40, 40, 42, 44, 45, 48, 49, 53, 53, 58, 59, 63, + 66, 70, 71, 72, 75, 78, 80, 83, 32, 32, 32, 32, 33, 34, 35, 35, 36, 37, + 38, 38, 40, 41, 42, 44, 46, 48, 49, 53, 53, 58, 59, 63, 65, 69, 71, 72, + 74, 77, 79, 80, 33, 33, 33, 33, 34, 35, 36, 36, 38, 39, 42, 42, 44, 45, + 46, 48, 50, 52, 53, 57, 57, 62, 63, 67, 69, 73, 74, 75, 78, 79, 80, 81, + 34, 34, 34, 33, 34, 35, 36, 37, 39, 39, 42, 43, 45, 46, 47, 49, 51, 53, + 54, 58, 58, 63, 64, 68, 70, 74, 75, 76, 79, 81, 84, 86, 36, 35, 35, 34, + 35, 36, 37, 38, 42, 42, 48, 48, 50, 51, 54, 55, 57, 59, 60, 63, 64, 68, + 69, 73, 75, 79, 80, 81, 84, 85, 85, 86, 36, 35, 35, 34, 35, 36, 38, 38, + 42, 43, 48, 49, 51, 52, 54, 55, 57, 59, 60, 64, 64, 68, 69, 73, 75, 79, + 80, 81, 84, 86, 88, 91, 39, 38, 38, 37, 38, 39, 40, 40, 44, 45, 50, 51, + 54, 55, 58, 59, 61, 64, 65, 68, 69, 73, 74, 78, 80, 84, 85, 86, 89, 91, + 91, 91, 41, 39, 39, 38, 39, 40, 40, 41, 45, 46, 51, 52, 55, 56, 59, 61, + 63, 65, 67, 70, 70, 75, 76, 80, 82, 86, 87, 88, 91, 92, 94, 96, 44, 42, + 42, 41, 41, 42, 42, 42, 46, 47, 54, 54, 58, 59, 63, 65, 67, 70, 71, 75, + 75, 79, 80, 84, 86, 90, 91, 92, 95, 97, 97, 97, 46, 44, 44, 42, 43, 44, + 44, 44, 48, 49, 55, 55, 59, 61, 65, 67, 69, 72, 74, 77, 78, 82, 83, 87, + 89, 93, 94, 95, 98, 98, 100, 103, 48, 46, 46, 44, 45, 45, 45, 46, 50, + 51, 57, 57, 61, 63, 67, 69, 71, 74, 76, 80, 80, 85, 86, 90, 93, 96, 98, + 99, 101, 104, 104, 103, 52, 50, 49, 48, 48, 48, 48, 48, 52, 53, 59, 59, + 64, 65, 70, 72, 74, 78, 80, 84, 85, 90, 91, 95, 97, 101, 103, 104, 106, + 106, 107, 110, 54, 51, 51, 49, 49, 50, 49, 49, 53, 54, 60, 60, 65, 67, + 71, 74, 76, 80, 82, 86, 87, 92, 93, 97, 100, 104, 105, 106, 109, 112, + 112, 110, 58, 56, 55, 53, 53, 53, 53, 53, 57, 58, 63, 64, 68, 70, 75, + 77, 80, 84, 86, 91, 91, 97, 98, 103, 105, 110, 111, 112, 115, 114, 115, + 118, 59, 56, 56, 54, 54, 54, 53, 53, 57, 58, 64, 64, 69, 70, 75, 78, 80, + 85, 87, 91, 92, 98, 99, 103, 106, 110, 112, 113, 116, 119, 120, 119, 65, + 62, 61, 59, 59, 59, 58, 58, 62, 63, 68, 68, 73, 75, 79, 82, 85, 90, 92, + 97, 98, 105, 106, 111, 114, 118, 120, 121, 124, 123, 123, 126, 66, 63, + 63, 60, 60, 60, 59, 59, 63, 64, 69, 69, 74, 76, 80, 83, 86, 91, 93, 98, + 99, 106, 107, 112, 115, 119, 121, 122, 125, 128, 129, 126, 71, 68, 67, + 65, 65, 64, 63, 63, 67, 68, 73, 73, 78, 80, 84, 87, 90, 95, 97, 103, + 103, 111, 112, 117, 120, 125, 127, 128, 131, 132, 132, 135, 74, 71, 70, + 68, 67, 67, 66, 65, 69, 70, 75, 75, 80, 82, 86, 89, 93, 97, 100, 105, + 106, 114, 115, 120, 123, 128, 130, 131, 135, 135, 138, 136, 80, 76, 75, + 72, 72, 71, 70, 69, 73, 74, 79, 79, 84, 86, 90, 93, 96, 101, 104, 110, + 110, 118, 119, 125, 128, 134, 136, 137, 140, 142, 140, 144, 81, 77, 77, + 74, 73, 73, 71, 71, 74, 75, 80, 80, 85, 87, 91, 94, 98, 103, 105, 111, + 112, 120, 121, 127, 130, 136, 137, 139, 142, 145, 148, 144, 83, 78, 78, + 75, 74, 74, 72, 72, 75, 76, 81, 81, 86, 88, 92, 95, 99, 104, 106, 112, + 113, 121, 122, 128, 131, 137, 139, 140, 144, 148, 150, 155, 86, 82, 81, + 78, 77, 77, 75, 74, 78, 79, 84, 84, 89, 91, 95, 98, 101, 106, 109, 115, + 116, 124, 125, 131, 135, 140, 142, 144, 147, 149, 153, 155, 89, 84, 84, + 80, 80, 79, 78, 77, 79, 81, 85, 86, 91, 92, 97, 98, 104, 106, 112, 114, + 119, 123, 128, 132, 135, 142, 145, 148, 149, 153, 154, 159, 91, 86, 86, + 82, 82, 81, 80, 79, 80, 84, 85, 88, 91, 94, 97, 100, 104, 107, 112, 115, + 120, 123, 129, 132, 138, 140, 148, 150, 153, 154, 159, 159, 93, 88, 88, + 84, 84, 83, 83, 80, 81, 86, 86, 91, 91, 96, 97, 103, 103, 110, 110, 118, + 119, 126, 126, 135, 136, 144, 144, 155, 155, 159, 159, 164], + /* Size 4x8 */ + [32, 35, 51, 77, 32, 36, 50, 72, 34, 42, 54, 75, 38, 51, 67, 87, 48, 59, + 80, 103, 60, 68, 92, 119, 72, 79, 104, 135, 81, 86, 112, 144], + /* Size 8x4 */ + [32, 32, 34, 38, 48, 60, 72, 81, 35, 36, 42, 51, 59, 68, 79, 86, 51, 50, + 54, 67, 80, 92, 104, 112, 77, 72, 75, 87, 103, 119, 135, 144], + /* Size 8x16 */ + [32, 31, 33, 40, 51, 65, 79, 87, 31, 32, 33, 39, 49, 61, 74, 82, 31, 32, + 34, 38, 47, 59, 71, 79, 32, 33, 36, 40, 48, 58, 69, 77, 33, 34, 38, 44, + 52, 62, 72, 78, 36, 35, 42, 51, 58, 68, 78, 84, 39, 38, 44, 54, 63, 73, + 84, 89, 44, 41, 46, 59, 69, 79, 90, 96, 48, 45, 50, 62, 74, 85, 96, 103, + 53, 49, 53, 66, 79, 92, 103, 111, 58, 54, 57, 70, 84, 98, 110, 118, 66, + 60, 63, 75, 90, 106, 119, 126, 74, 67, 69, 81, 97, 113, 128, 134, 81, + 73, 75, 86, 102, 120, 135, 143, 86, 78, 78, 90, 106, 124, 140, 147, 91, + 82, 80, 90, 103, 119, 137, 151], + /* Size 16x8 */ + [32, 31, 31, 32, 33, 36, 39, 44, 48, 53, 58, 66, 74, 81, 86, 91, 31, 32, + 32, 33, 34, 35, 38, 41, 45, 49, 54, 60, 67, 73, 78, 82, 33, 33, 34, 36, + 38, 42, 44, 46, 50, 53, 57, 63, 69, 75, 78, 80, 40, 39, 38, 40, 44, 51, + 54, 59, 62, 66, 70, 75, 81, 86, 90, 90, 51, 49, 47, 48, 52, 58, 63, 69, + 74, 79, 84, 90, 97, 102, 106, 103, 65, 61, 59, 58, 62, 68, 73, 79, 85, + 92, 98, 106, 113, 120, 124, 119, 79, 74, 71, 69, 72, 78, 84, 90, 96, + 103, 110, 119, 128, 135, 140, 137, 87, 82, 79, 77, 78, 84, 89, 96, 103, + 111, 118, 126, 134, 143, 147, 151], + /* Size 16x32 */ + [32, 31, 31, 32, 33, 36, 40, 44, 51, 53, 65, 66, 79, 81, 87, 90, 31, 32, + 32, 32, 33, 35, 39, 42, 49, 51, 62, 63, 75, 77, 83, 85, 31, 32, 32, 32, + 33, 35, 39, 42, 49, 51, 61, 62, 74, 76, 82, 85, 31, 32, 32, 33, 33, 34, + 38, 41, 47, 49, 59, 60, 72, 74, 79, 81, 31, 32, 32, 33, 34, 35, 38, 41, + 47, 49, 59, 60, 71, 73, 79, 81, 32, 32, 33, 34, 35, 36, 39, 42, 48, 50, + 59, 60, 71, 72, 78, 80, 32, 32, 33, 35, 36, 37, 40, 42, 48, 49, 58, 59, + 69, 71, 77, 80, 32, 33, 33, 35, 36, 38, 41, 42, 48, 49, 58, 59, 69, 70, + 75, 77, 33, 33, 34, 36, 38, 41, 44, 46, 52, 53, 62, 63, 72, 74, 78, 78, + 34, 34, 34, 37, 39, 42, 45, 48, 53, 54, 63, 64, 73, 75, 80, 83, 36, 34, + 35, 38, 42, 48, 51, 54, 58, 60, 68, 69, 78, 80, 84, 83, 36, 35, 35, 38, + 42, 48, 51, 54, 59, 60, 68, 69, 79, 80, 85, 87, 39, 37, 38, 40, 44, 50, + 54, 58, 63, 65, 73, 74, 84, 85, 89, 88, 40, 38, 39, 41, 45, 51, 56, 59, + 65, 67, 75, 76, 85, 87, 90, 93, 44, 41, 41, 43, 46, 53, 59, 63, 69, 71, + 79, 80, 90, 91, 96, 93, 46, 43, 43, 44, 48, 55, 60, 65, 72, 73, 82, 83, + 93, 94, 97, 100, 48, 45, 45, 46, 50, 56, 62, 67, 74, 76, 85, 86, 96, 98, + 103, 100, 52, 48, 48, 49, 52, 59, 65, 70, 78, 80, 90, 91, 101, 103, 105, + 107, 53, 49, 49, 50, 53, 60, 66, 71, 79, 82, 92, 93, 103, 105, 111, 107, + 58, 53, 53, 53, 57, 63, 69, 74, 83, 86, 97, 98, 109, 111, 113, 115, 58, + 54, 54, 54, 57, 63, 70, 75, 84, 87, 98, 99, 110, 112, 118, 115, 65, 60, + 59, 58, 62, 68, 74, 79, 89, 92, 105, 106, 118, 119, 122, 123, 66, 61, + 60, 59, 63, 69, 75, 80, 90, 93, 106, 107, 119, 121, 126, 123, 71, 65, + 65, 63, 67, 73, 79, 84, 94, 97, 111, 112, 125, 127, 131, 132, 74, 68, + 67, 66, 69, 75, 81, 86, 97, 100, 113, 115, 128, 130, 134, 132, 79, 72, + 72, 70, 73, 79, 85, 90, 101, 104, 118, 119, 133, 135, 141, 140, 81, 74, + 73, 71, 75, 80, 86, 91, 102, 105, 120, 121, 135, 137, 143, 140, 82, 75, + 74, 72, 75, 81, 87, 92, 103, 106, 121, 122, 136, 139, 147, 151, 86, 78, + 78, 75, 78, 84, 90, 95, 106, 109, 124, 125, 140, 142, 147, 151, 88, 81, + 80, 77, 80, 86, 90, 98, 105, 112, 122, 127, 140, 144, 152, 155, 91, 83, + 82, 79, 80, 88, 90, 100, 103, 114, 119, 130, 137, 148, 151, 155, 93, 85, + 85, 81, 81, 90, 90, 102, 103, 117, 117, 134, 134, 151, 152, 160], + /* Size 32x16 */ + [32, 31, 31, 31, 31, 32, 32, 32, 33, 34, 36, 36, 39, 40, 44, 46, 48, 52, + 53, 58, 58, 65, 66, 71, 74, 79, 81, 82, 86, 88, 91, 93, 31, 32, 32, 32, + 32, 32, 32, 33, 33, 34, 34, 35, 37, 38, 41, 43, 45, 48, 49, 53, 54, 60, + 61, 65, 68, 72, 74, 75, 78, 81, 83, 85, 31, 32, 32, 32, 32, 33, 33, 33, + 34, 34, 35, 35, 38, 39, 41, 43, 45, 48, 49, 53, 54, 59, 60, 65, 67, 72, + 73, 74, 78, 80, 82, 85, 32, 32, 32, 33, 33, 34, 35, 35, 36, 37, 38, 38, + 40, 41, 43, 44, 46, 49, 50, 53, 54, 58, 59, 63, 66, 70, 71, 72, 75, 77, + 79, 81, 33, 33, 33, 33, 34, 35, 36, 36, 38, 39, 42, 42, 44, 45, 46, 48, + 50, 52, 53, 57, 57, 62, 63, 67, 69, 73, 75, 75, 78, 80, 80, 81, 36, 35, + 35, 34, 35, 36, 37, 38, 41, 42, 48, 48, 50, 51, 53, 55, 56, 59, 60, 63, + 63, 68, 69, 73, 75, 79, 80, 81, 84, 86, 88, 90, 40, 39, 39, 38, 38, 39, + 40, 41, 44, 45, 51, 51, 54, 56, 59, 60, 62, 65, 66, 69, 70, 74, 75, 79, + 81, 85, 86, 87, 90, 90, 90, 90, 44, 42, 42, 41, 41, 42, 42, 42, 46, 48, + 54, 54, 58, 59, 63, 65, 67, 70, 71, 74, 75, 79, 80, 84, 86, 90, 91, 92, + 95, 98, 100, 102, 51, 49, 49, 47, 47, 48, 48, 48, 52, 53, 58, 59, 63, + 65, 69, 72, 74, 78, 79, 83, 84, 89, 90, 94, 97, 101, 102, 103, 106, 105, + 103, 103, 53, 51, 51, 49, 49, 50, 49, 49, 53, 54, 60, 60, 65, 67, 71, + 73, 76, 80, 82, 86, 87, 92, 93, 97, 100, 104, 105, 106, 109, 112, 114, + 117, 65, 62, 61, 59, 59, 59, 58, 58, 62, 63, 68, 68, 73, 75, 79, 82, 85, + 90, 92, 97, 98, 105, 106, 111, 113, 118, 120, 121, 124, 122, 119, 117, + 66, 63, 62, 60, 60, 60, 59, 59, 63, 64, 69, 69, 74, 76, 80, 83, 86, 91, + 93, 98, 99, 106, 107, 112, 115, 119, 121, 122, 125, 127, 130, 134, 79, + 75, 74, 72, 71, 71, 69, 69, 72, 73, 78, 79, 84, 85, 90, 93, 96, 101, + 103, 109, 110, 118, 119, 125, 128, 133, 135, 136, 140, 140, 137, 134, + 81, 77, 76, 74, 73, 72, 71, 70, 74, 75, 80, 80, 85, 87, 91, 94, 98, 103, + 105, 111, 112, 119, 121, 127, 130, 135, 137, 139, 142, 144, 148, 151, + 87, 83, 82, 79, 79, 78, 77, 75, 78, 80, 84, 85, 89, 90, 96, 97, 103, + 105, 111, 113, 118, 122, 126, 131, 134, 141, 143, 147, 147, 152, 151, + 152, 90, 85, 85, 81, 81, 80, 80, 77, 78, 83, 83, 87, 88, 93, 93, 100, + 100, 107, 107, 115, 115, 123, 123, 132, 132, 140, 140, 151, 151, 155, + 155, 160], + /* Size 4x16 */ + [31, 36, 53, 81, 32, 35, 51, 76, 32, 35, 49, 73, 32, 37, 49, 71, 33, 41, + 53, 74, 34, 48, 60, 80, 37, 50, 65, 85, 41, 53, 71, 91, 45, 56, 76, 98, + 49, 60, 82, 105, 54, 63, 87, 112, 61, 69, 93, 121, 68, 75, 100, 130, 74, + 80, 105, 137, 78, 84, 109, 142, 83, 88, 114, 148], + /* Size 16x4 */ + [31, 32, 32, 32, 33, 34, 37, 41, 45, 49, 54, 61, 68, 74, 78, 83, 36, 35, + 35, 37, 41, 48, 50, 53, 56, 60, 63, 69, 75, 80, 84, 88, 53, 51, 49, 49, + 53, 60, 65, 71, 76, 82, 87, 93, 100, 105, 109, 114, 81, 76, 73, 71, 74, + 80, 85, 91, 98, 105, 112, 121, 130, 137, 142, 148], + /* Size 8x32 */ + [32, 31, 33, 40, 51, 65, 79, 87, 31, 32, 33, 39, 49, 62, 75, 83, 31, 32, + 33, 39, 49, 61, 74, 82, 31, 32, 33, 38, 47, 59, 72, 79, 31, 32, 34, 38, + 47, 59, 71, 79, 32, 33, 35, 39, 48, 59, 71, 78, 32, 33, 36, 40, 48, 58, + 69, 77, 32, 33, 36, 41, 48, 58, 69, 75, 33, 34, 38, 44, 52, 62, 72, 78, + 34, 34, 39, 45, 53, 63, 73, 80, 36, 35, 42, 51, 58, 68, 78, 84, 36, 35, + 42, 51, 59, 68, 79, 85, 39, 38, 44, 54, 63, 73, 84, 89, 40, 39, 45, 56, + 65, 75, 85, 90, 44, 41, 46, 59, 69, 79, 90, 96, 46, 43, 48, 60, 72, 82, + 93, 97, 48, 45, 50, 62, 74, 85, 96, 103, 52, 48, 52, 65, 78, 90, 101, + 105, 53, 49, 53, 66, 79, 92, 103, 111, 58, 53, 57, 69, 83, 97, 109, 113, + 58, 54, 57, 70, 84, 98, 110, 118, 65, 59, 62, 74, 89, 105, 118, 122, 66, + 60, 63, 75, 90, 106, 119, 126, 71, 65, 67, 79, 94, 111, 125, 131, 74, + 67, 69, 81, 97, 113, 128, 134, 79, 72, 73, 85, 101, 118, 133, 141, 81, + 73, 75, 86, 102, 120, 135, 143, 82, 74, 75, 87, 103, 121, 136, 147, 86, + 78, 78, 90, 106, 124, 140, 147, 88, 80, 80, 90, 105, 122, 140, 152, 91, + 82, 80, 90, 103, 119, 137, 151, 93, 85, 81, 90, 103, 117, 134, 152], + /* Size 32x8 */ + [32, 31, 31, 31, 31, 32, 32, 32, 33, 34, 36, 36, 39, 40, 44, 46, 48, 52, + 53, 58, 58, 65, 66, 71, 74, 79, 81, 82, 86, 88, 91, 93, 31, 32, 32, 32, + 32, 33, 33, 33, 34, 34, 35, 35, 38, 39, 41, 43, 45, 48, 49, 53, 54, 59, + 60, 65, 67, 72, 73, 74, 78, 80, 82, 85, 33, 33, 33, 33, 34, 35, 36, 36, + 38, 39, 42, 42, 44, 45, 46, 48, 50, 52, 53, 57, 57, 62, 63, 67, 69, 73, + 75, 75, 78, 80, 80, 81, 40, 39, 39, 38, 38, 39, 40, 41, 44, 45, 51, 51, + 54, 56, 59, 60, 62, 65, 66, 69, 70, 74, 75, 79, 81, 85, 86, 87, 90, 90, + 90, 90, 51, 49, 49, 47, 47, 48, 48, 48, 52, 53, 58, 59, 63, 65, 69, 72, + 74, 78, 79, 83, 84, 89, 90, 94, 97, 101, 102, 103, 106, 105, 103, 103, + 65, 62, 61, 59, 59, 59, 58, 58, 62, 63, 68, 68, 73, 75, 79, 82, 85, 90, + 92, 97, 98, 105, 106, 111, 113, 118, 120, 121, 124, 122, 119, 117, 79, + 75, 74, 72, 71, 71, 69, 69, 72, 73, 78, 79, 84, 85, 90, 93, 96, 101, + 103, 109, 110, 118, 119, 125, 128, 133, 135, 136, 140, 140, 137, 134, + 87, 83, 82, 79, 79, 78, 77, 75, 78, 80, 84, 85, 89, 90, 96, 97, 103, + 105, 111, 113, 118, 122, 126, 131, 134, 141, 143, 147, 147, 152, 151, + 152] + ], + [ /* Chroma */ + /* Size 4x4 */ + [32, 46, 49, 58, 46, 53, 55, 62, 49, 55, 70, 78, 58, 62, 78, 91], + /* Size 8x8 */ + [31, 34, 42, 47, 49, 54, 60, 64, 34, 39, 45, 46, 47, 51, 56, 59, 42, 45, + 48, 49, 50, 53, 57, 60, 47, 46, 49, 55, 58, 61, 65, 66, 49, 47, 50, 58, + 65, 69, 73, 74, 54, 51, 53, 61, 69, 76, 82, 83, 60, 56, 57, 65, 73, 82, + 89, 92, 64, 59, 60, 66, 74, 83, 92, 96], + /* Size 16x16 */ + [32, 31, 31, 35, 40, 49, 48, 49, 50, 52, 54, 57, 61, 64, 66, 68, 31, 31, + 32, 37, 41, 47, 47, 46, 48, 49, 51, 54, 57, 60, 62, 64, 31, 32, 34, 39, + 43, 46, 46, 45, 46, 47, 49, 52, 55, 57, 59, 61, 35, 37, 39, 44, 46, 47, + 46, 45, 46, 47, 48, 51, 53, 56, 57, 59, 40, 41, 43, 46, 48, 50, 49, 48, + 49, 49, 51, 53, 55, 57, 59, 59, 49, 47, 46, 47, 50, 53, 53, 53, 54, 54, + 55, 57, 59, 61, 62, 62, 48, 47, 46, 46, 49, 53, 54, 55, 56, 57, 58, 60, + 62, 64, 65, 65, 49, 46, 45, 45, 48, 53, 55, 58, 60, 61, 62, 64, 66, 68, + 69, 69, 50, 48, 46, 46, 49, 54, 56, 60, 61, 63, 65, 67, 69, 71, 72, 72, + 52, 49, 47, 47, 49, 54, 57, 61, 63, 66, 68, 71, 73, 75, 76, 77, 54, 51, + 49, 48, 51, 55, 58, 62, 65, 68, 71, 74, 76, 78, 80, 81, 57, 54, 52, 51, + 53, 57, 60, 64, 67, 71, 74, 77, 80, 83, 84, 85, 61, 57, 55, 53, 55, 59, + 62, 66, 69, 73, 76, 80, 84, 87, 89, 89, 64, 60, 57, 56, 57, 61, 64, 68, + 71, 75, 78, 83, 87, 90, 92, 94, 66, 62, 59, 57, 59, 62, 65, 69, 72, 76, + 80, 84, 89, 92, 94, 96, 68, 64, 61, 59, 59, 62, 65, 69, 72, 77, 81, 85, + 89, 94, 96, 98], + /* Size 32x32 */ + [32, 31, 31, 30, 31, 33, 35, 36, 40, 41, 49, 49, 48, 48, 49, 50, 50, 52, + 52, 54, 54, 57, 57, 60, 61, 63, 64, 65, 66, 67, 68, 69, 31, 31, 31, 31, + 32, 34, 37, 38, 41, 42, 47, 47, 47, 47, 47, 47, 48, 49, 50, 52, 52, 54, + 55, 57, 58, 60, 61, 61, 63, 64, 64, 65, 31, 31, 31, 31, 32, 35, 37, 39, + 41, 42, 47, 47, 47, 46, 46, 47, 48, 49, 49, 51, 51, 54, 54, 56, 57, 59, + 60, 61, 62, 63, 64, 65, 30, 31, 31, 32, 33, 35, 38, 40, 42, 42, 46, 46, + 45, 45, 45, 45, 46, 47, 47, 49, 49, 52, 52, 54, 55, 57, 58, 58, 60, 61, + 61, 62, 31, 32, 32, 33, 34, 37, 39, 41, 43, 43, 46, 46, 46, 45, 45, 46, + 46, 47, 47, 49, 49, 51, 52, 54, 55, 57, 57, 58, 59, 60, 61, 62, 33, 34, + 35, 35, 37, 39, 41, 43, 44, 45, 47, 47, 46, 46, 45, 46, 46, 47, 47, 49, + 49, 51, 51, 53, 54, 56, 57, 57, 58, 59, 60, 61, 35, 37, 37, 38, 39, 41, + 44, 46, 46, 46, 47, 47, 46, 46, 45, 46, 46, 47, 47, 48, 48, 50, 51, 52, + 53, 55, 56, 56, 57, 58, 59, 61, 36, 38, 39, 40, 41, 43, 46, 47, 47, 47, + 48, 47, 46, 46, 45, 46, 46, 46, 47, 48, 48, 50, 50, 52, 53, 54, 55, 55, + 56, 57, 58, 58, 40, 41, 41, 42, 43, 44, 46, 47, 48, 48, 50, 49, 49, 49, + 48, 49, 49, 49, 49, 51, 51, 52, 53, 54, 55, 57, 57, 58, 59, 59, 59, 59, + 41, 42, 42, 42, 43, 45, 46, 47, 48, 48, 50, 50, 49, 49, 49, 49, 50, 50, + 50, 52, 52, 53, 53, 55, 56, 57, 58, 58, 59, 60, 61, 62, 49, 47, 47, 46, + 46, 47, 47, 48, 50, 50, 53, 53, 53, 53, 53, 54, 54, 54, 54, 55, 55, 56, + 57, 58, 59, 60, 61, 61, 62, 62, 62, 62, 49, 47, 47, 46, 46, 47, 47, 47, + 49, 50, 53, 53, 53, 53, 54, 54, 54, 54, 54, 55, 56, 57, 57, 59, 59, 61, + 61, 62, 63, 63, 64, 65, 48, 47, 47, 45, 46, 46, 46, 46, 49, 49, 53, 53, + 54, 54, 55, 56, 56, 57, 57, 58, 58, 60, 60, 61, 62, 63, 64, 64, 65, 66, + 65, 65, 48, 47, 46, 45, 45, 46, 46, 46, 49, 49, 53, 53, 54, 55, 56, 57, + 57, 58, 58, 59, 60, 61, 61, 63, 63, 65, 65, 65, 66, 66, 67, 68, 49, 47, + 46, 45, 45, 45, 45, 45, 48, 49, 53, 54, 55, 56, 58, 59, 60, 61, 61, 62, + 62, 63, 64, 65, 66, 67, 68, 68, 69, 70, 69, 68, 50, 47, 47, 45, 46, 46, + 46, 46, 49, 49, 54, 54, 56, 57, 59, 60, 60, 62, 62, 63, 64, 65, 65, 67, + 68, 69, 69, 70, 70, 70, 71, 71, 50, 48, 48, 46, 46, 46, 46, 46, 49, 50, + 54, 54, 56, 57, 60, 60, 61, 63, 63, 65, 65, 67, 67, 68, 69, 71, 71, 71, + 72, 73, 72, 71, 52, 49, 49, 47, 47, 47, 47, 46, 49, 50, 54, 54, 57, 58, + 61, 62, 63, 65, 65, 67, 67, 69, 70, 71, 72, 73, 74, 74, 75, 74, 74, 75, + 52, 50, 49, 47, 47, 47, 47, 47, 49, 50, 54, 54, 57, 58, 61, 62, 63, 65, + 66, 68, 68, 70, 71, 72, 73, 75, 75, 75, 76, 77, 77, 75, 54, 52, 51, 49, + 49, 49, 48, 48, 51, 52, 55, 55, 58, 59, 62, 63, 65, 67, 68, 70, 70, 73, + 73, 75, 76, 78, 78, 78, 79, 78, 78, 79, 54, 52, 51, 49, 49, 49, 48, 48, + 51, 52, 55, 56, 58, 60, 62, 64, 65, 67, 68, 70, 71, 73, 74, 75, 76, 78, + 78, 79, 80, 81, 81, 79, 57, 54, 54, 52, 51, 51, 50, 50, 52, 53, 56, 57, + 60, 61, 63, 65, 67, 69, 70, 73, 73, 76, 77, 79, 80, 82, 82, 83, 84, 83, + 82, 83, 57, 55, 54, 52, 52, 51, 51, 50, 53, 53, 57, 57, 60, 61, 64, 65, + 67, 70, 71, 73, 74, 77, 77, 79, 80, 82, 83, 83, 84, 85, 85, 83, 60, 57, + 56, 54, 54, 53, 52, 52, 54, 55, 58, 59, 61, 63, 65, 67, 68, 71, 72, 75, + 75, 79, 79, 82, 83, 85, 86, 86, 87, 87, 86, 87, 61, 58, 57, 55, 55, 54, + 53, 53, 55, 56, 59, 59, 62, 63, 66, 68, 69, 72, 73, 76, 76, 80, 80, 83, + 84, 86, 87, 88, 89, 89, 89, 87, 63, 60, 59, 57, 57, 56, 55, 54, 57, 57, + 60, 61, 63, 65, 67, 69, 71, 73, 75, 78, 78, 82, 82, 85, 86, 89, 89, 90, + 91, 92, 90, 91, 64, 61, 60, 58, 57, 57, 56, 55, 57, 58, 61, 61, 64, 65, + 68, 69, 71, 74, 75, 78, 78, 82, 83, 86, 87, 89, 90, 91, 92, 93, 94, 91, + 65, 61, 61, 58, 58, 57, 56, 55, 58, 58, 61, 62, 64, 65, 68, 70, 71, 74, + 75, 78, 79, 83, 83, 86, 88, 90, 91, 91, 93, 94, 94, 96, 66, 63, 62, 60, + 59, 58, 57, 56, 59, 59, 62, 63, 65, 66, 69, 70, 72, 75, 76, 79, 80, 84, + 84, 87, 89, 91, 92, 93, 94, 94, 96, 96, 67, 64, 63, 61, 60, 59, 58, 57, + 59, 60, 62, 63, 66, 66, 70, 70, 73, 74, 77, 78, 81, 83, 85, 87, 89, 92, + 93, 94, 94, 96, 96, 97, 68, 64, 64, 61, 61, 60, 59, 58, 59, 61, 62, 64, + 65, 67, 69, 71, 72, 74, 77, 78, 81, 82, 85, 86, 89, 90, 94, 94, 96, 96, + 98, 97, 69, 65, 65, 62, 62, 61, 61, 58, 59, 62, 62, 65, 65, 68, 68, 71, + 71, 75, 75, 79, 79, 83, 83, 87, 87, 91, 91, 96, 96, 97, 97, 99], + /* Size 4x8 */ + [31, 47, 50, 61, 36, 47, 47, 57, 43, 50, 50, 58, 45, 53, 58, 65, 47, 54, + 66, 74, 52, 56, 70, 82, 57, 60, 75, 90, 61, 63, 77, 93], + /* Size 8x4 */ + [31, 36, 43, 45, 47, 52, 57, 61, 47, 47, 50, 53, 54, 56, 60, 63, 50, 47, + 50, 58, 66, 70, 75, 77, 61, 57, 58, 65, 74, 82, 90, 93], + /* Size 8x16 */ + [32, 32, 40, 49, 51, 57, 63, 67, 31, 33, 41, 47, 49, 54, 59, 63, 31, 35, + 43, 46, 47, 51, 57, 60, 35, 39, 46, 46, 47, 50, 55, 58, 41, 43, 48, 49, + 49, 52, 57, 59, 49, 47, 50, 53, 54, 57, 60, 62, 48, 46, 49, 54, 57, 60, + 64, 65, 49, 45, 48, 56, 61, 64, 67, 69, 50, 46, 49, 57, 63, 67, 71, 73, + 52, 48, 50, 58, 65, 71, 75, 77, 54, 50, 51, 59, 67, 73, 78, 81, 57, 52, + 53, 61, 69, 77, 82, 85, 61, 55, 56, 63, 72, 80, 86, 88, 64, 58, 58, 65, + 73, 82, 89, 92, 66, 59, 59, 66, 75, 84, 91, 94, 68, 61, 59, 65, 72, 81, + 89, 95], + /* Size 16x8 */ + [32, 31, 31, 35, 41, 49, 48, 49, 50, 52, 54, 57, 61, 64, 66, 68, 32, 33, + 35, 39, 43, 47, 46, 45, 46, 48, 50, 52, 55, 58, 59, 61, 40, 41, 43, 46, + 48, 50, 49, 48, 49, 50, 51, 53, 56, 58, 59, 59, 49, 47, 46, 46, 49, 53, + 54, 56, 57, 58, 59, 61, 63, 65, 66, 65, 51, 49, 47, 47, 49, 54, 57, 61, + 63, 65, 67, 69, 72, 73, 75, 72, 57, 54, 51, 50, 52, 57, 60, 64, 67, 71, + 73, 77, 80, 82, 84, 81, 63, 59, 57, 55, 57, 60, 64, 67, 71, 75, 78, 82, + 86, 89, 91, 89, 67, 63, 60, 58, 59, 62, 65, 69, 73, 77, 81, 85, 88, 92, + 94, 95], + /* Size 16x32 */ + [32, 31, 32, 37, 40, 48, 49, 49, 51, 52, 57, 58, 63, 64, 67, 67, 31, 31, + 33, 38, 41, 47, 47, 47, 49, 50, 54, 55, 60, 61, 63, 64, 31, 31, 33, 38, + 41, 47, 47, 47, 49, 49, 54, 54, 59, 60, 63, 64, 30, 32, 33, 40, 42, 46, + 45, 45, 47, 48, 52, 52, 57, 58, 60, 61, 31, 33, 35, 41, 43, 46, 46, 45, + 47, 48, 51, 52, 57, 57, 60, 61, 33, 36, 37, 43, 44, 47, 46, 46, 47, 47, + 51, 52, 56, 57, 59, 60, 35, 38, 39, 45, 46, 47, 46, 45, 47, 47, 50, 51, + 55, 56, 58, 60, 37, 40, 41, 47, 47, 47, 46, 45, 46, 47, 50, 50, 54, 55, + 57, 58, 41, 42, 43, 47, 48, 49, 49, 48, 49, 50, 52, 53, 57, 57, 59, 58, + 42, 43, 43, 47, 48, 50, 49, 49, 50, 50, 53, 54, 57, 58, 60, 61, 49, 46, + 47, 48, 50, 53, 53, 53, 54, 54, 57, 57, 60, 61, 62, 61, 49, 46, 47, 48, + 50, 53, 53, 54, 54, 55, 57, 57, 61, 61, 63, 64, 48, 46, 46, 47, 49, 53, + 54, 56, 57, 57, 60, 60, 64, 64, 65, 64, 48, 45, 46, 46, 49, 53, 55, 56, + 58, 58, 61, 61, 65, 65, 66, 67, 49, 45, 45, 46, 48, 53, 56, 58, 61, 61, + 64, 64, 67, 68, 69, 67, 49, 46, 46, 46, 49, 53, 57, 59, 62, 62, 65, 66, + 69, 69, 70, 70, 50, 46, 46, 46, 49, 54, 57, 59, 63, 64, 67, 67, 71, 71, + 73, 71, 51, 47, 47, 47, 49, 54, 58, 61, 64, 66, 69, 70, 73, 74, 74, 74, + 52, 48, 48, 47, 50, 54, 58, 61, 65, 66, 71, 71, 75, 75, 77, 74, 54, 50, + 49, 48, 51, 55, 59, 62, 67, 68, 73, 73, 77, 78, 78, 78, 54, 50, 50, 49, + 51, 55, 59, 62, 67, 68, 73, 74, 78, 78, 81, 78, 57, 52, 52, 50, 52, 56, + 60, 64, 69, 70, 76, 77, 82, 82, 83, 82, 57, 52, 52, 51, 53, 57, 61, 64, + 69, 71, 77, 77, 82, 83, 85, 82, 60, 54, 54, 52, 55, 58, 62, 65, 71, 72, + 79, 79, 85, 86, 87, 86, 61, 56, 55, 53, 56, 59, 63, 66, 72, 73, 80, 81, + 86, 87, 88, 86, 63, 57, 57, 55, 57, 60, 64, 67, 73, 75, 82, 82, 89, 90, + 92, 90, 64, 58, 58, 55, 58, 61, 65, 68, 73, 75, 82, 83, 89, 90, 92, 90, + 64, 59, 58, 56, 58, 61, 65, 68, 74, 75, 83, 83, 90, 91, 94, 95, 66, 60, + 59, 57, 59, 62, 66, 69, 75, 76, 84, 85, 91, 92, 94, 95, 67, 61, 60, 58, + 59, 63, 66, 70, 74, 77, 82, 85, 91, 93, 96, 96, 68, 62, 61, 58, 59, 64, + 65, 71, 72, 78, 81, 86, 89, 94, 95, 96, 68, 62, 62, 59, 59, 65, 65, 71, + 71, 79, 79, 87, 87, 95, 95, 98], + /* Size 32x16 */ + [32, 31, 31, 30, 31, 33, 35, 37, 41, 42, 49, 49, 48, 48, 49, 49, 50, 51, + 52, 54, 54, 57, 57, 60, 61, 63, 64, 64, 66, 67, 68, 68, 31, 31, 31, 32, + 33, 36, 38, 40, 42, 43, 46, 46, 46, 45, 45, 46, 46, 47, 48, 50, 50, 52, + 52, 54, 56, 57, 58, 59, 60, 61, 62, 62, 32, 33, 33, 33, 35, 37, 39, 41, + 43, 43, 47, 47, 46, 46, 45, 46, 46, 47, 48, 49, 50, 52, 52, 54, 55, 57, + 58, 58, 59, 60, 61, 62, 37, 38, 38, 40, 41, 43, 45, 47, 47, 47, 48, 48, + 47, 46, 46, 46, 46, 47, 47, 48, 49, 50, 51, 52, 53, 55, 55, 56, 57, 58, + 58, 59, 40, 41, 41, 42, 43, 44, 46, 47, 48, 48, 50, 50, 49, 49, 48, 49, + 49, 49, 50, 51, 51, 52, 53, 55, 56, 57, 58, 58, 59, 59, 59, 59, 48, 47, + 47, 46, 46, 47, 47, 47, 49, 50, 53, 53, 53, 53, 53, 53, 54, 54, 54, 55, + 55, 56, 57, 58, 59, 60, 61, 61, 62, 63, 64, 65, 49, 47, 47, 45, 46, 46, + 46, 46, 49, 49, 53, 53, 54, 55, 56, 57, 57, 58, 58, 59, 59, 60, 61, 62, + 63, 64, 65, 65, 66, 66, 65, 65, 49, 47, 47, 45, 45, 46, 45, 45, 48, 49, + 53, 54, 56, 56, 58, 59, 59, 61, 61, 62, 62, 64, 64, 65, 66, 67, 68, 68, + 69, 70, 71, 71, 51, 49, 49, 47, 47, 47, 47, 46, 49, 50, 54, 54, 57, 58, + 61, 62, 63, 64, 65, 67, 67, 69, 69, 71, 72, 73, 73, 74, 75, 74, 72, 71, + 52, 50, 49, 48, 48, 47, 47, 47, 50, 50, 54, 55, 57, 58, 61, 62, 64, 66, + 66, 68, 68, 70, 71, 72, 73, 75, 75, 75, 76, 77, 78, 79, 57, 54, 54, 52, + 51, 51, 50, 50, 52, 53, 57, 57, 60, 61, 64, 65, 67, 69, 71, 73, 73, 76, + 77, 79, 80, 82, 82, 83, 84, 82, 81, 79, 58, 55, 54, 52, 52, 52, 51, 50, + 53, 54, 57, 57, 60, 61, 64, 66, 67, 70, 71, 73, 74, 77, 77, 79, 81, 82, + 83, 83, 85, 85, 86, 87, 63, 60, 59, 57, 57, 56, 55, 54, 57, 57, 60, 61, + 64, 65, 67, 69, 71, 73, 75, 77, 78, 82, 82, 85, 86, 89, 89, 90, 91, 91, + 89, 87, 64, 61, 60, 58, 57, 57, 56, 55, 57, 58, 61, 61, 64, 65, 68, 69, + 71, 74, 75, 78, 78, 82, 83, 86, 87, 90, 90, 91, 92, 93, 94, 95, 67, 63, + 63, 60, 60, 59, 58, 57, 59, 60, 62, 63, 65, 66, 69, 70, 73, 74, 77, 78, + 81, 83, 85, 87, 88, 92, 92, 94, 94, 96, 95, 95, 67, 64, 64, 61, 61, 60, + 60, 58, 58, 61, 61, 64, 64, 67, 67, 70, 71, 74, 74, 78, 78, 82, 82, 86, + 86, 90, 90, 95, 95, 96, 96, 98], + /* Size 4x16 */ + [31, 48, 52, 64, 31, 47, 49, 60, 33, 46, 48, 57, 38, 47, 47, 56, 42, 49, + 50, 57, 46, 53, 54, 61, 46, 53, 57, 64, 45, 53, 61, 68, 46, 54, 64, 71, + 48, 54, 66, 75, 50, 55, 68, 78, 52, 57, 71, 83, 56, 59, 73, 87, 58, 61, + 75, 90, 60, 62, 76, 92, 62, 64, 78, 94], + /* Size 16x4 */ + [31, 31, 33, 38, 42, 46, 46, 45, 46, 48, 50, 52, 56, 58, 60, 62, 48, 47, + 46, 47, 49, 53, 53, 53, 54, 54, 55, 57, 59, 61, 62, 64, 52, 49, 48, 47, + 50, 54, 57, 61, 64, 66, 68, 71, 73, 75, 76, 78, 64, 60, 57, 56, 57, 61, + 64, 68, 71, 75, 78, 83, 87, 90, 92, 94], + /* Size 8x32 */ + [32, 32, 40, 49, 51, 57, 63, 67, 31, 33, 41, 47, 49, 54, 60, 63, 31, 33, + 41, 47, 49, 54, 59, 63, 30, 33, 42, 45, 47, 52, 57, 60, 31, 35, 43, 46, + 47, 51, 57, 60, 33, 37, 44, 46, 47, 51, 56, 59, 35, 39, 46, 46, 47, 50, + 55, 58, 37, 41, 47, 46, 46, 50, 54, 57, 41, 43, 48, 49, 49, 52, 57, 59, + 42, 43, 48, 49, 50, 53, 57, 60, 49, 47, 50, 53, 54, 57, 60, 62, 49, 47, + 50, 53, 54, 57, 61, 63, 48, 46, 49, 54, 57, 60, 64, 65, 48, 46, 49, 55, + 58, 61, 65, 66, 49, 45, 48, 56, 61, 64, 67, 69, 49, 46, 49, 57, 62, 65, + 69, 70, 50, 46, 49, 57, 63, 67, 71, 73, 51, 47, 49, 58, 64, 69, 73, 74, + 52, 48, 50, 58, 65, 71, 75, 77, 54, 49, 51, 59, 67, 73, 77, 78, 54, 50, + 51, 59, 67, 73, 78, 81, 57, 52, 52, 60, 69, 76, 82, 83, 57, 52, 53, 61, + 69, 77, 82, 85, 60, 54, 55, 62, 71, 79, 85, 87, 61, 55, 56, 63, 72, 80, + 86, 88, 63, 57, 57, 64, 73, 82, 89, 92, 64, 58, 58, 65, 73, 82, 89, 92, + 64, 58, 58, 65, 74, 83, 90, 94, 66, 59, 59, 66, 75, 84, 91, 94, 67, 60, + 59, 66, 74, 82, 91, 96, 68, 61, 59, 65, 72, 81, 89, 95, 68, 62, 59, 65, + 71, 79, 87, 95], + /* Size 32x8 */ + [32, 31, 31, 30, 31, 33, 35, 37, 41, 42, 49, 49, 48, 48, 49, 49, 50, 51, + 52, 54, 54, 57, 57, 60, 61, 63, 64, 64, 66, 67, 68, 68, 32, 33, 33, 33, + 35, 37, 39, 41, 43, 43, 47, 47, 46, 46, 45, 46, 46, 47, 48, 49, 50, 52, + 52, 54, 55, 57, 58, 58, 59, 60, 61, 62, 40, 41, 41, 42, 43, 44, 46, 47, + 48, 48, 50, 50, 49, 49, 48, 49, 49, 49, 50, 51, 51, 52, 53, 55, 56, 57, + 58, 58, 59, 59, 59, 59, 49, 47, 47, 45, 46, 46, 46, 46, 49, 49, 53, 53, + 54, 55, 56, 57, 57, 58, 58, 59, 59, 60, 61, 62, 63, 64, 65, 65, 66, 66, + 65, 65, 51, 49, 49, 47, 47, 47, 47, 46, 49, 50, 54, 54, 57, 58, 61, 62, + 63, 64, 65, 67, 67, 69, 69, 71, 72, 73, 73, 74, 75, 74, 72, 71, 57, 54, + 54, 52, 51, 51, 50, 50, 52, 53, 57, 57, 60, 61, 64, 65, 67, 69, 71, 73, + 73, 76, 77, 79, 80, 82, 82, 83, 84, 82, 81, 79, 63, 60, 59, 57, 57, 56, + 55, 54, 57, 57, 60, 61, 64, 65, 67, 69, 71, 73, 75, 77, 78, 82, 82, 85, + 86, 89, 89, 90, 91, 91, 89, 87, 67, 63, 63, 60, 60, 59, 58, 57, 59, 60, + 62, 63, 65, 66, 69, 70, 73, 74, 77, 78, 81, 83, 85, 87, 88, 92, 92, 94, + 94, 96, 95, 95]] + ], + [ + [ /* Luma */ + /* Size 4x4 */ + [32, 34, 49, 72, 34, 48, 60, 79, 49, 60, 82, 104, 72, 79, 104, 134], + /* Size 8x8 */ + [32, 32, 34, 38, 46, 56, 68, 78, 32, 33, 35, 39, 45, 54, 64, 74, 34, 35, + 39, 45, 51, 58, 68, 76, 38, 39, 45, 54, 61, 69, 78, 86, 46, 45, 51, 61, + 71, 80, 90, 99, 56, 54, 58, 69, 80, 92, 103, 113, 68, 64, 68, 78, 90, + 103, 117, 128, 78, 74, 76, 86, 99, 113, 128, 140], + /* Size 16x16 */ + [32, 31, 31, 31, 32, 34, 36, 39, 44, 48, 54, 59, 65, 71, 80, 83, 31, 32, + 32, 32, 32, 34, 35, 38, 42, 46, 51, 56, 62, 68, 76, 78, 31, 32, 32, 32, + 32, 33, 34, 37, 41, 44, 49, 54, 59, 65, 72, 75, 31, 32, 32, 33, 34, 35, + 36, 39, 42, 45, 50, 54, 59, 64, 71, 74, 32, 32, 32, 34, 35, 37, 38, 40, + 42, 46, 49, 53, 58, 63, 69, 72, 34, 34, 33, 35, 37, 39, 42, 45, 47, 51, + 54, 58, 63, 68, 74, 76, 36, 35, 34, 36, 38, 42, 48, 50, 54, 57, 60, 64, + 68, 73, 79, 81, 39, 38, 37, 39, 40, 45, 50, 54, 58, 61, 65, 69, 73, 78, + 84, 86, 44, 42, 41, 42, 42, 47, 54, 58, 63, 67, 71, 75, 79, 84, 90, 92, + 48, 46, 44, 45, 46, 51, 57, 61, 67, 71, 76, 80, 85, 90, 96, 99, 54, 51, + 49, 50, 49, 54, 60, 65, 71, 76, 82, 87, 92, 97, 104, 106, 59, 56, 54, + 54, 53, 58, 64, 69, 75, 80, 87, 92, 98, 103, 110, 113, 65, 62, 59, 59, + 58, 63, 68, 73, 79, 85, 92, 98, 105, 111, 118, 121, 71, 68, 65, 64, 63, + 68, 73, 78, 84, 90, 97, 103, 111, 117, 125, 128, 80, 76, 72, 71, 69, 74, + 79, 84, 90, 96, 104, 110, 118, 125, 134, 137, 83, 78, 75, 74, 72, 76, + 81, 86, 92, 99, 106, 113, 121, 128, 137, 140], + /* Size 32x32 */ + [32, 31, 31, 31, 31, 31, 31, 32, 32, 34, 34, 36, 36, 39, 39, 44, 44, 48, + 48, 54, 54, 59, 59, 65, 65, 71, 71, 80, 80, 83, 83, 87, 31, 32, 32, 32, + 32, 32, 32, 32, 32, 34, 34, 35, 35, 38, 38, 42, 42, 46, 46, 51, 51, 56, + 56, 62, 62, 68, 68, 76, 76, 78, 78, 83, 31, 32, 32, 32, 32, 32, 32, 32, + 32, 34, 34, 35, 35, 38, 38, 42, 42, 46, 46, 51, 51, 56, 56, 62, 62, 68, + 68, 76, 76, 78, 78, 83, 31, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 34, + 34, 37, 37, 41, 41, 44, 44, 49, 49, 54, 54, 59, 59, 65, 65, 72, 72, 75, + 75, 79, 31, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 34, 34, 37, 37, 41, + 41, 44, 44, 49, 49, 54, 54, 59, 59, 65, 65, 72, 72, 75, 75, 79, 31, 32, + 32, 32, 32, 33, 33, 34, 34, 35, 35, 36, 36, 39, 39, 42, 42, 45, 45, 50, + 50, 54, 54, 59, 59, 64, 64, 71, 71, 74, 74, 77, 31, 32, 32, 32, 32, 33, + 33, 34, 34, 35, 35, 36, 36, 39, 39, 42, 42, 45, 45, 50, 50, 54, 54, 59, + 59, 64, 64, 71, 71, 74, 74, 77, 32, 32, 32, 32, 32, 34, 34, 35, 35, 37, + 37, 38, 38, 40, 40, 42, 42, 46, 46, 49, 49, 53, 53, 58, 58, 63, 63, 69, + 69, 72, 72, 75, 32, 32, 32, 32, 32, 34, 34, 35, 35, 37, 37, 38, 38, 40, + 40, 42, 42, 46, 46, 49, 49, 53, 53, 58, 58, 63, 63, 69, 69, 72, 72, 75, + 34, 34, 34, 33, 33, 35, 35, 37, 37, 39, 39, 42, 42, 45, 45, 47, 47, 51, + 51, 54, 54, 58, 58, 63, 63, 68, 68, 74, 74, 76, 76, 80, 34, 34, 34, 33, + 33, 35, 35, 37, 37, 39, 39, 42, 42, 45, 45, 47, 47, 51, 51, 54, 54, 58, + 58, 63, 63, 68, 68, 74, 74, 76, 76, 80, 36, 35, 35, 34, 34, 36, 36, 38, + 38, 42, 42, 48, 48, 50, 50, 54, 54, 57, 57, 60, 60, 64, 64, 68, 68, 73, + 73, 79, 79, 81, 81, 84, 36, 35, 35, 34, 34, 36, 36, 38, 38, 42, 42, 48, + 48, 50, 50, 54, 54, 57, 57, 60, 60, 64, 64, 68, 68, 73, 73, 79, 79, 81, + 81, 84, 39, 38, 38, 37, 37, 39, 39, 40, 40, 45, 45, 50, 50, 54, 54, 58, + 58, 61, 61, 65, 65, 69, 69, 73, 73, 78, 78, 84, 84, 86, 86, 90, 39, 38, + 38, 37, 37, 39, 39, 40, 40, 45, 45, 50, 50, 54, 54, 58, 58, 61, 61, 65, + 65, 69, 69, 73, 73, 78, 78, 84, 84, 86, 86, 90, 44, 42, 42, 41, 41, 42, + 42, 42, 42, 47, 47, 54, 54, 58, 58, 63, 63, 67, 67, 71, 71, 75, 75, 79, + 79, 84, 84, 90, 90, 92, 92, 96, 44, 42, 42, 41, 41, 42, 42, 42, 42, 47, + 47, 54, 54, 58, 58, 63, 63, 67, 67, 71, 71, 75, 75, 79, 79, 84, 84, 90, + 90, 92, 92, 96, 48, 46, 46, 44, 44, 45, 45, 46, 46, 51, 51, 57, 57, 61, + 61, 67, 67, 71, 71, 76, 76, 80, 80, 85, 85, 90, 90, 96, 96, 99, 99, 102, + 48, 46, 46, 44, 44, 45, 45, 46, 46, 51, 51, 57, 57, 61, 61, 67, 67, 71, + 71, 76, 76, 80, 80, 85, 85, 90, 90, 96, 96, 99, 99, 102, 54, 51, 51, 49, + 49, 50, 50, 49, 49, 54, 54, 60, 60, 65, 65, 71, 71, 76, 76, 82, 82, 87, + 87, 92, 92, 97, 97, 104, 104, 106, 106, 109, 54, 51, 51, 49, 49, 50, 50, + 49, 49, 54, 54, 60, 60, 65, 65, 71, 71, 76, 76, 82, 82, 87, 87, 92, 92, + 97, 97, 104, 104, 106, 106, 109, 59, 56, 56, 54, 54, 54, 54, 53, 53, 58, + 58, 64, 64, 69, 69, 75, 75, 80, 80, 87, 87, 92, 92, 98, 98, 103, 103, + 110, 110, 113, 113, 116, 59, 56, 56, 54, 54, 54, 54, 53, 53, 58, 58, 64, + 64, 69, 69, 75, 75, 80, 80, 87, 87, 92, 92, 98, 98, 103, 103, 110, 110, + 113, 113, 116, 65, 62, 62, 59, 59, 59, 59, 58, 58, 63, 63, 68, 68, 73, + 73, 79, 79, 85, 85, 92, 92, 98, 98, 105, 105, 111, 111, 118, 118, 121, + 121, 124, 65, 62, 62, 59, 59, 59, 59, 58, 58, 63, 63, 68, 68, 73, 73, + 79, 79, 85, 85, 92, 92, 98, 98, 105, 105, 111, 111, 118, 118, 121, 121, + 124, 71, 68, 68, 65, 65, 64, 64, 63, 63, 68, 68, 73, 73, 78, 78, 84, 84, + 90, 90, 97, 97, 103, 103, 111, 111, 117, 117, 125, 125, 128, 128, 132, + 71, 68, 68, 65, 65, 64, 64, 63, 63, 68, 68, 73, 73, 78, 78, 84, 84, 90, + 90, 97, 97, 103, 103, 111, 111, 117, 117, 125, 125, 128, 128, 132, 80, + 76, 76, 72, 72, 71, 71, 69, 69, 74, 74, 79, 79, 84, 84, 90, 90, 96, 96, + 104, 104, 110, 110, 118, 118, 125, 125, 134, 134, 137, 137, 141, 80, 76, + 76, 72, 72, 71, 71, 69, 69, 74, 74, 79, 79, 84, 84, 90, 90, 96, 96, 104, + 104, 110, 110, 118, 118, 125, 125, 134, 134, 137, 137, 141, 83, 78, 78, + 75, 75, 74, 74, 72, 72, 76, 76, 81, 81, 86, 86, 92, 92, 99, 99, 106, + 106, 113, 113, 121, 121, 128, 128, 137, 137, 140, 140, 144, 83, 78, 78, + 75, 75, 74, 74, 72, 72, 76, 76, 81, 81, 86, 86, 92, 92, 99, 99, 106, + 106, 113, 113, 121, 121, 128, 128, 137, 137, 140, 140, 144, 87, 83, 83, + 79, 79, 77, 77, 75, 75, 80, 80, 84, 84, 90, 90, 96, 96, 102, 102, 109, + 109, 116, 116, 124, 124, 132, 132, 141, 141, 144, 144, 149], + /* Size 4x8 */ + [32, 35, 51, 75, 32, 36, 50, 71, 34, 42, 54, 73, 37, 50, 65, 84, 45, 56, + 76, 96, 54, 63, 87, 110, 65, 73, 97, 125, 75, 81, 106, 136], + /* Size 8x4 */ + [32, 32, 34, 37, 45, 54, 65, 75, 35, 36, 42, 50, 56, 63, 73, 81, 51, 50, + 54, 65, 76, 87, 97, 106, 75, 71, 73, 84, 96, 110, 125, 136], + /* Size 8x16 */ + [32, 31, 32, 36, 44, 53, 65, 79, 31, 32, 32, 35, 42, 51, 62, 75, 31, 32, + 33, 34, 41, 49, 59, 72, 32, 32, 34, 36, 42, 50, 59, 71, 32, 33, 35, 38, + 42, 49, 58, 69, 34, 34, 37, 42, 48, 54, 63, 73, 36, 34, 38, 48, 54, 60, + 68, 78, 39, 37, 40, 50, 58, 65, 73, 84, 44, 41, 43, 53, 63, 71, 79, 90, + 48, 45, 46, 56, 67, 76, 85, 96, 53, 49, 50, 60, 71, 82, 92, 103, 58, 54, + 54, 63, 75, 87, 98, 110, 65, 60, 58, 68, 79, 92, 105, 118, 71, 65, 63, + 73, 84, 97, 111, 125, 79, 72, 70, 79, 90, 104, 118, 133, 82, 75, 72, 81, + 92, 106, 121, 136], + /* Size 16x8 */ + [32, 31, 31, 32, 32, 34, 36, 39, 44, 48, 53, 58, 65, 71, 79, 82, 31, 32, + 32, 32, 33, 34, 34, 37, 41, 45, 49, 54, 60, 65, 72, 75, 32, 32, 33, 34, + 35, 37, 38, 40, 43, 46, 50, 54, 58, 63, 70, 72, 36, 35, 34, 36, 38, 42, + 48, 50, 53, 56, 60, 63, 68, 73, 79, 81, 44, 42, 41, 42, 42, 48, 54, 58, + 63, 67, 71, 75, 79, 84, 90, 92, 53, 51, 49, 50, 49, 54, 60, 65, 71, 76, + 82, 87, 92, 97, 104, 106, 65, 62, 59, 59, 58, 63, 68, 73, 79, 85, 92, + 98, 105, 111, 118, 121, 79, 75, 72, 71, 69, 73, 78, 84, 90, 96, 103, + 110, 118, 125, 133, 136], + /* Size 16x32 */ + [32, 31, 31, 32, 32, 36, 36, 44, 44, 53, 53, 65, 65, 79, 79, 87, 31, 32, + 32, 32, 32, 35, 35, 42, 42, 51, 51, 62, 62, 75, 75, 82, 31, 32, 32, 32, + 32, 35, 35, 42, 42, 51, 51, 62, 62, 75, 75, 82, 31, 32, 32, 33, 33, 34, + 34, 41, 41, 49, 49, 59, 59, 72, 72, 78, 31, 32, 32, 33, 33, 34, 34, 41, + 41, 49, 49, 59, 59, 72, 72, 78, 32, 32, 32, 34, 34, 36, 36, 42, 42, 50, + 50, 59, 59, 71, 71, 77, 32, 32, 32, 34, 34, 36, 36, 42, 42, 50, 50, 59, + 59, 71, 71, 77, 32, 33, 33, 35, 35, 38, 38, 42, 42, 49, 49, 58, 58, 69, + 69, 75, 32, 33, 33, 35, 35, 38, 38, 42, 42, 49, 49, 58, 58, 69, 69, 75, + 34, 34, 34, 37, 37, 42, 42, 48, 48, 54, 54, 63, 63, 73, 73, 79, 34, 34, + 34, 37, 37, 42, 42, 48, 48, 54, 54, 63, 63, 73, 73, 79, 36, 34, 34, 38, + 38, 48, 48, 54, 54, 60, 60, 68, 68, 78, 78, 84, 36, 34, 34, 38, 38, 48, + 48, 54, 54, 60, 60, 68, 68, 78, 78, 84, 39, 37, 37, 40, 40, 50, 50, 58, + 58, 65, 65, 73, 73, 84, 84, 89, 39, 37, 37, 40, 40, 50, 50, 58, 58, 65, + 65, 73, 73, 84, 84, 89, 44, 41, 41, 43, 43, 53, 53, 63, 63, 71, 71, 79, + 79, 90, 90, 95, 44, 41, 41, 43, 43, 53, 53, 63, 63, 71, 71, 79, 79, 90, + 90, 95, 48, 45, 45, 46, 46, 56, 56, 67, 67, 76, 76, 85, 85, 96, 96, 102, + 48, 45, 45, 46, 46, 56, 56, 67, 67, 76, 76, 85, 85, 96, 96, 102, 53, 49, + 49, 50, 50, 60, 60, 71, 71, 82, 82, 92, 92, 103, 103, 109, 53, 49, 49, + 50, 50, 60, 60, 71, 71, 82, 82, 92, 92, 103, 103, 109, 58, 54, 54, 54, + 54, 63, 63, 75, 75, 87, 87, 98, 98, 110, 110, 116, 58, 54, 54, 54, 54, + 63, 63, 75, 75, 87, 87, 98, 98, 110, 110, 116, 65, 60, 60, 58, 58, 68, + 68, 79, 79, 92, 92, 105, 105, 118, 118, 124, 65, 60, 60, 58, 58, 68, 68, + 79, 79, 92, 92, 105, 105, 118, 118, 124, 71, 65, 65, 63, 63, 73, 73, 84, + 84, 97, 97, 111, 111, 125, 125, 132, 71, 65, 65, 63, 63, 73, 73, 84, 84, + 97, 97, 111, 111, 125, 125, 132, 79, 72, 72, 70, 70, 79, 79, 90, 90, + 104, 104, 118, 118, 133, 133, 141, 79, 72, 72, 70, 70, 79, 79, 90, 90, + 104, 104, 118, 118, 133, 133, 141, 82, 75, 75, 72, 72, 81, 81, 92, 92, + 106, 106, 121, 121, 136, 136, 144, 82, 75, 75, 72, 72, 81, 81, 92, 92, + 106, 106, 121, 121, 136, 136, 144, 87, 79, 79, 76, 76, 84, 84, 96, 96, + 109, 109, 124, 124, 141, 141, 149], + /* Size 32x16 */ + [32, 31, 31, 31, 31, 32, 32, 32, 32, 34, 34, 36, 36, 39, 39, 44, 44, 48, + 48, 53, 53, 58, 58, 65, 65, 71, 71, 79, 79, 82, 82, 87, 31, 32, 32, 32, + 32, 32, 32, 33, 33, 34, 34, 34, 34, 37, 37, 41, 41, 45, 45, 49, 49, 54, + 54, 60, 60, 65, 65, 72, 72, 75, 75, 79, 31, 32, 32, 32, 32, 32, 32, 33, + 33, 34, 34, 34, 34, 37, 37, 41, 41, 45, 45, 49, 49, 54, 54, 60, 60, 65, + 65, 72, 72, 75, 75, 79, 32, 32, 32, 33, 33, 34, 34, 35, 35, 37, 37, 38, + 38, 40, 40, 43, 43, 46, 46, 50, 50, 54, 54, 58, 58, 63, 63, 70, 70, 72, + 72, 76, 32, 32, 32, 33, 33, 34, 34, 35, 35, 37, 37, 38, 38, 40, 40, 43, + 43, 46, 46, 50, 50, 54, 54, 58, 58, 63, 63, 70, 70, 72, 72, 76, 36, 35, + 35, 34, 34, 36, 36, 38, 38, 42, 42, 48, 48, 50, 50, 53, 53, 56, 56, 60, + 60, 63, 63, 68, 68, 73, 73, 79, 79, 81, 81, 84, 36, 35, 35, 34, 34, 36, + 36, 38, 38, 42, 42, 48, 48, 50, 50, 53, 53, 56, 56, 60, 60, 63, 63, 68, + 68, 73, 73, 79, 79, 81, 81, 84, 44, 42, 42, 41, 41, 42, 42, 42, 42, 48, + 48, 54, 54, 58, 58, 63, 63, 67, 67, 71, 71, 75, 75, 79, 79, 84, 84, 90, + 90, 92, 92, 96, 44, 42, 42, 41, 41, 42, 42, 42, 42, 48, 48, 54, 54, 58, + 58, 63, 63, 67, 67, 71, 71, 75, 75, 79, 79, 84, 84, 90, 90, 92, 92, 96, + 53, 51, 51, 49, 49, 50, 50, 49, 49, 54, 54, 60, 60, 65, 65, 71, 71, 76, + 76, 82, 82, 87, 87, 92, 92, 97, 97, 104, 104, 106, 106, 109, 53, 51, 51, + 49, 49, 50, 50, 49, 49, 54, 54, 60, 60, 65, 65, 71, 71, 76, 76, 82, 82, + 87, 87, 92, 92, 97, 97, 104, 104, 106, 106, 109, 65, 62, 62, 59, 59, 59, + 59, 58, 58, 63, 63, 68, 68, 73, 73, 79, 79, 85, 85, 92, 92, 98, 98, 105, + 105, 111, 111, 118, 118, 121, 121, 124, 65, 62, 62, 59, 59, 59, 59, 58, + 58, 63, 63, 68, 68, 73, 73, 79, 79, 85, 85, 92, 92, 98, 98, 105, 105, + 111, 111, 118, 118, 121, 121, 124, 79, 75, 75, 72, 72, 71, 71, 69, 69, + 73, 73, 78, 78, 84, 84, 90, 90, 96, 96, 103, 103, 110, 110, 118, 118, + 125, 125, 133, 133, 136, 136, 141, 79, 75, 75, 72, 72, 71, 71, 69, 69, + 73, 73, 78, 78, 84, 84, 90, 90, 96, 96, 103, 103, 110, 110, 118, 118, + 125, 125, 133, 133, 136, 136, 141, 87, 82, 82, 78, 78, 77, 77, 75, 75, + 79, 79, 84, 84, 89, 89, 95, 95, 102, 102, 109, 109, 116, 116, 124, 124, + 132, 132, 141, 141, 144, 144, 149], + /* Size 4x16 */ + [31, 36, 53, 79, 32, 35, 51, 75, 32, 34, 49, 72, 32, 36, 50, 71, 33, 38, + 49, 69, 34, 42, 54, 73, 34, 48, 60, 78, 37, 50, 65, 84, 41, 53, 71, 90, + 45, 56, 76, 96, 49, 60, 82, 103, 54, 63, 87, 110, 60, 68, 92, 118, 65, + 73, 97, 125, 72, 79, 104, 133, 75, 81, 106, 136], + /* Size 16x4 */ + [31, 32, 32, 32, 33, 34, 34, 37, 41, 45, 49, 54, 60, 65, 72, 75, 36, 35, + 34, 36, 38, 42, 48, 50, 53, 56, 60, 63, 68, 73, 79, 81, 53, 51, 49, 50, + 49, 54, 60, 65, 71, 76, 82, 87, 92, 97, 104, 106, 79, 75, 72, 71, 69, + 73, 78, 84, 90, 96, 103, 110, 118, 125, 133, 136], + /* Size 8x32 */ + [32, 31, 32, 36, 44, 53, 65, 79, 31, 32, 32, 35, 42, 51, 62, 75, 31, 32, + 32, 35, 42, 51, 62, 75, 31, 32, 33, 34, 41, 49, 59, 72, 31, 32, 33, 34, + 41, 49, 59, 72, 32, 32, 34, 36, 42, 50, 59, 71, 32, 32, 34, 36, 42, 50, + 59, 71, 32, 33, 35, 38, 42, 49, 58, 69, 32, 33, 35, 38, 42, 49, 58, 69, + 34, 34, 37, 42, 48, 54, 63, 73, 34, 34, 37, 42, 48, 54, 63, 73, 36, 34, + 38, 48, 54, 60, 68, 78, 36, 34, 38, 48, 54, 60, 68, 78, 39, 37, 40, 50, + 58, 65, 73, 84, 39, 37, 40, 50, 58, 65, 73, 84, 44, 41, 43, 53, 63, 71, + 79, 90, 44, 41, 43, 53, 63, 71, 79, 90, 48, 45, 46, 56, 67, 76, 85, 96, + 48, 45, 46, 56, 67, 76, 85, 96, 53, 49, 50, 60, 71, 82, 92, 103, 53, 49, + 50, 60, 71, 82, 92, 103, 58, 54, 54, 63, 75, 87, 98, 110, 58, 54, 54, + 63, 75, 87, 98, 110, 65, 60, 58, 68, 79, 92, 105, 118, 65, 60, 58, 68, + 79, 92, 105, 118, 71, 65, 63, 73, 84, 97, 111, 125, 71, 65, 63, 73, 84, + 97, 111, 125, 79, 72, 70, 79, 90, 104, 118, 133, 79, 72, 70, 79, 90, + 104, 118, 133, 82, 75, 72, 81, 92, 106, 121, 136, 82, 75, 72, 81, 92, + 106, 121, 136, 87, 79, 76, 84, 96, 109, 124, 141], + /* Size 32x8 */ + [32, 31, 31, 31, 31, 32, 32, 32, 32, 34, 34, 36, 36, 39, 39, 44, 44, 48, + 48, 53, 53, 58, 58, 65, 65, 71, 71, 79, 79, 82, 82, 87, 31, 32, 32, 32, + 32, 32, 32, 33, 33, 34, 34, 34, 34, 37, 37, 41, 41, 45, 45, 49, 49, 54, + 54, 60, 60, 65, 65, 72, 72, 75, 75, 79, 32, 32, 32, 33, 33, 34, 34, 35, + 35, 37, 37, 38, 38, 40, 40, 43, 43, 46, 46, 50, 50, 54, 54, 58, 58, 63, + 63, 70, 70, 72, 72, 76, 36, 35, 35, 34, 34, 36, 36, 38, 38, 42, 42, 48, + 48, 50, 50, 53, 53, 56, 56, 60, 60, 63, 63, 68, 68, 73, 73, 79, 79, 81, + 81, 84, 44, 42, 42, 41, 41, 42, 42, 42, 42, 48, 48, 54, 54, 58, 58, 63, + 63, 67, 67, 71, 71, 75, 75, 79, 79, 84, 84, 90, 90, 92, 92, 96, 53, 51, + 51, 49, 49, 50, 50, 49, 49, 54, 54, 60, 60, 65, 65, 71, 71, 76, 76, 82, + 82, 87, 87, 92, 92, 97, 97, 104, 104, 106, 106, 109, 65, 62, 62, 59, 59, + 59, 59, 58, 58, 63, 63, 68, 68, 73, 73, 79, 79, 85, 85, 92, 92, 98, 98, + 105, 105, 111, 111, 118, 118, 121, 121, 124, 79, 75, 75, 72, 72, 71, 71, + 69, 69, 73, 73, 78, 78, 84, 84, 90, 90, 96, 96, 103, 103, 110, 110, 118, + 118, 125, 125, 133, 133, 136, 136, 141]], + [ /* Chroma */ + /* Size 4x4 */ + [32, 46, 47, 57, 46, 53, 54, 60, 47, 54, 66, 75, 57, 60, 75, 89], + /* Size 8x8 */ + [31, 34, 42, 47, 48, 52, 57, 61, 34, 39, 45, 46, 46, 49, 53, 57, 42, 45, + 48, 49, 50, 52, 55, 58, 47, 46, 49, 54, 56, 58, 61, 64, 48, 46, 50, 56, + 61, 65, 68, 71, 52, 49, 52, 58, 65, 71, 75, 79, 57, 53, 55, 61, 68, 75, + 82, 86, 61, 57, 58, 64, 71, 79, 86, 91], + /* Size 16x16 */ + [32, 31, 30, 33, 36, 41, 49, 48, 49, 50, 52, 54, 57, 60, 63, 65, 31, 31, + 31, 34, 38, 42, 47, 47, 47, 48, 50, 52, 54, 57, 60, 61, 30, 31, 32, 35, + 40, 42, 46, 45, 45, 46, 47, 49, 52, 54, 57, 58, 33, 34, 35, 39, 43, 45, + 47, 46, 45, 46, 47, 49, 51, 53, 56, 57, 36, 38, 40, 43, 47, 47, 48, 46, + 45, 46, 47, 48, 50, 52, 54, 55, 41, 42, 42, 45, 47, 48, 50, 49, 49, 50, + 50, 52, 53, 55, 57, 58, 49, 47, 46, 47, 48, 50, 53, 53, 53, 54, 54, 55, + 56, 58, 60, 61, 48, 47, 45, 46, 46, 49, 53, 54, 55, 56, 57, 58, 60, 61, + 63, 64, 49, 47, 45, 45, 45, 49, 53, 55, 58, 60, 61, 62, 63, 65, 67, 68, + 50, 48, 46, 46, 46, 50, 54, 56, 60, 61, 63, 65, 67, 68, 71, 71, 52, 50, + 47, 47, 47, 50, 54, 57, 61, 63, 66, 68, 70, 72, 75, 75, 54, 52, 49, 49, + 48, 52, 55, 58, 62, 65, 68, 71, 73, 75, 78, 79, 57, 54, 52, 51, 50, 53, + 56, 60, 63, 67, 70, 73, 76, 79, 82, 83, 60, 57, 54, 53, 52, 55, 58, 61, + 65, 68, 72, 75, 79, 82, 85, 86, 63, 60, 57, 56, 54, 57, 60, 63, 67, 71, + 75, 78, 82, 85, 89, 90, 65, 61, 58, 57, 55, 58, 61, 64, 68, 71, 75, 79, + 83, 86, 90, 91], + /* Size 32x32 */ + [32, 31, 31, 30, 30, 33, 33, 36, 36, 41, 41, 49, 49, 48, 48, 49, 49, 50, + 50, 52, 52, 54, 54, 57, 57, 60, 60, 63, 63, 65, 65, 67, 31, 31, 31, 31, + 31, 34, 34, 38, 38, 42, 42, 47, 47, 47, 47, 47, 47, 48, 48, 50, 50, 52, + 52, 54, 54, 57, 57, 60, 60, 61, 61, 63, 31, 31, 31, 31, 31, 34, 34, 38, + 38, 42, 42, 47, 47, 47, 47, 47, 47, 48, 48, 50, 50, 52, 52, 54, 54, 57, + 57, 60, 60, 61, 61, 63, 30, 31, 31, 32, 32, 35, 35, 40, 40, 42, 42, 46, + 46, 45, 45, 45, 45, 46, 46, 47, 47, 49, 49, 52, 52, 54, 54, 57, 57, 58, + 58, 60, 30, 31, 31, 32, 32, 35, 35, 40, 40, 42, 42, 46, 46, 45, 45, 45, + 45, 46, 46, 47, 47, 49, 49, 52, 52, 54, 54, 57, 57, 58, 58, 60, 33, 34, + 34, 35, 35, 39, 39, 43, 43, 45, 45, 47, 47, 46, 46, 45, 45, 46, 46, 47, + 47, 49, 49, 51, 51, 53, 53, 56, 56, 57, 57, 59, 33, 34, 34, 35, 35, 39, + 39, 43, 43, 45, 45, 47, 47, 46, 46, 45, 45, 46, 46, 47, 47, 49, 49, 51, + 51, 53, 53, 56, 56, 57, 57, 59, 36, 38, 38, 40, 40, 43, 43, 47, 47, 47, + 47, 48, 48, 46, 46, 45, 45, 46, 46, 47, 47, 48, 48, 50, 50, 52, 52, 54, + 54, 55, 55, 57, 36, 38, 38, 40, 40, 43, 43, 47, 47, 47, 47, 48, 48, 46, + 46, 45, 45, 46, 46, 47, 47, 48, 48, 50, 50, 52, 52, 54, 54, 55, 55, 57, + 41, 42, 42, 42, 42, 45, 45, 47, 47, 48, 48, 50, 50, 49, 49, 49, 49, 50, + 50, 50, 50, 52, 52, 53, 53, 55, 55, 57, 57, 58, 58, 60, 41, 42, 42, 42, + 42, 45, 45, 47, 47, 48, 48, 50, 50, 49, 49, 49, 49, 50, 50, 50, 50, 52, + 52, 53, 53, 55, 55, 57, 57, 58, 58, 60, 49, 47, 47, 46, 46, 47, 47, 48, + 48, 50, 50, 53, 53, 53, 53, 53, 53, 54, 54, 54, 54, 55, 55, 56, 56, 58, + 58, 60, 60, 61, 61, 62, 49, 47, 47, 46, 46, 47, 47, 48, 48, 50, 50, 53, + 53, 53, 53, 53, 53, 54, 54, 54, 54, 55, 55, 56, 56, 58, 58, 60, 60, 61, + 61, 62, 48, 47, 47, 45, 45, 46, 46, 46, 46, 49, 49, 53, 53, 54, 54, 55, + 55, 56, 56, 57, 57, 58, 58, 60, 60, 61, 61, 63, 63, 64, 64, 66, 48, 47, + 47, 45, 45, 46, 46, 46, 46, 49, 49, 53, 53, 54, 54, 55, 55, 56, 56, 57, + 57, 58, 58, 60, 60, 61, 61, 63, 63, 64, 64, 66, 49, 47, 47, 45, 45, 45, + 45, 45, 45, 49, 49, 53, 53, 55, 55, 58, 58, 60, 60, 61, 61, 62, 62, 63, + 63, 65, 65, 67, 67, 68, 68, 69, 49, 47, 47, 45, 45, 45, 45, 45, 45, 49, + 49, 53, 53, 55, 55, 58, 58, 60, 60, 61, 61, 62, 62, 63, 63, 65, 65, 67, + 67, 68, 68, 69, 50, 48, 48, 46, 46, 46, 46, 46, 46, 50, 50, 54, 54, 56, + 56, 60, 60, 61, 61, 63, 63, 65, 65, 67, 67, 68, 68, 71, 71, 71, 71, 72, + 50, 48, 48, 46, 46, 46, 46, 46, 46, 50, 50, 54, 54, 56, 56, 60, 60, 61, + 61, 63, 63, 65, 65, 67, 67, 68, 68, 71, 71, 71, 71, 72, 52, 50, 50, 47, + 47, 47, 47, 47, 47, 50, 50, 54, 54, 57, 57, 61, 61, 63, 63, 66, 66, 68, + 68, 70, 70, 72, 72, 75, 75, 75, 75, 76, 52, 50, 50, 47, 47, 47, 47, 47, + 47, 50, 50, 54, 54, 57, 57, 61, 61, 63, 63, 66, 66, 68, 68, 70, 70, 72, + 72, 75, 75, 75, 75, 76, 54, 52, 52, 49, 49, 49, 49, 48, 48, 52, 52, 55, + 55, 58, 58, 62, 62, 65, 65, 68, 68, 71, 71, 73, 73, 75, 75, 78, 78, 79, + 79, 80, 54, 52, 52, 49, 49, 49, 49, 48, 48, 52, 52, 55, 55, 58, 58, 62, + 62, 65, 65, 68, 68, 71, 71, 73, 73, 75, 75, 78, 78, 79, 79, 80, 57, 54, + 54, 52, 52, 51, 51, 50, 50, 53, 53, 56, 56, 60, 60, 63, 63, 67, 67, 70, + 70, 73, 73, 76, 76, 79, 79, 82, 82, 83, 83, 84, 57, 54, 54, 52, 52, 51, + 51, 50, 50, 53, 53, 56, 56, 60, 60, 63, 63, 67, 67, 70, 70, 73, 73, 76, + 76, 79, 79, 82, 82, 83, 83, 84, 60, 57, 57, 54, 54, 53, 53, 52, 52, 55, + 55, 58, 58, 61, 61, 65, 65, 68, 68, 72, 72, 75, 75, 79, 79, 82, 82, 85, + 85, 86, 86, 88, 60, 57, 57, 54, 54, 53, 53, 52, 52, 55, 55, 58, 58, 61, + 61, 65, 65, 68, 68, 72, 72, 75, 75, 79, 79, 82, 82, 85, 85, 86, 86, 88, + 63, 60, 60, 57, 57, 56, 56, 54, 54, 57, 57, 60, 60, 63, 63, 67, 67, 71, + 71, 75, 75, 78, 78, 82, 82, 85, 85, 89, 89, 90, 90, 92, 63, 60, 60, 57, + 57, 56, 56, 54, 54, 57, 57, 60, 60, 63, 63, 67, 67, 71, 71, 75, 75, 78, + 78, 82, 82, 85, 85, 89, 89, 90, 90, 92, 65, 61, 61, 58, 58, 57, 57, 55, + 55, 58, 58, 61, 61, 64, 64, 68, 68, 71, 71, 75, 75, 79, 79, 83, 83, 86, + 86, 90, 90, 91, 91, 93, 65, 61, 61, 58, 58, 57, 57, 55, 55, 58, 58, 61, + 61, 64, 64, 68, 68, 71, 71, 75, 75, 79, 79, 83, 83, 86, 86, 90, 90, 91, + 91, 93, 67, 63, 63, 60, 60, 59, 59, 57, 57, 60, 60, 62, 62, 66, 66, 69, + 69, 72, 72, 76, 76, 80, 80, 84, 84, 88, 88, 92, 92, 93, 93, 95], + /* Size 4x8 */ + [31, 47, 50, 60, 36, 47, 47, 56, 43, 50, 50, 57, 46, 53, 57, 64, 46, 54, + 64, 71, 50, 55, 68, 78, 54, 58, 72, 85, 59, 61, 75, 90], + /* Size 8x4 */ + [31, 36, 43, 46, 46, 50, 54, 59, 47, 47, 50, 53, 54, 55, 58, 61, 50, 47, + 50, 57, 64, 68, 72, 75, 60, 56, 57, 64, 71, 78, 85, 90], + /* Size 8x16 */ + [32, 31, 37, 48, 49, 52, 57, 63, 31, 31, 38, 47, 47, 50, 54, 60, 30, 32, + 40, 46, 45, 48, 52, 57, 33, 36, 43, 47, 46, 47, 51, 56, 37, 40, 47, 47, + 45, 47, 50, 54, 42, 43, 47, 50, 49, 50, 53, 57, 49, 46, 48, 53, 53, 54, + 57, 60, 48, 46, 47, 53, 56, 57, 60, 64, 49, 45, 46, 53, 58, 61, 64, 67, + 50, 46, 46, 54, 59, 64, 67, 71, 52, 48, 47, 54, 61, 66, 71, 75, 54, 50, + 49, 55, 62, 68, 73, 78, 57, 52, 50, 56, 64, 70, 76, 82, 60, 54, 52, 58, + 65, 72, 79, 85, 63, 57, 55, 60, 67, 75, 82, 89, 64, 59, 56, 61, 68, 75, + 83, 90], + /* Size 16x8 */ + [32, 31, 30, 33, 37, 42, 49, 48, 49, 50, 52, 54, 57, 60, 63, 64, 31, 31, + 32, 36, 40, 43, 46, 46, 45, 46, 48, 50, 52, 54, 57, 59, 37, 38, 40, 43, + 47, 47, 48, 47, 46, 46, 47, 49, 50, 52, 55, 56, 48, 47, 46, 47, 47, 50, + 53, 53, 53, 54, 54, 55, 56, 58, 60, 61, 49, 47, 45, 46, 45, 49, 53, 56, + 58, 59, 61, 62, 64, 65, 67, 68, 52, 50, 48, 47, 47, 50, 54, 57, 61, 64, + 66, 68, 70, 72, 75, 75, 57, 54, 52, 51, 50, 53, 57, 60, 64, 67, 71, 73, + 76, 79, 82, 83, 63, 60, 57, 56, 54, 57, 60, 64, 67, 71, 75, 78, 82, 85, + 89, 90], + /* Size 16x32 */ + [32, 31, 31, 37, 37, 48, 48, 49, 49, 52, 52, 57, 57, 63, 63, 66, 31, 31, + 31, 38, 38, 47, 47, 47, 47, 50, 50, 54, 54, 60, 60, 63, 31, 31, 31, 38, + 38, 47, 47, 47, 47, 50, 50, 54, 54, 60, 60, 63, 30, 32, 32, 40, 40, 46, + 46, 45, 45, 48, 48, 52, 52, 57, 57, 60, 30, 32, 32, 40, 40, 46, 46, 45, + 45, 48, 48, 52, 52, 57, 57, 60, 33, 36, 36, 43, 43, 47, 47, 46, 46, 47, + 47, 51, 51, 56, 56, 59, 33, 36, 36, 43, 43, 47, 47, 46, 46, 47, 47, 51, + 51, 56, 56, 59, 37, 40, 40, 47, 47, 47, 47, 45, 45, 47, 47, 50, 50, 54, + 54, 57, 37, 40, 40, 47, 47, 47, 47, 45, 45, 47, 47, 50, 50, 54, 54, 57, + 42, 43, 43, 47, 47, 50, 50, 49, 49, 50, 50, 53, 53, 57, 57, 60, 42, 43, + 43, 47, 47, 50, 50, 49, 49, 50, 50, 53, 53, 57, 57, 60, 49, 46, 46, 48, + 48, 53, 53, 53, 53, 54, 54, 57, 57, 60, 60, 62, 49, 46, 46, 48, 48, 53, + 53, 53, 53, 54, 54, 57, 57, 60, 60, 62, 48, 46, 46, 47, 47, 53, 53, 56, + 56, 57, 57, 60, 60, 64, 64, 66, 48, 46, 46, 47, 47, 53, 53, 56, 56, 57, + 57, 60, 60, 64, 64, 66, 49, 45, 45, 46, 46, 53, 53, 58, 58, 61, 61, 64, + 64, 67, 67, 69, 49, 45, 45, 46, 46, 53, 53, 58, 58, 61, 61, 64, 64, 67, + 67, 69, 50, 46, 46, 46, 46, 54, 54, 59, 59, 64, 64, 67, 67, 71, 71, 73, + 50, 46, 46, 46, 46, 54, 54, 59, 59, 64, 64, 67, 67, 71, 71, 73, 52, 48, + 48, 47, 47, 54, 54, 61, 61, 66, 66, 71, 71, 75, 75, 77, 52, 48, 48, 47, + 47, 54, 54, 61, 61, 66, 66, 71, 71, 75, 75, 77, 54, 50, 50, 49, 49, 55, + 55, 62, 62, 68, 68, 73, 73, 78, 78, 80, 54, 50, 50, 49, 49, 55, 55, 62, + 62, 68, 68, 73, 73, 78, 78, 80, 57, 52, 52, 50, 50, 56, 56, 64, 64, 70, + 70, 76, 76, 82, 82, 84, 57, 52, 52, 50, 50, 56, 56, 64, 64, 70, 70, 76, + 76, 82, 82, 84, 60, 54, 54, 52, 52, 58, 58, 65, 65, 72, 72, 79, 79, 85, + 85, 88, 60, 54, 54, 52, 52, 58, 58, 65, 65, 72, 72, 79, 79, 85, 85, 88, + 63, 57, 57, 55, 55, 60, 60, 67, 67, 75, 75, 82, 82, 89, 89, 92, 63, 57, + 57, 55, 55, 60, 60, 67, 67, 75, 75, 82, 82, 89, 89, 92, 64, 59, 59, 56, + 56, 61, 61, 68, 68, 75, 75, 83, 83, 90, 90, 93, 64, 59, 59, 56, 56, 61, + 61, 68, 68, 75, 75, 83, 83, 90, 90, 93, 66, 60, 60, 57, 57, 63, 63, 69, + 69, 77, 77, 84, 84, 92, 92, 95], + /* Size 32x16 */ + [32, 31, 31, 30, 30, 33, 33, 37, 37, 42, 42, 49, 49, 48, 48, 49, 49, 50, + 50, 52, 52, 54, 54, 57, 57, 60, 60, 63, 63, 64, 64, 66, 31, 31, 31, 32, + 32, 36, 36, 40, 40, 43, 43, 46, 46, 46, 46, 45, 45, 46, 46, 48, 48, 50, + 50, 52, 52, 54, 54, 57, 57, 59, 59, 60, 31, 31, 31, 32, 32, 36, 36, 40, + 40, 43, 43, 46, 46, 46, 46, 45, 45, 46, 46, 48, 48, 50, 50, 52, 52, 54, + 54, 57, 57, 59, 59, 60, 37, 38, 38, 40, 40, 43, 43, 47, 47, 47, 47, 48, + 48, 47, 47, 46, 46, 46, 46, 47, 47, 49, 49, 50, 50, 52, 52, 55, 55, 56, + 56, 57, 37, 38, 38, 40, 40, 43, 43, 47, 47, 47, 47, 48, 48, 47, 47, 46, + 46, 46, 46, 47, 47, 49, 49, 50, 50, 52, 52, 55, 55, 56, 56, 57, 48, 47, + 47, 46, 46, 47, 47, 47, 47, 50, 50, 53, 53, 53, 53, 53, 53, 54, 54, 54, + 54, 55, 55, 56, 56, 58, 58, 60, 60, 61, 61, 63, 48, 47, 47, 46, 46, 47, + 47, 47, 47, 50, 50, 53, 53, 53, 53, 53, 53, 54, 54, 54, 54, 55, 55, 56, + 56, 58, 58, 60, 60, 61, 61, 63, 49, 47, 47, 45, 45, 46, 46, 45, 45, 49, + 49, 53, 53, 56, 56, 58, 58, 59, 59, 61, 61, 62, 62, 64, 64, 65, 65, 67, + 67, 68, 68, 69, 49, 47, 47, 45, 45, 46, 46, 45, 45, 49, 49, 53, 53, 56, + 56, 58, 58, 59, 59, 61, 61, 62, 62, 64, 64, 65, 65, 67, 67, 68, 68, 69, + 52, 50, 50, 48, 48, 47, 47, 47, 47, 50, 50, 54, 54, 57, 57, 61, 61, 64, + 64, 66, 66, 68, 68, 70, 70, 72, 72, 75, 75, 75, 75, 77, 52, 50, 50, 48, + 48, 47, 47, 47, 47, 50, 50, 54, 54, 57, 57, 61, 61, 64, 64, 66, 66, 68, + 68, 70, 70, 72, 72, 75, 75, 75, 75, 77, 57, 54, 54, 52, 52, 51, 51, 50, + 50, 53, 53, 57, 57, 60, 60, 64, 64, 67, 67, 71, 71, 73, 73, 76, 76, 79, + 79, 82, 82, 83, 83, 84, 57, 54, 54, 52, 52, 51, 51, 50, 50, 53, 53, 57, + 57, 60, 60, 64, 64, 67, 67, 71, 71, 73, 73, 76, 76, 79, 79, 82, 82, 83, + 83, 84, 63, 60, 60, 57, 57, 56, 56, 54, 54, 57, 57, 60, 60, 64, 64, 67, + 67, 71, 71, 75, 75, 78, 78, 82, 82, 85, 85, 89, 89, 90, 90, 92, 63, 60, + 60, 57, 57, 56, 56, 54, 54, 57, 57, 60, 60, 64, 64, 67, 67, 71, 71, 75, + 75, 78, 78, 82, 82, 85, 85, 89, 89, 90, 90, 92, 66, 63, 63, 60, 60, 59, + 59, 57, 57, 60, 60, 62, 62, 66, 66, 69, 69, 73, 73, 77, 77, 80, 80, 84, + 84, 88, 88, 92, 92, 93, 93, 95], + /* Size 4x16 */ + [31, 48, 52, 63, 31, 47, 50, 60, 32, 46, 48, 57, 36, 47, 47, 56, 40, 47, + 47, 54, 43, 50, 50, 57, 46, 53, 54, 60, 46, 53, 57, 64, 45, 53, 61, 67, + 46, 54, 64, 71, 48, 54, 66, 75, 50, 55, 68, 78, 52, 56, 70, 82, 54, 58, + 72, 85, 57, 60, 75, 89, 59, 61, 75, 90], + /* Size 16x4 */ + [31, 31, 32, 36, 40, 43, 46, 46, 45, 46, 48, 50, 52, 54, 57, 59, 48, 47, + 46, 47, 47, 50, 53, 53, 53, 54, 54, 55, 56, 58, 60, 61, 52, 50, 48, 47, + 47, 50, 54, 57, 61, 64, 66, 68, 70, 72, 75, 75, 63, 60, 57, 56, 54, 57, + 60, 64, 67, 71, 75, 78, 82, 85, 89, 90], + /* Size 8x32 */ + [32, 31, 37, 48, 49, 52, 57, 63, 31, 31, 38, 47, 47, 50, 54, 60, 31, 31, + 38, 47, 47, 50, 54, 60, 30, 32, 40, 46, 45, 48, 52, 57, 30, 32, 40, 46, + 45, 48, 52, 57, 33, 36, 43, 47, 46, 47, 51, 56, 33, 36, 43, 47, 46, 47, + 51, 56, 37, 40, 47, 47, 45, 47, 50, 54, 37, 40, 47, 47, 45, 47, 50, 54, + 42, 43, 47, 50, 49, 50, 53, 57, 42, 43, 47, 50, 49, 50, 53, 57, 49, 46, + 48, 53, 53, 54, 57, 60, 49, 46, 48, 53, 53, 54, 57, 60, 48, 46, 47, 53, + 56, 57, 60, 64, 48, 46, 47, 53, 56, 57, 60, 64, 49, 45, 46, 53, 58, 61, + 64, 67, 49, 45, 46, 53, 58, 61, 64, 67, 50, 46, 46, 54, 59, 64, 67, 71, + 50, 46, 46, 54, 59, 64, 67, 71, 52, 48, 47, 54, 61, 66, 71, 75, 52, 48, + 47, 54, 61, 66, 71, 75, 54, 50, 49, 55, 62, 68, 73, 78, 54, 50, 49, 55, + 62, 68, 73, 78, 57, 52, 50, 56, 64, 70, 76, 82, 57, 52, 50, 56, 64, 70, + 76, 82, 60, 54, 52, 58, 65, 72, 79, 85, 60, 54, 52, 58, 65, 72, 79, 85, + 63, 57, 55, 60, 67, 75, 82, 89, 63, 57, 55, 60, 67, 75, 82, 89, 64, 59, + 56, 61, 68, 75, 83, 90, 64, 59, 56, 61, 68, 75, 83, 90, 66, 60, 57, 63, + 69, 77, 84, 92], + /* Size 32x8 */ + [32, 31, 31, 30, 30, 33, 33, 37, 37, 42, 42, 49, 49, 48, 48, 49, 49, 50, + 50, 52, 52, 54, 54, 57, 57, 60, 60, 63, 63, 64, 64, 66, 31, 31, 31, 32, + 32, 36, 36, 40, 40, 43, 43, 46, 46, 46, 46, 45, 45, 46, 46, 48, 48, 50, + 50, 52, 52, 54, 54, 57, 57, 59, 59, 60, 37, 38, 38, 40, 40, 43, 43, 47, + 47, 47, 47, 48, 48, 47, 47, 46, 46, 46, 46, 47, 47, 49, 49, 50, 50, 52, + 52, 55, 55, 56, 56, 57, 48, 47, 47, 46, 46, 47, 47, 47, 47, 50, 50, 53, + 53, 53, 53, 53, 53, 54, 54, 54, 54, 55, 55, 56, 56, 58, 58, 60, 60, 61, + 61, 63, 49, 47, 47, 45, 45, 46, 46, 45, 45, 49, 49, 53, 53, 56, 56, 58, + 58, 59, 59, 61, 61, 62, 62, 64, 64, 65, 65, 67, 67, 68, 68, 69, 52, 50, + 50, 48, 48, 47, 47, 47, 47, 50, 50, 54, 54, 57, 57, 61, 61, 64, 64, 66, + 66, 68, 68, 70, 70, 72, 72, 75, 75, 75, 75, 77, 57, 54, 54, 52, 52, 51, + 51, 50, 50, 53, 53, 57, 57, 60, 60, 64, 64, 67, 67, 71, 71, 73, 73, 76, + 76, 79, 79, 82, 82, 83, 83, 84, 63, 60, 60, 57, 57, 56, 56, 54, 54, 57, + 57, 60, 60, 64, 64, 67, 67, 71, 71, 75, 75, 78, 78, 82, 82, 85, 85, 89, + 89, 90, 90, 92]] + ], + [ + [ /* Luma */ + /* Size 4x4 */ + [32, 33, 45, 62, 33, 39, 51, 64, 45, 51, 71, 87, 62, 64, 87, 108], + /* Size 8x8 */ + [31, 32, 32, 35, 42, 51, 59, 69, 32, 32, 33, 35, 41, 49, 56, 65, 32, 33, + 35, 38, 43, 49, 56, 64, 35, 35, 38, 48, 54, 59, 66, 73, 42, 41, 43, 54, + 63, 71, 77, 85, 51, 49, 49, 59, 71, 81, 89, 97, 59, 56, 56, 66, 77, 89, + 98, 108, 69, 65, 64, 73, 85, 97, 108, 119], + /* Size 16x16 */ + [32, 31, 31, 31, 32, 34, 35, 38, 41, 45, 48, 54, 59, 65, 71, 80, 31, 32, + 32, 32, 32, 34, 35, 37, 40, 43, 46, 51, 56, 62, 68, 76, 31, 32, 32, 32, + 32, 33, 34, 36, 38, 41, 44, 49, 54, 59, 65, 72, 31, 32, 32, 33, 34, 35, + 36, 38, 40, 42, 45, 50, 54, 59, 64, 71, 32, 32, 32, 34, 35, 37, 38, 39, + 41, 43, 46, 49, 53, 58, 63, 69, 34, 34, 33, 35, 37, 39, 42, 44, 46, 48, + 51, 54, 58, 63, 68, 74, 35, 35, 34, 36, 38, 42, 46, 48, 50, 53, 55, 59, + 62, 67, 72, 78, 38, 37, 36, 38, 39, 44, 48, 51, 54, 57, 59, 63, 67, 71, + 76, 82, 41, 40, 38, 40, 41, 46, 50, 54, 57, 60, 63, 67, 71, 75, 80, 86, + 45, 43, 41, 42, 43, 48, 53, 57, 60, 65, 68, 72, 76, 81, 85, 91, 48, 46, + 44, 45, 46, 51, 55, 59, 63, 68, 71, 76, 80, 85, 90, 96, 54, 51, 49, 50, + 49, 54, 59, 63, 67, 72, 76, 82, 87, 92, 97, 104, 59, 56, 54, 54, 53, 58, + 62, 67, 71, 76, 80, 87, 92, 98, 103, 110, 65, 62, 59, 59, 58, 63, 67, + 71, 75, 81, 85, 92, 98, 105, 111, 118, 71, 68, 65, 64, 63, 68, 72, 76, + 80, 85, 90, 97, 103, 111, 117, 125, 80, 76, 72, 71, 69, 74, 78, 82, 86, + 91, 96, 104, 110, 118, 125, 134], + /* Size 32x32 */ + [32, 31, 31, 31, 31, 31, 31, 32, 32, 32, 34, 34, 35, 36, 38, 39, 41, 44, + 45, 48, 48, 53, 54, 57, 59, 62, 65, 67, 71, 72, 80, 80, 31, 31, 32, 32, + 32, 32, 32, 32, 32, 32, 34, 34, 35, 35, 37, 38, 40, 42, 43, 46, 46, 51, + 52, 55, 56, 59, 62, 64, 68, 69, 76, 76, 31, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 34, 34, 35, 35, 37, 38, 40, 42, 43, 46, 46, 51, 51, 55, 56, 59, + 62, 64, 68, 69, 76, 76, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, + 34, 34, 36, 38, 39, 41, 42, 45, 45, 49, 50, 53, 54, 57, 60, 62, 66, 66, + 73, 73, 31, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 34, 34, 36, 37, + 38, 41, 41, 44, 44, 49, 49, 52, 54, 56, 59, 61, 65, 65, 72, 72, 31, 32, + 32, 32, 32, 32, 33, 33, 33, 33, 34, 34, 35, 35, 37, 38, 39, 41, 42, 45, + 45, 49, 49, 52, 54, 56, 59, 61, 64, 65, 72, 72, 31, 32, 32, 32, 32, 33, + 33, 33, 34, 34, 35, 35, 36, 36, 38, 39, 40, 42, 42, 45, 45, 49, 50, 52, + 54, 56, 59, 60, 64, 65, 71, 71, 32, 32, 32, 32, 32, 33, 33, 34, 34, 34, + 35, 35, 36, 37, 38, 39, 40, 42, 43, 45, 45, 49, 49, 52, 54, 56, 59, 60, + 64, 64, 70, 70, 32, 32, 32, 32, 32, 33, 34, 34, 35, 35, 37, 37, 38, 38, + 39, 40, 41, 42, 43, 46, 46, 49, 49, 52, 53, 55, 58, 59, 63, 63, 69, 69, + 32, 32, 32, 32, 33, 33, 34, 34, 35, 35, 37, 37, 38, 38, 40, 41, 41, 43, + 43, 46, 46, 49, 50, 52, 54, 56, 58, 60, 63, 64, 70, 70, 34, 34, 34, 33, + 33, 34, 35, 35, 37, 37, 39, 39, 42, 42, 44, 45, 46, 47, 48, 51, 51, 54, + 54, 57, 58, 60, 63, 64, 68, 68, 74, 74, 34, 34, 34, 33, 33, 34, 35, 35, + 37, 37, 39, 39, 42, 42, 44, 45, 46, 47, 48, 51, 51, 54, 54, 57, 58, 60, + 63, 64, 68, 68, 74, 74, 35, 35, 35, 34, 34, 35, 36, 36, 38, 38, 42, 42, + 46, 47, 48, 49, 50, 52, 53, 55, 55, 58, 59, 61, 62, 64, 67, 68, 72, 72, + 78, 78, 36, 35, 35, 34, 34, 35, 36, 37, 38, 38, 42, 42, 47, 48, 50, 50, + 52, 54, 54, 57, 57, 59, 60, 62, 64, 66, 68, 69, 73, 73, 79, 79, 38, 37, + 37, 36, 36, 37, 38, 38, 39, 40, 44, 44, 48, 50, 51, 52, 54, 56, 57, 59, + 59, 62, 63, 65, 67, 69, 71, 72, 76, 76, 82, 82, 39, 38, 38, 38, 37, 38, + 39, 39, 40, 41, 45, 45, 49, 50, 52, 54, 55, 58, 58, 61, 61, 64, 65, 67, + 69, 71, 73, 74, 78, 78, 84, 84, 41, 40, 40, 39, 38, 39, 40, 40, 41, 41, + 46, 46, 50, 52, 54, 55, 57, 60, 60, 63, 63, 67, 67, 70, 71, 73, 75, 77, + 80, 81, 86, 86, 44, 42, 42, 41, 41, 41, 42, 42, 42, 43, 47, 47, 52, 54, + 56, 58, 60, 63, 64, 67, 67, 71, 71, 74, 75, 77, 79, 81, 84, 85, 90, 90, + 45, 43, 43, 42, 41, 42, 42, 43, 43, 43, 48, 48, 53, 54, 57, 58, 60, 64, + 65, 68, 68, 72, 72, 75, 76, 78, 81, 82, 85, 86, 91, 91, 48, 46, 46, 45, + 44, 45, 45, 45, 46, 46, 51, 51, 55, 57, 59, 61, 63, 67, 68, 71, 71, 75, + 76, 79, 80, 83, 85, 87, 90, 91, 96, 96, 48, 46, 46, 45, 44, 45, 45, 45, + 46, 46, 51, 51, 55, 57, 59, 61, 63, 67, 68, 71, 71, 75, 76, 79, 80, 83, + 85, 87, 90, 91, 96, 96, 53, 51, 51, 49, 49, 49, 49, 49, 49, 49, 54, 54, + 58, 59, 62, 64, 67, 71, 72, 75, 75, 81, 81, 85, 86, 89, 91, 93, 97, 97, + 103, 103, 54, 52, 51, 50, 49, 49, 50, 49, 49, 50, 54, 54, 59, 60, 63, + 65, 67, 71, 72, 76, 76, 81, 82, 85, 87, 89, 92, 94, 97, 98, 104, 104, + 57, 55, 55, 53, 52, 52, 52, 52, 52, 52, 57, 57, 61, 62, 65, 67, 70, 74, + 75, 79, 79, 85, 85, 89, 90, 93, 96, 98, 102, 102, 108, 108, 59, 56, 56, + 54, 54, 54, 54, 54, 53, 54, 58, 58, 62, 64, 67, 69, 71, 75, 76, 80, 80, + 86, 87, 90, 92, 95, 98, 99, 103, 104, 110, 110, 62, 59, 59, 57, 56, 56, + 56, 56, 55, 56, 60, 60, 64, 66, 69, 71, 73, 77, 78, 83, 83, 89, 89, 93, + 95, 98, 101, 103, 107, 108, 114, 114, 65, 62, 62, 60, 59, 59, 59, 59, + 58, 58, 63, 63, 67, 68, 71, 73, 75, 79, 81, 85, 85, 91, 92, 96, 98, 101, + 105, 106, 111, 111, 118, 118, 67, 64, 64, 62, 61, 61, 60, 60, 59, 60, + 64, 64, 68, 69, 72, 74, 77, 81, 82, 87, 87, 93, 94, 98, 99, 103, 106, + 108, 113, 113, 120, 120, 71, 68, 68, 66, 65, 64, 64, 64, 63, 63, 68, 68, + 72, 73, 76, 78, 80, 84, 85, 90, 90, 97, 97, 102, 103, 107, 111, 113, + 117, 118, 125, 125, 72, 69, 69, 66, 65, 65, 65, 64, 63, 64, 68, 68, 72, + 73, 76, 78, 81, 85, 86, 91, 91, 97, 98, 102, 104, 108, 111, 113, 118, + 119, 126, 126, 80, 76, 76, 73, 72, 72, 71, 70, 69, 70, 74, 74, 78, 79, + 82, 84, 86, 90, 91, 96, 96, 103, 104, 108, 110, 114, 118, 120, 125, 126, + 134, 134, 80, 76, 76, 73, 72, 72, 71, 70, 69, 70, 74, 74, 78, 79, 82, + 84, 86, 90, 91, 96, 96, 103, 104, 108, 110, 114, 118, 120, 125, 126, + 134, 134], + /* Size 4x8 */ + [32, 34, 43, 62, 32, 34, 42, 59, 33, 37, 44, 58, 35, 43, 54, 68, 41, 48, + 64, 79, 49, 54, 71, 91, 57, 60, 78, 101, 66, 68, 86, 111], + /* Size 8x4 */ + [32, 32, 33, 35, 41, 49, 57, 66, 34, 34, 37, 43, 48, 54, 60, 68, 43, 42, + 44, 54, 64, 71, 78, 86, 62, 59, 58, 68, 79, 91, 101, 111], + /* Size 8x16 */ + [32, 31, 32, 36, 44, 53, 62, 73, 31, 32, 32, 35, 42, 51, 59, 69, 31, 32, + 33, 34, 41, 49, 57, 66, 32, 32, 34, 36, 42, 50, 57, 65, 32, 33, 35, 38, + 42, 49, 56, 64, 34, 34, 37, 42, 48, 54, 61, 69, 35, 34, 38, 47, 52, 59, + 65, 73, 38, 36, 40, 49, 56, 63, 69, 77, 41, 39, 41, 51, 60, 67, 74, 81, + 44, 42, 43, 54, 64, 72, 79, 86, 48, 45, 46, 56, 67, 76, 83, 91, 53, 49, + 50, 60, 71, 82, 90, 99, 58, 54, 54, 63, 75, 87, 95, 105, 65, 60, 58, 68, + 79, 92, 102, 112, 71, 65, 63, 73, 84, 97, 108, 119, 79, 72, 70, 79, 90, + 104, 115, 127], + /* Size 16x8 */ + [32, 31, 31, 32, 32, 34, 35, 38, 41, 44, 48, 53, 58, 65, 71, 79, 31, 32, + 32, 32, 33, 34, 34, 36, 39, 42, 45, 49, 54, 60, 65, 72, 32, 32, 33, 34, + 35, 37, 38, 40, 41, 43, 46, 50, 54, 58, 63, 70, 36, 35, 34, 36, 38, 42, + 47, 49, 51, 54, 56, 60, 63, 68, 73, 79, 44, 42, 41, 42, 42, 48, 52, 56, + 60, 64, 67, 71, 75, 79, 84, 90, 53, 51, 49, 50, 49, 54, 59, 63, 67, 72, + 76, 82, 87, 92, 97, 104, 62, 59, 57, 57, 56, 61, 65, 69, 74, 79, 83, 90, + 95, 102, 108, 115, 73, 69, 66, 65, 64, 69, 73, 77, 81, 86, 91, 99, 105, + 112, 119, 127], + /* Size 16x32 */ + [32, 31, 31, 32, 32, 34, 36, 38, 44, 44, 53, 53, 62, 65, 73, 79, 31, 32, + 32, 32, 32, 34, 35, 37, 42, 43, 51, 51, 60, 62, 70, 75, 31, 32, 32, 32, + 32, 34, 35, 37, 42, 43, 51, 51, 59, 62, 69, 75, 31, 32, 32, 32, 32, 33, + 35, 36, 41, 42, 50, 50, 58, 60, 67, 73, 31, 32, 32, 32, 33, 33, 34, 36, + 41, 41, 49, 49, 57, 59, 66, 72, 31, 32, 32, 33, 33, 34, 35, 37, 41, 42, + 49, 49, 57, 59, 66, 71, 32, 32, 32, 33, 34, 35, 36, 38, 42, 43, 50, 50, + 57, 59, 65, 71, 32, 32, 32, 34, 34, 35, 37, 38, 42, 43, 49, 49, 56, 59, + 65, 70, 32, 32, 33, 34, 35, 37, 38, 39, 42, 43, 49, 49, 56, 58, 64, 69, + 32, 33, 33, 34, 35, 37, 39, 40, 43, 44, 50, 50, 56, 58, 64, 69, 34, 34, + 34, 36, 37, 39, 42, 44, 48, 48, 54, 54, 61, 63, 69, 73, 34, 34, 34, 36, + 37, 39, 42, 44, 48, 48, 54, 54, 61, 63, 69, 73, 35, 34, 34, 37, 38, 42, + 47, 48, 52, 53, 59, 59, 65, 67, 73, 77, 36, 35, 34, 37, 38, 43, 48, 49, + 54, 54, 60, 60, 66, 68, 74, 78, 38, 36, 36, 38, 40, 44, 49, 51, 56, 57, + 63, 63, 69, 71, 77, 81, 39, 38, 37, 40, 40, 45, 50, 52, 58, 58, 65, 65, + 71, 73, 79, 84, 41, 39, 39, 41, 41, 46, 51, 54, 60, 60, 67, 67, 74, 76, + 81, 86, 44, 41, 41, 42, 43, 48, 53, 56, 63, 64, 71, 71, 78, 79, 85, 90, + 44, 42, 42, 43, 43, 48, 54, 56, 64, 64, 72, 72, 79, 81, 86, 91, 48, 45, + 45, 46, 46, 51, 56, 59, 67, 67, 76, 76, 83, 85, 91, 96, 48, 45, 45, 46, + 46, 51, 56, 59, 67, 67, 76, 76, 83, 85, 91, 96, 53, 49, 49, 49, 49, 54, + 59, 62, 71, 71, 81, 81, 89, 91, 98, 103, 53, 50, 49, 50, 50, 54, 60, 63, + 71, 72, 82, 82, 90, 92, 99, 103, 57, 53, 52, 52, 52, 57, 62, 65, 74, 75, + 85, 85, 94, 96, 103, 108, 58, 54, 54, 54, 54, 58, 63, 67, 75, 76, 87, + 87, 95, 98, 105, 110, 61, 57, 57, 56, 56, 60, 66, 69, 77, 78, 89, 89, + 98, 101, 108, 114, 65, 60, 60, 59, 58, 63, 68, 71, 79, 80, 92, 92, 102, + 105, 112, 118, 67, 62, 61, 60, 60, 64, 69, 72, 81, 82, 94, 94, 103, 106, + 114, 120, 71, 66, 65, 64, 63, 68, 73, 76, 84, 85, 97, 97, 108, 111, 119, + 125, 72, 66, 66, 64, 64, 68, 73, 76, 85, 86, 98, 98, 108, 111, 119, 125, + 79, 73, 72, 71, 70, 74, 79, 82, 90, 91, 104, 104, 115, 118, 127, 133, + 79, 73, 72, 71, 70, 74, 79, 82, 90, 91, 104, 104, 115, 118, 127, 133], + /* Size 32x16 */ + [32, 31, 31, 31, 31, 31, 32, 32, 32, 32, 34, 34, 35, 36, 38, 39, 41, 44, + 44, 48, 48, 53, 53, 57, 58, 61, 65, 67, 71, 72, 79, 79, 31, 32, 32, 32, + 32, 32, 32, 32, 32, 33, 34, 34, 34, 35, 36, 38, 39, 41, 42, 45, 45, 49, + 50, 53, 54, 57, 60, 62, 66, 66, 73, 73, 31, 32, 32, 32, 32, 32, 32, 32, + 33, 33, 34, 34, 34, 34, 36, 37, 39, 41, 42, 45, 45, 49, 49, 52, 54, 57, + 60, 61, 65, 66, 72, 72, 32, 32, 32, 32, 32, 33, 33, 34, 34, 34, 36, 36, + 37, 37, 38, 40, 41, 42, 43, 46, 46, 49, 50, 52, 54, 56, 59, 60, 64, 64, + 71, 71, 32, 32, 32, 32, 33, 33, 34, 34, 35, 35, 37, 37, 38, 38, 40, 40, + 41, 43, 43, 46, 46, 49, 50, 52, 54, 56, 58, 60, 63, 64, 70, 70, 34, 34, + 34, 33, 33, 34, 35, 35, 37, 37, 39, 39, 42, 43, 44, 45, 46, 48, 48, 51, + 51, 54, 54, 57, 58, 60, 63, 64, 68, 68, 74, 74, 36, 35, 35, 35, 34, 35, + 36, 37, 38, 39, 42, 42, 47, 48, 49, 50, 51, 53, 54, 56, 56, 59, 60, 62, + 63, 66, 68, 69, 73, 73, 79, 79, 38, 37, 37, 36, 36, 37, 38, 38, 39, 40, + 44, 44, 48, 49, 51, 52, 54, 56, 56, 59, 59, 62, 63, 65, 67, 69, 71, 72, + 76, 76, 82, 82, 44, 42, 42, 41, 41, 41, 42, 42, 42, 43, 48, 48, 52, 54, + 56, 58, 60, 63, 64, 67, 67, 71, 71, 74, 75, 77, 79, 81, 84, 85, 90, 90, + 44, 43, 43, 42, 41, 42, 43, 43, 43, 44, 48, 48, 53, 54, 57, 58, 60, 64, + 64, 67, 67, 71, 72, 75, 76, 78, 80, 82, 85, 86, 91, 91, 53, 51, 51, 50, + 49, 49, 50, 49, 49, 50, 54, 54, 59, 60, 63, 65, 67, 71, 72, 76, 76, 81, + 82, 85, 87, 89, 92, 94, 97, 98, 104, 104, 53, 51, 51, 50, 49, 49, 50, + 49, 49, 50, 54, 54, 59, 60, 63, 65, 67, 71, 72, 76, 76, 81, 82, 85, 87, + 89, 92, 94, 97, 98, 104, 104, 62, 60, 59, 58, 57, 57, 57, 56, 56, 56, + 61, 61, 65, 66, 69, 71, 74, 78, 79, 83, 83, 89, 90, 94, 95, 98, 102, + 103, 108, 108, 115, 115, 65, 62, 62, 60, 59, 59, 59, 59, 58, 58, 63, 63, + 67, 68, 71, 73, 76, 79, 81, 85, 85, 91, 92, 96, 98, 101, 105, 106, 111, + 111, 118, 118, 73, 70, 69, 67, 66, 66, 65, 65, 64, 64, 69, 69, 73, 74, + 77, 79, 81, 85, 86, 91, 91, 98, 99, 103, 105, 108, 112, 114, 119, 119, + 127, 127, 79, 75, 75, 73, 72, 71, 71, 70, 69, 69, 73, 73, 77, 78, 81, + 84, 86, 90, 91, 96, 96, 103, 103, 108, 110, 114, 118, 120, 125, 125, + 133, 133], + /* Size 4x16 */ + [31, 34, 44, 65, 32, 34, 43, 62, 32, 33, 41, 59, 32, 35, 43, 59, 32, 37, + 43, 58, 34, 39, 48, 63, 34, 42, 53, 67, 36, 44, 57, 71, 39, 46, 60, 76, + 42, 48, 64, 81, 45, 51, 67, 85, 50, 54, 72, 92, 54, 58, 76, 98, 60, 63, + 80, 105, 66, 68, 85, 111, 73, 74, 91, 118], + /* Size 16x4 */ + [31, 32, 32, 32, 32, 34, 34, 36, 39, 42, 45, 50, 54, 60, 66, 73, 34, 34, + 33, 35, 37, 39, 42, 44, 46, 48, 51, 54, 58, 63, 68, 74, 44, 43, 41, 43, + 43, 48, 53, 57, 60, 64, 67, 72, 76, 80, 85, 91, 65, 62, 59, 59, 58, 63, + 67, 71, 76, 81, 85, 92, 98, 105, 111, 118], + /* Size 8x32 */ + [32, 31, 32, 36, 44, 53, 62, 73, 31, 32, 32, 35, 42, 51, 60, 70, 31, 32, + 32, 35, 42, 51, 59, 69, 31, 32, 32, 35, 41, 50, 58, 67, 31, 32, 33, 34, + 41, 49, 57, 66, 31, 32, 33, 35, 41, 49, 57, 66, 32, 32, 34, 36, 42, 50, + 57, 65, 32, 32, 34, 37, 42, 49, 56, 65, 32, 33, 35, 38, 42, 49, 56, 64, + 32, 33, 35, 39, 43, 50, 56, 64, 34, 34, 37, 42, 48, 54, 61, 69, 34, 34, + 37, 42, 48, 54, 61, 69, 35, 34, 38, 47, 52, 59, 65, 73, 36, 34, 38, 48, + 54, 60, 66, 74, 38, 36, 40, 49, 56, 63, 69, 77, 39, 37, 40, 50, 58, 65, + 71, 79, 41, 39, 41, 51, 60, 67, 74, 81, 44, 41, 43, 53, 63, 71, 78, 85, + 44, 42, 43, 54, 64, 72, 79, 86, 48, 45, 46, 56, 67, 76, 83, 91, 48, 45, + 46, 56, 67, 76, 83, 91, 53, 49, 49, 59, 71, 81, 89, 98, 53, 49, 50, 60, + 71, 82, 90, 99, 57, 52, 52, 62, 74, 85, 94, 103, 58, 54, 54, 63, 75, 87, + 95, 105, 61, 57, 56, 66, 77, 89, 98, 108, 65, 60, 58, 68, 79, 92, 102, + 112, 67, 61, 60, 69, 81, 94, 103, 114, 71, 65, 63, 73, 84, 97, 108, 119, + 72, 66, 64, 73, 85, 98, 108, 119, 79, 72, 70, 79, 90, 104, 115, 127, 79, + 72, 70, 79, 90, 104, 115, 127], + /* Size 32x8 */ + [32, 31, 31, 31, 31, 31, 32, 32, 32, 32, 34, 34, 35, 36, 38, 39, 41, 44, + 44, 48, 48, 53, 53, 57, 58, 61, 65, 67, 71, 72, 79, 79, 31, 32, 32, 32, + 32, 32, 32, 32, 33, 33, 34, 34, 34, 34, 36, 37, 39, 41, 42, 45, 45, 49, + 49, 52, 54, 57, 60, 61, 65, 66, 72, 72, 32, 32, 32, 32, 33, 33, 34, 34, + 35, 35, 37, 37, 38, 38, 40, 40, 41, 43, 43, 46, 46, 49, 50, 52, 54, 56, + 58, 60, 63, 64, 70, 70, 36, 35, 35, 35, 34, 35, 36, 37, 38, 39, 42, 42, + 47, 48, 49, 50, 51, 53, 54, 56, 56, 59, 60, 62, 63, 66, 68, 69, 73, 73, + 79, 79, 44, 42, 42, 41, 41, 41, 42, 42, 42, 43, 48, 48, 52, 54, 56, 58, + 60, 63, 64, 67, 67, 71, 71, 74, 75, 77, 79, 81, 84, 85, 90, 90, 53, 51, + 51, 50, 49, 49, 50, 49, 49, 50, 54, 54, 59, 60, 63, 65, 67, 71, 72, 76, + 76, 81, 82, 85, 87, 89, 92, 94, 97, 98, 104, 104, 62, 60, 59, 58, 57, + 57, 57, 56, 56, 56, 61, 61, 65, 66, 69, 71, 74, 78, 79, 83, 83, 89, 90, + 94, 95, 98, 102, 103, 108, 108, 115, 115, 73, 70, 69, 67, 66, 66, 65, + 65, 64, 64, 69, 69, 73, 74, 77, 79, 81, 85, 86, 91, 91, 98, 99, 103, + 105, 108, 112, 114, 119, 119, 127, 127]], + [ /* Chroma */ + /* Size 4x4 */ + [31, 42, 47, 53, 42, 48, 50, 54, 47, 50, 61, 67, 53, 54, 67, 78], + /* Size 8x8 */ + [31, 32, 38, 48, 47, 50, 53, 57, 32, 35, 42, 47, 45, 47, 50, 54, 38, 42, + 47, 48, 45, 47, 49, 52, 48, 47, 48, 53, 53, 54, 56, 58, 47, 45, 45, 53, + 58, 61, 63, 65, 50, 47, 47, 54, 61, 66, 69, 72, 53, 50, 49, 56, 63, 69, + 73, 77, 57, 54, 52, 58, 65, 72, 77, 82], + /* Size 16x16 */ + [32, 31, 30, 33, 36, 41, 47, 49, 49, 49, 50, 52, 54, 57, 60, 63, 31, 31, + 31, 34, 38, 42, 46, 47, 47, 47, 48, 50, 52, 54, 57, 60, 30, 31, 32, 35, + 40, 42, 45, 46, 45, 45, 46, 47, 49, 52, 54, 57, 33, 34, 35, 39, 43, 45, + 47, 46, 46, 45, 46, 47, 49, 51, 53, 56, 36, 38, 40, 43, 47, 47, 47, 47, + 46, 45, 46, 47, 48, 50, 52, 54, 41, 42, 42, 45, 47, 48, 50, 50, 49, 49, + 50, 50, 52, 53, 55, 57, 47, 46, 45, 47, 47, 50, 52, 52, 52, 52, 53, 53, + 55, 56, 58, 60, 49, 47, 46, 46, 47, 50, 52, 53, 54, 55, 55, 56, 57, 58, + 60, 62, 49, 47, 45, 46, 46, 49, 52, 54, 55, 57, 58, 59, 60, 61, 63, 65, + 49, 47, 45, 45, 45, 49, 52, 55, 57, 59, 60, 61, 63, 64, 66, 68, 50, 48, + 46, 46, 46, 50, 53, 55, 58, 60, 61, 63, 65, 67, 68, 71, 52, 50, 47, 47, + 47, 50, 53, 56, 59, 61, 63, 66, 68, 70, 72, 75, 54, 52, 49, 49, 48, 52, + 55, 57, 60, 63, 65, 68, 71, 73, 75, 78, 57, 54, 52, 51, 50, 53, 56, 58, + 61, 64, 67, 70, 73, 76, 79, 82, 60, 57, 54, 53, 52, 55, 58, 60, 63, 66, + 68, 72, 75, 79, 82, 85, 63, 60, 57, 56, 54, 57, 60, 62, 65, 68, 71, 75, + 78, 82, 85, 89], + /* Size 32x32 */ + [32, 31, 31, 30, 30, 32, 33, 34, 36, 37, 41, 41, 47, 49, 49, 48, 49, 49, + 49, 50, 50, 52, 52, 54, 54, 56, 57, 58, 60, 60, 63, 63, 31, 31, 31, 31, + 31, 32, 34, 35, 38, 38, 42, 42, 46, 48, 47, 47, 47, 47, 47, 48, 48, 50, + 50, 51, 52, 53, 54, 55, 57, 57, 60, 60, 31, 31, 31, 31, 31, 33, 34, 35, + 38, 39, 42, 42, 46, 47, 47, 47, 47, 47, 47, 48, 48, 49, 50, 51, 52, 53, + 54, 55, 57, 57, 60, 60, 30, 31, 31, 31, 31, 33, 35, 36, 39, 40, 42, 42, + 46, 47, 46, 46, 46, 45, 46, 47, 47, 48, 48, 50, 50, 51, 52, 53, 55, 55, + 58, 58, 30, 31, 31, 31, 32, 33, 35, 36, 40, 40, 42, 42, 45, 46, 46, 45, + 45, 45, 45, 46, 46, 47, 47, 49, 49, 51, 52, 52, 54, 54, 57, 57, 32, 32, + 33, 33, 33, 35, 37, 38, 41, 42, 43, 43, 46, 47, 46, 46, 45, 45, 45, 46, + 46, 47, 47, 49, 49, 50, 51, 52, 54, 54, 57, 57, 33, 34, 34, 35, 35, 37, + 39, 40, 43, 43, 45, 45, 47, 47, 46, 46, 46, 45, 45, 46, 46, 47, 47, 49, + 49, 50, 51, 52, 53, 54, 56, 56, 34, 35, 35, 36, 36, 38, 40, 41, 44, 44, + 45, 45, 47, 47, 47, 46, 46, 45, 45, 46, 46, 47, 47, 48, 49, 50, 51, 51, + 53, 53, 55, 55, 36, 38, 38, 39, 40, 41, 43, 44, 47, 47, 47, 47, 47, 48, + 47, 46, 46, 45, 45, 46, 46, 46, 47, 48, 48, 49, 50, 50, 52, 52, 54, 54, + 37, 38, 39, 40, 40, 42, 43, 44, 47, 47, 47, 47, 48, 48, 47, 47, 46, 45, + 46, 46, 46, 47, 47, 48, 48, 49, 50, 51, 52, 52, 55, 55, 41, 42, 42, 42, + 42, 43, 45, 45, 47, 47, 48, 48, 50, 50, 50, 49, 49, 49, 49, 50, 50, 50, + 50, 51, 52, 52, 53, 54, 55, 55, 57, 57, 41, 42, 42, 42, 42, 43, 45, 45, + 47, 47, 48, 48, 50, 50, 50, 49, 49, 49, 49, 50, 50, 50, 50, 51, 52, 52, + 53, 54, 55, 55, 57, 57, 47, 46, 46, 46, 45, 46, 47, 47, 47, 48, 50, 50, + 52, 52, 52, 52, 52, 52, 52, 53, 53, 53, 53, 54, 55, 55, 56, 56, 58, 58, + 60, 60, 49, 48, 47, 47, 46, 47, 47, 47, 48, 48, 50, 50, 52, 53, 53, 53, + 53, 53, 53, 54, 54, 54, 54, 55, 55, 56, 56, 57, 58, 58, 60, 60, 49, 47, + 47, 46, 46, 46, 46, 47, 47, 47, 50, 50, 52, 53, 53, 54, 54, 55, 55, 55, + 55, 56, 56, 57, 57, 58, 58, 59, 60, 60, 62, 62, 48, 47, 47, 46, 45, 46, + 46, 46, 46, 47, 49, 49, 52, 53, 54, 54, 55, 55, 56, 56, 56, 57, 57, 58, + 58, 59, 60, 60, 61, 62, 63, 63, 49, 47, 47, 46, 45, 45, 46, 46, 46, 46, + 49, 49, 52, 53, 54, 55, 55, 57, 57, 58, 58, 59, 59, 60, 60, 61, 61, 62, + 63, 63, 65, 65, 49, 47, 47, 45, 45, 45, 45, 45, 45, 45, 49, 49, 52, 53, + 55, 55, 57, 58, 59, 60, 60, 61, 61, 62, 62, 63, 63, 64, 65, 65, 67, 67, + 49, 47, 47, 46, 45, 45, 45, 45, 45, 46, 49, 49, 52, 53, 55, 56, 57, 59, + 59, 60, 60, 61, 61, 62, 63, 63, 64, 65, 66, 66, 68, 68, 50, 48, 48, 47, + 46, 46, 46, 46, 46, 46, 50, 50, 53, 54, 55, 56, 58, 60, 60, 61, 61, 63, + 63, 65, 65, 66, 67, 67, 68, 69, 71, 71, 50, 48, 48, 47, 46, 46, 46, 46, + 46, 46, 50, 50, 53, 54, 55, 56, 58, 60, 60, 61, 61, 63, 63, 65, 65, 66, + 67, 67, 68, 69, 71, 71, 52, 50, 49, 48, 47, 47, 47, 47, 46, 47, 50, 50, + 53, 54, 56, 57, 59, 61, 61, 63, 63, 66, 66, 67, 68, 69, 70, 71, 72, 72, + 74, 74, 52, 50, 50, 48, 47, 47, 47, 47, 47, 47, 50, 50, 53, 54, 56, 57, + 59, 61, 61, 63, 63, 66, 66, 68, 68, 69, 70, 71, 72, 73, 75, 75, 54, 51, + 51, 50, 49, 49, 49, 48, 48, 48, 51, 51, 54, 55, 57, 58, 60, 62, 62, 65, + 65, 67, 68, 69, 70, 71, 72, 73, 74, 75, 77, 77, 54, 52, 52, 50, 49, 49, + 49, 49, 48, 48, 52, 52, 55, 55, 57, 58, 60, 62, 63, 65, 65, 68, 68, 70, + 71, 72, 73, 74, 75, 76, 78, 78, 56, 53, 53, 51, 51, 50, 50, 50, 49, 49, + 52, 52, 55, 56, 58, 59, 61, 63, 63, 66, 66, 69, 69, 71, 72, 73, 75, 75, + 77, 77, 80, 80, 57, 54, 54, 52, 52, 51, 51, 51, 50, 50, 53, 53, 56, 56, + 58, 60, 61, 63, 64, 67, 67, 70, 70, 72, 73, 75, 76, 77, 79, 79, 82, 82, + 58, 55, 55, 53, 52, 52, 52, 51, 50, 51, 54, 54, 56, 57, 59, 60, 62, 64, + 65, 67, 67, 71, 71, 73, 74, 75, 77, 78, 80, 80, 83, 83, 60, 57, 57, 55, + 54, 54, 53, 53, 52, 52, 55, 55, 58, 58, 60, 61, 63, 65, 66, 68, 68, 72, + 72, 74, 75, 77, 79, 80, 82, 82, 85, 85, 60, 57, 57, 55, 54, 54, 54, 53, + 52, 52, 55, 55, 58, 58, 60, 62, 63, 65, 66, 69, 69, 72, 73, 75, 76, 77, + 79, 80, 82, 82, 85, 85, 63, 60, 60, 58, 57, 57, 56, 55, 54, 55, 57, 57, + 60, 60, 62, 63, 65, 67, 68, 71, 71, 74, 75, 77, 78, 80, 82, 83, 85, 85, + 89, 89, 63, 60, 60, 58, 57, 57, 56, 55, 54, 55, 57, 57, 60, 60, 62, 63, + 65, 67, 68, 71, 71, 74, 75, 77, 78, 80, 82, 83, 85, 85, 89, 89], + /* Size 4x8 */ + [31, 42, 47, 54, 33, 44, 45, 51, 40, 47, 46, 50, 47, 50, 54, 57, 45, 49, + 59, 64, 48, 50, 61, 70, 51, 52, 63, 75, 55, 55, 66, 79], + /* Size 8x4 */ + [31, 33, 40, 47, 45, 48, 51, 55, 42, 44, 47, 50, 49, 50, 52, 55, 47, 45, + 46, 54, 59, 61, 63, 66, 54, 51, 50, 57, 64, 70, 75, 79], + /* Size 8x16 */ + [32, 31, 37, 48, 49, 52, 56, 61, 31, 31, 38, 47, 47, 50, 53, 57, 30, 32, + 40, 46, 45, 48, 51, 55, 33, 36, 43, 47, 46, 47, 50, 54, 37, 40, 47, 47, + 45, 47, 49, 52, 42, 43, 47, 50, 49, 50, 53, 56, 47, 46, 48, 52, 53, 53, + 55, 58, 48, 46, 47, 53, 55, 56, 58, 61, 48, 45, 46, 53, 57, 59, 61, 63, + 49, 45, 46, 53, 58, 62, 64, 66, 50, 46, 46, 54, 59, 64, 66, 69, 52, 48, + 47, 54, 61, 66, 70, 73, 54, 50, 49, 55, 62, 68, 72, 76, 57, 52, 50, 56, + 64, 70, 75, 79, 60, 54, 52, 58, 65, 72, 77, 82, 63, 57, 55, 60, 67, 75, + 80, 86], + /* Size 16x8 */ + [32, 31, 30, 33, 37, 42, 47, 48, 48, 49, 50, 52, 54, 57, 60, 63, 31, 31, + 32, 36, 40, 43, 46, 46, 45, 45, 46, 48, 50, 52, 54, 57, 37, 38, 40, 43, + 47, 47, 48, 47, 46, 46, 46, 47, 49, 50, 52, 55, 48, 47, 46, 47, 47, 50, + 52, 53, 53, 53, 54, 54, 55, 56, 58, 60, 49, 47, 45, 46, 45, 49, 53, 55, + 57, 58, 59, 61, 62, 64, 65, 67, 52, 50, 48, 47, 47, 50, 53, 56, 59, 62, + 64, 66, 68, 70, 72, 75, 56, 53, 51, 50, 49, 53, 55, 58, 61, 64, 66, 70, + 72, 75, 77, 80, 61, 57, 55, 54, 52, 56, 58, 61, 63, 66, 69, 73, 76, 79, + 82, 86], + /* Size 16x32 */ + [32, 31, 31, 35, 37, 42, 48, 48, 49, 49, 52, 52, 56, 57, 61, 63, 31, 31, + 31, 36, 38, 42, 47, 47, 47, 47, 50, 50, 54, 54, 58, 60, 31, 31, 31, 36, + 38, 42, 47, 47, 47, 47, 50, 50, 53, 54, 57, 60, 30, 32, 32, 37, 39, 42, + 46, 46, 46, 46, 48, 48, 52, 52, 56, 58, 30, 32, 32, 37, 40, 42, 46, 46, + 45, 45, 48, 48, 51, 52, 55, 57, 32, 33, 34, 39, 41, 44, 46, 46, 45, 45, + 48, 48, 51, 51, 54, 57, 33, 35, 36, 40, 43, 45, 47, 46, 46, 46, 47, 47, + 50, 51, 54, 56, 34, 37, 37, 42, 44, 45, 47, 47, 45, 46, 47, 47, 50, 51, + 53, 55, 37, 40, 40, 45, 47, 47, 47, 47, 45, 46, 47, 47, 49, 50, 52, 54, + 37, 40, 40, 45, 47, 47, 48, 47, 46, 46, 47, 47, 49, 50, 53, 55, 42, 43, + 43, 46, 47, 48, 50, 50, 49, 49, 50, 50, 53, 53, 56, 57, 42, 43, 43, 46, + 47, 48, 50, 50, 49, 49, 50, 50, 53, 53, 56, 57, 47, 46, 46, 47, 48, 50, + 52, 52, 53, 53, 53, 53, 55, 56, 58, 60, 49, 47, 46, 47, 48, 50, 53, 53, + 53, 54, 54, 54, 56, 57, 59, 60, 48, 46, 46, 47, 47, 50, 53, 53, 55, 55, + 56, 56, 58, 58, 61, 62, 48, 46, 46, 46, 47, 50, 53, 54, 56, 56, 57, 57, + 59, 60, 62, 64, 48, 46, 45, 46, 46, 49, 53, 54, 57, 57, 59, 59, 61, 61, + 63, 65, 49, 45, 45, 45, 46, 49, 53, 55, 58, 59, 61, 61, 63, 64, 66, 67, + 49, 46, 45, 46, 46, 49, 53, 55, 58, 59, 62, 62, 64, 64, 66, 68, 50, 47, + 46, 46, 46, 50, 54, 55, 59, 60, 64, 64, 66, 67, 69, 71, 50, 47, 46, 46, + 46, 50, 54, 55, 59, 60, 64, 64, 66, 67, 69, 71, 52, 48, 48, 47, 47, 50, + 54, 56, 61, 61, 66, 66, 69, 70, 72, 74, 52, 48, 48, 47, 47, 50, 54, 56, + 61, 61, 66, 66, 70, 71, 73, 75, 53, 50, 49, 48, 48, 51, 55, 57, 62, 62, + 68, 68, 71, 72, 75, 77, 54, 50, 50, 49, 49, 52, 55, 57, 62, 63, 68, 68, + 72, 73, 76, 78, 55, 51, 51, 50, 49, 52, 56, 58, 63, 63, 69, 69, 74, 75, + 78, 80, 57, 52, 52, 51, 50, 53, 56, 58, 64, 64, 70, 70, 75, 76, 79, 82, + 58, 53, 53, 51, 51, 54, 57, 59, 64, 65, 71, 71, 76, 77, 80, 83, 60, 55, + 54, 53, 52, 55, 58, 60, 65, 66, 72, 72, 77, 79, 82, 85, 60, 55, 55, 53, + 53, 55, 59, 60, 65, 66, 73, 73, 78, 79, 83, 85, 63, 58, 57, 56, 55, 58, + 60, 62, 67, 68, 75, 75, 80, 82, 86, 89, 63, 58, 57, 56, 55, 58, 60, 62, + 67, 68, 75, 75, 80, 82, 86, 89], + /* Size 32x16 */ + [32, 31, 31, 30, 30, 32, 33, 34, 37, 37, 42, 42, 47, 49, 48, 48, 48, 49, + 49, 50, 50, 52, 52, 53, 54, 55, 57, 58, 60, 60, 63, 63, 31, 31, 31, 32, + 32, 33, 35, 37, 40, 40, 43, 43, 46, 47, 46, 46, 46, 45, 46, 47, 47, 48, + 48, 50, 50, 51, 52, 53, 55, 55, 58, 58, 31, 31, 31, 32, 32, 34, 36, 37, + 40, 40, 43, 43, 46, 46, 46, 46, 45, 45, 45, 46, 46, 48, 48, 49, 50, 51, + 52, 53, 54, 55, 57, 57, 35, 36, 36, 37, 37, 39, 40, 42, 45, 45, 46, 46, + 47, 47, 47, 46, 46, 45, 46, 46, 46, 47, 47, 48, 49, 50, 51, 51, 53, 53, + 56, 56, 37, 38, 38, 39, 40, 41, 43, 44, 47, 47, 47, 47, 48, 48, 47, 47, + 46, 46, 46, 46, 46, 47, 47, 48, 49, 49, 50, 51, 52, 53, 55, 55, 42, 42, + 42, 42, 42, 44, 45, 45, 47, 47, 48, 48, 50, 50, 50, 50, 49, 49, 49, 50, + 50, 50, 50, 51, 52, 52, 53, 54, 55, 55, 58, 58, 48, 47, 47, 46, 46, 46, + 47, 47, 47, 48, 50, 50, 52, 53, 53, 53, 53, 53, 53, 54, 54, 54, 54, 55, + 55, 56, 56, 57, 58, 59, 60, 60, 48, 47, 47, 46, 46, 46, 46, 47, 47, 47, + 50, 50, 52, 53, 53, 54, 54, 55, 55, 55, 55, 56, 56, 57, 57, 58, 58, 59, + 60, 60, 62, 62, 49, 47, 47, 46, 45, 45, 46, 45, 45, 46, 49, 49, 53, 53, + 55, 56, 57, 58, 58, 59, 59, 61, 61, 62, 62, 63, 64, 64, 65, 65, 67, 67, + 49, 47, 47, 46, 45, 45, 46, 46, 46, 46, 49, 49, 53, 54, 55, 56, 57, 59, + 59, 60, 60, 61, 61, 62, 63, 63, 64, 65, 66, 66, 68, 68, 52, 50, 50, 48, + 48, 48, 47, 47, 47, 47, 50, 50, 53, 54, 56, 57, 59, 61, 62, 64, 64, 66, + 66, 68, 68, 69, 70, 71, 72, 73, 75, 75, 52, 50, 50, 48, 48, 48, 47, 47, + 47, 47, 50, 50, 53, 54, 56, 57, 59, 61, 62, 64, 64, 66, 66, 68, 68, 69, + 70, 71, 72, 73, 75, 75, 56, 54, 53, 52, 51, 51, 50, 50, 49, 49, 53, 53, + 55, 56, 58, 59, 61, 63, 64, 66, 66, 69, 70, 71, 72, 74, 75, 76, 77, 78, + 80, 80, 57, 54, 54, 52, 52, 51, 51, 51, 50, 50, 53, 53, 56, 57, 58, 60, + 61, 64, 64, 67, 67, 70, 71, 72, 73, 75, 76, 77, 79, 79, 82, 82, 61, 58, + 57, 56, 55, 54, 54, 53, 52, 53, 56, 56, 58, 59, 61, 62, 63, 66, 66, 69, + 69, 72, 73, 75, 76, 78, 79, 80, 82, 83, 86, 86, 63, 60, 60, 58, 57, 57, + 56, 55, 54, 55, 57, 57, 60, 60, 62, 64, 65, 67, 68, 71, 71, 74, 75, 77, + 78, 80, 82, 83, 85, 85, 89, 89], + /* Size 4x16 */ + [31, 42, 49, 57, 31, 42, 47, 54, 32, 42, 45, 52, 35, 45, 46, 51, 40, 47, + 46, 50, 43, 48, 49, 53, 46, 50, 53, 56, 46, 50, 55, 58, 46, 49, 57, 61, + 46, 49, 59, 64, 47, 50, 60, 67, 48, 50, 61, 71, 50, 52, 63, 73, 52, 53, + 64, 76, 55, 55, 66, 79, 58, 58, 68, 82], + /* Size 16x4 */ + [31, 31, 32, 35, 40, 43, 46, 46, 46, 46, 47, 48, 50, 52, 55, 58, 42, 42, + 42, 45, 47, 48, 50, 50, 49, 49, 50, 50, 52, 53, 55, 58, 49, 47, 45, 46, + 46, 49, 53, 55, 57, 59, 60, 61, 63, 64, 66, 68, 57, 54, 52, 51, 50, 53, + 56, 58, 61, 64, 67, 71, 73, 76, 79, 82], + /* Size 8x32 */ + [32, 31, 37, 48, 49, 52, 56, 61, 31, 31, 38, 47, 47, 50, 54, 58, 31, 31, + 38, 47, 47, 50, 53, 57, 30, 32, 39, 46, 46, 48, 52, 56, 30, 32, 40, 46, + 45, 48, 51, 55, 32, 34, 41, 46, 45, 48, 51, 54, 33, 36, 43, 47, 46, 47, + 50, 54, 34, 37, 44, 47, 45, 47, 50, 53, 37, 40, 47, 47, 45, 47, 49, 52, + 37, 40, 47, 48, 46, 47, 49, 53, 42, 43, 47, 50, 49, 50, 53, 56, 42, 43, + 47, 50, 49, 50, 53, 56, 47, 46, 48, 52, 53, 53, 55, 58, 49, 46, 48, 53, + 53, 54, 56, 59, 48, 46, 47, 53, 55, 56, 58, 61, 48, 46, 47, 53, 56, 57, + 59, 62, 48, 45, 46, 53, 57, 59, 61, 63, 49, 45, 46, 53, 58, 61, 63, 66, + 49, 45, 46, 53, 58, 62, 64, 66, 50, 46, 46, 54, 59, 64, 66, 69, 50, 46, + 46, 54, 59, 64, 66, 69, 52, 48, 47, 54, 61, 66, 69, 72, 52, 48, 47, 54, + 61, 66, 70, 73, 53, 49, 48, 55, 62, 68, 71, 75, 54, 50, 49, 55, 62, 68, + 72, 76, 55, 51, 49, 56, 63, 69, 74, 78, 57, 52, 50, 56, 64, 70, 75, 79, + 58, 53, 51, 57, 64, 71, 76, 80, 60, 54, 52, 58, 65, 72, 77, 82, 60, 55, + 53, 59, 65, 73, 78, 83, 63, 57, 55, 60, 67, 75, 80, 86, 63, 57, 55, 60, + 67, 75, 80, 86], + /* Size 32x8 */ + [32, 31, 31, 30, 30, 32, 33, 34, 37, 37, 42, 42, 47, 49, 48, 48, 48, 49, + 49, 50, 50, 52, 52, 53, 54, 55, 57, 58, 60, 60, 63, 63, 31, 31, 31, 32, + 32, 34, 36, 37, 40, 40, 43, 43, 46, 46, 46, 46, 45, 45, 45, 46, 46, 48, + 48, 49, 50, 51, 52, 53, 54, 55, 57, 57, 37, 38, 38, 39, 40, 41, 43, 44, + 47, 47, 47, 47, 48, 48, 47, 47, 46, 46, 46, 46, 46, 47, 47, 48, 49, 49, + 50, 51, 52, 53, 55, 55, 48, 47, 47, 46, 46, 46, 47, 47, 47, 48, 50, 50, + 52, 53, 53, 53, 53, 53, 53, 54, 54, 54, 54, 55, 55, 56, 56, 57, 58, 59, + 60, 60, 49, 47, 47, 46, 45, 45, 46, 45, 45, 46, 49, 49, 53, 53, 55, 56, + 57, 58, 58, 59, 59, 61, 61, 62, 62, 63, 64, 64, 65, 65, 67, 67, 52, 50, + 50, 48, 48, 48, 47, 47, 47, 47, 50, 50, 53, 54, 56, 57, 59, 61, 62, 64, + 64, 66, 66, 68, 68, 69, 70, 71, 72, 73, 75, 75, 56, 54, 53, 52, 51, 51, + 50, 50, 49, 49, 53, 53, 55, 56, 58, 59, 61, 63, 64, 66, 66, 69, 70, 71, + 72, 74, 75, 76, 77, 78, 80, 80, 61, 58, 57, 56, 55, 54, 54, 53, 52, 53, + 56, 56, 58, 59, 61, 62, 63, 66, 66, 69, 69, 72, 73, 75, 76, 78, 79, 80, + 82, 83, 86, 86]] + ], + [ + [ /* Luma */ + /* Size 4x4 */ + [32, 33, 42, 55, 33, 38, 46, 57, 42, 46, 63, 75, 55, 57, 75, 92], + /* Size 8x8 */ + [31, 32, 32, 34, 38, 46, 52, 63, 32, 32, 32, 34, 37, 44, 49, 59, 32, 32, + 35, 37, 40, 45, 49, 58, 34, 34, 37, 42, 47, 52, 56, 65, 38, 37, 40, 47, + 54, 60, 65, 73, 46, 44, 45, 52, 60, 69, 75, 84, 52, 49, 49, 56, 65, 75, + 82, 92, 63, 59, 58, 65, 73, 84, 92, 105], + /* Size 16x16 */ + [32, 31, 31, 31, 32, 32, 34, 36, 38, 41, 44, 48, 54, 58, 61, 65, 31, 32, + 32, 32, 32, 32, 34, 35, 38, 40, 42, 46, 51, 55, 58, 62, 31, 32, 32, 32, + 32, 32, 33, 34, 37, 38, 41, 44, 49, 53, 56, 59, 31, 32, 32, 33, 33, 33, + 35, 36, 38, 40, 42, 45, 49, 53, 56, 59, 32, 32, 32, 33, 34, 34, 36, 37, + 39, 40, 42, 45, 49, 53, 55, 59, 32, 32, 32, 33, 34, 35, 37, 38, 40, 41, + 42, 46, 49, 52, 55, 58, 34, 34, 33, 35, 36, 37, 39, 42, 44, 46, 47, 51, + 54, 57, 60, 63, 36, 35, 34, 36, 37, 38, 42, 48, 50, 52, 54, 57, 60, 63, + 65, 68, 38, 38, 37, 38, 39, 40, 44, 50, 52, 54, 57, 60, 64, 67, 69, 72, + 41, 40, 38, 40, 40, 41, 46, 52, 54, 57, 60, 63, 67, 70, 73, 75, 44, 42, + 41, 42, 42, 42, 47, 54, 57, 60, 63, 67, 71, 74, 77, 79, 48, 46, 44, 45, + 45, 46, 51, 57, 60, 63, 67, 71, 76, 79, 82, 85, 54, 51, 49, 49, 49, 49, + 54, 60, 64, 67, 71, 76, 82, 86, 89, 92, 58, 55, 53, 53, 53, 52, 57, 63, + 67, 70, 74, 79, 86, 90, 93, 97, 61, 58, 56, 56, 55, 55, 60, 65, 69, 73, + 77, 82, 89, 93, 97, 101, 65, 62, 59, 59, 59, 58, 63, 68, 72, 75, 79, 85, + 92, 97, 101, 105], + /* Size 32x32 */ + [32, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 33, 34, 34, 36, 36, 38, 39, + 41, 44, 44, 47, 48, 50, 54, 54, 58, 59, 61, 65, 65, 70, 31, 31, 31, 32, + 32, 32, 32, 32, 32, 32, 32, 33, 34, 34, 35, 35, 38, 38, 40, 42, 42, 46, + 47, 49, 52, 52, 56, 57, 59, 63, 63, 67, 31, 31, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 33, 34, 34, 35, 35, 38, 38, 40, 42, 42, 45, 46, 48, 51, 51, + 55, 56, 58, 62, 62, 67, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, + 34, 34, 35, 35, 37, 38, 39, 42, 42, 45, 45, 47, 50, 50, 54, 55, 57, 61, + 61, 65, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 34, 34, 34, + 37, 37, 38, 41, 41, 44, 44, 46, 49, 49, 53, 54, 56, 59, 59, 64, 31, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 34, 34, 34, 37, 37, 38, 41, + 41, 44, 44, 46, 49, 49, 53, 54, 56, 59, 59, 64, 31, 32, 32, 32, 32, 32, + 33, 33, 33, 33, 33, 34, 35, 35, 36, 36, 38, 39, 40, 42, 42, 44, 45, 47, + 49, 49, 53, 54, 56, 59, 59, 63, 31, 32, 32, 32, 32, 32, 33, 33, 33, 34, + 34, 35, 35, 36, 36, 36, 38, 39, 40, 42, 42, 45, 45, 47, 50, 50, 53, 54, + 56, 59, 59, 63, 32, 32, 32, 32, 32, 32, 33, 33, 34, 34, 34, 35, 36, 36, + 37, 37, 39, 39, 40, 42, 42, 45, 45, 47, 49, 49, 53, 54, 55, 59, 59, 63, + 32, 32, 32, 32, 32, 32, 33, 34, 34, 35, 35, 36, 37, 37, 38, 38, 40, 40, + 41, 42, 42, 45, 46, 47, 49, 49, 52, 53, 55, 58, 58, 62, 32, 32, 32, 32, + 32, 32, 33, 34, 34, 35, 35, 36, 37, 37, 38, 38, 40, 40, 41, 42, 42, 45, + 46, 47, 49, 49, 52, 53, 55, 58, 58, 62, 33, 33, 33, 33, 33, 33, 34, 35, + 35, 36, 36, 38, 39, 40, 42, 42, 43, 44, 45, 46, 46, 49, 50, 51, 53, 53, + 56, 57, 59, 62, 62, 66, 34, 34, 34, 34, 33, 33, 35, 35, 36, 37, 37, 39, + 39, 41, 42, 42, 44, 45, 46, 47, 47, 50, 51, 52, 54, 54, 57, 58, 60, 63, + 63, 67, 34, 34, 34, 34, 34, 34, 35, 36, 36, 37, 37, 40, 41, 42, 45, 45, + 46, 47, 48, 50, 50, 52, 53, 54, 56, 56, 59, 60, 62, 65, 65, 69, 36, 35, + 35, 35, 34, 34, 36, 36, 37, 38, 38, 42, 42, 45, 48, 48, 50, 50, 52, 54, + 54, 56, 57, 58, 60, 60, 63, 64, 65, 68, 68, 72, 36, 35, 35, 35, 34, 34, + 36, 36, 37, 38, 38, 42, 42, 45, 48, 48, 50, 50, 52, 54, 54, 56, 57, 58, + 60, 60, 63, 64, 65, 68, 68, 72, 38, 38, 38, 37, 37, 37, 38, 38, 39, 40, + 40, 43, 44, 46, 50, 50, 52, 53, 54, 57, 57, 59, 60, 61, 64, 64, 67, 68, + 69, 72, 72, 76, 39, 38, 38, 38, 37, 37, 39, 39, 39, 40, 40, 44, 45, 47, + 50, 50, 53, 54, 55, 58, 58, 60, 61, 62, 65, 65, 68, 69, 70, 73, 73, 77, + 41, 40, 40, 39, 38, 38, 40, 40, 40, 41, 41, 45, 46, 48, 52, 52, 54, 55, + 57, 60, 60, 62, 63, 65, 67, 67, 70, 71, 73, 75, 75, 79, 44, 42, 42, 42, + 41, 41, 42, 42, 42, 42, 42, 46, 47, 50, 54, 54, 57, 58, 60, 63, 63, 66, + 67, 68, 71, 71, 74, 75, 77, 79, 79, 83, 44, 42, 42, 42, 41, 41, 42, 42, + 42, 42, 42, 46, 47, 50, 54, 54, 57, 58, 60, 63, 63, 66, 67, 68, 71, 71, + 74, 75, 77, 79, 79, 83, 47, 46, 45, 45, 44, 44, 44, 45, 45, 45, 45, 49, + 50, 52, 56, 56, 59, 60, 62, 66, 66, 69, 70, 72, 75, 75, 78, 79, 81, 84, + 84, 88, 48, 47, 46, 45, 44, 44, 45, 45, 45, 46, 46, 50, 51, 53, 57, 57, + 60, 61, 63, 67, 67, 70, 71, 73, 76, 76, 79, 80, 82, 85, 85, 89, 50, 49, + 48, 47, 46, 46, 47, 47, 47, 47, 47, 51, 52, 54, 58, 58, 61, 62, 65, 68, + 68, 72, 73, 75, 78, 78, 82, 83, 85, 88, 88, 92, 54, 52, 51, 50, 49, 49, + 49, 50, 49, 49, 49, 53, 54, 56, 60, 60, 64, 65, 67, 71, 71, 75, 76, 78, + 82, 82, 86, 87, 89, 92, 92, 96, 54, 52, 51, 50, 49, 49, 49, 50, 49, 49, + 49, 53, 54, 56, 60, 60, 64, 65, 67, 71, 71, 75, 76, 78, 82, 82, 86, 87, + 89, 92, 92, 96, 58, 56, 55, 54, 53, 53, 53, 53, 53, 52, 52, 56, 57, 59, + 63, 63, 67, 68, 70, 74, 74, 78, 79, 82, 86, 86, 90, 91, 93, 97, 97, 101, + 59, 57, 56, 55, 54, 54, 54, 54, 54, 53, 53, 57, 58, 60, 64, 64, 68, 69, + 71, 75, 75, 79, 80, 83, 87, 87, 91, 92, 94, 98, 98, 102, 61, 59, 58, 57, + 56, 56, 56, 56, 55, 55, 55, 59, 60, 62, 65, 65, 69, 70, 73, 77, 77, 81, + 82, 85, 89, 89, 93, 94, 97, 101, 101, 105, 65, 63, 62, 61, 59, 59, 59, + 59, 59, 58, 58, 62, 63, 65, 68, 68, 72, 73, 75, 79, 79, 84, 85, 88, 92, + 92, 97, 98, 101, 105, 105, 109, 65, 63, 62, 61, 59, 59, 59, 59, 59, 58, + 58, 62, 63, 65, 68, 68, 72, 73, 75, 79, 79, 84, 85, 88, 92, 92, 97, 98, + 101, 105, 105, 109, 70, 67, 67, 65, 64, 64, 63, 63, 63, 62, 62, 66, 67, + 69, 72, 72, 76, 77, 79, 83, 83, 88, 89, 92, 96, 96, 101, 102, 105, 109, + 109, 114], + /* Size 4x8 */ + [32, 32, 42, 56, 32, 33, 41, 53, 32, 35, 42, 52, 34, 37, 50, 59, 38, 40, + 58, 68, 44, 45, 66, 78, 50, 50, 71, 86, 61, 58, 79, 97], + /* Size 8x4 */ + [32, 32, 32, 34, 38, 44, 50, 61, 32, 33, 35, 37, 40, 45, 50, 58, 42, 41, + 42, 50, 58, 66, 71, 79, 56, 53, 52, 59, 68, 78, 86, 97], + /* Size 8x16 */ + [32, 31, 32, 35, 39, 44, 53, 65, 31, 32, 32, 35, 38, 42, 51, 62, 31, 32, + 33, 34, 37, 41, 49, 59, 31, 32, 34, 35, 38, 42, 49, 59, 32, 32, 34, 36, + 39, 42, 49, 58, 32, 33, 35, 37, 40, 42, 49, 58, 34, 34, 37, 41, 44, 48, + 54, 63, 36, 34, 38, 46, 50, 54, 60, 68, 38, 37, 40, 47, 52, 57, 64, 72, + 41, 39, 41, 49, 54, 60, 67, 76, 44, 41, 43, 51, 57, 63, 71, 79, 48, 45, + 46, 54, 60, 67, 76, 85, 53, 49, 50, 57, 64, 71, 82, 92, 57, 53, 53, 60, + 67, 74, 86, 97, 61, 56, 56, 63, 69, 77, 89, 100, 65, 60, 58, 66, 72, 79, + 92, 105], + /* Size 16x8 */ + [32, 31, 31, 31, 32, 32, 34, 36, 38, 41, 44, 48, 53, 57, 61, 65, 31, 32, + 32, 32, 32, 33, 34, 34, 37, 39, 41, 45, 49, 53, 56, 60, 32, 32, 33, 34, + 34, 35, 37, 38, 40, 41, 43, 46, 50, 53, 56, 58, 35, 35, 34, 35, 36, 37, + 41, 46, 47, 49, 51, 54, 57, 60, 63, 66, 39, 38, 37, 38, 39, 40, 44, 50, + 52, 54, 57, 60, 64, 67, 69, 72, 44, 42, 41, 42, 42, 42, 48, 54, 57, 60, + 63, 67, 71, 74, 77, 79, 53, 51, 49, 49, 49, 49, 54, 60, 64, 67, 71, 76, + 82, 86, 89, 92, 65, 62, 59, 59, 58, 58, 63, 68, 72, 76, 79, 85, 92, 97, + 100, 105], + /* Size 16x32 */ + [32, 31, 31, 31, 32, 32, 35, 36, 39, 44, 44, 51, 53, 58, 65, 65, 31, 32, + 32, 32, 32, 32, 35, 35, 38, 42, 42, 49, 52, 56, 63, 63, 31, 32, 32, 32, + 32, 32, 35, 35, 38, 42, 42, 49, 51, 55, 62, 62, 31, 32, 32, 32, 32, 32, + 34, 35, 37, 41, 41, 48, 50, 54, 61, 61, 31, 32, 32, 32, 33, 33, 34, 34, + 37, 41, 41, 47, 49, 53, 59, 59, 31, 32, 32, 32, 33, 33, 34, 34, 37, 41, + 41, 47, 49, 53, 59, 59, 31, 32, 32, 33, 34, 34, 35, 36, 38, 42, 42, 48, + 49, 53, 59, 59, 32, 32, 32, 33, 34, 34, 36, 36, 38, 42, 42, 48, 50, 53, + 59, 59, 32, 32, 32, 33, 34, 34, 36, 37, 39, 42, 42, 48, 49, 53, 58, 58, + 32, 32, 33, 34, 35, 35, 37, 38, 40, 42, 42, 48, 49, 52, 58, 58, 32, 32, + 33, 34, 35, 35, 37, 38, 40, 42, 42, 48, 49, 52, 58, 58, 33, 33, 33, 35, + 36, 36, 40, 41, 43, 46, 46, 52, 53, 56, 62, 62, 34, 34, 34, 35, 37, 37, + 41, 42, 44, 48, 48, 53, 54, 57, 63, 63, 34, 34, 34, 35, 37, 37, 43, 44, + 46, 50, 50, 55, 56, 59, 65, 65, 36, 35, 34, 36, 38, 38, 46, 48, 50, 54, + 54, 58, 60, 63, 68, 68, 36, 35, 34, 36, 38, 38, 46, 48, 50, 54, 54, 58, + 60, 63, 68, 68, 38, 37, 37, 38, 40, 40, 47, 50, 52, 57, 57, 62, 64, 67, + 72, 72, 39, 38, 37, 39, 40, 40, 48, 50, 53, 58, 58, 63, 65, 68, 73, 73, + 41, 39, 39, 40, 41, 41, 49, 51, 54, 60, 60, 66, 67, 70, 76, 76, 44, 41, + 41, 42, 43, 43, 51, 53, 57, 63, 63, 69, 71, 74, 79, 79, 44, 41, 41, 42, + 43, 43, 51, 53, 57, 63, 63, 69, 71, 74, 79, 79, 47, 44, 44, 44, 45, 45, + 53, 56, 59, 66, 66, 73, 75, 78, 84, 84, 48, 45, 45, 45, 46, 46, 54, 56, + 60, 67, 67, 74, 76, 79, 85, 85, 50, 47, 46, 47, 47, 47, 55, 58, 61, 68, + 68, 76, 78, 82, 88, 88, 53, 50, 49, 50, 50, 50, 57, 60, 64, 71, 71, 79, + 82, 86, 92, 92, 53, 50, 49, 50, 50, 50, 57, 60, 64, 71, 71, 79, 82, 86, + 92, 92, 57, 54, 53, 53, 53, 53, 60, 63, 67, 74, 74, 83, 86, 90, 97, 97, + 58, 55, 54, 54, 54, 54, 61, 63, 68, 75, 75, 84, 87, 91, 98, 98, 61, 57, + 56, 56, 56, 56, 63, 65, 69, 77, 77, 86, 89, 93, 100, 100, 65, 61, 60, + 59, 58, 58, 66, 68, 72, 79, 79, 89, 92, 97, 105, 105, 65, 61, 60, 59, + 58, 58, 66, 68, 72, 79, 79, 89, 92, 97, 105, 105, 70, 65, 64, 63, 62, + 62, 70, 72, 76, 83, 83, 93, 96, 101, 109, 109], + /* Size 32x16 */ + [32, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 33, 34, 34, 36, 36, 38, 39, + 41, 44, 44, 47, 48, 50, 53, 53, 57, 58, 61, 65, 65, 70, 31, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 33, 34, 34, 35, 35, 37, 38, 39, 41, 41, 44, + 45, 47, 50, 50, 54, 55, 57, 61, 61, 65, 31, 32, 32, 32, 32, 32, 32, 32, + 32, 33, 33, 33, 34, 34, 34, 34, 37, 37, 39, 41, 41, 44, 45, 46, 49, 49, + 53, 54, 56, 60, 60, 64, 31, 32, 32, 32, 32, 32, 33, 33, 33, 34, 34, 35, + 35, 35, 36, 36, 38, 39, 40, 42, 42, 44, 45, 47, 50, 50, 53, 54, 56, 59, + 59, 63, 32, 32, 32, 32, 33, 33, 34, 34, 34, 35, 35, 36, 37, 37, 38, 38, + 40, 40, 41, 43, 43, 45, 46, 47, 50, 50, 53, 54, 56, 58, 58, 62, 32, 32, + 32, 32, 33, 33, 34, 34, 34, 35, 35, 36, 37, 37, 38, 38, 40, 40, 41, 43, + 43, 45, 46, 47, 50, 50, 53, 54, 56, 58, 58, 62, 35, 35, 35, 34, 34, 34, + 35, 36, 36, 37, 37, 40, 41, 43, 46, 46, 47, 48, 49, 51, 51, 53, 54, 55, + 57, 57, 60, 61, 63, 66, 66, 70, 36, 35, 35, 35, 34, 34, 36, 36, 37, 38, + 38, 41, 42, 44, 48, 48, 50, 50, 51, 53, 53, 56, 56, 58, 60, 60, 63, 63, + 65, 68, 68, 72, 39, 38, 38, 37, 37, 37, 38, 38, 39, 40, 40, 43, 44, 46, + 50, 50, 52, 53, 54, 57, 57, 59, 60, 61, 64, 64, 67, 68, 69, 72, 72, 76, + 44, 42, 42, 41, 41, 41, 42, 42, 42, 42, 42, 46, 48, 50, 54, 54, 57, 58, + 60, 63, 63, 66, 67, 68, 71, 71, 74, 75, 77, 79, 79, 83, 44, 42, 42, 41, + 41, 41, 42, 42, 42, 42, 42, 46, 48, 50, 54, 54, 57, 58, 60, 63, 63, 66, + 67, 68, 71, 71, 74, 75, 77, 79, 79, 83, 51, 49, 49, 48, 47, 47, 48, 48, + 48, 48, 48, 52, 53, 55, 58, 58, 62, 63, 66, 69, 69, 73, 74, 76, 79, 79, + 83, 84, 86, 89, 89, 93, 53, 52, 51, 50, 49, 49, 49, 50, 49, 49, 49, 53, + 54, 56, 60, 60, 64, 65, 67, 71, 71, 75, 76, 78, 82, 82, 86, 87, 89, 92, + 92, 96, 58, 56, 55, 54, 53, 53, 53, 53, 53, 52, 52, 56, 57, 59, 63, 63, + 67, 68, 70, 74, 74, 78, 79, 82, 86, 86, 90, 91, 93, 97, 97, 101, 65, 63, + 62, 61, 59, 59, 59, 59, 58, 58, 58, 62, 63, 65, 68, 68, 72, 73, 76, 79, + 79, 84, 85, 88, 92, 92, 97, 98, 100, 105, 105, 109, 65, 63, 62, 61, 59, + 59, 59, 59, 58, 58, 58, 62, 63, 65, 68, 68, 72, 73, 76, 79, 79, 84, 85, + 88, 92, 92, 97, 98, 100, 105, 105, 109], + /* Size 4x16 */ + [31, 32, 44, 58, 32, 32, 42, 55, 32, 33, 41, 53, 32, 34, 42, 53, 32, 34, + 42, 53, 32, 35, 42, 52, 34, 37, 48, 57, 35, 38, 54, 63, 37, 40, 57, 67, + 39, 41, 60, 70, 41, 43, 63, 74, 45, 46, 67, 79, 50, 50, 71, 86, 54, 53, + 74, 90, 57, 56, 77, 93, 61, 58, 79, 97], + /* Size 16x4 */ + [31, 32, 32, 32, 32, 32, 34, 35, 37, 39, 41, 45, 50, 54, 57, 61, 32, 32, + 33, 34, 34, 35, 37, 38, 40, 41, 43, 46, 50, 53, 56, 58, 44, 42, 41, 42, + 42, 42, 48, 54, 57, 60, 63, 67, 71, 74, 77, 79, 58, 55, 53, 53, 53, 52, + 57, 63, 67, 70, 74, 79, 86, 90, 93, 97], + /* Size 8x32 */ + [32, 31, 32, 35, 39, 44, 53, 65, 31, 32, 32, 35, 38, 42, 52, 63, 31, 32, + 32, 35, 38, 42, 51, 62, 31, 32, 32, 34, 37, 41, 50, 61, 31, 32, 33, 34, + 37, 41, 49, 59, 31, 32, 33, 34, 37, 41, 49, 59, 31, 32, 34, 35, 38, 42, + 49, 59, 32, 32, 34, 36, 38, 42, 50, 59, 32, 32, 34, 36, 39, 42, 49, 58, + 32, 33, 35, 37, 40, 42, 49, 58, 32, 33, 35, 37, 40, 42, 49, 58, 33, 33, + 36, 40, 43, 46, 53, 62, 34, 34, 37, 41, 44, 48, 54, 63, 34, 34, 37, 43, + 46, 50, 56, 65, 36, 34, 38, 46, 50, 54, 60, 68, 36, 34, 38, 46, 50, 54, + 60, 68, 38, 37, 40, 47, 52, 57, 64, 72, 39, 37, 40, 48, 53, 58, 65, 73, + 41, 39, 41, 49, 54, 60, 67, 76, 44, 41, 43, 51, 57, 63, 71, 79, 44, 41, + 43, 51, 57, 63, 71, 79, 47, 44, 45, 53, 59, 66, 75, 84, 48, 45, 46, 54, + 60, 67, 76, 85, 50, 46, 47, 55, 61, 68, 78, 88, 53, 49, 50, 57, 64, 71, + 82, 92, 53, 49, 50, 57, 64, 71, 82, 92, 57, 53, 53, 60, 67, 74, 86, 97, + 58, 54, 54, 61, 68, 75, 87, 98, 61, 56, 56, 63, 69, 77, 89, 100, 65, 60, + 58, 66, 72, 79, 92, 105, 65, 60, 58, 66, 72, 79, 92, 105, 70, 64, 62, + 70, 76, 83, 96, 109], + /* Size 32x8 */ + [32, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 33, 34, 34, 36, 36, 38, 39, + 41, 44, 44, 47, 48, 50, 53, 53, 57, 58, 61, 65, 65, 70, 31, 32, 32, 32, + 32, 32, 32, 32, 32, 33, 33, 33, 34, 34, 34, 34, 37, 37, 39, 41, 41, 44, + 45, 46, 49, 49, 53, 54, 56, 60, 60, 64, 32, 32, 32, 32, 33, 33, 34, 34, + 34, 35, 35, 36, 37, 37, 38, 38, 40, 40, 41, 43, 43, 45, 46, 47, 50, 50, + 53, 54, 56, 58, 58, 62, 35, 35, 35, 34, 34, 34, 35, 36, 36, 37, 37, 40, + 41, 43, 46, 46, 47, 48, 49, 51, 51, 53, 54, 55, 57, 57, 60, 61, 63, 66, + 66, 70, 39, 38, 38, 37, 37, 37, 38, 38, 39, 40, 40, 43, 44, 46, 50, 50, + 52, 53, 54, 57, 57, 59, 60, 61, 64, 64, 67, 68, 69, 72, 72, 76, 44, 42, + 42, 41, 41, 41, 42, 42, 42, 42, 42, 46, 48, 50, 54, 54, 57, 58, 60, 63, + 63, 66, 67, 68, 71, 71, 74, 75, 77, 79, 79, 83, 53, 52, 51, 50, 49, 49, + 49, 50, 49, 49, 49, 53, 54, 56, 60, 60, 64, 65, 67, 71, 71, 75, 76, 78, + 82, 82, 86, 87, 89, 92, 92, 96, 65, 63, 62, 61, 59, 59, 59, 59, 58, 58, + 58, 62, 63, 65, 68, 68, 72, 73, 76, 79, 79, 84, 85, 88, 92, 92, 97, 98, + 100, 105, 105, 109]], + [ /* Chroma */ + /* Size 4x4 */ + [31, 41, 46, 51, 41, 48, 48, 51, 46, 48, 58, 62, 51, 51, 62, 71], + /* Size 8x8 */ + [31, 31, 38, 44, 47, 48, 50, 55, 31, 32, 40, 44, 45, 46, 47, 52, 38, 40, + 47, 47, 46, 46, 47, 50, 44, 44, 47, 50, 51, 51, 52, 54, 47, 45, 46, 51, + 54, 56, 57, 60, 48, 46, 46, 51, 56, 61, 63, 66, 50, 47, 47, 52, 57, 63, + 66, 70, 55, 52, 50, 54, 60, 66, 70, 76], + /* Size 16x16 */ + [32, 31, 30, 33, 34, 36, 41, 49, 48, 49, 49, 50, 52, 54, 55, 57, 31, 31, + 31, 34, 36, 38, 42, 47, 47, 47, 47, 48, 50, 51, 53, 54, 30, 31, 32, 34, + 37, 40, 42, 46, 45, 45, 45, 46, 47, 49, 50, 52, 33, 34, 34, 37, 40, 42, + 44, 47, 46, 46, 45, 46, 47, 49, 50, 51, 34, 36, 37, 40, 42, 45, 46, 47, + 46, 46, 45, 46, 47, 48, 49, 50, 36, 38, 40, 42, 45, 47, 47, 48, 47, 46, + 45, 46, 47, 48, 49, 50, 41, 42, 42, 44, 46, 47, 48, 50, 50, 49, 49, 50, + 50, 51, 52, 53, 49, 47, 46, 47, 47, 48, 50, 53, 53, 53, 53, 54, 54, 55, + 56, 56, 48, 47, 45, 46, 46, 47, 50, 53, 54, 54, 55, 56, 57, 58, 58, 59, + 49, 47, 45, 46, 46, 46, 49, 53, 54, 55, 57, 58, 59, 60, 60, 61, 49, 47, + 45, 45, 45, 45, 49, 53, 55, 57, 58, 60, 61, 62, 63, 63, 50, 48, 46, 46, + 46, 46, 50, 54, 56, 58, 60, 61, 63, 65, 66, 67, 52, 50, 47, 47, 47, 47, + 50, 54, 57, 59, 61, 63, 66, 68, 69, 70, 54, 51, 49, 49, 48, 48, 51, 55, + 58, 60, 62, 65, 68, 70, 71, 73, 55, 53, 50, 50, 49, 49, 52, 56, 58, 60, + 63, 66, 69, 71, 73, 74, 57, 54, 52, 51, 50, 50, 53, 56, 59, 61, 63, 67, + 70, 73, 74, 76], + /* Size 32x32 */ + [32, 31, 31, 31, 30, 30, 33, 33, 34, 36, 36, 40, 41, 44, 49, 49, 48, 48, + 49, 49, 49, 50, 50, 51, 52, 52, 54, 54, 55, 57, 57, 59, 31, 31, 31, 31, + 31, 31, 33, 34, 36, 38, 38, 41, 42, 44, 48, 48, 47, 47, 47, 47, 47, 48, + 49, 49, 50, 50, 52, 52, 53, 55, 55, 57, 31, 31, 31, 31, 31, 31, 34, 34, + 36, 38, 38, 41, 42, 44, 47, 47, 47, 47, 47, 47, 47, 48, 48, 49, 50, 50, + 51, 52, 53, 54, 54, 56, 31, 31, 31, 31, 31, 31, 34, 35, 36, 39, 39, 41, + 42, 44, 47, 47, 46, 46, 46, 46, 46, 47, 47, 48, 49, 49, 50, 51, 52, 53, + 53, 55, 30, 31, 31, 31, 32, 32, 34, 35, 37, 40, 40, 42, 42, 44, 46, 46, + 45, 45, 45, 45, 45, 46, 46, 47, 47, 47, 49, 49, 50, 52, 52, 54, 30, 31, + 31, 31, 32, 32, 34, 35, 37, 40, 40, 42, 42, 44, 46, 46, 45, 45, 45, 45, + 45, 46, 46, 47, 47, 47, 49, 49, 50, 52, 52, 54, 33, 33, 34, 34, 34, 34, + 37, 38, 40, 42, 42, 44, 44, 45, 47, 47, 46, 46, 46, 45, 45, 46, 46, 47, + 47, 47, 49, 49, 50, 51, 51, 53, 33, 34, 34, 35, 35, 35, 38, 39, 40, 43, + 43, 44, 45, 46, 47, 47, 46, 46, 46, 45, 45, 46, 46, 47, 47, 47, 49, 49, + 50, 51, 51, 53, 34, 36, 36, 36, 37, 37, 40, 40, 42, 45, 45, 45, 46, 46, + 47, 47, 46, 46, 46, 45, 45, 46, 46, 47, 47, 47, 48, 49, 49, 50, 50, 52, + 36, 38, 38, 39, 40, 40, 42, 43, 45, 47, 47, 47, 47, 47, 48, 48, 47, 46, + 46, 45, 45, 46, 46, 46, 47, 47, 48, 48, 49, 50, 50, 51, 36, 38, 38, 39, + 40, 40, 42, 43, 45, 47, 47, 47, 47, 47, 48, 48, 47, 46, 46, 45, 45, 46, + 46, 46, 47, 47, 48, 48, 49, 50, 50, 51, 40, 41, 41, 41, 42, 42, 44, 44, + 45, 47, 47, 48, 48, 49, 50, 50, 49, 49, 49, 48, 48, 49, 49, 49, 49, 49, + 51, 51, 51, 52, 52, 54, 41, 42, 42, 42, 42, 42, 44, 45, 46, 47, 47, 48, + 48, 49, 50, 50, 50, 49, 49, 49, 49, 50, 50, 50, 50, 50, 51, 52, 52, 53, + 53, 55, 44, 44, 44, 44, 44, 44, 45, 46, 46, 47, 47, 49, 49, 50, 51, 51, + 51, 51, 51, 51, 51, 51, 51, 51, 52, 52, 53, 53, 54, 54, 54, 56, 49, 48, + 47, 47, 46, 46, 47, 47, 47, 48, 48, 50, 50, 51, 53, 53, 53, 53, 53, 53, + 53, 54, 54, 54, 54, 54, 55, 55, 56, 56, 56, 58, 49, 48, 47, 47, 46, 46, + 47, 47, 47, 48, 48, 50, 50, 51, 53, 53, 53, 53, 53, 53, 53, 54, 54, 54, + 54, 54, 55, 55, 56, 56, 56, 58, 48, 47, 47, 46, 45, 45, 46, 46, 46, 47, + 47, 49, 50, 51, 53, 53, 54, 54, 54, 55, 55, 56, 56, 56, 57, 57, 58, 58, + 58, 59, 59, 60, 48, 47, 47, 46, 45, 45, 46, 46, 46, 46, 46, 49, 49, 51, + 53, 53, 54, 54, 55, 55, 55, 56, 56, 57, 57, 57, 58, 58, 59, 60, 60, 61, + 49, 47, 47, 46, 45, 45, 46, 46, 46, 46, 46, 49, 49, 51, 53, 53, 54, 55, + 55, 57, 57, 57, 58, 58, 59, 59, 60, 60, 60, 61, 61, 63, 49, 47, 47, 46, + 45, 45, 45, 45, 45, 45, 45, 48, 49, 51, 53, 53, 55, 55, 57, 58, 58, 59, + 60, 60, 61, 61, 62, 62, 63, 63, 63, 65, 49, 47, 47, 46, 45, 45, 45, 45, + 45, 45, 45, 48, 49, 51, 53, 53, 55, 55, 57, 58, 58, 59, 60, 60, 61, 61, + 62, 62, 63, 63, 63, 65, 50, 48, 48, 47, 46, 46, 46, 46, 46, 46, 46, 49, + 50, 51, 54, 54, 56, 56, 57, 59, 59, 61, 61, 62, 63, 63, 64, 64, 65, 66, + 66, 67, 50, 49, 48, 47, 46, 46, 46, 46, 46, 46, 46, 49, 50, 51, 54, 54, + 56, 56, 58, 60, 60, 61, 61, 62, 63, 63, 65, 65, 66, 67, 67, 68, 51, 49, + 49, 48, 47, 47, 47, 47, 47, 46, 46, 49, 50, 51, 54, 54, 56, 57, 58, 60, + 60, 62, 62, 63, 65, 65, 66, 66, 67, 68, 68, 70, 52, 50, 50, 49, 47, 47, + 47, 47, 47, 47, 47, 49, 50, 52, 54, 54, 57, 57, 59, 61, 61, 63, 63, 65, + 66, 66, 68, 68, 69, 70, 70, 72, 52, 50, 50, 49, 47, 47, 47, 47, 47, 47, + 47, 49, 50, 52, 54, 54, 57, 57, 59, 61, 61, 63, 63, 65, 66, 66, 68, 68, + 69, 70, 70, 72, 54, 52, 51, 50, 49, 49, 49, 49, 48, 48, 48, 51, 51, 53, + 55, 55, 58, 58, 60, 62, 62, 64, 65, 66, 68, 68, 70, 70, 71, 73, 73, 74, + 54, 52, 52, 51, 49, 49, 49, 49, 49, 48, 48, 51, 52, 53, 55, 55, 58, 58, + 60, 62, 62, 64, 65, 66, 68, 68, 70, 71, 72, 73, 73, 75, 55, 53, 53, 52, + 50, 50, 50, 50, 49, 49, 49, 51, 52, 54, 56, 56, 58, 59, 60, 63, 63, 65, + 66, 67, 69, 69, 71, 72, 73, 74, 74, 76, 57, 55, 54, 53, 52, 52, 51, 51, + 50, 50, 50, 52, 53, 54, 56, 56, 59, 60, 61, 63, 63, 66, 67, 68, 70, 70, + 73, 73, 74, 76, 76, 78, 57, 55, 54, 53, 52, 52, 51, 51, 50, 50, 50, 52, + 53, 54, 56, 56, 59, 60, 61, 63, 63, 66, 67, 68, 70, 70, 73, 73, 74, 76, + 76, 78, 59, 57, 56, 55, 54, 54, 53, 53, 52, 51, 51, 54, 55, 56, 58, 58, + 60, 61, 63, 65, 65, 67, 68, 70, 72, 72, 74, 75, 76, 78, 78, 80], + /* Size 4x8 */ + [31, 38, 47, 52, 32, 40, 45, 49, 39, 47, 45, 48, 44, 47, 51, 53, 46, 47, + 56, 58, 47, 46, 59, 64, 48, 47, 61, 68, 53, 50, 64, 73], + /* Size 8x4 */ + [31, 32, 39, 44, 46, 47, 48, 53, 38, 40, 47, 47, 47, 46, 47, 50, 47, 45, + 45, 51, 56, 59, 61, 64, 52, 49, 48, 53, 58, 64, 68, 73], + /* Size 8x16 */ + [32, 31, 37, 45, 48, 49, 52, 57, 31, 31, 38, 45, 47, 47, 50, 54, 30, 32, + 40, 44, 45, 45, 48, 52, 33, 35, 42, 46, 46, 45, 47, 51, 35, 37, 44, 46, + 46, 45, 47, 51, 37, 40, 47, 47, 47, 45, 47, 50, 42, 43, 47, 49, 50, 49, + 50, 53, 49, 46, 48, 52, 53, 53, 54, 57, 48, 46, 47, 51, 54, 55, 57, 59, + 48, 45, 46, 51, 54, 57, 59, 61, 49, 45, 46, 51, 55, 58, 61, 64, 50, 46, + 46, 52, 56, 59, 64, 67, 52, 48, 47, 53, 57, 61, 66, 71, 54, 49, 48, 54, + 58, 62, 68, 73, 55, 51, 49, 54, 58, 63, 69, 74, 57, 52, 50, 55, 59, 64, + 70, 76], + /* Size 16x8 */ + [32, 31, 30, 33, 35, 37, 42, 49, 48, 48, 49, 50, 52, 54, 55, 57, 31, 31, + 32, 35, 37, 40, 43, 46, 46, 45, 45, 46, 48, 49, 51, 52, 37, 38, 40, 42, + 44, 47, 47, 48, 47, 46, 46, 46, 47, 48, 49, 50, 45, 45, 44, 46, 46, 47, + 49, 52, 51, 51, 51, 52, 53, 54, 54, 55, 48, 47, 45, 46, 46, 47, 50, 53, + 54, 54, 55, 56, 57, 58, 58, 59, 49, 47, 45, 45, 45, 45, 49, 53, 55, 57, + 58, 59, 61, 62, 63, 64, 52, 50, 48, 47, 47, 47, 50, 54, 57, 59, 61, 64, + 66, 68, 69, 70, 57, 54, 52, 51, 51, 50, 53, 57, 59, 61, 64, 67, 71, 73, + 74, 76], + /* Size 16x32 */ + [32, 31, 31, 33, 37, 37, 45, 48, 48, 49, 49, 51, 52, 54, 57, 57, 31, 31, + 31, 34, 38, 38, 45, 47, 47, 47, 47, 50, 50, 52, 55, 55, 31, 31, 31, 34, + 38, 38, 45, 47, 47, 47, 47, 49, 50, 51, 54, 54, 31, 31, 32, 34, 39, 39, + 45, 46, 46, 46, 46, 48, 49, 51, 53, 53, 30, 32, 32, 35, 40, 40, 44, 46, + 45, 45, 45, 47, 48, 49, 52, 52, 30, 32, 32, 35, 40, 40, 44, 46, 45, 45, + 45, 47, 48, 49, 52, 52, 33, 34, 35, 37, 42, 42, 46, 47, 46, 45, 45, 47, + 47, 49, 51, 51, 33, 35, 36, 38, 43, 43, 46, 47, 46, 46, 46, 47, 47, 49, + 51, 51, 35, 37, 37, 40, 44, 44, 46, 47, 46, 45, 45, 47, 47, 48, 51, 51, + 37, 39, 40, 43, 47, 47, 47, 47, 47, 45, 45, 46, 47, 48, 50, 50, 37, 39, + 40, 43, 47, 47, 47, 47, 47, 45, 45, 46, 47, 48, 50, 50, 41, 42, 42, 44, + 47, 47, 49, 49, 49, 48, 48, 49, 50, 51, 52, 52, 42, 42, 43, 44, 47, 47, + 49, 50, 50, 49, 49, 50, 50, 51, 53, 53, 44, 44, 44, 45, 47, 47, 50, 51, + 51, 51, 51, 52, 52, 53, 54, 54, 49, 47, 46, 47, 48, 48, 52, 53, 53, 53, + 53, 54, 54, 55, 57, 57, 49, 47, 46, 47, 48, 48, 52, 53, 53, 53, 53, 54, + 54, 55, 57, 57, 48, 46, 46, 46, 47, 47, 51, 53, 54, 55, 55, 56, 57, 58, + 59, 59, 48, 46, 46, 46, 47, 47, 51, 53, 54, 56, 56, 57, 57, 58, 60, 60, + 48, 46, 45, 46, 46, 46, 51, 53, 54, 57, 57, 58, 59, 60, 61, 61, 49, 46, + 45, 45, 46, 46, 51, 53, 55, 58, 58, 61, 61, 62, 64, 64, 49, 46, 45, 45, + 46, 46, 51, 53, 55, 58, 58, 61, 61, 62, 64, 64, 50, 47, 46, 46, 46, 46, + 52, 54, 56, 59, 59, 62, 63, 64, 66, 66, 50, 47, 46, 46, 46, 46, 52, 54, + 56, 59, 59, 63, 64, 65, 67, 67, 51, 48, 47, 47, 47, 47, 52, 54, 56, 60, + 60, 64, 65, 66, 68, 68, 52, 48, 48, 47, 47, 47, 53, 54, 57, 61, 61, 65, + 66, 68, 71, 71, 52, 48, 48, 47, 47, 47, 53, 54, 57, 61, 61, 65, 66, 68, + 71, 71, 54, 50, 49, 49, 48, 48, 54, 55, 58, 62, 62, 67, 68, 70, 73, 73, + 54, 51, 50, 49, 49, 49, 54, 55, 58, 62, 62, 67, 68, 70, 73, 73, 55, 51, + 51, 50, 49, 49, 54, 56, 58, 63, 63, 68, 69, 71, 74, 74, 57, 53, 52, 51, + 50, 50, 55, 56, 59, 64, 64, 69, 70, 73, 76, 76, 57, 53, 52, 51, 50, 50, + 55, 56, 59, 64, 64, 69, 70, 73, 76, 76, 59, 55, 54, 53, 52, 52, 57, 58, + 61, 65, 65, 70, 72, 74, 78, 78], + /* Size 32x16 */ + [32, 31, 31, 31, 30, 30, 33, 33, 35, 37, 37, 41, 42, 44, 49, 49, 48, 48, + 48, 49, 49, 50, 50, 51, 52, 52, 54, 54, 55, 57, 57, 59, 31, 31, 31, 31, + 32, 32, 34, 35, 37, 39, 39, 42, 42, 44, 47, 47, 46, 46, 46, 46, 46, 47, + 47, 48, 48, 48, 50, 51, 51, 53, 53, 55, 31, 31, 31, 32, 32, 32, 35, 36, + 37, 40, 40, 42, 43, 44, 46, 46, 46, 46, 45, 45, 45, 46, 46, 47, 48, 48, + 49, 50, 51, 52, 52, 54, 33, 34, 34, 34, 35, 35, 37, 38, 40, 43, 43, 44, + 44, 45, 47, 47, 46, 46, 46, 45, 45, 46, 46, 47, 47, 47, 49, 49, 50, 51, + 51, 53, 37, 38, 38, 39, 40, 40, 42, 43, 44, 47, 47, 47, 47, 47, 48, 48, + 47, 47, 46, 46, 46, 46, 46, 47, 47, 47, 48, 49, 49, 50, 50, 52, 37, 38, + 38, 39, 40, 40, 42, 43, 44, 47, 47, 47, 47, 47, 48, 48, 47, 47, 46, 46, + 46, 46, 46, 47, 47, 47, 48, 49, 49, 50, 50, 52, 45, 45, 45, 45, 44, 44, + 46, 46, 46, 47, 47, 49, 49, 50, 52, 52, 51, 51, 51, 51, 51, 52, 52, 52, + 53, 53, 54, 54, 54, 55, 55, 57, 48, 47, 47, 46, 46, 46, 47, 47, 47, 47, + 47, 49, 50, 51, 53, 53, 53, 53, 53, 53, 53, 54, 54, 54, 54, 54, 55, 55, + 56, 56, 56, 58, 48, 47, 47, 46, 45, 45, 46, 46, 46, 47, 47, 49, 50, 51, + 53, 53, 54, 54, 54, 55, 55, 56, 56, 56, 57, 57, 58, 58, 58, 59, 59, 61, + 49, 47, 47, 46, 45, 45, 45, 46, 45, 45, 45, 48, 49, 51, 53, 53, 55, 56, + 57, 58, 58, 59, 59, 60, 61, 61, 62, 62, 63, 64, 64, 65, 49, 47, 47, 46, + 45, 45, 45, 46, 45, 45, 45, 48, 49, 51, 53, 53, 55, 56, 57, 58, 58, 59, + 59, 60, 61, 61, 62, 62, 63, 64, 64, 65, 51, 50, 49, 48, 47, 47, 47, 47, + 47, 46, 46, 49, 50, 52, 54, 54, 56, 57, 58, 61, 61, 62, 63, 64, 65, 65, + 67, 67, 68, 69, 69, 70, 52, 50, 50, 49, 48, 48, 47, 47, 47, 47, 47, 50, + 50, 52, 54, 54, 57, 57, 59, 61, 61, 63, 64, 65, 66, 66, 68, 68, 69, 70, + 70, 72, 54, 52, 51, 51, 49, 49, 49, 49, 48, 48, 48, 51, 51, 53, 55, 55, + 58, 58, 60, 62, 62, 64, 65, 66, 68, 68, 70, 70, 71, 73, 73, 74, 57, 55, + 54, 53, 52, 52, 51, 51, 51, 50, 50, 52, 53, 54, 57, 57, 59, 60, 61, 64, + 64, 66, 67, 68, 71, 71, 73, 73, 74, 76, 76, 78, 57, 55, 54, 53, 52, 52, + 51, 51, 51, 50, 50, 52, 53, 54, 57, 57, 59, 60, 61, 64, 64, 66, 67, 68, + 71, 71, 73, 73, 74, 76, 76, 78], + /* Size 4x16 */ + [31, 37, 49, 54, 31, 38, 47, 51, 32, 40, 45, 49, 34, 42, 45, 49, 37, 44, + 45, 48, 39, 47, 45, 48, 42, 47, 49, 51, 47, 48, 53, 55, 46, 47, 55, 58, + 46, 46, 57, 60, 46, 46, 58, 62, 47, 46, 59, 65, 48, 47, 61, 68, 50, 48, + 62, 70, 51, 49, 63, 71, 53, 50, 64, 73], + /* Size 16x4 */ + [31, 31, 32, 34, 37, 39, 42, 47, 46, 46, 46, 47, 48, 50, 51, 53, 37, 38, + 40, 42, 44, 47, 47, 48, 47, 46, 46, 46, 47, 48, 49, 50, 49, 47, 45, 45, + 45, 45, 49, 53, 55, 57, 58, 59, 61, 62, 63, 64, 54, 51, 49, 49, 48, 48, + 51, 55, 58, 60, 62, 65, 68, 70, 71, 73], + /* Size 8x32 */ + [32, 31, 37, 45, 48, 49, 52, 57, 31, 31, 38, 45, 47, 47, 50, 55, 31, 31, + 38, 45, 47, 47, 50, 54, 31, 32, 39, 45, 46, 46, 49, 53, 30, 32, 40, 44, + 45, 45, 48, 52, 30, 32, 40, 44, 45, 45, 48, 52, 33, 35, 42, 46, 46, 45, + 47, 51, 33, 36, 43, 46, 46, 46, 47, 51, 35, 37, 44, 46, 46, 45, 47, 51, + 37, 40, 47, 47, 47, 45, 47, 50, 37, 40, 47, 47, 47, 45, 47, 50, 41, 42, + 47, 49, 49, 48, 50, 52, 42, 43, 47, 49, 50, 49, 50, 53, 44, 44, 47, 50, + 51, 51, 52, 54, 49, 46, 48, 52, 53, 53, 54, 57, 49, 46, 48, 52, 53, 53, + 54, 57, 48, 46, 47, 51, 54, 55, 57, 59, 48, 46, 47, 51, 54, 56, 57, 60, + 48, 45, 46, 51, 54, 57, 59, 61, 49, 45, 46, 51, 55, 58, 61, 64, 49, 45, + 46, 51, 55, 58, 61, 64, 50, 46, 46, 52, 56, 59, 63, 66, 50, 46, 46, 52, + 56, 59, 64, 67, 51, 47, 47, 52, 56, 60, 65, 68, 52, 48, 47, 53, 57, 61, + 66, 71, 52, 48, 47, 53, 57, 61, 66, 71, 54, 49, 48, 54, 58, 62, 68, 73, + 54, 50, 49, 54, 58, 62, 68, 73, 55, 51, 49, 54, 58, 63, 69, 74, 57, 52, + 50, 55, 59, 64, 70, 76, 57, 52, 50, 55, 59, 64, 70, 76, 59, 54, 52, 57, + 61, 65, 72, 78], + /* Size 32x8 */ + [32, 31, 31, 31, 30, 30, 33, 33, 35, 37, 37, 41, 42, 44, 49, 49, 48, 48, + 48, 49, 49, 50, 50, 51, 52, 52, 54, 54, 55, 57, 57, 59, 31, 31, 31, 32, + 32, 32, 35, 36, 37, 40, 40, 42, 43, 44, 46, 46, 46, 46, 45, 45, 45, 46, + 46, 47, 48, 48, 49, 50, 51, 52, 52, 54, 37, 38, 38, 39, 40, 40, 42, 43, + 44, 47, 47, 47, 47, 47, 48, 48, 47, 47, 46, 46, 46, 46, 46, 47, 47, 47, + 48, 49, 49, 50, 50, 52, 45, 45, 45, 45, 44, 44, 46, 46, 46, 47, 47, 49, + 49, 50, 52, 52, 51, 51, 51, 51, 51, 52, 52, 52, 53, 53, 54, 54, 54, 55, + 55, 57, 48, 47, 47, 46, 45, 45, 46, 46, 46, 47, 47, 49, 50, 51, 53, 53, + 54, 54, 54, 55, 55, 56, 56, 56, 57, 57, 58, 58, 58, 59, 59, 61, 49, 47, + 47, 46, 45, 45, 45, 46, 45, 45, 45, 48, 49, 51, 53, 53, 55, 56, 57, 58, + 58, 59, 59, 60, 61, 61, 62, 62, 63, 64, 64, 65, 52, 50, 50, 49, 48, 48, + 47, 47, 47, 47, 47, 50, 50, 52, 54, 54, 57, 57, 59, 61, 61, 63, 64, 65, + 66, 66, 68, 68, 69, 70, 70, 72, 57, 55, 54, 53, 52, 52, 51, 51, 51, 50, + 50, 52, 53, 54, 57, 57, 59, 60, 61, 64, 64, 66, 67, 68, 71, 71, 73, 73, + 74, 76, 76, 78]] + ], + [ + [ /* Luma */ + /* Size 4x4 */ + [32, 32, 38, 51, 32, 35, 40, 49, 38, 40, 54, 64, 51, 49, 64, 81], + /* Size 8x8 */ + [31, 32, 32, 34, 35, 41, 47, 53, 32, 32, 32, 33, 34, 40, 44, 50, 32, 32, + 34, 35, 37, 41, 45, 51, 34, 33, 35, 39, 42, 47, 51, 55, 35, 34, 37, 42, + 48, 53, 57, 61, 41, 40, 41, 47, 53, 60, 65, 70, 47, 44, 45, 51, 57, 65, + 71, 77, 53, 50, 51, 55, 61, 70, 77, 85], + /* Size 16x16 */ + [32, 31, 31, 31, 31, 32, 32, 34, 36, 38, 39, 44, 47, 49, 54, 59, 31, 32, + 32, 32, 32, 32, 33, 34, 35, 37, 38, 42, 45, 47, 51, 56, 31, 32, 32, 32, + 32, 32, 33, 33, 34, 36, 37, 41, 44, 46, 50, 54, 31, 32, 32, 32, 32, 33, + 33, 34, 35, 36, 38, 41, 44, 45, 49, 54, 31, 32, 32, 32, 33, 34, 34, 35, + 36, 38, 39, 42, 45, 46, 50, 54, 32, 32, 32, 33, 34, 35, 36, 37, 38, 39, + 40, 42, 45, 46, 49, 53, 32, 33, 33, 33, 34, 36, 36, 38, 40, 41, 42, 44, + 47, 48, 51, 55, 34, 34, 33, 34, 35, 37, 38, 39, 42, 44, 45, 47, 50, 51, + 54, 58, 36, 35, 34, 35, 36, 38, 40, 42, 48, 50, 50, 54, 56, 57, 60, 64, + 38, 37, 36, 36, 38, 39, 41, 44, 50, 51, 52, 56, 58, 60, 63, 67, 39, 38, + 37, 38, 39, 40, 42, 45, 50, 52, 54, 58, 60, 62, 65, 69, 44, 42, 41, 41, + 42, 42, 44, 47, 54, 56, 58, 63, 66, 68, 71, 75, 47, 45, 44, 44, 45, 45, + 47, 50, 56, 58, 60, 66, 69, 71, 75, 79, 49, 47, 46, 45, 46, 46, 48, 51, + 57, 60, 62, 68, 71, 73, 77, 81, 54, 51, 50, 49, 50, 49, 51, 54, 60, 63, + 65, 71, 75, 77, 82, 87, 59, 56, 54, 54, 54, 53, 55, 58, 64, 67, 69, 75, + 79, 81, 87, 92], + /* Size 32x32 */ + [32, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 34, 34, 35, 36, 36, + 38, 39, 39, 42, 44, 44, 47, 48, 49, 53, 54, 55, 59, 59, 31, 31, 31, 31, + 32, 32, 32, 32, 32, 32, 32, 32, 33, 34, 34, 34, 35, 35, 37, 39, 39, 41, + 43, 43, 46, 47, 48, 51, 52, 53, 57, 57, 31, 31, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 33, 34, 34, 34, 35, 35, 37, 38, 38, 41, 42, 43, 45, 46, + 47, 51, 51, 53, 56, 56, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 33, 34, 34, 34, 35, 35, 37, 38, 38, 41, 42, 42, 45, 46, 47, 51, 51, 52, + 56, 56, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 34, + 34, 34, 36, 37, 37, 40, 41, 41, 44, 45, 46, 49, 50, 51, 54, 54, 31, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 34, 34, 34, 36, 37, + 37, 40, 41, 41, 44, 44, 45, 49, 49, 50, 54, 54, 31, 32, 32, 32, 32, 32, + 32, 32, 32, 33, 33, 33, 33, 34, 34, 34, 35, 35, 36, 38, 38, 40, 41, 41, + 44, 45, 45, 49, 49, 50, 54, 54, 31, 32, 32, 32, 32, 32, 32, 33, 33, 33, + 34, 34, 34, 35, 35, 35, 36, 36, 38, 39, 39, 41, 42, 42, 44, 45, 46, 49, + 50, 51, 54, 54, 31, 32, 32, 32, 32, 32, 32, 33, 33, 33, 34, 34, 34, 35, + 35, 36, 36, 36, 38, 39, 39, 41, 42, 42, 45, 45, 46, 49, 50, 51, 54, 54, + 32, 32, 32, 32, 32, 32, 33, 33, 33, 34, 34, 34, 35, 35, 35, 36, 37, 37, + 38, 39, 39, 41, 42, 42, 45, 45, 46, 49, 49, 51, 54, 54, 32, 32, 32, 32, + 32, 32, 33, 34, 34, 34, 35, 35, 36, 37, 37, 37, 38, 38, 39, 40, 40, 42, + 42, 43, 45, 46, 46, 49, 49, 50, 53, 53, 32, 32, 32, 32, 32, 32, 33, 34, + 34, 34, 35, 35, 36, 37, 37, 37, 38, 38, 39, 40, 40, 42, 42, 43, 45, 46, + 46, 49, 49, 50, 53, 53, 32, 33, 33, 33, 33, 33, 33, 34, 34, 35, 36, 36, + 36, 38, 38, 39, 40, 40, 41, 42, 42, 44, 44, 45, 47, 47, 48, 51, 51, 52, + 55, 55, 34, 34, 34, 34, 33, 33, 34, 35, 35, 35, 37, 37, 38, 39, 39, 41, + 42, 42, 44, 45, 45, 47, 47, 48, 50, 51, 51, 54, 54, 55, 58, 58, 34, 34, + 34, 34, 33, 33, 34, 35, 35, 35, 37, 37, 38, 39, 39, 41, 42, 42, 44, 45, + 45, 47, 47, 48, 50, 51, 51, 54, 54, 55, 58, 58, 35, 34, 34, 34, 34, 34, + 34, 35, 36, 36, 37, 37, 39, 41, 41, 43, 45, 45, 47, 47, 47, 49, 50, 51, + 53, 53, 54, 57, 57, 58, 61, 61, 36, 35, 35, 35, 34, 34, 35, 36, 36, 37, + 38, 38, 40, 42, 42, 45, 48, 48, 50, 50, 50, 53, 54, 54, 56, 57, 57, 59, + 60, 61, 64, 64, 36, 35, 35, 35, 34, 34, 35, 36, 36, 37, 38, 38, 40, 42, + 42, 45, 48, 48, 50, 50, 50, 53, 54, 54, 56, 57, 57, 59, 60, 61, 64, 64, + 38, 37, 37, 37, 36, 36, 36, 38, 38, 38, 39, 39, 41, 44, 44, 47, 50, 50, + 51, 52, 52, 55, 56, 56, 58, 59, 60, 62, 63, 64, 67, 67, 39, 39, 38, 38, + 37, 37, 38, 39, 39, 39, 40, 40, 42, 45, 45, 47, 50, 50, 52, 54, 54, 56, + 58, 58, 60, 61, 62, 64, 65, 66, 69, 69, 39, 39, 38, 38, 37, 37, 38, 39, + 39, 39, 40, 40, 42, 45, 45, 47, 50, 50, 52, 54, 54, 56, 58, 58, 60, 61, + 62, 64, 65, 66, 69, 69, 42, 41, 41, 41, 40, 40, 40, 41, 41, 41, 42, 42, + 44, 47, 47, 49, 53, 53, 55, 56, 56, 60, 61, 62, 64, 65, 66, 69, 69, 70, + 73, 73, 44, 43, 42, 42, 41, 41, 41, 42, 42, 42, 42, 42, 44, 47, 47, 50, + 54, 54, 56, 58, 58, 61, 63, 64, 66, 67, 68, 71, 71, 72, 75, 75, 44, 43, + 43, 42, 41, 41, 41, 42, 42, 42, 43, 43, 45, 48, 48, 51, 54, 54, 56, 58, + 58, 62, 64, 64, 66, 67, 68, 71, 72, 73, 76, 76, 47, 46, 45, 45, 44, 44, + 44, 44, 45, 45, 45, 45, 47, 50, 50, 53, 56, 56, 58, 60, 60, 64, 66, 66, + 69, 70, 71, 74, 75, 76, 79, 79, 48, 47, 46, 46, 45, 44, 45, 45, 45, 45, + 46, 46, 47, 51, 51, 53, 57, 57, 59, 61, 61, 65, 67, 67, 70, 71, 72, 75, + 76, 77, 80, 80, 49, 48, 47, 47, 46, 45, 45, 46, 46, 46, 46, 46, 48, 51, + 51, 54, 57, 57, 60, 62, 62, 66, 68, 68, 71, 72, 73, 77, 77, 78, 81, 81, + 53, 51, 51, 51, 49, 49, 49, 49, 49, 49, 49, 49, 51, 54, 54, 57, 59, 59, + 62, 64, 64, 69, 71, 71, 74, 75, 77, 81, 81, 83, 86, 86, 54, 52, 51, 51, + 50, 49, 49, 50, 50, 49, 49, 49, 51, 54, 54, 57, 60, 60, 63, 65, 65, 69, + 71, 72, 75, 76, 77, 81, 82, 83, 87, 87, 55, 53, 53, 52, 51, 50, 50, 51, + 51, 51, 50, 50, 52, 55, 55, 58, 61, 61, 64, 66, 66, 70, 72, 73, 76, 77, + 78, 83, 83, 85, 88, 88, 59, 57, 56, 56, 54, 54, 54, 54, 54, 54, 53, 53, + 55, 58, 58, 61, 64, 64, 67, 69, 69, 73, 75, 76, 79, 80, 81, 86, 87, 88, + 92, 92, 59, 57, 56, 56, 54, 54, 54, 54, 54, 54, 53, 53, 55, 58, 58, 61, + 64, 64, 67, 69, 69, 73, 75, 76, 79, 80, 81, 86, 87, 88, 92, 92], + /* Size 4x8 */ + [32, 32, 37, 52, 32, 33, 36, 49, 32, 34, 38, 49, 34, 37, 44, 54, 35, 38, + 49, 60, 40, 42, 55, 69, 46, 46, 59, 76, 52, 51, 64, 83], + /* Size 8x4 */ + [32, 32, 32, 34, 35, 40, 46, 52, 32, 33, 34, 37, 38, 42, 46, 51, 37, 36, + 38, 44, 49, 55, 59, 64, 52, 49, 49, 54, 60, 69, 76, 83], + /* Size 8x16 */ + [32, 31, 32, 32, 36, 44, 47, 53, 31, 32, 32, 33, 35, 42, 45, 51, 31, 32, + 32, 33, 35, 41, 44, 49, 31, 32, 33, 33, 35, 41, 44, 49, 32, 32, 34, 34, + 36, 42, 45, 50, 32, 33, 35, 36, 38, 42, 45, 49, 32, 33, 35, 36, 40, 44, + 47, 51, 34, 34, 36, 38, 42, 48, 50, 54, 36, 34, 37, 40, 48, 54, 56, 60, + 38, 36, 39, 41, 49, 56, 58, 63, 39, 37, 40, 42, 50, 58, 60, 65, 44, 41, + 42, 45, 53, 63, 66, 71, 47, 44, 45, 47, 56, 66, 69, 75, 49, 46, 47, 48, + 57, 67, 71, 77, 53, 49, 50, 51, 60, 71, 75, 82, 58, 54, 54, 55, 63, 75, + 79, 87], + /* Size 16x8 */ + [32, 31, 31, 31, 32, 32, 32, 34, 36, 38, 39, 44, 47, 49, 53, 58, 31, 32, + 32, 32, 32, 33, 33, 34, 34, 36, 37, 41, 44, 46, 49, 54, 32, 32, 32, 33, + 34, 35, 35, 36, 37, 39, 40, 42, 45, 47, 50, 54, 32, 33, 33, 33, 34, 36, + 36, 38, 40, 41, 42, 45, 47, 48, 51, 55, 36, 35, 35, 35, 36, 38, 40, 42, + 48, 49, 50, 53, 56, 57, 60, 63, 44, 42, 41, 41, 42, 42, 44, 48, 54, 56, + 58, 63, 66, 67, 71, 75, 47, 45, 44, 44, 45, 45, 47, 50, 56, 58, 60, 66, + 69, 71, 75, 79, 53, 51, 49, 49, 50, 49, 51, 54, 60, 63, 65, 71, 75, 77, + 82, 87], + /* Size 16x32 */ + [32, 31, 31, 31, 32, 32, 32, 35, 36, 38, 44, 44, 47, 53, 53, 59, 31, 32, + 32, 32, 32, 32, 33, 35, 35, 37, 43, 43, 46, 52, 52, 57, 31, 32, 32, 32, + 32, 32, 33, 35, 35, 37, 42, 42, 45, 51, 51, 56, 31, 32, 32, 32, 32, 32, + 33, 35, 35, 37, 42, 42, 45, 51, 51, 56, 31, 32, 32, 32, 32, 32, 33, 34, + 35, 36, 41, 41, 44, 49, 49, 54, 31, 32, 32, 32, 32, 33, 33, 34, 34, 36, + 41, 41, 44, 49, 49, 54, 31, 32, 32, 32, 33, 33, 33, 35, 35, 36, 41, 41, + 44, 49, 49, 54, 32, 32, 32, 32, 33, 34, 34, 36, 36, 38, 42, 42, 45, 49, + 49, 54, 32, 32, 32, 33, 34, 34, 34, 36, 36, 38, 42, 42, 45, 50, 50, 54, + 32, 32, 32, 33, 34, 34, 35, 37, 37, 38, 42, 42, 45, 49, 49, 54, 32, 32, + 33, 33, 35, 35, 36, 38, 38, 39, 42, 42, 45, 49, 49, 53, 32, 32, 33, 33, + 35, 35, 36, 38, 38, 39, 42, 42, 45, 49, 49, 53, 32, 33, 33, 33, 35, 36, + 36, 39, 40, 41, 44, 44, 47, 51, 51, 55, 34, 34, 34, 34, 36, 37, 38, 42, + 42, 44, 48, 48, 50, 54, 54, 58, 34, 34, 34, 34, 36, 37, 38, 42, 42, 44, + 48, 48, 50, 54, 54, 58, 35, 34, 34, 34, 37, 37, 39, 44, 45, 46, 50, 50, + 53, 57, 57, 61, 36, 35, 34, 35, 37, 38, 40, 47, 48, 49, 54, 54, 56, 60, + 60, 64, 36, 35, 34, 35, 37, 38, 40, 47, 48, 49, 54, 54, 56, 60, 60, 64, + 38, 37, 36, 37, 39, 40, 41, 48, 49, 51, 56, 56, 58, 63, 63, 67, 39, 38, + 37, 38, 40, 40, 42, 49, 50, 52, 58, 58, 60, 65, 65, 69, 39, 38, 37, 38, + 40, 40, 42, 49, 50, 52, 58, 58, 60, 65, 65, 69, 42, 40, 40, 40, 42, 42, + 44, 51, 52, 55, 61, 61, 64, 69, 69, 73, 44, 42, 41, 41, 42, 43, 45, 52, + 53, 56, 63, 63, 66, 71, 71, 75, 44, 42, 41, 41, 43, 43, 45, 52, 54, 56, + 63, 63, 66, 72, 72, 76, 47, 45, 44, 44, 45, 45, 47, 54, 56, 58, 66, 66, + 69, 75, 75, 79, 48, 46, 45, 45, 46, 46, 48, 55, 56, 59, 67, 67, 70, 76, + 76, 80, 49, 47, 46, 46, 47, 47, 48, 56, 57, 60, 67, 67, 71, 77, 77, 81, + 53, 50, 49, 49, 49, 49, 51, 58, 59, 62, 71, 71, 74, 81, 81, 86, 53, 51, + 49, 49, 50, 50, 51, 59, 60, 63, 71, 71, 75, 82, 82, 87, 55, 52, 51, 51, + 51, 51, 53, 60, 61, 64, 72, 72, 76, 83, 83, 88, 58, 55, 54, 54, 54, 54, + 55, 62, 63, 67, 75, 75, 79, 87, 87, 92, 58, 55, 54, 54, 54, 54, 55, 62, + 63, 67, 75, 75, 79, 87, 87, 92], + /* Size 32x16 */ + [32, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 34, 34, 35, 36, 36, + 38, 39, 39, 42, 44, 44, 47, 48, 49, 53, 53, 55, 58, 58, 31, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 33, 34, 34, 34, 35, 35, 37, 38, 38, 40, + 42, 42, 45, 46, 47, 50, 51, 52, 55, 55, 31, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 33, 33, 33, 34, 34, 34, 34, 34, 36, 37, 37, 40, 41, 41, 44, 45, + 46, 49, 49, 51, 54, 54, 31, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 33, + 33, 34, 34, 34, 35, 35, 37, 38, 38, 40, 41, 41, 44, 45, 46, 49, 49, 51, + 54, 54, 32, 32, 32, 32, 32, 32, 33, 33, 34, 34, 35, 35, 35, 36, 36, 37, + 37, 37, 39, 40, 40, 42, 42, 43, 45, 46, 47, 49, 50, 51, 54, 54, 32, 32, + 32, 32, 32, 33, 33, 34, 34, 34, 35, 35, 36, 37, 37, 37, 38, 38, 40, 40, + 40, 42, 43, 43, 45, 46, 47, 49, 50, 51, 54, 54, 32, 33, 33, 33, 33, 33, + 33, 34, 34, 35, 36, 36, 36, 38, 38, 39, 40, 40, 41, 42, 42, 44, 45, 45, + 47, 48, 48, 51, 51, 53, 55, 55, 35, 35, 35, 35, 34, 34, 35, 36, 36, 37, + 38, 38, 39, 42, 42, 44, 47, 47, 48, 49, 49, 51, 52, 52, 54, 55, 56, 58, + 59, 60, 62, 62, 36, 35, 35, 35, 35, 34, 35, 36, 36, 37, 38, 38, 40, 42, + 42, 45, 48, 48, 49, 50, 50, 52, 53, 54, 56, 56, 57, 59, 60, 61, 63, 63, + 38, 37, 37, 37, 36, 36, 36, 38, 38, 38, 39, 39, 41, 44, 44, 46, 49, 49, + 51, 52, 52, 55, 56, 56, 58, 59, 60, 62, 63, 64, 67, 67, 44, 43, 42, 42, + 41, 41, 41, 42, 42, 42, 42, 42, 44, 48, 48, 50, 54, 54, 56, 58, 58, 61, + 63, 63, 66, 67, 67, 71, 71, 72, 75, 75, 44, 43, 42, 42, 41, 41, 41, 42, + 42, 42, 42, 42, 44, 48, 48, 50, 54, 54, 56, 58, 58, 61, 63, 63, 66, 67, + 67, 71, 71, 72, 75, 75, 47, 46, 45, 45, 44, 44, 44, 45, 45, 45, 45, 45, + 47, 50, 50, 53, 56, 56, 58, 60, 60, 64, 66, 66, 69, 70, 71, 74, 75, 76, + 79, 79, 53, 52, 51, 51, 49, 49, 49, 49, 50, 49, 49, 49, 51, 54, 54, 57, + 60, 60, 63, 65, 65, 69, 71, 72, 75, 76, 77, 81, 82, 83, 87, 87, 53, 52, + 51, 51, 49, 49, 49, 49, 50, 49, 49, 49, 51, 54, 54, 57, 60, 60, 63, 65, + 65, 69, 71, 72, 75, 76, 77, 81, 82, 83, 87, 87, 59, 57, 56, 56, 54, 54, + 54, 54, 54, 54, 53, 53, 55, 58, 58, 61, 64, 64, 67, 69, 69, 73, 75, 76, + 79, 80, 81, 86, 87, 88, 92, 92], + /* Size 4x16 */ + [31, 32, 38, 53, 32, 32, 37, 51, 32, 32, 36, 49, 32, 33, 36, 49, 32, 34, + 38, 50, 32, 35, 39, 49, 33, 36, 41, 51, 34, 37, 44, 54, 35, 38, 49, 60, + 37, 40, 51, 63, 38, 40, 52, 65, 42, 43, 56, 71, 45, 45, 58, 75, 47, 47, + 60, 77, 51, 50, 63, 82, 55, 54, 67, 87], + /* Size 16x4 */ + [31, 32, 32, 32, 32, 32, 33, 34, 35, 37, 38, 42, 45, 47, 51, 55, 32, 32, + 32, 33, 34, 35, 36, 37, 38, 40, 40, 43, 45, 47, 50, 54, 38, 37, 36, 36, + 38, 39, 41, 44, 49, 51, 52, 56, 58, 60, 63, 67, 53, 51, 49, 49, 50, 49, + 51, 54, 60, 63, 65, 71, 75, 77, 82, 87], + /* Size 8x32 */ + [32, 31, 32, 32, 36, 44, 47, 53, 31, 32, 32, 33, 35, 43, 46, 52, 31, 32, + 32, 33, 35, 42, 45, 51, 31, 32, 32, 33, 35, 42, 45, 51, 31, 32, 32, 33, + 35, 41, 44, 49, 31, 32, 32, 33, 34, 41, 44, 49, 31, 32, 33, 33, 35, 41, + 44, 49, 32, 32, 33, 34, 36, 42, 45, 49, 32, 32, 34, 34, 36, 42, 45, 50, + 32, 32, 34, 35, 37, 42, 45, 49, 32, 33, 35, 36, 38, 42, 45, 49, 32, 33, + 35, 36, 38, 42, 45, 49, 32, 33, 35, 36, 40, 44, 47, 51, 34, 34, 36, 38, + 42, 48, 50, 54, 34, 34, 36, 38, 42, 48, 50, 54, 35, 34, 37, 39, 45, 50, + 53, 57, 36, 34, 37, 40, 48, 54, 56, 60, 36, 34, 37, 40, 48, 54, 56, 60, + 38, 36, 39, 41, 49, 56, 58, 63, 39, 37, 40, 42, 50, 58, 60, 65, 39, 37, + 40, 42, 50, 58, 60, 65, 42, 40, 42, 44, 52, 61, 64, 69, 44, 41, 42, 45, + 53, 63, 66, 71, 44, 41, 43, 45, 54, 63, 66, 72, 47, 44, 45, 47, 56, 66, + 69, 75, 48, 45, 46, 48, 56, 67, 70, 76, 49, 46, 47, 48, 57, 67, 71, 77, + 53, 49, 49, 51, 59, 71, 74, 81, 53, 49, 50, 51, 60, 71, 75, 82, 55, 51, + 51, 53, 61, 72, 76, 83, 58, 54, 54, 55, 63, 75, 79, 87, 58, 54, 54, 55, + 63, 75, 79, 87], + /* Size 32x8 */ + [32, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 34, 34, 35, 36, 36, + 38, 39, 39, 42, 44, 44, 47, 48, 49, 53, 53, 55, 58, 58, 31, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 33, 33, 33, 34, 34, 34, 34, 34, 36, 37, 37, 40, + 41, 41, 44, 45, 46, 49, 49, 51, 54, 54, 32, 32, 32, 32, 32, 32, 33, 33, + 34, 34, 35, 35, 35, 36, 36, 37, 37, 37, 39, 40, 40, 42, 42, 43, 45, 46, + 47, 49, 50, 51, 54, 54, 32, 33, 33, 33, 33, 33, 33, 34, 34, 35, 36, 36, + 36, 38, 38, 39, 40, 40, 41, 42, 42, 44, 45, 45, 47, 48, 48, 51, 51, 53, + 55, 55, 36, 35, 35, 35, 35, 34, 35, 36, 36, 37, 38, 38, 40, 42, 42, 45, + 48, 48, 49, 50, 50, 52, 53, 54, 56, 56, 57, 59, 60, 61, 63, 63, 44, 43, + 42, 42, 41, 41, 41, 42, 42, 42, 42, 42, 44, 48, 48, 50, 54, 54, 56, 58, + 58, 61, 63, 63, 66, 67, 67, 71, 71, 72, 75, 75, 47, 46, 45, 45, 44, 44, + 44, 45, 45, 45, 45, 45, 47, 50, 50, 53, 56, 56, 58, 60, 60, 64, 66, 66, + 69, 70, 71, 74, 75, 76, 79, 79, 53, 52, 51, 51, 49, 49, 49, 49, 50, 49, + 49, 49, 51, 54, 54, 57, 60, 60, 63, 65, 65, 69, 71, 72, 75, 76, 77, 81, + 82, 83, 87, 87]], + [ /* Chroma */ + /* Size 4x4 */ + [31, 38, 47, 49, 38, 47, 46, 46, 47, 46, 54, 57, 49, 46, 57, 66], + /* Size 8x8 */ + [31, 31, 35, 42, 48, 47, 49, 51, 31, 32, 36, 42, 46, 45, 46, 48, 35, 36, + 41, 45, 47, 45, 46, 48, 42, 42, 45, 48, 50, 49, 50, 51, 48, 46, 47, 50, + 53, 53, 54, 54, 47, 45, 45, 49, 53, 57, 59, 60, 49, 46, 46, 50, 54, 59, + 61, 64, 51, 48, 48, 51, 54, 60, 64, 68], + /* Size 16x16 */ + [32, 31, 30, 31, 33, 36, 38, 41, 49, 49, 48, 49, 50, 51, 52, 54, 31, 31, + 31, 32, 34, 38, 40, 42, 47, 47, 47, 47, 48, 48, 50, 52, 30, 31, 31, 32, + 35, 39, 41, 42, 46, 46, 46, 45, 46, 47, 48, 50, 31, 32, 32, 33, 36, 40, + 41, 43, 46, 46, 45, 45, 46, 46, 47, 49, 33, 34, 35, 36, 39, 43, 44, 45, + 47, 46, 46, 45, 46, 47, 47, 49, 36, 38, 39, 40, 43, 47, 47, 47, 48, 47, + 46, 45, 46, 46, 47, 48, 38, 40, 41, 41, 44, 47, 47, 48, 49, 48, 48, 47, + 47, 47, 48, 49, 41, 42, 42, 43, 45, 47, 48, 48, 50, 50, 49, 49, 50, 50, + 50, 52, 49, 47, 46, 46, 47, 48, 49, 50, 53, 53, 53, 53, 54, 54, 54, 55, + 49, 47, 46, 46, 46, 47, 48, 50, 53, 53, 54, 55, 55, 55, 56, 57, 48, 47, + 46, 45, 46, 46, 48, 49, 53, 54, 54, 55, 56, 56, 57, 58, 49, 47, 45, 45, + 45, 45, 47, 49, 53, 55, 55, 58, 59, 60, 61, 62, 50, 48, 46, 46, 46, 46, + 47, 50, 54, 55, 56, 59, 61, 61, 63, 64, 51, 48, 47, 46, 47, 46, 47, 50, + 54, 55, 56, 60, 61, 62, 64, 66, 52, 50, 48, 47, 47, 47, 48, 50, 54, 56, + 57, 61, 63, 64, 66, 68, 54, 52, 50, 49, 49, 48, 49, 52, 55, 57, 58, 62, + 64, 66, 68, 71], + /* Size 32x32 */ + [32, 31, 31, 31, 30, 30, 31, 33, 33, 34, 36, 36, 38, 41, 41, 45, 49, 49, + 49, 48, 48, 49, 49, 49, 50, 50, 51, 52, 52, 53, 54, 54, 31, 31, 31, 31, + 31, 31, 31, 34, 34, 35, 38, 38, 39, 42, 42, 45, 48, 48, 47, 47, 47, 47, + 47, 47, 49, 49, 49, 50, 50, 51, 53, 53, 31, 31, 31, 31, 31, 31, 32, 34, + 34, 35, 38, 38, 40, 42, 42, 45, 47, 47, 47, 47, 47, 47, 47, 47, 48, 48, + 48, 49, 50, 50, 52, 52, 31, 31, 31, 31, 31, 31, 32, 34, 34, 36, 38, 38, + 40, 42, 42, 45, 47, 47, 47, 47, 47, 47, 46, 47, 48, 48, 48, 49, 49, 50, + 52, 52, 30, 31, 31, 31, 31, 31, 32, 35, 35, 36, 39, 39, 41, 42, 42, 44, + 46, 46, 46, 46, 46, 45, 45, 45, 46, 47, 47, 48, 48, 48, 50, 50, 30, 31, + 31, 31, 31, 32, 32, 35, 35, 36, 40, 40, 41, 42, 42, 44, 46, 46, 46, 45, + 45, 45, 45, 45, 46, 46, 46, 47, 47, 48, 49, 49, 31, 31, 32, 32, 32, 32, + 33, 35, 36, 37, 40, 40, 41, 43, 43, 44, 46, 46, 46, 45, 45, 45, 45, 45, + 46, 46, 46, 47, 47, 48, 49, 49, 33, 34, 34, 34, 35, 35, 35, 38, 38, 40, + 43, 43, 43, 44, 44, 46, 47, 47, 46, 46, 46, 45, 45, 45, 46, 46, 47, 47, + 47, 48, 49, 49, 33, 34, 34, 34, 35, 35, 36, 38, 39, 40, 43, 43, 44, 45, + 45, 46, 47, 47, 46, 46, 46, 45, 45, 45, 46, 46, 47, 47, 47, 48, 49, 49, + 34, 35, 35, 36, 36, 36, 37, 40, 40, 41, 44, 44, 45, 45, 45, 46, 47, 47, + 47, 46, 46, 45, 45, 45, 46, 46, 46, 47, 47, 48, 49, 49, 36, 38, 38, 38, + 39, 40, 40, 43, 43, 44, 47, 47, 47, 47, 47, 47, 48, 48, 47, 46, 46, 45, + 45, 45, 46, 46, 46, 46, 47, 47, 48, 48, 36, 38, 38, 38, 39, 40, 40, 43, + 43, 44, 47, 47, 47, 47, 47, 47, 48, 48, 47, 46, 46, 45, 45, 45, 46, 46, + 46, 46, 47, 47, 48, 48, 38, 39, 40, 40, 41, 41, 41, 43, 44, 45, 47, 47, + 47, 48, 48, 48, 49, 49, 48, 48, 48, 47, 47, 47, 47, 47, 47, 48, 48, 48, + 49, 49, 41, 42, 42, 42, 42, 42, 43, 44, 45, 45, 47, 47, 48, 48, 48, 49, + 50, 50, 50, 49, 49, 49, 49, 49, 50, 50, 50, 50, 50, 51, 52, 52, 41, 42, + 42, 42, 42, 42, 43, 44, 45, 45, 47, 47, 48, 48, 48, 49, 50, 50, 50, 49, + 49, 49, 49, 49, 50, 50, 50, 50, 50, 51, 52, 52, 45, 45, 45, 45, 44, 44, + 44, 46, 46, 46, 47, 47, 48, 49, 49, 50, 51, 51, 51, 51, 51, 51, 51, 51, + 52, 52, 52, 52, 52, 52, 53, 53, 49, 48, 47, 47, 46, 46, 46, 47, 47, 47, + 48, 48, 49, 50, 50, 51, 53, 53, 53, 53, 53, 53, 53, 53, 54, 54, 54, 54, + 54, 54, 55, 55, 49, 48, 47, 47, 46, 46, 46, 47, 47, 47, 48, 48, 49, 50, + 50, 51, 53, 53, 53, 53, 53, 53, 53, 53, 54, 54, 54, 54, 54, 54, 55, 55, + 49, 47, 47, 47, 46, 46, 46, 46, 46, 47, 47, 47, 48, 50, 50, 51, 53, 53, + 53, 54, 54, 54, 55, 55, 55, 55, 55, 56, 56, 56, 57, 57, 48, 47, 47, 47, + 46, 45, 45, 46, 46, 46, 46, 46, 48, 49, 49, 51, 53, 53, 54, 54, 54, 55, + 55, 56, 56, 56, 56, 57, 57, 58, 58, 58, 48, 47, 47, 47, 46, 45, 45, 46, + 46, 46, 46, 46, 48, 49, 49, 51, 53, 53, 54, 54, 54, 55, 55, 56, 56, 56, + 56, 57, 57, 58, 58, 58, 49, 47, 47, 47, 45, 45, 45, 45, 45, 45, 45, 45, + 47, 49, 49, 51, 53, 53, 54, 55, 55, 57, 57, 58, 58, 59, 59, 60, 60, 60, + 61, 61, 49, 47, 47, 46, 45, 45, 45, 45, 45, 45, 45, 45, 47, 49, 49, 51, + 53, 53, 55, 55, 55, 57, 58, 58, 59, 60, 60, 61, 61, 61, 62, 62, 49, 47, + 47, 47, 45, 45, 45, 45, 45, 45, 45, 45, 47, 49, 49, 51, 53, 53, 55, 56, + 56, 58, 58, 59, 59, 60, 60, 61, 61, 62, 63, 63, 50, 49, 48, 48, 46, 46, + 46, 46, 46, 46, 46, 46, 47, 50, 50, 52, 54, 54, 55, 56, 56, 58, 59, 59, + 61, 61, 61, 63, 63, 63, 64, 64, 50, 49, 48, 48, 47, 46, 46, 46, 46, 46, + 46, 46, 47, 50, 50, 52, 54, 54, 55, 56, 56, 59, 60, 60, 61, 61, 62, 63, + 63, 64, 65, 65, 51, 49, 48, 48, 47, 46, 46, 47, 47, 46, 46, 46, 47, 50, + 50, 52, 54, 54, 55, 56, 56, 59, 60, 60, 61, 62, 62, 64, 64, 64, 66, 66, + 52, 50, 49, 49, 48, 47, 47, 47, 47, 47, 46, 46, 48, 50, 50, 52, 54, 54, + 56, 57, 57, 60, 61, 61, 63, 63, 64, 66, 66, 67, 68, 68, 52, 50, 50, 49, + 48, 47, 47, 47, 47, 47, 47, 47, 48, 50, 50, 52, 54, 54, 56, 57, 57, 60, + 61, 61, 63, 63, 64, 66, 66, 67, 68, 68, 53, 51, 50, 50, 48, 48, 48, 48, + 48, 48, 47, 47, 48, 51, 51, 52, 54, 54, 56, 58, 58, 60, 61, 62, 63, 64, + 64, 67, 67, 68, 69, 69, 54, 53, 52, 52, 50, 49, 49, 49, 49, 49, 48, 48, + 49, 52, 52, 53, 55, 55, 57, 58, 58, 61, 62, 63, 64, 65, 66, 68, 68, 69, + 71, 71, 54, 53, 52, 52, 50, 49, 49, 49, 49, 49, 48, 48, 49, 52, 52, 53, + 55, 55, 57, 58, 58, 61, 62, 63, 64, 65, 66, 68, 68, 69, 71, 71], + /* Size 4x8 */ + [31, 38, 47, 50, 31, 40, 46, 48, 36, 44, 47, 47, 42, 47, 50, 50, 47, 48, + 53, 54, 46, 46, 54, 60, 48, 46, 55, 64, 50, 48, 56, 67], + /* Size 8x4 */ + [31, 31, 36, 42, 47, 46, 48, 50, 38, 40, 44, 47, 48, 46, 46, 48, 47, 46, + 47, 50, 53, 54, 55, 56, 50, 48, 47, 50, 54, 60, 64, 67], + /* Size 8x16 */ + [32, 31, 35, 38, 48, 49, 50, 52, 31, 31, 37, 40, 47, 47, 48, 50, 30, 32, + 38, 40, 46, 45, 46, 48, 31, 33, 38, 41, 46, 45, 46, 48, 33, 36, 41, 44, + 47, 46, 46, 47, 37, 40, 45, 47, 47, 45, 46, 47, 39, 41, 46, 47, 48, 47, + 47, 48, 42, 43, 46, 48, 50, 49, 50, 50, 49, 46, 48, 49, 53, 53, 54, 54, + 48, 46, 47, 48, 53, 55, 55, 56, 48, 46, 46, 48, 53, 56, 56, 57, 49, 45, + 45, 47, 53, 58, 59, 61, 50, 46, 46, 48, 54, 59, 61, 63, 51, 47, 47, 48, + 54, 60, 61, 64, 52, 48, 47, 48, 54, 61, 63, 66, 54, 50, 49, 50, 55, 62, + 65, 68], + /* Size 16x8 */ + [32, 31, 30, 31, 33, 37, 39, 42, 49, 48, 48, 49, 50, 51, 52, 54, 31, 31, + 32, 33, 36, 40, 41, 43, 46, 46, 46, 45, 46, 47, 48, 50, 35, 37, 38, 38, + 41, 45, 46, 46, 48, 47, 46, 45, 46, 47, 47, 49, 38, 40, 40, 41, 44, 47, + 47, 48, 49, 48, 48, 47, 48, 48, 48, 50, 48, 47, 46, 46, 47, 47, 48, 50, + 53, 53, 53, 53, 54, 54, 54, 55, 49, 47, 45, 45, 46, 45, 47, 49, 53, 55, + 56, 58, 59, 60, 61, 62, 50, 48, 46, 46, 46, 46, 47, 50, 54, 55, 56, 59, + 61, 61, 63, 65, 52, 50, 48, 48, 47, 47, 48, 50, 54, 56, 57, 61, 63, 64, + 66, 68], + /* Size 16x32 */ + [32, 31, 31, 31, 35, 37, 38, 47, 48, 48, 49, 49, 50, 52, 52, 54, 31, 31, + 31, 32, 36, 38, 39, 46, 47, 47, 48, 48, 49, 50, 50, 53, 31, 31, 31, 32, + 37, 38, 40, 46, 47, 47, 47, 47, 48, 50, 50, 52, 31, 31, 31, 32, 37, 38, + 40, 46, 47, 47, 47, 47, 48, 50, 50, 52, 30, 31, 32, 32, 38, 39, 40, 45, + 46, 46, 45, 45, 46, 48, 48, 50, 30, 31, 32, 33, 38, 40, 41, 45, 46, 46, + 45, 45, 46, 48, 48, 50, 31, 32, 33, 33, 38, 40, 41, 45, 46, 46, 45, 45, + 46, 48, 48, 50, 33, 35, 35, 36, 41, 43, 43, 46, 47, 46, 45, 45, 46, 47, + 47, 49, 33, 35, 36, 36, 41, 43, 44, 46, 47, 46, 46, 46, 46, 47, 47, 49, + 34, 36, 37, 37, 42, 44, 45, 47, 47, 47, 45, 45, 46, 47, 47, 49, 37, 39, + 40, 41, 45, 47, 47, 47, 47, 47, 45, 45, 46, 47, 47, 48, 37, 39, 40, 41, + 45, 47, 47, 47, 47, 47, 45, 45, 46, 47, 47, 48, 39, 40, 41, 42, 46, 47, + 47, 48, 48, 48, 47, 47, 47, 48, 48, 50, 42, 42, 43, 43, 46, 47, 48, 50, + 50, 50, 49, 49, 50, 50, 50, 52, 42, 42, 43, 43, 46, 47, 48, 50, 50, 50, + 49, 49, 50, 50, 50, 52, 45, 45, 44, 45, 47, 47, 48, 51, 51, 51, 51, 51, + 52, 52, 52, 54, 49, 47, 46, 47, 48, 48, 49, 52, 53, 53, 53, 53, 54, 54, + 54, 55, 49, 47, 46, 47, 48, 48, 49, 52, 53, 53, 53, 53, 54, 54, 54, 55, + 48, 47, 46, 46, 47, 47, 48, 52, 53, 53, 55, 55, 55, 56, 56, 57, 48, 46, + 46, 46, 46, 47, 48, 52, 53, 54, 56, 56, 56, 57, 57, 59, 48, 46, 46, 46, + 46, 47, 48, 52, 53, 54, 56, 56, 56, 57, 57, 59, 49, 46, 45, 45, 46, 46, + 47, 52, 53, 54, 57, 57, 58, 60, 60, 61, 49, 46, 45, 45, 45, 46, 47, 52, + 53, 55, 58, 58, 59, 61, 61, 62, 49, 46, 45, 45, 46, 46, 47, 52, 53, 55, + 58, 58, 60, 61, 61, 63, 50, 47, 46, 46, 46, 46, 48, 53, 54, 55, 59, 59, + 61, 63, 63, 65, 50, 48, 46, 46, 46, 46, 48, 53, 54, 55, 59, 59, 61, 64, + 64, 65, 51, 48, 47, 47, 47, 47, 48, 53, 54, 55, 60, 60, 61, 64, 64, 66, + 52, 49, 48, 48, 47, 47, 48, 53, 54, 56, 61, 61, 63, 66, 66, 68, 52, 49, + 48, 48, 47, 47, 48, 53, 54, 56, 61, 61, 63, 66, 66, 68, 53, 50, 48, 48, + 48, 48, 49, 54, 54, 56, 61, 61, 63, 67, 67, 69, 54, 51, 50, 50, 49, 49, + 50, 55, 55, 57, 62, 62, 65, 68, 68, 71, 54, 51, 50, 50, 49, 49, 50, 55, + 55, 57, 62, 62, 65, 68, 68, 71], + /* Size 32x16 */ + [32, 31, 31, 31, 30, 30, 31, 33, 33, 34, 37, 37, 39, 42, 42, 45, 49, 49, + 48, 48, 48, 49, 49, 49, 50, 50, 51, 52, 52, 53, 54, 54, 31, 31, 31, 31, + 31, 31, 32, 35, 35, 36, 39, 39, 40, 42, 42, 45, 47, 47, 47, 46, 46, 46, + 46, 46, 47, 48, 48, 49, 49, 50, 51, 51, 31, 31, 31, 31, 32, 32, 33, 35, + 36, 37, 40, 40, 41, 43, 43, 44, 46, 46, 46, 46, 46, 45, 45, 45, 46, 46, + 47, 48, 48, 48, 50, 50, 31, 32, 32, 32, 32, 33, 33, 36, 36, 37, 41, 41, + 42, 43, 43, 45, 47, 47, 46, 46, 46, 45, 45, 45, 46, 46, 47, 48, 48, 48, + 50, 50, 35, 36, 37, 37, 38, 38, 38, 41, 41, 42, 45, 45, 46, 46, 46, 47, + 48, 48, 47, 46, 46, 46, 45, 46, 46, 46, 47, 47, 47, 48, 49, 49, 37, 38, + 38, 38, 39, 40, 40, 43, 43, 44, 47, 47, 47, 47, 47, 47, 48, 48, 47, 47, + 47, 46, 46, 46, 46, 46, 47, 47, 47, 48, 49, 49, 38, 39, 40, 40, 40, 41, + 41, 43, 44, 45, 47, 47, 47, 48, 48, 48, 49, 49, 48, 48, 48, 47, 47, 47, + 48, 48, 48, 48, 48, 49, 50, 50, 47, 46, 46, 46, 45, 45, 45, 46, 46, 47, + 47, 47, 48, 50, 50, 51, 52, 52, 52, 52, 52, 52, 52, 52, 53, 53, 53, 53, + 53, 54, 55, 55, 48, 47, 47, 47, 46, 46, 46, 47, 47, 47, 47, 47, 48, 50, + 50, 51, 53, 53, 53, 53, 53, 53, 53, 53, 54, 54, 54, 54, 54, 54, 55, 55, + 48, 47, 47, 47, 46, 46, 46, 46, 46, 47, 47, 47, 48, 50, 50, 51, 53, 53, + 53, 54, 54, 54, 55, 55, 55, 55, 55, 56, 56, 56, 57, 57, 49, 48, 47, 47, + 45, 45, 45, 45, 46, 45, 45, 45, 47, 49, 49, 51, 53, 53, 55, 56, 56, 57, + 58, 58, 59, 59, 60, 61, 61, 61, 62, 62, 49, 48, 47, 47, 45, 45, 45, 45, + 46, 45, 45, 45, 47, 49, 49, 51, 53, 53, 55, 56, 56, 57, 58, 58, 59, 59, + 60, 61, 61, 61, 62, 62, 50, 49, 48, 48, 46, 46, 46, 46, 46, 46, 46, 46, + 47, 50, 50, 52, 54, 54, 55, 56, 56, 58, 59, 60, 61, 61, 61, 63, 63, 63, + 65, 65, 52, 50, 50, 50, 48, 48, 48, 47, 47, 47, 47, 47, 48, 50, 50, 52, + 54, 54, 56, 57, 57, 60, 61, 61, 63, 64, 64, 66, 66, 67, 68, 68, 52, 50, + 50, 50, 48, 48, 48, 47, 47, 47, 47, 47, 48, 50, 50, 52, 54, 54, 56, 57, + 57, 60, 61, 61, 63, 64, 64, 66, 66, 67, 68, 68, 54, 53, 52, 52, 50, 50, + 50, 49, 49, 49, 48, 48, 50, 52, 52, 54, 55, 55, 57, 59, 59, 61, 62, 63, + 65, 65, 66, 68, 68, 69, 71, 71], + /* Size 4x16 */ + [31, 37, 48, 52, 31, 38, 47, 50, 31, 39, 46, 48, 32, 40, 46, 48, 35, 43, + 46, 47, 39, 47, 47, 47, 40, 47, 48, 48, 42, 47, 50, 50, 47, 48, 53, 54, + 47, 47, 53, 56, 46, 47, 54, 57, 46, 46, 55, 61, 47, 46, 55, 63, 48, 47, + 55, 64, 49, 47, 56, 66, 51, 49, 57, 68], + /* Size 16x4 */ + [31, 31, 31, 32, 35, 39, 40, 42, 47, 47, 46, 46, 47, 48, 49, 51, 37, 38, + 39, 40, 43, 47, 47, 47, 48, 47, 47, 46, 46, 47, 47, 49, 48, 47, 46, 46, + 46, 47, 48, 50, 53, 53, 54, 55, 55, 55, 56, 57, 52, 50, 48, 48, 47, 47, + 48, 50, 54, 56, 57, 61, 63, 64, 66, 68], + /* Size 8x32 */ + [32, 31, 35, 38, 48, 49, 50, 52, 31, 31, 36, 39, 47, 48, 49, 50, 31, 31, + 37, 40, 47, 47, 48, 50, 31, 31, 37, 40, 47, 47, 48, 50, 30, 32, 38, 40, + 46, 45, 46, 48, 30, 32, 38, 41, 46, 45, 46, 48, 31, 33, 38, 41, 46, 45, + 46, 48, 33, 35, 41, 43, 47, 45, 46, 47, 33, 36, 41, 44, 47, 46, 46, 47, + 34, 37, 42, 45, 47, 45, 46, 47, 37, 40, 45, 47, 47, 45, 46, 47, 37, 40, + 45, 47, 47, 45, 46, 47, 39, 41, 46, 47, 48, 47, 47, 48, 42, 43, 46, 48, + 50, 49, 50, 50, 42, 43, 46, 48, 50, 49, 50, 50, 45, 44, 47, 48, 51, 51, + 52, 52, 49, 46, 48, 49, 53, 53, 54, 54, 49, 46, 48, 49, 53, 53, 54, 54, + 48, 46, 47, 48, 53, 55, 55, 56, 48, 46, 46, 48, 53, 56, 56, 57, 48, 46, + 46, 48, 53, 56, 56, 57, 49, 45, 46, 47, 53, 57, 58, 60, 49, 45, 45, 47, + 53, 58, 59, 61, 49, 45, 46, 47, 53, 58, 60, 61, 50, 46, 46, 48, 54, 59, + 61, 63, 50, 46, 46, 48, 54, 59, 61, 64, 51, 47, 47, 48, 54, 60, 61, 64, + 52, 48, 47, 48, 54, 61, 63, 66, 52, 48, 47, 48, 54, 61, 63, 66, 53, 48, + 48, 49, 54, 61, 63, 67, 54, 50, 49, 50, 55, 62, 65, 68, 54, 50, 49, 50, + 55, 62, 65, 68], + /* Size 32x8 */ + [32, 31, 31, 31, 30, 30, 31, 33, 33, 34, 37, 37, 39, 42, 42, 45, 49, 49, + 48, 48, 48, 49, 49, 49, 50, 50, 51, 52, 52, 53, 54, 54, 31, 31, 31, 31, + 32, 32, 33, 35, 36, 37, 40, 40, 41, 43, 43, 44, 46, 46, 46, 46, 46, 45, + 45, 45, 46, 46, 47, 48, 48, 48, 50, 50, 35, 36, 37, 37, 38, 38, 38, 41, + 41, 42, 45, 45, 46, 46, 46, 47, 48, 48, 47, 46, 46, 46, 45, 46, 46, 46, + 47, 47, 47, 48, 49, 49, 38, 39, 40, 40, 40, 41, 41, 43, 44, 45, 47, 47, + 47, 48, 48, 48, 49, 49, 48, 48, 48, 47, 47, 47, 48, 48, 48, 48, 48, 49, + 50, 50, 48, 47, 47, 47, 46, 46, 46, 47, 47, 47, 47, 47, 48, 50, 50, 51, + 53, 53, 53, 53, 53, 53, 53, 53, 54, 54, 54, 54, 54, 54, 55, 55, 49, 48, + 47, 47, 45, 45, 45, 45, 46, 45, 45, 45, 47, 49, 49, 51, 53, 53, 55, 56, + 56, 57, 58, 58, 59, 59, 60, 61, 61, 61, 62, 62, 50, 49, 48, 48, 46, 46, + 46, 46, 46, 46, 46, 46, 47, 50, 50, 52, 54, 54, 55, 56, 56, 58, 59, 60, + 61, 61, 61, 63, 63, 63, 65, 65, 52, 50, 50, 50, 48, 48, 48, 47, 47, 47, + 47, 47, 48, 50, 50, 52, 54, 54, 56, 57, 57, 60, 61, 61, 63, 64, 64, 66, + 66, 67, 68, 68]] + ], + [ + [ /* Luma */ + /* Size 4x4 */ + [32, 32, 35, 43, 32, 34, 37, 43, 35, 37, 48, 54, 43, 43, 54, 65], + /* Size 8x8 */ + [31, 31, 32, 32, 34, 37, 43, 47, 31, 32, 32, 32, 34, 36, 41, 44, 32, 32, + 33, 34, 35, 38, 42, 45, 32, 32, 34, 35, 37, 39, 42, 46, 34, 34, 35, 37, + 41, 45, 49, 52, 37, 36, 38, 39, 45, 51, 56, 59, 43, 41, 42, 42, 49, 56, + 63, 67, 47, 44, 45, 46, 52, 59, 67, 71], + /* Size 16x16 */ + [32, 31, 31, 31, 31, 31, 32, 32, 34, 35, 36, 39, 41, 44, 47, 48, 31, 32, + 32, 32, 32, 32, 32, 33, 34, 35, 35, 38, 40, 42, 45, 46, 31, 32, 32, 32, + 32, 32, 32, 33, 34, 34, 35, 38, 39, 42, 45, 45, 31, 32, 32, 32, 32, 32, + 32, 33, 33, 34, 34, 37, 38, 41, 44, 44, 31, 32, 32, 32, 33, 33, 33, 34, + 35, 36, 36, 39, 40, 42, 44, 45, 31, 32, 32, 32, 33, 33, 34, 34, 35, 36, + 36, 39, 40, 42, 45, 45, 32, 32, 32, 32, 33, 34, 35, 36, 37, 38, 38, 40, + 41, 42, 45, 46, 32, 33, 33, 33, 34, 34, 36, 36, 38, 39, 40, 42, 43, 44, + 47, 47, 34, 34, 34, 33, 35, 35, 37, 38, 39, 42, 42, 45, 46, 47, 50, 51, + 35, 35, 34, 34, 36, 36, 38, 39, 42, 46, 47, 49, 50, 52, 55, 55, 36, 35, + 35, 34, 36, 36, 38, 40, 42, 47, 48, 50, 52, 54, 56, 57, 39, 38, 38, 37, + 39, 39, 40, 42, 45, 49, 50, 54, 55, 58, 60, 61, 41, 40, 39, 38, 40, 40, + 41, 43, 46, 50, 52, 55, 57, 60, 62, 63, 44, 42, 42, 41, 42, 42, 42, 44, + 47, 52, 54, 58, 60, 63, 66, 67, 47, 45, 45, 44, 44, 45, 45, 47, 50, 55, + 56, 60, 62, 66, 69, 70, 48, 46, 45, 44, 45, 45, 46, 47, 51, 55, 57, 61, + 63, 67, 70, 71], + /* Size 32x32 */ + [32, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 34, 34, 34, + 35, 36, 36, 38, 39, 39, 41, 44, 44, 45, 47, 48, 48, 51, 31, 31, 31, 31, + 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 33, 34, 34, 34, 35, 35, 35, 37, + 39, 39, 40, 43, 43, 44, 46, 47, 47, 50, 31, 31, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 33, 34, 34, 34, 35, 35, 35, 37, 38, 38, 40, 42, + 42, 43, 45, 46, 46, 49, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 33, 34, 34, 34, 35, 35, 35, 37, 38, 38, 40, 42, 42, 43, 45, 46, + 46, 49, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 34, + 34, 34, 34, 35, 35, 36, 38, 38, 39, 42, 42, 42, 45, 45, 45, 48, 31, 31, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 34, 34, 34, + 34, 36, 37, 37, 38, 41, 41, 41, 44, 44, 44, 47, 31, 31, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 34, 34, 34, 34, 36, 37, 37, + 38, 41, 41, 41, 44, 44, 44, 47, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 33, 33, 33, 33, 34, 34, 34, 34, 35, 35, 36, 38, 38, 39, 41, 41, 42, + 44, 45, 45, 47, 31, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 33, 33, 33, + 34, 35, 35, 35, 36, 36, 36, 37, 39, 39, 40, 42, 42, 42, 44, 45, 45, 48, + 31, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 33, 34, 34, 34, 35, 35, 35, + 36, 36, 36, 38, 39, 39, 40, 42, 42, 42, 45, 45, 45, 48, 31, 32, 32, 32, + 32, 32, 32, 32, 33, 33, 33, 33, 34, 34, 34, 35, 35, 35, 36, 36, 36, 38, + 39, 39, 40, 42, 42, 42, 45, 45, 45, 48, 32, 32, 32, 32, 32, 32, 32, 33, + 33, 33, 33, 34, 35, 35, 35, 36, 36, 36, 37, 37, 37, 39, 40, 40, 41, 42, + 42, 43, 45, 45, 45, 48, 32, 32, 32, 32, 32, 32, 32, 33, 33, 34, 34, 35, + 35, 35, 36, 37, 37, 37, 38, 38, 38, 39, 40, 40, 41, 42, 42, 43, 45, 46, + 46, 48, 32, 32, 32, 32, 32, 32, 32, 33, 33, 34, 34, 35, 35, 35, 36, 37, + 37, 37, 38, 38, 38, 39, 40, 40, 41, 42, 42, 43, 45, 46, 46, 48, 32, 33, + 33, 33, 33, 33, 33, 33, 34, 34, 34, 35, 36, 36, 36, 38, 38, 38, 39, 40, + 40, 41, 42, 42, 43, 44, 44, 45, 47, 47, 47, 50, 34, 34, 34, 34, 34, 33, + 33, 34, 35, 35, 35, 36, 37, 37, 38, 39, 39, 40, 42, 42, 42, 44, 45, 45, + 46, 47, 47, 48, 50, 51, 51, 53, 34, 34, 34, 34, 34, 33, 33, 34, 35, 35, + 35, 36, 37, 37, 38, 39, 39, 40, 42, 42, 42, 44, 45, 45, 46, 47, 47, 48, + 50, 51, 51, 53, 34, 34, 34, 34, 34, 34, 34, 34, 35, 35, 35, 36, 37, 37, + 38, 40, 40, 41, 43, 44, 44, 45, 46, 46, 47, 49, 49, 49, 51, 52, 52, 54, + 35, 35, 35, 35, 34, 34, 34, 34, 36, 36, 36, 37, 38, 38, 39, 42, 42, 43, + 46, 47, 47, 48, 49, 49, 50, 52, 52, 53, 55, 55, 55, 57, 36, 35, 35, 35, + 35, 34, 34, 35, 36, 36, 36, 37, 38, 38, 40, 42, 42, 44, 47, 48, 48, 50, + 50, 50, 52, 54, 54, 54, 56, 57, 57, 58, 36, 35, 35, 35, 35, 34, 34, 35, + 36, 36, 36, 37, 38, 38, 40, 42, 42, 44, 47, 48, 48, 50, 50, 50, 52, 54, + 54, 54, 56, 57, 57, 58, 38, 37, 37, 37, 36, 36, 36, 36, 37, 38, 38, 39, + 39, 39, 41, 44, 44, 45, 48, 50, 50, 51, 52, 52, 54, 56, 56, 57, 58, 59, + 59, 61, 39, 39, 38, 38, 38, 37, 37, 38, 39, 39, 39, 40, 40, 40, 42, 45, + 45, 46, 49, 50, 50, 52, 54, 54, 55, 58, 58, 58, 60, 61, 61, 63, 39, 39, + 38, 38, 38, 37, 37, 38, 39, 39, 39, 40, 40, 40, 42, 45, 45, 46, 49, 50, + 50, 52, 54, 54, 55, 58, 58, 58, 60, 61, 61, 63, 41, 40, 40, 40, 39, 38, + 38, 39, 40, 40, 40, 41, 41, 41, 43, 46, 46, 47, 50, 52, 52, 54, 55, 55, + 57, 60, 60, 60, 62, 63, 63, 66, 44, 43, 42, 42, 42, 41, 41, 41, 42, 42, + 42, 42, 42, 42, 44, 47, 47, 49, 52, 54, 54, 56, 58, 58, 60, 63, 63, 64, + 66, 67, 67, 69, 44, 43, 42, 42, 42, 41, 41, 41, 42, 42, 42, 42, 42, 42, + 44, 47, 47, 49, 52, 54, 54, 56, 58, 58, 60, 63, 63, 64, 66, 67, 67, 69, + 45, 44, 43, 43, 42, 41, 41, 42, 42, 42, 42, 43, 43, 43, 45, 48, 48, 49, + 53, 54, 54, 57, 58, 58, 60, 64, 64, 65, 67, 68, 68, 70, 47, 46, 45, 45, + 45, 44, 44, 44, 44, 45, 45, 45, 45, 45, 47, 50, 50, 51, 55, 56, 56, 58, + 60, 60, 62, 66, 66, 67, 69, 70, 70, 73, 48, 47, 46, 46, 45, 44, 44, 45, + 45, 45, 45, 45, 46, 46, 47, 51, 51, 52, 55, 57, 57, 59, 61, 61, 63, 67, + 67, 68, 70, 71, 71, 74, 48, 47, 46, 46, 45, 44, 44, 45, 45, 45, 45, 45, + 46, 46, 47, 51, 51, 52, 55, 57, 57, 59, 61, 61, 63, 67, 67, 68, 70, 71, + 71, 74, 51, 50, 49, 49, 48, 47, 47, 47, 48, 48, 48, 48, 48, 48, 50, 53, + 53, 54, 57, 58, 58, 61, 63, 63, 66, 69, 69, 70, 73, 74, 74, 77], + /* Size 4x8 */ + [31, 32, 35, 43, 32, 33, 34, 41, 32, 34, 36, 42, 32, 35, 38, 42, 34, 37, + 43, 49, 37, 40, 49, 56, 42, 43, 53, 63, 46, 46, 56, 67], + /* Size 8x4 */ + [31, 32, 32, 32, 34, 37, 42, 46, 32, 33, 34, 35, 37, 40, 43, 46, 35, 34, + 36, 38, 43, 49, 53, 56, 43, 41, 42, 42, 49, 56, 63, 67], + /* Size 8x16 */ + [32, 31, 31, 32, 35, 36, 44, 47, 31, 32, 32, 32, 35, 35, 42, 45, 31, 32, + 32, 32, 34, 35, 41, 45, 31, 32, 32, 33, 34, 34, 41, 44, 31, 32, 33, 34, + 35, 36, 42, 44, 32, 32, 33, 34, 36, 36, 42, 45, 32, 33, 34, 35, 37, 38, + 42, 45, 32, 33, 34, 36, 39, 40, 44, 47, 34, 34, 35, 37, 41, 42, 48, 50, + 35, 34, 36, 38, 45, 47, 52, 55, 36, 34, 36, 38, 46, 48, 54, 56, 39, 37, + 39, 40, 48, 50, 58, 60, 41, 39, 40, 41, 49, 51, 60, 62, 44, 41, 42, 43, + 51, 53, 63, 66, 47, 44, 44, 45, 53, 56, 66, 69, 48, 45, 45, 46, 54, 56, + 67, 70], + /* Size 16x8 */ + [32, 31, 31, 31, 31, 32, 32, 32, 34, 35, 36, 39, 41, 44, 47, 48, 31, 32, + 32, 32, 32, 32, 33, 33, 34, 34, 34, 37, 39, 41, 44, 45, 31, 32, 32, 32, + 33, 33, 34, 34, 35, 36, 36, 39, 40, 42, 44, 45, 32, 32, 32, 33, 34, 34, + 35, 36, 37, 38, 38, 40, 41, 43, 45, 46, 35, 35, 34, 34, 35, 36, 37, 39, + 41, 45, 46, 48, 49, 51, 53, 54, 36, 35, 35, 34, 36, 36, 38, 40, 42, 47, + 48, 50, 51, 53, 56, 56, 44, 42, 41, 41, 42, 42, 42, 44, 48, 52, 54, 58, + 60, 63, 66, 67, 47, 45, 45, 44, 44, 45, 45, 47, 50, 55, 56, 60, 62, 66, + 69, 70], + /* Size 16x32 */ + [32, 31, 31, 31, 31, 32, 32, 32, 35, 36, 36, 40, 44, 44, 47, 53, 31, 31, + 32, 32, 32, 32, 32, 33, 35, 35, 35, 39, 43, 43, 46, 52, 31, 32, 32, 32, + 32, 32, 32, 33, 35, 35, 35, 39, 42, 42, 45, 51, 31, 32, 32, 32, 32, 32, + 32, 33, 35, 35, 35, 39, 42, 42, 45, 51, 31, 32, 32, 32, 32, 32, 32, 33, + 34, 35, 35, 39, 41, 41, 45, 50, 31, 32, 32, 32, 32, 33, 33, 33, 34, 34, + 34, 38, 41, 41, 44, 49, 31, 32, 32, 32, 32, 33, 33, 33, 34, 34, 34, 38, + 41, 41, 44, 49, 31, 32, 32, 32, 32, 33, 33, 33, 34, 35, 35, 38, 41, 41, + 44, 49, 31, 32, 32, 32, 33, 34, 34, 34, 35, 36, 36, 39, 42, 42, 44, 49, + 32, 32, 32, 32, 33, 34, 34, 34, 36, 36, 36, 39, 42, 42, 45, 50, 32, 32, + 32, 32, 33, 34, 34, 34, 36, 36, 36, 39, 42, 42, 45, 50, 32, 32, 32, 32, + 33, 35, 35, 35, 37, 37, 37, 40, 42, 42, 45, 49, 32, 32, 33, 33, 34, 35, + 35, 36, 37, 38, 38, 41, 42, 42, 45, 49, 32, 32, 33, 33, 34, 35, 35, 36, + 37, 38, 38, 41, 42, 42, 45, 49, 32, 33, 33, 33, 34, 36, 36, 36, 39, 40, + 40, 42, 44, 44, 47, 51, 34, 34, 34, 34, 35, 37, 37, 38, 41, 42, 42, 45, + 48, 48, 50, 54, 34, 34, 34, 34, 35, 37, 37, 38, 41, 42, 42, 45, 48, 48, + 50, 54, 34, 34, 34, 34, 35, 37, 37, 38, 42, 43, 43, 46, 49, 49, 51, 55, + 35, 35, 34, 34, 36, 38, 38, 39, 45, 47, 47, 50, 52, 52, 55, 59, 36, 35, + 34, 34, 36, 38, 38, 40, 46, 48, 48, 51, 54, 54, 56, 60, 36, 35, 34, 34, + 36, 38, 38, 40, 46, 48, 48, 51, 54, 54, 56, 60, 38, 37, 36, 36, 37, 40, + 40, 41, 47, 49, 49, 53, 56, 56, 58, 63, 39, 38, 37, 37, 39, 40, 40, 42, + 48, 50, 50, 54, 58, 58, 60, 65, 39, 38, 37, 37, 39, 40, 40, 42, 48, 50, + 50, 54, 58, 58, 60, 65, 41, 40, 39, 39, 40, 41, 41, 43, 49, 51, 51, 56, + 60, 60, 62, 67, 44, 42, 41, 41, 42, 43, 43, 45, 51, 53, 53, 59, 63, 63, + 66, 71, 44, 42, 41, 41, 42, 43, 43, 45, 51, 53, 53, 59, 63, 63, 66, 71, + 44, 43, 42, 42, 42, 43, 43, 45, 51, 54, 54, 59, 64, 64, 67, 72, 47, 45, + 44, 44, 44, 45, 45, 47, 53, 56, 56, 61, 66, 66, 69, 75, 48, 46, 45, 45, + 45, 46, 46, 48, 54, 56, 56, 62, 67, 67, 70, 76, 48, 46, 45, 45, 45, 46, + 46, 48, 54, 56, 56, 62, 67, 67, 70, 76, 51, 49, 47, 47, 48, 48, 48, 50, + 56, 58, 58, 64, 69, 69, 73, 79], + /* Size 32x16 */ + [32, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 34, 34, 34, + 35, 36, 36, 38, 39, 39, 41, 44, 44, 44, 47, 48, 48, 51, 31, 31, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 34, 34, 34, 35, 35, 35, 37, + 38, 38, 40, 42, 42, 43, 45, 46, 46, 49, 31, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 33, 33, 33, 34, 34, 34, 34, 34, 34, 36, 37, 37, 39, 41, + 41, 42, 44, 45, 45, 47, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 33, 33, 33, 34, 34, 34, 34, 34, 34, 36, 37, 37, 39, 41, 41, 42, 44, 45, + 45, 47, 31, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 33, 34, 34, 34, 35, + 35, 35, 36, 36, 36, 37, 39, 39, 40, 42, 42, 42, 44, 45, 45, 48, 32, 32, + 32, 32, 32, 33, 33, 33, 34, 34, 34, 35, 35, 35, 36, 37, 37, 37, 38, 38, + 38, 40, 40, 40, 41, 43, 43, 43, 45, 46, 46, 48, 32, 32, 32, 32, 32, 33, + 33, 33, 34, 34, 34, 35, 35, 35, 36, 37, 37, 37, 38, 38, 38, 40, 40, 40, + 41, 43, 43, 43, 45, 46, 46, 48, 32, 33, 33, 33, 33, 33, 33, 33, 34, 34, + 34, 35, 36, 36, 36, 38, 38, 38, 39, 40, 40, 41, 42, 42, 43, 45, 45, 45, + 47, 48, 48, 50, 35, 35, 35, 35, 34, 34, 34, 34, 35, 36, 36, 37, 37, 37, + 39, 41, 41, 42, 45, 46, 46, 47, 48, 48, 49, 51, 51, 51, 53, 54, 54, 56, + 36, 35, 35, 35, 35, 34, 34, 35, 36, 36, 36, 37, 38, 38, 40, 42, 42, 43, + 47, 48, 48, 49, 50, 50, 51, 53, 53, 54, 56, 56, 56, 58, 36, 35, 35, 35, + 35, 34, 34, 35, 36, 36, 36, 37, 38, 38, 40, 42, 42, 43, 47, 48, 48, 49, + 50, 50, 51, 53, 53, 54, 56, 56, 56, 58, 40, 39, 39, 39, 39, 38, 38, 38, + 39, 39, 39, 40, 41, 41, 42, 45, 45, 46, 50, 51, 51, 53, 54, 54, 56, 59, + 59, 59, 61, 62, 62, 64, 44, 43, 42, 42, 41, 41, 41, 41, 42, 42, 42, 42, + 42, 42, 44, 48, 48, 49, 52, 54, 54, 56, 58, 58, 60, 63, 63, 64, 66, 67, + 67, 69, 44, 43, 42, 42, 41, 41, 41, 41, 42, 42, 42, 42, 42, 42, 44, 48, + 48, 49, 52, 54, 54, 56, 58, 58, 60, 63, 63, 64, 66, 67, 67, 69, 47, 46, + 45, 45, 45, 44, 44, 44, 44, 45, 45, 45, 45, 45, 47, 50, 50, 51, 55, 56, + 56, 58, 60, 60, 62, 66, 66, 67, 69, 70, 70, 73, 53, 52, 51, 51, 50, 49, + 49, 49, 49, 50, 50, 49, 49, 49, 51, 54, 54, 55, 59, 60, 60, 63, 65, 65, + 67, 71, 71, 72, 75, 76, 76, 79], + /* Size 4x16 */ + [31, 32, 36, 44, 32, 32, 35, 42, 32, 32, 35, 41, 32, 33, 34, 41, 32, 34, + 36, 42, 32, 34, 36, 42, 32, 35, 38, 42, 33, 36, 40, 44, 34, 37, 42, 48, + 35, 38, 47, 52, 35, 38, 48, 54, 38, 40, 50, 58, 40, 41, 51, 60, 42, 43, + 53, 63, 45, 45, 56, 66, 46, 46, 56, 67], + /* Size 16x4 */ + [31, 32, 32, 32, 32, 32, 32, 33, 34, 35, 35, 38, 40, 42, 45, 46, 32, 32, + 32, 33, 34, 34, 35, 36, 37, 38, 38, 40, 41, 43, 45, 46, 36, 35, 35, 34, + 36, 36, 38, 40, 42, 47, 48, 50, 51, 53, 56, 56, 44, 42, 41, 41, 42, 42, + 42, 44, 48, 52, 54, 58, 60, 63, 66, 67], + /* Size 8x32 */ + [32, 31, 31, 32, 35, 36, 44, 47, 31, 32, 32, 32, 35, 35, 43, 46, 31, 32, + 32, 32, 35, 35, 42, 45, 31, 32, 32, 32, 35, 35, 42, 45, 31, 32, 32, 32, + 34, 35, 41, 45, 31, 32, 32, 33, 34, 34, 41, 44, 31, 32, 32, 33, 34, 34, + 41, 44, 31, 32, 32, 33, 34, 35, 41, 44, 31, 32, 33, 34, 35, 36, 42, 44, + 32, 32, 33, 34, 36, 36, 42, 45, 32, 32, 33, 34, 36, 36, 42, 45, 32, 32, + 33, 35, 37, 37, 42, 45, 32, 33, 34, 35, 37, 38, 42, 45, 32, 33, 34, 35, + 37, 38, 42, 45, 32, 33, 34, 36, 39, 40, 44, 47, 34, 34, 35, 37, 41, 42, + 48, 50, 34, 34, 35, 37, 41, 42, 48, 50, 34, 34, 35, 37, 42, 43, 49, 51, + 35, 34, 36, 38, 45, 47, 52, 55, 36, 34, 36, 38, 46, 48, 54, 56, 36, 34, + 36, 38, 46, 48, 54, 56, 38, 36, 37, 40, 47, 49, 56, 58, 39, 37, 39, 40, + 48, 50, 58, 60, 39, 37, 39, 40, 48, 50, 58, 60, 41, 39, 40, 41, 49, 51, + 60, 62, 44, 41, 42, 43, 51, 53, 63, 66, 44, 41, 42, 43, 51, 53, 63, 66, + 44, 42, 42, 43, 51, 54, 64, 67, 47, 44, 44, 45, 53, 56, 66, 69, 48, 45, + 45, 46, 54, 56, 67, 70, 48, 45, 45, 46, 54, 56, 67, 70, 51, 47, 48, 48, + 56, 58, 69, 73], + /* Size 32x8 */ + [32, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 34, 34, 34, + 35, 36, 36, 38, 39, 39, 41, 44, 44, 44, 47, 48, 48, 51, 31, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 34, 34, 34, 34, 34, 34, 36, + 37, 37, 39, 41, 41, 42, 44, 45, 45, 47, 31, 32, 32, 32, 32, 32, 32, 32, + 33, 33, 33, 33, 34, 34, 34, 35, 35, 35, 36, 36, 36, 37, 39, 39, 40, 42, + 42, 42, 44, 45, 45, 48, 32, 32, 32, 32, 32, 33, 33, 33, 34, 34, 34, 35, + 35, 35, 36, 37, 37, 37, 38, 38, 38, 40, 40, 40, 41, 43, 43, 43, 45, 46, + 46, 48, 35, 35, 35, 35, 34, 34, 34, 34, 35, 36, 36, 37, 37, 37, 39, 41, + 41, 42, 45, 46, 46, 47, 48, 48, 49, 51, 51, 51, 53, 54, 54, 56, 36, 35, + 35, 35, 35, 34, 34, 35, 36, 36, 36, 37, 38, 38, 40, 42, 42, 43, 47, 48, + 48, 49, 50, 50, 51, 53, 53, 54, 56, 56, 56, 58, 44, 43, 42, 42, 41, 41, + 41, 41, 42, 42, 42, 42, 42, 42, 44, 48, 48, 49, 52, 54, 54, 56, 58, 58, + 60, 63, 63, 64, 66, 67, 67, 69, 47, 46, 45, 45, 45, 44, 44, 44, 44, 45, + 45, 45, 45, 45, 47, 50, 50, 51, 55, 56, 56, 58, 60, 60, 62, 66, 66, 67, + 69, 70, 70, 73]], + [ /* Chroma */ + /* Size 4x4 */ + [31, 37, 47, 47, 37, 44, 47, 45, 47, 47, 53, 53, 47, 45, 53, 59], + /* Size 8x8 */ + [31, 31, 34, 37, 43, 48, 47, 49, 31, 32, 35, 40, 43, 46, 45, 46, 34, 35, + 39, 43, 45, 46, 45, 46, 37, 40, 43, 47, 47, 47, 45, 46, 43, 43, 45, 47, + 49, 50, 50, 50, 48, 46, 46, 47, 50, 53, 55, 55, 47, 45, 45, 45, 50, 55, + 58, 60, 49, 46, 46, 46, 50, 55, 60, 61], + /* Size 16x16 */ + [32, 31, 31, 30, 33, 33, 36, 38, 41, 47, 49, 48, 49, 49, 50, 50, 31, 31, + 31, 31, 34, 34, 38, 40, 42, 46, 47, 47, 47, 47, 48, 48, 31, 31, 31, 31, + 34, 35, 39, 40, 42, 46, 47, 46, 46, 46, 47, 47, 30, 31, 31, 32, 34, 35, + 40, 41, 42, 45, 46, 45, 45, 45, 46, 46, 33, 34, 34, 34, 37, 38, 42, 43, + 44, 46, 47, 46, 46, 45, 46, 46, 33, 34, 35, 35, 38, 39, 43, 44, 45, 47, + 47, 46, 46, 45, 46, 46, 36, 38, 39, 40, 42, 43, 47, 47, 47, 47, 48, 46, + 46, 45, 46, 46, 38, 40, 40, 41, 43, 44, 47, 47, 48, 48, 49, 48, 47, 47, + 47, 47, 41, 42, 42, 42, 44, 45, 47, 48, 48, 50, 50, 49, 49, 49, 50, 50, + 47, 46, 46, 45, 46, 47, 47, 48, 50, 52, 52, 52, 52, 52, 53, 53, 49, 47, + 47, 46, 47, 47, 48, 49, 50, 52, 53, 53, 53, 53, 54, 54, 48, 47, 46, 45, + 46, 46, 46, 48, 49, 52, 53, 54, 55, 55, 56, 56, 49, 47, 46, 45, 46, 46, + 46, 47, 49, 52, 53, 55, 55, 57, 57, 58, 49, 47, 46, 45, 45, 45, 45, 47, + 49, 52, 53, 55, 57, 58, 59, 60, 50, 48, 47, 46, 46, 46, 46, 47, 50, 53, + 54, 56, 57, 59, 61, 61, 50, 48, 47, 46, 46, 46, 46, 47, 50, 53, 54, 56, + 58, 60, 61, 61], + /* Size 32x32 */ + [32, 31, 31, 31, 31, 30, 30, 31, 33, 33, 33, 35, 36, 36, 38, 41, 41, 43, + 47, 49, 49, 49, 48, 48, 49, 49, 49, 49, 50, 50, 50, 51, 31, 31, 31, 31, + 31, 31, 31, 31, 33, 34, 34, 36, 37, 37, 39, 42, 42, 43, 47, 48, 48, 48, + 47, 47, 47, 47, 47, 48, 49, 49, 49, 50, 31, 31, 31, 31, 31, 31, 31, 32, + 34, 34, 34, 37, 38, 38, 40, 42, 42, 43, 46, 47, 47, 47, 47, 47, 47, 47, + 47, 47, 48, 48, 48, 49, 31, 31, 31, 31, 31, 31, 31, 32, 34, 34, 34, 37, + 38, 38, 40, 42, 42, 43, 46, 47, 47, 47, 47, 47, 47, 47, 47, 47, 48, 48, + 48, 49, 31, 31, 31, 31, 31, 31, 31, 32, 34, 35, 35, 37, 39, 39, 40, 42, + 42, 43, 46, 47, 47, 46, 46, 46, 46, 46, 46, 46, 47, 47, 47, 48, 30, 31, + 31, 31, 31, 32, 32, 32, 34, 35, 35, 38, 40, 40, 41, 42, 42, 43, 45, 46, + 46, 46, 45, 45, 45, 45, 45, 45, 46, 46, 46, 47, 30, 31, 31, 31, 31, 32, + 32, 32, 34, 35, 35, 38, 40, 40, 41, 42, 42, 43, 45, 46, 46, 46, 45, 45, + 45, 45, 45, 45, 46, 46, 46, 47, 31, 31, 32, 32, 32, 32, 32, 33, 35, 36, + 36, 38, 40, 40, 41, 43, 43, 43, 46, 46, 46, 46, 45, 45, 45, 45, 45, 45, + 46, 46, 46, 47, 33, 33, 34, 34, 34, 34, 34, 35, 37, 38, 38, 41, 42, 42, + 43, 44, 44, 45, 46, 47, 47, 46, 46, 46, 46, 45, 45, 45, 46, 46, 46, 47, + 33, 34, 34, 34, 35, 35, 35, 36, 38, 39, 39, 41, 43, 43, 44, 45, 45, 45, + 47, 47, 47, 46, 46, 46, 46, 45, 45, 45, 46, 46, 46, 47, 33, 34, 34, 34, + 35, 35, 35, 36, 38, 39, 39, 41, 43, 43, 44, 45, 45, 45, 47, 47, 47, 46, + 46, 46, 46, 45, 45, 45, 46, 46, 46, 47, 35, 36, 37, 37, 37, 38, 38, 38, + 41, 41, 41, 44, 46, 46, 46, 46, 46, 46, 47, 47, 47, 47, 46, 46, 46, 45, + 45, 45, 46, 46, 46, 47, 36, 37, 38, 38, 39, 40, 40, 40, 42, 43, 43, 46, + 47, 47, 47, 47, 47, 47, 47, 48, 48, 47, 46, 46, 46, 45, 45, 45, 46, 46, + 46, 46, 36, 37, 38, 38, 39, 40, 40, 40, 42, 43, 43, 46, 47, 47, 47, 47, + 47, 47, 47, 48, 48, 47, 46, 46, 46, 45, 45, 45, 46, 46, 46, 46, 38, 39, + 40, 40, 40, 41, 41, 41, 43, 44, 44, 46, 47, 47, 47, 48, 48, 48, 48, 49, + 49, 48, 48, 48, 47, 47, 47, 47, 47, 47, 47, 48, 41, 42, 42, 42, 42, 42, + 42, 43, 44, 45, 45, 46, 47, 47, 48, 48, 48, 49, 50, 50, 50, 50, 49, 49, + 49, 49, 49, 49, 50, 50, 50, 50, 41, 42, 42, 42, 42, 42, 42, 43, 44, 45, + 45, 46, 47, 47, 48, 48, 48, 49, 50, 50, 50, 50, 49, 49, 49, 49, 49, 49, + 50, 50, 50, 50, 43, 43, 43, 43, 43, 43, 43, 43, 45, 45, 45, 46, 47, 47, + 48, 49, 49, 49, 50, 51, 51, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 51, + 47, 47, 46, 46, 46, 45, 45, 46, 46, 47, 47, 47, 47, 47, 48, 50, 50, 50, + 52, 52, 52, 52, 52, 52, 52, 52, 52, 52, 53, 53, 53, 53, 49, 48, 47, 47, + 47, 46, 46, 46, 47, 47, 47, 47, 48, 48, 49, 50, 50, 51, 52, 53, 53, 53, + 53, 53, 53, 53, 53, 53, 54, 54, 54, 54, 49, 48, 47, 47, 47, 46, 46, 46, + 47, 47, 47, 47, 48, 48, 49, 50, 50, 51, 52, 53, 53, 53, 53, 53, 53, 53, + 53, 53, 54, 54, 54, 54, 49, 48, 47, 47, 46, 46, 46, 46, 46, 46, 46, 47, + 47, 47, 48, 50, 50, 50, 52, 53, 53, 53, 54, 54, 54, 55, 55, 55, 55, 55, + 55, 56, 48, 47, 47, 47, 46, 45, 45, 45, 46, 46, 46, 46, 46, 46, 48, 49, + 49, 50, 52, 53, 53, 54, 54, 54, 55, 55, 55, 56, 56, 56, 56, 57, 48, 47, + 47, 47, 46, 45, 45, 45, 46, 46, 46, 46, 46, 46, 48, 49, 49, 50, 52, 53, + 53, 54, 54, 54, 55, 55, 55, 56, 56, 56, 56, 57, 49, 47, 47, 47, 46, 45, + 45, 45, 46, 46, 46, 46, 46, 46, 47, 49, 49, 50, 52, 53, 53, 54, 55, 55, + 55, 57, 57, 57, 57, 58, 58, 58, 49, 47, 47, 47, 46, 45, 45, 45, 45, 45, + 45, 45, 45, 45, 47, 49, 49, 50, 52, 53, 53, 55, 55, 55, 57, 58, 58, 59, + 59, 60, 60, 60, 49, 47, 47, 47, 46, 45, 45, 45, 45, 45, 45, 45, 45, 45, + 47, 49, 49, 50, 52, 53, 53, 55, 55, 55, 57, 58, 58, 59, 59, 60, 60, 60, + 49, 48, 47, 47, 46, 45, 45, 45, 45, 45, 45, 45, 45, 45, 47, 49, 49, 50, + 52, 53, 53, 55, 56, 56, 57, 59, 59, 59, 60, 60, 60, 61, 50, 49, 48, 48, + 47, 46, 46, 46, 46, 46, 46, 46, 46, 46, 47, 50, 50, 50, 53, 54, 54, 55, + 56, 56, 57, 59, 59, 60, 61, 61, 61, 62, 50, 49, 48, 48, 47, 46, 46, 46, + 46, 46, 46, 46, 46, 46, 47, 50, 50, 50, 53, 54, 54, 55, 56, 56, 58, 60, + 60, 60, 61, 61, 61, 63, 50, 49, 48, 48, 47, 46, 46, 46, 46, 46, 46, 46, + 46, 46, 47, 50, 50, 50, 53, 54, 54, 55, 56, 56, 58, 60, 60, 60, 61, 61, + 61, 63, 51, 50, 49, 49, 48, 47, 47, 47, 47, 47, 47, 47, 46, 46, 48, 50, + 50, 51, 53, 54, 54, 56, 57, 57, 58, 60, 60, 61, 62, 63, 63, 64], + /* Size 4x8 */ + [31, 38, 47, 48, 31, 40, 46, 45, 35, 43, 47, 46, 39, 47, 47, 45, 43, 47, + 50, 50, 47, 47, 53, 55, 46, 46, 53, 58, 48, 46, 54, 59], + /* Size 8x4 */ + [31, 31, 35, 39, 43, 47, 46, 48, 38, 40, 43, 47, 47, 47, 46, 46, 47, 46, + 47, 47, 50, 53, 53, 54, 48, 45, 46, 45, 50, 55, 58, 59], + /* Size 8x16 */ + [32, 31, 33, 37, 45, 48, 49, 50, 31, 31, 34, 38, 45, 47, 47, 48, 31, 32, + 34, 39, 45, 46, 46, 47, 30, 32, 35, 40, 44, 46, 45, 46, 33, 35, 37, 42, + 46, 47, 45, 46, 33, 36, 38, 43, 46, 47, 46, 46, 37, 40, 43, 47, 47, 47, + 45, 46, 39, 41, 43, 47, 48, 48, 47, 47, 42, 43, 44, 47, 49, 50, 49, 50, + 47, 46, 46, 48, 51, 52, 53, 53, 49, 46, 47, 48, 52, 53, 53, 54, 48, 46, + 46, 47, 51, 53, 56, 56, 48, 45, 46, 46, 51, 53, 57, 57, 49, 45, 45, 46, + 51, 53, 58, 59, 50, 46, 46, 46, 52, 54, 59, 61, 50, 46, 46, 46, 52, 54, + 59, 61], + /* Size 16x8 */ + [32, 31, 31, 30, 33, 33, 37, 39, 42, 47, 49, 48, 48, 49, 50, 50, 31, 31, + 32, 32, 35, 36, 40, 41, 43, 46, 46, 46, 45, 45, 46, 46, 33, 34, 34, 35, + 37, 38, 43, 43, 44, 46, 47, 46, 46, 45, 46, 46, 37, 38, 39, 40, 42, 43, + 47, 47, 47, 48, 48, 47, 46, 46, 46, 46, 45, 45, 45, 44, 46, 46, 47, 48, + 49, 51, 52, 51, 51, 51, 52, 52, 48, 47, 46, 46, 47, 47, 47, 48, 50, 52, + 53, 53, 53, 53, 54, 54, 49, 47, 46, 45, 45, 46, 45, 47, 49, 53, 53, 56, + 57, 58, 59, 59, 50, 48, 47, 46, 46, 46, 46, 47, 50, 53, 54, 56, 57, 59, + 61, 61], + /* Size 16x32 */ + [32, 31, 31, 31, 33, 37, 37, 38, 45, 48, 48, 49, 49, 49, 50, 52, 31, 31, + 31, 31, 33, 38, 38, 39, 45, 47, 47, 48, 48, 48, 49, 51, 31, 31, 31, 31, + 34, 38, 38, 40, 45, 47, 47, 47, 47, 47, 48, 50, 31, 31, 31, 31, 34, 38, + 38, 40, 45, 47, 47, 47, 47, 47, 48, 50, 31, 31, 32, 32, 34, 39, 39, 40, + 45, 46, 46, 46, 46, 46, 47, 49, 30, 31, 32, 32, 35, 40, 40, 41, 44, 46, + 46, 45, 45, 45, 46, 48, 30, 31, 32, 32, 35, 40, 40, 41, 44, 46, 46, 45, + 45, 45, 46, 48, 31, 32, 33, 33, 35, 40, 40, 41, 45, 46, 46, 45, 45, 45, + 46, 48, 33, 34, 35, 35, 37, 42, 42, 43, 46, 47, 47, 46, 45, 45, 46, 47, + 33, 35, 36, 36, 38, 43, 43, 44, 46, 47, 47, 46, 46, 46, 46, 47, 33, 35, + 36, 36, 38, 43, 43, 44, 46, 47, 47, 46, 46, 46, 46, 47, 35, 37, 38, 38, + 41, 45, 45, 46, 47, 47, 47, 46, 45, 45, 46, 47, 37, 39, 40, 40, 43, 47, + 47, 47, 47, 47, 47, 46, 45, 45, 46, 47, 37, 39, 40, 40, 43, 47, 47, 47, + 47, 47, 47, 46, 45, 45, 46, 47, 39, 40, 41, 41, 43, 47, 47, 47, 48, 48, + 48, 47, 47, 47, 47, 48, 42, 42, 43, 43, 44, 47, 47, 48, 49, 50, 50, 49, + 49, 49, 50, 50, 42, 42, 43, 43, 44, 47, 47, 48, 49, 50, 50, 49, 49, 49, + 50, 50, 43, 43, 43, 43, 45, 47, 47, 48, 50, 50, 50, 50, 50, 50, 50, 51, + 47, 46, 46, 46, 46, 48, 48, 48, 51, 52, 52, 52, 53, 53, 53, 53, 49, 47, + 46, 46, 47, 48, 48, 49, 52, 53, 53, 53, 53, 53, 54, 54, 49, 47, 46, 46, + 47, 48, 48, 49, 52, 53, 53, 53, 53, 53, 54, 54, 48, 47, 46, 46, 46, 47, + 47, 48, 52, 53, 53, 54, 55, 55, 55, 56, 48, 47, 46, 46, 46, 47, 47, 48, + 51, 53, 53, 54, 56, 56, 56, 57, 48, 47, 46, 46, 46, 47, 47, 48, 51, 53, + 53, 54, 56, 56, 56, 57, 48, 47, 45, 45, 46, 46, 46, 47, 51, 53, 53, 55, + 57, 57, 57, 59, 49, 46, 45, 45, 45, 46, 46, 47, 51, 53, 53, 56, 58, 58, + 59, 61, 49, 46, 45, 45, 45, 46, 46, 47, 51, 53, 53, 56, 58, 58, 59, 61, + 49, 47, 45, 45, 45, 46, 46, 47, 52, 53, 53, 56, 58, 58, 60, 62, 50, 48, + 46, 46, 46, 46, 46, 48, 52, 54, 54, 57, 59, 59, 61, 63, 50, 48, 46, 46, + 46, 46, 46, 48, 52, 54, 54, 57, 59, 59, 61, 64, 50, 48, 46, 46, 46, 46, + 46, 48, 52, 54, 54, 57, 59, 59, 61, 64, 51, 49, 47, 47, 47, 47, 47, 48, + 52, 54, 54, 58, 60, 60, 62, 65], + /* Size 32x16 */ + [32, 31, 31, 31, 31, 30, 30, 31, 33, 33, 33, 35, 37, 37, 39, 42, 42, 43, + 47, 49, 49, 48, 48, 48, 48, 49, 49, 49, 50, 50, 50, 51, 31, 31, 31, 31, + 31, 31, 31, 32, 34, 35, 35, 37, 39, 39, 40, 42, 42, 43, 46, 47, 47, 47, + 47, 47, 47, 46, 46, 47, 48, 48, 48, 49, 31, 31, 31, 31, 32, 32, 32, 33, + 35, 36, 36, 38, 40, 40, 41, 43, 43, 43, 46, 46, 46, 46, 46, 46, 45, 45, + 45, 45, 46, 46, 46, 47, 31, 31, 31, 31, 32, 32, 32, 33, 35, 36, 36, 38, + 40, 40, 41, 43, 43, 43, 46, 46, 46, 46, 46, 46, 45, 45, 45, 45, 46, 46, + 46, 47, 33, 33, 34, 34, 34, 35, 35, 35, 37, 38, 38, 41, 43, 43, 43, 44, + 44, 45, 46, 47, 47, 46, 46, 46, 46, 45, 45, 45, 46, 46, 46, 47, 37, 38, + 38, 38, 39, 40, 40, 40, 42, 43, 43, 45, 47, 47, 47, 47, 47, 47, 48, 48, + 48, 47, 47, 47, 46, 46, 46, 46, 46, 46, 46, 47, 37, 38, 38, 38, 39, 40, + 40, 40, 42, 43, 43, 45, 47, 47, 47, 47, 47, 47, 48, 48, 48, 47, 47, 47, + 46, 46, 46, 46, 46, 46, 46, 47, 38, 39, 40, 40, 40, 41, 41, 41, 43, 44, + 44, 46, 47, 47, 47, 48, 48, 48, 48, 49, 49, 48, 48, 48, 47, 47, 47, 47, + 48, 48, 48, 48, 45, 45, 45, 45, 45, 44, 44, 45, 46, 46, 46, 47, 47, 47, + 48, 49, 49, 50, 51, 52, 52, 52, 51, 51, 51, 51, 51, 52, 52, 52, 52, 52, + 48, 47, 47, 47, 46, 46, 46, 46, 47, 47, 47, 47, 47, 47, 48, 50, 50, 50, + 52, 53, 53, 53, 53, 53, 53, 53, 53, 53, 54, 54, 54, 54, 48, 47, 47, 47, + 46, 46, 46, 46, 47, 47, 47, 47, 47, 47, 48, 50, 50, 50, 52, 53, 53, 53, + 53, 53, 53, 53, 53, 53, 54, 54, 54, 54, 49, 48, 47, 47, 46, 45, 45, 45, + 46, 46, 46, 46, 46, 46, 47, 49, 49, 50, 52, 53, 53, 54, 54, 54, 55, 56, + 56, 56, 57, 57, 57, 58, 49, 48, 47, 47, 46, 45, 45, 45, 45, 46, 46, 45, + 45, 45, 47, 49, 49, 50, 53, 53, 53, 55, 56, 56, 57, 58, 58, 58, 59, 59, + 59, 60, 49, 48, 47, 47, 46, 45, 45, 45, 45, 46, 46, 45, 45, 45, 47, 49, + 49, 50, 53, 53, 53, 55, 56, 56, 57, 58, 58, 58, 59, 59, 59, 60, 50, 49, + 48, 48, 47, 46, 46, 46, 46, 46, 46, 46, 46, 46, 47, 50, 50, 50, 53, 54, + 54, 55, 56, 56, 57, 59, 59, 60, 61, 61, 61, 62, 52, 51, 50, 50, 49, 48, + 48, 48, 47, 47, 47, 47, 47, 47, 48, 50, 50, 51, 53, 54, 54, 56, 57, 57, + 59, 61, 61, 62, 63, 64, 64, 65], + /* Size 4x16 */ + [31, 37, 48, 49, 31, 38, 47, 47, 31, 39, 46, 46, 31, 40, 46, 45, 34, 42, + 47, 45, 35, 43, 47, 46, 39, 47, 47, 45, 40, 47, 48, 47, 42, 47, 50, 49, + 46, 48, 52, 53, 47, 48, 53, 53, 47, 47, 53, 56, 47, 46, 53, 57, 46, 46, + 53, 58, 48, 46, 54, 59, 48, 46, 54, 59], + /* Size 16x4 */ + [31, 31, 31, 31, 34, 35, 39, 40, 42, 46, 47, 47, 47, 46, 48, 48, 37, 38, + 39, 40, 42, 43, 47, 47, 47, 48, 48, 47, 46, 46, 46, 46, 48, 47, 46, 46, + 47, 47, 47, 48, 50, 52, 53, 53, 53, 53, 54, 54, 49, 47, 46, 45, 45, 46, + 45, 47, 49, 53, 53, 56, 57, 58, 59, 59], + /* Size 8x32 */ + [32, 31, 33, 37, 45, 48, 49, 50, 31, 31, 33, 38, 45, 47, 48, 49, 31, 31, + 34, 38, 45, 47, 47, 48, 31, 31, 34, 38, 45, 47, 47, 48, 31, 32, 34, 39, + 45, 46, 46, 47, 30, 32, 35, 40, 44, 46, 45, 46, 30, 32, 35, 40, 44, 46, + 45, 46, 31, 33, 35, 40, 45, 46, 45, 46, 33, 35, 37, 42, 46, 47, 45, 46, + 33, 36, 38, 43, 46, 47, 46, 46, 33, 36, 38, 43, 46, 47, 46, 46, 35, 38, + 41, 45, 47, 47, 45, 46, 37, 40, 43, 47, 47, 47, 45, 46, 37, 40, 43, 47, + 47, 47, 45, 46, 39, 41, 43, 47, 48, 48, 47, 47, 42, 43, 44, 47, 49, 50, + 49, 50, 42, 43, 44, 47, 49, 50, 49, 50, 43, 43, 45, 47, 50, 50, 50, 50, + 47, 46, 46, 48, 51, 52, 53, 53, 49, 46, 47, 48, 52, 53, 53, 54, 49, 46, + 47, 48, 52, 53, 53, 54, 48, 46, 46, 47, 52, 53, 55, 55, 48, 46, 46, 47, + 51, 53, 56, 56, 48, 46, 46, 47, 51, 53, 56, 56, 48, 45, 46, 46, 51, 53, + 57, 57, 49, 45, 45, 46, 51, 53, 58, 59, 49, 45, 45, 46, 51, 53, 58, 59, + 49, 45, 45, 46, 52, 53, 58, 60, 50, 46, 46, 46, 52, 54, 59, 61, 50, 46, + 46, 46, 52, 54, 59, 61, 50, 46, 46, 46, 52, 54, 59, 61, 51, 47, 47, 47, + 52, 54, 60, 62], + /* Size 32x8 */ + [32, 31, 31, 31, 31, 30, 30, 31, 33, 33, 33, 35, 37, 37, 39, 42, 42, 43, + 47, 49, 49, 48, 48, 48, 48, 49, 49, 49, 50, 50, 50, 51, 31, 31, 31, 31, + 32, 32, 32, 33, 35, 36, 36, 38, 40, 40, 41, 43, 43, 43, 46, 46, 46, 46, + 46, 46, 45, 45, 45, 45, 46, 46, 46, 47, 33, 33, 34, 34, 34, 35, 35, 35, + 37, 38, 38, 41, 43, 43, 43, 44, 44, 45, 46, 47, 47, 46, 46, 46, 46, 45, + 45, 45, 46, 46, 46, 47, 37, 38, 38, 38, 39, 40, 40, 40, 42, 43, 43, 45, + 47, 47, 47, 47, 47, 47, 48, 48, 48, 47, 47, 47, 46, 46, 46, 46, 46, 46, + 46, 47, 45, 45, 45, 45, 45, 44, 44, 45, 46, 46, 46, 47, 47, 47, 48, 49, + 49, 50, 51, 52, 52, 52, 51, 51, 51, 51, 51, 52, 52, 52, 52, 52, 48, 47, + 47, 47, 46, 46, 46, 46, 47, 47, 47, 47, 47, 47, 48, 50, 50, 50, 52, 53, + 53, 53, 53, 53, 53, 53, 53, 53, 54, 54, 54, 54, 49, 48, 47, 47, 46, 45, + 45, 45, 45, 46, 46, 45, 45, 45, 47, 49, 49, 50, 53, 53, 53, 55, 56, 56, + 57, 58, 58, 58, 59, 59, 59, 60, 50, 49, 48, 48, 47, 46, 46, 46, 46, 46, + 46, 46, 46, 46, 47, 50, 50, 50, 53, 54, 54, 55, 56, 56, 57, 59, 59, 60, + 61, 61, 61, 62]] + ], + [ + [ /* Luma */ + /* Size 4x4 */ + [32, 32, 34, 38, 32, 33, 35, 39, 34, 35, 39, 45, 38, 39, 45, 54], + /* Size 8x8 */ + [31, 31, 32, 32, 33, 34, 37, 41, 31, 32, 32, 32, 33, 34, 36, 39, 32, 32, + 32, 33, 34, 35, 37, 40, 32, 32, 33, 34, 35, 36, 38, 41, 33, 33, 34, 35, + 37, 39, 41, 44, 34, 34, 35, 36, 39, 43, 46, 49, 37, 36, 37, 38, 41, 46, + 51, 54, 41, 39, 40, 41, 44, 49, 54, 58], + /* Size 16x16 */ + [32, 31, 31, 31, 31, 31, 31, 32, 32, 34, 34, 36, 36, 39, 39, 44, 31, 32, + 32, 32, 32, 32, 32, 32, 32, 34, 34, 35, 35, 38, 38, 42, 31, 32, 32, 32, + 32, 32, 32, 32, 32, 34, 34, 35, 35, 38, 38, 42, 31, 32, 32, 32, 32, 32, + 32, 32, 32, 33, 33, 34, 34, 37, 37, 41, 31, 32, 32, 32, 32, 32, 32, 32, + 32, 33, 33, 34, 34, 37, 37, 41, 31, 32, 32, 32, 32, 33, 33, 34, 34, 35, + 35, 36, 36, 39, 39, 42, 31, 32, 32, 32, 32, 33, 33, 34, 34, 35, 35, 36, + 36, 39, 39, 42, 32, 32, 32, 32, 32, 34, 34, 35, 35, 37, 37, 38, 38, 40, + 40, 42, 32, 32, 32, 32, 32, 34, 34, 35, 35, 37, 37, 38, 38, 40, 40, 42, + 34, 34, 34, 33, 33, 35, 35, 37, 37, 39, 39, 42, 42, 45, 45, 47, 34, 34, + 34, 33, 33, 35, 35, 37, 37, 39, 39, 42, 42, 45, 45, 47, 36, 35, 35, 34, + 34, 36, 36, 38, 38, 42, 42, 48, 48, 50, 50, 54, 36, 35, 35, 34, 34, 36, + 36, 38, 38, 42, 42, 48, 48, 50, 50, 54, 39, 38, 38, 37, 37, 39, 39, 40, + 40, 45, 45, 50, 50, 54, 54, 58, 39, 38, 38, 37, 37, 39, 39, 40, 40, 45, + 45, 50, 50, 54, 54, 58, 44, 42, 42, 41, 41, 42, 42, 42, 42, 47, 47, 54, + 54, 58, 58, 63], + /* Size 32x32 */ + [32, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 33, + 34, 34, 34, 35, 36, 36, 36, 37, 39, 39, 39, 41, 44, 44, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 33, 34, 34, 34, 34, + 35, 35, 35, 37, 39, 39, 39, 41, 43, 43, 31, 31, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 34, 34, 34, 34, 35, 35, 35, 37, + 38, 38, 38, 40, 42, 42, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 33, 34, 34, 34, 34, 35, 35, 35, 37, 38, 38, 38, 40, + 42, 42, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 33, 34, 34, 34, 34, 35, 35, 35, 37, 38, 38, 38, 40, 42, 42, 31, 31, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 34, 34, + 34, 34, 35, 35, 35, 36, 38, 38, 38, 39, 41, 41, 31, 31, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 33, 34, 34, 34, + 34, 36, 37, 37, 37, 39, 41, 41, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 33, 34, 34, 34, 34, 36, 37, 37, + 37, 39, 41, 41, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 33, 33, 33, 33, 34, 34, 34, 34, 36, 37, 37, 37, 39, 41, 41, + 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 33, 33, 33, 33, 34, + 34, 34, 34, 35, 35, 35, 35, 37, 38, 38, 38, 40, 41, 41, 31, 32, 32, 32, + 32, 32, 32, 32, 32, 33, 33, 33, 33, 33, 34, 34, 34, 34, 35, 35, 35, 36, + 36, 36, 36, 38, 39, 39, 39, 40, 42, 42, 31, 32, 32, 32, 32, 32, 32, 32, + 32, 33, 33, 33, 33, 33, 34, 34, 34, 34, 35, 35, 35, 36, 36, 36, 36, 38, + 39, 39, 39, 40, 42, 42, 31, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, + 33, 33, 34, 34, 34, 34, 35, 35, 35, 36, 36, 36, 36, 38, 39, 39, 39, 40, + 42, 42, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 33, 34, 34, 34, + 34, 35, 36, 36, 36, 36, 37, 37, 37, 38, 40, 40, 40, 41, 42, 42, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 33, 34, 34, 34, 34, 35, 35, 35, 36, 37, 37, + 37, 37, 38, 38, 38, 39, 40, 40, 40, 41, 42, 42, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 33, 34, 34, 34, 34, 35, 35, 35, 36, 37, 37, 37, 37, 38, 38, + 38, 39, 40, 40, 40, 41, 42, 42, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, + 34, 34, 34, 34, 35, 35, 35, 36, 37, 37, 37, 37, 38, 38, 38, 39, 40, 40, + 40, 41, 42, 42, 33, 33, 33, 33, 33, 33, 33, 33, 33, 34, 34, 34, 34, 35, + 36, 36, 36, 37, 38, 38, 38, 39, 40, 40, 40, 41, 42, 42, 42, 44, 45, 45, + 34, 34, 34, 34, 34, 34, 33, 33, 33, 34, 35, 35, 35, 36, 37, 37, 37, 38, + 39, 39, 39, 41, 42, 42, 42, 44, 45, 45, 45, 46, 47, 47, 34, 34, 34, 34, + 34, 34, 33, 33, 33, 34, 35, 35, 35, 36, 37, 37, 37, 38, 39, 39, 39, 41, + 42, 42, 42, 44, 45, 45, 45, 46, 47, 47, 34, 34, 34, 34, 34, 34, 33, 33, + 33, 34, 35, 35, 35, 36, 37, 37, 37, 38, 39, 39, 39, 41, 42, 42, 42, 44, + 45, 45, 45, 46, 47, 47, 35, 34, 34, 34, 34, 34, 34, 34, 34, 35, 36, 36, + 36, 36, 37, 37, 37, 39, 41, 41, 41, 43, 45, 45, 45, 46, 47, 47, 47, 49, + 50, 50, 36, 35, 35, 35, 35, 35, 34, 34, 34, 35, 36, 36, 36, 37, 38, 38, + 38, 40, 42, 42, 42, 45, 48, 48, 48, 49, 50, 50, 50, 52, 54, 54, 36, 35, + 35, 35, 35, 35, 34, 34, 34, 35, 36, 36, 36, 37, 38, 38, 38, 40, 42, 42, + 42, 45, 48, 48, 48, 49, 50, 50, 50, 52, 54, 54, 36, 35, 35, 35, 35, 35, + 34, 34, 34, 35, 36, 36, 36, 37, 38, 38, 38, 40, 42, 42, 42, 45, 48, 48, + 48, 49, 50, 50, 50, 52, 54, 54, 37, 37, 37, 37, 37, 36, 36, 36, 36, 37, + 38, 38, 38, 38, 39, 39, 39, 41, 44, 44, 44, 46, 49, 49, 49, 51, 52, 52, + 52, 54, 56, 56, 39, 39, 38, 38, 38, 38, 37, 37, 37, 38, 39, 39, 39, 40, + 40, 40, 40, 42, 45, 45, 45, 47, 50, 50, 50, 52, 54, 54, 54, 56, 58, 58, + 39, 39, 38, 38, 38, 38, 37, 37, 37, 38, 39, 39, 39, 40, 40, 40, 40, 42, + 45, 45, 45, 47, 50, 50, 50, 52, 54, 54, 54, 56, 58, 58, 39, 39, 38, 38, + 38, 38, 37, 37, 37, 38, 39, 39, 39, 40, 40, 40, 40, 42, 45, 45, 45, 47, + 50, 50, 50, 52, 54, 54, 54, 56, 58, 58, 41, 41, 40, 40, 40, 39, 39, 39, + 39, 40, 40, 40, 40, 41, 41, 41, 41, 44, 46, 46, 46, 49, 52, 52, 52, 54, + 56, 56, 56, 58, 60, 60, 44, 43, 42, 42, 42, 41, 41, 41, 41, 41, 42, 42, + 42, 42, 42, 42, 42, 45, 47, 47, 47, 50, 54, 54, 54, 56, 58, 58, 58, 60, + 63, 63, 44, 43, 42, 42, 42, 41, 41, 41, 41, 41, 42, 42, 42, 42, 42, 42, + 42, 45, 47, 47, 47, 50, 54, 54, 54, 56, 58, 58, 58, 60, 63, 63], + /* Size 4x8 */ + [31, 32, 34, 39, 32, 32, 34, 38, 32, 33, 34, 38, 32, 33, 36, 40, 33, 34, + 38, 42, 34, 36, 41, 47, 37, 38, 44, 52, 40, 40, 46, 56], + /* Size 8x4 */ + [31, 32, 32, 32, 33, 34, 37, 40, 32, 32, 33, 33, 34, 36, 38, 40, 34, 34, + 34, 36, 38, 41, 44, 46, 39, 38, 38, 40, 42, 47, 52, 56], + /* Size 8x16 */ + [32, 31, 31, 32, 32, 36, 36, 44, 31, 32, 32, 32, 32, 35, 35, 42, 31, 32, + 32, 32, 32, 35, 35, 42, 31, 32, 32, 33, 33, 34, 34, 41, 31, 32, 32, 33, + 33, 34, 34, 41, 32, 32, 32, 34, 34, 36, 36, 42, 32, 32, 32, 34, 34, 36, + 36, 42, 32, 33, 33, 35, 35, 38, 38, 42, 32, 33, 33, 35, 35, 38, 38, 42, + 34, 34, 34, 37, 37, 42, 42, 48, 34, 34, 34, 37, 37, 42, 42, 48, 36, 34, + 34, 38, 38, 48, 48, 54, 36, 34, 34, 38, 38, 48, 48, 54, 39, 37, 37, 40, + 40, 50, 50, 58, 39, 37, 37, 40, 40, 50, 50, 58, 44, 41, 41, 43, 43, 53, + 53, 63], + /* Size 16x8 */ + [32, 31, 31, 31, 31, 32, 32, 32, 32, 34, 34, 36, 36, 39, 39, 44, 31, 32, + 32, 32, 32, 32, 32, 33, 33, 34, 34, 34, 34, 37, 37, 41, 31, 32, 32, 32, + 32, 32, 32, 33, 33, 34, 34, 34, 34, 37, 37, 41, 32, 32, 32, 33, 33, 34, + 34, 35, 35, 37, 37, 38, 38, 40, 40, 43, 32, 32, 32, 33, 33, 34, 34, 35, + 35, 37, 37, 38, 38, 40, 40, 43, 36, 35, 35, 34, 34, 36, 36, 38, 38, 42, + 42, 48, 48, 50, 50, 53, 36, 35, 35, 34, 34, 36, 36, 38, 38, 42, 42, 48, + 48, 50, 50, 53, 44, 42, 42, 41, 41, 42, 42, 42, 42, 48, 48, 54, 54, 58, + 58, 63], + /* Size 16x32 */ + [32, 31, 31, 31, 31, 32, 32, 32, 32, 34, 36, 36, 36, 39, 44, 44, 31, 31, + 31, 31, 31, 32, 32, 32, 32, 34, 35, 35, 35, 39, 43, 43, 31, 32, 32, 32, + 32, 32, 32, 32, 32, 34, 35, 35, 35, 38, 42, 42, 31, 32, 32, 32, 32, 32, + 32, 32, 32, 34, 35, 35, 35, 38, 42, 42, 31, 32, 32, 32, 32, 32, 32, 32, + 32, 34, 35, 35, 35, 38, 42, 42, 31, 32, 32, 32, 32, 32, 32, 32, 32, 34, + 35, 35, 35, 38, 41, 41, 31, 32, 32, 32, 32, 32, 33, 33, 33, 33, 34, 34, + 34, 37, 41, 41, 31, 32, 32, 32, 32, 32, 33, 33, 33, 33, 34, 34, 34, 37, + 41, 41, 31, 32, 32, 32, 32, 32, 33, 33, 33, 33, 34, 34, 34, 37, 41, 41, + 31, 32, 32, 32, 32, 33, 33, 33, 33, 34, 35, 35, 35, 38, 41, 41, 32, 32, + 32, 32, 32, 33, 34, 34, 34, 35, 36, 36, 36, 39, 42, 42, 32, 32, 32, 32, + 32, 33, 34, 34, 34, 35, 36, 36, 36, 39, 42, 42, 32, 32, 32, 32, 32, 33, + 34, 34, 34, 35, 36, 36, 36, 39, 42, 42, 32, 32, 32, 32, 32, 33, 34, 34, + 34, 36, 37, 37, 37, 40, 42, 42, 32, 32, 33, 33, 33, 34, 35, 35, 35, 37, + 38, 38, 38, 40, 42, 42, 32, 32, 33, 33, 33, 34, 35, 35, 35, 37, 38, 38, + 38, 40, 42, 42, 32, 32, 33, 33, 33, 34, 35, 35, 35, 37, 38, 38, 38, 40, + 42, 42, 33, 33, 33, 33, 33, 34, 36, 36, 36, 38, 40, 40, 40, 42, 45, 45, + 34, 34, 34, 34, 34, 35, 37, 37, 37, 39, 42, 42, 42, 45, 48, 48, 34, 34, + 34, 34, 34, 35, 37, 37, 37, 39, 42, 42, 42, 45, 48, 48, 34, 34, 34, 34, + 34, 35, 37, 37, 37, 39, 42, 42, 42, 45, 48, 48, 35, 34, 34, 34, 34, 36, + 37, 37, 37, 41, 45, 45, 45, 47, 50, 50, 36, 35, 34, 34, 34, 36, 38, 38, + 38, 43, 48, 48, 48, 51, 54, 54, 36, 35, 34, 34, 34, 36, 38, 38, 38, 43, + 48, 48, 48, 51, 54, 54, 36, 35, 34, 34, 34, 36, 38, 38, 38, 43, 48, 48, + 48, 51, 54, 54, 37, 37, 36, 36, 36, 38, 39, 39, 39, 44, 49, 49, 49, 52, + 56, 56, 39, 38, 37, 37, 37, 39, 40, 40, 40, 45, 50, 50, 50, 54, 58, 58, + 39, 38, 37, 37, 37, 39, 40, 40, 40, 45, 50, 50, 50, 54, 58, 58, 39, 38, + 37, 37, 37, 39, 40, 40, 40, 45, 50, 50, 50, 54, 58, 58, 41, 40, 39, 39, + 39, 40, 42, 42, 42, 46, 52, 52, 52, 56, 60, 60, 44, 42, 41, 41, 41, 42, + 43, 43, 43, 48, 53, 53, 53, 58, 63, 63, 44, 42, 41, 41, 41, 42, 43, 43, + 43, 48, 53, 53, 53, 58, 63, 63], + /* Size 32x16 */ + [32, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 33, + 34, 34, 34, 35, 36, 36, 36, 37, 39, 39, 39, 41, 44, 44, 31, 31, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 34, 34, 34, 34, + 35, 35, 35, 37, 38, 38, 38, 40, 42, 42, 31, 31, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 33, 33, 33, 33, 34, 34, 34, 34, 34, 34, 34, 36, + 37, 37, 37, 39, 41, 41, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 33, 33, 33, 33, 34, 34, 34, 34, 34, 34, 34, 36, 37, 37, 37, 39, + 41, 41, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, + 33, 33, 34, 34, 34, 34, 34, 34, 34, 36, 37, 37, 37, 39, 41, 41, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 33, 33, 34, 34, 34, 34, 35, 35, + 35, 36, 36, 36, 36, 38, 39, 39, 39, 40, 42, 42, 32, 32, 32, 32, 32, 32, + 33, 33, 33, 33, 34, 34, 34, 34, 35, 35, 35, 36, 37, 37, 37, 37, 38, 38, + 38, 39, 40, 40, 40, 42, 43, 43, 32, 32, 32, 32, 32, 32, 33, 33, 33, 33, + 34, 34, 34, 34, 35, 35, 35, 36, 37, 37, 37, 37, 38, 38, 38, 39, 40, 40, + 40, 42, 43, 43, 32, 32, 32, 32, 32, 32, 33, 33, 33, 33, 34, 34, 34, 34, + 35, 35, 35, 36, 37, 37, 37, 37, 38, 38, 38, 39, 40, 40, 40, 42, 43, 43, + 34, 34, 34, 34, 34, 34, 33, 33, 33, 34, 35, 35, 35, 36, 37, 37, 37, 38, + 39, 39, 39, 41, 43, 43, 43, 44, 45, 45, 45, 46, 48, 48, 36, 35, 35, 35, + 35, 35, 34, 34, 34, 35, 36, 36, 36, 37, 38, 38, 38, 40, 42, 42, 42, 45, + 48, 48, 48, 49, 50, 50, 50, 52, 53, 53, 36, 35, 35, 35, 35, 35, 34, 34, + 34, 35, 36, 36, 36, 37, 38, 38, 38, 40, 42, 42, 42, 45, 48, 48, 48, 49, + 50, 50, 50, 52, 53, 53, 36, 35, 35, 35, 35, 35, 34, 34, 34, 35, 36, 36, + 36, 37, 38, 38, 38, 40, 42, 42, 42, 45, 48, 48, 48, 49, 50, 50, 50, 52, + 53, 53, 39, 39, 38, 38, 38, 38, 37, 37, 37, 38, 39, 39, 39, 40, 40, 40, + 40, 42, 45, 45, 45, 47, 51, 51, 51, 52, 54, 54, 54, 56, 58, 58, 44, 43, + 42, 42, 42, 41, 41, 41, 41, 41, 42, 42, 42, 42, 42, 42, 42, 45, 48, 48, + 48, 50, 54, 54, 54, 56, 58, 58, 58, 60, 63, 63, 44, 43, 42, 42, 42, 41, + 41, 41, 41, 41, 42, 42, 42, 42, 42, 42, 42, 45, 48, 48, 48, 50, 54, 54, + 54, 56, 58, 58, 58, 60, 63, 63], + /* Size 4x16 */ + [31, 32, 34, 39, 32, 32, 34, 38, 32, 32, 34, 38, 32, 32, 33, 37, 32, 32, + 33, 37, 32, 33, 35, 39, 32, 33, 35, 39, 32, 34, 37, 40, 32, 34, 37, 40, + 34, 35, 39, 45, 34, 35, 39, 45, 35, 36, 43, 51, 35, 36, 43, 51, 38, 39, + 45, 54, 38, 39, 45, 54, 42, 42, 48, 58], + /* Size 16x4 */ + [31, 32, 32, 32, 32, 32, 32, 32, 32, 34, 34, 35, 35, 38, 38, 42, 32, 32, + 32, 32, 32, 33, 33, 34, 34, 35, 35, 36, 36, 39, 39, 42, 34, 34, 34, 33, + 33, 35, 35, 37, 37, 39, 39, 43, 43, 45, 45, 48, 39, 38, 38, 37, 37, 39, + 39, 40, 40, 45, 45, 51, 51, 54, 54, 58], + /* Size 8x32 */ + [32, 31, 31, 32, 32, 36, 36, 44, 31, 31, 31, 32, 32, 35, 35, 43, 31, 32, + 32, 32, 32, 35, 35, 42, 31, 32, 32, 32, 32, 35, 35, 42, 31, 32, 32, 32, + 32, 35, 35, 42, 31, 32, 32, 32, 32, 35, 35, 41, 31, 32, 32, 33, 33, 34, + 34, 41, 31, 32, 32, 33, 33, 34, 34, 41, 31, 32, 32, 33, 33, 34, 34, 41, + 31, 32, 32, 33, 33, 35, 35, 41, 32, 32, 32, 34, 34, 36, 36, 42, 32, 32, + 32, 34, 34, 36, 36, 42, 32, 32, 32, 34, 34, 36, 36, 42, 32, 32, 32, 34, + 34, 37, 37, 42, 32, 33, 33, 35, 35, 38, 38, 42, 32, 33, 33, 35, 35, 38, + 38, 42, 32, 33, 33, 35, 35, 38, 38, 42, 33, 33, 33, 36, 36, 40, 40, 45, + 34, 34, 34, 37, 37, 42, 42, 48, 34, 34, 34, 37, 37, 42, 42, 48, 34, 34, + 34, 37, 37, 42, 42, 48, 35, 34, 34, 37, 37, 45, 45, 50, 36, 34, 34, 38, + 38, 48, 48, 54, 36, 34, 34, 38, 38, 48, 48, 54, 36, 34, 34, 38, 38, 48, + 48, 54, 37, 36, 36, 39, 39, 49, 49, 56, 39, 37, 37, 40, 40, 50, 50, 58, + 39, 37, 37, 40, 40, 50, 50, 58, 39, 37, 37, 40, 40, 50, 50, 58, 41, 39, + 39, 42, 42, 52, 52, 60, 44, 41, 41, 43, 43, 53, 53, 63, 44, 41, 41, 43, + 43, 53, 53, 63], + /* Size 32x8 */ + [32, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 33, + 34, 34, 34, 35, 36, 36, 36, 37, 39, 39, 39, 41, 44, 44, 31, 31, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 33, 34, 34, 34, 34, + 34, 34, 34, 36, 37, 37, 37, 39, 41, 41, 31, 31, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 33, 33, 33, 33, 34, 34, 34, 34, 34, 34, 34, 36, + 37, 37, 37, 39, 41, 41, 32, 32, 32, 32, 32, 32, 33, 33, 33, 33, 34, 34, + 34, 34, 35, 35, 35, 36, 37, 37, 37, 37, 38, 38, 38, 39, 40, 40, 40, 42, + 43, 43, 32, 32, 32, 32, 32, 32, 33, 33, 33, 33, 34, 34, 34, 34, 35, 35, + 35, 36, 37, 37, 37, 37, 38, 38, 38, 39, 40, 40, 40, 42, 43, 43, 36, 35, + 35, 35, 35, 35, 34, 34, 34, 35, 36, 36, 36, 37, 38, 38, 38, 40, 42, 42, + 42, 45, 48, 48, 48, 49, 50, 50, 50, 52, 53, 53, 36, 35, 35, 35, 35, 35, + 34, 34, 34, 35, 36, 36, 36, 37, 38, 38, 38, 40, 42, 42, 42, 45, 48, 48, + 48, 49, 50, 50, 50, 52, 53, 53, 44, 43, 42, 42, 42, 41, 41, 41, 41, 41, + 42, 42, 42, 42, 42, 42, 42, 45, 48, 48, 48, 50, 54, 54, 54, 56, 58, 58, + 58, 60, 63, 63]], + [ /* Chroma */ + /* Size 4x4 */ + [31, 34, 42, 47, 34, 39, 45, 46, 42, 45, 48, 49, 47, 46, 49, 54], + /* Size 8x8 */ + [31, 31, 32, 35, 39, 45, 48, 48, 31, 31, 33, 37, 41, 44, 46, 46, 32, 33, + 35, 39, 42, 45, 46, 45, 35, 37, 39, 43, 45, 47, 47, 46, 39, 41, 42, 45, + 47, 48, 48, 47, 45, 44, 45, 47, 48, 50, 51, 51, 48, 46, 46, 47, 48, 51, + 53, 54, 48, 46, 45, 46, 47, 51, 54, 56], + /* Size 16x16 */ + [32, 31, 31, 30, 30, 33, 33, 36, 36, 41, 41, 49, 49, 48, 48, 49, 31, 31, + 31, 31, 31, 34, 34, 38, 38, 42, 42, 47, 47, 47, 47, 47, 31, 31, 31, 31, + 31, 34, 34, 38, 38, 42, 42, 47, 47, 47, 47, 47, 30, 31, 31, 32, 32, 35, + 35, 40, 40, 42, 42, 46, 46, 45, 45, 45, 30, 31, 31, 32, 32, 35, 35, 40, + 40, 42, 42, 46, 46, 45, 45, 45, 33, 34, 34, 35, 35, 39, 39, 43, 43, 45, + 45, 47, 47, 46, 46, 45, 33, 34, 34, 35, 35, 39, 39, 43, 43, 45, 45, 47, + 47, 46, 46, 45, 36, 38, 38, 40, 40, 43, 43, 47, 47, 47, 47, 48, 48, 46, + 46, 45, 36, 38, 38, 40, 40, 43, 43, 47, 47, 47, 47, 48, 48, 46, 46, 45, + 41, 42, 42, 42, 42, 45, 45, 47, 47, 48, 48, 50, 50, 49, 49, 49, 41, 42, + 42, 42, 42, 45, 45, 47, 47, 48, 48, 50, 50, 49, 49, 49, 49, 47, 47, 46, + 46, 47, 47, 48, 48, 50, 50, 53, 53, 53, 53, 53, 49, 47, 47, 46, 46, 47, + 47, 48, 48, 50, 50, 53, 53, 53, 53, 53, 48, 47, 47, 45, 45, 46, 46, 46, + 46, 49, 49, 53, 53, 54, 54, 55, 48, 47, 47, 45, 45, 46, 46, 46, 46, 49, + 49, 53, 53, 54, 54, 55, 49, 47, 47, 45, 45, 45, 45, 45, 45, 49, 49, 53, + 53, 55, 55, 58], + /* Size 32x32 */ + [32, 31, 31, 31, 31, 31, 30, 30, 30, 32, 33, 33, 33, 35, 36, 36, 36, 39, + 41, 41, 41, 45, 49, 49, 49, 49, 48, 48, 48, 49, 49, 49, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 32, 34, 34, 34, 35, 37, 37, 37, 39, 42, 42, 42, 45, + 48, 48, 48, 48, 48, 48, 48, 48, 48, 48, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 33, 34, 34, 34, 36, 38, 38, 38, 40, 42, 42, 42, 45, 47, 47, 47, 47, + 47, 47, 47, 47, 47, 47, 31, 31, 31, 31, 31, 31, 31, 31, 31, 33, 34, 34, + 34, 36, 38, 38, 38, 40, 42, 42, 42, 45, 47, 47, 47, 47, 47, 47, 47, 47, + 47, 47, 31, 31, 31, 31, 31, 31, 31, 31, 31, 33, 34, 34, 34, 36, 38, 38, + 38, 40, 42, 42, 42, 45, 47, 47, 47, 47, 47, 47, 47, 47, 47, 47, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 33, 35, 35, 35, 37, 39, 39, 39, 41, 42, 42, + 42, 44, 47, 47, 47, 46, 46, 46, 46, 46, 46, 46, 30, 31, 31, 31, 31, 31, + 32, 32, 32, 33, 35, 35, 35, 37, 40, 40, 40, 41, 42, 42, 42, 44, 46, 46, + 46, 46, 45, 45, 45, 45, 45, 45, 30, 31, 31, 31, 31, 31, 32, 32, 32, 33, + 35, 35, 35, 37, 40, 40, 40, 41, 42, 42, 42, 44, 46, 46, 46, 46, 45, 45, + 45, 45, 45, 45, 30, 31, 31, 31, 31, 31, 32, 32, 32, 33, 35, 35, 35, 37, + 40, 40, 40, 41, 42, 42, 42, 44, 46, 46, 46, 46, 45, 45, 45, 45, 45, 45, + 32, 32, 33, 33, 33, 33, 33, 33, 33, 35, 37, 37, 37, 39, 41, 41, 41, 42, + 43, 43, 43, 45, 47, 47, 47, 46, 46, 46, 46, 45, 45, 45, 33, 34, 34, 34, + 34, 35, 35, 35, 35, 37, 39, 39, 39, 41, 43, 43, 43, 44, 45, 45, 45, 46, + 47, 47, 47, 47, 46, 46, 46, 46, 45, 45, 33, 34, 34, 34, 34, 35, 35, 35, + 35, 37, 39, 39, 39, 41, 43, 43, 43, 44, 45, 45, 45, 46, 47, 47, 47, 47, + 46, 46, 46, 46, 45, 45, 33, 34, 34, 34, 34, 35, 35, 35, 35, 37, 39, 39, + 39, 41, 43, 43, 43, 44, 45, 45, 45, 46, 47, 47, 47, 47, 46, 46, 46, 46, + 45, 45, 35, 35, 36, 36, 36, 37, 37, 37, 37, 39, 41, 41, 41, 43, 45, 45, + 45, 45, 46, 46, 46, 47, 47, 47, 47, 47, 46, 46, 46, 46, 45, 45, 36, 37, + 38, 38, 38, 39, 40, 40, 40, 41, 43, 43, 43, 45, 47, 47, 47, 47, 47, 47, + 47, 47, 48, 48, 48, 47, 46, 46, 46, 46, 45, 45, 36, 37, 38, 38, 38, 39, + 40, 40, 40, 41, 43, 43, 43, 45, 47, 47, 47, 47, 47, 47, 47, 47, 48, 48, + 48, 47, 46, 46, 46, 46, 45, 45, 36, 37, 38, 38, 38, 39, 40, 40, 40, 41, + 43, 43, 43, 45, 47, 47, 47, 47, 47, 47, 47, 47, 48, 48, 48, 47, 46, 46, + 46, 46, 45, 45, 39, 39, 40, 40, 40, 41, 41, 41, 41, 42, 44, 44, 44, 45, + 47, 47, 47, 47, 48, 48, 48, 48, 49, 49, 49, 48, 48, 48, 48, 47, 47, 47, + 41, 42, 42, 42, 42, 42, 42, 42, 42, 43, 45, 45, 45, 46, 47, 47, 47, 48, + 48, 48, 48, 49, 50, 50, 50, 50, 49, 49, 49, 49, 49, 49, 41, 42, 42, 42, + 42, 42, 42, 42, 42, 43, 45, 45, 45, 46, 47, 47, 47, 48, 48, 48, 48, 49, + 50, 50, 50, 50, 49, 49, 49, 49, 49, 49, 41, 42, 42, 42, 42, 42, 42, 42, + 42, 43, 45, 45, 45, 46, 47, 47, 47, 48, 48, 48, 48, 49, 50, 50, 50, 50, + 49, 49, 49, 49, 49, 49, 45, 45, 45, 45, 45, 44, 44, 44, 44, 45, 46, 46, + 46, 47, 47, 47, 47, 48, 49, 49, 49, 50, 51, 51, 51, 51, 51, 51, 51, 51, + 51, 51, 49, 48, 47, 47, 47, 47, 46, 46, 46, 47, 47, 47, 47, 47, 48, 48, + 48, 49, 50, 50, 50, 51, 53, 53, 53, 53, 53, 53, 53, 53, 53, 53, 49, 48, + 47, 47, 47, 47, 46, 46, 46, 47, 47, 47, 47, 47, 48, 48, 48, 49, 50, 50, + 50, 51, 53, 53, 53, 53, 53, 53, 53, 53, 53, 53, 49, 48, 47, 47, 47, 47, + 46, 46, 46, 47, 47, 47, 47, 47, 48, 48, 48, 49, 50, 50, 50, 51, 53, 53, + 53, 53, 53, 53, 53, 53, 53, 53, 49, 48, 47, 47, 47, 46, 46, 46, 46, 46, + 47, 47, 47, 47, 47, 47, 47, 48, 50, 50, 50, 51, 53, 53, 53, 53, 53, 53, + 53, 54, 54, 54, 48, 48, 47, 47, 47, 46, 45, 45, 45, 46, 46, 46, 46, 46, + 46, 46, 46, 48, 49, 49, 49, 51, 53, 53, 53, 53, 54, 54, 54, 55, 55, 55, + 48, 48, 47, 47, 47, 46, 45, 45, 45, 46, 46, 46, 46, 46, 46, 46, 46, 48, + 49, 49, 49, 51, 53, 53, 53, 53, 54, 54, 54, 55, 55, 55, 48, 48, 47, 47, + 47, 46, 45, 45, 45, 46, 46, 46, 46, 46, 46, 46, 46, 48, 49, 49, 49, 51, + 53, 53, 53, 53, 54, 54, 54, 55, 55, 55, 49, 48, 47, 47, 47, 46, 45, 45, + 45, 45, 46, 46, 46, 46, 46, 46, 46, 47, 49, 49, 49, 51, 53, 53, 53, 54, + 55, 55, 55, 56, 57, 57, 49, 48, 47, 47, 47, 46, 45, 45, 45, 45, 45, 45, + 45, 45, 45, 45, 45, 47, 49, 49, 49, 51, 53, 53, 53, 54, 55, 55, 55, 57, + 58, 58, 49, 48, 47, 47, 47, 46, 45, 45, 45, 45, 45, 45, 45, 45, 45, 45, + 45, 47, 49, 49, 49, 51, 53, 53, 53, 54, 55, 55, 55, 57, 58, 58], + /* Size 4x8 */ + [31, 34, 42, 48, 31, 35, 42, 46, 33, 37, 44, 46, 36, 41, 46, 46, 40, 44, + 48, 48, 45, 46, 49, 51, 47, 47, 50, 54, 47, 46, 49, 55], + /* Size 8x4 */ + [31, 31, 33, 36, 40, 45, 47, 47, 34, 35, 37, 41, 44, 46, 47, 46, 42, 42, + 44, 46, 48, 49, 50, 49, 48, 46, 46, 46, 48, 51, 54, 55], + /* Size 8x16 */ + [32, 31, 31, 37, 37, 48, 48, 49, 31, 31, 31, 38, 38, 47, 47, 47, 31, 31, + 31, 38, 38, 47, 47, 47, 30, 32, 32, 40, 40, 46, 46, 45, 30, 32, 32, 40, + 40, 46, 46, 45, 33, 36, 36, 43, 43, 47, 47, 46, 33, 36, 36, 43, 43, 47, + 47, 46, 37, 40, 40, 47, 47, 47, 47, 45, 37, 40, 40, 47, 47, 47, 47, 45, + 42, 43, 43, 47, 47, 50, 50, 49, 42, 43, 43, 47, 47, 50, 50, 49, 49, 46, + 46, 48, 48, 53, 53, 53, 49, 46, 46, 48, 48, 53, 53, 53, 48, 46, 46, 47, + 47, 53, 53, 56, 48, 46, 46, 47, 47, 53, 53, 56, 49, 45, 45, 46, 46, 53, + 53, 58], + /* Size 16x8 */ + [32, 31, 31, 30, 30, 33, 33, 37, 37, 42, 42, 49, 49, 48, 48, 49, 31, 31, + 31, 32, 32, 36, 36, 40, 40, 43, 43, 46, 46, 46, 46, 45, 31, 31, 31, 32, + 32, 36, 36, 40, 40, 43, 43, 46, 46, 46, 46, 45, 37, 38, 38, 40, 40, 43, + 43, 47, 47, 47, 47, 48, 48, 47, 47, 46, 37, 38, 38, 40, 40, 43, 43, 47, + 47, 47, 47, 48, 48, 47, 47, 46, 48, 47, 47, 46, 46, 47, 47, 47, 47, 50, + 50, 53, 53, 53, 53, 53, 48, 47, 47, 46, 46, 47, 47, 47, 47, 50, 50, 53, + 53, 53, 53, 53, 49, 47, 47, 45, 45, 46, 46, 45, 45, 49, 49, 53, 53, 56, + 56, 58], + /* Size 16x32 */ + [32, 31, 31, 31, 31, 33, 37, 37, 37, 42, 48, 48, 48, 48, 49, 49, 31, 31, + 31, 31, 31, 34, 37, 37, 37, 42, 47, 47, 47, 48, 48, 48, 31, 31, 31, 31, + 31, 34, 38, 38, 38, 42, 47, 47, 47, 47, 47, 47, 31, 31, 31, 31, 31, 34, + 38, 38, 38, 42, 47, 47, 47, 47, 47, 47, 31, 31, 31, 31, 31, 34, 38, 38, + 38, 42, 47, 47, 47, 47, 47, 47, 31, 31, 32, 32, 32, 35, 39, 39, 39, 42, + 46, 46, 46, 46, 46, 46, 30, 31, 32, 32, 32, 35, 40, 40, 40, 42, 46, 46, + 46, 45, 45, 45, 30, 31, 32, 32, 32, 35, 40, 40, 40, 42, 46, 46, 46, 45, + 45, 45, 30, 31, 32, 32, 32, 35, 40, 40, 40, 42, 46, 46, 46, 45, 45, 45, + 32, 33, 34, 34, 34, 37, 41, 41, 41, 44, 46, 46, 46, 46, 45, 45, 33, 34, + 36, 36, 36, 39, 43, 43, 43, 45, 47, 47, 47, 46, 46, 46, 33, 34, 36, 36, + 36, 39, 43, 43, 43, 45, 47, 47, 47, 46, 46, 46, 33, 34, 36, 36, 36, 39, + 43, 43, 43, 45, 47, 47, 47, 46, 46, 46, 35, 36, 38, 38, 38, 41, 45, 45, + 45, 46, 47, 47, 47, 46, 45, 45, 37, 38, 40, 40, 40, 43, 47, 47, 47, 47, + 47, 47, 47, 46, 45, 45, 37, 38, 40, 40, 40, 43, 47, 47, 47, 47, 47, 47, + 47, 46, 45, 45, 37, 38, 40, 40, 40, 43, 47, 47, 47, 47, 47, 47, 47, 46, + 45, 45, 39, 40, 41, 41, 41, 44, 47, 47, 47, 48, 49, 49, 49, 48, 47, 47, + 42, 42, 43, 43, 43, 45, 47, 47, 47, 48, 50, 50, 50, 50, 49, 49, 42, 42, + 43, 43, 43, 45, 47, 47, 47, 48, 50, 50, 50, 50, 49, 49, 42, 42, 43, 43, + 43, 45, 47, 47, 47, 48, 50, 50, 50, 50, 49, 49, 45, 45, 44, 44, 44, 46, + 47, 47, 47, 49, 51, 51, 51, 51, 51, 51, 49, 48, 46, 46, 46, 47, 48, 48, + 48, 50, 53, 53, 53, 53, 53, 53, 49, 48, 46, 46, 46, 47, 48, 48, 48, 50, + 53, 53, 53, 53, 53, 53, 49, 48, 46, 46, 46, 47, 48, 48, 48, 50, 53, 53, + 53, 53, 53, 53, 48, 47, 46, 46, 46, 47, 47, 47, 47, 50, 53, 53, 53, 54, + 54, 54, 48, 47, 46, 46, 46, 46, 47, 47, 47, 50, 53, 53, 53, 54, 56, 56, + 48, 47, 46, 46, 46, 46, 47, 47, 47, 50, 53, 53, 53, 54, 56, 56, 48, 47, + 46, 46, 46, 46, 47, 47, 47, 50, 53, 53, 53, 54, 56, 56, 48, 47, 45, 45, + 45, 46, 46, 46, 46, 49, 53, 53, 53, 55, 57, 57, 49, 47, 45, 45, 45, 45, + 46, 46, 46, 49, 53, 53, 53, 56, 58, 58, 49, 47, 45, 45, 45, 45, 46, 46, + 46, 49, 53, 53, 53, 56, 58, 58], + /* Size 32x16 */ + [32, 31, 31, 31, 31, 31, 30, 30, 30, 32, 33, 33, 33, 35, 37, 37, 37, 39, + 42, 42, 42, 45, 49, 49, 49, 48, 48, 48, 48, 48, 49, 49, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 33, 34, 34, 34, 36, 38, 38, 38, 40, 42, 42, 42, 45, + 48, 48, 48, 47, 47, 47, 47, 47, 47, 47, 31, 31, 31, 31, 31, 32, 32, 32, + 32, 34, 36, 36, 36, 38, 40, 40, 40, 41, 43, 43, 43, 44, 46, 46, 46, 46, + 46, 46, 46, 45, 45, 45, 31, 31, 31, 31, 31, 32, 32, 32, 32, 34, 36, 36, + 36, 38, 40, 40, 40, 41, 43, 43, 43, 44, 46, 46, 46, 46, 46, 46, 46, 45, + 45, 45, 31, 31, 31, 31, 31, 32, 32, 32, 32, 34, 36, 36, 36, 38, 40, 40, + 40, 41, 43, 43, 43, 44, 46, 46, 46, 46, 46, 46, 46, 45, 45, 45, 33, 34, + 34, 34, 34, 35, 35, 35, 35, 37, 39, 39, 39, 41, 43, 43, 43, 44, 45, 45, + 45, 46, 47, 47, 47, 47, 46, 46, 46, 46, 45, 45, 37, 37, 38, 38, 38, 39, + 40, 40, 40, 41, 43, 43, 43, 45, 47, 47, 47, 47, 47, 47, 47, 47, 48, 48, + 48, 47, 47, 47, 47, 46, 46, 46, 37, 37, 38, 38, 38, 39, 40, 40, 40, 41, + 43, 43, 43, 45, 47, 47, 47, 47, 47, 47, 47, 47, 48, 48, 48, 47, 47, 47, + 47, 46, 46, 46, 37, 37, 38, 38, 38, 39, 40, 40, 40, 41, 43, 43, 43, 45, + 47, 47, 47, 47, 47, 47, 47, 47, 48, 48, 48, 47, 47, 47, 47, 46, 46, 46, + 42, 42, 42, 42, 42, 42, 42, 42, 42, 44, 45, 45, 45, 46, 47, 47, 47, 48, + 48, 48, 48, 49, 50, 50, 50, 50, 50, 50, 50, 49, 49, 49, 48, 47, 47, 47, + 47, 46, 46, 46, 46, 46, 47, 47, 47, 47, 47, 47, 47, 49, 50, 50, 50, 51, + 53, 53, 53, 53, 53, 53, 53, 53, 53, 53, 48, 47, 47, 47, 47, 46, 46, 46, + 46, 46, 47, 47, 47, 47, 47, 47, 47, 49, 50, 50, 50, 51, 53, 53, 53, 53, + 53, 53, 53, 53, 53, 53, 48, 47, 47, 47, 47, 46, 46, 46, 46, 46, 47, 47, + 47, 47, 47, 47, 47, 49, 50, 50, 50, 51, 53, 53, 53, 53, 53, 53, 53, 53, + 53, 53, 48, 48, 47, 47, 47, 46, 45, 45, 45, 46, 46, 46, 46, 46, 46, 46, + 46, 48, 50, 50, 50, 51, 53, 53, 53, 54, 54, 54, 54, 55, 56, 56, 49, 48, + 47, 47, 47, 46, 45, 45, 45, 45, 46, 46, 46, 45, 45, 45, 45, 47, 49, 49, + 49, 51, 53, 53, 53, 54, 56, 56, 56, 57, 58, 58, 49, 48, 47, 47, 47, 46, + 45, 45, 45, 45, 46, 46, 46, 45, 45, 45, 45, 47, 49, 49, 49, 51, 53, 53, + 53, 54, 56, 56, 56, 57, 58, 58], + /* Size 4x16 */ + [31, 33, 42, 48, 31, 34, 42, 47, 31, 34, 42, 47, 31, 35, 42, 45, 31, 35, + 42, 45, 34, 39, 45, 46, 34, 39, 45, 46, 38, 43, 47, 46, 38, 43, 47, 46, + 42, 45, 48, 50, 42, 45, 48, 50, 48, 47, 50, 53, 48, 47, 50, 53, 47, 46, + 50, 54, 47, 46, 50, 54, 47, 45, 49, 56], + /* Size 16x4 */ + [31, 31, 31, 31, 31, 34, 34, 38, 38, 42, 42, 48, 48, 47, 47, 47, 33, 34, + 34, 35, 35, 39, 39, 43, 43, 45, 45, 47, 47, 46, 46, 45, 42, 42, 42, 42, + 42, 45, 45, 47, 47, 48, 48, 50, 50, 50, 50, 49, 48, 47, 47, 45, 45, 46, + 46, 46, 46, 50, 50, 53, 53, 54, 54, 56], + /* Size 8x32 */ + [32, 31, 31, 37, 37, 48, 48, 49, 31, 31, 31, 37, 37, 47, 47, 48, 31, 31, + 31, 38, 38, 47, 47, 47, 31, 31, 31, 38, 38, 47, 47, 47, 31, 31, 31, 38, + 38, 47, 47, 47, 31, 32, 32, 39, 39, 46, 46, 46, 30, 32, 32, 40, 40, 46, + 46, 45, 30, 32, 32, 40, 40, 46, 46, 45, 30, 32, 32, 40, 40, 46, 46, 45, + 32, 34, 34, 41, 41, 46, 46, 45, 33, 36, 36, 43, 43, 47, 47, 46, 33, 36, + 36, 43, 43, 47, 47, 46, 33, 36, 36, 43, 43, 47, 47, 46, 35, 38, 38, 45, + 45, 47, 47, 45, 37, 40, 40, 47, 47, 47, 47, 45, 37, 40, 40, 47, 47, 47, + 47, 45, 37, 40, 40, 47, 47, 47, 47, 45, 39, 41, 41, 47, 47, 49, 49, 47, + 42, 43, 43, 47, 47, 50, 50, 49, 42, 43, 43, 47, 47, 50, 50, 49, 42, 43, + 43, 47, 47, 50, 50, 49, 45, 44, 44, 47, 47, 51, 51, 51, 49, 46, 46, 48, + 48, 53, 53, 53, 49, 46, 46, 48, 48, 53, 53, 53, 49, 46, 46, 48, 48, 53, + 53, 53, 48, 46, 46, 47, 47, 53, 53, 54, 48, 46, 46, 47, 47, 53, 53, 56, + 48, 46, 46, 47, 47, 53, 53, 56, 48, 46, 46, 47, 47, 53, 53, 56, 48, 45, + 45, 46, 46, 53, 53, 57, 49, 45, 45, 46, 46, 53, 53, 58, 49, 45, 45, 46, + 46, 53, 53, 58], + /* Size 32x8 */ + [32, 31, 31, 31, 31, 31, 30, 30, 30, 32, 33, 33, 33, 35, 37, 37, 37, 39, + 42, 42, 42, 45, 49, 49, 49, 48, 48, 48, 48, 48, 49, 49, 31, 31, 31, 31, + 31, 32, 32, 32, 32, 34, 36, 36, 36, 38, 40, 40, 40, 41, 43, 43, 43, 44, + 46, 46, 46, 46, 46, 46, 46, 45, 45, 45, 31, 31, 31, 31, 31, 32, 32, 32, + 32, 34, 36, 36, 36, 38, 40, 40, 40, 41, 43, 43, 43, 44, 46, 46, 46, 46, + 46, 46, 46, 45, 45, 45, 37, 37, 38, 38, 38, 39, 40, 40, 40, 41, 43, 43, + 43, 45, 47, 47, 47, 47, 47, 47, 47, 47, 48, 48, 48, 47, 47, 47, 47, 46, + 46, 46, 37, 37, 38, 38, 38, 39, 40, 40, 40, 41, 43, 43, 43, 45, 47, 47, + 47, 47, 47, 47, 47, 47, 48, 48, 48, 47, 47, 47, 47, 46, 46, 46, 48, 47, + 47, 47, 47, 46, 46, 46, 46, 46, 47, 47, 47, 47, 47, 47, 47, 49, 50, 50, + 50, 51, 53, 53, 53, 53, 53, 53, 53, 53, 53, 53, 48, 47, 47, 47, 47, 46, + 46, 46, 46, 46, 47, 47, 47, 47, 47, 47, 47, 49, 50, 50, 50, 51, 53, 53, + 53, 53, 53, 53, 53, 53, 53, 53, 49, 48, 47, 47, 47, 46, 45, 45, 45, 45, + 46, 46, 46, 45, 45, 45, 45, 47, 49, 49, 49, 51, 53, 53, 53, 54, 56, 56, + 56, 57, 58, 58]] + ], + [ + [ /* Luma */ + /* Size 4x4 */ + [32, 32, 32, 35, 32, 32, 33, 35, 32, 33, 35, 38, 35, 35, 38, 46], + /* Size 8x8 */ + [31, 31, 31, 32, 32, 32, 34, 35, 31, 32, 32, 32, 32, 33, 34, 35, 31, 32, + 32, 32, 32, 33, 33, 34, 32, 32, 32, 33, 34, 34, 35, 36, 32, 32, 32, 34, + 35, 35, 36, 38, 32, 33, 33, 34, 35, 36, 38, 40, 34, 34, 33, 35, 36, 38, + 39, 42, 35, 35, 34, 36, 38, 40, 42, 48], + /* Size 16x16 */ + [32, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 33, 34, 34, 36, 36, 31, 31, + 31, 32, 32, 32, 32, 32, 32, 32, 32, 33, 34, 34, 35, 35, 31, 31, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 33, 34, 34, 35, 35, 31, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 33, 34, 34, 35, 35, 31, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 33, 33, 34, 34, 34, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 33, 33, 34, 34, 34, 31, 32, 32, 32, 32, 32, 33, 33, 33, 33, 33, 34, + 35, 35, 36, 36, 31, 32, 32, 32, 32, 32, 33, 33, 33, 34, 34, 35, 35, 36, + 36, 36, 32, 32, 32, 32, 32, 32, 33, 33, 34, 34, 34, 35, 36, 36, 37, 37, + 32, 32, 32, 32, 32, 32, 33, 34, 34, 35, 35, 36, 37, 37, 38, 38, 32, 32, + 32, 32, 32, 32, 33, 34, 34, 35, 35, 36, 37, 37, 38, 38, 33, 33, 33, 33, + 33, 33, 34, 35, 35, 36, 36, 38, 39, 40, 42, 42, 34, 34, 34, 34, 33, 33, + 35, 35, 36, 37, 37, 39, 39, 41, 42, 42, 34, 34, 34, 34, 34, 34, 35, 36, + 36, 37, 37, 40, 41, 42, 45, 45, 36, 35, 35, 35, 34, 34, 36, 36, 37, 38, + 38, 42, 42, 45, 48, 48, 36, 35, 35, 35, 34, 34, 36, 36, 37, 38, 38, 42, + 42, 45, 48, 48], + /* Size 32x32 */ + [32, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, + 32, 32, 32, 32, 33, 34, 34, 34, 34, 35, 36, 36, 36, 37, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 33, 34, 34, 34, 34, 35, 35, 35, 35, 37, 31, 31, 31, 31, 31, 31, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 34, 34, 34, + 34, 35, 35, 35, 35, 36, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 34, 34, 34, 34, 35, 35, 35, + 35, 36, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 33, 33, 34, 34, 34, 34, 35, 35, 35, 35, 36, 31, 31, + 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 33, 33, 34, 34, 34, 34, 35, 35, 35, 35, 36, 31, 31, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 34, + 34, 34, 34, 34, 35, 35, 35, 36, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 33, 33, 34, 34, + 34, 34, 34, 35, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 33, 33, 34, 34, 34, 34, 34, 35, + 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 33, 33, 33, 33, 33, 34, 34, 34, 34, 34, 35, 31, 31, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, + 33, 33, 33, 33, 34, 34, 34, 34, 34, 35, 31, 31, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 34, 34, 34, 34, + 34, 35, 35, 35, 35, 36, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, + 33, 33, 33, 33, 33, 33, 33, 33, 33, 34, 34, 35, 35, 35, 35, 36, 36, 36, + 36, 37, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 33, 33, + 33, 34, 34, 34, 34, 34, 35, 35, 35, 35, 36, 36, 36, 36, 36, 37, 31, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 33, 33, 33, 34, 34, 34, + 34, 34, 35, 35, 35, 35, 36, 36, 36, 36, 36, 37, 31, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 33, 33, 33, 33, 33, 33, 34, 34, 34, 34, 34, 35, 35, + 35, 35, 36, 36, 36, 36, 36, 37, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 33, 33, 33, 33, 33, 34, 34, 34, 34, 34, 35, 35, 36, 36, 36, 36, 37, + 37, 37, 37, 38, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 34, + 34, 34, 34, 35, 35, 35, 35, 35, 36, 36, 36, 36, 37, 37, 38, 38, 38, 39, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 34, 34, 34, 34, 35, + 35, 35, 35, 36, 36, 37, 37, 37, 37, 38, 38, 38, 38, 39, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 33, 33, 34, 34, 34, 34, 35, 35, 35, 35, 36, + 36, 37, 37, 37, 37, 38, 38, 38, 38, 39, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 33, 33, 34, 34, 34, 34, 35, 35, 35, 35, 36, 36, 37, 37, 37, + 37, 38, 38, 38, 38, 39, 32, 32, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, + 34, 34, 34, 34, 35, 35, 36, 36, 36, 36, 37, 38, 38, 38, 38, 39, 40, 40, + 40, 41, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 34, 34, 35, 35, 35, + 35, 36, 36, 36, 36, 37, 38, 39, 39, 39, 40, 41, 42, 42, 42, 42, 34, 34, + 34, 34, 34, 34, 34, 33, 33, 33, 33, 34, 35, 35, 35, 35, 36, 36, 37, 37, + 37, 38, 39, 39, 39, 39, 41, 42, 42, 42, 42, 43, 34, 34, 34, 34, 34, 34, + 34, 33, 33, 33, 33, 34, 35, 35, 35, 35, 36, 36, 37, 37, 37, 38, 39, 39, + 39, 39, 41, 42, 42, 42, 42, 43, 34, 34, 34, 34, 34, 34, 34, 33, 33, 33, + 33, 34, 35, 35, 35, 35, 36, 36, 37, 37, 37, 38, 39, 39, 39, 39, 41, 42, + 42, 42, 42, 43, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 35, 36, + 36, 36, 36, 37, 37, 37, 37, 38, 40, 41, 41, 41, 42, 44, 45, 45, 45, 45, + 35, 35, 35, 35, 35, 35, 34, 34, 34, 34, 34, 35, 36, 36, 36, 36, 37, 37, + 38, 38, 38, 39, 41, 42, 42, 42, 44, 46, 47, 47, 47, 48, 36, 35, 35, 35, + 35, 35, 35, 34, 34, 34, 34, 35, 36, 36, 36, 36, 37, 38, 38, 38, 38, 40, + 42, 42, 42, 42, 45, 47, 48, 48, 48, 49, 36, 35, 35, 35, 35, 35, 35, 34, + 34, 34, 34, 35, 36, 36, 36, 36, 37, 38, 38, 38, 38, 40, 42, 42, 42, 42, + 45, 47, 48, 48, 48, 49, 36, 35, 35, 35, 35, 35, 35, 34, 34, 34, 34, 35, + 36, 36, 36, 36, 37, 38, 38, 38, 38, 40, 42, 42, 42, 42, 45, 47, 48, 48, + 48, 49, 37, 37, 36, 36, 36, 36, 36, 35, 35, 35, 35, 36, 37, 37, 37, 37, + 38, 39, 39, 39, 39, 41, 42, 43, 43, 43, 45, 48, 49, 49, 49, 50], + /* Size 4x8 */ + [31, 31, 32, 35, 32, 32, 32, 35, 32, 32, 33, 34, 32, 32, 34, 36, 32, 33, + 35, 38, 33, 33, 36, 40, 34, 34, 37, 42, 35, 34, 38, 48], + /* Size 8x4 */ + [31, 32, 32, 32, 32, 33, 34, 35, 31, 32, 32, 32, 33, 33, 34, 34, 32, 32, + 33, 34, 35, 36, 37, 38, 35, 35, 34, 36, 38, 40, 42, 48], + /* Size 8x16 */ + [32, 31, 31, 31, 32, 32, 35, 36, 31, 32, 32, 32, 32, 32, 35, 35, 31, 32, + 32, 32, 32, 32, 35, 35, 31, 32, 32, 32, 32, 32, 34, 35, 31, 32, 32, 32, + 33, 33, 34, 34, 31, 32, 32, 32, 33, 33, 34, 34, 31, 32, 32, 33, 34, 34, + 35, 36, 32, 32, 32, 33, 34, 34, 36, 36, 32, 32, 32, 33, 34, 34, 36, 37, + 32, 32, 33, 34, 35, 35, 37, 38, 32, 32, 33, 34, 35, 35, 37, 38, 33, 33, + 33, 35, 36, 36, 40, 41, 34, 34, 34, 35, 37, 37, 41, 42, 34, 34, 34, 35, + 37, 37, 43, 44, 36, 35, 34, 36, 38, 38, 46, 48, 36, 35, 34, 36, 38, 38, + 46, 48], + /* Size 16x8 */ + [32, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 33, 34, 34, 36, 36, 31, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 34, 34, 35, 35, 31, 32, 32, 32, + 32, 32, 32, 32, 32, 33, 33, 33, 34, 34, 34, 34, 31, 32, 32, 32, 32, 32, + 33, 33, 33, 34, 34, 35, 35, 35, 36, 36, 32, 32, 32, 32, 33, 33, 34, 34, + 34, 35, 35, 36, 37, 37, 38, 38, 32, 32, 32, 32, 33, 33, 34, 34, 34, 35, + 35, 36, 37, 37, 38, 38, 35, 35, 35, 34, 34, 34, 35, 36, 36, 37, 37, 40, + 41, 43, 46, 46, 36, 35, 35, 35, 34, 34, 36, 36, 37, 38, 38, 41, 42, 44, + 48, 48], + /* Size 16x32 */ + [32, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 33, 35, 36, 36, 36, 31, 31, + 31, 31, 31, 31, 32, 32, 32, 32, 32, 33, 35, 35, 35, 35, 31, 31, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 33, 35, 35, 35, 35, 31, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 33, 35, 35, 35, 35, 31, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 33, 35, 35, 35, 35, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 33, 35, 35, 35, 35, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, + 34, 35, 35, 35, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 34, 35, + 35, 35, 31, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 33, 34, 34, 34, 34, + 31, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 33, 34, 34, 34, 34, 31, 32, + 32, 32, 32, 32, 32, 32, 33, 33, 33, 33, 34, 34, 34, 34, 31, 32, 32, 32, + 32, 32, 33, 33, 33, 33, 33, 34, 35, 35, 35, 35, 31, 32, 32, 32, 32, 32, + 33, 33, 34, 34, 34, 34, 35, 36, 36, 36, 32, 32, 32, 32, 32, 32, 33, 34, + 34, 34, 34, 35, 36, 36, 36, 36, 32, 32, 32, 32, 32, 32, 33, 34, 34, 34, + 34, 35, 36, 36, 36, 36, 32, 32, 32, 32, 32, 32, 33, 34, 34, 34, 34, 35, + 36, 36, 36, 36, 32, 32, 32, 32, 32, 32, 33, 34, 34, 34, 34, 35, 36, 37, + 37, 37, 32, 32, 32, 33, 33, 33, 33, 34, 35, 35, 35, 36, 37, 38, 38, 38, + 32, 32, 32, 33, 33, 33, 34, 35, 35, 35, 35, 36, 37, 38, 38, 38, 32, 32, + 32, 33, 33, 33, 34, 35, 35, 35, 35, 36, 37, 38, 38, 38, 32, 32, 32, 33, + 33, 33, 34, 35, 35, 35, 35, 36, 37, 38, 38, 38, 32, 33, 33, 33, 33, 33, + 34, 35, 36, 36, 36, 37, 39, 40, 40, 40, 33, 33, 33, 33, 33, 33, 35, 36, + 36, 36, 36, 38, 40, 41, 41, 41, 34, 34, 34, 34, 34, 34, 35, 36, 37, 37, + 37, 39, 41, 42, 42, 42, 34, 34, 34, 34, 34, 34, 35, 36, 37, 37, 37, 39, + 41, 42, 42, 42, 34, 34, 34, 34, 34, 34, 35, 36, 37, 37, 37, 39, 41, 42, + 42, 42, 34, 34, 34, 34, 34, 34, 35, 37, 37, 37, 37, 40, 43, 44, 44, 44, + 35, 35, 34, 34, 34, 34, 36, 37, 38, 38, 38, 41, 45, 47, 47, 47, 36, 35, + 35, 34, 34, 34, 36, 37, 38, 38, 38, 42, 46, 48, 48, 48, 36, 35, 35, 34, + 34, 34, 36, 37, 38, 38, 38, 42, 46, 48, 48, 48, 36, 35, 35, 34, 34, 34, + 36, 37, 38, 38, 38, 42, 46, 48, 48, 48, 37, 36, 36, 36, 36, 36, 37, 38, + 39, 39, 39, 42, 46, 49, 49, 49], + /* Size 32x16 */ + [32, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 33, 34, 34, 34, 34, 35, 36, 36, 36, 37, 31, 31, 31, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, + 33, 34, 34, 34, 34, 35, 35, 35, 35, 36, 31, 31, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 34, 34, 34, + 34, 34, 35, 35, 35, 36, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 33, 33, 33, 33, 33, 33, 34, 34, 34, 34, 34, 34, 34, + 34, 36, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 33, 33, 33, 33, 33, 33, 34, 34, 34, 34, 34, 34, 34, 34, 36, 31, 31, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, + 33, 33, 33, 34, 34, 34, 34, 34, 34, 34, 34, 36, 31, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 33, 33, 33, 33, 33, 33, 33, 34, 34, 34, 34, 35, 35, + 35, 35, 35, 36, 36, 36, 36, 37, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 33, 33, 34, 34, 34, 34, 34, 35, 35, 35, 35, 36, 36, 36, 36, 37, 37, + 37, 37, 37, 38, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 33, 34, 34, + 34, 34, 34, 35, 35, 35, 35, 36, 36, 37, 37, 37, 37, 38, 38, 38, 38, 39, + 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 33, 34, 34, 34, 34, 34, 35, + 35, 35, 35, 36, 36, 37, 37, 37, 37, 38, 38, 38, 38, 39, 32, 32, 32, 32, + 32, 32, 32, 32, 33, 33, 33, 33, 34, 34, 34, 34, 34, 35, 35, 35, 35, 36, + 36, 37, 37, 37, 37, 38, 38, 38, 38, 39, 33, 33, 33, 33, 33, 33, 33, 33, + 33, 33, 33, 34, 34, 35, 35, 35, 35, 36, 36, 36, 36, 37, 38, 39, 39, 39, + 40, 41, 42, 42, 42, 42, 35, 35, 35, 35, 35, 35, 34, 34, 34, 34, 34, 35, + 35, 36, 36, 36, 36, 37, 37, 37, 37, 39, 40, 41, 41, 41, 43, 45, 46, 46, + 46, 46, 36, 35, 35, 35, 35, 35, 35, 35, 34, 34, 34, 35, 36, 36, 36, 36, + 37, 38, 38, 38, 38, 40, 41, 42, 42, 42, 44, 47, 48, 48, 48, 49, 36, 35, + 35, 35, 35, 35, 35, 35, 34, 34, 34, 35, 36, 36, 36, 36, 37, 38, 38, 38, + 38, 40, 41, 42, 42, 42, 44, 47, 48, 48, 48, 49, 36, 35, 35, 35, 35, 35, + 35, 35, 34, 34, 34, 35, 36, 36, 36, 36, 37, 38, 38, 38, 38, 40, 41, 42, + 42, 42, 44, 47, 48, 48, 48, 49], + /* Size 4x16 */ + [31, 31, 32, 36, 31, 32, 32, 35, 32, 32, 32, 35, 32, 32, 32, 35, 32, 32, + 33, 34, 32, 32, 33, 34, 32, 32, 34, 36, 32, 32, 34, 36, 32, 32, 34, 37, + 32, 33, 35, 38, 32, 33, 35, 38, 33, 33, 36, 41, 34, 34, 37, 42, 34, 34, + 37, 44, 35, 34, 38, 48, 35, 34, 38, 48], + /* Size 16x4 */ + [32, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 34, 34, 35, 35, 31, 32, + 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 34, 34, 34, 34, 32, 32, 32, 32, + 33, 33, 34, 34, 34, 35, 35, 36, 37, 37, 38, 38, 36, 35, 35, 35, 34, 34, + 36, 36, 37, 38, 38, 41, 42, 44, 48, 48], + /* Size 8x32 */ + [32, 31, 31, 31, 32, 32, 35, 36, 31, 31, 31, 32, 32, 32, 35, 35, 31, 32, + 32, 32, 32, 32, 35, 35, 31, 32, 32, 32, 32, 32, 35, 35, 31, 32, 32, 32, + 32, 32, 35, 35, 31, 32, 32, 32, 32, 32, 35, 35, 31, 32, 32, 32, 32, 32, + 34, 35, 31, 32, 32, 32, 32, 32, 34, 35, 31, 32, 32, 32, 33, 33, 34, 34, + 31, 32, 32, 32, 33, 33, 34, 34, 31, 32, 32, 32, 33, 33, 34, 34, 31, 32, + 32, 33, 33, 33, 35, 35, 31, 32, 32, 33, 34, 34, 35, 36, 32, 32, 32, 33, + 34, 34, 36, 36, 32, 32, 32, 33, 34, 34, 36, 36, 32, 32, 32, 33, 34, 34, + 36, 36, 32, 32, 32, 33, 34, 34, 36, 37, 32, 32, 33, 33, 35, 35, 37, 38, + 32, 32, 33, 34, 35, 35, 37, 38, 32, 32, 33, 34, 35, 35, 37, 38, 32, 32, + 33, 34, 35, 35, 37, 38, 32, 33, 33, 34, 36, 36, 39, 40, 33, 33, 33, 35, + 36, 36, 40, 41, 34, 34, 34, 35, 37, 37, 41, 42, 34, 34, 34, 35, 37, 37, + 41, 42, 34, 34, 34, 35, 37, 37, 41, 42, 34, 34, 34, 35, 37, 37, 43, 44, + 35, 34, 34, 36, 38, 38, 45, 47, 36, 35, 34, 36, 38, 38, 46, 48, 36, 35, + 34, 36, 38, 38, 46, 48, 36, 35, 34, 36, 38, 38, 46, 48, 37, 36, 36, 37, + 39, 39, 46, 49], + /* Size 32x8 */ + [32, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 33, 34, 34, 34, 34, 35, 36, 36, 36, 37, 31, 31, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, + 33, 34, 34, 34, 34, 34, 35, 35, 35, 36, 31, 31, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 33, 33, 33, 34, 34, 34, + 34, 34, 34, 34, 34, 36, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, + 33, 33, 33, 33, 33, 33, 34, 34, 34, 34, 35, 35, 35, 35, 35, 36, 36, 36, + 36, 37, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 33, 34, 34, 34, 34, + 34, 35, 35, 35, 35, 36, 36, 37, 37, 37, 37, 38, 38, 38, 38, 39, 32, 32, + 32, 32, 32, 32, 32, 32, 33, 33, 33, 33, 34, 34, 34, 34, 34, 35, 35, 35, + 35, 36, 36, 37, 37, 37, 37, 38, 38, 38, 38, 39, 35, 35, 35, 35, 35, 35, + 34, 34, 34, 34, 34, 35, 35, 36, 36, 36, 36, 37, 37, 37, 37, 39, 40, 41, + 41, 41, 43, 45, 46, 46, 46, 46, 36, 35, 35, 35, 35, 35, 35, 35, 34, 34, + 34, 35, 36, 36, 36, 36, 37, 38, 38, 38, 38, 40, 41, 42, 42, 42, 44, 47, + 48, 48, 48, 49]], + [ /* Chroma */ + /* Size 4x4 */ + [31, 32, 38, 46, 32, 34, 41, 46, 38, 41, 47, 47, 46, 46, 47, 52], + /* Size 8x8 */ + [31, 31, 30, 34, 36, 39, 42, 48, 31, 31, 31, 34, 37, 40, 42, 47, 30, 31, + 32, 35, 39, 41, 42, 46, 34, 34, 35, 39, 42, 44, 45, 47, 36, 37, 39, 42, + 46, 47, 47, 47, 39, 40, 41, 44, 47, 47, 48, 49, 42, 42, 42, 45, 47, 48, + 48, 50, 48, 47, 46, 47, 47, 49, 50, 53], + /* Size 16x16 */ + [32, 31, 31, 31, 30, 30, 33, 33, 34, 36, 36, 40, 41, 44, 49, 49, 31, 31, + 31, 31, 31, 31, 33, 34, 36, 38, 38, 41, 42, 44, 48, 48, 31, 31, 31, 31, + 31, 31, 34, 34, 36, 38, 38, 41, 42, 44, 47, 47, 31, 31, 31, 31, 31, 31, + 34, 35, 36, 39, 39, 41, 42, 44, 47, 47, 30, 31, 31, 31, 32, 32, 34, 35, + 37, 40, 40, 42, 42, 44, 46, 46, 30, 31, 31, 31, 32, 32, 34, 35, 37, 40, + 40, 42, 42, 44, 46, 46, 33, 33, 34, 34, 34, 34, 37, 38, 40, 42, 42, 44, + 44, 45, 47, 47, 33, 34, 34, 35, 35, 35, 38, 39, 40, 43, 43, 44, 45, 46, + 47, 47, 34, 36, 36, 36, 37, 37, 40, 40, 42, 45, 45, 45, 46, 46, 47, 47, + 36, 38, 38, 39, 40, 40, 42, 43, 45, 47, 47, 47, 47, 47, 48, 48, 36, 38, + 38, 39, 40, 40, 42, 43, 45, 47, 47, 47, 47, 47, 48, 48, 40, 41, 41, 41, + 42, 42, 44, 44, 45, 47, 47, 48, 48, 49, 50, 50, 41, 42, 42, 42, 42, 42, + 44, 45, 46, 47, 47, 48, 48, 49, 50, 50, 44, 44, 44, 44, 44, 44, 45, 46, + 46, 47, 47, 49, 49, 50, 51, 51, 49, 48, 47, 47, 46, 46, 47, 47, 47, 48, + 48, 50, 50, 51, 53, 53, 49, 48, 47, 47, 46, 46, 47, 47, 47, 48, 48, 50, + 50, 51, 53, 53], + /* Size 32x32 */ + [32, 31, 31, 31, 31, 31, 31, 30, 30, 30, 30, 31, 33, 33, 33, 33, 34, 36, + 36, 36, 36, 38, 40, 41, 41, 41, 44, 47, 49, 49, 49, 49, 31, 31, 31, 31, + 31, 31, 31, 31, 30, 30, 30, 32, 33, 34, 34, 34, 35, 36, 37, 37, 37, 39, + 41, 42, 42, 42, 44, 47, 48, 48, 48, 48, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 32, 33, 34, 34, 34, 36, 37, 38, 38, 38, 39, 41, 42, 42, 42, + 44, 46, 48, 48, 48, 47, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, + 34, 34, 34, 34, 36, 37, 38, 38, 38, 40, 41, 42, 42, 42, 44, 46, 47, 47, + 47, 47, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 34, 34, 34, 34, + 36, 37, 38, 38, 38, 40, 41, 42, 42, 42, 44, 46, 47, 47, 47, 47, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 34, 34, 34, 34, 36, 37, 38, 38, + 38, 40, 41, 42, 42, 42, 44, 46, 47, 47, 47, 47, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 33, 34, 35, 35, 35, 36, 38, 39, 39, 39, 40, 41, 42, + 42, 42, 44, 46, 47, 47, 47, 47, 30, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 33, 34, 35, 35, 35, 37, 38, 39, 39, 39, 41, 42, 42, 42, 42, 44, 46, + 46, 46, 46, 46, 30, 30, 31, 31, 31, 31, 31, 31, 32, 32, 32, 33, 34, 35, + 35, 35, 37, 39, 40, 40, 40, 41, 42, 42, 42, 42, 44, 45, 46, 46, 46, 46, + 30, 30, 31, 31, 31, 31, 31, 31, 32, 32, 32, 33, 34, 35, 35, 35, 37, 39, + 40, 40, 40, 41, 42, 42, 42, 42, 44, 45, 46, 46, 46, 46, 30, 30, 31, 31, + 31, 31, 31, 31, 32, 32, 32, 33, 34, 35, 35, 35, 37, 39, 40, 40, 40, 41, + 42, 42, 42, 42, 44, 45, 46, 46, 46, 46, 31, 32, 32, 32, 32, 32, 33, 33, + 33, 33, 33, 34, 36, 37, 37, 37, 38, 40, 41, 41, 41, 42, 43, 43, 43, 43, + 44, 46, 46, 46, 46, 46, 33, 33, 33, 34, 34, 34, 34, 34, 34, 34, 34, 36, + 37, 38, 38, 38, 40, 41, 42, 42, 42, 43, 44, 44, 44, 44, 45, 46, 47, 47, + 47, 46, 33, 34, 34, 34, 34, 34, 35, 35, 35, 35, 35, 37, 38, 39, 39, 39, + 40, 42, 43, 43, 43, 44, 44, 45, 45, 45, 46, 47, 47, 47, 47, 47, 33, 34, + 34, 34, 34, 34, 35, 35, 35, 35, 35, 37, 38, 39, 39, 39, 40, 42, 43, 43, + 43, 44, 44, 45, 45, 45, 46, 47, 47, 47, 47, 47, 33, 34, 34, 34, 34, 34, + 35, 35, 35, 35, 35, 37, 38, 39, 39, 39, 40, 42, 43, 43, 43, 44, 44, 45, + 45, 45, 46, 47, 47, 47, 47, 47, 34, 35, 36, 36, 36, 36, 36, 37, 37, 37, + 37, 38, 40, 40, 40, 40, 42, 44, 45, 45, 45, 45, 45, 46, 46, 46, 46, 47, + 47, 47, 47, 47, 36, 36, 37, 37, 37, 37, 38, 38, 39, 39, 39, 40, 41, 42, + 42, 42, 44, 46, 46, 46, 46, 47, 47, 47, 47, 47, 47, 47, 47, 47, 47, 47, + 36, 37, 38, 38, 38, 38, 39, 39, 40, 40, 40, 41, 42, 43, 43, 43, 45, 46, + 47, 47, 47, 47, 47, 47, 47, 47, 47, 47, 48, 48, 48, 47, 36, 37, 38, 38, + 38, 38, 39, 39, 40, 40, 40, 41, 42, 43, 43, 43, 45, 46, 47, 47, 47, 47, + 47, 47, 47, 47, 47, 47, 48, 48, 48, 47, 36, 37, 38, 38, 38, 38, 39, 39, + 40, 40, 40, 41, 42, 43, 43, 43, 45, 46, 47, 47, 47, 47, 47, 47, 47, 47, + 47, 47, 48, 48, 48, 47, 38, 39, 39, 40, 40, 40, 40, 41, 41, 41, 41, 42, + 43, 44, 44, 44, 45, 47, 47, 47, 47, 47, 48, 48, 48, 48, 48, 48, 49, 49, + 49, 48, 40, 41, 41, 41, 41, 41, 41, 42, 42, 42, 42, 43, 44, 44, 44, 44, + 45, 47, 47, 47, 47, 48, 48, 48, 48, 48, 49, 49, 50, 50, 50, 49, 41, 42, + 42, 42, 42, 42, 42, 42, 42, 42, 42, 43, 44, 45, 45, 45, 46, 47, 47, 47, + 47, 48, 48, 48, 48, 48, 49, 50, 50, 50, 50, 50, 41, 42, 42, 42, 42, 42, + 42, 42, 42, 42, 42, 43, 44, 45, 45, 45, 46, 47, 47, 47, 47, 48, 48, 48, + 48, 48, 49, 50, 50, 50, 50, 50, 41, 42, 42, 42, 42, 42, 42, 42, 42, 42, + 42, 43, 44, 45, 45, 45, 46, 47, 47, 47, 47, 48, 48, 48, 48, 48, 49, 50, + 50, 50, 50, 50, 44, 44, 44, 44, 44, 44, 44, 44, 44, 44, 44, 44, 45, 46, + 46, 46, 46, 47, 47, 47, 47, 48, 49, 49, 49, 49, 50, 51, 51, 51, 51, 51, + 47, 47, 46, 46, 46, 46, 46, 46, 45, 45, 45, 46, 46, 47, 47, 47, 47, 47, + 47, 47, 47, 48, 49, 50, 50, 50, 51, 52, 52, 52, 52, 52, 49, 48, 48, 47, + 47, 47, 47, 46, 46, 46, 46, 46, 47, 47, 47, 47, 47, 47, 48, 48, 48, 49, + 50, 50, 50, 50, 51, 52, 53, 53, 53, 53, 49, 48, 48, 47, 47, 47, 47, 46, + 46, 46, 46, 46, 47, 47, 47, 47, 47, 47, 48, 48, 48, 49, 50, 50, 50, 50, + 51, 52, 53, 53, 53, 53, 49, 48, 48, 47, 47, 47, 47, 46, 46, 46, 46, 46, + 47, 47, 47, 47, 47, 47, 48, 48, 48, 49, 50, 50, 50, 50, 51, 52, 53, 53, + 53, 53, 49, 48, 47, 47, 47, 47, 47, 46, 46, 46, 46, 46, 46, 47, 47, 47, + 47, 47, 47, 47, 47, 48, 49, 50, 50, 50, 51, 52, 53, 53, 53, 53], + /* Size 4x8 */ + [31, 31, 37, 48, 31, 31, 38, 47, 31, 32, 40, 46, 34, 36, 43, 47, 37, 39, + 46, 47, 39, 41, 47, 48, 42, 43, 47, 50, 48, 46, 48, 53], + /* Size 8x4 */ + [31, 31, 31, 34, 37, 39, 42, 48, 31, 31, 32, 36, 39, 41, 43, 46, 37, 38, + 40, 43, 46, 47, 47, 48, 48, 47, 46, 47, 47, 48, 50, 53], + /* Size 8x16 */ + [32, 31, 31, 33, 37, 37, 45, 48, 31, 31, 31, 34, 38, 38, 45, 47, 31, 31, + 31, 34, 38, 38, 45, 47, 31, 31, 32, 34, 39, 39, 45, 46, 30, 32, 32, 35, + 40, 40, 44, 46, 30, 32, 32, 35, 40, 40, 44, 46, 33, 34, 35, 37, 42, 42, + 46, 47, 33, 35, 36, 38, 43, 43, 46, 47, 35, 37, 37, 40, 44, 44, 46, 47, + 37, 39, 40, 43, 47, 47, 47, 47, 37, 39, 40, 43, 47, 47, 47, 47, 41, 42, + 42, 44, 47, 47, 49, 49, 42, 42, 43, 44, 47, 47, 49, 50, 44, 44, 44, 45, + 47, 47, 50, 51, 49, 47, 46, 47, 48, 48, 52, 53, 49, 47, 46, 47, 48, 48, + 52, 53], + /* Size 16x8 */ + [32, 31, 31, 31, 30, 30, 33, 33, 35, 37, 37, 41, 42, 44, 49, 49, 31, 31, + 31, 31, 32, 32, 34, 35, 37, 39, 39, 42, 42, 44, 47, 47, 31, 31, 31, 32, + 32, 32, 35, 36, 37, 40, 40, 42, 43, 44, 46, 46, 33, 34, 34, 34, 35, 35, + 37, 38, 40, 43, 43, 44, 44, 45, 47, 47, 37, 38, 38, 39, 40, 40, 42, 43, + 44, 47, 47, 47, 47, 47, 48, 48, 37, 38, 38, 39, 40, 40, 42, 43, 44, 47, + 47, 47, 47, 47, 48, 48, 45, 45, 45, 45, 44, 44, 46, 46, 46, 47, 47, 49, + 49, 50, 52, 52, 48, 47, 47, 46, 46, 46, 47, 47, 47, 47, 47, 49, 50, 51, + 53, 53], + /* Size 16x32 */ + [32, 31, 31, 31, 31, 31, 33, 35, 37, 37, 37, 40, 45, 48, 48, 48, 31, 31, + 31, 31, 31, 31, 33, 36, 37, 37, 37, 41, 45, 48, 48, 48, 31, 31, 31, 31, + 31, 31, 34, 36, 38, 38, 38, 41, 45, 47, 47, 47, 31, 31, 31, 31, 31, 31, + 34, 37, 38, 38, 38, 41, 45, 47, 47, 47, 31, 31, 31, 31, 31, 31, 34, 37, + 38, 38, 38, 41, 45, 47, 47, 47, 31, 31, 31, 31, 31, 31, 34, 37, 38, 38, + 38, 41, 45, 47, 47, 47, 31, 31, 31, 32, 32, 32, 34, 37, 39, 39, 39, 41, + 45, 46, 46, 46, 30, 31, 31, 32, 32, 32, 34, 38, 39, 39, 39, 42, 44, 46, + 46, 46, 30, 31, 32, 32, 32, 32, 35, 38, 40, 40, 40, 42, 44, 46, 46, 46, + 30, 31, 32, 32, 32, 32, 35, 38, 40, 40, 40, 42, 44, 46, 46, 46, 30, 31, + 32, 32, 32, 32, 35, 38, 40, 40, 40, 42, 44, 46, 46, 46, 31, 32, 33, 33, + 33, 33, 36, 39, 41, 41, 41, 43, 45, 46, 46, 46, 33, 34, 34, 35, 35, 35, + 37, 40, 42, 42, 42, 44, 46, 47, 47, 47, 33, 34, 35, 36, 36, 36, 38, 41, + 43, 43, 43, 44, 46, 47, 47, 47, 33, 34, 35, 36, 36, 36, 38, 41, 43, 43, + 43, 44, 46, 47, 47, 47, 33, 34, 35, 36, 36, 36, 38, 41, 43, 43, 43, 44, + 46, 47, 47, 47, 35, 36, 37, 37, 37, 37, 40, 43, 44, 44, 44, 45, 46, 47, + 47, 47, 36, 37, 38, 39, 39, 39, 42, 44, 46, 46, 46, 47, 47, 47, 47, 47, + 37, 38, 39, 40, 40, 40, 43, 45, 47, 47, 47, 47, 47, 47, 47, 47, 37, 38, + 39, 40, 40, 40, 43, 45, 47, 47, 47, 47, 47, 47, 47, 47, 37, 38, 39, 40, + 40, 40, 43, 45, 47, 47, 47, 47, 47, 47, 47, 47, 39, 39, 40, 41, 41, 41, + 43, 46, 47, 47, 47, 48, 48, 48, 48, 48, 41, 41, 42, 42, 42, 42, 44, 46, + 47, 47, 47, 48, 49, 49, 49, 49, 42, 42, 42, 43, 43, 43, 44, 46, 47, 47, + 47, 48, 49, 50, 50, 50, 42, 42, 42, 43, 43, 43, 44, 46, 47, 47, 47, 48, + 49, 50, 50, 50, 42, 42, 42, 43, 43, 43, 44, 46, 47, 47, 47, 48, 49, 50, + 50, 50, 44, 44, 44, 44, 44, 44, 45, 47, 47, 47, 47, 49, 50, 51, 51, 51, + 47, 46, 46, 46, 46, 46, 46, 47, 48, 48, 48, 49, 51, 52, 52, 52, 49, 48, + 47, 46, 46, 46, 47, 48, 48, 48, 48, 50, 52, 53, 53, 53, 49, 48, 47, 46, + 46, 46, 47, 48, 48, 48, 48, 50, 52, 53, 53, 53, 49, 48, 47, 46, 46, 46, + 47, 48, 48, 48, 48, 50, 52, 53, 53, 53, 49, 48, 47, 46, 46, 46, 47, 47, + 47, 47, 47, 49, 52, 53, 53, 53], + /* Size 32x16 */ + [32, 31, 31, 31, 31, 31, 31, 30, 30, 30, 30, 31, 33, 33, 33, 33, 35, 36, + 37, 37, 37, 39, 41, 42, 42, 42, 44, 47, 49, 49, 49, 49, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 32, 34, 34, 34, 34, 36, 37, 38, 38, 38, 39, + 41, 42, 42, 42, 44, 46, 48, 48, 48, 48, 31, 31, 31, 31, 31, 31, 31, 31, + 32, 32, 32, 33, 34, 35, 35, 35, 37, 38, 39, 39, 39, 40, 42, 42, 42, 42, + 44, 46, 47, 47, 47, 47, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 33, + 35, 36, 36, 36, 37, 39, 40, 40, 40, 41, 42, 43, 43, 43, 44, 46, 46, 46, + 46, 46, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 33, 35, 36, 36, 36, + 37, 39, 40, 40, 40, 41, 42, 43, 43, 43, 44, 46, 46, 46, 46, 46, 31, 31, + 31, 31, 31, 31, 32, 32, 32, 32, 32, 33, 35, 36, 36, 36, 37, 39, 40, 40, + 40, 41, 42, 43, 43, 43, 44, 46, 46, 46, 46, 46, 33, 33, 34, 34, 34, 34, + 34, 34, 35, 35, 35, 36, 37, 38, 38, 38, 40, 42, 43, 43, 43, 43, 44, 44, + 44, 44, 45, 46, 47, 47, 47, 47, 35, 36, 36, 37, 37, 37, 37, 38, 38, 38, + 38, 39, 40, 41, 41, 41, 43, 44, 45, 45, 45, 46, 46, 46, 46, 46, 47, 47, + 48, 48, 48, 47, 37, 37, 38, 38, 38, 38, 39, 39, 40, 40, 40, 41, 42, 43, + 43, 43, 44, 46, 47, 47, 47, 47, 47, 47, 47, 47, 47, 48, 48, 48, 48, 47, + 37, 37, 38, 38, 38, 38, 39, 39, 40, 40, 40, 41, 42, 43, 43, 43, 44, 46, + 47, 47, 47, 47, 47, 47, 47, 47, 47, 48, 48, 48, 48, 47, 37, 37, 38, 38, + 38, 38, 39, 39, 40, 40, 40, 41, 42, 43, 43, 43, 44, 46, 47, 47, 47, 47, + 47, 47, 47, 47, 47, 48, 48, 48, 48, 47, 40, 41, 41, 41, 41, 41, 41, 42, + 42, 42, 42, 43, 44, 44, 44, 44, 45, 47, 47, 47, 47, 48, 48, 48, 48, 48, + 49, 49, 50, 50, 50, 49, 45, 45, 45, 45, 45, 45, 45, 44, 44, 44, 44, 45, + 46, 46, 46, 46, 46, 47, 47, 47, 47, 48, 49, 49, 49, 49, 50, 51, 52, 52, + 52, 52, 48, 48, 47, 47, 47, 47, 46, 46, 46, 46, 46, 46, 47, 47, 47, 47, + 47, 47, 47, 47, 47, 48, 49, 50, 50, 50, 51, 52, 53, 53, 53, 53, 48, 48, + 47, 47, 47, 47, 46, 46, 46, 46, 46, 46, 47, 47, 47, 47, 47, 47, 47, 47, + 47, 48, 49, 50, 50, 50, 51, 52, 53, 53, 53, 53, 48, 48, 47, 47, 47, 47, + 46, 46, 46, 46, 46, 46, 47, 47, 47, 47, 47, 47, 47, 47, 47, 48, 49, 50, + 50, 50, 51, 52, 53, 53, 53, 53], + /* Size 4x16 */ + [31, 31, 37, 48, 31, 31, 38, 47, 31, 31, 38, 47, 31, 32, 39, 46, 31, 32, + 40, 46, 31, 32, 40, 46, 34, 35, 42, 47, 34, 36, 43, 47, 36, 37, 44, 47, + 38, 40, 47, 47, 38, 40, 47, 47, 41, 42, 47, 49, 42, 43, 47, 50, 44, 44, + 47, 51, 48, 46, 48, 53, 48, 46, 48, 53], + /* Size 16x4 */ + [31, 31, 31, 31, 31, 31, 34, 34, 36, 38, 38, 41, 42, 44, 48, 48, 31, 31, + 31, 32, 32, 32, 35, 36, 37, 40, 40, 42, 43, 44, 46, 46, 37, 38, 38, 39, + 40, 40, 42, 43, 44, 47, 47, 47, 47, 47, 48, 48, 48, 47, 47, 46, 46, 46, + 47, 47, 47, 47, 47, 49, 50, 51, 53, 53], + /* Size 8x32 */ + [32, 31, 31, 33, 37, 37, 45, 48, 31, 31, 31, 33, 37, 37, 45, 48, 31, 31, + 31, 34, 38, 38, 45, 47, 31, 31, 31, 34, 38, 38, 45, 47, 31, 31, 31, 34, + 38, 38, 45, 47, 31, 31, 31, 34, 38, 38, 45, 47, 31, 31, 32, 34, 39, 39, + 45, 46, 30, 31, 32, 34, 39, 39, 44, 46, 30, 32, 32, 35, 40, 40, 44, 46, + 30, 32, 32, 35, 40, 40, 44, 46, 30, 32, 32, 35, 40, 40, 44, 46, 31, 33, + 33, 36, 41, 41, 45, 46, 33, 34, 35, 37, 42, 42, 46, 47, 33, 35, 36, 38, + 43, 43, 46, 47, 33, 35, 36, 38, 43, 43, 46, 47, 33, 35, 36, 38, 43, 43, + 46, 47, 35, 37, 37, 40, 44, 44, 46, 47, 36, 38, 39, 42, 46, 46, 47, 47, + 37, 39, 40, 43, 47, 47, 47, 47, 37, 39, 40, 43, 47, 47, 47, 47, 37, 39, + 40, 43, 47, 47, 47, 47, 39, 40, 41, 43, 47, 47, 48, 48, 41, 42, 42, 44, + 47, 47, 49, 49, 42, 42, 43, 44, 47, 47, 49, 50, 42, 42, 43, 44, 47, 47, + 49, 50, 42, 42, 43, 44, 47, 47, 49, 50, 44, 44, 44, 45, 47, 47, 50, 51, + 47, 46, 46, 46, 48, 48, 51, 52, 49, 47, 46, 47, 48, 48, 52, 53, 49, 47, + 46, 47, 48, 48, 52, 53, 49, 47, 46, 47, 48, 48, 52, 53, 49, 47, 46, 47, + 47, 47, 52, 53], + /* Size 32x8 */ + [32, 31, 31, 31, 31, 31, 31, 30, 30, 30, 30, 31, 33, 33, 33, 33, 35, 36, + 37, 37, 37, 39, 41, 42, 42, 42, 44, 47, 49, 49, 49, 49, 31, 31, 31, 31, + 31, 31, 31, 31, 32, 32, 32, 33, 34, 35, 35, 35, 37, 38, 39, 39, 39, 40, + 42, 42, 42, 42, 44, 46, 47, 47, 47, 47, 31, 31, 31, 31, 31, 31, 32, 32, + 32, 32, 32, 33, 35, 36, 36, 36, 37, 39, 40, 40, 40, 41, 42, 43, 43, 43, + 44, 46, 46, 46, 46, 46, 33, 33, 34, 34, 34, 34, 34, 34, 35, 35, 35, 36, + 37, 38, 38, 38, 40, 42, 43, 43, 43, 43, 44, 44, 44, 44, 45, 46, 47, 47, + 47, 47, 37, 37, 38, 38, 38, 38, 39, 39, 40, 40, 40, 41, 42, 43, 43, 43, + 44, 46, 47, 47, 47, 47, 47, 47, 47, 47, 47, 48, 48, 48, 48, 47, 37, 37, + 38, 38, 38, 38, 39, 39, 40, 40, 40, 41, 42, 43, 43, 43, 44, 46, 47, 47, + 47, 47, 47, 47, 47, 47, 47, 48, 48, 48, 48, 47, 45, 45, 45, 45, 45, 45, + 45, 44, 44, 44, 44, 45, 46, 46, 46, 46, 46, 47, 47, 47, 47, 48, 49, 49, + 49, 49, 50, 51, 52, 52, 52, 52, 48, 48, 47, 47, 47, 47, 46, 46, 46, 46, + 46, 46, 47, 47, 47, 47, 47, 47, 47, 47, 47, 48, 49, 50, 50, 50, 51, 52, + 53, 53, 53, 53]] + ], + [ + [ /* Luma */ + /* Size 4x4 */ + [31, 32, 32, 32, 32, 32, 32, 33, 32, 32, 33, 34, 32, 33, 34, 35], + /* Size 8x8 */ + [31, 31, 31, 31, 32, 32, 32, 33, 31, 32, 32, 32, 32, 32, 32, 33, 31, 32, + 32, 32, 32, 32, 32, 33, 31, 32, 32, 32, 32, 32, 32, 33, 32, 32, 32, 32, + 33, 33, 34, 35, 32, 32, 32, 32, 33, 34, 34, 35, 32, 32, 32, 32, 34, 34, + 35, 36, 33, 33, 33, 33, 35, 35, 36, 38], + /* Size 16x16 */ + [32, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 34, 31, 31, + 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 33, 34, 31, 31, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 34, 31, 31, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 33, 34, 31, 31, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 33, 34, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 33, 33, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 33, 33, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, + 33, 34, 31, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 33, 33, 33, 34, 35, + 31, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 33, 34, 34, 34, 35, 31, 32, + 32, 32, 32, 32, 32, 32, 33, 33, 33, 33, 34, 34, 34, 35, 32, 32, 32, 32, + 32, 32, 32, 33, 33, 33, 33, 34, 35, 35, 35, 36, 32, 32, 32, 32, 32, 32, + 32, 33, 33, 34, 34, 35, 35, 35, 36, 37, 32, 32, 32, 32, 32, 32, 32, 33, + 33, 34, 34, 35, 35, 35, 36, 37, 32, 33, 33, 33, 33, 33, 33, 33, 34, 34, + 34, 35, 36, 36, 36, 38, 34, 34, 34, 34, 34, 33, 33, 34, 35, 35, 35, 36, + 37, 37, 38, 39], + /* Size 32x32 */ + [32, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 33, 34, 34, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 33, 34, 34, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 33, 33, 34, 34, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, + 34, 34, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 34, 34, 31, 31, + 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 34, 34, 31, 31, 31, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 33, 33, 34, 34, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 33, 33, 34, 34, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 34, 34, + 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 33, 31, 31, 31, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 33, 33, 33, 33, 33, 31, 31, 31, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 33, 33, 33, 33, 33, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, + 33, 33, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 33, 33, 31, 31, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 33, 33, 33, 33, 33, 33, 33, 33, 33, 34, 34, 31, 31, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 33, 33, 33, 33, 33, + 33, 33, 33, 33, 34, 34, 34, 34, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 34, + 34, 34, 35, 35, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 33, 33, 33, 33, 33, 33, 33, 33, 34, 34, 34, 34, 34, 34, 35, 35, 35, + 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, + 33, 33, 33, 33, 33, 34, 34, 34, 34, 34, 34, 35, 35, 35, 31, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 33, 33, 33, 33, + 33, 34, 34, 34, 34, 34, 34, 35, 35, 35, 31, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 33, 33, 33, 33, 33, 34, 34, 34, + 34, 34, 34, 35, 35, 35, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 33, 33, 33, 33, 33, 33, 33, 34, 34, 34, 34, 34, 34, 34, 35, 35, + 35, 35, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, + 33, 33, 33, 33, 33, 34, 34, 34, 35, 35, 35, 35, 35, 35, 36, 36, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 34, 34, 34, + 34, 34, 34, 35, 35, 35, 35, 35, 35, 36, 36, 36, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 34, 34, 34, 34, 34, 35, 35, + 35, 35, 35, 35, 36, 36, 37, 37, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 33, 33, 33, 34, 34, 34, 34, 34, 35, 35, 35, 35, 35, 35, + 36, 36, 37, 37, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 33, 33, 33, 34, 34, 34, 34, 34, 35, 35, 35, 35, 35, 35, 36, 36, 37, 37, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 33, 33, 33, 34, 34, + 34, 34, 34, 34, 35, 35, 35, 35, 35, 35, 36, 36, 37, 37, 32, 32, 33, 33, + 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 34, 34, 34, 34, 34, 34, 35, + 35, 35, 36, 36, 36, 36, 36, 37, 38, 38, 33, 33, 33, 33, 33, 33, 33, 33, + 33, 33, 33, 33, 33, 33, 33, 34, 34, 35, 35, 35, 35, 35, 35, 36, 36, 36, + 36, 36, 37, 38, 38, 38, 34, 34, 34, 34, 34, 34, 34, 34, 34, 33, 33, 33, + 33, 33, 34, 34, 35, 35, 35, 35, 35, 35, 36, 36, 37, 37, 37, 37, 38, 38, + 39, 39, 34, 34, 34, 34, 34, 34, 34, 34, 34, 33, 33, 33, 33, 33, 34, 34, + 35, 35, 35, 35, 35, 35, 36, 36, 37, 37, 37, 37, 38, 38, 39, 39], + /* Size 4x8 */ + [31, 31, 32, 32, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 32, 32, + 33, 34, 32, 32, 34, 34, 32, 33, 34, 35, 33, 33, 35, 36], + /* Size 8x4 */ + [31, 31, 32, 32, 32, 32, 32, 33, 31, 32, 32, 32, 32, 32, 33, 33, 32, 32, + 32, 32, 33, 34, 34, 35, 32, 32, 32, 33, 34, 34, 35, 36], + /* Size 8x16 */ + [32, 31, 31, 31, 31, 32, 32, 32, 31, 31, 32, 32, 32, 32, 32, 33, 31, 32, + 32, 32, 32, 32, 32, 33, 31, 32, 32, 32, 32, 32, 32, 33, 31, 32, 32, 32, + 32, 32, 32, 33, 31, 32, 32, 32, 32, 33, 33, 33, 31, 32, 32, 32, 32, 33, + 33, 33, 31, 32, 32, 32, 32, 33, 33, 33, 31, 32, 32, 32, 33, 34, 34, 34, + 32, 32, 32, 32, 33, 34, 34, 34, 32, 32, 32, 32, 33, 34, 34, 34, 32, 32, + 32, 32, 33, 35, 35, 35, 32, 32, 33, 33, 34, 35, 35, 36, 32, 32, 33, 33, + 34, 35, 35, 36, 32, 33, 33, 33, 34, 36, 36, 36, 34, 34, 34, 34, 35, 37, + 37, 38], + /* Size 16x8 */ + [32, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 34, 31, 31, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 34, 31, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 34, 31, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 33, 33, 33, 34, 31, 32, 32, 32, 32, 32, 32, 32, + 33, 33, 33, 33, 34, 34, 34, 35, 32, 32, 32, 32, 32, 33, 33, 33, 34, 34, + 34, 35, 35, 35, 36, 37, 32, 32, 32, 32, 32, 33, 33, 33, 34, 34, 34, 35, + 35, 35, 36, 37, 32, 33, 33, 33, 33, 33, 33, 33, 34, 34, 34, 35, 36, 36, + 36, 38], + /* Size 16x32 */ + [32, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 34, 31, 31, + 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 33, 34, 31, 31, 31, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 34, 31, 31, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 33, 34, 31, 31, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 33, 34, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 33, 34, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 33, 34, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 33, 34, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 34, + 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 31, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 33, 33, 33, 31, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 33, 33, 33, 33, 33, 33, 31, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 33, 33, 33, 33, 33, 33, 31, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 33, 33, 33, 33, 33, 33, 31, 32, 32, 32, 32, 32, 32, 32, 32, 33, + 33, 33, 33, 33, 33, 34, 31, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 33, + 33, 33, 34, 34, 31, 32, 32, 32, 32, 32, 32, 32, 33, 33, 34, 34, 34, 34, + 34, 35, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 34, 34, 34, 34, 34, 35, + 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 34, 34, 34, 34, 34, 35, 32, 32, + 32, 32, 32, 32, 32, 33, 33, 33, 34, 34, 34, 34, 34, 35, 32, 32, 32, 32, + 32, 32, 32, 33, 33, 33, 34, 34, 34, 34, 34, 35, 32, 32, 32, 32, 32, 32, + 32, 33, 33, 34, 34, 34, 34, 34, 35, 35, 32, 32, 32, 32, 32, 32, 32, 33, + 33, 34, 35, 35, 35, 35, 35, 36, 32, 32, 32, 32, 33, 33, 33, 33, 33, 34, + 35, 35, 35, 35, 36, 36, 32, 32, 32, 32, 33, 33, 33, 33, 34, 34, 35, 35, + 35, 35, 36, 37, 32, 32, 32, 32, 33, 33, 33, 33, 34, 34, 35, 35, 35, 35, + 36, 37, 32, 32, 32, 32, 33, 33, 33, 33, 34, 34, 35, 35, 35, 35, 36, 37, + 32, 32, 32, 33, 33, 33, 33, 33, 34, 34, 35, 35, 35, 35, 36, 37, 32, 33, + 33, 33, 33, 33, 33, 33, 34, 35, 36, 36, 36, 36, 36, 38, 33, 33, 33, 33, + 33, 33, 33, 34, 34, 35, 36, 36, 36, 36, 37, 38, 34, 34, 34, 34, 34, 34, + 34, 34, 35, 36, 37, 37, 37, 37, 38, 39, 34, 34, 34, 34, 34, 34, 34, 34, + 35, 36, 37, 37, 37, 37, 38, 39], + /* Size 32x16 */ + [32, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 34, 34, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 33, 33, 34, 34, 31, 31, 31, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 33, 33, 34, 34, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, + 34, 34, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 33, 33, 33, 33, 34, 34, 31, 31, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 33, 33, 33, 33, 33, 33, 33, 34, 34, 31, 31, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, + 33, 33, 33, 33, 33, 33, 34, 34, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, + 33, 34, 34, 34, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 33, 33, 33, 33, 33, 33, 33, 33, 33, 34, 34, 34, 34, 34, 34, 35, 35, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 33, + 33, 33, 33, 34, 34, 34, 34, 34, 34, 34, 35, 35, 36, 36, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 33, 33, 33, 33, 33, 33, 34, 34, 34, 34, 34, 34, + 35, 35, 35, 35, 35, 35, 36, 36, 37, 37, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 33, 33, 33, 33, 33, 33, 34, 34, 34, 34, 34, 34, 35, 35, 35, 35, + 35, 35, 36, 36, 37, 37, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, + 33, 33, 33, 33, 34, 34, 34, 34, 34, 34, 35, 35, 35, 35, 35, 35, 36, 36, + 37, 37, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 33, 33, 33, + 34, 34, 34, 34, 34, 34, 35, 35, 35, 35, 35, 35, 36, 36, 37, 37, 32, 33, + 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 34, 34, 34, 34, 34, + 34, 35, 35, 36, 36, 36, 36, 36, 36, 37, 38, 38, 34, 34, 34, 34, 34, 34, + 34, 34, 34, 33, 33, 33, 33, 33, 34, 34, 35, 35, 35, 35, 35, 35, 36, 36, + 37, 37, 37, 37, 38, 38, 39, 39], + /* Size 4x16 */ + [31, 31, 32, 32, 31, 32, 32, 32, 31, 32, 32, 32, 31, 32, 32, 32, 31, 32, + 32, 32, 32, 32, 32, 33, 32, 32, 32, 33, 32, 32, 33, 33, 32, 32, 33, 34, + 32, 32, 33, 34, 32, 32, 33, 34, 32, 32, 34, 35, 32, 33, 34, 35, 32, 33, + 34, 35, 33, 33, 35, 36, 34, 34, 36, 37], + /* Size 16x4 */ + [31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 34, 31, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 34, 32, 32, 32, 32, + 32, 32, 32, 33, 33, 33, 33, 34, 34, 34, 35, 36, 32, 32, 32, 32, 32, 33, + 33, 33, 34, 34, 34, 35, 35, 35, 36, 37], + /* Size 8x32 */ + [32, 31, 31, 31, 31, 32, 32, 32, 31, 31, 31, 31, 32, 32, 32, 33, 31, 31, + 32, 32, 32, 32, 32, 33, 31, 32, 32, 32, 32, 32, 32, 33, 31, 32, 32, 32, + 32, 32, 32, 33, 31, 32, 32, 32, 32, 32, 32, 33, 31, 32, 32, 32, 32, 32, + 32, 33, 31, 32, 32, 32, 32, 32, 32, 33, 31, 32, 32, 32, 32, 32, 32, 33, + 31, 32, 32, 32, 32, 32, 32, 33, 31, 32, 32, 32, 32, 33, 33, 33, 31, 32, + 32, 32, 32, 33, 33, 33, 31, 32, 32, 32, 32, 33, 33, 33, 31, 32, 32, 32, + 32, 33, 33, 33, 31, 32, 32, 32, 32, 33, 33, 33, 31, 32, 32, 32, 33, 33, + 33, 34, 31, 32, 32, 32, 33, 34, 34, 34, 32, 32, 32, 32, 33, 34, 34, 34, + 32, 32, 32, 32, 33, 34, 34, 34, 32, 32, 32, 32, 33, 34, 34, 34, 32, 32, + 32, 32, 33, 34, 34, 34, 32, 32, 32, 32, 33, 34, 34, 35, 32, 32, 32, 32, + 33, 35, 35, 35, 32, 32, 33, 33, 33, 35, 35, 36, 32, 32, 33, 33, 34, 35, + 35, 36, 32, 32, 33, 33, 34, 35, 35, 36, 32, 32, 33, 33, 34, 35, 35, 36, + 32, 32, 33, 33, 34, 35, 35, 36, 32, 33, 33, 33, 34, 36, 36, 36, 33, 33, + 33, 33, 34, 36, 36, 37, 34, 34, 34, 34, 35, 37, 37, 38, 34, 34, 34, 34, + 35, 37, 37, 38], + /* Size 32x8 */ + [32, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 34, 34, 31, 31, 31, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 33, 33, 34, 34, 31, 31, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, + 33, 33, 33, 33, 34, 34, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 33, 33, 33, 33, + 34, 34, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, + 33, 33, 33, 33, 33, 33, 33, 33, 34, 34, 34, 34, 34, 34, 35, 35, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 33, 33, 33, 34, 34, 34, 34, + 34, 34, 35, 35, 35, 35, 35, 35, 36, 36, 37, 37, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 33, 33, 33, 33, 33, 33, 34, 34, 34, 34, 34, 34, 35, 35, + 35, 35, 35, 35, 36, 36, 37, 37, 32, 33, 33, 33, 33, 33, 33, 33, 33, 33, + 33, 33, 33, 33, 33, 34, 34, 34, 34, 34, 34, 35, 35, 36, 36, 36, 36, 36, + 36, 37, 38, 38]], + [ /* Chroma */ + /* Size 4x4 */ + [31, 31, 34, 38, 31, 32, 35, 40, 34, 35, 39, 43, 38, 40, 43, 47], + /* Size 8x8 */ + [31, 31, 31, 30, 34, 35, 37, 40, 31, 31, 31, 31, 34, 35, 38, 41, 31, 31, + 31, 31, 35, 36, 39, 41, 30, 31, 31, 32, 35, 36, 40, 42, 34, 34, 35, 35, + 39, 40, 43, 44, 35, 35, 36, 36, 40, 41, 44, 45, 37, 38, 39, 40, 43, 44, + 47, 47, 40, 41, 41, 42, 44, 45, 47, 48], + /* Size 16x16 */ + [32, 31, 31, 31, 31, 30, 30, 31, 33, 33, 33, 35, 36, 36, 38, 41, 31, 31, + 31, 31, 31, 31, 31, 31, 33, 34, 34, 36, 37, 37, 39, 42, 31, 31, 31, 31, + 31, 31, 31, 32, 34, 34, 34, 37, 38, 38, 40, 42, 31, 31, 31, 31, 31, 31, + 31, 32, 34, 34, 34, 37, 38, 38, 40, 42, 31, 31, 31, 31, 31, 31, 31, 32, + 34, 35, 35, 37, 39, 39, 40, 42, 30, 31, 31, 31, 31, 32, 32, 32, 34, 35, + 35, 38, 40, 40, 41, 42, 30, 31, 31, 31, 31, 32, 32, 32, 34, 35, 35, 38, + 40, 40, 41, 42, 31, 31, 32, 32, 32, 32, 32, 33, 35, 36, 36, 38, 40, 40, + 41, 43, 33, 33, 34, 34, 34, 34, 34, 35, 37, 38, 38, 41, 42, 42, 43, 44, + 33, 34, 34, 34, 35, 35, 35, 36, 38, 39, 39, 41, 43, 43, 44, 45, 33, 34, + 34, 34, 35, 35, 35, 36, 38, 39, 39, 41, 43, 43, 44, 45, 35, 36, 37, 37, + 37, 38, 38, 38, 41, 41, 41, 44, 46, 46, 46, 46, 36, 37, 38, 38, 39, 40, + 40, 40, 42, 43, 43, 46, 47, 47, 47, 47, 36, 37, 38, 38, 39, 40, 40, 40, + 42, 43, 43, 46, 47, 47, 47, 47, 38, 39, 40, 40, 40, 41, 41, 41, 43, 44, + 44, 46, 47, 47, 47, 48, 41, 42, 42, 42, 42, 42, 42, 43, 44, 45, 45, 46, + 47, 47, 48, 48], + /* Size 32x32 */ + [32, 31, 31, 31, 31, 31, 31, 31, 31, 30, 30, 30, 30, 30, 31, 32, 33, 33, + 33, 33, 33, 34, 35, 36, 36, 36, 36, 37, 38, 40, 41, 41, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 30, 30, 30, 30, 31, 32, 33, 34, 34, 34, 34, 35, + 36, 37, 37, 37, 37, 37, 39, 40, 42, 42, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 32, 33, 34, 34, 34, 34, 35, 36, 37, 37, 37, + 37, 38, 39, 40, 42, 42, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 32, 32, 34, 34, 34, 34, 34, 35, 36, 38, 38, 38, 38, 38, 40, 41, + 42, 42, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 33, + 34, 34, 34, 34, 34, 35, 37, 38, 38, 38, 38, 39, 40, 41, 42, 42, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 33, 34, 34, 34, 34, + 34, 35, 37, 38, 38, 38, 38, 39, 40, 41, 42, 42, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 32, 33, 34, 34, 34, 34, 34, 35, 37, 38, + 38, 38, 38, 39, 40, 41, 42, 42, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 32, 33, 34, 34, 34, 34, 34, 36, 37, 38, 38, 38, 38, 39, + 40, 41, 42, 42, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 32, 33, 34, 35, 35, 35, 35, 36, 37, 38, 39, 39, 39, 39, 40, 41, 42, 42, + 30, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 33, 34, 35, + 35, 35, 35, 36, 37, 39, 39, 39, 39, 40, 40, 41, 42, 42, 30, 30, 31, 31, + 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 33, 34, 35, 35, 35, 35, 36, + 38, 39, 40, 40, 40, 40, 41, 42, 42, 42, 30, 30, 31, 31, 31, 31, 31, 31, + 31, 31, 32, 32, 32, 32, 32, 33, 34, 35, 35, 35, 35, 36, 38, 39, 40, 40, + 40, 40, 41, 42, 42, 42, 30, 30, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, + 32, 32, 32, 33, 34, 35, 35, 35, 35, 36, 38, 39, 40, 40, 40, 40, 41, 42, + 42, 42, 30, 30, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 33, + 34, 35, 35, 35, 35, 36, 38, 39, 40, 40, 40, 40, 41, 42, 42, 42, 31, 31, + 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 34, 35, 36, 36, 36, + 36, 37, 38, 40, 40, 40, 40, 41, 41, 42, 43, 43, 32, 32, 32, 32, 33, 33, + 33, 33, 33, 33, 33, 33, 33, 33, 34, 35, 36, 37, 37, 37, 37, 38, 39, 41, + 41, 41, 41, 42, 42, 43, 43, 43, 33, 33, 33, 34, 34, 34, 34, 34, 34, 34, + 34, 34, 34, 34, 35, 36, 37, 38, 38, 38, 38, 39, 41, 42, 42, 42, 42, 43, + 43, 44, 44, 44, 33, 34, 34, 34, 34, 34, 34, 34, 35, 35, 35, 35, 35, 35, + 36, 37, 38, 39, 39, 39, 39, 40, 41, 43, 43, 43, 43, 43, 44, 44, 45, 45, + 33, 34, 34, 34, 34, 34, 34, 34, 35, 35, 35, 35, 35, 35, 36, 37, 38, 39, + 39, 39, 39, 40, 41, 43, 43, 43, 43, 43, 44, 44, 45, 45, 33, 34, 34, 34, + 34, 34, 34, 34, 35, 35, 35, 35, 35, 35, 36, 37, 38, 39, 39, 39, 39, 40, + 41, 43, 43, 43, 43, 43, 44, 44, 45, 45, 33, 34, 34, 34, 34, 34, 34, 34, + 35, 35, 35, 35, 35, 35, 36, 37, 38, 39, 39, 39, 39, 40, 41, 43, 43, 43, + 43, 43, 44, 44, 45, 45, 34, 35, 35, 35, 35, 35, 35, 36, 36, 36, 36, 36, + 36, 36, 37, 38, 39, 40, 40, 40, 40, 41, 42, 44, 44, 44, 44, 44, 45, 45, + 45, 45, 35, 36, 36, 36, 37, 37, 37, 37, 37, 37, 38, 38, 38, 38, 38, 39, + 41, 41, 41, 41, 41, 42, 44, 45, 46, 46, 46, 46, 46, 46, 46, 46, 36, 37, + 37, 38, 38, 38, 38, 38, 38, 39, 39, 39, 39, 39, 40, 41, 42, 43, 43, 43, + 43, 44, 45, 46, 47, 47, 47, 47, 47, 47, 47, 47, 36, 37, 37, 38, 38, 38, + 38, 38, 39, 39, 40, 40, 40, 40, 40, 41, 42, 43, 43, 43, 43, 44, 46, 47, + 47, 47, 47, 47, 47, 47, 47, 47, 36, 37, 37, 38, 38, 38, 38, 38, 39, 39, + 40, 40, 40, 40, 40, 41, 42, 43, 43, 43, 43, 44, 46, 47, 47, 47, 47, 47, + 47, 47, 47, 47, 36, 37, 37, 38, 38, 38, 38, 38, 39, 39, 40, 40, 40, 40, + 40, 41, 42, 43, 43, 43, 43, 44, 46, 47, 47, 47, 47, 47, 47, 47, 47, 47, + 37, 37, 38, 38, 39, 39, 39, 39, 39, 40, 40, 40, 40, 40, 41, 42, 43, 43, + 43, 43, 43, 44, 46, 47, 47, 47, 47, 47, 47, 47, 47, 47, 38, 39, 39, 40, + 40, 40, 40, 40, 40, 40, 41, 41, 41, 41, 41, 42, 43, 44, 44, 44, 44, 45, + 46, 47, 47, 47, 47, 47, 47, 48, 48, 48, 40, 40, 40, 41, 41, 41, 41, 41, + 41, 41, 42, 42, 42, 42, 42, 43, 44, 44, 44, 44, 44, 45, 46, 47, 47, 47, + 47, 47, 48, 48, 48, 48, 41, 42, 42, 42, 42, 42, 42, 42, 42, 42, 42, 42, + 42, 42, 43, 43, 44, 45, 45, 45, 45, 45, 46, 47, 47, 47, 47, 47, 48, 48, + 48, 48, 41, 42, 42, 42, 42, 42, 42, 42, 42, 42, 42, 42, 42, 42, 43, 43, + 44, 45, 45, 45, 45, 45, 46, 47, 47, 47, 47, 47, 48, 48, 48, 48], + /* Size 4x8 */ + [31, 31, 35, 37, 31, 31, 36, 38, 31, 32, 37, 39, 31, 32, 37, 40, 34, 36, + 40, 43, 35, 37, 42, 44, 38, 40, 45, 47, 41, 42, 45, 47], + /* Size 8x4 */ + [31, 31, 31, 31, 34, 35, 38, 41, 31, 31, 32, 32, 36, 37, 40, 42, 35, 36, + 37, 37, 40, 42, 45, 45, 37, 38, 39, 40, 43, 44, 47, 47], + /* Size 8x16 */ + [32, 31, 31, 31, 33, 37, 37, 38, 31, 31, 31, 31, 33, 38, 38, 39, 31, 31, + 31, 31, 34, 38, 38, 40, 31, 31, 31, 31, 34, 38, 38, 40, 31, 31, 32, 32, + 34, 39, 39, 40, 30, 31, 32, 32, 35, 40, 40, 41, 30, 31, 32, 32, 35, 40, + 40, 41, 31, 32, 33, 33, 35, 40, 40, 41, 33, 34, 35, 35, 37, 42, 42, 43, + 33, 35, 36, 36, 38, 43, 43, 44, 33, 35, 36, 36, 38, 43, 43, 44, 35, 37, + 38, 38, 41, 45, 45, 46, 37, 39, 40, 40, 43, 47, 47, 47, 37, 39, 40, 40, + 43, 47, 47, 47, 39, 40, 41, 41, 43, 47, 47, 47, 42, 42, 43, 43, 44, 47, + 47, 48], + /* Size 16x8 */ + [32, 31, 31, 31, 31, 30, 30, 31, 33, 33, 33, 35, 37, 37, 39, 42, 31, 31, + 31, 31, 31, 31, 31, 32, 34, 35, 35, 37, 39, 39, 40, 42, 31, 31, 31, 31, + 32, 32, 32, 33, 35, 36, 36, 38, 40, 40, 41, 43, 31, 31, 31, 31, 32, 32, + 32, 33, 35, 36, 36, 38, 40, 40, 41, 43, 33, 33, 34, 34, 34, 35, 35, 35, + 37, 38, 38, 41, 43, 43, 43, 44, 37, 38, 38, 38, 39, 40, 40, 40, 42, 43, + 43, 45, 47, 47, 47, 47, 37, 38, 38, 38, 39, 40, 40, 40, 42, 43, 43, 45, + 47, 47, 47, 47, 38, 39, 40, 40, 40, 41, 41, 41, 43, 44, 44, 46, 47, 47, + 47, 48], + /* Size 16x32 */ + [32, 31, 31, 31, 31, 31, 31, 31, 33, 35, 37, 37, 37, 37, 38, 42, 31, 31, + 31, 31, 31, 31, 31, 31, 33, 35, 37, 37, 37, 37, 39, 42, 31, 31, 31, 31, + 31, 31, 31, 32, 33, 35, 38, 38, 38, 38, 39, 42, 31, 31, 31, 31, 31, 31, + 31, 32, 34, 36, 38, 38, 38, 38, 40, 42, 31, 31, 31, 31, 31, 31, 31, 32, + 34, 36, 38, 38, 38, 38, 40, 42, 31, 31, 31, 31, 31, 31, 31, 32, 34, 36, + 38, 38, 38, 38, 40, 42, 31, 31, 31, 31, 31, 31, 31, 32, 34, 36, 38, 38, + 38, 38, 40, 42, 31, 31, 31, 31, 31, 31, 31, 32, 34, 36, 38, 38, 38, 38, + 40, 42, 31, 31, 31, 31, 32, 32, 32, 32, 34, 36, 39, 39, 39, 39, 40, 42, + 30, 31, 31, 32, 32, 32, 32, 32, 34, 37, 39, 39, 39, 39, 40, 42, 30, 31, + 31, 32, 32, 32, 32, 33, 35, 37, 40, 40, 40, 40, 41, 42, 30, 31, 31, 32, + 32, 32, 32, 33, 35, 37, 40, 40, 40, 40, 41, 42, 30, 31, 31, 32, 32, 32, + 32, 33, 35, 37, 40, 40, 40, 40, 41, 42, 30, 31, 31, 32, 32, 32, 32, 33, + 35, 37, 40, 40, 40, 40, 41, 42, 31, 31, 32, 32, 33, 33, 33, 33, 35, 38, + 40, 40, 40, 40, 41, 43, 32, 32, 33, 33, 34, 34, 34, 34, 36, 39, 41, 41, + 41, 41, 42, 44, 33, 33, 34, 35, 35, 35, 35, 35, 37, 40, 42, 42, 42, 42, + 43, 44, 33, 34, 35, 35, 36, 36, 36, 36, 38, 40, 43, 43, 43, 43, 44, 45, + 33, 34, 35, 35, 36, 36, 36, 36, 38, 40, 43, 43, 43, 43, 44, 45, 33, 34, + 35, 35, 36, 36, 36, 36, 38, 40, 43, 43, 43, 43, 44, 45, 33, 34, 35, 35, + 36, 36, 36, 36, 38, 40, 43, 43, 43, 43, 44, 45, 34, 35, 36, 37, 37, 37, + 37, 37, 39, 42, 44, 44, 44, 44, 45, 45, 35, 36, 37, 38, 38, 38, 38, 39, + 41, 43, 45, 45, 45, 45, 46, 46, 36, 37, 38, 39, 39, 39, 39, 40, 42, 44, + 47, 47, 47, 47, 47, 47, 37, 38, 39, 40, 40, 40, 40, 41, 43, 45, 47, 47, + 47, 47, 47, 47, 37, 38, 39, 40, 40, 40, 40, 41, 43, 45, 47, 47, 47, 47, + 47, 47, 37, 38, 39, 40, 40, 40, 40, 41, 43, 45, 47, 47, 47, 47, 47, 47, + 37, 38, 39, 40, 40, 40, 40, 41, 43, 45, 47, 47, 47, 47, 47, 47, 39, 39, + 40, 41, 41, 41, 41, 42, 43, 45, 47, 47, 47, 47, 47, 48, 40, 41, 41, 42, + 42, 42, 42, 42, 44, 45, 47, 47, 47, 47, 47, 48, 42, 42, 42, 43, 43, 43, + 43, 43, 44, 46, 47, 47, 47, 47, 48, 48, 42, 42, 42, 43, 43, 43, 43, 43, + 44, 46, 47, 47, 47, 47, 48, 48], + /* Size 32x16 */ + [32, 31, 31, 31, 31, 31, 31, 31, 31, 30, 30, 30, 30, 30, 31, 32, 33, 33, + 33, 33, 33, 34, 35, 36, 37, 37, 37, 37, 39, 40, 42, 42, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 33, 34, 34, 34, 34, 35, + 36, 37, 38, 38, 38, 38, 39, 41, 42, 42, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 32, 33, 34, 35, 35, 35, 35, 36, 37, 38, 39, 39, + 39, 39, 40, 41, 42, 42, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, + 32, 32, 32, 33, 35, 35, 35, 35, 35, 37, 38, 39, 40, 40, 40, 40, 41, 42, + 43, 43, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 33, 34, + 35, 36, 36, 36, 36, 37, 38, 39, 40, 40, 40, 40, 41, 42, 43, 43, 31, 31, + 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 33, 34, 35, 36, 36, 36, + 36, 37, 38, 39, 40, 40, 40, 40, 41, 42, 43, 43, 31, 31, 31, 31, 31, 31, + 31, 31, 32, 32, 32, 32, 32, 32, 33, 34, 35, 36, 36, 36, 36, 37, 38, 39, + 40, 40, 40, 40, 41, 42, 43, 43, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, + 33, 33, 33, 33, 33, 34, 35, 36, 36, 36, 36, 37, 39, 40, 41, 41, 41, 41, + 42, 42, 43, 43, 33, 33, 33, 34, 34, 34, 34, 34, 34, 34, 35, 35, 35, 35, + 35, 36, 37, 38, 38, 38, 38, 39, 41, 42, 43, 43, 43, 43, 43, 44, 44, 44, + 35, 35, 35, 36, 36, 36, 36, 36, 36, 37, 37, 37, 37, 37, 38, 39, 40, 40, + 40, 40, 40, 42, 43, 44, 45, 45, 45, 45, 45, 45, 46, 46, 37, 37, 38, 38, + 38, 38, 38, 38, 39, 39, 40, 40, 40, 40, 40, 41, 42, 43, 43, 43, 43, 44, + 45, 47, 47, 47, 47, 47, 47, 47, 47, 47, 37, 37, 38, 38, 38, 38, 38, 38, + 39, 39, 40, 40, 40, 40, 40, 41, 42, 43, 43, 43, 43, 44, 45, 47, 47, 47, + 47, 47, 47, 47, 47, 47, 37, 37, 38, 38, 38, 38, 38, 38, 39, 39, 40, 40, + 40, 40, 40, 41, 42, 43, 43, 43, 43, 44, 45, 47, 47, 47, 47, 47, 47, 47, + 47, 47, 37, 37, 38, 38, 38, 38, 38, 38, 39, 39, 40, 40, 40, 40, 40, 41, + 42, 43, 43, 43, 43, 44, 45, 47, 47, 47, 47, 47, 47, 47, 47, 47, 38, 39, + 39, 40, 40, 40, 40, 40, 40, 40, 41, 41, 41, 41, 41, 42, 43, 44, 44, 44, + 44, 45, 46, 47, 47, 47, 47, 47, 47, 47, 48, 48, 42, 42, 42, 42, 42, 42, + 42, 42, 42, 42, 42, 42, 42, 42, 43, 44, 44, 45, 45, 45, 45, 45, 46, 47, + 47, 47, 47, 47, 48, 48, 48, 48], + /* Size 4x16 */ + [31, 31, 35, 37, 31, 31, 35, 38, 31, 31, 36, 38, 31, 31, 36, 38, 31, 32, + 36, 39, 31, 32, 37, 40, 31, 32, 37, 40, 31, 33, 38, 40, 33, 35, 40, 42, + 34, 36, 40, 43, 34, 36, 40, 43, 36, 38, 43, 45, 38, 40, 45, 47, 38, 40, + 45, 47, 39, 41, 45, 47, 42, 43, 46, 47], + /* Size 16x4 */ + [31, 31, 31, 31, 31, 31, 31, 31, 33, 34, 34, 36, 38, 38, 39, 42, 31, 31, + 31, 31, 32, 32, 32, 33, 35, 36, 36, 38, 40, 40, 41, 43, 35, 35, 36, 36, + 36, 37, 37, 38, 40, 40, 40, 43, 45, 45, 45, 46, 37, 38, 38, 38, 39, 40, + 40, 40, 42, 43, 43, 45, 47, 47, 47, 47], + /* Size 8x32 */ + [32, 31, 31, 31, 33, 37, 37, 38, 31, 31, 31, 31, 33, 37, 37, 39, 31, 31, + 31, 31, 33, 38, 38, 39, 31, 31, 31, 31, 34, 38, 38, 40, 31, 31, 31, 31, + 34, 38, 38, 40, 31, 31, 31, 31, 34, 38, 38, 40, 31, 31, 31, 31, 34, 38, + 38, 40, 31, 31, 31, 31, 34, 38, 38, 40, 31, 31, 32, 32, 34, 39, 39, 40, + 30, 31, 32, 32, 34, 39, 39, 40, 30, 31, 32, 32, 35, 40, 40, 41, 30, 31, + 32, 32, 35, 40, 40, 41, 30, 31, 32, 32, 35, 40, 40, 41, 30, 31, 32, 32, + 35, 40, 40, 41, 31, 32, 33, 33, 35, 40, 40, 41, 32, 33, 34, 34, 36, 41, + 41, 42, 33, 34, 35, 35, 37, 42, 42, 43, 33, 35, 36, 36, 38, 43, 43, 44, + 33, 35, 36, 36, 38, 43, 43, 44, 33, 35, 36, 36, 38, 43, 43, 44, 33, 35, + 36, 36, 38, 43, 43, 44, 34, 36, 37, 37, 39, 44, 44, 45, 35, 37, 38, 38, + 41, 45, 45, 46, 36, 38, 39, 39, 42, 47, 47, 47, 37, 39, 40, 40, 43, 47, + 47, 47, 37, 39, 40, 40, 43, 47, 47, 47, 37, 39, 40, 40, 43, 47, 47, 47, + 37, 39, 40, 40, 43, 47, 47, 47, 39, 40, 41, 41, 43, 47, 47, 47, 40, 41, + 42, 42, 44, 47, 47, 47, 42, 42, 43, 43, 44, 47, 47, 48, 42, 42, 43, 43, + 44, 47, 47, 48], + /* Size 32x8 */ + [32, 31, 31, 31, 31, 31, 31, 31, 31, 30, 30, 30, 30, 30, 31, 32, 33, 33, + 33, 33, 33, 34, 35, 36, 37, 37, 37, 37, 39, 40, 42, 42, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 33, 34, 35, 35, 35, 35, 36, + 37, 38, 39, 39, 39, 39, 40, 41, 42, 42, 31, 31, 31, 31, 31, 31, 31, 31, + 32, 32, 32, 32, 32, 32, 33, 34, 35, 36, 36, 36, 36, 37, 38, 39, 40, 40, + 40, 40, 41, 42, 43, 43, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, + 32, 32, 33, 34, 35, 36, 36, 36, 36, 37, 38, 39, 40, 40, 40, 40, 41, 42, + 43, 43, 33, 33, 33, 34, 34, 34, 34, 34, 34, 34, 35, 35, 35, 35, 35, 36, + 37, 38, 38, 38, 38, 39, 41, 42, 43, 43, 43, 43, 43, 44, 44, 44, 37, 37, + 38, 38, 38, 38, 38, 38, 39, 39, 40, 40, 40, 40, 40, 41, 42, 43, 43, 43, + 43, 44, 45, 47, 47, 47, 47, 47, 47, 47, 47, 47, 37, 37, 38, 38, 38, 38, + 38, 38, 39, 39, 40, 40, 40, 40, 40, 41, 42, 43, 43, 43, 43, 44, 45, 47, + 47, 47, 47, 47, 47, 47, 47, 47, 38, 39, 39, 40, 40, 40, 40, 40, 40, 40, + 41, 41, 41, 41, 41, 42, 43, 44, 44, 44, 44, 45, 46, 47, 47, 47, 47, 47, + 47, 47, 48, 48]] + ], + [ + [ /* Luma */ + /* Size 4x4 */ + [31, 31, 31, 32, 31, 32, 32, 32, 31, 32, 32, 32, 32, 32, 32, 33], + /* Size 8x8 */ + [31, 31, 31, 31, 31, 31, 32, 32, 31, 32, 32, 32, 32, 32, 32, 32, 31, 32, + 32, 32, 32, 32, 32, 32, 31, 32, 32, 32, 32, 32, 32, 32, 31, 32, 32, 32, + 32, 32, 32, 32, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 33, 33, 32, 32, 32, 32, 32, 32, 33, 33], + /* Size 16x16 */ + [32, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 31, 31, 31, 31, + 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, 31, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, 31, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 33, 31, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 33, 33, 33, 33, 33, 31, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 33, 33, 33, 33, 33, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 33, 33, 33, 33, 33, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, + 33, 33, 33, 33], + /* Size 32x32 */ + [32, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, + 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, 31, 31, 31, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, 31, 31, + 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, 31, 31, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, + 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, 31, 31, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, 31, 31, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, 31, 31, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 33, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 33, 33, 33, 33, 33, 31, 31, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 33, 33, 33, 33, 33, 33, 33, 33, 33, 31, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, + 33, 33, 33, 33, 33, 33, 33, 33, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 33, 33, 33, + 33, 33, 33, 33, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, + 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 31, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 31, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 33, + 33, 33, 33, 33, 33, 33, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 33, 33, 33, 33, 33, + 33, 33, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33], + /* Size 4x8 */ + [31, 31, 31, 32, 31, 32, 32, 32, 31, 32, 32, 32, 31, 32, 32, 32, 31, 32, + 32, 32, 31, 32, 32, 33, 32, 32, 32, 33, 32, 32, 32, 33], + /* Size 8x4 */ + [31, 31, 31, 31, 31, 31, 32, 32, 31, 32, 32, 32, 32, 32, 32, 32, 31, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33], + /* Size 8x16 */ + [32, 31, 31, 31, 31, 31, 31, 32, 31, 31, 31, 31, 31, 31, 32, 32, 31, 31, + 32, 32, 32, 32, 32, 32, 31, 32, 32, 32, 32, 32, 32, 32, 31, 32, 32, 32, + 32, 32, 32, 32, 31, 32, 32, 32, 32, 32, 32, 32, 31, 32, 32, 32, 32, 32, + 32, 32, 31, 32, 32, 32, 32, 32, 32, 32, 31, 32, 32, 32, 32, 32, 32, 32, + 31, 32, 32, 32, 32, 32, 32, 32, 31, 32, 32, 32, 32, 32, 32, 32, 31, 32, + 32, 32, 32, 32, 33, 33, 31, 32, 32, 32, 32, 32, 33, 33, 32, 32, 32, 32, + 32, 32, 33, 34, 32, 32, 32, 32, 32, 32, 33, 34, 32, 32, 32, 32, 32, 32, + 33, 34], + /* Size 16x8 */ + [32, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 31, 31, + 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, + 33, 33, 33, 33, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 34, + 34, 34], + /* Size 16x32 */ + [32, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 31, 31, 31, 31, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, 31, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 33, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 33, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, + 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 31, 31, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 31, 31, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 31, 31, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 31, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 33, 33, 33, 33, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 33, 33, 33, 33, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, + 33, 33, 33, 34, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, + 34, 34, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 34, 34, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 34, 34, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 34, 34, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 34, 34, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 33, 33, 33, 34, 34, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 33, 33, 33, 34, 34], + /* Size 32x16 */ + [32, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, 31, 31, 31, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, + 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, 31, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, 31, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, + 33, 33, 33, 33, 33, 33, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 33, 33, 33, 33, 33, + 33, 33, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 33, 33, 33, 33, 34, 34, 34, 34, 34, 34, 34, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 33, 33, 33, 33, 33, 33, + 34, 34, 34, 34, 34, 34, 34, 34], + /* Size 4x16 */ + [31, 31, 31, 32, 31, 31, 31, 32, 31, 32, 32, 32, 31, 32, 32, 32, 31, 32, + 32, 32, 31, 32, 32, 32, 31, 32, 32, 32, 31, 32, 32, 32, 31, 32, 32, 32, + 31, 32, 32, 32, 31, 32, 32, 32, 32, 32, 32, 33, 32, 32, 32, 33, 32, 32, + 32, 33, 32, 32, 32, 33, 32, 32, 32, 33], + /* Size 16x4 */ + [31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 31, 31, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 33, 33, 33, 33, 33], + /* Size 8x32 */ + [32, 31, 31, 31, 31, 31, 31, 32, 31, 31, 31, 31, 31, 31, 32, 32, 31, 31, + 31, 31, 31, 31, 32, 32, 31, 31, 32, 32, 32, 32, 32, 32, 31, 31, 32, 32, + 32, 32, 32, 32, 31, 32, 32, 32, 32, 32, 32, 32, 31, 32, 32, 32, 32, 32, + 32, 32, 31, 32, 32, 32, 32, 32, 32, 32, 31, 32, 32, 32, 32, 32, 32, 32, + 31, 32, 32, 32, 32, 32, 32, 32, 31, 32, 32, 32, 32, 32, 32, 32, 31, 32, + 32, 32, 32, 32, 32, 32, 31, 32, 32, 32, 32, 32, 32, 32, 31, 32, 32, 32, + 32, 32, 32, 32, 31, 32, 32, 32, 32, 32, 32, 32, 31, 32, 32, 32, 32, 32, + 32, 32, 31, 32, 32, 32, 32, 32, 32, 32, 31, 32, 32, 32, 32, 32, 32, 32, + 31, 32, 32, 32, 32, 32, 32, 32, 31, 32, 32, 32, 32, 32, 32, 32, 31, 32, + 32, 32, 32, 32, 32, 32, 31, 32, 32, 32, 32, 32, 32, 33, 31, 32, 32, 32, + 32, 32, 33, 33, 31, 32, 32, 32, 32, 32, 33, 33, 31, 32, 32, 32, 32, 32, + 33, 33, 32, 32, 32, 32, 32, 32, 33, 34, 32, 32, 32, 32, 32, 32, 33, 34, + 32, 32, 32, 32, 32, 32, 33, 34, 32, 32, 32, 32, 32, 32, 33, 34, 32, 32, + 32, 32, 32, 32, 33, 34, 32, 32, 32, 32, 32, 32, 33, 34, 32, 32, 32, 32, + 32, 32, 33, 34], + /* Size 32x8 */ + [32, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 31, 31, 31, 31, + 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, 31, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, + 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 31, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, + 33, 33, 33, 33, 33, 33, 33, 33, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 33, 34, 34, 34, + 34, 34, 34, 34]], + [ /* Chroma */ + /* Size 4x4 */ + [31, 31, 31, 34, 31, 31, 31, 35, 31, 31, 32, 35, 34, 35, 35, 39], + /* Size 8x8 */ + [31, 31, 31, 31, 30, 31, 33, 33, 31, 31, 31, 31, 31, 32, 34, 34, 31, 31, + 31, 31, 31, 32, 34, 34, 31, 31, 31, 31, 31, 32, 35, 35, 30, 31, 31, 31, + 32, 32, 35, 35, 31, 32, 32, 32, 32, 33, 36, 36, 33, 34, 34, 35, 35, 36, + 39, 39, 33, 34, 34, 35, 35, 36, 39, 39], + /* Size 16x16 */ + [32, 31, 31, 31, 31, 31, 31, 30, 30, 30, 30, 31, 33, 33, 33, 33, 31, 31, + 31, 31, 31, 31, 31, 31, 30, 30, 30, 32, 33, 34, 34, 34, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 32, 33, 34, 34, 34, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 32, 34, 34, 34, 34, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 32, 34, 34, 34, 34, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 32, 34, 34, 34, 34, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 33, + 34, 35, 35, 35, 30, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 33, 34, 35, + 35, 35, 30, 30, 31, 31, 31, 31, 31, 31, 32, 32, 32, 33, 34, 35, 35, 35, + 30, 30, 31, 31, 31, 31, 31, 31, 32, 32, 32, 33, 34, 35, 35, 35, 30, 30, + 31, 31, 31, 31, 31, 31, 32, 32, 32, 33, 34, 35, 35, 35, 31, 32, 32, 32, + 32, 32, 33, 33, 33, 33, 33, 34, 36, 37, 37, 37, 33, 33, 33, 34, 34, 34, + 34, 34, 34, 34, 34, 36, 37, 38, 38, 38, 33, 34, 34, 34, 34, 34, 35, 35, + 35, 35, 35, 37, 38, 39, 39, 39, 33, 34, 34, 34, 34, 34, 35, 35, 35, 35, + 35, 37, 38, 39, 39, 39, 33, 34, 34, 34, 34, 34, 35, 35, 35, 35, 35, 37, + 38, 39, 39, 39], + /* Size 32x32 */ + [32, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 30, 30, 30, 30, 30, + 30, 30, 30, 31, 31, 32, 33, 33, 33, 33, 33, 33, 33, 34, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 30, 30, 30, 30, 30, 30, 30, 31, + 31, 32, 33, 33, 33, 33, 33, 33, 33, 34, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 30, 30, 30, 30, 30, 30, 31, 32, 32, 33, 34, + 34, 34, 34, 34, 34, 34, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 33, 33, 34, 34, 34, 34, 34, + 34, 35, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 32, 33, 33, 34, 34, 34, 34, 34, 34, 35, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 32, 32, 33, 34, 34, 34, 34, 34, 34, 34, 35, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, 33, + 34, 34, 34, 34, 34, 34, 34, 35, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, 33, 34, 34, 34, 34, + 34, 34, 34, 35, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 32, 32, 33, 34, 34, 34, 34, 34, 34, 34, 35, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 32, 32, 33, 34, 34, 34, 34, 34, 34, 34, 35, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, + 32, 33, 34, 34, 34, 34, 34, 34, 34, 35, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, 33, 34, 35, + 35, 35, 35, 35, 35, 35, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 33, 33, 34, 35, 35, 35, 35, 35, + 35, 35, 30, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 32, 33, 33, 34, 35, 35, 35, 35, 35, 35, 36, 30, 30, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 32, 33, 34, 34, 35, 35, 35, 35, 35, 35, 36, 30, 30, 30, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 33, 34, + 34, 35, 35, 35, 35, 35, 35, 36, 30, 30, 30, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 33, 34, 34, 35, 35, 35, + 35, 35, 35, 36, 30, 30, 30, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 32, 32, 32, 32, 32, 32, 32, 33, 34, 34, 35, 35, 35, 35, 35, 35, 36, + 30, 30, 30, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, + 32, 32, 32, 32, 33, 34, 34, 35, 35, 35, 35, 35, 35, 36, 30, 30, 30, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, + 33, 34, 34, 35, 35, 35, 35, 35, 35, 36, 30, 30, 30, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 33, 34, 34, 35, + 35, 35, 35, 35, 35, 36, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 34, 34, 35, 36, 36, 36, 36, 36, + 36, 37, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 33, 33, + 33, 33, 33, 33, 33, 34, 34, 35, 36, 37, 37, 37, 37, 37, 37, 37, 32, 32, + 32, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 34, 34, 34, 34, 34, 34, + 34, 34, 35, 36, 37, 37, 37, 37, 37, 37, 37, 38, 33, 33, 33, 33, 33, 34, + 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 35, 36, 37, + 37, 38, 38, 38, 38, 38, 38, 39, 33, 33, 34, 34, 34, 34, 34, 34, 34, 34, + 34, 35, 35, 35, 35, 35, 35, 35, 35, 35, 35, 36, 37, 37, 38, 39, 39, 39, + 39, 39, 39, 40, 33, 33, 34, 34, 34, 34, 34, 34, 34, 34, 34, 35, 35, 35, + 35, 35, 35, 35, 35, 35, 35, 36, 37, 37, 38, 39, 39, 39, 39, 39, 39, 40, + 33, 33, 34, 34, 34, 34, 34, 34, 34, 34, 34, 35, 35, 35, 35, 35, 35, 35, + 35, 35, 35, 36, 37, 37, 38, 39, 39, 39, 39, 39, 39, 40, 33, 33, 34, 34, + 34, 34, 34, 34, 34, 34, 34, 35, 35, 35, 35, 35, 35, 35, 35, 35, 35, 36, + 37, 37, 38, 39, 39, 39, 39, 39, 39, 40, 33, 33, 34, 34, 34, 34, 34, 34, + 34, 34, 34, 35, 35, 35, 35, 35, 35, 35, 35, 35, 35, 36, 37, 37, 38, 39, + 39, 39, 39, 39, 39, 40, 33, 33, 34, 34, 34, 34, 34, 34, 34, 34, 34, 35, + 35, 35, 35, 35, 35, 35, 35, 35, 35, 36, 37, 37, 38, 39, 39, 39, 39, 39, + 39, 40, 34, 34, 34, 35, 35, 35, 35, 35, 35, 35, 35, 35, 35, 36, 36, 36, + 36, 36, 36, 36, 36, 37, 37, 38, 39, 40, 40, 40, 40, 40, 40, 40], + /* Size 4x8 */ + [31, 31, 31, 34, 31, 31, 31, 35, 31, 31, 31, 35, 31, 32, 32, 36, 31, 32, + 32, 36, 31, 33, 33, 37, 34, 36, 36, 40, 34, 36, 36, 40], + /* Size 8x4 */ + [31, 31, 31, 31, 31, 31, 34, 34, 31, 31, 31, 32, 32, 33, 36, 36, 31, 31, + 31, 32, 32, 33, 36, 36, 34, 35, 35, 36, 36, 37, 40, 40], + /* Size 8x16 */ + [32, 31, 31, 31, 31, 31, 33, 35, 31, 31, 31, 31, 31, 31, 33, 36, 31, 31, + 31, 31, 31, 31, 34, 36, 31, 31, 31, 31, 31, 31, 34, 37, 31, 31, 31, 31, + 31, 31, 34, 37, 31, 31, 31, 31, 31, 31, 34, 37, 31, 31, 31, 32, 32, 32, + 34, 37, 30, 31, 31, 32, 32, 32, 34, 38, 30, 31, 32, 32, 32, 32, 35, 38, + 30, 31, 32, 32, 32, 32, 35, 38, 30, 31, 32, 32, 32, 32, 35, 38, 31, 32, + 33, 33, 33, 33, 36, 39, 33, 34, 34, 35, 35, 35, 37, 40, 33, 34, 35, 36, + 36, 36, 38, 41, 33, 34, 35, 36, 36, 36, 38, 41, 33, 34, 35, 36, 36, 36, + 38, 41], + /* Size 16x8 */ + [32, 31, 31, 31, 31, 31, 31, 30, 30, 30, 30, 31, 33, 33, 33, 33, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 34, 34, 34, 34, 31, 31, 31, 31, + 31, 31, 31, 31, 32, 32, 32, 33, 34, 35, 35, 35, 31, 31, 31, 31, 31, 31, + 32, 32, 32, 32, 32, 33, 35, 36, 36, 36, 31, 31, 31, 31, 31, 31, 32, 32, + 32, 32, 32, 33, 35, 36, 36, 36, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, + 32, 33, 35, 36, 36, 36, 33, 33, 34, 34, 34, 34, 34, 34, 35, 35, 35, 36, + 37, 38, 38, 38, 35, 36, 36, 37, 37, 37, 37, 38, 38, 38, 38, 39, 40, 41, + 41, 41], + /* Size 16x32 */ + [32, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 33, 34, 35, 37, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 33, 34, 35, 37, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 32, 33, 34, 36, 37, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 32, 33, 35, 36, 38, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 32, 34, 35, 36, 38, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 33, 34, 35, 37, 38, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 33, + 34, 35, 37, 38, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 33, 34, 35, + 37, 38, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 33, 34, 35, 37, 38, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 33, 34, 35, 37, 38, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 33, 34, 35, 37, 38, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 33, 34, 35, 37, 38, 31, 31, 31, 31, 31, 32, + 32, 32, 32, 32, 32, 33, 34, 36, 37, 39, 31, 31, 31, 31, 31, 32, 32, 32, + 32, 32, 32, 33, 34, 36, 37, 39, 30, 31, 31, 31, 31, 32, 32, 32, 32, 32, + 32, 33, 34, 36, 38, 39, 30, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 33, + 35, 36, 38, 40, 30, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 33, 35, 36, + 38, 40, 30, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 33, 35, 36, 38, 40, + 30, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 33, 35, 36, 38, 40, 30, 31, + 31, 31, 32, 32, 32, 32, 32, 32, 32, 33, 35, 36, 38, 40, 30, 31, 31, 31, + 32, 32, 32, 32, 32, 32, 32, 33, 35, 36, 38, 40, 31, 31, 31, 32, 32, 33, + 33, 33, 33, 33, 33, 34, 35, 37, 38, 40, 31, 32, 32, 33, 33, 33, 33, 33, + 33, 33, 33, 35, 36, 37, 39, 41, 32, 32, 33, 33, 34, 34, 34, 34, 34, 34, + 34, 35, 37, 38, 40, 41, 33, 33, 34, 34, 34, 35, 35, 35, 35, 35, 35, 36, + 37, 39, 40, 42, 33, 34, 34, 35, 35, 36, 36, 36, 36, 36, 36, 37, 38, 40, + 41, 43, 33, 34, 34, 35, 35, 36, 36, 36, 36, 36, 36, 37, 38, 40, 41, 43, + 33, 34, 34, 35, 35, 36, 36, 36, 36, 36, 36, 37, 38, 40, 41, 43, 33, 34, + 34, 35, 35, 36, 36, 36, 36, 36, 36, 37, 38, 40, 41, 43, 33, 34, 34, 35, + 35, 36, 36, 36, 36, 36, 36, 37, 38, 40, 41, 43, 33, 34, 34, 35, 35, 36, + 36, 36, 36, 36, 36, 37, 38, 40, 41, 43, 34, 34, 35, 35, 36, 36, 36, 36, + 36, 36, 36, 38, 39, 40, 42, 44], + /* Size 32x16 */ + [32, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 30, 30, 30, 30, + 30, 30, 30, 31, 31, 32, 33, 33, 33, 33, 33, 33, 33, 34, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 32, 32, 33, 34, 34, 34, 34, 34, 34, 34, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 33, 34, 34, + 34, 34, 34, 34, 34, 35, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 33, 33, 34, 35, 35, 35, 35, 35, + 35, 35, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, + 32, 32, 32, 32, 32, 32, 33, 34, 34, 35, 35, 35, 35, 35, 35, 36, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 33, 33, 34, 35, 36, 36, 36, 36, 36, 36, 36, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 34, + 35, 36, 36, 36, 36, 36, 36, 36, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 34, 35, 36, 36, 36, + 36, 36, 36, 36, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 33, 33, 34, 35, 36, 36, 36, 36, 36, 36, 36, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 33, 33, 34, 35, 36, 36, 36, 36, 36, 36, 36, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, + 33, 34, 35, 36, 36, 36, 36, 36, 36, 36, 32, 32, 32, 32, 32, 33, 33, 33, + 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 34, 35, 35, 36, 37, + 37, 37, 37, 37, 37, 38, 33, 33, 33, 33, 34, 34, 34, 34, 34, 34, 34, 34, + 34, 34, 34, 35, 35, 35, 35, 35, 35, 35, 36, 37, 37, 38, 38, 38, 38, 38, + 38, 39, 34, 34, 34, 35, 35, 35, 35, 35, 35, 35, 35, 35, 36, 36, 36, 36, + 36, 36, 36, 36, 36, 37, 37, 38, 39, 40, 40, 40, 40, 40, 40, 40, 35, 35, + 36, 36, 36, 37, 37, 37, 37, 37, 37, 37, 37, 37, 38, 38, 38, 38, 38, 38, + 38, 38, 39, 40, 40, 41, 41, 41, 41, 41, 41, 42, 37, 37, 37, 38, 38, 38, + 38, 38, 38, 38, 38, 38, 39, 39, 39, 40, 40, 40, 40, 40, 40, 40, 41, 41, + 42, 43, 43, 43, 43, 43, 43, 44], + /* Size 4x16 */ + [31, 31, 31, 34, 31, 31, 31, 34, 31, 31, 31, 35, 31, 31, 31, 35, 31, 31, + 31, 35, 31, 31, 31, 35, 31, 32, 32, 36, 31, 32, 32, 36, 31, 32, 32, 36, + 31, 32, 32, 36, 31, 32, 32, 36, 32, 33, 33, 37, 33, 35, 35, 39, 34, 36, + 36, 40, 34, 36, 36, 40, 34, 36, 36, 40], + /* Size 16x4 */ + [31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 33, 34, 34, 34, 31, 31, + 31, 31, 31, 31, 32, 32, 32, 32, 32, 33, 35, 36, 36, 36, 31, 31, 31, 31, + 31, 31, 32, 32, 32, 32, 32, 33, 35, 36, 36, 36, 34, 34, 35, 35, 35, 35, + 36, 36, 36, 36, 36, 37, 39, 40, 40, 40], + /* Size 8x32 */ + [32, 31, 31, 31, 31, 31, 33, 35, 31, 31, 31, 31, 31, 31, 33, 35, 31, 31, + 31, 31, 31, 31, 33, 36, 31, 31, 31, 31, 31, 31, 33, 36, 31, 31, 31, 31, + 31, 31, 34, 36, 31, 31, 31, 31, 31, 31, 34, 37, 31, 31, 31, 31, 31, 31, + 34, 37, 31, 31, 31, 31, 31, 31, 34, 37, 31, 31, 31, 31, 31, 31, 34, 37, + 31, 31, 31, 31, 31, 31, 34, 37, 31, 31, 31, 31, 31, 31, 34, 37, 31, 31, + 31, 31, 31, 31, 34, 37, 31, 31, 31, 32, 32, 32, 34, 37, 31, 31, 31, 32, + 32, 32, 34, 37, 30, 31, 31, 32, 32, 32, 34, 38, 30, 31, 32, 32, 32, 32, + 35, 38, 30, 31, 32, 32, 32, 32, 35, 38, 30, 31, 32, 32, 32, 32, 35, 38, + 30, 31, 32, 32, 32, 32, 35, 38, 30, 31, 32, 32, 32, 32, 35, 38, 30, 31, + 32, 32, 32, 32, 35, 38, 31, 31, 32, 33, 33, 33, 35, 38, 31, 32, 33, 33, + 33, 33, 36, 39, 32, 33, 34, 34, 34, 34, 37, 40, 33, 34, 34, 35, 35, 35, + 37, 40, 33, 34, 35, 36, 36, 36, 38, 41, 33, 34, 35, 36, 36, 36, 38, 41, + 33, 34, 35, 36, 36, 36, 38, 41, 33, 34, 35, 36, 36, 36, 38, 41, 33, 34, + 35, 36, 36, 36, 38, 41, 33, 34, 35, 36, 36, 36, 38, 41, 34, 35, 36, 36, + 36, 36, 39, 42], + /* Size 32x8 */ + [32, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 30, 30, 30, 30, + 30, 30, 30, 31, 31, 32, 33, 33, 33, 33, 33, 33, 33, 34, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 32, 33, 34, 34, 34, 34, 34, 34, 34, 35, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 33, 34, 34, 35, + 35, 35, 35, 35, 35, 36, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 33, 33, 34, 35, 36, 36, 36, 36, 36, + 36, 36, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 33, 33, 34, 35, 36, 36, 36, 36, 36, 36, 36, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 33, 33, 34, 35, 36, 36, 36, 36, 36, 36, 36, 33, 33, 33, 33, 34, 34, + 34, 34, 34, 34, 34, 34, 34, 34, 34, 35, 35, 35, 35, 35, 35, 35, 36, 37, + 37, 38, 38, 38, 38, 38, 38, 39, 35, 35, 36, 36, 36, 37, 37, 37, 37, 37, + 37, 37, 37, 37, 38, 38, 38, 38, 38, 38, 38, 38, 39, 40, 40, 41, 41, 41, + 41, 41, 41, 42]], + ], + [ + [ /* Luma */ + /* Size 4x4 */ + [31, 31, 31, 31, 31, 32, 32, 32, 31, 32, 32, 32, 31, 32, 32, 32], + /* Size 8x8 */ + [31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 32, 32, 32, 32, 32, 31, 31, 32, 32, 32, 32, 32, 32, 31, 31, 32, 32, + 32, 32, 32, 32, 31, 31, 32, 32, 32, 32, 32, 32, 31, 31, 32, 32, 32, 32, + 32, 32, 31, 31, 32, 32, 32, 32, 32, 32], + /* Size 16x16 */ + [32, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 32, 32, 32, 32, 32, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, + 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, 31, 31, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, 31, 31, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, 31, 31, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32], + /* Size 32x32 */ + [32, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, + 32, 32, 32, 32, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, + 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, 31, 31, 31, 31, + 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, 31, 31, + 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, 31, 31, 31, 31, 31, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32], + /* Size 4x8 */ + [31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 31, 32, 32, 32, 31, 32, + 32, 32, 31, 32, 32, 32, 31, 32, 32, 32, 31, 32, 32, 32], + /* Size 8x4 */ + [31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 31, 31, + 32, 32, 32, 32, 32, 32, 31, 31, 32, 32, 32, 32, 32, 32], + /* Size 8x16 */ + [32, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 31, 31, 31, 32, + 32, 32, 32, 32, 31, 31, 32, 32, 32, 32, 32, 32, 31, 31, 32, 32, 32, 32, + 32, 32, 31, 31, 32, 32, 32, 32, 32, 32, 31, 31, 32, 32, 32, 32, 32, 32, + 31, 31, 32, 32, 32, 32, 32, 32, 31, 31, 32, 32, 32, 32, 32, 32, 31, 31, + 32, 32, 32, 32, 32, 32, 31, 31, 32, 32, 32, 32, 32, 32, 31, 31, 32, 32, + 32, 32, 32, 32, 31, 31, 32, 32, 32, 32, 32, 32, 31, 31, 32, 32, 32, 32, + 32, 32], + /* Size 16x8 */ + [32, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, 31, 31, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, 31, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32], + /* Size 16x32 */ + [32, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, + 32, 32, 32, 32, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, + 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, 31, 31, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, 31, 31, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, 31, 31, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, + 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, 31, 31, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, 31, 31, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, 31, 31, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, + 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, 31, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, 31, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, 31, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32], + /* Size 32x16 */ + [32, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, + 32, 32, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, + 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, 31, 31, 31, 31, + 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, 31, 31, + 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, 31, 31, 31, 31, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, + 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, 31, 31, 31, 31, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32], + /* Size 4x16 */ + [31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, 31, 32, + 32, 32, 31, 32, 32, 32, 31, 32, 32, 32, 31, 32, 32, 32, 31, 32, 32, 32, + 31, 32, 32, 32, 31, 32, 32, 32, 31, 32, 32, 32, 31, 32, 32, 32, 31, 32, + 32, 32, 31, 32, 32, 32, 31, 32, 32, 32], + /* Size 16x4 */ + [31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, 31, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, 31, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32], + /* Size 8x32 */ + [32, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, + 32, 32, 31, 31, 31, 32, 32, 32, 32, 32, 31, 31, 31, 32, 32, 32, 32, 32, + 31, 31, 32, 32, 32, 32, 32, 32, 31, 31, 32, 32, 32, 32, 32, 32, 31, 31, + 32, 32, 32, 32, 32, 32, 31, 31, 32, 32, 32, 32, 32, 32, 31, 31, 32, 32, + 32, 32, 32, 32, 31, 31, 32, 32, 32, 32, 32, 32, 31, 31, 32, 32, 32, 32, + 32, 32, 31, 31, 32, 32, 32, 32, 32, 32, 31, 31, 32, 32, 32, 32, 32, 32, + 31, 31, 32, 32, 32, 32, 32, 32, 31, 31, 32, 32, 32, 32, 32, 32, 31, 31, + 32, 32, 32, 32, 32, 32, 31, 31, 32, 32, 32, 32, 32, 32, 31, 31, 32, 32, + 32, 32, 32, 32, 31, 31, 32, 32, 32, 32, 32, 32, 31, 31, 32, 32, 32, 32, + 32, 32, 31, 31, 32, 32, 32, 32, 32, 32, 31, 31, 32, 32, 32, 32, 32, 32, + 31, 31, 32, 32, 32, 32, 32, 32, 31, 31, 32, 32, 32, 32, 32, 32, 31, 31, + 32, 32, 32, 32, 32, 32, 31, 31, 32, 32, 32, 32, 32, 32, 31, 31, 32, 32, + 32, 32, 32, 32], + /* Size 32x8 */ + [32, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, + 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, 31, 31, 31, 31, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32]], + [ /* Chroma */ + /* Size 4x4 */ + [31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31], + /* Size 8x8 */ + [31, 31, 31, 31, 31, 31, 31, 30, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 30, 31, 31, 31, 31, 31, 31, 31], + /* Size 16x16 */ + [32, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 30, 30, 30, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 30, 30, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 30, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 30, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 30, 30, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 30, 30, 30, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 32], + /* Size 32x32 */ + [32, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 30, 30, 30, 30, 30, 30, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 30, 30, 30, 30, 30, 30, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 30, 30, 30, 30, 30, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 30, 30, + 30, 30, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 30, 30, 30, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 30, 30, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 30, 30, 30, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 30, 30, 30, 30, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 30, 30, 30, 30, 30, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 30, 30, 30, 30, 30, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 32, 32, 30, 30, 30, 30, 30, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32], + /* Size 4x8 */ + [31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, 30, 31, 32, 32], + /* Size 8x4 */ + [31, 31, 31, 31, 31, 31, 31, 30, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 32, 32, 31, 31, 31, 31, 31, 31, 32, 32], + /* Size 8x16 */ + [32, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 31, 31, 31, 31, + 31, 32, 32, 32, 30, 31, 31, 31, 31, 32, 32, 32, 30, 31, 31, 31, 32, 32, + 32, 32], + /* Size 16x8 */ + [32, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 30, 30, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 32, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 32, 32, 32, 32, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 32, 32, 32, 32, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, + 32, 32], + /* Size 16x32 */ + [32, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 32, 32, 32, 32, 32, 32, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, + 32, 32, 32, 32, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, + 32, 32, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, + 30, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 30, 31, + 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 30, 30, 31, 31, + 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 30, 30, 31, 31, 31, 31, + 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 30, 30, 31, 31, 31, 31, 31, 31, + 32, 32, 32, 32, 32, 32, 32, 32], + /* Size 32x16 */ + [32, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 30, 30, 30, 30, 30, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 30, 30, 30, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, + 32, 32, 32, 32, 32, 32, 32, 32], + /* Size 4x16 */ + [31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 31, 31, + 32, 32, 31, 31, 32, 32, 30, 31, 32, 32], + /* Size 16x4 */ + [31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 30, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 32, 32, 32, 32], + /* Size 8x32 */ + [32, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 31, 31, 31, 31, 31, 32, + 32, 32, 31, 31, 31, 31, 31, 32, 32, 32, 31, 31, 31, 31, 31, 32, 32, 32, + 30, 31, 31, 31, 31, 32, 32, 32, 30, 31, 31, 31, 31, 32, 32, 32, 30, 31, + 31, 31, 32, 32, 32, 32, 30, 31, 31, 31, 32, 32, 32, 32, 30, 31, 31, 31, + 32, 32, 32, 32], + /* Size 32x8 */ + [32, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 30, 30, 30, 30, 30, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 32, 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, + 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 32, 32, 32, 32, 32, + 32, 32, 32, 32]] + ], + [ + [ /* Luma */ + /* Size 4x4 */ + [32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32], + /* Size 8x8 */ + [32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32], + /* Size 16x16 */ + [32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32], + /* Size 32x32 */ + [32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32], + /* Size 4x8 */ + [32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32], + /* Size 8x4 */ + [32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32], + /* Size 8x16 */ + [32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32], + /* Size 16x8 */ + [32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32], + /* Size 16x32 */ + [32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32], + /* Size 32x16 */ + [32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32], + /* Size 4x16 */ + [32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32], + /* Size 16x4 */ + [32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32], + /* Size 8x32 */ + [32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32], + /* Size 32x8 */ + [32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32]], + [ /* Chroma */ + /* Size 4x4 */ + [32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32], + /* Size 8x8 */ + [32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32], + /* Size 16x16 */ + [32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32], + /* Size 32x32 */ + [32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32], + /* Size 4x8 */ + [32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32], + /* Size 8x4 */ + [32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32], + /* Size 8x16 */ + [32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32], + /* Size 16x8 */ + [32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32], + /* Size 16x32 */ + [32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32], + /* Size 32x16 */ + [32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32], + /* Size 4x16 */ + [32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32], + /* Size 16x4 */ + [32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32], + /* Size 8x32 */ + [32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32], + /* Size 32x8 */ + [32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, + 32, 32, 32, 32]] + ] + ]; + + /// + /// Gets the inverse quantization matrix for a matrix level, color plane, and transform size. + /// + /// The quantization-matrix level. + /// The color plane; U and V select the shared chroma matrix. + /// The transform size whose raster coefficient weights are requested. + /// The inverse matrix weights in raster coefficient order. + public static ReadOnlySpan GetQuantizationMatrix(int level, Av1Plane plane, Av1TransformSize transformSize) + { + int[][][] levelMatrices = InverseWeightTable[level]; + + // The table stores one luma plane class and one shared chroma plane class. + int[][] planeMatrices = levelMatrices[Math.Min(1, (int)plane)]; + int transformMatrixIndex = TransformMatrixIndices[(int)transformSize]; + + return planeMatrices[transformMatrixIndex]; + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/Quantizers/Av1InverseQuantizer.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Quantizers/Av1InverseQuantizer.cs new file mode 100644 index 0000000000..3d8d1fa57f --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Quantizers/Av1InverseQuantizer.cs @@ -0,0 +1,137 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.Quantizers; + +/// +/// Reconstructs AV1 transform coefficients from quantized coefficient levels. +/// +internal sealed class Av1InverseQuantizer +{ + /// + /// The sequence-level color configuration that determines coefficient precision. + /// + private readonly ObuSequenceHeader sequenceHeader; + + /// + /// The frame-level segmentation and quantization configuration. + /// + private readonly ObuFrameHeader frameHeader; + + /// + /// The current per-segment, per-plane dequantization values, including any superblock delta-Q update. + /// + private Av1DeQuantizationContext deQuantsDeltaQ; + + /// + /// Initializes a new instance of the class. + /// + /// The sequence header that supplies coded bit depth and color configuration. + /// The frame header that supplies segmentation and quantization parameters. + public Av1InverseQuantizer(ObuSequenceHeader sequenceHeader, ObuFrameHeader frameHeader) + { + this.sequenceHeader = sequenceHeader; + this.frameHeader = frameHeader; + this.deQuantsDeltaQ = new(sequenceHeader, frameHeader); + } + + /// + /// Updates the active dequantization context for a superblock, applying its delta-Q value when signaled. + /// + /// The frame dequantization context to update and retain. + /// The superblock whose quantizer adjustment is applied. + public void UpdateDequant(Av1DeQuantizationContext deQuants, Av1SuperblockInfo superblockInfo) + { + Av1BitDepth bitDepth = this.sequenceHeader.ColorConfig.BitDepth; + this.deQuantsDeltaQ = deQuants; + if (this.frameHeader.DeltaQParameters.IsPresent) + { + for (int i = 0; i < Av1Constants.MaxSegmentCount; i++) + { + int currentQIndex = Av1QuantizationLookup.GetQIndex( + this.frameHeader.SegmentationParameters, + i, + superblockInfo.SuperblockQuantizerIndex); + + for (Av1Plane plane = 0; (int)plane < Av1Constants.MaxPlanes; plane++) + { + int dcDeltaQ = this.frameHeader.QuantizationParameters.DeltaQDc[(int)plane]; + int acDeltaQ = this.frameHeader.QuantizationParameters.DeltaQAc[(int)plane]; + + this.deQuantsDeltaQ.SetDc(i, plane, Av1QuantizationLookup.GetDcQuant(currentQIndex, dcDeltaQ, bitDepth)); + this.deQuantsDeltaQ.SetAc(i, plane, Av1QuantizationLookup.GetAcQuant(currentQIndex, acDeltaQ, bitDepth)); + } + } + } + } + + /// + /// Applies the active segment, plane, matrix, and transform scale to decoded coefficient magnitudes. + /// + public readonly ref struct TransformParameters + { + private readonly short dc; + private readonly short ac; + private readonly int minimum; + private readonly int maximum; + private readonly int shift; + private readonly ReadOnlySpan inverseMatrix; + + /// + /// Initializes a new instance of the struct. + /// + /// The active frame and superblock quantization values. + /// The block mode selecting the segment. + /// The transform type selecting frequency weighting. + /// The transform dimensions and coefficient scale. + /// The color plane selecting DC, AC, and matrix values. + public TransformParameters( + Av1InverseQuantizer quantizer, + Av1BlockModeInfo mode, + Av1TransformType transformType, + Av1TransformSize transformSize, + Av1Plane plane) + { + int bitCount = quantizer.sequenceHeader.ColorConfig.BitDepth.GetBitCount(); + this.minimum = -(1 << (7 + bitCount)); + this.maximum = (1 << (7 + bitCount)) - 1; + this.dc = quantizer.deQuantsDeltaQ.GetDc(mode.SegmentId, plane); + this.ac = quantizer.deQuantsDeltaQ.GetAc(mode.SegmentId, plane); + this.shift = transformSize.GetScale(); + + // Lossless segments and one-dimensional or identity transforms use the flat matrix. Matrix lookup + // happens once per transform, before the entropy loop supplies its nonzero magnitudes and signs. + int matrixLevel = quantizer.frameHeader.LosslessArray[mode.SegmentId] || + !quantizer.frameHeader.QuantizationParameters.IsUsingQMatrix || + transformType >= Av1TransformType.Identity + ? Av1ScanOrderConstants.QuantizationMatrixLevelCount - 1 + : quantizer.frameHeader.SegmentationParameters.QMLevel[(int)plane][mode.SegmentId]; + + this.inverseMatrix = Av1InverseQuantizationLookup.GetQuantizationMatrix(matrixLevel, plane, transformSize); + } + + /// + /// Dequantizes one coefficient magnitude and applies its sign and precision bounds. + /// + /// The nonnegative coefficient magnitude masked to twenty bits. + /// The coefficient's raster position. + /// Whether the decoded coefficient sign is negative. + /// The signed, scaled, and clipped transform coefficient. + public int Dequantize(int magnitude, int coefficientIndex, bool negative) + { + int dequant = coefficientIndex == 0 ? this.dc : this.ac; + + // Matrix weights have five fractional bits. Round the weighted quantizer first, then retain the + // normative 24-bit product before removing transform-size scaling. Sign and clipping follow the shift. + const int bias = 1 << (Av1Constants.QuantizationMatrixElementBitCount - 1); + dequant = ((this.inverseMatrix[coefficientIndex] * dequant) + bias) >> Av1Constants.QuantizationMatrixElementBitCount; + int coefficient = (int)(((long)magnitude * dequant) & 0xffffff) >> this.shift; + coefficient = negative ? -coefficient : coefficient; + return Av1Math.Clamp(coefficient, this.minimum, this.maximum); + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/Quantizers/Av1QuantizationLookup.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Quantizers/Av1QuantizationLookup.cs new file mode 100644 index 0000000000..ace7efab0f --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Quantizers/Av1QuantizationLookup.cs @@ -0,0 +1,287 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.Quantizers; + +/// +/// Provides the normative AV1 DC and AC dequantization values for each quantizer index and supported bit depth. +/// +internal static class Av1QuantizationLookup +{ + private const int LinearQuantizerScale = 4; + private const int LastLinearQuantizer = 61; + private const int PenultimateQuantizer = 62; + private const int PenultimateQuantizerIndex = 249; + + // AV1 transforms normally retain three fractional coefficient bits. The quantizer tables use the same Q3 + // scale, leaving coded coefficients in Q0 and reconstructed coefficients in Q3. + + // Transforms larger than 16x16 reduce coefficient scaling by one bit, and transforms larger than 32x32 reduce + // it by two bits to preserve numeric range. Quantization applies the same reduction to its step, so every + // reconstructed transform still reaches the inverse transform in Q3. + + // Encoder rate decisions intentionally retain bit-depth-specific quantizer values. The minimum table value is + // four because a smaller step would round to zero during fixed-point quantization. + + /// + /// The Q3 AC dequantization values for 8-bit samples, indexed by quantizer index. + /// + private static readonly short[] AcQlookup8 = [ + 4, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, + 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, + 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, + 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, + 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, + 102, 104, 106, 108, 110, 112, 114, 116, 118, 120, 122, 124, 126, 128, 130, 132, 134, 136, 138, + 140, 142, 144, 146, 148, 150, 152, 155, 158, 161, 164, 167, 170, 173, 176, 179, 182, 185, 188, + 191, 194, 197, 200, 203, 207, 211, 215, 219, 223, 227, 231, 235, 239, 243, 247, 251, 255, 260, + 265, 270, 275, 280, 285, 290, 295, 300, 305, 311, 317, 323, 329, 335, 341, 347, 353, 359, 366, + 373, 380, 387, 394, 401, 408, 416, 424, 432, 440, 448, 456, 465, 474, 483, 492, 501, 510, 520, + 530, 540, 550, 560, 571, 582, 593, 604, 615, 627, 639, 651, 663, 676, 689, 702, 715, 729, 743, + 757, 771, 786, 801, 816, 832, 848, 864, 881, 898, 915, 933, 951, 969, 988, 1007, 1026, 1046, 1066, + 1087, 1108, 1129, 1151, 1173, 1196, 1219, 1243, 1267, 1292, 1317, 1343, 1369, 1396, 1423, 1451, 1479, 1508, 1537, + 1567, 1597, 1628, 1660, 1692, 1725, 1759, 1793, 1828, + ]; + + /// + /// The Q3 AC dequantization values for 10-bit samples, indexed by quantizer index. + /// + private static readonly short[] AcQlookup10 = [ + 4, 9, 11, 13, 16, 18, 21, 24, 27, 30, 33, 37, 40, 44, 48, 51, 55, 59, 63, + 67, 71, 75, 79, 83, 88, 92, 96, 100, 105, 109, 114, 118, 122, 127, 131, 136, 140, 145, + 149, 154, 158, 163, 168, 172, 177, 181, 186, 190, 195, 199, 204, 208, 213, 217, 222, 226, 231, + 235, 240, 244, 249, 253, 258, 262, 267, 271, 275, 280, 284, 289, 293, 297, 302, 306, 311, 315, + 319, 324, 328, 332, 337, 341, 345, 349, 354, 358, 362, 367, 371, 375, 379, 384, 388, 392, 396, + 401, 409, 417, 425, 433, 441, 449, 458, 466, 474, 482, 490, 498, 506, 514, 523, 531, 539, 547, + 555, 563, 571, 579, 588, 596, 604, 616, 628, 640, 652, 664, 676, 688, 700, 713, 725, 737, 749, + 761, 773, 785, 797, 809, 825, 841, 857, 873, 889, 905, 922, 938, 954, 970, 986, 1002, 1018, 1038, + 1058, 1078, 1098, 1118, 1138, 1158, 1178, 1198, 1218, 1242, 1266, 1290, 1314, 1338, 1362, 1386, 1411, 1435, 1463, + 1491, 1519, 1547, 1575, 1603, 1631, 1663, 1695, 1727, 1759, 1791, 1823, 1859, 1895, 1931, 1967, 2003, 2039, 2079, + 2119, 2159, 2199, 2239, 2283, 2327, 2371, 2415, 2459, 2507, 2555, 2603, 2651, 2703, 2755, 2807, 2859, 2915, 2971, + 3027, 3083, 3143, 3203, 3263, 3327, 3391, 3455, 3523, 3591, 3659, 3731, 3803, 3876, 3952, 4028, 4104, 4184, 4264, + 4348, 4432, 4516, 4604, 4692, 4784, 4876, 4972, 5068, 5168, 5268, 5372, 5476, 5584, 5692, 5804, 5916, 6032, 6148, + 6268, 6388, 6512, 6640, 6768, 6900, 7036, 7172, 7312, + ]; + + /// + /// The Q3 AC dequantization values for 12-bit samples, indexed by quantizer index. + /// + private static readonly short[] AcQlookup12 = [ + 4, 13, 19, 27, 35, 44, 54, 64, 75, 87, 99, 112, 126, 139, 154, 168, + 183, 199, 214, 230, 247, 263, 280, 297, 314, 331, 349, 366, 384, 402, 420, 438, + 456, 475, 493, 511, 530, 548, 567, 586, 604, 623, 642, 660, 679, 698, 716, 735, + 753, 772, 791, 809, 828, 846, 865, 884, 902, 920, 939, 957, 976, 994, 1012, 1030, + 1049, 1067, 1085, 1103, 1121, 1139, 1157, 1175, 1193, 1211, 1229, 1246, 1264, 1282, 1299, 1317, + 1335, 1352, 1370, 1387, 1405, 1422, 1440, 1457, 1474, 1491, 1509, 1526, 1543, 1560, 1577, 1595, + 1627, 1660, 1693, 1725, 1758, 1791, 1824, 1856, 1889, 1922, 1954, 1987, 2020, 2052, 2085, 2118, + 2150, 2183, 2216, 2248, 2281, 2313, 2346, 2378, 2411, 2459, 2508, 2556, 2605, 2653, 2701, 2750, + 2798, 2847, 2895, 2943, 2992, 3040, 3088, 3137, 3185, 3234, 3298, 3362, 3426, 3491, 3555, 3619, + 3684, 3748, 3812, 3876, 3941, 4005, 4069, 4149, 4230, 4310, 4390, 4470, 4550, 4631, 4711, 4791, + 4871, 4967, 5064, 5160, 5256, 5352, 5448, 5544, 5641, 5737, 5849, 5961, 6073, 6185, 6297, 6410, + 6522, 6650, 6778, 6906, 7034, 7162, 7290, 7435, 7579, 7723, 7867, 8011, 8155, 8315, 8475, 8635, + 8795, 8956, 9132, 9308, 9484, 9660, 9836, 10028, 10220, 10412, 10604, 10812, 11020, 11228, 11437, 11661, + 11885, 12109, 12333, 12573, 12813, 13053, 13309, 13565, 13821, 14093, 14365, 14637, 14925, 15213, 15502, 15806, + 16110, 16414, 16734, 17054, 17390, 17726, 18062, 18414, 18766, 19134, 19502, 19886, 20270, 20670, 21070, 21486, + 21902, 22334, 22766, 23214, 23662, 24126, 24590, 25070, 25551, 26047, 26559, 27071, 27599, 28143, 28687, 29247, + ]; + + /// + /// The Q3 DC dequantization values for 8-bit samples, indexed by quantizer index. + /// + private static readonly short[] DcQlookup8 = [ + 4, 8, 8, 9, 10, 11, 12, 12, 13, 14, 15, 16, 17, 18, 19, 19, 20, 21, 22, 23, + 24, 25, 26, 26, 27, 28, 29, 30, 31, 32, 32, 33, 34, 35, 36, 37, 38, 38, 39, 40, + 41, 42, 43, 43, 44, 45, 46, 47, 48, 48, 49, 50, 51, 52, 53, 53, 54, 55, 56, 57, + 57, 58, 59, 60, 61, 62, 62, 63, 64, 65, 66, 66, 67, 68, 69, 70, 70, 71, 72, 73, + 74, 74, 75, 76, 77, 78, 78, 79, 80, 81, 81, 82, 83, 84, 85, 85, 87, 88, 90, 92, + 93, 95, 96, 98, 99, 101, 102, 104, 105, 107, 108, 110, 111, 113, 114, 116, 117, 118, 120, 121, + 123, 125, 127, 129, 131, 134, 136, 138, 140, 142, 144, 146, 148, 150, 152, 154, 156, 158, 161, 164, + 166, 169, 172, 174, 177, 180, 182, 185, 187, 190, 192, 195, 199, 202, 205, 208, 211, 214, 217, 220, + 223, 226, 230, 233, 237, 240, 243, 247, 250, 253, 257, 261, 265, 269, 272, 276, 280, 284, 288, 292, + 296, 300, 304, 309, 313, 317, 322, 326, 330, 335, 340, 344, 349, 354, 359, 364, 369, 374, 379, 384, + 389, 395, 400, 406, 411, 417, 423, 429, 435, 441, 447, 454, 461, 467, 475, 482, 489, 497, 505, 513, + 522, 530, 539, 549, 559, 569, 579, 590, 602, 614, 626, 640, 654, 668, 684, 700, 717, 736, 755, 775, + 796, 819, 843, 869, 896, 925, 955, 988, 1022, 1058, 1098, 1139, 1184, 1232, 1282, 1336, + ]; + + /// + /// The Q3 DC dequantization values for 10-bit samples, indexed by quantizer index. + /// + private static readonly short[] DcQlookup10 = [ + 4, 9, 10, 13, 15, 17, 20, 22, 25, 28, 31, 34, 37, 40, 43, 47, 50, 53, 57, + 60, 64, 68, 71, 75, 78, 82, 86, 90, 93, 97, 101, 105, 109, 113, 116, 120, 124, 128, + 132, 136, 140, 143, 147, 151, 155, 159, 163, 166, 170, 174, 178, 182, 185, 189, 193, 197, 200, + 204, 208, 212, 215, 219, 223, 226, 230, 233, 237, 241, 244, 248, 251, 255, 259, 262, 266, 269, + 273, 276, 280, 283, 287, 290, 293, 297, 300, 304, 307, 310, 314, 317, 321, 324, 327, 331, 334, + 337, 343, 350, 356, 362, 369, 375, 381, 387, 394, 400, 406, 412, 418, 424, 430, 436, 442, 448, + 454, 460, 466, 472, 478, 484, 490, 499, 507, 516, 525, 533, 542, 550, 559, 567, 576, 584, 592, + 601, 609, 617, 625, 634, 644, 655, 666, 676, 687, 698, 708, 718, 729, 739, 749, 759, 770, 782, + 795, 807, 819, 831, 844, 856, 868, 880, 891, 906, 920, 933, 947, 961, 975, 988, 1001, 1015, 1030, + 1045, 1061, 1076, 1090, 1105, 1120, 1137, 1153, 1170, 1186, 1202, 1218, 1236, 1253, 1271, 1288, 1306, 1323, 1342, + 1361, 1379, 1398, 1416, 1436, 1456, 1476, 1496, 1516, 1537, 1559, 1580, 1601, 1624, 1647, 1670, 1692, 1717, 1741, + 1766, 1791, 1817, 1844, 1871, 1900, 1929, 1958, 1990, 2021, 2054, 2088, 2123, 2159, 2197, 2236, 2276, 2319, 2363, + 2410, 2458, 2508, 2561, 2616, 2675, 2737, 2802, 2871, 2944, 3020, 3102, 3188, 3280, 3375, 3478, 3586, 3702, 3823, + 3953, 4089, 4236, 4394, 4559, 4737, 4929, 5130, 5347, + ]; + + /// + /// The Q3 DC dequantization values for 12-bit samples, indexed by quantizer index. + /// + private static readonly short[] DcQlookup12 = [ + 4, 12, 18, 25, 33, 41, 50, 60, 70, 80, 91, 103, 115, 127, 140, 153, + 166, 180, 194, 208, 222, 237, 251, 266, 281, 296, 312, 327, 343, 358, 374, 390, + 405, 421, 437, 453, 469, 484, 500, 516, 532, 548, 564, 580, 596, 611, 627, 643, + 659, 674, 690, 706, 721, 737, 752, 768, 783, 798, 814, 829, 844, 859, 874, 889, + 904, 919, 934, 949, 964, 978, 993, 1008, 1022, 1037, 1051, 1065, 1080, 1094, 1108, 1122, + 1136, 1151, 1165, 1179, 1192, 1206, 1220, 1234, 1248, 1261, 1275, 1288, 1302, 1315, 1329, 1342, + 1368, 1393, 1419, 1444, 1469, 1494, 1519, 1544, 1569, 1594, 1618, 1643, 1668, 1692, 1717, 1741, + 1765, 1789, 1814, 1838, 1862, 1885, 1909, 1933, 1957, 1992, 2027, 2061, 2096, 2130, 2165, 2199, + 2233, 2267, 2300, 2334, 2367, 2400, 2434, 2467, 2499, 2532, 2575, 2618, 2661, 2704, 2746, 2788, + 2830, 2872, 2913, 2954, 2995, 3036, 3076, 3127, 3177, 3226, 3275, 3324, 3373, 3421, 3469, 3517, + 3565, 3621, 3677, 3733, 3788, 3843, 3897, 3951, 4005, 4058, 4119, 4181, 4241, 4301, 4361, 4420, + 4479, 4546, 4612, 4677, 4742, 4807, 4871, 4942, 5013, 5083, 5153, 5222, 5291, 5367, 5442, 5517, + 5591, 5665, 5745, 5825, 5905, 5984, 6063, 6149, 6234, 6319, 6404, 6495, 6587, 6678, 6769, 6867, + 6966, 7064, 7163, 7269, 7376, 7483, 7599, 7715, 7832, 7958, 8085, 8214, 8352, 8492, 8635, 8788, + 8945, 9104, 9275, 9450, 9639, 9832, 10031, 10245, 10465, 10702, 10946, 11210, 11482, 11776, 12081, 12409, + 12750, 13118, 13501, 13913, 14343, 14807, 15290, 15812, 16356, 16943, 17575, 18237, 18949, 19718, 20521, 21387, + ]; + + /// + /// Converts a quantizer on libaom's external zero-through-63 scale to an AV1 quantizer index. + /// + /// The external quantizer. + /// The corresponding AV1 quantizer index. + public static int GetQIndex(int quantizer) + { + // Four qindex steps separate the regular entries. The final two entries use 249 and 255 so the external + // scale reaches AV1's complete qindex range without changing the spacing of its first 62 entries. + if (quantizer <= LastLinearQuantizer) + { + return quantizer * LinearQuantizerScale; + } + + return quantizer == PenultimateQuantizer ? PenultimateQuantizerIndex : Av1Constants.MaxQ; + } + + /// + /// Gets the DC dequantization value after applying a plane delta to the frame quantizer index. + /// + /// The frame or segment quantizer index. + /// The signed DC quantizer adjustment for the selected plane. + /// The coded sample bit depth. + /// The Q3 DC dequantization value. + public static short GetDcQuant(int qIndex, int dcDeltaQ, Av1BitDepth bitDepth) + { + // Plane deltas may move beyond the signaled 8-bit quantizer domain, where AV1 requires endpoint clamping. + int qClamped = Av1Math.Clamp(qIndex + dcDeltaQ, 0, Av1Constants.MaxQ); + switch (bitDepth) + { + case Av1BitDepth.EightBit: + return DcQlookup8[qClamped]; + case Av1BitDepth.TenBit: + return DcQlookup10[qClamped]; + case Av1BitDepth.TwelveBit: + return DcQlookup12[qClamped]; + default: + Guard.IsFalse(true, nameof(bitDepth), "bit_depth should be EB_EIGHT_BIT, EB_TEN_BIT or EB_TWELVE_BIT"); + return -1; + } + } + + /// + /// Gets the AC dequantization value after applying a plane delta to the frame quantizer index. + /// + /// The frame or segment quantizer index. + /// The signed AC quantizer adjustment for the selected plane. + /// The coded sample bit depth. + /// The Q3 AC dequantization value. + public static short GetAcQuant(int qIndex, int dcDeltaQ, Av1BitDepth bitDepth) + { + // Plane deltas may move beyond the signaled 8-bit quantizer domain, where AV1 requires endpoint clamping. + int qClamped = Av1Math.Clamp(qIndex + dcDeltaQ, 0, Av1Constants.MaxQ); + switch (bitDepth) + { + case Av1BitDepth.EightBit: + return AcQlookup8[qClamped]; + case Av1BitDepth.TenBit: + return AcQlookup10[qClamped]; + case Av1BitDepth.TwelveBit: + return AcQlookup12[qClamped]; + default: + Guard.IsFalse(true, nameof(bitDepth), "bit_depth should be EB_EIGHT_BIT, EB_TEN_BIT or EB_TWELVE_BIT"); + return -1; + } + } + + /// + /// Gets the quantizer index for a segment, including its alternative-quantizer feature when active. + /// + /// The frame segmentation configuration. + /// The zero-based AV1 segment identifier. + /// The frame's base quantizer index. + /// The segment quantizer index clamped to the AV1 quantizer domain. + public static int GetQIndex(ObuSegmentationParameters segmentationParameters, int segmentId, int baseQIndex) + { + if (segmentationParameters.IsFeatureActive(segmentId, ObuSegmentationLevelFeature.AlternativeQuantizer)) + { + int data = segmentationParameters.GetFeatureData(segmentId, (int)ObuSegmentationLevelFeature.AlternativeQuantizer); + int qIndex = baseQIndex + data; + return Av1Math.Clamp(qIndex, 0, Av1Constants.MaxQ); + } + else + { + return baseQIndex; + } + } + + /// + /// Derives the per-segment quantizer indices, lossless flags, and quantization-matrix levels for a frame. + /// + /// The frame header whose quantization state is updated. + public static void UpdateFrameQuantizationState(ObuFrameHeader frameHeader) + { + ObuQuantizationParameters quantization = frameHeader.QuantizationParameters; + ObuSegmentationParameters segmentation = frameHeader.SegmentationParameters; + frameHeader.CodedLossless = true; + for (int segmentId = 0; segmentId < Av1Constants.MaxSegmentCount; segmentId++) + { + int qIndex = GetQIndex(segmentation, segmentId, quantization.BaseQIndex); + quantization.QIndex[segmentId] = qIndex; + frameHeader.LosslessArray[segmentId] = qIndex == 0 && + quantization.DeltaQDc[(int)Av1Plane.Y] == 0 && + quantization.DeltaQAc[(int)Av1Plane.U] == 0 && + quantization.DeltaQDc[(int)Av1Plane.U] == 0 && + quantization.DeltaQAc[(int)Av1Plane.V] == 0 && + quantization.DeltaQDc[(int)Av1Plane.V] == 0; + + if (!frameHeader.LosslessArray[segmentId]) + { + frameHeader.CodedLossless = false; + } + + if (quantization.IsUsingQMatrix) + { + // Lossless segments use the identity matrix level; lossy segments inherit the + // plane-specific levels signaled by the frame quantization parameters. + segmentation.QMLevel[(int)Av1Plane.Y][segmentId] = frameHeader.LosslessArray[segmentId] + ? 15 + : quantization.QMatrix[(int)Av1Plane.Y]; + + segmentation.QMLevel[(int)Av1Plane.U][segmentId] = frameHeader.LosslessArray[segmentId] + ? 15 + : quantization.QMatrix[(int)Av1Plane.U]; + + segmentation.QMLevel[(int)Av1Plane.V][segmentId] = frameHeader.LosslessArray[segmentId] + ? 15 + : quantization.QMatrix[(int)Av1Plane.V]; + } + } + + frameHeader.AllLossless = frameHeader.CodedLossless && + frameHeader.FrameSize.FrameWidth == frameHeader.FrameSize.SuperResolutionUpscaledWidth; + + // Header syntax and tile quantization now observe the same derived frame state. + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/SuperResolution/Av1SuperResolutionDecoder.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/SuperResolution/Av1SuperResolutionDecoder.cs new file mode 100644 index 0000000000..aa80a8011d --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/SuperResolution/Av1SuperResolutionDecoder.cs @@ -0,0 +1,164 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Buffers; +using System.Runtime.InteropServices; +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.SuperResolution; + +/// +/// Applies normative AV1 horizontal super-resolution upscaling to a reconstructed still-image frame. +/// +internal sealed class Av1SuperResolutionDecoder +{ + /// + /// The sequence-level bit-depth and color-plane configuration. + /// + private readonly ObuSequenceHeader sequenceHeader; + + /// + /// The coded and upscaled frame dimensions. + /// + private readonly ObuFrameHeader frameHeader; + + /// + /// The reconstructed planes updated with the upscaled samples. + /// + private readonly Av1FrameBuffer frameBuffer; + + /// + /// Initializes a new instance of the class. + /// + /// The sequence header defining bit depth and color layout. + /// The frame header defining coded and upscaled dimensions. + /// The CDEF-filtered frame samples to upscale. + public Av1SuperResolutionDecoder( + ObuSequenceHeader sequenceHeader, + ObuFrameHeader frameHeader, + Av1FrameBuffer frameBuffer) + { + this.sequenceHeader = sequenceHeader; + this.frameHeader = frameHeader; + this.frameBuffer = frameBuffer; + } + + /// + /// Upscales each color plane horizontally and updates the visible frame dimensions. + /// + public void DecodeFrame() + { + ObuFrameSize frameSize = this.frameHeader.FrameSize; + int codedWidth = frameSize.FrameWidth; + int upscaledWidth = frameSize.SuperResolutionUpscaledWidth; + if (codedWidth != upscaledWidth) + { + int outputLength = upscaledWidth * this.frameBuffer.BytesPerSample; + using IMemoryOwner outputOwner = this.frameBuffer.MemoryAllocator.Allocate(outputLength); + Span output = outputOwner.Memory.Span[..outputLength]; + ObuColorConfig colorConfig = this.sequenceHeader.ColorConfig; + for (int planeIndex = 0; planeIndex < colorConfig.PlaneCount; planeIndex++) + { + Av1Plane plane = (Av1Plane)planeIndex; + int subsamplingX = plane != Av1Plane.Y && colorConfig.SubSamplingX ? 1 : 0; + int subsamplingY = plane != Av1Plane.Y && colorConfig.SubSamplingY ? 1 : 0; + this.UpscalePlane(plane, subsamplingX, subsamplingY, codedWidth, upscaledWidth, frameSize.FrameHeight, output); + } + } + + // Plane-region consumers must observe the reconstructed frame rather than the sequence maxima + // used to size the reusable backing allocations. + this.frameBuffer.Width = upscaledWidth; + this.frameBuffer.Height = frameSize.FrameHeight; + } + + /// + /// Upscales every row of one color plane with the normative fixed filter. + /// + /// The color plane to upscale. + /// The horizontal chroma subsampling shift. + /// The vertical chroma subsampling shift. + /// The coded luma width before upscaling. + /// The luma width after upscaling. + /// The unchanged luma height. + /// The allocator-owned row used to prevent source and destination overlap. + private void UpscalePlane( + Av1Plane plane, + int subsamplingX, + int subsamplingY, + int codedLumaWidth, + int upscaledLumaWidth, + int lumaHeight, + Span outputBuffer) + { + int codedWidth = Av1Math.DivideLog2Ceiling(codedLumaWidth, subsamplingX); + int upscaledWidth = Av1Math.DivideLog2Ceiling(upscaledLumaWidth, subsamplingX); + int reconstructedWidth = this.frameHeader.ModeInfoColumnCount << (Av1Constants.ModeInfoSizeLog2 - subsamplingX); + int height = Av1Math.DivideLog2Ceiling(lumaHeight, subsamplingY); + int step = Av1SuperResolutionFilter.GetConvolveStep(codedWidth, upscaledWidth); + int initialSubpixel = Av1SuperResolutionFilter.GetInitialSubpixel(codedWidth, upscaledWidth, step); + + Span lowBitDepthPlane = default; + Span highBitDepthPlane = default; + int stride; + if (this.frameBuffer.BytesPerSample == 2) + { + Span signedPlane = this.frameBuffer.DeriveBlockPointer16( + plane, + new Point(-Av1SuperResolutionFilter.SourceBorder, 0), + subsamplingX, + subsamplingY, + out stride); + + highBitDepthPlane = MemoryMarshal.Cast(signedPlane); + } + else + { + lowBitDepthPlane = this.frameBuffer.DeriveBlockPointer( + plane, + new Point(-Av1SuperResolutionFilter.SourceBorder, 0), + subsamplingX, + subsamplingY, + out stride); + } + + int sourceStart = Av1SuperResolutionFilter.SourceBorder; + int bitDepth = this.frameBuffer.BitDepth.GetBitCount(); + + // the reference decoder partitions the same continuous phase progression by tile column but does not pad + // internal boundaries. Filtering the complete row therefore produces the identical samples. + for (int row = 0; row < height; row++) + { + int planeOffset = stride + (row * stride); + if (this.frameBuffer.BytesPerSample == 2) + { + Span sourceRow = highBitDepthPlane.Slice(planeOffset, reconstructedWidth + (sourceStart * 2)); + Span reconstructedSamples = sourceRow.Slice(sourceStart, reconstructedWidth); + Span destinationSamples = highBitDepthPlane.Slice(planeOffset + sourceStart, upscaledWidth); + Span outputRow = MemoryMarshal.Cast(outputBuffer)[..upscaledWidth]; + + // The frame allocation already reserves decoder padding. Replicating its edge samples in place avoids + // copying the reconstructed row into a second working buffer before every filter pass. + sourceRow[..sourceStart].Fill(reconstructedSamples[0]); + sourceRow[(sourceStart + reconstructedWidth)..].Fill(reconstructedSamples[^1]); + Av1SuperResolutionFilter.UpscaleRow(sourceRow, outputRow, step, initialSubpixel, bitDepth); + + // Horizontal expansion cannot write directly over its input because later taps still address samples + // to the right. The single pooled row is copied back only after all of its source samples are consumed. + outputRow.CopyTo(destinationSamples); + } + else + { + Span sourceRow = lowBitDepthPlane.Slice(planeOffset, reconstructedWidth + (sourceStart * 2)); + Span reconstructedSamples = sourceRow.Slice(sourceStart, reconstructedWidth); + Span destinationSamples = lowBitDepthPlane.Slice(planeOffset + sourceStart, upscaledWidth); + Span outputRow = outputBuffer[..upscaledWidth]; + + sourceRow[..sourceStart].Fill(reconstructedSamples[0]); + sourceRow[(sourceStart + reconstructedWidth)..].Fill(reconstructedSamples[^1]); + Av1SuperResolutionFilter.UpscaleRow(sourceRow, outputRow, step, initialSubpixel); + outputRow.CopyTo(destinationSamples); + } + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/SuperResolution/Av1SuperResolutionFilter.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/SuperResolution/Av1SuperResolutionFilter.cs new file mode 100644 index 0000000000..8ca598c850 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/SuperResolution/Av1SuperResolutionFilter.cs @@ -0,0 +1,430 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.InteropServices; +using System.Runtime.Intrinsics; +using System.Runtime.Intrinsics.Arm; +using System.Runtime.Intrinsics.X86; +using SixLabors.ImageSharp.Common.Helpers; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.SuperResolution; + +/// +/// Applies the normative horizontal filter used by AV1 super-resolution upscaling. +/// +/// +/// Four output coordinates are evaluated together, but each coordinate starts as its own eight-lane vector of source +/// taps. Pairwise multiply-add produces four partial sums per output, and two horizontal reductions transpose those +/// four independent dot products into consecutive output lanes. This layout supports arbitrary fixed-point source +/// steps and filter phases without gathers or a temporary coefficient matrix. +/// +internal static class Av1SuperResolutionFilter +{ + /// + /// The number of source samples consumed for each output sample. + /// + public const int TapCount = 8; + + /// + /// The number of replicated source samples required at each row edge. + /// + public const int SourceBorder = (TapCount / 2) + 1; + + /// + /// The number of output samples evaluated together by the vector path. + /// + private const int OutputGroupSize = 4; + + /// + /// The number of fractional bits represented by each filter coefficient. + /// + private const int FilterBits = 7; + + /// + /// The rounding offset applied before removing the filter-coefficient fractional bits. + /// + private const int FilterRounding = 1 << (FilterBits - 1); + + /// + /// The number of filter phases in the normative table, expressed as a base-two exponent. + /// + private const int SubpixelBits = 6; + + /// + /// The number of fractional bits used by the horizontal sample position. + /// + private const int ScaleSubpixelBits = 14; + + /// + /// The mask selecting the fractional sample position. + /// + private const int ScaleSubpixelMask = (1 << ScaleSubpixelBits) - 1; + + /// + /// The number of low position bits omitted when selecting a filter phase. + /// + private const int ScaleExtraBits = ScaleSubpixelBits - SubpixelBits; + + /// + /// The half-step bias applied before filter-phase selection. + /// + private const int ScaleExtraOffset = 1 << (ScaleExtraBits - 1); + + /// + /// Gets the flattened 64-phase, 8-tap normative AV1 super-resolution filter table. + /// + private static ReadOnlySpan Filters => + [ + 0, 0, 0, 128, 0, 0, 0, 0, 0, 0, -1, 128, 2, -1, 0, 0, + 0, 1, -3, 127, 4, -2, 1, 0, 0, 1, -4, 127, 6, -3, 1, 0, + 0, 2, -6, 126, 8, -3, 1, 0, 0, 2, -7, 125, 11, -4, 1, 0, + -1, 2, -8, 125, 13, -5, 2, 0, -1, 3, -9, 124, 15, -6, 2, 0, + -1, 3, -10, 123, 18, -6, 2, -1, -1, 3, -11, 122, 20, -7, 3, -1, + -1, 4, -12, 121, 22, -8, 3, -1, -1, 4, -13, 120, 25, -9, 3, -1, + -1, 4, -14, 118, 28, -9, 3, -1, -1, 4, -15, 117, 30, -10, 4, -1, + -1, 5, -16, 116, 32, -11, 4, -1, -1, 5, -16, 114, 35, -12, 4, -1, + -1, 5, -17, 112, 38, -12, 4, -1, -1, 5, -18, 111, 40, -13, 5, -1, + -1, 5, -18, 109, 43, -14, 5, -1, -1, 6, -19, 107, 45, -14, 5, -1, + -1, 6, -19, 105, 48, -15, 5, -1, -1, 6, -19, 103, 51, -16, 5, -1, + -1, 6, -20, 101, 53, -16, 6, -1, -1, 6, -20, 99, 56, -17, 6, -1, + -1, 6, -20, 97, 58, -17, 6, -1, -1, 6, -20, 95, 61, -18, 6, -1, + -2, 7, -20, 93, 64, -18, 6, -2, -2, 7, -20, 91, 66, -19, 6, -1, + -2, 7, -20, 88, 69, -19, 6, -1, -2, 7, -20, 86, 71, -19, 6, -1, + -2, 7, -20, 84, 74, -20, 7, -2, -2, 7, -20, 81, 76, -20, 7, -1, + -2, 7, -20, 79, 79, -20, 7, -2, -1, 7, -20, 76, 81, -20, 7, -2, + -2, 7, -20, 74, 84, -20, 7, -2, -1, 6, -19, 71, 86, -20, 7, -2, + -1, 6, -19, 69, 88, -20, 7, -2, -1, 6, -19, 66, 91, -20, 7, -2, + -2, 6, -18, 64, 93, -20, 7, -2, -1, 6, -18, 61, 95, -20, 6, -1, + -1, 6, -17, 58, 97, -20, 6, -1, -1, 6, -17, 56, 99, -20, 6, -1, + -1, 6, -16, 53, 101, -20, 6, -1, -1, 5, -16, 51, 103, -19, 6, -1, + -1, 5, -15, 48, 105, -19, 6, -1, -1, 5, -14, 45, 107, -19, 6, -1, + -1, 5, -14, 43, 109, -18, 5, -1, -1, 5, -13, 40, 111, -18, 5, -1, + -1, 4, -12, 38, 112, -17, 5, -1, -1, 4, -12, 35, 114, -16, 5, -1, + -1, 4, -11, 32, 116, -16, 5, -1, -1, 4, -10, 30, 117, -15, 4, -1, + -1, 3, -9, 28, 118, -14, 4, -1, -1, 3, -9, 25, 120, -13, 4, -1, + -1, 3, -8, 22, 121, -12, 4, -1, -1, 3, -7, 20, 122, -11, 3, -1, + -1, 2, -6, 18, 123, -10, 3, -1, 0, 2, -6, 15, 124, -9, 3, -1, + 0, 2, -5, 13, 125, -8, 2, -1, 0, 1, -4, 11, 125, -7, 2, 0, + 0, 1, -3, 8, 126, -6, 2, 0, 0, 1, -3, 6, 127, -4, 1, 0, + 0, 1, -2, 4, 127, -3, 1, 0, 0, 0, -1, 2, 128, -1, 0, 0 + ]; + + /// + /// Derives the fixed-point source-position increment for an output row. + /// + /// The coded plane width. + /// The upscaled plane width. + /// The source-position increment with 14 fractional bits. + public static int GetConvolveStep(int inputLength, int outputLength) + => ((inputLength << ScaleSubpixelBits) + (outputLength / 2)) / outputLength; + + /// + /// Derives the initial fixed-point source position for an output row. + /// + /// The coded plane width. + /// The upscaled plane width. + /// The source-position increment returned by . + /// The initial source position with 14 fractional bits. + public static int GetInitialSubpixel(int inputLength, int outputLength, int step) + { + int error = (outputLength * step) - (inputLength << ScaleSubpixelBits); + int initial = (-((outputLength - inputLength) << (ScaleSubpixelBits - 1)) + (outputLength / 2)) / outputLength; + + initial += ScaleExtraOffset - (error / 2); + return initial & ScaleSubpixelMask; + } + + /// + /// Upscales one replicated-edge eight-bit source row into eight-bit output samples. + /// + /// The coded row with replicated samples on each edge. + /// The upscaled destination row. + /// The fixed-point source-position increment. + /// The initial fixed-point source position. + public static void UpscaleRow(ReadOnlySpan source, Span destination, int step, int initialSubpixel) + { + ref byte sourceBase = ref MemoryMarshal.GetReference(source); + ref byte destinationBase = ref MemoryMarshal.GetReference(destination); + int sourcePosition = initialSubpixel; + int column = 0; + if (Vector128.IsHardwareAccelerated) + { + for (; column <= destination.Length - OutputGroupSize; column += OutputGroupSize) + { + Vector128 filtered = FilterFour(ref sourceBase, sourcePosition, step); + Vector128 samples16 = Vector128_.PackUnsignedSaturate(filtered, Vector128.Zero); + Vector128 samples8 = Vector128_.PackUnsignedSaturate(samples16.AsInt16(), Vector128.Zero); + + Unsafe.WriteUnaligned(ref Unsafe.Add(ref destinationBase, column), samples8.AsUInt32().GetElement(0)); + sourcePosition += step * OutputGroupSize; + } + } + + for (; column < destination.Length; column++) + { + Unsafe.Add(ref destinationBase, column) = (byte)FilterOne(ref sourceBase, sourcePosition, byte.MaxValue); + sourcePosition += step; + } + } + + /// + /// Upscales one replicated-edge eight-bit source row into 16-bit output storage. + /// + /// The coded row with replicated samples on each edge. + /// The upscaled destination row. + /// The fixed-point source-position increment. + /// The initial fixed-point source position. + public static void UpscaleRow(ReadOnlySpan source, Span destination, int step, int initialSubpixel) + { + ref byte sourceBase = ref MemoryMarshal.GetReference(source); + ref ushort destinationBase = ref MemoryMarshal.GetReference(destination); + int sourcePosition = initialSubpixel; + int column = 0; + if (Vector128.IsHardwareAccelerated) + { + for (; column <= destination.Length - OutputGroupSize; column += OutputGroupSize) + { + Vector128 filtered = FilterFour(ref sourceBase, sourcePosition, step); + Vector128 samples = Vector128_.PackUnsignedSaturate(filtered, Vector128.Zero); + + samples.GetLower().StoreUnsafe(ref destinationBase, (nuint)column); + sourcePosition += step * OutputGroupSize; + } + } + + for (; column < destination.Length; column++) + { + Unsafe.Add(ref destinationBase, column) = (ushort)FilterOne(ref sourceBase, sourcePosition, byte.MaxValue); + sourcePosition += step; + } + } + + /// + /// Upscales one replicated-edge high-bit-depth source row into 16-bit output storage. + /// + /// The coded row with replicated samples on each edge. + /// The upscaled destination row. + /// The fixed-point source-position increment. + /// The initial fixed-point source position. + /// The encoded sample bit depth. + public static void UpscaleRow(ReadOnlySpan source, Span destination, int step, int initialSubpixel, int bitDepth) + { + ref ushort sourceBase = ref MemoryMarshal.GetReference(source); + ref ushort destinationBase = ref MemoryMarshal.GetReference(destination); + int maximum = (1 << bitDepth) - 1; + int sourcePosition = initialSubpixel; + int column = 0; + if (Vector128.IsHardwareAccelerated) + { + Vector128 maximumVector = Vector128.Create((ushort)maximum); + for (; column <= destination.Length - OutputGroupSize; column += OutputGroupSize) + { + Vector128 filtered = FilterFour(ref sourceBase, sourcePosition, step); + Vector128 samples = Vector128.Min(Vector128_.PackUnsignedSaturate(filtered, Vector128.Zero), maximumVector); + + samples.GetLower().StoreUnsafe(ref destinationBase, (nuint)column); + sourcePosition += step * OutputGroupSize; + } + } + + for (; column < destination.Length; column++) + { + Unsafe.Add(ref destinationBase, column) = (ushort)FilterOne(ref sourceBase, sourcePosition, maximum); + sourcePosition += step; + } + } + + /// + /// Evaluates four consecutive output positions from an eight-bit source row. + /// + /// The first sample in the replicated-edge source row. + /// The first fixed-point source position. + /// The fixed-point increment between output samples. + /// The rounded filter results in output order. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector128 FilterFour(ref byte source, int sourcePosition, int step) + { + GetOffsets(sourcePosition, out int sourceOffset0, out int filterOffset0); + GetOffsets(sourcePosition + step, out int sourceOffset1, out int filterOffset1); + GetOffsets(sourcePosition + (step * 2), out int sourceOffset2, out int filterOffset2); + GetOffsets(sourcePosition + (step * 3), out int sourceOffset3, out int filterOffset3); + + ref short filter = ref MemoryMarshal.GetReference(Filters); + + // Replicated edge storage guarantees all eight taps are contiguous even for the first and last output. Exact + // 64-bit reads avoid depending on additional row padding before widening each tap set to signed Int16 lanes. + Vector128 samples0 = Vector128.WidenLower(Vector128.CreateScalarUnsafe(Unsafe.ReadUnaligned(ref Unsafe.Add(ref source, sourceOffset0))).AsByte()).AsInt16(); + Vector128 samples1 = Vector128.WidenLower(Vector128.CreateScalarUnsafe(Unsafe.ReadUnaligned(ref Unsafe.Add(ref source, sourceOffset1))).AsByte()).AsInt16(); + Vector128 samples2 = Vector128.WidenLower(Vector128.CreateScalarUnsafe(Unsafe.ReadUnaligned(ref Unsafe.Add(ref source, sourceOffset2))).AsByte()).AsInt16(); + Vector128 samples3 = Vector128.WidenLower(Vector128.CreateScalarUnsafe(Unsafe.ReadUnaligned(ref Unsafe.Add(ref source, sourceOffset3))).AsByte()).AsInt16(); + return FilterFour( + samples0, + samples1, + samples2, + samples3, + Vector128.LoadUnsafe(ref filter, (nuint)filterOffset0), + Vector128.LoadUnsafe(ref filter, (nuint)filterOffset1), + Vector128.LoadUnsafe(ref filter, (nuint)filterOffset2), + Vector128.LoadUnsafe(ref filter, (nuint)filterOffset3)); + } + + /// + /// Evaluates four consecutive output positions from a high-bit-depth source row. + /// + /// The first sample in the replicated-edge source row. + /// The first fixed-point source position. + /// The fixed-point increment between output samples. + /// The rounded filter results in output order. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector128 FilterFour(ref ushort source, int sourcePosition, int step) + { + GetOffsets(sourcePosition, out int sourceOffset0, out int filterOffset0); + GetOffsets(sourcePosition + step, out int sourceOffset1, out int filterOffset1); + GetOffsets(sourcePosition + (step * 2), out int sourceOffset2, out int filterOffset2); + GetOffsets(sourcePosition + (step * 3), out int sourceOffset3, out int filterOffset3); + + ref short filter = ref MemoryMarshal.GetReference(Filters); + + // AV1 high-bit-depth samples are at most twelve bits, so their signed Int16 view remains positive. Keeping the + // tap and coefficient types equal enables the same pairwise multiply-add reduction as the eight-bit path. + return FilterFour( + Vector128.LoadUnsafe(ref source, (nuint)sourceOffset0).AsInt16(), + Vector128.LoadUnsafe(ref source, (nuint)sourceOffset1).AsInt16(), + Vector128.LoadUnsafe(ref source, (nuint)sourceOffset2).AsInt16(), + Vector128.LoadUnsafe(ref source, (nuint)sourceOffset3).AsInt16(), + Vector128.LoadUnsafe(ref filter, (nuint)filterOffset0), + Vector128.LoadUnsafe(ref filter, (nuint)filterOffset1), + Vector128.LoadUnsafe(ref filter, (nuint)filterOffset2), + Vector128.LoadUnsafe(ref filter, (nuint)filterOffset3)); + } + + /// + /// Multiplies and reduces four independent eight-tap filter inputs. + /// + /// The source samples for the first output. + /// The source samples for the second output. + /// The source samples for the third output. + /// The source samples for the fourth output. + /// The coefficients for the first output. + /// The coefficients for the second output. + /// The coefficients for the third output. + /// The coefficients for the fourth output. + /// The rounded filter results in output order. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector128 FilterFour( + Vector128 samples0, + Vector128 samples1, + Vector128 samples2, + Vector128 samples3, + Vector128 filter0, + Vector128 filter1, + Vector128 filter2, + Vector128 filter3) + { + Vector128 products0 = Vector128_.MultiplyAddAdjacent(samples0, filter0); + Vector128 products1 = Vector128_.MultiplyAddAdjacent(samples1, filter1); + Vector128 products2 = Vector128_.MultiplyAddAdjacent(samples2, filter2); + Vector128 products3 = Vector128_.MultiplyAddAdjacent(samples3, filter3); + + // the reference decoder reduces four independent filters in two horizontal-add stages so the four complete sums occupy + // consecutive lanes. Keeping that arrangement also allows both destination forms to use one packed store. + Vector128 pairs01 = HorizontalAdd(products0, products1); + Vector128 pairs23 = HorizontalAdd(products2, products3); + return (HorizontalAdd(pairs01, pairs23) + Vector128.Create(FilterRounding)) >> FilterBits; + } + + /// + /// Horizontally adds adjacent 32-bit lanes from two filter-product vectors. + /// + /// The first filter-product vector. + /// The second filter-product vector. + /// The adjacent sums from followed by those from . + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector128 HorizontalAdd(Vector128 left, Vector128 right) + { + if (Ssse3.IsSupported) + { + return Ssse3.HorizontalAdd(left, right); + } + + if (AdvSimd.Arm64.IsSupported) + { + return AdvSimd.Arm64.AddPairwise(left, right); + } + + if (AdvSimd.IsSupported) + { + Vector64 leftPairs = AdvSimd.AddPairwise(left.GetLower(), left.GetUpper()); + Vector64 rightPairs = AdvSimd.AddPairwise(right.GetLower(), right.GetUpper()); + return Vector128.Create(leftPairs, rightPairs); + } + + // The portable fallback selects even and odd lanes separately. ShuffleNative is valid here because every index + // is in range; adding the two permutations forms [x0+x1, x2+x3] for each source vector. + Vector128 evenIndices = Vector128.Create(0, 2, 0, 2); + Vector128 oddIndices = Vector128.Create(1, 3, 1, 3); + Vector128 leftPairsFallback = Vector128.ShuffleNative(left, evenIndices) + Vector128.ShuffleNative(left, oddIndices); + Vector128 rightPairsFallback = Vector128.ShuffleNative(right, evenIndices) + Vector128.ShuffleNative(right, oddIndices); + return Vector128.Create(leftPairsFallback.GetLower(), rightPairsFallback.GetLower()); + } + + /// + /// Evaluates one eight-bit source position for the scalar remainder. + /// + /// The first sample in the replicated-edge source row. + /// The fixed-point source position. + /// The largest permitted output sample. + /// The rounded and clipped output sample. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static int FilterOne(ref byte source, int sourcePosition, int maximum) + { + GetOffsets(sourcePosition, out int sourceOffset, out int filterOffset); + + ref short filter = ref MemoryMarshal.GetReference(Filters); + int sum = 0; + for (int tap = 0; tap < TapCount; tap++) + { + sum += Unsafe.Add(ref source, sourceOffset + tap) * Unsafe.Add(ref filter, filterOffset + tap); + } + + return Av1Math.Clip3(0, maximum, (sum + FilterRounding) >> FilterBits); + } + + /// + /// Evaluates one high-bit-depth source position for the scalar remainder. + /// + /// The first sample in the replicated-edge source row. + /// The fixed-point source position. + /// The largest permitted output sample. + /// The rounded and clipped output sample. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static int FilterOne(ref ushort source, int sourcePosition, int maximum) + { + GetOffsets(sourcePosition, out int sourceOffset, out int filterOffset); + + ref short filter = ref MemoryMarshal.GetReference(Filters); + int sum = 0; + for (int tap = 0; tap < TapCount; tap++) + { + sum += Unsafe.Add(ref source, sourceOffset + tap) * Unsafe.Add(ref filter, filterOffset + tap); + } + + return Av1Math.Clip3(0, maximum, (sum + FilterRounding) >> FilterBits); + } + + /// + /// Resolves the source and filter-table offsets for one fixed-point position. + /// + /// The fixed-point source position. + /// Receives the first source tap relative to the replicated-edge row. + /// Receives the first coefficient for the selected filter phase. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static void GetOffsets(int sourcePosition, out int sourceOffset, out int filterOffset) + { + int integerPosition = sourcePosition >> ScaleSubpixelBits; + int filterPhase = (sourcePosition & ScaleSubpixelMask) >> ScaleExtraBits; + sourceOffset = SourceBorder + integerPosition - (TapCount / 2); + filterOffset = filterPhase * TapCount; + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1BottomRightTopLeftConstants.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1BottomRightTopLeftConstants.cs new file mode 100644 index 0000000000..919452ce14 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1BottomRightTopLeftConstants.cs @@ -0,0 +1,682 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; + +/// +/// Provides packed AV1 top-right and bottom-left reference-availability tables for blocks within a superblock. +/// +/// +/// Each bit describes whether a reference edge has already been reconstructed for one block in the AV1 partition traversal order. +/// Separate tables preserve the alternate visit order used by mixed vertical partitions. +/// +internal static class Av1BottomRightTopLeftConstants +{ + // Tables to store if the top-right reference pixels are available. The flags + // are represented with bits, packed into 8-bit integers. E.g., for the 32x32 + // blocks in a 128x128 superblock, the index of the "o" block is 10 (in raster + // order), so its flag is stored at the 3rd bit of the 2nd entry in the table, + // i.e. (table[10 / 8] >> (10 % 8)) & 1. + // . . . . + // . . . . + // . . o . + // . . . . + + /// + /// Packed top-right availability bits for 4-by-4 blocks. + /// + private static readonly byte[] HasTopRight4x4 = [ + 255, 255, 255, 255, 85, 85, 85, 85, 119, 119, 119, 119, 85, 85, 85, 85, 127, 127, 127, 127, 85, 85, + 85, 85, 119, 119, 119, 119, 85, 85, 85, 85, 255, 127, 255, 127, 85, 85, 85, 85, 119, 119, 119, 119, + 85, 85, 85, 85, 127, 127, 127, 127, 85, 85, 85, 85, 119, 119, 119, 119, 85, 85, 85, 85, 255, 255, + 255, 127, 85, 85, 85, 85, 119, 119, 119, 119, 85, 85, 85, 85, 127, 127, 127, 127, 85, 85, 85, 85, + 119, 119, 119, 119, 85, 85, 85, 85, 255, 127, 255, 127, 85, 85, 85, 85, 119, 119, 119, 119, 85, 85, + 85, 85, 127, 127, 127, 127, 85, 85, 85, 85, 119, 119, 119, 119, 85, 85, 85, 85, + ]; + + /// + /// Packed top-right availability bits for 4-by-8 blocks. + /// + private static readonly byte[] HasTopRight4x8 = [ + 255, 255, 255, 255, 119, 119, 119, 119, 127, 127, 127, 127, 119, 119, 119, 119, 255, 127, 255, 127, 119, 119, + 119, 119, 127, 127, 127, 127, 119, 119, 119, 119, 255, 255, 255, 127, 119, 119, 119, 119, 127, 127, 127, 127, + 119, 119, 119, 119, 255, 127, 255, 127, 119, 119, 119, 119, 127, 127, 127, 127, 119, 119, 119, 119, + ]; + + /// + /// Packed top-right availability bits for 8-by-4 blocks. + /// + private static readonly byte[] HasTopRight8x4 = [ + 255, 255, 0, 0, 85, 85, 0, 0, 119, 119, 0, 0, 85, 85, 0, 0, 127, 127, 0, 0, 85, 85, + 0, 0, 119, 119, 0, 0, 85, 85, 0, 0, 255, 127, 0, 0, 85, 85, 0, 0, 119, 119, 0, 0, + 85, 85, 0, 0, 127, 127, 0, 0, 85, 85, 0, 0, 119, 119, 0, 0, 85, 85, 0, 0, + ]; + + /// + /// Packed top-right availability bits for 8-by-8 blocks. + /// + private static readonly byte[] HasTopRight8x8 = [ + 255, 255, 85, 85, 119, 119, 85, 85, 127, 127, 85, 85, 119, 119, 85, 85, + 255, 127, 85, 85, 119, 119, 85, 85, 127, 127, 85, 85, 119, 119, 85, 85, + ]; + + /// + /// Packed top-right availability bits for 8-by-16 blocks. + /// + private static readonly byte[] HasTopRight8x16 = [ + 255, + 255, + 119, + 119, + 127, + 127, + 119, + 119, + 255, + 127, + 119, + 119, + 127, + 127, + 119, + 119, + ]; + + /// + /// Packed top-right availability bits for 16-by-8 blocks. + /// + private static readonly byte[] HasTopRight16x8 = [ + 255, + 0, + 85, + 0, + 119, + 0, + 85, + 0, + 127, + 0, + 85, + 0, + 119, + 0, + 85, + 0, + ]; + + /// + /// Packed top-right availability bits for 16-by-16 blocks. + /// + private static readonly byte[] HasTopRight16x16 = [ + 255, + 85, + 119, + 85, + 127, + 85, + 119, + 85, + ]; + + /// + /// Packed top-right availability bits for 16-by-32 blocks. + /// + private static readonly byte[] HasTopRight16x32 = [255, 119, 127, 119]; + + /// + /// Packed top-right availability bits for 32-by-16 blocks. + /// + private static readonly byte[] HasTopRight32x16 = [15, 5, 7, 5]; + + /// + /// Packed top-right availability bits for 32-by-32 blocks. + /// + private static readonly byte[] HasTopRight32x32 = [95, 87]; + + /// + /// Packed top-right availability bits for 32-by-64 blocks. + /// + private static readonly byte[] HasTopRight32x64 = [127]; + + /// + /// Packed top-right availability bits for 64-by-32 blocks. + /// + private static readonly byte[] HasTopRight64x32 = [19]; + + /// + /// Packed top-right availability bits for 64-by-64 blocks. + /// + private static readonly byte[] HasTopRight64x64 = [7]; + + /// + /// Packed top-right availability bits for 64-by-128 blocks. + /// + private static readonly byte[] HasTopRight64x128 = [3]; + + /// + /// Packed top-right availability bits for 128-by-64 blocks. + /// + private static readonly byte[] HasTopRight128x64 = [1]; + + /// + /// Packed top-right availability bits for 128-by-128 blocks. + /// + private static readonly byte[] HasTopRight128x128 = [1]; + + /// + /// Packed top-right availability bits for 4-by-16 blocks. + /// + private static readonly byte[] HasTopRight4x16 = [ + 255, 255, 255, 255, 127, 127, 127, 127, 255, 127, 255, 127, 127, 127, 127, 127, + 255, 255, 255, 127, 127, 127, 127, 127, 255, 127, 255, 127, 127, 127, 127, 127, + ]; + + /// + /// Packed top-right availability bits for 16-by-4 blocks. + /// + private static readonly byte[] HasTopRight16x4 = [ + 255, 0, 0, 0, 85, 0, 0, 0, 119, 0, 0, 0, 85, 0, 0, 0, 127, 0, 0, 0, 85, 0, 0, 0, 119, 0, 0, 0, 85, 0, 0, 0, + ]; + + /// + /// Packed top-right availability bits for 8-by-32 blocks. + /// + private static readonly byte[] HasTopRight8x32 = [ + 255, + 255, + 127, + 127, + 255, + 127, + 127, + 127, + ]; + + /// + /// Packed top-right availability bits for 32-by-8 blocks. + /// + private static readonly byte[] HasTopRight32x8 = [ + 15, + 0, + 5, + 0, + 7, + 0, + 5, + 0, + ]; + + /// + /// Packed top-right availability bits for 16-by-64 blocks. + /// + private static readonly byte[] HasTopRight16x64 = [255, 127]; + + /// + /// Packed top-right availability bits for 64-by-16 blocks. + /// + private static readonly byte[] HasTopRight64x16 = [3, 1]; + + /// + /// Maps each supported AV1 block-size value to its standard top-right availability table. + /// + private static readonly byte[][] HasTopRightTables = [ + + // 4X4 + HasTopRight4x4, + + // 4X8, 8X4, 8X8 + HasTopRight4x8, + HasTopRight8x4, + HasTopRight8x8, + + // 8X16, 16X8, 16X16 + HasTopRight8x16, + HasTopRight16x8, + HasTopRight16x16, + + // 16X32, 32X16, 32X32 + HasTopRight16x32, + HasTopRight32x16, + HasTopRight32x32, + + // 32X64, 64X32, 64X64 + HasTopRight32x64, + HasTopRight64x32, + HasTopRight64x64, + + // 64x128, 128x64, 128x128 + HasTopRight64x128, + HasTopRight128x64, + HasTopRight128x128, + + // 4x16, 16x4, 8x32 + HasTopRight4x16, + HasTopRight16x4, + HasTopRight8x32, + + // 32x8, 16x64, 64x16 + HasTopRight32x8, + HasTopRight16x64, + HasTopRight64x16 + ]; + + /// + /// Packed top-right availability bits for 8-by-8 blocks visited by a mixed vertical partition. + /// + private static readonly byte[] HasTopRightVertical8x8 = [ + 255, 255, 0, 0, 119, 119, 0, 0, 127, 127, 0, 0, 119, 119, 0, 0, + 255, 127, 0, 0, 119, 119, 0, 0, 127, 127, 0, 0, 119, 119, 0, 0, + ]; + + /// + /// Packed top-right availability bits for 16-by-16 blocks visited by a mixed vertical partition. + /// + private static readonly byte[] HasTopRightVertical16x16 = [ + 255, + 0, + 119, + 0, + 127, + 0, + 119, + 0, + ]; + + /// + /// Packed top-right availability bits for 32-by-32 blocks visited by a mixed vertical partition. + /// + private static readonly byte[] HasTopRightVertical32x32 = [15, 7]; + + /// + /// Packed top-right availability bits for 64-by-64 blocks visited by a mixed vertical partition. + /// + private static readonly byte[] HasTopRightVertical64x64 = [3]; + + // The _vert_* tables are like the ordinary tables above, but describe the + // order we visit square blocks when doing a PARTITION_VERT_A or + // PARTITION_VERT_B. This is the same order as normal except for on the last + // split where we go vertically (TL, BL, TR, BR). We treat the rectangular block + // as a pair of squares, which means that these tables work correctly for both + // mixed vertical partition types. + // + // There are tables for each of the square sizes. Vertical rectangles (like + // BLOCK_16X32) use their respective "non-vert" table + + // Similar to the has_tr_* tables, but store if the bottom-left reference + // pixels are available. + + /// + /// Packed bottom-left availability bits for 4-by-4 blocks. + /// + private static readonly byte[] HasBottomLeft4x4 = [ + 84, 85, 85, 85, 16, 17, 17, 17, 84, 85, 85, 85, 0, 1, 1, 1, 84, 85, 85, 85, 16, 17, 17, 17, 84, 85, + 85, 85, 0, 0, 1, 0, 84, 85, 85, 85, 16, 17, 17, 17, 84, 85, 85, 85, 0, 1, 1, 1, 84, 85, 85, 85, + 16, 17, 17, 17, 84, 85, 85, 85, 0, 0, 0, 0, 84, 85, 85, 85, 16, 17, 17, 17, 84, 85, 85, 85, 0, 1, + 1, 1, 84, 85, 85, 85, 16, 17, 17, 17, 84, 85, 85, 85, 0, 0, 1, 0, 84, 85, 85, 85, 16, 17, 17, 17, + 84, 85, 85, 85, 0, 1, 1, 1, 84, 85, 85, 85, 16, 17, 17, 17, 84, 85, 85, 85, 0, 0, 0, 0, + ]; + + /// + /// Packed bottom-left availability bits for 4-by-8 blocks. + /// + private static readonly byte[] HasBottomLeft4x8 = [ + 16, 17, 17, 17, 0, 1, 1, 1, 16, 17, 17, 17, 0, 0, 1, 0, 16, 17, 17, 17, 0, 1, 1, 1, 16, 17, 17, 17, 0, 0, 0, 0, + 16, 17, 17, 17, 0, 1, 1, 1, 16, 17, 17, 17, 0, 0, 1, 0, 16, 17, 17, 17, 0, 1, 1, 1, 16, 17, 17, 17, 0, 0, 0, 0, + ]; + + /// + /// Packed bottom-left availability bits for 8-by-4 blocks. + /// + private static readonly byte[] HasBottomLeft8x4 = [ + 254, 255, 84, 85, 254, 255, 16, 17, 254, 255, 84, 85, 254, 255, 0, 1, 254, 255, 84, 85, 254, 255, + 16, 17, 254, 255, 84, 85, 254, 255, 0, 0, 254, 255, 84, 85, 254, 255, 16, 17, 254, 255, 84, 85, + 254, 255, 0, 1, 254, 255, 84, 85, 254, 255, 16, 17, 254, 255, 84, 85, 254, 255, 0, 0, + ]; + + /// + /// Packed bottom-left availability bits for 8-by-8 blocks. + /// + private static readonly byte[] HasBottomLeft8x8 = [ + 84, 85, 16, 17, 84, 85, 0, 1, 84, 85, 16, 17, 84, 85, 0, 0, + 84, 85, 16, 17, 84, 85, 0, 1, 84, 85, 16, 17, 84, 85, 0, 0, + ]; + + /// + /// Packed bottom-left availability bits for 8-by-16 blocks. + /// + private static readonly byte[] HasBottomLeft8x16 = [ + 16, + 17, + 0, + 1, + 16, + 17, + 0, + 0, + 16, + 17, + 0, + 1, + 16, + 17, + 0, + 0, + ]; + + /// + /// Packed bottom-left availability bits for 16-by-8 blocks. + /// + private static readonly byte[] HasBottomLeft16x8 = [ + 254, + 84, + 254, + 16, + 254, + 84, + 254, + 0, + 254, + 84, + 254, + 16, + 254, + 84, + 254, + 0, + ]; + + /// + /// Packed bottom-left availability bits for 16-by-16 blocks. + /// + private static readonly byte[] HasBottomLeft16x16 = [ + 84, + 16, + 84, + 0, + 84, + 16, + 84, + 0, + ]; + + /// + /// Packed bottom-left availability bits for 16-by-32 blocks. + /// + private static readonly byte[] HasBottomLeft16x32 = [16, 0, 16, 0]; + + /// + /// Packed bottom-left availability bits for 32-by-16 blocks. + /// + private static readonly byte[] HasBottomLeft32x16 = [78, 14, 78, 14]; + + /// + /// Packed bottom-left availability bits for 32-by-32 blocks. + /// + private static readonly byte[] HasBottomLeft32x32 = [4, 4]; + + /// + /// Packed bottom-left availability bits for 32-by-64 blocks. + /// + private static readonly byte[] HasBottomLeft32x64 = [0]; + + /// + /// Packed bottom-left availability bits for 64-by-32 blocks. + /// + private static readonly byte[] HasBottomLeft64x32 = [34]; + + /// + /// Packed bottom-left availability bits for 64-by-64 blocks. + /// + private static readonly byte[] HasBottomLeft64x64 = [0]; + + /// + /// Packed bottom-left availability bits for 64-by-128 blocks. + /// + private static readonly byte[] HasBottomLeft64x128 = [0]; + + /// + /// Packed bottom-left availability bits for 128-by-64 blocks. + /// + private static readonly byte[] HasBottomLeft128x64 = [0]; + + /// + /// Packed bottom-left availability bits for 128-by-128 blocks. + /// + private static readonly byte[] HasBottomLeft128x128 = [0]; + + /// + /// Packed bottom-left availability bits for 4-by-16 blocks. + /// + private static readonly byte[] HasBottomLeft4x16 = [ + 0, 1, 1, 1, 0, 0, 1, 0, 0, 1, 1, 1, 0, 0, 0, 0, 0, 1, 1, 1, 0, 0, 1, 0, 0, 1, 1, 1, 0, 0, 0, 0, + ]; + + /// + /// Packed bottom-left availability bits for 16-by-4 blocks. + /// + private static readonly byte[] HasBottomLeft16x4 = [ + 254, 254, 254, 84, 254, 254, 254, 16, 254, 254, 254, 84, 254, 254, 254, 0, + 254, 254, 254, 84, 254, 254, 254, 16, 254, 254, 254, 84, 254, 254, 254, 0, + ]; + + /// + /// Packed bottom-left availability bits for 8-by-32 blocks. + /// + private static readonly byte[] HasBottomLeft8x32 = [ + 0, + 1, + 0, + 0, + 0, + 1, + 0, + 0, + ]; + + /// + /// Packed bottom-left availability bits for 32-by-8 blocks. + /// + private static readonly byte[] HasBottomLeft32x8 = [ + 238, + 78, + 238, + 14, + 238, + 78, + 238, + 14, + ]; + + /// + /// Packed bottom-left availability bits for 16-by-64 blocks. + /// + private static readonly byte[] HasBottomLeft16x64 = [0, 0]; + + /// + /// Packed bottom-left availability bits for 64-by-16 blocks. + /// + private static readonly byte[] HasBottomLeft64x16 = [42, 42]; + + /// + /// Maps each supported AV1 block-size value to its standard bottom-left availability table. + /// + private static readonly byte[][] HasBottomLeftTables = [ + + // 4X4 + HasBottomLeft4x4, + + // 4X8, 8X4, 8X8 + HasBottomLeft4x8, + HasBottomLeft8x4, + HasBottomLeft8x8, + + // 8X16, 16X8, 16X16 + HasBottomLeft8x16, + HasBottomLeft16x8, + HasBottomLeft16x16, + + // 16X32, 32X16, 32X32 + HasBottomLeft16x32, + HasBottomLeft32x16, + HasBottomLeft32x32, + + // 32X64, 64X32, 64X64 + HasBottomLeft32x64, + HasBottomLeft64x32, + HasBottomLeft64x64, + + // 64x128, 128x64, 128x128 + HasBottomLeft64x128, + HasBottomLeft128x64, + HasBottomLeft128x128, + + // 4x16, 16x4, 8x32 + HasBottomLeft4x16, + HasBottomLeft16x4, + HasBottomLeft8x32, + + // 32x8, 16x64, 64x16 + HasBottomLeft32x8, + HasBottomLeft16x64, + HasBottomLeft64x16 + ]; + + /// + /// Packed bottom-left availability bits for 8-by-8 blocks visited by a mixed vertical partition. + /// + private static readonly byte[] HasBottomLeftVertical8x8 = [ + 254, 255, 16, 17, 254, 255, 0, 1, 254, 255, 16, 17, 254, 255, 0, 0, + 254, 255, 16, 17, 254, 255, 0, 1, 254, 255, 16, 17, 254, 255, 0, 0, + ]; + + /// + /// Packed bottom-left availability bits for 16-by-16 blocks visited by a mixed vertical partition. + /// + private static readonly byte[] HasBottomLeftVertical16x16 = [ + 254, + 16, + 254, + 0, + 254, + 16, + 254, + 0, + ]; + + /// + /// Packed bottom-left availability bits for 32-by-32 blocks visited by a mixed vertical partition. + /// + private static readonly byte[] HasBottomLeftVertical32x32 = [14, 14]; + + /// + /// Packed bottom-left availability bits for 64-by-64 blocks visited by a mixed vertical partition. + /// + private static readonly byte[] HasBottomLeftVertical64x64 = [2]; + + // The _vert_* tables are like the ordinary tables above, but describe the + // order we visit square blocks when doing a PARTITION_VERT_A or + // PARTITION_VERT_B. This is the same order as normal except for on the last + // split where we go vertically (TL, BL, TR, BR). We treat the rectangular block + // as a pair of squares, which means that these tables work correctly for both + // mixed vertical partition types. + // + // There are tables for each of the square sizes. Vertical rectangles (like + // BLOCK_16X32) use their respective "non-vert" table + + /// + /// Determines whether the top-right reference samples are available for a block at the specified traversal index. + /// + /// The partition type that determines the block traversal order. + /// The size of each block represented by the selected availability table. + /// The block's index in partition traversal order. + /// when the block may use its top-right reference samples; otherwise, . + public static bool HasTopRight(Av1PartitionType partitionType, Av1BlockSize blockSize, int blockIndex) + { + // Eight block flags share each byte; the quotient selects the byte and the + // remainder selects the bit within that byte. + int index1 = blockIndex / 8; + int index2 = blockIndex % 8; + ReadOnlySpan hasTopRightTable = GetHasTopRightTable(partitionType, blockSize); + return ((hasTopRightTable[index1] >> index2) & 1) > 0; + } + + /// + /// Determines whether the bottom-left reference samples are available for a block at the specified traversal index. + /// + /// The partition type that determines the block traversal order. + /// The size of each block represented by the selected availability table. + /// The block's index in partition traversal order. + /// when the block may use its bottom-left reference samples; otherwise, . + public static bool HasBottomLeft(Av1PartitionType partitionType, Av1BlockSize blockSize, int blockIndex) + { + // Eight block flags share each byte; the quotient selects the byte and the + // remainder selects the bit within that byte. + int index1 = blockIndex / 8; + int index2 = blockIndex % 8; + ReadOnlySpan hasBottomLeftTable = GetHasBottomLeftTable(partitionType, blockSize); + return ((hasBottomLeftTable[index1] >> index2) & 1) > 0; + } + + /// + /// Selects the top-right availability table for a block size and partition traversal order. + /// + /// The partition type that determines the block traversal order. + /// The block size whose availability table is selected. + /// The packed top-right availability table. + private static ReadOnlySpan GetHasTopRightTable(Av1PartitionType partition, Av1BlockSize blockSize) + { + // If this is a mixed vertical partition, look up block size in vertical order. + if (partition is Av1PartitionType.VerticalA or Av1PartitionType.VerticalB) + { + // libaom asserts that mixed-vertical traversal can select only vertical rectangles or squares. + // Listing those shapes directly keeps the impossible horizontal-rectangle states out of the table type. + return blockSize switch + { + Av1BlockSize.Block4x8 => HasTopRight4x8, + Av1BlockSize.Block8x8 => HasTopRightVertical8x8, + Av1BlockSize.Block8x16 => HasTopRight8x16, + Av1BlockSize.Block16x16 => HasTopRightVertical16x16, + Av1BlockSize.Block16x32 => HasTopRight16x32, + Av1BlockSize.Block32x32 => HasTopRightVertical32x32, + Av1BlockSize.Block32x64 => HasTopRight32x64, + Av1BlockSize.Block64x64 => HasTopRightVertical64x64, + Av1BlockSize.Block64x128 => HasTopRight64x128, + Av1BlockSize.Block128x128 => HasTopRight128x128, + _ => throw new InvalidOperationException("The mixed-vertical partition selected an invalid AV1 block size.") + }; + } + + return HasTopRightTables[(int)blockSize]; + } + + /// + /// Selects the bottom-left availability table for a block size and partition traversal order. + /// + /// The partition type that determines the block traversal order. + /// The block size whose availability table is selected. + /// The packed bottom-left availability table. + private static ReadOnlySpan GetHasBottomLeftTable(Av1PartitionType partition, Av1BlockSize blockSize) + { + // If this is a mixed vertical partition, look up block size in vertical order. + if (partition is Av1PartitionType.VerticalA or Av1PartitionType.VerticalB) + { + // The valid block shapes mirror the top-right table and the libaom traversal assertion. + return blockSize switch + { + Av1BlockSize.Block4x8 => HasBottomLeft4x8, + Av1BlockSize.Block8x8 => HasBottomLeftVertical8x8, + Av1BlockSize.Block8x16 => HasBottomLeft8x16, + Av1BlockSize.Block16x16 => HasBottomLeftVertical16x16, + Av1BlockSize.Block16x32 => HasBottomLeft16x32, + Av1BlockSize.Block32x32 => HasBottomLeftVertical32x32, + Av1BlockSize.Block32x64 => HasBottomLeft32x64, + Av1BlockSize.Block64x64 => HasBottomLeftVertical64x64, + Av1BlockSize.Block64x128 => HasBottomLeft64x128, + Av1BlockSize.Block128x128 => HasBottomLeft128x128, + _ => throw new InvalidOperationException("The mixed-vertical partition selected an invalid AV1 block size.") + }; + } + + return HasBottomLeftTables[(int)blockSize]; + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1ChromaPredictionMode.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1ChromaPredictionMode.cs new file mode 100644 index 0000000000..c443b86ce8 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1ChromaPredictionMode.cs @@ -0,0 +1,90 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; + +/// +/// Identifies the intra prediction mode used by the chroma planes of an AV1 coding block. +/// +internal enum Av1ChromaPredictionMode : byte +{ + /// + /// Predicts each sample from the average of the available top and left neighbors. + /// + DC, + + /// + /// Repeats the top neighboring row vertically through the block. + /// + Vertical, + + /// + /// Repeats the left neighboring column horizontally through the block. + /// + Horizontal, + + /// + /// Projects neighboring samples into the block at 45 degrees. + /// + Directional45Degrees, + + /// + /// Projects neighboring samples into the block at 135 degrees. + /// + Directional135Degrees, + + /// + /// Projects neighboring samples into the block at 113 degrees. + /// + Directional113Degrees, + + /// + /// Projects neighboring samples into the block at 157 degrees. + /// + Directional157Degrees, + + /// + /// Projects neighboring samples into the block at 203 degrees. + /// + Directional203Degrees, + + /// + /// Projects neighboring samples into the block at 67 degrees. + /// + Directional67Degrees, + + /// + /// Blends horizontal and vertical smooth predictions. + /// + Smooth, + + /// + /// Interpolates vertically between the top row and the bottom-left neighbor. + /// + SmoothVertical, + + /// + /// Interpolates horizontally between the left column and the top-right neighbor. + /// + SmoothHorizontal, + + /// + /// Selects the neighbor with the smallest gradient from the top-left reference. + /// + Paeth, + + /// + /// Predicts chroma from the reconstructed luma AC surface. + /// + ChromaFromLuma, + + /// + /// The exclusive upper bound of valid chroma intra-prediction modes. + /// + ModeCount, + + /// + /// Identifies an unavailable chroma prediction mode on an inter-predicted block. + /// + Invalid, +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1ChromaPredictionModeExtensions.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1ChromaPredictionModeExtensions.cs new file mode 100644 index 0000000000..1090a8037b --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1ChromaPredictionModeExtensions.cs @@ -0,0 +1,46 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; + +/// +/// Provides luma-equivalent prediction metadata for AV1 chroma intra-prediction modes. +/// +internal static class Av1ChromaPredictionModeExtensions +{ + /// + /// Maps a chroma intra-prediction mode to the equivalent luma intra-prediction mode. + /// + /// The chroma intra-prediction mode. + /// The luma mode with the same spatial predictor, or the invalid luma sentinel for an invalid chroma mode. + public static Av1PredictionMode ToLumaMode(this Av1ChromaPredictionMode mode) + => mode switch + { + Av1ChromaPredictionMode.DC => Av1PredictionMode.DC, + Av1ChromaPredictionMode.Vertical => Av1PredictionMode.Vertical, + Av1ChromaPredictionMode.Horizontal => Av1PredictionMode.Horizontal, + Av1ChromaPredictionMode.Directional45Degrees => Av1PredictionMode.Directional45Degrees, + Av1ChromaPredictionMode.Directional135Degrees => Av1PredictionMode.Directional135Degrees, + Av1ChromaPredictionMode.Directional113Degrees => Av1PredictionMode.Directional113Degrees, + Av1ChromaPredictionMode.Directional157Degrees => Av1PredictionMode.Directional157Degrees, + Av1ChromaPredictionMode.Directional203Degrees => Av1PredictionMode.Directional203Degrees, + Av1ChromaPredictionMode.Directional67Degrees => Av1PredictionMode.Directional67Degrees, + Av1ChromaPredictionMode.Smooth => Av1PredictionMode.Smooth, + Av1ChromaPredictionMode.SmoothVertical => Av1PredictionMode.SmoothVertical, + Av1ChromaPredictionMode.SmoothHorizontal => Av1PredictionMode.SmoothHorizontal, + Av1ChromaPredictionMode.Paeth => Av1PredictionMode.Paeth, + + // Chroma-from-luma adds its AC contribution to a DC prediction. the reference decoder's get_uv_mode() therefore maps it + // to DC when shared transform and neighbor metadata require the corresponding luma predictor. + Av1ChromaPredictionMode.ChromaFromLuma => Av1PredictionMode.DC, + _ => Av1PredictionMode.IntraInvalid, + }; + + /// + /// Determines whether a chroma intra-prediction mode projects samples along a coded angle. + /// + /// The chroma intra-prediction mode. + /// for a directional mode; otherwise, . + public static bool IsDirectional(this Av1ChromaPredictionMode mode) + => mode is >= Av1ChromaPredictionMode.Vertical and <= Av1ChromaPredictionMode.Directional67Degrees; +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1DcIntraPredictor.Operator.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1DcIntraPredictor.Operator.cs new file mode 100644 index 0000000000..51ff071fde --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1DcIntraPredictor.Operator.cs @@ -0,0 +1,434 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.InteropServices; +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; + +/// +/// Defines reference reduction and rounded mean arithmetic for AV1 DC intra prediction. +/// +internal static class Av1DcIntraPredictor +{ + /// + /// Defines scalar and SIMD reference reduction for AV1 DC intra prediction. + /// + internal interface IDcPredictionOperator + { + /// + /// Sums one 8-bit reference sample. + /// + /// The reference sample. + /// The sample value. + public static abstract int Sum(byte sample); + + /// + /// Sums sixteen 8-bit reference samples. + /// + /// The reference samples. + /// The exact sum. + public static abstract int Sum(Vector128 samples); + + /// + /// Sums thirty-two 8-bit reference samples. + /// + /// The reference samples. + /// The exact sum. + public static abstract int Sum(Vector256 samples); + + /// + /// Sums sixty-four 8-bit reference samples. + /// + /// The reference samples. + /// The exact sum. + public static abstract int Sum(Vector512 samples); + + /// + /// Sums one high-bit-depth reference sample. + /// + /// The reference sample. + /// The sample value. + public static abstract int Sum(short sample); + + /// + /// Sums eight high-bit-depth reference samples. + /// + /// The reference samples. + /// The exact sum. + public static abstract int Sum(Vector128 samples); + + /// + /// Sums sixteen high-bit-depth reference samples. + /// + /// The reference samples. + /// The exact sum. + public static abstract int Sum(Vector256 samples); + + /// + /// Sums thirty-two high-bit-depth reference samples. + /// + /// The reference samples. + /// The exact sum. + public static abstract int Sum(Vector512 samples); + + /// + /// Calculates the 8-bit DC prediction. + /// + /// The sum of available reference samples. + /// The number of available reference samples. + /// The rounded DC prediction. + public static abstract byte Predict(int sum, int count); + + /// + /// Calculates the high-bit-depth DC prediction. + /// + /// The sum of available reference samples. + /// The number of available reference samples. + /// The reconstructed sample precision. + /// The rounded DC prediction. + public static abstract short Predict(int sum, int count, int bitDepth); + } + + /// + /// Predicts an 8-bit DC block. + /// + public static void Predict(bool hasLeft, bool hasAbove, Span destination, int destinationStride, ReadOnlySpan above, ReadOnlySpan left, int width, int height) + => Predictor.Predict(hasLeft, hasAbove, destination, destinationStride, above, left, width, height); + + /// + /// Predicts a high-bit-depth DC block. + /// + public static void Predict(bool hasLeft, bool hasAbove, Span destination, int destinationStride, ReadOnlySpan above, ReadOnlySpan left, int width, int height, int bitDepth) + => Predictor.Predict(hasLeft, hasAbove, destination, destinationStride, above, left, width, height, bitDepth); + + /// + /// Predicts an 8-bit DC block without hardware intrinsics. + /// + public static void PredictScalar(bool hasLeft, bool hasAbove, Span destination, int destinationStride, ReadOnlySpan above, ReadOnlySpan left, int width, int height) + => Predictor.PredictScalar(hasLeft, hasAbove, destination, destinationStride, above, left, width, height); + + /// + /// Predicts a high-bit-depth DC block without hardware intrinsics. + /// + public static void PredictScalar(bool hasLeft, bool hasAbove, Span destination, int destinationStride, ReadOnlySpan above, ReadOnlySpan left, int width, int height, int bitDepth) + => Predictor.PredictScalar(hasLeft, hasAbove, destination, destinationStride, above, left, width, height, bitDepth); + + /// + /// Calculates the DC value from the available neighboring samples. + /// + internal readonly struct DcOperator : IDcPredictionOperator + { + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static int Sum(byte sample) => sample; + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static int Sum(Vector128 samples) + { + (Vector128 lower, Vector128 upper) = Vector128.Widen(samples); + + return Vector128.Sum(lower) + Vector128.Sum(upper); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static int Sum(Vector256 samples) + { + (Vector256 lower, Vector256 upper) = Vector256.Widen(samples); + + return Vector256.Sum(lower) + Vector256.Sum(upper); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static int Sum(Vector512 samples) + { + (Vector512 lower, Vector512 upper) = Vector512.Widen(samples); + + return Vector512.Sum(lower) + Vector512.Sum(upper); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static int Sum(short sample) => sample; + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static int Sum(Vector128 samples) + { + (Vector128 lower, Vector128 upper) = Vector128.Widen(samples); + + return Vector128.Sum(lower) + Vector128.Sum(upper); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static int Sum(Vector256 samples) + { + (Vector256 lower, Vector256 upper) = Vector256.Widen(samples); + + return Vector256.Sum(lower) + Vector256.Sum(upper); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static int Sum(Vector512 samples) + { + (Vector512 lower, Vector512 upper) = Vector512.Widen(samples); + + return Vector512.Sum(lower) + Vector512.Sum(upper); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static byte Predict(int sum, int count) => count == 0 ? (byte)128 : (byte)((sum + (count >> 1)) / count); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static short Predict(int sum, int count, int bitDepth) + => count == 0 ? (short)(1 << (bitDepth - 1)) : (short)((sum + (count >> 1)) / count); + } + + /// + /// Reconstructs DC blocks through one closed reduction operator. + /// + /// The reference reduction and rounded mean arithmetic. + private static class Predictor + where TOperator : struct, IDcPredictionOperator + { + /// + /// Predicts an 8-bit DC block. + /// + /// Whether the left reference is available. + /// Whether the top reference is available. + /// The destination block. + /// The distance between destination rows. + /// The top reference. + /// The left reference. + /// The block width. + /// The block height. + public static void Predict(bool hasLeft, bool hasAbove, Span destination, int destinationStride, ReadOnlySpan above, ReadOnlySpan left, int width, int height) + { + int count = (hasAbove ? width : 0) + (hasLeft ? height : 0); + int sum = (hasAbove ? Sum(above[..width]) : 0) + (hasLeft ? Sum(left[..height]) : 0); + byte prediction = TOperator.Predict(sum, count); + + for (int row = 0; row < height; row++) + { + destination.Slice(row * destinationStride, width).Fill(prediction); + } + } + + /// + /// Predicts a high-bit-depth DC block. + /// + /// Whether the left reference is available. + /// Whether the top reference is available. + /// The destination block. + /// The distance between destination rows. + /// The top reference. + /// The left reference. + /// The block width. + /// The block height. + /// The reconstructed sample precision. + public static void Predict(bool hasLeft, bool hasAbove, Span destination, int destinationStride, ReadOnlySpan above, ReadOnlySpan left, int width, int height, int bitDepth) + { + int count = (hasAbove ? width : 0) + (hasLeft ? height : 0); + int sum = (hasAbove ? Sum(above[..width]) : 0) + (hasLeft ? Sum(left[..height]) : 0); + short prediction = TOperator.Predict(sum, count, bitDepth); + + for (int row = 0; row < height; row++) + { + destination.Slice(row * destinationStride, width).Fill(prediction); + } + } + + /// + /// Predicts an 8-bit DC block without hardware intrinsics. + /// + /// Whether the left reference is available. + /// Whether the top reference is available. + /// The destination block. + /// The distance between destination rows. + /// The top reference. + /// The left reference. + /// The block width. + /// The block height. + public static void PredictScalar(bool hasLeft, bool hasAbove, Span destination, int destinationStride, ReadOnlySpan above, ReadOnlySpan left, int width, int height) + { + int count = (hasAbove ? width : 0) + (hasLeft ? height : 0); + int sum = (hasAbove ? SumScalar(above[..width]) : 0) + (hasLeft ? SumScalar(left[..height]) : 0); + byte prediction = TOperator.Predict(sum, count); + + for (int row = 0; row < height; row++) + { + ref byte destinationRow = ref destination[row * destinationStride]; + for (int column = 0; column < width; column++) + { + Unsafe.Add(ref destinationRow, column) = prediction; + } + } + } + + /// + /// Predicts a high-bit-depth DC block without hardware intrinsics. + /// + /// Whether the left reference is available. + /// Whether the top reference is available. + /// The destination block. + /// The distance between destination rows. + /// The top reference. + /// The left reference. + /// The block width. + /// The block height. + /// The reconstructed sample precision. + public static void PredictScalar(bool hasLeft, bool hasAbove, Span destination, int destinationStride, ReadOnlySpan above, ReadOnlySpan left, int width, int height, int bitDepth) + { + int count = (hasAbove ? width : 0) + (hasLeft ? height : 0); + int sum = (hasAbove ? SumScalar(above[..width]) : 0) + (hasLeft ? SumScalar(left[..height]) : 0); + short prediction = TOperator.Predict(sum, count, bitDepth); + + for (int row = 0; row < height; row++) + { + ref short destinationRow = ref destination[row * destinationStride]; + for (int column = 0; column < width; column++) + { + Unsafe.Add(ref destinationRow, column) = prediction; + } + } + } + + /// + /// Sums 8-bit references through the widest available SIMD widths. + /// + /// The reference samples. + /// The exact sum. + private static int Sum(ReadOnlySpan samples) + { + ref byte samplesBase = ref MemoryMarshal.GetReference(samples); + int sum = 0; + int index = 0; + + // The shared index deliberately continues through narrower widths. This handles every legal AV1 edge + // length without a separate dispatch tree and leaves only an incomplete final vector to scalar code. + if (Vector512.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector512Count(samples.Length - index); + for (; vectorCount > 0; vectorCount--, index += Vector512.Count) + { + sum += TOperator.Sum(Vector512.LoadUnsafe(ref samplesBase, (nuint)index)); + } + } + + if (Vector256.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector256Count(samples.Length - index); + for (; vectorCount > 0; vectorCount--, index += Vector256.Count) + { + sum += TOperator.Sum(Vector256.LoadUnsafe(ref samplesBase, (nuint)index)); + } + } + + if (Vector128.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector128Count(samples.Length - index); + for (; vectorCount > 0; vectorCount--, index += Vector128.Count) + { + sum += TOperator.Sum(Vector128.LoadUnsafe(ref samplesBase, (nuint)index)); + } + } + + for (; index < samples.Length; index++) + { + sum += TOperator.Sum(Unsafe.Add(ref samplesBase, index)); + } + + return sum; + } + + /// + /// Sums high-bit-depth references through the widest available SIMD widths. + /// + /// The reference samples. + /// The exact sum. + private static int Sum(ReadOnlySpan samples) + { + ref short samplesBase = ref MemoryMarshal.GetReference(samples); + int sum = 0; + int index = 0; + + if (Vector512.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector512Count(samples.Length - index); + for (; vectorCount > 0; vectorCount--, index += Vector512.Count) + { + sum += TOperator.Sum(Vector512.LoadUnsafe(ref samplesBase, (nuint)index)); + } + } + + if (Vector256.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector256Count(samples.Length - index); + for (; vectorCount > 0; vectorCount--, index += Vector256.Count) + { + sum += TOperator.Sum(Vector256.LoadUnsafe(ref samplesBase, (nuint)index)); + } + } + + if (Vector128.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector128Count(samples.Length - index); + for (; vectorCount > 0; vectorCount--, index += Vector128.Count) + { + sum += TOperator.Sum(Vector128.LoadUnsafe(ref samplesBase, (nuint)index)); + } + } + + for (; index < samples.Length; index++) + { + sum += TOperator.Sum(Unsafe.Add(ref samplesBase, index)); + } + + return sum; + } + + /// + /// Sums 8-bit references without hardware intrinsics. + /// + /// The reference samples. + /// The exact sum. + private static int SumScalar(ReadOnlySpan samples) + { + ref byte samplesBase = ref MemoryMarshal.GetReference(samples); + int sum = 0; + + for (int index = 0; index < samples.Length; index++) + { + sum += TOperator.Sum(Unsafe.Add(ref samplesBase, index)); + } + + return sum; + } + + /// + /// Sums high-bit-depth references without hardware intrinsics. + /// + /// The reference samples. + /// The exact sum. + private static int SumScalar(ReadOnlySpan samples) + { + ref short samplesBase = ref MemoryMarshal.GetReference(samples); + int sum = 0; + + for (int index = 0; index < samples.Length; index++) + { + sum += TOperator.Sum(Unsafe.Add(ref samplesBase, index)); + } + + return sum; + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1DirectionalIntraPredictor.Operations.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1DirectionalIntraPredictor.Operations.cs new file mode 100644 index 0000000000..7266b8b02f --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1DirectionalIntraPredictor.Operations.cs @@ -0,0 +1,950 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.InteropServices; +using System.Runtime.Intrinsics; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; + +/// +/// Provides packed projection kernels for AV1 directional intra prediction. Contiguous zone-one references map one +/// output sample to each lane. Upsampled references use native byte or 16-bit table shuffles to select alternating +/// half-sample positions, while zone-two left projections construct four independent coordinates per vector because +/// their bases advance by the directional derivative rather than by a fixed memory stride. Every interpolation uses +/// Q5 weights and the scalar continuation preserves the same rounding and endpoint-extension rules. +/// +internal static partial class Av1DirectionalIntraPredictor +{ + /// + /// Implements the directional traversal for one closed interpolation operator. + /// + private static partial class Predictor + where TOperator : struct, IDirectionalPredictionOperator + { + /// + /// Gets the indices of even bytes in one upsampled reference vector. + /// + private static Vector128 EvenByteIndices => Vector128.Create((byte)0, 2, 4, 6, 8, 10, 12, 14, 0, 2, 4, 6, 8, 10, 12, 14); + + /// + /// Gets the indices of odd bytes in one upsampled reference vector. + /// + private static Vector128 OddByteIndices => Vector128.Create((byte)1, 3, 5, 7, 9, 11, 13, 15, 1, 3, 5, 7, 9, 11, 13, 15); + + /// + /// Gets the indices of even high-bit-depth samples in one upsampled reference vector. + /// + private static Vector128 EvenShortIndices => Vector128.Create((short)0, 2, 4, 6, 0, 2, 4, 6); + + /// + /// Gets the indices of odd high-bit-depth samples in one upsampled reference vector. + /// + private static Vector128 OddShortIndices => Vector128.Create((short)1, 3, 5, 7, 1, 3, 5, 7); + + /// + /// Predicts one 8-bit zone 1 block using contiguous SIMD projection rows. + /// + /// The destination block origin. + /// The destination row stride. + /// The projected top reference. + /// Whether the reference contains half-sample positions. + /// The Q8 projection derivative. + /// The block width. + /// The block height. + private static void PredictZone1(Span destination, int destinationStride, ReadOnlySpan above, bool upsample, int derivative, int width, int height) + { + int upsampleShift = upsample ? 1 : 0; + int maximumBasis = (width + height - 1) << upsampleShift; + int fractionBits = 6 - upsampleShift; + int projection = derivative; + + for (int row = 0; row < height; row++, projection += derivative) + { + int basis = projection >> fractionBits; + int weight = ((projection << upsampleShift) & 0x3F) >> 1; + InterpolateRow(destination.Slice(row * destinationStride, width), above, basis, weight, upsample, maximumBasis); + } + } + + /// + /// Predicts one high-bit-depth zone 1 block using contiguous SIMD projection rows. + /// + /// The destination block origin. + /// The destination row stride. + /// The projected top reference. + /// Whether the reference contains half-sample positions. + /// The Q8 projection derivative. + /// The block width. + /// The block height. + private static void PredictZone1(Span destination, int destinationStride, ReadOnlySpan above, bool upsample, int derivative, int width, int height) + { + int upsampleShift = upsample ? 1 : 0; + int maximumBasis = (width + height - 1) << upsampleShift; + int fractionBits = 6 - upsampleShift; + int projection = derivative; + + for (int row = 0; row < height; row++, projection += derivative) + { + int basis = projection >> fractionBits; + int weight = ((projection << upsampleShift) & 0x3F) >> 1; + InterpolateRow(destination.Slice(row * destinationStride, width), above, basis, weight, upsample, maximumBasis); + } + } + + /// + /// Predicts one 8-bit zone 2 block using vectorized left gathers and contiguous top projection rows. + /// + /// The destination block origin. + /// The destination row stride. + /// The projected top reference. + /// The projected left reference. + /// Whether the top reference contains half-sample positions. + /// Whether the left reference contains half-sample positions. + /// The horizontal Q8 derivative. + /// The vertical Q8 derivative. + /// The block width. + /// The block height. + private static void PredictZone2(Span destination, int destinationStride, ReadOnlySpan above, ReadOnlySpan left, bool upsampleAbove, bool upsampleLeft, int dx, int dy, int width, int height) + { + int aboveShift = upsampleAbove ? 1 : 0; + int minimumTopBasis = -(1 << aboveShift); + int topFractionBits = 6 - aboveShift; + int topBasisIncrement = 1 << aboveShift; + int topProjection = -dx; + + for (int row = 0; row < height; row++, topProjection -= dx) + { + int topBasis = topProjection >> topFractionBits; + int leftCount = 0; + while (leftCount < width && topBasis < minimumTopBasis) + { + leftCount++; + topBasis += topBasisIncrement; + } + + Span destinationRow = destination.Slice(row * destinationStride, width); + int leftProjection = (row << 6) - dy; + InterpolateLeft(destinationRow[..leftCount], left, leftProjection, dy, upsampleLeft); + + if (leftCount < width) + { + int topWeight = ((topProjection << aboveShift) & 0x3F) >> 1; + InterpolateRow(destinationRow[leftCount..], above, topBasis, topWeight, upsampleAbove, int.MaxValue); + } + } + } + + /// + /// Predicts one high-bit-depth zone 2 block using vectorized left gathers and contiguous top projection rows. + /// + /// The destination block origin. + /// The destination row stride. + /// The projected top reference. + /// The projected left reference. + /// Whether the top reference contains half-sample positions. + /// Whether the left reference contains half-sample positions. + /// The horizontal Q8 derivative. + /// The vertical Q8 derivative. + /// The block width. + /// The block height. + private static void PredictZone2(Span destination, int destinationStride, ReadOnlySpan above, ReadOnlySpan left, bool upsampleAbove, bool upsampleLeft, int dx, int dy, int width, int height) + { + int aboveShift = upsampleAbove ? 1 : 0; + int minimumTopBasis = -(1 << aboveShift); + int topFractionBits = 6 - aboveShift; + int topBasisIncrement = 1 << aboveShift; + int topProjection = -dx; + + for (int row = 0; row < height; row++, topProjection -= dx) + { + int topBasis = topProjection >> topFractionBits; + int leftCount = 0; + while (leftCount < width && topBasis < minimumTopBasis) + { + leftCount++; + topBasis += topBasisIncrement; + } + + Span destinationRow = destination.Slice(row * destinationStride, width); + int leftProjection = (row << 6) - dy; + InterpolateLeft(destinationRow[..leftCount], left, leftProjection, dy, upsampleLeft); + + if (leftCount < width) + { + int topWeight = ((topProjection << aboveShift) & 0x3F) >> 1; + InterpolateRow(destinationRow[leftCount..], above, topBasis, topWeight, upsampleAbove, int.MaxValue); + } + } + } + + /// + /// Interpolates one 8-bit projection row. + /// + /// The destination row. + /// The projected reference samples. + /// The first integral reference coordinate. + /// The right-sample interpolation weight. + /// Whether consecutive output samples advance two reference positions. + /// The final extended reference coordinate, or when the row cannot reach it. + private static void InterpolateRow(Span destination, ReadOnlySpan reference, int basis, int weight, bool upsample, int maximumBasis) + { + ref byte destinationBase = ref MemoryMarshal.GetReference(destination); + ref byte referenceBase = ref MemoryMarshal.GetReference(reference); + int basisIncrement = upsample ? 2 : 1; + int validCount = maximumBasis == int.MaxValue || basis >= maximumBasis + ? maximumBasis == int.MaxValue ? destination.Length : 0 + : Math.Min(destination.Length, ((maximumBasis - 1 - basis) / basisIncrement) + 1); + int index = 0; + + if (!upsample) + { + // A single index is advanced through all supported widths. A narrower path consumes only the remainder + // left by the wider path, so the row is written once without requiring padded destination storage. + if (Vector512.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector512Count(validCount - index); + for (; vectorCount > 0; vectorCount--, index += Vector512.Count) + { + Vector512 left = Vector512.LoadUnsafe(ref referenceBase, (nuint)(basis + index)); + Vector512 right = Vector512.LoadUnsafe(ref referenceBase, (nuint)(basis + index + 1)); + TOperator.Interpolate(left, right, weight).StoreUnsafe(ref destinationBase, (nuint)index); + } + } + + if (Vector256.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector256Count(validCount - index); + for (; vectorCount > 0; vectorCount--, index += Vector256.Count) + { + Vector256 left = Vector256.LoadUnsafe(ref referenceBase, (nuint)(basis + index)); + Vector256 right = Vector256.LoadUnsafe(ref referenceBase, (nuint)(basis + index + 1)); + TOperator.Interpolate(left, right, weight).StoreUnsafe(ref destinationBase, (nuint)index); + } + } + + if (Vector128.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector128Count(validCount - index); + for (; vectorCount > 0; vectorCount--, index += Vector128.Count) + { + Vector128 left = Vector128.LoadUnsafe(ref referenceBase, (nuint)(basis + index)); + Vector128 right = Vector128.LoadUnsafe(ref referenceBase, (nuint)(basis + index + 1)); + TOperator.Interpolate(left, right, weight).StoreUnsafe(ref destinationBase, (nuint)index); + } + } + } + else if (Vector128.IsHardwareAccelerated) + { + int oneVectorFromEnd = validCount - 8; + for (; index <= oneVectorFromEnd; index += 8) + { + Vector128 source = Vector128.LoadUnsafe(ref referenceBase, (nuint)(basis + (index * 2))); + + // ShuffleNative maps to byte-table lookup on AdvSimd and PSHUFB on x86. Eight output samples are + // gathered from sixteen half-sample positions without scalar lane construction. + Vector128 left = Vector128.ShuffleNative(source, EvenByteIndices); + Vector128 right = Vector128.ShuffleNative(source, OddByteIndices); + Vector128 prediction = TOperator.Interpolate(left, right, weight); + Unsafe.As(ref Unsafe.Add(ref destinationBase, index)) = prediction.AsUInt64().ToScalar(); + } + } + + if (Vector128.IsHardwareAccelerated) + { + // Four-lane construction covers both the final non-upsampled remainder and targets without a native + // gather. Each lane carries an independently projected coordinate but shares the row's interpolation weight. + int oneVectorFromEnd = validCount - 4; + for (; index <= oneVectorFromEnd; index += 4) + { + int source = basis + (index * basisIncrement); + Vector128 left = Vector128.Create( + (int)Unsafe.Add(ref referenceBase, source), + Unsafe.Add(ref referenceBase, source + basisIncrement), + Unsafe.Add(ref referenceBase, source + (2 * basisIncrement)), + Unsafe.Add(ref referenceBase, source + (3 * basisIncrement))); + + Vector128 right = Vector128.Create( + (int)Unsafe.Add(ref referenceBase, source + 1), + Unsafe.Add(ref referenceBase, source + basisIncrement + 1), + Unsafe.Add(ref referenceBase, source + (2 * basisIncrement) + 1), + Unsafe.Add(ref referenceBase, source + (3 * basisIncrement) + 1)); + + StoreFourBytes(TOperator.Interpolate(left, right, Vector128.Create(weight)), ref Unsafe.Add(ref destinationBase, index)); + } + } + + for (; index < validCount; index++) + { + int source = basis + (index * basisIncrement); + Unsafe.Add(ref destinationBase, index) = (byte)(((Unsafe.Add(ref referenceBase, source) * (32 - weight)) + (Unsafe.Add(ref referenceBase, source + 1) * weight) + 16) >> 5); + } + + if (validCount < destination.Length) + { + destination[validCount..].Fill(Unsafe.Add(ref referenceBase, maximumBasis)); + } + } + + /// + /// Interpolates one high-bit-depth projection row. + /// + /// The destination row. + /// The projected reference samples. + /// The first integral reference coordinate. + /// The right-sample interpolation weight. + /// Whether consecutive output samples advance two reference positions. + /// The final extended reference coordinate, or when the row cannot reach it. + private static void InterpolateRow(Span destination, ReadOnlySpan reference, int basis, int weight, bool upsample, int maximumBasis) + { + ref short destinationBase = ref MemoryMarshal.GetReference(destination); + ref short referenceBase = ref MemoryMarshal.GetReference(reference); + int basisIncrement = upsample ? 2 : 1; + int validCount = maximumBasis == int.MaxValue || basis >= maximumBasis + ? maximumBasis == int.MaxValue ? destination.Length : 0 + : Math.Min(destination.Length, ((maximumBasis - 1 - basis) / basisIncrement) + 1); + int index = 0; + + if (!upsample) + { + // High-bit-depth samples stay in signed 16-bit storage, but interpolation widens to Int32 before the Q5 + // weighted sum. The largest supported 12-bit sample therefore cannot overflow an intermediate lane. + if (Vector512.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector512Count(validCount - index); + for (; vectorCount > 0; vectorCount--, index += Vector512.Count) + { + Vector512 left = Vector512.LoadUnsafe(ref referenceBase, (nuint)(basis + index)); + Vector512 right = Vector512.LoadUnsafe(ref referenceBase, (nuint)(basis + index + 1)); + TOperator.Interpolate(left, right, weight).StoreUnsafe(ref destinationBase, (nuint)index); + } + } + + if (Vector256.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector256Count(validCount - index); + for (; vectorCount > 0; vectorCount--, index += Vector256.Count) + { + Vector256 left = Vector256.LoadUnsafe(ref referenceBase, (nuint)(basis + index)); + Vector256 right = Vector256.LoadUnsafe(ref referenceBase, (nuint)(basis + index + 1)); + TOperator.Interpolate(left, right, weight).StoreUnsafe(ref destinationBase, (nuint)index); + } + } + + if (Vector128.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector128Count(validCount - index); + for (; vectorCount > 0; vectorCount--, index += Vector128.Count) + { + Vector128 left = Vector128.LoadUnsafe(ref referenceBase, (nuint)(basis + index)); + Vector128 right = Vector128.LoadUnsafe(ref referenceBase, (nuint)(basis + index + 1)); + TOperator.Interpolate(left, right, weight).StoreUnsafe(ref destinationBase, (nuint)index); + } + } + } + else if (Vector128.IsHardwareAccelerated) + { + int oneVectorFromEnd = validCount - 4; + for (; index <= oneVectorFromEnd; index += 4) + { + Vector128 source = Vector128.LoadUnsafe(ref referenceBase, (nuint)(basis + (index * 2))); + Vector128 left = Vector128.ShuffleNative(source, EvenShortIndices); + Vector128 right = Vector128.ShuffleNative(source, OddShortIndices); + Vector128 prediction = TOperator.Interpolate(left, right, weight); + Unsafe.As(ref Unsafe.Add(ref destinationBase, index)) = prediction.AsUInt64().ToScalar(); + } + } + + if (Vector128.IsHardwareAccelerated) + { + int oneVectorFromEnd = validCount - 4; + for (; index <= oneVectorFromEnd; index += 4) + { + int source = basis + (index * basisIncrement); + Vector128 left = Vector128.Create( + (int)Unsafe.Add(ref referenceBase, source), + Unsafe.Add(ref referenceBase, source + basisIncrement), + Unsafe.Add(ref referenceBase, source + (2 * basisIncrement)), + Unsafe.Add(ref referenceBase, source + (3 * basisIncrement))); + + Vector128 right = Vector128.Create( + (int)Unsafe.Add(ref referenceBase, source + 1), + Unsafe.Add(ref referenceBase, source + basisIncrement + 1), + Unsafe.Add(ref referenceBase, source + (2 * basisIncrement) + 1), + Unsafe.Add(ref referenceBase, source + (3 * basisIncrement) + 1)); + + StoreFourShorts(TOperator.Interpolate(left, right, Vector128.Create(weight)), ref Unsafe.Add(ref destinationBase, index)); + } + } + + for (; index < validCount; index++) + { + int source = basis + (index * basisIncrement); + Unsafe.Add(ref destinationBase, index) = (short)(((Unsafe.Add(ref referenceBase, source) * (32 - weight)) + (Unsafe.Add(ref referenceBase, source + 1) * weight) + 16) >> 5); + } + + if (validCount < destination.Length) + { + destination[validCount..].Fill(Unsafe.Add(ref referenceBase, maximumBasis)); + } + } + + /// + /// Interpolates the left-edge prefix of one 8-bit zone 2 row. + /// + /// The destination prefix. + /// The projected left reference. + /// The first Q6 left projection. + /// The Q8 derivative subtracted between columns. + /// Whether the left reference contains half-sample positions. + private static void InterpolateLeft(Span destination, ReadOnlySpan left, int projection, int derivative, bool upsample) + { + ref byte destinationBase = ref MemoryMarshal.GetReference(destination); + ref byte leftBase = ref MemoryMarshal.GetReference(left); + int upsampleShift = upsample ? 1 : 0; + int fractionBits = 6 - upsampleShift; + int index = 0; + + if (Vector128.IsHardwareAccelerated) + { + // Zone-two left references are not contiguous across output columns. Constructing the four source pairs + // directly avoids a temporary gather-index buffer and keeps the scalar continuation at the same offset. + int oneVectorFromEnd = destination.Length - 4; + for (; index <= oneVectorFromEnd; index += 4) + { + int projection0 = projection - (index * derivative); + int projection1 = projection0 - derivative; + int projection2 = projection1 - derivative; + int projection3 = projection2 - derivative; + int basis0 = projection0 >> fractionBits; + int basis1 = projection1 >> fractionBits; + int basis2 = projection2 >> fractionBits; + int basis3 = projection3 >> fractionBits; + Vector128 source0 = Vector128.Create((int)Unsafe.Add(ref leftBase, basis0), Unsafe.Add(ref leftBase, basis1), Unsafe.Add(ref leftBase, basis2), Unsafe.Add(ref leftBase, basis3)); + Vector128 source1 = Vector128.Create((int)Unsafe.Add(ref leftBase, basis0 + 1), Unsafe.Add(ref leftBase, basis1 + 1), Unsafe.Add(ref leftBase, basis2 + 1), Unsafe.Add(ref leftBase, basis3 + 1)); + Vector128 weights = Vector128.Create( + ((projection0 << upsampleShift) & 0x3F) >> 1, + ((projection1 << upsampleShift) & 0x3F) >> 1, + ((projection2 << upsampleShift) & 0x3F) >> 1, + ((projection3 << upsampleShift) & 0x3F) >> 1); + + StoreFourBytes(TOperator.Interpolate(source0, source1, weights), ref Unsafe.Add(ref destinationBase, index)); + } + } + + for (; index < destination.Length; index++) + { + int currentProjection = projection - (index * derivative); + int basis = currentProjection >> fractionBits; + int weight = ((currentProjection << upsampleShift) & 0x3F) >> 1; + Unsafe.Add(ref destinationBase, index) = (byte)(((Unsafe.Add(ref leftBase, basis) * (32 - weight)) + (Unsafe.Add(ref leftBase, basis + 1) * weight) + 16) >> 5); + } + } + + /// + /// Interpolates the left-edge prefix of one high-bit-depth zone 2 row. + /// + /// The destination prefix. + /// The projected left reference. + /// The first Q6 left projection. + /// The Q8 derivative subtracted between columns. + /// Whether the left reference contains half-sample positions. + private static void InterpolateLeft(Span destination, ReadOnlySpan left, int projection, int derivative, bool upsample) + { + ref short destinationBase = ref MemoryMarshal.GetReference(destination); + ref short leftBase = ref MemoryMarshal.GetReference(left); + int upsampleShift = upsample ? 1 : 0; + int fractionBits = 6 - upsampleShift; + int index = 0; + + if (Vector128.IsHardwareAccelerated) + { + int oneVectorFromEnd = destination.Length - 4; + for (; index <= oneVectorFromEnd; index += 4) + { + int projection0 = projection - (index * derivative); + int projection1 = projection0 - derivative; + int projection2 = projection1 - derivative; + int projection3 = projection2 - derivative; + int basis0 = projection0 >> fractionBits; + int basis1 = projection1 >> fractionBits; + int basis2 = projection2 >> fractionBits; + int basis3 = projection3 >> fractionBits; + Vector128 source0 = Vector128.Create((int)Unsafe.Add(ref leftBase, basis0), Unsafe.Add(ref leftBase, basis1), Unsafe.Add(ref leftBase, basis2), Unsafe.Add(ref leftBase, basis3)); + Vector128 source1 = Vector128.Create((int)Unsafe.Add(ref leftBase, basis0 + 1), Unsafe.Add(ref leftBase, basis1 + 1), Unsafe.Add(ref leftBase, basis2 + 1), Unsafe.Add(ref leftBase, basis3 + 1)); + Vector128 weights = Vector128.Create( + ((projection0 << upsampleShift) & 0x3F) >> 1, + ((projection1 << upsampleShift) & 0x3F) >> 1, + ((projection2 << upsampleShift) & 0x3F) >> 1, + ((projection3 << upsampleShift) & 0x3F) >> 1); + + StoreFourShorts(TOperator.Interpolate(source0, source1, weights), ref Unsafe.Add(ref destinationBase, index)); + } + } + + for (; index < destination.Length; index++) + { + int currentProjection = projection - (index * derivative); + int basis = currentProjection >> fractionBits; + int weight = ((currentProjection << upsampleShift) & 0x3F) >> 1; + Unsafe.Add(ref destinationBase, index) = (short)(((Unsafe.Add(ref leftBase, basis) * (32 - weight)) + (Unsafe.Add(ref leftBase, basis + 1) * weight) + 16) >> 5); + } + } + + /// + /// Stores four widened predictions as packed 8-bit samples. + /// + /// The widened predictions. + /// The first destination sample. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static void StoreFourBytes(Vector128 prediction, ref byte destination) + { + Vector128 narrowed16 = Vector128.Narrow(prediction.AsUInt32(), Vector128.Zero); + Vector128 narrowed8 = Vector128.Narrow(narrowed16, Vector128.Zero); + Unsafe.As(ref destination) = narrowed8.AsUInt32().GetElement(0); + } + + /// + /// Stores four widened predictions as packed high-bit-depth samples. + /// + /// The widened predictions. + /// The first destination sample. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static void StoreFourShorts(Vector128 prediction, ref short destination) + { + Vector128 narrowed = Vector128.Narrow(prediction, Vector128.Zero); + Unsafe.As(ref destination) = narrowed.AsUInt64().ToScalar(); + } + } + + /// + /// Implements directional intra prediction through one closed interpolation operator. + /// + /// The directional interpolation arithmetic. + private static partial class Predictor + where TOperator : struct, IDirectionalPredictionOperator + { + /// + /// Gets the Q8 directional derivatives indexed by acute prediction angle. + /// + private static ReadOnlySpan DirectionalIntraDerivative => + [ + + // Zero entries represent angles which AV1 never signals. Direct indexing avoids a search or division in + // each directional block while retaining the exact fixed-point projections from the normative table. + 0, 0, 0, 1023, 0, 0, 547, 0, 0, 372, 0, 0, 0, 0, 273, 0, 0, 215, 0, 0, 178, 0, 0, + 151, 0, 0, 132, 0, 0, 116, 0, 0, 102, 0, 0, 0, 90, 0, 0, 80, 0, 0, 71, 0, 0, 64, 0, 0, + 57, 0, 0, 51, 0, 0, 45, 0, 0, 0, 40, 0, 0, 35, 0, 0, 31, 0, 0, 27, 0, 0, 23, 0, 0, + 19, 0, 0, 15, 0, 0, 0, 0, 11, 0, 0, 7, 0, 0, 3, 0, 0, + ]; + + /// + /// Predicts an 8-bit directional block using the widest available SIMD path. + /// + /// The destination block origin. + /// The destination row stride in samples. + /// The predicted block dimensions. + /// The prepared top reference, including any required extension. + /// The prepared left reference, including any required extension. + /// Whether the top edge contains half-sample positions. + /// Whether the left edge contains half-sample positions. + /// The adjusted prediction angle. + /// The caller-owned block transposition workspace. + public static void Predict(Span destination, int destinationStride, Av1TransformSize transformSize, ReadOnlySpan above, ReadOnlySpan left, bool upsampleAbove, bool upsampleLeft, int angle, Span scratch) + { + int width = transformSize.GetWidth(); + int height = transformSize.GetHeight(); + + if (angle is > 0 and < 90) + { + PredictZone1(destination, destinationStride, above, upsampleAbove, GetDeltaX(angle), width, height); + } + else if (angle is > 90 and < 180) + { + PredictZone2(destination, destinationStride, above, left, upsampleAbove, upsampleLeft, GetDeltaX(angle), GetDeltaY(angle), width, height); + } + else if (angle is > 180 and < 270) + { + // the reference decoder computes zone 3 as a zone 1 block with swapped dimensions, then transposes it. This preserves + // contiguous reference reads and destination stores in both hot stages instead of scattering columns. + Span transposed = scratch[..(width * height)]; + PredictZone1(transposed, height, left, upsampleLeft, GetDeltaY(angle), height, width); + Transpose(transposed, destination, height, width, destinationStride); + } + else + { + Av1PredictionMode mode = angle == 90 ? Av1PredictionMode.Vertical : Av1PredictionMode.Horizontal; + Av1NonDirectionalIntraPredictorBase.GetPredictor(mode).Predict(destination, destinationStride, above, left, width, height); + } + } + + /// + /// Predicts a high-bit-depth directional block using the widest available SIMD path. + /// + /// The destination block origin. + /// The destination row stride in samples. + /// The predicted block dimensions. + /// The prepared top reference, including any required extension. + /// The prepared left reference, including any required extension. + /// Whether the top edge contains half-sample positions. + /// Whether the left edge contains half-sample positions. + /// The adjusted prediction angle. + /// The caller-owned block transposition workspace. + public static void Predict(Span destination, int destinationStride, Av1TransformSize transformSize, ReadOnlySpan above, ReadOnlySpan left, bool upsampleAbove, bool upsampleLeft, int angle, Span scratch) + { + int width = transformSize.GetWidth(); + int height = transformSize.GetHeight(); + + if (angle is > 0 and < 90) + { + PredictZone1(destination, destinationStride, above, upsampleAbove, GetDeltaX(angle), width, height); + } + else if (angle is > 90 and < 180) + { + PredictZone2(destination, destinationStride, above, left, upsampleAbove, upsampleLeft, GetDeltaX(angle), GetDeltaY(angle), width, height); + } + else if (angle is > 180 and < 270) + { + Span transposed = scratch[..(width * height)]; + PredictZone1(transposed, height, left, upsampleLeft, GetDeltaY(angle), height, width); + Transpose(transposed, destination, height, width, destinationStride); + } + else + { + Av1PredictionMode mode = angle == 90 ? Av1PredictionMode.Vertical : Av1PredictionMode.Horizontal; + Av1NonDirectionalIntraPredictorBase.GetPredictor(mode).Predict(destination, destinationStride, above, left, width, height); + } + } + + /// + /// Predicts an 8-bit directional block without hardware intrinsics. + /// + /// The destination block origin. + /// The destination row stride in samples. + /// The predicted block dimensions. + /// The prepared top reference, including any required extension. + /// The prepared left reference, including any required extension. + /// Whether the top edge contains half-sample positions. + /// Whether the left edge contains half-sample positions. + /// The adjusted prediction angle. + public static void PredictScalar(Span destination, int destinationStride, Av1TransformSize transformSize, ReadOnlySpan above, ReadOnlySpan left, bool upsampleAbove, bool upsampleLeft, int angle) + { + int width = transformSize.GetWidth(); + int height = transformSize.GetHeight(); + + if (angle is > 0 and < 90) + { + PredictZone1Scalar(destination, destinationStride, above, upsampleAbove, GetDeltaX(angle), width, height); + } + else if (angle is > 90 and < 180) + { + PredictZone2Scalar(destination, destinationStride, above, left, upsampleAbove, upsampleLeft, GetDeltaX(angle), GetDeltaY(angle), width, height); + } + else if (angle is > 180 and < 270) + { + PredictZone3Scalar(destination, destinationStride, left, upsampleLeft, GetDeltaY(angle), width, height); + } + else + { + Av1PredictionMode mode = angle == 90 ? Av1PredictionMode.Vertical : Av1PredictionMode.Horizontal; + Av1NonDirectionalIntraPredictorBase.GetPredictor(mode).PredictScalar(destination, destinationStride, above, left, width, height); + } + } + + /// + /// Predicts a high-bit-depth directional block without hardware intrinsics. + /// + /// The destination block origin. + /// The destination row stride in samples. + /// The predicted block dimensions. + /// The prepared top reference, including any required extension. + /// The prepared left reference, including any required extension. + /// Whether the top edge contains half-sample positions. + /// Whether the left edge contains half-sample positions. + /// The adjusted prediction angle. + public static void PredictScalar(Span destination, int destinationStride, Av1TransformSize transformSize, ReadOnlySpan above, ReadOnlySpan left, bool upsampleAbove, bool upsampleLeft, int angle) + { + int width = transformSize.GetWidth(); + int height = transformSize.GetHeight(); + + if (angle is > 0 and < 90) + { + PredictZone1Scalar(destination, destinationStride, above, upsampleAbove, GetDeltaX(angle), width, height); + } + else if (angle is > 90 and < 180) + { + PredictZone2Scalar(destination, destinationStride, above, left, upsampleAbove, upsampleLeft, GetDeltaX(angle), GetDeltaY(angle), width, height); + } + else if (angle is > 180 and < 270) + { + PredictZone3Scalar(destination, destinationStride, left, upsampleLeft, GetDeltaY(angle), width, height); + } + else + { + Av1PredictionMode mode = angle == 90 ? Av1PredictionMode.Vertical : Av1PredictionMode.Horizontal; + Av1NonDirectionalIntraPredictorBase.GetPredictor(mode).PredictScalar(destination, destinationStride, above, left, width, height); + } + } + + /// + /// Gets the horizontal Q8 projection derivative for an adjusted angle. + /// + /// The adjusted prediction angle. + /// The horizontal derivative, or one when the selected zone does not consume it. + public static int GetDeltaX(int angle) + => angle switch + { + > 0 and < 90 => DirectionalIntraDerivative[angle], + > 90 and < 180 => DirectionalIntraDerivative[180 - angle], + _ => 1, + }; + + /// + /// Gets the vertical Q8 projection derivative for an adjusted angle. + /// + /// The adjusted prediction angle. + /// The vertical derivative, or one when the selected zone does not consume it. + public static int GetDeltaY(int angle) + => angle switch + { + > 90 and < 180 => DirectionalIntraDerivative[angle - 90], + > 180 and < 270 => DirectionalIntraDerivative[270 - angle], + _ => 1, + }; + + /// + /// Predicts one 8-bit zone 1 block without hardware intrinsics. + /// + /// The destination block origin. + /// The destination row stride. + /// The projected top reference. + /// Whether the reference contains half-sample positions. + /// The Q8 projection derivative. + /// The block width. + /// The block height. + private static void PredictZone1Scalar(Span destination, int destinationStride, ReadOnlySpan above, bool upsample, int derivative, int width, int height) + { + int upsampleShift = upsample ? 1 : 0; + int maximumBasis = (width + height - 1) << upsampleShift; + int fractionBits = 6 - upsampleShift; + int basisIncrement = 1 << upsampleShift; + int projection = derivative; + ref byte aboveBase = ref Unsafe.AsRef(in above[0]); + + for (int row = 0; row < height; row++, projection += derivative) + { + int basis = projection >> fractionBits; + int weight = ((projection << upsampleShift) & 0x3F) >> 1; + ref byte destinationRow = ref destination[row * destinationStride]; + + for (int column = 0; column < width; column++, basis += basisIncrement) + { + Unsafe.Add(ref destinationRow, column) = basis < maximumBasis + ? TOperator.Interpolate(Unsafe.Add(ref aboveBase, basis), Unsafe.Add(ref aboveBase, basis + 1), weight) + : Unsafe.Add(ref aboveBase, maximumBasis); + } + } + } + + /// + /// Predicts one high-bit-depth zone 1 block without hardware intrinsics. + /// + /// The destination block origin. + /// The destination row stride. + /// The projected top reference. + /// Whether the reference contains half-sample positions. + /// The Q8 projection derivative. + /// The block width. + /// The block height. + private static void PredictZone1Scalar(Span destination, int destinationStride, ReadOnlySpan above, bool upsample, int derivative, int width, int height) + { + int upsampleShift = upsample ? 1 : 0; + int maximumBasis = (width + height - 1) << upsampleShift; + int fractionBits = 6 - upsampleShift; + int basisIncrement = 1 << upsampleShift; + int projection = derivative; + ref short aboveBase = ref Unsafe.AsRef(in above[0]); + + for (int row = 0; row < height; row++, projection += derivative) + { + int basis = projection >> fractionBits; + int weight = ((projection << upsampleShift) & 0x3F) >> 1; + ref short destinationRow = ref destination[row * destinationStride]; + + for (int column = 0; column < width; column++, basis += basisIncrement) + { + Unsafe.Add(ref destinationRow, column) = basis < maximumBasis + ? TOperator.Interpolate(Unsafe.Add(ref aboveBase, basis), Unsafe.Add(ref aboveBase, basis + 1), weight) + : Unsafe.Add(ref aboveBase, maximumBasis); + } + } + } + + /// + /// Predicts one 8-bit zone 2 block without hardware intrinsics. + /// + /// The destination block origin. + /// The destination row stride. + /// The projected top reference. + /// The projected left reference. + /// Whether the top reference contains half-sample positions. + /// Whether the left reference contains half-sample positions. + /// The horizontal Q8 derivative. + /// The vertical Q8 derivative. + /// The block width. + /// The block height. + private static void PredictZone2Scalar(Span destination, int destinationStride, ReadOnlySpan above, ReadOnlySpan left, bool upsampleAbove, bool upsampleLeft, int dx, int dy, int width, int height) + { + int aboveShift = upsampleAbove ? 1 : 0; + int leftShift = upsampleLeft ? 1 : 0; + int minimumTopBasis = -(1 << aboveShift); + int topFractionBits = 6 - aboveShift; + int leftFractionBits = 6 - leftShift; + int topBasisIncrement = 1 << aboveShift; + int topProjection = -dx; + ref byte aboveBase = ref Unsafe.AsRef(in above[0]); + ref byte leftBase = ref Unsafe.AsRef(in left[0]); + + for (int row = 0; row < height; row++, topProjection -= dx) + { + int topBasis = topProjection >> topFractionBits; + int topWeight = ((topProjection << aboveShift) & 0x3F) >> 1; + int leftProjection = (row << 6) - dy; + ref byte destinationRow = ref destination[row * destinationStride]; + + for (int column = 0; column < width; column++, topBasis += topBasisIncrement, leftProjection -= dy) + { + byte prediction; + if (topBasis >= minimumTopBasis) + { + prediction = TOperator.Interpolate(Unsafe.Add(ref aboveBase, topBasis), Unsafe.Add(ref aboveBase, topBasis + 1), topWeight); + } + else + { + int leftBasis = leftProjection >> leftFractionBits; + int leftWeight = ((leftProjection << leftShift) & 0x3F) >> 1; + prediction = TOperator.Interpolate(Unsafe.Add(ref leftBase, leftBasis), Unsafe.Add(ref leftBase, leftBasis + 1), leftWeight); + } + + Unsafe.Add(ref destinationRow, column) = prediction; + } + } + } + + /// + /// Predicts one high-bit-depth zone 2 block without hardware intrinsics. + /// + /// The destination block origin. + /// The destination row stride. + /// The projected top reference. + /// The projected left reference. + /// Whether the top reference contains half-sample positions. + /// Whether the left reference contains half-sample positions. + /// The horizontal Q8 derivative. + /// The vertical Q8 derivative. + /// The block width. + /// The block height. + private static void PredictZone2Scalar(Span destination, int destinationStride, ReadOnlySpan above, ReadOnlySpan left, bool upsampleAbove, bool upsampleLeft, int dx, int dy, int width, int height) + { + int aboveShift = upsampleAbove ? 1 : 0; + int leftShift = upsampleLeft ? 1 : 0; + int minimumTopBasis = -(1 << aboveShift); + int topFractionBits = 6 - aboveShift; + int leftFractionBits = 6 - leftShift; + int topBasisIncrement = 1 << aboveShift; + int topProjection = -dx; + ref short aboveBase = ref Unsafe.AsRef(in above[0]); + ref short leftBase = ref Unsafe.AsRef(in left[0]); + + for (int row = 0; row < height; row++, topProjection -= dx) + { + int topBasis = topProjection >> topFractionBits; + int topWeight = ((topProjection << aboveShift) & 0x3F) >> 1; + int leftProjection = (row << 6) - dy; + ref short destinationRow = ref destination[row * destinationStride]; + + for (int column = 0; column < width; column++, topBasis += topBasisIncrement, leftProjection -= dy) + { + short prediction; + if (topBasis >= minimumTopBasis) + { + prediction = TOperator.Interpolate(Unsafe.Add(ref aboveBase, topBasis), Unsafe.Add(ref aboveBase, topBasis + 1), topWeight); + } + else + { + int leftBasis = leftProjection >> leftFractionBits; + int leftWeight = ((leftProjection << leftShift) & 0x3F) >> 1; + prediction = TOperator.Interpolate(Unsafe.Add(ref leftBase, leftBasis), Unsafe.Add(ref leftBase, leftBasis + 1), leftWeight); + } + + Unsafe.Add(ref destinationRow, column) = prediction; + } + } + } + + /// + /// Predicts one 8-bit zone 3 block without hardware intrinsics. + /// + /// The destination block origin. + /// The destination row stride. + /// The projected left reference. + /// Whether the reference contains half-sample positions. + /// The Q8 projection derivative. + /// The block width. + /// The block height. + private static void PredictZone3Scalar(Span destination, int destinationStride, ReadOnlySpan left, bool upsample, int derivative, int width, int height) + { + int upsampleShift = upsample ? 1 : 0; + int maximumBasis = (width + height - 1) << upsampleShift; + int fractionBits = 6 - upsampleShift; + int basisIncrement = 1 << upsampleShift; + int projection = derivative; + ref byte leftBase = ref Unsafe.AsRef(in left[0]); + + for (int column = 0; column < width; column++, projection += derivative) + { + int basis = projection >> fractionBits; + int weight = ((projection << upsampleShift) & 0x3F) >> 1; + for (int row = 0; row < height; row++, basis += basisIncrement) + { + destination[(row * destinationStride) + column] = basis < maximumBasis + ? TOperator.Interpolate(Unsafe.Add(ref leftBase, basis), Unsafe.Add(ref leftBase, basis + 1), weight) + : Unsafe.Add(ref leftBase, maximumBasis); + } + } + } + + /// + /// Predicts one high-bit-depth zone 3 block without hardware intrinsics. + /// + /// The destination block origin. + /// The destination row stride. + /// The projected left reference. + /// Whether the reference contains half-sample positions. + /// The Q8 projection derivative. + /// The block width. + /// The block height. + private static void PredictZone3Scalar(Span destination, int destinationStride, ReadOnlySpan left, bool upsample, int derivative, int width, int height) + { + int upsampleShift = upsample ? 1 : 0; + int maximumBasis = (width + height - 1) << upsampleShift; + int fractionBits = 6 - upsampleShift; + int basisIncrement = 1 << upsampleShift; + int projection = derivative; + ref short leftBase = ref Unsafe.AsRef(in left[0]); + + for (int column = 0; column < width; column++, projection += derivative) + { + int basis = projection >> fractionBits; + int weight = ((projection << upsampleShift) & 0x3F) >> 1; + for (int row = 0; row < height; row++, basis += basisIncrement) + { + destination[(row * destinationStride) + column] = basis < maximumBasis + ? TOperator.Interpolate(Unsafe.Add(ref leftBase, basis), Unsafe.Add(ref leftBase, basis + 1), weight) + : Unsafe.Add(ref leftBase, maximumBasis); + } + } + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1DirectionalIntraPredictor.Operator.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1DirectionalIntraPredictor.Operator.cs new file mode 100644 index 0000000000..6815603392 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1DirectionalIntraPredictor.Operator.cs @@ -0,0 +1,239 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.Intrinsics; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; + +/// +/// Defines Q5 interpolation for AV1 directional intra prediction. +/// +internal static partial class Av1DirectionalIntraPredictor +{ + /// + /// The largest number of samples required to transpose a directional prediction block. + /// + public const int ScratchLength = 64 * 64; + + /// + /// Defines scalar and SIMD interpolation for AV1 directional intra prediction. + /// + internal interface IDirectionalPredictionOperator + { + /// + /// Interpolates one 8-bit pair. + /// + /// The first reference sample. + /// The second reference sample. + /// The Q5 weight of . + /// The interpolated sample. + public static abstract byte Interpolate(byte left, byte right, int weight); + + /// + /// Interpolates one high-bit-depth pair. + /// + /// The first reference sample. + /// The second reference sample. + /// The Q5 weight of . + /// The interpolated sample. + public static abstract short Interpolate(short left, short right, int weight); + + /// + /// Interpolates sixteen 8-bit pairs. + /// + /// The first reference samples. + /// The second reference samples. + /// The Q5 weight of . + /// The interpolated samples. + public static abstract Vector128 Interpolate(Vector128 left, Vector128 right, int weight); + + /// + /// Interpolates thirty-two 8-bit pairs. + /// + /// The first reference samples. + /// The second reference samples. + /// The Q5 weight of . + /// The interpolated samples. + public static abstract Vector256 Interpolate(Vector256 left, Vector256 right, int weight); + + /// + /// Interpolates sixty-four 8-bit pairs. + /// + /// The first reference samples. + /// The second reference samples. + /// The Q5 weight of . + /// The interpolated samples. + public static abstract Vector512 Interpolate(Vector512 left, Vector512 right, int weight); + + /// + /// Interpolates eight high-bit-depth pairs. + /// + /// The first reference samples. + /// The second reference samples. + /// The Q5 weight of . + /// The interpolated samples. + public static abstract Vector128 Interpolate(Vector128 left, Vector128 right, int weight); + + /// + /// Interpolates sixteen high-bit-depth pairs. + /// + /// The first reference samples. + /// The second reference samples. + /// The Q5 weight of . + /// The interpolated samples. + public static abstract Vector256 Interpolate(Vector256 left, Vector256 right, int weight); + + /// + /// Interpolates thirty-two high-bit-depth pairs. + /// + /// The first reference samples. + /// The second reference samples. + /// The Q5 weight of . + /// The interpolated samples. + public static abstract Vector512 Interpolate(Vector512 left, Vector512 right, int weight); + + /// + /// Interpolates four widened pairs with independent weights. + /// + /// The first reference samples. + /// The second reference samples. + /// The Q5 weights of . + /// The interpolated samples. + public static abstract Vector128 Interpolate(Vector128 left, Vector128 right, Vector128 weights); + } + + /// + /// Gets the horizontal Q8 projection derivative for an adjusted angle. + /// + public static int GetDeltaX(int angle) => Predictor.GetDeltaX(angle); + + /// + /// Gets the vertical Q8 projection derivative for an adjusted angle. + /// + public static int GetDeltaY(int angle) => Predictor.GetDeltaY(angle); + + /// + /// Predicts an 8-bit directional block. + /// + public static void Predict(Span destination, int destinationStride, Av1TransformSize transformSize, ReadOnlySpan above, ReadOnlySpan left, bool upsampleAbove, bool upsampleLeft, int angle, Span scratch) + => Predictor.Predict(destination, destinationStride, transformSize, above, left, upsampleAbove, upsampleLeft, angle, scratch); + + /// + /// Predicts a high-bit-depth directional block. + /// + public static void Predict(Span destination, int destinationStride, Av1TransformSize transformSize, ReadOnlySpan above, ReadOnlySpan left, bool upsampleAbove, bool upsampleLeft, int angle, Span scratch) + => Predictor.Predict(destination, destinationStride, transformSize, above, left, upsampleAbove, upsampleLeft, angle, scratch); + + /// + /// Predicts an 8-bit directional block without hardware intrinsics. + /// + public static void PredictScalar(Span destination, int destinationStride, Av1TransformSize transformSize, ReadOnlySpan above, ReadOnlySpan left, bool upsampleAbove, bool upsampleLeft, int angle) + => Predictor.PredictScalar(destination, destinationStride, transformSize, above, left, upsampleAbove, upsampleLeft, angle); + + /// + /// Predicts a high-bit-depth directional block without hardware intrinsics. + /// + public static void PredictScalar(Span destination, int destinationStride, Av1TransformSize transformSize, ReadOnlySpan above, ReadOnlySpan left, bool upsampleAbove, bool upsampleLeft, int angle) + => Predictor.PredictScalar(destination, destinationStride, transformSize, above, left, upsampleAbove, upsampleLeft, angle); + + /// + /// Interpolates projected neighboring samples for all directional prediction zones. + /// + internal readonly struct DirectionalOperator : IDirectionalPredictionOperator + { + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static byte Interpolate(byte left, byte right, int weight) + => (byte)(((left * (32 - weight)) + (right * weight) + 16) >> 5); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static short Interpolate(short left, short right, int weight) + => (short)(((left * (32 - weight)) + (right * weight) + 16) >> 5); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector128 Interpolate(Vector128 left, Vector128 right, int weight) + { + (Vector128 leftLow, Vector128 leftHigh) = Vector128.Widen(left); + (Vector128 rightLow, Vector128 rightHigh) = Vector128.Widen(right); + Vector128 rounding = Vector128.Create((ushort)16); + Vector128 low = ((leftLow * (ushort)(32 - weight)) + (rightLow * (ushort)weight) + rounding) >> 5; + Vector128 high = ((leftHigh * (ushort)(32 - weight)) + (rightHigh * (ushort)weight) + rounding) >> 5; + + return Vector128.Narrow(low, high); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector256 Interpolate(Vector256 left, Vector256 right, int weight) + { + (Vector256 leftLow, Vector256 leftHigh) = Vector256.Widen(left); + (Vector256 rightLow, Vector256 rightHigh) = Vector256.Widen(right); + Vector256 rounding = Vector256.Create((ushort)16); + Vector256 low = ((leftLow * (ushort)(32 - weight)) + (rightLow * (ushort)weight) + rounding) >> 5; + Vector256 high = ((leftHigh * (ushort)(32 - weight)) + (rightHigh * (ushort)weight) + rounding) >> 5; + + return Vector256.Narrow(low, high); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector512 Interpolate(Vector512 left, Vector512 right, int weight) + { + (Vector512 leftLow, Vector512 leftHigh) = Vector512.Widen(left); + (Vector512 rightLow, Vector512 rightHigh) = Vector512.Widen(right); + Vector512 rounding = Vector512.Create((ushort)16); + Vector512 low = ((leftLow * (ushort)(32 - weight)) + (rightLow * (ushort)weight) + rounding) >> 5; + Vector512 high = ((leftHigh * (ushort)(32 - weight)) + (rightHigh * (ushort)weight) + rounding) >> 5; + + return Vector512.Narrow(low, high); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector128 Interpolate(Vector128 left, Vector128 right, int weight) + { + (Vector128 leftLow, Vector128 leftHigh) = Vector128.Widen(left); + (Vector128 rightLow, Vector128 rightHigh) = Vector128.Widen(right); + Vector128 rounding = Vector128.Create(16); + Vector128 low = ((leftLow * (32 - weight)) + (rightLow * weight) + rounding) >> 5; + Vector128 high = ((leftHigh * (32 - weight)) + (rightHigh * weight) + rounding) >> 5; + + return Vector128.Narrow(low, high); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector256 Interpolate(Vector256 left, Vector256 right, int weight) + { + (Vector256 leftLow, Vector256 leftHigh) = Vector256.Widen(left); + (Vector256 rightLow, Vector256 rightHigh) = Vector256.Widen(right); + Vector256 rounding = Vector256.Create(16); + Vector256 low = ((leftLow * (32 - weight)) + (rightLow * weight) + rounding) >> 5; + Vector256 high = ((leftHigh * (32 - weight)) + (rightHigh * weight) + rounding) >> 5; + + return Vector256.Narrow(low, high); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector512 Interpolate(Vector512 left, Vector512 right, int weight) + { + (Vector512 leftLow, Vector512 leftHigh) = Vector512.Widen(left); + (Vector512 rightLow, Vector512 rightHigh) = Vector512.Widen(right); + Vector512 rounding = Vector512.Create(16); + Vector512 low = ((leftLow * (32 - weight)) + (rightLow * weight) + rounding) >> 5; + Vector512 high = ((leftHigh * (32 - weight)) + (rightHigh * weight) + rounding) >> 5; + + return Vector512.Narrow(low, high); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector128 Interpolate(Vector128 left, Vector128 right, Vector128 weights) + => ((left * (Vector128.Create(32) - weights)) + (right * weights) + Vector128.Create(16)) >> 5; + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1DirectionalIntraPredictor.Transpose.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1DirectionalIntraPredictor.Transpose.cs new file mode 100644 index 0000000000..96e1b9b484 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1DirectionalIntraPredictor.Transpose.cs @@ -0,0 +1,326 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.InteropServices; +using System.Runtime.Intrinsics; +using SixLabors.ImageSharp.Common.Helpers; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; + +/// +/// Provides SIMD block transposition for zone 3 directional prediction. AV1 block dimensions are multiples of four, +/// allowing the traversal to use complete eight-by-eight tiles where possible and complete four-by-four tiles for the +/// remaining small blocks. Unpack stages exchange coordinate bits inside registers; exact-width loads and stores keep +/// every access within the logical block even when the destination has no writable row padding. +/// +internal static partial class Av1DirectionalIntraPredictor +{ + /// + /// Implements the directional traversal for one closed interpolation operator. + /// + private static partial class Predictor + where TOperator : struct, IDirectionalPredictionOperator + { + /// + /// Transposes a contiguous 8-bit prediction block into the reconstructed destination. + /// + /// The contiguous source block. + /// The destination block origin. + /// The source row width. + /// The number of source rows. + /// The destination row stride. + private static void Transpose(ReadOnlySpan source, Span destination, int sourceWidth, int sourceHeight, int destinationStride) + { + if (Vector128.IsHardwareAccelerated) + { + // Selecting one tile size for the complete block keeps both loop increments aligned with the AV1 block + // dimensions. No partial SIMD tile reaches a neighboring prediction block. + int tileSize = sourceWidth >= 8 && sourceHeight >= 8 ? 8 : 4; + for (int y = 0; y < sourceHeight; y += tileSize) + { + for (int x = 0; x < sourceWidth; x += tileSize) + { + if (tileSize == 8) + { + Transpose8x8(source, destination, sourceWidth, destinationStride, x, y); + } + else + { + Transpose4x4(source, destination, sourceWidth, destinationStride, x, y); + } + } + } + + return; + } + + for (int y = 0; y < sourceHeight; y++) + { + for (int x = 0; x < sourceWidth; x++) + { + destination[(x * destinationStride) + y] = source[(y * sourceWidth) + x]; + } + } + } + + /// + /// Transposes a contiguous high-bit-depth prediction block into the reconstructed destination. + /// + /// The contiguous source block. + /// The destination block origin. + /// The source row width. + /// The number of source rows. + /// The destination row stride. + private static void Transpose(ReadOnlySpan source, Span destination, int sourceWidth, int sourceHeight, int destinationStride) + { + if (Vector128.IsHardwareAccelerated) + { + // The same tiling invariant applies to two-byte samples; only the register unpack granularity differs. + int tileSize = sourceWidth >= 8 && sourceHeight >= 8 ? 8 : 4; + for (int y = 0; y < sourceHeight; y += tileSize) + { + for (int x = 0; x < sourceWidth; x += tileSize) + { + if (tileSize == 8) + { + Transpose8x8(source, destination, sourceWidth, destinationStride, x, y); + } + else + { + Transpose4x4(source, destination, sourceWidth, destinationStride, x, y); + } + } + } + + return; + } + + for (int y = 0; y < sourceHeight; y++) + { + for (int x = 0; x < sourceWidth; x++) + { + destination[(x * destinationStride) + y] = source[(y * sourceWidth) + x]; + } + } + } + + /// + /// Transposes one eight-by-eight tile of 8-bit prediction samples. + /// + /// The contiguous source block. + /// The destination block origin. + /// The source row stride. + /// The destination row stride. + /// The tile X coordinate. + /// The tile Y coordinate. + private static void Transpose8x8(ReadOnlySpan source, Span destination, int sourceStride, int destinationStride, int x, int y) + { + ref byte sourceBase = ref MemoryMarshal.GetReference(source); + ref byte destinationBase = ref MemoryMarshal.GetReference(destination); + Vector128 row0 = LoadEightBytes(ref sourceBase, ((y + 0) * sourceStride) + x); + Vector128 row1 = LoadEightBytes(ref sourceBase, ((y + 1) * sourceStride) + x); + Vector128 row2 = LoadEightBytes(ref sourceBase, ((y + 2) * sourceStride) + x); + Vector128 row3 = LoadEightBytes(ref sourceBase, ((y + 3) * sourceStride) + x); + Vector128 row4 = LoadEightBytes(ref sourceBase, ((y + 4) * sourceStride) + x); + Vector128 row5 = LoadEightBytes(ref sourceBase, ((y + 5) * sourceStride) + x); + Vector128 row6 = LoadEightBytes(ref sourceBase, ((y + 6) * sourceStride) + x); + Vector128 row7 = LoadEightBytes(ref sourceBase, ((y + 7) * sourceStride) + x); + + // Three unpack stages exchange one, two, then four byte coordinates. Each final vector contains two + // complete source columns, which are written as two contiguous eight-byte destination rows. + Vector128 pair0 = Vector128_.UnpackLow(row0, row1); + Vector128 pair2 = Vector128_.UnpackLow(row2, row3); + Vector128 pair4 = Vector128_.UnpackLow(row4, row5); + Vector128 pair6 = Vector128_.UnpackLow(row6, row7); + Vector128 quad0 = Vector128_.UnpackLow(pair0.AsInt16(), pair2.AsInt16()); + Vector128 quad1 = Vector128_.UnpackHigh(pair0.AsInt16(), pair2.AsInt16()); + Vector128 quad4 = Vector128_.UnpackLow(pair4.AsInt16(), pair6.AsInt16()); + Vector128 quad5 = Vector128_.UnpackHigh(pair4.AsInt16(), pair6.AsInt16()); + Vector128 columns01 = Vector128_.UnpackLow(quad0.AsInt32(), quad4.AsInt32()); + Vector128 columns23 = Vector128_.UnpackHigh(quad0.AsInt32(), quad4.AsInt32()); + Vector128 columns45 = Vector128_.UnpackLow(quad1.AsInt32(), quad5.AsInt32()); + Vector128 columns67 = Vector128_.UnpackHigh(quad1.AsInt32(), quad5.AsInt32()); + + StoreEightBytes(columns01.AsUInt64().ToScalar(), ref destinationBase, ((x + 0) * destinationStride) + y); + StoreEightBytes(columns01.AsUInt64().GetElement(1), ref destinationBase, ((x + 1) * destinationStride) + y); + StoreEightBytes(columns23.AsUInt64().ToScalar(), ref destinationBase, ((x + 2) * destinationStride) + y); + StoreEightBytes(columns23.AsUInt64().GetElement(1), ref destinationBase, ((x + 3) * destinationStride) + y); + StoreEightBytes(columns45.AsUInt64().ToScalar(), ref destinationBase, ((x + 4) * destinationStride) + y); + StoreEightBytes(columns45.AsUInt64().GetElement(1), ref destinationBase, ((x + 5) * destinationStride) + y); + StoreEightBytes(columns67.AsUInt64().ToScalar(), ref destinationBase, ((x + 6) * destinationStride) + y); + StoreEightBytes(columns67.AsUInt64().GetElement(1), ref destinationBase, ((x + 7) * destinationStride) + y); + } + + /// + /// Transposes one four-by-four tile of 8-bit prediction samples. + /// + /// The contiguous source block. + /// The destination block origin. + /// The source row stride. + /// The destination row stride. + /// The tile X coordinate. + /// The tile Y coordinate. + private static void Transpose4x4(ReadOnlySpan source, Span destination, int sourceStride, int destinationStride, int x, int y) + { + ref byte sourceBase = ref MemoryMarshal.GetReference(source); + ref byte destinationBase = ref MemoryMarshal.GetReference(destination); + Vector128 row0 = LoadFourBytes(ref sourceBase, ((y + 0) * sourceStride) + x); + Vector128 row1 = LoadFourBytes(ref sourceBase, ((y + 1) * sourceStride) + x); + Vector128 row2 = LoadFourBytes(ref sourceBase, ((y + 2) * sourceStride) + x); + Vector128 row3 = LoadFourBytes(ref sourceBase, ((y + 3) * sourceStride) + x); + Vector128 pair0 = Vector128_.UnpackLow(row0, row1); + Vector128 pair1 = Vector128_.UnpackLow(row2, row3); + Vector128 columns = Vector128_.UnpackLow(pair0.AsInt16(), pair1.AsInt16()); + + Vector128 packedColumns = columns.AsUInt32(); + StoreFourBytes(packedColumns.GetElement(0), ref destinationBase, ((x + 0) * destinationStride) + y); + StoreFourBytes(packedColumns.GetElement(1), ref destinationBase, ((x + 1) * destinationStride) + y); + StoreFourBytes(packedColumns.GetElement(2), ref destinationBase, ((x + 2) * destinationStride) + y); + StoreFourBytes(packedColumns.GetElement(3), ref destinationBase, ((x + 3) * destinationStride) + y); + } + + /// + /// Transposes one eight-by-eight tile of high-bit-depth prediction samples. + /// + /// The contiguous source block. + /// The destination block origin. + /// The source row stride. + /// The destination row stride. + /// The tile X coordinate. + /// The tile Y coordinate. + private static void Transpose8x8(ReadOnlySpan source, Span destination, int sourceStride, int destinationStride, int x, int y) + { + ref short sourceBase = ref MemoryMarshal.GetReference(source); + ref short destinationBase = ref MemoryMarshal.GetReference(destination); + Vector128 row0 = Vector128.LoadUnsafe(ref sourceBase, (nuint)(((y + 0) * sourceStride) + x)); + Vector128 row1 = Vector128.LoadUnsafe(ref sourceBase, (nuint)(((y + 1) * sourceStride) + x)); + Vector128 row2 = Vector128.LoadUnsafe(ref sourceBase, (nuint)(((y + 2) * sourceStride) + x)); + Vector128 row3 = Vector128.LoadUnsafe(ref sourceBase, (nuint)(((y + 3) * sourceStride) + x)); + Vector128 row4 = Vector128.LoadUnsafe(ref sourceBase, (nuint)(((y + 4) * sourceStride) + x)); + Vector128 row5 = Vector128.LoadUnsafe(ref sourceBase, (nuint)(((y + 5) * sourceStride) + x)); + Vector128 row6 = Vector128.LoadUnsafe(ref sourceBase, (nuint)(((y + 6) * sourceStride) + x)); + Vector128 row7 = Vector128.LoadUnsafe(ref sourceBase, (nuint)(((y + 7) * sourceStride) + x)); + Vector128 pair0 = Vector128_.UnpackLow(row0, row1); + Vector128 pair1 = Vector128_.UnpackHigh(row0, row1); + Vector128 pair2 = Vector128_.UnpackLow(row2, row3); + Vector128 pair3 = Vector128_.UnpackHigh(row2, row3); + Vector128 pair4 = Vector128_.UnpackLow(row4, row5); + Vector128 pair5 = Vector128_.UnpackHigh(row4, row5); + Vector128 pair6 = Vector128_.UnpackLow(row6, row7); + Vector128 pair7 = Vector128_.UnpackHigh(row6, row7); + Vector128 quad0 = Vector128_.UnpackLow(pair0.AsInt32(), pair2.AsInt32()); + Vector128 quad1 = Vector128_.UnpackHigh(pair0.AsInt32(), pair2.AsInt32()); + Vector128 quad2 = Vector128_.UnpackLow(pair1.AsInt32(), pair3.AsInt32()); + Vector128 quad3 = Vector128_.UnpackHigh(pair1.AsInt32(), pair3.AsInt32()); + Vector128 quad4 = Vector128_.UnpackLow(pair4.AsInt32(), pair6.AsInt32()); + Vector128 quad5 = Vector128_.UnpackHigh(pair4.AsInt32(), pair6.AsInt32()); + Vector128 quad6 = Vector128_.UnpackLow(pair5.AsInt32(), pair7.AsInt32()); + Vector128 quad7 = Vector128_.UnpackHigh(pair5.AsInt32(), pair7.AsInt32()); + Vector128 column0 = Vector128_.UnpackLow(quad0.AsInt64(), quad4.AsInt64()).AsInt16(); + Vector128 column1 = Vector128_.UnpackHigh(quad0.AsInt64(), quad4.AsInt64()).AsInt16(); + Vector128 column2 = Vector128_.UnpackLow(quad1.AsInt64(), quad5.AsInt64()).AsInt16(); + Vector128 column3 = Vector128_.UnpackHigh(quad1.AsInt64(), quad5.AsInt64()).AsInt16(); + Vector128 column4 = Vector128_.UnpackLow(quad2.AsInt64(), quad6.AsInt64()).AsInt16(); + Vector128 column5 = Vector128_.UnpackHigh(quad2.AsInt64(), quad6.AsInt64()).AsInt16(); + Vector128 column6 = Vector128_.UnpackLow(quad3.AsInt64(), quad7.AsInt64()).AsInt16(); + Vector128 column7 = Vector128_.UnpackHigh(quad3.AsInt64(), quad7.AsInt64()).AsInt16(); + column0.StoreUnsafe(ref destinationBase, (nuint)(((x + 0) * destinationStride) + y)); + column1.StoreUnsafe(ref destinationBase, (nuint)(((x + 1) * destinationStride) + y)); + column2.StoreUnsafe(ref destinationBase, (nuint)(((x + 2) * destinationStride) + y)); + column3.StoreUnsafe(ref destinationBase, (nuint)(((x + 3) * destinationStride) + y)); + column4.StoreUnsafe(ref destinationBase, (nuint)(((x + 4) * destinationStride) + y)); + column5.StoreUnsafe(ref destinationBase, (nuint)(((x + 5) * destinationStride) + y)); + column6.StoreUnsafe(ref destinationBase, (nuint)(((x + 6) * destinationStride) + y)); + column7.StoreUnsafe(ref destinationBase, (nuint)(((x + 7) * destinationStride) + y)); + } + + /// + /// Transposes one four-by-four tile of high-bit-depth prediction samples. + /// + /// The contiguous source block. + /// The destination block origin. + /// The source row stride. + /// The destination row stride. + /// The tile X coordinate. + /// The tile Y coordinate. + private static void Transpose4x4(ReadOnlySpan source, Span destination, int sourceStride, int destinationStride, int x, int y) + { + ref short sourceBase = ref MemoryMarshal.GetReference(source); + ref short destinationBase = ref MemoryMarshal.GetReference(destination); + Vector128 row0 = LoadFourShorts(ref sourceBase, ((y + 0) * sourceStride) + x); + Vector128 row1 = LoadFourShorts(ref sourceBase, ((y + 1) * sourceStride) + x); + Vector128 row2 = LoadFourShorts(ref sourceBase, ((y + 2) * sourceStride) + x); + Vector128 row3 = LoadFourShorts(ref sourceBase, ((y + 3) * sourceStride) + x); + Vector128 pair0 = Vector128_.UnpackLow(row0, row1); + Vector128 pair1 = Vector128_.UnpackLow(row2, row3); + Vector128 columns01 = Vector128_.UnpackLow(pair0.AsInt32(), pair1.AsInt32()); + Vector128 columns23 = Vector128_.UnpackHigh(pair0.AsInt32(), pair1.AsInt32()); + + StoreFourShorts(columns01.AsUInt64().ToScalar(), ref destinationBase, ((x + 0) * destinationStride) + y); + StoreFourShorts(columns01.AsUInt64().GetElement(1), ref destinationBase, ((x + 1) * destinationStride) + y); + StoreFourShorts(columns23.AsUInt64().ToScalar(), ref destinationBase, ((x + 2) * destinationStride) + y); + StoreFourShorts(columns23.AsUInt64().GetElement(1), ref destinationBase, ((x + 3) * destinationStride) + y); + } + + /// + /// Loads eight bytes into the lower half of a vector without reading past a tile row. + /// + /// The source buffer origin. + /// The source offset. + /// The loaded bytes. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector128 LoadEightBytes(ref byte source, int offset) + => Vector128.Create(Unsafe.As(ref Unsafe.Add(ref source, offset)), 0UL).AsByte(); + + /// + /// Loads four bytes into the low vector lanes without reading past a tile row. + /// + /// The source buffer origin. + /// The source offset. + /// The loaded bytes. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector128 LoadFourBytes(ref byte source, int offset) + => Vector128.Create(Unsafe.As(ref Unsafe.Add(ref source, offset)), 0U, 0U, 0U).AsByte(); + + /// + /// Loads four high-bit-depth samples into the lower half of a vector without reading past a tile row. + /// + /// The source buffer origin. + /// The source offset. + /// The loaded samples. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector128 LoadFourShorts(ref short source, int offset) + => Vector128.Create(Unsafe.As(ref Unsafe.Add(ref source, offset)), 0UL).AsInt16(); + + /// + /// Stores the lower eight bytes of a vector. + /// + /// The packed bytes. + /// The destination buffer origin. + /// The destination offset. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static void StoreEightBytes(ulong source, ref byte destination, int offset) + => Unsafe.As(ref Unsafe.Add(ref destination, offset)) = source; + + /// + /// Stores four bytes from the low vector lanes. + /// + /// The packed bytes. + /// The destination buffer origin. + /// The destination offset. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static void StoreFourBytes(uint source, ref byte destination, int offset) + => Unsafe.As(ref Unsafe.Add(ref destination, offset)) = source; + + /// + /// Stores four high-bit-depth samples from the low vector lanes. + /// + /// The packed samples. + /// The destination buffer origin. + /// The destination offset. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static void StoreFourShorts(ulong source, ref short destination, int offset) + => Unsafe.As(ref Unsafe.Add(ref destination, offset)) = source; + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1FilterIntraPredictor.DcOperator.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1FilterIntraPredictor.DcOperator.cs new file mode 100644 index 0000000000..c696afef76 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1FilterIntraPredictor.DcOperator.cs @@ -0,0 +1,31 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; + +internal abstract partial class Av1FilterIntraPredictorBase +{ + /// + /// Defines the AV1 DC filter-intra coefficient operator. + /// + internal readonly struct DcOperator : IAv1FilterIntraPredictionOperator + { + /// + public static Av1FilterIntraMode Mode => Av1FilterIntraMode.DC; + + /// + public static ReadOnlySpan Taps => + [ + -6, 10, 0, 0, 0, 12, 0, + -5, 2, 10, 0, 0, 9, 0, + -3, 1, 1, 10, 0, 7, 0, + -3, 1, 1, 2, 10, 5, 0, + -4, 6, 0, 0, 0, 2, 12, + -3, 2, 6, 0, 0, 2, 9, + -3, 2, 2, 6, 0, 2, 7, + -3, 1, 2, 2, 6, 3, 5, + ]; + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1FilterIntraPredictor.Directional157Operator.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1FilterIntraPredictor.Directional157Operator.cs new file mode 100644 index 0000000000..5e8ddebcd1 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1FilterIntraPredictor.Directional157Operator.cs @@ -0,0 +1,31 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; + +internal abstract partial class Av1FilterIntraPredictorBase +{ + /// + /// Defines the AV1 157-degree directional filter-intra coefficient operator. + /// + internal readonly struct Directional157Operator : IAv1FilterIntraPredictionOperator + { + /// + public static Av1FilterIntraMode Mode => Av1FilterIntraMode.Directional157; + + /// + public static ReadOnlySpan Taps => + [ + -2, 8, 0, 0, 0, 10, 0, + -1, 3, 8, 0, 0, 6, 0, + -1, 2, 3, 8, 0, 4, 0, + 0, 1, 2, 3, 8, 2, 0, + -1, 4, 0, 0, 0, 3, 10, + -1, 3, 4, 0, 0, 4, 6, + -1, 2, 3, 4, 0, 4, 4, + -1, 2, 2, 3, 4, 3, 3, + ]; + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1FilterIntraPredictor.HorizontalOperator.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1FilterIntraPredictor.HorizontalOperator.cs new file mode 100644 index 0000000000..2b1da42d4e --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1FilterIntraPredictor.HorizontalOperator.cs @@ -0,0 +1,31 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; + +internal abstract partial class Av1FilterIntraPredictorBase +{ + /// + /// Defines the AV1 horizontal filter-intra coefficient operator. + /// + internal readonly struct HorizontalOperator : IAv1FilterIntraPredictionOperator + { + /// + public static Av1FilterIntraMode Mode => Av1FilterIntraMode.Horizontal; + + /// + public static ReadOnlySpan Taps => + [ + -8, 8, 0, 0, 0, 16, 0, + -8, 0, 8, 0, 0, 16, 0, + -8, 0, 0, 8, 0, 16, 0, + -8, 0, 0, 0, 8, 16, 0, + -4, 4, 0, 0, 0, 0, 16, + -4, 0, 4, 0, 0, 0, 16, + -4, 0, 0, 4, 0, 0, 16, + -4, 0, 0, 0, 4, 0, 16, + ]; + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1FilterIntraPredictor.Operator.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1FilterIntraPredictor.Operator.cs new file mode 100644 index 0000000000..c573f3166a --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1FilterIntraPredictor.Operator.cs @@ -0,0 +1,630 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.InteropServices; +using System.Runtime.Intrinsics; +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; + +/// +/// Provides recursive filter-intra traversal for the closed coefficient operators. +/// +internal abstract partial class Av1FilterIntraPredictorBase +{ + /// + /// Defines the coefficient set for one AV1 filter-intra prediction mode. + /// + internal interface IAv1FilterIntraPredictionOperator + { + /// + /// Gets the filter-intra mode implemented by the operator. + /// + public static abstract Av1FilterIntraMode Mode { get; } + + /// + /// Gets the eight seven-tap coefficient rows used by the operator. + /// + public static abstract ReadOnlySpan Taps { get; } + } + + /// + /// Applies one closed filter-intra coefficient operator using the widest useful SIMD width. + /// + /// The filter-intra coefficient set. + /// + /// AV1 filter-intra predicts a two-row by four-column group from seven samples that may include previously + /// predicted groups. The fixed-stride scratch surface preserves those dependencies with a one-sample top and left + /// border. SIMD lanes hold the eight outputs of one group in row-major order; they do not span independent groups, + /// because the next group can depend on the values just produced. + /// + internal sealed class Av1FilterIntraPredictor : Av1FilterIntraPredictorBase + where TOperator : struct, IAv1FilterIntraPredictionOperator + { + /// + public override Av1FilterIntraMode Mode => TOperator.Mode; + + /// + public override void Predict(Span destination, int destinationStride, ReadOnlySpan above, ReadOnlySpan left, int width, int height, Span scratch) + { + Span buffer = scratch[..ScratchLength]; + ref byte bufferBase = ref MemoryMarshal.GetReference(buffer); + ref byte aboveBase = ref MemoryMarshal.GetReference(above); + ref byte leftBase = ref MemoryMarshal.GetReference(left); + ref sbyte taps = ref MemoryMarshal.GetReference(TOperator.Taps); + Initialize(buffer, above, left, width, height, Unsafe.Subtract(ref aboveBase, 1)); + + if (Vector256.IsHardwareAccelerated) + { + // Each tap vector contains the coefficient at one tap position for the eight row-major outputs in a + // 2-by-4 group. Broadcasting the seven reconstructed inputs therefore evaluates all outputs together. + Vector256 tap0 = CreateTapVector256(ref taps, 0); + Vector256 tap1 = CreateTapVector256(ref taps, 1); + Vector256 tap2 = CreateTapVector256(ref taps, 2); + Vector256 tap3 = CreateTapVector256(ref taps, 3); + Vector256 tap4 = CreateTapVector256(ref taps, 4); + Vector256 tap5 = CreateTapVector256(ref taps, 5); + Vector256 tap6 = CreateTapVector256(ref taps, 6); + Vector256 rounding = Vector256.Create(8); + Vector256 maximum = Vector256.Create(255); + + for (int row = 1; row <= height; row += 2) + { + for (int column = 1; column <= width; column += 4) + { + int sourceOffset = ((row - 1) * BufferStride) + column - 1; + Vector256 prediction = Calculate( + tap0, + tap1, + tap2, + tap3, + tap4, + tap5, + tap6, + Unsafe.Add(ref bufferBase, sourceOffset), + Unsafe.Add(ref bufferBase, sourceOffset + 1), + Unsafe.Add(ref bufferBase, sourceOffset + 2), + Unsafe.Add(ref bufferBase, sourceOffset + 3), + Unsafe.Add(ref bufferBase, sourceOffset + 4), + Unsafe.Add(ref bufferBase, sourceOffset + BufferStride), + Unsafe.Add(ref bufferBase, sourceOffset + (2 * BufferStride))); + + prediction = Vector256.Clamp((prediction + rounding) >> 4, Vector256.Zero, maximum); + StoreEightBytes(prediction, ref bufferBase, (row * BufferStride) + column, ((row + 1) * BufferStride) + column); + } + } + } + else if (Vector128.IsHardwareAccelerated) + { + // A 128-bit vector covers one four-sample output row. Low and high coefficient vectors describe the + // first and second rows respectively while sharing the same seven reconstructed input broadcasts. + Vector128 tap0Low = CreateTapVector128(ref taps, 0, 0); + Vector128 tap1Low = CreateTapVector128(ref taps, 1, 0); + Vector128 tap2Low = CreateTapVector128(ref taps, 2, 0); + Vector128 tap3Low = CreateTapVector128(ref taps, 3, 0); + Vector128 tap4Low = CreateTapVector128(ref taps, 4, 0); + Vector128 tap5Low = CreateTapVector128(ref taps, 5, 0); + Vector128 tap6Low = CreateTapVector128(ref taps, 6, 0); + Vector128 tap0High = CreateTapVector128(ref taps, 0, 4); + Vector128 tap1High = CreateTapVector128(ref taps, 1, 4); + Vector128 tap2High = CreateTapVector128(ref taps, 2, 4); + Vector128 tap3High = CreateTapVector128(ref taps, 3, 4); + Vector128 tap4High = CreateTapVector128(ref taps, 4, 4); + Vector128 tap5High = CreateTapVector128(ref taps, 5, 4); + Vector128 tap6High = CreateTapVector128(ref taps, 6, 4); + Vector128 rounding = Vector128.Create(8); + Vector128 maximum = Vector128.Create(255); + + for (int row = 1; row <= height; row += 2) + { + for (int column = 1; column <= width; column += 4) + { + int sourceOffset = ((row - 1) * BufferStride) + column - 1; + int p0 = Unsafe.Add(ref bufferBase, sourceOffset); + int p1 = Unsafe.Add(ref bufferBase, sourceOffset + 1); + int p2 = Unsafe.Add(ref bufferBase, sourceOffset + 2); + int p3 = Unsafe.Add(ref bufferBase, sourceOffset + 3); + int p4 = Unsafe.Add(ref bufferBase, sourceOffset + 4); + int p5 = Unsafe.Add(ref bufferBase, sourceOffset + BufferStride); + int p6 = Unsafe.Add(ref bufferBase, sourceOffset + (2 * BufferStride)); + Vector128 low = Calculate(tap0Low, tap1Low, tap2Low, tap3Low, tap4Low, tap5Low, tap6Low, p0, p1, p2, p3, p4, p5, p6); + Vector128 high = Calculate(tap0High, tap1High, tap2High, tap3High, tap4High, tap5High, tap6High, p0, p1, p2, p3, p4, p5, p6); + low = Vector128.Clamp((low + rounding) >> 4, Vector128.Zero, maximum); + high = Vector128.Clamp((high + rounding) >> 4, Vector128.Zero, maximum); + StoreFourBytes(low, ref bufferBase, (row * BufferStride) + column); + StoreFourBytes(high, ref bufferBase, ((row + 1) * BufferStride) + column); + } + } + } + else + { + PredictGroupsScalar(buffer, width, height, 255, ref taps); + } + + CopyToDestination(buffer, destination, destinationStride, width, height); + } + + /// + public override void Predict(Span destination, int destinationStride, ReadOnlySpan above, ReadOnlySpan left, int width, int height, int bitDepth, Span scratch) + { + Span buffer = scratch[..ScratchLength]; + ref short bufferBase = ref MemoryMarshal.GetReference(buffer); + ref short aboveBase = ref MemoryMarshal.GetReference(above); + ref sbyte taps = ref MemoryMarshal.GetReference(TOperator.Taps); + Initialize(buffer, above, left, width, height, Unsafe.Subtract(ref aboveBase, 1)); + int maximum = (1 << bitDepth) - 1; + + if (Vector256.IsHardwareAccelerated) + { + // High-bit-depth storage changes only the final clamp and narrowing. The Int32 accumulator layout is + // identical to the eight-bit path, preserving all signed coefficient products before Q4 rounding. + Vector256 tap0 = CreateTapVector256(ref taps, 0); + Vector256 tap1 = CreateTapVector256(ref taps, 1); + Vector256 tap2 = CreateTapVector256(ref taps, 2); + Vector256 tap3 = CreateTapVector256(ref taps, 3); + Vector256 tap4 = CreateTapVector256(ref taps, 4); + Vector256 tap5 = CreateTapVector256(ref taps, 5); + Vector256 tap6 = CreateTapVector256(ref taps, 6); + Vector256 rounding = Vector256.Create(8); + Vector256 maximumVector = Vector256.Create(maximum); + + for (int row = 1; row <= height; row += 2) + { + for (int column = 1; column <= width; column += 4) + { + int sourceOffset = ((row - 1) * BufferStride) + column - 1; + Vector256 prediction = Calculate( + tap0, + tap1, + tap2, + tap3, + tap4, + tap5, + tap6, + Unsafe.Add(ref bufferBase, sourceOffset), + Unsafe.Add(ref bufferBase, sourceOffset + 1), + Unsafe.Add(ref bufferBase, sourceOffset + 2), + Unsafe.Add(ref bufferBase, sourceOffset + 3), + Unsafe.Add(ref bufferBase, sourceOffset + 4), + Unsafe.Add(ref bufferBase, sourceOffset + BufferStride), + Unsafe.Add(ref bufferBase, sourceOffset + (2 * BufferStride))); + + prediction = Vector256.Clamp((prediction + rounding) >> 4, Vector256.Zero, maximumVector); + StoreEightShorts(prediction, ref bufferBase, (row * BufferStride) + column, ((row + 1) * BufferStride) + column); + } + } + } + else if (Vector128.IsHardwareAccelerated) + { + Vector128 tap0Low = CreateTapVector128(ref taps, 0, 0); + Vector128 tap1Low = CreateTapVector128(ref taps, 1, 0); + Vector128 tap2Low = CreateTapVector128(ref taps, 2, 0); + Vector128 tap3Low = CreateTapVector128(ref taps, 3, 0); + Vector128 tap4Low = CreateTapVector128(ref taps, 4, 0); + Vector128 tap5Low = CreateTapVector128(ref taps, 5, 0); + Vector128 tap6Low = CreateTapVector128(ref taps, 6, 0); + Vector128 tap0High = CreateTapVector128(ref taps, 0, 4); + Vector128 tap1High = CreateTapVector128(ref taps, 1, 4); + Vector128 tap2High = CreateTapVector128(ref taps, 2, 4); + Vector128 tap3High = CreateTapVector128(ref taps, 3, 4); + Vector128 tap4High = CreateTapVector128(ref taps, 4, 4); + Vector128 tap5High = CreateTapVector128(ref taps, 5, 4); + Vector128 tap6High = CreateTapVector128(ref taps, 6, 4); + Vector128 rounding = Vector128.Create(8); + Vector128 maximumVector = Vector128.Create(maximum); + + for (int row = 1; row <= height; row += 2) + { + for (int column = 1; column <= width; column += 4) + { + int sourceOffset = ((row - 1) * BufferStride) + column - 1; + int p0 = Unsafe.Add(ref bufferBase, sourceOffset); + int p1 = Unsafe.Add(ref bufferBase, sourceOffset + 1); + int p2 = Unsafe.Add(ref bufferBase, sourceOffset + 2); + int p3 = Unsafe.Add(ref bufferBase, sourceOffset + 3); + int p4 = Unsafe.Add(ref bufferBase, sourceOffset + 4); + int p5 = Unsafe.Add(ref bufferBase, sourceOffset + BufferStride); + int p6 = Unsafe.Add(ref bufferBase, sourceOffset + (2 * BufferStride)); + Vector128 low = Calculate(tap0Low, tap1Low, tap2Low, tap3Low, tap4Low, tap5Low, tap6Low, p0, p1, p2, p3, p4, p5, p6); + Vector128 high = Calculate(tap0High, tap1High, tap2High, tap3High, tap4High, tap5High, tap6High, p0, p1, p2, p3, p4, p5, p6); + low = Vector128.Clamp((low + rounding) >> 4, Vector128.Zero, maximumVector); + high = Vector128.Clamp((high + rounding) >> 4, Vector128.Zero, maximumVector); + StoreFourShorts(low, ref bufferBase, (row * BufferStride) + column); + StoreFourShorts(high, ref bufferBase, ((row + 1) * BufferStride) + column); + } + } + } + else + { + PredictGroupsScalar(buffer, width, height, maximum, ref taps); + } + + CopyToDestination(buffer, destination, destinationStride, width, height); + } + + /// + public override void PredictScalar(Span destination, int destinationStride, ReadOnlySpan above, ReadOnlySpan left, int width, int height, Span scratch) + { + Span buffer = scratch[..ScratchLength]; + ref byte aboveBase = ref MemoryMarshal.GetReference(above); + ref sbyte taps = ref MemoryMarshal.GetReference(TOperator.Taps); + Initialize(buffer, above, left, width, height, Unsafe.Subtract(ref aboveBase, 1)); + PredictGroupsScalar(buffer, width, height, 255, ref taps); + CopyToDestinationScalar(buffer, destination, destinationStride, width, height); + } + + /// + public override void PredictScalar(Span destination, int destinationStride, ReadOnlySpan above, ReadOnlySpan left, int width, int height, int bitDepth, Span scratch) + { + Span buffer = scratch[..ScratchLength]; + ref short aboveBase = ref MemoryMarshal.GetReference(above); + ref sbyte taps = ref MemoryMarshal.GetReference(TOperator.Taps); + Initialize(buffer, above, left, width, height, Unsafe.Subtract(ref aboveBase, 1)); + PredictGroupsScalar(buffer, width, height, (1 << bitDepth) - 1, ref taps); + CopyToDestinationScalar(buffer, destination, destinationStride, width, height); + } + + /// + /// Initializes an 8-bit recursive filter-intra workspace from the prepared top and left references. + /// + /// The recursive workspace. + /// The top reference. + /// The left reference. + /// The block width. + /// The block height. + /// The shared top-left reference. + private static void Initialize(Span buffer, ReadOnlySpan above, ReadOnlySpan left, int width, int height, byte topLeft) + { + // Predictions use one-based coordinates in the workspace. Row zero and column zero retain the prepared + // references while later groups overwrite only the interior values on which following groups depend. + buffer[0] = topLeft; + above[..width].CopyTo(buffer[1..]); + for (int row = 0; row < height; row++) + { + buffer[(row + 1) * BufferStride] = left[row]; + } + } + + /// + /// Initializes a high-bit-depth recursive filter-intra workspace from the prepared top and left references. + /// + /// The recursive workspace. + /// The top reference. + /// The left reference. + /// The block width. + /// The block height. + /// The shared top-left reference. + private static void Initialize(Span buffer, ReadOnlySpan above, ReadOnlySpan left, int width, int height, short topLeft) + { + // Match the eight-bit one-based workspace so the recursive source offsets remain representation-agnostic. + buffer[0] = topLeft; + above[..width].CopyTo(buffer[1..]); + for (int row = 0; row < height; row++) + { + buffer[(row + 1) * BufferStride] = left[row]; + } + } + + /// + /// Creates the coefficients for eight output samples at one tap position. + /// + /// The first coefficient in the operator table. + /// The tap position. + /// The coefficient vector. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector256 CreateTapVector256(ref sbyte taps, int tap) + => Vector256.Create( + (int)Unsafe.Add(ref taps, tap), + Unsafe.Add(ref taps, 7 + tap), + Unsafe.Add(ref taps, 14 + tap), + Unsafe.Add(ref taps, 21 + tap), + Unsafe.Add(ref taps, 28 + tap), + Unsafe.Add(ref taps, 35 + tap), + Unsafe.Add(ref taps, 42 + tap), + Unsafe.Add(ref taps, 49 + tap)); + + /// + /// Creates the coefficients for four output samples at one tap position. + /// + /// The first coefficient in the operator table. + /// The tap position. + /// The first output sample. + /// The coefficient vector. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector128 CreateTapVector128(ref sbyte taps, int tap, int pixel) + => Vector128.Create( + (int)Unsafe.Add(ref taps, (pixel * 7) + tap), + Unsafe.Add(ref taps, ((pixel + 1) * 7) + tap), + Unsafe.Add(ref taps, ((pixel + 2) * 7) + tap), + Unsafe.Add(ref taps, ((pixel + 3) * 7) + tap)); + + /// + /// Calculates eight filtered predictions from seven reconstructed samples. + /// + /// The first tap coefficients. + /// The second tap coefficients. + /// The third tap coefficients. + /// The fourth tap coefficients. + /// The fifth tap coefficients. + /// The sixth tap coefficients. + /// The seventh tap coefficients. + /// The first reconstructed sample. + /// The second reconstructed sample. + /// The third reconstructed sample. + /// The fourth reconstructed sample. + /// The fifth reconstructed sample. + /// The sixth reconstructed sample. + /// The seventh reconstructed sample. + /// The unrounded predictions. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector256 Calculate( + Vector256 tap0, + Vector256 tap1, + Vector256 tap2, + Vector256 tap3, + Vector256 tap4, + Vector256 tap5, + Vector256 tap6, + int p0, + int p1, + int p2, + int p3, + int p4, + int p5, + int p6) + => (tap0 * p0) + (tap1 * p1) + (tap2 * p2) + (tap3 * p3) + (tap4 * p4) + (tap5 * p5) + (tap6 * p6); + + /// + /// Calculates four filtered predictions from seven reconstructed samples. + /// + /// The first tap coefficients. + /// The second tap coefficients. + /// The third tap coefficients. + /// The fourth tap coefficients. + /// The fifth tap coefficients. + /// The sixth tap coefficients. + /// The seventh tap coefficients. + /// The first reconstructed sample. + /// The second reconstructed sample. + /// The third reconstructed sample. + /// The fourth reconstructed sample. + /// The fifth reconstructed sample. + /// The sixth reconstructed sample. + /// The seventh reconstructed sample. + /// The unrounded predictions. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector128 Calculate( + Vector128 tap0, + Vector128 tap1, + Vector128 tap2, + Vector128 tap3, + Vector128 tap4, + Vector128 tap5, + Vector128 tap6, + int p0, + int p1, + int p2, + int p3, + int p4, + int p5, + int p6) + => (tap0 * p0) + (tap1 * p1) + (tap2 * p2) + (tap3 * p3) + (tap4 * p4) + (tap5 * p5) + (tap6 * p6); + + /// + /// Stores eight 8-bit predictions into the two recursive output rows. + /// + /// The clamped predictions. + /// The recursive workspace origin. + /// The first output-row offset. + /// The second output-row offset. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static void StoreEightBytes(Vector256 prediction, ref byte buffer, int firstRowOffset, int secondRowOffset) + { + Vector256 narrowed16 = Vector256.Narrow(prediction.AsUInt32(), Vector256.Zero); + Vector256 narrowed8 = Vector256.Narrow(narrowed16, Vector256.Zero); + Unsafe.As(ref Unsafe.Add(ref buffer, firstRowOffset)) = narrowed8.AsUInt32().GetElement(0); + Unsafe.As(ref Unsafe.Add(ref buffer, secondRowOffset)) = narrowed8.AsUInt32().GetElement(1); + } + + /// + /// Stores four 8-bit predictions into one recursive output row. + /// + /// The clamped predictions. + /// The recursive workspace origin. + /// The output-row offset. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static void StoreFourBytes(Vector128 prediction, ref byte buffer, int offset) + { + Vector128 narrowed16 = Vector128.Narrow(prediction.AsUInt32(), Vector128.Zero); + Vector128 narrowed8 = Vector128.Narrow(narrowed16, Vector128.Zero); + Unsafe.As(ref Unsafe.Add(ref buffer, offset)) = narrowed8.AsUInt32().GetElement(0); + } + + /// + /// Stores eight high-bit-depth predictions into the two recursive output rows. + /// + /// The clamped predictions. + /// The recursive workspace origin. + /// The first output-row offset. + /// The second output-row offset. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static void StoreEightShorts(Vector256 prediction, ref short buffer, int firstRowOffset, int secondRowOffset) + { + Vector256 narrowed = Vector256.Narrow(prediction, Vector256.Zero); + Unsafe.As(ref Unsafe.Add(ref buffer, firstRowOffset)) = narrowed.AsUInt64().ToScalar(); + Unsafe.As(ref Unsafe.Add(ref buffer, secondRowOffset)) = narrowed.AsUInt64().GetElement(1); + } + + /// + /// Stores four high-bit-depth predictions into one recursive output row. + /// + /// The clamped predictions. + /// The recursive workspace origin. + /// The output-row offset. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static void StoreFourShorts(Vector128 prediction, ref short buffer, int offset) + { + Vector128 narrowed = Vector128.Narrow(prediction, Vector128.Zero); + Unsafe.As(ref Unsafe.Add(ref buffer, offset)) = narrowed.AsUInt64().ToScalar(); + } + + /// + /// Calculates every 8-bit recursive filter group without hardware intrinsics. + /// + /// The initialized recursive workspace. + /// The block width. + /// The block height. + /// The maximum reconstructed sample. + /// The first filter coefficient. + private static void PredictGroupsScalar(Span buffer, int width, int height, int maximum, ref sbyte taps) + { + ref byte bufferBase = ref MemoryMarshal.GetReference(buffer); + for (int row = 1; row <= height; row += 2) + { + for (int column = 1; column <= width; column += 4) + { + int sourceOffset = ((row - 1) * BufferStride) + column - 1; + int p0 = Unsafe.Add(ref bufferBase, sourceOffset); + int p1 = Unsafe.Add(ref bufferBase, sourceOffset + 1); + int p2 = Unsafe.Add(ref bufferBase, sourceOffset + 2); + int p3 = Unsafe.Add(ref bufferBase, sourceOffset + 3); + int p4 = Unsafe.Add(ref bufferBase, sourceOffset + 4); + int p5 = Unsafe.Add(ref bufferBase, sourceOffset + BufferStride); + int p6 = Unsafe.Add(ref bufferBase, sourceOffset + (2 * BufferStride)); + for (int pixel = 0; pixel < 8; pixel++) + { + int tapOffset = pixel * 7; + int prediction = + (Unsafe.Add(ref taps, tapOffset) * p0) + + (Unsafe.Add(ref taps, tapOffset + 1) * p1) + + (Unsafe.Add(ref taps, tapOffset + 2) * p2) + + (Unsafe.Add(ref taps, tapOffset + 3) * p3) + + (Unsafe.Add(ref taps, tapOffset + 4) * p4) + + (Unsafe.Add(ref taps, tapOffset + 5) * p5) + + (Unsafe.Add(ref taps, tapOffset + 6) * p6); + + int destinationOffset = ((row + (pixel >> 2)) * BufferStride) + column + (pixel & 3); + Unsafe.Add(ref bufferBase, destinationOffset) = (byte)Math.Clamp((prediction + 8) >> 4, 0, maximum); + } + } + } + } + + /// + /// Calculates every high-bit-depth recursive filter group without hardware intrinsics. + /// + /// The initialized recursive workspace. + /// The block width. + /// The block height. + /// The maximum reconstructed sample. + /// The first filter coefficient. + private static void PredictGroupsScalar(Span buffer, int width, int height, int maximum, ref sbyte taps) + { + ref short bufferBase = ref MemoryMarshal.GetReference(buffer); + for (int row = 1; row <= height; row += 2) + { + for (int column = 1; column <= width; column += 4) + { + int sourceOffset = ((row - 1) * BufferStride) + column - 1; + int p0 = Unsafe.Add(ref bufferBase, sourceOffset); + int p1 = Unsafe.Add(ref bufferBase, sourceOffset + 1); + int p2 = Unsafe.Add(ref bufferBase, sourceOffset + 2); + int p3 = Unsafe.Add(ref bufferBase, sourceOffset + 3); + int p4 = Unsafe.Add(ref bufferBase, sourceOffset + 4); + int p5 = Unsafe.Add(ref bufferBase, sourceOffset + BufferStride); + int p6 = Unsafe.Add(ref bufferBase, sourceOffset + (2 * BufferStride)); + for (int pixel = 0; pixel < 8; pixel++) + { + int tapOffset = pixel * 7; + int prediction = + (Unsafe.Add(ref taps, tapOffset) * p0) + + (Unsafe.Add(ref taps, tapOffset + 1) * p1) + + (Unsafe.Add(ref taps, tapOffset + 2) * p2) + + (Unsafe.Add(ref taps, tapOffset + 3) * p3) + + (Unsafe.Add(ref taps, tapOffset + 4) * p4) + + (Unsafe.Add(ref taps, tapOffset + 5) * p5) + + (Unsafe.Add(ref taps, tapOffset + 6) * p6); + + int destinationOffset = ((row + (pixel >> 2)) * BufferStride) + column + (pixel & 3); + Unsafe.Add(ref bufferBase, destinationOffset) = (short)Math.Clamp((prediction + 8) >> 4, 0, maximum); + } + } + } + } + + /// + /// Copies an 8-bit recursive workspace into the strided destination. + /// + /// The completed recursive workspace. + /// The destination block origin. + /// The destination row stride. + /// The block width. + /// The block height. + private static void CopyToDestination(ReadOnlySpan buffer, Span destination, int destinationStride, int width, int height) + { + for (int row = 0; row < height; row++) + { + buffer.Slice(((row + 1) * BufferStride) + 1, width).CopyTo(destination.Slice(row * destinationStride, width)); + } + } + + /// + /// Copies a high-bit-depth recursive workspace into the strided destination. + /// + /// The completed recursive workspace. + /// The destination block origin. + /// The destination row stride. + /// The block width. + /// The block height. + private static void CopyToDestination(ReadOnlySpan buffer, Span destination, int destinationStride, int width, int height) + { + for (int row = 0; row < height; row++) + { + buffer.Slice(((row + 1) * BufferStride) + 1, width).CopyTo(destination.Slice(row * destinationStride, width)); + } + } + + /// + /// Copies an 8-bit recursive workspace into the strided destination without vectorized span copying. + /// + /// The completed recursive workspace. + /// The destination block origin. + /// The destination row stride. + /// The block width. + /// The block height. + private static void CopyToDestinationScalar(ReadOnlySpan buffer, Span destination, int destinationStride, int width, int height) + { + for (int row = 0; row < height; row++) + { + int sourceOffset = ((row + 1) * BufferStride) + 1; + int destinationOffset = row * destinationStride; + for (int column = 0; column < width; column++) + { + destination[destinationOffset + column] = buffer[sourceOffset + column]; + } + } + } + + /// + /// Copies a high-bit-depth recursive workspace into the strided destination without vectorized span copying. + /// + /// The completed recursive workspace. + /// The destination block origin. + /// The destination row stride. + /// The block width. + /// The block height. + private static void CopyToDestinationScalar(ReadOnlySpan buffer, Span destination, int destinationStride, int width, int height) + { + for (int row = 0; row < height; row++) + { + int sourceOffset = ((row + 1) * BufferStride) + 1; + int destinationOffset = row * destinationStride; + for (int column = 0; column < width; column++) + { + destination[destinationOffset + column] = buffer[sourceOffset + column]; + } + } + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1FilterIntraPredictor.PaethOperator.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1FilterIntraPredictor.PaethOperator.cs new file mode 100644 index 0000000000..bbbc93ab3b --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1FilterIntraPredictor.PaethOperator.cs @@ -0,0 +1,31 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; + +internal abstract partial class Av1FilterIntraPredictorBase +{ + /// + /// Defines the AV1 Paeth filter-intra coefficient operator. + /// + internal readonly struct PaethOperator : IAv1FilterIntraPredictionOperator + { + /// + public static Av1FilterIntraMode Mode => Av1FilterIntraMode.Paeth; + + /// + public static ReadOnlySpan Taps => + [ + -12, 14, 0, 0, 0, 14, 0, + -10, 0, 14, 0, 0, 12, 0, + -9, 0, 0, 14, 0, 11, 0, + -8, 0, 0, 0, 14, 10, 0, + -10, 12, 0, 0, 0, 0, 14, + -9, 1, 12, 0, 0, 0, 12, + -8, 0, 0, 12, 0, 1, 11, + -7, 0, 0, 1, 12, 1, 9, + ]; + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1FilterIntraPredictor.VerticalOperator.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1FilterIntraPredictor.VerticalOperator.cs new file mode 100644 index 0000000000..401621a015 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1FilterIntraPredictor.VerticalOperator.cs @@ -0,0 +1,31 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; + +internal abstract partial class Av1FilterIntraPredictorBase +{ + /// + /// Defines the AV1 vertical filter-intra coefficient operator. + /// + internal readonly struct VerticalOperator : IAv1FilterIntraPredictionOperator + { + /// + public static Av1FilterIntraMode Mode => Av1FilterIntraMode.Vertical; + + /// + public static ReadOnlySpan Taps => + [ + -10, 16, 0, 0, 0, 10, 0, + -6, 0, 16, 0, 0, 6, 0, + -4, 0, 0, 16, 0, 4, 0, + -2, 0, 0, 0, 16, 2, 0, + -10, 16, 0, 0, 0, 0, 10, + -6, 0, 16, 0, 0, 0, 6, + -4, 0, 0, 16, 0, 0, 4, + -2, 0, 0, 0, 16, 0, 2, + ]; + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1FilterIntraPredictorBase.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1FilterIntraPredictorBase.cs new file mode 100644 index 0000000000..7d59507115 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1FilterIntraPredictorBase.cs @@ -0,0 +1,121 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; + +/// +/// Reconstructs AV1 filter-intra prediction blocks from prepared neighboring samples. +/// +/// +/// The implementation follows the recursive filter-intra process in section 7.11.2.3 of the AV1 specification. +/// +internal abstract partial class Av1FilterIntraPredictorBase +{ + /// + /// The row stride of the recursive prediction workspace. + /// + public const int BufferStride = 33; + + /// + /// The number of samples in the recursive prediction workspace. + /// + public const int ScratchLength = BufferStride * BufferStride; + + /// + /// The DC filter-intra predictor. + /// + private static readonly Av1FilterIntraPredictor DcPredictor = new(); + + /// + /// The vertical filter-intra predictor. + /// + private static readonly Av1FilterIntraPredictor VerticalPredictor = new(); + + /// + /// The horizontal filter-intra predictor. + /// + private static readonly Av1FilterIntraPredictor HorizontalPredictor = new(); + + /// + /// The 157-degree directional filter-intra predictor. + /// + private static readonly Av1FilterIntraPredictor Directional157Predictor = new(); + + /// + /// The Paeth filter-intra predictor. + /// + private static readonly Av1FilterIntraPredictor PaethPredictor = new(); + + /// + /// Gets the filter-intra mode implemented by this predictor. + /// + public abstract Av1FilterIntraMode Mode { get; } + + /// + /// Gets the closed predictor for a filter-intra mode. + /// + /// The decoded filter-intra mode. + /// The predictor for . + public static Av1FilterIntraPredictorBase GetPredictor(Av1FilterIntraMode mode) + => mode switch + { + Av1FilterIntraMode.DC => DcPredictor, + Av1FilterIntraMode.Vertical => VerticalPredictor, + Av1FilterIntraMode.Horizontal => HorizontalPredictor, + Av1FilterIntraMode.Directional157 => Directional157Predictor, + Av1FilterIntraMode.Paeth => PaethPredictor, + _ => throw new InvalidImageContentException($"Filter-intra mode {mode} is not defined by AV1."), + }; + + /// + /// Predicts an 8-bit filter-intra block. + /// + /// The destination block origin. + /// The destination row stride in samples. + /// The prepared top reference, preceded in memory by the top-left sample. + /// The prepared left reference. + /// The block width. + /// The block height. + /// The caller-owned recursive prediction workspace. + public abstract void Predict(Span destination, int destinationStride, ReadOnlySpan above, ReadOnlySpan left, int width, int height, Span scratch); + + /// + /// Predicts a high-bit-depth filter-intra block. + /// + /// The destination block origin. + /// The destination row stride in samples. + /// The prepared top reference, preceded in memory by the top-left sample. + /// The prepared left reference. + /// The block width. + /// The block height. + /// The reconstructed sample precision. + /// The caller-owned recursive prediction workspace. + public abstract void Predict(Span destination, int destinationStride, ReadOnlySpan above, ReadOnlySpan left, int width, int height, int bitDepth, Span scratch); + + /// + /// Predicts an 8-bit filter-intra block without hardware intrinsics. + /// + /// The destination block origin. + /// The destination row stride in samples. + /// The prepared top reference, preceded in memory by the top-left sample. + /// The prepared left reference. + /// The block width. + /// The block height. + /// The caller-owned recursive prediction workspace. + public abstract void PredictScalar(Span destination, int destinationStride, ReadOnlySpan above, ReadOnlySpan left, int width, int height, Span scratch); + + /// + /// Predicts a high-bit-depth filter-intra block without hardware intrinsics. + /// + /// The destination block origin. + /// The destination row stride in samples. + /// The prepared top reference, preceded in memory by the top-left sample. + /// The prepared left reference. + /// The block width. + /// The block height. + /// The reconstructed sample precision. + /// The caller-owned recursive prediction workspace. + public abstract void PredictScalar(Span destination, int destinationStride, ReadOnlySpan above, ReadOnlySpan left, int width, int height, int bitDepth, Span scratch); +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1IntraEdgeFilter.Operations.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1IntraEdgeFilter.Operations.cs new file mode 100644 index 0000000000..f6c48c57cf --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1IntraEdgeFilter.Operations.cs @@ -0,0 +1,210 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.InteropServices; +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; + +internal static partial class Av1IntraEdgeFilter +{ + /// + /// Traverses one edge using the arithmetic of a closed smoothing operator. + /// + /// The filter-strength arithmetic. + private static class Filter + where TOperator : struct, IEdgeFilterOperator + { + /// + /// Filters all samples following the preserved first sample. + /// + /// The first edge sample. + /// The number of samples including the preserved sample. + /// The reusable source workspace. + public static void Apply(ref byte edge, int count, Span scratch) + { + // Each convolution reads the original edge. Duplicate its first sample once and its last sample + // twice so the five-tap windows implement endpoint clamping without per-lane boundary branches. + scratch[0] = edge; + MemoryMarshal.CreateReadOnlySpan(ref edge, count).CopyTo(scratch[1..]); + scratch.Slice(count + 1, 2).Fill(Unsafe.Add(ref edge, count - 1)); + + ref byte source = ref MemoryMarshal.GetReference(scratch); + int outputCount = count - 1; + int i = 0; + + // The same offset advances through descending SIMD widths. Adjacent lanes represent adjacent + // output samples, and only complete windows are loaded; the final incomplete window is scalar. + if (Vector512.IsHardwareAccelerated) + { + int vectorEnd = outputCount - Vector512.Count; + for (; i <= vectorEnd; i += Vector512.Count) + { + Vector512 s0 = Vector512.WidenLower(Vector512.Create( + Vector256.LoadUnsafe(ref source, (nuint)(i + 0)), Vector256.Zero)); + + Vector512 s1 = Vector512.WidenLower(Vector512.Create( + Vector256.LoadUnsafe(ref source, (nuint)(i + 1)), Vector256.Zero)); + + Vector512 s2 = Vector512.WidenLower(Vector512.Create( + Vector256.LoadUnsafe(ref source, (nuint)(i + 2)), Vector256.Zero)); + + Vector512 s3 = Vector512.WidenLower(Vector512.Create( + Vector256.LoadUnsafe(ref source, (nuint)(i + 3)), Vector256.Zero)); + + Vector512 s4 = Vector512.WidenLower(Vector512.Create( + Vector256.LoadUnsafe(ref source, (nuint)(i + 4)), Vector256.Zero)); + + Vector512 result = TOperator.Apply(s0, s1, s2, s3, s4); + Vector512.Narrow(result, Vector512.Zero).GetLower().StoreUnsafe(ref edge, (nuint)(i + 1)); + } + } + + if (Vector256.IsHardwareAccelerated) + { + int vectorEnd = outputCount - Vector256.Count; + for (; i <= vectorEnd; i += Vector256.Count) + { + Vector256 s0 = Vector256.WidenLower(Vector256.Create( + Vector128.LoadUnsafe(ref source, (nuint)(i + 0)), Vector128.Zero)); + + Vector256 s1 = Vector256.WidenLower(Vector256.Create( + Vector128.LoadUnsafe(ref source, (nuint)(i + 1)), Vector128.Zero)); + + Vector256 s2 = Vector256.WidenLower(Vector256.Create( + Vector128.LoadUnsafe(ref source, (nuint)(i + 2)), Vector128.Zero)); + + Vector256 s3 = Vector256.WidenLower(Vector256.Create( + Vector128.LoadUnsafe(ref source, (nuint)(i + 3)), Vector128.Zero)); + + Vector256 s4 = Vector256.WidenLower(Vector256.Create( + Vector128.LoadUnsafe(ref source, (nuint)(i + 4)), Vector128.Zero)); + + Vector256 result = TOperator.Apply(s0, s1, s2, s3, s4); + Vector256.Narrow(result, Vector256.Zero).GetLower().StoreUnsafe(ref edge, (nuint)(i + 1)); + } + } + + if (Vector128.IsHardwareAccelerated) + { + int vectorEnd = outputCount - Vector128.Count; + for (; i <= vectorEnd; i += Vector128.Count) + { + Vector128 s0 = Vector128.WidenLower(Vector128.Create( + Vector64.LoadUnsafe(ref source, (nuint)(i + 0)), Vector64.Zero)); + + Vector128 s1 = Vector128.WidenLower(Vector128.Create( + Vector64.LoadUnsafe(ref source, (nuint)(i + 1)), Vector64.Zero)); + + Vector128 s2 = Vector128.WidenLower(Vector128.Create( + Vector64.LoadUnsafe(ref source, (nuint)(i + 2)), Vector64.Zero)); + + Vector128 s3 = Vector128.WidenLower(Vector128.Create( + Vector64.LoadUnsafe(ref source, (nuint)(i + 3)), Vector64.Zero)); + + Vector128 s4 = Vector128.WidenLower(Vector128.Create( + Vector64.LoadUnsafe(ref source, (nuint)(i + 4)), Vector64.Zero)); + + Vector128 result = TOperator.Apply(s0, s1, s2, s3, s4); + Vector128.Narrow(result, Vector128.Zero).GetLower().StoreUnsafe(ref edge, (nuint)(i + 1)); + } + } + + for (; i < outputCount; i++) + { + int value = TOperator.Apply( + Unsafe.Add(ref source, i), + Unsafe.Add(ref source, i + 1), + Unsafe.Add(ref source, i + 2), + Unsafe.Add(ref source, i + 3), + Unsafe.Add(ref source, i + 4)); + + Unsafe.Add(ref edge, i + 1) = (byte)value; + } + } + + /// + /// Filters all samples following the preserved first sample. + /// + /// The first edge sample. + /// The number of samples including the preserved sample. + /// The reusable source workspace. + public static void Apply(ref short edge, int count, Span scratch) + { + // Each convolution reads the original edge. Duplicate its first sample once and its last sample + // twice so the five-tap windows implement endpoint clamping without per-lane boundary branches. + scratch[0] = edge; + MemoryMarshal.CreateReadOnlySpan(ref edge, count).CopyTo(scratch[1..]); + scratch.Slice(count + 1, 2).Fill(Unsafe.Add(ref edge, count - 1)); + + ref short source = ref MemoryMarshal.GetReference(scratch); + int outputCount = count - 1; + int i = 0; + + // The same offset advances through descending SIMD widths. Adjacent lanes represent adjacent + // output samples, and only complete windows are loaded; the final incomplete window is scalar. + // Nonnegative 12-bit samples have a maximum weighted sum of 65520. The rounding bias keeps + // that below 65536, so unsigned 16-bit lanes preserve the normative result at all strengths. + if (Vector512.IsHardwareAccelerated) + { + int vectorEnd = outputCount - Vector512.Count; + for (; i <= vectorEnd; i += Vector512.Count) + { + Vector512 s0 = Vector512.LoadUnsafe(ref source, (nuint)(i + 0)).AsUInt16(); + Vector512 s1 = Vector512.LoadUnsafe(ref source, (nuint)(i + 1)).AsUInt16(); + Vector512 s2 = Vector512.LoadUnsafe(ref source, (nuint)(i + 2)).AsUInt16(); + Vector512 s3 = Vector512.LoadUnsafe(ref source, (nuint)(i + 3)).AsUInt16(); + Vector512 s4 = Vector512.LoadUnsafe(ref source, (nuint)(i + 4)).AsUInt16(); + + Vector512 result = TOperator.Apply(s0, s1, s2, s3, s4); + result.AsInt16().StoreUnsafe(ref edge, (nuint)(i + 1)); + } + } + + if (Vector256.IsHardwareAccelerated) + { + int vectorEnd = outputCount - Vector256.Count; + for (; i <= vectorEnd; i += Vector256.Count) + { + Vector256 s0 = Vector256.LoadUnsafe(ref source, (nuint)(i + 0)).AsUInt16(); + Vector256 s1 = Vector256.LoadUnsafe(ref source, (nuint)(i + 1)).AsUInt16(); + Vector256 s2 = Vector256.LoadUnsafe(ref source, (nuint)(i + 2)).AsUInt16(); + Vector256 s3 = Vector256.LoadUnsafe(ref source, (nuint)(i + 3)).AsUInt16(); + Vector256 s4 = Vector256.LoadUnsafe(ref source, (nuint)(i + 4)).AsUInt16(); + + Vector256 result = TOperator.Apply(s0, s1, s2, s3, s4); + result.AsInt16().StoreUnsafe(ref edge, (nuint)(i + 1)); + } + } + + if (Vector128.IsHardwareAccelerated) + { + int vectorEnd = outputCount - Vector128.Count; + for (; i <= vectorEnd; i += Vector128.Count) + { + Vector128 s0 = Vector128.LoadUnsafe(ref source, (nuint)(i + 0)).AsUInt16(); + Vector128 s1 = Vector128.LoadUnsafe(ref source, (nuint)(i + 1)).AsUInt16(); + Vector128 s2 = Vector128.LoadUnsafe(ref source, (nuint)(i + 2)).AsUInt16(); + Vector128 s3 = Vector128.LoadUnsafe(ref source, (nuint)(i + 3)).AsUInt16(); + Vector128 s4 = Vector128.LoadUnsafe(ref source, (nuint)(i + 4)).AsUInt16(); + + Vector128 result = TOperator.Apply(s0, s1, s2, s3, s4); + result.AsInt16().StoreUnsafe(ref edge, (nuint)(i + 1)); + } + } + + for (; i < outputCount; i++) + { + int value = TOperator.Apply( + Unsafe.Add(ref source, i), + Unsafe.Add(ref source, i + 1), + Unsafe.Add(ref source, i + 2), + Unsafe.Add(ref source, i + 3), + Unsafe.Add(ref source, i + 4)); + + Unsafe.Add(ref edge, i + 1) = (short)value; + } + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1IntraEdgeFilter.Operator.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1IntraEdgeFilter.Operator.cs new file mode 100644 index 0000000000..1c2c20dc2a --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1IntraEdgeFilter.Operator.cs @@ -0,0 +1,113 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; + +/// +/// Smooths AV1 intra-reference edges while preserving their common-corner sample. +/// +internal static partial class Av1IntraEdgeFilter +{ + /// + /// The sample count required for a maximal edge and its repeated endpoints. + /// + public const int ScratchLength = (2 * Av1Constants.MaxTransformSize) + 4; + + /// + /// Defines the rounded smoothing arithmetic for one AV1 filter strength. + /// + internal interface IEdgeFilterOperator + { + /// + /// Filters one sample using the five neighboring positions. + /// + /// The samples two positions before the output. + /// The preceding samples. + /// The centered samples. + /// The following samples. + /// The samples two positions after the output. + /// The rounded filtered samples. + public static abstract int Apply(int a, int b, int c, int d, int e); + + /// + /// Filters eight samples using the five neighboring positions. + /// + /// The samples two positions before the output. + /// The preceding samples. + /// The centered samples. + /// The following samples. + /// The samples two positions after the output. + /// The rounded filtered samples. + public static abstract Vector128 Apply(Vector128 a, Vector128 b, Vector128 c, Vector128 d, Vector128 e); + + /// + /// Filters sixteen samples using the five neighboring positions. + /// + /// The samples two positions before the output. + /// The preceding samples. + /// The centered samples. + /// The following samples. + /// The samples two positions after the output. + /// The rounded filtered samples. + public static abstract Vector256 Apply(Vector256 a, Vector256 b, Vector256 c, Vector256 d, Vector256 e); + + /// + /// Filters thirty-two samples using the five neighboring positions. + /// + /// The samples two positions before the output. + /// The preceding samples. + /// The centered samples. + /// The following samples. + /// The samples two positions after the output. + /// The rounded filtered samples. + public static abstract Vector512 Apply(Vector512 a, Vector512 b, Vector512 c, Vector512 d, Vector512 e); + } + + /// + /// Filters an edge in place, leaving its first sample unchanged. + /// + /// The first edge sample, including the common corner when present. + /// The number of edge samples. + /// The smoothing strength from zero through three. + /// The source workspace with at least samples. + public static void Apply(ref byte edge, int count, int strength, Span scratch) + { + switch (strength) + { + case 1: + Filter.Apply(ref edge, count, scratch); + break; + case 2: + Filter.Apply(ref edge, count, scratch); + break; + case 3: + Filter.Apply(ref edge, count, scratch); + break; + } + } + + /// + /// Filters an edge in place, leaving its first sample unchanged. + /// + /// The first edge sample, including the common corner when present. + /// The number of edge samples. + /// The smoothing strength from zero through three. + /// The source workspace with at least samples. + public static void Apply(ref short edge, int count, int strength, Span scratch) + { + switch (strength) + { + case 1: + Filter.Apply(ref edge, count, scratch); + break; + case 2: + Filter.Apply(ref edge, count, scratch); + break; + case 3: + Filter.Apply(ref edge, count, scratch); + break; + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1IntraEdgeFilter.Strength1Operator.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1IntraEdgeFilter.Strength1Operator.cs new file mode 100644 index 0000000000..bdd76e67da --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1IntraEdgeFilter.Strength1Operator.cs @@ -0,0 +1,36 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; + +internal static partial class Av1IntraEdgeFilter +{ + /// + /// Applies the strength-1 three-tap edge smoothing kernel. + /// + internal readonly struct Strength1Operator : IEdgeFilterOperator + { + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static int Apply(int a, int b, int c, int d, int e) + => (b + (c << 1) + d + 2) >> 2; + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector128 Apply(Vector128 a, Vector128 b, Vector128 c, Vector128 d, Vector128 e) + => (b + (c << 1) + d + Vector128.Create((ushort)2)) >> 2; + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector256 Apply(Vector256 a, Vector256 b, Vector256 c, Vector256 d, Vector256 e) + => (b + (c << 1) + d + Vector256.Create((ushort)2)) >> 2; + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector512 Apply(Vector512 a, Vector512 b, Vector512 c, Vector512 d, Vector512 e) + => (b + (c << 1) + d + Vector512.Create((ushort)2)) >> 2; + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1IntraEdgeFilter.Strength2Operator.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1IntraEdgeFilter.Strength2Operator.cs new file mode 100644 index 0000000000..c4f2f2f40e --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1IntraEdgeFilter.Strength2Operator.cs @@ -0,0 +1,36 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; + +internal static partial class Av1IntraEdgeFilter +{ + /// + /// Applies the strength-2 three-tap edge smoothing kernel. + /// + internal readonly struct Strength2Operator : IEdgeFilterOperator + { + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static int Apply(int a, int b, int c, int d, int e) + => (((b + d) * 5) + (c * 6) + 8) >> 4; + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector128 Apply(Vector128 a, Vector128 b, Vector128 c, Vector128 d, Vector128 e) + => (((b + d) * Vector128.Create((ushort)5)) + (c * Vector128.Create((ushort)6)) + Vector128.Create((ushort)8)) >> 4; + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector256 Apply(Vector256 a, Vector256 b, Vector256 c, Vector256 d, Vector256 e) + => (((b + d) * Vector256.Create((ushort)5)) + (c * Vector256.Create((ushort)6)) + Vector256.Create((ushort)8)) >> 4; + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector512 Apply(Vector512 a, Vector512 b, Vector512 c, Vector512 d, Vector512 e) + => (((b + d) * Vector512.Create((ushort)5)) + (c * Vector512.Create((ushort)6)) + Vector512.Create((ushort)8)) >> 4; + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1IntraEdgeFilter.Strength3Operator.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1IntraEdgeFilter.Strength3Operator.cs new file mode 100644 index 0000000000..38aaac7658 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1IntraEdgeFilter.Strength3Operator.cs @@ -0,0 +1,36 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; + +internal static partial class Av1IntraEdgeFilter +{ + /// + /// Applies the strength-3 five-tap edge smoothing kernel. + /// + internal readonly struct Strength3Operator : IEdgeFilterOperator + { + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static int Apply(int a, int b, int c, int d, int e) + => (a + ((b + c + d) << 1) + e + 4) >> 3; + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector128 Apply(Vector128 a, Vector128 b, Vector128 c, Vector128 d, Vector128 e) + => (a + ((b + c + d) << 1) + e + Vector128.Create((ushort)4)) >> 3; + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector256 Apply(Vector256 a, Vector256 b, Vector256 c, Vector256 d, Vector256 e) + => (a + ((b + c + d) << 1) + e + Vector256.Create((ushort)4)) >> 3; + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector512 Apply(Vector512 a, Vector512 b, Vector512 c, Vector512 d, Vector512 e) + => (a + ((b + c + d) << 1) + e + Vector512.Create((ushort)4)) >> 3; + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1IntraEdgePreparation.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1IntraEdgePreparation.cs new file mode 100644 index 0000000000..1202f39bc5 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1IntraEdgePreparation.cs @@ -0,0 +1,289 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Numerics; +using System.Runtime.CompilerServices; +using System.Runtime.InteropServices; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; + +/// +/// Prepares directional intra-reference edges for AV1 smoothing and half-sample prediction. +/// +internal static class Av1IntraEdgePreparation +{ + /// + /// The number of samples reserved before the first edge sample. + /// + public const int ReferencePrefixLength = 16; + + /// + /// The total sample capacity of one edge including prefix and extension. + /// + public const int ReferenceBufferLength = (2 * Av1Constants.MaxTransformSize) + 32; + + /// + /// Filters and upsamples prepared directional reference edges. + /// + /// The byte or signed high-bit-depth sample type. + /// The top edge with writable prefix and extension. + /// The left edge with writable prefix and extension. + /// The transform width. + /// The transform height. + /// The adjusted directional angle. + /// The number of available top samples before extension. + /// The number of available left samples before extension. + /// Whether a relevant neighbor uses smooth prediction. + /// The coded sample precision. + /// The original-edge workspace with at least samples. + /// Whether the top edge contains half-sample positions. + /// Whether the left edge contains half-sample positions. + public static void Prepare( + Span above, + Span left, + int width, + int height, + int angle, + int topCount, + int leftCount, + bool filterType, + int bitDepth, + Span scratch, + out bool upsampleAbove, + out bool upsampleLeft) + where T : unmanaged, IBinaryInteger + { + bool needAbove = angle < 180; + bool needLeft = angle > 90; + bool needRight = angle < 90; + bool needBottom = angle > 180; + upsampleAbove = false; + upsampleLeft = false; + + // A missing sole edge produces a constant block from the perpendicular sample or midpoint offset. + // Its prepared edge already repeats that value. Upsampling its distinct corner would change it. + if ((!needAbove && leftCount == 0) || (!needLeft && topCount == 0)) + { + return; + } + + if (angle is not 90 and not 180) + { + if (needAbove && needLeft && width + height >= 24) + { + // The corner is one logical sample represented in both edge prefixes. Filter it first, + // then let both edge convolutions read the same rounded [5, 6, 5] corner value. + ref T corner = ref Unsafe.Subtract(ref above[0], 1); + int value = (5 * int.CreateChecked(left[0])) + + (6 * int.CreateChecked(corner)) + + (5 * int.CreateChecked(above[0])); + + corner = T.CreateChecked((value + 8) >> 4); + Unsafe.Subtract(ref left[0], 1) = corner; + } + + if (needAbove && topCount > 0) + { + int strength = IntraEdgeFilterStrength(width, height, angle - 90, filterType); + Filter(ref Unsafe.Subtract(ref above[0], 1), topCount + 1 + (needRight ? height : 0), strength, scratch); + } + + if (needLeft && leftCount > 0) + { + int strength = IntraEdgeFilterStrength(height, width, angle - 180, filterType); + Filter(ref Unsafe.Subtract(ref left[0], 1), leftCount + 1 + (needBottom ? width : 0), strength, scratch); + } + } + + upsampleAbove = UseUpsampling(width, height, angle - 90, filterType); + if (needAbove && upsampleAbove) + { + Upsample(above, width + (needRight ? height : 0), bitDepth, scratch); + } + + upsampleLeft = UseUpsampling(height, width, angle - 180, filterType); + if (needLeft && upsampleLeft) + { + Upsample(left, height + (needBottom ? width : 0), bitDepth, scratch); + } + } + + /// + /// Selects half-sample interpolation for a transform edge. + /// + /// The transform width. + /// The transform height. + /// The angle relative to the edge's cardinal direction. + /// Whether a relevant neighbor uses smooth prediction. + /// Whether the edge uses half-sample interpolation. + private static bool UseUpsampling(int width, int height, int delta, bool filterType) + { + int distance = Math.Abs(delta); + return distance > 0 && distance < 40 && width + height <= (filterType ? 8 : 16); + } + + /// + /// Dispatches edge smoothing to the concrete sample representation. + /// + /// The byte or signed high-bit-depth sample type. + /// The first edge sample, including the corner. + /// The number of edge samples. + /// The smoothing strength. + /// The reusable original-edge workspace. + private static void Filter(ref T edge, int count, int strength, Span scratch) + where T : unmanaged, IBinaryInteger + { + if (typeof(T) == typeof(byte)) + { + Av1IntraEdgeFilter.Apply(ref Unsafe.As(ref edge), count, strength, MemoryMarshal.Cast(scratch)); + } + else + { + Av1IntraEdgeFilter.Apply(ref Unsafe.As(ref edge), count, strength, MemoryMarshal.Cast(scratch)); + } + } + + /// + /// Dispatches half-sample interpolation to the concrete sample representation. + /// + /// The byte or signed high-bit-depth sample type. + /// The edge with writable prefix and extension. + /// The number of original edge samples. + /// The coded precision. + /// The reusable original-edge workspace. + private static void Upsample(Span edge, int count, int bitDepth, Span scratch) + where T : unmanaged, IBinaryInteger + { + if (typeof(T) == typeof(byte)) + { + Av1IntraEdgeUpsampler.Apply(MemoryMarshal.Cast(edge), count, MemoryMarshal.Cast(scratch)); + } + else + { + Av1IntraEdgeUpsampler.Apply(MemoryMarshal.Cast(edge), count, bitDepth, MemoryMarshal.Cast(scratch)); + } + } + + /// + /// Selects the AV1 intra-edge filter strength for the block dimensions and prediction angle. + /// + /// The edge's primary block dimension. + /// The edge's secondary block dimension. + /// The prediction angle relative to the edge's cardinal direction. + /// A value indicating whether a neighboring smooth mode selects the alternate thresholds. + /// The filter strength from zero for no filtering through three for the strongest kernel. + private static int IntraEdgeFilterStrength(int width, int height, int delta, bool filterType) + { + int d = Math.Abs(delta); + int strength = 0; + int widthHeight = width + height; + if (!filterType) + { + if (widthHeight <= 8) + { + if (d >= 56) + { + strength = 1; + } + } + else if (widthHeight <= 12) + { + if (d >= 40) + { + strength = 1; + } + } + else if (widthHeight <= 16) + { + if (d >= 40) + { + strength = 1; + } + } + else if (widthHeight <= 24) + { + if (d >= 8) + { + strength = 1; + } + + if (d >= 16) + { + strength = 2; + } + + if (d >= 32) + { + strength = 3; + } + } + else if (widthHeight <= 32) + { + if (d >= 1) + { + strength = 1; + } + + if (d >= 4) + { + strength = 2; + } + + if (d >= 32) + { + strength = 3; + } + } + else + { + if (d >= 1) + { + strength = 3; + } + } + } + else + { + if (widthHeight <= 8) + { + if (d >= 40) + { + strength = 1; + } + + if (d >= 64) + { + strength = 2; + } + } + else if (widthHeight <= 16) + { + if (d >= 20) + { + strength = 1; + } + + if (d >= 48) + { + strength = 2; + } + } + else if (widthHeight <= 24) + { + if (d >= 4) + { + strength = 3; + } + } + else + { + if (d >= 1) + { + strength = 3; + } + } + } + + return strength; + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1IntraEdgeUpsampler.FourTapOperator.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1IntraEdgeUpsampler.FourTapOperator.cs new file mode 100644 index 0000000000..b99523ce02 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1IntraEdgeUpsampler.FourTapOperator.cs @@ -0,0 +1,48 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; + +internal static partial class Av1IntraEdgeUpsampler +{ + /// + /// Applies the AV1 [-1, 9, 9, -1] interpolation kernel with Q4 rounding and clipping. + /// + internal readonly struct FourTapOperator : IEdgeUpsamplingOperator + { + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static int Interpolate(int a, int b, int c, int d, int maximum) + => Math.Clamp((((9 * (b + c)) - a - d) + 8) >> 4, 0, maximum); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector128 Interpolate(Vector128 a, Vector128 b, Vector128 c, Vector128 d, int maximum) + { + // Signed 32-bit lanes preserve negative overshoot and the 12-bit central sum, which can reach 73710. + Vector128 value = (((Vector128.Create(9) * (b + c)) - a - d) + Vector128.Create(8)) >> 4; + return Vector128.Clamp(value, Vector128.Zero, Vector128.Create(maximum)); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector256 Interpolate(Vector256 a, Vector256 b, Vector256 c, Vector256 d, int maximum) + { + // Signed 32-bit lanes preserve negative overshoot and the 12-bit central sum, which can reach 73710. + Vector256 value = (((Vector256.Create(9) * (b + c)) - a - d) + Vector256.Create(8)) >> 4; + return Vector256.Clamp(value, Vector256.Zero, Vector256.Create(maximum)); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector512 Interpolate(Vector512 a, Vector512 b, Vector512 c, Vector512 d, int maximum) + { + // Signed 32-bit lanes preserve negative overshoot and the 12-bit central sum, which can reach 73710. + Vector512 value = (((Vector512.Create(9) * (b + c)) - a - d) + Vector512.Create(8)) >> 4; + return Vector512.Clamp(value, Vector512.Zero, Vector512.Create(maximum)); + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1IntraEdgeUpsampler.Operations.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1IntraEdgeUpsampler.Operations.cs new file mode 100644 index 0000000000..106905c053 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1IntraEdgeUpsampler.Operations.cs @@ -0,0 +1,278 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.InteropServices; +using System.Runtime.Intrinsics; +using SixLabors.ImageSharp.Common.Helpers; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; + +internal static partial class Av1IntraEdgeUpsampler +{ + /// + /// Traverses a bounded edge using a closed interpolation operator. + /// + /// The four-tap interpolation arithmetic. + private static class Upsampler + where TOperator : struct, IEdgeUpsamplingOperator + { + /// + /// Inserts half samples using the original edge values and repeated endpoints. + /// + /// The edge with prefix and doubled output capacity. + /// The original sample count. + /// The reusable original-sample workspace. + public static void Apply(Span edge, int count, Span scratch) + { + ref byte destination = ref MemoryMarshal.GetReference(edge); + ref byte source = ref MemoryMarshal.GetReference(scratch); + + // Preserve the corner twice and the final sample once. Every SIMD load below covers exactly its + // input lanes, so the native 16+3-sample workspace also suffices for the widest interpolation. + source = Unsafe.Subtract(ref destination, 1); + Unsafe.Add(ref source, 1) = source; + edge[..count].CopyTo(scratch[2..]); + Unsafe.Add(ref source, count + 2) = edge[count - 1]; + Unsafe.Subtract(ref destination, 2) = source; + ref byte firstOutput = ref Unsafe.Subtract(ref destination, 1); + int i = 0; + + if (Vector512.IsHardwareAccelerated) + { + int vectorEnd = count - Vector512.Count; + for (; i <= vectorEnd; i += Vector512.Count) + { + Vector256 w0 = Vector256.WidenLower(Vector256.Create( + Vector128.LoadUnsafe(ref source, (nuint)(i + 0)), Vector128.Zero)); + + Vector512 s0 = Vector512.WidenLower(Vector512.Create(w0, Vector256.Zero)).AsInt32(); + + Vector256 w1 = Vector256.WidenLower(Vector256.Create( + Vector128.LoadUnsafe(ref source, (nuint)(i + 1)), Vector128.Zero)); + + Vector512 s1 = Vector512.WidenLower(Vector512.Create(w1, Vector256.Zero)).AsInt32(); + + Vector256 w2 = Vector256.WidenLower(Vector256.Create( + Vector128.LoadUnsafe(ref source, (nuint)(i + 2)), Vector128.Zero)); + + Vector512 s2 = Vector512.WidenLower(Vector512.Create(w2, Vector256.Zero)).AsInt32(); + + Vector256 w3 = Vector256.WidenLower(Vector256.Create( + Vector128.LoadUnsafe(ref source, (nuint)(i + 3)), Vector128.Zero)); + + Vector512 s3 = Vector512.WidenLower(Vector512.Create(w3, Vector256.Zero)).AsInt32(); + + Vector512 values = TOperator.Interpolate(s0, s1, s2, s3, 255); + Vector256 halfWords = Vector512.Narrow(values, Vector512.Zero).GetLower().AsUInt16(); + Vector128 halfSamples = Vector256.Narrow(halfWords, Vector256.Zero).GetLower(); + Vector128 originals = Vector128.LoadUnsafe(ref source, (nuint)(i + 2)); + + // Unpack the lower and upper eight pairs independently to retain linear sample order. + Vector128_.UnpackLow(halfSamples, originals).StoreUnsafe(ref firstOutput, (nuint)(2 * i)); + Vector128_.UnpackHigh(halfSamples, originals).StoreUnsafe(ref firstOutput, (nuint)((2 * i) + 16)); + } + } + + if (Vector256.IsHardwareAccelerated) + { + int vectorEnd = count - Vector256.Count; + for (; i <= vectorEnd; i += Vector256.Count) + { + Vector128 w0 = Vector128.WidenLower(Vector128.Create( + Vector64.LoadUnsafe(ref source, (nuint)(i + 0)), Vector64.Zero)); + + Vector256 s0 = Vector256.WidenLower(Vector256.Create(w0, Vector128.Zero)).AsInt32(); + + Vector128 w1 = Vector128.WidenLower(Vector128.Create( + Vector64.LoadUnsafe(ref source, (nuint)(i + 1)), Vector64.Zero)); + + Vector256 s1 = Vector256.WidenLower(Vector256.Create(w1, Vector128.Zero)).AsInt32(); + + Vector128 w2 = Vector128.WidenLower(Vector128.Create( + Vector64.LoadUnsafe(ref source, (nuint)(i + 2)), Vector64.Zero)); + + Vector256 s2 = Vector256.WidenLower(Vector256.Create(w2, Vector128.Zero)).AsInt32(); + + Vector128 w3 = Vector128.WidenLower(Vector128.Create( + Vector64.LoadUnsafe(ref source, (nuint)(i + 3)), Vector64.Zero)); + + Vector256 s3 = Vector256.WidenLower(Vector256.Create(w3, Vector128.Zero)).AsInt32(); + + Vector256 values = TOperator.Interpolate(s0, s1, s2, s3, 255); + Vector128 halfWords = Vector256.Narrow(values, Vector256.Zero).GetLower().AsUInt16(); + Vector128 halfSamples = Vector128.Narrow(halfWords, Vector128.Zero); + Vector128 originals = Vector128.Create(Vector64.LoadUnsafe(ref source, (nuint)(i + 2)), Vector64.Zero); + Vector128_.UnpackLow(halfSamples, originals).StoreUnsafe(ref firstOutput, (nuint)(2 * i)); + } + } + + if (Vector128.IsHardwareAccelerated) + { + int vectorEnd = count - Vector128.Count; + for (; i <= vectorEnd; i += Vector128.Count) + { + Vector128 b0 = Vector128.CreateScalar(Unsafe.As(ref Unsafe.Add(ref source, i + 0))).AsByte(); + Vector128 w0 = Vector128.WidenLower(b0); + Vector128 s0 = Vector128.WidenLower(w0).AsInt32(); + + Vector128 b1 = Vector128.CreateScalar(Unsafe.As(ref Unsafe.Add(ref source, i + 1))).AsByte(); + Vector128 w1 = Vector128.WidenLower(b1); + Vector128 s1 = Vector128.WidenLower(w1).AsInt32(); + + Vector128 b2 = Vector128.CreateScalar(Unsafe.As(ref Unsafe.Add(ref source, i + 2))).AsByte(); + Vector128 w2 = Vector128.WidenLower(b2); + Vector128 s2 = Vector128.WidenLower(w2).AsInt32(); + + Vector128 b3 = Vector128.CreateScalar(Unsafe.As(ref Unsafe.Add(ref source, i + 3))).AsByte(); + Vector128 w3 = Vector128.WidenLower(b3); + Vector128 s3 = Vector128.WidenLower(w3).AsInt32(); + + Vector128 values = TOperator.Interpolate(s0, s1, s2, s3, 255); + Vector128 halfSamples = Vector128.Narrow( + Vector128.Narrow(values, Vector128.Zero).AsUInt16(), Vector128.Zero); + + Vector128 originals = Vector128.CreateScalar(Unsafe.As(ref Unsafe.Add(ref source, i + 2))).AsByte(); + Vector128_.UnpackLow(halfSamples, originals).GetLower().StoreUnsafe(ref firstOutput, (nuint)(2 * i)); + } + } + + for (; i < count; i++) + { + int value = TOperator.Interpolate( + Unsafe.Add(ref source, i), + Unsafe.Add(ref source, i + 1), + Unsafe.Add(ref source, i + 2), + Unsafe.Add(ref source, i + 3), + 255); + + Unsafe.Add(ref destination, (2 * i) - 1) = (byte)value; + Unsafe.Add(ref destination, 2 * i) = Unsafe.Add(ref source, i + 2); + } + } + + /// + /// Inserts half samples using the original edge values and repeated endpoints. + /// + /// The edge with prefix and doubled output capacity. + /// The original sample count. + /// The maximum coded sample value. + /// The reusable original-sample workspace. + public static void Apply(Span edge, int count, int maximum, Span scratch) + { + ref short destination = ref MemoryMarshal.GetReference(edge); + ref short source = ref MemoryMarshal.GetReference(scratch); + + // Preserve the corner twice and the final sample once. Every SIMD load below covers exactly its + // input lanes, so the native 16+3-sample workspace also suffices for the widest interpolation. + source = Unsafe.Subtract(ref destination, 1); + Unsafe.Add(ref source, 1) = source; + edge[..count].CopyTo(scratch[2..]); + Unsafe.Add(ref source, count + 2) = edge[count - 1]; + Unsafe.Subtract(ref destination, 2) = source; + ref short firstOutput = ref Unsafe.Subtract(ref destination, 1); + int i = 0; + + if (Vector512.IsHardwareAccelerated) + { + int vectorEnd = count - Vector512.Count; + for (; i <= vectorEnd; i += Vector512.Count) + { + Vector512 s0 = Vector512.WidenLower(Vector512.Create( + Vector256.LoadUnsafe(ref source, (nuint)(i + 0)), Vector256.Zero)); + + Vector512 s1 = Vector512.WidenLower(Vector512.Create( + Vector256.LoadUnsafe(ref source, (nuint)(i + 1)), Vector256.Zero)); + + Vector512 s2 = Vector512.WidenLower(Vector512.Create( + Vector256.LoadUnsafe(ref source, (nuint)(i + 2)), Vector256.Zero)); + + Vector512 s3 = Vector512.WidenLower(Vector512.Create( + Vector256.LoadUnsafe(ref source, (nuint)(i + 3)), Vector256.Zero)); + + Vector512 values = TOperator.Interpolate(s0, s1, s2, s3, maximum); + Vector256 halfSamples = Vector512.Narrow(values, Vector512.Zero).GetLower(); + Vector256 originals = Vector256.LoadUnsafe(ref source, (nuint)(i + 2)); + + // Four contiguous groups of four pairs avoid treating lane-local unpack order as one + // linear 256-bit edge. Each store writes only prepared half samples and their originals. + Vector128_.UnpackLow(halfSamples.GetLower(), originals.GetLower()) + .StoreUnsafe(ref firstOutput, (nuint)(2 * i)); + + Vector128_.UnpackHigh(halfSamples.GetLower(), originals.GetLower()) + .StoreUnsafe(ref firstOutput, (nuint)((2 * i) + 8)); + + Vector128_.UnpackLow(halfSamples.GetUpper(), originals.GetUpper()) + .StoreUnsafe(ref firstOutput, (nuint)((2 * i) + 16)); + + Vector128_.UnpackHigh(halfSamples.GetUpper(), originals.GetUpper()) + .StoreUnsafe(ref firstOutput, (nuint)((2 * i) + 24)); + } + } + + if (Vector256.IsHardwareAccelerated) + { + int vectorEnd = count - Vector256.Count; + for (; i <= vectorEnd; i += Vector256.Count) + { + Vector256 s0 = Vector256.WidenLower(Vector256.Create( + Vector128.LoadUnsafe(ref source, (nuint)(i + 0)), Vector128.Zero)); + + Vector256 s1 = Vector256.WidenLower(Vector256.Create( + Vector128.LoadUnsafe(ref source, (nuint)(i + 1)), Vector128.Zero)); + + Vector256 s2 = Vector256.WidenLower(Vector256.Create( + Vector128.LoadUnsafe(ref source, (nuint)(i + 2)), Vector128.Zero)); + + Vector256 s3 = Vector256.WidenLower(Vector256.Create( + Vector128.LoadUnsafe(ref source, (nuint)(i + 3)), Vector128.Zero)); + + Vector256 values = TOperator.Interpolate(s0, s1, s2, s3, maximum); + Vector128 halfSamples = Vector256.Narrow(values, Vector256.Zero).GetLower(); + Vector128 originals = Vector128.LoadUnsafe(ref source, (nuint)(i + 2)); + Vector128_.UnpackLow(halfSamples, originals).StoreUnsafe(ref firstOutput, (nuint)(2 * i)); + Vector128_.UnpackHigh(halfSamples, originals).StoreUnsafe(ref firstOutput, (nuint)((2 * i) + 8)); + } + } + + if (Vector128.IsHardwareAccelerated) + { + int vectorEnd = count - Vector128.Count; + for (; i <= vectorEnd; i += Vector128.Count) + { + Vector128 s0 = Vector128.WidenLower(Vector128.Create( + Vector64.LoadUnsafe(ref source, (nuint)(i + 0)), Vector64.Zero)); + + Vector128 s1 = Vector128.WidenLower(Vector128.Create( + Vector64.LoadUnsafe(ref source, (nuint)(i + 1)), Vector64.Zero)); + + Vector128 s2 = Vector128.WidenLower(Vector128.Create( + Vector64.LoadUnsafe(ref source, (nuint)(i + 2)), Vector64.Zero)); + + Vector128 s3 = Vector128.WidenLower(Vector128.Create( + Vector64.LoadUnsafe(ref source, (nuint)(i + 3)), Vector64.Zero)); + + Vector128 values = TOperator.Interpolate(s0, s1, s2, s3, maximum); + Vector128 halfSamples = Vector128.Narrow(values, Vector128.Zero); + Vector128 originals = Vector128.Create( + Vector64.LoadUnsafe(ref source, (nuint)(i + 2)), Vector64.Zero); + + Vector128_.UnpackLow(halfSamples, originals).StoreUnsafe(ref firstOutput, (nuint)(2 * i)); + } + } + + for (; i < count; i++) + { + int value = TOperator.Interpolate( + Unsafe.Add(ref source, i), + Unsafe.Add(ref source, i + 1), + Unsafe.Add(ref source, i + 2), + Unsafe.Add(ref source, i + 3), + maximum); + + Unsafe.Add(ref destination, (2 * i) - 1) = (short)value; + Unsafe.Add(ref destination, 2 * i) = Unsafe.Add(ref source, i + 2); + } + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1IntraEdgeUpsampler.Operator.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1IntraEdgeUpsampler.Operator.cs new file mode 100644 index 0000000000..a8ed1fb5b2 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1IntraEdgeUpsampler.Operator.cs @@ -0,0 +1,91 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; + +/// +/// Inserts clipped half-sample positions into AV1 intra-reference edges. +/// +internal static partial class Av1IntraEdgeUpsampler +{ + /// + /// The maximum number of original samples permitted in an upsampled edge. + /// + public const int MaximumCount = 16; + + /// + /// The sample count required for the original edge, corner, and repeated endpoints. + /// + public const int ScratchLength = MaximumCount + 3; + + /// + /// Defines signed four-tap interpolation before sample narrowing. + /// + internal interface IEdgeUpsamplingOperator + { + /// + /// Interpolates half samples and clamps them to the coded range. + /// + /// The preceding samples. + /// The first central samples. + /// The second central samples. + /// The following samples. + /// The maximum coded sample. + /// The rounded and clipped half samples. + public static abstract int Interpolate(int a, int b, int c, int d, int maximum); + + /// + /// Interpolates half samples and clamps them to the coded range. + /// + /// The preceding samples. + /// The first central samples. + /// The second central samples. + /// The following samples. + /// The maximum coded sample. + /// The rounded and clipped half samples. + public static abstract Vector128 Interpolate(Vector128 a, Vector128 b, Vector128 c, Vector128 d, int maximum); + + /// + /// Interpolates half samples and clamps them to the coded range. + /// + /// The preceding samples. + /// The first central samples. + /// The second central samples. + /// The following samples. + /// The maximum coded sample. + /// The rounded and clipped half samples. + public static abstract Vector256 Interpolate(Vector256 a, Vector256 b, Vector256 c, Vector256 d, int maximum); + + /// + /// Interpolates half samples and clamps them to the coded range. + /// + /// The preceding samples. + /// The first central samples. + /// The second central samples. + /// The following samples. + /// The maximum coded sample. + /// The rounded and clipped half samples. + public static abstract Vector512 Interpolate(Vector512 a, Vector512 b, Vector512 c, Vector512 d, int maximum); + } + + /// + /// Inserts half samples before each original edge sample. + /// + /// The edge with writable prefix samples at -2 and -1 and room for the doubled extent. + /// The number of original edge samples, at most . + /// The original-sample workspace with at least samples. + public static void Apply(Span edge, int count, Span scratch) + => Upsampler.Apply(edge, count, scratch); + + /// + /// Inserts half samples before each original edge sample. + /// + /// The edge with writable prefix samples at -2 and -1 and room for the doubled extent. + /// The number of original edge samples, at most . + /// The coded precision used to clamp interpolation. + /// The original-sample workspace with at least samples. + public static void Apply(Span edge, int count, int bitDepth, Span scratch) + => Upsampler.Apply(edge, count, (1 << bitDepth) - 1, scratch); +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1IntraReferenceAvailability.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1IntraReferenceAvailability.cs new file mode 100644 index 0000000000..4c839b8381 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1IntraReferenceAvailability.cs @@ -0,0 +1,191 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; + +/// +/// Determines whether extended intra-prediction references have already been reconstructed. +/// +internal static class Av1IntraReferenceAvailability +{ + /// + /// Determines whether every bottom-left reference sample required by a transform is already reconstructed. + /// + /// The sequence superblock size. + /// The containing block size in the current plane's geometry. + /// The containing block row in 4-by-4 mode-information units. + /// The containing block column in 4-by-4 mode-information units. + /// A value indicating whether the required rows remain inside the frame and tile. + /// A value indicating whether reconstructed samples exist immediately to the left. + /// The partition type that determines reconstruction order. + /// The transform size whose extended edge is required. + /// The transform row offset within the containing block. + /// The transform column offset within the containing block. + /// The horizontal chroma subsampling shift. + /// The vertical chroma subsampling shift. + /// when the bottom-left reference extension is available; otherwise, . + public static bool HasBottomLeft(Av1BlockSize superblockSize, Av1BlockSize blockSize, int modeInfoRow, int modeInfoColumn, bool bottomAvailable, bool haveLeft, Av1PartitionType partition, Av1TransformSize transformSize, int blockModeInfoRowOffset, int blockModeInfoColumnOffset, int subX, int subY) + { + if (!bottomAvailable || !haveLeft) + { + return false; + } + + // A 128-wide block is reconstructed as two 64-wide regions in raster order, + // so the right half can consume references that already belong to the left half. + if (blockSize.GetWidth() > 64 && blockModeInfoColumnOffset > 0) + { + int block64WidthInUnits = Av1BlockSize.Block64x64.Get4x4WideCount(); + int planeBlockWidthInUnits64 = block64WidthInUnits >> subX; + int columnOffset64 = blockModeInfoColumnOffset % planeBlockWidthInUnits64; + if (columnOffset64 == 0) + { + // We are at the left edge of top-right or bottom-right 64x* block. + int block64HeightInUnits = Av1BlockSize.Block64x64.Get4x4HighCount(); + int planeBlockHeightInUnits64 = block64HeightInUnits >> subY; + int rowOffset64 = blockModeInfoRowOffset % planeBlockHeightInUnits64; + int planeBlockHeightInUnits = Math.Min(blockSize.Get4x4HighCount() >> subY, planeBlockHeightInUnits64); + + // Check if all bottom-left pixels are in the left 64x* block (which is + // already coded). + return rowOffset64 + transformSize.Get4x4HighCount() < planeBlockHeightInUnits; + } + } + + if (blockModeInfoColumnOffset > 0) + { + // Bottom-left pixels are in the bottom-left block, which is not available. + return false; + } + else + { + int blockHeightInUnits = blockSize.GetHeight() >> Av1TransformSize.Size4x4.GetBlockHeightLog2(); + int planeBlockHeightInUnits = Math.Max(blockHeightInUnits >> subY, 1); + int bottomLeftUnitCount = transformSize.Get4x4HighCount(); + + // All bottom-left pixels are in the left block, which is already available. + if (blockModeInfoRowOffset + bottomLeftUnitCount < planeBlockHeightInUnits) + { + return true; + } + + int blockWidthInModeInfoLog2 = blockSize.Get4x4WidthLog2(); + int blockHeightInModeInfoLog2 = blockSize.Get4x4HeightLog2(); + int superblockModeInfoSize = superblockSize.Get4x4HighCount(); + int blockRowInSuperblock = (modeInfoRow & (superblockModeInfoSize - 1)) >> blockHeightInModeInfoLog2; + int blockColumnInSuperblock = (modeInfoColumn & (superblockModeInfoSize - 1)) >> blockWidthInModeInfoLog2; + + // Leftmost column of superblock: so bottom-left pixels maybe in the left + // and/or bottom-left superblocks. But only the left superblock is + // available, so check if all required pixels fall in that superblock. + if (blockColumnInSuperblock == 0) + { + int blockStartRowOffset = blockRowInSuperblock << (blockHeightInModeInfoLog2 + Av1Constants.ModeInfoSizeLog2 - Av1TransformSize.Size4x4.GetBlockWidthLog2()) >> subY; + int rowOffsetInSuperblock = blockStartRowOffset + blockModeInfoRowOffset; + int superblockHeightInUnits = superblockModeInfoSize >> subY; + return rowOffsetInSuperblock + bottomLeftUnitCount < superblockHeightInUnits; + } + + // Bottom row of superblock (and not the leftmost column): so bottom-left + // pixels fall in the bottom superblock, which is not available yet. + if (((blockRowInSuperblock + 1) << blockHeightInModeInfoLog2) >= superblockModeInfoSize) + { + return false; + } + + // General case (neither leftmost column nor bottom row): check if the + // bottom-left block is coded before the current block. + int thisBlockIndex = ((blockRowInSuperblock + 0) << (Av1Constants.MaxSuperBlockSizeLog2 - Av1Constants.ModeInfoSizeLog2 - blockWidthInModeInfoLog2)) + blockColumnInSuperblock + 0; + return Av1BottomRightTopLeftConstants.HasBottomLeft(partition, blockSize, thisBlockIndex); + } + } + + /// + /// Determines whether every top-right reference sample required by a transform is already reconstructed. + /// + /// The sequence superblock size. + /// The containing block size in the current plane's geometry. + /// The containing block row in 4-by-4 mode-information units. + /// The containing block column in 4-by-4 mode-information units. + /// A value indicating whether reconstructed samples exist immediately above. + /// A value indicating whether the required columns remain inside the frame and tile. + /// The partition type that determines reconstruction order. + /// The transform size whose extended edge is required. + /// The transform row offset within the containing block. + /// The transform column offset within the containing block. + /// The horizontal chroma subsampling shift. + /// The vertical chroma subsampling shift. + /// when the top-right reference extension is available; otherwise, . + public static bool HasTopRight(Av1BlockSize superblockSize, Av1BlockSize blockSize, int modeInfoRow, int modeInfoColumn, bool haveTop, bool rightAvailable, Av1PartitionType partition, Av1TransformSize transformSize, int blockModeInfoRowOffset, int blockModeInfoColumnOffset, int subX, int subY) + { + if (!haveTop || !rightAvailable) + { + return false; + } + + int blockWideInUnits = blockSize.GetWidth() >> 2; + int planeBlockWidthInUnits = Math.Max(blockWideInUnits >> subX, 1); + int topRightUnitCount = transformSize.Get4x4WideCount(); + + if (blockModeInfoRowOffset > 0) + { + // Transforms below the first row obtain their top edge from the containing block, + // so only the reconstructed width to their right constrains availability. + if (blockSize.GetWidth() > 64) + { + // Special case: For 128x128 blocks, the transform unit whose + // top-right corner is at the center of the block does in fact have + // pixels available at its top-right corner. + int block64WidthInUnits = Av1BlockSize.Block64x64.Get4x4WideCount(); + int block64HeightInUnits = Av1BlockSize.Block64x64.Get4x4HighCount(); + if (blockModeInfoRowOffset == block64HeightInUnits >> subY && + blockModeInfoColumnOffset + topRightUnitCount == block64WidthInUnits >> subX) + { + return true; + } + + int planeBlockWidthInUnits64 = block64WidthInUnits >> subX; + int blockModeInfoColumnOffset64 = blockModeInfoColumnOffset % planeBlockWidthInUnits64; + return blockModeInfoColumnOffset64 + topRightUnitCount < planeBlockWidthInUnits64; + } + + return blockModeInfoColumnOffset + topRightUnitCount < planeBlockWidthInUnits; + } + else + { + // All top-right pixels are in the block above, which is already available. + if (blockModeInfoColumnOffset + topRightUnitCount < planeBlockWidthInUnits) + { + return true; + } + + int blockWidthInModeInfoLog2 = blockSize.Get4x4WidthLog2(); + int blockHeightInModeInfeLog2 = blockSize.Get4x4HeightLog2(); + int superBlockModeInfoSize = superblockSize.Get4x4HighCount(); + int blockRowInSuperblock = (modeInfoRow & (superBlockModeInfoSize - 1)) >> blockHeightInModeInfeLog2; + int blockColumnInSuperBlock = (modeInfoColumn & (superBlockModeInfoSize - 1)) >> blockWidthInModeInfoLog2; + + // Top row of superblock: so top-right pixels are in the top and/or + // top-right superblocks, both of which are already available. + if (blockRowInSuperblock == 0) + { + return true; + } + + // Rightmost column of superblock (and not the top row): so top-right pixels + // fall in the right superblock, which is not available yet. + if (((blockColumnInSuperBlock + 1) << blockWidthInModeInfoLog2) >= superBlockModeInfoSize) + { + return false; + } + + // General case (neither top row nor rightmost column): check if the + // top-right block is coded before the current block. + int thisBlockIndex = ((blockRowInSuperblock + 0) << (Av1Constants.MaxSuperBlockSizeLog2 - Av1Constants.ModeInfoSizeLog2 - blockWidthInModeInfoLog2)) + blockColumnInSuperBlock + 0; + return Av1BottomRightTopLeftConstants.HasTopRight(partition, blockSize, thisBlockIndex); + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1NeighborNeed.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1NeighborNeed.cs new file mode 100644 index 0000000000..3dba6f76ce --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1NeighborNeed.cs @@ -0,0 +1,41 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; + +/// +/// Identifies the neighboring sample regions required by an AV1 intra-prediction mode. +/// +[Flags] +internal enum Av1NeighborNeed +{ + /// + /// No neighboring samples are required. + /// + Nothing = 0, + + /// + /// Samples immediately left of the block are required. + /// + Left = 2, + + /// + /// Samples immediately above the block are required. + /// + Above = 4, + + /// + /// Samples extending right of the top edge are required. + /// + AboveRight = 8, + + /// + /// The sample diagonally above and left of the block is required. + /// + AboveLeft = 16, + + /// + /// Samples extending below the left edge are required. + /// + BottomLeft = 32, +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1NonDirectionalIntraPredictor.HorizontalOperator.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1NonDirectionalIntraPredictor.HorizontalOperator.cs new file mode 100644 index 0000000000..1cb367266a --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1NonDirectionalIntraPredictor.HorizontalOperator.cs @@ -0,0 +1,57 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; + +/// +/// Provides horizontal intra prediction for scalar and SIMD sample representations. +/// +internal abstract partial class Av1NonDirectionalIntraPredictorBase +{ + /// + /// Implements AV1 horizontal intra prediction for scalar and SIMD lanes. + /// + internal readonly struct HorizontalOperator : IAv1IntraPredictionOperator + { + /// + public static Av1PredictionMode Mode => Av1PredictionMode.Horizontal; + + /// + public static Av1IntraPredictionInputs Inputs => Av1IntraPredictionInputs.Left; + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static byte Predict(byte top, byte left, byte topLeft, byte topRight, byte bottomLeft, int columnWeight, int rowWeight) => left; + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector128 Predict(Vector128 top, Vector128 left, Vector128 topLeft, Vector128 topRight, Vector128 bottomLeft, ref int columnWeights, int rowWeight) => left; + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector256 Predict(Vector256 top, Vector256 left, Vector256 topLeft, Vector256 topRight, Vector256 bottomLeft, ref int columnWeights, int rowWeight) => left; + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector512 Predict(Vector512 top, Vector512 left, Vector512 topLeft, Vector512 topRight, Vector512 bottomLeft, ref int columnWeights, int rowWeight) => left; + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static short Predict(short top, short left, short topLeft, short topRight, short bottomLeft, int columnWeight, int rowWeight) => left; + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector128 Predict(Vector128 top, Vector128 left, Vector128 topLeft, Vector128 topRight, Vector128 bottomLeft, ref int columnWeights, int rowWeight) => left; + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector256 Predict(Vector256 top, Vector256 left, Vector256 topLeft, Vector256 topRight, Vector256 bottomLeft, ref int columnWeights, int rowWeight) => left; + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector512 Predict(Vector512 top, Vector512 left, Vector512 topLeft, Vector512 topRight, Vector512 bottomLeft, ref int columnWeights, int rowWeight) => left; + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1NonDirectionalIntraPredictor.Operator.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1NonDirectionalIntraPredictor.Operator.cs new file mode 100644 index 0000000000..25688ecc54 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1NonDirectionalIntraPredictor.Operator.cs @@ -0,0 +1,531 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.InteropServices; +using System.Runtime.Intrinsics; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; + +/// +/// Defines the neighbor-usage flags and scalar/SIMD contract for closed intra-prediction operators, and provides +/// their shared width-progressive SIMD traversal. +/// +internal abstract partial class Av1NonDirectionalIntraPredictorBase +{ + /// + /// Identifies the neighboring inputs consumed by an AV1 intra-prediction operator. + /// + [Flags] + internal enum Av1IntraPredictionInputs + { + /// + /// The operator does not consume neighboring samples. + /// + None = 0, + + /// + /// The operator consumes samples from the top reference. + /// + Top = 1, + + /// + /// The operator consumes samples from the left reference. + /// + Left = 2, + + /// + /// The operator consumes the shared top-left reference. + /// + TopLeft = 4, + + /// + /// The operator consumes the final top reference. + /// + TopRight = 8, + + /// + /// The operator consumes the final left reference. + /// + BottomLeft = 16, + + /// + /// The operator consumes the horizontal smooth weights. + /// + ColumnWeight = 32, + + /// + /// The operator consumes the vertical smooth weights. + /// + RowWeight = 64, + } + + /// + /// Defines the scalar and SIMD arithmetic for one non-directional AV1 intra-prediction mode. + /// + /// + /// Each overload performs the same lane-wise operation. The generic predictor traversal selects the widest + /// available overload, and the JIT specializes each static interface call for the closed operator type. + /// + internal interface IAv1IntraPredictionOperator + { + /// + /// Gets the prediction mode implemented by the operator. + /// + public static abstract Av1PredictionMode Mode { get; } + + /// + /// Gets the neighboring inputs consumed by the operator. + /// + public static abstract Av1IntraPredictionInputs Inputs { get; } + + /// + /// Predicts one 8-bit sample when hardware vectorization is unavailable. + /// + /// The top reference sample. + /// The left reference sample. + /// The shared top-left reference sample. + /// The final top reference sample. + /// The final left reference sample. + /// The horizontal Q8 smooth weight. + /// The vertical Q8 smooth weight. + /// The predicted sample. + public static abstract byte Predict(byte top, byte left, byte topLeft, byte topRight, byte bottomLeft, int columnWeight, int rowWeight); + + /// + /// Predicts sixteen 8-bit samples in parallel. + /// + /// The top reference samples. + /// The left reference sample in every lane. + /// The shared top-left reference sample in every lane. + /// The final top reference sample in every lane. + /// The final left reference sample in every lane. + /// The first horizontal Q8 smooth weight for these lanes. + /// The vertical Q8 smooth weight. + /// The predicted samples. + public static abstract Vector128 Predict(Vector128 top, Vector128 left, Vector128 topLeft, Vector128 topRight, Vector128 bottomLeft, ref int columnWeights, int rowWeight); + + /// + /// Predicts thirty-two 8-bit samples in parallel. + /// + /// The top reference samples. + /// The left reference sample in every lane. + /// The shared top-left reference sample in every lane. + /// The final top reference sample in every lane. + /// The final left reference sample in every lane. + /// The first horizontal Q8 smooth weight for these lanes. + /// The vertical Q8 smooth weight. + /// The predicted samples. + public static abstract Vector256 Predict(Vector256 top, Vector256 left, Vector256 topLeft, Vector256 topRight, Vector256 bottomLeft, ref int columnWeights, int rowWeight); + + /// + /// Predicts sixty-four 8-bit samples in parallel. + /// + /// The top reference samples. + /// The left reference sample in every lane. + /// The shared top-left reference sample in every lane. + /// The final top reference sample in every lane. + /// The final left reference sample in every lane. + /// The first horizontal Q8 smooth weight for these lanes. + /// The vertical Q8 smooth weight. + /// The predicted samples. + public static abstract Vector512 Predict(Vector512 top, Vector512 left, Vector512 topLeft, Vector512 topRight, Vector512 bottomLeft, ref int columnWeights, int rowWeight); + + /// + /// Predicts one high-bit-depth sample when hardware vectorization is unavailable. + /// + /// The top reference sample. + /// The left reference sample. + /// The shared top-left reference sample. + /// The final top reference sample. + /// The final left reference sample. + /// The horizontal Q8 smooth weight. + /// The vertical Q8 smooth weight. + /// The predicted sample. + public static abstract short Predict(short top, short left, short topLeft, short topRight, short bottomLeft, int columnWeight, int rowWeight); + + /// + /// Predicts eight high-bit-depth samples in parallel. + /// + /// The top reference samples. + /// The left reference sample in every lane. + /// The shared top-left reference sample in every lane. + /// The final top reference sample in every lane. + /// The final left reference sample in every lane. + /// The first horizontal Q8 smooth weight for these lanes. + /// The vertical Q8 smooth weight. + /// The predicted samples. + public static abstract Vector128 Predict(Vector128 top, Vector128 left, Vector128 topLeft, Vector128 topRight, Vector128 bottomLeft, ref int columnWeights, int rowWeight); + + /// + /// Predicts sixteen high-bit-depth samples in parallel. + /// + /// The top reference samples. + /// The left reference sample in every lane. + /// The shared top-left reference sample in every lane. + /// The final top reference sample in every lane. + /// The final left reference sample in every lane. + /// The first horizontal Q8 smooth weight for these lanes. + /// The vertical Q8 smooth weight. + /// The predicted samples. + public static abstract Vector256 Predict(Vector256 top, Vector256 left, Vector256 topLeft, Vector256 topRight, Vector256 bottomLeft, ref int columnWeights, int rowWeight); + + /// + /// Predicts thirty-two high-bit-depth samples in parallel. + /// + /// The top reference samples. + /// The left reference sample in every lane. + /// The shared top-left reference sample in every lane. + /// The final top reference sample in every lane. + /// The final left reference sample in every lane. + /// The first horizontal Q8 smooth weight for these lanes. + /// The vertical Q8 smooth weight. + /// The predicted samples. + public static abstract Vector512 Predict(Vector512 top, Vector512 left, Vector512 topLeft, Vector512 topRight, Vector512 bottomLeft, ref int columnWeights, int rowWeight); + } + + /// + /// Applies one closed non-directional AV1 prediction operator using the widest available SIMD width. + /// + /// The prediction-mode-specific arithmetic. + /// + /// Each lane produces one output column. Top samples and column weights vary by lane, while the current row's + /// left sample and row weight are broadcast. declares which references it uses; + /// because the operator type is closed, the JIT can remove unused loads and broadcasts from each prediction mode. + /// + internal sealed class Av1NonDirectionalIntraPredictor : Av1NonDirectionalIntraPredictorBase + where TOperator : struct, IAv1IntraPredictionOperator + { + /// + public override Av1PredictionMode Mode => TOperator.Mode; + + /// + public override void Predict(Span destination, int destinationStride, ReadOnlySpan above, ReadOnlySpan left, int width, int height) + { + ref byte destinationBase = ref MemoryMarshal.GetReference(destination); + ref byte topBase = ref MemoryMarshal.GetReference(above); + ref byte leftBase = ref MemoryMarshal.GetReference(left); + ref int columnWeightBase = ref MemoryMarshal.GetReference(SmoothWeights[width..]); + + Av1IntraPredictionInputs inputs = TOperator.Inputs; + bool usesTop = (inputs & Av1IntraPredictionInputs.Top) != 0; + bool usesLeft = (inputs & Av1IntraPredictionInputs.Left) != 0; + bool usesTopLeft = (inputs & Av1IntraPredictionInputs.TopLeft) != 0; + bool usesTopRight = (inputs & Av1IntraPredictionInputs.TopRight) != 0; + bool usesBottomLeft = (inputs & Av1IntraPredictionInputs.BottomLeft) != 0; + bool usesColumnWeight = (inputs & Av1IntraPredictionInputs.ColumnWeight) != 0; + bool usesRowWeight = (inputs & Av1IntraPredictionInputs.RowWeight) != 0; + + byte topLeft = usesTopLeft ? Unsafe.Subtract(ref topBase, 1) : default; + byte topRight = usesTopRight ? Unsafe.Add(ref topBase, width - 1) : default; + byte bottomLeft = usesBottomLeft ? Unsafe.Add(ref leftBase, height - 1) : default; + int processedColumns = 0; + + // Widths are cumulative rather than mutually exclusive. A wide vector advances the row prefix, then the + // narrower paths consume any complete vectors left before the scalar tail handles the final columns. + if (Vector512.IsHardwareAccelerated) + { + int vectorizedColumns = (int)(Numerics.Vector512Count(width) * (nuint)Vector512.Count); + if (vectorizedColumns > 0) + { + Vector512 topLeftVector = usesTopLeft ? Vector512.Create(topLeft) : default; + Vector512 topRightVector = usesTopRight ? Vector512.Create(topRight) : default; + Vector512 bottomLeftVector = usesBottomLeft ? Vector512.Create(bottomLeft) : default; + + for (int row = 0; row < height; row++) + { + Vector512 leftVector = usesLeft ? Vector512.Create(Unsafe.Add(ref leftBase, row)) : default; + int rowWeight = usesRowWeight ? SmoothWeights[height + row] : 0; + ref byte destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + + for (int column = 0; column < vectorizedColumns; column += Vector512.Count) + { + Vector512 top = usesTop ? Vector512.LoadUnsafe(ref topBase, (nuint)column) : default; + ref int columnWeights = ref usesColumnWeight ? ref Unsafe.Add(ref columnWeightBase, column) : ref columnWeightBase; + Vector512 prediction = TOperator.Predict(top, leftVector, topLeftVector, topRightVector, bottomLeftVector, ref columnWeights, rowWeight); + prediction.StoreUnsafe(ref destinationRow, (nuint)column); + } + } + + processedColumns = vectorizedColumns; + } + } + + if (Vector256.IsHardwareAccelerated) + { + int remainingColumns = width - processedColumns; + int vectorizedColumns = (int)(Numerics.Vector256Count(remainingColumns) * (nuint)Vector256.Count); + if (vectorizedColumns > 0) + { + Vector256 topLeftVector = usesTopLeft ? Vector256.Create(topLeft) : default; + Vector256 topRightVector = usesTopRight ? Vector256.Create(topRight) : default; + Vector256 bottomLeftVector = usesBottomLeft ? Vector256.Create(bottomLeft) : default; + int endColumn = processedColumns + vectorizedColumns; + + for (int row = 0; row < height; row++) + { + Vector256 leftVector = usesLeft ? Vector256.Create(Unsafe.Add(ref leftBase, row)) : default; + int rowWeight = usesRowWeight ? SmoothWeights[height + row] : 0; + ref byte destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + + for (int column = processedColumns; column < endColumn; column += Vector256.Count) + { + Vector256 top = usesTop ? Vector256.LoadUnsafe(ref topBase, (nuint)column) : default; + ref int columnWeights = ref usesColumnWeight ? ref Unsafe.Add(ref columnWeightBase, column) : ref columnWeightBase; + Vector256 prediction = TOperator.Predict(top, leftVector, topLeftVector, topRightVector, bottomLeftVector, ref columnWeights, rowWeight); + prediction.StoreUnsafe(ref destinationRow, (nuint)column); + } + } + + processedColumns = endColumn; + } + } + + if (Vector128.IsHardwareAccelerated) + { + int remainingColumns = width - processedColumns; + int vectorizedColumns = (int)(Numerics.Vector128Count(remainingColumns) * (nuint)Vector128.Count); + if (vectorizedColumns > 0) + { + Vector128 topLeftVector = usesTopLeft ? Vector128.Create(topLeft) : default; + Vector128 topRightVector = usesTopRight ? Vector128.Create(topRight) : default; + Vector128 bottomLeftVector = usesBottomLeft ? Vector128.Create(bottomLeft) : default; + int endColumn = processedColumns + vectorizedColumns; + + for (int row = 0; row < height; row++) + { + Vector128 leftVector = usesLeft ? Vector128.Create(Unsafe.Add(ref leftBase, row)) : default; + int rowWeight = usesRowWeight ? SmoothWeights[height + row] : 0; + ref byte destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + + for (int column = processedColumns; column < endColumn; column += Vector128.Count) + { + Vector128 top = usesTop ? Vector128.LoadUnsafe(ref topBase, (nuint)column) : default; + ref int columnWeights = ref usesColumnWeight ? ref Unsafe.Add(ref columnWeightBase, column) : ref columnWeightBase; + Vector128 prediction = TOperator.Predict(top, leftVector, topLeftVector, topRightVector, bottomLeftVector, ref columnWeights, rowWeight); + prediction.StoreUnsafe(ref destinationRow, (nuint)column); + } + } + + processedColumns = endColumn; + } + } + + // AV1 dimensions are multiples of four. The tail is normally zero on SIMD hardware, but retaining the + // scalar continuation keeps the traversal correct when intrinsics are disabled by FeatureTestRunner. + for (int row = 0; row < height; row++) + { + byte leftSample = usesLeft ? Unsafe.Add(ref leftBase, row) : default; + int rowWeight = usesRowWeight ? SmoothWeights[height + row] : 0; + ref byte destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + + for (int column = processedColumns; column < width; column++) + { + byte top = usesTop ? Unsafe.Add(ref topBase, column) : default; + int columnWeight = usesColumnWeight ? Unsafe.Add(ref columnWeightBase, column) : 0; + Unsafe.Add(ref destinationRow, column) = TOperator.Predict(top, leftSample, topLeft, topRight, bottomLeft, columnWeight, rowWeight); + } + } + } + + /// + public override void Predict(Span destination, int destinationStride, ReadOnlySpan above, ReadOnlySpan left, int width, int height) + { + ref short destinationBase = ref MemoryMarshal.GetReference(destination); + ref short topBase = ref MemoryMarshal.GetReference(above); + ref short leftBase = ref MemoryMarshal.GetReference(left); + ref int columnWeightBase = ref MemoryMarshal.GetReference(SmoothWeights[width..]); + + Av1IntraPredictionInputs inputs = TOperator.Inputs; + bool usesTop = (inputs & Av1IntraPredictionInputs.Top) != 0; + bool usesLeft = (inputs & Av1IntraPredictionInputs.Left) != 0; + bool usesTopLeft = (inputs & Av1IntraPredictionInputs.TopLeft) != 0; + bool usesTopRight = (inputs & Av1IntraPredictionInputs.TopRight) != 0; + bool usesBottomLeft = (inputs & Av1IntraPredictionInputs.BottomLeft) != 0; + bool usesColumnWeight = (inputs & Av1IntraPredictionInputs.ColumnWeight) != 0; + bool usesRowWeight = (inputs & Av1IntraPredictionInputs.RowWeight) != 0; + + short topLeft = usesTopLeft ? Unsafe.Subtract(ref topBase, 1) : default; + short topRight = usesTopRight ? Unsafe.Add(ref topBase, width - 1) : default; + short bottomLeft = usesBottomLeft ? Unsafe.Add(ref leftBase, height - 1) : default; + int processedColumns = 0; + + // High-bit-depth samples use signed storage but remain nonnegative. Each vector lane follows one output + // column, so the same width-progressive traversal is valid without inter-lane packing or saturation. + if (Vector512.IsHardwareAccelerated) + { + int vectorizedColumns = (int)(Numerics.Vector512Count(width) * (nuint)Vector512.Count); + if (vectorizedColumns > 0) + { + Vector512 topLeftVector = usesTopLeft ? Vector512.Create(topLeft) : default; + Vector512 topRightVector = usesTopRight ? Vector512.Create(topRight) : default; + Vector512 bottomLeftVector = usesBottomLeft ? Vector512.Create(bottomLeft) : default; + + for (int row = 0; row < height; row++) + { + Vector512 leftVector = usesLeft ? Vector512.Create(Unsafe.Add(ref leftBase, row)) : default; + int rowWeight = usesRowWeight ? SmoothWeights[height + row] : 0; + ref short destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + + for (int column = 0; column < vectorizedColumns; column += Vector512.Count) + { + Vector512 top = usesTop ? Vector512.LoadUnsafe(ref topBase, (nuint)column) : default; + ref int columnWeights = ref usesColumnWeight ? ref Unsafe.Add(ref columnWeightBase, column) : ref columnWeightBase; + Vector512 prediction = TOperator.Predict(top, leftVector, topLeftVector, topRightVector, bottomLeftVector, ref columnWeights, rowWeight); + prediction.StoreUnsafe(ref destinationRow, (nuint)column); + } + } + + processedColumns = vectorizedColumns; + } + } + + if (Vector256.IsHardwareAccelerated) + { + int remainingColumns = width - processedColumns; + int vectorizedColumns = (int)(Numerics.Vector256Count(remainingColumns) * (nuint)Vector256.Count); + if (vectorizedColumns > 0) + { + Vector256 topLeftVector = usesTopLeft ? Vector256.Create(topLeft) : default; + Vector256 topRightVector = usesTopRight ? Vector256.Create(topRight) : default; + Vector256 bottomLeftVector = usesBottomLeft ? Vector256.Create(bottomLeft) : default; + int endColumn = processedColumns + vectorizedColumns; + + for (int row = 0; row < height; row++) + { + Vector256 leftVector = usesLeft ? Vector256.Create(Unsafe.Add(ref leftBase, row)) : default; + int rowWeight = usesRowWeight ? SmoothWeights[height + row] : 0; + ref short destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + + for (int column = processedColumns; column < endColumn; column += Vector256.Count) + { + Vector256 top = usesTop ? Vector256.LoadUnsafe(ref topBase, (nuint)column) : default; + ref int columnWeights = ref usesColumnWeight ? ref Unsafe.Add(ref columnWeightBase, column) : ref columnWeightBase; + Vector256 prediction = TOperator.Predict(top, leftVector, topLeftVector, topRightVector, bottomLeftVector, ref columnWeights, rowWeight); + prediction.StoreUnsafe(ref destinationRow, (nuint)column); + } + } + + processedColumns = endColumn; + } + } + + if (Vector128.IsHardwareAccelerated) + { + int remainingColumns = width - processedColumns; + int vectorizedColumns = (int)(Numerics.Vector128Count(remainingColumns) * (nuint)Vector128.Count); + if (vectorizedColumns > 0) + { + Vector128 topLeftVector = usesTopLeft ? Vector128.Create(topLeft) : default; + Vector128 topRightVector = usesTopRight ? Vector128.Create(topRight) : default; + Vector128 bottomLeftVector = usesBottomLeft ? Vector128.Create(bottomLeft) : default; + int endColumn = processedColumns + vectorizedColumns; + + for (int row = 0; row < height; row++) + { + Vector128 leftVector = usesLeft ? Vector128.Create(Unsafe.Add(ref leftBase, row)) : default; + int rowWeight = usesRowWeight ? SmoothWeights[height + row] : 0; + ref short destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + + for (int column = processedColumns; column < endColumn; column += Vector128.Count) + { + Vector128 top = usesTop ? Vector128.LoadUnsafe(ref topBase, (nuint)column) : default; + ref int columnWeights = ref usesColumnWeight ? ref Unsafe.Add(ref columnWeightBase, column) : ref columnWeightBase; + Vector128 prediction = TOperator.Predict(top, leftVector, topLeftVector, topRightVector, bottomLeftVector, ref columnWeights, rowWeight); + prediction.StoreUnsafe(ref destinationRow, (nuint)column); + } + } + + processedColumns = endColumn; + } + } + + // Retain a scalar continuation for widths smaller than the available vectors and for forced-scalar test + // execution. AV1 block dimensions keep this tail short during normal hardware-accelerated decoding. + for (int row = 0; row < height; row++) + { + short leftSample = usesLeft ? Unsafe.Add(ref leftBase, row) : default; + int rowWeight = usesRowWeight ? SmoothWeights[height + row] : 0; + ref short destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + + for (int column = processedColumns; column < width; column++) + { + short top = usesTop ? Unsafe.Add(ref topBase, column) : default; + int columnWeight = usesColumnWeight ? Unsafe.Add(ref columnWeightBase, column) : 0; + Unsafe.Add(ref destinationRow, column) = TOperator.Predict(top, leftSample, topLeft, topRight, bottomLeft, columnWeight, rowWeight); + } + } + } + + /// + public override void PredictScalar(Span destination, int destinationStride, ReadOnlySpan above, ReadOnlySpan left, int width, int height) + { + ref byte destinationBase = ref MemoryMarshal.GetReference(destination); + ref byte topBase = ref MemoryMarshal.GetReference(above); + ref byte leftBase = ref MemoryMarshal.GetReference(left); + ref int columnWeightBase = ref MemoryMarshal.GetReference(SmoothWeights[width..]); + + Av1IntraPredictionInputs inputs = TOperator.Inputs; + bool usesTop = (inputs & Av1IntraPredictionInputs.Top) != 0; + bool usesLeft = (inputs & Av1IntraPredictionInputs.Left) != 0; + bool usesTopLeft = (inputs & Av1IntraPredictionInputs.TopLeft) != 0; + bool usesTopRight = (inputs & Av1IntraPredictionInputs.TopRight) != 0; + bool usesBottomLeft = (inputs & Av1IntraPredictionInputs.BottomLeft) != 0; + bool usesColumnWeight = (inputs & Av1IntraPredictionInputs.ColumnWeight) != 0; + bool usesRowWeight = (inputs & Av1IntraPredictionInputs.RowWeight) != 0; + byte topLeft = usesTopLeft ? Unsafe.Subtract(ref topBase, 1) : default; + byte topRight = usesTopRight ? Unsafe.Add(ref topBase, width - 1) : default; + byte bottomLeft = usesBottomLeft ? Unsafe.Add(ref leftBase, height - 1) : default; + + for (int row = 0; row < height; row++) + { + byte leftSample = usesLeft ? Unsafe.Add(ref leftBase, row) : default; + int rowWeight = usesRowWeight ? SmoothWeights[height + row] : 0; + ref byte destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + + for (int column = 0; column < width; column++) + { + byte top = usesTop ? Unsafe.Add(ref topBase, column) : default; + int columnWeight = usesColumnWeight ? Unsafe.Add(ref columnWeightBase, column) : 0; + Unsafe.Add(ref destinationRow, column) = TOperator.Predict(top, leftSample, topLeft, topRight, bottomLeft, columnWeight, rowWeight); + } + } + } + + /// + public override void PredictScalar(Span destination, int destinationStride, ReadOnlySpan above, ReadOnlySpan left, int width, int height) + { + ref short destinationBase = ref MemoryMarshal.GetReference(destination); + ref short topBase = ref MemoryMarshal.GetReference(above); + ref short leftBase = ref MemoryMarshal.GetReference(left); + ref int columnWeightBase = ref MemoryMarshal.GetReference(SmoothWeights[width..]); + + Av1IntraPredictionInputs inputs = TOperator.Inputs; + bool usesTop = (inputs & Av1IntraPredictionInputs.Top) != 0; + bool usesLeft = (inputs & Av1IntraPredictionInputs.Left) != 0; + bool usesTopLeft = (inputs & Av1IntraPredictionInputs.TopLeft) != 0; + bool usesTopRight = (inputs & Av1IntraPredictionInputs.TopRight) != 0; + bool usesBottomLeft = (inputs & Av1IntraPredictionInputs.BottomLeft) != 0; + bool usesColumnWeight = (inputs & Av1IntraPredictionInputs.ColumnWeight) != 0; + bool usesRowWeight = (inputs & Av1IntraPredictionInputs.RowWeight) != 0; + short topLeft = usesTopLeft ? Unsafe.Subtract(ref topBase, 1) : default; + short topRight = usesTopRight ? Unsafe.Add(ref topBase, width - 1) : default; + short bottomLeft = usesBottomLeft ? Unsafe.Add(ref leftBase, height - 1) : default; + + for (int row = 0; row < height; row++) + { + short leftSample = usesLeft ? Unsafe.Add(ref leftBase, row) : default; + int rowWeight = usesRowWeight ? SmoothWeights[height + row] : 0; + ref short destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + + for (int column = 0; column < width; column++) + { + short top = usesTop ? Unsafe.Add(ref topBase, column) : default; + int columnWeight = usesColumnWeight ? Unsafe.Add(ref columnWeightBase, column) : 0; + Unsafe.Add(ref destinationRow, column) = TOperator.Predict(top, leftSample, topLeft, topRight, bottomLeft, columnWeight, rowWeight); + } + } + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1NonDirectionalIntraPredictor.Packing.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1NonDirectionalIntraPredictor.Packing.cs new file mode 100644 index 0000000000..f425e56098 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1NonDirectionalIntraPredictor.Packing.cs @@ -0,0 +1,171 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; + +/// +/// Widens AV1 sample vectors for fixed-point prediction arithmetic and narrows completed results. +/// +/// +/// Widening preserves consecutive sample order while splitting packed vectors into equal low-to-high groups. Callers +/// perform interpolation in signed 32-bit lanes and clip before narrowing, so the unsigned byte overloads may use +/// unsigned narrowing and the high-bit-depth overloads may use signed narrowing without additional saturation logic. +/// +internal abstract partial class Av1NonDirectionalIntraPredictorBase +{ + /// + /// Widens sixteen 8-bit samples into four 32-bit vectors. + /// + /// The packed samples. + /// The first four widened samples. + /// The second four widened samples. + /// The third four widened samples. + /// The fourth four widened samples. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void Widen(Vector128 source, out Vector128 result0, out Vector128 result1, out Vector128 result2, out Vector128 result3) + { + (Vector128 low, Vector128 high) = Vector128.Widen(source); + (Vector128 low0, Vector128 low1) = Vector128.Widen(low); + (Vector128 high0, Vector128 high1) = Vector128.Widen(high); + result0 = low0.AsInt32(); + result1 = low1.AsInt32(); + result2 = high0.AsInt32(); + result3 = high1.AsInt32(); + } + + /// + /// Widens thirty-two 8-bit samples into four 32-bit vectors. + /// + /// The packed samples. + /// The first eight widened samples. + /// The second eight widened samples. + /// The third eight widened samples. + /// The fourth eight widened samples. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void Widen(Vector256 source, out Vector256 result0, out Vector256 result1, out Vector256 result2, out Vector256 result3) + { + (Vector256 low, Vector256 high) = Vector256.Widen(source); + (Vector256 low0, Vector256 low1) = Vector256.Widen(low); + (Vector256 high0, Vector256 high1) = Vector256.Widen(high); + result0 = low0.AsInt32(); + result1 = low1.AsInt32(); + result2 = high0.AsInt32(); + result3 = high1.AsInt32(); + } + + /// + /// Widens sixty-four 8-bit samples into four 32-bit vectors. + /// + /// The packed samples. + /// The first sixteen widened samples. + /// The second sixteen widened samples. + /// The third sixteen widened samples. + /// The fourth sixteen widened samples. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void Widen(Vector512 source, out Vector512 result0, out Vector512 result1, out Vector512 result2, out Vector512 result3) + { + (Vector512 low, Vector512 high) = Vector512.Widen(source); + (Vector512 low0, Vector512 low1) = Vector512.Widen(low); + (Vector512 high0, Vector512 high1) = Vector512.Widen(high); + result0 = low0.AsInt32(); + result1 = low1.AsInt32(); + result2 = high0.AsInt32(); + result3 = high1.AsInt32(); + } + + /// + /// Widens eight high-bit-depth samples into two 32-bit vectors. + /// + /// The packed samples. + /// The first four widened samples. + /// The second four widened samples. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void Widen(Vector128 source, out Vector128 result0, out Vector128 result1) + => (result0, result1) = Vector128.Widen(source); + + /// + /// Widens sixteen high-bit-depth samples into two 32-bit vectors. + /// + /// The packed samples. + /// The first eight widened samples. + /// The second eight widened samples. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void Widen(Vector256 source, out Vector256 result0, out Vector256 result1) + => (result0, result1) = Vector256.Widen(source); + + /// + /// Widens thirty-two high-bit-depth samples into two 32-bit vectors. + /// + /// The packed samples. + /// The first sixteen widened samples. + /// The second sixteen widened samples. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void Widen(Vector512 source, out Vector512 result0, out Vector512 result1) + => (result0, result1) = Vector512.Widen(source); + + /// + /// Narrows four 32-bit vectors into sixteen 8-bit samples. + /// + /// The first four samples. + /// The second four samples. + /// The third four samples. + /// The fourth four samples. + /// The packed samples. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector128 Narrow(Vector128 source0, Vector128 source1, Vector128 source2, Vector128 source3) + => Vector128.Narrow(Vector128.Narrow(source0.AsUInt32(), source1.AsUInt32()), Vector128.Narrow(source2.AsUInt32(), source3.AsUInt32())); + + /// + /// Narrows four 32-bit vectors into thirty-two 8-bit samples. + /// + /// The first eight samples. + /// The second eight samples. + /// The third eight samples. + /// The fourth eight samples. + /// The packed samples. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector256 Narrow(Vector256 source0, Vector256 source1, Vector256 source2, Vector256 source3) + => Vector256.Narrow(Vector256.Narrow(source0.AsUInt32(), source1.AsUInt32()), Vector256.Narrow(source2.AsUInt32(), source3.AsUInt32())); + + /// + /// Narrows four 32-bit vectors into sixty-four 8-bit samples. + /// + /// The first sixteen samples. + /// The second sixteen samples. + /// The third sixteen samples. + /// The fourth sixteen samples. + /// The packed samples. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector512 Narrow(Vector512 source0, Vector512 source1, Vector512 source2, Vector512 source3) + => Vector512.Narrow(Vector512.Narrow(source0.AsUInt32(), source1.AsUInt32()), Vector512.Narrow(source2.AsUInt32(), source3.AsUInt32())); + + /// + /// Narrows two 32-bit vectors into eight high-bit-depth samples. + /// + /// The first four samples. + /// The second four samples. + /// The packed samples. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector128 Narrow(Vector128 source0, Vector128 source1) => Vector128.Narrow(source0, source1); + + /// + /// Narrows two 32-bit vectors into sixteen high-bit-depth samples. + /// + /// The first eight samples. + /// The second eight samples. + /// The packed samples. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector256 Narrow(Vector256 source0, Vector256 source1) => Vector256.Narrow(source0, source1); + + /// + /// Narrows two 32-bit vectors into thirty-two high-bit-depth samples. + /// + /// The first sixteen samples. + /// The second sixteen samples. + /// The packed samples. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector512 Narrow(Vector512 source0, Vector512 source1) => Vector512.Narrow(source0, source1); +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1NonDirectionalIntraPredictor.PaethOperator.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1NonDirectionalIntraPredictor.PaethOperator.cs new file mode 100644 index 0000000000..b6f9b13cd3 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1NonDirectionalIntraPredictor.PaethOperator.cs @@ -0,0 +1,212 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; + +/// +/// Provides Paeth intra prediction for scalar and SIMD sample representations. +/// +internal abstract partial class Av1NonDirectionalIntraPredictorBase +{ + /// + /// Implements AV1 Paeth intra prediction for scalar and SIMD lanes. + /// + internal readonly struct PaethOperator : IAv1IntraPredictionOperator + { + /// + public static Av1PredictionMode Mode => Av1PredictionMode.Paeth; + + /// + public static Av1IntraPredictionInputs Inputs + => Av1IntraPredictionInputs.Top | Av1IntraPredictionInputs.Left | Av1IntraPredictionInputs.TopLeft; + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static byte Predict(byte top, byte left, byte topLeft, byte topRight, byte bottomLeft, int columnWeight, int rowWeight) + { + int basis = top + left - topLeft; + int distanceLeft = Math.Abs(basis - left); + int distanceTop = Math.Abs(basis - top); + int distanceTopLeft = Math.Abs(basis - topLeft); + + // AV1 resolves equal distances in left, top, top-left order. + return distanceLeft <= distanceTop && distanceLeft <= distanceTopLeft ? left : distanceTop <= distanceTopLeft ? top : topLeft; + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector128 Predict(Vector128 top, Vector128 left, Vector128 topLeft, Vector128 topRight, Vector128 bottomLeft, ref int columnWeights, int rowWeight) + => PredictPaeth(top, left, topLeft); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector256 Predict(Vector256 top, Vector256 left, Vector256 topLeft, Vector256 topRight, Vector256 bottomLeft, ref int columnWeights, int rowWeight) + => PredictPaeth(top, left, topLeft); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector512 Predict(Vector512 top, Vector512 left, Vector512 topLeft, Vector512 topRight, Vector512 bottomLeft, ref int columnWeights, int rowWeight) + => PredictPaeth(top, left, topLeft); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static short Predict(short top, short left, short topLeft, short topRight, short bottomLeft, int columnWeight, int rowWeight) + { + int basis = top + left - topLeft; + int distanceLeft = Math.Abs(basis - left); + int distanceTop = Math.Abs(basis - top); + int distanceTopLeft = Math.Abs(basis - topLeft); + + // High-bit-depth prediction has the same tie order as the 8-bit process. + return distanceLeft <= distanceTop && distanceLeft <= distanceTopLeft ? left : distanceTop <= distanceTopLeft ? top : topLeft; + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector128 Predict(Vector128 top, Vector128 left, Vector128 topLeft, Vector128 topRight, Vector128 bottomLeft, ref int columnWeights, int rowWeight) + => PredictPaeth(top, left, topLeft); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector256 Predict(Vector256 top, Vector256 left, Vector256 topLeft, Vector256 topRight, Vector256 bottomLeft, ref int columnWeights, int rowWeight) + => PredictPaeth(top, left, topLeft); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector512 Predict(Vector512 top, Vector512 left, Vector512 topLeft, Vector512 topRight, Vector512 bottomLeft, ref int columnWeights, int rowWeight) + => PredictPaeth(top, left, topLeft); + + /// + /// Selects the nearest Paeth reference for sixteen 8-bit lanes. + /// + /// The top candidates. + /// The left candidates. + /// The top-left candidates. + /// The selected candidates. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector128 PredictPaeth(Vector128 top, Vector128 left, Vector128 topLeft) + { + // The Paeth distances to left and top simplify to |top - topLeft| and |left - topLeft|. The established + // PNG predictor uses the same byte-lane identity, avoiding four widening stages for every sixteen samples. + Vector128 topMinusTopLeft = Vector128.SubtractSaturate(top, topLeft); + Vector128 leftMinusTopLeft = Vector128.SubtractSaturate(left, topLeft); + Vector128 distanceLeft = Vector128.SubtractSaturate(topLeft, top) | topMinusTopLeft; + Vector128 distanceTop = Vector128.SubtractSaturate(topLeft, left) | leftMinusTopLeft; + Vector128 sameDirection = Vector128.Equals(Vector128.Equals(topMinusTopLeft, Vector128.Zero), Vector128.Equals(leftMinusTopLeft, Vector128.Zero)); + Vector128 distanceTopLeft = sameDirection | Vector128.SubtractSaturate(distanceTop, distanceLeft) | Vector128.SubtractSaturate(distanceLeft, distanceTop); + Vector128 minimumTopTopLeft = Vector128.Min(distanceTopLeft, distanceTop); + Vector128 topOrTopLeft = Vector128.ConditionalSelect(Vector128.Equals(minimumTopTopLeft, distanceTop), top, topLeft); + return Vector128.ConditionalSelect(Vector128.Equals(Vector128.Min(minimumTopTopLeft, distanceLeft), distanceLeft), left, topOrTopLeft); + } + + /// + /// Selects the nearest Paeth reference for thirty-two 8-bit lanes. + /// + /// The top candidates. + /// The left candidates. + /// The top-left candidates. + /// The selected candidates. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector256 PredictPaeth(Vector256 top, Vector256 left, Vector256 topLeft) + { + Vector256 topMinusTopLeft = Vector256.SubtractSaturate(top, topLeft); + Vector256 leftMinusTopLeft = Vector256.SubtractSaturate(left, topLeft); + Vector256 distanceLeft = Vector256.SubtractSaturate(topLeft, top) | topMinusTopLeft; + Vector256 distanceTop = Vector256.SubtractSaturate(topLeft, left) | leftMinusTopLeft; + Vector256 sameDirection = Vector256.Equals(Vector256.Equals(topMinusTopLeft, Vector256.Zero), Vector256.Equals(leftMinusTopLeft, Vector256.Zero)); + Vector256 distanceTopLeft = sameDirection | Vector256.SubtractSaturate(distanceTop, distanceLeft) | Vector256.SubtractSaturate(distanceLeft, distanceTop); + Vector256 minimumTopTopLeft = Vector256.Min(distanceTopLeft, distanceTop); + Vector256 topOrTopLeft = Vector256.ConditionalSelect(Vector256.Equals(minimumTopTopLeft, distanceTop), top, topLeft); + return Vector256.ConditionalSelect(Vector256.Equals(Vector256.Min(minimumTopTopLeft, distanceLeft), distanceLeft), left, topOrTopLeft); + } + + /// + /// Selects the nearest Paeth reference for sixty-four 8-bit lanes. + /// + /// The top candidates. + /// The left candidates. + /// The top-left candidates. + /// The selected candidates. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector512 PredictPaeth(Vector512 top, Vector512 left, Vector512 topLeft) + { + Vector512 topMinusTopLeft = Vector512.SubtractSaturate(top, topLeft); + Vector512 leftMinusTopLeft = Vector512.SubtractSaturate(left, topLeft); + Vector512 distanceLeft = Vector512.SubtractSaturate(topLeft, top) | topMinusTopLeft; + Vector512 distanceTop = Vector512.SubtractSaturate(topLeft, left) | leftMinusTopLeft; + Vector512 sameDirection = Vector512.Equals(Vector512.Equals(topMinusTopLeft, Vector512.Zero), Vector512.Equals(leftMinusTopLeft, Vector512.Zero)); + Vector512 distanceTopLeft = sameDirection | Vector512.SubtractSaturate(distanceTop, distanceLeft) | Vector512.SubtractSaturate(distanceLeft, distanceTop); + Vector512 minimumTopTopLeft = Vector512.Min(distanceTopLeft, distanceTop); + Vector512 topOrTopLeft = Vector512.ConditionalSelect(Vector512.Equals(minimumTopTopLeft, distanceTop), top, topLeft); + return Vector512.ConditionalSelect(Vector512.Equals(Vector512.Min(minimumTopTopLeft, distanceLeft), distanceLeft), left, topOrTopLeft); + } + + /// + /// Selects the nearest Paeth reference for eight high-bit-depth lanes. + /// + /// The top candidates. + /// The left candidates. + /// The top-left candidates. + /// The selected candidates. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector128 PredictPaeth(Vector128 top, Vector128 left, Vector128 topLeft) + { + // AV1 samples are at most twelve bits. Both signed differences and their sum are therefore bounded by + // 8190, allowing the exact Paeth distances to stay in signed 16-bit lanes without widening. + Vector128 topDelta = top - topLeft; + Vector128 leftDelta = left - topLeft; + Vector128 distanceLeft = Vector128.Abs(topDelta); + Vector128 distanceTop = Vector128.Abs(leftDelta); + Vector128 distanceTopLeft = Vector128.Abs(topDelta + leftDelta); + + Vector128 minimumTopTopLeft = Vector128.Min(distanceTopLeft, distanceTop); + Vector128 topOrTopLeft = Vector128.ConditionalSelect(Vector128.Equals(minimumTopTopLeft, distanceTop), top, topLeft); + return Vector128.ConditionalSelect(Vector128.Equals(Vector128.Min(minimumTopTopLeft, distanceLeft), distanceLeft), left, topOrTopLeft); + } + + /// + /// Selects the nearest Paeth reference for sixteen high-bit-depth lanes. + /// + /// The top candidates. + /// The left candidates. + /// The top-left candidates. + /// The selected candidates. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector256 PredictPaeth(Vector256 top, Vector256 left, Vector256 topLeft) + { + Vector256 topDelta = top - topLeft; + Vector256 leftDelta = left - topLeft; + Vector256 distanceLeft = Vector256.Abs(topDelta); + Vector256 distanceTop = Vector256.Abs(leftDelta); + Vector256 distanceTopLeft = Vector256.Abs(topDelta + leftDelta); + + Vector256 minimumTopTopLeft = Vector256.Min(distanceTopLeft, distanceTop); + Vector256 topOrTopLeft = Vector256.ConditionalSelect(Vector256.Equals(minimumTopTopLeft, distanceTop), top, topLeft); + return Vector256.ConditionalSelect(Vector256.Equals(Vector256.Min(minimumTopTopLeft, distanceLeft), distanceLeft), left, topOrTopLeft); + } + + /// + /// Selects the nearest Paeth reference for thirty-two high-bit-depth lanes. + /// + /// The top candidates. + /// The left candidates. + /// The top-left candidates. + /// The selected candidates. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector512 PredictPaeth(Vector512 top, Vector512 left, Vector512 topLeft) + { + Vector512 topDelta = top - topLeft; + Vector512 leftDelta = left - topLeft; + Vector512 distanceLeft = Vector512.Abs(topDelta); + Vector512 distanceTop = Vector512.Abs(leftDelta); + Vector512 distanceTopLeft = Vector512.Abs(topDelta + leftDelta); + + Vector512 minimumTopTopLeft = Vector512.Min(distanceTopLeft, distanceTop); + Vector512 topOrTopLeft = Vector512.ConditionalSelect(Vector512.Equals(minimumTopTopLeft, distanceTop), top, topLeft); + return Vector512.ConditionalSelect(Vector512.Equals(Vector512.Min(minimumTopTopLeft, distanceLeft), distanceLeft), left, topOrTopLeft); + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1NonDirectionalIntraPredictor.SmoothHorizontalOperator.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1NonDirectionalIntraPredictor.SmoothHorizontalOperator.cs new file mode 100644 index 0000000000..01122bf6ad --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1NonDirectionalIntraPredictor.SmoothHorizontalOperator.cs @@ -0,0 +1,169 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; + +/// +/// Provides horizontal smooth intra prediction for scalar and SIMD sample representations. +/// +internal abstract partial class Av1NonDirectionalIntraPredictorBase +{ + /// + /// Implements horizontal AV1 smooth intra prediction for scalar and SIMD lanes. + /// + /// + /// Each lane uses its Q8 column weight to interpolate between the current row's left sample and the top-right + /// endpoint. Rewriting the complementary weight around 256 leaves one multiply per lane plus a shared endpoint and + /// rounding bias. + /// + internal readonly struct SmoothHorizontalOperator : IAv1IntraPredictionOperator + { + /// + /// The Q8 scale used by the smooth surface. + /// + private const int WeightScale = 256; + + /// + public static Av1PredictionMode Mode => Av1PredictionMode.SmoothHorizontal; + + /// + public static Av1IntraPredictionInputs Inputs + => Av1IntraPredictionInputs.Left | Av1IntraPredictionInputs.TopRight | Av1IntraPredictionInputs.ColumnWeight; + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static byte Predict(byte top, byte left, byte topLeft, byte topRight, byte bottomLeft, int columnWeight, int rowWeight) + => (byte)(((left * columnWeight) + (topRight * (WeightScale - columnWeight)) + (WeightScale >> 1)) >> 8); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector128 Predict(Vector128 top, Vector128 left, Vector128 topLeft, Vector128 topRight, Vector128 bottomLeft, ref int columnWeights, int rowWeight) + { + int leftSample = left.GetElement(0); + int topRightSample = topRight.GetElement(0); + int horizontalDelta = leftSample - topRightSample; + Vector128 endpointBias = Vector128.Create((topRightSample * WeightScale) + (WeightScale >> 1)); + + return Narrow( + Calculate(Vector128.LoadUnsafe(ref columnWeights), horizontalDelta, endpointBias), + Calculate(Vector128.LoadUnsafe(ref columnWeights, (nuint)Vector128.Count), horizontalDelta, endpointBias), + Calculate(Vector128.LoadUnsafe(ref columnWeights, (nuint)(2 * Vector128.Count)), horizontalDelta, endpointBias), + Calculate(Vector128.LoadUnsafe(ref columnWeights, (nuint)(3 * Vector128.Count)), horizontalDelta, endpointBias)); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector256 Predict(Vector256 top, Vector256 left, Vector256 topLeft, Vector256 topRight, Vector256 bottomLeft, ref int columnWeights, int rowWeight) + { + int leftSample = left.GetElement(0); + int topRightSample = topRight.GetElement(0); + int horizontalDelta = leftSample - topRightSample; + Vector256 endpointBias = Vector256.Create((topRightSample * WeightScale) + (WeightScale >> 1)); + + return Narrow( + Calculate(Vector256.LoadUnsafe(ref columnWeights), horizontalDelta, endpointBias), + Calculate(Vector256.LoadUnsafe(ref columnWeights, (nuint)Vector256.Count), horizontalDelta, endpointBias), + Calculate(Vector256.LoadUnsafe(ref columnWeights, (nuint)(2 * Vector256.Count)), horizontalDelta, endpointBias), + Calculate(Vector256.LoadUnsafe(ref columnWeights, (nuint)(3 * Vector256.Count)), horizontalDelta, endpointBias)); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector512 Predict(Vector512 top, Vector512 left, Vector512 topLeft, Vector512 topRight, Vector512 bottomLeft, ref int columnWeights, int rowWeight) + { + int leftSample = left.GetElement(0); + int topRightSample = topRight.GetElement(0); + int horizontalDelta = leftSample - topRightSample; + Vector512 endpointBias = Vector512.Create((topRightSample * WeightScale) + (WeightScale >> 1)); + + return Narrow( + Calculate(Vector512.LoadUnsafe(ref columnWeights), horizontalDelta, endpointBias), + Calculate(Vector512.LoadUnsafe(ref columnWeights, (nuint)Vector512.Count), horizontalDelta, endpointBias), + Calculate(Vector512.LoadUnsafe(ref columnWeights, (nuint)(2 * Vector512.Count)), horizontalDelta, endpointBias), + Calculate(Vector512.LoadUnsafe(ref columnWeights, (nuint)(3 * Vector512.Count)), horizontalDelta, endpointBias)); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static short Predict(short top, short left, short topLeft, short topRight, short bottomLeft, int columnWeight, int rowWeight) + => (short)(((left * columnWeight) + (topRight * (WeightScale - columnWeight)) + (WeightScale >> 1)) >> 8); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector128 Predict(Vector128 top, Vector128 left, Vector128 topLeft, Vector128 topRight, Vector128 bottomLeft, ref int columnWeights, int rowWeight) + { + int leftSample = left.GetElement(0); + int topRightSample = topRight.GetElement(0); + int horizontalDelta = leftSample - topRightSample; + Vector128 endpointBias = Vector128.Create((topRightSample * WeightScale) + (WeightScale >> 1)); + + return Narrow( + Calculate(Vector128.LoadUnsafe(ref columnWeights), horizontalDelta, endpointBias), + Calculate(Vector128.LoadUnsafe(ref columnWeights, (nuint)Vector128.Count), horizontalDelta, endpointBias)); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector256 Predict(Vector256 top, Vector256 left, Vector256 topLeft, Vector256 topRight, Vector256 bottomLeft, ref int columnWeights, int rowWeight) + { + int leftSample = left.GetElement(0); + int topRightSample = topRight.GetElement(0); + int horizontalDelta = leftSample - topRightSample; + Vector256 endpointBias = Vector256.Create((topRightSample * WeightScale) + (WeightScale >> 1)); + + return Narrow( + Calculate(Vector256.LoadUnsafe(ref columnWeights), horizontalDelta, endpointBias), + Calculate(Vector256.LoadUnsafe(ref columnWeights, (nuint)Vector256.Count), horizontalDelta, endpointBias)); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector512 Predict(Vector512 top, Vector512 left, Vector512 topLeft, Vector512 topRight, Vector512 bottomLeft, ref int columnWeights, int rowWeight) + { + int leftSample = left.GetElement(0); + int topRightSample = topRight.GetElement(0); + int horizontalDelta = leftSample - topRightSample; + Vector512 endpointBias = Vector512.Create((topRightSample * WeightScale) + (WeightScale >> 1)); + + return Narrow( + Calculate(Vector512.LoadUnsafe(ref columnWeights), horizontalDelta, endpointBias), + Calculate(Vector512.LoadUnsafe(ref columnWeights, (nuint)Vector512.Count), horizontalDelta, endpointBias)); + } + + /// + /// Calculates four horizontal smooth predictions. + /// + /// The horizontal Q8 weights. + /// The difference between the left sample and top-right endpoint. + /// The top-right endpoint and rounding bias in every lane. + /// The rounded predictions. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector128 Calculate(Vector128 columnWeights, int horizontalDelta, Vector128 endpointBias) + => ((columnWeights * horizontalDelta) + endpointBias) >> 8; + + /// + /// Calculates eight horizontal smooth predictions. + /// + /// The horizontal Q8 weights. + /// The difference between the left sample and top-right endpoint. + /// The top-right endpoint and rounding bias in every lane. + /// The rounded predictions. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector256 Calculate(Vector256 columnWeights, int horizontalDelta, Vector256 endpointBias) + => ((columnWeights * horizontalDelta) + endpointBias) >> 8; + + /// + /// Calculates sixteen horizontal smooth predictions. + /// + /// The horizontal Q8 weights. + /// The difference between the left sample and top-right endpoint. + /// The top-right endpoint and rounding bias in every lane. + /// The rounded predictions. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector512 Calculate(Vector512 columnWeights, int horizontalDelta, Vector512 endpointBias) + => ((columnWeights * horizontalDelta) + endpointBias) >> 8; + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1NonDirectionalIntraPredictor.SmoothOperator.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1NonDirectionalIntraPredictor.SmoothOperator.cs new file mode 100644 index 0000000000..00393f4806 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1NonDirectionalIntraPredictor.SmoothOperator.cs @@ -0,0 +1,249 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; + +/// +/// Provides two-dimensional smooth intra prediction for scalar and SIMD sample representations. +/// +internal abstract partial class Av1NonDirectionalIntraPredictorBase +{ + /// + /// Implements two-dimensional AV1 smooth intra prediction for scalar and SIMD lanes. + /// + /// + /// Horizontal and vertical Q8 interpolations are accumulated before one Q9 rounding shift. Expanding each + /// complementary weight around 256 reduces the lane equation to two products and a shared endpoint bias while + /// preserving the normative result exactly. + /// + internal readonly struct SmoothOperator : IAv1IntraPredictionOperator + { + /// + /// The Q8 scale used by each smooth surface. + /// + private const int WeightScale = 256; + + /// + public static Av1PredictionMode Mode => Av1PredictionMode.Smooth; + + /// + public static Av1IntraPredictionInputs Inputs + => Av1IntraPredictionInputs.Top + | Av1IntraPredictionInputs.Left + | Av1IntraPredictionInputs.TopRight + | Av1IntraPredictionInputs.BottomLeft + | Av1IntraPredictionInputs.ColumnWeight + | Av1IntraPredictionInputs.RowWeight; + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static byte Predict(byte top, byte left, byte topLeft, byte topRight, byte bottomLeft, int columnWeight, int rowWeight) + { + int prediction = (top * rowWeight) + (bottomLeft * (WeightScale - rowWeight)); + prediction += (left * columnWeight) + (topRight * (WeightScale - columnWeight)); + return (byte)((prediction + WeightScale) >> 9); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector128 Predict(Vector128 top, Vector128 left, Vector128 topLeft, Vector128 topRight, Vector128 bottomLeft, ref int columnWeights, int rowWeight) + { + Widen(top, out Vector128 top0, out Vector128 top1, out Vector128 top2, out Vector128 top3); + Vector128 weight0 = Vector128.LoadUnsafe(ref columnWeights); + Vector128 weight1 = Vector128.LoadUnsafe(ref columnWeights, (nuint)Vector128.Count); + Vector128 weight2 = Vector128.LoadUnsafe(ref columnWeights, (nuint)(2 * Vector128.Count)); + Vector128 weight3 = Vector128.LoadUnsafe(ref columnWeights, (nuint)(3 * Vector128.Count)); + int leftSample = left.GetElement(0); + int topRightSample = topRight.GetElement(0); + int bottomLeftSample = bottomLeft.GetElement(0); + int horizontalDelta = leftSample - topRightSample; + Vector128 bottomLeftVector = Vector128.Create(bottomLeftSample); + Vector128 endpointBias = Vector128.Create(((bottomLeftSample + topRightSample) * WeightScale) + WeightScale); + + return Narrow( + Calculate(top0, weight0, bottomLeftVector, horizontalDelta, endpointBias, rowWeight), + Calculate(top1, weight1, bottomLeftVector, horizontalDelta, endpointBias, rowWeight), + Calculate(top2, weight2, bottomLeftVector, horizontalDelta, endpointBias, rowWeight), + Calculate(top3, weight3, bottomLeftVector, horizontalDelta, endpointBias, rowWeight)); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector256 Predict(Vector256 top, Vector256 left, Vector256 topLeft, Vector256 topRight, Vector256 bottomLeft, ref int columnWeights, int rowWeight) + { + Widen(top, out Vector256 top0, out Vector256 top1, out Vector256 top2, out Vector256 top3); + Vector256 weight0 = Vector256.LoadUnsafe(ref columnWeights); + Vector256 weight1 = Vector256.LoadUnsafe(ref columnWeights, (nuint)Vector256.Count); + Vector256 weight2 = Vector256.LoadUnsafe(ref columnWeights, (nuint)(2 * Vector256.Count)); + Vector256 weight3 = Vector256.LoadUnsafe(ref columnWeights, (nuint)(3 * Vector256.Count)); + int leftSample = left.GetElement(0); + int topRightSample = topRight.GetElement(0); + int bottomLeftSample = bottomLeft.GetElement(0); + int horizontalDelta = leftSample - topRightSample; + Vector256 bottomLeftVector = Vector256.Create(bottomLeftSample); + Vector256 endpointBias = Vector256.Create(((bottomLeftSample + topRightSample) * WeightScale) + WeightScale); + + return Narrow( + Calculate(top0, weight0, bottomLeftVector, horizontalDelta, endpointBias, rowWeight), + Calculate(top1, weight1, bottomLeftVector, horizontalDelta, endpointBias, rowWeight), + Calculate(top2, weight2, bottomLeftVector, horizontalDelta, endpointBias, rowWeight), + Calculate(top3, weight3, bottomLeftVector, horizontalDelta, endpointBias, rowWeight)); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector512 Predict(Vector512 top, Vector512 left, Vector512 topLeft, Vector512 topRight, Vector512 bottomLeft, ref int columnWeights, int rowWeight) + { + Widen(top, out Vector512 top0, out Vector512 top1, out Vector512 top2, out Vector512 top3); + Vector512 weight0 = Vector512.LoadUnsafe(ref columnWeights); + Vector512 weight1 = Vector512.LoadUnsafe(ref columnWeights, (nuint)Vector512.Count); + Vector512 weight2 = Vector512.LoadUnsafe(ref columnWeights, (nuint)(2 * Vector512.Count)); + Vector512 weight3 = Vector512.LoadUnsafe(ref columnWeights, (nuint)(3 * Vector512.Count)); + int leftSample = left.GetElement(0); + int topRightSample = topRight.GetElement(0); + int bottomLeftSample = bottomLeft.GetElement(0); + int horizontalDelta = leftSample - topRightSample; + Vector512 bottomLeftVector = Vector512.Create(bottomLeftSample); + Vector512 endpointBias = Vector512.Create(((bottomLeftSample + topRightSample) * WeightScale) + WeightScale); + + return Narrow( + Calculate(top0, weight0, bottomLeftVector, horizontalDelta, endpointBias, rowWeight), + Calculate(top1, weight1, bottomLeftVector, horizontalDelta, endpointBias, rowWeight), + Calculate(top2, weight2, bottomLeftVector, horizontalDelta, endpointBias, rowWeight), + Calculate(top3, weight3, bottomLeftVector, horizontalDelta, endpointBias, rowWeight)); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static short Predict(short top, short left, short topLeft, short topRight, short bottomLeft, int columnWeight, int rowWeight) + { + int prediction = (top * rowWeight) + (bottomLeft * (WeightScale - rowWeight)); + prediction += (left * columnWeight) + (topRight * (WeightScale - columnWeight)); + return (short)((prediction + WeightScale) >> 9); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector128 Predict(Vector128 top, Vector128 left, Vector128 topLeft, Vector128 topRight, Vector128 bottomLeft, ref int columnWeights, int rowWeight) + { + Widen(top, out Vector128 top0, out Vector128 top1); + Vector128 weight0 = Vector128.LoadUnsafe(ref columnWeights); + Vector128 weight1 = Vector128.LoadUnsafe(ref columnWeights, (nuint)Vector128.Count); + int leftSample = left.GetElement(0); + int topRightSample = topRight.GetElement(0); + int bottomLeftSample = bottomLeft.GetElement(0); + int horizontalDelta = leftSample - topRightSample; + Vector128 bottomLeftVector = Vector128.Create(bottomLeftSample); + Vector128 endpointBias = Vector128.Create(((bottomLeftSample + topRightSample) * WeightScale) + WeightScale); + + return Narrow( + Calculate(top0, weight0, bottomLeftVector, horizontalDelta, endpointBias, rowWeight), + Calculate(top1, weight1, bottomLeftVector, horizontalDelta, endpointBias, rowWeight)); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector256 Predict(Vector256 top, Vector256 left, Vector256 topLeft, Vector256 topRight, Vector256 bottomLeft, ref int columnWeights, int rowWeight) + { + Widen(top, out Vector256 top0, out Vector256 top1); + Vector256 weight0 = Vector256.LoadUnsafe(ref columnWeights); + Vector256 weight1 = Vector256.LoadUnsafe(ref columnWeights, (nuint)Vector256.Count); + int leftSample = left.GetElement(0); + int topRightSample = topRight.GetElement(0); + int bottomLeftSample = bottomLeft.GetElement(0); + int horizontalDelta = leftSample - topRightSample; + Vector256 bottomLeftVector = Vector256.Create(bottomLeftSample); + Vector256 endpointBias = Vector256.Create(((bottomLeftSample + topRightSample) * WeightScale) + WeightScale); + + return Narrow( + Calculate(top0, weight0, bottomLeftVector, horizontalDelta, endpointBias, rowWeight), + Calculate(top1, weight1, bottomLeftVector, horizontalDelta, endpointBias, rowWeight)); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector512 Predict(Vector512 top, Vector512 left, Vector512 topLeft, Vector512 topRight, Vector512 bottomLeft, ref int columnWeights, int rowWeight) + { + Widen(top, out Vector512 top0, out Vector512 top1); + Vector512 weight0 = Vector512.LoadUnsafe(ref columnWeights); + Vector512 weight1 = Vector512.LoadUnsafe(ref columnWeights, (nuint)Vector512.Count); + int leftSample = left.GetElement(0); + int topRightSample = topRight.GetElement(0); + int bottomLeftSample = bottomLeft.GetElement(0); + int horizontalDelta = leftSample - topRightSample; + Vector512 bottomLeftVector = Vector512.Create(bottomLeftSample); + Vector512 endpointBias = Vector512.Create(((bottomLeftSample + topRightSample) * WeightScale) + WeightScale); + + return Narrow( + Calculate(top0, weight0, bottomLeftVector, horizontalDelta, endpointBias, rowWeight), + Calculate(top1, weight1, bottomLeftVector, horizontalDelta, endpointBias, rowWeight)); + } + + /// + /// Calculates four two-dimensional smooth predictions. + /// + /// The widened top samples. + /// The horizontal Q8 weights. + /// The bottom-left endpoint in every lane. + /// The difference between the left sample and top-right endpoint. + /// The combined endpoint and rounding bias in every lane. + /// The vertical Q8 weight. + /// The rounded predictions. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector128 Calculate( + Vector128 top, + Vector128 columnWeights, + Vector128 bottomLeft, + int horizontalDelta, + Vector128 endpointBias, + int rowWeight) + { + // Expanding the complementary weights gives the exact normative sum while reducing it to two vector + // multiplications: (top - bottom) * rowWeight + (left - right) * columnWeight + the endpoint bias. + return (((top - bottomLeft) * rowWeight) + (columnWeights * horizontalDelta) + endpointBias) >> 9; + } + + /// + /// Calculates eight two-dimensional smooth predictions. + /// + /// The widened top samples. + /// The horizontal Q8 weights. + /// The bottom-left endpoint in every lane. + /// The difference between the left sample and top-right endpoint. + /// The combined endpoint and rounding bias in every lane. + /// The vertical Q8 weight. + /// The rounded predictions. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector256 Calculate( + Vector256 top, + Vector256 columnWeights, + Vector256 bottomLeft, + int horizontalDelta, + Vector256 endpointBias, + int rowWeight) + => (((top - bottomLeft) * rowWeight) + (columnWeights * horizontalDelta) + endpointBias) >> 9; + + /// + /// Calculates sixteen two-dimensional smooth predictions. + /// + /// The widened top samples. + /// The horizontal Q8 weights. + /// The bottom-left endpoint in every lane. + /// The difference between the left sample and top-right endpoint. + /// The combined endpoint and rounding bias in every lane. + /// The vertical Q8 weight. + /// The rounded predictions. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector512 Calculate( + Vector512 top, + Vector512 columnWeights, + Vector512 bottomLeft, + int horizontalDelta, + Vector512 endpointBias, + int rowWeight) + => (((top - bottomLeft) * rowWeight) + (columnWeights * horizontalDelta) + endpointBias) >> 9; + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1NonDirectionalIntraPredictor.SmoothVerticalOperator.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1NonDirectionalIntraPredictor.SmoothVerticalOperator.cs new file mode 100644 index 0000000000..60099f0dca --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1NonDirectionalIntraPredictor.SmoothVerticalOperator.cs @@ -0,0 +1,171 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; + +/// +/// Provides vertical smooth intra prediction for scalar and SIMD sample representations. +/// +internal abstract partial class Av1NonDirectionalIntraPredictorBase +{ + /// + /// Implements vertical AV1 smooth intra prediction for scalar and SIMD lanes. + /// + /// + /// The row's Q8 weight is common to every lane, while top references vary by column. Rewriting the complementary + /// weight around 256 leaves one product per lane plus a broadcast bottom-left endpoint and rounding bias. + /// + internal readonly struct SmoothVerticalOperator : IAv1IntraPredictionOperator + { + /// + /// The Q8 scale used by the smooth surface. + /// + private const int WeightScale = 256; + + /// + public static Av1PredictionMode Mode => Av1PredictionMode.SmoothVertical; + + /// + public static Av1IntraPredictionInputs Inputs + => Av1IntraPredictionInputs.Top | Av1IntraPredictionInputs.BottomLeft | Av1IntraPredictionInputs.RowWeight; + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static byte Predict(byte top, byte left, byte topLeft, byte topRight, byte bottomLeft, int columnWeight, int rowWeight) + => (byte)(((top * rowWeight) + (bottomLeft * (WeightScale - rowWeight)) + (WeightScale >> 1)) >> 8); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector128 Predict(Vector128 top, Vector128 left, Vector128 topLeft, Vector128 topRight, Vector128 bottomLeft, ref int columnWeights, int rowWeight) + { + Widen(top, out Vector128 top0, out Vector128 top1, out Vector128 top2, out Vector128 top3); + int bottomLeftSample = bottomLeft.GetElement(0); + Vector128 bottomLeftVector = Vector128.Create(bottomLeftSample); + Vector128 endpointBias = Vector128.Create((bottomLeftSample * WeightScale) + (WeightScale >> 1)); + + return Narrow( + Calculate(top0, bottomLeftVector, endpointBias, rowWeight), + Calculate(top1, bottomLeftVector, endpointBias, rowWeight), + Calculate(top2, bottomLeftVector, endpointBias, rowWeight), + Calculate(top3, bottomLeftVector, endpointBias, rowWeight)); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector256 Predict(Vector256 top, Vector256 left, Vector256 topLeft, Vector256 topRight, Vector256 bottomLeft, ref int columnWeights, int rowWeight) + { + Widen(top, out Vector256 top0, out Vector256 top1, out Vector256 top2, out Vector256 top3); + int bottomLeftSample = bottomLeft.GetElement(0); + Vector256 bottomLeftVector = Vector256.Create(bottomLeftSample); + Vector256 endpointBias = Vector256.Create((bottomLeftSample * WeightScale) + (WeightScale >> 1)); + + return Narrow( + Calculate(top0, bottomLeftVector, endpointBias, rowWeight), + Calculate(top1, bottomLeftVector, endpointBias, rowWeight), + Calculate(top2, bottomLeftVector, endpointBias, rowWeight), + Calculate(top3, bottomLeftVector, endpointBias, rowWeight)); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector512 Predict(Vector512 top, Vector512 left, Vector512 topLeft, Vector512 topRight, Vector512 bottomLeft, ref int columnWeights, int rowWeight) + { + Widen(top, out Vector512 top0, out Vector512 top1, out Vector512 top2, out Vector512 top3); + int bottomLeftSample = bottomLeft.GetElement(0); + Vector512 bottomLeftVector = Vector512.Create(bottomLeftSample); + Vector512 endpointBias = Vector512.Create((bottomLeftSample * WeightScale) + (WeightScale >> 1)); + + return Narrow( + Calculate(top0, bottomLeftVector, endpointBias, rowWeight), + Calculate(top1, bottomLeftVector, endpointBias, rowWeight), + Calculate(top2, bottomLeftVector, endpointBias, rowWeight), + Calculate(top3, bottomLeftVector, endpointBias, rowWeight)); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static short Predict(short top, short left, short topLeft, short topRight, short bottomLeft, int columnWeight, int rowWeight) + => (short)(((top * rowWeight) + (bottomLeft * (WeightScale - rowWeight)) + (WeightScale >> 1)) >> 8); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector128 Predict(Vector128 top, Vector128 left, Vector128 topLeft, Vector128 topRight, Vector128 bottomLeft, ref int columnWeights, int rowWeight) + { + Widen(top, out Vector128 top0, out Vector128 top1); + int bottomLeftSample = bottomLeft.GetElement(0); + Vector128 bottomLeftVector = Vector128.Create(bottomLeftSample); + Vector128 endpointBias = Vector128.Create((bottomLeftSample * WeightScale) + (WeightScale >> 1)); + + return Narrow( + Calculate(top0, bottomLeftVector, endpointBias, rowWeight), + Calculate(top1, bottomLeftVector, endpointBias, rowWeight)); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector256 Predict(Vector256 top, Vector256 left, Vector256 topLeft, Vector256 topRight, Vector256 bottomLeft, ref int columnWeights, int rowWeight) + { + Widen(top, out Vector256 top0, out Vector256 top1); + int bottomLeftSample = bottomLeft.GetElement(0); + Vector256 bottomLeftVector = Vector256.Create(bottomLeftSample); + Vector256 endpointBias = Vector256.Create((bottomLeftSample * WeightScale) + (WeightScale >> 1)); + + return Narrow( + Calculate(top0, bottomLeftVector, endpointBias, rowWeight), + Calculate(top1, bottomLeftVector, endpointBias, rowWeight)); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector512 Predict(Vector512 top, Vector512 left, Vector512 topLeft, Vector512 topRight, Vector512 bottomLeft, ref int columnWeights, int rowWeight) + { + Widen(top, out Vector512 top0, out Vector512 top1); + int bottomLeftSample = bottomLeft.GetElement(0); + Vector512 bottomLeftVector = Vector512.Create(bottomLeftSample); + Vector512 endpointBias = Vector512.Create((bottomLeftSample * WeightScale) + (WeightScale >> 1)); + + return Narrow( + Calculate(top0, bottomLeftVector, endpointBias, rowWeight), + Calculate(top1, bottomLeftVector, endpointBias, rowWeight)); + } + + /// + /// Calculates four vertical smooth predictions. + /// + /// The widened top samples. + /// The bottom-left endpoint in every lane. + /// The bottom-left endpoint and rounding bias in every lane. + /// The vertical Q8 weight. + /// The rounded predictions. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector128 Calculate(Vector128 top, Vector128 bottomLeft, Vector128 endpointBias, int rowWeight) + => (((top - bottomLeft) * rowWeight) + endpointBias) >> 8; + + /// + /// Calculates eight vertical smooth predictions. + /// + /// The widened top samples. + /// The bottom-left endpoint in every lane. + /// The bottom-left endpoint and rounding bias in every lane. + /// The vertical Q8 weight. + /// The rounded predictions. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector256 Calculate(Vector256 top, Vector256 bottomLeft, Vector256 endpointBias, int rowWeight) + => (((top - bottomLeft) * rowWeight) + endpointBias) >> 8; + + /// + /// Calculates sixteen vertical smooth predictions. + /// + /// The widened top samples. + /// The bottom-left endpoint in every lane. + /// The bottom-left endpoint and rounding bias in every lane. + /// The vertical Q8 weight. + /// The rounded predictions. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector512 Calculate(Vector512 top, Vector512 bottomLeft, Vector512 endpointBias, int rowWeight) + => (((top - bottomLeft) * rowWeight) + endpointBias) >> 8; + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1NonDirectionalIntraPredictor.VerticalOperator.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1NonDirectionalIntraPredictor.VerticalOperator.cs new file mode 100644 index 0000000000..7c90f2f43d --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1NonDirectionalIntraPredictor.VerticalOperator.cs @@ -0,0 +1,57 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; + +/// +/// Provides vertical intra prediction for scalar and SIMD sample representations. +/// +internal abstract partial class Av1NonDirectionalIntraPredictorBase +{ + /// + /// Implements AV1 vertical intra prediction for scalar and SIMD lanes. + /// + internal readonly struct VerticalOperator : IAv1IntraPredictionOperator + { + /// + public static Av1PredictionMode Mode => Av1PredictionMode.Vertical; + + /// + public static Av1IntraPredictionInputs Inputs => Av1IntraPredictionInputs.Top; + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static byte Predict(byte top, byte left, byte topLeft, byte topRight, byte bottomLeft, int columnWeight, int rowWeight) => top; + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector128 Predict(Vector128 top, Vector128 left, Vector128 topLeft, Vector128 topRight, Vector128 bottomLeft, ref int columnWeights, int rowWeight) => top; + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector256 Predict(Vector256 top, Vector256 left, Vector256 topLeft, Vector256 topRight, Vector256 bottomLeft, ref int columnWeights, int rowWeight) => top; + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector512 Predict(Vector512 top, Vector512 left, Vector512 topLeft, Vector512 topRight, Vector512 bottomLeft, ref int columnWeights, int rowWeight) => top; + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static short Predict(short top, short left, short topLeft, short topRight, short bottomLeft, int columnWeight, int rowWeight) => top; + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector128 Predict(Vector128 top, Vector128 left, Vector128 topLeft, Vector128 topRight, Vector128 bottomLeft, ref int columnWeights, int rowWeight) => top; + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector256 Predict(Vector256 top, Vector256 left, Vector256 topLeft, Vector256 topRight, Vector256 bottomLeft, ref int columnWeights, int rowWeight) => top; + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector512 Predict(Vector512 top, Vector512 left, Vector512 topLeft, Vector512 topRight, Vector512 bottomLeft, ref int columnWeights, int rowWeight) => top; + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1NonDirectionalIntraPredictorBase.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1NonDirectionalIntraPredictorBase.cs new file mode 100644 index 0000000000..86f8b91868 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1NonDirectionalIntraPredictorBase.cs @@ -0,0 +1,131 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; + +/// +/// Reconstructs non-directional AV1 intra-prediction blocks from prepared neighboring samples. +/// +/// +/// The implementation covers the non-directional prediction processes in section 7.11.2 of the AV1 specification. +/// +internal abstract partial class Av1NonDirectionalIntraPredictorBase +{ + /// + /// The horizontal prediction operator. + /// + private static readonly Av1NonDirectionalIntraPredictor HorizontalPredictor = new(); + + /// + /// The vertical prediction operator. + /// + private static readonly Av1NonDirectionalIntraPredictor VerticalPredictor = new(); + + /// + /// The Paeth prediction operator. + /// + private static readonly Av1NonDirectionalIntraPredictor PaethPredictor = new(); + + /// + /// The two-dimensional smooth prediction operator. + /// + private static readonly Av1NonDirectionalIntraPredictor SmoothPredictor = new(); + + /// + /// The horizontal smooth prediction operator. + /// + private static readonly Av1NonDirectionalIntraPredictor SmoothHorizontalPredictor = new(); + + /// + /// The vertical smooth prediction operator. + /// + private static readonly Av1NonDirectionalIntraPredictor SmoothVerticalPredictor = new(); + + /// + /// Gets the Q8 smooth weights for every supported block dimension. + /// + private static ReadOnlySpan SmoothWeights => + [ + + // The first two entries are unused because the smallest AV1 prediction dimension is four samples. + 0, 0, + 255, 128, + 255, 149, 85, 64, + 255, 197, 146, 105, 73, 50, 37, 32, + 255, 225, 196, 170, 145, 123, 102, 84, 68, 54, 43, 33, 26, 20, 17, 16, + 255, 240, 225, 210, 196, 182, 169, 157, 145, 133, 122, 111, 101, 92, 83, 74, + 66, 59, 52, 45, 39, 34, 29, 25, 21, 17, 14, 12, 10, 9, 8, 8, + 255, 248, 240, 233, 225, 218, 210, 203, 196, 189, 182, 176, 169, 163, 156, + 150, 144, 138, 133, 127, 121, 116, 111, 106, 101, 96, 91, 86, 82, 77, 73, 69, + 65, 61, 57, 54, 50, 47, 44, 41, 38, 35, 32, 29, 27, 25, 22, 20, 18, 16, 15, + 13, 12, 10, 9, 8, 7, 6, 6, 5, 5, 4, 4, 4, + ]; + + /// + /// Gets the prediction mode implemented by this predictor. + /// + public abstract Av1PredictionMode Mode { get; } + + /// + /// Gets the closed predictor for a non-directional AV1 prediction mode. + /// + /// The decoded non-directional prediction mode. + /// The predictor for . + public static Av1NonDirectionalIntraPredictorBase GetPredictor(Av1PredictionMode mode) + => mode switch + { + Av1PredictionMode.Horizontal => HorizontalPredictor, + Av1PredictionMode.Vertical => VerticalPredictor, + Av1PredictionMode.Paeth => PaethPredictor, + Av1PredictionMode.Smooth => SmoothPredictor, + Av1PredictionMode.SmoothHorizontal => SmoothHorizontalPredictor, + Av1PredictionMode.SmoothVertical => SmoothVerticalPredictor, + _ => throw new InvalidImageContentException($"Prediction mode {mode} is not a non-directional AV1 mode."), + }; + + /// + /// Predicts an 8-bit block. + /// + /// The destination block origin. + /// The destination row stride in samples. + /// The prepared top reference, preceded in memory by the top-left sample. + /// The prepared left reference. + /// The block width in samples. + /// The block height in samples. + public abstract void Predict(Span destination, int destinationStride, ReadOnlySpan above, ReadOnlySpan left, int width, int height); + + /// + /// Predicts a high-bit-depth block. + /// + /// The destination block origin. + /// The destination row stride in samples. + /// The prepared top reference, preceded in memory by the top-left sample. + /// The prepared left reference. + /// The block width in samples. + /// The block height in samples. + public abstract void Predict(Span destination, int destinationStride, ReadOnlySpan above, ReadOnlySpan left, int width, int height); + + /// + /// Predicts an 8-bit block without hardware intrinsics. + /// + /// The destination block origin. + /// The destination row stride in samples. + /// The prepared top reference, preceded in memory by the top-left sample. + /// The prepared left reference. + /// The block width in samples. + /// The block height in samples. + public abstract void PredictScalar(Span destination, int destinationStride, ReadOnlySpan above, ReadOnlySpan left, int width, int height); + + /// + /// Predicts a high-bit-depth block without hardware intrinsics. + /// + /// The destination block origin. + /// The destination row stride in samples. + /// The prepared top reference, preceded in memory by the top-left sample. + /// The prepared left reference. + /// The block width in samples. + /// The block height in samples. + public abstract void PredictScalar(Span destination, int destinationStride, ReadOnlySpan above, ReadOnlySpan left, int width, int height); +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1PalettePredictor.Operator.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1PalettePredictor.Operator.cs new file mode 100644 index 0000000000..9317d8ee3e --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1PalettePredictor.Operator.cs @@ -0,0 +1,371 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.InteropServices; +using System.Runtime.Intrinsics; +using SixLabors.ImageSharp.Memory; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; + +/// +/// Defines the closed scalar/SIMD operator contract and traversal for AV1 palette prediction. +/// +internal static class Av1PalettePredictor +{ + /// + /// Multiplies a palette index by two to select the low byte of a high-bit-depth entry. + /// + private const ushort PaletteByteOffsetMultiplier = 0x0202; + + /// + /// Adds one to each odd control byte so each shuffled high-bit-depth sample retains both bytes. + /// + private const ushort PaletteHighByteOffset = 0x0100; + + /// + /// Defines scalar and SIMD palette-index lookup. + /// + private interface IPaletteOperator + { + /// + /// Predicts one 8-bit sample. + /// + /// The first palette entry. + /// The palette index. + /// The selected sample. + public static abstract byte Predict(ref byte palette, byte index); + + /// + /// Predicts sixteen 8-bit samples. + /// + /// The palette entries repeated in each 128-bit lane. + /// The palette indices. + /// The selected samples. + public static abstract Vector128 Predict(Vector128 palette, Vector128 indices); + + /// + /// Predicts thirty-two 8-bit samples. + /// + /// The palette entries repeated in each 128-bit lane. + /// The palette indices. + /// The selected samples. + public static abstract Vector256 Predict(Vector256 palette, Vector256 indices); + + /// + /// Predicts sixty-four 8-bit samples. + /// + /// The palette entries repeated in each 128-bit lane. + /// The palette indices. + /// The selected samples. + public static abstract Vector512 Predict(Vector512 palette, Vector512 indices); + + /// + /// Predicts one high-bit-depth sample. + /// + /// The first palette entry. + /// The palette index. + /// The selected sample. + public static abstract short Predict(ref ushort palette, byte index); + + /// + /// Predicts eight high-bit-depth samples. + /// + /// The palette bytes repeated in each 128-bit lane. + /// The palette indices. + /// The selected samples. + public static abstract Vector128 Predict(Vector128 palette, Vector128 indices); + + /// + /// Predicts sixteen high-bit-depth samples. + /// + /// The palette bytes repeated in each 128-bit lane. + /// The palette indices. + /// The selected samples. + public static abstract Vector256 Predict(Vector256 palette, Vector256 indices); + + /// + /// Predicts thirty-two high-bit-depth samples. + /// + /// The palette bytes repeated in each 128-bit lane. + /// The palette indices. + /// The selected samples. + public static abstract Vector512 Predict(Vector512 palette, Vector512 indices); + } + + /// + /// Reconstructs an 8-bit palette-predicted block. + /// + public static void Predict( + ReadOnlySpan paletteColors, + Buffer2DRegion colorIndexMap, + Span destination, + int destinationStride, + int width, + int height) + => Predictor.Predict(paletteColors, colorIndexMap, destination, destinationStride, width, height); + + /// + /// Reconstructs a high-bit-depth palette-predicted block. + /// + public static void Predict( + ReadOnlySpan paletteColors, + Buffer2DRegion colorIndexMap, + Span destination, + int destinationStride, + int width, + int height) + => Predictor.Predict(paletteColors, colorIndexMap, destination, destinationStride, width, height); + + /// + /// Maps decoded palette indices to reconstructed samples. + /// + private readonly struct PaletteOperator : IPaletteOperator + { + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static byte Predict(ref byte palette, byte index) => Unsafe.Add(ref palette, index); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector128 Predict(Vector128 palette, Vector128 indices) + => Vector128.ShuffleNative(palette, indices); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector256 Predict(Vector256 palette, Vector256 indices) + => Vector256.ShuffleNative(palette, indices); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector512 Predict(Vector512 palette, Vector512 indices) + => Vector512.ShuffleNative(palette, indices); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static short Predict(ref ushort palette, byte index) => (short)Unsafe.Add(ref palette, index); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector128 Predict(Vector128 palette, Vector128 indices) + { + Vector128 controls = (indices * Vector128.Create(PaletteByteOffsetMultiplier)) + Vector128.Create(PaletteHighByteOffset); + + return Vector128.ShuffleNative(palette, controls.AsByte()).AsInt16(); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector256 Predict(Vector256 palette, Vector256 indices) + { + Vector256 controls = (indices * Vector256.Create(PaletteByteOffsetMultiplier)) + Vector256.Create(PaletteHighByteOffset); + + return Vector256.ShuffleNative(palette, controls.AsByte()).AsInt16(); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector512 Predict(Vector512 palette, Vector512 indices) + { + Vector512 controls = (indices * Vector512.Create(PaletteByteOffsetMultiplier)) + Vector512.Create(PaletteHighByteOffset); + + return Vector512.ShuffleNative(palette, controls.AsByte()).AsInt16(); + } + } + + /// + /// Traverses palette blocks through one closed lookup operator. + /// + /// The palette lookup arithmetic. + private static class Predictor + where TOperator : struct, IPaletteOperator + { + /// + /// Reconstructs an 8-bit palette block. + /// + public static void Predict( + ReadOnlySpan paletteColors, + Buffer2DRegion colorIndexMap, + Span destination, + int destinationStride, + int width, + int height) + { + ref byte destinationBase = ref MemoryMarshal.GetReference(destination); + + // AV1 palettes contain at most eight colors. Repeating all eight entries in every 128-bit lane keeps native + // table lookup lane-local at every SIMD width and removes palette bounds work from the reconstruction loop. + ulong packedPalette = 0; + for (int index = 0; index < paletteColors.Length; index++) + { + packedPalette |= (ulong)(byte)paletteColors[index] << (index * 8); + } + + ref byte paletteBase = ref Unsafe.As(ref packedPalette); + Vector128 palette128 = Vector128.Create(packedPalette, packedPalette).AsByte(); + + for (int row = 0; row < height; row++) + { + ref byte mapRow = ref MemoryMarshal.GetReference(colorIndexMap.DangerousGetRowSpan(row)); + ref byte destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + int column = 0; + + if (Vector512.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector512Count(width - column); + + if (vectorCount > 0) + { + // Replicate the lookup table only when the row has a complete 64-lane batch. + Vector256 palette256 = Vector256.Create(palette128, palette128); + Vector512 palette512 = Vector512.Create(palette256, palette256); + + for (; vectorCount > 0; vectorCount--, column += Vector512.Count) + { + Vector512 indices = Vector512.LoadUnsafe(ref mapRow, (nuint)column); + TOperator.Predict(palette512, indices).StoreUnsafe(ref destinationRow, (nuint)column); + } + } + } + + if (Vector256.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector256Count(width - column); + + if (vectorCount > 0) + { + // The narrower table is likewise materialized only for a complete 32-lane remainder. + Vector256 palette256 = Vector256.Create(palette128, palette128); + + for (; vectorCount > 0; vectorCount--, column += Vector256.Count) + { + Vector256 indices = Vector256.LoadUnsafe(ref mapRow, (nuint)column); + TOperator.Predict(palette256, indices).StoreUnsafe(ref destinationRow, (nuint)column); + } + } + } + + if (Vector128.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector128Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector128.Count) + { + Vector128 indices = Vector128.LoadUnsafe(ref mapRow, (nuint)column); + TOperator.Predict(palette128, indices).StoreUnsafe(ref destinationRow, (nuint)column); + } + + int remaining = width - column; + if (remaining >= 8) + { + ulong packedIndices = Unsafe.ReadUnaligned(ref Unsafe.Add(ref mapRow, column)); + Vector128 prediction = TOperator.Predict(palette128, Vector128.CreateScalarUnsafe(packedIndices).AsByte()); + Unsafe.WriteUnaligned(ref Unsafe.Add(ref destinationRow, column), prediction.AsUInt64().ToScalar()); + column += 8; + remaining -= 8; + } + + if (remaining >= 4) + { + uint packedIndices = Unsafe.ReadUnaligned(ref Unsafe.Add(ref mapRow, column)); + Vector128 prediction = TOperator.Predict(palette128, Vector128.CreateScalarUnsafe(packedIndices).AsByte()); + Unsafe.WriteUnaligned(ref Unsafe.Add(ref destinationRow, column), prediction.AsUInt32().ToScalar()); + column += 4; + } + } + + for (; column < width; column++) + { + Unsafe.Add(ref destinationRow, column) = TOperator.Predict(ref paletteBase, Unsafe.Add(ref mapRow, column)); + } + } + } + + /// + /// Reconstructs a high-bit-depth palette block. + /// + public static void Predict( + ReadOnlySpan paletteColors, + Buffer2DRegion colorIndexMap, + Span destination, + int destinationStride, + int width, + int height) + { + ref short destinationBase = ref MemoryMarshal.GetReference(destination); + InlineArray8 paletteStorage = default; + paletteColors.CopyTo(paletteStorage); + + ref ushort paletteBase = ref paletteStorage[0]; + Vector128 palette128 = Vector128.LoadUnsafe(ref paletteBase).AsByte(); + + for (int row = 0; row < height; row++) + { + ref byte mapRow = ref MemoryMarshal.GetReference(colorIndexMap.DangerousGetRowSpan(row)); + ref short destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + int column = 0; + + if (Vector512.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector512Count(width - column); + + if (vectorCount > 0) + { + // High-bit-depth output has half as many lanes, so gate table replication with that lane count. + Vector256 palette256 = Vector256.Create(palette128, palette128); + Vector512 palette512 = Vector512.Create(palette256, palette256); + + for (; vectorCount > 0; vectorCount--, column += Vector512.Count) + { + (Vector256 lower, Vector256 upper) = Vector256.Widen(Vector256.LoadUnsafe(ref mapRow, (nuint)column)); + Vector512 indices = Vector512.Create(lower, upper); + TOperator.Predict(palette512, indices).StoreUnsafe(ref destinationRow, (nuint)column); + } + } + } + + if (Vector256.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector256Count(width - column); + + if (vectorCount > 0) + { + // Avoid creating the 256-bit table when the remainder belongs entirely to narrower paths. + Vector256 palette256 = Vector256.Create(palette128, palette128); + + for (; vectorCount > 0; vectorCount--, column += Vector256.Count) + { + (Vector128 lower, Vector128 upper) = Vector128.Widen(Vector128.LoadUnsafe(ref mapRow, (nuint)column)); + Vector256 indices = Vector256.Create(lower, upper); + TOperator.Predict(palette256, indices).StoreUnsafe(ref destinationRow, (nuint)column); + } + } + } + + if (Vector128.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector128Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector128.Count) + { + ulong packedIndices = Unsafe.ReadUnaligned(ref Unsafe.Add(ref mapRow, column)); + Vector128 indices = Vector128.WidenLower(Vector128.CreateScalarUnsafe(packedIndices).AsByte()); + TOperator.Predict(palette128, indices).StoreUnsafe(ref destinationRow, (nuint)column); + } + + if (width - column >= 4) + { + uint packedIndices = Unsafe.ReadUnaligned(ref Unsafe.Add(ref mapRow, column)); + Vector128 indices = Vector128.WidenLower(Vector128.CreateScalarUnsafe(packedIndices).AsByte()); + Vector128 prediction = TOperator.Predict(palette128, indices); + Unsafe.WriteUnaligned(ref Unsafe.As(ref Unsafe.Add(ref destinationRow, column)), prediction.AsUInt64().ToScalar()); + column += 4; + } + } + + for (; column < width; column++) + { + Unsafe.Add(ref destinationRow, column) = TOperator.Predict(ref paletteBase, Unsafe.Add(ref mapRow, column)); + } + } + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1PredictionDecoder.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1PredictionDecoder.cs new file mode 100644 index 0000000000..92928ab500 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1PredictionDecoder.cs @@ -0,0 +1,1167 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Numerics; +using System.Runtime.CompilerServices; +using System.Runtime.InteropServices; +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.ChromaFromLuma; +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; +using SixLabors.ImageSharp.Memory; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; + +/// +/// Reconstructs AV1 intra-predicted transform blocks from neighboring samples and decoded mode information. +/// +/// +/// This type implements the intra prediction portion of the AV1 reconstruction process for 8-, 10-, and 12-bit +/// samples. Intra-edge filtering and upsampling operate on caller-owned padded scratch: adjacent reference samples map +/// to adjacent SIMD lanes, exact-width stores interleave filtered half samples with the original edge, and scalar +/// continuations handle only incomplete vectors. The completed edges then feed the closed prediction operators. +/// +internal sealed class Av1PredictionDecoder +{ + /// + /// The number of samples reserved for one prepared AV1 intra-prediction edge. + /// + private const int ReferenceBufferLength = Av1IntraEdgePreparation.ReferenceBufferLength; + + /// + /// The padded sample count required by the widest intra-edge SIMD loads. + /// + private const int EdgeScratchLength = Av1IntraEdgeFilter.ScratchLength; + + /// + /// The number of high-bit-depth samples required by the reusable prediction workspace. + /// + public const int ScratchLength = Av1DirectionalIntraPredictor.ScratchLength + (2 * ReferenceBufferLength) + EdgeScratchLength; + + /// + /// The sequence-level syntax that controls chroma sampling, bit depth, superblock size, and intra-edge filtering. + /// + private readonly ObuSequenceHeader sequenceHeader; + + /// + /// The frame-level syntax that controls segment lossless state and prediction behavior. + /// + private readonly ObuFrameHeader frameHeader; + + /// + /// The frame-owned workspace shared by directional and filter-intra predictors. + /// + private readonly Memory predictorScratch; + + /// + /// The complete decoder-session palette color-index map state, when supplied by a decoder session. + /// + private readonly Av1TileReader.PaletteColorIndexMaps? paletteColorIndexMaps; + + /// + /// Initializes a new instance of the class. + /// + /// The decoded sequence header for the current image. + /// The decoded frame header for the current image. + /// The reusable predictor workspace owned by the containing block decoder. + /// The complete decoder-session palette map state. + public Av1PredictionDecoder( + ObuSequenceHeader sequenceHeader, + ObuFrameHeader frameHeader, + Memory predictorScratch, + Av1TileReader.PaletteColorIndexMaps? paletteColorIndexMaps = null) + { + this.sequenceHeader = sequenceHeader; + this.frameHeader = frameHeader; + this.predictorScratch = predictorScratch; + this.paletteColorIndexMaps = paletteColorIndexMaps; + } + + /// + /// Reconstructs an 8-bit intra-predicted transform block. + /// + /// The decoded partition and mode state for the containing block. + /// The color plane being reconstructed. + /// The dimensions of the transform block. + /// The tile boundaries used to determine neighboring-sample availability. + /// The sample buffer beginning at the row above the destination block. + /// The distance, in samples, between pixel rows. + /// The bit depth of the reconstructed samples. + /// The transform block's horizontal offset within the mode-information block. + /// The transform block's vertical offset within the mode-information block. + public void Decode( + ref Av1PartitionInfo partitionInfo, + Av1Plane plane, + Av1TransformSize transformSize, + Av1TileInfo tileInfo, + Span pixelBuffer, + int pixelStride, + Av1BitDepth bitDepth, + int blockModeInfoColumnOffset, + int blockModeInfoRowOffset) + => this.DecodeCore( + ref partitionInfo, + plane, + transformSize, + tileInfo, + pixelBuffer, + pixelStride, + bitDepth, + blockModeInfoColumnOffset, + blockModeInfoRowOffset); + + /// + /// Builds the intra predictor for an 8-bit inter-intra plane block in separate caller-owned storage. + /// + public void DecodeInterIntra( + ref Av1PartitionInfo partitionInfo, + Av1Plane plane, + Av1TileInfo tileInfo, + Span referenceBuffer, + int referenceStride, + Span destination, + int destinationStride, + Av1BitDepth bitDepth) + => this.DecodeInterIntraCore( + ref partitionInfo, + plane, + tileInfo, + referenceBuffer, + referenceStride, + destination, + destinationStride, + bitDepth); + + /// + /// Builds the intra predictor for a high-bit-depth inter-intra plane block in separate caller-owned storage. + /// + public void DecodeInterIntra( + ref Av1PartitionInfo partitionInfo, + Av1Plane plane, + Av1TileInfo tileInfo, + Span referenceBuffer, + int referenceStride, + Span destination, + int destinationStride, + Av1BitDepth bitDepth) + => this.DecodeInterIntraCore( + ref partitionInfo, + plane, + tileInfo, + referenceBuffer, + referenceStride, + destination, + destinationStride, + bitDepth); + + /// + /// Builds an inter-intra predictor from reconstructed frame neighbors without replacing those references. + /// + private void DecodeInterIntraCore( + ref Av1PartitionInfo partitionInfo, + Av1Plane plane, + Av1TileInfo tileInfo, + Span referenceBuffer, + int referenceStride, + Span destination, + int destinationStride, + Av1BitDepth bitDepth) + where T : unmanaged, IBinaryInteger + { + ObuColorConfig colorConfig = this.sequenceHeader.ColorConfig; + int subX = plane != Av1Plane.Y && colorConfig.SubSamplingX ? 1 : 0; + int subY = plane != Av1Plane.Y && colorConfig.SubSamplingY ? 1 : 0; + Av1BlockSize planeBlockSize = partitionInfo.ModeInfo.BlockSize.GetSubsampled(subX, subY); + Av1TransformSize transformSize = planeBlockSize.GetMaximumTransformSize(); + Av1PredictionMode mode = partitionInfo.ModeInfo.InterIntraMode switch + { + Av1InterIntraMode.Vertical => Av1PredictionMode.Vertical, + Av1InterIntraMode.Horizontal => Av1PredictionMode.Horizontal, + Av1InterIntraMode.Smooth => Av1PredictionMode.Smooth, + _ => Av1PredictionMode.DC, + }; + + Span topNeighbor = referenceBuffer; + ReadOnlySpan leftNeighbor = referenceBuffer[(referenceStride - 1)..]; + + // The reference decoder predicts one maximum-transform-sized plane block for inter-intra. Destination storage is separate + // because the inter predictor must remain intact until the final mask blend consumes both complete blocks. + this.PredictIntraBlock( + ref partitionInfo, + plane, + transformSize, + tileInfo, + destination, + destinationStride, + topNeighbor, + leftNeighbor, + referenceStride, + mode, + blockModeInfoColumnOffset: 0, + blockModeInfoRowOffset: 0, + bitDepth); + } + + /// + /// Reconstructs a 10-bit or 12-bit intra-predicted transform block. + /// + /// The decoded partition and mode state for the containing block. + /// The color plane being reconstructed. + /// The dimensions of the transform block. + /// The tile boundaries used to determine neighboring-sample availability. + /// The sample buffer beginning at the row above the destination block. + /// The distance, in samples, between pixel rows. + /// The bit depth of the reconstructed samples. + /// The transform block's horizontal offset within the mode-information block. + /// The transform block's vertical offset within the mode-information block. + /// Implements the intra prediction portion of section 7.11.2 of the AV1 specification. + public void Decode( + ref Av1PartitionInfo partitionInfo, + Av1Plane plane, + Av1TransformSize transformSize, + Av1TileInfo tileInfo, + Span pixelBuffer, + int pixelStride, + Av1BitDepth bitDepth, + int blockModeInfoColumnOffset, + int blockModeInfoRowOffset) + => this.DecodeCore( + ref partitionInfo, + plane, + transformSize, + tileInfo, + pixelBuffer, + pixelStride, + bitDepth, + blockModeInfoColumnOffset, + blockModeInfoRowOffset); + + /// + /// Reconstructs an intra-predicted transform block in its native sample representation. + /// + /// The 8-bit or high-bit-depth sample type. + /// The decoded partition and mode state for the containing block. + /// The color plane being reconstructed. + /// The dimensions of the transform block. + /// The tile boundaries used to determine neighboring-sample availability. + /// The sample buffer beginning at the row above the destination block. + /// The distance, in samples, between pixel rows. + /// The bit depth of the reconstructed samples. + /// The transform block's horizontal offset within the mode-information block. + /// The transform block's vertical offset within the mode-information block. + private void DecodeCore( + ref Av1PartitionInfo partitionInfo, + Av1Plane plane, + Av1TransformSize transformSize, + Av1TileInfo tileInfo, + Span pixelBuffer, + int pixelStride, + Av1BitDepth bitDepth, + int blockModeInfoColumnOffset, + int blockModeInfoRowOffset) + where T : unmanaged, IBinaryInteger + { + int stride = pixelStride; + + // Unlike the encoder's separate destination and reference pointers, this span begins at the + // previous row. That layout exposes the top, top-left, and strided left samples without copying. + Span topNeighbor = pixelBuffer; + Span leftNeighbor = pixelBuffer[(stride - 1)..]; + Span startOfPixels = pixelBuffer[stride..]; + + Av1PredictionMode mode = partitionInfo.ModeInfo.YMode; + if (plane != Av1Plane.Y && partitionInfo.ModeInfo.UvMode == Av1ChromaPredictionMode.ChromaFromLuma) + { + this.PredictIntraBlock( + ref partitionInfo, + plane, + transformSize, + tileInfo, + startOfPixels, + stride, + topNeighbor, + leftNeighbor, + stride, + Av1PredictionMode.DC, + blockModeInfoColumnOffset, + blockModeInfoRowOffset, + bitDepth); + + this.PredictChromaFromLumaBlock( + ref partitionInfo, + partitionInfo.ChromaFromLumaContext, + startOfPixels, + stride, + transformSize, + plane); + + return; + } + + if (plane != Av1Plane.Y) + { + // Chroma and luma modes are separate bitstream domains. Shared spatial predictors consume the explicit + // the reference decoder get_uv_mode() equivalent rather than relying on their matching ordinal values. + mode = partitionInfo.ModeInfo.UvMode.ToLumaMode(); + } + + this.PredictIntraBlock( + ref partitionInfo, + plane, + transformSize, + tileInfo, + startOfPixels, + stride, + topNeighbor, + leftNeighbor, + stride, + mode, + blockModeInfoColumnOffset, + blockModeInfoRowOffset, + bitDepth); + } + + /// + /// Applies chroma-from-luma scaling to the DC prediction for one chroma transform block. + /// + /// The 8-bit or high-bit-depth sample type. + /// The decoded partition and mode state for the containing block. + /// The block-level luma prediction context shared by the chroma planes. + /// The DC-predicted chroma samples that receive the luma-derived adjustment. + /// The distance, in samples, between pixel rows. + /// The dimensions of the chroma transform block. + /// The U or V plane being reconstructed. + private void PredictChromaFromLumaBlock( + ref Av1PartitionInfo partitionInfo, + Av1ChromaFromLumaContext? chromaFromLumaContext, + Span pixelBuffer, + int stride, + Av1TransformSize transformSize, + Av1Plane plane) + where T : unmanaged, IBinaryInteger + { + Av1BlockModeInfo modeInfo = partitionInfo.ModeInfo; + Av1BlockSize blockSize = modeInfo.BlockSize; + DebugGuard.MustBeLessThan((int)blockSize, (int)Av1BlockSize.AllSizes, nameof(blockSize)); + bool isChromaFromLumaAllowedFlag = blockSize.AllowsChromaFromLuma( + this.frameHeader.LosslessArray[modeInfo.SegmentId], + this.sequenceHeader.ColorConfig.SubSamplingX, + this.sequenceHeader.ColorConfig.SubSamplingY); + + DebugGuard.IsTrue(isChromaFromLumaAllowedFlag, "Chroma from Luma should be allowed then computing it."); + + if (chromaFromLumaContext == null) + { + throw new InvalidOperationException("CFL context should have been defined already."); + } + + // U computes the shared subsampled-luma parameters first; V reuses them for the + // same block because both chroma planes have identical sampling geometry. + if (!chromaFromLumaContext.AreParametersComputed) + { + chromaFromLumaContext.ComputeParameters(transformSize); + } + + int alphaQ3 = ChromaFromLumaIndexToAlpha(modeInfo.ChromaFromLumaAlphaIndex, modeInfo.ChromaFromLumaAlphaSign, plane); + + Av1BitDepth bitDepth = this.sequenceHeader.ColorConfig.BitDepth; + int width = transformSize.GetWidth(); + int height = transformSize.GetHeight(); + + if (typeof(T) == typeof(byte)) + { + Av1ChromaFromLumaPredictor.Predict(chromaFromLumaContext.Q3Buffer, MemoryMarshal.Cast(pixelBuffer), stride, alphaQ3, width, height); + } + else + { + Av1ChromaFromLumaPredictor.Predict(chromaFromLumaContext.Q3Buffer, MemoryMarshal.Cast(pixelBuffer), stride, alphaQ3, bitDepth.GetBitCount(), width, height); + } + } + + /// + /// Converts the packed chroma-from-luma magnitude and joint sign into a signed Q3 scaling factor. + /// + /// The packed U and V alpha magnitudes. + /// The joint U and V alpha-sign symbol. + /// The U or V plane whose alpha value is selected. + /// The signed Q3 alpha value for the selected chroma plane. + public static int ChromaFromLumaIndexToAlpha(int alphaIndex, int jointSign, Av1Plane plane) + { + int alphaSign = (plane == Av1Plane.U) ? Av1ChromaFromLumaMath.SignU(jointSign) : Av1ChromaFromLumaMath.SignV(jointSign); + if (alphaSign == Av1ChromaFromLumaMath.SignZero) + { + return 0; + } + + int absAlphaQ3 = (plane == Av1Plane.U) ? Av1ChromaFromLumaMath.IndexU(alphaIndex) : Av1ChromaFromLumaMath.IndexV(alphaIndex); + return (alphaSign == Av1ChromaFromLumaMath.SignPositive) ? absAlphaQ3 + 1 : -absAlphaQ3 - 1; + } + + /// + /// Determines available reference samples and dispatches prediction for one transform block. + /// + /// The 8-bit or high-bit-depth sample type. + /// The decoded partition and mode state for the containing block. + /// The color plane being reconstructed. + /// The dimensions of the transform block. + /// The tile boundaries used to determine neighboring-sample availability. + /// The destination samples for the transform block. + /// The distance, in samples, between destination rows. + /// The reconstructed samples along the top edge. + /// The reconstructed samples along the left edge. + /// The distance, in samples, between consecutive left-edge references. + /// The intra prediction mode to apply. + /// The transform block's horizontal offset within the mode-information block. + /// The transform block's vertical offset within the mode-information block. + /// The bit depth of the reconstructed samples. + private void PredictIntraBlock( + ref Av1PartitionInfo partitionInfo, + Av1Plane plane, + Av1TransformSize transformSize, + Av1TileInfo tileInfo, + Span pixelBuffer, + int pixelBufferStride, + Span topNeighbor, + ReadOnlySpan leftNeighbor, + int referenceStride, + Av1PredictionMode mode, + int blockModeInfoColumnOffset, + int blockModeInfoRowOffset, + Av1BitDepth bitDepth) + where T : unmanaged, IBinaryInteger + { + ObuColorConfig cc = this.sequenceHeader.ColorConfig; + int subX = plane != Av1Plane.Y ? cc.SubSamplingX ? 1 : 0 : 0; + int subY = plane != Av1Plane.Y ? cc.SubSamplingY ? 1 : 0 : 0; + + Av1BlockModeInfo modeInfo = partitionInfo.ModeInfo; + + int transformWidth = transformSize.GetWidth(); + int transformHeight = transformSize.GetHeight(); + int transformWidthInModeInfoUnits = transformSize.Get4x4WideCount(); + int transformHeightInModeInfoUnits = transformSize.Get4x4HighCount(); + + bool usePalette = modeInfo.GetPaletteSize(plane) > 0; + + if (usePalette) + { + ReadOnlySpan paletteColors = modeInfo.GetPaletteColors(plane); + Av1TileReader.PaletteColorIndexMaps? paletteColorIndexMapState = this.paletteColorIndexMaps; + if (paletteColorIndexMapState is null) + { + throw new InvalidOperationException("Palette prediction requires decoder-session color-index maps."); + } + + Av1TileReader.PaletteColorIndexMaps paletteColorIndexMaps = paletteColorIndexMapState.Value; + Buffer2D colorIndexBuffer = plane == Av1Plane.Y + ? paletteColorIndexMaps.Luma + : paletteColorIndexMaps.Chroma; + + Buffer2DRegion colorIndexMap = modeInfo.GetPaletteColorIndexMap(plane, colorIndexBuffer); + Buffer2DRegion transformColorIndexMap = colorIndexMap.GetSubRegion( + blockModeInfoColumnOffset << Av1Constants.ModeInfoSizeLog2, + blockModeInfoRowOffset << Av1Constants.ModeInfoSizeLog2, + transformWidth, + transformHeight); + + // Every transform reconstructs its own window of the block-level palette map. The row-oriented region + // keeps this traversal valid when the frame-owned map spans multiple allocator memory groups. + if (typeof(T) == typeof(byte)) + { + Span byteDestination = MemoryMarshal.Cast(pixelBuffer); + Av1PalettePredictor.Predict(paletteColors, transformColorIndexMap, byteDestination, pixelBufferStride, transformWidth, transformHeight); + } + else + { + Span highBitDepthDestination = MemoryMarshal.Cast(pixelBuffer); + Av1PalettePredictor.Predict(paletteColors, transformColorIndexMap, highBitDepthDestination, pixelBufferStride, transformWidth, transformHeight); + } + + return; + } + + Av1FilterIntraMode filterIntraMode = (plane == Av1Plane.Y && modeInfo.UseFilterIntra) + ? modeInfo.FilterIntraMode : Av1FilterIntraMode.AllFilterIntraModes; + + int angleDelta = modeInfo.GetAngleDelta(plane); + + Av1BlockSize blockSize = modeInfo.BlockSize; + bool haveTop = blockModeInfoRowOffset > 0 || (subY > 0 ? partitionInfo.AvailableAboveForChroma : partitionInfo.AvailableAbove); + bool haveLeft = blockModeInfoColumnOffset > 0 || (subX > 0 ? partitionInfo.AvailableLeftForChroma : partitionInfo.AvailableLeft); + + int modeInfoRow = -partitionInfo.ModeBlockToTopEdge >> (3 + Av1Constants.ModeInfoSizeLog2); + int modeInfoColumn = -partitionInfo.ModeBlockToLeftEdge >> (3 + Av1Constants.ModeInfoSizeLog2); + int xrOffset = 0; + int ydOffset = 0; + + // These distances bound edge extension at the coded frame rather than allowing + // a transform to read padding that happens to exist beyond the visible image. + int xr = (partitionInfo.ModeBlockToRightEdge >> (3 + subX)) + + (partitionInfo.GetWidthInPixels(plane) - (blockModeInfoColumnOffset << Av1Constants.ModeInfoSizeLog2) - transformWidth) - + xrOffset; + + int yd = (partitionInfo.ModeBlockToBottomEdge >> (3 + subY)) + + (partitionInfo.GetHeightInPixels(plane) - (blockModeInfoRowOffset << Av1Constants.ModeInfoSizeLog2) - transformHeight) - ydOffset; + + bool rightAvailable = modeInfoColumn + ((blockModeInfoColumnOffset + transformWidthInModeInfoUnits) << subX) < tileInfo.ModeInfoColumnEnd; + bool bottomAvailable = (yd > 0) && (modeInfoRow + ((blockModeInfoRowOffset + transformHeightInModeInfoUnits) << subY) < tileInfo.ModeInfoRowEnd); + + Av1PartitionType partition = modeInfo.PartitionType; + + // Chroma prediction geometry cannot be smaller than 4 by 4 after subsampling. + blockSize = ScaleChromaBlockSize(blockSize, subX == 1, subY == 1); + + bool haveTopRight = Av1IntraReferenceAvailability.HasTopRight( + this.sequenceHeader.SuperblockSize, + blockSize, + modeInfoRow, + modeInfoColumn, + haveTop, + rightAvailable, + partition, + transformSize, + blockModeInfoRowOffset, + blockModeInfoColumnOffset, + subX, + subY); + bool haveBottomLeft = Av1IntraReferenceAvailability.HasBottomLeft( + this.sequenceHeader.SuperblockSize, + blockSize, + modeInfoRow, + modeInfoColumn, + bottomAvailable, + haveLeft, + partition, + transformSize, + blockModeInfoRowOffset, + blockModeInfoColumnOffset, + subX, + subY); + + bool disableEdgeFilter = !this.sequenceHeader.EnableIntraEdgeFilter; + + // Calling all other intra predictors except CFL and palette. + this.DecodeBuildIntraPredictors( + ref partitionInfo, + topNeighbor, + leftNeighbor, + (nuint)referenceStride, + pixelBuffer, + (nuint)pixelBufferStride, + mode, + angleDelta, + filterIntraMode, + transformSize, + disableEdgeFilter, + haveTop ? Math.Min(transformWidth, xr + transformWidth) : 0, + haveTopRight ? Math.Min(transformWidth, xr) : 0, + haveLeft ? Math.Min(transformHeight, yd + transformHeight) : 0, + haveBottomLeft ? Math.Min(transformHeight, yd) : 0, + plane, + bitDepth.GetBitCount()); + } + + /// + /// Adjusts sub-8-by-8 luma block geometry to the minimum chroma prediction block size. + /// + /// The luma block size. + /// A value indicating whether chroma is horizontally subsampled. + /// A value indicating whether chroma is vertically subsampled. + /// The block size used to evaluate chroma reference availability. + private static Av1BlockSize ScaleChromaBlockSize(Av1BlockSize blockSize, bool subX, bool subY) + { + Av1BlockSize bs = blockSize; + switch (blockSize) + { + case Av1BlockSize.Block4x4: + if (subX && subY) + { + bs = Av1BlockSize.Block8x8; + } + else if (subX) + { + bs = Av1BlockSize.Block8x4; + } + else if (subY) + { + bs = Av1BlockSize.Block4x8; + } + + break; + case Av1BlockSize.Block4x8: + if (subX && subY) + { + bs = Av1BlockSize.Block8x8; + } + else if (subX) + { + bs = Av1BlockSize.Block8x8; + } + else if (subY) + { + bs = Av1BlockSize.Block4x8; + } + + break; + case Av1BlockSize.Block8x4: + if (subX && subY) + { + bs = Av1BlockSize.Block8x8; + } + else if (subX) + { + bs = Av1BlockSize.Block8x4; + } + else if (subY) + { + bs = Av1BlockSize.Block8x8; + } + + break; + case Av1BlockSize.Block4x16: + if (subX && subY) + { + bs = Av1BlockSize.Block8x16; + } + else if (subX) + { + bs = Av1BlockSize.Block8x16; + } + else if (subY) + { + bs = Av1BlockSize.Block4x16; + } + + break; + case Av1BlockSize.Block16x4: + if (subX && subY) + { + bs = Av1BlockSize.Block16x8; + } + else if (subX) + { + bs = Av1BlockSize.Block16x4; + } + else if (subY) + { + bs = Av1BlockSize.Block16x8; + } + + break; + default: + break; + } + + return bs; + } + + /// + /// Prepares normative reference-edge samples and runs the selected intra predictor. + /// + /// The 8-bit or high-bit-depth sample type. + /// The decoded partition and neighboring mode state. + /// The reconstructed top and top-right reference samples. + /// The reconstructed left and bottom-left reference samples. + /// The distance, in samples, between consecutive left-edge references. + /// The buffer that receives the prediction block. + /// The distance, in samples, between destination rows. + /// The intra prediction mode to apply. + /// The coded directional angle adjustment. + /// The selected filter intra mode, or the sentinel indicating that filter intra is disabled. + /// The dimensions of the prediction block. + /// A value indicating whether intra-edge filtering and upsampling are disabled. + /// The number of available top samples. + /// The number of available top-right extension samples. + /// The number of available left samples. + /// The number of available bottom-left extension samples. + /// The color plane being reconstructed. + /// The number of bits used to represent each sample. + private void DecodeBuildIntraPredictors( + ref Av1PartitionInfo partitionInfo, + Span aboveNeighbor, + ReadOnlySpan leftNeighbor, + nuint referenceStride, + Span destination, + nuint destinationStride, + Av1PredictionMode mode, + int angleDelta, + Av1FilterIntraMode filterIntraMode, + Av1TransformSize transformSize, + bool disableEdgeFilter, + int topPixelCount, + int topRightPixelCount, + int leftPixelCount, + int bottomLeftPixelCount, + Av1Plane plane, + int bitDepth) + where T : unmanaged, IBinaryInteger + { + int baseValue = 128 << (bitDepth - 8); + + // The frame-owned allocation is sized in high-bit-depth samples. Reinterpreting it as T gives the byte + // path additional capacity while preserving the same sample offsets for the larger short representation. + Span scratch = MemoryMarshal.Cast(this.predictorScratch.Span); + Span aboveData = scratch.Slice(Av1DirectionalIntraPredictor.ScratchLength, ReferenceBufferLength); + Span leftData = scratch.Slice(Av1DirectionalIntraPredictor.ScratchLength + ReferenceBufferLength, ReferenceBufferLength); + Span edgeScratch = scratch.Slice(Av1DirectionalIntraPredictor.ScratchLength + (2 * ReferenceBufferLength), EdgeScratchLength); + + // Prefix storage is required because AV1 addresses the shared top-left sample at -1 + // and writes upsampled edge samples as far back as -2. + aboveData.Fill(T.CreateChecked(baseValue - 1)); + leftData.Fill(T.CreateChecked(baseValue + 1)); + Span aboveRow = aboveData[Av1IntraEdgePreparation.ReferencePrefixLength..]; + Span leftColumn = leftData[Av1IntraEdgePreparation.ReferencePrefixLength..]; + int transformWidth = transformSize.GetWidth(); + int transformHeight = transformSize.GetHeight(); + bool isDirectionalMode = mode.IsDirectional(); + Av1NeighborNeed need = mode.GetNeighborNeed(); + bool needLeft = (need & Av1NeighborNeed.Left) == Av1NeighborNeed.Left; + bool needAbove = (need & Av1NeighborNeed.Above) == Av1NeighborNeed.Above; + bool needAboveLeft = (need & Av1NeighborNeed.AboveLeft) == Av1NeighborNeed.AboveLeft; + int angle = 0; + bool useFilterIntra = filterIntraMode != Av1FilterIntraMode.AllFilterIntraModes; + + if (isDirectionalMode) + { + angle = mode.ToAngle() + (angleDelta * Av1Constants.AngleStep); + if (angle <= 90) + { + needAbove = true; + needLeft = false; + needAboveLeft = true; + } + else if (angle < 180) + { + needAbove = true; + needLeft = true; + needAboveLeft = true; + } + else + { + needAbove = false; + needLeft = true; + needAboveLeft = true; + } + } + + if (useFilterIntra) + { + needAbove = true; + needLeft = true; + needAboveLeft = true; + } + + DebugGuard.MustBeGreaterThanOrEqualTo(topPixelCount, 0, nameof(topPixelCount)); + DebugGuard.MustBeGreaterThanOrEqualTo(topRightPixelCount, 0, nameof(topRightPixelCount)); + DebugGuard.MustBeGreaterThanOrEqualTo(leftPixelCount, 0, nameof(leftPixelCount)); + DebugGuard.MustBeGreaterThanOrEqualTo(bottomLeftPixelCount, 0, nameof(bottomLeftPixelCount)); + + if ((!needAbove && leftPixelCount == 0) || (!needLeft && topPixelCount == 0)) + { + // Pure horizontal or vertical prediction with its sole required edge missing + // degenerates to the first perpendicular sample or the normative midpoint offset. + T value; + if (needLeft) + { + value = topPixelCount > 0 ? aboveNeighbor[0] : T.CreateChecked(baseValue + 1); + } + else + { + value = leftPixelCount > 0 ? leftNeighbor[0] : T.CreateChecked(baseValue - 1); + } + + for (int i = 0; i < transformHeight; ++i) + { + destination.Slice(i * (int)destinationStride, transformWidth).Fill(value); + } + + return; + } + + // Copy the available left and bottom-left samples, then extend the final sample + // through any unavailable portion required by the selected predictor. + if (needLeft) + { + bool needBottom = (need & Av1NeighborNeed.BottomLeft) == Av1NeighborNeed.BottomLeft; + if (useFilterIntra) + { + needBottom = false; + } + + if (isDirectionalMode) + { + needBottom = angle > 180; + } + + int numLeftPixelsNeeded = transformHeight + (needBottom ? transformWidth : 0); + int i = 0; + if (leftPixelCount > 0) + { + for (; i < leftPixelCount; i++) + { + leftColumn[i] = leftNeighbor[i * (int)referenceStride]; + } + + if (needBottom && bottomLeftPixelCount > 0) + { + Guard.IsTrue(i == transformHeight, nameof(i), string.Empty); + for (; i < transformHeight + bottomLeftPixelCount; i++) + { + leftColumn[i] = leftNeighbor[i * (int)referenceStride]; + } + } + + if (i < numLeftPixelsNeeded) + { + leftColumn.Slice(i, numLeftPixelsNeeded - i).Fill(leftColumn[i - 1]); + } + } + else + { + if (topPixelCount > 0) + { + leftColumn[..numLeftPixelsNeeded].Fill(aboveNeighbor[0]); + } + else + { + leftColumn[..numLeftPixelsNeeded].Fill(T.CreateChecked(baseValue + 1)); + } + } + } + + // Prepare the top edge by the same copy-and-extend rule. Unlike the left edge, + // these samples are contiguous in the reconstructed pixel buffer. + if (needAbove) + { + bool needRight = (need & Av1NeighborNeed.AboveRight) == Av1NeighborNeed.AboveRight; + if (useFilterIntra) + { + needRight = false; + } + + if (isDirectionalMode) + { + needRight = angle < 90; + } + + int numTopPixelsNeeded = transformWidth + (needRight ? transformHeight : 0); + if (topPixelCount > 0) + { + aboveNeighbor[..topPixelCount].CopyTo(aboveRow); + int i = topPixelCount; + if (topRightPixelCount > 0) + { + Guard.IsTrue(topPixelCount == transformWidth, nameof(topPixelCount), string.Empty); + aboveNeighbor.Slice(transformWidth, topRightPixelCount).CopyTo(aboveRow[transformWidth..]); + i += topRightPixelCount; + } + + if (i < numTopPixelsNeeded) + { + aboveRow.Slice(i, numTopPixelsNeeded - i).Fill(aboveRow[i - 1]); + } + } + else + { + if (leftPixelCount > 0) + { + aboveRow[..numTopPixelsNeeded].Fill(leftNeighbor[0]); + } + else + { + aboveRow[..numTopPixelsNeeded].Fill(T.CreateChecked(baseValue - 1)); + } + } + } + + if (needAboveLeft) + { + // AV1 synthesizes the shared corner from the closest available edge when only + // one edge exists, and uses the bit-depth midpoint when neither edge exists. + ref T aboveLeft = ref Unsafe.Subtract(ref aboveRow[0], 1); + if (topPixelCount > 0 && leftPixelCount > 0) + { + aboveLeft = Unsafe.Subtract(ref aboveNeighbor[0], 1); + } + else if (topPixelCount > 0) + { + aboveLeft = aboveNeighbor[0]; + } + else if (leftPixelCount > 0) + { + aboveLeft = leftNeighbor[0]; + } + else + { + aboveLeft = T.CreateChecked(baseValue); + } + + Unsafe.Subtract(ref leftColumn[0], 1) = aboveLeft; + } + + if (useFilterIntra) + { + this.FilterIntraPredictor(destination, destinationStride, transformSize, aboveRow, leftColumn, filterIntraMode, bitDepth); + return; + } + + if (isDirectionalMode) + { + bool upsampleAbove = false; + bool upsampleLeft = false; + if (!disableEdgeFilter) + { + Av1IntraEdgePreparation.Prepare( + aboveRow, + leftColumn, + transformWidth, + transformHeight, + angle, + topPixelCount, + leftPixelCount, + GetFilterType(ref partitionInfo, plane), + bitDepth, + edgeScratch, + out upsampleAbove, + out upsampleLeft); + } + + this.DirectionalPredictor(destination, destinationStride, transformSize, aboveRow, leftColumn, upsampleAbove, upsampleLeft, angle); + return; + } + + if (mode == Av1PredictionMode.DC) + { + DcPredictor(leftPixelCount > 0, topPixelCount > 0, transformSize, destination, destinationStride, aboveRow, leftColumn, bitDepth); + } + else + { + GeneralPredictor(mode, transformSize, destination, destinationStride, aboveRow, leftColumn); + } + } + + /// + /// Dispatches DC prediction to the 8-bit or high-bit-depth implementation. + /// + /// The byte or 16-bit sample type. + /// A value indicating whether reconstructed left samples are available. + /// A value indicating whether reconstructed top samples are available. + /// The dimensions of the prediction block. + /// The buffer that receives the predicted samples. + /// The distance, in samples, between destination rows. + /// The prepared top reference samples. + /// The prepared left reference samples. + /// The number of bits used to represent each sample. + private static void DcPredictor(bool hasLeft, bool hasAbove, Av1TransformSize transformSize, Span destination, nuint destinationStride, Span above, Span left, int bitDepth) + where T : unmanaged + { + int width = transformSize.GetWidth(); + int height = transformSize.GetHeight(); + + // DecodeCore is reachable only through byte and short overloads, so this type + // dispatch permits shared reference preparation without boxing or allocating. + if (typeof(T) == typeof(byte)) + { + Av1DcIntraPredictor.Predict( + hasLeft, + hasAbove, + MemoryMarshal.Cast(destination), + (int)destinationStride, + MemoryMarshal.Cast(above), + MemoryMarshal.Cast(left), + width, + height); + } + else + { + Av1DcIntraPredictor.Predict( + hasLeft, + hasAbove, + MemoryMarshal.Cast(destination), + (int)destinationStride, + MemoryMarshal.Cast(above), + MemoryMarshal.Cast(left), + width, + height, + bitDepth); + } + } + + /// + /// Dispatches nondirectional prediction to the 8-bit or high-bit-depth implementation. + /// + /// The byte or 16-bit sample type. + /// The nondirectional prediction mode to apply. + /// The dimensions of the prediction block. + /// The buffer that receives the predicted samples. + /// The distance, in samples, between destination rows. + /// The prepared top reference samples. + /// The prepared left reference samples. + private static void GeneralPredictor(Av1PredictionMode mode, Av1TransformSize transformSize, Span destination, nuint destinationStride, Span above, Span left) + where T : unmanaged + { + int width = transformSize.GetWidth(); + int height = transformSize.GetHeight(); + Av1NonDirectionalIntraPredictorBase predictor = Av1NonDirectionalIntraPredictorBase.GetPredictor(mode); + + if (typeof(T) == typeof(byte)) + { + predictor.Predict( + MemoryMarshal.Cast(destination), + (int)destinationStride, + MemoryMarshal.Cast(above), + MemoryMarshal.Cast(left), + width, + height); + } + else + { + predictor.Predict( + MemoryMarshal.Cast(destination), + (int)destinationStride, + MemoryMarshal.Cast(above), + MemoryMarshal.Cast(left), + width, + height); + } + } + + /// + /// Dispatches directional prediction to the 8-bit or high-bit-depth implementation. + /// + /// The byte or 16-bit sample type. + /// The buffer that receives the predicted samples. + /// The distance, in samples, between destination rows. + /// The dimensions of the prediction block. + /// The prepared top reference samples. + /// The prepared left reference samples. + /// A value indicating whether the top edge was upsampled. + /// A value indicating whether the left edge was upsampled. + /// The adjusted prediction angle in degrees. + private void DirectionalPredictor(Span destination, nuint destinationStride, Av1TransformSize transformSize, Span above, Span left, bool upsampleAbove, bool upsampleLeft, int angle) + where T : unmanaged + { + if (typeof(T) == typeof(byte)) + { + Span scratch = MemoryMarshal.AsBytes(this.predictorScratch.Span)[..Av1DirectionalIntraPredictor.ScratchLength]; + Av1DirectionalIntraPredictor.Predict( + MemoryMarshal.Cast(destination), + (int)destinationStride, + transformSize, + MemoryMarshal.Cast(above), + MemoryMarshal.Cast(left), + upsampleAbove, + upsampleLeft, + angle, + scratch); + } + else + { + Av1DirectionalIntraPredictor.Predict( + MemoryMarshal.Cast(destination), + (int)destinationStride, + transformSize, + MemoryMarshal.Cast(above), + MemoryMarshal.Cast(left), + upsampleAbove, + upsampleLeft, + angle, + this.predictorScratch.Span); + } + } + + /// + /// Dispatches filter intra prediction to the 8-bit or high-bit-depth implementation. + /// + /// The byte or 16-bit sample type. + /// The buffer that receives the predicted samples. + /// The distance, in samples, between destination rows. + /// The dimensions of the prediction block. + /// The prepared top reference samples. + /// The prepared left reference samples. + /// The filter intra mode whose coefficient set is applied. + /// The number of bits used to represent each sample. + private void FilterIntraPredictor(Span destination, nuint destinationStride, Av1TransformSize transformSize, Span above, Span left, Av1FilterIntraMode mode, int bitDepth) + where T : unmanaged + { + int width = transformSize.GetWidth(); + int height = transformSize.GetHeight(); + Av1FilterIntraPredictorBase predictor = Av1FilterIntraPredictorBase.GetPredictor(mode); + + if (typeof(T) == typeof(byte)) + { + Span scratch = MemoryMarshal.AsBytes(this.predictorScratch.Span)[..Av1FilterIntraPredictorBase.ScratchLength]; + predictor.Predict( + MemoryMarshal.Cast(destination), + (int)destinationStride, + MemoryMarshal.Cast(above), + MemoryMarshal.Cast(left), + width, + height, + scratch); + } + else + { + predictor.Predict( + MemoryMarshal.Cast(destination), + (int)destinationStride, + MemoryMarshal.Cast(above), + MemoryMarshal.Cast(left), + width, + height, + bitDepth, + this.predictorScratch.Span[..Av1FilterIntraPredictorBase.ScratchLength]); + } + } + + /// + /// Determines the directional edge-filter threshold class from neighboring prediction modes. + /// + /// The decoded partition and neighboring mode state. + /// The color plane whose neighbors are inspected. + /// when either relevant neighbor uses a smooth mode; otherwise, . + private static bool GetFilterType(ref Av1PartitionInfo partitionInfo, Av1Plane plane) + { + Av1BlockModeInfo? above; + Av1BlockModeInfo? left; + if (plane == Av1Plane.Y) + { + above = partitionInfo.AboveModeInfo; + left = partitionInfo.LeftModeInfo; + } + else + { + above = partitionInfo.AboveModeInfoForChroma; + left = partitionInfo.LeftModeInfoForChroma; + } + + bool aboveIsSmooth = above is not null && IsSmooth(above.Value, plane); + bool leftIsSmooth = left is not null && IsSmooth(left.Value, plane); + return aboveIsSmooth || leftIsSmooth; + } + + /// + /// Determines whether a block uses any AV1 smooth intra prediction mode on the requested plane. + /// + /// The neighboring block's decoded mode state. + /// The luma or chroma plane class whose mode is inspected. + /// for smooth, smooth-horizontal, or smooth-vertical prediction; otherwise, . + private static bool IsSmooth(Av1BlockModeInfo modeInfo, Av1Plane plane) + { + if (plane == Av1Plane.Y) + { + Av1PredictionMode mode = modeInfo.YMode; + return mode is Av1PredictionMode.Smooth or + Av1PredictionMode.SmoothVertical or + Av1PredictionMode.SmoothHorizontal; + } + else + { + // Chroma modes use their own enum and carry only the intra predictors relevant to this neighbor check. + Av1ChromaPredictionMode uvMode = modeInfo.UvMode; + return uvMode is Av1ChromaPredictionMode.Smooth or + Av1ChromaPredictionMode.SmoothVertical or + Av1ChromaPredictionMode.SmoothHorizontal; + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1PredictionMode.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1PredictionMode.cs new file mode 100644 index 0000000000..720b78c790 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1PredictionMode.cs @@ -0,0 +1,190 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; + +/// +/// Identifies the luma intra and inter prediction modes used by an AV1 coding block. +/// +internal enum Av1PredictionMode : byte +{ + /// + /// Predicts each sample from the average of the available top and left neighbors. + /// + DC, + + /// + /// Repeats the top neighboring row vertically through the block. + /// + Vertical, + + /// + /// Repeats the left neighboring column horizontally through the block. + /// + Horizontal, + + /// + /// Projects neighboring samples into the block at 45 degrees. + /// + Directional45Degrees, + + /// + /// Projects neighboring samples into the block at 135 degrees. + /// + Directional135Degrees, + + /// + /// Projects neighboring samples into the block at 113 degrees. + /// + Directional113Degrees, + + /// + /// Projects neighboring samples into the block at 157 degrees. + /// + Directional157Degrees, + + /// + /// Projects neighboring samples into the block at 203 degrees. + /// + Directional203Degrees, + + /// + /// Projects neighboring samples into the block at 67 degrees. + /// + Directional67Degrees, + + /// + /// Blends horizontal and vertical smooth predictions. + /// + Smooth, + + /// + /// Interpolates vertically between the top row and the bottom-left neighbor. + /// + SmoothVertical, + + /// + /// Interpolates horizontally between the left column and the top-right neighbor. + /// + SmoothHorizontal, + + /// + /// Selects the neighbor with the smallest gradient from the top-left reference. + /// + Paeth, + + /// + /// Uses the nearest motion-vector candidate for one reference frame. + /// + NearestMotionVector = 13, + + /// + /// Uses a near motion-vector candidate for one reference frame. + /// + NearMotionVector = 14, + + /// + /// Uses the global-motion model for one reference frame. + /// + GlobalMotionVector = 15, + + /// + /// Decodes a new motion vector for one reference frame. + /// + NewMotionVector = 16, + + /// + /// Uses the nearest motion-vector candidate for both compound references. + /// + NearestNearestMotionVector = 17, + + /// + /// Uses a near motion-vector candidate for both compound references. + /// + NearNearMotionVector = 18, + + /// + /// Uses the nearest candidate for the first compound reference and decodes a new vector for the second. + /// + NearestNewMotionVector = 19, + + /// + /// Decodes a new vector for the first compound reference and uses the nearest candidate for the second. + /// + NewNearestMotionVector = 20, + + /// + /// Uses a near candidate for the first compound reference and decodes a new vector for the second. + /// + NearNewMotionVector = 21, + + /// + /// Decodes a new vector for the first compound reference and uses a near candidate for the second. + /// + NewNearMotionVector = 22, + + /// + /// Uses the global-motion model for both compound references. + /// + GlobalGlobalMotionVector = 23, + + /// + /// Decodes a new motion vector for both compound references. + /// + NewNewMotionVector = 24, + + /// + /// The first luma intra-prediction mode. + /// + IntraModeStart = DC, + + /// + /// The exclusive upper bound of luma intra-prediction modes. + /// + IntraModeEnd = Paeth + 1, + + /// + /// The number of luma intra-prediction modes. + /// + IntraModes = Paeth + 1, + + /// + /// The first single-reference inter-prediction mode. + /// + SingleInterModeStart = NearestMotionVector, + + /// + /// The exclusive upper bound of single-reference inter-prediction modes. + /// + SingleInterModeEnd = NearestNearestMotionVector, + + /// + /// The first compound-reference inter-prediction mode. + /// + CompoundInterModeStart = NearestNearestMotionVector, + + /// + /// The exclusive upper bound of compound-reference inter-prediction modes. + /// + CompoundInterModeEnd = NewNewMotionVector + 1, + + /// + /// The first inter-prediction mode. + /// + InterModeStart = NearestMotionVector, + + /// + /// The exclusive upper bound of all inter-prediction modes. + /// + InterModeEnd = NewNewMotionVector + 1, + + /// + /// The number of luma and inter prediction modes in the complete AV1 mode domain. + /// + PredictionModeCount = NewNewMotionVector + 1, + + /// + /// The invalid intra-mode sentinel matching the complete AV1 prediction-mode domain. + /// + IntraInvalid = PredictionModeCount, +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1PreditionModeExtensions.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1PreditionModeExtensions.cs new file mode 100644 index 0000000000..118a2cb238 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1PreditionModeExtensions.cs @@ -0,0 +1,99 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; + +/// +/// Provides transform, direction, angle, and neighbor metadata for AV1 intra-prediction modes. +/// +internal static class Av1PreditionModeExtensions +{ + /// + /// Maps each luma intra-prediction mode to its default two-dimensional transform type. + /// + private static readonly Av1TransformType[] IntraPreditionMode2TransformType = [ + Av1TransformType.DctDct, // DC + Av1TransformType.AdstDct, // V + Av1TransformType.DctAdst, // H + Av1TransformType.DctDct, // D45 + Av1TransformType.AdstAdst, // D135 + Av1TransformType.AdstDct, // D117 + Av1TransformType.DctAdst, // D153 + Av1TransformType.DctAdst, // D207 + Av1TransformType.AdstDct, // D63 + Av1TransformType.AdstAdst, // SMOOTH + Av1TransformType.AdstDct, // SMOOTH_V + Av1TransformType.DctAdst, // SMOOTH_H + Av1TransformType.AdstAdst, // PAETH + ]; + + /// + /// Maps each luma intra-prediction mode to the neighboring sample regions it consumes. + /// + private static readonly Av1NeighborNeed[] NeedsMap = [ + Av1NeighborNeed.Above | Av1NeighborNeed.Left, // DC + Av1NeighborNeed.Above, // V + Av1NeighborNeed.Left, // H + Av1NeighborNeed.Above | Av1NeighborNeed.AboveRight, // D45 + Av1NeighborNeed.Left | Av1NeighborNeed.Above | Av1NeighborNeed.AboveLeft, // D135 + Av1NeighborNeed.Left | Av1NeighborNeed.Above | Av1NeighborNeed.AboveLeft, // D113 + Av1NeighborNeed.Left | Av1NeighborNeed.Above | Av1NeighborNeed.AboveLeft, // D157 + Av1NeighborNeed.Left | Av1NeighborNeed.BottomLeft, // D203 + Av1NeighborNeed.Above | Av1NeighborNeed.AboveRight, // D67 + Av1NeighborNeed.Left | Av1NeighborNeed.Above, // SMOOTH + Av1NeighborNeed.Left | Av1NeighborNeed.Above, // SMOOTH_V + Av1NeighborNeed.Left | Av1NeighborNeed.Above, // SMOOTH_H + Av1NeighborNeed.Left | Av1NeighborNeed.Above | Av1NeighborNeed.AboveLeft, // PAETH + ]; + + /// + /// Maps each luma intra-prediction mode to its base directional angle in degrees, or zero for non-directional modes. + /// + private static readonly int[] AngleMap = [ + 0, + 90, + 180, + 45, + 135, + 113, + 157, + 203, + 67, + 0, + 0, + 0, + 0, + ]; + + /// + /// Gets the default transform type associated with an intra-prediction mode. + /// + /// The luma intra-prediction mode. + /// The default transform type. + public static Av1TransformType ToTransformType(this Av1PredictionMode mode) => IntraPreditionMode2TransformType[(int)mode]; + + /// + /// Determines whether an intra-prediction mode projects samples along a coded angle. + /// + /// The luma intra-prediction mode. + /// for a directional mode; otherwise, . + public static bool IsDirectional(this Av1PredictionMode mode) + => mode is >= Av1PredictionMode.Vertical and <= Av1PredictionMode.Directional67Degrees; + + /// + /// Gets the neighboring sample regions required by an intra-prediction mode. + /// + /// The luma intra-prediction mode. + /// The required neighboring sample flags. + public static Av1NeighborNeed GetNeighborNeed(this Av1PredictionMode mode) => NeedsMap[(int)mode]; + + /// + /// Gets the base prediction angle for an intra-prediction mode. + /// + /// The luma intra-prediction mode. + /// The prediction angle in degrees, or zero for a non-directional mode. + public static int ToAngle(this Av1PredictionMode mode) => AngleMap[(int)mode]; +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/ChromaFromLuma/Av1ChromaFromLumaContext.Operations.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/ChromaFromLuma/Av1ChromaFromLumaContext.Operations.cs new file mode 100644 index 0000000000..4e22a9bba0 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/ChromaFromLuma/Av1ChromaFromLumaContext.Operations.cs @@ -0,0 +1,510 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.InteropServices; +using System.Runtime.Intrinsics; +using System.Runtime.Intrinsics.Arm; +using System.Runtime.Intrinsics.X86; +using SixLabors.ImageSharp.Common.Helpers; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.ChromaFromLuma; + +/// +/// Provides packed luma subsampling and mean-subtraction operations for chroma-from-luma prediction. Consecutive lanes +/// represent chroma coordinates. Horizontal and vertical luma sums are converted directly to Q3, after which a vector +/// reduction derives the common rounded mean and lane-wise subtraction leaves the zero-mean AC predictor surface. +/// +internal partial class Av1ChromaFromLumaContext +{ + /// + /// Subsamples one reconstructed eight-bit luma block and removes its rounded Q3 mean. + /// + /// The reconstructed luma samples. + /// The distance, in samples, between input rows. + /// The fixed-stride Q3 predictor workspace. + /// The chroma transform dimensions. + /// Whether two horizontal luma samples map to each chroma sample. + /// Whether two vertical luma samples map to each chroma sample. + public static void PrepareBlock( + ReadOnlySpan input, + int inputStride, + Span output, + Av1TransformSize transformSize, + bool subsamplingX, + bool subsamplingY) + { + int lumaWidth = transformSize.GetWidth() << (subsamplingX ? 1 : 0); + int lumaHeight = transformSize.GetHeight() << (subsamplingY ? 1 : 0); + StoreSamples(input, inputStride, 0, lumaWidth, lumaHeight, output, subsamplingX, subsamplingY); + SubtractAverage(output, transformSize); + } + + /// + /// Subsamples one reconstructed high-bit-depth luma block and removes its rounded Q3 mean. + /// + /// The reconstructed luma samples. + /// The distance, in samples, between input rows. + /// The fixed-stride Q3 predictor workspace. + /// The chroma transform dimensions. + /// Whether two horizontal luma samples map to each chroma sample. + /// Whether two vertical luma samples map to each chroma sample. + public static void PrepareBlock( + ReadOnlySpan input, + int inputStride, + Span output, + Av1TransformSize transformSize, + bool subsamplingX, + bool subsamplingY) + { + int lumaWidth = transformSize.GetWidth() << (subsamplingX ? 1 : 0); + int lumaHeight = transformSize.GetHeight() << (subsamplingY ? 1 : 0); + StoreSamples(input, inputStride, 0, lumaWidth, lumaHeight, output, subsamplingX, subsamplingY); + SubtractAverage(output, transformSize); + } + + /// + /// Stores 8-bit reconstructed luma samples in the Q3 predictor surface. + /// + /// The reconstructed luma samples. + /// The distance, in samples, between input rows. + /// The first destination sample in the fixed-stride predictor buffer. + /// The luma width in samples. + /// The luma height in samples. + /// The fixed-stride Q3 predictor workspace. + /// Whether horizontal luma pairs are subsampled. + /// Whether vertical luma pairs are subsampled. + private static void StoreSamples( + ReadOnlySpan input, + int inputStride, + int outputOffset, + int width, + int height, + Span output, + bool subsamplingX, + bool subsamplingY) + { + ref byte inputBase = ref MemoryMarshal.GetReference(input); + ref short outputBase = ref MemoryMarshal.GetReference(output); + + if (!subsamplingX) + { + // One luma sample maps directly to one chroma sample, so multiplying by eight converts it to Q3. + for (int row = 0; row < height; row++) + { + ref byte inputRow = ref Unsafe.Add(ref inputBase, row * inputStride); + ref short outputRow = ref Unsafe.Add(ref outputBase, outputOffset + (row * BufferLine)); + int column = 0; + + if (Vector256.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector256Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector256.Count) + { + (Vector256 lower, Vector256 upper) = Vector256.Widen(Vector256.LoadUnsafe(ref inputRow, (nuint)column)); + + (lower << 3).AsInt16().StoreUnsafe(ref outputRow, (nuint)column); + (upper << 3).AsInt16().StoreUnsafe(ref outputRow, (nuint)(column + Vector256.Count)); + } + } + + if (Vector128.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector128Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector128.Count) + { + (Vector128 lower, Vector128 upper) = Vector128.Widen(Vector128.LoadUnsafe(ref inputRow, (nuint)column)); + + (lower << 3).AsInt16().StoreUnsafe(ref outputRow, (nuint)column); + (upper << 3).AsInt16().StoreUnsafe(ref outputRow, (nuint)(column + Vector128.Count)); + } + + if (column < width) + { + ulong packed = width - column == 4 + ? Unsafe.ReadUnaligned(ref Unsafe.Add(ref inputRow, column)) + : Unsafe.ReadUnaligned(ref Unsafe.Add(ref inputRow, column)); + Vector128 q3 = (Vector128.WidenLower(Vector128.CreateScalarUnsafe(packed).AsByte()) << 3).AsInt16(); + + if (width - column == 4) + { + Unsafe.WriteUnaligned(ref Unsafe.As(ref Unsafe.Add(ref outputRow, column)), q3.AsUInt64().ToScalar()); + } + else + { + q3.StoreUnsafe(ref outputRow, (nuint)column); + } + + column = width; + } + } + + for (; column < width; column++) + { + Unsafe.Add(ref outputRow, column) = (short)(Unsafe.Add(ref inputRow, column) << 3); + } + } + + return; + } + + Vector256 ones256 = Vector256.Create((sbyte)1); + Vector128 ones128 = Vector128.Create((sbyte)1); + int rowStep = subsamplingY ? 2 : 1; + int outputShift = subsamplingY ? 1 : 2; + for (int row = 0; row < height; row += rowStep) + { + ref byte inputRow = ref Unsafe.Add(ref inputBase, row * inputStride); + ref byte nextInputRow = ref Unsafe.Add(ref inputRow, subsamplingY ? inputStride : 0); + ref short outputRow = ref Unsafe.Add(ref outputBase, outputOffset + ((row >> (subsamplingY ? 1 : 0)) * BufferLine)); + int column = 0; + + if (Avx2.IsSupported) + { + nuint vectorCount = Numerics.Vector256Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector256.Count) + { + Vector256 sum = Avx2.MultiplyAddAdjacent(Vector256.LoadUnsafe(ref inputRow, (nuint)column), ones256); + if (subsamplingY) + { + sum += Avx2.MultiplyAddAdjacent(Vector256.LoadUnsafe(ref nextInputRow, (nuint)column), ones256); + } + + (sum << outputShift).StoreUnsafe(ref outputRow, (nuint)(column >> 1)); + } + } + + if (Vector128.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector128Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector128.Count) + { + Vector128 sum = PairSum(Vector128.LoadUnsafe(ref inputRow, (nuint)column), ones128); + if (subsamplingY) + { + sum += PairSum(Vector128.LoadUnsafe(ref nextInputRow, (nuint)column), ones128); + } + + (sum << outputShift).StoreUnsafe(ref outputRow, (nuint)(column >> 1)); + } + + if (column < width) + { + int remaining = width - column; + ulong packed = remaining == 4 + ? Unsafe.ReadUnaligned(ref Unsafe.Add(ref inputRow, column)) + : Unsafe.ReadUnaligned(ref Unsafe.Add(ref inputRow, column)); + Vector128 sum = PairSum(Vector128.CreateScalarUnsafe(packed).AsByte(), ones128); + if (subsamplingY) + { + packed = remaining == 4 + ? Unsafe.ReadUnaligned(ref Unsafe.Add(ref nextInputRow, column)) + : Unsafe.ReadUnaligned(ref Unsafe.Add(ref nextInputRow, column)); + sum += PairSum(Vector128.CreateScalarUnsafe(packed).AsByte(), ones128); + } + + Vector128 q3 = sum << outputShift; + if (remaining == 4) + { + Unsafe.WriteUnaligned(ref Unsafe.As(ref Unsafe.Add(ref outputRow, column >> 1)), q3.AsUInt32().ToScalar()); + } + else + { + Unsafe.WriteUnaligned(ref Unsafe.As(ref Unsafe.Add(ref outputRow, column >> 1)), q3.AsUInt64().ToScalar()); + } + + column = width; + } + } + + for (; column < width; column += 2) + { + int sum = Unsafe.Add(ref inputRow, column) + Unsafe.Add(ref inputRow, column + 1); + if (subsamplingY) + { + sum += Unsafe.Add(ref nextInputRow, column) + Unsafe.Add(ref nextInputRow, column + 1); + } + + Unsafe.Add(ref outputRow, column >> 1) = (short)(sum << outputShift); + } + } + } + + /// + /// Stores high-bit-depth reconstructed luma samples in the Q3 predictor surface. + /// + /// The reconstructed luma samples. + /// The distance, in samples, between input rows. + /// The first destination sample in the fixed-stride predictor buffer. + /// The luma width in samples. + /// The luma height in samples. + /// The fixed-stride Q3 predictor workspace. + /// Whether horizontal luma pairs are subsampled. + /// Whether vertical luma pairs are subsampled. + private static void StoreSamples( + ReadOnlySpan input, + int inputStride, + int outputOffset, + int width, + int height, + Span output, + bool subsamplingX, + bool subsamplingY) + { + ref short inputBase = ref MemoryMarshal.GetReference(input); + ref short outputBase = ref MemoryMarshal.GetReference(output); + + if (!subsamplingX) + { + for (int row = 0; row < height; row++) + { + ref short inputRow = ref Unsafe.Add(ref inputBase, row * inputStride); + ref short outputRow = ref Unsafe.Add(ref outputBase, outputOffset + (row * BufferLine)); + int column = 0; + + if (Vector256.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector256Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector256.Count) + { + (Vector256.LoadUnsafe(ref inputRow, (nuint)column) << 3).StoreUnsafe(ref outputRow, (nuint)column); + } + } + + if (Vector128.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector128Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector128.Count) + { + (Vector128.LoadUnsafe(ref inputRow, (nuint)column) << 3).StoreUnsafe(ref outputRow, (nuint)column); + } + + if (column < width) + { + ulong packed = Unsafe.ReadUnaligned(ref Unsafe.As(ref Unsafe.Add(ref inputRow, column))); + Vector128 q3 = Vector128.CreateScalarUnsafe(packed).AsInt16() << 3; + Unsafe.WriteUnaligned(ref Unsafe.As(ref Unsafe.Add(ref outputRow, column)), q3.AsUInt64().ToScalar()); + column += 4; + } + } + + for (; column < width; column++) + { + Unsafe.Add(ref outputRow, column) = (short)(Unsafe.Add(ref inputRow, column) << 3); + } + } + + return; + } + + Vector256 ones256 = Vector256.Create((short)1); + Vector128 ones128 = Vector128.Create((short)1); + int rowStep = subsamplingY ? 2 : 1; + int outputShift = subsamplingY ? 1 : 2; + for (int row = 0; row < height; row += rowStep) + { + ref short inputRow = ref Unsafe.Add(ref inputBase, row * inputStride); + ref short nextInputRow = ref Unsafe.Add(ref inputRow, subsamplingY ? inputStride : 0); + ref short outputRow = ref Unsafe.Add(ref outputBase, outputOffset + ((row >> (subsamplingY ? 1 : 0)) * BufferLine)); + int column = 0; + + if (Vector256.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector256Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector256.Count) + { + Vector256 sum = Vector256_.MultiplyAddAdjacent(Vector256.LoadUnsafe(ref inputRow, (nuint)column), ones256); + if (subsamplingY) + { + sum += Vector256_.MultiplyAddAdjacent(Vector256.LoadUnsafe(ref nextInputRow, (nuint)column), ones256); + } + + Vector256.Narrow(sum << outputShift, Vector256.Zero).GetLower().StoreUnsafe(ref outputRow, (nuint)(column >> 1)); + } + } + + if (Vector128.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector128Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector128.Count) + { + Vector128 sum = Vector128_.MultiplyAddAdjacent(Vector128.LoadUnsafe(ref inputRow, (nuint)column), ones128); + if (subsamplingY) + { + sum += Vector128_.MultiplyAddAdjacent(Vector128.LoadUnsafe(ref nextInputRow, (nuint)column), ones128); + } + + Vector128.Narrow(sum << outputShift, Vector128.Zero).GetLower().StoreUnsafe(ref outputRow, (nuint)(column >> 1)); + } + + if (column < width) + { + Vector128 samples = Vector128.CreateScalarUnsafe( + Unsafe.ReadUnaligned(ref Unsafe.As(ref Unsafe.Add(ref inputRow, column)))).AsInt16(); + Vector128 sum = Vector128_.MultiplyAddAdjacent(samples, ones128); + if (subsamplingY) + { + samples = Vector128.CreateScalarUnsafe( + Unsafe.ReadUnaligned(ref Unsafe.As(ref Unsafe.Add(ref nextInputRow, column)))).AsInt16(); + sum += Vector128_.MultiplyAddAdjacent(samples, ones128); + } + + Vector64 q3 = Vector128.Narrow(sum << outputShift, Vector128.Zero).GetLower(); + Unsafe.WriteUnaligned(ref Unsafe.As(ref Unsafe.Add(ref outputRow, column >> 1)), q3.AsUInt32().ToScalar()); + column += 4; + } + } + + for (; column < width; column += 2) + { + int sum = Unsafe.Add(ref inputRow, column) + Unsafe.Add(ref inputRow, column + 1); + if (subsamplingY) + { + sum += Unsafe.Add(ref nextInputRow, column) + Unsafe.Add(ref nextInputRow, column + 1); + } + + Unsafe.Add(ref outputRow, column >> 1) = (short)(sum << outputShift); + } + } + } + + /// + /// Subtracts the rounded Q3 average from each predictor sample, leaving the AC contribution used by CfL. + /// + /// The fixed-stride Q3 predictor workspace. + /// The populated predictor dimensions. + private static void SubtractAverage(Span buffer, Av1TransformSize transformSize) + { + int width = transformSize.GetWidth(); + int height = transformSize.GetHeight(); + + // Transform dimensions are powers of two, so division by the sample count is an exact right shift. Half + // the sample count is accumulated first to implement the normative nearest-integer rounding. + int sumQ3 = (width * height) >> 1; + ref short bufferBase = ref MemoryMarshal.GetReference(buffer); + + if (Vector256.IsHardwareAccelerated && width >= Vector256.Count) + { + for (int row = 0; row < height; row++) + { + int rowOffset = row * BufferLine; + for (int column = 0; column < width; column += Vector256.Count) + { + (Vector256 lower, Vector256 upper) = Vector256.Widen(Vector256.LoadUnsafe(ref bufferBase, (nuint)(rowOffset + column))); + + sumQ3 += Vector256.Sum(lower) + Vector256.Sum(upper); + } + } + } + else if (Vector128.IsHardwareAccelerated) + { + for (int row = 0; row < height; row++) + { + int rowOffset = row * BufferLine; + int column = 0; + nuint vectorCount = Numerics.Vector128Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector128.Count) + { + (Vector128 lower, Vector128 upper) = Vector128.Widen(Vector128.LoadUnsafe(ref bufferBase, (nuint)(rowOffset + column))); + + sumQ3 += Vector128.Sum(lower) + Vector128.Sum(upper); + } + + if (column < width) + { + ulong packed = Unsafe.ReadUnaligned(ref Unsafe.As(ref Unsafe.Add(ref bufferBase, rowOffset + column))); + sumQ3 += Vector128.Sum(Vector128.WidenLower(Vector128.CreateScalarUnsafe(packed).AsInt16())); + } + } + } + else + { + for (int row = 0; row < height; row++) + { + int rowOffset = row * BufferLine; + for (int column = 0; column < width; column++) + { + sumQ3 += Unsafe.Add(ref bufferBase, rowOffset + column); + } + } + } + + int pelCountLog2 = transformSize.GetBlockWidthLog2() + transformSize.GetBlockHeightLog2(); + short averageQ3 = (short)(sumQ3 >> pelCountLog2); + + if (Vector256.IsHardwareAccelerated && width >= Vector256.Count) + { + Vector256 average = Vector256.Create(averageQ3); + for (int row = 0; row < height; row++) + { + int rowOffset = row * BufferLine; + for (int column = 0; column < width; column += Vector256.Count) + { + (Vector256.LoadUnsafe(ref bufferBase, (nuint)(rowOffset + column)) - average).StoreUnsafe(ref bufferBase, (nuint)(rowOffset + column)); + } + } + + return; + } + + if (Vector128.IsHardwareAccelerated) + { + Vector128 average = Vector128.Create(averageQ3); + for (int row = 0; row < height; row++) + { + int rowOffset = row * BufferLine; + int column = 0; + nuint vectorCount = Numerics.Vector128Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector128.Count) + { + (Vector128.LoadUnsafe(ref bufferBase, (nuint)(rowOffset + column)) - average).StoreUnsafe(ref bufferBase, (nuint)(rowOffset + column)); + } + + if (column < width) + { + ulong packed = Unsafe.ReadUnaligned(ref Unsafe.As(ref Unsafe.Add(ref bufferBase, rowOffset + column))); + Vector128 result = Vector128.CreateScalarUnsafe(packed).AsInt16() - average; + Unsafe.WriteUnaligned(ref Unsafe.As(ref Unsafe.Add(ref bufferBase, rowOffset + column)), result.AsUInt64().ToScalar()); + } + } + + return; + } + + for (int row = 0; row < height; row++) + { + int rowOffset = row * BufferLine; + for (int column = 0; column < width; column++) + { + Unsafe.Add(ref bufferBase, rowOffset + column) -= averageQ3; + } + } + } + + /// + /// Adds adjacent 8-bit samples into eight 16-bit lanes. + /// + /// The packed source samples. + /// The multiplier used by the x86 adjacent multiply-add instruction. + /// The adjacent pair sums. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector128 PairSum(Vector128 samples, Vector128 ones) + { + if (Ssse3.IsSupported) + { + return Ssse3.MultiplyAddAdjacent(samples, ones); + } + + if (AdvSimd.IsSupported) + { + return AdvSimd.AddPairwiseWidening(samples).AsInt16(); + } + + // WebAssembly has byte shuffles but no pairwise-widening intrinsic. Grouping even and odd bytes before + // widening keeps all eight pair additions vectorized without adding a general-purpose utility wrapper. + Vector128 even = Vector128.Shuffle(samples, Vector128.Create((byte)0, 2, 4, 6, 8, 10, 12, 14, 255, 255, 255, 255, 255, 255, 255, 255)); + Vector128 odd = Vector128.Shuffle(samples, Vector128.Create((byte)1, 3, 5, 7, 9, 11, 13, 15, 255, 255, 255, 255, 255, 255, 255, 255)); + return (Vector128.WidenLower(even) + Vector128.WidenLower(odd)).AsInt16(); + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/ChromaFromLuma/Av1ChromaFromLumaContext.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/ChromaFromLuma/Av1ChromaFromLumaContext.cs new file mode 100644 index 0000000000..39e3de8803 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/ChromaFromLuma/Av1ChromaFromLumaContext.cs @@ -0,0 +1,222 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Numerics; +using System.Runtime.InteropServices; +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.ChromaFromLuma; + +/// +/// Accumulates subsampled luma samples and derives the zero-mean Q3 predictor surface used by AV1 chroma-from-luma prediction. +/// +internal sealed partial class Av1ChromaFromLumaContext +{ + /// + /// The fixed row stride and maximum dimension, in chroma samples, of the luma predictor buffer. + /// + public const int BufferLine = 32; + + /// + /// The number of samples in the fixed-stride chroma-from-luma workspace. + /// + public const int BufferLength = BufferLine * BufferLine; + + /// + /// The caller-owned fixed-stride luma predictor workspace. + /// + private readonly Memory q3Buffer; + + /// + /// The number of initialized predictor rows currently stored in . + /// + private int bufferHeight; + + /// + /// The number of initialized predictor columns currently stored in . + /// + private int bufferWidth; + + /// + /// Whether luma is subsampled by two along the horizontal axis for the chroma planes. + /// + private readonly bool subX; + + /// + /// Whether luma is subsampled by two along the vertical axis for the chroma planes. + /// + private readonly bool subY; + + /// + /// Initializes a new instance of the class. + /// + /// The AV1 color configuration that supplies chroma subsampling. + public Av1ChromaFromLumaContext(ObuColorConfig colorConfig) + : this(colorConfig, new short[BufferLength]) + { + } + + /// + /// Initializes a new instance of the class over caller-owned workspace. + /// + /// The AV1 color configuration that supplies chroma subsampling. + /// The fixed-stride signed Q3 workspace retained for the context lifetime. + public Av1ChromaFromLumaContext(ObuColorConfig colorConfig, Memory q3Buffer) + { + this.subX = colorConfig.SubSamplingX; + this.subY = colorConfig.SubSamplingY; + this.q3Buffer = q3Buffer; + } + + /// + /// Gets the fixed-stride luma predictor samples in signed Q3 fixed-point representation. + /// + public Span Q3Buffer => this.q3Buffer.Span; + + /// + /// Gets a value indicating whether edge padding and mean subtraction have been applied to the current samples. + /// + public bool AreParametersComputed { get; private set; } + + /// + /// Stores one reconstructed luma transform region in the chroma-resolution Q3 predictor buffer. + /// + /// The integer sample type of the reconstructed luma plane. + /// The reconstructed luma samples for the transform region. + /// The distance, in samples, between consecutive input rows. + /// The transform row relative to the chroma-from-luma block, in mode-info units. + /// The transform column relative to the chroma-from-luma block, in mode-info units. + /// The luma transform dimensions. + /// The coded luma block size used to resolve shared sub-8x8 chroma ownership. + /// The frame-relative luma row in 4x4 mode-info units. + /// The frame-relative luma column in 4x4 mode-info units. + public void Store( + Span input, + int inputStride, + int row, + int column, + Av1TransformSize transformSize, + Av1BlockSize blockSize, + int modeInfoRow, + int modeInfoColumn) + where T : unmanaged, IBinaryInteger + { + if (blockSize.GetHeight() == 4 || blockSize.GetWidth() == 4) + { + // Subsampled chroma shares one CfL surface across the adjacent sub-8x8 luma blocks. + if ((modeInfoRow & 1) != 0 && this.subY) + { + row++; + } + + if ((modeInfoColumn & 1) != 0 && this.subX) + { + column++; + } + } + + int subX = this.subX ? 1 : 0; + int subY = this.subY ? 1 : 0; + int width = transformSize.GetWidth(); + int height = transformSize.GetHeight(); + int storeRow = row << (Av1Constants.ModeInfoSizeLog2 - subY); + int storeColumn = column << (Av1Constants.ModeInfoSizeLog2 - subX); + int storeWidth = width >> subX; + int storeHeight = height >> subY; + + // New luma samples invalidate the previously padded, zero-mean surface. + this.AreParametersComputed = false; + + if (column == 0 && row == 0) + { + this.bufferWidth = storeWidth; + this.bufferHeight = storeHeight; + } + else + { + this.bufferWidth = Math.Max(storeColumn + storeWidth, this.bufferWidth); + this.bufferHeight = Math.Max(storeRow + storeHeight, this.bufferHeight); + } + + int outputOffset = (storeRow * BufferLine) + storeColumn; + + // Reconstruction reaches this method only through the byte and short decoder pipelines. Dispatching once + // here keeps sample conversion out of the row kernels and lets the JIT specialize both storage layouts. + if (typeof(T) == typeof(byte)) + { + StoreSamples( + MemoryMarshal.Cast(input), + inputStride, + outputOffset, + width, + height, + this.Q3Buffer, + this.subX, + this.subY); + } + else + { + StoreSamples( + MemoryMarshal.Cast(input), + inputStride, + outputOffset, + width, + height, + this.Q3Buffer, + this.subX, + this.subY); + } + } + + /// + /// Pads the populated predictor extent to the transform dimensions and subtracts its rounded mean. + /// + /// The chroma prediction transform dimensions. + public void ComputeParameters(Av1TransformSize transformSize) + { + Guard.IsFalse(this.AreParametersComputed, nameof(this.AreParametersComputed), "Do not call cfl_compute_parameters multiple time on the same values."); + this.Pad(transformSize.GetWidth(), transformSize.GetHeight()); + SubtractAverage(this.Q3Buffer, transformSize); + this.AreParametersComputed = true; + } + + /// + /// Extends the last initialized column and row to cover the requested predictor dimensions. + /// + /// The required predictor width in chroma samples. + /// The required predictor height in chroma samples. + private void Pad(int width, int height) + { + int differenceWidth = width - this.bufferWidth; + int differenceHeight = height - this.bufferHeight; + Span q3Buffer = this.Q3Buffer; + + if (differenceWidth > 0) + { + int minimumHeight = height - differenceHeight; + + // AV1 CfL edge extension repeats the final available sample when the coded luma extent is narrower. + for (int y = 0; y < minimumHeight; y++) + { + int rowOffset = y * BufferLine; + short lastPixel = q3Buffer[rowOffset + this.bufferWidth - 1]; + q3Buffer.Slice(rowOffset + this.bufferWidth, differenceWidth).Fill(lastPixel); + } + + this.bufferWidth = width; + } + + if (differenceHeight > 0) + { + // Missing bottom rows repeat the last available row after horizontal extension is complete. + for (int y = this.bufferHeight; y < height; y++) + { + int rowOffset = y * BufferLine; + q3Buffer.Slice(rowOffset - BufferLine, width).CopyTo(q3Buffer.Slice(rowOffset, width)); + } + + this.bufferHeight = height; + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/ChromaFromLuma/Av1ChromaFromLumaMath.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/ChromaFromLuma/Av1ChromaFromLumaMath.cs new file mode 100644 index 0000000000..1c74307f47 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/ChromaFromLuma/Av1ChromaFromLumaMath.cs @@ -0,0 +1,130 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.ChromaFromLuma; + +/// +/// Provides AV1 chroma-from-luma sign, magnitude-index, and entropy-context mappings. +/// +internal static class Av1ChromaFromLumaMath +{ + /// + /// The number of alpha sign states: zero, negative, and positive. + /// + private const int Signs = 3; + + /// + /// The number of bits occupied by each plane's packed alpha-magnitude index. + /// + private const int AlphabetSizeLog2 = 4; + + /// + /// The number of nonzero alpha magnitudes represented by each plane's alphabet. + /// + public const int AlphaMagnitudeCount = 1 << AlphabetSizeLog2; + + /// + /// The number of signed alpha candidates including zero. + /// + public const int AlphaCandidateCount = (AlphaMagnitudeCount * 2) + 1; + + /// + /// The candidate index representing a zero alpha. + /// + public const int AlphaZeroIndex = AlphaMagnitudeCount; + + /// + /// The alpha sign value representing a zero multiplier. + /// + public const int SignZero = 0; + + /// + /// The alpha sign value representing a negative multiplier. + /// + public const int SignNegative = 1; + + /// + /// The alpha sign value representing a positive multiplier. + /// + public const int SignPositive = 2; + + /// + /// Extracts the U-plane sign from a joint chroma sign symbol. + /// + /// The coded joint U/V sign symbol. + /// The U-plane sign state. + public static int SignU(int jointSign) => ((jointSign + 1) * 11) >> 5; + + /// + /// Extracts the V-plane sign from a joint chroma sign symbol. + /// + /// The coded joint U/V sign symbol. + /// The V-plane sign state. + public static int SignV(int jointSign) => (jointSign + 1) - (Signs * SignU(jointSign)); + + /// + /// Extracts the U-plane alpha-magnitude index from the high four bits of the packed index. + /// + /// The packed U/V alpha-magnitude index. + /// The U-plane magnitude index. + public static int IndexU(int index) => index >> AlphabetSizeLog2; + + /// + /// Extracts the V-plane alpha-magnitude index from the low four bits of the packed index. + /// + /// The packed U/V alpha-magnitude index. + /// The V-plane magnitude index. + public static int IndexV(int index) => index & ((1 << AlphabetSizeLog2) - 1); + + /// + /// Maps a joint sign symbol to the entropy context used for the U-plane alpha magnitude. + /// + /// The coded joint U/V sign symbol. + /// The U-plane alpha entropy context. + public static int ContextU(int jointSign) => jointSign + 1 - Signs; + + /// + /// Maps a joint sign symbol to the symmetric entropy context used for the V-plane alpha magnitude. + /// + /// The coded joint U/V sign symbol. + /// The V-plane alpha entropy context. + public static int ContextV(int jointSign) => (SignV(jointSign) * Signs) + SignU(jointSign) - Signs; + + /// + /// Converts a signed-candidate index to its alpha value in Q3 units. + /// + /// The candidate index in negative-to-positive order. + /// The signed Q3 alpha value. + public static int CandidateIndexToAlpha(int candidateIndex) => candidateIndex - AlphaZeroIndex; + + /// + /// Converts a signed Q3 alpha value to its coded sign state. + /// + /// The signed alpha value. + /// The zero, negative, or positive sign state. + public static int AlphaToSign(int alphaQ3) + => alphaQ3 == 0 ? SignZero : alphaQ3 < 0 ? SignNegative : SignPositive; + + /// + /// Converts a nonzero signed Q3 alpha value to its coded magnitude index. + /// + /// The signed alpha value. + /// The zero-based magnitude index, or zero for a zero alpha. + public static int AlphaToMagnitudeIndex(int alphaQ3) => alphaQ3 == 0 ? 0 : Math.Abs(alphaQ3) - 1; + + /// + /// Combines the U and V sign states into the coded joint symbol. + /// + /// The U-plane sign state. + /// The V-plane sign state. + /// The joint sign symbol. + public static int JointSign(int signU, int signV) => (signU * Signs) + signV - 1; + + /// + /// Packs the U and V alpha-magnitude indices into the coded byte. + /// + /// The U-plane magnitude index. + /// The V-plane magnitude index. + /// The packed magnitude indices. + public static int PackIndices(int indexU, int indexV) => (indexU << AlphabetSizeLog2) + indexV; +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/ChromaFromLuma/Av1ChromaFromLumaPredictor.Operator.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/ChromaFromLuma/Av1ChromaFromLumaPredictor.Operator.cs new file mode 100644 index 0000000000..5cfbec7361 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/ChromaFromLuma/Av1ChromaFromLumaPredictor.Operator.cs @@ -0,0 +1,324 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.InteropServices; +using System.Runtime.Intrinsics; +using System.Runtime.Intrinsics.Arm; +using System.Runtime.Intrinsics.X86; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.ChromaFromLuma; + +/// +/// Defines the closed scalar/SIMD operator contract and traversal for AV1 chroma-from-luma prediction. +/// +internal static partial class Av1ChromaFromLumaPredictor +{ + /// + /// The fixed row stride of the AV1 chroma-from-luma scratch buffer. + /// + private const int BufferLine = 32; + + /// + /// Defines scalar and SIMD signed Q3 arithmetic for AV1 chroma-from-luma prediction. + /// + private interface IChromaFromLumaOperator + { + /// + /// Predicts one chroma sample. + /// + /// The zero-mean Q3 luma sample. + /// The chroma DC prediction. + /// The signed Q3 chroma scaling factor. + /// The maximum sample value. + /// The predicted chroma sample. + public static abstract short Predict(short lumaQ3, short dc, int alphaQ3, short maximum); + + /// + /// Predicts eight chroma samples. + /// + /// The zero-mean Q3 luma samples. + /// The chroma DC prediction. + /// The signed Q3 chroma scaling factor. + /// The maximum sample value. + /// The predicted chroma samples. + public static abstract Vector128 Predict(Vector128 lumaQ3, short dc, int alphaQ3, short maximum); + + /// + /// Predicts sixteen chroma samples. + /// + /// The zero-mean Q3 luma samples. + /// The chroma DC prediction. + /// The signed Q3 chroma scaling factor. + /// The maximum sample value. + /// The predicted chroma samples. + public static abstract Vector256 Predict(Vector256 lumaQ3, short dc, int alphaQ3, short maximum); + + /// + /// Predicts thirty-two chroma samples. + /// + /// The zero-mean Q3 luma samples. + /// The chroma DC prediction. + /// The signed Q3 chroma scaling factor. + /// The maximum sample value. + /// The predicted chroma samples. + public static abstract Vector512 Predict(Vector512 lumaQ3, short dc, int alphaQ3, short maximum); + } + + /// + /// Applies chroma-from-luma prediction to an 8-bit chroma block. + /// + /// The zero-mean Q3 luma surface. + /// The DC-predicted chroma block that receives the luma adjustment. + /// The distance, in samples, between destination rows. + /// The signed Q3 chroma scaling factor. + /// The block width in samples. + /// The block height in samples. + public static void Predict(ReadOnlySpan lumaQ3, Span destination, int destinationStride, int alphaQ3, int width, int height) + => Predictor.Predict(lumaQ3, destination, destinationStride, alphaQ3, width, height); + + /// + /// Applies chroma-from-luma prediction to a high-bit-depth chroma block. + /// + /// The zero-mean Q3 luma surface. + /// The DC-predicted chroma block that receives the luma adjustment. + /// The distance, in samples, between destination rows. + /// The signed Q3 chroma scaling factor. + /// The number of bits used to represent each sample. + /// The block width in samples. + /// The block height in samples. + public static void Predict(ReadOnlySpan lumaQ3, Span destination, int destinationStride, int alphaQ3, int bitDepth, int width, int height) + => Predictor.Predict(lumaQ3, destination, destinationStride, alphaQ3, bitDepth, width, height); + + /// + /// Applies the decoded chroma scaling factor to zero-mean luma samples. + /// + private readonly struct ChromaFromLumaOperator : IChromaFromLumaOperator + { + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static short Predict(short lumaQ3, short dc, int alphaQ3, short maximum) + { + int scaledLumaQ0 = Av1Math.RoundPowerOf2Signed(alphaQ3 * lumaQ3, 6); + + return (short)Math.Clamp(dc + scaledLumaQ0, (short)0, maximum); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector128 Predict(Vector128 lumaQ3, short dc, int alphaQ3, short maximum) + { + Vector128 dcVector = Vector128.Create(dc); + Vector128 scaledLumaQ0; + + if (Ssse3.IsSupported) + { + Vector128 alphaSign = Vector128.Create((short)alphaQ3); + Vector128 alphaQ12 = Vector128.Create((short)(Math.Abs(alphaQ3) << 9)); + scaledLumaQ0 = Ssse3.MultiplyHighRoundScale(Ssse3.Abs(lumaQ3).AsInt16(), alphaQ12); + Vector128 signMask = (lumaQ3 ^ alphaSign) >> 15; + scaledLumaQ0 = (scaledLumaQ0 ^ signMask) - signMask; + } + else if (AdvSimd.IsSupported) + { + Vector128 alphaSign = Vector128.Create((short)alphaQ3); + Vector128 alphaQ12 = Vector128.Create((short)(Math.Abs(alphaQ3) << 9)); + scaledLumaQ0 = AdvSimd.MultiplyRoundedDoublingSaturateHigh(Vector128.Abs(lumaQ3), alphaQ12); + Vector128 signMask = (lumaQ3 ^ alphaSign) >> 15; + scaledLumaQ0 = (scaledLumaQ0 ^ signMask) - signMask; + } + else + { + // WebAssembly and other Vector128 targets do not expose packed rounded-high multiply. Widening keeps + // the same signed rounding rule without introducing a second scalar traversal. + (Vector128 lower, Vector128 upper) = Vector128.Widen(lumaQ3); + Vector128 alpha = Vector128.Create(alphaQ3); + lower *= alpha; + upper *= alpha; + lower = (lower + Vector128.Create(32) + (lower >> 31)) >> 6; + upper = (upper + Vector128.Create(32) + (upper >> 31)) >> 6; + scaledLumaQ0 = Vector128.Narrow(lower, upper); + } + + return Vector128.Clamp(scaledLumaQ0 + dcVector, Vector128.Zero, Vector128.Create(maximum)); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector256 Predict(Vector256 lumaQ3, short dc, int alphaQ3, short maximum) + { + Vector256 dcVector = Vector256.Create(dc); + Vector256 scaledLumaQ0; + + if (Avx2.IsSupported) + { + Vector256 alphaSign = Vector256.Create((short)alphaQ3); + Vector256 alphaQ12 = Vector256.Create((short)(Math.Abs(alphaQ3) << 9)); + scaledLumaQ0 = Avx2.MultiplyHighRoundScale(Avx2.Abs(lumaQ3).AsInt16(), alphaQ12); + Vector256 signMask = (lumaQ3 ^ alphaSign) >> 15; + scaledLumaQ0 = (scaledLumaQ0 ^ signMask) - signMask; + } + else + { + (Vector256 lower, Vector256 upper) = Vector256.Widen(lumaQ3); + Vector256 alpha = Vector256.Create(alphaQ3); + lower *= alpha; + upper *= alpha; + lower = (lower + Vector256.Create(32) + (lower >> 31)) >> 6; + upper = (upper + Vector256.Create(32) + (upper >> 31)) >> 6; + scaledLumaQ0 = Vector256.Narrow(lower, upper); + } + + return Vector256.Clamp(scaledLumaQ0 + dcVector, Vector256.Zero, Vector256.Create(maximum)); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector512 Predict(Vector512 lumaQ3, short dc, int alphaQ3, short maximum) + { + (Vector512 lower, Vector512 upper) = Vector512.Widen(lumaQ3); + Vector512 alpha = Vector512.Create(alphaQ3); + lower *= alpha; + upper *= alpha; + lower = (lower + Vector512.Create(32) + (lower >> 31)) >> 6; + upper = (upper + Vector512.Create(32) + (upper >> 31)) >> 6; + Vector512 scaledLumaQ0 = Vector512.Narrow(lower, upper); + + return Vector512.Clamp(scaledLumaQ0 + Vector512.Create(dc), Vector512.Zero, Vector512.Create(maximum)); + } + } + + /// + /// Traverses a chroma block through one closed prediction operator. + /// + /// The signed Q3 prediction arithmetic. + private static class Predictor + where TOperator : struct, IChromaFromLumaOperator + { + /// + /// Applies chroma-from-luma prediction to an 8-bit block. + /// + public static void Predict(ReadOnlySpan lumaQ3, Span destination, int destinationStride, int alphaQ3, int width, int height) + { + ref short lumaBase = ref MemoryMarshal.GetReference(lumaQ3); + ref byte destinationBase = ref MemoryMarshal.GetReference(destination); + short dc = destinationBase; + + // CfL follows DC prediction, so one sample supplies the base value for the complete block. The fixed + // scratch stride also makes exact-width Vector128 loads safe for the four-sample AV1 tail. + for (int row = 0; row < height; row++) + { + int lumaRowOffset = row * BufferLine; + int destinationRowOffset = row * destinationStride; + ref short lumaRow = ref Unsafe.Add(ref lumaBase, lumaRowOffset); + ref byte destinationRow = ref Unsafe.Add(ref destinationBase, destinationRowOffset); + int column = 0; + + if (Vector512.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector512Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector512.Count) + { + Vector512 prediction = TOperator.Predict(Vector512.LoadUnsafe(ref lumaRow, (nuint)column), dc, alphaQ3, byte.MaxValue); + Vector256 packed = Vector512.Narrow(prediction.AsUInt16(), Vector512.Zero).GetLower(); + packed.StoreUnsafe(ref destinationRow, (nuint)column); + } + } + + if (Vector256.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector256Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector256.Count) + { + Vector256 prediction = TOperator.Predict(Vector256.LoadUnsafe(ref lumaRow, (nuint)column), dc, alphaQ3, byte.MaxValue); + Vector128 packed = Vector256.Narrow(prediction.AsUInt16(), Vector256.Zero).GetLower(); + packed.StoreUnsafe(ref destinationRow, (nuint)column); + } + } + + if (Vector128.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector128Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector128.Count) + { + Vector128 prediction = TOperator.Predict(Vector128.LoadUnsafe(ref lumaRow, (nuint)column), dc, alphaQ3, byte.MaxValue); + Vector64 packed = Vector128.Narrow(prediction.AsUInt16(), Vector128.Zero).GetLower(); + packed.StoreUnsafe(ref destinationRow, (nuint)column); + } + + if (width - column >= 4) + { + Vector128 prediction = TOperator.Predict(Vector128.LoadUnsafe(ref lumaRow, (nuint)column), dc, alphaQ3, byte.MaxValue); + Vector64 packed = Vector128.Narrow(prediction.AsUInt16(), Vector128.Zero).GetLower(); + Unsafe.WriteUnaligned(ref Unsafe.Add(ref destinationRow, column), packed.AsUInt32().ToScalar()); + column += 4; + } + } + + for (; column < width; column++) + { + Unsafe.Add(ref destinationRow, column) = (byte)TOperator.Predict(Unsafe.Add(ref lumaRow, column), dc, alphaQ3, byte.MaxValue); + } + } + } + + /// + /// Applies chroma-from-luma prediction to a high-bit-depth block. + /// + public static void Predict(ReadOnlySpan lumaQ3, Span destination, int destinationStride, int alphaQ3, int bitDepth, int width, int height) + { + ref short lumaBase = ref MemoryMarshal.GetReference(lumaQ3); + ref short destinationBase = ref MemoryMarshal.GetReference(destination); + short dc = destinationBase; + short maximum = (short)((1 << bitDepth) - 1); + + for (int row = 0; row < height; row++) + { + int lumaRowOffset = row * BufferLine; + int destinationRowOffset = row * destinationStride; + ref short lumaRow = ref Unsafe.Add(ref lumaBase, lumaRowOffset); + ref short destinationRow = ref Unsafe.Add(ref destinationBase, destinationRowOffset); + int column = 0; + + if (Vector512.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector512Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector512.Count) + { + TOperator.Predict(Vector512.LoadUnsafe(ref lumaRow, (nuint)column), dc, alphaQ3, maximum).StoreUnsafe(ref destinationRow, (nuint)column); + } + } + + if (Vector256.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector256Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector256.Count) + { + TOperator.Predict(Vector256.LoadUnsafe(ref lumaRow, (nuint)column), dc, alphaQ3, maximum).StoreUnsafe(ref destinationRow, (nuint)column); + } + } + + if (Vector128.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector128Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector128.Count) + { + TOperator.Predict(Vector128.LoadUnsafe(ref lumaRow, (nuint)column), dc, alphaQ3, maximum).StoreUnsafe(ref destinationRow, (nuint)column); + } + + if (width - column >= 4) + { + Vector128 prediction = TOperator.Predict(Vector128.LoadUnsafe(ref lumaRow, (nuint)column), dc, alphaQ3, maximum); + Unsafe.WriteUnaligned(ref Unsafe.As(ref Unsafe.Add(ref destinationRow, column)), prediction.AsUInt64().ToScalar()); + column += 4; + } + } + + for (; column < width; column++) + { + Unsafe.Add(ref destinationRow, column) = TOperator.Predict(Unsafe.Add(ref lumaRow, column), dc, alphaQ3, maximum); + } + } + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1CompoundAveragePredictor.Operator.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1CompoundAveragePredictor.Operator.cs new file mode 100644 index 0000000000..7b2b721c03 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1CompoundAveragePredictor.Operator.cs @@ -0,0 +1,132 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.Intrinsics; + +using static SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter.Av1TranslationalInterPredictor; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; + +/// +/// Defines equal-weight compound prediction arithmetic. +/// +internal static partial class Av1CompoundAveragePredictor +{ + /// + /// Defines equal-weight compound averaging for scalar and SIMD lane groups. + /// + private interface IAv1CompoundAverageOperator + { + /// + /// Averages two 8-bit samples. + /// + /// The first sample. + /// The second sample. + /// The rounded average. + public static abstract byte Blend(byte first, byte second); + + /// + /// Averages two high-bit-depth samples. + /// + /// The first sample. + /// The second sample. + /// The rounded average. + public static abstract ushort Blend(ushort first, ushort second); + + /// + /// Averages two 128-bit vectors of 8-bit samples. + /// + /// The first samples. + /// The second samples. + /// The rounded averages. + public static abstract Vector128 Blend(Vector128 first, Vector128 second); + + /// + /// Averages two 256-bit vectors of 8-bit samples. + /// + /// The first samples. + /// The second samples. + /// The rounded averages. + public static abstract Vector256 Blend(Vector256 first, Vector256 second); + + /// + /// Averages two 512-bit vectors of 8-bit samples. + /// + /// The first samples. + /// The second samples. + /// The rounded averages. + public static abstract Vector512 Blend(Vector512 first, Vector512 second); + + /// + /// Averages two 128-bit vectors of high-bit-depth samples. + /// + /// The first samples. + /// The second samples. + /// The rounded averages. + public static abstract Vector128 Blend(Vector128 first, Vector128 second); + + /// + /// Averages two 256-bit vectors of high-bit-depth samples. + /// + /// The first samples. + /// The second samples. + /// The rounded averages. + public static abstract Vector256 Blend(Vector256 first, Vector256 second); + + /// + /// Averages two 512-bit vectors of high-bit-depth samples. + /// + /// The first samples. + /// The second samples. + /// The rounded averages. + public static abstract Vector512 Blend(Vector512 first, Vector512 second); + } + + /// + /// Implements equal-weight rounded averaging for scalar and SIMD lane groups. + /// + private readonly struct CompoundAverageOperator : IAv1CompoundAverageOperator + { + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static byte Blend(byte first, byte second) => (byte)((first + second + 1) >> 1); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static ushort Blend(ushort first, ushort second) => (ushort)((first + second + 1) >> 1); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector128 Blend(Vector128 first, Vector128 second) + => (first | second) - ((first ^ second) >> 1); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector256 Blend(Vector256 first, Vector256 second) + => (first | second) - ((first ^ second) >> 1); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector512 Blend(Vector512 first, Vector512 second) + { + // This identity is exactly (a + b + 1) >> 1 but cannot overflow unsigned lanes at any SIMD width. + return (first | second) - ((first ^ second) >> 1); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector128 Blend(Vector128 first, Vector128 second) + => (first | second) - ((first ^ second) >> 1); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector256 Blend(Vector256 first, Vector256 second) + => (first | second) - ((first ^ second) >> 1); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector512 Blend(Vector512 first, Vector512 second) + => (first | second) - ((first ^ second) >> 1); + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1CompoundAveragePredictor.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1CompoundAveragePredictor.cs new file mode 100644 index 0000000000..e0fd1f25c3 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1CompoundAveragePredictor.cs @@ -0,0 +1,281 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.InteropServices; +using System.Runtime.Intrinsics; + +using static SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter.Av1TranslationalInterPredictor; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; + +/// +/// Combines two AV1 inter predictors with equal-weight rounded averaging. +/// +internal static partial class Av1CompoundAveragePredictor +{ + /// + /// Averages an 8-bit predictor into an existing prediction block. + /// + /// The first predictor and combined output. + /// The distance between destination rows in samples. + /// The second predictor. + /// The distance between second-predictor rows in samples. + /// The active block width. + /// The active block height. + public static void Average( + Span destination, + int destinationStride, + ReadOnlySpan second, + int secondStride, + int width, + int height) + => Average(destination, destinationStride, second, secondStride, width, height); + + /// + /// Averages an 8-bit predictor through one closed compound operator. + /// + /// The equal-weight averaging arithmetic. + /// The first predictor and combined output. + /// The distance between destination rows. + /// The second predictor. + /// The distance between second-predictor rows. + /// The active block width. + /// The active block height. + private static void Average( + Span destination, + int destinationStride, + ReadOnlySpan second, + int secondStride, + int width, + int height) + where TOperator : struct, IAv1CompoundAverageOperator + { + for (int row = 0; row < height; row++) + { + Span destinationRow = destination.Slice(row * destinationStride, width); + ReadOnlySpan secondRow = second.Slice(row * secondStride, width); + ref byte destinationReference = ref MemoryMarshal.GetReference(destinationRow); + ref byte secondReference = ref MemoryMarshal.GetReference(secondRow); + int column = 0; + + if (Vector512.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector512Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector512.Count) + { + Vector512 firstVector = Vector512.LoadUnsafe(ref destinationReference, (nuint)column); + Vector512 secondVector = Vector512.LoadUnsafe(ref secondReference, (nuint)column); + TOperator.Blend(firstVector, secondVector).StoreUnsafe(ref destinationReference, (nuint)column); + } + } + + if (Vector256.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector256Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector256.Count) + { + Vector256 firstVector = Vector256.LoadUnsafe(ref destinationReference, (nuint)column); + Vector256 secondVector = Vector256.LoadUnsafe(ref secondReference, (nuint)column); + TOperator.Blend(firstVector, secondVector).StoreUnsafe(ref destinationReference, (nuint)column); + } + } + + if (Vector128.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector128Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector128.Count) + { + Vector128 firstVector = Vector128.LoadUnsafe(ref destinationReference, (nuint)column); + Vector128 secondVector = Vector128.LoadUnsafe(ref secondReference, (nuint)column); + TOperator.Blend(firstVector, secondVector).StoreUnsafe(ref destinationReference, (nuint)column); + } + } + + for (; column < width; column++) + { + destinationRow[column] = TOperator.Blend(destinationRow[column], secondRow[column]); + } + } + } + + /// + /// Averages a high-bit-depth predictor into an existing prediction block. + /// + /// The first predictor and combined output. + /// The distance between destination rows in samples. + /// The second predictor. + /// The distance between second-predictor rows in samples. + /// The active block width. + /// The active block height. + public static void Average( + Span destination, + int destinationStride, + ReadOnlySpan second, + int secondStride, + int width, + int height) + => Average(destination, destinationStride, second, secondStride, width, height); + + /// + /// Averages a high-bit-depth predictor through one closed compound operator. + /// + /// The equal-weight averaging arithmetic. + /// The first predictor and combined output. + /// The distance between destination rows. + /// The second predictor. + /// The distance between second-predictor rows. + /// The active block width. + /// The active block height. + private static void Average( + Span destination, + int destinationStride, + ReadOnlySpan second, + int secondStride, + int width, + int height) + where TOperator : struct, IAv1CompoundAverageOperator + { + for (int row = 0; row < height; row++) + { + Span destinationRow = destination.Slice(row * destinationStride, width); + ReadOnlySpan secondRow = second.Slice(row * secondStride, width); + ref ushort destinationReference = ref MemoryMarshal.GetReference(destinationRow); + ref ushort secondReference = ref MemoryMarshal.GetReference(secondRow); + int column = 0; + + if (Vector512.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector512Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector512.Count) + { + Vector512 firstVector = Vector512.LoadUnsafe(ref destinationReference, (nuint)column); + Vector512 secondVector = Vector512.LoadUnsafe(ref secondReference, (nuint)column); + TOperator.Blend(firstVector, secondVector).StoreUnsafe(ref destinationReference, (nuint)column); + } + } + + if (Vector256.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector256Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector256.Count) + { + Vector256 firstVector = Vector256.LoadUnsafe(ref destinationReference, (nuint)column); + Vector256 secondVector = Vector256.LoadUnsafe(ref secondReference, (nuint)column); + TOperator.Blend(firstVector, secondVector).StoreUnsafe(ref destinationReference, (nuint)column); + } + } + + if (Vector128.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector128Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector128.Count) + { + Vector128 firstVector = Vector128.LoadUnsafe(ref destinationReference, (nuint)column); + Vector128 secondVector = Vector128.LoadUnsafe(ref secondReference, (nuint)column); + TOperator.Blend(firstVector, secondVector).StoreUnsafe(ref destinationReference, (nuint)column); + } + } + + for (; column < width; column++) + { + destinationRow[column] = TOperator.Blend(destinationRow[column], secondRow[column]); + } + } + } + + /// + /// Averages an 8-bit predictor without explicit hardware intrinsics. + /// + /// The first predictor and combined output. + /// The distance between destination rows in samples. + /// The second predictor. + /// The distance between second-predictor rows in samples. + /// The active block width. + /// The active block height. + public static void AverageScalar( + Span destination, + int destinationStride, + ReadOnlySpan second, + int secondStride, + int width, + int height) + => AverageScalar(destination, destinationStride, second, secondStride, width, height); + + /// + /// Averages an 8-bit predictor through one closed scalar compound operator. + /// + /// The equal-weight averaging arithmetic. + /// The first predictor and combined output. + /// The distance between destination rows. + /// The second predictor. + /// The distance between second-predictor rows. + /// The active block width. + /// The active block height. + private static void AverageScalar( + Span destination, + int destinationStride, + ReadOnlySpan second, + int secondStride, + int width, + int height) + where TOperator : struct, IAv1CompoundAverageOperator + { + for (int row = 0; row < height; row++) + { + Span destinationRow = destination.Slice(row * destinationStride, width); + ReadOnlySpan secondRow = second.Slice(row * secondStride, width); + for (int column = 0; column < width; column++) + { + destinationRow[column] = TOperator.Blend(destinationRow[column], secondRow[column]); + } + } + } + + /// + /// Averages a high-bit-depth predictor without explicit hardware intrinsics. + /// + /// The first predictor and combined output. + /// The distance between destination rows in samples. + /// The second predictor. + /// The distance between second-predictor rows in samples. + /// The active block width. + /// The active block height. + public static void AverageScalar( + Span destination, + int destinationStride, + ReadOnlySpan second, + int secondStride, + int width, + int height) + => AverageScalar(destination, destinationStride, second, secondStride, width, height); + + /// + /// Averages a high-bit-depth predictor through one closed scalar compound operator. + /// + /// The equal-weight averaging arithmetic. + /// The first predictor and combined output. + /// The distance between destination rows. + /// The second predictor. + /// The distance between second-predictor rows. + /// The active block width. + /// The active block height. + private static void AverageScalar( + Span destination, + int destinationStride, + ReadOnlySpan second, + int secondStride, + int width, + int height) + where TOperator : struct, IAv1CompoundAverageOperator + { + for (int row = 0; row < height; row++) + { + Span destinationRow = destination.Slice(row * destinationStride, width); + ReadOnlySpan secondRow = second.Slice(row * secondStride, width); + for (int column = 0; column < width; column++) + { + destinationRow[column] = TOperator.Blend(destinationRow[column], secondRow[column]); + } + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1CompoundDistanceWeightedPredictor.Operator.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1CompoundDistanceWeightedPredictor.Operator.cs new file mode 100644 index 0000000000..43ae86623b --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1CompoundDistanceWeightedPredictor.Operator.cs @@ -0,0 +1,211 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.Intrinsics; + +using static SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter.Av1CompoundInterPredictor; +using static SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter.Av1TranslationalInterPredictor; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; + +/// +/// Defines distance-weighted compound prediction arithmetic. +/// +internal static partial class Av1CompoundDistanceWeightedPredictor +{ + /// + /// Defines distance-weighted compound blending for scalar and SIMD lane groups. + /// + private interface IAv1CompoundDistanceWeightedOperator + { + /// + /// Blends two 8-bit samples with display-distance weights. + /// + /// The first sample. + /// The second sample. + /// The first predictor weight. + /// The second predictor weight. + /// The weighted sample. + public static abstract byte Blend(byte first, byte second, int firstWeight, int secondWeight); + + /// + /// Blends two high-bit-depth samples with display-distance weights. + /// + /// The first sample. + /// The second sample. + /// The first predictor weight. + /// The second predictor weight. + /// The weighted sample. + public static abstract ushort Blend(ushort first, ushort second, int firstWeight, int secondWeight); + + /// + /// Blends 128-bit vectors of 8-bit samples with display-distance weights. + /// + /// The first samples. + /// The second samples. + /// The first predictor weight. + /// The second predictor weight. + /// The weighted samples. + public static abstract Vector128 Blend(Vector128 first, Vector128 second, int firstWeight, int secondWeight); + + /// + /// Blends 256-bit vectors of 8-bit samples with display-distance weights. + /// + /// The first samples. + /// The second samples. + /// The first predictor weight. + /// The second predictor weight. + /// The weighted samples. + public static abstract Vector256 Blend(Vector256 first, Vector256 second, int firstWeight, int secondWeight); + + /// + /// Blends 512-bit vectors of 8-bit samples with display-distance weights. + /// + /// The first samples. + /// The second samples. + /// The first predictor weight. + /// The second predictor weight. + /// The weighted samples. + public static abstract Vector512 Blend(Vector512 first, Vector512 second, int firstWeight, int secondWeight); + + /// + /// Blends 128-bit vectors of high-bit-depth samples with display-distance weights. + /// + /// The first samples. + /// The second samples. + /// The first predictor weight. + /// The second predictor weight. + /// The weighted samples. + public static abstract Vector128 Blend(Vector128 first, Vector128 second, int firstWeight, int secondWeight); + + /// + /// Blends 256-bit vectors of high-bit-depth samples with display-distance weights. + /// + /// The first samples. + /// The second samples. + /// The first predictor weight. + /// The second predictor weight. + /// The weighted samples. + public static abstract Vector256 Blend(Vector256 first, Vector256 second, int firstWeight, int secondWeight); + + /// + /// Blends 512-bit vectors of high-bit-depth samples with display-distance weights. + /// + /// The first samples. + /// The second samples. + /// The first predictor weight. + /// The second predictor weight. + /// The weighted samples. + public static abstract Vector512 Blend(Vector512 first, Vector512 second, int firstWeight, int secondWeight); + } + + /// + /// Implements display-distance-weighted blending for scalar and SIMD lane groups. + /// + private readonly struct CompoundDistanceWeightedOperator : IAv1CompoundDistanceWeightedOperator + { + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static byte Blend(byte first, byte second, int firstWeight, int secondWeight) + => (byte)(((first * firstWeight) + (second * secondWeight) + 8) >> DistanceWeightBits); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static ushort Blend(ushort first, ushort second, int firstWeight, int secondWeight) + => (ushort)(((first * firstWeight) + (second * secondWeight) + 8) >> DistanceWeightBits); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector128 Blend(Vector128 first, Vector128 second, int firstWeight, int secondWeight) + { + Av1NonDirectionalIntraPredictorBase.Widen(first, out Vector128 first0, out Vector128 first1, out Vector128 first2, out Vector128 first3); + Av1NonDirectionalIntraPredictorBase.Widen(second, out Vector128 second0, out Vector128 second1, out Vector128 second2, out Vector128 second3); + return Av1NonDirectionalIntraPredictorBase.Narrow( + Blend(first0, second0, firstWeight, secondWeight), + Blend(first1, second1, firstWeight, secondWeight), + Blend(first2, second2, firstWeight, secondWeight), + Blend(first3, second3, firstWeight, secondWeight)); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector256 Blend(Vector256 first, Vector256 second, int firstWeight, int secondWeight) + { + Av1NonDirectionalIntraPredictorBase.Widen(first, out Vector256 first0, out Vector256 first1, out Vector256 first2, out Vector256 first3); + Av1NonDirectionalIntraPredictorBase.Widen(second, out Vector256 second0, out Vector256 second1, out Vector256 second2, out Vector256 second3); + return Av1NonDirectionalIntraPredictorBase.Narrow( + Blend(first0, second0, firstWeight, secondWeight), + Blend(first1, second1, firstWeight, secondWeight), + Blend(first2, second2, firstWeight, secondWeight), + Blend(first3, second3, firstWeight, secondWeight)); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector512 Blend(Vector512 first, Vector512 second, int firstWeight, int secondWeight) + { + Av1NonDirectionalIntraPredictorBase.Widen(first, out Vector512 first0, out Vector512 first1, out Vector512 first2, out Vector512 first3); + Av1NonDirectionalIntraPredictorBase.Widen(second, out Vector512 second0, out Vector512 second1, out Vector512 second2, out Vector512 second3); + return Av1NonDirectionalIntraPredictorBase.Narrow( + Blend(first0, second0, firstWeight, secondWeight), + Blend(first1, second1, firstWeight, secondWeight), + Blend(first2, second2, firstWeight, secondWeight), + Blend(first3, second3, firstWeight, secondWeight)); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector128 Blend(Vector128 first, Vector128 second, int firstWeight, int secondWeight) + { + Av1NonDirectionalIntraPredictorBase.Widen(first.AsInt16(), out Vector128 first0, out Vector128 first1); + Av1NonDirectionalIntraPredictorBase.Widen(second.AsInt16(), out Vector128 second0, out Vector128 second1); + return Av1NonDirectionalIntraPredictorBase.Narrow( + Blend(first0, second0, firstWeight, secondWeight), + Blend(first1, second1, firstWeight, secondWeight)).AsUInt16(); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector256 Blend(Vector256 first, Vector256 second, int firstWeight, int secondWeight) + { + Av1NonDirectionalIntraPredictorBase.Widen(first.AsInt16(), out Vector256 first0, out Vector256 first1); + Av1NonDirectionalIntraPredictorBase.Widen(second.AsInt16(), out Vector256 second0, out Vector256 second1); + return Av1NonDirectionalIntraPredictorBase.Narrow( + Blend(first0, second0, firstWeight, secondWeight), + Blend(first1, second1, firstWeight, secondWeight)).AsUInt16(); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector512 Blend(Vector512 first, Vector512 second, int firstWeight, int secondWeight) + { + Av1NonDirectionalIntraPredictorBase.Widen(first.AsInt16(), out Vector512 first0, out Vector512 first1); + Av1NonDirectionalIntraPredictorBase.Widen(second.AsInt16(), out Vector512 second0, out Vector512 second1); + return Av1NonDirectionalIntraPredictorBase.Narrow( + Blend(first0, second0, firstWeight, secondWeight), + Blend(first1, second1, firstWeight, secondWeight)).AsUInt16(); + } + + /// + /// Applies display-distance weighting to 128-bit vectors of widened samples. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector128 Blend(Vector128 first, Vector128 second, int firstWeight, int secondWeight) + => ((first * Vector128.Create(firstWeight)) + (second * Vector128.Create(secondWeight)) + Vector128.Create(8)) >> DistanceWeightBits; + + /// + /// Applies display-distance weighting to 256-bit vectors of widened samples. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector256 Blend(Vector256 first, Vector256 second, int firstWeight, int secondWeight) + => ((first * Vector256.Create(firstWeight)) + (second * Vector256.Create(secondWeight)) + Vector256.Create(8)) >> DistanceWeightBits; + + /// + /// Applies display-distance weighting to 512-bit vectors of widened samples. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector512 Blend(Vector512 first, Vector512 second, int firstWeight, int secondWeight) + => ((first * Vector512.Create(firstWeight)) + (second * Vector512.Create(secondWeight)) + Vector512.Create(8)) >> DistanceWeightBits; + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1CompoundDistanceWeightedPredictor.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1CompoundDistanceWeightedPredictor.cs new file mode 100644 index 0000000000..f8d0c1a461 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1CompoundDistanceWeightedPredictor.cs @@ -0,0 +1,235 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.InteropServices; +using System.Runtime.Intrinsics; + +using static SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter.Av1CompoundInterPredictor; +using static SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter.Av1TranslationalInterPredictor; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; + +/// +/// Reconstructs display-distance-weighted compound prediction. +/// +internal static partial class Av1CompoundDistanceWeightedPredictor +{ + /// + /// Combines two 8-bit predictors with AV1 display-distance weights. + /// + public static void DistanceWeighted( + Span destination, + int destinationStride, + ReadOnlySpan second, + int secondStride, + int width, + int height, + int firstWeight, + int secondWeight) + => DistanceWeighted( + destination, + destinationStride, + second, + secondStride, + width, + height, + firstWeight, + secondWeight); + + /// + /// Executes one closed 8-bit distance-weighted compound operator. + /// + /// The compound arithmetic operator. + private static void DistanceWeighted( + Span destination, + int destinationStride, + ReadOnlySpan second, + int secondStride, + int width, + int height, + int firstWeight, + int secondWeight) + where TOperator : struct, IAv1CompoundDistanceWeightedOperator + { + for (int row = 0; row < height; row++) + { + Span destinationRow = destination.Slice(row * destinationStride, width); + ReadOnlySpan secondRow = second.Slice(row * secondStride, width); + ref byte destinationReference = ref MemoryMarshal.GetReference(destinationRow); + ref byte secondReference = ref MemoryMarshal.GetReference(secondRow); + int column = 0; + + if (Vector512.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector512Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector512.Count) + { + Vector512 firstVector = Vector512.LoadUnsafe(ref destinationReference, (nuint)column); + Vector512 secondVector = Vector512.LoadUnsafe(ref secondReference, (nuint)column); + TOperator.Blend(firstVector, secondVector, firstWeight, secondWeight).StoreUnsafe(ref destinationReference, (nuint)column); + } + } + + if (Vector256.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector256Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector256.Count) + { + Vector256 firstVector = Vector256.LoadUnsafe(ref destinationReference, (nuint)column); + Vector256 secondVector = Vector256.LoadUnsafe(ref secondReference, (nuint)column); + TOperator.Blend(firstVector, secondVector, firstWeight, secondWeight).StoreUnsafe(ref destinationReference, (nuint)column); + } + } + + if (Vector128.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector128Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector128.Count) + { + Vector128 firstVector = Vector128.LoadUnsafe(ref destinationReference, (nuint)column); + Vector128 secondVector = Vector128.LoadUnsafe(ref secondReference, (nuint)column); + TOperator.Blend(firstVector, secondVector, firstWeight, secondWeight).StoreUnsafe(ref destinationReference, (nuint)column); + } + } + + for (; column < width; column++) + { + destinationRow[column] = TOperator.Blend(destinationRow[column], secondRow[column], firstWeight, secondWeight); + } + } + } + + /// + /// Combines two high-bit-depth predictors with AV1 display-distance weights. + /// + public static void DistanceWeighted( + Span destination, + int destinationStride, + ReadOnlySpan second, + int secondStride, + int width, + int height, + int firstWeight, + int secondWeight) + => DistanceWeighted( + destination, + destinationStride, + second, + secondStride, + width, + height, + firstWeight, + secondWeight); + + /// + /// Executes one closed high-bit-depth distance-weighted compound operator. + /// + /// The compound arithmetic operator. + private static void DistanceWeighted( + Span destination, + int destinationStride, + ReadOnlySpan second, + int secondStride, + int width, + int height, + int firstWeight, + int secondWeight) + where TOperator : struct, IAv1CompoundDistanceWeightedOperator + { + for (int row = 0; row < height; row++) + { + Span destinationRow = destination.Slice(row * destinationStride, width); + ReadOnlySpan secondRow = second.Slice(row * secondStride, width); + ref ushort destinationReference = ref MemoryMarshal.GetReference(destinationRow); + ref ushort secondReference = ref MemoryMarshal.GetReference(secondRow); + int column = 0; + + if (Vector512.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector512Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector512.Count) + { + Vector512 firstVector = Vector512.LoadUnsafe(ref destinationReference, (nuint)column); + Vector512 secondVector = Vector512.LoadUnsafe(ref secondReference, (nuint)column); + TOperator.Blend(firstVector, secondVector, firstWeight, secondWeight).StoreUnsafe(ref destinationReference, (nuint)column); + } + } + + if (Vector256.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector256Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector256.Count) + { + Vector256 firstVector = Vector256.LoadUnsafe(ref destinationReference, (nuint)column); + Vector256 secondVector = Vector256.LoadUnsafe(ref secondReference, (nuint)column); + TOperator.Blend(firstVector, secondVector, firstWeight, secondWeight).StoreUnsafe(ref destinationReference, (nuint)column); + } + } + + if (Vector128.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector128Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector128.Count) + { + Vector128 firstVector = Vector128.LoadUnsafe(ref destinationReference, (nuint)column); + Vector128 secondVector = Vector128.LoadUnsafe(ref secondReference, (nuint)column); + TOperator.Blend(firstVector, secondVector, firstWeight, secondWeight).StoreUnsafe(ref destinationReference, (nuint)column); + } + } + + for (; column < width; column++) + { + destinationRow[column] = TOperator.Blend(destinationRow[column], secondRow[column], firstWeight, secondWeight); + } + } + } + + /// + /// Combines two 8-bit predictors with display-distance weights without explicit hardware intrinsics. + /// + public static void DistanceWeightedScalar( + Span destination, + int destinationStride, + ReadOnlySpan second, + int secondStride, + int width, + int height, + int firstWeight, + int secondWeight) + => DistanceWeightedScalar( + destination, + destinationStride, + second, + secondStride, + width, + height, + firstWeight, + secondWeight); + + /// + /// Executes one closed 8-bit distance-weighted compound operator without explicit hardware intrinsics. + /// + /// The compound arithmetic operator. + private static void DistanceWeightedScalar( + Span destination, + int destinationStride, + ReadOnlySpan second, + int secondStride, + int width, + int height, + int firstWeight, + int secondWeight) + where TOperator : struct, IAv1CompoundDistanceWeightedOperator + { + for (int row = 0; row < height; row++) + { + Span destinationRow = destination.Slice(row * destinationStride, width); + ReadOnlySpan secondRow = second.Slice(row * secondStride, width); + for (int column = 0; column < width; column++) + { + destinationRow[column] = TOperator.Blend(destinationRow[column], secondRow[column], firstWeight, secondWeight); + } + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1CompoundDistanceWeights.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1CompoundDistanceWeights.cs new file mode 100644 index 0000000000..d4338bd283 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1CompoundDistanceWeights.cs @@ -0,0 +1,64 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; + +/// +/// Derives AV1 compound weights from retained reference display distances. +/// +internal static class Av1CompoundDistanceWeights +{ + private const int MaximumFrameDistance = 31; + + /// + /// Derives the weights applied to the first and second predictors. + /// + public static void Derive( + ObuOrderHintInfo orderHintInfo, + ObuFrameHeader frameHeader, + Av1ReferenceFrameType firstReference, + Av1ReferenceFrameType secondReference, + out int firstWeight, + out int secondWeight) + { + ReadOnlySpan quantizedDistanceWeights = [2, 3, 2, 5, 2, 7, 1, MaximumFrameDistance]; + ReadOnlySpan quantizedDistanceLookup = [9, 7, 11, 5, 12, 4, 13, 3]; + ReadOnlySpan referenceFrameIndices = frameHeader.GetReferenceFrameIndices(); + ReadOnlySpan referenceOrderHints = frameHeader.GetReferenceOrderHints(); + int firstCanonicalIndex = (int)firstReference - (int)Av1ReferenceFrameType.Last; + int secondCanonicalIndex = (int)secondReference - (int)Av1ReferenceFrameType.Last; + uint firstOrderHint = referenceOrderHints[(int)referenceFrameIndices[firstCanonicalIndex]]; + uint secondOrderHint = referenceOrderHints[(int)referenceFrameIndices[secondCanonicalIndex]]; + int secondDistance = Av1Math.Clip3( + 0, + MaximumFrameDistance, + Math.Abs(orderHintInfo.GetRelativeDistance(secondOrderHint, frameHeader.OrderHint))); + + int firstDistance = Av1Math.Clip3( + 0, + MaximumFrameDistance, + Math.Abs(orderHintInfo.GetRelativeDistance(frameHeader.OrderHint, firstOrderHint))); + + int order = secondDistance <= firstDistance ? 1 : 0; + int weightClass = 3; + if (secondDistance != 0 && firstDistance != 0) + { + for (weightClass = 0; weightClass < 3; weightClass++) + { + int secondScaledDistance = secondDistance * quantizedDistanceWeights[(weightClass * 2) + order]; + int firstScaledDistance = firstDistance * quantizedDistanceWeights[(weightClass * 2) + (1 - order)]; + if ((secondDistance > firstDistance && secondScaledDistance < firstScaledDistance) || + (secondDistance <= firstDistance && secondScaledDistance > firstScaledDistance)) + { + break; + } + } + } + + firstWeight = quantizedDistanceLookup[(weightClass * 2) + order]; + secondWeight = quantizedDistanceLookup[(weightClass * 2) + (1 - order)]; + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1CompoundInterPredictor.Arithmetic.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1CompoundInterPredictor.Arithmetic.cs new file mode 100644 index 0000000000..bfc20d0137 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1CompoundInterPredictor.Arithmetic.cs @@ -0,0 +1,169 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; + +/// +/// Provides shared final-rounding arithmetic for compound intermediate reconstruction. +/// +internal static partial class Av1CompoundInterPredictor +{ + /// + /// Derives the bias and remaining fractional precision of a compound intermediate. + /// + public static void GetIntermediateRounding(int bitDepth, out int roundBits, out int roundOffset) + { + int intermediateRange = bitDepth + 7 - 3 + 2; + int round0 = 3 + Math.Max(intermediateRange - 16, 0); + int offsetBits = bitDepth + 14 - round0; + roundBits = 14 - round0 - CompoundRound1Bits; + roundOffset = (1 << (offsetBits - CompoundRound1Bits)) + + (1 << (offsetBits - CompoundRound1Bits - 1)); + } + + /// + /// Removes the compound bias and final fractional precision from 128-bit unsigned lanes. + /// + public static Vector128 FinalizeIntermediate(Vector128 value, int roundBits, int roundOffset) + { + Vector128 result = (value - Vector128.Create((ushort)roundOffset)).AsInt16(); + if (roundBits != 0) + { + result = (result + Vector128.Create((short)(1 << (roundBits - 1)))) >> roundBits; + } + + result = Vector128.Max(Vector128.Zero, Vector128.Min(Vector128.Create((short)byte.MaxValue), result)); + return result.AsUInt16(); + } + + /// + /// Removes the compound bias and final fractional precision from 256-bit unsigned lanes. + /// + public static Vector256 FinalizeIntermediate(Vector256 value, int roundBits, int roundOffset) + { + Vector256 result = (value - Vector256.Create((ushort)roundOffset)).AsInt16(); + if (roundBits != 0) + { + result = (result + Vector256.Create((short)(1 << (roundBits - 1)))) >> roundBits; + } + + result = Vector256.Max(Vector256.Zero, Vector256.Min(Vector256.Create((short)byte.MaxValue), result)); + return result.AsUInt16(); + } + + /// + /// Removes the compound bias and final fractional precision from 512-bit unsigned lanes. + /// + public static Vector512 FinalizeIntermediate(Vector512 value, int roundBits, int roundOffset) + { + Vector512 result = (value - Vector512.Create((ushort)roundOffset)).AsInt16(); + if (roundBits != 0) + { + result = (result + Vector512.Create((short)(1 << (roundBits - 1)))) >> roundBits; + } + + result = Vector512.Max(Vector512.Zero, Vector512.Min(Vector512.Create((short)byte.MaxValue), result)); + return result.AsUInt16(); + } + + /// + /// Removes compound bias and fractional precision from 128-bit high-bit-depth lanes. + /// + public static Vector128 FinalizeHighBitDepthIntermediate( + Vector128 value, + int roundBits, + int roundOffset, + int maximum) + { + Vector128 result = (value - Vector128.Create((ushort)roundOffset)).AsInt16(); + if (roundBits != 0) + { + result = (result + Vector128.Create((short)(1 << (roundBits - 1)))) >> roundBits; + } + + result = Vector128.Max(Vector128.Zero, Vector128.Min(Vector128.Create((short)maximum), result)); + return result.AsUInt16(); + } + + /// + /// Removes compound bias and fractional precision from 256-bit high-bit-depth lanes. + /// + public static Vector256 FinalizeHighBitDepthIntermediate( + Vector256 value, + int roundBits, + int roundOffset, + int maximum) + { + Vector256 result = (value - Vector256.Create((ushort)roundOffset)).AsInt16(); + if (roundBits != 0) + { + result = (result + Vector256.Create((short)(1 << (roundBits - 1)))) >> roundBits; + } + + result = Vector256.Max(Vector256.Zero, Vector256.Min(Vector256.Create((short)maximum), result)); + return result.AsUInt16(); + } + + /// + /// Removes compound bias and fractional precision from 512-bit high-bit-depth lanes. + /// + public static Vector512 FinalizeHighBitDepthIntermediate( + Vector512 value, + int roundBits, + int roundOffset, + int maximum) + { + Vector512 result = (value - Vector512.Create((ushort)roundOffset)).AsInt16(); + if (roundBits != 0) + { + result = (result + Vector512.Create((short)(1 << (roundBits - 1)))) >> roundBits; + } + + result = Vector512.Max(Vector512.Zero, Vector512.Min(Vector512.Create((short)maximum), result)); + return result.AsUInt16(); + } + + /// + /// Removes the compound bias and final fractional precision from 128-bit widened lanes. + /// + public static Vector128 FinalizeIntermediate(Vector128 value, int roundBits, int roundOffset) + { + Vector128 result = value - Vector128.Create(roundOffset); + if (roundBits != 0) + { + result = (result + Vector128.Create(1 << (roundBits - 1))) >> roundBits; + } + + return Vector128.Max(Vector128.Zero, Vector128.Min(Vector128.Create((int)byte.MaxValue), result)); + } + + /// + /// Removes the compound bias and final fractional precision from 256-bit widened lanes. + /// + public static Vector256 FinalizeIntermediate(Vector256 value, int roundBits, int roundOffset) + { + Vector256 result = value - Vector256.Create(roundOffset); + if (roundBits != 0) + { + result = (result + Vector256.Create(1 << (roundBits - 1))) >> roundBits; + } + + return Vector256.Max(Vector256.Zero, Vector256.Min(Vector256.Create((int)byte.MaxValue), result)); + } + + /// + /// Removes the compound bias and final fractional precision from 512-bit widened lanes. + /// + public static Vector512 FinalizeIntermediate(Vector512 value, int roundBits, int roundOffset) + { + Vector512 result = value - Vector512.Create(roundOffset); + if (roundBits != 0) + { + result = (result + Vector512.Create(1 << (roundBits - 1))) >> roundBits; + } + + return Vector512.Max(Vector512.Zero, Vector512.Min(Vector512.Create((int)byte.MaxValue), result)); + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1CompoundInterPredictor.Operator.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1CompoundInterPredictor.Operator.cs new file mode 100644 index 0000000000..9bef8f2050 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1CompoundInterPredictor.Operator.cs @@ -0,0 +1,586 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.Intrinsics; + +using static SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter.Av1TranslationalInterPredictor; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; + +/// +/// Defines biased compound-prediction conversion arithmetic. +/// +internal static partial class Av1CompoundInterPredictor +{ + /// + /// Defines biased compound-prediction conversion for scalar and SIMD lane groups. + /// + private interface IAv1CompoundPredictionOperator + { + /// + /// Converts one integer-position sample to the compound intermediate representation. + /// + /// The source sample. + /// The final reconstruction shift. + /// The compound intermediate bias. + /// The biased compound intermediate. + public static abstract ushort Copy(byte sample, int roundBits, int roundOffset); + + /// + /// Converts one high-bit-depth integer-position sample to the compound intermediate representation. + /// + /// The source sample. + /// The final reconstruction shift. + /// The compound intermediate bias. + /// The biased compound intermediate. + public static abstract ushort CopyHighBitDepth(ushort sample, int roundBits, int roundOffset); + + /// + /// Converts 128 bits of high-bit-depth integer-position samples to compound intermediates. + /// + /// The source samples. + /// The final reconstruction shift. + /// The compound intermediate bias. + /// The biased compound intermediates. + public static abstract Vector128 CopyHighBitDepth( + Vector128 samples, + int roundBits, + int roundOffset); + + /// + /// Converts 256 bits of high-bit-depth integer-position samples to compound intermediates. + /// + /// The source samples. + /// The final reconstruction shift. + /// The compound intermediate bias. + /// The biased compound intermediates. + public static abstract Vector256 CopyHighBitDepth( + Vector256 samples, + int roundBits, + int roundOffset); + + /// + /// Converts 512 bits of high-bit-depth integer-position samples to compound intermediates. + /// + /// The source samples. + /// The final reconstruction shift. + /// The compound intermediate bias. + /// The biased compound intermediates. + public static abstract Vector512 CopyHighBitDepth( + Vector512 samples, + int roundBits, + int roundOffset); + + /// + /// Converts 128 bits of integer-position samples to compound intermediates. + /// + /// The source samples. + /// The final reconstruction shift. + /// The compound intermediate bias. + /// Receives the lower widened intermediates. + /// Receives the upper widened intermediates. + public static abstract void Copy( + Vector128 samples, + int roundBits, + int roundOffset, + out Vector128 lower, + out Vector128 upper); + + /// + /// Converts 256 bits of integer-position samples to compound intermediates. + /// + /// The source samples. + /// The final reconstruction shift. + /// The compound intermediate bias. + /// Receives the lower widened intermediates. + /// Receives the upper widened intermediates. + public static abstract void Copy( + Vector256 samples, + int roundBits, + int roundOffset, + out Vector256 lower, + out Vector256 upper); + + /// + /// Converts 512 bits of integer-position samples to compound intermediates. + /// + /// The source samples. + /// The final reconstruction shift. + /// The compound intermediate bias. + /// Receives the lower widened intermediates. + /// Receives the upper widened intermediates. + public static abstract void Copy( + Vector512 samples, + int roundBits, + int roundOffset, + out Vector512 lower, + out Vector512 upper); + + /// + /// Applies direct-filter rounding and bias to one convolution result. + /// + /// The convolution result. + /// The shift applied before rounding. + /// The rounding shift. + /// The compound intermediate bias. + /// The biased compound intermediate. + public static abstract ushort PrepareDirect(int result, int preShift, int round, int roundOffset); + + /// + /// Applies direct-filter rounding and bias to 128-bit widened convolution results. + /// + /// The lower convolution results. + /// The upper convolution results. + /// The shift applied before rounding. + /// The rounding shift. + /// The compound intermediate bias. + /// The biased compound intermediates. + public static abstract Vector128 PrepareDirect( + Vector128 lower, + Vector128 upper, + int preShift, + int round, + int roundOffset); + + /// + /// Applies direct-filter rounding and bias to 256-bit widened convolution results. + /// + /// The lower convolution results. + /// The upper convolution results. + /// The shift applied before rounding. + /// The rounding shift. + /// The compound intermediate bias. + /// The biased compound intermediates. + public static abstract Vector256 PrepareDirect( + Vector256 lower, + Vector256 upper, + int preShift, + int round, + int roundOffset); + + /// + /// Applies direct-filter rounding and bias to 512-bit widened convolution results. + /// + /// The lower convolution results. + /// The upper convolution results. + /// The shift applied before rounding. + /// The rounding shift. + /// The compound intermediate bias. + /// The biased compound intermediates. + public static abstract Vector512 PrepareDirect( + Vector512 lower, + Vector512 upper, + int preShift, + int round, + int roundOffset); + + /// + /// Applies first-pass compound rounding to one biased horizontal convolution result. + /// + /// The biased horizontal convolution result. + /// The rounded intermediate. + public static abstract short PrepareHorizontal(int result); + + /// + /// Applies first-pass compound rounding to 128-bit widened horizontal convolution results. + /// + /// The lower convolution results. + /// The upper convolution results. + /// The rounded intermediates. + public static abstract Vector128 PrepareHorizontal(Vector128 lower, Vector128 upper); + + /// + /// Applies first-pass compound rounding to 256-bit widened horizontal convolution results. + /// + /// The lower convolution results. + /// The upper convolution results. + /// The rounded intermediates. + public static abstract Vector256 PrepareHorizontal(Vector256 lower, Vector256 upper); + + /// + /// Applies first-pass compound rounding to 512-bit widened horizontal convolution results. + /// + /// The lower convolution results. + /// The upper convolution results. + /// The rounded intermediates. + public static abstract Vector512 PrepareHorizontal(Vector512 lower, Vector512 upper); + + /// + /// Applies first-pass compound rounding to one biased high-bit-depth horizontal convolution result. + /// + /// The horizontal convolution result. + /// The bit-depth-dependent horizontal bias. + /// The bit-depth-dependent first-pass shift. + /// The rounded intermediate. + public static abstract short PrepareHighBitDepthHorizontal(int result, int bias, int round); + + /// + /// Applies first-pass compound rounding to 128-bit widened high-bit-depth horizontal results. + /// + /// The lower convolution results. + /// The upper convolution results. + /// The bit-depth-dependent horizontal bias. + /// The bit-depth-dependent first-pass shift. + /// The rounded intermediates. + public static abstract Vector128 PrepareHighBitDepthHorizontal( + Vector128 lower, + Vector128 upper, + int bias, + int round); + + /// + /// Applies first-pass compound rounding to 256-bit widened high-bit-depth horizontal results. + /// + /// The lower convolution results. + /// The upper convolution results. + /// The bit-depth-dependent horizontal bias. + /// The bit-depth-dependent first-pass shift. + /// The rounded intermediates. + public static abstract Vector256 PrepareHighBitDepthHorizontal( + Vector256 lower, + Vector256 upper, + int bias, + int round); + + /// + /// Applies first-pass compound rounding to 512-bit widened high-bit-depth horizontal results. + /// + /// The lower convolution results. + /// The upper convolution results. + /// The bit-depth-dependent horizontal bias. + /// The bit-depth-dependent first-pass shift. + /// The rounded intermediates. + public static abstract Vector512 PrepareHighBitDepthHorizontal( + Vector512 lower, + Vector512 upper, + int bias, + int round); + + /// + /// Applies second-pass compound rounding to one biased vertical convolution result. + /// + /// The biased vertical convolution result. + /// The compound intermediate. + public static abstract ushort PrepareVertical(int result); + + /// + /// Applies second-pass compound rounding to 128-bit widened vertical convolution results. + /// + /// The lower convolution results. + /// The upper convolution results. + /// The compound intermediates. + public static abstract Vector128 PrepareVertical(Vector128 lower, Vector128 upper); + + /// + /// Applies second-pass compound rounding to 256-bit widened vertical convolution results. + /// + /// The lower convolution results. + /// The upper convolution results. + /// The compound intermediates. + public static abstract Vector256 PrepareVertical(Vector256 lower, Vector256 upper); + + /// + /// Applies second-pass compound rounding to 512-bit widened vertical convolution results. + /// + /// The lower convolution results. + /// The upper convolution results. + /// The compound intermediates. + public static abstract Vector512 PrepareVertical(Vector512 lower, Vector512 upper); + + /// + /// Applies second-pass compound rounding to one biased high-bit-depth vertical convolution result. + /// + /// The vertical convolution result. + /// The bit-depth-dependent vertical bias. + /// The compound intermediate. + public static abstract ushort PrepareHighBitDepthVertical(int result, int bias); + + /// + /// Applies second-pass compound rounding to 128-bit widened high-bit-depth vertical results. + /// + /// The lower convolution results. + /// The upper convolution results. + /// The bit-depth-dependent vertical bias. + /// The compound intermediates. + public static abstract Vector128 PrepareHighBitDepthVertical( + Vector128 lower, + Vector128 upper, + int bias); + + /// + /// Applies second-pass compound rounding to 256-bit widened high-bit-depth vertical results. + /// + /// The lower convolution results. + /// The upper convolution results. + /// The bit-depth-dependent vertical bias. + /// The compound intermediates. + public static abstract Vector256 PrepareHighBitDepthVertical( + Vector256 lower, + Vector256 upper, + int bias); + + /// + /// Applies second-pass compound rounding to 512-bit widened high-bit-depth vertical results. + /// + /// The lower convolution results. + /// The upper convolution results. + /// The bit-depth-dependent vertical bias. + /// The compound intermediates. + public static abstract Vector512 PrepareHighBitDepthVertical( + Vector512 lower, + Vector512 upper, + int bias); + } + + /// + /// Implements AV1 compound-prediction conversion for scalar and SIMD lane groups. + /// + private readonly struct CompoundPredictionOperator : IAv1CompoundPredictionOperator + { + private const int HorizontalBias = 1 << (8 + FilterBits - 1); + private const int VerticalBias = 1 << (8 + (2 * FilterBits) - Round0Bits); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static ushort Copy(byte sample, int roundBits, int roundOffset) + => (ushort)((sample << roundBits) + roundOffset); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static ushort CopyHighBitDepth(ushort sample, int roundBits, int roundOffset) + => (ushort)((sample << roundBits) + roundOffset); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector128 CopyHighBitDepth( + Vector128 samples, + int roundBits, + int roundOffset) + => (samples << roundBits) + Vector128.Create((ushort)roundOffset); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector256 CopyHighBitDepth( + Vector256 samples, + int roundBits, + int roundOffset) + => (samples << roundBits) + Vector256.Create((ushort)roundOffset); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector512 CopyHighBitDepth( + Vector512 samples, + int roundBits, + int roundOffset) + => (samples << roundBits) + Vector512.Create((ushort)roundOffset); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void Copy( + Vector128 samples, + int roundBits, + int roundOffset, + out Vector128 lower, + out Vector128 upper) + { + Vector128 offset = Vector128.Create((ushort)roundOffset); + lower = (Vector128.WidenLower(samples) << roundBits) + offset; + upper = (Vector128.WidenUpper(samples) << roundBits) + offset; + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void Copy( + Vector256 samples, + int roundBits, + int roundOffset, + out Vector256 lower, + out Vector256 upper) + { + Vector256 offset = Vector256.Create((ushort)roundOffset); + lower = (Vector256.WidenLower(samples) << roundBits) + offset; + upper = (Vector256.WidenUpper(samples) << roundBits) + offset; + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void Copy( + Vector512 samples, + int roundBits, + int roundOffset, + out Vector512 lower, + out Vector512 upper) + { + Vector512 offset = Vector512.Create((ushort)roundOffset); + lower = (Vector512.WidenLower(samples) << roundBits) + offset; + upper = (Vector512.WidenUpper(samples) << roundBits) + offset; + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static ushort PrepareDirect(int result, int preShift, int round, int roundOffset) + => (ushort)(RoundPowerOfTwo(result << preShift, round) + roundOffset); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector128 PrepareDirect( + Vector128 lower, + Vector128 upper, + int preShift, + int round, + int roundOffset) + => Av1NonDirectionalIntraPredictorBase.Narrow( + RoundPowerOfTwo(lower << preShift, round) + Vector128.Create(roundOffset), + RoundPowerOfTwo(upper << preShift, round) + Vector128.Create(roundOffset)).AsUInt16(); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector256 PrepareDirect( + Vector256 lower, + Vector256 upper, + int preShift, + int round, + int roundOffset) + => Av1NonDirectionalIntraPredictorBase.Narrow( + RoundPowerOfTwo(lower << preShift, round) + Vector256.Create(roundOffset), + RoundPowerOfTwo(upper << preShift, round) + Vector256.Create(roundOffset)).AsUInt16(); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector512 PrepareDirect( + Vector512 lower, + Vector512 upper, + int preShift, + int round, + int roundOffset) + => Av1NonDirectionalIntraPredictorBase.Narrow( + RoundPowerOfTwo(lower << preShift, round) + Vector512.Create(roundOffset), + RoundPowerOfTwo(upper << preShift, round) + Vector512.Create(roundOffset)).AsUInt16(); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static short PrepareHorizontal(int result) + => (short)RoundPowerOfTwo(HorizontalBias + result, Round0Bits); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector128 PrepareHorizontal(Vector128 lower, Vector128 upper) + => Av1NonDirectionalIntraPredictorBase.Narrow( + RoundPowerOfTwo(lower + Vector128.Create(HorizontalBias), Round0Bits), + RoundPowerOfTwo(upper + Vector128.Create(HorizontalBias), Round0Bits)); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector256 PrepareHorizontal(Vector256 lower, Vector256 upper) + => Av1NonDirectionalIntraPredictorBase.Narrow( + RoundPowerOfTwo(lower + Vector256.Create(HorizontalBias), Round0Bits), + RoundPowerOfTwo(upper + Vector256.Create(HorizontalBias), Round0Bits)); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector512 PrepareHorizontal(Vector512 lower, Vector512 upper) + => Av1NonDirectionalIntraPredictorBase.Narrow( + RoundPowerOfTwo(lower + Vector512.Create(HorizontalBias), Round0Bits), + RoundPowerOfTwo(upper + Vector512.Create(HorizontalBias), Round0Bits)); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static short PrepareHighBitDepthHorizontal(int result, int bias, int round) + => (short)RoundPowerOfTwo(bias + result, round); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector128 PrepareHighBitDepthHorizontal( + Vector128 lower, + Vector128 upper, + int bias, + int round) + => Av1NonDirectionalIntraPredictorBase.Narrow( + RoundPowerOfTwo(lower + Vector128.Create(bias), round), + RoundPowerOfTwo(upper + Vector128.Create(bias), round)); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector256 PrepareHighBitDepthHorizontal( + Vector256 lower, + Vector256 upper, + int bias, + int round) + => Av1NonDirectionalIntraPredictorBase.Narrow( + RoundPowerOfTwo(lower + Vector256.Create(bias), round), + RoundPowerOfTwo(upper + Vector256.Create(bias), round)); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector512 PrepareHighBitDepthHorizontal( + Vector512 lower, + Vector512 upper, + int bias, + int round) + => Av1NonDirectionalIntraPredictorBase.Narrow( + RoundPowerOfTwo(lower + Vector512.Create(bias), round), + RoundPowerOfTwo(upper + Vector512.Create(bias), round)); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static ushort PrepareVertical(int result) + => (ushort)RoundPowerOfTwo(VerticalBias + result, CompoundRound1Bits); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector128 PrepareVertical(Vector128 lower, Vector128 upper) + => Av1NonDirectionalIntraPredictorBase.Narrow( + RoundPowerOfTwo(lower + Vector128.Create(VerticalBias), CompoundRound1Bits), + RoundPowerOfTwo(upper + Vector128.Create(VerticalBias), CompoundRound1Bits)).AsUInt16(); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector256 PrepareVertical(Vector256 lower, Vector256 upper) + => Av1NonDirectionalIntraPredictorBase.Narrow( + RoundPowerOfTwo(lower + Vector256.Create(VerticalBias), CompoundRound1Bits), + RoundPowerOfTwo(upper + Vector256.Create(VerticalBias), CompoundRound1Bits)).AsUInt16(); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector512 PrepareVertical(Vector512 lower, Vector512 upper) + => Av1NonDirectionalIntraPredictorBase.Narrow( + RoundPowerOfTwo(lower + Vector512.Create(VerticalBias), CompoundRound1Bits), + RoundPowerOfTwo(upper + Vector512.Create(VerticalBias), CompoundRound1Bits)).AsUInt16(); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static ushort PrepareHighBitDepthVertical(int result, int bias) + => (ushort)RoundPowerOfTwo(bias + result, CompoundRound1Bits); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector128 PrepareHighBitDepthVertical( + Vector128 lower, + Vector128 upper, + int bias) + => Av1NonDirectionalIntraPredictorBase.Narrow( + RoundPowerOfTwo(lower + Vector128.Create(bias), CompoundRound1Bits), + RoundPowerOfTwo(upper + Vector128.Create(bias), CompoundRound1Bits)).AsUInt16(); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector256 PrepareHighBitDepthVertical( + Vector256 lower, + Vector256 upper, + int bias) + => Av1NonDirectionalIntraPredictorBase.Narrow( + RoundPowerOfTwo(lower + Vector256.Create(bias), CompoundRound1Bits), + RoundPowerOfTwo(upper + Vector256.Create(bias), CompoundRound1Bits)).AsUInt16(); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector512 PrepareHighBitDepthVertical( + Vector512 lower, + Vector512 upper, + int bias) + => Av1NonDirectionalIntraPredictorBase.Narrow( + RoundPowerOfTwo(lower + Vector512.Create(bias), CompoundRound1Bits), + RoundPowerOfTwo(upper + Vector512.Create(bias), CompoundRound1Bits)).AsUInt16(); + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1CompoundInterPredictor.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1CompoundInterPredictor.cs new file mode 100644 index 0000000000..27baf014df --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1CompoundInterPredictor.cs @@ -0,0 +1,1168 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.InteropServices; +using System.Runtime.Intrinsics; + +using static SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter.Av1TranslationalInterPredictor; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; + +/// +/// Produces the biased high-precision intermediates required by compound inter prediction. +/// +internal static partial class Av1CompoundInterPredictor +{ + /// + /// The second-round shift retained by every compound convolution path. + /// + internal const int CompoundRound1Bits = 7; + + /// + /// The fixed-point precision used by AV1 distance weights. + /// + internal const int DistanceWeightBits = 4; + + /// + /// The fixed-point precision used by AV1 compound masks. + /// + internal const int MaskWeightBits = 6; + + /// + /// The inclusive upper bound for an AV1 compound-mask alpha value. + /// + internal const int MaximumMaskAlpha = 1 << MaskWeightBits; + + /// + /// Reconstructs one 8-bit translational reference into AV1's unsigned compound intermediate format. + /// + public static void PredictCompound( + ReadOnlySpan source, + int sourceStride, + int sourceOrigin, + Span destination, + int destinationStride, + int width, + int height, + Av1InterpolationFilter horizontalFilter, + Av1InterpolationFilter verticalFilter, + int horizontalPhase, + int verticalPhase, + Span scratch) + => PredictCompound( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + horizontalFilter, + verticalFilter, + horizontalPhase, + verticalPhase, + scratch, + useSimd: true); + + /// + /// Reconstructs one high-bit-depth translational reference into AV1's unsigned compound intermediate format. + /// + public static void PredictCompound( + ReadOnlySpan source, + int sourceStride, + int sourceOrigin, + Span destination, + int destinationStride, + int width, + int height, + Av1InterpolationFilter horizontalFilter, + Av1InterpolationFilter verticalFilter, + int horizontalPhase, + int verticalPhase, + int bitDepth, + Span scratch) + => PredictCompound( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + horizontalFilter, + verticalFilter, + horizontalPhase, + verticalPhase, + bitDepth, + scratch, + useSimd: true); + + /// + /// Executes one closed compound-prediction conversion operator. + /// + /// The compound-prediction conversion operator. + private static void PredictCompound( + ReadOnlySpan source, + int sourceStride, + int sourceOrigin, + Span destination, + int destinationStride, + int width, + int height, + Av1InterpolationFilter horizontalFilter, + Av1InterpolationFilter verticalFilter, + int horizontalPhase, + int verticalPhase, + Span scratch, + bool useSimd) + where TOperator : struct, IAv1CompoundPredictionOperator + { + ReadOnlySpan horizontalCoefficients = GetCompoundCoefficients(horizontalFilter, horizontalPhase, width <= 4); + ReadOnlySpan verticalCoefficients = GetCompoundCoefficients(verticalFilter, verticalPhase, height <= 4); + int roundBits = (2 * FilterBits) - Round0Bits - CompoundRound1Bits; + int offsetBits = 8 + (2 * FilterBits) - Round0Bits; + int roundOffset = (1 << (offsetBits - CompoundRound1Bits)) + + (1 << (offsetBits - CompoundRound1Bits - 1)); + + if (horizontalPhase == 0 && verticalPhase == 0) + { + CopyCompound( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + roundBits, + roundOffset, + useSimd); + + return; + } + + if (verticalPhase == 0) + { + GetEffectiveKernel(horizontalCoefficients, out int firstCoefficient, out int tapCount); + FilterCompoundDirect( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + horizontalCoefficients[firstCoefficient..], + tapCount, + firstCoefficient - 3, + tapStride: 1, + preShift: 0, + round: Round0Bits, + roundOffset, + useSimd); + + return; + } + + if (horizontalPhase == 0) + { + GetEffectiveKernel(verticalCoefficients, out int firstCoefficient, out int tapCount); + FilterCompoundDirect( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + verticalCoefficients[firstCoefficient..], + tapCount, + (firstCoefficient - 3) * sourceStride, + sourceStride, + FilterBits - Round0Bits, + CompoundRound1Bits, + roundOffset, + useSimd); + + return; + } + + GetEffectiveKernel(horizontalCoefficients, out int firstHorizontalCoefficient, out int horizontalTapCount); + GetEffectiveKernel(verticalCoefficients, out int firstVerticalCoefficient, out int verticalTapCount); + FilterCompound2D( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + horizontalCoefficients[firstHorizontalCoefficient..], + horizontalTapCount, + firstHorizontalCoefficient - 3, + verticalCoefficients[firstVerticalCoefficient..], + verticalTapCount, + firstVerticalCoefficient - 3, + scratch, + useSimd); + } + + /// + /// Executes one closed high-bit-depth compound-prediction conversion operator. + /// + /// The compound-prediction conversion operator. + private static void PredictCompound( + ReadOnlySpan source, + int sourceStride, + int sourceOrigin, + Span destination, + int destinationStride, + int width, + int height, + Av1InterpolationFilter horizontalFilter, + Av1InterpolationFilter verticalFilter, + int horizontalPhase, + int verticalPhase, + int bitDepth, + Span scratch, + bool useSimd) + where TOperator : struct, IAv1CompoundPredictionOperator + { + ReadOnlySpan horizontalCoefficients = GetCompoundCoefficients(horizontalFilter, horizontalPhase, width <= 4); + ReadOnlySpan verticalCoefficients = GetCompoundCoefficients(verticalFilter, verticalPhase, height <= 4); + + // The reference decoder increases the first-round shift only for 12-bit input. This keeps the signed horizontal + // intermediate within sixteen bits while preserving the same total Q14 convolution precision. + int intermediateRange = bitDepth + FilterBits - Round0Bits + 2; + int round0 = Round0Bits + Math.Max(intermediateRange - 16, 0); + int roundBits = (2 * FilterBits) - round0 - CompoundRound1Bits; + int offsetBits = bitDepth + (2 * FilterBits) - round0; + int roundOffset = (1 << (offsetBits - CompoundRound1Bits)) + + (1 << (offsetBits - CompoundRound1Bits - 1)); + + if (horizontalPhase == 0 && verticalPhase == 0) + { + CopyCompound( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + roundBits, + roundOffset, + useSimd); + + return; + } + + if (verticalPhase == 0) + { + GetEffectiveKernel(horizontalCoefficients, out int firstCoefficient, out int tapCount); + FilterCompoundDirect( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + horizontalCoefficients[firstCoefficient..], + tapCount, + firstCoefficient - 3, + tapStride: 1, + preShift: 0, + round: round0, + roundOffset, + useSimd); + + return; + } + + if (horizontalPhase == 0) + { + GetEffectiveKernel(verticalCoefficients, out int firstCoefficient, out int tapCount); + FilterCompoundDirect( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + verticalCoefficients[firstCoefficient..], + tapCount, + (firstCoefficient - 3) * sourceStride, + sourceStride, + FilterBits - round0, + CompoundRound1Bits, + roundOffset, + useSimd); + + return; + } + + GetEffectiveKernel(horizontalCoefficients, out int firstHorizontalCoefficient, out int horizontalTapCount); + GetEffectiveKernel(verticalCoefficients, out int firstVerticalCoefficient, out int verticalTapCount); + FilterCompound2D( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + horizontalCoefficients[firstHorizontalCoefficient..], + horizontalTapCount, + firstHorizontalCoefficient - 3, + verticalCoefficients[firstVerticalCoefficient..], + verticalTapCount, + firstVerticalCoefficient - 3, + bitDepth, + round0, + scratch, + useSimd); + } + + /// + /// Reconstructs one compound intermediate without explicit hardware intrinsics. + /// + public static void PredictCompoundScalar( + ReadOnlySpan source, + int sourceStride, + int sourceOrigin, + Span destination, + int destinationStride, + int width, + int height, + Av1InterpolationFilter horizontalFilter, + Av1InterpolationFilter verticalFilter, + int horizontalPhase, + int verticalPhase, + Span scratch) + => PredictCompound( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + horizontalFilter, + verticalFilter, + horizontalPhase, + verticalPhase, + scratch, + useSimd: false); + + /// + /// Reconstructs one high-bit-depth compound intermediate without explicit hardware intrinsics. + /// + public static void PredictCompoundScalar( + ReadOnlySpan source, + int sourceStride, + int sourceOrigin, + Span destination, + int destinationStride, + int width, + int height, + Av1InterpolationFilter horizontalFilter, + Av1InterpolationFilter verticalFilter, + int horizontalPhase, + int verticalPhase, + int bitDepth, + Span scratch) + => PredictCompound( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + horizontalFilter, + verticalFilter, + horizontalPhase, + verticalPhase, + bitDepth, + scratch, + useSimd: false); + + /// + /// Copies integer-position samples through one closed compound-prediction operator. + /// + /// The compound-prediction conversion operator. + private static void CopyCompound( + ReadOnlySpan source, + int sourceStride, + int sourceOrigin, + Span destination, + int destinationStride, + int width, + int height, + int roundBits, + int roundOffset, + bool useSimd) + where TOperator : struct, IAv1CompoundPredictionOperator + { + ref byte sourceBase = ref Unsafe.Add(ref MemoryMarshal.GetReference(source), sourceOrigin); + ref ushort destinationBase = ref MemoryMarshal.GetReference(destination); + + for (int row = 0; row < height; row++) + { + ref byte sourceRow = ref Unsafe.Add(ref sourceBase, row * sourceStride); + ref ushort destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + int column = 0; + + if (useSimd && Vector512.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector512Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector512.Count) + { + Vector512 samples = Vector512.LoadUnsafe(ref sourceRow, (nuint)column); + TOperator.Copy(samples, roundBits, roundOffset, out Vector512 lower, out Vector512 upper); + lower.StoreUnsafe(ref destinationRow, (nuint)column); + upper.StoreUnsafe(ref destinationRow, (nuint)(column + Vector512.Count)); + } + } + + if (useSimd && Vector256.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector256Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector256.Count) + { + Vector256 samples = Vector256.LoadUnsafe(ref sourceRow, (nuint)column); + TOperator.Copy(samples, roundBits, roundOffset, out Vector256 lower, out Vector256 upper); + lower.StoreUnsafe(ref destinationRow, (nuint)column); + upper.StoreUnsafe(ref destinationRow, (nuint)(column + Vector256.Count)); + } + } + + if (useSimd && Vector128.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector128Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector128.Count) + { + Vector128 samples = Vector128.LoadUnsafe(ref sourceRow, (nuint)column); + TOperator.Copy(samples, roundBits, roundOffset, out Vector128 lower, out Vector128 upper); + lower.StoreUnsafe(ref destinationRow, (nuint)column); + upper.StoreUnsafe(ref destinationRow, (nuint)(column + Vector128.Count)); + } + } + + for (; column < width; column++) + { + Unsafe.Add(ref destinationRow, column) = TOperator.Copy(Unsafe.Add(ref sourceRow, column), roundBits, roundOffset); + } + } + } + + /// + /// Copies high-bit-depth integer-position samples through one closed compound-prediction operator. + /// + /// The compound-prediction conversion operator. + private static void CopyCompound( + ReadOnlySpan source, + int sourceStride, + int sourceOrigin, + Span destination, + int destinationStride, + int width, + int height, + int roundBits, + int roundOffset, + bool useSimd) + where TOperator : struct, IAv1CompoundPredictionOperator + { + ref ushort sourceBase = ref Unsafe.Add(ref MemoryMarshal.GetReference(source), sourceOrigin); + ref ushort destinationBase = ref MemoryMarshal.GetReference(destination); + + for (int row = 0; row < height; row++) + { + ref ushort sourceRow = ref Unsafe.Add(ref sourceBase, row * sourceStride); + ref ushort destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + int column = 0; + + if (useSimd && Vector512.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector512Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector512.Count) + { + Vector512 samples = Vector512.LoadUnsafe(ref sourceRow, (nuint)column); + TOperator.CopyHighBitDepth(samples, roundBits, roundOffset) + .StoreUnsafe(ref destinationRow, (nuint)column); + } + } + + if (useSimd && Vector256.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector256Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector256.Count) + { + Vector256 samples = Vector256.LoadUnsafe(ref sourceRow, (nuint)column); + TOperator.CopyHighBitDepth(samples, roundBits, roundOffset) + .StoreUnsafe(ref destinationRow, (nuint)column); + } + } + + if (useSimd && Vector128.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector128Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector128.Count) + { + Vector128 samples = Vector128.LoadUnsafe(ref sourceRow, (nuint)column); + TOperator.CopyHighBitDepth(samples, roundBits, roundOffset) + .StoreUnsafe(ref destinationRow, (nuint)column); + } + } + + for (; column < width; column++) + { + Unsafe.Add(ref destinationRow, column) = TOperator.CopyHighBitDepth( + Unsafe.Add(ref sourceRow, column), + roundBits, + roundOffset); + } + } + } + + /// + /// Applies one compound convolution direction through one closed conversion operator. + /// + /// The compound-prediction conversion operator. + private static void FilterCompoundDirect( + ReadOnlySpan source, + int sourceStride, + int sourceOrigin, + Span destination, + int destinationStride, + int width, + int height, + ReadOnlySpan coefficients, + int tapCount, + int sourceOffset, + int tapStride, + int preShift, + int round, + int roundOffset, + bool useSimd) + where TOperator : struct, IAv1CompoundPredictionOperator + { + ref byte sourceBase = ref Unsafe.Add(ref MemoryMarshal.GetReference(source), sourceOrigin); + ref ushort destinationBase = ref MemoryMarshal.GetReference(destination); + ref short coefficientBase = ref MemoryMarshal.GetReference(coefficients); + + for (int row = 0; row < height; row++) + { + ref byte sourceRow = ref Unsafe.Add(ref sourceBase, (row * sourceStride) + sourceOffset); + ref ushort destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + int column = 0; + + if (useSimd && Vector512.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector512Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector512.Count) + { + Convolve( + ref sourceRow, + tapStride, + (nuint)column, + ref coefficientBase, + tapCount, + Vector512.Zero, + out Vector512 result0, + out Vector512 result1, + out Vector512 result2, + out Vector512 result3); + + TOperator.PrepareDirect(result0, result1, preShift, round, roundOffset) + .StoreUnsafe(ref destinationRow, (nuint)column); + + TOperator.PrepareDirect(result2, result3, preShift, round, roundOffset) + .StoreUnsafe(ref destinationRow, (nuint)(column + Vector512.Count)); + } + } + + if (useSimd && Vector256.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector256Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector256.Count) + { + Convolve( + ref sourceRow, + tapStride, + (nuint)column, + ref coefficientBase, + tapCount, + Vector256.Zero, + out Vector256 result0, + out Vector256 result1, + out Vector256 result2, + out Vector256 result3); + + TOperator.PrepareDirect(result0, result1, preShift, round, roundOffset) + .StoreUnsafe(ref destinationRow, (nuint)column); + + TOperator.PrepareDirect(result2, result3, preShift, round, roundOffset) + .StoreUnsafe(ref destinationRow, (nuint)(column + Vector256.Count)); + } + } + + if (useSimd && Vector128.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector128Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector128.Count) + { + Convolve( + ref sourceRow, + tapStride, + (nuint)column, + ref coefficientBase, + tapCount, + Vector128.Zero, + out Vector128 result0, + out Vector128 result1, + out Vector128 result2, + out Vector128 result3); + + TOperator.PrepareDirect(result0, result1, preShift, round, roundOffset) + .StoreUnsafe(ref destinationRow, (nuint)column); + + TOperator.PrepareDirect(result2, result3, preShift, round, roundOffset) + .StoreUnsafe(ref destinationRow, (nuint)(column + Vector128.Count)); + } + } + + for (; column < width; column++) + { + int result = ConvolveScalar( + ref Unsafe.Add(ref sourceRow, column), + tapStride, + ref coefficientBase, + tapCount); + + Unsafe.Add(ref destinationRow, column) = TOperator.PrepareDirect(result, preShift, round, roundOffset); + } + } + } + + /// + /// Applies one high-bit-depth compound convolution direction through one closed conversion operator. + /// + /// The compound-prediction conversion operator. + private static void FilterCompoundDirect( + ReadOnlySpan source, + int sourceStride, + int sourceOrigin, + Span destination, + int destinationStride, + int width, + int height, + ReadOnlySpan coefficients, + int tapCount, + int sourceOffset, + int tapStride, + int preShift, + int round, + int roundOffset, + bool useSimd) + where TOperator : struct, IAv1CompoundPredictionOperator + { + ref ushort sourceBase = ref Unsafe.Add(ref MemoryMarshal.GetReference(source), sourceOrigin); + ref ushort destinationBase = ref MemoryMarshal.GetReference(destination); + ref short coefficientBase = ref MemoryMarshal.GetReference(coefficients); + + for (int row = 0; row < height; row++) + { + ref ushort sourceRowUnsigned = ref Unsafe.Add(ref sourceBase, (row * sourceStride) + sourceOffset); + ref short sourceRow = ref Unsafe.As(ref sourceRowUnsigned); + ref ushort destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + int column = 0; + + if (useSimd && Vector512.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector512Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector512.Count) + { + Convolve( + ref sourceRow, + tapStride, + (nuint)column, + ref coefficientBase, + tapCount, + Vector512.Zero, + out Vector512 lower, + out Vector512 upper); + + TOperator.PrepareDirect(lower, upper, preShift, round, roundOffset) + .StoreUnsafe(ref destinationRow, (nuint)column); + } + } + + if (useSimd && Vector256.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector256Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector256.Count) + { + Convolve( + ref sourceRow, + tapStride, + (nuint)column, + ref coefficientBase, + tapCount, + Vector256.Zero, + out Vector256 lower, + out Vector256 upper); + + TOperator.PrepareDirect(lower, upper, preShift, round, roundOffset) + .StoreUnsafe(ref destinationRow, (nuint)column); + } + } + + if (useSimd && Vector128.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector128Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector128.Count) + { + Convolve( + ref sourceRow, + tapStride, + (nuint)column, + ref coefficientBase, + tapCount, + Vector128.Zero, + out Vector128 lower, + out Vector128 upper); + + TOperator.PrepareDirect(lower, upper, preShift, round, roundOffset) + .StoreUnsafe(ref destinationRow, (nuint)column); + } + } + + for (; column < width; column++) + { + int result = ConvolveScalar( + ref Unsafe.Add(ref sourceRowUnsigned, column), + tapStride, + ref coefficientBase, + tapCount); + + Unsafe.Add(ref destinationRow, column) = TOperator.PrepareDirect(result, preShift, round, roundOffset); + } + } + } + + /// + /// Applies separable compound convolution through caller-owned signed scratch. + /// + /// The compound-prediction conversion operator. + private static void FilterCompound2D( + ReadOnlySpan source, + int sourceStride, + int sourceOrigin, + Span destination, + int destinationStride, + int width, + int height, + ReadOnlySpan horizontalCoefficients, + int horizontalTapCount, + int horizontalSourceOffset, + ReadOnlySpan verticalCoefficients, + int verticalTapCount, + int verticalSourceOffset, + Span scratch, + bool useSimd) + where TOperator : struct, IAv1CompoundPredictionOperator + { + ref byte sourceBase = ref Unsafe.Add(ref MemoryMarshal.GetReference(source), sourceOrigin); + ref ushort destinationBase = ref MemoryMarshal.GetReference(destination); + ref short scratchBase = ref MemoryMarshal.GetReference(scratch); + ref short horizontalCoefficientBase = ref MemoryMarshal.GetReference(horizontalCoefficients); + ref short verticalCoefficientBase = ref MemoryMarshal.GetReference(verticalCoefficients); + int scratchStride = Math.Max(width, MinimumScratchStride); + int intermediateHeight = height + verticalTapCount - 1; + + // The horizontal pass retains Q7 precision in signed scratch. Each SIMD stage continues at the shared + // column offset so mixed-width rows need no padding stores and never cross the logical block edge. + for (int row = 0; row < intermediateHeight; row++) + { + ref byte sourceRow = ref Unsafe.Add( + ref sourceBase, + ((row + verticalSourceOffset) * sourceStride) + horizontalSourceOffset); + + ref short scratchRow = ref Unsafe.Add(ref scratchBase, row * scratchStride); + int column = 0; + + if (useSimd && Vector512.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector512Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector512.Count) + { + Convolve( + ref sourceRow, + 1, + (nuint)column, + ref horizontalCoefficientBase, + horizontalTapCount, + Vector512.Zero, + out Vector512 result0, + out Vector512 result1, + out Vector512 result2, + out Vector512 result3); + + TOperator.PrepareHorizontal(result0, result1).StoreUnsafe(ref scratchRow, (nuint)column); + TOperator.PrepareHorizontal(result2, result3) + .StoreUnsafe(ref scratchRow, (nuint)(column + Vector512.Count)); + } + } + + if (useSimd && Vector256.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector256Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector256.Count) + { + Convolve( + ref sourceRow, + 1, + (nuint)column, + ref horizontalCoefficientBase, + horizontalTapCount, + Vector256.Zero, + out Vector256 result0, + out Vector256 result1, + out Vector256 result2, + out Vector256 result3); + + TOperator.PrepareHorizontal(result0, result1).StoreUnsafe(ref scratchRow, (nuint)column); + TOperator.PrepareHorizontal(result2, result3) + .StoreUnsafe(ref scratchRow, (nuint)(column + Vector256.Count)); + } + } + + if (useSimd && Vector128.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector128Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector128.Count) + { + Convolve( + ref sourceRow, + 1, + (nuint)column, + ref horizontalCoefficientBase, + horizontalTapCount, + Vector128.Zero, + out Vector128 result0, + out Vector128 result1, + out Vector128 result2, + out Vector128 result3); + + TOperator.PrepareHorizontal(result0, result1).StoreUnsafe(ref scratchRow, (nuint)column); + TOperator.PrepareHorizontal(result2, result3) + .StoreUnsafe(ref scratchRow, (nuint)(column + Vector128.Count)); + } + } + + for (; column < width; column++) + { + int result = ConvolveScalar( + ref Unsafe.Add(ref sourceRow, column), + 1, + ref horizontalCoefficientBase, + horizontalTapCount); + + Unsafe.Add(ref scratchRow, column) = TOperator.PrepareHorizontal(result); + } + } + + for (int row = 0; row < height; row++) + { + ref short scratchRow = ref Unsafe.Add(ref scratchBase, row * scratchStride); + ref ushort destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + int column = 0; + + if (useSimd && Vector512.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector512Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector512.Count) + { + Convolve( + ref scratchRow, + scratchStride, + (nuint)column, + ref verticalCoefficientBase, + verticalTapCount, + Vector512.Zero, + out Vector512 lower, + out Vector512 upper); + + TOperator.PrepareVertical(lower, upper).StoreUnsafe(ref destinationRow, (nuint)column); + } + } + + if (useSimd && Vector256.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector256Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector256.Count) + { + Convolve( + ref scratchRow, + scratchStride, + (nuint)column, + ref verticalCoefficientBase, + verticalTapCount, + Vector256.Zero, + out Vector256 lower, + out Vector256 upper); + + TOperator.PrepareVertical(lower, upper).StoreUnsafe(ref destinationRow, (nuint)column); + } + } + + if (useSimd && Vector128.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector128Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector128.Count) + { + Convolve( + ref scratchRow, + scratchStride, + (nuint)column, + ref verticalCoefficientBase, + verticalTapCount, + Vector128.Zero, + out Vector128 lower, + out Vector128 upper); + + TOperator.PrepareVertical(lower, upper).StoreUnsafe(ref destinationRow, (nuint)column); + } + } + + for (; column < width; column++) + { + int result = ConvolveScalar( + ref Unsafe.Add(ref scratchRow, column), + scratchStride, + ref verticalCoefficientBase, + verticalTapCount); + + Unsafe.Add(ref destinationRow, column) = TOperator.PrepareVertical(result); + } + } + } + + /// + /// Applies separable high-bit-depth compound convolution through caller-owned signed scratch. + /// + /// The compound-prediction conversion operator. + private static void FilterCompound2D( + ReadOnlySpan source, + int sourceStride, + int sourceOrigin, + Span destination, + int destinationStride, + int width, + int height, + ReadOnlySpan horizontalCoefficients, + int horizontalTapCount, + int horizontalSourceOffset, + ReadOnlySpan verticalCoefficients, + int verticalTapCount, + int verticalSourceOffset, + int bitDepth, + int round0, + Span scratch, + bool useSimd) + where TOperator : struct, IAv1CompoundPredictionOperator + { + ref ushort sourceBase = ref Unsafe.Add(ref MemoryMarshal.GetReference(source), sourceOrigin); + ref ushort destinationBase = ref MemoryMarshal.GetReference(destination); + ref short scratchBase = ref MemoryMarshal.GetReference(scratch); + ref short horizontalCoefficientBase = ref MemoryMarshal.GetReference(horizontalCoefficients); + ref short verticalCoefficientBase = ref MemoryMarshal.GetReference(verticalCoefficients); + int scratchStride = Math.Max(width, MinimumScratchStride); + int intermediateHeight = height + verticalTapCount - 1; + int horizontalBias = 1 << (bitDepth + FilterBits - 1); + int verticalBias = 1 << (bitDepth + (2 * FilterBits) - round0); + + // High-bit-depth input is still below short.MaxValue. Reinterpreting the source lets the shared signed + // widening kernels apply negative filter coefficients without copying or allocating a conversion buffer. + for (int row = 0; row < intermediateHeight; row++) + { + ref ushort sourceRowUnsigned = ref Unsafe.Add( + ref sourceBase, + ((row + verticalSourceOffset) * sourceStride) + horizontalSourceOffset); + + ref short sourceRow = ref Unsafe.As(ref sourceRowUnsigned); + ref short scratchRow = ref Unsafe.Add(ref scratchBase, row * scratchStride); + int column = 0; + + if (useSimd && Vector512.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector512Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector512.Count) + { + Convolve( + ref sourceRow, + 1, + (nuint)column, + ref horizontalCoefficientBase, + horizontalTapCount, + Vector512.Zero, + out Vector512 lower, + out Vector512 upper); + + TOperator.PrepareHighBitDepthHorizontal(lower, upper, horizontalBias, round0) + .StoreUnsafe(ref scratchRow, (nuint)column); + } + } + + if (useSimd && Vector256.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector256Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector256.Count) + { + Convolve( + ref sourceRow, + 1, + (nuint)column, + ref horizontalCoefficientBase, + horizontalTapCount, + Vector256.Zero, + out Vector256 lower, + out Vector256 upper); + + TOperator.PrepareHighBitDepthHorizontal(lower, upper, horizontalBias, round0) + .StoreUnsafe(ref scratchRow, (nuint)column); + } + } + + if (useSimd && Vector128.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector128Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector128.Count) + { + Convolve( + ref sourceRow, + 1, + (nuint)column, + ref horizontalCoefficientBase, + horizontalTapCount, + Vector128.Zero, + out Vector128 lower, + out Vector128 upper); + + TOperator.PrepareHighBitDepthHorizontal(lower, upper, horizontalBias, round0) + .StoreUnsafe(ref scratchRow, (nuint)column); + } + } + + for (; column < width; column++) + { + int result = ConvolveScalar( + ref Unsafe.Add(ref sourceRowUnsigned, column), + 1, + ref horizontalCoefficientBase, + horizontalTapCount); + + Unsafe.Add(ref scratchRow, column) = + TOperator.PrepareHighBitDepthHorizontal(result, horizontalBias, round0); + } + } + + for (int row = 0; row < height; row++) + { + ref short scratchRow = ref Unsafe.Add(ref scratchBase, row * scratchStride); + ref ushort destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + int column = 0; + + if (useSimd && Vector512.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector512Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector512.Count) + { + Convolve( + ref scratchRow, + scratchStride, + (nuint)column, + ref verticalCoefficientBase, + verticalTapCount, + Vector512.Zero, + out Vector512 lower, + out Vector512 upper); + + TOperator.PrepareHighBitDepthVertical(lower, upper, verticalBias) + .StoreUnsafe(ref destinationRow, (nuint)column); + } + } + + if (useSimd && Vector256.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector256Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector256.Count) + { + Convolve( + ref scratchRow, + scratchStride, + (nuint)column, + ref verticalCoefficientBase, + verticalTapCount, + Vector256.Zero, + out Vector256 lower, + out Vector256 upper); + + TOperator.PrepareHighBitDepthVertical(lower, upper, verticalBias) + .StoreUnsafe(ref destinationRow, (nuint)column); + } + } + + if (useSimd && Vector128.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector128Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector128.Count) + { + Convolve( + ref scratchRow, + scratchStride, + (nuint)column, + ref verticalCoefficientBase, + verticalTapCount, + Vector128.Zero, + out Vector128 lower, + out Vector128 upper); + + TOperator.PrepareHighBitDepthVertical(lower, upper, verticalBias) + .StoreUnsafe(ref destinationRow, (nuint)column); + } + } + + for (; column < width; column++) + { + int result = ConvolveScalar( + ref Unsafe.Add(ref scratchRow, column), + scratchStride, + ref verticalCoefficientBase, + verticalTapCount); + + Unsafe.Add(ref destinationRow, column) = + TOperator.PrepareHighBitDepthVertical(result, verticalBias); + } + } + } + + /// + /// Gets the selected interpolation kernel for compound traversal. + /// + private static ReadOnlySpan GetCompoundCoefficients( + Av1InterpolationFilter filter, + int phase, + bool useReducedFilter) + => filter switch + { + Av1InterpolationFilter.Regular => RegularOperator.GetCoefficients(phase, useReducedFilter), + Av1InterpolationFilter.Smooth => SmoothOperator.GetCoefficients(phase, useReducedFilter), + Av1InterpolationFilter.Sharp => SharpOperator.GetCoefficients(phase, useReducedFilter), + _ => BilinearOperator.GetCoefficients(phase, useReducedFilter), + }; +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1CompoundIntermediateAveragePredictor.Operator.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1CompoundIntermediateAveragePredictor.Operator.cs new file mode 100644 index 0000000000..6a6eb78c25 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1CompoundIntermediateAveragePredictor.Operator.cs @@ -0,0 +1,259 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.Intrinsics; + +using static SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter.Av1CompoundInterPredictor; +using static SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter.Av1TranslationalInterPredictor; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; + +/// +/// Defines final equal-average compound-intermediate reconstruction. +/// +internal static partial class Av1CompoundIntermediateAveragePredictor +{ + /// + /// Defines equal-average finalization for scalar and SIMD lane groups. + /// + private interface IAv1CompoundIntermediateAverageOperator + { + /// + /// Equal-averages and finalizes one pair of compound intermediate samples. + /// + /// The first compound intermediate. + /// The second compound intermediate. + /// The final reconstruction shift. + /// The compound intermediate bias. + /// The reconstructed sample. + public static abstract byte Average(ushort first, ushort second, int roundBits, int roundOffset); + + /// + /// Equal-averages and finalizes one pair of high-bit-depth compound intermediate samples. + /// + /// The first compound intermediate. + /// The second compound intermediate. + /// The final reconstruction shift. + /// The compound intermediate bias. + /// The maximum reconstructed sample value. + /// The reconstructed sample. + public static abstract ushort AverageHighBitDepth( + ushort first, + ushort second, + int roundBits, + int roundOffset, + int maximum); + + /// + /// Equal-averages and finalizes 128 bits of compound intermediate samples. + /// + /// The lower first-predictor intermediates. + /// The upper first-predictor intermediates. + /// The lower second-predictor intermediates. + /// The upper second-predictor intermediates. + /// The final reconstruction shift. + /// The compound intermediate bias. + /// The reconstructed samples. + public static abstract Vector128 Average( + Vector128 first0, + Vector128 first1, + Vector128 second0, + Vector128 second1, + int roundBits, + int roundOffset); + + /// + /// Equal-averages and finalizes 256 bits of compound intermediate samples. + /// + /// The lower first-predictor intermediates. + /// The upper first-predictor intermediates. + /// The lower second-predictor intermediates. + /// The upper second-predictor intermediates. + /// The final reconstruction shift. + /// The compound intermediate bias. + /// The reconstructed samples. + public static abstract Vector256 Average( + Vector256 first0, + Vector256 first1, + Vector256 second0, + Vector256 second1, + int roundBits, + int roundOffset); + + /// + /// Equal-averages and finalizes 512 bits of compound intermediate samples. + /// + /// The lower first-predictor intermediates. + /// The upper first-predictor intermediates. + /// The lower second-predictor intermediates. + /// The upper second-predictor intermediates. + /// The final reconstruction shift. + /// The compound intermediate bias. + /// The reconstructed samples. + public static abstract Vector512 Average( + Vector512 first0, + Vector512 first1, + Vector512 second0, + Vector512 second1, + int roundBits, + int roundOffset); + + /// + /// Equal-averages and finalizes 128 bits of high-bit-depth compound intermediate samples. + /// + /// The first compound intermediates. + /// The second compound intermediates. + /// The final reconstruction shift. + /// The compound intermediate bias. + /// The maximum reconstructed sample value. + /// The reconstructed samples. + public static abstract Vector128 AverageHighBitDepth( + Vector128 first, + Vector128 second, + int roundBits, + int roundOffset, + int maximum); + + /// + /// Equal-averages and finalizes 256 bits of high-bit-depth compound intermediate samples. + /// + /// The first compound intermediates. + /// The second compound intermediates. + /// The final reconstruction shift. + /// The compound intermediate bias. + /// The maximum reconstructed sample value. + /// The reconstructed samples. + public static abstract Vector256 AverageHighBitDepth( + Vector256 first, + Vector256 second, + int roundBits, + int roundOffset, + int maximum); + + /// + /// Equal-averages and finalizes 512 bits of high-bit-depth compound intermediate samples. + /// + /// The first compound intermediates. + /// The second compound intermediates. + /// The final reconstruction shift. + /// The compound intermediate bias. + /// The maximum reconstructed sample value. + /// The reconstructed samples. + public static abstract Vector512 AverageHighBitDepth( + Vector512 first, + Vector512 second, + int roundBits, + int roundOffset, + int maximum); + } + + /// + /// Implements equal-average finalization for scalar and SIMD lane groups. + /// + private readonly struct CompoundIntermediateAverageOperator : IAv1CompoundIntermediateAverageOperator + { + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static byte Average(ushort first, ushort second, int roundBits, int roundOffset) + { + // The reference average deliberately truncates here. Finalization performs the sole rounding step. + int result = ((first + second) >> 1) - roundOffset; + return (byte)Math.Clamp(RoundPowerOfTwo(result, roundBits), 0, byte.MaxValue); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static ushort AverageHighBitDepth( + ushort first, + ushort second, + int roundBits, + int roundOffset, + int maximum) + { + int result = ((first + second) >> 1) - roundOffset; + return (ushort)Math.Clamp(RoundPowerOfTwo(result, roundBits), 0, maximum); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector128 Average( + Vector128 first0, + Vector128 first1, + Vector128 second0, + Vector128 second1, + int roundBits, + int roundOffset) + => Vector128.Narrow( + FinalizeIntermediate((first0 & second0) + ((first0 ^ second0) >> 1), roundBits, roundOffset), + FinalizeIntermediate((first1 & second1) + ((first1 ^ second1) >> 1), roundBits, roundOffset)); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector256 Average( + Vector256 first0, + Vector256 first1, + Vector256 second0, + Vector256 second1, + int roundBits, + int roundOffset) + => Vector256.Narrow( + FinalizeIntermediate((first0 & second0) + ((first0 ^ second0) >> 1), roundBits, roundOffset), + FinalizeIntermediate((first1 & second1) + ((first1 ^ second1) >> 1), roundBits, roundOffset)); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector512 Average( + Vector512 first0, + Vector512 first1, + Vector512 second0, + Vector512 second1, + int roundBits, + int roundOffset) + => Vector512.Narrow( + FinalizeIntermediate((first0 & second0) + ((first0 ^ second0) >> 1), roundBits, roundOffset), + FinalizeIntermediate((first1 & second1) + ((first1 ^ second1) >> 1), roundBits, roundOffset)); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector128 AverageHighBitDepth( + Vector128 first, + Vector128 second, + int roundBits, + int roundOffset, + int maximum) + => FinalizeHighBitDepthIntermediate( + (first & second) + ((first ^ second) >> 1), + roundBits, + roundOffset, + maximum); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector256 AverageHighBitDepth( + Vector256 first, + Vector256 second, + int roundBits, + int roundOffset, + int maximum) + => FinalizeHighBitDepthIntermediate( + (first & second) + ((first ^ second) >> 1), + roundBits, + roundOffset, + maximum); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector512 AverageHighBitDepth( + Vector512 first, + Vector512 second, + int roundBits, + int roundOffset, + int maximum) + => FinalizeHighBitDepthIntermediate( + (first & second) + ((first ^ second) >> 1), + roundBits, + roundOffset, + maximum); + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1CompoundIntermediateAveragePredictor.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1CompoundIntermediateAveragePredictor.cs new file mode 100644 index 0000000000..99b31068c6 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1CompoundIntermediateAveragePredictor.cs @@ -0,0 +1,225 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.InteropServices; +using System.Runtime.Intrinsics; + +using static SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter.Av1CompoundInterPredictor; +using static SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter.Av1TranslationalInterPredictor; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; + +/// +/// Reconstructs final samples by equal-averaging compound intermediates. +/// +internal static partial class Av1CompoundIntermediateAveragePredictor +{ + /// + /// Combines two compound intermediates by equal averaging. + /// + public static void AverageIntermediate( + Span destination, + int destinationStride, + ReadOnlySpan first, + int firstStride, + ReadOnlySpan second, + int secondStride, + int width, + int height, + int bitDepth) + => AverageIntermediate( + destination, + destinationStride, + first, + firstStride, + second, + secondStride, + width, + height, + bitDepth); + + /// + /// Executes one closed equal-average compound-intermediate operator. + /// + /// The compound-intermediate operator. + private static void AverageIntermediate( + Span destination, + int destinationStride, + ReadOnlySpan first, + int firstStride, + ReadOnlySpan second, + int secondStride, + int width, + int height, + int bitDepth) + where TOperator : struct, IAv1CompoundIntermediateAverageOperator + { + GetIntermediateRounding(bitDepth, out int roundBits, out int roundOffset); + for (int row = 0; row < height; row++) + { + Span destinationRow = destination.Slice(row * destinationStride, width); + ReadOnlySpan firstRow = first.Slice(row * firstStride, width); + ReadOnlySpan secondRow = second.Slice(row * secondStride, width); + ref byte destinationReference = ref MemoryMarshal.GetReference(destinationRow); + ref ushort firstReference = ref MemoryMarshal.GetReference(firstRow); + ref ushort secondReference = ref MemoryMarshal.GetReference(secondRow); + int column = 0; + + if (Vector512.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector512Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector512.Count) + { + Vector512 first0 = Vector512.LoadUnsafe(ref firstReference, (nuint)column); + Vector512 first1 = Vector512.LoadUnsafe(ref firstReference, (nuint)(column + Vector512.Count)); + Vector512 second0 = Vector512.LoadUnsafe(ref secondReference, (nuint)column); + Vector512 second1 = Vector512.LoadUnsafe(ref secondReference, (nuint)(column + Vector512.Count)); + TOperator.Average(first0, first1, second0, second1, roundBits, roundOffset) + .StoreUnsafe(ref destinationReference, (nuint)column); + } + } + + if (Vector256.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector256Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector256.Count) + { + Vector256 first0 = Vector256.LoadUnsafe(ref firstReference, (nuint)column); + Vector256 first1 = Vector256.LoadUnsafe(ref firstReference, (nuint)(column + Vector256.Count)); + Vector256 second0 = Vector256.LoadUnsafe(ref secondReference, (nuint)column); + Vector256 second1 = Vector256.LoadUnsafe(ref secondReference, (nuint)(column + Vector256.Count)); + TOperator.Average(first0, first1, second0, second1, roundBits, roundOffset) + .StoreUnsafe(ref destinationReference, (nuint)column); + } + } + + if (Vector128.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector128Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector128.Count) + { + Vector128 first0 = Vector128.LoadUnsafe(ref firstReference, (nuint)column); + Vector128 first1 = Vector128.LoadUnsafe( + ref firstReference, + (nuint)(column + Vector128.Count)); + + Vector128 second0 = Vector128.LoadUnsafe(ref secondReference, (nuint)column); + Vector128 second1 = Vector128.LoadUnsafe( + ref secondReference, + (nuint)(column + Vector128.Count)); + + TOperator.Average(first0, first1, second0, second1, roundBits, roundOffset).StoreUnsafe( + ref destinationReference, + (nuint)column); + } + } + + for (; column < width; column++) + { + destinationRow[column] = TOperator.Average(firstRow[column], secondRow[column], roundBits, roundOffset); + } + } + } + + /// + /// Combines two high-bit-depth compound intermediates by equal averaging. + /// + public static void AverageIntermediate( + Span destination, + int destinationStride, + ReadOnlySpan first, + int firstStride, + ReadOnlySpan second, + int secondStride, + int width, + int height, + int bitDepth) + => AverageIntermediate( + destination, + destinationStride, + first, + firstStride, + second, + secondStride, + width, + height, + bitDepth); + + /// + /// Executes one closed high-bit-depth equal-average compound-intermediate operator. + /// + /// The compound-intermediate operator. + private static void AverageIntermediate( + Span destination, + int destinationStride, + ReadOnlySpan first, + int firstStride, + ReadOnlySpan second, + int secondStride, + int width, + int height, + int bitDepth) + where TOperator : struct, IAv1CompoundIntermediateAverageOperator + { + GetIntermediateRounding(bitDepth, out int roundBits, out int roundOffset); + int maximum = (1 << bitDepth) - 1; + + for (int row = 0; row < height; row++) + { + Span destinationRow = destination.Slice(row * destinationStride, width); + ReadOnlySpan firstRow = first.Slice(row * firstStride, width); + ReadOnlySpan secondRow = second.Slice(row * secondStride, width); + ref ushort destinationReference = ref MemoryMarshal.GetReference(destinationRow); + ref ushort firstReference = ref MemoryMarshal.GetReference(firstRow); + ref ushort secondReference = ref MemoryMarshal.GetReference(secondRow); + int column = 0; + + if (Vector512.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector512Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector512.Count) + { + Vector512 firstVector = Vector512.LoadUnsafe(ref firstReference, (nuint)column); + Vector512 secondVector = Vector512.LoadUnsafe(ref secondReference, (nuint)column); + TOperator.AverageHighBitDepth(firstVector, secondVector, roundBits, roundOffset, maximum) + .StoreUnsafe(ref destinationReference, (nuint)column); + } + } + + if (Vector256.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector256Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector256.Count) + { + Vector256 firstVector = Vector256.LoadUnsafe(ref firstReference, (nuint)column); + Vector256 secondVector = Vector256.LoadUnsafe(ref secondReference, (nuint)column); + TOperator.AverageHighBitDepth(firstVector, secondVector, roundBits, roundOffset, maximum) + .StoreUnsafe(ref destinationReference, (nuint)column); + } + } + + if (Vector128.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector128Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector128.Count) + { + Vector128 firstVector = Vector128.LoadUnsafe(ref firstReference, (nuint)column); + Vector128 secondVector = Vector128.LoadUnsafe(ref secondReference, (nuint)column); + TOperator.AverageHighBitDepth(firstVector, secondVector, roundBits, roundOffset, maximum) + .StoreUnsafe(ref destinationReference, (nuint)column); + } + } + + for (; column < width; column++) + { + destinationRow[column] = TOperator.AverageHighBitDepth( + firstRow[column], + secondRow[column], + roundBits, + roundOffset, + maximum); + } + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1CompoundIntermediateDifferenceWeightedMaskBuilder.Operator.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1CompoundIntermediateDifferenceWeightedMaskBuilder.Operator.cs new file mode 100644 index 0000000000..9b0adeb5a3 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1CompoundIntermediateDifferenceWeightedMaskBuilder.Operator.cs @@ -0,0 +1,218 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.Intrinsics; + +using static SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter.Av1CompoundInterPredictor; +using static SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter.Av1TranslationalInterPredictor; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; + +/// +/// Defines difference-weighted mask generation from compound intermediates. +/// +internal static partial class Av1CompoundIntermediateDifferenceWeightedMaskBuilder +{ + /// + /// Defines difference-weighted mask generation for scalar and SIMD lane groups. + /// + private interface IAv1CompoundIntermediateDifferenceWeightedMaskOperator + { + /// + /// Creates one difference-weighted mask value from compound intermediate samples. + /// + /// The first compound intermediate. + /// The second compound intermediate. + /// The difference scaling shift. + /// Whether to invert the selected predictor. + /// The AV1 mask value. + public static abstract byte CreateMask(ushort first, ushort second, int differenceRound, bool invert); + + /// + /// Creates 128 bits of difference-weighted mask values from compound intermediate samples. + /// + /// The lower first-predictor intermediates. + /// The upper first-predictor intermediates. + /// The lower second-predictor intermediates. + /// The upper second-predictor intermediates. + /// The difference scaling shift. + /// Whether to invert the selected predictor. + /// The packed AV1 mask values. + public static abstract Vector128 CreateMask( + Vector128 first0, + Vector128 first1, + Vector128 second0, + Vector128 second1, + int differenceRound, + bool invert); + + /// + /// Creates 256 bits of difference-weighted mask values from compound intermediate samples. + /// + /// The lower first-predictor intermediates. + /// The upper first-predictor intermediates. + /// The lower second-predictor intermediates. + /// The upper second-predictor intermediates. + /// The difference scaling shift. + /// Whether to invert the selected predictor. + /// The packed AV1 mask values. + public static abstract Vector256 CreateMask( + Vector256 first0, + Vector256 first1, + Vector256 second0, + Vector256 second1, + int differenceRound, + bool invert); + + /// + /// Creates 512 bits of difference-weighted mask values from compound intermediate samples. + /// + /// The lower first-predictor intermediates. + /// The upper first-predictor intermediates. + /// The lower second-predictor intermediates. + /// The upper second-predictor intermediates. + /// The difference scaling shift. + /// Whether to invert the selected predictor. + /// The packed AV1 mask values. + public static abstract Vector512 CreateMask( + Vector512 first0, + Vector512 first1, + Vector512 second0, + Vector512 second1, + int differenceRound, + bool invert); + } + + /// + /// Implements difference-weighted mask generation for scalar and SIMD lane groups. + /// + private readonly struct CompoundIntermediateDifferenceWeightedMaskOperator : IAv1CompoundIntermediateDifferenceWeightedMaskOperator + { + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static byte CreateMask(ushort first, ushort second, int differenceRound, bool invert) + { + int difference = RoundPowerOfTwo(Math.Abs(first - second), differenceRound); + int alpha = Math.Min(MaximumMaskAlpha, 38 + (difference >> 4)); + return (byte)(invert ? MaximumMaskAlpha - alpha : alpha); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector128 CreateMask( + Vector128 first0, + Vector128 first1, + Vector128 second0, + Vector128 second1, + int differenceRound, + bool invert) + => Vector128.Narrow( + CreateMask(first0, second0, differenceRound, invert), + CreateMask(first1, second1, differenceRound, invert)); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector256 CreateMask( + Vector256 first0, + Vector256 first1, + Vector256 second0, + Vector256 second1, + int differenceRound, + bool invert) + => Vector256.Narrow( + CreateMask(first0, second0, differenceRound, invert), + CreateMask(first1, second1, differenceRound, invert)); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector512 CreateMask( + Vector512 first0, + Vector512 first1, + Vector512 second0, + Vector512 second1, + int differenceRound, + bool invert) + => Vector512.Narrow( + CreateMask(first0, second0, differenceRound, invert), + CreateMask(first1, second1, differenceRound, invert)); + + /// + /// Creates 128-bit unpacked mask values without losing the required pre-alpha rounding. + /// + private static Vector128 CreateMask(Vector128 first, Vector128 second, int differenceRound, bool invert) + { + Vector128 difference = Vector128.Max(first, second) - Vector128.Min(first, second); + Vector128 lower = CreateMaskAlpha(Vector128.WidenLower(difference).AsInt32(), differenceRound, invert); + Vector128 upper = CreateMaskAlpha(Vector128.WidenUpper(difference).AsInt32(), differenceRound, invert); + return Vector128.Narrow(lower, upper).AsUInt16(); + } + + /// + /// Creates 256-bit unpacked mask values without losing the required pre-alpha rounding. + /// + private static Vector256 CreateMask(Vector256 first, Vector256 second, int differenceRound, bool invert) + { + Vector256 difference = Vector256.Max(first, second) - Vector256.Min(first, second); + Vector256 lower = CreateMaskAlpha(Vector256.WidenLower(difference).AsInt32(), differenceRound, invert); + Vector256 upper = CreateMaskAlpha(Vector256.WidenUpper(difference).AsInt32(), differenceRound, invert); + return Vector256.Narrow(lower, upper).AsUInt16(); + } + + /// + /// Creates 512-bit unpacked mask values without losing the required pre-alpha rounding. + /// + private static Vector512 CreateMask(Vector512 first, Vector512 second, int differenceRound, bool invert) + { + Vector512 difference = Vector512.Max(first, second) - Vector512.Min(first, second); + Vector512 lower = CreateMaskAlpha(Vector512.WidenLower(difference).AsInt32(), differenceRound, invert); + Vector512 upper = CreateMaskAlpha(Vector512.WidenUpper(difference).AsInt32(), differenceRound, invert); + return Vector512.Narrow(lower, upper).AsUInt16(); + } + + /// + /// Converts 128-bit intermediate differences to the decoded type-38 mask range. + /// + private static Vector128 CreateMaskAlpha(Vector128 difference, int differenceRound, bool invert) + { + if (differenceRound != 0) + { + difference = (difference + Vector128.Create(1 << (differenceRound - 1))) >> differenceRound; + } + + Vector128 maximum = Vector128.Create(MaximumMaskAlpha); + Vector128 alpha = Vector128.Min(maximum, (difference >> 4) + Vector128.Create(38)); + return invert ? maximum - alpha : alpha; + } + + /// + /// Converts 256-bit intermediate differences to the decoded type-38 mask range. + /// + private static Vector256 CreateMaskAlpha(Vector256 difference, int differenceRound, bool invert) + { + if (differenceRound != 0) + { + difference = (difference + Vector256.Create(1 << (differenceRound - 1))) >> differenceRound; + } + + Vector256 maximum = Vector256.Create(MaximumMaskAlpha); + Vector256 alpha = Vector256.Min(maximum, (difference >> 4) + Vector256.Create(38)); + return invert ? maximum - alpha : alpha; + } + + /// + /// Converts 512-bit intermediate differences to the decoded type-38 mask range. + /// + private static Vector512 CreateMaskAlpha(Vector512 difference, int differenceRound, bool invert) + { + if (differenceRound != 0) + { + difference = (difference + Vector512.Create(1 << (differenceRound - 1))) >> differenceRound; + } + + Vector512 maximum = Vector512.Create(MaximumMaskAlpha); + Vector512 alpha = Vector512.Min(maximum, (difference >> 4) + Vector512.Create(38)); + return invert ? maximum - alpha : alpha; + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1CompoundIntermediateDifferenceWeightedMaskBuilder.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1CompoundIntermediateDifferenceWeightedMaskBuilder.cs new file mode 100644 index 0000000000..181b0e6a75 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1CompoundIntermediateDifferenceWeightedMaskBuilder.cs @@ -0,0 +1,134 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.InteropServices; +using System.Runtime.Intrinsics; +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +using static SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter.Av1CompoundInterPredictor; +using static SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter.Av1TranslationalInterPredictor; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; + +/// +/// Builds difference-weighted masks from compound intermediates. +/// +internal static partial class Av1CompoundIntermediateDifferenceWeightedMaskBuilder +{ + /// + /// Fills a luma-resolution difference-weighted mask from compound intermediates. + /// + public static void FillDifferenceWeightedIntermediateMask( + Span mask, + int maskStride, + ReadOnlySpan first, + int firstStride, + ReadOnlySpan second, + int secondStride, + int width, + int height, + int bitDepth, + Av1DifferenceWeightedMaskType maskType) + => FillDifferenceWeightedIntermediateMask( + mask, + maskStride, + first, + firstStride, + second, + secondStride, + width, + height, + bitDepth, + maskType); + + /// + /// Executes one closed difference-mask compound-intermediate operator. + /// + /// The compound-intermediate operator. + private static void FillDifferenceWeightedIntermediateMask( + Span mask, + int maskStride, + ReadOnlySpan first, + int firstStride, + ReadOnlySpan second, + int secondStride, + int width, + int height, + int bitDepth, + Av1DifferenceWeightedMaskType maskType) + where TOperator : struct, IAv1CompoundIntermediateDifferenceWeightedMaskOperator + { + bool invert = maskType == Av1DifferenceWeightedMaskType.Type38Inverse; + GetIntermediateRounding(bitDepth, out int roundBits, out _); + int differenceRound = roundBits + bitDepth - 8; + for (int row = 0; row < height; row++) + { + Span maskRow = mask.Slice(row * maskStride, width); + ReadOnlySpan firstRow = first.Slice(row * firstStride, width); + ReadOnlySpan secondRow = second.Slice(row * secondStride, width); + ref byte maskReference = ref MemoryMarshal.GetReference(maskRow); + ref ushort firstReference = ref MemoryMarshal.GetReference(firstRow); + ref ushort secondReference = ref MemoryMarshal.GetReference(secondRow); + int column = 0; + + if (Vector512.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector512Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector512.Count) + { + Vector512 first0 = Vector512.LoadUnsafe(ref firstReference, (nuint)column); + Vector512 first1 = Vector512.LoadUnsafe(ref firstReference, (nuint)(column + Vector512.Count)); + Vector512 second0 = Vector512.LoadUnsafe(ref secondReference, (nuint)column); + Vector512 second1 = Vector512.LoadUnsafe(ref secondReference, (nuint)(column + Vector512.Count)); + TOperator.CreateMask(first0, first1, second0, second1, differenceRound, invert) + .StoreUnsafe(ref maskReference, (nuint)column); + } + } + + if (Vector256.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector256Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector256.Count) + { + Vector256 first0 = Vector256.LoadUnsafe(ref firstReference, (nuint)column); + Vector256 first1 = Vector256.LoadUnsafe(ref firstReference, (nuint)(column + Vector256.Count)); + Vector256 second0 = Vector256.LoadUnsafe(ref secondReference, (nuint)column); + Vector256 second1 = Vector256.LoadUnsafe(ref secondReference, (nuint)(column + Vector256.Count)); + TOperator.CreateMask(first0, first1, second0, second1, differenceRound, invert) + .StoreUnsafe(ref maskReference, (nuint)column); + } + } + + if (Vector128.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector128Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector128.Count) + { + Vector128 first0 = Vector128.LoadUnsafe(ref firstReference, (nuint)column); + Vector128 first1 = Vector128.LoadUnsafe( + ref firstReference, + (nuint)(column + Vector128.Count)); + + Vector128 second0 = Vector128.LoadUnsafe(ref secondReference, (nuint)column); + Vector128 second1 = Vector128.LoadUnsafe( + ref secondReference, + (nuint)(column + Vector128.Count)); + + TOperator.CreateMask( + first0, + first1, + second0, + second1, + differenceRound, + invert).StoreUnsafe(ref maskReference, (nuint)column); + } + } + + for (; column < width; column++) + { + maskRow[column] = TOperator.CreateMask(firstRow[column], secondRow[column], differenceRound, invert); + } + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1CompoundIntermediateDistanceWeightedPredictor.Operator.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1CompoundIntermediateDistanceWeightedPredictor.Operator.cs new file mode 100644 index 0000000000..f876c16de1 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1CompoundIntermediateDistanceWeightedPredictor.Operator.cs @@ -0,0 +1,418 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.Intrinsics; + +using static SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter.Av1CompoundInterPredictor; +using static SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter.Av1TranslationalInterPredictor; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; + +/// +/// Defines final distance-weighted compound-intermediate reconstruction. +/// +internal static partial class Av1CompoundIntermediateDistanceWeightedPredictor +{ + /// + /// Defines distance-weighted finalization for scalar and SIMD lane groups. + /// + private interface IAv1CompoundIntermediateDistanceWeightedOperator + { + /// + /// Distance-weights and finalizes one pair of compound intermediate samples. + /// + /// The first compound intermediate. + /// The second compound intermediate. + /// The first predictor weight. + /// The second predictor weight. + /// The final reconstruction shift. + /// The compound intermediate bias. + /// The reconstructed sample. + public static abstract byte DistanceWeighted( + ushort first, + ushort second, + int firstWeight, + int secondWeight, + int roundBits, + int roundOffset); + + /// + /// Distance-weights and finalizes 128 bits of compound intermediate samples. + /// + /// The lower first-predictor intermediates. + /// The upper first-predictor intermediates. + /// The lower second-predictor intermediates. + /// The upper second-predictor intermediates. + /// The first predictor weight. + /// The second predictor weight. + /// The final reconstruction shift. + /// The compound intermediate bias. + /// The reconstructed samples. + public static abstract Vector128 DistanceWeighted( + Vector128 first0, + Vector128 first1, + Vector128 second0, + Vector128 second1, + int firstWeight, + int secondWeight, + int roundBits, + int roundOffset); + + /// + /// Distance-weights and finalizes 256 bits of compound intermediate samples. + /// + /// The lower first-predictor intermediates. + /// The upper first-predictor intermediates. + /// The lower second-predictor intermediates. + /// The upper second-predictor intermediates. + /// The first predictor weight. + /// The second predictor weight. + /// The final reconstruction shift. + /// The compound intermediate bias. + /// The reconstructed samples. + public static abstract Vector256 DistanceWeighted( + Vector256 first0, + Vector256 first1, + Vector256 second0, + Vector256 second1, + int firstWeight, + int secondWeight, + int roundBits, + int roundOffset); + + /// + /// Distance-weights and finalizes 512 bits of compound intermediate samples. + /// + /// The lower first-predictor intermediates. + /// The upper first-predictor intermediates. + /// The lower second-predictor intermediates. + /// The upper second-predictor intermediates. + /// The first predictor weight. + /// The second predictor weight. + /// The final reconstruction shift. + /// The compound intermediate bias. + /// The reconstructed samples. + public static abstract Vector512 DistanceWeighted( + Vector512 first0, + Vector512 first1, + Vector512 second0, + Vector512 second1, + int firstWeight, + int secondWeight, + int roundBits, + int roundOffset); + + /// + /// Distance-weights and finalizes one pair of high-bit-depth compound intermediate samples. + /// + /// The first compound intermediate. + /// The second compound intermediate. + /// The first predictor weight. + /// The second predictor weight. + /// The final reconstruction shift. + /// The compound intermediate bias. + /// The maximum reconstructed sample value. + /// The reconstructed sample. + public static abstract ushort DistanceWeightedHighBitDepth( + ushort first, + ushort second, + int firstWeight, + int secondWeight, + int roundBits, + int roundOffset, + int maximum); + + /// + /// Distance-weights and finalizes 128 bits of high-bit-depth compound intermediate samples. + /// + /// The first compound intermediates. + /// The second compound intermediates. + /// The first predictor weight. + /// The second predictor weight. + /// The final reconstruction shift. + /// The compound intermediate bias. + /// The maximum reconstructed sample value. + /// The reconstructed samples. + public static abstract Vector128 DistanceWeightedHighBitDepth( + Vector128 first, + Vector128 second, + int firstWeight, + int secondWeight, + int roundBits, + int roundOffset, + int maximum); + + /// + /// Distance-weights and finalizes 256 bits of high-bit-depth compound intermediate samples. + /// + /// The first compound intermediates. + /// The second compound intermediates. + /// The first predictor weight. + /// The second predictor weight. + /// The final reconstruction shift. + /// The compound intermediate bias. + /// The maximum reconstructed sample value. + /// The reconstructed samples. + public static abstract Vector256 DistanceWeightedHighBitDepth( + Vector256 first, + Vector256 second, + int firstWeight, + int secondWeight, + int roundBits, + int roundOffset, + int maximum); + + /// + /// Distance-weights and finalizes 512 bits of high-bit-depth compound intermediate samples. + /// + /// The first compound intermediates. + /// The second compound intermediates. + /// The first predictor weight. + /// The second predictor weight. + /// The final reconstruction shift. + /// The compound intermediate bias. + /// The maximum reconstructed sample value. + /// The reconstructed samples. + public static abstract Vector512 DistanceWeightedHighBitDepth( + Vector512 first, + Vector512 second, + int firstWeight, + int secondWeight, + int roundBits, + int roundOffset, + int maximum); + } + + /// + /// Implements distance-weighted finalization for scalar and SIMD lane groups. + /// + private readonly struct CompoundIntermediateDistanceWeightedOperator : IAv1CompoundIntermediateDistanceWeightedOperator + { + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static byte DistanceWeighted( + ushort first, + ushort second, + int firstWeight, + int secondWeight, + int roundBits, + int roundOffset) + { + int result = ((first * firstWeight) + (second * secondWeight)) >> DistanceWeightBits; + result -= roundOffset; + return (byte)Math.Clamp(RoundPowerOfTwo(result, roundBits), 0, byte.MaxValue); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static ushort DistanceWeightedHighBitDepth( + ushort first, + ushort second, + int firstWeight, + int secondWeight, + int roundBits, + int roundOffset, + int maximum) + { + int result = ((first * firstWeight) + (second * secondWeight)) >> DistanceWeightBits; + result -= roundOffset; + return (ushort)Math.Clamp(RoundPowerOfTwo(result, roundBits), 0, maximum); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector128 DistanceWeighted( + Vector128 first0, + Vector128 first1, + Vector128 second0, + Vector128 second1, + int firstWeight, + int secondWeight, + int roundBits, + int roundOffset) + => Vector128.Narrow( + DistanceWeighted(first0, second0, firstWeight, secondWeight, roundBits, roundOffset), + DistanceWeighted(first1, second1, firstWeight, secondWeight, roundBits, roundOffset)); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector256 DistanceWeighted( + Vector256 first0, + Vector256 first1, + Vector256 second0, + Vector256 second1, + int firstWeight, + int secondWeight, + int roundBits, + int roundOffset) + => Vector256.Narrow( + DistanceWeighted(first0, second0, firstWeight, secondWeight, roundBits, roundOffset), + DistanceWeighted(first1, second1, firstWeight, secondWeight, roundBits, roundOffset)); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector512 DistanceWeighted( + Vector512 first0, + Vector512 first1, + Vector512 second0, + Vector512 second1, + int firstWeight, + int secondWeight, + int roundBits, + int roundOffset) + => Vector512.Narrow( + DistanceWeighted(first0, second0, firstWeight, secondWeight, roundBits, roundOffset), + DistanceWeighted(first1, second1, firstWeight, secondWeight, roundBits, roundOffset)); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector128 DistanceWeightedHighBitDepth( + Vector128 first, + Vector128 second, + int firstWeight, + int secondWeight, + int roundBits, + int roundOffset, + int maximum) + { + Vector128 firstLower = Vector128.WidenLower(first); + Vector128 firstUpper = Vector128.WidenUpper(first); + Vector128 secondLower = Vector128.WidenLower(second); + Vector128 secondUpper = Vector128.WidenUpper(second); + Vector128 lower = + ((firstLower * Vector128.Create((uint)firstWeight)) + + (secondLower * Vector128.Create((uint)secondWeight))) >> DistanceWeightBits; + + Vector128 upper = + ((firstUpper * Vector128.Create((uint)firstWeight)) + + (secondUpper * Vector128.Create((uint)secondWeight))) >> DistanceWeightBits; + + return FinalizeHighBitDepthIntermediate( + Vector128.Narrow(lower, upper), + roundBits, + roundOffset, + maximum); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector256 DistanceWeightedHighBitDepth( + Vector256 first, + Vector256 second, + int firstWeight, + int secondWeight, + int roundBits, + int roundOffset, + int maximum) + { + Vector256 firstLower = Vector256.WidenLower(first); + Vector256 firstUpper = Vector256.WidenUpper(first); + Vector256 secondLower = Vector256.WidenLower(second); + Vector256 secondUpper = Vector256.WidenUpper(second); + Vector256 lower = + ((firstLower * Vector256.Create((uint)firstWeight)) + + (secondLower * Vector256.Create((uint)secondWeight))) >> DistanceWeightBits; + + Vector256 upper = + ((firstUpper * Vector256.Create((uint)firstWeight)) + + (secondUpper * Vector256.Create((uint)secondWeight))) >> DistanceWeightBits; + + return FinalizeHighBitDepthIntermediate( + Vector256.Narrow(lower, upper), + roundBits, + roundOffset, + maximum); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector512 DistanceWeightedHighBitDepth( + Vector512 first, + Vector512 second, + int firstWeight, + int secondWeight, + int roundBits, + int roundOffset, + int maximum) + { + Vector512 firstLower = Vector512.WidenLower(first); + Vector512 firstUpper = Vector512.WidenUpper(first); + Vector512 secondLower = Vector512.WidenLower(second); + Vector512 secondUpper = Vector512.WidenUpper(second); + Vector512 lower = + ((firstLower * Vector512.Create((uint)firstWeight)) + + (secondLower * Vector512.Create((uint)secondWeight))) >> DistanceWeightBits; + + Vector512 upper = + ((firstUpper * Vector512.Create((uint)firstWeight)) + + (secondUpper * Vector512.Create((uint)secondWeight))) >> DistanceWeightBits; + + return FinalizeHighBitDepthIntermediate( + Vector512.Narrow(lower, upper), + roundBits, + roundOffset, + maximum); + } + + /// + /// Distance-weights 128-bit lanes without overflowing the unsigned intermediate range. + /// + private static Vector128 DistanceWeighted( + Vector128 first, + Vector128 second, + int firstWeight, + int secondWeight, + int roundBits, + int roundOffset) + { + Vector128 firstLower = Vector128.WidenLower(first).AsInt32(); + Vector128 firstUpper = Vector128.WidenUpper(first).AsInt32(); + Vector128 secondLower = Vector128.WidenLower(second).AsInt32(); + Vector128 secondUpper = Vector128.WidenUpper(second).AsInt32(); + Vector128 lower = ((firstLower * firstWeight) + (secondLower * secondWeight)) >> DistanceWeightBits; + Vector128 upper = ((firstUpper * firstWeight) + (secondUpper * secondWeight)) >> DistanceWeightBits; + return Vector128.Narrow(FinalizeIntermediate(lower, roundBits, roundOffset), FinalizeIntermediate(upper, roundBits, roundOffset)).AsUInt16(); + } + + /// + /// Distance-weights 256-bit lanes without overflowing the unsigned intermediate range. + /// + private static Vector256 DistanceWeighted( + Vector256 first, + Vector256 second, + int firstWeight, + int secondWeight, + int roundBits, + int roundOffset) + { + Vector256 firstLower = Vector256.WidenLower(first).AsInt32(); + Vector256 firstUpper = Vector256.WidenUpper(first).AsInt32(); + Vector256 secondLower = Vector256.WidenLower(second).AsInt32(); + Vector256 secondUpper = Vector256.WidenUpper(second).AsInt32(); + Vector256 lower = ((firstLower * firstWeight) + (secondLower * secondWeight)) >> DistanceWeightBits; + Vector256 upper = ((firstUpper * firstWeight) + (secondUpper * secondWeight)) >> DistanceWeightBits; + return Vector256.Narrow(FinalizeIntermediate(lower, roundBits, roundOffset), FinalizeIntermediate(upper, roundBits, roundOffset)).AsUInt16(); + } + + /// + /// Distance-weights 512-bit lanes without overflowing the unsigned intermediate range. + /// + private static Vector512 DistanceWeighted( + Vector512 first, + Vector512 second, + int firstWeight, + int secondWeight, + int roundBits, + int roundOffset) + { + Vector512 firstLower = Vector512.WidenLower(first).AsInt32(); + Vector512 firstUpper = Vector512.WidenUpper(first).AsInt32(); + Vector512 secondLower = Vector512.WidenLower(second).AsInt32(); + Vector512 secondUpper = Vector512.WidenUpper(second).AsInt32(); + Vector512 lower = ((firstLower * firstWeight) + (secondLower * secondWeight)) >> DistanceWeightBits; + Vector512 upper = ((firstUpper * firstWeight) + (secondUpper * secondWeight)) >> DistanceWeightBits; + return Vector512.Narrow(FinalizeIntermediate(lower, roundBits, roundOffset), FinalizeIntermediate(upper, roundBits, roundOffset)).AsUInt16(); + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1CompoundIntermediateDistanceWeightedPredictor.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1CompoundIntermediateDistanceWeightedPredictor.cs new file mode 100644 index 0000000000..719306216e --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1CompoundIntermediateDistanceWeightedPredictor.cs @@ -0,0 +1,283 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.InteropServices; +using System.Runtime.Intrinsics; + +using static SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter.Av1CompoundInterPredictor; +using static SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter.Av1TranslationalInterPredictor; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; + +/// +/// Reconstructs final samples by distance-weighting compound intermediates. +/// +internal static partial class Av1CompoundIntermediateDistanceWeightedPredictor +{ + /// + /// Combines two compound intermediates using the decoded temporal-distance weights. + /// + public static void DistanceWeightedIntermediate( + Span destination, + int destinationStride, + ReadOnlySpan first, + int firstStride, + ReadOnlySpan second, + int secondStride, + int width, + int height, + int firstWeight, + int secondWeight, + int bitDepth) + => DistanceWeightedIntermediate( + destination, + destinationStride, + first, + firstStride, + second, + secondStride, + width, + height, + firstWeight, + secondWeight, + bitDepth); + + /// + /// Executes one closed distance-weighted compound-intermediate operator. + /// + /// The compound-intermediate operator. + private static void DistanceWeightedIntermediate( + Span destination, + int destinationStride, + ReadOnlySpan first, + int firstStride, + ReadOnlySpan second, + int secondStride, + int width, + int height, + int firstWeight, + int secondWeight, + int bitDepth) + where TOperator : struct, IAv1CompoundIntermediateDistanceWeightedOperator + { + GetIntermediateRounding(bitDepth, out int roundBits, out int roundOffset); + for (int row = 0; row < height; row++) + { + Span destinationRow = destination.Slice(row * destinationStride, width); + ReadOnlySpan firstRow = first.Slice(row * firstStride, width); + ReadOnlySpan secondRow = second.Slice(row * secondStride, width); + ref byte destinationReference = ref MemoryMarshal.GetReference(destinationRow); + ref ushort firstReference = ref MemoryMarshal.GetReference(firstRow); + ref ushort secondReference = ref MemoryMarshal.GetReference(secondRow); + int column = 0; + + if (Vector512.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector512Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector512.Count) + { + Vector512 first0 = Vector512.LoadUnsafe(ref firstReference, (nuint)column); + Vector512 first1 = Vector512.LoadUnsafe(ref firstReference, (nuint)(column + Vector512.Count)); + Vector512 second0 = Vector512.LoadUnsafe(ref secondReference, (nuint)column); + Vector512 second1 = Vector512.LoadUnsafe(ref secondReference, (nuint)(column + Vector512.Count)); + TOperator.DistanceWeighted( + first0, + first1, + second0, + second1, + firstWeight, + secondWeight, + roundBits, + roundOffset).StoreUnsafe(ref destinationReference, (nuint)column); + } + } + + if (Vector256.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector256Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector256.Count) + { + Vector256 first0 = Vector256.LoadUnsafe(ref firstReference, (nuint)column); + Vector256 first1 = Vector256.LoadUnsafe(ref firstReference, (nuint)(column + Vector256.Count)); + Vector256 second0 = Vector256.LoadUnsafe(ref secondReference, (nuint)column); + Vector256 second1 = Vector256.LoadUnsafe(ref secondReference, (nuint)(column + Vector256.Count)); + TOperator.DistanceWeighted( + first0, + first1, + second0, + second1, + firstWeight, + secondWeight, + roundBits, + roundOffset).StoreUnsafe(ref destinationReference, (nuint)column); + } + } + + if (Vector128.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector128Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector128.Count) + { + Vector128 first0 = Vector128.LoadUnsafe(ref firstReference, (nuint)column); + Vector128 first1 = Vector128.LoadUnsafe( + ref firstReference, + (nuint)(column + Vector128.Count)); + + Vector128 second0 = Vector128.LoadUnsafe(ref secondReference, (nuint)column); + Vector128 second1 = Vector128.LoadUnsafe( + ref secondReference, + (nuint)(column + Vector128.Count)); + + TOperator.DistanceWeighted( + first0, + first1, + second0, + second1, + firstWeight, + secondWeight, + roundBits, + roundOffset).StoreUnsafe(ref destinationReference, (nuint)column); + } + } + + for (; column < width; column++) + { + destinationRow[column] = TOperator.DistanceWeighted( + firstRow[column], + secondRow[column], + firstWeight, + secondWeight, + roundBits, + roundOffset); + } + } + } + + /// + /// Combines two high-bit-depth compound intermediates using the decoded display-distance weights. + /// + public static void DistanceWeightedIntermediate( + Span destination, + int destinationStride, + ReadOnlySpan first, + int firstStride, + ReadOnlySpan second, + int secondStride, + int width, + int height, + int firstWeight, + int secondWeight, + int bitDepth) + => DistanceWeightedIntermediate( + destination, + destinationStride, + first, + firstStride, + second, + secondStride, + width, + height, + firstWeight, + secondWeight, + bitDepth); + + /// + /// Executes one closed high-bit-depth distance-weighted compound-intermediate operator. + /// + /// The compound-intermediate operator. + private static void DistanceWeightedIntermediate( + Span destination, + int destinationStride, + ReadOnlySpan first, + int firstStride, + ReadOnlySpan second, + int secondStride, + int width, + int height, + int firstWeight, + int secondWeight, + int bitDepth) + where TOperator : struct, IAv1CompoundIntermediateDistanceWeightedOperator + { + GetIntermediateRounding(bitDepth, out int roundBits, out int roundOffset); + int maximum = (1 << bitDepth) - 1; + + for (int row = 0; row < height; row++) + { + Span destinationRow = destination.Slice(row * destinationStride, width); + ReadOnlySpan firstRow = first.Slice(row * firstStride, width); + ReadOnlySpan secondRow = second.Slice(row * secondStride, width); + ref ushort destinationReference = ref MemoryMarshal.GetReference(destinationRow); + ref ushort firstReference = ref MemoryMarshal.GetReference(firstRow); + ref ushort secondReference = ref MemoryMarshal.GetReference(secondRow); + int column = 0; + + if (Vector512.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector512Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector512.Count) + { + Vector512 firstVector = Vector512.LoadUnsafe(ref firstReference, (nuint)column); + Vector512 secondVector = Vector512.LoadUnsafe(ref secondReference, (nuint)column); + TOperator.DistanceWeightedHighBitDepth( + firstVector, + secondVector, + firstWeight, + secondWeight, + roundBits, + roundOffset, + maximum).StoreUnsafe(ref destinationReference, (nuint)column); + } + } + + if (Vector256.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector256Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector256.Count) + { + Vector256 firstVector = Vector256.LoadUnsafe(ref firstReference, (nuint)column); + Vector256 secondVector = Vector256.LoadUnsafe(ref secondReference, (nuint)column); + TOperator.DistanceWeightedHighBitDepth( + firstVector, + secondVector, + firstWeight, + secondWeight, + roundBits, + roundOffset, + maximum).StoreUnsafe(ref destinationReference, (nuint)column); + } + } + + if (Vector128.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector128Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector128.Count) + { + Vector128 firstVector = Vector128.LoadUnsafe(ref firstReference, (nuint)column); + Vector128 secondVector = Vector128.LoadUnsafe(ref secondReference, (nuint)column); + TOperator.DistanceWeightedHighBitDepth( + firstVector, + secondVector, + firstWeight, + secondWeight, + roundBits, + roundOffset, + maximum).StoreUnsafe(ref destinationReference, (nuint)column); + } + } + + for (; column < width; column++) + { + destinationRow[column] = TOperator.DistanceWeightedHighBitDepth( + firstRow[column], + secondRow[column], + firstWeight, + secondWeight, + roundBits, + roundOffset, + maximum); + } + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1CompoundIntermediateMaskBlendPredictor.Operator.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1CompoundIntermediateMaskBlendPredictor.Operator.cs new file mode 100644 index 0000000000..10feeaad40 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1CompoundIntermediateMaskBlendPredictor.Operator.cs @@ -0,0 +1,511 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.Intrinsics; + +using static SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter.Av1CompoundInterPredictor; +using static SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter.Av1TranslationalInterPredictor; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; + +/// +/// Defines final masked compound-intermediate reconstruction. +/// +internal static partial class Av1CompoundIntermediateMaskBlendPredictor +{ + /// + /// Defines masked compound-intermediate finalization for scalar and SIMD lane groups. + /// + private interface IAv1CompoundIntermediateMaskBlendOperator + { + /// + /// Alpha-blends and finalizes one pair of compound intermediate samples. + /// + /// The first compound intermediate. + /// The second compound intermediate. + /// The first-predictor weight in the AV1 mask range. + /// The final reconstruction shift. + /// The compound intermediate bias. + /// The reconstructed sample. + public static abstract byte Blend(ushort first, ushort second, byte alpha, int roundBits, int roundOffset); + + /// + /// Alpha-blends and finalizes 128 bits of compound intermediate samples. + /// + /// The lower first-predictor intermediates. + /// The upper first-predictor intermediates. + /// The lower second-predictor intermediates. + /// The upper second-predictor intermediates. + /// The first-predictor weights in the AV1 mask range. + /// The final reconstruction shift. + /// The compound intermediate bias. + /// The reconstructed samples. + public static abstract Vector128 Blend( + Vector128 first0, + Vector128 first1, + Vector128 second0, + Vector128 second1, + Vector128 alpha, + int roundBits, + int roundOffset); + + /// + /// Alpha-blends and finalizes 256 bits of compound intermediate samples. + /// + /// The lower first-predictor intermediates. + /// The upper first-predictor intermediates. + /// The lower second-predictor intermediates. + /// The upper second-predictor intermediates. + /// The first-predictor weights in the AV1 mask range. + /// The final reconstruction shift. + /// The compound intermediate bias. + /// The reconstructed samples. + public static abstract Vector256 Blend( + Vector256 first0, + Vector256 first1, + Vector256 second0, + Vector256 second1, + Vector256 alpha, + int roundBits, + int roundOffset); + + /// + /// Alpha-blends and finalizes 512 bits of compound intermediate samples. + /// + /// The lower first-predictor intermediates. + /// The upper first-predictor intermediates. + /// The lower second-predictor intermediates. + /// The upper second-predictor intermediates. + /// The first-predictor weights in the AV1 mask range. + /// The final reconstruction shift. + /// The compound intermediate bias. + /// The reconstructed samples. + public static abstract Vector512 Blend( + Vector512 first0, + Vector512 first1, + Vector512 second0, + Vector512 second1, + Vector512 alpha, + int roundBits, + int roundOffset); + + /// + /// Alpha-blends and finalizes one pair of high-bit-depth compound intermediate samples. + /// + /// The first compound intermediate. + /// The second compound intermediate. + /// The first-predictor weight in the AV1 mask range. + /// The final reconstruction shift. + /// The compound intermediate bias. + /// The maximum reconstructed sample value. + /// The reconstructed sample. + public static abstract ushort BlendHighBitDepth( + ushort first, + ushort second, + byte alpha, + int roundBits, + int roundOffset, + int maximum); + + /// + /// Alpha-blends and finalizes 128 bits of high-bit-depth compound intermediate samples. + /// + /// The lower first-predictor intermediates. + /// The upper first-predictor intermediates. + /// The lower second-predictor intermediates. + /// The upper second-predictor intermediates. + /// The first-predictor weights in the AV1 mask range. + /// The final reconstruction shift. + /// The compound intermediate bias. + /// The maximum reconstructed sample value. + /// The lower reconstructed samples. + /// The upper reconstructed samples. + public static abstract void BlendHighBitDepth( + Vector128 first0, + Vector128 first1, + Vector128 second0, + Vector128 second1, + Vector128 alpha, + int roundBits, + int roundOffset, + int maximum, + out Vector128 result0, + out Vector128 result1); + + /// + /// Alpha-blends and finalizes 256 bits of high-bit-depth compound intermediate samples. + /// + /// The lower first-predictor intermediates. + /// The upper first-predictor intermediates. + /// The lower second-predictor intermediates. + /// The upper second-predictor intermediates. + /// The first-predictor weights in the AV1 mask range. + /// The final reconstruction shift. + /// The compound intermediate bias. + /// The maximum reconstructed sample value. + /// The lower reconstructed samples. + /// The upper reconstructed samples. + public static abstract void BlendHighBitDepth( + Vector256 first0, + Vector256 first1, + Vector256 second0, + Vector256 second1, + Vector256 alpha, + int roundBits, + int roundOffset, + int maximum, + out Vector256 result0, + out Vector256 result1); + + /// + /// Alpha-blends and finalizes 512 bits of high-bit-depth compound intermediate samples. + /// + /// The lower first-predictor intermediates. + /// The upper first-predictor intermediates. + /// The lower second-predictor intermediates. + /// The upper second-predictor intermediates. + /// The first-predictor weights in the AV1 mask range. + /// The final reconstruction shift. + /// The compound intermediate bias. + /// The maximum reconstructed sample value. + /// The lower reconstructed samples. + /// The upper reconstructed samples. + public static abstract void BlendHighBitDepth( + Vector512 first0, + Vector512 first1, + Vector512 second0, + Vector512 second1, + Vector512 alpha, + int roundBits, + int roundOffset, + int maximum, + out Vector512 result0, + out Vector512 result1); + } + + /// + /// Implements masked compound-intermediate finalization for scalar and SIMD lane groups. + /// + private readonly struct CompoundIntermediateMaskBlendOperator : IAv1CompoundIntermediateMaskBlendOperator + { + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static byte Blend(ushort first, ushort second, byte alpha, int roundBits, int roundOffset) + { + // The Q6 blend truncates because final pixel rounding is still pending after bias removal. + int result = ((alpha * first) + ((MaximumMaskAlpha - alpha) * second)) >> MaskWeightBits; + result -= roundOffset; + return (byte)Math.Clamp(RoundPowerOfTwo(result, roundBits), 0, byte.MaxValue); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static ushort BlendHighBitDepth( + ushort first, + ushort second, + byte alpha, + int roundBits, + int roundOffset, + int maximum) + { + int result = ((alpha * first) + ((MaximumMaskAlpha - alpha) * second)) >> MaskWeightBits; + result -= roundOffset; + return (ushort)Math.Clamp(RoundPowerOfTwo(result, roundBits), 0, maximum); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector128 Blend( + Vector128 first0, + Vector128 first1, + Vector128 second0, + Vector128 second1, + Vector128 alpha, + int roundBits, + int roundOffset) + => Vector128.Narrow( + Blend(first0, second0, Vector128.WidenLower(alpha), roundBits, roundOffset), + Blend(first1, second1, Vector128.WidenUpper(alpha), roundBits, roundOffset)); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector256 Blend( + Vector256 first0, + Vector256 first1, + Vector256 second0, + Vector256 second1, + Vector256 alpha, + int roundBits, + int roundOffset) + => Vector256.Narrow( + Blend(first0, second0, Vector256.WidenLower(alpha), roundBits, roundOffset), + Blend(first1, second1, Vector256.WidenUpper(alpha), roundBits, roundOffset)); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector512 Blend( + Vector512 first0, + Vector512 first1, + Vector512 second0, + Vector512 second1, + Vector512 alpha, + int roundBits, + int roundOffset) + => Vector512.Narrow( + Blend(first0, second0, Vector512.WidenLower(alpha), roundBits, roundOffset), + Blend(first1, second1, Vector512.WidenUpper(alpha), roundBits, roundOffset)); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void BlendHighBitDepth( + Vector128 first0, + Vector128 first1, + Vector128 second0, + Vector128 second1, + Vector128 alpha, + int roundBits, + int roundOffset, + int maximum, + out Vector128 result0, + out Vector128 result1) + { + result0 = BlendHighBitDepth( + first0, + second0, + Vector128.WidenLower(alpha), + roundBits, + roundOffset, + maximum); + + result1 = BlendHighBitDepth( + first1, + second1, + Vector128.WidenUpper(alpha), + roundBits, + roundOffset, + maximum); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void BlendHighBitDepth( + Vector256 first0, + Vector256 first1, + Vector256 second0, + Vector256 second1, + Vector256 alpha, + int roundBits, + int roundOffset, + int maximum, + out Vector256 result0, + out Vector256 result1) + { + result0 = BlendHighBitDepth( + first0, + second0, + Vector256.WidenLower(alpha), + roundBits, + roundOffset, + maximum); + + result1 = BlendHighBitDepth( + first1, + second1, + Vector256.WidenUpper(alpha), + roundBits, + roundOffset, + maximum); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void BlendHighBitDepth( + Vector512 first0, + Vector512 first1, + Vector512 second0, + Vector512 second1, + Vector512 alpha, + int roundBits, + int roundOffset, + int maximum, + out Vector512 result0, + out Vector512 result1) + { + result0 = BlendHighBitDepth( + first0, + second0, + Vector512.WidenLower(alpha), + roundBits, + roundOffset, + maximum); + + result1 = BlendHighBitDepth( + first1, + second1, + Vector512.WidenUpper(alpha), + roundBits, + roundOffset, + maximum); + } + + /// + /// Alpha-blends 128-bit lanes after widening every product to signed 32-bit precision. + /// + private static Vector128 Blend( + Vector128 first, + Vector128 second, + Vector128 alpha, + int roundBits, + int roundOffset) + { + Vector128 firstLower = Vector128.WidenLower(first).AsInt32(); + Vector128 firstUpper = Vector128.WidenUpper(first).AsInt32(); + Vector128 secondLower = Vector128.WidenLower(second).AsInt32(); + Vector128 secondUpper = Vector128.WidenUpper(second).AsInt32(); + Vector128 alphaLower = Vector128.WidenLower(alpha).AsInt32(); + Vector128 alphaUpper = Vector128.WidenUpper(alpha).AsInt32(); + Vector128 maximum = Vector128.Create(MaximumMaskAlpha); + Vector128 lower = ((alphaLower * firstLower) + ((maximum - alphaLower) * secondLower)) >> MaskWeightBits; + Vector128 upper = ((alphaUpper * firstUpper) + ((maximum - alphaUpper) * secondUpper)) >> MaskWeightBits; + return Vector128.Narrow(FinalizeIntermediate(lower, roundBits, roundOffset), FinalizeIntermediate(upper, roundBits, roundOffset)).AsUInt16(); + } + + /// + /// Alpha-blends 256-bit lanes after widening every product to signed 32-bit precision. + /// + private static Vector256 Blend( + Vector256 first, + Vector256 second, + Vector256 alpha, + int roundBits, + int roundOffset) + { + Vector256 firstLower = Vector256.WidenLower(first).AsInt32(); + Vector256 firstUpper = Vector256.WidenUpper(first).AsInt32(); + Vector256 secondLower = Vector256.WidenLower(second).AsInt32(); + Vector256 secondUpper = Vector256.WidenUpper(second).AsInt32(); + Vector256 alphaLower = Vector256.WidenLower(alpha).AsInt32(); + Vector256 alphaUpper = Vector256.WidenUpper(alpha).AsInt32(); + Vector256 maximum = Vector256.Create(MaximumMaskAlpha); + Vector256 lower = ((alphaLower * firstLower) + ((maximum - alphaLower) * secondLower)) >> MaskWeightBits; + Vector256 upper = ((alphaUpper * firstUpper) + ((maximum - alphaUpper) * secondUpper)) >> MaskWeightBits; + return Vector256.Narrow(FinalizeIntermediate(lower, roundBits, roundOffset), FinalizeIntermediate(upper, roundBits, roundOffset)).AsUInt16(); + } + + /// + /// Alpha-blends 512-bit lanes after widening every product to signed 32-bit precision. + /// + private static Vector512 Blend( + Vector512 first, + Vector512 second, + Vector512 alpha, + int roundBits, + int roundOffset) + { + Vector512 firstLower = Vector512.WidenLower(first).AsInt32(); + Vector512 firstUpper = Vector512.WidenUpper(first).AsInt32(); + Vector512 secondLower = Vector512.WidenLower(second).AsInt32(); + Vector512 secondUpper = Vector512.WidenUpper(second).AsInt32(); + Vector512 alphaLower = Vector512.WidenLower(alpha).AsInt32(); + Vector512 alphaUpper = Vector512.WidenUpper(alpha).AsInt32(); + Vector512 maximum = Vector512.Create(MaximumMaskAlpha); + Vector512 lower = ((alphaLower * firstLower) + ((maximum - alphaLower) * secondLower)) >> MaskWeightBits; + Vector512 upper = ((alphaUpper * firstUpper) + ((maximum - alphaUpper) * secondUpper)) >> MaskWeightBits; + return Vector512.Narrow(FinalizeIntermediate(lower, roundBits, roundOffset), FinalizeIntermediate(upper, roundBits, roundOffset)).AsUInt16(); + } + + /// + /// Alpha-blends 128-bit high-bit-depth lanes without narrowing the unsigned intermediate range. + /// + private static Vector128 BlendHighBitDepth( + Vector128 first, + Vector128 second, + Vector128 alpha, + int roundBits, + int roundOffset, + int maximum) + { + Vector128 firstLower = Vector128.WidenLower(first); + Vector128 firstUpper = Vector128.WidenUpper(first); + Vector128 secondLower = Vector128.WidenLower(second); + Vector128 secondUpper = Vector128.WidenUpper(second); + Vector128 alphaLower = Vector128.WidenLower(alpha); + Vector128 alphaUpper = Vector128.WidenUpper(alpha); + Vector128 maximumAlpha = Vector128.Create((uint)MaximumMaskAlpha); + Vector128 lower = + ((alphaLower * firstLower) + ((maximumAlpha - alphaLower) * secondLower)) >> MaskWeightBits; + + Vector128 upper = + ((alphaUpper * firstUpper) + ((maximumAlpha - alphaUpper) * secondUpper)) >> MaskWeightBits; + + return FinalizeHighBitDepthIntermediate( + Vector128.Narrow(lower, upper), + roundBits, + roundOffset, + maximum); + } + + /// + /// Alpha-blends 256-bit high-bit-depth lanes without narrowing the unsigned intermediate range. + /// + private static Vector256 BlendHighBitDepth( + Vector256 first, + Vector256 second, + Vector256 alpha, + int roundBits, + int roundOffset, + int maximum) + { + Vector256 firstLower = Vector256.WidenLower(first); + Vector256 firstUpper = Vector256.WidenUpper(first); + Vector256 secondLower = Vector256.WidenLower(second); + Vector256 secondUpper = Vector256.WidenUpper(second); + Vector256 alphaLower = Vector256.WidenLower(alpha); + Vector256 alphaUpper = Vector256.WidenUpper(alpha); + Vector256 maximumAlpha = Vector256.Create((uint)MaximumMaskAlpha); + Vector256 lower = + ((alphaLower * firstLower) + ((maximumAlpha - alphaLower) * secondLower)) >> MaskWeightBits; + + Vector256 upper = + ((alphaUpper * firstUpper) + ((maximumAlpha - alphaUpper) * secondUpper)) >> MaskWeightBits; + + return FinalizeHighBitDepthIntermediate( + Vector256.Narrow(lower, upper), + roundBits, + roundOffset, + maximum); + } + + /// + /// Alpha-blends 512-bit high-bit-depth lanes without narrowing the unsigned intermediate range. + /// + private static Vector512 BlendHighBitDepth( + Vector512 first, + Vector512 second, + Vector512 alpha, + int roundBits, + int roundOffset, + int maximum) + { + Vector512 firstLower = Vector512.WidenLower(first); + Vector512 firstUpper = Vector512.WidenUpper(first); + Vector512 secondLower = Vector512.WidenLower(second); + Vector512 secondUpper = Vector512.WidenUpper(second); + Vector512 alphaLower = Vector512.WidenLower(alpha); + Vector512 alphaUpper = Vector512.WidenUpper(alpha); + Vector512 maximumAlpha = Vector512.Create((uint)MaximumMaskAlpha); + Vector512 lower = + ((alphaLower * firstLower) + ((maximumAlpha - alphaLower) * secondLower)) >> MaskWeightBits; + + Vector512 upper = + ((alphaUpper * firstUpper) + ((maximumAlpha - alphaUpper) * secondUpper)) >> MaskWeightBits; + + return FinalizeHighBitDepthIntermediate( + Vector512.Narrow(lower, upper), + roundBits, + roundOffset, + maximum); + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1CompoundIntermediateMaskBlendPredictor.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1CompoundIntermediateMaskBlendPredictor.cs new file mode 100644 index 0000000000..e2331c863a --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1CompoundIntermediateMaskBlendPredictor.cs @@ -0,0 +1,388 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.InteropServices; +using System.Runtime.Intrinsics; + +using static SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter.Av1CompoundInterPredictor; +using static SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter.Av1TranslationalInterPredictor; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; + +/// +/// Reconstructs final samples by alpha-blending compound intermediates. +/// +internal static partial class Av1CompoundIntermediateMaskBlendPredictor +{ + /// + /// Blends two compound intermediates through a luma-resolution mask. + /// + public static void BlendIntermediate( + Span destination, + int destinationStride, + ReadOnlySpan first, + int firstStride, + ReadOnlySpan second, + int secondStride, + ReadOnlySpan mask, + int maskStride, + int width, + int height, + int subX, + int subY, + int bitDepth) + => BlendIntermediate( + destination, + destinationStride, + first, + firstStride, + second, + secondStride, + mask, + maskStride, + width, + height, + subX, + subY, + bitDepth); + + /// + /// Executes one closed alpha-blend compound-intermediate operator. + /// + /// The compound-intermediate operator. + private static void BlendIntermediate( + Span destination, + int destinationStride, + ReadOnlySpan first, + int firstStride, + ReadOnlySpan second, + int secondStride, + ReadOnlySpan mask, + int maskStride, + int width, + int height, + int subX, + int subY, + int bitDepth) + where TOperator : struct, IAv1CompoundIntermediateMaskBlendOperator + { + GetIntermediateRounding(bitDepth, out int roundBits, out int roundOffset); + for (int row = 0; row < height; row++) + { + Span destinationRow = destination.Slice(row * destinationStride, width); + ReadOnlySpan firstRow = first.Slice(row * firstStride, width); + ReadOnlySpan secondRow = second.Slice(row * secondStride, width); + ref byte destinationReference = ref MemoryMarshal.GetReference(destinationRow); + ref ushort firstReference = ref MemoryMarshal.GetReference(firstRow); + ref ushort secondReference = ref MemoryMarshal.GetReference(secondRow); + int column = 0; + + if (Vector512.IsHardwareAccelerated && subX == 0 && subY == 0) + { + ref byte maskReference = ref MemoryMarshal.GetReference(mask); + int maskRowOffset = row * maskStride; + nuint vectorCount = Numerics.Vector512Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector512.Count) + { + Vector512 first0 = Vector512.LoadUnsafe(ref firstReference, (nuint)column); + Vector512 first1 = Vector512.LoadUnsafe(ref firstReference, (nuint)(column + Vector512.Count)); + Vector512 second0 = Vector512.LoadUnsafe(ref secondReference, (nuint)column); + Vector512 second1 = Vector512.LoadUnsafe(ref secondReference, (nuint)(column + Vector512.Count)); + Vector512 alpha = Vector512.LoadUnsafe(ref maskReference, (nuint)(maskRowOffset + column)); + TOperator.Blend(first0, first1, second0, second1, alpha, roundBits, roundOffset) + .StoreUnsafe(ref destinationReference, (nuint)column); + } + } + + if (Vector256.IsHardwareAccelerated && subX == 0 && subY == 0) + { + ref byte maskReference = ref MemoryMarshal.GetReference(mask); + int maskRowOffset = row * maskStride; + nuint vectorCount = Numerics.Vector256Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector256.Count) + { + Vector256 first0 = Vector256.LoadUnsafe(ref firstReference, (nuint)column); + Vector256 first1 = Vector256.LoadUnsafe(ref firstReference, (nuint)(column + Vector256.Count)); + Vector256 second0 = Vector256.LoadUnsafe(ref secondReference, (nuint)column); + Vector256 second1 = Vector256.LoadUnsafe(ref secondReference, (nuint)(column + Vector256.Count)); + Vector256 alpha = Vector256.LoadUnsafe(ref maskReference, (nuint)(maskRowOffset + column)); + TOperator.Blend(first0, first1, second0, second1, alpha, roundBits, roundOffset) + .StoreUnsafe(ref destinationReference, (nuint)column); + } + } + + if (Vector128.IsHardwareAccelerated && subX == 0 && subY == 0) + { + ref byte maskReference = ref MemoryMarshal.GetReference(mask); + int maskRowOffset = row * maskStride; + nuint vectorCount = Numerics.Vector128Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector128.Count) + { + Vector128 first0 = Vector128.LoadUnsafe(ref firstReference, (nuint)column); + Vector128 first1 = Vector128.LoadUnsafe( + ref firstReference, + (nuint)(column + Vector128.Count)); + + Vector128 second0 = Vector128.LoadUnsafe(ref secondReference, (nuint)column); + Vector128 second1 = Vector128.LoadUnsafe( + ref secondReference, + (nuint)(column + Vector128.Count)); + + Vector128 alpha = Vector128.LoadUnsafe( + ref maskReference, + (nuint)(maskRowOffset + column)); + + TOperator.Blend( + first0, + first1, + second0, + second1, + alpha, + roundBits, + roundOffset).StoreUnsafe(ref destinationReference, (nuint)column); + } + } + + for (; column < width; column++) + { + byte alpha = (byte)GetSubsampledMaskAlpha(mask, maskStride, row, column, subX, subY); + destinationRow[column] = TOperator.Blend(firstRow[column], secondRow[column], alpha, roundBits, roundOffset); + } + } + } + + /// + /// Blends two high-bit-depth compound intermediates through a luma-resolution mask. + /// + public static void BlendIntermediate( + Span destination, + int destinationStride, + ReadOnlySpan first, + int firstStride, + ReadOnlySpan second, + int secondStride, + ReadOnlySpan mask, + int maskStride, + int width, + int height, + int subX, + int subY, + int bitDepth) + => BlendIntermediate( + destination, + destinationStride, + first, + firstStride, + second, + secondStride, + mask, + maskStride, + width, + height, + subX, + subY, + bitDepth); + + /// + /// Executes one closed high-bit-depth alpha-blend compound-intermediate operator. + /// + /// The compound-intermediate operator. + private static void BlendIntermediate( + Span destination, + int destinationStride, + ReadOnlySpan first, + int firstStride, + ReadOnlySpan second, + int secondStride, + ReadOnlySpan mask, + int maskStride, + int width, + int height, + int subX, + int subY, + int bitDepth) + where TOperator : struct, IAv1CompoundIntermediateMaskBlendOperator + { + GetIntermediateRounding(bitDepth, out int roundBits, out int roundOffset); + int maximum = (1 << bitDepth) - 1; + + for (int row = 0; row < height; row++) + { + Span destinationRow = destination.Slice(row * destinationStride, width); + ReadOnlySpan firstRow = first.Slice(row * firstStride, width); + ReadOnlySpan secondRow = second.Slice(row * secondStride, width); + ref ushort destinationReference = ref MemoryMarshal.GetReference(destinationRow); + ref ushort firstReference = ref MemoryMarshal.GetReference(firstRow); + ref ushort secondReference = ref MemoryMarshal.GetReference(secondRow); + int column = 0; + + if (Vector512.IsHardwareAccelerated && subX == 0 && subY == 0) + { + ref byte maskReference = ref MemoryMarshal.GetReference(mask); + int maskRowOffset = row * maskStride; + nuint vectorCount = Numerics.Vector512Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector512.Count) + { + Vector512 first0 = Vector512.LoadUnsafe(ref firstReference, (nuint)column); + Vector512 first1 = Vector512.LoadUnsafe( + ref firstReference, + (nuint)(column + Vector512.Count)); + + Vector512 second0 = Vector512.LoadUnsafe(ref secondReference, (nuint)column); + Vector512 second1 = Vector512.LoadUnsafe( + ref secondReference, + (nuint)(column + Vector512.Count)); + + Vector512 alpha = Vector512.LoadUnsafe( + ref maskReference, + (nuint)(maskRowOffset + column)); + + TOperator.BlendHighBitDepth( + first0, + first1, + second0, + second1, + alpha, + roundBits, + roundOffset, + maximum, + out Vector512 result0, + out Vector512 result1); + + result0.StoreUnsafe(ref destinationReference, (nuint)column); + result1.StoreUnsafe( + ref destinationReference, + (nuint)(column + Vector512.Count)); + } + } + + if (Vector256.IsHardwareAccelerated && subX == 0 && subY == 0) + { + ref byte maskReference = ref MemoryMarshal.GetReference(mask); + int maskRowOffset = row * maskStride; + nuint vectorCount = Numerics.Vector256Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector256.Count) + { + Vector256 first0 = Vector256.LoadUnsafe(ref firstReference, (nuint)column); + Vector256 first1 = Vector256.LoadUnsafe( + ref firstReference, + (nuint)(column + Vector256.Count)); + + Vector256 second0 = Vector256.LoadUnsafe(ref secondReference, (nuint)column); + Vector256 second1 = Vector256.LoadUnsafe( + ref secondReference, + (nuint)(column + Vector256.Count)); + + Vector256 alpha = Vector256.LoadUnsafe( + ref maskReference, + (nuint)(maskRowOffset + column)); + + TOperator.BlendHighBitDepth( + first0, + first1, + second0, + second1, + alpha, + roundBits, + roundOffset, + maximum, + out Vector256 result0, + out Vector256 result1); + + result0.StoreUnsafe(ref destinationReference, (nuint)column); + result1.StoreUnsafe( + ref destinationReference, + (nuint)(column + Vector256.Count)); + } + } + + if (Vector128.IsHardwareAccelerated && subX == 0 && subY == 0) + { + ref byte maskReference = ref MemoryMarshal.GetReference(mask); + int maskRowOffset = row * maskStride; + nuint vectorCount = Numerics.Vector128Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector128.Count) + { + Vector128 first0 = Vector128.LoadUnsafe(ref firstReference, (nuint)column); + Vector128 first1 = Vector128.LoadUnsafe( + ref firstReference, + (nuint)(column + Vector128.Count)); + + Vector128 second0 = Vector128.LoadUnsafe(ref secondReference, (nuint)column); + Vector128 second1 = Vector128.LoadUnsafe( + ref secondReference, + (nuint)(column + Vector128.Count)); + + Vector128 alpha = Vector128.LoadUnsafe( + ref maskReference, + (nuint)(maskRowOffset + column)); + + TOperator.BlendHighBitDepth( + first0, + first1, + second0, + second1, + alpha, + roundBits, + roundOffset, + maximum, + out Vector128 result0, + out Vector128 result1); + + result0.StoreUnsafe(ref destinationReference, (nuint)column); + result1.StoreUnsafe( + ref destinationReference, + (nuint)(column + Vector128.Count)); + } + } + + for (; column < width; column++) + { + byte alpha = (byte)GetSubsampledMaskAlpha(mask, maskStride, row, column, subX, subY); + destinationRow[column] = TOperator.BlendHighBitDepth( + firstRow[column], + secondRow[column], + alpha, + roundBits, + roundOffset, + maximum); + } + } + } + + /// + /// Gets the mask alpha for one plane sample, averaging its two or four luma samples when required. + /// + private static int GetSubsampledMaskAlpha( + ReadOnlySpan mask, + int maskStride, + int row, + int column, + int subX, + int subY) + { + int maskRow = row << subY; + int maskColumn = column << subX; + int alpha = mask[(maskRow * maskStride) + maskColumn]; + if (subX != 0) + { + alpha += mask[(maskRow * maskStride) + maskColumn + 1]; + } + + if (subY != 0) + { + int lowerOffset = ((maskRow + 1) * maskStride) + maskColumn; + alpha += mask[lowerOffset]; + if (subX != 0) + { + alpha += mask[lowerOffset + 1]; + } + } + + int sampleCountShift = subX + subY; + return sampleCountShift == 0 + ? alpha + : RoundPowerOfTwo(alpha, sampleCountShift); + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1CompoundMaskBlendPredictor.Operator.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1CompoundMaskBlendPredictor.Operator.cs new file mode 100644 index 0000000000..d7df3a32ad --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1CompoundMaskBlendPredictor.Operator.cs @@ -0,0 +1,203 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.Intrinsics; + +using static SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter.Av1CompoundInterPredictor; +using static SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter.Av1TranslationalInterPredictor; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; + +/// +/// Defines alpha-masked compound prediction arithmetic. +/// +internal static partial class Av1CompoundMaskBlendPredictor +{ + /// + /// Defines alpha-masked compound blending for scalar and SIMD lane groups. + /// + private interface IAv1CompoundMaskBlendOperator + { + /// + /// Blends two 8-bit samples through an AV1 alpha value. + /// + /// The first sample. + /// The second sample. + /// The first-sample weight in the AV1 mask range. + /// The blended sample. + public static abstract byte Blend(byte first, byte second, byte alpha); + + /// + /// Blends two high-bit-depth samples through an AV1 alpha value. + /// + /// The first sample. + /// The second sample. + /// The first-sample weight in the AV1 mask range. + /// The blended sample. + public static abstract ushort Blend(ushort first, ushort second, byte alpha); + + /// + /// Blends 128-bit vectors of 8-bit samples through AV1 alpha values. + /// + /// The first samples. + /// The second samples. + /// The first-sample weights in the AV1 mask range. + /// The blended samples. + public static abstract Vector128 Blend(Vector128 first, Vector128 second, Vector128 alpha); + + /// + /// Blends 256-bit vectors of 8-bit samples through AV1 alpha values. + /// + /// The first samples. + /// The second samples. + /// The first-sample weights in the AV1 mask range. + /// The blended samples. + public static abstract Vector256 Blend(Vector256 first, Vector256 second, Vector256 alpha); + + /// + /// Blends 512-bit vectors of 8-bit samples through AV1 alpha values. + /// + /// The first samples. + /// The second samples. + /// The first-sample weights in the AV1 mask range. + /// The blended samples. + public static abstract Vector512 Blend(Vector512 first, Vector512 second, Vector512 alpha); + + /// + /// Blends 128-bit vectors of high-bit-depth samples through AV1 alpha values. + /// + /// The first samples. + /// The second samples. + /// The first-sample weights in the AV1 mask range. + /// The blended samples. + public static abstract Vector128 Blend(Vector128 first, Vector128 second, Vector128 alpha); + + /// + /// Blends 256-bit vectors of high-bit-depth samples through AV1 alpha values. + /// + /// The first samples. + /// The second samples. + /// The first-sample weights in the AV1 mask range. + /// The blended samples. + public static abstract Vector256 Blend(Vector256 first, Vector256 second, Vector256 alpha); + + /// + /// Blends 512-bit vectors of high-bit-depth samples through AV1 alpha values. + /// + /// The first samples. + /// The second samples. + /// The first-sample weights in the AV1 mask range. + /// The blended samples. + public static abstract Vector512 Blend(Vector512 first, Vector512 second, Vector512 alpha); + } + + /// + /// Implements AV1 alpha-mask blending for scalar and SIMD lane groups. + /// + private readonly struct CompoundMaskBlendOperator : IAv1CompoundMaskBlendOperator + { + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static byte Blend(byte first, byte second, byte alpha) + => (byte)(((alpha * first) + ((MaximumMaskAlpha - alpha) * second) + 32) >> MaskWeightBits); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static ushort Blend(ushort first, ushort second, byte alpha) + => (ushort)(((alpha * first) + ((MaximumMaskAlpha - alpha) * second) + 32) >> MaskWeightBits); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector128 Blend(Vector128 first, Vector128 second, Vector128 alpha) + { + Av1NonDirectionalIntraPredictorBase.Widen(first, out Vector128 first0, out Vector128 first1, out Vector128 first2, out Vector128 first3); + Av1NonDirectionalIntraPredictorBase.Widen(second, out Vector128 second0, out Vector128 second1, out Vector128 second2, out Vector128 second3); + Av1NonDirectionalIntraPredictorBase.Widen(alpha, out Vector128 alpha0, out Vector128 alpha1, out Vector128 alpha2, out Vector128 alpha3); + return Av1NonDirectionalIntraPredictorBase.Narrow( + Blend(first0, second0, alpha0), + Blend(first1, second1, alpha1), + Blend(first2, second2, alpha2), + Blend(first3, second3, alpha3)); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector256 Blend(Vector256 first, Vector256 second, Vector256 alpha) + { + Av1NonDirectionalIntraPredictorBase.Widen(first, out Vector256 first0, out Vector256 first1, out Vector256 first2, out Vector256 first3); + Av1NonDirectionalIntraPredictorBase.Widen(second, out Vector256 second0, out Vector256 second1, out Vector256 second2, out Vector256 second3); + Av1NonDirectionalIntraPredictorBase.Widen(alpha, out Vector256 alpha0, out Vector256 alpha1, out Vector256 alpha2, out Vector256 alpha3); + return Av1NonDirectionalIntraPredictorBase.Narrow( + Blend(first0, second0, alpha0), + Blend(first1, second1, alpha1), + Blend(first2, second2, alpha2), + Blend(first3, second3, alpha3)); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector512 Blend(Vector512 first, Vector512 second, Vector512 alpha) + { + Av1NonDirectionalIntraPredictorBase.Widen(first, out Vector512 first0, out Vector512 first1, out Vector512 first2, out Vector512 first3); + Av1NonDirectionalIntraPredictorBase.Widen(second, out Vector512 second0, out Vector512 second1, out Vector512 second2, out Vector512 second3); + Av1NonDirectionalIntraPredictorBase.Widen(alpha, out Vector512 alpha0, out Vector512 alpha1, out Vector512 alpha2, out Vector512 alpha3); + return Av1NonDirectionalIntraPredictorBase.Narrow( + Blend(first0, second0, alpha0), + Blend(first1, second1, alpha1), + Blend(first2, second2, alpha2), + Blend(first3, second3, alpha3)); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector128 Blend(Vector128 first, Vector128 second, Vector128 alpha) + { + Av1NonDirectionalIntraPredictorBase.Widen(first.AsInt16(), out Vector128 first0, out Vector128 first1); + Av1NonDirectionalIntraPredictorBase.Widen(second.AsInt16(), out Vector128 second0, out Vector128 second1); + Av1NonDirectionalIntraPredictorBase.Widen(alpha.AsInt16(), out Vector128 alpha0, out Vector128 alpha1); + return Av1NonDirectionalIntraPredictorBase.Narrow(Blend(first0, second0, alpha0), Blend(first1, second1, alpha1)).AsUInt16(); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector256 Blend(Vector256 first, Vector256 second, Vector256 alpha) + { + Av1NonDirectionalIntraPredictorBase.Widen(first.AsInt16(), out Vector256 first0, out Vector256 first1); + Av1NonDirectionalIntraPredictorBase.Widen(second.AsInt16(), out Vector256 second0, out Vector256 second1); + Av1NonDirectionalIntraPredictorBase.Widen(alpha.AsInt16(), out Vector256 alpha0, out Vector256 alpha1); + return Av1NonDirectionalIntraPredictorBase.Narrow(Blend(first0, second0, alpha0), Blend(first1, second1, alpha1)).AsUInt16(); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector512 Blend(Vector512 first, Vector512 second, Vector512 alpha) + { + Av1NonDirectionalIntraPredictorBase.Widen(first.AsInt16(), out Vector512 first0, out Vector512 first1); + Av1NonDirectionalIntraPredictorBase.Widen(second.AsInt16(), out Vector512 second0, out Vector512 second1); + Av1NonDirectionalIntraPredictorBase.Widen(alpha.AsInt16(), out Vector512 alpha0, out Vector512 alpha1); + return Av1NonDirectionalIntraPredictorBase.Narrow(Blend(first0, second0, alpha0), Blend(first1, second1, alpha1)).AsUInt16(); + } + + /// + /// Applies alpha-mask blending to 128-bit vectors of widened samples. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector128 Blend(Vector128 first, Vector128 second, Vector128 alpha) + => ((alpha * first) + ((Vector128.Create(MaximumMaskAlpha) - alpha) * second) + Vector128.Create(32)) >> MaskWeightBits; + + /// + /// Applies alpha-mask blending to 256-bit vectors of widened samples. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector256 Blend(Vector256 first, Vector256 second, Vector256 alpha) + => ((alpha * first) + ((Vector256.Create(MaximumMaskAlpha) - alpha) * second) + Vector256.Create(32)) >> MaskWeightBits; + + /// + /// Applies alpha-mask blending to 512-bit vectors of widened samples. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector512 Blend(Vector512 first, Vector512 second, Vector512 alpha) + => ((alpha * first) + ((Vector512.Create(MaximumMaskAlpha) - alpha) * second) + Vector512.Create(32)) >> MaskWeightBits; + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1CompoundMaskBlendPredictor.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1CompoundMaskBlendPredictor.cs new file mode 100644 index 0000000000..59400482ab --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1CompoundMaskBlendPredictor.cs @@ -0,0 +1,267 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.InteropServices; +using System.Runtime.Intrinsics; + +using static SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter.Av1CompoundInterPredictor; +using static SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter.Av1TranslationalInterPredictor; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; + +/// +/// Reconstructs alpha-masked compound prediction. +/// +internal static partial class Av1CompoundMaskBlendPredictor +{ + /// + /// Blends two 8-bit predictors through a contiguous AV1 alpha mask. + /// + public static void Blend( + Span destination, + int destinationStride, + ReadOnlySpan second, + int secondStride, + ReadOnlySpan mask, + int maskStride, + int width, + int height) + => Blend( + destination, + destinationStride, + second, + secondStride, + mask, + maskStride, + width, + height); + + /// + /// Executes one closed 8-bit masked compound operator. + /// + /// The compound arithmetic operator. + private static void Blend( + Span destination, + int destinationStride, + ReadOnlySpan second, + int secondStride, + ReadOnlySpan mask, + int maskStride, + int width, + int height) + where TOperator : struct, IAv1CompoundMaskBlendOperator + { + for (int row = 0; row < height; row++) + { + Span destinationRow = destination.Slice(row * destinationStride, width); + ReadOnlySpan secondRow = second.Slice(row * secondStride, width); + ReadOnlySpan maskRow = mask.Slice(row * maskStride, width); + ref byte destinationReference = ref MemoryMarshal.GetReference(destinationRow); + ref byte secondReference = ref MemoryMarshal.GetReference(secondRow); + ref byte maskReference = ref MemoryMarshal.GetReference(maskRow); + int column = 0; + + if (Vector512.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector512Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector512.Count) + { + Vector512 firstVector = Vector512.LoadUnsafe(ref destinationReference, (nuint)column); + Vector512 secondVector = Vector512.LoadUnsafe(ref secondReference, (nuint)column); + Vector512 maskVector = Vector512.LoadUnsafe(ref maskReference, (nuint)column); + TOperator.Blend(firstVector, secondVector, maskVector).StoreUnsafe(ref destinationReference, (nuint)column); + } + } + + if (Vector256.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector256Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector256.Count) + { + Vector256 firstVector = Vector256.LoadUnsafe(ref destinationReference, (nuint)column); + Vector256 secondVector = Vector256.LoadUnsafe(ref secondReference, (nuint)column); + Vector256 maskVector = Vector256.LoadUnsafe(ref maskReference, (nuint)column); + TOperator.Blend(firstVector, secondVector, maskVector).StoreUnsafe(ref destinationReference, (nuint)column); + } + } + + if (Vector128.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector128Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector128.Count) + { + Vector128 firstVector = Vector128.LoadUnsafe(ref destinationReference, (nuint)column); + Vector128 secondVector = Vector128.LoadUnsafe(ref secondReference, (nuint)column); + Vector128 maskVector = Vector128.LoadUnsafe(ref maskReference, (nuint)column); + TOperator.Blend(firstVector, secondVector, maskVector).StoreUnsafe(ref destinationReference, (nuint)column); + } + } + + for (; column < width; column++) + { + destinationRow[column] = TOperator.Blend(destinationRow[column], secondRow[column], maskRow[column]); + } + } + } + + /// + /// Blends two high-bit-depth predictors through a contiguous AV1 alpha mask. + /// + public static void Blend( + Span destination, + int destinationStride, + ReadOnlySpan second, + int secondStride, + ReadOnlySpan mask, + int maskStride, + int width, + int height) + => Blend( + destination, + destinationStride, + second, + secondStride, + mask, + maskStride, + width, + height); + + /// + /// Executes one closed high-bit-depth masked compound operator. + /// + /// The compound arithmetic operator. + private static void Blend( + Span destination, + int destinationStride, + ReadOnlySpan second, + int secondStride, + ReadOnlySpan mask, + int maskStride, + int width, + int height) + where TOperator : struct, IAv1CompoundMaskBlendOperator + { + for (int row = 0; row < height; row++) + { + Span destinationRow = destination.Slice(row * destinationStride, width); + ReadOnlySpan secondRow = second.Slice(row * secondStride, width); + ReadOnlySpan maskRow = mask.Slice(row * maskStride, width); + ref ushort destinationReference = ref MemoryMarshal.GetReference(destinationRow); + ref ushort secondReference = ref MemoryMarshal.GetReference(secondRow); + ref byte maskReference = ref MemoryMarshal.GetReference(maskRow); + int column = 0; + + if (Vector512.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector512Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector512.Count) + { + Vector512 firstVector = Vector512.LoadUnsafe(ref destinationReference, (nuint)column); + Vector512 secondVector = Vector512.LoadUnsafe(ref secondReference, (nuint)column); + Vector512 maskVector = LoadMask512(ref maskReference, column); + TOperator.Blend(firstVector, secondVector, maskVector).StoreUnsafe(ref destinationReference, (nuint)column); + } + } + + if (Vector256.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector256Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector256.Count) + { + Vector256 firstVector = Vector256.LoadUnsafe(ref destinationReference, (nuint)column); + Vector256 secondVector = Vector256.LoadUnsafe(ref secondReference, (nuint)column); + Vector256 maskVector = LoadMask256(ref maskReference, column); + TOperator.Blend(firstVector, secondVector, maskVector).StoreUnsafe(ref destinationReference, (nuint)column); + } + } + + if (Vector128.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector128Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector128.Count) + { + Vector128 firstVector = Vector128.LoadUnsafe(ref destinationReference, (nuint)column); + Vector128 secondVector = Vector128.LoadUnsafe(ref secondReference, (nuint)column); + Vector128 maskVector = LoadMask128(ref maskReference, column); + TOperator.Blend(firstVector, secondVector, maskVector).StoreUnsafe(ref destinationReference, (nuint)column); + } + } + + for (; column < width; column++) + { + destinationRow[column] = TOperator.Blend(destinationRow[column], secondRow[column], maskRow[column]); + } + } + } + + /// + /// Blends two 8-bit predictors through an alpha mask without explicit hardware intrinsics. + /// + public static void BlendScalar( + Span destination, + int destinationStride, + ReadOnlySpan second, + int secondStride, + ReadOnlySpan mask, + int maskStride, + int width, + int height) + => BlendScalar( + destination, + destinationStride, + second, + secondStride, + mask, + maskStride, + width, + height); + + /// + /// Executes one closed 8-bit masked compound operator without explicit hardware intrinsics. + /// + /// The compound arithmetic operator. + private static void BlendScalar( + Span destination, + int destinationStride, + ReadOnlySpan second, + int secondStride, + ReadOnlySpan mask, + int maskStride, + int width, + int height) + where TOperator : struct, IAv1CompoundMaskBlendOperator + { + for (int row = 0; row < height; row++) + { + Span destinationRow = destination.Slice(row * destinationStride, width); + ReadOnlySpan secondRow = second.Slice(row * secondStride, width); + ReadOnlySpan maskRow = mask.Slice(row * maskStride, width); + for (int column = 0; column < width; column++) + { + destinationRow[column] = TOperator.Blend(destinationRow[column], secondRow[column], maskRow[column]); + } + } + } + + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector128 LoadMask128(ref byte source, int offset) + { + Vector64 packed = Unsafe.As>(ref Unsafe.Add(ref source, offset)); + return Vector128.WidenLower(Vector128.Create(packed, Vector64.Zero)); + } + + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector256 LoadMask256(ref byte source, int offset) + { + Vector128 packed = Vector128.LoadUnsafe(ref source, (nuint)offset); + return Vector256.WidenLower(Vector256.Create(packed, Vector128.Zero)); + } + + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector512 LoadMask512(ref byte source, int offset) + { + Vector256 packed = Vector256.LoadUnsafe(ref source, (nuint)offset); + return Vector512.WidenLower(Vector512.Create(packed, Vector256.Zero)); + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1DifferenceWeightedMaskBuilder.Operator.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1DifferenceWeightedMaskBuilder.Operator.cs new file mode 100644 index 0000000000..c8d18c273b --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1DifferenceWeightedMaskBuilder.Operator.cs @@ -0,0 +1,254 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.Intrinsics; + +using static SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter.Av1CompoundInterPredictor; +using static SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter.Av1TranslationalInterPredictor; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; + +/// +/// Defines difference-weighted compound mask arithmetic. +/// +internal static partial class Av1DifferenceWeightedMaskBuilder +{ + /// + /// Defines difference-weighted compound mask generation for scalar and SIMD lane groups. + /// + private interface IAv1DifferenceWeightedMaskOperator + { + /// + /// Creates one mask value from two 8-bit predictor samples. + /// + /// The first predictor sample. + /// The second predictor sample. + /// The difference scaling shift. + /// Whether to invert the selected predictor. + /// The AV1 mask value. + public static abstract byte Create(byte first, byte second, int shift, bool invert); + + /// + /// Creates one mask value from two high-bit-depth predictor samples. + /// + /// The first predictor sample. + /// The second predictor sample. + /// The difference scaling shift. + /// Whether to invert the selected predictor. + /// The AV1 mask value. + public static abstract byte Create(ushort first, ushort second, int shift, bool invert); + + /// + /// Creates 128 bits of mask values from 8-bit predictor samples. + /// + /// The first predictor samples. + /// The second predictor samples. + /// The difference scaling shift. + /// Whether to invert the selected predictor. + /// The AV1 mask values. + public static abstract Vector128 Create(Vector128 first, Vector128 second, int shift, bool invert); + + /// + /// Creates 256 bits of mask values from 8-bit predictor samples. + /// + /// The first predictor samples. + /// The second predictor samples. + /// The difference scaling shift. + /// Whether to invert the selected predictor. + /// The AV1 mask values. + public static abstract Vector256 Create(Vector256 first, Vector256 second, int shift, bool invert); + + /// + /// Creates 512 bits of mask values from 8-bit predictor samples. + /// + /// The first predictor samples. + /// The second predictor samples. + /// The difference scaling shift. + /// Whether to invert the selected predictor. + /// The AV1 mask values. + public static abstract Vector512 Create(Vector512 first, Vector512 second, int shift, bool invert); + + /// + /// Creates 128 bits of packed mask values from high-bit-depth predictor samples. + /// + /// The lower first-predictor samples. + /// The upper first-predictor samples. + /// The lower second-predictor samples. + /// The upper second-predictor samples. + /// The difference scaling shift. + /// Whether to invert the selected predictor. + /// The packed AV1 mask values. + public static abstract Vector128 Create( + Vector128 first0, + Vector128 first1, + Vector128 second0, + Vector128 second1, + int shift, + bool invert); + + /// + /// Creates 256 bits of packed mask values from high-bit-depth predictor samples. + /// + /// The lower first-predictor samples. + /// The upper first-predictor samples. + /// The lower second-predictor samples. + /// The upper second-predictor samples. + /// The difference scaling shift. + /// Whether to invert the selected predictor. + /// The packed AV1 mask values. + public static abstract Vector256 Create( + Vector256 first0, + Vector256 first1, + Vector256 second0, + Vector256 second1, + int shift, + bool invert); + + /// + /// Creates 512 bits of packed mask values from high-bit-depth predictor samples. + /// + /// The lower first-predictor samples. + /// The upper first-predictor samples. + /// The lower second-predictor samples. + /// The upper second-predictor samples. + /// The difference scaling shift. + /// Whether to invert the selected predictor. + /// The packed AV1 mask values. + public static abstract Vector512 Create( + Vector512 first0, + Vector512 first1, + Vector512 second0, + Vector512 second1, + int shift, + bool invert); + } + + /// + /// Implements AV1 difference-weighted mask generation for scalar and SIMD lane groups. + /// + private readonly struct DifferenceWeightedMaskOperator : IAv1DifferenceWeightedMaskOperator + { + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static byte Create(byte first, byte second, int shift, bool invert) + => Create((ushort)Math.Abs(first - second), shift, invert); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static byte Create(ushort first, ushort second, int shift, bool invert) + => Create((ushort)Math.Abs(first - second), shift, invert); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector128 Create(Vector128 first, Vector128 second, int shift, bool invert) + { + Vector128 difference = Vector128.Max(first, second) - Vector128.Min(first, second); + Vector128 lower = CreateAlpha(Vector128.WidenLower(difference), shift, invert); + Vector128 upper = CreateAlpha(Vector128.WidenUpper(difference), shift, invert); + return Vector128.Narrow(lower, upper); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector256 Create(Vector256 first, Vector256 second, int shift, bool invert) + { + Vector256 difference = Vector256.Max(first, second) - Vector256.Min(first, second); + Vector256 lower = CreateAlpha(Vector256.WidenLower(difference), shift, invert); + Vector256 upper = CreateAlpha(Vector256.WidenUpper(difference), shift, invert); + return Vector256.Narrow(lower, upper); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector512 Create(Vector512 first, Vector512 second, int shift, bool invert) + { + Vector512 difference = Vector512.Max(first, second) - Vector512.Min(first, second); + Vector512 lower = CreateAlpha(Vector512.WidenLower(difference), shift, invert); + Vector512 upper = CreateAlpha(Vector512.WidenUpper(difference), shift, invert); + return Vector512.Narrow(lower, upper); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector128 Create( + Vector128 first0, + Vector128 first1, + Vector128 second0, + Vector128 second1, + int shift, + bool invert) + => Vector128.Narrow( + CreateAlpha(Vector128.Max(first0, second0) - Vector128.Min(first0, second0), shift, invert), + CreateAlpha(Vector128.Max(first1, second1) - Vector128.Min(first1, second1), shift, invert)); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector256 Create( + Vector256 first0, + Vector256 first1, + Vector256 second0, + Vector256 second1, + int shift, + bool invert) + => Vector256.Narrow( + CreateAlpha(Vector256.Max(first0, second0) - Vector256.Min(first0, second0), shift, invert), + CreateAlpha(Vector256.Max(first1, second1) - Vector256.Min(first1, second1), shift, invert)); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector512 Create( + Vector512 first0, + Vector512 first1, + Vector512 second0, + Vector512 second1, + int shift, + bool invert) + => Vector512.Narrow( + CreateAlpha(Vector512.Max(first0, second0) - Vector512.Min(first0, second0), shift, invert), + CreateAlpha(Vector512.Max(first1, second1) - Vector512.Min(first1, second1), shift, invert)); + + /// + /// Creates one mask value from an absolute predictor difference. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static byte Create(ushort difference, int shift, bool invert) + { + int alpha = Math.Min(MaximumMaskAlpha, 38 + (difference >> shift)); + return (byte)(invert ? MaximumMaskAlpha - alpha : alpha); + } + + /// + /// Creates 128-bit vectors of unpacked mask values from absolute predictor differences. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector128 CreateAlpha(Vector128 difference, int shift, bool invert) + { + Vector128 maximum = Vector128.Create((ushort)MaximumMaskAlpha); + Vector128 alpha = Vector128.Min(maximum, (difference >> shift) + Vector128.Create((ushort)38)); + return invert ? maximum - alpha : alpha; + } + + /// + /// Creates 256-bit vectors of unpacked mask values from absolute predictor differences. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector256 CreateAlpha(Vector256 difference, int shift, bool invert) + { + Vector256 maximum = Vector256.Create((ushort)MaximumMaskAlpha); + Vector256 alpha = Vector256.Min(maximum, (difference >> shift) + Vector256.Create((ushort)38)); + return invert ? maximum - alpha : alpha; + } + + /// + /// Creates 512-bit vectors of unpacked mask values from absolute predictor differences. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector512 CreateAlpha(Vector512 difference, int shift, bool invert) + { + Vector512 maximum = Vector512.Create((ushort)MaximumMaskAlpha); + Vector512 alpha = Vector512.Min(maximum, (difference >> shift) + Vector512.Create((ushort)38)); + return invert ? maximum - alpha : alpha; + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1DifferenceWeightedMaskBuilder.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1DifferenceWeightedMaskBuilder.cs new file mode 100644 index 0000000000..f4e0c9ff7b --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1DifferenceWeightedMaskBuilder.cs @@ -0,0 +1,214 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.InteropServices; +using System.Runtime.Intrinsics; +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +using static SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter.Av1CompoundInterPredictor; +using static SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter.Av1TranslationalInterPredictor; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; + +/// +/// Builds AV1 difference-weighted compound masks. +/// +internal static partial class Av1DifferenceWeightedMaskBuilder +{ + /// + /// Fills an 8-bit difference-weighted compound mask. + /// + public static void FillDifferenceWeightedMask( + Span mask, + int maskStride, + ReadOnlySpan first, + int firstStride, + ReadOnlySpan second, + int secondStride, + int width, + int height, + Av1DifferenceWeightedMaskType maskType) + => FillDifferenceWeightedMask( + mask, + maskStride, + first, + firstStride, + second, + secondStride, + width, + height, + maskType); + + /// + /// Executes one closed 8-bit difference-weighted mask operator. + /// + /// The difference-weighted mask operator. + private static void FillDifferenceWeightedMask( + Span mask, + int maskStride, + ReadOnlySpan first, + int firstStride, + ReadOnlySpan second, + int secondStride, + int width, + int height, + Av1DifferenceWeightedMaskType maskType) + where TOperator : struct, IAv1DifferenceWeightedMaskOperator + { + bool invert = maskType == Av1DifferenceWeightedMaskType.Type38Inverse; + for (int row = 0; row < height; row++) + { + Span maskRow = mask.Slice(row * maskStride, width); + ReadOnlySpan firstRow = first.Slice(row * firstStride, width); + ReadOnlySpan secondRow = second.Slice(row * secondStride, width); + ref byte maskReference = ref MemoryMarshal.GetReference(maskRow); + ref byte firstReference = ref MemoryMarshal.GetReference(firstRow); + ref byte secondReference = ref MemoryMarshal.GetReference(secondRow); + int column = 0; + + if (Vector512.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector512Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector512.Count) + { + Vector512 firstVector = Vector512.LoadUnsafe(ref firstReference, (nuint)column); + Vector512 secondVector = Vector512.LoadUnsafe(ref secondReference, (nuint)column); + TOperator.Create(firstVector, secondVector, 4, invert).StoreUnsafe(ref maskReference, (nuint)column); + } + } + + if (Vector256.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector256Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector256.Count) + { + Vector256 firstVector = Vector256.LoadUnsafe(ref firstReference, (nuint)column); + Vector256 secondVector = Vector256.LoadUnsafe(ref secondReference, (nuint)column); + TOperator.Create(firstVector, secondVector, 4, invert).StoreUnsafe(ref maskReference, (nuint)column); + } + } + + if (Vector128.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector128Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector128.Count) + { + Vector128 firstVector = Vector128.LoadUnsafe(ref firstReference, (nuint)column); + Vector128 secondVector = Vector128.LoadUnsafe(ref secondReference, (nuint)column); + TOperator.Create(firstVector, secondVector, 4, invert).StoreUnsafe(ref maskReference, (nuint)column); + } + } + + for (; column < width; column++) + { + maskRow[column] = TOperator.Create(firstRow[column], secondRow[column], 4, invert); + } + } + } + + /// + /// Fills a high-bit-depth difference-weighted compound mask. + /// + public static void FillDifferenceWeightedMask( + Span mask, + int maskStride, + ReadOnlySpan first, + int firstStride, + ReadOnlySpan second, + int secondStride, + int width, + int height, + int bitDepth, + Av1DifferenceWeightedMaskType maskType) + => FillDifferenceWeightedMask( + mask, + maskStride, + first, + firstStride, + second, + secondStride, + width, + height, + bitDepth, + maskType); + + /// + /// Executes one closed high-bit-depth difference-weighted mask operator. + /// + /// The difference-weighted mask operator. + private static void FillDifferenceWeightedMask( + Span mask, + int maskStride, + ReadOnlySpan first, + int firstStride, + ReadOnlySpan second, + int secondStride, + int width, + int height, + int bitDepth, + Av1DifferenceWeightedMaskType maskType) + where TOperator : struct, IAv1DifferenceWeightedMaskOperator + { + bool invert = maskType == Av1DifferenceWeightedMaskType.Type38Inverse; + int differenceShift = bitDepth - 8 + 4; + for (int row = 0; row < height; row++) + { + Span maskRow = mask.Slice(row * maskStride, width); + ReadOnlySpan firstRow = first.Slice(row * firstStride, width); + ReadOnlySpan secondRow = second.Slice(row * secondStride, width); + ref byte maskReference = ref MemoryMarshal.GetReference(maskRow); + ref ushort firstReference = ref MemoryMarshal.GetReference(firstRow); + ref ushort secondReference = ref MemoryMarshal.GetReference(secondRow); + int column = 0; + + // Two input vectors narrow to one packed byte mask. This keeps mask construction contiguous and avoids + // temporary buffers before the following vector blend consumes the complete plane block. + if (Vector512.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector512Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector512.Count) + { + Vector512 first0 = Vector512.LoadUnsafe(ref firstReference, (nuint)column); + Vector512 first1 = Vector512.LoadUnsafe(ref firstReference, (nuint)(column + Vector512.Count)); + Vector512 second0 = Vector512.LoadUnsafe(ref secondReference, (nuint)column); + Vector512 second1 = Vector512.LoadUnsafe(ref secondReference, (nuint)(column + Vector512.Count)); + TOperator.Create(first0, first1, second0, second1, differenceShift, invert) + .StoreUnsafe(ref maskReference, (nuint)column); + } + } + + if (Vector256.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector256Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector256.Count) + { + Vector256 first0 = Vector256.LoadUnsafe(ref firstReference, (nuint)column); + Vector256 first1 = Vector256.LoadUnsafe(ref firstReference, (nuint)(column + Vector256.Count)); + Vector256 second0 = Vector256.LoadUnsafe(ref secondReference, (nuint)column); + Vector256 second1 = Vector256.LoadUnsafe(ref secondReference, (nuint)(column + Vector256.Count)); + TOperator.Create(first0, first1, second0, second1, differenceShift, invert) + .StoreUnsafe(ref maskReference, (nuint)column); + } + } + + if (Vector128.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector128Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector128.Count) + { + Vector128 first0 = Vector128.LoadUnsafe(ref firstReference, (nuint)column); + Vector128 first1 = Vector128.LoadUnsafe(ref firstReference, (nuint)(column + Vector128.Count)); + Vector128 second0 = Vector128.LoadUnsafe(ref secondReference, (nuint)column); + Vector128 second1 = Vector128.LoadUnsafe(ref secondReference, (nuint)(column + Vector128.Count)); + TOperator.Create(first0, first1, second0, second1, differenceShift, invert) + .StoreUnsafe(ref maskReference, (nuint)column); + } + } + + for (; column < width; column++) + { + maskRow[column] = TOperator.Create(firstRow[column], secondRow[column], differenceShift, invert); + } + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1InterIntraMaskBuilder.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1InterIntraMaskBuilder.cs new file mode 100644 index 0000000000..b1e8eaa908 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1InterIntraMaskBuilder.cs @@ -0,0 +1,62 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.InteropServices; +using System.Runtime.Intrinsics; +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +using static SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter.Av1CompoundInterPredictor; +using static SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter.Av1TranslationalInterPredictor; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; + +/// +/// Builds AV1 inter-intra prediction masks. +/// +internal static partial class Av1InterIntraMaskBuilder +{ + /// + /// Gets the reference decoder's one-dimensional inter-intra alpha curve. + /// + private static ReadOnlySpan InterIntraWeights => + [ + 60, 58, 56, 54, 52, 50, 48, 47, 45, 44, 42, 41, 39, 38, 37, 35, + 34, 33, 32, 31, 30, 29, 28, 27, 26, 25, 24, 23, 22, 22, 21, 20, + 19, 19, 18, 18, 17, 16, 16, 15, 15, 14, 14, 13, 13, 12, 12, 12, + 11, 11, 10, 10, 10, 9, 9, 9, 8, 8, 8, 8, 7, 7, 7, 7, + 6, 6, 6, 6, 6, 5, 5, 5, 5, 5, 4, 4, 4, 4, 4, 4, + 4, 4, 3, 3, 3, 3, 3, 3, 3, 3, 3, 2, 2, 2, 2, 2, + 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 1, 1, 1, 1, 1, 1, + 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, + ]; + + /// + /// Fills a smooth inter-intra mask for one plane. + /// + public static void FillInterIntraMask( + Span mask, + int maskStride, + int width, + int height, + Av1InterIntraMode mode, + bool invert) + { + int sizeScale = 128 / Math.Max(width, height); + for (int row = 0; row < height; row++) + { + Span maskRow = mask.Slice(row * maskStride, width); + for (int column = 0; column < width; column++) + { + int alpha = mode switch + { + Av1InterIntraMode.Vertical => InterIntraWeights[row * sizeScale], + Av1InterIntraMode.Horizontal => InterIntraWeights[column * sizeScale], + Av1InterIntraMode.Smooth => InterIntraWeights[Math.Min(row, column) * sizeScale], + _ => 32, + }; + + maskRow[column] = (byte)(invert ? MaximumMaskAlpha - alpha : alpha); + } + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1InterpolationFilter.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1InterpolationFilter.cs new file mode 100644 index 0000000000..963d4dde94 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1InterpolationFilter.cs @@ -0,0 +1,35 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; + +/// +/// Identifies an AV1 interpolation-filter family or the frame-level switchable selection. +/// +internal enum Av1InterpolationFilter : byte +{ + /// + /// The regular interpolation-filter family. + /// + Regular = 0, + + /// + /// The smooth interpolation-filter family. + /// + Smooth = 1, + + /// + /// The sharp interpolation-filter family. + /// + Sharp = 2, + + /// + /// The bilinear interpolation-filter family. + /// + Bilinear = 3, + + /// + /// Indicates that each inter block selects its interpolation-filter family. + /// + Switchable = 4, +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1ObmcMask.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1ObmcMask.cs new file mode 100644 index 0000000000..044b94ea18 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1ObmcMask.cs @@ -0,0 +1,43 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; + +/// +/// Provides the one-dimensional alpha masks used by AV1 overlapping motion compensation. +/// +internal static class Av1ObmcMask +{ + /// + /// Gets the mask whose length matches one overlapping prediction axis. + /// + /// The power-of-two overlap length. + /// The alpha values applied to the regular block predictor. + public static ReadOnlySpan Get(int length) => length switch + { + 1 => [64], + 2 => [45, 64], + 4 => [39, 50, 59, 64], + 8 => [36, 42, 48, 53, 57, 61, 64, 64], + 16 => [34, 37, 40, 43, 46, 49, 52, 54, 56, 58, 60, 61, 64, 64, 64, 64], + 32 => + [ + 33, 35, 36, 38, 40, 41, 43, 44, + 45, 47, 48, 50, 51, 52, 53, 55, + 56, 57, 58, 59, 60, 60, 61, 62, + 64, 64, 64, 64, 64, 64, 64, 64 + ], + 64 => + [ + 33, 34, 35, 35, 36, 37, 38, 39, + 40, 40, 41, 42, 43, 44, 44, 44, + 45, 46, 47, 47, 48, 49, 50, 51, + 51, 51, 52, 52, 53, 54, 55, 56, + 56, 56, 57, 57, 58, 58, 59, 60, + 60, 60, 60, 60, 61, 62, 62, 62, + 62, 62, 63, 63, 63, 63, 64, 64, + 64, 64, 64, 64, 64, 64, 64, 64 + ], + _ => throw new ArgumentOutOfRangeException(nameof(length)) + }; +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1ReferenceScale.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1ReferenceScale.cs new file mode 100644 index 0000000000..822120c2e1 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1ReferenceScale.cs @@ -0,0 +1,94 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; + +/// +/// Converts current-frame prediction coordinates into a retained reference frame's sample grid. +/// +internal readonly struct Av1ReferenceScale +{ + /// + /// The identity scale in the normative Q14 representation. + /// + private const int IdentityScale = 1 << 14; + + /// + /// The number of fractional bits carried by scaled prediction positions and steps. + /// + public const int SubpixelBits = 10; + + /// + /// The mask selecting one scaled sample's fractional position. + /// + public const int SubpixelMask = (1 << SubpixelBits) - 1; + + /// + /// The half-unit offset that centers Q4 input coordinates on the Q10 reference grid. + /// + public const int ExtraOffset = 1 << (SubpixelBits - 4 - 1); + + /// + /// Initializes a new instance of the struct. + /// + /// The retained reference width. + /// The retained reference height. + /// The current coded-frame width. + /// The current coded-frame height. + public Av1ReferenceScale(int referenceWidth, int referenceHeight, int currentWidth, int currentHeight) + { + this.HorizontalScale = ((referenceWidth << 14) + (currentWidth >> 1)) / currentWidth; + this.VerticalScale = ((referenceHeight << 14) + (currentHeight >> 1)) / currentHeight; + this.HorizontalStep = (this.HorizontalScale + 8) >> 4; + this.VerticalStep = (this.VerticalScale + 8) >> 4; + } + + /// + /// Gets the horizontal Q14 scale factor. + /// + public int HorizontalScale { get; } + + /// + /// Gets the vertical Q14 scale factor. + /// + public int VerticalScale { get; } + + /// + /// Gets the horizontal per-output-sample step in Q10 reference samples. + /// + public int HorizontalStep { get; } + + /// + /// Gets the vertical per-output-sample step in Q10 reference samples. + /// + public int VerticalStep { get; } + + /// + /// Gets a value indicating whether either reference dimension differs from the current frame. + /// + public bool IsScaled => this.HorizontalScale != IdentityScale || this.VerticalScale != IdentityScale; + + /// + /// Scales one horizontal Q4 current-frame coordinate into the Q10 reference grid. + /// + public int ScaleHorizontal(int value) => Scale(value, this.HorizontalScale); + + /// + /// Scales one vertical Q4 current-frame coordinate into the Q10 reference grid. + /// + public int ScaleVertical(int value) => Scale(value, this.VerticalScale); + + /// + /// Applies the reference decoder's signed fixed-point rounding without relying on implementation-defined negative shifts. + /// + private static int Scale(int value, int scale) + { + long offset = (scale - IdentityScale) * 8L; + long scaled = ((long)value * scale) + offset; + const int shift = 8; + const long rounding = 1L << (shift - 1); + return scaled < 0 + ? (int)-((-scaled + rounding) >> shift) + : (int)((scaled + rounding) >> shift); + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1ScaledInterPredictor.Operator.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1ScaledInterPredictor.Operator.cs new file mode 100644 index 0000000000..13e1243a8e --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1ScaledInterPredictor.Operator.cs @@ -0,0 +1,459 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.Intrinsics; + +using static SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter.Av1TranslationalInterPredictor; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; + +/// +/// Defines variable-phase reference-scaled prediction arithmetic. +/// +internal static partial class Av1ScaledInterPredictor +{ + /// + /// Defines source and convolution arithmetic shared by variable-phase scaled prediction. + /// + private interface IAv1ScaledArithmeticOperator + { + /// + /// Loads one native source sample as a signed accumulator value. + /// + /// The native sample storage type. + /// The first native source sample. + /// The source sample offset. + /// The widened sample. + public static abstract int Load(ref T source, int index) + where T : unmanaged; + + /// + /// Accumulates one sample-coefficient product. + /// + /// The current convolution sum. + /// The source sample. + /// The signed Q7 coefficient. + /// The updated convolution sum. + public static abstract int MultiplyAdd(int accumulator, int sample, int coefficient); + + /// + /// Accumulates four independent sample-coefficient products. + /// + /// The current convolution sums. + /// The source samples. + /// The signed Q7 coefficients. + /// The updated convolution sums. + public static abstract Vector128 MultiplyAdd(Vector128 accumulator, Vector128 samples, Vector128 coefficients); + + /// + /// Accumulates eight independent sample-coefficient products. + /// + /// The current convolution sums. + /// The source samples. + /// The signed Q7 coefficients. + /// The updated convolution sums. + public static abstract Vector256 MultiplyAdd(Vector256 accumulator, Vector256 samples, Vector256 coefficients); + + /// + /// Accumulates sixteen independent sample-coefficient products. + /// + /// The current convolution sums. + /// The source samples. + /// The signed Q7 coefficients. + /// The updated convolution sums. + public static abstract Vector512 MultiplyAdd(Vector512 accumulator, Vector512 samples, Vector512 coefficients); + + /// + /// Convolves one intermediate sample column without hardware intrinsics. + /// + /// The first intermediate sample. + /// The distance between intermediate rows. + /// The first signed Q7 coefficient. + /// The number of active coefficients. + /// The exact convolution sum. + public static abstract int Convolve(ref short source, int sourceStride, ref short coefficients, int coefficientCount); + + /// + /// Convolves eight adjacent intermediate samples through a 128-bit lane group. + /// + /// The first intermediate sample. + /// The distance between intermediate rows. + /// The first column offset. + /// The first signed Q7 coefficient. + /// The number of active coefficients. + /// The initial convolution bias. + /// Receives the first four completed sums. + /// Receives the next four completed sums. + public static abstract void Convolve( + ref short source, + int sourceStride, + nuint sourceOffset, + ref short coefficients, + int coefficientCount, + Vector128 initial, + out Vector128 result0, + out Vector128 result1); + + /// + /// Convolves sixteen adjacent intermediate samples through a 256-bit lane group. + /// + /// The first intermediate sample. + /// The distance between intermediate rows. + /// The first column offset. + /// The first signed Q7 coefficient. + /// The number of active coefficients. + /// The initial convolution bias. + /// Receives the first eight completed sums. + /// Receives the next eight completed sums. + public static abstract void Convolve( + ref short source, + int sourceStride, + nuint sourceOffset, + ref short coefficients, + int coefficientCount, + Vector256 initial, + out Vector256 result0, + out Vector256 result1); + + /// + /// Convolves thirty-two adjacent intermediate samples through a 512-bit lane group. + /// + /// The first intermediate sample. + /// The distance between intermediate rows. + /// The first column offset. + /// The first signed Q7 coefficient. + /// The number of active coefficients. + /// The initial convolution bias. + /// Receives the first sixteen completed sums. + /// Receives the next sixteen completed sums. + public static abstract void Convolve( + ref short source, + int sourceStride, + nuint sourceOffset, + ref short coefficients, + int coefficientCount, + Vector512 initial, + out Vector512 result0, + out Vector512 result1); + } + + /// + /// Defines the output domain produced by variable-phase scaled prediction. + /// + private interface IAv1ScaledPredictionOperator + { + /// + /// Gets the vertical convolution shift for the selected output domain. + /// + /// The horizontal convolution shift. + /// The vertical convolution shift. + public static abstract int GetVerticalRound(int horizontalRound); + + /// + /// Gets the bias removed after vertical convolution for the selected output domain. + /// + /// The biased intermediate precision. + /// The vertical convolution shift. + /// The bias removed before storing the result. + public static abstract int GetRoundOffset(int offsetBits, int verticalRound); + + /// + /// Stores one completed prediction in the selected output domain. + /// + /// The native sample storage type. + /// The first destination sample. + /// The destination offset. + /// The completed prediction. + /// The decoded sample precision. + public static abstract void Store(ref T destination, int index, int value, int bitDepth) + where T : unmanaged; + + /// + /// Stores eight completed predictions in the selected output domain. + /// + /// The native sample storage type. + /// The first destination sample. + /// The destination offset. + /// The first four completed predictions. + /// The next four completed predictions. + /// The decoded sample precision. + public static abstract void Store(ref T destination, int index, Vector128 result0, Vector128 result1, int bitDepth) + where T : unmanaged; + + /// + /// Stores sixteen completed predictions in the selected output domain. + /// + /// The native sample storage type. + /// The first destination sample. + /// The destination offset. + /// The first eight completed predictions. + /// The next eight completed predictions. + /// The decoded sample precision. + public static abstract void Store(ref T destination, int index, Vector256 result0, Vector256 result1, int bitDepth) + where T : unmanaged; + + /// + /// Stores thirty-two completed predictions in the selected output domain. + /// + /// The native sample storage type. + /// The first destination sample. + /// The destination offset. + /// The first sixteen completed predictions. + /// The next sixteen completed predictions. + /// The decoded sample precision. + public static abstract void Store(ref T destination, int index, Vector512 result0, Vector512 result1, int bitDepth) + where T : unmanaged; + } + + /// + /// Produces native-pixel scaled prediction for scalar and SIMD lane groups. + /// + private readonly struct NativeOperator : IAv1ScaledArithmeticOperator, IAv1ScaledPredictionOperator + { + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static int GetVerticalRound(int horizontalRound) + => (2 * FilterBits) - horizontalRound; + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static int GetRoundOffset(int offsetBits, int verticalRound) + => (1 << (offsetBits - verticalRound)) + (1 << (offsetBits - verticalRound - 1)); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static int Load(ref T source, int index) + where T : unmanaged + { + // The only closed forms are byte and ushort. The JIT removes this storage choice from each specialization, + // leaving the shared variable-phase traversal free of duplicate 8-bit and high-bit-depth implementations. + if (typeof(T) == typeof(byte)) + { + return Unsafe.Add(ref Unsafe.As(ref source), index); + } + + return Unsafe.Add(ref Unsafe.As(ref source), index); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static int MultiplyAdd(int accumulator, int sample, int coefficient) + => accumulator + (sample * coefficient); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector128 MultiplyAdd(Vector128 accumulator, Vector128 samples, Vector128 coefficients) + => accumulator + (samples * coefficients); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector256 MultiplyAdd(Vector256 accumulator, Vector256 samples, Vector256 coefficients) + => accumulator + (samples * coefficients); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector512 MultiplyAdd(Vector512 accumulator, Vector512 samples, Vector512 coefficients) + => accumulator + (samples * coefficients); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static int Convolve(ref short source, int sourceStride, ref short coefficients, int coefficientCount) + => ConvolveScalar(ref source, sourceStride, ref coefficients, coefficientCount); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void Convolve( + ref short source, + int sourceStride, + nuint sourceOffset, + ref short coefficients, + int coefficientCount, + Vector128 initial, + out Vector128 result0, + out Vector128 result1) + => Av1TranslationalInterPredictor.Convolve( + ref source, + sourceStride, + sourceOffset, + ref coefficients, + coefficientCount, + initial, + out result0, + out result1); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void Convolve( + ref short source, + int sourceStride, + nuint sourceOffset, + ref short coefficients, + int coefficientCount, + Vector256 initial, + out Vector256 result0, + out Vector256 result1) + => Av1TranslationalInterPredictor.Convolve( + ref source, + sourceStride, + sourceOffset, + ref coefficients, + coefficientCount, + initial, + out result0, + out result1); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void Convolve( + ref short source, + int sourceStride, + nuint sourceOffset, + ref short coefficients, + int coefficientCount, + Vector512 initial, + out Vector512 result0, + out Vector512 result1) + => Av1TranslationalInterPredictor.Convolve( + ref source, + sourceStride, + sourceOffset, + ref coefficients, + coefficientCount, + initial, + out result0, + out result1); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void Store(ref T destination, int index, int value, int bitDepth) + where T : unmanaged + { + if (typeof(T) == typeof(byte)) + { + Unsafe.Add(ref Unsafe.As(ref destination), index) = (byte)Math.Clamp(value, byte.MinValue, byte.MaxValue); + return; + } + + Unsafe.Add(ref Unsafe.As(ref destination), index) = (ushort)Math.Clamp(value, 0, (1 << bitDepth) - 1); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void Store(ref T destination, int index, Vector128 result0, Vector128 result1, int bitDepth) + where T : unmanaged + { + if (typeof(T) == typeof(byte)) + { + PackBytes(result0, result1, Vector128.Zero, Vector128.Zero) + .GetLower() + .StoreUnsafe(ref Unsafe.As(ref destination), (nuint)index); + + return; + } + + PackHighBitDepth(result0, result1, (1 << bitDepth) - 1) + .StoreUnsafe(ref Unsafe.As(ref destination), (nuint)index); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void Store(ref T destination, int index, Vector256 result0, Vector256 result1, int bitDepth) + where T : unmanaged + { + if (typeof(T) == typeof(byte)) + { + PackBytes(result0, result1, Vector256.Zero, Vector256.Zero) + .GetLower() + .StoreUnsafe(ref Unsafe.As(ref destination), (nuint)index); + + return; + } + + PackHighBitDepth(result0, result1, (1 << bitDepth) - 1) + .StoreUnsafe(ref Unsafe.As(ref destination), (nuint)index); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void Store(ref T destination, int index, Vector512 result0, Vector512 result1, int bitDepth) + where T : unmanaged + { + if (typeof(T) == typeof(byte)) + { + PackBytes(result0, result1, Vector512.Zero, Vector512.Zero) + .GetLower() + .StoreUnsafe(ref Unsafe.As(ref destination), (nuint)index); + + return; + } + + PackHighBitDepth(result0, result1, (1 << bitDepth) - 1) + .StoreUnsafe(ref Unsafe.As(ref destination), (nuint)index); + } + } + + /// + /// Produces no-round compound intermediates for scalar and SIMD lane groups. + /// + private readonly struct CompoundOperator : IAv1ScaledPredictionOperator + { + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static int GetVerticalRound(int horizontalRound) + => Av1CompoundInterPredictor.CompoundRound1Bits; + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static int GetRoundOffset(int offsetBits, int verticalRound) => 0; + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void Store(ref T destination, int index, int value, int bitDepth) + where T : unmanaged + { + // Compound entry points close T as ushort. Their no-round values retain the positive convolution bias, + // so storing the normative unsigned intermediate needs neither pixel clipping nor a storage-type branch. + Unsafe.Add(ref Unsafe.As(ref destination), index) = (ushort)value; + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void Store( + ref T destination, + int index, + Vector128 result0, + Vector128 result1, + int bitDepth) + where T : unmanaged + => Av1NonDirectionalIntraPredictorBase.Narrow(result0, result1) + .AsUInt16() + .StoreUnsafe(ref Unsafe.As(ref destination), (nuint)index); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void Store( + ref T destination, + int index, + Vector256 result0, + Vector256 result1, + int bitDepth) + where T : unmanaged + => Av1NonDirectionalIntraPredictorBase.Narrow(result0, result1) + .AsUInt16() + .StoreUnsafe(ref Unsafe.As(ref destination), (nuint)index); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void Store( + ref T destination, + int index, + Vector512 result0, + Vector512 result1, + int bitDepth) + where T : unmanaged + => Av1NonDirectionalIntraPredictorBase.Narrow(result0, result1) + .AsUInt16() + .StoreUnsafe(ref Unsafe.As(ref destination), (nuint)index); + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1ScaledInterPredictor.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1ScaledInterPredictor.cs new file mode 100644 index 0000000000..a0414c7853 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1ScaledInterPredictor.cs @@ -0,0 +1,765 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.InteropServices; +using System.Runtime.Intrinsics; + +using static SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter.Av1TranslationalInterPredictor; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; + +/// +/// Reconstructs reference-scaled inter prediction through variable-phase separable convolution. +/// +internal static partial class Av1ScaledInterPredictor +{ + /// + /// Gets the scratch capacity required by one scaled prediction block. + /// + public static int GetScaledScratchLength(int width, int height, int verticalPhase, int verticalStep) + { + int intermediateHeight = ((((height - 1) * verticalStep) + verticalPhase) >> Av1ReferenceScale.SubpixelBits) + FilterCoefficientCount; + return Math.Max(width, Vector128.Count) * intermediateHeight; + } + + /// + /// Gets a dimension-only upper bound for one scaled prediction block's scratch capacity. + /// + public static int GetMaximumScaledScratchLength(int width, int height) + => Math.Max(width, Vector128.Count) * ((height * 2) + FilterCoefficientCount); + + /// + /// Reconstructs an 8-bit scaled prediction using variable source positions and phases. + /// + public static void PredictScaled( + ReadOnlySpan source, + int sourceStride, + int sourceOrigin, + Span destination, + int destinationStride, + int width, + int height, + Av1InterpolationFilter horizontalFilter, + Av1InterpolationFilter verticalFilter, + int horizontalPhase, + int horizontalStep, + int verticalPhase, + int verticalStep, + Span scratch) + => DispatchScaled( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + horizontalFilter, + verticalFilter, + horizontalPhase, + horizontalStep, + verticalPhase, + verticalStep, + 8, + scratch); + + /// + /// Reconstructs an 8-, 10-, or 12-bit scaled prediction using variable source positions and phases. + /// + public static void PredictScaled( + ReadOnlySpan source, + int sourceStride, + int sourceOrigin, + Span destination, + int destinationStride, + int width, + int height, + Av1InterpolationFilter horizontalFilter, + Av1InterpolationFilter verticalFilter, + int horizontalPhase, + int horizontalStep, + int verticalPhase, + int verticalStep, + int bitDepth, + Span scratch) + => DispatchScaled( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + horizontalFilter, + verticalFilter, + horizontalPhase, + horizontalStep, + verticalPhase, + verticalStep, + bitDepth, + scratch); + + /// + /// Reconstructs an 8-bit scaled predictor into the no-round compound intermediate domain. + /// + public static void PredictScaledCompound( + ReadOnlySpan source, + int sourceStride, + int sourceOrigin, + Span destination, + int destinationStride, + int width, + int height, + Av1InterpolationFilter horizontalFilter, + Av1InterpolationFilter verticalFilter, + int horizontalPhase, + int horizontalStep, + int verticalPhase, + int verticalStep, + Span scratch) + => DispatchScaled( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + horizontalFilter, + verticalFilter, + horizontalPhase, + horizontalStep, + verticalPhase, + verticalStep, + 8, + scratch); + + /// + /// Reconstructs an 8-, 10-, or 12-bit scaled predictor into the no-round compound intermediate domain. + /// + public static void PredictScaledCompound( + ReadOnlySpan source, + int sourceStride, + int sourceOrigin, + Span destination, + int destinationStride, + int width, + int height, + Av1InterpolationFilter horizontalFilter, + Av1InterpolationFilter verticalFilter, + int horizontalPhase, + int horizontalStep, + int verticalPhase, + int verticalStep, + int bitDepth, + Span scratch) + => DispatchScaled( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + horizontalFilter, + verticalFilter, + horizontalPhase, + horizontalStep, + verticalPhase, + verticalStep, + bitDepth, + scratch); + + /// + /// Selects the horizontal filter family for scaled prediction. + /// + private static void DispatchScaled( + ReadOnlySpan source, + int sourceStride, + int sourceOrigin, + Span destination, + int destinationStride, + int width, + int height, + Av1InterpolationFilter horizontalFilter, + Av1InterpolationFilter verticalFilter, + int horizontalPhase, + int horizontalStep, + int verticalPhase, + int verticalStep, + int bitDepth, + Span scratch) + where TSource : unmanaged + where TDestination : unmanaged + where TOperator : struct, IAv1ScaledPredictionOperator + { + switch (horizontalFilter) + { + case Av1InterpolationFilter.Regular: + DispatchScaledVertical( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + verticalFilter, + horizontalPhase, + horizontalStep, + verticalPhase, + verticalStep, + bitDepth, + scratch); + + break; + case Av1InterpolationFilter.Smooth: + DispatchScaledVertical( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + verticalFilter, + horizontalPhase, + horizontalStep, + verticalPhase, + verticalStep, + bitDepth, + scratch); + + break; + case Av1InterpolationFilter.Sharp: + DispatchScaledVertical( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + verticalFilter, + horizontalPhase, + horizontalStep, + verticalPhase, + verticalStep, + bitDepth, + scratch); + + break; + default: + DispatchScaledVertical( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + verticalFilter, + horizontalPhase, + horizontalStep, + verticalPhase, + verticalStep, + bitDepth, + scratch); + + break; + } + } + + /// + /// Selects the vertical filter family for a closed horizontal scaled-prediction operator. + /// + private static void DispatchScaledVertical( + ReadOnlySpan source, + int sourceStride, + int sourceOrigin, + Span destination, + int destinationStride, + int width, + int height, + Av1InterpolationFilter verticalFilter, + int horizontalPhase, + int horizontalStep, + int verticalPhase, + int verticalStep, + int bitDepth, + Span scratch) + where TSource : unmanaged + where TDestination : unmanaged + where TOperator : struct, IAv1ScaledPredictionOperator + where THorizontal : struct, IAv1InterPredictorOperator + { + switch (verticalFilter) + { + case Av1InterpolationFilter.Regular: + PredictScaled( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + horizontalPhase, + horizontalStep, + verticalPhase, + verticalStep, + bitDepth, + scratch); + + break; + case Av1InterpolationFilter.Smooth: + PredictScaled( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + horizontalPhase, + horizontalStep, + verticalPhase, + verticalStep, + bitDepth, + scratch); + + break; + case Av1InterpolationFilter.Sharp: + PredictScaled( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + horizontalPhase, + horizontalStep, + verticalPhase, + verticalStep, + bitDepth, + scratch); + + break; + default: + PredictScaled( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + horizontalPhase, + horizontalStep, + verticalPhase, + verticalStep, + bitDepth, + scratch); + + break; + } + } + + /// + /// Applies variable-phase horizontal filtering followed by variable-phase vertical filtering. + /// + private static void PredictScaled( + ReadOnlySpan source, + int sourceStride, + int sourceOrigin, + Span destination, + int destinationStride, + int width, + int height, + int horizontalPhase, + int horizontalStep, + int verticalPhase, + int verticalStep, + int bitDepth, + Span scratch) + where TSource : unmanaged + where TDestination : unmanaged + where TOperator : struct, IAv1ScaledPredictionOperator + where THorizontal : struct, IAv1InterPredictorOperator + where TVertical : struct, IAv1InterPredictorOperator + { + ref TSource sourceBase = ref Unsafe.Add(ref MemoryMarshal.GetReference(source), sourceOrigin); + ref TDestination destinationBase = ref MemoryMarshal.GetReference(destination); + ref short scratchBase = ref MemoryMarshal.GetReference(scratch); + int scratchStride = Math.Max(width, Vector128.Count); + int intermediateHeight = ((((height - 1) * verticalStep) + verticalPhase) >> Av1ReferenceScale.SubpixelBits) + FilterCoefficientCount; + int horizontalBias = 1 << (bitDepth + FilterBits - 1); + int intermediateRange = bitDepth + FilterBits - Round0Bits + 2; + int round0 = Round0Bits + Math.Max(intermediateRange - 16, 0); + bool useReducedHorizontalFilter = width <= 4; + bool useReducedVerticalFilter = height <= 4; + + // Scaled positions change both the integer source sample and filter phase at each output column. Four-lane + // vectors gather those independent positions into one multiply-accumulate chain without allocating an index map. + for (int row = 0; row < intermediateHeight; row++) + { + ref TSource sourceRow = ref Unsafe.Add(ref sourceBase, (row - 3) * sourceStride); + ref short scratchRow = ref Unsafe.Add(ref scratchBase, row * scratchStride); + int column = 0; + + if (Vector512.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector512Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector512.Count) + { + Vector512 result = FilterScaledHorizontalVector512( + ref sourceRow, + horizontalPhase, + horizontalStep, + column, + useReducedHorizontalFilter, + horizontalBias, + round0); + + Av1NonDirectionalIntraPredictorBase.Narrow(result, Vector512.Zero) + .GetLower() + .StoreUnsafe(ref scratchRow, (nuint)column); + } + } + + if (Vector256.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector256Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector256.Count) + { + Vector256 result = FilterScaledHorizontalVector256( + ref sourceRow, + horizontalPhase, + horizontalStep, + column, + useReducedHorizontalFilter, + horizontalBias, + round0); + + Av1NonDirectionalIntraPredictorBase.Narrow(result, Vector256.Zero) + .GetLower() + .StoreUnsafe(ref scratchRow, (nuint)column); + } + } + + if (Vector128.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector128Count(width - column); + for (; vectorCount > 0; vectorCount--, column += Vector128.Count) + { + Vector128 result = FilterScaledHorizontalVector128( + ref sourceRow, + horizontalPhase, + horizontalStep, + column, + useReducedHorizontalFilter, + horizontalBias, + round0); + + Av1NonDirectionalIntraPredictorBase.Narrow(result, Vector128.Zero) + .GetLower() + .StoreUnsafe(ref scratchRow, (nuint)column); + } + } + + for (; column < width; column++) + { + int position = horizontalPhase + (column * horizontalStep); + int sourceColumn = (position >> Av1ReferenceScale.SubpixelBits) - 3; + ReadOnlySpan coefficients = THorizontal.GetCoefficients( + (position & Av1ReferenceScale.SubpixelMask) >> 6, + useReducedHorizontalFilter); + + int sum = horizontalBias; + for (int tap = 0; tap < FilterCoefficientCount; tap++) + { + sum = NativeOperator.MultiplyAdd(sum, NativeOperator.Load(ref sourceRow, sourceColumn + tap), coefficients[tap]); + } + + Unsafe.Add(ref scratchRow, column) = (short)RoundPowerOfTwo(sum, round0); + } + } + + int round1 = TOperator.GetVerticalRound(round0); + int offsetBits = bitDepth + (2 * FilterBits) - round0; + int verticalBias = 1 << offsetBits; + int roundOffset = TOperator.GetRoundOffset(offsetBits, round1); + for (int row = 0; row < height; row++) + { + int position = verticalPhase + (row * verticalStep); + int sourceRowIndex = position >> Av1ReferenceScale.SubpixelBits; + ReadOnlySpan coefficients = TVertical.GetCoefficients( + (position & Av1ReferenceScale.SubpixelMask) >> 6, + useReducedVerticalFilter); + + ref short scratchRow = ref Unsafe.Add(ref scratchBase, sourceRowIndex * scratchStride); + ref short coefficientBase = ref MemoryMarshal.GetReference(coefficients); + ref TDestination destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + int column = 0; + + if (Vector512.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector512Count(width - column) / 2; + + if (vectorCount > 0) + { + // The scaled vertical kernel emits two register widths per iteration, so constants are needed only + // when the remaining row contains at least two complete vectors. + Vector512 initial = Vector512.Create(verticalBias); + Vector512 offset = Vector512.Create(roundOffset); + + for (; vectorCount > 0; vectorCount--, column += Vector512.Count * 2) + { + NativeOperator.Convolve( + ref scratchRow, + scratchStride, + (nuint)column, + ref coefficientBase, + FilterCoefficientCount, + initial, + out Vector512 result0, + out Vector512 result1); + + result0 = RoundPowerOfTwo(result0, round1) - offset; + result1 = RoundPowerOfTwo(result1, round1) - offset; + TOperator.Store(ref destinationRow, column, result0, result1, bitDepth); + } + } + } + + if (Vector256.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector256Count(width - column) / 2; + + if (vectorCount > 0) + { + // A single-vector remainder belongs to the next narrower tier and does not materialize YMM constants. + Vector256 initial = Vector256.Create(verticalBias); + Vector256 offset = Vector256.Create(roundOffset); + + for (; vectorCount > 0; vectorCount--, column += Vector256.Count * 2) + { + NativeOperator.Convolve( + ref scratchRow, + scratchStride, + (nuint)column, + ref coefficientBase, + FilterCoefficientCount, + initial, + out Vector256 result0, + out Vector256 result1); + + result0 = RoundPowerOfTwo(result0, round1) - offset; + result1 = RoundPowerOfTwo(result1, round1) - offset; + TOperator.Store(ref destinationRow, column, result0, result1, bitDepth); + } + } + } + + if (Vector128.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector128Count(width - column) / 2; + + if (vectorCount > 0) + { + // XMM constants are likewise skipped when fewer than eight output samples remain. + Vector128 initial = Vector128.Create(verticalBias); + Vector128 offset = Vector128.Create(roundOffset); + + for (; vectorCount > 0; vectorCount--, column += Vector128.Count * 2) + { + NativeOperator.Convolve( + ref scratchRow, + scratchStride, + (nuint)column, + ref coefficientBase, + FilterCoefficientCount, + initial, + out Vector128 result0, + out Vector128 result1); + + result0 = RoundPowerOfTwo(result0, round1) - offset; + result1 = RoundPowerOfTwo(result1, round1) - offset; + TOperator.Store(ref destinationRow, column, result0, result1, bitDepth); + } + } + } + + for (; column < width; column++) + { + int sum = verticalBias + NativeOperator.Convolve( + ref Unsafe.Add(ref scratchRow, column), + scratchStride, + ref coefficientBase, + FilterCoefficientCount); + + TOperator.Store( + ref destinationRow, + column, + RoundPowerOfTwo(sum, round1) - roundOffset, + bitDepth); + } + } + } + + /// + /// Filters four independently positioned horizontal samples through a closed scaled-prediction operator. + /// + private static Vector128 FilterScaledHorizontalVector128( + ref T source, + int phase, + int step, + int column, + bool useReducedFilter, + int bias, + int round) + where T : unmanaged + where TFilter : struct, IAv1InterPredictorOperator + { + Vector128 result = Vector128.Create(bias); + for (int tap = 0; tap < FilterCoefficientCount; tap++) + { + result = NativeOperator.MultiplyAdd( + result, + LoadScaledSamplesVector128(ref source, phase, step, column, tap), + LoadScaledCoefficientsVector128(phase, step, column, tap, useReducedFilter)); + } + + return RoundPowerOfTwo(result, round); + } + + /// + /// Filters eight independently positioned horizontal samples through a closed scaled-prediction operator. + /// + private static Vector256 FilterScaledHorizontalVector256( + ref T source, + int phase, + int step, + int column, + bool useReducedFilter, + int bias, + int round) + where T : unmanaged + where TFilter : struct, IAv1InterPredictorOperator + { + Vector256 result = Vector256.Create(bias); + for (int tap = 0; tap < FilterCoefficientCount; tap++) + { + Vector256 samples = Vector256.Create( + LoadScaledSamplesVector128(ref source, phase, step, column, tap), + LoadScaledSamplesVector128(ref source, phase, step, column + Vector128.Count, tap)); + + Vector256 coefficients = Vector256.Create( + LoadScaledCoefficientsVector128(phase, step, column, tap, useReducedFilter), + LoadScaledCoefficientsVector128(phase, step, column + Vector128.Count, tap, useReducedFilter)); + + result = NativeOperator.MultiplyAdd(result, samples, coefficients); + } + + return RoundPowerOfTwo(result, round); + } + + /// + /// Filters sixteen independently positioned horizontal samples through a closed scaled-prediction operator. + /// + private static Vector512 FilterScaledHorizontalVector512( + ref T source, + int phase, + int step, + int column, + bool useReducedFilter, + int bias, + int round) + where T : unmanaged + where TFilter : struct, IAv1InterPredictorOperator + { + Vector512 result = Vector512.Create(bias); + for (int tap = 0; tap < FilterCoefficientCount; tap++) + { + Vector256 sampleLower = Vector256.Create( + LoadScaledSamplesVector128(ref source, phase, step, column, tap), + LoadScaledSamplesVector128(ref source, phase, step, column + Vector128.Count, tap)); + + Vector256 sampleUpper = Vector256.Create( + LoadScaledSamplesVector128(ref source, phase, step, column + Vector256.Count, tap), + LoadScaledSamplesVector128(ref source, phase, step, column + Vector256.Count + Vector128.Count, tap)); + + Vector256 coefficientLower = Vector256.Create( + LoadScaledCoefficientsVector128(phase, step, column, tap, useReducedFilter), + LoadScaledCoefficientsVector128(phase, step, column + Vector128.Count, tap, useReducedFilter)); + + Vector256 coefficientUpper = Vector256.Create( + LoadScaledCoefficientsVector128(phase, step, column + Vector256.Count, tap, useReducedFilter), + LoadScaledCoefficientsVector128(phase, step, column + Vector256.Count + Vector128.Count, tap, useReducedFilter)); + + result = NativeOperator.MultiplyAdd( + result, + Vector512.Create(sampleLower, sampleUpper), + Vector512.Create(coefficientLower, coefficientUpper)); + } + + return RoundPowerOfTwo(result, round); + } + + /// + /// Gathers four variable-position source samples for one horizontal filter tap. + /// + private static Vector128 LoadScaledSamplesVector128( + ref T source, + int phase, + int step, + int column, + int tap) + where T : unmanaged + => Vector128.Create( + LoadScaledSample(ref source, phase, step, column, tap), + LoadScaledSample(ref source, phase, step, column + 1, tap), + LoadScaledSample(ref source, phase, step, column + 2, tap), + LoadScaledSample(ref source, phase, step, column + 3, tap)); + + /// + /// Gathers four variable-phase coefficients for one horizontal filter tap. + /// + private static Vector128 LoadScaledCoefficientsVector128( + int phase, + int step, + int column, + int tap, + bool useReducedFilter) + where TFilter : struct, IAv1InterPredictorOperator + => Vector128.Create( + LoadScaledCoefficient(phase, step, column, tap, useReducedFilter), + LoadScaledCoefficient(phase, step, column + 1, tap, useReducedFilter), + LoadScaledCoefficient(phase, step, column + 2, tap, useReducedFilter), + LoadScaledCoefficient(phase, step, column + 3, tap, useReducedFilter)); + + /// + /// Loads one variable-position source sample for a horizontal filter tap. + /// + private static int LoadScaledSample(ref T source, int phase, int step, int column, int tap) + where T : unmanaged + { + int position = phase + (column * step); + int sourceColumn = (position >> Av1ReferenceScale.SubpixelBits) - 3; + return NativeOperator.Load(ref source, sourceColumn + tap); + } + + /// + /// Loads one variable-phase horizontal filter coefficient. + /// + private static int LoadScaledCoefficient(int phase, int step, int column, int tap, bool useReducedFilter) + where TFilter : struct, IAv1InterPredictorOperator + { + int position = phase + (column * step); + return TFilter.GetCoefficients( + (position & Av1ReferenceScale.SubpixelMask) >> 6, + useReducedFilter)[tap]; + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1TranslationalInterPredictor.Arithmetic.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1TranslationalInterPredictor.Arithmetic.cs new file mode 100644 index 0000000000..cdf0d57f90 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1TranslationalInterPredictor.Arithmetic.cs @@ -0,0 +1,347 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; + +/// +/// Provides lane-wise convolution, rounding, clipping, and packing shared by every interpolation filter. +/// +internal static partial class Av1TranslationalInterPredictor +{ + /// + /// Convolves sixteen adjacent 8-bit samples into four signed 32-bit accumulator vectors. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void Convolve( + ref byte source, + int tapStride, + nuint column, + ref short coefficients, + int tapCount, + Vector128 initial, + out Vector128 result0, + out Vector128 result1, + out Vector128 result2, + out Vector128 result3) + { + result0 = initial; + result1 = initial; + result2 = initial; + result3 = initial; + + for (int tap = 0; tap < tapCount; tap++) + { + Vector128 samples = Vector128.LoadUnsafe(ref Unsafe.Add(ref source, tap * tapStride), column); + Av1NonDirectionalIntraPredictorBase.Widen(samples, out Vector128 samples0, out Vector128 samples1, out Vector128 samples2, out Vector128 samples3); + Vector128 coefficient = Vector128.Create((int)Unsafe.Add(ref coefficients, tap)); + + // Each widened vector retains four consecutive source columns. Applying the same tap coefficient to all + // lanes evaluates sixteen independent finite-impulse-response filters without a horizontal reduction. + result0 += samples0 * coefficient; + result1 += samples1 * coefficient; + result2 += samples2 * coefficient; + result3 += samples3 * coefficient; + } + } + + /// + /// Convolves thirty-two adjacent 8-bit samples into four signed 32-bit accumulator vectors. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void Convolve( + ref byte source, + int tapStride, + nuint column, + ref short coefficients, + int tapCount, + Vector256 initial, + out Vector256 result0, + out Vector256 result1, + out Vector256 result2, + out Vector256 result3) + { + result0 = initial; + result1 = initial; + result2 = initial; + result3 = initial; + + for (int tap = 0; tap < tapCount; tap++) + { + Vector256 samples = Vector256.LoadUnsafe(ref Unsafe.Add(ref source, tap * tapStride), column); + Av1NonDirectionalIntraPredictorBase.Widen(samples, out Vector256 samples0, out Vector256 samples1, out Vector256 samples2, out Vector256 samples3); + Vector256 coefficient = Vector256.Create((int)Unsafe.Add(ref coefficients, tap)); + + result0 += samples0 * coefficient; + result1 += samples1 * coefficient; + result2 += samples2 * coefficient; + result3 += samples3 * coefficient; + } + } + + /// + /// Convolves sixty-four adjacent 8-bit samples into four signed 32-bit accumulator vectors. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void Convolve( + ref byte source, + int tapStride, + nuint column, + ref short coefficients, + int tapCount, + Vector512 initial, + out Vector512 result0, + out Vector512 result1, + out Vector512 result2, + out Vector512 result3) + { + result0 = initial; + result1 = initial; + result2 = initial; + result3 = initial; + + for (int tap = 0; tap < tapCount; tap++) + { + Vector512 samples = Vector512.LoadUnsafe(ref Unsafe.Add(ref source, tap * tapStride), column); + Av1NonDirectionalIntraPredictorBase.Widen(samples, out Vector512 samples0, out Vector512 samples1, out Vector512 samples2, out Vector512 samples3); + Vector512 coefficient = Vector512.Create((int)Unsafe.Add(ref coefficients, tap)); + + result0 += samples0 * coefficient; + result1 += samples1 * coefficient; + result2 += samples2 * coefficient; + result3 += samples3 * coefficient; + } + } + + /// + /// Convolves eight adjacent nonnegative 16-bit samples into two signed 32-bit accumulator vectors. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void Convolve( + ref short source, + int tapStride, + nuint column, + ref short coefficients, + int tapCount, + Vector128 initial, + out Vector128 result0, + out Vector128 result1) + { + result0 = initial; + result1 = initial; + + for (int tap = 0; tap < tapCount; tap++) + { + Vector128 samples = Vector128.LoadUnsafe(ref Unsafe.Add(ref source, tap * tapStride), column); + Av1NonDirectionalIntraPredictorBase.Widen(samples, out Vector128 samples0, out Vector128 samples1); + Vector128 coefficient = Vector128.Create((int)Unsafe.Add(ref coefficients, tap)); + + // Reconstructed 10- and 12-bit samples and biased 2D intermediates are below short.MaxValue, so signed + // widening preserves their values while allowing negative interpolation coefficients. + result0 += samples0 * coefficient; + result1 += samples1 * coefficient; + } + } + + /// + /// Convolves sixteen adjacent nonnegative 16-bit samples into two signed 32-bit accumulator vectors. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void Convolve( + ref short source, + int tapStride, + nuint column, + ref short coefficients, + int tapCount, + Vector256 initial, + out Vector256 result0, + out Vector256 result1) + { + result0 = initial; + result1 = initial; + + for (int tap = 0; tap < tapCount; tap++) + { + Vector256 samples = Vector256.LoadUnsafe(ref Unsafe.Add(ref source, tap * tapStride), column); + Av1NonDirectionalIntraPredictorBase.Widen(samples, out Vector256 samples0, out Vector256 samples1); + Vector256 coefficient = Vector256.Create((int)Unsafe.Add(ref coefficients, tap)); + result0 += samples0 * coefficient; + result1 += samples1 * coefficient; + } + } + + /// + /// Convolves thirty-two adjacent nonnegative 16-bit samples into two signed 32-bit accumulator vectors. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void Convolve( + ref short source, + int tapStride, + nuint column, + ref short coefficients, + int tapCount, + Vector512 initial, + out Vector512 result0, + out Vector512 result1) + { + result0 = initial; + result1 = initial; + + for (int tap = 0; tap < tapCount; tap++) + { + Vector512 samples = Vector512.LoadUnsafe(ref Unsafe.Add(ref source, tap * tapStride), column); + Av1NonDirectionalIntraPredictorBase.Widen(samples, out Vector512 samples0, out Vector512 samples1); + Vector512 coefficient = Vector512.Create((int)Unsafe.Add(ref coefficients, tap)); + result0 += samples0 * coefficient; + result1 += samples1 * coefficient; + } + } + + /// + /// Applies AV1 power-of-two rounding to four-lane signed accumulators. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector128 RoundPowerOfTwo(Vector128 value, int bits) + => bits == 0 ? value : (value + Vector128.Create(1 << (bits - 1))) >> bits; + + /// + /// Applies AV1 power-of-two rounding to eight-lane signed accumulators. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector256 RoundPowerOfTwo(Vector256 value, int bits) + => bits == 0 ? value : (value + Vector256.Create(1 << (bits - 1))) >> bits; + + /// + /// Applies AV1 power-of-two rounding to sixteen-lane signed accumulators. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector512 RoundPowerOfTwo(Vector512 value, int bits) + => bits == 0 ? value : (value + Vector512.Create(1 << (bits - 1))) >> bits; + + /// + /// Clips and packs sixteen signed accumulators into 8-bit samples. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector128 PackBytes(Vector128 result0, Vector128 result1, Vector128 result2, Vector128 result3) + { + Vector128 maximum = Vector128.Create((int)byte.MaxValue); + result0 = Vector128.Clamp(result0, Vector128.Zero, maximum); + result1 = Vector128.Clamp(result1, Vector128.Zero, maximum); + result2 = Vector128.Clamp(result2, Vector128.Zero, maximum); + result3 = Vector128.Clamp(result3, Vector128.Zero, maximum); + return Av1NonDirectionalIntraPredictorBase.Narrow(result0, result1, result2, result3); + } + + /// + /// Clips and packs thirty-two signed accumulators into 8-bit samples. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector256 PackBytes(Vector256 result0, Vector256 result1, Vector256 result2, Vector256 result3) + { + Vector256 maximum = Vector256.Create((int)byte.MaxValue); + result0 = Vector256.Clamp(result0, Vector256.Zero, maximum); + result1 = Vector256.Clamp(result1, Vector256.Zero, maximum); + result2 = Vector256.Clamp(result2, Vector256.Zero, maximum); + result3 = Vector256.Clamp(result3, Vector256.Zero, maximum); + return Av1NonDirectionalIntraPredictorBase.Narrow(result0, result1, result2, result3); + } + + /// + /// Clips and packs sixty-four signed accumulators into 8-bit samples. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector512 PackBytes(Vector512 result0, Vector512 result1, Vector512 result2, Vector512 result3) + { + Vector512 maximum = Vector512.Create((int)byte.MaxValue); + result0 = Vector512.Clamp(result0, Vector512.Zero, maximum); + result1 = Vector512.Clamp(result1, Vector512.Zero, maximum); + result2 = Vector512.Clamp(result2, Vector512.Zero, maximum); + result3 = Vector512.Clamp(result3, Vector512.Zero, maximum); + return Av1NonDirectionalIntraPredictorBase.Narrow(result0, result1, result2, result3); + } + + /// + /// Clips and packs eight signed accumulators into high-bit-depth samples. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector128 PackHighBitDepth(Vector128 result0, Vector128 result1, int maximumValue) + { + Vector128 maximum = Vector128.Create(maximumValue); + result0 = Vector128.Clamp(result0, Vector128.Zero, maximum); + result1 = Vector128.Clamp(result1, Vector128.Zero, maximum); + return Av1NonDirectionalIntraPredictorBase.Narrow(result0, result1).AsUInt16(); + } + + /// + /// Clips and packs sixteen signed accumulators into high-bit-depth samples. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector256 PackHighBitDepth(Vector256 result0, Vector256 result1, int maximumValue) + { + Vector256 maximum = Vector256.Create(maximumValue); + result0 = Vector256.Clamp(result0, Vector256.Zero, maximum); + result1 = Vector256.Clamp(result1, Vector256.Zero, maximum); + return Av1NonDirectionalIntraPredictorBase.Narrow(result0, result1).AsUInt16(); + } + + /// + /// Clips and packs thirty-two signed accumulators into high-bit-depth samples. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector512 PackHighBitDepth(Vector512 result0, Vector512 result1, int maximumValue) + { + Vector512 maximum = Vector512.Create(maximumValue); + result0 = Vector512.Clamp(result0, Vector512.Zero, maximum); + result1 = Vector512.Clamp(result1, Vector512.Zero, maximum); + return Av1NonDirectionalIntraPredictorBase.Narrow(result0, result1).AsUInt16(); + } + + /// + /// Computes one signed Q7 convolution sum from 8-bit samples. + /// + public static int ConvolveScalar(ref byte source, int sourceStride, ref short coefficients, int tapCount) + { + int sum = 0; + for (int tap = 0; tap < tapCount; tap++) + { + sum += Unsafe.Add(ref coefficients, tap) * Unsafe.Add(ref source, tap * sourceStride); + } + + return sum; + } + + /// + /// Computes one signed Q7 convolution sum from high-bit-depth samples. + /// + public static int ConvolveScalar(ref ushort source, int sourceStride, ref short coefficients, int tapCount) + { + int sum = 0; + for (int tap = 0; tap < tapCount; tap++) + { + sum += Unsafe.Add(ref coefficients, tap) * Unsafe.Add(ref source, tap * sourceStride); + } + + return sum; + } + + /// + /// Computes one signed Q7 convolution sum from biased intermediate samples. + /// + public static int ConvolveScalar(ref short source, int sourceStride, ref short coefficients, int tapCount) + { + int sum = 0; + for (int tap = 0; tap < tapCount; tap++) + { + sum += Unsafe.Add(ref coefficients, tap) * Unsafe.Add(ref source, tap * sourceStride); + } + + return sum; + } + + /// + /// Rounds an integer after division by a power of two using AV1's unsigned-bias rule. + /// + public static int RoundPowerOfTwo(int value, int bits) => bits == 0 ? value : (value + (1 << (bits - 1))) >> bits; +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1TranslationalInterPredictor.BilinearOperator.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1TranslationalInterPredictor.BilinearOperator.cs new file mode 100644 index 0000000000..3cc1a08695 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1TranslationalInterPredictor.BilinearOperator.cs @@ -0,0 +1,20 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; + +/// +/// Defines bilinear interpolation for translational inter prediction. +/// +internal static partial class Av1TranslationalInterPredictor +{ + /// + /// Selects bilinear interpolation coefficients. + /// + internal readonly struct BilinearOperator : IAv1InterPredictorOperator + { + /// + public static ReadOnlySpan GetCoefficients(int phase, bool useReducedFilter) + => GetPhase(Bilinear, phase); + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1TranslationalInterPredictor.Dispatch.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1TranslationalInterPredictor.Dispatch.cs new file mode 100644 index 0000000000..26c3346fb8 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1TranslationalInterPredictor.Dispatch.cs @@ -0,0 +1,1020 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Numerics; +using System.Runtime.CompilerServices; +using System.Runtime.InteropServices; +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; + +/// +/// Selects interpolation filters and the widest supported traversal for a translational prediction block. +/// +internal static partial class Av1TranslationalInterPredictor +{ + /// + /// Selects an 8-bit horizontal interpolation operator. + /// + private static void Dispatch( + ReadOnlySpan source, + int sourceStride, + int sourceOrigin, + Span destination, + int destinationStride, + int width, + int height, + Av1InterpolationFilter horizontalFilter, + Av1InterpolationFilter verticalFilter, + int horizontalPhase, + int verticalPhase, + Span scratch) + { + switch (horizontalFilter) + { + case Av1InterpolationFilter.Regular: + DispatchVertical( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + verticalFilter, + horizontalPhase, + verticalPhase, + scratch); + + break; + case Av1InterpolationFilter.Smooth: + DispatchVertical( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + verticalFilter, + horizontalPhase, + verticalPhase, + scratch); + + break; + case Av1InterpolationFilter.Sharp: + DispatchVertical( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + verticalFilter, + horizontalPhase, + verticalPhase, + scratch); + + break; + default: + DispatchVertical( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + verticalFilter, + horizontalPhase, + verticalPhase, + scratch); + + break; + } + } + + /// + /// Selects a high-bit-depth horizontal interpolation operator. + /// + private static void Dispatch( + ReadOnlySpan source, + int sourceStride, + int sourceOrigin, + Span destination, + int destinationStride, + int width, + int height, + Av1InterpolationFilter horizontalFilter, + Av1InterpolationFilter verticalFilter, + int horizontalPhase, + int verticalPhase, + int bitDepth, + Span scratch) + { + switch (horizontalFilter) + { + case Av1InterpolationFilter.Regular: + DispatchVertical( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + verticalFilter, + horizontalPhase, + verticalPhase, + bitDepth, + scratch); + + break; + case Av1InterpolationFilter.Smooth: + DispatchVertical( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + verticalFilter, + horizontalPhase, + verticalPhase, + bitDepth, + scratch); + + break; + case Av1InterpolationFilter.Sharp: + DispatchVertical( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + verticalFilter, + horizontalPhase, + verticalPhase, + bitDepth, + scratch); + + break; + default: + DispatchVertical( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + verticalFilter, + horizontalPhase, + verticalPhase, + bitDepth, + scratch); + + break; + } + } + + /// + /// Selects a closed 8-bit horizontal interpolation operator for explicit scalar execution. + /// + private static void DispatchScalar( + ReadOnlySpan source, + int sourceStride, + int sourceOrigin, + Span destination, + int destinationStride, + int width, + int height, + Av1InterpolationFilter horizontalFilter, + Av1InterpolationFilter verticalFilter, + int horizontalPhase, + int verticalPhase, + Span scratch) + { + // The benchmark/test entry point closes the same production operators explicitly, but terminates in the + // scalar kernels without carrying a runtime mode flag through the SIMD-first decoder path. + switch (horizontalFilter) + { + case Av1InterpolationFilter.Regular: + DispatchVerticalScalar( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + verticalFilter, + horizontalPhase, + verticalPhase, + scratch); + + break; + case Av1InterpolationFilter.Smooth: + DispatchVerticalScalar( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + verticalFilter, + horizontalPhase, + verticalPhase, + scratch); + + break; + case Av1InterpolationFilter.Sharp: + DispatchVerticalScalar( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + verticalFilter, + horizontalPhase, + verticalPhase, + scratch); + + break; + default: + DispatchVerticalScalar( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + verticalFilter, + horizontalPhase, + verticalPhase, + scratch); + + break; + } + } + + /// + /// Selects a closed high-bit-depth horizontal interpolation operator for explicit scalar execution. + /// + private static void DispatchScalar( + ReadOnlySpan source, + int sourceStride, + int sourceOrigin, + Span destination, + int destinationStride, + int width, + int height, + Av1InterpolationFilter horizontalFilter, + Av1InterpolationFilter verticalFilter, + int horizontalPhase, + int verticalPhase, + int bitDepth, + Span scratch) + { + // Closing the production table operators here keeps scalar parity coverage on the same normative Q7 data. + switch (horizontalFilter) + { + case Av1InterpolationFilter.Regular: + DispatchVerticalScalar( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + verticalFilter, + horizontalPhase, + verticalPhase, + bitDepth, + scratch); + + break; + case Av1InterpolationFilter.Smooth: + DispatchVerticalScalar( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + verticalFilter, + horizontalPhase, + verticalPhase, + bitDepth, + scratch); + + break; + case Av1InterpolationFilter.Sharp: + DispatchVerticalScalar( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + verticalFilter, + horizontalPhase, + verticalPhase, + bitDepth, + scratch); + + break; + default: + DispatchVerticalScalar( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + verticalFilter, + horizontalPhase, + verticalPhase, + bitDepth, + scratch); + + break; + } + } + + /// + /// Copies an 8-bit integer-position block using the widest vector that fits a complete row prefix. + /// + private static void Copy( + ReadOnlySpan source, + int sourceStride, + int sourceOrigin, + Span destination, + int destinationStride, + int width, + int height) + { + ref byte sourceBase = ref Unsafe.Add(ref MemoryMarshal.GetReference(source), sourceOrigin); + ref byte destinationBase = ref MemoryMarshal.GetReference(destination); + + if (Vector512.IsHardwareAccelerated && Vector.Count == Vector512.Count && width >= Vector512.Count) + { + int vectorEnd = (int)(Numerics.Vector512Count(width) * (nuint)Vector512.Count); + for (int row = 0; row < height; row++) + { + ref byte sourceRow = ref Unsafe.Add(ref sourceBase, row * sourceStride); + ref byte destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + int column = 0; + + for (; column < vectorEnd; column += Vector512.Count) + { + Vector512.LoadUnsafe(ref sourceRow, (nuint)column).StoreUnsafe(ref destinationRow, (nuint)column); + } + + for (; column < width; column++) + { + Unsafe.Add(ref destinationRow, column) = Unsafe.Add(ref sourceRow, column); + } + } + + return; + } + + if (Vector256.IsHardwareAccelerated && width >= Vector256.Count) + { + int vectorEnd = (int)(Numerics.Vector256Count(width) * (nuint)Vector256.Count); + for (int row = 0; row < height; row++) + { + ref byte sourceRow = ref Unsafe.Add(ref sourceBase, row * sourceStride); + ref byte destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + int column = 0; + + for (; column < vectorEnd; column += Vector256.Count) + { + Vector256.LoadUnsafe(ref sourceRow, (nuint)column).StoreUnsafe(ref destinationRow, (nuint)column); + } + + for (; column < width; column++) + { + Unsafe.Add(ref destinationRow, column) = Unsafe.Add(ref sourceRow, column); + } + } + + return; + } + + if (Vector128.IsHardwareAccelerated) + { + for (int row = 0; row < height; row++) + { + ref byte sourceRow = ref Unsafe.Add(ref sourceBase, row * sourceStride); + ref byte destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + + if (width < Vector128.Count) + { + StorePartial(Vector128.LoadUnsafe(ref sourceRow), ref destinationRow, width); + continue; + } + + int vectorEnd = (int)(Numerics.Vector128Count(width) * (nuint)Vector128.Count); + int column = 0; + for (; column < vectorEnd; column += Vector128.Count) + { + Vector128.LoadUnsafe(ref sourceRow, (nuint)column).StoreUnsafe(ref destinationRow, (nuint)column); + } + + for (; column < width; column++) + { + Unsafe.Add(ref destinationRow, column) = Unsafe.Add(ref sourceRow, column); + } + } + + return; + } + + CopyScalar(source, sourceStride, sourceOrigin, destination, destinationStride, width, height); + } + + /// + /// Copies a high-bit-depth integer-position block using the widest vector that fits a complete row prefix. + /// + private static void Copy( + ReadOnlySpan source, + int sourceStride, + int sourceOrigin, + Span destination, + int destinationStride, + int width, + int height) + { + ref ushort sourceBase = ref Unsafe.Add(ref MemoryMarshal.GetReference(source), sourceOrigin); + ref ushort destinationBase = ref MemoryMarshal.GetReference(destination); + + if (Vector512.IsHardwareAccelerated && Vector.Count == Vector512.Count && width >= Vector512.Count) + { + int vectorEnd = (int)(Numerics.Vector512Count(width) * (nuint)Vector512.Count); + for (int row = 0; row < height; row++) + { + ref ushort sourceRow = ref Unsafe.Add(ref sourceBase, row * sourceStride); + ref ushort destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + int column = 0; + + for (; column < vectorEnd; column += Vector512.Count) + { + Vector512.LoadUnsafe(ref sourceRow, (nuint)column).StoreUnsafe(ref destinationRow, (nuint)column); + } + + for (; column < width; column++) + { + Unsafe.Add(ref destinationRow, column) = Unsafe.Add(ref sourceRow, column); + } + } + + return; + } + + if (Vector256.IsHardwareAccelerated && width >= Vector256.Count) + { + int vectorEnd = (int)(Numerics.Vector256Count(width) * (nuint)Vector256.Count); + for (int row = 0; row < height; row++) + { + ref ushort sourceRow = ref Unsafe.Add(ref sourceBase, row * sourceStride); + ref ushort destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + int column = 0; + + for (; column < vectorEnd; column += Vector256.Count) + { + Vector256.LoadUnsafe(ref sourceRow, (nuint)column).StoreUnsafe(ref destinationRow, (nuint)column); + } + + for (; column < width; column++) + { + Unsafe.Add(ref destinationRow, column) = Unsafe.Add(ref sourceRow, column); + } + } + + return; + } + + if (Vector128.IsHardwareAccelerated) + { + for (int row = 0; row < height; row++) + { + ref ushort sourceRow = ref Unsafe.Add(ref sourceBase, row * sourceStride); + ref ushort destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + + if (width < Vector128.Count) + { + // Subsampled sub-8x8 chroma can be two samples wide, so retain the vector load while limiting + // the store to the logical row width. + StorePartial(Vector128.LoadUnsafe(ref sourceRow), ref destinationRow, width); + continue; + } + + int vectorEnd = (int)(Numerics.Vector128Count(width) * (nuint)Vector128.Count); + int column = 0; + for (; column < vectorEnd; column += Vector128.Count) + { + Vector128.LoadUnsafe(ref sourceRow, (nuint)column).StoreUnsafe(ref destinationRow, (nuint)column); + } + + for (; column < width; column++) + { + Unsafe.Add(ref destinationRow, column) = Unsafe.Add(ref sourceRow, column); + } + } + + return; + } + + CopyScalar(source, sourceStride, sourceOrigin, destination, destinationStride, width, height); + } + + /// + /// Applies a one-dimensional 8-bit filter using one SIMD width for the complete block. + /// + private static void FilterDirect( + ReadOnlySpan source, + int sourceStride, + int sourceOrigin, + Span destination, + int destinationStride, + int width, + int height, + ReadOnlySpan coefficients, + int tapCount, + int sourceOffset, + int tapStride, + int firstRound, + int secondRound) + { + if (Vector512.IsHardwareAccelerated && Vector.Count == Vector512.Count && width >= Vector512.Count) + { + FilterDirect( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + coefficients, + tapCount, + sourceOffset, + tapStride, + firstRound, + secondRound, + Vector512.Zero); + + return; + } + + if (Vector256.IsHardwareAccelerated && width >= Vector256.Count) + { + FilterDirect( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + coefficients, + tapCount, + sourceOffset, + tapStride, + firstRound, + secondRound, + Vector256.Zero); + + return; + } + + if (Vector128.IsHardwareAccelerated) + { + FilterDirect( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + coefficients, + tapCount, + sourceOffset, + tapStride, + firstRound, + secondRound, + Vector128.Zero); + + return; + } + + FilterDirectScalar( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + coefficients, + tapCount, + sourceOffset, + tapStride, + firstRound, + secondRound); + } + + /// + /// Applies a one-dimensional high-bit-depth filter using one SIMD width for the complete block. + /// + private static void FilterDirect( + ReadOnlySpan source, + int sourceStride, + int sourceOrigin, + Span destination, + int destinationStride, + int width, + int height, + ReadOnlySpan coefficients, + int tapCount, + int sourceOffset, + int tapStride, + int firstRound, + int secondRound, + int bitDepth) + { + if (Vector512.IsHardwareAccelerated && Vector.Count == Vector512.Count && width >= Vector512.Count) + { + FilterDirect( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + coefficients, + tapCount, + sourceOffset, + tapStride, + firstRound, + secondRound, + bitDepth, + Vector512.Zero); + + return; + } + + if (Vector256.IsHardwareAccelerated && width >= Vector256.Count) + { + FilterDirect( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + coefficients, + tapCount, + sourceOffset, + tapStride, + firstRound, + secondRound, + bitDepth, + Vector256.Zero); + + return; + } + + if (Vector128.IsHardwareAccelerated) + { + FilterDirect( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + coefficients, + tapCount, + sourceOffset, + tapStride, + firstRound, + secondRound, + bitDepth, + Vector128.Zero); + + return; + } + + FilterDirectScalar( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + coefficients, + tapCount, + sourceOffset, + tapStride, + firstRound, + secondRound, + bitDepth); + } + + /// + /// Applies separable two-dimensional filtering to an 8-bit block using one SIMD width. + /// + private static void Filter2D( + ReadOnlySpan source, + int sourceStride, + int sourceOrigin, + Span destination, + int destinationStride, + int width, + int height, + ReadOnlySpan horizontalCoefficients, + int horizontalTapCount, + int horizontalSourceOffset, + ReadOnlySpan verticalCoefficients, + int verticalTapCount, + int verticalSourceOffset, + int bitDepth, + Span scratch) + { + if (Vector512.IsHardwareAccelerated && Vector.Count == Vector512.Count && width >= Vector512.Count) + { + Filter2D( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + horizontalCoefficients, + horizontalTapCount, + horizontalSourceOffset, + verticalCoefficients, + verticalTapCount, + verticalSourceOffset, + bitDepth, + Round0Bits, + scratch, + Vector512.Zero); + + return; + } + + if (Vector256.IsHardwareAccelerated && width >= Vector256.Count) + { + Filter2D( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + horizontalCoefficients, + horizontalTapCount, + horizontalSourceOffset, + verticalCoefficients, + verticalTapCount, + verticalSourceOffset, + bitDepth, + Round0Bits, + scratch, + Vector256.Zero); + + return; + } + + if (Vector128.IsHardwareAccelerated) + { + Filter2D( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + horizontalCoefficients, + horizontalTapCount, + horizontalSourceOffset, + verticalCoefficients, + verticalTapCount, + verticalSourceOffset, + bitDepth, + Round0Bits, + scratch, + Vector128.Zero); + + return; + } + + Filter2DScalar( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + horizontalCoefficients, + horizontalTapCount, + horizontalSourceOffset, + verticalCoefficients, + verticalTapCount, + verticalSourceOffset, + bitDepth, + Round0Bits, + scratch); + } + + /// + /// Applies separable two-dimensional filtering to a high-bit-depth block using one SIMD width. + /// + private static void Filter2D( + ReadOnlySpan source, + int sourceStride, + int sourceOrigin, + Span destination, + int destinationStride, + int width, + int height, + ReadOnlySpan horizontalCoefficients, + int horizontalTapCount, + int horizontalSourceOffset, + ReadOnlySpan verticalCoefficients, + int verticalTapCount, + int verticalSourceOffset, + int bitDepth, + int round0, + Span scratch) + { + if (Vector512.IsHardwareAccelerated && Vector.Count == Vector512.Count && width >= Vector512.Count) + { + Filter2D( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + horizontalCoefficients, + horizontalTapCount, + horizontalSourceOffset, + verticalCoefficients, + verticalTapCount, + verticalSourceOffset, + bitDepth, + round0, + scratch, + Vector512.Zero); + + return; + } + + if (Vector256.IsHardwareAccelerated && width >= Vector256.Count) + { + Filter2D( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + horizontalCoefficients, + horizontalTapCount, + horizontalSourceOffset, + verticalCoefficients, + verticalTapCount, + verticalSourceOffset, + bitDepth, + round0, + scratch, + Vector256.Zero); + + return; + } + + if (Vector128.IsHardwareAccelerated) + { + Filter2D( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + horizontalCoefficients, + horizontalTapCount, + horizontalSourceOffset, + verticalCoefficients, + verticalTapCount, + verticalSourceOffset, + bitDepth, + round0, + scratch, + Vector128.Zero); + + return; + } + + Filter2DScalar( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + horizontalCoefficients, + horizontalTapCount, + horizontalSourceOffset, + verticalCoefficients, + verticalTapCount, + verticalSourceOffset, + bitDepth, + round0, + scratch); + } + + /// + /// Copies an 8-bit integer-position prediction without explicit hardware intrinsics. + /// + private static void CopyScalar( + ReadOnlySpan source, + int sourceStride, + int sourceOrigin, + Span destination, + int destinationStride, + int width, + int height) + { + ref byte sourceBase = ref Unsafe.Add(ref MemoryMarshal.GetReference(source), sourceOrigin); + ref byte destinationBase = ref MemoryMarshal.GetReference(destination); + + for (int row = 0; row < height; row++) + { + ref byte sourceRow = ref Unsafe.Add(ref sourceBase, row * sourceStride); + ref byte destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + + for (int column = 0; column < width; column++) + { + Unsafe.Add(ref destinationRow, column) = Unsafe.Add(ref sourceRow, column); + } + } + } + + /// + /// Copies a high-bit-depth integer-position prediction without explicit hardware intrinsics. + /// + private static void CopyScalar( + ReadOnlySpan source, + int sourceStride, + int sourceOrigin, + Span destination, + int destinationStride, + int width, + int height) + { + ref ushort sourceBase = ref Unsafe.Add(ref MemoryMarshal.GetReference(source), sourceOrigin); + ref ushort destinationBase = ref MemoryMarshal.GetReference(destination); + + for (int row = 0; row < height; row++) + { + ref ushort sourceRow = ref Unsafe.Add(ref sourceBase, row * sourceStride); + ref ushort destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + + for (int column = 0; column < width; column++) + { + Unsafe.Add(ref destinationRow, column) = Unsafe.Add(ref sourceRow, column); + } + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1TranslationalInterPredictor.Filters.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1TranslationalInterPredictor.Filters.cs new file mode 100644 index 0000000000..9e919704da --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1TranslationalInterPredictor.Filters.cs @@ -0,0 +1,153 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; + +/// +/// Provides the normative Q7 interpolation coefficients used by AV1 inter prediction. +/// +internal static partial class Av1TranslationalInterPredictor +{ + /// + /// The number of stored coefficient positions in every decoder interpolation kernel. + /// + internal const int FilterCoefficientCount = 8; + + /// + /// Gets the regular eight-tap kernels for the sixteen subpixel phases. + /// + private static ReadOnlySpan RegularEightTap => + [ + 0, 0, 0, 128, 0, 0, 0, 0, + 0, 2, -6, 126, 8, -2, 0, 0, + 0, 2, -10, 122, 18, -4, 0, 0, + 0, 2, -12, 116, 28, -8, 2, 0, + 0, 2, -14, 110, 38, -10, 2, 0, + 0, 2, -14, 102, 48, -12, 2, 0, + 0, 2, -16, 94, 58, -12, 2, 0, + 0, 2, -14, 84, 66, -12, 2, 0, + 0, 2, -14, 76, 76, -14, 2, 0, + 0, 2, -12, 66, 84, -14, 2, 0, + 0, 2, -12, 58, 94, -16, 2, 0, + 0, 2, -12, 48, 102, -14, 2, 0, + 0, 2, -10, 38, 110, -14, 2, 0, + 0, 2, -8, 28, 116, -12, 2, 0, + 0, 0, -4, 18, 122, -10, 2, 0, + 0, 0, -2, 8, 126, -6, 2, 0, + ]; + + /// + /// Gets the smooth eight-tap kernels for the sixteen subpixel phases. + /// + private static ReadOnlySpan SmoothEightTap => + [ + 0, 0, 0, 128, 0, 0, 0, 0, + 0, 2, 28, 62, 34, 2, 0, 0, + 0, 0, 26, 62, 36, 4, 0, 0, + 0, 0, 22, 62, 40, 4, 0, 0, + 0, 0, 20, 60, 42, 6, 0, 0, + 0, 0, 18, 58, 44, 8, 0, 0, + 0, 0, 16, 56, 46, 10, 0, 0, + 0, -2, 16, 54, 48, 12, 0, 0, + 0, -2, 14, 52, 52, 14, -2, 0, + 0, 0, 12, 48, 54, 16, -2, 0, + 0, 0, 10, 46, 56, 16, 0, 0, + 0, 0, 8, 44, 58, 18, 0, 0, + 0, 0, 6, 42, 60, 20, 0, 0, + 0, 0, 4, 40, 62, 22, 0, 0, + 0, 0, 4, 36, 62, 26, 0, 0, + 0, 0, 2, 34, 62, 28, 2, 0, + ]; + + /// + /// Gets the sharp eight-tap kernels for the sixteen subpixel phases. + /// + private static ReadOnlySpan SharpEightTap => + [ + 0, 0, 0, 128, 0, 0, 0, 0, + -2, 2, -6, 126, 8, -2, 2, 0, + -2, 6, -12, 124, 16, -6, 4, -2, + -2, 8, -18, 120, 26, -10, 6, -2, + -4, 10, -22, 116, 38, -14, 6, -2, + -4, 10, -22, 108, 48, -18, 8, -2, + -4, 10, -24, 100, 60, -20, 8, -2, + -4, 10, -24, 90, 70, -22, 10, -2, + -4, 12, -24, 80, 80, -24, 12, -4, + -2, 10, -22, 70, 90, -24, 10, -4, + -2, 8, -20, 60, 100, -24, 10, -4, + -2, 8, -18, 48, 108, -22, 10, -4, + -2, 6, -14, 38, 116, -22, 10, -4, + -2, 6, -10, 26, 120, -18, 8, -2, + -2, 4, -6, 16, 124, -12, 6, -2, + 0, 2, -2, 8, 126, -6, 2, -2, + ]; + + /// + /// Gets the regular reduced kernels selected when a block dimension is at most four samples. + /// + private static ReadOnlySpan RegularFourTap => + [ + 0, 0, 0, 128, 0, 0, 0, 0, + 0, 0, -4, 126, 8, -2, 0, 0, + 0, 0, -8, 122, 18, -4, 0, 0, + 0, 0, -10, 116, 28, -6, 0, 0, + 0, 0, -12, 110, 38, -8, 0, 0, + 0, 0, -12, 102, 48, -10, 0, 0, + 0, 0, -14, 94, 58, -10, 0, 0, + 0, 0, -12, 84, 66, -10, 0, 0, + 0, 0, -12, 76, 76, -12, 0, 0, + 0, 0, -10, 66, 84, -12, 0, 0, + 0, 0, -10, 58, 94, -14, 0, 0, + 0, 0, -10, 48, 102, -12, 0, 0, + 0, 0, -8, 38, 110, -12, 0, 0, + 0, 0, -6, 28, 116, -10, 0, 0, + 0, 0, -4, 18, 122, -8, 0, 0, + 0, 0, -2, 8, 126, -4, 0, 0, + ]; + + /// + /// Gets the smooth reduced kernels selected when a block dimension is at most four samples. + /// + private static ReadOnlySpan SmoothFourTap => + [ + 0, 0, 0, 128, 0, 0, 0, 0, + 0, 0, 30, 62, 34, 2, 0, 0, + 0, 0, 26, 62, 36, 4, 0, 0, + 0, 0, 22, 62, 40, 4, 0, 0, + 0, 0, 20, 60, 42, 6, 0, 0, + 0, 0, 18, 58, 44, 8, 0, 0, + 0, 0, 16, 56, 46, 10, 0, 0, + 0, 0, 14, 54, 48, 12, 0, 0, + 0, 0, 12, 52, 52, 12, 0, 0, + 0, 0, 12, 48, 54, 14, 0, 0, + 0, 0, 10, 46, 56, 16, 0, 0, + 0, 0, 8, 44, 58, 18, 0, 0, + 0, 0, 6, 42, 60, 20, 0, 0, + 0, 0, 4, 40, 62, 22, 0, 0, + 0, 0, 4, 36, 62, 26, 0, 0, + 0, 0, 2, 34, 62, 30, 0, 0, + ]; + + /// + /// Gets the bilinear kernels for the sixteen subpixel phases. + /// + private static ReadOnlySpan Bilinear => + [ + 0, 0, 0, 128, 0, 0, 0, 0, + 0, 0, 0, 120, 8, 0, 0, 0, + 0, 0, 0, 112, 16, 0, 0, 0, + 0, 0, 0, 104, 24, 0, 0, 0, + 0, 0, 0, 96, 32, 0, 0, 0, + 0, 0, 0, 88, 40, 0, 0, 0, + 0, 0, 0, 80, 48, 0, 0, 0, + 0, 0, 0, 72, 56, 0, 0, 0, + 0, 0, 0, 64, 64, 0, 0, 0, + 0, 0, 0, 56, 72, 0, 0, 0, + 0, 0, 0, 48, 80, 0, 0, 0, + 0, 0, 0, 40, 88, 0, 0, 0, + 0, 0, 0, 32, 96, 0, 0, 0, + 0, 0, 0, 24, 104, 0, 0, 0, + 0, 0, 0, 16, 112, 0, 0, 0, + 0, 0, 0, 8, 120, 0, 0, 0, + ]; +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1TranslationalInterPredictor.OneDimension.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1TranslationalInterPredictor.OneDimension.cs new file mode 100644 index 0000000000..e25e8bbbd8 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1TranslationalInterPredictor.OneDimension.cs @@ -0,0 +1,565 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.InteropServices; +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; + +/// +/// Provides SIMD kernels for horizontal-only and vertical-only single-reference filtering. +/// +internal static partial class Av1TranslationalInterPredictor +{ + /// + /// Filters an 8-bit block in sixteen-sample vectors. + /// + private static void FilterDirect( + ReadOnlySpan source, + int sourceStride, + int sourceOrigin, + Span destination, + int destinationStride, + int width, + int height, + ReadOnlySpan coefficients, + int tapCount, + int sourceOffset, + int tapStride, + int firstRound, + int secondRound, + Vector128 initial) + { + ref byte sourceBase = ref Unsafe.Add(ref MemoryMarshal.GetReference(source), sourceOrigin); + ref byte destinationBase = ref MemoryMarshal.GetReference(destination); + ref short coefficientBase = ref MemoryMarshal.GetReference(coefficients); + + for (int row = 0; row < height; row++) + { + ref byte sourceRow = ref Unsafe.Add(ref sourceBase, (row * sourceStride) + sourceOffset); + ref byte destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + int processedColumns = 0; + + if (width < Vector128.Count) + { + // Four- and eight-sample AV1 blocks are smaller than one byte vector. Reference-plane padding makes + // the complete load readable, while the width-specific store leaves adjacent destination samples intact. + Convolve( + ref sourceRow, + tapStride, + 0, + ref coefficientBase, + tapCount, + initial, + out Vector128 result0, + out Vector128 result1, + out Vector128 result2, + out Vector128 result3); + + Round(ref result0, ref result1, ref result2, ref result3, firstRound, secondRound); + StorePartial(PackBytes(result0, result1, result2, result3), ref destinationRow, width); + continue; + } + + nuint vectorCount = Numerics.Vector128Count(width - processedColumns); + for (; vectorCount > 0; vectorCount--, processedColumns += Vector128.Count) + { + Convolve( + ref sourceRow, + tapStride, + (nuint)processedColumns, + ref coefficientBase, + tapCount, + initial, + out Vector128 result0, + out Vector128 result1, + out Vector128 result2, + out Vector128 result3); + + Round(ref result0, ref result1, ref result2, ref result3, firstRound, secondRound); + PackBytes(result0, result1, result2, result3).StoreUnsafe(ref destinationRow, (nuint)processedColumns); + } + + FilterDirectTail(ref sourceRow, ref destinationRow, processedColumns, width, tapStride, ref coefficientBase, tapCount, firstRound, secondRound); + } + } + + /// + /// Filters an 8-bit block in thirty-two-sample vectors. + /// + private static void FilterDirect( + ReadOnlySpan source, + int sourceStride, + int sourceOrigin, + Span destination, + int destinationStride, + int width, + int height, + ReadOnlySpan coefficients, + int tapCount, + int sourceOffset, + int tapStride, + int firstRound, + int secondRound, + Vector256 initial) + { + ref byte sourceBase = ref Unsafe.Add(ref MemoryMarshal.GetReference(source), sourceOrigin); + ref byte destinationBase = ref MemoryMarshal.GetReference(destination); + ref short coefficientBase = ref MemoryMarshal.GetReference(coefficients); + int vectorEnd = (int)(Numerics.Vector256Count(width) * (nuint)Vector256.Count); + + for (int row = 0; row < height; row++) + { + ref byte sourceRow = ref Unsafe.Add(ref sourceBase, (row * sourceStride) + sourceOffset); + ref byte destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + int processedColumns = 0; + + for (; processedColumns < vectorEnd; processedColumns += Vector256.Count) + { + Convolve( + ref sourceRow, + tapStride, + (nuint)processedColumns, + ref coefficientBase, + tapCount, + initial, + out Vector256 result0, + out Vector256 result1, + out Vector256 result2, + out Vector256 result3); + + Round(ref result0, ref result1, ref result2, ref result3, firstRound, secondRound); + PackBytes(result0, result1, result2, result3).StoreUnsafe(ref destinationRow, (nuint)processedColumns); + } + + FilterDirectTail(ref sourceRow, ref destinationRow, processedColumns, width, tapStride, ref coefficientBase, tapCount, firstRound, secondRound); + } + } + + /// + /// Filters an 8-bit block in sixty-four-sample vectors. + /// + private static void FilterDirect( + ReadOnlySpan source, + int sourceStride, + int sourceOrigin, + Span destination, + int destinationStride, + int width, + int height, + ReadOnlySpan coefficients, + int tapCount, + int sourceOffset, + int tapStride, + int firstRound, + int secondRound, + Vector512 initial) + { + ref byte sourceBase = ref Unsafe.Add(ref MemoryMarshal.GetReference(source), sourceOrigin); + ref byte destinationBase = ref MemoryMarshal.GetReference(destination); + ref short coefficientBase = ref MemoryMarshal.GetReference(coefficients); + int vectorEnd = (int)(Numerics.Vector512Count(width) * (nuint)Vector512.Count); + + for (int row = 0; row < height; row++) + { + ref byte sourceRow = ref Unsafe.Add(ref sourceBase, (row * sourceStride) + sourceOffset); + ref byte destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + int processedColumns = 0; + + for (; processedColumns < vectorEnd; processedColumns += Vector512.Count) + { + Convolve( + ref sourceRow, + tapStride, + (nuint)processedColumns, + ref coefficientBase, + tapCount, + initial, + out Vector512 result0, + out Vector512 result1, + out Vector512 result2, + out Vector512 result3); + + Round(ref result0, ref result1, ref result2, ref result3, firstRound, secondRound); + PackBytes(result0, result1, result2, result3).StoreUnsafe(ref destinationRow, (nuint)processedColumns); + } + + FilterDirectTail(ref sourceRow, ref destinationRow, processedColumns, width, tapStride, ref coefficientBase, tapCount, firstRound, secondRound); + } + } + + /// + /// Filters a high-bit-depth block in eight-sample vectors. + /// + private static void FilterDirect( + ReadOnlySpan source, + int sourceStride, + int sourceOrigin, + Span destination, + int destinationStride, + int width, + int height, + ReadOnlySpan coefficients, + int tapCount, + int sourceOffset, + int tapStride, + int firstRound, + int secondRound, + int bitDepth, + Vector128 initial) + { + ref ushort sourceBase = ref Unsafe.Add(ref MemoryMarshal.GetReference(source), sourceOrigin); + ref ushort destinationBase = ref MemoryMarshal.GetReference(destination); + ref short coefficientBase = ref MemoryMarshal.GetReference(coefficients); + int maximum = (1 << bitDepth) - 1; + + for (int row = 0; row < height; row++) + { + ref ushort sourceRowUnsigned = ref Unsafe.Add(ref sourceBase, (row * sourceStride) + sourceOffset); + ref short sourceRow = ref Unsafe.As(ref sourceRowUnsigned); + ref ushort destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + int processedColumns = 0; + + if (width < Vector128.Count) + { + Convolve(ref sourceRow, tapStride, 0, ref coefficientBase, tapCount, initial, out Vector128 result0, out Vector128 result1); + Round(ref result0, ref result1, firstRound, secondRound); + + // Subsampled sub-8x8 chroma can be two samples wide. Keep the full source load for throughput, but + // store only the logical row so the adjacent luma owner's chroma prediction remains intact. + StorePartial(PackHighBitDepth(result0, result1, maximum), ref destinationRow, width); + continue; + } + + nuint vectorCount = Numerics.Vector128Count(width - processedColumns); + for (; vectorCount > 0; vectorCount--, processedColumns += Vector128.Count) + { + Convolve(ref sourceRow, tapStride, (nuint)processedColumns, ref coefficientBase, tapCount, initial, out Vector128 result0, out Vector128 result1); + Round(ref result0, ref result1, firstRound, secondRound); + PackHighBitDepth(result0, result1, maximum).StoreUnsafe(ref destinationRow, (nuint)processedColumns); + } + + FilterDirectTail(ref sourceRowUnsigned, ref destinationRow, processedColumns, width, tapStride, ref coefficientBase, tapCount, firstRound, secondRound, maximum); + } + } + + /// + /// Filters a high-bit-depth block in sixteen-sample vectors. + /// + private static void FilterDirect( + ReadOnlySpan source, + int sourceStride, + int sourceOrigin, + Span destination, + int destinationStride, + int width, + int height, + ReadOnlySpan coefficients, + int tapCount, + int sourceOffset, + int tapStride, + int firstRound, + int secondRound, + int bitDepth, + Vector256 initial) + { + ref ushort sourceBase = ref Unsafe.Add(ref MemoryMarshal.GetReference(source), sourceOrigin); + ref ushort destinationBase = ref MemoryMarshal.GetReference(destination); + ref short coefficientBase = ref MemoryMarshal.GetReference(coefficients); + int maximum = (1 << bitDepth) - 1; + int vectorEnd = (int)(Numerics.Vector256Count(width) * (nuint)Vector256.Count); + + for (int row = 0; row < height; row++) + { + ref ushort sourceRowUnsigned = ref Unsafe.Add(ref sourceBase, (row * sourceStride) + sourceOffset); + ref short sourceRow = ref Unsafe.As(ref sourceRowUnsigned); + ref ushort destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + int processedColumns = 0; + + for (; processedColumns < vectorEnd; processedColumns += Vector256.Count) + { + Convolve(ref sourceRow, tapStride, (nuint)processedColumns, ref coefficientBase, tapCount, initial, out Vector256 result0, out Vector256 result1); + Round(ref result0, ref result1, firstRound, secondRound); + PackHighBitDepth(result0, result1, maximum).StoreUnsafe(ref destinationRow, (nuint)processedColumns); + } + + FilterDirectTail(ref sourceRowUnsigned, ref destinationRow, processedColumns, width, tapStride, ref coefficientBase, tapCount, firstRound, secondRound, maximum); + } + } + + /// + /// Filters a high-bit-depth block in thirty-two-sample vectors. + /// + private static void FilterDirect( + ReadOnlySpan source, + int sourceStride, + int sourceOrigin, + Span destination, + int destinationStride, + int width, + int height, + ReadOnlySpan coefficients, + int tapCount, + int sourceOffset, + int tapStride, + int firstRound, + int secondRound, + int bitDepth, + Vector512 initial) + { + ref ushort sourceBase = ref Unsafe.Add(ref MemoryMarshal.GetReference(source), sourceOrigin); + ref ushort destinationBase = ref MemoryMarshal.GetReference(destination); + ref short coefficientBase = ref MemoryMarshal.GetReference(coefficients); + int maximum = (1 << bitDepth) - 1; + int vectorEnd = (int)(Numerics.Vector512Count(width) * (nuint)Vector512.Count); + + for (int row = 0; row < height; row++) + { + ref ushort sourceRowUnsigned = ref Unsafe.Add(ref sourceBase, (row * sourceStride) + sourceOffset); + ref short sourceRow = ref Unsafe.As(ref sourceRowUnsigned); + ref ushort destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + int processedColumns = 0; + + for (; processedColumns < vectorEnd; processedColumns += Vector512.Count) + { + Convolve(ref sourceRow, tapStride, (nuint)processedColumns, ref coefficientBase, tapCount, initial, out Vector512 result0, out Vector512 result1); + Round(ref result0, ref result1, firstRound, secondRound); + PackHighBitDepth(result0, result1, maximum).StoreUnsafe(ref destinationRow, (nuint)processedColumns); + } + + FilterDirectTail(ref sourceRowUnsigned, ref destinationRow, processedColumns, width, tapStride, ref coefficientBase, tapCount, firstRound, secondRound, maximum); + } + } + + /// + /// Applies the two direct-filter rounding stages to sixteen 8-bit results. + /// + private static void Round(ref Vector128 result0, ref Vector128 result1, ref Vector128 result2, ref Vector128 result3, int firstRound, int secondRound) + { + result0 = RoundPowerOfTwo(RoundPowerOfTwo(result0, firstRound), secondRound); + result1 = RoundPowerOfTwo(RoundPowerOfTwo(result1, firstRound), secondRound); + result2 = RoundPowerOfTwo(RoundPowerOfTwo(result2, firstRound), secondRound); + result3 = RoundPowerOfTwo(RoundPowerOfTwo(result3, firstRound), secondRound); + } + + /// + /// Applies the two direct-filter rounding stages to thirty-two 8-bit results. + /// + private static void Round(ref Vector256 result0, ref Vector256 result1, ref Vector256 result2, ref Vector256 result3, int firstRound, int secondRound) + { + result0 = RoundPowerOfTwo(RoundPowerOfTwo(result0, firstRound), secondRound); + result1 = RoundPowerOfTwo(RoundPowerOfTwo(result1, firstRound), secondRound); + result2 = RoundPowerOfTwo(RoundPowerOfTwo(result2, firstRound), secondRound); + result3 = RoundPowerOfTwo(RoundPowerOfTwo(result3, firstRound), secondRound); + } + + /// + /// Applies the two direct-filter rounding stages to sixty-four 8-bit results. + /// + private static void Round(ref Vector512 result0, ref Vector512 result1, ref Vector512 result2, ref Vector512 result3, int firstRound, int secondRound) + { + result0 = RoundPowerOfTwo(RoundPowerOfTwo(result0, firstRound), secondRound); + result1 = RoundPowerOfTwo(RoundPowerOfTwo(result1, firstRound), secondRound); + result2 = RoundPowerOfTwo(RoundPowerOfTwo(result2, firstRound), secondRound); + result3 = RoundPowerOfTwo(RoundPowerOfTwo(result3, firstRound), secondRound); + } + + /// + /// Applies the two direct-filter rounding stages to eight high-bit-depth results. + /// + private static void Round(ref Vector128 result0, ref Vector128 result1, int firstRound, int secondRound) + { + result0 = RoundPowerOfTwo(RoundPowerOfTwo(result0, firstRound), secondRound); + result1 = RoundPowerOfTwo(RoundPowerOfTwo(result1, firstRound), secondRound); + } + + /// + /// Applies the two direct-filter rounding stages to sixteen high-bit-depth results. + /// + private static void Round(ref Vector256 result0, ref Vector256 result1, int firstRound, int secondRound) + { + result0 = RoundPowerOfTwo(RoundPowerOfTwo(result0, firstRound), secondRound); + result1 = RoundPowerOfTwo(RoundPowerOfTwo(result1, firstRound), secondRound); + } + + /// + /// Applies the two direct-filter rounding stages to thirty-two high-bit-depth results. + /// + private static void Round(ref Vector512 result0, ref Vector512 result1, int firstRound, int secondRound) + { + result0 = RoundPowerOfTwo(RoundPowerOfTwo(result0, firstRound), secondRound); + result1 = RoundPowerOfTwo(RoundPowerOfTwo(result1, firstRound), secondRound); + } + + /// + /// Finishes an 8-bit row after its selected vector width. + /// + private static void FilterDirectTail( + ref byte source, + ref byte destination, + int firstColumn, + int width, + int tapStride, + ref short coefficients, + int tapCount, + int firstRound, + int secondRound) + { + for (int column = firstColumn; column < width; column++) + { + int sum = ConvolveScalar(ref Unsafe.Add(ref source, column), tapStride, ref coefficients, tapCount); + sum = RoundPowerOfTwo(sum, firstRound); + sum = RoundPowerOfTwo(sum, secondRound); + Unsafe.Add(ref destination, column) = (byte)Math.Clamp(sum, byte.MinValue, byte.MaxValue); + } + } + + /// + /// Finishes a high-bit-depth row after its selected vector width. + /// + private static void FilterDirectTail( + ref ushort source, + ref ushort destination, + int firstColumn, + int width, + int tapStride, + ref short coefficients, + int tapCount, + int firstRound, + int secondRound, + int maximum) + { + for (int column = firstColumn; column < width; column++) + { + int sum = ConvolveScalar(ref Unsafe.Add(ref source, column), tapStride, ref coefficients, tapCount); + sum = RoundPowerOfTwo(sum, firstRound); + sum = RoundPowerOfTwo(sum, secondRound); + Unsafe.Add(ref destination, column) = (ushort)Math.Clamp(sum, 0, maximum); + } + } + + /// + /// Stores the two-, four-, or eight-sample prefix of a sixteen-byte prediction vector. + /// + private static void StorePartial(Vector128 value, ref byte destination, int width) + { + if (width == 8) + { + value.GetLower().StoreUnsafe(ref destination); + } + else if (width == 4) + { + Unsafe.As(ref destination) = value.AsUInt32().GetElement(0); + } + else + { + Unsafe.As(ref destination) = value.AsUInt16().GetElement(0); + } + } + + /// + /// Stores the two-, four-, or eight-sample prefix of an eight-ushort prediction vector. + /// + private static void StorePartial(Vector128 value, ref ushort destination, int width) + { + if (width == 8) + { + value.StoreUnsafe(ref destination); + } + else if (width == 4) + { + value.GetLower().StoreUnsafe(ref destination); + } + else + { + Unsafe.As(ref destination) = value.AsUInt32().GetElement(0); + } + } + + /// + /// Applies a one-dimensional 8-bit filter without explicit hardware intrinsics. + /// + private static void FilterDirectScalar( + ReadOnlySpan source, + int sourceStride, + int sourceOrigin, + Span destination, + int destinationStride, + int width, + int height, + ReadOnlySpan coefficients, + int tapCount, + int sourceOffset, + int tapStride, + int firstRound, + int secondRound) + { + ref byte sourceBase = ref Unsafe.Add(ref MemoryMarshal.GetReference(source), sourceOrigin); + ref byte destinationBase = ref MemoryMarshal.GetReference(destination); + ref short coefficientBase = ref MemoryMarshal.GetReference(coefficients); + + for (int row = 0; row < height; row++) + { + ref byte sourceRow = ref Unsafe.Add(ref sourceBase, (row * sourceStride) + sourceOffset); + ref byte destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + + for (int column = 0; column < width; column++) + { + int sum = ConvolveScalar(ref Unsafe.Add(ref sourceRow, column), tapStride, ref coefficientBase, tapCount); + sum = RoundPowerOfTwo(sum, firstRound); + + if (secondRound != 0) + { + sum = RoundPowerOfTwo(sum, secondRound); + } + + Unsafe.Add(ref destinationRow, column) = (byte)Math.Clamp(sum, byte.MinValue, byte.MaxValue); + } + } + } + + /// + /// Applies a one-dimensional high-bit-depth filter without explicit hardware intrinsics. + /// + private static void FilterDirectScalar( + ReadOnlySpan source, + int sourceStride, + int sourceOrigin, + Span destination, + int destinationStride, + int width, + int height, + ReadOnlySpan coefficients, + int tapCount, + int sourceOffset, + int tapStride, + int firstRound, + int secondRound, + int bitDepth) + { + ref ushort sourceBase = ref Unsafe.Add(ref MemoryMarshal.GetReference(source), sourceOrigin); + ref ushort destinationBase = ref MemoryMarshal.GetReference(destination); + ref short coefficientBase = ref MemoryMarshal.GetReference(coefficients); + int maximum = (1 << bitDepth) - 1; + + for (int row = 0; row < height; row++) + { + ref ushort sourceRow = ref Unsafe.Add(ref sourceBase, (row * sourceStride) + sourceOffset); + ref ushort destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + + for (int column = 0; column < width; column++) + { + int sum = ConvolveScalar(ref Unsafe.Add(ref sourceRow, column), tapStride, ref coefficientBase, tapCount); + sum = RoundPowerOfTwo(sum, firstRound); + + if (secondRound != 0) + { + sum = RoundPowerOfTwo(sum, secondRound); + } + + Unsafe.Add(ref destinationRow, column) = (ushort)Math.Clamp(sum, 0, maximum); + } + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1TranslationalInterPredictor.Operator.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1TranslationalInterPredictor.Operator.cs new file mode 100644 index 0000000000..2aed6afbf3 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1TranslationalInterPredictor.Operator.cs @@ -0,0 +1,791 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; + +/// +/// Defines interpolation operators and the generic traversal used by translational inter prediction. +/// +internal static partial class Av1TranslationalInterPredictor +{ + /// + /// Supplies the normative Q7 coefficient kernel for one AV1 interpolation-filter family. + /// + /// + /// The closed operator type lets the JIT inline table selection into each horizontal and vertical filter pair. + /// Width reduction is selected once per block dimension rather than inside the sample loops. + /// + internal interface IAv1InterPredictorOperator + { + /// + /// Gets the eight coefficients for a one-sixteenth-sample phase. + /// + /// The fractional phase in the inclusive range zero through fifteen. + /// Indicates whether the coded block dimension is at most four samples. + /// The Q7 coefficients in increasing source-sample order. + public static abstract ReadOnlySpan GetCoefficients(int phase, bool useReducedFilter); + } + + /// + /// Selects an 8-bit vertical interpolation operator for a closed horizontal operator. + /// + /// The horizontal filter family. + private static void DispatchVertical( + ReadOnlySpan source, + int sourceStride, + int sourceOrigin, + Span destination, + int destinationStride, + int width, + int height, + Av1InterpolationFilter verticalFilter, + int horizontalPhase, + int verticalPhase, + Span scratch) + where THorizontal : struct, IAv1InterPredictorOperator + { + switch (verticalFilter) + { + case Av1InterpolationFilter.Regular: + Predict( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + horizontalPhase, + verticalPhase, + scratch); + + break; + case Av1InterpolationFilter.Smooth: + Predict( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + horizontalPhase, + verticalPhase, + scratch); + + break; + case Av1InterpolationFilter.Sharp: + Predict( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + horizontalPhase, + verticalPhase, + scratch); + + break; + default: + Predict( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + horizontalPhase, + verticalPhase, + scratch); + + break; + } + } + + /// + /// Selects a high-bit-depth vertical interpolation operator for a closed horizontal operator. + /// + /// The horizontal filter family. + private static void DispatchVertical( + ReadOnlySpan source, + int sourceStride, + int sourceOrigin, + Span destination, + int destinationStride, + int width, + int height, + Av1InterpolationFilter verticalFilter, + int horizontalPhase, + int verticalPhase, + int bitDepth, + Span scratch) + where THorizontal : struct, IAv1InterPredictorOperator + { + switch (verticalFilter) + { + case Av1InterpolationFilter.Regular: + Predict( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + horizontalPhase, + verticalPhase, + bitDepth, + scratch); + + break; + case Av1InterpolationFilter.Smooth: + Predict( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + horizontalPhase, + verticalPhase, + bitDepth, + scratch); + + break; + case Av1InterpolationFilter.Sharp: + Predict( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + horizontalPhase, + verticalPhase, + bitDepth, + scratch); + + break; + default: + Predict( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + horizontalPhase, + verticalPhase, + bitDepth, + scratch); + + break; + } + } + + /// + /// Selects an 8-bit vertical interpolation operator for explicit scalar execution. + /// + /// The closed horizontal filter family selected from the production operator set. + private static void DispatchVerticalScalar( + ReadOnlySpan source, + int sourceStride, + int sourceOrigin, + Span destination, + int destinationStride, + int width, + int height, + Av1InterpolationFilter verticalFilter, + int horizontalPhase, + int verticalPhase, + Span scratch) + where THorizontal : struct, IAv1InterPredictorOperator + { + switch (verticalFilter) + { + case Av1InterpolationFilter.Regular: + PredictScalar( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + horizontalPhase, + verticalPhase, + scratch); + + break; + case Av1InterpolationFilter.Smooth: + PredictScalar( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + horizontalPhase, + verticalPhase, + scratch); + + break; + case Av1InterpolationFilter.Sharp: + PredictScalar( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + horizontalPhase, + verticalPhase, + scratch); + + break; + default: + PredictScalar( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + horizontalPhase, + verticalPhase, + scratch); + + break; + } + } + + /// + /// Selects a high-bit-depth vertical interpolation operator for explicit scalar execution. + /// + /// The closed horizontal filter family selected from the production operator set. + private static void DispatchVerticalScalar( + ReadOnlySpan source, + int sourceStride, + int sourceOrigin, + Span destination, + int destinationStride, + int width, + int height, + Av1InterpolationFilter verticalFilter, + int horizontalPhase, + int verticalPhase, + int bitDepth, + Span scratch) + where THorizontal : struct, IAv1InterPredictorOperator + { + switch (verticalFilter) + { + case Av1InterpolationFilter.Regular: + PredictScalar( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + horizontalPhase, + verticalPhase, + bitDepth, + scratch); + + break; + case Av1InterpolationFilter.Smooth: + PredictScalar( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + horizontalPhase, + verticalPhase, + bitDepth, + scratch); + + break; + case Av1InterpolationFilter.Sharp: + PredictScalar( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + horizontalPhase, + verticalPhase, + bitDepth, + scratch); + + break; + default: + PredictScalar( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + horizontalPhase, + verticalPhase, + bitDepth, + scratch); + + break; + } + } + + /// + /// Executes one closed 8-bit interpolation-filter pair. + /// + /// The horizontal filter family. + /// The vertical filter family. + private static void Predict( + ReadOnlySpan source, + int sourceStride, + int sourceOrigin, + Span destination, + int destinationStride, + int width, + int height, + int horizontalPhase, + int verticalPhase, + Span scratch) + where THorizontal : struct, IAv1InterPredictorOperator + where TVertical : struct, IAv1InterPredictorOperator + { + if (horizontalPhase == 0 && verticalPhase == 0) + { + Copy(source, sourceStride, sourceOrigin, destination, destinationStride, width, height); + return; + } + + if (verticalPhase == 0) + { + ReadOnlySpan coefficients = THorizontal.GetCoefficients(horizontalPhase, width <= 4); + GetEffectiveKernel(coefficients, out int firstCoefficient, out int tapCount); + + FilterDirect( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + coefficients[firstCoefficient..], + tapCount, + firstCoefficient - 3, + 1, + Round0Bits, + FilterBits - Round0Bits); + + return; + } + + if (horizontalPhase == 0) + { + ReadOnlySpan coefficients = TVertical.GetCoefficients(verticalPhase, height <= 4); + GetEffectiveKernel(coefficients, out int firstCoefficient, out int tapCount); + + FilterDirect( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + coefficients[firstCoefficient..], + tapCount, + (firstCoefficient - 3) * sourceStride, + sourceStride, + FilterBits, + 0); + + return; + } + + ReadOnlySpan horizontalCoefficients = THorizontal.GetCoefficients(horizontalPhase, width <= 4); + ReadOnlySpan verticalCoefficients = TVertical.GetCoefficients(verticalPhase, height <= 4); + GetEffectiveKernel(horizontalCoefficients, out int firstHorizontalCoefficient, out int horizontalTapCount); + GetEffectiveKernel(verticalCoefficients, out int firstVerticalCoefficient, out int verticalTapCount); + + Filter2D( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + horizontalCoefficients[firstHorizontalCoefficient..], + horizontalTapCount, + firstHorizontalCoefficient - 3, + verticalCoefficients[firstVerticalCoefficient..], + verticalTapCount, + firstVerticalCoefficient - 3, + 8, + scratch); + } + + /// + /// Executes one closed high-bit-depth interpolation-filter pair. + /// + /// The horizontal filter family. + /// The vertical filter family. + private static void Predict( + ReadOnlySpan source, + int sourceStride, + int sourceOrigin, + Span destination, + int destinationStride, + int width, + int height, + int horizontalPhase, + int verticalPhase, + int bitDepth, + Span scratch) + where THorizontal : struct, IAv1InterPredictorOperator + where TVertical : struct, IAv1InterPredictorOperator + { + if (horizontalPhase == 0 && verticalPhase == 0) + { + Copy(source, sourceStride, sourceOrigin, destination, destinationStride, width, height); + return; + } + + // Twelve-bit samples require two additional first-pass rounding bits to keep the reference decoder's signed intermediate + // within sixteen bits. The second pass gives those bits back, preserving a total Q14 shift. + int intermediateRange = bitDepth + FilterBits - Round0Bits + 2; + int round0 = Round0Bits + Math.Max(intermediateRange - 16, 0); + + if (verticalPhase == 0) + { + ReadOnlySpan coefficients = THorizontal.GetCoefficients(horizontalPhase, width <= 4); + GetEffectiveKernel(coefficients, out int firstCoefficient, out int tapCount); + + FilterDirect( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + coefficients[firstCoefficient..], + tapCount, + firstCoefficient - 3, + 1, + round0, + FilterBits - round0, + bitDepth); + + return; + } + + if (horizontalPhase == 0) + { + ReadOnlySpan coefficients = TVertical.GetCoefficients(verticalPhase, height <= 4); + GetEffectiveKernel(coefficients, out int firstCoefficient, out int tapCount); + + FilterDirect( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + coefficients[firstCoefficient..], + tapCount, + (firstCoefficient - 3) * sourceStride, + sourceStride, + FilterBits, + 0, + bitDepth); + + return; + } + + ReadOnlySpan horizontalCoefficients = THorizontal.GetCoefficients(horizontalPhase, width <= 4); + ReadOnlySpan verticalCoefficients = TVertical.GetCoefficients(verticalPhase, height <= 4); + GetEffectiveKernel(horizontalCoefficients, out int firstHorizontalCoefficient, out int horizontalTapCount); + GetEffectiveKernel(verticalCoefficients, out int firstVerticalCoefficient, out int verticalTapCount); + + Filter2D( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + horizontalCoefficients[firstHorizontalCoefficient..], + horizontalTapCount, + firstHorizontalCoefficient - 3, + verticalCoefficients[firstVerticalCoefficient..], + verticalTapCount, + firstVerticalCoefficient - 3, + bitDepth, + round0, + scratch); + } + + /// + /// Executes one closed 8-bit interpolation-filter pair through the explicit scalar kernels. + /// + /// The horizontal filter family. + /// The vertical filter family. + private static void PredictScalar( + ReadOnlySpan source, + int sourceStride, + int sourceOrigin, + Span destination, + int destinationStride, + int width, + int height, + int horizontalPhase, + int verticalPhase, + Span scratch) + where THorizontal : struct, IAv1InterPredictorOperator + where TVertical : struct, IAv1InterPredictorOperator + { + if (horizontalPhase == 0 && verticalPhase == 0) + { + CopyScalar(source, sourceStride, sourceOrigin, destination, destinationStride, width, height); + return; + } + + if (verticalPhase == 0) + { + ReadOnlySpan coefficients = THorizontal.GetCoefficients(horizontalPhase, width <= 4); + GetEffectiveKernel(coefficients, out int firstCoefficient, out int tapCount); + + FilterDirectScalar( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + coefficients[firstCoefficient..], + tapCount, + firstCoefficient - 3, + 1, + Round0Bits, + FilterBits - Round0Bits); + + return; + } + + if (horizontalPhase == 0) + { + ReadOnlySpan coefficients = TVertical.GetCoefficients(verticalPhase, height <= 4); + GetEffectiveKernel(coefficients, out int firstCoefficient, out int tapCount); + + FilterDirectScalar( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + coefficients[firstCoefficient..], + tapCount, + (firstCoefficient - 3) * sourceStride, + sourceStride, + FilterBits, + 0); + + return; + } + + ReadOnlySpan horizontalCoefficients = THorizontal.GetCoefficients(horizontalPhase, width <= 4); + ReadOnlySpan verticalCoefficients = TVertical.GetCoefficients(verticalPhase, height <= 4); + GetEffectiveKernel(horizontalCoefficients, out int firstHorizontalCoefficient, out int horizontalTapCount); + GetEffectiveKernel(verticalCoefficients, out int firstVerticalCoefficient, out int verticalTapCount); + + Filter2DScalar( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + horizontalCoefficients[firstHorizontalCoefficient..], + horizontalTapCount, + firstHorizontalCoefficient - 3, + verticalCoefficients[firstVerticalCoefficient..], + verticalTapCount, + firstVerticalCoefficient - 3, + 8, + Round0Bits, + scratch); + } + + /// + /// Executes one closed high-bit-depth interpolation-filter pair through the explicit scalar kernels. + /// + /// The horizontal filter family. + /// The vertical filter family. + private static void PredictScalar( + ReadOnlySpan source, + int sourceStride, + int sourceOrigin, + Span destination, + int destinationStride, + int width, + int height, + int horizontalPhase, + int verticalPhase, + int bitDepth, + Span scratch) + where THorizontal : struct, IAv1InterPredictorOperator + where TVertical : struct, IAv1InterPredictorOperator + { + if (horizontalPhase == 0 && verticalPhase == 0) + { + CopyScalar(source, sourceStride, sourceOrigin, destination, destinationStride, width, height); + return; + } + + // Scalar parity uses the same first-pass range correction as the SIMD traversal so twelve-bit + // intermediates remain signed-16-bit without changing the complete Q14 rounding distance. + int intermediateRange = bitDepth + FilterBits - Round0Bits + 2; + int round0 = Round0Bits + Math.Max(intermediateRange - 16, 0); + + if (verticalPhase == 0) + { + ReadOnlySpan coefficients = THorizontal.GetCoefficients(horizontalPhase, width <= 4); + GetEffectiveKernel(coefficients, out int firstCoefficient, out int tapCount); + + FilterDirectScalar( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + coefficients[firstCoefficient..], + tapCount, + firstCoefficient - 3, + 1, + round0, + FilterBits - round0, + bitDepth); + + return; + } + + if (horizontalPhase == 0) + { + ReadOnlySpan coefficients = TVertical.GetCoefficients(verticalPhase, height <= 4); + GetEffectiveKernel(coefficients, out int firstCoefficient, out int tapCount); + + FilterDirectScalar( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + coefficients[firstCoefficient..], + tapCount, + (firstCoefficient - 3) * sourceStride, + sourceStride, + FilterBits, + 0, + bitDepth); + + return; + } + + ReadOnlySpan horizontalCoefficients = THorizontal.GetCoefficients(horizontalPhase, width <= 4); + ReadOnlySpan verticalCoefficients = TVertical.GetCoefficients(verticalPhase, height <= 4); + GetEffectiveKernel(horizontalCoefficients, out int firstHorizontalCoefficient, out int horizontalTapCount); + GetEffectiveKernel(verticalCoefficients, out int firstVerticalCoefficient, out int verticalTapCount); + + Filter2DScalar( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + horizontalCoefficients[firstHorizontalCoefficient..], + horizontalTapCount, + firstHorizontalCoefficient - 3, + verticalCoefficients[firstVerticalCoefficient..], + verticalTapCount, + firstVerticalCoefficient - 3, + bitDepth, + round0, + scratch); + } + + /// + /// Finds the centered nonzero portion of an eight-position interpolation kernel. + /// + /// The selected Q7 phase kernel. + /// Receives the first coefficient used by the effective kernel. + /// Receives the effective two-, four-, six-, or eight-tap length. + public static void GetEffectiveKernel(ReadOnlySpan coefficients, out int firstCoefficient, out int tapCount) + { + // This matches the reference decoder's get_filter_tap decision. Reducing symmetric zero endpoints avoids source loads and + // multiply-adds while retaining the original tap-to-source alignment through firstCoefficient. + if (coefficients[0] != 0 || coefficients[7] != 0) + { + firstCoefficient = 0; + tapCount = 8; + } + else if (coefficients[1] != 0 || coefficients[6] != 0) + { + firstCoefficient = 1; + tapCount = 6; + } + else if (coefficients[2] != 0 || coefficients[5] != 0) + { + firstCoefficient = 2; + tapCount = 4; + } + else + { + firstCoefficient = 3; + tapCount = 2; + } + } + + /// + /// Selects one eight-coefficient phase from a flattened interpolation table. + /// + /// The sixteen consecutive phase kernels. + /// The selected one-sixteenth-sample phase. + /// The selected Q7 coefficient kernel. + private static ReadOnlySpan GetPhase(ReadOnlySpan table, int phase) => table.Slice(phase * FilterCoefficientCount, FilterCoefficientCount); +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1TranslationalInterPredictor.RegularOperator.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1TranslationalInterPredictor.RegularOperator.cs new file mode 100644 index 0000000000..dfa24eb0a9 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1TranslationalInterPredictor.RegularOperator.cs @@ -0,0 +1,20 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; + +/// +/// Defines regular interpolation for translational inter prediction. +/// +internal static partial class Av1TranslationalInterPredictor +{ + /// + /// Selects regular interpolation coefficients. + /// + internal readonly struct RegularOperator : IAv1InterPredictorOperator + { + /// + public static ReadOnlySpan GetCoefficients(int phase, bool useReducedFilter) + => GetPhase(useReducedFilter ? RegularFourTap : RegularEightTap, phase); + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1TranslationalInterPredictor.SharpOperator.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1TranslationalInterPredictor.SharpOperator.cs new file mode 100644 index 0000000000..9c49a72cd4 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1TranslationalInterPredictor.SharpOperator.cs @@ -0,0 +1,24 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; + +/// +/// Defines sharp interpolation for translational inter prediction. +/// +internal static partial class Av1TranslationalInterPredictor +{ + /// + /// Selects sharp interpolation coefficients. + /// + internal readonly struct SharpOperator : IAv1InterPredictorOperator + { + /// + public static ReadOnlySpan GetCoefficients(int phase, bool useReducedFilter) + { + // AV1 defines sharp filtering on four-sample blocks to be identical to its reduced regular filter. + // Selecting that table here removes the distinction before the hot traversal is instantiated. + return GetPhase(useReducedFilter ? RegularFourTap : SharpEightTap, phase); + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1TranslationalInterPredictor.SmoothOperator.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1TranslationalInterPredictor.SmoothOperator.cs new file mode 100644 index 0000000000..ae0517ef78 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1TranslationalInterPredictor.SmoothOperator.cs @@ -0,0 +1,20 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; + +/// +/// Defines smooth interpolation for translational inter prediction. +/// +internal static partial class Av1TranslationalInterPredictor +{ + /// + /// Selects smooth interpolation coefficients. + /// + internal readonly struct SmoothOperator : IAv1InterPredictorOperator + { + /// + public static ReadOnlySpan GetCoefficients(int phase, bool useReducedFilter) + => GetPhase(useReducedFilter ? SmoothFourTap : SmoothEightTap, phase); + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1TranslationalInterPredictor.TwoDimensions.Byte.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1TranslationalInterPredictor.TwoDimensions.Byte.cs new file mode 100644 index 0000000000..f917c91791 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1TranslationalInterPredictor.TwoDimensions.Byte.cs @@ -0,0 +1,490 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.InteropServices; +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; + +/// +/// Provides separable SIMD convolution for 8-bit single-reference prediction. +/// +internal static partial class Av1TranslationalInterPredictor +{ + /// + /// Filters an 8-bit block in sixteen-sample vectors through caller-owned signed scratch. + /// + private static void Filter2D( + ReadOnlySpan source, + int sourceStride, + int sourceOrigin, + Span destination, + int destinationStride, + int width, + int height, + ReadOnlySpan horizontalCoefficients, + int horizontalTapCount, + int horizontalSourceOffset, + ReadOnlySpan verticalCoefficients, + int verticalTapCount, + int verticalSourceOffset, + int bitDepth, + int round0, + Span scratch, + Vector128 initial) + { + ref byte sourceBase = ref Unsafe.Add(ref MemoryMarshal.GetReference(source), sourceOrigin); + ref byte destinationBase = ref MemoryMarshal.GetReference(destination); + ref short scratchBase = ref MemoryMarshal.GetReference(scratch); + ref short horizontalCoefficientBase = ref MemoryMarshal.GetReference(horizontalCoefficients); + ref short verticalCoefficientBase = ref MemoryMarshal.GetReference(verticalCoefficients); + int scratchStride = Math.Max(width, MinimumScratchStride); + int intermediateHeight = height + verticalTapCount - 1; + Vector128 horizontalInitial = initial + Vector128.Create(1 << (bitDepth + FilterBits - 1)); + + for (int row = 0; row < intermediateHeight; row++) + { + ref byte sourceRow = ref Unsafe.Add(ref sourceBase, ((row + verticalSourceOffset) * sourceStride) + horizontalSourceOffset); + ref short scratchRow = ref Unsafe.Add(ref scratchBase, row * scratchStride); + int processedColumns = 0; + + if (width < Vector128.Count) + { + Convolve( + ref sourceRow, + 1, + 0, + ref horizontalCoefficientBase, + horizontalTapCount, + horizontalInitial, + out Vector128 result0, + out Vector128 result1, + out Vector128 result2, + out Vector128 result3); + + Av1NonDirectionalIntraPredictorBase.Narrow(RoundPowerOfTwo(result0, round0), RoundPowerOfTwo(result1, round0)).StoreUnsafe(ref scratchRow); + Av1NonDirectionalIntraPredictorBase.Narrow(RoundPowerOfTwo(result2, round0), RoundPowerOfTwo(result3, round0)).StoreUnsafe(ref scratchRow, (nuint)Vector128.Count); + continue; + } + + nuint vectorCount = Numerics.Vector128Count(width - processedColumns); + for (; vectorCount > 0; vectorCount--, processedColumns += Vector128.Count) + { + Convolve( + ref sourceRow, + 1, + (nuint)processedColumns, + ref horizontalCoefficientBase, + horizontalTapCount, + horizontalInitial, + out Vector128 result0, + out Vector128 result1, + out Vector128 result2, + out Vector128 result3); + + Av1NonDirectionalIntraPredictorBase.Narrow(RoundPowerOfTwo(result0, round0), RoundPowerOfTwo(result1, round0)).StoreUnsafe(ref scratchRow, (nuint)processedColumns); + Av1NonDirectionalIntraPredictorBase.Narrow( + RoundPowerOfTwo(result2, round0), + RoundPowerOfTwo(result3, round0)).StoreUnsafe( + ref scratchRow, + (nuint)(processedColumns + Vector128.Count)); + } + + FilterHorizontalTail(ref sourceRow, ref scratchRow, processedColumns, width, ref horizontalCoefficientBase, horizontalTapCount, bitDepth, round0); + } + + int round1 = (2 * FilterBits) - round0; + int offsetBits = bitDepth + (2 * FilterBits) - round0; + Vector128 verticalInitial = initial + Vector128.Create(1 << offsetBits); + Vector128 roundOffset = Vector128.Create((1 << (offsetBits - round1)) + (1 << (offsetBits - round1 - 1))); + + for (int row = 0; row < height; row++) + { + ref short scratchRow = ref Unsafe.Add(ref scratchBase, row * scratchStride); + ref byte destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + int processedColumns = 0; + + if (width < Vector128.Count) + { + Convolve( + ref scratchRow, + scratchStride, + 0, + ref verticalCoefficientBase, + verticalTapCount, + verticalInitial, + out Vector128 result0, + out Vector128 result1); + + Convolve( + ref Unsafe.Add(ref scratchRow, Vector128.Count), + scratchStride, + 0, + ref verticalCoefficientBase, + verticalTapCount, + verticalInitial, + out Vector128 result2, + out Vector128 result3); + + result0 = RoundPowerOfTwo(result0, round1) - roundOffset; + result1 = RoundPowerOfTwo(result1, round1) - roundOffset; + result2 = RoundPowerOfTwo(result2, round1) - roundOffset; + result3 = RoundPowerOfTwo(result3, round1) - roundOffset; + StorePartial(PackBytes(result0, result1, result2, result3), ref destinationRow, width); + continue; + } + + nuint vectorCount = Numerics.Vector128Count(width - processedColumns); + for (; vectorCount > 0; vectorCount--, processedColumns += Vector128.Count) + { + Convolve( + ref scratchRow, + scratchStride, + (nuint)processedColumns, + ref verticalCoefficientBase, + verticalTapCount, + verticalInitial, + out Vector128 result0, + out Vector128 result1); + + Convolve( + ref scratchRow, + scratchStride, + (nuint)(processedColumns + Vector128.Count), + ref verticalCoefficientBase, + verticalTapCount, + verticalInitial, + out Vector128 result2, + out Vector128 result3); + + result0 = RoundPowerOfTwo(result0, round1) - roundOffset; + result1 = RoundPowerOfTwo(result1, round1) - roundOffset; + result2 = RoundPowerOfTwo(result2, round1) - roundOffset; + result3 = RoundPowerOfTwo(result3, round1) - roundOffset; + PackBytes(result0, result1, result2, result3).StoreUnsafe(ref destinationRow, (nuint)processedColumns); + } + + FilterVerticalTail(ref scratchRow, ref destinationRow, processedColumns, width, scratchStride, ref verticalCoefficientBase, verticalTapCount, bitDepth, round0); + } + } + + /// + /// Filters an 8-bit block in thirty-two-sample vectors through caller-owned signed scratch. + /// + private static void Filter2D( + ReadOnlySpan source, + int sourceStride, + int sourceOrigin, + Span destination, + int destinationStride, + int width, + int height, + ReadOnlySpan horizontalCoefficients, + int horizontalTapCount, + int horizontalSourceOffset, + ReadOnlySpan verticalCoefficients, + int verticalTapCount, + int verticalSourceOffset, + int bitDepth, + int round0, + Span scratch, + Vector256 initial) + { + ref byte sourceBase = ref Unsafe.Add(ref MemoryMarshal.GetReference(source), sourceOrigin); + ref byte destinationBase = ref MemoryMarshal.GetReference(destination); + ref short scratchBase = ref MemoryMarshal.GetReference(scratch); + ref short horizontalCoefficientBase = ref MemoryMarshal.GetReference(horizontalCoefficients); + ref short verticalCoefficientBase = ref MemoryMarshal.GetReference(verticalCoefficients); + int scratchStride = Math.Max(width, MinimumScratchStride); + int intermediateHeight = height + verticalTapCount - 1; + Vector256 horizontalInitial = initial + Vector256.Create(1 << (bitDepth + FilterBits - 1)); + int vectorEnd = (int)(Numerics.Vector256Count(width) * (nuint)Vector256.Count); + + for (int row = 0; row < intermediateHeight; row++) + { + ref byte sourceRow = ref Unsafe.Add(ref sourceBase, ((row + verticalSourceOffset) * sourceStride) + horizontalSourceOffset); + ref short scratchRow = ref Unsafe.Add(ref scratchBase, row * scratchStride); + int processedColumns = 0; + + for (; processedColumns < vectorEnd; processedColumns += Vector256.Count) + { + Convolve( + ref sourceRow, + 1, + (nuint)processedColumns, + ref horizontalCoefficientBase, + horizontalTapCount, + horizontalInitial, + out Vector256 result0, + out Vector256 result1, + out Vector256 result2, + out Vector256 result3); + + Av1NonDirectionalIntraPredictorBase.Narrow(RoundPowerOfTwo(result0, round0), RoundPowerOfTwo(result1, round0)).StoreUnsafe(ref scratchRow, (nuint)processedColumns); + Av1NonDirectionalIntraPredictorBase.Narrow( + RoundPowerOfTwo(result2, round0), + RoundPowerOfTwo(result3, round0)).StoreUnsafe( + ref scratchRow, + (nuint)(processedColumns + Vector256.Count)); + } + + FilterHorizontalTail(ref sourceRow, ref scratchRow, processedColumns, width, ref horizontalCoefficientBase, horizontalTapCount, bitDepth, round0); + } + + int round1 = (2 * FilterBits) - round0; + int offsetBits = bitDepth + (2 * FilterBits) - round0; + Vector256 verticalInitial = initial + Vector256.Create(1 << offsetBits); + Vector256 roundOffset = Vector256.Create((1 << (offsetBits - round1)) + (1 << (offsetBits - round1 - 1))); + + for (int row = 0; row < height; row++) + { + ref short scratchRow = ref Unsafe.Add(ref scratchBase, row * scratchStride); + ref byte destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + int processedColumns = 0; + + for (; processedColumns < vectorEnd; processedColumns += Vector256.Count) + { + Convolve( + ref scratchRow, + scratchStride, + (nuint)processedColumns, + ref verticalCoefficientBase, + verticalTapCount, + verticalInitial, + out Vector256 result0, + out Vector256 result1); + + Convolve( + ref scratchRow, + scratchStride, + (nuint)(processedColumns + Vector256.Count), + ref verticalCoefficientBase, + verticalTapCount, + verticalInitial, + out Vector256 result2, + out Vector256 result3); + + result0 = RoundPowerOfTwo(result0, round1) - roundOffset; + result1 = RoundPowerOfTwo(result1, round1) - roundOffset; + result2 = RoundPowerOfTwo(result2, round1) - roundOffset; + result3 = RoundPowerOfTwo(result3, round1) - roundOffset; + PackBytes(result0, result1, result2, result3).StoreUnsafe(ref destinationRow, (nuint)processedColumns); + } + + FilterVerticalTail(ref scratchRow, ref destinationRow, processedColumns, width, scratchStride, ref verticalCoefficientBase, verticalTapCount, bitDepth, round0); + } + } + + /// + /// Filters an 8-bit block in sixty-four-sample vectors through caller-owned signed scratch. + /// + private static void Filter2D( + ReadOnlySpan source, + int sourceStride, + int sourceOrigin, + Span destination, + int destinationStride, + int width, + int height, + ReadOnlySpan horizontalCoefficients, + int horizontalTapCount, + int horizontalSourceOffset, + ReadOnlySpan verticalCoefficients, + int verticalTapCount, + int verticalSourceOffset, + int bitDepth, + int round0, + Span scratch, + Vector512 initial) + { + ref byte sourceBase = ref Unsafe.Add(ref MemoryMarshal.GetReference(source), sourceOrigin); + ref byte destinationBase = ref MemoryMarshal.GetReference(destination); + ref short scratchBase = ref MemoryMarshal.GetReference(scratch); + ref short horizontalCoefficientBase = ref MemoryMarshal.GetReference(horizontalCoefficients); + ref short verticalCoefficientBase = ref MemoryMarshal.GetReference(verticalCoefficients); + int scratchStride = Math.Max(width, MinimumScratchStride); + int intermediateHeight = height + verticalTapCount - 1; + Vector512 horizontalInitial = initial + Vector512.Create(1 << (bitDepth + FilterBits - 1)); + int vectorEnd = (int)(Numerics.Vector512Count(width) * (nuint)Vector512.Count); + + for (int row = 0; row < intermediateHeight; row++) + { + ref byte sourceRow = ref Unsafe.Add(ref sourceBase, ((row + verticalSourceOffset) * sourceStride) + horizontalSourceOffset); + ref short scratchRow = ref Unsafe.Add(ref scratchBase, row * scratchStride); + int processedColumns = 0; + + for (; processedColumns < vectorEnd; processedColumns += Vector512.Count) + { + Convolve( + ref sourceRow, + 1, + (nuint)processedColumns, + ref horizontalCoefficientBase, + horizontalTapCount, + horizontalInitial, + out Vector512 result0, + out Vector512 result1, + out Vector512 result2, + out Vector512 result3); + + Av1NonDirectionalIntraPredictorBase.Narrow(RoundPowerOfTwo(result0, round0), RoundPowerOfTwo(result1, round0)).StoreUnsafe(ref scratchRow, (nuint)processedColumns); + Av1NonDirectionalIntraPredictorBase.Narrow( + RoundPowerOfTwo(result2, round0), + RoundPowerOfTwo(result3, round0)).StoreUnsafe( + ref scratchRow, + (nuint)(processedColumns + Vector512.Count)); + } + + FilterHorizontalTail(ref sourceRow, ref scratchRow, processedColumns, width, ref horizontalCoefficientBase, horizontalTapCount, bitDepth, round0); + } + + int round1 = (2 * FilterBits) - round0; + int offsetBits = bitDepth + (2 * FilterBits) - round0; + Vector512 verticalInitial = initial + Vector512.Create(1 << offsetBits); + Vector512 roundOffset = Vector512.Create((1 << (offsetBits - round1)) + (1 << (offsetBits - round1 - 1))); + + for (int row = 0; row < height; row++) + { + ref short scratchRow = ref Unsafe.Add(ref scratchBase, row * scratchStride); + ref byte destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + int processedColumns = 0; + + for (; processedColumns < vectorEnd; processedColumns += Vector512.Count) + { + Convolve( + ref scratchRow, + scratchStride, + (nuint)processedColumns, + ref verticalCoefficientBase, + verticalTapCount, + verticalInitial, + out Vector512 result0, + out Vector512 result1); + + Convolve( + ref scratchRow, + scratchStride, + (nuint)(processedColumns + Vector512.Count), + ref verticalCoefficientBase, + verticalTapCount, + verticalInitial, + out Vector512 result2, + out Vector512 result3); + + result0 = RoundPowerOfTwo(result0, round1) - roundOffset; + result1 = RoundPowerOfTwo(result1, round1) - roundOffset; + result2 = RoundPowerOfTwo(result2, round1) - roundOffset; + result3 = RoundPowerOfTwo(result3, round1) - roundOffset; + PackBytes(result0, result1, result2, result3).StoreUnsafe(ref destinationRow, (nuint)processedColumns); + } + + FilterVerticalTail(ref scratchRow, ref destinationRow, processedColumns, width, scratchStride, ref verticalCoefficientBase, verticalTapCount, bitDepth, round0); + } + } + + /// + /// Finishes an 8-bit horizontal intermediate row after its selected vector width. + /// + private static void FilterHorizontalTail(ref byte source, ref short scratch, int firstColumn, int width, ref short coefficients, int tapCount, int bitDepth, int round0) + { + int horizontalBias = 1 << (bitDepth + FilterBits - 1); + for (int column = firstColumn; column < width; column++) + { + int sum = horizontalBias + ConvolveScalar(ref Unsafe.Add(ref source, column), 1, ref coefficients, tapCount); + Unsafe.Add(ref scratch, column) = (short)RoundPowerOfTwo(sum, round0); + } + } + + /// + /// Finishes an 8-bit vertical output row after its selected vector width. + /// + private static void FilterVerticalTail( + ref short scratch, + ref byte destination, + int firstColumn, + int width, + int scratchStride, + ref short coefficients, + int tapCount, + int bitDepth, + int round0) + { + int round1 = (2 * FilterBits) - round0; + int offsetBits = bitDepth + (2 * FilterBits) - round0; + int verticalBias = 1 << offsetBits; + int roundOffset = (1 << (offsetBits - round1)) + (1 << (offsetBits - round1 - 1)); + + for (int column = firstColumn; column < width; column++) + { + int sum = verticalBias + ConvolveScalar(ref Unsafe.Add(ref scratch, column), scratchStride, ref coefficients, tapCount); + int result = RoundPowerOfTwo(sum, round1) - roundOffset; + Unsafe.Add(ref destination, column) = (byte)Math.Clamp(result, byte.MinValue, byte.MaxValue); + } + } + + /// + /// Applies separable two-dimensional filtering to an 8-bit block without explicit hardware intrinsics. + /// + private static void Filter2DScalar( + ReadOnlySpan source, + int sourceStride, + int sourceOrigin, + Span destination, + int destinationStride, + int width, + int height, + ReadOnlySpan horizontalCoefficients, + int horizontalTapCount, + int horizontalSourceOffset, + ReadOnlySpan verticalCoefficients, + int verticalTapCount, + int verticalSourceOffset, + int bitDepth, + int round0, + Span scratch) + { + ref byte sourceBase = ref Unsafe.Add(ref MemoryMarshal.GetReference(source), sourceOrigin); + ref byte destinationBase = ref MemoryMarshal.GetReference(destination); + ref short scratchBase = ref MemoryMarshal.GetReference(scratch); + ref short horizontalCoefficientBase = ref MemoryMarshal.GetReference(horizontalCoefficients); + ref short verticalCoefficientBase = ref MemoryMarshal.GetReference(verticalCoefficients); + int scratchStride = Math.Max(width, MinimumScratchStride); + int intermediateHeight = height + verticalTapCount - 1; + int horizontalBias = 1 << (bitDepth + FilterBits - 1); + + // The first intermediate row corresponds to the uppermost vertical tap. Horizontal filtering therefore starts + // above the nominal source origin and writes one row for every vertical-tap position needed by the final pass. + for (int row = 0; row < intermediateHeight; row++) + { + ref byte sourceRow = ref Unsafe.Add(ref sourceBase, ((row + verticalSourceOffset) * sourceStride) + horizontalSourceOffset); + ref short scratchRow = ref Unsafe.Add(ref scratchBase, row * scratchStride); + + for (int column = 0; column < width; column++) + { + int sum = horizontalBias + ConvolveScalar(ref Unsafe.Add(ref sourceRow, column), 1, ref horizontalCoefficientBase, horizontalTapCount); + Unsafe.Add(ref scratchRow, column) = (short)RoundPowerOfTwo(sum, round0); + } + } + + int round1 = (2 * FilterBits) - round0; + int offsetBits = bitDepth + (2 * FilterBits) - round0; + int verticalBias = 1 << offsetBits; + int roundOffset = (1 << (offsetBits - round1)) + (1 << (offsetBits - round1 - 1)); + + // The biased first pass keeps every intermediate nonnegative and representable by a signed 16-bit lane. + // Removing both bias terms after the vertical Q7 filter reproduces the reference decoder's single-reference rounding exactly. + for (int row = 0; row < height; row++) + { + ref short scratchRow = ref Unsafe.Add(ref scratchBase, row * scratchStride); + ref byte destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + + for (int column = 0; column < width; column++) + { + int sum = verticalBias + ConvolveScalar(ref Unsafe.Add(ref scratchRow, column), scratchStride, ref verticalCoefficientBase, verticalTapCount); + int result = RoundPowerOfTwo(sum, round1) - roundOffset; + Unsafe.Add(ref destinationRow, column) = (byte)Math.Clamp(result, byte.MinValue, byte.MaxValue); + } + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1TranslationalInterPredictor.TwoDimensions.UInt16.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1TranslationalInterPredictor.TwoDimensions.UInt16.cs new file mode 100644 index 0000000000..c7cf8e0d3e --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1TranslationalInterPredictor.TwoDimensions.UInt16.cs @@ -0,0 +1,520 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.InteropServices; +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; + +/// +/// Provides separable SIMD convolution for unsigned 16-bit single-reference prediction. +/// +internal static partial class Av1TranslationalInterPredictor +{ + /// + /// Filters a high-bit-depth block in eight-sample vectors through caller-owned signed scratch. + /// + private static void Filter2D( + ReadOnlySpan source, + int sourceStride, + int sourceOrigin, + Span destination, + int destinationStride, + int width, + int height, + ReadOnlySpan horizontalCoefficients, + int horizontalTapCount, + int horizontalSourceOffset, + ReadOnlySpan verticalCoefficients, + int verticalTapCount, + int verticalSourceOffset, + int bitDepth, + int round0, + Span scratch, + Vector128 initial) + { + ref ushort sourceBase = ref Unsafe.Add(ref MemoryMarshal.GetReference(source), sourceOrigin); + ref ushort destinationBase = ref MemoryMarshal.GetReference(destination); + ref short scratchBase = ref MemoryMarshal.GetReference(scratch); + ref short horizontalCoefficientBase = ref MemoryMarshal.GetReference(horizontalCoefficients); + ref short verticalCoefficientBase = ref MemoryMarshal.GetReference(verticalCoefficients); + int scratchStride = Math.Max(width, MinimumScratchStride); + int intermediateHeight = height + verticalTapCount - 1; + Vector128 horizontalInitial = initial + Vector128.Create(1 << (bitDepth + FilterBits - 1)); + + for (int row = 0; row < intermediateHeight; row++) + { + ref ushort sourceRowUnsigned = ref Unsafe.Add( + ref sourceBase, + ((row + verticalSourceOffset) * sourceStride) + horizontalSourceOffset); + + ref short sourceRow = ref Unsafe.As(ref sourceRowUnsigned); + ref short scratchRow = ref Unsafe.Add(ref scratchBase, row * scratchStride); + int processedColumns = 0; + + if (width < Vector128.Count) + { + // Subsampled sub-8x8 chroma can be two samples wide. The reference plane is padded for the full source + // load, and the minimum scratch stride preserves all eight intermediate lanes for the vertical pass. + Convolve( + ref sourceRow, + 1, + 0, + ref horizontalCoefficientBase, + horizontalTapCount, + horizontalInitial, + out Vector128 result0, + out Vector128 result1); + + Av1NonDirectionalIntraPredictorBase.Narrow( + RoundPowerOfTwo(result0, round0), + RoundPowerOfTwo(result1, round0)).StoreUnsafe(ref scratchRow); + + continue; + } + + nuint vectorCount = Numerics.Vector128Count(width - processedColumns); + for (; vectorCount > 0; vectorCount--, processedColumns += Vector128.Count) + { + Convolve( + ref sourceRow, + 1, + (nuint)processedColumns, + ref horizontalCoefficientBase, + horizontalTapCount, + horizontalInitial, + out Vector128 result0, + out Vector128 result1); + + Av1NonDirectionalIntraPredictorBase.Narrow( + RoundPowerOfTwo(result0, round0), + RoundPowerOfTwo(result1, round0)).StoreUnsafe(ref scratchRow, (nuint)processedColumns); + } + + FilterHorizontalTail( + ref sourceRowUnsigned, + ref scratchRow, + processedColumns, + width, + ref horizontalCoefficientBase, + horizontalTapCount, + bitDepth, + round0); + } + + int round1 = (2 * FilterBits) - round0; + int offsetBits = bitDepth + (2 * FilterBits) - round0; + Vector128 verticalInitial = initial + Vector128.Create(1 << offsetBits); + Vector128 roundOffset = Vector128.Create( + (1 << (offsetBits - round1)) + (1 << (offsetBits - round1 - 1))); + + int maximum = (1 << bitDepth) - 1; + + for (int row = 0; row < height; row++) + { + ref short scratchRow = ref Unsafe.Add(ref scratchBase, row * scratchStride); + ref ushort destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + int processedColumns = 0; + + if (width < Vector128.Count) + { + Convolve( + ref scratchRow, + scratchStride, + 0, + ref verticalCoefficientBase, + verticalTapCount, + verticalInitial, + out Vector128 result0, + out Vector128 result1); + + result0 = RoundPowerOfTwo(result0, round1) - roundOffset; + result1 = RoundPowerOfTwo(result1, round1) - roundOffset; + + // Retain the vector convolution for two- and four-sample rows while leaving adjacent destination + // samples owned by the neighboring luma block untouched. + StorePartial(PackHighBitDepth(result0, result1, maximum), ref destinationRow, width); + continue; + } + + nuint vectorCount = Numerics.Vector128Count(width - processedColumns); + for (; vectorCount > 0; vectorCount--, processedColumns += Vector128.Count) + { + Convolve( + ref scratchRow, + scratchStride, + (nuint)processedColumns, + ref verticalCoefficientBase, + verticalTapCount, + verticalInitial, + out Vector128 result0, + out Vector128 result1); + + result0 = RoundPowerOfTwo(result0, round1) - roundOffset; + result1 = RoundPowerOfTwo(result1, round1) - roundOffset; + PackHighBitDepth(result0, result1, maximum).StoreUnsafe(ref destinationRow, (nuint)processedColumns); + } + + FilterVerticalTail( + ref scratchRow, + ref destinationRow, + processedColumns, + width, + scratchStride, + ref verticalCoefficientBase, + verticalTapCount, + bitDepth, + round0); + } + } + + /// + /// Filters a high-bit-depth block in sixteen-sample vectors through caller-owned signed scratch. + /// + private static void Filter2D( + ReadOnlySpan source, + int sourceStride, + int sourceOrigin, + Span destination, + int destinationStride, + int width, + int height, + ReadOnlySpan horizontalCoefficients, + int horizontalTapCount, + int horizontalSourceOffset, + ReadOnlySpan verticalCoefficients, + int verticalTapCount, + int verticalSourceOffset, + int bitDepth, + int round0, + Span scratch, + Vector256 initial) + { + ref ushort sourceBase = ref Unsafe.Add(ref MemoryMarshal.GetReference(source), sourceOrigin); + ref ushort destinationBase = ref MemoryMarshal.GetReference(destination); + ref short scratchBase = ref MemoryMarshal.GetReference(scratch); + ref short horizontalCoefficientBase = ref MemoryMarshal.GetReference(horizontalCoefficients); + ref short verticalCoefficientBase = ref MemoryMarshal.GetReference(verticalCoefficients); + int scratchStride = Math.Max(width, MinimumScratchStride); + int intermediateHeight = height + verticalTapCount - 1; + Vector256 horizontalInitial = initial + Vector256.Create(1 << (bitDepth + FilterBits - 1)); + int vectorEnd = (int)(Numerics.Vector256Count(width) * (nuint)Vector256.Count); + + for (int row = 0; row < intermediateHeight; row++) + { + ref ushort sourceRowUnsigned = ref Unsafe.Add( + ref sourceBase, + ((row + verticalSourceOffset) * sourceStride) + horizontalSourceOffset); + + ref short sourceRow = ref Unsafe.As(ref sourceRowUnsigned); + ref short scratchRow = ref Unsafe.Add(ref scratchBase, row * scratchStride); + int processedColumns = 0; + + for (; processedColumns < vectorEnd; processedColumns += Vector256.Count) + { + Convolve( + ref sourceRow, + 1, + (nuint)processedColumns, + ref horizontalCoefficientBase, + horizontalTapCount, + horizontalInitial, + out Vector256 result0, + out Vector256 result1); + + Av1NonDirectionalIntraPredictorBase.Narrow( + RoundPowerOfTwo(result0, round0), + RoundPowerOfTwo(result1, round0)).StoreUnsafe(ref scratchRow, (nuint)processedColumns); + } + + FilterHorizontalTail( + ref sourceRowUnsigned, + ref scratchRow, + processedColumns, + width, + ref horizontalCoefficientBase, + horizontalTapCount, + bitDepth, + round0); + } + + int round1 = (2 * FilterBits) - round0; + int offsetBits = bitDepth + (2 * FilterBits) - round0; + Vector256 verticalInitial = initial + Vector256.Create(1 << offsetBits); + Vector256 roundOffset = Vector256.Create( + (1 << (offsetBits - round1)) + (1 << (offsetBits - round1 - 1))); + + int maximum = (1 << bitDepth) - 1; + + for (int row = 0; row < height; row++) + { + ref short scratchRow = ref Unsafe.Add(ref scratchBase, row * scratchStride); + ref ushort destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + int processedColumns = 0; + + for (; processedColumns < vectorEnd; processedColumns += Vector256.Count) + { + Convolve( + ref scratchRow, + scratchStride, + (nuint)processedColumns, + ref verticalCoefficientBase, + verticalTapCount, + verticalInitial, + out Vector256 result0, + out Vector256 result1); + + result0 = RoundPowerOfTwo(result0, round1) - roundOffset; + result1 = RoundPowerOfTwo(result1, round1) - roundOffset; + PackHighBitDepth(result0, result1, maximum).StoreUnsafe(ref destinationRow, (nuint)processedColumns); + } + + FilterVerticalTail( + ref scratchRow, + ref destinationRow, + processedColumns, + width, + scratchStride, + ref verticalCoefficientBase, + verticalTapCount, + bitDepth, + round0); + } + } + + /// + /// Filters a high-bit-depth block in thirty-two-sample vectors through caller-owned signed scratch. + /// + private static void Filter2D( + ReadOnlySpan source, + int sourceStride, + int sourceOrigin, + Span destination, + int destinationStride, + int width, + int height, + ReadOnlySpan horizontalCoefficients, + int horizontalTapCount, + int horizontalSourceOffset, + ReadOnlySpan verticalCoefficients, + int verticalTapCount, + int verticalSourceOffset, + int bitDepth, + int round0, + Span scratch, + Vector512 initial) + { + ref ushort sourceBase = ref Unsafe.Add(ref MemoryMarshal.GetReference(source), sourceOrigin); + ref ushort destinationBase = ref MemoryMarshal.GetReference(destination); + ref short scratchBase = ref MemoryMarshal.GetReference(scratch); + ref short horizontalCoefficientBase = ref MemoryMarshal.GetReference(horizontalCoefficients); + ref short verticalCoefficientBase = ref MemoryMarshal.GetReference(verticalCoefficients); + int scratchStride = Math.Max(width, MinimumScratchStride); + int intermediateHeight = height + verticalTapCount - 1; + Vector512 horizontalInitial = initial + Vector512.Create(1 << (bitDepth + FilterBits - 1)); + int vectorEnd = (int)(Numerics.Vector512Count(width) * (nuint)Vector512.Count); + + for (int row = 0; row < intermediateHeight; row++) + { + ref ushort sourceRowUnsigned = ref Unsafe.Add( + ref sourceBase, + ((row + verticalSourceOffset) * sourceStride) + horizontalSourceOffset); + + ref short sourceRow = ref Unsafe.As(ref sourceRowUnsigned); + ref short scratchRow = ref Unsafe.Add(ref scratchBase, row * scratchStride); + int processedColumns = 0; + + for (; processedColumns < vectorEnd; processedColumns += Vector512.Count) + { + Convolve( + ref sourceRow, + 1, + (nuint)processedColumns, + ref horizontalCoefficientBase, + horizontalTapCount, + horizontalInitial, + out Vector512 result0, + out Vector512 result1); + + Av1NonDirectionalIntraPredictorBase.Narrow( + RoundPowerOfTwo(result0, round0), + RoundPowerOfTwo(result1, round0)).StoreUnsafe(ref scratchRow, (nuint)processedColumns); + } + + FilterHorizontalTail( + ref sourceRowUnsigned, + ref scratchRow, + processedColumns, + width, + ref horizontalCoefficientBase, + horizontalTapCount, + bitDepth, + round0); + } + + int round1 = (2 * FilterBits) - round0; + int offsetBits = bitDepth + (2 * FilterBits) - round0; + Vector512 verticalInitial = initial + Vector512.Create(1 << offsetBits); + Vector512 roundOffset = Vector512.Create( + (1 << (offsetBits - round1)) + (1 << (offsetBits - round1 - 1))); + + int maximum = (1 << bitDepth) - 1; + + for (int row = 0; row < height; row++) + { + ref short scratchRow = ref Unsafe.Add(ref scratchBase, row * scratchStride); + ref ushort destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + int processedColumns = 0; + + for (; processedColumns < vectorEnd; processedColumns += Vector512.Count) + { + Convolve( + ref scratchRow, + scratchStride, + (nuint)processedColumns, + ref verticalCoefficientBase, + verticalTapCount, + verticalInitial, + out Vector512 result0, + out Vector512 result1); + + result0 = RoundPowerOfTwo(result0, round1) - roundOffset; + result1 = RoundPowerOfTwo(result1, round1) - roundOffset; + PackHighBitDepth(result0, result1, maximum).StoreUnsafe(ref destinationRow, (nuint)processedColumns); + } + + FilterVerticalTail( + ref scratchRow, + ref destinationRow, + processedColumns, + width, + scratchStride, + ref verticalCoefficientBase, + verticalTapCount, + bitDepth, + round0); + } + } + + /// + /// Finishes a high-bit-depth horizontal intermediate row after its selected vector width. + /// + private static void FilterHorizontalTail( + ref ushort source, + ref short scratch, + int firstColumn, + int width, + ref short coefficients, + int tapCount, + int bitDepth, + int round0) + { + int horizontalBias = 1 << (bitDepth + FilterBits - 1); + for (int column = firstColumn; column < width; column++) + { + int sum = horizontalBias + ConvolveScalar( + ref Unsafe.Add(ref source, column), + 1, + ref coefficients, + tapCount); + + Unsafe.Add(ref scratch, column) = (short)RoundPowerOfTwo(sum, round0); + } + } + + /// + /// Finishes a high-bit-depth vertical output row after its selected vector width. + /// + private static void FilterVerticalTail( + ref short scratch, + ref ushort destination, + int firstColumn, + int width, + int scratchStride, + ref short coefficients, + int tapCount, + int bitDepth, + int round0) + { + int round1 = (2 * FilterBits) - round0; + int offsetBits = bitDepth + (2 * FilterBits) - round0; + int verticalBias = 1 << offsetBits; + int roundOffset = (1 << (offsetBits - round1)) + (1 << (offsetBits - round1 - 1)); + int maximum = (1 << bitDepth) - 1; + + for (int column = firstColumn; column < width; column++) + { + int sum = verticalBias + ConvolveScalar( + ref Unsafe.Add(ref scratch, column), + scratchStride, + ref coefficients, + tapCount); + + int result = RoundPowerOfTwo(sum, round1) - roundOffset; + Unsafe.Add(ref destination, column) = (ushort)Math.Clamp(result, 0, maximum); + } + } + + /// + /// Applies separable two-dimensional filtering to a high-bit-depth block without explicit hardware intrinsics. + /// + private static void Filter2DScalar( + ReadOnlySpan source, + int sourceStride, + int sourceOrigin, + Span destination, + int destinationStride, + int width, + int height, + ReadOnlySpan horizontalCoefficients, + int horizontalTapCount, + int horizontalSourceOffset, + ReadOnlySpan verticalCoefficients, + int verticalTapCount, + int verticalSourceOffset, + int bitDepth, + int round0, + Span scratch) + { + ref ushort sourceBase = ref Unsafe.Add(ref MemoryMarshal.GetReference(source), sourceOrigin); + ref ushort destinationBase = ref MemoryMarshal.GetReference(destination); + ref short scratchBase = ref MemoryMarshal.GetReference(scratch); + ref short horizontalCoefficientBase = ref MemoryMarshal.GetReference(horizontalCoefficients); + ref short verticalCoefficientBase = ref MemoryMarshal.GetReference(verticalCoefficients); + int scratchStride = Math.Max(width, MinimumScratchStride); + int intermediateHeight = height + verticalTapCount - 1; + int horizontalBias = 1 << (bitDepth + FilterBits - 1); + + for (int row = 0; row < intermediateHeight; row++) + { + ref ushort sourceRow = ref Unsafe.Add(ref sourceBase, ((row + verticalSourceOffset) * sourceStride) + horizontalSourceOffset); + ref short scratchRow = ref Unsafe.Add(ref scratchBase, row * scratchStride); + + for (int column = 0; column < width; column++) + { + int sum = horizontalBias + ConvolveScalar(ref Unsafe.Add(ref sourceRow, column), 1, ref horizontalCoefficientBase, horizontalTapCount); + Unsafe.Add(ref scratchRow, column) = (short)RoundPowerOfTwo(sum, round0); + } + } + + int round1 = (2 * FilterBits) - round0; + int offsetBits = bitDepth + (2 * FilterBits) - round0; + int verticalBias = 1 << offsetBits; + int roundOffset = (1 << (offsetBits - round1)) + (1 << (offsetBits - round1 - 1)); + int maximum = (1 << bitDepth) - 1; + + for (int row = 0; row < height; row++) + { + ref short scratchRow = ref Unsafe.Add(ref scratchBase, row * scratchStride); + ref ushort destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + + for (int column = 0; column < width; column++) + { + int sum = verticalBias + ConvolveScalar(ref Unsafe.Add(ref scratchRow, column), scratchStride, ref verticalCoefficientBase, verticalTapCount); + int result = RoundPowerOfTwo(sum, round1) - roundOffset; + Unsafe.Add(ref destinationRow, column) = (ushort)Math.Clamp(result, 0, maximum); + } + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1TranslationalInterPredictor.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1TranslationalInterPredictor.cs new file mode 100644 index 0000000000..a45b853e04 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1TranslationalInterPredictor.cs @@ -0,0 +1,234 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; + +/// +/// Reconstructs single-reference translational AV1 inter-prediction blocks. +/// +/// +/// +/// sourceOrigin identifies the integer sample selected by motion-vector scaling within the complete +/// padded reference plane. A filtered axis can consume three samples before the block and four samples after it. Byte +/// rows narrower than sixteen samples and 16-bit rows narrower than eight samples must additionally permit a complete +/// 128-bit source load at every selected tap. The frame prediction border provides this storage; no destination padding +/// is required. +/// +/// +/// Two-dimensional filtering uses caller-owned scratch so block reconstruction does not allocate. The scratch span must +/// contain at least elements when both phases are nonzero and may be empty for +/// copy or one-dimensional filtering. +/// +/// +internal static partial class Av1TranslationalInterPredictor +{ + /// + /// The number of fractional bits in each interpolation coefficient. + /// + internal const int FilterBits = 7; + + /// + /// The normal first-round shift used by the reference decoder single-reference convolution. + /// + internal const int Round0Bits = 3; + + /// + /// The maximum number of source rows added by an eight-tap vertical filter. + /// + internal const int MaximumExtraRows = FilterCoefficientCount - 1; + + /// + /// The minimum scratch stride that lets a 128-bit byte kernel handle four- and eight-sample blocks. + /// + internal const int MinimumScratchStride = 16; + + /// + /// Gets the maximum number of signed 16-bit elements required for one two-dimensional prediction block. + /// + /// The prediction width in samples. + /// The prediction height in samples. + /// The scratch capacity required by either sample-storage overload. + public static int GetScratchLength(int width, int height) => Math.Max(width, MinimumScratchStride) * (height + MaximumExtraRows); + + /// + /// Reconstructs an 8-bit translational prediction using the widest supported SIMD kernel. + /// + /// The complete padded reference plane containing every source sample used by the block. + /// The distance between reference rows in samples. + /// The nonnegative, zero-based index of the integer-position source sample within . + /// The prediction block destination. + /// The distance between destination rows in samples. + /// The prediction width in samples. + /// The prediction height in samples. + /// The horizontal interpolation filter. + /// The vertical interpolation filter. + /// The horizontal phase in one-sixteenth-sample units. + /// The vertical phase in one-sixteenth-sample units. + /// + /// Caller-owned signed intermediate storage sized by when both phases are nonzero. + /// + public static void Predict( + ReadOnlySpan source, + int sourceStride, + int sourceOrigin, + Span destination, + int destinationStride, + int width, + int height, + Av1InterpolationFilter horizontalFilter, + Av1InterpolationFilter verticalFilter, + int horizontalPhase, + int verticalPhase, + Span scratch) + => Dispatch( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + horizontalFilter, + verticalFilter, + horizontalPhase, + verticalPhase, + scratch); + + /// + /// Reconstructs an 8-, 10-, or 12-bit translational prediction using the widest supported SIMD kernel. + /// + /// The complete padded reference plane containing every source sample used by the block. + /// The distance between reference rows in samples. + /// The nonnegative, zero-based index of the integer-position source sample within . + /// The prediction block destination. + /// The distance between destination rows in samples. + /// The prediction width in samples. + /// The prediction height in samples. + /// The horizontal interpolation filter. + /// The vertical interpolation filter. + /// The horizontal phase in one-sixteenth-sample units. + /// The vertical phase in one-sixteenth-sample units. + /// The decoded sample precision: 8, 10, or 12 bits. + /// + /// Caller-owned signed intermediate storage sized by when both phases are nonzero. + /// + public static void Predict( + ReadOnlySpan source, + int sourceStride, + int sourceOrigin, + Span destination, + int destinationStride, + int width, + int height, + Av1InterpolationFilter horizontalFilter, + Av1InterpolationFilter verticalFilter, + int horizontalPhase, + int verticalPhase, + int bitDepth, + Span scratch) + => Dispatch( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + horizontalFilter, + verticalFilter, + horizontalPhase, + verticalPhase, + bitDepth, + scratch); + + /// + /// Reconstructs an 8-bit translational prediction without explicit hardware intrinsics. + /// + /// The complete padded reference plane containing every source sample used by the block. + /// The distance between reference rows in samples. + /// The nonnegative, zero-based index of the integer-position source sample within . + /// The prediction block destination. + /// The distance between destination rows in samples. + /// The prediction width in samples. + /// The prediction height in samples. + /// The horizontal interpolation filter. + /// The vertical interpolation filter. + /// The horizontal phase in one-sixteenth-sample units. + /// The vertical phase in one-sixteenth-sample units. + /// + /// Caller-owned signed intermediate storage sized by when both phases are nonzero. + /// + public static void PredictScalar( + ReadOnlySpan source, + int sourceStride, + int sourceOrigin, + Span destination, + int destinationStride, + int width, + int height, + Av1InterpolationFilter horizontalFilter, + Av1InterpolationFilter verticalFilter, + int horizontalPhase, + int verticalPhase, + Span scratch) + => DispatchScalar( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + horizontalFilter, + verticalFilter, + horizontalPhase, + verticalPhase, + scratch); + + /// + /// Reconstructs an 8-, 10-, or 12-bit translational prediction without explicit hardware intrinsics. + /// + /// The complete padded reference plane containing every source sample used by the block. + /// The distance between reference rows in samples. + /// The nonnegative, zero-based index of the integer-position source sample within . + /// The prediction block destination. + /// The distance between destination rows in samples. + /// The prediction width in samples. + /// The prediction height in samples. + /// The horizontal interpolation filter. + /// The vertical interpolation filter. + /// The horizontal phase in one-sixteenth-sample units. + /// The vertical phase in one-sixteenth-sample units. + /// The decoded sample precision: 8, 10, or 12 bits. + /// + /// Caller-owned signed intermediate storage sized by when both phases are nonzero. + /// + public static void PredictScalar( + ReadOnlySpan source, + int sourceStride, + int sourceOrigin, + Span destination, + int destinationStride, + int width, + int height, + Av1InterpolationFilter horizontalFilter, + Av1InterpolationFilter verticalFilter, + int horizontalPhase, + int verticalPhase, + int bitDepth, + Span scratch) + => DispatchScalar( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + horizontalFilter, + verticalFilter, + horizontalPhase, + verticalPhase, + bitDepth, + scratch); +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1WarpedInterPredictor.Filters.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1WarpedInterPredictor.Filters.cs new file mode 100644 index 0000000000..9d6bc439f4 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1WarpedInterPredictor.Filters.cs @@ -0,0 +1,212 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using static SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter.Av1TranslationalInterPredictor; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; + +/// +/// Provides the normative Q7 filter kernels for affine warped-motion prediction. +/// +internal static partial class Av1WarpedInterPredictor +{ + /// + /// Gets the 193 consecutive eight-tap warped-filter phases spanning fractional positions [-1, 2]. + /// + private static ReadOnlySpan WarpedFilter => + [ + 0, 0, 127, 1, 0, 0, 0, 0, + 0, -1, 127, 2, 0, 0, 0, 0, + 1, -3, 127, 4, -1, 0, 0, 0, + 1, -4, 126, 6, -2, 1, 0, 0, + 1, -5, 126, 8, -3, 1, 0, 0, + 1, -6, 125, 11, -4, 1, 0, 0, + 1, -7, 124, 13, -4, 1, 0, 0, + 2, -8, 123, 15, -5, 1, 0, 0, + 2, -9, 122, 18, -6, 1, 0, 0, + 2, -10, 121, 20, -6, 1, 0, 0, + 2, -11, 120, 22, -7, 2, 0, 0, + 2, -12, 119, 25, -8, 2, 0, 0, + 3, -13, 117, 27, -8, 2, 0, 0, + 3, -13, 116, 29, -9, 2, 0, 0, + 3, -14, 114, 32, -10, 3, 0, 0, + 3, -15, 113, 35, -10, 2, 0, 0, + 3, -15, 111, 37, -11, 3, 0, 0, + 3, -16, 109, 40, -11, 3, 0, 0, + 3, -16, 108, 42, -12, 3, 0, 0, + 4, -17, 106, 45, -13, 3, 0, 0, + 4, -17, 104, 47, -13, 3, 0, 0, + 4, -17, 102, 50, -14, 3, 0, 0, + 4, -17, 100, 52, -14, 3, 0, 0, + 4, -18, 98, 55, -15, 4, 0, 0, + 4, -18, 96, 58, -15, 3, 0, 0, + 4, -18, 94, 60, -16, 4, 0, 0, + 4, -18, 91, 63, -16, 4, 0, 0, + 4, -18, 89, 65, -16, 4, 0, 0, + 4, -18, 87, 68, -17, 4, 0, 0, + 4, -18, 85, 70, -17, 4, 0, 0, + 4, -18, 82, 73, -17, 4, 0, 0, + 4, -18, 80, 75, -17, 4, 0, 0, + 4, -18, 78, 78, -18, 4, 0, 0, + 4, -17, 75, 80, -18, 4, 0, 0, + 4, -17, 73, 82, -18, 4, 0, 0, + 4, -17, 70, 85, -18, 4, 0, 0, + 4, -17, 68, 87, -18, 4, 0, 0, + 4, -16, 65, 89, -18, 4, 0, 0, + 4, -16, 63, 91, -18, 4, 0, 0, + 4, -16, 60, 94, -18, 4, 0, 0, + 3, -15, 58, 96, -18, 4, 0, 0, + 4, -15, 55, 98, -18, 4, 0, 0, + 3, -14, 52, 100, -17, 4, 0, 0, + 3, -14, 50, 102, -17, 4, 0, 0, + 3, -13, 47, 104, -17, 4, 0, 0, + 3, -13, 45, 106, -17, 4, 0, 0, + 3, -12, 42, 108, -16, 3, 0, 0, + 3, -11, 40, 109, -16, 3, 0, 0, + 3, -11, 37, 111, -15, 3, 0, 0, + 2, -10, 35, 113, -15, 3, 0, 0, + 3, -10, 32, 114, -14, 3, 0, 0, + 2, -9, 29, 116, -13, 3, 0, 0, + 2, -8, 27, 117, -13, 3, 0, 0, + 2, -8, 25, 119, -12, 2, 0, 0, + 2, -7, 22, 120, -11, 2, 0, 0, + 1, -6, 20, 121, -10, 2, 0, 0, + 1, -6, 18, 122, -9, 2, 0, 0, + 1, -5, 15, 123, -8, 2, 0, 0, + 1, -4, 13, 124, -7, 1, 0, 0, + 1, -4, 11, 125, -6, 1, 0, 0, + 1, -3, 8, 126, -5, 1, 0, 0, + 1, -2, 6, 126, -4, 1, 0, 0, + 0, -1, 4, 127, -3, 1, 0, 0, + 0, 0, 2, 127, -1, 0, 0, 0, + 0, 0, 0, 127, 1, 0, 0, 0, + 0, 0, -1, 127, 2, 0, 0, 0, + 0, 1, -3, 127, 4, -2, 1, 0, + 0, 1, -5, 127, 6, -2, 1, 0, + 0, 2, -6, 126, 8, -3, 1, 0, + -1, 2, -7, 126, 11, -4, 2, -1, + -1, 3, -8, 125, 13, -5, 2, -1, + -1, 3, -10, 124, 16, -6, 3, -1, + -1, 4, -11, 123, 18, -7, 3, -1, + -1, 4, -12, 122, 20, -7, 3, -1, + -1, 4, -13, 121, 23, -8, 3, -1, + -2, 5, -14, 120, 25, -9, 4, -1, + -1, 5, -15, 119, 27, -10, 4, -1, + -1, 5, -16, 118, 30, -11, 4, -1, + -2, 6, -17, 116, 33, -12, 5, -1, + -2, 6, -17, 114, 35, -12, 5, -1, + -2, 6, -18, 113, 38, -13, 5, -1, + -2, 7, -19, 111, 41, -14, 6, -2, + -2, 7, -19, 110, 43, -15, 6, -2, + -2, 7, -20, 108, 46, -15, 6, -2, + -2, 7, -20, 106, 49, -16, 6, -2, + -2, 7, -21, 104, 51, -16, 7, -2, + -2, 7, -21, 102, 54, -17, 7, -2, + -2, 8, -21, 100, 56, -18, 7, -2, + -2, 8, -22, 98, 59, -18, 7, -2, + -2, 8, -22, 96, 62, -19, 7, -2, + -2, 8, -22, 94, 64, -19, 7, -2, + -2, 8, -22, 91, 67, -20, 8, -2, + -2, 8, -22, 89, 69, -20, 8, -2, + -2, 8, -22, 87, 72, -21, 8, -2, + -2, 8, -21, 84, 74, -21, 8, -2, + -2, 8, -22, 82, 77, -21, 8, -2, + -2, 8, -21, 79, 79, -21, 8, -2, + -2, 8, -21, 77, 82, -22, 8, -2, + -2, 8, -21, 74, 84, -21, 8, -2, + -2, 8, -21, 72, 87, -22, 8, -2, + -2, 8, -20, 69, 89, -22, 8, -2, + -2, 8, -20, 67, 91, -22, 8, -2, + -2, 7, -19, 64, 94, -22, 8, -2, + -2, 7, -19, 62, 96, -22, 8, -2, + -2, 7, -18, 59, 98, -22, 8, -2, + -2, 7, -18, 56, 100, -21, 8, -2, + -2, 7, -17, 54, 102, -21, 7, -2, + -2, 7, -16, 51, 104, -21, 7, -2, + -2, 6, -16, 49, 106, -20, 7, -2, + -2, 6, -15, 46, 108, -20, 7, -2, + -2, 6, -15, 43, 110, -19, 7, -2, + -2, 6, -14, 41, 111, -19, 7, -2, + -1, 5, -13, 38, 113, -18, 6, -2, + -1, 5, -12, 35, 114, -17, 6, -2, + -1, 5, -12, 33, 116, -17, 6, -2, + -1, 4, -11, 30, 118, -16, 5, -1, + -1, 4, -10, 27, 119, -15, 5, -1, + -1, 4, -9, 25, 120, -14, 5, -2, + -1, 3, -8, 23, 121, -13, 4, -1, + -1, 3, -7, 20, 122, -12, 4, -1, + -1, 3, -7, 18, 123, -11, 4, -1, + -1, 3, -6, 16, 124, -10, 3, -1, + -1, 2, -5, 13, 125, -8, 3, -1, + -1, 2, -4, 11, 126, -7, 2, -1, + 0, 1, -3, 8, 126, -6, 2, 0, + 0, 1, -2, 6, 127, -5, 1, 0, + 0, 1, -2, 4, 127, -3, 1, 0, + 0, 0, 0, 2, 127, -1, 0, 0, + 0, 0, 0, 1, 127, 0, 0, 0, + 0, 0, 0, -1, 127, 2, 0, 0, + 0, 0, 1, -3, 127, 4, -1, 0, + 0, 0, 1, -4, 126, 6, -2, 1, + 0, 0, 1, -5, 126, 8, -3, 1, + 0, 0, 1, -6, 125, 11, -4, 1, + 0, 0, 1, -7, 124, 13, -4, 1, + 0, 0, 2, -8, 123, 15, -5, 1, + 0, 0, 2, -9, 122, 18, -6, 1, + 0, 0, 2, -10, 121, 20, -6, 1, + 0, 0, 2, -11, 120, 22, -7, 2, + 0, 0, 2, -12, 119, 25, -8, 2, + 0, 0, 3, -13, 117, 27, -8, 2, + 0, 0, 3, -13, 116, 29, -9, 2, + 0, 0, 3, -14, 114, 32, -10, 3, + 0, 0, 3, -15, 113, 35, -10, 2, + 0, 0, 3, -15, 111, 37, -11, 3, + 0, 0, 3, -16, 109, 40, -11, 3, + 0, 0, 3, -16, 108, 42, -12, 3, + 0, 0, 4, -17, 106, 45, -13, 3, + 0, 0, 4, -17, 104, 47, -13, 3, + 0, 0, 4, -17, 102, 50, -14, 3, + 0, 0, 4, -17, 100, 52, -14, 3, + 0, 0, 4, -18, 98, 55, -15, 4, + 0, 0, 4, -18, 96, 58, -15, 3, + 0, 0, 4, -18, 94, 60, -16, 4, + 0, 0, 4, -18, 91, 63, -16, 4, + 0, 0, 4, -18, 89, 65, -16, 4, + 0, 0, 4, -18, 87, 68, -17, 4, + 0, 0, 4, -18, 85, 70, -17, 4, + 0, 0, 4, -18, 82, 73, -17, 4, + 0, 0, 4, -18, 80, 75, -17, 4, + 0, 0, 4, -18, 78, 78, -18, 4, + 0, 0, 4, -17, 75, 80, -18, 4, + 0, 0, 4, -17, 73, 82, -18, 4, + 0, 0, 4, -17, 70, 85, -18, 4, + 0, 0, 4, -17, 68, 87, -18, 4, + 0, 0, 4, -16, 65, 89, -18, 4, + 0, 0, 4, -16, 63, 91, -18, 4, + 0, 0, 4, -16, 60, 94, -18, 4, + 0, 0, 3, -15, 58, 96, -18, 4, + 0, 0, 4, -15, 55, 98, -18, 4, + 0, 0, 3, -14, 52, 100, -17, 4, + 0, 0, 3, -14, 50, 102, -17, 4, + 0, 0, 3, -13, 47, 104, -17, 4, + 0, 0, 3, -13, 45, 106, -17, 4, + 0, 0, 3, -12, 42, 108, -16, 3, + 0, 0, 3, -11, 40, 109, -16, 3, + 0, 0, 3, -11, 37, 111, -15, 3, + 0, 0, 2, -10, 35, 113, -15, 3, + 0, 0, 3, -10, 32, 114, -14, 3, + 0, 0, 2, -9, 29, 116, -13, 3, + 0, 0, 2, -8, 27, 117, -13, 3, + 0, 0, 2, -8, 25, 119, -12, 2, + 0, 0, 2, -7, 22, 120, -11, 2, + 0, 0, 1, -6, 20, 121, -10, 2, + 0, 0, 1, -6, 18, 122, -9, 2, + 0, 0, 1, -5, 15, 123, -8, 2, + 0, 0, 1, -4, 13, 124, -7, 1, + 0, 0, 1, -4, 11, 125, -6, 1, + 0, 0, 1, -3, 8, 126, -5, 1, + 0, 0, 1, -2, 6, 126, -4, 1, + 0, 0, 0, -1, 4, 127, -3, 1, + 0, 0, 0, 0, 2, 127, -1, 0, + 0, 0, 0, 0, 2, 127, -1, 0, + ]; +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1WarpedInterPredictor.Operator.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1WarpedInterPredictor.Operator.cs new file mode 100644 index 0000000000..c148a36a94 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1WarpedInterPredictor.Operator.cs @@ -0,0 +1,141 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.Intrinsics; + +using static SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter.Av1TranslationalInterPredictor; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; + +/// +/// Defines affine warped-motion prediction arithmetic. +/// +internal static partial class Av1WarpedInterPredictor +{ + /// + /// Defines the affine warped-motion dot product for scalar and SIMD lane groups. + /// + /// + /// Each SIMD overload contains consecutive independent eight-tap filters. The generic warped traversal + /// gathers source windows and coefficient phases, while the closed operator owns the exact multiply-and-sum arithmetic. + /// + private interface IAv1WarpedPredictionOperator + { + /// + /// Convolves one eight-sample window without hardware intrinsics. + /// + /// The first source sample. + /// The distance between source samples. + /// The first signed Q7 coefficient. + /// The exact dot product. + public static abstract int Convolve(ref byte source, int sourceStride, ref short coefficients); + + /// + /// Convolves one high-bit-depth eight-sample window without hardware intrinsics. + /// + /// The first source sample. + /// The distance between source samples. + /// The first signed Q7 coefficient. + /// The exact dot product. + public static abstract int Convolve(ref ushort source, int sourceStride, ref short coefficients); + + /// + /// Convolves one packed eight-sample window. + /// + /// The unsigned samples. + /// The signed Q7 coefficients. + /// The exact dot product. + public static abstract int Convolve(Vector128 samples, Vector128 coefficients); + + /// + /// Convolves two packed eight-sample windows. + /// + /// The two unsigned sample windows. + /// The two signed Q7 coefficient windows. + /// The two exact dot products in the low lanes. + public static abstract Vector128 Convolve(Vector256 samples, Vector256 coefficients); + + /// + /// Convolves four packed eight-sample windows. + /// + /// The four unsigned sample windows. + /// The four signed Q7 coefficient windows. + /// The four exact dot products. + public static abstract Vector128 Convolve(Vector512 samples, Vector512 coefficients); + } + + /// + /// Implements the affine warped-motion dot product for scalar and SIMD lane groups. + /// + private readonly struct WarpedOperator : IAv1WarpedPredictionOperator + { + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static int Convolve(ref byte source, int sourceStride, ref short coefficients) + { + int sum = 0; + for (int index = 0; index < FilterCoefficientCount; index++) + { + sum += Unsafe.Add(ref source, index * sourceStride) * Unsafe.Add(ref coefficients, index); + } + + return sum; + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static int Convolve(ref ushort source, int sourceStride, ref short coefficients) + { + int sum = 0; + for (int index = 0; index < FilterCoefficientCount; index++) + { + sum += Unsafe.Add(ref source, index * sourceStride) * Unsafe.Add(ref coefficients, index); + } + + return sum; + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static int Convolve(Vector128 samples, Vector128 coefficients) + { + (Vector128 sampleLower, Vector128 sampleUpper) = Vector128.Widen(samples); + (Vector128 coefficientLower, Vector128 coefficientUpper) = Vector128.Widen(coefficients); + return Vector128.Sum(sampleLower.AsInt32() * coefficientLower) + + Vector128.Sum(sampleUpper.AsInt32() * coefficientUpper); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector128 Convolve(Vector256 samples, Vector256 coefficients) + { + // Widen preserves the two eight-tap windows as separate 256-bit results. Reducing each product vector + // therefore produces the two independent predictions without horizontal lane shuffles. + (Vector256 sample0, Vector256 sample1) = Vector256.Widen(samples); + (Vector256 coefficient0, Vector256 coefficient1) = Vector256.Widen(coefficients); + return Vector128.Create( + Vector256.Sum(sample0.AsInt32() * coefficient0), + Vector256.Sum(sample1.AsInt32() * coefficient1), + 0, + 0); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector128 Convolve(Vector512 samples, Vector512 coefficients) + { + // The four eight-tap windows occupy four consecutive 256-bit quarters after widening. Multiplication + // remains 512-bit; quarter reductions recover the four independent scalar dot products in output order. + (Vector512 sampleLower, Vector512 sampleUpper) = Vector512.Widen(samples); + (Vector512 coefficientLower, Vector512 coefficientUpper) = Vector512.Widen(coefficients); + Vector512 productLower = sampleLower.AsInt32() * coefficientLower; + Vector512 productUpper = sampleUpper.AsInt32() * coefficientUpper; + return Vector128.Create( + Vector256.Sum(productLower.GetLower()), + Vector256.Sum(productLower.GetUpper()), + Vector256.Sum(productUpper.GetLower()), + Vector256.Sum(productUpper.GetUpper())); + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1WarpedInterPredictor.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1WarpedInterPredictor.cs new file mode 100644 index 0000000000..04a52d1a42 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1WarpedInterPredictor.cs @@ -0,0 +1,1215 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.InteropServices; +using System.Runtime.Intrinsics; +using SixLabors.ImageSharp.Formats.Heif.Av1.Motion; + +using static SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter.Av1TranslationalInterPredictor; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; + +/// +/// Reconstructs local and global affine warped-motion prediction blocks. +/// +internal static partial class Av1WarpedInterPredictor +{ + /// + /// The number of rows in one warped filter's horizontal intermediate tile. + /// + private const int WarpedIntermediateRows = 15; + + /// + /// The number of columns in one warped filter tile. + /// + private const int WarpedTileSize = 8; + + /// + /// The number of low model bits removed when addressing the warped filter table. + /// + private const int WarpedDifferencePrecisionBits = 10; + + /// + /// The number of fractional positions in one warped pixel. + /// + private const int WarpedPixelPrecisionShifts = 64; + + /// + /// Gets the number of signed 16-bit elements required by warped prediction. + /// + public const int WarpedScratchLength = WarpedIntermediateRows * WarpedTileSize; + + /// + /// Reconstructs an 8-bit affine warped prediction using the widest supported convolution operator. + /// + public static void PredictWarped( + ReadOnlySpan source, + int sourceStride, + Point sourceOrigin, + int sourceWidth, + int sourceHeight, + Span destination, + int destinationStride, + Point destinationPosition, + int width, + int height, + int subsamplingX, + int subsamplingY, + Av1GlobalMotionParameters parameters, + Span scratch) + => PredictWarped( + source, + sourceStride, + sourceOrigin, + sourceWidth, + sourceHeight, + destination, + destinationStride, + destinationPosition, + width, + height, + subsamplingX, + subsamplingY, + parameters, + scratch, + useHardwareIntrinsics: true); + + /// + /// Reconstructs an 8-bit affine warped reference into AV1's unsigned compound intermediate format. + /// + public static void PredictWarpedCompound( + ReadOnlySpan source, + int sourceStride, + Point sourceOrigin, + int sourceWidth, + int sourceHeight, + Span destination, + int destinationStride, + Point destinationPosition, + int width, + int height, + int subsamplingX, + int subsamplingY, + Av1GlobalMotionParameters parameters, + Span scratch) + => PredictWarpedCompound( + source, + sourceStride, + sourceOrigin, + sourceWidth, + sourceHeight, + destination, + destinationStride, + destinationPosition, + width, + height, + subsamplingX, + subsamplingY, + parameters, + scratch, + useHardwareIntrinsics: true); + + /// + /// Reconstructs a high-bit-depth affine warped prediction using the widest supported convolution operator. + /// + public static void PredictWarped( + ReadOnlySpan source, + int sourceStride, + Point sourceOrigin, + int sourceWidth, + int sourceHeight, + Span destination, + int destinationStride, + Point destinationPosition, + int width, + int height, + int subsamplingX, + int subsamplingY, + int bitDepth, + Av1GlobalMotionParameters parameters, + Span scratch) + => PredictWarped( + source, + sourceStride, + sourceOrigin, + sourceWidth, + sourceHeight, + destination, + destinationStride, + destinationPosition, + width, + height, + subsamplingX, + subsamplingY, + bitDepth, + parameters, + scratch, + useHardwareIntrinsics: true); + + /// + /// Reconstructs a high-bit-depth affine warped reference into AV1's unsigned compound intermediate format. + /// + public static void PredictWarpedCompound( + ReadOnlySpan source, + int sourceStride, + Point sourceOrigin, + int sourceWidth, + int sourceHeight, + Span destination, + int destinationStride, + Point destinationPosition, + int width, + int height, + int subsamplingX, + int subsamplingY, + int bitDepth, + Av1GlobalMotionParameters parameters, + Span scratch) + => PredictWarpedCompound( + source, + sourceStride, + sourceOrigin, + sourceWidth, + sourceHeight, + destination, + destinationStride, + destinationPosition, + width, + height, + subsamplingX, + subsamplingY, + bitDepth, + parameters, + scratch, + useHardwareIntrinsics: true); + + /// + /// Reconstructs an 8-bit affine warped prediction without explicit hardware intrinsics. + /// + public static void PredictWarpedScalar( + ReadOnlySpan source, + int sourceStride, + Point sourceOrigin, + int sourceWidth, + int sourceHeight, + Span destination, + int destinationStride, + Point destinationPosition, + int width, + int height, + int subsamplingX, + int subsamplingY, + Av1GlobalMotionParameters parameters, + Span scratch) + => PredictWarped( + source, + sourceStride, + sourceOrigin, + sourceWidth, + sourceHeight, + destination, + destinationStride, + destinationPosition, + width, + height, + subsamplingX, + subsamplingY, + parameters, + scratch, + useHardwareIntrinsics: false); + + /// + /// Reconstructs an 8-bit affine warped reference into compound intermediates without explicit hardware intrinsics. + /// + public static void PredictWarpedCompoundScalar( + ReadOnlySpan source, + int sourceStride, + Point sourceOrigin, + int sourceWidth, + int sourceHeight, + Span destination, + int destinationStride, + Point destinationPosition, + int width, + int height, + int subsamplingX, + int subsamplingY, + Av1GlobalMotionParameters parameters, + Span scratch) + => PredictWarpedCompound( + source, + sourceStride, + sourceOrigin, + sourceWidth, + sourceHeight, + destination, + destinationStride, + destinationPosition, + width, + height, + subsamplingX, + subsamplingY, + parameters, + scratch, + useHardwareIntrinsics: false); + + /// + /// Reconstructs a high-bit-depth affine warped prediction without explicit hardware intrinsics. + /// + public static void PredictWarpedScalar( + ReadOnlySpan source, + int sourceStride, + Point sourceOrigin, + int sourceWidth, + int sourceHeight, + Span destination, + int destinationStride, + Point destinationPosition, + int width, + int height, + int subsamplingX, + int subsamplingY, + int bitDepth, + Av1GlobalMotionParameters parameters, + Span scratch) + => PredictWarped( + source, + sourceStride, + sourceOrigin, + sourceWidth, + sourceHeight, + destination, + destinationStride, + destinationPosition, + width, + height, + subsamplingX, + subsamplingY, + bitDepth, + parameters, + scratch, + useHardwareIntrinsics: false); + + /// + /// Reconstructs a high-bit-depth affine warped reference into compound intermediates without explicit hardware intrinsics. + /// + public static void PredictWarpedCompoundScalar( + ReadOnlySpan source, + int sourceStride, + Point sourceOrigin, + int sourceWidth, + int sourceHeight, + Span destination, + int destinationStride, + Point destinationPosition, + int width, + int height, + int subsamplingX, + int subsamplingY, + int bitDepth, + Av1GlobalMotionParameters parameters, + Span scratch) + => PredictWarpedCompound( + source, + sourceStride, + sourceOrigin, + sourceWidth, + sourceHeight, + destination, + destinationStride, + destinationPosition, + width, + height, + subsamplingX, + subsamplingY, + bitDepth, + parameters, + scratch, + useHardwareIntrinsics: false); + + /// + /// Reconstructs one 8-bit warped block through a closed convolution operator. + /// + private static void PredictWarped( + ReadOnlySpan source, + int sourceStride, + Point sourceOrigin, + int sourceWidth, + int sourceHeight, + Span destination, + int destinationStride, + Point destinationPosition, + int width, + int height, + int subsamplingX, + int subsamplingY, + Av1GlobalMotionParameters parameters, + Span scratch, + bool useHardwareIntrinsics) + where TOperator : struct, IAv1WarpedPredictionOperator + { + ref byte sourceBase = ref MemoryMarshal.GetReference(source); + ref byte destinationBase = ref MemoryMarshal.GetReference(destination); + Span intermediate = MemoryMarshal.Cast(scratch)[..WarpedScratchLength]; + int horizontalBias = 1 << (8 + FilterBits - 1); + int verticalBias = 1 << (8 + (2 * FilterBits) - Round0Bits); + int verticalRound = (2 * FilterBits) - Round0Bits; + + for (int tileRow = destinationPosition.Y; tileRow < destinationPosition.Y + height; tileRow += WarpedTileSize) + { + for (int tileColumn = destinationPosition.X; tileColumn < destinationPosition.X + width; tileColumn += WarpedTileSize) + { + DeriveWarpedTilePosition( + parameters, + tileColumn, + tileRow, + subsamplingX, + subsamplingY, + out int integerX, + out int integerY, + out int phaseX, + out int phaseY); + + FilterWarpedHorizontal( + ref sourceBase, + sourceStride, + sourceOrigin, + sourceHeight, + integerX, + integerY, + phaseX, + parameters, + intermediate, + horizontalBias, + Round0Bits, + useHardwareIntrinsics); + + int tileHeight = Math.Min(WarpedTileSize, destinationPosition.Y + height - tileRow); + int tileWidth = Math.Min(WarpedTileSize, destinationPosition.X + width - tileColumn); + for (int row = 0; row < tileHeight; row++) + { + int phase = phaseY + (parameters.Delta * row); + int destinationRowOffset = (tileRow - destinationPosition.Y + row) * destinationStride; + ref ushort intermediateSource = ref intermediate[row * WarpedTileSize]; + ref byte destinationRow = ref Unsafe.Add( + ref destinationBase, + destinationRowOffset + tileColumn - destinationPosition.X); + + FilterWarpedVertical( + ref intermediateSource, + ref destinationRow, + tileWidth, + phase, + parameters.Gamma, + verticalBias, + verticalRound, + useHardwareIntrinsics); + } + } + } + } + + /// + /// Produces the unsigned horizontal intermediate tile for an 8-bit source plane. + /// + private static void FilterWarpedHorizontal( + ref byte sourceBase, + int sourceStride, + Point sourceOrigin, + int sourceHeight, + int integerX, + int integerY, + int phaseX, + Av1GlobalMotionParameters parameters, + Span intermediate, + int bias, + int round, + bool useHardwareIntrinsics) + where TOperator : struct, IAv1WarpedPredictionOperator + { + for (int row = -7; row < 8; row++) + { + int sourceY = Math.Clamp(integerY + row, 0, sourceHeight - 1); + int sourceIndex = ((sourceOrigin.Y + sourceY) * sourceStride) + sourceOrigin.X + integerX - 7; + ref byte sourceRow = ref Unsafe.Add(ref sourceBase, sourceIndex); + ref ushort intermediateRow = ref intermediate[(row + 7) * WarpedTileSize]; + int phase = phaseX + (parameters.Beta * (row + 4)); + int column = 0; + + // Eight neighboring windows use different warped phases. Packing complete eight-tap windows into + // descending SIMD widths preserves those independent coefficients while leaving no per-row buffers. + if (useHardwareIntrinsics && Vector512.IsHardwareAccelerated) + { + int oneVectorFromEnd = WarpedTileSize - 4; + for (; column <= oneVectorFromEnd; column += 4) + { + Vector128 sums = ConvolveWarpedVector512( + ref Unsafe.Add(ref sourceRow, column), + phase + (column * parameters.Alpha), + parameters.Alpha); + + for (int lane = 0; lane < 4; lane++) + { + Unsafe.Add(ref intermediateRow, column + lane) = + (ushort)RoundPowerOfTwoScalar(bias + sums.GetElement(lane), round); + } + } + } + + if (useHardwareIntrinsics && Vector256.IsHardwareAccelerated) + { + int oneVectorFromEnd = WarpedTileSize - 2; + for (; column <= oneVectorFromEnd; column += 2) + { + Vector128 sums = ConvolveWarpedVector256( + ref Unsafe.Add(ref sourceRow, column), + phase + (column * parameters.Alpha), + parameters.Alpha); + + Unsafe.Add(ref intermediateRow, column) = (ushort)RoundPowerOfTwoScalar(bias + sums.GetElement(0), round); + Unsafe.Add(ref intermediateRow, column + 1) = (ushort)RoundPowerOfTwoScalar(bias + sums.GetElement(1), round); + } + } + + if (useHardwareIntrinsics && Vector128.IsHardwareAccelerated) + { + for (; column < WarpedTileSize; column++) + { + int sum = ConvolveWarpedVector128( + ref Unsafe.Add(ref sourceRow, column), + phase + (column * parameters.Alpha)); + + Unsafe.Add(ref intermediateRow, column) = (ushort)RoundPowerOfTwoScalar(bias + sum, round); + } + } + + for (; column < WarpedTileSize; column++) + { + ref short coefficients = ref GetWarpedFilterReference(phase + (column * parameters.Alpha)); + int sum = TOperator.Convolve(ref Unsafe.Add(ref sourceRow, column), 1, ref coefficients); + Unsafe.Add(ref intermediateRow, column) = (ushort)RoundPowerOfTwoScalar(bias + sum, round); + } + } + } + + /// + /// Produces the unsigned horizontal intermediate tile for a high-bit-depth source plane. + /// + private static void FilterWarpedHorizontal( + ref ushort sourceBase, + int sourceStride, + Point sourceOrigin, + int sourceHeight, + int integerX, + int integerY, + int phaseX, + Av1GlobalMotionParameters parameters, + Span intermediate, + int bias, + int round, + bool useHardwareIntrinsics) + where TOperator : struct, IAv1WarpedPredictionOperator + { + for (int row = -7; row < 8; row++) + { + int sourceY = Math.Clamp(integerY + row, 0, sourceHeight - 1); + int sourceIndex = ((sourceOrigin.Y + sourceY) * sourceStride) + sourceOrigin.X + integerX - 7; + ref ushort sourceRow = ref Unsafe.Add(ref sourceBase, sourceIndex); + ref ushort intermediateRow = ref intermediate[(row + 7) * WarpedTileSize]; + int phase = phaseX + (parameters.Beta * (row + 4)); + int column = 0; + + if (useHardwareIntrinsics && Vector512.IsHardwareAccelerated) + { + int oneVectorFromEnd = WarpedTileSize - 4; + for (; column <= oneVectorFromEnd; column += 4) + { + Vector128 sums = ConvolveWarpedVector512( + ref Unsafe.Add(ref sourceRow, column), + 1, + phase + (column * parameters.Alpha), + parameters.Alpha); + + for (int lane = 0; lane < 4; lane++) + { + Unsafe.Add(ref intermediateRow, column + lane) = + (ushort)RoundPowerOfTwoScalar(bias + sums.GetElement(lane), round); + } + } + } + + if (useHardwareIntrinsics && Vector256.IsHardwareAccelerated) + { + int oneVectorFromEnd = WarpedTileSize - 2; + for (; column <= oneVectorFromEnd; column += 2) + { + Vector128 sums = ConvolveWarpedVector256( + ref Unsafe.Add(ref sourceRow, column), + 1, + phase + (column * parameters.Alpha), + parameters.Alpha); + + Unsafe.Add(ref intermediateRow, column) = (ushort)RoundPowerOfTwoScalar(bias + sums.GetElement(0), round); + Unsafe.Add(ref intermediateRow, column + 1) = (ushort)RoundPowerOfTwoScalar(bias + sums.GetElement(1), round); + } + } + + if (useHardwareIntrinsics && Vector128.IsHardwareAccelerated) + { + for (; column < WarpedTileSize; column++) + { + int sum = ConvolveWarpedVector128( + ref Unsafe.Add(ref sourceRow, column), + 1, + phase + (column * parameters.Alpha)); + + Unsafe.Add(ref intermediateRow, column) = (ushort)RoundPowerOfTwoScalar(bias + sum, round); + } + } + + for (; column < WarpedTileSize; column++) + { + ref short coefficients = ref GetWarpedFilterReference(phase + (column * parameters.Alpha)); + int sum = TOperator.Convolve(ref Unsafe.Add(ref sourceRow, column), 1, ref coefficients); + Unsafe.Add(ref intermediateRow, column) = (ushort)RoundPowerOfTwoScalar(bias + sum, round); + } + } + } + + /// + /// Completes one 8-bit vertical warped-filter row. + /// + private static void FilterWarpedVertical( + ref ushort source, + ref byte destination, + int width, + int phase, + int phaseStep, + int bias, + int round, + bool useHardwareIntrinsics) + where TOperator : struct, IAv1WarpedPredictionOperator + { + int column = 0; + if (useHardwareIntrinsics && Vector512.IsHardwareAccelerated) + { + int oneVectorFromEnd = width - 4; + for (; column <= oneVectorFromEnd; column += 4) + { + Vector128 sums = ConvolveWarpedVector512( + ref Unsafe.Add(ref source, column), + WarpedTileSize, + phase + (column * phaseStep), + phaseStep); + + for (int lane = 0; lane < 4; lane++) + { + Unsafe.Add(ref destination, column + lane) = FinishWarpedByte(sums.GetElement(lane), bias, round); + } + } + } + + if (useHardwareIntrinsics && Vector256.IsHardwareAccelerated) + { + int oneVectorFromEnd = width - 2; + for (; column <= oneVectorFromEnd; column += 2) + { + Vector128 sums = ConvolveWarpedVector256( + ref Unsafe.Add(ref source, column), + WarpedTileSize, + phase + (column * phaseStep), + phaseStep); + + Unsafe.Add(ref destination, column) = FinishWarpedByte(sums.GetElement(0), bias, round); + Unsafe.Add(ref destination, column + 1) = FinishWarpedByte(sums.GetElement(1), bias, round); + } + } + + if (useHardwareIntrinsics && Vector128.IsHardwareAccelerated) + { + for (; column < width; column++) + { + int sum = ConvolveWarpedVector128( + ref Unsafe.Add(ref source, column), + WarpedTileSize, + phase + (column * phaseStep)); + + Unsafe.Add(ref destination, column) = FinishWarpedByte(sum, bias, round); + } + } + + for (; column < width; column++) + { + ref short coefficients = ref GetWarpedFilterReference(phase + (column * phaseStep)); + int sum = TOperator.Convolve(ref Unsafe.Add(ref source, column), WarpedTileSize, ref coefficients); + Unsafe.Add(ref destination, column) = FinishWarpedByte(sum, bias, round); + } + } + + /// + /// Completes one compound-intermediate vertical warped-filter row. + /// + private static void FilterWarpedCompoundVertical( + ref ushort source, + ref ushort destination, + int width, + int phase, + int phaseStep, + int bias, + bool useHardwareIntrinsics) + where TOperator : struct, IAv1WarpedPredictionOperator + { + int column = 0; + if (useHardwareIntrinsics && Vector512.IsHardwareAccelerated) + { + int oneVectorFromEnd = width - 4; + for (; column <= oneVectorFromEnd; column += 4) + { + Vector128 sums = ConvolveWarpedVector512( + ref Unsafe.Add(ref source, column), + WarpedTileSize, + phase + (column * phaseStep), + phaseStep); + + for (int lane = 0; lane < 4; lane++) + { + Unsafe.Add(ref destination, column + lane) = FinishWarpedCompound(sums.GetElement(lane), bias); + } + } + } + + if (useHardwareIntrinsics && Vector256.IsHardwareAccelerated) + { + int oneVectorFromEnd = width - 2; + for (; column <= oneVectorFromEnd; column += 2) + { + Vector128 sums = ConvolveWarpedVector256( + ref Unsafe.Add(ref source, column), + WarpedTileSize, + phase + (column * phaseStep), + phaseStep); + + Unsafe.Add(ref destination, column) = FinishWarpedCompound(sums.GetElement(0), bias); + Unsafe.Add(ref destination, column + 1) = FinishWarpedCompound(sums.GetElement(1), bias); + } + } + + if (useHardwareIntrinsics && Vector128.IsHardwareAccelerated) + { + for (; column < width; column++) + { + int sum = ConvolveWarpedVector128( + ref Unsafe.Add(ref source, column), + WarpedTileSize, + phase + (column * phaseStep)); + + Unsafe.Add(ref destination, column) = FinishWarpedCompound(sum, bias); + } + } + + for (; column < width; column++) + { + ref short coefficients = ref GetWarpedFilterReference(phase + (column * phaseStep)); + int sum = TOperator.Convolve(ref Unsafe.Add(ref source, column), WarpedTileSize, ref coefficients); + Unsafe.Add(ref destination, column) = FinishWarpedCompound(sum, bias); + } + } + + /// + /// Completes one high-bit-depth vertical warped-filter row. + /// + private static void FilterWarpedVertical( + ref ushort source, + ref ushort destination, + int width, + int phase, + int phaseStep, + int bias, + int round, + int bitDepth, + int maximum, + bool useHardwareIntrinsics) + where TOperator : struct, IAv1WarpedPredictionOperator + { + int column = 0; + if (useHardwareIntrinsics && Vector512.IsHardwareAccelerated) + { + int oneVectorFromEnd = width - 4; + for (; column <= oneVectorFromEnd; column += 4) + { + Vector128 sums = ConvolveWarpedVector512( + ref Unsafe.Add(ref source, column), + WarpedTileSize, + phase + (column * phaseStep), + phaseStep); + + for (int lane = 0; lane < 4; lane++) + { + Unsafe.Add(ref destination, column + lane) = FinishWarpedHighBitDepth(sums.GetElement(lane), bias, round, bitDepth, maximum); + } + } + } + + if (useHardwareIntrinsics && Vector256.IsHardwareAccelerated) + { + int oneVectorFromEnd = width - 2; + for (; column <= oneVectorFromEnd; column += 2) + { + Vector128 sums = ConvolveWarpedVector256( + ref Unsafe.Add(ref source, column), + WarpedTileSize, + phase + (column * phaseStep), + phaseStep); + + Unsafe.Add(ref destination, column) = FinishWarpedHighBitDepth(sums.GetElement(0), bias, round, bitDepth, maximum); + Unsafe.Add(ref destination, column + 1) = FinishWarpedHighBitDepth(sums.GetElement(1), bias, round, bitDepth, maximum); + } + } + + if (useHardwareIntrinsics && Vector128.IsHardwareAccelerated) + { + for (; column < width; column++) + { + int sum = ConvolveWarpedVector128( + ref Unsafe.Add(ref source, column), + WarpedTileSize, + phase + (column * phaseStep)); + + Unsafe.Add(ref destination, column) = FinishWarpedHighBitDepth(sum, bias, round, bitDepth, maximum); + } + } + + for (; column < width; column++) + { + ref short coefficients = ref GetWarpedFilterReference(phase + (column * phaseStep)); + int sum = TOperator.Convolve(ref Unsafe.Add(ref source, column), WarpedTileSize, ref coefficients); + Unsafe.Add(ref destination, column) = FinishWarpedHighBitDepth(sum, bias, round, bitDepth, maximum); + } + } + + /// + /// Convolves four adjacent 8-bit source windows with independent phases. + /// + private static Vector128 ConvolveWarpedVector512(ref byte source, int phase, int phaseStep) + where TOperator : struct, IAv1WarpedPredictionOperator + { + Vector256 sampleLower = Vector256.Create(LoadWarpedWindow(ref source), LoadWarpedWindow(ref Unsafe.Add(ref source, 1))); + Vector256 sampleUpper = Vector256.Create(LoadWarpedWindow(ref Unsafe.Add(ref source, 2)), LoadWarpedWindow(ref Unsafe.Add(ref source, 3))); + Vector256 coefficientLower = Vector256.Create(LoadWarpedCoefficients(phase), LoadWarpedCoefficients(phase + phaseStep)); + Vector256 coefficientUpper = Vector256.Create(LoadWarpedCoefficients(phase + (2 * phaseStep)), LoadWarpedCoefficients(phase + (3 * phaseStep))); + return TOperator.Convolve(Vector512.Create(sampleLower, sampleUpper), Vector512.Create(coefficientLower, coefficientUpper)); + } + + /// + /// Convolves two adjacent 8-bit source windows with independent phases. + /// + private static Vector128 ConvolveWarpedVector256(ref byte source, int phase, int phaseStep) + where TOperator : struct, IAv1WarpedPredictionOperator + => TOperator.Convolve( + Vector256.Create(LoadWarpedWindow(ref source), LoadWarpedWindow(ref Unsafe.Add(ref source, 1))), + Vector256.Create(LoadWarpedCoefficients(phase), LoadWarpedCoefficients(phase + phaseStep))); + + /// + /// Convolves one 8-bit source window. + /// + private static int ConvolveWarpedVector128(ref byte source, int phase) + where TOperator : struct, IAv1WarpedPredictionOperator + => TOperator.Convolve(LoadWarpedWindow(ref source), LoadWarpedCoefficients(phase)); + + /// + /// Convolves four adjacent high-bit-depth source windows with independent phases. + /// + private static Vector128 ConvolveWarpedVector512(ref ushort source, int sourceStride, int phase, int phaseStep) + where TOperator : struct, IAv1WarpedPredictionOperator + { + Vector256 sampleLower = Vector256.Create(LoadWarpedWindow(ref source, sourceStride), LoadWarpedWindow(ref Unsafe.Add(ref source, 1), sourceStride)); + Vector256 sampleUpper = Vector256.Create(LoadWarpedWindow(ref Unsafe.Add(ref source, 2), sourceStride), LoadWarpedWindow(ref Unsafe.Add(ref source, 3), sourceStride)); + Vector256 coefficientLower = Vector256.Create(LoadWarpedCoefficients(phase), LoadWarpedCoefficients(phase + phaseStep)); + Vector256 coefficientUpper = Vector256.Create(LoadWarpedCoefficients(phase + (2 * phaseStep)), LoadWarpedCoefficients(phase + (3 * phaseStep))); + return TOperator.Convolve(Vector512.Create(sampleLower, sampleUpper), Vector512.Create(coefficientLower, coefficientUpper)); + } + + /// + /// Convolves two adjacent high-bit-depth source windows with independent phases. + /// + private static Vector128 ConvolveWarpedVector256(ref ushort source, int sourceStride, int phase, int phaseStep) + where TOperator : struct, IAv1WarpedPredictionOperator + => TOperator.Convolve( + Vector256.Create(LoadWarpedWindow(ref source, sourceStride), LoadWarpedWindow(ref Unsafe.Add(ref source, 1), sourceStride)), + Vector256.Create(LoadWarpedCoefficients(phase), LoadWarpedCoefficients(phase + phaseStep))); + + /// + /// Convolves one high-bit-depth source window. + /// + private static int ConvolveWarpedVector128(ref ushort source, int sourceStride, int phase) + where TOperator : struct, IAv1WarpedPredictionOperator + => TOperator.Convolve(LoadWarpedWindow(ref source, sourceStride), LoadWarpedCoefficients(phase)); + + /// + /// Loads eight adjacent unsigned byte samples as unsigned 16-bit lanes. + /// + private static Vector128 LoadWarpedWindow(ref byte source) + { + Vector128 packed = Vector128.LoadUnsafe(ref source); + return Vector128.Widen(packed).Lower; + } + + /// + /// Loads eight unsigned high-bit-depth samples with the supplied spacing. + /// + private static Vector128 LoadWarpedWindow(ref ushort source, int sourceStride) + { + if (sourceStride == 1) + { + return Vector128.LoadUnsafe(ref source); + } + + return Vector128.Create( + source, + Unsafe.Add(ref source, sourceStride), + Unsafe.Add(ref source, sourceStride * 2), + Unsafe.Add(ref source, sourceStride * 3), + Unsafe.Add(ref source, sourceStride * 4), + Unsafe.Add(ref source, sourceStride * 5), + Unsafe.Add(ref source, sourceStride * 6), + Unsafe.Add(ref source, sourceStride * 7)); + } + + /// + /// Loads one signed Q7 warped-filter phase. + /// + private static Vector128 LoadWarpedCoefficients(int phase) + => Vector128.LoadUnsafe(ref GetWarpedFilterReference(phase)); + + /// + /// Applies the final 8-bit warped-prediction rounding and clipping. + /// + private static byte FinishWarpedByte(int sum, int bias, int round) + { + int value = RoundPowerOfTwoScalar(bias + sum, round) - (1 << 7) - (1 << 8); + return (byte)Math.Clamp(value, byte.MinValue, byte.MaxValue); + } + + /// + /// Applies the compound-intermediate warped-prediction rounding. + /// + private static ushort FinishWarpedCompound(int sum, int bias) + => (ushort)RoundPowerOfTwoScalar(bias + sum, Av1CompoundInterPredictor.CompoundRound1Bits); + + /// + /// Applies the final high-bit-depth warped-prediction rounding and clipping. + /// + private static ushort FinishWarpedHighBitDepth(int sum, int bias, int round, int bitDepth, int maximum) + { + int value = RoundPowerOfTwoScalar(bias + sum, round) - (1 << (bitDepth - 1)) - (1 << bitDepth); + return (ushort)Math.Clamp(value, 0, maximum); + } + + /// + /// Reconstructs one 8-bit warped reference without discarding the compound convolution precision. + /// + private static void PredictWarpedCompound( + ReadOnlySpan source, + int sourceStride, + Point sourceOrigin, + int sourceWidth, + int sourceHeight, + Span destination, + int destinationStride, + Point destinationPosition, + int width, + int height, + int subsamplingX, + int subsamplingY, + Av1GlobalMotionParameters parameters, + Span scratch, + bool useHardwareIntrinsics) + where TOperator : struct, IAv1WarpedPredictionOperator + { + ref byte sourceBase = ref MemoryMarshal.GetReference(source); + ref ushort destinationBase = ref MemoryMarshal.GetReference(destination); + Span intermediate = MemoryMarshal.Cast(scratch)[..WarpedScratchLength]; + int horizontalBias = 1 << (8 + FilterBits - 1); + int verticalBias = 1 << (8 + (2 * FilterBits) - Round0Bits); + + for (int tileRow = destinationPosition.Y; tileRow < destinationPosition.Y + height; tileRow += WarpedTileSize) + { + for (int tileColumn = destinationPosition.X; tileColumn < destinationPosition.X + width; tileColumn += WarpedTileSize) + { + DeriveWarpedTilePosition( + parameters, + tileColumn, + tileRow, + subsamplingX, + subsamplingY, + out int integerX, + out int integerY, + out int phaseX, + out int phaseY); + + FilterWarpedHorizontal( + ref sourceBase, + sourceStride, + sourceOrigin, + sourceHeight, + integerX, + integerY, + phaseX, + parameters, + intermediate, + horizontalBias, + Round0Bits, + useHardwareIntrinsics); + + int tileHeight = Math.Min(WarpedTileSize, destinationPosition.Y + height - tileRow); + int tileWidth = Math.Min(WarpedTileSize, destinationPosition.X + width - tileColumn); + for (int row = 0; row < tileHeight; row++) + { + int phase = phaseY + (parameters.Delta * row); + int destinationRowOffset = (tileRow - destinationPosition.Y + row) * destinationStride; + ref ushort intermediateSource = ref intermediate[row * WarpedTileSize]; + ref ushort destinationRow = ref Unsafe.Add( + ref destinationBase, + destinationRowOffset + tileColumn - destinationPosition.X); + + FilterWarpedCompoundVertical( + ref intermediateSource, + ref destinationRow, + tileWidth, + phase, + parameters.Gamma, + verticalBias, + useHardwareIntrinsics); + } + } + } + } + + /// + /// Reconstructs one high-bit-depth warped reference without discarding the compound convolution precision. + /// + private static void PredictWarpedCompound( + ReadOnlySpan source, + int sourceStride, + Point sourceOrigin, + int sourceWidth, + int sourceHeight, + Span destination, + int destinationStride, + Point destinationPosition, + int width, + int height, + int subsamplingX, + int subsamplingY, + int bitDepth, + Av1GlobalMotionParameters parameters, + Span scratch, + bool useHardwareIntrinsics) + where TOperator : struct, IAv1WarpedPredictionOperator + { + ref ushort sourceBase = ref MemoryMarshal.GetReference(source); + ref ushort destinationBase = ref MemoryMarshal.GetReference(destination); + Span intermediate = MemoryMarshal.Cast(scratch)[..WarpedScratchLength]; + + // Twelve-bit input raises round0 so every biased horizontal sample fits in the shared 16-bit scratch tile. + // Compound prediction keeps round1 at seven; its final blend removes the remaining two normative bits. + int intermediateRange = bitDepth + FilterBits - Round0Bits + 2; + int round0 = Round0Bits + Math.Max(intermediateRange - 16, 0); + int horizontalBias = 1 << (bitDepth + FilterBits - 1); + int verticalBias = 1 << (bitDepth + (2 * FilterBits) - round0); + + for (int tileRow = destinationPosition.Y; tileRow < destinationPosition.Y + height; tileRow += WarpedTileSize) + { + for (int tileColumn = destinationPosition.X; tileColumn < destinationPosition.X + width; tileColumn += WarpedTileSize) + { + DeriveWarpedTilePosition( + parameters, + tileColumn, + tileRow, + subsamplingX, + subsamplingY, + out int integerX, + out int integerY, + out int phaseX, + out int phaseY); + + FilterWarpedHorizontal( + ref sourceBase, + sourceStride, + sourceOrigin, + sourceHeight, + integerX, + integerY, + phaseX, + parameters, + intermediate, + horizontalBias, + round0, + useHardwareIntrinsics); + + int tileHeight = Math.Min(WarpedTileSize, destinationPosition.Y + height - tileRow); + int tileWidth = Math.Min(WarpedTileSize, destinationPosition.X + width - tileColumn); + for (int row = 0; row < tileHeight; row++) + { + int phase = phaseY + (parameters.Delta * row); + int destinationRowOffset = (tileRow - destinationPosition.Y + row) * destinationStride; + ref ushort intermediateSource = ref intermediate[row * WarpedTileSize]; + ref ushort destinationRow = ref Unsafe.Add( + ref destinationBase, + destinationRowOffset + tileColumn - destinationPosition.X); + + FilterWarpedCompoundVertical( + ref intermediateSource, + ref destinationRow, + tileWidth, + phase, + parameters.Gamma, + verticalBias, + useHardwareIntrinsics); + } + } + } + } + + /// + /// Reconstructs one high-bit-depth warped block through a closed convolution operator. + /// + private static void PredictWarped( + ReadOnlySpan source, + int sourceStride, + Point sourceOrigin, + int sourceWidth, + int sourceHeight, + Span destination, + int destinationStride, + Point destinationPosition, + int width, + int height, + int subsamplingX, + int subsamplingY, + int bitDepth, + Av1GlobalMotionParameters parameters, + Span scratch, + bool useHardwareIntrinsics) + where TOperator : struct, IAv1WarpedPredictionOperator + { + ref ushort sourceBase = ref MemoryMarshal.GetReference(source); + ref ushort destinationBase = ref MemoryMarshal.GetReference(destination); + Span intermediate = MemoryMarshal.Cast(scratch)[..WarpedScratchLength]; + + // Twelve-bit prediction increases round0 by two so the biased horizontal intermediate remains representable + // in sixteen bits. Reducing round1 by the same amount preserves the complete normative Q14 shift. + int intermediateRange = bitDepth + FilterBits - Round0Bits + 2; + int round0 = Round0Bits + Math.Max(intermediateRange - 16, 0); + int verticalRound = (2 * FilterBits) - round0; + int horizontalBias = 1 << (bitDepth + FilterBits - 1); + int verticalBias = 1 << (bitDepth + (2 * FilterBits) - round0); + int maximum = (1 << bitDepth) - 1; + + for (int tileRow = destinationPosition.Y; tileRow < destinationPosition.Y + height; tileRow += WarpedTileSize) + { + for (int tileColumn = destinationPosition.X; tileColumn < destinationPosition.X + width; tileColumn += WarpedTileSize) + { + DeriveWarpedTilePosition( + parameters, + tileColumn, + tileRow, + subsamplingX, + subsamplingY, + out int integerX, + out int integerY, + out int phaseX, + out int phaseY); + + FilterWarpedHorizontal( + ref sourceBase, + sourceStride, + sourceOrigin, + sourceHeight, + integerX, + integerY, + phaseX, + parameters, + intermediate, + horizontalBias, + round0, + useHardwareIntrinsics); + + int tileHeight = Math.Min(WarpedTileSize, destinationPosition.Y + height - tileRow); + int tileWidth = Math.Min(WarpedTileSize, destinationPosition.X + width - tileColumn); + for (int row = 0; row < tileHeight; row++) + { + int phase = phaseY + (parameters.Delta * row); + int destinationRowOffset = (tileRow - destinationPosition.Y + row) * destinationStride; + ref ushort intermediateSource = ref intermediate[row * WarpedTileSize]; + ref ushort destinationRow = ref Unsafe.Add( + ref destinationBase, + destinationRowOffset + tileColumn - destinationPosition.X); + + FilterWarpedVertical( + ref intermediateSource, + ref destinationRow, + tileWidth, + phase, + parameters.Gamma, + verticalBias, + verticalRound, + bitDepth, + maximum, + useHardwareIntrinsics); + } + } + } + } + + /// + /// Projects the center of one 8x8 output tile and derives its integer source position and reduced phases. + /// + private static void DeriveWarpedTilePosition( + Av1GlobalMotionParameters parameters, + int tileColumn, + int tileRow, + int subsamplingX, + int subsamplingY, + out int integerX, + out int integerY, + out int phaseX, + out int phaseY) + { + int sourceX = (tileColumn + 4) << subsamplingX; + int sourceY = (tileRow + 4) << subsamplingY; + long projectedX = ((long)parameters[2] * sourceX) + ((long)parameters[3] * sourceY) + parameters[0]; + long projectedY = ((long)parameters[4] * sourceX) + ((long)parameters[5] * sourceY) + parameters[1]; + long planeX = projectedX >> subsamplingX; + long planeY = projectedY >> subsamplingY; + integerX = (int)(planeX >> Av1GlobalMotionParameters.ModelPrecisionBits); + integerY = (int)(planeY >> Av1GlobalMotionParameters.ModelPrecisionBits); + phaseX = (int)planeX & (Av1GlobalMotionParameters.ModelScale - 1); + phaseY = (int)planeY & (Av1GlobalMotionParameters.ModelScale - 1); + phaseX += (-4 * parameters.Alpha) + (-4 * parameters.Beta); + phaseY += (-4 * parameters.Gamma) + (-4 * parameters.Delta); + + // Shear parameters are quantized to 64-model-unit steps. Clearing the same low bits after the tile-center + // projection keeps negative and positive phases on the exact filter-table grid used by the bitstream model. + phaseX &= -1 << 6; + phaseY &= -1 << 6; + } + + /// + /// Gets a reference to the first coefficient for one reduced warped-filter phase. + /// + private static ref short GetWarpedFilterReference(int phase) + { + int filterIndex = ((phase + (1 << (WarpedDifferencePrecisionBits - 1))) >> WarpedDifferencePrecisionBits) + + WarpedPixelPrecisionShifts; + + return ref Unsafe.Add(ref MemoryMarshal.GetReference(WarpedFilter), filterIndex * FilterCoefficientCount); + } + + /// + /// Divides a nonnegative value by a power of two with nearest-integer rounding. + /// + private static int RoundPowerOfTwoScalar(int value, int bitCount) + => (value + (1 << (bitCount - 1))) >> bitCount; +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1WedgeMask.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1WedgeMask.cs new file mode 100644 index 0000000000..f79da5774f --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1WedgeMask.cs @@ -0,0 +1,204 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; + +/// +/// Produces AV1 wedge masks in caller-owned plane-sized storage. +/// +internal static class Av1WedgeMask +{ + private const int MaximumAlpha = 64; + private const int MasterSize = 64; + + /// + /// Gets the odd-row oblique prototype defined by the reference decoder. + /// + private static ReadOnlySpan MasterObliqueOdd => + [ + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 2, 6, 18, + 37, 53, 60, 63, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, + 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, + ]; + + /// + /// Gets the even-row oblique prototype defined by the reference decoder. + /// + private static ReadOnlySpan MasterObliqueEven => + [ + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 4, 11, 27, + 46, 58, 62, 63, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, + 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, + ]; + + /// + /// Gets the vertical prototype defined by the reference decoder. + /// + private static ReadOnlySpan MasterVertical => + [ + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 7, 21, + 43, 57, 62, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, + 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, + ]; + + /// + /// Gets the codebook used when block height exceeds block width. + /// + private static ReadOnlySpan HeightGreaterCodebook => + [ + 2, 4, 4, 3, 4, 4, 4, 4, 4, 5, 4, 4, + 0, 4, 2, 0, 4, 4, 0, 4, 6, 1, 4, 4, + 2, 4, 2, 2, 4, 6, 5, 4, 2, 5, 4, 6, + 3, 2, 4, 3, 6, 4, 4, 2, 4, 4, 6, 4, + ]; + + /// + /// Gets the codebook used when block width exceeds block height. + /// + private static ReadOnlySpan HeightLessCodebook => + [ + 2, 4, 4, 3, 4, 4, 4, 4, 4, 5, 4, 4, + 1, 2, 4, 1, 4, 4, 1, 6, 4, 0, 4, 4, + 2, 4, 2, 2, 4, 6, 5, 4, 2, 5, 4, 6, + 3, 2, 4, 3, 6, 4, 4, 2, 4, 4, 6, 4, + ]; + + /// + /// Gets the codebook used by square blocks. + /// + private static ReadOnlySpan EqualCodebook => + [ + 2, 4, 4, 3, 4, 4, 4, 4, 4, 5, 4, 4, + 0, 4, 2, 0, 4, 6, 1, 2, 4, 1, 6, 4, + 2, 4, 2, 2, 4, 6, 5, 4, 2, 5, 4, 6, + 3, 2, 4, 3, 6, 4, 4, 2, 4, 4, 6, 4, + ]; + + /// + /// Fills one luma or subsampled chroma mask for a selected wedge. + /// + /// The caller-owned plane mask. + /// The distance between destination rows. + /// The luma block size selecting the wedge codebook. + /// The wedge index in the inclusive range zero through fifteen. + /// The signaled compound wedge orientation. + /// The horizontal plane subsampling shift. + /// The vertical plane subsampling shift. + /// Whether to complement the resulting mask. + public static void Fill( + Span destination, + int destinationStride, + Av1BlockSize blockSize, + int wedgeIndex, + bool wedgeSign, + int subX, + int subY, + bool invert) + { + int lumaWidth = blockSize.GetWidth(); + int lumaHeight = blockSize.GetHeight(); + int width = Math.Max(4, lumaWidth >> subX); + int height = Math.Max(4, lumaHeight >> subY); + ReadOnlySpan codebook = lumaHeight > lumaWidth + ? HeightGreaterCodebook + : lumaHeight < lumaWidth ? HeightLessCodebook : EqualCodebook; + + int codebookOffset = wedgeIndex * 3; + int direction = codebook[codebookOffset]; + int horizontalOffset = (codebook[codebookOffset + 1] * lumaWidth) >> 3; + int verticalOffset = (codebook[codebookOffset + 2] * lumaHeight) >> 3; + bool negative = wedgeSign ^ GetSignFlip(blockSize, wedgeIndex); + int masterRow = (MasterSize / 2) - verticalOffset; + int masterColumn = (MasterSize / 2) - horizontalOffset; + + // Chroma masks are the rounded average of the corresponding two or four luma-mask samples. Producing the + // plane mask once keeps the vector blend contiguous and avoids gathering mask bytes in every SIMD lane. + for (int row = 0; row < height; row++) + { + Span destinationRow = destination.Slice(row * destinationStride, width); + int lumaRow = row << subY; + for (int column = 0; column < width; column++) + { + int lumaColumn = column << subX; + int mask = GetMasterValue(direction, negative, masterRow + lumaRow, masterColumn + lumaColumn); + if (subX != 0) + { + mask += GetMasterValue(direction, negative, masterRow + lumaRow, masterColumn + lumaColumn + 1); + } + + if (subY != 0) + { + int lowerMask = GetMasterValue(direction, negative, masterRow + lumaRow + 1, masterColumn + lumaColumn); + if (subX != 0) + { + lowerMask += GetMasterValue(direction, negative, masterRow + lumaRow + 1, masterColumn + lumaColumn + 1); + } + + mask += lowerMask; + } + + int sampleCountShift = subX + subY; + if (sampleCountShift != 0) + { + mask = (mask + (1 << (sampleCountShift - 1))) >> sampleCountShift; + } + + destinationRow[column] = (byte)(invert ? MaximumAlpha - mask : mask); + } + } + } + + /// + /// Reads one value from the generated 64 by 64 master mask. + /// + private static int GetMasterValue(int direction, bool negative, int row, int column) + { + int value = direction switch + { + 0 => MasterVertical[row], + 1 => MasterVertical[column], + 2 => GetOblique63(column, row), + 3 => GetOblique63(row, column), + 4 => MaximumAlpha - GetOblique63(row, MasterSize - 1 - column), + _ => MaximumAlpha - GetOblique63(column, MasterSize - 1 - row), + }; + + return negative ? MaximumAlpha - value : value; + } + + /// + /// Reads one value from the shifted oblique-63 master prototype. + /// + private static int GetOblique63(int row, int column) + { + bool oddRow = (row & 1) != 0; + int shift = (oddRow ? 15 : 16) - (row >> 1); + int sourceColumn = Av1Math.Clip3(0, MasterSize - 1, column - shift); + return oddRow ? MasterObliqueOdd[sourceColumn] : MasterObliqueEven[sourceColumn]; + } + + /// + /// Gets the reference decoder's canonical sign flip for a block and wedge index. + /// + private static bool GetSignFlip(Av1BlockSize blockSize, int wedgeIndex) + { + ReadOnlySpan signFlips = blockSize switch + { + Av1BlockSize.Block8x8 or Av1BlockSize.Block16x16 or Av1BlockSize.Block32x32 => + [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 0, 1], + Av1BlockSize.Block8x32 => + [1, 1, 1, 1, 0, 1, 1, 1, 0, 1, 0, 1, 1, 1, 0, 1], + Av1BlockSize.Block32x8 => + [1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 0, 1, 0, 1, 0, 1], + _ => + [1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 0, 1, 1, 1, 0, 1], + }; + + return signFlips[wedgeIndex] != 0; + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/IntraBlockCopy/Av1IntraBlockCopyBilinearPredictor.Operator.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/IntraBlockCopy/Av1IntraBlockCopyBilinearPredictor.Operator.cs new file mode 100644 index 0000000000..9454741fc4 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/IntraBlockCopy/Av1IntraBlockCopyBilinearPredictor.Operator.cs @@ -0,0 +1,251 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.IntraBlockCopy; + +/// +/// Defines the closed bilinear intra-block-copy interpolation operator. +/// +internal static partial class Av1IntraBlockCopyBilinearPredictor +{ + /// + /// Defines bilinear intra-block-copy filtering for scalar and SIMD lane groups. + /// + private interface IAv1IntraBlockCopyBilinearOperator + { + /// + /// Filters one 8-bit sample. + /// + /// The integer-position source sample. + /// The source sample one column to the right. + /// The source sample one row below. + /// The source sample one row below and one column to the right. + /// The filtered 8-bit sample. + public static abstract byte Filter(byte topLeft, byte topRight, byte bottomLeft, byte bottomRight); + + /// + /// Filters sixteen 8-bit samples in parallel. + /// + /// The integer-position source samples. + /// The source samples one column to the right. + /// The source samples one row below. + /// The source samples one row below and one column to the right. + /// The filtered 8-bit samples. + public static abstract Vector128 Filter( + Vector128 topLeft, + Vector128 topRight, + Vector128 bottomLeft, + Vector128 bottomRight); + + /// + /// Filters thirty-two 8-bit samples in parallel. + /// + /// The integer-position source samples. + /// The source samples one column to the right. + /// The source samples one row below. + /// The source samples one row below and one column to the right. + /// The filtered 8-bit samples. + public static abstract Vector256 Filter( + Vector256 topLeft, + Vector256 topRight, + Vector256 bottomLeft, + Vector256 bottomRight); + + /// + /// Filters sixty-four 8-bit samples in parallel. + /// + /// The integer-position source samples. + /// The source samples one column to the right. + /// The source samples one row below. + /// The source samples one row below and one column to the right. + /// The filtered 8-bit samples. + public static abstract Vector512 Filter( + Vector512 topLeft, + Vector512 topRight, + Vector512 bottomLeft, + Vector512 bottomRight); + + /// + /// Filters one high-bit-depth sample. + /// + /// The integer-position source sample. + /// The source sample one column to the right. + /// The source sample one row below. + /// The source sample one row below and one column to the right. + /// The filtered high-bit-depth sample. + public static abstract short Filter(short topLeft, short topRight, short bottomLeft, short bottomRight); + + /// + /// Filters eight high-bit-depth samples in parallel. + /// + /// The integer-position source samples. + /// The source samples one column to the right. + /// The source samples one row below. + /// The source samples one row below and one column to the right. + /// The filtered high-bit-depth samples. + public static abstract Vector128 Filter( + Vector128 topLeft, + Vector128 topRight, + Vector128 bottomLeft, + Vector128 bottomRight); + + /// + /// Filters sixteen high-bit-depth samples in parallel. + /// + /// The integer-position source samples. + /// The source samples one column to the right. + /// The source samples one row below. + /// The source samples one row below and one column to the right. + /// The filtered high-bit-depth samples. + public static abstract Vector256 Filter( + Vector256 topLeft, + Vector256 topRight, + Vector256 bottomLeft, + Vector256 bottomRight); + + /// + /// Filters thirty-two high-bit-depth samples in parallel. + /// + /// The integer-position source samples. + /// The source samples one column to the right. + /// The source samples one row below. + /// The source samples one row below and one column to the right. + /// The filtered high-bit-depth samples. + public static abstract Vector512 Filter( + Vector512 topLeft, + Vector512 topRight, + Vector512 bottomLeft, + Vector512 bottomRight); + } + + /// + /// Applies the separable two-dimensional interpolation required when both source axes have a half-sample phase. + /// + /// + /// The offsets in the reference decoder's separable two-pass implementation cancel algebraically to + /// (topLeft + topRight + bottomLeft + bottomRight + 2) >> 2, so the closed operator produces the exact + /// result directly without an intermediate image buffer. + /// + /// Byte lanes widen to unsigned 16-bit halves before the four-source sum, while high-bit-depth lanes widen to + /// unsigned 32-bit halves. Narrowing recombines those halves in source-column order after the rounded result has + /// returned to the original sample range. + /// + private readonly struct IntraBlockCopyBilinearOperator : IAv1IntraBlockCopyBilinearOperator + { + /// + public static byte Filter(byte topLeft, byte topRight, byte bottomLeft, byte bottomRight) + => (byte)((topLeft + topRight + bottomLeft + bottomRight + 2) >> 2); + + /// + public static Vector128 Filter( + Vector128 topLeft, + Vector128 topRight, + Vector128 bottomLeft, + Vector128 bottomRight) + { + (Vector128 topLeftLow, Vector128 topLeftHigh) = Vector128.Widen(topLeft); + (Vector128 topRightLow, Vector128 topRightHigh) = Vector128.Widen(topRight); + (Vector128 bottomLeftLow, Vector128 bottomLeftHigh) = Vector128.Widen(bottomLeft); + (Vector128 bottomRightLow, Vector128 bottomRightHigh) = Vector128.Widen(bottomRight); + + // Four byte samples can sum to 1020, so ushort lanes preserve the complete value before AV1's +2 + // rounding term and divide-by-four shift. Narrowing is exact because the result remains in byte range. + Vector128 low = (topLeftLow + topRightLow + bottomLeftLow + bottomRightLow + Vector128.Create((ushort)2)) >> 2; + Vector128 high = (topLeftHigh + topRightHigh + bottomLeftHigh + bottomRightHigh + Vector128.Create((ushort)2)) >> 2; + return Vector128.Narrow(low, high); + } + + /// + public static Vector256 Filter( + Vector256 topLeft, + Vector256 topRight, + Vector256 bottomLeft, + Vector256 bottomRight) + { + (Vector256 topLeftLow, Vector256 topLeftHigh) = Vector256.Widen(topLeft); + (Vector256 topRightLow, Vector256 topRightHigh) = Vector256.Widen(topRight); + (Vector256 bottomLeftLow, Vector256 bottomLeftHigh) = Vector256.Widen(bottomLeft); + (Vector256 bottomRightLow, Vector256 bottomRightHigh) = Vector256.Widen(bottomRight); + Vector256 rounding = Vector256.Create((ushort)2); + Vector256 low = (topLeftLow + topRightLow + bottomLeftLow + bottomRightLow + rounding) >> 2; + Vector256 high = (topLeftHigh + topRightHigh + bottomLeftHigh + bottomRightHigh + rounding) >> 2; + return Vector256.Narrow(low, high); + } + + /// + public static Vector512 Filter( + Vector512 topLeft, + Vector512 topRight, + Vector512 bottomLeft, + Vector512 bottomRight) + { + (Vector512 topLeftLow, Vector512 topLeftHigh) = Vector512.Widen(topLeft); + (Vector512 topRightLow, Vector512 topRightHigh) = Vector512.Widen(topRight); + (Vector512 bottomLeftLow, Vector512 bottomLeftHigh) = Vector512.Widen(bottomLeft); + (Vector512 bottomRightLow, Vector512 bottomRightHigh) = Vector512.Widen(bottomRight); + Vector512 rounding = Vector512.Create((ushort)2); + Vector512 low = (topLeftLow + topRightLow + bottomLeftLow + bottomRightLow + rounding) >> 2; + Vector512 high = (topLeftHigh + topRightHigh + bottomLeftHigh + bottomRightHigh + rounding) >> 2; + return Vector512.Narrow(low, high); + } + + /// + public static short Filter(short topLeft, short topRight, short bottomLeft, short bottomRight) + => (short)((topLeft + topRight + bottomLeft + bottomRight + 2) >> 2); + + /// + public static Vector128 Filter( + Vector128 topLeft, + Vector128 topRight, + Vector128 bottomLeft, + Vector128 bottomRight) + { + (Vector128 topLeftLow, Vector128 topLeftHigh) = Vector128.Widen(topLeft.AsUInt16()); + (Vector128 topRightLow, Vector128 topRightHigh) = Vector128.Widen(topRight.AsUInt16()); + (Vector128 bottomLeftLow, Vector128 bottomLeftHigh) = Vector128.Widen(bottomLeft.AsUInt16()); + (Vector128 bottomRightLow, Vector128 bottomRightHigh) = Vector128.Widen(bottomRight.AsUInt16()); + + // High-bit-depth storage is signed for integration with transform code, but reconstructed samples are + // nonnegative. Unsigned widening therefore preserves 10- and 12-bit values through the four-input sum. + Vector128 low = (topLeftLow + topRightLow + bottomLeftLow + bottomRightLow + Vector128.Create(2U)) >> 2; + Vector128 high = (topLeftHigh + topRightHigh + bottomLeftHigh + bottomRightHigh + Vector128.Create(2U)) >> 2; + return Vector128.Narrow(low, high).AsInt16(); + } + + /// + public static Vector256 Filter( + Vector256 topLeft, + Vector256 topRight, + Vector256 bottomLeft, + Vector256 bottomRight) + { + (Vector256 topLeftLow, Vector256 topLeftHigh) = Vector256.Widen(topLeft.AsUInt16()); + (Vector256 topRightLow, Vector256 topRightHigh) = Vector256.Widen(topRight.AsUInt16()); + (Vector256 bottomLeftLow, Vector256 bottomLeftHigh) = Vector256.Widen(bottomLeft.AsUInt16()); + (Vector256 bottomRightLow, Vector256 bottomRightHigh) = Vector256.Widen(bottomRight.AsUInt16()); + Vector256 rounding = Vector256.Create(2U); + Vector256 low = (topLeftLow + topRightLow + bottomLeftLow + bottomRightLow + rounding) >> 2; + Vector256 high = (topLeftHigh + topRightHigh + bottomLeftHigh + bottomRightHigh + rounding) >> 2; + return Vector256.Narrow(low, high).AsInt16(); + } + + /// + public static Vector512 Filter( + Vector512 topLeft, + Vector512 topRight, + Vector512 bottomLeft, + Vector512 bottomRight) + { + (Vector512 topLeftLow, Vector512 topLeftHigh) = Vector512.Widen(topLeft.AsUInt16()); + (Vector512 topRightLow, Vector512 topRightHigh) = Vector512.Widen(topRight.AsUInt16()); + (Vector512 bottomLeftLow, Vector512 bottomLeftHigh) = Vector512.Widen(bottomLeft.AsUInt16()); + (Vector512 bottomRightLow, Vector512 bottomRightHigh) = Vector512.Widen(bottomRight.AsUInt16()); + Vector512 rounding = Vector512.Create(2U); + Vector512 low = (topLeftLow + topRightLow + bottomLeftLow + bottomRightLow + rounding) >> 2; + Vector512 high = (topLeftHigh + topRightHigh + bottomLeftHigh + bottomRightHigh + rounding) >> 2; + return Vector512.Narrow(low, high).AsInt16(); + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/IntraBlockCopy/Av1IntraBlockCopyBilinearPredictor.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/IntraBlockCopy/Av1IntraBlockCopyBilinearPredictor.cs new file mode 100644 index 0000000000..3d4555f211 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/IntraBlockCopy/Av1IntraBlockCopyBilinearPredictor.cs @@ -0,0 +1,394 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.InteropServices; +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.IntraBlockCopy; + +/// +/// Provides the family-owned scalar and width-progressive SIMD traversal for bilinear intra-block-copy prediction. +/// +internal static partial class Av1IntraBlockCopyBilinearPredictor +{ + /// + /// Reconstructs an 8-bit filtered intra-block-copy prediction. + /// + public static void Predict( + ReadOnlySpan source, + int sourceStride, + Span destination, + int destinationStride, + int width, + int height) + => Predict(source, sourceStride, destination, destinationStride, width, height); + + /// + /// Reconstructs a high-bit-depth filtered intra-block-copy prediction. + /// + public static void Predict( + ReadOnlySpan source, + int sourceStride, + Span destination, + int destinationStride, + int width, + int height) + => Predict(source, sourceStride, destination, destinationStride, width, height); + + /// + /// Reconstructs an 8-bit filtered intra-block-copy prediction without explicit hardware intrinsics. + /// + public static void PredictScalar( + ReadOnlySpan source, + int sourceStride, + Span destination, + int destinationStride, + int width, + int height) + => PredictScalar(source, sourceStride, destination, destinationStride, width, height); + + /// + /// Reconstructs a high-bit-depth filtered intra-block-copy prediction without explicit hardware intrinsics. + /// + public static void PredictScalar( + ReadOnlySpan source, + int sourceStride, + Span destination, + int destinationStride, + int width, + int height) + => PredictScalar(source, sourceStride, destination, destinationStride, width, height); + + /// + /// Applies one closed interpolation operator to an 8-bit source block. + /// + /// The source-phase-specific interpolation arithmetic. + private static void Predict( + ReadOnlySpan source, + int sourceStride, + Span destination, + int destinationStride, + int width, + int height) + where TOperator : struct, IAv1IntraBlockCopyBilinearOperator + { + ref byte sourceBase = ref MemoryMarshal.GetReference(source); + ref byte destinationBase = ref MemoryMarshal.GetReference(destination); + + if (Vector128.IsHardwareAccelerated && width is 4 or 8) + { + // AV1 permits 4- and 8-sample transform widths, both smaller than a byte Vector128. The frame allocation's + // 72-sample prediction border makes each full source load readable; exact-width stores avoid touching + // destination padding. + for (int row = 0; row < height; row++) + { + ref byte sourceRow = ref Unsafe.Add(ref sourceBase, row * sourceStride); + ref byte destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + Vector128 topLeft = Vector128.LoadUnsafe(ref sourceRow); + Vector128 topRight = Vector128.LoadUnsafe(ref sourceRow, 1); + Vector128 bottomLeft = Vector128.LoadUnsafe(ref sourceRow, (nuint)sourceStride); + Vector128 bottomRight = Vector128.LoadUnsafe(ref sourceRow, (nuint)(sourceStride + 1)); + + Vector128 prediction = TOperator.Filter(topLeft, topRight, bottomLeft, bottomRight); + if (width == 8) + { + prediction.GetLower().StoreUnsafe(ref destinationRow); + } + else + { + Unsafe.As(ref destinationRow) = prediction.AsUInt32().GetElement(0); + } + } + + return; + } + + int processedColumns = 0; + + // AV1 transform widths are powers of two. The widest supported tier normally consumes the complete row; the + // cumulative narrower tiers preserve the same contract for future legal widths without over-reading a tail. + if (Vector512.IsHardwareAccelerated) + { + int vectorizedColumns = (int)(Numerics.Vector512Count(width) * (nuint)Vector512.Count); + if (vectorizedColumns > 0) + { + for (int row = 0; row < height; row++) + { + ref byte sourceRow = ref Unsafe.Add(ref sourceBase, row * sourceStride); + ref byte destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + + for (int column = 0; column < vectorizedColumns; column += Vector512.Count) + { + Vector512 topLeft = Vector512.LoadUnsafe(ref sourceRow, (nuint)column); + Vector512 topRight = Vector512.LoadUnsafe(ref sourceRow, (nuint)(column + 1)); + Vector512 bottomLeft = Vector512.LoadUnsafe(ref sourceRow, (nuint)(sourceStride + column)); + Vector512 bottomRight = Vector512.LoadUnsafe(ref sourceRow, (nuint)(sourceStride + column + 1)); + + TOperator.Filter(topLeft, topRight, bottomLeft, bottomRight).StoreUnsafe(ref destinationRow, (nuint)column); + } + } + + processedColumns = vectorizedColumns; + } + } + + if (Vector256.IsHardwareAccelerated) + { + int remainingColumns = width - processedColumns; + int vectorizedColumns = (int)(Numerics.Vector256Count(remainingColumns) * (nuint)Vector256.Count); + int endColumn = processedColumns + vectorizedColumns; + + for (int row = 0; row < height; row++) + { + ref byte sourceRow = ref Unsafe.Add(ref sourceBase, row * sourceStride); + ref byte destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + + for (int column = processedColumns; column < endColumn; column += Vector256.Count) + { + Vector256 topLeft = Vector256.LoadUnsafe(ref sourceRow, (nuint)column); + Vector256 topRight = Vector256.LoadUnsafe(ref sourceRow, (nuint)(column + 1)); + Vector256 bottomLeft = Vector256.LoadUnsafe(ref sourceRow, (nuint)(sourceStride + column)); + Vector256 bottomRight = Vector256.LoadUnsafe(ref sourceRow, (nuint)(sourceStride + column + 1)); + + TOperator.Filter(topLeft, topRight, bottomLeft, bottomRight).StoreUnsafe(ref destinationRow, (nuint)column); + } + } + + processedColumns = endColumn; + } + + if (Vector128.IsHardwareAccelerated) + { + int remainingColumns = width - processedColumns; + int vectorizedColumns = (int)(Numerics.Vector128Count(remainingColumns) * (nuint)Vector128.Count); + int endColumn = processedColumns + vectorizedColumns; + + for (int row = 0; row < height; row++) + { + ref byte sourceRow = ref Unsafe.Add(ref sourceBase, row * sourceStride); + ref byte destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + + for (int column = processedColumns; column < endColumn; column += Vector128.Count) + { + Vector128 topLeft = Vector128.LoadUnsafe(ref sourceRow, (nuint)column); + Vector128 topRight = Vector128.LoadUnsafe(ref sourceRow, (nuint)(column + 1)); + Vector128 bottomLeft = Vector128.LoadUnsafe(ref sourceRow, (nuint)(sourceStride + column)); + Vector128 bottomRight = Vector128.LoadUnsafe(ref sourceRow, (nuint)(sourceStride + column + 1)); + + TOperator.Filter(topLeft, topRight, bottomLeft, bottomRight).StoreUnsafe(ref destinationRow, (nuint)column); + } + } + + processedColumns = endColumn; + } + + // FeatureTestRunner can disable every intrinsic tier. Keeping the scalar continuation in the same traversal + // proves the fallback without changing source addressing or the normative rounding performed by the operator. + for (int row = 0; row < height; row++) + { + ref byte sourceRow = ref Unsafe.Add(ref sourceBase, row * sourceStride); + ref byte destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + + for (int column = processedColumns; column < width; column++) + { + byte topLeft = Unsafe.Add(ref sourceRow, column); + byte topRight = Unsafe.Add(ref sourceRow, column + 1); + byte bottomLeft = Unsafe.Add(ref sourceRow, sourceStride + column); + byte bottomRight = Unsafe.Add(ref sourceRow, sourceStride + column + 1); + + Unsafe.Add(ref destinationRow, column) = TOperator.Filter(topLeft, topRight, bottomLeft, bottomRight); + } + } + } + + /// + /// Applies one closed interpolation operator to a high-bit-depth source block. + /// + /// The source-phase-specific interpolation arithmetic. + private static void Predict( + ReadOnlySpan source, + int sourceStride, + Span destination, + int destinationStride, + int width, + int height) + where TOperator : struct, IAv1IntraBlockCopyBilinearOperator + { + ref short sourceBase = ref MemoryMarshal.GetReference(source); + ref short destinationBase = ref MemoryMarshal.GetReference(destination); + + if (Vector128.IsHardwareAccelerated && width == 4) + { + // Four high-bit-depth samples occupy the lower half of a Vector128. The frame allocation's prediction + // border makes the full source load readable; storing only the lower four lanes avoids relying on writable + // samples beyond the transform boundary. + for (int row = 0; row < height; row++) + { + ref short sourceRow = ref Unsafe.Add(ref sourceBase, row * sourceStride); + ref short destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + Vector128 topLeft = Vector128.LoadUnsafe(ref sourceRow); + Vector128 topRight = Vector128.LoadUnsafe(ref sourceRow, 1); + Vector128 bottomLeft = Vector128.LoadUnsafe(ref sourceRow, (nuint)sourceStride); + Vector128 bottomRight = Vector128.LoadUnsafe(ref sourceRow, (nuint)(sourceStride + 1)); + + TOperator.Filter(topLeft, topRight, bottomLeft, bottomRight).GetLower().StoreUnsafe(ref destinationRow); + } + + return; + } + + int processedColumns = 0; + + // High-bit-depth lanes hold half as many samples, but retain the same descending-width traversal and one scalar + // continuation as the byte path. + if (Vector512.IsHardwareAccelerated) + { + int vectorizedColumns = (int)(Numerics.Vector512Count(width) * (nuint)Vector512.Count); + if (vectorizedColumns > 0) + { + for (int row = 0; row < height; row++) + { + ref short sourceRow = ref Unsafe.Add(ref sourceBase, row * sourceStride); + ref short destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + + for (int column = 0; column < vectorizedColumns; column += Vector512.Count) + { + Vector512 topLeft = Vector512.LoadUnsafe(ref sourceRow, (nuint)column); + Vector512 topRight = Vector512.LoadUnsafe(ref sourceRow, (nuint)(column + 1)); + Vector512 bottomLeft = Vector512.LoadUnsafe(ref sourceRow, (nuint)(sourceStride + column)); + Vector512 bottomRight = Vector512.LoadUnsafe(ref sourceRow, (nuint)(sourceStride + column + 1)); + + TOperator.Filter(topLeft, topRight, bottomLeft, bottomRight).StoreUnsafe(ref destinationRow, (nuint)column); + } + } + + processedColumns = vectorizedColumns; + } + } + + if (Vector256.IsHardwareAccelerated) + { + int remainingColumns = width - processedColumns; + int vectorizedColumns = (int)(Numerics.Vector256Count(remainingColumns) * (nuint)Vector256.Count); + int endColumn = processedColumns + vectorizedColumns; + + for (int row = 0; row < height; row++) + { + ref short sourceRow = ref Unsafe.Add(ref sourceBase, row * sourceStride); + ref short destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + + for (int column = processedColumns; column < endColumn; column += Vector256.Count) + { + Vector256 topLeft = Vector256.LoadUnsafe(ref sourceRow, (nuint)column); + Vector256 topRight = Vector256.LoadUnsafe(ref sourceRow, (nuint)(column + 1)); + Vector256 bottomLeft = Vector256.LoadUnsafe(ref sourceRow, (nuint)(sourceStride + column)); + Vector256 bottomRight = Vector256.LoadUnsafe(ref sourceRow, (nuint)(sourceStride + column + 1)); + + TOperator.Filter(topLeft, topRight, bottomLeft, bottomRight).StoreUnsafe(ref destinationRow, (nuint)column); + } + } + + processedColumns = endColumn; + } + + if (Vector128.IsHardwareAccelerated) + { + int remainingColumns = width - processedColumns; + int vectorizedColumns = (int)(Numerics.Vector128Count(remainingColumns) * (nuint)Vector128.Count); + int endColumn = processedColumns + vectorizedColumns; + + for (int row = 0; row < height; row++) + { + ref short sourceRow = ref Unsafe.Add(ref sourceBase, row * sourceStride); + ref short destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + + for (int column = processedColumns; column < endColumn; column += Vector128.Count) + { + Vector128 topLeft = Vector128.LoadUnsafe(ref sourceRow, (nuint)column); + Vector128 topRight = Vector128.LoadUnsafe(ref sourceRow, (nuint)(column + 1)); + Vector128 bottomLeft = Vector128.LoadUnsafe(ref sourceRow, (nuint)(sourceStride + column)); + Vector128 bottomRight = Vector128.LoadUnsafe(ref sourceRow, (nuint)(sourceStride + column + 1)); + + TOperator.Filter(topLeft, topRight, bottomLeft, bottomRight).StoreUnsafe(ref destinationRow, (nuint)column); + } + } + + processedColumns = endColumn; + } + + for (int row = 0; row < height; row++) + { + ref short sourceRow = ref Unsafe.Add(ref sourceBase, row * sourceStride); + ref short destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + + for (int column = processedColumns; column < width; column++) + { + short topLeft = Unsafe.Add(ref sourceRow, column); + short topRight = Unsafe.Add(ref sourceRow, column + 1); + short bottomLeft = Unsafe.Add(ref sourceRow, sourceStride + column); + short bottomRight = Unsafe.Add(ref sourceRow, sourceStride + column + 1); + + Unsafe.Add(ref destinationRow, column) = TOperator.Filter(topLeft, topRight, bottomLeft, bottomRight); + } + } + } + + /// + /// Applies one closed interpolation operator to an 8-bit source block without explicit hardware intrinsics. + /// + /// The source-phase-specific interpolation arithmetic. + private static void PredictScalar( + ReadOnlySpan source, + int sourceStride, + Span destination, + int destinationStride, + int width, + int height) + where TOperator : struct, IAv1IntraBlockCopyBilinearOperator + { + for (int row = 0; row < height; row++) + { + int sourceRow = row * sourceStride; + int destinationRow = row * destinationStride; + + for (int column = 0; column < width; column++) + { + byte topLeft = source[sourceRow + column]; + byte topRight = source[sourceRow + column + 1]; + byte bottomLeft = source[sourceRow + sourceStride + column]; + byte bottomRight = source[sourceRow + sourceStride + column + 1]; + destination[destinationRow + column] = TOperator.Filter(topLeft, topRight, bottomLeft, bottomRight); + } + } + } + + /// + /// Applies one closed interpolation operator to a high-bit-depth source block without explicit hardware intrinsics. + /// + /// The source-phase-specific interpolation arithmetic. + private static void PredictScalar( + ReadOnlySpan source, + int sourceStride, + Span destination, + int destinationStride, + int width, + int height) + where TOperator : struct, IAv1IntraBlockCopyBilinearOperator + { + for (int row = 0; row < height; row++) + { + int sourceRow = row * sourceStride; + int destinationRow = row * destinationStride; + + for (int column = 0; column < width; column++) + { + short topLeft = source[sourceRow + column]; + short topRight = source[sourceRow + column + 1]; + short bottomLeft = source[sourceRow + sourceStride + column]; + short bottomRight = source[sourceRow + sourceStride + column + 1]; + destination[destinationRow + column] = TOperator.Filter(topLeft, topRight, bottomLeft, bottomRight); + } + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/IntraBlockCopy/Av1IntraBlockCopyHorizontalPredictor.Operator.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/IntraBlockCopy/Av1IntraBlockCopyHorizontalPredictor.Operator.cs new file mode 100644 index 0000000000..a9a4d4304c --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/IntraBlockCopy/Av1IntraBlockCopyHorizontalPredictor.Operator.cs @@ -0,0 +1,180 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.IntraBlockCopy; + +/// +/// Defines the closed horizontal intra-block-copy interpolation operator. +/// +internal static partial class Av1IntraBlockCopyHorizontalPredictor +{ + /// + /// Defines horizontal intra-block-copy filtering for scalar and SIMD lane groups. + /// + private interface IAv1IntraBlockCopyHorizontalOperator + { + /// + /// Filters one 8-bit sample. + /// + /// The integer-position source sample. + /// The source sample one column to the right. + /// The filtered 8-bit sample. + public static abstract byte Filter(byte left, byte right); + + /// + /// Filters sixteen 8-bit samples in parallel. + /// + /// The integer-position source samples. + /// The source samples one column to the right. + /// The filtered 8-bit samples. + public static abstract Vector128 Filter( + Vector128 left, + Vector128 right); + + /// + /// Filters thirty-two 8-bit samples in parallel. + /// + /// The integer-position source samples. + /// The source samples one column to the right. + /// The filtered 8-bit samples. + public static abstract Vector256 Filter( + Vector256 left, + Vector256 right); + + /// + /// Filters sixty-four 8-bit samples in parallel. + /// + /// The integer-position source samples. + /// The source samples one column to the right. + /// The filtered 8-bit samples. + public static abstract Vector512 Filter( + Vector512 left, + Vector512 right); + + /// + /// Filters one high-bit-depth sample. + /// + /// The integer-position source sample. + /// The source sample one column to the right. + /// The filtered high-bit-depth sample. + public static abstract short Filter(short left, short right); + + /// + /// Filters eight high-bit-depth samples in parallel. + /// + /// The integer-position source samples. + /// The source samples one column to the right. + /// The filtered high-bit-depth samples. + public static abstract Vector128 Filter( + Vector128 left, + Vector128 right); + + /// + /// Filters sixteen high-bit-depth samples in parallel. + /// + /// The integer-position source samples. + /// The source samples one column to the right. + /// The filtered high-bit-depth samples. + public static abstract Vector256 Filter( + Vector256 left, + Vector256 right); + + /// + /// Filters thirty-two high-bit-depth samples in parallel. + /// + /// The integer-position source samples. + /// The source samples one column to the right. + /// The filtered high-bit-depth samples. + public static abstract Vector512 Filter( + Vector512 left, + Vector512 right); + } + + /// + /// Averages horizontally adjacent source samples for a half-sample horizontal phase. + /// + private readonly struct IntraBlockCopyHorizontalOperator : IAv1IntraBlockCopyHorizontalOperator + { + /// + public static byte Filter(byte left, byte right) => (byte)((left + right + 1) >> 1); + + /// + public static Vector128 Filter(Vector128 left, Vector128 right) + => AverageRounded(left, right); + + /// + public static Vector256 Filter(Vector256 left, Vector256 right) + => AverageRounded(left, right); + + /// + public static Vector512 Filter(Vector512 left, Vector512 right) + => AverageRounded(left, right); + + /// + public static short Filter(short left, short right) => (short)((left + right + 1) >> 1); + + /// + public static Vector128 Filter(Vector128 left, Vector128 right) + => AverageRounded(left, right); + + /// + public static Vector256 Filter(Vector256 left, Vector256 right) + => AverageRounded(left, right); + + /// + public static Vector512 Filter(Vector512 left, Vector512 right) + => AverageRounded(left, right); + + /// + /// Computes a rounded average without overflowing unsigned byte lanes. + /// + private static Vector128 AverageRounded(Vector128 left, Vector128 right) + => (left | right) - ((left ^ right) >> 1); + + /// + /// Computes a rounded average without overflowing unsigned byte lanes. + /// + private static Vector256 AverageRounded(Vector256 left, Vector256 right) + => (left | right) - ((left ^ right) >> 1); + + /// + /// Computes a rounded average without overflowing unsigned byte lanes. + /// + private static Vector512 AverageRounded(Vector512 left, Vector512 right) + => (left | right) - ((left ^ right) >> 1); + + /// + /// Computes a rounded average without overflowing nonnegative high-bit-depth lanes. + /// + private static Vector128 AverageRounded(Vector128 left, Vector128 right) + { + Vector128 leftUnsigned = left.AsUInt16(); + Vector128 rightUnsigned = right.AsUInt16(); + + // This identity computes ceil((a + b) / 2) without an overflowing lane-wise addition. + return ((leftUnsigned | rightUnsigned) - ((leftUnsigned ^ rightUnsigned) >> 1)).AsInt16(); + } + + /// + /// Computes a rounded average without overflowing nonnegative high-bit-depth lanes. + /// + private static Vector256 AverageRounded(Vector256 left, Vector256 right) + { + Vector256 leftUnsigned = left.AsUInt16(); + Vector256 rightUnsigned = right.AsUInt16(); + return ((leftUnsigned | rightUnsigned) - ((leftUnsigned ^ rightUnsigned) >> 1)).AsInt16(); + } + + /// + /// Computes a rounded average without overflowing nonnegative high-bit-depth lanes. + /// + private static Vector512 AverageRounded(Vector512 left, Vector512 right) + { + Vector512 leftUnsigned = left.AsUInt16(); + Vector512 rightUnsigned = right.AsUInt16(); + return ((leftUnsigned | rightUnsigned) - ((leftUnsigned ^ rightUnsigned) >> 1)).AsInt16(); + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/IntraBlockCopy/Av1IntraBlockCopyHorizontalPredictor.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/IntraBlockCopy/Av1IntraBlockCopyHorizontalPredictor.cs new file mode 100644 index 0000000000..597f3a8872 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/IntraBlockCopy/Av1IntraBlockCopyHorizontalPredictor.cs @@ -0,0 +1,370 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.InteropServices; +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.IntraBlockCopy; + +/// +/// Provides the family-owned scalar and width-progressive SIMD traversal for horizontal intra-block-copy prediction. +/// +internal static partial class Av1IntraBlockCopyHorizontalPredictor +{ + /// + /// Reconstructs an 8-bit filtered intra-block-copy prediction. + /// + public static void Predict( + ReadOnlySpan source, + int sourceStride, + Span destination, + int destinationStride, + int width, + int height) + => Predict(source, sourceStride, destination, destinationStride, width, height); + + /// + /// Reconstructs a high-bit-depth filtered intra-block-copy prediction. + /// + public static void Predict( + ReadOnlySpan source, + int sourceStride, + Span destination, + int destinationStride, + int width, + int height) + => Predict(source, sourceStride, destination, destinationStride, width, height); + + /// + /// Reconstructs an 8-bit filtered intra-block-copy prediction without explicit hardware intrinsics. + /// + public static void PredictScalar( + ReadOnlySpan source, + int sourceStride, + Span destination, + int destinationStride, + int width, + int height) + => PredictScalar(source, sourceStride, destination, destinationStride, width, height); + + /// + /// Reconstructs a high-bit-depth filtered intra-block-copy prediction without explicit hardware intrinsics. + /// + public static void PredictScalar( + ReadOnlySpan source, + int sourceStride, + Span destination, + int destinationStride, + int width, + int height) + => PredictScalar(source, sourceStride, destination, destinationStride, width, height); + + /// + /// Applies one closed interpolation operator to an 8-bit source block. + /// + /// The source-phase-specific interpolation arithmetic. + private static void Predict( + ReadOnlySpan source, + int sourceStride, + Span destination, + int destinationStride, + int width, + int height) + where TOperator : struct, IAv1IntraBlockCopyHorizontalOperator + { + ref byte sourceBase = ref MemoryMarshal.GetReference(source); + ref byte destinationBase = ref MemoryMarshal.GetReference(destination); + + if (Vector128.IsHardwareAccelerated && width is 4 or 8) + { + // AV1 permits 4- and 8-sample transform widths, both smaller than a byte Vector128. The frame allocation's + // 72-sample prediction border makes each full source load readable; exact-width stores avoid touching + // destination padding. + for (int row = 0; row < height; row++) + { + ref byte sourceRow = ref Unsafe.Add(ref sourceBase, row * sourceStride); + ref byte destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + Vector128 topLeft = Vector128.LoadUnsafe(ref sourceRow); + Vector128 topRight = Vector128.LoadUnsafe(ref sourceRow, 1); + + Vector128 prediction = TOperator.Filter(topLeft, topRight); + if (width == 8) + { + prediction.GetLower().StoreUnsafe(ref destinationRow); + } + else + { + Unsafe.As(ref destinationRow) = prediction.AsUInt32().GetElement(0); + } + } + + return; + } + + int processedColumns = 0; + + // AV1 transform widths are powers of two. The widest supported tier normally consumes the complete row; the + // cumulative narrower tiers preserve the same contract for future legal widths without over-reading a tail. + if (Vector512.IsHardwareAccelerated) + { + int vectorizedColumns = (int)(Numerics.Vector512Count(width) * (nuint)Vector512.Count); + if (vectorizedColumns > 0) + { + for (int row = 0; row < height; row++) + { + ref byte sourceRow = ref Unsafe.Add(ref sourceBase, row * sourceStride); + ref byte destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + + for (int column = 0; column < vectorizedColumns; column += Vector512.Count) + { + Vector512 topLeft = Vector512.LoadUnsafe(ref sourceRow, (nuint)column); + Vector512 topRight = Vector512.LoadUnsafe(ref sourceRow, (nuint)(column + 1)); + + TOperator.Filter(topLeft, topRight).StoreUnsafe(ref destinationRow, (nuint)column); + } + } + + processedColumns = vectorizedColumns; + } + } + + if (Vector256.IsHardwareAccelerated) + { + int remainingColumns = width - processedColumns; + int vectorizedColumns = (int)(Numerics.Vector256Count(remainingColumns) * (nuint)Vector256.Count); + int endColumn = processedColumns + vectorizedColumns; + + for (int row = 0; row < height; row++) + { + ref byte sourceRow = ref Unsafe.Add(ref sourceBase, row * sourceStride); + ref byte destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + + for (int column = processedColumns; column < endColumn; column += Vector256.Count) + { + Vector256 topLeft = Vector256.LoadUnsafe(ref sourceRow, (nuint)column); + Vector256 topRight = Vector256.LoadUnsafe(ref sourceRow, (nuint)(column + 1)); + + TOperator.Filter(topLeft, topRight).StoreUnsafe(ref destinationRow, (nuint)column); + } + } + + processedColumns = endColumn; + } + + if (Vector128.IsHardwareAccelerated) + { + int remainingColumns = width - processedColumns; + int vectorizedColumns = (int)(Numerics.Vector128Count(remainingColumns) * (nuint)Vector128.Count); + int endColumn = processedColumns + vectorizedColumns; + + for (int row = 0; row < height; row++) + { + ref byte sourceRow = ref Unsafe.Add(ref sourceBase, row * sourceStride); + ref byte destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + + for (int column = processedColumns; column < endColumn; column += Vector128.Count) + { + Vector128 topLeft = Vector128.LoadUnsafe(ref sourceRow, (nuint)column); + Vector128 topRight = Vector128.LoadUnsafe(ref sourceRow, (nuint)(column + 1)); + + TOperator.Filter(topLeft, topRight).StoreUnsafe(ref destinationRow, (nuint)column); + } + } + + processedColumns = endColumn; + } + + // FeatureTestRunner can disable every intrinsic tier. Keeping the scalar continuation in the same traversal + // proves the fallback without changing source addressing or the normative rounding performed by the operator. + for (int row = 0; row < height; row++) + { + ref byte sourceRow = ref Unsafe.Add(ref sourceBase, row * sourceStride); + ref byte destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + + for (int column = processedColumns; column < width; column++) + { + byte topLeft = Unsafe.Add(ref sourceRow, column); + byte topRight = Unsafe.Add(ref sourceRow, column + 1); + + Unsafe.Add(ref destinationRow, column) = TOperator.Filter(topLeft, topRight); + } + } + } + + /// + /// Applies one closed interpolation operator to a high-bit-depth source block. + /// + /// The source-phase-specific interpolation arithmetic. + private static void Predict( + ReadOnlySpan source, + int sourceStride, + Span destination, + int destinationStride, + int width, + int height) + where TOperator : struct, IAv1IntraBlockCopyHorizontalOperator + { + ref short sourceBase = ref MemoryMarshal.GetReference(source); + ref short destinationBase = ref MemoryMarshal.GetReference(destination); + + if (Vector128.IsHardwareAccelerated && width == 4) + { + // Four high-bit-depth samples occupy the lower half of a Vector128. The frame allocation's prediction + // border makes the full source load readable; storing only the lower four lanes avoids relying on writable + // samples beyond the transform boundary. + for (int row = 0; row < height; row++) + { + ref short sourceRow = ref Unsafe.Add(ref sourceBase, row * sourceStride); + ref short destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + Vector128 topLeft = Vector128.LoadUnsafe(ref sourceRow); + Vector128 topRight = Vector128.LoadUnsafe(ref sourceRow, 1); + + TOperator.Filter(topLeft, topRight).GetLower().StoreUnsafe(ref destinationRow); + } + + return; + } + + int processedColumns = 0; + + // High-bit-depth lanes hold half as many samples, but retain the same descending-width traversal and one scalar + // continuation as the byte path. + if (Vector512.IsHardwareAccelerated) + { + int vectorizedColumns = (int)(Numerics.Vector512Count(width) * (nuint)Vector512.Count); + if (vectorizedColumns > 0) + { + for (int row = 0; row < height; row++) + { + ref short sourceRow = ref Unsafe.Add(ref sourceBase, row * sourceStride); + ref short destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + + for (int column = 0; column < vectorizedColumns; column += Vector512.Count) + { + Vector512 topLeft = Vector512.LoadUnsafe(ref sourceRow, (nuint)column); + Vector512 topRight = Vector512.LoadUnsafe(ref sourceRow, (nuint)(column + 1)); + + TOperator.Filter(topLeft, topRight).StoreUnsafe(ref destinationRow, (nuint)column); + } + } + + processedColumns = vectorizedColumns; + } + } + + if (Vector256.IsHardwareAccelerated) + { + int remainingColumns = width - processedColumns; + int vectorizedColumns = (int)(Numerics.Vector256Count(remainingColumns) * (nuint)Vector256.Count); + int endColumn = processedColumns + vectorizedColumns; + + for (int row = 0; row < height; row++) + { + ref short sourceRow = ref Unsafe.Add(ref sourceBase, row * sourceStride); + ref short destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + + for (int column = processedColumns; column < endColumn; column += Vector256.Count) + { + Vector256 topLeft = Vector256.LoadUnsafe(ref sourceRow, (nuint)column); + Vector256 topRight = Vector256.LoadUnsafe(ref sourceRow, (nuint)(column + 1)); + + TOperator.Filter(topLeft, topRight).StoreUnsafe(ref destinationRow, (nuint)column); + } + } + + processedColumns = endColumn; + } + + if (Vector128.IsHardwareAccelerated) + { + int remainingColumns = width - processedColumns; + int vectorizedColumns = (int)(Numerics.Vector128Count(remainingColumns) * (nuint)Vector128.Count); + int endColumn = processedColumns + vectorizedColumns; + + for (int row = 0; row < height; row++) + { + ref short sourceRow = ref Unsafe.Add(ref sourceBase, row * sourceStride); + ref short destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + + for (int column = processedColumns; column < endColumn; column += Vector128.Count) + { + Vector128 topLeft = Vector128.LoadUnsafe(ref sourceRow, (nuint)column); + Vector128 topRight = Vector128.LoadUnsafe(ref sourceRow, (nuint)(column + 1)); + + TOperator.Filter(topLeft, topRight).StoreUnsafe(ref destinationRow, (nuint)column); + } + } + + processedColumns = endColumn; + } + + for (int row = 0; row < height; row++) + { + ref short sourceRow = ref Unsafe.Add(ref sourceBase, row * sourceStride); + ref short destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + + for (int column = processedColumns; column < width; column++) + { + short topLeft = Unsafe.Add(ref sourceRow, column); + short topRight = Unsafe.Add(ref sourceRow, column + 1); + + Unsafe.Add(ref destinationRow, column) = TOperator.Filter(topLeft, topRight); + } + } + } + + /// + /// Applies one closed interpolation operator to an 8-bit source block without explicit hardware intrinsics. + /// + /// The source-phase-specific interpolation arithmetic. + private static void PredictScalar( + ReadOnlySpan source, + int sourceStride, + Span destination, + int destinationStride, + int width, + int height) + where TOperator : struct, IAv1IntraBlockCopyHorizontalOperator + { + for (int row = 0; row < height; row++) + { + int sourceRow = row * sourceStride; + int destinationRow = row * destinationStride; + + for (int column = 0; column < width; column++) + { + byte topLeft = source[sourceRow + column]; + byte topRight = source[sourceRow + column + 1]; + destination[destinationRow + column] = TOperator.Filter(topLeft, topRight); + } + } + } + + /// + /// Applies one closed interpolation operator to a high-bit-depth source block without explicit hardware intrinsics. + /// + /// The source-phase-specific interpolation arithmetic. + private static void PredictScalar( + ReadOnlySpan source, + int sourceStride, + Span destination, + int destinationStride, + int width, + int height) + where TOperator : struct, IAv1IntraBlockCopyHorizontalOperator + { + for (int row = 0; row < height; row++) + { + int sourceRow = row * sourceStride; + int destinationRow = row * destinationStride; + + for (int column = 0; column < width; column++) + { + short topLeft = source[sourceRow + column]; + short topRight = source[sourceRow + column + 1]; + destination[destinationRow + column] = TOperator.Filter(topLeft, topRight); + } + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/IntraBlockCopy/Av1IntraBlockCopyPredictor.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/IntraBlockCopy/Av1IntraBlockCopyPredictor.cs new file mode 100644 index 0000000000..8e75b35730 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/IntraBlockCopy/Av1IntraBlockCopyPredictor.cs @@ -0,0 +1,231 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.IntraBlockCopy; + +/// +/// Reconstructs AV1 intra-block-copy predictions from an earlier region of the current frame. +/// +/// +/// Whole-sample luma displacements can map to half-sample chroma positions. The predictor therefore selects direct +/// copy, horizontal two-tap, vertical two-tap, or separable two-dimensional bilinear reconstruction per plane. +/// Filtered paths use the widest preferred SIMD width and retain an explicit scalar fallback for feature-disabled +/// execution. Narrow rows read from the frame buffer's prediction padding but use exact-width stores, so vectorization +/// never depends on writable destination padding. +/// +internal static partial class Av1IntraBlockCopyPredictor +{ + /// + /// Reconstructs an 8-bit intra-block-copy prediction. + /// + /// The source region beginning at the integer sample preceding any half-sample phase. + /// The distance, in samples, between source rows. + /// The destination block origin. + /// The distance, in samples, between destination rows. + /// The prediction width in samples. + /// The prediction height in samples. + /// Indicates whether the horizontal source phase is one half-sample. + /// Indicates whether the vertical source phase is one half-sample. + public static void Predict( + ReadOnlySpan source, + int sourceStride, + Span destination, + int destinationStride, + int width, + int height, + bool halfX, + bool halfY) + { + if (!halfX && !halfY) + { + Copy(source, sourceStride, destination, destinationStride, width, height); + } + else if (halfX && halfY) + { + Av1IntraBlockCopyBilinearPredictor.Predict(source, sourceStride, destination, destinationStride, width, height); + } + else if (halfX) + { + Av1IntraBlockCopyHorizontalPredictor.Predict(source, sourceStride, destination, destinationStride, width, height); + } + else + { + Av1IntraBlockCopyVerticalPredictor.Predict(source, sourceStride, destination, destinationStride, width, height); + } + } + + /// + /// Reconstructs a high-bit-depth intra-block-copy prediction. + /// + /// The source region beginning at the integer sample preceding any half-sample phase. + /// The distance, in samples, between source rows. + /// The destination block origin. + /// The distance, in samples, between destination rows. + /// The prediction width in samples. + /// The prediction height in samples. + /// Indicates whether the horizontal source phase is one half-sample. + /// Indicates whether the vertical source phase is one half-sample. + public static void Predict( + ReadOnlySpan source, + int sourceStride, + Span destination, + int destinationStride, + int width, + int height, + bool halfX, + bool halfY) + { + if (!halfX && !halfY) + { + Copy(source, sourceStride, destination, destinationStride, width, height); + } + else if (halfX && halfY) + { + Av1IntraBlockCopyBilinearPredictor.Predict(source, sourceStride, destination, destinationStride, width, height); + } + else if (halfX) + { + Av1IntraBlockCopyHorizontalPredictor.Predict(source, sourceStride, destination, destinationStride, width, height); + } + else + { + Av1IntraBlockCopyVerticalPredictor.Predict(source, sourceStride, destination, destinationStride, width, height); + } + } + + /// + /// Reconstructs an 8-bit intra-block-copy prediction without explicit hardware intrinsics. + /// + /// The source region beginning at the integer sample preceding any half-sample phase. + /// The distance, in samples, between source rows. + /// The destination block origin. + /// The distance, in samples, between destination rows. + /// The prediction width in samples. + /// The prediction height in samples. + /// Indicates whether the horizontal source phase is one half-sample. + /// Indicates whether the vertical source phase is one half-sample. + public static void PredictScalar( + ReadOnlySpan source, + int sourceStride, + Span destination, + int destinationStride, + int width, + int height, + bool halfX, + bool halfY) + { + if (!halfX && !halfY) + { + CopyScalar(source, sourceStride, destination, destinationStride, width, height); + } + else if (halfX && halfY) + { + Av1IntraBlockCopyBilinearPredictor.PredictScalar(source, sourceStride, destination, destinationStride, width, height); + } + else if (halfX) + { + Av1IntraBlockCopyHorizontalPredictor.PredictScalar(source, sourceStride, destination, destinationStride, width, height); + } + else + { + Av1IntraBlockCopyVerticalPredictor.PredictScalar(source, sourceStride, destination, destinationStride, width, height); + } + } + + /// + /// Reconstructs a high-bit-depth intra-block-copy prediction without explicit hardware intrinsics. + /// + /// The source region beginning at the integer sample preceding any half-sample phase. + /// The distance, in samples, between source rows. + /// The destination block origin. + /// The distance, in samples, between destination rows. + /// The prediction width in samples. + /// The prediction height in samples. + /// Indicates whether the horizontal source phase is one half-sample. + /// Indicates whether the vertical source phase is one half-sample. + public static void PredictScalar( + ReadOnlySpan source, + int sourceStride, + Span destination, + int destinationStride, + int width, + int height, + bool halfX, + bool halfY) + { + if (!halfX && !halfY) + { + CopyScalar(source, sourceStride, destination, destinationStride, width, height); + } + else if (halfX && halfY) + { + Av1IntraBlockCopyBilinearPredictor.PredictScalar(source, sourceStride, destination, destinationStride, width, height); + } + else if (halfX) + { + Av1IntraBlockCopyHorizontalPredictor.PredictScalar(source, sourceStride, destination, destinationStride, width, height); + } + else + { + Av1IntraBlockCopyVerticalPredictor.PredictScalar(source, sourceStride, destination, destinationStride, width, height); + } + } + + /// + /// Copies an 8-bit whole-sample source block to its destination. + /// + private static void Copy(ReadOnlySpan source, int sourceStride, Span destination, int destinationStride, int width, int height) + { + // Span copying delegates each complete row to the runtime's overlap-safe native-width implementation. The + // displacement validity rules keep source and destination blocks separate, so no intermediate buffer is needed. + for (int row = 0; row < height; row++) + { + source.Slice(row * sourceStride, width).CopyTo(destination.Slice(row * destinationStride, width)); + } + } + + /// + /// Copies a high-bit-depth whole-sample source block to its destination. + /// + private static void Copy(ReadOnlySpan source, int sourceStride, Span destination, int destinationStride, int width, int height) + { + for (int row = 0; row < height; row++) + { + source.Slice(row * sourceStride, width).CopyTo(destination.Slice(row * destinationStride, width)); + } + } + + /// + /// Copies an 8-bit whole-sample source block with scalar sample assignments. + /// + private static void CopyScalar(ReadOnlySpan source, int sourceStride, Span destination, int destinationStride, int width, int height) + { + for (int row = 0; row < height; row++) + { + for (int column = 0; column < width; column++) + { + destination[(row * destinationStride) + column] = source[(row * sourceStride) + column]; + } + } + } + + /// + /// Copies a high-bit-depth whole-sample source block with scalar sample assignments. + /// + private static void CopyScalar( + ReadOnlySpan source, + int sourceStride, + Span destination, + int destinationStride, + int width, + int height) + { + for (int row = 0; row < height; row++) + { + for (int column = 0; column < width; column++) + { + destination[(row * destinationStride) + column] = source[(row * sourceStride) + column]; + } + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/IntraBlockCopy/Av1IntraBlockCopyVerticalPredictor.Operator.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/IntraBlockCopy/Av1IntraBlockCopyVerticalPredictor.Operator.cs new file mode 100644 index 0000000000..2757c6e39c --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/IntraBlockCopy/Av1IntraBlockCopyVerticalPredictor.Operator.cs @@ -0,0 +1,180 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.IntraBlockCopy; + +/// +/// Defines the closed vertical intra-block-copy interpolation operator. +/// +internal static partial class Av1IntraBlockCopyVerticalPredictor +{ + /// + /// Defines vertical intra-block-copy filtering for scalar and SIMD lane groups. + /// + private interface IAv1IntraBlockCopyVerticalOperator + { + /// + /// Filters one 8-bit sample. + /// + /// The integer-position source sample. + /// The source sample one row below. + /// The filtered 8-bit sample. + public static abstract byte Filter(byte top, byte bottom); + + /// + /// Filters sixteen 8-bit samples in parallel. + /// + /// The integer-position source samples. + /// The source samples one row below. + /// The filtered 8-bit samples. + public static abstract Vector128 Filter( + Vector128 top, + Vector128 bottom); + + /// + /// Filters thirty-two 8-bit samples in parallel. + /// + /// The integer-position source samples. + /// The source samples one row below. + /// The filtered 8-bit samples. + public static abstract Vector256 Filter( + Vector256 top, + Vector256 bottom); + + /// + /// Filters sixty-four 8-bit samples in parallel. + /// + /// The integer-position source samples. + /// The source samples one row below. + /// The filtered 8-bit samples. + public static abstract Vector512 Filter( + Vector512 top, + Vector512 bottom); + + /// + /// Filters one high-bit-depth sample. + /// + /// The integer-position source sample. + /// The source sample one row below. + /// The filtered high-bit-depth sample. + public static abstract short Filter(short top, short bottom); + + /// + /// Filters eight high-bit-depth samples in parallel. + /// + /// The integer-position source samples. + /// The source samples one row below. + /// The filtered high-bit-depth samples. + public static abstract Vector128 Filter( + Vector128 top, + Vector128 bottom); + + /// + /// Filters sixteen high-bit-depth samples in parallel. + /// + /// The integer-position source samples. + /// The source samples one row below. + /// The filtered high-bit-depth samples. + public static abstract Vector256 Filter( + Vector256 top, + Vector256 bottom); + + /// + /// Filters thirty-two high-bit-depth samples in parallel. + /// + /// The integer-position source samples. + /// The source samples one row below. + /// The filtered high-bit-depth samples. + public static abstract Vector512 Filter( + Vector512 top, + Vector512 bottom); + } + + /// + /// Averages vertically adjacent source samples for a half-sample vertical phase. + /// + private readonly struct IntraBlockCopyVerticalOperator : IAv1IntraBlockCopyVerticalOperator + { + /// + public static byte Filter(byte top, byte bottom) => (byte)((top + bottom + 1) >> 1); + + /// + public static Vector128 Filter(Vector128 top, Vector128 bottom) + => AverageRounded(top, bottom); + + /// + public static Vector256 Filter(Vector256 top, Vector256 bottom) + => AverageRounded(top, bottom); + + /// + public static Vector512 Filter(Vector512 top, Vector512 bottom) + => AverageRounded(top, bottom); + + /// + public static short Filter(short top, short bottom) => (short)((top + bottom + 1) >> 1); + + /// + public static Vector128 Filter(Vector128 top, Vector128 bottom) + => AverageRounded(top, bottom); + + /// + public static Vector256 Filter(Vector256 top, Vector256 bottom) + => AverageRounded(top, bottom); + + /// + public static Vector512 Filter(Vector512 top, Vector512 bottom) + => AverageRounded(top, bottom); + + /// + /// Computes a rounded average without overflowing unsigned byte lanes. + /// + private static Vector128 AverageRounded(Vector128 left, Vector128 right) + => (left | right) - ((left ^ right) >> 1); + + /// + /// Computes a rounded average without overflowing unsigned byte lanes. + /// + private static Vector256 AverageRounded(Vector256 left, Vector256 right) + => (left | right) - ((left ^ right) >> 1); + + /// + /// Computes a rounded average without overflowing unsigned byte lanes. + /// + private static Vector512 AverageRounded(Vector512 left, Vector512 right) + => (left | right) - ((left ^ right) >> 1); + + /// + /// Computes a rounded average without overflowing nonnegative high-bit-depth lanes. + /// + private static Vector128 AverageRounded(Vector128 left, Vector128 right) + { + Vector128 leftUnsigned = left.AsUInt16(); + Vector128 rightUnsigned = right.AsUInt16(); + + // This identity computes ceil((a + b) / 2) without an overflowing lane-wise addition. + return ((leftUnsigned | rightUnsigned) - ((leftUnsigned ^ rightUnsigned) >> 1)).AsInt16(); + } + + /// + /// Computes a rounded average without overflowing nonnegative high-bit-depth lanes. + /// + private static Vector256 AverageRounded(Vector256 left, Vector256 right) + { + Vector256 leftUnsigned = left.AsUInt16(); + Vector256 rightUnsigned = right.AsUInt16(); + return ((leftUnsigned | rightUnsigned) - ((leftUnsigned ^ rightUnsigned) >> 1)).AsInt16(); + } + + /// + /// Computes a rounded average without overflowing nonnegative high-bit-depth lanes. + /// + private static Vector512 AverageRounded(Vector512 left, Vector512 right) + { + Vector512 leftUnsigned = left.AsUInt16(); + Vector512 rightUnsigned = right.AsUInt16(); + return ((leftUnsigned | rightUnsigned) - ((leftUnsigned ^ rightUnsigned) >> 1)).AsInt16(); + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/IntraBlockCopy/Av1IntraBlockCopyVerticalPredictor.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/IntraBlockCopy/Av1IntraBlockCopyVerticalPredictor.cs new file mode 100644 index 0000000000..e3afaec567 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/IntraBlockCopy/Av1IntraBlockCopyVerticalPredictor.cs @@ -0,0 +1,370 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.InteropServices; +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.IntraBlockCopy; + +/// +/// Provides the family-owned scalar and width-progressive SIMD traversal for vertical intra-block-copy prediction. +/// +internal static partial class Av1IntraBlockCopyVerticalPredictor +{ + /// + /// Reconstructs an 8-bit filtered intra-block-copy prediction. + /// + public static void Predict( + ReadOnlySpan source, + int sourceStride, + Span destination, + int destinationStride, + int width, + int height) + => Predict(source, sourceStride, destination, destinationStride, width, height); + + /// + /// Reconstructs a high-bit-depth filtered intra-block-copy prediction. + /// + public static void Predict( + ReadOnlySpan source, + int sourceStride, + Span destination, + int destinationStride, + int width, + int height) + => Predict(source, sourceStride, destination, destinationStride, width, height); + + /// + /// Reconstructs an 8-bit filtered intra-block-copy prediction without explicit hardware intrinsics. + /// + public static void PredictScalar( + ReadOnlySpan source, + int sourceStride, + Span destination, + int destinationStride, + int width, + int height) + => PredictScalar(source, sourceStride, destination, destinationStride, width, height); + + /// + /// Reconstructs a high-bit-depth filtered intra-block-copy prediction without explicit hardware intrinsics. + /// + public static void PredictScalar( + ReadOnlySpan source, + int sourceStride, + Span destination, + int destinationStride, + int width, + int height) + => PredictScalar(source, sourceStride, destination, destinationStride, width, height); + + /// + /// Applies one closed interpolation operator to an 8-bit source block. + /// + /// The source-phase-specific interpolation arithmetic. + private static void Predict( + ReadOnlySpan source, + int sourceStride, + Span destination, + int destinationStride, + int width, + int height) + where TOperator : struct, IAv1IntraBlockCopyVerticalOperator + { + ref byte sourceBase = ref MemoryMarshal.GetReference(source); + ref byte destinationBase = ref MemoryMarshal.GetReference(destination); + + if (Vector128.IsHardwareAccelerated && width is 4 or 8) + { + // AV1 permits 4- and 8-sample transform widths, both smaller than a byte Vector128. The frame allocation's + // 72-sample prediction border makes each full source load readable; exact-width stores avoid touching + // destination padding. + for (int row = 0; row < height; row++) + { + ref byte sourceRow = ref Unsafe.Add(ref sourceBase, row * sourceStride); + ref byte destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + Vector128 topLeft = Vector128.LoadUnsafe(ref sourceRow); + Vector128 bottomLeft = Vector128.LoadUnsafe(ref sourceRow, (nuint)sourceStride); + + Vector128 prediction = TOperator.Filter(topLeft, bottomLeft); + if (width == 8) + { + prediction.GetLower().StoreUnsafe(ref destinationRow); + } + else + { + Unsafe.As(ref destinationRow) = prediction.AsUInt32().GetElement(0); + } + } + + return; + } + + int processedColumns = 0; + + // AV1 transform widths are powers of two. The widest supported tier normally consumes the complete row; the + // cumulative narrower tiers preserve the same contract for future legal widths without over-reading a tail. + if (Vector512.IsHardwareAccelerated) + { + int vectorizedColumns = (int)(Numerics.Vector512Count(width) * (nuint)Vector512.Count); + if (vectorizedColumns > 0) + { + for (int row = 0; row < height; row++) + { + ref byte sourceRow = ref Unsafe.Add(ref sourceBase, row * sourceStride); + ref byte destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + + for (int column = 0; column < vectorizedColumns; column += Vector512.Count) + { + Vector512 topLeft = Vector512.LoadUnsafe(ref sourceRow, (nuint)column); + Vector512 bottomLeft = Vector512.LoadUnsafe(ref sourceRow, (nuint)(sourceStride + column)); + + TOperator.Filter(topLeft, bottomLeft).StoreUnsafe(ref destinationRow, (nuint)column); + } + } + + processedColumns = vectorizedColumns; + } + } + + if (Vector256.IsHardwareAccelerated) + { + int remainingColumns = width - processedColumns; + int vectorizedColumns = (int)(Numerics.Vector256Count(remainingColumns) * (nuint)Vector256.Count); + int endColumn = processedColumns + vectorizedColumns; + + for (int row = 0; row < height; row++) + { + ref byte sourceRow = ref Unsafe.Add(ref sourceBase, row * sourceStride); + ref byte destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + + for (int column = processedColumns; column < endColumn; column += Vector256.Count) + { + Vector256 topLeft = Vector256.LoadUnsafe(ref sourceRow, (nuint)column); + Vector256 bottomLeft = Vector256.LoadUnsafe(ref sourceRow, (nuint)(sourceStride + column)); + + TOperator.Filter(topLeft, bottomLeft).StoreUnsafe(ref destinationRow, (nuint)column); + } + } + + processedColumns = endColumn; + } + + if (Vector128.IsHardwareAccelerated) + { + int remainingColumns = width - processedColumns; + int vectorizedColumns = (int)(Numerics.Vector128Count(remainingColumns) * (nuint)Vector128.Count); + int endColumn = processedColumns + vectorizedColumns; + + for (int row = 0; row < height; row++) + { + ref byte sourceRow = ref Unsafe.Add(ref sourceBase, row * sourceStride); + ref byte destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + + for (int column = processedColumns; column < endColumn; column += Vector128.Count) + { + Vector128 topLeft = Vector128.LoadUnsafe(ref sourceRow, (nuint)column); + Vector128 bottomLeft = Vector128.LoadUnsafe(ref sourceRow, (nuint)(sourceStride + column)); + + TOperator.Filter(topLeft, bottomLeft).StoreUnsafe(ref destinationRow, (nuint)column); + } + } + + processedColumns = endColumn; + } + + // FeatureTestRunner can disable every intrinsic tier. Keeping the scalar continuation in the same traversal + // proves the fallback without changing source addressing or the normative rounding performed by the operator. + for (int row = 0; row < height; row++) + { + ref byte sourceRow = ref Unsafe.Add(ref sourceBase, row * sourceStride); + ref byte destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + + for (int column = processedColumns; column < width; column++) + { + byte topLeft = Unsafe.Add(ref sourceRow, column); + byte bottomLeft = Unsafe.Add(ref sourceRow, sourceStride + column); + + Unsafe.Add(ref destinationRow, column) = TOperator.Filter(topLeft, bottomLeft); + } + } + } + + /// + /// Applies one closed interpolation operator to a high-bit-depth source block. + /// + /// The source-phase-specific interpolation arithmetic. + private static void Predict( + ReadOnlySpan source, + int sourceStride, + Span destination, + int destinationStride, + int width, + int height) + where TOperator : struct, IAv1IntraBlockCopyVerticalOperator + { + ref short sourceBase = ref MemoryMarshal.GetReference(source); + ref short destinationBase = ref MemoryMarshal.GetReference(destination); + + if (Vector128.IsHardwareAccelerated && width == 4) + { + // Four high-bit-depth samples occupy the lower half of a Vector128. The frame allocation's prediction + // border makes the full source load readable; storing only the lower four lanes avoids relying on writable + // samples beyond the transform boundary. + for (int row = 0; row < height; row++) + { + ref short sourceRow = ref Unsafe.Add(ref sourceBase, row * sourceStride); + ref short destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + Vector128 topLeft = Vector128.LoadUnsafe(ref sourceRow); + Vector128 bottomLeft = Vector128.LoadUnsafe(ref sourceRow, (nuint)sourceStride); + + TOperator.Filter(topLeft, bottomLeft).GetLower().StoreUnsafe(ref destinationRow); + } + + return; + } + + int processedColumns = 0; + + // High-bit-depth lanes hold half as many samples, but retain the same descending-width traversal and one scalar + // continuation as the byte path. + if (Vector512.IsHardwareAccelerated) + { + int vectorizedColumns = (int)(Numerics.Vector512Count(width) * (nuint)Vector512.Count); + if (vectorizedColumns > 0) + { + for (int row = 0; row < height; row++) + { + ref short sourceRow = ref Unsafe.Add(ref sourceBase, row * sourceStride); + ref short destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + + for (int column = 0; column < vectorizedColumns; column += Vector512.Count) + { + Vector512 topLeft = Vector512.LoadUnsafe(ref sourceRow, (nuint)column); + Vector512 bottomLeft = Vector512.LoadUnsafe(ref sourceRow, (nuint)(sourceStride + column)); + + TOperator.Filter(topLeft, bottomLeft).StoreUnsafe(ref destinationRow, (nuint)column); + } + } + + processedColumns = vectorizedColumns; + } + } + + if (Vector256.IsHardwareAccelerated) + { + int remainingColumns = width - processedColumns; + int vectorizedColumns = (int)(Numerics.Vector256Count(remainingColumns) * (nuint)Vector256.Count); + int endColumn = processedColumns + vectorizedColumns; + + for (int row = 0; row < height; row++) + { + ref short sourceRow = ref Unsafe.Add(ref sourceBase, row * sourceStride); + ref short destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + + for (int column = processedColumns; column < endColumn; column += Vector256.Count) + { + Vector256 topLeft = Vector256.LoadUnsafe(ref sourceRow, (nuint)column); + Vector256 bottomLeft = Vector256.LoadUnsafe(ref sourceRow, (nuint)(sourceStride + column)); + + TOperator.Filter(topLeft, bottomLeft).StoreUnsafe(ref destinationRow, (nuint)column); + } + } + + processedColumns = endColumn; + } + + if (Vector128.IsHardwareAccelerated) + { + int remainingColumns = width - processedColumns; + int vectorizedColumns = (int)(Numerics.Vector128Count(remainingColumns) * (nuint)Vector128.Count); + int endColumn = processedColumns + vectorizedColumns; + + for (int row = 0; row < height; row++) + { + ref short sourceRow = ref Unsafe.Add(ref sourceBase, row * sourceStride); + ref short destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + + for (int column = processedColumns; column < endColumn; column += Vector128.Count) + { + Vector128 topLeft = Vector128.LoadUnsafe(ref sourceRow, (nuint)column); + Vector128 bottomLeft = Vector128.LoadUnsafe(ref sourceRow, (nuint)(sourceStride + column)); + + TOperator.Filter(topLeft, bottomLeft).StoreUnsafe(ref destinationRow, (nuint)column); + } + } + + processedColumns = endColumn; + } + + for (int row = 0; row < height; row++) + { + ref short sourceRow = ref Unsafe.Add(ref sourceBase, row * sourceStride); + ref short destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + + for (int column = processedColumns; column < width; column++) + { + short topLeft = Unsafe.Add(ref sourceRow, column); + short bottomLeft = Unsafe.Add(ref sourceRow, sourceStride + column); + + Unsafe.Add(ref destinationRow, column) = TOperator.Filter(topLeft, bottomLeft); + } + } + } + + /// + /// Applies one closed interpolation operator to an 8-bit source block without explicit hardware intrinsics. + /// + /// The source-phase-specific interpolation arithmetic. + private static void PredictScalar( + ReadOnlySpan source, + int sourceStride, + Span destination, + int destinationStride, + int width, + int height) + where TOperator : struct, IAv1IntraBlockCopyVerticalOperator + { + for (int row = 0; row < height; row++) + { + int sourceRow = row * sourceStride; + int destinationRow = row * destinationStride; + + for (int column = 0; column < width; column++) + { + byte topLeft = source[sourceRow + column]; + byte bottomLeft = source[sourceRow + sourceStride + column]; + destination[destinationRow + column] = TOperator.Filter(topLeft, bottomLeft); + } + } + } + + /// + /// Applies one closed interpolation operator to a high-bit-depth source block without explicit hardware intrinsics. + /// + /// The source-phase-specific interpolation arithmetic. + private static void PredictScalar( + ReadOnlySpan source, + int sourceStride, + Span destination, + int destinationStride, + int width, + int height) + where TOperator : struct, IAv1IntraBlockCopyVerticalOperator + { + for (int row = 0; row < height; row++) + { + int sourceRow = row * sourceStride; + int destinationRow = row * destinationStride; + + for (int column = 0; column < width; column++) + { + short topLeft = source[sourceRow + column]; + short bottomLeft = source[sourceRow + sourceStride + column]; + destination[destinationRow + column] = TOperator.Filter(topLeft, bottomLeft); + } + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Readme.md b/src/ImageSharp/Formats/Heif/Av1/Readme.md new file mode 100644 index 0000000000..cc18ba1e01 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Readme.md @@ -0,0 +1,71 @@ +# Open Bitstream Unit + +An OBU is a unit of syntax encoded in an AV1 bitstream. HEIF image items can contain still-picture, +progressive, or dependent-frame AV1 payloads, so the decoder handles both intra and inter frames. + +An OBU section for AVIF consists of the following headers: + +## Temporal delimiter + +In AV1 sequences this marks a temporal-unit boundary. Many encoders write one at the start of the payload. + +## Sequence header + +This is the common header for a sequence of frames. A still picture can use the reduced syntax selected by +`ReducedStillPictureHeader`; progressive and dependent-frame payloads use the complete sequence syntax. + +## Frame header + +Frame-header, redundant-frame-header, and combined-frame OBUs define the syntax of a coded frame. + +## Tile group + +Defines the tile range and contains the entropy-coded payload for each tile in that range. + +# Tiling + +In AV1 a frame is made up of 1 or more tiles. The parameters for each tile are entropy encoded using the context aware symbol coding. +These parameters are contained in an OBU tile group header. + +## Superblock + +A tile consists of one or more superblocks. Superblocks can be either 64x64 or 128x128 pixels in size. +This choice is made per frame, and is specified in the `ObuFrameHeader`. +A superblock contains one or more partitions that subdivide the area. + +## Partition + +A superblock contains one or more partitions. The partition type determines how the area is split. +Partitions can contain other partitions and blocks. + +## Block + +## Transform Block + +A transform block is the smallest image area that shares transform parameters. A block contains one or more mode-information units. + +## ModeInfo + +The smallest unit in the frame. It determines the parameters for an area of 4 by 4 pixels. + +# References + +[AV1 embedded in HEIF](https://aomediacodec.github.io/av1-isobmff) + +[AV1 specification](https://aomediacodec.github.io/av1-spec/av1-spec.pdf) + +[AVIF specification](https://aomediacodec.github.io/av1-avif) + +[Official AV1 reference implementation](https://aomedia.googlesource.com/aom/) + +[SVT-AV1 encoder](https://gitlab.com/AOMediaCodec/SVT-AV1) + +[libavif AVIF container implementation](https://github.com/AOMediaCodec/libavif) + +[Paper describing the techniques used in AV1](https://arxiv.org/pdf/2008.06091) + +# Test images + +[Netflix image repository](http://download.opencontent.netflix.com/?prefix=AV1/) + +[AVIF sample images](https://github.com/link-u/avif-sample-images) diff --git a/src/ImageSharp/Formats/Heif/Av1/ReferenceFrames/Av1ReferenceFrame.cs b/src/ImageSharp/Formats/Heif/Av1/ReferenceFrames/Av1ReferenceFrame.cs new file mode 100644 index 0000000000..b90457ea22 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/ReferenceFrames/Av1ReferenceFrame.cs @@ -0,0 +1,220 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1.Entropy; +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.ReferenceFrames; + +/// +/// Owns the completed decoded state retained for one AV1 reference or presentation frame. +/// +/// +/// Reference-map owners contain reconstruction samples after the normative in-loop filters and before film-grain +/// synthesis. A presentation-only owner may instead contain the independently synthesized grained output. +/// +internal sealed class Av1ReferenceFrame : IDisposable +{ + /// + /// The completed sample planes while this instance owns them. + /// + private Av1FrameBuffer? frameBuffer; + + /// + /// The compact reference state and optional entropy snapshot retained while this frame occupies the reference map. + /// + private ReferenceOwnership? referenceOwnership; + + /// + /// Initializes a new instance of the class for presentation-only ownership. + /// + /// + /// The completed sample buffer. Ownership transfers to this instance when construction succeeds. + /// + /// + /// The completed frame header associated with the reconstructed samples. The caller must not mutate the header + /// after transferring it to this instance. + /// + public Av1ReferenceFrame(Av1FrameBuffer frameBuffer, ObuFrameHeader frameHeader) + { + this.frameBuffer = frameBuffer; + this.FrameHeader = frameHeader; + } + + /// + /// Initializes a new instance of the class with retained compact reference state. + /// + /// + /// The completed sample buffer. Ownership transfers to this instance when construction succeeds. + /// + /// + /// The completed frame header associated with the reconstructed samples. The caller must not mutate the header + /// after transferring it to this instance. + /// + /// The completed reconstruction state from which reference syntax is retained. + public Av1ReferenceFrame(Av1FrameBuffer frameBuffer, ObuFrameHeader frameHeader, Av1FrameInfo frameInfo) + : this(frameBuffer, frameHeader) + { + this.referenceOwnership = new(frameInfo.AcquireReferenceState(), null); + } + + /// + /// Initializes a new instance of the class and takes ownership of decoded samples + /// and the entropy snapshot retained by a refreshed reference frame. + /// + /// + /// The completed sample buffer. Ownership transfers to this instance when construction succeeds. + /// + /// + /// The completed frame header associated with the reconstructed samples. The caller must not mutate the header + /// after transferring it to this instance. + /// + /// + /// The completed per-block state associated with the reconstructed samples. The caller must not mutate the state + /// after transferring it to this instance. + /// + /// The completed entropy snapshot selected for later primary-reference use. + /// The decoder-session owner to which the snapshot is returned. + public Av1ReferenceFrame( + Av1FrameBuffer frameBuffer, + ObuFrameHeader frameHeader, + Av1FrameInfo frameInfo, + Av1FrameEntropyContext entropyContext, + Av1FrameEntropyContexts entropyContextOwner) + : this(frameBuffer, frameHeader) + { + this.referenceOwnership = new( + frameInfo.AcquireReferenceState(), + new EntropyOwnership(entropyContext, entropyContextOwner)); + } + + /// + /// Gets the completed sample buffer owned by this frame. + /// + public Av1FrameBuffer FrameBuffer + { + get + { + return this.frameBuffer + ?? throw new ObjectDisposedException(nameof(Av1ReferenceFrame)); + } + } + + /// + /// Gets the completed header that describes the retained frame. + /// + public ObuFrameHeader FrameHeader { get; } + + /// + /// Gets the compact segment and motion state associated with the retained frame. + /// + public Av1FrameInfo.ReferenceState ReferenceState + { + get + { + ReferenceOwnership? ownership = this.referenceOwnership; + if (ownership is null) + { + throw new InvalidOperationException("A presentation-only AV1 frame has no retained reference state."); + } + + return ownership.Value.ReferenceState; + } + } + + /// + /// Gets the entropy context retained for primary-reference use, or for a presentation-only + /// frame. + /// + public Av1FrameEntropyContext? EntropyContext => this.referenceOwnership?.Entropy?.Context; + + /// + /// Restores the retained frame context to the normative defaults selected by this frame's quantizer band. + /// + public void ResetEntropyContext() + { + EntropyOwnership? entropy = this.referenceOwnership?.Entropy; + if (entropy is null) + { + throw new InvalidOperationException("The AV1 reference frame has no retained entropy context."); + } + + entropy.Value.Context.ResetToDefaults(this.FrameHeader.QuantizationParameters.BaseQIndex); + } + + /// + /// Transfers the completed sample planes out of this frame owner. + /// + /// The completed sample planes now owned by the caller. + public Av1FrameBuffer TakeFrameBuffer() + { + Av1FrameBuffer result = this.frameBuffer + ?? throw new ObjectDisposedException(nameof(Av1ReferenceFrame)); + + this.frameBuffer = null; + return result; + } + + /// + /// Releases the owned completed sample planes and returns any retained entropy snapshot to its decoder session. + /// + public void Dispose() + { + ReferenceOwnership? ownership = this.referenceOwnership; + this.referenceOwnership = null; + if (ownership is not null) + { + // Clearing the complete ownership state before returning either resource makes repeated disposal harmless + // when one frame owner occupies multiple reference-map slots. + ReferenceOwnership activeOwnership = ownership.Value; + EntropyOwnership? entropy = activeOwnership.Entropy; + if (entropy is not null) + { + EntropyOwnership activeEntropy = entropy.Value; + activeEntropy.Owner.ReturnSnapshot(activeEntropy.Context); + } + + activeOwnership.ReferenceState.ReleaseOwner(); + } + + this.frameBuffer?.Dispose(); + this.frameBuffer = null; + } + + /// + /// Carries the complete state retained only by frames that can be selected as references. + /// + private readonly struct ReferenceOwnership( + Av1FrameInfo.ReferenceState referenceState, + EntropyOwnership? entropy) + { + /// + /// Gets the retained segment and motion state. + /// + public Av1FrameInfo.ReferenceState ReferenceState { get; } = referenceState; + + /// + /// Gets the retained entropy snapshot and its return owner when one was published. + /// + public EntropyOwnership? Entropy { get; } = entropy; + } + + /// + /// Pairs a retained entropy snapshot with the decoder-session owner that must receive it on release. + /// + private readonly struct EntropyOwnership( + Av1FrameEntropyContext context, + Av1FrameEntropyContexts owner) + { + /// + /// Gets the retained entropy snapshot. + /// + public Av1FrameEntropyContext Context { get; } = context; + + /// + /// Gets the decoder-session owner that receives the snapshot. + /// + public Av1FrameEntropyContexts Owner { get; } = owner; + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/ReferenceFrames/Av1ReferenceFrameBorder.cs b/src/ImageSharp/Formats/Heif/Av1/ReferenceFrames/Av1ReferenceFrameBorder.cs new file mode 100644 index 0000000000..c8cbf5712c --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/ReferenceFrames/Av1ReferenceFrameBorder.cs @@ -0,0 +1,111 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.InteropServices; +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Memory; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.ReferenceFrames; + +/// +/// Extends reconstructed AV1 edge samples through the padded reference-frame border. +/// +internal static class Av1ReferenceFrameBorder +{ + /// + /// Replicates every visible plane edge through its complete decoder padding. + /// + /// The post-restoration reference frame whose padding is extended. + public static void Extend(Av1FrameBuffer frameBuffer) + { + ObuColorConfig colorConfig = frameBuffer.ColorConfig; + int subsamplingX = !colorConfig.IsMonochrome && colorConfig.SubSamplingX ? 1 : 0; + int subsamplingY = !colorConfig.IsMonochrome && colorConfig.SubSamplingY ? 1 : 0; + + ExtendPlane( + frameBuffer, + frameBuffer.GetPlaneBuffer(Av1Plane.Y), + frameBuffer.OriginX, + frameBuffer.OriginY, + frameBuffer.Width, + frameBuffer.Height); + + if (!colorConfig.IsMonochrome) + { + int chromaWidth = Av1Math.DivideLog2Ceiling(frameBuffer.Width, subsamplingX); + int chromaHeight = Av1Math.DivideLog2Ceiling(frameBuffer.Height, subsamplingY); + int chromaOriginX = frameBuffer.OriginX >> subsamplingX; + int chromaOriginY = frameBuffer.OriginY >> subsamplingY; + + ExtendPlane(frameBuffer, frameBuffer.GetPlaneBuffer(Av1Plane.U), chromaOriginX, chromaOriginY, chromaWidth, chromaHeight); + ExtendPlane(frameBuffer, frameBuffer.GetPlaneBuffer(Av1Plane.V), chromaOriginX, chromaOriginY, chromaWidth, chromaHeight); + } + } + + /// + /// Selects the native sample representation for one byte-backed plane. + /// + /// The frame that defines the native sample size. + /// The padded plane allocation. + /// The horizontal visible origin in plane samples. + /// The vertical visible origin in rows. + /// The visible plane width. + /// The visible plane height. + private static void ExtendPlane( + Av1FrameBuffer frameBuffer, + Buffer2D buffer, + int originX, + int originY, + int width, + int height) + { + if (frameBuffer.BytesPerSample == 2) + { + ExtendPlane(MemoryMarshal.Cast(buffer.DangerousGetSingleSpan()), buffer.Width >> 1, originX, originY, width, height); + } + else + { + ExtendPlane(buffer.DangerousGetSingleSpan(), buffer.Width, originX, originY, width, height); + } + } + + /// + /// Extends one native sample plane horizontally and then vertically. + /// + /// The native eight-bit or high-bit-depth sample type. + /// The complete padded plane allocation. + /// The number of native samples between adjacent rows. + /// The horizontal visible origin in plane samples. + /// The vertical visible origin in rows. + /// The visible plane width. + /// The visible plane height. + private static void ExtendPlane(Span plane, int stride, int originX, int originY, int width, int height) + where TSample : unmanaged + { + int rightStart = originX + width; + int rightLength = stride - rightStart; + + for (int row = 0; row < height; row++) + { + Span destinationRow = plane.Slice((originY + row) * stride, stride); + + // Span.Fill maps these long constant runs to the runtime's vectorized fill implementation. Extending the + // horizontal edges first also makes each later full-row copy include complete left and right padding. + destinationRow[..originX].Fill(destinationRow[originX]); + destinationRow.Slice(rightStart, rightLength).Fill(destinationRow[rightStart - 1]); + } + + ReadOnlySpan firstVisibleRow = plane.Slice(originY * stride, stride); + for (int row = 0; row < originY; row++) + { + firstVisibleRow.CopyTo(plane.Slice(row * stride, stride)); + } + + int bottomStart = originY + height; + ReadOnlySpan lastVisibleRow = plane.Slice((bottomStart - 1) * stride, stride); + for (int row = bottomStart; row < plane.Length / stride; row++) + { + lastVisibleRow.CopyTo(plane.Slice(row * stride, stride)); + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/ReferenceFrames/Av1ReferenceFrameDerivation.cs b/src/ImageSharp/Formats/Heif/Av1/ReferenceFrames/Av1ReferenceFrameDerivation.cs new file mode 100644 index 0000000000..48c73ebf61 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/ReferenceFrames/Av1ReferenceFrameDerivation.cs @@ -0,0 +1,279 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.ReferenceFrames; + +/// +/// Derives the seven AV1 inter-reference map indices from short reference signaling. +/// +internal static class Av1ReferenceFrameDerivation +{ + /// + /// The shifted-order sentinel used for a reference-map slot that is not available to the current frame. + /// + private const int UnavailableSortIndex = -1; + + /// + /// The reference-type value subtracted when indexing the seven-entry inter-reference map. + /// + private const int ReferenceIndexOffset = (int)Av1ReferenceFrameType.Last; + + /// + /// Gets the order in which unassigned backward roles are replaced by forward references. + /// + private static ReadOnlySpan RemainingReferenceOrder => + [ + Av1ReferenceFrameType.Last2, + Av1ReferenceFrameType.Last3, + Av1ReferenceFrameType.Backward, + Av1ReferenceFrameType.Alternate2, + Av1ReferenceFrameType.Alternate, + ]; + + /// + /// Derives the reference-map slot selected for each inter-reference type when an AV1 frame uses short reference + /// signaling. + /// + /// The current frame order hint in the active modulo order-hint domain. + /// The number of bits in the active order-hint domain. + /// The explicitly signaled reference-map slot for . + /// The explicitly signaled reference-map slot for . + /// The eight persisted reference-map order hints. + /// + /// The eight values indicating whether each persisted reference-map slot owns a decoded frame. An empty slot is + /// excluded from derivation. + /// + /// + /// The destination for seven slot indices ordered from through + /// . + /// + /// + /// The signaled LAST or GOLDEN slot is empty, refers to the current frame, or refers to a future frame. + /// + /// + /// The caller owns the fixed AV1 table-size invariants: and + /// contain eight entries, while contains + /// seven entries. The signaled indices and order hints have already been read from their bounded bit fields. Every + /// destination entry is overwritten on success, and multiple reference types may select the same slot. Frame-ID + /// validity is a separate conformance state that the caller checks for every resolved reference after derivation. + /// + public static void DeriveShortSignaledReferences( + uint currentOrderHint, + int orderHintBitWidth, + uint lastFrameIndex, + uint goldenFrameIndex, + ReadOnlySpan slotOrderHints, + ReadOnlySpan slotOccupancy, + Span referenceFrameIndices) + { + int lastMapIndex = (int)lastFrameIndex; + int goldenMapIndex = (int)goldenFrameIndex; + + if (!slotOccupancy[lastMapIndex]) + { + // Unlike an unused empty slot, the explicitly signaled LAST slot must own a decoded frame before any + // derived mapping can be consumed. the reference decoder rejects the missing reference at this frame-header boundary. + throw new InvalidImageContentException("An AV1 inter frame requests an unavailable LAST reference."); + } + + if (!slotOccupancy[goldenMapIndex]) + { + // GOLDEN is the other explicitly signaled slot and has the same ownership requirement as LAST. + throw new InvalidImageContentException("An AV1 inter frame requests an unavailable GOLDEN reference."); + } + + int currentFrameSortIndex = 1 << (orderHintBitWidth - 1); + int orderHintMask = currentFrameSortIndex - 1; + InlineArray8 referenceInfo = default; + int lastFrameSortIndex = UnavailableSortIndex; + int goldenFrameSortIndex = UnavailableSortIndex; + + for (int mapIndex = 0; mapIndex < Av1Constants.ReferenceFrameCount; mapIndex++) + { + ref ReferenceFrameInfo info = ref referenceInfo[mapIndex]; + info.MapIndex = mapIndex; + info.SortIndex = UnavailableSortIndex; + + if (!slotOccupancy[mapIndex]) + { + // the reference decoder gives absent reference buffers sort index -1. Keeping empty managed slots in the same + // leading partition prevents their stale order hints from participating in temporal selection. + continue; + } + + int difference = (int)slotOrderHints[mapIndex] - (int)currentOrderHint; + + // get_relative_dist folds the unsigned order-hint difference into the signed half-open interval + // [-2^(bits-1), 2^(bits-1)). Adding the half-range makes -1 available as the absence sentinel while valid + // entries sort from zero through the complete modulo domain. + difference = (difference & orderHintMask) - (difference & currentFrameSortIndex); + info.SortIndex = currentFrameSortIndex + difference; + + if (mapIndex == lastMapIndex) + { + lastFrameSortIndex = info.SortIndex; + } + + if (mapIndex == goldenMapIndex) + { + goldenFrameSortIndex = info.SortIndex; + } + } + + if (lastFrameSortIndex >= currentFrameSortIndex) + { + throw new InvalidImageContentException("An AV1 inter frame requests a current or future frame as LAST."); + } + + if (goldenFrameSortIndex >= currentFrameSortIndex) + { + throw new InvalidImageContentException("An AV1 inter frame requests a current or future frame as GOLDEN."); + } + + // the reference decoder sorts first by shifted output order and then by reference-map index. The explicit tie break is + // normative: equal order hints select the highest map index for latest references and the lowest for earliest + // references. Insertion sort is bounded to eight inline entries and does not allocate or require general sort + // infrastructure at the frame-header boundary. + for (int index = 1; index < Av1Constants.ReferenceFrameCount; index++) + { + ReferenceFrameInfo current = referenceInfo[index]; + int insertionIndex = index; + + while (insertionIndex > 0) + { + ReferenceFrameInfo previous = referenceInfo[insertionIndex - 1]; + if (previous.SortIndex < current.SortIndex || + (previous.SortIndex == current.SortIndex && previous.MapIndex <= current.MapIndex)) + { + break; + } + + referenceInfo[insertionIndex] = previous; + insertionIndex--; + } + + referenceInfo[insertionIndex] = current; + } + + InlineArray8 assignedReferences = default; + int lastReferenceIndex = (int)Av1ReferenceFrameType.Last - ReferenceIndexOffset; + int goldenReferenceIndex = (int)Av1ReferenceFrameType.Golden - ReferenceIndexOffset; + referenceFrameIndices[lastReferenceIndex] = lastFrameIndex; + referenceFrameIndices[goldenReferenceIndex] = goldenFrameIndex; + assignedReferences[lastReferenceIndex] = true; + assignedReferences[goldenReferenceIndex] = true; + + int forwardStartIndex = 0; + int forwardEndIndex = Av1Constants.ReferenceFrameCount - 1; + + // Empty entries sort before every occupied shifted hint. The first current-or-future entry then divides the + // remaining sorted table into forward references on the left and backward references on the right. + for (int index = 0; index < Av1Constants.ReferenceFrameCount; index++) + { + if (referenceInfo[index].SortIndex == UnavailableSortIndex) + { + forwardStartIndex++; + continue; + } + + if (referenceInfo[index].SortIndex >= currentFrameSortIndex) + { + forwardEndIndex = index - 1; + break; + } + } + + int backwardStartIndex = forwardEndIndex + 1; + int backwardEndIndex = Av1Constants.ReferenceFrameCount - 1; + int alternateReferenceIndex = (int)Av1ReferenceFrameType.Alternate - ReferenceIndexOffset; + int backwardReferenceIndex = (int)Av1ReferenceFrameType.Backward - ReferenceIndexOffset; + int alternate2ReferenceIndex = (int)Av1ReferenceFrameType.Alternate2 - ReferenceIndexOffset; + + if (backwardStartIndex <= backwardEndIndex) + { + // ALTREF receives the frame farthest into the future. The sorted-map-index tie break selects the highest + // slot when multiple frames share that order hint, matching both the specification and the reference decoder. + referenceFrameIndices[alternateReferenceIndex] = (uint)referenceInfo[backwardEndIndex].MapIndex; + assignedReferences[alternateReferenceIndex] = true; + backwardEndIndex--; + } + + if (backwardStartIndex <= backwardEndIndex) + { + // BWDREF receives the nearest future frame and therefore consumes the low end of the backward partition. + referenceFrameIndices[backwardReferenceIndex] = (uint)referenceInfo[backwardStartIndex].MapIndex; + assignedReferences[backwardReferenceIndex] = true; + backwardStartIndex++; + } + + if (backwardStartIndex <= backwardEndIndex) + { + // ALTREF2 receives the next-nearest remaining future frame. No further backward lookup follows, so the + // lower boundary does not need to advance after this assignment. + referenceFrameIndices[alternate2ReferenceIndex] = (uint)referenceInfo[backwardStartIndex].MapIndex; + assignedReferences[alternate2ReferenceIndex] = true; + } + + ReadOnlySpan remainingReferenceOrder = RemainingReferenceOrder; + int remainingIndex; + + for (remainingIndex = 0; remainingIndex < remainingReferenceOrder.Length; remainingIndex++) + { + int referenceIndex = (int)remainingReferenceOrder[remainingIndex] - ReferenceIndexOffset; + if (assignedReferences[referenceIndex]) + { + continue; + } + + // LAST and GOLDEN were already assigned explicitly and cannot be reused while an unassigned forward slot + // remains. Moving from the high end chooses the remaining frames in anti-chronological order. + while (forwardStartIndex <= forwardEndIndex && + (referenceInfo[forwardEndIndex].MapIndex == lastMapIndex || + referenceInfo[forwardEndIndex].MapIndex == goldenMapIndex)) + { + forwardEndIndex--; + } + + if (forwardStartIndex > forwardEndIndex) + { + break; + } + + referenceFrameIndices[referenceIndex] = (uint)referenceInfo[forwardEndIndex].MapIndex; + assignedReferences[referenceIndex] = true; + forwardEndIndex--; + } + + for (; remainingIndex < remainingReferenceOrder.Length; remainingIndex++) + { + int referenceIndex = (int)remainingReferenceOrder[remainingIndex] - ReferenceIndexOffset; + if (assignedReferences[referenceIndex]) + { + continue; + } + + // AV1 requires every unfilled role to reuse the earliest available forward frame. At least LAST and GOLDEN + // are occupied forward references, so forwardStartIndex always identifies a usable slot at this point. + referenceFrameIndices[referenceIndex] = (uint)referenceInfo[forwardStartIndex].MapIndex; + assignedReferences[referenceIndex] = true; + } + } + + /// + /// Stores one reference-map slot and its shifted order for fixed-size sorting. + /// + private struct ReferenceFrameInfo + { + /// + /// Gets or sets the zero-based slot in the eight-entry persisted reference map. + /// + public int MapIndex { get; set; } + + /// + /// Gets or sets the order hint shifted around the current frame, or when unavailable. + /// + public int SortIndex { get; set; } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/ReferenceFrames/Av1ReferenceFrameStore.cs b/src/ImageSharp/Formats/Heif/Av1/ReferenceFrames/Av1ReferenceFrameStore.cs new file mode 100644 index 0000000000..82192dbadc --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/ReferenceFrames/Av1ReferenceFrameStore.cs @@ -0,0 +1,342 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.ReferenceFrames; + +/// +/// Owns the reference map and selected presentation output for one bounded AV1 decoder session. +/// +/// +/// Several slots and the selected output may identify the same . The store preserves +/// that sharing without allocating reference-count objects and releases a frame only after its final owning reference +/// has been replaced or cleared. This type is not thread safe; one decoder session serializes commit and disposal. +/// +internal sealed class Av1ReferenceFrameStore : IDisposable +{ + /// + /// The number of reference slots defined by the AV1 uncompressed frame header. + /// + private const int SlotCount = Av1Constants.ReferenceFrameCount; + + /// + /// Stores the frame owner selected by each reference-map slot without allocating a managed array. + /// + private InlineArray8 frames; + + /// + /// The most recent shown frame retained for presentation at the end of the bounded image payload. + /// + private Av1ReferenceFrame? outputFrame; + + /// + /// Gets the most recent shown frame retained for presentation. + /// + public Av1ReferenceFrame? OutputFrame => this.outputFrame; + + /// + /// Resolves one reference-map slot. + /// + /// The zero-based reference-map slot in the inclusive range 0 through 7. + /// The retained frame, or when the slot has not been populated. + public Av1ReferenceFrame? Resolve(int slot) => this.frames[slot]; + + /// + /// Resolves a reference-map slot that an earlier syntax boundary has established as occupied. + /// + /// The zero-based reference-map slot. + /// The retained frame in the selected slot. + public Av1ReferenceFrame ResolveRequired(int slot) + { + return this.Resolve(slot) + ?? throw new InvalidImageContentException($"The AV1 reference-map slot {slot} has not been populated."); + } + + /// + /// Resolves the presentation output established by the completed bounded payload. + /// + /// The retained frame selected for presentation. + public Av1ReferenceFrame ResolveOutput() + { + return this.outputFrame + ?? throw new InvalidImageContentException("The AV1 payload did not produce a shown frame."); + } + + /// + /// Writes whether each reference-map slot currently owns a reconstructed frame. + /// + /// The eight-entry destination receiving the current slot occupancy. + public void FillOccupancy(Span destination) + { + // Physical ownership is intentionally independent from frame-ID validity. Short reference signaling sorts + // every occupied slot first, then the uncompressed-header parser validates each derived role separately. + for (int slot = 0; slot < SlotCount; slot++) + { + destination[slot] = this.frames[slot] is not null; + } + } + + /// + /// Commits a completed frame to the reference map and, when shown, retains it for presentation. + /// + /// + /// The mask whose bit n replaces reference-map slot n. Only the low eight bits describe AV1 slots. + /// + /// The completed frame to retain in every selected ownership role. + /// Whether the completed frame replaces the previously retained presentation output. + /// + /// when the frame is retained as a reference or presentation output and ownership transfers + /// to this store; otherwise , in which case no state changes and the caller retains ownership. + /// + /// + /// The caller must invoke this method only after reconstruction and all normative in-loop filters have completed. + /// Once ownership transfers, the caller must not dispose the frame. A frame passed here must not already be owned by + /// this store. + /// + public bool Commit(uint refreshFrameFlags, Av1ReferenceFrame frame, bool showFrame) + { + refreshFrameFlags &= byte.MaxValue; + if (refreshFrameFlags == 0 && !showFrame) + { + // A hidden frame with a zero refresh mask has no remaining role in an image-decoder session. + return false; + } + + InlineArray8 replacedFrames = default; + Av1ReferenceFrame? replacedOutputFrame = showFrame ? this.outputFrame : null; + + // Capture displaced owners in inline storage, then publish the complete slot and output transition before + // releasing anything. A shown frame may also occupy reference slots, so both ownership domains must change as + // one operation. + for (int slot = 0; slot < SlotCount; slot++) + { + if ((refreshFrameFlags & (1U << slot)) != 0) + { + replacedFrames[slot] = this.frames[slot]; + this.frames[slot] = frame; + } + } + + if (showFrame) + { + this.outputFrame = frame; + } + + for (int replacedIndex = 0; replacedIndex < SlotCount; replacedIndex++) + { + Av1ReferenceFrame? replacedFrame = replacedFrames[replacedIndex]; + + if (replacedFrame is null) + { + continue; + } + + if (ReferenceEquals(replacedFrame, replacedOutputFrame)) + { + // Let the displaced-output path release this shared owner after every slot candidate has been removed. + replacedFrames[replacedIndex] = null; + continue; + } + + // A displaced frame remains owned when any unrefreshed slot or the selected output still references it. + // Eight fixed slots make the bounded identity scan cheaper than allocated reference-count state. + if (this.IsRetained(replacedFrame)) + { + replacedFrames[replacedIndex] = null; + } + } + + DisposeUnique(ref replacedFrames); + + if (replacedOutputFrame is not null && !this.IsRetained(replacedOutputFrame)) + { + replacedOutputFrame.Dispose(); + } + + return true; + } + + /// + /// Replaces the selected presentation output with an independently owned completed frame. + /// + /// The completed presentation frame whose ownership transfers to this store. + /// + /// This path is used when film grain requires presentation samples to differ from the ungrained reconstruction + /// retained by the reference map. + /// + public void CommitOutput(Av1ReferenceFrame frame) + { + Av1ReferenceFrame? replacedFrame = this.outputFrame; + this.outputFrame = frame; + + // The previous output may still be retained by one or more reference slots. Release it only after publishing + // the new output and confirming that no reference-map identity remains. + if (replacedFrame is not null && !this.IsRetained(replacedFrame)) + { + replacedFrame.Dispose(); + } + } + + /// + /// Selects one retained reference for presentation and applies the key-frame reference-map reset when required. + /// + /// The zero-based reference-map slot selected by the frame header. + /// The retained frame selected for presentation. + public Av1ReferenceFrame ShowExisting(int slot) + { + Av1ReferenceFrame selectedFrame = this.ResolveRequired(slot); + Av1ReferenceFrame? replacedOutputFrame = this.outputFrame; + this.outputFrame = selectedFrame; + + if (selectedFrame.FrameHeader.FrameType == ObuFrameType.KeyFrame) + { + InlineArray8 replacedFrames = this.frames; + + // Showing a hidden key frame starts a new coded-video-sequence state. All eight reference-map slots now + // identify that same reconstructed owner, so publish every alias before releasing displaced frames. + for (int mapSlot = 0; mapSlot < SlotCount; mapSlot++) + { + this.frames[mapSlot] = selectedFrame; + if (ReferenceEquals(replacedFrames[mapSlot], selectedFrame)) + { + replacedFrames[mapSlot] = null; + } + } + + // A key frame may be presented through show_existing_frame only once. The retained owner carries this + // conformance state because every slot alias must observe the transition. + selectedFrame.FrameHeader.ShowableFrame = false; + + if (replacedOutputFrame is not null) + { + // Let the displaced-output path release a detached shared owner after all of its old slot aliases have + // been removed from the replacement set. + for (int mapSlot = 0; mapSlot < SlotCount; mapSlot++) + { + if (ReferenceEquals(replacedFrames[mapSlot], replacedOutputFrame)) + { + replacedFrames[mapSlot] = null; + } + } + } + + DisposeUnique(ref replacedFrames); + } + + if (replacedOutputFrame is not null && !this.IsRetained(replacedOutputFrame)) + { + replacedOutputFrame.Dispose(); + } + + return selectedFrame; + } + + /// + /// Transfers the selected presentation frame out of this store and releases every other retained frame. + /// + /// The selected presentation frame now owned by the caller. + public Av1ReferenceFrame TakeOutput() + { + Av1ReferenceFrame result = this.ResolveOutput(); + this.outputFrame = null; + + // The caller becomes the sole owner of the selected output. Remove all slot aliases before Reset releases the + // remaining session references so the sample buffer can transfer without copying. + for (int slot = 0; slot < SlotCount; slot++) + { + if (ReferenceEquals(this.frames[slot], result)) + { + this.frames[slot] = null; + } + } + + this.Reset(); + return result; + } + + /// + /// Clears all reference-map slots and releases every uniquely retained frame. + /// + public void Reset() + { + InlineArray8 releasedFrames = this.frames; + this.frames = default; + Av1ReferenceFrame? releasedOutputFrame = this.outputFrame; + this.outputFrame = null; + + // Clear the live map before disposal so the store cannot expose a partially reset ownership state. When the + // output aliases a slot, let the output path perform the single release after the duplicate slot is removed. + if (releasedOutputFrame is not null) + { + for (int slot = 0; slot < SlotCount; slot++) + { + if (ReferenceEquals(releasedFrames[slot], releasedOutputFrame)) + { + releasedFrames[slot] = null; + } + } + } + + DisposeUnique(ref releasedFrames); + releasedOutputFrame?.Dispose(); + } + + /// + /// Releases every uniquely retained frame and clears all reference-map slots. + /// + public void Dispose() => this.Reset(); + + /// + /// Determines whether the live reference map or presentation output retains a frame. + /// + /// The frame whose ownership is queried. + /// when the store still owns the frame. + private bool IsRetained(Av1ReferenceFrame frame) + { + if (ReferenceEquals(this.outputFrame, frame)) + { + return true; + } + + for (int slot = 0; slot < SlotCount; slot++) + { + if (ReferenceEquals(this.frames[slot], frame)) + { + return true; + } + } + + return false; + } + + /// + /// Releases each distinct frame owner in a fixed-size set exactly once. + /// + /// The inline set of frame references to release. + private static void DisposeUnique(ref InlineArray8 frames) + { + for (int frameIndex = 0; frameIndex < SlotCount; frameIndex++) + { + Av1ReferenceFrame? frame = frames[frameIndex]; + + if (frame is null) + { + continue; + } + + // Null every later alias before disposal. The store intentionally represents shared slot ownership through + // object identity, so no separately allocated reference-count state is needed for the eight-entry map. + for (int duplicateIndex = frameIndex + 1; duplicateIndex < SlotCount; duplicateIndex++) + { + if (ReferenceEquals(frames[duplicateIndex], frame)) + { + frames[duplicateIndex] = null; + } + } + + frames[frameIndex] = null; + frame.Dispose(); + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1BlockModeInfo.cs b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1BlockModeInfo.cs new file mode 100644 index 0000000000..ba9453aa29 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1BlockModeInfo.cs @@ -0,0 +1,465 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Diagnostics.CodeAnalysis; +using SixLabors.ImageSharp.Formats.Heif.Av1.Motion; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; +using SixLabors.ImageSharp.Memory; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +/// +/// Stores block-size, intra/inter prediction, transform, and palette decisions shared by AV1 block processing. +/// +internal struct Av1BlockModeInfo +{ + /// + /// Stores the primary and optional secondary reference-frame labels. + /// + private InlineArray2 referenceFrames; + + /// + /// Stores the motion vector associated with each reference-frame label. + /// + private InlineArray2 motionVectors; + + /// + /// Stores the vertical and horizontal subpixel interpolation filters in that order. + /// + private InlineArray2 interpolationFilters; + + /// + /// The palette size for the luma plane. + /// + private int lumaPaletteSize; + + /// + /// The palette size shared by both chroma planes. + /// + private int chromaPaletteSize; + + /// + /// Stores the decoded luma palette colors. + /// + private InlineArray8 lumaPaletteColors; + + /// + /// Stores the decoded blue-difference chroma palette colors. + /// + private InlineArray8 chromaBluePaletteColors; + + /// + /// Stores the decoded red-difference chroma palette colors. + /// + private InlineArray8 chromaRedPaletteColors; + + /// + /// Stores the luma palette color-index map. + /// + private Rectangle lumaPaletteColorIndexBounds; + + /// + /// Stores the shared chroma palette color-index map. + /// + private Rectangle chromaPaletteColorIndexBounds; + + /// + /// The directional prediction angle adjustment for luma. + /// + private int lumaAngleDelta; + + /// + /// The directional prediction angle adjustment shared by both chroma planes. + /// + private int chromaAngleDelta; + + /// + /// The plane-relative index of the first luma transform. + /// + private int firstLumaTransformLocation; + + /// + /// The plane-relative index of the first chroma transform. + /// + private int firstChromaTransformLocation; + + /// + /// The number of luma transform units. + /// + private int lumaTransformUnitCount; + + /// + /// The number of transform units for one chroma plane. + /// + private int chromaTransformUnitCount; + + /// + /// Initializes a new instance of the structure. + /// + /// The decoded block size. + /// The block origin relative to its superblock in 4x4 mode-information units. + public Av1BlockModeInfo(Av1BlockSize blockSize, Point positionInSuperblock) + { + this.BlockSize = blockSize; + this.PositionInSuperblock = positionInSuperblock; + + // Both entries begin absent because inter syntax has not selected either reference yet. Intra parsing replaces + // the primary entry with the current frame while retaining None as the optional secondary reference. + this.referenceFrames[0] = Av1ReferenceFrameType.None; + this.referenceFrames[1] = Av1ReferenceFrameType.None; + } + + /// + /// Gets the decoded block size. + /// + public Av1BlockSize BlockSize { get; } + + /// + /// Gets or sets the frame storage index shared by every mode-information position covered by this block. + /// + public int ModeInfoIndex { get; set; } + + /// + /// Gets or sets the for the luminance channel. + /// + public Av1PredictionMode YMode { get; set; } + + /// + /// Gets the primary and optional secondary reference-frame labels. + /// + /// + /// Index zero is the primary reference. Index one is for a single-reference + /// block, for an inter-intra block, or the secondary inter-reference label + /// for compound prediction. + /// + [UnscopedRef] + public Span ReferenceFrames => this.referenceFrames; + + /// + /// Gets the decoded motion vectors corresponding to . + /// + [UnscopedRef] + public Span MotionVectors => this.motionVectors; + + /// + /// Gets the interpolation filters used for vertical and horizontal subpixel prediction. + /// + /// + /// Index zero is the vertical filter and index one is the horizontal filter, matching the reference decoder's + /// InterpFilters.y_filter and InterpFilters.x_filter layout. + /// + [UnscopedRef] + public Span InterpolationFilters => this.interpolationFilters; + + /// + /// Gets or sets the selected index in the derived reference-motion-vector stack. + /// + /// + /// The AV1 syntax constrains this value to the inclusive range zero through two. + /// + public byte ReferenceMotionVectorIndex { get; set; } + + /// + /// Gets or sets the motion model used to construct inter prediction. + /// + public Av1MotionMode MotionMode { get; set; } + + /// + /// Gets or sets the affine model derived for local warped prediction. + /// + public Av1GlobalMotionParameters WarpedMotionParameters { get; set; } + + /// + /// Gets or sets the intra predictor blended with a single-reference inter predictor. + /// + public Av1InterIntraMode InterIntraMode { get; set; } + + /// + /// Gets or sets a value indicating whether inter-intra prediction uses a wedge mask. + /// + public bool UseInterIntraWedge { get; set; } + + /// + /// Gets or sets the inter-intra wedge-mask index in the inclusive range 0 through 15. + /// + public byte InterIntraWedgeIndex { get; set; } + + /// + /// Gets or sets a value indicating whether compound prediction uses the masked-compound mode group. + /// + public bool CompoundGroupIndex { get; set; } + + /// + /// Gets or sets a value indicating whether unmasked compound prediction uses average blending. + /// A value of selects distance-weighted blending. + /// + public bool CompoundIndex { get; set; } + + /// + /// Gets or sets the compound blending method selected for two inter predictors. + /// + public Av1CompoundType CompoundType { get; set; } + + /// + /// Gets or sets the compound wedge-mask index in the inclusive range 0 through 15. + /// + public byte CompoundWedgeIndex { get; set; } + + /// + /// Gets or sets a value indicating whether the compound wedge mask is inverted. + /// + public bool CompoundWedgeSign { get; set; } + + /// + /// Gets or sets the orientation of the difference-weighted compound mask. + /// + public Av1DifferenceWeightedMaskType DifferenceWeightedMaskType { get; set; } + + /// + /// Gets or sets a value indicating whether residual coefficients are omitted for the block. + /// + public bool Skip { get; set; } + + /// + /// Gets or sets the partition type that produced the block. + /// + public Av1PartitionType PartitionType { get; set; } + + /// + /// Gets or sets a value indicating whether compound skip mode is selected. + /// + public bool SkipMode { get; set; } + + /// + /// Gets or sets the segmentation identifier assigned to the block. + /// + public int SegmentId { get; set; } + + /// + /// Gets or sets a value indicating whether temporal prediction supplied the segment identifier. + /// + public bool SegmentIdPredicted { get; set; } + + /// + /// Gets or sets the chroma intra-prediction mode. + /// + public Av1ChromaPredictionMode UvMode { get; set; } + + /// + /// Gets or sets a value indicating whether intra block copy is selected. + /// + public bool UseIntraBlockCopy { get; set; } + + /// + /// Gets or sets the intra-block-copy displacement vector in one-eighth-sample units. + /// + public Av1MotionVector DisplacementVector { get; set; } + + /// + /// Gets or sets the packed chroma-from-luma alpha magnitude indices. + /// + public int ChromaFromLumaAlphaIndex { get; set; } + + /// + /// Gets or sets the joint chroma-from-luma alpha sign value. + /// + public int ChromaFromLumaAlphaSign { get; set; } + + /// + /// Gets or sets a value indicating whether filter-intra prediction is enabled for the block. + /// + public bool UseFilterIntra { get; set; } + + /// + /// Gets the position relative to the superblock in 4x4 mode-information units. + /// + public Point PositionInSuperblock { get; } + + /// + /// Gets or sets the filter-intra mode selected for the block. + /// + public Av1FilterIntraMode FilterIntraMode { get; set; } + + /// + /// Gets the directional prediction angle adjustment for a color plane. + /// + /// The color plane. + /// The luma adjustment or the adjustment shared by both chroma planes. + public int GetAngleDelta(Av1Plane plane) => plane == Av1Plane.Y ? this.lumaAngleDelta : this.chromaAngleDelta; + + /// + /// Sets the directional prediction angle adjustment for a plane class. + /// + /// The luma or chroma plane class. + /// The directional prediction angle adjustment. + public void SetAngleDelta(Av1PlaneType planeType, int value) + { + if (planeType == Av1PlaneType.Y) + { + this.lumaAngleDelta = value; + } + else + { + this.chromaAngleDelta = value; + } + } + + /// + /// Gets the plane-relative index of the first transform for a color plane. + /// + /// The color plane. + /// The first transform index for luma or the selected chroma plane. + public int GetFirstTransformLocation(Av1Plane plane) + => plane == Av1Plane.Y ? this.firstLumaTransformLocation : this.firstChromaTransformLocation; + + /// + /// Gets the plane-relative index of the first transform for a plane class. + /// + /// The luma or chroma plane class. + /// The first transform index for the plane class. + public int GetFirstTransformLocation(Av1PlaneType planeType) + => planeType == Av1PlaneType.Y ? this.firstLumaTransformLocation : this.firstChromaTransformLocation; + + /// + /// Sets the plane-relative index of the first transform for a plane class. + /// + /// The luma or chroma plane class. + /// The first transform index. + public void SetFirstTransformLocation(Av1PlaneType planeType, int value) + { + if (planeType == Av1PlaneType.Y) + { + this.firstLumaTransformLocation = value; + } + else + { + this.firstChromaTransformLocation = value; + } + } + + /// + /// Gets the number of transform units for a color plane. + /// + /// The color plane. + /// The luma count or the count for one chroma plane. + public int GetTransformUnitCount(Av1Plane plane) + => plane == Av1Plane.Y ? this.lumaTransformUnitCount : this.chromaTransformUnitCount; + + /// + /// Gets the number of transform units for a plane class. + /// + /// The luma or chroma plane class. + /// The transform-unit count for the plane class. + public int GetTransformUnitCount(Av1PlaneType planeType) + => planeType == Av1PlaneType.Y ? this.lumaTransformUnitCount : this.chromaTransformUnitCount; + + /// + /// Sets the number of transform units for a plane class. + /// + /// The luma or chroma plane class. + /// The transform-unit count. + public void SetTransformUnitCount(Av1PlaneType planeType, int value) + { + if (planeType == Av1PlaneType.Y) + { + this.lumaTransformUnitCount = value; + } + else + { + this.chromaTransformUnitCount = value; + } + } + + /// + /// Gets the palette size for the specified color plane. + /// + /// The color plane. + /// The palette size for the plane. + public int GetPaletteSize(Av1Plane plane) => plane == Av1Plane.Y ? this.lumaPaletteSize : this.chromaPaletteSize; + + /// + /// Gets the palette size for the specified plane class. + /// + /// The luma or chroma plane class. + /// The palette size for the plane class. + public int GetPaletteSize(Av1PlaneType planeType) => planeType == Av1PlaneType.Y ? this.lumaPaletteSize : this.chromaPaletteSize; + + /// + /// Sets the luma and shared chroma palette sizes. + /// + /// The luma palette size. + /// The palette size shared by the chroma planes. + public void SetPaletteSizes(int ySize, int uvSize) + { + this.lumaPaletteSize = ySize; + this.chromaPaletteSize = uvSize; + } + + /// + /// Gets the decoded palette colors for a color plane. + /// + /// The color plane. + /// The palette colors in prediction-index order. + [UnscopedRef] + public ReadOnlySpan GetPaletteColors(Av1Plane plane) + { + if (plane == Av1Plane.Y) + { + return this.lumaPaletteColors[..this.lumaPaletteSize]; + } + + return plane == Av1Plane.U + ? this.chromaBluePaletteColors[..this.chromaPaletteSize] + : this.chromaRedPaletteColors[..this.chromaPaletteSize]; + } + + /// + /// Stores the decoded palette colors for a color plane. + /// + /// The color plane. + /// The palette colors in prediction-index order. + public void SetPaletteColors(Av1Plane plane, ReadOnlySpan colors) + { + if (plane == Av1Plane.Y) + { + colors.CopyTo(this.lumaPaletteColors); + } + else if (plane == Av1Plane.U) + { + colors.CopyTo(this.chromaBluePaletteColors); + } + else + { + colors.CopyTo(this.chromaRedPaletteColors); + } + } + + /// + /// Gets the palette color-index map for a color plane. + /// + /// The color plane. + /// The decoder-session palette map for the selected plane class. + /// The luma map for or the shared chroma map for either chroma plane. + public Buffer2DRegion GetPaletteColorIndexMap(Av1Plane plane, Buffer2D colorIndexMap) + => new( + colorIndexMap, + plane == Av1Plane.Y ? this.lumaPaletteColorIndexBounds : this.chromaPaletteColorIndexBounds); + + /// + /// Stores the palette color-index map for a plane class. + /// + /// The luma or shared chroma plane class. + /// The row-major color-index bounds including coded-block edge padding. + public void SetPaletteColorIndexMap(Av1PlaneType planeType, Rectangle bounds) + { + if (planeType == Av1PlaneType.Y) + { + this.lumaPaletteColorIndexBounds = bounds; + } + else + { + this.chromaPaletteColorIndexBounds = bounds; + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1ComponentType.cs b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1ComponentType.cs new file mode 100644 index 0000000000..2490f254c7 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1ComponentType.cs @@ -0,0 +1,40 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +/// +/// Identifies the luma or chroma component class used by AV1 entropy contexts. +/// +internal enum Av1ComponentType +{ + /// + /// The luma component. + /// + Luminance = 0, + + /// + /// Both chroma components. + /// + Chroma = 1, + + /// + /// The blue-difference chroma component. + /// + ChromaCb = 2, + + /// + /// The red-difference chroma component. + /// + ChromaCr = 3, + + /// + /// The luma and both chroma components. + /// + All = 4, + + /// + /// No component. + /// + None = 15 +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1CompoundType.cs b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1CompoundType.cs new file mode 100644 index 0000000000..35c4c71f65 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1CompoundType.cs @@ -0,0 +1,30 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +/// +/// Identifies the blending method used to combine two AV1 inter predictors. +/// +internal enum Av1CompoundType : byte +{ + /// + /// Averages both predictors with equal weights. + /// + Average = 0, + + /// + /// Weights predictors from their relative display-order distances. + /// + DistanceWeighted = 1, + + /// + /// Selects per-pixel weights from a signaled wedge mask. + /// + Wedge = 2, + + /// + /// Derives per-pixel weights from the difference between both predictors. + /// + DifferenceWeighted = 3, +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1DifferenceWeightedMaskType.cs b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1DifferenceWeightedMaskType.cs new file mode 100644 index 0000000000..81dbcfa702 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1DifferenceWeightedMaskType.cs @@ -0,0 +1,20 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +/// +/// Identifies the orientation of an AV1 difference-weighted compound mask. +/// +internal enum Av1DifferenceWeightedMaskType : byte +{ + /// + /// Applies the predictor-difference adjustment to a base alpha weight of 38 on AV1's 0-through-64 blend scale. + /// + Type38 = 0, + + /// + /// Applies the complement of the type-38 predictor-difference mask. + /// + Type38Inverse = 1, +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1EncoderBlockModeInfo.cs b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1EncoderBlockModeInfo.cs new file mode 100644 index 0000000000..45452cf3f3 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1EncoderBlockModeInfo.cs @@ -0,0 +1,156 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +/// +/// Stores encoder-selected prediction, transform, skip, and palette state for one block. +/// +internal struct Av1EncoderBlockModeInfo +{ + /// The residual-skip flag in the packed prediction state. + private const byte SkipMask = 1 << 0; + + /// The compound-skip flag in the packed prediction state. + private const byte SkipModeMask = 1 << 1; + + /// The intra-block-copy flag in the packed prediction state. + private const byte IntraBlockCopyMask = 1 << 2; + + /// The three low bits store the segment identifier, followed by the primary reference. + private const int ReferenceFrameShift = 3; + + /// Both segment identifiers and primary references have eight possible values. + private const int SegmentAndReferenceMask = 7; + + /// The vertical filter follows the three prediction flags. + private const int VerticalFilterShift = 3; + + /// The horizontal filter follows the two-bit vertical filter. + private const int HorizontalFilterShift = 5; + + /// Two bits represent each concrete interpolation filter, excluding the frame-level switchable sentinel. + private const int InterpolationFilterMask = 3; + + // Primary references never use the absent-secondary sentinel. Pack their three bits beside the segment, + // and the concrete filters beside the flags, so adding inter syntax does not enlarge the frame-wide grid. + private byte blockSize; + private byte partitionType; + private byte flags; + private byte segmentAndReference; + private byte transformSize; + private byte mode; + private byte uvMode; + + /// + /// Gets or sets the encoded block size. + /// + public Av1BlockSize BlockSize + { + readonly get => (Av1BlockSize)this.blockSize; + set => this.blockSize = (byte)value; + } + + /// + /// Gets or sets the partition type that produced the block. + /// + public Av1PartitionType PartitionType + { + readonly get => (Av1PartitionType)this.partitionType; + set => this.partitionType = (byte)value; + } + + /// + /// Gets or sets a value indicating whether residual coefficients are omitted for the block. + /// + public bool Skip + { + readonly get => (this.flags & SkipMask) != 0; + set => this.flags = value ? (byte)(this.flags | SkipMask) : (byte)(this.flags & ~SkipMask); + } + + /// + /// Gets or sets a value indicating whether compound skip mode is selected. + /// + public bool SkipMode + { + readonly get => (this.flags & SkipModeMask) != 0; + set => this.flags = value ? (byte)(this.flags | SkipModeMask) : (byte)(this.flags & ~SkipModeMask); + } + + /// + /// Gets or sets a value indicating whether intra block copy is selected. + /// + public bool UseIntraBlockCopy + { + readonly get => (this.flags & IntraBlockCopyMask) != 0; + set => this.flags = value ? (byte)(this.flags | IntraBlockCopyMask) : (byte)(this.flags & ~IntraBlockCopyMask); + } + + /// + /// Gets or sets the segmentation identifier assigned to the block. + /// + public int SegmentId + { + readonly get => this.segmentAndReference & SegmentAndReferenceMask; + set => this.segmentAndReference = (byte)((this.segmentAndReference & ~SegmentAndReferenceMask) | value); + } + + /// + /// Gets or sets the luma transform size selected for the block. + /// + public Av1TransformSize TransformSize + { + readonly get => (Av1TransformSize)this.transformSize; + set => this.transformSize = (byte)value; + } + + /// + /// Gets or sets the luma prediction mode written for the block. + /// + public Av1PredictionMode Mode + { + readonly get => (Av1PredictionMode)this.mode; + set => this.mode = (byte)value; + } + + /// + /// Gets or sets the chroma prediction mode written for the block. + /// + public Av1ChromaPredictionMode UvMode + { + readonly get => (Av1ChromaPredictionMode)this.uvMode; + set => this.uvMode = (byte)value; + } + + /// + /// Gets or sets the primary prediction reference selected for the block. + /// + public Av1ReferenceFrameType ReferenceFrame + { + readonly get => (Av1ReferenceFrameType)(this.segmentAndReference >> ReferenceFrameShift); + set => this.segmentAndReference = (byte)((this.segmentAndReference & SegmentAndReferenceMask) | ((int)value << ReferenceFrameShift)); + } + + /// + /// Gets or sets the concrete vertical interpolation filter selected for the block. + /// + public Av1InterpolationFilter VerticalInterpolationFilter + { + readonly get => (Av1InterpolationFilter)((this.flags >> VerticalFilterShift) & InterpolationFilterMask); + set => this.flags = (byte)((this.flags & ~(InterpolationFilterMask << VerticalFilterShift)) | ((int)value << VerticalFilterShift)); + } + + /// + /// Gets or sets the concrete horizontal interpolation filter selected for the block. + /// + public Av1InterpolationFilter HorizontalInterpolationFilter + { + readonly get => (Av1InterpolationFilter)((this.flags >> HorizontalFilterShift) & InterpolationFilterMask); + set => this.flags = (byte)((this.flags & ~(InterpolationFilterMask << HorizontalFilterShift)) | ((int)value << HorizontalFilterShift)); + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1EncoderBlockStruct.cs b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1EncoderBlockStruct.cs new file mode 100644 index 0000000000..04354869e0 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1EncoderBlockStruct.cs @@ -0,0 +1,85 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Diagnostics.CodeAnalysis; +using System.Runtime.InteropServices; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +/// +/// Stores encoder block geometry and its selected coding-mode information. +/// +[StructLayout(LayoutKind.Sequential, Pack = 1, Size = StorageSize)] +internal struct Av1EncoderBlockStruct +{ + /// + /// The fixed byte width of one packed final-block decision. + /// + public const int StorageSize = 8; + + /// + /// Stores the block's prediction-unit syntax inline. + /// + private Av1EncoderPredictionUnit predictionUnit; + + // These syntax values have AV1-defined byte-sized ranges. Storing their encoded widths explicitly + // prevents CLR field alignment from inflating every entry in the 1,024-element decision workspace. + private byte hasChroma; + private byte quantizationIndex; + private byte segmentId; + private byte filterIntraMode; + + /// + /// Gets or sets a value indicating whether this luma block owns the corresponding chroma syntax. + /// + public bool HasChroma + { + readonly get => this.hasChroma != 0; + set => this.hasChroma = value ? (byte)1 : (byte)0; + } + + /// + /// Gets or sets the quantizer index used for the block. + /// + public int QuantizationIndex + { + readonly get => this.quantizationIndex; + set => this.quantizationIndex = (byte)value; + } + + /// + /// Gets or sets the segmentation identifier assigned to the block. + /// + public int SegmentId + { + readonly get => this.segmentId; + set => this.segmentId = (byte)value; + } + + /// + /// Gets or sets the filter-intra mode selected for the block. + /// + public Av1FilterIntraMode FilterIntraMode + { + readonly get => (Av1FilterIntraMode)this.filterIntraMode; + set => this.filterIntraMode = (byte)value; + } + + /// + /// Gets or sets the dynamic-reference-list index selected for an inter block. + /// + /// + /// Filter-intra and inter prediction are mutually exclusive, so both syntax branches share one packed byte. + /// + public int ReferenceMotionVectorIndex + { + readonly get => this.filterIntraMode; + set => this.filterIntraMode = (byte)value; + } + + /// + /// Gets the encoder prediction-unit state for the block. + /// + [UnscopedRef] + public ref Av1EncoderPredictionUnit PredictionUnit => ref this.predictionUnit; +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1EncoderCoefficientBuffer.cs b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1EncoderCoefficientBuffer.cs new file mode 100644 index 0000000000..a7765c75c5 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1EncoderCoefficientBuffer.cs @@ -0,0 +1,156 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.InteropServices; +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Memory; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +/// +/// Owns the raster-ordered transform coefficients retained for every superblock in one encoded frame. +/// +internal sealed class Av1EncoderCoefficientBuffer : IDisposable +{ + /// + /// The number of coefficients represented by one entry in libaom's EOB arrays. + /// + public const int TransformBlockUnitCoefficientCount = 1 << (Av1Constants.ModeInfoSizeLog2 * 2); + + /// + /// Stores one complete superblock's coefficients and packed transform-block state in each row. + /// + private readonly Buffer2D storage; + + /// + /// Initializes a new instance of the class. + /// + /// The configuration providing the frame allocator. + /// The sequence header defining superblock and chroma geometry. + /// The coded luma width. + /// The coded luma height. + public Av1EncoderCoefficientBuffer( + Configuration configuration, + ObuSequenceHeader sequenceHeader, + int width, + int height) + { + int superblockSizeLog2 = sequenceHeader.SuperblockSizeLog2; + int superblockSize = 1 << superblockSizeLog2; + this.SuperblockColumnCount = Av1Math.DivideLog2Ceiling(width, superblockSizeLog2); + this.SuperblockRowCount = Av1Math.DivideLog2Ceiling(height, superblockSizeLog2); + this.SuperblockCount = this.SuperblockColumnCount * this.SuperblockRowCount; + this.LumaCoefficientCount = superblockSize * superblockSize; + + ObuColorConfig colorConfig = sequenceHeader.ColorConfig; + int chromaSubsampling = (colorConfig.SubSamplingX ? 1 : 0) + (colorConfig.SubSamplingY ? 1 : 0); + this.ChromaCoefficientCount = colorConfig.IsMonochrome ? 0 : this.LumaCoefficientCount >> chromaSubsampling; + this.CoefficientsPerSuperblock = this.LumaCoefficientCount + (2 * this.ChromaCoefficientCount); + this.LumaTransformBlockCount = this.LumaCoefficientCount / TransformBlockUnitCoefficientCount; + this.ChromaTransformBlockCount = this.ChromaCoefficientCount / TransformBlockUnitCoefficientCount; + this.TransformBlocksPerSuperblock = this.LumaTransformBlockCount + (2 * this.ChromaTransformBlockCount); + int storageElementsPerSuperblock = this.CoefficientsPerSuperblock + this.TransformBlocksPerSuperblock; + + // libaom stores finalized coefficients by raster-ordered superblock. A two-dimensional owner preserves that + // layout, and packing the EOB/type state into the same row removes its two additional frame-sized allocations. + this.storage = configuration.MemoryAllocator.Allocate2D( + storageElementsPerSuperblock, + this.SuperblockCount); + } + + /// + /// Gets the number of superblock columns covering the coded frame. + /// + public int SuperblockColumnCount { get; } + + /// + /// Gets the number of superblock rows covering the coded frame. + /// + public int SuperblockRowCount { get; } + + /// + /// Gets the number of superblocks covering the coded frame. + /// + public int SuperblockCount { get; } + + /// + /// Gets the number of luma coefficient positions reserved for each superblock. + /// + public int LumaCoefficientCount { get; } + + /// + /// Gets the number of coefficient positions reserved for each chroma plane in each superblock. + /// + public int ChromaCoefficientCount { get; } + + /// + /// Gets the number of coefficient positions reserved for each complete superblock. + /// + public int CoefficientsPerSuperblock { get; } + + /// + /// Gets the number of luma transform-block positions reserved for each superblock. + /// + public int LumaTransformBlockCount { get; } + + /// + /// Gets the number of transform-block positions reserved for each chroma plane in each superblock. + /// + public int ChromaTransformBlockCount { get; } + + /// + /// Gets the number of transform-block positions reserved for each complete superblock. + /// + public int TransformBlocksPerSuperblock { get; } + + /// + /// Gets the total number of coefficient positions retained for the frame. + /// + public long TotalCoefficientCount => (long)this.CoefficientsPerSuperblock * this.SuperblockCount; + + /// + /// Gets one component plane's coefficient span for a raster-ordered superblock. + /// + /// The raster-ordered superblock index. + /// The requested component plane. + /// The complete coefficient span reserved for that plane and superblock. + public Span GetPlaneSpan(int superblockIndex, Av1Plane plane) + { + Span superblock = this.storage.DangerousGetRowSpan(superblockIndex); + return plane switch + { + Av1Plane.Y => superblock[..this.LumaCoefficientCount], + Av1Plane.U => superblock.Slice(this.LumaCoefficientCount, this.ChromaCoefficientCount), + _ => superblock.Slice( + this.LumaCoefficientCount + this.ChromaCoefficientCount, + this.ChromaCoefficientCount) + }; + } + + /// + /// Gets one component plane's transform-block state for a raster-ordered superblock. + /// + /// The raster-ordered superblock index. + /// The requested component plane. + /// One state entry for every 4x4 coefficient unit in the plane. + public Span GetTransformBlockSpan(int superblockIndex, Av1Plane plane) + { + Span superblock = this.storage.DangerousGetRowSpan(superblockIndex); + Span transformBlocks = + MemoryMarshal.Cast(superblock[this.CoefficientsPerSuperblock..]); + + return plane switch + { + Av1Plane.Y => transformBlocks[..this.LumaTransformBlockCount], + Av1Plane.U => transformBlocks.Slice(this.LumaTransformBlockCount, this.ChromaTransformBlockCount), + _ => transformBlocks.Slice( + this.LumaTransformBlockCount + this.ChromaTransformBlockCount, + this.ChromaTransformBlockCount) + }; + } + + /// + /// Releases the frame coefficient storage. + /// + public void Dispose() => this.storage.Dispose(); +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1EncoderCommon.cs b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1EncoderCommon.cs new file mode 100644 index 0000000000..d548ce822f --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1EncoderCommon.cs @@ -0,0 +1,37 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +/// +/// Holds sequence, frame, and macroblock state shared across AV1 encoder stages. +/// +internal class Av1EncoderCommon +{ + /// + /// Gets or sets the frame height in 4x4 mode-information units. + /// + public int ModeInfoRowCount { get; set; } + + /// + /// Gets or sets the frame width in 4x4 mode-information units. + /// + public int ModeInfoColumnCount { get; set; } + + /// + /// Gets or sets the row stride of frame mode information in 4x4 units. + /// + public int ModeInfoStride { get; set; } + + /// + /// Gets or sets the coded frame dimensions. + /// + public required ObuFrameSize FrameSize { get; set; } + + /// + /// Gets or sets the tile layout for the current frame. + /// + public required ObuTileGroupHeader TilesInfo { get; set; } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1EncoderDisplacementVector.cs b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1EncoderDisplacementVector.cs new file mode 100644 index 0000000000..2ab0d32f54 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1EncoderDisplacementVector.cs @@ -0,0 +1,20 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +/// +/// Stores one encoder-selected integer displacement vector in the AV1 signed component domain. +/// +internal struct Av1EncoderDisplacementVector +{ + /// + /// The vertical component in one-eighth-sample units. + /// + public short Row; + + /// + /// The horizontal component in one-eighth-sample units. + /// + public short Column; +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1EncoderModeInfoBuffer.cs b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1EncoderModeInfoBuffer.cs new file mode 100644 index 0000000000..19dbe9eefe --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1EncoderModeInfoBuffer.cs @@ -0,0 +1,92 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Buffers; +using System.Runtime.CompilerServices; +using SixLabors.ImageSharp.Memory; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +/// +/// Owns the allocation-index grid and mode-information values for one encoded AV1 frame. +/// +internal sealed class Av1EncoderModeInfoBuffer : IDisposable +{ + private const int CodedDimensionAlignmentLog2 = 3; + private const int ModeInfoAlignmentLog2 = Av1Constants.MaxSuperBlockSizeLog2 - Av1Constants.ModeInfoSizeLog2; + private IMemoryOwner? owner; + private readonly ByteMemoryManager grid; + private readonly ByteMemoryManager allocation; + + /// + /// Initializes a new instance of the class. + /// + /// The configuration providing the memory allocator. + /// The visible frame width in luma samples. + /// The visible frame height in luma samples. + /// Whether each allocated mode-information value represents an 8x8 region. + public Av1EncoderModeInfoBuffer( + Configuration configuration, + int frameWidth, + int frameHeight, + bool disallow4x4AllFrames) + { + this.ModeInfoColumnCount = Av1Math.AlignPowerOf2(frameWidth, CodedDimensionAlignmentLog2) >> Av1Constants.ModeInfoSizeLog2; + this.ModeInfoRowCount = Av1Math.AlignPowerOf2(frameHeight, CodedDimensionAlignmentLog2) >> Av1Constants.ModeInfoSizeLog2; + this.ModeInfoStride = Av1Math.AlignPowerOf2(this.ModeInfoColumnCount, ModeInfoAlignmentLog2); + int alignedModeInfoRowCount = Av1Math.AlignPowerOf2(this.ModeInfoRowCount, ModeInfoAlignmentLog2); + int allocationShift = disallow4x4AllFrames ? 1 : 0; + int gridLength = checked(this.ModeInfoStride * alignedModeInfoRowCount); + int allocationLength = checked((this.ModeInfoStride >> allocationShift) * (alignedModeInfoRowCount >> allocationShift)); + int gridByteLength = checked(gridLength * sizeof(int)); + int allocationByteLength = checked(allocationLength * Unsafe.SizeOf()); + int storageLength = checked(gridByteLength + allocationByteLength); + + // The pointer grid and value allocation share one frame lifetime. Packing both regions into one clean + // owner retains libaom's independent typed layouts without its separate allocation and cleanup paths. + this.owner = configuration.MemoryAllocator.Allocate(storageLength, AllocationOptions.Clean); + Memory storage = this.owner.Memory[..storageLength]; + this.grid = new ByteMemoryManager(storage[..gridByteLength]); + this.allocation = new ByteMemoryManager(storage.Slice(gridByteLength, allocationByteLength)); + this.Disallow4x4AllFrames = disallow4x4AllFrames; + } + + /// + /// Gets the visible frame height in 4x4 mode-information units. + /// + public int ModeInfoRowCount { get; } + + /// + /// Gets the visible frame width in 4x4 mode-information units. + /// + public int ModeInfoColumnCount { get; } + + /// + /// Gets the aligned row stride of the allocation-index grid in 4x4 mode-information units. + /// + public int ModeInfoStride { get; } + + /// + /// Gets a value indicating whether each allocated mode-information value represents an 8x8 region. + /// + public bool Disallow4x4AllFrames { get; } + + /// + /// Gets the frame grid that maps each 4x4 position to its mode-information allocation index. + /// + public Memory Grid => this.grid.Memory; + + /// + /// Gets the contiguous mode-information values addressed by . + /// + public Memory Allocation => this.allocation.Memory; + + /// + /// Returns the packed frame storage to the configured memory allocator. + /// + public void Dispose() + { + this.owner?.Dispose(); + this.owner = null; + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1EncoderPaletteInfo.cs b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1EncoderPaletteInfo.cs new file mode 100644 index 0000000000..db3dfca31a --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1EncoderPaletteInfo.cs @@ -0,0 +1,54 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Diagnostics.CodeAnalysis; +using System.Runtime.InteropServices; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +/// +/// Stores the selected luma and chroma palette sizes and colors for one encoder block. +/// +[StructLayout(LayoutKind.Sequential, Size = StorageSize)] +internal struct Av1EncoderPaletteInfo +{ + /// + /// The packed size of two palette sizes and three eight-color planes. + /// + public const int StorageSize = 50; + + private InlineArray2 paletteSizes; + private InlineArray24 paletteColors; + + /// + /// Gets the writable luma and shared chroma palette sizes. + /// + [UnscopedRef] + public Span PaletteSizes => this.paletteSizes; + + /// + /// Gets the selected colors for one color plane. + /// + /// The color plane. + /// The selected colors in palette-index order. + [UnscopedRef] + public Span GetColors(Av1Plane plane) + { + int planeIndex = (int)plane; + int paletteSize = this.paletteSizes[planeIndex == 0 ? 0 : 1]; + Span colors = this.paletteColors; + return colors.Slice(planeIndex * Av1Constants.PaletteMaxSize, paletteSize); + } + + /// + /// Stores the selected colors for one color plane. + /// + /// The color plane. + /// The colors in palette-index order. + public void SetColors(Av1Plane plane, ReadOnlySpan colors) + { + int offset = (int)plane * Av1Constants.PaletteMaxSize; + Span destination = this.paletteColors; + colors.CopyTo(destination[offset..]); + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1EncoderPaletteMapBuffer.cs b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1EncoderPaletteMapBuffer.cs new file mode 100644 index 0000000000..1740cbecf5 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1EncoderPaletteMapBuffer.cs @@ -0,0 +1,58 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Memory; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +/// +/// Exposes reusable luma and chroma palette color-index maps over superblock-workspace storage. +/// +internal sealed class Av1EncoderPaletteMapBuffer : IDisposable +{ + /// + /// The width and height of each maximum-superblock map. + /// + public const int MapLength = 1 << Av1Constants.MaxSuperBlockSizeLog2; + + /// + /// The combined byte length of the luma and chroma maps. + /// + public const int StorageLength = 2 * MapLength * MapLength; + + private readonly Buffer2D luma; + private readonly Buffer2D chroma; + + /// + /// Initializes a new instance of the class. + /// + /// The superblock-owned backing storage. + public Av1EncoderPaletteMapBuffer(Memory storage) + { + int mapArea = MapLength * MapLength; + Memory mapStorage = storage[..StorageLength]; + this.luma = Buffer2D.WrapMemory(mapStorage[..mapArea], MapLength, MapLength); + this.chroma = Buffer2D.WrapMemory(mapStorage[mapArea..], MapLength, MapLength); + } + + /// + /// Gets a block-sized view of the luma or shared chroma color-index map. + /// + /// The luma or shared chroma plane class. + /// The padded plane-block width. + /// The padded plane-block height. + /// The reusable map region beginning at the workspace origin. + public Buffer2DRegion GetMap(Av1PlaneType planeType, int width, int height) + => new( + planeType == Av1PlaneType.Y ? this.luma : this.chroma, + new Rectangle(0, 0, width, height)); + + /// + /// Releases the non-owning two-dimensional wrappers. + /// + public void Dispose() + { + this.luma.Dispose(); + this.chroma.Dispose(); + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1EncoderPictureBuffer.cs b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1EncoderPictureBuffer.cs new file mode 100644 index 0000000000..cb70eb6094 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1EncoderPictureBuffer.cs @@ -0,0 +1,391 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Buffers; +using System.Runtime.CompilerServices; +using SixLabors.ImageSharp.Formats.Heif.Av1.Motion; +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Memory; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +/// +/// Owns reusable mode information, segmentation data, and tile-neighbor contexts for fixed-geometry AV1 pictures. +/// +internal sealed class Av1EncoderPictureBuffer : IDisposable +{ + private readonly Av1EncoderModeInfoBuffer modeInfo; + private readonly IMemoryOwner stateStorage; + + /// + /// The exact packed state region cleared between frames without touching excess pool capacity. + /// + private readonly Memory stateMemory; + + private readonly ByteMemoryManager partitionContextMemory; + private readonly Av1NeighborArrayUnit[] partitionContexts; + private readonly Av1NeighborArrayUnit[] lumaCoefficientContexts; + private readonly Av1NeighborArrayUnit[] blueCoefficientContexts; + private readonly Av1NeighborArrayUnit[] redCoefficientContexts; + private readonly Av1NeighborArrayUnit[] transformContexts; + private readonly Av1NeighborArrayUnit[] paletteContexts; + + /// + /// Initializes a new instance of the class. + /// + /// The configuration providing picture-lifetime memory. + /// The sequence header defining superblock and chroma geometry. + /// The frame header defining dimensions and tiles. + /// The visible luma width. + /// The visible luma height. + /// Whether each allocated mode-information value represents an 8x8 region. + public Av1EncoderPictureBuffer( + Configuration configuration, + ObuSequenceHeader sequenceHeader, + ObuFrameHeader frameHeader, + int width, + int height, + bool disallow4x4AllFrames) + : this( + configuration, + sequenceHeader, + frameHeader, + width, + height, + disallow4x4AllFrames, + frameHeader.AllowScreenContentTools, + frameHeader.AllowIntraBlockCopy || !frameHeader.IsIntra, + frameHeader.AllowIntraBlockCopy) + { + } + + /// + /// Initializes a new instance of the class with the maximum state + /// required by a fixed-geometry sequence. + /// + /// The configuration providing picture-lifetime memory. + /// The sequence header defining superblock and chroma geometry. + /// The initial frame header defining dimensions and tiles. + /// The visible luma width. + /// The visible luma height. + /// Whether each allocated mode-information value represents an 8x8 region. + /// Whether palette neighbor state can be required by any frame. + /// Whether inter or intra-block-copy vectors can be required by any frame. + /// Whether intra-block-copy search state can be required by any frame. + public Av1EncoderPictureBuffer( + Configuration configuration, + ObuSequenceHeader sequenceHeader, + ObuFrameHeader frameHeader, + int width, + int height, + bool disallow4x4AllFrames, + bool allocateScreenContentState, + bool allocateMotionVectorState, + bool allocateIntraBlockCopySearch) + { + const int ContextAlignmentLog2 = Av1Constants.MaxSuperBlockSizeLog2 - Av1Constants.ModeInfoSizeLog2; + this.modeInfo = new Av1EncoderModeInfoBuffer( + configuration, + width, + height, + disallow4x4AllFrames); + + try + { + int alignedModeInfoRowCount = Av1Math.AlignPowerOf2(this.modeInfo.ModeInfoRowCount, ContextAlignmentLog2); + int lumaLeftLength = alignedModeInfoRowCount; + int lumaTopLength = this.modeInfo.ModeInfoStride; + ObuColorConfig colorConfig = sequenceHeader.ColorConfig; + int chromaLeftLength = colorConfig.IsMonochrome + ? 0 + : lumaLeftLength >> (colorConfig.SubSamplingY ? 1 : 0); + + int chromaTopLength = colorConfig.IsMonochrome + ? 0 + : lumaTopLength >> (colorConfig.SubSamplingX ? 1 : 0); + + int tileCount = frameHeader.TilesInfo.TileColumnCount * frameHeader.TilesInfo.TileRowCount; + int lumaContextLength = checked(lumaLeftLength + lumaTopLength); + int chromaContextLength = checked(chromaLeftLength + chromaTopLength); + int byteContextLengthPerTile = checked((2 * lumaContextLength) + (2 * chromaContextLength)); + int partitionContextLength = checked(tileCount * lumaContextLength); + int segmentationLength = checked(this.modeInfo.ModeInfoColumnCount * this.modeInfo.ModeInfoRowCount); + int partitionContextSize = Unsafe.SizeOf(); + int partitionStorageOffset = checked( + ((segmentationLength + partitionContextSize - 1) / partitionContextSize) * partitionContextSize); + + int partitionStorageLength = checked( + partitionContextLength * Unsafe.SizeOf()); + + int byteContextStorageOffset = checked(partitionStorageOffset + partitionStorageLength); + int byteContextStorageLength = checked(tileCount * byteContextLengthPerTile); + int byteContextStorageEnd = checked(byteContextStorageOffset + byteContextStorageLength); + int paletteLeftLength = alignedModeInfoRowCount; + int paletteTopLength = this.modeInfo.ModeInfoStride; + int paletteContextLength = checked(paletteLeftLength + paletteTopLength); + int paletteStorageOffset = allocateScreenContentState + ? Av1Math.AlignPowerOf2(byteContextStorageEnd, 1) + : byteContextStorageEnd; + + int paletteStorageLength = allocateScreenContentState + ? checked(tileCount * paletteContextLength * Unsafe.SizeOf()) + : 0; + + int paletteStorageEnd = checked(paletteStorageOffset + paletteStorageLength); + int displacementVectorLength = allocateMotionVectorState ? this.modeInfo.Allocation.Length : 0; + int displacementVectorStorageOffset = allocateMotionVectorState + ? Av1Math.AlignPowerOf2(paletteStorageEnd, 1) + : paletteStorageEnd; + + int displacementVectorStorageLength = checked( + displacementVectorLength * Unsafe.SizeOf()); + + int displacementVectorStorageEnd = checked(displacementVectorStorageOffset + displacementVectorStorageLength); + int intraBlockCopySearchStorageOffset = allocateIntraBlockCopySearch + ? Av1Math.AlignPowerOf2(displacementVectorStorageEnd, 2) + : displacementVectorStorageEnd; + + int intraBlockCopySearchStorageLength = allocateIntraBlockCopySearch + ? Av1IntraBlockCopySearchIndex.GetStorageLength(width, height) + : 0; + + int intraBlockCopySearchStorageEnd = checked( + intraBlockCopySearchStorageOffset + intraBlockCopySearchStorageLength); + + int tileStateStorageOffset = Av1Math.AlignPowerOf2(intraBlockCopySearchStorageEnd, 2); + int cdefPresetLength = tileCount * Av1Constants.CdefUnitsPerSuperblock; + int tileStateLength = cdefPresetLength + (3 * tileCount); + int tileStateStorageLength = tileStateLength * sizeof(int); + int stateStorageLength = checked(tileStateStorageOffset + tileStateStorageLength); + + // Segmentation and every tile edge share one clean picture lifetime. The partition region begins at its + // native alignment. CDEF, quantizer, and encoded-tile bounds occupy one aligned trailing integer region + // instead of allocating separate managed arrays for every picture. + this.stateStorage = configuration.MemoryAllocator.Allocate( + stateStorageLength, + AllocationOptions.Clean); + + this.stateMemory = this.stateStorage.Memory[..stateStorageLength]; + Memory stateStorage = this.stateMemory; + this.partitionContextMemory = new ByteMemoryManager( + stateStorage.Slice(partitionStorageOffset, partitionStorageLength)); + + Memory partitionStorage = this.partitionContextMemory.Memory; + Memory byteContextStorage = stateStorage.Slice(byteContextStorageOffset, byteContextStorageLength); + this.partitionContexts = new Av1NeighborArrayUnit[tileCount]; + this.lumaCoefficientContexts = new Av1NeighborArrayUnit[tileCount]; + this.blueCoefficientContexts = new Av1NeighborArrayUnit[tileCount]; + this.redCoefficientContexts = new Av1NeighborArrayUnit[tileCount]; + this.transformContexts = new Av1NeighborArrayUnit[tileCount]; + Memory paletteStorage = Memory.Empty; + if (allocateScreenContentState) + { + // Palette entries contain 16-bit colors, so their packed typed region begins at an even byte offset. + ByteMemoryManager paletteMemory = new( + stateStorage.Slice(paletteStorageOffset, paletteStorageLength)); + + paletteStorage = paletteMemory.Memory; + this.paletteContexts = new Av1NeighborArrayUnit[tileCount]; + } + else + { + this.paletteContexts = []; + } + + Memory displacementVectors = Memory.Empty; + if (allocateMotionVectorState) + { + // Each component lies strictly inside plus or minus 16384. Two signed 16-bit fields preserve both + // inter and intra-block-copy vectors without expanding every compact mode-information entry. + ByteMemoryManager displacementVectorMemory = new( + stateStorage.Slice(displacementVectorStorageOffset, displacementVectorStorageLength)); + + displacementVectors = displacementVectorMemory.Memory; + } + + Av1IntraBlockCopySearchIndex intraBlockCopySearch = default; + if (allocateIntraBlockCopySearch) + { + // The search index casts its packed workspace to 32-bit links, so its non-owning region begins at + // a four-byte boundary inside the existing picture-state rent. + intraBlockCopySearch = new Av1IntraBlockCopySearchIndex( + stateStorage.Slice(intraBlockCopySearchStorageOffset, intraBlockCopySearchStorageLength), + width, + height); + } + + ByteMemoryManager tileStateMemory = new( + stateStorage.Slice(tileStateStorageOffset, tileStateStorageLength)); + + Memory tileState = tileStateMemory.Memory; + Memory cdefPreset = tileState[..cdefPresetLength]; + Memory previousQIndex = tileState.Slice(cdefPresetLength, tileCount); + Memory tileDataOffsets = tileState.Slice(cdefPresetLength + tileCount, tileCount); + Memory tileDataLengths = tileState.Slice(cdefPresetLength + (2 * tileCount), tileCount); + cdefPreset.Span.Fill(-1); + for (int tileIndex = 0; tileIndex < tileCount; tileIndex++) + { + this.partitionContexts[tileIndex] = new Av1NeighborArrayUnit( + partitionStorage.Slice(tileIndex * lumaContextLength, lumaContextLength), + lumaLeftLength, + lumaTopLength) + { + GranularityNormalLog2 = Av1Constants.ModeInfoSizeLog2 + }; + + int byteContextOffset = tileIndex * byteContextLengthPerTile; + this.lumaCoefficientContexts[tileIndex] = new Av1NeighborArrayUnit( + byteContextStorage.Slice(byteContextOffset, lumaContextLength), + lumaLeftLength, + lumaTopLength) + { + GranularityNormalLog2 = Av1Constants.ModeInfoSizeLog2 + }; + + byteContextOffset += lumaContextLength; + this.blueCoefficientContexts[tileIndex] = new Av1NeighborArrayUnit( + byteContextStorage.Slice(byteContextOffset, chromaContextLength), + chromaLeftLength, + chromaTopLength) + { + GranularityNormalLog2 = Av1Constants.ModeInfoSizeLog2 + }; + + byteContextOffset += chromaContextLength; + this.redCoefficientContexts[tileIndex] = new Av1NeighborArrayUnit( + byteContextStorage.Slice(byteContextOffset, chromaContextLength), + chromaLeftLength, + chromaTopLength) + { + GranularityNormalLog2 = Av1Constants.ModeInfoSizeLog2 + }; + + byteContextOffset += chromaContextLength; + this.transformContexts[tileIndex] = new Av1NeighborArrayUnit( + byteContextStorage.Slice(byteContextOffset, lumaContextLength), + lumaLeftLength, + lumaTopLength) + { + GranularityNormalLog2 = Av1Constants.ModeInfoSizeLog2 + }; + + // Variable-transform contexts consult both edges without separate availability flags. The largest + // transform makes an unavailable edge compare as unsplit until a coded neighbor publishes its size. + this.transformContexts[tileIndex].Left.Fill((byte)Av1Constants.MaxTransformSize); + this.transformContexts[tileIndex].Top.Fill((byte)Av1Constants.MaxTransformSize); + + if (allocateScreenContentState) + { + this.paletteContexts[tileIndex] = new Av1NeighborArrayUnit( + paletteStorage.Slice(tileIndex * paletteContextLength, paletteContextLength), + paletteLeftLength, + paletteTopLength) + { + GranularityNormalLog2 = Av1Constants.ModeInfoSizeLog2 + }; + } + + previousQIndex.Span[tileIndex] = frameHeader.QuantizationParameters.BaseQIndex; + } + + this.Picture = new Av1PictureControlSet + { + PartitionContexts = this.partitionContexts, + LuminanceDcSignLevelCoefficientNeighbors = this.lumaCoefficientContexts, + CbDcSignLevelCoefficientNeighbors = this.blueCoefficientContexts, + CrDcSignLevelCoefficientNeighbors = this.redCoefficientContexts, + TransformFunctionContexts = this.transformContexts, + PaletteContexts = this.paletteContexts, + Sequence = new Av1SequenceControlSet { SequenceHeader = sequenceHeader }, + Parent = new Av1PictureParentControlSet + { + Common = new Av1EncoderCommon + { + ModeInfoColumnCount = this.modeInfo.ModeInfoColumnCount, + ModeInfoRowCount = this.modeInfo.ModeInfoRowCount, + ModeInfoStride = this.modeInfo.ModeInfoStride, + FrameSize = frameHeader.FrameSize, + TilesInfo = frameHeader.TilesInfo + }, + FrameHeader = frameHeader, + PreviousQIndex = previousQIndex + }, + SegmentationNeighborMap = stateStorage[..segmentationLength], + ModeInfoGrid = this.modeInfo.Grid, + ModeInfoAllocation = this.modeInfo.Allocation, + DisplacementVectors = displacementVectors, + IntraBlockCopySearch = intraBlockCopySearch, + ModeInfoStride = this.modeInfo.ModeInfoStride, + Disallow4x4AllFrames = this.modeInfo.Disallow4x4AllFrames, + CdefPreset = cdefPreset, + TileDataOffsets = tileDataOffsets, + TileDataLengths = tileDataLengths + }; + } + catch + { + // The context objects only borrow these two owners. A failed constructor must release the + // completed allocations itself because the enclosing sequence never receives this picture. + this.stateStorage?.Dispose(); + this.modeInfo.Dispose(); + throw; + } + } + + /// + /// Gets the non-owning picture state consumed by superblock analysis and tile writing. + /// + public Av1PictureControlSet Picture { get; } + + /// + /// Restores clean per-frame state while retaining every fixed-geometry allocation. + /// + /// The frame header consumed by the next encoding pass. + public void Reset(ObuFrameHeader frameHeader) + { + this.modeInfo.Grid.Span.Clear(); + this.modeInfo.Allocation.Span.Clear(); + this.stateMemory.Span.Clear(); + + // Transform contexts begin at the largest transform size until an encoded neighbor publishes its + // selected size. This sentinel must be restored after the packed state owner is cleared. + foreach (Av1NeighborArrayUnit context in this.transformContexts) + { + context.Left.Fill((byte)Av1Constants.MaxTransformSize); + context.Top.Fill((byte)Av1Constants.MaxTransformSize); + } + + this.Picture.CdefPreset.Span.Fill(-1); + this.Picture.Parent.PreviousQIndex.Span.Fill(frameHeader.QuantizationParameters.BaseQIndex); + this.Picture.Parent.FrameHeader = frameHeader; + this.Picture.Parent.Common.FrameSize = frameHeader.FrameSize; + this.Picture.Parent.Common.TilesInfo = frameHeader.TilesInfo; + } + + /// + /// Returns every picture-lifetime allocation to the configured allocator. + /// + public void Dispose() + { + foreach (Av1NeighborArrayUnit context in this.partitionContexts) + { + context.Dispose(); + } + + foreach (Av1NeighborArrayUnit context in this.paletteContexts) + { + context.Dispose(); + } + + for (int tileIndex = 0; tileIndex < this.lumaCoefficientContexts.Length; tileIndex++) + { + this.lumaCoefficientContexts[tileIndex].Dispose(); + this.blueCoefficientContexts[tileIndex].Dispose(); + this.redCoefficientContexts[tileIndex].Dispose(); + this.transformContexts[tileIndex].Dispose(); + } + + this.stateStorage.Dispose(); + this.modeInfo.Dispose(); + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1EncoderPredictionUnit.cs b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1EncoderPredictionUnit.cs new file mode 100644 index 0000000000..eaf17068e1 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1EncoderPredictionUnit.cs @@ -0,0 +1,35 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Diagnostics.CodeAnalysis; +using System.Runtime.InteropServices; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +/// +/// Stores encoder-selected intra prediction modes and directional-angle adjustments for one block. +/// +[StructLayout(LayoutKind.Sequential, Pack = 1, Size = 4)] +internal struct Av1EncoderPredictionUnit +{ + /// + /// Stores the signed luma and chroma directional adjustments. + /// + private InlineArray2 angleDelta; + + /// + /// Gets the directional angle adjustment for each prediction plane. + /// + [UnscopedRef] + public Span AngleDelta => this.angleDelta; + + /// + /// Gets or sets the chroma-from-luma alpha magnitude index. + /// + public byte ChromaFromLumaIndex { get; set; } + + /// + /// Gets or sets the packed chroma-from-luma alpha signs for the U and V planes. + /// + public sbyte ChromaFromLumaSigns { get; set; } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1EncoderSuperblockWorkspace.cs b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1EncoderSuperblockWorkspace.cs new file mode 100644 index 0000000000..34238e3d5d --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1EncoderSuperblockWorkspace.cs @@ -0,0 +1,115 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Buffers; +using System.Runtime.InteropServices; +using SixLabors.ImageSharp.Formats.Heif.Av1.Motion; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +/// +/// Owns the reusable final-block and partition decisions for one AV1 superblock. +/// +internal sealed class Av1EncoderSuperblockWorkspace : IDisposable +{ + /// + /// The maximum number of 4x4 final blocks in a 128x128 superblock. + /// + public const int MaximumFinalBlockCount = 1 << (2 * (Av1Constants.MaxSuperBlockSizeLog2 - Av1Constants.ModeInfoSizeLog2)); + + /// + /// The maximum number of partition nodes in the complete 128x128 through 8x8 quadtree. + /// + public const int MaximumPartitionCount = 1 + 4 + 16 + 64 + 256; + + /// + /// The decision-region length in packed final-block storage elements. + /// + public const int DecisionStorageLength = MaximumFinalBlockCount + ((MaximumPartitionCount + Av1EncoderBlockStruct.StorageSize - 1) / Av1EncoderBlockStruct.StorageSize); + + /// + /// The byte length of the aligned final-block and partition decision region. + /// + public const int DecisionStorageByteLength = DecisionStorageLength * Av1EncoderBlockStruct.StorageSize; + + /// + /// The complete byte length of the decision and palette-map regions. + /// + public const int StorageByteLength = DecisionStorageByteLength + Av1EncoderPaletteMapBuffer.StorageLength; + + private const int PartitionStorageOffset = MaximumFinalBlockCount * Av1EncoderBlockStruct.StorageSize; + + private readonly IMemoryOwner owner; + private readonly Av1EncoderPaletteMapBuffer paletteMaps; + private Av1EncoderPaletteInfo paletteInfo; + private Av1ReferenceMotionVectors referenceMotionVectors; + + /// + /// Initializes a new instance of the class. + /// + /// The configuration providing the encoder allocator. + public Av1EncoderSuperblockWorkspace(Configuration configuration) + { + this.owner = configuration.MemoryAllocator.Allocate(StorageByteLength); + Memory storage = this.owner.Memory[..StorageByteLength]; + + // Decisions and palette maps have the same serial superblock lifetime. Keeping both regions in one + // owner preserves their distinct layouts while removing a separate palette allocation and cleanup path. + this.paletteMaps = new Av1EncoderPaletteMapBuffer( + storage.Slice(DecisionStorageByteLength, Av1EncoderPaletteMapBuffer.StorageLength)); + + this.Reset(); + } + + /// + /// Gets the maximum-size final-block decision span in partition traversal order. + /// + public Span FinalBlocks + => MemoryMarshal.Cast(this.owner.Memory.Span[..DecisionStorageByteLength])[..MaximumFinalBlockCount]; + + /// + /// Gets the maximum-size partition-type span in partition-tree preorder. + /// + public Span PartitionTypes => this.owner.Memory.Span.Slice(PartitionStorageOffset, MaximumPartitionCount); + + /// + /// Gets the palette sizes and colors selected for the block currently being written. + /// + public ref Av1EncoderPaletteInfo PaletteInfo => ref this.paletteInfo; + + /// + /// Gets the reusable reference-vector stack used while writing inter syntax. + /// + public ref Av1ReferenceMotionVectors ReferenceMotionVectors => ref this.referenceMotionVectors; + + /// + /// Gets the reusable palette maps within the superblock-workspace owner. + /// + /// The reusable luma and chroma palette maps. + public Av1EncoderPaletteMapBuffer GetPaletteMaps() => this.paletteMaps; + + /// + /// Clears all decisions before the workspace is reused for another superblock. + /// + public void Reset() + { + // Zero selects a real filter-intra kernel, so each cleared block must carry the disabled sentinel explicitly. + Av1EncoderBlockStruct initialBlock = new() + { + FilterIntraMode = Av1FilterIntraMode.AllFilterIntraModes + }; + + this.FinalBlocks.Fill(initialBlock); + this.PartitionTypes.Clear(); + this.paletteInfo = default; + } + + /// + /// Releases the reusable superblock workspace. + /// + public void Dispose() + { + this.paletteMaps.Dispose(); + this.owner.Dispose(); + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1EncoderTransformBlockState.cs b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1EncoderTransformBlockState.cs new file mode 100644 index 0000000000..bab0da76be --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1EncoderTransformBlockState.cs @@ -0,0 +1,42 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.InteropServices; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +/// +/// Stores the entropy syntax retained for one AV1 transform block. +/// +[StructLayout(LayoutKind.Sequential, Size = sizeof(int))] +internal struct Av1EncoderTransformBlockState +{ + /// + /// Stores the position after the final nonzero coefficient. + /// + private ushort endOfBlock; + + /// + /// Stores the selected transform type. + /// + private Av1TransformType transformType; + + /// + /// Gets or sets the position after the final nonzero coefficient. + /// + public ushort EndOfBlock + { + readonly get => this.endOfBlock; + set => this.endOfBlock = value; + } + + /// + /// Gets or sets the selected transform type. + /// + public Av1TransformType TransformType + { + readonly get => this.transformType; + set => this.transformType = value; + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1EntropyCodingContext.cs b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1EntropyCodingContext.cs new file mode 100644 index 0000000000..d61947a7c5 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1EntropyCodingContext.cs @@ -0,0 +1,41 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +/// +/// Provides tile-writer operations that maintain encoder entropy-neighbor state. +/// +internal partial class Av1TileWriter +{ + /// + /// Tracks the mode and coefficient positions while entropy-coding one AV1 superblock. + /// + internal class Av1EntropyCodingContext + { + /// + /// Gets the reusable macroblock edge and neighbor state for the current entropy-coding operation. + /// + public required Av1MacroBlockD MacroBlock { get; init; } + + /// + /// Gets or sets the macroblock mode information currently being encoded. + /// + public required Av1MacroBlockModeInfo MacroBlockModeInfo { get; set; } + + /// + /// Gets or sets the pixel origin of the current superblock. + /// + public Point SuperblockOrigin { get; set; } + + /// + /// Gets or sets the number of luma coefficient positions consumed in the current superblock. + /// + public int CodedAreaSuperblock { get; set; } + + /// + /// Gets or sets the number of chroma coefficient positions consumed in the current superblock. + /// + public int CodedAreaSuperblockUv { get; set; } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1FilterIntraMode.cs b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1FilterIntraMode.cs new file mode 100644 index 0000000000..17c08e5acc --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1FilterIntraMode.cs @@ -0,0 +1,40 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +/// +/// Identifies the filter-intra predictor kernel selected for an AV1 block. +/// +internal enum Av1FilterIntraMode +{ + /// + /// The filter-intra DC predictor. + /// + DC, + + /// + /// The filter-intra vertical predictor. + /// + Vertical, + + /// + /// The filter-intra horizontal predictor. + /// + Horizontal, + + /// + /// The filter-intra 157-degree directional predictor. + /// + Directional157, + + /// + /// The filter-intra Paeth predictor. + /// + Paeth, + + /// + /// The number of filter-intra modes. + /// + AllFilterIntraModes, +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1FilterIntraModeExtensions.cs b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1FilterIntraModeExtensions.cs new file mode 100644 index 0000000000..c27af0242b --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1FilterIntraModeExtensions.cs @@ -0,0 +1,26 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +/// +/// Provides validity checks for AV1 filter-intra modes. +/// +internal static class Av1FilterIntraModeExtensions +{ + /// + /// Maps filter-intra syntax values to the directional mode used by the predictor. + /// + private static readonly Av1PredictionMode[] IntraDirection = + [Av1PredictionMode.DC, Av1PredictionMode.Vertical, Av1PredictionMode.Horizontal, Av1PredictionMode.Directional157Degrees, Av1PredictionMode.DC]; + + /// + /// Gets the intra-prediction direction associated with the specified filter-intra mode. + /// + /// The filter-intra mode. + /// The corresponding intra-prediction direction. + public static Av1PredictionMode ToIntraDirection(this Av1FilterIntraMode mode) + => IntraDirection[(int)mode]; +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1FrameInfo.MotionField.cs b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1FrameInfo.MotionField.cs new file mode 100644 index 0000000000..37d55bc3da --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1FrameInfo.MotionField.cs @@ -0,0 +1,855 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Buffers; +using SixLabors.ImageSharp.Formats.Heif.Av1.Motion; +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Formats.Heif.Av1.ReferenceFrames; +using SixLabors.ImageSharp.Memory; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +/// +/// Owns the retained and projected per-8x8 motion fields associated with a decoded AV1 frame. +/// +internal partial class Av1FrameInfo +{ + /// + /// The maximum number of reference frames projected into one temporal motion field. + /// + private const int MotionFieldProjectionCount = 3; + + /// + /// The maximum source motion-vector magnitude retained for later temporal projection. + /// + private const int ReferenceMotionVectorLimit = 4095; + + /// + /// The width or height of the largest AV1 superblock in 4x4 mode-information units. + /// + private const int MaximumSuperblockModeInfoSize = 1 << (Av1Constants.MaxSuperBlockSizeLog2 - Av1Constants.ModeInfoSizeLog2); + + /// + /// The base-two logarithm of . + /// + private const int MaximumSuperblockModeInfoSizeLog2 = Av1Constants.MaxSuperBlockSizeLog2 - Av1Constants.ModeInfoSizeLog2; + + /// + /// The base-two reduction from 4x4 mode-information coordinates to the 8x8 motion-field grid. + /// + private const int MotionFieldModeInfoShift = 1; + + /// + /// The base-two reduction from one-eighth-sample motion vectors to offsets on the 8x8 motion-field grid. + /// + private const int MotionVectorToFieldOffsetShift = 4 + Av1Constants.ModeInfoSizeLog2; + + /// + /// The maximum horizontal projection displacement, measured in 8x8 motion-field blocks. + /// + private const int MaximumHorizontalFieldOffset = 8; + + /// + /// Owns the selected motion vector and logical reference for every retained 8x8 frame position. + /// + private MotionFieldStorage? retainedMotionField; + + /// + /// Owns motion vectors projected from retained frames into the current frame's 8x8 grid. + /// + private MotionFieldStorage? temporalMotionField; + + /// + /// The compact reference state detached from reconstruction storage after this frame refreshes the reference map. + /// + private ReferenceState? referenceState; + + /// + /// The number of tile-reader and decoder-result owners retaining this reconstruction state. + /// + private int ownerCount = 1; + + /// + /// Indicates whether this frame state still owns the initial lease created with the instance. + /// + private bool ownsInitialLease = true; + + /// + /// Stores the order hint selected by each logical inter-reference type for later projections from this frame. + /// + private InlineArray8 motionFieldReferenceOrderHints; + + /// + /// Stores whether each logical inter-reference type lies after, at, or before the current frame in display order. + /// + private InlineArray8 motionFieldReferenceSides; + + /// + /// The order hint of the current frame represented by this mode-information owner. + /// + private uint motionFieldOrderHint; + + /// + /// The active frame width in 4x4 mode-information units. + /// + private int activeModeInfoColumnCount; + + /// + /// The active frame height in 4x4 mode-information units. + /// + private int activeModeInfoRowCount; + + /// + /// Allocates and derives the motion fields required by one decoded frame. + /// + /// The decoder configuration providing motion-field storage. + /// The sequence header defining motion-field enablement and order-hint precision. + /// The current frame header and its seven resolved inter-reference roles. + /// The retained reconstructed frames selected by the current reference map. + public void InitializeMotionField( + Configuration configuration, + ObuSequenceHeader sequenceHeader, + ObuFrameHeader frameHeader, + Av1ReferenceFrameStore referenceFrames) + { + if (!sequenceHeader.OrderHintInfo.EnableOrderHint) + { + return; + } + + this.motionFieldOrderHint = frameHeader.OrderHint; + this.activeModeInfoColumnCount = frameHeader.ModeInfoColumnCount; + this.activeModeInfoRowCount = frameHeader.ModeInfoRowCount; + int retainedMotionFieldStride = (this.activeModeInfoColumnCount + 1) >> MotionFieldModeInfoShift; + + if (frameHeader.IsIntra) + { + // Intra frames retain an empty source field. They can occupy reference slots, but the reference decoder rejects them as + // projection sources before consulting their reference-order-hint metadata. + return; + } + + ReadOnlySpan referenceFrameIndices = frameHeader.GetReferenceFrameIndices(); + SelectedReferenceFrames selectedReferences = new(referenceFrames, referenceFrameIndices); + ObuOrderHintInfo orderHintInfo = sequenceHeader.OrderHintInfo; + + // Capture the seven logical-role order hints before this frame refreshes any physical map slots. The reference decoder keeps + // the same snapshot on RefCntBuffer so a later frame can project this frame's stored motion vectors. + for (int referenceIndex = 0; referenceIndex < Av1Constants.ReferencesPerFrame; referenceIndex++) + { + Av1ReferenceFrameType referenceFrameType = (Av1ReferenceFrameType)(referenceIndex + 1); + Av1ReferenceFrame referenceFrame = selectedReferences[referenceFrameType]; + uint referenceOrderHint = referenceFrame.FrameHeader.OrderHint; + this.motionFieldReferenceOrderHints[(int)referenceFrameType] = referenceOrderHint; + + int relativeDistance = orderHintInfo.GetRelativeDistance(referenceOrderHint, frameHeader.OrderHint); + this.motionFieldReferenceSides[(int)referenceFrameType] = relativeDistance > 0 + ? (sbyte)1 + : referenceOrderHint == frameHeader.OrderHint ? (sbyte)-1 : (sbyte)0; + } + + if (!sequenceHeader.OrderHintInfo.EnableReferenceFrameMotionVectors) + { + // Spatial reference extension still needs the sign classification above when temporal motion fields are + // disabled. Retained and projected 8x8 storage belongs exclusively to the temporal-motion-vector tool. + return; + } + + // The retained field transfers to compact reference state after reconstruction. Keeping it allocator-backed + // avoids placing a frame-sized array on the managed heap. Clean storage is required because an all-zero entry + // denotes the normative empty field. + int retainedRowCount = (this.activeModeInfoRowCount + 1) >> MotionFieldModeInfoShift; + IMemoryOwner retainedMotionFieldOwner = + configuration.MemoryAllocator.Allocate( + retainedMotionFieldStride * retainedRowCount, + AllocationOptions.Clean); + + this.retainedMotionField = new(retainedMotionFieldOwner, retainedMotionFieldStride); + + if (!frameHeader.UseReferenceFrameMotionVectors) + { + return; + } + + // the reference decoder aligns the projected field stride to the largest superblock even for a 64x64 sequence. This keeps + // later temporal-candidate addressing independent of the current sequence's selected superblock size. + int alignedModeInfoColumnCount = Av1Math.AlignPowerOf2( + this.activeModeInfoColumnCount, + MaximumSuperblockModeInfoSizeLog2); + + int temporalMotionFieldStride = alignedModeInfoColumnCount >> MotionFieldModeInfoShift; + int temporalRowCount = (this.activeModeInfoRowCount + MaximumSuperblockModeInfoSize) >> MotionFieldModeInfoShift; + IMemoryOwner temporalMotionFieldOwner = + configuration.MemoryAllocator.Allocate( + temporalMotionFieldStride * temporalRowCount, + AllocationOptions.Clean); + + MotionFieldStorage temporalMotionField = + new(temporalMotionFieldOwner, temporalMotionFieldStride); + + this.temporalMotionField = temporalMotionField; + + // AV1 examines LAST, BWDREF, ALTREF2, ALTREF, and LAST2 in this normative order and admits at most three + // projection sources. LAST always consumes the first budget position, forward references consume one only + // when eligible projection succeeds, and LAST2 fills the final unused position in the reverse direction. + int remainingProjectionCount = MotionFieldProjectionCount; + Av1ReferenceFrame lastFrame = selectedReferences[Av1ReferenceFrameType.Last]; + Av1ReferenceFrame goldenFrame = selectedReferences[Av1ReferenceFrameType.Golden]; + uint alternateOfLastOrderHint = + lastFrame.ReferenceState.MotionFieldReferenceOrderHints[(int)Av1ReferenceFrameType.Alternate]; + + // A LAST frame whose ALTREF order matches GOLDEN is an overlay. Projecting it would duplicate the overlay's + // temporal source, but the reference decoder still consumes one position from the three-source projection budget. + if (alternateOfLastOrderHint != goldenFrame.FrameHeader.OrderHint) + { + _ = this.ProjectMotionField(sequenceHeader, frameHeader, lastFrame, temporalMotionField, reverseDirection: true); + } + + remainingProjectionCount--; + Av1ReferenceFrame backwardFrame = selectedReferences[Av1ReferenceFrameType.Backward]; + + if (orderHintInfo.GetRelativeDistance(backwardFrame.FrameHeader.OrderHint, frameHeader.OrderHint) > 0 && + this.ProjectMotionField(sequenceHeader, frameHeader, backwardFrame, temporalMotionField, reverseDirection: false)) + { + remainingProjectionCount--; + } + + Av1ReferenceFrame alternate2Frame = selectedReferences[Av1ReferenceFrameType.Alternate2]; + + if (orderHintInfo.GetRelativeDistance(alternate2Frame.FrameHeader.OrderHint, frameHeader.OrderHint) > 0 && + this.ProjectMotionField(sequenceHeader, frameHeader, alternate2Frame, temporalMotionField, reverseDirection: false)) + { + remainingProjectionCount--; + } + + Av1ReferenceFrame alternateFrame = selectedReferences[Av1ReferenceFrameType.Alternate]; + + if (remainingProjectionCount > 0 && + orderHintInfo.GetRelativeDistance(alternateFrame.FrameHeader.OrderHint, frameHeader.OrderHint) > 0 && + this.ProjectMotionField(sequenceHeader, frameHeader, alternateFrame, temporalMotionField, reverseDirection: false)) + { + remainingProjectionCount--; + } + + if (remainingProjectionCount > 0) + { + Av1ReferenceFrame last2Frame = selectedReferences[Av1ReferenceFrameType.Last2]; + _ = this.ProjectMotionField(sequenceHeader, frameHeader, last2Frame, temporalMotionField, reverseDirection: true); + } + } + + /// + /// Gets the temporal motion vector projected over a 4x4 mode-information position. + /// + /// The zero-based 4x4 row. + /// The zero-based 4x4 column. + /// Receives the retained source vector in one-eighth-sample units. + /// Receives the positive temporal distance from the source to its reference. + /// when a projected vector covers the requested position. + public bool TryGetTemporalMotionVector( + int modeInfoRow, + int modeInfoColumn, + out Av1MotionVector motionVector, + out int referenceFrameOffset) + { + MotionFieldStorage? temporalMotionFieldState = this.temporalMotionField; + if (temporalMotionFieldState is null) + { + motionVector = default; + referenceFrameOffset = 0; + return false; + } + + MotionFieldStorage temporalMotionField = temporalMotionFieldState.Value; + int index = ((modeInfoRow >> MotionFieldModeInfoShift) * temporalMotionField.Stride) + + (modeInfoColumn >> MotionFieldModeInfoShift); + + TemporalMotionFieldEntry entry = temporalMotionField.Owner.Memory.Span[index]; + motionVector = entry.MotionVector; + referenceFrameOffset = entry.ReferenceFrameOffset; + return referenceFrameOffset > 0; + } + + /// + /// Gets a value indicating whether a canonical inter reference has positive AV1 sign bias. + /// + /// The canonical inter-reference role. + /// for a future reference; otherwise, . + public bool IsReferenceSignBiased(Av1ReferenceFrameType referenceFrame) => this.motionFieldReferenceSides[(int)referenceFrame] > 0; + + /// + /// Gets a temporal motion-field entry projected to a selected canonical inter reference. + /// + /// The zero-based 4x4 row. + /// The zero-based 4x4 column. + /// The canonical inter-reference role targeted by the candidate. + /// The sequence modulo order-hint configuration. + /// A value indicating whether one-eighth-sample precision may be retained. + /// A value indicating whether integer-sample precision is required. + /// Receives the projected and precision-reduced candidate. + /// when a temporal entry covers the requested position. + public bool TryGetProjectedTemporalMotionVector( + int modeInfoRow, + int modeInfoColumn, + Av1ReferenceFrameType referenceFrame, + ObuOrderHintInfo orderHintInfo, + bool allowHighPrecision, + bool forceInteger, + out Av1MotionVector motionVector) + { + if (!this.TryGetTemporalMotionVector( + modeInfoRow, + modeInfoColumn, + out Av1MotionVector sourceMotionVector, + out int sourceReferenceOffset)) + { + motionVector = default; + return false; + } + + int targetReferenceOffset = orderHintInfo.GetRelativeDistance( + this.motionFieldOrderHint, + this.motionFieldReferenceOrderHints[(int)referenceFrame]); + + // The projected field retains the source frame's original vector and its source-to-reference distance. + // Candidate construction therefore applies the second normative ratio for the current frame's selected role. + motionVector = sourceMotionVector + .ProjectTemporal(targetReferenceOffset, sourceReferenceOffset) + .LowerPrecision(allowHighPrecision, forceInteger); + + return true; + } + + /// + /// Writes the retained per-8x8 motion-field entries covered by one completed mode-information block. + /// + /// The completed block mode information. + /// The block origin in frame-relative 4x4 units. + private void UpdateRetainedMotionField(Av1BlockModeInfo modeInfo, Point modeInfoPosition) + { + MotionFieldStorage? retainedMotionFieldState = this.retainedMotionField; + if (retainedMotionFieldState is null) + { + return; + } + + MotionFieldStorage retainedMotionField = retainedMotionFieldState.Value; + Span retainedEntries = retainedMotionField.Owner.Memory.Span; + + Av1ReferenceFrameType selectedReference = Av1ReferenceFrameType.None; + Av1MotionVector selectedMotionVector = default; + Span referenceFrames = modeInfo.ReferenceFrames; + Span motionVectors = modeInfo.MotionVectors; + + // Compound blocks may offer two vectors. the reference decoder retains the last eligible forward-or-past reference after + // excluding same-order, future, and out-of-range vectors, so preserve that overwrite order exactly. + for (int referenceIndex = 0; referenceIndex < 2; referenceIndex++) + { + Av1ReferenceFrameType referenceFrame = referenceFrames[referenceIndex]; + Av1MotionVector motionVector = motionVectors[referenceIndex]; + + if (referenceFrame > Av1ReferenceFrameType.Intra && + this.motionFieldReferenceSides[(int)referenceFrame] == 0 && + Math.Abs(motionVector.Row) <= ReferenceMotionVectorLimit && + Math.Abs(motionVector.Column) <= ReferenceMotionVectorLimit) + { + selectedReference = referenceFrame; + selectedMotionVector = motionVector; + } + } + + int blockModeInfoWidth = Math.Min( + modeInfo.BlockSize.Get4x4WideCount(), + this.activeModeInfoColumnCount - modeInfoPosition.X); + + int blockModeInfoHeight = Math.Min( + modeInfo.BlockSize.Get4x4HighCount(), + this.activeModeInfoRowCount - modeInfoPosition.Y); + + int fieldWidth = (blockModeInfoWidth + 1) >> MotionFieldModeInfoShift; + int fieldHeight = (blockModeInfoHeight + 1) >> MotionFieldModeInfoShift; + int firstFieldRow = modeInfoPosition.Y >> MotionFieldModeInfoShift; + int firstFieldColumn = modeInfoPosition.X >> MotionFieldModeInfoShift; + RetainedMotionFieldEntry entry = new(selectedMotionVector, selectedReference); + + // One decoded block supplies the same retained candidate to every covered 8x8 cell. Filling each contiguous + // row lets the runtime select its optimized span implementation while later sub-8x8 blocks retain the + // normative ability to overwrite the shared cell in traversal order. + for (int row = 0; row < fieldHeight; row++) + { + int rowOffset = ((firstFieldRow + row) * retainedMotionField.Stride) + firstFieldColumn; + retainedEntries.Slice(rowOffset, fieldWidth).Fill(entry); + } + } + + /// + /// Projects one retained frame's motion field into the current frame's temporal candidate grid. + /// + /// The sequence header defining the modulo order-hint domain. + /// The current frame header. + /// The retained frame whose stored motion vectors are projected. + /// The complete destination field storage for the current frame. + /// + /// A value indicating whether the start-to-current distance and spatial displacement are reversed for a past frame. + /// + /// when the retained frame is an eligible projection source. + private bool ProjectMotionField( + ObuSequenceHeader sequenceHeader, + ObuFrameHeader frameHeader, + Av1ReferenceFrame startFrame, + MotionFieldStorage temporalMotionField, + bool reverseDirection) + { + ObuFrameHeader startFrameHeader = startFrame.FrameHeader; + if (startFrameHeader.IsIntra || + startFrameHeader.ModeInfoRowCount != this.activeModeInfoRowCount || + startFrameHeader.ModeInfoColumnCount != this.activeModeInfoColumnCount) + { + // AV1 does not rescale temporal fields. Intra sources contain no inter motion, and a differently sized + // source has no one-to-one 8x8 grid on which the normative projection can operate. + return false; + } + + ReferenceState startFrameState = startFrame.ReferenceState; + ObuOrderHintInfo orderHintInfo = sequenceHeader.OrderHintInfo; + int startToCurrentFrameOffset = orderHintInfo.GetRelativeDistance( + startFrameHeader.OrderHint, + frameHeader.OrderHint); + + if (reverseDirection) + { + startToCurrentFrameOffset = -startToCurrentFrameOffset; + } + + int sourceRowCount = (this.activeModeInfoRowCount + 1) >> MotionFieldModeInfoShift; + int sourceColumnCount = (this.activeModeInfoColumnCount + 1) >> MotionFieldModeInfoShift; + int destinationRowCount = this.activeModeInfoRowCount >> MotionFieldModeInfoShift; + int destinationColumnCount = this.activeModeInfoColumnCount >> MotionFieldModeInfoShift; + MotionFieldStorage? retainedMotionFieldState = startFrameState.RetainedMotionField; + if (retainedMotionFieldState is null) + { + return false; + } + + MotionFieldStorage retainedMotionField = retainedMotionFieldState.Value; + + ReadOnlySpan sourceEntries = retainedMotionField.Owner.Memory.Span; + Span destinationEntries = temporalMotionField.Owner.Memory.Span; + + for (int blockRow = 0; blockRow < sourceRowCount; blockRow++) + { + int sourceRowOffset = blockRow * retainedMotionField.Stride; + for (int blockColumn = 0; blockColumn < sourceColumnCount; blockColumn++) + { + RetainedMotionFieldEntry source = sourceEntries[sourceRowOffset + blockColumn]; + if (source.ReferenceFrame <= Av1ReferenceFrameType.Intra) + { + continue; + } + + int referenceFrameOffset = orderHintInfo.GetRelativeDistance( + startFrameHeader.OrderHint, + startFrameState.MotionFieldReferenceOrderHints[(int)source.ReferenceFrame]); + + bool positionIsValid = Math.Abs(referenceFrameOffset) <= Av1MotionVector.MaximumTemporalDistance && + referenceFrameOffset > 0 && + Math.Abs(startToCurrentFrameOffset) <= Av1MotionVector.MaximumTemporalDistance; + + if (!positionIsValid) + { + continue; + } + + Av1MotionVector projected = source.MotionVector.ProjectTemporal(startToCurrentFrameOffset, referenceFrameOffset); + + if (!TryGetProjectedBlockPosition( + blockRow, + blockColumn, + projected, + reverseDirection, + destinationRowCount, + destinationColumnCount, + out int projectedRow, + out int projectedColumn)) + { + continue; + } + + // The projected vector selects the destination cell, but AV1 stores the original forward vector and + // its source-to-reference distance there. Candidate scaling later uses both values for its own target. + int destinationOffset = (projectedRow * temporalMotionField.Stride) + projectedColumn; + destinationEntries[destinationOffset] = new(source.MotionVector, referenceFrameOffset); + } + } + + return true; + } + + /// + /// Detaches the compact state required while this decoded frame occupies the reference map. + /// + public ReferenceState PrepareReferenceState() + { + ReferenceState? state = this.referenceState; + if (state is null) + { + // libaom's RefCntBuffer retains only the segment map, 8x8 motion field, and reference-order hints from + // block reconstruction. Transfer those owners without copying and leave all other frame syntax local. + state = new ReferenceState( + this.segmentIds, + this.segmentIdColumnCount, + this.segmentIdRowCount, + this.retainedMotionField, + this.motionFieldReferenceOrderHints); + + this.segmentIds = null; + this.retainedMotionField = null; + this.referenceState = state; + } + + return state; + } + + /// + /// Acquires the prepared compact state for one reference-frame owner. + /// + /// The retained reference state with one ownership lease for the caller. + public ReferenceState AcquireReferenceState() + { + ReferenceState state = this.PrepareReferenceState(); + + state.AddOwner(); + return state; + } + + /// + /// Adds one owner for this frame state. + /// + public void AddOwner() + { + // One decoder session serializes tile parsing, reference-map updates, and output transfer. A direct count is + // therefore sufficient and avoids both atomic operations and a separately allocated shared-owner object. + this.ownerCount++; + } + + /// + /// Releases the initial owner created with this frame state. + /// + public void Dispose() + { + if (this.ownsInitialLease) + { + // Av1TileReader can complete through both the OBU lifecycle and decoder cleanup. Keeping the initial lease + // idempotent lets either path dispose safely without affecting reference-frame or result-state owners. + this.ownsInitialLease = false; + this.ReleaseOwner(); + } + } + + /// + /// Releases one owner and returns allocator-backed frame storage after the final owner is released. + /// + public void ReleaseOwner() + { + this.ownerCount--; + if (this.ownerCount == 0) + { + // Reconstruction-only syntax expires with the tile reader and optional decoder-result owner. A detached + // reference state has its own lease and can outlive this full frame state. + this.referenceState?.ReleaseOwner(); + this.referenceState = null; + this.retainedMotionField?.Dispose(); + this.retainedMotionField = null; + this.temporalMotionField?.Dispose(); + this.temporalMotionField = null; + + for (int plane = 0; plane < Av1Constants.MaxPlanes; plane++) + { + this.loopRestorationUnits[plane]?.Dispose(); + } + + this.segmentIds?.Dispose(); + this.coefficientScratch.Dispose(); + this.deltaLoopFilter.Dispose(); + this.cdefStrength.Dispose(); + this.quantizerIndices.Dispose(); + this.transformInfoScratch.Dispose(); + this.modeInfoMap.Dispose(); + this.modeInfoCounts.Dispose(); + this.modeInfos.Dispose(); + } + } + + /// + /// Maps a projected motion vector to its bounded destination on the current 8x8 field. + /// + /// The source 8x8 row. + /// The source 8x8 column. + /// The temporally projected vector in one-eighth-sample units. + /// Whether the vector moves backwards from the source position. + /// The number of complete 8x8 rows in the current frame. + /// The number of complete 8x8 columns in the current frame. + /// Receives the projected 8x8 row. + /// Receives the projected 8x8 column. + /// when the destination lies in the permitted projection window. + private static bool TryGetProjectedBlockPosition( + int blockRow, + int blockColumn, + Av1MotionVector motionVector, + bool reverseDirection, + int rowCount, + int columnCount, + out int projectedRow, + out int projectedColumn) + { + int baseBlockRow = (blockRow >> 3) << 3; + int baseBlockColumn = (blockColumn >> 3) << 3; + + // One field cell spans 8 samples, while vectors use one-eighth-sample units; dividing by 64 converts between + // them. C# integer division truncates toward zero, matching the reference decoder's explicit signed-shift construction. + int rowOffset = motionVector.Row / (1 << MotionVectorToFieldOffsetShift); + int columnOffset = motionVector.Column / (1 << MotionVectorToFieldOffsetShift); + projectedRow = reverseDirection ? blockRow - rowOffset : blockRow + rowOffset; + projectedColumn = reverseDirection ? blockColumn - columnOffset : blockColumn + columnOffset; + + if (projectedRow < 0 || projectedRow >= rowCount || projectedColumn < 0 || projectedColumn >= columnCount) + { + return false; + } + + // AV1 keeps a projection in the same 64x64 row band and permits one additional 64-sample horizontal band on + // either side. This bounds temporal-candidate lookup while accommodating common lateral motion. + return projectedRow >= baseBlockRow && + projectedRow < baseBlockRow + 8 && + projectedColumn >= baseBlockColumn - MaximumHorizontalFieldOffset && + projectedColumn < baseBlockColumn + 8 + MaximumHorizontalFieldOffset; + } + + /// + /// Couples one allocator-owned motion-field buffer with the stride required to address it. + /// + internal readonly struct MotionFieldStorage + where T : struct + { + /// + /// Initializes a new instance of the struct. + /// + public MotionFieldStorage(IMemoryOwner owner, int stride) + { + this.Owner = owner; + this.Stride = stride; + } + + /// + /// Gets the allocator-owned field entries. + /// + public IMemoryOwner Owner { get; } + + /// + /// Gets the number of entries in one field row. + /// + public int Stride { get; } + + /// + /// Returns the field entries to their allocator. + /// + public void Dispose() => this.Owner.Dispose(); + } + + /// + /// Carries the complete seven-role reference mapping captured before the current frame refreshes map slots. + /// + private readonly struct SelectedReferenceFrames + { + /// + /// Initializes a new instance of the struct. + /// + public SelectedReferenceFrames(Av1ReferenceFrameStore referenceFrames, ReadOnlySpan referenceFrameIndices) + { + this.Last = referenceFrames.ResolveRequired((int)referenceFrameIndices[0]); + this.Last2 = referenceFrames.ResolveRequired((int)referenceFrameIndices[1]); + this.Last3 = referenceFrames.ResolveRequired((int)referenceFrameIndices[2]); + this.Golden = referenceFrames.ResolveRequired((int)referenceFrameIndices[3]); + this.Backward = referenceFrames.ResolveRequired((int)referenceFrameIndices[4]); + this.Alternate2 = referenceFrames.ResolveRequired((int)referenceFrameIndices[5]); + this.Alternate = referenceFrames.ResolveRequired((int)referenceFrameIndices[6]); + } + + /// + /// Gets the retained LAST frame. + /// + public Av1ReferenceFrame Last { get; } + + /// + /// Gets the retained LAST2 frame. + /// + public Av1ReferenceFrame Last2 { get; } + + /// + /// Gets the retained LAST3 frame. + /// + public Av1ReferenceFrame Last3 { get; } + + /// + /// Gets the retained GOLDEN frame. + /// + public Av1ReferenceFrame Golden { get; } + + /// + /// Gets the retained BWDREF frame. + /// + public Av1ReferenceFrame Backward { get; } + + /// + /// Gets the retained ALTREF2 frame. + /// + public Av1ReferenceFrame Alternate2 { get; } + + /// + /// Gets the retained ALTREF frame. + /// + public Av1ReferenceFrame Alternate { get; } + + /// + /// Gets the retained frame for one canonical inter-reference role. + /// + public Av1ReferenceFrame this[Av1ReferenceFrameType referenceFrame] => referenceFrame switch + { + Av1ReferenceFrameType.Last => this.Last, + Av1ReferenceFrameType.Last2 => this.Last2, + Av1ReferenceFrameType.Last3 => this.Last3, + Av1ReferenceFrameType.Golden => this.Golden, + Av1ReferenceFrameType.Backward => this.Backward, + Av1ReferenceFrameType.Alternate2 => this.Alternate2, + Av1ReferenceFrameType.Alternate => this.Alternate, + _ => throw new InvalidOperationException("Motion fields only use canonical inter-reference roles.") + }; + } + + /// + /// Stores one motion vector and logical reference retained for projection by a later frame. + /// + internal readonly struct RetainedMotionFieldEntry + { + /// + /// Initializes a new instance of the struct. + /// + /// The retained motion vector in one-eighth-sample units. + /// The logical reference targeted by the vector. + public RetainedMotionFieldEntry(Av1MotionVector motionVector, Av1ReferenceFrameType referenceFrame) + { + this.MotionVector = motionVector; + this.ReferenceFrame = referenceFrame; + } + + /// + /// Gets the retained motion vector in one-eighth-sample units. + /// + public Av1MotionVector MotionVector { get; } + + /// + /// Gets the logical reference targeted by . + /// + public Av1ReferenceFrameType ReferenceFrame { get; } + } + + /// + /// Stores one temporal candidate projected over the current frame's 8x8 grid. + /// + private readonly struct TemporalMotionFieldEntry + { + /// + /// Initializes a new instance of the struct. + /// + /// The retained source vector in one-eighth-sample units. + /// The positive temporal distance from the source to its reference. + public TemporalMotionFieldEntry(Av1MotionVector motionVector, int referenceFrameOffset) + { + this.MotionVector = motionVector; + this.ReferenceFrameOffset = referenceFrameOffset; + } + + /// + /// Gets the retained source vector in one-eighth-sample units. + /// + public Av1MotionVector MotionVector { get; } + + /// + /// Gets the positive temporal distance from the source frame to its reference. + /// + public int ReferenceFrameOffset { get; } + } + + /// + /// Owns only the per-frame syntax retained by libaom's reference buffer after reconstruction completes. + /// + internal sealed class ReferenceState + { + /// + /// The number of frame-info and reference-frame owners retaining this state. + /// + private int ownerCount = 1; + + /// + /// Initializes a new instance of the class by taking ownership of retained buffers. + /// + public ReferenceState( + Buffer2D? segmentIds, + int segmentIdColumnCount, + int segmentIdRowCount, + MotionFieldStorage? retainedMotionField, + InlineArray8 motionFieldReferenceOrderHints) + { + this.SegmentIds = segmentIds; + this.SegmentIdColumnCount = segmentIdColumnCount; + this.SegmentIdRowCount = segmentIdRowCount; + this.RetainedMotionField = retainedMotionField; + this.MotionFieldReferenceOrderHints = motionFieldReferenceOrderHints; + } + + /// + /// Gets the retained 4x4 segmentation map, or when segmentation is disabled. + /// + public Buffer2D? SegmentIds { get; private set; } + + /// + /// Gets the number of active 4x4 columns in . + /// + public int SegmentIdColumnCount { get; } + + /// + /// Gets the number of active 4x4 rows in . + /// + public int SegmentIdRowCount { get; } + + /// + /// Gets the retained per-8x8 motion field, or when the temporal tool is disabled. + /// + public MotionFieldStorage? RetainedMotionField { get; private set; } + + /// + /// Gets the order hints selected by this frame's seven logical inter-reference roles. + /// + public InlineArray8 MotionFieldReferenceOrderHints { get; } + + /// + /// Adds one owner for this retained state. + /// + public void AddOwner() => this.ownerCount++; + + /// + /// Releases one owner and returns retained buffers after the final lease. + /// + public void ReleaseOwner() + { + this.ownerCount--; + if (this.ownerCount == 0) + { + this.RetainedMotionField?.Dispose(); + this.RetainedMotionField = null; + this.SegmentIds?.Dispose(); + this.SegmentIds = null; + } + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1FrameInfo.cs b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1FrameInfo.cs new file mode 100644 index 0000000000..11958d5eac --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1FrameInfo.cs @@ -0,0 +1,899 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Buffers; +using SixLabors.ImageSharp.Formats.Heif.Av1.Motion; +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; +using SixLabors.ImageSharp.Memory; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +/// +/// Identifies the inter-prediction features selected by decoded AV1 block syntax. +/// +[Flags] +internal enum Av1InterPredictionFeatures +{ + /// + /// No tracked inter-prediction feature was selected. + /// + None = 0, + + /// + /// Distance-weighted compound prediction was selected. + /// + DistanceWeightedCompound = 1 << 0, + + /// + /// A non-inverted wedge compound mask was selected. + /// + WedgeCompound = 1 << 1, + + /// + /// An inverted wedge compound mask was selected. + /// + InvertedWedgeCompound = 1 << 2, + + /// + /// The first difference-weighted compound mask orientation was selected. + /// + DifferenceWeightedCompound = 1 << 3, + + /// + /// The inverted difference-weighted compound mask orientation was selected. + /// + InvertedDifferenceWeightedCompound = 1 << 4, + + /// + /// Smooth inter-intra prediction was selected. + /// + SmoothInterIntra = 1 << 5, + + /// + /// Wedge inter-intra prediction was selected. + /// + WedgeInterIntra = 1 << 6, + + /// + /// Overlapping motion compensation was selected. + /// + Obmc = 1 << 7, + + /// + /// Local warped-motion prediction was selected. + /// + LocalWarp = 1 << 8, + + /// + /// Non-translational global warped-motion prediction was selected. + /// + GlobalWarp = 1 << 9 +} + +/// +/// Owns the mode, motion, segmentation, transform, coefficient, quantizer, and filter state decoded for one AV1 frame. +/// +internal sealed partial class Av1FrameInfo : IDisposable +{ + /// + /// The allocator that owns frame-sized syntax and retained-reference state. + /// + private readonly MemoryAllocator memoryAllocator; + + /// + /// The raster coefficient slots reserved for one 4x4 mode-information unit. + /// + public const int CoefficientCountPerModeInfo = 16; + + /// + /// Owns the luma and chroma coefficient scratch for the superblock currently being decoded. + /// + private readonly IMemoryOwner coefficientScratch; + + /// + /// The number of luma coefficient entries at the start of . + /// + private readonly int lumaCoefficientCount; + + /// + /// The number of coefficient entries reserved for each chroma plane. + /// + private readonly int chromaCoefficientCount; + + /// + /// The width and height of a superblock in 4x4 mode-information units. + /// + private readonly int modeInfoSizePerSuperblock; + + /// + /// The number of 4x4 mode-information positions in one square superblock. + /// + private readonly int modeInfoCountPerSuperblock; + + /// + /// The number of columns in the frame superblock grid. + /// + private readonly int superblockColumnCount; + + /// + /// The number of rows in the frame superblock grid. + /// + private readonly int superblockRowCount; + + /// + /// The base-2 reduction from luma coefficient capacity to per-chroma-plane capacity. + /// + private readonly int subsamplingFactor; + + /// + /// Stores decoded block mode information in bitstream traversal order. + /// + private readonly MemoryGroup modeInfos; + + /// + /// Stores the number of mode-information records written to each superblock row in . + /// + private readonly Buffer2D modeInfoCounts; + + /// + /// Maps every frame-relative 4x4 position to its covering entry in . + /// + private readonly Av1FrameModeInfoMap modeInfoMap; + + /// + /// Stores the decoded segment identifier for each active 4x4 mode-information position in row-major order. + /// + private Buffer2D? segmentIds; + + /// + /// The number of active 4x4 columns in one row of . + /// + private int segmentIdColumnCount; + + /// + /// The number of active 4x4 rows represented by . + /// + private int segmentIdRowCount; + + /// + /// Owns the luma and shared-chroma transform-information scratch for the superblock currently being decoded. + /// + private readonly IMemoryOwner transformInfoScratch; + + /// + /// Stores the active base quantizer index for each frame superblock. + /// + private readonly Buffer2D quantizerIndices; + + /// + /// The base-2 number of constrained directional enhancement filter entries allocated per superblock. + /// + private readonly int cdefStrengthFactorLog2; + + /// + /// Stores constrained directional enhancement filter strengths grouped by superblock. + /// + private readonly Buffer2D cdefStrength; + + /// + /// The base-2 number of loop-filter delta values stored per superblock. + /// + private readonly int deltaLoopFactorLog2 = 2; + + /// + /// Stores the four loop-filter delta values for each superblock. + /// + private readonly Buffer2D deltaLoopFilter; + + /// + /// Stores raster-ordered loop-restoration units for each color plane. + /// + private InlineArray4?> loopRestorationUnits; + + /// + /// Stores the number of loop-restoration unit columns for each color plane. + /// + private InlineArray4 loopRestorationUnitColumns; + + /// + /// The number of loop-restoration unit rows allocated for each color plane. + /// + private InlineArray4 loopRestorationUnitRows; + + /// + /// Initializes a new instance of the class using sequence-maximum dimensions. + /// + /// The sequence header defining maximum dimensions, superblock size, and color sampling. + public Av1FrameInfo(ObuSequenceHeader sequenceHeader) + : this(Configuration.Default, sequenceHeader, sequenceHeader.MaxFrameWidth, sequenceHeader.MaxFrameHeight) + { + } + + /// + /// Initializes a new instance of the class for one active coded frame. + /// + /// The sequence header defining superblock size and color sampling. + /// The frame header defining the active coded dimensions. + public Av1FrameInfo(ObuSequenceHeader sequenceHeader, ObuFrameHeader frameHeader) + : this( + Configuration.Default, + sequenceHeader, + frameHeader.FrameSize.FrameWidth, + frameHeader.FrameSize.FrameHeight) + { + } + + /// + /// Initializes a new instance of the class for one active coded frame. + /// + /// The decoder configuration providing frame-sized storage. + /// The sequence header defining superblock size and color sampling. + /// The frame header defining the active coded dimensions. + public Av1FrameInfo(Configuration configuration, ObuSequenceHeader sequenceHeader, ObuFrameHeader frameHeader) + : this( + configuration, + sequenceHeader, + frameHeader.FrameSize.FrameWidth, + frameHeader.FrameSize.FrameHeight) + { + } + + /// + /// Initializes a new instance of the class for explicit coded dimensions. + /// + /// The decoder configuration providing frame-sized storage. + /// The sequence header defining superblock size and color sampling. + /// The active coded frame width. + /// The active coded frame height. + private Av1FrameInfo(Configuration configuration, ObuSequenceHeader sequenceHeader, int frameWidth, int frameHeight) + { + // A FrameInfo instance belongs to one coded frame, so transient syntax storage follows that frame rather + // than the potentially much larger sequence maximum declared by an untrusted stream. + this.memoryAllocator = configuration.MemoryAllocator; + int superblockSizeLog2 = sequenceHeader.SuperblockSizeLog2; + int superblockAlignedWidth = Av1Math.AlignPowerOf2(frameWidth, superblockSizeLog2); + int superblockAlignedHeight = Av1Math.AlignPowerOf2(frameHeight, superblockSizeLog2); + this.superblockColumnCount = superblockAlignedWidth >> superblockSizeLog2; + this.superblockRowCount = superblockAlignedHeight >> superblockSizeLog2; + int superblockCount = this.superblockColumnCount * this.superblockRowCount; + this.modeInfoSizePerSuperblock = 1 << (superblockSizeLog2 - Av1Constants.ModeInfoSizeLog2); + this.modeInfoCountPerSuperblock = this.modeInfoSizePerSuperblock * this.modeInfoSizePerSuperblock; + bool subX = sequenceHeader.ColorConfig.SubSamplingX; + bool subY = sequenceHeader.ColorConfig.SubSamplingY; + + // Chroma capacity scales by two for each sampled axis: 4:4:4 => 0, 4:2:2 => 1, 4:2:0 => 2. + this.subsamplingFactor = (subX && subY) ? 2 : (subX && !subY) ? 1 : (!subX && !subY) ? 0 : -1; + Guard.IsFalse(this.subsamplingFactor == -1, nameof(this.subsamplingFactor), "Invalid combination of subsampling."); + this.lumaCoefficientCount = this.modeInfoCountPerSuperblock * CoefficientCountPerModeInfo; + this.chromaCoefficientCount = sequenceHeader.ColorConfig.IsMonochrome + ? 0 + : this.lumaCoefficientCount >> this.subsamplingFactor; + + IMemoryOwner? allocatedCoefficientScratch = null; + MemoryGroup? allocatedModeInfos = null; + Buffer2D? allocatedModeInfoCounts = null; + Av1FrameModeInfoMap? allocatedModeInfoMap = null; + IMemoryOwner? allocatedTransformInfoScratch = null; + Buffer2D? allocatedQuantizerIndices = null; + Buffer2D? allocatedCdefStrength = null; + Buffer2D? allocatedDeltaLoopFilter = null; + + try + { + // Reconstruction consumes one superblock before parsing the next. One allocator-owned scratch surface + // therefore covers all three planes without per-block arrays or unmanaged ownership outside ImageSharp. + int coefficientScratchLength = checked(this.lumaCoefficientCount + (2 * this.chromaCoefficientCount)); + allocatedCoefficientScratch = this.memoryAllocator.Allocate(coefficientScratchLength, AllocationOptions.Clean); + + // A decoded block can cover multiple 4x4 positions. Each allocator-backed row stores one + // superblock's traversal records, while the map resolves every covered 4x4 position to them. + AllocationOptions clean = AllocationOptions.Clean; + + // Mode information survives until frame completion, but one superblock row is much larger than a + // constrained allocator segment. Store it as a discontiguous group and address records by packed index. + long modeInfoLength = (long)this.modeInfoCountPerSuperblock * superblockCount; + allocatedModeInfos = this.memoryAllocator.AllocateGroup(modeInfoLength, 1, clean); + allocatedModeInfoCounts = this.memoryAllocator.Allocate2D(1, superblockCount, clean); + allocatedModeInfoMap = new Av1FrameModeInfoMap( + this.memoryAllocator, + new Size( + this.modeInfoSizePerSuperblock * this.superblockColumnCount, + this.modeInfoSizePerSuperblock * this.superblockRowCount)); + + // Tile parsing reconstructs each superblock before advancing to the next one. A single three-plane + // scratch owner therefore preserves every active transform while avoiding frame-wide retained copies. + int transformInfoScratchLength = checked(this.modeInfoCountPerSuperblock * 3); + allocatedTransformInfoScratch = this.memoryAllocator.Allocate(transformInfoScratchLength, clean); + allocatedQuantizerIndices = this.memoryAllocator.Allocate2D(1, superblockCount, clean); + + // A 128x128 superblock contains four 64x64 CDEF filter blocks; a 64x64 superblock contains one. + this.cdefStrengthFactorLog2 = (superblockSizeLog2 - 6) << 1; + allocatedCdefStrength = this.memoryAllocator.Allocate2D(1 << this.cdefStrengthFactorLog2, superblockCount, clean); + allocatedCdefStrength.MemoryGroup.Fill(-1); + allocatedDeltaLoopFilter = this.memoryAllocator.Allocate2D(1 << this.deltaLoopFactorLog2, superblockCount, clean); + } + catch + { + allocatedDeltaLoopFilter?.Dispose(); + allocatedCdefStrength?.Dispose(); + allocatedQuantizerIndices?.Dispose(); + allocatedTransformInfoScratch?.Dispose(); + allocatedModeInfoMap?.Dispose(); + allocatedModeInfoCounts?.Dispose(); + allocatedModeInfos?.Dispose(); + allocatedCoefficientScratch?.Dispose(); + throw; + } + + this.coefficientScratch = allocatedCoefficientScratch; + this.modeInfos = allocatedModeInfos; + this.modeInfoCounts = allocatedModeInfoCounts; + this.modeInfoMap = allocatedModeInfoMap; + this.transformInfoScratch = allocatedTransformInfoScratch; + this.quantizerIndices = allocatedQuantizerIndices; + this.cdefStrength = allocatedCdefStrength; + this.deltaLoopFilter = allocatedDeltaLoopFilter; + } + + /// + /// Gets the total mode-information capacity allocated for the frame. + /// + public int ModeInfoCount => checked((int)this.modeInfos.TotalLength); + + /// + /// Gets the width or height of one square superblock in 4x4 mode-information units. + /// + public int SuperblockModeInfoSize => this.modeInfoSizePerSuperblock; + + /// + /// Gets a bit mask containing every luma transform type decoded in this frame. + /// + public int LumaTransformTypeCoverage { get; private set; } + + /// + /// Gets the inter-prediction features selected by coding blocks in this frame. + /// + public Av1InterPredictionFeatures InterPredictionFeatures { get; private set; } + + /// + /// Records one decoded luma transform type before the current-superblock scratch is reused. + /// + /// The decoded luma transform type. + public void RecordLumaTransformType(Av1TransformType transformType) => + this.LumaTransformTypeCoverage |= 1 << (int)transformType; + + /// + /// Records the inter-prediction features selected by one completed coding block. + /// + /// The completed block mode information. + /// The frame header containing global-motion parameters. + public void RecordInterPredictionFeatures(Av1BlockModeInfo modeInfo, ObuFrameHeader frameHeader) + { + if (modeInfo.ReferenceFrames[0] <= Av1ReferenceFrameType.Intra) + { + return; + } + + Av1InterPredictionFeatures features = Av1InterPredictionFeatures.None; + if (modeInfo.MotionMode == Av1MotionMode.Obmc) + { + features |= Av1InterPredictionFeatures.Obmc; + } + + if (modeInfo.MotionMode == Av1MotionMode.Warped) + { + features |= Av1InterPredictionFeatures.LocalWarp; + } + + if (modeInfo.YMode is Av1PredictionMode.GlobalMotionVector or Av1PredictionMode.GlobalGlobalMotionVector && + Math.Min(modeInfo.BlockSize.GetWidth(), modeInfo.BlockSize.GetHeight()) >= 8) + { + int referenceCount = modeInfo.ReferenceFrames[1] > Av1ReferenceFrameType.Intra ? 2 : 1; + Span globalMotionParameters = frameHeader.GetGlobalMotionParameters(); + + for (int referenceIndex = 0; referenceIndex < referenceCount; referenceIndex++) + { + int canonicalReferenceIndex = + (int)modeInfo.ReferenceFrames[referenceIndex] - (int)Av1ReferenceFrameType.Last; + + Av1GlobalMotionParameters parameters = globalMotionParameters[canonicalReferenceIndex]; + if (parameters.Type > Av1GlobalMotionType.Translation && !parameters.IsInvalid) + { + features |= Av1InterPredictionFeatures.GlobalWarp; + } + } + } + + if (modeInfo.ReferenceFrames[1] == Av1ReferenceFrameType.Intra) + { + features |= modeInfo.UseInterIntraWedge + ? Av1InterPredictionFeatures.WedgeInterIntra + : Av1InterPredictionFeatures.SmoothInterIntra; + } + else if (modeInfo.ReferenceFrames[1] > Av1ReferenceFrameType.Intra) + { + features |= modeInfo.CompoundType switch + { + Av1CompoundType.DistanceWeighted => Av1InterPredictionFeatures.DistanceWeightedCompound, + Av1CompoundType.Wedge => modeInfo.CompoundWedgeSign + ? Av1InterPredictionFeatures.InvertedWedgeCompound + : Av1InterPredictionFeatures.WedgeCompound, + Av1CompoundType.DifferenceWeighted => modeInfo.DifferenceWeightedMaskType == Av1DifferenceWeightedMaskType.Type38Inverse + ? Av1InterPredictionFeatures.InvertedDifferenceWeightedCompound + : Av1InterPredictionFeatures.DifferenceWeightedCompound, + _ => Av1InterPredictionFeatures.None, + }; + } + + this.InterPredictionFeatures |= features; + } + + /// + /// Initializes the active frame's contiguous segment map and applies whole-map inheritance when requested. + /// + /// The frame header defining active geometry and segmentation update behavior. + /// + /// The retained state selected by the primary reference, or when no primary reference exists. + /// + public void InitializeSegmentIds(ObuFrameHeader frameHeader, ReferenceState? primaryReferenceState) + { + ObuSegmentationParameters segmentationParameters = frameHeader.SegmentationParameters; + if (!segmentationParameters.Enabled) + { + // A disabled map is normatively all zero. Empty storage represents that state without retaining one byte + // for every 4x4 position on frames that cannot use segmentation. + return; + } + + this.segmentIdColumnCount = frameHeader.ModeInfoColumnCount; + this.segmentIdRowCount = frameHeader.ModeInfoRowCount; + this.segmentIds = this.memoryAllocator.Allocate2D( + this.segmentIdColumnCount, + this.segmentIdRowCount, + AllocationOptions.Clean); + + Buffer2D? primarySegmentIds = primaryReferenceState?.SegmentIds; + if (segmentationParameters.SegmentationUpdateMap == 0 && + primaryReferenceState is not null && + primarySegmentIds is not null && + primaryReferenceState.SegmentIdColumnCount == this.segmentIdColumnCount && + primaryReferenceState.SegmentIdRowCount == this.segmentIdRowCount) + { + // AV1 decodemv.c copies the selected primary frame's block coverage when update_map is zero. Copying the + // same contiguous map once establishes the identical final state without repeating a row copy per block. + primarySegmentIds.CopyTo(this.segmentIds); + } + } + + /// + /// Gets the segment identifier stored at one active 4x4 mode-information position. + /// + /// The zero-based mode-information row. + /// The zero-based mode-information column. + /// The segment identifier stored at the requested position. + public byte GetSegmentId(int row, int column) + { + Buffer2D segmentIds = this.segmentIds + ?? this.referenceState?.SegmentIds + ?? throw new InvalidOperationException("The AV1 frame has no active segmentation map."); + + return segmentIds[column, row]; + } + + /// + /// Gets the minimum retained segment identifier across a block's clipped mode-information coverage. + /// + /// + /// The retained primary-frame state, or when no compatible map is available. + /// + /// The block size whose 4x4 coverage is inspected. + /// The block origin in frame-relative 4x4 units. + /// + /// The minimum retained segment identifier, or zero when no same-sized retained segmentation map is available. + /// + public int GetPredictedSegmentId(ReferenceState? primaryReferenceState, Av1BlockSize blockSize, Point modeInfoPosition) + { + Buffer2D? primarySegmentIds = primaryReferenceState?.SegmentIds; + if (primaryReferenceState is null || + primarySegmentIds is null || + primaryReferenceState.SegmentIdColumnCount != this.segmentIdColumnCount || + primaryReferenceState.SegmentIdRowCount != this.segmentIdRowCount) + { + // the reference decoder exposes the prior map only when both mode-info dimensions match the active frame. Treating a + // differently sized retained map as absent prevents coordinates from being reinterpreted with a new stride. + return 0; + } + + int columnCount = Math.Min(blockSize.Get4x4WideCount(), this.segmentIdColumnCount - modeInfoPosition.X); + int rowCount = Math.Min(blockSize.Get4x4HighCount(), this.segmentIdRowCount - modeInfoPosition.Y); + int segmentId = Av1Constants.MaxSegmentCount; + + // Temporal prediction uses the minimum over every clipped 4x4 cell, not merely the block origin. This is the + // dec_get_segment_id rule used when segmentation_temporal_update selects the retained primary map. + for (int row = 0; row < rowCount; row++) + { + ReadOnlySpan segmentRow = primarySegmentIds + .DangerousGetRowSpan(modeInfoPosition.Y + row) + .Slice(modeInfoPosition.X, columnCount); + + for (int column = 0; column < segmentRow.Length; column++) + { + segmentId = Math.Min(segmentId, segmentRow[column]); + } + } + + return segmentId; + } + + /// + /// Writes one segment identifier over a block's clipped mode-information coverage. + /// + /// The block size whose 4x4 coverage is updated. + /// The block origin in frame-relative 4x4 units. + /// The decoded segment identifier. + public void SetSegmentId(Av1BlockSize blockSize, Point modeInfoPosition, int segmentId) + { + int columnCount = Math.Min(blockSize.Get4x4WideCount(), this.segmentIdColumnCount - modeInfoPosition.X); + int rowCount = Math.Min(blockSize.Get4x4HighCount(), this.segmentIdRowCount - modeInfoPosition.Y); + Buffer2D segmentIds = this.segmentIds + ?? throw new InvalidOperationException("The AV1 frame has no writable segmentation map."); + + // Each block contributes one ID to all covered 4x4 cells. Filling contiguous row slices retains the native + // row-major layout without the per-row object indirection of the previous jagged map. + for (int row = 0; row < rowCount; row++) + { + segmentIds + .DangerousGetRowSpan(modeInfoPosition.Y + row) + .Slice(modeInfoPosition.X, columnCount) + .Fill((byte)segmentId); + } + } + + /// + /// Allocates the loop-restoration unit grid described by the active frame header. + /// + /// The sequence header defining the plane count and chroma subsampling. + /// The frame header defining upscaled dimensions and restoration-unit sizes. + public void InitializeLoopRestoration(ObuSequenceHeader sequenceHeader, ObuFrameHeader frameHeader) + { + ObuColorConfig colorConfig = sequenceHeader.ColorConfig; + for (int planeIndex = 0; planeIndex < colorConfig.PlaneCount; planeIndex++) + { + ObuLoopRestorationItem item = frameHeader.LoopRestorationParameters.Items[planeIndex]; + if (item.Type == ObuRestorationType.None) + { + continue; + } + + Av1Plane plane = (Av1Plane)planeIndex; + int subsamplingX = plane != Av1Plane.Y && colorConfig.SubSamplingX ? 1 : 0; + int subsamplingY = plane != Av1Plane.Y && colorConfig.SubSamplingY ? 1 : 0; + int planeWidth = Av1Math.DivideLog2Ceiling(frameHeader.FrameSize.SuperResolutionUpscaledWidth, subsamplingX); + int planeHeight = Av1Math.DivideLog2Ceiling(frameHeader.FrameSize.FrameHeight, subsamplingY); + + // A final unit may extend to 150 percent of the nominal size, so AV1 rounds the + // unit count to nearest instead of unconditionally rounding a partial unit upward. + int columnCount = Math.Max((planeWidth + (item.Size >> 1)) / item.Size, 1); + int rowCount = Math.Max((planeHeight + (item.Size >> 1)) / item.Size, 1); + this.loopRestorationUnitColumns[planeIndex] = columnCount; + this.loopRestorationUnitRows[planeIndex] = rowCount; + this.loopRestorationUnits[planeIndex] = this.memoryAllocator.Allocate2D( + columnCount, + rowCount, + AllocationOptions.Clean); + } + } + + /// + /// Gets the superblock view at the specified frame-grid position. + /// + /// The position in the frame superblock grid. + /// The superblock view. + public Av1SuperblockInfo GetSuperblock(Point index) => new(this, index); + + /// + /// Gets the number of mode-information records parsed for a specified superblock. + /// + /// The position in the frame superblock grid. + /// The number of parsed records. + public int GetModeInfoCount(Point index) + { + int storageRow = (index.Y * this.superblockColumnCount) + index.X; + return this.modeInfoCounts[0, storageRow]; + } + + /// + /// Gets the mode information covering the origin of a specified superblock. + /// + /// The position in the frame superblock grid. + /// The mode information covering the superblock origin. + public Av1BlockModeInfo GetModeInfo(Point superblockIndex) => this.GetModeInfo(superblockIndex, Point.Empty); + + /// + /// Gets the mode information covering a position relative to a specified superblock. + /// + /// The position in the frame superblock grid. + /// The position within the superblock in 4x4 mode-information units. + /// The mode information covering the position. + public Av1BlockModeInfo GetModeInfo(Point superblockIndex, Point modeInfoIndex) + { + Point location = this.GetModeInfoPosition(superblockIndex, modeInfoIndex); + return this.GetModeInfoByStorageIndex(this.modeInfoMap[location]); + } + + /// + /// Gets the mode information record covering the specified frame-relative mode information position. + /// + /// The frame-relative position in 4x4 mode-information units. + /// The mode information covering the position. + public Av1BlockModeInfo GetModeInfoAt(Point modeInfoPosition) + => this.GetModeInfoByStorageIndex(this.modeInfoMap[modeInfoPosition]); + + /// + /// Gets the mode information records parsed for the specified superblock in bitstream order. + /// + /// The position in the frame superblock grid. + /// The number of parsed records to return. + /// A discontiguous view of the parsed mode-information records. + public ModeInfoCollection GetModeInfos(Point superblockIndex, int count) + { + int storageRow = (superblockIndex.Y * this.superblockColumnCount) + superblockIndex.X; + return new ModeInfoCollection(this, storageRow * this.modeInfoCountPerSuperblock, count); + } + + /// + /// Gets the transform-information scratch for one plane of the current superblock. + /// + /// The zero-based plane index. + /// The luma storage for plane zero; otherwise, the shared chroma storage. + public Span GetSuperblockTransform(int plane) + { + if (plane == 0) + { + return this.GetSuperblockTransformY(); + } + + return this.GetSuperblockTransformUv(); + } + + /// + /// Gets the luma transform-information scratch for the current superblock. + /// + /// The current-superblock luma transform-information span. + public Span GetSuperblockTransformY() + => this.transformInfoScratch.GetSpan()[..this.modeInfoCountPerSuperblock]; + + /// + /// Gets the shared chroma transform-information scratch for the current superblock. + /// + /// The current-superblock chroma transform-information span. + public Span GetSuperblockTransformUv() + => this.transformInfoScratch.GetSpan().Slice( + this.modeInfoCountPerSuperblock, + this.modeInfoCountPerSuperblock << 1); + + /// + /// Gets the luma coefficient scratch reused for the current superblock. + /// + /// The current superblock luma coefficient span. + public Span GetCoefficientsY() + => this.coefficientScratch.GetSpan()[..this.lumaCoefficientCount]; + + /// + /// Gets the blue-difference chroma coefficient scratch reused for the current superblock. + /// + /// The current superblock blue-difference chroma coefficient span. + public Span GetCoefficientsU() + => this.coefficientScratch.GetSpan().Slice(this.lumaCoefficientCount, this.chromaCoefficientCount); + + /// + /// Gets the red-difference chroma coefficient scratch reused for the current superblock. + /// + /// The current superblock red-difference chroma coefficient span. + public Span GetCoefficientsV() + => this.coefficientScratch.GetSpan().Slice( + this.lumaCoefficientCount + this.chromaCoefficientCount, + this.chromaCoefficientCount); + + /// + /// Gets a reference to the active base quantizer index for a specified superblock. + /// + /// The position in the frame superblock grid. + /// A reference to the superblock base quantizer index. + public ref int GetQuantizerIndex(Point index) + { + int storageRow = (index.Y * this.superblockColumnCount) + index.X; + return ref this.quantizerIndices[0, storageRow]; + } + + /// + /// Gets the constrained directional enhancement filter strengths for a specified superblock. + /// + /// The position in the frame superblock grid. + /// The superblock filter-strength span. + public Span GetCdefStrength(Point index) + { + int storageRow = (index.Y * this.superblockColumnCount) + index.X; + return this.cdefStrength.DangerousGetRowSpan(storageRow); + } + + /// + /// Resets every constrained directional enhancement filter strength for a superblock to its unassigned value. + /// + /// The position in the frame superblock grid. + public void ClearCdef(Point index) + { + Span cdefs = this.GetCdefStrength(index); + for (int i = 0; i < cdefs.Length; i++) + { + cdefs[i] = -1; + } + } + + /// + /// Gets the four loop-filter delta values for a specified superblock. + /// + /// The position in the frame superblock grid. + /// The superblock loop-filter delta span. + public Span GetDeltaLoopFilter(Point index) + { + int storageRow = (index.Y * this.superblockColumnCount) + index.X; + return this.deltaLoopFilter.DangerousGetRowSpan(storageRow); + } + + /// + /// Gets the number of loop-restoration unit columns allocated for a color plane. + /// + /// The zero-based color-plane index. + /// The number of restoration-unit columns. + public int GetLoopRestorationUnitColumnCount(int plane) => this.loopRestorationUnitColumns[plane]; + + /// + /// Gets the number of loop-restoration unit rows allocated for a color plane. + /// + /// The zero-based color-plane index. + /// The number of restoration-unit rows. + public int GetLoopRestorationUnitRowCount(int plane) => this.loopRestorationUnitRows[plane]; + + /// + /// Gets the loop-restoration unit at a plane-relative grid position. + /// + /// The zero-based color-plane index. + /// The restoration-unit row. + /// The restoration-unit column. + /// The decoded restoration-unit information. + public ref Av1LoopRestorationUnit GetLoopRestorationUnit(int plane, int row, int column) + { + Buffer2D units = this.loopRestorationUnits[plane] + ?? throw new InvalidOperationException("The selected AV1 plane has no loop-restoration units."); + + return ref units[column, row]; + } + + /// + /// Stores decoded mode information and maps every 4x4 position covered by its block. + /// + /// The decoded block mode information. + /// The containing superblock. + public void UpdateModeInfo(Av1BlockModeInfo modeInfo, Av1SuperblockInfo superblockInfo) + { + Point modeInfoPosition = this.GetModeInfoPosition(superblockInfo.Position, modeInfo.PositionInSuperblock); + int storageRow = (superblockInfo.Position.Y * this.superblockColumnCount) + superblockInfo.Position.X; + ref int modeInfoCount = ref this.modeInfoCounts[0, storageRow]; + DebugGuard.MustBeLessThan(modeInfoCount, this.modeInfoCountPerSuperblock, nameof(modeInfoCount)); + int storageIndex = (storageRow * this.modeInfoCountPerSuperblock) + modeInfoCount; + modeInfo.ModeInfoIndex = this.modeInfoMap.NextIndex; + this.GetModeInfoByStorageIndex(storageIndex) = modeInfo; + modeInfoCount++; + this.UpdateRetainedMotionField(modeInfo, modeInfoPosition); + this.modeInfoMap.Update(modeInfoPosition, modeInfo.BlockSize, storageIndex); + } + + /// + /// Gets a reference to one packed mode-information record across allocator segments. + /// + /// The packed frame-storage index. + /// A reference to the selected mode information. + private ref Av1BlockModeInfo GetModeInfoByStorageIndex(int storageIndex) + { + int bufferIndex = storageIndex / this.modeInfos.BufferLength; + int elementIndex = storageIndex - (bufferIndex * this.modeInfos.BufferLength); + return ref this.modeInfos[bufferIndex].Span[elementIndex]; + } + + /// + /// Converts a superblock-relative mode-information position to frame-relative coordinates. + /// + /// The position in the frame superblock grid. + /// The position within the superblock in 4x4 units. + /// The frame-relative position in 4x4 mode-information units. + private Point GetModeInfoPosition(Point superblockPosition, Point positionInSuperblock) + { + int x = (superblockPosition.X * this.modeInfoSizePerSuperblock) + positionInSuperblock.X; + int y = (superblockPosition.Y * this.modeInfoSizePerSuperblock) + positionInSuperblock.Y; + return new Point(x, y); + } + + /// + /// Provides indexed and reference-preserving traversal over one superblock's discontiguous mode information. + /// + public readonly struct ModeInfoCollection + { + private readonly Av1FrameInfo owner; + private readonly int startIndex; + + /// + /// Initializes a new instance of the struct. + /// + /// The frame that owns the mode-information group. + /// The packed index of the first record. + /// The number of records in the view. + public ModeInfoCollection(Av1FrameInfo owner, int startIndex, int length) + { + this.owner = owner; + this.startIndex = startIndex; + this.Length = length; + } + + /// + /// Gets the number of records in the view. + /// + public int Length { get; } + + /// + /// Gets a reference to the record at the specified traversal index. + /// + /// The zero-based traversal index. + public ref Av1BlockModeInfo this[int index] => + ref this.owner.GetModeInfoByStorageIndex(this.startIndex + index); + + /// + /// Creates a reference-preserving enumerator over the records. + /// + /// The initialized enumerator. + public Enumerator GetEnumerator() => new(this.owner, this.startIndex, this.Length); + + /// + /// Enumerates one superblock's mode-information records without flattening allocator segments. + /// + public struct Enumerator + { + private readonly Av1FrameInfo owner; + private readonly int startIndex; + private readonly int length; + private int index; + + /// + /// Initializes a new instance of the struct. + /// + /// The frame that owns the mode-information group. + /// The packed index of the first record. + /// The number of records in the view. + public Enumerator(Av1FrameInfo owner, int startIndex, int length) + { + this.owner = owner; + this.startIndex = startIndex; + this.length = length; + this.index = -1; + } + + /// + /// Gets a reference to the current record. + /// + public ref Av1BlockModeInfo Current => + ref this.owner.GetModeInfoByStorageIndex(this.startIndex + this.index); + + /// + /// Advances to the next record. + /// + /// when another record is available. + public bool MoveNext() + { + this.index++; + return this.index < this.length; + } + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1FrameModeInfoMap.cs b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1FrameModeInfoMap.cs new file mode 100644 index 0000000000..31e1321a4b --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1FrameModeInfoMap.cs @@ -0,0 +1,113 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Memory; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +/// +/// Provides frame-wide lookup and storage for decoded AV1 mode-information blocks. +/// +internal partial class Av1FrameInfo +{ + /// + /// Mapping of values, from position to index into the . + /// + public sealed class Av1FrameModeInfoMap : IDisposable + { + /// + /// Stores the mode-information index assigned to each aligned 4x4 frame location. + /// + private readonly MemoryGroup offsets; + + /// + /// The dimensions of in 4x4 mode-information units. + /// + private readonly Size alignedModeInfoCount; + + /// + /// Initializes a new instance of the class using the default allocator. + /// + /// The aligned frame dimensions in 4x4 mode-information units. + public Av1FrameModeInfoMap(Size modeInfoCount) + : this(Configuration.Default.MemoryAllocator, modeInfoCount) + { + } + + /// + /// Initializes a new instance of the class. + /// + /// The allocator providing frame-sized storage. + /// The aligned frame dimensions in 4x4 mode-information units. + public Av1FrameModeInfoMap(MemoryAllocator memoryAllocator, Size modeInfoCount) + { + this.alignedModeInfoCount = modeInfoCount; + this.NextIndex = 0; + long offsetCount = (long)this.alignedModeInfoCount.Width * this.alignedModeInfoCount.Height; + this.offsets = memoryAllocator.AllocateGroup(offsetCount, 1, AllocationOptions.Clean); + } + + /// + /// Gets the next index to use. + /// + public int NextIndex { get; private set; } + + /// + /// Gets the mode-information index mapped to the specified 4x4 location. + /// + /// The location in 4x4 mode-information units. + public int this[Point location] + { + get + { + long offset = ((long)location.Y * this.alignedModeInfoCount.Width) + location.X; + return this.offsets.GetRemainingSliceOfBuffer(offset)[0]; + } + } + + /// + /// Maps every 4x4 location covered by a decoded block to the next mode-information index. + /// + /// The block origin in 4x4 mode-information units. + /// The decoded block size. + /// The packed frame-storage index assigned to the decoded block. + public void Update(Point modeInfoLocation, Av1BlockSize blockSize, int storageIndex) + { + int bw4 = blockSize.Get4x4WideCount(); + int bh4 = blockSize.Get4x4HighCount(); + DebugGuard.MustBeGreaterThanOrEqualTo(modeInfoLocation.Y, 0, nameof(modeInfoLocation)); + DebugGuard.MustBeLessThanOrEqualTo(modeInfoLocation.Y + bh4, this.alignedModeInfoCount.Height, nameof(modeInfoLocation)); + DebugGuard.MustBeGreaterThanOrEqualTo(modeInfoLocation.X, 0, nameof(modeInfoLocation)); + DebugGuard.MustBeLessThanOrEqualTo(modeInfoLocation.X + bw4, this.alignedModeInfoCount.Width, nameof(modeInfoLocation)); + + // Every 4x4 cell covered by the block must resolve to the same mode information, + // because later blocks query their above and left neighbors at cell granularity. + for (int i = modeInfoLocation.Y; i < modeInfoLocation.Y + bh4; i++) + { + long offset = ((long)i * this.alignedModeInfoCount.Width) + modeInfoLocation.X; + int remaining = bw4; + while (remaining > 0) + { + Span destination = this.offsets.GetRemainingSliceOfBuffer(offset); + int count = Math.Min(remaining, destination.Length); + destination[..count].Fill(storageIndex); + offset += count; + remaining -= count; + } + } + + this.NextIndex++; + } + + /// + /// Maps every 4x4 location covered by a decoded block to its traversal-order index. + /// + /// The block origin in 4x4 mode-information units. + /// The decoded block size. + public void Update(Point modeInfoLocation, Av1BlockSize blockSize) + => this.Update(modeInfoLocation, blockSize, this.NextIndex); + + /// + public void Dispose() => this.offsets.Dispose(); + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1InterIntraMode.cs b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1InterIntraMode.cs new file mode 100644 index 0000000000..bd101ab63b --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1InterIntraMode.cs @@ -0,0 +1,30 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +/// +/// Identifies the intra predictor blended with a single-reference inter predictor. +/// +internal enum Av1InterIntraMode : byte +{ + /// + /// Uses a DC intra predictor. + /// + DC = 0, + + /// + /// Uses a vertical intra predictor. + /// + Vertical = 1, + + /// + /// Uses a horizontal intra predictor. + /// + Horizontal = 2, + + /// + /// Uses a smooth intra predictor. + /// + Smooth = 3, +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1LevelBuffer.cs b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1LevelBuffer.cs new file mode 100644 index 0000000000..63ddd315db --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1LevelBuffer.cs @@ -0,0 +1,165 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Buffers; +using System.Runtime.CompilerServices; +using System.Runtime.InteropServices; +using SixLabors.ImageSharp.Memory; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +/// +/// Owns the padded absolute-coefficient level plane used to derive AV1 coefficient entropy contexts. +/// +internal sealed class Av1LevelBuffer : IDisposable +{ + /// + /// Owns the padded level storage until the buffer is disposed. + /// + private IMemoryOwner? memory; + + /// + /// Initializes a new instance of the class for the maximum entropy-coded + /// coefficient dimensions. + /// + /// The configuration providing the memory allocator. + public Av1LevelBuffer(Configuration configuration) + : this( + configuration, + new Size(Av1Constants.MaxTransformSize / 2, Av1Constants.MaxTransformSize / 2)) + { + } + + /// + /// Initializes a new instance of the class for the specified coefficient dimensions. + /// + /// The configuration providing the memory allocator. + /// The unpadded coefficient dimensions. + public Av1LevelBuffer(Configuration configuration, Size size) + { + this.Size = size; + + // Coefficient-context derivation reads fixed neighboring offsets around the coded transform. + // Keeping those offsets inside one clean allocation avoids branches at transform boundaries. + int totalHeight = Av1Constants.TransformPadTop + size.Height + Av1Constants.TransformPadBottom; + this.Stride = Av1Constants.TransformPadHorizontal + size.Width; + this.memory = configuration.MemoryAllocator.Allocate(this.Stride * totalHeight, AllocationOptions.Clean); + } + + /// + /// Gets the unpadded coefficient dimensions. + /// + public Size Size { get; private set; } + + /// + /// Gets the padded row stride in bytes. + /// + public int Stride { get; private set; } + + /// + /// Gets the coefficient level at the specified unpadded position. + /// + /// The coefficient position. + public int this[Point position] => this.GetRow(position.Y)[position.X]; + + /// + /// Initializes the unpadded level plane from raster-ordered coefficient magnitudes. + /// + /// The coefficient levels to copy. + public void Initialize(ReadOnlySpan coefficientBuffer) + { + ObjectDisposedException.ThrowIf(this.memory == null, this); + ArgumentOutOfRangeException.ThrowIfLessThan(coefficientBuffer.Length, this.Size.Width * this.Size.Height, nameof(coefficientBuffer)); + for (int y = 0; y < this.Size.Height; y++) + { + ref byte destRef = ref this.GetRow(y)[0]; + ref int sourceRef = ref Unsafe.Add( + ref MemoryMarshal.GetReference(coefficientBuffer), + y * this.Size.Width); + + for (int x = 0; x < this.Size.Width; x++) + { + // Entropy contexts use the absolute level, saturated to the signed-byte range used by the + // normative nonzero-map context calculation. + destRef = (byte)Math.Min(Math.Abs(sourceRef), sbyte.MaxValue); + destRef = ref Unsafe.Add(ref destRef, 1); + sourceRef = ref Unsafe.Add(ref sourceRef, 1); + } + } + } + + /// + /// Converts a raster-order coefficient index to its two-dimensional position. + /// + /// The raster-order coefficient index. + /// The corresponding coefficient position. + public Point GetPosition(int index) + { + int x = index % this.Size.Width; + int y = index / this.Size.Width; + return new Point(x, y); + } + + /// + /// Gets a padded coefficient row for the specified position. + /// + /// A position whose vertical coordinate selects the row. + /// The selected row, including its horizontal context padding. + public Span GetRow(Point pos) + => this.GetRow(pos.Y); + + /// + /// Gets a padded coefficient row by its unpadded vertical coordinate. + /// + /// The row coordinate, which may address the top context padding. + /// The selected row, including its horizontal context padding. + public Span GetRow(int y) + { + ObjectDisposedException.ThrowIf(this.memory == null, this); + ArgumentOutOfRangeException.ThrowIfLessThan(y, -Av1Constants.TransformPadTop); + int row = y + Av1Constants.TransformPadTop; + return this.memory.Memory.Span.Slice(row * this.Stride, this.Size.Width + Av1Constants.TransformPadHorizontal); + } + + /// + public void Dispose() + { + this.memory?.Dispose(); + this.memory = null; + } + + /// + /// Selects new active coefficient dimensions and clears their padded context storage. + /// + /// The unpadded coefficient dimensions. + public void Reset(Size size) => this.Reset(size, clear: true); + + /// + /// Selects new active coefficient dimensions and optionally clears their padded context storage. + /// + /// The unpadded coefficient dimensions. + /// Indicates whether to clear the active level plane and its context padding. + public void Reset(Size size, bool clear) + { + ObjectDisposedException.ThrowIf(this.memory == null, this); + this.Size = size; + this.Stride = Av1Constants.TransformPadHorizontal + size.Width; + + if (clear) + { + // Tile parsing is sequential, so one maximum-sized rent can serve every transform. Clear only the active + // layout because stale neighboring levels would otherwise select the wrong coefficient distributions. + int totalHeight = Av1Constants.TransformPadTop + size.Height + Av1Constants.TransformPadBottom; + this.memory.Memory.Span[..(this.Stride * totalHeight)].Clear(); + } + } + + /// + /// Clears all coefficient levels and context padding. + /// + public void Clear() + { + ObjectDisposedException.ThrowIf(this.memory == null, this); + this.memory.Memory.Span.Clear(); + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1LoopRestorationUnit.cs b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1LoopRestorationUnit.cs new file mode 100644 index 0000000000..023d9dfd1e --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1LoopRestorationUnit.cs @@ -0,0 +1,35 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +/// +/// Stores the filter selection and coefficients decoded for one AV1 loop-restoration unit. +/// +internal struct Av1LoopRestorationUnit +{ + /// + /// The three transmitted symmetric vertical Wiener coefficients. + /// + public InlineArray3 WienerVertical; + + /// + /// The three transmitted symmetric horizontal Wiener coefficients. + /// + public InlineArray3 WienerHorizontal; + + /// + /// The two self-guided projection coefficients. + /// + public InlineArray2 SgrProjectionCoefficients; + + /// + /// Gets or sets the restoration filter selected for the unit. + /// + public Av1RestorationFilterType FilterType { get; set; } + + /// + /// Gets or sets the self-guided filter parameter-set index. + /// + public int SgrParameterSet { get; set; } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1MacroBlockD.cs b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1MacroBlockD.cs new file mode 100644 index 0000000000..33598f85ec --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1MacroBlockD.cs @@ -0,0 +1,89 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +/// +/// Holds encoder-side macroblock edges and neighboring mode information. +/// +internal class Av1MacroBlockD +{ + /// + /// Stores the frame's mode-information allocation-index grid. + /// + private Memory modeInfoGrid; + + /// + /// Stores the frame's contiguous mode-information values. + /// + private Memory modeInfoAllocation; + + /// + /// Stores the current block's linear position in . + /// + private int modeInfoIndex; + + /// + /// Gets or sets the tile containing the current block. + /// + public required Av1TileInfo Tile { get; set; } + + /// + /// Gets or sets a value indicating whether an above block is available within the tile. + /// + public bool IsUpAvailable { get; set; } + + /// + /// Gets or sets a value indicating whether a left block is available within the tile. + /// + public bool IsLeftAvailable { get; set; } + + /// + /// Gets or sets the row stride of the frame mode-information map. + /// + public int ModeInfoStride { get; set; } + + /// + /// Gets or sets the signed distance from the block to the top frame edge in one-eighth-sample units. + /// + public int ToTopEdge { get; set; } + + /// + /// Gets or sets the signed distance from the block to the bottom frame edge in one-eighth-sample units. + /// + public int ToBottomEdge { get; set; } + + /// + /// Gets or sets the signed distance from the block to the left frame edge in one-eighth-sample units. + /// + public int ToLeftEdge { get; set; } + + /// + /// Gets or sets the signed distance from the block to the right frame edge in one-eighth-sample units. + /// + public int ToRightEdge { get; set; } + + /// + /// Selects the current entry in the frame-owned mode-information grid. + /// + /// The frame-owned mode-information allocation-index grid. + /// The frame-owned contiguous mode-information values. + /// The current block's linear grid index. + public void SetModeInfoGrid(Memory grid, Memory allocation, int index) + { + this.modeInfoGrid = grid; + this.modeInfoAllocation = allocation; + this.modeInfoIndex = index; + } + + /// + /// Gets a mode-information entry relative to the current block. + /// + /// The signed linear offset from the current block. + /// A reference to the mapped neighboring or current entry. + public ref Av1MacroBlockModeInfo GetRelativeModeInfo(int offset) + { + int allocationIndex = this.modeInfoGrid.Span[this.modeInfoIndex + offset]; + return ref this.modeInfoAllocation.Span[allocationIndex]; + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1MacroBlockModeInfo.cs b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1MacroBlockModeInfo.cs new file mode 100644 index 0000000000..d39355b94b --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1MacroBlockModeInfo.cs @@ -0,0 +1,26 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +/// +/// Stores the encoder's selected modes and references for an AV1 macroblock. +/// +internal struct Av1MacroBlockModeInfo +{ + private byte cdefStrength; + + /// + /// Stores the prediction, transform, and segmentation decisions for the block. + /// + public Av1EncoderBlockModeInfo Block; + + /// + /// Gets or sets the constrained directional enhancement filter strength for the block. + /// + public int CdefStrength + { + readonly get => this.cdefStrength; + set => this.cdefStrength = (byte)value; + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1MotionMode.cs b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1MotionMode.cs new file mode 100644 index 0000000000..ed9977b1e9 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1MotionMode.cs @@ -0,0 +1,25 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +/// +/// Identifies the motion model used to construct an AV1 inter predictor. +/// +internal enum Av1MotionMode : byte +{ + /// + /// Uses translational motion compensation without neighboring-block overlap. + /// + SimpleTranslation = 0, + + /// + /// Blends the block with predictions derived from overlapping above and left neighbors. + /// + Obmc = 1, + + /// + /// Uses a locally derived warped-motion model. + /// + Warped = 2, +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1NeighborArrayUnit.cs b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1NeighborArrayUnit.cs new file mode 100644 index 0000000000..d9d45ce5a2 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1NeighborArrayUnit.cs @@ -0,0 +1,198 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Buffers; +using SixLabors.ImageSharp.Memory; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +/// +/// Stores left and top neighbor values at the granularity required by AV1 encoder contexts. +/// +/// The context value type. +internal sealed class Av1NeighborArrayUnit : IDisposable + where T : struct +{ + /// + /// Owns the contiguous neighbor storage until this instance is disposed. + /// + private IMemoryOwner? owner; + + /// + /// The contiguous neighbor storage, whether owned directly or supplied by a picture owner. + /// + private Memory memory; + + /// + /// Indicates whether the neighbor view has been disposed. + /// + private bool isDisposed; + + /// + /// The number of context values exposed to blocks on the right. + /// + private readonly int leftLength; + + /// + /// The number of context values exposed to blocks below. + /// + private readonly int topLength; + + /// + /// Initializes a new instance of the class. + /// + /// The configuration providing the memory allocator. + /// The number of values in the left-neighbor storage. + /// The number of values in the top-neighbor storage. + public Av1NeighborArrayUnit(Configuration configuration, int leftSize, int topSize) + { + this.leftLength = leftSize; + this.topLength = topSize; + int totalLength = checked(leftSize + topSize); + + // Both context edges share the picture lifetime, so one clean allocator-backed buffer + // preserves their zero-initialized starting state without separate owner lifetimes. + this.owner = configuration.MemoryAllocator.Allocate(totalLength, AllocationOptions.Clean); + this.memory = this.owner.Memory[..totalLength]; + } + + /// + /// Initializes a new instance of the class over non-owning picture-lifetime storage. + /// + /// The contiguous left and top context storage. + /// The number of values in the left-neighbor storage. + /// The number of values in the top-neighbor storage. + public Av1NeighborArrayUnit(Memory memory, int leftSize, int topSize) + { + this.leftLength = leftSize; + this.topLength = topSize; + this.memory = memory[..checked(leftSize + topSize)]; + } + + /// + /// Selects which neighbor arrays receive an update. + /// + [Flags] + public enum UnitMask + { + /// + /// Update the left-neighbor storage. + /// + Left = 1, + + /// + /// Update the top-neighbor storage. + /// + Top = 2, + } + + /// + /// Gets the left-neighbor storage. + /// + public Span Left + { + get + { + ObjectDisposedException.ThrowIf(this.isDisposed, this); + return this.memory.Span[..this.leftLength]; + } + } + + /// + /// Gets the top-neighbor storage. + /// + public Span Top + { + get + { + ObjectDisposedException.ThrowIf(this.isDisposed, this); + return this.memory.Span.Slice(this.leftLength, this.topLength); + } + } + + /// + /// Gets or sets the base-2 logarithm of the top and left context granularity in samples. + /// + public required int GranularityNormalLog2 { get; set; } + + /// + /// Gets the left-neighbor unit index for a sample position. + /// + /// The sample position. + /// The left-neighbor unit index. + public int GetLeftIndex(Point loc) => loc.Y >> this.GranularityNormalLog2; + + /// + /// Gets the top-neighbor unit index for a sample position. + /// + /// The sample position. + /// The top-neighbor unit index. + public int GetTopIndex(Point loc) => loc.X >> this.GranularityNormalLog2; + + /// + /// Returns the neighbor storage to the configured memory allocator. + /// + public void Dispose() + { + this.owner?.Dispose(); + this.owner = null; + this.memory = Memory.Empty; + this.isDisposed = true; + } + + /// + /// Writes one context unit across the selected block edges. + /// + /// The context value to publish. + /// The block origin in samples. + /// The block dimensions in samples. + /// The neighbor arrays to update. + public void UnitModeWrite(T value, Point origin, Size blockSize, UnitMask mask) + { + if ((mask & UnitMask.Top) == UnitMask.Top) + { + // Top Neighbor Array + // ----------12345678--------------------- + // ^ ^ + // | | + // | | + // xxxxxxxx + // x x + // x x + // 12345678 + // + // The top neighbor array is updated with the samples from the + // bottom row of the source block + // + // Index = org_x + int offset = this.GetTopIndex(origin); + int count = blockSize.Width >> this.GranularityNormalLog2; + + // One packed value represents each AV1 edge unit. Filling the covered range mirrors the + // contiguous above-context update without retaining a caller-owned span. + this.Top.Slice(offset, count).Fill(value); + } + + if ((mask & UnitMask.Left) == UnitMask.Left) + { + // Left Neighbor Array + // + // | + // | + // 1 xxxxxxx1 + // 2 <---- x 2 + // 3 <---- x 3 + // 4 xxxxxxx4 + // | + // | + // + // The left neighbor array is updated with the samples from the + // right column of the source block + // + // Index = org_y + int offset = this.GetLeftIndex(origin); + int count = blockSize.Height >> this.GranularityNormalLog2; + this.Left.Slice(offset, count).Fill(value); + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1PaletteCache.cs b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1PaletteCache.cs new file mode 100644 index 0000000000..79aa93183f --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1PaletteCache.cs @@ -0,0 +1,66 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +/// +/// Builds the sorted unique palette-color cache shared by AV1 encoder and decoder syntax. +/// +internal static class Av1PaletteCache +{ + /// + /// Merges the sorted colors from the available above and left block palettes. + /// + /// The above block's sorted base colors. + /// The left block's sorted base colors. + /// The destination cache, which can hold both palettes. + /// The number of unique colors written to . + public static int Merge( + ReadOnlySpan aboveColors, + ReadOnlySpan leftColors, + Span cache) + { + int aboveIndex = 0; + int leftIndex = 0; + int count = 0; + while (aboveIndex < aboveColors.Length && leftIndex < leftColors.Length) + { + ushort aboveColor = aboveColors[aboveIndex]; + ushort leftColor = leftColors[leftIndex]; + if (leftColor < aboveColor) + { + Add(cache, ref count, leftColor); + leftIndex++; + } + else + { + Add(cache, ref count, aboveColor); + aboveIndex++; + if (leftColor == aboveColor) + { + leftIndex++; + } + } + } + + while (aboveIndex < aboveColors.Length) + { + Add(cache, ref count, aboveColors[aboveIndex++]); + } + + while (leftIndex < leftColors.Length) + { + Add(cache, ref count, leftColors[leftIndex++]); + } + + return count; + } + + private static void Add(Span cache, ref int count, ushort color) + { + if (count == 0 || cache[count - 1] != color) + { + cache[count++] = color; + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1PaletteColorMap.cs b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1PaletteColorMap.cs new file mode 100644 index 0000000000..f17e584ac6 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1PaletteColorMap.cs @@ -0,0 +1,101 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Memory; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +/// +/// Derives AV1 palette color-index ordering and entropy contexts from preceding spatial indices. +/// +internal static class Av1PaletteColorMap +{ + /// + /// Derives the palette color order, current color-order index, and entropy context for one map position. + /// + /// The partially or completely populated color-index map. + /// The current map row. + /// The current map column. + /// The number of colors in the palette. + /// The current palette index, or a negative value while decoding. + /// The destination color order for the current context. + /// The current index in , or a negative value while decoding. + /// The color-index entropy context in the range from zero through four. + public static int GetContext( + Buffer2DRegion colorIndexMap, + int row, + int column, + int paletteSize, + int colorIndex, + Span colorOrder, + out int colorOrderIndex) + { + Span scores = stackalloc int[Av1Constants.PaletteMaxSize]; + scores.Clear(); + ReadOnlySpan currentRow = colorIndexMap.DangerousGetRowSpan(row); + if (column > 0) + { + scores[currentRow[column - 1]] += 2; + } + + if (row > 0) + { + ReadOnlySpan aboveRow = colorIndexMap.DangerousGetRowSpan(row - 1); + if (column > 0) + { + scores[aboveRow[column - 1]]++; + } + + scores[aboveRow[column]] += 2; + } + + Span inverseColorOrder = stackalloc int[Av1Constants.PaletteMaxSize]; + for (int i = 0; i < Av1Constants.PaletteMaxSize; i++) + { + colorOrder[i] = (byte)i; + inverseColorOrder[i] = i; + } + + // Stable descending score order keeps lower palette indices ahead when neighboring scores tie. + for (int i = 0; i < 3; i++) + { + int maximumScore = scores[i]; + int maximumIndex = i; + for (int j = i + 1; j < paletteSize; j++) + { + if (scores[j] > maximumScore) + { + maximumScore = scores[j]; + maximumIndex = j; + } + } + + if (maximumIndex != i) + { + byte maximumColor = colorOrder[maximumIndex]; + for (int j = maximumIndex; j > i; j--) + { + scores[j] = scores[j - 1]; + colorOrder[j] = colorOrder[j - 1]; + inverseColorOrder[colorOrder[j]] = j; + } + + scores[i] = maximumScore; + colorOrder[i] = maximumColor; + inverseColorOrder[maximumColor] = i; + } + } + + colorOrderIndex = colorIndex < 0 ? -1 : inverseColorOrder[colorIndex]; + int contextHash = scores[0] + (2 * scores[1]) + (2 * scores[2]); + return contextHash switch + { + 2 => 0, + 5 => 4, + 6 => 3, + 7 => 2, + 8 => 1, + _ => -1 + }; + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1ParseAboveNeighbor4x4Context.cs b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1ParseAboveNeighbor4x4Context.cs new file mode 100644 index 0000000000..1737c29faa --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1ParseAboveNeighbor4x4Context.cs @@ -0,0 +1,162 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Buffers; +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; +using SixLabors.ImageSharp.Memory; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +/// +/// Stores entropy, partition, and transform contexts for 4-by-4 blocks above the current block. +/// +internal sealed class Av1ParseAboveNeighbor4x4Context : IDisposable +{ + /// + /// The region containing transform-width contexts. + /// + private const int TransformWidthRegionIndex = 0; + + /// + /// The region containing partition-width contexts. + /// + private const int PartitionWidthRegionIndex = 1; + + /// + /// The first region containing a color plane's coefficient contexts. + /// + private const int PlaneContextRegionStart = 2; + + /// + /// Owns the contiguous above-neighbor storage until this instance is disposed. + /// + private IMemoryOwner? memory; + + /// + /// The number of mode-information columns stored in each logical region. + /// + private readonly int contextLength; + + /// + /// Initializes a new instance of the class. + /// + /// The configuration providing the memory allocator. + /// The number of color planes. + /// The frame width in 4x4 mode-information columns. + public Av1ParseAboveNeighbor4x4Context(Configuration configuration, int planesCount, int modeInfoColumnCount) + { + this.contextLength = modeInfoColumnCount; + int regionCount = PlaneContextRegionStart + planesCount; + int totalLength = checked(regionCount * modeInfoColumnCount); + + // Every region spans the same aligned frame width and shares the tile-reader lifetime. + // One clean rent replaces the jagged array and its per-region arrays while preserving zero initialization. + this.memory = configuration.MemoryAllocator.Allocate(totalLength, AllocationOptions.Clean); + } + + /// + /// Gets a buffer holding the partition context of the previous 4x4 block row. + /// + public Span AbovePartitionWidth => this.GetRegion(PartitionWidthRegionIndex); + + /// + /// Gets a buffer holding the transform sizes of the previous 4x4 block row. + /// + public Span AboveTransformWidth => this.GetRegion(TransformWidthRegionIndex); + + /// + /// Gets the coefficient context row for the specified plane. + /// + /// The zero-based plane index. + /// The coefficient contexts for the plane. + public Span GetContext(int plane) => this.GetRegion(PlaneContextRegionStart + plane); + + /// + /// Returns the above-neighbor storage to the configured memory allocator. + /// + public void Dispose() + { + this.memory?.Dispose(); + this.memory = null; + } + + /// + /// Resets above-neighbor state for the active tile-column range. + /// + /// The sequence header describing the color planes. + /// The first mode-information column in the tile. + /// The exclusive end mode-information column in the tile. + public void Clear(ObuSequenceHeader sequenceHeader, int modeInfoColumnStart, int modeInfoColumnEnd) + { + int planeCount = sequenceHeader.ColorConfig.PlaneCount; + int width = modeInfoColumnEnd - modeInfoColumnStart; + this.AboveTransformWidth[..width].Fill(Av1TransformSize.Size64x64.GetWidth()); + this.AbovePartitionWidth[..width].Clear(); + for (int i = 0; i < planeCount; i++) + { + this.GetContext(i)[..width].Clear(); + } + } + + /// + /// Updates the above partition context for every 4x4 column covered by a block. + /// + /// The block origin in frame mode-information units. + /// The active tile boundaries. + /// The size produced by the decoded partition. + /// The parent block size. + public void UpdatePartition(Point modeInfoLocation, Av1TileInfo tileInfo, Av1BlockSize subSize, Av1BlockSize blockSize) + { + // Above contexts are tile-local even though block positions are frame-relative. + int startIndex = modeInfoLocation.X - tileInfo.ModeInfoColumnStart; + int bw = blockSize.Get4x4WideCount(); + int value = Av1PartitionContext.GetAboveContext(subSize); + + DebugGuard.MustBeLessThanOrEqualTo(startIndex, this.AboveTransformWidth.Length - bw, nameof(startIndex)); + this.AbovePartitionWidth.Slice(startIndex, bw).Fill(value); + } + + /// + /// Updates the above transform-size context for every 4x4 column covered by a block. + /// + /// The block origin in frame mode-information units. + /// The active tile boundaries. + /// The selected transform size. + /// The decoded block size. + /// A value indicating whether the block omits residual coefficients. + public void UpdateTransformation(Point modeInfoLocation, Av1TileInfo tileInfo, Av1TransformSize transformSize, Av1BlockSize blockSize, bool skip) + { + int startIndex = modeInfoLocation.X - tileInfo.ModeInfoColumnStart; + int transformWidth = transformSize.GetWidth(); + int n4w = blockSize.Get4x4WideCount(); + if (skip) + { + // Skipped blocks expose the full block width as their effective transform extent. + transformWidth = n4w << Av1Constants.ModeInfoSizeLog2; + } + + DebugGuard.MustBeLessThanOrEqualTo(startIndex, this.AboveTransformWidth.Length - n4w, nameof(startIndex)); + this.AboveTransformWidth.Slice(startIndex, n4w).Fill(transformWidth); + } + + /// + /// Clears a range of above coefficient contexts for one plane. + /// + /// The zero-based plane index. + /// The first context index to clear. + /// The number of context entries to clear. + public void ClearContext(int plane, int offset, int length) + => this.GetContext(plane).Slice(offset, length).Clear(); + + /// + /// Gets one logical row from the contiguous above-neighbor allocation. + /// + /// The zero-based logical region index. + /// The requested context row. + private Span GetRegion(int regionIndex) + { + ObjectDisposedException.ThrowIf(this.memory is null, this); + return this.memory.Memory.Span.Slice(regionIndex * this.contextLength, this.contextLength); + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1ParseLeftNeighbor4x4Context.cs b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1ParseLeftNeighbor4x4Context.cs new file mode 100644 index 0000000000..e433c2b114 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1ParseLeftNeighbor4x4Context.cs @@ -0,0 +1,161 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Buffers; +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; +using SixLabors.ImageSharp.Memory; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +/// +/// Stores entropy, partition, and transform contexts for 4-by-4 blocks left of the current block. +/// +internal sealed class Av1ParseLeftNeighbor4x4Context : IDisposable +{ + /// + /// The region containing transform-height contexts. + /// + private const int TransformHeightRegionIndex = 0; + + /// + /// The region containing partition-height contexts. + /// + private const int PartitionHeightRegionIndex = 1; + + /// + /// The first region containing a color plane's coefficient contexts. + /// + private const int PlaneContextRegionStart = 2; + + /// + /// Owns the contiguous left-neighbor storage until this instance is disposed. + /// + private IMemoryOwner? memory; + + /// + /// The number of superblock-row entries stored in each logical region. + /// + private readonly int contextLength; + + /// + /// Initializes a new instance of the class. + /// + /// The configuration providing the memory allocator. + /// The number of color planes. + /// The superblock height in 4x4 mode-information rows. + public Av1ParseLeftNeighbor4x4Context(Configuration configuration, int planesCount, int superblockModeInfoSize) + { + this.contextLength = superblockModeInfoSize; + int regionCount = PlaneContextRegionStart + planesCount; + int totalLength = checked(regionCount * superblockModeInfoSize); + + // Every region spans the same superblock height and shares the tile-reader lifetime. + // One clean rent replaces the jagged array and its per-region arrays while preserving zero initialization. + this.memory = configuration.MemoryAllocator.Allocate(totalLength, AllocationOptions.Clean); + } + + /// + /// Gets a buffer holding the partition context of the left 4x4 blocks corresponding + /// to the current super block row. + /// + public Span LeftPartitionHeight => this.GetRegion(PartitionHeightRegionIndex); + + /// + /// Gets a buffer holding the transform sizes of the left 4x4 blocks corresponding + /// to the current super block row. + /// + public Span LeftTransformHeight => this.GetRegion(TransformHeightRegionIndex); + + /// + /// Returns the left-neighbor storage to the configured memory allocator. + /// + public void Dispose() + { + this.memory?.Dispose(); + this.memory = null; + } + + /// + /// Resets all left-neighbor state for a new superblock row. + /// + /// The sequence header describing the superblock size and color planes. + public void Clear(ObuSequenceHeader sequenceHeader) + { + int blockCount = sequenceHeader.SuperblockModeInfoSize; + int planeCount = sequenceHeader.ColorConfig.PlaneCount; + this.LeftTransformHeight[..blockCount].Fill(Av1TransformSize.Size64x64.GetHeight()); + this.LeftPartitionHeight[..blockCount].Clear(); + for (int i = 0; i < planeCount; i++) + { + this.GetContext(i)[..blockCount].Clear(); + } + } + + /// + /// Updates the left partition context for every 4x4 row covered by a block. + /// + /// The block origin in frame mode-information units. + /// The active superblock location. + /// The size produced by the decoded partition. + /// The parent block size. + public void UpdatePartition(Point modeInfoLocation, Av1SuperblockInfo superblockInfo, Av1BlockSize subSize, Av1BlockSize blockSize) + { + // The left context is reused for each superblock row, so address it relative to the superblock origin. + int startIndex = (modeInfoLocation.Y - superblockInfo.ModeInfoPosition.Y) & Av1PartitionContext.Mask; + int bh = blockSize.Get4x4HighCount(); + int value = Av1PartitionContext.GetLeftContext(subSize); + DebugGuard.MustBeLessThanOrEqualTo(startIndex, this.LeftPartitionHeight.Length - bh, nameof(startIndex)); + this.LeftPartitionHeight.Slice(startIndex, bh).Fill(value); + } + + /// + /// Updates the left transform-size context for every 4x4 row covered by a block. + /// + /// The block origin in frame mode-information units. + /// The active superblock location. + /// The selected transform size. + /// The decoded block size. + /// A value indicating whether the block omits residual coefficients. + public void UpdateTransformation(Point modeInfoLocation, Av1SuperblockInfo superblockInfo, Av1TransformSize transformSize, Av1BlockSize blockSize, bool skip) + { + int startIndex = modeInfoLocation.Y - superblockInfo.ModeInfoPosition.Y; + int transformHeight = transformSize.GetHeight(); + int n4h = blockSize.Get4x4HighCount(); + if (skip) + { + // Skipped blocks expose the full block height as their effective transform extent. + transformHeight = n4h << Av1Constants.ModeInfoSizeLog2; + } + + DebugGuard.MustBeLessThanOrEqualTo(startIndex, this.LeftTransformHeight.Length - n4h, nameof(startIndex)); + this.LeftTransformHeight.Slice(startIndex, n4h).Fill(transformHeight); + } + + /// + /// Clears a range of left coefficient contexts for one plane. + /// + /// The zero-based plane index. + /// The first context index to clear. + /// The number of context entries to clear. + public void ClearContext(int plane, int offset, int length) + => this.GetContext(plane).Slice(offset, length).Clear(); + + /// + /// Gets the coefficient context column for the specified plane. + /// + /// The zero-based plane index. + /// The coefficient contexts for the plane. + public Span GetContext(int plane) => this.GetRegion(PlaneContextRegionStart + plane); + + /// + /// Gets one logical column from the contiguous left-neighbor allocation. + /// + /// The zero-based logical region index. + /// The requested context column. + private Span GetRegion(int regionIndex) + { + ObjectDisposedException.ThrowIf(this.memory is null, this); + return this.memory.Memory.Span.Slice(regionIndex * this.contextLength, this.contextLength); + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1PartitionContext.cs b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1PartitionContext.cs new file mode 100644 index 0000000000..ef7e9b738a --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1PartitionContext.cs @@ -0,0 +1,85 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Numerics; +using System.Runtime.InteropServices; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +/// +/// Stores the above and left five-bit AV1 partition contexts for a mode-information position. +/// +/// +/// Each set bit records a split at one block-size level. For example, 11111 records splits from +/// 128 by 128 through 8 by 8, while 10000 records only the 128 by 128 split. +/// +[StructLayout(LayoutKind.Sequential, Size = StorageSize)] +internal struct Av1PartitionContext : IMinMaxValue +{ + /// + /// The packed size of the above and left context bytes. + /// + public const int StorageSize = 2; + + /// + /// Maps each block size to the five-bit context stored for an above neighbor. + /// + private static readonly int[] AboveLookup = + [31, 31, 30, 30, 30, 28, 28, 28, 24, 24, 24, 16, 16, 16, 0, 0, 31, 28, 30, 24, 28, 16]; + + /// + /// Maps each block size to the five-bit context stored for a left neighbor. + /// + private static readonly int[] LeftLookup = + [31, 30, 31, 30, 28, 30, 28, 24, 28, 24, 16, 24, 16, 0, 16, 0, 28, 31, 24, 30, 16, 28]; + + /// + /// The mask used to convert a frame mode-information row to its position within a 128-sample superblock. + /// + public const int Mask = (1 << (7 - 2)) - 1; + + /// + /// Initializes a new instance of the struct. + /// + /// The context stored for blocks below this block. + /// The context stored for blocks to the right of this block. + public Av1PartitionContext(byte above, byte left) + { + this.Above = above; + this.Left = left; + } + + /// + /// Gets the sentinel partition context used for an unpopulated neighbor. + /// + public static Av1PartitionContext MaxValue => new(byte.MaxValue, byte.MaxValue); + + /// + /// Gets the partition context with no split levels recorded. + /// + public static Av1PartitionContext MinValue => default; + + /// + /// Gets or sets the five-bit context derived from the left neighbor. + /// + public byte Left { get; set; } + + /// + /// Gets or sets the five-bit context derived from the above neighbor. + /// + public byte Above { get; set; } + + /// + /// Gets the above-neighbor partition context for the specified block size. + /// + /// The block size. + /// The five-bit above-neighbor context. + public static int GetAboveContext(Av1BlockSize blockSize) => AboveLookup[(int)blockSize]; + + /// + /// Gets the left-neighbor partition context for the specified block size. + /// + /// The block size. + /// The five-bit left-neighbor context. + public static int GetLeftContext(Av1BlockSize blockSize) => LeftLookup[(int)blockSize]; +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1PartitionInfo.cs b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1PartitionInfo.cs new file mode 100644 index 0000000000..d2e9ed4f85 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1PartitionInfo.cs @@ -0,0 +1,386 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Diagnostics.CodeAnalysis; +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.ChromaFromLuma; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +/// +/// Describes a decoded AV1 partition's block geometry, neighbors, and frame-boundary availability. +/// +internal ref struct Av1PartitionInfo +{ + /// + /// The decoded mode information populated for this partition. + /// + private Av1BlockModeInfo modeInfo; + + /// + /// The luma block width in samples. + /// + private int lumaWidthInPixels; + + /// + /// The shared chroma block width in samples. + /// + private int chromaWidthInPixels; + + /// + /// The luma block height in samples. + /// + private int lumaHeightInPixels; + + /// + /// The shared chroma block height in samples. + /// + private int chromaHeightInPixels; + + /// + /// Initializes a new instance of the structure. + /// + /// The decoded mode information for the partition block. + /// The containing superblock. + /// A value indicating whether the partition has chroma samples. + /// The partition type that produced the block. + public Av1PartitionInfo(Av1BlockModeInfo modeInfo, Av1SuperblockInfo superblockInfo, bool isChroma, Av1PartitionType partitionType) + { + this.modeInfo = modeInfo; + this.SuperblockInfo = superblockInfo; + this.IsChroma = isChroma; + this.Type = partitionType; + } + + /// + /// Gets the decoded block mode information. + /// + [UnscopedRef] + public ref Av1BlockModeInfo ModeInfo => ref this.modeInfo; + + /// + /// Gets the this partition resides inside. + /// + public Av1SuperblockInfo SuperblockInfo { get; } + + /// + /// Gets a value indicating whether the partition has chroma samples at its current luma position. + /// + public bool IsChroma { get; } + + /// + /// Gets the partition type that produced the block. + /// + public Av1PartitionType Type { get; } + + /// + /// Gets or sets a value indicating whether the information from the block above can be used on the luma plane. + /// + public bool AvailableAbove { get; set; } + + /// + /// Gets or sets a value indicating whether the information from the block left can be used on the luma plane. + /// + public bool AvailableLeft { get; set; } + + /// + /// Gets or sets a value indicating whether the information from the block above can be used on the chroma plane. + /// + public bool AvailableAboveForChroma { get; set; } + + /// + /// Gets or sets a value indicating whether the information from the block left can be used on the chroma plane. + /// + public bool AvailableLeftForChroma { get; set; } + + /// + /// Gets or sets the horizontal location of the block in units of 4x4 luma samples. + /// + public int ColumnIndex { get; set; } + + /// + /// Gets or sets the vertical location of the block in units of 4x4 luma samples. + /// + public int RowIndex { get; set; } + + /// + /// Gets or sets the mode information covering the immediately above luma neighbor. + /// + public Av1BlockModeInfo? AboveModeInfo { get; set; } + + /// + /// Gets or sets the mode information covering the immediately left luma neighbor. + /// + public Av1BlockModeInfo? LeftModeInfo { get; set; } + + /// + /// Gets or sets the mode information covering the above chroma neighbor. + /// + public Av1BlockModeInfo? AboveModeInfoForChroma { get; set; } + + /// + /// Gets or sets the mode information covering the left chroma neighbor. + /// + public Av1BlockModeInfo? LeftModeInfoForChroma { get; set; } + + /// + /// Gets the reference-frame types selected for the block. + /// + [UnscopedRef] + public Span ReferenceFrames => this.ModeInfo.ReferenceFrames; + + /// + /// Gets the signed distance from the block to the left frame edge in one-eighth-sample units. + /// + public int ModeBlockToLeftEdge { get; private set; } + + /// + /// Gets the signed distance from the block to the right frame edge in one-eighth-sample units. + /// + public int ModeBlockToRightEdge { get; private set; } + + /// + /// Gets the signed distance from the block to the top frame edge in one-eighth-sample units. + /// + public int ModeBlockToTopEdge { get; private set; } + + /// + /// Gets the signed distance from the block to the bottom frame edge in one-eighth-sample units. + /// + public int ModeBlockToBottomEdge { get; private set; } + + /// + /// Gets or sets the neighboring luma samples used by chroma-from-luma prediction. + /// + public Av1ChromaFromLumaContext? ChromaFromLumaContext { get; set; } + + /// + /// Gets the block width in samples for a color plane. + /// + /// The luma, blue-difference, or red-difference plane. + /// The block width in samples for the requested plane. + public int GetWidthInPixels(Av1Plane plane) => plane == Av1Plane.Y ? this.lumaWidthInPixels : this.chromaWidthInPixels; + + /// + /// Gets the block height in samples for a color plane. + /// + /// The luma, blue-difference, or red-difference plane. + /// The block height in samples for the requested plane. + public int GetHeightInPixels(Av1Plane plane) => plane == Av1Plane.Y ? this.lumaHeightInPixels : this.chromaHeightInPixels; + + /// + /// Computes tile-neighbor availability, frame-edge distances, and per-plane block dimensions. + /// + /// The sequence header describing color subsampling. + /// The frame header describing coded dimensions. + /// The active tile boundaries. + public void ComputeBoundaryOffsets(ObuSequenceHeader sequenceHeader, ObuFrameHeader frameHeader, Av1TileInfo tileInfo) + { + Av1BlockSize blockSize = this.ModeInfo.BlockSize; + int bw4 = blockSize.Get4x4WideCount(); + int bh4 = blockSize.Get4x4HighCount(); + int subX = sequenceHeader.ColorConfig.SubSamplingX ? 1 : 0; + int subY = sequenceHeader.ColorConfig.SubSamplingY ? 1 : 0; + this.AvailableAbove = this.RowIndex > tileInfo.ModeInfoRowStart; + this.AvailableLeft = this.ColumnIndex > tileInfo.ModeInfoColumnStart; + this.AvailableAboveForChroma = this.AvailableAbove; + this.AvailableLeftForChroma = this.AvailableLeft; + + int shift = Av1Constants.ModeInfoSizeLog2 + 3; + this.ModeBlockToLeftEdge = -this.ColumnIndex << shift; + this.ModeBlockToRightEdge = (frameHeader.ModeInfoColumnCount - bw4 - this.ColumnIndex) << shift; + this.ModeBlockToTopEdge = -this.RowIndex << shift; + this.ModeBlockToBottomEdge = (frameHeader.ModeInfoRowCount - bh4 - this.RowIndex) << shift; + + // The bitstream expresses block size on the luma grid. Chroma dimensions are derived by + // subsampling that grid while retaining at least one 4x4 chroma unit for narrow blocks. + const int modeInfoSize = 1 << Av1Constants.ModeInfoSizeLog2; + this.lumaWidthInPixels = bw4 * modeInfoSize; + this.lumaHeightInPixels = bh4 * modeInfoSize; + this.chromaWidthInPixels = Math.Max(1, bw4 >> subX) * modeInfoSize; + this.chromaHeightInPixels = Math.Max(1, bh4 >> subY) * modeInfoSize; + } + + /// + /// Resolves the decoded luma and chroma mode information for available above and left neighbors. + /// + /// The color-plane subsampling configuration. + public void PopulateModeInfoNeighbors(ObuColorConfig colorConfig) + { + if (this.AvailableAbove) + { + this.AboveModeInfo = this.SuperblockInfo.GetModeInfoAt(new Point(this.ColumnIndex, this.RowIndex - 1)); + } + + if (this.AvailableLeft) + { + this.LeftModeInfo = this.SuperblockInfo.GetModeInfoAt(new Point(this.ColumnIndex - 1, this.RowIndex)); + } + + if (!this.IsChroma) + { + return; + } + + int subX = colorConfig.SubSamplingX ? 1 : 0; + int subY = colorConfig.SubSamplingY ? 1 : 0; + int chromaBaseColumn = this.ColumnIndex - (this.ColumnIndex & subX); + int chromaBaseRow = this.RowIndex - (this.RowIndex & subY); + + // Chroma neighbors refer to the bottom-right luma mode covered by each adjacent chroma block. + if (this.AvailableAboveForChroma) + { + this.AboveModeInfoForChroma = this.SuperblockInfo.GetModeInfoAt(new Point(chromaBaseColumn + subX, chromaBaseRow - 1)); + } + + if (this.AvailableLeftForChroma) + { + this.LeftModeInfoForChroma = this.SuperblockInfo.GetModeInfoAt(new Point(chromaBaseColumn - 1, chromaBaseRow + subY)); + } + } + + /// + /// Gets the luma transform type covering a transform position from a subsampled chroma plane. + /// + /// The transform position in 4x4 units of the target plane. + /// Indicates whether the target plane is horizontally subsampled. + /// Indicates whether the target plane is vertically subsampled. + /// The transform type decoded at the corresponding luma-grid position. + public Av1TransformType GetLumaTransformType(Point planePosition, bool subX, bool subY) + { + int lumaColumn = planePosition.X << (subX ? 1 : 0); + int lumaRow = planePosition.Y << (subY ? 1 : 0); + int first = this.ModeInfo.GetFirstTransformLocation(Av1Plane.Y); + int count = this.ModeInfo.GetTransformUnitCount(Av1Plane.Y); + Span lumaTransforms = this.SuperblockInfo.GetTransformInfoY().Slice(first, count); + foreach (Av1TransformInfo transform in lumaTransforms) + { + int width = transform.Size.Get4x4WideCount(); + int height = transform.Size.Get4x4HighCount(); + if (lumaColumn >= transform.OffsetX && lumaColumn < transform.OffsetX + width && + lumaRow >= transform.OffsetY && lumaRow < transform.OffsetY + height) + { + return transform.Type; + } + } + + throw new InvalidImageContentException("Missing luma transform for inter-predicted chroma."); + } + + /// + /// Gets the block width clipped to the right frame edge. + /// + /// The luma block size. + /// A value indicating whether the target plane is horizontally subsampled. + /// The clipped width in 4x4 units of the target plane. + public int GetMaxBlockWide(Av1BlockSize blockSize, bool subX) + { + int maxBlockWide = blockSize.GetWidth(); + if (this.ModeBlockToRightEdge < 0) + { + int shift = subX ? 4 : 3; + maxBlockWide += this.ModeBlockToRightEdge >> shift; + } + + return maxBlockWide >> 2; + } + + /// + /// Gets the block height clipped to the bottom frame edge. + /// + /// The luma block size. + /// A value indicating whether the target plane is vertically subsampled. + /// The clipped height in 4x4 units of the target plane. + public int GetMaxBlockHigh(Av1BlockSize blockSize, bool subY) + { + int maxBlockHigh = blockSize.GetHeight(); + if (this.ModeBlockToBottomEdge < 0) + { + int shift = subY ? 4 : 3; + maxBlockHigh += this.ModeBlockToBottomEdge >> shift; + } + + return maxBlockHigh >> 2; + } + + /// + /// Determines whether the current block can use the block at its top-right search position. + /// + /// The superblock width in 4x4 mode-information units. + /// when the top-right block has already been decoded; otherwise, . + public bool HasTopRight(int superblockModeInfoSize) => HasTopRight( + this.ModeInfo.BlockSize, + this.Type, + this.RowIndex, + this.ColumnIndex, + superblockModeInfoSize); + + /// + /// Determines whether a block can use its top-right search position from raster and partition geometry. + /// + /// The current block size. + /// The partition type that produced the block. + /// The block row in 4x4 mode-information units. + /// The block column in 4x4 mode-information units. + /// The superblock width in 4x4 mode-information units. + /// when the top-right block precedes the current block; otherwise, . + internal static bool HasTopRight( + Av1BlockSize blockSize, + Av1PartitionType partitionType, + int rowIndex, + int columnIndex, + int superblockModeInfoSize) + { + int width = blockSize.Get4x4WideCount(); + int height = blockSize.Get4x4HighCount(); + int squareSize = Math.Max(width, height); + if (squareSize > 16) + { + return false; + } + + int row = rowIndex & (superblockModeInfoSize - 1); + int column = columnIndex & (superblockModeInfoSize - 1); + bool hasTopRight = !((row & squareSize) != 0 && (column & squareSize) != 0); + int traversalSize = squareSize; + + // Split partitions decode three quadrants before the bottom-right quadrant. Walking the enclosing split levels + // excludes a right-hand block whenever traversal has not reached that block yet. + while (traversalSize < superblockModeInfoSize) + { + if ((column & traversalSize) == 0) + { + break; + } + + if ((column & (traversalSize << 1)) != 0 && (row & (traversalSize << 1)) != 0) + { + hasTopRight = false; + break; + } + + traversalSize <<= 1; + } + + // Rectangular partitions override the square traversal rule because their sub-blocks are decoded along the + // long axis. Earlier vertical rectangles have a completed row above; later horizontal rectangles do not. + if (width < height && ((columnIndex + width) & (height - 1)) != 0) + { + hasTopRight = true; + } + + if (width > height && (rowIndex & (width - 1)) != 0) + { + hasTopRight = false; + } + + // The lower-left square of a vertical-A partition precedes its right-hand rectangle in bitstream order. + if (partitionType == Av1PartitionType.VerticalA && width == height && (row & traversalSize) != 0) + { + hasTopRight = false; + } + + return hasTopRight; + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1PictureControlSet.cs b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1PictureControlSet.cs new file mode 100644 index 0000000000..aa96f70b98 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1PictureControlSet.cs @@ -0,0 +1,207 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1.Motion; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +/// +/// Holds the coding decisions, buffers, and sequence context for one AV1 picture pass. +/// +internal class Av1PictureControlSet +{ + /// + /// Gets or sets the partition neighbor contexts for each tile. + /// + public required Av1NeighborArrayUnit[] PartitionContexts { get; set; } + + /// + /// Gets or sets the luma DC-sign and coefficient-level neighbor contexts for each tile. + /// + public required Av1NeighborArrayUnit[] LuminanceDcSignLevelCoefficientNeighbors { get; set; } + + /// + /// Gets or sets the red-difference chroma DC-sign and coefficient-level neighbor contexts for each tile. + /// + public required Av1NeighborArrayUnit[] CrDcSignLevelCoefficientNeighbors { get; set; } + + /// + /// Gets or sets the blue-difference chroma DC-sign and coefficient-level neighbor contexts for each tile. + /// + public required Av1NeighborArrayUnit[] CbDcSignLevelCoefficientNeighbors { get; set; } + + /// + /// Gets or sets the transform-function neighbor contexts for each tile. + /// + public required Av1NeighborArrayUnit[] TransformFunctionContexts { get; set; } + + /// + /// Gets or sets the palette sizes and base colors exposed by the above and left block edges for each tile. + /// + public Av1NeighborArrayUnit[] PaletteContexts { get; set; } = []; + + /// + /// Gets or sets the sequence-wide encoder state. + /// + public required Av1SequenceControlSet Sequence { get; set; } + + /// + /// Gets or sets the parent picture state shared across coding passes. + /// + public required Av1PictureParentControlSet Parent { get; set; } + + /// + /// Gets or sets the frame segmentation identifiers used for spatial prediction. + /// + public required Memory SegmentationNeighborMap { get; set; } + + /// + /// Gets or sets the frame grid that maps each 4x4 position to its mode-information allocation index. + /// + public required Memory ModeInfoGrid { get; set; } + + /// + /// Gets or sets the contiguous mode-information storage addressed by . + /// + public required Memory ModeInfoAllocation { get; set; } + + /// + /// Gets or sets the packed displacement vectors present only when the frame permits intra-block copy. + /// + public Memory DisplacementVectors { get; set; } + + /// + /// Gets or sets the non-owning visible-frame hash index used by intra-block-copy motion search. + /// + public Av1IntraBlockCopySearchIndex IntraBlockCopySearch { get; set; } + + /// + /// Gets or sets the row stride of in 4x4 mode-information units. + /// + public int ModeInfoStride { get; set; } + + /// + /// Gets or sets a value indicating whether the mode-information backing store uses 8x8 rather than 4x4 granularity. + /// + public bool Disallow4x4AllFrames { get; set; } + + /// + /// Gets or sets the constrained directional enhancement filter presets for each tile. + /// Each tile occupies consecutive entries. + /// + public required Memory CdefPreset { get; set; } + + /// + /// Gets or sets the starting byte of each tile in the shared encoded output buffer. + /// + public required Memory TileDataOffsets { get; set; } + + /// + /// Gets or sets the encoded byte length of each tile. + /// + public required Memory TileDataLengths { get; set; } + + /// + /// Gets the mode-information entry mapped to a frame position. + /// + /// The frame position in 4x4 mode-information units. + /// A reference to the mapped mode-information entry. + public ref Av1MacroBlockModeInfo GetFromModeInfoGrid(Point position) + { + int gridOffset = (position.Y * this.ModeInfoStride) + position.X; + int allocationOffset = this.ModeInfoGrid.Span[gridOffset]; + return ref this.ModeInfoAllocation.Span[allocationOffset]; + } + + /// + /// Gets the displacement vector mapped to a frame position. + /// + /// The frame position in 4x4 mode-information units. + /// The displacement vector retained for the covering block. + public Av1MotionVector GetDisplacementVector(Point position) + { + int gridOffset = (position.Y * this.ModeInfoStride) + position.X; + int allocationOffset = this.ModeInfoGrid.Span[gridOffset]; + Av1EncoderDisplacementVector vector = this.DisplacementVectors.Span[allocationOffset]; + return new Av1MotionVector(vector.Row, vector.Column); + } + + /// + /// Stores the displacement vector selected at a block origin. + /// + /// The block position in 4x4 mode-information units. + /// The selected integer displacement vector. + public void SetDisplacementVector(Point modeInfoPosition, Av1MotionVector vector) + { + int modeInfoStride = this.ModeInfoStride; + int disallow4x4 = this.Disallow4x4AllFrames ? 1 : 0; + int allocationOffset = ((modeInfoPosition.Y >> disallow4x4) * (modeInfoStride >> disallow4x4)) + (modeInfoPosition.X >> disallow4x4); + this.DisplacementVectors.Span[allocationOffset] = new Av1EncoderDisplacementVector + { + Row = (short)vector.Row, + Column = (short)vector.Column + }; + } + + /// + /// Gets the macroblock mode information allocated at a block origin. + /// + /// The block position in 4x4 mode-information units. + /// A reference to the macroblock mode information at the origin. + public ref Av1MacroBlockModeInfo GetMacroBlockModeInfo(Point modeInfoPosition) + { + int modeInfoStride = this.ModeInfoStride; + int disallow4x4 = this.Disallow4x4AllFrames ? 1 : 0; + int allocationOffset = ((modeInfoPosition.Y >> disallow4x4) * (modeInfoStride >> disallow4x4)) + (modeInfoPosition.X >> disallow4x4); + return ref this.ModeInfoAllocation.Span[allocationOffset]; + } + + /// + /// Maps every coded 4x4 position covered by a block to the block's mode-information allocation entry. + /// + /// The block position in 4x4 mode-information units. + /// The coded block size. + public void MapModeInfoBlock(Point modeInfoPosition, Av1BlockSize blockSize) + { + int modeInfoStride = this.ModeInfoStride; + int disallow4x4 = this.Disallow4x4AllFrames ? 1 : 0; + int allocationOffset = ((modeInfoPosition.Y >> disallow4x4) * (modeInfoStride >> disallow4x4)) + (modeInfoPosition.X >> disallow4x4); + int mappedWidth = Math.Min(this.Parent.Common.ModeInfoColumnCount - modeInfoPosition.X, blockSize.Get4x4WideCount()); + int mappedHeight = Math.Min(this.Parent.Common.ModeInfoRowCount - modeInfoPosition.Y, blockSize.Get4x4HighCount()); + Span grid = this.ModeInfoGrid.Span; + + // Libaom's pointer grid aliases every covered 4x4 entry to one mode-info allocation. Integer indices keep + // the same aliasing without one managed object and one managed reference per grid position. + for (int row = 0; row < mappedHeight; row++) + { + int gridOffset = ((modeInfoPosition.Y + row) * modeInfoStride) + modeInfoPosition.X; + grid.Slice(gridOffset, mappedWidth).Fill(allocationOffset); + } + } + + /// + /// Writes a segment identifier to every segmentation-map entry covered by a block. + /// + /// The block size. + /// The block origin in samples. + /// The segment identifier. + public void UpdateSegmentation(Av1BlockSize blockSize, Point origin, int segmentId) + { + Av1EncoderCommon cm = this.Parent.Common; + Span segment_ids = this.SegmentationNeighborMap.Span; + int mi_col = origin.X >> Av1Constants.ModeInfoSizeLog2; + int mi_row = origin.Y >> Av1Constants.ModeInfoSizeLog2; + int mi_offset = (mi_row * cm.ModeInfoColumnCount) + mi_col; + int bw = blockSize.Get4x4WideCount(); + int bh = blockSize.Get4x4HighCount(); + int xmis = Math.Min(cm.ModeInfoColumnCount - mi_col, bw); + int ymis = Math.Min(cm.ModeInfoRowCount - mi_row, bh); + + // Clip edge blocks to the coded mode-information grid before filling complete rows. + for (int y = 0; y < ymis; ++y) + { + int offset = mi_offset + (y * cm.ModeInfoColumnCount); + segment_ids.Slice(offset, xmis).Fill((byte)segmentId); + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1PictureParentControlSet.cs b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1PictureParentControlSet.cs new file mode 100644 index 0000000000..108102fb2d --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1PictureParentControlSet.cs @@ -0,0 +1,32 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +/// +/// Holds encoder state that is shared by all coding passes for one AV1 picture. +/// +internal class Av1PictureParentControlSet +{ + /// + /// Gets or sets frame dimensions and tile state shared by encoder stages. + /// + public required Av1EncoderCommon Common { get; set; } + + /// + /// Gets or sets the frame header being encoded. + /// + public required ObuFrameHeader FrameHeader { get; set; } + + /// + /// Gets or sets the preceding quantizer index for each tile context. + /// + public required Memory PreviousQIndex { get; set; } + + /// + /// Gets or sets the encoder palette-search level. + /// + public int PaletteLevel { get; set; } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1PlaneType.cs b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1PlaneType.cs new file mode 100644 index 0000000000..f6b70e5425 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1PlaneType.cs @@ -0,0 +1,20 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +/// +/// Identifies whether AV1 block processing targets the luma plane or either chroma plane. +/// +internal enum Av1PlaneType : int +{ + /// + /// The luma plane. + /// + Y, + + /// + /// Either chroma plane. + /// + Uv +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1ReferenceFrameType.cs b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1ReferenceFrameType.cs new file mode 100644 index 0000000000..ffc1c98afc --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1ReferenceFrameType.cs @@ -0,0 +1,55 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +/// +/// Identifies the current or retained frame used to predict an AV1 coding block. +/// +internal enum Av1ReferenceFrameType : sbyte +{ + /// + /// Indicates that the optional secondary reference is absent. + /// + None = -1, + + /// + /// References the current frame for intra prediction. + /// + Intra = 0, + + /// + /// References the most recent forward prediction frame. + /// + Last = 1, + + /// + /// References the second most recent forward prediction frame. + /// + Last2 = 2, + + /// + /// References the third most recent forward prediction frame. + /// + Last3 = 3, + + /// + /// References the long-term golden forward prediction frame. + /// + Golden = 4, + + /// + /// References the nearest backward prediction frame. + /// + Backward = 5, + + /// + /// References the secondary alternate backward prediction frame. + /// + Alternate2 = 6, + + /// + /// References the alternate backward prediction frame. + /// + Alternate = 7, +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1RestorationFilterType.cs b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1RestorationFilterType.cs new file mode 100644 index 0000000000..6f99bbc6b9 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1RestorationFilterType.cs @@ -0,0 +1,25 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +/// +/// Identifies the filter selected for one AV1 loop-restoration unit. +/// +internal enum Av1RestorationFilterType +{ + /// + /// Leaves the restoration-unit samples unchanged. + /// + None, + + /// + /// Applies the separable Wiener restoration filter. + /// + Wiener, + + /// + /// Applies self-guided restoration projection. + /// + SgrProjection, +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1SequenceControlSet.cs b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1SequenceControlSet.cs new file mode 100644 index 0000000000..907205ae77 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1SequenceControlSet.cs @@ -0,0 +1,22 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +/// +/// Holds encoder configuration and sequence-wide state shared by AV1 pictures. +/// +internal class Av1SequenceControlSet +{ + /// + /// Gets or sets the sequence header that governs encoded pictures. + /// + public required ObuSequenceHeader SequenceHeader { get; set; } + + /// + /// Gets or sets the maximum number of encoded blocks allocated for a picture. + /// + public int MaxBlockCount { get; set; } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1Superblock.cs b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1Superblock.cs new file mode 100644 index 0000000000..73d38af89c --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1Superblock.cs @@ -0,0 +1,35 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +/// +/// Holds encoder-side block decisions and transform data for one AV1 superblock. +/// +internal class Av1Superblock +{ + /// + /// Gets or sets the reusable final-block and partition-decision workspace. + /// + public required Av1EncoderSuperblockWorkspace Workspace { get; set; } + + /// + /// Gets the final encoder decisions in partition traversal order. + /// + public Span FinalBlocks => this.Workspace.FinalBlocks; + + /// + /// Gets or sets the tile containing the superblock. + /// + public required Av1TileInfo TileInfo { get; set; } + + /// + /// Gets the selected partition type for each partition-tree node. + /// + public Span CodingUnitPartitionTypes => this.Workspace.PartitionTypes; + + /// + /// Gets or sets the superblock index within the picture. + /// + public int Index { get; set; } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1SuperblockInfo.cs b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1SuperblockInfo.cs new file mode 100644 index 0000000000..638cef860b --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1SuperblockInfo.cs @@ -0,0 +1,129 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +/// +/// Stores the partition tree and decoded mode information for one AV1 superblock. +/// +internal readonly struct Av1SuperblockInfo +{ + /// + /// Provides the frame-owned state and current-superblock scratch used by this view. + /// + private readonly Av1FrameInfo frameInfo; + + /// + /// Initializes a new instance of the struct. + /// + /// The owning frame information. + /// The superblock position in the frame superblock grid. + public Av1SuperblockInfo(Av1FrameInfo frameInfo, Point position) + { + this.Position = position; + this.frameInfo = frameInfo; + } + + /// + /// Gets the position of this superblock in the frame superblock grid. + /// + public Point Position { get; } + + /// + /// Gets the frame-relative superblock origin in 4x4 mode-information units. + /// + public Point ModeInfoPosition => this.Position * this.frameInfo.SuperblockModeInfoSize; + + /// + /// Gets a reference to the active base quantizer index for this superblock. + /// + public ref int SuperblockQuantizerIndex => ref this.frameInfo.GetQuantizerIndex(this.Position); + + /// + /// Gets the mode information that covers the superblock origin. + /// + public Av1BlockModeInfo SuperblockModeInfo => this.GetModeInfo(new Point(0, 0)); + + /// + /// Gets the luma coefficient scratch for the current superblock. + /// + public Span CoefficientsY => this.frameInfo.GetCoefficientsY(); + + /// + /// Gets the blue-difference chroma coefficient scratch for the current superblock. + /// + public Span CoefficientsU => this.frameInfo.GetCoefficientsU(); + + /// + /// Gets the red-difference chroma coefficient scratch for the current superblock. + /// + public Span CoefficientsV => this.frameInfo.GetCoefficientsV(); + + /// + /// Gets the constrained directional enhancement filter strengths for this superblock. + /// + public Span CdefStrength => this.frameInfo.GetCdefStrength(this.Position); + + /// + /// Gets the loop-filter deltas for this superblock. + /// + public Span SuperblockDeltaLoopFilter => this.frameInfo.GetDeltaLoopFilter(this.Position); + + /// + /// Gets the number of mode-information records parsed for this superblock. + /// + public int BlockCount => this.frameInfo.GetModeInfoCount(this.Position); + + /// + /// Gets the luma transform-information scratch for the current superblock. + /// + /// The current-superblock luma transform-information span. + public Span GetTransformInfoY() => this.frameInfo.GetSuperblockTransformY(); + + /// + /// Gets the shared chroma transform-information scratch for the current superblock. + /// + /// The current-superblock chroma transform-information span. + public Span GetTransformInfoUv() => this.frameInfo.GetSuperblockTransformUv(); + + /// + /// Gets the transform-information storage for the specified color plane. + /// + /// The zero-based color-plane index. + /// The transform-information span for the plane. + public Span GetTransformInfo(int plane) => this.frameInfo.GetSuperblockTransform(plane); + + /// + /// Gets the mode information records parsed for this superblock in bitstream order. + /// + /// The mode information records for the superblock. + public Av1FrameInfo.ModeInfoCollection GetModeInfos() => + this.frameInfo.GetModeInfos(this.Position, this.BlockCount); + + /// + /// Gets the mode information covering a position relative to this superblock. + /// + /// The position in 4x4 mode-information units relative to the superblock. + /// The mode information covering the position. + public Av1BlockModeInfo GetModeInfo(Point index) => this.frameInfo.GetModeInfo(this.Position, index); + + /// + /// Gets the mode information covering a frame-relative position. + /// + /// The frame-relative position in 4x4 mode-information units. + /// The mode information covering the position. + public Av1BlockModeInfo GetModeInfoAt(Point index) => this.frameInfo.GetModeInfoAt(index); + + /// + /// Gets the coefficient storage for the specified color plane. + /// + /// The color plane. + /// The coefficient span for the plane, or an empty span for an unsupported value. + public Span GetCoefficients(Av1Plane plane) => plane switch + { + Av1Plane.Y => this.CoefficientsY, + Av1Plane.U => this.CoefficientsU, + Av1Plane.V => this.CoefficientsV, + _ => [] + }; +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1TileInfo.cs b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1TileInfo.cs new file mode 100644 index 0000000000..26e0e95268 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1TileInfo.cs @@ -0,0 +1,92 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +/// +/// Describes one AV1 tile's superblock and mode-information boundaries. +/// +internal sealed class Av1TileInfo +{ + /// + /// Initializes a new instance of the class for the specified tile coordinates. + /// + /// The tile row index. + /// The tile column index. + /// The frame header that defines the tile layout. + public Av1TileInfo(int row, int column, ObuFrameHeader frameHeader) + { + this.SetTileRow(frameHeader.TilesInfo, frameHeader.ModeInfoRowCount, row); + this.SetTileColumn(frameHeader.TilesInfo, frameHeader.ModeInfoColumnCount, column); + } + + /// + /// Initializes a new instance of the class by copying another tile description. + /// + /// The tile description to copy. + public Av1TileInfo(Av1TileInfo tileInfo) + { + this.ModeInfoColumnStart = tileInfo.ModeInfoColumnStart; + this.ModeInfoColumnEnd = tileInfo.ModeInfoColumnEnd; + this.ModeInfoRowStart = tileInfo.ModeInfoRowStart; + this.ModeInfoRowEnd = tileInfo.ModeInfoRowEnd; + this.TileIndex = tileInfo.TileIndex; + } + + /// + /// Gets the first mode-information row in the tile. + /// + public int ModeInfoRowStart { get; private set; } + + /// + /// Gets the exclusive end mode-information row in the tile. + /// + public int ModeInfoRowEnd { get; private set; } + + /// + /// Gets the first mode-information column in the tile. + /// + public int ModeInfoColumnStart { get; private set; } + + /// + /// Gets the exclusive end mode-information column in the tile. + /// + public int ModeInfoColumnEnd { get; private set; } + + /// + /// Gets the tile column and row indices. + /// + public Point TileIndex { get; private set; } + + /// + /// Selects the tile row and updates its mode-information boundaries. + /// + /// The tile layout. + /// The coded frame height in mode-information rows. + /// The tile row index. + public void SetTileRow(ObuTileGroupHeader tileGroupHeader, int modeInfoRowCount, int row) + { + this.ModeInfoRowStart = tileGroupHeader.TileRowStartModeInfo[row]; + this.ModeInfoRowEnd = Math.Min(tileGroupHeader.TileRowStartModeInfo[row + 1], modeInfoRowCount); + Point loc = this.TileIndex; + loc.Y = row; + this.TileIndex = loc; + } + + /// + /// Selects the tile column and updates its mode-information boundaries. + /// + /// The tile layout. + /// The coded frame width in mode-information columns. + /// The tile column index. + public void SetTileColumn(ObuTileGroupHeader tileGroupHeader, int modeInfoColumnCount, int column) + { + this.ModeInfoColumnStart = tileGroupHeader.TileColumnStartModeInfo[column]; + this.ModeInfoColumnEnd = Math.Min(tileGroupHeader.TileColumnStartModeInfo[column + 1], modeInfoColumnCount); + Point loc = this.TileIndex; + loc.X = column; + this.TileIndex = loc; + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1TileReader.cs b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1TileReader.cs new file mode 100644 index 0000000000..4eadc10365 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1TileReader.cs @@ -0,0 +1,3462 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using SixLabors.ImageSharp.Formats.Heif.Av1; +using SixLabors.ImageSharp.Formats.Heif.Av1.Entropy; +using SixLabors.ImageSharp.Formats.Heif.Av1.Motion; +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline; +using SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.Quantizers; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; +using SixLabors.ImageSharp.Formats.Heif.Av1.ReferenceFrames; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; +using SixLabors.ImageSharp.Memory; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +/// +/// Parses partition, mode, transform, and coefficient syntax for one AV1 tile. +/// +internal sealed class Av1TileReader : IAv1TileReader, IDisposable +{ + /// + /// The minimum value of the first self-guided projection coefficient. + /// + private const int SgrProjectionCoefficient0Minimum = -96; + + /// + /// The minimum value of the second self-guided projection coefficient. + /// + private const int SgrProjectionCoefficient1Minimum = -32; + + /// + /// The number of values in either self-guided projection coefficient domain. + /// + private const int SgrProjectionCoefficientValueCount = 128; + + /// + /// The subexponential group-size exponent for self-guided projection coefficients. + /// + private const int SgrProjectionSubexponentialK = 4; + + /// + /// The two self-guided filter radii selected by each parameter-set index. + /// + private static readonly int[][] SgrProjectionRadii = + [ + [2, 1], [2, 1], [2, 1], [2, 1], [2, 1], [2, 1], [2, 1], [2, 1], + [2, 1], [2, 1], [0, 1], [0, 1], [0, 1], [0, 1], [2, 0], [2, 0] + ]; + + /// + /// Stores two preceding self-guided restoration coefficients for each of the three color planes, indexed by + /// (plane * 2) + coefficient. + /// + private InlineArray6 referenceSgrXqd; + + /// + /// Stores three preceding Wiener taps for both passes of each of the three color planes, indexed by + /// (((plane * 2) + pass) * 3) + tap. + /// + private InlineArray18 referenceLrWiener; + + /// + /// Tracks entropy, partition, and transform state above the current block. + /// + private readonly Av1ParseAboveNeighbor4x4Context aboveNeighborContext; + + /// + /// Tracks entropy, partition, and transform state left of the current block. + /// + private readonly Av1ParseLeftNeighbor4x4Context leftNeighborContext; + + /// + /// The quantizer index carried between delta-quantized blocks in the current tile. + /// + private int currentQuantizerIndex; + + /// + /// The coefficient quantizer updated before residual syntax consumes the active superblock delta-Q state. + /// + private readonly Av1InverseQuantizer inverseQuantizer; + + /// + /// The frame's base per-segment and per-plane dequantization values. + /// + private readonly Av1DeQuantizationContext deQuants; + + /// + /// Stores the loop-filter delta values carried between superblocks in the current tile. + /// + private InlineArray4 currentDeltaLoopFilter; + + /// + /// Stores per-plane transform counts for each forced 64x64 residual region. + /// + private InlineArray4> transformUnitCount; + + /// + /// Tracks the first unassigned transform-information index for luma and shared chroma storage. + /// + private InlineArray4 firstTransformOffset; + + /// + /// Tracks the next coefficient slot for each color plane within the current superblock. + /// + private InlineArray4 coefficientIndex; + + /// + /// Reusable padded coefficient-context storage for the sequential transform traversal. + /// + private readonly Av1LevelBuffer coefficientLevels; + + /// + /// Reusable luma palette indices for the coding blocks in one superblock. + /// + private readonly Buffer2D lumaPaletteColorIndexMap; + + /// + /// Reusable chroma palette indices for the coding blocks in one superblock. + /// + private readonly Buffer2D chromaPaletteColorIndexMap; + + /// + /// Indicates whether this reader owns and disposes the palette maps. + /// + private readonly bool ownsPaletteColorIndexMaps; + + /// + /// Reusable storage for the eight spatial displacement-vector candidates permitted by AV1. + /// + private InlineArray8 displacementVectorCandidates; + + /// + /// Reusable storage for the spatial weight associated with each displacement-vector candidate. + /// + private InlineArray8 displacementVectorWeights; + + /// + /// Reusable counts of the canonical references selected by the immediately above and left blocks. + /// + private InlineArray8 neighborReferenceCounts; + + /// + /// Reusable fixed-capacity storage for one block's weighted reference-motion-vector candidates. + /// + private Av1ReferenceMotionVectors referenceMotionVectors; + + /// + /// Reusable fixed-capacity state for motion-mode eligibility and local warped-motion projection. + /// + private readonly Av1MotionVariationCandidates motionVariationCandidates = new(); + + /// + /// Provides allocator and decoder configuration to tile entropy decoding. + /// + private readonly Configuration configuration; + + /// + /// The decoder-session entropy contexts reused by every tile in the current frame. + /// + private readonly Av1FrameEntropyContexts entropyContexts; + + /// + /// The retained primary frame whose segment map supplies temporal segment-ID predictions. + /// + private readonly Av1FrameInfo.ReferenceState? primaryReferenceState; + + /// + /// The retained reconstructed frames used to determine reference scaling during inter mode parsing. + /// + private readonly Av1ReferenceFrameStore? referenceFrames; + + /// + /// Initializes a new instance of the class for syntax parsing without reconstruction. + /// + /// The decoder configuration. + /// The active AV1 sequence header. + /// The frame header whose tiles will be parsed. + public Av1TileReader(Configuration configuration, ObuSequenceHeader sequenceHeader, ObuFrameHeader frameHeader) + : this(configuration, sequenceHeader, frameHeader, new(frameHeader.QuantizationParameters.BaseQIndex), null, null) + { + } + + /// + /// Initializes a new instance of the class with decoder-session entropy state. + /// + /// The decoder configuration. + /// The active AV1 sequence header. + /// The frame header whose tiles will be parsed. + /// The entropy contexts reused by the owning decoder session. + /// + /// The retained primary-reference entropy context, or when the frame selects defaults. + /// + /// + /// The retained reconstructed frames, or for the intra-only syntax reader. + /// + public Av1TileReader( + Configuration configuration, + ObuSequenceHeader sequenceHeader, + ObuFrameHeader frameHeader, + Av1FrameEntropyContexts entropyContexts, + Av1FrameEntropyContext? primaryReferenceContext, + Av1ReferenceFrameStore? referenceFrames) + : this(configuration, sequenceHeader, frameHeader, entropyContexts, primaryReferenceContext, referenceFrames, null) + { + } + + /// + /// Initializes a new instance of the class with decoder-session entropy and palette state. + /// + /// The decoder configuration. + /// The active AV1 sequence header. + /// The frame header whose tiles will be parsed. + /// The entropy contexts reused by the owning decoder session. + /// + /// The retained primary-reference entropy context, or when the frame selects defaults. + /// + /// The retained reconstructed frames. + /// The decoder-session luma palette scratch map. + /// The decoder-session chroma palette scratch map. + public Av1TileReader( + Configuration configuration, + ObuSequenceHeader sequenceHeader, + ObuFrameHeader frameHeader, + Av1FrameEntropyContexts entropyContexts, + Av1FrameEntropyContext? primaryReferenceContext, + Av1ReferenceFrameStore referenceFrames, + Buffer2D lumaPaletteColorIndexMap, + Buffer2D chromaPaletteColorIndexMap) + : this( + configuration, + sequenceHeader, + frameHeader, + entropyContexts, + primaryReferenceContext, + referenceFrames, + new PaletteColorIndexMaps(lumaPaletteColorIndexMap, chromaPaletteColorIndexMap)) + { + } + + /// + /// Initializes a new instance of the class with explicit palette-map ownership. + /// + /// The decoder configuration. + /// The active AV1 sequence header. + /// The frame header whose tiles will be parsed. + /// The entropy contexts reused by the owning decoder session. + /// The retained primary-reference entropy context. + /// The retained reconstructed frames. + /// + /// Shared palette maps, or when the reader allocates and owns both maps. + /// + private Av1TileReader( + Configuration configuration, + ObuSequenceHeader sequenceHeader, + ObuFrameHeader frameHeader, + Av1FrameEntropyContexts entropyContexts, + Av1FrameEntropyContext? primaryReferenceContext, + Av1ReferenceFrameStore? referenceFrames, + PaletteColorIndexMaps? sharedPaletteColorIndexMaps) + { + this.referenceMotionVectors = default; + this.FrameHeader = frameHeader; + this.configuration = configuration; + this.SequenceHeader = sequenceHeader; + this.entropyContexts = entropyContexts; + this.referenceFrames = referenceFrames; + this.ownsPaletteColorIndexMaps = sharedPaletteColorIndexMaps is null; + this.entropyContexts.BeginFrame(frameHeader.QuantizationParameters.BaseQIndex, primaryReferenceContext); + this.inverseQuantizer = new(sequenceHeader, frameHeader); + this.deQuants = new(sequenceHeader, frameHeader); + + // FrameInfo owns traversal records for this coded frame and one superblock of coefficient scratch. + this.FrameInfo = new(this.configuration, this.SequenceHeader, this.FrameHeader); + if (referenceFrames is not null) + { + byte? primaryReferenceSlot = this.FrameHeader.PrimaryReferenceSlot; + if (primaryReferenceSlot is not null) + { + // The uncompressed-header parser has already validated this slot. Segment samples remain in the + // retained reference state and are copied only for whole-map inheritance. + this.primaryReferenceState = referenceFrames.ResolveRequired(primaryReferenceSlot.Value).ReferenceState; + } + } + + // Above contexts span the aligned frame width, while left contexts are reused for each superblock row. + int planesCount = sequenceHeader.ColorConfig.PlaneCount; + int modeInfoWideColumnCount = Av1Math.AlignPowerOf2( + frameHeader.ModeInfoColumnCount, + sequenceHeader.SuperblockSizeLog2 - Av1Constants.ModeInfoSizeLog2); + + try + { + this.FrameInfo.InitializeSegmentIds(this.FrameHeader, this.primaryReferenceState); + this.FrameInfo.InitializeLoopRestoration(this.SequenceHeader, this.FrameHeader); + this.aboveNeighborContext = new Av1ParseAboveNeighbor4x4Context(configuration, planesCount, modeInfoWideColumnCount); + } + catch + { + // FrameInfo owns the active frame's syntax storage. Return it if segmentation, restoration, or the + // first neighbor context fails, because no constructed reader reaches the caller's using statement. + this.FrameInfo.Dispose(); + throw; + } + + try + { + this.leftNeighborContext = new Av1ParseLeftNeighbor4x4Context(configuration, planesCount, sequenceHeader.SuperblockModeInfoSize); + } + catch + { + // The second context can fail after both frame state and the above context have acquired owners. + // Neither survives failed construction, so unwind both completed owners here. + this.aboveNeighborContext.Dispose(); + this.FrameInfo.Dispose(); + throw; + } + + try + { + this.coefficientLevels = new Av1LevelBuffer(configuration); + } + catch + { + // The coefficient scratch allocation follows both neighbor contexts. Unwind those successful rents when + // construction cannot publish an owning tile reader. + this.aboveNeighborContext.Dispose(); + this.leftNeighborContext.Dispose(); + this.FrameInfo.Dispose(); + throw; + } + + if (sharedPaletteColorIndexMaps is null) + { + Buffer2D? ownedLumaPaletteColorIndexMap = null; + Buffer2D? ownedChromaPaletteColorIndexMap = null; + + try + { + // Standalone syntax readers have no decoder-session owner. They still use the same fixed + // maximum-superblock bound and return both maps when the reader is disposed. + int paletteMapLength = 1 << Av1Constants.MaxSuperBlockSizeLog2; + ownedLumaPaletteColorIndexMap = configuration.MemoryAllocator.Allocate2D(paletteMapLength, paletteMapLength); + ownedChromaPaletteColorIndexMap = configuration.MemoryAllocator.Allocate2D(paletteMapLength, paletteMapLength); + this.lumaPaletteColorIndexMap = ownedLumaPaletteColorIndexMap; + this.chromaPaletteColorIndexMap = ownedChromaPaletteColorIndexMap; + } + catch + { + ownedChromaPaletteColorIndexMap?.Dispose(); + ownedLumaPaletteColorIndexMap?.Dispose(); + this.coefficientLevels.Dispose(); + this.leftNeighborContext.Dispose(); + this.aboveNeighborContext.Dispose(); + this.FrameInfo.Dispose(); + throw; + } + } + else + { + PaletteColorIndexMaps paletteColorIndexMaps = sharedPaletteColorIndexMaps.Value; + this.lumaPaletteColorIndexMap = paletteColorIndexMaps.Luma; + this.chromaPaletteColorIndexMap = paletteColorIndexMaps.Chroma; + } + + if (referenceFrames is not null) + { + try + { + // Motion storage is acquired after every other constructor allocation. If this final acquisition + // fails, the catch can return every successfully created allocator-owned resource in one place. + this.FrameInfo.InitializeMotionField(configuration, this.SequenceHeader, this.FrameHeader, referenceFrames); + } + catch + { + if (this.ownsPaletteColorIndexMaps) + { + this.chromaPaletteColorIndexMap.Dispose(); + this.lumaPaletteColorIndexMap.Dispose(); + } + + this.coefficientLevels.Dispose(); + this.leftNeighborContext.Dispose(); + this.aboveNeighborContext.Dispose(); + this.FrameInfo.Dispose(); + throw; + } + } + } + + /// + /// Initializes a new instance of the class that reconstructs parsed superblocks. + /// + /// The decoder configuration. + /// The active AV1 sequence header. + /// The frame header whose tiles will be parsed. + /// The frame decoder that reconstructs each parsed superblock. + public Av1TileReader(Configuration configuration, ObuSequenceHeader sequenceHeader, ObuFrameHeader frameHeader, IAv1FrameDecoder frameDecoder) + : this(configuration, sequenceHeader, frameHeader) + => this.FrameDecoder = frameDecoder; + + /// + /// Gets the default self-guided restoration projection coefficients for each color plane. + /// + private static ReadOnlySpan SgrprojXqdMid => [-32, 31]; + + /// + /// Gets the default Wiener restoration taps retained between restoration units. + /// + private static ReadOnlySpan WienerTapsMid => [3, -7, 15]; + + /// + /// Gets the minimum transmitted value for each independent Wiener coefficient. + /// + private static ReadOnlySpan WienerCoefficientMinimum => [-5, -23, -17]; + + /// + /// Gets the number of possible transmitted values for each independent Wiener coefficient. + /// + private static ReadOnlySpan WienerCoefficientValueCount => [16, 32, 64]; + + /// + /// Gets the subexponential group-size exponent for each independent Wiener coefficient. + /// + private static ReadOnlySpan WienerCoefficientSubexponentialK => [1, 2, 3]; + + /// + /// Gets the signed DC-context contribution for each packed coefficient sign class. + /// + private static ReadOnlySpan Signs => [0, -1, 1]; + + /// + /// Gets the AV1 DC-sign entropy context for each summed neighboring sign value. + /// + private static ReadOnlySpan DcSignContexts => [ + 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, + 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2]; + + /// + /// Gets the frame header whose tile syntax is being parsed. + /// + public ObuFrameHeader FrameHeader { get; } + + /// + /// Gets the sequence header governing the frame. + /// + public ObuSequenceHeader SequenceHeader { get; } + + /// + /// Gets the frame-owned mode, motion, transform, coefficient, quantizer, and filter state populated by tile parsing. + /// + public Av1FrameInfo FrameInfo { get; } + + /// + /// Gets or sets the optional decoder that reconstructs each superblock immediately after its syntax is parsed. + /// + public IAv1FrameDecoder? FrameDecoder { get; set; } + + /// + /// Gets the completed frame entropy context selected by the context-update tile. + /// + /// + /// The context contains either normative defaults or the selected primary-reference state until the signaled + /// update tile has decoded successfully. Callers that retain it beyond this reader's frame lifecycle must copy it + /// through . + /// + public Av1FrameEntropyContext FrameEntropyContext => this.entropyContexts.Published; + + /// + /// Gets the decoder-session entropy owner that publishes the completed frame context. + /// + public Av1FrameEntropyContexts EntropyContexts => this.entropyContexts; + + /// + /// Returns tile-neighbor storage and the reader's frame-state lease to the configured memory allocator. + /// + public void Dispose() + { + this.aboveNeighborContext.Dispose(); + this.leftNeighborContext.Dispose(); + this.coefficientLevels.Dispose(); + if (this.ownsPaletteColorIndexMaps) + { + this.lumaPaletteColorIndexMap.Dispose(); + this.chromaPaletteColorIndexMap.Dispose(); + } + + this.FrameInfo.Dispose(); + } + + /// + /// Parses one tile's partition, mode, transform, coefficient, and filter syntax in superblock order. + /// + /// The entropy-coded tile payload. + /// The zero-based tile index in row-major order. + public void ReadTile(Span tileData, int tileNum) + { + // AV1 tiles never inherit adaptation from another tile in the same frame. Reusing one graph is safe because + // parsing is sequential and every entry is restored from the unchanged frame base before the range decoder is + // constructed. + this.entropyContexts.Working.CopyFrom(this.entropyContexts.Base); + + // The frame syntax exposes a disable flag, while the range reader follows the reference decoder's positive + // allow_update_cdf convention. + Av1SymbolDecoder reader = new( + this.configuration, + tileData, + this.entropyContexts.Working, + !this.FrameHeader.DisableCdfUpdate); + + int tileColumnIndex = tileNum % this.FrameHeader.TilesInfo.TileColumnCount; + int tileRowIndex = tileNum / this.FrameHeader.TilesInfo.TileColumnCount; + + int modeInfoColumnStart = this.FrameHeader.TilesInfo.TileColumnStartModeInfo[tileColumnIndex]; + int modeInfoColumnEnd = this.FrameHeader.TilesInfo.TileColumnStartModeInfo[tileColumnIndex + 1]; + int modeInfoRowStart = this.FrameHeader.TilesInfo.TileRowStartModeInfo[tileRowIndex]; + int modeInfoRowEnd = this.FrameHeader.TilesInfo.TileRowStartModeInfo[tileRowIndex + 1]; + this.aboveNeighborContext.Clear(this.SequenceHeader, modeInfoColumnStart, modeInfoColumnEnd); + this.currentQuantizerIndex = this.FrameHeader.QuantizationParameters.BaseQIndex; + this.ClearLoopFilterDelta(); + int planesCount = this.SequenceHeader.ColorConfig.PlaneCount; + + // AV1 fixes restoration reference storage at three planes, two directions or projection coefficients, and + // three transmitted Wiener taps. Populate the inline value storage in place so every tile starts from the + // normative differential-coding defaults without constructing jagged arrays. + Span sgrReferences = this.referenceSgrXqd; + Span wienerReferences = this.referenceLrWiener; + for (int plane = 0; plane < planesCount; plane++) + { + SgrprojXqdMid.CopyTo(sgrReferences.Slice(plane * 2, 2)); + for (int pass = 0; pass < 2; pass++) + { + int referenceOffset = ((plane * 2) + pass) * Av1Constants.WienerCoefficientCount; + WienerTapsMid.CopyTo(wienerReferences.Slice(referenceOffset, Av1Constants.WienerCoefficientCount)); + } + } + + Av1TileInfo tileInfo = new(tileRowIndex, tileColumnIndex, this.FrameHeader); + Av1BlockSize superBlockSize = this.SequenceHeader.SuperblockSize; + int superBlock4x4Size = this.SequenceHeader.SuperblockSize.Get4x4WideCount(); + int superBlockSizeLog2 = this.SequenceHeader.SuperblockSizeLog2; + for (int row = modeInfoRowStart; row < modeInfoRowEnd; row += superBlock4x4Size) + { + int superBlockRow = (row << Av1Constants.ModeInfoSizeLog2) >> superBlockSizeLog2; + this.leftNeighborContext.Clear(this.SequenceHeader); + for (int column = modeInfoColumnStart; column < modeInfoColumnEnd; column += superBlock4x4Size) + { + int superBlockColumn = (column << Av1Constants.ModeInfoSizeLog2) >> superBlockSizeLog2; + Point superblockPosition = new(superBlockColumn, superBlockRow); + Av1SuperblockInfo superblockInfo = this.FrameInfo.GetSuperblock(superblockPosition); + + Point modeInfoPosition = new(column, row); + superblockInfo.CoefficientsY.Clear(); + superblockInfo.CoefficientsU.Clear(); + superblockInfo.CoefficientsV.Clear(); + this.FrameInfo.ClearCdef(superblockPosition); + this.firstTransformOffset[0] = 0; + this.firstTransformOffset[1] = 0; + this.coefficientIndex[..Av1Constants.MaxPlanes].Clear(); + this.ReadLoopRestoration(ref reader, modeInfoPosition, superBlockSize); + this.ParsePartition(ref reader, modeInfoPosition, superBlockSize, superblockInfo, tileInfo); + + // Identify-only parsing omits a frame decoder but still populates the complete syntax model. + this.FrameDecoder?.DecodeSuperblock(modeInfoPosition, superblockInfo, tileInfo); + } + } + + // Range decoding may read implicit zero padding while normalizing its final interval. Validate the logical + // stopping position before publishing either pixels or adapted CDF state so a truncated tile cannot commit. + reader.ValidateTrailingBits(); + + if (!this.FrameHeader.DisableFrameEndUpdateCdf && tileNum == this.FrameHeader.TilesInfo.ContextUpdateTileId) + { + // the reference decoder publishes only context_update_tile_id after every tile has independently started from the frame + // base, then clears its CDF counters. Snapshotting into a third reusable graph preserves the unchanged base + // for tiles that follow the selected tile in bitstream order. + this.entropyContexts.Working.SnapshotTo(this.entropyContexts.Published); + } + } + + /// + /// Completes the current coded frame. + /// + public void CompleteFrame() + { + // Tile parsing and optional incremental superblock reconstruction finish inside ReadTile. The owning AV1 + // decoder uses this lifecycle boundary to assemble native planes. A directly created tile reader owns its + // neighbor-context rents, so the same boundary must return them before ObuReader releases the frame instance. + this.Dispose(); + } + + /// + /// Resets the loop-filter delta predictors before parsing a tile. + /// + private void ClearLoopFilterDelta() + => this.currentDeltaLoopFilter[..Av1Constants.FrameLoopFilterCount].Clear(); + + /// + /// Reads loop-restoration unit syntax that begins at a superblock location. + /// + /// The tile symbol decoder. + /// The superblock origin in 4x4 mode-information units. + /// The superblock size. + private void ReadLoopRestoration(ref Av1SymbolDecoder reader, Point modeInfoLocation, Av1BlockSize superBlockSize) + { + ObuColorConfig colorConfig = this.SequenceHeader.ColorConfig; + int planesCount = colorConfig.PlaneCount; + for (int plane = 0; plane < planesCount; plane++) + { + ObuLoopRestorationItem item = this.FrameHeader.LoopRestorationParameters.Items[plane]; + if (item.Type == ObuRestorationType.None) + { + continue; + } + + int subsamplingX = plane > 0 && colorConfig.SubSamplingX ? 1 : 0; + int subsamplingY = plane > 0 && colorConfig.SubSamplingY ? 1 : 0; + int planeHeight = Av1Math.DivideLog2Ceiling(this.FrameHeader.FrameSize.FrameHeight, subsamplingY); + int unitColumnCount = this.FrameInfo.GetLoopRestorationUnitColumnCount(plane); + int unitRowCount = Math.Max((planeHeight + (item.Size >> 1)) / item.Size, 1); + int superblockModeInfoSize = superBlockSize.Get4x4WideCount(); + int modeInfoColumnEnd = modeInfoLocation.X + superblockModeInfoSize; + int modeInfoRowEnd = modeInfoLocation.Y + superblockModeInfoSize; + int modeInfoSampleWidth = (1 << Av1Constants.ModeInfoSizeLog2) >> subsamplingX; + int modeInfoSampleHeight = (1 << Av1Constants.ModeInfoSizeLog2) >> subsamplingY; + bool usesSuperResolution = + this.FrameHeader.FrameSize.FrameWidth != this.FrameHeader.FrameSize.SuperResolutionUpscaledWidth; + + int columnNumeratorScale = usesSuperResolution + ? modeInfoSampleWidth * this.FrameHeader.FrameSize.SuperResolutionDenominator + : modeInfoSampleWidth; + + int columnDenominator = usesSuperResolution + ? item.Size * Av1Constants.ScaleNumerator + : item.Size; + + int rowDenominator = item.Size; + + // Restoration syntax is attached to the superblock containing each unit's upper-left + // corner. Super-resolution changes only the horizontal corner conversion. + int unitColumnStart = DivideCeiling(modeInfoLocation.X * columnNumeratorScale, columnDenominator); + int unitColumnEnd = Math.Min(DivideCeiling(modeInfoColumnEnd * columnNumeratorScale, columnDenominator), unitColumnCount); + int unitRowStart = DivideCeiling(modeInfoLocation.Y * modeInfoSampleHeight, rowDenominator); + int unitRowEnd = Math.Min(DivideCeiling(modeInfoRowEnd * modeInfoSampleHeight, rowDenominator), unitRowCount); + for (int unitRow = unitRowStart; unitRow < unitRowEnd; unitRow++) + { + for (int unitColumn = unitColumnStart; unitColumn < unitColumnEnd; unitColumn++) + { + ref Av1LoopRestorationUnit unit = ref this.FrameInfo.GetLoopRestorationUnit(plane, unitRow, unitColumn); + this.ReadLoopRestorationUnit(ref reader, item.Type, plane, ref unit); + } + } + } + } + + /// + /// Reads the filter selection and coefficients for one loop-restoration unit. + /// + /// The tile symbol decoder. + /// The restoration mode allowed by the frame header. + /// The zero-based color-plane index. + /// The destination restoration-unit information. + private void ReadLoopRestorationUnit( + ref Av1SymbolDecoder reader, + ObuRestorationType frameType, + int plane, + ref Av1LoopRestorationUnit unit) + { + unit.FilterType = frameType switch + { + ObuRestorationType.Switchable => reader.ReadSwitchableRestorationType(), + ObuRestorationType.Wiener => reader.ReadWienerRestoration() + ? Av1RestorationFilterType.Wiener + : Av1RestorationFilterType.None, + ObuRestorationType.SgrProj => reader.ReadSgrProjectionRestoration() + ? Av1RestorationFilterType.SgrProjection + : Av1RestorationFilterType.None, + _ => Av1RestorationFilterType.None, + }; + + if (unit.FilterType == Av1RestorationFilterType.Wiener) + { + this.ReadWienerFilter(ref reader, plane, ref unit); + } + else if (unit.FilterType == Av1RestorationFilterType.SgrProjection) + { + this.ReadSgrProjectionFilter(ref reader, plane, ref unit); + } + } + + /// + /// Reads the symmetric vertical and horizontal Wiener coefficients for one restoration unit. + /// + /// The tile symbol decoder. + /// The zero-based color-plane index. + /// The destination restoration-unit information. + private void ReadWienerFilter(ref Av1SymbolDecoder reader, int plane, ref Av1LoopRestorationUnit unit) + { + for (int pass = 0; pass < 2; pass++) + { + int firstCoefficient = plane == 0 ? 0 : 1; + if (pass == 0) + { + unit.WienerVertical[0] = 0; + } + else + { + unit.WienerHorizontal[0] = 0; + } + + for (int coefficient = firstCoefficient; coefficient < Av1Constants.WienerCoefficientCount; coefficient++) + { + int referenceIndex = (((plane * 2) + pass) * Av1Constants.WienerCoefficientCount) + coefficient; + int minimum = WienerCoefficientMinimum[coefficient]; + int value = reader.ReadReferenceSubexponential( + WienerCoefficientValueCount[coefficient], + WienerCoefficientSubexponentialK[coefficient], + this.referenceLrWiener[referenceIndex] - minimum); + + value += minimum; + if (pass == 0) + { + unit.WienerVertical[coefficient] = value; + } + else + { + unit.WienerHorizontal[coefficient] = value; + } + + this.referenceLrWiener[referenceIndex] = value; + } + } + } + + /// + /// Reads the parameter-set index and projection coefficients for one self-guided restoration unit. + /// + /// The tile symbol decoder. + /// The zero-based color-plane index. + /// The destination restoration-unit information. + private void ReadSgrProjectionFilter(ref Av1SymbolDecoder reader, int plane, ref Av1LoopRestorationUnit unit) + { + unit.SgrParameterSet = reader.ReadLiteral(4); + int[] radii = SgrProjectionRadii[unit.SgrParameterSet]; + Span coefficients = unit.SgrProjectionCoefficients; + Span allReferences = this.referenceSgrXqd; + Span references = allReferences.Slice(plane * 2, 2); + if (radii[0] == 0) + { + coefficients[0] = 0; + coefficients[1] = ReadSgrProjectionCoefficient(ref reader, references[1], SgrProjectionCoefficient1Minimum); + } + else if (radii[1] == 0) + { + coefficients[0] = ReadSgrProjectionCoefficient(ref reader, references[0], SgrProjectionCoefficient0Minimum); + + // When the second filter is disabled, AV1 derives the missing projection coefficient + // so the combined projection retains its fixed seven-bit scale. + coefficients[1] = Av1Math.Clip3( + SgrProjectionCoefficient1Minimum, + SgrProjectionCoefficient1Minimum + SgrProjectionCoefficientValueCount - 1, + SgrProjectionCoefficientValueCount - coefficients[0]); + } + else + { + coefficients[0] = ReadSgrProjectionCoefficient(ref reader, references[0], SgrProjectionCoefficient0Minimum); + coefficients[1] = ReadSgrProjectionCoefficient(ref reader, references[1], SgrProjectionCoefficient1Minimum); + } + + coefficients.CopyTo(references); + } + + /// + /// Reads one differentially coded self-guided projection coefficient. + /// + /// The tile symbol decoder. + /// The preceding coefficient value for the plane. + /// The minimum value in the coefficient domain. + /// The decoded signed coefficient. + private static int ReadSgrProjectionCoefficient(ref Av1SymbolDecoder reader, int reference, int minimum) + => reader.ReadReferenceSubexponential( + SgrProjectionCoefficientValueCount, + SgrProjectionSubexponentialK, + reference - minimum) + minimum; + + /// + /// Divides a non-negative numerator by a positive denominator and rounds upward. + /// + /// The non-negative numerator. + /// The positive denominator. + /// The ceiling of the quotient. + private static int DivideCeiling(int numerator, int denominator) + => (numerator + denominator - 1) / denominator; + + /// + /// Decodes AV1 partition syntax and recursively visits each resulting coding block. + /// + /// The tile symbol decoder. + /// The parent block origin in 4x4 mode-information units. + /// The parent block size. + /// The containing superblock. + /// The active tile boundaries. + /// Implements AV1 section 5.11.4. + private void ParsePartition(ref Av1SymbolDecoder reader, Point modeInfoLocation, Av1BlockSize blockSize, Av1SuperblockInfo superblockInfo, Av1TileInfo tileInfo) + { + int columnIndex = modeInfoLocation.X; + int rowIndex = modeInfoLocation.Y; + if (modeInfoLocation.Y >= this.FrameHeader.ModeInfoRowCount || modeInfoLocation.X >= this.FrameHeader.ModeInfoColumnCount) + { + return; + } + + int block4x4Size = blockSize.Get4x4WideCount(); + int halfBlock4x4Size = block4x4Size >> 1; + int quarterBlock4x4Size = halfBlock4x4Size >> 1; + bool hasRows = (modeInfoLocation.Y + halfBlock4x4Size) < this.FrameHeader.ModeInfoRowCount; + bool hasColumns = (modeInfoLocation.X + halfBlock4x4Size) < this.FrameHeader.ModeInfoColumnCount; + Av1PartitionType partitionType = Av1PartitionType.None; + if (blockSize >= Av1BlockSize.Block8x8) + { + int ctx = this.GetPartitionPlaneContext(modeInfoLocation, blockSize, tileInfo, superblockInfo); + partitionType = Av1PartitionType.Split; + if (hasRows && hasColumns) + { + partitionType = reader.ReadPartitionType(ctx); + } + else if (hasColumns) + { + partitionType = reader.ReadSplitOrHorizontal(blockSize, ctx); + } + else if (hasRows) + { + partitionType = reader.ReadSplitOrVertical(blockSize, ctx); + } + } + + Av1BlockSize subSize = partitionType.GetBlockSubSize(blockSize); + if (subSize == Av1BlockSize.Invalid) + { + throw new InvalidImageContentException($"The decoded AV1 partition type {partitionType} is invalid for block size {blockSize}."); + } + + ObuColorConfig colorConfig = this.SequenceHeader.ColorConfig; + if (subSize.GetSubsampled(colorConfig.SubSamplingX, colorConfig.SubSamplingY) == Av1BlockSize.Invalid) + { + // Luma partition syntax can describe a sub-8x8 shape that has no legal representation after chroma + // subsampling. Reject it before any block state is published, matching the reference decoder's decode_partition boundary. + throw new InvalidImageContentException($"The decoded AV1 block size {subSize} is invalid for the sequence chroma subsampling."); + } + + Av1BlockSize splitSize = Av1PartitionType.Split.GetBlockSubSize(blockSize); + + // Partition syntax is depth-first. The visit order here is also the order in which mode, + // transform, and coefficient records are appended to their frame-owned arrays. + switch (partitionType) + { + case Av1PartitionType.Split: + Point loc1 = new(modeInfoLocation.X + halfBlock4x4Size, modeInfoLocation.Y); + Point loc2 = new(modeInfoLocation.X, modeInfoLocation.Y + halfBlock4x4Size); + Point loc3 = new(modeInfoLocation.X + halfBlock4x4Size, modeInfoLocation.Y + halfBlock4x4Size); + this.ParsePartition(ref reader, modeInfoLocation, subSize, superblockInfo, tileInfo); + this.ParsePartition(ref reader, loc1, subSize, superblockInfo, tileInfo); + this.ParsePartition(ref reader, loc2, subSize, superblockInfo, tileInfo); + this.ParsePartition(ref reader, loc3, subSize, superblockInfo, tileInfo); + break; + case Av1PartitionType.None: + this.ParseBlock(ref reader, modeInfoLocation, subSize, superblockInfo, tileInfo, Av1PartitionType.None); + break; + case Av1PartitionType.Horizontal: + this.ParseBlock(ref reader, modeInfoLocation, subSize, superblockInfo, tileInfo, Av1PartitionType.Horizontal); + if (hasRows) + { + Point halfLocation = new(columnIndex, rowIndex + halfBlock4x4Size); + this.ParseBlock(ref reader, halfLocation, subSize, superblockInfo, tileInfo, Av1PartitionType.Horizontal); + } + + break; + case Av1PartitionType.Vertical: + this.ParseBlock(ref reader, modeInfoLocation, subSize, superblockInfo, tileInfo, Av1PartitionType.Vertical); + if (hasColumns) + { + Point halfLocation = new(columnIndex + halfBlock4x4Size, rowIndex); + this.ParseBlock(ref reader, halfLocation, subSize, superblockInfo, tileInfo, Av1PartitionType.Vertical); + } + + break; + case Av1PartitionType.HorizontalA: + this.ParseBlock(ref reader, modeInfoLocation, splitSize, superblockInfo, tileInfo, Av1PartitionType.HorizontalA); + Point locHorA1 = new(columnIndex + halfBlock4x4Size, rowIndex); + this.ParseBlock(ref reader, locHorA1, splitSize, superblockInfo, tileInfo, Av1PartitionType.HorizontalA); + Point locHorA2 = new(columnIndex, rowIndex + halfBlock4x4Size); + this.ParseBlock(ref reader, locHorA2, subSize, superblockInfo, tileInfo, Av1PartitionType.HorizontalA); + break; + case Av1PartitionType.HorizontalB: + this.ParseBlock(ref reader, modeInfoLocation, subSize, superblockInfo, tileInfo, Av1PartitionType.HorizontalB); + Point locHorB1 = new(columnIndex, rowIndex + halfBlock4x4Size); + this.ParseBlock(ref reader, locHorB1, splitSize, superblockInfo, tileInfo, Av1PartitionType.HorizontalB); + Point locHorB2 = new(columnIndex + halfBlock4x4Size, rowIndex + halfBlock4x4Size); + this.ParseBlock(ref reader, locHorB2, splitSize, superblockInfo, tileInfo, Av1PartitionType.HorizontalB); + break; + case Av1PartitionType.VerticalA: + this.ParseBlock(ref reader, modeInfoLocation, splitSize, superblockInfo, tileInfo, Av1PartitionType.VerticalA); + Point locVertA1 = new(columnIndex, rowIndex + halfBlock4x4Size); + this.ParseBlock(ref reader, locVertA1, splitSize, superblockInfo, tileInfo, Av1PartitionType.VerticalA); + Point locVertA2 = new(columnIndex + halfBlock4x4Size, rowIndex); + this.ParseBlock(ref reader, locVertA2, subSize, superblockInfo, tileInfo, Av1PartitionType.VerticalA); + break; + case Av1PartitionType.VerticalB: + this.ParseBlock(ref reader, modeInfoLocation, subSize, superblockInfo, tileInfo, Av1PartitionType.VerticalB); + Point locVertB1 = new(columnIndex + halfBlock4x4Size, rowIndex); + this.ParseBlock(ref reader, locVertB1, splitSize, superblockInfo, tileInfo, Av1PartitionType.VerticalB); + Point locVertB2 = new(columnIndex + halfBlock4x4Size, rowIndex + halfBlock4x4Size); + this.ParseBlock(ref reader, locVertB2, splitSize, superblockInfo, tileInfo, Av1PartitionType.VerticalB); + break; + case Av1PartitionType.Horizontal4: + for (int i = 0; i < 4; i++) + { + int currentBlockRow = rowIndex + (i * quarterBlock4x4Size); + if (i > 0 && currentBlockRow >= this.FrameHeader.ModeInfoRowCount) + { + break; + } + + Point currentLocation = new(modeInfoLocation.X, currentBlockRow); + this.ParseBlock(ref reader, currentLocation, subSize, superblockInfo, tileInfo, Av1PartitionType.Horizontal4); + } + + break; + case Av1PartitionType.Vertical4: + for (int i = 0; i < 4; i++) + { + int currentBlockColumn = columnIndex + (i * quarterBlock4x4Size); + if (i > 0 && currentBlockColumn >= this.FrameHeader.ModeInfoColumnCount) + { + break; + } + + Point currentLocation = new(currentBlockColumn, modeInfoLocation.Y); + this.ParseBlock(ref reader, currentLocation, subSize, superblockInfo, tileInfo, Av1PartitionType.Vertical4); + } + + break; + default: + throw new InvalidImageContentException($"The decoded AV1 partition type {partitionType} is invalid."); + } + + this.UpdatePartitionContext(new Point(columnIndex, rowIndex), tileInfo, superblockInfo, subSize, blockSize, partitionType); + } + + /// + /// Parses all syntax associated with one final coding block and stores its frame mode information. + /// + /// The tile symbol decoder. + /// The block origin in 4x4 mode-information units. + /// The final block size. + /// The containing superblock. + /// The active tile boundaries. + /// The partition type that produced the block. + private void ParseBlock(ref Av1SymbolDecoder reader, Point modeInfoLocation, Av1BlockSize blockSize, Av1SuperblockInfo superblockInfo, Av1TileInfo tileInfo, Av1PartitionType partitionType) + { + int rowIndex = modeInfoLocation.Y; + int columnIndex = modeInfoLocation.X; + int block4x4Width = blockSize.Get4x4WideCount(); + int block4x4Height = blockSize.Get4x4HighCount(); + Point superblockLocation = superblockInfo.Position * this.SequenceHeader.SuperblockModeInfoSize; + Point locationInSuperblock = new Point(modeInfoLocation.X - superblockLocation.X, modeInfoLocation.Y - superblockLocation.Y); + Av1BlockModeInfo blockModeInfo = new(blockSize, locationInSuperblock); + blockModeInfo.PartitionType = partitionType; + blockModeInfo.SetFirstTransformLocation(Av1PlaneType.Y, this.firstTransformOffset[0]); + blockModeInfo.SetFirstTransformLocation(Av1PlaneType.Uv, this.firstTransformOffset[1]); + bool hasChroma = HasChroma(this.SequenceHeader, modeInfoLocation, blockSize); + Av1PartitionInfo partitionInfo = new(blockModeInfo, superblockInfo, hasChroma, partitionType); + partitionInfo.ColumnIndex = columnIndex; + partitionInfo.RowIndex = rowIndex; + partitionInfo.ComputeBoundaryOffsets(this.SequenceHeader, this.FrameHeader, tileInfo); + if (hasChroma) + { + if (this.SequenceHeader.ColorConfig.SubSamplingY && block4x4Height == 1) + { + partitionInfo.AvailableAboveForChroma = IsInside(tileInfo, rowIndex - 2, columnIndex); + } + + if (this.SequenceHeader.ColorConfig.SubSamplingX && block4x4Width == 1) + { + partitionInfo.AvailableLeftForChroma = IsInside(tileInfo, rowIndex, columnIndex - 2); + } + } + + partitionInfo.PopulateModeInfoNeighbors(this.SequenceHeader.ColorConfig); + + this.ReadModeInfo(ref reader, ref partitionInfo, tileInfo); + + this.ReadPaletteTokens(ref reader, ref partitionInfo); + this.ReadBlockTransformSize(ref reader, modeInfoLocation, ref partitionInfo, superblockInfo, tileInfo); + + if (partitionInfo.ModeInfo.Skip) + { + this.ResetSkipContext(ref partitionInfo, tileInfo); + } + + this.Residual(ref reader, ref partitionInfo, superblockInfo, tileInfo, blockSize); + + // Record compact frame evidence before later frames release this frame's full mode-information graph. + this.FrameInfo.RecordInterPredictionFeatures(partitionInfo.ModeInfo, this.FrameHeader); + + // Store the record only after all syntax has populated it, then map every covered 4x4 position. + this.FrameInfo.UpdateModeInfo(partitionInfo.ModeInfo, superblockInfo); + } + + /// + /// Clears coefficient neighbor contexts across every plane of a skipped block. + /// + /// The skipped block and its frame position. + /// The active tile boundaries. + /// Implements AV1 section 5.11.37. + private void ResetSkipContext(ref Av1PartitionInfo partitionInfo, Av1TileInfo tileInfo) + { + // Subsampled 4x4 luma blocks can share chroma ownership with an adjacent luma block. A skipped block that is + // not the chroma reference must preserve those shared coefficient contexts for the owning block. + int planesCount = partitionInfo.IsChroma ? this.SequenceHeader.ColorConfig.PlaneCount : 1; + for (int i = 0; i < planesCount; i++) + { + int subX = (i > 0 && this.SequenceHeader.ColorConfig.SubSamplingX) ? 1 : 0; + int subY = (i > 0 && this.SequenceHeader.ColorConfig.SubSamplingY) ? 1 : 0; + Av1BlockSize planeBlockSize = partitionInfo.ModeInfo.BlockSize.GetSubsampled(subX, subY); + DebugGuard.IsTrue(planeBlockSize != Av1BlockSize.Invalid, nameof(planeBlockSize)); + int txsWide = planeBlockSize.GetWidth() >> 2; + int txsHigh = planeBlockSize.GetHeight() >> 2; + int aboveOffset = (partitionInfo.ColumnIndex - tileInfo.ModeInfoColumnStart) >> subX; + int leftOffset = (partitionInfo.RowIndex - partitionInfo.SuperblockInfo.ModeInfoPosition.Y) >> subY; + this.aboveNeighborContext.ClearContext(i, aboveOffset, txsWide); + this.leftNeighborContext.ClearContext(i, leftOffset, txsHigh); + } + } + + /// + /// Parses every luma and chroma transform block and its coefficients for a coding block. + /// + /// The tile symbol decoder. + /// The current coding block. + /// The containing superblock and coefficient storage. + /// The active tile boundaries. + /// The coding block size. + /// Implements AV1 section 5.11.34. + private void Residual( + ref Av1SymbolDecoder reader, + ref Av1PartitionInfo partitionInfo, + Av1SuperblockInfo superblockInfo, + Av1TileInfo tileInfo, + Av1BlockSize blockSize) + { + // Mode syntax has established delta-Q before residual decoding. Keep dequantization at this parsing + // boundary so each signed level is published once in the form consumed by inverse reconstruction. + this.inverseQuantizer.UpdateDequant(this.deQuants, superblockInfo); + int maxBlocksWide = partitionInfo.GetMaxBlockWide(blockSize, false); + int maxBlocksHigh = partitionInfo.GetMaxBlockHigh(blockSize, false); + Av1BlockSize maxUnitSize = Av1BlockSize.Block64x64; + int modeUnitBlocksWide = maxUnitSize.GetWidth() >> 2; + int modeUnitBlocksHigh = maxUnitSize.GetHeight() >> 2; + modeUnitBlocksWide = Math.Min(maxBlocksWide, modeUnitBlocksWide); + modeUnitBlocksHigh = Math.Min(maxBlocksHigh, modeUnitBlocksHigh); + int planeCount = this.SequenceHeader.ColorConfig.PlaneCount; + bool isLossless = this.FrameHeader.LosslessArray[partitionInfo.ModeInfo.SegmentId]; + bool isLosslessBlock = isLossless && (blockSize >= Av1BlockSize.Block64x64) && (blockSize <= Av1BlockSize.Block128x128); + int subSampling = (this.SequenceHeader.ColorConfig.SubSamplingX ? 1 : 0) + (this.SequenceHeader.ColorConfig.SubSamplingY ? 1 : 0); + int chromaTransformUnitCount = isLosslessBlock + ? (maxBlocksWide * maxBlocksHigh) >> subSampling + : partitionInfo.ModeInfo.GetTransformUnitCount(Av1PlaneType.Uv); + + int lumaTransformInfoIndex = partitionInfo.ModeInfo.GetFirstTransformLocation(Av1PlaneType.Y); + int chromaBlueTransformInfoIndex = partitionInfo.ModeInfo.GetFirstTransformLocation(Av1PlaneType.Uv); + int chromaRedTransformInfoIndex = chromaBlueTransformInfoIndex + chromaTransformUnitCount; + int forceSplitCount = 0; + + // AV1 forces residual traversal into at most 64x64 regions even when the coding block is larger. + // transformUnitCount preserves the transform geometry generated for each such region and plane. + for (int row = 0; row < maxBlocksHigh; row += modeUnitBlocksHigh) + { + for (int column = 0; column < maxBlocksWide; column += modeUnitBlocksWide) + { + for (int plane = 0; plane < planeCount; ++plane) + { + int totalTransformUnitCount; + int transformUnitCount; + int subX = (plane > 0 && this.SequenceHeader.ColorConfig.SubSamplingX) ? 1 : 0; + int subY = (plane > 0 && this.SequenceHeader.ColorConfig.SubSamplingY) ? 1 : 0; + int transformInfoIndex = plane switch + { + 0 => lumaTransformInfoIndex, + 1 => chromaBlueTransformInfoIndex, + _ => chromaRedTransformInfoIndex, + }; + + if (plane != 0 && !partitionInfo.IsChroma) + { + continue; + } + + Span transformInfoSpan = (plane == 0) ? superblockInfo.GetTransformInfoY() : superblockInfo.GetTransformInfoUv(); + if (isLosslessBlock) + { + // Lossless coding fixes transforms at 4x4, so count each clipped 4x4 unit + // directly after applying the plane's chroma subsampling. + int unitHeight = Av1Math.RoundPowerOf2(Math.Min(modeUnitBlocksHigh + row, maxBlocksHigh), 0); + int unitWidth = Av1Math.RoundPowerOf2(Math.Min(modeUnitBlocksWide + column, maxBlocksWide), 0); + DebugGuard.IsTrue(transformInfoSpan[transformInfoIndex].Size == Av1TransformSize.Size4x4, "Lossless frame shall have transform units of size 4x4."); + transformUnitCount = ((unitWidth - column) * (unitHeight - row)) >> (subX + subY); + } + else + { + totalTransformUnitCount = partitionInfo.ModeInfo.GetTransformUnitCount((Av1Plane)plane); + transformUnitCount = this.transformUnitCount[plane][forceSplitCount]; + + DebugGuard.IsFalse(totalTransformUnitCount == 0, nameof(totalTransformUnitCount), string.Empty); + DebugGuard.IsTrue( + totalTransformUnitCount == + this.transformUnitCount[plane][0] + this.transformUnitCount[plane][1] + + this.transformUnitCount[plane][2] + this.transformUnitCount[plane][3], + nameof(totalTransformUnitCount), + string.Empty); + } + + DebugGuard.IsFalse(transformUnitCount == 0, nameof(transformUnitCount), string.Empty); + for (int tu = 0; tu < transformUnitCount; tu++) + { + ref Av1TransformInfo transformInfo = ref transformInfoSpan[transformInfoIndex]; + DebugGuard.MustBeLessThanOrEqualTo(transformInfo.OffsetX, maxBlocksWide, nameof(transformInfo)); + DebugGuard.MustBeLessThanOrEqualTo(transformInfo.OffsetY, maxBlocksHigh, nameof(transformInfo)); + + int coefficientIndex = this.coefficientIndex[plane]; + int endOfBlock = 0; + int blockColumn = transformInfo.OffsetX; + int blockRow = transformInfo.OffsetY; + int startX = (partitionInfo.ColumnIndex >> subX) + blockColumn; + int startY = (partitionInfo.RowIndex >> subY) + blockRow; + + if (startX >= (this.FrameHeader.ModeInfoColumnCount >> subX) || + startY >= (this.FrameHeader.ModeInfoRowCount >> subY)) + { + return; + } + + if (!partitionInfo.ModeInfo.Skip) + { + Span coefficientBuffer = superblockInfo.GetCoefficients((Av1Plane)plane)[coefficientIndex..]; + endOfBlock = this.ParseTransformBlock( + ref reader, + ref partitionInfo, + tileInfo, + coefficientBuffer, + ref transformInfo, + plane, + blockColumn, + blockRow, + startX, + startY, + transformInfo.Size, + subX != 0, + subY != 0); + } + + // Each transform reserves its nominal area even when its residual is empty. EOB belongs + // to the descriptor, so the raster coefficient region contains no packed metadata prefix. + this.coefficientIndex[plane] += transformInfo.Size.GetWidth() * transformInfo.Size.GetHeight(); + transformInfo.EndOfBlock = (ushort)endOfBlock; + + transformInfoIndex++; + } + + // Each plane advances independently because its transform descriptors occupy a separate + // contiguous range. Scalar cursors avoid allocating a three-element array for every block. + switch (plane) + { + case 0: + lumaTransformInfoIndex = transformInfoIndex; + break; + case 1: + chromaBlueTransformInfoIndex = transformInfoIndex; + break; + default: + chromaRedTransformInfoIndex = transformInfoIndex; + break; + } + } + + forceSplitCount++; + } + } + } + + /// + /// Determines whether a luma coding block owns chroma mode and residual syntax at its frame position. + /// + /// The sequence header describing chroma subsampling. + /// The block origin in 4x4 luma mode-information units. + /// The luma block size. + /// when the block is a chroma reference position; otherwise, . + public static bool HasChroma(ObuSequenceHeader sequenceHeader, Point modeInfoLocation, Av1BlockSize blockSize) + { + int blockWide = blockSize.Get4x4WideCount(); + int blockHigh = blockSize.Get4x4HighCount(); + bool subX = sequenceHeader.ColorConfig.SubSamplingX; + bool subY = sequenceHeader.ColorConfig.SubSamplingY; + bool hasChroma = ((modeInfoLocation.Y & 0x01) != 0 || (blockHigh & 0x01) == 0 || !subY) && + ((modeInfoLocation.X & 0x01) != 0 || (blockWide & 0x01) == 0 || !subX); + return hasChroma; + } + + /// + /// Derives a transform block's entropy context and decodes its coefficient syntax. + /// + /// The tile symbol decoder. + /// The containing coding block. + /// The active tile boundaries. + /// The destination beginning at this transform's coefficient slot. + /// The transform geometry and syntax state to populate. + /// The zero-based color-plane index. + /// The transform's horizontal offset within the coding block in 4x4 units. + /// The transform's vertical offset within the coding block in 4x4 units. + /// The frame-relative transform column in 4x4 units of the target plane. + /// The frame-relative transform row in 4x4 units of the target plane. + /// The transform size. + /// A value indicating whether the target plane is horizontally subsampled. + /// A value indicating whether the target plane is vertically subsampled. + /// The decoded end-of-block coefficient position, or zero for an all-zero transform. + /// + /// Implements AV1 section 5.11.35. + /// + private int ParseTransformBlock( + ref Av1SymbolDecoder reader, + ref Av1PartitionInfo partitionInfo, + Av1TileInfo tileInfo, + Span coefficientBuffer, + ref Av1TransformInfo transformInfo, + int plane, + int blockColumn, + int blockRow, + int startX, + int startY, + Av1TransformSize transformSize, + bool subX, + bool subY) + { + int endOfBlock = 0; + Av1BlockSize planeBlockSize = partitionInfo.ModeInfo.BlockSize.GetSubsampled(subX, subY); + int transformBlockUnitWideCount = transformSize.Get4x4WideCount(); + int transformBlockUnitHighCount = transformSize.Get4x4HighCount(); + + if (partitionInfo.ModeBlockToRightEdge < 0) + { + int blocksWide = partitionInfo.GetMaxBlockWide(planeBlockSize, subX); + transformBlockUnitWideCount = Math.Min(transformBlockUnitWideCount, blocksWide - blockColumn); + } + + if (partitionInfo.ModeBlockToBottomEdge < 0) + { + int blocksHigh = partitionInfo.GetMaxBlockHigh(planeBlockSize, subY); + transformBlockUnitHighCount = Math.Min(transformBlockUnitHighCount, blocksHigh - blockRow); + } + + int aboveContextOffset = startX - (tileInfo.ModeInfoColumnStart >> (subX ? 1 : 0)); + int superblockRow = partitionInfo.SuperblockInfo.ModeInfoPosition.Y >> (subY ? 1 : 0); + int leftContextOffset = startY - superblockRow; + + // Above contexts are tile-column relative, while left contexts are reused from the start of each + // superblock row. Slicing both arrays here gives the entropy derivation the same pointer bases as the reference decoder. + Av1TransformBlockContext transformBlockContext = this.GetTransformBlockContext( + transformSize, + plane, + planeBlockSize, + transformBlockUnitHighCount, + transformBlockUnitWideCount, + aboveContextOffset, + leftContextOffset); + + endOfBlock = this.ParseCoefficients( + ref reader, + ref partitionInfo, + blockRow, + blockColumn, + aboveContextOffset, + leftContextOffset, + plane, + transformBlockContext, + transformSize, + ref transformInfo, + coefficientBuffer); + + if (plane == 0) + { + this.FrameInfo.RecordLumaTransformType(transformInfo.Type); + } + + return endOfBlock; + } + + /// + /// Decodes transform coefficients and updates the coefficient neighbor contexts for one color plane. + /// + /// The tile symbol decoder. + /// The containing coding block. + /// The transform row within the coding block in 4x4 units of the target plane. + /// The transform column within the coding block in 4x4 units of the target plane. + /// The first tile-relative above context covered by the transform. + /// The first superblock-row-relative left context covered by the transform. + /// The zero-based color-plane index. + /// The coefficient skip and DC-sign entropy contexts. + /// The transform size. + /// The transform syntax state to populate. + /// The destination beginning at this transform's coefficient slot. + /// The decoded end-of-block coefficient position, or zero for an all-zero transform. + /// + /// Implements AV1 section 5.11.39. + /// + private int ParseCoefficients( + ref Av1SymbolDecoder reader, + ref Av1PartitionInfo partitionInfo, + int blockRow, + int blockColumn, + int aboveOffset, + int leftOffset, + int plane, + Av1TransformBlockContext transformBlockContext, + Av1TransformSize transformSize, + ref Av1TransformInfo transformInfo, + Span coefficientBuffer) + { + int width = transformSize.GetWidth(); + int height = transformSize.GetHeight(); + Av1TransformSize transformSizeContext = Av1SymbolContextHelper.GetTransformSizeContext(transformSize); + Av1PlaneType planeType = (Av1PlaneType)Math.Min(plane, 1); + Point blockPosition = new(blockColumn, blockRow); + bool isLossless = this.FrameHeader.LosslessArray[partitionInfo.ModeInfo.SegmentId]; + bool subX = plane > 0 && this.SequenceHeader.ColorConfig.SubSamplingX; + bool subY = plane > 0 && this.SequenceHeader.ColorConfig.SubSamplingY; + Av1BlockSize planeBlockSize = partitionInfo.ModeInfo.BlockSize.GetSubsampled(subX, subY); + int blocksWide = partitionInfo.GetMaxBlockWide(planeBlockSize, subX); + int blocksHigh = partitionInfo.GetMaxBlockHigh(planeBlockSize, subY); + bool usesInterTransformSet = partitionInfo.ModeInfo.ReferenceFrames[0] >= Av1ReferenceFrameType.Last || partitionInfo.ModeInfo.UseIntraBlockCopy; + Av1TransformType lumaTransformType = usesInterTransformSet && plane > 0 && !isLossless + ? partitionInfo.GetLumaTransformType(blockPosition, subX, subY) + : Av1TransformType.DctDct; + + return reader.ReadCoefficients( + partitionInfo.ModeInfo, + blockPosition, + this.aboveNeighborContext.GetContext(plane), + this.leftNeighborContext.GetContext(plane), + aboveOffset, + leftOffset, + plane, + blocksWide, + blocksHigh, + transformBlockContext, + transformSize, + isLossless, + this.FrameHeader.UseReducedTransformSet, + lumaTransformType, + ref transformInfo, + partitionInfo.ModeBlockToRightEdge, + partitionInfo.ModeBlockToBottomEdge, + this.coefficientLevels, + coefficientBuffer, + this.inverseQuantizer); + } + + /// + /// Derives coefficient skip and DC-sign contexts from the transform block's above and left neighbors. + /// + /// The transform size. + /// The zero-based color-plane index. + /// The containing block size on the target plane. + /// The transform height clipped to the frame in 4x4 units. + /// The transform width clipped to the frame in 4x4 units. + /// The first tile-relative above context covered by the transform. + /// The first superblock-row-relative left context covered by the transform. + /// The derived transform-block entropy contexts. + private Av1TransformBlockContext GetTransformBlockContext( + Av1TransformSize transformSize, + int plane, + Av1BlockSize planeBlockSize, + int transformBlockUnitHighCount, + int transformBlockUnitWideCount, + int aboveOffset, + int leftOffset) + { + Av1TransformBlockContext transformBlockContext = default; + ReadOnlySpan aboveContext = this.aboveNeighborContext.GetContext(plane)[aboveOffset..]; + ReadOnlySpan leftContext = this.leftNeighborContext.GetContext(plane)[leftOffset..]; + int dcSign = 0; + int k = 0; + int mask = (1 << Av1Constants.CoefficientContextBitCount) - 1; + + // The high bits of each neighbor value encode its DC sign class. Summing both edges maps + // negative, balanced, and positive neighborhoods to the AV1 DC-sign context. + do + { + uint sign = (uint)aboveContext[k] >> Av1Constants.CoefficientContextBitCount; + DebugGuard.MustBeLessThanOrEqualTo(sign, 2U, nameof(sign)); + dcSign += Signs[(int)sign]; + } + while (++k < transformBlockUnitWideCount); + + k = 0; + do + { + uint sign = (uint)leftContext[k] >> Av1Constants.CoefficientContextBitCount; + DebugGuard.MustBeLessThanOrEqualTo(sign, 2U, nameof(sign)); + dcSign += Signs[(int)sign]; + } + while (++k < transformBlockUnitHighCount); + + transformBlockContext.DcSignContext = DcSignContexts[dcSign + (Av1Constants.MaxTransformSizeUnit << 1)]; + + if (plane == 0) + { + if (planeBlockSize == transformSize.ToBlockSize()) + { + transformBlockContext.SkipContext = 0; + } + else + { + // Luma skip contexts preserve both the weakest neighboring level and whether either edge is stronger. + int top = 0; + int left = 0; + + k = 0; + do + { + top |= aboveContext[k]; + } + while (++k < transformBlockUnitWideCount); + top &= mask; + + k = 0; + do + { + left |= leftContext[k]; + } + while (++k < transformBlockUnitHighCount); + left &= mask; + + transformBlockContext.SkipContext = Av1SymbolContextHelper.GetTransformBlockSkipContext(top, left); + } + } + else + { + // Chroma needs only the presence of nonzero levels on each edge, plus an offset that + // distinguishes a transform smaller than its containing plane block. + int contextBase = GetEntropyContext(transformSize, aboveContext, leftContext); + int contextOffset = planeBlockSize.GetPelsLog2Count() > transformSize.ToBlockSize().GetPelsLog2Count() ? 10 : 7; + transformBlockContext.SkipContext = contextBase + contextOffset; + } + + return transformBlockContext; + } + + /// + /// Determines whether the above and left edges contain nonzero chroma coefficient contexts. + /// + /// The transform size that selects how many edge entries to inspect. + /// The above coefficient contexts. + /// The left coefficient contexts. + /// The sum of the nonzero-above and nonzero-left flags. + private static int GetEntropyContext(Av1TransformSize transformSize, ReadOnlySpan above, ReadOnlySpan left) + { + bool aboveEntropyContext = false; + bool leftEntropyContext = false; + int transformBlockUnitWideCount = transformSize.Get4x4WideCount(); + int transformBlockUnitHighCount = transformSize.Get4x4HighCount(); + + // the reference decoder tests the context bytes through packed native loads. Enumerating the same transform-width and + // transform-height entries avoids unaligned reads while preserving the required any-nonzero result. + for (int i = 0; i < transformBlockUnitWideCount; i++) + { + if (above[i] != 0) + { + aboveEntropyContext = true; + break; + } + } + + for (int i = 0; i < transformBlockUnitHighCount; i++) + { + if (left[i] != 0) + { + leftEntropyContext = true; + break; + } + } + + return (aboveEntropyContext ? 1 : 0) + (leftEntropyContext ? 1 : 0); + } + + /// + /// Selects the transform size for a coding block from lossless, explicit-selection, or maximum-size rules. + /// + /// The tile symbol decoder. + /// The current coding block. + /// The containing superblock. + /// The active tile boundaries. + /// A value indicating whether transform-size selection syntax is allowed at this node. + /// The selected transform size. + /// Implements AV1 section 5.11.15. + private Av1TransformSize ReadTransformSize( + ref Av1SymbolDecoder reader, + ref Av1PartitionInfo partitionInfo, + Av1SuperblockInfo superblockInfo, + Av1TileInfo tileInfo, + bool allowSelect) + { + ref Av1BlockModeInfo modeInfo = ref partitionInfo.ModeInfo; + if (this.FrameHeader.LosslessArray[modeInfo.SegmentId]) + { + return Av1TransformSize.Size4x4; + } + + if (modeInfo.BlockSize > Av1BlockSize.Block4x4 && allowSelect && this.FrameHeader.TransformMode == Av1TransformMode.Select) + { + return this.ReadSelectedTransformSize(ref reader, ref partitionInfo, superblockInfo, tileInfo); + } + + return modeInfo.BlockSize.GetMaximumTransformSize(); + } + + /// + /// Reads a transform size using the available above and left transform-size contexts. + /// + /// The tile symbol decoder. + /// The current coding block. + /// The containing superblock. + /// The active tile boundaries. + /// The decoded transform size. + private Av1TransformSize ReadSelectedTransformSize( + ref Av1SymbolDecoder reader, + ref Av1PartitionInfo partitionInfo, + Av1SuperblockInfo superblockInfo, + Av1TileInfo tileInfo) + { + int context = 0; + Av1TransformSize maxTransformSize = partitionInfo.ModeInfo.BlockSize.GetMaximumTransformSize(); + int aboveWidth = this.aboveNeighborContext.AboveTransformWidth[partitionInfo.ColumnIndex - tileInfo.ModeInfoColumnStart]; + int above = (aboveWidth >= maxTransformSize.GetWidth()) ? 1 : 0; + int leftHeight = this.leftNeighborContext.LeftTransformHeight[partitionInfo.RowIndex - superblockInfo.ModeInfoPosition.Y]; + int left = (leftHeight >= maxTransformSize.GetHeight()) ? 1 : 0; + bool hasAbove = partitionInfo.AvailableAbove; + bool hasLeft = partitionInfo.AvailableLeft; + + // Inter neighbors expose their coding-block extent here rather than their residual transform extent. This + // keeps intra transform-size selection independent of whether the neighboring inter block split its tree. + if (hasAbove) + { + Av1BlockModeInfo aboveModeInfo = superblockInfo.GetModeInfoAt( + new Point(partitionInfo.ColumnIndex, partitionInfo.RowIndex - 1)); + + if (aboveModeInfo.UseIntraBlockCopy || aboveModeInfo.ReferenceFrames[0] > Av1ReferenceFrameType.Intra) + { + above = aboveModeInfo.BlockSize.GetWidth() >= maxTransformSize.GetWidth() ? 1 : 0; + } + } + + if (hasLeft) + { + Av1BlockModeInfo leftModeInfo = superblockInfo.GetModeInfoAt( + new Point(partitionInfo.ColumnIndex - 1, partitionInfo.RowIndex)); + + if (leftModeInfo.UseIntraBlockCopy || leftModeInfo.ReferenceFrames[0] > Av1ReferenceFrameType.Intra) + { + left = leftModeInfo.BlockSize.GetHeight() >= maxTransformSize.GetHeight() ? 1 : 0; + } + } + + if (hasAbove && hasLeft) + { + context = above + left; + } + else if (hasAbove) + { + context = above; + } + else if (hasLeft) + { + context = left; + } + else + { + context = 0; + } + + return reader.ReadTransformSize(partitionInfo.ModeInfo.BlockSize, context); + } + + /// + /// Reads a coding block's transform size, updates neighbor contexts, and creates its transform geometry records. + /// + /// The tile symbol decoder. + /// The block origin in 4x4 mode-information units. + /// The current coding block. + /// The containing superblock. + /// The active tile boundaries. + private void ReadBlockTransformSize( + ref Av1SymbolDecoder reader, + Point modeInfoLocation, + ref Av1PartitionInfo partitionInfo, + Av1SuperblockInfo superblockInfo, + Av1TileInfo tileInfo) + { + Av1BlockSize blockSize = partitionInfo.ModeInfo.BlockSize; + int block4x4Width = blockSize.Get4x4WideCount(); + int block4x4Height = blockSize.Get4x4HighCount(); + + ref Av1BlockModeInfo modeInfo = ref partitionInfo.ModeInfo; + bool usesInterTransformSyntax = modeInfo.ReferenceFrames[0] >= Av1ReferenceFrameType.Last || modeInfo.UseIntraBlockCopy; + + this.transformUnitCount[(int)Av1Plane.Y][..4].Clear(); + this.transformUnitCount[(int)Av1Plane.U][..4].Clear(); + this.transformUnitCount[(int)Av1Plane.V][..4].Clear(); + + if (usesInterTransformSyntax && + !modeInfo.Skip && + this.FrameHeader.TransformMode == Av1TransformMode.Select && + blockSize > Av1BlockSize.Block4x4) + { + this.ReadVariableTransformInfo( + ref reader, + modeInfoLocation, + ref partitionInfo, + superblockInfo, + tileInfo); + + return; + } + + // A skipped inter block derives its maximum transform size without a symbol. Intra blocks still select a + // transform size when the frame enables selection because skip_txfm does not suppress their size syntax. + bool allowSelect = !usesInterTransformSyntax || !modeInfo.Skip; + Av1TransformSize transformSize = this.ReadTransformSize( + ref reader, + ref partitionInfo, + superblockInfo, + tileInfo, + allowSelect); + + bool skippedInterBlock = usesInterTransformSyntax && modeInfo.Skip; + this.aboveNeighborContext.UpdateTransformation(modeInfoLocation, tileInfo, transformSize, blockSize, skippedInterBlock); + this.leftNeighborContext.UpdateTransformation(modeInfoLocation, superblockInfo, transformSize, blockSize, skippedInterBlock); + this.UpdateTransformInfo(ref partitionInfo, superblockInfo, blockSize, transformSize); + } + + /// + /// Reads the recursive luma transform partition used by a non-skipped inter block. + /// + /// The tile symbol decoder. + /// The coding-block origin in frame mode-information units. + /// The current coding block. + /// The containing superblock. + /// The active tile boundaries. + private void ReadVariableTransformInfo( + ref Av1SymbolDecoder reader, + Point modeInfoLocation, + ref Av1PartitionInfo partitionInfo, + Av1SuperblockInfo superblockInfo, + Av1TileInfo tileInfo) + { + Av1BlockSize blockSize = partitionInfo.ModeInfo.BlockSize; + Av1TransformSize maximumTransformSize = blockSize.GetMaximumTransformSize(); + int maximumBlocksWide = partitionInfo.GetMaxBlockWide(blockSize, false); + int maximumBlocksHigh = partitionInfo.GetMaxBlockHigh(blockSize, false); + int regionWidth = maximumTransformSize.Get4x4WideCount(); + int regionHeight = maximumTransformSize.Get4x4HighCount(); + int transformInfoIndex = partitionInfo.ModeInfo.GetFirstTransformLocation(Av1PlaneType.Y); + int totalTransformUnitCount = 0; + int regionIndex = 0; + + // Large blocks are visited as independent maximum-transform regions. Keeping the same region order as residual + // parsing lets each region retain an exact transform count without a second map or temporary allocation. + for (int blockRow = 0; blockRow < maximumBlocksHigh; blockRow += regionHeight) + { + for (int blockColumn = 0; blockColumn < maximumBlocksWide; blockColumn += regionWidth) + { + int firstRegionTransform = totalTransformUnitCount; + this.ReadVariableTransformNode( + ref reader, + modeInfoLocation, + ref partitionInfo, + superblockInfo, + tileInfo, + maximumTransformSize, + depth: 0, + blockRow, + blockColumn, + ref transformInfoIndex, + ref totalTransformUnitCount); + + this.transformUnitCount[(int)Av1Plane.Y][regionIndex] = totalTransformUnitCount - firstRegionTransform; + regionIndex++; + } + } + + this.UpdateTransformInfo( + ref partitionInfo, + superblockInfo, + blockSize, + maximumTransformSize, + preserveLuma: true, + existingLumaTransformUnitCount: totalTransformUnitCount); + } + + /// + /// Reads one node of the inter variable-transform tree and appends its leaf transform descriptors. + /// + private void ReadVariableTransformNode( + ref Av1SymbolDecoder reader, + Point modeInfoLocation, + ref Av1PartitionInfo partitionInfo, + Av1SuperblockInfo superblockInfo, + Av1TileInfo tileInfo, + Av1TransformSize transformSize, + int depth, + int blockRow, + int blockColumn, + ref int transformInfoIndex, + ref int transformUnitCount) + { + Av1BlockSize blockSize = partitionInfo.ModeInfo.BlockSize; + int maximumBlocksWide = partitionInfo.GetMaxBlockWide(blockSize, false); + int maximumBlocksHigh = partitionInfo.GetMaxBlockHigh(blockSize, false); + if (blockRow >= maximumBlocksHigh || blockColumn >= maximumBlocksWide) + { + return; + } + + bool split = false; + if (transformSize > Av1TransformSize.Size4x4 && depth < Av1Constants.MaxVarTransform) + { + int aboveOffset = modeInfoLocation.X - tileInfo.ModeInfoColumnStart + blockColumn; + int leftOffset = modeInfoLocation.Y - superblockInfo.ModeInfoPosition.Y + blockRow; + int transformWidth = transformSize.GetWidth(); + int transformHeight = transformSize.GetHeight(); + int above = this.aboveNeighborContext.AboveTransformWidth[aboveOffset] < transformWidth ? 1 : 0; + int left = this.leftNeighborContext.LeftTransformHeight[leftOffset] < transformHeight ? 1 : 0; + int maximumDimension = Math.Max(blockSize.GetWidth(), blockSize.GetHeight()); + Av1TransformSize maximumSquareTransform = maximumDimension switch + { + >= 64 => Av1TransformSize.Size64x64, + >= 32 => Av1TransformSize.Size32x32, + >= 16 => Av1TransformSize.Size16x16, + _ => Av1TransformSize.Size8x8 + }; + + int category = ((transformSize.GetSquareUpSize() != maximumSquareTransform && maximumSquareTransform > Av1TransformSize.Size8x8) ? 1 : 0) + + ((((int)Av1TransformSize.SquareSizes - 1) - (int)maximumSquareTransform) * 2); + + int context = (category * 3) + above + left; + split = reader.ReadTransformPartition(context); + } + + if (split) + { + Av1TransformSize subTransformSize = transformSize.GetSubSize(); + int subWidth = subTransformSize.Get4x4WideCount(); + int subHeight = subTransformSize.Get4x4HighCount(); + int width = transformSize.Get4x4WideCount(); + int height = transformSize.Get4x4HighCount(); + for (int row = 0; row < height; row += subHeight) + { + for (int column = 0; column < width; column += subWidth) + { + this.ReadVariableTransformNode( + ref reader, + modeInfoLocation, + ref partitionInfo, + superblockInfo, + tileInfo, + subTransformSize, + depth + 1, + blockRow + row, + blockColumn + column, + ref transformInfoIndex, + ref transformUnitCount); + } + } + + return; + } + + Span transformInfo = superblockInfo.GetTransformInfoY(); + transformInfo[transformInfoIndex] = new Av1TransformInfo(transformSize, blockColumn, blockRow); + transformInfoIndex++; + transformUnitCount++; + + Point transformLocation = new(modeInfoLocation.X + blockColumn, modeInfoLocation.Y + blockRow); + Av1BlockSize transformBlockSize = transformSize.ToBlockSize(); + this.aboveNeighborContext.UpdateTransformation(transformLocation, tileInfo, transformSize, transformBlockSize, false); + this.leftNeighborContext.UpdateTransformation(transformLocation, superblockInfo, transformSize, transformBlockSize, false); + } + + /// + /// Populates luma and chroma transform-information records in residual traversal order. + /// + /// The current coding block. + /// The containing superblock and transform storage. + /// The coding block size. + /// The selected luma transform size. + /// Indicates whether variable-transform traversal already populated luma descriptors. + /// The number of luma descriptors already populated. + private unsafe void UpdateTransformInfo( + ref Av1PartitionInfo partitionInfo, + Av1SuperblockInfo superblockInfo, + Av1BlockSize blockSize, + Av1TransformSize transformSize, + bool preserveLuma = false, + int existingLumaTransformUnitCount = 0) + { + int transformInfoYIndex = partitionInfo.ModeInfo.GetFirstTransformLocation(Av1PlaneType.Y); + int transformInfoUvIndex = partitionInfo.ModeInfo.GetFirstTransformLocation(Av1PlaneType.Uv); + Span lumaTransformInfo = superblockInfo.GetTransformInfoY(); + Span chromaTransformInfo = superblockInfo.GetTransformInfoUv(); + int totalLumaTransformUnitCount = existingLumaTransformUnitCount; + int totalChromaTransformUnitCount = 0; + int forceSplitCount = 0; + bool subX = this.SequenceHeader.ColorConfig.SubSamplingX; + bool subY = this.SequenceHeader.ColorConfig.SubSamplingY; + int maxBlockWide = partitionInfo.GetMaxBlockWide(blockSize, false); + int maxBlockHigh = partitionInfo.GetMaxBlockHigh(blockSize, false); + int width = 64 >> 2; + int height = 64 >> 2; + width = Math.Min(width, maxBlockWide); + height = Math.Min(height, maxBlockHigh); + + bool isLossLess = this.FrameHeader.LosslessArray[partitionInfo.ModeInfo.SegmentId]; + Av1TransformSize transformSizeUv = isLossLess ? Av1TransformSize.Size4x4 : blockSize.GetMaxUvTransformSize(subX, subY); + + // Residual syntax visits at most 64x64 luma regions. Record transform geometry in the same + // nested region/row/column order so coefficient parsing and reconstruction consume matching spans. + for (int idy = 0; idy < maxBlockHigh; idy += height) + { + for (int idx = 0; idx < maxBlockWide; idx += width, forceSplitCount++) + { + int lumaTransformUnitCount = 0; + int chromaTransformUnitCount = 0; + + int unitHeight = Av1Math.RoundPowerOf2(Math.Min(height + idy, maxBlockHigh), 0); + int unitWidth = Av1Math.RoundPowerOf2(Math.Min(width + idx, maxBlockWide), 0); + if (!preserveLuma) + { + // Luma transform offsets remain relative to the coding block in 4x4 luma units. + int lumaStepColumn = transformSize.Get4x4WideCount(); + int lumaStepRow = transformSize.Get4x4HighCount(); + for (int blockRow = idy; blockRow < unitHeight; blockRow += lumaStepRow) + { + for (int blockColumn = idx; blockColumn < unitWidth; blockColumn += lumaStepColumn) + { + lumaTransformInfo[transformInfoYIndex] = new Av1TransformInfo( + transformSize, blockColumn, blockRow); + transformInfoYIndex++; + lumaTransformUnitCount++; + totalLumaTransformUnitCount++; + } + } + + this.transformUnitCount[(int)Av1Plane.Y][forceSplitCount] = lumaTransformUnitCount; + } + + if (this.SequenceHeader.ColorConfig.IsMonochrome || !partitionInfo.IsChroma) + { + continue; + } + + // Chroma geometry is rounded to the subsampling grid before stepping its transform size. + int stepColumn = transformSizeUv.Get4x4WideCount(); + int stepRow = transformSizeUv.Get4x4HighCount(); + + unitHeight = Av1Math.RoundPowerOf2(Math.Min(height + idy, maxBlockHigh), subY ? 1 : 0); + unitWidth = Av1Math.RoundPowerOf2(Math.Min(width + idx, maxBlockWide), subX ? 1 : 0); + int planeRow = idy >> (subY ? 1 : 0); + int planeColumn = idx >> (subX ? 1 : 0); + + // The 64x64 region cursor is expressed on the luma grid. Chroma transform offsets use the + // target plane's 4x4 grid, matching the reference decoder's row/column subsampling before transform traversal. + for (int blockRow = planeRow; blockRow < unitHeight; blockRow += stepRow) + { + for (int blockColumn = planeColumn; blockColumn < unitWidth; blockColumn += stepColumn) + { + chromaTransformInfo[transformInfoUvIndex] = new Av1TransformInfo( + transformSizeUv, blockColumn, blockRow); + transformInfoUvIndex++; + chromaTransformUnitCount++; + totalChromaTransformUnitCount++; + } + } + + this.transformUnitCount[(int)Av1Plane.U][forceSplitCount] = chromaTransformUnitCount; + this.transformUnitCount[(int)Av1Plane.V][forceSplitCount] = chromaTransformUnitCount; + } + } + + // U and V share transform geometry, so append a second copy for V after the complete U sequence. + if (totalChromaTransformUnitCount != 0) + { + DebugGuard.IsTrue( + (transformInfoUvIndex - totalChromaTransformUnitCount) == + partitionInfo.ModeInfo.GetFirstTransformLocation(Av1PlaneType.Uv), + nameof(totalChromaTransformUnitCount)); + int originalIndex = transformInfoUvIndex - totalChromaTransformUnitCount; + ref Av1TransformInfo originalInfo = ref chromaTransformInfo[originalIndex]; + ref Av1TransformInfo infoV = ref chromaTransformInfo[transformInfoUvIndex]; + for (int i = 0; i < totalChromaTransformUnitCount; i++) + { + // U and V share transform geometry, but their entropy state and coefficients remain independent. + infoV = new Av1TransformInfo(originalInfo); + originalInfo = ref Unsafe.Add(ref originalInfo, 1); + infoV = ref Unsafe.Add(ref infoV, 1); + } + } + + partitionInfo.ModeInfo.SetTransformUnitCount(Av1PlaneType.Y, totalLumaTransformUnitCount); + partitionInfo.ModeInfo.SetTransformUnitCount(Av1PlaneType.Uv, totalChromaTransformUnitCount); + + this.firstTransformOffset[(int)Av1PlaneType.Y] += totalLumaTransformUnitCount; + this.firstTransformOffset[(int)Av1PlaneType.Uv] += totalChromaTransformUnitCount << 1; + } + + /// + /// Reads luma and chroma palette-map tokens when a block selects palette prediction. + /// + /// The tile symbol decoder. + /// The current coding block. + /// Implements AV1 section 5.11.49. + private void ReadPaletteTokens(ref Av1SymbolDecoder reader, ref Av1PartitionInfo partitionInfo) + { + ref Av1BlockModeInfo modeInfo = ref partitionInfo.ModeInfo; + if (modeInfo.GetPaletteSize(Av1PlaneType.Y) != 0) + { + GetPaletteMapDimensions( + ref partitionInfo, + Av1PlaneType.Y, + this.SequenceHeader.ColorConfig, + out int planeWidth, + out int planeHeight, + out int rows, + out int columns); + + Point position = modeInfo.PositionInSuperblock; + Rectangle bounds = new(position.X << Av1Constants.ModeInfoSizeLog2, position.Y << Av1Constants.ModeInfoSizeLog2, planeWidth, planeHeight); + Buffer2DRegion colorIndexMap = new(this.lumaPaletteColorIndexMap, bounds); + + DecodePaletteColorMap( + ref reader, + modeInfo.GetPaletteSize(Av1PlaneType.Y), + Av1PlaneType.Y, + planeWidth, + planeHeight, + rows, + columns, + colorIndexMap); + + if (this.FrameDecoder is not null) + { + modeInfo.SetPaletteColorIndexMap(Av1PlaneType.Y, colorIndexMap.Bounds); + } + } + + if (modeInfo.GetPaletteSize(Av1PlaneType.Uv) != 0) + { + GetPaletteMapDimensions( + ref partitionInfo, + Av1PlaneType.Uv, + this.SequenceHeader.ColorConfig, + out int planeWidth, + out int planeHeight, + out int rows, + out int columns); + + int subX = this.SequenceHeader.ColorConfig.SubSamplingX ? 1 : 0; + int subY = this.SequenceHeader.ColorConfig.SubSamplingY ? 1 : 0; + Point position = modeInfo.PositionInSuperblock; + Rectangle bounds = new((position.X << Av1Constants.ModeInfoSizeLog2) >> subX, (position.Y << Av1Constants.ModeInfoSizeLog2) >> subY, planeWidth, planeHeight); + Buffer2DRegion colorIndexMap = new(this.chromaPaletteColorIndexMap, bounds); + + DecodePaletteColorMap( + ref reader, + modeInfo.GetPaletteSize(Av1PlaneType.Uv), + Av1PlaneType.Uv, + planeWidth, + planeHeight, + rows, + columns, + colorIndexMap); + + if (this.FrameDecoder is not null) + { + modeInfo.SetPaletteColorIndexMap(Av1PlaneType.Uv, colorIndexMap.Bounds); + } + } + } + + /// + /// Reads the prediction, segmentation, skip, quantizer, and filter mode information for a coding block. + /// + /// The tile symbol decoder. + /// The current coding block. + /// The active tile boundaries. + /// Implements the frame-type dispatch in AV1 section 5.11.6. + private void ReadModeInfo(ref Av1SymbolDecoder reader, ref Av1PartitionInfo partitionInfo, Av1TileInfo tileInfo) + { + if (this.FrameHeader.IsIntra) + { + this.ReadIntraFrameModeInfo(ref reader, ref partitionInfo, tileInfo); + } + else + { + this.ReadInterFrameModeInfo(ref reader, ref partitionInfo, tileInfo); + } + } + + /// + /// Reads the common inter-frame block prefix and the supported intra or inter prediction branch in bitstream order. + /// + /// The tile symbol decoder. + /// The current coding block and its neighbors. + /// The active tile boundaries used by reference-motion-vector searches. + /// Implements the prefix, intra, and translational inter branches of AV1 section 5.11.7. + public void ReadInterFrameModeInfo(ref Av1SymbolDecoder reader, ref Av1PartitionInfo partitionInfo, Av1TileInfo tileInfo) + { + ref Av1BlockModeInfo modeInfo = ref partitionInfo.ModeInfo; + modeInfo.MotionVectors.Clear(); + this.ReadInterSegmentId(ref reader, ref partitionInfo, beforeSkip: true); + + modeInfo.SkipMode = this.ReadSkipMode(ref reader, ref partitionInfo); + modeInfo.Skip = modeInfo.SkipMode || this.ReadSkip(ref reader, ref partitionInfo); + + if (!this.FrameHeader.SegmentationParameters.SegmentIdPrecedesSkip) + { + this.ReadInterSegmentId(ref reader, ref partitionInfo, beforeSkip: false); + } + + this.ReadCdef(ref reader, ref partitionInfo); + + if (this.FrameHeader.DeltaQParameters.IsPresent) + { + this.ReadDeltaQuantizerIndex(ref reader, ref partitionInfo); + this.ReadDeltaLoopFilter(ref reader, ref partitionInfo); + } + + bool isInterBlock = modeInfo.SkipMode || this.ReadIsInter(ref reader, ref partitionInfo); + if (isInterBlock) + { + modeInfo.SetPaletteSizes(0, 0); + modeInfo.UvMode = Av1ChromaPredictionMode.DC; + this.ReadReferenceFrames(ref reader, ref partitionInfo); + + Av1ReferenceFrameType referenceFrame = modeInfo.ReferenceFrames[0]; + Av1ReferenceFrameType secondaryReferenceFrame = modeInfo.ReferenceFrames[1]; + bool isCompound = secondaryReferenceFrame > Av1ReferenceFrameType.Intra; + + ref Av1ReferenceMotionVectors referenceMotionVectors = ref this.referenceMotionVectors; + referenceMotionVectors.Build( + ref partitionInfo, + tileInfo, + this.FrameInfo, + this.SequenceHeader, + this.FrameHeader, + referenceFrame, + secondaryReferenceFrame); + + ObuSegmentationParameters segmentationParameters = this.FrameHeader.SegmentationParameters; + int segmentId = modeInfo.SegmentId; + bool usesForcedGlobalMotion = + segmentationParameters.IsFeatureActive(segmentId, ObuSegmentationLevelFeature.Skip) || + segmentationParameters.IsFeatureActive(segmentId, ObuSegmentationLevelFeature.GlobalMotionVector); + + modeInfo.ReferenceMotionVectorIndex = 0; + modeInfo.YMode = modeInfo.SkipMode + ? Av1PredictionMode.NearestNearestMotionVector + : usesForcedGlobalMotion + ? Av1PredictionMode.GlobalMotionVector + : isCompound + ? reader.ReadInterCompoundMode(referenceMotionVectors.ModeContext) + : reader.ReadInterMode(referenceMotionVectors.ModeContext); + + bool modeIsCompound = modeInfo.YMode is >= Av1PredictionMode.CompoundInterModeStart and < Av1PredictionMode.CompoundInterModeEnd; + if (isCompound != modeIsCompound) + { + throw new InvalidImageContentException("AV1 inter prediction mode does not match its reference-frame count."); + } + + if (modeInfo.YMode is Av1PredictionMode.NewMotionVector or Av1PredictionMode.NewNewMotionVector) + { + // NEWMV and NEW_NEWMV can advance across candidates zero through two. Each transmitted one selects + // the next candidate and exposes one further DRL decision when the stack contains it. + for (int index = 0; index < 2 && referenceMotionVectors.Count > index + 1; index++) + { + int context = Av1SymbolContextHelper.GetDrlContext(referenceMotionVectors.Weights, index); + bool advance = reader.ReadDrl(context); + modeInfo.ReferenceMotionVectorIndex = (byte)(index + (advance ? 1 : 0)); + if (!advance) + { + break; + } + } + } + else if (modeInfo.YMode is + Av1PredictionMode.NearMotionVector or + Av1PredictionMode.NearNearMotionVector or + Av1PredictionMode.NearNewMotionVector or + Av1PredictionMode.NewNearMotionVector) + { + // Modes containing NEARMV reserve candidate zero for NEARESTMV, so their two DRL decisions examine + // pairs one/two and two/three while storing a zero-based offset from the first near candidate. + for (int index = 1; index < 3 && referenceMotionVectors.Count > index + 1; index++) + { + int context = Av1SymbolContextHelper.GetDrlContext(referenceMotionVectors.Weights, index); + bool advance = reader.ReadDrl(context); + modeInfo.ReferenceMotionVectorIndex = (byte)(index + (advance ? 1 : 0) - 1); + if (!advance) + { + break; + } + } + } + + Av1MotionVectorPrecision precision = this.FrameHeader.MotionVectorPrecision; + + Span motionVectors = modeInfo.MotionVectors; + if (!isCompound) + { + motionVectors[0] = modeInfo.YMode switch + { + Av1PredictionMode.NewMotionVector => reader.ReadMotionVector( + referenceMotionVectors.GetNewReference(modeInfo.ReferenceMotionVectorIndex), + precision), + Av1PredictionMode.NearestMotionVector => referenceMotionVectors.Nearest, + Av1PredictionMode.NearMotionVector => referenceMotionVectors.GetNearReference(modeInfo.ReferenceMotionVectorIndex), + Av1PredictionMode.GlobalMotionVector => this.FrameHeader.GetGlobalMotionParameters()[(int)referenceFrame - 1].GetMotionVector( + this.FrameHeader.AllowHighPrecisionMotionVector, + modeInfo.BlockSize, + new Point(partitionInfo.ColumnIndex, partitionInfo.RowIndex), + this.FrameHeader.ForceIntegerMotionVector), + _ => throw new InvalidImageContentException("Invalid single-reference AV1 inter mode.") + }; + } + else + { + int referenceMotionVectorIndex = modeInfo.ReferenceMotionVectorIndex; + int newReferenceIndex = modeInfo.YMode is Av1PredictionMode.NearNewMotionVector or Av1PredictionMode.NewNearMotionVector + ? referenceMotionVectorIndex + 1 + : referenceMotionVectorIndex; + + Av1MotionVector primaryNearest = referenceMotionVectors.GetCompoundNearestReference(0); + Av1MotionVector secondaryNearest = referenceMotionVectors.GetCompoundNearestReference(1); + Av1MotionVector primaryNear = referenceMotionVectors.GetCompoundNearReference(referenceMotionVectorIndex, 0); + Av1MotionVector secondaryNear = referenceMotionVectors.GetCompoundNearReference(referenceMotionVectorIndex, 1); + + switch (modeInfo.YMode) + { + case Av1PredictionMode.NearestNearestMotionVector: + motionVectors[0] = primaryNearest; + motionVectors[1] = secondaryNearest; + break; + case Av1PredictionMode.NearNearMotionVector: + motionVectors[0] = primaryNear; + motionVectors[1] = secondaryNear; + break; + case Av1PredictionMode.NearestNewMotionVector: + motionVectors[0] = primaryNearest; + motionVectors[1] = reader.ReadMotionVector( + referenceMotionVectors.GetCompoundNewReference(newReferenceIndex, 1), + precision); + + break; + case Av1PredictionMode.NewNearestMotionVector: + motionVectors[0] = reader.ReadMotionVector( + referenceMotionVectors.GetCompoundNewReference(newReferenceIndex, 0), + precision); + + motionVectors[1] = secondaryNearest; + break; + case Av1PredictionMode.NearNewMotionVector: + motionVectors[0] = primaryNear; + motionVectors[1] = reader.ReadMotionVector( + referenceMotionVectors.GetCompoundNewReference(newReferenceIndex, 1), + precision); + + break; + case Av1PredictionMode.NewNearMotionVector: + motionVectors[0] = reader.ReadMotionVector( + referenceMotionVectors.GetCompoundNewReference(newReferenceIndex, 0), + precision); + + motionVectors[1] = secondaryNear; + break; + case Av1PredictionMode.GlobalGlobalMotionVector: + motionVectors[0] = this.FrameHeader.GetGlobalMotionParameters()[(int)referenceFrame - 1].GetMotionVector( + this.FrameHeader.AllowHighPrecisionMotionVector, + modeInfo.BlockSize, + new Point(partitionInfo.ColumnIndex, partitionInfo.RowIndex), + this.FrameHeader.ForceIntegerMotionVector); + + motionVectors[1] = this.FrameHeader.GetGlobalMotionParameters()[(int)secondaryReferenceFrame - 1].GetMotionVector( + this.FrameHeader.AllowHighPrecisionMotionVector, + modeInfo.BlockSize, + new Point(partitionInfo.ColumnIndex, partitionInfo.RowIndex), + this.FrameHeader.ForceIntegerMotionVector); + + break; + case Av1PredictionMode.NewNewMotionVector: + motionVectors[0] = reader.ReadMotionVector( + referenceMotionVectors.GetCompoundNewReference(newReferenceIndex, 0), + precision); + + motionVectors[1] = reader.ReadMotionVector( + referenceMotionVectors.GetCompoundNewReference(newReferenceIndex, 1), + precision); + + break; + default: + throw new InvalidImageContentException("Invalid compound-reference AV1 inter mode."); + } + } + + for (int index = 0; index < (isCompound ? 2 : 1); index++) + { + if (!motionVectors[index].IsValid) + { + throw new InvalidImageContentException("AV1 motion-vector component is outside the permitted range."); + } + } + + modeInfo.MotionMode = Av1MotionMode.SimpleTranslation; + modeInfo.UseInterIntraWedge = false; + modeInfo.InterIntraWedgeIndex = 0; + + int minimumBlockDimension = Math.Min(modeInfo.BlockSize.GetWidth(), modeInfo.BlockSize.GetHeight()); + if (!isCompound && !modeInfo.SkipMode && + this.SequenceHeader.EnableInterIntraCompound && + modeInfo.BlockSize is >= Av1BlockSize.Block8x8 and <= Av1BlockSize.Block32x32 && + reader.ReadIsInterIntra(modeInfo.BlockSize)) + { + // The synthetic INTRA_FRAME second reference is part of the decoded mode state: it suppresses motion + // variation syntax and lets reconstruction distinguish inter-intra from a regular single-reference block. + modeInfo.ReferenceFrames[1] = Av1ReferenceFrameType.Intra; + modeInfo.InterIntraMode = reader.ReadInterIntraMode(modeInfo.BlockSize); + modeInfo.SetAngleDelta(Av1PlaneType.Y, 0); + modeInfo.SetAngleDelta(Av1PlaneType.Uv, 0); + modeInfo.UseFilterIntra = false; + modeInfo.UseInterIntraWedge = reader.ReadUseInterIntraWedge(modeInfo.BlockSize); + if (modeInfo.UseInterIntraWedge) + { + modeInfo.InterIntraWedgeIndex = reader.ReadWedgeIndex(modeInfo.BlockSize); + } + } + + if (!isCompound && + modeInfo.ReferenceFrames[1] != Av1ReferenceFrameType.Intra && + this.FrameHeader.IsMotionModeSwitchable && + minimumBlockDimension >= 8 && + !modeInfo.SkipMode) + { + Av1MotionVariationCandidates candidates = this.motionVariationCandidates; + candidates.Build(ref partitionInfo, tileInfo, this.SequenceHeader, this.FrameHeader, referenceFrame); + + Av1GlobalMotionParameters selectedGlobalMotion = this.FrameHeader.GetGlobalMotionParameters()[(int)referenceFrame - 1]; + bool hasFixedGlobalMotionMode = + !this.FrameHeader.ForceIntegerMotionVector && + modeInfo.YMode == Av1PredictionMode.GlobalMotionVector && + selectedGlobalMotion.Type > Av1GlobalMotionType.Translation; + + if (candidates.HasOverlappableNeighbor && !hasFixedGlobalMotionMode) + { + bool allowWarpedMotion = false; + if (candidates.Count > 0 && this.FrameHeader.AllowWarpedMotion && !this.FrameHeader.ForceIntegerMotionVector) + { + int canonicalReferenceIndex = (int)referenceFrame - (int)Av1ReferenceFrameType.Last; + uint referenceSlot = this.FrameHeader.GetReferenceFrameIndices()[canonicalReferenceIndex]; + Av1ReferenceFrameStore referenceFrames = this.referenceFrames + ?? throw new InvalidImageContentException("AV1 warped-motion syntax requires a reconstructed reference map."); + + Av1FrameBuffer referenceFrameBuffer = referenceFrames.ResolveRequired((int)referenceSlot).FrameBuffer; + + // Local warped motion is excluded for a scaled reference. Width and height equality are the + // identity-scale test because both dimensions form the decoder's reference scale factors. + allowWarpedMotion = + referenceFrameBuffer.Width == this.FrameHeader.FrameSize.FrameWidth && + referenceFrameBuffer.Height == this.FrameHeader.FrameSize.FrameHeight; + } + + modeInfo.MotionMode = reader.ReadMotionMode(modeInfo.BlockSize, allowWarpedMotion); + if (modeInfo.MotionMode == Av1MotionMode.Warped) + { + modeInfo.WarpedMotionParameters = Av1GlobalMotionParameters.DeriveLocalProjection( + candidates.SourcePoints, + candidates.ReferencePoints, + modeInfo.BlockSize, + motionVectors[0], + new Point(partitionInfo.ColumnIndex, partitionInfo.RowIndex)); + } + } + } + + modeInfo.CompoundGroupIndex = false; + modeInfo.CompoundIndex = true; + modeInfo.CompoundType = Av1CompoundType.Average; + modeInfo.CompoundWedgeIndex = 0; + modeInfo.CompoundWedgeSign = false; + modeInfo.DifferenceWeightedMaskType = Av1DifferenceWeightedMaskType.Type38; + if (isCompound && !modeInfo.SkipMode) + { + bool maskedCompoundUsed = minimumBlockDimension >= 8 && this.SequenceHeader.EnableMaskedCompound; + if (maskedCompoundUsed) + { + int groupContext = Av1SymbolContextHelper.GetCompoundGroupIndexContext( + partitionInfo.AboveModeInfo, + partitionInfo.LeftModeInfo); + + modeInfo.CompoundGroupIndex = reader.ReadCompoundGroupIndex(groupContext); + } + + if (!modeInfo.CompoundGroupIndex) + { + if (this.SequenceHeader.OrderHintInfo.EnableJointCompound) + { + int compoundIndexContext = Av1SymbolContextHelper.GetCompoundIndexContext( + this.SequenceHeader.OrderHintInfo, + this.FrameHeader, + modeInfo, + partitionInfo.AboveModeInfo, + partitionInfo.LeftModeInfo); + + modeInfo.CompoundIndex = reader.ReadCompoundIndex(compoundIndexContext); + modeInfo.CompoundType = modeInfo.CompoundIndex + ? Av1CompoundType.Average + : Av1CompoundType.DistanceWeighted; + } + } + else + { + bool supportsWedge = modeInfo.BlockSize is + Av1BlockSize.Block8x8 or + Av1BlockSize.Block8x16 or + Av1BlockSize.Block16x8 or + Av1BlockSize.Block16x16 or + Av1BlockSize.Block16x32 or + Av1BlockSize.Block32x16 or + Av1BlockSize.Block32x32 or + Av1BlockSize.Block8x32 or + Av1BlockSize.Block32x8; + + modeInfo.CompoundType = supportsWedge + ? reader.ReadMaskedCompoundType(modeInfo.BlockSize) + : Av1CompoundType.DifferenceWeighted; + + if (modeInfo.CompoundType == Av1CompoundType.Wedge) + { + modeInfo.CompoundWedgeIndex = reader.ReadWedgeIndex(modeInfo.BlockSize); + modeInfo.CompoundWedgeSign = reader.ReadLiteral(1) != 0; + } + else + { + modeInfo.DifferenceWeightedMaskType = (Av1DifferenceWeightedMaskType)reader.ReadLiteral(1); + } + } + } + + Span interpolationFilters = modeInfo.InterpolationFilters; + Av1InterpolationFilter frameInterpolationFilter = this.FrameHeader.InterpolationFilter; + Av1GlobalMotionParameters globalMotion = this.FrameHeader.GetGlobalMotionParameters()[(int)referenceFrame - 1]; + bool usesNonTranslationalGlobalMotion = + minimumBlockDimension >= 8 && + ((modeInfo.YMode == Av1PredictionMode.GlobalMotionVector && globalMotion.Type != Av1GlobalMotionType.Translation) || + (modeInfo.YMode == Av1PredictionMode.GlobalGlobalMotionVector && + globalMotion.Type != Av1GlobalMotionType.Translation && + this.FrameHeader.GetGlobalMotionParameters()[(int)secondaryReferenceFrame - 1].Type != Av1GlobalMotionType.Translation)); + + if (modeInfo.SkipMode || modeInfo.MotionMode == Av1MotionMode.Warped || usesNonTranslationalGlobalMotion) + { + // Blocks that do not use separable interpolation carry no filter symbols. A switchable frame falls + // back to the regular family so every stored mode record contains an actual predictor selection. + interpolationFilters.Fill( + frameInterpolationFilter == Av1InterpolationFilter.Switchable + ? Av1InterpolationFilter.Regular + : frameInterpolationFilter); + } + else if (frameInterpolationFilter != Av1InterpolationFilter.Switchable) + { + interpolationFilters.Fill(frameInterpolationFilter); + } + else + { + // Filter storage is vertical then horizontal. AV1 transmits in the same order and reuses the vertical + // choice for both axes when the sequence disables independent dual-filter selection. + int verticalContext = Av1SymbolContextHelper.GetSwitchableInterpolationContext( + modeInfo, + partitionInfo.AboveModeInfo, + partitionInfo.LeftModeInfo, + direction: 0); + + interpolationFilters[0] = reader.ReadSwitchableInterpolationFilter(verticalContext); + if (this.SequenceHeader.EnableDualFilter) + { + int horizontalContext = Av1SymbolContextHelper.GetSwitchableInterpolationContext( + modeInfo, + partitionInfo.AboveModeInfo, + partitionInfo.LeftModeInfo, + direction: 1); + + interpolationFilters[1] = reader.ReadSwitchableInterpolationFilter(horizontalContext); + } + else + { + interpolationFilters[1] = interpolationFilters[0]; + } + } + + return; + } + + modeInfo.ReferenceFrames[0] = Av1ReferenceFrameType.Intra; + modeInfo.ReferenceFrames[1] = Av1ReferenceFrameType.None; + modeInfo.SetPaletteSizes(0, 0); + this.ReadConventionalIntraMode(ref reader, ref partitionInfo, reader.ReadInterFrameYMode(modeInfo.BlockSize)); + } + + /// + /// Reads all intra-frame mode syntax for a coding block in bitstream order. + /// + /// The tile symbol decoder. + /// The current coding block and its neighbors. + /// The active tile boundaries. + /// Implements AV1 section 5.11.7. + private void ReadIntraFrameModeInfo(ref Av1SymbolDecoder reader, ref Av1PartitionInfo partitionInfo, Av1TileInfo tileInfo) + { + if (this.FrameHeader.SegmentationParameters.SegmentIdPrecedesSkip) + { + this.IntraSegmentId(ref reader, ref partitionInfo); + } + + partitionInfo.ModeInfo.Skip = this.ReadSkip(ref reader, ref partitionInfo); + if (!this.FrameHeader.SegmentationParameters.SegmentIdPrecedesSkip) + { + this.IntraSegmentId(ref reader, ref partitionInfo); + } + + this.ReadCdef(ref reader, ref partitionInfo); + + if (this.FrameHeader.DeltaQParameters.IsPresent) + { + this.ReadDeltaQuantizerIndex(ref reader, ref partitionInfo); + this.ReadDeltaLoopFilter(ref reader, ref partitionInfo); + } + + // Independently decodable still-image blocks reference only the current intra frame. + partitionInfo.ReferenceFrames[0] = Av1ReferenceFrameType.Intra; + partitionInfo.ReferenceFrames[1] = Av1ReferenceFrameType.None; + partitionInfo.ModeInfo.SetPaletteSizes(0, 0); + bool useIntraBlockCopy = false; + if (this.AllowIntraBlockCopy()) + { + useIntraBlockCopy = reader.ReadUseIntraBlockCopy(); + } + + if (useIntraBlockCopy) + { + partitionInfo.ModeInfo.UseIntraBlockCopy = true; + partitionInfo.ModeInfo.YMode = Av1PredictionMode.DC; + partitionInfo.ModeInfo.UvMode = Av1ChromaPredictionMode.DC; + + Av1MotionVector reference = Av1IntraBlockCopy.FindReference( + ref partitionInfo, + tileInfo, + this.SequenceHeader.SuperblockModeInfoSize, + this.displacementVectorCandidates, + this.displacementVectorWeights); + + Av1MotionVector displacement = reader.ReadDisplacementVector(reference); + + if (!Av1IntraBlockCopy.IsValid(displacement, ref partitionInfo, tileInfo, this.SequenceHeader)) + { + throw new InvalidImageContentException( + $"Invalid AV1 intra-block-copy displacement vector at ({partitionInfo.ColumnIndex}, {partitionInfo.RowIndex}) " + + $"for {partitionInfo.ModeInfo.BlockSize}: reference ({reference.Row}, {reference.Column}), " + + $"decoded ({displacement.Row}, {displacement.Column})."); + } + + partitionInfo.ModeInfo.DisplacementVector = displacement; + } + else + { + Av1PredictionMode yMode = reader.ReadYMode(partitionInfo.AboveModeInfo, partitionInfo.LeftModeInfo); + this.ReadConventionalIntraMode(ref reader, ref partitionInfo, yMode); + } + } + + /// + /// Reads conventional luma and chroma intra-prediction details after the frame branch selects the luma mode CDF. + /// + /// The tile symbol decoder. + /// The current coding block. + /// The luma prediction mode selected by the frame-appropriate distribution. + private void ReadConventionalIntraMode( + ref Av1SymbolDecoder reader, + ref Av1PartitionInfo partitionInfo, + Av1PredictionMode yMode) + { + ref Av1BlockModeInfo modeInfo = ref partitionInfo.ModeInfo; + modeInfo.YMode = yMode; + modeInfo.SetAngleDelta(Av1PlaneType.Y, IntraAngleInfo(ref reader, yMode, modeInfo.BlockSize)); + + if (partitionInfo.IsChroma && !this.SequenceHeader.ColorConfig.IsMonochrome) + { + modeInfo.UvMode = reader.ReadIntraModeUv(yMode, this.IsChromaForLumaAllowed(ref partitionInfo)); + + if (modeInfo.UvMode == Av1ChromaPredictionMode.ChromaFromLuma) + { + ReadChromaFromLumaAlphas(ref reader, ref modeInfo); + } + + modeInfo.SetAngleDelta( + Av1PlaneType.Uv, + IntraAngleInfo(ref reader, modeInfo.UvMode.ToLumaMode(), modeInfo.BlockSize)); + } + else + { + modeInfo.UvMode = Av1ChromaPredictionMode.DC; + } + + if (modeInfo.BlockSize >= Av1BlockSize.Block8x8 && + modeInfo.BlockSize.GetWidth() <= 64 && + modeInfo.BlockSize.GetHeight() <= 64 && + this.FrameHeader.AllowScreenContentTools) + { + this.PaletteModeInfo(ref reader, ref partitionInfo); + } + + this.FilterIntraModeInfo(ref reader, ref partitionInfo); + } + + /// + /// Determines whether the frame header permits intra block copy for an intra frame. + /// + /// + /// when the frame and sequence enable intra block copy; otherwise, . + /// + private bool AllowIntraBlockCopy() + => (this.FrameHeader.FrameType is ObuFrameType.KeyFrame or ObuFrameType.IntraOnlyFrame) && + (this.SequenceHeader.ForceScreenContentTools > 0) && + this.FrameHeader.AllowIntraBlockCopy; + + /// + /// Determines whether chroma-from-luma prediction is available for a coding block. + /// + /// The current coding block. + /// + /// when the lossless transform or block dimensions permit chroma-from-luma prediction; + /// otherwise, . + /// + private bool IsChromaForLumaAllowed(ref Av1PartitionInfo partitionInfo) + { + if (this.FrameHeader.LosslessArray[partitionInfo.ModeInfo.SegmentId]) + { + // Lossless mode fixes transforms at 4x4, so CfL is available only when the subsampled + // plane block is itself 4x4 and therefore has no smaller transform partition. + bool subX = this.SequenceHeader.ColorConfig.SubSamplingX; + bool subY = this.SequenceHeader.ColorConfig.SubSamplingY; + Av1BlockSize planeBlockSize = partitionInfo.ModeInfo.BlockSize.GetSubsampled(subX, subY); + return planeBlockSize == Av1BlockSize.Block4x4; + } + + // Outside lossless mode, AV1 limits CfL to luma blocks no larger than 32x32. + return partitionInfo.ModeInfo.BlockSize.GetWidth() <= 32 && partitionInfo.ModeInfo.BlockSize.GetHeight() <= 32; + } + + /// + /// Reads filter-intra selection for an eligible DC-predicted luma block. + /// + /// The tile symbol decoder. + /// The current coding block. + private void FilterIntraModeInfo(ref Av1SymbolDecoder reader, ref Av1PartitionInfo partitionInfo) + { + partitionInfo.ModeInfo.UseFilterIntra = false; + if (this.SequenceHeader.EnableFilterIntra && + partitionInfo.ModeInfo.YMode == Av1PredictionMode.DC && + partitionInfo.ModeInfo.GetPaletteSize(Av1PlaneType.Y) == 0 && + Math.Max(partitionInfo.ModeInfo.BlockSize.GetWidth(), partitionInfo.ModeInfo.BlockSize.GetHeight()) <= 32) + { + Av1FilterIntraMode filterIntraMode = reader.ReadFilterUltraMode(partitionInfo.ModeInfo.BlockSize); + if (filterIntraMode != Av1FilterIntraMode.AllFilterIntraModes) + { + partitionInfo.ModeInfo.UseFilterIntra = true; + partitionInfo.ModeInfo.FilterIntraMode = filterIntraMode; + } + } + } + + /// + /// Reads palette size and color syntax for an eligible screen-content block. + /// + /// The tile symbol decoder. + /// The current coding block. + /// Implements AV1 section 5.11.46. + private void PaletteModeInfo(ref Av1SymbolDecoder reader, ref Av1PartitionInfo partitionInfo) + { + ref Av1BlockModeInfo modeInfo = ref partitionInfo.ModeInfo; + Av1BlockSize blockSize = modeInfo.BlockSize; + + // The palette block-size context is the base-two block-area difference from an 8-by-8 block. + int blockSizeContext = Av1Math.Log2(blockSize.GetWidth() * blockSize.GetHeight()) - 6; + int yPaletteSize = 0; + int uvPaletteSize = 0; + int bitDepth = this.SequenceHeader.ColorConfig.BitDepth.GetBitCount(); + if (modeInfo.YMode == Av1PredictionMode.DC) + { + int neighborContext = 0; + if (partitionInfo.AboveModeInfo is not null && partitionInfo.AboveModeInfo.Value.GetPaletteSize(Av1PlaneType.Y) != 0) + { + neighborContext++; + } + + if (partitionInfo.LeftModeInfo is not null && partitionInfo.LeftModeInfo.Value.GetPaletteSize(Av1PlaneType.Y) != 0) + { + neighborContext++; + } + + if (reader.ReadPaletteYMode(blockSizeContext, neighborContext)) + { + yPaletteSize = reader.ReadPaletteSize(blockSizeContext, Av1PlaneType.Y); + Span yColors = stackalloc ushort[Av1Constants.PaletteMaxSize]; + ReadPaletteColorsY(ref reader, ref partitionInfo, yPaletteSize, bitDepth, yColors); + modeInfo.SetPaletteColors(Av1Plane.Y, yColors[..yPaletteSize]); + } + } + + if (this.SequenceHeader.ColorConfig.PlaneCount > 1 && + modeInfo.UvMode == Av1ChromaPredictionMode.DC && + partitionInfo.IsChroma && + reader.ReadPaletteUvMode(yPaletteSize != 0)) + { + uvPaletteSize = reader.ReadPaletteSize(blockSizeContext, Av1PlaneType.Uv); + Span uColors = stackalloc ushort[Av1Constants.PaletteMaxSize]; + Span vColors = stackalloc ushort[Av1Constants.PaletteMaxSize]; + ReadPaletteColorsUv(ref reader, ref partitionInfo, uvPaletteSize, bitDepth, uColors, vColors); + modeInfo.SetPaletteColors(Av1Plane.U, uColors[..uvPaletteSize]); + modeInfo.SetPaletteColors(Av1Plane.V, vColors[..uvPaletteSize]); + } + + modeInfo.SetPaletteSizes(yPaletteSize, uvPaletteSize); + } + + /// + /// Reads the sorted luma palette colors, including selections from neighboring palette caches. + /// + /// The tile symbol decoder. + /// The current coding block and its palette neighbors. + /// The number of luma palette colors. + /// The number of bits in each color sample. + /// The destination palette-color buffer. + private static void ReadPaletteColorsY( + ref Av1SymbolDecoder reader, + ref Av1PartitionInfo partitionInfo, + int paletteSize, + int bitDepth, + scoped Span colors) + { + Span colorCache = stackalloc ushort[Av1Constants.PaletteMaxSize * 2]; + int cacheSize = GetPaletteCache(ref partitionInfo, Av1Plane.Y, colorCache); + reader.ReadPaletteYColors(colorCache[..cacheSize], paletteSize, bitDepth, colors); + } + + /// + /// Reads the U and V palette colors, including neighboring U colors and optional V delta coding. + /// + /// The tile symbol decoder. + /// The current coding block and its palette neighbors. + /// The number of chroma palette colors. + /// The number of bits in each color sample. + /// The destination U palette-color buffer. + /// The destination V palette-color buffer. + private static void ReadPaletteColorsUv( + ref Av1SymbolDecoder reader, + ref Av1PartitionInfo partitionInfo, + int paletteSize, + int bitDepth, + scoped Span uColors, + scoped Span vColors) + { + Span colorCache = stackalloc ushort[Av1Constants.PaletteMaxSize * 2]; + int cacheSize = GetPaletteCache(ref partitionInfo, Av1Plane.U, colorCache); + reader.ReadPaletteUvColors(colorCache[..cacheSize], paletteSize, bitDepth, uColors, vColors); + } + + /// + /// Builds the sorted unique palette cache from the available above and left block palettes. + /// + /// The current coding block and its decoded neighbors. + /// The luma or U plane whose sorted base colors form the cache. + /// The destination cache, which can hold both neighboring palettes. + /// The number of colors written to . + private static int GetPaletteCache(ref Av1PartitionInfo partitionInfo, Av1Plane plane, scoped Span cache) + { + // AV1 deliberately excludes the block above at a 64-by-64 superblock-row boundary. + int minimumSuperblockHeight = Av1BlockSize.Block64x64.Get4x4HighCount(); + Av1BlockModeInfo? aboveModeInfo = partitionInfo.RowIndex % minimumSuperblockHeight == 0 + ? null + : partitionInfo.AboveModeInfo; + Av1BlockModeInfo? leftModeInfo = partitionInfo.LeftModeInfo; + + Av1BlockModeInfo above = aboveModeInfo.GetValueOrDefault(); + Av1BlockModeInfo left = leftModeInfo.GetValueOrDefault(); + int abovePaletteSize = aboveModeInfo is null ? 0 : above.GetPaletteSize(plane); + int leftPaletteSize = leftModeInfo is null ? 0 : left.GetPaletteSize(plane); + ReadOnlySpan aboveColors = aboveModeInfo is null ? [] : above.GetPaletteColors(plane); + ReadOnlySpan leftColors = leftModeInfo is null ? [] : left.GetPaletteColors(plane); + return Av1PaletteCache.Merge( + aboveColors[..abovePaletteSize], + leftColors[..leftPaletteSize], + cache); + } + + /// + /// Computes the padded plane dimensions and the portion that lies inside the coded image. + /// + /// The current coding block and frame-edge distances. + /// The luma or shared chroma plane class. + /// The sequence chroma-subsampling configuration. + /// The padded plane-block width in samples. + /// The padded plane-block height in samples. + /// The number of plane-block rows inside the coded image. + /// The number of plane-block columns inside the coded image. + private static void GetPaletteMapDimensions( + ref Av1PartitionInfo partitionInfo, + Av1PlaneType planeType, + ObuColorConfig colorConfig, + out int planeWidth, + out int planeHeight, + out int rows, + out int columns) + { + int subX = planeType == Av1PlaneType.Uv && colorConfig.SubSamplingX ? 1 : 0; + int subY = planeType == Av1PlaneType.Uv && colorConfig.SubSamplingY ? 1 : 0; + int blockWidth = partitionInfo.ModeInfo.BlockSize.GetWidth(); + int blockHeight = partitionInfo.ModeInfo.BlockSize.GetHeight(); + int columnsInsideImage = partitionInfo.ModeBlockToRightEdge >= 0 + ? blockWidth + : blockWidth + (partitionInfo.ModeBlockToRightEdge >> 3); + int rowsInsideImage = partitionInfo.ModeBlockToBottomEdge >= 0 + ? blockHeight + : blockHeight + (partitionInfo.ModeBlockToBottomEdge >> 3); + + planeWidth = blockWidth >> subX; + planeHeight = blockHeight >> subY; + columns = columnsInsideImage >> subX; + rows = rowsInsideImage >> subY; + } + + /// + /// Decodes a padded palette color-index map in AV1 diagonal wavefront order. + /// + /// The tile symbol decoder. + /// The number of palette colors. + /// The luma or shared chroma plane class. + /// The padded plane-block width. + /// The padded plane-block height. + /// The number of rows inside the coded image. + /// The number of columns inside the coded image. + /// The row-addressable destination map. + private static void DecodePaletteColorMap( + ref Av1SymbolDecoder reader, + int paletteSize, + Av1PlaneType planeType, + int planeWidth, + int planeHeight, + int rows, + int columns, + Buffer2DRegion colorIndexMap) + { + reader.ReadPaletteColorMap(paletteSize, planeType, rows, columns, colorIndexMap); + + if (columns < planeWidth) + { + // Blocks clipped by the right image edge repeat their final coded column into the padded block area. + for (int row = 0; row < rows; row++) + { + Span colorIndexRow = colorIndexMap.DangerousGetRowSpan(row); + colorIndexRow.Slice(columns, planeWidth - columns) + .Fill(colorIndexRow[columns - 1]); + } + } + + // Blocks clipped by the bottom image edge repeat their final coded row for later transform reconstruction. + ReadOnlySpan finalRow = colorIndexMap.DangerousGetRowSpan(rows - 1); + for (int row = rows; row < planeHeight; row++) + { + finalRow.CopyTo(colorIndexMap.DangerousGetRowSpan(row)); + } + } + + /// + /// Reads the joint signs and nonzero alpha magnitudes for chroma-from-luma prediction. + /// + /// The tile symbol decoder. + /// The block mode information to populate. + /// Implements AV1 section 5.11.45. + private static void ReadChromaFromLumaAlphas(ref Av1SymbolDecoder reader, ref Av1BlockModeInfo modeInfo) + { + int jointSignPlus1 = reader.ReadChromFromLumaSign() + 1; + int index = 0; + if (jointSignPlus1 >= 3) + { + index = reader.ReadChromaFromLumaAlphaU(jointSignPlus1) << Av1Constants.ChromaFromLumaAlphabetSizeLog2; + } + + if (jointSignPlus1 % 3 != 0) + { + index += reader.ReadChromaFromLumaAlphaV(jointSignPlus1); + } + + modeInfo.ChromaFromLumaAlphaSign = jointSignPlus1 - 1; + modeInfo.ChromaFromLumaAlphaIndex = index; + } + + /// + /// Reads a directional intra-prediction angle adjustment when the block and mode permit one. + /// + /// The tile symbol decoder. + /// The selected luma or chroma prediction mode. + /// The block size. + /// The signed angle adjustment. + /// Implements AV1 sections 5.11.42 and 5.11.43. + private static int IntraAngleInfo(ref Av1SymbolDecoder reader, Av1PredictionMode mode, Av1BlockSize blockSize) + { + int angleDelta = 0; + if (blockSize >= Av1BlockSize.Block8x8 && IsDirectionalMode(mode)) + { + int symbol = reader.ReadAngleDelta(mode); + angleDelta = symbol - Av1Constants.MaxAngleDelta; + } + + return angleDelta; + } + + /// + /// Determines whether a prediction mode belongs to the AV1 directional-mode range. + /// + /// The prediction mode. + /// for a directional mode; otherwise, . + private static bool IsDirectionalMode(Av1PredictionMode mode) + => mode is >= Av1PredictionMode.Vertical and <= Av1PredictionMode.Directional67Degrees; + + /// + /// Reads or inherits a segment identifier and writes it over every 4x4 position covered by the block. + /// + /// The tile symbol decoder. + /// The current coding block. + /// Implements AV1 section 5.11.8. + private void IntraSegmentId(ref Av1SymbolDecoder reader, ref Av1PartitionInfo partitionInfo) + { + if (!this.FrameHeader.SegmentationParameters.Enabled) + { + // Disabled segmentation assigns the default ID without allocating or populating a retained map. + return; + } + + this.ReadSegmentId(ref reader, ref partitionInfo); + Point modeInfoPosition = new(partitionInfo.ColumnIndex, partitionInfo.RowIndex); + this.FrameInfo.SetSegmentId(partitionInfo.ModeInfo.BlockSize, modeInfoPosition, partitionInfo.ModeInfo.SegmentId); + } + + /// + /// Reads or inherits the segment identifier for one inter-frame block and updates its 4x4 map coverage. + /// + /// The tile symbol decoder. + /// The current coding block and its available neighbors. + /// Whether this invocation precedes the block's residual-skip decision. + /// + /// Implements read_inter_segment_id from AV1 section 5.11.8. + /// + public void ReadInterSegmentId(ref Av1SymbolDecoder reader, ref Av1PartitionInfo partitionInfo, bool beforeSkip) + { + ObuSegmentationParameters segmentationParameters = this.FrameHeader.SegmentationParameters; + ref Av1BlockModeInfo modeInfo = ref partitionInfo.ModeInfo; + + if (!segmentationParameters.Enabled) + { + // Disabled segmentation has no allocated map and normatively assigns segment zero. + modeInfo.SegmentId = 0; + return; + } + + Point modeInfoPosition = new(partitionInfo.ColumnIndex, partitionInfo.RowIndex); + + if (segmentationParameters.SegmentationUpdateMap == 0) + { + // The frame map was inherited as one contiguous copy during reader construction. Resolve the same clipped + // minimum that the reference decoder obtains from last_frame_seg_map so block state and the already copied map agree. + modeInfo.SegmentId = this.FrameInfo.GetPredictedSegmentId(this.primaryReferenceState, modeInfo.BlockSize, modeInfoPosition); + return; + } + + if (beforeSkip) + { + if (!segmentationParameters.SegmentIdPrecedesSkip) + { + // The caller invokes this once before skip for every inter block; post-skip segment syntax owns this case. + return; + } + } + else if (modeInfo.Skip) + { + if (segmentationParameters.SegmentationTemporalUpdate == 1) + { + // Skipped blocks use the spatial segment predictor and signal no temporal-prediction bit. + modeInfo.SegmentIdPredicted = false; + } + + this.ReadSegmentId(ref reader, ref partitionInfo); + this.FrameInfo.SetSegmentId(modeInfo.BlockSize, modeInfoPosition, modeInfo.SegmentId); + return; + } + + if (segmentationParameters.SegmentationTemporalUpdate == 1) + { + // The binary context counts only neighboring blocks that themselves selected the retained map. Segment + // values do not participate in this decision. + int context = Av1SymbolContextHelper.GetSegmentIdPredictedContext(partitionInfo.AboveModeInfo, partitionInfo.LeftModeInfo); + modeInfo.SegmentIdPredicted = reader.ReadSegmentIdPredicted(context); + if (modeInfo.SegmentIdPredicted) + { + modeInfo.SegmentId = this.FrameInfo.GetPredictedSegmentId(this.primaryReferenceState, modeInfo.BlockSize, modeInfoPosition); + } + else + { + this.ReadSegmentId(ref reader, ref partitionInfo); + } + } + else + { + this.ReadSegmentId(ref reader, ref partitionInfo); + } + + this.FrameInfo.SetSegmentId(modeInfo.BlockSize, modeInfoPosition, modeInfo.SegmentId); + } + + /// + /// Predicts and, when required, decodes the spatially coded segment identifier for a block. + /// + /// The tile symbol decoder. + /// The current coding block and its available neighbors. + /// Implements AV1 section 5.11.9. + private void ReadSegmentId(ref Av1SymbolDecoder reader, ref Av1PartitionInfo partitionInfo) + { + int predictor; + int prevUL = -1; + int prevU = -1; + int prevL = -1; + int columnIndex = partitionInfo.ColumnIndex; + int rowIndex = partitionInfo.RowIndex; + if (partitionInfo.AvailableAbove && partitionInfo.AvailableLeft) + { + prevUL = this.FrameInfo.GetSegmentId(rowIndex - 1, columnIndex - 1); + } + + if (partitionInfo.AvailableAbove) + { + prevU = this.FrameInfo.GetSegmentId(rowIndex - 1, columnIndex); + } + + if (partitionInfo.AvailableLeft) + { + prevL = this.FrameInfo.GetSegmentId(rowIndex, columnIndex - 1); + } + + if (prevU == -1) + { + predictor = prevL == -1 ? 0 : prevL; + } + else if (prevL == -1) + { + predictor = prevU; + } + else + { + predictor = prevU == prevUL ? prevU : prevL; + } + + if (partitionInfo.ModeInfo.Skip) + { + partitionInfo.ModeInfo.SegmentId = predictor; + } + else + { + // Any unavailable neighbor selects the edge context; otherwise, agreement among two + // or three neighbors increases the specificity of the segment-ID distribution. + int ctx = prevUL < 0 ? 0 + : prevUL == prevU && prevUL == prevL ? 2 + : prevUL == prevU || prevUL == prevL || prevU == prevL ? 1 : 0; + int lastActiveSegmentId = this.FrameHeader.SegmentationParameters.LastActiveSegmentId; + int segmentId = Av1SymbolContextHelper.NegativeDeinterleave(reader.ReadSegmentId(ctx), predictor, lastActiveSegmentId + 1); + if (segmentId is < 0 || segmentId > lastActiveSegmentId) + { + // The coded alphabet always contains eight symbols, even when the frame activates fewer segments. + // Validate the reconstructed ID at the same corruption boundary as the reference decoder's read_segment_id. + throw new InvalidImageContentException("The decoded AV1 segment identifier exceeds the active segment range."); + } + + partitionInfo.ModeInfo.SegmentId = segmentId; + } + } + + /// + /// Reads the constrained directional enhancement filter strength for the block's 64x64 filter unit. + /// + /// The tile symbol decoder. + /// The current coding block. + private void ReadCdef(ref Av1SymbolDecoder reader, ref Av1PartitionInfo partitionInfo) + { + if (partitionInfo.ModeInfo.Skip || this.FrameHeader.CodedLossless || !this.SequenceHeader.EnableCdef || this.FrameHeader.AllowIntraBlockCopy) + { + return; + } + + int cdefSize4 = Av1BlockSize.Block64x64.Get4x4WideCount(); + int superblockMask = this.SequenceHeader.SuperblockModeInfoSize - 1; + int rowInSuperblock = partitionInfo.RowIndex & superblockMask; + int columnInSuperblock = partitionInfo.ColumnIndex & superblockMask; + int unitRow = rowInSuperblock / cdefSize4; + int unitColumn = columnInSuperblock / cdefSize4; + int index = this.SequenceHeader.SuperblockSize == Av1BlockSize.Block128x128 ? unitColumn + (unitRow << 1) : 0; + Span cdefStrength = partitionInfo.SuperblockInfo.CdefStrength; + if (cdefStrength[index] == -1) + { + int cdefStrengthIndex = reader.ReadCdfStrength(this.FrameHeader.CdefParameters.BitCount); + int blockWidth4 = partitionInfo.ModeInfo.BlockSize.Get4x4WideCount(); + int blockHeight4 = partitionInfo.ModeInfo.BlockSize.Get4x4HighCount(); + int lastUnitRow = (rowInSuperblock + blockHeight4 - 1) / cdefSize4; + int lastUnitColumn = (columnInSuperblock + blockWidth4 - 1) / cdefSize4; + + // A coding block can cover the top-left cell of more than one 64x64 CDEF unit. the reference decoder + // stores the index on shared mode information, so the frame-owned unit map must mirror it. + for (int coveredUnitRow = unitRow; coveredUnitRow <= lastUnitRow; coveredUnitRow++) + { + for (int coveredUnitColumn = unitColumn; coveredUnitColumn <= lastUnitColumn; coveredUnitColumn++) + { + int coveredIndex = this.SequenceHeader.SuperblockSize == Av1BlockSize.Block128x128 + ? coveredUnitColumn + (coveredUnitRow << 1) + : 0; + + cdefStrength[coveredIndex] = cdefStrengthIndex; + } + } + } + } + + /// + /// Reads and accumulates the loop-filter delta values carried by a coding block. + /// + /// The tile symbol decoder. + /// The current coding block and superblock delta storage. + private void ReadDeltaLoopFilter(ref Av1SymbolDecoder reader, ref Av1PartitionInfo partitionInfo) + { + if (!this.FrameHeader.DeltaLoopFilterParameters.IsPresent || partitionInfo.ModeInfo.PositionInSuperblock != Point.Empty) + { + return; + } + + Av1BlockSize superBlockSize = this.SequenceHeader.Use128x128Superblock ? Av1BlockSize.Block128x128 : Av1BlockSize.Block64x64; + if (partitionInfo.ModeInfo.BlockSize != superBlockSize || !partitionInfo.ModeInfo.Skip) + { + int frameLoopFilterCount = 1; + if (this.FrameHeader.DeltaLoopFilterParameters.IsMulti) + { + frameLoopFilterCount = this.SequenceHeader.ColorConfig.PlaneCount > 1 ? Av1Constants.FrameLoopFilterCount : Av1Constants.FrameLoopFilterCount - 2; + } + + for (int i = 0; i < frameLoopFilterCount; i++) + { + int reducedDeltaLoopFilterLevel = reader.ReadDeltaLoopFilter(this.FrameHeader.DeltaLoopFilterParameters.IsMulti, i); + int deltaLoopFilterResolution = this.FrameHeader.DeltaLoopFilterParameters.Resolution; + this.currentDeltaLoopFilter[i] = Av1Math.Clip3( + -Av1Constants.MaxLoopFilter, + Av1Constants.MaxLoopFilter, + this.currentDeltaLoopFilter[i] + (reducedDeltaLoopFilterLevel * deltaLoopFilterResolution)); + } + } + + // Delta-LF values are predicted across superblocks within a tile, but every block in one superblock observes + // the same resulting values. Snapshot the predictors so later filtering does not depend on parse order. + this.currentDeltaLoopFilter[..Av1Constants.FrameLoopFilterCount].CopyTo(partitionInfo.SuperblockInfo.SuperblockDeltaLoopFilter); + } + + /// + /// Reads or infers the residual-skip flag for a coding block. + /// + /// The tile symbol decoder. + /// The current coding block and its available neighbors. + /// when the block omits residual coefficients; otherwise, . + private bool ReadSkip(ref Av1SymbolDecoder reader, ref Av1PartitionInfo partitionInfo) + { + int segmentId = partitionInfo.ModeInfo.SegmentId; + if (this.FrameHeader.SegmentationParameters.SegmentIdPrecedesSkip && + this.FrameHeader.SegmentationParameters.IsFeatureActive(segmentId, ObuSegmentationLevelFeature.Skip)) + { + return true; + } + else + { + int aboveSkip = partitionInfo.AboveModeInfo is not null && partitionInfo.AboveModeInfo.Value.Skip ? 1 : 0; + int leftSkip = partitionInfo.LeftModeInfo is not null && partitionInfo.LeftModeInfo.Value.Skip ? 1 : 0; + return reader.ReadSkip(aboveSkip + leftSkip); + } + } + + /// + /// Reads compound skip-mode selection when the frame, segment, and block geometry permit it. + /// + /// The tile symbol decoder. + /// The current coding block and its available neighbors. + /// when the block selects the frame's derived skip-mode reference pair. + private bool ReadSkipMode(ref Av1SymbolDecoder reader, ref Av1PartitionInfo partitionInfo) + { + ref Av1BlockModeInfo modeInfo = ref partitionInfo.ModeInfo; + ObuSegmentationParameters segmentationParameters = this.FrameHeader.SegmentationParameters; + int segmentId = modeInfo.SegmentId; + + if (!this.FrameHeader.SkipModeParameters.SkipModeFlag || + segmentationParameters.IsFeatureActive(segmentId, ObuSegmentationLevelFeature.Skip) || + Math.Min(modeInfo.BlockSize.GetWidth(), modeInfo.BlockSize.GetHeight()) < 8 || + segmentationParameters.IsFeatureActive(segmentId, ObuSegmentationLevelFeature.ReferenceFrame) || + segmentationParameters.IsFeatureActive(segmentId, ObuSegmentationLevelFeature.GlobalMotionVector)) + { + // Segment reference and global-motion features force single-reference prediction, while skip mode always + // selects the derived compound pair. The syntax therefore omits the skip-mode symbol in either case. + return false; + } + + int aboveSkipMode = partitionInfo.AboveModeInfo is not null && partitionInfo.AboveModeInfo.Value.SkipMode ? 1 : 0; + int leftSkipMode = partitionInfo.LeftModeInfo is not null && partitionInfo.LeftModeInfo.Value.SkipMode ? 1 : 0; + return reader.ReadSkipMode(aboveSkipMode + leftSkipMode); + } + + /// + /// Reads or infers whether an inter-frame coding block uses inter prediction. + /// + /// The tile symbol decoder. + /// The current coding block and its available neighbors. + /// for an inter-coded block; otherwise, . + private bool ReadIsInter(ref Av1SymbolDecoder reader, ref Av1PartitionInfo partitionInfo) + { + ObuSegmentationParameters segmentationParameters = this.FrameHeader.SegmentationParameters; + int segmentId = partitionInfo.ModeInfo.SegmentId; + + if (segmentationParameters.IsFeatureActive(segmentId, ObuSegmentationLevelFeature.ReferenceFrame)) + { + // Reference feature values use the same numeric labels as Av1ReferenceFrameType. INTRA_FRAME is zero; + // every canonical inter reference begins at LAST_FRAME and therefore has a positive value. + int referenceFrame = segmentationParameters.GetFeatureData(segmentId, (int)ObuSegmentationLevelFeature.ReferenceFrame); + return referenceFrame >= (int)Av1ReferenceFrameType.Last; + } + + if (segmentationParameters.IsFeatureActive(segmentId, ObuSegmentationLevelFeature.GlobalMotionVector)) + { + return true; + } + + int context = Av1SymbolContextHelper.GetIntraInterContext(partitionInfo.AboveModeInfo, partitionInfo.LeftModeInfo); + return reader.ReadIsInter(context); + } + + /// + /// Reads or infers the retained reference-frame labels selected by an inter-coded block. + /// + /// The tile symbol decoder. + /// The current coding block and its available neighbors. + private void ReadReferenceFrames(ref Av1SymbolDecoder reader, ref Av1PartitionInfo partitionInfo) + { + ref Av1BlockModeInfo modeInfo = ref partitionInfo.ModeInfo; + Span references = modeInfo.ReferenceFrames; + if (modeInfo.SkipMode) + { + ObuSkipModeParameters skipModeParameters = this.FrameHeader.SkipModeParameters; + references[0] = skipModeParameters.FirstReferenceFrame; + references[1] = skipModeParameters.SecondReferenceFrame; + return; + } + + ObuSegmentationParameters segmentationParameters = this.FrameHeader.SegmentationParameters; + int segmentId = modeInfo.SegmentId; + if (segmentationParameters.IsFeatureActive(segmentId, ObuSegmentationLevelFeature.ReferenceFrame)) + { + references[0] = (Av1ReferenceFrameType)segmentationParameters.GetFeatureData( + segmentId, + (int)ObuSegmentationLevelFeature.ReferenceFrame); + + references[1] = Av1ReferenceFrameType.None; + return; + } + + if (segmentationParameters.IsFeatureActive(segmentId, ObuSegmentationLevelFeature.Skip) || + segmentationParameters.IsFeatureActive(segmentId, ObuSegmentationLevelFeature.GlobalMotionVector)) + { + references[0] = Av1ReferenceFrameType.Last; + references[1] = Av1ReferenceFrameType.None; + return; + } + + Span referenceCounts = this.neighborReferenceCounts; + Av1SymbolContextHelper.CollectNeighborReferenceCounts( + partitionInfo.AboveModeInfo, + partitionInfo.LeftModeInfo, + referenceCounts); + + bool compoundReferenceAllowed = Math.Min(modeInfo.BlockSize.GetWidth(), modeInfo.BlockSize.GetHeight()) >= 8; + if (compoundReferenceAllowed && this.FrameHeader.ReferenceMode == ObuReferenceMode.ReferenceModeSelect) + { + int context = Av1SymbolContextHelper.GetReferenceModeContext( + partitionInfo.AboveModeInfo, + partitionInfo.LeftModeInfo); + + if (reader.ReadIsCompoundReference(context)) + { + int typeContext = Av1SymbolContextHelper.GetCompoundReferenceTypeContext( + partitionInfo.AboveModeInfo, + partitionInfo.LeftModeInfo); + + if (!reader.ReadCompoundReferenceIsBidirectional(typeContext)) + { + int directionContext = Av1SymbolContextHelper.GetUnidirectionalCompoundBackwardContext(referenceCounts); + if (reader.ReadUnidirectionalCompoundReference(directionContext, decision: 0)) + { + references[0] = Av1ReferenceFrameType.Backward; + references[1] = Av1ReferenceFrameType.Alternate; + } + else + { + int unidirectionalForwardGroupContext = Av1SymbolContextHelper.GetUnidirectionalCompoundLast3OrGoldenContext(referenceCounts); + if (!reader.ReadUnidirectionalCompoundReference(unidirectionalForwardGroupContext, decision: 1)) + { + references[0] = Av1ReferenceFrameType.Last; + references[1] = Av1ReferenceFrameType.Last2; + } + else + { + int forwardChoiceContext = Av1SymbolContextHelper.GetUnidirectionalCompoundGoldenContext(referenceCounts); + references[0] = Av1ReferenceFrameType.Last; + references[1] = reader.ReadUnidirectionalCompoundReference(forwardChoiceContext, decision: 2) + ? Av1ReferenceFrameType.Golden + : Av1ReferenceFrameType.Last3; + } + } + + return; + } + + int forwardGroupContext = Av1SymbolContextHelper.GetCompoundForwardLast3OrGoldenContext(referenceCounts); + if (!reader.ReadCompoundForwardReference(forwardGroupContext, decision: 0)) + { + int forwardChoiceContext = Av1SymbolContextHelper.GetCompoundForwardLast2Context(referenceCounts); + references[0] = reader.ReadCompoundForwardReference(forwardChoiceContext, decision: 1) + ? Av1ReferenceFrameType.Last2 + : Av1ReferenceFrameType.Last; + } + else + { + int forwardChoiceContext = Av1SymbolContextHelper.GetCompoundForwardGoldenContext(referenceCounts); + references[0] = reader.ReadCompoundForwardReference(forwardChoiceContext, decision: 2) + ? Av1ReferenceFrameType.Golden + : Av1ReferenceFrameType.Last3; + } + + int backwardGroupContext = Av1SymbolContextHelper.GetCompoundBackwardAlternateContext(referenceCounts); + if (reader.ReadCompoundBackwardReference(backwardGroupContext, decision: 0)) + { + references[1] = Av1ReferenceFrameType.Alternate; + } + else + { + int backwardChoiceContext = Av1SymbolContextHelper.GetCompoundBackwardAlternate2Context(referenceCounts); + references[1] = reader.ReadCompoundBackwardReference(backwardChoiceContext, decision: 1) + ? Av1ReferenceFrameType.Alternate2 + : Av1ReferenceFrameType.Backward; + } + + return; + } + } + + Av1ReferenceFrameType reference; + if (reader.ReadSingleReferenceIsBackward(Av1SymbolContextHelper.GetSingleReferenceBackwardContext(referenceCounts))) + { + if (reader.ReadSingleReferenceIsAlternate(Av1SymbolContextHelper.GetSingleReferenceAlternateContext(referenceCounts))) + { + reference = Av1ReferenceFrameType.Alternate; + } + else + { + reference = reader.ReadSingleReferenceIsAlternate2( + Av1SymbolContextHelper.GetSingleReferenceAlternate2Context(referenceCounts)) + ? Av1ReferenceFrameType.Alternate2 + : Av1ReferenceFrameType.Backward; + } + } + else if (reader.ReadSingleReferenceIsLast3OrGolden( + Av1SymbolContextHelper.GetSingleReferenceLast3OrGoldenContext(referenceCounts))) + { + reference = reader.ReadSingleReferenceIsGolden(Av1SymbolContextHelper.GetSingleReferenceGoldenContext(referenceCounts)) + ? Av1ReferenceFrameType.Golden + : Av1ReferenceFrameType.Last3; + } + else + { + reference = reader.ReadSingleReferenceIsLast2(Av1SymbolContextHelper.GetSingleReferenceLast2Context(referenceCounts)) + ? Av1ReferenceFrameType.Last2 + : Av1ReferenceFrameType.Last; + } + + references[0] = reference; + references[1] = Av1ReferenceFrameType.None; + } + + /// + /// Reads and accumulates a superblock quantizer-index delta when the block carries one. + /// + /// The tile symbol decoder. + /// The current coding block and superblock quantizer storage. + private void ReadDeltaQuantizerIndex(ref Av1SymbolDecoder reader, ref Av1PartitionInfo partitionInfo) + { + if (!this.FrameHeader.DeltaQParameters.IsPresent || partitionInfo.ModeInfo.PositionInSuperblock != Point.Empty) + { + return; + } + + if (partitionInfo.ModeInfo.BlockSize != this.SequenceHeader.SuperblockSize || !partitionInfo.ModeInfo.Skip) + { + int reducedDeltaQuantizerIndex = reader.ReadDeltaQuantizerIndex(); + int deltaQuantizerResolution = this.FrameHeader.DeltaQParameters.Resolution; + this.currentQuantizerIndex = Av1Math.Clip3( + 1, + Av1Constants.MaxQ, + this.currentQuantizerIndex + (reducedDeltaQuantizerIndex * deltaQuantizerResolution)); + } + + partitionInfo.SuperblockInfo.SuperblockQuantizerIndex = this.currentQuantizerIndex; + } + + /// + /// Determines whether a frame-relative mode-information position lies inside the active tile. + /// + /// The active tile boundaries. + /// The frame-relative mode-information row. + /// The frame-relative mode-information column. + /// when the position lies within the active tile; otherwise, . + private static bool IsInside(Av1TileInfo tileInfo, int rowIndex, int columnIndex) => + columnIndex >= tileInfo.ModeInfoColumnStart && + columnIndex < tileInfo.ModeInfoColumnEnd && + rowIndex >= tileInfo.ModeInfoRowStart && + rowIndex < tileInfo.ModeInfoRowEnd; + + /// + /// Derives the partition entropy context from the current split bit of the above and left neighbors. + /// + /// The partition origin in 4x4 mode-information units. + /// The square parent block size. + /// The active tile boundaries. + /// The containing superblock. + /// The partition entropy context. + private int GetPartitionPlaneContext(Point location, Av1BlockSize blockSize, Av1TileInfo tileInfo, Av1SuperblockInfo superblockInfo) + { + // The five stored split bits begin at the 8x8 partition point, so normalize the block-size log to that bit index. + int aboveCtx = this.aboveNeighborContext.AbovePartitionWidth[location.X - tileInfo.ModeInfoColumnStart]; + int leftCtx = this.leftNeighborContext.LeftPartitionHeight[(location.Y - superblockInfo.ModeInfoPosition.Y) & Av1PartitionContext.Mask]; + int blockSizeLog = blockSize.Get4x4WidthLog2() - Av1BlockSize.Block8x8.Get4x4WidthLog2(); + int above = (aboveCtx >> blockSizeLog) & 0x1; + int left = (leftCtx >> blockSizeLog) & 0x1; + DebugGuard.IsTrue(blockSize.Get4x4WidthLog2() == blockSize.Get4x4HeightLog2(), "Blocks should be square."); + DebugGuard.MustBeGreaterThanOrEqualTo(blockSizeLog, 0, nameof(blockSizeLog)); + return ((left << 1) + above) + (blockSizeLog * Av1Constants.PartitionProbabilitySet); + } + + /// + /// Publishes the decoded partition sizes to the above and left neighbor contexts. + /// + /// The parent block origin in 4x4 mode-information units. + /// The active tile boundaries. + /// The containing superblock. + /// The primary size produced by the partition. + /// The parent block size. + /// The decoded partition type. + private void UpdatePartitionContext(Point modeInfoLocation, Av1TileInfo tileLoc, Av1SuperblockInfo superblockInfo, Av1BlockSize subSize, Av1BlockSize blockSize, Av1PartitionType partition) + { + if (blockSize >= Av1BlockSize.Block8x8) + { + int hbs = blockSize.Get4x4WideCount() / 2; + Av1BlockSize blockSize2 = Av1PartitionType.Split.GetBlockSubSize(blockSize); + switch (partition) + { + case Av1PartitionType.Split: + if (blockSize != Av1BlockSize.Block8x8) + { + break; + } + + goto PARTITIONS; + case Av1PartitionType.None: + case Av1PartitionType.Horizontal: + case Av1PartitionType.Vertical: + case Av1PartitionType.Horizontal4: + case Av1PartitionType.Vertical4: + PARTITIONS: + this.aboveNeighborContext.UpdatePartition(modeInfoLocation, tileLoc, subSize, blockSize); + this.leftNeighborContext.UpdatePartition(modeInfoLocation, superblockInfo, subSize, blockSize); + break; + case Av1PartitionType.HorizontalA: + this.aboveNeighborContext.UpdatePartition(modeInfoLocation, tileLoc, blockSize2, subSize); + this.leftNeighborContext.UpdatePartition(modeInfoLocation, superblockInfo, blockSize2, subSize); + Point locHorizontalA = new(modeInfoLocation.X, modeInfoLocation.Y + hbs); + this.aboveNeighborContext.UpdatePartition(locHorizontalA, tileLoc, subSize, subSize); + this.leftNeighborContext.UpdatePartition(locHorizontalA, superblockInfo, subSize, subSize); + break; + case Av1PartitionType.HorizontalB: + this.aboveNeighborContext.UpdatePartition(modeInfoLocation, tileLoc, subSize, subSize); + this.leftNeighborContext.UpdatePartition(modeInfoLocation, superblockInfo, subSize, subSize); + Point locHorizontalB = new(modeInfoLocation.X, modeInfoLocation.Y + hbs); + this.aboveNeighborContext.UpdatePartition(locHorizontalB, tileLoc, blockSize2, subSize); + this.leftNeighborContext.UpdatePartition(locHorizontalB, superblockInfo, blockSize2, subSize); + break; + case Av1PartitionType.VerticalA: + this.aboveNeighborContext.UpdatePartition(modeInfoLocation, tileLoc, blockSize2, subSize); + this.leftNeighborContext.UpdatePartition(modeInfoLocation, superblockInfo, blockSize2, subSize); + Point locVerticalA = new(modeInfoLocation.X + hbs, modeInfoLocation.Y); + this.aboveNeighborContext.UpdatePartition(locVerticalA, tileLoc, subSize, subSize); + this.leftNeighborContext.UpdatePartition(locVerticalA, superblockInfo, subSize, subSize); + break; + case Av1PartitionType.VerticalB: + this.aboveNeighborContext.UpdatePartition(modeInfoLocation, tileLoc, subSize, subSize); + this.leftNeighborContext.UpdatePartition(modeInfoLocation, superblockInfo, subSize, subSize); + Point locVerticalB = new(modeInfoLocation.X + hbs, modeInfoLocation.Y); + this.aboveNeighborContext.UpdatePartition(locVerticalB, tileLoc, blockSize2, subSize); + this.leftNeighborContext.UpdatePartition(locVerticalB, superblockInfo, blockSize2, subSize); + break; + default: + throw new InvalidImageContentException($"Unknown partition type: {partition}"); + } + } + } + + /// + /// Carries the two shared palette maps as one valid constructor state. + /// + internal readonly struct PaletteColorIndexMaps(Buffer2D luma, Buffer2D chroma) + { + /// + /// Gets the shared luma palette map. + /// + public Buffer2D Luma { get; } = luma; + + /// + /// Gets the shared chroma palette map. + /// + public Buffer2D Chroma { get; } = chroma; + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1TileWriter.BlockEncoding.cs b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1TileWriter.BlockEncoding.cs new file mode 100644 index 0000000000..fd44f70736 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1TileWriter.BlockEncoding.cs @@ -0,0 +1,80 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1.Entropy; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +/// +/// Defines the final-block decision contract used by interleaved tile encoding. +/// +internal partial class Av1TileWriter +{ + /// + /// Supplies a final block decision immediately before its symbols are written. + /// + internal interface IBlockEncodingHandler + { + /// + /// Selects the partition used for the current tree node. + /// + /// The live tile symbol encoder. + /// The tile-local macroblock state. + /// The absolute luma-sample origin. + /// The zero-based tile index. + /// The current square partition size. + /// The partition retained before live analysis. + /// The partition to encode. + Av1PartitionType SelectPartition( + Av1SymbolEncoder writer, + Av1MacroBlockD macroBlock, + Point blockOrigin, + ushort tileIndex, + Av1BlockSize blockSize, + Av1PartitionType preparedPartition); + + /// + /// Encodes one final block against the current reconstructed neighbors and live tile probabilities. + /// + /// The live tile symbol encoder. + /// The current block's mapped neighbor state. + /// The absolute luma-sample origin. + /// The zero-based tile index. + /// The mode information to publish. + /// The encoder block state to publish. + /// The current block's palette sizes and colors. + void EncodeBlock( + Av1SymbolEncoder writer, + Av1MacroBlockD macroBlock, + Point blockOrigin, + ushort tileIndex, + ref Av1MacroBlockModeInfo modeInfo, + ref Av1EncoderBlockStruct block, + ref Av1EncoderPaletteInfo paletteInfo); + } + + private readonly struct PrecomputedBlockEncodingHandler : IBlockEncodingHandler + { + /// + public Av1PartitionType SelectPartition( + Av1SymbolEncoder writer, + Av1MacroBlockD macroBlock, + Point blockOrigin, + ushort tileIndex, + Av1BlockSize blockSize, + Av1PartitionType preparedPartition) + => preparedPartition; + + /// + public void EncodeBlock( + Av1SymbolEncoder writer, + Av1MacroBlockD macroBlock, + Point blockOrigin, + ushort tileIndex, + ref Av1MacroBlockModeInfo modeInfo, + ref Av1EncoderBlockStruct block, + ref Av1EncoderPaletteInfo paletteInfo) + { + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1TileWriter.cs b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1TileWriter.cs new file mode 100644 index 0000000000..235d0f27d9 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1TileWriter.cs @@ -0,0 +1,2661 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1.Entropy; +using SixLabors.ImageSharp.Formats.Heif.Av1.Motion; +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; +using SixLabors.ImageSharp.Memory; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +/// +/// Writes the partition, mode, transform, and coefficient syntax for one AV1 tile. +/// +internal partial class Av1TileWriter +{ + /// + /// Maps each AV1 block size to the five-bit partition contexts written to its bottom and right edges. + /// + /// + /// Each set bit represents a split level from 128x128 through 8x8. For example, 11111 + /// records every split level, while 10000 records only the 128x128 split. + /// + private static readonly Av1PartitionContext[] PartitionContextLookup = + [ + new(31, 31), // 4X4 - {0b11111, 0b11111} + new(31, 30), // 4X8 - {0b11111, 0b11110} + new(30, 31), // 8X4 - {0b11110, 0b11111} + new(30, 30), // 8X8 - {0b11110, 0b11110} + new(30, 28), // 8X16 - {0b11110, 0b11100} + new(28, 30), // 16X8 - {0b11100, 0b11110} + new(28, 28), // 16X16 - {0b11100, 0b11100} + new(28, 24), // 16X32 - {0b11100, 0b11000} + new(24, 28), // 32X16 - {0b11000, 0b11100} + new(24, 24), // 32X32 - {0b11000, 0b11000} + new(24, 16), // 32X64 - {0b11000, 0b10000} + new(16, 24), // 64X32 - {0b10000, 0b11000} + new(16, 16), // 64X64 - {0b10000, 0b10000} + new(16, 0), // 64X128- {0b10000, 0b00000} + new(0, 16), // 128X64- {0b00000, 0b10000} + new(0, 0), // 128X128-{0b00000, 0b00000} + new(31, 28), // 4X16 - {0b11111, 0b11100} + new(28, 31), // 16X4 - {0b11100, 0b11111} + new(30, 24), // 8X32 - {0b11110, 0b11000} + new(24, 30), // 32X8 - {0b11000, 0b11110} + new(28, 16), // 16X64 - {0b11100, 0b10000} + new(16, 28), // 64X16 - {0b10000, 0b11100} + ]; + + /// + /// Maps neighboring intra prediction modes to the key-frame luma-mode entropy contexts. + /// + private static readonly byte[] IntraModeContextLookup = [0, 1, 2, 3, 4, 4, 4, 4, 3, 0, 1, 2, 0]; + + /// + /// Writes the partition tree and each final coding block for a superblock. + /// + /// The picture coding state. + /// The entropy-coding position state for the superblock. + /// The tile symbol encoder. + /// The encoder decisions for the superblock. + /// The transformed coefficients retained by raster-ordered superblock. + /// The zero-based tile index. + public static void WriteSuperblock( + Av1PictureControlSet pcs, + Av1EntropyCodingContext ec_ctx, + Av1SymbolEncoder writer, + Av1Superblock superblock, + Av1EncoderCoefficientBuffer coefficientBuffer, + ushort tileIndex) + { + PrecomputedBlockEncodingHandler blockEncoder = default; + WriteSuperblock( + pcs, + ec_ctx, + writer, + superblock, + coefficientBuffer, + tileIndex, + ref blockEncoder); + } + + /// + /// Writes a partition tree while producing each final block against the immediately preceding tile state. + /// + /// The value type that produces final-block decisions. + /// The picture coding state. + /// The entropy-coding position state for the superblock. + /// The tile symbol encoder. + /// The encoder decisions for the superblock. + /// The transformed coefficients retained by raster-ordered superblock. + /// The zero-based tile index. + /// The handler invoked for each final block. + public static void WriteSuperblock( + Av1PictureControlSet pcs, + Av1EntropyCodingContext ec_ctx, + Av1SymbolEncoder writer, + Av1Superblock superblock, + Av1EncoderCoefficientBuffer coefficientBuffer, + ushort tileIndex, + ref TBlockEncoder blockEncoder) + where TBlockEncoder : struct, IBlockEncodingHandler + { + ec_ctx.CodedAreaSuperblock = 0; + ec_ctx.CodedAreaSuperblockUv = 0; + ec_ctx.MacroBlock.Tile = superblock.TileInfo; + int partitionIndex = 0; + int finalBlockIndex = 0; + + // Current libaom writes the selected partition tree recursively from the superblock origin. Keeping the + // decisions in preorder removes the global geometry catalog and keeps traversal state on this stack. + WritePartitionTree( + pcs, + ec_ctx, + writer, + superblock, + coefficientBuffer, + tileIndex, + pcs.Sequence.SequenceHeader.SuperblockSize, + ec_ctx.SuperblockOrigin, + ref partitionIndex, + ref finalBlockIndex, + ref blockEncoder); + } + + /// + /// Writes one selected partition node and recursively visits its split children. + /// + private static void WritePartitionTree( + Av1PictureControlSet pcs, + Av1EntropyCodingContext entropyCodingContext, + Av1SymbolEncoder writer, + Av1Superblock superblock, + Av1EncoderCoefficientBuffer coefficientBuffer, + ushort tileIndex, + Av1BlockSize blockSize, + Point blockOrigin, + ref int partitionIndex, + ref int finalBlockIndex, + ref TBlockEncoder blockEncoder) + where TBlockEncoder : struct, IBlockEncodingHandler + { + Av1EncoderCommon common = pcs.Parent.Common; + int modeInfoRow = blockOrigin.Y >> Av1Constants.ModeInfoSizeLog2; + int modeInfoColumn = blockOrigin.X >> Av1Constants.ModeInfoSizeLog2; + if (modeInfoRow >= common.ModeInfoRowCount || modeInfoColumn >= common.ModeInfoColumnCount) + { + return; + } + + int currentPartitionIndex = partitionIndex++; + Av1PartitionType preparedPartition = + (Av1PartitionType)superblock.CodingUnitPartitionTypes[currentPartitionIndex]; + + Av1PartitionType partition = blockEncoder.SelectPartition( + writer, + entropyCodingContext.MacroBlock, + blockOrigin, + tileIndex, + blockSize, + preparedPartition); + + superblock.CodingUnitPartitionTypes[currentPartitionIndex] = (byte)partition; + Av1BlockSize subSize = partition.GetBlockSubSize(blockSize); + int halfBlockSize = blockSize.GetWidth() >> 1; + int quarterBlockSize = blockSize.GetWidth() >> 2; + + EncodePartition( + pcs, + writer, + blockSize, + partition, + blockOrigin, + pcs.PartitionContexts[tileIndex]); + + switch (partition) + { + case Av1PartitionType.None: + WriteFinalBlock( + pcs, + entropyCodingContext, + writer, + superblock, + coefficientBuffer, + tileIndex, + blockOrigin, + ref finalBlockIndex, + ref blockEncoder); + + break; + case Av1PartitionType.Horizontal: + WriteFinalBlock( + pcs, + entropyCodingContext, + writer, + superblock, + coefficientBuffer, + tileIndex, + blockOrigin, + ref finalBlockIndex, + ref blockEncoder); + + if (modeInfoRow + (blockSize.Get4x4HighCount() >> 1) < common.ModeInfoRowCount) + { + WriteFinalBlock( + pcs, + entropyCodingContext, + writer, + superblock, + coefficientBuffer, + tileIndex, + blockOrigin + new Size(0, halfBlockSize), + ref finalBlockIndex, + ref blockEncoder); + } + + break; + case Av1PartitionType.Vertical: + WriteFinalBlock( + pcs, + entropyCodingContext, + writer, + superblock, + coefficientBuffer, + tileIndex, + blockOrigin, + ref finalBlockIndex, + ref blockEncoder); + + if (modeInfoColumn + (blockSize.Get4x4WideCount() >> 1) < common.ModeInfoColumnCount) + { + WriteFinalBlock( + pcs, + entropyCodingContext, + writer, + superblock, + coefficientBuffer, + tileIndex, + blockOrigin + new Size(halfBlockSize, 0), + ref finalBlockIndex, + ref blockEncoder); + } + + break; + case Av1PartitionType.Split: + if (blockSize == Av1BlockSize.Block8x8) + { + // A split 8x8 node terminates in four 4x4 coding blocks. AV1 does not carry another + // partition symbol at that size, so the children are final blocks rather than tree nodes. + for (int childIndex = 0; childIndex < 4; childIndex++) + { + Point childOrigin = blockOrigin + new Size( + (childIndex & 1) * halfBlockSize, + (childIndex >> 1) * halfBlockSize); + + Point childModeInfoPosition = childOrigin >> Av1Constants.ModeInfoSizeLog2; + if (childModeInfoPosition.Y >= common.ModeInfoRowCount || + childModeInfoPosition.X >= common.ModeInfoColumnCount) + { + continue; + } + + WriteFinalBlock( + pcs, + entropyCodingContext, + writer, + superblock, + coefficientBuffer, + tileIndex, + childOrigin, + ref finalBlockIndex, + ref blockEncoder); + } + } + else + { + WritePartitionTree( + pcs, + entropyCodingContext, + writer, + superblock, + coefficientBuffer, + tileIndex, + subSize, + blockOrigin, + ref partitionIndex, + ref finalBlockIndex, + ref blockEncoder); + + WritePartitionTree( + pcs, + entropyCodingContext, + writer, + superblock, + coefficientBuffer, + tileIndex, + subSize, + blockOrigin + new Size(halfBlockSize, 0), + ref partitionIndex, + ref finalBlockIndex, + ref blockEncoder); + + WritePartitionTree( + pcs, + entropyCodingContext, + writer, + superblock, + coefficientBuffer, + tileIndex, + subSize, + blockOrigin + new Size(0, halfBlockSize), + ref partitionIndex, + ref finalBlockIndex, + ref blockEncoder); + + WritePartitionTree( + pcs, + entropyCodingContext, + writer, + superblock, + coefficientBuffer, + tileIndex, + subSize, + blockOrigin + new Size(halfBlockSize, halfBlockSize), + ref partitionIndex, + ref finalBlockIndex, + ref blockEncoder); + } + + break; + case Av1PartitionType.HorizontalA: + WriteFinalBlock( + pcs, + entropyCodingContext, + writer, + superblock, + coefficientBuffer, + tileIndex, + blockOrigin, + ref finalBlockIndex, + ref blockEncoder); + WriteFinalBlock( + pcs, + entropyCodingContext, + writer, + superblock, + coefficientBuffer, + tileIndex, + blockOrigin + new Size(halfBlockSize, 0), + ref finalBlockIndex, + ref blockEncoder); + WriteFinalBlock( + pcs, + entropyCodingContext, + writer, + superblock, + coefficientBuffer, + tileIndex, + blockOrigin + new Size(0, halfBlockSize), + ref finalBlockIndex, + ref blockEncoder); + + break; + case Av1PartitionType.HorizontalB: + WriteFinalBlock( + pcs, + entropyCodingContext, + writer, + superblock, + coefficientBuffer, + tileIndex, + blockOrigin, + ref finalBlockIndex, + ref blockEncoder); + WriteFinalBlock( + pcs, + entropyCodingContext, + writer, + superblock, + coefficientBuffer, + tileIndex, + blockOrigin + new Size(0, halfBlockSize), + ref finalBlockIndex, + ref blockEncoder); + WriteFinalBlock( + pcs, + entropyCodingContext, + writer, + superblock, + coefficientBuffer, + tileIndex, + blockOrigin + new Size(halfBlockSize, halfBlockSize), + ref finalBlockIndex, + ref blockEncoder); + + break; + case Av1PartitionType.VerticalA: + WriteFinalBlock( + pcs, + entropyCodingContext, + writer, + superblock, + coefficientBuffer, + tileIndex, + blockOrigin, + ref finalBlockIndex, + ref blockEncoder); + WriteFinalBlock( + pcs, + entropyCodingContext, + writer, + superblock, + coefficientBuffer, + tileIndex, + blockOrigin + new Size(0, halfBlockSize), + ref finalBlockIndex, + ref blockEncoder); + WriteFinalBlock( + pcs, + entropyCodingContext, + writer, + superblock, + coefficientBuffer, + tileIndex, + blockOrigin + new Size(halfBlockSize, 0), + ref finalBlockIndex, + ref blockEncoder); + + break; + case Av1PartitionType.VerticalB: + WriteFinalBlock( + pcs, + entropyCodingContext, + writer, + superblock, + coefficientBuffer, + tileIndex, + blockOrigin, + ref finalBlockIndex, + ref blockEncoder); + WriteFinalBlock( + pcs, + entropyCodingContext, + writer, + superblock, + coefficientBuffer, + tileIndex, + blockOrigin + new Size(halfBlockSize, 0), + ref finalBlockIndex, + ref blockEncoder); + WriteFinalBlock( + pcs, + entropyCodingContext, + writer, + superblock, + coefficientBuffer, + tileIndex, + blockOrigin + new Size(halfBlockSize, halfBlockSize), + ref finalBlockIndex, + ref blockEncoder); + + break; + case Av1PartitionType.Horizontal4: + for (int childIndex = 0; childIndex < 4; childIndex++) + { + Point childOrigin = blockOrigin + new Size(0, childIndex * quarterBlockSize); + if (childIndex > 0 && + (childOrigin.Y >> Av1Constants.ModeInfoSizeLog2) >= common.ModeInfoRowCount) + { + break; + } + + WriteFinalBlock( + pcs, + entropyCodingContext, + writer, + superblock, + coefficientBuffer, + tileIndex, + childOrigin, + ref finalBlockIndex, + ref blockEncoder); + } + + break; + case Av1PartitionType.Vertical4: + for (int childIndex = 0; childIndex < 4; childIndex++) + { + Point childOrigin = blockOrigin + new Size(childIndex * quarterBlockSize, 0); + if (childIndex > 0 && + (childOrigin.X >> Av1Constants.ModeInfoSizeLog2) >= common.ModeInfoColumnCount) + { + break; + } + + WriteFinalBlock( + pcs, + entropyCodingContext, + writer, + superblock, + coefficientBuffer, + tileIndex, + childOrigin, + ref finalBlockIndex, + ref blockEncoder); + } + + break; + } + + UpdatePartitionContexts( + pcs.PartitionContexts[tileIndex], + blockOrigin, + subSize, + blockSize, + partition); + } + + /// + /// Writes the next final block selected by partition traversal. + /// + private static void WriteFinalBlock( + Av1PictureControlSet pcs, + Av1EntropyCodingContext entropyCodingContext, + Av1SymbolEncoder writer, + Av1Superblock superblock, + Av1EncoderCoefficientBuffer coefficientBuffer, + ushort tileIndex, + Point blockOrigin, + ref int finalBlockIndex, + ref TBlockEncoder blockEncoder) + where TBlockEncoder : struct, IBlockEncodingHandler + { + ref Av1EncoderBlockStruct block = ref superblock.FinalBlocks[finalBlockIndex++]; + WriteModesBlock( + pcs, + entropyCodingContext, + writer, + superblock, + ref block, + tileIndex, + blockOrigin, + coefficientBuffer, + ref blockEncoder); + } + + /// + /// Publishes the partition contexts produced by one completed partition node. + /// + internal static void UpdatePartitionContexts( + Av1NeighborArrayUnit neighbors, + Point blockOrigin, + Av1BlockSize subSize, + Av1BlockSize blockSize, + Av1PartitionType partition) + { + if (blockSize < Av1BlockSize.Block8x8) + { + return; + } + + int halfBlockSize = blockSize.GetWidth() >> 1; + Av1BlockSize splitSize = Av1PartitionType.Split.GetBlockSubSize(blockSize); + switch (partition) + { + case Av1PartitionType.Split: + if (blockSize != Av1BlockSize.Block8x8) + { + return; + } + + UpdatePartitionContext(neighbors, blockOrigin, subSize, blockSize); + break; + case Av1PartitionType.None: + case Av1PartitionType.Horizontal: + case Av1PartitionType.Vertical: + case Av1PartitionType.Horizontal4: + case Av1PartitionType.Vertical4: + UpdatePartitionContext(neighbors, blockOrigin, subSize, blockSize); + break; + case Av1PartitionType.HorizontalA: + UpdatePartitionContext(neighbors, blockOrigin, splitSize, subSize); + UpdatePartitionContext( + neighbors, + blockOrigin + new Size(0, halfBlockSize), + subSize, + subSize); + + break; + case Av1PartitionType.HorizontalB: + UpdatePartitionContext(neighbors, blockOrigin, subSize, subSize); + UpdatePartitionContext( + neighbors, + blockOrigin + new Size(0, halfBlockSize), + splitSize, + subSize); + + break; + case Av1PartitionType.VerticalA: + UpdatePartitionContext(neighbors, blockOrigin, splitSize, subSize); + UpdatePartitionContext( + neighbors, + blockOrigin + new Size(halfBlockSize, 0), + subSize, + subSize); + + break; + case Av1PartitionType.VerticalB: + UpdatePartitionContext(neighbors, blockOrigin, subSize, subSize); + UpdatePartitionContext( + neighbors, + blockOrigin + new Size(halfBlockSize, 0), + splitSize, + subSize); + + break; + } + } + + /// + /// Writes one partition-context value across the complete parent edges. + /// + private static void UpdatePartitionContext( + Av1NeighborArrayUnit neighbors, + Point blockOrigin, + Av1BlockSize contextBlockSize, + Av1BlockSize coveredBlockSize) + { + Av1PartitionContext context = PartitionContextLookup[(int)contextBlockSize]; + Av1NeighborArrayUnit.UnitMask edgeMask = + Av1NeighborArrayUnit.UnitMask.Left | + Av1NeighborArrayUnit.UnitMask.Top; + + neighbors.UnitModeWrite( + context, + blockOrigin, + new Size(coveredBlockSize.GetWidth(), coveredBlockSize.GetHeight()), + edgeMask); + } + + /// + /// Gets the partition-symbol rate from the above and left contexts available at a block origin. + /// + /// The picture coding state. + /// The live tile symbol encoder. + /// The square parent block size. + /// The partition type to measure. + /// The block origin in samples. + /// The partition neighbor arrays for the tile. + /// The rate cost in 1/512-bit units. + public static int GetPartitionCost( + Av1PictureControlSet pcs, + Av1SymbolEncoder writer, + Av1BlockSize blockSize, + Av1PartitionType partitionType, + Point blockOrigin, + Av1NeighborArrayUnit partitionContexts) + { + int context = GetPartitionContext( + pcs, + blockSize, + blockOrigin, + partitionContexts, + out bool hasRows, + out bool hasColumns); + + if (!hasRows && !hasColumns) + { + return 0; + } + + if (hasRows && hasColumns) + { + return writer.GetPartitionTypeCost(partitionType, context); + } + + return !hasRows + ? writer.GetSplitOrHorizontalCost(partitionType, blockSize, context) + : writer.GetSplitOrVerticalCost(partitionType, blockSize, context); + } + + /// + /// Writes a partition symbol using the above and left partition contexts available at a block origin. + /// + /// The picture coding state. + /// The tile symbol encoder. + /// The square parent block size. + /// The selected partition type. + /// The block origin in samples. + /// The partition neighbor arrays for the tile. + public static void EncodePartition( + Av1PictureControlSet pcs, + Av1SymbolEncoder writer, + Av1BlockSize blockSize, + Av1PartitionType partitionType, + Point blockOrigin, + Av1NeighborArrayUnit partition_context_na) + { + bool is_partition_point = blockSize >= Av1BlockSize.Block8x8; + + if (!is_partition_point) + { + return; + } + + int context_index = GetPartitionContext( + pcs, + blockSize, + blockOrigin, + partition_context_na, + out bool has_rows, + out bool has_cols); + + if (!has_rows && !has_cols) + { + Guard.IsTrue(partitionType == Av1PartitionType.Split, nameof(partitionType), "Partition outside frame boundaries should have Split type."); + return; + } + + if (has_rows && has_cols) + { + writer.WritePartitionType(partitionType, context_index); + } + else if (!has_rows && has_cols) + { + writer.WriteSplitOrHorizontal(partitionType, blockSize, context_index); + } + else + { + writer.WriteSplitOrVertical(partitionType, blockSize, context_index); + } + + return; + } + + private static int GetPartitionContext( + Av1PictureControlSet pcs, + Av1BlockSize blockSize, + Point blockOrigin, + Av1NeighborArrayUnit partitionContexts, + out bool hasRows, + out bool hasColumns) + { + int halfBlockModeInfoCount = blockSize.Get4x4WideCount() >> 1; + Point modeInfoPosition = blockOrigin >> Av1Constants.ModeInfoSizeLog2; + hasRows = modeInfoPosition.Y + halfBlockModeInfoCount < pcs.Parent.Common.ModeInfoRowCount; + hasColumns = modeInfoPosition.X + halfBlockModeInfoCount < pcs.Parent.Common.ModeInfoColumnCount; + int leftIndex = partitionContexts.GetLeftIndex(blockOrigin); + int topIndex = partitionContexts.GetTopIndex(blockOrigin); + byte aboveContext = partitionContexts.Top[topIndex].Above == byte.MaxValue + ? (byte)0 + : partitionContexts.Top[topIndex].Above; + + byte leftContext = partitionContexts.Left[leftIndex].Left == byte.MaxValue + ? (byte)0 + : partitionContexts.Left[leftIndex].Left; + + int blockSizeLog2 = blockSize.Get4x4WidthLog2() - 1; + int above = (aboveContext >> blockSizeLog2) & 1; + int left = (leftContext >> blockSizeLog2) & 1; + + Guard.IsTrue(blockSize.Get4x4WidthLog2() == blockSize.Get4x4HeightLog2(), nameof(blockSize), "Blocks need to be square."); + Guard.IsTrue(blockSizeLog2 >= 0, nameof(blockSizeLog2), "bsl needs to be a positive integer."); + + // Each square block-size level owns four contexts selected by the current split bit of its neighbors. + return ((left * 2) + above) + (blockSizeLog2 * Av1Constants.PartitionProbabilitySet); + } + + /// + /// Writes the segmentation, prediction, transform, coefficient, and filter syntax for one final coding block. + /// + /// The picture coding state. + /// The entropy-coding position state for the superblock. + /// The tile symbol encoder. + /// The containing superblock. + /// The final encoder decisions for the block. + /// The zero-based tile index. + /// The absolute luma-sample origin of the block. + /// The transformed coefficients retained by raster-ordered superblock. + /// The final-block decision producer. + private static void WriteModesBlock( + Av1PictureControlSet pcs, + Av1EntropyCodingContext entropyCodingContext, + Av1SymbolEncoder writer, + Av1Superblock tb_ptr, + ref Av1EncoderBlockStruct blk_ptr, + ushort tile_idx, + Point blockOrigin, + Av1EncoderCoefficientBuffer coefficientBuffer, + ref TBlockEncoder blockEncoder) + where TBlockEncoder : struct, IBlockEncodingHandler + { + Av1SequenceControlSet scs = pcs.Sequence; + ObuFrameHeader frm_hdr = pcs.Parent.FrameHeader; + Av1NeighborArrayUnit luma_dc_sign_level_coeff_na = pcs.LuminanceDcSignLevelCoefficientNeighbors[tile_idx]; + Av1NeighborArrayUnit cr_dc_sign_level_coeff_na = pcs.CrDcSignLevelCoefficientNeighbors[tile_idx]; + Av1NeighborArrayUnit cb_dc_sign_level_coeff_na = pcs.CbDcSignLevelCoefficientNeighbors[tile_idx]; + int mi_row = blockOrigin.Y >> Av1Constants.ModeInfoSizeLog2; + int mi_col = blockOrigin.X >> Av1Constants.ModeInfoSizeLog2; + int mi_stride = pcs.Parent.Common.ModeInfoStride; + Point modeInfoPosition = new(mi_col, mi_row); + ref Av1MacroBlockModeInfo macroBlockModeInfo = ref pcs.GetMacroBlockModeInfo(modeInfoPosition); + Av1BlockSize blockSize = macroBlockModeInfo.Block.BlockSize; + pcs.MapModeInfoBlock(modeInfoPosition, blockSize); + Av1MacroBlockD macroBlock = entropyCodingContext.MacroBlock; + + Guard.MustBeLessThan((int)blockSize, (int)Av1BlockSize.AllSizes, nameof(blockSize)); + + SetModeInfoRowAndColumn( + pcs, + macroBlock, + macroBlock.Tile, + modeInfoPosition, + blockSize, + mi_stride, + pcs.Parent.Common.ModeInfoRowCount, + pcs.Parent.Common.ModeInfoColumnCount); + + // Producing the decision here exposes exactly the reconstructed neighbors, coefficient contexts, + // and adaptive probabilities that the following syntax writes. + tb_ptr.Workspace.PaletteInfo = default; + ref Av1EncoderPaletteInfo paletteInfo = ref tb_ptr.Workspace.PaletteInfo; + blockEncoder.EncodeBlock( + writer, + macroBlock, + blockOrigin, + tile_idx, + ref macroBlockModeInfo, + ref blk_ptr, + ref paletteInfo); + + bool skipWritingCoefficients = macroBlockModeInfo.Block.Skip; + + // Segmentation, skip, filter, and quantizer syntax precede the prediction-domain branch in both + // intra and inter frames. Keeping this prefix shared preserves the decoder's symbol order. + { + if (pcs.Parent.FrameHeader.SegmentationParameters.Enabled && pcs.Parent.FrameHeader.SegmentationParameters.SegmentIdPrecedesSkip) + { + WriteSegmentId( + pcs, + writer, + blockSize, + blockOrigin, + macroBlock, + ref blk_ptr, + skipWritingCoefficients, + beforeSkip: true); + } + + EncodeSkipCoefficients(writer, macroBlock, skipWritingCoefficients); + + if (pcs.Parent.FrameHeader.SegmentationParameters.Enabled && !pcs.Parent.FrameHeader.SegmentationParameters.SegmentIdPrecedesSkip) + { + WriteSegmentId( + pcs, + writer, + blockSize, + blockOrigin, + macroBlock, + ref blk_ptr, + skipWritingCoefficients, + beforeSkip: false); + } + + WriteCdef( + scs, + pcs, + writer, + tile_idx, + skipWritingCoefficients, + modeInfoPosition); + + if (pcs.Parent.FrameHeader.DeltaQParameters.IsPresent) + { + int current_q_index = blk_ptr.QuantizationIndex; + bool super_block_upper_left = (((blockOrigin.Y >> 2) & (scs.SequenceHeader.SuperblockModeInfoSize - 1)) == 0) && + (((blockOrigin.X >> 2) & (scs.SequenceHeader.SuperblockModeInfoSize - 1)) == 0); + if ((blockSize != scs.SequenceHeader.SuperblockSize || !skipWritingCoefficients) && super_block_upper_left) + { + Guard.MustBeGreaterThan(current_q_index, 0, nameof(current_q_index)); + int reduced_delta_qindex = (current_q_index - pcs.Parent.PreviousQIndex.Span[tile_idx]) / + frm_hdr.DeltaQParameters.Resolution; + + writer.WriteDeltaQuantizerIndex(reduced_delta_qindex); + pcs.Parent.PreviousQIndex.Span[tile_idx] = current_q_index; + } + } + + bool isInterBlock = macroBlockModeInfo.Block.ReferenceFrame > Av1ReferenceFrameType.Intra; + bool isGlobalMotionForced = false; + bool isReferenceForced = false; + if (!frm_hdr.IsIntra) + { + ObuSegmentationParameters segmentation = frm_hdr.SegmentationParameters; + int segmentId = macroBlockModeInfo.Block.SegmentId; + isReferenceForced = segmentation.IsFeatureActive( + segmentId, + ObuSegmentationLevelFeature.ReferenceFrame); + + isGlobalMotionForced = segmentation.IsFeatureActive( + segmentId, + ObuSegmentationLevelFeature.GlobalMotionVector); + + if (!isReferenceForced && !isGlobalMotionForced) + { + int intraInterContext = GetIntraInterContext(macroBlock); + writer.WriteIsInter(isInterBlock, intraInterContext); + } + } + + Av1PredictionMode lumaMode = macroBlockModeInfo.Block.Mode; + Av1ChromaPredictionMode intra_chroma_mode = macroBlockModeInfo.Block.UvMode; + if (isInterBlock) + { + if (!isReferenceForced && !isGlobalMotionForced) + { + Span referenceCounts = stackalloc byte[Av1Constants.ReferenceFrameCount]; + CollectNeighborReferenceCounts(macroBlock, referenceCounts); + writer.WriteSingleReference( + macroBlockModeInfo.Block.ReferenceFrame, + referenceCounts); + } + + if (!isGlobalMotionForced) + { + ref Av1ReferenceMotionVectors referenceMotionVectors = ref tb_ptr.Workspace.ReferenceMotionVectors; + referenceMotionVectors.Build( + pcs, + macroBlock, + modeInfoPosition, + blockSize, + macroBlockModeInfo.Block.PartitionType, + scs.SequenceHeader, + frm_hdr, + macroBlockModeInfo.Block.ReferenceFrame); + + writer.WriteInterMode(lumaMode, referenceMotionVectors.ModeContext); + int referenceMotionVectorIndex = blk_ptr.ReferenceMotionVectorIndex; + if (lumaMode == Av1PredictionMode.NearMotionVector) + { + // NEARMV reserves stack entry zero for NEARESTMV, so its DRL decisions advance from + // near entry zero to one and then from one to two. + for (int index = 1; index < 3 && referenceMotionVectors.Count > index + 1; index++) + { + bool advance = referenceMotionVectorIndex >= index; + int context = Av1SymbolContextHelper.GetDrlContext(referenceMotionVectors.Weights, index); + writer.WriteDynamicReferenceList(advance, context); + if (!advance) + { + break; + } + } + } + else if (lumaMode == Av1PredictionMode.NewMotionVector) + { + // NEWMV begins at stack entry zero and can advance through entries one and two. + for (int index = 0; index < 2 && referenceMotionVectors.Count > index + 1; index++) + { + bool advance = referenceMotionVectorIndex > index; + int context = Av1SymbolContextHelper.GetDrlContext(referenceMotionVectors.Weights, index); + writer.WriteDynamicReferenceList(advance, context); + if (!advance) + { + break; + } + } + + Av1MotionVector vector = pcs.GetDisplacementVector(modeInfoPosition); + writer.WriteMotionVector( + vector, + referenceMotionVectors.GetNewReference(referenceMotionVectorIndex), + frm_hdr.MotionVectorPrecision); + } + } + + if (UsesSwitchableInterpolation(frm_hdr, macroBlockModeInfo.Block)) + { + // The vertical symbol is first and supplies both axes unless the sequence enables dual filters. + int verticalContext = Av1SymbolContextHelper.GetSwitchableInterpolationContext( + macroBlockModeInfo.Block, + macroBlock, + direction: 0); + + writer.WriteSwitchableInterpolationFilter(macroBlockModeInfo.Block.VerticalInterpolationFilter, verticalContext); + if (scs.SequenceHeader.EnableDualFilter) + { + int horizontalContext = Av1SymbolContextHelper.GetSwitchableInterpolationContext( + macroBlockModeInfo.Block, + macroBlock, + direction: 1); + + writer.WriteSwitchableInterpolationFilter(macroBlockModeInfo.Block.HorizontalInterpolationFilter, horizontalContext); + } + } + } + else if (IsIntraBlockCopyAllowed(pcs.Parent.FrameHeader/*, pcs.Parent.SliceType*/)) + { + WriteIntraBlockCopyInfo( + pcs, + writer, + macroBlock, + modeInfoPosition, + macroBlockModeInfo); + } + + if (!isInterBlock && !macroBlockModeInfo.Block.UseIntraBlockCopy) + { + EncodeIntraLumaMode( + writer, + frm_hdr, + macroBlockModeInfo, + macroBlock, + ref blk_ptr, + blockSize, + lumaMode); + } + + if (!isInterBlock && !macroBlockModeInfo.Block.UseIntraBlockCopy) + { + if (blk_ptr.HasChroma) + { + EncodeIntraChromaMode( + writer, + frm_hdr, + scs.SequenceHeader.ColorConfig, + macroBlockModeInfo, + ref blk_ptr, + blockSize, + lumaMode, + intra_chroma_mode); + } + } + + bool paletteAllowed = !isInterBlock && + !macroBlockModeInfo.Block.UseIntraBlockCopy && + IsPaletteAllowed(frm_hdr.AllowScreenContentTools, blockSize); + + if (paletteAllowed) + { + WritePaletteModeInfo( + scs, + pcs, + writer, + macroBlock, + macroBlockModeInfo, + ref paletteInfo, + blockSize, + blockOrigin, + tile_idx, + blk_ptr.HasChroma); + } + + if (!isInterBlock && + !macroBlockModeInfo.Block.UseIntraBlockCopy && + IsFilterIntraAllowed( + scs.SequenceHeader.EnableFilterIntra, + blockSize, + paletteInfo.PaletteSizes[0], + lumaMode)) + { + writer.WriteFilterIntraMode(blk_ptr.FilterIntraMode, blockSize); + } + + if (paletteAllowed) + { + ObuColorConfig colorConfig = scs.SequenceHeader.ColorConfig; + int palettePlaneCount = Math.Min(2, colorConfig.PlaneCount); + for (int plane = 0; plane < palettePlaneCount; ++plane) + { + int paletteSize = paletteInfo.PaletteSizes[plane]; + if (paletteSize == 0) + { + continue; + } + + Av1PlaneType planeType = (Av1PlaneType)plane; + int subX = planeType == Av1PlaneType.Uv && colorConfig.SubSamplingX ? 1 : 0; + int subY = planeType == Av1PlaneType.Uv && colorConfig.SubSamplingY ? 1 : 0; + int blockWidth = blockSize.GetWidth(); + int blockHeight = blockSize.GetHeight(); + int planeWidth = blockWidth >> subX; + int planeHeight = blockHeight >> subY; + + // Palette syntax covers coded alignment samples too. Visible-frame clipping would omit symbols + // that the decoder consumes before transform syntax and corrupt the remainder of the tile. + int columns = (blockWidth + (Math.Min(0, macroBlock.ToRightEdge) >> 3)) >> subX; + int rows = (blockHeight + (Math.Min(0, macroBlock.ToBottomEdge) >> 3)) >> subY; + Buffer2DRegion colorIndexMap = tb_ptr.Workspace + .GetPaletteMaps() + .GetMap(planeType, planeWidth, planeHeight); + + writer.WritePaletteColorMap( + paletteSize, + planeType, + rows, + columns, + colorIndexMap); + } + } + + WriteTransformSize( + pcs, + writer, + ref macroBlockModeInfo, + macroBlock, + blockSize, + blockOrigin, + tile_idx); + + entropyCodingContext.MacroBlockModeInfo = macroBlockModeInfo; + if (!skipWritingCoefficients) + { + EncodeCoefficients1d( + pcs, + entropyCodingContext, + writer, + ref blk_ptr, + blockOrigin, + lumaMode, + blockSize, + coefficientBuffer, + tb_ptr.Index, + luma_dc_sign_level_coeff_na, + cr_dc_sign_level_coeff_na, + cb_dc_sign_level_coeff_na); + } + } + + // Palette colors are published only after the current block's mode and map have consumed the preceding edges. + if (frm_hdr.AllowScreenContentTools) + { + const Av1NeighborArrayUnit.UnitMask PaletteContextMask = + Av1NeighborArrayUnit.UnitMask.Left | + Av1NeighborArrayUnit.UnitMask.Top; + + pcs.PaletteContexts[tile_idx].UnitModeWrite( + paletteInfo, + blockOrigin, + new Size(blockSize.GetWidth(), blockSize.GetHeight()), + PaletteContextMask); + } + + // Coefficient neighbor state follows the same post-symbol ownership boundary. + UpdateNeighbors(pcs, entropyCodingContext, blockOrigin, ref blk_ptr, tile_idx, blockSize); + } + + /// + /// Derives the uniform intra transform-size context from the current above and left edges. + /// + /// The retained transform widths and heights. + /// The reusable macroblock edge and neighbor state. + /// The block origin in samples. + /// The block size defining the maximum transform. + /// The uniform transform-size context. + public static int GetTransformSizeContext( + Av1NeighborArrayUnit transformContexts, + Av1MacroBlockD macroBlock, + Point blockOrigin, + Av1BlockSize blockSize) + { + Av1TransformSize maximumTransformSize = blockSize.GetMaximumTransformSize(); + int above = transformContexts.Top[transformContexts.GetTopIndex(blockOrigin)] >= maximumTransformSize.GetWidth() ? 1 : 0; + int left = transformContexts.Left[transformContexts.GetLeftIndex(blockOrigin)] >= maximumTransformSize.GetHeight() ? 1 : 0; + + // Inter neighbors contribute their coding-block extent, not their residual-transform extent. + if (macroBlock.IsUpAvailable) + { + ref Av1MacroBlockModeInfo aboveModeInfo = + ref macroBlock.GetRelativeModeInfo(-macroBlock.ModeInfoStride); + + if (aboveModeInfo.Block.ReferenceFrame > Av1ReferenceFrameType.Intra || + aboveModeInfo.Block.UseIntraBlockCopy) + { + above = aboveModeInfo.Block.BlockSize.GetWidth() >= maximumTransformSize.GetWidth() ? 1 : 0; + } + } + + if (macroBlock.IsLeftAvailable) + { + ref Av1MacroBlockModeInfo leftModeInfo = ref macroBlock.GetRelativeModeInfo(-1); + if (leftModeInfo.Block.ReferenceFrame > Av1ReferenceFrameType.Intra || + leftModeInfo.Block.UseIntraBlockCopy) + { + left = leftModeInfo.Block.BlockSize.GetHeight() >= maximumTransformSize.GetHeight() ? 1 : 0; + } + } + + return macroBlock.IsUpAvailable + ? macroBlock.IsLeftAvailable ? above + left : above + : macroBlock.IsLeftAvailable ? left : 0; + } + + /// + /// Writes or derives the block transform size and publishes its edge contexts. + /// + /// The picture coding state. + /// The tile symbol encoder. + /// The selected block modes. + /// The reusable macroblock edge and neighbor state. + /// The block size. + /// The block origin in samples. + /// The zero-based tile index. + internal static void WriteTransformSize( + Av1PictureControlSet pcs, + Av1SymbolEncoder writer, + ref Av1MacroBlockModeInfo macroBlockModeInfo, + Av1MacroBlockD macroBlock, + Av1BlockSize blockSize, + Point blockOrigin, + int tileIndex) + { + ObuFrameHeader frameHeader = pcs.Parent.FrameHeader; + bool isLossless = frameHeader.LosslessArray[macroBlockModeInfo.Block.SegmentId]; + bool isInter = macroBlockModeInfo.Block.ReferenceFrame > Av1ReferenceFrameType.Intra || + macroBlockModeInfo.Block.UseIntraBlockCopy; + bool writesUniformTransformSize = !isLossless && + frameHeader.TransformMode == Av1TransformMode.Select && + !isInter && + blockSize > Av1BlockSize.Block4x4; + bool writesVariableTransformSize = !isLossless && + frameHeader.TransformMode == Av1TransformMode.Select && + isInter && + !macroBlockModeInfo.Block.Skip && + blockSize > Av1BlockSize.Block4x4; + + Av1TransformSize transformSize = isLossless + ? Av1TransformSize.Size4x4 + : writesUniformTransformSize || writesVariableTransformSize + ? macroBlockModeInfo.Block.TransformSize + : blockSize.GetMaximumTransformSize(); + + macroBlockModeInfo.Block.TransformSize = transformSize; + Av1NeighborArrayUnit transformContexts = pcs.TransformFunctionContexts[tileIndex]; + if (writesUniformTransformSize) + { + int context = GetTransformSizeContext(transformContexts, macroBlock, blockOrigin, blockSize); + writer.WriteTransformSize(blockSize, transformSize, context); + } + else if (writesVariableTransformSize) + { + int topIndex = transformContexts.GetTopIndex(blockOrigin); + int leftIndex = transformContexts.GetLeftIndex(blockOrigin); + int context = Av1SymbolContextHelper.GetTransformPartitionContext( + transformContexts.Top[topIndex], + transformContexts.Left[leftIndex], + blockSize, + blockSize.GetMaximumTransformSize()); + + // Current inter decisions retain the maximum transform, so their variable-transform tree has one unsplit root. + writer.WriteTransformPartition(false, context); + } + + Size blockDimensions = new(blockSize.GetWidth(), blockSize.GetHeight()); + + // Above entries retain transform widths and left entries retain heights, including rectangular selections. + transformContexts.UnitModeWrite( + (byte)transformSize.GetWidth(), + blockOrigin, + blockDimensions, + Av1NeighborArrayUnit.UnitMask.Top); + + transformContexts.UnitModeWrite( + (byte)transformSize.GetHeight(), + blockOrigin, + blockDimensions, + Av1NeighborArrayUnit.UnitMask.Left); + } + + /// + /// Gets the chroma prediction-mode and directional-angle rate from the live tile probabilities. + /// + /// The live tile symbol encoder. + /// The current frame syntax and segment lossless state. + /// The sequence chroma subsampling configuration. + /// The selected block modes. + /// The luma block size. + /// The selected luma prediction mode. + /// The candidate chroma prediction mode. + /// The signed directional-angle adjustment. + /// The chroma mode and directional-angle rate in 1/512-bit units. + public static int GetChromaModeCost( + Av1SymbolEncoder writer, + ObuFrameHeader frameHeader, + ObuColorConfig colorConfig, + Av1MacroBlockModeInfo macroBlockModeInfo, + Av1BlockSize blockSize, + Av1PredictionMode lumaMode, + Av1ChromaPredictionMode chromaMode, + int angleDelta) + { + bool isChromaFromLumaAllowed = IsChromaFromLumaAllowed( + frameHeader, + colorConfig, + macroBlockModeInfo, + blockSize); + + int cost = writer.GetChromaModeCost(chromaMode, isChromaFromLumaAllowed, lumaMode); + if (blockSize >= Av1BlockSize.Block8x8 && chromaMode.IsDirectional()) + { + cost += writer.GetAngleDeltaCost(angleDelta + Av1Constants.MaxAngleDelta, chromaMode.ToLumaMode()); + } + + return cost; + } + + /// + /// Writes the chroma intra mode, chroma-from-luma alpha values, and directional angle adjustment for a block. + /// + /// The tile symbol encoder. + /// The current frame syntax and segment lossless state. + /// The sequence chroma subsampling configuration. + /// The selected block modes. + /// The encoder prediction-unit state. + /// The luma block size. + /// The selected luma prediction mode. + /// The selected chroma prediction mode. + public static void EncodeIntraChromaMode( + Av1SymbolEncoder writer, + ObuFrameHeader frameHeader, + ObuColorConfig colorConfig, + Av1MacroBlockModeInfo macroBlockModeInfo, + ref Av1EncoderBlockStruct blk_ptr, + Av1BlockSize blockSize, + Av1PredictionMode lumaMode, + Av1ChromaPredictionMode chromaMode) + { + bool isChromaFromLumaAllowed = IsChromaFromLumaAllowed( + frameHeader, + colorConfig, + macroBlockModeInfo, + blockSize); + + writer.WriteChromaMode(chromaMode, isChromaFromLumaAllowed, lumaMode); + + if (chromaMode == Av1ChromaPredictionMode.ChromaFromLuma) + { + writer.WriteChromaFromLumaAlphas( + blk_ptr.PredictionUnit.ChromaFromLumaIndex, + blk_ptr.PredictionUnit.ChromaFromLumaSigns); + } + + if (blockSize >= Av1BlockSize.Block8x8 && macroBlockModeInfo.Block.UvMode.IsDirectional()) + { + writer.WriteAngleDelta( + blk_ptr.PredictionUnit.AngleDelta[(int)Av1PlaneType.Uv] + Av1Constants.MaxAngleDelta, + chromaMode.ToLumaMode()); + } + } + + private static bool IsChromaFromLumaAllowed( + ObuFrameHeader frameHeader, + ObuColorConfig colorConfig, + Av1MacroBlockModeInfo macroBlockModeInfo, + Av1BlockSize blockSize) + => blockSize.AllowsChromaFromLuma( + frameHeader.LosslessArray[macroBlockModeInfo.Block.SegmentId], + colorConfig.SubSamplingX, + colorConfig.SubSamplingY); + + /// + /// Gets the above and left key-frame contexts used to write an intra luma mode. + /// + /// The current macroblock and its mapped neighbors. + /// The context derived from the above luma mode. + /// The context derived from the left luma mode. + private static void GetYModeContext(Av1MacroBlockD xd, out byte above_ctx, out byte left_ctx) + { + Av1PredictionMode intraLumaLeftMode = Av1PredictionMode.DC; + Av1PredictionMode intraLumaTopMode = Av1PredictionMode.DC; + if (xd.IsLeftAvailable) + { + // Key-frame neighbors are intra blocks, so their luma modes directly select the context class. + intraLumaLeftMode = xd.GetRelativeModeInfo(-1).Block.Mode; + } + + if (xd.IsUpAvailable) + { + intraLumaTopMode = xd.GetRelativeModeInfo(-xd.ModeInfoStride).Block.Mode; + } + + above_ctx = IntraModeContextLookup[(int)intraLumaTopMode]; + left_ctx = IntraModeContextLookup[(int)intraLumaLeftMode]; + } + + /// + /// Gets the luma mode rate from the frame-appropriate distribution. + /// + /// The live tile symbol encoder. + /// The current block's mapped neighbor state. + /// The selected block size. + /// The candidate luma mode. + /// The signed directional-angle adjustment. + /// Whether the frame uses key-frame neighbor-conditioned mode syntax. + /// The luma mode and directional-angle rate in 1/512-bit units. + public static int GetLumaModeCost( + Av1SymbolEncoder writer, + Av1MacroBlockD macroBlock, + Av1BlockSize blockSize, + Av1PredictionMode mode, + int angleDelta, + bool isIntraFrame) + { + int cost; + if (isIntraFrame) + { + GetYModeContext(macroBlock, out byte topContext, out byte leftContext); + cost = writer.GetLumaModeCost(mode, topContext, leftContext); + } + else + { + cost = writer.GetInterFrameLumaModeCost(mode, blockSize); + } + + if (blockSize >= Av1BlockSize.Block8x8 && mode.IsDirectional()) + { + cost += writer.GetAngleDeltaCost(angleDelta + Av1Constants.MaxAngleDelta, mode); + } + + return cost; + } + + /// + /// Writes the frame-appropriate luma prediction mode and any directional angle adjustment. + /// + /// The tile symbol encoder. + /// The frame header that selects the luma-mode probability model. + /// The selected block modes. + /// The reusable macroblock edge and neighbor state. + /// The encoder prediction-unit state. + /// The block size. + /// The selected luma prediction mode. + private static void EncodeIntraLumaMode( + Av1SymbolEncoder writer, + ObuFrameHeader frameHeader, + Av1MacroBlockModeInfo macroBlockModeInfo, + Av1MacroBlockD macroBlock, + ref Av1EncoderBlockStruct blk_ptr, + Av1BlockSize blockSize, + Av1PredictionMode lumaMode) + { + if (frameHeader.IsIntra) + { + GetYModeContext(macroBlock, out byte topContext, out byte leftContext); + writer.WriteLumaMode(lumaMode, topContext, leftContext); + } + else + { + writer.WriteInterFrameLumaMode(lumaMode, blockSize); + } + + if (blockSize >= Av1BlockSize.Block8x8 && macroBlockModeInfo.Block.Mode.IsDirectional()) + { + writer.WriteAngleDelta(blk_ptr.PredictionUnit.AngleDelta[(int)Av1PlaneType.Y] + Av1Constants.MaxAngleDelta, lumaMode); + } + } + + /// + /// Gets the prediction-domain context from the immediately above and left encoder blocks. + /// + /// The current block's mapped neighbor state. + /// The context in the inclusive range zero through three. + public static int GetIntraInterContext(Av1MacroBlockD macroBlock) + { + bool hasAbove = macroBlock.IsUpAvailable; + bool hasLeft = macroBlock.IsLeftAvailable; + if (hasAbove && hasLeft) + { + bool aboveIsIntra = macroBlock + .GetRelativeModeInfo(-macroBlock.ModeInfoStride) + .Block.ReferenceFrame <= Av1ReferenceFrameType.Intra; + + bool leftIsIntra = macroBlock + .GetRelativeModeInfo(-1) + .Block.ReferenceFrame <= Av1ReferenceFrameType.Intra; + + if (aboveIsIntra && leftIsIntra) + { + return 3; + } + + return aboveIsIntra || leftIsIntra ? 1 : 0; + } + + if (hasAbove) + { + return macroBlock + .GetRelativeModeInfo(-macroBlock.ModeInfoStride) + .Block.ReferenceFrame <= Av1ReferenceFrameType.Intra + ? 2 + : 0; + } + + if (hasLeft) + { + return macroBlock + .GetRelativeModeInfo(-1) + .Block.ReferenceFrame <= Av1ReferenceFrameType.Intra + ? 2 + : 0; + } + + return 0; + } + + /// + /// Counts the single-reference labels used by the immediately above and left encoded blocks. + /// + /// The current block's mapped neighbor state. + /// The eight-entry destination indexed by reference-frame label. + public static void CollectNeighborReferenceCounts( + Av1MacroBlockD macroBlock, + Span referenceCounts) + { + // The caller supplies short-lived fixed storage for one block. Clearing it here keeps unavailable + // neighbors from retaining votes collected for a preceding block. + referenceCounts.Clear(); + if (macroBlock.IsUpAvailable) + { + Av1ReferenceFrameType referenceFrame = macroBlock + .GetRelativeModeInfo(-macroBlock.ModeInfoStride) + .Block.ReferenceFrame; + + if (referenceFrame > Av1ReferenceFrameType.Intra) + { + referenceCounts[(int)referenceFrame]++; + } + } + + if (macroBlock.IsLeftAvailable) + { + Av1ReferenceFrameType referenceFrame = macroBlock + .GetRelativeModeInfo(-1) + .Block.ReferenceFrame; + + if (referenceFrame > Av1ReferenceFrameType.Intra) + { + referenceCounts[(int)referenceFrame]++; + } + } + } + + /// + /// Writes luma and chroma palette-mode syntax for a block. + /// + /// The sequence coding state. + /// The picture coding state. + /// The tile symbol encoder. + /// The current block's mapped neighbor state. + /// The selected block modes. + /// The selected palette sizes and colors. + /// The block size. + /// The absolute luma-sample origin. + /// The zero-based tile index. + /// Whether the block owns chroma syntax. + internal static void WritePaletteModeInfo( + Av1SequenceControlSet scs, + Av1PictureControlSet pcs, + Av1SymbolEncoder writer, + Av1MacroBlockD macroBlock, + Av1MacroBlockModeInfo macroBlockModeInfo, + ref Av1EncoderPaletteInfo paletteInfo, + Av1BlockSize blockSize, + Point blockOrigin, + int tileIndex, + bool hasChroma) + { + int blockSizeContext = GetPaletteBlockSizeContext(blockSize); + Av1NeighborArrayUnit paletteContexts = pcs.PaletteContexts[tileIndex]; + int yPaletteSize = paletteInfo.PaletteSizes[0]; + if (macroBlockModeInfo.Block.Mode == Av1PredictionMode.DC) + { + int neighborContext = GetPaletteYModeContext(paletteContexts, macroBlock, blockOrigin); + writer.WritePaletteYMode(yPaletteSize != 0, blockSizeContext, neighborContext); + if (yPaletteSize != 0) + { + writer.WritePaletteSize(yPaletteSize, blockSizeContext, Av1PlaneType.Y); + Span colorCache = stackalloc ushort[2 * Av1Constants.PaletteMaxSize]; + int cacheSize = GetPaletteCache( + paletteContexts, + macroBlock, + blockOrigin, + Av1Plane.Y, + colorCache); + + writer.WritePaletteYColors( + colorCache[..cacheSize], + paletteInfo.GetColors(Av1Plane.Y), + scs.SequenceHeader.ColorConfig.BitDepth.GetBitCount()); + } + } + + int uvPaletteSize = paletteInfo.PaletteSizes[1]; + if (scs.SequenceHeader.ColorConfig.PlaneCount > 1 && + macroBlockModeInfo.Block.UvMode == Av1ChromaPredictionMode.DC && + hasChroma) + { + writer.WritePaletteUvMode(uvPaletteSize != 0, yPaletteSize != 0); + if (uvPaletteSize != 0) + { + writer.WritePaletteSize(uvPaletteSize, blockSizeContext, Av1PlaneType.Uv); + Span colorCache = stackalloc ushort[2 * Av1Constants.PaletteMaxSize]; + int cacheSize = GetPaletteCache( + paletteContexts, + macroBlock, + blockOrigin, + Av1Plane.U, + colorCache); + + writer.WritePaletteUvColors( + colorCache[..cacheSize], + paletteInfo.GetColors(Av1Plane.U), + paletteInfo.GetColors(Av1Plane.V), + scs.SequenceHeader.ColorConfig.BitDepth.GetBitCount()); + } + } + } + + /// + /// Builds the sorted palette-color cache from the available above and left encoder edges. + /// + internal static int GetPaletteCache( + Av1NeighborArrayUnit paletteContexts, + Av1MacroBlockD macroBlock, + Point blockOrigin, + Av1Plane plane, + Span cache) + { + // AV1 excludes the above palette at each 64-sample row boundary, even with 128x128 superblocks. + bool hasAbove = macroBlock.IsUpAvailable && + (blockOrigin.Y & (Av1BlockSize.Block64x64.GetHeight() - 1)) != 0; + + Av1EncoderPaletteInfo above = hasAbove + ? paletteContexts.Top[paletteContexts.GetTopIndex(blockOrigin)] + : default; + + Av1EncoderPaletteInfo left = macroBlock.IsLeftAvailable + ? paletteContexts.Left[paletteContexts.GetLeftIndex(blockOrigin)] + : default; + + ReadOnlySpan aboveColors = hasAbove ? above.GetColors(plane) : []; + ReadOnlySpan leftColors = macroBlock.IsLeftAvailable ? left.GetColors(plane) : []; + return Av1PaletteCache.Merge(aboveColors, leftColors, cache); + } + + /// + /// Gets the palette probability context derived from the logarithmic block area. + /// + internal static int GetPaletteBlockSizeContext(Av1BlockSize blockSize) + => Av1Math.Log2(blockSize.GetWidth() * blockSize.GetHeight()) - 6; + + /// + /// Counts the available above and left luma neighbors that selected palette mode. + /// + internal static int GetPaletteYModeContext( + Av1NeighborArrayUnit paletteContexts, + Av1MacroBlockD macroBlock, + Point blockOrigin) + { + int neighborContext = 0; + if (macroBlock.IsUpAvailable && + paletteContexts.Top[paletteContexts.GetTopIndex(blockOrigin)].PaletteSizes[0] != 0) + { + neighborContext++; + } + + if (macroBlock.IsLeftAvailable && + paletteContexts.Left[paletteContexts.GetLeftIndex(blockOrigin)].PaletteSizes[0] != 0) + { + neighborContext++; + } + + return neighborContext; + } + + /// + /// Determines whether filter-intra syntax is available for a block mode. + /// + /// A value indicating whether the sequence enables filter-intra prediction. + /// The block size. + /// The selected luma palette size. + /// The selected luma prediction mode. + /// when the block can use filter-intra prediction; otherwise, . + private static bool IsFilterIntraAllowed( + bool enableFilterIntra, + Av1BlockSize blockSize, + int paletteSize, + Av1PredictionMode mode) + => mode == Av1PredictionMode.DC && paletteSize == 0 && IsFilterIntraAllowedBlockSize(enableFilterIntra, blockSize); + + /// + /// Determines whether filter-intra prediction is enabled for a block size. + /// + /// A value indicating whether the sequence enables filter-intra prediction. + /// The block size. + /// when filter-intra prediction supports the block dimensions; otherwise, . + internal static bool IsFilterIntraAllowedBlockSize(bool enableFilterIntra, Av1BlockSize blockSize) + { + if (!enableFilterIntra) + { + return false; + } + + return blockSize.GetWidth() <= 32 && blockSize.GetHeight() <= 32; + } + + /// + /// Writes the intra-block-copy selection and displacement-vector syntax for a block. + /// + /// The frame-owned mode and displacement state. + /// The tile symbol encoder. + /// The current block's frame edges and tile availability. + /// The block origin in 4x4 mode-information units. + /// The selected block modes. + public static void WriteIntraBlockCopyInfo( + Av1PictureControlSet picture, + Av1SymbolEncoder writer, + Av1MacroBlockD macroBlock, + Point modeInfoPosition, + Av1MacroBlockModeInfo macroBlockModeInfo) + { + bool useIntraBlockCopy = macroBlockModeInfo.Block.UseIntraBlockCopy; + writer.WriteUseIntraBlockCopy(useIntraBlockCopy); + if (useIntraBlockCopy) + { + Span candidates = stackalloc Av1MotionVector[8]; + Span weights = stackalloc int[8]; + Av1MotionVector reference = Av1IntraBlockCopy.FindReference( + picture, + macroBlock, + modeInfoPosition, + macroBlockModeInfo.Block.BlockSize, + macroBlockModeInfo.Block.PartitionType, + candidates, + weights); + + writer.WriteDisplacementVector( + picture.GetDisplacementVector(modeInfoPosition), + reference); + } + } + + /// + /// Determines whether a single-reference encoder block carries switchable interpolation symbols. + /// + /// The current frame header. + /// The selected block syntax. + /// Whether the block writes a vertical filter and, when enabled, a horizontal filter. + public static bool UsesSwitchableInterpolation(ObuFrameHeader frameHeader, Av1EncoderBlockModeInfo modeInfo) + { + if (frameHeader.InterpolationFilter != Av1InterpolationFilter.Switchable || modeInfo.SkipMode) + { + return false; + } + + // Global identity and affine models infer the regular filter on blocks at least 8x8. Translation still + // carries filter symbols, including integer translations. Residual skip does not suppress these symbols. + return modeInfo.Mode != Av1PredictionMode.GlobalMotionVector || + Math.Min(modeInfo.BlockSize.GetWidth(), modeInfo.BlockSize.GetHeight()) < Av1BlockSize.Block8x8.GetWidth() || + frameHeader.GetGlobalMotionParameters()[(int)modeInfo.ReferenceFrame - 1].Type == Av1GlobalMotionType.Translation; + } + + /// + /// Determines whether the current frame permits intra block copy. + /// + /// The current frame header. + /// when both screen-content tools and intra block copy are enabled; otherwise, . + private static bool IsIntraBlockCopyAllowed(ObuFrameHeader frameHeader) + => frameHeader.AllowScreenContentTools && frameHeader.AllowIntraBlockCopy; + + /// + /// Updates coefficient neighbor arrays after writing a block. + /// + /// The picture coding state. + /// The entropy-coding position state for the superblock. + /// The block origin in samples. + /// The encoder block state. + /// The zero-based tile index. + /// The block size. + private static void UpdateNeighbors( + Av1PictureControlSet pcs, + Av1EntropyCodingContext entropyCodingContext, + Point blockOrigin, + ref Av1EncoderBlockStruct blk_ptr, + ushort tile_idx, + Av1BlockSize blockSize) + { + Av1NeighborArrayUnit luma_dc_sign_level_coeff_na = pcs.LuminanceDcSignLevelCoefficientNeighbors[tile_idx]; + Av1NeighborArrayUnit cr_dc_sign_level_coeff_na = pcs.CrDcSignLevelCoefficientNeighbors[tile_idx]; + Av1NeighborArrayUnit cb_dc_sign_level_coeff_na = pcs.CbDcSignLevelCoefficientNeighbors[tile_idx]; + Point modeInfoPosition = blockOrigin >> Av1Constants.ModeInfoSizeLog2; + ref Av1MacroBlockModeInfo mbmi = ref pcs.GetMacroBlockModeInfo(modeInfoPosition); + bool skip_coeff = mbmi.Block.Skip; + + Size size = new(blockSize.GetWidth(), blockSize.GetHeight()); + if (skip_coeff) + { + // A skipped block has an all-zero residual, so publish a zero sign/level context over its edges + // and advance coefficient positions without reading transform units. + luma_dc_sign_level_coeff_na.UnitModeWrite( + 0, + blockOrigin, + size, + Av1NeighborArrayUnit.UnitMask.Left | Av1NeighborArrayUnit.UnitMask.Top); + + ObuColorConfig colorConfig = pcs.Sequence.SequenceHeader.ColorConfig; + if (blk_ptr.HasChroma && !colorConfig.IsMonochrome) + { + int subsamplingX = colorConfig.SubSamplingX ? 1 : 0; + int subsamplingY = colorConfig.SubSamplingY ? 1 : 0; + Point chromaOrigin = GetChromaBlockOrigin(blockOrigin, subsamplingX, subsamplingY); + Av1BlockSize chromaBlockSize = blockSize.GetSubsampled(colorConfig.SubSamplingX, colorConfig.SubSamplingY); + Size chromaSize = new(chromaBlockSize.GetWidth(), chromaBlockSize.GetHeight()); + + cb_dc_sign_level_coeff_na.UnitModeWrite( + 0, + chromaOrigin, + chromaSize, + Av1NeighborArrayUnit.UnitMask.Left | Av1NeighborArrayUnit.UnitMask.Top); + cr_dc_sign_level_coeff_na.UnitModeWrite( + 0, + chromaOrigin, + chromaSize, + Av1NeighborArrayUnit.UnitMask.Left | Av1NeighborArrayUnit.UnitMask.Top); + entropyCodingContext.CodedAreaSuperblockUv += chromaSize.Width * chromaSize.Height; + } + + entropyCodingContext.CodedAreaSuperblock += size.Width * size.Height; + } + } + + /// + /// Determines whether screen-content tools and block dimensions permit palette mode. + /// + /// A value indicating whether screen-content tools are enabled. + /// The block size. + /// when palette mode is available for the block; otherwise, . + internal static bool IsPaletteAllowed(bool allowScreenContentTools, Av1BlockSize blockSize) + => allowScreenContentTools && + blockSize.GetWidth() <= 64 && + blockSize.GetHeight() <= 64 && + blockSize >= Av1BlockSize.Block8x8; + + /// + /// Writes the constrained directional enhancement filter strength at its first coded block in a filter unit. + /// + /// The sequence coding state. + /// The picture coding state. + /// The tile symbol encoder. + /// The zero-based tile index. + /// A value indicating whether the current block omits residual coefficients. + /// The block position in 4x4 mode-information units. + internal static void WriteCdef( + Av1SequenceControlSet scs, + Av1PictureControlSet pcs, + Av1SymbolEncoder writer, + int tileIndex, + bool skip, + Point modeInfoPosition) + { + ObuFrameHeader frameHeader = pcs.Parent.FrameHeader; + + if (frameHeader.CodedLossless || frameHeader.AllowIntraBlockCopy) + { + return; + } + + Span cdefPreset = pcs.CdefPreset.Span.Slice( + tileIndex * Av1Constants.CdefUnitsPerSuperblock, + Av1Constants.CdefUnitsPerSuperblock); + + // Each superblock begins with all contained 64x64 filter units unassigned. + if ((modeInfoPosition.Y & (scs.SequenceHeader.SuperblockModeInfoSize - 1)) == 0 && + (modeInfoPosition.X & (scs.SequenceHeader.SuperblockModeInfoSize - 1)) == 0) + { + cdefPreset.Fill(-1); + } + + // The strength is coded once, at the first non-skipped block in each 64x64 CDEF filter unit. + int cdefSize = 1 << (6 - Av1Constants.ModeInfoSizeLog2); + int unitColumn = (modeInfoPosition.X & cdefSize) != 0 ? 1 : 0; + int unitRow = (modeInfoPosition.Y & cdefSize) != 0 ? 1 : 0; + int index = scs.SequenceHeader.Use128x128Superblock ? unitColumn + (2 * unitRow) : 0; + + if (cdefPreset[index] == -1 && !skip) + { + int firstBlockMask = ~(cdefSize - 1); + Point firstBlockPosition = new( + modeInfoPosition.X & firstBlockMask, + modeInfoPosition.Y & firstBlockMask); + ref Av1MacroBlockModeInfo firstBlock = ref pcs.GetFromModeInfoGrid(firstBlockPosition); + + // CDEF strength belongs to the first mode-info block in the 64x64 filter unit even when skipped + // blocks delay transmission until a later coding block. + writer.WriteCdefStrength(firstBlock.CdefStrength, frameHeader.CdefParameters.BitCount); + cdefPreset[index] = firstBlock.CdefStrength; + } + } + + /// + /// Populates a macroblock's frame edges, tile-neighbor availability, and rectangular-partition context. + /// + /// The picture coding state. + /// The macroblock state to populate. + /// The active tile boundaries. + /// The block position in 4x4 mode-information units. + /// The block size. + /// The row stride of the mode-information grid. + /// The coded frame height in mode-information rows. + /// The coded frame width in mode-information columns. + internal static void SetModeInfoRowAndColumn( + Av1PictureControlSet pcs, + Av1MacroBlockD macroBlock, + Av1TileInfo tile, + Point modeInfoPosition, + Av1BlockSize blockSize, + int modeInfoStride, + int modeInfoRowCount, + int modeInfoColumnCount) + { + macroBlock.ToTopEdge = -((modeInfoPosition.Y << Av1Constants.ModeInfoSizeLog2) << 3); + int blockModeInfoHeight = blockSize.Get4x4HighCount(); + int blockModeInfoWidth = blockSize.Get4x4WideCount(); + macroBlock.ToBottomEdge = ((modeInfoRowCount - blockModeInfoHeight - modeInfoPosition.Y) << Av1Constants.ModeInfoSizeLog2) << 3; + macroBlock.ToLeftEdge = -((modeInfoPosition.X << Av1Constants.ModeInfoSizeLog2) << 3); + macroBlock.ToRightEdge = ((modeInfoColumnCount - blockModeInfoWidth - modeInfoPosition.X) << Av1Constants.ModeInfoSizeLog2) << 3; + + macroBlock.ModeInfoStride = modeInfoStride; + + // Prediction cannot cross tile boundaries even when frame mode information exists there. + macroBlock.IsUpAvailable = modeInfoPosition.Y > tile.ModeInfoRowStart; + macroBlock.IsLeftAvailable = modeInfoPosition.X > tile.ModeInfoColumnStart; + int modeInfoIndex = (modeInfoPosition.Y * modeInfoStride) + modeInfoPosition.X; + macroBlock.SetModeInfoGrid(pcs.ModeInfoGrid, pcs.ModeInfoAllocation, modeInfoIndex); + } + + /// + /// Writes luma and chroma transform coefficients for a block in plane order. + /// + /// The picture coding state. + /// The entropy-coding position state for the superblock. + /// The tile symbol encoder. + /// The encoder block state. + /// The block origin in samples. + /// The luma prediction direction. + /// The luma block size. + /// The transformed coefficients retained by raster-ordered superblock. + /// The raster-ordered index of the containing superblock. + /// The luma coefficient neighbor contexts. + /// The red-difference chroma coefficient neighbor contexts. + /// The blue-difference chroma coefficient neighbor contexts. + private static void EncodeCoefficients1d( + Av1PictureControlSet pcs, + Av1EntropyCodingContext ec_ctx, + Av1SymbolEncoder writer, + ref Av1EncoderBlockStruct blk_ptr, + Point blockOrigin, + Av1PredictionMode intraLumaDir, + Av1BlockSize planeBlockSize, + Av1EncoderCoefficientBuffer coefficientBuffer, + int superblockIndex, + Av1NeighborArrayUnit luma_dc_sign_level_coeff_na, + Av1NeighborArrayUnit cr_dc_sign_level_coeff_na, + Av1NeighborArrayUnit cb_dc_sign_level_coeff_na) + { + EncodeTransformCoefficientRegions( + pcs, + ec_ctx, + writer, + ref blk_ptr, + blockOrigin, + intraLumaDir, + planeBlockSize, + coefficientBuffer, + superblockIndex, + luma_dc_sign_level_coeff_na, + cr_dc_sign_level_coeff_na, + cb_dc_sign_level_coeff_na); + } + + /// + /// Writes each luma transform block and updates its DC-sign and coefficient-level neighbor contexts. + /// + /// The picture coding state. + /// The entropy-coding position state for the superblock. + /// The tile symbol encoder. + /// The encoder block state. + /// The block origin in samples. + /// The luma prediction direction. + /// The luma block size. + /// The transformed coefficients retained by raster-ordered superblock. + /// The raster-ordered index of the containing superblock. + /// The luma coefficient neighbor contexts. + public static void EncodeTransformCoefficientsY( + Av1PictureControlSet pcs, + Av1EntropyCodingContext entropyCodingContext, + Av1SymbolEncoder writer, + ref Av1EncoderBlockStruct blk_ptr, + Point blockOrigin, + Av1PredictionMode intraLumaDir, + Av1BlockSize plane_bsize, + Av1EncoderCoefficientBuffer coefficientBuffer, + int superblockIndex, + Av1NeighborArrayUnit luma_dc_sign_level_coeff_na) + { + Av1MacroBlockD macroBlock = entropyCodingContext.MacroBlock; + int maximumBlocksWide = plane_bsize.GetWidth(); + int maximumBlocksHigh = plane_bsize.GetHeight(); + if (macroBlock.ToRightEdge < 0) + { + maximumBlocksWide += macroBlock.ToRightEdge >> 3; + } + + if (macroBlock.ToBottomEdge < 0) + { + maximumBlocksHigh += macroBlock.ToBottomEdge >> 3; + } + + maximumBlocksWide >>= Av1Constants.ModeInfoSizeLog2; + maximumBlocksHigh >>= Av1Constants.ModeInfoSizeLog2; + int maximumUnitBlocksWide = Math.Min( + Av1BlockSize.Block64x64.Get4x4WideCount(), + maximumBlocksWide); + + int maximumUnitBlocksHigh = Math.Min( + Av1BlockSize.Block64x64.Get4x4HighCount(), + maximumBlocksHigh); + + for (int regionRow = 0; regionRow < maximumBlocksHigh; regionRow += maximumUnitBlocksHigh) + { + int unitBottom = Math.Min(regionRow + maximumUnitBlocksHigh, maximumBlocksHigh); + for (int regionColumn = 0; regionColumn < maximumBlocksWide; regionColumn += maximumUnitBlocksWide) + { + int unitRight = Math.Min(regionColumn + maximumUnitBlocksWide, maximumBlocksWide); + EncodeTransformCoefficientRegion( + pcs, + entropyCodingContext, + writer, + ref blk_ptr, + blockOrigin, + intraLumaDir, + plane_bsize, + Av1Plane.Y, + coefficientBuffer, + superblockIndex, + luma_dc_sign_level_coeff_na, + regionRow, + regionColumn, + unitBottom, + unitRight); + } + } + } + + /// + /// Writes both chroma transform blocks and updates their DC-sign and coefficient-level neighbor contexts. + /// + /// The picture coding state. + /// The entropy-coding position state for the superblock. + /// The tile symbol encoder. + /// The encoder block state. + /// The luma block origin in samples. + /// The luma prediction direction used by coefficient contexts. + /// The luma block size. + /// The transformed coefficients retained by raster-ordered superblock. + /// The raster-ordered index of the containing superblock. + /// The red-difference chroma coefficient neighbor contexts. + /// The blue-difference chroma coefficient neighbor contexts. + private static void EncodeTransformCoefficientsUv( + Av1PictureControlSet pcs, + Av1EntropyCodingContext entropyCodingContext, + Av1SymbolEncoder writer, + ref Av1EncoderBlockStruct blk_ptr, + Point blockOrigin, + Av1PredictionMode intraLumaDir, + Av1BlockSize plane_bsize, + Av1EncoderCoefficientBuffer coefficientBuffer, + int superblockIndex, + Av1NeighborArrayUnit cr_dc_sign_level_coeff_na, + Av1NeighborArrayUnit cb_dc_sign_level_coeff_na) + { + ObuColorConfig colorConfig = pcs.Sequence.SequenceHeader.ColorConfig; + if (!blk_ptr.HasChroma || colorConfig.IsMonochrome) + { + return; + } + + int subsamplingX = colorConfig.SubSamplingX ? 1 : 0; + int subsamplingY = colorConfig.SubSamplingY ? 1 : 0; + Av1BlockSize chromaBlockSize = plane_bsize.GetSubsampled(colorConfig.SubSamplingX, colorConfig.SubSamplingY); + Point chromaBlockOrigin = GetChromaBlockOrigin(blockOrigin, subsamplingX, subsamplingY); + Av1MacroBlockD macroBlock = entropyCodingContext.MacroBlock; + int maximumBlocksWide = chromaBlockSize.GetWidth(); + int maximumBlocksHigh = chromaBlockSize.GetHeight(); + if (macroBlock.ToRightEdge < 0) + { + maximumBlocksWide += macroBlock.ToRightEdge >> (3 + subsamplingX); + } + + if (macroBlock.ToBottomEdge < 0) + { + maximumBlocksHigh += macroBlock.ToBottomEdge >> (3 + subsamplingY); + } + + maximumBlocksWide >>= Av1Constants.ModeInfoSizeLog2; + maximumBlocksHigh >>= Av1Constants.ModeInfoSizeLog2; + Av1BlockSize maximumUnitBlockSize = + Av1BlockSize.Block64x64.GetSubsampled(colorConfig.SubSamplingX, colorConfig.SubSamplingY); + + int maximumUnitBlocksWide = Math.Min(maximumUnitBlockSize.Get4x4WideCount(), maximumBlocksWide); + int maximumUnitBlocksHigh = Math.Min(maximumUnitBlockSize.Get4x4HighCount(), maximumBlocksHigh); + + for (int regionRow = 0; regionRow < maximumBlocksHigh; regionRow += maximumUnitBlocksHigh) + { + int unitBottom = Math.Min(regionRow + maximumUnitBlocksHigh, maximumBlocksHigh); + for (int regionColumn = 0; regionColumn < maximumBlocksWide; regionColumn += maximumUnitBlocksWide) + { + int unitRight = Math.Min(regionColumn + maximumUnitBlocksWide, maximumBlocksWide); + EncodeTransformCoefficientRegion( + pcs, + entropyCodingContext, + writer, + ref blk_ptr, + chromaBlockOrigin, + intraLumaDir, + plane_bsize, + Av1Plane.U, + coefficientBuffer, + superblockIndex, + cb_dc_sign_level_coeff_na, + regionRow, + regionColumn, + unitBottom, + unitRight); + + EncodeTransformCoefficientRegion( + pcs, + entropyCodingContext, + writer, + ref blk_ptr, + chromaBlockOrigin, + intraLumaDir, + plane_bsize, + Av1Plane.V, + coefficientBuffer, + superblockIndex, + cr_dc_sign_level_coeff_na, + regionRow, + regionColumn, + unitBottom, + unitRight); + } + } + } + + private static void EncodeTransformCoefficientRegions( + Av1PictureControlSet pcs, + Av1EntropyCodingContext entropyCodingContext, + Av1SymbolEncoder writer, + ref Av1EncoderBlockStruct block, + Point blockOrigin, + Av1PredictionMode intraLumaMode, + Av1BlockSize blockSize, + Av1EncoderCoefficientBuffer coefficientBuffer, + int superblockIndex, + Av1NeighborArrayUnit lumaCoefficientNeighbors, + Av1NeighborArrayUnit redCoefficientNeighbors, + Av1NeighborArrayUnit blueCoefficientNeighbors) + { + Av1MacroBlockD macroBlock = entropyCodingContext.MacroBlock; + int maximumBlocksWide = blockSize.GetWidth(); + int maximumBlocksHigh = blockSize.GetHeight(); + if (macroBlock.ToRightEdge < 0) + { + maximumBlocksWide += macroBlock.ToRightEdge >> 3; + } + + if (macroBlock.ToBottomEdge < 0) + { + maximumBlocksHigh += macroBlock.ToBottomEdge >> 3; + } + + maximumBlocksWide >>= Av1Constants.ModeInfoSizeLog2; + maximumBlocksHigh >>= Av1Constants.ModeInfoSizeLog2; + int maximumUnitBlocksWide = Math.Min( + Av1BlockSize.Block64x64.Get4x4WideCount(), + maximumBlocksWide); + + int maximumUnitBlocksHigh = Math.Min( + Av1BlockSize.Block64x64.Get4x4HighCount(), + maximumBlocksHigh); + + ObuColorConfig colorConfig = pcs.Sequence.SequenceHeader.ColorConfig; + bool hasChroma = block.HasChroma && !colorConfig.IsMonochrome; + int subsamplingX = colorConfig.SubSamplingX ? 1 : 0; + int subsamplingY = colorConfig.SubSamplingY ? 1 : 0; + Point chromaBlockOrigin = GetChromaBlockOrigin(blockOrigin, subsamplingX, subsamplingY); + + // Residual syntax is region-major, then plane-major. Keeping the three plane calls together + // prevents a 128x128 block from emitting later luma regions before earlier chroma regions. + for (int regionRow = 0; regionRow < maximumBlocksHigh; regionRow += maximumUnitBlocksHigh) + { + int unitBottom = Math.Min(regionRow + maximumUnitBlocksHigh, maximumBlocksHigh); + for (int regionColumn = 0; regionColumn < maximumBlocksWide; regionColumn += maximumUnitBlocksWide) + { + int unitRight = Math.Min(regionColumn + maximumUnitBlocksWide, maximumBlocksWide); + EncodeTransformCoefficientRegion( + pcs, + entropyCodingContext, + writer, + ref block, + blockOrigin, + intraLumaMode, + blockSize, + Av1Plane.Y, + coefficientBuffer, + superblockIndex, + lumaCoefficientNeighbors, + regionRow, + regionColumn, + unitBottom, + unitRight); + + if (hasChroma) + { + int chromaRegionRow = regionRow >> subsamplingY; + int chromaRegionColumn = regionColumn >> subsamplingX; + + // Region limits count 4x4 units. Round the subsampled end upward so a chroma-owning + // 4x4, 4x8, or 8x4 luma block still emits its shared 4x4 chroma transform. + int chromaUnitBottom = Av1Math.RoundPowerOf2(unitBottom, subsamplingY); + int chromaUnitRight = Av1Math.RoundPowerOf2(unitRight, subsamplingX); + EncodeTransformCoefficientRegion( + pcs, + entropyCodingContext, + writer, + ref block, + chromaBlockOrigin, + intraLumaMode, + blockSize, + Av1Plane.U, + coefficientBuffer, + superblockIndex, + blueCoefficientNeighbors, + chromaRegionRow, + chromaRegionColumn, + chromaUnitBottom, + chromaUnitRight); + + EncodeTransformCoefficientRegion( + pcs, + entropyCodingContext, + writer, + ref block, + chromaBlockOrigin, + intraLumaMode, + blockSize, + Av1Plane.V, + coefficientBuffer, + superblockIndex, + redCoefficientNeighbors, + chromaRegionRow, + chromaRegionColumn, + chromaUnitBottom, + chromaUnitRight); + } + } + } + } + + private static void EncodeTransformCoefficientRegion( + Av1PictureControlSet pcs, + Av1EntropyCodingContext entropyCodingContext, + Av1SymbolEncoder writer, + ref Av1EncoderBlockStruct block, + Point planeBlockOrigin, + Av1PredictionMode intraLumaMode, + Av1BlockSize lumaBlockSize, + Av1Plane plane, + Av1EncoderCoefficientBuffer coefficientBuffer, + int superblockIndex, + Av1NeighborArrayUnit coefficientNeighbors, + int regionRow, + int regionColumn, + int unitBottom, + int unitRight) + { + ObuFrameHeader frameHeader = pcs.Parent.FrameHeader; + ObuColorConfig colorConfig = pcs.Sequence.SequenceHeader.ColorConfig; + bool isLuma = plane == Av1Plane.Y; + Av1BlockSize planeBlockSize = isLuma + ? lumaBlockSize + : lumaBlockSize.GetSubsampled(colorConfig.SubSamplingX, colorConfig.SubSamplingY); + + Av1TransformSize transformSize = isLuma + ? entropyCodingContext.MacroBlockModeInfo.Block.TransformSize + : frameHeader.LosslessArray[entropyCodingContext.MacroBlockModeInfo.Block.SegmentId] + ? Av1TransformSize.Size4x4 + : lumaBlockSize.GetMaxUvTransformSize(colorConfig.SubSamplingX, colorConfig.SubSamplingY); + + int transformBlockWidth = transformSize.Get4x4WideCount(); + int transformBlockHeight = transformSize.Get4x4HighCount(); + int transformWidth = transformSize.GetWidth(); + int transformHeight = transformSize.GetHeight(); + bool usesInterTransformSet = + entropyCodingContext.MacroBlockModeInfo.Block.ReferenceFrame > Av1ReferenceFrameType.Intra || + entropyCodingContext.MacroBlockModeInfo.Block.UseIntraBlockCopy; + Av1ComponentType componentType = isLuma + ? Av1ComponentType.Luminance + : Av1ComponentType.Chroma; + + Span planeCoefficients = coefficientBuffer.GetPlaneSpan(superblockIndex, plane); + Span planeTransformBlocks = + coefficientBuffer.GetTransformBlockSpan(superblockIndex, plane); + + int codedArea = isLuma + ? entropyCodingContext.CodedAreaSuperblock + : entropyCodingContext.CodedAreaSuperblockUv; + + for (int blockRow = regionRow; blockRow < unitBottom; blockRow += transformBlockHeight) + { + for (int blockColumn = regionColumn; blockColumn < unitRight; blockColumn += transformBlockWidth) + { + int transformStateIndex = + codedArea / Av1EncoderCoefficientBuffer.TransformBlockUnitCoefficientCount; + + ref Av1EncoderTransformBlockState transformBlock = + ref planeTransformBlocks[transformStateIndex]; + + Point transformOrigin = planeBlockOrigin + new Size( + blockColumn << Av1Constants.ModeInfoSizeLog2, + blockRow << Av1Constants.ModeInfoSizeLog2); + + Span coefficients = planeCoefficients[codedArea..]; + Av1TransformBlockContext blockContext = GetTransformBlockContexts( + componentType, + coefficientNeighbors, + transformOrigin, + planeBlockSize, + transformSize); + + Av1TransformType transformType = transformBlock.TransformType; + if (isLuma && transformBlock.EndOfBlock == 0) + { + // Empty luma transforms carry no transform-type symbol, so retain the canonical state. + transformType = transformBlock.TransformType = Av1TransformType.DctDct; + } + + int culLevel = writer.WriteCoefficients( + transformSize, + transformType, + intraLumaMode, + coefficients, + componentType, + blockContext, + transformBlock.EndOfBlock, + frameHeader.UseReducedTransformSet, + block.FilterIntraMode, + usesInterTransformSet); + + coefficientNeighbors.UnitModeWrite( + (byte)culLevel, + transformOrigin, + new Size(transformWidth, transformHeight), + Av1NeighborArrayUnit.UnitMask.Top | Av1NeighborArrayUnit.UnitMask.Left); + + codedArea += transformWidth * transformHeight; + } + } + + if (isLuma) + { + entropyCodingContext.CodedAreaSuperblock = codedArea; + } + else if (plane == Av1Plane.V) + { + // U and V share the same per-plane coded-area positions; advance only after V completes the region. + entropyCodingContext.CodedAreaSuperblockUv = codedArea; + } + } + + /// + /// Converts a luma origin to the shared 4x4 chroma-block origin for the active subsampling. + /// + /// The luma sample position. + /// The horizontal chroma subsampling shift. + /// The vertical chroma subsampling shift. + /// The aligned origin in chroma samples. + public static Point GetChromaBlockOrigin(Point lumaOrigin, int subsamplingX, int subsamplingY) + => new( + (lumaOrigin.X >> (Av1Constants.ModeInfoSizeLog2 + subsamplingX)) << Av1Constants.ModeInfoSizeLog2, + (lumaOrigin.Y >> (Av1Constants.ModeInfoSizeLog2 + subsamplingY)) << Av1Constants.ModeInfoSizeLog2); + + /// + /// Derives coefficient skip and DC-sign contexts from the transform block's above and left neighbors. + /// + /// The luma or chroma component class. + /// The packed DC-sign and coefficient-level neighbor contexts. + /// The transform-block origin in samples of the target plane. + /// The containing block size on the target plane. + /// The transform size. + /// The coefficient skip and DC-sign contexts selected by both transform edges. + public static Av1TransformBlockContext GetTransformBlockContexts( + Av1ComponentType plane, + Av1NeighborArrayUnit dcSignLevelCoefficientNeighborArray, + Point blockOrigin, + Av1BlockSize planeBlockSize, + Av1TransformSize transformSize) + { + int leftIndex = dcSignLevelCoefficientNeighborArray.GetLeftIndex(blockOrigin); + int topIndex = dcSignLevelCoefficientNeighborArray.GetTopIndex(blockOrigin); + int transformBlockWidth = transformSize.Get4x4WideCount(); + int transformBlockHeight = transformSize.Get4x4HighCount(); + ReadOnlySpan topContexts = dcSignLevelCoefficientNeighborArray.Top.Slice(topIndex, transformBlockWidth); + ReadOnlySpan leftContexts = dcSignLevelCoefficientNeighborArray.Left.Slice(leftIndex, transformBlockHeight); + + return GetTransformBlockContexts(plane, topContexts, leftContexts, planeBlockSize, transformSize); + } + + /// + /// Derives coefficient skip and DC-sign contexts from explicit transform-edge contexts. + /// + /// The luma or chroma component class. + /// The packed contexts immediately above the transform block. + /// The packed contexts immediately left of the transform block. + /// The containing block size on the target plane. + /// The transform size. + /// The coefficient skip and DC-sign contexts selected by both transform edges. + public static Av1TransformBlockContext GetTransformBlockContexts( + Av1ComponentType plane, + ReadOnlySpan topContexts, + ReadOnlySpan leftContexts, + Av1BlockSize planeBlockSize, + Av1TransformSize transformSize) + { + int dcSign = 0; + int top = 0; + int left = 0; + + // Each context packs a coefficient-level class in the low bits and the DC sign class above it. + // Accumulating both values in one traversal supplies every luma and chroma context without scratch storage. + foreach (byte context in topContexts) + { + byte sign = (byte)(context >> Av1Constants.CoefficientContextBitCount); + DebugGuard.MustBeLessThanOrEqualTo(sign, (byte)2, nameof(sign)); + if (sign == 1) + { + dcSign--; + } + else if (sign == 2) + { + dcSign++; + } + + top |= context; + } + + foreach (byte context in leftContexts) + { + byte sign = (byte)(context >> Av1Constants.CoefficientContextBitCount); + DebugGuard.MustBeLessThanOrEqualTo(sign, (byte)2, nameof(sign)); + if (sign == 1) + { + dcSign--; + } + else if (sign == 2) + { + dcSign++; + } + + left |= context; + } + + Av1TransformBlockContext blockContext = default; + blockContext.DcSignContext = dcSign > 0 ? 2 : dcSign < 0 ? 1 : 0; + if (plane == Av1ComponentType.Luminance) + { + if (planeBlockSize == transformSize.ToBlockSize()) + { + blockContext.SkipContext = 0; + } + else + { + top &= Av1Constants.CoefficientContextMask; + left &= Av1Constants.CoefficientContextMask; + blockContext.SkipContext = Av1SymbolContextHelper.GetTransformBlockSkipContext(top, left); + } + } + else + { + // Chroma contexts use only the presence of nonzero levels on each edge, plus an offset + // that distinguishes a transform smaller than its containing plane block. + int contextBase = (left != 0 ? 1 : 0) + (top != 0 ? 1 : 0); + int contextOffset = planeBlockSize.GetPelsLog2Count() > transformSize.ToBlockSize().GetPelsLog2Count() ? 10 : 7; + blockContext.SkipContext = contextBase + contextOffset; + } + + return blockContext; + } + + /// + /// Writes or predicts a block segment identifier and updates the frame segmentation map. + /// + /// The picture coding state. + /// The tile symbol encoder. + /// The block size. + /// The block origin in samples. + /// The reusable macroblock edge and neighbor state. + /// The encoder block state. + /// A value indicating whether residual coefficients are omitted. + /// Whether the segment identifier is written before the skip flag. + private static void WriteSegmentId( + Av1PictureControlSet pcs, + Av1SymbolEncoder writer, + Av1BlockSize blockSize, + Point blockOrigin, + Av1MacroBlockD macroBlock, + ref Av1EncoderBlockStruct block, + bool skip, + bool beforeSkip) + { + ObuSegmentationParameters segmentation_params = pcs.Parent.FrameHeader.SegmentationParameters; + if (!segmentation_params.Enabled) + { + return; + } + + int spatial_pred = GetSpatialSegmentationPrediction(pcs, macroBlock, blockOrigin, out int cdf_num); + if (!beforeSkip && skip) + { + // Post-skip segment syntax can infer the spatial predictor once the decoder already knows the block is skipped. + pcs.UpdateSegmentation(blockSize, blockOrigin, spatial_pred); + block.SegmentId = spatial_pred; + return; + } + + int coded_id = Av1SymbolContextHelper.NegativeDeinterleave(block.SegmentId, spatial_pred, segmentation_params.LastActiveSegmentId + 1); + writer.WriteSegmentId(coded_id, cdf_num); + pcs.UpdateSegmentation(blockSize, blockOrigin, block.SegmentId); + } + + /// + /// Derives a segment identifier predictor and entropy context from the upper-left, above, and left neighbors. + /// + /// The picture coding state. + /// The current macroblock and its neighbor availability. + /// The block origin in samples. + /// The entropy context selected by matching neighbor identifiers. + /// The spatially predicted segment identifier. + private static int GetSpatialSegmentationPrediction( + Av1PictureControlSet pcs, + Av1MacroBlockD xd, + Point blockOrigin, + out int cdf_index) + { + const int unavailableSegmentId = -1; + int prev_ul = unavailableSegmentId; + int prev_l = unavailableSegmentId; + int prev_u = unavailableSegmentId; + + int mi_col = blockOrigin.X >> Av1Constants.ModeInfoSizeLog2; + int mi_row = blockOrigin.Y >> Av1Constants.ModeInfoSizeLog2; + bool left_available = xd.IsLeftAvailable; + bool up_available = xd.IsUpAvailable; + Av1EncoderCommon cm = pcs.Parent.Common; + Span segmentation_map = pcs.SegmentationNeighborMap.Span; + + if (up_available && left_available) + { + prev_ul = Av1SymbolContextHelper.GetSegmentId(cm, segmentation_map, Av1BlockSize.Block4x4, new Point(mi_col - 1, mi_row - 1)); + } + + if (up_available) + { + prev_u = Av1SymbolContextHelper.GetSegmentId(cm, segmentation_map, Av1BlockSize.Block4x4, new Point(mi_col, mi_row - 1)); + } + + if (left_available) + { + prev_l = Av1SymbolContextHelper.GetSegmentId(cm, segmentation_map, Av1BlockSize.Block4x4, new Point(mi_col - 1, mi_row)); + } + + // The entropy context records whether zero, two, or all three neighboring IDs agree. + // Any unavailable neighbor falls back to the least-specific context. + if (prev_ul < 0 || prev_u < 0 || prev_l < 0) + { + cdf_index = 0; + } + else if ((prev_ul == prev_u) && (prev_ul == prev_l)) + { + cdf_index = 2; + } + else if ((prev_ul == prev_u) || (prev_ul == prev_l) || (prev_u == prev_l)) + { + cdf_index = 1; + } + else + { + cdf_index = 0; + } + + // Select the majority value when possible; otherwise AV1 gives the left neighbor precedence. + if (prev_u == unavailableSegmentId) + { + return prev_l == unavailableSegmentId ? 0 : prev_l; + } + + if (prev_l == unavailableSegmentId) + { + return prev_u; + } + + return (prev_ul == prev_u) ? prev_u : prev_l; + } + + /// + /// Gets the block skip context from the available above and left modes. + /// + /// The reusable macroblock edge and neighbor state. + /// The sum of the available above and left skip states. + public static int GetSkipContext(Av1MacroBlockD macroBlock) + { + bool aboveSkipped = macroBlock.IsUpAvailable && + macroBlock.GetRelativeModeInfo(-macroBlock.ModeInfoStride).Block.Skip; + + bool leftSkipped = macroBlock.IsLeftAvailable && macroBlock.GetRelativeModeInfo(-1).Block.Skip; + return (aboveSkipped ? 1 : 0) + (leftSkipped ? 1 : 0); + } + + /// + /// Selects block skip when it is cheaper than retaining empty transform syntax. + /// + /// The live tile symbol encoder. + /// The neighboring block skip context. + /// The complete coefficient rate for the empty transforms. + /// + /// when block skip has a strictly lower rate; otherwise, . + /// + public static bool ShouldSkipCoefficients( + Av1SymbolEncoder writer, + int skipContext, + int emptyTransformRate) + { + int skipRate = writer.GetSkipCost(true, skipContext); + int nonSkipRate = writer.GetSkipCost(false, skipContext) + emptyTransformRate; + + // Current libaom keeps intra blocks non-skipped. Empty transforms make both choices + // decoder-identical, so select skip only when its complete live rate is strictly lower. + return skipRate < nonSkipRate; + } + + /// + /// Writes the block skip flag using the sum of available above and left skip states as its context. + /// + /// The tile symbol encoder. + /// The reusable macroblock edge and neighbor state. + /// The skip value to write. + public static void EncodeSkipCoefficients(Av1SymbolEncoder writer, Av1MacroBlockD macroBlock, bool skip) + => writer.WriteSkip(skip, GetSkipContext(macroBlock)); +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1TransformBlockContext.cs b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1TransformBlockContext.cs new file mode 100644 index 0000000000..ac7fedf60d --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1TransformBlockContext.cs @@ -0,0 +1,20 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +/// +/// Carries the neighboring coefficient contexts used to entropy-code an AV1 transform block. +/// +internal struct Av1TransformBlockContext +{ + /// + /// Gets or sets the context used to decode the sign of the DC coefficient. + /// + public int DcSignContext { get; set; } + + /// + /// Gets or sets the neighboring transform-block skip context. + /// + public int SkipContext { get; set; } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1TransformInfo.cs b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1TransformInfo.cs new file mode 100644 index 0000000000..872c9aec24 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1TransformInfo.cs @@ -0,0 +1,69 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +/// +/// Describes the size, position, type, and residual state of one AV1 transform block. +/// +internal struct Av1TransformInfo +{ + /// + /// Initializes a new instance of the struct with a 4x4 transform at the origin. + /// + public Av1TransformInfo() + : this(Av1TransformSize.Size4x4, 0, 0) + { + } + + /// + /// Initializes a new instance of the struct. + /// + /// The transform size. + /// The horizontal offset in mode-information units. + /// The vertical offset in mode-information units. + public Av1TransformInfo(Av1TransformSize size, int offsetX, int offsetY) + { + this.Size = size; + this.OffsetX = offsetX; + this.OffsetY = offsetY; + } + + /// + /// Initializes a new instance of the struct. + /// + /// The to copy the information from. + public Av1TransformInfo(Av1TransformInfo originalInfo) + { + this.Size = originalInfo.Size; + this.OffsetX = originalInfo.OffsetX; + this.OffsetY = originalInfo.OffsetY; + } + + /// + /// Gets or sets the transform size used for this transform block. + /// + public Av1TransformSize Size { get; set; } + + /// + /// Gets or sets the transform type used for this transform block. + /// + public Av1TransformType Type { get; set; } + + /// + /// Gets or sets the horizontal offset of this block in mode-information units. + /// + public int OffsetX { get; set; } + + /// + /// Gets or sets the vertical offset of this block in mode-information units. + /// + public int OffsetY { get; set; } + + /// + /// Gets or sets the end position of the coded coefficients in entropy scan order; zero means no residual. + /// + public ushort EndOfBlock { get; set; } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Transform/Av1BlockDecoder.cs b/src/ImageSharp/Formats/Heif/Av1/Transform/Av1BlockDecoder.cs new file mode 100644 index 0000000000..a74d100d43 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Transform/Av1BlockDecoder.cs @@ -0,0 +1,2107 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Buffers; +using System.Runtime.InteropServices; +using SixLabors.ImageSharp.Formats.Heif.Av1.Motion; +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.LoopFilter; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.ChromaFromLuma; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.IntraBlockCopy; +using SixLabors.ImageSharp.Formats.Heif.Av1.ReferenceFrames; +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +/// +/// Reconstructs AV1 transform blocks by combining prediction, inverse quantization, and inverse transforms. +/// +internal sealed class Av1BlockDecoder : IDisposable +{ + /// + /// The sequence-level syntax that determines superblock size, plane layout, and sample depth. + /// + private readonly ObuSequenceHeader sequenceHeader; + + /// + /// The current frame syntax that determines quantization, lossless segments, and reconstruction geometry. + /// + private readonly ObuFrameHeader frameHeader; + + /// + /// The reconstructed Y, U, and V sample planes receiving prediction and residual output. + /// + private readonly Av1FrameBuffer frameBuffer; + + /// + /// The per-plane transform-size map consumed after reconstruction by the deblocking stage. + /// + private readonly Av1LoopFilterContext loopFilterContext; + + /// + /// The retained reconstructed frames addressable by inter prediction. + /// + private readonly Av1ReferenceFrameStore referenceFrames; + + /// + /// Owns the reusable inverse-transform and prediction storage. + /// + private readonly IMemoryOwner workspaceOwner; + + /// + /// The prediction workspace offset in signed-short storage elements. + /// + private readonly int predictionScratchOffset; + + /// + /// The reusable predictor portion of , excluding compound and chroma-from-luma storage. + /// + private readonly int predictorWorkingLength; + + /// + /// Reconstructs intra-predicted blocks using the frame-owned prediction workspace. + /// + private readonly Av1PredictionDecoder predictionDecoder; + + /// + /// Indicates whether transform traversal must also populate loop-filter parameters. + /// + private readonly bool isLoopFilterEnabled; + + /// + /// The next raster coefficient region for each plane in the current superblock. + /// + private InlineArray4 currentCoefficientIndex; + + /// + /// Accumulates reconstructed luma samples until a chroma-from-luma prediction block can consume them. + /// + private readonly Av1ChromaFromLumaContext chromaFromLumaContext; + + /// + /// Initializes a new instance of the class. + /// + /// The decoded sequence header. + /// The decoded frame header. + /// The frame buffer receiving reconstructed samples. + /// The transform-size map populated while reconstructing blocks. + /// The retained reconstructed frames selected by inter blocks. + /// The complete decoder-session palette map state. + public Av1BlockDecoder( + ObuSequenceHeader sequenceHeader, + ObuFrameHeader frameHeader, + Av1FrameBuffer frameBuffer, + Av1LoopFilterContext loopFilterContext, + Av1ReferenceFrameStore referenceFrames, + Av1TileReader.PaletteColorIndexMaps? paletteColorIndexMaps = null) + { + this.sequenceHeader = sequenceHeader; + this.frameHeader = frameHeader; + this.frameBuffer = frameBuffer; + this.loopFilterContext = loopFilterContext; + this.referenceFrames = referenceFrames; + int maximumBlockLength = 1 << sequenceHeader.SuperblockSizeLog2; + int maximumBlockArea = maximumBlockLength * maximumBlockLength; + int predictorWorkingLength = Math.Max( + Av1PredictionDecoder.ScratchLength, + Math.Max( + Av1TranslationalInterPredictor.GetScratchLength(maximumBlockLength, maximumBlockLength), + Av1ScaledInterPredictor.GetMaximumScaledScratchLength(maximumBlockLength, maximumBlockLength))); + + int compoundMaskLength = (maximumBlockArea + 1) >> 1; + int predictorWorkingOffset = (2 * maximumBlockArea) + compoundMaskLength; + int chromaFromLumaOffset = predictorWorkingOffset + predictorWorkingLength; + int predictionScratchLength = chromaFromLumaOffset + Av1ChromaFromLumaContext.BufferLength; + this.predictionScratchOffset = Av1TransformWorkspace.MaximumLength * 2; + + // Integer workspaces occupy even signed-short slices so one allocator owner can retain the complete block + // lifetime while prediction still receives the Memory contract needed by its reusable context. + this.workspaceOwner = this.frameBuffer.MemoryAllocator.Allocate( + this.predictionScratchOffset + predictionScratchLength); + + Memory predictionScratch = this.workspaceOwner.Memory[this.predictionScratchOffset..]; + this.predictorWorkingLength = predictorWorkingLength; + this.predictionDecoder = new( + sequenceHeader, + frameHeader, + predictionScratch.Slice(predictorWorkingOffset, predictorWorkingLength), + paletteColorIndexMaps); + this.isLoopFilterEnabled = frameHeader.LoopFilterParameters.FilterLevel[0] != 0 || + frameHeader.LoopFilterParameters.FilterLevel[1] != 0; + + this.chromaFromLumaContext = new( + sequenceHeader.ColorConfig, + predictionScratch.Slice(chromaFromLumaOffset, Av1ChromaFromLumaContext.BufferLength)); + } + + /// + /// Releases the pooled reconstruction workspaces owned by this decoder. + /// + public void Dispose() => this.workspaceOwner.Dispose(); + + /// + /// Resets the per-plane coefficient-region cursors before reconstructing a superblock. + /// + /// The superblock whose coefficient streams will be consumed. + public void UpdateSuperblock(Av1SuperblockInfo superblockInfo) + { + // Each superblock owns independent coefficient regions for Y, U, and V. Every transform advances its + // plane cursor by its nominal area, including transforms with no coded residual. + this.currentCoefficientIndex[0] = 0; + this.currentCoefficientIndex[1] = 0; + this.currentCoefficientIndex[2] = 0; + } + + /// + /// Reconstructs every luma and chroma transform unit belonging to one decoded AV1 block. + /// + /// The decoded prediction, segmentation, skip, and transform state. + /// The block origin in units of four luma samples. + /// The decoded block size. + /// The owning superblock's transform and coefficient storage. + /// The tile boundaries used to determine neighbor availability. + public void DecodeBlock(Av1BlockModeInfo modeInfo, Point modeInfoPosition, Av1BlockSize blockSize, Av1SuperblockInfo superblockInfo, Av1TileInfo tileInfo) + { + Span transformWorkspace = MemoryMarshal.Cast( + this.workspaceOwner.Memory.Span[..(Av1TransformWorkspace.MaximumLength * 2)]); + + ObuColorConfig colorConfig = this.sequenceHeader.ColorConfig; + Av1TransformType transformType; + Av1TransformSize transformSize; + int transformUnitCount; + bool hasChroma = Av1TileReader.HasChroma(this.sequenceHeader, modeInfoPosition, blockSize); + Av1PartitionInfo partitionInfo = new(modeInfo, superblockInfo, hasChroma, modeInfo.PartitionType) + { + ColumnIndex = modeInfoPosition.X, + RowIndex = modeInfoPosition.Y, + ChromaFromLumaContext = this.chromaFromLumaContext + }; + + partitionInfo.ComputeBoundaryOffsets(this.sequenceHeader, this.frameHeader, tileInfo); + + if (hasChroma) + { + // A one-unit luma edge maps to the same chroma sample as the adjacent unit on a subsampled axis. In that + // case the usable chroma neighbor is two mode-info units away rather than immediately above or left. + if (colorConfig.SubSamplingY && blockSize.Get4x4HighCount() == 1) + { + partitionInfo.AvailableAboveForChroma = modeInfoPosition.Y - 2 >= tileInfo.ModeInfoRowStart; + } + + if (colorConfig.SubSamplingX && blockSize.Get4x4WideCount() == 1) + { + partitionInfo.AvailableLeftForChroma = modeInfoPosition.X - 2 >= tileInfo.ModeInfoColumnStart; + } + } + + partitionInfo.PopulateModeInfoNeighbors(colorConfig); + + int maxBlocksWide = partitionInfo.GetMaxBlockWide(blockSize, false); + int maxBlocksHigh = partitionInfo.GetMaxBlockHigh(blockSize, false); + + bool isLossless = this.frameHeader.LosslessArray[modeInfo.SegmentId]; + bool isLosslessBlock = isLossless && ((blockSize >= Av1BlockSize.Block64x64) && (blockSize <= Av1BlockSize.Block128x128)); + int chromaTransformUnitCount = isLosslessBlock + ? (maxBlocksWide * maxBlocksHigh) >> ((colorConfig.SubSamplingX ? 1 : 0) + (colorConfig.SubSamplingY ? 1 : 0)) + : modeInfo.GetTransformUnitCount(Av1Plane.U); + + bool isInterBlock = modeInfo.ReferenceFrames[0] >= Av1ReferenceFrameType.Last; + InterReferenceBuffers? interReferenceBuffers = null; + bool isCompound = modeInfo.ReferenceFrames[1] > Av1ReferenceFrameType.Intra; + bool isInterIntra = modeInfo.ReferenceFrames[1] == Av1ReferenceFrameType.Intra; + int firstCompoundWeight = 8; + int secondCompoundWeight = 8; + if (isInterBlock) + { + Av1FrameBuffer primaryReferenceFrameBuffer = this.ResolveReferenceFrame(modeInfo.ReferenceFrames[0]); + Av1FrameBuffer secondaryReferenceFrameBuffer = primaryReferenceFrameBuffer; + if (isCompound) + { + secondaryReferenceFrameBuffer = this.ResolveReferenceFrame(modeInfo.ReferenceFrames[1]); + if (modeInfo.CompoundType == Av1CompoundType.DistanceWeighted) + { + Av1CompoundDistanceWeights.Derive( + this.sequenceHeader.OrderHintInfo, + this.frameHeader, + modeInfo.ReferenceFrames[0], + modeInfo.ReferenceFrames[1], + out firstCompoundWeight, + out secondCompoundWeight); + } + } + + // A non-compound block aliases the unused secondary slot to its required primary frame. This keeps the + // published inter state complete without manufacturing a nullable second half. + interReferenceBuffers = new(primaryReferenceFrameBuffer, secondaryReferenceFrameBuffer); + } + + bool highBitDepth = this.frameBuffer.BytesPerSample == 2; + for (int plane = 0; plane < colorConfig.PlaneCount; plane++) + { + int subX = (plane > 0) && colorConfig.SubSamplingX ? 1 : 0; + int subY = (plane > 0) && colorConfig.SubSamplingY ? 1 : 0; + + if (plane != 0 && !partitionInfo.IsChroma) + { + continue; + } + + // Luma transform descriptors occupy their own stream. U and V share one stream, with the V descriptors + // following the U descriptors for this block, so the V base includes the complete U transform-unit count. + int transformInfoIndex = plane switch + { + 2 => modeInfo.GetFirstTransformLocation(Av1Plane.V) + chromaTransformUnitCount, + 1 => modeInfo.GetFirstTransformLocation(Av1Plane.U), + 0 => modeInfo.GetFirstTransformLocation(Av1Plane.Y), + _ => throw new InvalidImageContentException("Maximum of 3 color planes") + }; + Span transformInfo = superblockInfo.GetTransformInfo(plane)[transformInfoIndex..]; + + if (isLosslessBlock) + { + Guard.IsTrue(transformInfo[0].Size == Av1TransformSize.Size4x4, nameof(transformInfo), "Lossless may only have 4x4 blocks."); + transformUnitCount = (maxBlocksWide * maxBlocksHigh) >> (subX + subY); + } + else + { + transformUnitCount = modeInfo.GetTransformUnitCount((Av1Plane)plane); + } + + Guard.IsFalse(transformUnitCount == 0, nameof(transformUnitCount), "Must have at least a single transform unit to decode."); + + Point pixelPosition = new( + (modeInfoPosition.X >> subX) << Av1Constants.ModeInfoSizeLog2, + (modeInfoPosition.Y >> subY) << Av1Constants.ModeInfoSizeLog2); + + Span blockReconstructionBuffer = default; + Span highBitDepthBlockReconstructionBuffer = default; + int reconstructionStride; + + // Prediction reads the row immediately above the destination through negative-relative neighbor offsets. + // The frame-buffer helpers therefore return a span beginning one logical sample row before the block. + if (highBitDepth) + { + highBitDepthBlockReconstructionBuffer = this.frameBuffer.DeriveBlockPointer16((Av1Plane)plane, pixelPosition, subX, subY, out reconstructionStride); + } + else + { + blockReconstructionBuffer = this.frameBuffer.DeriveBlockPointer((Av1Plane)plane, pixelPosition, subX, subY, out reconstructionStride); + } + + if (interReferenceBuffers is not null) + { + InterReferenceBuffers referenceBuffers = interReferenceBuffers.Value; + Av1FrameBuffer primaryReferenceFrameBuffer = referenceBuffers.Primary; + + int predictionWidth = Math.Max(4, blockSize.GetWidth() >> subX); + int predictionHeight = Math.Max(4, blockSize.GetHeight() >> subY); + int maximumBlockLength = 1 << this.sequenceHeader.SuperblockSizeLog2; + int maximumBlockArea = maximumBlockLength * maximumBlockLength; + int compoundMaskStorageLength = (maximumBlockArea + 1) >> 1; + Span predictionStorage = this.workspaceOwner.Memory.Span[this.predictionScratchOffset..]; + Span secondPredictionStorage = predictionStorage[..maximumBlockArea]; + Span firstCompoundPrediction = MemoryMarshal.Cast( + predictionStorage.Slice(maximumBlockArea, maximumBlockArea)); + + Span compoundMask = MemoryMarshal.AsBytes( + predictionStorage.Slice(2 * maximumBlockArea, compoundMaskStorageLength))[..(blockSize.GetWidth() * blockSize.GetHeight())]; + + Span predictionScratch = + predictionStorage.Slice( + (2 * maximumBlockArea) + compoundMaskStorageLength, + this.predictorWorkingLength); + + Span secondPrediction = MemoryMarshal.AsBytes(secondPredictionStorage)[..(predictionWidth * predictionHeight)]; + Span highBitDepthSecondPrediction = MemoryMarshal.Cast(secondPredictionStorage)[..(predictionWidth * predictionHeight)]; + bool usesSub8x8ChromaPrediction = + plane != 0 && + !isCompound && + this.TryPredictSub8x8Chroma( + ref partitionInfo, + modeInfoPosition, + blockSize, + plane, + subX, + subY, + pixelPosition, + predictionWidth, + predictionHeight, + blockReconstructionBuffer, + highBitDepthBlockReconstructionBuffer, + reconstructionStride, + predictionScratch); + + int referenceCount = usesSub8x8ChromaPrediction ? 0 : isCompound ? 2 : 1; + + // Every compound predictor is combined before its final rounding step. Warped prediction has its own + // convolution kernels, but the reference decoder writes their output into the same unsigned no-round domain. + bool useHighBitDepthCompoundIntermediates = + highBitDepth && + modeInfo.CompoundType is ( + Av1CompoundType.Average or + Av1CompoundType.DistanceWeighted or + Av1CompoundType.Wedge or + Av1CompoundType.DifferenceWeighted); + + bool useCompoundIntermediates = + isCompound && + (!highBitDepth || useHighBitDepthCompoundIntermediates); + + for (int referenceIndex = 0; referenceIndex < referenceCount; referenceIndex++) + { + Av1FrameBuffer activeReferenceFrameBuffer = referenceIndex == 0 + ? referenceBuffers.Primary + : referenceBuffers.Secondary; + + Av1MotionVector motionVector = modeInfo.MotionVectors[referenceIndex]; + int destinationStride = useCompoundIntermediates + ? predictionWidth + : referenceIndex == 0 ? reconstructionStride : predictionWidth; + bool isScaledReference = activeReferenceFrameBuffer.Width != this.frameHeader.FrameSize.FrameWidth || + activeReferenceFrameBuffer.Height != this.frameHeader.FrameSize.FrameHeight; + + // Warped prediction is selected per plane. In subsampled frames an otherwise qualifying 8x8 luma + // block has a 4x4 chroma prediction, which the reference decoder deliberately reconstructs with the translational + // center motion vector. Scaled references and integer-only frames exclude both local and global warp. + bool canUseWarpedPrediction = + !isScaledReference && + !this.frameHeader.ForceIntegerMotionVector && + predictionWidth >= 8 && + predictionHeight >= 8; + + Av1GlobalMotionParameters warpedMotionParameters = modeInfo.WarpedMotionParameters; + bool useWarpedPrediction = + canUseWarpedPrediction && + referenceIndex == 0 && + modeInfo.MotionMode == Av1MotionMode.Warped && + !warpedMotionParameters.IsInvalid; + + if (canUseWarpedPrediction && !useWarpedPrediction) + { + bool usesGlobalMotion = modeInfo.YMode == Av1PredictionMode.GlobalGlobalMotionVector || + (referenceIndex == 0 && modeInfo.YMode == Av1PredictionMode.GlobalMotionVector); + + if (usesGlobalMotion) + { + int canonicalReferenceIndex = + (int)modeInfo.ReferenceFrames[referenceIndex] - (int)Av1ReferenceFrameType.Last; + + Av1GlobalMotionParameters globalMotionParameters = + this.frameHeader.GetGlobalMotionParameters()[canonicalReferenceIndex]; + + // Identity and translation GLOBALMV modes use their derived center vector. Rotation/zoom and + // affine models use the complete matrix only when the decoded shear parameters are valid. + if (globalMotionParameters.Type > Av1GlobalMotionType.Translation && + !globalMotionParameters.IsInvalid) + { + warpedMotionParameters = globalMotionParameters; + useWarpedPrediction = true; + } + } + } + + if (useWarpedPrediction) + { + int referencePlaneWidth = Av1Math.DivideLog2Ceiling(activeReferenceFrameBuffer.Width, subX); + int referencePlaneHeight = Av1Math.DivideLog2Ceiling(activeReferenceFrameBuffer.Height, subY); + if (highBitDepth) + { + Span source = activeReferenceFrameBuffer.GetPaddedPlaneSpan16( + (Av1Plane)plane, + subX, + subY, + out int sourceStride, + out Point sourceOrigin); + + if (useCompoundIntermediates) + { + Span destination = referenceIndex == 0 + ? firstCompoundPrediction + : highBitDepthSecondPrediction; + + Av1WarpedInterPredictor.PredictWarpedCompound( + source, + sourceStride, + sourceOrigin, + referencePlaneWidth, + referencePlaneHeight, + destination, + destinationStride, + pixelPosition, + predictionWidth, + predictionHeight, + subX, + subY, + this.frameBuffer.BitDepth.GetBitCount(), + warpedMotionParameters, + predictionScratch); + } + else + { + Span destination = MemoryMarshal.Cast( + referenceIndex == 0 + ? highBitDepthBlockReconstructionBuffer[reconstructionStride..] + : secondPredictionStorage); + + Av1WarpedInterPredictor.PredictWarped( + source, + sourceStride, + sourceOrigin, + referencePlaneWidth, + referencePlaneHeight, + destination, + destinationStride, + pixelPosition, + predictionWidth, + predictionHeight, + subX, + subY, + this.frameBuffer.BitDepth.GetBitCount(), + warpedMotionParameters, + predictionScratch); + } + } + else + { + Span source = activeReferenceFrameBuffer.GetPaddedPlaneSpan( + (Av1Plane)plane, + subX, + subY, + out int sourceStride, + out Point sourceOrigin); + + if (useCompoundIntermediates) + { + Span destination = referenceIndex == 0 + ? firstCompoundPrediction + : highBitDepthSecondPrediction; + + Av1WarpedInterPredictor.PredictWarpedCompound( + source, + sourceStride, + sourceOrigin, + referencePlaneWidth, + referencePlaneHeight, + destination, + destinationStride, + pixelPosition, + predictionWidth, + predictionHeight, + subX, + subY, + warpedMotionParameters, + predictionScratch); + } + else + { + Span destination = referenceIndex == 0 + ? blockReconstructionBuffer[reconstructionStride..] + : secondPrediction; + + Av1WarpedInterPredictor.PredictWarped( + source, + sourceStride, + sourceOrigin, + referencePlaneWidth, + referencePlaneHeight, + destination, + destinationStride, + pixelPosition, + predictionWidth, + predictionHeight, + subX, + subY, + warpedMotionParameters, + predictionScratch); + } + } + + continue; + } + + if (isScaledReference) + { + Span scaledDestination = default; + Span scaledHighBitDepthDestination = default; + Span scaledCompoundDestination = default; + if (useCompoundIntermediates) + { + scaledCompoundDestination = referenceIndex == 0 + ? firstCompoundPrediction + : highBitDepthSecondPrediction; + } + else if (highBitDepth) + { + scaledHighBitDepthDestination = referenceIndex == 0 + ? MemoryMarshal.Cast(highBitDepthBlockReconstructionBuffer[reconstructionStride..]) + : highBitDepthSecondPrediction; + } + else + { + scaledDestination = referenceIndex == 0 + ? blockReconstructionBuffer[reconstructionStride..] + : secondPrediction; + } + + this.PredictScaledReference( + activeReferenceFrameBuffer, + motionVector, + plane, + subX, + subY, + pixelPosition, + predictionWidth, + predictionHeight, + modeInfo.InterpolationFilters[1], + modeInfo.InterpolationFilters[0], + scaledDestination, + scaledHighBitDepthDestination, + scaledCompoundDestination, + destinationStride, + predictionScratch); + + continue; + } + + // AV1 predicts the complete declared plane block even when its luma extent crosses the frame boundary. + // Subsampled dimensions retain the mandatory four-sample minimum used by set_plane_n4 in the reference decoder. + int horizontalMotionQ4 = motionVector.Column << (1 - subX); + int verticalMotionQ4 = motionVector.Row << (1 - subY); + int horizontalExtensionQ4 = (4 + predictionWidth) << 4; + int verticalExtensionQ4 = (4 + predictionHeight) << 4; + int horizontalEdgeScale = 1 << (1 - subX); + int verticalEdgeScale = 1 << (1 - subY); + + // The UMV clamp is expressed in one-sixteenth plane-sample units. A 128-sample block can legally + // address 135 samples beyond an edge once its prediction extent and eight-tap filter support are + // included; the frame-owned prediction border keeps that source directly addressable. + horizontalMotionQ4 = Av1Math.Clip3( + (partitionInfo.ModeBlockToLeftEdge * horizontalEdgeScale) - horizontalExtensionQ4, + (partitionInfo.ModeBlockToRightEdge * horizontalEdgeScale) + horizontalExtensionQ4 - 16, + horizontalMotionQ4); + + verticalMotionQ4 = Av1Math.Clip3( + (partitionInfo.ModeBlockToTopEdge * verticalEdgeScale) - verticalExtensionQ4, + (partitionInfo.ModeBlockToBottomEdge * verticalEdgeScale) + verticalExtensionQ4 - 16, + verticalMotionQ4); + + int sourceColumnQ4 = (pixelPosition.X << 4) + horizontalMotionQ4; + int sourceRowQ4 = (pixelPosition.Y << 4) + verticalMotionQ4; + + // Motion vectors use one-eighth luma-sample units. Shifting by one minus the plane subsampling converts + // them directly to the predictor's one-sixteenth-plane-sample phase; masking then preserves the signed + // floor used to select the integer source sample. + int horizontalPhase = sourceColumnQ4 & 15; + int verticalPhase = sourceRowQ4 & 15; + + if (highBitDepth) + { + Span source = activeReferenceFrameBuffer.GetPaddedPlaneSpan16( + (Av1Plane)plane, + subX, + subY, + out int sourceStride, + out Point sourceOrigin); + + int sourceIndex = + ((sourceOrigin.Y + (sourceRowQ4 >> 4)) * sourceStride) + sourceOrigin.X + (sourceColumnQ4 >> 4); + + if (useCompoundIntermediates) + { + Span destination = referenceIndex == 0 + ? firstCompoundPrediction + : highBitDepthSecondPrediction; + + Av1CompoundInterPredictor.PredictCompound( + source, + sourceStride, + sourceIndex, + destination, + predictionWidth, + predictionWidth, + predictionHeight, + modeInfo.InterpolationFilters[1], + modeInfo.InterpolationFilters[0], + horizontalPhase, + verticalPhase, + this.frameBuffer.BitDepth.GetBitCount(), + predictionScratch); + } + else + { + Span destination = referenceIndex == 0 + ? MemoryMarshal.Cast(highBitDepthBlockReconstructionBuffer[reconstructionStride..]) + : highBitDepthSecondPrediction; + + Av1TranslationalInterPredictor.Predict( + source, + sourceStride, + sourceIndex, + destination, + destinationStride, + predictionWidth, + predictionHeight, + modeInfo.InterpolationFilters[1], + modeInfo.InterpolationFilters[0], + horizontalPhase, + verticalPhase, + this.frameBuffer.BitDepth.GetBitCount(), + predictionScratch); + } + } + else + { + Span source = activeReferenceFrameBuffer.GetPaddedPlaneSpan( + (Av1Plane)plane, + subX, + subY, + out int sourceStride, + out Point sourceOrigin); + + int sourceIndex = + ((sourceOrigin.Y + (sourceRowQ4 >> 4)) * sourceStride) + sourceOrigin.X + (sourceColumnQ4 >> 4); + + if (useCompoundIntermediates) + { + Span destination = referenceIndex == 0 + ? firstCompoundPrediction + : highBitDepthSecondPrediction; + + Av1CompoundInterPredictor.PredictCompound( + source, + sourceStride, + sourceIndex, + destination, + predictionWidth, + predictionWidth, + predictionHeight, + modeInfo.InterpolationFilters[1], + modeInfo.InterpolationFilters[0], + horizontalPhase, + verticalPhase, + predictionScratch); + } + else + { + Span destination = referenceIndex == 0 + ? blockReconstructionBuffer[reconstructionStride..] + : secondPrediction; + + Av1TranslationalInterPredictor.Predict( + source, + sourceStride, + sourceIndex, + destination, + destinationStride, + predictionWidth, + predictionHeight, + modeInfo.InterpolationFilters[1], + modeInfo.InterpolationFilters[0], + horizontalPhase, + verticalPhase, + predictionScratch); + } + } + } + + if (isCompound) + { + if (useCompoundIntermediates) + { + ReadOnlySpan first = firstCompoundPrediction[..(predictionWidth * predictionHeight)]; + if (highBitDepth) + { + Span highBitDepthDestination = MemoryMarshal.Cast( + highBitDepthBlockReconstructionBuffer[reconstructionStride..]); + + if (modeInfo.CompoundType == Av1CompoundType.DistanceWeighted) + { + // Distance weighting must consume the no-round intermediates. Equal-averaging the + // already filtered references loses the decoded display-distance contribution. + Av1CompoundIntermediateDistanceWeightedPredictor.DistanceWeightedIntermediate( + highBitDepthDestination, + reconstructionStride, + first, + predictionWidth, + highBitDepthSecondPrediction, + predictionWidth, + predictionWidth, + predictionHeight, + firstCompoundWeight, + secondCompoundWeight, + this.frameBuffer.BitDepth.GetBitCount()); + } + else if (modeInfo.CompoundType == Av1CompoundType.Wedge) + { + Av1WedgeMask.Fill( + compoundMask, + predictionWidth, + blockSize, + modeInfo.CompoundWedgeIndex, + modeInfo.CompoundWedgeSign, + subX, + subY, + invert: false); + + // Masked compound prediction must blend the same no-round intermediates as the reference decoder's + // high-bit-depth d16 path so the mask is applied before the sole final rounding step. + Av1CompoundIntermediateMaskBlendPredictor.BlendIntermediate( + highBitDepthDestination, + reconstructionStride, + first, + predictionWidth, + highBitDepthSecondPrediction, + predictionWidth, + compoundMask, + predictionWidth, + predictionWidth, + predictionHeight, + subX: 0, + subY: 0, + this.frameBuffer.BitDepth.GetBitCount()); + } + else if (modeInfo.CompoundType == Av1CompoundType.DifferenceWeighted) + { + int lumaWidth = blockSize.GetWidth(); + if (plane == 0) + { + // Difference-weighted chroma reuses the luma-derived segment mask. Building it + // only for plane zero preserves that decoded contract before chroma subsampling. + Av1CompoundIntermediateDifferenceWeightedMaskBuilder.FillDifferenceWeightedIntermediateMask( + compoundMask, + lumaWidth, + first, + predictionWidth, + highBitDepthSecondPrediction, + predictionWidth, + predictionWidth, + predictionHeight, + this.frameBuffer.BitDepth.GetBitCount(), + modeInfo.DifferenceWeightedMaskType); + } + + // The d16 mask and final blend consume the same no-round intermediates. Rounding + // either reference first changes both the derived mask and the reconstructed sample. + Av1CompoundIntermediateMaskBlendPredictor.BlendIntermediate( + highBitDepthDestination, + reconstructionStride, + first, + predictionWidth, + highBitDepthSecondPrediction, + predictionWidth, + compoundMask, + lumaWidth, + predictionWidth, + predictionHeight, + subX, + subY, + this.frameBuffer.BitDepth.GetBitCount()); + } + else + { + Av1CompoundIntermediateAveragePredictor.AverageIntermediate( + highBitDepthDestination, + reconstructionStride, + first, + predictionWidth, + highBitDepthSecondPrediction, + predictionWidth, + predictionWidth, + predictionHeight, + this.frameBuffer.BitDepth.GetBitCount()); + } + } + else + { + Span destination = blockReconstructionBuffer[reconstructionStride..]; + switch (modeInfo.CompoundType) + { + case Av1CompoundType.Average: + Av1CompoundIntermediateAveragePredictor.AverageIntermediate( + destination, + reconstructionStride, + first, + predictionWidth, + highBitDepthSecondPrediction, + predictionWidth, + predictionWidth, + predictionHeight, + bitDepth: 8); + + break; + case Av1CompoundType.DistanceWeighted: + Av1CompoundIntermediateDistanceWeightedPredictor.DistanceWeightedIntermediate( + destination, + reconstructionStride, + first, + predictionWidth, + highBitDepthSecondPrediction, + predictionWidth, + predictionWidth, + predictionHeight, + firstCompoundWeight, + secondCompoundWeight, + bitDepth: 8); + + break; + case Av1CompoundType.Wedge: + Av1WedgeMask.Fill( + compoundMask, + predictionWidth, + blockSize, + modeInfo.CompoundWedgeIndex, + modeInfo.CompoundWedgeSign, + subX, + subY, + invert: false); + + Av1CompoundIntermediateMaskBlendPredictor.BlendIntermediate( + destination, + reconstructionStride, + first, + predictionWidth, + highBitDepthSecondPrediction, + predictionWidth, + compoundMask, + predictionWidth, + predictionWidth, + predictionHeight, + subX: 0, + subY: 0, + bitDepth: 8); + + break; + default: + int lumaWidth = blockSize.GetWidth(); + if (plane == 0) + { + Av1CompoundIntermediateDifferenceWeightedMaskBuilder.FillDifferenceWeightedIntermediateMask( + compoundMask, + lumaWidth, + first, + predictionWidth, + highBitDepthSecondPrediction, + predictionWidth, + predictionWidth, + predictionHeight, + bitDepth: 8, + modeInfo.DifferenceWeightedMaskType); + } + + Av1CompoundIntermediateMaskBlendPredictor.BlendIntermediate( + destination, + reconstructionStride, + first, + predictionWidth, + highBitDepthSecondPrediction, + predictionWidth, + compoundMask, + lumaWidth, + predictionWidth, + predictionHeight, + subX, + subY, + bitDepth: 8); + + break; + } + } + } + else if (highBitDepth) + { + Span destination = MemoryMarshal.Cast( + highBitDepthBlockReconstructionBuffer[reconstructionStride..]); + + switch (modeInfo.CompoundType) + { + case Av1CompoundType.Average: + Av1CompoundAveragePredictor.Average( + destination, + reconstructionStride, + highBitDepthSecondPrediction, + predictionWidth, + predictionWidth, + predictionHeight); + + break; + case Av1CompoundType.DistanceWeighted: + Av1CompoundDistanceWeightedPredictor.DistanceWeighted( + destination, + reconstructionStride, + highBitDepthSecondPrediction, + predictionWidth, + predictionWidth, + predictionHeight, + firstCompoundWeight, + secondCompoundWeight); + + break; + case Av1CompoundType.Wedge: + Av1WedgeMask.Fill( + compoundMask, + predictionWidth, + blockSize, + modeInfo.CompoundWedgeIndex, + modeInfo.CompoundWedgeSign, + subX, + subY, + invert: false); + + Av1CompoundMaskBlendPredictor.Blend( + destination, + reconstructionStride, + highBitDepthSecondPrediction, + predictionWidth, + compoundMask, + predictionWidth, + predictionWidth, + predictionHeight); + + break; + default: + Av1DifferenceWeightedMaskBuilder.FillDifferenceWeightedMask( + compoundMask, + predictionWidth, + destination, + reconstructionStride, + highBitDepthSecondPrediction, + predictionWidth, + predictionWidth, + predictionHeight, + this.frameBuffer.BitDepth.GetBitCount(), + modeInfo.DifferenceWeightedMaskType); + + Av1CompoundMaskBlendPredictor.Blend( + destination, + reconstructionStride, + highBitDepthSecondPrediction, + predictionWidth, + compoundMask, + predictionWidth, + predictionWidth, + predictionHeight); + + break; + } + } + else + { + Span destination = blockReconstructionBuffer[reconstructionStride..]; + switch (modeInfo.CompoundType) + { + case Av1CompoundType.Average: + Av1CompoundAveragePredictor.Average( + destination, + reconstructionStride, + secondPrediction, + predictionWidth, + predictionWidth, + predictionHeight); + + break; + case Av1CompoundType.DistanceWeighted: + Av1CompoundDistanceWeightedPredictor.DistanceWeighted( + destination, + reconstructionStride, + secondPrediction, + predictionWidth, + predictionWidth, + predictionHeight, + firstCompoundWeight, + secondCompoundWeight); + + break; + case Av1CompoundType.Wedge: + Av1WedgeMask.Fill( + compoundMask, + predictionWidth, + blockSize, + modeInfo.CompoundWedgeIndex, + modeInfo.CompoundWedgeSign, + subX, + subY, + invert: false); + + Av1CompoundMaskBlendPredictor.Blend( + destination, + reconstructionStride, + secondPrediction, + predictionWidth, + compoundMask, + predictionWidth, + predictionWidth, + predictionHeight); + + break; + default: + Av1DifferenceWeightedMaskBuilder.FillDifferenceWeightedMask( + compoundMask, + predictionWidth, + destination, + reconstructionStride, + secondPrediction, + predictionWidth, + predictionWidth, + predictionHeight, + modeInfo.DifferenceWeightedMaskType); + + Av1CompoundMaskBlendPredictor.Blend( + destination, + reconstructionStride, + secondPrediction, + predictionWidth, + compoundMask, + predictionWidth, + predictionWidth, + predictionHeight); + + break; + } + } + } + else if (isInterIntra) + { + if (highBitDepth) + { + this.predictionDecoder.DecodeInterIntra( + ref partitionInfo, + (Av1Plane)plane, + tileInfo, + highBitDepthBlockReconstructionBuffer, + reconstructionStride, + secondPredictionStorage[..(predictionWidth * predictionHeight)], + predictionWidth, + this.frameBuffer.BitDepth); + + if (modeInfo.UseInterIntraWedge) + { + Av1WedgeMask.Fill( + compoundMask, + predictionWidth, + blockSize, + modeInfo.InterIntraWedgeIndex, + wedgeSign: false, + subX, + subY, + invert: true); + } + else + { + Av1InterIntraMaskBuilder.FillInterIntraMask( + compoundMask, + predictionWidth, + predictionWidth, + predictionHeight, + modeInfo.InterIntraMode, + invert: true); + } + + Av1CompoundMaskBlendPredictor.Blend( + MemoryMarshal.Cast(highBitDepthBlockReconstructionBuffer[reconstructionStride..]), + reconstructionStride, + highBitDepthSecondPrediction, + predictionWidth, + compoundMask, + predictionWidth, + predictionWidth, + predictionHeight); + } + else + { + this.predictionDecoder.DecodeInterIntra( + ref partitionInfo, + (Av1Plane)plane, + tileInfo, + blockReconstructionBuffer, + reconstructionStride, + secondPrediction, + predictionWidth, + this.frameBuffer.BitDepth); + + if (modeInfo.UseInterIntraWedge) + { + Av1WedgeMask.Fill( + compoundMask, + predictionWidth, + blockSize, + modeInfo.InterIntraWedgeIndex, + wedgeSign: false, + subX, + subY, + invert: true); + } + else + { + Av1InterIntraMaskBuilder.FillInterIntraMask( + compoundMask, + predictionWidth, + predictionWidth, + predictionHeight, + modeInfo.InterIntraMode, + invert: true); + } + + Av1CompoundMaskBlendPredictor.Blend( + blockReconstructionBuffer[reconstructionStride..], + reconstructionStride, + secondPrediction, + predictionWidth, + compoundMask, + predictionWidth, + predictionWidth, + predictionHeight); + } + } + else if (modeInfo.MotionMode == Av1MotionMode.Obmc) + { + this.ApplyOverlappedMotionCompensation( + ref partitionInfo, + plane, + subX, + subY, + predictionWidth, + predictionHeight, + blockReconstructionBuffer, + highBitDepthBlockReconstructionBuffer, + reconstructionStride, + secondPrediction, + highBitDepthSecondPrediction, + compoundMask, + predictionScratch); + } + } + + for (int tu = 0; tu < transformUnitCount; tu++) + { + Span transformBlockReconstructionBuffer = default; + Span highBitDepthTransformBlockReconstructionBuffer = default; + int transformBlockOffset; + + transformSize = transformInfo[0].Size; + Span coefficients = superblockInfo.GetCoefficients((Av1Plane)plane)[this.currentCoefficientIndex[plane]..]; + + // Transform offsets are stored in mode-info units. Reconstruction strides are expressed in logical + // samples for both storage pipelines, so no byte scaling is applied to the high-bit-depth offset. + transformBlockOffset = ((transformInfo[0].OffsetY * reconstructionStride) + transformInfo[0].OffsetX) << Av1Constants.ModeInfoSizeLog2; + if (highBitDepth) + { + highBitDepthTransformBlockReconstructionBuffer = highBitDepthBlockReconstructionBuffer[transformBlockOffset..]; + } + else + { + transformBlockReconstructionBuffer = blockReconstructionBuffer[transformBlockOffset..]; + } + + if (this.isLoopFilterEnabled) + { + // U and V share transform geometry. Store the chroma map once so the later plane passes consume + // identical sizes without retaining duplicate state. + if (plane != 2) + { + Point transformPosition = new( + (modeInfoPosition.X >> subX) + transformInfo[0].OffsetX, + (modeInfoPosition.Y >> subY) + transformInfo[0].OffsetY); + + this.loopFilterContext.SetTransformSize((Av1Plane)plane, transformPosition, transformSize); + } + } + + // Intra-block copy is signaled on an intra-only frame but follows AV1's inter prediction and transform + // rules. Its validated displacement always references an earlier reconstructed region of this frame. + if (modeInfo.UseIntraBlockCopy) + { + // the reference decoder predicts the complete coding block before traversing its residual transforms. The mandatory + // 256-pixel source delay prevents overlap, and the two-tap interpolation is translation-invariant, + // so predicting the matching source rectangle for each transform unit produces the same samples. + Point transformPixelPosition = new( + pixelPosition.X + (transformInfo[0].OffsetX << Av1Constants.ModeInfoSizeLog2), + pixelPosition.Y + (transformInfo[0].OffsetY << Av1Constants.ModeInfoSizeLog2)); + + // Displacement vectors use one-eighth luma-sample units. Converting them to the plane's q4 grid + // leaves luma on an integer sample and can leave subsampled chroma exactly at phase eight. + int sourceColumnQ4 = (transformPixelPosition.X << 4) + + (modeInfo.DisplacementVector.Column << (1 - subX)); + + int sourceRowQ4 = (transformPixelPosition.Y << 4) + + (modeInfo.DisplacementVector.Row << (1 - subY)); + + int sourcePhaseX = sourceColumnQ4 & 15; + int sourcePhaseY = sourceRowQ4 & 15; + DebugGuard.IsTrue(sourcePhaseX is 0 or 8, "Intra-block-copy horizontal phase must be an integer or half sample."); + DebugGuard.IsTrue(sourcePhaseY is 0 or 8, "Intra-block-copy vertical phase must be an integer or half sample."); + + Point sourcePixelPosition = new(sourceColumnQ4 >> 4, sourceRowQ4 >> 4); + int transformWidth = transformSize.GetWidth(); + int transformHeight = transformSize.GetHeight(); + + if (highBitDepth) + { + Span source = this.frameBuffer.DeriveBlockPointer16( + (Av1Plane)plane, + sourcePixelPosition, + subX, + subY, + out int sourceStride); + + Av1IntraBlockCopyPredictor.Predict( + source[sourceStride..], + sourceStride, + highBitDepthTransformBlockReconstructionBuffer[reconstructionStride..], + reconstructionStride, + transformWidth, + transformHeight, + sourcePhaseX != 0, + sourcePhaseY != 0); + } + else + { + Span source = this.frameBuffer.DeriveBlockPointer( + (Av1Plane)plane, + sourcePixelPosition, + subX, + subY, + out int sourceStride); + + Av1IntraBlockCopyPredictor.Predict( + source[sourceStride..], + sourceStride, + transformBlockReconstructionBuffer[reconstructionStride..], + reconstructionStride, + transformWidth, + transformHeight, + sourcePhaseX != 0, + sourcePhaseY != 0); + } + } + else if (!isInterBlock) + { + // Conventional intra prediction consumes the reference-prefixed destination span before the + // transform residual is reconstructed over its first output row. + if (highBitDepth) + { + this.predictionDecoder.Decode( + ref partitionInfo, + (Av1Plane)plane, + transformSize, + tileInfo, + highBitDepthTransformBlockReconstructionBuffer, + reconstructionStride, + this.frameBuffer.BitDepth, + transformInfo[0].OffsetX, + transformInfo[0].OffsetY); + } + else + { + this.predictionDecoder.Decode( + ref partitionInfo, + (Av1Plane)plane, + transformSize, + tileInfo, + transformBlockReconstructionBuffer, + reconstructionStride, + this.frameBuffer.BitDepth, + transformInfo[0].OffsetX, + transformInfo[0].OffsetY); + } + } + + int endOfBlock = transformInfo[0].EndOfBlock; + if (endOfBlock != 0) + { + transformType = transformInfo[0].Type; + + // Entropy decoding has already applied quantization, scan placement, and coefficient clipping. + // Prediction includes a top-reference row; inverse reconstruction begins one stride after it. + if (highBitDepth) + { + Av1InverseTransformer.ReconstructHighBitDepth( + coefficients, + highBitDepthTransformBlockReconstructionBuffer[reconstructionStride..], + reconstructionStride, + transformSize, + transformType, + plane, + endOfBlock, + isLossless, + this.frameBuffer.BitDepth, + transformWorkspace); + } + else + { + Av1InverseTransformer.Reconstruct8Bit( + coefficients, + transformBlockReconstructionBuffer[reconstructionStride..], + reconstructionStride, + transformSize, + transformType, + plane, + endOfBlock, + isLossless, + transformWorkspace); + } + } + + this.currentCoefficientIndex[plane] += transformSize.GetWidth() * transformSize.GetHeight(); + + // Store Luma for CFL if required! + if (plane == (int)Av1Plane.Y && StoreChromaFromLumaRequired(colorConfig, ref partitionInfo)) + { + // The predictor span begins on the previous row; CFL storage consumes reconstructed samples from + // the transform block itself, hence the explicit one-stride advance for both sample pipelines. + if (highBitDepth) + { + this.chromaFromLumaContext.Store( + highBitDepthTransformBlockReconstructionBuffer[reconstructionStride..], + reconstructionStride, + transformInfo[0].OffsetY, + transformInfo[0].OffsetX, + transformSize, + blockSize, + modeInfoPosition.Y, + modeInfoPosition.X); + } + else + { + this.chromaFromLumaContext.Store( + transformBlockReconstructionBuffer[reconstructionStride..], + reconstructionStride, + transformInfo[0].OffsetY, + transformInfo[0].OffsetX, + transformSize, + blockSize, + modeInfoPosition.Y, + modeInfoPosition.X); + } + } + + // Transform descriptors and their coefficient regions follow the same per-plane traversal order. + transformInfo = transformInfo[1..]; + } + } + } + + /// + /// Reconstructs a subsampled chroma block assembled from multiple neighboring luma inter blocks. + /// + private bool TryPredictSub8x8Chroma( + ref Av1PartitionInfo partitionInfo, + Point modeInfoPosition, + Av1BlockSize blockSize, + int plane, + int subX, + int subY, + Point pixelPosition, + int predictionWidth, + int predictionHeight, + Span blockReconstructionBuffer, + Span highBitDepthBlockReconstructionBuffer, + int reconstructionStride, + Span predictionScratch) + { + bool isSub4X = blockSize.GetWidth() == 4 && subX != 0; + bool isSub4Y = blockSize.GetHeight() == 4 && subY != 0; + if (!isSub4X && !isSub4Y) + { + return false; + } + + int rowStart = isSub4Y ? -1 : 0; + int columnStart = isSub4X ? -1 : 0; + + // One chroma block can cover two or four independently decoded luma blocks. the reference decoder enters this path only + // when every contributing owner is a conventional inter block; otherwise the current block supplies the + // complete chroma prediction through the ordinary path. + for (int row = rowStart; row <= 0; row++) + { + for (int column = columnStart; column <= 0; column++) + { + Av1BlockModeInfo candidate = partitionInfo.SuperblockInfo.GetModeInfoAt( + new Point(modeInfoPosition.X + column, modeInfoPosition.Y + row)); + + if (candidate.ReferenceFrames[0] < Av1ReferenceFrameType.Last || candidate.UseIntraBlockCopy) + { + return false; + } + } + } + + int subPredictionWidth = blockSize.GetWidth() >> subX; + int subPredictionHeight = blockSize.GetHeight() >> subY; + int modeRow = rowStart; + + // Chroma ownership is assigned to the bottom-right luma mode record on each subsampled axis. Consequently + // pixelPosition is already the top-left of this assembled plane block even when its first luma owner is at + // row or column -1. Each subprediction writes directly into its final rectangle without a staging copy. + for (int y = 0; y < predictionHeight; y += subPredictionHeight) + { + int modeColumn = columnStart; + for (int x = 0; x < predictionWidth; x += subPredictionWidth) + { + Av1BlockModeInfo candidate = partitionInfo.SuperblockInfo.GetModeInfoAt( + new Point(modeInfoPosition.X + modeColumn, modeInfoPosition.Y + modeRow)); + + Av1FrameBuffer referenceFrameBuffer = this.ResolveReferenceFrame(candidate.ReferenceFrames[0]); + Av1MotionVector motionVector = candidate.MotionVectors[0]; + Point subPredictionOrigin = new(pixelPosition.X + x, pixelPosition.Y + y); + int destinationOffset = reconstructionStride + (y * reconstructionStride) + x; + bool isScaledReference = + referenceFrameBuffer.Width != this.frameHeader.FrameSize.FrameWidth || + referenceFrameBuffer.Height != this.frameHeader.FrameSize.FrameHeight; + + if (isScaledReference) + { + Span scaledDestination = default; + Span scaledHighBitDepthDestination = default; + if (this.frameBuffer.BytesPerSample == 2) + { + scaledHighBitDepthDestination = MemoryMarshal.Cast( + highBitDepthBlockReconstructionBuffer[destinationOffset..]); + } + else + { + scaledDestination = blockReconstructionBuffer[destinationOffset..]; + } + + this.PredictScaledReference( + referenceFrameBuffer, + motionVector, + plane, + subX, + subY, + subPredictionOrigin, + subPredictionWidth, + subPredictionHeight, + candidate.InterpolationFilters[1], + candidate.InterpolationFilters[0], + scaledDestination, + scaledHighBitDepthDestination, + default, + reconstructionStride, + predictionScratch); + } + else + { + int horizontalMotionQ4 = motionVector.Column << (1 - subX); + int verticalMotionQ4 = motionVector.Row << (1 - subY); + int horizontalExtensionQ4 = (4 + subPredictionWidth) << 4; + int verticalExtensionQ4 = (4 + subPredictionHeight) << 4; + int horizontalEdgeScale = 1 << (1 - subX); + int verticalEdgeScale = 1 << (1 - subY); + + // The block-relative UMV edges belong to the current coding block, while each contributing luma + // owner supplies only its motion vector and interpolation filters. This is the same split used by + // the reference decoder's sub-8x8 chroma builder. + horizontalMotionQ4 = Av1Math.Clip3( + (partitionInfo.ModeBlockToLeftEdge * horizontalEdgeScale) - horizontalExtensionQ4, + (partitionInfo.ModeBlockToRightEdge * horizontalEdgeScale) + horizontalExtensionQ4 - 16, + horizontalMotionQ4); + + verticalMotionQ4 = Av1Math.Clip3( + (partitionInfo.ModeBlockToTopEdge * verticalEdgeScale) - verticalExtensionQ4, + (partitionInfo.ModeBlockToBottomEdge * verticalEdgeScale) + verticalExtensionQ4 - 16, + verticalMotionQ4); + + int sourceColumnQ4 = (subPredictionOrigin.X << 4) + horizontalMotionQ4; + int sourceRowQ4 = (subPredictionOrigin.Y << 4) + verticalMotionQ4; + int horizontalPhase = sourceColumnQ4 & 15; + int verticalPhase = sourceRowQ4 & 15; + + if (this.frameBuffer.BytesPerSample == 2) + { + Span source = referenceFrameBuffer.GetPaddedPlaneSpan16( + (Av1Plane)plane, + subX, + subY, + out int sourceStride, + out Point sourceOrigin); + + int sourceIndex = + ((sourceOrigin.Y + (sourceRowQ4 >> 4)) * sourceStride) + + sourceOrigin.X + + (sourceColumnQ4 >> 4); + + Span destination = MemoryMarshal.Cast( + highBitDepthBlockReconstructionBuffer[destinationOffset..]); + + Av1TranslationalInterPredictor.Predict( + source, + sourceStride, + sourceIndex, + destination, + reconstructionStride, + subPredictionWidth, + subPredictionHeight, + candidate.InterpolationFilters[1], + candidate.InterpolationFilters[0], + horizontalPhase, + verticalPhase, + this.frameBuffer.BitDepth.GetBitCount(), + predictionScratch); + } + else + { + Span source = referenceFrameBuffer.GetPaddedPlaneSpan( + (Av1Plane)plane, + subX, + subY, + out int sourceStride, + out Point sourceOrigin); + + int sourceIndex = + ((sourceOrigin.Y + (sourceRowQ4 >> 4)) * sourceStride) + + sourceOrigin.X + + (sourceColumnQ4 >> 4); + + Av1TranslationalInterPredictor.Predict( + source, + sourceStride, + sourceIndex, + blockReconstructionBuffer[destinationOffset..], + reconstructionStride, + subPredictionWidth, + subPredictionHeight, + candidate.InterpolationFilters[1], + candidate.InterpolationFilters[0], + horizontalPhase, + verticalPhase, + predictionScratch); + } + } + + modeColumn++; + } + + modeRow++; + } + + return true; + } + + /// + /// Resolves one canonical retained reference frame. + /// + private Av1FrameBuffer ResolveReferenceFrame(Av1ReferenceFrameType referenceFrame) + { + int canonicalReferenceIndex = (int)referenceFrame - (int)Av1ReferenceFrameType.Last; + uint referenceSlot = this.frameHeader.GetReferenceFrameIndices()[canonicalReferenceIndex]; + + // The uncompressed-header parser validates each selected slot and the reference store remains unchanged + // until frame reconstruction completes, so every parsed inter block resolves the same retained owner. + return this.referenceFrames.ResolveRequired((int)referenceSlot).FrameBuffer; + } + + /// + /// Predicts one block from a retained reference whose visible dimensions differ from the current coded frame. + /// + private void PredictScaledReference( + Av1FrameBuffer referenceFrameBuffer, + Av1MotionVector motionVector, + int plane, + int subX, + int subY, + Point predictionOrigin, + int predictionWidth, + int predictionHeight, + Av1InterpolationFilter horizontalFilter, + Av1InterpolationFilter verticalFilter, + Span destination, + Span highBitDepthDestination, + Span compoundDestination, + int destinationStride, + Span predictionScratch) + { + Av1ReferenceScale scale = new( + referenceFrameBuffer.Width, + referenceFrameBuffer.Height, + this.frameHeader.FrameSize.FrameWidth, + this.frameHeader.FrameSize.FrameHeight); + + int currentColumnQ4 = (predictionOrigin.X << 4) + (motionVector.Column << (1 - subX)); + int currentRowQ4 = (predictionOrigin.Y << 4) + (motionVector.Row << (1 - subY)); + int sourceColumnQ10 = scale.ScaleHorizontal(currentColumnQ4) + Av1ReferenceScale.ExtraOffset; + int sourceRowQ10 = scale.ScaleVertical(currentRowQ4) + Av1ReferenceScale.ExtraOffset; + int referencePlaneWidth = Av1Math.DivideLog2Ceiling(referenceFrameBuffer.Width, subX); + int referencePlaneHeight = Av1Math.DivideLog2Ceiling(referenceFrameBuffer.Height, subY); + int horizontalMargin = (Av1FrameBuffer.DecoderPaddingValue >> subX) - 4; + int verticalMargin = (Av1FrameBuffer.DecoderPaddingValue >> subY) - 4; + + // The scaled coordinate clamp is intentionally wider than the ordinary block-relative UMV clamp. The retained + // frame owns the normative border, so every variable-phase eight-tap source remains directly addressable. + sourceColumnQ10 = Av1Math.Clip3( + -horizontalMargin << Av1ReferenceScale.SubpixelBits, + (referencePlaneWidth + 4) << Av1ReferenceScale.SubpixelBits, + sourceColumnQ10); + + sourceRowQ10 = Av1Math.Clip3( + -verticalMargin << Av1ReferenceScale.SubpixelBits, + (referencePlaneHeight + 4) << Av1ReferenceScale.SubpixelBits, + sourceRowQ10); + + int horizontalPhase = sourceColumnQ10 & Av1ReferenceScale.SubpixelMask; + int verticalPhase = sourceRowQ10 & Av1ReferenceScale.SubpixelMask; + if (this.frameBuffer.BytesPerSample == 2) + { + Span source = referenceFrameBuffer.GetPaddedPlaneSpan16( + (Av1Plane)plane, + subX, + subY, + out int sourceStride, + out Point sourceOrigin); + + int sourceIndex = + ((sourceOrigin.Y + (sourceRowQ10 >> Av1ReferenceScale.SubpixelBits)) * sourceStride) + + sourceOrigin.X + + (sourceColumnQ10 >> Av1ReferenceScale.SubpixelBits); + + if (compoundDestination.IsEmpty) + { + Av1ScaledInterPredictor.PredictScaled( + source, + sourceStride, + sourceIndex, + highBitDepthDestination, + destinationStride, + predictionWidth, + predictionHeight, + horizontalFilter, + verticalFilter, + horizontalPhase, + scale.HorizontalStep, + verticalPhase, + scale.VerticalStep, + this.frameBuffer.BitDepth.GetBitCount(), + predictionScratch); + } + else + { + Av1ScaledInterPredictor.PredictScaledCompound( + source, + sourceStride, + sourceIndex, + compoundDestination, + destinationStride, + predictionWidth, + predictionHeight, + horizontalFilter, + verticalFilter, + horizontalPhase, + scale.HorizontalStep, + verticalPhase, + scale.VerticalStep, + this.frameBuffer.BitDepth.GetBitCount(), + predictionScratch); + } + } + else + { + Span source = referenceFrameBuffer.GetPaddedPlaneSpan( + (Av1Plane)plane, + subX, + subY, + out int sourceStride, + out Point sourceOrigin); + + int sourceIndex = + ((sourceOrigin.Y + (sourceRowQ10 >> Av1ReferenceScale.SubpixelBits)) * sourceStride) + + sourceOrigin.X + + (sourceColumnQ10 >> Av1ReferenceScale.SubpixelBits); + + if (compoundDestination.IsEmpty) + { + Av1ScaledInterPredictor.PredictScaled( + source, + sourceStride, + sourceIndex, + destination, + destinationStride, + predictionWidth, + predictionHeight, + horizontalFilter, + verticalFilter, + horizontalPhase, + scale.HorizontalStep, + verticalPhase, + scale.VerticalStep, + predictionScratch); + } + else + { + Av1ScaledInterPredictor.PredictScaledCompound( + source, + sourceStride, + sourceIndex, + compoundDestination, + destinationStride, + predictionWidth, + predictionHeight, + horizontalFilter, + verticalFilter, + horizontalPhase, + scale.HorizontalStep, + verticalPhase, + scale.VerticalStep, + predictionScratch); + } + } + } + + /// + /// Blends predictions from eligible above and left neighbors into one regular inter prediction. + /// + private void ApplyOverlappedMotionCompensation( + ref Av1PartitionInfo partitionInfo, + int plane, + int subX, + int subY, + int predictionWidth, + int predictionHeight, + Span blockReconstructionBuffer, + Span highBitDepthBlockReconstructionBuffer, + int reconstructionStride, + Span neighborPrediction, + Span highBitDepthNeighborPrediction, + Span maskStorage, + Span predictionScratch) + { + Av1BlockSize blockSize = partitionInfo.ModeInfo.BlockSize; + int blockWidthInModeInfoUnits = blockSize.Get4x4WideCount(); + int blockHeightInModeInfoUnits = blockSize.Get4x4HighCount(); + int blockColumn = partitionInfo.ColumnIndex; + int blockRow = partitionInfo.RowIndex; + bool highBitDepth = this.frameBuffer.BytesPerSample == 2; + + // Chroma planes smaller than 8x8 use left overlap only. This is the AV1 bandwidth rule for 4x4, + // 8x4, and 4x8 plane blocks; luma cannot reach those sizes when motion variation is selectable. + bool skipAbove = (predictionWidth == 4 && predictionHeight <= 8) || + (predictionWidth == 8 && predictionHeight == 4); + + if (partitionInfo.AvailableAbove && !skipAbove) + { + int maximumNeighbors = Math.Min(4, blockSize.Get4x4WidthLog2()); + int endColumn = Math.Min(blockColumn + blockWidthInModeInfoUnits, this.frameHeader.ModeInfoColumnCount); + int neighborCount = 0; + for (int aboveColumn = blockColumn; aboveColumn < endColumn && neighborCount < maximumNeighbors;) + { + Av1BlockModeInfo candidate = partitionInfo.SuperblockInfo.GetModeInfoAt(new Point(aboveColumn, blockRow - 1)); + int step = Math.Min(candidate.BlockSize.Get4x4WideCount(), Av1BlockSize.Block64x64.Get4x4WideCount()); + if (step == 1) + { + // A four-sample neighbor is one half of the chroma-bearing eight-sample pair. the reference decoder aligns + // the traversal to the pair start and reads prediction state from its second mode record. + aboveColumn &= ~1; + candidate = partitionInfo.SuperblockInfo.GetModeInfoAt(new Point(aboveColumn + 1, blockRow - 1)); + step = 2; + } + + if (IsOverlappable(candidate)) + { + int relativeColumn = aboveColumn - blockColumn; + int neighborWidthInModeInfoUnits = Math.Min(blockWidthInModeInfoUnits, step); + int neighborWidth = (neighborWidthInModeInfoUnits << Av1Constants.ModeInfoSizeLog2) >> subX; + int neighborHeight = Math.Clamp( + blockSize.GetHeight() >> (subY + 1), + 4, + Av1BlockSize.Block64x64.GetHeight() >> (subY + 1)); + + Point predictionOrigin = new( + (aboveColumn << Av1Constants.ModeInfoSizeLog2) >> subX, + (blockRow << Av1Constants.ModeInfoSizeLog2) >> subY); + + this.PredictObmcNeighbor( + candidate, + plane, + subX, + subY, + predictionOrigin, + neighborWidth, + neighborHeight, + neighborPrediction, + highBitDepthNeighborPrediction, + predictionScratch); + + int overlapHeight = (Math.Min(blockSize.GetHeight(), Av1BlockSize.Block64x64.GetHeight()) >> 1) >> subY; + int destinationColumn = (relativeColumn << Av1Constants.ModeInfoSizeLog2) >> subX; + ReadOnlySpan verticalMask = Av1ObmcMask.Get(overlapHeight); + Span expandedMask = maskStorage[..(neighborWidth * overlapHeight)]; + for (int row = 0; row < overlapHeight; row++) + { + // The vertical mask has one alpha per row. Expanding it into the reusable scratch plane lets + // the existing SIMD masked blender process complete rows without a specialized duplicate path. + expandedMask.Slice(row * neighborWidth, neighborWidth).Fill(verticalMask[row]); + } + + if (highBitDepth) + { + Av1CompoundMaskBlendPredictor.Blend( + MemoryMarshal.Cast(highBitDepthBlockReconstructionBuffer[reconstructionStride..])[destinationColumn..], + reconstructionStride, + highBitDepthNeighborPrediction, + neighborWidth, + expandedMask, + neighborWidth, + neighborWidth, + overlapHeight); + } + else + { + Av1CompoundMaskBlendPredictor.Blend( + blockReconstructionBuffer[reconstructionStride..][destinationColumn..], + reconstructionStride, + neighborPrediction, + neighborWidth, + expandedMask, + neighborWidth, + neighborWidth, + overlapHeight); + } + + neighborCount++; + } + + aboveColumn += step; + } + } + + if (partitionInfo.AvailableLeft) + { + int maximumNeighbors = Math.Min(4, blockSize.Get4x4HeightLog2()); + int endRow = Math.Min(blockRow + blockHeightInModeInfoUnits, this.frameHeader.ModeInfoRowCount); + int neighborCount = 0; + for (int leftRow = blockRow; leftRow < endRow && neighborCount < maximumNeighbors;) + { + Av1BlockModeInfo candidate = partitionInfo.SuperblockInfo.GetModeInfoAt(new Point(blockColumn - 1, leftRow)); + int step = Math.Min(candidate.BlockSize.Get4x4HighCount(), Av1BlockSize.Block64x64.Get4x4HighCount()); + if (step == 1) + { + // The vertical traversal applies the corresponding pairing rule to four-sample-high blocks. + leftRow &= ~1; + candidate = partitionInfo.SuperblockInfo.GetModeInfoAt(new Point(blockColumn - 1, leftRow + 1)); + step = 2; + } + + if (IsOverlappable(candidate)) + { + int relativeRow = leftRow - blockRow; + int neighborHeightInModeInfoUnits = Math.Min(blockHeightInModeInfoUnits, step); + int neighborWidth = Math.Clamp( + blockSize.GetWidth() >> (subX + 1), + 4, + Av1BlockSize.Block64x64.GetWidth() >> (subX + 1)); + + int neighborHeight = (neighborHeightInModeInfoUnits << Av1Constants.ModeInfoSizeLog2) >> subY; + Point predictionOrigin = new( + (blockColumn << Av1Constants.ModeInfoSizeLog2) >> subX, + (leftRow << Av1Constants.ModeInfoSizeLog2) >> subY); + + this.PredictObmcNeighbor( + candidate, + plane, + subX, + subY, + predictionOrigin, + neighborWidth, + neighborHeight, + neighborPrediction, + highBitDepthNeighborPrediction, + predictionScratch); + + int overlapWidth = (Math.Min(blockSize.GetWidth(), Av1BlockSize.Block64x64.GetWidth()) >> 1) >> subX; + int destinationRow = (relativeRow << Av1Constants.ModeInfoSizeLog2) >> subY; + ReadOnlySpan horizontalMask = Av1ObmcMask.Get(overlapWidth); + if (highBitDepth) + { + Av1CompoundMaskBlendPredictor.Blend( + MemoryMarshal.Cast(highBitDepthBlockReconstructionBuffer[reconstructionStride..])[(destinationRow * reconstructionStride)..], + reconstructionStride, + highBitDepthNeighborPrediction, + neighborWidth, + horizontalMask, + 0, + overlapWidth, + neighborHeight); + } + else + { + Av1CompoundMaskBlendPredictor.Blend( + blockReconstructionBuffer[reconstructionStride..][(destinationRow * reconstructionStride)..], + reconstructionStride, + neighborPrediction, + neighborWidth, + horizontalMask, + 0, + overlapWidth, + neighborHeight); + } + + neighborCount++; + } + + leftRow += step; + } + } + } + + /// + /// Builds one neighboring block's primary translational predictor into the reusable OBMC workspace. + /// + private void PredictObmcNeighbor( + Av1BlockModeInfo neighbor, + int plane, + int subX, + int subY, + Point predictionOrigin, + int predictionWidth, + int predictionHeight, + Span destination, + Span highBitDepthDestination, + Span predictionScratch) + { + Av1FrameBuffer referenceFrameBuffer = this.ResolveReferenceFrame(neighbor.ReferenceFrames[0]); + Av1MotionVector motionVector = neighbor.MotionVectors[0]; + bool isScaledReference = referenceFrameBuffer.Width != this.frameHeader.FrameSize.FrameWidth || + referenceFrameBuffer.Height != this.frameHeader.FrameSize.FrameHeight; + + if (isScaledReference) + { + this.PredictScaledReference( + referenceFrameBuffer, + motionVector, + plane, + subX, + subY, + predictionOrigin, + predictionWidth, + predictionHeight, + neighbor.InterpolationFilters[1], + neighbor.InterpolationFilters[0], + destination, + highBitDepthDestination, + default, + predictionWidth, + predictionScratch); + + return; + } + + int sourceColumnQ4 = (predictionOrigin.X << 4) + (motionVector.Column << (1 - subX)); + int sourceRowQ4 = (predictionOrigin.Y << 4) + (motionVector.Row << (1 - subY)); + int horizontalExtensionQ4 = (4 + predictionWidth) << 4; + int verticalExtensionQ4 = (4 + predictionHeight) << 4; + int framePlaneWidth = (this.frameHeader.ModeInfoColumnCount << Av1Constants.ModeInfoSizeLog2) >> subX; + int framePlaneHeight = (this.frameHeader.ModeInfoRowCount << Av1Constants.ModeInfoSizeLog2) >> subY; + + // the reference decoder clamps the motion vector relative to each neighbor rectangle. Once the neighbor origin is added, the + // prediction extent remains in the left/top limit but cancels from the right/bottom limit. Keeping this + // asymmetry avoids counting the OBMC rectangle twice when the source lies beyond the far frame edge. + sourceColumnQ4 = Av1Math.Clip3( + -horizontalExtensionQ4, + ((framePlaneWidth + 4) << 4) - 16, + sourceColumnQ4); + + sourceRowQ4 = Av1Math.Clip3( + -verticalExtensionQ4, + ((framePlaneHeight + 4) << 4) - 16, + sourceRowQ4); + + int horizontalPhase = sourceColumnQ4 & 15; + int verticalPhase = sourceRowQ4 & 15; + if (this.frameBuffer.BytesPerSample == 2) + { + Span source = referenceFrameBuffer.GetPaddedPlaneSpan16( + (Av1Plane)plane, + subX, + subY, + out int sourceStride, + out Point sourceOrigin); + + int sourceIndex = + ((sourceOrigin.Y + (sourceRowQ4 >> 4)) * sourceStride) + sourceOrigin.X + (sourceColumnQ4 >> 4); + + Av1TranslationalInterPredictor.Predict( + source, + sourceStride, + sourceIndex, + highBitDepthDestination, + predictionWidth, + predictionWidth, + predictionHeight, + neighbor.InterpolationFilters[1], + neighbor.InterpolationFilters[0], + horizontalPhase, + verticalPhase, + this.frameBuffer.BitDepth.GetBitCount(), + predictionScratch); + } + else + { + Span source = referenceFrameBuffer.GetPaddedPlaneSpan( + (Av1Plane)plane, + subX, + subY, + out int sourceStride, + out Point sourceOrigin); + + int sourceIndex = + ((sourceOrigin.Y + (sourceRowQ4 >> 4)) * sourceStride) + sourceOrigin.X + (sourceColumnQ4 >> 4); + + Av1TranslationalInterPredictor.Predict( + source, + sourceStride, + sourceIndex, + destination, + predictionWidth, + predictionWidth, + predictionHeight, + neighbor.InterpolationFilters[1], + neighbor.InterpolationFilters[0], + horizontalPhase, + verticalPhase, + predictionScratch); + } + } + + /// + /// Determines whether a decoded neighbor supplies an inter predictor for OBMC. + /// + private static bool IsOverlappable(Av1BlockModeInfo candidate) + => candidate.UseIntraBlockCopy || candidate.ReferenceFrames[0] > Av1ReferenceFrameType.Intra; + + /// + /// Determines whether reconstructed luma samples must be retained for a later chroma-from-luma prediction. + /// + /// The sequence color-plane configuration. + /// The current block and its prediction modes. + /// + /// when chroma is present and the current luma block can contribute to a chroma-from-luma block. + /// + private static bool StoreChromaFromLumaRequired(ObuColorConfig colorConfig, ref Av1PartitionInfo partitionInfo) + => !colorConfig.IsMonochrome && + (!partitionInfo.IsChroma || partitionInfo.ModeInfo.UvMode == Av1ChromaPredictionMode.ChromaFromLuma); + + /// + /// Carries a complete pair of retained buffers through the inter-only reconstruction branch. + /// + private readonly struct InterReferenceBuffers + { + /// + /// Initializes a new instance of the struct. + /// + public InterReferenceBuffers(Av1FrameBuffer primary, Av1FrameBuffer secondary) + { + this.Primary = primary; + this.Secondary = secondary; + } + + /// + /// Gets the primary retained frame. + /// + public Av1FrameBuffer Primary { get; } + + /// + /// Gets the secondary retained frame. + /// + public Av1FrameBuffer Secondary { get; } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Transform/Av1ForwardTransformer.cs b/src/ImageSharp/Formats/Heif/Av1/Transform/Av1ForwardTransformer.cs new file mode 100644 index 0000000000..14671c8cba --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Transform/Av1ForwardTransformer.cs @@ -0,0 +1,1216 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.InteropServices; +using System.Runtime.Intrinsics; +using System.Runtime.Intrinsics.X86; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform.Forward; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +/// +/// Converts spatial residual samples into AV1 transform coefficients. +/// +/// +/// The SIMD pipeline transposes rows into lanes before invoking the one-dimensional operators. One vector then holds +/// the same transform position from several independent axes, allowing the complete stage network to run lane-wise. +/// Eight-bit blocks use saturating 16-bit stages where their normative ranges permit it; high-bit-depth and scalar +/// fallback paths retain 32-bit stages. Both representations produce the same row-major coefficient contract. +/// +internal static partial class Av1ForwardTransformer +{ + /// + /// Resolves and applies the configured two-dimensional AV1 forward transform. + /// + /// The spatial residual samples. + /// The destination transform coefficients. + /// The number of input samples between rows. + /// The compound transform type. + /// The transform-block dimensions. + /// The source sample bit depth. + /// The reusable workspace owned by the containing encode operation. + public static void Transform2d( + ReadOnlySpan input, + Span coefficients, + uint stride, + Av1TransformType transformType, + Av1TransformSize transformSize, + int bitDepth, + Span workspace) + { + Av1Transform2dFlipConfiguration config = Av1Transform2dFlipConfiguration.CreateForward(transformType, transformSize, bitDepth); + Guard.MustBeSizedAtLeast(workspace, Av1TransformWorkspace.GetRequiredLength(transformSize), nameof(workspace)); + + DispatchColumn(input, coefficients, stride, bitDepth, ref config, workspace); + } + + /// + /// Applies the reversible four-by-four transform required by coded-lossless AV1 blocks. + /// + /// The spatial residual samples. + /// The destination transform coefficients. + /// The number of input samples between rows. + public static void TransformLossless4x4(ReadOnlySpan input, Span coefficients, uint stride) + { + int inputStride = (int)stride; + + if (Vector128.IsHardwareAccelerated) + { + Vector128 row0 = Vector128.Create((int)input[0], input[1], input[2], input[3]); + Vector128 row1 = Vector128.Create( + input[inputStride], + input[inputStride + 1], + input[inputStride + 2], + input[inputStride + 3]); + + Vector128 row2 = Vector128.Create( + input[2 * inputStride], + input[(2 * inputStride) + 1], + input[(2 * inputStride) + 2], + input[(2 * inputStride) + 3]); + + Vector128 row3 = Vector128.Create( + input[3 * inputStride], + input[(3 * inputStride) + 1], + input[(3 * inputStride) + 2], + input[(3 * inputStride) + 3]); + + // Each lane initially holds one column. The first stage transforms those columns in parallel, then the + // transpose makes each transformed column a row so the same lane-wise network can process the other axis. + TransformLosslessStage(ref row0, ref row1, ref row2, ref row3); + Av1Transform2dOperations.Transpose(ref row0, ref row1, ref row2, ref row3); + TransformLosslessStage(ref row0, ref row1, ref row2, ref row3); + + // The entropy pipeline stores transform positions in row-major order, while the reversible reference + // walk leaves the two frequency axes exchanged. Normalize that boundary before scan-order traversal. + Av1Transform2dOperations.Transpose(ref row0, ref row1, ref row2, ref row3); + + ref int coefficientBase = ref MemoryMarshal.GetReference(coefficients); + (row0 * 4).StoreUnsafe(ref coefficientBase); + (row1 * 4).StoreUnsafe(ref coefficientBase, 4); + (row2 * 4).StoreUnsafe(ref coefficientBase, 8); + (row3 * 4).StoreUnsafe(ref coefficientBase, 12); + return; + } + + // The first pass writes transposed columns into the destination, matching the layout consumed in-place by + // the second pass. This keeps the scalar fallback allocation-free without a temporary matrix. + for (int column = 0; column < 4; column++) + { + int a = input[column]; + int b = input[inputStride + column]; + int c = input[(2 * inputStride) + column]; + int d = input[(3 * inputStride) + column]; + + a += b; + d -= c; + int e = (a - d) >> 1; + b = e - b; + c = e - c; + a -= c; + d += b; + + int offset = column * 4; + coefficients[offset] = a; + coefficients[offset + 1] = c; + coefficients[offset + 2] = d; + coefficients[offset + 3] = b; + } + + for (int column = 0; column < 4; column++) + { + int a = coefficients[column]; + int b = coefficients[4 + column]; + int c = coefficients[8 + column]; + int d = coefficients[12 + column]; + + a += b; + d -= c; + int e = (a - d) >> 1; + b = e - b; + c = e - c; + a -= c; + d += b; + + coefficients[column] = a * 4; + coefficients[4 + column] = c * 4; + coefficients[8 + column] = d * 4; + coefficients[12 + column] = b * 4; + } + + // Normalize the scalar reference walk to the row-major coefficient contract used by entropy coding. + (coefficients[1], coefficients[4]) = (coefficients[4], coefficients[1]); + (coefficients[2], coefficients[8]) = (coefficients[8], coefficients[2]); + (coefficients[3], coefficients[12]) = (coefficients[12], coefficients[3]); + (coefficients[6], coefficients[9]) = (coefficients[9], coefficients[6]); + (coefficients[7], coefficients[13]) = (coefficients[13], coefficients[7]); + (coefficients[11], coefficients[14]) = (coefficients[14], coefficients[11]); + } + + /// + /// Applies one axis of the reversible four-point transform to four independent SIMD lanes. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static void TransformLosslessStage( + ref Vector128 row0, + ref Vector128 row1, + ref Vector128 row2, + ref Vector128 row3) + { + Vector128 a = row0 + row1; + Vector128 d = row3 - row2; + Vector128 e = (a - d) >> 1; + Vector128 b = e - row1; + Vector128 c = e - row2; + a -= c; + d += b; + + row0 = a; + row1 = c; + row2 = d; + row3 = b; + } + + /// + /// Selects the concrete column operator for a transform block. + /// + /// The spatial residual samples. + /// The destination transform coefficients. + /// The number of input samples between rows. + /// The source sample bit depth. + /// The resolved transform functions, shifts, and axis orientation. + /// The reusable transform workspace. + private static void DispatchColumn( + ReadOnlySpan input, + Span coefficients, + uint stride, + int bitDepth, + ref Av1Transform2dFlipConfiguration config, + Span workspace) + { + switch (config.TransformFunctionTypeColumn) + { + case Av1TransformFunctionType.Dct4: + DispatchRow(input, coefficients, stride, bitDepth, ref config, workspace); + break; + case Av1TransformFunctionType.Dct8: + DispatchRow(input, coefficients, stride, bitDepth, ref config, workspace); + break; + case Av1TransformFunctionType.Dct16: + DispatchRow(input, coefficients, stride, bitDepth, ref config, workspace); + break; + case Av1TransformFunctionType.Dct32: + DispatchRow(input, coefficients, stride, bitDepth, ref config, workspace); + break; + case Av1TransformFunctionType.Dct64: + DispatchRow(input, coefficients, stride, bitDepth, ref config, workspace); + break; + case Av1TransformFunctionType.Adst4: + DispatchRow(input, coefficients, stride, bitDepth, ref config, workspace); + break; + case Av1TransformFunctionType.Adst8: + DispatchRow(input, coefficients, stride, bitDepth, ref config, workspace); + break; + case Av1TransformFunctionType.Adst16: + DispatchRow(input, coefficients, stride, bitDepth, ref config, workspace); + break; + case Av1TransformFunctionType.Identity4: + DispatchRow(input, coefficients, stride, bitDepth, ref config, workspace); + break; + case Av1TransformFunctionType.Identity8: + DispatchRow(input, coefficients, stride, bitDepth, ref config, workspace); + break; + case Av1TransformFunctionType.Identity16: + DispatchRow(input, coefficients, stride, bitDepth, ref config, workspace); + break; + case Av1TransformFunctionType.Identity32: + DispatchRow(input, coefficients, stride, bitDepth, ref config, workspace); + break; + default: + throw new InvalidImageContentException($"The {config.TransformFunctionTypeColumn} column transform is not valid for {config.TransformSize}."); + } + } + + /// + /// Selects the concrete row operator after the column operator has been specialized. + /// + /// The column transform operator selected for the block. + /// The spatial residual samples. + /// The destination transform coefficients. + /// The number of input samples between rows. + /// The source sample bit depth. + /// The resolved transform functions, shifts, and axis orientation. + /// The reusable transform workspace. + private static void DispatchRow( + ReadOnlySpan input, + Span coefficients, + uint stride, + int bitDepth, + ref Av1Transform2dFlipConfiguration config, + Span workspace) + where TColumnOperator : struct, IAv1ForwardTransform1dOperator + { + switch (config.TransformFunctionTypeRow) + { + case Av1TransformFunctionType.Dct4: + Transform2d(input, coefficients, stride, bitDepth, ref config, workspace); + break; + case Av1TransformFunctionType.Dct8: + Transform2d(input, coefficients, stride, bitDepth, ref config, workspace); + break; + case Av1TransformFunctionType.Dct16: + Transform2d(input, coefficients, stride, bitDepth, ref config, workspace); + break; + case Av1TransformFunctionType.Dct32: + Transform2d(input, coefficients, stride, bitDepth, ref config, workspace); + break; + case Av1TransformFunctionType.Dct64: + Transform2d(input, coefficients, stride, bitDepth, ref config, workspace); + break; + case Av1TransformFunctionType.Adst4: + Transform2d(input, coefficients, stride, bitDepth, ref config, workspace); + break; + case Av1TransformFunctionType.Adst8: + Transform2d(input, coefficients, stride, bitDepth, ref config, workspace); + break; + case Av1TransformFunctionType.Adst16: + Transform2d(input, coefficients, stride, bitDepth, ref config, workspace); + break; + case Av1TransformFunctionType.Identity4: + Transform2d(input, coefficients, stride, bitDepth, ref config, workspace); + break; + case Av1TransformFunctionType.Identity8: + Transform2d(input, coefficients, stride, bitDepth, ref config, workspace); + break; + case Av1TransformFunctionType.Identity16: + Transform2d(input, coefficients, stride, bitDepth, ref config, workspace); + break; + case Av1TransformFunctionType.Identity32: + Transform2d(input, coefficients, stride, bitDepth, ref config, workspace); + break; + default: + throw new InvalidImageContentException($"The {config.TransformFunctionTypeRow} row transform is not valid for {config.TransformSize}."); + } + } + + /// + /// Applies the specialized operator pair using the sample representation selected for the coded bit depth. + /// + /// The column transform operator. + /// The row transform operator. + /// The spatial residual samples. + /// The destination transform coefficients. + /// The number of input samples between rows. + /// The source sample bit depth. + /// The resolved transform functions, shifts, and axis orientation. + /// The reusable transform workspace. + private static void Transform2d( + ReadOnlySpan input, + Span coefficients, + uint stride, + int bitDepth, + ref Av1Transform2dFlipConfiguration config, + Span workspace) + where TColumnOperator : struct, IAv1ForwardTransform1dOperator + where TRowOperator : struct, IAv1ForwardTransform1dOperator + { + // Highway keeps eight-bit transform stages in Int16 lanes and promotes only the large rectangular layouts. + // The independent scalar reference uses Int32, so hardware without packed Int16 support follows that exact fallback instead. + if (bitDepth == 8 && Vector128.IsHardwareAccelerated) + { + TransformPacked(input, coefficients, stride, ref config, workspace); + return; + } + + TransformExpanded(input, coefficients, stride, ref config, workspace); + } + + /// + /// Applies the signed Int16 stage pipeline used for eight-bit residuals. + /// + /// The column transform operator. + /// The row transform operator. + /// The spatial residual samples. + /// The destination transform coefficients. + /// The number of input samples between rows. + /// The resolved transform functions, shifts, and axis orientation. + /// The reusable transform workspace. + private static void TransformPacked( + ReadOnlySpan input, + Span coefficients, + uint stride, + ref Av1Transform2dFlipConfiguration config, + Span workspace) + where TColumnOperator : struct, IAv1ForwardTransform1dOperator + where TRowOperator : struct, IAv1ForwardTransform1dOperator + { + int width = config.TransformSize.GetWidth(); + int height = config.TransformSize.GetHeight(); + + // Packed short stages halve the arithmetic width and AVX-512BW doubles their lane count. Highway selects + // this representation by ISA capability, independently of the runtime preference used for generic vectors. + int blockLaneCount = Avx512BW.IsSupported + ? Vector512.Count + : Avx2.IsSupported ? Vector256.Count : Vector128.Count; + + int blockWidth = Math.Max(width, blockLaneCount); + int blockHeight = Math.Max(height, blockLaneCount); + int blockArea = blockWidth * blockHeight; + int packedBlockLength = (blockArea + 1) / 2; + bool promote = (blockWidth == 64 && blockHeight >= 32) || (blockWidth >= 32 && blockHeight == 64); + int dataOffset = Av1TransformWorkspace.Vector512StorageLength; + Span buffer0 = MemoryMarshal.Cast(workspace.Slice(dataOffset, packedBlockLength)); + ref short buffer0Base = ref MemoryMarshal.GetReference(buffer0); + + LoadPacked(input, stride, ref buffer0Base, blockWidth, width, height, config.Shift0, config.FlipUpsideDown, config.FlipLeftToRight); + TransformPackedAxis(buffer0, width, blockWidth, blockWidth, config.CosBitColumn, workspace); + + int retainedHeight = Math.Min(height, 32); + int retainedWidth = Math.Min(width, 32); + bool normalizeRectangle = Math.Abs(config.TransformSize.GetRectangleLogRatio()) == 1; + + if (promote) + { + Span buffer1 = workspace.Slice(dataOffset + packedBlockLength, blockArea); + int scratchOffset = dataOffset + packedBlockLength + blockArea; + Span scratch = workspace.Slice(scratchOffset); + ref int buffer1Base = ref MemoryMarshal.GetReference(buffer1); + + TransposeAndPromote( + ref buffer0Base, + blockWidth, + ref buffer1Base, + blockHeight, + width, + height, + -config.Shift1, + scratch); + + int rowOutputStride = width == 64 && height == 64 ? 32 : blockHeight; + + TransformExpandedAxis(buffer1, retainedHeight, blockHeight, rowOutputStride, config.CosBitRow, workspace); + ref int coefficientBase = ref MemoryMarshal.GetReference(coefficients); + + TransposeExpanded( + ref buffer1Base, + rowOutputStride, + ref coefficientBase, + retainedWidth, + retainedHeight, + retainedWidth, + -config.Shift2, + normalizeRectangle, + scratch); + + return; + } + + Span buffer1Packed = MemoryMarshal.Cast(workspace.Slice(dataOffset + packedBlockLength, packedBlockLength)); + int packedScratchOffset = dataOffset + (2 * packedBlockLength); + Span packedScratch = workspace.Slice(packedScratchOffset); + ref short buffer1PackedBase = ref MemoryMarshal.GetReference(buffer1Packed); + + TransposePacked( + ref buffer0Base, + blockWidth, + ref buffer1PackedBase, + blockHeight, + width, + height, + -config.Shift1, + false, + packedScratch); + + TransformPackedAxis(buffer1Packed, height, blockHeight, blockHeight, config.CosBitRow, workspace); + + // The second transform produces horizontal frequency in rows and vertical frequency in lanes. Transposing + // once more adapts the reference decoder's native layout to the row-major coefficient contract used by ImageSharp. + TransposePacked( + ref buffer1PackedBase, + blockHeight, + ref buffer0Base, + retainedWidth, + retainedHeight, + retainedWidth, + -config.Shift2, + normalizeRectangle, + packedScratch); + + StorePacked(ref buffer0Base, retainedWidth, retainedHeight, coefficients); + } + + /// + /// Applies the signed Int32 stage pipeline used for high-bit-depth residuals and scalar fallback. + /// + /// The column transform operator. + /// The row transform operator. + /// The spatial residual samples. + /// The destination transform coefficients. + /// The number of input samples between rows. + /// The resolved transform functions, shifts, and axis orientation. + /// The reusable transform workspace. + private static void TransformExpanded( + ReadOnlySpan input, + Span coefficients, + uint stride, + ref Av1Transform2dFlipConfiguration config, + Span workspace) + where TColumnOperator : struct, IAv1ForwardTransform1dOperator + where TRowOperator : struct, IAv1ForwardTransform1dOperator + { + int width = config.TransformSize.GetWidth(); + int height = config.TransformSize.GetHeight(); + int blockLaneCount = Vector512.IsHardwareAccelerated + ? Vector512.Count + : Vector256.IsHardwareAccelerated ? Vector256.Count : Vector128.IsHardwareAccelerated ? Vector128.Count : 1; + + int blockWidth = Math.Max(width, blockLaneCount); + int blockHeight = Math.Max(height, blockLaneCount); + int blockArea = blockWidth * blockHeight; + int dataOffset = Av1TransformWorkspace.Vector512StorageLength; + Span buffer0 = workspace.Slice(dataOffset, blockArea); + Span buffer1 = workspace.Slice(dataOffset + blockArea, blockArea); + Span scratch = workspace.Slice(dataOffset + (2 * blockArea)); + ref int buffer0Base = ref MemoryMarshal.GetReference(buffer0); + ref int buffer1Base = ref MemoryMarshal.GetReference(buffer1); + + LoadExpanded(input, stride, ref buffer0Base, blockWidth, width, height, config.Shift0, config.FlipUpsideDown, config.FlipLeftToRight); + TransformExpandedAxis(buffer0, width, blockWidth, blockWidth, config.CosBitColumn, workspace); + + TransposeExpanded( + ref buffer0Base, + blockWidth, + ref buffer1Base, + blockHeight, + width, + height, + -config.Shift1, + false, + scratch); + + int retainedHeight = Math.Min(height, 32); + int retainedWidth = Math.Min(width, 32); + int rowOutputStride = width == 64 && height == 64 ? 32 : blockHeight; + + TransformExpandedAxis(buffer1, retainedHeight, blockHeight, rowOutputStride, config.CosBitRow, workspace); + bool normalizeRectangle = Math.Abs(config.TransformSize.GetRectangleLogRatio()) == 1; + ref int coefficientBase = ref MemoryMarshal.GetReference(coefficients); + + TransposeExpanded( + ref buffer1Base, + rowOutputStride, + ref coefficientBase, + retainedWidth, + retainedHeight, + retainedWidth, + -config.Shift2, + normalizeRectangle, + scratch); + } + + /// + /// Loads, flips, and scales one eight-bit residual block into packed transform storage. + /// + /// The spatial residual samples. + /// The number of input samples between rows. + /// The first value in the packed transform block. + /// The number of packed values between destination rows. + /// The transform-block width. + /// The transform-block height. + /// The initial transform scaling shift. + /// Whether to reverse the input row order. + /// Whether to reverse the samples within each row. + private static void LoadPacked( + ReadOnlySpan input, + uint inputStride, + ref short destination, + int destinationStride, + int width, + int height, + int shift, + bool flipUpsideDown, + bool flipLeftToRight) + { + ref short inputBase = ref MemoryMarshal.GetReference(input); + + for (int row = 0; row < height; row++) + { + int sourceRow = flipUpsideDown ? height - row - 1 : row; + ref short source = ref Unsafe.Add(ref inputBase, sourceRow * (int)inputStride); + ref short target = ref Unsafe.Add(ref destination, row * destinationStride); + + if (Avx512BW.IsSupported && width >= Vector512.Count) + { + for (int column = 0; column < width; column += Vector512.Count) + { + int sourceColumn = flipLeftToRight ? width - column - Vector512.Count : column; + Vector512 value = Vector512.LoadUnsafe(ref source, (nuint)sourceColumn); + value = flipLeftToRight ? Av1Transform2dOperations.Reverse(value) : value; + Av1Transform2dOperations.RoundShift(value, -shift).StoreUnsafe(ref target, (nuint)column); + } + + continue; + } + + if (Avx2.IsSupported && width >= Vector256.Count) + { + for (int column = 0; column < width; column += Vector256.Count) + { + int sourceColumn = flipLeftToRight ? width - column - Vector256.Count : column; + Vector256 value = Vector256.LoadUnsafe(ref source, (nuint)sourceColumn); + value = flipLeftToRight ? Av1Transform2dOperations.Reverse(value) : value; + Av1Transform2dOperations.RoundShift(value, -shift).StoreUnsafe(ref target, (nuint)column); + } + + continue; + } + + if (width >= Vector128.Count) + { + for (int column = 0; column < width; column += Vector128.Count) + { + int sourceColumn = flipLeftToRight ? width - column - Vector128.Count : column; + Vector128 value = Vector128.LoadUnsafe(ref source, (nuint)sourceColumn); + value = flipLeftToRight ? Av1Transform2dOperations.Reverse(value) : value; + Av1Transform2dOperations.RoundShift(value, -shift).StoreUnsafe(ref target, (nuint)column); + } + + continue; + } + + // Four-point transforms occupy the lower half of the padded Vector128 row. Loading each source value + // explicitly avoids reading beyond a caller row whose stride is exactly four samples. + for (int column = 0; column < width; column++) + { + int sourceColumn = flipLeftToRight ? width - column - 1 : column; + target = (short)(Unsafe.Add(ref source, sourceColumn) << shift); + target = ref Unsafe.Add(ref target, 1); + } + } + } + + /// + /// Loads, flips, widens, and scales one residual block into signed thirty-two-bit transform storage. + /// + /// The spatial residual samples. + /// The number of input samples between rows. + /// The first value in the expanded transform block. + /// The number of expanded values between destination rows. + /// The transform-block width. + /// The transform-block height. + /// The initial transform scaling shift. + /// Whether to reverse the input row order. + /// Whether to reverse the samples within each row. + private static void LoadExpanded( + ReadOnlySpan input, + uint inputStride, + ref int destination, + int destinationStride, + int width, + int height, + int shift, + bool flipUpsideDown, + bool flipLeftToRight) + { + ref short inputBase = ref MemoryMarshal.GetReference(input); + + for (int row = 0; row < height; row++) + { + int sourceRow = flipUpsideDown ? height - row - 1 : row; + ref short source = ref Unsafe.Add(ref inputBase, sourceRow * (int)inputStride); + ref int target = ref Unsafe.Add(ref destination, row * destinationStride); + + if (Vector512.IsHardwareAccelerated && width >= Vector512.Count) + { + for (int column = 0; column < width; column += Vector512.Count) + { + int sourceColumn = flipLeftToRight ? width - column - Vector512.Count : column; + Vector512 value = Av1Transform2dOperations.Load16Int16(ref Unsafe.Add(ref source, sourceColumn)); + value = flipLeftToRight ? Av1Transform2dOperations.Reverse(value) : value; + Av1Transform2dOperations.RoundShift(value, -shift).StoreUnsafe(ref target, (nuint)column); + } + + continue; + } + + if (Vector256.IsHardwareAccelerated && width >= Vector256.Count) + { + for (int column = 0; column < width; column += Vector256.Count) + { + int sourceColumn = flipLeftToRight ? width - column - Vector256.Count : column; + Vector256 value = Av1Transform2dOperations.Load8Int16(ref Unsafe.Add(ref source, sourceColumn)); + value = flipLeftToRight ? Av1Transform2dOperations.Reverse(value) : value; + Av1Transform2dOperations.RoundShift(value, -shift).StoreUnsafe(ref target, (nuint)column); + } + + continue; + } + + if (Vector128.IsHardwareAccelerated) + { + for (int column = 0; column < width; column += Vector128.Count) + { + int sourceColumn = flipLeftToRight ? width - column - Vector128.Count : column; + Vector128 value = Av1Transform2dOperations.Load4Int16(ref Unsafe.Add(ref source, sourceColumn)); + value = flipLeftToRight ? Av1Transform2dOperations.Reverse(value) : value; + Av1Transform2dOperations.RoundShift(value, -shift).StoreUnsafe(ref target, (nuint)column); + } + + continue; + } + + for (int column = 0; column < width; column++) + { + int sourceColumn = flipLeftToRight ? width - column - 1 : column; + target = Unsafe.Add(ref source, sourceColumn) << shift; + target = ref Unsafe.Add(ref target, 1); + } + } + } + + /// + /// Applies one packed transform axis using the widest efficient lane count available for the block. + /// + /// The semantic transform operator. + /// The packed transform block. + /// The number of independent axes. + /// The number of packed values between input positions. + /// The number of packed values between output positions. + /// The fixed-point precision of the cosine constants. + /// The reusable transform-stage workspace. + private static void TransformPackedAxis( + Span buffer, + int transformCount, + int inputStride, + int outputStride, + int cosBit, + Span workspace) + where TOperator : struct, IAv1ForwardTransform1dOperator + { + if (Avx512BW.IsSupported && transformCount >= Vector512.Count) + { + TransformPackedVector512(buffer, transformCount, inputStride, outputStride, cosBit, workspace); + return; + } + + if (Avx2.IsSupported && transformCount >= Vector256.Count) + { + TransformPackedVector256(buffer, transformCount, inputStride, outputStride, cosBit, workspace); + return; + } + + TransformPackedVector128(buffer, transformCount, inputStride, outputStride, cosBit, workspace); + } + + /// + /// Applies one expanded transform axis using the widest efficient lane count available for the block. + /// + /// The semantic transform operator. + /// The expanded transform block. + /// The number of independent axes. + /// The number of expanded values between input positions. + /// The number of expanded values between output positions. + /// The fixed-point precision of the cosine constants. + /// The reusable transform-stage workspace. + private static void TransformExpandedAxis( + Span buffer, + int transformCount, + int inputStride, + int outputStride, + int cosBit, + Span workspace) + where TOperator : struct, IAv1ForwardTransform1dOperator + { + if (Vector512.IsHardwareAccelerated && transformCount >= Vector512.Count) + { + TransformExpandedVector512(buffer, transformCount, inputStride, outputStride, cosBit, workspace); + return; + } + + if (Vector256.IsHardwareAccelerated && transformCount >= Vector256.Count) + { + TransformExpandedVector256(buffer, transformCount, inputStride, outputStride, cosBit, workspace); + return; + } + + if (Vector128.IsHardwareAccelerated && transformCount >= Vector128.Count) + { + TransformExpandedVector128(buffer, transformCount, inputStride, outputStride, cosBit, workspace); + return; + } + + TransformExpandedScalar(buffer, transformCount, inputStride, outputStride, cosBit, workspace); + } + + /// + /// Applies a packed transform to thirty-two independent axes. + /// + private static void TransformPackedVector512( + Span buffer, + int transformCount, + int inputStride, + int outputStride, + int cosBit, + Span workspace) + where TOperator : struct, IAv1ForwardTransform1dOperator + { + ref Av1TransformVector> buffer0 = + ref Unsafe.As>>(ref MemoryMarshal.GetReference(workspace)); + ref Av1TransformVector> buffer1 = + ref Unsafe.Add(ref buffer0, 1); + ref short source = ref MemoryMarshal.GetReference(buffer); + nint inputByteStride = inputStride * sizeof(short); + nint outputByteStride = outputStride * sizeof(short); + + for (int batch = 0; batch < transformCount; batch += Vector512.Count) + { + ref byte values = ref Unsafe.As(ref Unsafe.Add(ref source, batch)); + + TOperator.Transform(ref values, inputByteStride, outputByteStride, ref buffer0, ref buffer1, cosBit); + } + } + + /// + /// Applies a packed transform to sixteen independent axes. + /// + private static void TransformPackedVector256( + Span buffer, + int transformCount, + int inputStride, + int outputStride, + int cosBit, + Span workspace) + where TOperator : struct, IAv1ForwardTransform1dOperator + { + ref Av1TransformVector> buffer0 = + ref Unsafe.As>>(ref MemoryMarshal.GetReference(workspace)); + ref Av1TransformVector> buffer1 = + ref Unsafe.Add(ref buffer0, 1); + ref short source = ref MemoryMarshal.GetReference(buffer); + nint inputByteStride = inputStride * sizeof(short); + nint outputByteStride = outputStride * sizeof(short); + + for (int batch = 0; batch < transformCount; batch += Vector256.Count) + { + ref byte values = ref Unsafe.As(ref Unsafe.Add(ref source, batch)); + + TOperator.Transform(ref values, inputByteStride, outputByteStride, ref buffer0, ref buffer1, cosBit); + } + } + + /// + /// Applies a packed transform to eight independent axes. + /// + private static void TransformPackedVector128( + Span buffer, + int transformCount, + int inputStride, + int outputStride, + int cosBit, + Span workspace) + where TOperator : struct, IAv1ForwardTransform1dOperator + { + ref Av1TransformVector> buffer0 = + ref Unsafe.As>>(ref MemoryMarshal.GetReference(workspace)); + ref Av1TransformVector> buffer1 = + ref Unsafe.Add(ref buffer0, 1); + ref short source = ref MemoryMarshal.GetReference(buffer); + nint inputByteStride = inputStride * sizeof(short); + nint outputByteStride = outputStride * sizeof(short); + + for (int batch = 0; batch < transformCount; batch += Vector128.Count) + { + ref byte values = ref Unsafe.As(ref Unsafe.Add(ref source, batch)); + + TOperator.Transform(ref values, inputByteStride, outputByteStride, ref buffer0, ref buffer1, cosBit); + } + } + + /// + /// Applies an expanded transform to sixteen independent axes. + /// + private static void TransformExpandedVector512( + Span buffer, + int transformCount, + int inputStride, + int outputStride, + int cosBit, + Span workspace) + where TOperator : struct, IAv1ForwardTransform1dOperator + { + ref Av1TransformVector> buffer0 = + ref Unsafe.As>>(ref MemoryMarshal.GetReference(workspace)); + ref Av1TransformVector> buffer1 = + ref Unsafe.Add(ref buffer0, 1); + ref int source = ref MemoryMarshal.GetReference(buffer); + nint inputByteStride = inputStride * sizeof(int); + nint outputByteStride = outputStride * sizeof(int); + + for (int batch = 0; batch < transformCount; batch += Vector512.Count) + { + ref byte values = ref Unsafe.As(ref Unsafe.Add(ref source, batch)); + + TOperator.Transform(ref values, inputByteStride, outputByteStride, ref buffer0, ref buffer1, cosBit); + } + } + + /// + /// Applies an expanded transform to eight independent axes. + /// + private static void TransformExpandedVector256( + Span buffer, + int transformCount, + int inputStride, + int outputStride, + int cosBit, + Span workspace) + where TOperator : struct, IAv1ForwardTransform1dOperator + { + ref Av1TransformVector> buffer0 = + ref Unsafe.As>>(ref MemoryMarshal.GetReference(workspace)); + ref Av1TransformVector> buffer1 = + ref Unsafe.Add(ref buffer0, 1); + ref int source = ref MemoryMarshal.GetReference(buffer); + nint inputByteStride = inputStride * sizeof(int); + nint outputByteStride = outputStride * sizeof(int); + + for (int batch = 0; batch < transformCount; batch += Vector256.Count) + { + ref byte values = ref Unsafe.As(ref Unsafe.Add(ref source, batch)); + + TOperator.Transform(ref values, inputByteStride, outputByteStride, ref buffer0, ref buffer1, cosBit); + } + } + + /// + /// Applies an expanded transform to four independent axes. + /// + private static void TransformExpandedVector128( + Span buffer, + int transformCount, + int inputStride, + int outputStride, + int cosBit, + Span workspace) + where TOperator : struct, IAv1ForwardTransform1dOperator + { + ref Av1TransformVector> buffer0 = + ref Unsafe.As>>(ref MemoryMarshal.GetReference(workspace)); + ref Av1TransformVector> buffer1 = + ref Unsafe.Add(ref buffer0, 1); + ref int source = ref MemoryMarshal.GetReference(buffer); + nint inputByteStride = inputStride * sizeof(int); + nint outputByteStride = outputStride * sizeof(int); + + for (int batch = 0; batch < transformCount; batch += Vector128.Count) + { + ref byte values = ref Unsafe.As(ref Unsafe.Add(ref source, batch)); + + TOperator.Transform(ref values, inputByteStride, outputByteStride, ref buffer0, ref buffer1, cosBit); + } + } + + /// + /// Applies an expanded transform to one axis. + /// + private static void TransformExpandedScalar( + Span buffer, + int transformCount, + int inputStride, + int outputStride, + int cosBit, + Span workspace) + where TOperator : struct, IAv1ForwardTransform1dOperator + { + ref Av1TransformVector buffer0 = + ref Unsafe.As>(ref MemoryMarshal.GetReference(workspace)); + ref Av1TransformVector buffer1 = + ref Unsafe.Add(ref buffer0, 1); + ref int source = ref MemoryMarshal.GetReference(buffer); + nint inputByteStride = inputStride * sizeof(int); + nint outputByteStride = outputStride * sizeof(int); + + for (int batch = 0; batch < transformCount; batch++) + { + ref byte values = ref Unsafe.As(ref Unsafe.Add(ref source, batch)); + + TOperator.Transform(ref values, inputByteStride, outputByteStride, ref buffer0, ref buffer1, cosBit); + } + } + + /// + /// Transposes packed transform storage while applying an AV1 pipeline shift and optional rectangle scaling. + /// + /// The first value in the source block. + /// The number of packed values between source rows. + /// The first value in the destination block. + /// The number of packed values between destination rows. + /// The number of source columns. + /// The number of source rows. + /// The signed AV1 scaling shift. + /// Whether to apply the square-root-of-two rectangle normalization. + /// The reusable transpose workspace. + private static void TransposePacked( + ref short source, + int sourceStride, + ref short destination, + int destinationStride, + int sourceWidth, + int sourceHeight, + int roundShift, + bool normalizeRectangle, + Span scratch) + { + int tileSize = Avx2.IsSupported && Math.Min(sourceWidth, sourceHeight) >= 16 ? 16 : Math.Min(sourceWidth, sourceHeight) >= 8 ? 8 : 4; + Span transposeScratch = MemoryMarshal.Cast(scratch); + + for (int row = 0; row < sourceHeight; row += tileSize) + { + for (int column = 0; column < sourceWidth; column += tileSize) + { + ref short tileSource = ref Unsafe.Add(ref source, (row * sourceStride) + column); + ref short tileDestination = ref Unsafe.Add(ref destination, (column * destinationStride) + row); + + if (tileSize == 16) + { + Av1Transform2dOperations.Transpose16x16Int16( + ref tileSource, + sourceStride, + ref tileDestination, + destinationStride, + transposeScratch, + roundShift, + normalizeRectangle); + } + else if (tileSize == 8) + { + Av1Transform2dOperations.Transpose8x8Int16( + ref tileSource, + sourceStride, + ref tileDestination, + destinationStride, + roundShift, + normalizeRectangle); + } + else + { + Av1Transform2dOperations.Transpose4x4Int16( + ref tileSource, + sourceStride, + ref tileDestination, + destinationStride, + roundShift, + normalizeRectangle); + } + } + } + } + + /// + /// Promotes and transposes the large packed layouts at the same axis boundary as the reference decoder. + /// + /// The first packed value in the source block. + /// The number of packed values between source rows. + /// The first expanded value in the destination block. + /// The number of expanded values between destination rows. + /// The number of source columns. + /// The number of source rows. + /// The signed AV1 scaling shift. + /// The reusable conversion and transpose workspace. + private static void TransposeAndPromote( + ref short source, + int sourceStride, + ref int destination, + int destinationStride, + int sourceWidth, + int sourceHeight, + int roundShift, + Span scratch) + { + int tileSize = Avx512BW.IsSupported ? 16 : Vector256.IsHardwareAccelerated ? 8 : 4; + + for (int row = 0; row < sourceHeight; row += tileSize) + { + for (int column = 0; column < sourceWidth; column += tileSize) + { + ref short tileSource = ref Unsafe.Add(ref source, (row * sourceStride) + column); + ref int tileDestination = ref Unsafe.Add(ref destination, (column * destinationStride) + row); + + if (tileSize == 16) + { + Av1Transform2dOperations.Transpose16x16Int16ToInt32( + ref tileSource, + sourceStride, + ref tileDestination, + destinationStride, + scratch[..256], + MemoryMarshal.Cast(scratch[256..]), + roundShift, + false); + } + else if (tileSize == 8) + { + Av1Transform2dOperations.Transpose8x8Int16ToInt32( + ref tileSource, + sourceStride, + ref tileDestination, + destinationStride, + scratch[..64], + roundShift, + false); + } + else + { + Av1Transform2dOperations.Transpose4x4Int16ToInt32( + ref tileSource, + sourceStride, + ref tileDestination, + destinationStride, + scratch[..16], + roundShift, + false); + } + } + } + } + + /// + /// Transposes signed thirty-two-bit transform storage while applying the configured terminal operations. + /// + /// The first value in the source block. + /// The number of expanded values between source rows. + /// The first value in the destination block. + /// The number of expanded values between destination rows. + /// The number of source columns. + /// The number of source rows. + /// The signed AV1 scaling shift. + /// Whether to apply the square-root-of-two rectangle normalization. + /// The reusable transpose workspace. + private static void TransposeExpanded( + ref int source, + int sourceStride, + ref int destination, + int destinationStride, + int sourceWidth, + int sourceHeight, + int roundShift, + bool normalizeRectangle, + Span scratch) + { + bool useVector512 = Vector512.IsHardwareAccelerated && Math.Min(sourceWidth, sourceHeight) >= 16; + int tileSize = useVector512 + ? 16 + : Vector256.IsHardwareAccelerated && Math.Min(sourceWidth, sourceHeight) >= 8 ? 8 : Vector128.IsHardwareAccelerated ? 4 : 1; + + Span transposeScratch = MemoryMarshal.Cast(scratch); + + for (int row = 0; row < sourceHeight; row += tileSize) + { + for (int column = 0; column < sourceWidth; column += tileSize) + { + ref int tileSource = ref Unsafe.Add(ref source, (row * sourceStride) + column); + ref int tileDestination = ref Unsafe.Add(ref destination, (column * destinationStride) + row); + + if (tileSize == 16) + { + Av1Transform2dOperations.Transpose16x16Avx512( + ref tileSource, + sourceStride, + ref tileDestination, + destinationStride, + transposeScratch, + roundShift, + normalizeRectangle); + } + else if (tileSize == 8) + { + Av1Transform2dOperations.Transpose8x8Int32( + ref tileSource, + sourceStride, + ref tileDestination, + destinationStride, + roundShift, + normalizeRectangle); + } + else if (tileSize == 4) + { + Av1Transform2dOperations.Transpose4x4Int32( + ref tileSource, + sourceStride, + ref tileDestination, + destinationStride, + roundShift, + normalizeRectangle); + } + else + { + int value = Av1Math.RoundShift(tileSource, roundShift); + tileDestination = normalizeRectangle + ? Av1Transform1dMath.HalfButterfly(Av1Transform1dMath.NewSqrt2, value, 0, 0, Av1Transform1dMath.NewSqrt2Bits) + : value; + } + } + } + } + + /// + /// Widens the completed packed coefficient matrix into its external signed thirty-two-bit representation. + /// + /// The first packed transform coefficient. + /// The coefficient matrix width. + /// The coefficient matrix height. + /// The destination signed thirty-two-bit coefficients. + private static void StorePacked(ref short source, int width, int height, Span destination) + { + ref int destinationBase = ref MemoryMarshal.GetReference(destination); + + for (int row = 0; row < height; row++) + { + ref short sourceRow = ref Unsafe.Add(ref source, row * width); + ref int destinationRow = ref Unsafe.Add(ref destinationBase, row * width); + int column = 0; + + if (Avx512BW.IsSupported) + { + nuint vector512Count = Numerics.Vector512Count(width - column); + for (; vector512Count > 0; vector512Count--, column += Vector512.Count) + { + (Vector512 lower, Vector512 upper) = Vector512.Widen(Vector512.LoadUnsafe(ref sourceRow, (nuint)column)); + + lower.StoreUnsafe(ref destinationRow, (nuint)column); + upper.StoreUnsafe(ref destinationRow, (nuint)(column + Vector512.Count)); + } + } + + if (Avx2.IsSupported) + { + nuint vector256Count = Numerics.Vector256Count(width - column); + for (; vector256Count > 0; vector256Count--, column += Vector256.Count) + { + (Vector256 lower, Vector256 upper) = Vector256.Widen(Vector256.LoadUnsafe(ref sourceRow, (nuint)column)); + + lower.StoreUnsafe(ref destinationRow, (nuint)column); + upper.StoreUnsafe(ref destinationRow, (nuint)(column + Vector256.Count)); + } + } + + nuint vector128Count = Numerics.Vector128Count(width - column); + for (; vector128Count > 0; vector128Count--, column += Vector128.Count) + { + (Vector128 lower, Vector128 upper) = Vector128.Widen(Vector128.LoadUnsafe(ref sourceRow, (nuint)column)); + + lower.StoreUnsafe(ref destinationRow, (nuint)column); + upper.StoreUnsafe(ref destinationRow, (nuint)(column + Vector128.Count)); + } + + if (column < width) + { + Vector128 value = Vector128.WidenLower( + Vector128.Create(Unsafe.As(ref Unsafe.Add(ref sourceRow, column)), 0UL).AsInt16()); + + value.StoreUnsafe(ref destinationRow, (nuint)column); + } + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Transform/Av1IdentityTransform1d.cs b/src/ImageSharp/Formats/Heif/Av1/Transform/Av1IdentityTransform1d.cs new file mode 100644 index 0000000000..c60891e659 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Transform/Av1IdentityTransform1d.cs @@ -0,0 +1,133 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +/// +/// Provides the shared SIMD arithmetic for AV1 identity-transform operators. +/// +/// +/// Transform-vector fields represent positions along an identity transform and lanes represent independent axes. +/// Scaling is lane-local; a zero fractional-bit count uses exact multiplication, while fixed-point variants use the +/// same rounded butterfly primitive as DCT and ADST operators. +/// +internal static class Av1IdentityTransform1d +{ + /// + /// Scales four independent identity-transform axes in parallel. + /// + /// The source values for four transform axes. + /// The destination values for four transform axes. + /// The number of values in each axis. + /// The fixed-point identity scale. + /// The number of fractional bits in . + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void Transform(ref Av1TransformVector> input, ref Av1TransformVector> output, int length, int multiplier, int fractionalBits) + { + if (fractionalBits == 0) + { + for (int i = 0; i < length; i++) + { + output[i] = input[i] * multiplier; + } + + return; + } + + for (int i = 0; i < length; i++) + { + output[i] = Av1Transform1dMath.MultiplyRound(input[i], multiplier, fractionalBits); + } + } + + /// + /// Scales eight independent identity-transform axes in parallel. + /// + /// The source values for eight transform axes. + /// The destination values for eight transform axes. + /// The number of values in each axis. + /// The fixed-point identity scale. + /// The number of fractional bits in . + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void Transform(ref Av1TransformVector> input, ref Av1TransformVector> output, int length, int multiplier, int fractionalBits) + { + if (fractionalBits == 0) + { + for (int i = 0; i < length; i++) + { + output[i] = input[i] * multiplier; + } + + return; + } + + for (int i = 0; i < length; i++) + { + output[i] = Av1Transform1dMath.MultiplyRound(input[i], multiplier, fractionalBits); + } + } + + /// + /// Scales four independent identity-transform axes with widened fixed-point products. + /// + /// The source values for four transform axes. + /// The destination values for four transform axes. + /// The number of values in each axis. + /// The fixed-point identity scale. + /// The number of fractional bits in . + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void TransformWidened(ref Av1TransformVector> input, ref Av1TransformVector> output, int length, int multiplier, int fractionalBits) + { + for (int i = 0; i < length; i++) + { + output[i] = Av1Transform1dMath.MultiplyRoundWidened(input[i], multiplier, fractionalBits); + } + } + + /// + /// Scales eight independent identity-transform axes with widened fixed-point products. + /// + /// The source values for eight transform axes. + /// The destination values for eight transform axes. + /// The number of values in each axis. + /// The fixed-point identity scale. + /// The number of fractional bits in . + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void TransformWidened(ref Av1TransformVector> input, ref Av1TransformVector> output, int length, int multiplier, int fractionalBits) + { + for (int i = 0; i < length; i++) + { + output[i] = Av1Transform1dMath.MultiplyRoundWidened(input[i], multiplier, fractionalBits); + } + } + + /// + /// Scales sixteen independent identity-transform axes in parallel. + /// + /// The source values for sixteen transform axes. + /// The destination values for sixteen transform axes. + /// The number of values in each axis. + /// The fixed-point identity scale. + /// The number of fractional bits in . + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void Transform(ref Av1TransformVector> input, ref Av1TransformVector> output, int length, int multiplier, int fractionalBits) + { + if (fractionalBits == 0) + { + for (int i = 0; i < length; i++) + { + output[i] = input[i] * multiplier; + } + + return; + } + + for (int i = 0; i < length; i++) + { + output[i] = Av1Transform1dMath.MultiplyRound(input[i], multiplier, fractionalBits); + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Transform/Av1Inverse2dTransformer.cs b/src/ImageSharp/Formats/Heif/Av1/Transform/Av1Inverse2dTransformer.cs new file mode 100644 index 0000000000..fd7d0518a1 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Transform/Av1Inverse2dTransformer.cs @@ -0,0 +1,689 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.InteropServices; +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +/// +/// Applies separable two-dimensional AV1 inverse transforms and reconstructs decoded samples. +/// +/// +/// Coefficients are transposed so that each SIMD lane represents an independent transform axis and each vector field +/// represents one coefficient position. The column and row operators can then use the scalar stage graph without +/// cross-lane permutations. Reconstruction adds the final residuals to their matching prediction lanes before +/// narrowing to the decoded sample depth. +/// +internal static partial class Av1Inverse2dTransformer +{ + /// + /// Applies an inverse transform and adds its residual to high-bit-depth predicted samples. + /// + /// The dequantized coefficients in raster order. + /// The predicted samples read by reconstruction. + /// The number of read samples between rows. + /// The destination reconstructed samples. + /// The number of destination samples between rows. + /// The per-axis transform, flip, shift, and range configuration. + /// The reusable transform workspace. + /// The coded sample bit depth. + public static void Transform2dAdd( + ReadOnlySpan input, + ReadOnlySpan outputForRead, + int strideForRead, + Span outputForWrite, + int strideForWrite, + ref Av1Transform2dFlipConfiguration config, + Span workspace, + int bitDepth) + => Transform2dAdd( + input, + outputForRead, + strideForRead, + outputForWrite, + strideForWrite, + ref config, + workspace, + bitDepth); + + /// + /// Applies an inverse transform and adds its residual to eight-bit predicted samples. + /// + /// The dequantized coefficients in raster order. + /// The predicted samples read by reconstruction. + /// The number of read samples between rows. + /// The destination reconstructed samples. + /// The number of destination samples between rows. + /// The per-axis transform, flip, shift, and range configuration. + /// The reusable transform workspace. + public static void Transform2dAdd( + ReadOnlySpan input, + ReadOnlySpan outputForRead, + int strideForRead, + Span outputForWrite, + int strideForWrite, + ref Av1Transform2dFlipConfiguration config, + Span workspace) + => Transform2dAdd( + input, + outputForRead, + strideForRead, + outputForWrite, + strideForWrite, + ref config, + workspace, + 8); + + /// + /// Initializes the transform ranges and selects the concrete column operator. + /// + private static void Transform2dAdd( + ReadOnlySpan input, + ReadOnlySpan outputForRead, + int strideForRead, + Span outputForWrite, + int strideForWrite, + ref Av1Transform2dFlipConfiguration config, + Span workspace, + int bitDepth) + where TSample : unmanaged + where TOutputOperator : struct, Av1InverseTransformer.IAv1InverseTransformOutputOperator + { + Guard.MustBeSizedAtLeast(workspace, Av1TransformWorkspace.GetRequiredLength(config.TransformSize), nameof(workspace)); + switch (config.TransformFunctionTypeColumn) + { + case Av1TransformFunctionType.Dct4: + DispatchRow( + input, outputForRead, strideForRead, outputForWrite, strideForWrite, ref config, workspace, bitDepth); + + break; + case Av1TransformFunctionType.Dct8: + DispatchRow( + input, outputForRead, strideForRead, outputForWrite, strideForWrite, ref config, workspace, bitDepth); + + break; + case Av1TransformFunctionType.Dct16: + DispatchRow( + input, outputForRead, strideForRead, outputForWrite, strideForWrite, ref config, workspace, bitDepth); + + break; + case Av1TransformFunctionType.Dct32: + DispatchRow( + input, outputForRead, strideForRead, outputForWrite, strideForWrite, ref config, workspace, bitDepth); + + break; + case Av1TransformFunctionType.Dct64: + DispatchRow( + input, outputForRead, strideForRead, outputForWrite, strideForWrite, ref config, workspace, bitDepth); + + break; + case Av1TransformFunctionType.Adst4: + DispatchRow( + input, outputForRead, strideForRead, outputForWrite, strideForWrite, ref config, workspace, bitDepth); + + break; + case Av1TransformFunctionType.Adst8: + DispatchRow( + input, outputForRead, strideForRead, outputForWrite, strideForWrite, ref config, workspace, bitDepth); + + break; + case Av1TransformFunctionType.Adst16: + DispatchRow( + input, outputForRead, strideForRead, outputForWrite, strideForWrite, ref config, workspace, bitDepth); + + break; + case Av1TransformFunctionType.Identity4: + DispatchRow( + input, outputForRead, strideForRead, outputForWrite, strideForWrite, ref config, workspace, bitDepth); + + break; + case Av1TransformFunctionType.Identity8: + DispatchRow( + input, outputForRead, strideForRead, outputForWrite, strideForWrite, ref config, workspace, bitDepth); + + break; + case Av1TransformFunctionType.Identity16: + DispatchRow( + input, outputForRead, strideForRead, outputForWrite, strideForWrite, ref config, workspace, bitDepth); + + break; + case Av1TransformFunctionType.Identity32: + DispatchRow( + input, outputForRead, strideForRead, outputForWrite, strideForWrite, ref config, workspace, bitDepth); + + break; + default: + throw new InvalidImageContentException($"The {config.TransformFunctionTypeColumn} column transform is not valid for {config.TransformSize}."); + } + } + + /// + /// Selects the concrete row operator after the column operator has been specialized. + /// + private static void DispatchRow( + ReadOnlySpan input, + ReadOnlySpan outputForRead, + int strideForRead, + Span outputForWrite, + int strideForWrite, + ref Av1Transform2dFlipConfiguration config, + Span workspace, + int bitDepth) + where TSample : unmanaged + where TOutputOperator : struct, Av1InverseTransformer.IAv1InverseTransformOutputOperator + where TColumnOperator : struct, IAv1Transform1dOperator + { + switch (config.TransformFunctionTypeRow) + { + case Av1TransformFunctionType.Dct4: + Transform2d( + input, outputForRead, strideForRead, outputForWrite, strideForWrite, ref config, workspace, bitDepth); + + break; + case Av1TransformFunctionType.Dct8: + Transform2d( + input, outputForRead, strideForRead, outputForWrite, strideForWrite, ref config, workspace, bitDepth); + + break; + case Av1TransformFunctionType.Dct16: + Transform2d( + input, outputForRead, strideForRead, outputForWrite, strideForWrite, ref config, workspace, bitDepth); + + break; + case Av1TransformFunctionType.Dct32: + Transform2d( + input, outputForRead, strideForRead, outputForWrite, strideForWrite, ref config, workspace, bitDepth); + + break; + case Av1TransformFunctionType.Dct64: + Transform2d( + input, outputForRead, strideForRead, outputForWrite, strideForWrite, ref config, workspace, bitDepth); + + break; + case Av1TransformFunctionType.Adst4: + Transform2d( + input, outputForRead, strideForRead, outputForWrite, strideForWrite, ref config, workspace, bitDepth); + + break; + case Av1TransformFunctionType.Adst8: + Transform2d( + input, outputForRead, strideForRead, outputForWrite, strideForWrite, ref config, workspace, bitDepth); + + break; + case Av1TransformFunctionType.Adst16: + Transform2d( + input, outputForRead, strideForRead, outputForWrite, strideForWrite, ref config, workspace, bitDepth); + + break; + case Av1TransformFunctionType.Identity4: + Transform2d( + input, outputForRead, strideForRead, outputForWrite, strideForWrite, ref config, workspace, bitDepth); + + break; + case Av1TransformFunctionType.Identity8: + Transform2d( + input, outputForRead, strideForRead, outputForWrite, strideForWrite, ref config, workspace, bitDepth); + + break; + case Av1TransformFunctionType.Identity16: + Transform2d( + input, outputForRead, strideForRead, outputForWrite, strideForWrite, ref config, workspace, bitDepth); + + break; + case Av1TransformFunctionType.Identity32: + Transform2d( + input, outputForRead, strideForRead, outputForWrite, strideForWrite, ref config, workspace, bitDepth); + + break; + default: + throw new InvalidImageContentException($"The {config.TransformFunctionTypeRow} row transform is not valid for {config.TransformSize}."); + } + } + + /// + /// Applies the specialized operator pair using the production lane width selected for the block and processor. + /// + private static void Transform2d( + ReadOnlySpan input, + ReadOnlySpan outputForRead, + int strideForRead, + Span outputForWrite, + int strideForWrite, + ref Av1Transform2dFlipConfiguration config, + Span workspace, + int bitDepth) + where TSample : unmanaged + where TOutputOperator : struct, Av1InverseTransformer.IAv1InverseTransformOutputOperator + where TColumnOperator : struct, IAv1Transform1dOperator + where TRowOperator : struct, IAv1Transform1dOperator + { + int width = config.TransformSize.GetWidth(); + int height = config.TransformSize.GetHeight(); + + if (Vector256.IsHardwareAccelerated && width >= Vector256.Count && height >= Vector256.Count) + { + Transform2dVector256( + input, outputForRead, strideForRead, outputForWrite, strideForWrite, ref config, workspace, bitDepth); + + return; + } + + if (Vector128.IsHardwareAccelerated) + { + Transform2dVector128( + input, outputForRead, strideForRead, outputForWrite, strideForWrite, ref config, workspace, bitDepth); + + return; + } + + Transform2dScalar( + input, outputForRead, strideForRead, outputForWrite, strideForWrite, ref config, workspace, bitDepth); + } + + /// + /// Applies both inverse-transform axes with eight samples packed into each SIMD vector. + /// + /// The reconstructed sample storage type. + /// The operator that adds and clips inverse residuals. + /// The one-dimensional operator applied down each column. + /// The one-dimensional operator applied across each row. + /// The dequantized transform coefficients. + /// The prediction samples. + /// The number of prediction samples between rows. + /// The destination reconstruction samples. + /// The number of destination samples between rows. + /// The transform dimensions, operators, flips, and fixed-point settings. + /// The reusable storage for SIMD vectors and transposed coefficients. + /// The coded sample bit depth used to clamp reconstructed values. + public static void Transform2dVector256( + ReadOnlySpan input, + ReadOnlySpan outputForRead, + int strideForRead, + Span outputForWrite, + int strideForWrite, + ref Av1Transform2dFlipConfiguration config, + Span workspace, + int bitDepth) + where TSample : unmanaged + where TOutputOperator : struct, Av1InverseTransformer.IAv1InverseTransformOutputOperator + where TColumnOperator : struct, IAv1Transform1dOperator + where TRowOperator : struct, IAv1Transform1dOperator + { + const int laneCount = 8; + const int vectorLength = Av1Constants.MaxTransformSize * laneCount; + + int width = config.TransformSize.GetWidth(); + int height = config.TransformSize.GetHeight(); + Av1TransformSize adjustedTransformSize = config.TransformSize.GetAdjusted(); + int inputWidth = adjustedTransformSize.GetWidth(); + int inputHeight = adjustedTransformSize.GetHeight(); + int shift0 = config.Shift0; + int shift1 = config.Shift1; + bool normalizeRectangle = Math.Abs(config.TransformSize.GetRectangleLogRatio()) == 1; + byte rowClampBits = (byte)(bitDepth + 8); + byte columnClampBits = (byte)Math.Max(bitDepth + 6, 16); + + // Three transform vectors occupy the fixed prefix of the caller-owned workspace. Reinterpreting that storage + // gives constant field offsets to the one-dimensional operators; the remaining raster buffer holds the first + // axis result without allocating or aliasing any active stage vector. + ref int workspaceBase = ref MemoryMarshal.GetReference(workspace); + ref Av1TransformVector> tempIn = ref Unsafe.As>>(ref workspaceBase); + ref Av1TransformVector> tempOut = ref Unsafe.As>>(ref Unsafe.Add(ref workspaceBase, vectorLength)); + ref Av1TransformVector> step = ref Unsafe.As>>(ref Unsafe.Add(ref workspaceBase, 2 * vectorLength)); + Span buffer = workspace.Slice(Av1TransformWorkspace.Vector256StorageLength, width * height); + ref int inputBase = ref MemoryMarshal.GetReference(input); + ref int bufferBase = ref MemoryMarshal.GetReference(buffer); + + // Rows are transposed into lanes so the complete 1-D operator runs once for eight rows. + for (int row = 0; row < height; row += laneCount) + { + for (int column = 0; column < width; column += laneCount) + { + bool hasCodedCoefficients = row < inputHeight && column < inputWidth; + Vector256 row0; + Vector256 row1; + Vector256 row2; + Vector256 row3; + Vector256 row4; + Vector256 row5; + Vector256 row6; + Vector256 row7; + + if (hasCodedCoefficients) + { + row0 = Vector256.LoadUnsafe(ref inputBase, (nuint)(((row + 0) * inputWidth) + column)); + row1 = Vector256.LoadUnsafe(ref inputBase, (nuint)(((row + 1) * inputWidth) + column)); + row2 = Vector256.LoadUnsafe(ref inputBase, (nuint)(((row + 2) * inputWidth) + column)); + row3 = Vector256.LoadUnsafe(ref inputBase, (nuint)(((row + 3) * inputWidth) + column)); + row4 = Vector256.LoadUnsafe(ref inputBase, (nuint)(((row + 4) * inputWidth) + column)); + row5 = Vector256.LoadUnsafe(ref inputBase, (nuint)(((row + 5) * inputWidth) + column)); + row6 = Vector256.LoadUnsafe(ref inputBase, (nuint)(((row + 6) * inputWidth) + column)); + row7 = Vector256.LoadUnsafe(ref inputBase, (nuint)(((row + 7) * inputWidth) + column)); + } + else + { + row0 = Vector256.Zero; + row1 = Vector256.Zero; + row2 = Vector256.Zero; + row3 = Vector256.Zero; + row4 = Vector256.Zero; + row5 = Vector256.Zero; + row6 = Vector256.Zero; + row7 = Vector256.Zero; + } + + Av1Transform2dOperations.Transpose(ref row0, ref row1, ref row2, ref row3, ref row4, ref row5, ref row6, ref row7); + tempIn[column + 0] = PrepareInverseRow(row0, normalizeRectangle, rowClampBits); + tempIn[column + 1] = PrepareInverseRow(row1, normalizeRectangle, rowClampBits); + tempIn[column + 2] = PrepareInverseRow(row2, normalizeRectangle, rowClampBits); + tempIn[column + 3] = PrepareInverseRow(row3, normalizeRectangle, rowClampBits); + tempIn[column + 4] = PrepareInverseRow(row4, normalizeRectangle, rowClampBits); + tempIn[column + 5] = PrepareInverseRow(row5, normalizeRectangle, rowClampBits); + tempIn[column + 6] = PrepareInverseRow(row6, normalizeRectangle, rowClampBits); + tempIn[column + 7] = PrepareInverseRow(row7, normalizeRectangle, rowClampBits); + } + + TRowOperator.Transform(ref tempIn, ref tempOut, ref step, config.CosBitRow, config.StageRangeRow); + + for (int column = 0; column < width; column += laneCount) + { + Vector256 row0 = Av1Transform2dOperations.RoundShift(tempOut[column + 0], -shift0); + Vector256 row1 = Av1Transform2dOperations.RoundShift(tempOut[column + 1], -shift0); + Vector256 row2 = Av1Transform2dOperations.RoundShift(tempOut[column + 2], -shift0); + Vector256 row3 = Av1Transform2dOperations.RoundShift(tempOut[column + 3], -shift0); + Vector256 row4 = Av1Transform2dOperations.RoundShift(tempOut[column + 4], -shift0); + Vector256 row5 = Av1Transform2dOperations.RoundShift(tempOut[column + 5], -shift0); + Vector256 row6 = Av1Transform2dOperations.RoundShift(tempOut[column + 6], -shift0); + Vector256 row7 = Av1Transform2dOperations.RoundShift(tempOut[column + 7], -shift0); + Av1Transform2dOperations.Transpose(ref row0, ref row1, ref row2, ref row3, ref row4, ref row5, ref row6, ref row7); + row0.StoreUnsafe(ref bufferBase, (nuint)(((row + 0) * width) + column)); + row1.StoreUnsafe(ref bufferBase, (nuint)(((row + 1) * width) + column)); + row2.StoreUnsafe(ref bufferBase, (nuint)(((row + 2) * width) + column)); + row3.StoreUnsafe(ref bufferBase, (nuint)(((row + 3) * width) + column)); + row4.StoreUnsafe(ref bufferBase, (nuint)(((row + 4) * width) + column)); + row5.StoreUnsafe(ref bufferBase, (nuint)(((row + 5) * width) + column)); + row6.StoreUnsafe(ref bufferBase, (nuint)(((row + 6) * width) + column)); + row7.StoreUnsafe(ref bufferBase, (nuint)(((row + 7) * width) + column)); + } + } + + ref TSample readBase = ref MemoryMarshal.GetReference(outputForRead); + ref TSample writeBase = ref MemoryMarshal.GetReference(outputForWrite); + + // The intermediate rows already contain contiguous column groups, avoiding a second transpose. Horizontal and + // vertical flips are folded into these loads and row selections so flipped transforms need no reversal pass. + for (int column = 0; column < width; column += laneCount) + { + int sourceColumn = config.FlipLeftToRight ? width - column - laneCount : column; + + for (int row = 0; row < height; row++) + { + Vector256 value = Vector256.LoadUnsafe(ref bufferBase, (nuint)((row * width) + sourceColumn)); + value = config.FlipLeftToRight ? Av1Transform2dOperations.Reverse(value) : value; + tempIn[row] = Av1Transform1dMath.Clamp(value, columnClampBits); + } + + TColumnOperator.Transform(ref tempIn, ref tempOut, ref step, config.CosBitColumn, config.StageRangeColumn); + + for (int row = 0; row < height; row++) + { + int sourceRow = config.FlipUpsideDown ? height - row - 1 : row; + Vector256 residual = Av1Transform2dOperations.RoundShift(tempOut[sourceRow], -shift1); + ref TSample prediction = ref Unsafe.Add(ref readBase, (row * strideForRead) + column); + ref TSample destination = ref Unsafe.Add(ref writeBase, (row * strideForWrite) + column); + TOutputOperator.Add(ref prediction, ref destination, residual, bitDepth); + } + } + } + + /// + /// Applies both inverse-transform axes with four samples packed into each SIMD vector. + /// + /// The reconstructed sample storage type. + /// The operator that adds and clips inverse residuals. + /// The one-dimensional operator applied down each column. + /// The one-dimensional operator applied across each row. + /// The dequantized transform coefficients. + /// The prediction samples. + /// The number of prediction samples between rows. + /// The destination reconstruction samples. + /// The number of destination samples between rows. + /// The transform dimensions, operators, flips, and fixed-point settings. + /// The reusable storage for SIMD vectors and transposed coefficients. + /// The coded sample bit depth used to clamp reconstructed values. + public static void Transform2dVector128( + ReadOnlySpan input, + ReadOnlySpan outputForRead, + int strideForRead, + Span outputForWrite, + int strideForWrite, + ref Av1Transform2dFlipConfiguration config, + Span workspace, + int bitDepth) + where TSample : unmanaged + where TOutputOperator : struct, Av1InverseTransformer.IAv1InverseTransformOutputOperator + where TColumnOperator : struct, IAv1Transform1dOperator + where TRowOperator : struct, IAv1Transform1dOperator + { + const int laneCount = 4; + const int vectorLength = Av1Constants.MaxTransformSize * laneCount; + + int width = config.TransformSize.GetWidth(); + int height = config.TransformSize.GetHeight(); + Av1TransformSize adjustedTransformSize = config.TransformSize.GetAdjusted(); + int inputWidth = adjustedTransformSize.GetWidth(); + int inputHeight = adjustedTransformSize.GetHeight(); + int shift0 = config.Shift0; + int shift1 = config.Shift1; + bool normalizeRectangle = Math.Abs(config.TransformSize.GetRectangleLogRatio()) == 1; + byte rowClampBits = (byte)(bitDepth + 8); + byte columnClampBits = (byte)Math.Max(bitDepth + 6, 16); + + // The 128-bit workspace has the same three-vector plus raster-buffer layout as the 256-bit path. Only the + // number of independent axes represented by each vector changes from eight to four. + ref int workspaceBase = ref MemoryMarshal.GetReference(workspace); + ref Av1TransformVector> tempIn = ref Unsafe.As>>(ref workspaceBase); + ref Av1TransformVector> tempOut = ref Unsafe.As>>(ref Unsafe.Add(ref workspaceBase, vectorLength)); + ref Av1TransformVector> step = ref Unsafe.As>>(ref Unsafe.Add(ref workspaceBase, 2 * vectorLength)); + Span buffer = workspace.Slice(Av1TransformWorkspace.Vector128StorageLength, width * height); + ref int inputBase = ref MemoryMarshal.GetReference(input); + ref int bufferBase = ref MemoryMarshal.GetReference(buffer); + + // A 4-by-4 transpose changes four raster rows into four coefficient-position vectors. Each lane then remains + // one independent row throughout the complete first-axis stage network. + for (int row = 0; row < height; row += laneCount) + { + for (int column = 0; column < width; column += laneCount) + { + bool hasCodedCoefficients = row < inputHeight && column < inputWidth; + Vector128 row0; + Vector128 row1; + Vector128 row2; + Vector128 row3; + + if (hasCodedCoefficients) + { + row0 = Vector128.LoadUnsafe(ref inputBase, (nuint)(((row + 0) * inputWidth) + column)); + row1 = Vector128.LoadUnsafe(ref inputBase, (nuint)(((row + 1) * inputWidth) + column)); + row2 = Vector128.LoadUnsafe(ref inputBase, (nuint)(((row + 2) * inputWidth) + column)); + row3 = Vector128.LoadUnsafe(ref inputBase, (nuint)(((row + 3) * inputWidth) + column)); + } + else + { + row0 = Vector128.Zero; + row1 = Vector128.Zero; + row2 = Vector128.Zero; + row3 = Vector128.Zero; + } + + Av1Transform2dOperations.Transpose(ref row0, ref row1, ref row2, ref row3); + tempIn[column + 0] = PrepareInverseRow(row0, normalizeRectangle, rowClampBits); + tempIn[column + 1] = PrepareInverseRow(row1, normalizeRectangle, rowClampBits); + tempIn[column + 2] = PrepareInverseRow(row2, normalizeRectangle, rowClampBits); + tempIn[column + 3] = PrepareInverseRow(row3, normalizeRectangle, rowClampBits); + } + + TRowOperator.Transform(ref tempIn, ref tempOut, ref step, config.CosBitRow, config.StageRangeRow); + + for (int column = 0; column < width; column += laneCount) + { + Vector128 row0 = Av1Transform2dOperations.RoundShift(tempOut[column + 0], -shift0); + Vector128 row1 = Av1Transform2dOperations.RoundShift(tempOut[column + 1], -shift0); + Vector128 row2 = Av1Transform2dOperations.RoundShift(tempOut[column + 2], -shift0); + Vector128 row3 = Av1Transform2dOperations.RoundShift(tempOut[column + 3], -shift0); + Av1Transform2dOperations.Transpose(ref row0, ref row1, ref row2, ref row3); + row0.StoreUnsafe(ref bufferBase, (nuint)(((row + 0) * width) + column)); + row1.StoreUnsafe(ref bufferBase, (nuint)(((row + 1) * width) + column)); + row2.StoreUnsafe(ref bufferBase, (nuint)(((row + 2) * width) + column)); + row3.StoreUnsafe(ref bufferBase, (nuint)(((row + 3) * width) + column)); + } + } + + ref TSample readBase = ref MemoryMarshal.GetReference(outputForRead); + ref TSample writeBase = ref MemoryMarshal.GetReference(outputForWrite); + + // Contiguous four-column groups become the independent lanes for the second axis. Flip selection is applied + // while reading the intermediate block and selecting completed rows, avoiding any extra copy or reversal. + for (int column = 0; column < width; column += laneCount) + { + int sourceColumn = config.FlipLeftToRight ? width - column - laneCount : column; + + for (int row = 0; row < height; row++) + { + Vector128 value = Vector128.LoadUnsafe(ref bufferBase, (nuint)((row * width) + sourceColumn)); + value = config.FlipLeftToRight ? Av1Transform2dOperations.Reverse(value) : value; + tempIn[row] = Av1Transform1dMath.Clamp(value, columnClampBits); + } + + TColumnOperator.Transform(ref tempIn, ref tempOut, ref step, config.CosBitColumn, config.StageRangeColumn); + + for (int row = 0; row < height; row++) + { + int sourceRow = config.FlipUpsideDown ? height - row - 1 : row; + Vector128 residual = Av1Transform2dOperations.RoundShift(tempOut[sourceRow], -shift1); + ref TSample prediction = ref Unsafe.Add(ref readBase, (row * strideForRead) + column); + ref TSample destination = ref Unsafe.Add(ref writeBase, (row * strideForWrite) + column); + TOutputOperator.Add(ref prediction, ref destination, residual, bitDepth); + } + } + } + + /// + /// Applies both inverse-transform axes when hardware vectorization is unavailable. + /// + /// The reconstructed sample storage type. + /// The operator that adds and clips inverse residuals. + /// The one-dimensional operator applied down each column. + /// The one-dimensional operator applied across each row. + /// The dequantized transform coefficients. + /// The prediction samples. + /// The number of prediction samples between rows. + /// The destination reconstruction samples. + /// The number of destination samples between rows. + /// The transform dimensions, operators, flips, and fixed-point settings. + /// The reusable storage for transform stages and transposed coefficients. + /// The coded sample bit depth used to clamp reconstructed values. + public static void Transform2dScalar( + ReadOnlySpan input, + ReadOnlySpan outputForRead, + int strideForRead, + Span outputForWrite, + int strideForWrite, + ref Av1Transform2dFlipConfiguration config, + Span workspace, + int bitDepth) + where TSample : unmanaged + where TOutputOperator : struct, Av1InverseTransformer.IAv1InverseTransformOutputOperator + where TColumnOperator : struct, IAv1Transform1dOperator + where TRowOperator : struct, IAv1Transform1dOperator + { + int width = config.TransformSize.GetWidth(); + int height = config.TransformSize.GetHeight(); + Av1TransformSize adjustedTransformSize = config.TransformSize.GetAdjusted(); + int inputWidth = adjustedTransformSize.GetWidth(); + int inputHeight = adjustedTransformSize.GetHeight(); + int vectorLength = Math.Max(width, height); + int shift0 = config.Shift0; + int shift1 = config.Shift1; + bool normalizeRectangle = Math.Abs(config.TransformSize.GetRectangleLogRatio()) == 1; + byte rowClampBits = (byte)(bitDepth + 8); + byte columnClampBits = (byte)Math.Max(bitDepth + 6, 16); + Span tempIn = workspace[..vectorLength]; + Span tempOut = workspace.Slice(vectorLength, vectorLength); + Span step = workspace.Slice(2 * vectorLength, vectorLength); + Span buffer = workspace.Slice(3 * vectorLength, width * height); + + for (int row = 0; row < height; row++) + { + int rowOffset = row * width; + tempIn[..width].Clear(); + + if (row < inputHeight) + { + int inputOffset = row * inputWidth; + + for (int column = 0; column < inputWidth; column++) + { + int value = input[inputOffset + column]; + value = normalizeRectangle + ? Av1Math.RoundShift((long)value * Av1InverseTransformMath.NewInverseSqrt2, Av1InverseTransformMath.NewSqrt2BitCount) + : value; + tempIn[column] = Av1Transform1dMath.Clamp(value, rowClampBits); + } + } + + TRowOperator.Transform(tempIn, tempOut, step, config.CosBitRow, config.StageRangeRow); + Av1InverseTransformMath.RoundShiftArray(tempOut, width, -shift0); + tempOut[..width].CopyTo(buffer.Slice(rowOffset, width)); + } + + for (int column = 0; column < width; column++) + { + int sourceColumn = config.FlipLeftToRight ? width - column - 1 : column; + + for (int row = 0; row < height; row++) + { + tempIn[row] = Av1Transform1dMath.Clamp(buffer[(row * width) + sourceColumn], columnClampBits); + } + + TColumnOperator.Transform(tempIn, tempOut, step, config.CosBitColumn, config.StageRangeColumn); + Av1InverseTransformMath.RoundShiftArray(tempOut, height, -shift1); + + for (int row = 0; row < height; row++) + { + int sourceRow = config.FlipUpsideDown ? height - row - 1 : row; + int readIndex = (row * strideForRead) + column; + int writeIndex = (row * strideForWrite) + column; + outputForWrite[writeIndex] = TOutputOperator.Add(outputForRead[readIndex], tempOut[sourceRow], bitDepth); + } + } + } + + /// + /// Applies rectangular normalization and the row-input clamp to four coefficient lanes. + /// + private static Vector128 PrepareInverseRow(Vector128 value, bool normalizeRectangle, byte clampBits) + { + if (normalizeRectangle) + { + value = Av1Transform1dMath.MultiplyRound(value, Av1InverseTransformMath.NewInverseSqrt2, Av1InverseTransformMath.NewSqrt2BitCount); + } + + return Av1Transform1dMath.Clamp(value, clampBits); + } + + /// + /// Applies rectangular normalization and the row-input clamp to eight coefficient lanes. + /// + private static Vector256 PrepareInverseRow(Vector256 value, bool normalizeRectangle, byte clampBits) + { + if (normalizeRectangle) + { + value = Av1Transform1dMath.MultiplyRound(value, Av1InverseTransformMath.NewInverseSqrt2, Av1InverseTransformMath.NewSqrt2BitCount); + } + + return Av1Transform1dMath.Clamp(value, clampBits); + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Transform/Av1InverseTransformMath.cs b/src/ImageSharp/Formats/Heif/Av1/Transform/Av1InverseTransformMath.cs new file mode 100644 index 0000000000..c575b244a6 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Transform/Av1InverseTransformMath.cs @@ -0,0 +1,301 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +/// +/// Contains fixed-point constants and quantizer lookup tables shared by AV1 inverse transforms. +/// +internal static class Av1InverseTransformMath +{ + /// + /// The fixed-point representation of 1 / sqrt(2) at fractional bits. + /// + public const int NewInverseSqrt2 = 2896; + + /// + /// The number of fractional bits used by . + /// + public const int NewSqrt2BitCount = 12; + + /// + /// Gets the normative AC dequantizer value indexed by bit-depth category and quantizer index. + /// + public static readonly int[,] AcQLookup = new int[3, 256] + { + { + 4, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, + 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, + 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, + 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, + 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, + 102, 104, 106, 108, 110, 112, 114, 116, 118, 120, 122, 124, 126, 128, 130, 132, 134, 136, 138, + 140, 142, 144, 146, 148, 150, 152, 155, 158, 161, 164, 167, 170, 173, 176, 179, 182, 185, 188, + 191, 194, 197, 200, 203, 207, 211, 215, 219, 223, 227, 231, 235, 239, 243, 247, 251, 255, 260, + 265, 270, 275, 280, 285, 290, 295, 300, 305, 311, 317, 323, 329, 335, 341, 347, 353, 359, 366, + 373, 380, 387, 394, 401, 408, 416, 424, 432, 440, 448, 456, 465, 474, 483, 492, 501, 510, 520, + 530, 540, 550, 560, 571, 582, 593, 604, 615, 627, 639, 651, 663, 676, 689, 702, 715, 729, 743, + 757, 771, 786, 801, 816, 832, 848, 864, 881, 898, 915, 933, 951, 969, 988, 1007, 1026, 1046, 1066, + 1087, 1108, 1129, 1151, 1173, 1196, 1219, 1243, 1267, 1292, 1317, 1343, 1369, 1396, 1423, 1451, 1479, 1508, 1537, + 1567, 1597, 1628, 1660, 1692, 1725, 1759, 1793, 1828, + }, + { + 4, 9, 11, 13, 16, 18, 21, 24, 27, 30, 33, 37, 40, 44, 48, 51, 55, 59, 63, + 67, 71, 75, 79, 83, 88, 92, 96, 100, 105, 109, 114, 118, 122, 127, 131, 136, 140, 145, + 149, 154, 158, 163, 168, 172, 177, 181, 186, 190, 195, 199, 204, 208, 213, 217, 222, 226, 231, + 235, 240, 244, 249, 253, 258, 262, 267, 271, 275, 280, 284, 289, 293, 297, 302, 306, 311, 315, + 319, 324, 328, 332, 337, 341, 345, 349, 354, 358, 362, 367, 371, 375, 379, 384, 388, 392, 396, + 401, 409, 417, 425, 433, 441, 449, 458, 466, 474, 482, 490, 498, 506, 514, 523, 531, 539, 547, + 555, 563, 571, 579, 588, 596, 604, 616, 628, 640, 652, 664, 676, 688, 700, 713, 725, 737, 749, + 761, 773, 785, 797, 809, 825, 841, 857, 873, 889, 905, 922, 938, 954, 970, 986, 1002, 1018, 1038, + 1058, 1078, 1098, 1118, 1138, 1158, 1178, 1198, 1218, 1242, 1266, 1290, 1314, 1338, 1362, 1386, 1411, 1435, 1463, + 1491, 1519, 1547, 1575, 1603, 1631, 1663, 1695, 1727, 1759, 1791, 1823, 1859, 1895, 1931, 1967, 2003, 2039, 2079, + 2119, 2159, 2199, 2239, 2283, 2327, 2371, 2415, 2459, 2507, 2555, 2603, 2651, 2703, 2755, 2807, 2859, 2915, 2971, + 3027, 3083, 3143, 3203, 3263, 3327, 3391, 3455, 3523, 3591, 3659, 3731, 3803, 3876, 3952, 4028, 4104, 4184, 4264, + 4348, 4432, 4516, 4604, 4692, 4784, 4876, 4972, 5068, 5168, 5268, 5372, 5476, 5584, 5692, 5804, 5916, 6032, 6148, + 6268, 6388, 6512, 6640, 6768, 6900, 7036, 7172, 7312, + }, + { + 4, 13, 19, 27, 35, 44, 54, 64, 75, 87, 99, 112, 126, 139, 154, 168, + 183, 199, 214, 230, 247, 263, 280, 297, 314, 331, 349, 366, 384, 402, 420, 438, + 456, 475, 493, 511, 530, 548, 567, 586, 604, 623, 642, 660, 679, 698, 716, 735, + 753, 772, 791, 809, 828, 846, 865, 884, 902, 920, 939, 957, 976, 994, 1012, 1030, + 1049, 1067, 1085, 1103, 1121, 1139, 1157, 1175, 1193, 1211, 1229, 1246, 1264, 1282, 1299, 1317, + 1335, 1352, 1370, 1387, 1405, 1422, 1440, 1457, 1474, 1491, 1509, 1526, 1543, 1560, 1577, 1595, + 1627, 1660, 1693, 1725, 1758, 1791, 1824, 1856, 1889, 1922, 1954, 1987, 2020, 2052, 2085, 2118, + 2150, 2183, 2216, 2248, 2281, 2313, 2346, 2378, 2411, 2459, 2508, 2556, 2605, 2653, 2701, 2750, + 2798, 2847, 2895, 2943, 2992, 3040, 3088, 3137, 3185, 3234, 3298, 3362, 3426, 3491, 3555, 3619, + 3684, 3748, 3812, 3876, 3941, 4005, 4069, 4149, 4230, 4310, 4390, 4470, 4550, 4631, 4711, 4791, + 4871, 4967, 5064, 5160, 5256, 5352, 5448, 5544, 5641, 5737, 5849, 5961, 6073, 6185, 6297, 6410, + 6522, 6650, 6778, 6906, 7034, 7162, 7290, 7435, 7579, 7723, 7867, 8011, 8155, 8315, 8475, 8635, + 8795, 8956, 9132, 9308, 9484, 9660, 9836, 10028, 10220, 10412, 10604, 10812, 11020, 11228, 11437, 11661, + 11885, 12109, 12333, 12573, 12813, 13053, 13309, 13565, 13821, 14093, 14365, 14637, 14925, 15213, 15502, 15806, + 16110, 16414, 16734, 17054, 17390, 17726, 18062, 18414, 18766, 19134, 19502, 19886, 20270, 20670, 21070, 21486, + 21902, 22334, 22766, 23214, 23662, 24126, 24590, 25070, 25551, 26047, 26559, 27071, 27599, 28143, 28687, 29247, + } + }; + + /// + /// Contains the normative DC dequantizer values indexed by bit-depth category and quantizer index. + /// + private static readonly int[,] DcQLookup = new int[3, 256] + { + { + 4, 8, 8, 9, 10, 11, 12, 12, 13, 14, 15, 16, 17, 18, 19, 19, 20, 21, 22, 23, + 24, 25, 26, 26, 27, 28, 29, 30, 31, 32, 32, 33, 34, 35, 36, 37, 38, 38, 39, 40, + 41, 42, 43, 43, 44, 45, 46, 47, 48, 48, 49, 50, 51, 52, 53, 53, 54, 55, 56, 57, + 57, 58, 59, 60, 61, 62, 62, 63, 64, 65, 66, 66, 67, 68, 69, 70, 70, 71, 72, 73, + 74, 74, 75, 76, 77, 78, 78, 79, 80, 81, 81, 82, 83, 84, 85, 85, 87, 88, 90, 92, + 93, 95, 96, 98, 99, 101, 102, 104, 105, 107, 108, 110, 111, 113, 114, 116, 117, 118, 120, 121, + 123, 125, 127, 129, 131, 134, 136, 138, 140, 142, 144, 146, 148, 150, 152, 154, 156, 158, 161, 164, + 166, 169, 172, 174, 177, 180, 182, 185, 187, 190, 192, 195, 199, 202, 205, 208, 211, 214, 217, 220, + 223, 226, 230, 233, 237, 240, 243, 247, 250, 253, 257, 261, 265, 269, 272, 276, 280, 284, 288, 292, + 296, 300, 304, 309, 313, 317, 322, 326, 330, 335, 340, 344, 349, 354, 359, 364, 369, 374, 379, 384, + 389, 395, 400, 406, 411, 417, 423, 429, 435, 441, 447, 454, 461, 467, 475, 482, 489, 497, 505, 513, + 522, 530, 539, 549, 559, 569, 579, 590, 602, 614, 626, 640, 654, 668, 684, 700, 717, 736, 755, 775, + 796, 819, 843, 869, 896, 925, 955, 988, 1022, 1058, 1098, 1139, 1184, 1232, 1282, 1336, + }, + { + 4, 9, 10, 13, 15, 17, 20, 22, 25, 28, 31, 34, 37, 40, 43, 47, 50, 53, 57, + 60, 64, 68, 71, 75, 78, 82, 86, 90, 93, 97, 101, 105, 109, 113, 116, 120, 124, 128, + 132, 136, 140, 143, 147, 151, 155, 159, 163, 166, 170, 174, 178, 182, 185, 189, 193, 197, 200, + 204, 208, 212, 215, 219, 223, 226, 230, 233, 237, 241, 244, 248, 251, 255, 259, 262, 266, 269, + 273, 276, 280, 283, 287, 290, 293, 297, 300, 304, 307, 310, 314, 317, 321, 324, 327, 331, 334, + 337, 343, 350, 356, 362, 369, 375, 381, 387, 394, 400, 406, 412, 418, 424, 430, 436, 442, 448, + 454, 460, 466, 472, 478, 484, 490, 499, 507, 516, 525, 533, 542, 550, 559, 567, 576, 584, 592, + 601, 609, 617, 625, 634, 644, 655, 666, 676, 687, 698, 708, 718, 729, 739, 749, 759, 770, 782, + 795, 807, 819, 831, 844, 856, 868, 880, 891, 906, 920, 933, 947, 961, 975, 988, 1001, 1015, 1030, + 1045, 1061, 1076, 1090, 1105, 1120, 1137, 1153, 1170, 1186, 1202, 1218, 1236, 1253, 1271, 1288, 1306, 1323, 1342, + 1361, 1379, 1398, 1416, 1436, 1456, 1476, 1496, 1516, 1537, 1559, 1580, 1601, 1624, 1647, 1670, 1692, 1717, 1741, + 1766, 1791, 1817, 1844, 1871, 1900, 1929, 1958, 1990, 2021, 2054, 2088, 2123, 2159, 2197, 2236, 2276, 2319, 2363, + 2410, 2458, 2508, 2561, 2616, 2675, 2737, 2802, 2871, 2944, 3020, 3102, 3188, 3280, 3375, 3478, 3586, 3702, 3823, + 3953, 4089, 4236, 4394, 4559, 4737, 4929, 5130, 5347, + }, + { + 4, 12, 18, 25, 33, 41, 50, 60, 70, 80, 91, 103, 115, 127, 140, 153, + 166, 180, 194, 208, 222, 237, 251, 266, 281, 296, 312, 327, 343, 358, 374, 390, + 405, 421, 437, 453, 469, 484, 500, 516, 532, 548, 564, 580, 596, 611, 627, 643, + 659, 674, 690, 706, 721, 737, 752, 768, 783, 798, 814, 829, 844, 859, 874, 889, + 904, 919, 934, 949, 964, 978, 993, 1008, 1022, 1037, 1051, 1065, 1080, 1094, 1108, 1122, + 1136, 1151, 1165, 1179, 1192, 1206, 1220, 1234, 1248, 1261, 1275, 1288, 1302, 1315, 1329, 1342, + 1368, 1393, 1419, 1444, 1469, 1494, 1519, 1544, 1569, 1594, 1618, 1643, 1668, 1692, 1717, 1741, + 1765, 1789, 1814, 1838, 1862, 1885, 1909, 1933, 1957, 1992, 2027, 2061, 2096, 2130, 2165, 2199, + 2233, 2267, 2300, 2334, 2367, 2400, 2434, 2467, 2499, 2532, 2575, 2618, 2661, 2704, 2746, 2788, + 2830, 2872, 2913, 2954, 2995, 3036, 3076, 3127, 3177, 3226, 3275, 3324, 3373, 3421, 3469, 3517, + 3565, 3621, 3677, 3733, 3788, 3843, 3897, 3951, 4005, 4058, 4119, 4181, 4241, 4301, 4361, 4420, + 4479, 4546, 4612, 4677, 4742, 4807, 4871, 4942, 5013, 5083, 5153, 5222, 5291, 5367, 5442, 5517, + 5591, 5665, 5745, 5825, 5905, 5984, 6063, 6149, 6234, 6319, 6404, 6495, 6587, 6678, 6769, 6867, + 6966, 7064, 7163, 7269, 7376, 7483, 7599, 7715, 7832, 7958, 8085, 8214, 8352, 8492, 8635, 8788, + 8945, 9104, 9275, 9450, 9639, 9832, 10031, 10245, 10465, 10702, 10946, 11210, 11482, 11776, 12081, 12409, + 12750, 13118, 13501, 13913, 14343, 14807, 15290, 15812, 16356, 16943, 17575, 18237, 18949, 19718, 20521, 21387, + } + }; + + /// + /// Gets a clipped DC dequantizer value. + /// + /// The segment quantizer index. + /// The plane-specific DC quantizer delta. + /// The coded sample bit depth. + /// The DC dequantizer value. + public static int GetDcQuantization(int qIndex, int delta, Av1BitDepth bitDepth) + => DcQLookup[(int)bitDepth, Av1Math.Clip3(0, 255, qIndex + delta)]; + + /// + /// Gets a clipped AC dequantizer value. + /// + /// The segment quantizer index. + /// The plane-specific AC quantizer delta. + /// The coded sample bit depth. + /// The AC dequantizer value. + public static int GetAcQuantization(int qIndex, int delta, Av1BitDepth bitDepth) + => AcQLookup[(int)bitDepth, Av1Math.Clip3(0, 255, qIndex + delta)]; + + /// + /// Gets the encoder zero-bin factor selected by quantizer magnitude and sample bit depth. + /// + /// The base quantizer index. + /// The coded sample bit depth. + /// The zero-bin factor. + public static int GetQzbinFactor(int q, Av1BitDepth bitDepth) + { + int quant = GetDcQuantization(q, 0, bitDepth); + + // Scaling the eight-bit threshold by four for every two added sample bits preserves the quantizer decision + // at equal normalized signal levels: 148 for 8-bit, 592 for 10-bit, and 2368 for 12-bit. + int shift = (int)bitDepth << 1; + int threshold = (1 << shift) * 148; + return q == 0 ? 64 : (quant < threshold ? 84 : 80); + } + + /// + /// Computes the fixed-point multiplier and shift used to replace division by a quantizer. + /// + /// Receives the reciprocal multiplier without its implicit leading bit. + /// Receives the reciprocal scaling shift. + /// The positive quantizer divisor. + public static void InvertQuantization(out int quantization, out int shift, int d) + { + uint t; + int l, m; + t = (uint)d; + for (l = 0; t > 1; l++) + { + t >>= 1; + } + + m = 1 + ((1 << (16 + l)) / d); + quantization = m - (1 << 16); + shift = 1 << (16 - l); + } + + /// + /// Adds an inverse-transform residual to an eight-bit predicted sample and clips the result. + /// + /// The predicted sample. + /// The inverse-transform residual. + /// The reconstructed eight-bit sample. + public static byte ClipPixelAdd(byte dest, long trans) + { + trans = CheckRange(trans, 8); + return (byte)ClipPixelHighBitDepth(dest + trans, 8); + } + + /// + /// Adds an inverse-transform residual to a high-bit-depth predicted sample and clips the result. + /// + /// The predicted sample stored in the signed transform representation. + /// The inverse-transform residual. + /// The coded sample bit depth. + /// The reconstructed sample stored in the signed transform representation. + public static short ClipPixelAdd(short dest, long trans, int bitDepth) + { + trans = CheckRange(trans, bitDepth); + return ClipPixelHighBitDepth(dest + trans, bitDepth); + } + + /// + /// Clips a reconstructed sample to the unsigned range selected by its bit depth. + /// + /// The unclipped reconstructed sample. + /// The coded sample bit depth. + /// The clipped sample stored in the signed transform representation. + private static short ClipPixelHighBitDepth(long val, int bd) => bd switch + { + 10 => (short)Av1Math.Clamp(val, 0, 1023), + 12 => (short)Av1Math.Clamp(val, 0, 4095), + _ => (short)Av1Math.Clamp(val, 0, 255), + }; + + /// + /// Applies a signed fixed-point shift to the requested prefix of an integer buffer. + /// + /// The transform-stage values. + /// The number of values to update. + /// A positive rounded-right shift or a negative exact-left shift. + public static void RoundShiftArray(Span arr, int size, int bit) + { + int i; + if (bit == 0) + { + return; + } + else + { + if (bit > 0) + { + for (i = 0; i < size; i++) + { + arr[i] = Av1Math.RoundShift(arr[i], bit); + } + } + else + { + for (i = 0; i < size; i++) + { + arr[i] = arr[i] * (1 << (-bit)); + } + } + } + } + + /// + /// Restricts an inverse-transform residual to the intermediate range permitted for the sample bit depth. + /// + /// The inverse-transform residual. + /// The coded sample bit depth. + /// The range-limited residual. + private static long CheckRange(long input, int bd) + { + // AV1 TX case + // - 8 bit: signed 16 bit integer + // - 10 bit: signed 18 bit integer + // - 12 bit: signed 20 bit integer + // - max quantization error = 1828 << (bd - 8) + int maximum = (1 << (7 + bd)) - 1 + (914 << (bd - 7)); + int minimum = -maximum - 1; + return Av1Math.Clamp(input, minimum, maximum); + } + + /// + /// Gets the maximum coded coefficient count retained for a transform size. + /// + /// The signaled transform size. + /// The maximum coefficient end position represented by AV1 syntax. + public static int GetMaxEndOfBuffer(Av1TransformSize transformSize) + { + if (transformSize is Av1TransformSize.Size64x64 or Av1TransformSize.Size64x32 or Av1TransformSize.Size32x64) + { + return 1024; + } + + if (transformSize is Av1TransformSize.Size16x64 or Av1TransformSize.Size64x16) + { + return 512; + } + + return transformSize.GetSize2d(); + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Transform/Av1InverseTransformer.ByteOutputOperator.cs b/src/ImageSharp/Formats/Heif/Av1/Transform/Av1InverseTransformer.ByteOutputOperator.cs new file mode 100644 index 0000000000..8d9a71d80c --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Transform/Av1InverseTransformer.ByteOutputOperator.cs @@ -0,0 +1,61 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +/// +/// Defines eight-bit inverse-transform reconstruction arithmetic. +/// +internal static partial class Av1InverseTransformer +{ + /// + /// Reconstructs eight-bit samples from predicted values and inverse-transform residuals. + /// + internal readonly struct ByteOutputOperator : IAv1InverseTransformOutputOperator + { + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static byte Add(byte prediction, int residual, int bitDepth) + { + _ = bitDepth; + return (byte)Math.Clamp(prediction + residual, byte.MinValue, byte.MaxValue); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void Add(ref byte prediction, ref byte destination, Vector128 residual, int bitDepth) + { + _ = bitDepth; + + // Read and write exactly four bytes. The unused upper lanes only participate in narrowing and never reach + // memory, which keeps reconstruction valid at a tightly packed row boundary. + uint packed = Unsafe.ReadUnaligned(ref prediction); + Vector128 predicted16 = Vector128.WidenLower(Vector128.CreateScalarUnsafe(packed).AsByte()); + Vector128 predicted32 = Vector128.WidenLower(predicted16).AsInt32(); + Vector128 reconstructed = Vector128.Clamp(predicted32 + residual, Vector128.Zero, Vector128.Create((int)byte.MaxValue)); + Vector128 reconstructed16 = Vector128.Narrow(reconstructed.AsUInt32(), Vector128.Zero); + Vector128 reconstructed8 = Vector128.Narrow(reconstructed16, Vector128.Zero); + Unsafe.WriteUnaligned(ref destination, reconstructed8.AsUInt32().ToScalar()); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void Add(ref byte prediction, ref byte destination, Vector256 residual, int bitDepth) + { + _ = bitDepth; + + // Eight byte predictions widen through UInt16 into the eight Int32 residual lanes. The final 64-bit store + // covers only those reconstructed samples and does not require destination padding. + ulong packed = Unsafe.ReadUnaligned(ref prediction); + Vector128 predicted16 = Vector128.WidenLower(Vector128.CreateScalarUnsafe(packed).AsByte()); + Vector256 predicted32 = Vector256.Create(Vector128.WidenLower(predicted16), Vector128.WidenUpper(predicted16)).AsInt32(); + Vector256 reconstructed = Vector256.Clamp(predicted32 + residual, Vector256.Zero, Vector256.Create((int)byte.MaxValue)); + Vector128 reconstructed16 = Vector128.Narrow(reconstructed.GetLower().AsUInt32(), reconstructed.GetUpper().AsUInt32()); + Vector128 reconstructed8 = Vector128.Narrow(reconstructed16, Vector128.Zero); + Unsafe.WriteUnaligned(ref destination, reconstructed8.AsUInt64().ToScalar()); + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Transform/Av1InverseTransformer.HighBitDepthOutputOperator.cs b/src/ImageSharp/Formats/Heif/Av1/Transform/Av1InverseTransformer.HighBitDepthOutputOperator.cs new file mode 100644 index 0000000000..2bfb5b5c02 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Transform/Av1InverseTransformer.HighBitDepthOutputOperator.cs @@ -0,0 +1,50 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.Intrinsics; +using SixLabors.ImageSharp.Common.Helpers; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +/// +/// Defines high-bit-depth inverse-transform reconstruction arithmetic. +/// +internal static partial class Av1InverseTransformer +{ + /// + /// Reconstructs high-bit-depth samples from predicted values and inverse-transform residuals. + /// + internal readonly struct HighBitDepthOutputOperator : IAv1InverseTransformOutputOperator + { + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static short Add(short prediction, int residual, int bitDepth) + => (short)Math.Clamp(prediction + residual, 0, (1 << bitDepth) - 1); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void Add(ref short prediction, ref short destination, Vector128 residual, int bitDepth) + { + ulong packed = Unsafe.ReadUnaligned(ref Unsafe.As(ref prediction)); + Vector128 predicted = Vector128.WidenLower(Vector128.CreateScalarUnsafe(packed).AsInt16()); + Vector128 reconstructed = + Vector128.Clamp(predicted + residual, Vector128.Zero, Vector128.Create((1 << bitDepth) - 1)); + + Vector128 narrowed = Vector128.Narrow(reconstructed, Vector128.Zero); + Unsafe.WriteUnaligned(ref Unsafe.As(ref destination), narrowed.AsUInt64().ToScalar()); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void Add(ref short prediction, ref short destination, Vector256 residual, int bitDepth) + { + Vector256 predicted = Vector256_.Widen(Vector128.LoadUnsafe(ref prediction)); + Vector256 reconstructed = + Vector256.Clamp(predicted + residual, Vector256.Zero, Vector256.Create((1 << bitDepth) - 1)); + + Vector128 narrowed = Vector128.Narrow(reconstructed.GetLower(), reconstructed.GetUpper()); + narrowed.StoreUnsafe(ref destination); + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Transform/Av1InverseTransformer.Operator.cs b/src/ImageSharp/Formats/Heif/Av1/Transform/Av1InverseTransformer.Operator.cs new file mode 100644 index 0000000000..1ce7495aec --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Transform/Av1InverseTransformer.Operator.cs @@ -0,0 +1,47 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +/// +/// Defines the inverse-transform reconstruction operator contract. +/// +internal static partial class Av1InverseTransformer +{ + /// + /// Defines how inverse-transform residuals are added to decoded samples. + /// + /// The decoded sample storage type. + internal interface IAv1InverseTransformOutputOperator + where TSample : unmanaged + { + /// + /// Adds one residual to a predicted sample and clips the result to the coded bit depth. + /// + /// The predicted sample. + /// The inverse-transform residual. + /// The coded sample bit depth. + /// The reconstructed sample. + public static abstract TSample Add(TSample prediction, int residual, int bitDepth); + + /// + /// Adds four residuals to predicted samples and stores the clipped results. + /// + /// The first predicted sample. + /// The first destination sample. + /// The four inverse-transform residuals. + /// The coded sample bit depth. + public static abstract void Add(ref TSample prediction, ref TSample destination, Vector128 residual, int bitDepth); + + /// + /// Adds eight residuals to predicted samples and stores the clipped results. + /// + /// The first predicted sample. + /// The first destination sample. + /// The eight inverse-transform residuals. + /// The coded sample bit depth. + public static abstract void Add(ref TSample prediction, ref TSample destination, Vector256 residual, int bitDepth); + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Transform/Av1InverseTransformer.cs b/src/ImageSharp/Formats/Heif/Av1/Transform/Av1InverseTransformer.cs new file mode 100644 index 0000000000..852ac8e73e --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Transform/Av1InverseTransformer.cs @@ -0,0 +1,138 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +/// +/// Reconstructs decoded AV1 transform coefficients into prediction sample buffers. +/// +internal static partial class Av1InverseTransformer +{ + /// + /// Reconstructs an eight-bit transform block in place by adding its inverse-transform residual. + /// + /// The dequantized transform coefficients. + /// The predicted samples and reconstruction destination. + /// The number of samples between rows. + /// The transform-block dimensions. + /// The compound transform type. + /// The zero-based Y, U, or V plane index. + /// The decoded coefficient end position. + /// Whether the segment uses lossless transform rules. + /// The reusable transform workspace for the containing block decode. + public static void Reconstruct8Bit( + ReadOnlySpan coefficientsBuffer, + Span reconstructionBuffer, + int reconstructionStride, + Av1TransformSize transformSize, + Av1TransformType transformType, + int plane, + int numberOfCoefficients, + bool isLossless, + Span workspace) + { + Av1TransformFunctionParameters transformFunctionParameters = new() + { + TransformType = transformType, + TransformSize = transformSize, + EndOfBuffer = numberOfCoefficients, + IsLossless = isLossless, + BitDepth = 8, + Is16BitPipeline = false + }; + + Av1InverseTransformerFactory.InverseTransformAdd( + coefficientsBuffer, reconstructionBuffer, reconstructionStride, reconstructionBuffer, reconstructionStride, transformFunctionParameters, workspace); + } + + /// + /// Reconstructs an eight-bit transform block from a separate prediction buffer. + /// + /// The dequantized transform coefficients. + /// The predicted samples read by reconstruction. + /// The number of prediction samples between rows. + /// The destination reconstructed samples. + /// The number of destination samples between rows. + /// The transform-block dimensions. + /// The compound transform type. + /// The zero-based Y, U, or V plane index. + /// The decoded coefficient end position. + /// Whether the segment uses lossless transform rules. + /// The reusable transform workspace for the containing block decode. + public static void Reconstruct8Bit( + ReadOnlySpan coefficientsBuffer, + Span reconstructionBufferRead, + int reconstructionReadStride, + Span reconstructionBufferWrite, + int reconstructionWriteStride, + Av1TransformSize transformSize, + Av1TransformType transformType, + int plane, + int numberOfCoefficients, + bool isLossless, + Span workspace) + { + Av1TransformFunctionParameters transformFunctionParameters = new() + { + TransformType = transformType, + TransformSize = transformSize, + EndOfBuffer = numberOfCoefficients, + IsLossless = isLossless, + BitDepth = 8, + Is16BitPipeline = false + }; + + // Separate prediction and destination buffers require every sample to be copied or reconstructed. Restricting + // traversal to the coded coefficient end position would leave the untouched prediction region unwritten. + transformFunctionParameters.EndOfBuffer = Av1InverseTransformMath.GetMaxEndOfBuffer(transformSize); + + Av1InverseTransformerFactory.InverseTransformAdd( + coefficientsBuffer, + reconstructionBufferRead, + reconstructionReadStride, + reconstructionBufferWrite, + reconstructionWriteStride, + transformFunctionParameters, + workspace); + } + + /// + /// Reconstructs a high-bit-depth transform block in place by adding its inverse-transform residual. + /// + /// The dequantized transform coefficients. + /// The predicted samples and reconstruction destination. + /// The number of logical samples between rows. + /// The transform-block dimensions. + /// The compound transform type. + /// The zero-based Y, U, or V plane index. + /// The decoded coefficient end position. + /// Whether the segment uses lossless transform rules. + /// The coded sample bit depth. + /// The reusable transform workspace for the containing block decode. + /// Implements the reconstruction operation in AV1 section 7.11.2. + public static void ReconstructHighBitDepth( + ReadOnlySpan coefficientsBuffer, + Span reconstructionBuffer, + int reconstructionStride, + Av1TransformSize transformSize, + Av1TransformType transformType, + int plane, + int numberOfCoefficients, + bool isLossless, + Av1BitDepth bitDepth, + Span workspace) + { + Av1TransformFunctionParameters transformFunctionParameters = new() + { + TransformType = transformType, + TransformSize = transformSize, + EndOfBuffer = numberOfCoefficients, + IsLossless = isLossless, + BitDepth = bitDepth.GetBitCount(), + Is16BitPipeline = true + }; + + Av1InverseTransformerFactory.InverseTransformAdd( + coefficientsBuffer, reconstructionBuffer, reconstructionStride, reconstructionBuffer, reconstructionStride, transformFunctionParameters, workspace); + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Transform/Av1InverseTransformerFactory.cs b/src/ImageSharp/Formats/Heif/Av1/Transform/Av1InverseTransformerFactory.cs new file mode 100644 index 0000000000..2578574431 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Transform/Av1InverseTransformerFactory.cs @@ -0,0 +1,98 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +/// +/// Selects and runs the AV1 inverse-transform pipeline for byte or high-bit-depth sample storage. +/// +internal static class Av1InverseTransformerFactory +{ + /// + /// Applies an inverse transform and adds its residual to eight-bit predicted samples. + /// + /// The dequantized transform coefficients. + /// The predicted samples read by reconstruction. + /// The number of read samples between rows. + /// The destination reconstructed samples. + /// The number of destination samples between rows. + /// The transform type, dimensions, bit depth, and pipeline selection. + /// The reusable transform workspace for the containing decode operation. + public static void InverseTransformAdd( + ReadOnlySpan coefficients, + Span readBuffer, + int readStride, + Span writeBuffer, + int writeStride, + in Av1TransformFunctionParameters transformFunctionParameters, + Span workspace) + { + Guard.MustBeLessThanOrEqualTo(transformFunctionParameters.BitDepth, 8, nameof(transformFunctionParameters)); + Guard.IsFalse(transformFunctionParameters.Is16BitPipeline, nameof(transformFunctionParameters), "Calling 8-bit pipeline while 16-bit is requested."); + + if (transformFunctionParameters.IsLossless) + { + Av1InverseWalshHadamardTransformer.TransformAdd( + coefficients, + readBuffer, + readStride, + writeBuffer, + writeStride, + transformFunctionParameters.EndOfBuffer, + workspace); + + return; + } + + Av1Transform2dFlipConfiguration config = Av1Transform2dFlipConfiguration.CreateInverse( + transformFunctionParameters.TransformType, + transformFunctionParameters.TransformSize, + transformFunctionParameters.BitDepth); + + Av1Inverse2dTransformer.Transform2dAdd(coefficients, readBuffer, readStride, writeBuffer, writeStride, ref config, workspace); + } + + /// + /// Applies an inverse transform and adds its residual to high-bit-depth predicted samples. + /// + /// The dequantized transform coefficients. + /// The predicted samples read by reconstruction. + /// The number of read samples between rows. + /// The destination reconstructed samples. + /// The number of destination samples between rows. + /// The transform type, dimensions, bit depth, and pipeline selection. + /// The reusable transform workspace for the containing decode operation. + public static void InverseTransformAdd( + ReadOnlySpan coefficients, + Span readBuffer, + int readStride, + Span writeBuffer, + int writeStride, + in Av1TransformFunctionParameters transformFunctionParameters, + Span workspace) + { + Guard.IsTrue(transformFunctionParameters.Is16BitPipeline, nameof(transformFunctionParameters), "Calling 16-bit pipeline while 8-bit is requested."); + + if (transformFunctionParameters.IsLossless) + { + Av1InverseWalshHadamardTransformer.TransformAdd( + coefficients, + readBuffer, + readStride, + writeBuffer, + writeStride, + transformFunctionParameters.EndOfBuffer, + workspace, + transformFunctionParameters.BitDepth); + + return; + } + + Av1Transform2dFlipConfiguration config = Av1Transform2dFlipConfiguration.CreateInverse( + transformFunctionParameters.TransformType, + transformFunctionParameters.TransformSize, + transformFunctionParameters.BitDepth); + + Av1Inverse2dTransformer.Transform2dAdd(coefficients, readBuffer, readStride, writeBuffer, writeStride, ref config, workspace, transformFunctionParameters.BitDepth); + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Transform/Av1InverseWalshHadamardTransformer.cs b/src/ImageSharp/Formats/Heif/Av1/Transform/Av1InverseWalshHadamardTransformer.cs new file mode 100644 index 0000000000..80f173b32e --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Transform/Av1InverseWalshHadamardTransformer.cs @@ -0,0 +1,312 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.InteropServices; +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +/// +/// Applies the reversible four-by-four inverse Walsh-Hadamard transform used by lossless AV1 segments. +/// +/// +/// The vector path stores one transform row in each and one column position in each lane. +/// Register transposes exchange the two transform dimensions between identical reversible butterflies. Reconstruction +/// then adds four consecutive residual lanes to each prediction row with exact-width output stores. +/// +internal static class Av1InverseWalshHadamardTransformer +{ + /// + /// The coefficient normalization shift applied before the first transform dimension. + /// + private const int UnitQuantizationShift = 2; + + /// + /// Reconstructs a lossless transform block into eight-bit sample storage. + /// + /// The sixteen dequantized coefficients in raster order. + /// The predicted samples read by reconstruction. + /// The number of read samples between rows. + /// The destination reconstructed samples. + /// The number of destination samples between rows. + /// The decoded coefficient end position. + /// The reusable transform workspace. + public static void TransformAdd( + ReadOnlySpan coefficients, + Span readBuffer, + int readStride, + Span writeBuffer, + int writeStride, + int coefficientCount, + Span workspace) + => TransformAdd( + coefficients, + readBuffer, + readStride, + writeBuffer, + writeStride, + coefficientCount, + workspace, + 8); + + /// + /// Reconstructs a lossless transform block into high-bit-depth sample storage. + /// + /// The sixteen dequantized coefficients in raster order. + /// The predicted samples read by reconstruction. + /// The number of read samples between rows. + /// The destination reconstructed samples. + /// The number of destination samples between rows. + /// The decoded coefficient end position. + /// The reusable transform workspace. + /// The coded sample bit depth. + public static void TransformAdd( + ReadOnlySpan coefficients, + Span readBuffer, + int readStride, + Span writeBuffer, + int writeStride, + int coefficientCount, + Span workspace, + int bitDepth) + => TransformAdd( + coefficients, + readBuffer, + readStride, + writeBuffer, + writeStride, + coefficientCount, + workspace, + bitDepth); + + /// + /// Selects the packed or scalar four-by-four reconstruction path. + /// + private static void TransformAdd( + ReadOnlySpan coefficients, + Span readBuffer, + int readStride, + Span writeBuffer, + int writeStride, + int coefficientCount, + Span workspace, + int bitDepth) + where TSample : unmanaged + where TOutputOperator : struct, Av1InverseTransformer.IAv1InverseTransformOutputOperator + { + if (Vector128.IsHardwareAccelerated) + { + TransformVector(coefficients, readBuffer, readStride, writeBuffer, writeStride, coefficientCount, bitDepth); + return; + } + + TransformScalar(coefficients, readBuffer, readStride, writeBuffer, writeStride, coefficientCount, workspace, bitDepth); + } + + /// + /// Applies both reversible transform dimensions to four packed coefficient rows. + /// + private static void TransformVector( + ReadOnlySpan coefficients, + Span readBuffer, + int readStride, + Span writeBuffer, + int writeStride, + int coefficientCount, + int bitDepth) + where TSample : unmanaged + where TOutputOperator : struct, Av1InverseTransformer.IAv1InverseTransformOutputOperator + { + ref int coefficientBase = ref MemoryMarshal.GetReference(coefficients); + Vector128 row0; + Vector128 row1; + Vector128 row2; + Vector128 row3; + + if (coefficientCount == 1) + { + // The DC-only form bypasses fifteen known-zero coefficients and both full butterflies. These divisions + // deliberately use arithmetic shifts because negative coefficients must round toward negative infinity. + int first = Unsafe.Add(ref coefficientBase, 0) >> UnitQuantizationShift; + int half = first >> 1; + Vector128 intermediate = Vector128.Create(first - half, half, half, half); + + row1 = intermediate >> 1; + row0 = intermediate - row1; + row2 = row1; + row3 = row1; + } + else + { + row0 = Vector128.LoadUnsafe(ref coefficientBase) >> UnitQuantizationShift; + row1 = Vector128.LoadUnsafe(ref coefficientBase, 4) >> UnitQuantizationShift; + row2 = Vector128.LoadUnsafe(ref coefficientBase, 8) >> UnitQuantizationShift; + row3 = Vector128.LoadUnsafe(ref coefficientBase, 12) >> UnitQuantizationShift; + + // Entropy decoding normalizes AV1's column-major coefficient positions to the row-major transform + // workspace. Restore the normative dimension order before either reversible butterfly performs its + // signed half shift; swapping the dimensions after those shifts would not preserve lossless rounding. + Av1Transform2dOperations.Transpose(ref row0, ref row1, ref row2, ref row3); + Transform(ref row0, ref row1, ref row2, ref row3); + + // The first pass produces four packed intermediate columns. Transposition turns those columns into rows + // so the same reversible butterfly implements the second dimension without scratch. + Av1Transform2dOperations.Transpose(ref row0, ref row1, ref row2, ref row3); + Transform(ref row0, ref row1, ref row2, ref row3); + } + + AddRows(readBuffer, readStride, writeBuffer, writeStride, row0, row1, row2, row3, bitDepth); + } + + /// + /// Applies both reversible transform dimensions without hardware intrinsics. + /// + private static void TransformScalar( + ReadOnlySpan coefficients, + Span readBuffer, + int readStride, + Span writeBuffer, + int writeStride, + int coefficientCount, + Span workspace, + int bitDepth) + where TSample : unmanaged + where TOutputOperator : struct, Av1InverseTransformer.IAv1InverseTransformOutputOperator + { + ref TSample readBase = ref MemoryMarshal.GetReference(readBuffer); + ref TSample writeBase = ref MemoryMarshal.GetReference(writeBuffer); + + if (coefficientCount == 1) + { + int first = coefficients[0] >> UnitQuantizationShift; + int half = first >> 1; + int firstResidual = first - half; + + for (int column = 0; column < 4; column++) + { + int intermediate = column == 0 ? firstResidual : half; + int repeatedResidual = intermediate >> 1; + int topResidual = intermediate - repeatedResidual; + + Unsafe.Add(ref writeBase, column) = TOutputOperator.Add(Unsafe.Add(ref readBase, column), topResidual, bitDepth); + + for (int row = 1; row < 4; row++) + { + int readOffset = (row * readStride) + column; + int writeOffset = (row * writeStride) + column; + Unsafe.Add(ref writeBase, writeOffset) = TOutputOperator.Add(Unsafe.Add(ref readBase, readOffset), repeatedResidual, bitDepth); + } + } + + return; + } + + ref int coefficientBase = ref MemoryMarshal.GetReference(coefficients); + ref int intermediateBase = ref MemoryMarshal.GetReference(workspace); + + // Entropy decoding stores the transposed scan in row-major order, so each contiguous local row is one + // normative transform column. Writing those results down the intermediate columns preserves the reference decoder's + // dimension order without a separate transpose or per-block allocation. + for (int row = 0; row < 4; row++) + { + int coefficientOffset = row * 4; + int a = Unsafe.Add(ref coefficientBase, coefficientOffset) >> UnitQuantizationShift; + int c = Unsafe.Add(ref coefficientBase, coefficientOffset + 1) >> UnitQuantizationShift; + int d = Unsafe.Add(ref coefficientBase, coefficientOffset + 2) >> UnitQuantizationShift; + int b = Unsafe.Add(ref coefficientBase, coefficientOffset + 3) >> UnitQuantizationShift; + + Transform(ref a, ref b, ref c, ref d); + Unsafe.Add(ref intermediateBase, row) = a; + Unsafe.Add(ref intermediateBase, 4 + row) = b; + Unsafe.Add(ref intermediateBase, 8 + row) = c; + Unsafe.Add(ref intermediateBase, 12 + row) = d; + } + + for (int column = 0; column < 4; column++) + { + int intermediateOffset = column * 4; + int a = Unsafe.Add(ref intermediateBase, intermediateOffset); + int c = Unsafe.Add(ref intermediateBase, intermediateOffset + 1); + int d = Unsafe.Add(ref intermediateBase, intermediateOffset + 2); + int b = Unsafe.Add(ref intermediateBase, intermediateOffset + 3); + + Transform(ref a, ref b, ref c, ref d); + + Unsafe.Add(ref writeBase, column) = TOutputOperator.Add(Unsafe.Add(ref readBase, column), a, bitDepth); + Unsafe.Add(ref writeBase, writeStride + column) = TOutputOperator.Add(Unsafe.Add(ref readBase, readStride + column), b, bitDepth); + Unsafe.Add(ref writeBase, (2 * writeStride) + column) = TOutputOperator.Add(Unsafe.Add(ref readBase, (2 * readStride) + column), c, bitDepth); + Unsafe.Add(ref writeBase, (3 * writeStride) + column) = TOutputOperator.Add(Unsafe.Add(ref readBase, (3 * readStride) + column), d, bitDepth); + } + } + + /// + /// Applies one packed four-point reversible Walsh-Hadamard dimension. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static void Transform( + ref Vector128 row0, + ref Vector128 row1, + ref Vector128 row2, + ref Vector128 row3) + { + Vector128 a = row0; + Vector128 c = row1; + Vector128 d = row2; + Vector128 b = row3; + + a += c; + d -= b; + Vector128 middle = (a - d) >> 1; + b = middle - b; + c = middle - c; + a -= b; + d += c; + + // The transform's arithmetic names the fourth input b and the second input c. Restore raster row order + // explicitly so the transpose and packed output stages see a, b, c, d exactly as the reference does. + row0 = a; + row1 = b; + row2 = c; + row3 = d; + } + + /// + /// Applies one scalar four-point reversible Walsh-Hadamard dimension. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static void Transform(ref int a, ref int b, ref int c, ref int d) + { + a += c; + d -= b; + int middle = (a - d) >> 1; + b = middle - b; + c = middle - c; + a -= b; + d += c; + } + + /// + /// Adds four packed residual rows to their prediction rows through the active sample operator. + /// + private static void AddRows( + Span readBuffer, + int readStride, + Span writeBuffer, + int writeStride, + Vector128 row0, + Vector128 row1, + Vector128 row2, + Vector128 row3, + int bitDepth) + where TSample : unmanaged + where TOutputOperator : struct, Av1InverseTransformer.IAv1InverseTransformOutputOperator + { + ref TSample readBase = ref MemoryMarshal.GetReference(readBuffer); + ref TSample writeBase = ref MemoryMarshal.GetReference(writeBuffer); + TOutputOperator.Add(ref readBase, ref writeBase, row0, bitDepth); + TOutputOperator.Add(ref Unsafe.Add(ref readBase, readStride), ref Unsafe.Add(ref writeBase, writeStride), row1, bitDepth); + TOutputOperator.Add(ref Unsafe.Add(ref readBase, 2 * readStride), ref Unsafe.Add(ref writeBase, 2 * writeStride), row2, bitDepth); + TOutputOperator.Add(ref Unsafe.Add(ref readBase, 3 * readStride), ref Unsafe.Add(ref writeBase, 3 * writeStride), row3, bitDepth); + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Transform/Av1ScanOrder.cs b/src/ImageSharp/Formats/Heif/Av1/Transform/Av1ScanOrder.cs new file mode 100644 index 0000000000..7c056a8b9c --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Transform/Av1ScanOrder.cs @@ -0,0 +1,64 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +/// +/// Describes the forward scan, inverse scan, and entropy-neighbor mapping for an AV1 transform size. +/// +internal readonly struct Av1ScanOrder +{ + /// + /// The coefficient positions in coded traversal order. + /// + private readonly short[] scan; + + /// + /// The coded position of each raster-order coefficient. + /// + private readonly short[] inverseScan; + + /// + /// The coefficient-neighbor mapping used to derive entropy contexts. + /// + private readonly short[] neighbors; + + /// + /// Initializes a new instance of the struct when only coefficient traversal is required. + /// + /// The coefficient positions in coded traversal order. + public Av1ScanOrder(short[] scan) + { + this.scan = scan; + this.inverseScan = []; + this.neighbors = []; + } + + /// + /// Initializes a new instance of the struct with complete entropy-context mappings. + /// + /// The coefficient positions in coded traversal order. + /// The coded position of each raster-order coefficient. + /// The coefficient neighbors used to derive entropy contexts. + public Av1ScanOrder(short[] scan, short[] inverseScan, short[] neighbors) + { + this.scan = scan; + this.inverseScan = inverseScan; + this.neighbors = neighbors; + } + + /// + /// Gets the coefficient positions in coded traversal order. + /// + public ReadOnlySpan Scan => this.scan; + + /// + /// Gets the coded position of each raster-order coefficient. + /// + public ReadOnlySpan InverseScan => this.inverseScan; + + /// + /// Gets the coefficient-neighbor mapping used for entropy contexts. + /// + public ReadOnlySpan Neighbors => this.neighbors; +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Transform/Av1ScanOrderConstants.cs b/src/ImageSharp/Formats/Heif/Av1/Transform/Av1ScanOrderConstants.cs new file mode 100644 index 0000000000..8e4154a9d2 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Transform/Av1ScanOrderConstants.cs @@ -0,0 +1,1307 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +/// +/// Contains the normative coefficient scan orders and quantization-matrix dimensions for AV1 transform blocks. +/// +internal static class Av1ScanOrderConstants +{ + /// + /// The number of bits used to signal a quantization-matrix level. + /// + public const int QuantizationMatrixLevelBitCount = 4; + + /// + /// The number of quantization-matrix levels, including the flat matrix. + /// + public const int QuantizationMatrixLevelCount = 1 << QuantizationMatrixLevelBitCount; + + /// + /// Maps default scan positions to raster coefficient indices for a 4x4 transform. + /// + private static readonly short[] DefaultScan4x4 = [0, 1, 4, 8, 5, 2, 3, 6, 9, 12, 13, 10, 7, 11, 14, 15]; + + /// + /// Maps default scan positions to raster coefficient indices for an 8x8 transform. + /// + private static readonly short[] DefaultScan8x8 = [ + 0, 1, 8, 16, 9, 2, 3, 10, 17, 24, 32, 25, 18, 11, 4, 5, 12, 19, 26, 33, 40, 48, + 41, 34, 27, 20, 13, 6, 7, 14, 21, 28, 35, 42, 49, 56, 57, 50, 43, 36, 29, 22, 15, 23, + 30, 37, 44, 51, 58, 59, 52, 45, 38, 31, 39, 46, 53, 60, 61, 54, 47, 55, 62, 63]; + + /// + /// Maps default scan positions to raster coefficient indices for a 16x16 transform. + /// + private static readonly short[] DefaultScan16x16 = [ + 0, 1, 16, 32, 17, 2, 3, 18, 33, 48, 64, 49, 34, 19, 4, 5, 20, 35, 50, 65, 80, 96, + 81, 66, 51, 36, 21, 6, 7, 22, 37, 52, 67, 82, 97, 112, 128, 113, 98, 83, 68, 53, 38, 23, + 8, 9, 24, 39, 54, 69, 84, 99, 114, 129, 144, 160, 145, 130, 115, 100, 85, 70, 55, 40, 25, 10, + 11, 26, 41, 56, 71, 86, 101, 116, 131, 146, 161, 176, 192, 177, 162, 147, 132, 117, 102, 87, 72, 57, + 42, 27, 12, 13, 28, 43, 58, 73, 88, 103, 118, 133, 148, 163, 178, 193, 208, 224, 209, 194, 179, 164, + 149, 134, 119, 104, 89, 74, 59, 44, 29, 14, 15, 30, 45, 60, 75, 90, 105, 120, 135, 150, 165, 180, + 195, 210, 225, 240, 241, 226, 211, 196, 181, 166, 151, 136, 121, 106, 91, 76, 61, 46, 31, 47, 62, 77, + 92, 107, 122, 137, 152, 167, 182, 197, 212, 227, 242, 243, 228, 213, 198, 183, 168, 153, 138, 123, 108, 93, + 78, 63, 79, 94, 109, 124, 139, 154, 169, 184, 199, 214, 229, 244, 245, 230, 215, 200, 185, 170, 155, 140, + 125, 110, 95, 111, 126, 141, 156, 171, 186, 201, 216, 231, 246, 247, 232, 217, 202, 187, 172, 157, 142, 127, + 143, 158, 173, 188, 203, 218, 233, 248, 249, 234, 219, 204, 189, 174, 159, 175, 190, 205, 220, 235, 250, 251, + 236, 221, 206, 191, 207, 222, 237, 252, 253, 238, 223, 239, 254, 255]; + + /// + /// Maps default scan positions to raster coefficient indices for a 32x32 transform. + /// + private static readonly short[] DefaultScan32x32 = [ + 0, 1, 32, 64, 33, 2, 3, 34, 65, 96, 128, 97, 66, 35, 4, 5, 36, 67, 98, 129, 160, + 192, 161, 130, 99, 68, 37, 6, 7, 38, 69, 100, 131, 162, 193, 224, 256, 225, 194, 163, 132, 101, + 70, 39, 8, 9, 40, 71, 102, 133, 164, 195, 226, 257, 288, 320, 289, 258, 227, 196, 165, 134, 103, + 72, 41, 10, 11, 42, 73, 104, 135, 166, 197, 228, 259, 290, 321, 352, 384, 353, 322, 291, 260, 229, + 198, 167, 136, 105, 74, 43, 12, 13, 44, 75, 106, 137, 168, 199, 230, 261, 292, 323, 354, 385, 416, + 448, 417, 386, 355, 324, 293, 262, 231, 200, 169, 138, 107, 76, 45, 14, 15, 46, 77, 108, 139, 170, + 201, 232, 263, 294, 325, 356, 387, 418, 449, 480, 512, 481, 450, 419, 388, 357, 326, 295, 264, 233, 202, + 171, 140, 109, 78, 47, 16, 17, 48, 79, 110, 141, 172, 203, 234, 265, 296, 327, 358, 389, 420, 451, + 482, 513, 544, 576, 545, 514, 483, 452, 421, 390, 359, 328, 297, 266, 235, 204, 173, 142, 111, 80, 49, + 18, 19, 50, 81, 112, 143, 174, 205, 236, 267, 298, 329, 360, 391, 422, 453, 484, 515, 546, 577, 608, + 640, 609, 578, 547, 516, 485, 454, 423, 392, 361, 330, 299, 268, 237, 206, 175, 144, 113, 82, 51, 20, + 21, 52, 83, 114, 145, 176, 207, 238, 269, 300, 331, 362, 393, 424, 455, 486, 517, 548, 579, 610, 641, + 672, 704, 673, 642, 611, 580, 549, 518, 487, 456, 425, 394, 363, 332, 301, 270, 239, 208, 177, 146, 115, + 84, 53, 22, 23, 54, 85, 116, 147, 178, 209, 240, 271, 302, 333, 364, 395, 426, 457, 488, 519, 550, + 581, 612, 643, 674, 705, 736, 768, 737, 706, 675, 644, 613, 582, 551, 520, 489, 458, 427, 396, 365, 334, + 303, 272, 241, 210, 179, 148, 117, 86, 55, 24, 25, 56, 87, 118, 149, 180, 211, 242, 273, 304, 335, + 366, 397, 428, 459, 490, 521, 552, 583, 614, 645, 676, 707, 738, 769, 800, 832, 801, 770, 739, 708, 677, + 646, 615, 584, 553, 522, 491, 460, 429, 398, 367, 336, 305, 274, 243, 212, 181, 150, 119, 88, 57, 26, + 27, 58, 89, 120, 151, 182, 213, 244, 275, 306, 337, 368, 399, 430, 461, 492, 523, 554, 585, 616, 647, + 678, 709, 740, 771, 802, 833, 864, 896, 865, 834, 803, 772, 741, 710, 679, 648, 617, 586, 555, 524, 493, + 462, 431, 400, 369, 338, 307, 276, 245, 214, 183, 152, 121, 90, 59, 28, 29, 60, 91, 122, 153, 184, + 215, 246, 277, 308, 339, 370, 401, 432, 463, 494, 525, 556, 587, 618, 649, 680, 711, 742, 773, 804, 835, + 866, 897, 928, 960, 929, 898, 867, 836, 805, 774, 743, 712, 681, 650, 619, 588, 557, 526, 495, 464, 433, + 402, 371, 340, 309, 278, 247, 216, 185, 154, 123, 92, 61, 30, 31, 62, 93, 124, 155, 186, 217, 248, + 279, 310, 341, 372, 403, 434, 465, 496, 527, 558, 589, 620, 651, 682, 713, 744, 775, 806, 837, 868, 899, + 930, 961, 992, 993, 962, 931, 900, 869, 838, 807, 776, 745, 714, 683, 652, 621, 590, 559, 528, 497, 466, + 435, 404, 373, 342, 311, 280, 249, 218, 187, 156, 125, 94, 63, 95, 126, 157, 188, 219, 250, 281, 312, + 343, 374, 405, 436, 467, 498, 529, 560, 591, 622, 653, 684, 715, 746, 777, 808, 839, 870, 901, 932, 963, + 994, 995, 964, 933, 902, 871, 840, 809, 778, 747, 716, 685, 654, 623, 592, 561, 530, 499, 468, 437, 406, + 375, 344, 313, 282, 251, 220, 189, 158, 127, 159, 190, 221, 252, 283, 314, 345, 376, 407, 438, 469, 500, + 531, 562, 593, 624, 655, 686, 717, 748, 779, 810, 841, 872, 903, 934, 965, 996, 997, 966, 935, 904, 873, + 842, 811, 780, 749, 718, 687, 656, 625, 594, 563, 532, 501, 470, 439, 408, 377, 346, 315, 284, 253, 222, + 191, 223, 254, 285, 316, 347, 378, 409, 440, 471, 502, 533, 564, 595, 626, 657, 688, 719, 750, 781, 812, + 843, 874, 905, 936, 967, 998, 999, 968, 937, 906, 875, 844, 813, 782, 751, 720, 689, 658, 627, 596, 565, + 534, 503, 472, 441, 410, 379, 348, 317, 286, 255, 287, 318, 349, 380, 411, 442, 473, 504, 535, 566, 597, + 628, 659, 690, 721, 752, 783, 814, 845, 876, 907, 938, 969, 1000, 1001, 970, 939, 908, 877, 846, 815, 784, + 753, 722, 691, 660, 629, 598, 567, 536, 505, 474, 443, 412, 381, 350, 319, 351, 382, 413, 444, 475, 506, + 537, 568, 599, 630, 661, 692, 723, 754, 785, 816, 847, 878, 909, 940, 971, 1002, 1003, 972, 941, 910, 879, + 848, 817, 786, 755, 724, 693, 662, 631, 600, 569, 538, 507, 476, 445, 414, 383, 415, 446, 477, 508, 539, + 570, 601, 632, 663, 694, 725, 756, 787, 818, 849, 880, 911, 942, 973, 1004, 1005, 974, 943, 912, 881, 850, + 819, 788, 757, 726, 695, 664, 633, 602, 571, 540, 509, 478, 447, 479, 510, 541, 572, 603, 634, 665, 696, + 727, 758, 789, 820, 851, 882, 913, 944, 975, 1006, 1007, 976, 945, 914, 883, 852, 821, 790, 759, 728, 697, + 666, 635, 604, 573, 542, 511, 543, 574, 605, 636, 667, 698, 729, 760, 791, 822, 853, 884, 915, 946, 977, + 1008, 1009, 978, 947, 916, 885, 854, 823, 792, 761, 730, 699, 668, 637, 606, 575, 607, 638, 669, 700, 731, + 762, 793, 824, 855, 886, 917, 948, 979, 1010, 1011, 980, 949, 918, 887, 856, 825, 794, 763, 732, 701, 670, + 639, 671, 702, 733, 764, 795, 826, 857, 888, 919, 950, 981, 1012, 1013, 982, 951, 920, 889, 858, 827, 796, + 765, 734, 703, 735, 766, 797, 828, 859, 890, 921, 952, 983, 1014, 1015, 984, 953, 922, 891, 860, 829, 798, + 767, 799, 830, 861, 892, 923, 954, 985, 1016, 1017, 986, 955, 924, 893, 862, 831, 863, 894, 925, 956, 987, + 1018, 1019, 988, 957, 926, 895, 927, 958, 989, 1020, 1021, 990, 959, 991, 1022, 1023]; + + /// + /// Maps default scan positions to raster coefficient indices for a 4x8 transform. + /// + private static readonly short[] DefaultScan4x8 = [ + 0, 1, 4, 2, 5, 8, 3, 6, 9, 12, 7, 10, 13, 16, 11, 14, + 17, 20, 15, 18, 21, 24, 19, 22, 25, 28, 23, 26, 29, 27, 30, 31,]; + + /// + /// Maps default scan positions to raster coefficient indices for an 8x4 transform. + /// + private static readonly short[] DefaultScan8x4 = [ + 0, 8, 1, 16, 9, 2, 24, 17, 10, 3, 25, 18, 11, 4, 26, 19, + 12, 5, 27, 20, 13, 6, 28, 21, 14, 7, 29, 22, 15, 30, 23, 31,]; + + /// + /// Maps default scan positions to raster coefficient indices for an 8x16 transform. + /// + private static readonly short[] DefaultScan8x16 = [ + 0, 1, 8, 2, 9, 16, 3, 10, 17, 24, 4, 11, 18, 25, 32, 5, 12, 19, 26, 33, 40, 6, + 13, 20, 27, 34, 41, 48, 7, 14, 21, 28, 35, 42, 49, 56, 15, 22, 29, 36, 43, 50, 57, 64, + 23, 30, 37, 44, 51, 58, 65, 72, 31, 38, 45, 52, 59, 66, 73, 80, 39, 46, 53, 60, 67, 74, + 81, 88, 47, 54, 61, 68, 75, 82, 89, 96, 55, 62, 69, 76, 83, 90, 97, 104, 63, 70, 77, 84, + 91, 98, 105, 112, 71, 78, 85, 92, 99, 106, 113, 120, 79, 86, 93, 100, 107, 114, 121, 87, 94, 101, + 108, 115, 122, 95, 102, 109, 116, 123, 103, 110, 117, 124, 111, 118, 125, 119, 126, 127,]; + + /// + /// Maps default scan positions to raster coefficient indices for a 16x8 transform. + /// + private static readonly short[] DefaultScan16x8 = [ + 0, 16, 1, 32, 17, 2, 48, 33, 18, 3, 64, 49, 34, 19, 4, 80, 65, 50, 35, 20, 5, 96, + 81, 66, 51, 36, 21, 6, 112, 97, 82, 67, 52, 37, 22, 7, 113, 98, 83, 68, 53, 38, 23, 8, + 114, 99, 84, 69, 54, 39, 24, 9, 115, 100, 85, 70, 55, 40, 25, 10, 116, 101, 86, 71, 56, 41, + 26, 11, 117, 102, 87, 72, 57, 42, 27, 12, 118, 103, 88, 73, 58, 43, 28, 13, 119, 104, 89, 74, + 59, 44, 29, 14, 120, 105, 90, 75, 60, 45, 30, 15, 121, 106, 91, 76, 61, 46, 31, 122, 107, 92, + 77, 62, 47, 123, 108, 93, 78, 63, 124, 109, 94, 79, 125, 110, 95, 126, 111, 127,]; + + /// + /// Maps default scan positions to raster coefficient indices for a 16x32 transform. + /// + private static readonly short[] DefaultScan16x32 = [ + 0, 1, 16, 2, 17, 32, 3, 18, 33, 48, 4, 19, 34, 49, 64, 5, 20, 35, 50, 65, 80, 6, 21, + 36, 51, 66, 81, 96, 7, 22, 37, 52, 67, 82, 97, 112, 8, 23, 38, 53, 68, 83, 98, 113, 128, 9, + 24, 39, 54, 69, 84, 99, 114, 129, 144, 10, 25, 40, 55, 70, 85, 100, 115, 130, 145, 160, 11, 26, 41, + 56, 71, 86, 101, 116, 131, 146, 161, 176, 12, 27, 42, 57, 72, 87, 102, 117, 132, 147, 162, 177, 192, 13, + 28, 43, 58, 73, 88, 103, 118, 133, 148, 163, 178, 193, 208, 14, 29, 44, 59, 74, 89, 104, 119, 134, 149, + 164, 179, 194, 209, 224, 15, 30, 45, 60, 75, 90, 105, 120, 135, 150, 165, 180, 195, 210, 225, 240, 31, 46, + 61, 76, 91, 106, 121, 136, 151, 166, 181, 196, 211, 226, 241, 256, 47, 62, 77, 92, 107, 122, 137, 152, 167, + 182, 197, 212, 227, 242, 257, 272, 63, 78, 93, 108, 123, 138, 153, 168, 183, 198, 213, 228, 243, 258, 273, 288, + 79, 94, 109, 124, 139, 154, 169, 184, 199, 214, 229, 244, 259, 274, 289, 304, 95, 110, 125, 140, 155, 170, 185, + 200, 215, 230, 245, 260, 275, 290, 305, 320, 111, 126, 141, 156, 171, 186, 201, 216, 231, 246, 261, 276, 291, 306, + 321, 336, 127, 142, 157, 172, 187, 202, 217, 232, 247, 262, 277, 292, 307, 322, 337, 352, 143, 158, 173, 188, 203, + 218, 233, 248, 263, 278, 293, 308, 323, 338, 353, 368, 159, 174, 189, 204, 219, 234, 249, 264, 279, 294, 309, 324, + 339, 354, 369, 384, 175, 190, 205, 220, 235, 250, 265, 280, 295, 310, 325, 340, 355, 370, 385, 400, 191, 206, 221, + 236, 251, 266, 281, 296, 311, 326, 341, 356, 371, 386, 401, 416, 207, 222, 237, 252, 267, 282, 297, 312, 327, 342, + 357, 372, 387, 402, 417, 432, 223, 238, 253, 268, 283, 298, 313, 328, 343, 358, 373, 388, 403, 418, 433, 448, 239, + 254, 269, 284, 299, 314, 329, 344, 359, 374, 389, 404, 419, 434, 449, 464, 255, 270, 285, 300, 315, 330, 345, 360, + 375, 390, 405, 420, 435, 450, 465, 480, 271, 286, 301, 316, 331, 346, 361, 376, 391, 406, 421, 436, 451, 466, 481, + 496, 287, 302, 317, 332, 347, 362, 377, 392, 407, 422, 437, 452, 467, 482, 497, 303, 318, 333, 348, 363, 378, 393, + 408, 423, 438, 453, 468, 483, 498, 319, 334, 349, 364, 379, 394, 409, 424, 439, 454, 469, 484, 499, 335, 350, 365, + 380, 395, 410, 425, 440, 455, 470, 485, 500, 351, 366, 381, 396, 411, 426, 441, 456, 471, 486, 501, 367, 382, 397, + 412, 427, 442, 457, 472, 487, 502, 383, 398, 413, 428, 443, 458, 473, 488, 503, 399, 414, 429, 444, 459, 474, 489, + 504, 415, 430, 445, 460, 475, 490, 505, 431, 446, 461, 476, 491, 506, 447, 462, 477, 492, 507, 463, 478, 493, 508, + 479, 494, 509, 495, 510, 511,]; + + /// + /// Maps default scan positions to raster coefficient indices for a 32x16 transform. + /// + private static readonly short[] DefaultScan32x16 = [ + 0, 32, 1, 64, 33, 2, 96, 65, 34, 3, 128, 97, 66, 35, 4, 160, 129, 98, 67, 36, 5, 192, 161, + 130, 99, 68, 37, 6, 224, 193, 162, 131, 100, 69, 38, 7, 256, 225, 194, 163, 132, 101, 70, 39, 8, 288, + 257, 226, 195, 164, 133, 102, 71, 40, 9, 320, 289, 258, 227, 196, 165, 134, 103, 72, 41, 10, 352, 321, 290, + 259, 228, 197, 166, 135, 104, 73, 42, 11, 384, 353, 322, 291, 260, 229, 198, 167, 136, 105, 74, 43, 12, 416, + 385, 354, 323, 292, 261, 230, 199, 168, 137, 106, 75, 44, 13, 448, 417, 386, 355, 324, 293, 262, 231, 200, 169, + 138, 107, 76, 45, 14, 480, 449, 418, 387, 356, 325, 294, 263, 232, 201, 170, 139, 108, 77, 46, 15, 481, 450, + 419, 388, 357, 326, 295, 264, 233, 202, 171, 140, 109, 78, 47, 16, 482, 451, 420, 389, 358, 327, 296, 265, 234, + 203, 172, 141, 110, 79, 48, 17, 483, 452, 421, 390, 359, 328, 297, 266, 235, 204, 173, 142, 111, 80, 49, 18, + 484, 453, 422, 391, 360, 329, 298, 267, 236, 205, 174, 143, 112, 81, 50, 19, 485, 454, 423, 392, 361, 330, 299, + 268, 237, 206, 175, 144, 113, 82, 51, 20, 486, 455, 424, 393, 362, 331, 300, 269, 238, 207, 176, 145, 114, 83, + 52, 21, 487, 456, 425, 394, 363, 332, 301, 270, 239, 208, 177, 146, 115, 84, 53, 22, 488, 457, 426, 395, 364, + 333, 302, 271, 240, 209, 178, 147, 116, 85, 54, 23, 489, 458, 427, 396, 365, 334, 303, 272, 241, 210, 179, 148, + 117, 86, 55, 24, 490, 459, 428, 397, 366, 335, 304, 273, 242, 211, 180, 149, 118, 87, 56, 25, 491, 460, 429, + 398, 367, 336, 305, 274, 243, 212, 181, 150, 119, 88, 57, 26, 492, 461, 430, 399, 368, 337, 306, 275, 244, 213, + 182, 151, 120, 89, 58, 27, 493, 462, 431, 400, 369, 338, 307, 276, 245, 214, 183, 152, 121, 90, 59, 28, 494, + 463, 432, 401, 370, 339, 308, 277, 246, 215, 184, 153, 122, 91, 60, 29, 495, 464, 433, 402, 371, 340, 309, 278, + 247, 216, 185, 154, 123, 92, 61, 30, 496, 465, 434, 403, 372, 341, 310, 279, 248, 217, 186, 155, 124, 93, 62, + 31, 497, 466, 435, 404, 373, 342, 311, 280, 249, 218, 187, 156, 125, 94, 63, 498, 467, 436, 405, 374, 343, 312, + 281, 250, 219, 188, 157, 126, 95, 499, 468, 437, 406, 375, 344, 313, 282, 251, 220, 189, 158, 127, 500, 469, 438, + 407, 376, 345, 314, 283, 252, 221, 190, 159, 501, 470, 439, 408, 377, 346, 315, 284, 253, 222, 191, 502, 471, 440, + 409, 378, 347, 316, 285, 254, 223, 503, 472, 441, 410, 379, 348, 317, 286, 255, 504, 473, 442, 411, 380, 349, 318, + 287, 505, 474, 443, 412, 381, 350, 319, 506, 475, 444, 413, 382, 351, 507, 476, 445, 414, 383, 508, 477, 446, 415, + 509, 478, 447, 510, 479, 511,]; + + /// + /// Maps default scan positions to raster coefficient indices for a 4x16 transform. + /// + private static readonly short[] DefaultScan4x16 = [ + 0, 1, 4, 2, 5, 8, 3, 6, 9, 12, 7, 10, 13, 16, 11, 14, 17, 20, 15, 18, 21, 24, + 19, 22, 25, 28, 23, 26, 29, 32, 27, 30, 33, 36, 31, 34, 37, 40, 35, 38, 41, 44, 39, 42, + 45, 48, 43, 46, 49, 52, 47, 50, 53, 56, 51, 54, 57, 60, 55, 58, 61, 59, 62, 63,]; + + /// + /// Maps default scan positions to raster coefficient indices for a 16x4 transform. + /// + private static readonly short[] DefaultScan16x4 = [ + 0, 16, 1, 32, 17, 2, 48, 33, 18, 3, 49, 34, 19, 4, 50, 35, 20, 5, 51, 36, 21, 6, + 52, 37, 22, 7, 53, 38, 23, 8, 54, 39, 24, 9, 55, 40, 25, 10, 56, 41, 26, 11, 57, 42, + 27, 12, 58, 43, 28, 13, 59, 44, 29, 14, 60, 45, 30, 15, 61, 46, 31, 62, 47, 63,]; + + /// + /// Maps default scan positions to raster coefficient indices for an 8x32 transform. + /// + private static readonly short[] DefaultScan8x32 = [ + 0, 1, 8, 2, 9, 16, 3, 10, 17, 24, 4, 11, 18, 25, 32, 5, 12, 19, 26, 33, 40, 6, + 13, 20, 27, 34, 41, 48, 7, 14, 21, 28, 35, 42, 49, 56, 15, 22, 29, 36, 43, 50, 57, 64, + 23, 30, 37, 44, 51, 58, 65, 72, 31, 38, 45, 52, 59, 66, 73, 80, 39, 46, 53, 60, 67, 74, + 81, 88, 47, 54, 61, 68, 75, 82, 89, 96, 55, 62, 69, 76, 83, 90, 97, 104, 63, 70, 77, 84, + 91, 98, 105, 112, 71, 78, 85, 92, 99, 106, 113, 120, 79, 86, 93, 100, 107, 114, 121, 128, 87, 94, + 101, 108, 115, 122, 129, 136, 95, 102, 109, 116, 123, 130, 137, 144, 103, 110, 117, 124, 131, 138, 145, 152, + 111, 118, 125, 132, 139, 146, 153, 160, 119, 126, 133, 140, 147, 154, 161, 168, 127, 134, 141, 148, 155, 162, + 169, 176, 135, 142, 149, 156, 163, 170, 177, 184, 143, 150, 157, 164, 171, 178, 185, 192, 151, 158, 165, 172, + 179, 186, 193, 200, 159, 166, 173, 180, 187, 194, 201, 208, 167, 174, 181, 188, 195, 202, 209, 216, 175, 182, + 189, 196, 203, 210, 217, 224, 183, 190, 197, 204, 211, 218, 225, 232, 191, 198, 205, 212, 219, 226, 233, 240, + 199, 206, 213, 220, 227, 234, 241, 248, 207, 214, 221, 228, 235, 242, 249, 215, 222, 229, 236, 243, 250, 223, + 230, 237, 244, 251, 231, 238, 245, 252, 239, 246, 253, 247, 254, 255,]; + + /// + /// Maps default scan positions to raster coefficient indices for a 32x8 transform. + /// + private static readonly short[] DefaultScan32x8 = [ + 0, 32, 1, 64, 33, 2, 96, 65, 34, 3, 128, 97, 66, 35, 4, 160, 129, 98, 67, 36, 5, 192, + 161, 130, 99, 68, 37, 6, 224, 193, 162, 131, 100, 69, 38, 7, 225, 194, 163, 132, 101, 70, 39, 8, + 226, 195, 164, 133, 102, 71, 40, 9, 227, 196, 165, 134, 103, 72, 41, 10, 228, 197, 166, 135, 104, 73, + 42, 11, 229, 198, 167, 136, 105, 74, 43, 12, 230, 199, 168, 137, 106, 75, 44, 13, 231, 200, 169, 138, + 107, 76, 45, 14, 232, 201, 170, 139, 108, 77, 46, 15, 233, 202, 171, 140, 109, 78, 47, 16, 234, 203, + 172, 141, 110, 79, 48, 17, 235, 204, 173, 142, 111, 80, 49, 18, 236, 205, 174, 143, 112, 81, 50, 19, + 237, 206, 175, 144, 113, 82, 51, 20, 238, 207, 176, 145, 114, 83, 52, 21, 239, 208, 177, 146, 115, 84, + 53, 22, 240, 209, 178, 147, 116, 85, 54, 23, 241, 210, 179, 148, 117, 86, 55, 24, 242, 211, 180, 149, + 118, 87, 56, 25, 243, 212, 181, 150, 119, 88, 57, 26, 244, 213, 182, 151, 120, 89, 58, 27, 245, 214, + 183, 152, 121, 90, 59, 28, 246, 215, 184, 153, 122, 91, 60, 29, 247, 216, 185, 154, 123, 92, 61, 30, + 248, 217, 186, 155, 124, 93, 62, 31, 249, 218, 187, 156, 125, 94, 63, 250, 219, 188, 157, 126, 95, 251, + 220, 189, 158, 127, 252, 221, 190, 159, 253, 222, 191, 254, 223, 255,]; + + /// + /// Maps column-oriented scan positions to raster coefficient indices for a 4x4 transform. + /// + private static readonly short[] MatrixColumnScan4x4 = [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15]; + + /// + /// Maps column-oriented scan positions to raster coefficient indices for an 8x8 transform. + /// + private static readonly short[] MatrixColumnScan8x8 = [ + 0, 8, 16, 24, 32, 40, 48, 56, 1, 9, 17, 25, 33, 41, 49, 57, 2, 10, 18, 26, 34, 42, + 50, 58, 3, 11, 19, 27, 35, 43, 51, 59, 4, 12, 20, 28, 36, 44, 52, 60, 5, 13, 21, 29, + 37, 45, 53, 61, 6, 14, 22, 30, 38, 46, 54, 62, 7, 15, 23, 31, 39, 47, 55, 63]; + + /// + /// Maps column-oriented scan positions to raster coefficient indices for a 16x16 transform. + /// + private static readonly short[] MatrixColumnScan16x16 = [ + 0, 16, 32, 48, 64, 80, 96, 112, 128, 144, 160, 176, 192, 208, 224, 240, 1, 17, 33, 49, 65, 81, + 97, 113, 129, 145, 161, 177, 193, 209, 225, 241, 2, 18, 34, 50, 66, 82, 98, 114, 130, 146, 162, 178, + 194, 210, 226, 242, 3, 19, 35, 51, 67, 83, 99, 115, 131, 147, 163, 179, 195, 211, 227, 243, 4, 20, + 36, 52, 68, 84, 100, 116, 132, 148, 164, 180, 196, 212, 228, 244, 5, 21, 37, 53, 69, 85, 101, 117, + 133, 149, 165, 181, 197, 213, 229, 245, 6, 22, 38, 54, 70, 86, 102, 118, 134, 150, 166, 182, 198, 214, + 230, 246, 7, 23, 39, 55, 71, 87, 103, 119, 135, 151, 167, 183, 199, 215, 231, 247, 8, 24, 40, 56, + 72, 88, 104, 120, 136, 152, 168, 184, 200, 216, 232, 248, 9, 25, 41, 57, 73, 89, 105, 121, 137, 153, + 169, 185, 201, 217, 233, 249, 10, 26, 42, 58, 74, 90, 106, 122, 138, 154, 170, 186, 202, 218, 234, 250, + 11, 27, 43, 59, 75, 91, 107, 123, 139, 155, 171, 187, 203, 219, 235, 251, 12, 28, 44, 60, 76, 92, + 108, 124, 140, 156, 172, 188, 204, 220, 236, 252, 13, 29, 45, 61, 77, 93, 109, 125, 141, 157, 173, 189, + 205, 221, 237, 253, 14, 30, 46, 62, 78, 94, 110, 126, 142, 158, 174, 190, 206, 222, 238, 254, 15, 31, + 47, 63, 79, 95, 111, 127, 143, 159, 175, 191, 207, 223, 239, 255,]; + + /// + /// Maps column-oriented scan positions to raster coefficient indices for a 4x8 transform. + /// + private static readonly short[] MatrixColumnScan4x8 = [ + 0, 4, 8, 12, 16, 20, 24, 28, 1, 5, 9, 13, 17, 21, 25, 29, + 2, 6, 10, 14, 18, 22, 26, 30, 3, 7, 11, 15, 19, 23, 27, 31,]; + + /// + /// Maps column-oriented scan positions to raster coefficient indices for an 8x4 transform. + /// + private static readonly short[] MatrixColumnScan8x4 = [ + 0, 8, 16, 24, 1, 9, 17, 25, 2, 10, 18, 26, 3, 11, 19, 27, + 4, 12, 20, 28, 5, 13, 21, 29, 6, 14, 22, 30, 7, 15, 23, 31,]; + + /// + /// Maps column-oriented scan positions to raster coefficient indices for an 8x16 transform. + /// + private static readonly short[] MatrixColumnScan8x16 = [ + 0, 8, 16, 24, 32, 40, 48, 56, 64, 72, 80, 88, 96, 104, 112, 120, 1, 9, 17, 25, 33, 41, + 49, 57, 65, 73, 81, 89, 97, 105, 113, 121, 2, 10, 18, 26, 34, 42, 50, 58, 66, 74, 82, 90, + 98, 106, 114, 122, 3, 11, 19, 27, 35, 43, 51, 59, 67, 75, 83, 91, 99, 107, 115, 123, 4, 12, + 20, 28, 36, 44, 52, 60, 68, 76, 84, 92, 100, 108, 116, 124, 5, 13, 21, 29, 37, 45, 53, 61, + 69, 77, 85, 93, 101, 109, 117, 125, 6, 14, 22, 30, 38, 46, 54, 62, 70, 78, 86, 94, 102, 110, + 118, 126, 7, 15, 23, 31, 39, 47, 55, 63, 71, 79, 87, 95, 103, 111, 119, 127,]; + + /// + /// Maps column-oriented scan positions to raster coefficient indices for a 16x8 transform. + /// + private static readonly short[] MatrixColumnScan16x8 = [ + 0, 16, 32, 48, 64, 80, 96, 112, 1, 17, 33, 49, 65, 81, 97, 113, 2, 18, 34, 50, 66, 82, + 98, 114, 3, 19, 35, 51, 67, 83, 99, 115, 4, 20, 36, 52, 68, 84, 100, 116, 5, 21, 37, 53, + 69, 85, 101, 117, 6, 22, 38, 54, 70, 86, 102, 118, 7, 23, 39, 55, 71, 87, 103, 119, 8, 24, + 40, 56, 72, 88, 104, 120, 9, 25, 41, 57, 73, 89, 105, 121, 10, 26, 42, 58, 74, 90, 106, 122, + 11, 27, 43, 59, 75, 91, 107, 123, 12, 28, 44, 60, 76, 92, 108, 124, 13, 29, 45, 61, 77, 93, + 109, 125, 14, 30, 46, 62, 78, 94, 110, 126, 15, 31, 47, 63, 79, 95, 111, 127,]; + + /// + /// Maps column-oriented scan positions to raster coefficient indices for a 4x16 transform. + /// + private static readonly short[] MatrixColumnScan4x16 = [ + 0, 4, 8, 12, 16, 20, 24, 28, 32, 36, 40, 44, 48, 52, 56, 60, 1, 5, 9, 13, 17, 21, + 25, 29, 33, 37, 41, 45, 49, 53, 57, 61, 2, 6, 10, 14, 18, 22, 26, 30, 34, 38, 42, 46, + 50, 54, 58, 62, 3, 7, 11, 15, 19, 23, 27, 31, 35, 39, 43, 47, 51, 55, 59, 63,]; + + /// + /// Maps column-oriented scan positions to raster coefficient indices for a 16x4 transform. + /// + private static readonly short[] MatrixColumnScan16x4 = [ + 0, 16, 32, 48, 1, 17, 33, 49, 2, 18, 34, 50, 3, 19, 35, 51, 4, 20, 36, 52, 5, 21, + 37, 53, 6, 22, 38, 54, 7, 23, 39, 55, 8, 24, 40, 56, 9, 25, 41, 57, 10, 26, 42, 58, + 11, 27, 43, 59, 12, 28, 44, 60, 13, 29, 45, 61, 14, 30, 46, 62, 15, 31, 47, 63,]; + + /// + /// Maps column-oriented scan positions to raster coefficient indices for an 8x32 transform. + /// + private static readonly short[] MatrixColumnScan8x32 = [ + 0, 8, 16, 24, 32, 40, 48, 56, 64, 72, 80, 88, 96, 104, 112, 120, 128, 136, 144, 152, 160, 168, + 176, 184, 192, 200, 208, 216, 224, 232, 240, 248, 1, 9, 17, 25, 33, 41, 49, 57, 65, 73, 81, 89, + 97, 105, 113, 121, 129, 137, 145, 153, 161, 169, 177, 185, 193, 201, 209, 217, 225, 233, 241, 249, 2, 10, + 18, 26, 34, 42, 50, 58, 66, 74, 82, 90, 98, 106, 114, 122, 130, 138, 146, 154, 162, 170, 178, 186, + 194, 202, 210, 218, 226, 234, 242, 250, 3, 11, 19, 27, 35, 43, 51, 59, 67, 75, 83, 91, 99, 107, + 115, 123, 131, 139, 147, 155, 163, 171, 179, 187, 195, 203, 211, 219, 227, 235, 243, 251, 4, 12, 20, 28, + 36, 44, 52, 60, 68, 76, 84, 92, 100, 108, 116, 124, 132, 140, 148, 156, 164, 172, 180, 188, 196, 204, + 212, 220, 228, 236, 244, 252, 5, 13, 21, 29, 37, 45, 53, 61, 69, 77, 85, 93, 101, 109, 117, 125, + 133, 141, 149, 157, 165, 173, 181, 189, 197, 205, 213, 221, 229, 237, 245, 253, 6, 14, 22, 30, 38, 46, + 54, 62, 70, 78, 86, 94, 102, 110, 118, 126, 134, 142, 150, 158, 166, 174, 182, 190, 198, 206, 214, 222, + 230, 238, 246, 254, 7, 15, 23, 31, 39, 47, 55, 63, 71, 79, 87, 95, 103, 111, 119, 127, 135, 143, + 151, 159, 167, 175, 183, 191, 199, 207, 215, 223, 231, 239, 247, 255,]; + + /// + /// Maps column-oriented scan positions to raster coefficient indices for a 32x8 transform. + /// + private static readonly short[] MatrixColumnScan32x8 = [ + 0, 32, 64, 96, 128, 160, 192, 224, 1, 33, 65, 97, 129, 161, 193, 225, 2, 34, 66, 98, 130, 162, 194, 226, + 3, 35, 67, 99, 131, 163, 195, 227, 4, 36, 68, 100, 132, 164, 196, 228, 5, 37, 69, 101, 133, 165, 197, 229, + 6, 38, 70, 102, 134, 166, 198, 230, 7, 39, 71, 103, 135, 167, 199, 231, 8, 40, 72, 104, 136, 168, 200, 232, + 9, 41, 73, 105, 137, 169, 201, 233, 10, 42, 74, 106, 138, 170, 202, 234, 11, 43, 75, 107, 139, 171, 203, 235, + 12, 44, 76, 108, 140, 172, 204, 236, 13, 45, 77, 109, 141, 173, 205, 237, 14, 46, 78, 110, 142, 174, 206, 238, + 15, 47, 79, 111, 143, 175, 207, 239, 16, 48, 80, 112, 144, 176, 208, 240, 17, 49, 81, 113, 145, 177, 209, 241, + 18, 50, 82, 114, 146, 178, 210, 242, 19, 51, 83, 115, 147, 179, 211, 243, 20, 52, 84, 116, 148, 180, 212, 244, + 21, 53, 85, 117, 149, 181, 213, 245, 22, 54, 86, 118, 150, 182, 214, 246, 23, 55, 87, 119, 151, 183, 215, 247, + 24, 56, 88, 120, 152, 184, 216, 248, 25, 57, 89, 121, 153, 185, 217, 249, 26, 58, 90, 122, 154, 186, 218, 250, + 27, 59, 91, 123, 155, 187, 219, 251, 28, 60, 92, 124, 156, 188, 220, 252, 29, 61, 93, 125, 157, 189, 221, 253, + 30, 62, 94, 126, 158, 190, 222, 254, 31, 63, 95, 127, 159, 191, 223, 255,]; + + /// + /// Maps row-oriented scan positions to raster coefficient indices for a 4x4 transform. + /// + private static readonly short[] MatrixRowScan4x4 = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]; + + /// + /// Maps row-oriented scan positions to raster coefficient indices for an 8x8 transform. + /// + private static readonly short[] MatrixRowScan8x8 = [ + 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, + 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, + 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63]; + + /// + /// Maps row-oriented scan positions to raster coefficient indices for a 16x16 transform. + /// + private static readonly short[] MatrixRowScan16x16 = [ + 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, + 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, + 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, + 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, + 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, + 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, + 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, + 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, + 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, + 198, 199, 200, 201, 202, 203, 204, 205, 206, 207, 208, 209, 210, 211, 212, 213, 214, 215, 216, 217, 218, 219, + 220, 221, 222, 223, 224, 225, 226, 227, 228, 229, 230, 231, 232, 233, 234, 235, 236, 237, 238, 239, 240, 241, + 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 255,]; + + /// + /// Maps row-oriented scan positions to raster coefficient indices for a 4x8 transform. + /// + private static readonly short[] MatrixRowScan4x8 = [ + 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, + 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31,]; + + /// + /// Maps row-oriented scan positions to raster coefficient indices for an 8x4 transform. + /// + private static readonly short[] MatrixRowScan8x4 = [ + 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, + 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31,]; + + /// + /// Maps row-oriented scan positions to raster coefficient indices for an 8x16 transform. + /// + private static readonly short[] MatrixRowScan8x16 = [ + 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, + 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, + 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, + 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, + 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, + 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127,]; + + /// + /// Maps row-oriented scan positions to raster coefficient indices for a 16x8 transform. + /// + private static readonly short[] MatrixRowScan16x8 = [ + 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, + 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, + 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, + 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, + 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, + 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127,]; + + /// + /// Maps row-oriented scan positions to raster coefficient indices for a 4x16 transform. + /// + private static readonly short[] MatrixRowScan4x16 = [ + 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, + 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, + 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63,]; + + /// + /// Maps row-oriented scan positions to raster coefficient indices for a 16x4 transform. + /// + private static readonly short[] MatrixRowScan16x4 = [ + 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, + 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, + 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63,]; + + /// + /// Maps row-oriented scan positions to raster coefficient indices for an 8x32 transform. + /// + private static readonly short[] MatrixRowScan8x32 = [ + 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, + 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, + 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, + 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, + 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, + 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, + 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, + 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, + 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, + 198, 199, 200, 201, 202, 203, 204, 205, 206, 207, 208, 209, 210, 211, 212, 213, 214, 215, 216, 217, 218, 219, + 220, 221, 222, 223, 224, 225, 226, 227, 228, 229, 230, 231, 232, 233, 234, 235, 236, 237, 238, 239, 240, 241, + 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 255,]; + + /// + /// Maps row-oriented scan positions to raster coefficient indices for a 32x8 transform. + /// + private static readonly short[] MatrixRowScan32x8 = [ + 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, + 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, + 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, + 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, + 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, + 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, + 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, + 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, + 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, + 198, 199, 200, 201, 202, 203, 204, 205, 206, 207, 208, 209, 210, 211, 212, 213, 214, 215, 216, 217, 218, 219, + 220, 221, 222, 223, 224, 225, 226, 227, 228, 229, 230, 231, 232, 233, 234, 235, 236, 237, 238, 239, 240, 241, + 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 255,]; + + // InverseScan is not used (yet) for AVIF coding, leave these arrays empty for now. + + /// + /// The inverse mapping of the default coefficient scan for a 4x4 transform; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] DefaultInverseScan4x4 = []; + + /// + /// The inverse mapping of the default coefficient scan for an 8x8 transform; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] DefaultInverseScan8x8 = []; + + /// + /// The inverse mapping of the default coefficient scan for a 16x16 transform; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] DefaultInverseScan16x16 = []; + + /// + /// The inverse mapping of the default coefficient scan for a 32x32 transform; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] DefaultInverseScan32x32 = []; + + /// + /// The inverse mapping of the default coefficient scan for a 4x8 transform; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] DefaultInverseScan4x8 = []; + + /// + /// The inverse mapping of the default coefficient scan for an 8x4 transform; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] DefaultInverseScan8x4 = []; + + /// + /// The inverse mapping of the default coefficient scan for an 8x16 transform; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] DefaultInverseScan8x16 = []; + + /// + /// The inverse mapping of the default coefficient scan for a 16x8 transform; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] DefaultInverseScan16x8 = []; + + /// + /// The inverse mapping of the default coefficient scan for a 16x32 transform; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] DefaultInverseScan16x32 = []; + + /// + /// The inverse mapping of the default coefficient scan for a 32x16 transform; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] DefaultInverseScan32x16 = []; + + /// + /// The inverse mapping of the default coefficient scan for a 4x16 transform; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] DefaultInverseScan4x16 = []; + + /// + /// The inverse mapping of the default coefficient scan for a 16x4 transform; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] DefaultInverseScan16x4 = []; + + /// + /// The inverse mapping of the default coefficient scan for an 8x32 transform; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] DefaultInverseScan8x32 = []; + + /// + /// The inverse mapping of the default coefficient scan for a 32x8 transform; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] DefaultInverseScan32x8 = []; + + /// + /// The inverse mapping of the column-oriented coefficient scan for a 4x4 transform; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] MatrixColumnInverseScan4x4 = []; + + /// + /// The inverse mapping of the column-oriented coefficient scan for an 8x8 transform; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] MatrixColumnInverseScan8x8 = []; + + /// + /// The inverse mapping of the column-oriented coefficient scan for a 16x16 transform; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] MatrixColumnInverseScan16x16 = []; + + /// + /// The inverse mapping of the column-oriented coefficient scan for a 32x32 transform; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] MatrixColumnInverseScan32x32 = []; + + /// + /// The inverse mapping of the column-oriented coefficient scan for a 4x8 transform; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] MatrixColumnInverseScan4x8 = []; + + /// + /// The inverse mapping of the column-oriented coefficient scan for an 8x4 transform; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] MatrixColumnInverseScan8x4 = []; + + /// + /// The inverse mapping of the column-oriented coefficient scan for an 8x16 transform; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] MatrixColumnInverseScan8x16 = []; + + /// + /// The inverse mapping of the column-oriented coefficient scan for a 16x8 transform; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] MatrixColumnInverseScan16x8 = []; + + /// + /// The inverse mapping of the column-oriented coefficient scan for a 16x32 transform; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] MatrixColumnInverseScan16x32 = []; + + /// + /// The inverse mapping of the column-oriented coefficient scan for a 32x16 transform; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] MatrixColumnInverseScan32x16 = []; + + /// + /// The inverse mapping of the column-oriented coefficient scan for a 4x16 transform; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] MatrixColumnInverseScan4x16 = []; + + /// + /// The inverse mapping of the column-oriented coefficient scan for a 16x4 transform; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] MatrixColumnInverseScan16x4 = []; + + /// + /// The inverse mapping of the column-oriented coefficient scan for an 8x32 transform; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] MatrixColumnInverseScan8x32 = []; + + /// + /// The inverse mapping of the column-oriented coefficient scan for a 32x8 transform; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] MatrixColumnInverseScan32x8 = []; + + /// + /// The inverse mapping of the row-oriented coefficient scan for a 4x4 transform; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] MatrixRowInverseScan4x4 = []; + + /// + /// The inverse mapping of the row-oriented coefficient scan for an 8x8 transform; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] MatrixRowInverseScan8x8 = []; + + /// + /// The inverse mapping of the row-oriented coefficient scan for a 16x16 transform; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] MatrixRowInverseScan16x16 = []; + + /// + /// The inverse mapping of the row-oriented coefficient scan for a 32x32 transform; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] MatrixRowInverseScan32x32 = []; + + /// + /// The inverse mapping of the row-oriented coefficient scan for a 4x8 transform; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] MatrixRowInverseScan4x8 = []; + + /// + /// The inverse mapping of the row-oriented coefficient scan for an 8x4 transform; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] MatrixRowInverseScan8x4 = []; + + /// + /// The inverse mapping of the row-oriented coefficient scan for an 8x16 transform; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] MatrixRowInverseScan8x16 = []; + + /// + /// The inverse mapping of the row-oriented coefficient scan for a 16x8 transform; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] MatrixRowInverseScan16x8 = []; + + /// + /// The inverse mapping of the row-oriented coefficient scan for a 16x32 transform; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] MatrixRowInverseScan16x32 = []; + + /// + /// The inverse mapping of the row-oriented coefficient scan for a 32x16 transform; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] MatrixRowInverseScan32x16 = []; + + /// + /// The inverse mapping of the row-oriented coefficient scan for a 4x16 transform; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] MatrixRowInverseScan4x16 = []; + + /// + /// The inverse mapping of the row-oriented coefficient scan for a 16x4 transform; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] MatrixRowInverseScan16x4 = []; + + /// + /// The inverse mapping of the row-oriented coefficient scan for an 8x32 transform; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] MatrixRowInverseScan8x32 = []; + + /// + /// The inverse mapping of the row-oriented coefficient scan for a 32x8 transform; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] MatrixRowInverseScan32x8 = []; + + // Neighbors are not used (yet) for AVIF coding, leave these arrays empty for now. + + /// + /// The entropy-neighbor lookup for the default 4x4 coefficient scan; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] DefaultScan4x4Neighbors = []; + + /// + /// The entropy-neighbor lookup for the default 8x8 coefficient scan; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] DefaultScan8x8Neighbors = []; + + /// + /// The entropy-neighbor lookup for the default 16x16 coefficient scan; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] DefaultScan16x16Neighbors = []; + + /// + /// The entropy-neighbor lookup for the default 32x32 coefficient scan; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] DefaultScan32x32Neighbors = []; + + /// + /// The entropy-neighbor lookup for the default 4x8 coefficient scan; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] DefaultScan4x8Neighbors = []; + + /// + /// The entropy-neighbor lookup for the default 8x4 coefficient scan; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] DefaultScan8x4Neighbors = []; + + /// + /// The entropy-neighbor lookup for the default 8x16 coefficient scan; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] DefaultScan8x16Neighbors = []; + + /// + /// The entropy-neighbor lookup for the default 16x8 coefficient scan; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] DefaultScan16x8Neighbors = []; + + /// + /// The entropy-neighbor lookup for the default 16x32 coefficient scan; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] DefaultScan16x32Neighbors = []; + + /// + /// The entropy-neighbor lookup for the default 32x16 coefficient scan; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] DefaultScan32x16Neighbors = []; + + /// + /// The entropy-neighbor lookup for the default 4x16 coefficient scan; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] DefaultScan4x16Neighbors = []; + + /// + /// The entropy-neighbor lookup for the default 16x4 coefficient scan; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] DefaultScan16x4Neighbors = []; + + /// + /// The entropy-neighbor lookup for the default 8x32 coefficient scan; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] DefaultScan8x32Neighbors = []; + + /// + /// The entropy-neighbor lookup for the default 32x8 coefficient scan; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] DefaultScan32x8Neighbors = []; + + /// + /// The entropy-neighbor lookup for the column-oriented 4x4 coefficient scan; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] MatrixColumnScan4x4Neighbors = []; + + /// + /// The entropy-neighbor lookup for the column-oriented 8x8 coefficient scan; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] MatrixColumnScan8x8Neighbors = []; + + /// + /// The entropy-neighbor lookup for the column-oriented 16x16 coefficient scan; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] MatrixColumnScan16x16Neighbors = []; + + /// + /// The entropy-neighbor lookup for the column-oriented 32x32 coefficient scan; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] MatrixColumnScan32x32Neighbors = []; + + /// + /// The entropy-neighbor lookup for the column-oriented 4x8 coefficient scan; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] MatrixColumnScan4x8Neighbors = []; + + /// + /// The entropy-neighbor lookup for the column-oriented 8x4 coefficient scan; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] MatrixColumnScan8x4Neighbors = []; + + /// + /// The entropy-neighbor lookup for the column-oriented 8x16 coefficient scan; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] MatrixColumnScan8x16Neighbors = []; + + /// + /// The entropy-neighbor lookup for the column-oriented 16x8 coefficient scan; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] MatrixColumnScan16x8Neighbors = []; + + /// + /// The entropy-neighbor lookup for the column-oriented 16x32 coefficient scan; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] MatrixColumnScan16x32Neighbors = []; + + /// + /// The entropy-neighbor lookup for the column-oriented 32x16 coefficient scan; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] MatrixColumnScan32x16Neighbors = []; + + /// + /// The entropy-neighbor lookup for the column-oriented 4x16 coefficient scan; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] MatrixColumnScan4x16Neighbors = []; + + /// + /// The entropy-neighbor lookup for the column-oriented 16x4 coefficient scan; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] MatrixColumnScan16x4Neighbors = []; + + /// + /// The entropy-neighbor lookup for the column-oriented 8x32 coefficient scan; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] MatrixColumnScan8x32Neighbors = []; + + /// + /// The entropy-neighbor lookup for the column-oriented 32x8 coefficient scan; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] MatrixColumnScan32x8Neighbors = []; + + /// + /// The entropy-neighbor lookup for the row-oriented 4x4 coefficient scan; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] MatrixRowScan4x4Neighbors = []; + + /// + /// The entropy-neighbor lookup for the row-oriented 8x8 coefficient scan; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] MatrixRowScan8x8Neighbors = []; + + /// + /// The entropy-neighbor lookup for the row-oriented 16x16 coefficient scan; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] MatrixRowScan16x16Neighbors = []; + + /// + /// The entropy-neighbor lookup for the row-oriented 32x32 coefficient scan; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] MatrixRowScan32x32Neighbors = []; + + /// + /// The entropy-neighbor lookup for the row-oriented 4x8 coefficient scan; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] MatrixRowScan4x8Neighbors = []; + + /// + /// The entropy-neighbor lookup for the row-oriented 8x4 coefficient scan; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] MatrixRowScan8x4Neighbors = []; + + /// + /// The entropy-neighbor lookup for the row-oriented 8x16 coefficient scan; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] MatrixRowScan8x16Neighbors = []; + + /// + /// The entropy-neighbor lookup for the row-oriented 16x8 coefficient scan; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] MatrixRowScan16x8Neighbors = []; + + /// + /// The entropy-neighbor lookup for the row-oriented 16x32 coefficient scan; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] MatrixRowScan16x32Neighbors = []; + + /// + /// The entropy-neighbor lookup for the row-oriented 32x16 coefficient scan; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] MatrixRowScan32x16Neighbors = []; + + /// + /// The entropy-neighbor lookup for the row-oriented 4x16 coefficient scan; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] MatrixRowScan4x16Neighbors = []; + + /// + /// The entropy-neighbor lookup for the row-oriented 16x4 coefficient scan; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] MatrixRowScan16x4Neighbors = []; + + /// + /// The entropy-neighbor lookup for the row-oriented 8x32 coefficient scan; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] MatrixRowScan8x32Neighbors = []; + + /// + /// The entropy-neighbor lookup for the row-oriented 32x8 coefficient scan; reserved for AV1 syntax that still-image decoding does not consume. + /// + private static readonly short[] MatrixRowScan32x8Neighbors = []; + + /// + /// Maps transform size and compound transform type to coefficient and entropy-neighbor traversal tables. + /// + private static readonly Av1ScanOrder[][] ScanOrders = + [ + + // Transform size 4x4 + [ + new(DefaultScan4x4, DefaultInverseScan4x4, DefaultScan4x4Neighbors), + new(DefaultScan4x4, DefaultInverseScan4x4, DefaultScan4x4Neighbors), + new(DefaultScan4x4, DefaultInverseScan4x4, DefaultScan4x4Neighbors), + new(DefaultScan4x4, DefaultInverseScan4x4, DefaultScan4x4Neighbors), + new(DefaultScan4x4, DefaultInverseScan4x4, DefaultScan4x4Neighbors), + new(DefaultScan4x4, DefaultInverseScan4x4, DefaultScan4x4Neighbors), + new(DefaultScan4x4, DefaultInverseScan4x4, DefaultScan4x4Neighbors), + new(DefaultScan4x4, DefaultInverseScan4x4, DefaultScan4x4Neighbors), + new(DefaultScan4x4, DefaultInverseScan4x4, DefaultScan4x4Neighbors), + new(DefaultScan4x4, DefaultInverseScan4x4, DefaultScan4x4Neighbors), + new(MatrixRowScan4x4, MatrixRowInverseScan4x4, MatrixRowScan4x4Neighbors), + new(MatrixColumnScan4x4, MatrixColumnInverseScan4x4, MatrixColumnScan4x4Neighbors), + new(MatrixRowScan4x4, MatrixRowInverseScan4x4, MatrixRowScan4x4Neighbors), + new(MatrixColumnScan4x4, MatrixColumnInverseScan4x4, MatrixColumnScan4x4Neighbors), + new(MatrixRowScan4x4, MatrixRowInverseScan4x4, MatrixRowScan4x4Neighbors), + new(MatrixColumnScan4x4, MatrixColumnInverseScan4x4, MatrixColumnScan4x4Neighbors), + ], + + // Transform size 8x8 + [ + new(DefaultScan8x8, DefaultInverseScan8x8, DefaultScan8x8Neighbors), + new(DefaultScan8x8, DefaultInverseScan8x8, DefaultScan8x8Neighbors), + new(DefaultScan8x8, DefaultInverseScan8x8, DefaultScan8x8Neighbors), + new(DefaultScan8x8, DefaultInverseScan8x8, DefaultScan8x8Neighbors), + new(DefaultScan8x8, DefaultInverseScan8x8, DefaultScan8x8Neighbors), + new(DefaultScan8x8, DefaultInverseScan8x8, DefaultScan8x8Neighbors), + new(DefaultScan8x8, DefaultInverseScan8x8, DefaultScan8x8Neighbors), + new(DefaultScan8x8, DefaultInverseScan8x8, DefaultScan8x8Neighbors), + new(DefaultScan8x8, DefaultInverseScan8x8, DefaultScan8x8Neighbors), + new(DefaultScan8x8, DefaultInverseScan8x8, DefaultScan8x8Neighbors), + new(MatrixRowScan8x8, MatrixRowInverseScan8x8, MatrixRowScan8x8Neighbors), + new(MatrixColumnScan8x8, MatrixColumnInverseScan8x8, MatrixColumnScan8x8Neighbors), + new(MatrixRowScan8x8, MatrixRowInverseScan8x8, MatrixRowScan8x8Neighbors), + new(MatrixColumnScan8x8, MatrixColumnInverseScan8x8, MatrixColumnScan8x8Neighbors), + new(MatrixRowScan8x8, MatrixRowInverseScan8x8, MatrixRowScan8x8Neighbors), + new(MatrixColumnScan8x8, MatrixColumnInverseScan8x8, MatrixColumnScan8x8Neighbors), + ], + + // Transform size 16x16 + [ + new(DefaultScan16x16, DefaultInverseScan16x16, DefaultScan16x16Neighbors), + new(DefaultScan16x16, DefaultInverseScan16x16, DefaultScan16x16Neighbors), + new(DefaultScan16x16, DefaultInverseScan16x16, DefaultScan16x16Neighbors), + new(DefaultScan16x16, DefaultInverseScan16x16, DefaultScan16x16Neighbors), + new(DefaultScan16x16, DefaultInverseScan16x16, DefaultScan16x16Neighbors), + new(DefaultScan16x16, DefaultInverseScan16x16, DefaultScan16x16Neighbors), + new(DefaultScan16x16, DefaultInverseScan16x16, DefaultScan16x16Neighbors), + new(DefaultScan16x16, DefaultInverseScan16x16, DefaultScan16x16Neighbors), + new(DefaultScan16x16, DefaultInverseScan16x16, DefaultScan16x16Neighbors), + new(DefaultScan16x16, DefaultInverseScan16x16, DefaultScan16x16Neighbors), + new(MatrixRowScan16x16, MatrixRowInverseScan16x16, MatrixRowScan16x16Neighbors), + new(MatrixColumnScan16x16, MatrixColumnInverseScan16x16, MatrixColumnScan16x16Neighbors), + new(MatrixRowScan16x16, MatrixRowInverseScan16x16, MatrixRowScan16x16Neighbors), + new(MatrixColumnScan16x16, MatrixColumnInverseScan16x16, MatrixColumnScan16x16Neighbors), + new(MatrixRowScan16x16, MatrixRowInverseScan16x16, MatrixRowScan16x16Neighbors), + new(MatrixColumnScan16x16, MatrixColumnInverseScan16x16, MatrixColumnScan16x16Neighbors), + ], + + // Transform size 32x32 + [ + new(DefaultScan32x32, DefaultInverseScan32x32, DefaultScan32x32Neighbors), + new(DefaultScan32x32, DefaultInverseScan32x32, DefaultScan32x32Neighbors), + new(DefaultScan32x32, DefaultInverseScan32x32, DefaultScan32x32Neighbors), + new(DefaultScan32x32, DefaultInverseScan32x32, DefaultScan32x32Neighbors), + new(DefaultScan32x32, DefaultInverseScan32x32, DefaultScan32x32Neighbors), + new(DefaultScan32x32, DefaultInverseScan32x32, DefaultScan32x32Neighbors), + new(DefaultScan32x32, DefaultInverseScan32x32, DefaultScan32x32Neighbors), + new(DefaultScan32x32, DefaultInverseScan32x32, DefaultScan32x32Neighbors), + new(DefaultScan32x32, DefaultInverseScan32x32, DefaultScan32x32Neighbors), + new(DefaultScan32x32, DefaultInverseScan32x32, DefaultScan32x32Neighbors), + new(DefaultScan32x32, MatrixRowInverseScan32x32, MatrixRowScan32x32Neighbors), + new(DefaultScan32x32, MatrixColumnInverseScan32x32, MatrixColumnScan32x32Neighbors), + new(DefaultScan32x32, MatrixRowInverseScan32x32, MatrixRowScan32x32Neighbors), + new(DefaultScan32x32, MatrixColumnInverseScan32x32, MatrixColumnScan32x32Neighbors), + new(DefaultScan32x32, MatrixRowInverseScan32x32, MatrixRowScan32x32Neighbors), + new(DefaultScan32x32, MatrixColumnInverseScan32x32, MatrixColumnScan32x32Neighbors), + ], + [ + + // Transform size 64X64 + // Half of the coefficients of tx64 at higher frequencies are set to + // zeros. So tx32's scan order is used. + new(DefaultScan32x32, DefaultInverseScan32x32, DefaultScan32x32Neighbors), + new(DefaultScan32x32, DefaultInverseScan32x32, DefaultScan32x32Neighbors), + new(DefaultScan32x32, DefaultInverseScan32x32, DefaultScan32x32Neighbors), + new(DefaultScan32x32, DefaultInverseScan32x32, DefaultScan32x32Neighbors), + new(DefaultScan32x32, DefaultInverseScan32x32, DefaultScan32x32Neighbors), + new(DefaultScan32x32, DefaultInverseScan32x32, DefaultScan32x32Neighbors), + new(DefaultScan32x32, DefaultInverseScan32x32, DefaultScan32x32Neighbors), + new(DefaultScan32x32, DefaultInverseScan32x32, DefaultScan32x32Neighbors), + new(DefaultScan32x32, DefaultInverseScan32x32, DefaultScan32x32Neighbors), + new(DefaultScan32x32, DefaultInverseScan32x32, DefaultScan32x32Neighbors), + new(DefaultScan32x32, MatrixRowInverseScan32x32, MatrixRowScan32x32Neighbors), + new(DefaultScan32x32, MatrixColumnInverseScan32x32, MatrixColumnScan32x32Neighbors), + new(DefaultScan32x32, MatrixRowInverseScan32x32, MatrixRowScan32x32Neighbors), + new(DefaultScan32x32, MatrixColumnInverseScan32x32, MatrixColumnScan32x32Neighbors), + new(DefaultScan32x32, MatrixRowInverseScan32x32, MatrixRowScan32x32Neighbors), + new(DefaultScan32x32, MatrixColumnInverseScan32x32, MatrixColumnScan32x32Neighbors), + ], + [ + + // Transform size 4X8 + new(DefaultScan4x8, DefaultInverseScan4x8, DefaultScan4x8Neighbors), + new(DefaultScan4x8, DefaultInverseScan4x8, DefaultScan4x8Neighbors), + new(DefaultScan4x8, DefaultInverseScan4x8, DefaultScan4x8Neighbors), + new(DefaultScan4x8, DefaultInverseScan4x8, DefaultScan4x8Neighbors), + new(DefaultScan4x8, DefaultInverseScan4x8, DefaultScan4x8Neighbors), + new(DefaultScan4x8, DefaultInverseScan4x8, DefaultScan4x8Neighbors), + new(DefaultScan4x8, DefaultInverseScan4x8, DefaultScan4x8Neighbors), + new(DefaultScan4x8, DefaultInverseScan4x8, DefaultScan4x8Neighbors), + new(DefaultScan4x8, DefaultInverseScan4x8, DefaultScan4x8Neighbors), + new(DefaultScan4x8, DefaultInverseScan4x8, DefaultScan4x8Neighbors), + new(MatrixRowScan4x8, MatrixRowInverseScan4x8, MatrixRowScan4x8Neighbors), + new(MatrixColumnScan4x8, MatrixColumnInverseScan4x8, MatrixColumnScan4x8Neighbors), + new(MatrixRowScan4x8, MatrixRowInverseScan4x8, MatrixRowScan4x8Neighbors), + new(MatrixColumnScan4x8, MatrixColumnInverseScan4x8, MatrixColumnScan4x8Neighbors), + new(MatrixRowScan4x8, MatrixRowInverseScan4x8, MatrixRowScan4x8Neighbors), + new(MatrixColumnScan4x8, MatrixColumnInverseScan4x8, MatrixColumnScan4x8Neighbors), + ], + [ + + // Transform size 8X4 + new(DefaultScan8x4, DefaultInverseScan8x4, DefaultScan8x4Neighbors), + new(DefaultScan8x4, DefaultInverseScan8x4, DefaultScan8x4Neighbors), + new(DefaultScan8x4, DefaultInverseScan8x4, DefaultScan8x4Neighbors), + new(DefaultScan8x4, DefaultInverseScan8x4, DefaultScan8x4Neighbors), + new(DefaultScan8x4, DefaultInverseScan8x4, DefaultScan8x4Neighbors), + new(DefaultScan8x4, DefaultInverseScan8x4, DefaultScan8x4Neighbors), + new(DefaultScan8x4, DefaultInverseScan8x4, DefaultScan8x4Neighbors), + new(DefaultScan8x4, DefaultInverseScan8x4, DefaultScan8x4Neighbors), + new(DefaultScan8x4, DefaultInverseScan8x4, DefaultScan8x4Neighbors), + new(DefaultScan8x4, DefaultInverseScan8x4, DefaultScan8x4Neighbors), + new(MatrixRowScan8x4, MatrixRowInverseScan8x4, MatrixRowScan8x4Neighbors), + new(MatrixColumnScan8x4, MatrixColumnInverseScan8x4, MatrixColumnScan8x4Neighbors), + new(MatrixRowScan8x4, MatrixRowInverseScan8x4, MatrixRowScan8x4Neighbors), + new(MatrixColumnScan8x4, MatrixColumnInverseScan8x4, MatrixColumnScan8x4Neighbors), + new(MatrixRowScan8x4, MatrixRowInverseScan8x4, MatrixRowScan8x4Neighbors), + new(MatrixColumnScan8x4, MatrixColumnInverseScan8x4, MatrixColumnScan8x4Neighbors), + ], + [ + + // Transform size 8X16 + new(DefaultScan8x16, DefaultInverseScan8x16, DefaultScan8x16Neighbors), + new(DefaultScan8x16, DefaultInverseScan8x16, DefaultScan8x16Neighbors), + new(DefaultScan8x16, DefaultInverseScan8x16, DefaultScan8x16Neighbors), + new(DefaultScan8x16, DefaultInverseScan8x16, DefaultScan8x16Neighbors), + new(DefaultScan8x16, DefaultInverseScan8x16, DefaultScan8x16Neighbors), + new(DefaultScan8x16, DefaultInverseScan8x16, DefaultScan8x16Neighbors), + new(DefaultScan8x16, DefaultInverseScan8x16, DefaultScan8x16Neighbors), + new(DefaultScan8x16, DefaultInverseScan8x16, DefaultScan8x16Neighbors), + new(DefaultScan8x16, DefaultInverseScan8x16, DefaultScan8x16Neighbors), + new(DefaultScan8x16, DefaultInverseScan8x16, DefaultScan8x16Neighbors), + new(MatrixRowScan8x16, MatrixRowInverseScan8x16, MatrixRowScan8x16Neighbors), + new(MatrixColumnScan8x16, MatrixColumnInverseScan8x16, MatrixColumnScan8x16Neighbors), + new(MatrixRowScan8x16, MatrixRowInverseScan8x16, MatrixRowScan8x16Neighbors), + new(MatrixColumnScan8x16, MatrixColumnInverseScan8x16, MatrixColumnScan8x16Neighbors), + new(MatrixRowScan8x16, MatrixRowInverseScan8x16, MatrixRowScan8x16Neighbors), + new(MatrixColumnScan8x16, MatrixColumnInverseScan8x16, MatrixColumnScan8x16Neighbors), + ], + [ + + // Transform size 16X8 + new(DefaultScan16x8, DefaultInverseScan16x8, DefaultScan16x8Neighbors), + new(DefaultScan16x8, DefaultInverseScan16x8, DefaultScan16x8Neighbors), + new(DefaultScan16x8, DefaultInverseScan16x8, DefaultScan16x8Neighbors), + new(DefaultScan16x8, DefaultInverseScan16x8, DefaultScan16x8Neighbors), + new(DefaultScan16x8, DefaultInverseScan16x8, DefaultScan16x8Neighbors), + new(DefaultScan16x8, DefaultInverseScan16x8, DefaultScan16x8Neighbors), + new(DefaultScan16x8, DefaultInverseScan16x8, DefaultScan16x8Neighbors), + new(DefaultScan16x8, DefaultInverseScan16x8, DefaultScan16x8Neighbors), + new(DefaultScan16x8, DefaultInverseScan16x8, DefaultScan16x8Neighbors), + new(DefaultScan16x8, DefaultInverseScan16x8, DefaultScan16x8Neighbors), + new(MatrixRowScan16x8, MatrixRowInverseScan16x8, MatrixRowScan16x8Neighbors), + new(MatrixColumnScan16x8, MatrixColumnInverseScan16x8, MatrixColumnScan16x8Neighbors), + new(MatrixRowScan16x8, MatrixRowInverseScan16x8, MatrixRowScan16x8Neighbors), + new(MatrixColumnScan16x8, MatrixColumnInverseScan16x8, MatrixColumnScan16x8Neighbors), + new(MatrixRowScan16x8, MatrixRowInverseScan16x8, MatrixRowScan16x8Neighbors), + new(MatrixColumnScan16x8, MatrixColumnInverseScan16x8, MatrixColumnScan16x8Neighbors), + ], + [ + + // Transform size 16X32 + new(DefaultScan16x32, DefaultInverseScan16x32, DefaultScan16x32Neighbors), + new(DefaultScan16x32, DefaultInverseScan16x32, DefaultScan16x32Neighbors), + new(DefaultScan16x32, DefaultInverseScan16x32, DefaultScan16x32Neighbors), + new(DefaultScan16x32, DefaultInverseScan16x32, DefaultScan16x32Neighbors), + new(DefaultScan16x32, DefaultInverseScan16x32, DefaultScan16x32Neighbors), + new(DefaultScan16x32, DefaultInverseScan16x32, DefaultScan16x32Neighbors), + new(DefaultScan16x32, DefaultInverseScan16x32, DefaultScan16x32Neighbors), + new(DefaultScan16x32, DefaultInverseScan16x32, DefaultScan16x32Neighbors), + new(DefaultScan16x32, DefaultInverseScan16x32, DefaultScan16x32Neighbors), + new(DefaultScan16x32, DefaultInverseScan16x32, DefaultScan16x32Neighbors), + new(DefaultScan16x32, MatrixRowInverseScan16x32, MatrixRowScan16x32Neighbors), + new(DefaultScan16x32, MatrixColumnInverseScan16x32, MatrixColumnScan16x32Neighbors), + new(DefaultScan16x32, MatrixRowInverseScan16x32, MatrixRowScan16x32Neighbors), + new(DefaultScan16x32, MatrixColumnInverseScan16x32, MatrixColumnScan16x32Neighbors), + new(DefaultScan16x32, MatrixRowInverseScan16x32, MatrixRowScan16x32Neighbors), + new(DefaultScan16x32, MatrixColumnInverseScan16x32, MatrixColumnScan16x32Neighbors), + ], + [ + + // Transform size 32X16 + new(DefaultScan32x16, DefaultInverseScan32x16, DefaultScan32x16Neighbors), + new(DefaultScan32x16, DefaultInverseScan32x16, DefaultScan32x16Neighbors), + new(DefaultScan32x16, DefaultInverseScan32x16, DefaultScan32x16Neighbors), + new(DefaultScan32x16, DefaultInverseScan32x16, DefaultScan32x16Neighbors), + new(DefaultScan32x16, DefaultInverseScan32x16, DefaultScan32x16Neighbors), + new(DefaultScan32x16, DefaultInverseScan32x16, DefaultScan32x16Neighbors), + new(DefaultScan32x16, DefaultInverseScan32x16, DefaultScan32x16Neighbors), + new(DefaultScan32x16, DefaultInverseScan32x16, DefaultScan32x16Neighbors), + new(DefaultScan32x16, DefaultInverseScan32x16, DefaultScan32x16Neighbors), + new(DefaultScan32x16, DefaultInverseScan32x16, DefaultScan32x16Neighbors), + new(DefaultScan32x16, MatrixRowInverseScan32x16, MatrixRowScan32x16Neighbors), + new(DefaultScan32x16, MatrixColumnInverseScan32x16, MatrixColumnScan32x16Neighbors), + new(DefaultScan32x16, MatrixRowInverseScan32x16, MatrixRowScan32x16Neighbors), + new(DefaultScan32x16, MatrixColumnInverseScan32x16, MatrixColumnScan32x16Neighbors), + new(DefaultScan32x16, MatrixRowInverseScan32x16, MatrixRowScan32x16Neighbors), + new(DefaultScan32x16, MatrixColumnInverseScan32x16, MatrixColumnScan32x16Neighbors), + ], + [ + + // Transform size 32X64 + // Half of the coefficients of tx64 at higher frequencies are set to + // zeros. So tx32's scan order is used. + new(DefaultScan32x32, DefaultInverseScan32x32, DefaultScan32x32Neighbors), + new(DefaultScan32x32, DefaultInverseScan32x32, DefaultScan32x32Neighbors), + new(DefaultScan32x32, DefaultInverseScan32x32, DefaultScan32x32Neighbors), + new(DefaultScan32x32, DefaultInverseScan32x32, DefaultScan32x32Neighbors), + new(DefaultScan32x32, DefaultInverseScan32x32, DefaultScan32x32Neighbors), + new(DefaultScan32x32, DefaultInverseScan32x32, DefaultScan32x32Neighbors), + new(DefaultScan32x32, DefaultInverseScan32x32, DefaultScan32x32Neighbors), + new(DefaultScan32x32, DefaultInverseScan32x32, DefaultScan32x32Neighbors), + new(DefaultScan32x32, DefaultInverseScan32x32, DefaultScan32x32Neighbors), + new(DefaultScan32x32, DefaultInverseScan32x32, DefaultScan32x32Neighbors), + new(DefaultScan32x32, MatrixRowInverseScan32x32, MatrixRowScan32x32Neighbors), + new(DefaultScan32x32, MatrixColumnInverseScan32x32, MatrixColumnScan32x32Neighbors), + new(DefaultScan32x32, MatrixRowInverseScan32x32, MatrixRowScan32x32Neighbors), + new(DefaultScan32x32, MatrixColumnInverseScan32x32, MatrixColumnScan32x32Neighbors), + new(DefaultScan32x32, MatrixRowInverseScan32x32, MatrixRowScan32x32Neighbors), + new(DefaultScan32x32, MatrixColumnInverseScan32x32, MatrixColumnScan32x32Neighbors), + ], + [ + + // Transform size 64X32 + // Half of the coefficients of tx64 at higher frequencies are set to + // zeros. So tx32's scan order is used. + new(DefaultScan32x32, DefaultInverseScan32x32, DefaultScan32x32Neighbors), + new(DefaultScan32x32, DefaultInverseScan32x32, DefaultScan32x32Neighbors), + new(DefaultScan32x32, DefaultInverseScan32x32, DefaultScan32x32Neighbors), + new(DefaultScan32x32, DefaultInverseScan32x32, DefaultScan32x32Neighbors), + new(DefaultScan32x32, DefaultInverseScan32x32, DefaultScan32x32Neighbors), + new(DefaultScan32x32, DefaultInverseScan32x32, DefaultScan32x32Neighbors), + new(DefaultScan32x32, DefaultInverseScan32x32, DefaultScan32x32Neighbors), + new(DefaultScan32x32, DefaultInverseScan32x32, DefaultScan32x32Neighbors), + new(DefaultScan32x32, DefaultInverseScan32x32, DefaultScan32x32Neighbors), + new(DefaultScan32x32, DefaultInverseScan32x32, DefaultScan32x32Neighbors), + new(DefaultScan32x32, MatrixRowInverseScan32x32, MatrixRowScan32x32Neighbors), + new(DefaultScan32x32, MatrixColumnInverseScan32x32, MatrixColumnScan32x32Neighbors), + new(DefaultScan32x32, MatrixRowInverseScan32x32, MatrixRowScan32x32Neighbors), + new(DefaultScan32x32, MatrixColumnInverseScan32x32, MatrixColumnScan32x32Neighbors), + new(DefaultScan32x32, MatrixRowInverseScan32x32, MatrixRowScan32x32Neighbors), + new(DefaultScan32x32, MatrixColumnInverseScan32x32, MatrixColumnScan32x32Neighbors), + ], + [ + + // Transform size 4X16 + new(DefaultScan4x16, DefaultInverseScan4x16, DefaultScan4x16Neighbors), + new(DefaultScan4x16, DefaultInverseScan4x16, DefaultScan4x16Neighbors), + new(DefaultScan4x16, DefaultInverseScan4x16, DefaultScan4x16Neighbors), + new(DefaultScan4x16, DefaultInverseScan4x16, DefaultScan4x16Neighbors), + new(DefaultScan4x16, DefaultInverseScan4x16, DefaultScan4x16Neighbors), + new(DefaultScan4x16, DefaultInverseScan4x16, DefaultScan4x16Neighbors), + new(DefaultScan4x16, DefaultInverseScan4x16, DefaultScan4x16Neighbors), + new(DefaultScan4x16, DefaultInverseScan4x16, DefaultScan4x16Neighbors), + new(DefaultScan4x16, DefaultInverseScan4x16, DefaultScan4x16Neighbors), + new(DefaultScan4x16, DefaultInverseScan4x16, DefaultScan4x16Neighbors), + new(MatrixRowScan4x16, MatrixRowInverseScan4x16, MatrixRowScan4x16Neighbors), + new(MatrixColumnScan4x16, MatrixColumnInverseScan4x16, MatrixColumnScan4x16Neighbors), + new(MatrixRowScan4x16, MatrixRowInverseScan4x16, MatrixRowScan4x16Neighbors), + new(MatrixColumnScan4x16, MatrixColumnInverseScan4x16, MatrixColumnScan4x16Neighbors), + new(MatrixRowScan4x16, MatrixRowInverseScan4x16, MatrixRowScan4x16Neighbors), + new(MatrixColumnScan4x16, MatrixColumnInverseScan4x16, MatrixColumnScan4x16Neighbors), + ], + [ + + // Transform size 16X4 + new(DefaultScan16x4, DefaultInverseScan16x4, DefaultScan16x4Neighbors), + new(DefaultScan16x4, DefaultInverseScan16x4, DefaultScan16x4Neighbors), + new(DefaultScan16x4, DefaultInverseScan16x4, DefaultScan16x4Neighbors), + new(DefaultScan16x4, DefaultInverseScan16x4, DefaultScan16x4Neighbors), + new(DefaultScan16x4, DefaultInverseScan16x4, DefaultScan16x4Neighbors), + new(DefaultScan16x4, DefaultInverseScan16x4, DefaultScan16x4Neighbors), + new(DefaultScan16x4, DefaultInverseScan16x4, DefaultScan16x4Neighbors), + new(DefaultScan16x4, DefaultInverseScan16x4, DefaultScan16x4Neighbors), + new(DefaultScan16x4, DefaultInverseScan16x4, DefaultScan16x4Neighbors), + new(DefaultScan16x4, DefaultInverseScan16x4, DefaultScan16x4Neighbors), + new(MatrixRowScan16x4, MatrixRowInverseScan16x4, MatrixRowScan16x4Neighbors), + new(MatrixColumnScan16x4, MatrixColumnInverseScan16x4, MatrixColumnScan16x4Neighbors), + new(MatrixRowScan16x4, MatrixRowInverseScan16x4, MatrixRowScan16x4Neighbors), + new(MatrixColumnScan16x4, MatrixColumnInverseScan16x4, MatrixColumnScan16x4Neighbors), + new(MatrixRowScan16x4, MatrixRowInverseScan16x4, MatrixRowScan16x4Neighbors), + new(MatrixColumnScan16x4, MatrixColumnInverseScan16x4, MatrixColumnScan16x4Neighbors), + ], + [ + + // Transform size 8X32 + new(DefaultScan8x32, DefaultInverseScan8x32, DefaultScan8x32Neighbors), + new(DefaultScan8x32, DefaultInverseScan8x32, DefaultScan8x32Neighbors), + new(DefaultScan8x32, DefaultInverseScan8x32, DefaultScan8x32Neighbors), + new(DefaultScan8x32, DefaultInverseScan8x32, DefaultScan8x32Neighbors), + new(DefaultScan8x32, DefaultInverseScan8x32, DefaultScan8x32Neighbors), + new(DefaultScan8x32, DefaultInverseScan8x32, DefaultScan8x32Neighbors), + new(DefaultScan8x32, DefaultInverseScan8x32, DefaultScan8x32Neighbors), + new(DefaultScan8x32, DefaultInverseScan8x32, DefaultScan8x32Neighbors), + new(DefaultScan8x32, DefaultInverseScan8x32, DefaultScan8x32Neighbors), + new(DefaultScan8x32, DefaultInverseScan8x32, DefaultScan8x32Neighbors), + new(MatrixRowScan8x32, MatrixRowInverseScan8x32, MatrixRowScan8x32Neighbors), + new(MatrixColumnScan8x32, MatrixColumnInverseScan8x32, MatrixColumnScan8x32Neighbors), + new(MatrixRowScan8x32, MatrixRowInverseScan8x32, MatrixRowScan8x32Neighbors), + new(MatrixColumnScan8x32, MatrixColumnInverseScan8x32, MatrixColumnScan8x32Neighbors), + new(MatrixRowScan8x32, MatrixRowInverseScan8x32, MatrixRowScan8x32Neighbors), + new(MatrixColumnScan8x32, MatrixColumnInverseScan8x32, MatrixColumnScan8x32Neighbors), + ], + [ + + // Transform size 32X8 + new(DefaultScan32x8, DefaultInverseScan32x8, DefaultScan32x8Neighbors), + new(DefaultScan32x8, DefaultInverseScan32x8, DefaultScan32x8Neighbors), + new(DefaultScan32x8, DefaultInverseScan32x8, DefaultScan32x8Neighbors), + new(DefaultScan32x8, DefaultInverseScan32x8, DefaultScan32x8Neighbors), + new(DefaultScan32x8, DefaultInverseScan32x8, DefaultScan32x8Neighbors), + new(DefaultScan32x8, DefaultInverseScan32x8, DefaultScan32x8Neighbors), + new(DefaultScan32x8, DefaultInverseScan32x8, DefaultScan32x8Neighbors), + new(DefaultScan32x8, DefaultInverseScan32x8, DefaultScan32x8Neighbors), + new(DefaultScan32x8, DefaultInverseScan32x8, DefaultScan32x8Neighbors), + new(DefaultScan32x8, DefaultInverseScan32x8, DefaultScan32x8Neighbors), + new(MatrixRowScan32x8, MatrixRowInverseScan32x8, MatrixRowScan32x8Neighbors), + new(MatrixColumnScan32x8, MatrixColumnInverseScan32x8, MatrixColumnScan32x8Neighbors), + new(MatrixRowScan32x8, MatrixRowInverseScan32x8, MatrixRowScan32x8Neighbors), + new(MatrixColumnScan32x8, MatrixColumnInverseScan32x8, MatrixColumnScan32x8Neighbors), + new(MatrixRowScan32x8, MatrixRowInverseScan32x8, MatrixRowScan32x8Neighbors), + new(MatrixColumnScan32x8, MatrixColumnInverseScan32x8, MatrixColumnScan32x8Neighbors), + ], + [ + + // Transform size 16X64 + // Half of the coefficients of tx64 at higher frequencies are set to + // zeros. So tx32's scan order is used. + new(DefaultScan16x32, DefaultInverseScan16x32, DefaultScan16x32Neighbors), + new(DefaultScan16x32, DefaultInverseScan16x32, DefaultScan16x32Neighbors), + new(DefaultScan16x32, DefaultInverseScan16x32, DefaultScan16x32Neighbors), + new(DefaultScan16x32, DefaultInverseScan16x32, DefaultScan16x32Neighbors), + new(DefaultScan16x32, DefaultInverseScan16x32, DefaultScan16x32Neighbors), + new(DefaultScan16x32, DefaultInverseScan16x32, DefaultScan16x32Neighbors), + new(DefaultScan16x32, DefaultInverseScan16x32, DefaultScan16x32Neighbors), + new(DefaultScan16x32, DefaultInverseScan16x32, DefaultScan16x32Neighbors), + new(DefaultScan16x32, DefaultInverseScan16x32, DefaultScan16x32Neighbors), + new(DefaultScan16x32, DefaultInverseScan16x32, DefaultScan16x32Neighbors), + new(DefaultScan16x32, MatrixRowInverseScan16x32, MatrixRowScan16x32Neighbors), + new(DefaultScan16x32, MatrixColumnInverseScan16x32, MatrixColumnScan16x32Neighbors), + new(DefaultScan16x32, MatrixRowInverseScan16x32, MatrixRowScan16x32Neighbors), + new(DefaultScan16x32, MatrixColumnInverseScan16x32, MatrixColumnScan16x32Neighbors), + new(DefaultScan16x32, MatrixRowInverseScan16x32, MatrixRowScan16x32Neighbors), + new(DefaultScan16x32, MatrixColumnInverseScan16x32, MatrixColumnScan16x32Neighbors), + ], + [ + + // Transform size 64X16 + // Half of the coefficients of tx64 at higher frequencies are set to + // zeros. So tx32's scan order is used. + new(DefaultScan32x16, DefaultInverseScan32x16, DefaultScan32x16Neighbors), + new(DefaultScan32x16, DefaultInverseScan32x16, DefaultScan32x16Neighbors), + new(DefaultScan32x16, DefaultInverseScan32x16, DefaultScan32x16Neighbors), + new(DefaultScan32x16, DefaultInverseScan32x16, DefaultScan32x16Neighbors), + new(DefaultScan32x16, DefaultInverseScan32x16, DefaultScan32x16Neighbors), + new(DefaultScan32x16, DefaultInverseScan32x16, DefaultScan32x16Neighbors), + new(DefaultScan32x16, DefaultInverseScan32x16, DefaultScan32x16Neighbors), + new(DefaultScan32x16, DefaultInverseScan32x16, DefaultScan32x16Neighbors), + new(DefaultScan32x16, DefaultInverseScan32x16, DefaultScan32x16Neighbors), + new(DefaultScan32x16, DefaultInverseScan32x16, DefaultScan32x16Neighbors), + new(DefaultScan32x16, MatrixRowInverseScan32x16, MatrixRowScan32x16Neighbors), + new(DefaultScan32x16, MatrixColumnInverseScan32x16, MatrixColumnScan32x16Neighbors), + new(DefaultScan32x16, MatrixRowInverseScan32x16, MatrixRowScan32x16Neighbors), + new(DefaultScan32x16, MatrixColumnInverseScan32x16, MatrixColumnScan32x16Neighbors), + new(DefaultScan32x16, MatrixRowInverseScan32x16, MatrixRowScan32x16Neighbors), + new(DefaultScan32x16, MatrixColumnInverseScan32x16, MatrixColumnScan32x16Neighbors), + ] + ]; + + /// + /// Gets the coefficient traversal and entropy-neighbor mappings for a transform block. + /// + /// The transform-block dimensions. + /// The compound transform type. + /// The selected scan order. + public static Av1ScanOrder GetScanOrder(Av1TransformSize transformSize, Av1TransformType transformType) + => ScanOrders[(int)transformSize][(int)transformType]; +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Transform/Av1SinusConstants.cs b/src/ImageSharp/Formats/Heif/Av1/Transform/Av1SinusConstants.cs new file mode 100644 index 0000000000..615f8a1fa2 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Transform/Av1SinusConstants.cs @@ -0,0 +1,152 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +/// +/// Contains the fixed-point sine and cosine tables used by the normative AV1 transform stages. +/// +internal static class Av1SinusConstants +{ + /// + /// The smallest supported number of fractional bits in a cosine lookup table. + /// + public const int MinimumCosinusBit = 10; + + /// + /// Fixed-point cosine values indexed by precision minus and angle step. + /// + /// + /// Each value is round(cos(pi * angle / 128) * 2^precision). + /// + private static readonly int[][] CosinusPiArray = + [ + [ + 1024, 1024, 1023, 1021, 1019, 1016, 1013, 1009, 1004, 999, 993, 987, 980, 972, 964, 955, + 946, 936, 926, 915, 903, 891, 878, 865, 851, 837, 822, 807, 792, 775, 759, 742, + 724, 706, 688, 669, 650, 630, 610, 590, 569, 548, 526, 505, 483, 460, 438, 415, + 392, 369, 345, 321, 297, 273, 249, 224, 200, 175, 150, 125, 100, 75, 50, 25 + ], + [ + 2048, 2047, 2046, 2042, 2038, 2033, 2026, 2018, 2009, 1998, 1987, 1974, 1960, 1945, 1928, 1911, + 1892, 1872, 1851, 1829, 1806, 1782, 1757, 1730, 1703, 1674, 1645, 1615, 1583, 1551, 1517, 1483, + 1448, 1412, 1375, 1338, 1299, 1260, 1220, 1179, 1138, 1096, 1053, 1009, 965, 921, 876, 830, + 784, 737, 690, 642, 595, 546, 498, 449, 400, 350, 301, 251, 201, 151, 100, 50 + ], + [ + 4096, 4095, 4091, 4085, 4076, 4065, 4052, 4036, 4017, 3996, 3973, 3948, 3920, 3889, 3857, 3822, + 3784, 3745, 3703, 3659, 3612, 3564, 3513, 3461, 3406, 3349, 3290, 3229, 3166, 3102, 3035, 2967, + 2896, 2824, 2751, 2675, 2598, 2520, 2440, 2359, 2276, 2191, 2106, 2019, 1931, 1842, 1751, 1660, + 1567, 1474, 1380, 1285, 1189, 1092, 995, 897, 799, 700, 601, 501, 401, 301, 201, 101 + ], + [ + 8192, 8190, 8182, 8170, 8153, 8130, 8103, 8071, 8035, 7993, 7946, 7895, 7839, 7779, 7713, 7643, + 7568, 7489, 7405, 7317, 7225, 7128, 7027, 6921, 6811, 6698, 6580, 6458, 6333, 6203, 6070, 5933, + 5793, 5649, 5501, 5351, 5197, 5040, 4880, 4717, 4551, 4383, 4212, 4038, 3862, 3683, 3503, 3320, + 3135, 2948, 2760, 2570, 2378, 2185, 1990, 1795, 1598, 1401, 1202, 1003, 803, 603, 402, 201 + ], + [ + 16384, 16379, 16364, 16340, 16305, 16261, 16207, 16143, 16069, 15986, 15893, 15791, 15679, 15557, 15426, 15286, + 15137, 14978, 14811, 14635, 14449, 14256, 14053, 13842, 13623, 13395, 13160, 12916, 12665, 12406, 12140, 11866, + 11585, 11297, 11003, 10702, 10394, 10080, 9760, 9434, 9102, 8765, 8423, 8076, 7723, 7366, 7005, 6639, + 6270, 5897, 5520, 5139, 4756, 4370, 3981, 3590, 3196, 2801, 2404, 2006, 1606, 1205, 804, 402 + ], + [ + 32768, 32758, 32729, 32679, 32610, 32522, 32413, 32286, 32138, 31972, 31786, 31581, 31357, 31114, 30853, 30572, + 30274, 29957, 29622, 29269, 28899, 28511, 28106, 27684, 27246, 26791, 26320, 25833, 25330, 24812, 24279, 23732, + 23170, 22595, 22006, 21403, 20788, 20160, 19520, 18868, 18205, 17531, 16846, 16151, 15447, 14733, 14010, 13279, + 12540, 11793, 11039, 10279, 9512, 8740, 7962, 7180, 6393, 5602, 4808, 4011, 3212, 2411, 1608, 804 + ], + [ + 65536, 65516, 65457, 65358, 65220, 65043, 64827, 64571, 64277, 63944, 63572, 63162, 62714, 62228, 61705, 61145, + 60547, 59914, 59244, 58538, 57798, 57022, 56212, 55368, 54491, 53581, 52639, 51665, 50660, 49624, 48559, 47464, + 46341, 45190, 44011, 42806, 41576, 40320, 39040, 37736, 36410, 35062, 33692, 32303, 30893, 29466, 28020, 26558, + 25080, 23586, 22078, 20557, 19024, 17479, 15924, 14359, 12785, 11204, 9616, 8022, 6424, 4821, 3216, 1608 + ] + ]; + + /// + /// Fixed-point sine values indexed by precision minus and angle step. + /// + /// + /// Values follow round((sqrt(2) * sin(angle * pi / 9) * 2 / 3) * 2^precision), adjusted so + /// the first two nonzero elements sum to the fourth. + /// + private static readonly int[][] SinusPiArray = + [ + [0, 330, 621, 836, 951], + [0, 660, 1241, 1672, 1901], + [0, 1321, 2482, 3344, 3803], + [0, 2642, 4964, 6689, 7606], + [0, 5283, 9929, 13377, 15212], + [0, 10566, 19858, 26755, 30424], + [0, 21133, 39716, 53510, 60849] + ]; + + /// + /// One quadrant of the signed cosine table used by directional intra prediction. + /// + private static readonly int[] Cosinus128Lookup = [ + 4096, 4095, 4091, 4085, 4076, 4065, 4052, 4036, + 4017, 3996, 3973, 3948, 3920, 3889, 3857, 3822, + 3784, 3745, 3703, 3659, 3612, 3564, 3513, 3461, + 3406, 3349, 3290, 3229, 3166, 3102, 3035, 2967, + 2896, 2824, 2751, 2675, 2598, 2520, 2440, 2359, + 2276, 2191, 2106, 2019, 1931, 1842, 1751, 1660, + 1567, 1474, 1380, 1285, 1189, 1092, 995, 897, + 799, 700, 601, 501, 401, 301, 201, 101, 0 + ]; + + /// + /// Gets the transform cosine table for a fixed-point precision. + /// + /// The number of fractional bits. + /// The cosine table for the requested precision. + public static Span CosinusPi(int n) => CosinusPiArray[n - MinimumCosinusBit]; + + /// + /// Gets the transform sine table for a fixed-point precision. + /// + /// The number of fractional bits. + /// The sine table for the requested precision. + public static Span SinusPi(int n) => SinusPiArray[n - MinimumCosinusBit]; + + /// + /// Spec: 7.13.2.1 Butterfly functions + /// + /// + /// Gets a directional-prediction sine value for an angle in 128-step circle units. + /// + /// The signed angle. + /// The signed fixed-point sine value. + public static int Sinus128(int angle) => Cosinus128(angle - 64); + + /// + /// Spec: 7.13.2.1 Butterfly functions + /// + /// + /// Gets a directional-prediction cosine value for an angle in 128-step circle units. + /// + /// The signed angle. + /// The signed fixed-point cosine value. + public static int Cosinus128(int angle) + { + int angle2 = angle & 255; + if (angle2 is >= 0 and <= 64) + { + return Cosinus128Lookup[angle2]; + } + + if (angle2 <= 128) + { + return -Cosinus128Lookup[128 - angle2]; + } + + if (angle2 <= 192) + { + return -Cosinus128Lookup[angle2 - 128]; + } + + return Cosinus128Lookup[256 - angle2]; + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Transform/Av1Transform1dMath.cs b/src/ImageSharp/Formats/Heif/Av1/Transform/Av1Transform1dMath.cs new file mode 100644 index 0000000000..1a97c68e87 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Transform/Av1Transform1dMath.cs @@ -0,0 +1,610 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.Intrinsics; +using System.Runtime.Intrinsics.X86; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +/// +/// Provides the fixed-point arithmetic shared by the scalar and SIMD AV1 one-dimensional transform kernels. +/// +/// +/// The general transform path keeps one independent axis in each signed 32-bit lane. Low-bit-depth forward transforms +/// additionally use signed 16-bit lanes and whole-butterfly AVX2 or AVX-512 multiply-add operations, matching AV1's +/// stage saturation points before packing. Scalar overloads preserve the same rounding and serve as the fallback. +/// +internal static class Av1Transform1dMath +{ + /// + /// The signed stage width whose fixed-point terminal operations require widened SIMD intermediates. + /// + public const byte WidenedIntermediateBitCount = 20; + + /// + /// The fixed-point representation of the square root of two with twelve fractional bits. + /// + public const int NewSqrt2 = 5793; + + /// + /// The number of fractional bits in . + /// + public const int NewSqrt2Bits = 12; + + /// + /// Calculates one output of a rounded, weighted two-input butterfly. + /// + /// The first fixed-point weight. + /// The first input value. + /// The second fixed-point weight. + /// The second input value. + /// The number of fractional bits in each weight. + /// The rounded fixed-point result. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static int HalfButterfly(int weight0, int input0, int weight1, int input1, int cosBit) + { + // The scalar path widens before multiplication so it remains an exact oracle for stress inputs outside the + // bounded production range as well as for conformant transform stages. + long weightedSum = ((long)weight0 * input0) + ((long)weight1 * input1); + return (int)((weightedSum + (1L << (cosBit - 1))) >> cosBit); + } + + /// + /// Clamps a transform-stage value to the signed range represented by a bit count. + /// + /// The transform-stage value. + /// The width of the signed range. + /// The value clamped to the permitted stage range. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static int Clamp(int value, byte bitCount) + { + long maximum = (1L << (bitCount - 1)) - 1; + long minimum = -(1L << (bitCount - 1)); + return (int)Math.Clamp(value, minimum, maximum); + } + + /// + /// Calculates four outputs of a rounded, weighted two-input butterfly in parallel. + /// + /// The first fixed-point weight. + /// The first four input values. + /// The second fixed-point weight. + /// The second four input values. + /// The number of fractional bits in each weight. + /// The four rounded fixed-point results. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector128 HalfButterfly(int weight0, Vector128 input0, int weight1, Vector128 input1, int cosBit) + { + // The transform stage ranges bound this sequence so the low 32-bit products and sum produce the normative + // result. Keeping those operations in Int32 lanes maps directly to the optimized SSE and Neon kernels. + Vector128 weightedSum = (input0 * weight0) + (input1 * weight1); + return (weightedSum + Vector128.Create(1 << (cosBit - 1))) >> cosBit; + } + + /// + /// Calculates eight outputs of a rounded, weighted two-input butterfly in parallel. + /// + /// The first fixed-point weight. + /// The first eight input values. + /// The second fixed-point weight. + /// The second eight input values. + /// The number of fractional bits in each weight. + /// The eight rounded fixed-point results. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector256 HalfButterfly(int weight0, Vector256 input0, int weight1, Vector256 input1, int cosBit) + { + // The bounded stage inputs allow the complete butterfly to remain in 32-bit lanes, letting the JIT emit the + // AVX2 multiply/add/shift sequence instead of splitting every input into widened 64-bit vectors. + Vector256 weightedSum = (input0 * weight0) + (input1 * weight1); + return (weightedSum + Vector256.Create(1 << (cosBit - 1))) >> cosBit; + } + + /// + /// Calculates sixteen outputs of a rounded, weighted two-input butterfly in parallel. + /// + /// The first fixed-point weight. + /// The first sixteen input values. + /// The second fixed-point weight. + /// The second sixteen input values. + /// The number of fractional bits in each weight. + /// The sixteen rounded fixed-point results. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector512 HalfButterfly(int weight0, Vector512 input0, int weight1, Vector512 input1, int cosBit) + { + // AV1 stage ranges keep the products and sum inside the normative wrapping Int32 domain. Preserving that lane + // width lets 512-bit SIMD evaluate sixteen independent transform axes without widened intermediate vectors. + Vector512 weightedSum = (input0 * weight0) + (input1 * weight1); + return (weightedSum + Vector512.Create(1 << (cosBit - 1))) >> cosBit; + } + + /// + /// Adds and subtracts thirty-two pairs of low-bit-depth transform values with signed saturation. + /// + /// The first thirty-two input values. + /// The second thirty-two input values. + /// The thirty-two saturated sums. + /// The thirty-two saturated differences. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void AddSubtract( + in Vector512 input0, + in Vector512 input1, + out Vector512 sum, + out Vector512 difference) + { + // Read both operands before either destination is written because the reference decoder deliberately permits an input + // buffer to alias one or both outputs while alternating between its two fixed transform-stage buffers. + Vector512 left = input0; + Vector512 right = input1; + + sum = Vector512.AddSaturate(left, right); + difference = Vector512.SubtractSaturate(left, right); + } + + /// + /// Calculates both outputs of thirty-two rounded low-bit-depth butterflies in parallel. + /// + /// The first fixed-point weight. + /// The second fixed-point weight. + /// The first thirty-two input values. + /// The second thirty-two input values. + /// The first thirty-two saturated, rounded results. + /// The second thirty-two saturated, rounded results. + /// The number of fractional bits in each weight. + /// The rounding offset for the widened intermediate values. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void Butterfly( + int weight0, + int weight1, + in Vector512 input0, + in Vector512 input1, + out Vector512 output0, + out Vector512 output1, + int cosBit, + in Vector512 rounding) + { + // VPMADDWD evaluates adjacent Int16 products into Int32 lanes. Both outputs reuse the same interleaved + // inputs, matching the reference decoder's whole butterfly instead of loading and unpacking each input pair twice. + Vector512 left = input0; + Vector512 right = input1; + Vector512 interleavedLower = Avx512BW.UnpackLow(left, right); + Vector512 interleavedUpper = Avx512BW.UnpackHigh(left, right); + Vector512 weight0Values = Vector512.Create((short)weight0); + Vector512 weight1Values = Vector512.Create((short)weight1); + Vector512 weights0 = Avx512BW.UnpackLow(weight0Values, weight1Values); + Vector512 weights1 = Avx512BW.UnpackLow(weight1Values, Vector512.Create((short)-weight0)); + Vector512 output0Lower = Avx512BW.MultiplyAddAdjacent(interleavedLower, weights0); + Vector512 output0Upper = Avx512BW.MultiplyAddAdjacent(interleavedUpper, weights0); + Vector512 output1Lower = Avx512BW.MultiplyAddAdjacent(interleavedLower, weights1); + Vector512 output1Upper = Avx512BW.MultiplyAddAdjacent(interleavedUpper, weights1); + + output0Lower = (output0Lower + rounding) >> cosBit; + output0Upper = (output0Upper + rounding) >> cosBit; + output1Lower = (output1Lower + rounding) >> cosBit; + output1Upper = (output1Upper + rounding) >> cosBit; + + // VPACKSSDW restores the original lane order within each 128-bit block and narrows with the saturation + // required by the low-bit-depth AV1 stage arithmetic. + output0 = Avx512BW.PackSignedSaturate(output0Lower, output0Upper); + output1 = Avx512BW.PackSignedSaturate(output1Lower, output1Upper); + } + + /// + /// Adds and subtracts sixteen pairs of low-bit-depth transform values with signed saturation. + /// + /// The first sixteen input values. + /// The second sixteen input values. + /// The sixteen saturated sums. + /// The sixteen saturated differences. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void AddSubtract( + in Vector256 input0, + in Vector256 input1, + out Vector256 sum, + out Vector256 difference) + { + Vector256 left = input0; + Vector256 right = input1; + + sum = Vector256.AddSaturate(left, right); + difference = Vector256.SubtractSaturate(left, right); + } + + /// + /// Calculates both outputs of sixteen rounded low-bit-depth butterflies in parallel. + /// + /// The first fixed-point weight. + /// The second fixed-point weight. + /// The first sixteen input values. + /// The second sixteen input values. + /// The first sixteen saturated, rounded results. + /// The second sixteen saturated, rounded results. + /// The number of fractional bits in each weight. + /// The rounding offset for the widened intermediate values. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void Butterfly( + int weight0, + int weight1, + in Vector256 input0, + in Vector256 input1, + out Vector256 output0, + out Vector256 output1, + int cosBit, + in Vector256 rounding) + { + Vector256 left = input0; + Vector256 right = input1; + Vector256 interleavedLower = Avx2.UnpackLow(left, right); + Vector256 interleavedUpper = Avx2.UnpackHigh(left, right); + Vector256 weight0Values = Vector256.Create((short)weight0); + Vector256 weight1Values = Vector256.Create((short)weight1); + Vector256 weights0 = Avx2.UnpackLow(weight0Values, weight1Values); + Vector256 weights1 = Avx2.UnpackLow(weight1Values, Vector256.Create((short)-weight0)); + Vector256 output0Lower = Avx2.MultiplyAddAdjacent(interleavedLower, weights0); + Vector256 output0Upper = Avx2.MultiplyAddAdjacent(interleavedUpper, weights0); + Vector256 output1Lower = Avx2.MultiplyAddAdjacent(interleavedLower, weights1); + Vector256 output1Upper = Avx2.MultiplyAddAdjacent(interleavedUpper, weights1); + + output0Lower = (output0Lower + rounding) >> cosBit; + output0Upper = (output0Upper + rounding) >> cosBit; + output1Lower = (output1Lower + rounding) >> cosBit; + output1Upper = (output1Upper + rounding) >> cosBit; + + output0 = Avx2.PackSignedSaturate(output0Lower, output0Upper); + output1 = Avx2.PackSignedSaturate(output1Lower, output1Upper); + } + + /// + /// Calculates both outputs of eight rounded low-bit-depth butterflies in parallel. + /// + /// The first fixed-point weight. + /// The second fixed-point weight. + /// The first eight input values. + /// The second eight input values. + /// The first eight saturated, rounded results. + /// The second eight saturated, rounded results. + /// The number of fractional bits in each weight. + /// The rounding offset for the widened intermediate values. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void Butterfly( + int weight0, + int weight1, + in Vector128 input0, + in Vector128 input1, + out Vector128 output0, + out Vector128 output1, + int cosBit, + in Vector128 rounding) + { + Vector128 left = input0; + Vector128 right = input1; + + if (Sse2.IsSupported) + { + // PMADDWD is the native x86 form of Highway's pairwise widening multiply-add. Interleaving once lets + // both butterfly outputs reuse the same input arrangement before signed-saturating demotion. + Vector128 interleavedLower = Sse2.UnpackLow(left, right); + Vector128 interleavedUpper = Sse2.UnpackHigh(left, right); + Vector128 weight0Values = Vector128.Create((short)weight0); + Vector128 weight1Values = Vector128.Create((short)weight1); + Vector128 weights0 = Sse2.UnpackLow(weight0Values, weight1Values); + Vector128 weights1 = Sse2.UnpackLow(weight1Values, Vector128.Create((short)-weight0)); + Vector128 output0Lower = Sse2.MultiplyAddAdjacent(interleavedLower, weights0); + Vector128 output0Upper = Sse2.MultiplyAddAdjacent(interleavedUpper, weights0); + Vector128 output1Lower = Sse2.MultiplyAddAdjacent(interleavedLower, weights1); + Vector128 output1Upper = Sse2.MultiplyAddAdjacent(interleavedUpper, weights1); + + output0Lower = (output0Lower + rounding) >> cosBit; + output0Upper = (output0Upper + rounding) >> cosBit; + output1Lower = (output1Lower + rounding) >> cosBit; + output1Upper = (output1Upper + rounding) >> cosBit; + + output0 = Sse2.PackSignedSaturate(output0Lower, output0Upper); + output1 = Sse2.PackSignedSaturate(output1Lower, output1Upper); + return; + } + + // AdvSimd and WebAssembly do not expose PMADDWD. Widen both inputs once and retain the complete operation + // in Vector128 lanes so those targets still execute the transform as a whole SIMD butterfly. + (Vector128 leftLower, Vector128 leftUpper) = Vector128.Widen(left); + (Vector128 rightLower, Vector128 rightUpper) = Vector128.Widen(right); + + Vector128 weight0Vector = Vector128.Create(weight0); + Vector128 weight1Vector = Vector128.Create(weight1); + Vector128 output0LowerVector = ((leftLower * weight0Vector) + (rightLower * weight1Vector) + rounding) >> cosBit; + Vector128 output0UpperVector = ((leftUpper * weight0Vector) + (rightUpper * weight1Vector) + rounding) >> cosBit; + Vector128 output1LowerVector = ((leftLower * weight1Vector) - (rightLower * weight0Vector) + rounding) >> cosBit; + Vector128 output1UpperVector = ((leftUpper * weight1Vector) - (rightUpper * weight0Vector) + rounding) >> cosBit; + Vector128 minimum = Vector128.Create((int)short.MinValue); + Vector128 maximum = Vector128.Create((int)short.MaxValue); + + output0LowerVector = Vector128.Clamp(output0LowerVector, minimum, maximum); + output0UpperVector = Vector128.Clamp(output0UpperVector, minimum, maximum); + output1LowerVector = Vector128.Clamp(output1LowerVector, minimum, maximum); + output1UpperVector = Vector128.Clamp(output1UpperVector, minimum, maximum); + output0 = Vector128.Narrow(output0LowerVector, output0UpperVector); + output1 = Vector128.Narrow(output1LowerVector, output1UpperVector); + } + + /// + /// Clamps four transform-stage values to the signed range represented by a bit count. + /// + /// The four transform-stage values. + /// The width of the signed range. + /// The values clamped to the permitted stage range. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector128 Clamp(Vector128 value, byte bitCount) + { + int maximum = (1 << (bitCount - 1)) - 1; + int minimum = -(1 << (bitCount - 1)); + return Vector128.Clamp(value, Vector128.Create(minimum), Vector128.Create(maximum)); + } + + /// + /// Clamps eight transform-stage values to the signed range represented by a bit count. + /// + /// The eight transform-stage values. + /// The width of the signed range. + /// The values clamped to the permitted stage range. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector256 Clamp(Vector256 value, byte bitCount) + { + int maximum = (1 << (bitCount - 1)) - 1; + int minimum = -(1 << (bitCount - 1)); + return Vector256.Clamp(value, Vector256.Create(minimum), Vector256.Create(maximum)); + } + + /// + /// Clamps sixteen transform-stage values to the signed range represented by a bit count. + /// + /// The sixteen transform-stage values. + /// The width of the signed range. + /// The values clamped to the permitted stage range. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector512 Clamp(Vector512 value, byte bitCount) + { + int maximum = (1 << (bitCount - 1)) - 1; + int minimum = -(1 << (bitCount - 1)); + return Vector512.Clamp(value, Vector512.Create(minimum), Vector512.Create(maximum)); + } + + /// + /// Multiplies and rounds four fixed-point values in parallel. + /// + /// The four values to scale. + /// The fixed-point multiplier. + /// The number of fractional bits in the multiplier. + /// The four rounded results. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector128 MultiplyRound(Vector128 value, int multiplier, int fractionalBits) + => HalfButterfly(multiplier, value, 0, Vector128.Zero, fractionalBits); + + /// + /// Multiplies and rounds eight fixed-point values in parallel. + /// + /// The eight values to scale. + /// The fixed-point multiplier. + /// The number of fractional bits in the multiplier. + /// The eight rounded results. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector256 MultiplyRound(Vector256 value, int multiplier, int fractionalBits) + => HalfButterfly(multiplier, value, 0, Vector256.Zero, fractionalBits); + + /// + /// Multiplies and rounds sixteen fixed-point values in parallel. + /// + /// The sixteen values to scale. + /// The fixed-point multiplier. + /// The number of fractional bits in the multiplier. + /// The sixteen rounded results. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector512 MultiplyRound(Vector512 value, int multiplier, int fractionalBits) + => HalfButterfly(multiplier, value, 0, Vector512.Zero, fractionalBits); + + /// + /// Multiplies and rounds four fixed-point values with signed sixty-four-bit intermediate lanes. + /// + /// The four values to scale. + /// The fixed-point multiplier. + /// The number of fractional bits in the multiplier. + /// The four rounded results. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector128 MultiplyRoundWidened(Vector128 value, int multiplier, int fractionalBits) + { + (Vector128 lower, Vector128 upper) = Vector128.Widen(value); + Vector128 rounding = Vector128.Create(1L << (fractionalBits - 1)); + + // The reference decoder's high-bit-depth identity kernels multiply in signed 64-bit lanes. Widen before both the + // product and rounding addition so a valid 20-bit twelve-bit row value cannot wrap through Int32. + lower = ((lower * multiplier) + rounding) >> fractionalBits; + upper = ((upper * multiplier) + rounding) >> fractionalBits; + return Vector128.Narrow(lower, upper); + } + + /// + /// Multiplies and rounds eight fixed-point values with signed sixty-four-bit intermediate lanes. + /// + /// The eight values to scale. + /// The fixed-point multiplier. + /// The number of fractional bits in the multiplier. + /// The eight rounded results. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector256 MultiplyRoundWidened(Vector256 value, int multiplier, int fractionalBits) + { + (Vector256 lower, Vector256 upper) = Vector256.Widen(value); + Vector256 rounding = Vector256.Create(1L << (fractionalBits - 1)); + + lower = ((lower * multiplier) + rounding) >> fractionalBits; + upper = ((upper * multiplier) + rounding) >> fractionalBits; + return Vector256.Narrow(lower, upper); + } + + /// + /// Multiplies four scalar inputs by fixed-point weights and rounds their sum. + /// + /// The first fixed-point weight. + /// The first input value. + /// The second fixed-point weight. + /// The second input value. + /// The third fixed-point weight. + /// The third input value. + /// The fourth fixed-point weight. + /// The fourth input value. + /// The number of fractional bits in each weight. + /// The rounded fixed-point sum. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static int MultiplyAdd4(int weight0, int input0, int weight1, int input1, int weight2, int input2, int weight3, int input3, int fractionalBits) + { + long weightedSum = ((long)weight0 * input0) + ((long)weight1 * input1) + ((long)weight2 * input2) + ((long)weight3 * input3); + return (int)((weightedSum + (1L << (fractionalBits - 1))) >> fractionalBits); + } + + /// + /// Multiplies four sets of four inputs by fixed-point weights and rounds their sums. + /// + /// The first fixed-point weight. + /// The first four input values. + /// The second fixed-point weight. + /// The second four input values. + /// The third fixed-point weight. + /// The third four input values. + /// The fourth fixed-point weight. + /// The fourth four input values. + /// The number of fractional bits in each weight. + /// The four rounded fixed-point sums. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector128 MultiplyAdd4( + int weight0, + Vector128 input0, + int weight1, + Vector128 input1, + int weight2, + Vector128 input2, + int weight3, + Vector128 input3, + int fractionalBits) + { + // The four-point ADST factorization has the same bounded-intermediate contract as the butterfly stages. + // Accumulating in Int32 lanes preserves the normative result and avoids eight widening operations per sum. + Vector128 weightedSum = (input0 * weight0) + (input1 * weight1) + (input2 * weight2) + (input3 * weight3); + return (weightedSum + Vector128.Create(1 << (fractionalBits - 1))) >> fractionalBits; + } + + /// + /// Multiplies four sets of eight inputs by fixed-point weights and rounds their sums. + /// + /// The first fixed-point weight. + /// The first eight input values. + /// The second fixed-point weight. + /// The second eight input values. + /// The third fixed-point weight. + /// The third eight input values. + /// The fourth fixed-point weight. + /// The fourth eight input values. + /// The number of fractional bits in each weight. + /// The eight rounded fixed-point sums. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector256 MultiplyAdd4( + int weight0, + Vector256 input0, + int weight1, + Vector256 input1, + int weight2, + Vector256 input2, + int weight3, + Vector256 input3, + int fractionalBits) + { + Vector256 weightedSum = (input0 * weight0) + (input1 * weight1) + (input2 * weight2) + (input3 * weight3); + return (weightedSum + Vector256.Create(1 << (fractionalBits - 1))) >> fractionalBits; + } + + /// + /// Multiplies four sets of sixteen inputs by fixed-point weights and rounds their sums. + /// + /// The first fixed-point weight. + /// The first sixteen input values. + /// The second fixed-point weight. + /// The second sixteen input values. + /// The third fixed-point weight. + /// The third sixteen input values. + /// The fourth fixed-point weight. + /// The fourth sixteen input values. + /// The number of fractional bits in each weight. + /// The sixteen rounded fixed-point sums. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector512 MultiplyAdd4( + int weight0, + Vector512 input0, + int weight1, + Vector512 input1, + int weight2, + Vector512 input2, + int weight3, + Vector512 input3, + int fractionalBits) + { + Vector512 weightedSum = (input0 * weight0) + (input1 * weight1) + (input2 * weight2) + (input3 * weight3); + return (weightedSum + Vector512.Create(1 << (fractionalBits - 1))) >> fractionalBits; + } + + /// + /// Multiplies four sets of four inputs in signed thirty-two-bit lanes, then widens the terminal rounding step. + /// + /// The first fixed-point weight. + /// The first four input values. + /// The second fixed-point weight. + /// The second four input values. + /// The third fixed-point weight. + /// The third four input values. + /// The fourth fixed-point weight. + /// The fourth four input values. + /// The number of fractional bits in each weight. + /// The four rounded fixed-point sums. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector128 MultiplyAdd4WidenedRound( + int weight0, + Vector128 input0, + int weight1, + Vector128 input1, + int weight2, + Vector128 input2, + int weight3, + Vector128 input3, + int fractionalBits) + { + // the reference decoder keeps conformant ADST4 sine products and their factorized sums in Int32, then widens the terminal + // scaling and rounding. Preserve that exact boundary instead of widening every transform multiplication. + Vector128 weightedSum = (input0 * weight0) + (input1 * weight1) + (input2 * weight2) + (input3 * weight3); + return MultiplyRoundWidened(weightedSum, 1, fractionalBits); + } + + /// + /// Multiplies four sets of eight inputs in signed thirty-two-bit lanes, then widens the terminal rounding step. + /// + /// The first fixed-point weight. + /// The first eight input values. + /// The second fixed-point weight. + /// The second eight input values. + /// The third fixed-point weight. + /// The third eight input values. + /// The fourth fixed-point weight. + /// The fourth eight input values. + /// The number of fractional bits in each weight. + /// The eight rounded fixed-point sums. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector256 MultiplyAdd4WidenedRound( + int weight0, + Vector256 input0, + int weight1, + Vector256 input1, + int weight2, + Vector256 input2, + int weight3, + Vector256 input3, + int fractionalBits) + { + Vector256 weightedSum = (input0 * weight0) + (input1 * weight1) + (input2 * weight2) + (input3 * weight3); + return MultiplyRoundWidened(weightedSum, 1, fractionalBits); + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Transform/Av1Transform2dFlipConfiguration.cs b/src/ImageSharp/Formats/Heif/Av1/Transform/Av1Transform2dFlipConfiguration.cs new file mode 100644 index 0000000000..905136ea1d --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Transform/Av1Transform2dFlipConfiguration.cs @@ -0,0 +1,500 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +/// +/// Resolves an AV1 compound transform into its per-axis functions, flips, shifts, and stage ranges. +/// +internal ref struct Av1Transform2dFlipConfiguration +{ + /// + /// The maximum number of fixed-point stages in any supported one-dimensional transform. + /// + public const int MaxStageNumber = 12; + + /// + /// The base-two logarithm of the smallest supported transform dimension. + /// + private const int SmallestTransformSizeLog2 = 2; + + /// + /// The fixed-point cosine precision used by every inverse transform. + /// + private const int InverseCosBit = 12; + + /// + /// The fixed-point shifts applied between successive stages of the configured transform pipeline. + /// + private InlineArray3 shift; + + /// + /// The signed-bit ranges produced by the column transform stages. + /// + private InlineArray12 stageRangeColumn; + + /// + /// The signed-bit ranges produced by the row transform stages. + /// + private InlineArray12 stageRangeRow; + + /// + /// Initializes a new instance of the struct. + /// + /// The compound horizontal and vertical transform type. + /// The transform-block dimensions. + /// The coded sample bit depth. + /// Whether to configure the forward transform pipeline. + private Av1Transform2dFlipConfiguration(Av1TransformType transformType, Av1TransformSize transformSize, int bitDepth, bool isForward) + { + this = default; + + // Resolve the axis operators and fixed-point settings once so the hot traversal contains no per-row + // transform-type lookup or flip decision. + this.TransformSize = transformSize; + this.TransformType = transformType; + this.SetFlip(transformType); + this.TransformTypeColumn = VerticalType[(int)transformType]; + this.TransformTypeRow = HorizontalType[(int)transformType]; + int transformWidthIndex = transformSize.GetBlockWidthLog2() - SmallestTransformSizeLog2; + int transformHeightIndex = transformSize.GetBlockHeightLog2() - SmallestTransformSizeLog2; + this.TransformFunctionTypeColumn = TransformFunctionTypeMap[(transformHeightIndex * 4) + (int)this.TransformTypeColumn]; + this.TransformFunctionTypeRow = TransformFunctionTypeMap[(transformWidthIndex * 4) + (int)this.TransformTypeRow]; + this.StageNumberColumn = this.TransformFunctionTypeColumn != Av1TransformFunctionType.Invalid ? StageNumberList[(int)this.TransformFunctionTypeColumn] : -1; + this.StageNumberRow = this.TransformFunctionTypeRow != Av1TransformFunctionType.Invalid ? StageNumberList[(int)this.TransformFunctionTypeRow] : -1; + + if (isForward) + { + int shiftIndex = (int)transformSize * 3; + this.shift[0] = ForwardShiftMap[shiftIndex]; + this.shift[1] = ForwardShiftMap[shiftIndex + 1]; + this.shift[2] = ForwardShiftMap[shiftIndex + 2]; + this.CosBitColumn = ForwardCosBitColumnMap[(transformWidthIndex * 5) + transformHeightIndex]; + this.CosBitRow = ForwardCosBitRowMap[(transformWidthIndex * 5) + transformHeightIndex]; + this.InitializeForwardStageRange(); + this.GenerateForwardStageRange(bitDepth); + } + else + { + int shiftIndex = (int)transformSize * 2; + this.shift[0] = InverseShiftMap[shiftIndex]; + this.shift[1] = InverseShiftMap[shiftIndex + 1]; + this.CosBitColumn = InverseCosBit; + this.CosBitRow = InverseCosBit; + this.GenerateInverseStageRange(bitDepth); + } + } + + /// + /// Gets the function applied down the transform columns for each compound transform type. + /// + private static ReadOnlySpan VerticalType => + [ + Av1TransformType1d.Dct, + Av1TransformType1d.Adst, + Av1TransformType1d.Dct, + Av1TransformType1d.Adst, + Av1TransformType1d.FlipAdst, + Av1TransformType1d.Dct, + Av1TransformType1d.FlipAdst, + Av1TransformType1d.Adst, + Av1TransformType1d.FlipAdst, + Av1TransformType1d.Identity, + Av1TransformType1d.Dct, + Av1TransformType1d.Identity, + Av1TransformType1d.Adst, + Av1TransformType1d.Identity, + Av1TransformType1d.FlipAdst, + Av1TransformType1d.Identity, + ]; + + /// + /// Gets the function applied across the transform rows for each compound transform type. + /// + private static ReadOnlySpan HorizontalType => + [ + Av1TransformType1d.Dct, + Av1TransformType1d.Dct, + Av1TransformType1d.Adst, + Av1TransformType1d.Adst, + Av1TransformType1d.Dct, + Av1TransformType1d.FlipAdst, + Av1TransformType1d.FlipAdst, + Av1TransformType1d.FlipAdst, + Av1TransformType1d.Adst, + Av1TransformType1d.Identity, + Av1TransformType1d.Identity, + Av1TransformType1d.Dct, + Av1TransformType1d.Identity, + Av1TransformType1d.Adst, + Av1TransformType1d.Identity, + Av1TransformType1d.FlipAdst, + ]; + + /// + /// Gets the three normative forward fixed-point shifts for every transform size. + /// + private static ReadOnlySpan ForwardShiftMap => + [ + 2, 0, 0, // 4x4 + 2, -1, 0, // 8x8 + 2, -2, 0, // 16x16 + 2, -4, 0, // 32x32 + 0, -2, -2, // 64x64 + 2, -1, 0, // 4x8 + 2, -1, 0, // 8x4 + 2, -2, 0, // 8x16 + 2, -2, 0, // 16x8 + 2, -4, 0, // 16x32 + 2, -4, 0, // 32x16 + 0, -2, -2, // 32x64 + 2, -4, -2, // 64x32 + 2, -1, 0, // 4x16 + 2, -1, 0, // 16x4 + 2, -2, 0, // 8x32 + 2, -2, 0, // 32x8 + 0, -2, 0, // 16x64 + 2, -4, 0, // 64x16 + ]; + + /// + /// Gets the two normative inverse fixed-point shifts for every transform size. + /// + private static ReadOnlySpan InverseShiftMap => + [ + 0, -4, // 4x4 + -1, -4, // 8x8 + -2, -4, // 16x16 + -2, -4, // 32x32 + -2, -4, // 64x64 + 0, -4, // 4x8 + 0, -4, // 8x4 + -1, -4, // 8x16 + -1, -4, // 16x8 + -1, -4, // 16x32 + -1, -4, // 32x16 + -1, -4, // 32x64 + -1, -4, // 64x32 + -1, -4, // 4x16 + -1, -4, // 16x4 + -2, -4, // 8x32 + -2, -4, // 32x8 + -2, -4, // 16x64 + -2, -4, // 64x16 + ]; + + /// + /// Gets column-transform cosine precision by width and height logarithm. + /// + private static ReadOnlySpan ForwardCosBitColumnMap => + [ + 13, 13, 13, 0, 0, + 13, 13, 13, 12, 0, + 13, 13, 13, 12, 13, + 0, 13, 13, 12, 13, + 0, 0, 13, 12, 13, + ]; + + /// + /// Gets row-transform cosine precision by width and height logarithm. + /// + private static ReadOnlySpan ForwardCosBitRowMap => + [ + 13, 13, 12, 0, 0, + 13, 13, 13, 12, 0, + 13, 13, 12, 13, 12, + 0, 12, 13, 12, 11, + 0, 0, 12, 11, 10, + ]; + + /// + /// Gets the concrete staged function for each transform dimension and one-dimensional type. + /// + private static ReadOnlySpan TransformFunctionTypeMap => + [ + Av1TransformFunctionType.Dct4, Av1TransformFunctionType.Adst4, Av1TransformFunctionType.Adst4, Av1TransformFunctionType.Identity4, + Av1TransformFunctionType.Dct8, Av1TransformFunctionType.Adst8, Av1TransformFunctionType.Adst8, Av1TransformFunctionType.Identity8, + Av1TransformFunctionType.Dct16, Av1TransformFunctionType.Adst16, Av1TransformFunctionType.Adst16, Av1TransformFunctionType.Identity16, + Av1TransformFunctionType.Dct32, Av1TransformFunctionType.Invalid, Av1TransformFunctionType.Invalid, Av1TransformFunctionType.Identity32, + Av1TransformFunctionType.Dct64, Av1TransformFunctionType.Invalid, Av1TransformFunctionType.Invalid, Av1TransformFunctionType.Invalid, + ]; + + /// + /// Gets the number of fixed-point stages executed by each concrete transform function. + /// + private static ReadOnlySpan StageNumberList => + [ + 4, // TXFM_TYPE_DCT4 + 6, // TXFM_TYPE_DCT8 + 8, // TXFM_TYPE_DCT16 + 10, // TXFM_TYPE_DCT32 + 12, // TXFM_TYPE_DCT64 + 7, // TXFM_TYPE_ADST4 + 8, // TXFM_TYPE_ADST8 + 10, // TXFM_TYPE_ADST16 + 1, // TXFM_TYPE_IDENTITY4 + 1, // TXFM_TYPE_IDENTITY8 + 1, // TXFM_TYPE_IDENTITY16 + 1, // TXFM_TYPE_IDENTITY32 + ]; + + /// + /// Gets twice the non-scaled bit range required after every transform stage. + /// + private static ReadOnlySpan RangeMulti2Map => + [ + 0, 2, 3, 3, 0, 0, 0, 0, 0, 0, 0, 0, // fdct4_range_mult2 + 0, 2, 4, 5, 5, 5, 0, 0, 0, 0, 0, 0, // fdct8_range_mult2 + 0, 2, 4, 6, 7, 7, 7, 7, 0, 0, 0, 0, // fdct16_range_mult2 + 0, 2, 4, 6, 8, 9, 9, 9, 9, 9, 0, 0, // fdct32_range_mult2 + 0, 2, 4, 6, 8, 10, 11, 11, 11, 11, 11, 11, // fdct64_range_mult2 + 0, 2, 4, 3, 3, 3, 3, 0, 0, 0, 0, 0, // fadst4_range_mult2 + 0, 0, 1, 3, 3, 5, 5, 5, 0, 0, 0, 0, // fadst8_range_mult2 + 0, 0, 1, 3, 3, 5, 5, 7, 7, 7, 0, 0, // fadst16_range_mult2 + 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, // fidtx4_range_mult2 + 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, // fidtx8_range_mult2 + 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, // fidtx16_range_mult2 + 4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, // fidtx32_range_mult2 + ]; + + /// + /// Gets the fixed-point cosine precision used by the column transform. + /// + public int CosBitColumn { get; } + + /// + /// Gets the fixed-point cosine precision used by the row transform. + /// + public int CosBitRow { get; } + + /// + /// Gets the one-dimensional transform type applied down columns. + /// + public Av1TransformType1d TransformTypeColumn { get; } + + /// + /// Gets the one-dimensional transform type applied across rows. + /// + public Av1TransformType1d TransformTypeRow { get; } + + /// + /// Gets the concrete staged transform function applied down columns. + /// + public Av1TransformFunctionType TransformFunctionTypeColumn { get; } + + /// + /// Gets the concrete staged transform function applied across rows. + /// + public Av1TransformFunctionType TransformFunctionTypeRow { get; } + + /// + /// Gets the number of fixed-point stages in the column transform. + /// + public int StageNumberColumn { get; } + + /// + /// Gets the number of fixed-point stages in the row transform. + /// + public int StageNumberRow { get; } + + /// + /// Gets the transform-block dimensions. + /// + public Av1TransformSize TransformSize { get; } + + /// + /// Gets the compound horizontal and vertical transform type. + /// + public Av1TransformType TransformType { get; } + + /// + /// Gets a value indicating whether column input is traversed from bottom to top. + /// + public bool FlipUpsideDown { get; private set; } + + /// + /// Gets a value indicating whether row output is written from right to left. + /// + public bool FlipLeftToRight { get; private set; } + + /// + /// Gets the first fixed-point pipeline shift. + /// + public readonly int Shift0 => this.shift[0]; + + /// + /// Gets the second fixed-point pipeline shift. + /// + public readonly int Shift1 => this.shift[1]; + + /// + /// Gets the terminal forward-transform shift, or zero for an inverse transform. + /// + public readonly int Shift2 => this.shift[2]; + + /// + /// Gets the allowed signed-bit range after each column-transform stage. + /// + public readonly InlineArray12 StageRangeColumn => this.stageRangeColumn; + + /// + /// Gets the allowed signed-bit range after each row-transform stage. + /// + public readonly InlineArray12 StageRangeRow => this.stageRangeRow; + + /// + /// Creates the configuration used to transform spatial residuals into coefficients. + /// + /// The compound horizontal and vertical transform type. + /// The transform-block dimensions. + /// The coded sample bit depth. + /// The forward transform configuration. + public static Av1Transform2dFlipConfiguration CreateForward(Av1TransformType transformType, Av1TransformSize transformSize, int bitDepth) + => new(transformType, transformSize, bitDepth, true); + + /// + /// Creates the configuration used to reconstruct samples from transform coefficients. + /// + /// The compound horizontal and vertical transform type. + /// The transform-block dimensions. + /// The coded sample bit depth. + /// The inverse transform configuration. + public static Av1Transform2dFlipConfiguration CreateInverse(Av1TransformType transformType, Av1TransformSize transformSize, int bitDepth) + => new(transformType, transformSize, bitDepth, false); + + /// + /// Determines whether the transform type is permitted for the configured dimensions. + /// + /// when the transform combination is valid for the transform size. + public bool IsAllowed() + { + // AV1 selects the legal transform set from the block's square-up size: blocks up to 16x16 allow all sixteen + // types, a 32x32 square-up allows DCT and identity, and larger square-up sizes allow DCT only. + return this.TransformSize switch + { + Av1TransformSize.Size32x32 or Av1TransformSize.Size16x32 or Av1TransformSize.Size32x16 or + Av1TransformSize.Size8x32 or Av1TransformSize.Size32x8 => + this.TransformType is Av1TransformType.DctDct or Av1TransformType.Identity, + Av1TransformSize.Size64x64 or Av1TransformSize.Size32x64 or Av1TransformSize.Size64x32 or + Av1TransformSize.Size16x64 or Av1TransformSize.Size64x16 => this.TransformType == Av1TransformType.DctDct, + _ => true, + }; + } + + /// + /// Derives the axis traversal directions encoded by a compound transform type. + /// + /// The compound transform type. + private void SetFlip(Av1TransformType transformType) + { + switch (transformType) + { + case Av1TransformType.DctDct: + case Av1TransformType.AdstDct: + case Av1TransformType.DctAdst: + case Av1TransformType.AdstAdst: + this.FlipUpsideDown = false; + this.FlipLeftToRight = false; + break; + case Av1TransformType.Identity: + case Av1TransformType.VerticalDct: + case Av1TransformType.HorizontalDct: + case Av1TransformType.VerticalAdst: + case Av1TransformType.HorizontalAdst: + this.FlipUpsideDown = false; + this.FlipLeftToRight = false; + break; + case Av1TransformType.FlipAdstDct: + case Av1TransformType.FlipAdstAdst: + case Av1TransformType.VerticalFlipAdst: + this.FlipUpsideDown = true; + this.FlipLeftToRight = false; + break; + case Av1TransformType.DctFlipAdst: + case Av1TransformType.AdstFlipAdst: + case Av1TransformType.HorizontalFlipAdst: + this.FlipUpsideDown = false; + this.FlipLeftToRight = true; + break; + case Av1TransformType.FlipAdstFlipAdst: + this.FlipUpsideDown = true; + this.FlipLeftToRight = true; + break; + default: + Guard.IsTrue(false, nameof(transformType), "Unknown transform type for determining flip."); + break; + } + } + + /// + /// Initializes the per-stage signed-bit ranges before input depth and pipeline shifts are applied. + /// + private void InitializeForwardStageRange() + { + if (this.TransformFunctionTypeColumn != Av1TransformFunctionType.Invalid) + { + int columnRangeOffset = (int)this.TransformFunctionTypeColumn * MaxStageNumber; + int columnStageCount = this.StageNumberColumn; + + for (int i = 0; i < columnStageCount; ++i) + { + this.stageRangeColumn[i] = (byte)((RangeMulti2Map[columnRangeOffset + i] + 1) >> 1); + } + + if (this.TransformFunctionTypeRow != Av1TransformFunctionType.Invalid) + { + int rowStageCount = this.StageNumberRow; + int rowRangeOffset = (int)this.TransformFunctionTypeRow * MaxStageNumber; + int columnRange = RangeMulti2Map[columnRangeOffset + this.StageNumberColumn - 1]; + + for (int i = 0; i < rowStageCount; ++i) + { + this.stageRangeRow[i] = (byte)((columnRange + RangeMulti2Map[rowRangeOffset + i] + 1) >> 1); + } + } + } + } + + /// + /// Adds input bit depth and inter-stage shifts to the non-scaled forward stage ranges. + /// + /// The coded sample bit depth. + private void GenerateForwardStageRange(int bitDepth) + { + for (int i = 0; i < this.StageNumberColumn; ++i) + { + this.stageRangeColumn[i] = (byte)(this.stageRangeColumn[i] + this.Shift0 + bitDepth + 1); + } + + for (int i = 0; i < this.StageNumberRow; ++i) + { + this.stageRangeRow[i] = (byte)(this.stageRangeRow[i] + this.Shift0 + this.Shift1 + bitDepth + 1); + } + } + + /// + /// Sets the optimized inverse stage ranges used to clamp intermediate values at the coded bit depth. + /// + /// The coded sample bit depth. + private void GenerateInverseStageRange(int bitDepth) + { + byte rowRange = bitDepth switch + { + 8 => 16, + 10 => 18, + _ => 20, + }; + + byte columnRange = bitDepth == 12 ? (byte)18 : (byte)16; + + for (int i = 0; i < this.StageNumberColumn; ++i) + { + this.stageRangeColumn[i] = columnRange; + } + + for (int i = 0; i < this.StageNumberRow; ++i) + { + this.stageRangeRow[i] = rowRange; + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Transform/Av1Transform2dOperations.cs b/src/ImageSharp/Formats/Heif/Av1/Transform/Av1Transform2dOperations.cs new file mode 100644 index 0000000000..5aa89f7804 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Transform/Av1Transform2dOperations.cs @@ -0,0 +1,1058 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.InteropServices; +using System.Runtime.Intrinsics; +using System.Runtime.Intrinsics.X86; +using SixLabors.ImageSharp.Common.Helpers; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +/// +/// Provides the SIMD data-layout operations shared by AV1 two-dimensional transforms. +/// +/// +/// The one-dimensional operators expect one transform position per vector and one independent axis per lane. These +/// routines transpose rectangular sample tiles into that structure, then transpose the completed axes back to raster +/// order. Every shuffle is consequently an index-bit exchange between row and column coordinates; it does not alter +/// the signed fixed-point sample representation. +/// +internal static class Av1Transform2dOperations +{ + /// + /// Gets the row order produced by the final AVX-512 16-by-16 transpose concatenation. + /// + private static ReadOnlySpan Vector512TransposeStoreOrder => [0, 2, 1, 3, 4, 6, 5, 7, 8, 10, 9, 11, 12, 14, 13, 15]; + + /// + /// Gets the row order produced by the final sixteen-bit 16-by-16 transpose concatenation. + /// + private static ReadOnlySpan Int16TransposeStoreOrder => [0, 4, 2, 6, 1, 5, 3, 7, 8, 12, 10, 14, 9, 13, 11, 15]; + + /// + /// Loads four signed sixteen-bit values and widens them to four signed thirty-two-bit lanes. + /// + /// The first source value. + /// The four widened values. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector128 Load4Int16(ref short source) + { + ulong packed = Unsafe.ReadUnaligned(ref Unsafe.As(ref source)); + return Vector128.WidenLower(Vector128.CreateScalarUnsafe(packed).AsInt16()); + } + + /// + /// Loads eight signed sixteen-bit values and widens them to eight signed thirty-two-bit lanes. + /// + /// The first source value. + /// The eight widened values. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector256 Load8Int16(ref short source) + => Vector256_.Widen(Vector128.LoadUnsafe(ref source)); + + /// + /// Loads sixteen signed sixteen-bit values and widens them to sixteen signed thirty-two-bit lanes. + /// + /// The first source value. + /// The sixteen widened values. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector512 Load16Int16(ref short source) + { + (Vector256 lower, Vector256 upper) = Vector256.Widen(Vector256.LoadUnsafe(ref source)); + + return Vector512.Create(lower, upper); + } + + /// + /// Applies a signed AV1 pipeline shift to four values in parallel. + /// + /// The values to shift. + /// A positive rounded-right shift or a negative exact-left shift. + /// The shifted values. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector128 RoundShift(Vector128 value, int bit) + { + if (bit > 0) + { + return (value + Vector128.Create(1 << (bit - 1))) >> bit; + } + + return bit < 0 ? value << -bit : value; + } + + /// + /// Applies a signed AV1 pipeline shift to eight values in parallel. + /// + /// The values to shift. + /// A positive rounded-right shift or a negative exact-left shift. + /// The shifted values. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector256 RoundShift(Vector256 value, int bit) + { + if (bit > 0) + { + return (value + Vector256.Create(1 << (bit - 1))) >> bit; + } + + return bit < 0 ? value << -bit : value; + } + + /// + /// Applies a signed AV1 pipeline shift to sixteen values in parallel. + /// + /// The values to shift. + /// A positive rounded-right shift or a negative exact-left shift. + /// The shifted values. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector512 RoundShift(Vector512 value, int bit) + { + if (bit > 0) + { + return (value + Vector512.Create(1 << (bit - 1))) >> bit; + } + + return bit < 0 ? value << -bit : value; + } + + /// + /// Applies a signed AV1 pipeline shift to eight signed sixteen-bit values in parallel. + /// + /// The values to shift. + /// A positive rounded-right shift or a negative exact-left shift. + /// The shifted values. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector128 RoundShift(Vector128 value, int bit) + { + if (bit > 0) + { + return (value + Vector128.Create((short)(1 << (bit - 1)))) >> bit; + } + + return bit < 0 ? value << -bit : value; + } + + /// + /// Applies a signed AV1 pipeline shift to sixteen signed sixteen-bit values in parallel. + /// + /// The values to shift. + /// A positive rounded-right shift or a negative exact-left shift. + /// The shifted values. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector256 RoundShift(Vector256 value, int bit) + { + if (bit > 0) + { + // Conformant low-bit-depth stage ranges leave room for the rounding bias, so this intentionally uses the + // wrapping add used by the reference decoder rather than changing the normative result with a saturating instruction. + return (value + Vector256.Create((short)(1 << (bit - 1)))) >> bit; + } + + return bit < 0 ? value << -bit : value; + } + + /// + /// Applies a signed AV1 pipeline shift to thirty-two signed sixteen-bit values in parallel. + /// + /// The values to shift. + /// A positive rounded-right shift or a negative exact-left shift. + /// The shifted values. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector512 RoundShift(Vector512 value, int bit) + { + if (bit > 0) + { + return (value + Vector512.Create((short)(1 << (bit - 1)))) >> bit; + } + + return bit < 0 ? value << -bit : value; + } + + /// + /// Transposes one 4-by-4 tile of signed sixteen-bit values and applies the configured pipeline operations. + /// + /// The first value of the source tile. + /// The number of signed sixteen-bit values between source rows. + /// The first value of the destination tile. + /// The number of signed sixteen-bit values between destination rows. + /// The signed AV1 pipeline shift applied before transposition. + /// Whether to apply the AV1 square-root-of-two rectangular normalization. + public static void Transpose4x4Int16( + ref short source, + int sourceStride, + ref short destination, + int destinationStride, + int roundShift, + bool normalizeRectangle) + { + Vector128 row0 = Load4Short(ref source); + Vector128 row1 = Load4Short(ref Unsafe.Add(ref source, sourceStride)); + Vector128 row2 = Load4Short(ref Unsafe.Add(ref source, 2 * sourceStride)); + Vector128 row3 = Load4Short(ref Unsafe.Add(ref source, 3 * sourceStride)); + + row0 = Finish(row0, roundShift, normalizeRectangle); + row1 = Finish(row1, roundShift, normalizeRectangle); + row2 = Finish(row2, roundShift, normalizeRectangle); + row3 = Finish(row3, roundShift, normalizeRectangle); + + // Only the lower four lanes belong to the tile. Interleaving at Int16 and Int32 granularity exchanges the + // two row-index bits with the corresponding column-index bits without touching adjacent padded storage. + Vector128 pair0 = Vector128_.UnpackLow(row0, row1); + Vector128 pair1 = Vector128_.UnpackLow(row2, row3); + Vector128 columns01 = Vector128_.UnpackLow(pair0.AsInt32(), pair1.AsInt32()); + Vector128 columns23 = Vector128_.UnpackHigh(pair0.AsInt32(), pair1.AsInt32()); + + Store4Int16(columns01.AsUInt64().ToScalar(), ref destination); + Store4Int16(columns01.AsUInt64().GetElement(1), ref Unsafe.Add(ref destination, destinationStride)); + Store4Int16(columns23.AsUInt64().ToScalar(), ref Unsafe.Add(ref destination, 2 * destinationStride)); + Store4Int16(columns23.AsUInt64().GetElement(1), ref Unsafe.Add(ref destination, 3 * destinationStride)); + } + + /// + /// Transposes one 8-by-8 tile of signed sixteen-bit values and applies the configured pipeline operations. + /// + /// The first value of the source tile. + /// The number of signed sixteen-bit values between source rows. + /// The first value of the destination tile. + /// The number of signed sixteen-bit values between destination rows. + /// The signed AV1 pipeline shift applied before transposition. + /// Whether to apply the AV1 square-root-of-two rectangular normalization. + public static void Transpose8x8Int16( + ref short source, + int sourceStride, + ref short destination, + int destinationStride, + int roundShift, + bool normalizeRectangle) + { + Vector128 row0 = Finish(Vector128.LoadUnsafe(ref source), roundShift, normalizeRectangle); + Vector128 row1 = Finish(Vector128.LoadUnsafe(ref source, (nuint)sourceStride), roundShift, normalizeRectangle); + Vector128 row2 = Finish(Vector128.LoadUnsafe(ref source, (nuint)(2 * sourceStride)), roundShift, normalizeRectangle); + Vector128 row3 = Finish(Vector128.LoadUnsafe(ref source, (nuint)(3 * sourceStride)), roundShift, normalizeRectangle); + Vector128 row4 = Finish(Vector128.LoadUnsafe(ref source, (nuint)(4 * sourceStride)), roundShift, normalizeRectangle); + Vector128 row5 = Finish(Vector128.LoadUnsafe(ref source, (nuint)(5 * sourceStride)), roundShift, normalizeRectangle); + Vector128 row6 = Finish(Vector128.LoadUnsafe(ref source, (nuint)(6 * sourceStride)), roundShift, normalizeRectangle); + Vector128 row7 = Finish(Vector128.LoadUnsafe(ref source, (nuint)(7 * sourceStride)), roundShift, normalizeRectangle); + + Vector128 pair0 = Vector128_.UnpackLow(row0, row1); + Vector128 pair1 = Vector128_.UnpackHigh(row0, row1); + Vector128 pair2 = Vector128_.UnpackLow(row2, row3); + Vector128 pair3 = Vector128_.UnpackHigh(row2, row3); + Vector128 pair4 = Vector128_.UnpackLow(row4, row5); + Vector128 pair5 = Vector128_.UnpackHigh(row4, row5); + Vector128 pair6 = Vector128_.UnpackLow(row6, row7); + Vector128 pair7 = Vector128_.UnpackHigh(row6, row7); + Vector128 quad0 = Vector128_.UnpackLow(pair0.AsInt32(), pair2.AsInt32()); + Vector128 quad1 = Vector128_.UnpackHigh(pair0.AsInt32(), pair2.AsInt32()); + Vector128 quad2 = Vector128_.UnpackLow(pair1.AsInt32(), pair3.AsInt32()); + Vector128 quad3 = Vector128_.UnpackHigh(pair1.AsInt32(), pair3.AsInt32()); + Vector128 quad4 = Vector128_.UnpackLow(pair4.AsInt32(), pair6.AsInt32()); + Vector128 quad5 = Vector128_.UnpackHigh(pair4.AsInt32(), pair6.AsInt32()); + Vector128 quad6 = Vector128_.UnpackLow(pair5.AsInt32(), pair7.AsInt32()); + Vector128 quad7 = Vector128_.UnpackHigh(pair5.AsInt32(), pair7.AsInt32()); + + Vector128_.UnpackLow(quad0.AsInt64(), quad4.AsInt64()).AsInt16().StoreUnsafe(ref destination); + Vector128_.UnpackHigh(quad0.AsInt64(), quad4.AsInt64()).AsInt16().StoreUnsafe(ref destination, (nuint)destinationStride); + Vector128_.UnpackLow(quad1.AsInt64(), quad5.AsInt64()).AsInt16().StoreUnsafe(ref destination, (nuint)(2 * destinationStride)); + Vector128_.UnpackHigh(quad1.AsInt64(), quad5.AsInt64()).AsInt16().StoreUnsafe(ref destination, (nuint)(3 * destinationStride)); + Vector128_.UnpackLow(quad2.AsInt64(), quad6.AsInt64()).AsInt16().StoreUnsafe(ref destination, (nuint)(4 * destinationStride)); + Vector128_.UnpackHigh(quad2.AsInt64(), quad6.AsInt64()).AsInt16().StoreUnsafe(ref destination, (nuint)(5 * destinationStride)); + Vector128_.UnpackLow(quad3.AsInt64(), quad7.AsInt64()).AsInt16().StoreUnsafe(ref destination, (nuint)(6 * destinationStride)); + Vector128_.UnpackHigh(quad3.AsInt64(), quad7.AsInt64()).AsInt16().StoreUnsafe(ref destination, (nuint)(7 * destinationStride)); + } + + /// + /// Transposes one 16-by-16 tile of signed sixteen-bit values and applies the configured pipeline shift. + /// + /// The first value of the source tile. + /// The number of signed sixteen-bit values between source rows. + /// The first value of the destination tile. + /// The number of signed sixteen-bit values between destination rows. + /// The reusable storage for the widening transpose stages. + /// The signed AV1 pipeline shift applied before transposition. + /// Whether to apply the AV1 square-root-of-two rectangular normalization. + public static void Transpose16x16Int16( + ref short source, + int sourceStride, + ref short destination, + int destinationStride, + Span scratch, + int roundShift, + bool normalizeRectangle) + { + ref long scratch64 = ref MemoryMarshal.GetReference(scratch); + ref int scratch32 = ref Unsafe.As(ref scratch64); + + // Pairing adjacent rows widens groups of two Int16 values into Int32 storage. The widening is a bitwise + // reinterpretation: it preserves all sixteen source bits while progressively exchanging row and column bits. + for (int row = 0; row < 16; row += 2) + { + Vector256 even = Vector256.LoadUnsafe(ref source, (nuint)(row * sourceStride)); + Vector256 odd = Vector256.LoadUnsafe(ref source, (nuint)((row + 1) * sourceStride)); + even = RoundShift(even, roundShift); + odd = RoundShift(odd, roundShift); + + if (normalizeRectangle) + { + even = Forward.Av1ForwardTransformArithmetic>.MultiplyRound( + even, + Av1Transform1dMath.NewSqrt2, + Av1Transform1dMath.NewSqrt2Bits); + + odd = Forward.Av1ForwardTransformArithmetic>.MultiplyRound( + odd, + Av1Transform1dMath.NewSqrt2, + Av1Transform1dMath.NewSqrt2Bits); + } + + Avx2.UnpackLow(even, odd).AsInt32().StoreUnsafe(ref scratch32, (nuint)(row * 8)); + Avx2.UnpackHigh(even, odd).AsInt32().StoreUnsafe(ref scratch32, (nuint)((row + 1) * 8)); + } + + // The Int32 and Int64 views exchange the next two index bits without allocating another temporary buffer. + // Each group is fully consumed before its destination slots overwrite the same scratch locations. + for (int row = 0; row < 16; row += 4) + { + for (int offset = 0; offset < 2; offset++) + { + Vector256 lower = Vector256.LoadUnsafe(ref scratch32, (nuint)((row + offset) * 8)); + Vector256 upper = Vector256.LoadUnsafe(ref scratch32, (nuint)((row + offset + 2) * 8)); + Avx2.UnpackLow(lower, upper).AsInt64().StoreUnsafe(ref scratch64, (nuint)((row + offset) * 4)); + Avx2.UnpackHigh(lower, upper).AsInt64().StoreUnsafe(ref scratch64, (nuint)((row + offset + 2) * 4)); + } + } + + for (int row = 0; row < 16; row += 8) + { + for (int offset = 0; offset < 4; offset++) + { + Vector256 lower = Vector256.LoadUnsafe(ref scratch64, (nuint)((row + offset) * 4)); + Vector256 upper = Vector256.LoadUnsafe(ref scratch64, (nuint)((row + offset + 4) * 4)); + Avx2.UnpackLow(lower, upper).StoreUnsafe(ref scratch64, (nuint)((row + offset) * 4)); + Avx2.UnpackHigh(lower, upper).StoreUnsafe(ref scratch64, (nuint)((row + offset + 4) * 4)); + } + } + + // Concatenating the matching 128-bit halves restores sixteen Int16 lanes per output row. The staged unpack + // order produces a fixed row permutation, so the compile-time table maps each register to its true column. + for (int row = 0; row < 8; row++) + { + Vector256 lower = Vector256.LoadUnsafe(ref scratch64, (nuint)(row * 4)); + Vector256 upper = Vector256.LoadUnsafe(ref scratch64, (nuint)((row + 8) * 4)); + Vector256 lowerResult = Vector256.Create(lower.GetLower(), upper.GetLower()).AsInt16(); + Vector256 upperResult = Vector256.Create(lower.GetUpper(), upper.GetUpper()).AsInt16(); + int lowerDestinationRow = Int16TransposeStoreOrder[row]; + int upperDestinationRow = Int16TransposeStoreOrder[row + 8]; + lowerResult.StoreUnsafe(ref destination, (nuint)(lowerDestinationRow * destinationStride)); + upperResult.StoreUnsafe(ref destination, (nuint)(upperDestinationRow * destinationStride)); + } + } + + /// + /// Promotes and transposes one 16-by-16 tile of signed sixteen-bit values. + /// + /// The first value of the source tile. + /// The number of signed sixteen-bit values between source rows. + /// The first value of the destination tile. + /// The number of signed thirty-two-bit values between destination rows. + /// The reusable storage for the promoted source tile. + /// The reusable storage for the widening transpose stages. + /// The signed AV1 pipeline shift applied after promotion. + /// Whether to apply the AV1 square-root-of-two rectangular normalization. + public static void Transpose16x16Int16ToInt32( + ref short source, + int sourceStride, + ref int destination, + int destinationStride, + Span promotionBuffer, + Span transposeScratch, + int roundShift, + bool normalizeRectangle) + { + ref int promotionBase = ref MemoryMarshal.GetReference(promotionBuffer); + + // The large low-bit-depth transforms widen at the axis boundary. Applying the pipeline shift after widening + // is significant: a left shift that is valid in Int32 is not required to remain representable in Int16. + for (int row = 0; row < 16; row++) + { + Vector256 packed = Vector256.LoadUnsafe(ref source, (nuint)(row * sourceStride)); + (Vector256 lower, Vector256 upper) = Vector256.Widen(packed); + + Vector512.Create(lower, upper).StoreUnsafe(ref promotionBase, (nuint)(row * 16)); + } + + // Once promoted, the same bounded transpose used by the high-bit-depth AVX-512 path supplies the exact + // the reference decoder staging order and performs the axis-boundary shift in signed thirty-two-bit lanes. + Transpose16x16Avx512( + ref promotionBase, + 16, + ref destination, + destinationStride, + transposeScratch, + roundShift, + normalizeRectangle); + } + + /// + /// Promotes and transposes one 8-by-8 tile of signed sixteen-bit values. + /// + /// The first value of the source tile. + /// The number of signed sixteen-bit values between source rows. + /// The first value of the destination tile. + /// The number of signed thirty-two-bit values between destination rows. + /// The reusable storage for the promoted source tile. + /// The signed AV1 pipeline shift applied after promotion. + /// Whether to apply the AV1 square-root-of-two rectangular normalization. + public static void Transpose8x8Int16ToInt32( + ref short source, + int sourceStride, + ref int destination, + int destinationStride, + Span promotionBuffer, + int roundShift, + bool normalizeRectangle) + { + ref int promotionBase = ref MemoryMarshal.GetReference(promotionBuffer); + + // AVX2 processes eight Int32 transform axes at a time. Widening each packed row before the axis shift follows + // the reference decoder's Repartition boundary and prevents valid Int32 intermediates from wrapping in Int16. + for (int row = 0; row < 8; row++) + { + Vector128 packed = Vector128.LoadUnsafe(ref source, (nuint)(row * sourceStride)); + (Vector128 lower, Vector128 upper) = Vector128.Widen(packed); + Vector256 promoted = Finish(Vector256.Create(lower, upper), roundShift, normalizeRectangle); + + promoted.StoreUnsafe(ref promotionBase, (nuint)(row * 8)); + } + + Transpose8x8Int32(ref promotionBase, 8, ref destination, destinationStride, 0, false); + } + + /// + /// Promotes and transposes one 4-by-4 tile of signed sixteen-bit values. + /// + /// The first value of the source tile. + /// The number of signed sixteen-bit values between source rows. + /// The first value of the destination tile. + /// The number of signed thirty-two-bit values between destination rows. + /// The reusable storage for the promoted source tile. + /// The signed AV1 pipeline shift applied after promotion. + /// Whether to apply the AV1 square-root-of-two rectangular normalization. + public static void Transpose4x4Int16ToInt32( + ref short source, + int sourceStride, + ref int destination, + int destinationStride, + Span promotionBuffer, + int roundShift, + bool normalizeRectangle) + { + ref int promotionBase = ref MemoryMarshal.GetReference(promotionBuffer); + + // The portable vector path retains four independent Int32 axes. Only the lower half is populated because a + // four-wide tile must not read the padded values belonging to its neighboring transform tile. + for (int row = 0; row < 4; row++) + { + Vector128 packed = Load4Short(ref Unsafe.Add(ref source, row * sourceStride)); + (Vector128 promoted, _) = Vector128.Widen(packed); + promoted = Finish(promoted, roundShift, normalizeRectangle); + + promoted.StoreUnsafe(ref promotionBase, (nuint)(row * 4)); + } + + Transpose4x4Int32(ref promotionBase, 4, ref destination, destinationStride, 0, false); + } + + /// + /// Reverses four signed thirty-two-bit lanes. + /// + /// The values to reverse. + /// The values in reverse lane order. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector128 Reverse(Vector128 value) + => Vector128.ShuffleNative(value, Vector128.Create(3, 2, 1, 0)); + + /// + /// Reverses eight signed sixteen-bit lanes. + /// + /// The values to reverse. + /// The values in reverse lane order. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector128 Reverse(Vector128 value) + => Vector128.ShuffleNative(value, Vector128.Create((short)7, 6, 5, 4, 3, 2, 1, 0)); + + /// + /// Reverses eight signed thirty-two-bit lanes. + /// + /// The values to reverse. + /// The values in reverse lane order. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector256 Reverse(Vector256 value) + => Vector256.ShuffleNative(value, Vector256.Create(7, 6, 5, 4, 3, 2, 1, 0)); + + /// + /// Reverses sixteen signed sixteen-bit lanes. + /// + /// The values to reverse. + /// The values in reverse lane order. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector256 Reverse(Vector256 value) + => Vector256.ShuffleNative(value, Vector256.Create((short)15, 14, 13, 12, 11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0)); + + /// + /// Reverses sixteen signed thirty-two-bit lanes. + /// + /// The values to reverse. + /// The values in reverse lane order. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector512 Reverse(Vector512 value) + => Vector512.ShuffleNative(value, Vector512.Create(15, 14, 13, 12, 11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0)); + + /// + /// Reverses thirty-two signed sixteen-bit lanes. + /// + /// The values to reverse. + /// The values in reverse lane order. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector512 Reverse(Vector512 value) + => Vector512.ShuffleNative( + value, + Vector512.Create((short)31, 30, 29, 28, 27, 26, 25, 24, 23, 22, 21, 20, 19, 18, 17, 16, 15, 14, 13, 12, 11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0)); + + /// + /// Transposes a four-by-four matrix of signed thirty-two-bit lanes. + /// + /// The first input row, replaced by the first output row. + /// The second input row, replaced by the second output row. + /// The third input row, replaced by the third output row. + /// The fourth input row, replaced by the fourth output row. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void Transpose(ref Vector128 row0, ref Vector128 row1, ref Vector128 row2, ref Vector128 row3) + { + Vector128 pairs01Low = Vector128_.UnpackLow(row0, row1); + Vector128 pairs01High = Vector128_.UnpackHigh(row0, row1); + Vector128 pairs23Low = Vector128_.UnpackLow(row2, row3); + Vector128 pairs23High = Vector128_.UnpackHigh(row2, row3); + + row0 = Vector128_.UnpackLow(pairs01Low.AsInt64(), pairs23Low.AsInt64()).AsInt32(); + row1 = Vector128_.UnpackHigh(pairs01Low.AsInt64(), pairs23Low.AsInt64()).AsInt32(); + row2 = Vector128_.UnpackLow(pairs01High.AsInt64(), pairs23High.AsInt64()).AsInt32(); + row3 = Vector128_.UnpackHigh(pairs01High.AsInt64(), pairs23High.AsInt64()).AsInt32(); + } + + /// + /// Transposes one 4-by-4 tile of signed thirty-two-bit values and applies the configured pipeline operations. + /// + /// The first value of the source tile. + /// The number of values between source rows. + /// The first value of the destination tile. + /// The number of values between destination rows. + /// The signed AV1 pipeline shift applied before transposition. + /// Whether to apply the AV1 square-root-of-two rectangular normalization. + public static void Transpose4x4Int32( + ref int source, + int sourceStride, + ref int destination, + int destinationStride, + int roundShift, + bool normalizeRectangle) + { + Vector128 row0 = Finish(Vector128.LoadUnsafe(ref source), roundShift, normalizeRectangle); + Vector128 row1 = Finish(Vector128.LoadUnsafe(ref source, (nuint)sourceStride), roundShift, normalizeRectangle); + Vector128 row2 = Finish(Vector128.LoadUnsafe(ref source, (nuint)(2 * sourceStride)), roundShift, normalizeRectangle); + Vector128 row3 = Finish(Vector128.LoadUnsafe(ref source, (nuint)(3 * sourceStride)), roundShift, normalizeRectangle); + + Transpose(ref row0, ref row1, ref row2, ref row3); + row0.StoreUnsafe(ref destination); + row1.StoreUnsafe(ref destination, (nuint)destinationStride); + row2.StoreUnsafe(ref destination, (nuint)(2 * destinationStride)); + row3.StoreUnsafe(ref destination, (nuint)(3 * destinationStride)); + } + + /// + /// Transposes an eight-by-eight matrix of signed thirty-two-bit lanes. + /// + /// The first input row, replaced by the first output row. + /// The second input row, replaced by the second output row. + /// The third input row, replaced by the third output row. + /// The fourth input row, replaced by the fourth output row. + /// The fifth input row, replaced by the fifth output row. + /// The sixth input row, replaced by the sixth output row. + /// The seventh input row, replaced by the seventh output row. + /// The eighth input row, replaced by the eighth output row. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void Transpose( + ref Vector256 row0, + ref Vector256 row1, + ref Vector256 row2, + ref Vector256 row3, + ref Vector256 row4, + ref Vector256 row5, + ref Vector256 row6, + ref Vector256 row7) + { + Vector128 column0Lower = row0.GetLower(); + Vector128 column1Lower = row1.GetLower(); + Vector128 column2Lower = row2.GetLower(); + Vector128 column3Lower = row3.GetLower(); + Transpose(ref column0Lower, ref column1Lower, ref column2Lower, ref column3Lower); + + Vector128 column0Upper = row4.GetLower(); + Vector128 column1Upper = row5.GetLower(); + Vector128 column2Upper = row6.GetLower(); + Vector128 column3Upper = row7.GetLower(); + Transpose(ref column0Upper, ref column1Upper, ref column2Upper, ref column3Upper); + + Vector128 column4Lower = row0.GetUpper(); + Vector128 column5Lower = row1.GetUpper(); + Vector128 column6Lower = row2.GetUpper(); + Vector128 column7Lower = row3.GetUpper(); + Transpose(ref column4Lower, ref column5Lower, ref column6Lower, ref column7Lower); + + Vector128 column4Upper = row4.GetUpper(); + Vector128 column5Upper = row5.GetUpper(); + Vector128 column6Upper = row6.GetUpper(); + Vector128 column7Upper = row7.GetUpper(); + Transpose(ref column4Upper, ref column5Upper, ref column6Upper, ref column7Upper); + + row0 = Vector256.Create(column0Lower, column0Upper); + row1 = Vector256.Create(column1Lower, column1Upper); + row2 = Vector256.Create(column2Lower, column2Upper); + row3 = Vector256.Create(column3Lower, column3Upper); + row4 = Vector256.Create(column4Lower, column4Upper); + row5 = Vector256.Create(column5Lower, column5Upper); + row6 = Vector256.Create(column6Lower, column6Upper); + row7 = Vector256.Create(column7Lower, column7Upper); + } + + /// + /// Transposes one 8-by-8 tile of signed thirty-two-bit values and applies the configured pipeline operations. + /// + /// The first value of the source tile. + /// The number of values between source rows. + /// The first value of the destination tile. + /// The number of values between destination rows. + /// The signed AV1 pipeline shift applied before transposition. + /// Whether to apply the AV1 square-root-of-two rectangular normalization. + public static void Transpose8x8Int32( + ref int source, + int sourceStride, + ref int destination, + int destinationStride, + int roundShift, + bool normalizeRectangle) + { + Vector256 row0 = Finish(Vector256.LoadUnsafe(ref source), roundShift, normalizeRectangle); + Vector256 row1 = Finish(Vector256.LoadUnsafe(ref source, (nuint)sourceStride), roundShift, normalizeRectangle); + Vector256 row2 = Finish(Vector256.LoadUnsafe(ref source, (nuint)(2 * sourceStride)), roundShift, normalizeRectangle); + Vector256 row3 = Finish(Vector256.LoadUnsafe(ref source, (nuint)(3 * sourceStride)), roundShift, normalizeRectangle); + Vector256 row4 = Finish(Vector256.LoadUnsafe(ref source, (nuint)(4 * sourceStride)), roundShift, normalizeRectangle); + Vector256 row5 = Finish(Vector256.LoadUnsafe(ref source, (nuint)(5 * sourceStride)), roundShift, normalizeRectangle); + Vector256 row6 = Finish(Vector256.LoadUnsafe(ref source, (nuint)(6 * sourceStride)), roundShift, normalizeRectangle); + Vector256 row7 = Finish(Vector256.LoadUnsafe(ref source, (nuint)(7 * sourceStride)), roundShift, normalizeRectangle); + + Transpose(ref row0, ref row1, ref row2, ref row3, ref row4, ref row5, ref row6, ref row7); + row0.StoreUnsafe(ref destination); + row1.StoreUnsafe(ref destination, (nuint)destinationStride); + row2.StoreUnsafe(ref destination, (nuint)(2 * destinationStride)); + row3.StoreUnsafe(ref destination, (nuint)(3 * destinationStride)); + row4.StoreUnsafe(ref destination, (nuint)(4 * destinationStride)); + row5.StoreUnsafe(ref destination, (nuint)(5 * destinationStride)); + row6.StoreUnsafe(ref destination, (nuint)(6 * destinationStride)); + row7.StoreUnsafe(ref destination, (nuint)(7 * destinationStride)); + } + + /// + /// Transposes a sixteen-by-sixteen matrix of signed thirty-two-bit values with the AVX-512 staging layout. + /// + /// The first value in the source matrix. + /// The number of values between source rows. + /// The first value in the destination matrix. + /// The number of values between destination rows. + /// The caller-owned storage for sixteen vectors of signed sixty-four-bit lanes. + /// The right shift applied with AV1 signed rounding before transposition. + /// Whether to apply the AV1 square-root-of-two rectangular normalization. + public static void Transpose16x16Avx512( + ref int source, + int sourceStride, + ref int destination, + int destinationStride, + Span scratch, + int roundShift, + bool normalizeRectangle) + { + ref long scratchBase = ref MemoryMarshal.GetReference(scratch); + + // the reference decoder widens the lane grouping after each local interleave rather than retaining all sixteen rows in + // registers. The bounded scratch keeps the live register set small and prevents the JIT from spilling a + // four-stage, sixteen-register cross-vector permutation network into its own stack frame. + for (int row = 0; row < 16; row += 2) + { + Vector512 even = Vector512.LoadUnsafe(ref source, (nuint)(row * sourceStride)); + Vector512 odd = Vector512.LoadUnsafe(ref source, (nuint)((row + 1) * sourceStride)); + even = RoundShift(even, roundShift); + odd = RoundShift(odd, roundShift); + + if (normalizeRectangle) + { + even = Av1Transform1dMath.MultiplyRound(even, Av1Transform1dMath.NewSqrt2, Av1Transform1dMath.NewSqrt2Bits); + odd = Av1Transform1dMath.MultiplyRound(odd, Av1Transform1dMath.NewSqrt2, Av1Transform1dMath.NewSqrt2Bits); + } + + Avx512F.UnpackLow(even, odd).AsInt64().StoreUnsafe(ref scratchBase, (nuint)(row * 8)); + Avx512F.UnpackHigh(even, odd).AsInt64().StoreUnsafe(ref scratchBase, (nuint)((row + 1) * 8)); + } + + // The second stage exchanges the next row and column bits with 64-bit unpack operations. Each iteration + // reads its complete four-row group before replacing that group in scratch. + for (int row = 0; row < 16; row += 4) + { + for (int offset = 0; offset < 2; offset++) + { + Vector512 lower = Vector512.LoadUnsafe(ref scratchBase, (nuint)((row + offset) * 8)); + Vector512 upper = Vector512.LoadUnsafe(ref scratchBase, (nuint)((row + offset + 2) * 8)); + Avx512F.UnpackLow(lower, upper).StoreUnsafe(ref scratchBase, (nuint)((row + offset) * 8)); + Avx512F.UnpackHigh(lower, upper).StoreUnsafe(ref scratchBase, (nuint)((row + offset + 2) * 8)); + } + } + + Vector512 evenBlockIndices = Vector512.Create(0L, 1L, 8L, 9L, 4L, 5L, 12L, 13L); + Vector512 oddBlockIndices = Vector512.Create(2L, 3L, 10L, 11L, 6L, 7L, 14L, 15L); + + // Highway's LocalInterleaveEvenBlocks and LocalInterleaveOddBlocks exchange the third matrix-index bit with + // one two-table lookup per result. The index vectors address the lower source as 0-7 and the upper as 8-15. + for (int row = 0; row < 16; row += 8) + { + for (int offset = 0; offset < 4; offset++) + { + Vector512 lower = Vector512.LoadUnsafe(ref scratchBase, (nuint)((row + offset) * 8)); + Vector512 upper = Vector512.LoadUnsafe(ref scratchBase, (nuint)((row + offset + 4) * 8)); + Avx512F.PermuteVar8x64x2(lower, evenBlockIndices, upper).StoreUnsafe(ref scratchBase, (nuint)((row + offset) * 8)); + Avx512F.PermuteVar8x64x2(lower, oddBlockIndices, upper).StoreUnsafe(ref scratchBase, (nuint)((row + offset + 4) * 8)); + } + } + + // The final 128-bit-block concatenations complete the transpose. The store order is the fixed permutation + // produced by the reference decoder's three preceding local-interleave stages. + for (int row = 0; row < 8; row++) + { + Vector512 lower = Vector512.LoadUnsafe(ref scratchBase, (nuint)(row * 8)); + Vector512 upper = Vector512.LoadUnsafe(ref scratchBase, (nuint)((row + 8) * 8)); + Vector512 lowerResult = Avx512F.Shuffle4x128(lower.AsInt32(), upper.AsInt32(), 0x44); + Vector512 upperResult = Avx512F.Shuffle4x128(lower.AsInt32(), upper.AsInt32(), 0xEE); + int lowerDestinationRow = Vector512TransposeStoreOrder[row]; + int upperDestinationRow = Vector512TransposeStoreOrder[row + 8]; + lowerResult.StoreUnsafe(ref destination, (nuint)(lowerDestinationRow * destinationStride)); + upperResult.StoreUnsafe(ref destination, (nuint)(upperDestinationRow * destinationStride)); + } + } + + /// + /// Transposes a sixteen-by-sixteen matrix of signed thirty-two-bit lanes. + /// + /// The first input row, replaced by the first output row. + /// The second input row, replaced by the second output row. + /// The third input row, replaced by the third output row. + /// The fourth input row, replaced by the fourth output row. + /// The fifth input row, replaced by the fifth output row. + /// The sixth input row, replaced by the sixth output row. + /// The seventh input row, replaced by the seventh output row. + /// The eighth input row, replaced by the eighth output row. + /// The ninth input row, replaced by the ninth output row. + /// The tenth input row, replaced by the tenth output row. + /// The eleventh input row, replaced by the eleventh output row. + /// The twelfth input row, replaced by the twelfth output row. + /// The thirteenth input row, replaced by the thirteenth output row. + /// The fourteenth input row, replaced by the fourteenth output row. + /// The fifteenth input row, replaced by the fifteenth output row. + /// The sixteenth input row, replaced by the sixteenth output row. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void Transpose( + ref Vector512 row0, + ref Vector512 row1, + ref Vector512 row2, + ref Vector512 row3, + ref Vector512 row4, + ref Vector512 row5, + ref Vector512 row6, + ref Vector512 row7, + ref Vector512 row8, + ref Vector512 row9, + ref Vector512 row10, + ref Vector512 row11, + ref Vector512 row12, + ref Vector512 row13, + ref Vector512 row14, + ref Vector512 row15) + { + if (Avx512F.IsSupported) + { + // Each permutation stage exchanges one row-index bit with the matching column-index bit. After four + // stages the vector index identifies the source column and the lane index identifies the source row. + // This keeps the complete transpose in 512-bit registers instead of decomposing it into 128-bit tiles. + Vector512 stage0Lower = Vector512.Create(0, 16, 2, 18, 4, 20, 6, 22, 8, 24, 10, 26, 12, 28, 14, 30); + Vector512 stage0Upper = Vector512.Create(1, 17, 3, 19, 5, 21, 7, 23, 9, 25, 11, 27, 13, 29, 15, 31); + Vector512 lowerSource = row0; + Vector512 upperSource = row1; + row0 = Avx512F.PermuteVar16x32x2(lowerSource, stage0Lower, upperSource); + row1 = Avx512F.PermuteVar16x32x2(lowerSource, stage0Upper, upperSource); + lowerSource = row2; + upperSource = row3; + row2 = Avx512F.PermuteVar16x32x2(lowerSource, stage0Lower, upperSource); + row3 = Avx512F.PermuteVar16x32x2(lowerSource, stage0Upper, upperSource); + lowerSource = row4; + upperSource = row5; + row4 = Avx512F.PermuteVar16x32x2(lowerSource, stage0Lower, upperSource); + row5 = Avx512F.PermuteVar16x32x2(lowerSource, stage0Upper, upperSource); + lowerSource = row6; + upperSource = row7; + row6 = Avx512F.PermuteVar16x32x2(lowerSource, stage0Lower, upperSource); + row7 = Avx512F.PermuteVar16x32x2(lowerSource, stage0Upper, upperSource); + lowerSource = row8; + upperSource = row9; + row8 = Avx512F.PermuteVar16x32x2(lowerSource, stage0Lower, upperSource); + row9 = Avx512F.PermuteVar16x32x2(lowerSource, stage0Upper, upperSource); + lowerSource = row10; + upperSource = row11; + row10 = Avx512F.PermuteVar16x32x2(lowerSource, stage0Lower, upperSource); + row11 = Avx512F.PermuteVar16x32x2(lowerSource, stage0Upper, upperSource); + lowerSource = row12; + upperSource = row13; + row12 = Avx512F.PermuteVar16x32x2(lowerSource, stage0Lower, upperSource); + row13 = Avx512F.PermuteVar16x32x2(lowerSource, stage0Upper, upperSource); + lowerSource = row14; + upperSource = row15; + row14 = Avx512F.PermuteVar16x32x2(lowerSource, stage0Lower, upperSource); + row15 = Avx512F.PermuteVar16x32x2(lowerSource, stage0Upper, upperSource); + + Vector512 stage1Lower = Vector512.Create(0, 1, 16, 17, 4, 5, 20, 21, 8, 9, 24, 25, 12, 13, 28, 29); + Vector512 stage1Upper = Vector512.Create(2, 3, 18, 19, 6, 7, 22, 23, 10, 11, 26, 27, 14, 15, 30, 31); + lowerSource = row0; + upperSource = row2; + row0 = Avx512F.PermuteVar16x32x2(lowerSource, stage1Lower, upperSource); + row2 = Avx512F.PermuteVar16x32x2(lowerSource, stage1Upper, upperSource); + lowerSource = row1; + upperSource = row3; + row1 = Avx512F.PermuteVar16x32x2(lowerSource, stage1Lower, upperSource); + row3 = Avx512F.PermuteVar16x32x2(lowerSource, stage1Upper, upperSource); + lowerSource = row4; + upperSource = row6; + row4 = Avx512F.PermuteVar16x32x2(lowerSource, stage1Lower, upperSource); + row6 = Avx512F.PermuteVar16x32x2(lowerSource, stage1Upper, upperSource); + lowerSource = row5; + upperSource = row7; + row5 = Avx512F.PermuteVar16x32x2(lowerSource, stage1Lower, upperSource); + row7 = Avx512F.PermuteVar16x32x2(lowerSource, stage1Upper, upperSource); + lowerSource = row8; + upperSource = row10; + row8 = Avx512F.PermuteVar16x32x2(lowerSource, stage1Lower, upperSource); + row10 = Avx512F.PermuteVar16x32x2(lowerSource, stage1Upper, upperSource); + lowerSource = row9; + upperSource = row11; + row9 = Avx512F.PermuteVar16x32x2(lowerSource, stage1Lower, upperSource); + row11 = Avx512F.PermuteVar16x32x2(lowerSource, stage1Upper, upperSource); + lowerSource = row12; + upperSource = row14; + row12 = Avx512F.PermuteVar16x32x2(lowerSource, stage1Lower, upperSource); + row14 = Avx512F.PermuteVar16x32x2(lowerSource, stage1Upper, upperSource); + lowerSource = row13; + upperSource = row15; + row13 = Avx512F.PermuteVar16x32x2(lowerSource, stage1Lower, upperSource); + row15 = Avx512F.PermuteVar16x32x2(lowerSource, stage1Upper, upperSource); + + Vector512 stage2Lower = Vector512.Create(0, 1, 2, 3, 16, 17, 18, 19, 8, 9, 10, 11, 24, 25, 26, 27); + Vector512 stage2Upper = Vector512.Create(4, 5, 6, 7, 20, 21, 22, 23, 12, 13, 14, 15, 28, 29, 30, 31); + lowerSource = row0; + upperSource = row4; + row0 = Avx512F.PermuteVar16x32x2(lowerSource, stage2Lower, upperSource); + row4 = Avx512F.PermuteVar16x32x2(lowerSource, stage2Upper, upperSource); + lowerSource = row1; + upperSource = row5; + row1 = Avx512F.PermuteVar16x32x2(lowerSource, stage2Lower, upperSource); + row5 = Avx512F.PermuteVar16x32x2(lowerSource, stage2Upper, upperSource); + lowerSource = row2; + upperSource = row6; + row2 = Avx512F.PermuteVar16x32x2(lowerSource, stage2Lower, upperSource); + row6 = Avx512F.PermuteVar16x32x2(lowerSource, stage2Upper, upperSource); + lowerSource = row3; + upperSource = row7; + row3 = Avx512F.PermuteVar16x32x2(lowerSource, stage2Lower, upperSource); + row7 = Avx512F.PermuteVar16x32x2(lowerSource, stage2Upper, upperSource); + lowerSource = row8; + upperSource = row12; + row8 = Avx512F.PermuteVar16x32x2(lowerSource, stage2Lower, upperSource); + row12 = Avx512F.PermuteVar16x32x2(lowerSource, stage2Upper, upperSource); + lowerSource = row9; + upperSource = row13; + row9 = Avx512F.PermuteVar16x32x2(lowerSource, stage2Lower, upperSource); + row13 = Avx512F.PermuteVar16x32x2(lowerSource, stage2Upper, upperSource); + lowerSource = row10; + upperSource = row14; + row10 = Avx512F.PermuteVar16x32x2(lowerSource, stage2Lower, upperSource); + row14 = Avx512F.PermuteVar16x32x2(lowerSource, stage2Upper, upperSource); + lowerSource = row11; + upperSource = row15; + row11 = Avx512F.PermuteVar16x32x2(lowerSource, stage2Lower, upperSource); + row15 = Avx512F.PermuteVar16x32x2(lowerSource, stage2Upper, upperSource); + + Vector512 stage3Lower = Vector512.Create(0, 1, 2, 3, 4, 5, 6, 7, 16, 17, 18, 19, 20, 21, 22, 23); + Vector512 stage3Upper = Vector512.Create(8, 9, 10, 11, 12, 13, 14, 15, 24, 25, 26, 27, 28, 29, 30, 31); + lowerSource = row0; + upperSource = row8; + row0 = Avx512F.PermuteVar16x32x2(lowerSource, stage3Lower, upperSource); + row8 = Avx512F.PermuteVar16x32x2(lowerSource, stage3Upper, upperSource); + lowerSource = row1; + upperSource = row9; + row1 = Avx512F.PermuteVar16x32x2(lowerSource, stage3Lower, upperSource); + row9 = Avx512F.PermuteVar16x32x2(lowerSource, stage3Upper, upperSource); + lowerSource = row2; + upperSource = row10; + row2 = Avx512F.PermuteVar16x32x2(lowerSource, stage3Lower, upperSource); + row10 = Avx512F.PermuteVar16x32x2(lowerSource, stage3Upper, upperSource); + lowerSource = row3; + upperSource = row11; + row3 = Avx512F.PermuteVar16x32x2(lowerSource, stage3Lower, upperSource); + row11 = Avx512F.PermuteVar16x32x2(lowerSource, stage3Upper, upperSource); + lowerSource = row4; + upperSource = row12; + row4 = Avx512F.PermuteVar16x32x2(lowerSource, stage3Lower, upperSource); + row12 = Avx512F.PermuteVar16x32x2(lowerSource, stage3Upper, upperSource); + lowerSource = row5; + upperSource = row13; + row5 = Avx512F.PermuteVar16x32x2(lowerSource, stage3Lower, upperSource); + row13 = Avx512F.PermuteVar16x32x2(lowerSource, stage3Upper, upperSource); + lowerSource = row6; + upperSource = row14; + row6 = Avx512F.PermuteVar16x32x2(lowerSource, stage3Lower, upperSource); + row14 = Avx512F.PermuteVar16x32x2(lowerSource, stage3Upper, upperSource); + lowerSource = row7; + upperSource = row15; + row7 = Avx512F.PermuteVar16x32x2(lowerSource, stage3Lower, upperSource); + row15 = Avx512F.PermuteVar16x32x2(lowerSource, stage3Upper, upperSource); + return; + } + + // A 16x16 transpose consists of four independent 8x8 quadrants. Reusing the established 256-bit transpose + // keeps the portable layout path branch-free while the transform arithmetic itself remains in 512-bit lanes. + // Preserve the bottom-left quadrant before row8-row15 become upper-column output storage. Emitting those upper + // columns first avoids keeping all four quadrants live across the complete operation. + Vector256 lowerBottom0 = row8.GetLower(); + Vector256 lowerBottom1 = row9.GetLower(); + Vector256 lowerBottom2 = row10.GetLower(); + Vector256 lowerBottom3 = row11.GetLower(); + Vector256 lowerBottom4 = row12.GetLower(); + Vector256 lowerBottom5 = row13.GetLower(); + Vector256 lowerBottom6 = row14.GetLower(); + Vector256 lowerBottom7 = row15.GetLower(); + Vector256 upperTop0 = row0.GetUpper(); + Vector256 upperTop1 = row1.GetUpper(); + Vector256 upperTop2 = row2.GetUpper(); + Vector256 upperTop3 = row3.GetUpper(); + Vector256 upperTop4 = row4.GetUpper(); + Vector256 upperTop5 = row5.GetUpper(); + Vector256 upperTop6 = row6.GetUpper(); + Vector256 upperTop7 = row7.GetUpper(); + Vector256 upperBottom0 = row8.GetUpper(); + Vector256 upperBottom1 = row9.GetUpper(); + Vector256 upperBottom2 = row10.GetUpper(); + Vector256 upperBottom3 = row11.GetUpper(); + Vector256 upperBottom4 = row12.GetUpper(); + Vector256 upperBottom5 = row13.GetUpper(); + Vector256 upperBottom6 = row14.GetUpper(); + Vector256 upperBottom7 = row15.GetUpper(); + + Transpose(ref upperTop0, ref upperTop1, ref upperTop2, ref upperTop3, ref upperTop4, ref upperTop5, ref upperTop6, ref upperTop7); + Transpose(ref upperBottom0, ref upperBottom1, ref upperBottom2, ref upperBottom3, ref upperBottom4, ref upperBottom5, ref upperBottom6, ref upperBottom7); + + row8 = Vector512.Create(upperTop0, upperBottom0); + row9 = Vector512.Create(upperTop1, upperBottom1); + row10 = Vector512.Create(upperTop2, upperBottom2); + row11 = Vector512.Create(upperTop3, upperBottom3); + row12 = Vector512.Create(upperTop4, upperBottom4); + row13 = Vector512.Create(upperTop5, upperBottom5); + row14 = Vector512.Create(upperTop6, upperBottom6); + row15 = Vector512.Create(upperTop7, upperBottom7); + + Vector256 lowerTop0 = row0.GetLower(); + Vector256 lowerTop1 = row1.GetLower(); + Vector256 lowerTop2 = row2.GetLower(); + Vector256 lowerTop3 = row3.GetLower(); + Vector256 lowerTop4 = row4.GetLower(); + Vector256 lowerTop5 = row5.GetLower(); + Vector256 lowerTop6 = row6.GetLower(); + Vector256 lowerTop7 = row7.GetLower(); + + Transpose(ref lowerTop0, ref lowerTop1, ref lowerTop2, ref lowerTop3, ref lowerTop4, ref lowerTop5, ref lowerTop6, ref lowerTop7); + Transpose(ref lowerBottom0, ref lowerBottom1, ref lowerBottom2, ref lowerBottom3, ref lowerBottom4, ref lowerBottom5, ref lowerBottom6, ref lowerBottom7); + + row0 = Vector512.Create(lowerTop0, lowerBottom0); + row1 = Vector512.Create(lowerTop1, lowerBottom1); + row2 = Vector512.Create(lowerTop2, lowerBottom2); + row3 = Vector512.Create(lowerTop3, lowerBottom3); + row4 = Vector512.Create(lowerTop4, lowerBottom4); + row5 = Vector512.Create(lowerTop5, lowerBottom5); + row6 = Vector512.Create(lowerTop6, lowerBottom6); + row7 = Vector512.Create(lowerTop7, lowerBottom7); + } + + /// + /// Applies the terminal operations which the reference decoder performs before transposing a signed sixteen-bit tile. + /// + /// The packed transform values. + /// The signed AV1 pipeline shift. + /// Whether to apply square-root-of-two rectangular normalization. + /// The shifted and normalized values. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector128 Finish(Vector128 value, int roundShift, bool normalizeRectangle) + { + value = RoundShift(value, roundShift); + return normalizeRectangle + ? Forward.Av1ForwardTransformArithmetic>.MultiplyRound( + value, + Av1Transform1dMath.NewSqrt2, + Av1Transform1dMath.NewSqrt2Bits) + : value; + } + + /// + /// Applies the terminal operations which the reference decoder performs before transposing four signed thirty-two-bit lanes. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector128 Finish(Vector128 value, int roundShift, bool normalizeRectangle) + { + value = RoundShift(value, roundShift); + return normalizeRectangle + ? Av1Transform1dMath.MultiplyRound(value, Av1Transform1dMath.NewSqrt2, Av1Transform1dMath.NewSqrt2Bits) + : value; + } + + /// + /// Applies the terminal operations which the reference decoder performs before transposing eight signed thirty-two-bit lanes. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector256 Finish(Vector256 value, int roundShift, bool normalizeRectangle) + { + value = RoundShift(value, roundShift); + return normalizeRectangle + ? Av1Transform1dMath.MultiplyRound(value, Av1Transform1dMath.NewSqrt2, Av1Transform1dMath.NewSqrt2Bits) + : value; + } + + /// + /// Loads four signed sixteen-bit values without reading outside the source tile. + /// + /// The first source value. + /// The four source values in the lower vector lanes. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector128 Load4Short(ref short source) + => Vector128.Create(Unsafe.As(ref source), 0UL).AsInt16(); + + /// + /// Stores the lower four signed sixteen-bit lanes without writing outside the destination tile. + /// + /// The packed lower lanes. + /// The first destination value. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static void Store4Int16(ulong value, ref short destination) + => Unsafe.As(ref destination) = value; +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Transform/Av1TransformClass.cs b/src/ImageSharp/Formats/Heif/Av1/Transform/Av1TransformClass.cs new file mode 100644 index 0000000000..7593843739 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Transform/Av1TransformClass.cs @@ -0,0 +1,25 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +/// +/// Classifies AV1 transforms by the axes along which non-identity transforms operate. +/// +internal enum Av1TransformClass +{ + /// + /// A two-dimensional transform with non-identity processing on both axes. + /// + Class2D = 0, + + /// + /// A horizontal transform with identity processing on the vertical axis. + /// + ClassHorizontal = 1, + + /// + /// A vertical transform with identity processing on the horizontal axis. + /// + ClassVertical = 2, +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Transform/Av1TransformFunctionParameters.cs b/src/ImageSharp/Formats/Heif/Av1/Transform/Av1TransformFunctionParameters.cs new file mode 100644 index 0000000000..080e6aa8b2 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Transform/Av1TransformFunctionParameters.cs @@ -0,0 +1,40 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +/// +/// Carries the syntax and sample-storage parameters required to reconstruct one AV1 transform block. +/// +internal struct Av1TransformFunctionParameters +{ + /// + /// Gets or sets the compound transform type. + /// + public Av1TransformType TransformType { get; set; } + + /// + /// Gets or sets the transform-block dimensions. + /// + public Av1TransformSize TransformSize { get; set; } + + /// + /// Gets or sets the number of coefficient positions represented by the decoded coefficient buffer. + /// + public int EndOfBuffer { get; set; } + + /// + /// Gets or sets a value indicating whether the coded segment uses the AV1 lossless transform rules. + /// + public bool IsLossless { get; set; } + + /// + /// Gets or sets the decoded sample bit depth. + /// + public int BitDepth { get; set; } + + /// + /// Gets or sets a value indicating whether reconstructed samples use the 16-bit storage pipeline. + /// + public bool Is16BitPipeline { get; set; } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Transform/Av1TransformFunctionType.cs b/src/ImageSharp/Formats/Heif/Av1/Transform/Av1TransformFunctionType.cs new file mode 100644 index 0000000000..1ecb083271 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Transform/Av1TransformFunctionType.cs @@ -0,0 +1,75 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +/// +/// Identifies a concrete one-dimensional AV1 transform function and length. +/// +internal enum Av1TransformFunctionType +{ + /// + /// A four-sample discrete cosine transform. + /// + Dct4, + + /// + /// An eight-sample discrete cosine transform. + /// + Dct8, + + /// + /// A sixteen-sample discrete cosine transform. + /// + Dct16, + + /// + /// A thirty-two-sample discrete cosine transform. + /// + Dct32, + + /// + /// A sixty-four-sample discrete cosine transform. + /// + Dct64, + + /// + /// A four-sample asymmetric discrete sine transform. + /// + Adst4, + + /// + /// An eight-sample asymmetric discrete sine transform. + /// + Adst8, + + /// + /// A sixteen-sample asymmetric discrete sine transform. + /// + Adst16, + + /// + /// A four-sample identity transform. + /// + Identity4, + + /// + /// An eight-sample identity transform. + /// + Identity8, + + /// + /// A sixteen-sample identity transform. + /// + Identity16, + + /// + /// A thirty-two-sample identity transform. + /// + Identity32, + + /// + /// No valid transform function. + /// + Invalid, +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Transform/Av1TransformMode.cs b/src/ImageSharp/Formats/Heif/Av1/Transform/Av1TransformMode.cs new file mode 100644 index 0000000000..b600921171 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Transform/Av1TransformMode.cs @@ -0,0 +1,25 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +/// +/// Identifies how transform-block sizes are selected within an AV1 frame. +/// +internal enum Av1TransformMode : byte +{ + /// + /// Every transform block is four by four samples. + /// + Only4x4 = 0, + + /// + /// Each block uses the largest permitted transform size. + /// + Largest = 1, + + /// + /// Transform-block sizes are selected by block-level syntax. + /// + Select = 2, +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Transform/Av1TransformSetType.cs b/src/ImageSharp/Formats/Heif/Av1/Transform/Av1TransformSetType.cs new file mode 100644 index 0000000000..512ca6880b --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Transform/Av1TransformSetType.cs @@ -0,0 +1,47 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +/// +/// Identifies the set of transform combinations allowed for an AV1 block. +/// +internal enum Av1TransformSetType +{ + /// + /// Allowed transforms: DCT only. + /// + DctOnly, + + /// + /// Allowed transforms: DCT + Identity only + /// + InterSet3, + + /// + /// Allowed transforms: Discrete Trig transforms w/o flip (4) + Identity (1) + /// + /// Referenced in spec as TX_SET_INTRA_2. + IntraSet2, + + /// + /// Allowed transforms: Discrete Trig transforms w/o flip (4) + Identity (1) + 1D Hor/vert DCT (2) + /// + /// Referenced in spec as TX_SET_INTRA_1. + IntraSet1, + + /// + /// Allowed transforms: Discrete Trig transforms w/ flip (9) + Identity (1) + 1D Hor/Ver DCT (2) + /// + InterSet2, + + /// + /// Allowed transforms: Discrete Trig transforms w/ flip (9) + Identity (1) + 1D Hor/Ver (6) + /// + InterSet1, + + /// + /// The number of defined transform sets. + /// + AllSets +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Transform/Av1TransformSize.cs b/src/ImageSharp/Formats/Heif/Av1/Transform/Av1TransformSize.cs new file mode 100644 index 0000000000..7d1aa856ce --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Transform/Av1TransformSize.cs @@ -0,0 +1,120 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +/// +/// Identifies every square and rectangular transform-block size defined by AV1. +/// +internal enum Av1TransformSize : byte +{ + /// + /// A 4-by-4 transform block. + /// + Size4x4 = 0, + + /// + /// An 8-by-8 transform block. + /// + Size8x8 = 1, + + /// + /// A 16-by-16 transform block. + /// + Size16x16 = 2, + + /// + /// A 32-by-32 transform block. + /// + Size32x32 = 3, + + /// + /// A 64-by-64 transform block. + /// + Size64x64 = 4, + + /// + /// A 4-by-8 transform block. + /// + Size4x8 = 5, + + /// + /// An 8-by-4 transform block. + /// + Size8x4 = 6, + + /// + /// An 8-by-16 transform block. + /// + Size8x16 = 7, + + /// + /// A 16-by-8 transform block. + /// + Size16x8 = 8, + + /// + /// A 16-by-32 transform block. + /// + Size16x32 = 9, + + /// + /// A 32-by-16 transform block. + /// + Size32x16 = 10, + + /// + /// A 32-by-64 transform block. + /// + Size32x64 = 11, + + /// + /// A 64-by-32 transform block. + /// + Size64x32 = 12, + + /// + /// A 4-by-16 transform block. + /// + Size4x16 = 13, + + /// + /// A 16-by-4 transform block. + /// + Size16x4 = 14, + + /// + /// An 8-by-32 transform block. + /// + Size8x32 = 15, + + /// + /// A 32-by-8 transform block. + /// + Size32x8 = 16, + + /// + /// A 16-by-64 transform block. + /// + Size16x64 = 17, + + /// + /// A 64-by-16 transform block. + /// + Size64x16 = 18, + + /// + /// The number of defined transform-block sizes. + /// + AllSizes = 19, + + /// + /// The number of square transform-block sizes. + /// + SquareSizes = Size4x8, + + /// + /// No valid transform-block size. + /// + Invalid = 255, +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Transform/Av1TransformSizeExtensions.cs b/src/ImageSharp/Formats/Heif/Av1/Transform/Av1TransformSizeExtensions.cs new file mode 100644 index 0000000000..483322391f --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Transform/Av1TransformSizeExtensions.cs @@ -0,0 +1,317 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +/// +/// Provides dimensions, block mappings, scaling values, and subdivision rules for AV1 transform sizes. +/// +internal static class Av1TransformSizeExtensions +{ + /// + /// The coefficient count for each transform-size enum value. + /// + private static readonly int[] Size2d = [ + 16, 64, 256, 1024, 4096, 32, 32, 128, 128, 512, 512, 2048, 2048, 64, 64, 256, 256, 1024, 1024]; + + /// + /// The transform size produced by one level of subdivision for each transform-size enum value. + /// + private static readonly Av1TransformSize[] SubTransformSize = [ + Av1TransformSize.Size4x4, // TX_4X4 + Av1TransformSize.Size4x4, // TX_8X8 + Av1TransformSize.Size8x8, // TX_16X16 + Av1TransformSize.Size16x16, // TX_32X32 + Av1TransformSize.Size32x32, // TX_64X64 + Av1TransformSize.Size4x4, // TX_4X8 + Av1TransformSize.Size4x4, // TX_8X4 + Av1TransformSize.Size8x8, // TX_8X16 + Av1TransformSize.Size8x8, // TX_16X8 + Av1TransformSize.Size16x16, // TX_16X32 + Av1TransformSize.Size16x16, // TX_32X16 + Av1TransformSize.Size32x32, // TX_32X64 + Av1TransformSize.Size32x32, // TX_64X32 + Av1TransformSize.Size4x8, // TX_4X16 + Av1TransformSize.Size8x4, // TX_16X4 + Av1TransformSize.Size8x16, // TX_8X32 + Av1TransformSize.Size16x8, // TX_32X8 + Av1TransformSize.Size16x32, // TX_16X64 + Av1TransformSize.Size32x16, // TX_64X16 + ]; + + /// + /// Transform widths in units of four samples. + /// + private static readonly int[] WideUnit = [1, 2, 4, 8, 16, 1, 2, 2, 4, 4, 8, 8, 16, 1, 4, 2, 8, 4, 16]; + + /// + /// Transform heights in units of four samples. + /// + private static readonly int[] HighUnit = [1, 2, 4, 8, 16, 2, 1, 4, 2, 8, 4, 16, 8, 4, 1, 8, 2, 16, 4]; + + /// + /// Maps each transform size to the block size with matching dimensions. + /// + private static readonly Av1BlockSize[] BlockSize = [ + Av1BlockSize.Block4x4, // TX_4X4 + Av1BlockSize.Block8x8, // TX_8X8 + Av1BlockSize.Block16x16, // TX_16X16 + Av1BlockSize.Block32x32, // TX_32X32 + Av1BlockSize.Block64x64, // TX_64X64 + Av1BlockSize.Block4x8, // TX_4X8 + Av1BlockSize.Block8x4, // TX_8X4 + Av1BlockSize.Block8x16, // TX_8X16 + Av1BlockSize.Block16x8, // TX_16X8 + Av1BlockSize.Block16x32, // TX_16X32 + Av1BlockSize.Block32x16, // TX_32X16 + Av1BlockSize.Block32x64, // TX_32X64 + Av1BlockSize.Block64x32, // TX_64X32 + Av1BlockSize.Block4x16, // TX_4X16 + Av1BlockSize.Block16x4, // TX_16X4 + Av1BlockSize.Block8x32, // TX_8X32 + Av1BlockSize.Block32x8, // TX_32X8 + Av1BlockSize.Block16x64, // TX_16X64 + Av1BlockSize.Block64x16, // TX_64X16 + ]; + + /// + /// Maps each transform size to the square transform based on its smaller dimension. + /// + private static readonly Av1TransformSize[] SquareMap = [ + Av1TransformSize.Size4x4, // TX_4X4 + Av1TransformSize.Size8x8, // TX_8X8 + Av1TransformSize.Size16x16, // TX_16X16 + Av1TransformSize.Size32x32, // TX_32X32 + Av1TransformSize.Size64x64, // TX_64X64 + Av1TransformSize.Size4x4, // TX_4X8 + Av1TransformSize.Size4x4, // TX_8X4 + Av1TransformSize.Size8x8, // TX_8X16 + Av1TransformSize.Size8x8, // TX_16X8 + Av1TransformSize.Size16x16, // TX_16X32 + Av1TransformSize.Size16x16, // TX_32X16 + Av1TransformSize.Size32x32, // TX_32X64 + Av1TransformSize.Size32x32, // TX_64X32 + Av1TransformSize.Size4x4, // TX_4X16 + Av1TransformSize.Size4x4, // TX_16X4 + Av1TransformSize.Size8x8, // TX_8X32 + Av1TransformSize.Size8x8, // TX_32X8 + Av1TransformSize.Size16x16, // TX_16X64 + Av1TransformSize.Size16x16, // TX_64X16 + ]; + + /// + /// Maps each transform size to the square transform based on its larger dimension. + /// + private static readonly Av1TransformSize[] SquareUpMap = [ + Av1TransformSize.Size4x4, // TX_4X4 + Av1TransformSize.Size8x8, // TX_8X8 + Av1TransformSize.Size16x16, // TX_16X16 + Av1TransformSize.Size32x32, // TX_32X32 + Av1TransformSize.Size64x64, // TX_64X64 + Av1TransformSize.Size8x8, // TX_4X8 + Av1TransformSize.Size8x8, // TX_8X4 + Av1TransformSize.Size16x16, // TX_8X16 + Av1TransformSize.Size16x16, // TX_16X8 + Av1TransformSize.Size32x32, // TX_16X32 + Av1TransformSize.Size32x32, // TX_32X16 + Av1TransformSize.Size64x64, // TX_32X64 + Av1TransformSize.Size64x64, // TX_64X32 + Av1TransformSize.Size16x16, // TX_4X16 + Av1TransformSize.Size16x16, // TX_16X4 + Av1TransformSize.Size32x32, // TX_8X32 + Av1TransformSize.Size32x32, // TX_32X8 + Av1TransformSize.Size64x64, // TX_16X64 + Av1TransformSize.Size64x64, // TX_64X16 + ]; + + /// + /// Contains min(log2(width), 5) + min(log2(height), 5) - 4 for each transform size. + /// + private static readonly int[] Log2Minus4 = [ + 0, // TX_4X4 + 2, // TX_8X8 + 4, // TX_16X16 + 6, // TX_32X32 + 6, // TX_64X64 + 1, // TX_4X8 + 1, // TX_8X4 + 3, // TX_8X16 + 3, // TX_16X8 + 5, // TX_16X32 + 5, // TX_32X16 + 6, // TX_32X64 + 6, // TX_64X32 + 2, // TX_4X16 + 2, // TX_16X4 + 4, // TX_8X32 + 4, // TX_32X8 + 5, // TX_16X64 + 5, // TX_64X16 + ]; + + /// + /// Transform widths expressed as base-two logarithms of sample counts. + /// + private static readonly int[] BlockWidthLog2 = [ + 2, 3, 4, 5, 6, 2, 3, 3, 4, 4, 5, 5, 6, 2, 4, 3, 5, 4, 6, + ]; + + /// + /// Transform heights expressed as base-two logarithms of sample counts. + /// + private static readonly int[] BlockHeightLog2 = [ + 2, 3, 4, 5, 6, 3, 2, 4, 3, 5, 4, 6, 5, 4, 2, 5, 3, 6, 4, + ]; + + /// + /// Gets the number of coefficient positions in a transform block. + /// + /// The transform size. + /// The transform width multiplied by its height. + public static int GetSize2d(this Av1TransformSize size) => Size2d[(int)size]; + + /// + /// Gets the inverse-quantization scale category for a transform size. + /// + /// The transform size. + /// Zero for up to 256 coefficients, one for up to 1024, or two for larger transforms. + public static int GetScale(this Av1TransformSize size) + { + int pels = Size2d[(int)size]; + return (pels > 1024) ? 2 : (pels > 256) ? 1 : 0; + } + + /// + /// Gets the transform width in samples. + /// + /// The transform size. + /// The transform width in samples. + public static int GetWidth(this Av1TransformSize size) => WideUnit[(int)size] << 2; + + /// + /// Gets the transform height in samples. + /// + /// The transform size. + /// The transform height in samples. + public static int GetHeight(this Av1TransformSize size) => HighUnit[(int)size] << 2; + + /// + /// Gets the transform width in units of four samples. + /// + /// The transform size. + /// The number of four-sample columns. + public static int Get4x4WideCount(this Av1TransformSize size) => WideUnit[(int)size]; + + /// + /// Gets the transform height in units of four samples. + /// + /// The transform size. + /// The number of four-sample rows. + public static int Get4x4HighCount(this Av1TransformSize size) => HighUnit[(int)size]; + + /// + /// Gets the next smaller transform size used when a transform block is subdivided. + /// + /// The transform size. + /// The transform's subdivision size. + public static Av1TransformSize GetSubSize(this Av1TransformSize size) => SubTransformSize[(int)size]; + + /// + /// Gets the square transform based on the smaller dimension of a rectangular transform. + /// + /// The transform size. + /// The corresponding square transform size. + public static Av1TransformSize GetSquareSize(this Av1TransformSize size) => SquareMap[(int)size]; + + /// + /// Gets the square transform based on the larger dimension of a rectangular transform. + /// + /// The transform size. + /// The corresponding enclosing square transform size. + public static Av1TransformSize GetSquareUpSize(this Av1TransformSize size) => SquareUpMap[(int)size]; + + /// + /// Gets the block size having the same dimensions as a transform size. + /// + /// The transform size. + /// The dimensionally equivalent block size. + public static Av1BlockSize ToBlockSize(this Av1TransformSize transformSize) => BlockSize[(int)transformSize]; + + /// + /// Gets the capped sum of the transform-dimension logarithms minus four. + /// + /// The transform size. + /// The context value used by AV1 transform syntax. + public static int GetLog2Minus4(this Av1TransformSize size) => Log2Minus4[(int)size]; + + /// + /// Gets the transform size used by coefficient and quantization tables that cap dimensions at 32 samples. + /// + /// The signaled transform size. + /// The adjusted transform size. + public static Av1TransformSize GetAdjusted(this Av1TransformSize size) => size switch + { + Av1TransformSize.Size64x64 or Av1TransformSize.Size64x32 or Av1TransformSize.Size32x64 => Av1TransformSize.Size32x32, + Av1TransformSize.Size64x16 => Av1TransformSize.Size32x16, + Av1TransformSize.Size16x64 => Av1TransformSize.Size16x32, + _ => size + }; + + /// + /// Gets the base-two logarithm of the transform width in samples. + /// + /// The transform size. + /// The base-two width logarithm. + public static int GetBlockWidthLog2(this Av1TransformSize size) => BlockWidthLog2[(int)size]; + + /// + /// Gets the base-two logarithm of the transform height in samples. + /// + /// The transform size. + /// The base-two height logarithm. + public static int GetBlockHeightLog2(this Av1TransformSize size) => BlockHeightLog2[(int)size]; + + /// + /// Gets the signed base-two ratio between transform width and height. + /// + /// The transform size. + /// Zero for square transforms, positive when wider, or negative when taller. + public static int GetRectangleLogRatio(this Av1TransformSize size) + { + int col = GetWidth(size); + int row = GetHeight(size); + if (col == row) + { + return 0; + } + + if (col > row) + { + if (col == row * 2) + { + return 1; + } + + if (col == row * 4) + { + return 2; + } + + throw new InvalidImageContentException("Unsupported transform size"); + } + else + { + if (row == col * 2) + { + return -1; + } + + if (row == col * 4) + { + return -2; + } + + throw new InvalidImageContentException("Unsupported transform size"); + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Transform/Av1TransformType.cs b/src/ImageSharp/Formats/Heif/Av1/Transform/Av1TransformType.cs new file mode 100644 index 0000000000..4d52b464b0 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Transform/Av1TransformType.cs @@ -0,0 +1,100 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +/// +/// Identifies the horizontal and vertical transform combination signaled for an AV1 transform block. +/// +internal enum Av1TransformType : byte +{ + /// + /// DCT in both horizontal and vertical. + /// + DctDct, + + /// + /// ADST in vertical, DCT in horizontal. + /// + AdstDct, + + /// + /// DCT in vertical, ADST in horizontal. + /// + DctAdst, + + /// + /// ADST in both directions. + /// + AdstAdst, + + /// + /// Flipped ADST vertically and DCT horizontally. + /// + FlipAdstDct, + + /// + /// DCT vertically and flipped ADST horizontally. + /// + DctFlipAdst, + + /// + /// Flipped ADST in both directions. + /// + FlipAdstFlipAdst, + + /// + /// ADST vertically and flipped ADST horizontally. + /// + AdstFlipAdst, + + /// + /// Flipped ADST vertically and ADST horizontally. + /// + FlipAdstAdst, + + /// + /// Identity transforms in both directions. + /// + Identity, + + /// + /// DCT vertically and identity horizontally. + /// + VerticalDct, + + /// + /// Identity vertically and DCT horizontally. + /// + HorizontalDct, + + /// + /// ADST vertically and identity horizontally. + /// + VerticalAdst, + + /// + /// Identity vertically and ADST horizontally. + /// + HorizontalAdst, + + /// + /// Flipped ADST vertically and identity horizontally. + /// + VerticalFlipAdst, + + /// + /// Identity vertically and flipped ADST horizontally. + /// + HorizontalFlipAdst, + + /// + /// Number of Transform types. + /// + AllTransformTypes, + + /// + /// Invalid value. + /// + Invalid, +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Transform/Av1TransformType1d.cs b/src/ImageSharp/Formats/Heif/Av1/Transform/Av1TransformType1d.cs new file mode 100644 index 0000000000..16f80d9436 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Transform/Av1TransformType1d.cs @@ -0,0 +1,30 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +/// +/// Identifies the one-dimensional transform applied along one axis of an AV1 transform block. +/// +internal enum Av1TransformType1d +{ + /// + /// A discrete cosine transform. + /// + Dct, + + /// + /// An asymmetric discrete sine transform. + /// + Adst, + + /// + /// A flipped asymmetric discrete sine transform. + /// + FlipAdst, + + /// + /// An identity transform. + /// + Identity +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Transform/Av1TransformTypeExtensions.cs b/src/ImageSharp/Formats/Heif/Av1/Transform/Av1TransformTypeExtensions.cs new file mode 100644 index 0000000000..d073393a52 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Transform/Av1TransformTypeExtensions.cs @@ -0,0 +1,60 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +/// +/// Provides AV1 transform-class and transform-set lookups for compound transform types. +/// +internal static class Av1TransformTypeExtensions +{ + /// + /// Maps each compound transform type to its entropy-context transform class. + /// + private static readonly Av1TransformClass[] Type2Class = [ + Av1TransformClass.Class2D, // DCT_DCT + Av1TransformClass.Class2D, // ADST_DCT + Av1TransformClass.Class2D, // DCT_ADST + Av1TransformClass.Class2D, // ADST_ADST + Av1TransformClass.Class2D, // FLIPADST_DCT + Av1TransformClass.Class2D, // DCT_FLIPADST + Av1TransformClass.Class2D, // FLIPADST_FLIPADST + Av1TransformClass.Class2D, // ADST_FLIPADST + Av1TransformClass.Class2D, // FLIPADST_ADST + Av1TransformClass.Class2D, // IDTX + Av1TransformClass.ClassVertical, // V_DCT + Av1TransformClass.ClassHorizontal, // H_DCT + Av1TransformClass.ClassVertical, // V_ADST + Av1TransformClass.ClassHorizontal, // H_ADST + Av1TransformClass.ClassVertical, // V_FLIPADST + Av1TransformClass.ClassHorizontal, // H_FLIPADST + ]; + + /// + /// Indicates which compound transform types are enabled by each transform-set type. + /// + private static readonly bool[][] ExtendedTransformUsed = [ + [true, false, false, false, false, false, false, false, false, false, false, false, false, false, false, false], + [true, false, false, false, false, false, false, false, false, true, false, false, false, false, false, false], + [true, true, true, true, false, false, false, false, false, true, false, false, false, false, false, false], + [true, true, true, true, false, false, false, false, false, true, true, true, false, false, false, false], + [true, true, true, true, true, true, true, true, true, true, true, true, false, false, false, false], + [true, true, true, true, true, true, true, true, true, true, true, true, true, true, true, true], + ]; + + /// + /// Gets the entropy-context class of a compound transform type. + /// + /// The compound transform type. + /// The two-dimensional, horizontal, or vertical transform class. + public static Av1TransformClass ToClass(this Av1TransformType transformType) => Type2Class[(int)transformType]; + + /// + /// Determines whether a compound transform type belongs to an allowed transform set. + /// + /// The compound transform type. + /// The allowed transform set. + /// when the transform is enabled by the set. + public static bool IsExtendedSetUsed(this Av1TransformType transformType, Av1TransformSetType setType) + => ExtendedTransformUsed[(int)setType][(int)transformType]; +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Transform/Av1TransformVector.cs b/src/ImageSharp/Formats/Heif/Av1/Transform/Av1TransformVector.cs new file mode 100644 index 0000000000..3bc8198a30 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Transform/Av1TransformVector.cs @@ -0,0 +1,106 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Diagnostics.CodeAnalysis; +using System.Runtime.CompilerServices; +using System.Runtime.InteropServices; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +/// +/// Stores the fixed set of SIMD values used by one bulk AV1 transform axis. +/// +/// The SIMD vector type used for parallel transform lanes. +/// +/// Field Vn stores transform position n for every independent axis in the vector lanes. A +/// therefore carries four 32-bit axes, while a +/// carries eight. Transform stages operate vertically through +/// these fields and never mix lanes, so their scalar fixed-point rounding and clamping rules remain unchanged. +/// +[StructLayout(LayoutKind.Sequential)] +internal struct Av1TransformVector + where TVector : struct +{ + // Explicit fields make each transform position a constant field offset. An indexed inline-array accessor was not + // expanded inside the larger stage networks, which introduced a helper call for every coefficient access. + public TVector V0; + public TVector V1; + public TVector V2; + public TVector V3; + public TVector V4; + public TVector V5; + public TVector V6; + public TVector V7; + public TVector V8; + public TVector V9; + public TVector V10; + public TVector V11; + public TVector V12; + public TVector V13; + public TVector V14; + public TVector V15; + public TVector V16; + public TVector V17; + public TVector V18; + public TVector V19; + public TVector V20; + public TVector V21; + public TVector V22; + public TVector V23; + public TVector V24; + public TVector V25; + public TVector V26; + public TVector V27; + public TVector V28; + public TVector V29; + public TVector V30; + public TVector V31; + public TVector V32; + public TVector V33; + public TVector V34; + public TVector V35; + public TVector V36; + public TVector V37; + public TVector V38; + public TVector V39; + public TVector V40; + public TVector V41; + public TVector V42; + public TVector V43; + public TVector V44; + public TVector V45; + public TVector V46; + public TVector V47; + public TVector V48; + public TVector V49; + public TVector V50; + public TVector V51; + public TVector V52; + public TVector V53; + public TVector V54; + public TVector V55; + public TVector V56; + public TVector V57; + public TVector V58; + public TVector V59; + public TVector V60; + public TVector V61; + public TVector V62; + public TVector V63; + + /// + /// Gets a reference to the SIMD value at the requested transform position. + /// + /// The zero-based transform position. + /// The SIMD value at the requested position. + [UnscopedRef] + public ref TVector this[int index] + { + [MethodImpl(MethodImplOptions.AggressiveInlining)] + get + { + ref TVector first = ref Unsafe.As, TVector>(ref this); + return ref Unsafe.Add(ref first, (uint)index); + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Transform/Av1TransformWorkspace.cs b/src/ImageSharp/Formats/Heif/Av1/Transform/Av1TransformWorkspace.cs new file mode 100644 index 0000000000..76e7033420 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Transform/Av1TransformWorkspace.cs @@ -0,0 +1,62 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +/// +/// Describes the reusable storage required by an AV1 two-dimensional transform operation. +/// +/// +/// The workspace reserves three maximum-length transform vectors for input, output, and stage exchange, followed by +/// full-block raster buffers and the fixed AVX-512 transpose area. All storage is expressed as integers so callers can +/// rent one buffer and reinterpret aligned prefixes for 16-bit or 32-bit SIMD lanes without per-block allocations. +/// +internal static class Av1TransformWorkspace +{ + /// + /// The number of integer elements occupied by the three 512-bit transform vectors. + /// + public const int Vector512StorageLength = 3 * Av1Constants.MaxTransformSize * 16; + + /// + /// The number of integer elements occupied by the AVX-512 transpose scratch. + /// + public const int Vector512TransposeStorageLength = 16 * 8 * 2; + + /// + /// The number of integer elements occupied by the three 256-bit transform vectors. + /// + public const int Vector256StorageLength = 3 * Av1Constants.MaxTransformSize * 8; + + /// + /// The number of integer elements occupied by the three 128-bit transform vectors. + /// + public const int Vector128StorageLength = 3 * Av1Constants.MaxTransformSize * 4; + + /// + /// The number of integers required for the largest supported transform block. + /// + public const int MaximumLength = + (2 * Av1Constants.MaxTransformSize * Av1Constants.MaxTransformSize) + + Vector512StorageLength + + Vector512TransposeStorageLength; + + /// + /// Gets the number of integers required for a transform size. + /// + /// The transform-block dimensions. + /// The required workspace length. + public static int GetRequiredLength(Av1TransformSize transformSize) + { + // The widest packed transform evaluates thirty-two independent axes together. Small AV1 blocks are padded + // to that lane count, so their workspace requirement is determined by the vector tile rather than the + // coded coefficient count. + int width = Math.Max(transformSize.GetWidth(), Vector512.Count); + int height = Math.Max(transformSize.GetHeight(), Vector512.Count); + int blockLength = width * height; + + return (2 * blockLength) + Vector512StorageLength + Vector512TransposeStorageLength; + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Transform/Forward/Av1ForwardTransformArithmetic.cs b/src/ImageSharp/Formats/Heif/Av1/Transform/Forward/Av1ForwardTransformArithmetic.cs new file mode 100644 index 0000000000..ed4a120b8c --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Transform/Forward/Av1ForwardTransformArithmetic.cs @@ -0,0 +1,864 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.Intrinsics; +using System.Runtime.Intrinsics.X86; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Transform.Forward; + +/// +/// Provides the sample-type and vector-width arithmetic used by the shared AV1 forward-transform stage networks. +/// +/// The scalar or SIMD value containing independent transform axes. +/// +/// Each closed is either one scalar axis or a vector of independent axes. The +/// branches are resolved when the generic type is compiled, so they select arithmetic once +/// without adding per-stage runtime dispatch. Signed 16-bit representations use the saturating operations required by +/// the packed transform pipeline; 32-bit representations retain the normative wrapping fixed-point arithmetic. +/// +internal static class Av1ForwardTransformArithmetic + where TValue : struct +{ + /// + /// Creates the rounding value used by fixed-point transform multiplications. + /// + /// The number of fractional bits in the transform constants. + /// The rounding value in the widened lane shape used by . + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Av1TransformRounding CreateRounding(int cosBit) + { + int value = 1 << (cosBit - 1); + Av1TransformRounding rounding = default; + + // The closed TValue makes this a compile-time shape selection. Only the matching explicit-layout field is + // initialized and subsequently read, keeping the broadcast outside every butterfly in the stage network. + if (typeof(TValue) == typeof(Vector128) || typeof(TValue) == typeof(Vector128)) + { + rounding.Vector128 = Vector128.Create(value); + } + else if (typeof(TValue) == typeof(Vector256) || typeof(TValue) == typeof(Vector256)) + { + rounding.Vector256 = Vector256.Create(value); + } + else if (typeof(TValue) == typeof(Vector512) || typeof(TValue) == typeof(Vector512)) + { + rounding.Vector512 = Vector512.Create(value); + } + else + { + rounding.Scalar = value; + } + + return rounding; + } + + /// + /// Adds two transform values using the lane arithmetic required by the selected sample type. + /// + /// The left operand. + /// The right operand. + /// The lane-wise sum. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static TValue Add(TValue left, TValue right) + { + if (typeof(TValue) == typeof(short)) + { + int resultValue = As(left) + As(right); + short result = (short)Math.Clamp(resultValue, short.MinValue, short.MaxValue); + return As(result); + } + + if (typeof(TValue) == typeof(int)) + { + int result = As(left) + As(right); + return As(result); + } + + if (typeof(TValue) == typeof(Vector128)) + { + Vector128 result = Vector128.AddSaturate(As>(left), As>(right)); + return As, TValue>(result); + } + + if (typeof(TValue) == typeof(Vector128)) + { + Vector128 result = As>(left) + As>(right); + return As, TValue>(result); + } + + if (typeof(TValue) == typeof(Vector256)) + { + Vector256 result = Vector256.AddSaturate(As>(left), As>(right)); + return As, TValue>(result); + } + + if (typeof(TValue) == typeof(Vector256)) + { + Vector256 result = As>(left) + As>(right); + return As, TValue>(result); + } + + if (typeof(TValue) == typeof(Vector512)) + { + Vector512 result = Vector512.AddSaturate(As>(left), As>(right)); + return As, TValue>(result); + } + + Vector512 vector = As>(left) + As>(right); + return As, TValue>(vector); + } + + /// + /// Subtracts one transform value from another using the lane arithmetic required by the selected sample type. + /// + /// The left operand. + /// The right operand. + /// The lane-wise difference. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static TValue Subtract(TValue left, TValue right) + { + if (typeof(TValue) == typeof(short)) + { + int resultValue = As(left) - As(right); + short result = (short)Math.Clamp(resultValue, short.MinValue, short.MaxValue); + return As(result); + } + + if (typeof(TValue) == typeof(int)) + { + int result = As(left) - As(right); + return As(result); + } + + if (typeof(TValue) == typeof(Vector128)) + { + Vector128 result = Vector128.SubtractSaturate(As>(left), As>(right)); + return As, TValue>(result); + } + + if (typeof(TValue) == typeof(Vector128)) + { + Vector128 result = As>(left) - As>(right); + return As, TValue>(result); + } + + if (typeof(TValue) == typeof(Vector256)) + { + Vector256 result = Vector256.SubtractSaturate(As>(left), As>(right)); + return As, TValue>(result); + } + + if (typeof(TValue) == typeof(Vector256)) + { + Vector256 result = As>(left) - As>(right); + return As, TValue>(result); + } + + if (typeof(TValue) == typeof(Vector512)) + { + Vector512 result = Vector512.SubtractSaturate(As>(left), As>(right)); + return As, TValue>(result); + } + + Vector512 vector = As>(left) - As>(right); + return As, TValue>(vector); + } + + /// + /// Negates a transform value using wrapping lane arithmetic. + /// + /// The value to negate. + /// The lane-wise negated value. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static TValue Negate(TValue value) + { + if (typeof(TValue) == typeof(short)) + { + short result = unchecked((short)-As(value)); + return As(result); + } + + if (typeof(TValue) == typeof(int)) + { + int result = -As(value); + return As(result); + } + + if (typeof(TValue) == typeof(Vector128)) + { + Vector128 result = Vector128.Zero - As>(value); + return As, TValue>(result); + } + + if (typeof(TValue) == typeof(Vector128)) + { + Vector128 result = -As>(value); + return As, TValue>(result); + } + + if (typeof(TValue) == typeof(Vector256)) + { + Vector256 result = Vector256.Zero - As>(value); + return As, TValue>(result); + } + + if (typeof(TValue) == typeof(Vector256)) + { + Vector256 result = -As>(value); + return As, TValue>(result); + } + + if (typeof(TValue) == typeof(Vector512)) + { + Vector512 result = Vector512.Zero - As>(value); + return As, TValue>(result); + } + + Vector512 vector = -As>(value); + return As, TValue>(vector); + } + + /// + /// Adds and subtracts two transform values, saturating only the signed sixteen-bit representations. + /// + /// The left operand. + /// The right operand. + /// The lane-wise sum. + /// The lane-wise difference. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void AddSubtract(TValue left, TValue right, out TValue sum, out TValue difference) + { + if (typeof(TValue) == typeof(short)) + { + int leftValue = As(left); + int rightValue = As(right); + short sumValue = (short)Math.Clamp(leftValue + rightValue, short.MinValue, short.MaxValue); + short differenceValue = (short)Math.Clamp(leftValue - rightValue, short.MinValue, short.MaxValue); + + sum = As(sumValue); + difference = As(differenceValue); + return; + } + + if (typeof(TValue) == typeof(int)) + { + int leftValue = As(left); + int rightValue = As(right); + int sumValue = leftValue + rightValue; + int differenceValue = leftValue - rightValue; + + sum = As(sumValue); + difference = As(differenceValue); + return; + } + + if (typeof(TValue) == typeof(Vector128)) + { + Vector128 leftValue = As>(left); + Vector128 rightValue = As>(right); + Vector128 sumValue = Vector128.AddSaturate(leftValue, rightValue); + Vector128 differenceValue = Vector128.SubtractSaturate(leftValue, rightValue); + + sum = As, TValue>(sumValue); + difference = As, TValue>(differenceValue); + return; + } + + if (typeof(TValue) == typeof(Vector128)) + { + Vector128 leftValue = As>(left); + Vector128 rightValue = As>(right); + + sum = As, TValue>(leftValue + rightValue); + difference = As, TValue>(leftValue - rightValue); + return; + } + + if (typeof(TValue) == typeof(Vector256)) + { + Vector256 leftValue = As>(left); + Vector256 rightValue = As>(right); + Vector256 sumValue = Vector256.AddSaturate(leftValue, rightValue); + Vector256 differenceValue = Vector256.SubtractSaturate(leftValue, rightValue); + + sum = As, TValue>(sumValue); + difference = As, TValue>(differenceValue); + return; + } + + if (typeof(TValue) == typeof(Vector256)) + { + Vector256 leftValue = As>(left); + Vector256 rightValue = As>(right); + + sum = As, TValue>(leftValue + rightValue); + difference = As, TValue>(leftValue - rightValue); + return; + } + + if (typeof(TValue) == typeof(Vector512)) + { + Vector512 leftValue = As>(left); + Vector512 rightValue = As>(right); + Vector512 sumValue = Vector512.AddSaturate(leftValue, rightValue); + Vector512 differenceValue = Vector512.SubtractSaturate(leftValue, rightValue); + + sum = As, TValue>(sumValue); + difference = As, TValue>(differenceValue); + return; + } + + Vector512 leftVector = As>(left); + Vector512 rightVector = As>(right); + + sum = As, TValue>(leftVector + rightVector); + difference = As, TValue>(leftVector - rightVector); + } + + /// + /// Calculates both outputs of a rounded, weighted two-input butterfly. + /// + /// The first fixed-point weight. + /// The second fixed-point weight. + /// The first transform value. + /// The second transform value. + /// The first rounded result. + /// The second rounded result. + /// The number of fractional bits in each weight. + /// The rounding value created for this transform. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void Butterfly( + int weight0, + int weight1, + TValue input0, + TValue input1, + out TValue output0, + out TValue output1, + int cosBit, + in Av1TransformRounding rounding) + { + if (typeof(TValue) == typeof(Vector128)) + { + Vector128 left = As>(input0); + Vector128 right = As>(input1); + + Av1Transform1dMath.Butterfly(weight0, weight1, in left, in right, out Vector128 result0, out Vector128 result1, cosBit, in rounding.Vector128); + output0 = As, TValue>(result0); + output1 = As, TValue>(result1); + return; + } + + if (typeof(TValue) == typeof(Vector256)) + { + Vector256 left = As>(input0); + Vector256 right = As>(input1); + + Av1Transform1dMath.Butterfly(weight0, weight1, in left, in right, out Vector256 result0, out Vector256 result1, cosBit, in rounding.Vector256); + output0 = As, TValue>(result0); + output1 = As, TValue>(result1); + return; + } + + if (typeof(TValue) == typeof(Vector512)) + { + Vector512 left = As>(input0); + Vector512 right = As>(input1); + + Av1Transform1dMath.Butterfly(weight0, weight1, in left, in right, out Vector512 result0, out Vector512 result1, cosBit, in rounding.Vector512); + output0 = As, TValue>(result0); + output1 = As, TValue>(result1); + return; + } + + output0 = HalfButterfly(weight0, input0, weight1, input1, cosBit, in rounding); + output1 = HalfButterfly(weight1, input0, -weight0, input1, cosBit, in rounding); + } + + /// + /// Calculates one output of a rounded, weighted two-input butterfly. + /// + /// The first fixed-point weight. + /// The first transform value. + /// The second fixed-point weight. + /// The second transform value. + /// The number of fractional bits in each weight. + /// The rounding value created for this transform. + /// The rounded lane-wise weighted sum. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static TValue HalfButterfly( + int weight0, + TValue input0, + int weight1, + TValue input1, + int cosBit, + in Av1TransformRounding rounding) + { + if (typeof(TValue) == typeof(short)) + { + int weighted = (weight0 * As(input0)) + (weight1 * As(input1)); + short result = (short)Math.Clamp((weighted + rounding.Scalar) >> cosBit, short.MinValue, short.MaxValue); + return As(result); + } + + if (typeof(TValue) == typeof(int)) + { + int weighted = (weight0 * As(input0)) + (weight1 * As(input1)); + int result = (weighted + rounding.Scalar) >> cosBit; + return As(result); + } + + if (typeof(TValue) == typeof(Vector128)) + { + Vector128 result = MultiplyRound( + As>(input0), + weight0, + As>(input1), + weight1, + cosBit, + rounding.Vector128); + + return As, TValue>(result); + } + + if (typeof(TValue) == typeof(Vector128)) + { + Vector128 result = ((As>(input0) * Vector128.Create(weight0)) + + (As>(input1) * Vector128.Create(weight1)) + + rounding.Vector128) >> cosBit; + + return As, TValue>(result); + } + + if (typeof(TValue) == typeof(Vector256)) + { + Vector256 result = MultiplyRound( + As>(input0), + weight0, + As>(input1), + weight1, + cosBit, + rounding.Vector256); + + return As, TValue>(result); + } + + if (typeof(TValue) == typeof(Vector256)) + { + Vector256 result = ((As>(input0) * Vector256.Create(weight0)) + + (As>(input1) * Vector256.Create(weight1)) + + rounding.Vector256) >> cosBit; + + return As, TValue>(result); + } + + if (typeof(TValue) == typeof(Vector512)) + { + Vector512 result = MultiplyRound( + As>(input0), + weight0, + As>(input1), + weight1, + cosBit, + rounding.Vector512); + + return As, TValue>(result); + } + + Vector512 vector = ((As>(input0) * Vector512.Create(weight0)) + + (As>(input1) * Vector512.Create(weight1)) + + rounding.Vector512) >> cosBit; + + return As, TValue>(vector); + } + + /// + /// Multiplies a transform value by a fixed-point constant and applies the requested rounding shift. + /// + /// The transform value. + /// The fixed-point multiplier. + /// The number of fractional bits in the multiplier. + /// The rounded lane-wise product. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static TValue MultiplyRound(TValue value, int multiplier, int shift) + { + Av1TransformRounding rounding = CreateRounding(shift); + return HalfButterfly(multiplier, value, 0, default, shift, in rounding); + } + + /// + /// Shifts each transform lane left without saturation. + /// + /// The transform value. + /// The shift count. + /// The shifted lane values. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static TValue ShiftLeft(TValue value, int count) + { + if (typeof(TValue) == typeof(short)) + { + short result = unchecked((short)(As(value) << count)); + return As(result); + } + + if (typeof(TValue) == typeof(int)) + { + int result = As(value) << count; + return As(result); + } + + if (typeof(TValue) == typeof(Vector128)) + { + Vector128 result = As>(value) << count; + return As, TValue>(result); + } + + if (typeof(TValue) == typeof(Vector128)) + { + Vector128 result = As>(value) << count; + return As, TValue>(result); + } + + if (typeof(TValue) == typeof(Vector256)) + { + Vector256 result = As>(value) << count; + return As, TValue>(result); + } + + if (typeof(TValue) == typeof(Vector256)) + { + Vector256 result = As>(value) << count; + return As, TValue>(result); + } + + if (typeof(TValue) == typeof(Vector512)) + { + Vector512 result = As>(value) << count; + return As, TValue>(result); + } + + Vector512 vector = As>(value) << count; + return As, TValue>(vector); + } + + /// + /// Calculates one rounded sum containing four independently weighted transform values. + /// + /// The first fixed-point weight. + /// The first transform value. + /// The second fixed-point weight. + /// The second transform value. + /// The third fixed-point weight. + /// The third transform value. + /// The fourth fixed-point weight. + /// The fourth transform value. + /// The number of fractional bits in each weight. + /// The rounding value created for this transform. + /// The rounded lane-wise weighted sum. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static TValue MultiplyAddRound( + int weight0, + TValue input0, + int weight1, + TValue input1, + int weight2, + TValue input2, + int weight3, + TValue input3, + int cosBit, + in Av1TransformRounding rounding) + { + if (typeof(TValue) == typeof(short)) + { + int weighted = (weight0 * As(input0)) + + (weight1 * As(input1)) + + (weight2 * As(input2)) + + (weight3 * As(input3)); + + short result = (short)Math.Clamp((weighted + rounding.Scalar) >> cosBit, short.MinValue, short.MaxValue); + return As(result); + } + + if (typeof(TValue) == typeof(int)) + { + int weighted = (weight0 * As(input0)) + + (weight1 * As(input1)) + + (weight2 * As(input2)) + + (weight3 * As(input3)); + + int result = (weighted + rounding.Scalar) >> cosBit; + return As(result); + } + + if (typeof(TValue) == typeof(Vector128)) + { + Vector128 result = MultiplyRound( + As>(input0), + weight0, + As>(input1), + weight1, + As>(input2), + weight2, + As>(input3), + weight3, + cosBit, + rounding.Vector128); + + return As, TValue>(result); + } + + if (typeof(TValue) == typeof(Vector128)) + { + Vector128 result = ((As>(input0) * Vector128.Create(weight0)) + + (As>(input1) * Vector128.Create(weight1)) + + (As>(input2) * Vector128.Create(weight2)) + + (As>(input3) * Vector128.Create(weight3)) + + rounding.Vector128) >> cosBit; + + return As, TValue>(result); + } + + if (typeof(TValue) == typeof(Vector256)) + { + Vector256 result = MultiplyRound( + As>(input0), + weight0, + As>(input1), + weight1, + As>(input2), + weight2, + As>(input3), + weight3, + cosBit, + rounding.Vector256); + + return As, TValue>(result); + } + + if (typeof(TValue) == typeof(Vector256)) + { + Vector256 result = ((As>(input0) * Vector256.Create(weight0)) + + (As>(input1) * Vector256.Create(weight1)) + + (As>(input2) * Vector256.Create(weight2)) + + (As>(input3) * Vector256.Create(weight3)) + + rounding.Vector256) >> cosBit; + + return As, TValue>(result); + } + + if (typeof(TValue) == typeof(Vector512)) + { + Vector512 result = MultiplyRound( + As>(input0), + weight0, + As>(input1), + weight1, + As>(input2), + weight2, + As>(input3), + weight3, + cosBit, + rounding.Vector512); + + return As, TValue>(result); + } + + Vector512 vector = ((As>(input0) * Vector512.Create(weight0)) + + (As>(input1) * Vector512.Create(weight1)) + + (As>(input2) * Vector512.Create(weight2)) + + (As>(input3) * Vector512.Create(weight3)) + + rounding.Vector512) >> cosBit; + + return As, TValue>(vector); + } + + /// + /// Converts one value type to another equal-sized value type without changing its bits. + /// + /// The source value type. + /// The destination value type. + /// The value to reinterpret. + /// The reinterpreted value. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static TTo As(TFrom value) + where TFrom : struct + where TTo : struct + => Unsafe.As(ref value); + + /// + /// Calculates and narrows two weighted 128-bit signed sixteen-bit vectors. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector128 MultiplyRound( + Vector128 input0, + int weight0, + Vector128 input1, + int weight1, + int cosBit, + Vector128 rounding) + { + // Widening preserves lane order on every Vector128 implementation. The explicit clamp gives Narrow the + // signed-saturating demotion semantics used by Highway on x86, Arm, and WebAssembly. + (Vector128 input0Lower, Vector128 input0Upper) = Vector128.Widen(input0); + (Vector128 input1Lower, Vector128 input1Upper) = Vector128.Widen(input1); + + Vector128 weight0Vector = Vector128.Create(weight0); + Vector128 weight1Vector = Vector128.Create(weight1); + Vector128 lower = ((input0Lower * weight0Vector) + (input1Lower * weight1Vector) + rounding) >> cosBit; + Vector128 upper = ((input0Upper * weight0Vector) + (input1Upper * weight1Vector) + rounding) >> cosBit; + Vector128 minimum = Vector128.Create((int)short.MinValue); + Vector128 maximum = Vector128.Create((int)short.MaxValue); + + lower = Vector128.Clamp(lower, minimum, maximum); + upper = Vector128.Clamp(upper, minimum, maximum); + return Vector128.Narrow(lower, upper); + } + + /// + /// Calculates and narrows two weighted 256-bit signed sixteen-bit vectors. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector256 MultiplyRound( + Vector256 input0, + int weight0, + Vector256 input1, + int weight1, + int cosBit, + Vector256 rounding) + { + // The AVX2 path mirrors Highway's WidenMulPairwiseAdd primitive: adjacent Int16 products become Int32 + // sums, then VPACKSSDW restores the original lane width with signed saturation. + Vector256 lowerInputs = Avx2.UnpackLow(input0, input1); + Vector256 upperInputs = Avx2.UnpackHigh(input0, input1); + Vector256 weights = Avx2.UnpackLow(Vector256.Create((short)weight0), Vector256.Create((short)weight1)); + Vector256 lower = (Avx2.MultiplyAddAdjacent(lowerInputs, weights) + rounding) >> cosBit; + Vector256 upper = (Avx2.MultiplyAddAdjacent(upperInputs, weights) + rounding) >> cosBit; + return Avx2.PackSignedSaturate(lower, upper); + } + + /// + /// Calculates and narrows two weighted 512-bit signed sixteen-bit vectors. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector512 MultiplyRound( + Vector512 input0, + int weight0, + Vector512 input1, + int weight1, + int cosBit, + Vector512 rounding) + { + // Unpacking forms adjacent (input0, input1) pairs independently inside each 128-bit lane. Pairwise multiply-add + // widens those pairs to Int32 for rounding, and the final pack restores original lane order with saturation. + Vector512 lowerInputs = Avx512BW.UnpackLow(input0, input1); + Vector512 upperInputs = Avx512BW.UnpackHigh(input0, input1); + Vector512 weights = Avx512BW.UnpackLow(Vector512.Create((short)weight0), Vector512.Create((short)weight1)); + Vector512 lower = (Avx512BW.MultiplyAddAdjacent(lowerInputs, weights) + rounding) >> cosBit; + Vector512 upper = (Avx512BW.MultiplyAddAdjacent(upperInputs, weights) + rounding) >> cosBit; + return Avx512BW.PackSignedSaturate(lower, upper); + } + + /// + /// Calculates and narrows four weighted 128-bit signed sixteen-bit vectors. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector128 MultiplyRound( + Vector128 input0, + int weight0, + Vector128 input1, + int weight1, + Vector128 input2, + int weight2, + Vector128 input3, + int weight3, + int cosBit, + Vector128 rounding) + { + // Four products can exceed Int16 even though the completed stage value cannot. Widening each input first keeps + // the full fixed-point sum until rounding; explicit clamping supplies the required saturating demotion. + (Vector128 input0Lower, Vector128 input0Upper) = Vector128.Widen(input0); + (Vector128 input1Lower, Vector128 input1Upper) = Vector128.Widen(input1); + (Vector128 input2Lower, Vector128 input2Upper) = Vector128.Widen(input2); + (Vector128 input3Lower, Vector128 input3Upper) = Vector128.Widen(input3); + + Vector128 lower = ((input0Lower * Vector128.Create(weight0)) + + (input1Lower * Vector128.Create(weight1)) + + (input2Lower * Vector128.Create(weight2)) + + (input3Lower * Vector128.Create(weight3)) + + rounding) >> cosBit; + + Vector128 upper = ((input0Upper * Vector128.Create(weight0)) + + (input1Upper * Vector128.Create(weight1)) + + (input2Upper * Vector128.Create(weight2)) + + (input3Upper * Vector128.Create(weight3)) + + rounding) >> cosBit; + + Vector128 minimum = Vector128.Create((int)short.MinValue); + Vector128 maximum = Vector128.Create((int)short.MaxValue); + lower = Vector128.Clamp(lower, minimum, maximum); + upper = Vector128.Clamp(upper, minimum, maximum); + return Vector128.Narrow(lower, upper); + } + + /// + /// Calculates and narrows four weighted 256-bit signed sixteen-bit vectors. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector256 MultiplyRound( + Vector256 input0, + int weight0, + Vector256 input1, + int weight1, + Vector256 input2, + int weight2, + Vector256 input3, + int weight3, + int cosBit, + Vector256 rounding) + { + // The two unpack streams contain alternating input pairs for the lower and upper lane groups. Adding the two + // pairwise products completes each four-term dot product before the rounded saturating pack restores Int16. + Vector256 weights01 = Avx2.UnpackLow(Vector256.Create((short)weight0), Vector256.Create((short)weight1)); + Vector256 weights23 = Avx2.UnpackLow(Vector256.Create((short)weight2), Vector256.Create((short)weight3)); + Vector256 lower = Avx2.MultiplyAddAdjacent(Avx2.UnpackLow(input0, input1), weights01) + + Avx2.MultiplyAddAdjacent(Avx2.UnpackLow(input2, input3), weights23); + + Vector256 upper = Avx2.MultiplyAddAdjacent(Avx2.UnpackHigh(input0, input1), weights01) + + Avx2.MultiplyAddAdjacent(Avx2.UnpackHigh(input2, input3), weights23); + + lower = (lower + rounding) >> cosBit; + upper = (upper + rounding) >> cosBit; + return Avx2.PackSignedSaturate(lower, upper); + } + + /// + /// Calculates and narrows four weighted 512-bit signed sixteen-bit vectors. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector512 MultiplyRound( + Vector512 input0, + int weight0, + Vector512 input1, + int weight1, + Vector512 input2, + int weight2, + Vector512 input3, + int weight3, + int cosBit, + Vector512 rounding) + { + // AVX-512BW preserves the same lane-local pair layout as the 256-bit path. Two pairwise dot products form each + // four-term result in Int32, after which rounding and signed saturation return thirty-two independent axes. + Vector512 weights01 = Avx512BW.UnpackLow(Vector512.Create((short)weight0), Vector512.Create((short)weight1)); + Vector512 weights23 = Avx512BW.UnpackLow(Vector512.Create((short)weight2), Vector512.Create((short)weight3)); + Vector512 lower = Avx512BW.MultiplyAddAdjacent(Avx512BW.UnpackLow(input0, input1), weights01) + + Avx512BW.MultiplyAddAdjacent(Avx512BW.UnpackLow(input2, input3), weights23); + + Vector512 upper = Avx512BW.MultiplyAddAdjacent(Avx512BW.UnpackHigh(input0, input1), weights01) + + Avx512BW.MultiplyAddAdjacent(Avx512BW.UnpackHigh(input2, input3), weights23); + + lower = (lower + rounding) >> cosBit; + upper = (upper + rounding) >> cosBit; + return Avx512BW.PackSignedSaturate(lower, upper); + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Transform/Forward/Av1ForwardTransformer.Adst16Operator.cs b/src/ImageSharp/Formats/Heif/Av1/Transform/Forward/Av1ForwardTransformer.Adst16Operator.cs new file mode 100644 index 0000000000..fa18febc84 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Transform/Forward/Av1ForwardTransformer.Adst16Operator.cs @@ -0,0 +1,1077 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.Intrinsics; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform.Forward; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +/// +/// Defines the sixteen-point forward ADST operator. +/// +internal static partial class Av1ForwardTransformer +{ + /// + /// Implements the sixteen-point forward asymmetric discrete sine transform. + /// + internal readonly struct Adst16Operator : IAv1ForwardTransform1dOperator + { + /// + /// Gets the first cosine index for each final rotation. + /// + private static ReadOnlySpan FinalWeights => [2, 10, 18, 26, 34, 42, 50, 58]; + + /// + /// Gets the fixed coefficient permutation. + /// + private static ReadOnlySpan OutputOrder => [1, 14, 3, 12, 5, 10, 7, 8, 9, 6, 11, 4, 13, 2, 15, 0]; + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector buffer0, + ref Av1TransformVector buffer1, + int cosBit) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + Av1TransformRounding rounding = Av1ForwardTransformArithmetic.CreateRounding(cosBit); + + // Stage 1 is the bit-reversed ADST input order with the normative alternating signs. + buffer0[0] = Load(ref values, inputStride, 0); + buffer0[1] = Av1ForwardTransformArithmetic.Negate(Load(ref values, inputStride, 15)); + buffer0[2] = Av1ForwardTransformArithmetic.Negate(Load(ref values, inputStride, 7)); + buffer0[3] = Load(ref values, inputStride, 8); + buffer0[4] = Av1ForwardTransformArithmetic.Negate(Load(ref values, inputStride, 3)); + buffer0[5] = Load(ref values, inputStride, 12); + buffer0[6] = Load(ref values, inputStride, 4); + buffer0[7] = Av1ForwardTransformArithmetic.Negate(Load(ref values, inputStride, 11)); + buffer0[8] = Av1ForwardTransformArithmetic.Negate(Load(ref values, inputStride, 1)); + buffer0[9] = Load(ref values, inputStride, 14); + buffer0[10] = Load(ref values, inputStride, 6); + buffer0[11] = Av1ForwardTransformArithmetic.Negate(Load(ref values, inputStride, 9)); + buffer0[12] = Load(ref values, inputStride, 2); + buffer0[13] = Av1ForwardTransformArithmetic.Negate(Load(ref values, inputStride, 13)); + buffer0[14] = Av1ForwardTransformArithmetic.Negate(Load(ref values, inputStride, 5)); + buffer0[15] = Load(ref values, inputStride, 10); + + // Stage 2 rotates the second pair in each group of four while copying the first pair unchanged. + for (int group = 0; group < 16; group += 4) + { + buffer1[group] = buffer0[group]; + buffer1[group + 1] = buffer0[group + 1]; + Butterfly(cospi[32], cospi[32], buffer0[group + 2], buffer0[group + 3], ref buffer1, group + 2, group + 3, cosBit, in rounding); + } + + // Stage 3 combines adjacent pairs within each group of four. + for (int group = 0; group < 16; group += 4) + { + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic.AddSubtract( + buffer1[group + i], + buffer1[group + i + 2], + out buffer0[group + i], + out buffer0[group + i + 2]); + } + } + + // Stage 4 rotates the upper pair of each eight-value group by pi/8. + for (int group = 0; group < 16; group += 8) + { + for (int i = 0; i < 4; i++) + { + buffer1[group + i] = buffer0[group + i]; + } + + buffer1[group + 4] = Av1ForwardTransformArithmetic.HalfButterfly( + cospi[16], buffer0[group + 4], cospi[48], buffer0[group + 5], cosBit, in rounding); + + buffer1[group + 5] = Av1ForwardTransformArithmetic.HalfButterfly( + cospi[48], buffer0[group + 4], -cospi[16], buffer0[group + 5], cosBit, in rounding); + + buffer1[group + 6] = Av1ForwardTransformArithmetic.HalfButterfly( + -cospi[48], buffer0[group + 6], cospi[16], buffer0[group + 7], cosBit, in rounding); + + buffer1[group + 7] = Av1ForwardTransformArithmetic.HalfButterfly( + cospi[16], buffer0[group + 6], cospi[48], buffer0[group + 7], cosBit, in rounding); + } + + // Stage 5 combines the lower and upper quartets within each eight-value group. + for (int group = 0; group < 16; group += 8) + { + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic.AddSubtract( + buffer1[group + i], + buffer1[group + i + 4], + out buffer0[group + i], + out buffer0[group + i + 4]); + } + } + + // Stage 6 rotates the upper octet by pi/16 while retaining the completed lower octet. + for (int i = 0; i < 8; i++) + { + buffer1[i] = buffer0[i]; + } + + buffer1[8] = Av1ForwardTransformArithmetic.HalfButterfly(cospi[8], buffer0[8], cospi[56], buffer0[9], cosBit, in rounding); + buffer1[9] = Av1ForwardTransformArithmetic.HalfButterfly(cospi[56], buffer0[8], -cospi[8], buffer0[9], cosBit, in rounding); + buffer1[10] = Av1ForwardTransformArithmetic.HalfButterfly(cospi[40], buffer0[10], cospi[24], buffer0[11], cosBit, in rounding); + buffer1[11] = Av1ForwardTransformArithmetic.HalfButterfly(cospi[24], buffer0[10], -cospi[40], buffer0[11], cosBit, in rounding); + buffer1[12] = Av1ForwardTransformArithmetic.HalfButterfly(-cospi[56], buffer0[12], cospi[8], buffer0[13], cosBit, in rounding); + buffer1[13] = Av1ForwardTransformArithmetic.HalfButterfly(cospi[8], buffer0[12], cospi[56], buffer0[13], cosBit, in rounding); + buffer1[14] = Av1ForwardTransformArithmetic.HalfButterfly(-cospi[24], buffer0[14], cospi[40], buffer0[15], cosBit, in rounding); + buffer1[15] = Av1ForwardTransformArithmetic.HalfButterfly(cospi[40], buffer0[14], cospi[24], buffer0[15], cosBit, in rounding); + + // Stage 7 creates the eight final butterfly pairs spanning both octets. + for (int i = 0; i < 8; i++) + { + Av1ForwardTransformArithmetic.AddSubtract(buffer1[i], buffer1[i + 8], out buffer0[i], out buffer0[i + 8]); + } + + ReadOnlySpan finalWeights = FinalWeights; + + // Stage 8 applies the final odd-angle rotations. The compact weight table preserves their normative order + // without allocating a per-call array or duplicating the complementary cosine-index calculation. + for (int pair = 0; pair < 8; pair++) + { + int first = finalWeights[pair]; + int second = 64 - first; + int index = pair * 2; + buffer1[index] = Av1ForwardTransformArithmetic.HalfButterfly( + cospi[first], buffer0[index], cospi[second], buffer0[index + 1], cosBit, in rounding); + + buffer1[index + 1] = Av1ForwardTransformArithmetic.HalfButterfly( + cospi[second], buffer0[index], -cospi[first], buffer0[index + 1], cosBit, in rounding); + } + + ReadOnlySpan outputOrder = OutputOrder; + + // Stage 9 maps the rotated values to ascending AV1 ADST coefficient order. + for (int i = 0; i < 16; i++) + { + Store(ref values, outputStride, i, buffer1[outputOrder[i]]); + } + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector buffer0, + ref Av1TransformVector buffer1, + int cosBit) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + Av1TransformRounding rounding = Av1ForwardTransformArithmetic.CreateRounding(cosBit); + + // Stage 1 is the bit-reversed ADST input order with the normative alternating signs. + buffer0[0] = Load(ref values, inputStride, 0); + buffer0[1] = Av1ForwardTransformArithmetic.Negate(Load(ref values, inputStride, 15)); + buffer0[2] = Av1ForwardTransformArithmetic.Negate(Load(ref values, inputStride, 7)); + buffer0[3] = Load(ref values, inputStride, 8); + buffer0[4] = Av1ForwardTransformArithmetic.Negate(Load(ref values, inputStride, 3)); + buffer0[5] = Load(ref values, inputStride, 12); + buffer0[6] = Load(ref values, inputStride, 4); + buffer0[7] = Av1ForwardTransformArithmetic.Negate(Load(ref values, inputStride, 11)); + buffer0[8] = Av1ForwardTransformArithmetic.Negate(Load(ref values, inputStride, 1)); + buffer0[9] = Load(ref values, inputStride, 14); + buffer0[10] = Load(ref values, inputStride, 6); + buffer0[11] = Av1ForwardTransformArithmetic.Negate(Load(ref values, inputStride, 9)); + buffer0[12] = Load(ref values, inputStride, 2); + buffer0[13] = Av1ForwardTransformArithmetic.Negate(Load(ref values, inputStride, 13)); + buffer0[14] = Av1ForwardTransformArithmetic.Negate(Load(ref values, inputStride, 5)); + buffer0[15] = Load(ref values, inputStride, 10); + + // Stage 2 rotates the second pair in each group of four while copying the first pair unchanged. + for (int group = 0; group < 16; group += 4) + { + buffer1[group] = buffer0[group]; + buffer1[group + 1] = buffer0[group + 1]; + Butterfly(cospi[32], cospi[32], buffer0[group + 2], buffer0[group + 3], ref buffer1, group + 2, group + 3, cosBit, in rounding); + } + + // Stage 3 combines adjacent pairs within each group of four. + for (int group = 0; group < 16; group += 4) + { + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic.AddSubtract( + buffer1[group + i], + buffer1[group + i + 2], + out buffer0[group + i], + out buffer0[group + i + 2]); + } + } + + // Stage 4 rotates the upper pair of each eight-value group by pi/8. + for (int group = 0; group < 16; group += 8) + { + for (int i = 0; i < 4; i++) + { + buffer1[group + i] = buffer0[group + i]; + } + + buffer1[group + 4] = Av1ForwardTransformArithmetic.HalfButterfly( + cospi[16], buffer0[group + 4], cospi[48], buffer0[group + 5], cosBit, in rounding); + + buffer1[group + 5] = Av1ForwardTransformArithmetic.HalfButterfly( + cospi[48], buffer0[group + 4], -cospi[16], buffer0[group + 5], cosBit, in rounding); + + buffer1[group + 6] = Av1ForwardTransformArithmetic.HalfButterfly( + -cospi[48], buffer0[group + 6], cospi[16], buffer0[group + 7], cosBit, in rounding); + + buffer1[group + 7] = Av1ForwardTransformArithmetic.HalfButterfly( + cospi[16], buffer0[group + 6], cospi[48], buffer0[group + 7], cosBit, in rounding); + } + + // Stage 5 combines the lower and upper quartets within each eight-value group. + for (int group = 0; group < 16; group += 8) + { + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic.AddSubtract( + buffer1[group + i], + buffer1[group + i + 4], + out buffer0[group + i], + out buffer0[group + i + 4]); + } + } + + // Stage 6 rotates the upper octet by pi/16 while retaining the completed lower octet. + for (int i = 0; i < 8; i++) + { + buffer1[i] = buffer0[i]; + } + + buffer1[8] = Av1ForwardTransformArithmetic.HalfButterfly(cospi[8], buffer0[8], cospi[56], buffer0[9], cosBit, in rounding); + buffer1[9] = Av1ForwardTransformArithmetic.HalfButterfly(cospi[56], buffer0[8], -cospi[8], buffer0[9], cosBit, in rounding); + buffer1[10] = Av1ForwardTransformArithmetic.HalfButterfly(cospi[40], buffer0[10], cospi[24], buffer0[11], cosBit, in rounding); + buffer1[11] = Av1ForwardTransformArithmetic.HalfButterfly(cospi[24], buffer0[10], -cospi[40], buffer0[11], cosBit, in rounding); + buffer1[12] = Av1ForwardTransformArithmetic.HalfButterfly(-cospi[56], buffer0[12], cospi[8], buffer0[13], cosBit, in rounding); + buffer1[13] = Av1ForwardTransformArithmetic.HalfButterfly(cospi[8], buffer0[12], cospi[56], buffer0[13], cosBit, in rounding); + buffer1[14] = Av1ForwardTransformArithmetic.HalfButterfly(-cospi[24], buffer0[14], cospi[40], buffer0[15], cosBit, in rounding); + buffer1[15] = Av1ForwardTransformArithmetic.HalfButterfly(cospi[40], buffer0[14], cospi[24], buffer0[15], cosBit, in rounding); + + // Stage 7 creates the eight final butterfly pairs spanning both octets. + for (int i = 0; i < 8; i++) + { + Av1ForwardTransformArithmetic.AddSubtract(buffer1[i], buffer1[i + 8], out buffer0[i], out buffer0[i + 8]); + } + + ReadOnlySpan finalWeights = FinalWeights; + + // Stage 8 applies the final odd-angle rotations. The compact weight table preserves their normative order + // without allocating a per-call array or duplicating the complementary cosine-index calculation. + for (int pair = 0; pair < 8; pair++) + { + int first = finalWeights[pair]; + int second = 64 - first; + int index = pair * 2; + buffer1[index] = Av1ForwardTransformArithmetic.HalfButterfly( + cospi[first], buffer0[index], cospi[second], buffer0[index + 1], cosBit, in rounding); + + buffer1[index + 1] = Av1ForwardTransformArithmetic.HalfButterfly( + cospi[second], buffer0[index], -cospi[first], buffer0[index + 1], cosBit, in rounding); + } + + ReadOnlySpan outputOrder = OutputOrder; + + // Stage 9 maps the rotated values to ascending AV1 ADST coefficient order. + for (int i = 0; i < 16; i++) + { + Store(ref values, outputStride, i, buffer1[outputOrder[i]]); + } + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + Av1TransformRounding rounding = Av1ForwardTransformArithmetic>.CreateRounding(cosBit); + + // Stage 1 is the bit-reversed ADST input order with the normative alternating signs. + buffer0[0] = Load>(ref values, inputStride, 0); + buffer0[1] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 15)); + buffer0[2] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 7)); + buffer0[3] = Load>(ref values, inputStride, 8); + buffer0[4] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 3)); + buffer0[5] = Load>(ref values, inputStride, 12); + buffer0[6] = Load>(ref values, inputStride, 4); + buffer0[7] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 11)); + buffer0[8] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 1)); + buffer0[9] = Load>(ref values, inputStride, 14); + buffer0[10] = Load>(ref values, inputStride, 6); + buffer0[11] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 9)); + buffer0[12] = Load>(ref values, inputStride, 2); + buffer0[13] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 13)); + buffer0[14] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 5)); + buffer0[15] = Load>(ref values, inputStride, 10); + + // Stage 2 rotates the second pair in each group of four while copying the first pair unchanged. + for (int group = 0; group < 16; group += 4) + { + buffer1[group] = buffer0[group]; + buffer1[group + 1] = buffer0[group + 1]; + Butterfly(cospi[32], cospi[32], buffer0[group + 2], buffer0[group + 3], ref buffer1, group + 2, group + 3, cosBit, in rounding); + } + + // Stage 3 combines adjacent pairs within each group of four. + for (int group = 0; group < 16; group += 4) + { + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract( + buffer1[group + i], + buffer1[group + i + 2], + out buffer0[group + i], + out buffer0[group + i + 2]); + } + } + + // Stage 4 rotates the upper pair of each eight-value group by pi/8. + for (int group = 0; group < 16; group += 8) + { + for (int i = 0; i < 4; i++) + { + buffer1[group + i] = buffer0[group + i]; + } + + buffer1[group + 4] = Av1ForwardTransformArithmetic>.HalfButterfly( + cospi[16], buffer0[group + 4], cospi[48], buffer0[group + 5], cosBit, in rounding); + + buffer1[group + 5] = Av1ForwardTransformArithmetic>.HalfButterfly( + cospi[48], buffer0[group + 4], -cospi[16], buffer0[group + 5], cosBit, in rounding); + + buffer1[group + 6] = Av1ForwardTransformArithmetic>.HalfButterfly( + -cospi[48], buffer0[group + 6], cospi[16], buffer0[group + 7], cosBit, in rounding); + + buffer1[group + 7] = Av1ForwardTransformArithmetic>.HalfButterfly( + cospi[16], buffer0[group + 6], cospi[48], buffer0[group + 7], cosBit, in rounding); + } + + // Stage 5 combines the lower and upper quartets within each eight-value group. + for (int group = 0; group < 16; group += 8) + { + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract( + buffer1[group + i], + buffer1[group + i + 4], + out buffer0[group + i], + out buffer0[group + i + 4]); + } + } + + // Stage 6 rotates the upper octet by pi/16 while retaining the completed lower octet. + for (int i = 0; i < 8; i++) + { + buffer1[i] = buffer0[i]; + } + + buffer1[8] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[8], buffer0[8], cospi[56], buffer0[9], cosBit, in rounding); + buffer1[9] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[56], buffer0[8], -cospi[8], buffer0[9], cosBit, in rounding); + buffer1[10] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[40], buffer0[10], cospi[24], buffer0[11], cosBit, in rounding); + buffer1[11] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[24], buffer0[10], -cospi[40], buffer0[11], cosBit, in rounding); + buffer1[12] = Av1ForwardTransformArithmetic>.HalfButterfly(-cospi[56], buffer0[12], cospi[8], buffer0[13], cosBit, in rounding); + buffer1[13] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[8], buffer0[12], cospi[56], buffer0[13], cosBit, in rounding); + buffer1[14] = Av1ForwardTransformArithmetic>.HalfButterfly(-cospi[24], buffer0[14], cospi[40], buffer0[15], cosBit, in rounding); + buffer1[15] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[40], buffer0[14], cospi[24], buffer0[15], cosBit, in rounding); + + // Stage 7 creates the eight final butterfly pairs spanning both octets. + for (int i = 0; i < 8; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[i], buffer1[i + 8], out buffer0[i], out buffer0[i + 8]); + } + + ReadOnlySpan finalWeights = FinalWeights; + + // Stage 8 applies the final odd-angle rotations. The compact weight table preserves their normative order + // without allocating a per-call array or duplicating the complementary cosine-index calculation. + for (int pair = 0; pair < 8; pair++) + { + int first = finalWeights[pair]; + int second = 64 - first; + int index = pair * 2; + buffer1[index] = Av1ForwardTransformArithmetic>.HalfButterfly( + cospi[first], buffer0[index], cospi[second], buffer0[index + 1], cosBit, in rounding); + + buffer1[index + 1] = Av1ForwardTransformArithmetic>.HalfButterfly( + cospi[second], buffer0[index], -cospi[first], buffer0[index + 1], cosBit, in rounding); + } + + ReadOnlySpan outputOrder = OutputOrder; + + // Stage 9 maps the rotated values to ascending AV1 ADST coefficient order. + for (int i = 0; i < 16; i++) + { + Store(ref values, outputStride, i, buffer1[outputOrder[i]]); + } + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + Av1TransformRounding rounding = Av1ForwardTransformArithmetic>.CreateRounding(cosBit); + + // Stage 1 is the bit-reversed ADST input order with the normative alternating signs. + buffer0[0] = Load>(ref values, inputStride, 0); + buffer0[1] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 15)); + buffer0[2] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 7)); + buffer0[3] = Load>(ref values, inputStride, 8); + buffer0[4] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 3)); + buffer0[5] = Load>(ref values, inputStride, 12); + buffer0[6] = Load>(ref values, inputStride, 4); + buffer0[7] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 11)); + buffer0[8] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 1)); + buffer0[9] = Load>(ref values, inputStride, 14); + buffer0[10] = Load>(ref values, inputStride, 6); + buffer0[11] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 9)); + buffer0[12] = Load>(ref values, inputStride, 2); + buffer0[13] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 13)); + buffer0[14] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 5)); + buffer0[15] = Load>(ref values, inputStride, 10); + + // Stage 2 rotates the second pair in each group of four while copying the first pair unchanged. + for (int group = 0; group < 16; group += 4) + { + buffer1[group] = buffer0[group]; + buffer1[group + 1] = buffer0[group + 1]; + Butterfly(cospi[32], cospi[32], buffer0[group + 2], buffer0[group + 3], ref buffer1, group + 2, group + 3, cosBit, in rounding); + } + + // Stage 3 combines adjacent pairs within each group of four. + for (int group = 0; group < 16; group += 4) + { + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract( + buffer1[group + i], + buffer1[group + i + 2], + out buffer0[group + i], + out buffer0[group + i + 2]); + } + } + + // Stage 4 rotates the upper pair of each eight-value group by pi/8. + for (int group = 0; group < 16; group += 8) + { + for (int i = 0; i < 4; i++) + { + buffer1[group + i] = buffer0[group + i]; + } + + buffer1[group + 4] = Av1ForwardTransformArithmetic>.HalfButterfly( + cospi[16], buffer0[group + 4], cospi[48], buffer0[group + 5], cosBit, in rounding); + + buffer1[group + 5] = Av1ForwardTransformArithmetic>.HalfButterfly( + cospi[48], buffer0[group + 4], -cospi[16], buffer0[group + 5], cosBit, in rounding); + + buffer1[group + 6] = Av1ForwardTransformArithmetic>.HalfButterfly( + -cospi[48], buffer0[group + 6], cospi[16], buffer0[group + 7], cosBit, in rounding); + + buffer1[group + 7] = Av1ForwardTransformArithmetic>.HalfButterfly( + cospi[16], buffer0[group + 6], cospi[48], buffer0[group + 7], cosBit, in rounding); + } + + // Stage 5 combines the lower and upper quartets within each eight-value group. + for (int group = 0; group < 16; group += 8) + { + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract( + buffer1[group + i], + buffer1[group + i + 4], + out buffer0[group + i], + out buffer0[group + i + 4]); + } + } + + // Stage 6 rotates the upper octet by pi/16 while retaining the completed lower octet. + for (int i = 0; i < 8; i++) + { + buffer1[i] = buffer0[i]; + } + + buffer1[8] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[8], buffer0[8], cospi[56], buffer0[9], cosBit, in rounding); + buffer1[9] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[56], buffer0[8], -cospi[8], buffer0[9], cosBit, in rounding); + buffer1[10] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[40], buffer0[10], cospi[24], buffer0[11], cosBit, in rounding); + buffer1[11] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[24], buffer0[10], -cospi[40], buffer0[11], cosBit, in rounding); + buffer1[12] = Av1ForwardTransformArithmetic>.HalfButterfly(-cospi[56], buffer0[12], cospi[8], buffer0[13], cosBit, in rounding); + buffer1[13] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[8], buffer0[12], cospi[56], buffer0[13], cosBit, in rounding); + buffer1[14] = Av1ForwardTransformArithmetic>.HalfButterfly(-cospi[24], buffer0[14], cospi[40], buffer0[15], cosBit, in rounding); + buffer1[15] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[40], buffer0[14], cospi[24], buffer0[15], cosBit, in rounding); + + // Stage 7 creates the eight final butterfly pairs spanning both octets. + for (int i = 0; i < 8; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[i], buffer1[i + 8], out buffer0[i], out buffer0[i + 8]); + } + + ReadOnlySpan finalWeights = FinalWeights; + + // Stage 8 applies the final odd-angle rotations. The compact weight table preserves their normative order + // without allocating a per-call array or duplicating the complementary cosine-index calculation. + for (int pair = 0; pair < 8; pair++) + { + int first = finalWeights[pair]; + int second = 64 - first; + int index = pair * 2; + buffer1[index] = Av1ForwardTransformArithmetic>.HalfButterfly( + cospi[first], buffer0[index], cospi[second], buffer0[index + 1], cosBit, in rounding); + + buffer1[index + 1] = Av1ForwardTransformArithmetic>.HalfButterfly( + cospi[second], buffer0[index], -cospi[first], buffer0[index + 1], cosBit, in rounding); + } + + ReadOnlySpan outputOrder = OutputOrder; + + // Stage 9 maps the rotated values to ascending AV1 ADST coefficient order. + for (int i = 0; i < 16; i++) + { + Store(ref values, outputStride, i, buffer1[outputOrder[i]]); + } + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + Av1TransformRounding rounding = Av1ForwardTransformArithmetic>.CreateRounding(cosBit); + + // Stage 1 is the bit-reversed ADST input order with the normative alternating signs. + buffer0[0] = Load>(ref values, inputStride, 0); + buffer0[1] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 15)); + buffer0[2] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 7)); + buffer0[3] = Load>(ref values, inputStride, 8); + buffer0[4] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 3)); + buffer0[5] = Load>(ref values, inputStride, 12); + buffer0[6] = Load>(ref values, inputStride, 4); + buffer0[7] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 11)); + buffer0[8] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 1)); + buffer0[9] = Load>(ref values, inputStride, 14); + buffer0[10] = Load>(ref values, inputStride, 6); + buffer0[11] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 9)); + buffer0[12] = Load>(ref values, inputStride, 2); + buffer0[13] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 13)); + buffer0[14] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 5)); + buffer0[15] = Load>(ref values, inputStride, 10); + + // Stage 2 rotates the second pair in each group of four while copying the first pair unchanged. + for (int group = 0; group < 16; group += 4) + { + buffer1[group] = buffer0[group]; + buffer1[group + 1] = buffer0[group + 1]; + Butterfly(cospi[32], cospi[32], buffer0[group + 2], buffer0[group + 3], ref buffer1, group + 2, group + 3, cosBit, in rounding); + } + + // Stage 3 combines adjacent pairs within each group of four. + for (int group = 0; group < 16; group += 4) + { + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract( + buffer1[group + i], + buffer1[group + i + 2], + out buffer0[group + i], + out buffer0[group + i + 2]); + } + } + + // Stage 4 rotates the upper pair of each eight-value group by pi/8. + for (int group = 0; group < 16; group += 8) + { + for (int i = 0; i < 4; i++) + { + buffer1[group + i] = buffer0[group + i]; + } + + buffer1[group + 4] = Av1ForwardTransformArithmetic>.HalfButterfly( + cospi[16], buffer0[group + 4], cospi[48], buffer0[group + 5], cosBit, in rounding); + + buffer1[group + 5] = Av1ForwardTransformArithmetic>.HalfButterfly( + cospi[48], buffer0[group + 4], -cospi[16], buffer0[group + 5], cosBit, in rounding); + + buffer1[group + 6] = Av1ForwardTransformArithmetic>.HalfButterfly( + -cospi[48], buffer0[group + 6], cospi[16], buffer0[group + 7], cosBit, in rounding); + + buffer1[group + 7] = Av1ForwardTransformArithmetic>.HalfButterfly( + cospi[16], buffer0[group + 6], cospi[48], buffer0[group + 7], cosBit, in rounding); + } + + // Stage 5 combines the lower and upper quartets within each eight-value group. + for (int group = 0; group < 16; group += 8) + { + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract( + buffer1[group + i], + buffer1[group + i + 4], + out buffer0[group + i], + out buffer0[group + i + 4]); + } + } + + // Stage 6 rotates the upper octet by pi/16 while retaining the completed lower octet. + for (int i = 0; i < 8; i++) + { + buffer1[i] = buffer0[i]; + } + + buffer1[8] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[8], buffer0[8], cospi[56], buffer0[9], cosBit, in rounding); + buffer1[9] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[56], buffer0[8], -cospi[8], buffer0[9], cosBit, in rounding); + buffer1[10] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[40], buffer0[10], cospi[24], buffer0[11], cosBit, in rounding); + buffer1[11] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[24], buffer0[10], -cospi[40], buffer0[11], cosBit, in rounding); + buffer1[12] = Av1ForwardTransformArithmetic>.HalfButterfly(-cospi[56], buffer0[12], cospi[8], buffer0[13], cosBit, in rounding); + buffer1[13] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[8], buffer0[12], cospi[56], buffer0[13], cosBit, in rounding); + buffer1[14] = Av1ForwardTransformArithmetic>.HalfButterfly(-cospi[24], buffer0[14], cospi[40], buffer0[15], cosBit, in rounding); + buffer1[15] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[40], buffer0[14], cospi[24], buffer0[15], cosBit, in rounding); + + // Stage 7 creates the eight final butterfly pairs spanning both octets. + for (int i = 0; i < 8; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[i], buffer1[i + 8], out buffer0[i], out buffer0[i + 8]); + } + + ReadOnlySpan finalWeights = FinalWeights; + + // Stage 8 applies the final odd-angle rotations. The compact weight table preserves their normative order + // without allocating a per-call array or duplicating the complementary cosine-index calculation. + for (int pair = 0; pair < 8; pair++) + { + int first = finalWeights[pair]; + int second = 64 - first; + int index = pair * 2; + buffer1[index] = Av1ForwardTransformArithmetic>.HalfButterfly( + cospi[first], buffer0[index], cospi[second], buffer0[index + 1], cosBit, in rounding); + + buffer1[index + 1] = Av1ForwardTransformArithmetic>.HalfButterfly( + cospi[second], buffer0[index], -cospi[first], buffer0[index + 1], cosBit, in rounding); + } + + ReadOnlySpan outputOrder = OutputOrder; + + // Stage 9 maps the rotated values to ascending AV1 ADST coefficient order. + for (int i = 0; i < 16; i++) + { + Store(ref values, outputStride, i, buffer1[outputOrder[i]]); + } + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + Av1TransformRounding rounding = Av1ForwardTransformArithmetic>.CreateRounding(cosBit); + + // Stage 1 is the bit-reversed ADST input order with the normative alternating signs. + buffer0[0] = Load>(ref values, inputStride, 0); + buffer0[1] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 15)); + buffer0[2] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 7)); + buffer0[3] = Load>(ref values, inputStride, 8); + buffer0[4] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 3)); + buffer0[5] = Load>(ref values, inputStride, 12); + buffer0[6] = Load>(ref values, inputStride, 4); + buffer0[7] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 11)); + buffer0[8] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 1)); + buffer0[9] = Load>(ref values, inputStride, 14); + buffer0[10] = Load>(ref values, inputStride, 6); + buffer0[11] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 9)); + buffer0[12] = Load>(ref values, inputStride, 2); + buffer0[13] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 13)); + buffer0[14] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 5)); + buffer0[15] = Load>(ref values, inputStride, 10); + + // Stage 2 rotates the second pair in each group of four while copying the first pair unchanged. + for (int group = 0; group < 16; group += 4) + { + buffer1[group] = buffer0[group]; + buffer1[group + 1] = buffer0[group + 1]; + Butterfly(cospi[32], cospi[32], buffer0[group + 2], buffer0[group + 3], ref buffer1, group + 2, group + 3, cosBit, in rounding); + } + + // Stage 3 combines adjacent pairs within each group of four. + for (int group = 0; group < 16; group += 4) + { + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract( + buffer1[group + i], + buffer1[group + i + 2], + out buffer0[group + i], + out buffer0[group + i + 2]); + } + } + + // Stage 4 rotates the upper pair of each eight-value group by pi/8. + for (int group = 0; group < 16; group += 8) + { + for (int i = 0; i < 4; i++) + { + buffer1[group + i] = buffer0[group + i]; + } + + buffer1[group + 4] = Av1ForwardTransformArithmetic>.HalfButterfly( + cospi[16], buffer0[group + 4], cospi[48], buffer0[group + 5], cosBit, in rounding); + + buffer1[group + 5] = Av1ForwardTransformArithmetic>.HalfButterfly( + cospi[48], buffer0[group + 4], -cospi[16], buffer0[group + 5], cosBit, in rounding); + + buffer1[group + 6] = Av1ForwardTransformArithmetic>.HalfButterfly( + -cospi[48], buffer0[group + 6], cospi[16], buffer0[group + 7], cosBit, in rounding); + + buffer1[group + 7] = Av1ForwardTransformArithmetic>.HalfButterfly( + cospi[16], buffer0[group + 6], cospi[48], buffer0[group + 7], cosBit, in rounding); + } + + // Stage 5 combines the lower and upper quartets within each eight-value group. + for (int group = 0; group < 16; group += 8) + { + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract( + buffer1[group + i], + buffer1[group + i + 4], + out buffer0[group + i], + out buffer0[group + i + 4]); + } + } + + // Stage 6 rotates the upper octet by pi/16 while retaining the completed lower octet. + for (int i = 0; i < 8; i++) + { + buffer1[i] = buffer0[i]; + } + + buffer1[8] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[8], buffer0[8], cospi[56], buffer0[9], cosBit, in rounding); + buffer1[9] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[56], buffer0[8], -cospi[8], buffer0[9], cosBit, in rounding); + buffer1[10] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[40], buffer0[10], cospi[24], buffer0[11], cosBit, in rounding); + buffer1[11] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[24], buffer0[10], -cospi[40], buffer0[11], cosBit, in rounding); + buffer1[12] = Av1ForwardTransformArithmetic>.HalfButterfly(-cospi[56], buffer0[12], cospi[8], buffer0[13], cosBit, in rounding); + buffer1[13] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[8], buffer0[12], cospi[56], buffer0[13], cosBit, in rounding); + buffer1[14] = Av1ForwardTransformArithmetic>.HalfButterfly(-cospi[24], buffer0[14], cospi[40], buffer0[15], cosBit, in rounding); + buffer1[15] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[40], buffer0[14], cospi[24], buffer0[15], cosBit, in rounding); + + // Stage 7 creates the eight final butterfly pairs spanning both octets. + for (int i = 0; i < 8; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[i], buffer1[i + 8], out buffer0[i], out buffer0[i + 8]); + } + + ReadOnlySpan finalWeights = FinalWeights; + + // Stage 8 applies the final odd-angle rotations. The compact weight table preserves their normative order + // without allocating a per-call array or duplicating the complementary cosine-index calculation. + for (int pair = 0; pair < 8; pair++) + { + int first = finalWeights[pair]; + int second = 64 - first; + int index = pair * 2; + buffer1[index] = Av1ForwardTransformArithmetic>.HalfButterfly( + cospi[first], buffer0[index], cospi[second], buffer0[index + 1], cosBit, in rounding); + + buffer1[index + 1] = Av1ForwardTransformArithmetic>.HalfButterfly( + cospi[second], buffer0[index], -cospi[first], buffer0[index + 1], cosBit, in rounding); + } + + ReadOnlySpan outputOrder = OutputOrder; + + // Stage 9 maps the rotated values to ascending AV1 ADST coefficient order. + for (int i = 0; i < 16; i++) + { + Store(ref values, outputStride, i, buffer1[outputOrder[i]]); + } + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + Av1TransformRounding rounding = Av1ForwardTransformArithmetic>.CreateRounding(cosBit); + + // Stage 1 is the bit-reversed ADST input order with the normative alternating signs. + buffer0[0] = Load>(ref values, inputStride, 0); + buffer0[1] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 15)); + buffer0[2] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 7)); + buffer0[3] = Load>(ref values, inputStride, 8); + buffer0[4] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 3)); + buffer0[5] = Load>(ref values, inputStride, 12); + buffer0[6] = Load>(ref values, inputStride, 4); + buffer0[7] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 11)); + buffer0[8] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 1)); + buffer0[9] = Load>(ref values, inputStride, 14); + buffer0[10] = Load>(ref values, inputStride, 6); + buffer0[11] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 9)); + buffer0[12] = Load>(ref values, inputStride, 2); + buffer0[13] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 13)); + buffer0[14] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 5)); + buffer0[15] = Load>(ref values, inputStride, 10); + + // Stage 2 rotates the second pair in each group of four while copying the first pair unchanged. + for (int group = 0; group < 16; group += 4) + { + buffer1[group] = buffer0[group]; + buffer1[group + 1] = buffer0[group + 1]; + Butterfly(cospi[32], cospi[32], buffer0[group + 2], buffer0[group + 3], ref buffer1, group + 2, group + 3, cosBit, in rounding); + } + + // Stage 3 combines adjacent pairs within each group of four. + for (int group = 0; group < 16; group += 4) + { + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract( + buffer1[group + i], + buffer1[group + i + 2], + out buffer0[group + i], + out buffer0[group + i + 2]); + } + } + + // Stage 4 rotates the upper pair of each eight-value group by pi/8. + for (int group = 0; group < 16; group += 8) + { + for (int i = 0; i < 4; i++) + { + buffer1[group + i] = buffer0[group + i]; + } + + buffer1[group + 4] = Av1ForwardTransformArithmetic>.HalfButterfly( + cospi[16], buffer0[group + 4], cospi[48], buffer0[group + 5], cosBit, in rounding); + + buffer1[group + 5] = Av1ForwardTransformArithmetic>.HalfButterfly( + cospi[48], buffer0[group + 4], -cospi[16], buffer0[group + 5], cosBit, in rounding); + + buffer1[group + 6] = Av1ForwardTransformArithmetic>.HalfButterfly( + -cospi[48], buffer0[group + 6], cospi[16], buffer0[group + 7], cosBit, in rounding); + + buffer1[group + 7] = Av1ForwardTransformArithmetic>.HalfButterfly( + cospi[16], buffer0[group + 6], cospi[48], buffer0[group + 7], cosBit, in rounding); + } + + // Stage 5 combines the lower and upper quartets within each eight-value group. + for (int group = 0; group < 16; group += 8) + { + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract( + buffer1[group + i], + buffer1[group + i + 4], + out buffer0[group + i], + out buffer0[group + i + 4]); + } + } + + // Stage 6 rotates the upper octet by pi/16 while retaining the completed lower octet. + for (int i = 0; i < 8; i++) + { + buffer1[i] = buffer0[i]; + } + + buffer1[8] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[8], buffer0[8], cospi[56], buffer0[9], cosBit, in rounding); + buffer1[9] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[56], buffer0[8], -cospi[8], buffer0[9], cosBit, in rounding); + buffer1[10] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[40], buffer0[10], cospi[24], buffer0[11], cosBit, in rounding); + buffer1[11] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[24], buffer0[10], -cospi[40], buffer0[11], cosBit, in rounding); + buffer1[12] = Av1ForwardTransformArithmetic>.HalfButterfly(-cospi[56], buffer0[12], cospi[8], buffer0[13], cosBit, in rounding); + buffer1[13] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[8], buffer0[12], cospi[56], buffer0[13], cosBit, in rounding); + buffer1[14] = Av1ForwardTransformArithmetic>.HalfButterfly(-cospi[24], buffer0[14], cospi[40], buffer0[15], cosBit, in rounding); + buffer1[15] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[40], buffer0[14], cospi[24], buffer0[15], cosBit, in rounding); + + // Stage 7 creates the eight final butterfly pairs spanning both octets. + for (int i = 0; i < 8; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[i], buffer1[i + 8], out buffer0[i], out buffer0[i + 8]); + } + + ReadOnlySpan finalWeights = FinalWeights; + + // Stage 8 applies the final odd-angle rotations. The compact weight table preserves their normative order + // without allocating a per-call array or duplicating the complementary cosine-index calculation. + for (int pair = 0; pair < 8; pair++) + { + int first = finalWeights[pair]; + int second = 64 - first; + int index = pair * 2; + buffer1[index] = Av1ForwardTransformArithmetic>.HalfButterfly( + cospi[first], buffer0[index], cospi[second], buffer0[index + 1], cosBit, in rounding); + + buffer1[index + 1] = Av1ForwardTransformArithmetic>.HalfButterfly( + cospi[second], buffer0[index], -cospi[first], buffer0[index + 1], cosBit, in rounding); + } + + ReadOnlySpan outputOrder = OutputOrder; + + // Stage 9 maps the rotated values to ascending AV1 ADST coefficient order. + for (int i = 0; i < 16; i++) + { + Store(ref values, outputStride, i, buffer1[outputOrder[i]]); + } + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + Av1TransformRounding rounding = Av1ForwardTransformArithmetic>.CreateRounding(cosBit); + + // Stage 1 is the bit-reversed ADST input order with the normative alternating signs. + buffer0[0] = Load>(ref values, inputStride, 0); + buffer0[1] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 15)); + buffer0[2] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 7)); + buffer0[3] = Load>(ref values, inputStride, 8); + buffer0[4] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 3)); + buffer0[5] = Load>(ref values, inputStride, 12); + buffer0[6] = Load>(ref values, inputStride, 4); + buffer0[7] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 11)); + buffer0[8] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 1)); + buffer0[9] = Load>(ref values, inputStride, 14); + buffer0[10] = Load>(ref values, inputStride, 6); + buffer0[11] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 9)); + buffer0[12] = Load>(ref values, inputStride, 2); + buffer0[13] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 13)); + buffer0[14] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 5)); + buffer0[15] = Load>(ref values, inputStride, 10); + + // Stage 2 rotates the second pair in each group of four while copying the first pair unchanged. + for (int group = 0; group < 16; group += 4) + { + buffer1[group] = buffer0[group]; + buffer1[group + 1] = buffer0[group + 1]; + Butterfly(cospi[32], cospi[32], buffer0[group + 2], buffer0[group + 3], ref buffer1, group + 2, group + 3, cosBit, in rounding); + } + + // Stage 3 combines adjacent pairs within each group of four. + for (int group = 0; group < 16; group += 4) + { + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract( + buffer1[group + i], + buffer1[group + i + 2], + out buffer0[group + i], + out buffer0[group + i + 2]); + } + } + + // Stage 4 rotates the upper pair of each eight-value group by pi/8. + for (int group = 0; group < 16; group += 8) + { + for (int i = 0; i < 4; i++) + { + buffer1[group + i] = buffer0[group + i]; + } + + buffer1[group + 4] = Av1ForwardTransformArithmetic>.HalfButterfly( + cospi[16], buffer0[group + 4], cospi[48], buffer0[group + 5], cosBit, in rounding); + + buffer1[group + 5] = Av1ForwardTransformArithmetic>.HalfButterfly( + cospi[48], buffer0[group + 4], -cospi[16], buffer0[group + 5], cosBit, in rounding); + + buffer1[group + 6] = Av1ForwardTransformArithmetic>.HalfButterfly( + -cospi[48], buffer0[group + 6], cospi[16], buffer0[group + 7], cosBit, in rounding); + + buffer1[group + 7] = Av1ForwardTransformArithmetic>.HalfButterfly( + cospi[16], buffer0[group + 6], cospi[48], buffer0[group + 7], cosBit, in rounding); + } + + // Stage 5 combines the lower and upper quartets within each eight-value group. + for (int group = 0; group < 16; group += 8) + { + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract( + buffer1[group + i], + buffer1[group + i + 4], + out buffer0[group + i], + out buffer0[group + i + 4]); + } + } + + // Stage 6 rotates the upper octet by pi/16 while retaining the completed lower octet. + for (int i = 0; i < 8; i++) + { + buffer1[i] = buffer0[i]; + } + + buffer1[8] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[8], buffer0[8], cospi[56], buffer0[9], cosBit, in rounding); + buffer1[9] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[56], buffer0[8], -cospi[8], buffer0[9], cosBit, in rounding); + buffer1[10] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[40], buffer0[10], cospi[24], buffer0[11], cosBit, in rounding); + buffer1[11] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[24], buffer0[10], -cospi[40], buffer0[11], cosBit, in rounding); + buffer1[12] = Av1ForwardTransformArithmetic>.HalfButterfly(-cospi[56], buffer0[12], cospi[8], buffer0[13], cosBit, in rounding); + buffer1[13] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[8], buffer0[12], cospi[56], buffer0[13], cosBit, in rounding); + buffer1[14] = Av1ForwardTransformArithmetic>.HalfButterfly(-cospi[24], buffer0[14], cospi[40], buffer0[15], cosBit, in rounding); + buffer1[15] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[40], buffer0[14], cospi[24], buffer0[15], cosBit, in rounding); + + // Stage 7 creates the eight final butterfly pairs spanning both octets. + for (int i = 0; i < 8; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[i], buffer1[i + 8], out buffer0[i], out buffer0[i + 8]); + } + + ReadOnlySpan finalWeights = FinalWeights; + + // Stage 8 applies the final odd-angle rotations. The compact weight table preserves their normative order + // without allocating a per-call array or duplicating the complementary cosine-index calculation. + for (int pair = 0; pair < 8; pair++) + { + int first = finalWeights[pair]; + int second = 64 - first; + int index = pair * 2; + buffer1[index] = Av1ForwardTransformArithmetic>.HalfButterfly( + cospi[first], buffer0[index], cospi[second], buffer0[index + 1], cosBit, in rounding); + + buffer1[index + 1] = Av1ForwardTransformArithmetic>.HalfButterfly( + cospi[second], buffer0[index], -cospi[first], buffer0[index + 1], cosBit, in rounding); + } + + ReadOnlySpan outputOrder = OutputOrder; + + // Stage 9 maps the rotated values to ascending AV1 ADST coefficient order. + for (int i = 0; i < 16; i++) + { + Store(ref values, outputStride, i, buffer1[outputOrder[i]]); + } + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Transform/Forward/Av1ForwardTransformer.Adst4Operator.cs b/src/ImageSharp/Formats/Heif/Av1/Transform/Forward/Av1ForwardTransformer.Adst4Operator.cs new file mode 100644 index 0000000000..6416b8a6c6 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Transform/Forward/Av1ForwardTransformer.Adst4Operator.cs @@ -0,0 +1,411 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.Intrinsics; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform.Forward; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +/// +/// Defines the four-point forward ADST operator. +/// +internal static partial class Av1ForwardTransformer +{ + /// + /// Implements the four-point forward asymmetric discrete sine transform. + /// + internal readonly struct Adst4Operator : IAv1ForwardTransform1dOperator + { + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector buffer0, + ref Av1TransformVector buffer1, + int cosBit) + { + _ = buffer0; + _ = buffer1; + + ReadOnlySpan sinpi = Av1SinusConstants.SinusPi(cosBit); + Av1TransformRounding rounding = Av1ForwardTransformArithmetic.CreateRounding(cosBit); + int input0 = Load(ref values, inputStride, 0); + int input1 = Load(ref values, inputStride, 1); + int input2 = Load(ref values, inputStride, 2); + int input3 = Load(ref values, inputStride, 3); + int input01 = Av1ForwardTransformArithmetic.Add(input0, input1); + + // Packed lanes form input0 + input1 before widening, matching Highway's observable saturating arithmetic. + int output0 = Av1ForwardTransformArithmetic.MultiplyAddRound( + sinpi[1], input0, sinpi[2], input1, sinpi[3], input2, sinpi[4], input3, cosBit, in rounding); + + int output1 = Av1ForwardTransformArithmetic.MultiplyAddRound( + sinpi[3], input01, -sinpi[3], input3, 0, input0, 0, input0, cosBit, in rounding); + + int output2 = Av1ForwardTransformArithmetic.MultiplyAddRound( + sinpi[4], input0, -sinpi[1], input1, -sinpi[3], input2, sinpi[2], input3, cosBit, in rounding); + + // This expression preserves Highway's widened w2 - w0 + 3 * v5 sequence with one rounding point. + int output3 = Av1ForwardTransformArithmetic.MultiplyAddRound( + sinpi[4] - sinpi[1], + input0, + -sinpi[1] - sinpi[2], + input1, + sinpi[3], + input2, + sinpi[2] - sinpi[4], + input3, + cosBit, + in rounding); + + Store(ref values, outputStride, 0, output0); + Store(ref values, outputStride, 1, output1); + Store(ref values, outputStride, 2, output2); + Store(ref values, outputStride, 3, output3); + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector buffer0, + ref Av1TransformVector buffer1, + int cosBit) + { + _ = buffer0; + _ = buffer1; + + ReadOnlySpan sinpi = Av1SinusConstants.SinusPi(cosBit); + Av1TransformRounding rounding = Av1ForwardTransformArithmetic.CreateRounding(cosBit); + short input0 = Load(ref values, inputStride, 0); + short input1 = Load(ref values, inputStride, 1); + short input2 = Load(ref values, inputStride, 2); + short input3 = Load(ref values, inputStride, 3); + short input01 = Av1ForwardTransformArithmetic.Add(input0, input1); + + // Packed lanes form input0 + input1 before widening, matching Highway's observable saturating arithmetic. + short output0 = Av1ForwardTransformArithmetic.MultiplyAddRound( + sinpi[1], input0, sinpi[2], input1, sinpi[3], input2, sinpi[4], input3, cosBit, in rounding); + + short output1 = Av1ForwardTransformArithmetic.MultiplyAddRound( + sinpi[3], input01, -sinpi[3], input3, 0, input0, 0, input0, cosBit, in rounding); + + short output2 = Av1ForwardTransformArithmetic.MultiplyAddRound( + sinpi[4], input0, -sinpi[1], input1, -sinpi[3], input2, sinpi[2], input3, cosBit, in rounding); + + // This expression preserves Highway's widened w2 - w0 + 3 * v5 sequence with one rounding point. + short output3 = Av1ForwardTransformArithmetic.MultiplyAddRound( + sinpi[4] - sinpi[1], + input0, + -sinpi[1] - sinpi[2], + input1, + sinpi[3], + input2, + sinpi[2] - sinpi[4], + input3, + cosBit, + in rounding); + + Store(ref values, outputStride, 0, output0); + Store(ref values, outputStride, 1, output1); + Store(ref values, outputStride, 2, output2); + Store(ref values, outputStride, 3, output3); + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + _ = buffer0; + _ = buffer1; + + ReadOnlySpan sinpi = Av1SinusConstants.SinusPi(cosBit); + Av1TransformRounding rounding = Av1ForwardTransformArithmetic>.CreateRounding(cosBit); + Vector128 input0 = Load>(ref values, inputStride, 0); + Vector128 input1 = Load>(ref values, inputStride, 1); + Vector128 input2 = Load>(ref values, inputStride, 2); + Vector128 input3 = Load>(ref values, inputStride, 3); + Vector128 input01 = Av1ForwardTransformArithmetic>.Add(input0, input1); + + // Packed lanes form input0 + input1 before widening, matching Highway's observable saturating arithmetic. + Vector128 output0 = Av1ForwardTransformArithmetic>.MultiplyAddRound( + sinpi[1], input0, sinpi[2], input1, sinpi[3], input2, sinpi[4], input3, cosBit, in rounding); + + Vector128 output1 = Av1ForwardTransformArithmetic>.MultiplyAddRound( + sinpi[3], input01, -sinpi[3], input3, 0, input0, 0, input0, cosBit, in rounding); + + Vector128 output2 = Av1ForwardTransformArithmetic>.MultiplyAddRound( + sinpi[4], input0, -sinpi[1], input1, -sinpi[3], input2, sinpi[2], input3, cosBit, in rounding); + + // This expression preserves Highway's widened w2 - w0 + 3 * v5 sequence with one rounding point. + Vector128 output3 = Av1ForwardTransformArithmetic>.MultiplyAddRound( + sinpi[4] - sinpi[1], + input0, + -sinpi[1] - sinpi[2], + input1, + sinpi[3], + input2, + sinpi[2] - sinpi[4], + input3, + cosBit, + in rounding); + + Store(ref values, outputStride, 0, output0); + Store(ref values, outputStride, 1, output1); + Store(ref values, outputStride, 2, output2); + Store(ref values, outputStride, 3, output3); + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + _ = buffer0; + _ = buffer1; + + ReadOnlySpan sinpi = Av1SinusConstants.SinusPi(cosBit); + Av1TransformRounding rounding = Av1ForwardTransformArithmetic>.CreateRounding(cosBit); + Vector256 input0 = Load>(ref values, inputStride, 0); + Vector256 input1 = Load>(ref values, inputStride, 1); + Vector256 input2 = Load>(ref values, inputStride, 2); + Vector256 input3 = Load>(ref values, inputStride, 3); + Vector256 input01 = Av1ForwardTransformArithmetic>.Add(input0, input1); + + // Packed lanes form input0 + input1 before widening, matching Highway's observable saturating arithmetic. + Vector256 output0 = Av1ForwardTransformArithmetic>.MultiplyAddRound( + sinpi[1], input0, sinpi[2], input1, sinpi[3], input2, sinpi[4], input3, cosBit, in rounding); + + Vector256 output1 = Av1ForwardTransformArithmetic>.MultiplyAddRound( + sinpi[3], input01, -sinpi[3], input3, 0, input0, 0, input0, cosBit, in rounding); + + Vector256 output2 = Av1ForwardTransformArithmetic>.MultiplyAddRound( + sinpi[4], input0, -sinpi[1], input1, -sinpi[3], input2, sinpi[2], input3, cosBit, in rounding); + + // This expression preserves Highway's widened w2 - w0 + 3 * v5 sequence with one rounding point. + Vector256 output3 = Av1ForwardTransformArithmetic>.MultiplyAddRound( + sinpi[4] - sinpi[1], + input0, + -sinpi[1] - sinpi[2], + input1, + sinpi[3], + input2, + sinpi[2] - sinpi[4], + input3, + cosBit, + in rounding); + + Store(ref values, outputStride, 0, output0); + Store(ref values, outputStride, 1, output1); + Store(ref values, outputStride, 2, output2); + Store(ref values, outputStride, 3, output3); + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + _ = buffer0; + _ = buffer1; + + ReadOnlySpan sinpi = Av1SinusConstants.SinusPi(cosBit); + Av1TransformRounding rounding = Av1ForwardTransformArithmetic>.CreateRounding(cosBit); + Vector512 input0 = Load>(ref values, inputStride, 0); + Vector512 input1 = Load>(ref values, inputStride, 1); + Vector512 input2 = Load>(ref values, inputStride, 2); + Vector512 input3 = Load>(ref values, inputStride, 3); + Vector512 input01 = Av1ForwardTransformArithmetic>.Add(input0, input1); + + // Packed lanes form input0 + input1 before widening, matching Highway's observable saturating arithmetic. + Vector512 output0 = Av1ForwardTransformArithmetic>.MultiplyAddRound( + sinpi[1], input0, sinpi[2], input1, sinpi[3], input2, sinpi[4], input3, cosBit, in rounding); + + Vector512 output1 = Av1ForwardTransformArithmetic>.MultiplyAddRound( + sinpi[3], input01, -sinpi[3], input3, 0, input0, 0, input0, cosBit, in rounding); + + Vector512 output2 = Av1ForwardTransformArithmetic>.MultiplyAddRound( + sinpi[4], input0, -sinpi[1], input1, -sinpi[3], input2, sinpi[2], input3, cosBit, in rounding); + + // This expression preserves Highway's widened w2 - w0 + 3 * v5 sequence with one rounding point. + Vector512 output3 = Av1ForwardTransformArithmetic>.MultiplyAddRound( + sinpi[4] - sinpi[1], + input0, + -sinpi[1] - sinpi[2], + input1, + sinpi[3], + input2, + sinpi[2] - sinpi[4], + input3, + cosBit, + in rounding); + + Store(ref values, outputStride, 0, output0); + Store(ref values, outputStride, 1, output1); + Store(ref values, outputStride, 2, output2); + Store(ref values, outputStride, 3, output3); + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + _ = buffer0; + _ = buffer1; + + ReadOnlySpan sinpi = Av1SinusConstants.SinusPi(cosBit); + Av1TransformRounding rounding = Av1ForwardTransformArithmetic>.CreateRounding(cosBit); + Vector128 input0 = Load>(ref values, inputStride, 0); + Vector128 input1 = Load>(ref values, inputStride, 1); + Vector128 input2 = Load>(ref values, inputStride, 2); + Vector128 input3 = Load>(ref values, inputStride, 3); + Vector128 input01 = Av1ForwardTransformArithmetic>.Add(input0, input1); + + // Packed lanes form input0 + input1 before widening, matching Highway's observable saturating arithmetic. + Vector128 output0 = Av1ForwardTransformArithmetic>.MultiplyAddRound( + sinpi[1], input0, sinpi[2], input1, sinpi[3], input2, sinpi[4], input3, cosBit, in rounding); + + Vector128 output1 = Av1ForwardTransformArithmetic>.MultiplyAddRound( + sinpi[3], input01, -sinpi[3], input3, 0, input0, 0, input0, cosBit, in rounding); + + Vector128 output2 = Av1ForwardTransformArithmetic>.MultiplyAddRound( + sinpi[4], input0, -sinpi[1], input1, -sinpi[3], input2, sinpi[2], input3, cosBit, in rounding); + + // This expression preserves Highway's widened w2 - w0 + 3 * v5 sequence with one rounding point. + Vector128 output3 = Av1ForwardTransformArithmetic>.MultiplyAddRound( + sinpi[4] - sinpi[1], + input0, + -sinpi[1] - sinpi[2], + input1, + sinpi[3], + input2, + sinpi[2] - sinpi[4], + input3, + cosBit, + in rounding); + + Store(ref values, outputStride, 0, output0); + Store(ref values, outputStride, 1, output1); + Store(ref values, outputStride, 2, output2); + Store(ref values, outputStride, 3, output3); + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + _ = buffer0; + _ = buffer1; + + ReadOnlySpan sinpi = Av1SinusConstants.SinusPi(cosBit); + Av1TransformRounding rounding = Av1ForwardTransformArithmetic>.CreateRounding(cosBit); + Vector256 input0 = Load>(ref values, inputStride, 0); + Vector256 input1 = Load>(ref values, inputStride, 1); + Vector256 input2 = Load>(ref values, inputStride, 2); + Vector256 input3 = Load>(ref values, inputStride, 3); + Vector256 input01 = Av1ForwardTransformArithmetic>.Add(input0, input1); + + // Packed lanes form input0 + input1 before widening, matching Highway's observable saturating arithmetic. + Vector256 output0 = Av1ForwardTransformArithmetic>.MultiplyAddRound( + sinpi[1], input0, sinpi[2], input1, sinpi[3], input2, sinpi[4], input3, cosBit, in rounding); + + Vector256 output1 = Av1ForwardTransformArithmetic>.MultiplyAddRound( + sinpi[3], input01, -sinpi[3], input3, 0, input0, 0, input0, cosBit, in rounding); + + Vector256 output2 = Av1ForwardTransformArithmetic>.MultiplyAddRound( + sinpi[4], input0, -sinpi[1], input1, -sinpi[3], input2, sinpi[2], input3, cosBit, in rounding); + + // This expression preserves Highway's widened w2 - w0 + 3 * v5 sequence with one rounding point. + Vector256 output3 = Av1ForwardTransformArithmetic>.MultiplyAddRound( + sinpi[4] - sinpi[1], + input0, + -sinpi[1] - sinpi[2], + input1, + sinpi[3], + input2, + sinpi[2] - sinpi[4], + input3, + cosBit, + in rounding); + + Store(ref values, outputStride, 0, output0); + Store(ref values, outputStride, 1, output1); + Store(ref values, outputStride, 2, output2); + Store(ref values, outputStride, 3, output3); + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + _ = buffer0; + _ = buffer1; + + ReadOnlySpan sinpi = Av1SinusConstants.SinusPi(cosBit); + Av1TransformRounding rounding = Av1ForwardTransformArithmetic>.CreateRounding(cosBit); + Vector512 input0 = Load>(ref values, inputStride, 0); + Vector512 input1 = Load>(ref values, inputStride, 1); + Vector512 input2 = Load>(ref values, inputStride, 2); + Vector512 input3 = Load>(ref values, inputStride, 3); + Vector512 input01 = Av1ForwardTransformArithmetic>.Add(input0, input1); + + // Packed lanes form input0 + input1 before widening, matching Highway's observable saturating arithmetic. + Vector512 output0 = Av1ForwardTransformArithmetic>.MultiplyAddRound( + sinpi[1], input0, sinpi[2], input1, sinpi[3], input2, sinpi[4], input3, cosBit, in rounding); + + Vector512 output1 = Av1ForwardTransformArithmetic>.MultiplyAddRound( + sinpi[3], input01, -sinpi[3], input3, 0, input0, 0, input0, cosBit, in rounding); + + Vector512 output2 = Av1ForwardTransformArithmetic>.MultiplyAddRound( + sinpi[4], input0, -sinpi[1], input1, -sinpi[3], input2, sinpi[2], input3, cosBit, in rounding); + + // This expression preserves Highway's widened w2 - w0 + 3 * v5 sequence with one rounding point. + Vector512 output3 = Av1ForwardTransformArithmetic>.MultiplyAddRound( + sinpi[4] - sinpi[1], + input0, + -sinpi[1] - sinpi[2], + input1, + sinpi[3], + input2, + sinpi[2] - sinpi[4], + input3, + cosBit, + in rounding); + + Store(ref values, outputStride, 0, output0); + Store(ref values, outputStride, 1, output1); + Store(ref values, outputStride, 2, output2); + Store(ref values, outputStride, 3, output3); + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Transform/Forward/Av1ForwardTransformer.Adst8Operator.cs b/src/ImageSharp/Formats/Heif/Av1/Transform/Forward/Av1ForwardTransformer.Adst8Operator.cs new file mode 100644 index 0000000000..17bdaa53d1 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Transform/Forward/Av1ForwardTransformer.Adst8Operator.cs @@ -0,0 +1,656 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.Intrinsics; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform.Forward; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +/// +/// Defines the eight-point forward ADST operator. +/// +internal static partial class Av1ForwardTransformer +{ + /// + /// Implements the eight-point forward asymmetric discrete sine transform. + /// + internal readonly struct Adst8Operator : IAv1ForwardTransform1dOperator + { + /// + /// Gets the fixed coefficient permutation. + /// + private static ReadOnlySpan OutputOrder => [1, 6, 3, 4, 5, 2, 7, 0]; + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector buffer0, + ref Av1TransformVector buffer1, + int cosBit) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + Av1TransformRounding rounding = Av1ForwardTransformArithmetic.CreateRounding(cosBit); + + // Stage 1 applies the ADST permutation and signs while the source block is still read-only. + buffer0[0] = Load(ref values, inputStride, 0); + buffer0[1] = Av1ForwardTransformArithmetic.Negate(Load(ref values, inputStride, 7)); + buffer0[2] = Av1ForwardTransformArithmetic.Negate(Load(ref values, inputStride, 3)); + buffer0[3] = Load(ref values, inputStride, 4); + buffer0[4] = Av1ForwardTransformArithmetic.Negate(Load(ref values, inputStride, 1)); + buffer0[5] = Load(ref values, inputStride, 6); + buffer0[6] = Load(ref values, inputStride, 2); + buffer0[7] = Av1ForwardTransformArithmetic.Negate(Load(ref values, inputStride, 5)); + + // Stage 2 rotates the second pair in each four-value group while copying the already aligned pairs. + buffer1[0] = buffer0[0]; + buffer1[1] = buffer0[1]; + Butterfly(cospi[32], cospi[32], buffer0[2], buffer0[3], ref buffer1, 2, 3, cosBit, in rounding); + buffer1[4] = buffer0[4]; + buffer1[5] = buffer0[5]; + Butterfly(cospi[32], cospi[32], buffer0[6], buffer0[7], ref buffer1, 6, 7, cosBit, in rounding); + + // Stage 3 combines the rotated and copied pairs into two independent four-value groups. + for (int group = 0; group < 8; group += 4) + { + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic.AddSubtract( + buffer1[group + i], + buffer1[group + i + 2], + out buffer0[group + i], + out buffer0[group + i + 2]); + } + } + + // Stage 4 rotates the upper group by pi/8 while the completed lower group passes through unchanged. + for (int i = 0; i < 4; i++) + { + buffer1[i] = buffer0[i]; + } + + buffer1[4] = Av1ForwardTransformArithmetic.HalfButterfly(cospi[16], buffer0[4], cospi[48], buffer0[5], cosBit, in rounding); + buffer1[5] = Av1ForwardTransformArithmetic.HalfButterfly(cospi[48], buffer0[4], -cospi[16], buffer0[5], cosBit, in rounding); + buffer1[6] = Av1ForwardTransformArithmetic.HalfButterfly(-cospi[48], buffer0[6], cospi[16], buffer0[7], cosBit, in rounding); + buffer1[7] = Av1ForwardTransformArithmetic.HalfButterfly(cospi[16], buffer0[6], cospi[48], buffer0[7], cosBit, in rounding); + + // Stage 5 creates the four final butterfly pairs spanning the two groups. + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic.AddSubtract(buffer1[i], buffer1[i + 4], out buffer0[i], out buffer0[i + 4]); + } + + // Stage 6 applies the remaining odd-angle rotations. + buffer1[0] = Av1ForwardTransformArithmetic.HalfButterfly(cospi[4], buffer0[0], cospi[60], buffer0[1], cosBit, in rounding); + buffer1[1] = Av1ForwardTransformArithmetic.HalfButterfly(cospi[60], buffer0[0], -cospi[4], buffer0[1], cosBit, in rounding); + buffer1[2] = Av1ForwardTransformArithmetic.HalfButterfly(cospi[20], buffer0[2], cospi[44], buffer0[3], cosBit, in rounding); + buffer1[3] = Av1ForwardTransformArithmetic.HalfButterfly(cospi[44], buffer0[2], -cospi[20], buffer0[3], cosBit, in rounding); + buffer1[4] = Av1ForwardTransformArithmetic.HalfButterfly(cospi[36], buffer0[4], cospi[28], buffer0[5], cosBit, in rounding); + buffer1[5] = Av1ForwardTransformArithmetic.HalfButterfly(cospi[28], buffer0[4], -cospi[36], buffer0[5], cosBit, in rounding); + buffer1[6] = Av1ForwardTransformArithmetic.HalfButterfly(cospi[52], buffer0[6], cospi[12], buffer0[7], cosBit, in rounding); + buffer1[7] = Av1ForwardTransformArithmetic.HalfButterfly(cospi[12], buffer0[6], -cospi[52], buffer0[7], cosBit, in rounding); + + ReadOnlySpan outputOrder = OutputOrder; + + // Stage 7 maps the rotated values to ascending AV1 ADST coefficient order. + for (int i = 0; i < 8; i++) + { + Store(ref values, outputStride, i, buffer1[outputOrder[i]]); + } + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector buffer0, + ref Av1TransformVector buffer1, + int cosBit) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + Av1TransformRounding rounding = Av1ForwardTransformArithmetic.CreateRounding(cosBit); + + // Stage 1 applies the ADST permutation and signs while the source block is still read-only. + buffer0[0] = Load(ref values, inputStride, 0); + buffer0[1] = Av1ForwardTransformArithmetic.Negate(Load(ref values, inputStride, 7)); + buffer0[2] = Av1ForwardTransformArithmetic.Negate(Load(ref values, inputStride, 3)); + buffer0[3] = Load(ref values, inputStride, 4); + buffer0[4] = Av1ForwardTransformArithmetic.Negate(Load(ref values, inputStride, 1)); + buffer0[5] = Load(ref values, inputStride, 6); + buffer0[6] = Load(ref values, inputStride, 2); + buffer0[7] = Av1ForwardTransformArithmetic.Negate(Load(ref values, inputStride, 5)); + + // Stage 2 rotates the second pair in each four-value group while copying the already aligned pairs. + buffer1[0] = buffer0[0]; + buffer1[1] = buffer0[1]; + Butterfly(cospi[32], cospi[32], buffer0[2], buffer0[3], ref buffer1, 2, 3, cosBit, in rounding); + buffer1[4] = buffer0[4]; + buffer1[5] = buffer0[5]; + Butterfly(cospi[32], cospi[32], buffer0[6], buffer0[7], ref buffer1, 6, 7, cosBit, in rounding); + + // Stage 3 combines the rotated and copied pairs into two independent four-value groups. + for (int group = 0; group < 8; group += 4) + { + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic.AddSubtract( + buffer1[group + i], + buffer1[group + i + 2], + out buffer0[group + i], + out buffer0[group + i + 2]); + } + } + + // Stage 4 rotates the upper group by pi/8 while the completed lower group passes through unchanged. + for (int i = 0; i < 4; i++) + { + buffer1[i] = buffer0[i]; + } + + buffer1[4] = Av1ForwardTransformArithmetic.HalfButterfly(cospi[16], buffer0[4], cospi[48], buffer0[5], cosBit, in rounding); + buffer1[5] = Av1ForwardTransformArithmetic.HalfButterfly(cospi[48], buffer0[4], -cospi[16], buffer0[5], cosBit, in rounding); + buffer1[6] = Av1ForwardTransformArithmetic.HalfButterfly(-cospi[48], buffer0[6], cospi[16], buffer0[7], cosBit, in rounding); + buffer1[7] = Av1ForwardTransformArithmetic.HalfButterfly(cospi[16], buffer0[6], cospi[48], buffer0[7], cosBit, in rounding); + + // Stage 5 creates the four final butterfly pairs spanning the two groups. + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic.AddSubtract(buffer1[i], buffer1[i + 4], out buffer0[i], out buffer0[i + 4]); + } + + // Stage 6 applies the remaining odd-angle rotations. + buffer1[0] = Av1ForwardTransformArithmetic.HalfButterfly(cospi[4], buffer0[0], cospi[60], buffer0[1], cosBit, in rounding); + buffer1[1] = Av1ForwardTransformArithmetic.HalfButterfly(cospi[60], buffer0[0], -cospi[4], buffer0[1], cosBit, in rounding); + buffer1[2] = Av1ForwardTransformArithmetic.HalfButterfly(cospi[20], buffer0[2], cospi[44], buffer0[3], cosBit, in rounding); + buffer1[3] = Av1ForwardTransformArithmetic.HalfButterfly(cospi[44], buffer0[2], -cospi[20], buffer0[3], cosBit, in rounding); + buffer1[4] = Av1ForwardTransformArithmetic.HalfButterfly(cospi[36], buffer0[4], cospi[28], buffer0[5], cosBit, in rounding); + buffer1[5] = Av1ForwardTransformArithmetic.HalfButterfly(cospi[28], buffer0[4], -cospi[36], buffer0[5], cosBit, in rounding); + buffer1[6] = Av1ForwardTransformArithmetic.HalfButterfly(cospi[52], buffer0[6], cospi[12], buffer0[7], cosBit, in rounding); + buffer1[7] = Av1ForwardTransformArithmetic.HalfButterfly(cospi[12], buffer0[6], -cospi[52], buffer0[7], cosBit, in rounding); + + ReadOnlySpan outputOrder = OutputOrder; + + // Stage 7 maps the rotated values to ascending AV1 ADST coefficient order. + for (int i = 0; i < 8; i++) + { + Store(ref values, outputStride, i, buffer1[outputOrder[i]]); + } + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + Av1TransformRounding rounding = Av1ForwardTransformArithmetic>.CreateRounding(cosBit); + + // Stage 1 applies the ADST permutation and signs while the source block is still read-only. + buffer0[0] = Load>(ref values, inputStride, 0); + buffer0[1] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 7)); + buffer0[2] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 3)); + buffer0[3] = Load>(ref values, inputStride, 4); + buffer0[4] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 1)); + buffer0[5] = Load>(ref values, inputStride, 6); + buffer0[6] = Load>(ref values, inputStride, 2); + buffer0[7] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 5)); + + // Stage 2 rotates the second pair in each four-value group while copying the already aligned pairs. + buffer1[0] = buffer0[0]; + buffer1[1] = buffer0[1]; + Butterfly(cospi[32], cospi[32], buffer0[2], buffer0[3], ref buffer1, 2, 3, cosBit, in rounding); + buffer1[4] = buffer0[4]; + buffer1[5] = buffer0[5]; + Butterfly(cospi[32], cospi[32], buffer0[6], buffer0[7], ref buffer1, 6, 7, cosBit, in rounding); + + // Stage 3 combines the rotated and copied pairs into two independent four-value groups. + for (int group = 0; group < 8; group += 4) + { + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract( + buffer1[group + i], + buffer1[group + i + 2], + out buffer0[group + i], + out buffer0[group + i + 2]); + } + } + + // Stage 4 rotates the upper group by pi/8 while the completed lower group passes through unchanged. + for (int i = 0; i < 4; i++) + { + buffer1[i] = buffer0[i]; + } + + buffer1[4] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[16], buffer0[4], cospi[48], buffer0[5], cosBit, in rounding); + buffer1[5] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[48], buffer0[4], -cospi[16], buffer0[5], cosBit, in rounding); + buffer1[6] = Av1ForwardTransformArithmetic>.HalfButterfly(-cospi[48], buffer0[6], cospi[16], buffer0[7], cosBit, in rounding); + buffer1[7] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[16], buffer0[6], cospi[48], buffer0[7], cosBit, in rounding); + + // Stage 5 creates the four final butterfly pairs spanning the two groups. + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[i], buffer1[i + 4], out buffer0[i], out buffer0[i + 4]); + } + + // Stage 6 applies the remaining odd-angle rotations. + buffer1[0] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[4], buffer0[0], cospi[60], buffer0[1], cosBit, in rounding); + buffer1[1] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[60], buffer0[0], -cospi[4], buffer0[1], cosBit, in rounding); + buffer1[2] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[20], buffer0[2], cospi[44], buffer0[3], cosBit, in rounding); + buffer1[3] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[44], buffer0[2], -cospi[20], buffer0[3], cosBit, in rounding); + buffer1[4] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[36], buffer0[4], cospi[28], buffer0[5], cosBit, in rounding); + buffer1[5] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[28], buffer0[4], -cospi[36], buffer0[5], cosBit, in rounding); + buffer1[6] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[52], buffer0[6], cospi[12], buffer0[7], cosBit, in rounding); + buffer1[7] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[12], buffer0[6], -cospi[52], buffer0[7], cosBit, in rounding); + + ReadOnlySpan outputOrder = OutputOrder; + + // Stage 7 maps the rotated values to ascending AV1 ADST coefficient order. + for (int i = 0; i < 8; i++) + { + Store(ref values, outputStride, i, buffer1[outputOrder[i]]); + } + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + Av1TransformRounding rounding = Av1ForwardTransformArithmetic>.CreateRounding(cosBit); + + // Stage 1 applies the ADST permutation and signs while the source block is still read-only. + buffer0[0] = Load>(ref values, inputStride, 0); + buffer0[1] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 7)); + buffer0[2] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 3)); + buffer0[3] = Load>(ref values, inputStride, 4); + buffer0[4] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 1)); + buffer0[5] = Load>(ref values, inputStride, 6); + buffer0[6] = Load>(ref values, inputStride, 2); + buffer0[7] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 5)); + + // Stage 2 rotates the second pair in each four-value group while copying the already aligned pairs. + buffer1[0] = buffer0[0]; + buffer1[1] = buffer0[1]; + Butterfly(cospi[32], cospi[32], buffer0[2], buffer0[3], ref buffer1, 2, 3, cosBit, in rounding); + buffer1[4] = buffer0[4]; + buffer1[5] = buffer0[5]; + Butterfly(cospi[32], cospi[32], buffer0[6], buffer0[7], ref buffer1, 6, 7, cosBit, in rounding); + + // Stage 3 combines the rotated and copied pairs into two independent four-value groups. + for (int group = 0; group < 8; group += 4) + { + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract( + buffer1[group + i], + buffer1[group + i + 2], + out buffer0[group + i], + out buffer0[group + i + 2]); + } + } + + // Stage 4 rotates the upper group by pi/8 while the completed lower group passes through unchanged. + for (int i = 0; i < 4; i++) + { + buffer1[i] = buffer0[i]; + } + + buffer1[4] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[16], buffer0[4], cospi[48], buffer0[5], cosBit, in rounding); + buffer1[5] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[48], buffer0[4], -cospi[16], buffer0[5], cosBit, in rounding); + buffer1[6] = Av1ForwardTransformArithmetic>.HalfButterfly(-cospi[48], buffer0[6], cospi[16], buffer0[7], cosBit, in rounding); + buffer1[7] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[16], buffer0[6], cospi[48], buffer0[7], cosBit, in rounding); + + // Stage 5 creates the four final butterfly pairs spanning the two groups. + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[i], buffer1[i + 4], out buffer0[i], out buffer0[i + 4]); + } + + // Stage 6 applies the remaining odd-angle rotations. + buffer1[0] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[4], buffer0[0], cospi[60], buffer0[1], cosBit, in rounding); + buffer1[1] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[60], buffer0[0], -cospi[4], buffer0[1], cosBit, in rounding); + buffer1[2] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[20], buffer0[2], cospi[44], buffer0[3], cosBit, in rounding); + buffer1[3] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[44], buffer0[2], -cospi[20], buffer0[3], cosBit, in rounding); + buffer1[4] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[36], buffer0[4], cospi[28], buffer0[5], cosBit, in rounding); + buffer1[5] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[28], buffer0[4], -cospi[36], buffer0[5], cosBit, in rounding); + buffer1[6] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[52], buffer0[6], cospi[12], buffer0[7], cosBit, in rounding); + buffer1[7] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[12], buffer0[6], -cospi[52], buffer0[7], cosBit, in rounding); + + ReadOnlySpan outputOrder = OutputOrder; + + // Stage 7 maps the rotated values to ascending AV1 ADST coefficient order. + for (int i = 0; i < 8; i++) + { + Store(ref values, outputStride, i, buffer1[outputOrder[i]]); + } + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + Av1TransformRounding rounding = Av1ForwardTransformArithmetic>.CreateRounding(cosBit); + + // Stage 1 applies the ADST permutation and signs while the source block is still read-only. + buffer0[0] = Load>(ref values, inputStride, 0); + buffer0[1] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 7)); + buffer0[2] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 3)); + buffer0[3] = Load>(ref values, inputStride, 4); + buffer0[4] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 1)); + buffer0[5] = Load>(ref values, inputStride, 6); + buffer0[6] = Load>(ref values, inputStride, 2); + buffer0[7] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 5)); + + // Stage 2 rotates the second pair in each four-value group while copying the already aligned pairs. + buffer1[0] = buffer0[0]; + buffer1[1] = buffer0[1]; + Butterfly(cospi[32], cospi[32], buffer0[2], buffer0[3], ref buffer1, 2, 3, cosBit, in rounding); + buffer1[4] = buffer0[4]; + buffer1[5] = buffer0[5]; + Butterfly(cospi[32], cospi[32], buffer0[6], buffer0[7], ref buffer1, 6, 7, cosBit, in rounding); + + // Stage 3 combines the rotated and copied pairs into two independent four-value groups. + for (int group = 0; group < 8; group += 4) + { + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract( + buffer1[group + i], + buffer1[group + i + 2], + out buffer0[group + i], + out buffer0[group + i + 2]); + } + } + + // Stage 4 rotates the upper group by pi/8 while the completed lower group passes through unchanged. + for (int i = 0; i < 4; i++) + { + buffer1[i] = buffer0[i]; + } + + buffer1[4] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[16], buffer0[4], cospi[48], buffer0[5], cosBit, in rounding); + buffer1[5] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[48], buffer0[4], -cospi[16], buffer0[5], cosBit, in rounding); + buffer1[6] = Av1ForwardTransformArithmetic>.HalfButterfly(-cospi[48], buffer0[6], cospi[16], buffer0[7], cosBit, in rounding); + buffer1[7] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[16], buffer0[6], cospi[48], buffer0[7], cosBit, in rounding); + + // Stage 5 creates the four final butterfly pairs spanning the two groups. + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[i], buffer1[i + 4], out buffer0[i], out buffer0[i + 4]); + } + + // Stage 6 applies the remaining odd-angle rotations. + buffer1[0] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[4], buffer0[0], cospi[60], buffer0[1], cosBit, in rounding); + buffer1[1] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[60], buffer0[0], -cospi[4], buffer0[1], cosBit, in rounding); + buffer1[2] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[20], buffer0[2], cospi[44], buffer0[3], cosBit, in rounding); + buffer1[3] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[44], buffer0[2], -cospi[20], buffer0[3], cosBit, in rounding); + buffer1[4] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[36], buffer0[4], cospi[28], buffer0[5], cosBit, in rounding); + buffer1[5] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[28], buffer0[4], -cospi[36], buffer0[5], cosBit, in rounding); + buffer1[6] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[52], buffer0[6], cospi[12], buffer0[7], cosBit, in rounding); + buffer1[7] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[12], buffer0[6], -cospi[52], buffer0[7], cosBit, in rounding); + + ReadOnlySpan outputOrder = OutputOrder; + + // Stage 7 maps the rotated values to ascending AV1 ADST coefficient order. + for (int i = 0; i < 8; i++) + { + Store(ref values, outputStride, i, buffer1[outputOrder[i]]); + } + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + Av1TransformRounding rounding = Av1ForwardTransformArithmetic>.CreateRounding(cosBit); + + // Stage 1 applies the ADST permutation and signs while the source block is still read-only. + buffer0[0] = Load>(ref values, inputStride, 0); + buffer0[1] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 7)); + buffer0[2] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 3)); + buffer0[3] = Load>(ref values, inputStride, 4); + buffer0[4] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 1)); + buffer0[5] = Load>(ref values, inputStride, 6); + buffer0[6] = Load>(ref values, inputStride, 2); + buffer0[7] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 5)); + + // Stage 2 rotates the second pair in each four-value group while copying the already aligned pairs. + buffer1[0] = buffer0[0]; + buffer1[1] = buffer0[1]; + Butterfly(cospi[32], cospi[32], buffer0[2], buffer0[3], ref buffer1, 2, 3, cosBit, in rounding); + buffer1[4] = buffer0[4]; + buffer1[5] = buffer0[5]; + Butterfly(cospi[32], cospi[32], buffer0[6], buffer0[7], ref buffer1, 6, 7, cosBit, in rounding); + + // Stage 3 combines the rotated and copied pairs into two independent four-value groups. + for (int group = 0; group < 8; group += 4) + { + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract( + buffer1[group + i], + buffer1[group + i + 2], + out buffer0[group + i], + out buffer0[group + i + 2]); + } + } + + // Stage 4 rotates the upper group by pi/8 while the completed lower group passes through unchanged. + for (int i = 0; i < 4; i++) + { + buffer1[i] = buffer0[i]; + } + + buffer1[4] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[16], buffer0[4], cospi[48], buffer0[5], cosBit, in rounding); + buffer1[5] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[48], buffer0[4], -cospi[16], buffer0[5], cosBit, in rounding); + buffer1[6] = Av1ForwardTransformArithmetic>.HalfButterfly(-cospi[48], buffer0[6], cospi[16], buffer0[7], cosBit, in rounding); + buffer1[7] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[16], buffer0[6], cospi[48], buffer0[7], cosBit, in rounding); + + // Stage 5 creates the four final butterfly pairs spanning the two groups. + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[i], buffer1[i + 4], out buffer0[i], out buffer0[i + 4]); + } + + // Stage 6 applies the remaining odd-angle rotations. + buffer1[0] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[4], buffer0[0], cospi[60], buffer0[1], cosBit, in rounding); + buffer1[1] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[60], buffer0[0], -cospi[4], buffer0[1], cosBit, in rounding); + buffer1[2] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[20], buffer0[2], cospi[44], buffer0[3], cosBit, in rounding); + buffer1[3] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[44], buffer0[2], -cospi[20], buffer0[3], cosBit, in rounding); + buffer1[4] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[36], buffer0[4], cospi[28], buffer0[5], cosBit, in rounding); + buffer1[5] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[28], buffer0[4], -cospi[36], buffer0[5], cosBit, in rounding); + buffer1[6] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[52], buffer0[6], cospi[12], buffer0[7], cosBit, in rounding); + buffer1[7] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[12], buffer0[6], -cospi[52], buffer0[7], cosBit, in rounding); + + ReadOnlySpan outputOrder = OutputOrder; + + // Stage 7 maps the rotated values to ascending AV1 ADST coefficient order. + for (int i = 0; i < 8; i++) + { + Store(ref values, outputStride, i, buffer1[outputOrder[i]]); + } + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + Av1TransformRounding rounding = Av1ForwardTransformArithmetic>.CreateRounding(cosBit); + + // Stage 1 applies the ADST permutation and signs while the source block is still read-only. + buffer0[0] = Load>(ref values, inputStride, 0); + buffer0[1] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 7)); + buffer0[2] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 3)); + buffer0[3] = Load>(ref values, inputStride, 4); + buffer0[4] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 1)); + buffer0[5] = Load>(ref values, inputStride, 6); + buffer0[6] = Load>(ref values, inputStride, 2); + buffer0[7] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 5)); + + // Stage 2 rotates the second pair in each four-value group while copying the already aligned pairs. + buffer1[0] = buffer0[0]; + buffer1[1] = buffer0[1]; + Butterfly(cospi[32], cospi[32], buffer0[2], buffer0[3], ref buffer1, 2, 3, cosBit, in rounding); + buffer1[4] = buffer0[4]; + buffer1[5] = buffer0[5]; + Butterfly(cospi[32], cospi[32], buffer0[6], buffer0[7], ref buffer1, 6, 7, cosBit, in rounding); + + // Stage 3 combines the rotated and copied pairs into two independent four-value groups. + for (int group = 0; group < 8; group += 4) + { + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract( + buffer1[group + i], + buffer1[group + i + 2], + out buffer0[group + i], + out buffer0[group + i + 2]); + } + } + + // Stage 4 rotates the upper group by pi/8 while the completed lower group passes through unchanged. + for (int i = 0; i < 4; i++) + { + buffer1[i] = buffer0[i]; + } + + buffer1[4] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[16], buffer0[4], cospi[48], buffer0[5], cosBit, in rounding); + buffer1[5] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[48], buffer0[4], -cospi[16], buffer0[5], cosBit, in rounding); + buffer1[6] = Av1ForwardTransformArithmetic>.HalfButterfly(-cospi[48], buffer0[6], cospi[16], buffer0[7], cosBit, in rounding); + buffer1[7] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[16], buffer0[6], cospi[48], buffer0[7], cosBit, in rounding); + + // Stage 5 creates the four final butterfly pairs spanning the two groups. + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[i], buffer1[i + 4], out buffer0[i], out buffer0[i + 4]); + } + + // Stage 6 applies the remaining odd-angle rotations. + buffer1[0] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[4], buffer0[0], cospi[60], buffer0[1], cosBit, in rounding); + buffer1[1] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[60], buffer0[0], -cospi[4], buffer0[1], cosBit, in rounding); + buffer1[2] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[20], buffer0[2], cospi[44], buffer0[3], cosBit, in rounding); + buffer1[3] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[44], buffer0[2], -cospi[20], buffer0[3], cosBit, in rounding); + buffer1[4] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[36], buffer0[4], cospi[28], buffer0[5], cosBit, in rounding); + buffer1[5] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[28], buffer0[4], -cospi[36], buffer0[5], cosBit, in rounding); + buffer1[6] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[52], buffer0[6], cospi[12], buffer0[7], cosBit, in rounding); + buffer1[7] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[12], buffer0[6], -cospi[52], buffer0[7], cosBit, in rounding); + + ReadOnlySpan outputOrder = OutputOrder; + + // Stage 7 maps the rotated values to ascending AV1 ADST coefficient order. + for (int i = 0; i < 8; i++) + { + Store(ref values, outputStride, i, buffer1[outputOrder[i]]); + } + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + Av1TransformRounding rounding = Av1ForwardTransformArithmetic>.CreateRounding(cosBit); + + // Stage 1 applies the ADST permutation and signs while the source block is still read-only. + buffer0[0] = Load>(ref values, inputStride, 0); + buffer0[1] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 7)); + buffer0[2] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 3)); + buffer0[3] = Load>(ref values, inputStride, 4); + buffer0[4] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 1)); + buffer0[5] = Load>(ref values, inputStride, 6); + buffer0[6] = Load>(ref values, inputStride, 2); + buffer0[7] = Av1ForwardTransformArithmetic>.Negate(Load>(ref values, inputStride, 5)); + + // Stage 2 rotates the second pair in each four-value group while copying the already aligned pairs. + buffer1[0] = buffer0[0]; + buffer1[1] = buffer0[1]; + Butterfly(cospi[32], cospi[32], buffer0[2], buffer0[3], ref buffer1, 2, 3, cosBit, in rounding); + buffer1[4] = buffer0[4]; + buffer1[5] = buffer0[5]; + Butterfly(cospi[32], cospi[32], buffer0[6], buffer0[7], ref buffer1, 6, 7, cosBit, in rounding); + + // Stage 3 combines the rotated and copied pairs into two independent four-value groups. + for (int group = 0; group < 8; group += 4) + { + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract( + buffer1[group + i], + buffer1[group + i + 2], + out buffer0[group + i], + out buffer0[group + i + 2]); + } + } + + // Stage 4 rotates the upper group by pi/8 while the completed lower group passes through unchanged. + for (int i = 0; i < 4; i++) + { + buffer1[i] = buffer0[i]; + } + + buffer1[4] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[16], buffer0[4], cospi[48], buffer0[5], cosBit, in rounding); + buffer1[5] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[48], buffer0[4], -cospi[16], buffer0[5], cosBit, in rounding); + buffer1[6] = Av1ForwardTransformArithmetic>.HalfButterfly(-cospi[48], buffer0[6], cospi[16], buffer0[7], cosBit, in rounding); + buffer1[7] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[16], buffer0[6], cospi[48], buffer0[7], cosBit, in rounding); + + // Stage 5 creates the four final butterfly pairs spanning the two groups. + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[i], buffer1[i + 4], out buffer0[i], out buffer0[i + 4]); + } + + // Stage 6 applies the remaining odd-angle rotations. + buffer1[0] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[4], buffer0[0], cospi[60], buffer0[1], cosBit, in rounding); + buffer1[1] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[60], buffer0[0], -cospi[4], buffer0[1], cosBit, in rounding); + buffer1[2] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[20], buffer0[2], cospi[44], buffer0[3], cosBit, in rounding); + buffer1[3] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[44], buffer0[2], -cospi[20], buffer0[3], cosBit, in rounding); + buffer1[4] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[36], buffer0[4], cospi[28], buffer0[5], cosBit, in rounding); + buffer1[5] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[28], buffer0[4], -cospi[36], buffer0[5], cosBit, in rounding); + buffer1[6] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[52], buffer0[6], cospi[12], buffer0[7], cosBit, in rounding); + buffer1[7] = Av1ForwardTransformArithmetic>.HalfButterfly(cospi[12], buffer0[6], -cospi[52], buffer0[7], cosBit, in rounding); + + ReadOnlySpan outputOrder = OutputOrder; + + // Stage 7 maps the rotated values to ascending AV1 ADST coefficient order. + for (int i = 0; i < 8; i++) + { + Store(ref values, outputStride, i, buffer1[outputOrder[i]]); + } + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Transform/Forward/Av1ForwardTransformer.Dct16Operator.cs b/src/ImageSharp/Formats/Heif/Av1/Transform/Forward/Av1ForwardTransformer.Dct16Operator.cs new file mode 100644 index 0000000000..ce7497096d --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Transform/Forward/Av1ForwardTransformer.Dct16Operator.cs @@ -0,0 +1,1603 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.Intrinsics; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform.Forward; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +/// +/// Implements the sixteen-point forward DCT stage network. +/// +internal static partial class Av1ForwardTransformer +{ + /// + /// Implements the sixteen-point forward transform for every supported lane width. + /// + internal readonly struct Dct16Operator : IAv1ForwardTransform1dOperator + { + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector buffer0, + ref Av1TransformVector buffer1, + int cosBit) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + Av1TransformRounding rounding = Av1ForwardTransformArithmetic.CreateRounding(cosBit); + + // Stage 1 forms the mirror-symmetric pairs consumed by the recursive even and odd factorizations. + for (int i = 0; i < 8; i++) + { + Av1ForwardTransformArithmetic.AddSubtract( + Load(ref values, inputStride, i), + Load(ref values, inputStride, 15 - i), + out buffer0[i], + out buffer0[15 - i]); + } + + // Stage 2 begins the recursive factorization of the even half and rotates the central odd pairs by pi/4. + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic.AddSubtract(buffer0[i], buffer0[7 - i], out buffer1[i], out buffer1[7 - i]); + } + + Av1ForwardTransformArithmetic.Butterfly( + -cospi[32], + cospi[32], + buffer0[10], + buffer0[13], + out buffer1[10], + out buffer1[13], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic.Butterfly( + -cospi[32], + cospi[32], + buffer0[11], + buffer0[12], + out buffer1[11], + out buffer1[12], + cosBit, + in rounding); + + // Stage 3 reduces both eight-value groups into the four-value units consumed by the terminal rotations. + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic.AddSubtract(buffer1[i], buffer1[3 - i], out buffer0[i], out buffer0[3 - i]); + } + + Av1ForwardTransformArithmetic.Butterfly( + -cospi[32], + cospi[32], + buffer1[5], + buffer1[6], + out buffer0[5], + out buffer0[6], + cosBit, + in rounding); + + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic.AddSubtract(buffer0[8 + i], buffer1[11 - i], out buffer0[8 + i], out buffer0[11 - i]); + Av1ForwardTransformArithmetic.AddSubtract(buffer0[15 - i], buffer1[12 + i], out buffer0[15 - i], out buffer0[12 + i]); + } + + // The even coefficients become final at stages 4 and 5, so they are written directly to their AV1 order. + Av1ForwardTransformArithmetic.Butterfly( + cospi[32], + cospi[32], + buffer0[0], + buffer0[1], + out int output0, + out int output8, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic.Butterfly( + cospi[16], + cospi[48], + buffer0[3], + buffer0[2], + out int output4, + out int output12, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic.AddSubtract(buffer1[4], buffer0[5], out buffer1[4], out buffer1[5]); + Av1ForwardTransformArithmetic.AddSubtract(buffer1[7], buffer0[6], out buffer1[7], out buffer1[6]); + Av1ForwardTransformArithmetic.Butterfly( + -cospi[16], + cospi[48], + buffer0[9], + buffer0[14], + out buffer1[9], + out buffer1[14], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic.Butterfly( + -cospi[48], + -cospi[16], + buffer0[10], + buffer0[13], + out buffer1[10], + out buffer1[13], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic.Butterfly( + cospi[8], + cospi[56], + buffer1[7], + buffer1[4], + out int output2, + out int output14, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic.Butterfly( + cospi[40], + cospi[24], + buffer1[6], + buffer1[5], + out int output10, + out int output6, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic.AddSubtract(buffer0[8], buffer1[9], out buffer0[8], out buffer0[9]); + Av1ForwardTransformArithmetic.AddSubtract(buffer0[11], buffer1[10], out buffer0[11], out buffer0[10]); + Av1ForwardTransformArithmetic.AddSubtract(buffer0[12], buffer1[13], out buffer0[12], out buffer0[13]); + Av1ForwardTransformArithmetic.AddSubtract(buffer0[15], buffer1[14], out buffer0[15], out buffer0[14]); + + // Stage 6 applies the final pi/32 odd-frequency rotations. The following stores perform only the normative + // coefficient permutation, so each rotation result is named by its final destination. + Av1ForwardTransformArithmetic.Butterfly( + cospi[4], + cospi[60], + buffer0[15], + buffer0[8], + out int output1, + out int output15, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic.Butterfly( + cospi[36], + cospi[28], + buffer0[14], + buffer0[9], + out int output9, + out int output7, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic.Butterfly( + cospi[20], + cospi[44], + buffer0[13], + buffer0[10], + out int output5, + out int output11, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic.Butterfly( + cospi[52], + cospi[12], + buffer0[12], + buffer0[11], + out int output13, + out int output3, + cosBit, + in rounding); + + Store(ref values, outputStride, 0, output0); + Store(ref values, outputStride, 1, output1); + Store(ref values, outputStride, 2, output2); + Store(ref values, outputStride, 3, output3); + Store(ref values, outputStride, 4, output4); + Store(ref values, outputStride, 5, output5); + Store(ref values, outputStride, 6, output6); + Store(ref values, outputStride, 7, output7); + Store(ref values, outputStride, 8, output8); + Store(ref values, outputStride, 9, output9); + Store(ref values, outputStride, 10, output10); + Store(ref values, outputStride, 11, output11); + Store(ref values, outputStride, 12, output12); + Store(ref values, outputStride, 13, output13); + Store(ref values, outputStride, 14, output14); + Store(ref values, outputStride, 15, output15); + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector buffer0, + ref Av1TransformVector buffer1, + int cosBit) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + Av1TransformRounding rounding = Av1ForwardTransformArithmetic.CreateRounding(cosBit); + + // Stage 1 forms the mirror-symmetric pairs consumed by the recursive even and odd factorizations. + for (int i = 0; i < 8; i++) + { + Av1ForwardTransformArithmetic.AddSubtract( + Load(ref values, inputStride, i), + Load(ref values, inputStride, 15 - i), + out buffer0[i], + out buffer0[15 - i]); + } + + // Stage 2 begins the recursive factorization of the even half and rotates the central odd pairs by pi/4. + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic.AddSubtract(buffer0[i], buffer0[7 - i], out buffer1[i], out buffer1[7 - i]); + } + + Av1ForwardTransformArithmetic.Butterfly( + -cospi[32], + cospi[32], + buffer0[10], + buffer0[13], + out buffer1[10], + out buffer1[13], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic.Butterfly( + -cospi[32], + cospi[32], + buffer0[11], + buffer0[12], + out buffer1[11], + out buffer1[12], + cosBit, + in rounding); + + // Stage 3 reduces both eight-value groups into the four-value units consumed by the terminal rotations. + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic.AddSubtract(buffer1[i], buffer1[3 - i], out buffer0[i], out buffer0[3 - i]); + } + + Av1ForwardTransformArithmetic.Butterfly( + -cospi[32], + cospi[32], + buffer1[5], + buffer1[6], + out buffer0[5], + out buffer0[6], + cosBit, + in rounding); + + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic.AddSubtract(buffer0[8 + i], buffer1[11 - i], out buffer0[8 + i], out buffer0[11 - i]); + Av1ForwardTransformArithmetic.AddSubtract(buffer0[15 - i], buffer1[12 + i], out buffer0[15 - i], out buffer0[12 + i]); + } + + // The even coefficients become final at stages 4 and 5, so they are written directly to their AV1 order. + Av1ForwardTransformArithmetic.Butterfly( + cospi[32], + cospi[32], + buffer0[0], + buffer0[1], + out short output0, + out short output8, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic.Butterfly( + cospi[16], + cospi[48], + buffer0[3], + buffer0[2], + out short output4, + out short output12, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic.AddSubtract(buffer1[4], buffer0[5], out buffer1[4], out buffer1[5]); + Av1ForwardTransformArithmetic.AddSubtract(buffer1[7], buffer0[6], out buffer1[7], out buffer1[6]); + Av1ForwardTransformArithmetic.Butterfly( + -cospi[16], + cospi[48], + buffer0[9], + buffer0[14], + out buffer1[9], + out buffer1[14], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic.Butterfly( + -cospi[48], + -cospi[16], + buffer0[10], + buffer0[13], + out buffer1[10], + out buffer1[13], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic.Butterfly( + cospi[8], + cospi[56], + buffer1[7], + buffer1[4], + out short output2, + out short output14, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic.Butterfly( + cospi[40], + cospi[24], + buffer1[6], + buffer1[5], + out short output10, + out short output6, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic.AddSubtract(buffer0[8], buffer1[9], out buffer0[8], out buffer0[9]); + Av1ForwardTransformArithmetic.AddSubtract(buffer0[11], buffer1[10], out buffer0[11], out buffer0[10]); + Av1ForwardTransformArithmetic.AddSubtract(buffer0[12], buffer1[13], out buffer0[12], out buffer0[13]); + Av1ForwardTransformArithmetic.AddSubtract(buffer0[15], buffer1[14], out buffer0[15], out buffer0[14]); + + // Stage 6 applies the final pi/32 odd-frequency rotations. The following stores perform only the normative + // coefficient permutation, so each rotation result is named by its final destination. + Av1ForwardTransformArithmetic.Butterfly( + cospi[4], + cospi[60], + buffer0[15], + buffer0[8], + out short output1, + out short output15, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic.Butterfly( + cospi[36], + cospi[28], + buffer0[14], + buffer0[9], + out short output9, + out short output7, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic.Butterfly( + cospi[20], + cospi[44], + buffer0[13], + buffer0[10], + out short output5, + out short output11, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic.Butterfly( + cospi[52], + cospi[12], + buffer0[12], + buffer0[11], + out short output13, + out short output3, + cosBit, + in rounding); + + Store(ref values, outputStride, 0, output0); + Store(ref values, outputStride, 1, output1); + Store(ref values, outputStride, 2, output2); + Store(ref values, outputStride, 3, output3); + Store(ref values, outputStride, 4, output4); + Store(ref values, outputStride, 5, output5); + Store(ref values, outputStride, 6, output6); + Store(ref values, outputStride, 7, output7); + Store(ref values, outputStride, 8, output8); + Store(ref values, outputStride, 9, output9); + Store(ref values, outputStride, 10, output10); + Store(ref values, outputStride, 11, output11); + Store(ref values, outputStride, 12, output12); + Store(ref values, outputStride, 13, output13); + Store(ref values, outputStride, 14, output14); + Store(ref values, outputStride, 15, output15); + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + Av1TransformRounding rounding = Av1ForwardTransformArithmetic>.CreateRounding(cosBit); + + // Stage 1 forms the mirror-symmetric pairs consumed by the recursive even and odd factorizations. + for (int i = 0; i < 8; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract( + Load>(ref values, inputStride, i), + Load>(ref values, inputStride, 15 - i), + out buffer0[i], + out buffer0[15 - i]); + } + + // Stage 2 begins the recursive factorization of the even half and rotates the central odd pairs by pi/4. + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[i], buffer0[7 - i], out buffer1[i], out buffer1[7 - i]); + } + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[32], + cospi[32], + buffer0[10], + buffer0[13], + out buffer1[10], + out buffer1[13], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[32], + cospi[32], + buffer0[11], + buffer0[12], + out buffer1[11], + out buffer1[12], + cosBit, + in rounding); + + // Stage 3 reduces both eight-value groups into the four-value units consumed by the terminal rotations. + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[i], buffer1[3 - i], out buffer0[i], out buffer0[3 - i]); + } + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[32], + cospi[32], + buffer1[5], + buffer1[6], + out buffer0[5], + out buffer0[6], + cosBit, + in rounding); + + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[8 + i], buffer1[11 - i], out buffer0[8 + i], out buffer0[11 - i]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[15 - i], buffer1[12 + i], out buffer0[15 - i], out buffer0[12 + i]); + } + + // The even coefficients become final at stages 4 and 5, so they are written directly to their AV1 order. + Av1ForwardTransformArithmetic>.Butterfly( + cospi[32], + cospi[32], + buffer0[0], + buffer0[1], + out Vector128 output0, + out Vector128 output8, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + cospi[16], + cospi[48], + buffer0[3], + buffer0[2], + out Vector128 output4, + out Vector128 output12, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[4], buffer0[5], out buffer1[4], out buffer1[5]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[7], buffer0[6], out buffer1[7], out buffer1[6]); + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[16], + cospi[48], + buffer0[9], + buffer0[14], + out buffer1[9], + out buffer1[14], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[48], + -cospi[16], + buffer0[10], + buffer0[13], + out buffer1[10], + out buffer1[13], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + cospi[8], + cospi[56], + buffer1[7], + buffer1[4], + out Vector128 output2, + out Vector128 output14, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + cospi[40], + cospi[24], + buffer1[6], + buffer1[5], + out Vector128 output10, + out Vector128 output6, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[8], buffer1[9], out buffer0[8], out buffer0[9]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[11], buffer1[10], out buffer0[11], out buffer0[10]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[12], buffer1[13], out buffer0[12], out buffer0[13]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[15], buffer1[14], out buffer0[15], out buffer0[14]); + + // Stage 6 applies the final pi/32 odd-frequency rotations. The following stores perform only the normative + // coefficient permutation, so each rotation result is named by its final destination. + Av1ForwardTransformArithmetic>.Butterfly( + cospi[4], + cospi[60], + buffer0[15], + buffer0[8], + out Vector128 output1, + out Vector128 output15, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + cospi[36], + cospi[28], + buffer0[14], + buffer0[9], + out Vector128 output9, + out Vector128 output7, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + cospi[20], + cospi[44], + buffer0[13], + buffer0[10], + out Vector128 output5, + out Vector128 output11, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + cospi[52], + cospi[12], + buffer0[12], + buffer0[11], + out Vector128 output13, + out Vector128 output3, + cosBit, + in rounding); + + Store(ref values, outputStride, 0, output0); + Store(ref values, outputStride, 1, output1); + Store(ref values, outputStride, 2, output2); + Store(ref values, outputStride, 3, output3); + Store(ref values, outputStride, 4, output4); + Store(ref values, outputStride, 5, output5); + Store(ref values, outputStride, 6, output6); + Store(ref values, outputStride, 7, output7); + Store(ref values, outputStride, 8, output8); + Store(ref values, outputStride, 9, output9); + Store(ref values, outputStride, 10, output10); + Store(ref values, outputStride, 11, output11); + Store(ref values, outputStride, 12, output12); + Store(ref values, outputStride, 13, output13); + Store(ref values, outputStride, 14, output14); + Store(ref values, outputStride, 15, output15); + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + Av1TransformRounding rounding = Av1ForwardTransformArithmetic>.CreateRounding(cosBit); + + // Stage 1 forms the mirror-symmetric pairs consumed by the recursive even and odd factorizations. + for (int i = 0; i < 8; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract( + Load>(ref values, inputStride, i), + Load>(ref values, inputStride, 15 - i), + out buffer0[i], + out buffer0[15 - i]); + } + + // Stage 2 begins the recursive factorization of the even half and rotates the central odd pairs by pi/4. + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[i], buffer0[7 - i], out buffer1[i], out buffer1[7 - i]); + } + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[32], + cospi[32], + buffer0[10], + buffer0[13], + out buffer1[10], + out buffer1[13], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[32], + cospi[32], + buffer0[11], + buffer0[12], + out buffer1[11], + out buffer1[12], + cosBit, + in rounding); + + // Stage 3 reduces both eight-value groups into the four-value units consumed by the terminal rotations. + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[i], buffer1[3 - i], out buffer0[i], out buffer0[3 - i]); + } + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[32], + cospi[32], + buffer1[5], + buffer1[6], + out buffer0[5], + out buffer0[6], + cosBit, + in rounding); + + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[8 + i], buffer1[11 - i], out buffer0[8 + i], out buffer0[11 - i]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[15 - i], buffer1[12 + i], out buffer0[15 - i], out buffer0[12 + i]); + } + + // The even coefficients become final at stages 4 and 5, so they are written directly to their AV1 order. + Av1ForwardTransformArithmetic>.Butterfly( + cospi[32], + cospi[32], + buffer0[0], + buffer0[1], + out Vector256 output0, + out Vector256 output8, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + cospi[16], + cospi[48], + buffer0[3], + buffer0[2], + out Vector256 output4, + out Vector256 output12, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[4], buffer0[5], out buffer1[4], out buffer1[5]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[7], buffer0[6], out buffer1[7], out buffer1[6]); + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[16], + cospi[48], + buffer0[9], + buffer0[14], + out buffer1[9], + out buffer1[14], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[48], + -cospi[16], + buffer0[10], + buffer0[13], + out buffer1[10], + out buffer1[13], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + cospi[8], + cospi[56], + buffer1[7], + buffer1[4], + out Vector256 output2, + out Vector256 output14, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + cospi[40], + cospi[24], + buffer1[6], + buffer1[5], + out Vector256 output10, + out Vector256 output6, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[8], buffer1[9], out buffer0[8], out buffer0[9]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[11], buffer1[10], out buffer0[11], out buffer0[10]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[12], buffer1[13], out buffer0[12], out buffer0[13]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[15], buffer1[14], out buffer0[15], out buffer0[14]); + + // Stage 6 applies the final pi/32 odd-frequency rotations. The following stores perform only the normative + // coefficient permutation, so each rotation result is named by its final destination. + Av1ForwardTransformArithmetic>.Butterfly( + cospi[4], + cospi[60], + buffer0[15], + buffer0[8], + out Vector256 output1, + out Vector256 output15, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + cospi[36], + cospi[28], + buffer0[14], + buffer0[9], + out Vector256 output9, + out Vector256 output7, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + cospi[20], + cospi[44], + buffer0[13], + buffer0[10], + out Vector256 output5, + out Vector256 output11, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + cospi[52], + cospi[12], + buffer0[12], + buffer0[11], + out Vector256 output13, + out Vector256 output3, + cosBit, + in rounding); + + Store(ref values, outputStride, 0, output0); + Store(ref values, outputStride, 1, output1); + Store(ref values, outputStride, 2, output2); + Store(ref values, outputStride, 3, output3); + Store(ref values, outputStride, 4, output4); + Store(ref values, outputStride, 5, output5); + Store(ref values, outputStride, 6, output6); + Store(ref values, outputStride, 7, output7); + Store(ref values, outputStride, 8, output8); + Store(ref values, outputStride, 9, output9); + Store(ref values, outputStride, 10, output10); + Store(ref values, outputStride, 11, output11); + Store(ref values, outputStride, 12, output12); + Store(ref values, outputStride, 13, output13); + Store(ref values, outputStride, 14, output14); + Store(ref values, outputStride, 15, output15); + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + Av1TransformRounding rounding = Av1ForwardTransformArithmetic>.CreateRounding(cosBit); + + // Stage 1 forms the mirror-symmetric pairs consumed by the recursive even and odd factorizations. + for (int i = 0; i < 8; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract( + Load>(ref values, inputStride, i), + Load>(ref values, inputStride, 15 - i), + out buffer0[i], + out buffer0[15 - i]); + } + + // Stage 2 begins the recursive factorization of the even half and rotates the central odd pairs by pi/4. + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[i], buffer0[7 - i], out buffer1[i], out buffer1[7 - i]); + } + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[32], + cospi[32], + buffer0[10], + buffer0[13], + out buffer1[10], + out buffer1[13], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[32], + cospi[32], + buffer0[11], + buffer0[12], + out buffer1[11], + out buffer1[12], + cosBit, + in rounding); + + // Stage 3 reduces both eight-value groups into the four-value units consumed by the terminal rotations. + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[i], buffer1[3 - i], out buffer0[i], out buffer0[3 - i]); + } + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[32], + cospi[32], + buffer1[5], + buffer1[6], + out buffer0[5], + out buffer0[6], + cosBit, + in rounding); + + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[8 + i], buffer1[11 - i], out buffer0[8 + i], out buffer0[11 - i]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[15 - i], buffer1[12 + i], out buffer0[15 - i], out buffer0[12 + i]); + } + + // The even coefficients become final at stages 4 and 5, so they are written directly to their AV1 order. + Av1ForwardTransformArithmetic>.Butterfly( + cospi[32], + cospi[32], + buffer0[0], + buffer0[1], + out Vector512 output0, + out Vector512 output8, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + cospi[16], + cospi[48], + buffer0[3], + buffer0[2], + out Vector512 output4, + out Vector512 output12, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[4], buffer0[5], out buffer1[4], out buffer1[5]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[7], buffer0[6], out buffer1[7], out buffer1[6]); + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[16], + cospi[48], + buffer0[9], + buffer0[14], + out buffer1[9], + out buffer1[14], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[48], + -cospi[16], + buffer0[10], + buffer0[13], + out buffer1[10], + out buffer1[13], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + cospi[8], + cospi[56], + buffer1[7], + buffer1[4], + out Vector512 output2, + out Vector512 output14, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + cospi[40], + cospi[24], + buffer1[6], + buffer1[5], + out Vector512 output10, + out Vector512 output6, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[8], buffer1[9], out buffer0[8], out buffer0[9]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[11], buffer1[10], out buffer0[11], out buffer0[10]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[12], buffer1[13], out buffer0[12], out buffer0[13]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[15], buffer1[14], out buffer0[15], out buffer0[14]); + + // Stage 6 applies the final pi/32 odd-frequency rotations. The following stores perform only the normative + // coefficient permutation, so each rotation result is named by its final destination. + Av1ForwardTransformArithmetic>.Butterfly( + cospi[4], + cospi[60], + buffer0[15], + buffer0[8], + out Vector512 output1, + out Vector512 output15, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + cospi[36], + cospi[28], + buffer0[14], + buffer0[9], + out Vector512 output9, + out Vector512 output7, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + cospi[20], + cospi[44], + buffer0[13], + buffer0[10], + out Vector512 output5, + out Vector512 output11, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + cospi[52], + cospi[12], + buffer0[12], + buffer0[11], + out Vector512 output13, + out Vector512 output3, + cosBit, + in rounding); + + Store(ref values, outputStride, 0, output0); + Store(ref values, outputStride, 1, output1); + Store(ref values, outputStride, 2, output2); + Store(ref values, outputStride, 3, output3); + Store(ref values, outputStride, 4, output4); + Store(ref values, outputStride, 5, output5); + Store(ref values, outputStride, 6, output6); + Store(ref values, outputStride, 7, output7); + Store(ref values, outputStride, 8, output8); + Store(ref values, outputStride, 9, output9); + Store(ref values, outputStride, 10, output10); + Store(ref values, outputStride, 11, output11); + Store(ref values, outputStride, 12, output12); + Store(ref values, outputStride, 13, output13); + Store(ref values, outputStride, 14, output14); + Store(ref values, outputStride, 15, output15); + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + Av1TransformRounding rounding = Av1ForwardTransformArithmetic>.CreateRounding(cosBit); + + // Stage 1 forms the mirror-symmetric pairs consumed by the recursive even and odd factorizations. + for (int i = 0; i < 8; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract( + Load>(ref values, inputStride, i), + Load>(ref values, inputStride, 15 - i), + out buffer0[i], + out buffer0[15 - i]); + } + + // Stage 2 begins the recursive factorization of the even half and rotates the central odd pairs by pi/4. + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[i], buffer0[7 - i], out buffer1[i], out buffer1[7 - i]); + } + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[32], + cospi[32], + buffer0[10], + buffer0[13], + out buffer1[10], + out buffer1[13], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[32], + cospi[32], + buffer0[11], + buffer0[12], + out buffer1[11], + out buffer1[12], + cosBit, + in rounding); + + // Stage 3 reduces both eight-value groups into the four-value units consumed by the terminal rotations. + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[i], buffer1[3 - i], out buffer0[i], out buffer0[3 - i]); + } + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[32], + cospi[32], + buffer1[5], + buffer1[6], + out buffer0[5], + out buffer0[6], + cosBit, + in rounding); + + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[8 + i], buffer1[11 - i], out buffer0[8 + i], out buffer0[11 - i]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[15 - i], buffer1[12 + i], out buffer0[15 - i], out buffer0[12 + i]); + } + + // The even coefficients become final at stages 4 and 5, so they are written directly to their AV1 order. + Av1ForwardTransformArithmetic>.Butterfly( + cospi[32], + cospi[32], + buffer0[0], + buffer0[1], + out Vector128 output0, + out Vector128 output8, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + cospi[16], + cospi[48], + buffer0[3], + buffer0[2], + out Vector128 output4, + out Vector128 output12, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[4], buffer0[5], out buffer1[4], out buffer1[5]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[7], buffer0[6], out buffer1[7], out buffer1[6]); + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[16], + cospi[48], + buffer0[9], + buffer0[14], + out buffer1[9], + out buffer1[14], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[48], + -cospi[16], + buffer0[10], + buffer0[13], + out buffer1[10], + out buffer1[13], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + cospi[8], + cospi[56], + buffer1[7], + buffer1[4], + out Vector128 output2, + out Vector128 output14, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + cospi[40], + cospi[24], + buffer1[6], + buffer1[5], + out Vector128 output10, + out Vector128 output6, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[8], buffer1[9], out buffer0[8], out buffer0[9]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[11], buffer1[10], out buffer0[11], out buffer0[10]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[12], buffer1[13], out buffer0[12], out buffer0[13]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[15], buffer1[14], out buffer0[15], out buffer0[14]); + + // Stage 6 applies the final pi/32 odd-frequency rotations. The following stores perform only the normative + // coefficient permutation, so each rotation result is named by its final destination. + Av1ForwardTransformArithmetic>.Butterfly( + cospi[4], + cospi[60], + buffer0[15], + buffer0[8], + out Vector128 output1, + out Vector128 output15, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + cospi[36], + cospi[28], + buffer0[14], + buffer0[9], + out Vector128 output9, + out Vector128 output7, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + cospi[20], + cospi[44], + buffer0[13], + buffer0[10], + out Vector128 output5, + out Vector128 output11, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + cospi[52], + cospi[12], + buffer0[12], + buffer0[11], + out Vector128 output13, + out Vector128 output3, + cosBit, + in rounding); + + Store(ref values, outputStride, 0, output0); + Store(ref values, outputStride, 1, output1); + Store(ref values, outputStride, 2, output2); + Store(ref values, outputStride, 3, output3); + Store(ref values, outputStride, 4, output4); + Store(ref values, outputStride, 5, output5); + Store(ref values, outputStride, 6, output6); + Store(ref values, outputStride, 7, output7); + Store(ref values, outputStride, 8, output8); + Store(ref values, outputStride, 9, output9); + Store(ref values, outputStride, 10, output10); + Store(ref values, outputStride, 11, output11); + Store(ref values, outputStride, 12, output12); + Store(ref values, outputStride, 13, output13); + Store(ref values, outputStride, 14, output14); + Store(ref values, outputStride, 15, output15); + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + Av1TransformRounding rounding = Av1ForwardTransformArithmetic>.CreateRounding(cosBit); + + // Stage 1 forms the mirror-symmetric pairs consumed by the recursive even and odd factorizations. + for (int i = 0; i < 8; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract( + Load>(ref values, inputStride, i), + Load>(ref values, inputStride, 15 - i), + out buffer0[i], + out buffer0[15 - i]); + } + + // Stage 2 begins the recursive factorization of the even half and rotates the central odd pairs by pi/4. + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[i], buffer0[7 - i], out buffer1[i], out buffer1[7 - i]); + } + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[32], + cospi[32], + buffer0[10], + buffer0[13], + out buffer1[10], + out buffer1[13], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[32], + cospi[32], + buffer0[11], + buffer0[12], + out buffer1[11], + out buffer1[12], + cosBit, + in rounding); + + // Stage 3 reduces both eight-value groups into the four-value units consumed by the terminal rotations. + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[i], buffer1[3 - i], out buffer0[i], out buffer0[3 - i]); + } + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[32], + cospi[32], + buffer1[5], + buffer1[6], + out buffer0[5], + out buffer0[6], + cosBit, + in rounding); + + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[8 + i], buffer1[11 - i], out buffer0[8 + i], out buffer0[11 - i]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[15 - i], buffer1[12 + i], out buffer0[15 - i], out buffer0[12 + i]); + } + + // The even coefficients become final at stages 4 and 5, so they are written directly to their AV1 order. + Av1ForwardTransformArithmetic>.Butterfly( + cospi[32], + cospi[32], + buffer0[0], + buffer0[1], + out Vector256 output0, + out Vector256 output8, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + cospi[16], + cospi[48], + buffer0[3], + buffer0[2], + out Vector256 output4, + out Vector256 output12, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[4], buffer0[5], out buffer1[4], out buffer1[5]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[7], buffer0[6], out buffer1[7], out buffer1[6]); + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[16], + cospi[48], + buffer0[9], + buffer0[14], + out buffer1[9], + out buffer1[14], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[48], + -cospi[16], + buffer0[10], + buffer0[13], + out buffer1[10], + out buffer1[13], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + cospi[8], + cospi[56], + buffer1[7], + buffer1[4], + out Vector256 output2, + out Vector256 output14, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + cospi[40], + cospi[24], + buffer1[6], + buffer1[5], + out Vector256 output10, + out Vector256 output6, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[8], buffer1[9], out buffer0[8], out buffer0[9]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[11], buffer1[10], out buffer0[11], out buffer0[10]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[12], buffer1[13], out buffer0[12], out buffer0[13]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[15], buffer1[14], out buffer0[15], out buffer0[14]); + + // Stage 6 applies the final pi/32 odd-frequency rotations. The following stores perform only the normative + // coefficient permutation, so each rotation result is named by its final destination. + Av1ForwardTransformArithmetic>.Butterfly( + cospi[4], + cospi[60], + buffer0[15], + buffer0[8], + out Vector256 output1, + out Vector256 output15, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + cospi[36], + cospi[28], + buffer0[14], + buffer0[9], + out Vector256 output9, + out Vector256 output7, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + cospi[20], + cospi[44], + buffer0[13], + buffer0[10], + out Vector256 output5, + out Vector256 output11, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + cospi[52], + cospi[12], + buffer0[12], + buffer0[11], + out Vector256 output13, + out Vector256 output3, + cosBit, + in rounding); + + Store(ref values, outputStride, 0, output0); + Store(ref values, outputStride, 1, output1); + Store(ref values, outputStride, 2, output2); + Store(ref values, outputStride, 3, output3); + Store(ref values, outputStride, 4, output4); + Store(ref values, outputStride, 5, output5); + Store(ref values, outputStride, 6, output6); + Store(ref values, outputStride, 7, output7); + Store(ref values, outputStride, 8, output8); + Store(ref values, outputStride, 9, output9); + Store(ref values, outputStride, 10, output10); + Store(ref values, outputStride, 11, output11); + Store(ref values, outputStride, 12, output12); + Store(ref values, outputStride, 13, output13); + Store(ref values, outputStride, 14, output14); + Store(ref values, outputStride, 15, output15); + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + Av1TransformRounding rounding = Av1ForwardTransformArithmetic>.CreateRounding(cosBit); + + // Stage 1 forms the mirror-symmetric pairs consumed by the recursive even and odd factorizations. + for (int i = 0; i < 8; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract( + Load>(ref values, inputStride, i), + Load>(ref values, inputStride, 15 - i), + out buffer0[i], + out buffer0[15 - i]); + } + + // Stage 2 begins the recursive factorization of the even half and rotates the central odd pairs by pi/4. + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[i], buffer0[7 - i], out buffer1[i], out buffer1[7 - i]); + } + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[32], + cospi[32], + buffer0[10], + buffer0[13], + out buffer1[10], + out buffer1[13], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[32], + cospi[32], + buffer0[11], + buffer0[12], + out buffer1[11], + out buffer1[12], + cosBit, + in rounding); + + // Stage 3 reduces both eight-value groups into the four-value units consumed by the terminal rotations. + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[i], buffer1[3 - i], out buffer0[i], out buffer0[3 - i]); + } + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[32], + cospi[32], + buffer1[5], + buffer1[6], + out buffer0[5], + out buffer0[6], + cosBit, + in rounding); + + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[8 + i], buffer1[11 - i], out buffer0[8 + i], out buffer0[11 - i]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[15 - i], buffer1[12 + i], out buffer0[15 - i], out buffer0[12 + i]); + } + + // The even coefficients become final at stages 4 and 5, so they are written directly to their AV1 order. + Av1ForwardTransformArithmetic>.Butterfly( + cospi[32], + cospi[32], + buffer0[0], + buffer0[1], + out Vector512 output0, + out Vector512 output8, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + cospi[16], + cospi[48], + buffer0[3], + buffer0[2], + out Vector512 output4, + out Vector512 output12, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[4], buffer0[5], out buffer1[4], out buffer1[5]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[7], buffer0[6], out buffer1[7], out buffer1[6]); + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[16], + cospi[48], + buffer0[9], + buffer0[14], + out buffer1[9], + out buffer1[14], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[48], + -cospi[16], + buffer0[10], + buffer0[13], + out buffer1[10], + out buffer1[13], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + cospi[8], + cospi[56], + buffer1[7], + buffer1[4], + out Vector512 output2, + out Vector512 output14, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + cospi[40], + cospi[24], + buffer1[6], + buffer1[5], + out Vector512 output10, + out Vector512 output6, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[8], buffer1[9], out buffer0[8], out buffer0[9]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[11], buffer1[10], out buffer0[11], out buffer0[10]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[12], buffer1[13], out buffer0[12], out buffer0[13]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[15], buffer1[14], out buffer0[15], out buffer0[14]); + + // Stage 6 applies the final pi/32 odd-frequency rotations. The following stores perform only the normative + // coefficient permutation, so each rotation result is named by its final destination. + Av1ForwardTransformArithmetic>.Butterfly( + cospi[4], + cospi[60], + buffer0[15], + buffer0[8], + out Vector512 output1, + out Vector512 output15, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + cospi[36], + cospi[28], + buffer0[14], + buffer0[9], + out Vector512 output9, + out Vector512 output7, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + cospi[20], + cospi[44], + buffer0[13], + buffer0[10], + out Vector512 output5, + out Vector512 output11, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + cospi[52], + cospi[12], + buffer0[12], + buffer0[11], + out Vector512 output13, + out Vector512 output3, + cosBit, + in rounding); + + Store(ref values, outputStride, 0, output0); + Store(ref values, outputStride, 1, output1); + Store(ref values, outputStride, 2, output2); + Store(ref values, outputStride, 3, output3); + Store(ref values, outputStride, 4, output4); + Store(ref values, outputStride, 5, output5); + Store(ref values, outputStride, 6, output6); + Store(ref values, outputStride, 7, output7); + Store(ref values, outputStride, 8, output8); + Store(ref values, outputStride, 9, output9); + Store(ref values, outputStride, 10, output10); + Store(ref values, outputStride, 11, output11); + Store(ref values, outputStride, 12, output12); + Store(ref values, outputStride, 13, output13); + Store(ref values, outputStride, 14, output14); + Store(ref values, outputStride, 15, output15); + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Transform/Forward/Av1ForwardTransformer.Dct32Operator.cs b/src/ImageSharp/Formats/Heif/Av1/Transform/Forward/Av1ForwardTransformer.Dct32Operator.cs new file mode 100644 index 0000000000..1624d7bbb1 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Transform/Forward/Av1ForwardTransformer.Dct32Operator.cs @@ -0,0 +1,1963 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.Intrinsics; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform.Forward; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +/// +/// Implements the thirty-two-point forward DCT stage network. +/// +internal static partial class Av1ForwardTransformer +{ + /// + /// Implements the thirty-two-point forward transform for every supported lane width. + /// + internal readonly struct Dct32Operator : IAv1ForwardTransform1dOperator + { + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector buffer0, + ref Av1TransformVector buffer1, + int cosBit) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + Av1TransformRounding rounding = Av1ForwardTransformArithmetic.CreateRounding(cosBit); + + // Stage 1 consumes the source block completely before any final coefficient is stored back into it. + for (int i = 0; i < 16; i++) + { + Av1ForwardTransformArithmetic.AddSubtract( + Load(ref values, inputStride, i), + Load(ref values, inputStride, 31 - i), + out buffer1[i], + out buffer1[31 - i]); + } + + // Stage 2 starts the recursive radix-2 factorization and rotates the central odd-frequency pairs. + for (int i = 0; i < 8; i++) + { + Av1ForwardTransformArithmetic.AddSubtract(buffer1[i], buffer1[15 - i], out buffer0[i], out buffer0[15 - i]); + } + + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic.Butterfly( + -cospi[32], + cospi[32], + buffer1[20 + i], + buffer1[27 - i], + out buffer0[20 + i], + out buffer0[27 - i], + cosBit, + in rounding); + } + + // Stage 3 reduces the even half and folds the next odd-frequency groups into paired sums and differences. + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic.AddSubtract(buffer0[i], buffer0[7 - i], out buffer1[i], out buffer1[7 - i]); + } + + Av1ForwardTransformArithmetic.Butterfly( + -cospi[32], + cospi[32], + buffer0[10], + buffer0[13], + out buffer1[10], + out buffer1[13], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic.Butterfly( + -cospi[32], + cospi[32], + buffer0[11], + buffer0[12], + out buffer1[11], + out buffer1[12], + cosBit, + in rounding); + + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic.AddSubtract(buffer1[16 + i], buffer0[23 - i], out buffer1[16 + i], out buffer1[23 - i]); + Av1ForwardTransformArithmetic.AddSubtract(buffer1[31 - i], buffer0[24 + i], out buffer1[31 - i], out buffer1[24 + i]); + } + + // Stage 4 continues the factorization as independent eight-value groups. + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic.AddSubtract(buffer1[i], buffer1[3 - i], out buffer0[i], out buffer0[3 - i]); + } + + Av1ForwardTransformArithmetic.Butterfly( + -cospi[32], + cospi[32], + buffer1[5], + buffer1[6], + out buffer0[5], + out buffer0[6], + cosBit, + in rounding); + + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic.AddSubtract(buffer0[8 + i], buffer1[11 - i], out buffer0[8 + i], out buffer0[11 - i]); + Av1ForwardTransformArithmetic.AddSubtract(buffer0[15 - i], buffer1[12 + i], out buffer0[15 - i], out buffer0[12 + i]); + } + + Av1ForwardTransformArithmetic.Butterfly( + -cospi[16], + cospi[48], + buffer1[18], + buffer1[29], + out buffer0[18], + out buffer0[29], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic.Butterfly( + -cospi[16], + cospi[48], + buffer1[19], + buffer1[28], + out buffer0[19], + out buffer0[28], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic.Butterfly( + -cospi[48], + -cospi[16], + buffer1[20], + buffer1[27], + out buffer0[20], + out buffer0[27], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic.Butterfly( + -cospi[48], + -cospi[16], + buffer1[21], + buffer1[26], + out buffer0[21], + out buffer0[26], + cosBit, + in rounding); + + // Stage 5 completes the low-frequency DCT and rotates the first separated odd groups. Final coefficients are + // retired directly to the block instead of being copied through a third workspace. + ButterflyStore(cospi[32], cospi[32], buffer0[0], buffer0[1], ref values, outputStride, 0, 16, cosBit, in rounding); + ButterflyStore(cospi[16], cospi[48], buffer0[3], buffer0[2], ref values, outputStride, 8, 24, cosBit, in rounding); + Av1ForwardTransformArithmetic.AddSubtract(buffer1[4], buffer0[5], out buffer1[4], out buffer1[5]); + Av1ForwardTransformArithmetic.AddSubtract(buffer1[7], buffer0[6], out buffer1[7], out buffer1[6]); + Av1ForwardTransformArithmetic.Butterfly( + -cospi[16], + cospi[48], + buffer0[9], + buffer0[14], + out buffer1[9], + out buffer1[14], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic.Butterfly( + -cospi[48], + -cospi[16], + buffer0[10], + buffer0[13], + out buffer1[10], + out buffer1[13], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic.AddSubtract(buffer1[16], buffer0[19], out buffer1[16], out buffer1[19]); + Av1ForwardTransformArithmetic.AddSubtract(buffer1[17], buffer0[18], out buffer1[17], out buffer1[18]); + Av1ForwardTransformArithmetic.AddSubtract(buffer1[23], buffer0[20], out buffer1[23], out buffer1[20]); + Av1ForwardTransformArithmetic.AddSubtract(buffer1[22], buffer0[21], out buffer1[22], out buffer1[21]); + Av1ForwardTransformArithmetic.AddSubtract(buffer1[24], buffer0[27], out buffer1[24], out buffer1[27]); + Av1ForwardTransformArithmetic.AddSubtract(buffer1[25], buffer0[26], out buffer1[25], out buffer1[26]); + Av1ForwardTransformArithmetic.AddSubtract(buffer1[31], buffer0[28], out buffer1[31], out buffer1[28]); + Av1ForwardTransformArithmetic.AddSubtract(buffer1[30], buffer0[29], out buffer1[30], out buffer1[29]); + + // Stage 6 merges adjacent odd-frequency terms with the sign pattern required by the next rotations. + ButterflyStore(cospi[8], cospi[56], buffer1[7], buffer1[4], ref values, outputStride, 4, 28, cosBit, in rounding); + ButterflyStore(cospi[40], cospi[24], buffer1[6], buffer1[5], ref values, outputStride, 20, 12, cosBit, in rounding); + Av1ForwardTransformArithmetic.AddSubtract(buffer0[8], buffer1[9], out buffer0[8], out buffer0[9]); + Av1ForwardTransformArithmetic.AddSubtract(buffer0[11], buffer1[10], out buffer0[11], out buffer0[10]); + Av1ForwardTransformArithmetic.AddSubtract(buffer0[12], buffer1[13], out buffer0[12], out buffer0[13]); + Av1ForwardTransformArithmetic.AddSubtract(buffer0[15], buffer1[14], out buffer0[15], out buffer0[14]); + Av1ForwardTransformArithmetic.Butterfly( + -cospi[8], + cospi[56], + buffer1[17], + buffer1[30], + out buffer0[17], + out buffer0[30], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic.Butterfly( + -cospi[56], + -cospi[8], + buffer1[18], + buffer1[29], + out buffer0[18], + out buffer0[29], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic.Butterfly( + -cospi[40], + cospi[24], + buffer1[21], + buffer1[26], + out buffer0[21], + out buffer0[26], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic.Butterfly( + -cospi[24], + -cospi[40], + buffer1[22], + buffer1[25], + out buffer0[22], + out buffer0[25], + cosBit, + in rounding); + + // Stage 7 applies the pi/32 rotations to the next odd-frequency level. + ButterflyStore(cospi[4], cospi[60], buffer0[15], buffer0[8], ref values, outputStride, 2, 30, cosBit, in rounding); + ButterflyStore(cospi[36], cospi[28], buffer0[14], buffer0[9], ref values, outputStride, 18, 14, cosBit, in rounding); + ButterflyStore(cospi[20], cospi[44], buffer0[13], buffer0[10], ref values, outputStride, 10, 22, cosBit, in rounding); + ButterflyStore(cospi[52], cospi[12], buffer0[12], buffer0[11], ref values, outputStride, 26, 6, cosBit, in rounding); + Av1ForwardTransformArithmetic.AddSubtract(buffer1[16], buffer0[17], out buffer1[16], out buffer1[17]); + Av1ForwardTransformArithmetic.AddSubtract(buffer1[19], buffer0[18], out buffer1[19], out buffer1[18]); + Av1ForwardTransformArithmetic.AddSubtract(buffer1[20], buffer0[21], out buffer1[20], out buffer1[21]); + Av1ForwardTransformArithmetic.AddSubtract(buffer1[23], buffer0[22], out buffer1[23], out buffer1[22]); + Av1ForwardTransformArithmetic.AddSubtract(buffer1[24], buffer0[25], out buffer1[24], out buffer1[25]); + Av1ForwardTransformArithmetic.AddSubtract(buffer1[27], buffer0[26], out buffer1[27], out buffer1[26]); + Av1ForwardTransformArithmetic.AddSubtract(buffer1[28], buffer0[29], out buffer1[28], out buffer1[29]); + Av1ForwardTransformArithmetic.AddSubtract(buffer1[31], buffer0[30], out buffer1[31], out buffer1[30]); + + // Stages 8 and 9 fuse the terminal pi/64 rotations with the output permutation because none of their results + // are consumed by another arithmetic stage. + ButterflyStore(cospi[2], cospi[62], buffer1[31], buffer1[16], ref values, outputStride, 1, 31, cosBit, in rounding); + ButterflyStore(cospi[34], cospi[30], buffer1[30], buffer1[17], ref values, outputStride, 17, 15, cosBit, in rounding); + ButterflyStore(cospi[18], cospi[46], buffer1[29], buffer1[18], ref values, outputStride, 9, 23, cosBit, in rounding); + ButterflyStore(cospi[50], cospi[14], buffer1[28], buffer1[19], ref values, outputStride, 25, 7, cosBit, in rounding); + ButterflyStore(cospi[10], cospi[54], buffer1[27], buffer1[20], ref values, outputStride, 5, 27, cosBit, in rounding); + ButterflyStore(cospi[42], cospi[22], buffer1[26], buffer1[21], ref values, outputStride, 21, 11, cosBit, in rounding); + ButterflyStore(cospi[26], cospi[38], buffer1[25], buffer1[22], ref values, outputStride, 13, 19, cosBit, in rounding); + ButterflyStore(cospi[58], cospi[6], buffer1[24], buffer1[23], ref values, outputStride, 29, 3, cosBit, in rounding); + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector buffer0, + ref Av1TransformVector buffer1, + int cosBit) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + Av1TransformRounding rounding = Av1ForwardTransformArithmetic.CreateRounding(cosBit); + + // Stage 1 consumes the source block completely before any final coefficient is stored back into it. + for (int i = 0; i < 16; i++) + { + Av1ForwardTransformArithmetic.AddSubtract( + Load(ref values, inputStride, i), + Load(ref values, inputStride, 31 - i), + out buffer1[i], + out buffer1[31 - i]); + } + + // Stage 2 starts the recursive radix-2 factorization and rotates the central odd-frequency pairs. + for (int i = 0; i < 8; i++) + { + Av1ForwardTransformArithmetic.AddSubtract(buffer1[i], buffer1[15 - i], out buffer0[i], out buffer0[15 - i]); + } + + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic.Butterfly( + -cospi[32], + cospi[32], + buffer1[20 + i], + buffer1[27 - i], + out buffer0[20 + i], + out buffer0[27 - i], + cosBit, + in rounding); + } + + // Stage 3 reduces the even half and folds the next odd-frequency groups into paired sums and differences. + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic.AddSubtract(buffer0[i], buffer0[7 - i], out buffer1[i], out buffer1[7 - i]); + } + + Av1ForwardTransformArithmetic.Butterfly( + -cospi[32], + cospi[32], + buffer0[10], + buffer0[13], + out buffer1[10], + out buffer1[13], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic.Butterfly( + -cospi[32], + cospi[32], + buffer0[11], + buffer0[12], + out buffer1[11], + out buffer1[12], + cosBit, + in rounding); + + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic.AddSubtract(buffer1[16 + i], buffer0[23 - i], out buffer1[16 + i], out buffer1[23 - i]); + Av1ForwardTransformArithmetic.AddSubtract(buffer1[31 - i], buffer0[24 + i], out buffer1[31 - i], out buffer1[24 + i]); + } + + // Stage 4 continues the factorization as independent eight-value groups. + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic.AddSubtract(buffer1[i], buffer1[3 - i], out buffer0[i], out buffer0[3 - i]); + } + + Av1ForwardTransformArithmetic.Butterfly( + -cospi[32], + cospi[32], + buffer1[5], + buffer1[6], + out buffer0[5], + out buffer0[6], + cosBit, + in rounding); + + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic.AddSubtract(buffer0[8 + i], buffer1[11 - i], out buffer0[8 + i], out buffer0[11 - i]); + Av1ForwardTransformArithmetic.AddSubtract(buffer0[15 - i], buffer1[12 + i], out buffer0[15 - i], out buffer0[12 + i]); + } + + Av1ForwardTransformArithmetic.Butterfly( + -cospi[16], + cospi[48], + buffer1[18], + buffer1[29], + out buffer0[18], + out buffer0[29], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic.Butterfly( + -cospi[16], + cospi[48], + buffer1[19], + buffer1[28], + out buffer0[19], + out buffer0[28], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic.Butterfly( + -cospi[48], + -cospi[16], + buffer1[20], + buffer1[27], + out buffer0[20], + out buffer0[27], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic.Butterfly( + -cospi[48], + -cospi[16], + buffer1[21], + buffer1[26], + out buffer0[21], + out buffer0[26], + cosBit, + in rounding); + + // Stage 5 completes the low-frequency DCT and rotates the first separated odd groups. Final coefficients are + // retired directly to the block instead of being copied through a third workspace. + ButterflyStore(cospi[32], cospi[32], buffer0[0], buffer0[1], ref values, outputStride, 0, 16, cosBit, in rounding); + ButterflyStore(cospi[16], cospi[48], buffer0[3], buffer0[2], ref values, outputStride, 8, 24, cosBit, in rounding); + Av1ForwardTransformArithmetic.AddSubtract(buffer1[4], buffer0[5], out buffer1[4], out buffer1[5]); + Av1ForwardTransformArithmetic.AddSubtract(buffer1[7], buffer0[6], out buffer1[7], out buffer1[6]); + Av1ForwardTransformArithmetic.Butterfly( + -cospi[16], + cospi[48], + buffer0[9], + buffer0[14], + out buffer1[9], + out buffer1[14], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic.Butterfly( + -cospi[48], + -cospi[16], + buffer0[10], + buffer0[13], + out buffer1[10], + out buffer1[13], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic.AddSubtract(buffer1[16], buffer0[19], out buffer1[16], out buffer1[19]); + Av1ForwardTransformArithmetic.AddSubtract(buffer1[17], buffer0[18], out buffer1[17], out buffer1[18]); + Av1ForwardTransformArithmetic.AddSubtract(buffer1[23], buffer0[20], out buffer1[23], out buffer1[20]); + Av1ForwardTransformArithmetic.AddSubtract(buffer1[22], buffer0[21], out buffer1[22], out buffer1[21]); + Av1ForwardTransformArithmetic.AddSubtract(buffer1[24], buffer0[27], out buffer1[24], out buffer1[27]); + Av1ForwardTransformArithmetic.AddSubtract(buffer1[25], buffer0[26], out buffer1[25], out buffer1[26]); + Av1ForwardTransformArithmetic.AddSubtract(buffer1[31], buffer0[28], out buffer1[31], out buffer1[28]); + Av1ForwardTransformArithmetic.AddSubtract(buffer1[30], buffer0[29], out buffer1[30], out buffer1[29]); + + // Stage 6 merges adjacent odd-frequency terms with the sign pattern required by the next rotations. + ButterflyStore(cospi[8], cospi[56], buffer1[7], buffer1[4], ref values, outputStride, 4, 28, cosBit, in rounding); + ButterflyStore(cospi[40], cospi[24], buffer1[6], buffer1[5], ref values, outputStride, 20, 12, cosBit, in rounding); + Av1ForwardTransformArithmetic.AddSubtract(buffer0[8], buffer1[9], out buffer0[8], out buffer0[9]); + Av1ForwardTransformArithmetic.AddSubtract(buffer0[11], buffer1[10], out buffer0[11], out buffer0[10]); + Av1ForwardTransformArithmetic.AddSubtract(buffer0[12], buffer1[13], out buffer0[12], out buffer0[13]); + Av1ForwardTransformArithmetic.AddSubtract(buffer0[15], buffer1[14], out buffer0[15], out buffer0[14]); + Av1ForwardTransformArithmetic.Butterfly( + -cospi[8], + cospi[56], + buffer1[17], + buffer1[30], + out buffer0[17], + out buffer0[30], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic.Butterfly( + -cospi[56], + -cospi[8], + buffer1[18], + buffer1[29], + out buffer0[18], + out buffer0[29], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic.Butterfly( + -cospi[40], + cospi[24], + buffer1[21], + buffer1[26], + out buffer0[21], + out buffer0[26], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic.Butterfly( + -cospi[24], + -cospi[40], + buffer1[22], + buffer1[25], + out buffer0[22], + out buffer0[25], + cosBit, + in rounding); + + // Stage 7 applies the pi/32 rotations to the next odd-frequency level. + ButterflyStore(cospi[4], cospi[60], buffer0[15], buffer0[8], ref values, outputStride, 2, 30, cosBit, in rounding); + ButterflyStore(cospi[36], cospi[28], buffer0[14], buffer0[9], ref values, outputStride, 18, 14, cosBit, in rounding); + ButterflyStore(cospi[20], cospi[44], buffer0[13], buffer0[10], ref values, outputStride, 10, 22, cosBit, in rounding); + ButterflyStore(cospi[52], cospi[12], buffer0[12], buffer0[11], ref values, outputStride, 26, 6, cosBit, in rounding); + Av1ForwardTransformArithmetic.AddSubtract(buffer1[16], buffer0[17], out buffer1[16], out buffer1[17]); + Av1ForwardTransformArithmetic.AddSubtract(buffer1[19], buffer0[18], out buffer1[19], out buffer1[18]); + Av1ForwardTransformArithmetic.AddSubtract(buffer1[20], buffer0[21], out buffer1[20], out buffer1[21]); + Av1ForwardTransformArithmetic.AddSubtract(buffer1[23], buffer0[22], out buffer1[23], out buffer1[22]); + Av1ForwardTransformArithmetic.AddSubtract(buffer1[24], buffer0[25], out buffer1[24], out buffer1[25]); + Av1ForwardTransformArithmetic.AddSubtract(buffer1[27], buffer0[26], out buffer1[27], out buffer1[26]); + Av1ForwardTransformArithmetic.AddSubtract(buffer1[28], buffer0[29], out buffer1[28], out buffer1[29]); + Av1ForwardTransformArithmetic.AddSubtract(buffer1[31], buffer0[30], out buffer1[31], out buffer1[30]); + + // Stages 8 and 9 fuse the terminal pi/64 rotations with the output permutation because none of their results + // are consumed by another arithmetic stage. + ButterflyStore(cospi[2], cospi[62], buffer1[31], buffer1[16], ref values, outputStride, 1, 31, cosBit, in rounding); + ButterflyStore(cospi[34], cospi[30], buffer1[30], buffer1[17], ref values, outputStride, 17, 15, cosBit, in rounding); + ButterflyStore(cospi[18], cospi[46], buffer1[29], buffer1[18], ref values, outputStride, 9, 23, cosBit, in rounding); + ButterflyStore(cospi[50], cospi[14], buffer1[28], buffer1[19], ref values, outputStride, 25, 7, cosBit, in rounding); + ButterflyStore(cospi[10], cospi[54], buffer1[27], buffer1[20], ref values, outputStride, 5, 27, cosBit, in rounding); + ButterflyStore(cospi[42], cospi[22], buffer1[26], buffer1[21], ref values, outputStride, 21, 11, cosBit, in rounding); + ButterflyStore(cospi[26], cospi[38], buffer1[25], buffer1[22], ref values, outputStride, 13, 19, cosBit, in rounding); + ButterflyStore(cospi[58], cospi[6], buffer1[24], buffer1[23], ref values, outputStride, 29, 3, cosBit, in rounding); + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + Av1TransformRounding rounding = Av1ForwardTransformArithmetic>.CreateRounding(cosBit); + + // Stage 1 consumes the source block completely before any final coefficient is stored back into it. + for (int i = 0; i < 16; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract( + Load>(ref values, inputStride, i), + Load>(ref values, inputStride, 31 - i), + out buffer1[i], + out buffer1[31 - i]); + } + + // Stage 2 starts the recursive radix-2 factorization and rotates the central odd-frequency pairs. + for (int i = 0; i < 8; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[i], buffer1[15 - i], out buffer0[i], out buffer0[15 - i]); + } + + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[32], + cospi[32], + buffer1[20 + i], + buffer1[27 - i], + out buffer0[20 + i], + out buffer0[27 - i], + cosBit, + in rounding); + } + + // Stage 3 reduces the even half and folds the next odd-frequency groups into paired sums and differences. + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[i], buffer0[7 - i], out buffer1[i], out buffer1[7 - i]); + } + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[32], + cospi[32], + buffer0[10], + buffer0[13], + out buffer1[10], + out buffer1[13], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[32], + cospi[32], + buffer0[11], + buffer0[12], + out buffer1[11], + out buffer1[12], + cosBit, + in rounding); + + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[16 + i], buffer0[23 - i], out buffer1[16 + i], out buffer1[23 - i]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[31 - i], buffer0[24 + i], out buffer1[31 - i], out buffer1[24 + i]); + } + + // Stage 4 continues the factorization as independent eight-value groups. + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[i], buffer1[3 - i], out buffer0[i], out buffer0[3 - i]); + } + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[32], + cospi[32], + buffer1[5], + buffer1[6], + out buffer0[5], + out buffer0[6], + cosBit, + in rounding); + + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[8 + i], buffer1[11 - i], out buffer0[8 + i], out buffer0[11 - i]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[15 - i], buffer1[12 + i], out buffer0[15 - i], out buffer0[12 + i]); + } + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[16], + cospi[48], + buffer1[18], + buffer1[29], + out buffer0[18], + out buffer0[29], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[16], + cospi[48], + buffer1[19], + buffer1[28], + out buffer0[19], + out buffer0[28], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[48], + -cospi[16], + buffer1[20], + buffer1[27], + out buffer0[20], + out buffer0[27], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[48], + -cospi[16], + buffer1[21], + buffer1[26], + out buffer0[21], + out buffer0[26], + cosBit, + in rounding); + + // Stage 5 completes the low-frequency DCT and rotates the first separated odd groups. Final coefficients are + // retired directly to the block instead of being copied through a third workspace. + ButterflyStore(cospi[32], cospi[32], buffer0[0], buffer0[1], ref values, outputStride, 0, 16, cosBit, in rounding); + ButterflyStore(cospi[16], cospi[48], buffer0[3], buffer0[2], ref values, outputStride, 8, 24, cosBit, in rounding); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[4], buffer0[5], out buffer1[4], out buffer1[5]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[7], buffer0[6], out buffer1[7], out buffer1[6]); + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[16], + cospi[48], + buffer0[9], + buffer0[14], + out buffer1[9], + out buffer1[14], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[48], + -cospi[16], + buffer0[10], + buffer0[13], + out buffer1[10], + out buffer1[13], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[16], buffer0[19], out buffer1[16], out buffer1[19]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[17], buffer0[18], out buffer1[17], out buffer1[18]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[23], buffer0[20], out buffer1[23], out buffer1[20]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[22], buffer0[21], out buffer1[22], out buffer1[21]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[24], buffer0[27], out buffer1[24], out buffer1[27]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[25], buffer0[26], out buffer1[25], out buffer1[26]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[31], buffer0[28], out buffer1[31], out buffer1[28]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[30], buffer0[29], out buffer1[30], out buffer1[29]); + + // Stage 6 merges adjacent odd-frequency terms with the sign pattern required by the next rotations. + ButterflyStore(cospi[8], cospi[56], buffer1[7], buffer1[4], ref values, outputStride, 4, 28, cosBit, in rounding); + ButterflyStore(cospi[40], cospi[24], buffer1[6], buffer1[5], ref values, outputStride, 20, 12, cosBit, in rounding); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[8], buffer1[9], out buffer0[8], out buffer0[9]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[11], buffer1[10], out buffer0[11], out buffer0[10]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[12], buffer1[13], out buffer0[12], out buffer0[13]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[15], buffer1[14], out buffer0[15], out buffer0[14]); + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[8], + cospi[56], + buffer1[17], + buffer1[30], + out buffer0[17], + out buffer0[30], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[56], + -cospi[8], + buffer1[18], + buffer1[29], + out buffer0[18], + out buffer0[29], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[40], + cospi[24], + buffer1[21], + buffer1[26], + out buffer0[21], + out buffer0[26], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[24], + -cospi[40], + buffer1[22], + buffer1[25], + out buffer0[22], + out buffer0[25], + cosBit, + in rounding); + + // Stage 7 applies the pi/32 rotations to the next odd-frequency level. + ButterflyStore(cospi[4], cospi[60], buffer0[15], buffer0[8], ref values, outputStride, 2, 30, cosBit, in rounding); + ButterflyStore(cospi[36], cospi[28], buffer0[14], buffer0[9], ref values, outputStride, 18, 14, cosBit, in rounding); + ButterflyStore(cospi[20], cospi[44], buffer0[13], buffer0[10], ref values, outputStride, 10, 22, cosBit, in rounding); + ButterflyStore(cospi[52], cospi[12], buffer0[12], buffer0[11], ref values, outputStride, 26, 6, cosBit, in rounding); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[16], buffer0[17], out buffer1[16], out buffer1[17]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[19], buffer0[18], out buffer1[19], out buffer1[18]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[20], buffer0[21], out buffer1[20], out buffer1[21]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[23], buffer0[22], out buffer1[23], out buffer1[22]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[24], buffer0[25], out buffer1[24], out buffer1[25]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[27], buffer0[26], out buffer1[27], out buffer1[26]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[28], buffer0[29], out buffer1[28], out buffer1[29]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[31], buffer0[30], out buffer1[31], out buffer1[30]); + + // Stages 8 and 9 fuse the terminal pi/64 rotations with the output permutation because none of their results + // are consumed by another arithmetic stage. + ButterflyStore(cospi[2], cospi[62], buffer1[31], buffer1[16], ref values, outputStride, 1, 31, cosBit, in rounding); + ButterflyStore(cospi[34], cospi[30], buffer1[30], buffer1[17], ref values, outputStride, 17, 15, cosBit, in rounding); + ButterflyStore(cospi[18], cospi[46], buffer1[29], buffer1[18], ref values, outputStride, 9, 23, cosBit, in rounding); + ButterflyStore(cospi[50], cospi[14], buffer1[28], buffer1[19], ref values, outputStride, 25, 7, cosBit, in rounding); + ButterflyStore(cospi[10], cospi[54], buffer1[27], buffer1[20], ref values, outputStride, 5, 27, cosBit, in rounding); + ButterflyStore(cospi[42], cospi[22], buffer1[26], buffer1[21], ref values, outputStride, 21, 11, cosBit, in rounding); + ButterflyStore(cospi[26], cospi[38], buffer1[25], buffer1[22], ref values, outputStride, 13, 19, cosBit, in rounding); + ButterflyStore(cospi[58], cospi[6], buffer1[24], buffer1[23], ref values, outputStride, 29, 3, cosBit, in rounding); + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + Av1TransformRounding rounding = Av1ForwardTransformArithmetic>.CreateRounding(cosBit); + + // Stage 1 consumes the source block completely before any final coefficient is stored back into it. + for (int i = 0; i < 16; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract( + Load>(ref values, inputStride, i), + Load>(ref values, inputStride, 31 - i), + out buffer1[i], + out buffer1[31 - i]); + } + + // Stage 2 starts the recursive radix-2 factorization and rotates the central odd-frequency pairs. + for (int i = 0; i < 8; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[i], buffer1[15 - i], out buffer0[i], out buffer0[15 - i]); + } + + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[32], + cospi[32], + buffer1[20 + i], + buffer1[27 - i], + out buffer0[20 + i], + out buffer0[27 - i], + cosBit, + in rounding); + } + + // Stage 3 reduces the even half and folds the next odd-frequency groups into paired sums and differences. + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[i], buffer0[7 - i], out buffer1[i], out buffer1[7 - i]); + } + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[32], + cospi[32], + buffer0[10], + buffer0[13], + out buffer1[10], + out buffer1[13], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[32], + cospi[32], + buffer0[11], + buffer0[12], + out buffer1[11], + out buffer1[12], + cosBit, + in rounding); + + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[16 + i], buffer0[23 - i], out buffer1[16 + i], out buffer1[23 - i]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[31 - i], buffer0[24 + i], out buffer1[31 - i], out buffer1[24 + i]); + } + + // Stage 4 continues the factorization as independent eight-value groups. + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[i], buffer1[3 - i], out buffer0[i], out buffer0[3 - i]); + } + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[32], + cospi[32], + buffer1[5], + buffer1[6], + out buffer0[5], + out buffer0[6], + cosBit, + in rounding); + + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[8 + i], buffer1[11 - i], out buffer0[8 + i], out buffer0[11 - i]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[15 - i], buffer1[12 + i], out buffer0[15 - i], out buffer0[12 + i]); + } + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[16], + cospi[48], + buffer1[18], + buffer1[29], + out buffer0[18], + out buffer0[29], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[16], + cospi[48], + buffer1[19], + buffer1[28], + out buffer0[19], + out buffer0[28], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[48], + -cospi[16], + buffer1[20], + buffer1[27], + out buffer0[20], + out buffer0[27], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[48], + -cospi[16], + buffer1[21], + buffer1[26], + out buffer0[21], + out buffer0[26], + cosBit, + in rounding); + + // Stage 5 completes the low-frequency DCT and rotates the first separated odd groups. Final coefficients are + // retired directly to the block instead of being copied through a third workspace. + ButterflyStore(cospi[32], cospi[32], buffer0[0], buffer0[1], ref values, outputStride, 0, 16, cosBit, in rounding); + ButterflyStore(cospi[16], cospi[48], buffer0[3], buffer0[2], ref values, outputStride, 8, 24, cosBit, in rounding); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[4], buffer0[5], out buffer1[4], out buffer1[5]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[7], buffer0[6], out buffer1[7], out buffer1[6]); + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[16], + cospi[48], + buffer0[9], + buffer0[14], + out buffer1[9], + out buffer1[14], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[48], + -cospi[16], + buffer0[10], + buffer0[13], + out buffer1[10], + out buffer1[13], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[16], buffer0[19], out buffer1[16], out buffer1[19]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[17], buffer0[18], out buffer1[17], out buffer1[18]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[23], buffer0[20], out buffer1[23], out buffer1[20]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[22], buffer0[21], out buffer1[22], out buffer1[21]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[24], buffer0[27], out buffer1[24], out buffer1[27]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[25], buffer0[26], out buffer1[25], out buffer1[26]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[31], buffer0[28], out buffer1[31], out buffer1[28]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[30], buffer0[29], out buffer1[30], out buffer1[29]); + + // Stage 6 merges adjacent odd-frequency terms with the sign pattern required by the next rotations. + ButterflyStore(cospi[8], cospi[56], buffer1[7], buffer1[4], ref values, outputStride, 4, 28, cosBit, in rounding); + ButterflyStore(cospi[40], cospi[24], buffer1[6], buffer1[5], ref values, outputStride, 20, 12, cosBit, in rounding); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[8], buffer1[9], out buffer0[8], out buffer0[9]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[11], buffer1[10], out buffer0[11], out buffer0[10]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[12], buffer1[13], out buffer0[12], out buffer0[13]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[15], buffer1[14], out buffer0[15], out buffer0[14]); + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[8], + cospi[56], + buffer1[17], + buffer1[30], + out buffer0[17], + out buffer0[30], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[56], + -cospi[8], + buffer1[18], + buffer1[29], + out buffer0[18], + out buffer0[29], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[40], + cospi[24], + buffer1[21], + buffer1[26], + out buffer0[21], + out buffer0[26], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[24], + -cospi[40], + buffer1[22], + buffer1[25], + out buffer0[22], + out buffer0[25], + cosBit, + in rounding); + + // Stage 7 applies the pi/32 rotations to the next odd-frequency level. + ButterflyStore(cospi[4], cospi[60], buffer0[15], buffer0[8], ref values, outputStride, 2, 30, cosBit, in rounding); + ButterflyStore(cospi[36], cospi[28], buffer0[14], buffer0[9], ref values, outputStride, 18, 14, cosBit, in rounding); + ButterflyStore(cospi[20], cospi[44], buffer0[13], buffer0[10], ref values, outputStride, 10, 22, cosBit, in rounding); + ButterflyStore(cospi[52], cospi[12], buffer0[12], buffer0[11], ref values, outputStride, 26, 6, cosBit, in rounding); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[16], buffer0[17], out buffer1[16], out buffer1[17]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[19], buffer0[18], out buffer1[19], out buffer1[18]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[20], buffer0[21], out buffer1[20], out buffer1[21]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[23], buffer0[22], out buffer1[23], out buffer1[22]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[24], buffer0[25], out buffer1[24], out buffer1[25]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[27], buffer0[26], out buffer1[27], out buffer1[26]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[28], buffer0[29], out buffer1[28], out buffer1[29]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[31], buffer0[30], out buffer1[31], out buffer1[30]); + + // Stages 8 and 9 fuse the terminal pi/64 rotations with the output permutation because none of their results + // are consumed by another arithmetic stage. + ButterflyStore(cospi[2], cospi[62], buffer1[31], buffer1[16], ref values, outputStride, 1, 31, cosBit, in rounding); + ButterflyStore(cospi[34], cospi[30], buffer1[30], buffer1[17], ref values, outputStride, 17, 15, cosBit, in rounding); + ButterflyStore(cospi[18], cospi[46], buffer1[29], buffer1[18], ref values, outputStride, 9, 23, cosBit, in rounding); + ButterflyStore(cospi[50], cospi[14], buffer1[28], buffer1[19], ref values, outputStride, 25, 7, cosBit, in rounding); + ButterflyStore(cospi[10], cospi[54], buffer1[27], buffer1[20], ref values, outputStride, 5, 27, cosBit, in rounding); + ButterflyStore(cospi[42], cospi[22], buffer1[26], buffer1[21], ref values, outputStride, 21, 11, cosBit, in rounding); + ButterflyStore(cospi[26], cospi[38], buffer1[25], buffer1[22], ref values, outputStride, 13, 19, cosBit, in rounding); + ButterflyStore(cospi[58], cospi[6], buffer1[24], buffer1[23], ref values, outputStride, 29, 3, cosBit, in rounding); + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + Av1TransformRounding rounding = Av1ForwardTransformArithmetic>.CreateRounding(cosBit); + + // Stage 1 consumes the source block completely before any final coefficient is stored back into it. + for (int i = 0; i < 16; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract( + Load>(ref values, inputStride, i), + Load>(ref values, inputStride, 31 - i), + out buffer1[i], + out buffer1[31 - i]); + } + + // Stage 2 starts the recursive radix-2 factorization and rotates the central odd-frequency pairs. + for (int i = 0; i < 8; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[i], buffer1[15 - i], out buffer0[i], out buffer0[15 - i]); + } + + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[32], + cospi[32], + buffer1[20 + i], + buffer1[27 - i], + out buffer0[20 + i], + out buffer0[27 - i], + cosBit, + in rounding); + } + + // Stage 3 reduces the even half and folds the next odd-frequency groups into paired sums and differences. + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[i], buffer0[7 - i], out buffer1[i], out buffer1[7 - i]); + } + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[32], + cospi[32], + buffer0[10], + buffer0[13], + out buffer1[10], + out buffer1[13], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[32], + cospi[32], + buffer0[11], + buffer0[12], + out buffer1[11], + out buffer1[12], + cosBit, + in rounding); + + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[16 + i], buffer0[23 - i], out buffer1[16 + i], out buffer1[23 - i]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[31 - i], buffer0[24 + i], out buffer1[31 - i], out buffer1[24 + i]); + } + + // Stage 4 continues the factorization as independent eight-value groups. + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[i], buffer1[3 - i], out buffer0[i], out buffer0[3 - i]); + } + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[32], + cospi[32], + buffer1[5], + buffer1[6], + out buffer0[5], + out buffer0[6], + cosBit, + in rounding); + + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[8 + i], buffer1[11 - i], out buffer0[8 + i], out buffer0[11 - i]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[15 - i], buffer1[12 + i], out buffer0[15 - i], out buffer0[12 + i]); + } + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[16], + cospi[48], + buffer1[18], + buffer1[29], + out buffer0[18], + out buffer0[29], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[16], + cospi[48], + buffer1[19], + buffer1[28], + out buffer0[19], + out buffer0[28], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[48], + -cospi[16], + buffer1[20], + buffer1[27], + out buffer0[20], + out buffer0[27], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[48], + -cospi[16], + buffer1[21], + buffer1[26], + out buffer0[21], + out buffer0[26], + cosBit, + in rounding); + + // Stage 5 completes the low-frequency DCT and rotates the first separated odd groups. Final coefficients are + // retired directly to the block instead of being copied through a third workspace. + ButterflyStore(cospi[32], cospi[32], buffer0[0], buffer0[1], ref values, outputStride, 0, 16, cosBit, in rounding); + ButterflyStore(cospi[16], cospi[48], buffer0[3], buffer0[2], ref values, outputStride, 8, 24, cosBit, in rounding); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[4], buffer0[5], out buffer1[4], out buffer1[5]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[7], buffer0[6], out buffer1[7], out buffer1[6]); + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[16], + cospi[48], + buffer0[9], + buffer0[14], + out buffer1[9], + out buffer1[14], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[48], + -cospi[16], + buffer0[10], + buffer0[13], + out buffer1[10], + out buffer1[13], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[16], buffer0[19], out buffer1[16], out buffer1[19]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[17], buffer0[18], out buffer1[17], out buffer1[18]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[23], buffer0[20], out buffer1[23], out buffer1[20]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[22], buffer0[21], out buffer1[22], out buffer1[21]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[24], buffer0[27], out buffer1[24], out buffer1[27]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[25], buffer0[26], out buffer1[25], out buffer1[26]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[31], buffer0[28], out buffer1[31], out buffer1[28]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[30], buffer0[29], out buffer1[30], out buffer1[29]); + + // Stage 6 merges adjacent odd-frequency terms with the sign pattern required by the next rotations. + ButterflyStore(cospi[8], cospi[56], buffer1[7], buffer1[4], ref values, outputStride, 4, 28, cosBit, in rounding); + ButterflyStore(cospi[40], cospi[24], buffer1[6], buffer1[5], ref values, outputStride, 20, 12, cosBit, in rounding); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[8], buffer1[9], out buffer0[8], out buffer0[9]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[11], buffer1[10], out buffer0[11], out buffer0[10]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[12], buffer1[13], out buffer0[12], out buffer0[13]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[15], buffer1[14], out buffer0[15], out buffer0[14]); + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[8], + cospi[56], + buffer1[17], + buffer1[30], + out buffer0[17], + out buffer0[30], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[56], + -cospi[8], + buffer1[18], + buffer1[29], + out buffer0[18], + out buffer0[29], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[40], + cospi[24], + buffer1[21], + buffer1[26], + out buffer0[21], + out buffer0[26], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[24], + -cospi[40], + buffer1[22], + buffer1[25], + out buffer0[22], + out buffer0[25], + cosBit, + in rounding); + + // Stage 7 applies the pi/32 rotations to the next odd-frequency level. + ButterflyStore(cospi[4], cospi[60], buffer0[15], buffer0[8], ref values, outputStride, 2, 30, cosBit, in rounding); + ButterflyStore(cospi[36], cospi[28], buffer0[14], buffer0[9], ref values, outputStride, 18, 14, cosBit, in rounding); + ButterflyStore(cospi[20], cospi[44], buffer0[13], buffer0[10], ref values, outputStride, 10, 22, cosBit, in rounding); + ButterflyStore(cospi[52], cospi[12], buffer0[12], buffer0[11], ref values, outputStride, 26, 6, cosBit, in rounding); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[16], buffer0[17], out buffer1[16], out buffer1[17]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[19], buffer0[18], out buffer1[19], out buffer1[18]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[20], buffer0[21], out buffer1[20], out buffer1[21]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[23], buffer0[22], out buffer1[23], out buffer1[22]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[24], buffer0[25], out buffer1[24], out buffer1[25]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[27], buffer0[26], out buffer1[27], out buffer1[26]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[28], buffer0[29], out buffer1[28], out buffer1[29]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[31], buffer0[30], out buffer1[31], out buffer1[30]); + + // Stages 8 and 9 fuse the terminal pi/64 rotations with the output permutation because none of their results + // are consumed by another arithmetic stage. + ButterflyStore(cospi[2], cospi[62], buffer1[31], buffer1[16], ref values, outputStride, 1, 31, cosBit, in rounding); + ButterflyStore(cospi[34], cospi[30], buffer1[30], buffer1[17], ref values, outputStride, 17, 15, cosBit, in rounding); + ButterflyStore(cospi[18], cospi[46], buffer1[29], buffer1[18], ref values, outputStride, 9, 23, cosBit, in rounding); + ButterflyStore(cospi[50], cospi[14], buffer1[28], buffer1[19], ref values, outputStride, 25, 7, cosBit, in rounding); + ButterflyStore(cospi[10], cospi[54], buffer1[27], buffer1[20], ref values, outputStride, 5, 27, cosBit, in rounding); + ButterflyStore(cospi[42], cospi[22], buffer1[26], buffer1[21], ref values, outputStride, 21, 11, cosBit, in rounding); + ButterflyStore(cospi[26], cospi[38], buffer1[25], buffer1[22], ref values, outputStride, 13, 19, cosBit, in rounding); + ButterflyStore(cospi[58], cospi[6], buffer1[24], buffer1[23], ref values, outputStride, 29, 3, cosBit, in rounding); + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + Av1TransformRounding rounding = Av1ForwardTransformArithmetic>.CreateRounding(cosBit); + + // Stage 1 consumes the source block completely before any final coefficient is stored back into it. + for (int i = 0; i < 16; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract( + Load>(ref values, inputStride, i), + Load>(ref values, inputStride, 31 - i), + out buffer1[i], + out buffer1[31 - i]); + } + + // Stage 2 starts the recursive radix-2 factorization and rotates the central odd-frequency pairs. + for (int i = 0; i < 8; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[i], buffer1[15 - i], out buffer0[i], out buffer0[15 - i]); + } + + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[32], + cospi[32], + buffer1[20 + i], + buffer1[27 - i], + out buffer0[20 + i], + out buffer0[27 - i], + cosBit, + in rounding); + } + + // Stage 3 reduces the even half and folds the next odd-frequency groups into paired sums and differences. + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[i], buffer0[7 - i], out buffer1[i], out buffer1[7 - i]); + } + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[32], + cospi[32], + buffer0[10], + buffer0[13], + out buffer1[10], + out buffer1[13], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[32], + cospi[32], + buffer0[11], + buffer0[12], + out buffer1[11], + out buffer1[12], + cosBit, + in rounding); + + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[16 + i], buffer0[23 - i], out buffer1[16 + i], out buffer1[23 - i]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[31 - i], buffer0[24 + i], out buffer1[31 - i], out buffer1[24 + i]); + } + + // Stage 4 continues the factorization as independent eight-value groups. + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[i], buffer1[3 - i], out buffer0[i], out buffer0[3 - i]); + } + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[32], + cospi[32], + buffer1[5], + buffer1[6], + out buffer0[5], + out buffer0[6], + cosBit, + in rounding); + + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[8 + i], buffer1[11 - i], out buffer0[8 + i], out buffer0[11 - i]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[15 - i], buffer1[12 + i], out buffer0[15 - i], out buffer0[12 + i]); + } + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[16], + cospi[48], + buffer1[18], + buffer1[29], + out buffer0[18], + out buffer0[29], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[16], + cospi[48], + buffer1[19], + buffer1[28], + out buffer0[19], + out buffer0[28], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[48], + -cospi[16], + buffer1[20], + buffer1[27], + out buffer0[20], + out buffer0[27], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[48], + -cospi[16], + buffer1[21], + buffer1[26], + out buffer0[21], + out buffer0[26], + cosBit, + in rounding); + + // Stage 5 completes the low-frequency DCT and rotates the first separated odd groups. Final coefficients are + // retired directly to the block instead of being copied through a third workspace. + ButterflyStore(cospi[32], cospi[32], buffer0[0], buffer0[1], ref values, outputStride, 0, 16, cosBit, in rounding); + ButterflyStore(cospi[16], cospi[48], buffer0[3], buffer0[2], ref values, outputStride, 8, 24, cosBit, in rounding); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[4], buffer0[5], out buffer1[4], out buffer1[5]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[7], buffer0[6], out buffer1[7], out buffer1[6]); + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[16], + cospi[48], + buffer0[9], + buffer0[14], + out buffer1[9], + out buffer1[14], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[48], + -cospi[16], + buffer0[10], + buffer0[13], + out buffer1[10], + out buffer1[13], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[16], buffer0[19], out buffer1[16], out buffer1[19]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[17], buffer0[18], out buffer1[17], out buffer1[18]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[23], buffer0[20], out buffer1[23], out buffer1[20]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[22], buffer0[21], out buffer1[22], out buffer1[21]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[24], buffer0[27], out buffer1[24], out buffer1[27]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[25], buffer0[26], out buffer1[25], out buffer1[26]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[31], buffer0[28], out buffer1[31], out buffer1[28]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[30], buffer0[29], out buffer1[30], out buffer1[29]); + + // Stage 6 merges adjacent odd-frequency terms with the sign pattern required by the next rotations. + ButterflyStore(cospi[8], cospi[56], buffer1[7], buffer1[4], ref values, outputStride, 4, 28, cosBit, in rounding); + ButterflyStore(cospi[40], cospi[24], buffer1[6], buffer1[5], ref values, outputStride, 20, 12, cosBit, in rounding); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[8], buffer1[9], out buffer0[8], out buffer0[9]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[11], buffer1[10], out buffer0[11], out buffer0[10]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[12], buffer1[13], out buffer0[12], out buffer0[13]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[15], buffer1[14], out buffer0[15], out buffer0[14]); + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[8], + cospi[56], + buffer1[17], + buffer1[30], + out buffer0[17], + out buffer0[30], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[56], + -cospi[8], + buffer1[18], + buffer1[29], + out buffer0[18], + out buffer0[29], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[40], + cospi[24], + buffer1[21], + buffer1[26], + out buffer0[21], + out buffer0[26], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[24], + -cospi[40], + buffer1[22], + buffer1[25], + out buffer0[22], + out buffer0[25], + cosBit, + in rounding); + + // Stage 7 applies the pi/32 rotations to the next odd-frequency level. + ButterflyStore(cospi[4], cospi[60], buffer0[15], buffer0[8], ref values, outputStride, 2, 30, cosBit, in rounding); + ButterflyStore(cospi[36], cospi[28], buffer0[14], buffer0[9], ref values, outputStride, 18, 14, cosBit, in rounding); + ButterflyStore(cospi[20], cospi[44], buffer0[13], buffer0[10], ref values, outputStride, 10, 22, cosBit, in rounding); + ButterflyStore(cospi[52], cospi[12], buffer0[12], buffer0[11], ref values, outputStride, 26, 6, cosBit, in rounding); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[16], buffer0[17], out buffer1[16], out buffer1[17]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[19], buffer0[18], out buffer1[19], out buffer1[18]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[20], buffer0[21], out buffer1[20], out buffer1[21]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[23], buffer0[22], out buffer1[23], out buffer1[22]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[24], buffer0[25], out buffer1[24], out buffer1[25]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[27], buffer0[26], out buffer1[27], out buffer1[26]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[28], buffer0[29], out buffer1[28], out buffer1[29]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[31], buffer0[30], out buffer1[31], out buffer1[30]); + + // Stages 8 and 9 fuse the terminal pi/64 rotations with the output permutation because none of their results + // are consumed by another arithmetic stage. + ButterflyStore(cospi[2], cospi[62], buffer1[31], buffer1[16], ref values, outputStride, 1, 31, cosBit, in rounding); + ButterflyStore(cospi[34], cospi[30], buffer1[30], buffer1[17], ref values, outputStride, 17, 15, cosBit, in rounding); + ButterflyStore(cospi[18], cospi[46], buffer1[29], buffer1[18], ref values, outputStride, 9, 23, cosBit, in rounding); + ButterflyStore(cospi[50], cospi[14], buffer1[28], buffer1[19], ref values, outputStride, 25, 7, cosBit, in rounding); + ButterflyStore(cospi[10], cospi[54], buffer1[27], buffer1[20], ref values, outputStride, 5, 27, cosBit, in rounding); + ButterflyStore(cospi[42], cospi[22], buffer1[26], buffer1[21], ref values, outputStride, 21, 11, cosBit, in rounding); + ButterflyStore(cospi[26], cospi[38], buffer1[25], buffer1[22], ref values, outputStride, 13, 19, cosBit, in rounding); + ButterflyStore(cospi[58], cospi[6], buffer1[24], buffer1[23], ref values, outputStride, 29, 3, cosBit, in rounding); + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + Av1TransformRounding rounding = Av1ForwardTransformArithmetic>.CreateRounding(cosBit); + + // Stage 1 consumes the source block completely before any final coefficient is stored back into it. + for (int i = 0; i < 16; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract( + Load>(ref values, inputStride, i), + Load>(ref values, inputStride, 31 - i), + out buffer1[i], + out buffer1[31 - i]); + } + + // Stage 2 starts the recursive radix-2 factorization and rotates the central odd-frequency pairs. + for (int i = 0; i < 8; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[i], buffer1[15 - i], out buffer0[i], out buffer0[15 - i]); + } + + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[32], + cospi[32], + buffer1[20 + i], + buffer1[27 - i], + out buffer0[20 + i], + out buffer0[27 - i], + cosBit, + in rounding); + } + + // Stage 3 reduces the even half and folds the next odd-frequency groups into paired sums and differences. + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[i], buffer0[7 - i], out buffer1[i], out buffer1[7 - i]); + } + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[32], + cospi[32], + buffer0[10], + buffer0[13], + out buffer1[10], + out buffer1[13], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[32], + cospi[32], + buffer0[11], + buffer0[12], + out buffer1[11], + out buffer1[12], + cosBit, + in rounding); + + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[16 + i], buffer0[23 - i], out buffer1[16 + i], out buffer1[23 - i]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[31 - i], buffer0[24 + i], out buffer1[31 - i], out buffer1[24 + i]); + } + + // Stage 4 continues the factorization as independent eight-value groups. + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[i], buffer1[3 - i], out buffer0[i], out buffer0[3 - i]); + } + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[32], + cospi[32], + buffer1[5], + buffer1[6], + out buffer0[5], + out buffer0[6], + cosBit, + in rounding); + + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[8 + i], buffer1[11 - i], out buffer0[8 + i], out buffer0[11 - i]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[15 - i], buffer1[12 + i], out buffer0[15 - i], out buffer0[12 + i]); + } + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[16], + cospi[48], + buffer1[18], + buffer1[29], + out buffer0[18], + out buffer0[29], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[16], + cospi[48], + buffer1[19], + buffer1[28], + out buffer0[19], + out buffer0[28], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[48], + -cospi[16], + buffer1[20], + buffer1[27], + out buffer0[20], + out buffer0[27], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[48], + -cospi[16], + buffer1[21], + buffer1[26], + out buffer0[21], + out buffer0[26], + cosBit, + in rounding); + + // Stage 5 completes the low-frequency DCT and rotates the first separated odd groups. Final coefficients are + // retired directly to the block instead of being copied through a third workspace. + ButterflyStore(cospi[32], cospi[32], buffer0[0], buffer0[1], ref values, outputStride, 0, 16, cosBit, in rounding); + ButterflyStore(cospi[16], cospi[48], buffer0[3], buffer0[2], ref values, outputStride, 8, 24, cosBit, in rounding); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[4], buffer0[5], out buffer1[4], out buffer1[5]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[7], buffer0[6], out buffer1[7], out buffer1[6]); + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[16], + cospi[48], + buffer0[9], + buffer0[14], + out buffer1[9], + out buffer1[14], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[48], + -cospi[16], + buffer0[10], + buffer0[13], + out buffer1[10], + out buffer1[13], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[16], buffer0[19], out buffer1[16], out buffer1[19]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[17], buffer0[18], out buffer1[17], out buffer1[18]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[23], buffer0[20], out buffer1[23], out buffer1[20]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[22], buffer0[21], out buffer1[22], out buffer1[21]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[24], buffer0[27], out buffer1[24], out buffer1[27]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[25], buffer0[26], out buffer1[25], out buffer1[26]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[31], buffer0[28], out buffer1[31], out buffer1[28]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[30], buffer0[29], out buffer1[30], out buffer1[29]); + + // Stage 6 merges adjacent odd-frequency terms with the sign pattern required by the next rotations. + ButterflyStore(cospi[8], cospi[56], buffer1[7], buffer1[4], ref values, outputStride, 4, 28, cosBit, in rounding); + ButterflyStore(cospi[40], cospi[24], buffer1[6], buffer1[5], ref values, outputStride, 20, 12, cosBit, in rounding); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[8], buffer1[9], out buffer0[8], out buffer0[9]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[11], buffer1[10], out buffer0[11], out buffer0[10]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[12], buffer1[13], out buffer0[12], out buffer0[13]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[15], buffer1[14], out buffer0[15], out buffer0[14]); + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[8], + cospi[56], + buffer1[17], + buffer1[30], + out buffer0[17], + out buffer0[30], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[56], + -cospi[8], + buffer1[18], + buffer1[29], + out buffer0[18], + out buffer0[29], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[40], + cospi[24], + buffer1[21], + buffer1[26], + out buffer0[21], + out buffer0[26], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[24], + -cospi[40], + buffer1[22], + buffer1[25], + out buffer0[22], + out buffer0[25], + cosBit, + in rounding); + + // Stage 7 applies the pi/32 rotations to the next odd-frequency level. + ButterflyStore(cospi[4], cospi[60], buffer0[15], buffer0[8], ref values, outputStride, 2, 30, cosBit, in rounding); + ButterflyStore(cospi[36], cospi[28], buffer0[14], buffer0[9], ref values, outputStride, 18, 14, cosBit, in rounding); + ButterflyStore(cospi[20], cospi[44], buffer0[13], buffer0[10], ref values, outputStride, 10, 22, cosBit, in rounding); + ButterflyStore(cospi[52], cospi[12], buffer0[12], buffer0[11], ref values, outputStride, 26, 6, cosBit, in rounding); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[16], buffer0[17], out buffer1[16], out buffer1[17]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[19], buffer0[18], out buffer1[19], out buffer1[18]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[20], buffer0[21], out buffer1[20], out buffer1[21]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[23], buffer0[22], out buffer1[23], out buffer1[22]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[24], buffer0[25], out buffer1[24], out buffer1[25]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[27], buffer0[26], out buffer1[27], out buffer1[26]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[28], buffer0[29], out buffer1[28], out buffer1[29]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[31], buffer0[30], out buffer1[31], out buffer1[30]); + + // Stages 8 and 9 fuse the terminal pi/64 rotations with the output permutation because none of their results + // are consumed by another arithmetic stage. + ButterflyStore(cospi[2], cospi[62], buffer1[31], buffer1[16], ref values, outputStride, 1, 31, cosBit, in rounding); + ButterflyStore(cospi[34], cospi[30], buffer1[30], buffer1[17], ref values, outputStride, 17, 15, cosBit, in rounding); + ButterflyStore(cospi[18], cospi[46], buffer1[29], buffer1[18], ref values, outputStride, 9, 23, cosBit, in rounding); + ButterflyStore(cospi[50], cospi[14], buffer1[28], buffer1[19], ref values, outputStride, 25, 7, cosBit, in rounding); + ButterflyStore(cospi[10], cospi[54], buffer1[27], buffer1[20], ref values, outputStride, 5, 27, cosBit, in rounding); + ButterflyStore(cospi[42], cospi[22], buffer1[26], buffer1[21], ref values, outputStride, 21, 11, cosBit, in rounding); + ButterflyStore(cospi[26], cospi[38], buffer1[25], buffer1[22], ref values, outputStride, 13, 19, cosBit, in rounding); + ButterflyStore(cospi[58], cospi[6], buffer1[24], buffer1[23], ref values, outputStride, 29, 3, cosBit, in rounding); + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + Av1TransformRounding rounding = Av1ForwardTransformArithmetic>.CreateRounding(cosBit); + + // Stage 1 consumes the source block completely before any final coefficient is stored back into it. + for (int i = 0; i < 16; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract( + Load>(ref values, inputStride, i), + Load>(ref values, inputStride, 31 - i), + out buffer1[i], + out buffer1[31 - i]); + } + + // Stage 2 starts the recursive radix-2 factorization and rotates the central odd-frequency pairs. + for (int i = 0; i < 8; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[i], buffer1[15 - i], out buffer0[i], out buffer0[15 - i]); + } + + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[32], + cospi[32], + buffer1[20 + i], + buffer1[27 - i], + out buffer0[20 + i], + out buffer0[27 - i], + cosBit, + in rounding); + } + + // Stage 3 reduces the even half and folds the next odd-frequency groups into paired sums and differences. + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[i], buffer0[7 - i], out buffer1[i], out buffer1[7 - i]); + } + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[32], + cospi[32], + buffer0[10], + buffer0[13], + out buffer1[10], + out buffer1[13], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[32], + cospi[32], + buffer0[11], + buffer0[12], + out buffer1[11], + out buffer1[12], + cosBit, + in rounding); + + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[16 + i], buffer0[23 - i], out buffer1[16 + i], out buffer1[23 - i]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[31 - i], buffer0[24 + i], out buffer1[31 - i], out buffer1[24 + i]); + } + + // Stage 4 continues the factorization as independent eight-value groups. + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[i], buffer1[3 - i], out buffer0[i], out buffer0[3 - i]); + } + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[32], + cospi[32], + buffer1[5], + buffer1[6], + out buffer0[5], + out buffer0[6], + cosBit, + in rounding); + + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[8 + i], buffer1[11 - i], out buffer0[8 + i], out buffer0[11 - i]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[15 - i], buffer1[12 + i], out buffer0[15 - i], out buffer0[12 + i]); + } + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[16], + cospi[48], + buffer1[18], + buffer1[29], + out buffer0[18], + out buffer0[29], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[16], + cospi[48], + buffer1[19], + buffer1[28], + out buffer0[19], + out buffer0[28], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[48], + -cospi[16], + buffer1[20], + buffer1[27], + out buffer0[20], + out buffer0[27], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[48], + -cospi[16], + buffer1[21], + buffer1[26], + out buffer0[21], + out buffer0[26], + cosBit, + in rounding); + + // Stage 5 completes the low-frequency DCT and rotates the first separated odd groups. Final coefficients are + // retired directly to the block instead of being copied through a third workspace. + ButterflyStore(cospi[32], cospi[32], buffer0[0], buffer0[1], ref values, outputStride, 0, 16, cosBit, in rounding); + ButterflyStore(cospi[16], cospi[48], buffer0[3], buffer0[2], ref values, outputStride, 8, 24, cosBit, in rounding); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[4], buffer0[5], out buffer1[4], out buffer1[5]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[7], buffer0[6], out buffer1[7], out buffer1[6]); + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[16], + cospi[48], + buffer0[9], + buffer0[14], + out buffer1[9], + out buffer1[14], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[48], + -cospi[16], + buffer0[10], + buffer0[13], + out buffer1[10], + out buffer1[13], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[16], buffer0[19], out buffer1[16], out buffer1[19]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[17], buffer0[18], out buffer1[17], out buffer1[18]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[23], buffer0[20], out buffer1[23], out buffer1[20]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[22], buffer0[21], out buffer1[22], out buffer1[21]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[24], buffer0[27], out buffer1[24], out buffer1[27]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[25], buffer0[26], out buffer1[25], out buffer1[26]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[31], buffer0[28], out buffer1[31], out buffer1[28]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[30], buffer0[29], out buffer1[30], out buffer1[29]); + + // Stage 6 merges adjacent odd-frequency terms with the sign pattern required by the next rotations. + ButterflyStore(cospi[8], cospi[56], buffer1[7], buffer1[4], ref values, outputStride, 4, 28, cosBit, in rounding); + ButterflyStore(cospi[40], cospi[24], buffer1[6], buffer1[5], ref values, outputStride, 20, 12, cosBit, in rounding); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[8], buffer1[9], out buffer0[8], out buffer0[9]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[11], buffer1[10], out buffer0[11], out buffer0[10]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[12], buffer1[13], out buffer0[12], out buffer0[13]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[15], buffer1[14], out buffer0[15], out buffer0[14]); + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[8], + cospi[56], + buffer1[17], + buffer1[30], + out buffer0[17], + out buffer0[30], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[56], + -cospi[8], + buffer1[18], + buffer1[29], + out buffer0[18], + out buffer0[29], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[40], + cospi[24], + buffer1[21], + buffer1[26], + out buffer0[21], + out buffer0[26], + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[24], + -cospi[40], + buffer1[22], + buffer1[25], + out buffer0[22], + out buffer0[25], + cosBit, + in rounding); + + // Stage 7 applies the pi/32 rotations to the next odd-frequency level. + ButterflyStore(cospi[4], cospi[60], buffer0[15], buffer0[8], ref values, outputStride, 2, 30, cosBit, in rounding); + ButterflyStore(cospi[36], cospi[28], buffer0[14], buffer0[9], ref values, outputStride, 18, 14, cosBit, in rounding); + ButterflyStore(cospi[20], cospi[44], buffer0[13], buffer0[10], ref values, outputStride, 10, 22, cosBit, in rounding); + ButterflyStore(cospi[52], cospi[12], buffer0[12], buffer0[11], ref values, outputStride, 26, 6, cosBit, in rounding); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[16], buffer0[17], out buffer1[16], out buffer1[17]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[19], buffer0[18], out buffer1[19], out buffer1[18]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[20], buffer0[21], out buffer1[20], out buffer1[21]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[23], buffer0[22], out buffer1[23], out buffer1[22]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[24], buffer0[25], out buffer1[24], out buffer1[25]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[27], buffer0[26], out buffer1[27], out buffer1[26]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[28], buffer0[29], out buffer1[28], out buffer1[29]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[31], buffer0[30], out buffer1[31], out buffer1[30]); + + // Stages 8 and 9 fuse the terminal pi/64 rotations with the output permutation because none of their results + // are consumed by another arithmetic stage. + ButterflyStore(cospi[2], cospi[62], buffer1[31], buffer1[16], ref values, outputStride, 1, 31, cosBit, in rounding); + ButterflyStore(cospi[34], cospi[30], buffer1[30], buffer1[17], ref values, outputStride, 17, 15, cosBit, in rounding); + ButterflyStore(cospi[18], cospi[46], buffer1[29], buffer1[18], ref values, outputStride, 9, 23, cosBit, in rounding); + ButterflyStore(cospi[50], cospi[14], buffer1[28], buffer1[19], ref values, outputStride, 25, 7, cosBit, in rounding); + ButterflyStore(cospi[10], cospi[54], buffer1[27], buffer1[20], ref values, outputStride, 5, 27, cosBit, in rounding); + ButterflyStore(cospi[42], cospi[22], buffer1[26], buffer1[21], ref values, outputStride, 21, 11, cosBit, in rounding); + ButterflyStore(cospi[26], cospi[38], buffer1[25], buffer1[22], ref values, outputStride, 13, 19, cosBit, in rounding); + ButterflyStore(cospi[58], cospi[6], buffer1[24], buffer1[23], ref values, outputStride, 29, 3, cosBit, in rounding); + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Transform/Forward/Av1ForwardTransformer.Dct4Operator.cs b/src/ImageSharp/Formats/Heif/Av1/Transform/Forward/Av1ForwardTransformer.Dct4Operator.cs new file mode 100644 index 0000000000..2987fc3db2 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Transform/Forward/Av1ForwardTransformer.Dct4Operator.cs @@ -0,0 +1,395 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.Intrinsics; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform.Forward; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +/// +/// Defines the four-point forward DCT operator. +/// +internal static partial class Av1ForwardTransformer +{ + /// + /// Implements the four-point forward discrete cosine transform. + /// + internal readonly struct Dct4Operator : IAv1ForwardTransform1dOperator + { + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector buffer0, + ref Av1TransformVector buffer1, + int cosBit) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + Av1TransformRounding rounding = Av1ForwardTransformArithmetic.CreateRounding(cosBit); + + int input0 = Load(ref values, inputStride, 0); + int input1 = Load(ref values, inputStride, 1); + int input2 = Load(ref values, inputStride, 2); + int input3 = Load(ref values, inputStride, 3); + + // The paired stage keeps the axes in their native lane representation. Packed short lanes therefore retain + // Highway's saturating add/subtract behavior before the widening butterfly multiplication. + Av1ForwardTransformArithmetic.AddSubtract(input0, input3, out buffer0[0], out buffer0[3]); + Av1ForwardTransformArithmetic.AddSubtract(input1, input2, out buffer0[1], out buffer0[2]); + Av1ForwardTransformArithmetic.Butterfly( + cospi[32], + cospi[32], + buffer0[0], + buffer0[1], + out int output0, + out int output2, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic.Butterfly( + cospi[16], + cospi[48], + buffer0[3], + buffer0[2], + out int output1, + out int output3, + cosBit, + in rounding); + + Store(ref values, outputStride, 0, output0); + Store(ref values, outputStride, 1, output1); + Store(ref values, outputStride, 2, output2); + Store(ref values, outputStride, 3, output3); + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector buffer0, + ref Av1TransformVector buffer1, + int cosBit) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + Av1TransformRounding rounding = Av1ForwardTransformArithmetic.CreateRounding(cosBit); + + short input0 = Load(ref values, inputStride, 0); + short input1 = Load(ref values, inputStride, 1); + short input2 = Load(ref values, inputStride, 2); + short input3 = Load(ref values, inputStride, 3); + + // The paired stage keeps the axes in their native lane representation. Packed short lanes therefore retain + // Highway's saturating add/subtract behavior before the widening butterfly multiplication. + Av1ForwardTransformArithmetic.AddSubtract(input0, input3, out buffer0[0], out buffer0[3]); + Av1ForwardTransformArithmetic.AddSubtract(input1, input2, out buffer0[1], out buffer0[2]); + Av1ForwardTransformArithmetic.Butterfly( + cospi[32], + cospi[32], + buffer0[0], + buffer0[1], + out short output0, + out short output2, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic.Butterfly( + cospi[16], + cospi[48], + buffer0[3], + buffer0[2], + out short output1, + out short output3, + cosBit, + in rounding); + + Store(ref values, outputStride, 0, output0); + Store(ref values, outputStride, 1, output1); + Store(ref values, outputStride, 2, output2); + Store(ref values, outputStride, 3, output3); + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + Av1TransformRounding rounding = Av1ForwardTransformArithmetic>.CreateRounding(cosBit); + + Vector128 input0 = Load>(ref values, inputStride, 0); + Vector128 input1 = Load>(ref values, inputStride, 1); + Vector128 input2 = Load>(ref values, inputStride, 2); + Vector128 input3 = Load>(ref values, inputStride, 3); + + // The paired stage keeps the axes in their native lane representation. Packed short lanes therefore retain + // Highway's saturating add/subtract behavior before the widening butterfly multiplication. + Av1ForwardTransformArithmetic>.AddSubtract(input0, input3, out buffer0[0], out buffer0[3]); + Av1ForwardTransformArithmetic>.AddSubtract(input1, input2, out buffer0[1], out buffer0[2]); + Av1ForwardTransformArithmetic>.Butterfly( + cospi[32], + cospi[32], + buffer0[0], + buffer0[1], + out Vector128 output0, + out Vector128 output2, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + cospi[16], + cospi[48], + buffer0[3], + buffer0[2], + out Vector128 output1, + out Vector128 output3, + cosBit, + in rounding); + + Store(ref values, outputStride, 0, output0); + Store(ref values, outputStride, 1, output1); + Store(ref values, outputStride, 2, output2); + Store(ref values, outputStride, 3, output3); + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + Av1TransformRounding rounding = Av1ForwardTransformArithmetic>.CreateRounding(cosBit); + + Vector256 input0 = Load>(ref values, inputStride, 0); + Vector256 input1 = Load>(ref values, inputStride, 1); + Vector256 input2 = Load>(ref values, inputStride, 2); + Vector256 input3 = Load>(ref values, inputStride, 3); + + // The paired stage keeps the axes in their native lane representation. Packed short lanes therefore retain + // Highway's saturating add/subtract behavior before the widening butterfly multiplication. + Av1ForwardTransformArithmetic>.AddSubtract(input0, input3, out buffer0[0], out buffer0[3]); + Av1ForwardTransformArithmetic>.AddSubtract(input1, input2, out buffer0[1], out buffer0[2]); + Av1ForwardTransformArithmetic>.Butterfly( + cospi[32], + cospi[32], + buffer0[0], + buffer0[1], + out Vector256 output0, + out Vector256 output2, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + cospi[16], + cospi[48], + buffer0[3], + buffer0[2], + out Vector256 output1, + out Vector256 output3, + cosBit, + in rounding); + + Store(ref values, outputStride, 0, output0); + Store(ref values, outputStride, 1, output1); + Store(ref values, outputStride, 2, output2); + Store(ref values, outputStride, 3, output3); + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + Av1TransformRounding rounding = Av1ForwardTransformArithmetic>.CreateRounding(cosBit); + + Vector512 input0 = Load>(ref values, inputStride, 0); + Vector512 input1 = Load>(ref values, inputStride, 1); + Vector512 input2 = Load>(ref values, inputStride, 2); + Vector512 input3 = Load>(ref values, inputStride, 3); + + // The paired stage keeps the axes in their native lane representation. Packed short lanes therefore retain + // Highway's saturating add/subtract behavior before the widening butterfly multiplication. + Av1ForwardTransformArithmetic>.AddSubtract(input0, input3, out buffer0[0], out buffer0[3]); + Av1ForwardTransformArithmetic>.AddSubtract(input1, input2, out buffer0[1], out buffer0[2]); + Av1ForwardTransformArithmetic>.Butterfly( + cospi[32], + cospi[32], + buffer0[0], + buffer0[1], + out Vector512 output0, + out Vector512 output2, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + cospi[16], + cospi[48], + buffer0[3], + buffer0[2], + out Vector512 output1, + out Vector512 output3, + cosBit, + in rounding); + + Store(ref values, outputStride, 0, output0); + Store(ref values, outputStride, 1, output1); + Store(ref values, outputStride, 2, output2); + Store(ref values, outputStride, 3, output3); + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + Av1TransformRounding rounding = Av1ForwardTransformArithmetic>.CreateRounding(cosBit); + + Vector128 input0 = Load>(ref values, inputStride, 0); + Vector128 input1 = Load>(ref values, inputStride, 1); + Vector128 input2 = Load>(ref values, inputStride, 2); + Vector128 input3 = Load>(ref values, inputStride, 3); + + // The paired stage keeps the axes in their native lane representation. Packed short lanes therefore retain + // Highway's saturating add/subtract behavior before the widening butterfly multiplication. + Av1ForwardTransformArithmetic>.AddSubtract(input0, input3, out buffer0[0], out buffer0[3]); + Av1ForwardTransformArithmetic>.AddSubtract(input1, input2, out buffer0[1], out buffer0[2]); + Av1ForwardTransformArithmetic>.Butterfly( + cospi[32], + cospi[32], + buffer0[0], + buffer0[1], + out Vector128 output0, + out Vector128 output2, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + cospi[16], + cospi[48], + buffer0[3], + buffer0[2], + out Vector128 output1, + out Vector128 output3, + cosBit, + in rounding); + + Store(ref values, outputStride, 0, output0); + Store(ref values, outputStride, 1, output1); + Store(ref values, outputStride, 2, output2); + Store(ref values, outputStride, 3, output3); + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + Av1TransformRounding rounding = Av1ForwardTransformArithmetic>.CreateRounding(cosBit); + + Vector256 input0 = Load>(ref values, inputStride, 0); + Vector256 input1 = Load>(ref values, inputStride, 1); + Vector256 input2 = Load>(ref values, inputStride, 2); + Vector256 input3 = Load>(ref values, inputStride, 3); + + // The paired stage keeps the axes in their native lane representation. Packed short lanes therefore retain + // Highway's saturating add/subtract behavior before the widening butterfly multiplication. + Av1ForwardTransformArithmetic>.AddSubtract(input0, input3, out buffer0[0], out buffer0[3]); + Av1ForwardTransformArithmetic>.AddSubtract(input1, input2, out buffer0[1], out buffer0[2]); + Av1ForwardTransformArithmetic>.Butterfly( + cospi[32], + cospi[32], + buffer0[0], + buffer0[1], + out Vector256 output0, + out Vector256 output2, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + cospi[16], + cospi[48], + buffer0[3], + buffer0[2], + out Vector256 output1, + out Vector256 output3, + cosBit, + in rounding); + + Store(ref values, outputStride, 0, output0); + Store(ref values, outputStride, 1, output1); + Store(ref values, outputStride, 2, output2); + Store(ref values, outputStride, 3, output3); + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + Av1TransformRounding rounding = Av1ForwardTransformArithmetic>.CreateRounding(cosBit); + + Vector512 input0 = Load>(ref values, inputStride, 0); + Vector512 input1 = Load>(ref values, inputStride, 1); + Vector512 input2 = Load>(ref values, inputStride, 2); + Vector512 input3 = Load>(ref values, inputStride, 3); + + // The paired stage keeps the axes in their native lane representation. Packed short lanes therefore retain + // Highway's saturating add/subtract behavior before the widening butterfly multiplication. + Av1ForwardTransformArithmetic>.AddSubtract(input0, input3, out buffer0[0], out buffer0[3]); + Av1ForwardTransformArithmetic>.AddSubtract(input1, input2, out buffer0[1], out buffer0[2]); + Av1ForwardTransformArithmetic>.Butterfly( + cospi[32], + cospi[32], + buffer0[0], + buffer0[1], + out Vector512 output0, + out Vector512 output2, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + cospi[16], + cospi[48], + buffer0[3], + buffer0[2], + out Vector512 output1, + out Vector512 output3, + cosBit, + in rounding); + + Store(ref values, outputStride, 0, output0); + Store(ref values, outputStride, 1, output1); + Store(ref values, outputStride, 2, output2); + Store(ref values, outputStride, 3, output3); + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Transform/Forward/Av1ForwardTransformer.Dct64Operator.cs b/src/ImageSharp/Formats/Heif/Av1/Transform/Forward/Av1ForwardTransformer.Dct64Operator.cs new file mode 100644 index 0000000000..b4f52dfeda --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Transform/Forward/Av1ForwardTransformer.Dct64Operator.cs @@ -0,0 +1,1929 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.Intrinsics; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform.Forward; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +/// +/// Implements the sixty-four-point forward DCT stage network. +/// +internal static partial class Av1ForwardTransformer +{ + /// + /// Implements the sixty-four-point forward transform for every supported lane width. + /// + internal readonly struct Dct64Operator : IAv1ForwardTransform1dOperator + { + /// + /// Identifies coefficient positions whose final value resides in the first stage buffer. + /// + private const ulong Dct64Buffer0OutputMask = + (1UL << 2) | (1UL << 6) | (1UL << 8) | (1UL << 10) | (1UL << 14) | + (1UL << 18) | (1UL << 22) | (1UL << 24) | (1UL << 26) | (1UL << 30) | + (1UL << 34) | (1UL << 38) | (1UL << 40) | (1UL << 42) | (1UL << 46) | + (1UL << 50) | (1UL << 54) | (1UL << 56) | (1UL << 58) | (1UL << 62); + + /// + /// Gets the stage-nine rotation order for the middle quarter of the transform. + /// + private static ReadOnlySpan Dct64Stage9RotationOrder => [2, 34, 18, 50, 10, 42, 26, 58]; + + /// + /// Gets the stage-ten rotation order for the upper half of the transform. + /// + private static ReadOnlySpan Dct64Stage10RotationOrder => [1, 33, 17, 49, 9, 41, 25, 57, 5, 37, 21, 53, 13, 45, 29, 61]; + + /// + /// Gets the mapping from coefficient order to the final staged value. + /// + private static ReadOnlySpan Dct64OutputOrder => + [ + 0, 32, 16, 48, 8, 40, 24, 56, 4, 36, 20, 52, 12, 44, 28, 60, + 2, 34, 18, 50, 10, 42, 26, 58, 6, 38, 22, 54, 14, 46, 30, 62, + 1, 33, 17, 49, 9, 41, 25, 57, 5, 37, 21, 53, 13, 45, 29, 61, + 3, 35, 19, 51, 11, 43, 27, 59, 7, 39, 23, 55, 15, 47, 31, 63, + ]; + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector buffer0, + ref Av1TransformVector buffer1, + int cosBit) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + Av1TransformRounding rounding = Av1ForwardTransformArithmetic.CreateRounding(cosBit); + + // Stage 1 consumes every spatial value before the strided block becomes available for final coefficients. + for (int i = 0; i < 32; i++) + { + Av1ForwardTransformArithmetic.AddSubtract( + Load(ref values, inputStride, i), + Load(ref values, inputStride, 63 - i), + out buffer0[i], + out buffer0[63 - i]); + } + + // Stage 2 begins the recursive radix-2 factorization and rotates the central odd-frequency pairs. + for (int i = 0; i < 16; i++) + { + Av1ForwardTransformArithmetic.AddSubtract(buffer0[i], buffer0[31 - i], out buffer1[i], out buffer1[31 - i]); + } + + for (int i = 0; i < 8; i++) + { + Butterfly(-cospi[32], cospi[32], buffer0[40 + i], buffer0[55 - i], ref buffer1, 40 + i, 55 - i, cosBit, in rounding); + } + + // Stage 3 reduces the even half and folds the next odd-frequency groups into paired sums and differences. + for (int i = 0; i < 8; i++) + { + Av1ForwardTransformArithmetic.AddSubtract(buffer1[i], buffer1[15 - i], out buffer0[i], out buffer0[15 - i]); + } + + for (int i = 0; i < 4; i++) + { + Butterfly(-cospi[32], cospi[32], buffer1[20 + i], buffer1[27 - i], ref buffer0, 20 + i, 27 - i, cosBit, in rounding); + } + + for (int i = 0; i < 8; i++) + { + Av1ForwardTransformArithmetic.AddSubtract(buffer0[32 + i], buffer1[47 - i], out buffer0[32 + i], out buffer0[47 - i]); + } + + for (int i = 0; i < 8; i++) + { + Av1ForwardTransformArithmetic.AddSubtract(buffer0[63 - i], buffer1[48 + i], out buffer0[63 - i], out buffer0[48 + i]); + } + + // Stage 4 continues the factorization as independent sixteen-value groups. + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic.AddSubtract(buffer0[i], buffer0[7 - i], out buffer1[i], out buffer1[7 - i]); + } + + for (int i = 0; i < 2; i++) + { + Butterfly(-cospi[32], cospi[32], buffer0[10 + i], buffer0[13 - i], ref buffer1, 10 + i, 13 - i, cosBit, in rounding); + } + + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic.AddSubtract(buffer1[16 + i], buffer0[23 - i], out buffer1[16 + i], out buffer1[23 - i]); + Av1ForwardTransformArithmetic.AddSubtract(buffer1[31 - i], buffer0[24 + i], out buffer1[31 - i], out buffer1[24 + i]); + Butterfly(-cospi[16], cospi[48], buffer0[36 + i], buffer0[59 - i], ref buffer1, 36 + i, 59 - i, cosBit, in rounding); + } + + for (int i = 4; i < 8; i++) + { + Butterfly(-cospi[48], -cospi[16], buffer0[36 + i], buffer0[59 - i], ref buffer1, 36 + i, 59 - i, cosBit, in rounding); + } + + // Stage 5 reduces the sixteen-value groups into the eight-value DCT and ADST building blocks. + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic.AddSubtract(buffer1[i], buffer1[3 - i], out buffer0[i], out buffer0[3 - i]); + } + + Butterfly(-cospi[32], cospi[32], buffer1[5], buffer1[6], ref buffer0, 5, 6, cosBit, in rounding); + + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic.AddSubtract(buffer0[8 + i], buffer1[11 - i], out buffer0[8 + i], out buffer0[11 - i]); + Av1ForwardTransformArithmetic.AddSubtract(buffer0[15 - i], buffer1[12 + i], out buffer0[15 - i], out buffer0[12 + i]); + Butterfly(-cospi[16], cospi[48], buffer1[18 + i], buffer1[29 - i], ref buffer0, 18 + i, 29 - i, cosBit, in rounding); + } + + for (int i = 2; i < 4; i++) + { + Butterfly(-cospi[48], -cospi[16], buffer1[18 + i], buffer1[29 - i], ref buffer0, 18 + i, 29 - i, cosBit, in rounding); + } + + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic.AddSubtract(buffer0[32 + i], buffer1[39 - i], out buffer0[32 + i], out buffer0[39 - i]); + Av1ForwardTransformArithmetic.AddSubtract(buffer0[47 - i], buffer1[40 + i], out buffer0[47 - i], out buffer0[40 + i]); + Av1ForwardTransformArithmetic.AddSubtract(buffer0[48 + i], buffer1[55 - i], out buffer0[48 + i], out buffer0[55 - i]); + Av1ForwardTransformArithmetic.AddSubtract(buffer0[63 - i], buffer1[56 + i], out buffer0[63 - i], out buffer0[56 + i]); + } + + // Stage 6 completes the low-frequency DCT and rotates the first separated odd-frequency groups. + Butterfly(cospi[32], cospi[32], buffer0[0], buffer0[1], ref buffer1, 0, 1, cosBit, in rounding); + Butterfly(cospi[16], cospi[48], buffer0[3], buffer0[2], ref buffer1, 2, 3, cosBit, in rounding); + Av1ForwardTransformArithmetic.AddSubtract(buffer1[4], buffer0[5], out buffer1[4], out buffer1[5]); + Av1ForwardTransformArithmetic.AddSubtract(buffer1[7], buffer0[6], out buffer1[7], out buffer1[6]); + Butterfly(-cospi[16], cospi[48], buffer0[9], buffer0[14], ref buffer1, 9, 14, cosBit, in rounding); + Butterfly(-cospi[48], -cospi[16], buffer0[10], buffer0[13], ref buffer1, 10, 13, cosBit, in rounding); + + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic.AddSubtract(buffer1[16 + i], buffer0[19 - i], out buffer1[16 + i], out buffer1[19 - i]); + Av1ForwardTransformArithmetic.AddSubtract(buffer1[23 - i], buffer0[20 + i], out buffer1[23 - i], out buffer1[20 + i]); + Av1ForwardTransformArithmetic.AddSubtract(buffer1[24 + i], buffer0[27 - i], out buffer1[24 + i], out buffer1[27 - i]); + Av1ForwardTransformArithmetic.AddSubtract(buffer1[31 - i], buffer0[28 + i], out buffer1[31 - i], out buffer1[28 + i]); + Butterfly(-cospi[8], cospi[56], buffer0[34 + i], buffer0[61 - i], ref buffer1, 34 + i, 61 - i, cosBit, in rounding); + Butterfly(-cospi[40], cospi[24], buffer0[42 + i], buffer0[53 - i], ref buffer1, 42 + i, 53 - i, cosBit, in rounding); + } + + for (int i = 2; i < 4; i++) + { + Butterfly(-cospi[56], -cospi[8], buffer0[34 + i], buffer0[61 - i], ref buffer1, 34 + i, 61 - i, cosBit, in rounding); + Butterfly(-cospi[24], -cospi[40], buffer0[42 + i], buffer0[53 - i], ref buffer1, 42 + i, 53 - i, cosBit, in rounding); + } + + // Stage 7 merges adjacent odd-frequency terms with the sign pattern required by the next rotations. + Butterfly(cospi[8], cospi[56], buffer1[7], buffer1[4], ref buffer0, 4, 7, cosBit, in rounding); + Butterfly(cospi[40], cospi[24], buffer1[6], buffer1[5], ref buffer0, 5, 6, cosBit, in rounding); + Av1ForwardTransformArithmetic.AddSubtract(buffer0[8], buffer1[9], out buffer0[8], out buffer0[9]); + Av1ForwardTransformArithmetic.AddSubtract(buffer0[11], buffer1[10], out buffer0[11], out buffer0[10]); + Av1ForwardTransformArithmetic.AddSubtract(buffer0[12], buffer1[13], out buffer0[12], out buffer0[13]); + Av1ForwardTransformArithmetic.AddSubtract(buffer0[15], buffer1[14], out buffer0[15], out buffer0[14]); + Butterfly(-cospi[8], cospi[56], buffer1[17], buffer1[30], ref buffer0, 17, 30, cosBit, in rounding); + Butterfly(-cospi[56], -cospi[8], buffer1[18], buffer1[29], ref buffer0, 18, 29, cosBit, in rounding); + Butterfly(-cospi[40], cospi[24], buffer1[21], buffer1[26], ref buffer0, 21, 26, cosBit, in rounding); + Butterfly(-cospi[24], -cospi[40], buffer1[22], buffer1[25], ref buffer0, 22, 25, cosBit, in rounding); + + for (int group = 0; group < 4; group++) + { + int offset = group * 8; + + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic.AddSubtract( + buffer0[32 + offset + i], + buffer1[35 + offset - i], + out buffer0[32 + offset + i], + out buffer0[35 + offset - i]); + + Av1ForwardTransformArithmetic.AddSubtract( + buffer0[39 + offset - i], + buffer1[36 + offset + i], + out buffer0[39 + offset - i], + out buffer0[36 + offset + i]); + } + } + + // Stage 8 applies the next level of odd-frequency rotations. + Butterfly(cospi[4], cospi[60], buffer0[15], buffer0[8], ref buffer1, 8, 15, cosBit, in rounding); + Butterfly(cospi[36], cospi[28], buffer0[14], buffer0[9], ref buffer1, 9, 14, cosBit, in rounding); + Butterfly(cospi[20], cospi[44], buffer0[13], buffer0[10], ref buffer1, 10, 13, cosBit, in rounding); + Butterfly(cospi[52], cospi[12], buffer0[12], buffer0[11], ref buffer1, 11, 12, cosBit, in rounding); + + Av1ForwardTransformArithmetic.AddSubtract(buffer1[16], buffer0[17], out buffer1[16], out buffer1[17]); + Av1ForwardTransformArithmetic.AddSubtract(buffer1[19], buffer0[18], out buffer1[19], out buffer1[18]); + Av1ForwardTransformArithmetic.AddSubtract(buffer1[20], buffer0[21], out buffer1[20], out buffer1[21]); + Av1ForwardTransformArithmetic.AddSubtract(buffer1[23], buffer0[22], out buffer1[23], out buffer1[22]); + Av1ForwardTransformArithmetic.AddSubtract(buffer1[24], buffer0[25], out buffer1[24], out buffer1[25]); + Av1ForwardTransformArithmetic.AddSubtract(buffer1[27], buffer0[26], out buffer1[27], out buffer1[26]); + Av1ForwardTransformArithmetic.AddSubtract(buffer1[28], buffer0[29], out buffer1[28], out buffer1[29]); + Av1ForwardTransformArithmetic.AddSubtract(buffer1[31], buffer0[30], out buffer1[31], out buffer1[30]); + + Butterfly(-cospi[4], cospi[60], buffer0[33], buffer0[62], ref buffer1, 33, 62, cosBit, in rounding); + Butterfly(-cospi[60], -cospi[4], buffer0[34], buffer0[61], ref buffer1, 34, 61, cosBit, in rounding); + Butterfly(-cospi[36], cospi[28], buffer0[37], buffer0[58], ref buffer1, 37, 58, cosBit, in rounding); + Butterfly(-cospi[28], -cospi[36], buffer0[38], buffer0[57], ref buffer1, 38, 57, cosBit, in rounding); + Butterfly(-cospi[20], cospi[44], buffer0[41], buffer0[54], ref buffer1, 41, 54, cosBit, in rounding); + Butterfly(-cospi[44], -cospi[20], buffer0[42], buffer0[53], ref buffer1, 42, 53, cosBit, in rounding); + Butterfly(-cospi[52], cospi[12], buffer0[45], buffer0[50], ref buffer1, 45, 50, cosBit, in rounding); + Butterfly(-cospi[12], -cospi[52], buffer0[46], buffer0[49], ref buffer1, 46, 49, cosBit, in rounding); + + // Stage 9 merges the remaining odd-frequency pairs before their terminal rotations. The table preserves the + // non-linear rotation order while keeping the constants in compile-time data. + for (int i = 0; i < 8; i++) + { + int low = 16 + i; + int high = 31 - i; + int odd = Dct64Stage9RotationOrder[i]; + Butterfly(cospi[odd], cospi[64 - odd], buffer1[high], buffer1[low], ref buffer0, low, high, cosBit, in rounding); + } + + Av1ForwardTransformArithmetic.AddSubtract(buffer0[32], buffer1[33], out buffer0[32], out buffer0[33]); + Av1ForwardTransformArithmetic.AddSubtract(buffer0[35], buffer1[34], out buffer0[35], out buffer0[34]); + Av1ForwardTransformArithmetic.AddSubtract(buffer0[36], buffer1[37], out buffer0[36], out buffer0[37]); + Av1ForwardTransformArithmetic.AddSubtract(buffer0[39], buffer1[38], out buffer0[39], out buffer0[38]); + Av1ForwardTransformArithmetic.AddSubtract(buffer0[40], buffer1[41], out buffer0[40], out buffer0[41]); + Av1ForwardTransformArithmetic.AddSubtract(buffer0[43], buffer1[42], out buffer0[43], out buffer0[42]); + Av1ForwardTransformArithmetic.AddSubtract(buffer0[44], buffer1[45], out buffer0[44], out buffer0[45]); + Av1ForwardTransformArithmetic.AddSubtract(buffer0[47], buffer1[46], out buffer0[47], out buffer0[46]); + Av1ForwardTransformArithmetic.AddSubtract(buffer0[48], buffer1[49], out buffer0[48], out buffer0[49]); + Av1ForwardTransformArithmetic.AddSubtract(buffer0[51], buffer1[50], out buffer0[51], out buffer0[50]); + Av1ForwardTransformArithmetic.AddSubtract(buffer0[52], buffer1[53], out buffer0[52], out buffer0[53]); + Av1ForwardTransformArithmetic.AddSubtract(buffer0[55], buffer1[54], out buffer0[55], out buffer0[54]); + Av1ForwardTransformArithmetic.AddSubtract(buffer0[56], buffer1[57], out buffer0[56], out buffer0[57]); + Av1ForwardTransformArithmetic.AddSubtract(buffer0[59], buffer1[58], out buffer0[59], out buffer0[58]); + Av1ForwardTransformArithmetic.AddSubtract(buffer0[60], buffer1[61], out buffer0[60], out buffer0[61]); + Av1ForwardTransformArithmetic.AddSubtract(buffer0[63], buffer1[62], out buffer0[63], out buffer0[62]); + + // Stage 10 applies the pi/64 rotations to the penultimate odd-frequency level. + for (int i = 0; i < 16; i++) + { + int low = 32 + i; + int high = 63 - i; + int odd = Dct64Stage10RotationOrder[i]; + Butterfly(cospi[odd], cospi[64 - odd], buffer0[high], buffer0[low], ref buffer1, low, high, cosBit, in rounding); + } + + // Stage 11 applies the terminal permutation. The fused stages omit pass-through copies, so sources 4-7 and + // 16-31 remain in buffer0 at retirement, + // while every other source resides in buffer1. The mask maps that ownership through AV1 coefficient order. + ReadOnlySpan outputOrder = Dct64OutputOrder; + + for (int i = 0; i < 64; i++) + { + int sourceIndex = outputOrder[i]; + int value = ((Dct64Buffer0OutputMask >> i) & 1) != 0 ? buffer0[sourceIndex] : buffer1[sourceIndex]; + + Store(ref values, outputStride, i, value); + } + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector buffer0, + ref Av1TransformVector buffer1, + int cosBit) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + Av1TransformRounding rounding = Av1ForwardTransformArithmetic.CreateRounding(cosBit); + + // Stage 1 consumes every spatial value before the strided block becomes available for final coefficients. + for (int i = 0; i < 32; i++) + { + Av1ForwardTransformArithmetic.AddSubtract( + Load(ref values, inputStride, i), + Load(ref values, inputStride, 63 - i), + out buffer0[i], + out buffer0[63 - i]); + } + + // Stage 2 begins the recursive radix-2 factorization and rotates the central odd-frequency pairs. + for (int i = 0; i < 16; i++) + { + Av1ForwardTransformArithmetic.AddSubtract(buffer0[i], buffer0[31 - i], out buffer1[i], out buffer1[31 - i]); + } + + for (int i = 0; i < 8; i++) + { + Butterfly(-cospi[32], cospi[32], buffer0[40 + i], buffer0[55 - i], ref buffer1, 40 + i, 55 - i, cosBit, in rounding); + } + + // Stage 3 reduces the even half and folds the next odd-frequency groups into paired sums and differences. + for (int i = 0; i < 8; i++) + { + Av1ForwardTransformArithmetic.AddSubtract(buffer1[i], buffer1[15 - i], out buffer0[i], out buffer0[15 - i]); + } + + for (int i = 0; i < 4; i++) + { + Butterfly(-cospi[32], cospi[32], buffer1[20 + i], buffer1[27 - i], ref buffer0, 20 + i, 27 - i, cosBit, in rounding); + } + + for (int i = 0; i < 8; i++) + { + Av1ForwardTransformArithmetic.AddSubtract(buffer0[32 + i], buffer1[47 - i], out buffer0[32 + i], out buffer0[47 - i]); + } + + for (int i = 0; i < 8; i++) + { + Av1ForwardTransformArithmetic.AddSubtract(buffer0[63 - i], buffer1[48 + i], out buffer0[63 - i], out buffer0[48 + i]); + } + + // Stage 4 continues the factorization as independent sixteen-value groups. + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic.AddSubtract(buffer0[i], buffer0[7 - i], out buffer1[i], out buffer1[7 - i]); + } + + for (int i = 0; i < 2; i++) + { + Butterfly(-cospi[32], cospi[32], buffer0[10 + i], buffer0[13 - i], ref buffer1, 10 + i, 13 - i, cosBit, in rounding); + } + + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic.AddSubtract(buffer1[16 + i], buffer0[23 - i], out buffer1[16 + i], out buffer1[23 - i]); + Av1ForwardTransformArithmetic.AddSubtract(buffer1[31 - i], buffer0[24 + i], out buffer1[31 - i], out buffer1[24 + i]); + Butterfly(-cospi[16], cospi[48], buffer0[36 + i], buffer0[59 - i], ref buffer1, 36 + i, 59 - i, cosBit, in rounding); + } + + for (int i = 4; i < 8; i++) + { + Butterfly(-cospi[48], -cospi[16], buffer0[36 + i], buffer0[59 - i], ref buffer1, 36 + i, 59 - i, cosBit, in rounding); + } + + // Stage 5 reduces the sixteen-value groups into the eight-value DCT and ADST building blocks. + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic.AddSubtract(buffer1[i], buffer1[3 - i], out buffer0[i], out buffer0[3 - i]); + } + + Butterfly(-cospi[32], cospi[32], buffer1[5], buffer1[6], ref buffer0, 5, 6, cosBit, in rounding); + + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic.AddSubtract(buffer0[8 + i], buffer1[11 - i], out buffer0[8 + i], out buffer0[11 - i]); + Av1ForwardTransformArithmetic.AddSubtract(buffer0[15 - i], buffer1[12 + i], out buffer0[15 - i], out buffer0[12 + i]); + Butterfly(-cospi[16], cospi[48], buffer1[18 + i], buffer1[29 - i], ref buffer0, 18 + i, 29 - i, cosBit, in rounding); + } + + for (int i = 2; i < 4; i++) + { + Butterfly(-cospi[48], -cospi[16], buffer1[18 + i], buffer1[29 - i], ref buffer0, 18 + i, 29 - i, cosBit, in rounding); + } + + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic.AddSubtract(buffer0[32 + i], buffer1[39 - i], out buffer0[32 + i], out buffer0[39 - i]); + Av1ForwardTransformArithmetic.AddSubtract(buffer0[47 - i], buffer1[40 + i], out buffer0[47 - i], out buffer0[40 + i]); + Av1ForwardTransformArithmetic.AddSubtract(buffer0[48 + i], buffer1[55 - i], out buffer0[48 + i], out buffer0[55 - i]); + Av1ForwardTransformArithmetic.AddSubtract(buffer0[63 - i], buffer1[56 + i], out buffer0[63 - i], out buffer0[56 + i]); + } + + // Stage 6 completes the low-frequency DCT and rotates the first separated odd-frequency groups. + Butterfly(cospi[32], cospi[32], buffer0[0], buffer0[1], ref buffer1, 0, 1, cosBit, in rounding); + Butterfly(cospi[16], cospi[48], buffer0[3], buffer0[2], ref buffer1, 2, 3, cosBit, in rounding); + Av1ForwardTransformArithmetic.AddSubtract(buffer1[4], buffer0[5], out buffer1[4], out buffer1[5]); + Av1ForwardTransformArithmetic.AddSubtract(buffer1[7], buffer0[6], out buffer1[7], out buffer1[6]); + Butterfly(-cospi[16], cospi[48], buffer0[9], buffer0[14], ref buffer1, 9, 14, cosBit, in rounding); + Butterfly(-cospi[48], -cospi[16], buffer0[10], buffer0[13], ref buffer1, 10, 13, cosBit, in rounding); + + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic.AddSubtract(buffer1[16 + i], buffer0[19 - i], out buffer1[16 + i], out buffer1[19 - i]); + Av1ForwardTransformArithmetic.AddSubtract(buffer1[23 - i], buffer0[20 + i], out buffer1[23 - i], out buffer1[20 + i]); + Av1ForwardTransformArithmetic.AddSubtract(buffer1[24 + i], buffer0[27 - i], out buffer1[24 + i], out buffer1[27 - i]); + Av1ForwardTransformArithmetic.AddSubtract(buffer1[31 - i], buffer0[28 + i], out buffer1[31 - i], out buffer1[28 + i]); + Butterfly(-cospi[8], cospi[56], buffer0[34 + i], buffer0[61 - i], ref buffer1, 34 + i, 61 - i, cosBit, in rounding); + Butterfly(-cospi[40], cospi[24], buffer0[42 + i], buffer0[53 - i], ref buffer1, 42 + i, 53 - i, cosBit, in rounding); + } + + for (int i = 2; i < 4; i++) + { + Butterfly(-cospi[56], -cospi[8], buffer0[34 + i], buffer0[61 - i], ref buffer1, 34 + i, 61 - i, cosBit, in rounding); + Butterfly(-cospi[24], -cospi[40], buffer0[42 + i], buffer0[53 - i], ref buffer1, 42 + i, 53 - i, cosBit, in rounding); + } + + // Stage 7 merges adjacent odd-frequency terms with the sign pattern required by the next rotations. + Butterfly(cospi[8], cospi[56], buffer1[7], buffer1[4], ref buffer0, 4, 7, cosBit, in rounding); + Butterfly(cospi[40], cospi[24], buffer1[6], buffer1[5], ref buffer0, 5, 6, cosBit, in rounding); + Av1ForwardTransformArithmetic.AddSubtract(buffer0[8], buffer1[9], out buffer0[8], out buffer0[9]); + Av1ForwardTransformArithmetic.AddSubtract(buffer0[11], buffer1[10], out buffer0[11], out buffer0[10]); + Av1ForwardTransformArithmetic.AddSubtract(buffer0[12], buffer1[13], out buffer0[12], out buffer0[13]); + Av1ForwardTransformArithmetic.AddSubtract(buffer0[15], buffer1[14], out buffer0[15], out buffer0[14]); + Butterfly(-cospi[8], cospi[56], buffer1[17], buffer1[30], ref buffer0, 17, 30, cosBit, in rounding); + Butterfly(-cospi[56], -cospi[8], buffer1[18], buffer1[29], ref buffer0, 18, 29, cosBit, in rounding); + Butterfly(-cospi[40], cospi[24], buffer1[21], buffer1[26], ref buffer0, 21, 26, cosBit, in rounding); + Butterfly(-cospi[24], -cospi[40], buffer1[22], buffer1[25], ref buffer0, 22, 25, cosBit, in rounding); + + for (int group = 0; group < 4; group++) + { + int offset = group * 8; + + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic.AddSubtract( + buffer0[32 + offset + i], + buffer1[35 + offset - i], + out buffer0[32 + offset + i], + out buffer0[35 + offset - i]); + + Av1ForwardTransformArithmetic.AddSubtract( + buffer0[39 + offset - i], + buffer1[36 + offset + i], + out buffer0[39 + offset - i], + out buffer0[36 + offset + i]); + } + } + + // Stage 8 applies the next level of odd-frequency rotations. + Butterfly(cospi[4], cospi[60], buffer0[15], buffer0[8], ref buffer1, 8, 15, cosBit, in rounding); + Butterfly(cospi[36], cospi[28], buffer0[14], buffer0[9], ref buffer1, 9, 14, cosBit, in rounding); + Butterfly(cospi[20], cospi[44], buffer0[13], buffer0[10], ref buffer1, 10, 13, cosBit, in rounding); + Butterfly(cospi[52], cospi[12], buffer0[12], buffer0[11], ref buffer1, 11, 12, cosBit, in rounding); + + Av1ForwardTransformArithmetic.AddSubtract(buffer1[16], buffer0[17], out buffer1[16], out buffer1[17]); + Av1ForwardTransformArithmetic.AddSubtract(buffer1[19], buffer0[18], out buffer1[19], out buffer1[18]); + Av1ForwardTransformArithmetic.AddSubtract(buffer1[20], buffer0[21], out buffer1[20], out buffer1[21]); + Av1ForwardTransformArithmetic.AddSubtract(buffer1[23], buffer0[22], out buffer1[23], out buffer1[22]); + Av1ForwardTransformArithmetic.AddSubtract(buffer1[24], buffer0[25], out buffer1[24], out buffer1[25]); + Av1ForwardTransformArithmetic.AddSubtract(buffer1[27], buffer0[26], out buffer1[27], out buffer1[26]); + Av1ForwardTransformArithmetic.AddSubtract(buffer1[28], buffer0[29], out buffer1[28], out buffer1[29]); + Av1ForwardTransformArithmetic.AddSubtract(buffer1[31], buffer0[30], out buffer1[31], out buffer1[30]); + + Butterfly(-cospi[4], cospi[60], buffer0[33], buffer0[62], ref buffer1, 33, 62, cosBit, in rounding); + Butterfly(-cospi[60], -cospi[4], buffer0[34], buffer0[61], ref buffer1, 34, 61, cosBit, in rounding); + Butterfly(-cospi[36], cospi[28], buffer0[37], buffer0[58], ref buffer1, 37, 58, cosBit, in rounding); + Butterfly(-cospi[28], -cospi[36], buffer0[38], buffer0[57], ref buffer1, 38, 57, cosBit, in rounding); + Butterfly(-cospi[20], cospi[44], buffer0[41], buffer0[54], ref buffer1, 41, 54, cosBit, in rounding); + Butterfly(-cospi[44], -cospi[20], buffer0[42], buffer0[53], ref buffer1, 42, 53, cosBit, in rounding); + Butterfly(-cospi[52], cospi[12], buffer0[45], buffer0[50], ref buffer1, 45, 50, cosBit, in rounding); + Butterfly(-cospi[12], -cospi[52], buffer0[46], buffer0[49], ref buffer1, 46, 49, cosBit, in rounding); + + // Stage 9 merges the remaining odd-frequency pairs before their terminal rotations. The table preserves the + // non-linear rotation order while keeping the constants in compile-time data. + for (int i = 0; i < 8; i++) + { + int low = 16 + i; + int high = 31 - i; + int odd = Dct64Stage9RotationOrder[i]; + Butterfly(cospi[odd], cospi[64 - odd], buffer1[high], buffer1[low], ref buffer0, low, high, cosBit, in rounding); + } + + Av1ForwardTransformArithmetic.AddSubtract(buffer0[32], buffer1[33], out buffer0[32], out buffer0[33]); + Av1ForwardTransformArithmetic.AddSubtract(buffer0[35], buffer1[34], out buffer0[35], out buffer0[34]); + Av1ForwardTransformArithmetic.AddSubtract(buffer0[36], buffer1[37], out buffer0[36], out buffer0[37]); + Av1ForwardTransformArithmetic.AddSubtract(buffer0[39], buffer1[38], out buffer0[39], out buffer0[38]); + Av1ForwardTransformArithmetic.AddSubtract(buffer0[40], buffer1[41], out buffer0[40], out buffer0[41]); + Av1ForwardTransformArithmetic.AddSubtract(buffer0[43], buffer1[42], out buffer0[43], out buffer0[42]); + Av1ForwardTransformArithmetic.AddSubtract(buffer0[44], buffer1[45], out buffer0[44], out buffer0[45]); + Av1ForwardTransformArithmetic.AddSubtract(buffer0[47], buffer1[46], out buffer0[47], out buffer0[46]); + Av1ForwardTransformArithmetic.AddSubtract(buffer0[48], buffer1[49], out buffer0[48], out buffer0[49]); + Av1ForwardTransformArithmetic.AddSubtract(buffer0[51], buffer1[50], out buffer0[51], out buffer0[50]); + Av1ForwardTransformArithmetic.AddSubtract(buffer0[52], buffer1[53], out buffer0[52], out buffer0[53]); + Av1ForwardTransformArithmetic.AddSubtract(buffer0[55], buffer1[54], out buffer0[55], out buffer0[54]); + Av1ForwardTransformArithmetic.AddSubtract(buffer0[56], buffer1[57], out buffer0[56], out buffer0[57]); + Av1ForwardTransformArithmetic.AddSubtract(buffer0[59], buffer1[58], out buffer0[59], out buffer0[58]); + Av1ForwardTransformArithmetic.AddSubtract(buffer0[60], buffer1[61], out buffer0[60], out buffer0[61]); + Av1ForwardTransformArithmetic.AddSubtract(buffer0[63], buffer1[62], out buffer0[63], out buffer0[62]); + + // Stage 10 applies the pi/64 rotations to the penultimate odd-frequency level. + for (int i = 0; i < 16; i++) + { + int low = 32 + i; + int high = 63 - i; + int odd = Dct64Stage10RotationOrder[i]; + Butterfly(cospi[odd], cospi[64 - odd], buffer0[high], buffer0[low], ref buffer1, low, high, cosBit, in rounding); + } + + // Stage 11 applies the terminal permutation. The fused stages omit pass-through copies, so sources 4-7 and + // 16-31 remain in buffer0 at retirement, + // while every other source resides in buffer1. The mask maps that ownership through AV1 coefficient order. + ReadOnlySpan outputOrder = Dct64OutputOrder; + + for (int i = 0; i < 64; i++) + { + int sourceIndex = outputOrder[i]; + short value = ((Dct64Buffer0OutputMask >> i) & 1) != 0 ? buffer0[sourceIndex] : buffer1[sourceIndex]; + + Store(ref values, outputStride, i, value); + } + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + Av1TransformRounding rounding = Av1ForwardTransformArithmetic>.CreateRounding(cosBit); + + // Stage 1 consumes every spatial value before the strided block becomes available for final coefficients. + for (int i = 0; i < 32; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract( + Load>(ref values, inputStride, i), + Load>(ref values, inputStride, 63 - i), + out buffer0[i], + out buffer0[63 - i]); + } + + // Stage 2 begins the recursive radix-2 factorization and rotates the central odd-frequency pairs. + for (int i = 0; i < 16; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[i], buffer0[31 - i], out buffer1[i], out buffer1[31 - i]); + } + + for (int i = 0; i < 8; i++) + { + Butterfly(-cospi[32], cospi[32], buffer0[40 + i], buffer0[55 - i], ref buffer1, 40 + i, 55 - i, cosBit, in rounding); + } + + // Stage 3 reduces the even half and folds the next odd-frequency groups into paired sums and differences. + for (int i = 0; i < 8; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[i], buffer1[15 - i], out buffer0[i], out buffer0[15 - i]); + } + + for (int i = 0; i < 4; i++) + { + Butterfly(-cospi[32], cospi[32], buffer1[20 + i], buffer1[27 - i], ref buffer0, 20 + i, 27 - i, cosBit, in rounding); + } + + for (int i = 0; i < 8; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[32 + i], buffer1[47 - i], out buffer0[32 + i], out buffer0[47 - i]); + } + + for (int i = 0; i < 8; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[63 - i], buffer1[48 + i], out buffer0[63 - i], out buffer0[48 + i]); + } + + // Stage 4 continues the factorization as independent sixteen-value groups. + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[i], buffer0[7 - i], out buffer1[i], out buffer1[7 - i]); + } + + for (int i = 0; i < 2; i++) + { + Butterfly(-cospi[32], cospi[32], buffer0[10 + i], buffer0[13 - i], ref buffer1, 10 + i, 13 - i, cosBit, in rounding); + } + + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[16 + i], buffer0[23 - i], out buffer1[16 + i], out buffer1[23 - i]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[31 - i], buffer0[24 + i], out buffer1[31 - i], out buffer1[24 + i]); + Butterfly(-cospi[16], cospi[48], buffer0[36 + i], buffer0[59 - i], ref buffer1, 36 + i, 59 - i, cosBit, in rounding); + } + + for (int i = 4; i < 8; i++) + { + Butterfly(-cospi[48], -cospi[16], buffer0[36 + i], buffer0[59 - i], ref buffer1, 36 + i, 59 - i, cosBit, in rounding); + } + + // Stage 5 reduces the sixteen-value groups into the eight-value DCT and ADST building blocks. + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[i], buffer1[3 - i], out buffer0[i], out buffer0[3 - i]); + } + + Butterfly(-cospi[32], cospi[32], buffer1[5], buffer1[6], ref buffer0, 5, 6, cosBit, in rounding); + + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[8 + i], buffer1[11 - i], out buffer0[8 + i], out buffer0[11 - i]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[15 - i], buffer1[12 + i], out buffer0[15 - i], out buffer0[12 + i]); + Butterfly(-cospi[16], cospi[48], buffer1[18 + i], buffer1[29 - i], ref buffer0, 18 + i, 29 - i, cosBit, in rounding); + } + + for (int i = 2; i < 4; i++) + { + Butterfly(-cospi[48], -cospi[16], buffer1[18 + i], buffer1[29 - i], ref buffer0, 18 + i, 29 - i, cosBit, in rounding); + } + + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[32 + i], buffer1[39 - i], out buffer0[32 + i], out buffer0[39 - i]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[47 - i], buffer1[40 + i], out buffer0[47 - i], out buffer0[40 + i]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[48 + i], buffer1[55 - i], out buffer0[48 + i], out buffer0[55 - i]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[63 - i], buffer1[56 + i], out buffer0[63 - i], out buffer0[56 + i]); + } + + // Stage 6 completes the low-frequency DCT and rotates the first separated odd-frequency groups. + Butterfly(cospi[32], cospi[32], buffer0[0], buffer0[1], ref buffer1, 0, 1, cosBit, in rounding); + Butterfly(cospi[16], cospi[48], buffer0[3], buffer0[2], ref buffer1, 2, 3, cosBit, in rounding); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[4], buffer0[5], out buffer1[4], out buffer1[5]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[7], buffer0[6], out buffer1[7], out buffer1[6]); + Butterfly(-cospi[16], cospi[48], buffer0[9], buffer0[14], ref buffer1, 9, 14, cosBit, in rounding); + Butterfly(-cospi[48], -cospi[16], buffer0[10], buffer0[13], ref buffer1, 10, 13, cosBit, in rounding); + + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[16 + i], buffer0[19 - i], out buffer1[16 + i], out buffer1[19 - i]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[23 - i], buffer0[20 + i], out buffer1[23 - i], out buffer1[20 + i]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[24 + i], buffer0[27 - i], out buffer1[24 + i], out buffer1[27 - i]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[31 - i], buffer0[28 + i], out buffer1[31 - i], out buffer1[28 + i]); + Butterfly(-cospi[8], cospi[56], buffer0[34 + i], buffer0[61 - i], ref buffer1, 34 + i, 61 - i, cosBit, in rounding); + Butterfly(-cospi[40], cospi[24], buffer0[42 + i], buffer0[53 - i], ref buffer1, 42 + i, 53 - i, cosBit, in rounding); + } + + for (int i = 2; i < 4; i++) + { + Butterfly(-cospi[56], -cospi[8], buffer0[34 + i], buffer0[61 - i], ref buffer1, 34 + i, 61 - i, cosBit, in rounding); + Butterfly(-cospi[24], -cospi[40], buffer0[42 + i], buffer0[53 - i], ref buffer1, 42 + i, 53 - i, cosBit, in rounding); + } + + // Stage 7 merges adjacent odd-frequency terms with the sign pattern required by the next rotations. + Butterfly(cospi[8], cospi[56], buffer1[7], buffer1[4], ref buffer0, 4, 7, cosBit, in rounding); + Butterfly(cospi[40], cospi[24], buffer1[6], buffer1[5], ref buffer0, 5, 6, cosBit, in rounding); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[8], buffer1[9], out buffer0[8], out buffer0[9]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[11], buffer1[10], out buffer0[11], out buffer0[10]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[12], buffer1[13], out buffer0[12], out buffer0[13]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[15], buffer1[14], out buffer0[15], out buffer0[14]); + Butterfly(-cospi[8], cospi[56], buffer1[17], buffer1[30], ref buffer0, 17, 30, cosBit, in rounding); + Butterfly(-cospi[56], -cospi[8], buffer1[18], buffer1[29], ref buffer0, 18, 29, cosBit, in rounding); + Butterfly(-cospi[40], cospi[24], buffer1[21], buffer1[26], ref buffer0, 21, 26, cosBit, in rounding); + Butterfly(-cospi[24], -cospi[40], buffer1[22], buffer1[25], ref buffer0, 22, 25, cosBit, in rounding); + + for (int group = 0; group < 4; group++) + { + int offset = group * 8; + + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract( + buffer0[32 + offset + i], + buffer1[35 + offset - i], + out buffer0[32 + offset + i], + out buffer0[35 + offset - i]); + + Av1ForwardTransformArithmetic>.AddSubtract( + buffer0[39 + offset - i], + buffer1[36 + offset + i], + out buffer0[39 + offset - i], + out buffer0[36 + offset + i]); + } + } + + // Stage 8 applies the next level of odd-frequency rotations. + Butterfly(cospi[4], cospi[60], buffer0[15], buffer0[8], ref buffer1, 8, 15, cosBit, in rounding); + Butterfly(cospi[36], cospi[28], buffer0[14], buffer0[9], ref buffer1, 9, 14, cosBit, in rounding); + Butterfly(cospi[20], cospi[44], buffer0[13], buffer0[10], ref buffer1, 10, 13, cosBit, in rounding); + Butterfly(cospi[52], cospi[12], buffer0[12], buffer0[11], ref buffer1, 11, 12, cosBit, in rounding); + + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[16], buffer0[17], out buffer1[16], out buffer1[17]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[19], buffer0[18], out buffer1[19], out buffer1[18]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[20], buffer0[21], out buffer1[20], out buffer1[21]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[23], buffer0[22], out buffer1[23], out buffer1[22]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[24], buffer0[25], out buffer1[24], out buffer1[25]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[27], buffer0[26], out buffer1[27], out buffer1[26]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[28], buffer0[29], out buffer1[28], out buffer1[29]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[31], buffer0[30], out buffer1[31], out buffer1[30]); + + Butterfly(-cospi[4], cospi[60], buffer0[33], buffer0[62], ref buffer1, 33, 62, cosBit, in rounding); + Butterfly(-cospi[60], -cospi[4], buffer0[34], buffer0[61], ref buffer1, 34, 61, cosBit, in rounding); + Butterfly(-cospi[36], cospi[28], buffer0[37], buffer0[58], ref buffer1, 37, 58, cosBit, in rounding); + Butterfly(-cospi[28], -cospi[36], buffer0[38], buffer0[57], ref buffer1, 38, 57, cosBit, in rounding); + Butterfly(-cospi[20], cospi[44], buffer0[41], buffer0[54], ref buffer1, 41, 54, cosBit, in rounding); + Butterfly(-cospi[44], -cospi[20], buffer0[42], buffer0[53], ref buffer1, 42, 53, cosBit, in rounding); + Butterfly(-cospi[52], cospi[12], buffer0[45], buffer0[50], ref buffer1, 45, 50, cosBit, in rounding); + Butterfly(-cospi[12], -cospi[52], buffer0[46], buffer0[49], ref buffer1, 46, 49, cosBit, in rounding); + + // Stage 9 merges the remaining odd-frequency pairs before their terminal rotations. The table preserves the + // non-linear rotation order while keeping the constants in compile-time data. + for (int i = 0; i < 8; i++) + { + int low = 16 + i; + int high = 31 - i; + int odd = Dct64Stage9RotationOrder[i]; + Butterfly(cospi[odd], cospi[64 - odd], buffer1[high], buffer1[low], ref buffer0, low, high, cosBit, in rounding); + } + + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[32], buffer1[33], out buffer0[32], out buffer0[33]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[35], buffer1[34], out buffer0[35], out buffer0[34]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[36], buffer1[37], out buffer0[36], out buffer0[37]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[39], buffer1[38], out buffer0[39], out buffer0[38]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[40], buffer1[41], out buffer0[40], out buffer0[41]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[43], buffer1[42], out buffer0[43], out buffer0[42]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[44], buffer1[45], out buffer0[44], out buffer0[45]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[47], buffer1[46], out buffer0[47], out buffer0[46]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[48], buffer1[49], out buffer0[48], out buffer0[49]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[51], buffer1[50], out buffer0[51], out buffer0[50]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[52], buffer1[53], out buffer0[52], out buffer0[53]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[55], buffer1[54], out buffer0[55], out buffer0[54]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[56], buffer1[57], out buffer0[56], out buffer0[57]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[59], buffer1[58], out buffer0[59], out buffer0[58]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[60], buffer1[61], out buffer0[60], out buffer0[61]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[63], buffer1[62], out buffer0[63], out buffer0[62]); + + // Stage 10 applies the pi/64 rotations to the penultimate odd-frequency level. + for (int i = 0; i < 16; i++) + { + int low = 32 + i; + int high = 63 - i; + int odd = Dct64Stage10RotationOrder[i]; + Butterfly(cospi[odd], cospi[64 - odd], buffer0[high], buffer0[low], ref buffer1, low, high, cosBit, in rounding); + } + + // Stage 11 applies the terminal permutation. The fused stages omit pass-through copies, so sources 4-7 and + // 16-31 remain in buffer0 at retirement, + // while every other source resides in buffer1. The mask maps that ownership through AV1 coefficient order. + ReadOnlySpan outputOrder = Dct64OutputOrder; + + for (int i = 0; i < 64; i++) + { + int sourceIndex = outputOrder[i]; + Vector128 value = ((Dct64Buffer0OutputMask >> i) & 1) != 0 ? buffer0[sourceIndex] : buffer1[sourceIndex]; + + Store(ref values, outputStride, i, value); + } + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + Av1TransformRounding rounding = Av1ForwardTransformArithmetic>.CreateRounding(cosBit); + + // Stage 1 consumes every spatial value before the strided block becomes available for final coefficients. + for (int i = 0; i < 32; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract( + Load>(ref values, inputStride, i), + Load>(ref values, inputStride, 63 - i), + out buffer0[i], + out buffer0[63 - i]); + } + + // Stage 2 begins the recursive radix-2 factorization and rotates the central odd-frequency pairs. + for (int i = 0; i < 16; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[i], buffer0[31 - i], out buffer1[i], out buffer1[31 - i]); + } + + for (int i = 0; i < 8; i++) + { + Butterfly(-cospi[32], cospi[32], buffer0[40 + i], buffer0[55 - i], ref buffer1, 40 + i, 55 - i, cosBit, in rounding); + } + + // Stage 3 reduces the even half and folds the next odd-frequency groups into paired sums and differences. + for (int i = 0; i < 8; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[i], buffer1[15 - i], out buffer0[i], out buffer0[15 - i]); + } + + for (int i = 0; i < 4; i++) + { + Butterfly(-cospi[32], cospi[32], buffer1[20 + i], buffer1[27 - i], ref buffer0, 20 + i, 27 - i, cosBit, in rounding); + } + + for (int i = 0; i < 8; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[32 + i], buffer1[47 - i], out buffer0[32 + i], out buffer0[47 - i]); + } + + for (int i = 0; i < 8; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[63 - i], buffer1[48 + i], out buffer0[63 - i], out buffer0[48 + i]); + } + + // Stage 4 continues the factorization as independent sixteen-value groups. + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[i], buffer0[7 - i], out buffer1[i], out buffer1[7 - i]); + } + + for (int i = 0; i < 2; i++) + { + Butterfly(-cospi[32], cospi[32], buffer0[10 + i], buffer0[13 - i], ref buffer1, 10 + i, 13 - i, cosBit, in rounding); + } + + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[16 + i], buffer0[23 - i], out buffer1[16 + i], out buffer1[23 - i]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[31 - i], buffer0[24 + i], out buffer1[31 - i], out buffer1[24 + i]); + Butterfly(-cospi[16], cospi[48], buffer0[36 + i], buffer0[59 - i], ref buffer1, 36 + i, 59 - i, cosBit, in rounding); + } + + for (int i = 4; i < 8; i++) + { + Butterfly(-cospi[48], -cospi[16], buffer0[36 + i], buffer0[59 - i], ref buffer1, 36 + i, 59 - i, cosBit, in rounding); + } + + // Stage 5 reduces the sixteen-value groups into the eight-value DCT and ADST building blocks. + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[i], buffer1[3 - i], out buffer0[i], out buffer0[3 - i]); + } + + Butterfly(-cospi[32], cospi[32], buffer1[5], buffer1[6], ref buffer0, 5, 6, cosBit, in rounding); + + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[8 + i], buffer1[11 - i], out buffer0[8 + i], out buffer0[11 - i]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[15 - i], buffer1[12 + i], out buffer0[15 - i], out buffer0[12 + i]); + Butterfly(-cospi[16], cospi[48], buffer1[18 + i], buffer1[29 - i], ref buffer0, 18 + i, 29 - i, cosBit, in rounding); + } + + for (int i = 2; i < 4; i++) + { + Butterfly(-cospi[48], -cospi[16], buffer1[18 + i], buffer1[29 - i], ref buffer0, 18 + i, 29 - i, cosBit, in rounding); + } + + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[32 + i], buffer1[39 - i], out buffer0[32 + i], out buffer0[39 - i]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[47 - i], buffer1[40 + i], out buffer0[47 - i], out buffer0[40 + i]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[48 + i], buffer1[55 - i], out buffer0[48 + i], out buffer0[55 - i]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[63 - i], buffer1[56 + i], out buffer0[63 - i], out buffer0[56 + i]); + } + + // Stage 6 completes the low-frequency DCT and rotates the first separated odd-frequency groups. + Butterfly(cospi[32], cospi[32], buffer0[0], buffer0[1], ref buffer1, 0, 1, cosBit, in rounding); + Butterfly(cospi[16], cospi[48], buffer0[3], buffer0[2], ref buffer1, 2, 3, cosBit, in rounding); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[4], buffer0[5], out buffer1[4], out buffer1[5]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[7], buffer0[6], out buffer1[7], out buffer1[6]); + Butterfly(-cospi[16], cospi[48], buffer0[9], buffer0[14], ref buffer1, 9, 14, cosBit, in rounding); + Butterfly(-cospi[48], -cospi[16], buffer0[10], buffer0[13], ref buffer1, 10, 13, cosBit, in rounding); + + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[16 + i], buffer0[19 - i], out buffer1[16 + i], out buffer1[19 - i]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[23 - i], buffer0[20 + i], out buffer1[23 - i], out buffer1[20 + i]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[24 + i], buffer0[27 - i], out buffer1[24 + i], out buffer1[27 - i]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[31 - i], buffer0[28 + i], out buffer1[31 - i], out buffer1[28 + i]); + Butterfly(-cospi[8], cospi[56], buffer0[34 + i], buffer0[61 - i], ref buffer1, 34 + i, 61 - i, cosBit, in rounding); + Butterfly(-cospi[40], cospi[24], buffer0[42 + i], buffer0[53 - i], ref buffer1, 42 + i, 53 - i, cosBit, in rounding); + } + + for (int i = 2; i < 4; i++) + { + Butterfly(-cospi[56], -cospi[8], buffer0[34 + i], buffer0[61 - i], ref buffer1, 34 + i, 61 - i, cosBit, in rounding); + Butterfly(-cospi[24], -cospi[40], buffer0[42 + i], buffer0[53 - i], ref buffer1, 42 + i, 53 - i, cosBit, in rounding); + } + + // Stage 7 merges adjacent odd-frequency terms with the sign pattern required by the next rotations. + Butterfly(cospi[8], cospi[56], buffer1[7], buffer1[4], ref buffer0, 4, 7, cosBit, in rounding); + Butterfly(cospi[40], cospi[24], buffer1[6], buffer1[5], ref buffer0, 5, 6, cosBit, in rounding); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[8], buffer1[9], out buffer0[8], out buffer0[9]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[11], buffer1[10], out buffer0[11], out buffer0[10]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[12], buffer1[13], out buffer0[12], out buffer0[13]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[15], buffer1[14], out buffer0[15], out buffer0[14]); + Butterfly(-cospi[8], cospi[56], buffer1[17], buffer1[30], ref buffer0, 17, 30, cosBit, in rounding); + Butterfly(-cospi[56], -cospi[8], buffer1[18], buffer1[29], ref buffer0, 18, 29, cosBit, in rounding); + Butterfly(-cospi[40], cospi[24], buffer1[21], buffer1[26], ref buffer0, 21, 26, cosBit, in rounding); + Butterfly(-cospi[24], -cospi[40], buffer1[22], buffer1[25], ref buffer0, 22, 25, cosBit, in rounding); + + for (int group = 0; group < 4; group++) + { + int offset = group * 8; + + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract( + buffer0[32 + offset + i], + buffer1[35 + offset - i], + out buffer0[32 + offset + i], + out buffer0[35 + offset - i]); + + Av1ForwardTransformArithmetic>.AddSubtract( + buffer0[39 + offset - i], + buffer1[36 + offset + i], + out buffer0[39 + offset - i], + out buffer0[36 + offset + i]); + } + } + + // Stage 8 applies the next level of odd-frequency rotations. + Butterfly(cospi[4], cospi[60], buffer0[15], buffer0[8], ref buffer1, 8, 15, cosBit, in rounding); + Butterfly(cospi[36], cospi[28], buffer0[14], buffer0[9], ref buffer1, 9, 14, cosBit, in rounding); + Butterfly(cospi[20], cospi[44], buffer0[13], buffer0[10], ref buffer1, 10, 13, cosBit, in rounding); + Butterfly(cospi[52], cospi[12], buffer0[12], buffer0[11], ref buffer1, 11, 12, cosBit, in rounding); + + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[16], buffer0[17], out buffer1[16], out buffer1[17]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[19], buffer0[18], out buffer1[19], out buffer1[18]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[20], buffer0[21], out buffer1[20], out buffer1[21]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[23], buffer0[22], out buffer1[23], out buffer1[22]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[24], buffer0[25], out buffer1[24], out buffer1[25]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[27], buffer0[26], out buffer1[27], out buffer1[26]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[28], buffer0[29], out buffer1[28], out buffer1[29]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[31], buffer0[30], out buffer1[31], out buffer1[30]); + + Butterfly(-cospi[4], cospi[60], buffer0[33], buffer0[62], ref buffer1, 33, 62, cosBit, in rounding); + Butterfly(-cospi[60], -cospi[4], buffer0[34], buffer0[61], ref buffer1, 34, 61, cosBit, in rounding); + Butterfly(-cospi[36], cospi[28], buffer0[37], buffer0[58], ref buffer1, 37, 58, cosBit, in rounding); + Butterfly(-cospi[28], -cospi[36], buffer0[38], buffer0[57], ref buffer1, 38, 57, cosBit, in rounding); + Butterfly(-cospi[20], cospi[44], buffer0[41], buffer0[54], ref buffer1, 41, 54, cosBit, in rounding); + Butterfly(-cospi[44], -cospi[20], buffer0[42], buffer0[53], ref buffer1, 42, 53, cosBit, in rounding); + Butterfly(-cospi[52], cospi[12], buffer0[45], buffer0[50], ref buffer1, 45, 50, cosBit, in rounding); + Butterfly(-cospi[12], -cospi[52], buffer0[46], buffer0[49], ref buffer1, 46, 49, cosBit, in rounding); + + // Stage 9 merges the remaining odd-frequency pairs before their terminal rotations. The table preserves the + // non-linear rotation order while keeping the constants in compile-time data. + for (int i = 0; i < 8; i++) + { + int low = 16 + i; + int high = 31 - i; + int odd = Dct64Stage9RotationOrder[i]; + Butterfly(cospi[odd], cospi[64 - odd], buffer1[high], buffer1[low], ref buffer0, low, high, cosBit, in rounding); + } + + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[32], buffer1[33], out buffer0[32], out buffer0[33]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[35], buffer1[34], out buffer0[35], out buffer0[34]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[36], buffer1[37], out buffer0[36], out buffer0[37]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[39], buffer1[38], out buffer0[39], out buffer0[38]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[40], buffer1[41], out buffer0[40], out buffer0[41]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[43], buffer1[42], out buffer0[43], out buffer0[42]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[44], buffer1[45], out buffer0[44], out buffer0[45]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[47], buffer1[46], out buffer0[47], out buffer0[46]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[48], buffer1[49], out buffer0[48], out buffer0[49]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[51], buffer1[50], out buffer0[51], out buffer0[50]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[52], buffer1[53], out buffer0[52], out buffer0[53]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[55], buffer1[54], out buffer0[55], out buffer0[54]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[56], buffer1[57], out buffer0[56], out buffer0[57]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[59], buffer1[58], out buffer0[59], out buffer0[58]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[60], buffer1[61], out buffer0[60], out buffer0[61]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[63], buffer1[62], out buffer0[63], out buffer0[62]); + + // Stage 10 applies the pi/64 rotations to the penultimate odd-frequency level. + for (int i = 0; i < 16; i++) + { + int low = 32 + i; + int high = 63 - i; + int odd = Dct64Stage10RotationOrder[i]; + Butterfly(cospi[odd], cospi[64 - odd], buffer0[high], buffer0[low], ref buffer1, low, high, cosBit, in rounding); + } + + // Stage 11 applies the terminal permutation. The fused stages omit pass-through copies, so sources 4-7 and + // 16-31 remain in buffer0 at retirement, + // while every other source resides in buffer1. The mask maps that ownership through AV1 coefficient order. + ReadOnlySpan outputOrder = Dct64OutputOrder; + + for (int i = 0; i < 64; i++) + { + int sourceIndex = outputOrder[i]; + Vector256 value = ((Dct64Buffer0OutputMask >> i) & 1) != 0 ? buffer0[sourceIndex] : buffer1[sourceIndex]; + + Store(ref values, outputStride, i, value); + } + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + Av1TransformRounding rounding = Av1ForwardTransformArithmetic>.CreateRounding(cosBit); + + // Stage 1 consumes every spatial value before the strided block becomes available for final coefficients. + for (int i = 0; i < 32; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract( + Load>(ref values, inputStride, i), + Load>(ref values, inputStride, 63 - i), + out buffer0[i], + out buffer0[63 - i]); + } + + // Stage 2 begins the recursive radix-2 factorization and rotates the central odd-frequency pairs. + for (int i = 0; i < 16; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[i], buffer0[31 - i], out buffer1[i], out buffer1[31 - i]); + } + + for (int i = 0; i < 8; i++) + { + Butterfly(-cospi[32], cospi[32], buffer0[40 + i], buffer0[55 - i], ref buffer1, 40 + i, 55 - i, cosBit, in rounding); + } + + // Stage 3 reduces the even half and folds the next odd-frequency groups into paired sums and differences. + for (int i = 0; i < 8; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[i], buffer1[15 - i], out buffer0[i], out buffer0[15 - i]); + } + + for (int i = 0; i < 4; i++) + { + Butterfly(-cospi[32], cospi[32], buffer1[20 + i], buffer1[27 - i], ref buffer0, 20 + i, 27 - i, cosBit, in rounding); + } + + for (int i = 0; i < 8; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[32 + i], buffer1[47 - i], out buffer0[32 + i], out buffer0[47 - i]); + } + + for (int i = 0; i < 8; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[63 - i], buffer1[48 + i], out buffer0[63 - i], out buffer0[48 + i]); + } + + // Stage 4 continues the factorization as independent sixteen-value groups. + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[i], buffer0[7 - i], out buffer1[i], out buffer1[7 - i]); + } + + for (int i = 0; i < 2; i++) + { + Butterfly(-cospi[32], cospi[32], buffer0[10 + i], buffer0[13 - i], ref buffer1, 10 + i, 13 - i, cosBit, in rounding); + } + + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[16 + i], buffer0[23 - i], out buffer1[16 + i], out buffer1[23 - i]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[31 - i], buffer0[24 + i], out buffer1[31 - i], out buffer1[24 + i]); + Butterfly(-cospi[16], cospi[48], buffer0[36 + i], buffer0[59 - i], ref buffer1, 36 + i, 59 - i, cosBit, in rounding); + } + + for (int i = 4; i < 8; i++) + { + Butterfly(-cospi[48], -cospi[16], buffer0[36 + i], buffer0[59 - i], ref buffer1, 36 + i, 59 - i, cosBit, in rounding); + } + + // Stage 5 reduces the sixteen-value groups into the eight-value DCT and ADST building blocks. + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[i], buffer1[3 - i], out buffer0[i], out buffer0[3 - i]); + } + + Butterfly(-cospi[32], cospi[32], buffer1[5], buffer1[6], ref buffer0, 5, 6, cosBit, in rounding); + + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[8 + i], buffer1[11 - i], out buffer0[8 + i], out buffer0[11 - i]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[15 - i], buffer1[12 + i], out buffer0[15 - i], out buffer0[12 + i]); + Butterfly(-cospi[16], cospi[48], buffer1[18 + i], buffer1[29 - i], ref buffer0, 18 + i, 29 - i, cosBit, in rounding); + } + + for (int i = 2; i < 4; i++) + { + Butterfly(-cospi[48], -cospi[16], buffer1[18 + i], buffer1[29 - i], ref buffer0, 18 + i, 29 - i, cosBit, in rounding); + } + + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[32 + i], buffer1[39 - i], out buffer0[32 + i], out buffer0[39 - i]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[47 - i], buffer1[40 + i], out buffer0[47 - i], out buffer0[40 + i]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[48 + i], buffer1[55 - i], out buffer0[48 + i], out buffer0[55 - i]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[63 - i], buffer1[56 + i], out buffer0[63 - i], out buffer0[56 + i]); + } + + // Stage 6 completes the low-frequency DCT and rotates the first separated odd-frequency groups. + Butterfly(cospi[32], cospi[32], buffer0[0], buffer0[1], ref buffer1, 0, 1, cosBit, in rounding); + Butterfly(cospi[16], cospi[48], buffer0[3], buffer0[2], ref buffer1, 2, 3, cosBit, in rounding); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[4], buffer0[5], out buffer1[4], out buffer1[5]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[7], buffer0[6], out buffer1[7], out buffer1[6]); + Butterfly(-cospi[16], cospi[48], buffer0[9], buffer0[14], ref buffer1, 9, 14, cosBit, in rounding); + Butterfly(-cospi[48], -cospi[16], buffer0[10], buffer0[13], ref buffer1, 10, 13, cosBit, in rounding); + + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[16 + i], buffer0[19 - i], out buffer1[16 + i], out buffer1[19 - i]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[23 - i], buffer0[20 + i], out buffer1[23 - i], out buffer1[20 + i]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[24 + i], buffer0[27 - i], out buffer1[24 + i], out buffer1[27 - i]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[31 - i], buffer0[28 + i], out buffer1[31 - i], out buffer1[28 + i]); + Butterfly(-cospi[8], cospi[56], buffer0[34 + i], buffer0[61 - i], ref buffer1, 34 + i, 61 - i, cosBit, in rounding); + Butterfly(-cospi[40], cospi[24], buffer0[42 + i], buffer0[53 - i], ref buffer1, 42 + i, 53 - i, cosBit, in rounding); + } + + for (int i = 2; i < 4; i++) + { + Butterfly(-cospi[56], -cospi[8], buffer0[34 + i], buffer0[61 - i], ref buffer1, 34 + i, 61 - i, cosBit, in rounding); + Butterfly(-cospi[24], -cospi[40], buffer0[42 + i], buffer0[53 - i], ref buffer1, 42 + i, 53 - i, cosBit, in rounding); + } + + // Stage 7 merges adjacent odd-frequency terms with the sign pattern required by the next rotations. + Butterfly(cospi[8], cospi[56], buffer1[7], buffer1[4], ref buffer0, 4, 7, cosBit, in rounding); + Butterfly(cospi[40], cospi[24], buffer1[6], buffer1[5], ref buffer0, 5, 6, cosBit, in rounding); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[8], buffer1[9], out buffer0[8], out buffer0[9]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[11], buffer1[10], out buffer0[11], out buffer0[10]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[12], buffer1[13], out buffer0[12], out buffer0[13]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[15], buffer1[14], out buffer0[15], out buffer0[14]); + Butterfly(-cospi[8], cospi[56], buffer1[17], buffer1[30], ref buffer0, 17, 30, cosBit, in rounding); + Butterfly(-cospi[56], -cospi[8], buffer1[18], buffer1[29], ref buffer0, 18, 29, cosBit, in rounding); + Butterfly(-cospi[40], cospi[24], buffer1[21], buffer1[26], ref buffer0, 21, 26, cosBit, in rounding); + Butterfly(-cospi[24], -cospi[40], buffer1[22], buffer1[25], ref buffer0, 22, 25, cosBit, in rounding); + + for (int group = 0; group < 4; group++) + { + int offset = group * 8; + + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract( + buffer0[32 + offset + i], + buffer1[35 + offset - i], + out buffer0[32 + offset + i], + out buffer0[35 + offset - i]); + + Av1ForwardTransformArithmetic>.AddSubtract( + buffer0[39 + offset - i], + buffer1[36 + offset + i], + out buffer0[39 + offset - i], + out buffer0[36 + offset + i]); + } + } + + // Stage 8 applies the next level of odd-frequency rotations. + Butterfly(cospi[4], cospi[60], buffer0[15], buffer0[8], ref buffer1, 8, 15, cosBit, in rounding); + Butterfly(cospi[36], cospi[28], buffer0[14], buffer0[9], ref buffer1, 9, 14, cosBit, in rounding); + Butterfly(cospi[20], cospi[44], buffer0[13], buffer0[10], ref buffer1, 10, 13, cosBit, in rounding); + Butterfly(cospi[52], cospi[12], buffer0[12], buffer0[11], ref buffer1, 11, 12, cosBit, in rounding); + + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[16], buffer0[17], out buffer1[16], out buffer1[17]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[19], buffer0[18], out buffer1[19], out buffer1[18]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[20], buffer0[21], out buffer1[20], out buffer1[21]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[23], buffer0[22], out buffer1[23], out buffer1[22]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[24], buffer0[25], out buffer1[24], out buffer1[25]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[27], buffer0[26], out buffer1[27], out buffer1[26]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[28], buffer0[29], out buffer1[28], out buffer1[29]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[31], buffer0[30], out buffer1[31], out buffer1[30]); + + Butterfly(-cospi[4], cospi[60], buffer0[33], buffer0[62], ref buffer1, 33, 62, cosBit, in rounding); + Butterfly(-cospi[60], -cospi[4], buffer0[34], buffer0[61], ref buffer1, 34, 61, cosBit, in rounding); + Butterfly(-cospi[36], cospi[28], buffer0[37], buffer0[58], ref buffer1, 37, 58, cosBit, in rounding); + Butterfly(-cospi[28], -cospi[36], buffer0[38], buffer0[57], ref buffer1, 38, 57, cosBit, in rounding); + Butterfly(-cospi[20], cospi[44], buffer0[41], buffer0[54], ref buffer1, 41, 54, cosBit, in rounding); + Butterfly(-cospi[44], -cospi[20], buffer0[42], buffer0[53], ref buffer1, 42, 53, cosBit, in rounding); + Butterfly(-cospi[52], cospi[12], buffer0[45], buffer0[50], ref buffer1, 45, 50, cosBit, in rounding); + Butterfly(-cospi[12], -cospi[52], buffer0[46], buffer0[49], ref buffer1, 46, 49, cosBit, in rounding); + + // Stage 9 merges the remaining odd-frequency pairs before their terminal rotations. The table preserves the + // non-linear rotation order while keeping the constants in compile-time data. + for (int i = 0; i < 8; i++) + { + int low = 16 + i; + int high = 31 - i; + int odd = Dct64Stage9RotationOrder[i]; + Butterfly(cospi[odd], cospi[64 - odd], buffer1[high], buffer1[low], ref buffer0, low, high, cosBit, in rounding); + } + + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[32], buffer1[33], out buffer0[32], out buffer0[33]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[35], buffer1[34], out buffer0[35], out buffer0[34]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[36], buffer1[37], out buffer0[36], out buffer0[37]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[39], buffer1[38], out buffer0[39], out buffer0[38]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[40], buffer1[41], out buffer0[40], out buffer0[41]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[43], buffer1[42], out buffer0[43], out buffer0[42]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[44], buffer1[45], out buffer0[44], out buffer0[45]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[47], buffer1[46], out buffer0[47], out buffer0[46]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[48], buffer1[49], out buffer0[48], out buffer0[49]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[51], buffer1[50], out buffer0[51], out buffer0[50]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[52], buffer1[53], out buffer0[52], out buffer0[53]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[55], buffer1[54], out buffer0[55], out buffer0[54]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[56], buffer1[57], out buffer0[56], out buffer0[57]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[59], buffer1[58], out buffer0[59], out buffer0[58]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[60], buffer1[61], out buffer0[60], out buffer0[61]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[63], buffer1[62], out buffer0[63], out buffer0[62]); + + // Stage 10 applies the pi/64 rotations to the penultimate odd-frequency level. + for (int i = 0; i < 16; i++) + { + int low = 32 + i; + int high = 63 - i; + int odd = Dct64Stage10RotationOrder[i]; + Butterfly(cospi[odd], cospi[64 - odd], buffer0[high], buffer0[low], ref buffer1, low, high, cosBit, in rounding); + } + + // Stage 11 applies the terminal permutation. The fused stages omit pass-through copies, so sources 4-7 and + // 16-31 remain in buffer0 at retirement, + // while every other source resides in buffer1. The mask maps that ownership through AV1 coefficient order. + ReadOnlySpan outputOrder = Dct64OutputOrder; + + for (int i = 0; i < 64; i++) + { + int sourceIndex = outputOrder[i]; + Vector512 value = ((Dct64Buffer0OutputMask >> i) & 1) != 0 ? buffer0[sourceIndex] : buffer1[sourceIndex]; + + Store(ref values, outputStride, i, value); + } + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + Av1TransformRounding rounding = Av1ForwardTransformArithmetic>.CreateRounding(cosBit); + + // Stage 1 consumes every spatial value before the strided block becomes available for final coefficients. + for (int i = 0; i < 32; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract( + Load>(ref values, inputStride, i), + Load>(ref values, inputStride, 63 - i), + out buffer0[i], + out buffer0[63 - i]); + } + + // Stage 2 begins the recursive radix-2 factorization and rotates the central odd-frequency pairs. + for (int i = 0; i < 16; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[i], buffer0[31 - i], out buffer1[i], out buffer1[31 - i]); + } + + for (int i = 0; i < 8; i++) + { + Butterfly(-cospi[32], cospi[32], buffer0[40 + i], buffer0[55 - i], ref buffer1, 40 + i, 55 - i, cosBit, in rounding); + } + + // Stage 3 reduces the even half and folds the next odd-frequency groups into paired sums and differences. + for (int i = 0; i < 8; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[i], buffer1[15 - i], out buffer0[i], out buffer0[15 - i]); + } + + for (int i = 0; i < 4; i++) + { + Butterfly(-cospi[32], cospi[32], buffer1[20 + i], buffer1[27 - i], ref buffer0, 20 + i, 27 - i, cosBit, in rounding); + } + + for (int i = 0; i < 8; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[32 + i], buffer1[47 - i], out buffer0[32 + i], out buffer0[47 - i]); + } + + for (int i = 0; i < 8; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[63 - i], buffer1[48 + i], out buffer0[63 - i], out buffer0[48 + i]); + } + + // Stage 4 continues the factorization as independent sixteen-value groups. + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[i], buffer0[7 - i], out buffer1[i], out buffer1[7 - i]); + } + + for (int i = 0; i < 2; i++) + { + Butterfly(-cospi[32], cospi[32], buffer0[10 + i], buffer0[13 - i], ref buffer1, 10 + i, 13 - i, cosBit, in rounding); + } + + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[16 + i], buffer0[23 - i], out buffer1[16 + i], out buffer1[23 - i]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[31 - i], buffer0[24 + i], out buffer1[31 - i], out buffer1[24 + i]); + Butterfly(-cospi[16], cospi[48], buffer0[36 + i], buffer0[59 - i], ref buffer1, 36 + i, 59 - i, cosBit, in rounding); + } + + for (int i = 4; i < 8; i++) + { + Butterfly(-cospi[48], -cospi[16], buffer0[36 + i], buffer0[59 - i], ref buffer1, 36 + i, 59 - i, cosBit, in rounding); + } + + // Stage 5 reduces the sixteen-value groups into the eight-value DCT and ADST building blocks. + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[i], buffer1[3 - i], out buffer0[i], out buffer0[3 - i]); + } + + Butterfly(-cospi[32], cospi[32], buffer1[5], buffer1[6], ref buffer0, 5, 6, cosBit, in rounding); + + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[8 + i], buffer1[11 - i], out buffer0[8 + i], out buffer0[11 - i]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[15 - i], buffer1[12 + i], out buffer0[15 - i], out buffer0[12 + i]); + Butterfly(-cospi[16], cospi[48], buffer1[18 + i], buffer1[29 - i], ref buffer0, 18 + i, 29 - i, cosBit, in rounding); + } + + for (int i = 2; i < 4; i++) + { + Butterfly(-cospi[48], -cospi[16], buffer1[18 + i], buffer1[29 - i], ref buffer0, 18 + i, 29 - i, cosBit, in rounding); + } + + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[32 + i], buffer1[39 - i], out buffer0[32 + i], out buffer0[39 - i]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[47 - i], buffer1[40 + i], out buffer0[47 - i], out buffer0[40 + i]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[48 + i], buffer1[55 - i], out buffer0[48 + i], out buffer0[55 - i]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[63 - i], buffer1[56 + i], out buffer0[63 - i], out buffer0[56 + i]); + } + + // Stage 6 completes the low-frequency DCT and rotates the first separated odd-frequency groups. + Butterfly(cospi[32], cospi[32], buffer0[0], buffer0[1], ref buffer1, 0, 1, cosBit, in rounding); + Butterfly(cospi[16], cospi[48], buffer0[3], buffer0[2], ref buffer1, 2, 3, cosBit, in rounding); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[4], buffer0[5], out buffer1[4], out buffer1[5]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[7], buffer0[6], out buffer1[7], out buffer1[6]); + Butterfly(-cospi[16], cospi[48], buffer0[9], buffer0[14], ref buffer1, 9, 14, cosBit, in rounding); + Butterfly(-cospi[48], -cospi[16], buffer0[10], buffer0[13], ref buffer1, 10, 13, cosBit, in rounding); + + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[16 + i], buffer0[19 - i], out buffer1[16 + i], out buffer1[19 - i]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[23 - i], buffer0[20 + i], out buffer1[23 - i], out buffer1[20 + i]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[24 + i], buffer0[27 - i], out buffer1[24 + i], out buffer1[27 - i]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[31 - i], buffer0[28 + i], out buffer1[31 - i], out buffer1[28 + i]); + Butterfly(-cospi[8], cospi[56], buffer0[34 + i], buffer0[61 - i], ref buffer1, 34 + i, 61 - i, cosBit, in rounding); + Butterfly(-cospi[40], cospi[24], buffer0[42 + i], buffer0[53 - i], ref buffer1, 42 + i, 53 - i, cosBit, in rounding); + } + + for (int i = 2; i < 4; i++) + { + Butterfly(-cospi[56], -cospi[8], buffer0[34 + i], buffer0[61 - i], ref buffer1, 34 + i, 61 - i, cosBit, in rounding); + Butterfly(-cospi[24], -cospi[40], buffer0[42 + i], buffer0[53 - i], ref buffer1, 42 + i, 53 - i, cosBit, in rounding); + } + + // Stage 7 merges adjacent odd-frequency terms with the sign pattern required by the next rotations. + Butterfly(cospi[8], cospi[56], buffer1[7], buffer1[4], ref buffer0, 4, 7, cosBit, in rounding); + Butterfly(cospi[40], cospi[24], buffer1[6], buffer1[5], ref buffer0, 5, 6, cosBit, in rounding); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[8], buffer1[9], out buffer0[8], out buffer0[9]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[11], buffer1[10], out buffer0[11], out buffer0[10]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[12], buffer1[13], out buffer0[12], out buffer0[13]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[15], buffer1[14], out buffer0[15], out buffer0[14]); + Butterfly(-cospi[8], cospi[56], buffer1[17], buffer1[30], ref buffer0, 17, 30, cosBit, in rounding); + Butterfly(-cospi[56], -cospi[8], buffer1[18], buffer1[29], ref buffer0, 18, 29, cosBit, in rounding); + Butterfly(-cospi[40], cospi[24], buffer1[21], buffer1[26], ref buffer0, 21, 26, cosBit, in rounding); + Butterfly(-cospi[24], -cospi[40], buffer1[22], buffer1[25], ref buffer0, 22, 25, cosBit, in rounding); + + for (int group = 0; group < 4; group++) + { + int offset = group * 8; + + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract( + buffer0[32 + offset + i], + buffer1[35 + offset - i], + out buffer0[32 + offset + i], + out buffer0[35 + offset - i]); + + Av1ForwardTransformArithmetic>.AddSubtract( + buffer0[39 + offset - i], + buffer1[36 + offset + i], + out buffer0[39 + offset - i], + out buffer0[36 + offset + i]); + } + } + + // Stage 8 applies the next level of odd-frequency rotations. + Butterfly(cospi[4], cospi[60], buffer0[15], buffer0[8], ref buffer1, 8, 15, cosBit, in rounding); + Butterfly(cospi[36], cospi[28], buffer0[14], buffer0[9], ref buffer1, 9, 14, cosBit, in rounding); + Butterfly(cospi[20], cospi[44], buffer0[13], buffer0[10], ref buffer1, 10, 13, cosBit, in rounding); + Butterfly(cospi[52], cospi[12], buffer0[12], buffer0[11], ref buffer1, 11, 12, cosBit, in rounding); + + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[16], buffer0[17], out buffer1[16], out buffer1[17]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[19], buffer0[18], out buffer1[19], out buffer1[18]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[20], buffer0[21], out buffer1[20], out buffer1[21]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[23], buffer0[22], out buffer1[23], out buffer1[22]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[24], buffer0[25], out buffer1[24], out buffer1[25]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[27], buffer0[26], out buffer1[27], out buffer1[26]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[28], buffer0[29], out buffer1[28], out buffer1[29]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[31], buffer0[30], out buffer1[31], out buffer1[30]); + + Butterfly(-cospi[4], cospi[60], buffer0[33], buffer0[62], ref buffer1, 33, 62, cosBit, in rounding); + Butterfly(-cospi[60], -cospi[4], buffer0[34], buffer0[61], ref buffer1, 34, 61, cosBit, in rounding); + Butterfly(-cospi[36], cospi[28], buffer0[37], buffer0[58], ref buffer1, 37, 58, cosBit, in rounding); + Butterfly(-cospi[28], -cospi[36], buffer0[38], buffer0[57], ref buffer1, 38, 57, cosBit, in rounding); + Butterfly(-cospi[20], cospi[44], buffer0[41], buffer0[54], ref buffer1, 41, 54, cosBit, in rounding); + Butterfly(-cospi[44], -cospi[20], buffer0[42], buffer0[53], ref buffer1, 42, 53, cosBit, in rounding); + Butterfly(-cospi[52], cospi[12], buffer0[45], buffer0[50], ref buffer1, 45, 50, cosBit, in rounding); + Butterfly(-cospi[12], -cospi[52], buffer0[46], buffer0[49], ref buffer1, 46, 49, cosBit, in rounding); + + // Stage 9 merges the remaining odd-frequency pairs before their terminal rotations. The table preserves the + // non-linear rotation order while keeping the constants in compile-time data. + for (int i = 0; i < 8; i++) + { + int low = 16 + i; + int high = 31 - i; + int odd = Dct64Stage9RotationOrder[i]; + Butterfly(cospi[odd], cospi[64 - odd], buffer1[high], buffer1[low], ref buffer0, low, high, cosBit, in rounding); + } + + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[32], buffer1[33], out buffer0[32], out buffer0[33]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[35], buffer1[34], out buffer0[35], out buffer0[34]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[36], buffer1[37], out buffer0[36], out buffer0[37]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[39], buffer1[38], out buffer0[39], out buffer0[38]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[40], buffer1[41], out buffer0[40], out buffer0[41]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[43], buffer1[42], out buffer0[43], out buffer0[42]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[44], buffer1[45], out buffer0[44], out buffer0[45]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[47], buffer1[46], out buffer0[47], out buffer0[46]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[48], buffer1[49], out buffer0[48], out buffer0[49]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[51], buffer1[50], out buffer0[51], out buffer0[50]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[52], buffer1[53], out buffer0[52], out buffer0[53]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[55], buffer1[54], out buffer0[55], out buffer0[54]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[56], buffer1[57], out buffer0[56], out buffer0[57]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[59], buffer1[58], out buffer0[59], out buffer0[58]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[60], buffer1[61], out buffer0[60], out buffer0[61]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[63], buffer1[62], out buffer0[63], out buffer0[62]); + + // Stage 10 applies the pi/64 rotations to the penultimate odd-frequency level. + for (int i = 0; i < 16; i++) + { + int low = 32 + i; + int high = 63 - i; + int odd = Dct64Stage10RotationOrder[i]; + Butterfly(cospi[odd], cospi[64 - odd], buffer0[high], buffer0[low], ref buffer1, low, high, cosBit, in rounding); + } + + // Stage 11 applies the terminal permutation. The fused stages omit pass-through copies, so sources 4-7 and + // 16-31 remain in buffer0 at retirement, + // while every other source resides in buffer1. The mask maps that ownership through AV1 coefficient order. + ReadOnlySpan outputOrder = Dct64OutputOrder; + + for (int i = 0; i < 64; i++) + { + int sourceIndex = outputOrder[i]; + Vector128 value = ((Dct64Buffer0OutputMask >> i) & 1) != 0 ? buffer0[sourceIndex] : buffer1[sourceIndex]; + + Store(ref values, outputStride, i, value); + } + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + Av1TransformRounding rounding = Av1ForwardTransformArithmetic>.CreateRounding(cosBit); + + // Stage 1 consumes every spatial value before the strided block becomes available for final coefficients. + for (int i = 0; i < 32; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract( + Load>(ref values, inputStride, i), + Load>(ref values, inputStride, 63 - i), + out buffer0[i], + out buffer0[63 - i]); + } + + // Stage 2 begins the recursive radix-2 factorization and rotates the central odd-frequency pairs. + for (int i = 0; i < 16; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[i], buffer0[31 - i], out buffer1[i], out buffer1[31 - i]); + } + + for (int i = 0; i < 8; i++) + { + Butterfly(-cospi[32], cospi[32], buffer0[40 + i], buffer0[55 - i], ref buffer1, 40 + i, 55 - i, cosBit, in rounding); + } + + // Stage 3 reduces the even half and folds the next odd-frequency groups into paired sums and differences. + for (int i = 0; i < 8; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[i], buffer1[15 - i], out buffer0[i], out buffer0[15 - i]); + } + + for (int i = 0; i < 4; i++) + { + Butterfly(-cospi[32], cospi[32], buffer1[20 + i], buffer1[27 - i], ref buffer0, 20 + i, 27 - i, cosBit, in rounding); + } + + for (int i = 0; i < 8; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[32 + i], buffer1[47 - i], out buffer0[32 + i], out buffer0[47 - i]); + } + + for (int i = 0; i < 8; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[63 - i], buffer1[48 + i], out buffer0[63 - i], out buffer0[48 + i]); + } + + // Stage 4 continues the factorization as independent sixteen-value groups. + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[i], buffer0[7 - i], out buffer1[i], out buffer1[7 - i]); + } + + for (int i = 0; i < 2; i++) + { + Butterfly(-cospi[32], cospi[32], buffer0[10 + i], buffer0[13 - i], ref buffer1, 10 + i, 13 - i, cosBit, in rounding); + } + + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[16 + i], buffer0[23 - i], out buffer1[16 + i], out buffer1[23 - i]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[31 - i], buffer0[24 + i], out buffer1[31 - i], out buffer1[24 + i]); + Butterfly(-cospi[16], cospi[48], buffer0[36 + i], buffer0[59 - i], ref buffer1, 36 + i, 59 - i, cosBit, in rounding); + } + + for (int i = 4; i < 8; i++) + { + Butterfly(-cospi[48], -cospi[16], buffer0[36 + i], buffer0[59 - i], ref buffer1, 36 + i, 59 - i, cosBit, in rounding); + } + + // Stage 5 reduces the sixteen-value groups into the eight-value DCT and ADST building blocks. + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[i], buffer1[3 - i], out buffer0[i], out buffer0[3 - i]); + } + + Butterfly(-cospi[32], cospi[32], buffer1[5], buffer1[6], ref buffer0, 5, 6, cosBit, in rounding); + + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[8 + i], buffer1[11 - i], out buffer0[8 + i], out buffer0[11 - i]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[15 - i], buffer1[12 + i], out buffer0[15 - i], out buffer0[12 + i]); + Butterfly(-cospi[16], cospi[48], buffer1[18 + i], buffer1[29 - i], ref buffer0, 18 + i, 29 - i, cosBit, in rounding); + } + + for (int i = 2; i < 4; i++) + { + Butterfly(-cospi[48], -cospi[16], buffer1[18 + i], buffer1[29 - i], ref buffer0, 18 + i, 29 - i, cosBit, in rounding); + } + + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[32 + i], buffer1[39 - i], out buffer0[32 + i], out buffer0[39 - i]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[47 - i], buffer1[40 + i], out buffer0[47 - i], out buffer0[40 + i]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[48 + i], buffer1[55 - i], out buffer0[48 + i], out buffer0[55 - i]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[63 - i], buffer1[56 + i], out buffer0[63 - i], out buffer0[56 + i]); + } + + // Stage 6 completes the low-frequency DCT and rotates the first separated odd-frequency groups. + Butterfly(cospi[32], cospi[32], buffer0[0], buffer0[1], ref buffer1, 0, 1, cosBit, in rounding); + Butterfly(cospi[16], cospi[48], buffer0[3], buffer0[2], ref buffer1, 2, 3, cosBit, in rounding); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[4], buffer0[5], out buffer1[4], out buffer1[5]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[7], buffer0[6], out buffer1[7], out buffer1[6]); + Butterfly(-cospi[16], cospi[48], buffer0[9], buffer0[14], ref buffer1, 9, 14, cosBit, in rounding); + Butterfly(-cospi[48], -cospi[16], buffer0[10], buffer0[13], ref buffer1, 10, 13, cosBit, in rounding); + + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[16 + i], buffer0[19 - i], out buffer1[16 + i], out buffer1[19 - i]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[23 - i], buffer0[20 + i], out buffer1[23 - i], out buffer1[20 + i]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[24 + i], buffer0[27 - i], out buffer1[24 + i], out buffer1[27 - i]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[31 - i], buffer0[28 + i], out buffer1[31 - i], out buffer1[28 + i]); + Butterfly(-cospi[8], cospi[56], buffer0[34 + i], buffer0[61 - i], ref buffer1, 34 + i, 61 - i, cosBit, in rounding); + Butterfly(-cospi[40], cospi[24], buffer0[42 + i], buffer0[53 - i], ref buffer1, 42 + i, 53 - i, cosBit, in rounding); + } + + for (int i = 2; i < 4; i++) + { + Butterfly(-cospi[56], -cospi[8], buffer0[34 + i], buffer0[61 - i], ref buffer1, 34 + i, 61 - i, cosBit, in rounding); + Butterfly(-cospi[24], -cospi[40], buffer0[42 + i], buffer0[53 - i], ref buffer1, 42 + i, 53 - i, cosBit, in rounding); + } + + // Stage 7 merges adjacent odd-frequency terms with the sign pattern required by the next rotations. + Butterfly(cospi[8], cospi[56], buffer1[7], buffer1[4], ref buffer0, 4, 7, cosBit, in rounding); + Butterfly(cospi[40], cospi[24], buffer1[6], buffer1[5], ref buffer0, 5, 6, cosBit, in rounding); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[8], buffer1[9], out buffer0[8], out buffer0[9]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[11], buffer1[10], out buffer0[11], out buffer0[10]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[12], buffer1[13], out buffer0[12], out buffer0[13]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[15], buffer1[14], out buffer0[15], out buffer0[14]); + Butterfly(-cospi[8], cospi[56], buffer1[17], buffer1[30], ref buffer0, 17, 30, cosBit, in rounding); + Butterfly(-cospi[56], -cospi[8], buffer1[18], buffer1[29], ref buffer0, 18, 29, cosBit, in rounding); + Butterfly(-cospi[40], cospi[24], buffer1[21], buffer1[26], ref buffer0, 21, 26, cosBit, in rounding); + Butterfly(-cospi[24], -cospi[40], buffer1[22], buffer1[25], ref buffer0, 22, 25, cosBit, in rounding); + + for (int group = 0; group < 4; group++) + { + int offset = group * 8; + + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract( + buffer0[32 + offset + i], + buffer1[35 + offset - i], + out buffer0[32 + offset + i], + out buffer0[35 + offset - i]); + + Av1ForwardTransformArithmetic>.AddSubtract( + buffer0[39 + offset - i], + buffer1[36 + offset + i], + out buffer0[39 + offset - i], + out buffer0[36 + offset + i]); + } + } + + // Stage 8 applies the next level of odd-frequency rotations. + Butterfly(cospi[4], cospi[60], buffer0[15], buffer0[8], ref buffer1, 8, 15, cosBit, in rounding); + Butterfly(cospi[36], cospi[28], buffer0[14], buffer0[9], ref buffer1, 9, 14, cosBit, in rounding); + Butterfly(cospi[20], cospi[44], buffer0[13], buffer0[10], ref buffer1, 10, 13, cosBit, in rounding); + Butterfly(cospi[52], cospi[12], buffer0[12], buffer0[11], ref buffer1, 11, 12, cosBit, in rounding); + + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[16], buffer0[17], out buffer1[16], out buffer1[17]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[19], buffer0[18], out buffer1[19], out buffer1[18]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[20], buffer0[21], out buffer1[20], out buffer1[21]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[23], buffer0[22], out buffer1[23], out buffer1[22]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[24], buffer0[25], out buffer1[24], out buffer1[25]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[27], buffer0[26], out buffer1[27], out buffer1[26]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[28], buffer0[29], out buffer1[28], out buffer1[29]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[31], buffer0[30], out buffer1[31], out buffer1[30]); + + Butterfly(-cospi[4], cospi[60], buffer0[33], buffer0[62], ref buffer1, 33, 62, cosBit, in rounding); + Butterfly(-cospi[60], -cospi[4], buffer0[34], buffer0[61], ref buffer1, 34, 61, cosBit, in rounding); + Butterfly(-cospi[36], cospi[28], buffer0[37], buffer0[58], ref buffer1, 37, 58, cosBit, in rounding); + Butterfly(-cospi[28], -cospi[36], buffer0[38], buffer0[57], ref buffer1, 38, 57, cosBit, in rounding); + Butterfly(-cospi[20], cospi[44], buffer0[41], buffer0[54], ref buffer1, 41, 54, cosBit, in rounding); + Butterfly(-cospi[44], -cospi[20], buffer0[42], buffer0[53], ref buffer1, 42, 53, cosBit, in rounding); + Butterfly(-cospi[52], cospi[12], buffer0[45], buffer0[50], ref buffer1, 45, 50, cosBit, in rounding); + Butterfly(-cospi[12], -cospi[52], buffer0[46], buffer0[49], ref buffer1, 46, 49, cosBit, in rounding); + + // Stage 9 merges the remaining odd-frequency pairs before their terminal rotations. The table preserves the + // non-linear rotation order while keeping the constants in compile-time data. + for (int i = 0; i < 8; i++) + { + int low = 16 + i; + int high = 31 - i; + int odd = Dct64Stage9RotationOrder[i]; + Butterfly(cospi[odd], cospi[64 - odd], buffer1[high], buffer1[low], ref buffer0, low, high, cosBit, in rounding); + } + + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[32], buffer1[33], out buffer0[32], out buffer0[33]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[35], buffer1[34], out buffer0[35], out buffer0[34]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[36], buffer1[37], out buffer0[36], out buffer0[37]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[39], buffer1[38], out buffer0[39], out buffer0[38]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[40], buffer1[41], out buffer0[40], out buffer0[41]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[43], buffer1[42], out buffer0[43], out buffer0[42]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[44], buffer1[45], out buffer0[44], out buffer0[45]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[47], buffer1[46], out buffer0[47], out buffer0[46]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[48], buffer1[49], out buffer0[48], out buffer0[49]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[51], buffer1[50], out buffer0[51], out buffer0[50]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[52], buffer1[53], out buffer0[52], out buffer0[53]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[55], buffer1[54], out buffer0[55], out buffer0[54]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[56], buffer1[57], out buffer0[56], out buffer0[57]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[59], buffer1[58], out buffer0[59], out buffer0[58]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[60], buffer1[61], out buffer0[60], out buffer0[61]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[63], buffer1[62], out buffer0[63], out buffer0[62]); + + // Stage 10 applies the pi/64 rotations to the penultimate odd-frequency level. + for (int i = 0; i < 16; i++) + { + int low = 32 + i; + int high = 63 - i; + int odd = Dct64Stage10RotationOrder[i]; + Butterfly(cospi[odd], cospi[64 - odd], buffer0[high], buffer0[low], ref buffer1, low, high, cosBit, in rounding); + } + + // Stage 11 applies the terminal permutation. The fused stages omit pass-through copies, so sources 4-7 and + // 16-31 remain in buffer0 at retirement, + // while every other source resides in buffer1. The mask maps that ownership through AV1 coefficient order. + ReadOnlySpan outputOrder = Dct64OutputOrder; + + for (int i = 0; i < 64; i++) + { + int sourceIndex = outputOrder[i]; + Vector256 value = ((Dct64Buffer0OutputMask >> i) & 1) != 0 ? buffer0[sourceIndex] : buffer1[sourceIndex]; + + Store(ref values, outputStride, i, value); + } + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + Av1TransformRounding rounding = Av1ForwardTransformArithmetic>.CreateRounding(cosBit); + + // Stage 1 consumes every spatial value before the strided block becomes available for final coefficients. + for (int i = 0; i < 32; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract( + Load>(ref values, inputStride, i), + Load>(ref values, inputStride, 63 - i), + out buffer0[i], + out buffer0[63 - i]); + } + + // Stage 2 begins the recursive radix-2 factorization and rotates the central odd-frequency pairs. + for (int i = 0; i < 16; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[i], buffer0[31 - i], out buffer1[i], out buffer1[31 - i]); + } + + for (int i = 0; i < 8; i++) + { + Butterfly(-cospi[32], cospi[32], buffer0[40 + i], buffer0[55 - i], ref buffer1, 40 + i, 55 - i, cosBit, in rounding); + } + + // Stage 3 reduces the even half and folds the next odd-frequency groups into paired sums and differences. + for (int i = 0; i < 8; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[i], buffer1[15 - i], out buffer0[i], out buffer0[15 - i]); + } + + for (int i = 0; i < 4; i++) + { + Butterfly(-cospi[32], cospi[32], buffer1[20 + i], buffer1[27 - i], ref buffer0, 20 + i, 27 - i, cosBit, in rounding); + } + + for (int i = 0; i < 8; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[32 + i], buffer1[47 - i], out buffer0[32 + i], out buffer0[47 - i]); + } + + for (int i = 0; i < 8; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[63 - i], buffer1[48 + i], out buffer0[63 - i], out buffer0[48 + i]); + } + + // Stage 4 continues the factorization as independent sixteen-value groups. + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[i], buffer0[7 - i], out buffer1[i], out buffer1[7 - i]); + } + + for (int i = 0; i < 2; i++) + { + Butterfly(-cospi[32], cospi[32], buffer0[10 + i], buffer0[13 - i], ref buffer1, 10 + i, 13 - i, cosBit, in rounding); + } + + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[16 + i], buffer0[23 - i], out buffer1[16 + i], out buffer1[23 - i]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[31 - i], buffer0[24 + i], out buffer1[31 - i], out buffer1[24 + i]); + Butterfly(-cospi[16], cospi[48], buffer0[36 + i], buffer0[59 - i], ref buffer1, 36 + i, 59 - i, cosBit, in rounding); + } + + for (int i = 4; i < 8; i++) + { + Butterfly(-cospi[48], -cospi[16], buffer0[36 + i], buffer0[59 - i], ref buffer1, 36 + i, 59 - i, cosBit, in rounding); + } + + // Stage 5 reduces the sixteen-value groups into the eight-value DCT and ADST building blocks. + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[i], buffer1[3 - i], out buffer0[i], out buffer0[3 - i]); + } + + Butterfly(-cospi[32], cospi[32], buffer1[5], buffer1[6], ref buffer0, 5, 6, cosBit, in rounding); + + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[8 + i], buffer1[11 - i], out buffer0[8 + i], out buffer0[11 - i]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[15 - i], buffer1[12 + i], out buffer0[15 - i], out buffer0[12 + i]); + Butterfly(-cospi[16], cospi[48], buffer1[18 + i], buffer1[29 - i], ref buffer0, 18 + i, 29 - i, cosBit, in rounding); + } + + for (int i = 2; i < 4; i++) + { + Butterfly(-cospi[48], -cospi[16], buffer1[18 + i], buffer1[29 - i], ref buffer0, 18 + i, 29 - i, cosBit, in rounding); + } + + for (int i = 0; i < 4; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[32 + i], buffer1[39 - i], out buffer0[32 + i], out buffer0[39 - i]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[47 - i], buffer1[40 + i], out buffer0[47 - i], out buffer0[40 + i]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[48 + i], buffer1[55 - i], out buffer0[48 + i], out buffer0[55 - i]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[63 - i], buffer1[56 + i], out buffer0[63 - i], out buffer0[56 + i]); + } + + // Stage 6 completes the low-frequency DCT and rotates the first separated odd-frequency groups. + Butterfly(cospi[32], cospi[32], buffer0[0], buffer0[1], ref buffer1, 0, 1, cosBit, in rounding); + Butterfly(cospi[16], cospi[48], buffer0[3], buffer0[2], ref buffer1, 2, 3, cosBit, in rounding); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[4], buffer0[5], out buffer1[4], out buffer1[5]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[7], buffer0[6], out buffer1[7], out buffer1[6]); + Butterfly(-cospi[16], cospi[48], buffer0[9], buffer0[14], ref buffer1, 9, 14, cosBit, in rounding); + Butterfly(-cospi[48], -cospi[16], buffer0[10], buffer0[13], ref buffer1, 10, 13, cosBit, in rounding); + + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[16 + i], buffer0[19 - i], out buffer1[16 + i], out buffer1[19 - i]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[23 - i], buffer0[20 + i], out buffer1[23 - i], out buffer1[20 + i]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[24 + i], buffer0[27 - i], out buffer1[24 + i], out buffer1[27 - i]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[31 - i], buffer0[28 + i], out buffer1[31 - i], out buffer1[28 + i]); + Butterfly(-cospi[8], cospi[56], buffer0[34 + i], buffer0[61 - i], ref buffer1, 34 + i, 61 - i, cosBit, in rounding); + Butterfly(-cospi[40], cospi[24], buffer0[42 + i], buffer0[53 - i], ref buffer1, 42 + i, 53 - i, cosBit, in rounding); + } + + for (int i = 2; i < 4; i++) + { + Butterfly(-cospi[56], -cospi[8], buffer0[34 + i], buffer0[61 - i], ref buffer1, 34 + i, 61 - i, cosBit, in rounding); + Butterfly(-cospi[24], -cospi[40], buffer0[42 + i], buffer0[53 - i], ref buffer1, 42 + i, 53 - i, cosBit, in rounding); + } + + // Stage 7 merges adjacent odd-frequency terms with the sign pattern required by the next rotations. + Butterfly(cospi[8], cospi[56], buffer1[7], buffer1[4], ref buffer0, 4, 7, cosBit, in rounding); + Butterfly(cospi[40], cospi[24], buffer1[6], buffer1[5], ref buffer0, 5, 6, cosBit, in rounding); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[8], buffer1[9], out buffer0[8], out buffer0[9]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[11], buffer1[10], out buffer0[11], out buffer0[10]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[12], buffer1[13], out buffer0[12], out buffer0[13]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[15], buffer1[14], out buffer0[15], out buffer0[14]); + Butterfly(-cospi[8], cospi[56], buffer1[17], buffer1[30], ref buffer0, 17, 30, cosBit, in rounding); + Butterfly(-cospi[56], -cospi[8], buffer1[18], buffer1[29], ref buffer0, 18, 29, cosBit, in rounding); + Butterfly(-cospi[40], cospi[24], buffer1[21], buffer1[26], ref buffer0, 21, 26, cosBit, in rounding); + Butterfly(-cospi[24], -cospi[40], buffer1[22], buffer1[25], ref buffer0, 22, 25, cosBit, in rounding); + + for (int group = 0; group < 4; group++) + { + int offset = group * 8; + + for (int i = 0; i < 2; i++) + { + Av1ForwardTransformArithmetic>.AddSubtract( + buffer0[32 + offset + i], + buffer1[35 + offset - i], + out buffer0[32 + offset + i], + out buffer0[35 + offset - i]); + + Av1ForwardTransformArithmetic>.AddSubtract( + buffer0[39 + offset - i], + buffer1[36 + offset + i], + out buffer0[39 + offset - i], + out buffer0[36 + offset + i]); + } + } + + // Stage 8 applies the next level of odd-frequency rotations. + Butterfly(cospi[4], cospi[60], buffer0[15], buffer0[8], ref buffer1, 8, 15, cosBit, in rounding); + Butterfly(cospi[36], cospi[28], buffer0[14], buffer0[9], ref buffer1, 9, 14, cosBit, in rounding); + Butterfly(cospi[20], cospi[44], buffer0[13], buffer0[10], ref buffer1, 10, 13, cosBit, in rounding); + Butterfly(cospi[52], cospi[12], buffer0[12], buffer0[11], ref buffer1, 11, 12, cosBit, in rounding); + + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[16], buffer0[17], out buffer1[16], out buffer1[17]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[19], buffer0[18], out buffer1[19], out buffer1[18]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[20], buffer0[21], out buffer1[20], out buffer1[21]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[23], buffer0[22], out buffer1[23], out buffer1[22]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[24], buffer0[25], out buffer1[24], out buffer1[25]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[27], buffer0[26], out buffer1[27], out buffer1[26]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[28], buffer0[29], out buffer1[28], out buffer1[29]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[31], buffer0[30], out buffer1[31], out buffer1[30]); + + Butterfly(-cospi[4], cospi[60], buffer0[33], buffer0[62], ref buffer1, 33, 62, cosBit, in rounding); + Butterfly(-cospi[60], -cospi[4], buffer0[34], buffer0[61], ref buffer1, 34, 61, cosBit, in rounding); + Butterfly(-cospi[36], cospi[28], buffer0[37], buffer0[58], ref buffer1, 37, 58, cosBit, in rounding); + Butterfly(-cospi[28], -cospi[36], buffer0[38], buffer0[57], ref buffer1, 38, 57, cosBit, in rounding); + Butterfly(-cospi[20], cospi[44], buffer0[41], buffer0[54], ref buffer1, 41, 54, cosBit, in rounding); + Butterfly(-cospi[44], -cospi[20], buffer0[42], buffer0[53], ref buffer1, 42, 53, cosBit, in rounding); + Butterfly(-cospi[52], cospi[12], buffer0[45], buffer0[50], ref buffer1, 45, 50, cosBit, in rounding); + Butterfly(-cospi[12], -cospi[52], buffer0[46], buffer0[49], ref buffer1, 46, 49, cosBit, in rounding); + + // Stage 9 merges the remaining odd-frequency pairs before their terminal rotations. The table preserves the + // non-linear rotation order while keeping the constants in compile-time data. + for (int i = 0; i < 8; i++) + { + int low = 16 + i; + int high = 31 - i; + int odd = Dct64Stage9RotationOrder[i]; + Butterfly(cospi[odd], cospi[64 - odd], buffer1[high], buffer1[low], ref buffer0, low, high, cosBit, in rounding); + } + + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[32], buffer1[33], out buffer0[32], out buffer0[33]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[35], buffer1[34], out buffer0[35], out buffer0[34]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[36], buffer1[37], out buffer0[36], out buffer0[37]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[39], buffer1[38], out buffer0[39], out buffer0[38]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[40], buffer1[41], out buffer0[40], out buffer0[41]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[43], buffer1[42], out buffer0[43], out buffer0[42]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[44], buffer1[45], out buffer0[44], out buffer0[45]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[47], buffer1[46], out buffer0[47], out buffer0[46]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[48], buffer1[49], out buffer0[48], out buffer0[49]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[51], buffer1[50], out buffer0[51], out buffer0[50]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[52], buffer1[53], out buffer0[52], out buffer0[53]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[55], buffer1[54], out buffer0[55], out buffer0[54]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[56], buffer1[57], out buffer0[56], out buffer0[57]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[59], buffer1[58], out buffer0[59], out buffer0[58]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[60], buffer1[61], out buffer0[60], out buffer0[61]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[63], buffer1[62], out buffer0[63], out buffer0[62]); + + // Stage 10 applies the pi/64 rotations to the penultimate odd-frequency level. + for (int i = 0; i < 16; i++) + { + int low = 32 + i; + int high = 63 - i; + int odd = Dct64Stage10RotationOrder[i]; + Butterfly(cospi[odd], cospi[64 - odd], buffer0[high], buffer0[low], ref buffer1, low, high, cosBit, in rounding); + } + + // Stage 11 applies the terminal permutation. The fused stages omit pass-through copies, so sources 4-7 and + // 16-31 remain in buffer0 at retirement, + // while every other source resides in buffer1. The mask maps that ownership through AV1 coefficient order. + ReadOnlySpan outputOrder = Dct64OutputOrder; + + for (int i = 0; i < 64; i++) + { + int sourceIndex = outputOrder[i]; + Vector512 value = ((Dct64Buffer0OutputMask >> i) & 1) != 0 ? buffer0[sourceIndex] : buffer1[sourceIndex]; + + Store(ref values, outputStride, i, value); + } + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Transform/Forward/Av1ForwardTransformer.Dct8Operator.cs b/src/ImageSharp/Formats/Heif/Av1/Transform/Forward/Av1ForwardTransformer.Dct8Operator.cs new file mode 100644 index 0000000000..c2223d45fe --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Transform/Forward/Av1ForwardTransformer.Dct8Operator.cs @@ -0,0 +1,859 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.Intrinsics; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform.Forward; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +/// +/// Implements the eight-point forward DCT stage network. +/// +internal static partial class Av1ForwardTransformer +{ + /// + /// Implements the eight-point forward transform for every supported lane width. + /// + internal readonly struct Dct8Operator : IAv1ForwardTransform1dOperator + { + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector buffer0, + ref Av1TransformVector buffer1, + int cosBit) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + Av1TransformRounding rounding = Av1ForwardTransformArithmetic.CreateRounding(cosBit); + + // Stages 1 and 2 split the even and odd terms. The asymmetric destinations mirror Highway's buffer + // ownership, allowing the later even butterflies to write their final coefficients directly to the block. + Av1ForwardTransformArithmetic.AddSubtract( + Load(ref values, inputStride, 0), + Load(ref values, inputStride, 7), + out buffer0[0], + out buffer1[7]); + + Av1ForwardTransformArithmetic.AddSubtract( + Load(ref values, inputStride, 1), + Load(ref values, inputStride, 6), + out buffer0[1], + out buffer0[6]); + + Av1ForwardTransformArithmetic.AddSubtract( + Load(ref values, inputStride, 2), + Load(ref values, inputStride, 5), + out buffer0[2], + out buffer0[5]); + + Av1ForwardTransformArithmetic.AddSubtract( + Load(ref values, inputStride, 3), + Load(ref values, inputStride, 4), + out buffer0[3], + out buffer1[4]); + + Av1ForwardTransformArithmetic.AddSubtract(buffer0[0], buffer0[3], out buffer1[0], out buffer1[3]); + Av1ForwardTransformArithmetic.AddSubtract(buffer0[1], buffer0[2], out buffer1[1], out buffer1[2]); + Av1ForwardTransformArithmetic.Butterfly( + -cospi[32], + cospi[32], + buffer0[5], + buffer0[6], + out buffer1[5], + out buffer1[6], + cosBit, + in rounding); + + // Stage 3 completes the even half directly in coefficient order and prepares the four remaining odd terms. + Av1ForwardTransformArithmetic.Butterfly( + cospi[32], + cospi[32], + buffer1[0], + buffer1[1], + out int output0, + out int output4, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic.Butterfly( + cospi[16], + cospi[48], + buffer1[3], + buffer1[2], + out int output2, + out int output6, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic.AddSubtract(buffer1[4], buffer1[5], out buffer0[4], out buffer0[5]); + Av1ForwardTransformArithmetic.AddSubtract(buffer1[7], buffer1[6], out buffer0[7], out buffer0[6]); + + // Highway fuses the final two stages because no intermediate value is reused after either rotation. + Av1ForwardTransformArithmetic.Butterfly( + cospi[8], + cospi[56], + buffer0[7], + buffer0[4], + out int output1, + out int output7, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic.Butterfly( + cospi[40], + cospi[24], + buffer0[6], + buffer0[5], + out int output5, + out int output3, + cosBit, + in rounding); + + Store(ref values, outputStride, 0, output0); + Store(ref values, outputStride, 1, output1); + Store(ref values, outputStride, 2, output2); + Store(ref values, outputStride, 3, output3); + Store(ref values, outputStride, 4, output4); + Store(ref values, outputStride, 5, output5); + Store(ref values, outputStride, 6, output6); + Store(ref values, outputStride, 7, output7); + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector buffer0, + ref Av1TransformVector buffer1, + int cosBit) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + Av1TransformRounding rounding = Av1ForwardTransformArithmetic.CreateRounding(cosBit); + + // Stages 1 and 2 split the even and odd terms. The asymmetric destinations mirror Highway's buffer + // ownership, allowing the later even butterflies to write their final coefficients directly to the block. + Av1ForwardTransformArithmetic.AddSubtract( + Load(ref values, inputStride, 0), + Load(ref values, inputStride, 7), + out buffer0[0], + out buffer1[7]); + + Av1ForwardTransformArithmetic.AddSubtract( + Load(ref values, inputStride, 1), + Load(ref values, inputStride, 6), + out buffer0[1], + out buffer0[6]); + + Av1ForwardTransformArithmetic.AddSubtract( + Load(ref values, inputStride, 2), + Load(ref values, inputStride, 5), + out buffer0[2], + out buffer0[5]); + + Av1ForwardTransformArithmetic.AddSubtract( + Load(ref values, inputStride, 3), + Load(ref values, inputStride, 4), + out buffer0[3], + out buffer1[4]); + + Av1ForwardTransformArithmetic.AddSubtract(buffer0[0], buffer0[3], out buffer1[0], out buffer1[3]); + Av1ForwardTransformArithmetic.AddSubtract(buffer0[1], buffer0[2], out buffer1[1], out buffer1[2]); + Av1ForwardTransformArithmetic.Butterfly( + -cospi[32], + cospi[32], + buffer0[5], + buffer0[6], + out buffer1[5], + out buffer1[6], + cosBit, + in rounding); + + // Stage 3 completes the even half directly in coefficient order and prepares the four remaining odd terms. + Av1ForwardTransformArithmetic.Butterfly( + cospi[32], + cospi[32], + buffer1[0], + buffer1[1], + out short output0, + out short output4, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic.Butterfly( + cospi[16], + cospi[48], + buffer1[3], + buffer1[2], + out short output2, + out short output6, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic.AddSubtract(buffer1[4], buffer1[5], out buffer0[4], out buffer0[5]); + Av1ForwardTransformArithmetic.AddSubtract(buffer1[7], buffer1[6], out buffer0[7], out buffer0[6]); + + // Highway fuses the final two stages because no intermediate value is reused after either rotation. + Av1ForwardTransformArithmetic.Butterfly( + cospi[8], + cospi[56], + buffer0[7], + buffer0[4], + out short output1, + out short output7, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic.Butterfly( + cospi[40], + cospi[24], + buffer0[6], + buffer0[5], + out short output5, + out short output3, + cosBit, + in rounding); + + Store(ref values, outputStride, 0, output0); + Store(ref values, outputStride, 1, output1); + Store(ref values, outputStride, 2, output2); + Store(ref values, outputStride, 3, output3); + Store(ref values, outputStride, 4, output4); + Store(ref values, outputStride, 5, output5); + Store(ref values, outputStride, 6, output6); + Store(ref values, outputStride, 7, output7); + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + Av1TransformRounding rounding = Av1ForwardTransformArithmetic>.CreateRounding(cosBit); + + // Stages 1 and 2 split the even and odd terms. The asymmetric destinations mirror Highway's buffer + // ownership, allowing the later even butterflies to write their final coefficients directly to the block. + Av1ForwardTransformArithmetic>.AddSubtract( + Load>(ref values, inputStride, 0), + Load>(ref values, inputStride, 7), + out buffer0[0], + out buffer1[7]); + + Av1ForwardTransformArithmetic>.AddSubtract( + Load>(ref values, inputStride, 1), + Load>(ref values, inputStride, 6), + out buffer0[1], + out buffer0[6]); + + Av1ForwardTransformArithmetic>.AddSubtract( + Load>(ref values, inputStride, 2), + Load>(ref values, inputStride, 5), + out buffer0[2], + out buffer0[5]); + + Av1ForwardTransformArithmetic>.AddSubtract( + Load>(ref values, inputStride, 3), + Load>(ref values, inputStride, 4), + out buffer0[3], + out buffer1[4]); + + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[0], buffer0[3], out buffer1[0], out buffer1[3]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[1], buffer0[2], out buffer1[1], out buffer1[2]); + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[32], + cospi[32], + buffer0[5], + buffer0[6], + out buffer1[5], + out buffer1[6], + cosBit, + in rounding); + + // Stage 3 completes the even half directly in coefficient order and prepares the four remaining odd terms. + Av1ForwardTransformArithmetic>.Butterfly( + cospi[32], + cospi[32], + buffer1[0], + buffer1[1], + out Vector128 output0, + out Vector128 output4, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + cospi[16], + cospi[48], + buffer1[3], + buffer1[2], + out Vector128 output2, + out Vector128 output6, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[4], buffer1[5], out buffer0[4], out buffer0[5]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[7], buffer1[6], out buffer0[7], out buffer0[6]); + + // Highway fuses the final two stages because no intermediate value is reused after either rotation. + Av1ForwardTransformArithmetic>.Butterfly( + cospi[8], + cospi[56], + buffer0[7], + buffer0[4], + out Vector128 output1, + out Vector128 output7, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + cospi[40], + cospi[24], + buffer0[6], + buffer0[5], + out Vector128 output5, + out Vector128 output3, + cosBit, + in rounding); + + Store(ref values, outputStride, 0, output0); + Store(ref values, outputStride, 1, output1); + Store(ref values, outputStride, 2, output2); + Store(ref values, outputStride, 3, output3); + Store(ref values, outputStride, 4, output4); + Store(ref values, outputStride, 5, output5); + Store(ref values, outputStride, 6, output6); + Store(ref values, outputStride, 7, output7); + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + Av1TransformRounding rounding = Av1ForwardTransformArithmetic>.CreateRounding(cosBit); + + // Stages 1 and 2 split the even and odd terms. The asymmetric destinations mirror Highway's buffer + // ownership, allowing the later even butterflies to write their final coefficients directly to the block. + Av1ForwardTransformArithmetic>.AddSubtract( + Load>(ref values, inputStride, 0), + Load>(ref values, inputStride, 7), + out buffer0[0], + out buffer1[7]); + + Av1ForwardTransformArithmetic>.AddSubtract( + Load>(ref values, inputStride, 1), + Load>(ref values, inputStride, 6), + out buffer0[1], + out buffer0[6]); + + Av1ForwardTransformArithmetic>.AddSubtract( + Load>(ref values, inputStride, 2), + Load>(ref values, inputStride, 5), + out buffer0[2], + out buffer0[5]); + + Av1ForwardTransformArithmetic>.AddSubtract( + Load>(ref values, inputStride, 3), + Load>(ref values, inputStride, 4), + out buffer0[3], + out buffer1[4]); + + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[0], buffer0[3], out buffer1[0], out buffer1[3]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[1], buffer0[2], out buffer1[1], out buffer1[2]); + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[32], + cospi[32], + buffer0[5], + buffer0[6], + out buffer1[5], + out buffer1[6], + cosBit, + in rounding); + + // Stage 3 completes the even half directly in coefficient order and prepares the four remaining odd terms. + Av1ForwardTransformArithmetic>.Butterfly( + cospi[32], + cospi[32], + buffer1[0], + buffer1[1], + out Vector256 output0, + out Vector256 output4, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + cospi[16], + cospi[48], + buffer1[3], + buffer1[2], + out Vector256 output2, + out Vector256 output6, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[4], buffer1[5], out buffer0[4], out buffer0[5]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[7], buffer1[6], out buffer0[7], out buffer0[6]); + + // Highway fuses the final two stages because no intermediate value is reused after either rotation. + Av1ForwardTransformArithmetic>.Butterfly( + cospi[8], + cospi[56], + buffer0[7], + buffer0[4], + out Vector256 output1, + out Vector256 output7, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + cospi[40], + cospi[24], + buffer0[6], + buffer0[5], + out Vector256 output5, + out Vector256 output3, + cosBit, + in rounding); + + Store(ref values, outputStride, 0, output0); + Store(ref values, outputStride, 1, output1); + Store(ref values, outputStride, 2, output2); + Store(ref values, outputStride, 3, output3); + Store(ref values, outputStride, 4, output4); + Store(ref values, outputStride, 5, output5); + Store(ref values, outputStride, 6, output6); + Store(ref values, outputStride, 7, output7); + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + Av1TransformRounding rounding = Av1ForwardTransformArithmetic>.CreateRounding(cosBit); + + // Stages 1 and 2 split the even and odd terms. The asymmetric destinations mirror Highway's buffer + // ownership, allowing the later even butterflies to write their final coefficients directly to the block. + Av1ForwardTransformArithmetic>.AddSubtract( + Load>(ref values, inputStride, 0), + Load>(ref values, inputStride, 7), + out buffer0[0], + out buffer1[7]); + + Av1ForwardTransformArithmetic>.AddSubtract( + Load>(ref values, inputStride, 1), + Load>(ref values, inputStride, 6), + out buffer0[1], + out buffer0[6]); + + Av1ForwardTransformArithmetic>.AddSubtract( + Load>(ref values, inputStride, 2), + Load>(ref values, inputStride, 5), + out buffer0[2], + out buffer0[5]); + + Av1ForwardTransformArithmetic>.AddSubtract( + Load>(ref values, inputStride, 3), + Load>(ref values, inputStride, 4), + out buffer0[3], + out buffer1[4]); + + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[0], buffer0[3], out buffer1[0], out buffer1[3]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[1], buffer0[2], out buffer1[1], out buffer1[2]); + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[32], + cospi[32], + buffer0[5], + buffer0[6], + out buffer1[5], + out buffer1[6], + cosBit, + in rounding); + + // Stage 3 completes the even half directly in coefficient order and prepares the four remaining odd terms. + Av1ForwardTransformArithmetic>.Butterfly( + cospi[32], + cospi[32], + buffer1[0], + buffer1[1], + out Vector512 output0, + out Vector512 output4, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + cospi[16], + cospi[48], + buffer1[3], + buffer1[2], + out Vector512 output2, + out Vector512 output6, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[4], buffer1[5], out buffer0[4], out buffer0[5]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[7], buffer1[6], out buffer0[7], out buffer0[6]); + + // Highway fuses the final two stages because no intermediate value is reused after either rotation. + Av1ForwardTransformArithmetic>.Butterfly( + cospi[8], + cospi[56], + buffer0[7], + buffer0[4], + out Vector512 output1, + out Vector512 output7, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + cospi[40], + cospi[24], + buffer0[6], + buffer0[5], + out Vector512 output5, + out Vector512 output3, + cosBit, + in rounding); + + Store(ref values, outputStride, 0, output0); + Store(ref values, outputStride, 1, output1); + Store(ref values, outputStride, 2, output2); + Store(ref values, outputStride, 3, output3); + Store(ref values, outputStride, 4, output4); + Store(ref values, outputStride, 5, output5); + Store(ref values, outputStride, 6, output6); + Store(ref values, outputStride, 7, output7); + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + Av1TransformRounding rounding = Av1ForwardTransformArithmetic>.CreateRounding(cosBit); + + // Stages 1 and 2 split the even and odd terms. The asymmetric destinations mirror Highway's buffer + // ownership, allowing the later even butterflies to write their final coefficients directly to the block. + Av1ForwardTransformArithmetic>.AddSubtract( + Load>(ref values, inputStride, 0), + Load>(ref values, inputStride, 7), + out buffer0[0], + out buffer1[7]); + + Av1ForwardTransformArithmetic>.AddSubtract( + Load>(ref values, inputStride, 1), + Load>(ref values, inputStride, 6), + out buffer0[1], + out buffer0[6]); + + Av1ForwardTransformArithmetic>.AddSubtract( + Load>(ref values, inputStride, 2), + Load>(ref values, inputStride, 5), + out buffer0[2], + out buffer0[5]); + + Av1ForwardTransformArithmetic>.AddSubtract( + Load>(ref values, inputStride, 3), + Load>(ref values, inputStride, 4), + out buffer0[3], + out buffer1[4]); + + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[0], buffer0[3], out buffer1[0], out buffer1[3]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[1], buffer0[2], out buffer1[1], out buffer1[2]); + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[32], + cospi[32], + buffer0[5], + buffer0[6], + out buffer1[5], + out buffer1[6], + cosBit, + in rounding); + + // Stage 3 completes the even half directly in coefficient order and prepares the four remaining odd terms. + Av1ForwardTransformArithmetic>.Butterfly( + cospi[32], + cospi[32], + buffer1[0], + buffer1[1], + out Vector128 output0, + out Vector128 output4, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + cospi[16], + cospi[48], + buffer1[3], + buffer1[2], + out Vector128 output2, + out Vector128 output6, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[4], buffer1[5], out buffer0[4], out buffer0[5]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[7], buffer1[6], out buffer0[7], out buffer0[6]); + + // Highway fuses the final two stages because no intermediate value is reused after either rotation. + Av1ForwardTransformArithmetic>.Butterfly( + cospi[8], + cospi[56], + buffer0[7], + buffer0[4], + out Vector128 output1, + out Vector128 output7, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + cospi[40], + cospi[24], + buffer0[6], + buffer0[5], + out Vector128 output5, + out Vector128 output3, + cosBit, + in rounding); + + Store(ref values, outputStride, 0, output0); + Store(ref values, outputStride, 1, output1); + Store(ref values, outputStride, 2, output2); + Store(ref values, outputStride, 3, output3); + Store(ref values, outputStride, 4, output4); + Store(ref values, outputStride, 5, output5); + Store(ref values, outputStride, 6, output6); + Store(ref values, outputStride, 7, output7); + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + Av1TransformRounding rounding = Av1ForwardTransformArithmetic>.CreateRounding(cosBit); + + // Stages 1 and 2 split the even and odd terms. The asymmetric destinations mirror Highway's buffer + // ownership, allowing the later even butterflies to write their final coefficients directly to the block. + Av1ForwardTransformArithmetic>.AddSubtract( + Load>(ref values, inputStride, 0), + Load>(ref values, inputStride, 7), + out buffer0[0], + out buffer1[7]); + + Av1ForwardTransformArithmetic>.AddSubtract( + Load>(ref values, inputStride, 1), + Load>(ref values, inputStride, 6), + out buffer0[1], + out buffer0[6]); + + Av1ForwardTransformArithmetic>.AddSubtract( + Load>(ref values, inputStride, 2), + Load>(ref values, inputStride, 5), + out buffer0[2], + out buffer0[5]); + + Av1ForwardTransformArithmetic>.AddSubtract( + Load>(ref values, inputStride, 3), + Load>(ref values, inputStride, 4), + out buffer0[3], + out buffer1[4]); + + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[0], buffer0[3], out buffer1[0], out buffer1[3]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[1], buffer0[2], out buffer1[1], out buffer1[2]); + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[32], + cospi[32], + buffer0[5], + buffer0[6], + out buffer1[5], + out buffer1[6], + cosBit, + in rounding); + + // Stage 3 completes the even half directly in coefficient order and prepares the four remaining odd terms. + Av1ForwardTransformArithmetic>.Butterfly( + cospi[32], + cospi[32], + buffer1[0], + buffer1[1], + out Vector256 output0, + out Vector256 output4, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + cospi[16], + cospi[48], + buffer1[3], + buffer1[2], + out Vector256 output2, + out Vector256 output6, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[4], buffer1[5], out buffer0[4], out buffer0[5]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[7], buffer1[6], out buffer0[7], out buffer0[6]); + + // Highway fuses the final two stages because no intermediate value is reused after either rotation. + Av1ForwardTransformArithmetic>.Butterfly( + cospi[8], + cospi[56], + buffer0[7], + buffer0[4], + out Vector256 output1, + out Vector256 output7, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + cospi[40], + cospi[24], + buffer0[6], + buffer0[5], + out Vector256 output5, + out Vector256 output3, + cosBit, + in rounding); + + Store(ref values, outputStride, 0, output0); + Store(ref values, outputStride, 1, output1); + Store(ref values, outputStride, 2, output2); + Store(ref values, outputStride, 3, output3); + Store(ref values, outputStride, 4, output4); + Store(ref values, outputStride, 5, output5); + Store(ref values, outputStride, 6, output6); + Store(ref values, outputStride, 7, output7); + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + Av1TransformRounding rounding = Av1ForwardTransformArithmetic>.CreateRounding(cosBit); + + // Stages 1 and 2 split the even and odd terms. The asymmetric destinations mirror Highway's buffer + // ownership, allowing the later even butterflies to write their final coefficients directly to the block. + Av1ForwardTransformArithmetic>.AddSubtract( + Load>(ref values, inputStride, 0), + Load>(ref values, inputStride, 7), + out buffer0[0], + out buffer1[7]); + + Av1ForwardTransformArithmetic>.AddSubtract( + Load>(ref values, inputStride, 1), + Load>(ref values, inputStride, 6), + out buffer0[1], + out buffer0[6]); + + Av1ForwardTransformArithmetic>.AddSubtract( + Load>(ref values, inputStride, 2), + Load>(ref values, inputStride, 5), + out buffer0[2], + out buffer0[5]); + + Av1ForwardTransformArithmetic>.AddSubtract( + Load>(ref values, inputStride, 3), + Load>(ref values, inputStride, 4), + out buffer0[3], + out buffer1[4]); + + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[0], buffer0[3], out buffer1[0], out buffer1[3]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer0[1], buffer0[2], out buffer1[1], out buffer1[2]); + Av1ForwardTransformArithmetic>.Butterfly( + -cospi[32], + cospi[32], + buffer0[5], + buffer0[6], + out buffer1[5], + out buffer1[6], + cosBit, + in rounding); + + // Stage 3 completes the even half directly in coefficient order and prepares the four remaining odd terms. + Av1ForwardTransformArithmetic>.Butterfly( + cospi[32], + cospi[32], + buffer1[0], + buffer1[1], + out Vector512 output0, + out Vector512 output4, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + cospi[16], + cospi[48], + buffer1[3], + buffer1[2], + out Vector512 output2, + out Vector512 output6, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[4], buffer1[5], out buffer0[4], out buffer0[5]); + Av1ForwardTransformArithmetic>.AddSubtract(buffer1[7], buffer1[6], out buffer0[7], out buffer0[6]); + + // Highway fuses the final two stages because no intermediate value is reused after either rotation. + Av1ForwardTransformArithmetic>.Butterfly( + cospi[8], + cospi[56], + buffer0[7], + buffer0[4], + out Vector512 output1, + out Vector512 output7, + cosBit, + in rounding); + + Av1ForwardTransformArithmetic>.Butterfly( + cospi[40], + cospi[24], + buffer0[6], + buffer0[5], + out Vector512 output5, + out Vector512 output3, + cosBit, + in rounding); + + Store(ref values, outputStride, 0, output0); + Store(ref values, outputStride, 1, output1); + Store(ref values, outputStride, 2, output2); + Store(ref values, outputStride, 3, output3); + Store(ref values, outputStride, 4, output4); + Store(ref values, outputStride, 5, output5); + Store(ref values, outputStride, 6, output6); + Store(ref values, outputStride, 7, output7); + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Transform/Forward/Av1ForwardTransformer.Identity16Operator.cs b/src/ImageSharp/Formats/Heif/Av1/Transform/Forward/Av1ForwardTransformer.Identity16Operator.cs new file mode 100644 index 0000000000..51b0878f16 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Transform/Forward/Av1ForwardTransformer.Identity16Operator.cs @@ -0,0 +1,235 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.Intrinsics; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform.Forward; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +/// +/// Defines the sixteen-point forward identity operator. +/// +internal static partial class Av1ForwardTransformer +{ + /// + /// Implements the sixteen-point forward identity transform. + /// + internal readonly struct Identity16Operator : IAv1ForwardTransform1dOperator + { + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector buffer0, + ref Av1TransformVector buffer1, + int cosBit) + { + _ = buffer0; + _ = buffer1; + _ = cosBit; + + // The length-specific normalization is applied directly in the semantic operator so each scalar + // or SIMD overload retains the exact AV1 identity-transform arithmetic without a forwarding layer. + for (int i = 0; i < 16; i++) + { + int input = Load(ref values, inputStride, i); + int output = Av1ForwardTransformArithmetic.MultiplyRound( + input, + 2 * Av1Transform1dMath.NewSqrt2, + Av1Transform1dMath.NewSqrt2Bits); + + Store(ref values, outputStride, i, output); + } + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector buffer0, + ref Av1TransformVector buffer1, + int cosBit) + { + _ = buffer0; + _ = buffer1; + _ = cosBit; + + // The length-specific normalization is applied directly in the semantic operator so each scalar + // or SIMD overload retains the exact AV1 identity-transform arithmetic without a forwarding layer. + for (int i = 0; i < 16; i++) + { + short input = Load(ref values, inputStride, i); + short output = Av1ForwardTransformArithmetic.MultiplyRound( + input, + 2 * Av1Transform1dMath.NewSqrt2, + Av1Transform1dMath.NewSqrt2Bits); + + Store(ref values, outputStride, i, output); + } + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + _ = buffer0; + _ = buffer1; + _ = cosBit; + + // The length-specific normalization is applied directly in the semantic operator so each scalar + // or SIMD overload retains the exact AV1 identity-transform arithmetic without a forwarding layer. + for (int i = 0; i < 16; i++) + { + Vector128 input = Load>(ref values, inputStride, i); + Vector128 output = Av1ForwardTransformArithmetic>.MultiplyRound( + input, + 2 * Av1Transform1dMath.NewSqrt2, + Av1Transform1dMath.NewSqrt2Bits); + + Store(ref values, outputStride, i, output); + } + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + _ = buffer0; + _ = buffer1; + _ = cosBit; + + // The length-specific normalization is applied directly in the semantic operator so each scalar + // or SIMD overload retains the exact AV1 identity-transform arithmetic without a forwarding layer. + for (int i = 0; i < 16; i++) + { + Vector256 input = Load>(ref values, inputStride, i); + Vector256 output = Av1ForwardTransformArithmetic>.MultiplyRound( + input, + 2 * Av1Transform1dMath.NewSqrt2, + Av1Transform1dMath.NewSqrt2Bits); + + Store(ref values, outputStride, i, output); + } + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + _ = buffer0; + _ = buffer1; + _ = cosBit; + + // The length-specific normalization is applied directly in the semantic operator so each scalar + // or SIMD overload retains the exact AV1 identity-transform arithmetic without a forwarding layer. + for (int i = 0; i < 16; i++) + { + Vector512 input = Load>(ref values, inputStride, i); + Vector512 output = Av1ForwardTransformArithmetic>.MultiplyRound( + input, + 2 * Av1Transform1dMath.NewSqrt2, + Av1Transform1dMath.NewSqrt2Bits); + + Store(ref values, outputStride, i, output); + } + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + _ = buffer0; + _ = buffer1; + _ = cosBit; + + // The length-specific normalization is applied directly in the semantic operator so each scalar + // or SIMD overload retains the exact AV1 identity-transform arithmetic without a forwarding layer. + for (int i = 0; i < 16; i++) + { + Vector128 input = Load>(ref values, inputStride, i); + Vector128 output = Av1ForwardTransformArithmetic>.MultiplyRound( + input, + 2 * Av1Transform1dMath.NewSqrt2, + Av1Transform1dMath.NewSqrt2Bits); + + Store(ref values, outputStride, i, output); + } + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + _ = buffer0; + _ = buffer1; + _ = cosBit; + + // The length-specific normalization is applied directly in the semantic operator so each scalar + // or SIMD overload retains the exact AV1 identity-transform arithmetic without a forwarding layer. + for (int i = 0; i < 16; i++) + { + Vector256 input = Load>(ref values, inputStride, i); + Vector256 output = Av1ForwardTransformArithmetic>.MultiplyRound( + input, + 2 * Av1Transform1dMath.NewSqrt2, + Av1Transform1dMath.NewSqrt2Bits); + + Store(ref values, outputStride, i, output); + } + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + _ = buffer0; + _ = buffer1; + _ = cosBit; + + // The length-specific normalization is applied directly in the semantic operator so each scalar + // or SIMD overload retains the exact AV1 identity-transform arithmetic without a forwarding layer. + for (int i = 0; i < 16; i++) + { + Vector512 input = Load>(ref values, inputStride, i); + Vector512 output = Av1ForwardTransformArithmetic>.MultiplyRound( + input, + 2 * Av1Transform1dMath.NewSqrt2, + Av1Transform1dMath.NewSqrt2Bits); + + Store(ref values, outputStride, i, output); + } + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Transform/Forward/Av1ForwardTransformer.Identity32Operator.cs b/src/ImageSharp/Formats/Heif/Av1/Transform/Forward/Av1ForwardTransformer.Identity32Operator.cs new file mode 100644 index 0000000000..3d6a179f59 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Transform/Forward/Av1ForwardTransformer.Identity32Operator.cs @@ -0,0 +1,211 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.Intrinsics; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform.Forward; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +/// +/// Defines the thirty-two-point forward identity operator. +/// +internal static partial class Av1ForwardTransformer +{ + /// + /// Implements the thirty-two-point forward identity transform. + /// + internal readonly struct Identity32Operator : IAv1ForwardTransform1dOperator + { + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector buffer0, + ref Av1TransformVector buffer1, + int cosBit) + { + _ = buffer0; + _ = buffer1; + _ = cosBit; + + // The length-specific normalization is applied directly in the semantic operator so each scalar + // or SIMD overload retains the exact AV1 identity-transform arithmetic without a forwarding layer. + for (int i = 0; i < 32; i++) + { + int input = Load(ref values, inputStride, i); + int output = Av1ForwardTransformArithmetic.ShiftLeft(input, 2); + + Store(ref values, outputStride, i, output); + } + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector buffer0, + ref Av1TransformVector buffer1, + int cosBit) + { + _ = buffer0; + _ = buffer1; + _ = cosBit; + + // The length-specific normalization is applied directly in the semantic operator so each scalar + // or SIMD overload retains the exact AV1 identity-transform arithmetic without a forwarding layer. + for (int i = 0; i < 32; i++) + { + short input = Load(ref values, inputStride, i); + short output = Av1ForwardTransformArithmetic.ShiftLeft(input, 2); + + Store(ref values, outputStride, i, output); + } + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + _ = buffer0; + _ = buffer1; + _ = cosBit; + + // The length-specific normalization is applied directly in the semantic operator so each scalar + // or SIMD overload retains the exact AV1 identity-transform arithmetic without a forwarding layer. + for (int i = 0; i < 32; i++) + { + Vector128 input = Load>(ref values, inputStride, i); + Vector128 output = Av1ForwardTransformArithmetic>.ShiftLeft(input, 2); + + Store(ref values, outputStride, i, output); + } + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + _ = buffer0; + _ = buffer1; + _ = cosBit; + + // The length-specific normalization is applied directly in the semantic operator so each scalar + // or SIMD overload retains the exact AV1 identity-transform arithmetic without a forwarding layer. + for (int i = 0; i < 32; i++) + { + Vector256 input = Load>(ref values, inputStride, i); + Vector256 output = Av1ForwardTransformArithmetic>.ShiftLeft(input, 2); + + Store(ref values, outputStride, i, output); + } + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + _ = buffer0; + _ = buffer1; + _ = cosBit; + + // The length-specific normalization is applied directly in the semantic operator so each scalar + // or SIMD overload retains the exact AV1 identity-transform arithmetic without a forwarding layer. + for (int i = 0; i < 32; i++) + { + Vector512 input = Load>(ref values, inputStride, i); + Vector512 output = Av1ForwardTransformArithmetic>.ShiftLeft(input, 2); + + Store(ref values, outputStride, i, output); + } + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + _ = buffer0; + _ = buffer1; + _ = cosBit; + + // The length-specific normalization is applied directly in the semantic operator so each scalar + // or SIMD overload retains the exact AV1 identity-transform arithmetic without a forwarding layer. + for (int i = 0; i < 32; i++) + { + Vector128 input = Load>(ref values, inputStride, i); + Vector128 output = Av1ForwardTransformArithmetic>.ShiftLeft(input, 2); + + Store(ref values, outputStride, i, output); + } + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + _ = buffer0; + _ = buffer1; + _ = cosBit; + + // The length-specific normalization is applied directly in the semantic operator so each scalar + // or SIMD overload retains the exact AV1 identity-transform arithmetic without a forwarding layer. + for (int i = 0; i < 32; i++) + { + Vector256 input = Load>(ref values, inputStride, i); + Vector256 output = Av1ForwardTransformArithmetic>.ShiftLeft(input, 2); + + Store(ref values, outputStride, i, output); + } + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + _ = buffer0; + _ = buffer1; + _ = cosBit; + + // The length-specific normalization is applied directly in the semantic operator so each scalar + // or SIMD overload retains the exact AV1 identity-transform arithmetic without a forwarding layer. + for (int i = 0; i < 32; i++) + { + Vector512 input = Load>(ref values, inputStride, i); + Vector512 output = Av1ForwardTransformArithmetic>.ShiftLeft(input, 2); + + Store(ref values, outputStride, i, output); + } + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Transform/Forward/Av1ForwardTransformer.Identity4Operator.cs b/src/ImageSharp/Formats/Heif/Av1/Transform/Forward/Av1ForwardTransformer.Identity4Operator.cs new file mode 100644 index 0000000000..cb2e20676c --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Transform/Forward/Av1ForwardTransformer.Identity4Operator.cs @@ -0,0 +1,235 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.Intrinsics; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform.Forward; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +/// +/// Defines the four-point forward identity operator. +/// +internal static partial class Av1ForwardTransformer +{ + /// + /// Implements the four-point forward identity transform. + /// + internal readonly struct Identity4Operator : IAv1ForwardTransform1dOperator + { + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector buffer0, + ref Av1TransformVector buffer1, + int cosBit) + { + _ = buffer0; + _ = buffer1; + _ = cosBit; + + // The length-specific normalization is applied directly in the semantic operator so each scalar + // or SIMD overload retains the exact AV1 identity-transform arithmetic without a forwarding layer. + for (int i = 0; i < 4; i++) + { + int input = Load(ref values, inputStride, i); + int output = Av1ForwardTransformArithmetic.MultiplyRound( + input, + 1 * Av1Transform1dMath.NewSqrt2, + Av1Transform1dMath.NewSqrt2Bits); + + Store(ref values, outputStride, i, output); + } + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector buffer0, + ref Av1TransformVector buffer1, + int cosBit) + { + _ = buffer0; + _ = buffer1; + _ = cosBit; + + // The length-specific normalization is applied directly in the semantic operator so each scalar + // or SIMD overload retains the exact AV1 identity-transform arithmetic without a forwarding layer. + for (int i = 0; i < 4; i++) + { + short input = Load(ref values, inputStride, i); + short output = Av1ForwardTransformArithmetic.MultiplyRound( + input, + 1 * Av1Transform1dMath.NewSqrt2, + Av1Transform1dMath.NewSqrt2Bits); + + Store(ref values, outputStride, i, output); + } + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + _ = buffer0; + _ = buffer1; + _ = cosBit; + + // The length-specific normalization is applied directly in the semantic operator so each scalar + // or SIMD overload retains the exact AV1 identity-transform arithmetic without a forwarding layer. + for (int i = 0; i < 4; i++) + { + Vector128 input = Load>(ref values, inputStride, i); + Vector128 output = Av1ForwardTransformArithmetic>.MultiplyRound( + input, + 1 * Av1Transform1dMath.NewSqrt2, + Av1Transform1dMath.NewSqrt2Bits); + + Store(ref values, outputStride, i, output); + } + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + _ = buffer0; + _ = buffer1; + _ = cosBit; + + // The length-specific normalization is applied directly in the semantic operator so each scalar + // or SIMD overload retains the exact AV1 identity-transform arithmetic without a forwarding layer. + for (int i = 0; i < 4; i++) + { + Vector256 input = Load>(ref values, inputStride, i); + Vector256 output = Av1ForwardTransformArithmetic>.MultiplyRound( + input, + 1 * Av1Transform1dMath.NewSqrt2, + Av1Transform1dMath.NewSqrt2Bits); + + Store(ref values, outputStride, i, output); + } + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + _ = buffer0; + _ = buffer1; + _ = cosBit; + + // The length-specific normalization is applied directly in the semantic operator so each scalar + // or SIMD overload retains the exact AV1 identity-transform arithmetic without a forwarding layer. + for (int i = 0; i < 4; i++) + { + Vector512 input = Load>(ref values, inputStride, i); + Vector512 output = Av1ForwardTransformArithmetic>.MultiplyRound( + input, + 1 * Av1Transform1dMath.NewSqrt2, + Av1Transform1dMath.NewSqrt2Bits); + + Store(ref values, outputStride, i, output); + } + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + _ = buffer0; + _ = buffer1; + _ = cosBit; + + // The length-specific normalization is applied directly in the semantic operator so each scalar + // or SIMD overload retains the exact AV1 identity-transform arithmetic without a forwarding layer. + for (int i = 0; i < 4; i++) + { + Vector128 input = Load>(ref values, inputStride, i); + Vector128 output = Av1ForwardTransformArithmetic>.MultiplyRound( + input, + 1 * Av1Transform1dMath.NewSqrt2, + Av1Transform1dMath.NewSqrt2Bits); + + Store(ref values, outputStride, i, output); + } + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + _ = buffer0; + _ = buffer1; + _ = cosBit; + + // The length-specific normalization is applied directly in the semantic operator so each scalar + // or SIMD overload retains the exact AV1 identity-transform arithmetic without a forwarding layer. + for (int i = 0; i < 4; i++) + { + Vector256 input = Load>(ref values, inputStride, i); + Vector256 output = Av1ForwardTransformArithmetic>.MultiplyRound( + input, + 1 * Av1Transform1dMath.NewSqrt2, + Av1Transform1dMath.NewSqrt2Bits); + + Store(ref values, outputStride, i, output); + } + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + _ = buffer0; + _ = buffer1; + _ = cosBit; + + // The length-specific normalization is applied directly in the semantic operator so each scalar + // or SIMD overload retains the exact AV1 identity-transform arithmetic without a forwarding layer. + for (int i = 0; i < 4; i++) + { + Vector512 input = Load>(ref values, inputStride, i); + Vector512 output = Av1ForwardTransformArithmetic>.MultiplyRound( + input, + 1 * Av1Transform1dMath.NewSqrt2, + Av1Transform1dMath.NewSqrt2Bits); + + Store(ref values, outputStride, i, output); + } + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Transform/Forward/Av1ForwardTransformer.Identity8Operator.cs b/src/ImageSharp/Formats/Heif/Av1/Transform/Forward/Av1ForwardTransformer.Identity8Operator.cs new file mode 100644 index 0000000000..ec86687103 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Transform/Forward/Av1ForwardTransformer.Identity8Operator.cs @@ -0,0 +1,211 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.Intrinsics; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform.Forward; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +/// +/// Defines the eight-point forward identity operator. +/// +internal static partial class Av1ForwardTransformer +{ + /// + /// Implements the eight-point forward identity transform. + /// + internal readonly struct Identity8Operator : IAv1ForwardTransform1dOperator + { + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector buffer0, + ref Av1TransformVector buffer1, + int cosBit) + { + _ = buffer0; + _ = buffer1; + _ = cosBit; + + // The length-specific normalization is applied directly in the semantic operator so each scalar + // or SIMD overload retains the exact AV1 identity-transform arithmetic without a forwarding layer. + for (int i = 0; i < 8; i++) + { + int input = Load(ref values, inputStride, i); + int output = Av1ForwardTransformArithmetic.ShiftLeft(input, 1); + + Store(ref values, outputStride, i, output); + } + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector buffer0, + ref Av1TransformVector buffer1, + int cosBit) + { + _ = buffer0; + _ = buffer1; + _ = cosBit; + + // The length-specific normalization is applied directly in the semantic operator so each scalar + // or SIMD overload retains the exact AV1 identity-transform arithmetic without a forwarding layer. + for (int i = 0; i < 8; i++) + { + short input = Load(ref values, inputStride, i); + short output = Av1ForwardTransformArithmetic.ShiftLeft(input, 1); + + Store(ref values, outputStride, i, output); + } + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + _ = buffer0; + _ = buffer1; + _ = cosBit; + + // The length-specific normalization is applied directly in the semantic operator so each scalar + // or SIMD overload retains the exact AV1 identity-transform arithmetic without a forwarding layer. + for (int i = 0; i < 8; i++) + { + Vector128 input = Load>(ref values, inputStride, i); + Vector128 output = Av1ForwardTransformArithmetic>.ShiftLeft(input, 1); + + Store(ref values, outputStride, i, output); + } + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + _ = buffer0; + _ = buffer1; + _ = cosBit; + + // The length-specific normalization is applied directly in the semantic operator so each scalar + // or SIMD overload retains the exact AV1 identity-transform arithmetic without a forwarding layer. + for (int i = 0; i < 8; i++) + { + Vector256 input = Load>(ref values, inputStride, i); + Vector256 output = Av1ForwardTransformArithmetic>.ShiftLeft(input, 1); + + Store(ref values, outputStride, i, output); + } + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + _ = buffer0; + _ = buffer1; + _ = cosBit; + + // The length-specific normalization is applied directly in the semantic operator so each scalar + // or SIMD overload retains the exact AV1 identity-transform arithmetic without a forwarding layer. + for (int i = 0; i < 8; i++) + { + Vector512 input = Load>(ref values, inputStride, i); + Vector512 output = Av1ForwardTransformArithmetic>.ShiftLeft(input, 1); + + Store(ref values, outputStride, i, output); + } + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + _ = buffer0; + _ = buffer1; + _ = cosBit; + + // The length-specific normalization is applied directly in the semantic operator so each scalar + // or SIMD overload retains the exact AV1 identity-transform arithmetic without a forwarding layer. + for (int i = 0; i < 8; i++) + { + Vector128 input = Load>(ref values, inputStride, i); + Vector128 output = Av1ForwardTransformArithmetic>.ShiftLeft(input, 1); + + Store(ref values, outputStride, i, output); + } + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + _ = buffer0; + _ = buffer1; + _ = cosBit; + + // The length-specific normalization is applied directly in the semantic operator so each scalar + // or SIMD overload retains the exact AV1 identity-transform arithmetic without a forwarding layer. + for (int i = 0; i < 8; i++) + { + Vector256 input = Load>(ref values, inputStride, i); + Vector256 output = Av1ForwardTransformArithmetic>.ShiftLeft(input, 1); + + Store(ref values, outputStride, i, output); + } + } + + /// + public static void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit) + { + _ = buffer0; + _ = buffer1; + _ = cosBit; + + // The length-specific normalization is applied directly in the semantic operator so each scalar + // or SIMD overload retains the exact AV1 identity-transform arithmetic without a forwarding layer. + for (int i = 0; i < 8; i++) + { + Vector512 input = Load>(ref values, inputStride, i); + Vector512 output = Av1ForwardTransformArithmetic>.ShiftLeft(input, 1); + + Store(ref values, outputStride, i, output); + } + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Transform/Forward/Av1ForwardTransformer.Operations.cs b/src/ImageSharp/Formats/Heif/Av1/Transform/Forward/Av1ForwardTransformer.Operations.cs new file mode 100644 index 0000000000..fa19fd5319 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Transform/Forward/Av1ForwardTransformer.Operations.cs @@ -0,0 +1,116 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform.Forward; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +/// +/// Defines shared forward-transform storage and rotation primitives. +/// +internal static partial class Av1ForwardTransformer +{ + /// + /// Loads one scalar or SIMD transform value from strided block storage. + /// + /// The scalar or SIMD value to load. + /// The first value in the transform block. + /// The byte distance between consecutive transform positions. + /// The transform position to load. + /// The requested transform value. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static TValue Load(ref byte values, nint stride, int index) + where TValue : struct + => Unsafe.ReadUnaligned(ref Unsafe.AddByteOffset(ref values, index * stride)); + + /// + /// Stores one scalar or SIMD transform value in strided block storage. + /// + /// The scalar or SIMD value to store. + /// The first value in the transform block. + /// The byte distance between consecutive transform positions. + /// The transform position to store. + /// The transform value. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static void Store(ref byte values, nint stride, int index, TValue value) + where TValue : struct + => Unsafe.WriteUnaligned(ref Unsafe.AddByteOffset(ref values, index * stride), value); + + /// + /// Applies one paired rotation and stores both results directly in the strided transform block. + /// + /// The scalar or SIMD value containing the independent transform axes. + /// The first fixed-point rotation weight. + /// The second fixed-point rotation weight. + /// The first rotation input. + /// The second rotation input. + /// The first value in the transform block. + /// The byte distance between consecutive transform positions. + /// The transform position for the first result. + /// The transform position for the second result. + /// The fixed-point precision of the rotation weights. + /// The lane-width-specific rounding constants. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static void ButterflyStore( + int weight0, + int weight1, + TValue input0, + TValue input1, + ref byte values, + nint stride, + int outputIndex0, + int outputIndex1, + int cosBit, + in Av1TransformRounding rounding) + where TValue : struct + { + Av1ForwardTransformArithmetic.Butterfly( + weight0, + weight1, + input0, + input1, + out TValue output0, + out TValue output1, + cosBit, + in rounding); + + Store(ref values, stride, outputIndex0, output0); + Store(ref values, stride, outputIndex1, output1); + } + + /// + /// Applies one paired rotation into two positions of a transform-stage buffer. + /// + /// The scalar or SIMD value containing the independent transform axes. + /// The first fixed-point rotation weight. + /// The second fixed-point rotation weight. + /// The first rotation input. + /// The second rotation input. + /// The transform-stage buffer receiving both results. + /// The buffer position for the first result. + /// The buffer position for the second result. + /// The fixed-point precision of the rotation weights. + /// The lane-width-specific rounding constants. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static void Butterfly( + int weight0, + int weight1, + TValue input0, + TValue input1, + ref Av1TransformVector output, + int outputIndex0, + int outputIndex1, + int cosBit, + in Av1TransformRounding rounding) + where TValue : struct + => Av1ForwardTransformArithmetic.Butterfly( + weight0, + weight1, + input0, + input1, + out output[outputIndex0], + out output[outputIndex1], + cosBit, + in rounding); +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Transform/Forward/Av1ForwardTransformer.Operator.cs b/src/ImageSharp/Formats/Heif/Av1/Transform/Forward/Av1ForwardTransformer.Operator.cs new file mode 100644 index 0000000000..fc4cace530 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Transform/Forward/Av1ForwardTransformer.Operator.cs @@ -0,0 +1,158 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +/// +/// Defines the operator contract for one-dimensional AV1 forward transforms. +/// +internal static partial class Av1ForwardTransformer +{ + /// + /// Defines the scalar and SIMD arithmetic for one AV1 forward transform. + /// + /// + /// Every overload applies the same stage network to independent transform axes. The family traversal selects one + /// concrete lane width, while the closed semantic operator lets the JIT resolve the static call before the stages. + /// + internal interface IAv1ForwardTransform1dOperator + { + /// + /// Transforms one expanded axis without hardware vectorization. + /// + /// The first value in the strided transform storage. + /// The byte distance between consecutive input positions. + /// The byte distance between consecutive output positions. + /// The first transform-stage workspace buffer. + /// The second transform-stage workspace buffer. + /// The fixed-point precision of the transform constants. + public static abstract void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector buffer0, + ref Av1TransformVector buffer1, + int cosBit); + + /// + /// Transforms one packed axis without hardware vectorization. + /// + /// The first value in the strided transform storage. + /// The byte distance between consecutive input positions. + /// The byte distance between consecutive output positions. + /// The first transform-stage workspace buffer. + /// The second transform-stage workspace buffer. + /// The fixed-point precision of the transform constants. + public static abstract void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector buffer0, + ref Av1TransformVector buffer1, + int cosBit); + + /// + /// Transforms eight packed axes in parallel. + /// + /// The first value in the strided transform storage. + /// The byte distance between consecutive input positions. + /// The byte distance between consecutive output positions. + /// The first transform-stage workspace buffer. + /// The second transform-stage workspace buffer. + /// The fixed-point precision of the transform constants. + public static abstract void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit); + + /// + /// Transforms sixteen packed axes in parallel. + /// + /// The first value in the strided transform storage. + /// The byte distance between consecutive input positions. + /// The byte distance between consecutive output positions. + /// The first transform-stage workspace buffer. + /// The second transform-stage workspace buffer. + /// The fixed-point precision of the transform constants. + public static abstract void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit); + + /// + /// Transforms thirty-two packed axes in parallel. + /// + /// The first value in the strided transform storage. + /// The byte distance between consecutive input positions. + /// The byte distance between consecutive output positions. + /// The first transform-stage workspace buffer. + /// The second transform-stage workspace buffer. + /// The fixed-point precision of the transform constants. + public static abstract void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit); + + /// + /// Transforms four expanded axes in parallel. + /// + /// The first value in the strided transform storage. + /// The byte distance between consecutive input positions. + /// The byte distance between consecutive output positions. + /// The first transform-stage workspace buffer. + /// The second transform-stage workspace buffer. + /// The fixed-point precision of the transform constants. + public static abstract void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit); + + /// + /// Transforms eight expanded axes in parallel. + /// + /// The first value in the strided transform storage. + /// The byte distance between consecutive input positions. + /// The byte distance between consecutive output positions. + /// The first transform-stage workspace buffer. + /// The second transform-stage workspace buffer. + /// The fixed-point precision of the transform constants. + public static abstract void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit); + + /// + /// Transforms sixteen expanded axes in parallel. + /// + /// The first value in the strided transform storage. + /// The byte distance between consecutive input positions. + /// The byte distance between consecutive output positions. + /// The first transform-stage workspace buffer. + /// The second transform-stage workspace buffer. + /// The fixed-point precision of the transform constants. + public static abstract void Transform( + ref byte values, + nint inputStride, + nint outputStride, + ref Av1TransformVector> buffer0, + ref Av1TransformVector> buffer1, + int cosBit); + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Transform/Forward/Av1TransformRounding.cs b/src/ImageSharp/Formats/Heif/Av1/Transform/Forward/Av1TransformRounding.cs new file mode 100644 index 0000000000..7893511c8c --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Transform/Forward/Av1TransformRounding.cs @@ -0,0 +1,42 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.InteropServices; +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Transform.Forward; + +/// +/// Stores the fixed-point rounding value in the lane shape selected for one forward transform. +/// +/// +/// The fields overlap because a closed transform instantiation reads exactly one representation. This keeps the +/// rounding broadcast outside the butterfly sequence without increasing the caller-owned transform workspace. +/// +[StructLayout(LayoutKind.Explicit)] +internal struct Av1TransformRounding +{ + /// + /// The scalar rounding value. + /// + [FieldOffset(0)] + public int Scalar; + + /// + /// The four-lane rounding value used by 128-bit widened arithmetic. + /// + [FieldOffset(0)] + public Vector128 Vector128; + + /// + /// The eight-lane rounding value used by 256-bit widened arithmetic. + /// + [FieldOffset(0)] + public Vector256 Vector256; + + /// + /// The sixteen-lane rounding value used by 512-bit widened arithmetic. + /// + [FieldOffset(0)] + public Vector512 Vector512; +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Transform/Inverse/Av1Inverse2dTransformer.Adst16Operator.cs b/src/ImageSharp/Formats/Heif/Av1/Transform/Inverse/Av1Inverse2dTransformer.Adst16Operator.cs new file mode 100644 index 0000000000..4c0a80f97a --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Transform/Inverse/Av1Inverse2dTransformer.Adst16Operator.cs @@ -0,0 +1,571 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +/// +/// Defines the 16-point AV1 inverse asymmetric discrete sine transform operator. +/// +/// +/// Vector fields represent transform positions and vector lanes represent independent axes. The SIMD overloads apply +/// the same staged rotations, fixed-point rounding, and range clamps as the scalar overload without mixing axes. +/// +internal static partial class Av1Inverse2dTransformer +{ + internal readonly struct Adst16Operator : IAv1Transform1dOperator + { + /// + /// Applies the normative 16-point AV1 inverse asymmetric discrete sine transform. + /// + /// The sixteen frequency-domain coefficients. + /// The sixteen spatial-domain residual values. + /// The sixteen-element stage buffer owned by the containing two-dimensional transform. + /// The fixed-point precision of the cosine constants. + /// The signed-bit range assigned to each transform stage. + public static void Transform(ReadOnlySpan input, Span output, Span step, int cosBit, InlineArray12 stageRange) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + int stage = 0; + + // Stage 1 permutes the coefficients into the signed order used by the ADST factorization. + stage++; + output[0] = input[15]; + output[1] = input[0]; + output[2] = input[13]; + output[3] = input[2]; + output[4] = input[11]; + output[5] = input[4]; + output[6] = input[9]; + output[7] = input[6]; + output[8] = input[7]; + output[9] = input[8]; + output[10] = input[5]; + output[11] = input[10]; + output[12] = input[3]; + output[13] = input[12]; + output[14] = input[1]; + output[15] = input[14]; + + // Stage 2 applies the terminal odd-angle rotations in reverse. + stage++; + step[0] = Av1Transform1dMath.HalfButterfly(cospi[2], output[0], cospi[62], output[1], cosBit); + step[1] = Av1Transform1dMath.HalfButterfly(cospi[62], output[0], -cospi[2], output[1], cosBit); + step[2] = Av1Transform1dMath.HalfButterfly(cospi[10], output[2], cospi[54], output[3], cosBit); + step[3] = Av1Transform1dMath.HalfButterfly(cospi[54], output[2], -cospi[10], output[3], cosBit); + step[4] = Av1Transform1dMath.HalfButterfly(cospi[18], output[4], cospi[46], output[5], cosBit); + step[5] = Av1Transform1dMath.HalfButterfly(cospi[46], output[4], -cospi[18], output[5], cosBit); + step[6] = Av1Transform1dMath.HalfButterfly(cospi[26], output[6], cospi[38], output[7], cosBit); + step[7] = Av1Transform1dMath.HalfButterfly(cospi[38], output[6], -cospi[26], output[7], cosBit); + step[8] = Av1Transform1dMath.HalfButterfly(cospi[34], output[8], cospi[30], output[9], cosBit); + step[9] = Av1Transform1dMath.HalfButterfly(cospi[30], output[8], -cospi[34], output[9], cosBit); + step[10] = Av1Transform1dMath.HalfButterfly(cospi[42], output[10], cospi[22], output[11], cosBit); + step[11] = Av1Transform1dMath.HalfButterfly(cospi[22], output[10], -cospi[42], output[11], cosBit); + step[12] = Av1Transform1dMath.HalfButterfly(cospi[50], output[12], cospi[14], output[13], cosBit); + step[13] = Av1Transform1dMath.HalfButterfly(cospi[14], output[12], -cospi[50], output[13], cosBit); + step[14] = Av1Transform1dMath.HalfButterfly(cospi[58], output[14], cospi[6], output[15], cosBit); + step[15] = Av1Transform1dMath.HalfButterfly(cospi[6], output[14], -cospi[58], output[15], cosBit); + + // Stage 3 separates the complete butterfly into two eight-sample halves and clamps each lane. + stage++; + output[0] = Av1Transform1dMath.Clamp(step[0] + step[8], stageRange[stage]); + output[1] = Av1Transform1dMath.Clamp(step[1] + step[9], stageRange[stage]); + output[2] = Av1Transform1dMath.Clamp(step[2] + step[10], stageRange[stage]); + output[3] = Av1Transform1dMath.Clamp(step[3] + step[11], stageRange[stage]); + output[4] = Av1Transform1dMath.Clamp(step[4] + step[12], stageRange[stage]); + output[5] = Av1Transform1dMath.Clamp(step[5] + step[13], stageRange[stage]); + output[6] = Av1Transform1dMath.Clamp(step[6] + step[14], stageRange[stage]); + output[7] = Av1Transform1dMath.Clamp(step[7] + step[15], stageRange[stage]); + output[8] = Av1Transform1dMath.Clamp(step[0] - step[8], stageRange[stage]); + output[9] = Av1Transform1dMath.Clamp(step[1] - step[9], stageRange[stage]); + output[10] = Av1Transform1dMath.Clamp(step[2] - step[10], stageRange[stage]); + output[11] = Av1Transform1dMath.Clamp(step[3] - step[11], stageRange[stage]); + output[12] = Av1Transform1dMath.Clamp(step[4] - step[12], stageRange[stage]); + output[13] = Av1Transform1dMath.Clamp(step[5] - step[13], stageRange[stage]); + output[14] = Av1Transform1dMath.Clamp(step[6] - step[14], stageRange[stage]); + output[15] = Av1Transform1dMath.Clamp(step[7] - step[15], stageRange[stage]); + + // Stage 4 reverses the pi/16 rotations in the upper half. + stage++; + step[0] = output[0]; + step[1] = output[1]; + step[2] = output[2]; + step[3] = output[3]; + step[4] = output[4]; + step[5] = output[5]; + step[6] = output[6]; + step[7] = output[7]; + step[8] = Av1Transform1dMath.HalfButterfly(cospi[8], output[8], cospi[56], output[9], cosBit); + step[9] = Av1Transform1dMath.HalfButterfly(cospi[56], output[8], -cospi[8], output[9], cosBit); + step[10] = Av1Transform1dMath.HalfButterfly(cospi[40], output[10], cospi[24], output[11], cosBit); + step[11] = Av1Transform1dMath.HalfButterfly(cospi[24], output[10], -cospi[40], output[11], cosBit); + step[12] = Av1Transform1dMath.HalfButterfly(-cospi[56], output[12], cospi[8], output[13], cosBit); + step[13] = Av1Transform1dMath.HalfButterfly(cospi[8], output[12], cospi[56], output[13], cosBit); + step[14] = Av1Transform1dMath.HalfButterfly(-cospi[24], output[14], cospi[40], output[15], cosBit); + step[15] = Av1Transform1dMath.HalfButterfly(cospi[40], output[14], cospi[24], output[15], cosBit); + + // Stage 5 separates each eight-sample half into four-sample groups and clamps each lane. + stage++; + output[0] = Av1Transform1dMath.Clamp(step[0] + step[4], stageRange[stage]); + output[1] = Av1Transform1dMath.Clamp(step[1] + step[5], stageRange[stage]); + output[2] = Av1Transform1dMath.Clamp(step[2] + step[6], stageRange[stage]); + output[3] = Av1Transform1dMath.Clamp(step[3] + step[7], stageRange[stage]); + output[4] = Av1Transform1dMath.Clamp(step[0] - step[4], stageRange[stage]); + output[5] = Av1Transform1dMath.Clamp(step[1] - step[5], stageRange[stage]); + output[6] = Av1Transform1dMath.Clamp(step[2] - step[6], stageRange[stage]); + output[7] = Av1Transform1dMath.Clamp(step[3] - step[7], stageRange[stage]); + output[8] = Av1Transform1dMath.Clamp(step[8] + step[12], stageRange[stage]); + output[9] = Av1Transform1dMath.Clamp(step[9] + step[13], stageRange[stage]); + output[10] = Av1Transform1dMath.Clamp(step[10] + step[14], stageRange[stage]); + output[11] = Av1Transform1dMath.Clamp(step[11] + step[15], stageRange[stage]); + output[12] = Av1Transform1dMath.Clamp(step[8] - step[12], stageRange[stage]); + output[13] = Av1Transform1dMath.Clamp(step[9] - step[13], stageRange[stage]); + output[14] = Av1Transform1dMath.Clamp(step[10] - step[14], stageRange[stage]); + output[15] = Av1Transform1dMath.Clamp(step[11] - step[15], stageRange[stage]); + + // Stage 6 reverses the pi/8 and 3pi/8 rotations. + stage++; + step[0] = output[0]; + step[1] = output[1]; + step[2] = output[2]; + step[3] = output[3]; + step[4] = Av1Transform1dMath.HalfButterfly(cospi[16], output[4], cospi[48], output[5], cosBit); + step[5] = Av1Transform1dMath.HalfButterfly(cospi[48], output[4], -cospi[16], output[5], cosBit); + step[6] = Av1Transform1dMath.HalfButterfly(-cospi[48], output[6], cospi[16], output[7], cosBit); + step[7] = Av1Transform1dMath.HalfButterfly(cospi[16], output[6], cospi[48], output[7], cosBit); + step[8] = output[8]; + step[9] = output[9]; + step[10] = output[10]; + step[11] = output[11]; + step[12] = Av1Transform1dMath.HalfButterfly(cospi[16], output[12], cospi[48], output[13], cosBit); + step[13] = Av1Transform1dMath.HalfButterfly(cospi[48], output[12], -cospi[16], output[13], cosBit); + step[14] = Av1Transform1dMath.HalfButterfly(-cospi[48], output[14], cospi[16], output[15], cosBit); + step[15] = Av1Transform1dMath.HalfButterfly(cospi[16], output[14], cospi[48], output[15], cosBit); + + // Stage 7 separates the four-sample groups into adjacent coefficient pairs and clamps each lane. + stage++; + output[0] = Av1Transform1dMath.Clamp(step[0] + step[2], stageRange[stage]); + output[1] = Av1Transform1dMath.Clamp(step[1] + step[3], stageRange[stage]); + output[2] = Av1Transform1dMath.Clamp(step[0] - step[2], stageRange[stage]); + output[3] = Av1Transform1dMath.Clamp(step[1] - step[3], stageRange[stage]); + output[4] = Av1Transform1dMath.Clamp(step[4] + step[6], stageRange[stage]); + output[5] = Av1Transform1dMath.Clamp(step[5] + step[7], stageRange[stage]); + output[6] = Av1Transform1dMath.Clamp(step[4] - step[6], stageRange[stage]); + output[7] = Av1Transform1dMath.Clamp(step[5] - step[7], stageRange[stage]); + output[8] = Av1Transform1dMath.Clamp(step[8] + step[10], stageRange[stage]); + output[9] = Av1Transform1dMath.Clamp(step[9] + step[11], stageRange[stage]); + output[10] = Av1Transform1dMath.Clamp(step[8] - step[10], stageRange[stage]); + output[11] = Av1Transform1dMath.Clamp(step[9] - step[11], stageRange[stage]); + output[12] = Av1Transform1dMath.Clamp(step[12] + step[14], stageRange[stage]); + output[13] = Av1Transform1dMath.Clamp(step[13] + step[15], stageRange[stage]); + output[14] = Av1Transform1dMath.Clamp(step[12] - step[14], stageRange[stage]); + output[15] = Av1Transform1dMath.Clamp(step[13] - step[15], stageRange[stage]); + + // Stage 8 reverses the pi/4 rotations for the middle pairs. + step[0] = output[0]; + step[1] = output[1]; + step[2] = Av1Transform1dMath.HalfButterfly(cospi[32], output[2], cospi[32], output[3], cosBit); + step[3] = Av1Transform1dMath.HalfButterfly(cospi[32], output[2], -cospi[32], output[3], cosBit); + step[4] = output[4]; + step[5] = output[5]; + step[6] = Av1Transform1dMath.HalfButterfly(cospi[32], output[6], cospi[32], output[7], cosBit); + step[7] = Av1Transform1dMath.HalfButterfly(cospi[32], output[6], -cospi[32], output[7], cosBit); + step[8] = output[8]; + step[9] = output[9]; + step[10] = Av1Transform1dMath.HalfButterfly(cospi[32], output[10], cospi[32], output[11], cosBit); + step[11] = Av1Transform1dMath.HalfButterfly(cospi[32], output[10], -cospi[32], output[11], cosBit); + step[12] = output[12]; + step[13] = output[13]; + step[14] = Av1Transform1dMath.HalfButterfly(cospi[32], output[14], cospi[32], output[15], cosBit); + step[15] = Av1Transform1dMath.HalfButterfly(cospi[32], output[14], -cospi[32], output[15], cosBit); + + // Stage 9 applies the AV1 signs and permutation that restore spatial sample order. + output[0] = step[0]; + output[1] = -step[8]; + output[2] = step[12]; + output[3] = -step[4]; + output[4] = step[6]; + output[5] = -step[14]; + output[6] = step[10]; + output[7] = -step[2]; + output[8] = step[3]; + output[9] = -step[11]; + output[10] = step[15]; + output[11] = -step[7]; + output[12] = step[5]; + output[13] = -step[13]; + output[14] = step[9]; + output[15] = -step[1]; + } + + /// + public static void Transform( + ref Av1TransformVector> input, + ref Av1TransformVector> output, + ref Av1TransformVector> step, + int cosBit, + InlineArray12 stageRange) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + int stage = 0; + + // Stage 1 permutes the coefficients into the signed order used by the ADST factorization. + stage++; + output.V0 = input.V15; + output.V1 = input.V0; + output.V2 = input.V13; + output.V3 = input.V2; + output.V4 = input.V11; + output.V5 = input.V4; + output.V6 = input.V9; + output.V7 = input.V6; + output.V8 = input.V7; + output.V9 = input.V8; + output.V10 = input.V5; + output.V11 = input.V10; + output.V12 = input.V3; + output.V13 = input.V12; + output.V14 = input.V1; + output.V15 = input.V14; + + // Stage 2 applies the terminal odd-angle rotations in reverse. + stage++; + step.V0 = Av1Transform1dMath.HalfButterfly(cospi[2], output.V0, cospi[62], output.V1, cosBit); + step.V1 = Av1Transform1dMath.HalfButterfly(cospi[62], output.V0, -cospi[2], output.V1, cosBit); + step.V2 = Av1Transform1dMath.HalfButterfly(cospi[10], output.V2, cospi[54], output.V3, cosBit); + step.V3 = Av1Transform1dMath.HalfButterfly(cospi[54], output.V2, -cospi[10], output.V3, cosBit); + step.V4 = Av1Transform1dMath.HalfButterfly(cospi[18], output.V4, cospi[46], output.V5, cosBit); + step.V5 = Av1Transform1dMath.HalfButterfly(cospi[46], output.V4, -cospi[18], output.V5, cosBit); + step.V6 = Av1Transform1dMath.HalfButterfly(cospi[26], output.V6, cospi[38], output.V7, cosBit); + step.V7 = Av1Transform1dMath.HalfButterfly(cospi[38], output.V6, -cospi[26], output.V7, cosBit); + step.V8 = Av1Transform1dMath.HalfButterfly(cospi[34], output.V8, cospi[30], output.V9, cosBit); + step.V9 = Av1Transform1dMath.HalfButterfly(cospi[30], output.V8, -cospi[34], output.V9, cosBit); + step.V10 = Av1Transform1dMath.HalfButterfly(cospi[42], output.V10, cospi[22], output.V11, cosBit); + step.V11 = Av1Transform1dMath.HalfButterfly(cospi[22], output.V10, -cospi[42], output.V11, cosBit); + step.V12 = Av1Transform1dMath.HalfButterfly(cospi[50], output.V12, cospi[14], output.V13, cosBit); + step.V13 = Av1Transform1dMath.HalfButterfly(cospi[14], output.V12, -cospi[50], output.V13, cosBit); + step.V14 = Av1Transform1dMath.HalfButterfly(cospi[58], output.V14, cospi[6], output.V15, cosBit); + step.V15 = Av1Transform1dMath.HalfButterfly(cospi[6], output.V14, -cospi[58], output.V15, cosBit); + + // Stage 3 separates the complete butterfly into two eight-sample halves and clamps each lane. + stage++; + output.V0 = Av1Transform1dMath.Clamp(step.V0 + step.V8, stageRange[stage]); + output.V1 = Av1Transform1dMath.Clamp(step.V1 + step.V9, stageRange[stage]); + output.V2 = Av1Transform1dMath.Clamp(step.V2 + step.V10, stageRange[stage]); + output.V3 = Av1Transform1dMath.Clamp(step.V3 + step.V11, stageRange[stage]); + output.V4 = Av1Transform1dMath.Clamp(step.V4 + step.V12, stageRange[stage]); + output.V5 = Av1Transform1dMath.Clamp(step.V5 + step.V13, stageRange[stage]); + output.V6 = Av1Transform1dMath.Clamp(step.V6 + step.V14, stageRange[stage]); + output.V7 = Av1Transform1dMath.Clamp(step.V7 + step.V15, stageRange[stage]); + output.V8 = Av1Transform1dMath.Clamp(step.V0 - step.V8, stageRange[stage]); + output.V9 = Av1Transform1dMath.Clamp(step.V1 - step.V9, stageRange[stage]); + output.V10 = Av1Transform1dMath.Clamp(step.V2 - step.V10, stageRange[stage]); + output.V11 = Av1Transform1dMath.Clamp(step.V3 - step.V11, stageRange[stage]); + output.V12 = Av1Transform1dMath.Clamp(step.V4 - step.V12, stageRange[stage]); + output.V13 = Av1Transform1dMath.Clamp(step.V5 - step.V13, stageRange[stage]); + output.V14 = Av1Transform1dMath.Clamp(step.V6 - step.V14, stageRange[stage]); + output.V15 = Av1Transform1dMath.Clamp(step.V7 - step.V15, stageRange[stage]); + + // Stage 4 reverses the pi/16 rotations in the upper half. + stage++; + step.V0 = output.V0; + step.V1 = output.V1; + step.V2 = output.V2; + step.V3 = output.V3; + step.V4 = output.V4; + step.V5 = output.V5; + step.V6 = output.V6; + step.V7 = output.V7; + step.V8 = Av1Transform1dMath.HalfButterfly(cospi[8], output.V8, cospi[56], output.V9, cosBit); + step.V9 = Av1Transform1dMath.HalfButterfly(cospi[56], output.V8, -cospi[8], output.V9, cosBit); + step.V10 = Av1Transform1dMath.HalfButterfly(cospi[40], output.V10, cospi[24], output.V11, cosBit); + step.V11 = Av1Transform1dMath.HalfButterfly(cospi[24], output.V10, -cospi[40], output.V11, cosBit); + step.V12 = Av1Transform1dMath.HalfButterfly(-cospi[56], output.V12, cospi[8], output.V13, cosBit); + step.V13 = Av1Transform1dMath.HalfButterfly(cospi[8], output.V12, cospi[56], output.V13, cosBit); + step.V14 = Av1Transform1dMath.HalfButterfly(-cospi[24], output.V14, cospi[40], output.V15, cosBit); + step.V15 = Av1Transform1dMath.HalfButterfly(cospi[40], output.V14, cospi[24], output.V15, cosBit); + + // Stage 5 separates each eight-sample half into four-sample groups and clamps each lane. + stage++; + output.V0 = Av1Transform1dMath.Clamp(step.V0 + step.V4, stageRange[stage]); + output.V1 = Av1Transform1dMath.Clamp(step.V1 + step.V5, stageRange[stage]); + output.V2 = Av1Transform1dMath.Clamp(step.V2 + step.V6, stageRange[stage]); + output.V3 = Av1Transform1dMath.Clamp(step.V3 + step.V7, stageRange[stage]); + output.V4 = Av1Transform1dMath.Clamp(step.V0 - step.V4, stageRange[stage]); + output.V5 = Av1Transform1dMath.Clamp(step.V1 - step.V5, stageRange[stage]); + output.V6 = Av1Transform1dMath.Clamp(step.V2 - step.V6, stageRange[stage]); + output.V7 = Av1Transform1dMath.Clamp(step.V3 - step.V7, stageRange[stage]); + output.V8 = Av1Transform1dMath.Clamp(step.V8 + step.V12, stageRange[stage]); + output.V9 = Av1Transform1dMath.Clamp(step.V9 + step.V13, stageRange[stage]); + output.V10 = Av1Transform1dMath.Clamp(step.V10 + step.V14, stageRange[stage]); + output.V11 = Av1Transform1dMath.Clamp(step.V11 + step.V15, stageRange[stage]); + output.V12 = Av1Transform1dMath.Clamp(step.V8 - step.V12, stageRange[stage]); + output.V13 = Av1Transform1dMath.Clamp(step.V9 - step.V13, stageRange[stage]); + output.V14 = Av1Transform1dMath.Clamp(step.V10 - step.V14, stageRange[stage]); + output.V15 = Av1Transform1dMath.Clamp(step.V11 - step.V15, stageRange[stage]); + + // Stage 6 reverses the pi/8 and 3pi/8 rotations. + stage++; + step.V0 = output.V0; + step.V1 = output.V1; + step.V2 = output.V2; + step.V3 = output.V3; + step.V4 = Av1Transform1dMath.HalfButterfly(cospi[16], output.V4, cospi[48], output.V5, cosBit); + step.V5 = Av1Transform1dMath.HalfButterfly(cospi[48], output.V4, -cospi[16], output.V5, cosBit); + step.V6 = Av1Transform1dMath.HalfButterfly(-cospi[48], output.V6, cospi[16], output.V7, cosBit); + step.V7 = Av1Transform1dMath.HalfButterfly(cospi[16], output.V6, cospi[48], output.V7, cosBit); + step.V8 = output.V8; + step.V9 = output.V9; + step.V10 = output.V10; + step.V11 = output.V11; + step.V12 = Av1Transform1dMath.HalfButterfly(cospi[16], output.V12, cospi[48], output.V13, cosBit); + step.V13 = Av1Transform1dMath.HalfButterfly(cospi[48], output.V12, -cospi[16], output.V13, cosBit); + step.V14 = Av1Transform1dMath.HalfButterfly(-cospi[48], output.V14, cospi[16], output.V15, cosBit); + step.V15 = Av1Transform1dMath.HalfButterfly(cospi[16], output.V14, cospi[48], output.V15, cosBit); + + // Stage 7 separates the four-sample groups into adjacent coefficient pairs and clamps each lane. + stage++; + output.V0 = Av1Transform1dMath.Clamp(step.V0 + step.V2, stageRange[stage]); + output.V1 = Av1Transform1dMath.Clamp(step.V1 + step.V3, stageRange[stage]); + output.V2 = Av1Transform1dMath.Clamp(step.V0 - step.V2, stageRange[stage]); + output.V3 = Av1Transform1dMath.Clamp(step.V1 - step.V3, stageRange[stage]); + output.V4 = Av1Transform1dMath.Clamp(step.V4 + step.V6, stageRange[stage]); + output.V5 = Av1Transform1dMath.Clamp(step.V5 + step.V7, stageRange[stage]); + output.V6 = Av1Transform1dMath.Clamp(step.V4 - step.V6, stageRange[stage]); + output.V7 = Av1Transform1dMath.Clamp(step.V5 - step.V7, stageRange[stage]); + output.V8 = Av1Transform1dMath.Clamp(step.V8 + step.V10, stageRange[stage]); + output.V9 = Av1Transform1dMath.Clamp(step.V9 + step.V11, stageRange[stage]); + output.V10 = Av1Transform1dMath.Clamp(step.V8 - step.V10, stageRange[stage]); + output.V11 = Av1Transform1dMath.Clamp(step.V9 - step.V11, stageRange[stage]); + output.V12 = Av1Transform1dMath.Clamp(step.V12 + step.V14, stageRange[stage]); + output.V13 = Av1Transform1dMath.Clamp(step.V13 + step.V15, stageRange[stage]); + output.V14 = Av1Transform1dMath.Clamp(step.V12 - step.V14, stageRange[stage]); + output.V15 = Av1Transform1dMath.Clamp(step.V13 - step.V15, stageRange[stage]); + + // Stage 8 reverses the pi/4 rotations for the middle pairs. + step.V0 = output.V0; + step.V1 = output.V1; + step.V2 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V2, cospi[32], output.V3, cosBit); + step.V3 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V2, -cospi[32], output.V3, cosBit); + step.V4 = output.V4; + step.V5 = output.V5; + step.V6 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V6, cospi[32], output.V7, cosBit); + step.V7 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V6, -cospi[32], output.V7, cosBit); + step.V8 = output.V8; + step.V9 = output.V9; + step.V10 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V10, cospi[32], output.V11, cosBit); + step.V11 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V10, -cospi[32], output.V11, cosBit); + step.V12 = output.V12; + step.V13 = output.V13; + step.V14 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V14, cospi[32], output.V15, cosBit); + step.V15 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V14, -cospi[32], output.V15, cosBit); + + // Stage 9 applies the AV1 signs and permutation that restore spatial sample order. + output.V0 = step.V0; + output.V1 = -step.V8; + output.V2 = step.V12; + output.V3 = -step.V4; + output.V4 = step.V6; + output.V5 = -step.V14; + output.V6 = step.V10; + output.V7 = -step.V2; + output.V8 = step.V3; + output.V9 = -step.V11; + output.V10 = step.V15; + output.V11 = -step.V7; + output.V12 = step.V5; + output.V13 = -step.V13; + output.V14 = step.V9; + output.V15 = -step.V1; + } + + /// + /// Applies the transform to four independent axes in parallel. + /// + /// The source values for the parallel transform axes. + /// The destination values for the parallel transform axes. + /// The fixed stage storage for the parallel transform axes. + /// The fixed-point precision of the cosine constants. + /// The signed-bit range assigned to each transform stage. + public static void Transform( + ref Av1TransformVector> input, + ref Av1TransformVector> output, + ref Av1TransformVector> step, + int cosBit, + InlineArray12 stageRange) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + int stage = 0; + + // Stage 1 permutes the coefficients into the signed order used by the ADST factorization. + stage++; + output.V0 = input.V15; + output.V1 = input.V0; + output.V2 = input.V13; + output.V3 = input.V2; + output.V4 = input.V11; + output.V5 = input.V4; + output.V6 = input.V9; + output.V7 = input.V6; + output.V8 = input.V7; + output.V9 = input.V8; + output.V10 = input.V5; + output.V11 = input.V10; + output.V12 = input.V3; + output.V13 = input.V12; + output.V14 = input.V1; + output.V15 = input.V14; + + // Stage 2 applies the terminal odd-angle rotations in reverse. + stage++; + step.V0 = Av1Transform1dMath.HalfButterfly(cospi[2], output.V0, cospi[62], output.V1, cosBit); + step.V1 = Av1Transform1dMath.HalfButterfly(cospi[62], output.V0, -cospi[2], output.V1, cosBit); + step.V2 = Av1Transform1dMath.HalfButterfly(cospi[10], output.V2, cospi[54], output.V3, cosBit); + step.V3 = Av1Transform1dMath.HalfButterfly(cospi[54], output.V2, -cospi[10], output.V3, cosBit); + step.V4 = Av1Transform1dMath.HalfButterfly(cospi[18], output.V4, cospi[46], output.V5, cosBit); + step.V5 = Av1Transform1dMath.HalfButterfly(cospi[46], output.V4, -cospi[18], output.V5, cosBit); + step.V6 = Av1Transform1dMath.HalfButterfly(cospi[26], output.V6, cospi[38], output.V7, cosBit); + step.V7 = Av1Transform1dMath.HalfButterfly(cospi[38], output.V6, -cospi[26], output.V7, cosBit); + step.V8 = Av1Transform1dMath.HalfButterfly(cospi[34], output.V8, cospi[30], output.V9, cosBit); + step.V9 = Av1Transform1dMath.HalfButterfly(cospi[30], output.V8, -cospi[34], output.V9, cosBit); + step.V10 = Av1Transform1dMath.HalfButterfly(cospi[42], output.V10, cospi[22], output.V11, cosBit); + step.V11 = Av1Transform1dMath.HalfButterfly(cospi[22], output.V10, -cospi[42], output.V11, cosBit); + step.V12 = Av1Transform1dMath.HalfButterfly(cospi[50], output.V12, cospi[14], output.V13, cosBit); + step.V13 = Av1Transform1dMath.HalfButterfly(cospi[14], output.V12, -cospi[50], output.V13, cosBit); + step.V14 = Av1Transform1dMath.HalfButterfly(cospi[58], output.V14, cospi[6], output.V15, cosBit); + step.V15 = Av1Transform1dMath.HalfButterfly(cospi[6], output.V14, -cospi[58], output.V15, cosBit); + + // Stage 3 separates the complete butterfly into two eight-sample halves and clamps each lane. + stage++; + output.V0 = Av1Transform1dMath.Clamp(step.V0 + step.V8, stageRange[stage]); + output.V1 = Av1Transform1dMath.Clamp(step.V1 + step.V9, stageRange[stage]); + output.V2 = Av1Transform1dMath.Clamp(step.V2 + step.V10, stageRange[stage]); + output.V3 = Av1Transform1dMath.Clamp(step.V3 + step.V11, stageRange[stage]); + output.V4 = Av1Transform1dMath.Clamp(step.V4 + step.V12, stageRange[stage]); + output.V5 = Av1Transform1dMath.Clamp(step.V5 + step.V13, stageRange[stage]); + output.V6 = Av1Transform1dMath.Clamp(step.V6 + step.V14, stageRange[stage]); + output.V7 = Av1Transform1dMath.Clamp(step.V7 + step.V15, stageRange[stage]); + output.V8 = Av1Transform1dMath.Clamp(step.V0 - step.V8, stageRange[stage]); + output.V9 = Av1Transform1dMath.Clamp(step.V1 - step.V9, stageRange[stage]); + output.V10 = Av1Transform1dMath.Clamp(step.V2 - step.V10, stageRange[stage]); + output.V11 = Av1Transform1dMath.Clamp(step.V3 - step.V11, stageRange[stage]); + output.V12 = Av1Transform1dMath.Clamp(step.V4 - step.V12, stageRange[stage]); + output.V13 = Av1Transform1dMath.Clamp(step.V5 - step.V13, stageRange[stage]); + output.V14 = Av1Transform1dMath.Clamp(step.V6 - step.V14, stageRange[stage]); + output.V15 = Av1Transform1dMath.Clamp(step.V7 - step.V15, stageRange[stage]); + + // Stage 4 reverses the pi/16 rotations in the upper half. + stage++; + step.V0 = output.V0; + step.V1 = output.V1; + step.V2 = output.V2; + step.V3 = output.V3; + step.V4 = output.V4; + step.V5 = output.V5; + step.V6 = output.V6; + step.V7 = output.V7; + step.V8 = Av1Transform1dMath.HalfButterfly(cospi[8], output.V8, cospi[56], output.V9, cosBit); + step.V9 = Av1Transform1dMath.HalfButterfly(cospi[56], output.V8, -cospi[8], output.V9, cosBit); + step.V10 = Av1Transform1dMath.HalfButterfly(cospi[40], output.V10, cospi[24], output.V11, cosBit); + step.V11 = Av1Transform1dMath.HalfButterfly(cospi[24], output.V10, -cospi[40], output.V11, cosBit); + step.V12 = Av1Transform1dMath.HalfButterfly(-cospi[56], output.V12, cospi[8], output.V13, cosBit); + step.V13 = Av1Transform1dMath.HalfButterfly(cospi[8], output.V12, cospi[56], output.V13, cosBit); + step.V14 = Av1Transform1dMath.HalfButterfly(-cospi[24], output.V14, cospi[40], output.V15, cosBit); + step.V15 = Av1Transform1dMath.HalfButterfly(cospi[40], output.V14, cospi[24], output.V15, cosBit); + + // Stage 5 separates each eight-sample half into four-sample groups and clamps each lane. + stage++; + output.V0 = Av1Transform1dMath.Clamp(step.V0 + step.V4, stageRange[stage]); + output.V1 = Av1Transform1dMath.Clamp(step.V1 + step.V5, stageRange[stage]); + output.V2 = Av1Transform1dMath.Clamp(step.V2 + step.V6, stageRange[stage]); + output.V3 = Av1Transform1dMath.Clamp(step.V3 + step.V7, stageRange[stage]); + output.V4 = Av1Transform1dMath.Clamp(step.V0 - step.V4, stageRange[stage]); + output.V5 = Av1Transform1dMath.Clamp(step.V1 - step.V5, stageRange[stage]); + output.V6 = Av1Transform1dMath.Clamp(step.V2 - step.V6, stageRange[stage]); + output.V7 = Av1Transform1dMath.Clamp(step.V3 - step.V7, stageRange[stage]); + output.V8 = Av1Transform1dMath.Clamp(step.V8 + step.V12, stageRange[stage]); + output.V9 = Av1Transform1dMath.Clamp(step.V9 + step.V13, stageRange[stage]); + output.V10 = Av1Transform1dMath.Clamp(step.V10 + step.V14, stageRange[stage]); + output.V11 = Av1Transform1dMath.Clamp(step.V11 + step.V15, stageRange[stage]); + output.V12 = Av1Transform1dMath.Clamp(step.V8 - step.V12, stageRange[stage]); + output.V13 = Av1Transform1dMath.Clamp(step.V9 - step.V13, stageRange[stage]); + output.V14 = Av1Transform1dMath.Clamp(step.V10 - step.V14, stageRange[stage]); + output.V15 = Av1Transform1dMath.Clamp(step.V11 - step.V15, stageRange[stage]); + + // Stage 6 reverses the pi/8 and 3pi/8 rotations. + stage++; + step.V0 = output.V0; + step.V1 = output.V1; + step.V2 = output.V2; + step.V3 = output.V3; + step.V4 = Av1Transform1dMath.HalfButterfly(cospi[16], output.V4, cospi[48], output.V5, cosBit); + step.V5 = Av1Transform1dMath.HalfButterfly(cospi[48], output.V4, -cospi[16], output.V5, cosBit); + step.V6 = Av1Transform1dMath.HalfButterfly(-cospi[48], output.V6, cospi[16], output.V7, cosBit); + step.V7 = Av1Transform1dMath.HalfButterfly(cospi[16], output.V6, cospi[48], output.V7, cosBit); + step.V8 = output.V8; + step.V9 = output.V9; + step.V10 = output.V10; + step.V11 = output.V11; + step.V12 = Av1Transform1dMath.HalfButterfly(cospi[16], output.V12, cospi[48], output.V13, cosBit); + step.V13 = Av1Transform1dMath.HalfButterfly(cospi[48], output.V12, -cospi[16], output.V13, cosBit); + step.V14 = Av1Transform1dMath.HalfButterfly(-cospi[48], output.V14, cospi[16], output.V15, cosBit); + step.V15 = Av1Transform1dMath.HalfButterfly(cospi[16], output.V14, cospi[48], output.V15, cosBit); + + // Stage 7 separates the four-sample groups into adjacent coefficient pairs and clamps each lane. + stage++; + output.V0 = Av1Transform1dMath.Clamp(step.V0 + step.V2, stageRange[stage]); + output.V1 = Av1Transform1dMath.Clamp(step.V1 + step.V3, stageRange[stage]); + output.V2 = Av1Transform1dMath.Clamp(step.V0 - step.V2, stageRange[stage]); + output.V3 = Av1Transform1dMath.Clamp(step.V1 - step.V3, stageRange[stage]); + output.V4 = Av1Transform1dMath.Clamp(step.V4 + step.V6, stageRange[stage]); + output.V5 = Av1Transform1dMath.Clamp(step.V5 + step.V7, stageRange[stage]); + output.V6 = Av1Transform1dMath.Clamp(step.V4 - step.V6, stageRange[stage]); + output.V7 = Av1Transform1dMath.Clamp(step.V5 - step.V7, stageRange[stage]); + output.V8 = Av1Transform1dMath.Clamp(step.V8 + step.V10, stageRange[stage]); + output.V9 = Av1Transform1dMath.Clamp(step.V9 + step.V11, stageRange[stage]); + output.V10 = Av1Transform1dMath.Clamp(step.V8 - step.V10, stageRange[stage]); + output.V11 = Av1Transform1dMath.Clamp(step.V9 - step.V11, stageRange[stage]); + output.V12 = Av1Transform1dMath.Clamp(step.V12 + step.V14, stageRange[stage]); + output.V13 = Av1Transform1dMath.Clamp(step.V13 + step.V15, stageRange[stage]); + output.V14 = Av1Transform1dMath.Clamp(step.V12 - step.V14, stageRange[stage]); + output.V15 = Av1Transform1dMath.Clamp(step.V13 - step.V15, stageRange[stage]); + + // Stage 8 reverses the pi/4 rotations for the middle pairs. + step.V0 = output.V0; + step.V1 = output.V1; + step.V2 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V2, cospi[32], output.V3, cosBit); + step.V3 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V2, -cospi[32], output.V3, cosBit); + step.V4 = output.V4; + step.V5 = output.V5; + step.V6 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V6, cospi[32], output.V7, cosBit); + step.V7 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V6, -cospi[32], output.V7, cosBit); + step.V8 = output.V8; + step.V9 = output.V9; + step.V10 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V10, cospi[32], output.V11, cosBit); + step.V11 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V10, -cospi[32], output.V11, cosBit); + step.V12 = output.V12; + step.V13 = output.V13; + step.V14 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V14, cospi[32], output.V15, cosBit); + step.V15 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V14, -cospi[32], output.V15, cosBit); + + // Stage 9 applies the AV1 signs and permutation that restore spatial sample order. + output.V0 = step.V0; + output.V1 = -step.V8; + output.V2 = step.V12; + output.V3 = -step.V4; + output.V4 = step.V6; + output.V5 = -step.V14; + output.V6 = step.V10; + output.V7 = -step.V2; + output.V8 = step.V3; + output.V9 = -step.V11; + output.V10 = step.V15; + output.V11 = -step.V7; + output.V12 = step.V5; + output.V13 = -step.V13; + output.V14 = step.V9; + output.V15 = -step.V1; + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Transform/Inverse/Av1Inverse2dTransformer.Adst4Operator.cs b/src/ImageSharp/Formats/Heif/Av1/Transform/Inverse/Av1Inverse2dTransformer.Adst4Operator.cs new file mode 100644 index 0000000000..ba1badf334 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Transform/Inverse/Av1Inverse2dTransformer.Adst4Operator.cs @@ -0,0 +1,145 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +/// +/// Defines the four-point AV1 inverse asymmetric discrete sine transform operator. +/// +/// +/// Vector fields represent transform positions and vector lanes represent independent axes. The SIMD overloads apply +/// the same staged rotations, fixed-point rounding, and range clamps as the scalar overload without mixing axes. +/// +internal static partial class Av1Inverse2dTransformer +{ + internal readonly struct Adst4Operator : IAv1Transform1dOperator + { + /// + /// Applies the normative four-point AV1 inverse asymmetric discrete sine transform. + /// + /// The four frequency-domain coefficients. + /// The four spatial-domain residual values. + /// The stage buffer owned by the containing two-dimensional transform. + /// The fixed-point precision of the sine constants. + /// The signed-bit range assigned to each transform stage. + public static void Transform(ReadOnlySpan input, Span output, Span step, int cosBit, InlineArray12 stageRange) + { + ReadOnlySpan sinpi = Av1SinusConstants.SinusPi(cosBit); + + // the reference decoder widens the complete four-point factorization because the products retain their fixed-point scale + // until the final shift. The stage buffer is therefore unnecessary for this transform size. + long x0 = input[0]; + long x1 = input[1]; + long x2 = input[2]; + long x3 = input[3]; + + _ = step; + _ = stageRange; + + // Avoid the multiplications for the all-zero coefficient vector, matching the reference decoder's scalar kernel. + if ((x0 | x1 | x2 | x3) == 0) + { + output[..4].Clear(); + return; + } + + // Stages 1 and 2 form the seven sine products and the one unscaled combination used by stage 3. + long s0 = sinpi[1] * x0; + long s1 = sinpi[2] * x0; + long s2 = sinpi[3] * x1; + long s3 = sinpi[4] * x2; + long s4 = sinpi[1] * x2; + long s5 = sinpi[2] * x3; + long s6 = sinpi[4] * x3; + long s7 = (x0 - x2) + x3; + + // Stages 3 through 6 combine the products while preserving the fixed-point scale until the final rounding. + s0 += s3; + s1 -= s4; + s3 = s2; + s2 = sinpi[3] * s7; + s0 += s5; + s1 -= s6; + x0 = s0 + s3; + x1 = s1 + s3; + x2 = s2; + x3 = (s0 + s1) - s3; + + output[0] = Av1Math.RoundShift(x0, cosBit); + output[1] = Av1Math.RoundShift(x1, cosBit); + output[2] = Av1Math.RoundShift(x2, cosBit); + output[3] = Av1Math.RoundShift(x3, cosBit); + } + + /// + public static void Transform( + ref Av1TransformVector> input, + ref Av1TransformVector> output, + ref Av1TransformVector> step, + int cosBit, + InlineArray12 stageRange) + { + bool widenedRound = stageRange[0] >= Av1Transform1dMath.WidenedIntermediateBitCount; + + ReadOnlySpan sinpi = Av1SinusConstants.SinusPi(cosBit); + Vector128 x0 = input.V0; + Vector128 x1 = input.V1; + Vector128 x2 = input.V2; + Vector128 x3 = input.V3; + + // The reference decoder retains the sine-table scale in Int32 products and sums, but performs the twelve-bit row + // kernel's terminal scaling and rounding in Int64. This is the only stage whose rounding bias can overflow + // a valid Int32 fixed-point sum. + if (widenedRound) + { + output.V0 = Av1Transform1dMath.MultiplyAdd4WidenedRound(sinpi[1], x0, sinpi[3], x1, sinpi[4], x2, sinpi[2], x3, cosBit); + output.V1 = Av1Transform1dMath.MultiplyAdd4WidenedRound(sinpi[2], x0, sinpi[3], x1, -sinpi[1], x2, -sinpi[4], x3, cosBit); + output.V2 = Av1Transform1dMath.MultiplyAdd4WidenedRound(sinpi[3], x0, 0, x1, -sinpi[3], x2, sinpi[3], x3, cosBit); + output.V3 = Av1Transform1dMath.MultiplyAdd4WidenedRound(sinpi[1] + sinpi[2], x0, -sinpi[3], x1, sinpi[4] - sinpi[1], x2, sinpi[2] - sinpi[4], x3, cosBit); + return; + } + + output.V0 = Av1Transform1dMath.MultiplyAdd4(sinpi[1], x0, sinpi[3], x1, sinpi[4], x2, sinpi[2], x3, cosBit); + output.V1 = Av1Transform1dMath.MultiplyAdd4(sinpi[2], x0, sinpi[3], x1, -sinpi[1], x2, -sinpi[4], x3, cosBit); + output.V2 = Av1Transform1dMath.MultiplyAdd4(sinpi[3], x0, 0, x1, -sinpi[3], x2, sinpi[3], x3, cosBit); + output.V3 = Av1Transform1dMath.MultiplyAdd4(sinpi[1] + sinpi[2], x0, -sinpi[3], x1, sinpi[4] - sinpi[1], x2, sinpi[2] - sinpi[4], x3, cosBit); + + _ = step; + } + + /// + public static void Transform( + ref Av1TransformVector> input, + ref Av1TransformVector> output, + ref Av1TransformVector> step, + int cosBit, + InlineArray12 stageRange) + { + bool widenedRound = stageRange[0] >= Av1Transform1dMath.WidenedIntermediateBitCount; + + ReadOnlySpan sinpi = Av1SinusConstants.SinusPi(cosBit); + Vector256 x0 = input.V0; + Vector256 x1 = input.V1; + Vector256 x2 = input.V2; + Vector256 x3 = input.V3; + + if (widenedRound) + { + output.V0 = Av1Transform1dMath.MultiplyAdd4WidenedRound(sinpi[1], x0, sinpi[3], x1, sinpi[4], x2, sinpi[2], x3, cosBit); + output.V1 = Av1Transform1dMath.MultiplyAdd4WidenedRound(sinpi[2], x0, sinpi[3], x1, -sinpi[1], x2, -sinpi[4], x3, cosBit); + output.V2 = Av1Transform1dMath.MultiplyAdd4WidenedRound(sinpi[3], x0, 0, x1, -sinpi[3], x2, sinpi[3], x3, cosBit); + output.V3 = Av1Transform1dMath.MultiplyAdd4WidenedRound(sinpi[1] + sinpi[2], x0, -sinpi[3], x1, sinpi[4] - sinpi[1], x2, sinpi[2] - sinpi[4], x3, cosBit); + return; + } + + output.V0 = Av1Transform1dMath.MultiplyAdd4(sinpi[1], x0, sinpi[3], x1, sinpi[4], x2, sinpi[2], x3, cosBit); + output.V1 = Av1Transform1dMath.MultiplyAdd4(sinpi[2], x0, sinpi[3], x1, -sinpi[1], x2, -sinpi[4], x3, cosBit); + output.V2 = Av1Transform1dMath.MultiplyAdd4(sinpi[3], x0, 0, x1, -sinpi[3], x2, sinpi[3], x3, cosBit); + output.V3 = Av1Transform1dMath.MultiplyAdd4(sinpi[1] + sinpi[2], x0, -sinpi[3], x1, sinpi[4] - sinpi[1], x2, sinpi[2] - sinpi[4], x3, cosBit); + + _ = step; + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Transform/Inverse/Av1Inverse2dTransformer.Adst8Operator.cs b/src/ImageSharp/Formats/Heif/Av1/Transform/Inverse/Av1Inverse2dTransformer.Adst8Operator.cs new file mode 100644 index 0000000000..c741ec7dce --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Transform/Inverse/Av1Inverse2dTransformer.Adst8Operator.cs @@ -0,0 +1,292 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +/// +/// Defines the eight-point AV1 inverse asymmetric discrete sine transform operator. +/// +/// +/// Vector fields represent transform positions and vector lanes represent independent axes. The SIMD overloads apply +/// the same staged rotations, fixed-point rounding, and range clamps as the scalar overload without mixing axes. +/// +internal static partial class Av1Inverse2dTransformer +{ + internal readonly struct Adst8Operator : IAv1Transform1dOperator + { + /// + /// Applies the normative eight-point AV1 inverse asymmetric discrete sine transform. + /// + /// The eight frequency-domain coefficients. + /// The eight spatial-domain residual values. + /// The eight-element stage buffer owned by the containing two-dimensional transform. + /// The fixed-point precision of the cosine constants. + /// The signed-bit range assigned to each transform stage. + public static void Transform(ReadOnlySpan input, Span output, Span step, int cosBit, InlineArray12 stageRange) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + int stage = 0; + + // Stage 1 permutes the coefficients into the signed order used by the ADST factorization. + stage++; + output[0] = input[7]; + output[1] = input[0]; + output[2] = input[5]; + output[3] = input[2]; + output[4] = input[3]; + output[5] = input[4]; + output[6] = input[1]; + output[7] = input[6]; + + // Stage 2 applies the terminal odd-angle rotations in reverse. + stage++; + step[0] = Av1Transform1dMath.HalfButterfly(cospi[4], output[0], cospi[60], output[1], cosBit); + step[1] = Av1Transform1dMath.HalfButterfly(cospi[60], output[0], -cospi[4], output[1], cosBit); + step[2] = Av1Transform1dMath.HalfButterfly(cospi[20], output[2], cospi[44], output[3], cosBit); + step[3] = Av1Transform1dMath.HalfButterfly(cospi[44], output[2], -cospi[20], output[3], cosBit); + step[4] = Av1Transform1dMath.HalfButterfly(cospi[36], output[4], cospi[28], output[5], cosBit); + step[5] = Av1Transform1dMath.HalfButterfly(cospi[28], output[4], -cospi[36], output[5], cosBit); + step[6] = Av1Transform1dMath.HalfButterfly(cospi[52], output[6], cospi[12], output[7], cosBit); + step[7] = Av1Transform1dMath.HalfButterfly(cospi[12], output[6], -cospi[52], output[7], cosBit); + + // Stage 3 separates the complete butterfly into two four-sample halves and clamps each lane. + stage++; + output[0] = Av1Transform1dMath.Clamp(step[0] + step[4], stageRange[stage]); + output[1] = Av1Transform1dMath.Clamp(step[1] + step[5], stageRange[stage]); + output[2] = Av1Transform1dMath.Clamp(step[2] + step[6], stageRange[stage]); + output[3] = Av1Transform1dMath.Clamp(step[3] + step[7], stageRange[stage]); + output[4] = Av1Transform1dMath.Clamp(step[0] - step[4], stageRange[stage]); + output[5] = Av1Transform1dMath.Clamp(step[1] - step[5], stageRange[stage]); + output[6] = Av1Transform1dMath.Clamp(step[2] - step[6], stageRange[stage]); + output[7] = Av1Transform1dMath.Clamp(step[3] - step[7], stageRange[stage]); + + // Stage 4 reverses the pi/8 and 3pi/8 rotations in the upper half. + stage++; + step[0] = output[0]; + step[1] = output[1]; + step[2] = output[2]; + step[3] = output[3]; + step[4] = Av1Transform1dMath.HalfButterfly(cospi[16], output[4], cospi[48], output[5], cosBit); + step[5] = Av1Transform1dMath.HalfButterfly(cospi[48], output[4], -cospi[16], output[5], cosBit); + step[6] = Av1Transform1dMath.HalfButterfly(-cospi[48], output[6], cospi[16], output[7], cosBit); + step[7] = Av1Transform1dMath.HalfButterfly(cospi[16], output[6], cospi[48], output[7], cosBit); + + // Stage 5 separates the four-sample halves into adjacent coefficient pairs and clamps each lane. + stage++; + output[0] = Av1Transform1dMath.Clamp(step[0] + step[2], stageRange[stage]); + output[1] = Av1Transform1dMath.Clamp(step[1] + step[3], stageRange[stage]); + output[2] = Av1Transform1dMath.Clamp(step[0] - step[2], stageRange[stage]); + output[3] = Av1Transform1dMath.Clamp(step[1] - step[3], stageRange[stage]); + output[4] = Av1Transform1dMath.Clamp(step[4] + step[6], stageRange[stage]); + output[5] = Av1Transform1dMath.Clamp(step[5] + step[7], stageRange[stage]); + output[6] = Av1Transform1dMath.Clamp(step[4] - step[6], stageRange[stage]); + output[7] = Av1Transform1dMath.Clamp(step[5] - step[7], stageRange[stage]); + + // Stage 6 reverses the pi/4 rotations for the middle pairs. + stage++; + step[0] = output[0]; + step[1] = output[1]; + step[2] = Av1Transform1dMath.HalfButterfly(cospi[32], output[2], cospi[32], output[3], cosBit); + step[3] = Av1Transform1dMath.HalfButterfly(cospi[32], output[2], -cospi[32], output[3], cosBit); + step[4] = output[4]; + step[5] = output[5]; + step[6] = Av1Transform1dMath.HalfButterfly(cospi[32], output[6], cospi[32], output[7], cosBit); + step[7] = Av1Transform1dMath.HalfButterfly(cospi[32], output[6], -cospi[32], output[7], cosBit); + + // Stage 7 applies the AV1 signs and permutation that restore spatial sample order. + output[0] = step[0]; + output[1] = -step[4]; + output[2] = step[6]; + output[3] = -step[2]; + output[4] = step[3]; + output[5] = -step[7]; + output[6] = step[5]; + output[7] = -step[1]; + } + + /// + public static void Transform( + ref Av1TransformVector> input, + ref Av1TransformVector> output, + ref Av1TransformVector> step, + int cosBit, + InlineArray12 stageRange) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + int stage = 0; + + // Stage 1 permutes the coefficients into the signed order used by the ADST factorization. + stage++; + output.V0 = input.V7; + output.V1 = input.V0; + output.V2 = input.V5; + output.V3 = input.V2; + output.V4 = input.V3; + output.V5 = input.V4; + output.V6 = input.V1; + output.V7 = input.V6; + + // Stage 2 applies the terminal odd-angle rotations in reverse. + stage++; + step.V0 = Av1Transform1dMath.HalfButterfly(cospi[4], output.V0, cospi[60], output.V1, cosBit); + step.V1 = Av1Transform1dMath.HalfButterfly(cospi[60], output.V0, -cospi[4], output.V1, cosBit); + step.V2 = Av1Transform1dMath.HalfButterfly(cospi[20], output.V2, cospi[44], output.V3, cosBit); + step.V3 = Av1Transform1dMath.HalfButterfly(cospi[44], output.V2, -cospi[20], output.V3, cosBit); + step.V4 = Av1Transform1dMath.HalfButterfly(cospi[36], output.V4, cospi[28], output.V5, cosBit); + step.V5 = Av1Transform1dMath.HalfButterfly(cospi[28], output.V4, -cospi[36], output.V5, cosBit); + step.V6 = Av1Transform1dMath.HalfButterfly(cospi[52], output.V6, cospi[12], output.V7, cosBit); + step.V7 = Av1Transform1dMath.HalfButterfly(cospi[12], output.V6, -cospi[52], output.V7, cosBit); + + // Stage 3 separates the complete butterfly into two four-sample halves and clamps each lane. + stage++; + output.V0 = Av1Transform1dMath.Clamp(step.V0 + step.V4, stageRange[stage]); + output.V1 = Av1Transform1dMath.Clamp(step.V1 + step.V5, stageRange[stage]); + output.V2 = Av1Transform1dMath.Clamp(step.V2 + step.V6, stageRange[stage]); + output.V3 = Av1Transform1dMath.Clamp(step.V3 + step.V7, stageRange[stage]); + output.V4 = Av1Transform1dMath.Clamp(step.V0 - step.V4, stageRange[stage]); + output.V5 = Av1Transform1dMath.Clamp(step.V1 - step.V5, stageRange[stage]); + output.V6 = Av1Transform1dMath.Clamp(step.V2 - step.V6, stageRange[stage]); + output.V7 = Av1Transform1dMath.Clamp(step.V3 - step.V7, stageRange[stage]); + + // Stage 4 reverses the pi/8 and 3pi/8 rotations in the upper half. + stage++; + step.V0 = output.V0; + step.V1 = output.V1; + step.V2 = output.V2; + step.V3 = output.V3; + step.V4 = Av1Transform1dMath.HalfButterfly(cospi[16], output.V4, cospi[48], output.V5, cosBit); + step.V5 = Av1Transform1dMath.HalfButterfly(cospi[48], output.V4, -cospi[16], output.V5, cosBit); + step.V6 = Av1Transform1dMath.HalfButterfly(-cospi[48], output.V6, cospi[16], output.V7, cosBit); + step.V7 = Av1Transform1dMath.HalfButterfly(cospi[16], output.V6, cospi[48], output.V7, cosBit); + + // Stage 5 separates the four-sample halves into adjacent coefficient pairs and clamps each lane. + stage++; + output.V0 = Av1Transform1dMath.Clamp(step.V0 + step.V2, stageRange[stage]); + output.V1 = Av1Transform1dMath.Clamp(step.V1 + step.V3, stageRange[stage]); + output.V2 = Av1Transform1dMath.Clamp(step.V0 - step.V2, stageRange[stage]); + output.V3 = Av1Transform1dMath.Clamp(step.V1 - step.V3, stageRange[stage]); + output.V4 = Av1Transform1dMath.Clamp(step.V4 + step.V6, stageRange[stage]); + output.V5 = Av1Transform1dMath.Clamp(step.V5 + step.V7, stageRange[stage]); + output.V6 = Av1Transform1dMath.Clamp(step.V4 - step.V6, stageRange[stage]); + output.V7 = Av1Transform1dMath.Clamp(step.V5 - step.V7, stageRange[stage]); + + // Stage 6 reverses the pi/4 rotations for the middle pairs. + stage++; + step.V0 = output.V0; + step.V1 = output.V1; + step.V2 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V2, cospi[32], output.V3, cosBit); + step.V3 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V2, -cospi[32], output.V3, cosBit); + step.V4 = output.V4; + step.V5 = output.V5; + step.V6 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V6, cospi[32], output.V7, cosBit); + step.V7 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V6, -cospi[32], output.V7, cosBit); + + // Stage 7 applies the AV1 signs and permutation that restore spatial sample order. + output.V0 = step.V0; + output.V1 = -step.V4; + output.V2 = step.V6; + output.V3 = -step.V2; + output.V4 = step.V3; + output.V5 = -step.V7; + output.V6 = step.V5; + output.V7 = -step.V1; + } + + /// + /// Applies the transform to four independent axes in parallel. + /// + /// The source values for the parallel transform axes. + /// The destination values for the parallel transform axes. + /// The fixed stage storage for the parallel transform axes. + /// The fixed-point precision of the cosine constants. + /// The signed-bit range assigned to each transform stage. + public static void Transform( + ref Av1TransformVector> input, + ref Av1TransformVector> output, + ref Av1TransformVector> step, + int cosBit, + InlineArray12 stageRange) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + int stage = 0; + + // Stage 1 permutes the coefficients into the signed order used by the ADST factorization. + stage++; + output.V0 = input.V7; + output.V1 = input.V0; + output.V2 = input.V5; + output.V3 = input.V2; + output.V4 = input.V3; + output.V5 = input.V4; + output.V6 = input.V1; + output.V7 = input.V6; + + // Stage 2 applies the terminal odd-angle rotations in reverse. + stage++; + step.V0 = Av1Transform1dMath.HalfButterfly(cospi[4], output.V0, cospi[60], output.V1, cosBit); + step.V1 = Av1Transform1dMath.HalfButterfly(cospi[60], output.V0, -cospi[4], output.V1, cosBit); + step.V2 = Av1Transform1dMath.HalfButterfly(cospi[20], output.V2, cospi[44], output.V3, cosBit); + step.V3 = Av1Transform1dMath.HalfButterfly(cospi[44], output.V2, -cospi[20], output.V3, cosBit); + step.V4 = Av1Transform1dMath.HalfButterfly(cospi[36], output.V4, cospi[28], output.V5, cosBit); + step.V5 = Av1Transform1dMath.HalfButterfly(cospi[28], output.V4, -cospi[36], output.V5, cosBit); + step.V6 = Av1Transform1dMath.HalfButterfly(cospi[52], output.V6, cospi[12], output.V7, cosBit); + step.V7 = Av1Transform1dMath.HalfButterfly(cospi[12], output.V6, -cospi[52], output.V7, cosBit); + + // Stage 3 separates the complete butterfly into two four-sample halves and clamps each lane. + stage++; + output.V0 = Av1Transform1dMath.Clamp(step.V0 + step.V4, stageRange[stage]); + output.V1 = Av1Transform1dMath.Clamp(step.V1 + step.V5, stageRange[stage]); + output.V2 = Av1Transform1dMath.Clamp(step.V2 + step.V6, stageRange[stage]); + output.V3 = Av1Transform1dMath.Clamp(step.V3 + step.V7, stageRange[stage]); + output.V4 = Av1Transform1dMath.Clamp(step.V0 - step.V4, stageRange[stage]); + output.V5 = Av1Transform1dMath.Clamp(step.V1 - step.V5, stageRange[stage]); + output.V6 = Av1Transform1dMath.Clamp(step.V2 - step.V6, stageRange[stage]); + output.V7 = Av1Transform1dMath.Clamp(step.V3 - step.V7, stageRange[stage]); + + // Stage 4 reverses the pi/8 and 3pi/8 rotations in the upper half. + stage++; + step.V0 = output.V0; + step.V1 = output.V1; + step.V2 = output.V2; + step.V3 = output.V3; + step.V4 = Av1Transform1dMath.HalfButterfly(cospi[16], output.V4, cospi[48], output.V5, cosBit); + step.V5 = Av1Transform1dMath.HalfButterfly(cospi[48], output.V4, -cospi[16], output.V5, cosBit); + step.V6 = Av1Transform1dMath.HalfButterfly(-cospi[48], output.V6, cospi[16], output.V7, cosBit); + step.V7 = Av1Transform1dMath.HalfButterfly(cospi[16], output.V6, cospi[48], output.V7, cosBit); + + // Stage 5 separates the four-sample halves into adjacent coefficient pairs and clamps each lane. + stage++; + output.V0 = Av1Transform1dMath.Clamp(step.V0 + step.V2, stageRange[stage]); + output.V1 = Av1Transform1dMath.Clamp(step.V1 + step.V3, stageRange[stage]); + output.V2 = Av1Transform1dMath.Clamp(step.V0 - step.V2, stageRange[stage]); + output.V3 = Av1Transform1dMath.Clamp(step.V1 - step.V3, stageRange[stage]); + output.V4 = Av1Transform1dMath.Clamp(step.V4 + step.V6, stageRange[stage]); + output.V5 = Av1Transform1dMath.Clamp(step.V5 + step.V7, stageRange[stage]); + output.V6 = Av1Transform1dMath.Clamp(step.V4 - step.V6, stageRange[stage]); + output.V7 = Av1Transform1dMath.Clamp(step.V5 - step.V7, stageRange[stage]); + + // Stage 6 reverses the pi/4 rotations for the middle pairs. + stage++; + step.V0 = output.V0; + step.V1 = output.V1; + step.V2 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V2, cospi[32], output.V3, cosBit); + step.V3 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V2, -cospi[32], output.V3, cosBit); + step.V4 = output.V4; + step.V5 = output.V5; + step.V6 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V6, cospi[32], output.V7, cosBit); + step.V7 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V6, -cospi[32], output.V7, cosBit); + + // Stage 7 applies the AV1 signs and permutation that restore spatial sample order. + output.V0 = step.V0; + output.V1 = -step.V4; + output.V2 = step.V6; + output.V3 = -step.V2; + output.V4 = step.V3; + output.V5 = -step.V7; + output.V6 = step.V5; + output.V7 = -step.V1; + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Transform/Inverse/Av1Inverse2dTransformer.Dct16Operator.cs b/src/ImageSharp/Formats/Heif/Av1/Transform/Inverse/Av1Inverse2dTransformer.Dct16Operator.cs new file mode 100644 index 0000000000..7db0baec48 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Transform/Inverse/Av1Inverse2dTransformer.Dct16Operator.cs @@ -0,0 +1,478 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +/// +/// Defines the 16-point AV1 inverse discrete cosine transform operator. +/// +/// +/// Vector fields represent transform positions and vector lanes represent independent axes. The SIMD overloads apply +/// the same staged butterflies, fixed-point rounding, and range clamps as the scalar overload without mixing axes. +/// +internal static partial class Av1Inverse2dTransformer +{ + internal readonly struct Dct16Operator : IAv1Transform1dOperator + { + /// + /// Applies the normative 16-point AV1 inverse discrete cosine transform. + /// + /// The sixteen frequency-domain coefficients. + /// The sixteen spatial-domain residual values. + /// The sixteen-element stage buffer owned by the containing two-dimensional transform. + /// The fixed-point precision of the cosine constants. + /// The signed-bit range assigned to each transform stage. + public static void Transform(ReadOnlySpan input, Span output, Span step, int cosBit, InlineArray12 stageRange) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + int stage = 0; + + // Stage 1 permutes frequency-ordered coefficients into the recursive DCT factorization order. + stage++; + output[0] = input[0]; + output[1] = input[8]; + output[2] = input[4]; + output[3] = input[12]; + output[4] = input[2]; + output[5] = input[10]; + output[6] = input[6]; + output[7] = input[14]; + output[8] = input[1]; + output[9] = input[9]; + output[10] = input[5]; + output[11] = input[13]; + output[12] = input[3]; + output[13] = input[11]; + output[14] = input[7]; + output[15] = input[15]; + + // Stage 2 rotates the highest odd-frequency coefficient pairs by their pi/32 angles. + stage++; + step[0] = output[0]; + step[1] = output[1]; + step[2] = output[2]; + step[3] = output[3]; + step[4] = output[4]; + step[5] = output[5]; + step[6] = output[6]; + step[7] = output[7]; + step[8] = Av1Transform1dMath.HalfButterfly(cospi[60], output[8], -cospi[4], output[15], cosBit); + step[9] = Av1Transform1dMath.HalfButterfly(cospi[28], output[9], -cospi[36], output[14], cosBit); + step[10] = Av1Transform1dMath.HalfButterfly(cospi[44], output[10], -cospi[20], output[13], cosBit); + step[11] = Av1Transform1dMath.HalfButterfly(cospi[12], output[11], -cospi[52], output[12], cosBit); + step[12] = Av1Transform1dMath.HalfButterfly(cospi[52], output[11], cospi[12], output[12], cosBit); + step[13] = Av1Transform1dMath.HalfButterfly(cospi[20], output[10], cospi[44], output[13], cosBit); + step[14] = Av1Transform1dMath.HalfButterfly(cospi[36], output[9], cospi[28], output[14], cosBit); + step[15] = Av1Transform1dMath.HalfButterfly(cospi[4], output[8], cospi[60], output[15], cosBit); + + // Stage 3 reconstructs the embedded eight-point groups and combines adjacent odd terms. + stage++; + byte range = stageRange[stage]; + output[0] = step[0]; + output[1] = step[1]; + output[2] = step[2]; + output[3] = step[3]; + output[4] = Av1Transform1dMath.HalfButterfly(cospi[56], step[4], -cospi[8], step[7], cosBit); + output[5] = Av1Transform1dMath.HalfButterfly(cospi[24], step[5], -cospi[40], step[6], cosBit); + output[6] = Av1Transform1dMath.HalfButterfly(cospi[40], step[5], cospi[24], step[6], cosBit); + output[7] = Av1Transform1dMath.HalfButterfly(cospi[8], step[4], cospi[56], step[7], cosBit); + output[8] = Av1Transform1dMath.Clamp(step[8] + step[9], range); + output[9] = Av1Transform1dMath.Clamp(step[8] - step[9], range); + output[10] = Av1Transform1dMath.Clamp(step[11] - step[10], range); + output[11] = Av1Transform1dMath.Clamp(step[10] + step[11], range); + output[12] = Av1Transform1dMath.Clamp(step[12] + step[13], range); + output[13] = Av1Transform1dMath.Clamp(step[12] - step[13], range); + output[14] = Av1Transform1dMath.Clamp(step[15] - step[14], range); + output[15] = Av1Transform1dMath.Clamp(step[14] + step[15], range); + + // Stage 4 completes the low-frequency four-point DCT and rotates the next odd-frequency pairs. + stage++; + range = stageRange[stage]; + step[0] = Av1Transform1dMath.HalfButterfly(cospi[32], output[0], cospi[32], output[1], cosBit); + step[1] = Av1Transform1dMath.HalfButterfly(cospi[32], output[0], -cospi[32], output[1], cosBit); + step[2] = Av1Transform1dMath.HalfButterfly(cospi[48], output[2], -cospi[16], output[3], cosBit); + step[3] = Av1Transform1dMath.HalfButterfly(cospi[16], output[2], cospi[48], output[3], cosBit); + step[4] = Av1Transform1dMath.Clamp(output[4] + output[5], range); + step[5] = Av1Transform1dMath.Clamp(output[4] - output[5], range); + step[6] = Av1Transform1dMath.Clamp(output[7] - output[6], range); + step[7] = Av1Transform1dMath.Clamp(output[6] + output[7], range); + step[8] = output[8]; + step[9] = Av1Transform1dMath.HalfButterfly(-cospi[16], output[9], cospi[48], output[14], cosBit); + step[10] = Av1Transform1dMath.HalfButterfly(-cospi[48], output[10], -cospi[16], output[13], cosBit); + step[11] = output[11]; + step[12] = output[12]; + step[13] = Av1Transform1dMath.HalfButterfly(-cospi[16], output[10], cospi[48], output[13], cosBit); + step[14] = Av1Transform1dMath.HalfButterfly(cospi[48], output[9], cospi[16], output[14], cosBit); + step[15] = output[15]; + + // Stage 5 widens the reconstructed groups through their next butterfly level. + stage++; + range = stageRange[stage]; + output[0] = Av1Transform1dMath.Clamp(step[0] + step[3], range); + output[1] = Av1Transform1dMath.Clamp(step[1] + step[2], range); + output[2] = Av1Transform1dMath.Clamp(step[1] - step[2], range); + output[3] = Av1Transform1dMath.Clamp(step[0] - step[3], range); + output[4] = step[4]; + output[5] = Av1Transform1dMath.HalfButterfly(-cospi[32], step[5], cospi[32], step[6], cosBit); + output[6] = Av1Transform1dMath.HalfButterfly(cospi[32], step[5], cospi[32], step[6], cosBit); + output[7] = step[7]; + output[8] = Av1Transform1dMath.Clamp(step[8] + step[11], range); + output[9] = Av1Transform1dMath.Clamp(step[9] + step[10], range); + output[10] = Av1Transform1dMath.Clamp(step[9] - step[10], range); + output[11] = Av1Transform1dMath.Clamp(step[8] - step[11], range); + output[12] = Av1Transform1dMath.Clamp(step[15] - step[12], range); + output[13] = Av1Transform1dMath.Clamp(step[14] - step[13], range); + output[14] = Av1Transform1dMath.Clamp(step[13] + step[14], range); + output[15] = Av1Transform1dMath.Clamp(step[12] + step[15], range); + + // Stage 6 applies the remaining pi/4 rotations before the terminal spatial merge. + stage++; + range = stageRange[stage]; + step[0] = Av1Transform1dMath.Clamp(output[0] + output[7], range); + step[1] = Av1Transform1dMath.Clamp(output[1] + output[6], range); + step[2] = Av1Transform1dMath.Clamp(output[2] + output[5], range); + step[3] = Av1Transform1dMath.Clamp(output[3] + output[4], range); + step[4] = Av1Transform1dMath.Clamp(output[3] - output[4], range); + step[5] = Av1Transform1dMath.Clamp(output[2] - output[5], range); + step[6] = Av1Transform1dMath.Clamp(output[1] - output[6], range); + step[7] = Av1Transform1dMath.Clamp(output[0] - output[7], range); + step[8] = output[8]; + step[9] = output[9]; + step[10] = Av1Transform1dMath.HalfButterfly(-cospi[32], output[10], cospi[32], output[13], cosBit); + step[11] = Av1Transform1dMath.HalfButterfly(-cospi[32], output[11], cospi[32], output[12], cosBit); + step[12] = Av1Transform1dMath.HalfButterfly(cospi[32], output[11], cospi[32], output[12], cosBit); + step[13] = Av1Transform1dMath.HalfButterfly(cospi[32], output[10], cospi[32], output[13], cosBit); + step[14] = output[14]; + step[15] = output[15]; + + // Stage 7 merges the even and odd halves into spatial order and clamps every result. + stage++; + range = stageRange[stage]; + output[0] = Av1Transform1dMath.Clamp(step[0] + step[15], range); + output[1] = Av1Transform1dMath.Clamp(step[1] + step[14], range); + output[2] = Av1Transform1dMath.Clamp(step[2] + step[13], range); + output[3] = Av1Transform1dMath.Clamp(step[3] + step[12], range); + output[4] = Av1Transform1dMath.Clamp(step[4] + step[11], range); + output[5] = Av1Transform1dMath.Clamp(step[5] + step[10], range); + output[6] = Av1Transform1dMath.Clamp(step[6] + step[9], range); + output[7] = Av1Transform1dMath.Clamp(step[7] + step[8], range); + output[8] = Av1Transform1dMath.Clamp(step[7] - step[8], range); + output[9] = Av1Transform1dMath.Clamp(step[6] - step[9], range); + output[10] = Av1Transform1dMath.Clamp(step[5] - step[10], range); + output[11] = Av1Transform1dMath.Clamp(step[4] - step[11], range); + output[12] = Av1Transform1dMath.Clamp(step[3] - step[12], range); + output[13] = Av1Transform1dMath.Clamp(step[2] - step[13], range); + output[14] = Av1Transform1dMath.Clamp(step[1] - step[14], range); + output[15] = Av1Transform1dMath.Clamp(step[0] - step[15], range); + } + + /// + public static void Transform( + ref Av1TransformVector> input, + ref Av1TransformVector> output, + ref Av1TransformVector> step, + int cosBit, + InlineArray12 stageRange) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + int stage = 0; + + // Stage 1 permutes frequency-ordered coefficients into the recursive DCT factorization order. + stage++; + output.V0 = input.V0; + output.V1 = input.V8; + output.V2 = input.V4; + output.V3 = input.V12; + output.V4 = input.V2; + output.V5 = input.V10; + output.V6 = input.V6; + output.V7 = input.V14; + output.V8 = input.V1; + output.V9 = input.V9; + output.V10 = input.V5; + output.V11 = input.V13; + output.V12 = input.V3; + output.V13 = input.V11; + output.V14 = input.V7; + output.V15 = input.V15; + + // Stage 2 rotates the highest odd-frequency coefficient pairs by their pi/32 angles. + stage++; + step.V0 = output.V0; + step.V1 = output.V1; + step.V2 = output.V2; + step.V3 = output.V3; + step.V4 = output.V4; + step.V5 = output.V5; + step.V6 = output.V6; + step.V7 = output.V7; + step.V8 = Av1Transform1dMath.HalfButterfly(cospi[60], output.V8, -cospi[4], output.V15, cosBit); + step.V9 = Av1Transform1dMath.HalfButterfly(cospi[28], output.V9, -cospi[36], output.V14, cosBit); + step.V10 = Av1Transform1dMath.HalfButterfly(cospi[44], output.V10, -cospi[20], output.V13, cosBit); + step.V11 = Av1Transform1dMath.HalfButterfly(cospi[12], output.V11, -cospi[52], output.V12, cosBit); + step.V12 = Av1Transform1dMath.HalfButterfly(cospi[52], output.V11, cospi[12], output.V12, cosBit); + step.V13 = Av1Transform1dMath.HalfButterfly(cospi[20], output.V10, cospi[44], output.V13, cosBit); + step.V14 = Av1Transform1dMath.HalfButterfly(cospi[36], output.V9, cospi[28], output.V14, cosBit); + step.V15 = Av1Transform1dMath.HalfButterfly(cospi[4], output.V8, cospi[60], output.V15, cosBit); + + // Stage 3 reconstructs the embedded eight-point groups and combines adjacent odd terms. + stage++; + byte range = stageRange[stage]; + output.V0 = step.V0; + output.V1 = step.V1; + output.V2 = step.V2; + output.V3 = step.V3; + output.V4 = Av1Transform1dMath.HalfButterfly(cospi[56], step.V4, -cospi[8], step.V7, cosBit); + output.V5 = Av1Transform1dMath.HalfButterfly(cospi[24], step.V5, -cospi[40], step.V6, cosBit); + output.V6 = Av1Transform1dMath.HalfButterfly(cospi[40], step.V5, cospi[24], step.V6, cosBit); + output.V7 = Av1Transform1dMath.HalfButterfly(cospi[8], step.V4, cospi[56], step.V7, cosBit); + output.V8 = Av1Transform1dMath.Clamp(step.V8 + step.V9, range); + output.V9 = Av1Transform1dMath.Clamp(step.V8 - step.V9, range); + output.V10 = Av1Transform1dMath.Clamp(step.V11 - step.V10, range); + output.V11 = Av1Transform1dMath.Clamp(step.V10 + step.V11, range); + output.V12 = Av1Transform1dMath.Clamp(step.V12 + step.V13, range); + output.V13 = Av1Transform1dMath.Clamp(step.V12 - step.V13, range); + output.V14 = Av1Transform1dMath.Clamp(step.V15 - step.V14, range); + output.V15 = Av1Transform1dMath.Clamp(step.V14 + step.V15, range); + + // Stage 4 completes the low-frequency four-point DCT and rotates the next odd-frequency pairs. + stage++; + range = stageRange[stage]; + step.V0 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V0, cospi[32], output.V1, cosBit); + step.V1 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V0, -cospi[32], output.V1, cosBit); + step.V2 = Av1Transform1dMath.HalfButterfly(cospi[48], output.V2, -cospi[16], output.V3, cosBit); + step.V3 = Av1Transform1dMath.HalfButterfly(cospi[16], output.V2, cospi[48], output.V3, cosBit); + step.V4 = Av1Transform1dMath.Clamp(output.V4 + output.V5, range); + step.V5 = Av1Transform1dMath.Clamp(output.V4 - output.V5, range); + step.V6 = Av1Transform1dMath.Clamp(output.V7 - output.V6, range); + step.V7 = Av1Transform1dMath.Clamp(output.V6 + output.V7, range); + step.V8 = output.V8; + step.V9 = Av1Transform1dMath.HalfButterfly(-cospi[16], output.V9, cospi[48], output.V14, cosBit); + step.V10 = Av1Transform1dMath.HalfButterfly(-cospi[48], output.V10, -cospi[16], output.V13, cosBit); + step.V11 = output.V11; + step.V12 = output.V12; + step.V13 = Av1Transform1dMath.HalfButterfly(-cospi[16], output.V10, cospi[48], output.V13, cosBit); + step.V14 = Av1Transform1dMath.HalfButterfly(cospi[48], output.V9, cospi[16], output.V14, cosBit); + step.V15 = output.V15; + + // Stage 5 widens the reconstructed groups through their next butterfly level. + stage++; + range = stageRange[stage]; + output.V0 = Av1Transform1dMath.Clamp(step.V0 + step.V3, range); + output.V1 = Av1Transform1dMath.Clamp(step.V1 + step.V2, range); + output.V2 = Av1Transform1dMath.Clamp(step.V1 - step.V2, range); + output.V3 = Av1Transform1dMath.Clamp(step.V0 - step.V3, range); + output.V4 = step.V4; + output.V5 = Av1Transform1dMath.HalfButterfly(-cospi[32], step.V5, cospi[32], step.V6, cosBit); + output.V6 = Av1Transform1dMath.HalfButterfly(cospi[32], step.V5, cospi[32], step.V6, cosBit); + output.V7 = step.V7; + output.V8 = Av1Transform1dMath.Clamp(step.V8 + step.V11, range); + output.V9 = Av1Transform1dMath.Clamp(step.V9 + step.V10, range); + output.V10 = Av1Transform1dMath.Clamp(step.V9 - step.V10, range); + output.V11 = Av1Transform1dMath.Clamp(step.V8 - step.V11, range); + output.V12 = Av1Transform1dMath.Clamp(step.V15 - step.V12, range); + output.V13 = Av1Transform1dMath.Clamp(step.V14 - step.V13, range); + output.V14 = Av1Transform1dMath.Clamp(step.V13 + step.V14, range); + output.V15 = Av1Transform1dMath.Clamp(step.V12 + step.V15, range); + + // Stage 6 applies the remaining pi/4 rotations before the terminal spatial merge. + stage++; + range = stageRange[stage]; + step.V0 = Av1Transform1dMath.Clamp(output.V0 + output.V7, range); + step.V1 = Av1Transform1dMath.Clamp(output.V1 + output.V6, range); + step.V2 = Av1Transform1dMath.Clamp(output.V2 + output.V5, range); + step.V3 = Av1Transform1dMath.Clamp(output.V3 + output.V4, range); + step.V4 = Av1Transform1dMath.Clamp(output.V3 - output.V4, range); + step.V5 = Av1Transform1dMath.Clamp(output.V2 - output.V5, range); + step.V6 = Av1Transform1dMath.Clamp(output.V1 - output.V6, range); + step.V7 = Av1Transform1dMath.Clamp(output.V0 - output.V7, range); + step.V8 = output.V8; + step.V9 = output.V9; + step.V10 = Av1Transform1dMath.HalfButterfly(-cospi[32], output.V10, cospi[32], output.V13, cosBit); + step.V11 = Av1Transform1dMath.HalfButterfly(-cospi[32], output.V11, cospi[32], output.V12, cosBit); + step.V12 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V11, cospi[32], output.V12, cosBit); + step.V13 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V10, cospi[32], output.V13, cosBit); + step.V14 = output.V14; + step.V15 = output.V15; + + // Stage 7 merges the even and odd halves into spatial order and clamps every result. + stage++; + range = stageRange[stage]; + output.V0 = Av1Transform1dMath.Clamp(step.V0 + step.V15, range); + output.V1 = Av1Transform1dMath.Clamp(step.V1 + step.V14, range); + output.V2 = Av1Transform1dMath.Clamp(step.V2 + step.V13, range); + output.V3 = Av1Transform1dMath.Clamp(step.V3 + step.V12, range); + output.V4 = Av1Transform1dMath.Clamp(step.V4 + step.V11, range); + output.V5 = Av1Transform1dMath.Clamp(step.V5 + step.V10, range); + output.V6 = Av1Transform1dMath.Clamp(step.V6 + step.V9, range); + output.V7 = Av1Transform1dMath.Clamp(step.V7 + step.V8, range); + output.V8 = Av1Transform1dMath.Clamp(step.V7 - step.V8, range); + output.V9 = Av1Transform1dMath.Clamp(step.V6 - step.V9, range); + output.V10 = Av1Transform1dMath.Clamp(step.V5 - step.V10, range); + output.V11 = Av1Transform1dMath.Clamp(step.V4 - step.V11, range); + output.V12 = Av1Transform1dMath.Clamp(step.V3 - step.V12, range); + output.V13 = Av1Transform1dMath.Clamp(step.V2 - step.V13, range); + output.V14 = Av1Transform1dMath.Clamp(step.V1 - step.V14, range); + output.V15 = Av1Transform1dMath.Clamp(step.V0 - step.V15, range); + } + + /// + /// Applies the transform to four independent axes in parallel. + /// + /// The source values for the parallel transform axes. + /// The destination values for the parallel transform axes. + /// The fixed stage storage for the parallel transform axes. + /// The fixed-point precision of the cosine constants. + /// The signed-bit range assigned to each transform stage. + public static void Transform( + ref Av1TransformVector> input, + ref Av1TransformVector> output, + ref Av1TransformVector> step, + int cosBit, + InlineArray12 stageRange) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + int stage = 0; + + // Stage 1 permutes frequency-ordered coefficients into the recursive DCT factorization order. + stage++; + output.V0 = input.V0; + output.V1 = input.V8; + output.V2 = input.V4; + output.V3 = input.V12; + output.V4 = input.V2; + output.V5 = input.V10; + output.V6 = input.V6; + output.V7 = input.V14; + output.V8 = input.V1; + output.V9 = input.V9; + output.V10 = input.V5; + output.V11 = input.V13; + output.V12 = input.V3; + output.V13 = input.V11; + output.V14 = input.V7; + output.V15 = input.V15; + + // Stage 2 rotates the highest odd-frequency coefficient pairs by their pi/32 angles. + stage++; + step.V0 = output.V0; + step.V1 = output.V1; + step.V2 = output.V2; + step.V3 = output.V3; + step.V4 = output.V4; + step.V5 = output.V5; + step.V6 = output.V6; + step.V7 = output.V7; + step.V8 = Av1Transform1dMath.HalfButterfly(cospi[60], output.V8, -cospi[4], output.V15, cosBit); + step.V9 = Av1Transform1dMath.HalfButterfly(cospi[28], output.V9, -cospi[36], output.V14, cosBit); + step.V10 = Av1Transform1dMath.HalfButterfly(cospi[44], output.V10, -cospi[20], output.V13, cosBit); + step.V11 = Av1Transform1dMath.HalfButterfly(cospi[12], output.V11, -cospi[52], output.V12, cosBit); + step.V12 = Av1Transform1dMath.HalfButterfly(cospi[52], output.V11, cospi[12], output.V12, cosBit); + step.V13 = Av1Transform1dMath.HalfButterfly(cospi[20], output.V10, cospi[44], output.V13, cosBit); + step.V14 = Av1Transform1dMath.HalfButterfly(cospi[36], output.V9, cospi[28], output.V14, cosBit); + step.V15 = Av1Transform1dMath.HalfButterfly(cospi[4], output.V8, cospi[60], output.V15, cosBit); + + // Stage 3 reconstructs the embedded eight-point groups and combines adjacent odd terms. + stage++; + byte range = stageRange[stage]; + output.V0 = step.V0; + output.V1 = step.V1; + output.V2 = step.V2; + output.V3 = step.V3; + output.V4 = Av1Transform1dMath.HalfButterfly(cospi[56], step.V4, -cospi[8], step.V7, cosBit); + output.V5 = Av1Transform1dMath.HalfButterfly(cospi[24], step.V5, -cospi[40], step.V6, cosBit); + output.V6 = Av1Transform1dMath.HalfButterfly(cospi[40], step.V5, cospi[24], step.V6, cosBit); + output.V7 = Av1Transform1dMath.HalfButterfly(cospi[8], step.V4, cospi[56], step.V7, cosBit); + output.V8 = Av1Transform1dMath.Clamp(step.V8 + step.V9, range); + output.V9 = Av1Transform1dMath.Clamp(step.V8 - step.V9, range); + output.V10 = Av1Transform1dMath.Clamp(step.V11 - step.V10, range); + output.V11 = Av1Transform1dMath.Clamp(step.V10 + step.V11, range); + output.V12 = Av1Transform1dMath.Clamp(step.V12 + step.V13, range); + output.V13 = Av1Transform1dMath.Clamp(step.V12 - step.V13, range); + output.V14 = Av1Transform1dMath.Clamp(step.V15 - step.V14, range); + output.V15 = Av1Transform1dMath.Clamp(step.V14 + step.V15, range); + + // Stage 4 completes the low-frequency four-point DCT and rotates the next odd-frequency pairs. + stage++; + range = stageRange[stage]; + step.V0 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V0, cospi[32], output.V1, cosBit); + step.V1 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V0, -cospi[32], output.V1, cosBit); + step.V2 = Av1Transform1dMath.HalfButterfly(cospi[48], output.V2, -cospi[16], output.V3, cosBit); + step.V3 = Av1Transform1dMath.HalfButterfly(cospi[16], output.V2, cospi[48], output.V3, cosBit); + step.V4 = Av1Transform1dMath.Clamp(output.V4 + output.V5, range); + step.V5 = Av1Transform1dMath.Clamp(output.V4 - output.V5, range); + step.V6 = Av1Transform1dMath.Clamp(output.V7 - output.V6, range); + step.V7 = Av1Transform1dMath.Clamp(output.V6 + output.V7, range); + step.V8 = output.V8; + step.V9 = Av1Transform1dMath.HalfButterfly(-cospi[16], output.V9, cospi[48], output.V14, cosBit); + step.V10 = Av1Transform1dMath.HalfButterfly(-cospi[48], output.V10, -cospi[16], output.V13, cosBit); + step.V11 = output.V11; + step.V12 = output.V12; + step.V13 = Av1Transform1dMath.HalfButterfly(-cospi[16], output.V10, cospi[48], output.V13, cosBit); + step.V14 = Av1Transform1dMath.HalfButterfly(cospi[48], output.V9, cospi[16], output.V14, cosBit); + step.V15 = output.V15; + + // Stage 5 widens the reconstructed groups through their next butterfly level. + stage++; + range = stageRange[stage]; + output.V0 = Av1Transform1dMath.Clamp(step.V0 + step.V3, range); + output.V1 = Av1Transform1dMath.Clamp(step.V1 + step.V2, range); + output.V2 = Av1Transform1dMath.Clamp(step.V1 - step.V2, range); + output.V3 = Av1Transform1dMath.Clamp(step.V0 - step.V3, range); + output.V4 = step.V4; + output.V5 = Av1Transform1dMath.HalfButterfly(-cospi[32], step.V5, cospi[32], step.V6, cosBit); + output.V6 = Av1Transform1dMath.HalfButterfly(cospi[32], step.V5, cospi[32], step.V6, cosBit); + output.V7 = step.V7; + output.V8 = Av1Transform1dMath.Clamp(step.V8 + step.V11, range); + output.V9 = Av1Transform1dMath.Clamp(step.V9 + step.V10, range); + output.V10 = Av1Transform1dMath.Clamp(step.V9 - step.V10, range); + output.V11 = Av1Transform1dMath.Clamp(step.V8 - step.V11, range); + output.V12 = Av1Transform1dMath.Clamp(step.V15 - step.V12, range); + output.V13 = Av1Transform1dMath.Clamp(step.V14 - step.V13, range); + output.V14 = Av1Transform1dMath.Clamp(step.V13 + step.V14, range); + output.V15 = Av1Transform1dMath.Clamp(step.V12 + step.V15, range); + + // Stage 6 applies the remaining pi/4 rotations before the terminal spatial merge. + stage++; + range = stageRange[stage]; + step.V0 = Av1Transform1dMath.Clamp(output.V0 + output.V7, range); + step.V1 = Av1Transform1dMath.Clamp(output.V1 + output.V6, range); + step.V2 = Av1Transform1dMath.Clamp(output.V2 + output.V5, range); + step.V3 = Av1Transform1dMath.Clamp(output.V3 + output.V4, range); + step.V4 = Av1Transform1dMath.Clamp(output.V3 - output.V4, range); + step.V5 = Av1Transform1dMath.Clamp(output.V2 - output.V5, range); + step.V6 = Av1Transform1dMath.Clamp(output.V1 - output.V6, range); + step.V7 = Av1Transform1dMath.Clamp(output.V0 - output.V7, range); + step.V8 = output.V8; + step.V9 = output.V9; + step.V10 = Av1Transform1dMath.HalfButterfly(-cospi[32], output.V10, cospi[32], output.V13, cosBit); + step.V11 = Av1Transform1dMath.HalfButterfly(-cospi[32], output.V11, cospi[32], output.V12, cosBit); + step.V12 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V11, cospi[32], output.V12, cosBit); + step.V13 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V10, cospi[32], output.V13, cosBit); + step.V14 = output.V14; + step.V15 = output.V15; + + // Stage 7 merges the even and odd halves into spatial order and clamps every result. + stage++; + range = stageRange[stage]; + output.V0 = Av1Transform1dMath.Clamp(step.V0 + step.V15, range); + output.V1 = Av1Transform1dMath.Clamp(step.V1 + step.V14, range); + output.V2 = Av1Transform1dMath.Clamp(step.V2 + step.V13, range); + output.V3 = Av1Transform1dMath.Clamp(step.V3 + step.V12, range); + output.V4 = Av1Transform1dMath.Clamp(step.V4 + step.V11, range); + output.V5 = Av1Transform1dMath.Clamp(step.V5 + step.V10, range); + output.V6 = Av1Transform1dMath.Clamp(step.V6 + step.V9, range); + output.V7 = Av1Transform1dMath.Clamp(step.V7 + step.V8, range); + output.V8 = Av1Transform1dMath.Clamp(step.V7 - step.V8, range); + output.V9 = Av1Transform1dMath.Clamp(step.V6 - step.V9, range); + output.V10 = Av1Transform1dMath.Clamp(step.V5 - step.V10, range); + output.V11 = Av1Transform1dMath.Clamp(step.V4 - step.V11, range); + output.V12 = Av1Transform1dMath.Clamp(step.V3 - step.V12, range); + output.V13 = Av1Transform1dMath.Clamp(step.V2 - step.V13, range); + output.V14 = Av1Transform1dMath.Clamp(step.V1 - step.V14, range); + output.V15 = Av1Transform1dMath.Clamp(step.V0 - step.V15, range); + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Transform/Inverse/Av1Inverse2dTransformer.Dct32Operator.cs b/src/ImageSharp/Formats/Heif/Av1/Transform/Inverse/Av1Inverse2dTransformer.Dct32Operator.cs new file mode 100644 index 0000000000..9808f02bf6 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Transform/Inverse/Av1Inverse2dTransformer.Dct32Operator.cs @@ -0,0 +1,1030 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +/// +/// Defines the 32-point AV1 inverse discrete cosine transform operator. +/// +/// +/// Vector fields represent transform positions and vector lanes represent independent axes. The SIMD overloads apply +/// the same staged butterflies, fixed-point rounding, and range clamps as the scalar overload without mixing axes. +/// +internal static partial class Av1Inverse2dTransformer +{ + internal readonly struct Dct32Operator : IAv1Transform1dOperator + { + /// + /// Applies the normative 32-point AV1 inverse discrete cosine transform. + /// + /// The 32 frequency-domain coefficients. + /// The 32 spatial-domain residual values. + /// The 32-element stage buffer owned by the containing two-dimensional transform. + /// The fixed-point precision of the cosine constants. + /// The signed-bit range assigned to each transform stage. + public static void Transform(ReadOnlySpan input, Span output, Span step, int cosBit, InlineArray12 stageRange) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + int stage = 0; + + // Stage 1 permutes frequency-ordered coefficients into the recursive DCT factorization order. + stage++; + output[0] = input[0]; + output[1] = input[16]; + output[2] = input[8]; + output[3] = input[24]; + output[4] = input[4]; + output[5] = input[20]; + output[6] = input[12]; + output[7] = input[28]; + output[8] = input[2]; + output[9] = input[18]; + output[10] = input[10]; + output[11] = input[26]; + output[12] = input[6]; + output[13] = input[22]; + output[14] = input[14]; + output[15] = input[30]; + output[16] = input[1]; + output[17] = input[17]; + output[18] = input[9]; + output[19] = input[25]; + output[20] = input[5]; + output[21] = input[21]; + output[22] = input[13]; + output[23] = input[29]; + output[24] = input[3]; + output[25] = input[19]; + output[26] = input[11]; + output[27] = input[27]; + output[28] = input[7]; + output[29] = input[23]; + output[30] = input[15]; + output[31] = input[31]; + + // Stage 2 rotates the highest odd-frequency coefficient pairs by their pi/64 angles. + stage++; + step[0] = output[0]; + step[1] = output[1]; + step[2] = output[2]; + step[3] = output[3]; + step[4] = output[4]; + step[5] = output[5]; + step[6] = output[6]; + step[7] = output[7]; + step[8] = output[8]; + step[9] = output[9]; + step[10] = output[10]; + step[11] = output[11]; + step[12] = output[12]; + step[13] = output[13]; + step[14] = output[14]; + step[15] = output[15]; + step[16] = Av1Transform1dMath.HalfButterfly(cospi[62], output[16], -cospi[2], output[31], cosBit); + step[17] = Av1Transform1dMath.HalfButterfly(cospi[30], output[17], -cospi[34], output[30], cosBit); + step[18] = Av1Transform1dMath.HalfButterfly(cospi[46], output[18], -cospi[18], output[29], cosBit); + step[19] = Av1Transform1dMath.HalfButterfly(cospi[14], output[19], -cospi[50], output[28], cosBit); + step[20] = Av1Transform1dMath.HalfButterfly(cospi[54], output[20], -cospi[10], output[27], cosBit); + step[21] = Av1Transform1dMath.HalfButterfly(cospi[22], output[21], -cospi[42], output[26], cosBit); + step[22] = Av1Transform1dMath.HalfButterfly(cospi[38], output[22], -cospi[26], output[25], cosBit); + step[23] = Av1Transform1dMath.HalfButterfly(cospi[6], output[23], -cospi[58], output[24], cosBit); + step[24] = Av1Transform1dMath.HalfButterfly(cospi[58], output[23], cospi[6], output[24], cosBit); + step[25] = Av1Transform1dMath.HalfButterfly(cospi[26], output[22], cospi[38], output[25], cosBit); + step[26] = Av1Transform1dMath.HalfButterfly(cospi[42], output[21], cospi[22], output[26], cosBit); + step[27] = Av1Transform1dMath.HalfButterfly(cospi[10], output[20], cospi[54], output[27], cosBit); + step[28] = Av1Transform1dMath.HalfButterfly(cospi[50], output[19], cospi[14], output[28], cosBit); + step[29] = Av1Transform1dMath.HalfButterfly(cospi[18], output[18], cospi[46], output[29], cosBit); + step[30] = Av1Transform1dMath.HalfButterfly(cospi[34], output[17], cospi[30], output[30], cosBit); + step[31] = Av1Transform1dMath.HalfButterfly(cospi[2], output[16], cospi[62], output[31], cosBit); + + // Stage 3 reconstructs the first nested groups and combines their adjacent odd terms. + stage++; + byte range = stageRange[stage]; + output[0] = step[0]; + output[1] = step[1]; + output[2] = step[2]; + output[3] = step[3]; + output[4] = step[4]; + output[5] = step[5]; + output[6] = step[6]; + output[7] = step[7]; + output[8] = Av1Transform1dMath.HalfButterfly(cospi[60], step[8], -cospi[4], step[15], cosBit); + output[9] = Av1Transform1dMath.HalfButterfly(cospi[28], step[9], -cospi[36], step[14], cosBit); + output[10] = Av1Transform1dMath.HalfButterfly(cospi[44], step[10], -cospi[20], step[13], cosBit); + output[11] = Av1Transform1dMath.HalfButterfly(cospi[12], step[11], -cospi[52], step[12], cosBit); + output[12] = Av1Transform1dMath.HalfButterfly(cospi[52], step[11], cospi[12], step[12], cosBit); + output[13] = Av1Transform1dMath.HalfButterfly(cospi[20], step[10], cospi[44], step[13], cosBit); + output[14] = Av1Transform1dMath.HalfButterfly(cospi[36], step[9], cospi[28], step[14], cosBit); + output[15] = Av1Transform1dMath.HalfButterfly(cospi[4], step[8], cospi[60], step[15], cosBit); + output[16] = Av1Transform1dMath.Clamp(step[16] + step[17], range); + output[17] = Av1Transform1dMath.Clamp(step[16] - step[17], range); + output[18] = Av1Transform1dMath.Clamp(-step[18] + step[19], range); + output[19] = Av1Transform1dMath.Clamp(step[18] + step[19], range); + output[20] = Av1Transform1dMath.Clamp(step[20] + step[21], range); + output[21] = Av1Transform1dMath.Clamp(step[20] - step[21], range); + output[22] = Av1Transform1dMath.Clamp(-step[22] + step[23], range); + output[23] = Av1Transform1dMath.Clamp(step[22] + step[23], range); + output[24] = Av1Transform1dMath.Clamp(step[24] + step[25], range); + output[25] = Av1Transform1dMath.Clamp(step[24] - step[25], range); + output[26] = Av1Transform1dMath.Clamp(-step[26] + step[27], range); + output[27] = Av1Transform1dMath.Clamp(step[26] + step[27], range); + output[28] = Av1Transform1dMath.Clamp(step[28] + step[29], range); + output[29] = Av1Transform1dMath.Clamp(step[28] - step[29], range); + output[30] = Av1Transform1dMath.Clamp(-step[30] + step[31], range); + output[31] = Av1Transform1dMath.Clamp(step[30] + step[31], range); + + // Stage 4 rotates the next odd-frequency level while preserving completed low-frequency lanes. + stage++; + range = stageRange[stage]; + step[0] = output[0]; + step[1] = output[1]; + step[2] = output[2]; + step[3] = output[3]; + step[4] = Av1Transform1dMath.HalfButterfly(cospi[56], output[4], -cospi[8], output[7], cosBit); + step[5] = Av1Transform1dMath.HalfButterfly(cospi[24], output[5], -cospi[40], output[6], cosBit); + step[6] = Av1Transform1dMath.HalfButterfly(cospi[40], output[5], cospi[24], output[6], cosBit); + step[7] = Av1Transform1dMath.HalfButterfly(cospi[8], output[4], cospi[56], step[7], cosBit); + step[8] = Av1Transform1dMath.Clamp(output[8] + output[9], range); + step[9] = Av1Transform1dMath.Clamp(output[8] - output[9], range); + step[10] = Av1Transform1dMath.Clamp(-output[10] + output[11], range); + step[11] = Av1Transform1dMath.Clamp(output[10] + output[11], range); + step[12] = Av1Transform1dMath.Clamp(output[12] + output[13], range); + step[13] = Av1Transform1dMath.Clamp(output[12] - output[13], range); + step[14] = Av1Transform1dMath.Clamp(-output[14] + output[15], range); + step[15] = Av1Transform1dMath.Clamp(output[14] + output[15], range); + step[16] = output[16]; + step[17] = Av1Transform1dMath.HalfButterfly(-cospi[8], output[17], cospi[56], output[30], cosBit); + step[18] = Av1Transform1dMath.HalfButterfly(-cospi[56], output[18], -cospi[8], output[29], cosBit); + step[19] = output[19]; + step[20] = output[20]; + step[21] = Av1Transform1dMath.HalfButterfly(-cospi[40], output[21], cospi[24], output[26], cosBit); + step[22] = Av1Transform1dMath.HalfButterfly(-cospi[24], output[22], -cospi[40], output[25], cosBit); + step[23] = output[23]; + step[24] = output[24]; + step[25] = Av1Transform1dMath.HalfButterfly(-cospi[40], output[22], cospi[24], output[25], cosBit); + step[26] = Av1Transform1dMath.HalfButterfly(cospi[24], output[21], cospi[40], output[26], cosBit); + step[27] = output[27]; + step[28] = output[28]; + step[29] = Av1Transform1dMath.HalfButterfly(-cospi[8], output[18], cospi[56], output[29], cosBit); + step[30] = Av1Transform1dMath.HalfButterfly(cospi[56], output[17], cospi[8], output[30], cosBit); + step[31] = output[31]; + + // Stage 5 reconstructs the embedded eight-point groups and combines adjacent odd terms. + stage++; + range = stageRange[stage]; + output[0] = Av1Transform1dMath.HalfButterfly(cospi[32], step[0], cospi[32], step[1], cosBit); + output[1] = Av1Transform1dMath.HalfButterfly(cospi[32], step[0], -cospi[32], step[1], cosBit); + output[2] = Av1Transform1dMath.HalfButterfly(cospi[48], step[2], -cospi[16], step[3], cosBit); + output[3] = Av1Transform1dMath.HalfButterfly(cospi[16], step[2], cospi[48], step[3], cosBit); + output[4] = Av1Transform1dMath.Clamp(step[4] + step[5], range); + output[5] = Av1Transform1dMath.Clamp(step[4] - step[5], range); + output[6] = Av1Transform1dMath.Clamp(-step[6] + step[7], range); + output[7] = Av1Transform1dMath.Clamp(step[6] + step[7], range); + output[8] = step[8]; + output[9] = Av1Transform1dMath.HalfButterfly(-cospi[16], step[9], cospi[48], step[14], cosBit); + output[10] = Av1Transform1dMath.HalfButterfly(-cospi[48], step[10], -cospi[16], step[13], cosBit); + output[11] = step[11]; + output[12] = step[12]; + output[13] = Av1Transform1dMath.HalfButterfly(-cospi[16], step[10], cospi[48], step[13], cosBit); + output[14] = Av1Transform1dMath.HalfButterfly(cospi[48], step[9], cospi[16], step[14], cosBit); + output[15] = step[15]; + output[16] = Av1Transform1dMath.Clamp(step[16] + step[19], range); + output[17] = Av1Transform1dMath.Clamp(step[17] + step[18], range); + output[18] = Av1Transform1dMath.Clamp(step[17] - step[18], range); + output[19] = Av1Transform1dMath.Clamp(step[16] - step[19], range); + output[20] = Av1Transform1dMath.Clamp(-step[20] + step[23], range); + output[21] = Av1Transform1dMath.Clamp(-step[21] + step[22], range); + output[22] = Av1Transform1dMath.Clamp(step[21] + step[22], range); + output[23] = Av1Transform1dMath.Clamp(step[20] + step[23], range); + output[24] = Av1Transform1dMath.Clamp(step[24] + step[27], range); + output[25] = Av1Transform1dMath.Clamp(step[25] + step[26], range); + output[26] = Av1Transform1dMath.Clamp(step[25] - step[26], range); + output[27] = Av1Transform1dMath.Clamp(step[24] - step[27], range); + output[28] = Av1Transform1dMath.Clamp(-step[28] + step[31], range); + output[29] = Av1Transform1dMath.Clamp(-step[29] + step[30], range); + output[30] = Av1Transform1dMath.Clamp(step[29] + step[30], range); + output[31] = Av1Transform1dMath.Clamp(step[28] + step[31], range); + + // Stage 6 completes the low-frequency four-point DCT and rotates the next odd-frequency pairs. + stage++; + range = stageRange[stage]; + step[0] = Av1Transform1dMath.Clamp(output[0] + output[3], range); + step[1] = Av1Transform1dMath.Clamp(output[1] + output[2], range); + step[2] = Av1Transform1dMath.Clamp(output[1] - output[2], range); + step[3] = Av1Transform1dMath.Clamp(output[0] - output[3], range); + step[4] = output[4]; + step[5] = Av1Transform1dMath.HalfButterfly(-cospi[32], output[5], cospi[32], output[6], cosBit); + step[6] = Av1Transform1dMath.HalfButterfly(cospi[32], output[5], cospi[32], output[6], cosBit); + step[7] = output[7]; + step[8] = Av1Transform1dMath.Clamp(output[8] + output[11], range); + step[9] = Av1Transform1dMath.Clamp(output[9] + output[10], range); + step[10] = Av1Transform1dMath.Clamp(output[9] - output[10], range); + step[11] = Av1Transform1dMath.Clamp(output[8] - output[11], range); + step[12] = Av1Transform1dMath.Clamp(-output[12] + output[15], range); + step[13] = Av1Transform1dMath.Clamp(-output[13] + output[14], range); + step[14] = Av1Transform1dMath.Clamp(output[13] + output[14], range); + step[15] = Av1Transform1dMath.Clamp(output[12] + output[15], range); + step[16] = output[16]; + step[17] = output[17]; + step[18] = Av1Transform1dMath.HalfButterfly(-cospi[16], output[18], cospi[48], output[29], cosBit); + step[19] = Av1Transform1dMath.HalfButterfly(-cospi[16], output[19], cospi[48], output[28], cosBit); + step[20] = Av1Transform1dMath.HalfButterfly(-cospi[48], output[20], -cospi[16], output[27], cosBit); + step[21] = Av1Transform1dMath.HalfButterfly(-cospi[48], output[21], -cospi[16], output[26], cosBit); + step[22] = output[22]; + step[23] = output[23]; + step[24] = output[24]; + step[25] = output[25]; + step[26] = Av1Transform1dMath.HalfButterfly(-cospi[16], output[21], cospi[48], output[26], cosBit); + step[27] = Av1Transform1dMath.HalfButterfly(-cospi[16], output[20], cospi[48], output[27], cosBit); + step[28] = Av1Transform1dMath.HalfButterfly(cospi[48], output[19], cospi[16], output[28], cosBit); + step[29] = Av1Transform1dMath.HalfButterfly(cospi[48], output[18], cospi[16], output[29], cosBit); + step[30] = output[30]; + step[31] = output[31]; + + // Stage 7 widens the reconstructed groups through their next butterfly level. + stage++; + range = stageRange[stage]; + output[0] = Av1Transform1dMath.Clamp(step[0] + step[7], range); + output[1] = Av1Transform1dMath.Clamp(step[1] + step[6], range); + output[2] = Av1Transform1dMath.Clamp(step[2] + step[5], range); + output[3] = Av1Transform1dMath.Clamp(step[3] + step[4], range); + output[4] = Av1Transform1dMath.Clamp(step[3] - step[4], range); + output[5] = Av1Transform1dMath.Clamp(step[2] - step[5], range); + output[6] = Av1Transform1dMath.Clamp(step[1] - step[6], range); + output[7] = Av1Transform1dMath.Clamp(step[0] - step[7], range); + output[8] = step[8]; + output[9] = step[9]; + output[10] = Av1Transform1dMath.HalfButterfly(-cospi[32], step[10], cospi[32], step[13], cosBit); + output[11] = Av1Transform1dMath.HalfButterfly(-cospi[32], step[11], cospi[32], step[12], cosBit); + output[12] = Av1Transform1dMath.HalfButterfly(cospi[32], step[11], cospi[32], step[12], cosBit); + output[13] = Av1Transform1dMath.HalfButterfly(cospi[32], step[10], cospi[32], step[13], cosBit); + output[14] = step[14]; + output[15] = step[15]; + output[16] = Av1Transform1dMath.Clamp(step[16] + step[23], range); + output[17] = Av1Transform1dMath.Clamp(step[17] + step[22], range); + output[18] = Av1Transform1dMath.Clamp(step[18] + step[21], range); + output[19] = Av1Transform1dMath.Clamp(step[19] + step[20], range); + output[20] = Av1Transform1dMath.Clamp(step[19] - step[20], range); + output[21] = Av1Transform1dMath.Clamp(step[18] - step[21], range); + output[22] = Av1Transform1dMath.Clamp(step[17] - step[22], range); + output[23] = Av1Transform1dMath.Clamp(step[16] - step[23], range); + output[24] = Av1Transform1dMath.Clamp(-step[24] + step[31], range); + output[25] = Av1Transform1dMath.Clamp(-step[25] + step[30], range); + output[26] = Av1Transform1dMath.Clamp(-step[26] + step[29], range); + output[27] = Av1Transform1dMath.Clamp(-step[27] + step[28], range); + output[28] = Av1Transform1dMath.Clamp(step[27] + step[28], range); + output[29] = Av1Transform1dMath.Clamp(step[26] + step[29], range); + output[30] = Av1Transform1dMath.Clamp(step[25] + step[30], range); + output[31] = Av1Transform1dMath.Clamp(step[24] + step[31], range); + + // Stage 8 applies the remaining pi/4 rotations before the terminal spatial merge. + stage++; + range = stageRange[stage]; + step[0] = Av1Transform1dMath.Clamp(output[0] + output[15], range); + step[1] = Av1Transform1dMath.Clamp(output[1] + output[14], range); + step[2] = Av1Transform1dMath.Clamp(output[2] + output[13], range); + step[3] = Av1Transform1dMath.Clamp(output[3] + output[12], range); + step[4] = Av1Transform1dMath.Clamp(output[4] + output[11], range); + step[5] = Av1Transform1dMath.Clamp(output[5] + output[10], range); + step[6] = Av1Transform1dMath.Clamp(output[6] + output[9], range); + step[7] = Av1Transform1dMath.Clamp(output[7] + output[8], range); + step[8] = Av1Transform1dMath.Clamp(output[7] - output[8], range); + step[9] = Av1Transform1dMath.Clamp(output[6] - output[9], range); + step[10] = Av1Transform1dMath.Clamp(output[5] - output[10], range); + step[11] = Av1Transform1dMath.Clamp(output[4] - output[11], range); + step[12] = Av1Transform1dMath.Clamp(output[3] - output[12], range); + step[13] = Av1Transform1dMath.Clamp(output[2] - output[13], range); + step[14] = Av1Transform1dMath.Clamp(output[1] - output[14], range); + step[15] = Av1Transform1dMath.Clamp(output[0] - output[15], range); + step[16] = output[16]; + step[17] = output[17]; + step[18] = output[18]; + step[19] = output[19]; + step[20] = Av1Transform1dMath.HalfButterfly(-cospi[32], output[20], cospi[32], output[27], cosBit); + step[21] = Av1Transform1dMath.HalfButterfly(-cospi[32], output[21], cospi[32], output[26], cosBit); + step[22] = Av1Transform1dMath.HalfButterfly(-cospi[32], output[22], cospi[32], output[25], cosBit); + step[23] = Av1Transform1dMath.HalfButterfly(-cospi[32], output[23], cospi[32], output[24], cosBit); + step[24] = Av1Transform1dMath.HalfButterfly(cospi[32], output[23], cospi[32], output[24], cosBit); + step[25] = Av1Transform1dMath.HalfButterfly(cospi[32], output[22], cospi[32], output[25], cosBit); + step[26] = Av1Transform1dMath.HalfButterfly(cospi[32], output[21], cospi[32], output[26], cosBit); + step[27] = Av1Transform1dMath.HalfButterfly(cospi[32], output[20], cospi[32], output[27], cosBit); + step[28] = output[28]; + step[29] = output[29]; + step[30] = output[30]; + step[31] = output[31]; + + // Stage 9 merges the even and odd halves into spatial order and clamps every result. + stage++; + range = stageRange[stage]; + output[0] = Av1Transform1dMath.Clamp(step[0] + step[31], range); + output[1] = Av1Transform1dMath.Clamp(step[1] + step[30], range); + output[2] = Av1Transform1dMath.Clamp(step[2] + step[29], range); + output[3] = Av1Transform1dMath.Clamp(step[3] + step[28], range); + output[4] = Av1Transform1dMath.Clamp(step[4] + step[27], range); + output[5] = Av1Transform1dMath.Clamp(step[5] + step[26], range); + output[6] = Av1Transform1dMath.Clamp(step[6] + step[25], range); + output[7] = Av1Transform1dMath.Clamp(step[7] + step[24], range); + output[8] = Av1Transform1dMath.Clamp(step[8] + step[23], range); + output[9] = Av1Transform1dMath.Clamp(step[9] + step[22], range); + output[10] = Av1Transform1dMath.Clamp(step[10] + step[21], range); + output[11] = Av1Transform1dMath.Clamp(step[11] + step[20], range); + output[12] = Av1Transform1dMath.Clamp(step[12] + step[19], range); + output[13] = Av1Transform1dMath.Clamp(step[13] + step[18], range); + output[14] = Av1Transform1dMath.Clamp(step[14] + step[17], range); + output[15] = Av1Transform1dMath.Clamp(step[15] + step[16], range); + output[16] = Av1Transform1dMath.Clamp(step[15] - step[16], range); + output[17] = Av1Transform1dMath.Clamp(step[14] - step[17], range); + output[18] = Av1Transform1dMath.Clamp(step[13] - step[18], range); + output[19] = Av1Transform1dMath.Clamp(step[12] - step[19], range); + output[20] = Av1Transform1dMath.Clamp(step[11] - step[20], range); + output[21] = Av1Transform1dMath.Clamp(step[10] - step[21], range); + output[22] = Av1Transform1dMath.Clamp(step[9] - step[22], range); + output[23] = Av1Transform1dMath.Clamp(step[8] - step[23], range); + output[24] = Av1Transform1dMath.Clamp(step[7] - step[24], range); + output[25] = Av1Transform1dMath.Clamp(step[6] - step[25], range); + output[26] = Av1Transform1dMath.Clamp(step[5] - step[26], range); + output[27] = Av1Transform1dMath.Clamp(step[4] - step[27], range); + output[28] = Av1Transform1dMath.Clamp(step[3] - step[28], range); + output[29] = Av1Transform1dMath.Clamp(step[2] - step[29], range); + output[30] = Av1Transform1dMath.Clamp(step[1] - step[30], range); + output[31] = Av1Transform1dMath.Clamp(step[0] - step[31], range); + } + + /// + public static void Transform( + ref Av1TransformVector> input, + ref Av1TransformVector> output, + ref Av1TransformVector> step, + int cosBit, + InlineArray12 stageRange) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + int stage = 0; + + // Stage 1 permutes frequency-ordered coefficients into the recursive DCT factorization order. + stage++; + output.V0 = input.V0; + output.V1 = input.V16; + output.V2 = input.V8; + output.V3 = input.V24; + output.V4 = input.V4; + output.V5 = input.V20; + output.V6 = input.V12; + output.V7 = input.V28; + output.V8 = input.V2; + output.V9 = input.V18; + output.V10 = input.V10; + output.V11 = input.V26; + output.V12 = input.V6; + output.V13 = input.V22; + output.V14 = input.V14; + output.V15 = input.V30; + output.V16 = input.V1; + output.V17 = input.V17; + output.V18 = input.V9; + output.V19 = input.V25; + output.V20 = input.V5; + output.V21 = input.V21; + output.V22 = input.V13; + output.V23 = input.V29; + output.V24 = input.V3; + output.V25 = input.V19; + output.V26 = input.V11; + output.V27 = input.V27; + output.V28 = input.V7; + output.V29 = input.V23; + output.V30 = input.V15; + output.V31 = input.V31; + + // Stage 2 rotates the highest odd-frequency coefficient pairs by their pi/64 angles. + stage++; + step.V0 = output.V0; + step.V1 = output.V1; + step.V2 = output.V2; + step.V3 = output.V3; + step.V4 = output.V4; + step.V5 = output.V5; + step.V6 = output.V6; + step.V7 = output.V7; + step.V8 = output.V8; + step.V9 = output.V9; + step.V10 = output.V10; + step.V11 = output.V11; + step.V12 = output.V12; + step.V13 = output.V13; + step.V14 = output.V14; + step.V15 = output.V15; + step.V16 = Av1Transform1dMath.HalfButterfly(cospi[62], output.V16, -cospi[2], output.V31, cosBit); + step.V17 = Av1Transform1dMath.HalfButterfly(cospi[30], output.V17, -cospi[34], output.V30, cosBit); + step.V18 = Av1Transform1dMath.HalfButterfly(cospi[46], output.V18, -cospi[18], output.V29, cosBit); + step.V19 = Av1Transform1dMath.HalfButterfly(cospi[14], output.V19, -cospi[50], output.V28, cosBit); + step.V20 = Av1Transform1dMath.HalfButterfly(cospi[54], output.V20, -cospi[10], output.V27, cosBit); + step.V21 = Av1Transform1dMath.HalfButterfly(cospi[22], output.V21, -cospi[42], output.V26, cosBit); + step.V22 = Av1Transform1dMath.HalfButterfly(cospi[38], output.V22, -cospi[26], output.V25, cosBit); + step.V23 = Av1Transform1dMath.HalfButterfly(cospi[6], output.V23, -cospi[58], output.V24, cosBit); + step.V24 = Av1Transform1dMath.HalfButterfly(cospi[58], output.V23, cospi[6], output.V24, cosBit); + step.V25 = Av1Transform1dMath.HalfButterfly(cospi[26], output.V22, cospi[38], output.V25, cosBit); + step.V26 = Av1Transform1dMath.HalfButterfly(cospi[42], output.V21, cospi[22], output.V26, cosBit); + step.V27 = Av1Transform1dMath.HalfButterfly(cospi[10], output.V20, cospi[54], output.V27, cosBit); + step.V28 = Av1Transform1dMath.HalfButterfly(cospi[50], output.V19, cospi[14], output.V28, cosBit); + step.V29 = Av1Transform1dMath.HalfButterfly(cospi[18], output.V18, cospi[46], output.V29, cosBit); + step.V30 = Av1Transform1dMath.HalfButterfly(cospi[34], output.V17, cospi[30], output.V30, cosBit); + step.V31 = Av1Transform1dMath.HalfButterfly(cospi[2], output.V16, cospi[62], output.V31, cosBit); + + // Stage 3 reconstructs the first nested groups and combines their adjacent odd terms. + stage++; + byte range = stageRange[stage]; + output.V0 = step.V0; + output.V1 = step.V1; + output.V2 = step.V2; + output.V3 = step.V3; + output.V4 = step.V4; + output.V5 = step.V5; + output.V6 = step.V6; + output.V7 = step.V7; + output.V8 = Av1Transform1dMath.HalfButterfly(cospi[60], step.V8, -cospi[4], step.V15, cosBit); + output.V9 = Av1Transform1dMath.HalfButterfly(cospi[28], step.V9, -cospi[36], step.V14, cosBit); + output.V10 = Av1Transform1dMath.HalfButterfly(cospi[44], step.V10, -cospi[20], step.V13, cosBit); + output.V11 = Av1Transform1dMath.HalfButterfly(cospi[12], step.V11, -cospi[52], step.V12, cosBit); + output.V12 = Av1Transform1dMath.HalfButterfly(cospi[52], step.V11, cospi[12], step.V12, cosBit); + output.V13 = Av1Transform1dMath.HalfButterfly(cospi[20], step.V10, cospi[44], step.V13, cosBit); + output.V14 = Av1Transform1dMath.HalfButterfly(cospi[36], step.V9, cospi[28], step.V14, cosBit); + output.V15 = Av1Transform1dMath.HalfButterfly(cospi[4], step.V8, cospi[60], step.V15, cosBit); + output.V16 = Av1Transform1dMath.Clamp(step.V16 + step.V17, range); + output.V17 = Av1Transform1dMath.Clamp(step.V16 - step.V17, range); + output.V18 = Av1Transform1dMath.Clamp(-step.V18 + step.V19, range); + output.V19 = Av1Transform1dMath.Clamp(step.V18 + step.V19, range); + output.V20 = Av1Transform1dMath.Clamp(step.V20 + step.V21, range); + output.V21 = Av1Transform1dMath.Clamp(step.V20 - step.V21, range); + output.V22 = Av1Transform1dMath.Clamp(-step.V22 + step.V23, range); + output.V23 = Av1Transform1dMath.Clamp(step.V22 + step.V23, range); + output.V24 = Av1Transform1dMath.Clamp(step.V24 + step.V25, range); + output.V25 = Av1Transform1dMath.Clamp(step.V24 - step.V25, range); + output.V26 = Av1Transform1dMath.Clamp(-step.V26 + step.V27, range); + output.V27 = Av1Transform1dMath.Clamp(step.V26 + step.V27, range); + output.V28 = Av1Transform1dMath.Clamp(step.V28 + step.V29, range); + output.V29 = Av1Transform1dMath.Clamp(step.V28 - step.V29, range); + output.V30 = Av1Transform1dMath.Clamp(-step.V30 + step.V31, range); + output.V31 = Av1Transform1dMath.Clamp(step.V30 + step.V31, range); + + // Stage 4 rotates the next odd-frequency level while preserving completed low-frequency lanes. + stage++; + range = stageRange[stage]; + step.V0 = output.V0; + step.V1 = output.V1; + step.V2 = output.V2; + step.V3 = output.V3; + step.V4 = Av1Transform1dMath.HalfButterfly(cospi[56], output.V4, -cospi[8], output.V7, cosBit); + step.V5 = Av1Transform1dMath.HalfButterfly(cospi[24], output.V5, -cospi[40], output.V6, cosBit); + step.V6 = Av1Transform1dMath.HalfButterfly(cospi[40], output.V5, cospi[24], output.V6, cosBit); + step.V7 = Av1Transform1dMath.HalfButterfly(cospi[8], output.V4, cospi[56], step.V7, cosBit); + step.V8 = Av1Transform1dMath.Clamp(output.V8 + output.V9, range); + step.V9 = Av1Transform1dMath.Clamp(output.V8 - output.V9, range); + step.V10 = Av1Transform1dMath.Clamp(-output.V10 + output.V11, range); + step.V11 = Av1Transform1dMath.Clamp(output.V10 + output.V11, range); + step.V12 = Av1Transform1dMath.Clamp(output.V12 + output.V13, range); + step.V13 = Av1Transform1dMath.Clamp(output.V12 - output.V13, range); + step.V14 = Av1Transform1dMath.Clamp(-output.V14 + output.V15, range); + step.V15 = Av1Transform1dMath.Clamp(output.V14 + output.V15, range); + step.V16 = output.V16; + step.V17 = Av1Transform1dMath.HalfButterfly(-cospi[8], output.V17, cospi[56], output.V30, cosBit); + step.V18 = Av1Transform1dMath.HalfButterfly(-cospi[56], output.V18, -cospi[8], output.V29, cosBit); + step.V19 = output.V19; + step.V20 = output.V20; + step.V21 = Av1Transform1dMath.HalfButterfly(-cospi[40], output.V21, cospi[24], output.V26, cosBit); + step.V22 = Av1Transform1dMath.HalfButterfly(-cospi[24], output.V22, -cospi[40], output.V25, cosBit); + step.V23 = output.V23; + step.V24 = output.V24; + step.V25 = Av1Transform1dMath.HalfButterfly(-cospi[40], output.V22, cospi[24], output.V25, cosBit); + step.V26 = Av1Transform1dMath.HalfButterfly(cospi[24], output.V21, cospi[40], output.V26, cosBit); + step.V27 = output.V27; + step.V28 = output.V28; + step.V29 = Av1Transform1dMath.HalfButterfly(-cospi[8], output.V18, cospi[56], output.V29, cosBit); + step.V30 = Av1Transform1dMath.HalfButterfly(cospi[56], output.V17, cospi[8], output.V30, cosBit); + step.V31 = output.V31; + + // Stage 5 reconstructs the embedded eight-point groups and combines adjacent odd terms. + stage++; + range = stageRange[stage]; + output.V0 = Av1Transform1dMath.HalfButterfly(cospi[32], step.V0, cospi[32], step.V1, cosBit); + output.V1 = Av1Transform1dMath.HalfButterfly(cospi[32], step.V0, -cospi[32], step.V1, cosBit); + output.V2 = Av1Transform1dMath.HalfButterfly(cospi[48], step.V2, -cospi[16], step.V3, cosBit); + output.V3 = Av1Transform1dMath.HalfButterfly(cospi[16], step.V2, cospi[48], step.V3, cosBit); + output.V4 = Av1Transform1dMath.Clamp(step.V4 + step.V5, range); + output.V5 = Av1Transform1dMath.Clamp(step.V4 - step.V5, range); + output.V6 = Av1Transform1dMath.Clamp(-step.V6 + step.V7, range); + output.V7 = Av1Transform1dMath.Clamp(step.V6 + step.V7, range); + output.V8 = step.V8; + output.V9 = Av1Transform1dMath.HalfButterfly(-cospi[16], step.V9, cospi[48], step.V14, cosBit); + output.V10 = Av1Transform1dMath.HalfButterfly(-cospi[48], step.V10, -cospi[16], step.V13, cosBit); + output.V11 = step.V11; + output.V12 = step.V12; + output.V13 = Av1Transform1dMath.HalfButterfly(-cospi[16], step.V10, cospi[48], step.V13, cosBit); + output.V14 = Av1Transform1dMath.HalfButterfly(cospi[48], step.V9, cospi[16], step.V14, cosBit); + output.V15 = step.V15; + output.V16 = Av1Transform1dMath.Clamp(step.V16 + step.V19, range); + output.V17 = Av1Transform1dMath.Clamp(step.V17 + step.V18, range); + output.V18 = Av1Transform1dMath.Clamp(step.V17 - step.V18, range); + output.V19 = Av1Transform1dMath.Clamp(step.V16 - step.V19, range); + output.V20 = Av1Transform1dMath.Clamp(-step.V20 + step.V23, range); + output.V21 = Av1Transform1dMath.Clamp(-step.V21 + step.V22, range); + output.V22 = Av1Transform1dMath.Clamp(step.V21 + step.V22, range); + output.V23 = Av1Transform1dMath.Clamp(step.V20 + step.V23, range); + output.V24 = Av1Transform1dMath.Clamp(step.V24 + step.V27, range); + output.V25 = Av1Transform1dMath.Clamp(step.V25 + step.V26, range); + output.V26 = Av1Transform1dMath.Clamp(step.V25 - step.V26, range); + output.V27 = Av1Transform1dMath.Clamp(step.V24 - step.V27, range); + output.V28 = Av1Transform1dMath.Clamp(-step.V28 + step.V31, range); + output.V29 = Av1Transform1dMath.Clamp(-step.V29 + step.V30, range); + output.V30 = Av1Transform1dMath.Clamp(step.V29 + step.V30, range); + output.V31 = Av1Transform1dMath.Clamp(step.V28 + step.V31, range); + + // Stage 6 completes the low-frequency four-point DCT and rotates the next odd-frequency pairs. + stage++; + range = stageRange[stage]; + step.V0 = Av1Transform1dMath.Clamp(output.V0 + output.V3, range); + step.V1 = Av1Transform1dMath.Clamp(output.V1 + output.V2, range); + step.V2 = Av1Transform1dMath.Clamp(output.V1 - output.V2, range); + step.V3 = Av1Transform1dMath.Clamp(output.V0 - output.V3, range); + step.V4 = output.V4; + step.V5 = Av1Transform1dMath.HalfButterfly(-cospi[32], output.V5, cospi[32], output.V6, cosBit); + step.V6 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V5, cospi[32], output.V6, cosBit); + step.V7 = output.V7; + step.V8 = Av1Transform1dMath.Clamp(output.V8 + output.V11, range); + step.V9 = Av1Transform1dMath.Clamp(output.V9 + output.V10, range); + step.V10 = Av1Transform1dMath.Clamp(output.V9 - output.V10, range); + step.V11 = Av1Transform1dMath.Clamp(output.V8 - output.V11, range); + step.V12 = Av1Transform1dMath.Clamp(-output.V12 + output.V15, range); + step.V13 = Av1Transform1dMath.Clamp(-output.V13 + output.V14, range); + step.V14 = Av1Transform1dMath.Clamp(output.V13 + output.V14, range); + step.V15 = Av1Transform1dMath.Clamp(output.V12 + output.V15, range); + step.V16 = output.V16; + step.V17 = output.V17; + step.V18 = Av1Transform1dMath.HalfButterfly(-cospi[16], output.V18, cospi[48], output.V29, cosBit); + step.V19 = Av1Transform1dMath.HalfButterfly(-cospi[16], output.V19, cospi[48], output.V28, cosBit); + step.V20 = Av1Transform1dMath.HalfButterfly(-cospi[48], output.V20, -cospi[16], output.V27, cosBit); + step.V21 = Av1Transform1dMath.HalfButterfly(-cospi[48], output.V21, -cospi[16], output.V26, cosBit); + step.V22 = output.V22; + step.V23 = output.V23; + step.V24 = output.V24; + step.V25 = output.V25; + step.V26 = Av1Transform1dMath.HalfButterfly(-cospi[16], output.V21, cospi[48], output.V26, cosBit); + step.V27 = Av1Transform1dMath.HalfButterfly(-cospi[16], output.V20, cospi[48], output.V27, cosBit); + step.V28 = Av1Transform1dMath.HalfButterfly(cospi[48], output.V19, cospi[16], output.V28, cosBit); + step.V29 = Av1Transform1dMath.HalfButterfly(cospi[48], output.V18, cospi[16], output.V29, cosBit); + step.V30 = output.V30; + step.V31 = output.V31; + + // Stage 7 widens the reconstructed groups through their next butterfly level. + stage++; + range = stageRange[stage]; + output.V0 = Av1Transform1dMath.Clamp(step.V0 + step.V7, range); + output.V1 = Av1Transform1dMath.Clamp(step.V1 + step.V6, range); + output.V2 = Av1Transform1dMath.Clamp(step.V2 + step.V5, range); + output.V3 = Av1Transform1dMath.Clamp(step.V3 + step.V4, range); + output.V4 = Av1Transform1dMath.Clamp(step.V3 - step.V4, range); + output.V5 = Av1Transform1dMath.Clamp(step.V2 - step.V5, range); + output.V6 = Av1Transform1dMath.Clamp(step.V1 - step.V6, range); + output.V7 = Av1Transform1dMath.Clamp(step.V0 - step.V7, range); + output.V8 = step.V8; + output.V9 = step.V9; + output.V10 = Av1Transform1dMath.HalfButterfly(-cospi[32], step.V10, cospi[32], step.V13, cosBit); + output.V11 = Av1Transform1dMath.HalfButterfly(-cospi[32], step.V11, cospi[32], step.V12, cosBit); + output.V12 = Av1Transform1dMath.HalfButterfly(cospi[32], step.V11, cospi[32], step.V12, cosBit); + output.V13 = Av1Transform1dMath.HalfButterfly(cospi[32], step.V10, cospi[32], step.V13, cosBit); + output.V14 = step.V14; + output.V15 = step.V15; + output.V16 = Av1Transform1dMath.Clamp(step.V16 + step.V23, range); + output.V17 = Av1Transform1dMath.Clamp(step.V17 + step.V22, range); + output.V18 = Av1Transform1dMath.Clamp(step.V18 + step.V21, range); + output.V19 = Av1Transform1dMath.Clamp(step.V19 + step.V20, range); + output.V20 = Av1Transform1dMath.Clamp(step.V19 - step.V20, range); + output.V21 = Av1Transform1dMath.Clamp(step.V18 - step.V21, range); + output.V22 = Av1Transform1dMath.Clamp(step.V17 - step.V22, range); + output.V23 = Av1Transform1dMath.Clamp(step.V16 - step.V23, range); + output.V24 = Av1Transform1dMath.Clamp(-step.V24 + step.V31, range); + output.V25 = Av1Transform1dMath.Clamp(-step.V25 + step.V30, range); + output.V26 = Av1Transform1dMath.Clamp(-step.V26 + step.V29, range); + output.V27 = Av1Transform1dMath.Clamp(-step.V27 + step.V28, range); + output.V28 = Av1Transform1dMath.Clamp(step.V27 + step.V28, range); + output.V29 = Av1Transform1dMath.Clamp(step.V26 + step.V29, range); + output.V30 = Av1Transform1dMath.Clamp(step.V25 + step.V30, range); + output.V31 = Av1Transform1dMath.Clamp(step.V24 + step.V31, range); + + // Stage 8 applies the remaining pi/4 rotations before the terminal spatial merge. + stage++; + range = stageRange[stage]; + step.V0 = Av1Transform1dMath.Clamp(output.V0 + output.V15, range); + step.V1 = Av1Transform1dMath.Clamp(output.V1 + output.V14, range); + step.V2 = Av1Transform1dMath.Clamp(output.V2 + output.V13, range); + step.V3 = Av1Transform1dMath.Clamp(output.V3 + output.V12, range); + step.V4 = Av1Transform1dMath.Clamp(output.V4 + output.V11, range); + step.V5 = Av1Transform1dMath.Clamp(output.V5 + output.V10, range); + step.V6 = Av1Transform1dMath.Clamp(output.V6 + output.V9, range); + step.V7 = Av1Transform1dMath.Clamp(output.V7 + output.V8, range); + step.V8 = Av1Transform1dMath.Clamp(output.V7 - output.V8, range); + step.V9 = Av1Transform1dMath.Clamp(output.V6 - output.V9, range); + step.V10 = Av1Transform1dMath.Clamp(output.V5 - output.V10, range); + step.V11 = Av1Transform1dMath.Clamp(output.V4 - output.V11, range); + step.V12 = Av1Transform1dMath.Clamp(output.V3 - output.V12, range); + step.V13 = Av1Transform1dMath.Clamp(output.V2 - output.V13, range); + step.V14 = Av1Transform1dMath.Clamp(output.V1 - output.V14, range); + step.V15 = Av1Transform1dMath.Clamp(output.V0 - output.V15, range); + step.V16 = output.V16; + step.V17 = output.V17; + step.V18 = output.V18; + step.V19 = output.V19; + step.V20 = Av1Transform1dMath.HalfButterfly(-cospi[32], output.V20, cospi[32], output.V27, cosBit); + step.V21 = Av1Transform1dMath.HalfButterfly(-cospi[32], output.V21, cospi[32], output.V26, cosBit); + step.V22 = Av1Transform1dMath.HalfButterfly(-cospi[32], output.V22, cospi[32], output.V25, cosBit); + step.V23 = Av1Transform1dMath.HalfButterfly(-cospi[32], output.V23, cospi[32], output.V24, cosBit); + step.V24 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V23, cospi[32], output.V24, cosBit); + step.V25 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V22, cospi[32], output.V25, cosBit); + step.V26 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V21, cospi[32], output.V26, cosBit); + step.V27 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V20, cospi[32], output.V27, cosBit); + step.V28 = output.V28; + step.V29 = output.V29; + step.V30 = output.V30; + step.V31 = output.V31; + + // Stage 9 merges the even and odd halves into spatial order and clamps every result. + stage++; + range = stageRange[stage]; + output.V0 = Av1Transform1dMath.Clamp(step.V0 + step.V31, range); + output.V1 = Av1Transform1dMath.Clamp(step.V1 + step.V30, range); + output.V2 = Av1Transform1dMath.Clamp(step.V2 + step.V29, range); + output.V3 = Av1Transform1dMath.Clamp(step.V3 + step.V28, range); + output.V4 = Av1Transform1dMath.Clamp(step.V4 + step.V27, range); + output.V5 = Av1Transform1dMath.Clamp(step.V5 + step.V26, range); + output.V6 = Av1Transform1dMath.Clamp(step.V6 + step.V25, range); + output.V7 = Av1Transform1dMath.Clamp(step.V7 + step.V24, range); + output.V8 = Av1Transform1dMath.Clamp(step.V8 + step.V23, range); + output.V9 = Av1Transform1dMath.Clamp(step.V9 + step.V22, range); + output.V10 = Av1Transform1dMath.Clamp(step.V10 + step.V21, range); + output.V11 = Av1Transform1dMath.Clamp(step.V11 + step.V20, range); + output.V12 = Av1Transform1dMath.Clamp(step.V12 + step.V19, range); + output.V13 = Av1Transform1dMath.Clamp(step.V13 + step.V18, range); + output.V14 = Av1Transform1dMath.Clamp(step.V14 + step.V17, range); + output.V15 = Av1Transform1dMath.Clamp(step.V15 + step.V16, range); + output.V16 = Av1Transform1dMath.Clamp(step.V15 - step.V16, range); + output.V17 = Av1Transform1dMath.Clamp(step.V14 - step.V17, range); + output.V18 = Av1Transform1dMath.Clamp(step.V13 - step.V18, range); + output.V19 = Av1Transform1dMath.Clamp(step.V12 - step.V19, range); + output.V20 = Av1Transform1dMath.Clamp(step.V11 - step.V20, range); + output.V21 = Av1Transform1dMath.Clamp(step.V10 - step.V21, range); + output.V22 = Av1Transform1dMath.Clamp(step.V9 - step.V22, range); + output.V23 = Av1Transform1dMath.Clamp(step.V8 - step.V23, range); + output.V24 = Av1Transform1dMath.Clamp(step.V7 - step.V24, range); + output.V25 = Av1Transform1dMath.Clamp(step.V6 - step.V25, range); + output.V26 = Av1Transform1dMath.Clamp(step.V5 - step.V26, range); + output.V27 = Av1Transform1dMath.Clamp(step.V4 - step.V27, range); + output.V28 = Av1Transform1dMath.Clamp(step.V3 - step.V28, range); + output.V29 = Av1Transform1dMath.Clamp(step.V2 - step.V29, range); + output.V30 = Av1Transform1dMath.Clamp(step.V1 - step.V30, range); + output.V31 = Av1Transform1dMath.Clamp(step.V0 - step.V31, range); + } + + /// + /// Applies the transform to four independent axes in parallel. + /// + /// The source values for the parallel transform axes. + /// The destination values for the parallel transform axes. + /// The fixed stage storage for the parallel transform axes. + /// The fixed-point precision of the cosine constants. + /// The signed-bit range assigned to each transform stage. + public static void Transform( + ref Av1TransformVector> input, + ref Av1TransformVector> output, + ref Av1TransformVector> step, + int cosBit, + InlineArray12 stageRange) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + int stage = 0; + + // Stage 1 permutes frequency-ordered coefficients into the recursive DCT factorization order. + stage++; + output.V0 = input.V0; + output.V1 = input.V16; + output.V2 = input.V8; + output.V3 = input.V24; + output.V4 = input.V4; + output.V5 = input.V20; + output.V6 = input.V12; + output.V7 = input.V28; + output.V8 = input.V2; + output.V9 = input.V18; + output.V10 = input.V10; + output.V11 = input.V26; + output.V12 = input.V6; + output.V13 = input.V22; + output.V14 = input.V14; + output.V15 = input.V30; + output.V16 = input.V1; + output.V17 = input.V17; + output.V18 = input.V9; + output.V19 = input.V25; + output.V20 = input.V5; + output.V21 = input.V21; + output.V22 = input.V13; + output.V23 = input.V29; + output.V24 = input.V3; + output.V25 = input.V19; + output.V26 = input.V11; + output.V27 = input.V27; + output.V28 = input.V7; + output.V29 = input.V23; + output.V30 = input.V15; + output.V31 = input.V31; + + // Stage 2 rotates the highest odd-frequency coefficient pairs by their pi/64 angles. + stage++; + step.V0 = output.V0; + step.V1 = output.V1; + step.V2 = output.V2; + step.V3 = output.V3; + step.V4 = output.V4; + step.V5 = output.V5; + step.V6 = output.V6; + step.V7 = output.V7; + step.V8 = output.V8; + step.V9 = output.V9; + step.V10 = output.V10; + step.V11 = output.V11; + step.V12 = output.V12; + step.V13 = output.V13; + step.V14 = output.V14; + step.V15 = output.V15; + step.V16 = Av1Transform1dMath.HalfButterfly(cospi[62], output.V16, -cospi[2], output.V31, cosBit); + step.V17 = Av1Transform1dMath.HalfButterfly(cospi[30], output.V17, -cospi[34], output.V30, cosBit); + step.V18 = Av1Transform1dMath.HalfButterfly(cospi[46], output.V18, -cospi[18], output.V29, cosBit); + step.V19 = Av1Transform1dMath.HalfButterfly(cospi[14], output.V19, -cospi[50], output.V28, cosBit); + step.V20 = Av1Transform1dMath.HalfButterfly(cospi[54], output.V20, -cospi[10], output.V27, cosBit); + step.V21 = Av1Transform1dMath.HalfButterfly(cospi[22], output.V21, -cospi[42], output.V26, cosBit); + step.V22 = Av1Transform1dMath.HalfButterfly(cospi[38], output.V22, -cospi[26], output.V25, cosBit); + step.V23 = Av1Transform1dMath.HalfButterfly(cospi[6], output.V23, -cospi[58], output.V24, cosBit); + step.V24 = Av1Transform1dMath.HalfButterfly(cospi[58], output.V23, cospi[6], output.V24, cosBit); + step.V25 = Av1Transform1dMath.HalfButterfly(cospi[26], output.V22, cospi[38], output.V25, cosBit); + step.V26 = Av1Transform1dMath.HalfButterfly(cospi[42], output.V21, cospi[22], output.V26, cosBit); + step.V27 = Av1Transform1dMath.HalfButterfly(cospi[10], output.V20, cospi[54], output.V27, cosBit); + step.V28 = Av1Transform1dMath.HalfButterfly(cospi[50], output.V19, cospi[14], output.V28, cosBit); + step.V29 = Av1Transform1dMath.HalfButterfly(cospi[18], output.V18, cospi[46], output.V29, cosBit); + step.V30 = Av1Transform1dMath.HalfButterfly(cospi[34], output.V17, cospi[30], output.V30, cosBit); + step.V31 = Av1Transform1dMath.HalfButterfly(cospi[2], output.V16, cospi[62], output.V31, cosBit); + + // Stage 3 reconstructs the first nested groups and combines their adjacent odd terms. + stage++; + byte range = stageRange[stage]; + output.V0 = step.V0; + output.V1 = step.V1; + output.V2 = step.V2; + output.V3 = step.V3; + output.V4 = step.V4; + output.V5 = step.V5; + output.V6 = step.V6; + output.V7 = step.V7; + output.V8 = Av1Transform1dMath.HalfButterfly(cospi[60], step.V8, -cospi[4], step.V15, cosBit); + output.V9 = Av1Transform1dMath.HalfButterfly(cospi[28], step.V9, -cospi[36], step.V14, cosBit); + output.V10 = Av1Transform1dMath.HalfButterfly(cospi[44], step.V10, -cospi[20], step.V13, cosBit); + output.V11 = Av1Transform1dMath.HalfButterfly(cospi[12], step.V11, -cospi[52], step.V12, cosBit); + output.V12 = Av1Transform1dMath.HalfButterfly(cospi[52], step.V11, cospi[12], step.V12, cosBit); + output.V13 = Av1Transform1dMath.HalfButterfly(cospi[20], step.V10, cospi[44], step.V13, cosBit); + output.V14 = Av1Transform1dMath.HalfButterfly(cospi[36], step.V9, cospi[28], step.V14, cosBit); + output.V15 = Av1Transform1dMath.HalfButterfly(cospi[4], step.V8, cospi[60], step.V15, cosBit); + output.V16 = Av1Transform1dMath.Clamp(step.V16 + step.V17, range); + output.V17 = Av1Transform1dMath.Clamp(step.V16 - step.V17, range); + output.V18 = Av1Transform1dMath.Clamp(-step.V18 + step.V19, range); + output.V19 = Av1Transform1dMath.Clamp(step.V18 + step.V19, range); + output.V20 = Av1Transform1dMath.Clamp(step.V20 + step.V21, range); + output.V21 = Av1Transform1dMath.Clamp(step.V20 - step.V21, range); + output.V22 = Av1Transform1dMath.Clamp(-step.V22 + step.V23, range); + output.V23 = Av1Transform1dMath.Clamp(step.V22 + step.V23, range); + output.V24 = Av1Transform1dMath.Clamp(step.V24 + step.V25, range); + output.V25 = Av1Transform1dMath.Clamp(step.V24 - step.V25, range); + output.V26 = Av1Transform1dMath.Clamp(-step.V26 + step.V27, range); + output.V27 = Av1Transform1dMath.Clamp(step.V26 + step.V27, range); + output.V28 = Av1Transform1dMath.Clamp(step.V28 + step.V29, range); + output.V29 = Av1Transform1dMath.Clamp(step.V28 - step.V29, range); + output.V30 = Av1Transform1dMath.Clamp(-step.V30 + step.V31, range); + output.V31 = Av1Transform1dMath.Clamp(step.V30 + step.V31, range); + + // Stage 4 rotates the next odd-frequency level while preserving completed low-frequency lanes. + stage++; + range = stageRange[stage]; + step.V0 = output.V0; + step.V1 = output.V1; + step.V2 = output.V2; + step.V3 = output.V3; + step.V4 = Av1Transform1dMath.HalfButterfly(cospi[56], output.V4, -cospi[8], output.V7, cosBit); + step.V5 = Av1Transform1dMath.HalfButterfly(cospi[24], output.V5, -cospi[40], output.V6, cosBit); + step.V6 = Av1Transform1dMath.HalfButterfly(cospi[40], output.V5, cospi[24], output.V6, cosBit); + step.V7 = Av1Transform1dMath.HalfButterfly(cospi[8], output.V4, cospi[56], step.V7, cosBit); + step.V8 = Av1Transform1dMath.Clamp(output.V8 + output.V9, range); + step.V9 = Av1Transform1dMath.Clamp(output.V8 - output.V9, range); + step.V10 = Av1Transform1dMath.Clamp(-output.V10 + output.V11, range); + step.V11 = Av1Transform1dMath.Clamp(output.V10 + output.V11, range); + step.V12 = Av1Transform1dMath.Clamp(output.V12 + output.V13, range); + step.V13 = Av1Transform1dMath.Clamp(output.V12 - output.V13, range); + step.V14 = Av1Transform1dMath.Clamp(-output.V14 + output.V15, range); + step.V15 = Av1Transform1dMath.Clamp(output.V14 + output.V15, range); + step.V16 = output.V16; + step.V17 = Av1Transform1dMath.HalfButterfly(-cospi[8], output.V17, cospi[56], output.V30, cosBit); + step.V18 = Av1Transform1dMath.HalfButterfly(-cospi[56], output.V18, -cospi[8], output.V29, cosBit); + step.V19 = output.V19; + step.V20 = output.V20; + step.V21 = Av1Transform1dMath.HalfButterfly(-cospi[40], output.V21, cospi[24], output.V26, cosBit); + step.V22 = Av1Transform1dMath.HalfButterfly(-cospi[24], output.V22, -cospi[40], output.V25, cosBit); + step.V23 = output.V23; + step.V24 = output.V24; + step.V25 = Av1Transform1dMath.HalfButterfly(-cospi[40], output.V22, cospi[24], output.V25, cosBit); + step.V26 = Av1Transform1dMath.HalfButterfly(cospi[24], output.V21, cospi[40], output.V26, cosBit); + step.V27 = output.V27; + step.V28 = output.V28; + step.V29 = Av1Transform1dMath.HalfButterfly(-cospi[8], output.V18, cospi[56], output.V29, cosBit); + step.V30 = Av1Transform1dMath.HalfButterfly(cospi[56], output.V17, cospi[8], output.V30, cosBit); + step.V31 = output.V31; + + // Stage 5 reconstructs the embedded eight-point groups and combines adjacent odd terms. + stage++; + range = stageRange[stage]; + output.V0 = Av1Transform1dMath.HalfButterfly(cospi[32], step.V0, cospi[32], step.V1, cosBit); + output.V1 = Av1Transform1dMath.HalfButterfly(cospi[32], step.V0, -cospi[32], step.V1, cosBit); + output.V2 = Av1Transform1dMath.HalfButterfly(cospi[48], step.V2, -cospi[16], step.V3, cosBit); + output.V3 = Av1Transform1dMath.HalfButterfly(cospi[16], step.V2, cospi[48], step.V3, cosBit); + output.V4 = Av1Transform1dMath.Clamp(step.V4 + step.V5, range); + output.V5 = Av1Transform1dMath.Clamp(step.V4 - step.V5, range); + output.V6 = Av1Transform1dMath.Clamp(-step.V6 + step.V7, range); + output.V7 = Av1Transform1dMath.Clamp(step.V6 + step.V7, range); + output.V8 = step.V8; + output.V9 = Av1Transform1dMath.HalfButterfly(-cospi[16], step.V9, cospi[48], step.V14, cosBit); + output.V10 = Av1Transform1dMath.HalfButterfly(-cospi[48], step.V10, -cospi[16], step.V13, cosBit); + output.V11 = step.V11; + output.V12 = step.V12; + output.V13 = Av1Transform1dMath.HalfButterfly(-cospi[16], step.V10, cospi[48], step.V13, cosBit); + output.V14 = Av1Transform1dMath.HalfButterfly(cospi[48], step.V9, cospi[16], step.V14, cosBit); + output.V15 = step.V15; + output.V16 = Av1Transform1dMath.Clamp(step.V16 + step.V19, range); + output.V17 = Av1Transform1dMath.Clamp(step.V17 + step.V18, range); + output.V18 = Av1Transform1dMath.Clamp(step.V17 - step.V18, range); + output.V19 = Av1Transform1dMath.Clamp(step.V16 - step.V19, range); + output.V20 = Av1Transform1dMath.Clamp(-step.V20 + step.V23, range); + output.V21 = Av1Transform1dMath.Clamp(-step.V21 + step.V22, range); + output.V22 = Av1Transform1dMath.Clamp(step.V21 + step.V22, range); + output.V23 = Av1Transform1dMath.Clamp(step.V20 + step.V23, range); + output.V24 = Av1Transform1dMath.Clamp(step.V24 + step.V27, range); + output.V25 = Av1Transform1dMath.Clamp(step.V25 + step.V26, range); + output.V26 = Av1Transform1dMath.Clamp(step.V25 - step.V26, range); + output.V27 = Av1Transform1dMath.Clamp(step.V24 - step.V27, range); + output.V28 = Av1Transform1dMath.Clamp(-step.V28 + step.V31, range); + output.V29 = Av1Transform1dMath.Clamp(-step.V29 + step.V30, range); + output.V30 = Av1Transform1dMath.Clamp(step.V29 + step.V30, range); + output.V31 = Av1Transform1dMath.Clamp(step.V28 + step.V31, range); + + // Stage 6 completes the low-frequency four-point DCT and rotates the next odd-frequency pairs. + stage++; + range = stageRange[stage]; + step.V0 = Av1Transform1dMath.Clamp(output.V0 + output.V3, range); + step.V1 = Av1Transform1dMath.Clamp(output.V1 + output.V2, range); + step.V2 = Av1Transform1dMath.Clamp(output.V1 - output.V2, range); + step.V3 = Av1Transform1dMath.Clamp(output.V0 - output.V3, range); + step.V4 = output.V4; + step.V5 = Av1Transform1dMath.HalfButterfly(-cospi[32], output.V5, cospi[32], output.V6, cosBit); + step.V6 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V5, cospi[32], output.V6, cosBit); + step.V7 = output.V7; + step.V8 = Av1Transform1dMath.Clamp(output.V8 + output.V11, range); + step.V9 = Av1Transform1dMath.Clamp(output.V9 + output.V10, range); + step.V10 = Av1Transform1dMath.Clamp(output.V9 - output.V10, range); + step.V11 = Av1Transform1dMath.Clamp(output.V8 - output.V11, range); + step.V12 = Av1Transform1dMath.Clamp(-output.V12 + output.V15, range); + step.V13 = Av1Transform1dMath.Clamp(-output.V13 + output.V14, range); + step.V14 = Av1Transform1dMath.Clamp(output.V13 + output.V14, range); + step.V15 = Av1Transform1dMath.Clamp(output.V12 + output.V15, range); + step.V16 = output.V16; + step.V17 = output.V17; + step.V18 = Av1Transform1dMath.HalfButterfly(-cospi[16], output.V18, cospi[48], output.V29, cosBit); + step.V19 = Av1Transform1dMath.HalfButterfly(-cospi[16], output.V19, cospi[48], output.V28, cosBit); + step.V20 = Av1Transform1dMath.HalfButterfly(-cospi[48], output.V20, -cospi[16], output.V27, cosBit); + step.V21 = Av1Transform1dMath.HalfButterfly(-cospi[48], output.V21, -cospi[16], output.V26, cosBit); + step.V22 = output.V22; + step.V23 = output.V23; + step.V24 = output.V24; + step.V25 = output.V25; + step.V26 = Av1Transform1dMath.HalfButterfly(-cospi[16], output.V21, cospi[48], output.V26, cosBit); + step.V27 = Av1Transform1dMath.HalfButterfly(-cospi[16], output.V20, cospi[48], output.V27, cosBit); + step.V28 = Av1Transform1dMath.HalfButterfly(cospi[48], output.V19, cospi[16], output.V28, cosBit); + step.V29 = Av1Transform1dMath.HalfButterfly(cospi[48], output.V18, cospi[16], output.V29, cosBit); + step.V30 = output.V30; + step.V31 = output.V31; + + // Stage 7 widens the reconstructed groups through their next butterfly level. + stage++; + range = stageRange[stage]; + output.V0 = Av1Transform1dMath.Clamp(step.V0 + step.V7, range); + output.V1 = Av1Transform1dMath.Clamp(step.V1 + step.V6, range); + output.V2 = Av1Transform1dMath.Clamp(step.V2 + step.V5, range); + output.V3 = Av1Transform1dMath.Clamp(step.V3 + step.V4, range); + output.V4 = Av1Transform1dMath.Clamp(step.V3 - step.V4, range); + output.V5 = Av1Transform1dMath.Clamp(step.V2 - step.V5, range); + output.V6 = Av1Transform1dMath.Clamp(step.V1 - step.V6, range); + output.V7 = Av1Transform1dMath.Clamp(step.V0 - step.V7, range); + output.V8 = step.V8; + output.V9 = step.V9; + output.V10 = Av1Transform1dMath.HalfButterfly(-cospi[32], step.V10, cospi[32], step.V13, cosBit); + output.V11 = Av1Transform1dMath.HalfButterfly(-cospi[32], step.V11, cospi[32], step.V12, cosBit); + output.V12 = Av1Transform1dMath.HalfButterfly(cospi[32], step.V11, cospi[32], step.V12, cosBit); + output.V13 = Av1Transform1dMath.HalfButterfly(cospi[32], step.V10, cospi[32], step.V13, cosBit); + output.V14 = step.V14; + output.V15 = step.V15; + output.V16 = Av1Transform1dMath.Clamp(step.V16 + step.V23, range); + output.V17 = Av1Transform1dMath.Clamp(step.V17 + step.V22, range); + output.V18 = Av1Transform1dMath.Clamp(step.V18 + step.V21, range); + output.V19 = Av1Transform1dMath.Clamp(step.V19 + step.V20, range); + output.V20 = Av1Transform1dMath.Clamp(step.V19 - step.V20, range); + output.V21 = Av1Transform1dMath.Clamp(step.V18 - step.V21, range); + output.V22 = Av1Transform1dMath.Clamp(step.V17 - step.V22, range); + output.V23 = Av1Transform1dMath.Clamp(step.V16 - step.V23, range); + output.V24 = Av1Transform1dMath.Clamp(-step.V24 + step.V31, range); + output.V25 = Av1Transform1dMath.Clamp(-step.V25 + step.V30, range); + output.V26 = Av1Transform1dMath.Clamp(-step.V26 + step.V29, range); + output.V27 = Av1Transform1dMath.Clamp(-step.V27 + step.V28, range); + output.V28 = Av1Transform1dMath.Clamp(step.V27 + step.V28, range); + output.V29 = Av1Transform1dMath.Clamp(step.V26 + step.V29, range); + output.V30 = Av1Transform1dMath.Clamp(step.V25 + step.V30, range); + output.V31 = Av1Transform1dMath.Clamp(step.V24 + step.V31, range); + + // Stage 8 applies the remaining pi/4 rotations before the terminal spatial merge. + stage++; + range = stageRange[stage]; + step.V0 = Av1Transform1dMath.Clamp(output.V0 + output.V15, range); + step.V1 = Av1Transform1dMath.Clamp(output.V1 + output.V14, range); + step.V2 = Av1Transform1dMath.Clamp(output.V2 + output.V13, range); + step.V3 = Av1Transform1dMath.Clamp(output.V3 + output.V12, range); + step.V4 = Av1Transform1dMath.Clamp(output.V4 + output.V11, range); + step.V5 = Av1Transform1dMath.Clamp(output.V5 + output.V10, range); + step.V6 = Av1Transform1dMath.Clamp(output.V6 + output.V9, range); + step.V7 = Av1Transform1dMath.Clamp(output.V7 + output.V8, range); + step.V8 = Av1Transform1dMath.Clamp(output.V7 - output.V8, range); + step.V9 = Av1Transform1dMath.Clamp(output.V6 - output.V9, range); + step.V10 = Av1Transform1dMath.Clamp(output.V5 - output.V10, range); + step.V11 = Av1Transform1dMath.Clamp(output.V4 - output.V11, range); + step.V12 = Av1Transform1dMath.Clamp(output.V3 - output.V12, range); + step.V13 = Av1Transform1dMath.Clamp(output.V2 - output.V13, range); + step.V14 = Av1Transform1dMath.Clamp(output.V1 - output.V14, range); + step.V15 = Av1Transform1dMath.Clamp(output.V0 - output.V15, range); + step.V16 = output.V16; + step.V17 = output.V17; + step.V18 = output.V18; + step.V19 = output.V19; + step.V20 = Av1Transform1dMath.HalfButterfly(-cospi[32], output.V20, cospi[32], output.V27, cosBit); + step.V21 = Av1Transform1dMath.HalfButterfly(-cospi[32], output.V21, cospi[32], output.V26, cosBit); + step.V22 = Av1Transform1dMath.HalfButterfly(-cospi[32], output.V22, cospi[32], output.V25, cosBit); + step.V23 = Av1Transform1dMath.HalfButterfly(-cospi[32], output.V23, cospi[32], output.V24, cosBit); + step.V24 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V23, cospi[32], output.V24, cosBit); + step.V25 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V22, cospi[32], output.V25, cosBit); + step.V26 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V21, cospi[32], output.V26, cosBit); + step.V27 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V20, cospi[32], output.V27, cosBit); + step.V28 = output.V28; + step.V29 = output.V29; + step.V30 = output.V30; + step.V31 = output.V31; + + // Stage 9 merges the even and odd halves into spatial order and clamps every result. + stage++; + range = stageRange[stage]; + output.V0 = Av1Transform1dMath.Clamp(step.V0 + step.V31, range); + output.V1 = Av1Transform1dMath.Clamp(step.V1 + step.V30, range); + output.V2 = Av1Transform1dMath.Clamp(step.V2 + step.V29, range); + output.V3 = Av1Transform1dMath.Clamp(step.V3 + step.V28, range); + output.V4 = Av1Transform1dMath.Clamp(step.V4 + step.V27, range); + output.V5 = Av1Transform1dMath.Clamp(step.V5 + step.V26, range); + output.V6 = Av1Transform1dMath.Clamp(step.V6 + step.V25, range); + output.V7 = Av1Transform1dMath.Clamp(step.V7 + step.V24, range); + output.V8 = Av1Transform1dMath.Clamp(step.V8 + step.V23, range); + output.V9 = Av1Transform1dMath.Clamp(step.V9 + step.V22, range); + output.V10 = Av1Transform1dMath.Clamp(step.V10 + step.V21, range); + output.V11 = Av1Transform1dMath.Clamp(step.V11 + step.V20, range); + output.V12 = Av1Transform1dMath.Clamp(step.V12 + step.V19, range); + output.V13 = Av1Transform1dMath.Clamp(step.V13 + step.V18, range); + output.V14 = Av1Transform1dMath.Clamp(step.V14 + step.V17, range); + output.V15 = Av1Transform1dMath.Clamp(step.V15 + step.V16, range); + output.V16 = Av1Transform1dMath.Clamp(step.V15 - step.V16, range); + output.V17 = Av1Transform1dMath.Clamp(step.V14 - step.V17, range); + output.V18 = Av1Transform1dMath.Clamp(step.V13 - step.V18, range); + output.V19 = Av1Transform1dMath.Clamp(step.V12 - step.V19, range); + output.V20 = Av1Transform1dMath.Clamp(step.V11 - step.V20, range); + output.V21 = Av1Transform1dMath.Clamp(step.V10 - step.V21, range); + output.V22 = Av1Transform1dMath.Clamp(step.V9 - step.V22, range); + output.V23 = Av1Transform1dMath.Clamp(step.V8 - step.V23, range); + output.V24 = Av1Transform1dMath.Clamp(step.V7 - step.V24, range); + output.V25 = Av1Transform1dMath.Clamp(step.V6 - step.V25, range); + output.V26 = Av1Transform1dMath.Clamp(step.V5 - step.V26, range); + output.V27 = Av1Transform1dMath.Clamp(step.V4 - step.V27, range); + output.V28 = Av1Transform1dMath.Clamp(step.V3 - step.V28, range); + output.V29 = Av1Transform1dMath.Clamp(step.V2 - step.V29, range); + output.V30 = Av1Transform1dMath.Clamp(step.V1 - step.V30, range); + output.V31 = Av1Transform1dMath.Clamp(step.V0 - step.V31, range); + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Transform/Inverse/Av1Inverse2dTransformer.Dct4Operator.cs b/src/ImageSharp/Formats/Heif/Av1/Transform/Inverse/Av1Inverse2dTransformer.Dct4Operator.cs new file mode 100644 index 0000000000..e9abc192fe --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Transform/Inverse/Av1Inverse2dTransformer.Dct4Operator.cs @@ -0,0 +1,115 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +/// +/// Defines the four-point AV1 inverse discrete cosine transform operator. +/// +/// +/// Vector fields represent transform positions and vector lanes represent independent axes. The SIMD overloads apply +/// the same staged butterflies, fixed-point rounding, and range clamps as the scalar overload without mixing axes. +/// +internal static partial class Av1Inverse2dTransformer +{ + internal readonly struct Dct4Operator : IAv1Transform1dOperator + { + /// + /// Applies the normative four-point AV1 inverse discrete cosine transform. + /// + /// The four frequency-domain coefficients. + /// The four spatial-domain residual values. + /// The four-element stage buffer owned by the containing two-dimensional transform. + /// The fixed-point precision of the cosine constants. + /// The signed-bit range assigned to each transform stage. + public static void Transform(ReadOnlySpan input, Span output, Span step, int cosBit, InlineArray12 stageRange) + { + // AV1 stores coefficients in frequency order; this permutation restores the order expected by the staged DCT. + output[0] = input[0]; + output[1] = input[2]; + output[2] = input[1]; + output[3] = input[3]; + + // Rotate the even and odd coefficient pairs using the same fixed-point basis as the forward transform. + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + step[0] = Av1Transform1dMath.HalfButterfly(cospi[32], output[0], cospi[32], output[1], cosBit); + step[1] = Av1Transform1dMath.HalfButterfly(cospi[32], output[0], -cospi[32], output[1], cosBit); + step[2] = Av1Transform1dMath.HalfButterfly(cospi[48], output[2], -cospi[16], output[3], cosBit); + step[3] = Av1Transform1dMath.HalfButterfly(cospi[16], output[2], cospi[48], output[3], cosBit); + + // The terminal butterflies reconstruct spatial order and clamp every result to the normative stage range. + byte range = stageRange[3]; + output[0] = Av1Transform1dMath.Clamp(step[0] + step[3], range); + output[1] = Av1Transform1dMath.Clamp(step[1] + step[2], range); + output[2] = Av1Transform1dMath.Clamp(step[1] - step[2], range); + output[3] = Av1Transform1dMath.Clamp(step[0] - step[3], range); + } + + /// + public static void Transform( + ref Av1TransformVector> input, + ref Av1TransformVector> output, + ref Av1TransformVector> step, + int cosBit, + InlineArray12 stageRange) + { + // AV1 stores coefficients in frequency order; this permutation restores the order expected by the staged DCT. + output.V0 = input.V0; + output.V1 = input.V2; + output.V2 = input.V1; + output.V3 = input.V3; + + // Rotate the even and odd coefficient pairs using the same fixed-point basis as the forward transform. + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + step.V0 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V0, cospi[32], output.V1, cosBit); + step.V1 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V0, -cospi[32], output.V1, cosBit); + step.V2 = Av1Transform1dMath.HalfButterfly(cospi[48], output.V2, -cospi[16], output.V3, cosBit); + step.V3 = Av1Transform1dMath.HalfButterfly(cospi[16], output.V2, cospi[48], output.V3, cosBit); + + // The terminal butterflies reconstruct spatial order and clamp every result to the normative stage range. + byte range = stageRange[3]; + output.V0 = Av1Transform1dMath.Clamp(step.V0 + step.V3, range); + output.V1 = Av1Transform1dMath.Clamp(step.V1 + step.V2, range); + output.V2 = Av1Transform1dMath.Clamp(step.V1 - step.V2, range); + output.V3 = Av1Transform1dMath.Clamp(step.V0 - step.V3, range); + } + + /// + /// Applies the transform to four independent axes in parallel. + /// + /// The source values for the parallel transform axes. + /// The destination values for the parallel transform axes. + /// The fixed stage storage for the parallel transform axes. + /// The fixed-point precision of the cosine constants. + /// The signed-bit range assigned to each transform stage. + public static void Transform( + ref Av1TransformVector> input, + ref Av1TransformVector> output, + ref Av1TransformVector> step, + int cosBit, + InlineArray12 stageRange) + { + // AV1 stores coefficients in frequency order; this permutation restores the order expected by the staged DCT. + output.V0 = input.V0; + output.V1 = input.V2; + output.V2 = input.V1; + output.V3 = input.V3; + + // Rotate the even and odd coefficient pairs using the same fixed-point basis as the forward transform. + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + step.V0 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V0, cospi[32], output.V1, cosBit); + step.V1 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V0, -cospi[32], output.V1, cosBit); + step.V2 = Av1Transform1dMath.HalfButterfly(cospi[48], output.V2, -cospi[16], output.V3, cosBit); + step.V3 = Av1Transform1dMath.HalfButterfly(cospi[16], output.V2, cospi[48], output.V3, cosBit); + + // The terminal butterflies reconstruct spatial order and clamp every result to the normative stage range. + byte range = stageRange[3]; + output.V0 = Av1Transform1dMath.Clamp(step.V0 + step.V3, range); + output.V1 = Av1Transform1dMath.Clamp(step.V1 + step.V2, range); + output.V2 = Av1Transform1dMath.Clamp(step.V1 - step.V2, range); + output.V3 = Av1Transform1dMath.Clamp(step.V0 - step.V3, range); + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Transform/Inverse/Av1Inverse2dTransformer.Dct64Operator.cs b/src/ImageSharp/Formats/Heif/Av1/Transform/Inverse/Av1Inverse2dTransformer.Dct64Operator.cs new file mode 100644 index 0000000000..020940df70 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Transform/Inverse/Av1Inverse2dTransformer.Dct64Operator.cs @@ -0,0 +1,2275 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +/// +/// Defines the 64-point AV1 inverse discrete cosine transform operator. +/// +/// +/// Vector fields represent transform positions and vector lanes represent independent axes. The SIMD overloads apply +/// the same staged butterflies, fixed-point rounding, and range clamps as the scalar overload without mixing axes. +/// +internal static partial class Av1Inverse2dTransformer +{ + internal readonly struct Dct64Operator : IAv1Transform1dOperator + { + /// + /// Applies the normative 64-point AV1 inverse discrete cosine transform. + /// + /// The 64 frequency-domain coefficients. + /// The 64 spatial-domain residual values. + /// The 64-element stage buffer owned by the containing two-dimensional transform. + /// The fixed-point precision of the cosine constants. + /// The signed-bit range assigned to each transform stage. + public static void Transform(ReadOnlySpan input, Span output, Span step, int cosBit, InlineArray12 stageRange) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + int stage = 0; + + // Stage 1 permutes frequency-ordered coefficients into the recursive DCT factorization order. + stage++; + output[0] = input[0]; + output[1] = input[32]; + output[2] = input[16]; + output[3] = input[48]; + output[4] = input[8]; + output[5] = input[40]; + output[6] = input[24]; + output[7] = input[56]; + output[8] = input[4]; + output[9] = input[36]; + output[10] = input[20]; + output[11] = input[52]; + output[12] = input[12]; + output[13] = input[44]; + output[14] = input[28]; + output[15] = input[60]; + output[16] = input[2]; + output[17] = input[34]; + output[18] = input[18]; + output[19] = input[50]; + output[20] = input[10]; + output[21] = input[42]; + output[22] = input[26]; + output[23] = input[58]; + output[24] = input[6]; + output[25] = input[38]; + output[26] = input[22]; + output[27] = input[54]; + output[28] = input[14]; + output[29] = input[46]; + output[30] = input[30]; + output[31] = input[62]; + output[32] = input[1]; + output[33] = input[33]; + output[34] = input[17]; + output[35] = input[49]; + output[36] = input[9]; + output[37] = input[41]; + output[38] = input[25]; + output[39] = input[57]; + output[40] = input[5]; + output[41] = input[37]; + output[42] = input[21]; + output[43] = input[53]; + output[44] = input[13]; + output[45] = input[45]; + output[46] = input[29]; + output[47] = input[61]; + output[48] = input[3]; + output[49] = input[35]; + output[50] = input[19]; + output[51] = input[51]; + output[52] = input[11]; + output[53] = input[43]; + output[54] = input[27]; + output[55] = input[59]; + output[56] = input[7]; + output[57] = input[39]; + output[58] = input[23]; + output[59] = input[55]; + output[60] = input[15]; + output[61] = input[47]; + output[62] = input[31]; + output[63] = input[63]; + + // Stage 2 rotates the highest odd-frequency coefficient pairs by their pi/128 angles. + stage++; + step[0] = output[0]; + step[1] = output[1]; + step[2] = output[2]; + step[3] = output[3]; + step[4] = output[4]; + step[5] = output[5]; + step[6] = output[6]; + step[7] = output[7]; + step[8] = output[8]; + step[9] = output[9]; + step[10] = output[10]; + step[11] = output[11]; + step[12] = output[12]; + step[13] = output[13]; + step[14] = output[14]; + step[15] = output[15]; + step[16] = output[16]; + step[17] = output[17]; + step[18] = output[18]; + step[19] = output[19]; + step[20] = output[20]; + step[21] = output[21]; + step[22] = output[22]; + step[23] = output[23]; + step[24] = output[24]; + step[25] = output[25]; + step[26] = output[26]; + step[27] = output[27]; + step[28] = output[28]; + step[29] = output[29]; + step[30] = output[30]; + step[31] = output[31]; + step[32] = Av1Transform1dMath.HalfButterfly(cospi[63], output[32], -cospi[1], output[63], cosBit); + step[33] = Av1Transform1dMath.HalfButterfly(cospi[31], output[33], -cospi[33], output[62], cosBit); + step[34] = Av1Transform1dMath.HalfButterfly(cospi[47], output[34], -cospi[17], output[61], cosBit); + step[35] = Av1Transform1dMath.HalfButterfly(cospi[15], output[35], -cospi[49], output[60], cosBit); + step[36] = Av1Transform1dMath.HalfButterfly(cospi[55], output[36], -cospi[9], output[59], cosBit); + step[37] = Av1Transform1dMath.HalfButterfly(cospi[23], output[37], -cospi[41], output[58], cosBit); + step[38] = Av1Transform1dMath.HalfButterfly(cospi[39], output[38], -cospi[25], output[57], cosBit); + step[39] = Av1Transform1dMath.HalfButterfly(cospi[7], output[39], -cospi[57], output[56], cosBit); + step[40] = Av1Transform1dMath.HalfButterfly(cospi[59], output[40], -cospi[5], output[55], cosBit); + step[41] = Av1Transform1dMath.HalfButterfly(cospi[27], output[41], -cospi[37], output[54], cosBit); + step[42] = Av1Transform1dMath.HalfButterfly(cospi[43], output[42], -cospi[21], output[53], cosBit); + step[43] = Av1Transform1dMath.HalfButterfly(cospi[11], output[43], -cospi[53], output[52], cosBit); + step[44] = Av1Transform1dMath.HalfButterfly(cospi[51], output[44], -cospi[13], output[51], cosBit); + step[45] = Av1Transform1dMath.HalfButterfly(cospi[19], output[45], -cospi[45], output[50], cosBit); + step[46] = Av1Transform1dMath.HalfButterfly(cospi[35], output[46], -cospi[29], output[49], cosBit); + step[47] = Av1Transform1dMath.HalfButterfly(cospi[3], output[47], -cospi[61], output[48], cosBit); + step[48] = Av1Transform1dMath.HalfButterfly(cospi[61], output[47], cospi[3], output[48], cosBit); + step[49] = Av1Transform1dMath.HalfButterfly(cospi[29], output[46], cospi[35], output[49], cosBit); + step[50] = Av1Transform1dMath.HalfButterfly(cospi[45], output[45], cospi[19], output[50], cosBit); + step[51] = Av1Transform1dMath.HalfButterfly(cospi[13], output[44], cospi[51], output[51], cosBit); + step[52] = Av1Transform1dMath.HalfButterfly(cospi[53], output[43], cospi[11], output[52], cosBit); + step[53] = Av1Transform1dMath.HalfButterfly(cospi[21], output[42], cospi[43], output[53], cosBit); + step[54] = Av1Transform1dMath.HalfButterfly(cospi[37], output[41], cospi[27], output[54], cosBit); + step[55] = Av1Transform1dMath.HalfButterfly(cospi[5], output[40], cospi[59], output[55], cosBit); + step[56] = Av1Transform1dMath.HalfButterfly(cospi[57], output[39], cospi[7], output[56], cosBit); + step[57] = Av1Transform1dMath.HalfButterfly(cospi[25], output[38], cospi[39], output[57], cosBit); + step[58] = Av1Transform1dMath.HalfButterfly(cospi[41], output[37], cospi[23], output[58], cosBit); + step[59] = Av1Transform1dMath.HalfButterfly(cospi[9], output[36], cospi[55], output[59], cosBit); + step[60] = Av1Transform1dMath.HalfButterfly(cospi[49], output[35], cospi[15], output[60], cosBit); + step[61] = Av1Transform1dMath.HalfButterfly(cospi[17], output[34], cospi[47], output[61], cosBit); + step[62] = Av1Transform1dMath.HalfButterfly(cospi[33], output[33], cospi[31], output[62], cosBit); + step[63] = Av1Transform1dMath.HalfButterfly(cospi[1], output[32], cospi[63], output[63], cosBit); + + // Stage 3 reconstructs the first nested groups and combines their adjacent odd terms. + stage++; + output[0] = step[0]; + output[1] = step[1]; + output[2] = step[2]; + output[3] = step[3]; + output[4] = step[4]; + output[5] = step[5]; + output[6] = step[6]; + output[7] = step[7]; + output[8] = step[8]; + output[9] = step[9]; + output[10] = step[10]; + output[11] = step[11]; + output[12] = step[12]; + output[13] = step[13]; + output[14] = step[14]; + output[15] = step[15]; + output[16] = Av1Transform1dMath.HalfButterfly(cospi[62], step[16], -cospi[2], step[31], cosBit); + output[17] = Av1Transform1dMath.HalfButterfly(cospi[30], step[17], -cospi[34], step[30], cosBit); + output[18] = Av1Transform1dMath.HalfButterfly(cospi[46], step[18], -cospi[18], step[29], cosBit); + output[19] = Av1Transform1dMath.HalfButterfly(cospi[14], step[19], -cospi[50], step[28], cosBit); + output[20] = Av1Transform1dMath.HalfButterfly(cospi[54], step[20], -cospi[10], step[27], cosBit); + output[21] = Av1Transform1dMath.HalfButterfly(cospi[22], step[21], -cospi[42], step[26], cosBit); + output[22] = Av1Transform1dMath.HalfButterfly(cospi[38], step[22], -cospi[26], step[25], cosBit); + output[23] = Av1Transform1dMath.HalfButterfly(cospi[6], step[23], -cospi[58], step[24], cosBit); + output[24] = Av1Transform1dMath.HalfButterfly(cospi[58], step[23], cospi[6], step[24], cosBit); + output[25] = Av1Transform1dMath.HalfButterfly(cospi[26], step[22], cospi[38], step[25], cosBit); + output[26] = Av1Transform1dMath.HalfButterfly(cospi[42], step[21], cospi[22], step[26], cosBit); + output[27] = Av1Transform1dMath.HalfButterfly(cospi[10], step[20], cospi[54], step[27], cosBit); + output[28] = Av1Transform1dMath.HalfButterfly(cospi[50], step[19], cospi[14], step[28], cosBit); + output[29] = Av1Transform1dMath.HalfButterfly(cospi[18], step[18], cospi[46], step[29], cosBit); + output[30] = Av1Transform1dMath.HalfButterfly(cospi[34], step[17], cospi[30], step[30], cosBit); + output[31] = Av1Transform1dMath.HalfButterfly(cospi[2], step[16], cospi[62], step[31], cosBit); + output[32] = Av1Transform1dMath.Clamp(step[32] + step[33], stageRange[stage]); + output[33] = Av1Transform1dMath.Clamp(step[32] - step[33], stageRange[stage]); + output[34] = Av1Transform1dMath.Clamp(-step[34] + step[35], stageRange[stage]); + output[35] = Av1Transform1dMath.Clamp(step[34] + step[35], stageRange[stage]); + output[36] = Av1Transform1dMath.Clamp(step[36] + step[37], stageRange[stage]); + output[37] = Av1Transform1dMath.Clamp(step[36] - step[37], stageRange[stage]); + output[38] = Av1Transform1dMath.Clamp(-step[38] + step[39], stageRange[stage]); + output[39] = Av1Transform1dMath.Clamp(step[38] + step[39], stageRange[stage]); + output[40] = Av1Transform1dMath.Clamp(step[40] + step[41], stageRange[stage]); + output[41] = Av1Transform1dMath.Clamp(step[40] - step[41], stageRange[stage]); + output[42] = Av1Transform1dMath.Clamp(-step[42] + step[43], stageRange[stage]); + output[43] = Av1Transform1dMath.Clamp(step[42] + step[43], stageRange[stage]); + output[44] = Av1Transform1dMath.Clamp(step[44] + step[45], stageRange[stage]); + output[45] = Av1Transform1dMath.Clamp(step[44] - step[45], stageRange[stage]); + output[46] = Av1Transform1dMath.Clamp(-step[46] + step[47], stageRange[stage]); + output[47] = Av1Transform1dMath.Clamp(step[46] + step[47], stageRange[stage]); + output[48] = Av1Transform1dMath.Clamp(step[48] + step[49], stageRange[stage]); + output[49] = Av1Transform1dMath.Clamp(step[48] - step[49], stageRange[stage]); + output[50] = Av1Transform1dMath.Clamp(-step[50] + step[51], stageRange[stage]); + output[51] = Av1Transform1dMath.Clamp(step[50] + step[51], stageRange[stage]); + output[52] = Av1Transform1dMath.Clamp(step[52] + step[53], stageRange[stage]); + output[53] = Av1Transform1dMath.Clamp(step[52] - step[53], stageRange[stage]); + output[54] = Av1Transform1dMath.Clamp(-step[54] + step[55], stageRange[stage]); + output[55] = Av1Transform1dMath.Clamp(step[54] + step[55], stageRange[stage]); + output[56] = Av1Transform1dMath.Clamp(step[56] + step[57], stageRange[stage]); + output[57] = Av1Transform1dMath.Clamp(step[56] - step[57], stageRange[stage]); + output[58] = Av1Transform1dMath.Clamp(-step[58] + step[59], stageRange[stage]); + output[59] = Av1Transform1dMath.Clamp(step[58] + step[59], stageRange[stage]); + output[60] = Av1Transform1dMath.Clamp(step[60] + step[61], stageRange[stage]); + output[61] = Av1Transform1dMath.Clamp(step[60] - step[61], stageRange[stage]); + output[62] = Av1Transform1dMath.Clamp(-step[62] + step[63], stageRange[stage]); + output[63] = Av1Transform1dMath.Clamp(step[62] + step[63], stageRange[stage]); + + // Stage 4 rotates the next odd-frequency level while preserving completed low-frequency lanes. + stage++; + step[0] = output[0]; + step[1] = output[1]; + step[2] = output[2]; + step[3] = output[3]; + step[4] = output[4]; + step[5] = output[5]; + step[6] = output[6]; + step[7] = output[7]; + step[8] = Av1Transform1dMath.HalfButterfly(cospi[60], output[8], -cospi[4], output[15], cosBit); + step[9] = Av1Transform1dMath.HalfButterfly(cospi[28], output[9], -cospi[36], output[14], cosBit); + step[10] = Av1Transform1dMath.HalfButterfly(cospi[44], output[10], -cospi[20], output[13], cosBit); + step[11] = Av1Transform1dMath.HalfButterfly(cospi[12], output[11], -cospi[52], output[12], cosBit); + step[12] = Av1Transform1dMath.HalfButterfly(cospi[52], output[11], cospi[12], output[12], cosBit); + step[13] = Av1Transform1dMath.HalfButterfly(cospi[20], output[10], cospi[44], output[13], cosBit); + step[14] = Av1Transform1dMath.HalfButterfly(cospi[36], output[9], cospi[28], output[14], cosBit); + step[15] = Av1Transform1dMath.HalfButterfly(cospi[4], output[8], cospi[60], output[15], cosBit); + step[16] = Av1Transform1dMath.Clamp(output[16] + output[17], stageRange[stage]); + step[17] = Av1Transform1dMath.Clamp(output[16] - output[17], stageRange[stage]); + step[18] = Av1Transform1dMath.Clamp(-output[18] + output[19], stageRange[stage]); + step[19] = Av1Transform1dMath.Clamp(output[18] + output[19], stageRange[stage]); + step[20] = Av1Transform1dMath.Clamp(output[20] + output[21], stageRange[stage]); + step[21] = Av1Transform1dMath.Clamp(output[20] - output[21], stageRange[stage]); + step[22] = Av1Transform1dMath.Clamp(-output[22] + output[23], stageRange[stage]); + step[23] = Av1Transform1dMath.Clamp(output[22] + output[23], stageRange[stage]); + step[24] = Av1Transform1dMath.Clamp(output[24] + output[25], stageRange[stage]); + step[25] = Av1Transform1dMath.Clamp(output[24] - output[25], stageRange[stage]); + step[26] = Av1Transform1dMath.Clamp(-output[26] + output[27], stageRange[stage]); + step[27] = Av1Transform1dMath.Clamp(output[26] + output[27], stageRange[stage]); + step[28] = Av1Transform1dMath.Clamp(output[28] + output[29], stageRange[stage]); + step[29] = Av1Transform1dMath.Clamp(output[28] - output[29], stageRange[stage]); + step[30] = Av1Transform1dMath.Clamp(-output[30] + output[31], stageRange[stage]); + step[31] = Av1Transform1dMath.Clamp(output[30] + output[31], stageRange[stage]); + step[32] = output[32]; + step[33] = Av1Transform1dMath.HalfButterfly(-cospi[4], output[33], cospi[60], output[62], cosBit); + step[34] = Av1Transform1dMath.HalfButterfly(-cospi[60], output[34], -cospi[4], output[61], cosBit); + step[35] = output[35]; + step[36] = output[36]; + step[37] = Av1Transform1dMath.HalfButterfly(-cospi[36], output[37], cospi[28], output[58], cosBit); + step[38] = Av1Transform1dMath.HalfButterfly(-cospi[28], output[38], -cospi[36], output[57], cosBit); + step[39] = output[39]; + step[40] = output[40]; + step[41] = Av1Transform1dMath.HalfButterfly(-cospi[20], output[41], cospi[44], output[54], cosBit); + step[42] = Av1Transform1dMath.HalfButterfly(-cospi[44], output[42], -cospi[20], output[53], cosBit); + step[43] = output[43]; + step[44] = output[44]; + step[45] = Av1Transform1dMath.HalfButterfly(-cospi[52], output[45], cospi[12], output[50], cosBit); + step[46] = Av1Transform1dMath.HalfButterfly(-cospi[12], output[46], -cospi[52], output[49], cosBit); + step[47] = output[47]; + step[48] = output[48]; + step[49] = Av1Transform1dMath.HalfButterfly(-cospi[52], output[46], cospi[12], output[49], cosBit); + step[50] = Av1Transform1dMath.HalfButterfly(cospi[12], output[45], cospi[52], output[50], cosBit); + step[51] = output[51]; + step[52] = output[52]; + step[53] = Av1Transform1dMath.HalfButterfly(-cospi[20], output[42], cospi[44], output[53], cosBit); + step[54] = Av1Transform1dMath.HalfButterfly(cospi[44], output[41], cospi[20], output[54], cosBit); + step[55] = output[55]; + step[56] = output[56]; + step[57] = Av1Transform1dMath.HalfButterfly(-cospi[36], output[38], cospi[28], output[57], cosBit); + step[58] = Av1Transform1dMath.HalfButterfly(cospi[28], output[37], cospi[36], output[58], cosBit); + step[59] = output[59]; + step[60] = output[60]; + step[61] = Av1Transform1dMath.HalfButterfly(-cospi[4], output[34], cospi[60], output[61], cosBit); + step[62] = Av1Transform1dMath.HalfButterfly(cospi[60], output[33], cospi[4], output[62], cosBit); + step[63] = output[63]; + + // Stage 5 widens the nested groups through the next butterfly level. + stage++; + output[0] = step[0]; + output[1] = step[1]; + output[2] = step[2]; + output[3] = step[3]; + output[4] = Av1Transform1dMath.HalfButterfly(cospi[56], step[4], -cospi[8], step[7], cosBit); + output[5] = Av1Transform1dMath.HalfButterfly(cospi[24], step[5], -cospi[40], step[6], cosBit); + output[6] = Av1Transform1dMath.HalfButterfly(cospi[40], step[5], cospi[24], step[6], cosBit); + output[7] = Av1Transform1dMath.HalfButterfly(cospi[8], step[4], cospi[56], step[7], cosBit); + output[8] = Av1Transform1dMath.Clamp(step[8] + step[9], stageRange[stage]); + output[9] = Av1Transform1dMath.Clamp(step[8] - step[9], stageRange[stage]); + output[10] = Av1Transform1dMath.Clamp(-step[10] + step[11], stageRange[stage]); + output[11] = Av1Transform1dMath.Clamp(step[10] + step[11], stageRange[stage]); + output[12] = Av1Transform1dMath.Clamp(step[12] + step[13], stageRange[stage]); + output[13] = Av1Transform1dMath.Clamp(step[12] - step[13], stageRange[stage]); + output[14] = Av1Transform1dMath.Clamp(-step[14] + step[15], stageRange[stage]); + output[15] = Av1Transform1dMath.Clamp(step[14] + step[15], stageRange[stage]); + output[16] = step[16]; + output[17] = Av1Transform1dMath.HalfButterfly(-cospi[8], step[17], cospi[56], step[30], cosBit); + output[18] = Av1Transform1dMath.HalfButterfly(-cospi[56], step[18], -cospi[8], step[29], cosBit); + output[19] = step[19]; + output[20] = step[20]; + output[21] = Av1Transform1dMath.HalfButterfly(-cospi[40], step[21], cospi[24], step[26], cosBit); + output[22] = Av1Transform1dMath.HalfButterfly(-cospi[24], step[22], -cospi[40], step[25], cosBit); + output[23] = step[23]; + output[24] = step[24]; + output[25] = Av1Transform1dMath.HalfButterfly(-cospi[40], step[22], cospi[24], step[25], cosBit); + output[26] = Av1Transform1dMath.HalfButterfly(cospi[24], step[21], cospi[40], step[26], cosBit); + output[27] = step[27]; + output[28] = step[28]; + output[29] = Av1Transform1dMath.HalfButterfly(-cospi[8], step[18], cospi[56], step[29], cosBit); + output[30] = Av1Transform1dMath.HalfButterfly(cospi[56], step[17], cospi[8], step[30], cosBit); + output[31] = step[31]; + output[32] = Av1Transform1dMath.Clamp(step[32] + step[35], stageRange[stage]); + output[33] = Av1Transform1dMath.Clamp(step[33] + step[34], stageRange[stage]); + output[34] = Av1Transform1dMath.Clamp(step[33] - step[34], stageRange[stage]); + output[35] = Av1Transform1dMath.Clamp(step[32] - step[35], stageRange[stage]); + output[36] = Av1Transform1dMath.Clamp(-step[36] + step[39], stageRange[stage]); + output[37] = Av1Transform1dMath.Clamp(-step[37] + step[38], stageRange[stage]); + output[38] = Av1Transform1dMath.Clamp(step[37] + step[38], stageRange[stage]); + output[39] = Av1Transform1dMath.Clamp(step[36] + step[39], stageRange[stage]); + output[40] = Av1Transform1dMath.Clamp(step[40] + step[43], stageRange[stage]); + output[41] = Av1Transform1dMath.Clamp(step[41] + step[42], stageRange[stage]); + output[42] = Av1Transform1dMath.Clamp(step[41] - step[42], stageRange[stage]); + output[43] = Av1Transform1dMath.Clamp(step[40] - step[43], stageRange[stage]); + output[44] = Av1Transform1dMath.Clamp(-step[44] + step[47], stageRange[stage]); + output[45] = Av1Transform1dMath.Clamp(-step[45] + step[46], stageRange[stage]); + output[46] = Av1Transform1dMath.Clamp(step[45] + step[46], stageRange[stage]); + output[47] = Av1Transform1dMath.Clamp(step[44] + step[47], stageRange[stage]); + output[48] = Av1Transform1dMath.Clamp(step[48] + step[51], stageRange[stage]); + output[49] = Av1Transform1dMath.Clamp(step[49] + step[50], stageRange[stage]); + output[50] = Av1Transform1dMath.Clamp(step[49] - step[50], stageRange[stage]); + output[51] = Av1Transform1dMath.Clamp(step[48] - step[51], stageRange[stage]); + output[52] = Av1Transform1dMath.Clamp(-step[52] + step[55], stageRange[stage]); + output[53] = Av1Transform1dMath.Clamp(-step[53] + step[54], stageRange[stage]); + output[54] = Av1Transform1dMath.Clamp(step[53] + step[54], stageRange[stage]); + output[55] = Av1Transform1dMath.Clamp(step[52] + step[55], stageRange[stage]); + output[56] = Av1Transform1dMath.Clamp(step[56] + step[59], stageRange[stage]); + output[57] = Av1Transform1dMath.Clamp(step[57] + step[58], stageRange[stage]); + output[58] = Av1Transform1dMath.Clamp(step[57] - step[58], stageRange[stage]); + output[59] = Av1Transform1dMath.Clamp(step[56] - step[59], stageRange[stage]); + output[60] = Av1Transform1dMath.Clamp(-step[60] + step[63], stageRange[stage]); + output[61] = Av1Transform1dMath.Clamp(-step[61] + step[62], stageRange[stage]); + output[62] = Av1Transform1dMath.Clamp(step[61] + step[62], stageRange[stage]); + output[63] = Av1Transform1dMath.Clamp(step[60] + step[63], stageRange[stage]); + + // Stage 6 rotates the next odd-frequency level while preserving completed low-frequency lanes. + stage++; + step[0] = Av1Transform1dMath.HalfButterfly(cospi[32], output[0], cospi[32], output[1], cosBit); + step[1] = Av1Transform1dMath.HalfButterfly(cospi[32], output[0], -cospi[32], output[1], cosBit); + step[2] = Av1Transform1dMath.HalfButterfly(cospi[48], output[2], -cospi[16], output[3], cosBit); + step[3] = Av1Transform1dMath.HalfButterfly(cospi[16], output[2], cospi[48], output[3], cosBit); + step[4] = Av1Transform1dMath.Clamp(output[4] + output[5], stageRange[stage]); + step[5] = Av1Transform1dMath.Clamp(output[4] - output[5], stageRange[stage]); + step[6] = Av1Transform1dMath.Clamp(-output[6] + output[7], stageRange[stage]); + step[7] = Av1Transform1dMath.Clamp(output[6] + output[7], stageRange[stage]); + step[8] = output[8]; + step[9] = Av1Transform1dMath.HalfButterfly(-cospi[16], output[9], cospi[48], output[14], cosBit); + step[10] = Av1Transform1dMath.HalfButterfly(-cospi[48], output[10], -cospi[16], output[13], cosBit); + step[11] = output[11]; + step[12] = output[12]; + step[13] = Av1Transform1dMath.HalfButterfly(-cospi[16], output[10], cospi[48], output[13], cosBit); + step[14] = Av1Transform1dMath.HalfButterfly(cospi[48], output[9], cospi[16], output[14], cosBit); + step[15] = output[15]; + step[16] = Av1Transform1dMath.Clamp(output[16] + output[19], stageRange[stage]); + step[17] = Av1Transform1dMath.Clamp(output[17] + output[18], stageRange[stage]); + step[18] = Av1Transform1dMath.Clamp(output[17] - output[18], stageRange[stage]); + step[19] = Av1Transform1dMath.Clamp(output[16] - output[19], stageRange[stage]); + step[20] = Av1Transform1dMath.Clamp(-output[20] + output[23], stageRange[stage]); + step[21] = Av1Transform1dMath.Clamp(-output[21] + output[22], stageRange[stage]); + step[22] = Av1Transform1dMath.Clamp(output[21] + output[22], stageRange[stage]); + step[23] = Av1Transform1dMath.Clamp(output[20] + output[23], stageRange[stage]); + step[24] = Av1Transform1dMath.Clamp(output[24] + output[27], stageRange[stage]); + step[25] = Av1Transform1dMath.Clamp(output[25] + output[26], stageRange[stage]); + step[26] = Av1Transform1dMath.Clamp(output[25] - output[26], stageRange[stage]); + step[27] = Av1Transform1dMath.Clamp(output[24] - output[27], stageRange[stage]); + step[28] = Av1Transform1dMath.Clamp(-output[28] + output[31], stageRange[stage]); + step[29] = Av1Transform1dMath.Clamp(-output[29] + output[30], stageRange[stage]); + step[30] = Av1Transform1dMath.Clamp(output[29] + output[30], stageRange[stage]); + step[31] = Av1Transform1dMath.Clamp(output[28] + output[31], stageRange[stage]); + step[32] = output[32]; + step[33] = output[33]; + step[34] = Av1Transform1dMath.HalfButterfly(-cospi[8], output[34], cospi[56], output[61], cosBit); + step[35] = Av1Transform1dMath.HalfButterfly(-cospi[8], output[35], cospi[56], output[60], cosBit); + step[36] = Av1Transform1dMath.HalfButterfly(-cospi[56], output[36], -cospi[8], output[59], cosBit); + step[37] = Av1Transform1dMath.HalfButterfly(-cospi[56], output[37], -cospi[8], output[58], cosBit); + step[38] = output[38]; + step[39] = output[39]; + step[40] = output[40]; + step[41] = output[41]; + step[42] = Av1Transform1dMath.HalfButterfly(-cospi[40], output[42], cospi[24], output[53], cosBit); + step[43] = Av1Transform1dMath.HalfButterfly(-cospi[40], output[43], cospi[24], output[52], cosBit); + step[44] = Av1Transform1dMath.HalfButterfly(-cospi[24], output[44], -cospi[40], output[51], cosBit); + step[45] = Av1Transform1dMath.HalfButterfly(-cospi[24], output[45], -cospi[40], output[50], cosBit); + step[46] = output[46]; + step[47] = output[47]; + step[48] = output[48]; + step[49] = output[49]; + step[50] = Av1Transform1dMath.HalfButterfly(-cospi[40], output[45], cospi[24], output[50], cosBit); + step[51] = Av1Transform1dMath.HalfButterfly(-cospi[40], output[44], cospi[24], output[51], cosBit); + step[52] = Av1Transform1dMath.HalfButterfly(cospi[24], output[43], cospi[40], output[52], cosBit); + step[53] = Av1Transform1dMath.HalfButterfly(cospi[24], output[42], cospi[40], output[53], cosBit); + step[54] = output[54]; + step[55] = output[55]; + step[56] = output[56]; + step[57] = output[57]; + step[58] = Av1Transform1dMath.HalfButterfly(-cospi[8], output[37], cospi[56], output[58], cosBit); + step[59] = Av1Transform1dMath.HalfButterfly(-cospi[8], output[36], cospi[56], output[59], cosBit); + step[60] = Av1Transform1dMath.HalfButterfly(cospi[56], output[35], cospi[8], output[60], cosBit); + step[61] = Av1Transform1dMath.HalfButterfly(cospi[56], output[34], cospi[8], output[61], cosBit); + step[62] = output[62]; + step[63] = output[63]; + + // Stage 7 reconstructs the embedded sixteen-point groups and combines adjacent odd terms. + stage++; + output[0] = Av1Transform1dMath.Clamp(step[0] + step[3], stageRange[stage]); + output[1] = Av1Transform1dMath.Clamp(step[1] + step[2], stageRange[stage]); + output[2] = Av1Transform1dMath.Clamp(step[1] - step[2], stageRange[stage]); + output[3] = Av1Transform1dMath.Clamp(step[0] - step[3], stageRange[stage]); + output[4] = step[4]; + output[5] = Av1Transform1dMath.HalfButterfly(-cospi[32], step[5], cospi[32], step[6], cosBit); + output[6] = Av1Transform1dMath.HalfButterfly(cospi[32], step[5], cospi[32], step[6], cosBit); + output[7] = step[7]; + output[8] = Av1Transform1dMath.Clamp(step[8] + step[11], stageRange[stage]); + output[9] = Av1Transform1dMath.Clamp(step[9] + step[10], stageRange[stage]); + output[10] = Av1Transform1dMath.Clamp(step[9] - step[10], stageRange[stage]); + output[11] = Av1Transform1dMath.Clamp(step[8] - step[11], stageRange[stage]); + output[12] = Av1Transform1dMath.Clamp(-step[12] + step[15], stageRange[stage]); + output[13] = Av1Transform1dMath.Clamp(-step[13] + step[14], stageRange[stage]); + output[14] = Av1Transform1dMath.Clamp(step[13] + step[14], stageRange[stage]); + output[15] = Av1Transform1dMath.Clamp(step[12] + step[15], stageRange[stage]); + output[16] = step[16]; + output[17] = step[17]; + output[18] = Av1Transform1dMath.HalfButterfly(-cospi[16], step[18], cospi[48], step[29], cosBit); + output[19] = Av1Transform1dMath.HalfButterfly(-cospi[16], step[19], cospi[48], step[28], cosBit); + output[20] = Av1Transform1dMath.HalfButterfly(-cospi[48], step[20], -cospi[16], step[27], cosBit); + output[21] = Av1Transform1dMath.HalfButterfly(-cospi[48], step[21], -cospi[16], step[26], cosBit); + output[22] = step[22]; + output[23] = step[23]; + output[24] = step[24]; + output[25] = step[25]; + output[26] = Av1Transform1dMath.HalfButterfly(-cospi[16], step[21], cospi[48], step[26], cosBit); + output[27] = Av1Transform1dMath.HalfButterfly(-cospi[16], step[20], cospi[48], step[27], cosBit); + output[28] = Av1Transform1dMath.HalfButterfly(cospi[48], step[19], cospi[16], step[28], cosBit); + output[29] = Av1Transform1dMath.HalfButterfly(cospi[48], step[18], cospi[16], step[29], cosBit); + output[30] = step[30]; + output[31] = step[31]; + output[32] = Av1Transform1dMath.Clamp(step[32] + step[39], stageRange[stage]); + output[33] = Av1Transform1dMath.Clamp(step[33] + step[38], stageRange[stage]); + output[34] = Av1Transform1dMath.Clamp(step[34] + step[37], stageRange[stage]); + output[35] = Av1Transform1dMath.Clamp(step[35] + step[36], stageRange[stage]); + output[36] = Av1Transform1dMath.Clamp(step[35] - step[36], stageRange[stage]); + output[37] = Av1Transform1dMath.Clamp(step[34] - step[37], stageRange[stage]); + output[38] = Av1Transform1dMath.Clamp(step[33] - step[38], stageRange[stage]); + output[39] = Av1Transform1dMath.Clamp(step[32] - step[39], stageRange[stage]); + output[40] = Av1Transform1dMath.Clamp(-step[40] + step[47], stageRange[stage]); + output[41] = Av1Transform1dMath.Clamp(-step[41] + step[46], stageRange[stage]); + output[42] = Av1Transform1dMath.Clamp(-step[42] + step[45], stageRange[stage]); + output[43] = Av1Transform1dMath.Clamp(-step[43] + step[44], stageRange[stage]); + output[44] = Av1Transform1dMath.Clamp(step[43] + step[44], stageRange[stage]); + output[45] = Av1Transform1dMath.Clamp(step[42] + step[45], stageRange[stage]); + output[46] = Av1Transform1dMath.Clamp(step[41] + step[46], stageRange[stage]); + output[47] = Av1Transform1dMath.Clamp(step[40] + step[47], stageRange[stage]); + output[48] = Av1Transform1dMath.Clamp(step[48] + step[55], stageRange[stage]); + output[49] = Av1Transform1dMath.Clamp(step[49] + step[54], stageRange[stage]); + output[50] = Av1Transform1dMath.Clamp(step[50] + step[53], stageRange[stage]); + output[51] = Av1Transform1dMath.Clamp(step[51] + step[52], stageRange[stage]); + output[52] = Av1Transform1dMath.Clamp(step[51] - step[52], stageRange[stage]); + output[53] = Av1Transform1dMath.Clamp(step[50] - step[53], stageRange[stage]); + output[54] = Av1Transform1dMath.Clamp(step[49] - step[54], stageRange[stage]); + output[55] = Av1Transform1dMath.Clamp(step[48] - step[55], stageRange[stage]); + output[56] = Av1Transform1dMath.Clamp(-step[56] + step[63], stageRange[stage]); + output[57] = Av1Transform1dMath.Clamp(-step[57] + step[62], stageRange[stage]); + output[58] = Av1Transform1dMath.Clamp(-step[58] + step[61], stageRange[stage]); + output[59] = Av1Transform1dMath.Clamp(-step[59] + step[60], stageRange[stage]); + output[60] = Av1Transform1dMath.Clamp(step[59] + step[60], stageRange[stage]); + output[61] = Av1Transform1dMath.Clamp(step[58] + step[61], stageRange[stage]); + output[62] = Av1Transform1dMath.Clamp(step[57] + step[62], stageRange[stage]); + output[63] = Av1Transform1dMath.Clamp(step[56] + step[63], stageRange[stage]); + + // Stage 8 completes the embedded eight-point groups and rotates their odd-frequency pairs. + stage++; + step[0] = Av1Transform1dMath.Clamp(output[0] + output[7], stageRange[stage]); + step[1] = Av1Transform1dMath.Clamp(output[1] + output[6], stageRange[stage]); + step[2] = Av1Transform1dMath.Clamp(output[2] + output[5], stageRange[stage]); + step[3] = Av1Transform1dMath.Clamp(output[3] + output[4], stageRange[stage]); + step[4] = Av1Transform1dMath.Clamp(output[3] - output[4], stageRange[stage]); + step[5] = Av1Transform1dMath.Clamp(output[2] - output[5], stageRange[stage]); + step[6] = Av1Transform1dMath.Clamp(output[1] - output[6], stageRange[stage]); + step[7] = Av1Transform1dMath.Clamp(output[0] - output[7], stageRange[stage]); + step[8] = output[8]; + step[9] = output[9]; + step[10] = Av1Transform1dMath.HalfButterfly(-cospi[32], output[10], cospi[32], output[13], cosBit); + step[11] = Av1Transform1dMath.HalfButterfly(-cospi[32], output[11], cospi[32], output[12], cosBit); + step[12] = Av1Transform1dMath.HalfButterfly(cospi[32], output[11], cospi[32], output[12], cosBit); + step[13] = Av1Transform1dMath.HalfButterfly(cospi[32], output[10], cospi[32], output[13], cosBit); + step[14] = output[14]; + step[15] = output[15]; + step[16] = Av1Transform1dMath.Clamp(output[16] + output[23], stageRange[stage]); + step[17] = Av1Transform1dMath.Clamp(output[17] + output[22], stageRange[stage]); + step[18] = Av1Transform1dMath.Clamp(output[18] + output[21], stageRange[stage]); + step[19] = Av1Transform1dMath.Clamp(output[19] + output[20], stageRange[stage]); + step[20] = Av1Transform1dMath.Clamp(output[19] - output[20], stageRange[stage]); + step[21] = Av1Transform1dMath.Clamp(output[18] - output[21], stageRange[stage]); + step[22] = Av1Transform1dMath.Clamp(output[17] - output[22], stageRange[stage]); + step[23] = Av1Transform1dMath.Clamp(output[16] - output[23], stageRange[stage]); + step[24] = Av1Transform1dMath.Clamp(-output[24] + output[31], stageRange[stage]); + step[25] = Av1Transform1dMath.Clamp(-output[25] + output[30], stageRange[stage]); + step[26] = Av1Transform1dMath.Clamp(-output[26] + output[29], stageRange[stage]); + step[27] = Av1Transform1dMath.Clamp(-output[27] + output[28], stageRange[stage]); + step[28] = Av1Transform1dMath.Clamp(output[27] + output[28], stageRange[stage]); + step[29] = Av1Transform1dMath.Clamp(output[26] + output[29], stageRange[stage]); + step[30] = Av1Transform1dMath.Clamp(output[25] + output[30], stageRange[stage]); + step[31] = Av1Transform1dMath.Clamp(output[24] + output[31], stageRange[stage]); + step[32] = output[32]; + step[33] = output[33]; + step[34] = output[34]; + step[35] = output[35]; + step[36] = Av1Transform1dMath.HalfButterfly(-cospi[16], output[36], cospi[48], output[59], cosBit); + step[37] = Av1Transform1dMath.HalfButterfly(-cospi[16], output[37], cospi[48], output[58], cosBit); + step[38] = Av1Transform1dMath.HalfButterfly(-cospi[16], output[38], cospi[48], output[57], cosBit); + step[39] = Av1Transform1dMath.HalfButterfly(-cospi[16], output[39], cospi[48], output[56], cosBit); + step[40] = Av1Transform1dMath.HalfButterfly(-cospi[48], output[40], -cospi[16], output[55], cosBit); + step[41] = Av1Transform1dMath.HalfButterfly(-cospi[48], output[41], -cospi[16], output[54], cosBit); + step[42] = Av1Transform1dMath.HalfButterfly(-cospi[48], output[42], -cospi[16], output[53], cosBit); + step[43] = Av1Transform1dMath.HalfButterfly(-cospi[48], output[43], -cospi[16], output[52], cosBit); + step[44] = output[44]; + step[45] = output[45]; + step[46] = output[46]; + step[47] = output[47]; + step[48] = output[48]; + step[49] = output[49]; + step[50] = output[50]; + step[51] = output[51]; + step[52] = Av1Transform1dMath.HalfButterfly(-cospi[16], output[43], cospi[48], output[52], cosBit); + step[53] = Av1Transform1dMath.HalfButterfly(-cospi[16], output[42], cospi[48], output[53], cosBit); + step[54] = Av1Transform1dMath.HalfButterfly(-cospi[16], output[41], cospi[48], output[54], cosBit); + step[55] = Av1Transform1dMath.HalfButterfly(-cospi[16], output[40], cospi[48], output[55], cosBit); + step[56] = Av1Transform1dMath.HalfButterfly(cospi[48], output[39], cospi[16], output[56], cosBit); + step[57] = Av1Transform1dMath.HalfButterfly(cospi[48], output[38], cospi[16], output[57], cosBit); + step[58] = Av1Transform1dMath.HalfButterfly(cospi[48], output[37], cospi[16], output[58], cosBit); + step[59] = Av1Transform1dMath.HalfButterfly(cospi[48], output[36], cospi[16], output[59], cosBit); + step[60] = output[60]; + step[61] = output[61]; + step[62] = output[62]; + step[63] = output[63]; + + // Stage 9 widens the reconstructed groups through their next butterfly level. + stage++; + output[0] = Av1Transform1dMath.Clamp(step[0] + step[15], stageRange[stage]); + output[1] = Av1Transform1dMath.Clamp(step[1] + step[14], stageRange[stage]); + output[2] = Av1Transform1dMath.Clamp(step[2] + step[13], stageRange[stage]); + output[3] = Av1Transform1dMath.Clamp(step[3] + step[12], stageRange[stage]); + output[4] = Av1Transform1dMath.Clamp(step[4] + step[11], stageRange[stage]); + output[5] = Av1Transform1dMath.Clamp(step[5] + step[10], stageRange[stage]); + output[6] = Av1Transform1dMath.Clamp(step[6] + step[9], stageRange[stage]); + output[7] = Av1Transform1dMath.Clamp(step[7] + step[8], stageRange[stage]); + output[8] = Av1Transform1dMath.Clamp(step[7] - step[8], stageRange[stage]); + output[9] = Av1Transform1dMath.Clamp(step[6] - step[9], stageRange[stage]); + output[10] = Av1Transform1dMath.Clamp(step[5] - step[10], stageRange[stage]); + output[11] = Av1Transform1dMath.Clamp(step[4] - step[11], stageRange[stage]); + output[12] = Av1Transform1dMath.Clamp(step[3] - step[12], stageRange[stage]); + output[13] = Av1Transform1dMath.Clamp(step[2] - step[13], stageRange[stage]); + output[14] = Av1Transform1dMath.Clamp(step[1] - step[14], stageRange[stage]); + output[15] = Av1Transform1dMath.Clamp(step[0] - step[15], stageRange[stage]); + output[16] = step[16]; + output[17] = step[17]; + output[18] = step[18]; + output[19] = step[19]; + output[20] = Av1Transform1dMath.HalfButterfly(-cospi[32], step[20], cospi[32], step[27], cosBit); + output[21] = Av1Transform1dMath.HalfButterfly(-cospi[32], step[21], cospi[32], step[26], cosBit); + output[22] = Av1Transform1dMath.HalfButterfly(-cospi[32], step[22], cospi[32], step[25], cosBit); + output[23] = Av1Transform1dMath.HalfButterfly(-cospi[32], step[23], cospi[32], step[24], cosBit); + output[24] = Av1Transform1dMath.HalfButterfly(cospi[32], step[23], cospi[32], step[24], cosBit); + output[25] = Av1Transform1dMath.HalfButterfly(cospi[32], step[22], cospi[32], step[25], cosBit); + output[26] = Av1Transform1dMath.HalfButterfly(cospi[32], step[21], cospi[32], step[26], cosBit); + output[27] = Av1Transform1dMath.HalfButterfly(cospi[32], step[20], cospi[32], step[27], cosBit); + output[28] = step[28]; + output[29] = step[29]; + output[30] = step[30]; + output[31] = step[31]; + output[32] = Av1Transform1dMath.Clamp(step[32] + step[47], stageRange[stage]); + output[33] = Av1Transform1dMath.Clamp(step[33] + step[46], stageRange[stage]); + output[34] = Av1Transform1dMath.Clamp(step[34] + step[45], stageRange[stage]); + output[35] = Av1Transform1dMath.Clamp(step[35] + step[44], stageRange[stage]); + output[36] = Av1Transform1dMath.Clamp(step[36] + step[43], stageRange[stage]); + output[37] = Av1Transform1dMath.Clamp(step[37] + step[42], stageRange[stage]); + output[38] = Av1Transform1dMath.Clamp(step[38] + step[41], stageRange[stage]); + output[39] = Av1Transform1dMath.Clamp(step[39] + step[40], stageRange[stage]); + output[40] = Av1Transform1dMath.Clamp(step[39] - step[40], stageRange[stage]); + output[41] = Av1Transform1dMath.Clamp(step[38] - step[41], stageRange[stage]); + output[42] = Av1Transform1dMath.Clamp(step[37] - step[42], stageRange[stage]); + output[43] = Av1Transform1dMath.Clamp(step[36] - step[43], stageRange[stage]); + output[44] = Av1Transform1dMath.Clamp(step[35] - step[44], stageRange[stage]); + output[45] = Av1Transform1dMath.Clamp(step[34] - step[45], stageRange[stage]); + output[46] = Av1Transform1dMath.Clamp(step[33] - step[46], stageRange[stage]); + output[47] = Av1Transform1dMath.Clamp(step[32] - step[47], stageRange[stage]); + output[48] = Av1Transform1dMath.Clamp(-step[48] + step[63], stageRange[stage]); + output[49] = Av1Transform1dMath.Clamp(-step[49] + step[62], stageRange[stage]); + output[50] = Av1Transform1dMath.Clamp(-step[50] + step[61], stageRange[stage]); + output[51] = Av1Transform1dMath.Clamp(-step[51] + step[60], stageRange[stage]); + output[52] = Av1Transform1dMath.Clamp(-step[52] + step[59], stageRange[stage]); + output[53] = Av1Transform1dMath.Clamp(-step[53] + step[58], stageRange[stage]); + output[54] = Av1Transform1dMath.Clamp(-step[54] + step[57], stageRange[stage]); + output[55] = Av1Transform1dMath.Clamp(-step[55] + step[56], stageRange[stage]); + output[56] = Av1Transform1dMath.Clamp(step[55] + step[56], stageRange[stage]); + output[57] = Av1Transform1dMath.Clamp(step[54] + step[57], stageRange[stage]); + output[58] = Av1Transform1dMath.Clamp(step[53] + step[58], stageRange[stage]); + output[59] = Av1Transform1dMath.Clamp(step[52] + step[59], stageRange[stage]); + output[60] = Av1Transform1dMath.Clamp(step[51] + step[60], stageRange[stage]); + output[61] = Av1Transform1dMath.Clamp(step[50] + step[61], stageRange[stage]); + output[62] = Av1Transform1dMath.Clamp(step[49] + step[62], stageRange[stage]); + output[63] = Av1Transform1dMath.Clamp(step[48] + step[63], stageRange[stage]); + + // Stage 10 applies the remaining pi/4 rotations before the terminal spatial merge. + stage++; + step[0] = Av1Transform1dMath.Clamp(output[0] + output[31], stageRange[stage]); + step[1] = Av1Transform1dMath.Clamp(output[1] + output[30], stageRange[stage]); + step[2] = Av1Transform1dMath.Clamp(output[2] + output[29], stageRange[stage]); + step[3] = Av1Transform1dMath.Clamp(output[3] + output[28], stageRange[stage]); + step[4] = Av1Transform1dMath.Clamp(output[4] + output[27], stageRange[stage]); + step[5] = Av1Transform1dMath.Clamp(output[5] + output[26], stageRange[stage]); + step[6] = Av1Transform1dMath.Clamp(output[6] + output[25], stageRange[stage]); + step[7] = Av1Transform1dMath.Clamp(output[7] + output[24], stageRange[stage]); + step[8] = Av1Transform1dMath.Clamp(output[8] + output[23], stageRange[stage]); + step[9] = Av1Transform1dMath.Clamp(output[9] + output[22], stageRange[stage]); + step[10] = Av1Transform1dMath.Clamp(output[10] + output[21], stageRange[stage]); + step[11] = Av1Transform1dMath.Clamp(output[11] + output[20], stageRange[stage]); + step[12] = Av1Transform1dMath.Clamp(output[12] + output[19], stageRange[stage]); + step[13] = Av1Transform1dMath.Clamp(output[13] + output[18], stageRange[stage]); + step[14] = Av1Transform1dMath.Clamp(output[14] + output[17], stageRange[stage]); + step[15] = Av1Transform1dMath.Clamp(output[15] + output[16], stageRange[stage]); + step[16] = Av1Transform1dMath.Clamp(output[15] - output[16], stageRange[stage]); + step[17] = Av1Transform1dMath.Clamp(output[14] - output[17], stageRange[stage]); + step[18] = Av1Transform1dMath.Clamp(output[13] - output[18], stageRange[stage]); + step[19] = Av1Transform1dMath.Clamp(output[12] - output[19], stageRange[stage]); + step[20] = Av1Transform1dMath.Clamp(output[11] - output[20], stageRange[stage]); + step[21] = Av1Transform1dMath.Clamp(output[10] - output[21], stageRange[stage]); + step[22] = Av1Transform1dMath.Clamp(output[9] - output[22], stageRange[stage]); + step[23] = Av1Transform1dMath.Clamp(output[8] - output[23], stageRange[stage]); + step[24] = Av1Transform1dMath.Clamp(output[7] - output[24], stageRange[stage]); + step[25] = Av1Transform1dMath.Clamp(output[6] - output[25], stageRange[stage]); + step[26] = Av1Transform1dMath.Clamp(output[5] - output[26], stageRange[stage]); + step[27] = Av1Transform1dMath.Clamp(output[4] - output[27], stageRange[stage]); + step[28] = Av1Transform1dMath.Clamp(output[3] - output[28], stageRange[stage]); + step[29] = Av1Transform1dMath.Clamp(output[2] - output[29], stageRange[stage]); + step[30] = Av1Transform1dMath.Clamp(output[1] - output[30], stageRange[stage]); + step[31] = Av1Transform1dMath.Clamp(output[0] - output[31], stageRange[stage]); + step[32] = output[32]; + step[33] = output[33]; + step[34] = output[34]; + step[35] = output[35]; + step[36] = output[36]; + step[37] = output[37]; + step[38] = output[38]; + step[39] = output[39]; + step[40] = Av1Transform1dMath.HalfButterfly(-cospi[32], output[40], cospi[32], output[55], cosBit); + step[41] = Av1Transform1dMath.HalfButterfly(-cospi[32], output[41], cospi[32], output[54], cosBit); + step[42] = Av1Transform1dMath.HalfButterfly(-cospi[32], output[42], cospi[32], output[53], cosBit); + step[43] = Av1Transform1dMath.HalfButterfly(-cospi[32], output[43], cospi[32], output[52], cosBit); + step[44] = Av1Transform1dMath.HalfButterfly(-cospi[32], output[44], cospi[32], output[51], cosBit); + step[45] = Av1Transform1dMath.HalfButterfly(-cospi[32], output[45], cospi[32], output[50], cosBit); + step[46] = Av1Transform1dMath.HalfButterfly(-cospi[32], output[46], cospi[32], output[49], cosBit); + step[47] = Av1Transform1dMath.HalfButterfly(-cospi[32], output[47], cospi[32], output[48], cosBit); + step[48] = Av1Transform1dMath.HalfButterfly(cospi[32], output[47], cospi[32], output[48], cosBit); + step[49] = Av1Transform1dMath.HalfButterfly(cospi[32], output[46], cospi[32], output[49], cosBit); + step[50] = Av1Transform1dMath.HalfButterfly(cospi[32], output[45], cospi[32], output[50], cosBit); + step[51] = Av1Transform1dMath.HalfButterfly(cospi[32], output[44], cospi[32], output[51], cosBit); + step[52] = Av1Transform1dMath.HalfButterfly(cospi[32], output[43], cospi[32], output[52], cosBit); + step[53] = Av1Transform1dMath.HalfButterfly(cospi[32], output[42], cospi[32], output[53], cosBit); + step[54] = Av1Transform1dMath.HalfButterfly(cospi[32], output[41], cospi[32], output[54], cosBit); + step[55] = Av1Transform1dMath.HalfButterfly(cospi[32], output[40], cospi[32], output[55], cosBit); + step[56] = output[56]; + step[57] = output[57]; + step[58] = output[58]; + step[59] = output[59]; + step[60] = output[60]; + step[61] = output[61]; + step[62] = output[62]; + step[63] = output[63]; + + // Stage 11 merges the even and odd halves into spatial order and clamps every result. + stage++; + output[0] = Av1Transform1dMath.Clamp(step[0] + step[63], stageRange[stage]); + output[1] = Av1Transform1dMath.Clamp(step[1] + step[62], stageRange[stage]); + output[2] = Av1Transform1dMath.Clamp(step[2] + step[61], stageRange[stage]); + output[3] = Av1Transform1dMath.Clamp(step[3] + step[60], stageRange[stage]); + output[4] = Av1Transform1dMath.Clamp(step[4] + step[59], stageRange[stage]); + output[5] = Av1Transform1dMath.Clamp(step[5] + step[58], stageRange[stage]); + output[6] = Av1Transform1dMath.Clamp(step[6] + step[57], stageRange[stage]); + output[7] = Av1Transform1dMath.Clamp(step[7] + step[56], stageRange[stage]); + output[8] = Av1Transform1dMath.Clamp(step[8] + step[55], stageRange[stage]); + output[9] = Av1Transform1dMath.Clamp(step[9] + step[54], stageRange[stage]); + output[10] = Av1Transform1dMath.Clamp(step[10] + step[53], stageRange[stage]); + output[11] = Av1Transform1dMath.Clamp(step[11] + step[52], stageRange[stage]); + output[12] = Av1Transform1dMath.Clamp(step[12] + step[51], stageRange[stage]); + output[13] = Av1Transform1dMath.Clamp(step[13] + step[50], stageRange[stage]); + output[14] = Av1Transform1dMath.Clamp(step[14] + step[49], stageRange[stage]); + output[15] = Av1Transform1dMath.Clamp(step[15] + step[48], stageRange[stage]); + output[16] = Av1Transform1dMath.Clamp(step[16] + step[47], stageRange[stage]); + output[17] = Av1Transform1dMath.Clamp(step[17] + step[46], stageRange[stage]); + output[18] = Av1Transform1dMath.Clamp(step[18] + step[45], stageRange[stage]); + output[19] = Av1Transform1dMath.Clamp(step[19] + step[44], stageRange[stage]); + output[20] = Av1Transform1dMath.Clamp(step[20] + step[43], stageRange[stage]); + output[21] = Av1Transform1dMath.Clamp(step[21] + step[42], stageRange[stage]); + output[22] = Av1Transform1dMath.Clamp(step[22] + step[41], stageRange[stage]); + output[23] = Av1Transform1dMath.Clamp(step[23] + step[40], stageRange[stage]); + output[24] = Av1Transform1dMath.Clamp(step[24] + step[39], stageRange[stage]); + output[25] = Av1Transform1dMath.Clamp(step[25] + step[38], stageRange[stage]); + output[26] = Av1Transform1dMath.Clamp(step[26] + step[37], stageRange[stage]); + output[27] = Av1Transform1dMath.Clamp(step[27] + step[36], stageRange[stage]); + output[28] = Av1Transform1dMath.Clamp(step[28] + step[35], stageRange[stage]); + output[29] = Av1Transform1dMath.Clamp(step[29] + step[34], stageRange[stage]); + output[30] = Av1Transform1dMath.Clamp(step[30] + step[33], stageRange[stage]); + output[31] = Av1Transform1dMath.Clamp(step[31] + step[32], stageRange[stage]); + output[32] = Av1Transform1dMath.Clamp(step[31] - step[32], stageRange[stage]); + output[33] = Av1Transform1dMath.Clamp(step[30] - step[33], stageRange[stage]); + output[34] = Av1Transform1dMath.Clamp(step[29] - step[34], stageRange[stage]); + output[35] = Av1Transform1dMath.Clamp(step[28] - step[35], stageRange[stage]); + output[36] = Av1Transform1dMath.Clamp(step[27] - step[36], stageRange[stage]); + output[37] = Av1Transform1dMath.Clamp(step[26] - step[37], stageRange[stage]); + output[38] = Av1Transform1dMath.Clamp(step[25] - step[38], stageRange[stage]); + output[39] = Av1Transform1dMath.Clamp(step[24] - step[39], stageRange[stage]); + output[40] = Av1Transform1dMath.Clamp(step[23] - step[40], stageRange[stage]); + output[41] = Av1Transform1dMath.Clamp(step[22] - step[41], stageRange[stage]); + output[42] = Av1Transform1dMath.Clamp(step[21] - step[42], stageRange[stage]); + output[43] = Av1Transform1dMath.Clamp(step[20] - step[43], stageRange[stage]); + output[44] = Av1Transform1dMath.Clamp(step[19] - step[44], stageRange[stage]); + output[45] = Av1Transform1dMath.Clamp(step[18] - step[45], stageRange[stage]); + output[46] = Av1Transform1dMath.Clamp(step[17] - step[46], stageRange[stage]); + output[47] = Av1Transform1dMath.Clamp(step[16] - step[47], stageRange[stage]); + output[48] = Av1Transform1dMath.Clamp(step[15] - step[48], stageRange[stage]); + output[49] = Av1Transform1dMath.Clamp(step[14] - step[49], stageRange[stage]); + output[50] = Av1Transform1dMath.Clamp(step[13] - step[50], stageRange[stage]); + output[51] = Av1Transform1dMath.Clamp(step[12] - step[51], stageRange[stage]); + output[52] = Av1Transform1dMath.Clamp(step[11] - step[52], stageRange[stage]); + output[53] = Av1Transform1dMath.Clamp(step[10] - step[53], stageRange[stage]); + output[54] = Av1Transform1dMath.Clamp(step[9] - step[54], stageRange[stage]); + output[55] = Av1Transform1dMath.Clamp(step[8] - step[55], stageRange[stage]); + output[56] = Av1Transform1dMath.Clamp(step[7] - step[56], stageRange[stage]); + output[57] = Av1Transform1dMath.Clamp(step[6] - step[57], stageRange[stage]); + output[58] = Av1Transform1dMath.Clamp(step[5] - step[58], stageRange[stage]); + output[59] = Av1Transform1dMath.Clamp(step[4] - step[59], stageRange[stage]); + output[60] = Av1Transform1dMath.Clamp(step[3] - step[60], stageRange[stage]); + output[61] = Av1Transform1dMath.Clamp(step[2] - step[61], stageRange[stage]); + output[62] = Av1Transform1dMath.Clamp(step[1] - step[62], stageRange[stage]); + output[63] = Av1Transform1dMath.Clamp(step[0] - step[63], stageRange[stage]); + } + + /// + public static void Transform( + ref Av1TransformVector> input, + ref Av1TransformVector> output, + ref Av1TransformVector> step, + int cosBit, + InlineArray12 stageRange) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + int stage = 0; + + // Stage 1 permutes frequency-ordered coefficients into the recursive DCT factorization order. + stage++; + output.V0 = input.V0; + output.V1 = input.V32; + output.V2 = input.V16; + output.V3 = input.V48; + output.V4 = input.V8; + output.V5 = input.V40; + output.V6 = input.V24; + output.V7 = input.V56; + output.V8 = input.V4; + output.V9 = input.V36; + output.V10 = input.V20; + output.V11 = input.V52; + output.V12 = input.V12; + output.V13 = input.V44; + output.V14 = input.V28; + output.V15 = input.V60; + output.V16 = input.V2; + output.V17 = input.V34; + output.V18 = input.V18; + output.V19 = input.V50; + output.V20 = input.V10; + output.V21 = input.V42; + output.V22 = input.V26; + output.V23 = input.V58; + output.V24 = input.V6; + output.V25 = input.V38; + output.V26 = input.V22; + output.V27 = input.V54; + output.V28 = input.V14; + output.V29 = input.V46; + output.V30 = input.V30; + output.V31 = input.V62; + output.V32 = input.V1; + output.V33 = input.V33; + output.V34 = input.V17; + output.V35 = input.V49; + output.V36 = input.V9; + output.V37 = input.V41; + output.V38 = input.V25; + output.V39 = input.V57; + output.V40 = input.V5; + output.V41 = input.V37; + output.V42 = input.V21; + output.V43 = input.V53; + output.V44 = input.V13; + output.V45 = input.V45; + output.V46 = input.V29; + output.V47 = input.V61; + output.V48 = input.V3; + output.V49 = input.V35; + output.V50 = input.V19; + output.V51 = input.V51; + output.V52 = input.V11; + output.V53 = input.V43; + output.V54 = input.V27; + output.V55 = input.V59; + output.V56 = input.V7; + output.V57 = input.V39; + output.V58 = input.V23; + output.V59 = input.V55; + output.V60 = input.V15; + output.V61 = input.V47; + output.V62 = input.V31; + output.V63 = input.V63; + + // Stage 2 rotates the highest odd-frequency coefficient pairs by their pi/128 angles. + stage++; + step.V0 = output.V0; + step.V1 = output.V1; + step.V2 = output.V2; + step.V3 = output.V3; + step.V4 = output.V4; + step.V5 = output.V5; + step.V6 = output.V6; + step.V7 = output.V7; + step.V8 = output.V8; + step.V9 = output.V9; + step.V10 = output.V10; + step.V11 = output.V11; + step.V12 = output.V12; + step.V13 = output.V13; + step.V14 = output.V14; + step.V15 = output.V15; + step.V16 = output.V16; + step.V17 = output.V17; + step.V18 = output.V18; + step.V19 = output.V19; + step.V20 = output.V20; + step.V21 = output.V21; + step.V22 = output.V22; + step.V23 = output.V23; + step.V24 = output.V24; + step.V25 = output.V25; + step.V26 = output.V26; + step.V27 = output.V27; + step.V28 = output.V28; + step.V29 = output.V29; + step.V30 = output.V30; + step.V31 = output.V31; + step.V32 = Av1Transform1dMath.HalfButterfly(cospi[63], output.V32, -cospi[1], output.V63, cosBit); + step.V33 = Av1Transform1dMath.HalfButterfly(cospi[31], output.V33, -cospi[33], output.V62, cosBit); + step.V34 = Av1Transform1dMath.HalfButterfly(cospi[47], output.V34, -cospi[17], output.V61, cosBit); + step.V35 = Av1Transform1dMath.HalfButterfly(cospi[15], output.V35, -cospi[49], output.V60, cosBit); + step.V36 = Av1Transform1dMath.HalfButterfly(cospi[55], output.V36, -cospi[9], output.V59, cosBit); + step.V37 = Av1Transform1dMath.HalfButterfly(cospi[23], output.V37, -cospi[41], output.V58, cosBit); + step.V38 = Av1Transform1dMath.HalfButterfly(cospi[39], output.V38, -cospi[25], output.V57, cosBit); + step.V39 = Av1Transform1dMath.HalfButterfly(cospi[7], output.V39, -cospi[57], output.V56, cosBit); + step.V40 = Av1Transform1dMath.HalfButterfly(cospi[59], output.V40, -cospi[5], output.V55, cosBit); + step.V41 = Av1Transform1dMath.HalfButterfly(cospi[27], output.V41, -cospi[37], output.V54, cosBit); + step.V42 = Av1Transform1dMath.HalfButterfly(cospi[43], output.V42, -cospi[21], output.V53, cosBit); + step.V43 = Av1Transform1dMath.HalfButterfly(cospi[11], output.V43, -cospi[53], output.V52, cosBit); + step.V44 = Av1Transform1dMath.HalfButterfly(cospi[51], output.V44, -cospi[13], output.V51, cosBit); + step.V45 = Av1Transform1dMath.HalfButterfly(cospi[19], output.V45, -cospi[45], output.V50, cosBit); + step.V46 = Av1Transform1dMath.HalfButterfly(cospi[35], output.V46, -cospi[29], output.V49, cosBit); + step.V47 = Av1Transform1dMath.HalfButterfly(cospi[3], output.V47, -cospi[61], output.V48, cosBit); + step.V48 = Av1Transform1dMath.HalfButterfly(cospi[61], output.V47, cospi[3], output.V48, cosBit); + step.V49 = Av1Transform1dMath.HalfButterfly(cospi[29], output.V46, cospi[35], output.V49, cosBit); + step.V50 = Av1Transform1dMath.HalfButterfly(cospi[45], output.V45, cospi[19], output.V50, cosBit); + step.V51 = Av1Transform1dMath.HalfButterfly(cospi[13], output.V44, cospi[51], output.V51, cosBit); + step.V52 = Av1Transform1dMath.HalfButterfly(cospi[53], output.V43, cospi[11], output.V52, cosBit); + step.V53 = Av1Transform1dMath.HalfButterfly(cospi[21], output.V42, cospi[43], output.V53, cosBit); + step.V54 = Av1Transform1dMath.HalfButterfly(cospi[37], output.V41, cospi[27], output.V54, cosBit); + step.V55 = Av1Transform1dMath.HalfButterfly(cospi[5], output.V40, cospi[59], output.V55, cosBit); + step.V56 = Av1Transform1dMath.HalfButterfly(cospi[57], output.V39, cospi[7], output.V56, cosBit); + step.V57 = Av1Transform1dMath.HalfButterfly(cospi[25], output.V38, cospi[39], output.V57, cosBit); + step.V58 = Av1Transform1dMath.HalfButterfly(cospi[41], output.V37, cospi[23], output.V58, cosBit); + step.V59 = Av1Transform1dMath.HalfButterfly(cospi[9], output.V36, cospi[55], output.V59, cosBit); + step.V60 = Av1Transform1dMath.HalfButterfly(cospi[49], output.V35, cospi[15], output.V60, cosBit); + step.V61 = Av1Transform1dMath.HalfButterfly(cospi[17], output.V34, cospi[47], output.V61, cosBit); + step.V62 = Av1Transform1dMath.HalfButterfly(cospi[33], output.V33, cospi[31], output.V62, cosBit); + step.V63 = Av1Transform1dMath.HalfButterfly(cospi[1], output.V32, cospi[63], output.V63, cosBit); + + // Stage 3 reconstructs the first nested groups and combines their adjacent odd terms. + stage++; + output.V0 = step.V0; + output.V1 = step.V1; + output.V2 = step.V2; + output.V3 = step.V3; + output.V4 = step.V4; + output.V5 = step.V5; + output.V6 = step.V6; + output.V7 = step.V7; + output.V8 = step.V8; + output.V9 = step.V9; + output.V10 = step.V10; + output.V11 = step.V11; + output.V12 = step.V12; + output.V13 = step.V13; + output.V14 = step.V14; + output.V15 = step.V15; + output.V16 = Av1Transform1dMath.HalfButterfly(cospi[62], step.V16, -cospi[2], step.V31, cosBit); + output.V17 = Av1Transform1dMath.HalfButterfly(cospi[30], step.V17, -cospi[34], step.V30, cosBit); + output.V18 = Av1Transform1dMath.HalfButterfly(cospi[46], step.V18, -cospi[18], step.V29, cosBit); + output.V19 = Av1Transform1dMath.HalfButterfly(cospi[14], step.V19, -cospi[50], step.V28, cosBit); + output.V20 = Av1Transform1dMath.HalfButterfly(cospi[54], step.V20, -cospi[10], step.V27, cosBit); + output.V21 = Av1Transform1dMath.HalfButterfly(cospi[22], step.V21, -cospi[42], step.V26, cosBit); + output.V22 = Av1Transform1dMath.HalfButterfly(cospi[38], step.V22, -cospi[26], step.V25, cosBit); + output.V23 = Av1Transform1dMath.HalfButterfly(cospi[6], step.V23, -cospi[58], step.V24, cosBit); + output.V24 = Av1Transform1dMath.HalfButterfly(cospi[58], step.V23, cospi[6], step.V24, cosBit); + output.V25 = Av1Transform1dMath.HalfButterfly(cospi[26], step.V22, cospi[38], step.V25, cosBit); + output.V26 = Av1Transform1dMath.HalfButterfly(cospi[42], step.V21, cospi[22], step.V26, cosBit); + output.V27 = Av1Transform1dMath.HalfButterfly(cospi[10], step.V20, cospi[54], step.V27, cosBit); + output.V28 = Av1Transform1dMath.HalfButterfly(cospi[50], step.V19, cospi[14], step.V28, cosBit); + output.V29 = Av1Transform1dMath.HalfButterfly(cospi[18], step.V18, cospi[46], step.V29, cosBit); + output.V30 = Av1Transform1dMath.HalfButterfly(cospi[34], step.V17, cospi[30], step.V30, cosBit); + output.V31 = Av1Transform1dMath.HalfButterfly(cospi[2], step.V16, cospi[62], step.V31, cosBit); + output.V32 = Av1Transform1dMath.Clamp(step.V32 + step.V33, stageRange[stage]); + output.V33 = Av1Transform1dMath.Clamp(step.V32 - step.V33, stageRange[stage]); + output.V34 = Av1Transform1dMath.Clamp(-step.V34 + step.V35, stageRange[stage]); + output.V35 = Av1Transform1dMath.Clamp(step.V34 + step.V35, stageRange[stage]); + output.V36 = Av1Transform1dMath.Clamp(step.V36 + step.V37, stageRange[stage]); + output.V37 = Av1Transform1dMath.Clamp(step.V36 - step.V37, stageRange[stage]); + output.V38 = Av1Transform1dMath.Clamp(-step.V38 + step.V39, stageRange[stage]); + output.V39 = Av1Transform1dMath.Clamp(step.V38 + step.V39, stageRange[stage]); + output.V40 = Av1Transform1dMath.Clamp(step.V40 + step.V41, stageRange[stage]); + output.V41 = Av1Transform1dMath.Clamp(step.V40 - step.V41, stageRange[stage]); + output.V42 = Av1Transform1dMath.Clamp(-step.V42 + step.V43, stageRange[stage]); + output.V43 = Av1Transform1dMath.Clamp(step.V42 + step.V43, stageRange[stage]); + output.V44 = Av1Transform1dMath.Clamp(step.V44 + step.V45, stageRange[stage]); + output.V45 = Av1Transform1dMath.Clamp(step.V44 - step.V45, stageRange[stage]); + output.V46 = Av1Transform1dMath.Clamp(-step.V46 + step.V47, stageRange[stage]); + output.V47 = Av1Transform1dMath.Clamp(step.V46 + step.V47, stageRange[stage]); + output.V48 = Av1Transform1dMath.Clamp(step.V48 + step.V49, stageRange[stage]); + output.V49 = Av1Transform1dMath.Clamp(step.V48 - step.V49, stageRange[stage]); + output.V50 = Av1Transform1dMath.Clamp(-step.V50 + step.V51, stageRange[stage]); + output.V51 = Av1Transform1dMath.Clamp(step.V50 + step.V51, stageRange[stage]); + output.V52 = Av1Transform1dMath.Clamp(step.V52 + step.V53, stageRange[stage]); + output.V53 = Av1Transform1dMath.Clamp(step.V52 - step.V53, stageRange[stage]); + output.V54 = Av1Transform1dMath.Clamp(-step.V54 + step.V55, stageRange[stage]); + output.V55 = Av1Transform1dMath.Clamp(step.V54 + step.V55, stageRange[stage]); + output.V56 = Av1Transform1dMath.Clamp(step.V56 + step.V57, stageRange[stage]); + output.V57 = Av1Transform1dMath.Clamp(step.V56 - step.V57, stageRange[stage]); + output.V58 = Av1Transform1dMath.Clamp(-step.V58 + step.V59, stageRange[stage]); + output.V59 = Av1Transform1dMath.Clamp(step.V58 + step.V59, stageRange[stage]); + output.V60 = Av1Transform1dMath.Clamp(step.V60 + step.V61, stageRange[stage]); + output.V61 = Av1Transform1dMath.Clamp(step.V60 - step.V61, stageRange[stage]); + output.V62 = Av1Transform1dMath.Clamp(-step.V62 + step.V63, stageRange[stage]); + output.V63 = Av1Transform1dMath.Clamp(step.V62 + step.V63, stageRange[stage]); + + // Stage 4 rotates the next odd-frequency level while preserving completed low-frequency lanes. + stage++; + step.V0 = output.V0; + step.V1 = output.V1; + step.V2 = output.V2; + step.V3 = output.V3; + step.V4 = output.V4; + step.V5 = output.V5; + step.V6 = output.V6; + step.V7 = output.V7; + step.V8 = Av1Transform1dMath.HalfButterfly(cospi[60], output.V8, -cospi[4], output.V15, cosBit); + step.V9 = Av1Transform1dMath.HalfButterfly(cospi[28], output.V9, -cospi[36], output.V14, cosBit); + step.V10 = Av1Transform1dMath.HalfButterfly(cospi[44], output.V10, -cospi[20], output.V13, cosBit); + step.V11 = Av1Transform1dMath.HalfButterfly(cospi[12], output.V11, -cospi[52], output.V12, cosBit); + step.V12 = Av1Transform1dMath.HalfButterfly(cospi[52], output.V11, cospi[12], output.V12, cosBit); + step.V13 = Av1Transform1dMath.HalfButterfly(cospi[20], output.V10, cospi[44], output.V13, cosBit); + step.V14 = Av1Transform1dMath.HalfButterfly(cospi[36], output.V9, cospi[28], output.V14, cosBit); + step.V15 = Av1Transform1dMath.HalfButterfly(cospi[4], output.V8, cospi[60], output.V15, cosBit); + step.V16 = Av1Transform1dMath.Clamp(output.V16 + output.V17, stageRange[stage]); + step.V17 = Av1Transform1dMath.Clamp(output.V16 - output.V17, stageRange[stage]); + step.V18 = Av1Transform1dMath.Clamp(-output.V18 + output.V19, stageRange[stage]); + step.V19 = Av1Transform1dMath.Clamp(output.V18 + output.V19, stageRange[stage]); + step.V20 = Av1Transform1dMath.Clamp(output.V20 + output.V21, stageRange[stage]); + step.V21 = Av1Transform1dMath.Clamp(output.V20 - output.V21, stageRange[stage]); + step.V22 = Av1Transform1dMath.Clamp(-output.V22 + output.V23, stageRange[stage]); + step.V23 = Av1Transform1dMath.Clamp(output.V22 + output.V23, stageRange[stage]); + step.V24 = Av1Transform1dMath.Clamp(output.V24 + output.V25, stageRange[stage]); + step.V25 = Av1Transform1dMath.Clamp(output.V24 - output.V25, stageRange[stage]); + step.V26 = Av1Transform1dMath.Clamp(-output.V26 + output.V27, stageRange[stage]); + step.V27 = Av1Transform1dMath.Clamp(output.V26 + output.V27, stageRange[stage]); + step.V28 = Av1Transform1dMath.Clamp(output.V28 + output.V29, stageRange[stage]); + step.V29 = Av1Transform1dMath.Clamp(output.V28 - output.V29, stageRange[stage]); + step.V30 = Av1Transform1dMath.Clamp(-output.V30 + output.V31, stageRange[stage]); + step.V31 = Av1Transform1dMath.Clamp(output.V30 + output.V31, stageRange[stage]); + step.V32 = output.V32; + step.V33 = Av1Transform1dMath.HalfButterfly(-cospi[4], output.V33, cospi[60], output.V62, cosBit); + step.V34 = Av1Transform1dMath.HalfButterfly(-cospi[60], output.V34, -cospi[4], output.V61, cosBit); + step.V35 = output.V35; + step.V36 = output.V36; + step.V37 = Av1Transform1dMath.HalfButterfly(-cospi[36], output.V37, cospi[28], output.V58, cosBit); + step.V38 = Av1Transform1dMath.HalfButterfly(-cospi[28], output.V38, -cospi[36], output.V57, cosBit); + step.V39 = output.V39; + step.V40 = output.V40; + step.V41 = Av1Transform1dMath.HalfButterfly(-cospi[20], output.V41, cospi[44], output.V54, cosBit); + step.V42 = Av1Transform1dMath.HalfButterfly(-cospi[44], output.V42, -cospi[20], output.V53, cosBit); + step.V43 = output.V43; + step.V44 = output.V44; + step.V45 = Av1Transform1dMath.HalfButterfly(-cospi[52], output.V45, cospi[12], output.V50, cosBit); + step.V46 = Av1Transform1dMath.HalfButterfly(-cospi[12], output.V46, -cospi[52], output.V49, cosBit); + step.V47 = output.V47; + step.V48 = output.V48; + step.V49 = Av1Transform1dMath.HalfButterfly(-cospi[52], output.V46, cospi[12], output.V49, cosBit); + step.V50 = Av1Transform1dMath.HalfButterfly(cospi[12], output.V45, cospi[52], output.V50, cosBit); + step.V51 = output.V51; + step.V52 = output.V52; + step.V53 = Av1Transform1dMath.HalfButterfly(-cospi[20], output.V42, cospi[44], output.V53, cosBit); + step.V54 = Av1Transform1dMath.HalfButterfly(cospi[44], output.V41, cospi[20], output.V54, cosBit); + step.V55 = output.V55; + step.V56 = output.V56; + step.V57 = Av1Transform1dMath.HalfButterfly(-cospi[36], output.V38, cospi[28], output.V57, cosBit); + step.V58 = Av1Transform1dMath.HalfButterfly(cospi[28], output.V37, cospi[36], output.V58, cosBit); + step.V59 = output.V59; + step.V60 = output.V60; + step.V61 = Av1Transform1dMath.HalfButterfly(-cospi[4], output.V34, cospi[60], output.V61, cosBit); + step.V62 = Av1Transform1dMath.HalfButterfly(cospi[60], output.V33, cospi[4], output.V62, cosBit); + step.V63 = output.V63; + + // Stage 5 widens the nested groups through the next butterfly level. + stage++; + output.V0 = step.V0; + output.V1 = step.V1; + output.V2 = step.V2; + output.V3 = step.V3; + output.V4 = Av1Transform1dMath.HalfButterfly(cospi[56], step.V4, -cospi[8], step.V7, cosBit); + output.V5 = Av1Transform1dMath.HalfButterfly(cospi[24], step.V5, -cospi[40], step.V6, cosBit); + output.V6 = Av1Transform1dMath.HalfButterfly(cospi[40], step.V5, cospi[24], step.V6, cosBit); + output.V7 = Av1Transform1dMath.HalfButterfly(cospi[8], step.V4, cospi[56], step.V7, cosBit); + output.V8 = Av1Transform1dMath.Clamp(step.V8 + step.V9, stageRange[stage]); + output.V9 = Av1Transform1dMath.Clamp(step.V8 - step.V9, stageRange[stage]); + output.V10 = Av1Transform1dMath.Clamp(-step.V10 + step.V11, stageRange[stage]); + output.V11 = Av1Transform1dMath.Clamp(step.V10 + step.V11, stageRange[stage]); + output.V12 = Av1Transform1dMath.Clamp(step.V12 + step.V13, stageRange[stage]); + output.V13 = Av1Transform1dMath.Clamp(step.V12 - step.V13, stageRange[stage]); + output.V14 = Av1Transform1dMath.Clamp(-step.V14 + step.V15, stageRange[stage]); + output.V15 = Av1Transform1dMath.Clamp(step.V14 + step.V15, stageRange[stage]); + output.V16 = step.V16; + output.V17 = Av1Transform1dMath.HalfButterfly(-cospi[8], step.V17, cospi[56], step.V30, cosBit); + output.V18 = Av1Transform1dMath.HalfButterfly(-cospi[56], step.V18, -cospi[8], step.V29, cosBit); + output.V19 = step.V19; + output.V20 = step.V20; + output.V21 = Av1Transform1dMath.HalfButterfly(-cospi[40], step.V21, cospi[24], step.V26, cosBit); + output.V22 = Av1Transform1dMath.HalfButterfly(-cospi[24], step.V22, -cospi[40], step.V25, cosBit); + output.V23 = step.V23; + output.V24 = step.V24; + output.V25 = Av1Transform1dMath.HalfButterfly(-cospi[40], step.V22, cospi[24], step.V25, cosBit); + output.V26 = Av1Transform1dMath.HalfButterfly(cospi[24], step.V21, cospi[40], step.V26, cosBit); + output.V27 = step.V27; + output.V28 = step.V28; + output.V29 = Av1Transform1dMath.HalfButterfly(-cospi[8], step.V18, cospi[56], step.V29, cosBit); + output.V30 = Av1Transform1dMath.HalfButterfly(cospi[56], step.V17, cospi[8], step.V30, cosBit); + output.V31 = step.V31; + output.V32 = Av1Transform1dMath.Clamp(step.V32 + step.V35, stageRange[stage]); + output.V33 = Av1Transform1dMath.Clamp(step.V33 + step.V34, stageRange[stage]); + output.V34 = Av1Transform1dMath.Clamp(step.V33 - step.V34, stageRange[stage]); + output.V35 = Av1Transform1dMath.Clamp(step.V32 - step.V35, stageRange[stage]); + output.V36 = Av1Transform1dMath.Clamp(-step.V36 + step.V39, stageRange[stage]); + output.V37 = Av1Transform1dMath.Clamp(-step.V37 + step.V38, stageRange[stage]); + output.V38 = Av1Transform1dMath.Clamp(step.V37 + step.V38, stageRange[stage]); + output.V39 = Av1Transform1dMath.Clamp(step.V36 + step.V39, stageRange[stage]); + output.V40 = Av1Transform1dMath.Clamp(step.V40 + step.V43, stageRange[stage]); + output.V41 = Av1Transform1dMath.Clamp(step.V41 + step.V42, stageRange[stage]); + output.V42 = Av1Transform1dMath.Clamp(step.V41 - step.V42, stageRange[stage]); + output.V43 = Av1Transform1dMath.Clamp(step.V40 - step.V43, stageRange[stage]); + output.V44 = Av1Transform1dMath.Clamp(-step.V44 + step.V47, stageRange[stage]); + output.V45 = Av1Transform1dMath.Clamp(-step.V45 + step.V46, stageRange[stage]); + output.V46 = Av1Transform1dMath.Clamp(step.V45 + step.V46, stageRange[stage]); + output.V47 = Av1Transform1dMath.Clamp(step.V44 + step.V47, stageRange[stage]); + output.V48 = Av1Transform1dMath.Clamp(step.V48 + step.V51, stageRange[stage]); + output.V49 = Av1Transform1dMath.Clamp(step.V49 + step.V50, stageRange[stage]); + output.V50 = Av1Transform1dMath.Clamp(step.V49 - step.V50, stageRange[stage]); + output.V51 = Av1Transform1dMath.Clamp(step.V48 - step.V51, stageRange[stage]); + output.V52 = Av1Transform1dMath.Clamp(-step.V52 + step.V55, stageRange[stage]); + output.V53 = Av1Transform1dMath.Clamp(-step.V53 + step.V54, stageRange[stage]); + output.V54 = Av1Transform1dMath.Clamp(step.V53 + step.V54, stageRange[stage]); + output.V55 = Av1Transform1dMath.Clamp(step.V52 + step.V55, stageRange[stage]); + output.V56 = Av1Transform1dMath.Clamp(step.V56 + step.V59, stageRange[stage]); + output.V57 = Av1Transform1dMath.Clamp(step.V57 + step.V58, stageRange[stage]); + output.V58 = Av1Transform1dMath.Clamp(step.V57 - step.V58, stageRange[stage]); + output.V59 = Av1Transform1dMath.Clamp(step.V56 - step.V59, stageRange[stage]); + output.V60 = Av1Transform1dMath.Clamp(-step.V60 + step.V63, stageRange[stage]); + output.V61 = Av1Transform1dMath.Clamp(-step.V61 + step.V62, stageRange[stage]); + output.V62 = Av1Transform1dMath.Clamp(step.V61 + step.V62, stageRange[stage]); + output.V63 = Av1Transform1dMath.Clamp(step.V60 + step.V63, stageRange[stage]); + + // Stage 6 rotates the next odd-frequency level while preserving completed low-frequency lanes. + stage++; + step.V0 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V0, cospi[32], output.V1, cosBit); + step.V1 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V0, -cospi[32], output.V1, cosBit); + step.V2 = Av1Transform1dMath.HalfButterfly(cospi[48], output.V2, -cospi[16], output.V3, cosBit); + step.V3 = Av1Transform1dMath.HalfButterfly(cospi[16], output.V2, cospi[48], output.V3, cosBit); + step.V4 = Av1Transform1dMath.Clamp(output.V4 + output.V5, stageRange[stage]); + step.V5 = Av1Transform1dMath.Clamp(output.V4 - output.V5, stageRange[stage]); + step.V6 = Av1Transform1dMath.Clamp(-output.V6 + output.V7, stageRange[stage]); + step.V7 = Av1Transform1dMath.Clamp(output.V6 + output.V7, stageRange[stage]); + step.V8 = output.V8; + step.V9 = Av1Transform1dMath.HalfButterfly(-cospi[16], output.V9, cospi[48], output.V14, cosBit); + step.V10 = Av1Transform1dMath.HalfButterfly(-cospi[48], output.V10, -cospi[16], output.V13, cosBit); + step.V11 = output.V11; + step.V12 = output.V12; + step.V13 = Av1Transform1dMath.HalfButterfly(-cospi[16], output.V10, cospi[48], output.V13, cosBit); + step.V14 = Av1Transform1dMath.HalfButterfly(cospi[48], output.V9, cospi[16], output.V14, cosBit); + step.V15 = output.V15; + step.V16 = Av1Transform1dMath.Clamp(output.V16 + output.V19, stageRange[stage]); + step.V17 = Av1Transform1dMath.Clamp(output.V17 + output.V18, stageRange[stage]); + step.V18 = Av1Transform1dMath.Clamp(output.V17 - output.V18, stageRange[stage]); + step.V19 = Av1Transform1dMath.Clamp(output.V16 - output.V19, stageRange[stage]); + step.V20 = Av1Transform1dMath.Clamp(-output.V20 + output.V23, stageRange[stage]); + step.V21 = Av1Transform1dMath.Clamp(-output.V21 + output.V22, stageRange[stage]); + step.V22 = Av1Transform1dMath.Clamp(output.V21 + output.V22, stageRange[stage]); + step.V23 = Av1Transform1dMath.Clamp(output.V20 + output.V23, stageRange[stage]); + step.V24 = Av1Transform1dMath.Clamp(output.V24 + output.V27, stageRange[stage]); + step.V25 = Av1Transform1dMath.Clamp(output.V25 + output.V26, stageRange[stage]); + step.V26 = Av1Transform1dMath.Clamp(output.V25 - output.V26, stageRange[stage]); + step.V27 = Av1Transform1dMath.Clamp(output.V24 - output.V27, stageRange[stage]); + step.V28 = Av1Transform1dMath.Clamp(-output.V28 + output.V31, stageRange[stage]); + step.V29 = Av1Transform1dMath.Clamp(-output.V29 + output.V30, stageRange[stage]); + step.V30 = Av1Transform1dMath.Clamp(output.V29 + output.V30, stageRange[stage]); + step.V31 = Av1Transform1dMath.Clamp(output.V28 + output.V31, stageRange[stage]); + step.V32 = output.V32; + step.V33 = output.V33; + step.V34 = Av1Transform1dMath.HalfButterfly(-cospi[8], output.V34, cospi[56], output.V61, cosBit); + step.V35 = Av1Transform1dMath.HalfButterfly(-cospi[8], output.V35, cospi[56], output.V60, cosBit); + step.V36 = Av1Transform1dMath.HalfButterfly(-cospi[56], output.V36, -cospi[8], output.V59, cosBit); + step.V37 = Av1Transform1dMath.HalfButterfly(-cospi[56], output.V37, -cospi[8], output.V58, cosBit); + step.V38 = output.V38; + step.V39 = output.V39; + step.V40 = output.V40; + step.V41 = output.V41; + step.V42 = Av1Transform1dMath.HalfButterfly(-cospi[40], output.V42, cospi[24], output.V53, cosBit); + step.V43 = Av1Transform1dMath.HalfButterfly(-cospi[40], output.V43, cospi[24], output.V52, cosBit); + step.V44 = Av1Transform1dMath.HalfButterfly(-cospi[24], output.V44, -cospi[40], output.V51, cosBit); + step.V45 = Av1Transform1dMath.HalfButterfly(-cospi[24], output.V45, -cospi[40], output.V50, cosBit); + step.V46 = output.V46; + step.V47 = output.V47; + step.V48 = output.V48; + step.V49 = output.V49; + step.V50 = Av1Transform1dMath.HalfButterfly(-cospi[40], output.V45, cospi[24], output.V50, cosBit); + step.V51 = Av1Transform1dMath.HalfButterfly(-cospi[40], output.V44, cospi[24], output.V51, cosBit); + step.V52 = Av1Transform1dMath.HalfButterfly(cospi[24], output.V43, cospi[40], output.V52, cosBit); + step.V53 = Av1Transform1dMath.HalfButterfly(cospi[24], output.V42, cospi[40], output.V53, cosBit); + step.V54 = output.V54; + step.V55 = output.V55; + step.V56 = output.V56; + step.V57 = output.V57; + step.V58 = Av1Transform1dMath.HalfButterfly(-cospi[8], output.V37, cospi[56], output.V58, cosBit); + step.V59 = Av1Transform1dMath.HalfButterfly(-cospi[8], output.V36, cospi[56], output.V59, cosBit); + step.V60 = Av1Transform1dMath.HalfButterfly(cospi[56], output.V35, cospi[8], output.V60, cosBit); + step.V61 = Av1Transform1dMath.HalfButterfly(cospi[56], output.V34, cospi[8], output.V61, cosBit); + step.V62 = output.V62; + step.V63 = output.V63; + + // Stage 7 reconstructs the embedded sixteen-point groups and combines adjacent odd terms. + stage++; + output.V0 = Av1Transform1dMath.Clamp(step.V0 + step.V3, stageRange[stage]); + output.V1 = Av1Transform1dMath.Clamp(step.V1 + step.V2, stageRange[stage]); + output.V2 = Av1Transform1dMath.Clamp(step.V1 - step.V2, stageRange[stage]); + output.V3 = Av1Transform1dMath.Clamp(step.V0 - step.V3, stageRange[stage]); + output.V4 = step.V4; + output.V5 = Av1Transform1dMath.HalfButterfly(-cospi[32], step.V5, cospi[32], step.V6, cosBit); + output.V6 = Av1Transform1dMath.HalfButterfly(cospi[32], step.V5, cospi[32], step.V6, cosBit); + output.V7 = step.V7; + output.V8 = Av1Transform1dMath.Clamp(step.V8 + step.V11, stageRange[stage]); + output.V9 = Av1Transform1dMath.Clamp(step.V9 + step.V10, stageRange[stage]); + output.V10 = Av1Transform1dMath.Clamp(step.V9 - step.V10, stageRange[stage]); + output.V11 = Av1Transform1dMath.Clamp(step.V8 - step.V11, stageRange[stage]); + output.V12 = Av1Transform1dMath.Clamp(-step.V12 + step.V15, stageRange[stage]); + output.V13 = Av1Transform1dMath.Clamp(-step.V13 + step.V14, stageRange[stage]); + output.V14 = Av1Transform1dMath.Clamp(step.V13 + step.V14, stageRange[stage]); + output.V15 = Av1Transform1dMath.Clamp(step.V12 + step.V15, stageRange[stage]); + output.V16 = step.V16; + output.V17 = step.V17; + output.V18 = Av1Transform1dMath.HalfButterfly(-cospi[16], step.V18, cospi[48], step.V29, cosBit); + output.V19 = Av1Transform1dMath.HalfButterfly(-cospi[16], step.V19, cospi[48], step.V28, cosBit); + output.V20 = Av1Transform1dMath.HalfButterfly(-cospi[48], step.V20, -cospi[16], step.V27, cosBit); + output.V21 = Av1Transform1dMath.HalfButterfly(-cospi[48], step.V21, -cospi[16], step.V26, cosBit); + output.V22 = step.V22; + output.V23 = step.V23; + output.V24 = step.V24; + output.V25 = step.V25; + output.V26 = Av1Transform1dMath.HalfButterfly(-cospi[16], step.V21, cospi[48], step.V26, cosBit); + output.V27 = Av1Transform1dMath.HalfButterfly(-cospi[16], step.V20, cospi[48], step.V27, cosBit); + output.V28 = Av1Transform1dMath.HalfButterfly(cospi[48], step.V19, cospi[16], step.V28, cosBit); + output.V29 = Av1Transform1dMath.HalfButterfly(cospi[48], step.V18, cospi[16], step.V29, cosBit); + output.V30 = step.V30; + output.V31 = step.V31; + output.V32 = Av1Transform1dMath.Clamp(step.V32 + step.V39, stageRange[stage]); + output.V33 = Av1Transform1dMath.Clamp(step.V33 + step.V38, stageRange[stage]); + output.V34 = Av1Transform1dMath.Clamp(step.V34 + step.V37, stageRange[stage]); + output.V35 = Av1Transform1dMath.Clamp(step.V35 + step.V36, stageRange[stage]); + output.V36 = Av1Transform1dMath.Clamp(step.V35 - step.V36, stageRange[stage]); + output.V37 = Av1Transform1dMath.Clamp(step.V34 - step.V37, stageRange[stage]); + output.V38 = Av1Transform1dMath.Clamp(step.V33 - step.V38, stageRange[stage]); + output.V39 = Av1Transform1dMath.Clamp(step.V32 - step.V39, stageRange[stage]); + output.V40 = Av1Transform1dMath.Clamp(-step.V40 + step.V47, stageRange[stage]); + output.V41 = Av1Transform1dMath.Clamp(-step.V41 + step.V46, stageRange[stage]); + output.V42 = Av1Transform1dMath.Clamp(-step.V42 + step.V45, stageRange[stage]); + output.V43 = Av1Transform1dMath.Clamp(-step.V43 + step.V44, stageRange[stage]); + output.V44 = Av1Transform1dMath.Clamp(step.V43 + step.V44, stageRange[stage]); + output.V45 = Av1Transform1dMath.Clamp(step.V42 + step.V45, stageRange[stage]); + output.V46 = Av1Transform1dMath.Clamp(step.V41 + step.V46, stageRange[stage]); + output.V47 = Av1Transform1dMath.Clamp(step.V40 + step.V47, stageRange[stage]); + output.V48 = Av1Transform1dMath.Clamp(step.V48 + step.V55, stageRange[stage]); + output.V49 = Av1Transform1dMath.Clamp(step.V49 + step.V54, stageRange[stage]); + output.V50 = Av1Transform1dMath.Clamp(step.V50 + step.V53, stageRange[stage]); + output.V51 = Av1Transform1dMath.Clamp(step.V51 + step.V52, stageRange[stage]); + output.V52 = Av1Transform1dMath.Clamp(step.V51 - step.V52, stageRange[stage]); + output.V53 = Av1Transform1dMath.Clamp(step.V50 - step.V53, stageRange[stage]); + output.V54 = Av1Transform1dMath.Clamp(step.V49 - step.V54, stageRange[stage]); + output.V55 = Av1Transform1dMath.Clamp(step.V48 - step.V55, stageRange[stage]); + output.V56 = Av1Transform1dMath.Clamp(-step.V56 + step.V63, stageRange[stage]); + output.V57 = Av1Transform1dMath.Clamp(-step.V57 + step.V62, stageRange[stage]); + output.V58 = Av1Transform1dMath.Clamp(-step.V58 + step.V61, stageRange[stage]); + output.V59 = Av1Transform1dMath.Clamp(-step.V59 + step.V60, stageRange[stage]); + output.V60 = Av1Transform1dMath.Clamp(step.V59 + step.V60, stageRange[stage]); + output.V61 = Av1Transform1dMath.Clamp(step.V58 + step.V61, stageRange[stage]); + output.V62 = Av1Transform1dMath.Clamp(step.V57 + step.V62, stageRange[stage]); + output.V63 = Av1Transform1dMath.Clamp(step.V56 + step.V63, stageRange[stage]); + + // Stage 8 completes the embedded eight-point groups and rotates their odd-frequency pairs. + stage++; + step.V0 = Av1Transform1dMath.Clamp(output.V0 + output.V7, stageRange[stage]); + step.V1 = Av1Transform1dMath.Clamp(output.V1 + output.V6, stageRange[stage]); + step.V2 = Av1Transform1dMath.Clamp(output.V2 + output.V5, stageRange[stage]); + step.V3 = Av1Transform1dMath.Clamp(output.V3 + output.V4, stageRange[stage]); + step.V4 = Av1Transform1dMath.Clamp(output.V3 - output.V4, stageRange[stage]); + step.V5 = Av1Transform1dMath.Clamp(output.V2 - output.V5, stageRange[stage]); + step.V6 = Av1Transform1dMath.Clamp(output.V1 - output.V6, stageRange[stage]); + step.V7 = Av1Transform1dMath.Clamp(output.V0 - output.V7, stageRange[stage]); + step.V8 = output.V8; + step.V9 = output.V9; + step.V10 = Av1Transform1dMath.HalfButterfly(-cospi[32], output.V10, cospi[32], output.V13, cosBit); + step.V11 = Av1Transform1dMath.HalfButterfly(-cospi[32], output.V11, cospi[32], output.V12, cosBit); + step.V12 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V11, cospi[32], output.V12, cosBit); + step.V13 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V10, cospi[32], output.V13, cosBit); + step.V14 = output.V14; + step.V15 = output.V15; + step.V16 = Av1Transform1dMath.Clamp(output.V16 + output.V23, stageRange[stage]); + step.V17 = Av1Transform1dMath.Clamp(output.V17 + output.V22, stageRange[stage]); + step.V18 = Av1Transform1dMath.Clamp(output.V18 + output.V21, stageRange[stage]); + step.V19 = Av1Transform1dMath.Clamp(output.V19 + output.V20, stageRange[stage]); + step.V20 = Av1Transform1dMath.Clamp(output.V19 - output.V20, stageRange[stage]); + step.V21 = Av1Transform1dMath.Clamp(output.V18 - output.V21, stageRange[stage]); + step.V22 = Av1Transform1dMath.Clamp(output.V17 - output.V22, stageRange[stage]); + step.V23 = Av1Transform1dMath.Clamp(output.V16 - output.V23, stageRange[stage]); + step.V24 = Av1Transform1dMath.Clamp(-output.V24 + output.V31, stageRange[stage]); + step.V25 = Av1Transform1dMath.Clamp(-output.V25 + output.V30, stageRange[stage]); + step.V26 = Av1Transform1dMath.Clamp(-output.V26 + output.V29, stageRange[stage]); + step.V27 = Av1Transform1dMath.Clamp(-output.V27 + output.V28, stageRange[stage]); + step.V28 = Av1Transform1dMath.Clamp(output.V27 + output.V28, stageRange[stage]); + step.V29 = Av1Transform1dMath.Clamp(output.V26 + output.V29, stageRange[stage]); + step.V30 = Av1Transform1dMath.Clamp(output.V25 + output.V30, stageRange[stage]); + step.V31 = Av1Transform1dMath.Clamp(output.V24 + output.V31, stageRange[stage]); + step.V32 = output.V32; + step.V33 = output.V33; + step.V34 = output.V34; + step.V35 = output.V35; + step.V36 = Av1Transform1dMath.HalfButterfly(-cospi[16], output.V36, cospi[48], output.V59, cosBit); + step.V37 = Av1Transform1dMath.HalfButterfly(-cospi[16], output.V37, cospi[48], output.V58, cosBit); + step.V38 = Av1Transform1dMath.HalfButterfly(-cospi[16], output.V38, cospi[48], output.V57, cosBit); + step.V39 = Av1Transform1dMath.HalfButterfly(-cospi[16], output.V39, cospi[48], output.V56, cosBit); + step.V40 = Av1Transform1dMath.HalfButterfly(-cospi[48], output.V40, -cospi[16], output.V55, cosBit); + step.V41 = Av1Transform1dMath.HalfButterfly(-cospi[48], output.V41, -cospi[16], output.V54, cosBit); + step.V42 = Av1Transform1dMath.HalfButterfly(-cospi[48], output.V42, -cospi[16], output.V53, cosBit); + step.V43 = Av1Transform1dMath.HalfButterfly(-cospi[48], output.V43, -cospi[16], output.V52, cosBit); + step.V44 = output.V44; + step.V45 = output.V45; + step.V46 = output.V46; + step.V47 = output.V47; + step.V48 = output.V48; + step.V49 = output.V49; + step.V50 = output.V50; + step.V51 = output.V51; + step.V52 = Av1Transform1dMath.HalfButterfly(-cospi[16], output.V43, cospi[48], output.V52, cosBit); + step.V53 = Av1Transform1dMath.HalfButterfly(-cospi[16], output.V42, cospi[48], output.V53, cosBit); + step.V54 = Av1Transform1dMath.HalfButterfly(-cospi[16], output.V41, cospi[48], output.V54, cosBit); + step.V55 = Av1Transform1dMath.HalfButterfly(-cospi[16], output.V40, cospi[48], output.V55, cosBit); + step.V56 = Av1Transform1dMath.HalfButterfly(cospi[48], output.V39, cospi[16], output.V56, cosBit); + step.V57 = Av1Transform1dMath.HalfButterfly(cospi[48], output.V38, cospi[16], output.V57, cosBit); + step.V58 = Av1Transform1dMath.HalfButterfly(cospi[48], output.V37, cospi[16], output.V58, cosBit); + step.V59 = Av1Transform1dMath.HalfButterfly(cospi[48], output.V36, cospi[16], output.V59, cosBit); + step.V60 = output.V60; + step.V61 = output.V61; + step.V62 = output.V62; + step.V63 = output.V63; + + // Stage 9 widens the reconstructed groups through their next butterfly level. + stage++; + output.V0 = Av1Transform1dMath.Clamp(step.V0 + step.V15, stageRange[stage]); + output.V1 = Av1Transform1dMath.Clamp(step.V1 + step.V14, stageRange[stage]); + output.V2 = Av1Transform1dMath.Clamp(step.V2 + step.V13, stageRange[stage]); + output.V3 = Av1Transform1dMath.Clamp(step.V3 + step.V12, stageRange[stage]); + output.V4 = Av1Transform1dMath.Clamp(step.V4 + step.V11, stageRange[stage]); + output.V5 = Av1Transform1dMath.Clamp(step.V5 + step.V10, stageRange[stage]); + output.V6 = Av1Transform1dMath.Clamp(step.V6 + step.V9, stageRange[stage]); + output.V7 = Av1Transform1dMath.Clamp(step.V7 + step.V8, stageRange[stage]); + output.V8 = Av1Transform1dMath.Clamp(step.V7 - step.V8, stageRange[stage]); + output.V9 = Av1Transform1dMath.Clamp(step.V6 - step.V9, stageRange[stage]); + output.V10 = Av1Transform1dMath.Clamp(step.V5 - step.V10, stageRange[stage]); + output.V11 = Av1Transform1dMath.Clamp(step.V4 - step.V11, stageRange[stage]); + output.V12 = Av1Transform1dMath.Clamp(step.V3 - step.V12, stageRange[stage]); + output.V13 = Av1Transform1dMath.Clamp(step.V2 - step.V13, stageRange[stage]); + output.V14 = Av1Transform1dMath.Clamp(step.V1 - step.V14, stageRange[stage]); + output.V15 = Av1Transform1dMath.Clamp(step.V0 - step.V15, stageRange[stage]); + output.V16 = step.V16; + output.V17 = step.V17; + output.V18 = step.V18; + output.V19 = step.V19; + output.V20 = Av1Transform1dMath.HalfButterfly(-cospi[32], step.V20, cospi[32], step.V27, cosBit); + output.V21 = Av1Transform1dMath.HalfButterfly(-cospi[32], step.V21, cospi[32], step.V26, cosBit); + output.V22 = Av1Transform1dMath.HalfButterfly(-cospi[32], step.V22, cospi[32], step.V25, cosBit); + output.V23 = Av1Transform1dMath.HalfButterfly(-cospi[32], step.V23, cospi[32], step.V24, cosBit); + output.V24 = Av1Transform1dMath.HalfButterfly(cospi[32], step.V23, cospi[32], step.V24, cosBit); + output.V25 = Av1Transform1dMath.HalfButterfly(cospi[32], step.V22, cospi[32], step.V25, cosBit); + output.V26 = Av1Transform1dMath.HalfButterfly(cospi[32], step.V21, cospi[32], step.V26, cosBit); + output.V27 = Av1Transform1dMath.HalfButterfly(cospi[32], step.V20, cospi[32], step.V27, cosBit); + output.V28 = step.V28; + output.V29 = step.V29; + output.V30 = step.V30; + output.V31 = step.V31; + output.V32 = Av1Transform1dMath.Clamp(step.V32 + step.V47, stageRange[stage]); + output.V33 = Av1Transform1dMath.Clamp(step.V33 + step.V46, stageRange[stage]); + output.V34 = Av1Transform1dMath.Clamp(step.V34 + step.V45, stageRange[stage]); + output.V35 = Av1Transform1dMath.Clamp(step.V35 + step.V44, stageRange[stage]); + output.V36 = Av1Transform1dMath.Clamp(step.V36 + step.V43, stageRange[stage]); + output.V37 = Av1Transform1dMath.Clamp(step.V37 + step.V42, stageRange[stage]); + output.V38 = Av1Transform1dMath.Clamp(step.V38 + step.V41, stageRange[stage]); + output.V39 = Av1Transform1dMath.Clamp(step.V39 + step.V40, stageRange[stage]); + output.V40 = Av1Transform1dMath.Clamp(step.V39 - step.V40, stageRange[stage]); + output.V41 = Av1Transform1dMath.Clamp(step.V38 - step.V41, stageRange[stage]); + output.V42 = Av1Transform1dMath.Clamp(step.V37 - step.V42, stageRange[stage]); + output.V43 = Av1Transform1dMath.Clamp(step.V36 - step.V43, stageRange[stage]); + output.V44 = Av1Transform1dMath.Clamp(step.V35 - step.V44, stageRange[stage]); + output.V45 = Av1Transform1dMath.Clamp(step.V34 - step.V45, stageRange[stage]); + output.V46 = Av1Transform1dMath.Clamp(step.V33 - step.V46, stageRange[stage]); + output.V47 = Av1Transform1dMath.Clamp(step.V32 - step.V47, stageRange[stage]); + output.V48 = Av1Transform1dMath.Clamp(-step.V48 + step.V63, stageRange[stage]); + output.V49 = Av1Transform1dMath.Clamp(-step.V49 + step.V62, stageRange[stage]); + output.V50 = Av1Transform1dMath.Clamp(-step.V50 + step.V61, stageRange[stage]); + output.V51 = Av1Transform1dMath.Clamp(-step.V51 + step.V60, stageRange[stage]); + output.V52 = Av1Transform1dMath.Clamp(-step.V52 + step.V59, stageRange[stage]); + output.V53 = Av1Transform1dMath.Clamp(-step.V53 + step.V58, stageRange[stage]); + output.V54 = Av1Transform1dMath.Clamp(-step.V54 + step.V57, stageRange[stage]); + output.V55 = Av1Transform1dMath.Clamp(-step.V55 + step.V56, stageRange[stage]); + output.V56 = Av1Transform1dMath.Clamp(step.V55 + step.V56, stageRange[stage]); + output.V57 = Av1Transform1dMath.Clamp(step.V54 + step.V57, stageRange[stage]); + output.V58 = Av1Transform1dMath.Clamp(step.V53 + step.V58, stageRange[stage]); + output.V59 = Av1Transform1dMath.Clamp(step.V52 + step.V59, stageRange[stage]); + output.V60 = Av1Transform1dMath.Clamp(step.V51 + step.V60, stageRange[stage]); + output.V61 = Av1Transform1dMath.Clamp(step.V50 + step.V61, stageRange[stage]); + output.V62 = Av1Transform1dMath.Clamp(step.V49 + step.V62, stageRange[stage]); + output.V63 = Av1Transform1dMath.Clamp(step.V48 + step.V63, stageRange[stage]); + + // Stage 10 applies the remaining pi/4 rotations before the terminal spatial merge. + stage++; + step.V0 = Av1Transform1dMath.Clamp(output.V0 + output.V31, stageRange[stage]); + step.V1 = Av1Transform1dMath.Clamp(output.V1 + output.V30, stageRange[stage]); + step.V2 = Av1Transform1dMath.Clamp(output.V2 + output.V29, stageRange[stage]); + step.V3 = Av1Transform1dMath.Clamp(output.V3 + output.V28, stageRange[stage]); + step.V4 = Av1Transform1dMath.Clamp(output.V4 + output.V27, stageRange[stage]); + step.V5 = Av1Transform1dMath.Clamp(output.V5 + output.V26, stageRange[stage]); + step.V6 = Av1Transform1dMath.Clamp(output.V6 + output.V25, stageRange[stage]); + step.V7 = Av1Transform1dMath.Clamp(output.V7 + output.V24, stageRange[stage]); + step.V8 = Av1Transform1dMath.Clamp(output.V8 + output.V23, stageRange[stage]); + step.V9 = Av1Transform1dMath.Clamp(output.V9 + output.V22, stageRange[stage]); + step.V10 = Av1Transform1dMath.Clamp(output.V10 + output.V21, stageRange[stage]); + step.V11 = Av1Transform1dMath.Clamp(output.V11 + output.V20, stageRange[stage]); + step.V12 = Av1Transform1dMath.Clamp(output.V12 + output.V19, stageRange[stage]); + step.V13 = Av1Transform1dMath.Clamp(output.V13 + output.V18, stageRange[stage]); + step.V14 = Av1Transform1dMath.Clamp(output.V14 + output.V17, stageRange[stage]); + step.V15 = Av1Transform1dMath.Clamp(output.V15 + output.V16, stageRange[stage]); + step.V16 = Av1Transform1dMath.Clamp(output.V15 - output.V16, stageRange[stage]); + step.V17 = Av1Transform1dMath.Clamp(output.V14 - output.V17, stageRange[stage]); + step.V18 = Av1Transform1dMath.Clamp(output.V13 - output.V18, stageRange[stage]); + step.V19 = Av1Transform1dMath.Clamp(output.V12 - output.V19, stageRange[stage]); + step.V20 = Av1Transform1dMath.Clamp(output.V11 - output.V20, stageRange[stage]); + step.V21 = Av1Transform1dMath.Clamp(output.V10 - output.V21, stageRange[stage]); + step.V22 = Av1Transform1dMath.Clamp(output.V9 - output.V22, stageRange[stage]); + step.V23 = Av1Transform1dMath.Clamp(output.V8 - output.V23, stageRange[stage]); + step.V24 = Av1Transform1dMath.Clamp(output.V7 - output.V24, stageRange[stage]); + step.V25 = Av1Transform1dMath.Clamp(output.V6 - output.V25, stageRange[stage]); + step.V26 = Av1Transform1dMath.Clamp(output.V5 - output.V26, stageRange[stage]); + step.V27 = Av1Transform1dMath.Clamp(output.V4 - output.V27, stageRange[stage]); + step.V28 = Av1Transform1dMath.Clamp(output.V3 - output.V28, stageRange[stage]); + step.V29 = Av1Transform1dMath.Clamp(output.V2 - output.V29, stageRange[stage]); + step.V30 = Av1Transform1dMath.Clamp(output.V1 - output.V30, stageRange[stage]); + step.V31 = Av1Transform1dMath.Clamp(output.V0 - output.V31, stageRange[stage]); + step.V32 = output.V32; + step.V33 = output.V33; + step.V34 = output.V34; + step.V35 = output.V35; + step.V36 = output.V36; + step.V37 = output.V37; + step.V38 = output.V38; + step.V39 = output.V39; + step.V40 = Av1Transform1dMath.HalfButterfly(-cospi[32], output.V40, cospi[32], output.V55, cosBit); + step.V41 = Av1Transform1dMath.HalfButterfly(-cospi[32], output.V41, cospi[32], output.V54, cosBit); + step.V42 = Av1Transform1dMath.HalfButterfly(-cospi[32], output.V42, cospi[32], output.V53, cosBit); + step.V43 = Av1Transform1dMath.HalfButterfly(-cospi[32], output.V43, cospi[32], output.V52, cosBit); + step.V44 = Av1Transform1dMath.HalfButterfly(-cospi[32], output.V44, cospi[32], output.V51, cosBit); + step.V45 = Av1Transform1dMath.HalfButterfly(-cospi[32], output.V45, cospi[32], output.V50, cosBit); + step.V46 = Av1Transform1dMath.HalfButterfly(-cospi[32], output.V46, cospi[32], output.V49, cosBit); + step.V47 = Av1Transform1dMath.HalfButterfly(-cospi[32], output.V47, cospi[32], output.V48, cosBit); + step.V48 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V47, cospi[32], output.V48, cosBit); + step.V49 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V46, cospi[32], output.V49, cosBit); + step.V50 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V45, cospi[32], output.V50, cosBit); + step.V51 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V44, cospi[32], output.V51, cosBit); + step.V52 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V43, cospi[32], output.V52, cosBit); + step.V53 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V42, cospi[32], output.V53, cosBit); + step.V54 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V41, cospi[32], output.V54, cosBit); + step.V55 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V40, cospi[32], output.V55, cosBit); + step.V56 = output.V56; + step.V57 = output.V57; + step.V58 = output.V58; + step.V59 = output.V59; + step.V60 = output.V60; + step.V61 = output.V61; + step.V62 = output.V62; + step.V63 = output.V63; + + // Stage 11 merges the even and odd halves into spatial order and clamps every result. + stage++; + output.V0 = Av1Transform1dMath.Clamp(step.V0 + step.V63, stageRange[stage]); + output.V1 = Av1Transform1dMath.Clamp(step.V1 + step.V62, stageRange[stage]); + output.V2 = Av1Transform1dMath.Clamp(step.V2 + step.V61, stageRange[stage]); + output.V3 = Av1Transform1dMath.Clamp(step.V3 + step.V60, stageRange[stage]); + output.V4 = Av1Transform1dMath.Clamp(step.V4 + step.V59, stageRange[stage]); + output.V5 = Av1Transform1dMath.Clamp(step.V5 + step.V58, stageRange[stage]); + output.V6 = Av1Transform1dMath.Clamp(step.V6 + step.V57, stageRange[stage]); + output.V7 = Av1Transform1dMath.Clamp(step.V7 + step.V56, stageRange[stage]); + output.V8 = Av1Transform1dMath.Clamp(step.V8 + step.V55, stageRange[stage]); + output.V9 = Av1Transform1dMath.Clamp(step.V9 + step.V54, stageRange[stage]); + output.V10 = Av1Transform1dMath.Clamp(step.V10 + step.V53, stageRange[stage]); + output.V11 = Av1Transform1dMath.Clamp(step.V11 + step.V52, stageRange[stage]); + output.V12 = Av1Transform1dMath.Clamp(step.V12 + step.V51, stageRange[stage]); + output.V13 = Av1Transform1dMath.Clamp(step.V13 + step.V50, stageRange[stage]); + output.V14 = Av1Transform1dMath.Clamp(step.V14 + step.V49, stageRange[stage]); + output.V15 = Av1Transform1dMath.Clamp(step.V15 + step.V48, stageRange[stage]); + output.V16 = Av1Transform1dMath.Clamp(step.V16 + step.V47, stageRange[stage]); + output.V17 = Av1Transform1dMath.Clamp(step.V17 + step.V46, stageRange[stage]); + output.V18 = Av1Transform1dMath.Clamp(step.V18 + step.V45, stageRange[stage]); + output.V19 = Av1Transform1dMath.Clamp(step.V19 + step.V44, stageRange[stage]); + output.V20 = Av1Transform1dMath.Clamp(step.V20 + step.V43, stageRange[stage]); + output.V21 = Av1Transform1dMath.Clamp(step.V21 + step.V42, stageRange[stage]); + output.V22 = Av1Transform1dMath.Clamp(step.V22 + step.V41, stageRange[stage]); + output.V23 = Av1Transform1dMath.Clamp(step.V23 + step.V40, stageRange[stage]); + output.V24 = Av1Transform1dMath.Clamp(step.V24 + step.V39, stageRange[stage]); + output.V25 = Av1Transform1dMath.Clamp(step.V25 + step.V38, stageRange[stage]); + output.V26 = Av1Transform1dMath.Clamp(step.V26 + step.V37, stageRange[stage]); + output.V27 = Av1Transform1dMath.Clamp(step.V27 + step.V36, stageRange[stage]); + output.V28 = Av1Transform1dMath.Clamp(step.V28 + step.V35, stageRange[stage]); + output.V29 = Av1Transform1dMath.Clamp(step.V29 + step.V34, stageRange[stage]); + output.V30 = Av1Transform1dMath.Clamp(step.V30 + step.V33, stageRange[stage]); + output.V31 = Av1Transform1dMath.Clamp(step.V31 + step.V32, stageRange[stage]); + output.V32 = Av1Transform1dMath.Clamp(step.V31 - step.V32, stageRange[stage]); + output.V33 = Av1Transform1dMath.Clamp(step.V30 - step.V33, stageRange[stage]); + output.V34 = Av1Transform1dMath.Clamp(step.V29 - step.V34, stageRange[stage]); + output.V35 = Av1Transform1dMath.Clamp(step.V28 - step.V35, stageRange[stage]); + output.V36 = Av1Transform1dMath.Clamp(step.V27 - step.V36, stageRange[stage]); + output.V37 = Av1Transform1dMath.Clamp(step.V26 - step.V37, stageRange[stage]); + output.V38 = Av1Transform1dMath.Clamp(step.V25 - step.V38, stageRange[stage]); + output.V39 = Av1Transform1dMath.Clamp(step.V24 - step.V39, stageRange[stage]); + output.V40 = Av1Transform1dMath.Clamp(step.V23 - step.V40, stageRange[stage]); + output.V41 = Av1Transform1dMath.Clamp(step.V22 - step.V41, stageRange[stage]); + output.V42 = Av1Transform1dMath.Clamp(step.V21 - step.V42, stageRange[stage]); + output.V43 = Av1Transform1dMath.Clamp(step.V20 - step.V43, stageRange[stage]); + output.V44 = Av1Transform1dMath.Clamp(step.V19 - step.V44, stageRange[stage]); + output.V45 = Av1Transform1dMath.Clamp(step.V18 - step.V45, stageRange[stage]); + output.V46 = Av1Transform1dMath.Clamp(step.V17 - step.V46, stageRange[stage]); + output.V47 = Av1Transform1dMath.Clamp(step.V16 - step.V47, stageRange[stage]); + output.V48 = Av1Transform1dMath.Clamp(step.V15 - step.V48, stageRange[stage]); + output.V49 = Av1Transform1dMath.Clamp(step.V14 - step.V49, stageRange[stage]); + output.V50 = Av1Transform1dMath.Clamp(step.V13 - step.V50, stageRange[stage]); + output.V51 = Av1Transform1dMath.Clamp(step.V12 - step.V51, stageRange[stage]); + output.V52 = Av1Transform1dMath.Clamp(step.V11 - step.V52, stageRange[stage]); + output.V53 = Av1Transform1dMath.Clamp(step.V10 - step.V53, stageRange[stage]); + output.V54 = Av1Transform1dMath.Clamp(step.V9 - step.V54, stageRange[stage]); + output.V55 = Av1Transform1dMath.Clamp(step.V8 - step.V55, stageRange[stage]); + output.V56 = Av1Transform1dMath.Clamp(step.V7 - step.V56, stageRange[stage]); + output.V57 = Av1Transform1dMath.Clamp(step.V6 - step.V57, stageRange[stage]); + output.V58 = Av1Transform1dMath.Clamp(step.V5 - step.V58, stageRange[stage]); + output.V59 = Av1Transform1dMath.Clamp(step.V4 - step.V59, stageRange[stage]); + output.V60 = Av1Transform1dMath.Clamp(step.V3 - step.V60, stageRange[stage]); + output.V61 = Av1Transform1dMath.Clamp(step.V2 - step.V61, stageRange[stage]); + output.V62 = Av1Transform1dMath.Clamp(step.V1 - step.V62, stageRange[stage]); + output.V63 = Av1Transform1dMath.Clamp(step.V0 - step.V63, stageRange[stage]); + } + + /// + /// Applies the transform to four independent axes in parallel. + /// + /// The source values for the parallel transform axes. + /// The destination values for the parallel transform axes. + /// The fixed stage storage for the parallel transform axes. + /// The fixed-point precision of the cosine constants. + /// The signed-bit range assigned to each transform stage. + public static void Transform( + ref Av1TransformVector> input, + ref Av1TransformVector> output, + ref Av1TransformVector> step, + int cosBit, + InlineArray12 stageRange) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + int stage = 0; + + // Stage 1 permutes frequency-ordered coefficients into the recursive DCT factorization order. + stage++; + output.V0 = input.V0; + output.V1 = input.V32; + output.V2 = input.V16; + output.V3 = input.V48; + output.V4 = input.V8; + output.V5 = input.V40; + output.V6 = input.V24; + output.V7 = input.V56; + output.V8 = input.V4; + output.V9 = input.V36; + output.V10 = input.V20; + output.V11 = input.V52; + output.V12 = input.V12; + output.V13 = input.V44; + output.V14 = input.V28; + output.V15 = input.V60; + output.V16 = input.V2; + output.V17 = input.V34; + output.V18 = input.V18; + output.V19 = input.V50; + output.V20 = input.V10; + output.V21 = input.V42; + output.V22 = input.V26; + output.V23 = input.V58; + output.V24 = input.V6; + output.V25 = input.V38; + output.V26 = input.V22; + output.V27 = input.V54; + output.V28 = input.V14; + output.V29 = input.V46; + output.V30 = input.V30; + output.V31 = input.V62; + output.V32 = input.V1; + output.V33 = input.V33; + output.V34 = input.V17; + output.V35 = input.V49; + output.V36 = input.V9; + output.V37 = input.V41; + output.V38 = input.V25; + output.V39 = input.V57; + output.V40 = input.V5; + output.V41 = input.V37; + output.V42 = input.V21; + output.V43 = input.V53; + output.V44 = input.V13; + output.V45 = input.V45; + output.V46 = input.V29; + output.V47 = input.V61; + output.V48 = input.V3; + output.V49 = input.V35; + output.V50 = input.V19; + output.V51 = input.V51; + output.V52 = input.V11; + output.V53 = input.V43; + output.V54 = input.V27; + output.V55 = input.V59; + output.V56 = input.V7; + output.V57 = input.V39; + output.V58 = input.V23; + output.V59 = input.V55; + output.V60 = input.V15; + output.V61 = input.V47; + output.V62 = input.V31; + output.V63 = input.V63; + + // Stage 2 rotates the highest odd-frequency coefficient pairs by their pi/128 angles. + stage++; + step.V0 = output.V0; + step.V1 = output.V1; + step.V2 = output.V2; + step.V3 = output.V3; + step.V4 = output.V4; + step.V5 = output.V5; + step.V6 = output.V6; + step.V7 = output.V7; + step.V8 = output.V8; + step.V9 = output.V9; + step.V10 = output.V10; + step.V11 = output.V11; + step.V12 = output.V12; + step.V13 = output.V13; + step.V14 = output.V14; + step.V15 = output.V15; + step.V16 = output.V16; + step.V17 = output.V17; + step.V18 = output.V18; + step.V19 = output.V19; + step.V20 = output.V20; + step.V21 = output.V21; + step.V22 = output.V22; + step.V23 = output.V23; + step.V24 = output.V24; + step.V25 = output.V25; + step.V26 = output.V26; + step.V27 = output.V27; + step.V28 = output.V28; + step.V29 = output.V29; + step.V30 = output.V30; + step.V31 = output.V31; + step.V32 = Av1Transform1dMath.HalfButterfly(cospi[63], output.V32, -cospi[1], output.V63, cosBit); + step.V33 = Av1Transform1dMath.HalfButterfly(cospi[31], output.V33, -cospi[33], output.V62, cosBit); + step.V34 = Av1Transform1dMath.HalfButterfly(cospi[47], output.V34, -cospi[17], output.V61, cosBit); + step.V35 = Av1Transform1dMath.HalfButterfly(cospi[15], output.V35, -cospi[49], output.V60, cosBit); + step.V36 = Av1Transform1dMath.HalfButterfly(cospi[55], output.V36, -cospi[9], output.V59, cosBit); + step.V37 = Av1Transform1dMath.HalfButterfly(cospi[23], output.V37, -cospi[41], output.V58, cosBit); + step.V38 = Av1Transform1dMath.HalfButterfly(cospi[39], output.V38, -cospi[25], output.V57, cosBit); + step.V39 = Av1Transform1dMath.HalfButterfly(cospi[7], output.V39, -cospi[57], output.V56, cosBit); + step.V40 = Av1Transform1dMath.HalfButterfly(cospi[59], output.V40, -cospi[5], output.V55, cosBit); + step.V41 = Av1Transform1dMath.HalfButterfly(cospi[27], output.V41, -cospi[37], output.V54, cosBit); + step.V42 = Av1Transform1dMath.HalfButterfly(cospi[43], output.V42, -cospi[21], output.V53, cosBit); + step.V43 = Av1Transform1dMath.HalfButterfly(cospi[11], output.V43, -cospi[53], output.V52, cosBit); + step.V44 = Av1Transform1dMath.HalfButterfly(cospi[51], output.V44, -cospi[13], output.V51, cosBit); + step.V45 = Av1Transform1dMath.HalfButterfly(cospi[19], output.V45, -cospi[45], output.V50, cosBit); + step.V46 = Av1Transform1dMath.HalfButterfly(cospi[35], output.V46, -cospi[29], output.V49, cosBit); + step.V47 = Av1Transform1dMath.HalfButterfly(cospi[3], output.V47, -cospi[61], output.V48, cosBit); + step.V48 = Av1Transform1dMath.HalfButterfly(cospi[61], output.V47, cospi[3], output.V48, cosBit); + step.V49 = Av1Transform1dMath.HalfButterfly(cospi[29], output.V46, cospi[35], output.V49, cosBit); + step.V50 = Av1Transform1dMath.HalfButterfly(cospi[45], output.V45, cospi[19], output.V50, cosBit); + step.V51 = Av1Transform1dMath.HalfButterfly(cospi[13], output.V44, cospi[51], output.V51, cosBit); + step.V52 = Av1Transform1dMath.HalfButterfly(cospi[53], output.V43, cospi[11], output.V52, cosBit); + step.V53 = Av1Transform1dMath.HalfButterfly(cospi[21], output.V42, cospi[43], output.V53, cosBit); + step.V54 = Av1Transform1dMath.HalfButterfly(cospi[37], output.V41, cospi[27], output.V54, cosBit); + step.V55 = Av1Transform1dMath.HalfButterfly(cospi[5], output.V40, cospi[59], output.V55, cosBit); + step.V56 = Av1Transform1dMath.HalfButterfly(cospi[57], output.V39, cospi[7], output.V56, cosBit); + step.V57 = Av1Transform1dMath.HalfButterfly(cospi[25], output.V38, cospi[39], output.V57, cosBit); + step.V58 = Av1Transform1dMath.HalfButterfly(cospi[41], output.V37, cospi[23], output.V58, cosBit); + step.V59 = Av1Transform1dMath.HalfButterfly(cospi[9], output.V36, cospi[55], output.V59, cosBit); + step.V60 = Av1Transform1dMath.HalfButterfly(cospi[49], output.V35, cospi[15], output.V60, cosBit); + step.V61 = Av1Transform1dMath.HalfButterfly(cospi[17], output.V34, cospi[47], output.V61, cosBit); + step.V62 = Av1Transform1dMath.HalfButterfly(cospi[33], output.V33, cospi[31], output.V62, cosBit); + step.V63 = Av1Transform1dMath.HalfButterfly(cospi[1], output.V32, cospi[63], output.V63, cosBit); + + // Stage 3 reconstructs the first nested groups and combines their adjacent odd terms. + stage++; + output.V0 = step.V0; + output.V1 = step.V1; + output.V2 = step.V2; + output.V3 = step.V3; + output.V4 = step.V4; + output.V5 = step.V5; + output.V6 = step.V6; + output.V7 = step.V7; + output.V8 = step.V8; + output.V9 = step.V9; + output.V10 = step.V10; + output.V11 = step.V11; + output.V12 = step.V12; + output.V13 = step.V13; + output.V14 = step.V14; + output.V15 = step.V15; + output.V16 = Av1Transform1dMath.HalfButterfly(cospi[62], step.V16, -cospi[2], step.V31, cosBit); + output.V17 = Av1Transform1dMath.HalfButterfly(cospi[30], step.V17, -cospi[34], step.V30, cosBit); + output.V18 = Av1Transform1dMath.HalfButterfly(cospi[46], step.V18, -cospi[18], step.V29, cosBit); + output.V19 = Av1Transform1dMath.HalfButterfly(cospi[14], step.V19, -cospi[50], step.V28, cosBit); + output.V20 = Av1Transform1dMath.HalfButterfly(cospi[54], step.V20, -cospi[10], step.V27, cosBit); + output.V21 = Av1Transform1dMath.HalfButterfly(cospi[22], step.V21, -cospi[42], step.V26, cosBit); + output.V22 = Av1Transform1dMath.HalfButterfly(cospi[38], step.V22, -cospi[26], step.V25, cosBit); + output.V23 = Av1Transform1dMath.HalfButterfly(cospi[6], step.V23, -cospi[58], step.V24, cosBit); + output.V24 = Av1Transform1dMath.HalfButterfly(cospi[58], step.V23, cospi[6], step.V24, cosBit); + output.V25 = Av1Transform1dMath.HalfButterfly(cospi[26], step.V22, cospi[38], step.V25, cosBit); + output.V26 = Av1Transform1dMath.HalfButterfly(cospi[42], step.V21, cospi[22], step.V26, cosBit); + output.V27 = Av1Transform1dMath.HalfButterfly(cospi[10], step.V20, cospi[54], step.V27, cosBit); + output.V28 = Av1Transform1dMath.HalfButterfly(cospi[50], step.V19, cospi[14], step.V28, cosBit); + output.V29 = Av1Transform1dMath.HalfButterfly(cospi[18], step.V18, cospi[46], step.V29, cosBit); + output.V30 = Av1Transform1dMath.HalfButterfly(cospi[34], step.V17, cospi[30], step.V30, cosBit); + output.V31 = Av1Transform1dMath.HalfButterfly(cospi[2], step.V16, cospi[62], step.V31, cosBit); + output.V32 = Av1Transform1dMath.Clamp(step.V32 + step.V33, stageRange[stage]); + output.V33 = Av1Transform1dMath.Clamp(step.V32 - step.V33, stageRange[stage]); + output.V34 = Av1Transform1dMath.Clamp(-step.V34 + step.V35, stageRange[stage]); + output.V35 = Av1Transform1dMath.Clamp(step.V34 + step.V35, stageRange[stage]); + output.V36 = Av1Transform1dMath.Clamp(step.V36 + step.V37, stageRange[stage]); + output.V37 = Av1Transform1dMath.Clamp(step.V36 - step.V37, stageRange[stage]); + output.V38 = Av1Transform1dMath.Clamp(-step.V38 + step.V39, stageRange[stage]); + output.V39 = Av1Transform1dMath.Clamp(step.V38 + step.V39, stageRange[stage]); + output.V40 = Av1Transform1dMath.Clamp(step.V40 + step.V41, stageRange[stage]); + output.V41 = Av1Transform1dMath.Clamp(step.V40 - step.V41, stageRange[stage]); + output.V42 = Av1Transform1dMath.Clamp(-step.V42 + step.V43, stageRange[stage]); + output.V43 = Av1Transform1dMath.Clamp(step.V42 + step.V43, stageRange[stage]); + output.V44 = Av1Transform1dMath.Clamp(step.V44 + step.V45, stageRange[stage]); + output.V45 = Av1Transform1dMath.Clamp(step.V44 - step.V45, stageRange[stage]); + output.V46 = Av1Transform1dMath.Clamp(-step.V46 + step.V47, stageRange[stage]); + output.V47 = Av1Transform1dMath.Clamp(step.V46 + step.V47, stageRange[stage]); + output.V48 = Av1Transform1dMath.Clamp(step.V48 + step.V49, stageRange[stage]); + output.V49 = Av1Transform1dMath.Clamp(step.V48 - step.V49, stageRange[stage]); + output.V50 = Av1Transform1dMath.Clamp(-step.V50 + step.V51, stageRange[stage]); + output.V51 = Av1Transform1dMath.Clamp(step.V50 + step.V51, stageRange[stage]); + output.V52 = Av1Transform1dMath.Clamp(step.V52 + step.V53, stageRange[stage]); + output.V53 = Av1Transform1dMath.Clamp(step.V52 - step.V53, stageRange[stage]); + output.V54 = Av1Transform1dMath.Clamp(-step.V54 + step.V55, stageRange[stage]); + output.V55 = Av1Transform1dMath.Clamp(step.V54 + step.V55, stageRange[stage]); + output.V56 = Av1Transform1dMath.Clamp(step.V56 + step.V57, stageRange[stage]); + output.V57 = Av1Transform1dMath.Clamp(step.V56 - step.V57, stageRange[stage]); + output.V58 = Av1Transform1dMath.Clamp(-step.V58 + step.V59, stageRange[stage]); + output.V59 = Av1Transform1dMath.Clamp(step.V58 + step.V59, stageRange[stage]); + output.V60 = Av1Transform1dMath.Clamp(step.V60 + step.V61, stageRange[stage]); + output.V61 = Av1Transform1dMath.Clamp(step.V60 - step.V61, stageRange[stage]); + output.V62 = Av1Transform1dMath.Clamp(-step.V62 + step.V63, stageRange[stage]); + output.V63 = Av1Transform1dMath.Clamp(step.V62 + step.V63, stageRange[stage]); + + // Stage 4 rotates the next odd-frequency level while preserving completed low-frequency lanes. + stage++; + step.V0 = output.V0; + step.V1 = output.V1; + step.V2 = output.V2; + step.V3 = output.V3; + step.V4 = output.V4; + step.V5 = output.V5; + step.V6 = output.V6; + step.V7 = output.V7; + step.V8 = Av1Transform1dMath.HalfButterfly(cospi[60], output.V8, -cospi[4], output.V15, cosBit); + step.V9 = Av1Transform1dMath.HalfButterfly(cospi[28], output.V9, -cospi[36], output.V14, cosBit); + step.V10 = Av1Transform1dMath.HalfButterfly(cospi[44], output.V10, -cospi[20], output.V13, cosBit); + step.V11 = Av1Transform1dMath.HalfButterfly(cospi[12], output.V11, -cospi[52], output.V12, cosBit); + step.V12 = Av1Transform1dMath.HalfButterfly(cospi[52], output.V11, cospi[12], output.V12, cosBit); + step.V13 = Av1Transform1dMath.HalfButterfly(cospi[20], output.V10, cospi[44], output.V13, cosBit); + step.V14 = Av1Transform1dMath.HalfButterfly(cospi[36], output.V9, cospi[28], output.V14, cosBit); + step.V15 = Av1Transform1dMath.HalfButterfly(cospi[4], output.V8, cospi[60], output.V15, cosBit); + step.V16 = Av1Transform1dMath.Clamp(output.V16 + output.V17, stageRange[stage]); + step.V17 = Av1Transform1dMath.Clamp(output.V16 - output.V17, stageRange[stage]); + step.V18 = Av1Transform1dMath.Clamp(-output.V18 + output.V19, stageRange[stage]); + step.V19 = Av1Transform1dMath.Clamp(output.V18 + output.V19, stageRange[stage]); + step.V20 = Av1Transform1dMath.Clamp(output.V20 + output.V21, stageRange[stage]); + step.V21 = Av1Transform1dMath.Clamp(output.V20 - output.V21, stageRange[stage]); + step.V22 = Av1Transform1dMath.Clamp(-output.V22 + output.V23, stageRange[stage]); + step.V23 = Av1Transform1dMath.Clamp(output.V22 + output.V23, stageRange[stage]); + step.V24 = Av1Transform1dMath.Clamp(output.V24 + output.V25, stageRange[stage]); + step.V25 = Av1Transform1dMath.Clamp(output.V24 - output.V25, stageRange[stage]); + step.V26 = Av1Transform1dMath.Clamp(-output.V26 + output.V27, stageRange[stage]); + step.V27 = Av1Transform1dMath.Clamp(output.V26 + output.V27, stageRange[stage]); + step.V28 = Av1Transform1dMath.Clamp(output.V28 + output.V29, stageRange[stage]); + step.V29 = Av1Transform1dMath.Clamp(output.V28 - output.V29, stageRange[stage]); + step.V30 = Av1Transform1dMath.Clamp(-output.V30 + output.V31, stageRange[stage]); + step.V31 = Av1Transform1dMath.Clamp(output.V30 + output.V31, stageRange[stage]); + step.V32 = output.V32; + step.V33 = Av1Transform1dMath.HalfButterfly(-cospi[4], output.V33, cospi[60], output.V62, cosBit); + step.V34 = Av1Transform1dMath.HalfButterfly(-cospi[60], output.V34, -cospi[4], output.V61, cosBit); + step.V35 = output.V35; + step.V36 = output.V36; + step.V37 = Av1Transform1dMath.HalfButterfly(-cospi[36], output.V37, cospi[28], output.V58, cosBit); + step.V38 = Av1Transform1dMath.HalfButterfly(-cospi[28], output.V38, -cospi[36], output.V57, cosBit); + step.V39 = output.V39; + step.V40 = output.V40; + step.V41 = Av1Transform1dMath.HalfButterfly(-cospi[20], output.V41, cospi[44], output.V54, cosBit); + step.V42 = Av1Transform1dMath.HalfButterfly(-cospi[44], output.V42, -cospi[20], output.V53, cosBit); + step.V43 = output.V43; + step.V44 = output.V44; + step.V45 = Av1Transform1dMath.HalfButterfly(-cospi[52], output.V45, cospi[12], output.V50, cosBit); + step.V46 = Av1Transform1dMath.HalfButterfly(-cospi[12], output.V46, -cospi[52], output.V49, cosBit); + step.V47 = output.V47; + step.V48 = output.V48; + step.V49 = Av1Transform1dMath.HalfButterfly(-cospi[52], output.V46, cospi[12], output.V49, cosBit); + step.V50 = Av1Transform1dMath.HalfButterfly(cospi[12], output.V45, cospi[52], output.V50, cosBit); + step.V51 = output.V51; + step.V52 = output.V52; + step.V53 = Av1Transform1dMath.HalfButterfly(-cospi[20], output.V42, cospi[44], output.V53, cosBit); + step.V54 = Av1Transform1dMath.HalfButterfly(cospi[44], output.V41, cospi[20], output.V54, cosBit); + step.V55 = output.V55; + step.V56 = output.V56; + step.V57 = Av1Transform1dMath.HalfButterfly(-cospi[36], output.V38, cospi[28], output.V57, cosBit); + step.V58 = Av1Transform1dMath.HalfButterfly(cospi[28], output.V37, cospi[36], output.V58, cosBit); + step.V59 = output.V59; + step.V60 = output.V60; + step.V61 = Av1Transform1dMath.HalfButterfly(-cospi[4], output.V34, cospi[60], output.V61, cosBit); + step.V62 = Av1Transform1dMath.HalfButterfly(cospi[60], output.V33, cospi[4], output.V62, cosBit); + step.V63 = output.V63; + + // Stage 5 widens the nested groups through the next butterfly level. + stage++; + output.V0 = step.V0; + output.V1 = step.V1; + output.V2 = step.V2; + output.V3 = step.V3; + output.V4 = Av1Transform1dMath.HalfButterfly(cospi[56], step.V4, -cospi[8], step.V7, cosBit); + output.V5 = Av1Transform1dMath.HalfButterfly(cospi[24], step.V5, -cospi[40], step.V6, cosBit); + output.V6 = Av1Transform1dMath.HalfButterfly(cospi[40], step.V5, cospi[24], step.V6, cosBit); + output.V7 = Av1Transform1dMath.HalfButterfly(cospi[8], step.V4, cospi[56], step.V7, cosBit); + output.V8 = Av1Transform1dMath.Clamp(step.V8 + step.V9, stageRange[stage]); + output.V9 = Av1Transform1dMath.Clamp(step.V8 - step.V9, stageRange[stage]); + output.V10 = Av1Transform1dMath.Clamp(-step.V10 + step.V11, stageRange[stage]); + output.V11 = Av1Transform1dMath.Clamp(step.V10 + step.V11, stageRange[stage]); + output.V12 = Av1Transform1dMath.Clamp(step.V12 + step.V13, stageRange[stage]); + output.V13 = Av1Transform1dMath.Clamp(step.V12 - step.V13, stageRange[stage]); + output.V14 = Av1Transform1dMath.Clamp(-step.V14 + step.V15, stageRange[stage]); + output.V15 = Av1Transform1dMath.Clamp(step.V14 + step.V15, stageRange[stage]); + output.V16 = step.V16; + output.V17 = Av1Transform1dMath.HalfButterfly(-cospi[8], step.V17, cospi[56], step.V30, cosBit); + output.V18 = Av1Transform1dMath.HalfButterfly(-cospi[56], step.V18, -cospi[8], step.V29, cosBit); + output.V19 = step.V19; + output.V20 = step.V20; + output.V21 = Av1Transform1dMath.HalfButterfly(-cospi[40], step.V21, cospi[24], step.V26, cosBit); + output.V22 = Av1Transform1dMath.HalfButterfly(-cospi[24], step.V22, -cospi[40], step.V25, cosBit); + output.V23 = step.V23; + output.V24 = step.V24; + output.V25 = Av1Transform1dMath.HalfButterfly(-cospi[40], step.V22, cospi[24], step.V25, cosBit); + output.V26 = Av1Transform1dMath.HalfButterfly(cospi[24], step.V21, cospi[40], step.V26, cosBit); + output.V27 = step.V27; + output.V28 = step.V28; + output.V29 = Av1Transform1dMath.HalfButterfly(-cospi[8], step.V18, cospi[56], step.V29, cosBit); + output.V30 = Av1Transform1dMath.HalfButterfly(cospi[56], step.V17, cospi[8], step.V30, cosBit); + output.V31 = step.V31; + output.V32 = Av1Transform1dMath.Clamp(step.V32 + step.V35, stageRange[stage]); + output.V33 = Av1Transform1dMath.Clamp(step.V33 + step.V34, stageRange[stage]); + output.V34 = Av1Transform1dMath.Clamp(step.V33 - step.V34, stageRange[stage]); + output.V35 = Av1Transform1dMath.Clamp(step.V32 - step.V35, stageRange[stage]); + output.V36 = Av1Transform1dMath.Clamp(-step.V36 + step.V39, stageRange[stage]); + output.V37 = Av1Transform1dMath.Clamp(-step.V37 + step.V38, stageRange[stage]); + output.V38 = Av1Transform1dMath.Clamp(step.V37 + step.V38, stageRange[stage]); + output.V39 = Av1Transform1dMath.Clamp(step.V36 + step.V39, stageRange[stage]); + output.V40 = Av1Transform1dMath.Clamp(step.V40 + step.V43, stageRange[stage]); + output.V41 = Av1Transform1dMath.Clamp(step.V41 + step.V42, stageRange[stage]); + output.V42 = Av1Transform1dMath.Clamp(step.V41 - step.V42, stageRange[stage]); + output.V43 = Av1Transform1dMath.Clamp(step.V40 - step.V43, stageRange[stage]); + output.V44 = Av1Transform1dMath.Clamp(-step.V44 + step.V47, stageRange[stage]); + output.V45 = Av1Transform1dMath.Clamp(-step.V45 + step.V46, stageRange[stage]); + output.V46 = Av1Transform1dMath.Clamp(step.V45 + step.V46, stageRange[stage]); + output.V47 = Av1Transform1dMath.Clamp(step.V44 + step.V47, stageRange[stage]); + output.V48 = Av1Transform1dMath.Clamp(step.V48 + step.V51, stageRange[stage]); + output.V49 = Av1Transform1dMath.Clamp(step.V49 + step.V50, stageRange[stage]); + output.V50 = Av1Transform1dMath.Clamp(step.V49 - step.V50, stageRange[stage]); + output.V51 = Av1Transform1dMath.Clamp(step.V48 - step.V51, stageRange[stage]); + output.V52 = Av1Transform1dMath.Clamp(-step.V52 + step.V55, stageRange[stage]); + output.V53 = Av1Transform1dMath.Clamp(-step.V53 + step.V54, stageRange[stage]); + output.V54 = Av1Transform1dMath.Clamp(step.V53 + step.V54, stageRange[stage]); + output.V55 = Av1Transform1dMath.Clamp(step.V52 + step.V55, stageRange[stage]); + output.V56 = Av1Transform1dMath.Clamp(step.V56 + step.V59, stageRange[stage]); + output.V57 = Av1Transform1dMath.Clamp(step.V57 + step.V58, stageRange[stage]); + output.V58 = Av1Transform1dMath.Clamp(step.V57 - step.V58, stageRange[stage]); + output.V59 = Av1Transform1dMath.Clamp(step.V56 - step.V59, stageRange[stage]); + output.V60 = Av1Transform1dMath.Clamp(-step.V60 + step.V63, stageRange[stage]); + output.V61 = Av1Transform1dMath.Clamp(-step.V61 + step.V62, stageRange[stage]); + output.V62 = Av1Transform1dMath.Clamp(step.V61 + step.V62, stageRange[stage]); + output.V63 = Av1Transform1dMath.Clamp(step.V60 + step.V63, stageRange[stage]); + + // Stage 6 rotates the next odd-frequency level while preserving completed low-frequency lanes. + stage++; + step.V0 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V0, cospi[32], output.V1, cosBit); + step.V1 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V0, -cospi[32], output.V1, cosBit); + step.V2 = Av1Transform1dMath.HalfButterfly(cospi[48], output.V2, -cospi[16], output.V3, cosBit); + step.V3 = Av1Transform1dMath.HalfButterfly(cospi[16], output.V2, cospi[48], output.V3, cosBit); + step.V4 = Av1Transform1dMath.Clamp(output.V4 + output.V5, stageRange[stage]); + step.V5 = Av1Transform1dMath.Clamp(output.V4 - output.V5, stageRange[stage]); + step.V6 = Av1Transform1dMath.Clamp(-output.V6 + output.V7, stageRange[stage]); + step.V7 = Av1Transform1dMath.Clamp(output.V6 + output.V7, stageRange[stage]); + step.V8 = output.V8; + step.V9 = Av1Transform1dMath.HalfButterfly(-cospi[16], output.V9, cospi[48], output.V14, cosBit); + step.V10 = Av1Transform1dMath.HalfButterfly(-cospi[48], output.V10, -cospi[16], output.V13, cosBit); + step.V11 = output.V11; + step.V12 = output.V12; + step.V13 = Av1Transform1dMath.HalfButterfly(-cospi[16], output.V10, cospi[48], output.V13, cosBit); + step.V14 = Av1Transform1dMath.HalfButterfly(cospi[48], output.V9, cospi[16], output.V14, cosBit); + step.V15 = output.V15; + step.V16 = Av1Transform1dMath.Clamp(output.V16 + output.V19, stageRange[stage]); + step.V17 = Av1Transform1dMath.Clamp(output.V17 + output.V18, stageRange[stage]); + step.V18 = Av1Transform1dMath.Clamp(output.V17 - output.V18, stageRange[stage]); + step.V19 = Av1Transform1dMath.Clamp(output.V16 - output.V19, stageRange[stage]); + step.V20 = Av1Transform1dMath.Clamp(-output.V20 + output.V23, stageRange[stage]); + step.V21 = Av1Transform1dMath.Clamp(-output.V21 + output.V22, stageRange[stage]); + step.V22 = Av1Transform1dMath.Clamp(output.V21 + output.V22, stageRange[stage]); + step.V23 = Av1Transform1dMath.Clamp(output.V20 + output.V23, stageRange[stage]); + step.V24 = Av1Transform1dMath.Clamp(output.V24 + output.V27, stageRange[stage]); + step.V25 = Av1Transform1dMath.Clamp(output.V25 + output.V26, stageRange[stage]); + step.V26 = Av1Transform1dMath.Clamp(output.V25 - output.V26, stageRange[stage]); + step.V27 = Av1Transform1dMath.Clamp(output.V24 - output.V27, stageRange[stage]); + step.V28 = Av1Transform1dMath.Clamp(-output.V28 + output.V31, stageRange[stage]); + step.V29 = Av1Transform1dMath.Clamp(-output.V29 + output.V30, stageRange[stage]); + step.V30 = Av1Transform1dMath.Clamp(output.V29 + output.V30, stageRange[stage]); + step.V31 = Av1Transform1dMath.Clamp(output.V28 + output.V31, stageRange[stage]); + step.V32 = output.V32; + step.V33 = output.V33; + step.V34 = Av1Transform1dMath.HalfButterfly(-cospi[8], output.V34, cospi[56], output.V61, cosBit); + step.V35 = Av1Transform1dMath.HalfButterfly(-cospi[8], output.V35, cospi[56], output.V60, cosBit); + step.V36 = Av1Transform1dMath.HalfButterfly(-cospi[56], output.V36, -cospi[8], output.V59, cosBit); + step.V37 = Av1Transform1dMath.HalfButterfly(-cospi[56], output.V37, -cospi[8], output.V58, cosBit); + step.V38 = output.V38; + step.V39 = output.V39; + step.V40 = output.V40; + step.V41 = output.V41; + step.V42 = Av1Transform1dMath.HalfButterfly(-cospi[40], output.V42, cospi[24], output.V53, cosBit); + step.V43 = Av1Transform1dMath.HalfButterfly(-cospi[40], output.V43, cospi[24], output.V52, cosBit); + step.V44 = Av1Transform1dMath.HalfButterfly(-cospi[24], output.V44, -cospi[40], output.V51, cosBit); + step.V45 = Av1Transform1dMath.HalfButterfly(-cospi[24], output.V45, -cospi[40], output.V50, cosBit); + step.V46 = output.V46; + step.V47 = output.V47; + step.V48 = output.V48; + step.V49 = output.V49; + step.V50 = Av1Transform1dMath.HalfButterfly(-cospi[40], output.V45, cospi[24], output.V50, cosBit); + step.V51 = Av1Transform1dMath.HalfButterfly(-cospi[40], output.V44, cospi[24], output.V51, cosBit); + step.V52 = Av1Transform1dMath.HalfButterfly(cospi[24], output.V43, cospi[40], output.V52, cosBit); + step.V53 = Av1Transform1dMath.HalfButterfly(cospi[24], output.V42, cospi[40], output.V53, cosBit); + step.V54 = output.V54; + step.V55 = output.V55; + step.V56 = output.V56; + step.V57 = output.V57; + step.V58 = Av1Transform1dMath.HalfButterfly(-cospi[8], output.V37, cospi[56], output.V58, cosBit); + step.V59 = Av1Transform1dMath.HalfButterfly(-cospi[8], output.V36, cospi[56], output.V59, cosBit); + step.V60 = Av1Transform1dMath.HalfButterfly(cospi[56], output.V35, cospi[8], output.V60, cosBit); + step.V61 = Av1Transform1dMath.HalfButterfly(cospi[56], output.V34, cospi[8], output.V61, cosBit); + step.V62 = output.V62; + step.V63 = output.V63; + + // Stage 7 reconstructs the embedded sixteen-point groups and combines adjacent odd terms. + stage++; + output.V0 = Av1Transform1dMath.Clamp(step.V0 + step.V3, stageRange[stage]); + output.V1 = Av1Transform1dMath.Clamp(step.V1 + step.V2, stageRange[stage]); + output.V2 = Av1Transform1dMath.Clamp(step.V1 - step.V2, stageRange[stage]); + output.V3 = Av1Transform1dMath.Clamp(step.V0 - step.V3, stageRange[stage]); + output.V4 = step.V4; + output.V5 = Av1Transform1dMath.HalfButterfly(-cospi[32], step.V5, cospi[32], step.V6, cosBit); + output.V6 = Av1Transform1dMath.HalfButterfly(cospi[32], step.V5, cospi[32], step.V6, cosBit); + output.V7 = step.V7; + output.V8 = Av1Transform1dMath.Clamp(step.V8 + step.V11, stageRange[stage]); + output.V9 = Av1Transform1dMath.Clamp(step.V9 + step.V10, stageRange[stage]); + output.V10 = Av1Transform1dMath.Clamp(step.V9 - step.V10, stageRange[stage]); + output.V11 = Av1Transform1dMath.Clamp(step.V8 - step.V11, stageRange[stage]); + output.V12 = Av1Transform1dMath.Clamp(-step.V12 + step.V15, stageRange[stage]); + output.V13 = Av1Transform1dMath.Clamp(-step.V13 + step.V14, stageRange[stage]); + output.V14 = Av1Transform1dMath.Clamp(step.V13 + step.V14, stageRange[stage]); + output.V15 = Av1Transform1dMath.Clamp(step.V12 + step.V15, stageRange[stage]); + output.V16 = step.V16; + output.V17 = step.V17; + output.V18 = Av1Transform1dMath.HalfButterfly(-cospi[16], step.V18, cospi[48], step.V29, cosBit); + output.V19 = Av1Transform1dMath.HalfButterfly(-cospi[16], step.V19, cospi[48], step.V28, cosBit); + output.V20 = Av1Transform1dMath.HalfButterfly(-cospi[48], step.V20, -cospi[16], step.V27, cosBit); + output.V21 = Av1Transform1dMath.HalfButterfly(-cospi[48], step.V21, -cospi[16], step.V26, cosBit); + output.V22 = step.V22; + output.V23 = step.V23; + output.V24 = step.V24; + output.V25 = step.V25; + output.V26 = Av1Transform1dMath.HalfButterfly(-cospi[16], step.V21, cospi[48], step.V26, cosBit); + output.V27 = Av1Transform1dMath.HalfButterfly(-cospi[16], step.V20, cospi[48], step.V27, cosBit); + output.V28 = Av1Transform1dMath.HalfButterfly(cospi[48], step.V19, cospi[16], step.V28, cosBit); + output.V29 = Av1Transform1dMath.HalfButterfly(cospi[48], step.V18, cospi[16], step.V29, cosBit); + output.V30 = step.V30; + output.V31 = step.V31; + output.V32 = Av1Transform1dMath.Clamp(step.V32 + step.V39, stageRange[stage]); + output.V33 = Av1Transform1dMath.Clamp(step.V33 + step.V38, stageRange[stage]); + output.V34 = Av1Transform1dMath.Clamp(step.V34 + step.V37, stageRange[stage]); + output.V35 = Av1Transform1dMath.Clamp(step.V35 + step.V36, stageRange[stage]); + output.V36 = Av1Transform1dMath.Clamp(step.V35 - step.V36, stageRange[stage]); + output.V37 = Av1Transform1dMath.Clamp(step.V34 - step.V37, stageRange[stage]); + output.V38 = Av1Transform1dMath.Clamp(step.V33 - step.V38, stageRange[stage]); + output.V39 = Av1Transform1dMath.Clamp(step.V32 - step.V39, stageRange[stage]); + output.V40 = Av1Transform1dMath.Clamp(-step.V40 + step.V47, stageRange[stage]); + output.V41 = Av1Transform1dMath.Clamp(-step.V41 + step.V46, stageRange[stage]); + output.V42 = Av1Transform1dMath.Clamp(-step.V42 + step.V45, stageRange[stage]); + output.V43 = Av1Transform1dMath.Clamp(-step.V43 + step.V44, stageRange[stage]); + output.V44 = Av1Transform1dMath.Clamp(step.V43 + step.V44, stageRange[stage]); + output.V45 = Av1Transform1dMath.Clamp(step.V42 + step.V45, stageRange[stage]); + output.V46 = Av1Transform1dMath.Clamp(step.V41 + step.V46, stageRange[stage]); + output.V47 = Av1Transform1dMath.Clamp(step.V40 + step.V47, stageRange[stage]); + output.V48 = Av1Transform1dMath.Clamp(step.V48 + step.V55, stageRange[stage]); + output.V49 = Av1Transform1dMath.Clamp(step.V49 + step.V54, stageRange[stage]); + output.V50 = Av1Transform1dMath.Clamp(step.V50 + step.V53, stageRange[stage]); + output.V51 = Av1Transform1dMath.Clamp(step.V51 + step.V52, stageRange[stage]); + output.V52 = Av1Transform1dMath.Clamp(step.V51 - step.V52, stageRange[stage]); + output.V53 = Av1Transform1dMath.Clamp(step.V50 - step.V53, stageRange[stage]); + output.V54 = Av1Transform1dMath.Clamp(step.V49 - step.V54, stageRange[stage]); + output.V55 = Av1Transform1dMath.Clamp(step.V48 - step.V55, stageRange[stage]); + output.V56 = Av1Transform1dMath.Clamp(-step.V56 + step.V63, stageRange[stage]); + output.V57 = Av1Transform1dMath.Clamp(-step.V57 + step.V62, stageRange[stage]); + output.V58 = Av1Transform1dMath.Clamp(-step.V58 + step.V61, stageRange[stage]); + output.V59 = Av1Transform1dMath.Clamp(-step.V59 + step.V60, stageRange[stage]); + output.V60 = Av1Transform1dMath.Clamp(step.V59 + step.V60, stageRange[stage]); + output.V61 = Av1Transform1dMath.Clamp(step.V58 + step.V61, stageRange[stage]); + output.V62 = Av1Transform1dMath.Clamp(step.V57 + step.V62, stageRange[stage]); + output.V63 = Av1Transform1dMath.Clamp(step.V56 + step.V63, stageRange[stage]); + + // Stage 8 completes the embedded eight-point groups and rotates their odd-frequency pairs. + stage++; + step.V0 = Av1Transform1dMath.Clamp(output.V0 + output.V7, stageRange[stage]); + step.V1 = Av1Transform1dMath.Clamp(output.V1 + output.V6, stageRange[stage]); + step.V2 = Av1Transform1dMath.Clamp(output.V2 + output.V5, stageRange[stage]); + step.V3 = Av1Transform1dMath.Clamp(output.V3 + output.V4, stageRange[stage]); + step.V4 = Av1Transform1dMath.Clamp(output.V3 - output.V4, stageRange[stage]); + step.V5 = Av1Transform1dMath.Clamp(output.V2 - output.V5, stageRange[stage]); + step.V6 = Av1Transform1dMath.Clamp(output.V1 - output.V6, stageRange[stage]); + step.V7 = Av1Transform1dMath.Clamp(output.V0 - output.V7, stageRange[stage]); + step.V8 = output.V8; + step.V9 = output.V9; + step.V10 = Av1Transform1dMath.HalfButterfly(-cospi[32], output.V10, cospi[32], output.V13, cosBit); + step.V11 = Av1Transform1dMath.HalfButterfly(-cospi[32], output.V11, cospi[32], output.V12, cosBit); + step.V12 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V11, cospi[32], output.V12, cosBit); + step.V13 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V10, cospi[32], output.V13, cosBit); + step.V14 = output.V14; + step.V15 = output.V15; + step.V16 = Av1Transform1dMath.Clamp(output.V16 + output.V23, stageRange[stage]); + step.V17 = Av1Transform1dMath.Clamp(output.V17 + output.V22, stageRange[stage]); + step.V18 = Av1Transform1dMath.Clamp(output.V18 + output.V21, stageRange[stage]); + step.V19 = Av1Transform1dMath.Clamp(output.V19 + output.V20, stageRange[stage]); + step.V20 = Av1Transform1dMath.Clamp(output.V19 - output.V20, stageRange[stage]); + step.V21 = Av1Transform1dMath.Clamp(output.V18 - output.V21, stageRange[stage]); + step.V22 = Av1Transform1dMath.Clamp(output.V17 - output.V22, stageRange[stage]); + step.V23 = Av1Transform1dMath.Clamp(output.V16 - output.V23, stageRange[stage]); + step.V24 = Av1Transform1dMath.Clamp(-output.V24 + output.V31, stageRange[stage]); + step.V25 = Av1Transform1dMath.Clamp(-output.V25 + output.V30, stageRange[stage]); + step.V26 = Av1Transform1dMath.Clamp(-output.V26 + output.V29, stageRange[stage]); + step.V27 = Av1Transform1dMath.Clamp(-output.V27 + output.V28, stageRange[stage]); + step.V28 = Av1Transform1dMath.Clamp(output.V27 + output.V28, stageRange[stage]); + step.V29 = Av1Transform1dMath.Clamp(output.V26 + output.V29, stageRange[stage]); + step.V30 = Av1Transform1dMath.Clamp(output.V25 + output.V30, stageRange[stage]); + step.V31 = Av1Transform1dMath.Clamp(output.V24 + output.V31, stageRange[stage]); + step.V32 = output.V32; + step.V33 = output.V33; + step.V34 = output.V34; + step.V35 = output.V35; + step.V36 = Av1Transform1dMath.HalfButterfly(-cospi[16], output.V36, cospi[48], output.V59, cosBit); + step.V37 = Av1Transform1dMath.HalfButterfly(-cospi[16], output.V37, cospi[48], output.V58, cosBit); + step.V38 = Av1Transform1dMath.HalfButterfly(-cospi[16], output.V38, cospi[48], output.V57, cosBit); + step.V39 = Av1Transform1dMath.HalfButterfly(-cospi[16], output.V39, cospi[48], output.V56, cosBit); + step.V40 = Av1Transform1dMath.HalfButterfly(-cospi[48], output.V40, -cospi[16], output.V55, cosBit); + step.V41 = Av1Transform1dMath.HalfButterfly(-cospi[48], output.V41, -cospi[16], output.V54, cosBit); + step.V42 = Av1Transform1dMath.HalfButterfly(-cospi[48], output.V42, -cospi[16], output.V53, cosBit); + step.V43 = Av1Transform1dMath.HalfButterfly(-cospi[48], output.V43, -cospi[16], output.V52, cosBit); + step.V44 = output.V44; + step.V45 = output.V45; + step.V46 = output.V46; + step.V47 = output.V47; + step.V48 = output.V48; + step.V49 = output.V49; + step.V50 = output.V50; + step.V51 = output.V51; + step.V52 = Av1Transform1dMath.HalfButterfly(-cospi[16], output.V43, cospi[48], output.V52, cosBit); + step.V53 = Av1Transform1dMath.HalfButterfly(-cospi[16], output.V42, cospi[48], output.V53, cosBit); + step.V54 = Av1Transform1dMath.HalfButterfly(-cospi[16], output.V41, cospi[48], output.V54, cosBit); + step.V55 = Av1Transform1dMath.HalfButterfly(-cospi[16], output.V40, cospi[48], output.V55, cosBit); + step.V56 = Av1Transform1dMath.HalfButterfly(cospi[48], output.V39, cospi[16], output.V56, cosBit); + step.V57 = Av1Transform1dMath.HalfButterfly(cospi[48], output.V38, cospi[16], output.V57, cosBit); + step.V58 = Av1Transform1dMath.HalfButterfly(cospi[48], output.V37, cospi[16], output.V58, cosBit); + step.V59 = Av1Transform1dMath.HalfButterfly(cospi[48], output.V36, cospi[16], output.V59, cosBit); + step.V60 = output.V60; + step.V61 = output.V61; + step.V62 = output.V62; + step.V63 = output.V63; + + // Stage 9 widens the reconstructed groups through their next butterfly level. + stage++; + output.V0 = Av1Transform1dMath.Clamp(step.V0 + step.V15, stageRange[stage]); + output.V1 = Av1Transform1dMath.Clamp(step.V1 + step.V14, stageRange[stage]); + output.V2 = Av1Transform1dMath.Clamp(step.V2 + step.V13, stageRange[stage]); + output.V3 = Av1Transform1dMath.Clamp(step.V3 + step.V12, stageRange[stage]); + output.V4 = Av1Transform1dMath.Clamp(step.V4 + step.V11, stageRange[stage]); + output.V5 = Av1Transform1dMath.Clamp(step.V5 + step.V10, stageRange[stage]); + output.V6 = Av1Transform1dMath.Clamp(step.V6 + step.V9, stageRange[stage]); + output.V7 = Av1Transform1dMath.Clamp(step.V7 + step.V8, stageRange[stage]); + output.V8 = Av1Transform1dMath.Clamp(step.V7 - step.V8, stageRange[stage]); + output.V9 = Av1Transform1dMath.Clamp(step.V6 - step.V9, stageRange[stage]); + output.V10 = Av1Transform1dMath.Clamp(step.V5 - step.V10, stageRange[stage]); + output.V11 = Av1Transform1dMath.Clamp(step.V4 - step.V11, stageRange[stage]); + output.V12 = Av1Transform1dMath.Clamp(step.V3 - step.V12, stageRange[stage]); + output.V13 = Av1Transform1dMath.Clamp(step.V2 - step.V13, stageRange[stage]); + output.V14 = Av1Transform1dMath.Clamp(step.V1 - step.V14, stageRange[stage]); + output.V15 = Av1Transform1dMath.Clamp(step.V0 - step.V15, stageRange[stage]); + output.V16 = step.V16; + output.V17 = step.V17; + output.V18 = step.V18; + output.V19 = step.V19; + output.V20 = Av1Transform1dMath.HalfButterfly(-cospi[32], step.V20, cospi[32], step.V27, cosBit); + output.V21 = Av1Transform1dMath.HalfButterfly(-cospi[32], step.V21, cospi[32], step.V26, cosBit); + output.V22 = Av1Transform1dMath.HalfButterfly(-cospi[32], step.V22, cospi[32], step.V25, cosBit); + output.V23 = Av1Transform1dMath.HalfButterfly(-cospi[32], step.V23, cospi[32], step.V24, cosBit); + output.V24 = Av1Transform1dMath.HalfButterfly(cospi[32], step.V23, cospi[32], step.V24, cosBit); + output.V25 = Av1Transform1dMath.HalfButterfly(cospi[32], step.V22, cospi[32], step.V25, cosBit); + output.V26 = Av1Transform1dMath.HalfButterfly(cospi[32], step.V21, cospi[32], step.V26, cosBit); + output.V27 = Av1Transform1dMath.HalfButterfly(cospi[32], step.V20, cospi[32], step.V27, cosBit); + output.V28 = step.V28; + output.V29 = step.V29; + output.V30 = step.V30; + output.V31 = step.V31; + output.V32 = Av1Transform1dMath.Clamp(step.V32 + step.V47, stageRange[stage]); + output.V33 = Av1Transform1dMath.Clamp(step.V33 + step.V46, stageRange[stage]); + output.V34 = Av1Transform1dMath.Clamp(step.V34 + step.V45, stageRange[stage]); + output.V35 = Av1Transform1dMath.Clamp(step.V35 + step.V44, stageRange[stage]); + output.V36 = Av1Transform1dMath.Clamp(step.V36 + step.V43, stageRange[stage]); + output.V37 = Av1Transform1dMath.Clamp(step.V37 + step.V42, stageRange[stage]); + output.V38 = Av1Transform1dMath.Clamp(step.V38 + step.V41, stageRange[stage]); + output.V39 = Av1Transform1dMath.Clamp(step.V39 + step.V40, stageRange[stage]); + output.V40 = Av1Transform1dMath.Clamp(step.V39 - step.V40, stageRange[stage]); + output.V41 = Av1Transform1dMath.Clamp(step.V38 - step.V41, stageRange[stage]); + output.V42 = Av1Transform1dMath.Clamp(step.V37 - step.V42, stageRange[stage]); + output.V43 = Av1Transform1dMath.Clamp(step.V36 - step.V43, stageRange[stage]); + output.V44 = Av1Transform1dMath.Clamp(step.V35 - step.V44, stageRange[stage]); + output.V45 = Av1Transform1dMath.Clamp(step.V34 - step.V45, stageRange[stage]); + output.V46 = Av1Transform1dMath.Clamp(step.V33 - step.V46, stageRange[stage]); + output.V47 = Av1Transform1dMath.Clamp(step.V32 - step.V47, stageRange[stage]); + output.V48 = Av1Transform1dMath.Clamp(-step.V48 + step.V63, stageRange[stage]); + output.V49 = Av1Transform1dMath.Clamp(-step.V49 + step.V62, stageRange[stage]); + output.V50 = Av1Transform1dMath.Clamp(-step.V50 + step.V61, stageRange[stage]); + output.V51 = Av1Transform1dMath.Clamp(-step.V51 + step.V60, stageRange[stage]); + output.V52 = Av1Transform1dMath.Clamp(-step.V52 + step.V59, stageRange[stage]); + output.V53 = Av1Transform1dMath.Clamp(-step.V53 + step.V58, stageRange[stage]); + output.V54 = Av1Transform1dMath.Clamp(-step.V54 + step.V57, stageRange[stage]); + output.V55 = Av1Transform1dMath.Clamp(-step.V55 + step.V56, stageRange[stage]); + output.V56 = Av1Transform1dMath.Clamp(step.V55 + step.V56, stageRange[stage]); + output.V57 = Av1Transform1dMath.Clamp(step.V54 + step.V57, stageRange[stage]); + output.V58 = Av1Transform1dMath.Clamp(step.V53 + step.V58, stageRange[stage]); + output.V59 = Av1Transform1dMath.Clamp(step.V52 + step.V59, stageRange[stage]); + output.V60 = Av1Transform1dMath.Clamp(step.V51 + step.V60, stageRange[stage]); + output.V61 = Av1Transform1dMath.Clamp(step.V50 + step.V61, stageRange[stage]); + output.V62 = Av1Transform1dMath.Clamp(step.V49 + step.V62, stageRange[stage]); + output.V63 = Av1Transform1dMath.Clamp(step.V48 + step.V63, stageRange[stage]); + + // Stage 10 applies the remaining pi/4 rotations before the terminal spatial merge. + stage++; + step.V0 = Av1Transform1dMath.Clamp(output.V0 + output.V31, stageRange[stage]); + step.V1 = Av1Transform1dMath.Clamp(output.V1 + output.V30, stageRange[stage]); + step.V2 = Av1Transform1dMath.Clamp(output.V2 + output.V29, stageRange[stage]); + step.V3 = Av1Transform1dMath.Clamp(output.V3 + output.V28, stageRange[stage]); + step.V4 = Av1Transform1dMath.Clamp(output.V4 + output.V27, stageRange[stage]); + step.V5 = Av1Transform1dMath.Clamp(output.V5 + output.V26, stageRange[stage]); + step.V6 = Av1Transform1dMath.Clamp(output.V6 + output.V25, stageRange[stage]); + step.V7 = Av1Transform1dMath.Clamp(output.V7 + output.V24, stageRange[stage]); + step.V8 = Av1Transform1dMath.Clamp(output.V8 + output.V23, stageRange[stage]); + step.V9 = Av1Transform1dMath.Clamp(output.V9 + output.V22, stageRange[stage]); + step.V10 = Av1Transform1dMath.Clamp(output.V10 + output.V21, stageRange[stage]); + step.V11 = Av1Transform1dMath.Clamp(output.V11 + output.V20, stageRange[stage]); + step.V12 = Av1Transform1dMath.Clamp(output.V12 + output.V19, stageRange[stage]); + step.V13 = Av1Transform1dMath.Clamp(output.V13 + output.V18, stageRange[stage]); + step.V14 = Av1Transform1dMath.Clamp(output.V14 + output.V17, stageRange[stage]); + step.V15 = Av1Transform1dMath.Clamp(output.V15 + output.V16, stageRange[stage]); + step.V16 = Av1Transform1dMath.Clamp(output.V15 - output.V16, stageRange[stage]); + step.V17 = Av1Transform1dMath.Clamp(output.V14 - output.V17, stageRange[stage]); + step.V18 = Av1Transform1dMath.Clamp(output.V13 - output.V18, stageRange[stage]); + step.V19 = Av1Transform1dMath.Clamp(output.V12 - output.V19, stageRange[stage]); + step.V20 = Av1Transform1dMath.Clamp(output.V11 - output.V20, stageRange[stage]); + step.V21 = Av1Transform1dMath.Clamp(output.V10 - output.V21, stageRange[stage]); + step.V22 = Av1Transform1dMath.Clamp(output.V9 - output.V22, stageRange[stage]); + step.V23 = Av1Transform1dMath.Clamp(output.V8 - output.V23, stageRange[stage]); + step.V24 = Av1Transform1dMath.Clamp(output.V7 - output.V24, stageRange[stage]); + step.V25 = Av1Transform1dMath.Clamp(output.V6 - output.V25, stageRange[stage]); + step.V26 = Av1Transform1dMath.Clamp(output.V5 - output.V26, stageRange[stage]); + step.V27 = Av1Transform1dMath.Clamp(output.V4 - output.V27, stageRange[stage]); + step.V28 = Av1Transform1dMath.Clamp(output.V3 - output.V28, stageRange[stage]); + step.V29 = Av1Transform1dMath.Clamp(output.V2 - output.V29, stageRange[stage]); + step.V30 = Av1Transform1dMath.Clamp(output.V1 - output.V30, stageRange[stage]); + step.V31 = Av1Transform1dMath.Clamp(output.V0 - output.V31, stageRange[stage]); + step.V32 = output.V32; + step.V33 = output.V33; + step.V34 = output.V34; + step.V35 = output.V35; + step.V36 = output.V36; + step.V37 = output.V37; + step.V38 = output.V38; + step.V39 = output.V39; + step.V40 = Av1Transform1dMath.HalfButterfly(-cospi[32], output.V40, cospi[32], output.V55, cosBit); + step.V41 = Av1Transform1dMath.HalfButterfly(-cospi[32], output.V41, cospi[32], output.V54, cosBit); + step.V42 = Av1Transform1dMath.HalfButterfly(-cospi[32], output.V42, cospi[32], output.V53, cosBit); + step.V43 = Av1Transform1dMath.HalfButterfly(-cospi[32], output.V43, cospi[32], output.V52, cosBit); + step.V44 = Av1Transform1dMath.HalfButterfly(-cospi[32], output.V44, cospi[32], output.V51, cosBit); + step.V45 = Av1Transform1dMath.HalfButterfly(-cospi[32], output.V45, cospi[32], output.V50, cosBit); + step.V46 = Av1Transform1dMath.HalfButterfly(-cospi[32], output.V46, cospi[32], output.V49, cosBit); + step.V47 = Av1Transform1dMath.HalfButterfly(-cospi[32], output.V47, cospi[32], output.V48, cosBit); + step.V48 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V47, cospi[32], output.V48, cosBit); + step.V49 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V46, cospi[32], output.V49, cosBit); + step.V50 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V45, cospi[32], output.V50, cosBit); + step.V51 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V44, cospi[32], output.V51, cosBit); + step.V52 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V43, cospi[32], output.V52, cosBit); + step.V53 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V42, cospi[32], output.V53, cosBit); + step.V54 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V41, cospi[32], output.V54, cosBit); + step.V55 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V40, cospi[32], output.V55, cosBit); + step.V56 = output.V56; + step.V57 = output.V57; + step.V58 = output.V58; + step.V59 = output.V59; + step.V60 = output.V60; + step.V61 = output.V61; + step.V62 = output.V62; + step.V63 = output.V63; + + // Stage 11 merges the even and odd halves into spatial order and clamps every result. + stage++; + output.V0 = Av1Transform1dMath.Clamp(step.V0 + step.V63, stageRange[stage]); + output.V1 = Av1Transform1dMath.Clamp(step.V1 + step.V62, stageRange[stage]); + output.V2 = Av1Transform1dMath.Clamp(step.V2 + step.V61, stageRange[stage]); + output.V3 = Av1Transform1dMath.Clamp(step.V3 + step.V60, stageRange[stage]); + output.V4 = Av1Transform1dMath.Clamp(step.V4 + step.V59, stageRange[stage]); + output.V5 = Av1Transform1dMath.Clamp(step.V5 + step.V58, stageRange[stage]); + output.V6 = Av1Transform1dMath.Clamp(step.V6 + step.V57, stageRange[stage]); + output.V7 = Av1Transform1dMath.Clamp(step.V7 + step.V56, stageRange[stage]); + output.V8 = Av1Transform1dMath.Clamp(step.V8 + step.V55, stageRange[stage]); + output.V9 = Av1Transform1dMath.Clamp(step.V9 + step.V54, stageRange[stage]); + output.V10 = Av1Transform1dMath.Clamp(step.V10 + step.V53, stageRange[stage]); + output.V11 = Av1Transform1dMath.Clamp(step.V11 + step.V52, stageRange[stage]); + output.V12 = Av1Transform1dMath.Clamp(step.V12 + step.V51, stageRange[stage]); + output.V13 = Av1Transform1dMath.Clamp(step.V13 + step.V50, stageRange[stage]); + output.V14 = Av1Transform1dMath.Clamp(step.V14 + step.V49, stageRange[stage]); + output.V15 = Av1Transform1dMath.Clamp(step.V15 + step.V48, stageRange[stage]); + output.V16 = Av1Transform1dMath.Clamp(step.V16 + step.V47, stageRange[stage]); + output.V17 = Av1Transform1dMath.Clamp(step.V17 + step.V46, stageRange[stage]); + output.V18 = Av1Transform1dMath.Clamp(step.V18 + step.V45, stageRange[stage]); + output.V19 = Av1Transform1dMath.Clamp(step.V19 + step.V44, stageRange[stage]); + output.V20 = Av1Transform1dMath.Clamp(step.V20 + step.V43, stageRange[stage]); + output.V21 = Av1Transform1dMath.Clamp(step.V21 + step.V42, stageRange[stage]); + output.V22 = Av1Transform1dMath.Clamp(step.V22 + step.V41, stageRange[stage]); + output.V23 = Av1Transform1dMath.Clamp(step.V23 + step.V40, stageRange[stage]); + output.V24 = Av1Transform1dMath.Clamp(step.V24 + step.V39, stageRange[stage]); + output.V25 = Av1Transform1dMath.Clamp(step.V25 + step.V38, stageRange[stage]); + output.V26 = Av1Transform1dMath.Clamp(step.V26 + step.V37, stageRange[stage]); + output.V27 = Av1Transform1dMath.Clamp(step.V27 + step.V36, stageRange[stage]); + output.V28 = Av1Transform1dMath.Clamp(step.V28 + step.V35, stageRange[stage]); + output.V29 = Av1Transform1dMath.Clamp(step.V29 + step.V34, stageRange[stage]); + output.V30 = Av1Transform1dMath.Clamp(step.V30 + step.V33, stageRange[stage]); + output.V31 = Av1Transform1dMath.Clamp(step.V31 + step.V32, stageRange[stage]); + output.V32 = Av1Transform1dMath.Clamp(step.V31 - step.V32, stageRange[stage]); + output.V33 = Av1Transform1dMath.Clamp(step.V30 - step.V33, stageRange[stage]); + output.V34 = Av1Transform1dMath.Clamp(step.V29 - step.V34, stageRange[stage]); + output.V35 = Av1Transform1dMath.Clamp(step.V28 - step.V35, stageRange[stage]); + output.V36 = Av1Transform1dMath.Clamp(step.V27 - step.V36, stageRange[stage]); + output.V37 = Av1Transform1dMath.Clamp(step.V26 - step.V37, stageRange[stage]); + output.V38 = Av1Transform1dMath.Clamp(step.V25 - step.V38, stageRange[stage]); + output.V39 = Av1Transform1dMath.Clamp(step.V24 - step.V39, stageRange[stage]); + output.V40 = Av1Transform1dMath.Clamp(step.V23 - step.V40, stageRange[stage]); + output.V41 = Av1Transform1dMath.Clamp(step.V22 - step.V41, stageRange[stage]); + output.V42 = Av1Transform1dMath.Clamp(step.V21 - step.V42, stageRange[stage]); + output.V43 = Av1Transform1dMath.Clamp(step.V20 - step.V43, stageRange[stage]); + output.V44 = Av1Transform1dMath.Clamp(step.V19 - step.V44, stageRange[stage]); + output.V45 = Av1Transform1dMath.Clamp(step.V18 - step.V45, stageRange[stage]); + output.V46 = Av1Transform1dMath.Clamp(step.V17 - step.V46, stageRange[stage]); + output.V47 = Av1Transform1dMath.Clamp(step.V16 - step.V47, stageRange[stage]); + output.V48 = Av1Transform1dMath.Clamp(step.V15 - step.V48, stageRange[stage]); + output.V49 = Av1Transform1dMath.Clamp(step.V14 - step.V49, stageRange[stage]); + output.V50 = Av1Transform1dMath.Clamp(step.V13 - step.V50, stageRange[stage]); + output.V51 = Av1Transform1dMath.Clamp(step.V12 - step.V51, stageRange[stage]); + output.V52 = Av1Transform1dMath.Clamp(step.V11 - step.V52, stageRange[stage]); + output.V53 = Av1Transform1dMath.Clamp(step.V10 - step.V53, stageRange[stage]); + output.V54 = Av1Transform1dMath.Clamp(step.V9 - step.V54, stageRange[stage]); + output.V55 = Av1Transform1dMath.Clamp(step.V8 - step.V55, stageRange[stage]); + output.V56 = Av1Transform1dMath.Clamp(step.V7 - step.V56, stageRange[stage]); + output.V57 = Av1Transform1dMath.Clamp(step.V6 - step.V57, stageRange[stage]); + output.V58 = Av1Transform1dMath.Clamp(step.V5 - step.V58, stageRange[stage]); + output.V59 = Av1Transform1dMath.Clamp(step.V4 - step.V59, stageRange[stage]); + output.V60 = Av1Transform1dMath.Clamp(step.V3 - step.V60, stageRange[stage]); + output.V61 = Av1Transform1dMath.Clamp(step.V2 - step.V61, stageRange[stage]); + output.V62 = Av1Transform1dMath.Clamp(step.V1 - step.V62, stageRange[stage]); + output.V63 = Av1Transform1dMath.Clamp(step.V0 - step.V63, stageRange[stage]); + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Transform/Inverse/Av1Inverse2dTransformer.Dct8Operator.cs b/src/ImageSharp/Formats/Heif/Av1/Transform/Inverse/Av1Inverse2dTransformer.Dct8Operator.cs new file mode 100644 index 0000000000..409b973301 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Transform/Inverse/Av1Inverse2dTransformer.Dct8Operator.cs @@ -0,0 +1,235 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +/// +/// Defines the eight-point AV1 inverse discrete cosine transform operator. +/// +/// +/// Vector fields represent transform positions and vector lanes represent independent axes. The SIMD overloads apply +/// the same staged butterflies, fixed-point rounding, and range clamps as the scalar overload without mixing axes. +/// +internal static partial class Av1Inverse2dTransformer +{ + internal readonly struct Dct8Operator : IAv1Transform1dOperator + { + /// + /// Applies the normative eight-point AV1 inverse discrete cosine transform. + /// + /// The eight frequency-domain coefficients. + /// The eight spatial-domain residual values. + /// The eight-element stage buffer owned by the containing two-dimensional transform. + /// The fixed-point precision of the cosine constants. + /// The signed-bit range assigned to each transform stage. + public static void Transform(ReadOnlySpan input, Span output, Span step, int cosBit, InlineArray12 stageRange) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + int stage = 0; + + // Stage 1 permutes frequency-ordered coefficients into the recursive DCT factorization order. + stage++; + output[0] = input[0]; + output[1] = input[4]; + output[2] = input[2]; + output[3] = input[6]; + output[4] = input[1]; + output[5] = input[5]; + output[6] = input[3]; + output[7] = input[7]; + + // Stage 2 rotates the odd-frequency coefficient pairs by their pi/16 angles. + stage++; + step[0] = output[0]; + step[1] = output[1]; + step[2] = output[2]; + step[3] = output[3]; + step[4] = Av1Transform1dMath.HalfButterfly(cospi[56], output[4], -cospi[8], output[7], cosBit); + step[5] = Av1Transform1dMath.HalfButterfly(cospi[24], output[5], -cospi[40], output[6], cosBit); + step[6] = Av1Transform1dMath.HalfButterfly(cospi[40], output[5], cospi[24], output[6], cosBit); + step[7] = Av1Transform1dMath.HalfButterfly(cospi[8], output[4], cospi[56], output[7], cosBit); + + // Stage 3 reconstructs the even four-point DCT and combines adjacent odd terms. + stage++; + byte range = stageRange[stage]; + output[0] = Av1Transform1dMath.HalfButterfly(cospi[32], step[0], cospi[32], step[1], cosBit); + output[1] = Av1Transform1dMath.HalfButterfly(cospi[32], step[0], -cospi[32], step[1], cosBit); + output[2] = Av1Transform1dMath.HalfButterfly(cospi[48], step[2], -cospi[16], step[3], cosBit); + output[3] = Av1Transform1dMath.HalfButterfly(cospi[16], step[2], cospi[48], step[3], cosBit); + output[4] = Av1Transform1dMath.Clamp(step[4] + step[5], range); + output[5] = Av1Transform1dMath.Clamp(step[4] - step[5], range); + output[6] = Av1Transform1dMath.Clamp(step[7] - step[6], range); + output[7] = Av1Transform1dMath.Clamp(step[6] + step[7], range); + + // Stage 4 completes the even butterflies and applies the remaining pi/4 odd rotation. + stage++; + step[0] = Av1Transform1dMath.Clamp(output[0] + output[3], range); + step[1] = Av1Transform1dMath.Clamp(output[1] + output[2], range); + step[2] = Av1Transform1dMath.Clamp(output[1] - output[2], range); + step[3] = Av1Transform1dMath.Clamp(output[0] - output[3], range); + step[4] = output[4]; + step[5] = Av1Transform1dMath.HalfButterfly(-cospi[32], output[5], cospi[32], output[6], cosBit); + step[6] = Av1Transform1dMath.HalfButterfly(cospi[32], output[5], cospi[32], output[6], cosBit); + step[7] = output[7]; + + // Stage 5 merges the even and odd halves into spatial order and clamps every result. + stage++; + range = stageRange[stage]; + output[0] = Av1Transform1dMath.Clamp(step[0] + step[7], range); + output[1] = Av1Transform1dMath.Clamp(step[1] + step[6], range); + output[2] = Av1Transform1dMath.Clamp(step[2] + step[5], range); + output[3] = Av1Transform1dMath.Clamp(step[3] + step[4], range); + output[4] = Av1Transform1dMath.Clamp(step[3] - step[4], range); + output[5] = Av1Transform1dMath.Clamp(step[2] - step[5], range); + output[6] = Av1Transform1dMath.Clamp(step[1] - step[6], range); + output[7] = Av1Transform1dMath.Clamp(step[0] - step[7], range); + } + + /// + public static void Transform( + ref Av1TransformVector> input, + ref Av1TransformVector> output, + ref Av1TransformVector> step, + int cosBit, + InlineArray12 stageRange) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + int stage = 0; + + // Stage 1 permutes frequency-ordered coefficients into the recursive DCT factorization order. + stage++; + output.V0 = input.V0; + output.V1 = input.V4; + output.V2 = input.V2; + output.V3 = input.V6; + output.V4 = input.V1; + output.V5 = input.V5; + output.V6 = input.V3; + output.V7 = input.V7; + + // Stage 2 rotates the odd-frequency coefficient pairs by their pi/16 angles. + stage++; + step.V0 = output.V0; + step.V1 = output.V1; + step.V2 = output.V2; + step.V3 = output.V3; + step.V4 = Av1Transform1dMath.HalfButterfly(cospi[56], output.V4, -cospi[8], output.V7, cosBit); + step.V5 = Av1Transform1dMath.HalfButterfly(cospi[24], output.V5, -cospi[40], output.V6, cosBit); + step.V6 = Av1Transform1dMath.HalfButterfly(cospi[40], output.V5, cospi[24], output.V6, cosBit); + step.V7 = Av1Transform1dMath.HalfButterfly(cospi[8], output.V4, cospi[56], output.V7, cosBit); + + // Stage 3 reconstructs the even four-point DCT and combines adjacent odd terms. + stage++; + byte range = stageRange[stage]; + output.V0 = Av1Transform1dMath.HalfButterfly(cospi[32], step.V0, cospi[32], step.V1, cosBit); + output.V1 = Av1Transform1dMath.HalfButterfly(cospi[32], step.V0, -cospi[32], step.V1, cosBit); + output.V2 = Av1Transform1dMath.HalfButterfly(cospi[48], step.V2, -cospi[16], step.V3, cosBit); + output.V3 = Av1Transform1dMath.HalfButterfly(cospi[16], step.V2, cospi[48], step.V3, cosBit); + output.V4 = Av1Transform1dMath.Clamp(step.V4 + step.V5, range); + output.V5 = Av1Transform1dMath.Clamp(step.V4 - step.V5, range); + output.V6 = Av1Transform1dMath.Clamp(step.V7 - step.V6, range); + output.V7 = Av1Transform1dMath.Clamp(step.V6 + step.V7, range); + + // Stage 4 completes the even butterflies and applies the remaining pi/4 odd rotation. + stage++; + step.V0 = Av1Transform1dMath.Clamp(output.V0 + output.V3, range); + step.V1 = Av1Transform1dMath.Clamp(output.V1 + output.V2, range); + step.V2 = Av1Transform1dMath.Clamp(output.V1 - output.V2, range); + step.V3 = Av1Transform1dMath.Clamp(output.V0 - output.V3, range); + step.V4 = output.V4; + step.V5 = Av1Transform1dMath.HalfButterfly(-cospi[32], output.V5, cospi[32], output.V6, cosBit); + step.V6 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V5, cospi[32], output.V6, cosBit); + step.V7 = output.V7; + + // Stage 5 merges the even and odd halves into spatial order and clamps every result. + stage++; + range = stageRange[stage]; + output.V0 = Av1Transform1dMath.Clamp(step.V0 + step.V7, range); + output.V1 = Av1Transform1dMath.Clamp(step.V1 + step.V6, range); + output.V2 = Av1Transform1dMath.Clamp(step.V2 + step.V5, range); + output.V3 = Av1Transform1dMath.Clamp(step.V3 + step.V4, range); + output.V4 = Av1Transform1dMath.Clamp(step.V3 - step.V4, range); + output.V5 = Av1Transform1dMath.Clamp(step.V2 - step.V5, range); + output.V6 = Av1Transform1dMath.Clamp(step.V1 - step.V6, range); + output.V7 = Av1Transform1dMath.Clamp(step.V0 - step.V7, range); + } + + /// + /// Applies the transform to four independent axes in parallel. + /// + /// The source values for the parallel transform axes. + /// The destination values for the parallel transform axes. + /// The fixed stage storage for the parallel transform axes. + /// The fixed-point precision of the cosine constants. + /// The signed-bit range assigned to each transform stage. + public static void Transform( + ref Av1TransformVector> input, + ref Av1TransformVector> output, + ref Av1TransformVector> step, + int cosBit, + InlineArray12 stageRange) + { + ReadOnlySpan cospi = Av1SinusConstants.CosinusPi(cosBit); + int stage = 0; + + // Stage 1 permutes frequency-ordered coefficients into the recursive DCT factorization order. + stage++; + output.V0 = input.V0; + output.V1 = input.V4; + output.V2 = input.V2; + output.V3 = input.V6; + output.V4 = input.V1; + output.V5 = input.V5; + output.V6 = input.V3; + output.V7 = input.V7; + + // Stage 2 rotates the odd-frequency coefficient pairs by their pi/16 angles. + stage++; + step.V0 = output.V0; + step.V1 = output.V1; + step.V2 = output.V2; + step.V3 = output.V3; + step.V4 = Av1Transform1dMath.HalfButterfly(cospi[56], output.V4, -cospi[8], output.V7, cosBit); + step.V5 = Av1Transform1dMath.HalfButterfly(cospi[24], output.V5, -cospi[40], output.V6, cosBit); + step.V6 = Av1Transform1dMath.HalfButterfly(cospi[40], output.V5, cospi[24], output.V6, cosBit); + step.V7 = Av1Transform1dMath.HalfButterfly(cospi[8], output.V4, cospi[56], output.V7, cosBit); + + // Stage 3 reconstructs the even four-point DCT and combines adjacent odd terms. + stage++; + byte range = stageRange[stage]; + output.V0 = Av1Transform1dMath.HalfButterfly(cospi[32], step.V0, cospi[32], step.V1, cosBit); + output.V1 = Av1Transform1dMath.HalfButterfly(cospi[32], step.V0, -cospi[32], step.V1, cosBit); + output.V2 = Av1Transform1dMath.HalfButterfly(cospi[48], step.V2, -cospi[16], step.V3, cosBit); + output.V3 = Av1Transform1dMath.HalfButterfly(cospi[16], step.V2, cospi[48], step.V3, cosBit); + output.V4 = Av1Transform1dMath.Clamp(step.V4 + step.V5, range); + output.V5 = Av1Transform1dMath.Clamp(step.V4 - step.V5, range); + output.V6 = Av1Transform1dMath.Clamp(step.V7 - step.V6, range); + output.V7 = Av1Transform1dMath.Clamp(step.V6 + step.V7, range); + + // Stage 4 completes the even butterflies and applies the remaining pi/4 odd rotation. + stage++; + step.V0 = Av1Transform1dMath.Clamp(output.V0 + output.V3, range); + step.V1 = Av1Transform1dMath.Clamp(output.V1 + output.V2, range); + step.V2 = Av1Transform1dMath.Clamp(output.V1 - output.V2, range); + step.V3 = Av1Transform1dMath.Clamp(output.V0 - output.V3, range); + step.V4 = output.V4; + step.V5 = Av1Transform1dMath.HalfButterfly(-cospi[32], output.V5, cospi[32], output.V6, cosBit); + step.V6 = Av1Transform1dMath.HalfButterfly(cospi[32], output.V5, cospi[32], output.V6, cosBit); + step.V7 = output.V7; + + // Stage 5 merges the even and odd halves into spatial order and clamps every result. + stage++; + range = stageRange[stage]; + output.V0 = Av1Transform1dMath.Clamp(step.V0 + step.V7, range); + output.V1 = Av1Transform1dMath.Clamp(step.V1 + step.V6, range); + output.V2 = Av1Transform1dMath.Clamp(step.V2 + step.V5, range); + output.V3 = Av1Transform1dMath.Clamp(step.V3 + step.V4, range); + output.V4 = Av1Transform1dMath.Clamp(step.V3 - step.V4, range); + output.V5 = Av1Transform1dMath.Clamp(step.V2 - step.V5, range); + output.V6 = Av1Transform1dMath.Clamp(step.V1 - step.V6, range); + output.V7 = Av1Transform1dMath.Clamp(step.V0 - step.V7, range); + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Transform/Inverse/Av1Inverse2dTransformer.Identity16Operator.cs b/src/ImageSharp/Formats/Heif/Av1/Transform/Inverse/Av1Inverse2dTransformer.Identity16Operator.cs new file mode 100644 index 0000000000..3d2a7d8d0b --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Transform/Inverse/Av1Inverse2dTransformer.Identity16Operator.cs @@ -0,0 +1,84 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +/// +/// Defines the sixteen-point AV1 inverse identity transform operator. +/// +/// +/// Vector fields represent transform positions and vector lanes represent independent axes. Scaling is lane-local, +/// so the SIMD overloads preserve the scalar fixed-point multiplier and rounding for every axis. +/// +internal static partial class Av1Inverse2dTransformer +{ + internal readonly struct Identity16Operator : IAv1Transform1dOperator + { + /// + /// Applies the normative sixteen-point AV1 inverse identity transform. + /// + /// The sixteen frequency-domain coefficients. + /// The sixteen scaled spatial-domain values. + /// Unused stage storage supplied by the common transform-kernel contract. + /// Unused cosine precision supplied by the common transform-kernel contract. + /// The signed-bit range assigned to the transform output. + public static void Transform(ReadOnlySpan input, Span output, Span step, int cosBit, InlineArray12 stageRange) + { + _ = step; + _ = cosBit; + _ = stageRange; + + // The AV1 identity transform preserves coefficient order while applying the twice the square-root-of-two fixed-point scale required for 2-D normalization. + for (int i = 0; i < 16; i++) + { + output[i] = Av1Math.RoundShift((long)input[i] * (2 * Av1Transform1dMath.NewSqrt2), Av1Transform1dMath.NewSqrt2Bits); + } + } + + /// + public static void Transform( + ref Av1TransformVector> input, + ref Av1TransformVector> output, + ref Av1TransformVector> step, + int cosBit, + InlineArray12 stageRange) + { + // The doubled scale exceeds Int32 only for the 20-bit twelve-bit row range. Widen that exact product and + // rounding sequence, matching the reference decoder without changing the established lower-range SIMD path. + if (stageRange[0] >= Av1Transform1dMath.WidenedIntermediateBitCount) + { + Av1IdentityTransform1d.TransformWidened(ref input, ref output, 16, 2 * Av1Transform1dMath.NewSqrt2, Av1Transform1dMath.NewSqrt2Bits); + } + else + { + Av1IdentityTransform1d.Transform(ref input, ref output, 16, 2 * Av1Transform1dMath.NewSqrt2, Av1Transform1dMath.NewSqrt2Bits); + } + + _ = step; + _ = cosBit; + } + + /// + public static void Transform( + ref Av1TransformVector> input, + ref Av1TransformVector> output, + ref Av1TransformVector> step, + int cosBit, + InlineArray12 stageRange) + { + if (stageRange[0] >= Av1Transform1dMath.WidenedIntermediateBitCount) + { + Av1IdentityTransform1d.TransformWidened(ref input, ref output, 16, 2 * Av1Transform1dMath.NewSqrt2, Av1Transform1dMath.NewSqrt2Bits); + } + else + { + Av1IdentityTransform1d.Transform(ref input, ref output, 16, 2 * Av1Transform1dMath.NewSqrt2, Av1Transform1dMath.NewSqrt2Bits); + } + + _ = step; + _ = cosBit; + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Transform/Inverse/Av1Inverse2dTransformer.Identity32Operator.cs b/src/ImageSharp/Formats/Heif/Av1/Transform/Inverse/Av1Inverse2dTransformer.Identity32Operator.cs new file mode 100644 index 0000000000..933fb8060b --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Transform/Inverse/Av1Inverse2dTransformer.Identity32Operator.cs @@ -0,0 +1,68 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +/// +/// Defines the thirty-two-point AV1 inverse identity transform operator. +/// +/// +/// Vector fields represent transform positions and vector lanes represent independent axes. Scaling is lane-local, +/// so the SIMD overloads preserve the scalar fixed-point multiplier and rounding for every axis. +/// +internal static partial class Av1Inverse2dTransformer +{ + internal readonly struct Identity32Operator : IAv1Transform1dOperator + { + /// + /// Applies the normative thirty-two-point AV1 inverse identity transform. + /// + /// The thirty-two frequency-domain coefficients. + /// The thirty-two scaled spatial-domain values. + /// Unused stage storage supplied by the common transform-kernel contract. + /// Unused cosine precision supplied by the common transform-kernel contract. + /// The signed-bit range assigned to the transform output. + public static void Transform(ReadOnlySpan input, Span output, Span step, int cosBit, InlineArray12 stageRange) + { + _ = step; + _ = cosBit; + _ = stageRange; + + // The AV1 identity transform preserves coefficient order while applying the exact factor-of-four scale required for 2-D normalization. + for (int i = 0; i < 32; i++) + { + output[i] = input[i] * 4; + } + } + + /// + public static void Transform( + ref Av1TransformVector> input, + ref Av1TransformVector> output, + ref Av1TransformVector> step, + int cosBit, + InlineArray12 stageRange) + { + Av1IdentityTransform1d.Transform(ref input, ref output, 32, 4, 0); + _ = step; + _ = cosBit; + _ = stageRange; + } + + /// + public static void Transform( + ref Av1TransformVector> input, + ref Av1TransformVector> output, + ref Av1TransformVector> step, + int cosBit, + InlineArray12 stageRange) + { + Av1IdentityTransform1d.Transform(ref input, ref output, 32, 4, 0); + _ = step; + _ = cosBit; + _ = stageRange; + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Transform/Inverse/Av1Inverse2dTransformer.Identity4Operator.cs b/src/ImageSharp/Formats/Heif/Av1/Transform/Inverse/Av1Inverse2dTransformer.Identity4Operator.cs new file mode 100644 index 0000000000..5789e0e842 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Transform/Inverse/Av1Inverse2dTransformer.Identity4Operator.cs @@ -0,0 +1,84 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +/// +/// Defines the four-point AV1 inverse identity transform operator. +/// +/// +/// Vector fields represent transform positions and vector lanes represent independent axes. Scaling is lane-local, +/// so the SIMD overloads preserve the scalar fixed-point multiplier and rounding for every axis. +/// +internal static partial class Av1Inverse2dTransformer +{ + internal readonly struct Identity4Operator : IAv1Transform1dOperator + { + /// + /// Applies the normative four-point AV1 inverse identity transform. + /// + /// The four frequency-domain coefficients. + /// The four scaled spatial-domain values. + /// Unused stage storage supplied by the common transform-kernel contract. + /// Unused cosine precision supplied by the common transform-kernel contract. + /// The signed-bit range assigned to the transform output. + public static void Transform(ReadOnlySpan input, Span output, Span step, int cosBit, InlineArray12 stageRange) + { + _ = step; + _ = cosBit; + _ = stageRange; + + // The AV1 identity transform preserves coefficient order while applying the square-root-of-two fixed-point scale required for 2-D normalization. + for (int i = 0; i < 4; i++) + { + output[i] = Av1Math.RoundShift((long)input[i] * Av1Transform1dMath.NewSqrt2, Av1Transform1dMath.NewSqrt2Bits); + } + } + + /// + public static void Transform( + ref Av1TransformVector> input, + ref Av1TransformVector> output, + ref Av1TransformVector> step, + int cosBit, + InlineArray12 stageRange) + { + // Only a twelve-bit row transform has the 20-bit input range that can overflow this fixed-point product. + // Match the reference decoder's high-bit-depth kernel there while retaining the compact Int32 path for narrower ranges. + if (stageRange[0] >= Av1Transform1dMath.WidenedIntermediateBitCount) + { + Av1IdentityTransform1d.TransformWidened(ref input, ref output, 4, Av1Transform1dMath.NewSqrt2, Av1Transform1dMath.NewSqrt2Bits); + } + else + { + Av1IdentityTransform1d.Transform(ref input, ref output, 4, Av1Transform1dMath.NewSqrt2, Av1Transform1dMath.NewSqrt2Bits); + } + + _ = step; + _ = cosBit; + } + + /// + public static void Transform( + ref Av1TransformVector> input, + ref Av1TransformVector> output, + ref Av1TransformVector> step, + int cosBit, + InlineArray12 stageRange) + { + if (stageRange[0] >= Av1Transform1dMath.WidenedIntermediateBitCount) + { + Av1IdentityTransform1d.TransformWidened(ref input, ref output, 4, Av1Transform1dMath.NewSqrt2, Av1Transform1dMath.NewSqrt2Bits); + } + else + { + Av1IdentityTransform1d.Transform(ref input, ref output, 4, Av1Transform1dMath.NewSqrt2, Av1Transform1dMath.NewSqrt2Bits); + } + + _ = step; + _ = cosBit; + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Transform/Inverse/Av1Inverse2dTransformer.Identity8Operator.cs b/src/ImageSharp/Formats/Heif/Av1/Transform/Inverse/Av1Inverse2dTransformer.Identity8Operator.cs new file mode 100644 index 0000000000..c4d6f36f7c --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Transform/Inverse/Av1Inverse2dTransformer.Identity8Operator.cs @@ -0,0 +1,68 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +/// +/// Defines the eight-point AV1 inverse identity transform operator. +/// +/// +/// Vector fields represent transform positions and vector lanes represent independent axes. Scaling is lane-local, +/// so the SIMD overloads preserve the scalar fixed-point multiplier and rounding for every axis. +/// +internal static partial class Av1Inverse2dTransformer +{ + internal readonly struct Identity8Operator : IAv1Transform1dOperator + { + /// + /// Applies the normative eight-point AV1 inverse identity transform. + /// + /// The eight frequency-domain coefficients. + /// The eight scaled spatial-domain values. + /// Unused stage storage supplied by the common transform-kernel contract. + /// Unused cosine precision supplied by the common transform-kernel contract. + /// The signed-bit range assigned to the transform output. + public static void Transform(ReadOnlySpan input, Span output, Span step, int cosBit, InlineArray12 stageRange) + { + _ = step; + _ = cosBit; + _ = stageRange; + + // The AV1 identity transform preserves coefficient order while applying the exact factor-of-two scale required for 2-D normalization. + for (int i = 0; i < 8; i++) + { + output[i] = input[i] * 2; + } + } + + /// + public static void Transform( + ref Av1TransformVector> input, + ref Av1TransformVector> output, + ref Av1TransformVector> step, + int cosBit, + InlineArray12 stageRange) + { + Av1IdentityTransform1d.Transform(ref input, ref output, 8, 2, 0); + _ = step; + _ = cosBit; + _ = stageRange; + } + + /// + public static void Transform( + ref Av1TransformVector> input, + ref Av1TransformVector> output, + ref Av1TransformVector> step, + int cosBit, + InlineArray12 stageRange) + { + Av1IdentityTransform1d.Transform(ref input, ref output, 8, 2, 0); + _ = step; + _ = cosBit; + _ = stageRange; + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Transform/Inverse/Av1Inverse2dTransformer.Operator.cs b/src/ImageSharp/Formats/Heif/Av1/Transform/Inverse/Av1Inverse2dTransformer.Operator.cs new file mode 100644 index 0000000000..d88e575d3d --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Transform/Inverse/Av1Inverse2dTransformer.Operator.cs @@ -0,0 +1,62 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +/// +/// Defines the inverse-transform operator contract. +/// +internal static partial class Av1Inverse2dTransformer +{ + /// + /// Defines the scalar and SIMD arithmetic for one AV1 one-dimensional inverse transform. + /// + /// + /// Each overload performs the same staged fixed-point transform. Vector fields identify coefficient positions, + /// while vector lanes identify independent rows or columns. + /// + internal interface IAv1Transform1dOperator + { + /// + /// Transforms one axis when hardware vectorization is unavailable. + /// + /// The source values for the transform axis. + /// The destination values for the transform axis. + /// The fixed stage storage for the transform axis. + /// The fixed-point precision of the cosine constants. + /// The signed-bit range assigned to each transform stage. + public static abstract void Transform(ReadOnlySpan input, Span output, Span step, int cosBit, InlineArray12 stageRange); + + /// + /// Transforms four independent axes in parallel. + /// + /// The source values for four transform axes. + /// The destination values for four transform axes. + /// The fixed stage storage for four transform axes. + /// The fixed-point precision of the cosine constants. + /// The signed-bit range assigned to each transform stage. + public static abstract void Transform( + ref Av1TransformVector> input, + ref Av1TransformVector> output, + ref Av1TransformVector> step, + int cosBit, + InlineArray12 stageRange); + + /// + /// Transforms eight independent axes in parallel. + /// + /// The source values for eight transform axes. + /// The destination values for eight transform axes. + /// The fixed stage storage for eight transform axes. + /// The fixed-point precision of the cosine constants. + /// The signed-bit range assigned to each transform stage. + public static abstract void Transform( + ref Av1TransformVector> input, + ref Av1TransformVector> output, + ref Av1TransformVector> step, + int cosBit, + InlineArray12 stageRange); + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/av1-spec.pdf b/src/ImageSharp/Formats/Heif/Av1/av1-spec.pdf new file mode 100644 index 0000000000..7b634be629 Binary files /dev/null and b/src/ImageSharp/Formats/Heif/Av1/av1-spec.pdf differ diff --git a/src/ImageSharp/Formats/Heif/Av1HeifItemDecoder.cs b/src/ImageSharp/Formats/Heif/Av1HeifItemDecoder.cs new file mode 100644 index 0000000000..eb496c9f2e --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1HeifItemDecoder.cs @@ -0,0 +1,167 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1; +using SixLabors.ImageSharp.Formats.Heif.Components.Alpha; +using SixLabors.ImageSharp.Metadata.Profiles.Cicp; +using SixLabors.ImageSharp.PixelFormats; + +namespace SixLabors.ImageSharp.Formats.Heif; + +/// +/// Decodes a single AV1-coded HEIF image item. +/// +/// The destination pixel type. +internal sealed class Av1HeifItemDecoder : IHeifItemDecoder, IHeifAlphaItemDecoder + where TPixel : unmanaged, IPixel +{ + /// + /// Gets the AV1-coded image item type. + /// + public Heif4CharCode Type => Heif4CharCode.Av01; + + /// + /// Gets the AV1 compression method. + /// + public HeifCompressionMethod CompressionMethod => HeifCompressionMethod.Av1; + + /// + /// Decodes the encoded AV1 payload of an image item. + /// + /// The general options governing the containing HEIF decode. + /// The HEIF item whose encoded payload is being decoded. + /// The encoded AV1 payload. + /// + /// The container color description that supplies unspecified color information in the AV1 sequence header. + /// + /// The token used to cancel the payload decode. + /// The decoded image. + public Image DecodeItemData( + DecoderOptions options, + HeifItem item, + Span data, + CicpProfile? colorProfile, + CancellationToken cancellationToken) + { + cancellationToken.ThrowIfCancellationRequested(); + Span itemData = GetItemData(item, data); + Av1CodecConfiguration codecConfiguration = ValidateItemData( + options, + item, + itemData, + out HeifContentLightLevel? obuContentLightLevel, + out HeifMasteringDisplayColorVolume? obuMasteringDisplayColorVolume); + + byte operatingPointIndex = item.Av1OperatingPointSelector?.Index ?? 0; + + using Av1Decoder decoder = new(options.Configuration, operatingPointIndex); + Image image = decoder.Decode( + itemData, + colorProfile, + codecConfiguration, + item.Av1LayeredImageIndex, + item.Extent); + + HeifMetadata metadata = image.Metadata.GetHeifMetadata(); + metadata.CompressionMethod = this.CompressionMethod; + metadata.BitDepth = codecConfiguration.BitDepth; + metadata.IsMonochrome = codecConfiguration.IsMonochrome; + metadata.ContentLightLevel = item.ContentLightLevel ?? obuContentLightLevel; + metadata.MasteringDisplayColorVolume = item.MasteringDisplayColorVolume ?? obuMasteringDisplayColorVolume; + return image; + } + + /// + public void DecodeAlphaItemData( + DecoderOptions options, + HeifItem item, + Span data, + ImageFrame destination, + Size outputSize, + Rectangle destinationRectangle, + bool premultiplied, + CancellationToken cancellationToken) + { + cancellationToken.ThrowIfCancellationRequested(); + Span itemData = GetItemData(item, data); + Av1CodecConfiguration codecConfiguration = ValidateItemData(options, item, itemData, out _, out _); + if (!codecConfiguration.IsMonochrome) + { + throw new InvalidImageContentException($"AV1 alpha image item {item.Id} is not monochrome."); + } + + byte operatingPointIndex = item.Av1OperatingPointSelector?.Index ?? 0; + + using Av1Decoder decoder = new(options.Configuration, operatingPointIndex); + decoder.DecodeAlpha( + itemData, + item.CicpProfile, + codecConfiguration, + default, + destination, + outputSize, + destinationRectangle, + premultiplied, + item.Av1LayeredImageIndex); + } + + /// + /// Gets the cumulative item bytes required by an explicit AV1 spatial-layer selection. + /// + /// The AV1 image item containing optional layered-image properties. + /// The complete logical image-item payload. + /// The complete payload for final-layer decoding, or the cumulative prefix through the selected layer. + private static Span GetItemData(HeifItem item, Span data) + { + Av1LayeredImageIndex? layeredImageIndex = item.Av1LayeredImageIndex; + if (layeredImageIndex is null) + { + return data; + } + + int payloadLength = layeredImageIndex.Value.GetPayloadLength(data.Length, item.Av1LayerSelector); + return data[..payloadLength]; + } + + /// + /// Validates an AV1 item description against its encoded payload and returns the required codec configuration. + /// + /// The general options governing the containing HEIF decode. + /// The AV1 image item being validated. + /// The encoded AV1 payload. + /// Receives content-light metadata found in the AV1 payload. + /// Receives mastering-display metadata found in the AV1 payload. + /// The validated item-associated AV1 codec configuration. + private static Av1CodecConfiguration ValidateItemData( + DecoderOptions options, + HeifItem item, + ReadOnlySpan data, + out HeifContentLightLevel? obuContentLightLevel, + out HeifMasteringDisplayColorVolume? obuMasteringDisplayColorVolume) + { + Av1CodecConfiguration codecConfiguration = item.Av1CodecConfiguration + ?? throw new InvalidImageContentException($"AV1 image item {item.Id} has no codec configuration property."); + + if (item.ChannelBitDepths is not null) + { + foreach (byte channelBitDepth in item.ChannelBitDepths) + { + if (channelBitDepth != (byte)codecConfiguration.BitDepth) + { + throw new InvalidImageContentException( + $"AV1 image item {item.Id} has mismatched pixel-information and codec-configuration bit depths."); + } + } + } + + codecConfiguration.ValidateItemData( + data, + item.ContentLightLevel, + item.MasteringDisplayColorVolume, + options, + out obuContentLightLevel, + out obuMasteringDisplayColorVolume); + + return codecConfiguration; + } +} diff --git a/src/ImageSharp/Formats/Heif/Components/Alpha/HeifPlanarAlphaCompositor.cs b/src/ImageSharp/Formats/Heif/Components/Alpha/HeifPlanarAlphaCompositor.cs new file mode 100644 index 0000000000..d37b5eabe4 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Components/Alpha/HeifPlanarAlphaCompositor.cs @@ -0,0 +1,173 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Buffers; +using System.Numerics.Tensors; +using SixLabors.ImageSharp.Formats.Heif.Components; +using SixLabors.ImageSharp.Memory; +using SixLabors.ImageSharp.PixelFormats; +using SixLabors.ImageSharp.Processing.Processors.Transforms; + +namespace SixLabors.ImageSharp.Formats.Heif.Components.Alpha; + +/// +/// Composes a reconstructed HEIF luma plane directly into the alpha channel of a packed destination frame. +/// +internal static class HeifPlanarAlphaCompositor +{ + /// + /// Composes a native codec luma plane into a destination image region without materializing an intermediate image. + /// + /// The destination pixel type. + /// The codec adapter exposing the reconstructed component planes. + /// The native unsigned sample storage type. + /// The SIMD widening operations for the sample type. + /// The configuration used for pooled allocation and pixel conversion. + /// The native reconstructed component planes. + /// The packed destination frame receiving alpha values. + /// The resolved H.273 component-range parameters. + /// The visible luma rectangle within the reconstructed plane. + /// The complete presented size of the auxiliary image or grid tile. + /// The destination region receiving the top-left portion of the presented alpha image. + /// Whether stored color samples must be converted to unassociated alpha. + public static void Compose( + Configuration configuration, + TBuffer buffer, + ImageFrame destination, + in HeifColorConversionParameters parameters, + Rectangle sourceRectangle, + Size outputSize, + Rectangle destinationRectangle, + bool premultiplied) + where TPixel : unmanaged, IPixel + where TBuffer : struct, IHeifPlanarSampleBuffer + where TSample : unmanaged + where TLoader : struct, IHeifSampleConverter + { + int sourceWidth = sourceRectangle.Width; + int sourceHeight = sourceRectangle.Height; + int outputWidth = outputSize.Width; + int outputHeight = outputSize.Height; + int composedWidth = destinationRectangle.Width; + int composedHeight = destinationRectangle.Height; + + if (sourceWidth == outputWidth && sourceHeight == outputHeight) + { + using IMemoryOwner componentOwner = configuration.MemoryAllocator.Allocate(composedWidth); + using IMemoryOwner alphaOwner = configuration.MemoryAllocator.Allocate(composedWidth); + using IMemoryOwner colorOwner = configuration.MemoryAllocator.Allocate(composedWidth); + Span alpha = componentOwner.GetSpan()[..composedWidth]; + Span packedAlpha = alphaOwner.GetSpan()[..composedWidth]; + Span packedColor = colorOwner.GetSpan()[..composedWidth]; + + // The overwhelmingly common path reads the codec plane once and immediately packs the corresponding + // destination row. No resize maps or full-plane staging are required. + for (int y = 0; y < composedHeight; y++) + { + ReadOnlySpan source = buffer.GetLumaRowSpan(sourceRectangle.Y + y).Slice(sourceRectangle.X, composedWidth); + NormalizeAlphaRow(source, alpha, in parameters); + ApplyAlphaRow(configuration, destination, destinationRectangle.X, destinationRectangle.Y + y, alpha, packedAlpha, packedColor, premultiplied); + } + + return; + } + + // Alpha scaling must match KnownResamplers.Box. That public instance is exposed as IResampler, while + // ResizeKernelMap requires the concrete struct so Radius and GetValue remain statically dispatched. + // BoxResampler is stateless, making its default value behaviorally identical to the known instance. + BoxResampler boxResampler = default; + using ResizeKernelMap horizontalKernels = ResizeKernelMap.Calculate(in boxResampler, outputWidth, sourceWidth, configuration.MemoryAllocator); + using ResizeKernelMap verticalKernels = ResizeKernelMap.Calculate(in boxResampler, outputHeight, sourceHeight, configuration.MemoryAllocator); + using HeifPlanarAlphaResizeWorker worker = new( + configuration, + buffer, + destination, + in parameters, + sourceRectangle, + destinationRectangle, + horizontalKernels, + verticalKernels, + premultiplied); + + worker.Compose(); + } + + /// + /// Widens and normalizes one native luma row to unbounded alpha values before packing or resampling. + /// + /// The native unsigned sample storage type. + /// The SIMD widening operations for the sample type. + /// The native luma samples. + /// The normalized alpha samples. + /// The resolved H.273 component-range parameters. + public static void NormalizeAlphaRow( + ReadOnlySpan source, + Span destination, + in HeifColorConversionParameters parameters) + where TSample : unmanaged + where TLoader : struct, IHeifSampleConverter + { + HeifSampleConversion.ConvertSamplesToFloat(source, destination); + + // Alpha auxiliaries use the luma code-value range but no color matrix. TensorPrimitives keeps this bulk + // normalization SIMD-first on every supported architecture and clamps before resampling, matching the + // established conversion to a bounded L16 plane. + TensorPrimitives.Subtract(destination, parameters.LumaBias, destination); + TensorPrimitives.Multiply(destination, 1F / parameters.LumaScale, destination); + TensorPrimitives.Clamp(destination, 0F, 1F, destination); + } + + /// + /// Packs and composes one normalized alpha row into the destination frame. + /// + /// The destination pixel type. + /// The configuration used for pixel conversion. + /// The packed destination frame receiving alpha values. + /// The horizontal start of the destination region. + /// The destination row receiving alpha values. + /// The normalized alpha samples. + /// The reusable 16-bit alpha packing row. + /// The reusable high-bit-depth destination color row. + /// Whether stored color samples must be converted to unassociated alpha. + public static void ApplyAlphaRow( + Configuration configuration, + ImageFrame destination, + int destinationX, + int destinationY, + ReadOnlySpan alpha, + Span packedAlpha, + Span packedColor, + bool premultiplied) + where TPixel : unmanaged, IPixel + { + int width = alpha.Length; + Span destinationRow = destination.PixelBuffer.DangerousGetRowSpan(destinationY).Slice(destinationX, width); + PixelOperations pixelOperations = PixelOperations.Instance; + + HeifSampleConversion.PackL16(alpha, packedAlpha); + pixelOperations.ToRgba64(configuration, destinationRow, packedColor); + if (premultiplied) + { + for (int x = 0; x < width; x++) + { + Rgba64 pixel = packedColor[x]; + pixel.A = packedAlpha[x].PackedValue; + + // Transparent associated samples have no recoverable color. Nonzero samples use the pixel type's + // established conversion so unassociation retains ImageSharp's clamping and rounding behavior. + packedColor[x] = pixel.A == 0 + ? new Rgba64(0, 0, 0, 0) + : Rgba64.FromAssociatedScaledVector4(pixel.ToScaledVector4()); + } + } + else + { + for (int x = 0; x < width; x++) + { + packedColor[x].A = packedAlpha[x].PackedValue; + } + } + + pixelOperations.FromRgba64(configuration, packedColor, destinationRow); + } +} diff --git a/src/ImageSharp/Formats/Heif/Components/Alpha/HeifPlanarAlphaEncoder.cs b/src/ImageSharp/Formats/Heif/Components/Alpha/HeifPlanarAlphaEncoder.cs new file mode 100644 index 0000000000..f9df35ad95 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Components/Alpha/HeifPlanarAlphaEncoder.cs @@ -0,0 +1,227 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Buffers; +using System.Runtime.CompilerServices; +using System.Runtime.InteropServices; +using System.Runtime.Intrinsics; +using SixLabors.ImageSharp.Memory; +using SixLabors.ImageSharp.PixelFormats; + +namespace SixLabors.ImageSharp.Formats.Heif.Components.Alpha; + +/// +/// Converts packed ImageSharp alpha values into one native HEIF monochrome plane. +/// +internal static class HeifPlanarAlphaEncoder +{ + /// + /// The number of float elements sharing storage with one packed value. + /// + private const int Rgba64FloatElementCount = 2; + + /// + /// The additional float element receiving one extracted alpha value. + /// + private const int AlphaFloatElementCount = 1; + + /// + /// The complete reusable row-storage length per source pixel. + /// + private const int RowFloatElementCount = Rgba64FloatElementCount + AlphaFloatElementCount; + + /// + /// Gets the reusable row-storage length required for the specified source width. + /// + /// The source-row width. + /// The required number of float elements. + public static int GetRowStorageLength(int width) => width * RowFloatElementCount; + + /// + /// Converts one packed image frame into a full-range native alpha plane. + /// + /// The packed source pixel type. + /// The codec adapter exposing the destination plane. + /// The native unsigned sample storage type. + /// The SIMD narrowing and storage operations for the sample type. + /// The configuration used for row allocation and pixel conversion. + /// The packed source image frame. + /// The monochrome destination buffer. + public static void Convert( + Configuration configuration, + ImageFrame image, + TBuffer buffer) + where TPixel : unmanaged, IPixel + where TBuffer : struct, IHeifPlanarSampleBuffer + where TSample : unmanaged + where TStorer : struct, IHeifSampleConverter + { + Rectangle sourceRectangle = new(0, 0, image.Width, image.Height); + Convert(configuration, image, sourceRectangle, buffer); + } + + /// + /// Converts one packed image region into a full-range native alpha plane. + /// + /// The packed source pixel type. + /// The codec adapter exposing the destination plane. + /// The native unsigned sample storage type. + /// The SIMD narrowing and storage operations for the sample type. + /// The configuration used for row allocation and pixel conversion. + /// The packed source image frame. + /// The source region mapped to the complete destination plane. + /// The monochrome destination buffer. + public static void Convert( + Configuration configuration, + ImageFrame image, + Rectangle sourceRectangle, + TBuffer buffer) + where TPixel : unmanaged, IPixel + where TBuffer : struct, IHeifPlanarSampleBuffer + where TSample : unmanaged + where TStorer : struct, IHeifSampleConverter + { + int width = sourceRectangle.Width; + + // Rgba64 preserves the source pixel's normalized alpha precision before quantization to the requested AV1 + // depth. Both row views share one owner because their lifetimes never escape this conversion operation. + using IMemoryOwner rowOwner = configuration.MemoryAllocator.Allocate( + GetRowStorageLength(width)); + + Convert( + configuration, + image, + sourceRectangle, + buffer, + rowOwner.GetSpan()); + } + + /// + /// Converts one packed image region using caller-owned reusable row storage. + /// + /// The packed source pixel type. + /// The codec adapter exposing the destination plane. + /// The native unsigned sample storage type. + /// The SIMD narrowing and storage operations for the sample type. + /// The configuration used for pixel conversion. + /// The packed source image frame. + /// The source region mapped to the complete destination plane. + /// The monochrome destination buffer. + /// Storage for one packed high-precision row and its extracted alpha values. + public static void Convert( + Configuration configuration, + ImageFrame image, + Rectangle sourceRectangle, + TBuffer buffer, + Span rowStorage) + where TPixel : unmanaged, IPixel + where TBuffer : struct, IHeifPlanarSampleBuffer + where TSample : unmanaged + where TStorer : struct, IHeifSampleConverter + { + int width = sourceRectangle.Width; + Span packed = MemoryMarshal.Cast( + rowStorage[..(width * Rgba64FloatElementCount)]); + + Span alpha = rowStorage.Slice( + width * Rgba64FloatElementCount, + width * AlphaFloatElementCount); + + float maximum = (1 << buffer.LumaBitDepth) - 1; + float scale = maximum / ushort.MaxValue; + for (int y = 0; y < sourceRectangle.Height; y++) + { + ReadOnlySpan source = image.PixelBuffer + .DangerousGetRowSpan(sourceRectangle.Y + y) + .Slice(sourceRectangle.X, sourceRectangle.Width); + + PixelOperations.Instance.ToRgba64(configuration, source, packed); + ExtractAlpha(packed, alpha); + HeifSampleConversion.WriteSamples( + alpha, + buffer.GetLumaRowSpan(y), + scale, + 0F, + maximum); + } + } + + /// + /// Deinterleaves alpha values from one packed high-precision row. + /// + private static void ExtractAlpha(ReadOnlySpan source, Span destination) + { + ref Rgba64 sourceBase = ref MemoryMarshal.GetReference(source); + ref float destinationBase = ref MemoryMarshal.GetReference(destination); + int i = 0; + + // Packed RGBA requires a gather before conversion. Constructing vectors from the alpha fields keeps the + // widening and stores SIMD-wide without copying or transposing the complete packed row. + if (Vector512.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector512Count(destination.Length); + for (nuint vectorIndex = 0; vectorIndex < vectorCount; vectorIndex++) + { + int offset = (int)(vectorIndex * (uint)Vector512.Count); + Vector512 values = Vector512.Create( + CreateAlphaVector256(ref Unsafe.Add(ref sourceBase, offset)), + CreateAlphaVector256(ref Unsafe.Add(ref sourceBase, offset + Vector256.Count))); + + Vector512.ConvertToSingle(values).StoreUnsafe( + ref destinationBase, + (nuint)offset); + } + + i = (int)(vectorCount * (uint)Vector512.Count); + } + + if (Vector256.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector256Count(destination.Length - i); + for (nuint vectorIndex = 0; vectorIndex < vectorCount; vectorIndex++) + { + int offset = i + (int)(vectorIndex * (uint)Vector256.Count); + Vector256 values = CreateAlphaVector256(ref Unsafe.Add(ref sourceBase, offset)); + Vector256.ConvertToSingle(values).StoreUnsafe( + ref destinationBase, + (nuint)offset); + } + + i += (int)(vectorCount * (uint)Vector256.Count); + } + + if (Vector128.IsHardwareAccelerated) + { + nuint vectorCount = Numerics.Vector128Count(destination.Length - i); + for (nuint vectorIndex = 0; vectorIndex < vectorCount; vectorIndex++) + { + int offset = i + (int)(vectorIndex * (uint)Vector128.Count); + Vector128 values = CreateAlphaVector128(ref Unsafe.Add(ref sourceBase, offset)); + Vector128.ConvertToSingle(values).StoreUnsafe( + ref destinationBase, + (nuint)offset); + } + + i += (int)(vectorCount * (uint)Vector128.Count); + } + + for (; i < destination.Length; i++) + { + Unsafe.Add(ref destinationBase, i) = Unsafe.Add(ref sourceBase, i).A; + } + } + + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector128 CreateAlphaVector128(ref Rgba64 source) + => Vector128.Create( + (uint)source.A, + Unsafe.Add(ref source, 1).A, + Unsafe.Add(ref source, 2).A, + Unsafe.Add(ref source, 3).A); + + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector256 CreateAlphaVector256(ref Rgba64 source) + => Vector256.Create( + CreateAlphaVector128(ref source), + CreateAlphaVector128(ref Unsafe.Add(ref source, Vector128.Count))); +} diff --git a/src/ImageSharp/Formats/Heif/Components/Alpha/HeifPlanarAlphaResizeWorker.cs b/src/ImageSharp/Formats/Heif/Components/Alpha/HeifPlanarAlphaResizeWorker.cs new file mode 100644 index 0000000000..4b54240927 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Components/Alpha/HeifPlanarAlphaResizeWorker.cs @@ -0,0 +1,323 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Buffers; +using System.Numerics; +using System.Runtime.CompilerServices; +using System.Runtime.InteropServices; +using SixLabors.ImageSharp.Memory; +using SixLabors.ImageSharp.PixelFormats; +using SixLabors.ImageSharp.Processing.Processors.Transforms; + +namespace SixLabors.ImageSharp.Formats.Heif.Components.Alpha; + +/// +/// Resizes a native HEIF luma plane and composes the result as alpha using a bounded sliding window. +/// +/// The destination pixel type. +/// The codec adapter exposing the reconstructed component planes. +/// The native unsigned sample storage type. +/// The SIMD widening operations for the sample type. +internal sealed class HeifPlanarAlphaResizeWorker : IDisposable + where TPixel : unmanaged, IPixel + where TBuffer : struct, IHeifPlanarSampleBuffer + where TSample : unmanaged + where TLoader : struct, IHeifSampleConverter +{ + /// + /// The configuration used for pooled allocation and pixel conversion. + /// + private readonly Configuration configuration; + + /// + /// The codec-native component planes. + /// + private readonly TBuffer buffer; + + /// + /// The packed color frame receiving alpha values. + /// + private readonly ImageFrame destination; + + /// + /// The resolved H.273 component-range parameters. + /// + private readonly HeifColorConversionParameters parameters; + + /// + /// The visible luma rectangle within the reconstructed plane. + /// + private readonly Rectangle sourceRectangle; + + /// + /// The destination region receiving the resized alpha plane. + /// + private readonly Rectangle destinationRectangle; + + /// + /// The horizontal box-filter kernels for the full presented width. + /// + private readonly ResizeKernelMap horizontalKernels; + + /// + /// The vertical box-filter kernels for the full presented height. + /// + private readonly ResizeKernelMap verticalKernels; + + /// + /// The transposed horizontally filtered rows retained by the sliding window. + /// + private readonly Buffer2D transposedFirstPassBuffer; + + /// + /// The reusable normalized source or resized destination row. + /// + private readonly IMemoryOwner componentOwner; + + /// + /// The reusable replicated source row consumed by the shared resize kernels. + /// + private readonly IMemoryOwner sourceVectorOwner; + + /// + /// The reusable 16-bit source and destination alpha packing row. + /// + private readonly IMemoryOwner alphaOwner; + + /// + /// The reusable high-bit-depth destination color row. + /// + private readonly IMemoryOwner colorOwner; + + /// + /// Whether stored color samples must be converted to unassociated alpha. + /// + private readonly bool premultiplied; + + /// + /// The number of source rows retained when the window advances. + /// + private readonly int windowBandHeight; + + /// + /// The total number of source rows retained by the bounded working window. + /// + private readonly int workerHeight; + + /// + /// The source-row interval currently represented by the transposed first-pass buffer. + /// + private RowInterval currentWindow; + + /// + /// Initializes a new instance of the class. + /// + /// The configuration used for pooled allocation and pixel conversion. + /// The codec-native component planes. + /// The packed color frame receiving alpha values. + /// The resolved H.273 component-range parameters. + /// The visible luma rectangle within the reconstructed plane. + /// The destination region receiving the top-left portion of the presented alpha plane. + /// The horizontal box-filter kernels for the full presented width. + /// The vertical box-filter kernels for the full presented height. + /// Whether stored color samples must be converted to unassociated alpha. + public HeifPlanarAlphaResizeWorker( + Configuration configuration, + TBuffer buffer, + ImageFrame destination, + in HeifColorConversionParameters parameters, + Rectangle sourceRectangle, + Rectangle destinationRectangle, + ResizeKernelMap horizontalKernels, + ResizeKernelMap verticalKernels, + bool premultiplied) + { + this.configuration = configuration; + this.buffer = buffer; + this.destination = destination; + this.parameters = parameters; + this.sourceRectangle = sourceRectangle; + this.destinationRectangle = destinationRectangle; + this.premultiplied = premultiplied; + + this.horizontalKernels = horizontalKernels; + this.verticalKernels = verticalKernels; + + // Retaining one complete maximum-diameter band is sufficient for every vertical kernel that crosses a + // window boundary. Those first-pass rows can be copied forward instead of normalized and filtered again. + this.windowBandHeight = this.verticalKernels.MaxDiameter; + + // As in ResizeWorker, the first pass is stored transposed as [destination X][source Y]. Bounding the source-Y + // dimension by the configured working-buffer limit keeps memory independent of the complete alpha-plane size. + int workingBufferLimitInBytes = Math.Min( + configuration.WorkingBufferSizeHintInBytes, + configuration.MemoryAllocator.GetBufferCapacityInBytes()); + + int windowBandCount = ResizeHelper.CalculateResizeWorkerHeightInWindowBands( + this.windowBandHeight, + destinationRectangle.Width, + workingBufferLimitInBytes); + + // A whole number of bands lets Slide retain exactly one overlap band and fill the remaining window with rows + // that have not entered the first pass before. + this.workerHeight = Math.Min(sourceRectangle.Height, windowBandCount * this.windowBandHeight); + this.transposedFirstPassBuffer = configuration.MemoryAllocator.Allocate2D( + this.workerHeight, + destinationRectangle.Width, + preferContiguosImageBuffers: true, + options: AllocationOptions.Clean); + + this.componentOwner = configuration.MemoryAllocator.Allocate(Math.Max(sourceRectangle.Width, destinationRectangle.Width)); + this.sourceVectorOwner = configuration.MemoryAllocator.Allocate(sourceRectangle.Width); + this.alphaOwner = configuration.MemoryAllocator.Allocate(Math.Max(sourceRectangle.Width, destinationRectangle.Width)); + this.colorOwner = configuration.MemoryAllocator.Allocate(destinationRectangle.Width); + this.currentWindow = new RowInterval(0, this.workerHeight); + } + + /// + /// Releases all allocator-owned working buffers. + /// + public void Dispose() + { + this.transposedFirstPassBuffer.Dispose(); + this.componentOwner.Dispose(); + this.sourceVectorOwner.Dispose(); + this.alphaOwner.Dispose(); + this.colorOwner.Dispose(); + } + + /// + /// Resizes and composes the complete requested destination rectangle. + /// + public void Compose() + { + // Populate the horizontal first pass for the initial bounded source-row interval. Later windows retain their + // overlap and calculate only newly entering rows. + this.CalculateFirstPassValues(this.currentWindow); + + Span transposed = this.transposedFirstPassBuffer.DangerousGetSingleSpan(); + Span resizedAlpha = this.componentOwner.GetSpan()[..this.destinationRectangle.Width]; + Span packedAlpha = this.alphaOwner.GetSpan()[..this.destinationRectangle.Width]; + Span packedColor = this.colorOwner.GetSpan()[..this.destinationRectangle.Width]; + ReadOnlySpan verticalKernelSpan = this.verticalKernels.GetKernelSpan(); + ref ResizeKernel verticalKernelBase = ref MemoryMarshal.GetReference(verticalKernelSpan); + ref float resizedAlphaBase = ref MemoryMarshal.GetReference(resizedAlpha); + int currentWindowMin = this.currentWindow.Min; + int currentWindowMax = this.currentWindow.Max; + nuint width = (uint)this.destinationRectangle.Width; + nuint workerHeight = (uint)this.workerHeight; + nuint twoWorkerHeights = workerHeight * 2; + + for (int y = 0; y < this.destinationRectangle.Height; y++) + { + ref ResizeKernel kernel = ref Unsafe.Add(ref verticalKernelBase, y); + int kernelEnd = kernel.StartIndex + kernel.Length; + + // Destination kernels advance monotonically through source Y. Slide until the complete kernel lies in + // the cached first-pass interval; the retained overlap prevents any shared source row being recalculated. + while (kernelEnd > currentWindowMax) + { + this.Slide(); + currentWindowMin = this.currentWindow.Min; + currentWindowMax = this.currentWindow.Max; + } + + // Values for one destination X are contiguous along source Y in the transposed buffer. ConvolveCore + // therefore reads the vertical kernel without gathers, while workerHeight advances to the next X column. + ref Vector4 column = ref transposed[kernel.StartIndex - currentWindowMin]; + nuint x = 0; + for (; x + 1 < width; x += 2) + { + Unsafe.Add(ref resizedAlphaBase, x) = kernel.ConvolveCore(ref column).X; + ref Vector4 nextColumn = ref Unsafe.Add(ref column, workerHeight); + Unsafe.Add(ref resizedAlphaBase, x + 1) = kernel.ConvolveCore(ref nextColumn).X; + column = ref Unsafe.Add(ref column, twoWorkerHeights); + } + + if (x < width) + { + Unsafe.Add(ref resizedAlphaBase, x) = kernel.ConvolveCore(ref column).X; + } + + HeifPlanarAlphaCompositor.ApplyAlphaRow( + this.configuration, + this.destination, + this.destinationRectangle.X, + this.destinationRectangle.Y + y, + resizedAlpha, + packedAlpha, + packedColor, + this.premultiplied); + } + } + + /// + /// Advances the bounded working window while preserving its overlapping source-row band. + /// + private void Slide() + { + // The old bottom band is the only set of first-pass rows that a future kernel can share with the new window. + // Its height equals the largest vertical-kernel diameter, covering the maximum possible overlap. + int minimumY = this.currentWindow.Max - this.windowBandHeight; + int maximumY = Math.Min(minimumY + this.workerHeight, this.sourceRectangle.Height); + + // Buffer2D columns represent source Y because the first pass is transposed. Move the retained bottom band to + // offset zero for every destination-X column before replacing the remainder of the window. + this.transposedFirstPassBuffer.DangerousCopyColumns( + this.workerHeight - this.windowBandHeight, + 0, + this.windowBandHeight); + + this.currentWindow = new RowInterval(minimumY, maximumY); + + // The retained band already contains normalized and horizontally filtered values. Only rows below it are new. + this.CalculateFirstPassValues(this.currentWindow.Slice(this.windowBandHeight)); + } + + /// + /// Normalizes and horizontally filters the source rows entering the current working window. + /// + /// The source-row interval requiring first-pass values. + private void CalculateFirstPassValues(RowInterval interval) + { + int sourceWidth = this.sourceRectangle.Width; + int destinationWidth = this.destinationRectangle.Width; + Span normalized = this.componentOwner.GetSpan()[..sourceWidth]; + Span sourceAlpha = this.alphaOwner.GetSpan()[..sourceWidth]; + Span sourceVectors = this.sourceVectorOwner.GetSpan()[..sourceWidth]; + Span transposed = this.transposedFirstPassBuffer.DangerousGetSingleSpan(); + ReadOnlySpan horizontalKernelSpan = this.horizontalKernels.GetKernelSpan(); + ref ResizeKernel horizontalKernelBase = ref MemoryMarshal.GetReference(horizontalKernelSpan); + nuint workerHeight = (uint)this.workerHeight; + + for (int y = interval.Min; y < interval.Max; y++) + { + ReadOnlySpan source = this.buffer.GetLumaRowSpan(this.sourceRectangle.Y + y).Slice(this.sourceRectangle.X, sourceWidth); + HeifPlanarAlphaCompositor.NormalizeAlphaRow(source, normalized, in this.parameters); + + // ResizeKernel is the same SIMD convolution primitive used by the general image resizer. Replicating alpha + // into Vector4 lets that kernel operate on the planar row, while the L16 round trip preserves the result of + // the removed Image path without materializing the complete alpha image. + HeifSampleConversion.PackL16(normalized, sourceAlpha); + PixelOperations.Instance.ToVector4(this.configuration, sourceAlpha, sourceVectors, PixelConversionModifiers.Scale); + + // The source row is horizontally filtered once for every destination X and stored at [X][window Y]. A + // vertical kernel can then reuse this first-pass row wherever adjacent destination kernels overlap it. + ref Vector4 firstPass = ref transposed[y - this.currentWindow.Min]; + int x = 0; + for (; x + 1 < destinationWidth; x += 2) + { + ref ResizeKernel kernel0 = ref Unsafe.Add(ref horizontalKernelBase, x); + ref ResizeKernel kernel1 = ref Unsafe.Add(ref horizontalKernelBase, x + 1); + Unsafe.Add(ref firstPass, (nuint)x * workerHeight) = kernel0.Convolve(sourceVectors); + Unsafe.Add(ref firstPass, (nuint)(x + 1) * workerHeight) = kernel1.Convolve(sourceVectors); + } + + if (x < destinationWidth) + { + ref ResizeKernel kernel = ref Unsafe.Add(ref horizontalKernelBase, x); + Unsafe.Add(ref firstPass, (nuint)x * workerHeight) = kernel.Convolve(sourceVectors); + } + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Components/Alpha/IHeifAlphaItemDecoder.cs b/src/ImageSharp/Formats/Heif/Components/Alpha/IHeifAlphaItemDecoder.cs new file mode 100644 index 0000000000..bb4e1cf874 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Components/Alpha/IHeifAlphaItemDecoder.cs @@ -0,0 +1,35 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.PixelFormats; + +namespace SixLabors.ImageSharp.Formats.Heif.Components.Alpha; + +/// +/// Decodes one coded HEIF auxiliary alpha item directly into a packed color frame. +/// +/// The destination color pixel type. +internal interface IHeifAlphaItemDecoder + where TPixel : unmanaged, IPixel +{ + /// + /// Decodes and composes one coded auxiliary alpha item. + /// + /// The general options governing the containing HEIF decode. + /// The auxiliary image item whose encoded payload is being decoded. + /// The encoded auxiliary payload. + /// The packed color frame receiving alpha values. + /// The complete presented size of the auxiliary image or grid tile. + /// The destination region receiving the top-left portion of the presented alpha image. + /// Whether stored color samples must be converted to unassociated alpha. + /// The token used to cancel the payload decode. + public void DecodeAlphaItemData( + DecoderOptions options, + HeifItem item, + Span data, + ImageFrame destination, + Size outputSize, + Rectangle destinationRectangle, + bool premultiplied, + CancellationToken cancellationToken); +} diff --git a/src/ImageSharp/Formats/Heif/Components/ColorConverters/HeifColorConversionParameters.cs b/src/ImageSharp/Formats/Heif/Components/ColorConverters/HeifColorConversionParameters.cs new file mode 100644 index 0000000000..9432facd59 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Components/ColorConverters/HeifColorConversionParameters.cs @@ -0,0 +1,528 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Metadata.Profiles.Cicp; + +namespace SixLabors.ImageSharp.Formats.Heif.Components; + +/// +/// Stores the resolved H.273 values shared by every scalar and SIMD lane. +/// +internal readonly struct HeifColorConversionParameters +{ + /// + /// Initializes a new instance of the struct. + /// + /// The red luma coefficient. + /// The green luma coefficient. + /// The blue luma coefficient. + /// The signaled H.273 matrix-coefficient code point. + /// Whether encoded components use their complete numeric range. + /// The signaled transfer characteristics. + /// The constant-luminance chroma scales. + /// The encoded luma bias. + /// The encoded luma range. + /// The encoded chroma midpoint. + /// The encoded chroma range. + /// The largest encoded luma sample value. + /// The largest encoded chroma sample value. + /// The reversible transform's RGB code-value bias. + /// The reversible transform's RGB code-value range. + /// The largest reversible transform RGB code value. + public HeifColorConversionParameters( + float kr, + float kg, + float kb, + CicpMatrixCoefficients matrixCoefficients, + bool isFullRange, + CicpTransferCharacteristics transferCharacteristics, + in HeifConstantLuminanceScales constantLuminanceScales, + float lumaBias, + float lumaScale, + float chromaBias, + float chromaScale, + float lumaSampleMaximum, + float chromaSampleMaximum, + float rgbBias, + float rgbScale, + float rgbSampleMaximum) + { + this.Kr = kr; + this.Kg = kg; + this.Kb = kb; + this.RedChromaScale = 2F * (1F - kr); + this.BlueChromaScale = 2F * (1F - kb); + this.GreenRedChromaNumerator = kr * (1F - kr); + this.GreenBlueChromaNumerator = kb * (1F - kb); + this.GreenRedChromaScale = 2F * this.GreenRedChromaNumerator / kg; + this.GreenBlueChromaScale = 2F * this.GreenBlueChromaNumerator / kg; + this.MatrixCoefficients = matrixCoefficients; + this.IsFullRange = isFullRange; + this.TransferCharacteristics = transferCharacteristics; + this.ConstantLuminanceScales = constantLuminanceScales; + this.LumaBias = lumaBias; + this.LumaScale = lumaScale; + this.ChromaBias = chromaBias; + this.ChromaScale = chromaScale; + this.LumaSampleMaximum = lumaSampleMaximum; + this.ChromaSampleMaximum = chromaSampleMaximum; + this.RgbBias = rgbBias; + this.RgbScale = rgbScale; + this.RgbSampleMaximum = rgbSampleMaximum; + } + + /// + /// Gets the red luma coefficient. + /// + public float Kr { get; } + + /// + /// Gets the green luma coefficient. + /// + public float Kg { get; } + + /// + /// Gets the blue luma coefficient. + /// + public float Kb { get; } + + /// + /// Gets the red contribution from the red-difference component. + /// + public float RedChromaScale { get; } + + /// + /// Gets the blue contribution from the blue-difference component. + /// + public float BlueChromaScale { get; } + + /// + /// Gets the red-difference term in the green-component numerator. + /// + public float GreenRedChromaNumerator { get; } + + /// + /// Gets the blue-difference term in the green-component numerator. + /// + public float GreenBlueChromaNumerator { get; } + + /// + /// Gets the red-difference subtraction from green. + /// + public float GreenRedChromaScale { get; } + + /// + /// Gets the blue-difference subtraction from green. + /// + public float GreenBlueChromaScale { get; } + + /// + /// Gets the signaled H.273 matrix-coefficient code point. + /// + public CicpMatrixCoefficients MatrixCoefficients { get; } + + /// + /// Gets a value indicating whether encoded components use their complete numeric range. + /// + public bool IsFullRange { get; } + + /// + /// Gets the signaled transfer characteristics. + /// + public CicpTransferCharacteristics TransferCharacteristics { get; } + + /// + /// Gets the constant-luminance chroma scales. + /// + public HeifConstantLuminanceScales ConstantLuminanceScales { get; } + + /// + /// Gets the encoded luma bias. + /// + public float LumaBias { get; } + + /// + /// Gets the encoded luma range. + /// + public float LumaScale { get; } + + /// + /// Gets the encoded chroma midpoint. + /// + public float ChromaBias { get; } + + /// + /// Gets the encoded chroma range. + /// + public float ChromaScale { get; } + + /// + /// Gets the largest encoded luma sample value. + /// + public float LumaSampleMaximum { get; } + + /// + /// Gets the largest encoded chroma sample value. + /// + public float ChromaSampleMaximum { get; } + + /// + /// Gets the reversible transform's RGB code-value bias. + /// + public float RgbBias { get; } + + /// + /// Gets the reversible transform's RGB code-value range. + /// + public float RgbScale { get; } + + /// + /// Gets the largest reversible transform RGB code value. + /// + public float RgbSampleMaximum { get; } + + /// + /// Resolves the H.273 matrix coefficients and sample ranges used by AV1 image items. + /// + /// The H.273 color-primary code point. + /// The H.273 transfer-characteristic code point. + /// The H.273 matrix-coefficient code point. + /// Whether encoded components use their complete numeric range. + /// The encoded luma precision in bits. + /// The encoded chroma precision in bits. + /// Whether the image contains only luma samples. + /// Whether both chroma planes have luma resolution. + /// The resolved conversion operation. + /// The immutable values used by each scalar and SIMD conversion lane. + public static HeifColorConversionParameters Create( + CicpColorPrimaries colorPrimaries, + CicpTransferCharacteristics transferCharacteristics, + CicpMatrixCoefficients matrixCoefficients, + bool isFullRange, + int lumaBitDepth, + int chromaBitDepth, + bool isMonochrome, + bool hasFullResolutionChroma, + out HeifColorConversionMode mode) + { + mode = HeifColorConversionMode.Coefficients; + float kr = 0F; + float kb = 0F; + + // H.273 assigns one closed arithmetic operation to each non-reserved matrix code point. Resolving the + // operation once keeps both codec wrappers and every vector lane free from per-sample format dispatch. + switch (matrixCoefficients) + { + case CicpMatrixCoefficients.Identity: + mode = HeifColorConversionMode.Identity; + break; + case CicpMatrixCoefficients.ItuRBt709_6: + kr = 0.2126F; + kb = 0.0722F; + break; + case CicpMatrixCoefficients.Fcc47: + kr = 0.30F; + kb = 0.11F; + break; + case CicpMatrixCoefficients.ItuRBt601_7_625: + case CicpMatrixCoefficients.ItuRBt601_7_525: + case CicpMatrixCoefficients.Unspecified: + kr = 0.299F; + kb = 0.114F; + break; + case CicpMatrixCoefficients.SmpteSt240: + kr = 0.212F; + kb = 0.087F; + break; + case CicpMatrixCoefficients.YCgCo: + mode = HeifColorConversionMode.YCgCo; + break; + case CicpMatrixCoefficients.ItuRBt2020_2_Ncl: + kr = 0.2627F; + kb = 0.0593F; + break; + case CicpMatrixCoefficients.ItuRBt2020_2_Cl: + mode = HeifColorConversionMode.ConstantLuminance; + kr = 0.2627F; + kb = 0.0593F; + break; + case CicpMatrixCoefficients.SmpteSt2085: + mode = HeifColorConversionMode.Smpte2085; + break; + case CicpMatrixCoefficients.ChromaDerivedNcl: + GetChromaticityDerivedCoefficients(colorPrimaries, out kr, out kb); + break; + case CicpMatrixCoefficients.ChromaDerivedCl: + mode = HeifColorConversionMode.ConstantLuminance; + GetChromaticityDerivedCoefficients(colorPrimaries, out kr, out kb); + break; + case CicpMatrixCoefficients.ICtCp: + mode = HeifColorConversionMode.ICtCp; + break; + case CicpMatrixCoefficients.IptC2: + mode = HeifColorConversionMode.IptC2; + break; + case CicpMatrixCoefficients.YCgCoRe: + case CicpMatrixCoefficients.YCgCoRo: + mode = HeifColorConversionMode.YCgCoReversible; + break; + default: + throw new InvalidImageContentException($"The image declares reserved H.273 matrix coefficients '{(byte)matrixCoefficients}'."); + } + + bool requiresFullResolutionChroma = mode is HeifColorConversionMode.Identity or HeifColorConversionMode.YCgCoReversible; + if (requiresFullResolutionChroma && !isMonochrome && !hasFullResolutionChroma) + { + throw new InvalidImageContentException($"H.273 matrix coefficients '{matrixCoefficients}' require 4:4:4 sampling."); + } + + bool requiresEqualBitDepth = mode is HeifColorConversionMode.Identity or HeifColorConversionMode.YCgCoReversible; + if (requiresEqualBitDepth && !isMonochrome && lumaBitDepth != chromaBitDepth) + { + throw new InvalidImageContentException($"H.273 matrix coefficients '{matrixCoefficients}' require equal component bit depths."); + } + + float kg = 1F - kr - kb; + int lumaDepthScale = 1 << (lumaBitDepth - 8); + int chromaDepthScale = 1 << (chromaBitDepth - 8); + float lumaSampleMaximum = (1 << lumaBitDepth) - 1; + float chromaSampleMaximum = (1 << chromaBitDepth) - 1; + float chromaBias = 128F * chromaDepthScale; + float lumaBias = isFullRange ? 0F : 16F * lumaDepthScale; + float lumaScale = isFullRange ? lumaSampleMaximum : 219F * lumaDepthScale; + + // Limited-range YCgCo first range-adjusts RGB through the luma range, so its difference components use + // 219 codes. Conventional YCbCr uses the independently specified 224-code chroma excursion. + float chromaScale = isFullRange + ? chromaSampleMaximum + : (mode == HeifColorConversionMode.YCgCo ? 219F : 224F) * chromaDepthScale; + + float rgbBias = 0F; + float rgbScale = 1F; + float rgbSampleMaximum = 1F; + if (mode == HeifColorConversionMode.YCgCoReversible) + { + int bitOffset = matrixCoefficients == CicpMatrixCoefficients.YCgCoRe ? 2 : 1; + int rgbBitDepth = lumaBitDepth - bitOffset; + float rgbDepthScale = MathF.ScaleB(1F, rgbBitDepth - 8); + rgbSampleMaximum = (1 << rgbBitDepth) - 1; + rgbBias = isFullRange ? 0F : 16F * rgbDepthScale; + rgbScale = isFullRange ? rgbSampleMaximum : 219F * rgbDepthScale; + + // The reversible lifting transform works on integer code values. It performs RGB range adjustment + // internally, while the row traversal normalizes all three encoded components to their full code range. + lumaBias = 0F; + lumaScale = lumaSampleMaximum; + chromaScale = chromaSampleMaximum; + } + + HeifConstantLuminanceScales constantLuminanceScales = mode == HeifColorConversionMode.ConstantLuminance + ? new HeifConstantLuminanceScales(transferCharacteristics, kr, kb) + : default; + + return new HeifColorConversionParameters( + kr, + kg, + kb, + matrixCoefficients, + isFullRange, + transferCharacteristics, + in constantLuminanceScales, + lumaBias, + lumaScale, + chromaBias, + chromaScale, + lumaSampleMaximum, + chromaSampleMaximum, + rgbBias, + rgbScale, + rgbSampleMaximum); + } + + /// + /// Computes the luma coefficients defined by an H.273 primary-chromaticity code point. + /// + /// The H.273 color-primary code point. + /// The resulting red luma coefficient. + /// The resulting blue luma coefficient. + private static void GetChromaticityDerivedCoefficients(CicpColorPrimaries colorPrimaries, out float kr, out float kb) + { + float redX; + float redY; + float greenX; + float greenY; + float blueX; + float blueY; + float whiteX; + float whiteY; + + switch (colorPrimaries) + { + case CicpColorPrimaries.ItuRBt470_6M: + redX = 0.67F; + redY = 0.33F; + greenX = 0.21F; + greenY = 0.71F; + blueX = 0.14F; + blueY = 0.08F; + whiteX = 0.310F; + whiteY = 0.316F; + break; + case CicpColorPrimaries.ItuRBt601_7_625: + redX = 0.64F; + redY = 0.33F; + greenX = 0.29F; + greenY = 0.60F; + blueX = 0.15F; + blueY = 0.06F; + whiteX = 0.3127F; + whiteY = 0.3290F; + break; + case CicpColorPrimaries.ItuRBt601_7_525: + case CicpColorPrimaries.SmpteSt240: + redX = 0.630F; + redY = 0.340F; + greenX = 0.310F; + greenY = 0.595F; + blueX = 0.155F; + blueY = 0.070F; + whiteX = 0.3127F; + whiteY = 0.3290F; + break; + case CicpColorPrimaries.GenericFilm: + redX = 0.681F; + redY = 0.319F; + greenX = 0.243F; + greenY = 0.692F; + blueX = 0.145F; + blueY = 0.049F; + whiteX = 0.310F; + whiteY = 0.316F; + break; + case CicpColorPrimaries.ItuRBt2020_2: + redX = 0.708F; + redY = 0.292F; + greenX = 0.170F; + greenY = 0.797F; + blueX = 0.131F; + blueY = 0.046F; + whiteX = 0.3127F; + whiteY = 0.3290F; + break; + case CicpColorPrimaries.SmpteSt428_1: + redX = 1F; + redY = 0F; + greenX = 0F; + greenY = 1F; + blueX = 0F; + blueY = 0F; + whiteX = 1F / 3F; + whiteY = 1F / 3F; + break; + case CicpColorPrimaries.SmpteRp431_2: + redX = 0.680F; + redY = 0.320F; + greenX = 0.265F; + greenY = 0.690F; + blueX = 0.150F; + blueY = 0.060F; + whiteX = 0.314F; + whiteY = 0.351F; + break; + case CicpColorPrimaries.SmpteEg432_1: + redX = 0.680F; + redY = 0.320F; + greenX = 0.265F; + greenY = 0.690F; + blueX = 0.150F; + blueY = 0.060F; + whiteX = 0.3127F; + whiteY = 0.3290F; + break; + case CicpColorPrimaries.EbuTech3213E: + redX = 0.630F; + redY = 0.340F; + greenX = 0.295F; + greenY = 0.605F; + blueX = 0.155F; + blueY = 0.077F; + whiteX = 0.3127F; + whiteY = 0.3290F; + break; + default: + // Unspecified primaries cannot define a chromaticity-derived matrix. The established libavif + // behavior supplies BT.709/D65 so the image has one deterministic interpretation. + redX = 0.64F; + redY = 0.33F; + greenX = 0.30F; + greenY = 0.60F; + blueX = 0.15F; + blueY = 0.06F; + whiteX = 0.3127F; + whiteY = 0.3290F; + break; + } + + float redZ = 1F - (redX + redY); + float greenZ = 1F - (greenX + greenY); + float blueZ = 1F - (blueX + blueY); + float whiteZ = 1F - (whiteX + whiteY); + + // H.273 equations 39 and 40 solve the RGB-to-XYZ primary matrix at the signaled white point. Expanding the + // determinant avoids a general matrix inversion and gives both codec paths the same rounding sequence. + float denominator = whiteY * + ((redX * ((greenY * blueZ) - (blueY * greenZ))) + + (greenX * ((blueY * redZ) - (redY * blueZ))) + + (blueX * ((redY * greenZ) - (greenY * redZ)))); + + kr = (redY * + ((whiteX * ((greenY * blueZ) - (blueY * greenZ))) + + (whiteY * ((blueX * greenZ) - (greenX * blueZ))) + + (whiteZ * ((greenX * blueY) - (blueX * greenY))))) / denominator; + + kb = (blueY * + ((whiteX * ((redY * greenZ) - (greenY * redZ))) + + (whiteY * ((greenX * redZ) - (redX * greenZ))) + + (whiteZ * ((redX * greenY) - (greenX * redY))))) / denominator; + } +} + +/// +/// Stores the H.273 chroma normalization constants for constant-luminance conversion. +/// +internal readonly struct HeifConstantLuminanceScales +{ + /// + /// Initializes a new instance of the struct. + /// + /// The signaled transfer characteristics. + /// The red luma coefficient. + /// The blue luma coefficient. + public HeifConstantLuminanceScales(CicpTransferCharacteristics transferCharacteristics, float kr, float kb) + { + this.NegativeBlue = HeifTransferFunctions.ToGamma(transferCharacteristics, 1F - kb); + this.PositiveBlue = 1F - HeifTransferFunctions.ToGamma(transferCharacteristics, kb); + this.NegativeRed = HeifTransferFunctions.ToGamma(transferCharacteristics, 1F - kr); + this.PositiveRed = 1F - HeifTransferFunctions.ToGamma(transferCharacteristics, kr); + } + + /// + /// Gets the negative blue-difference scale. + /// + public float NegativeBlue { get; } + + /// + /// Gets the positive blue-difference scale. + /// + public float PositiveBlue { get; } + + /// + /// Gets the negative red-difference scale. + /// + public float NegativeRed { get; } + + /// + /// Gets the positive red-difference scale. + /// + public float PositiveRed { get; } +} diff --git a/src/ImageSharp/Formats/Heif/Components/ColorConverters/HeifColorConverter.CoefficientOperator.cs b/src/ImageSharp/Formats/Heif/Components/ColorConverters/HeifColorConverter.CoefficientOperator.cs new file mode 100644 index 0000000000..60dda9f47c --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Components/ColorConverters/HeifColorConverter.CoefficientOperator.cs @@ -0,0 +1,158 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Components; + +/// +/// Provides coefficient-matrix YCbCr conversion for scalar and SIMD lanes. +/// +internal abstract partial class HeifColorConverterBase +{ + /// + /// Implements coefficient-based YCbCr conversion for scalar and SIMD lanes. + /// + internal readonly struct HeifCoefficientColorOperator : IHeifColorOperator + { + /// + public static bool ChromaUsesLumaRange => false; + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void ConvertToRgb(ref float y, ref float cb, ref float cr, in HeifColorConversionParameters parameters) + { + // Preserve the H.273 operation order. Pre-dividing the two green contributions changes rounding at + // exact output-code boundaries for high-bit-depth images. + float r = y + (parameters.RedChromaScale * cr); + float g = y - ((2F * ((parameters.GreenRedChromaNumerator * cr) + + (parameters.GreenBlueChromaNumerator * cb))) / parameters.Kg); + + float b = y + (parameters.BlueChromaScale * cb); + + y = r; + cb = g; + cr = b; + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void ConvertToRgb(ref Vector128 y, ref Vector128 cb, ref Vector128 cr, in HeifColorConversionParameters parameters) + { + Vector128 r = y + (Vector128.Create(parameters.RedChromaScale) * cr); + Vector128 greenRed = Vector128.Create(parameters.GreenRedChromaNumerator) * cr; + Vector128 greenBlue = Vector128.Create(parameters.GreenBlueChromaNumerator) * cb; + Vector128 g = y - ((Vector128.Create(2F) * (greenRed + greenBlue)) / Vector128.Create(parameters.Kg)); + Vector128 b = y + (Vector128.Create(parameters.BlueChromaScale) * cb); + + y = r; + cb = g; + cr = b; + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void ConvertToRgb(ref Vector256 y, ref Vector256 cb, ref Vector256 cr, in HeifColorConversionParameters parameters) + { + Vector256 r = y + (Vector256.Create(parameters.RedChromaScale) * cr); + Vector256 greenRed = Vector256.Create(parameters.GreenRedChromaNumerator) * cr; + Vector256 greenBlue = Vector256.Create(parameters.GreenBlueChromaNumerator) * cb; + Vector256 g = y - ((Vector256.Create(2F) * (greenRed + greenBlue)) / Vector256.Create(parameters.Kg)); + Vector256 b = y + (Vector256.Create(parameters.BlueChromaScale) * cb); + + y = r; + cb = g; + cr = b; + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void ConvertToRgb(ref Vector512 y, ref Vector512 cb, ref Vector512 cr, in HeifColorConversionParameters parameters) + { + Vector512 r = y + (Vector512.Create(parameters.RedChromaScale) * cr); + Vector512 greenRed = Vector512.Create(parameters.GreenRedChromaNumerator) * cr; + Vector512 greenBlue = Vector512.Create(parameters.GreenBlueChromaNumerator) * cb; + Vector512 g = y - ((Vector512.Create(2F) * (greenRed + greenBlue)) / Vector512.Create(parameters.Kg)); + Vector512 b = y + (Vector512.Create(parameters.BlueChromaScale) * cb); + + y = r; + cb = g; + cr = b; + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void ConvertFromRgb( + float r, + float g, + float b, + in HeifColorConversionParameters parameters, + out float y, + out float cb, + out float cr) + { + // Luma is shared by both chroma equations, so calculate it once before projecting blue and red. + y = (parameters.Kr * r) + (parameters.Kg * g) + (parameters.Kb * b); + cb = (b - y) / parameters.BlueChromaScale; + cr = (r - y) / parameters.RedChromaScale; + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void ConvertFromRgb( + Vector128 r, + Vector128 g, + Vector128 b, + in HeifColorConversionParameters parameters, + out Vector128 y, + out Vector128 cb, + out Vector128 cr) + { + y = Vector128.MultiplyAddEstimate( + Vector128.Create(parameters.Kr), + r, + Vector128.MultiplyAddEstimate(Vector128.Create(parameters.Kg), g, Vector128.Create(parameters.Kb) * b)); + cb = (b - y) / Vector128.Create(parameters.BlueChromaScale); + cr = (r - y) / Vector128.Create(parameters.RedChromaScale); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void ConvertFromRgb( + Vector256 r, + Vector256 g, + Vector256 b, + in HeifColorConversionParameters parameters, + out Vector256 y, + out Vector256 cb, + out Vector256 cr) + { + y = Vector256.MultiplyAddEstimate( + Vector256.Create(parameters.Kr), + r, + Vector256.MultiplyAddEstimate(Vector256.Create(parameters.Kg), g, Vector256.Create(parameters.Kb) * b)); + cb = (b - y) / Vector256.Create(parameters.BlueChromaScale); + cr = (r - y) / Vector256.Create(parameters.RedChromaScale); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void ConvertFromRgb( + Vector512 r, + Vector512 g, + Vector512 b, + in HeifColorConversionParameters parameters, + out Vector512 y, + out Vector512 cb, + out Vector512 cr) + { + y = Vector512.MultiplyAddEstimate( + Vector512.Create(parameters.Kr), + r, + Vector512.MultiplyAddEstimate(Vector512.Create(parameters.Kg), g, Vector512.Create(parameters.Kb) * b)); + cb = (b - y) / Vector512.Create(parameters.BlueChromaScale); + cr = (r - y) / Vector512.Create(parameters.RedChromaScale); + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Components/ColorConverters/HeifColorConverter.ConstantLuminanceOperator.cs b/src/ImageSharp/Formats/Heif/Components/ColorConverters/HeifColorConverter.ConstantLuminanceOperator.cs new file mode 100644 index 0000000000..98ad00352c --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Components/ColorConverters/HeifColorConverter.ConstantLuminanceOperator.cs @@ -0,0 +1,252 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Components; + +/// +/// Provides constant-luminance YCbCr conversion for scalar and SIMD lanes. +/// +internal abstract partial class HeifColorConverterBase +{ + /// + /// Implements H.273 constant-luminance conversion for scalar and SIMD lanes. + /// + internal readonly struct HeifConstantLuminanceColorOperator : IHeifColorOperator + { + /// + public static bool ChromaUsesLumaRange => false; + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void ConvertToRgb(ref float y, ref float cb, ref float cr, in HeifColorConversionParameters parameters) + { + HeifConstantLuminanceScales scales = parameters.ConstantLuminanceScales; + + // Constant-luminance chroma has different positive and negative divisors. Reconstruct nonlinear + // red and blue first, then solve for green in linear light using the signaled transfer curve. + float nonlinearBlue = y + (2F * (cb <= 0F ? scales.NegativeBlue : scales.PositiveBlue) * cb); + float nonlinearRed = y + (2F * (cr <= 0F ? scales.NegativeRed : scales.PositiveRed) * cr); + float linearY = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, y); + float linearBlue = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, nonlinearBlue); + float linearRed = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, nonlinearRed); + float linearGreen = (linearY - (parameters.Kr * linearRed) - (parameters.Kb * linearBlue)) / parameters.Kg; + y = nonlinearRed; + cb = HeifTransferFunctions.ToGamma(parameters.TransferCharacteristics, linearGreen); + cr = nonlinearBlue; + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void ConvertToRgb(ref Vector128 y, ref Vector128 cb, ref Vector128 cr, in HeifColorConversionParameters parameters) + { + HeifConstantLuminanceScales scales = parameters.ConstantLuminanceScales; + Vector128 blueScale = Vector128.ConditionalSelect( + Vector128.LessThanOrEqual(cb, Vector128.Zero), + Vector128.Create(scales.NegativeBlue), + Vector128.Create(scales.PositiveBlue)); + Vector128 redScale = Vector128.ConditionalSelect( + Vector128.LessThanOrEqual(cr, Vector128.Zero), + Vector128.Create(scales.NegativeRed), + Vector128.Create(scales.PositiveRed)); + Vector128 nonlinearBlue = Vector128.MultiplyAddEstimate(Vector128.Create(2F) * blueScale, cb, y); + Vector128 nonlinearRed = Vector128.MultiplyAddEstimate(Vector128.Create(2F) * redScale, cr, y); + Vector128 linearY = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, y); + Vector128 linearBlue = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, nonlinearBlue); + Vector128 linearRed = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, nonlinearRed); + Vector128 linearGreen = ( + linearY - (Vector128.Create(parameters.Kr) * linearRed) - (Vector128.Create(parameters.Kb) * linearBlue)) + / Vector128.Create(parameters.Kg); + + y = nonlinearRed; + cb = HeifTransferFunctions.ToGamma(parameters.TransferCharacteristics, linearGreen); + cr = nonlinearBlue; + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void ConvertToRgb(ref Vector256 y, ref Vector256 cb, ref Vector256 cr, in HeifColorConversionParameters parameters) + { + HeifConstantLuminanceScales scales = parameters.ConstantLuminanceScales; + Vector256 blueScale = Vector256.ConditionalSelect( + Vector256.LessThanOrEqual(cb, Vector256.Zero), + Vector256.Create(scales.NegativeBlue), + Vector256.Create(scales.PositiveBlue)); + Vector256 redScale = Vector256.ConditionalSelect( + Vector256.LessThanOrEqual(cr, Vector256.Zero), + Vector256.Create(scales.NegativeRed), + Vector256.Create(scales.PositiveRed)); + Vector256 nonlinearBlue = Vector256.MultiplyAddEstimate(Vector256.Create(2F) * blueScale, cb, y); + Vector256 nonlinearRed = Vector256.MultiplyAddEstimate(Vector256.Create(2F) * redScale, cr, y); + Vector256 linearY = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, y); + Vector256 linearBlue = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, nonlinearBlue); + Vector256 linearRed = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, nonlinearRed); + Vector256 linearGreen = ( + linearY - (Vector256.Create(parameters.Kr) * linearRed) - (Vector256.Create(parameters.Kb) * linearBlue)) + / Vector256.Create(parameters.Kg); + + y = nonlinearRed; + cb = HeifTransferFunctions.ToGamma(parameters.TransferCharacteristics, linearGreen); + cr = nonlinearBlue; + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void ConvertToRgb(ref Vector512 y, ref Vector512 cb, ref Vector512 cr, in HeifColorConversionParameters parameters) + { + HeifConstantLuminanceScales scales = parameters.ConstantLuminanceScales; + Vector512 blueScale = Vector512.ConditionalSelect( + Vector512.LessThanOrEqual(cb, Vector512.Zero), + Vector512.Create(scales.NegativeBlue), + Vector512.Create(scales.PositiveBlue)); + Vector512 redScale = Vector512.ConditionalSelect( + Vector512.LessThanOrEqual(cr, Vector512.Zero), + Vector512.Create(scales.NegativeRed), + Vector512.Create(scales.PositiveRed)); + Vector512 nonlinearBlue = Vector512.MultiplyAddEstimate(Vector512.Create(2F) * blueScale, cb, y); + Vector512 nonlinearRed = Vector512.MultiplyAddEstimate(Vector512.Create(2F) * redScale, cr, y); + Vector512 linearY = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, y); + Vector512 linearBlue = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, nonlinearBlue); + Vector512 linearRed = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, nonlinearRed); + Vector512 linearGreen = ( + linearY - (Vector512.Create(parameters.Kr) * linearRed) - (Vector512.Create(parameters.Kb) * linearBlue)) + / Vector512.Create(parameters.Kg); + + y = nonlinearRed; + cb = HeifTransferFunctions.ToGamma(parameters.TransferCharacteristics, linearGreen); + cr = nonlinearBlue; + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void ConvertFromRgb( + float r, + float g, + float b, + in HeifColorConversionParameters parameters, + out float y, + out float cb, + out float cr) + { + // Luma is formed in linear light. The nonlinear red and blue differences then choose the + // sign-dependent denominators that define constant-luminance Cb and Cr. + float linearRed = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, r); + float linearGreen = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, g); + float linearBlue = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, b); + float linearY = (parameters.Kr * linearRed) + (parameters.Kg * linearGreen) + (parameters.Kb * linearBlue); + y = HeifTransferFunctions.ToGamma(parameters.TransferCharacteristics, linearY); + float blueDifference = b - y; + float redDifference = r - y; + HeifConstantLuminanceScales scales = parameters.ConstantLuminanceScales; + cb = blueDifference / (2F * (blueDifference <= 0F ? scales.NegativeBlue : scales.PositiveBlue)); + cr = redDifference / (2F * (redDifference <= 0F ? scales.NegativeRed : scales.PositiveRed)); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void ConvertFromRgb( + Vector128 r, + Vector128 g, + Vector128 b, + in HeifColorConversionParameters parameters, + out Vector128 y, + out Vector128 cb, + out Vector128 cr) + { + Vector128 linearRed = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, r); + Vector128 linearGreen = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, g); + Vector128 linearBlue = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, b); + Vector128 linearY = Vector128.MultiplyAddEstimate( + Vector128.Create(parameters.Kr), + linearRed, + Vector128.MultiplyAddEstimate(Vector128.Create(parameters.Kg), linearGreen, Vector128.Create(parameters.Kb) * linearBlue)); + y = HeifTransferFunctions.ToGamma(parameters.TransferCharacteristics, linearY); + Vector128 blueDifference = b - y; + Vector128 redDifference = r - y; + HeifConstantLuminanceScales scales = parameters.ConstantLuminanceScales; + Vector128 blueScale = Vector128.ConditionalSelect( + Vector128.LessThanOrEqual(blueDifference, Vector128.Zero), + Vector128.Create(scales.NegativeBlue), + Vector128.Create(scales.PositiveBlue)); + Vector128 redScale = Vector128.ConditionalSelect( + Vector128.LessThanOrEqual(redDifference, Vector128.Zero), + Vector128.Create(scales.NegativeRed), + Vector128.Create(scales.PositiveRed)); + + cb = blueDifference / (Vector128.Create(2F) * blueScale); + cr = redDifference / (Vector128.Create(2F) * redScale); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void ConvertFromRgb( + Vector256 r, + Vector256 g, + Vector256 b, + in HeifColorConversionParameters parameters, + out Vector256 y, + out Vector256 cb, + out Vector256 cr) + { + Vector256 linearRed = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, r); + Vector256 linearGreen = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, g); + Vector256 linearBlue = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, b); + Vector256 linearY = Vector256.MultiplyAddEstimate( + Vector256.Create(parameters.Kr), + linearRed, + Vector256.MultiplyAddEstimate(Vector256.Create(parameters.Kg), linearGreen, Vector256.Create(parameters.Kb) * linearBlue)); + y = HeifTransferFunctions.ToGamma(parameters.TransferCharacteristics, linearY); + Vector256 blueDifference = b - y; + Vector256 redDifference = r - y; + HeifConstantLuminanceScales scales = parameters.ConstantLuminanceScales; + Vector256 blueScale = Vector256.ConditionalSelect( + Vector256.LessThanOrEqual(blueDifference, Vector256.Zero), + Vector256.Create(scales.NegativeBlue), + Vector256.Create(scales.PositiveBlue)); + Vector256 redScale = Vector256.ConditionalSelect( + Vector256.LessThanOrEqual(redDifference, Vector256.Zero), + Vector256.Create(scales.NegativeRed), + Vector256.Create(scales.PositiveRed)); + + cb = blueDifference / (Vector256.Create(2F) * blueScale); + cr = redDifference / (Vector256.Create(2F) * redScale); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void ConvertFromRgb( + Vector512 r, + Vector512 g, + Vector512 b, + in HeifColorConversionParameters parameters, + out Vector512 y, + out Vector512 cb, + out Vector512 cr) + { + Vector512 linearRed = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, r); + Vector512 linearGreen = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, g); + Vector512 linearBlue = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, b); + Vector512 linearY = Vector512.MultiplyAddEstimate( + Vector512.Create(parameters.Kr), + linearRed, + Vector512.MultiplyAddEstimate(Vector512.Create(parameters.Kg), linearGreen, Vector512.Create(parameters.Kb) * linearBlue)); + y = HeifTransferFunctions.ToGamma(parameters.TransferCharacteristics, linearY); + Vector512 blueDifference = b - y; + Vector512 redDifference = r - y; + HeifConstantLuminanceScales scales = parameters.ConstantLuminanceScales; + Vector512 blueScale = Vector512.ConditionalSelect( + Vector512.LessThanOrEqual(blueDifference, Vector512.Zero), + Vector512.Create(scales.NegativeBlue), + Vector512.Create(scales.PositiveBlue)); + Vector512 redScale = Vector512.ConditionalSelect( + Vector512.LessThanOrEqual(redDifference, Vector512.Zero), + Vector512.Create(scales.NegativeRed), + Vector512.Create(scales.PositiveRed)); + + cb = blueDifference / (Vector512.Create(2F) * blueScale); + cr = redDifference / (Vector512.Create(2F) * redScale); + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Components/ColorConverters/HeifColorConverter.ICtCpOperator.cs b/src/ImageSharp/Formats/Heif/Components/ColorConverters/HeifColorConverter.ICtCpOperator.cs new file mode 100644 index 0000000000..af37f6d5ed --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Components/ColorConverters/HeifColorConverter.ICtCpOperator.cs @@ -0,0 +1,535 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.Intrinsics; +using SixLabors.ImageSharp.Metadata.Profiles.Cicp; + +namespace SixLabors.ImageSharp.Formats.Heif.Components; + +/// +/// Provides ICtCp transfer-domain matrix conversion for scalar and SIMD lanes. +/// +internal abstract partial class HeifColorConverterBase +{ + /// + /// Implements BT.2100 ICtCp conversion for scalar and SIMD lanes. + /// + /// + /// H.273 equations 72-74 and 75-77 define the forward PQ and HLG matrices from nonlinear LMS to I, Ct, and Cp. Decoding requires the + /// inverse matrices, whose non-identity terms are used below as L' = I + aCt + bCp, M' = I - aCt - bCp, and + /// S' = I + cCt + dCp. Each rational constant is the exact result of inverting the corresponding integer-over-4096 matrix; + /// the explicit double-precision division preserves that value until the compile-time conversion to . + /// + internal readonly struct HeifICtCpColorOperator : IHeifColorOperator + { + /// + /// The PQ Ct contribution to nonlinear L, derived by inverting H.273 equations 72-74. + /// + public const float PqCtToL = (float)(1_112_064D / 129_174_029D); + + /// + /// The PQ Cp contribution to nonlinear L, derived by inverting H.273 equations 72-74. + /// + public const float PqCpToL = (float)(14_342_144D / 129_174_029D); + + /// + /// The PQ Ct contribution to nonlinear S, derived by inverting H.273 equations 72-74. + /// + public const float PqCtToS = (float)(72_341_504D / 129_174_029D); + + /// + /// The PQ Cp contribution to nonlinear S, derived by inverting H.273 equations 72-74. + /// + public const float PqCpToS = (float)(-41_416_704D / 129_174_029D); + + /// + /// The HLG Ct contribution to nonlinear L, derived by inverting H.273 equations 75-77. + /// + public const float HlgCtToL = (float)(6_144D / 390_875D); + + /// + /// The HLG Cp contribution to nonlinear L, derived by inverting H.273 equations 75-77. + /// + public const float HlgCpToL = (float)(16_384D / 78_175D); + + /// + /// The HLG Ct contribution to nonlinear S, derived by inverting H.273 equations 75-77. + /// + public const float HlgCtToS = (float)(1_197_568D / 1_172_625D); + + /// + /// The HLG Cp contribution to nonlinear S, derived by inverting H.273 equations 75-77. + /// + public const float HlgCpToS = (float)(-141_952D / 234_525D); + + /// + /// The linear L contribution to red. + /// + public const float LToRed = 3.4366066943330784F; + + /// + /// The linear M contribution to red. + /// + public const float MToRed = -2.50645211865627F; + + /// + /// The linear S contribution to red. + /// + public const float SToRed = 0.06984542432319148F; + + /// + /// The linear L contribution to green. + /// + public const float LToGreen = -0.7913295555989287F; + + /// + /// The linear M contribution to green. + /// + public const float MToGreen = 1.9836004517922907F; + + /// + /// The linear S contribution to green. + /// + public const float SToGreen = -0.192270896193362F; + + /// + /// The linear L contribution to blue. + /// + public const float LToBlue = -0.025949899690592672F; + + /// + /// The linear M contribution to blue. + /// + public const float MToBlue = -0.09891371471172644F; + + /// + /// The linear S contribution to blue. + /// + public const float SToBlue = 1.1248636144023192F; + + /// + /// The linear red contribution to L. + /// + public const float RedToL = 1688F / 4096F; + + /// + /// The linear green contribution to L. + /// + public const float GreenToL = 2146F / 4096F; + + /// + /// The linear blue contribution to L. + /// + public const float BlueToL = 262F / 4096F; + + /// + /// The linear red contribution to M. + /// + public const float RedToM = 683F / 4096F; + + /// + /// The linear green contribution to M. + /// + public const float GreenToM = 2951F / 4096F; + + /// + /// The linear blue contribution to M. + /// + public const float BlueToM = 462F / 4096F; + + /// + /// The linear red contribution to S. + /// + public const float RedToS = 99F / 4096F; + + /// + /// The linear green contribution to S. + /// + public const float GreenToS = 309F / 4096F; + + /// + /// The linear blue contribution to S. + /// + public const float BlueToS = 3688F / 4096F; + + /// + /// The PQ nonlinear L contribution to Ct. + /// + public const float PqLToCt = 6610F / 4096F; + + /// + /// The PQ nonlinear M contribution to Ct. + /// + public const float PqMToCt = -13613F / 4096F; + + /// + /// The PQ nonlinear S contribution to Ct. + /// + public const float PqSToCt = 7003F / 4096F; + + /// + /// The PQ nonlinear L contribution to Cp. + /// + public const float PqLToCp = 17933F / 4096F; + + /// + /// The PQ nonlinear M contribution to Cp. + /// + public const float PqMToCp = -17390F / 4096F; + + /// + /// The PQ nonlinear S contribution to Cp. + /// + public const float PqSToCp = -543F / 4096F; + + /// + /// The HLG nonlinear L contribution to Ct. + /// + public const float HlgLToCt = 3625F / 4096F; + + /// + /// The HLG nonlinear M contribution to Ct. + /// + public const float HlgMToCt = -7465F / 4096F; + + /// + /// The HLG nonlinear S contribution to Ct. + /// + public const float HlgSToCt = 3840F / 4096F; + + /// + /// The HLG nonlinear L contribution to Cp. + /// + public const float HlgLToCp = 9500F / 4096F; + + /// + /// The HLG nonlinear M contribution to Cp. + /// + public const float HlgMToCp = -9212F / 4096F; + + /// + /// The HLG nonlinear S contribution to Cp. + /// + public const float HlgSToCp = -288F / 4096F; + + /// + public static bool ChromaUsesLumaRange => false; + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void ConvertToRgb(ref float intensity, ref float ct, ref float cp, in HeifColorConversionParameters parameters) + { + bool isHlg = parameters.TransferCharacteristics == CicpTransferCharacteristics.AribStdB67; + float ctToL = isHlg ? HlgCtToL : PqCtToL; + float cpToL = isHlg ? HlgCpToL : PqCpToL; + + // The inverse ICtCp matrix first reconstructs nonlinear LMS. The transfer curve is then + // removed before the fixed LMS-to-RGB matrix and reapplied to the three output primaries. + float nonlinearL = intensity + (ctToL * ct) + (cpToL * cp); + float nonlinearM = intensity - (ctToL * ct) - (cpToL * cp); + float nonlinearS = intensity + ((isHlg ? HlgCtToS : PqCtToS) * ct) + ((isHlg ? HlgCpToS : PqCpToS) * cp); + float linearL = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, nonlinearL); + float linearM = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, nonlinearM); + float linearS = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, nonlinearS); + float linearRed = (LToRed * linearL) + (MToRed * linearM) + (SToRed * linearS); + float linearGreen = (LToGreen * linearL) + (MToGreen * linearM) + (SToGreen * linearS); + float linearBlue = (LToBlue * linearL) + (MToBlue * linearM) + (SToBlue * linearS); + + intensity = HeifTransferFunctions.ToGamma(parameters.TransferCharacteristics, linearRed); + ct = HeifTransferFunctions.ToGamma(parameters.TransferCharacteristics, linearGreen); + cp = HeifTransferFunctions.ToGamma(parameters.TransferCharacteristics, linearBlue); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void ConvertToRgb( + ref Vector128 intensity, + ref Vector128 ct, + ref Vector128 cp, + in HeifColorConversionParameters parameters) + { + bool isHlg = parameters.TransferCharacteristics == CicpTransferCharacteristics.AribStdB67; + Vector128 ctContribution = Vector128.Create(isHlg ? HlgCtToL : PqCtToL) * ct; + Vector128 cpContribution = Vector128.Create(isHlg ? HlgCpToL : PqCpToL) * cp; + Vector128 nonlinearL = intensity + ctContribution + cpContribution; + Vector128 nonlinearM = intensity - ctContribution - cpContribution; + Vector128 nonlinearS = Vector128.MultiplyAddEstimate( + Vector128.Create(isHlg ? HlgCpToS : PqCpToS), + cp, + Vector128.MultiplyAddEstimate(Vector128.Create(isHlg ? HlgCtToS : PqCtToS), ct, intensity)); + Vector128 linearL = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, nonlinearL); + Vector128 linearM = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, nonlinearM); + Vector128 linearS = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, nonlinearS); + Vector128 linearRed = Vector128.MultiplyAddEstimate( + Vector128.Create(SToRed), + linearS, + Vector128.MultiplyAddEstimate(Vector128.Create(MToRed), linearM, Vector128.Create(LToRed) * linearL)); + Vector128 linearGreen = Vector128.MultiplyAddEstimate( + Vector128.Create(SToGreen), + linearS, + Vector128.MultiplyAddEstimate(Vector128.Create(MToGreen), linearM, Vector128.Create(LToGreen) * linearL)); + Vector128 linearBlue = Vector128.MultiplyAddEstimate( + Vector128.Create(SToBlue), + linearS, + Vector128.MultiplyAddEstimate(Vector128.Create(MToBlue), linearM, Vector128.Create(LToBlue) * linearL)); + + intensity = HeifTransferFunctions.ToGamma(parameters.TransferCharacteristics, linearRed); + ct = HeifTransferFunctions.ToGamma(parameters.TransferCharacteristics, linearGreen); + cp = HeifTransferFunctions.ToGamma(parameters.TransferCharacteristics, linearBlue); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void ConvertToRgb( + ref Vector256 intensity, + ref Vector256 ct, + ref Vector256 cp, + in HeifColorConversionParameters parameters) + { + bool isHlg = parameters.TransferCharacteristics == CicpTransferCharacteristics.AribStdB67; + Vector256 ctContribution = Vector256.Create(isHlg ? HlgCtToL : PqCtToL) * ct; + Vector256 cpContribution = Vector256.Create(isHlg ? HlgCpToL : PqCpToL) * cp; + Vector256 nonlinearL = intensity + ctContribution + cpContribution; + Vector256 nonlinearM = intensity - ctContribution - cpContribution; + Vector256 nonlinearS = Vector256.MultiplyAddEstimate( + Vector256.Create(isHlg ? HlgCpToS : PqCpToS), + cp, + Vector256.MultiplyAddEstimate(Vector256.Create(isHlg ? HlgCtToS : PqCtToS), ct, intensity)); + Vector256 linearL = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, nonlinearL); + Vector256 linearM = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, nonlinearM); + Vector256 linearS = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, nonlinearS); + Vector256 linearRed = Vector256.MultiplyAddEstimate( + Vector256.Create(SToRed), + linearS, + Vector256.MultiplyAddEstimate(Vector256.Create(MToRed), linearM, Vector256.Create(LToRed) * linearL)); + Vector256 linearGreen = Vector256.MultiplyAddEstimate( + Vector256.Create(SToGreen), + linearS, + Vector256.MultiplyAddEstimate(Vector256.Create(MToGreen), linearM, Vector256.Create(LToGreen) * linearL)); + Vector256 linearBlue = Vector256.MultiplyAddEstimate( + Vector256.Create(SToBlue), + linearS, + Vector256.MultiplyAddEstimate(Vector256.Create(MToBlue), linearM, Vector256.Create(LToBlue) * linearL)); + + intensity = HeifTransferFunctions.ToGamma(parameters.TransferCharacteristics, linearRed); + ct = HeifTransferFunctions.ToGamma(parameters.TransferCharacteristics, linearGreen); + cp = HeifTransferFunctions.ToGamma(parameters.TransferCharacteristics, linearBlue); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void ConvertToRgb( + ref Vector512 intensity, + ref Vector512 ct, + ref Vector512 cp, + in HeifColorConversionParameters parameters) + { + bool isHlg = parameters.TransferCharacteristics == CicpTransferCharacteristics.AribStdB67; + Vector512 ctContribution = Vector512.Create(isHlg ? HlgCtToL : PqCtToL) * ct; + Vector512 cpContribution = Vector512.Create(isHlg ? HlgCpToL : PqCpToL) * cp; + Vector512 nonlinearL = intensity + ctContribution + cpContribution; + Vector512 nonlinearM = intensity - ctContribution - cpContribution; + Vector512 nonlinearS = Vector512.MultiplyAddEstimate( + Vector512.Create(isHlg ? HlgCpToS : PqCpToS), + cp, + Vector512.MultiplyAddEstimate(Vector512.Create(isHlg ? HlgCtToS : PqCtToS), ct, intensity)); + Vector512 linearL = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, nonlinearL); + Vector512 linearM = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, nonlinearM); + Vector512 linearS = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, nonlinearS); + Vector512 linearRed = Vector512.MultiplyAddEstimate( + Vector512.Create(SToRed), + linearS, + Vector512.MultiplyAddEstimate(Vector512.Create(MToRed), linearM, Vector512.Create(LToRed) * linearL)); + Vector512 linearGreen = Vector512.MultiplyAddEstimate( + Vector512.Create(SToGreen), + linearS, + Vector512.MultiplyAddEstimate(Vector512.Create(MToGreen), linearM, Vector512.Create(LToGreen) * linearL)); + Vector512 linearBlue = Vector512.MultiplyAddEstimate( + Vector512.Create(SToBlue), + linearS, + Vector512.MultiplyAddEstimate(Vector512.Create(MToBlue), linearM, Vector512.Create(LToBlue) * linearL)); + + intensity = HeifTransferFunctions.ToGamma(parameters.TransferCharacteristics, linearRed); + ct = HeifTransferFunctions.ToGamma(parameters.TransferCharacteristics, linearGreen); + cp = HeifTransferFunctions.ToGamma(parameters.TransferCharacteristics, linearBlue); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void ConvertFromRgb( + float red, + float green, + float blue, + in HeifColorConversionParameters parameters, + out float intensity, + out float ct, + out float cp) + { + // ICtCp is defined in nonlinear LMS. Convert RGB to linear light, apply the LMS matrix, then + // apply the signaled PQ or HLG transfer curve before deriving intensity and the chroma axes. + float linearRed = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, red); + float linearGreen = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, green); + float linearBlue = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, blue); + float nonlinearL = HeifTransferFunctions.ToGamma( + parameters.TransferCharacteristics, + (RedToL * linearRed) + (GreenToL * linearGreen) + (BlueToL * linearBlue)); + float nonlinearM = HeifTransferFunctions.ToGamma( + parameters.TransferCharacteristics, + (RedToM * linearRed) + (GreenToM * linearGreen) + (BlueToM * linearBlue)); + float nonlinearS = HeifTransferFunctions.ToGamma( + parameters.TransferCharacteristics, + (RedToS * linearRed) + (GreenToS * linearGreen) + (BlueToS * linearBlue)); + intensity = 0.5F * (nonlinearL + nonlinearM); + bool isHlg = parameters.TransferCharacteristics == CicpTransferCharacteristics.AribStdB67; + ct = isHlg + ? (HlgLToCt * nonlinearL) + (HlgMToCt * nonlinearM) + (HlgSToCt * nonlinearS) + : (PqLToCt * nonlinearL) + (PqMToCt * nonlinearM) + (PqSToCt * nonlinearS); + + cp = isHlg + ? (HlgLToCp * nonlinearL) + (HlgMToCp * nonlinearM) + (HlgSToCp * nonlinearS) + : (PqLToCp * nonlinearL) + (PqMToCp * nonlinearM) + (PqSToCp * nonlinearS); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void ConvertFromRgb( + Vector128 red, + Vector128 green, + Vector128 blue, + in HeifColorConversionParameters parameters, + out Vector128 intensity, + out Vector128 ct, + out Vector128 cp) + { + Vector128 linearRed = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, red); + Vector128 linearGreen = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, green); + Vector128 linearBlue = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, blue); + Vector128 linearL = Vector128.MultiplyAddEstimate( + Vector128.Create(RedToL), + linearRed, + Vector128.MultiplyAddEstimate(Vector128.Create(GreenToL), linearGreen, Vector128.Create(BlueToL) * linearBlue)); + Vector128 linearM = Vector128.MultiplyAddEstimate( + Vector128.Create(RedToM), + linearRed, + Vector128.MultiplyAddEstimate(Vector128.Create(GreenToM), linearGreen, Vector128.Create(BlueToM) * linearBlue)); + Vector128 linearS = Vector128.MultiplyAddEstimate( + Vector128.Create(RedToS), + linearRed, + Vector128.MultiplyAddEstimate(Vector128.Create(GreenToS), linearGreen, Vector128.Create(BlueToS) * linearBlue)); + Vector128 nonlinearL = HeifTransferFunctions.ToGamma(parameters.TransferCharacteristics, linearL); + Vector128 nonlinearM = HeifTransferFunctions.ToGamma(parameters.TransferCharacteristics, linearM); + Vector128 nonlinearS = HeifTransferFunctions.ToGamma(parameters.TransferCharacteristics, linearS); + intensity = Vector128.Create(0.5F) * (nonlinearL + nonlinearM); + bool isHlg = parameters.TransferCharacteristics == CicpTransferCharacteristics.AribStdB67; + float lToCt = isHlg ? HlgLToCt : PqLToCt; + float mToCt = isHlg ? HlgMToCt : PqMToCt; + float sToCt = isHlg ? HlgSToCt : PqSToCt; + float lToCp = isHlg ? HlgLToCp : PqLToCp; + float mToCp = isHlg ? HlgMToCp : PqMToCp; + float sToCp = isHlg ? HlgSToCp : PqSToCp; + ct = Vector128.MultiplyAddEstimate( + Vector128.Create(lToCt), + nonlinearL, + Vector128.MultiplyAddEstimate(Vector128.Create(mToCt), nonlinearM, Vector128.Create(sToCt) * nonlinearS)); + cp = Vector128.MultiplyAddEstimate( + Vector128.Create(lToCp), + nonlinearL, + Vector128.MultiplyAddEstimate(Vector128.Create(mToCp), nonlinearM, Vector128.Create(sToCp) * nonlinearS)); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void ConvertFromRgb( + Vector256 red, + Vector256 green, + Vector256 blue, + in HeifColorConversionParameters parameters, + out Vector256 intensity, + out Vector256 ct, + out Vector256 cp) + { + Vector256 linearRed = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, red); + Vector256 linearGreen = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, green); + Vector256 linearBlue = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, blue); + Vector256 linearL = Vector256.MultiplyAddEstimate( + Vector256.Create(RedToL), + linearRed, + Vector256.MultiplyAddEstimate(Vector256.Create(GreenToL), linearGreen, Vector256.Create(BlueToL) * linearBlue)); + Vector256 linearM = Vector256.MultiplyAddEstimate( + Vector256.Create(RedToM), + linearRed, + Vector256.MultiplyAddEstimate(Vector256.Create(GreenToM), linearGreen, Vector256.Create(BlueToM) * linearBlue)); + Vector256 linearS = Vector256.MultiplyAddEstimate( + Vector256.Create(RedToS), + linearRed, + Vector256.MultiplyAddEstimate(Vector256.Create(GreenToS), linearGreen, Vector256.Create(BlueToS) * linearBlue)); + Vector256 nonlinearL = HeifTransferFunctions.ToGamma(parameters.TransferCharacteristics, linearL); + Vector256 nonlinearM = HeifTransferFunctions.ToGamma(parameters.TransferCharacteristics, linearM); + Vector256 nonlinearS = HeifTransferFunctions.ToGamma(parameters.TransferCharacteristics, linearS); + intensity = Vector256.Create(0.5F) * (nonlinearL + nonlinearM); + bool isHlg = parameters.TransferCharacteristics == CicpTransferCharacteristics.AribStdB67; + float lToCt = isHlg ? HlgLToCt : PqLToCt; + float mToCt = isHlg ? HlgMToCt : PqMToCt; + float sToCt = isHlg ? HlgSToCt : PqSToCt; + float lToCp = isHlg ? HlgLToCp : PqLToCp; + float mToCp = isHlg ? HlgMToCp : PqMToCp; + float sToCp = isHlg ? HlgSToCp : PqSToCp; + ct = Vector256.MultiplyAddEstimate( + Vector256.Create(lToCt), + nonlinearL, + Vector256.MultiplyAddEstimate(Vector256.Create(mToCt), nonlinearM, Vector256.Create(sToCt) * nonlinearS)); + cp = Vector256.MultiplyAddEstimate( + Vector256.Create(lToCp), + nonlinearL, + Vector256.MultiplyAddEstimate(Vector256.Create(mToCp), nonlinearM, Vector256.Create(sToCp) * nonlinearS)); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void ConvertFromRgb( + Vector512 red, + Vector512 green, + Vector512 blue, + in HeifColorConversionParameters parameters, + out Vector512 intensity, + out Vector512 ct, + out Vector512 cp) + { + Vector512 linearRed = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, red); + Vector512 linearGreen = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, green); + Vector512 linearBlue = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, blue); + Vector512 linearL = Vector512.MultiplyAddEstimate( + Vector512.Create(RedToL), + linearRed, + Vector512.MultiplyAddEstimate(Vector512.Create(GreenToL), linearGreen, Vector512.Create(BlueToL) * linearBlue)); + Vector512 linearM = Vector512.MultiplyAddEstimate( + Vector512.Create(RedToM), + linearRed, + Vector512.MultiplyAddEstimate(Vector512.Create(GreenToM), linearGreen, Vector512.Create(BlueToM) * linearBlue)); + Vector512 linearS = Vector512.MultiplyAddEstimate( + Vector512.Create(RedToS), + linearRed, + Vector512.MultiplyAddEstimate(Vector512.Create(GreenToS), linearGreen, Vector512.Create(BlueToS) * linearBlue)); + Vector512 nonlinearL = HeifTransferFunctions.ToGamma(parameters.TransferCharacteristics, linearL); + Vector512 nonlinearM = HeifTransferFunctions.ToGamma(parameters.TransferCharacteristics, linearM); + Vector512 nonlinearS = HeifTransferFunctions.ToGamma(parameters.TransferCharacteristics, linearS); + intensity = Vector512.Create(0.5F) * (nonlinearL + nonlinearM); + bool isHlg = parameters.TransferCharacteristics == CicpTransferCharacteristics.AribStdB67; + float lToCt = isHlg ? HlgLToCt : PqLToCt; + float mToCt = isHlg ? HlgMToCt : PqMToCt; + float sToCt = isHlg ? HlgSToCt : PqSToCt; + float lToCp = isHlg ? HlgLToCp : PqLToCp; + float mToCp = isHlg ? HlgMToCp : PqMToCp; + float sToCp = isHlg ? HlgSToCp : PqSToCp; + ct = Vector512.MultiplyAddEstimate( + Vector512.Create(lToCt), + nonlinearL, + Vector512.MultiplyAddEstimate(Vector512.Create(mToCt), nonlinearM, Vector512.Create(sToCt) * nonlinearS)); + cp = Vector512.MultiplyAddEstimate( + Vector512.Create(lToCp), + nonlinearL, + Vector512.MultiplyAddEstimate(Vector512.Create(mToCp), nonlinearM, Vector512.Create(sToCp) * nonlinearS)); + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Components/ColorConverters/HeifColorConverter.IdentityOperator.cs b/src/ImageSharp/Formats/Heif/Components/ColorConverters/HeifColorConverter.IdentityOperator.cs new file mode 100644 index 0000000000..80bafa8f44 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Components/ColorConverters/HeifColorConverter.IdentityOperator.cs @@ -0,0 +1,141 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Components; + +/// +/// Provides identity-matrix component reordering for scalar and SIMD color conversion. +/// +internal abstract partial class HeifColorConverterBase +{ + /// + /// Implements direct G, B, and R plane mapping for scalar and SIMD lanes. + /// + internal readonly struct HeifIdentityColorOperator : IHeifColorOperator + { + /// + public static bool ChromaUsesLumaRange => true; + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void ConvertToRgb(ref float green, ref float blue, ref float red, in HeifColorConversionParameters parameters) + { + // H.273 identity-matrix signaling stores the planes in G, B, R order. Rotate the three references in place + // so the shared traversal always leaves component0/component1/component2 as R, G, B. + float g = green; + green = red; + red = blue; + blue = g; + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void ConvertToRgb( + ref Vector128 green, + ref Vector128 blue, + ref Vector128 red, + in HeifColorConversionParameters parameters) + { + Vector128 g = green; + green = red; + red = blue; + blue = g; + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void ConvertToRgb( + ref Vector256 green, + ref Vector256 blue, + ref Vector256 red, + in HeifColorConversionParameters parameters) + { + Vector256 g = green; + green = red; + red = blue; + blue = g; + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void ConvertToRgb( + ref Vector512 green, + ref Vector512 blue, + ref Vector512 red, + in HeifColorConversionParameters parameters) + { + Vector512 g = green; + green = red; + red = blue; + blue = g; + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void ConvertFromRgb( + float red, + float green, + float blue, + in HeifColorConversionParameters parameters, + out float component0, + out float component1, + out float component2) + { + // H.273 identity-matrix signaling stores RGB input as G, B, R without matrix arithmetic. + component0 = green; + component1 = blue; + component2 = red; + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void ConvertFromRgb( + Vector128 red, + Vector128 green, + Vector128 blue, + in HeifColorConversionParameters parameters, + out Vector128 component0, + out Vector128 component1, + out Vector128 component2) + { + component0 = green; + component1 = blue; + component2 = red; + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void ConvertFromRgb( + Vector256 red, + Vector256 green, + Vector256 blue, + in HeifColorConversionParameters parameters, + out Vector256 component0, + out Vector256 component1, + out Vector256 component2) + { + component0 = green; + component1 = blue; + component2 = red; + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void ConvertFromRgb( + Vector512 red, + Vector512 green, + Vector512 blue, + in HeifColorConversionParameters parameters, + out Vector512 component0, + out Vector512 component1, + out Vector512 component2) + { + component0 = green; + component1 = blue; + component2 = red; + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Components/ColorConverters/HeifColorConverter.IptC2Operator.cs b/src/ImageSharp/Formats/Heif/Components/ColorConverters/HeifColorConverter.IptC2Operator.cs new file mode 100644 index 0000000000..8abd23afe8 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Components/ColorConverters/HeifColorConverter.IptC2Operator.cs @@ -0,0 +1,505 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Components; + +/// +/// Provides IPT-C2 transfer-domain matrix conversion for scalar and SIMD lanes. +/// +internal abstract partial class HeifColorConverterBase +{ + /// + /// Implements IPT-C2 conversion for scalar and SIMD lanes. + /// + internal readonly struct HeifIptC2ColorOperator : IHeifColorOperator + { + /// + /// The linear red contribution to L. + /// + public const float RedToL = 1747F / 4096F; + + /// + /// The linear green contribution to L. + /// + public const float GreenToL = 2169F / 4096F; + + /// + /// The linear blue contribution to L. + /// + public const float BlueToL = 180F / 4096F; + + /// + /// The linear red contribution to M. + /// + public const float RedToM = 673F / 4096F; + + /// + /// The linear green contribution to M. + /// + public const float GreenToM = 3029F / 4096F; + + /// + /// The linear blue contribution to M. + /// + public const float BlueToM = 394F / 4096F; + + /// + /// The linear red contribution to S. + /// + public const float RedToS = 50F / 4096F; + + /// + /// The linear green contribution to S. + /// + public const float GreenToS = 207F / 4096F; + + /// + /// The linear blue contribution to S. + /// + public const float BlueToS = 3839F / 4096F; + + /// + /// The nonlinear L contribution to intensity. + /// + public const float LToIntensity = 1638F / 4096F; + + /// + /// The nonlinear M contribution to intensity. + /// + public const float MToIntensity = 1638F / 4096F; + + /// + /// The nonlinear S contribution to intensity. + /// + public const float SToIntensity = 820F / 4096F; + + /// + /// The nonlinear L contribution to the protan axis. + /// + public const float LToProtan = 18248F / 4096F; + + /// + /// The nonlinear M contribution to the protan axis. + /// + public const float MToProtan = -19870F / 4096F; + + /// + /// The nonlinear S contribution to the protan axis. + /// + public const float SToProtan = 1622F / 4096F; + + /// + /// The nonlinear L contribution to the tritan axis. + /// + public const float LToTritan = 3300F / 4096F; + + /// + /// The nonlinear M contribution to the tritan axis. + /// + public const float MToTritan = 1463F / 4096F; + + /// + /// The nonlinear S contribution to the tritan axis. + /// + public const float SToTritan = -4763F / 4096F; + + /// + /// The protan contribution to nonlinear L. + /// + public const float ProtanToL = 0.0975578875686935F; + + /// + /// The tritan contribution to nonlinear L. + /// + public const float TritanToL = 0.20538292958984272F; + + /// + /// The protan contribution to nonlinear M. + /// + public const float ProtanToM = -0.11388362209560723F; + + /// + /// The tritan contribution to nonlinear M. + /// + public const float TritanToM = 0.13337828363655785F; + + /// + /// The protan contribution to nonlinear S. + /// + public const float ProtanToS = 0.032611650189127685F; + + /// + /// The tritan contribution to nonlinear S. + /// + public const float TritanToS = -0.6766961795912734F; + + /// + /// The linear L contribution to red. + /// + public const float LToRed = 3.2374662424353895F; + + /// + /// The linear M contribution to red. + /// + public const float MToRed = -2.324205800020636F; + + /// + /// The linear S contribution to red. + /// + public const float SToRed = 0.08673955758524626F; + + /// + /// The linear L contribution to green. + /// + public const float LToGreen = -0.7188754693535147F; + + /// + /// The linear M contribution to green. + /// + public const float MToGreen = 1.877899954242238F; + + /// + /// The linear S contribution to green. + /// + public const float SToGreen = -0.1590244848887234F; + + /// + /// The linear L contribution to blue. + /// + public const float LToBlue = -0.003403513926958051F; + + /// + /// The linear M contribution to blue. + /// + public const float MToBlue = -0.07098593397424108F; + + /// + /// The linear S contribution to blue. + /// + public const float SToBlue = 1.074389447901199F; + + /// + public static bool ChromaUsesLumaRange => false; + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void ConvertToRgb(ref float intensity, ref float protan, ref float tritan, in HeifColorConversionParameters parameters) + { + // IPT-C2 stores opponent axes around intensity in nonlinear LMS. Undo both matrices around the + // signaled transfer function so the final RGB values remain in the source signal domain. + float nonlinearL = intensity + (ProtanToL * protan) + (TritanToL * tritan); + float nonlinearM = intensity + (ProtanToM * protan) + (TritanToM * tritan); + float nonlinearS = intensity + (ProtanToS * protan) + (TritanToS * tritan); + float linearL = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, nonlinearL); + float linearM = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, nonlinearM); + float linearS = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, nonlinearS); + float linearRed = (LToRed * linearL) + (MToRed * linearM) + (SToRed * linearS); + float linearGreen = (LToGreen * linearL) + (MToGreen * linearM) + (SToGreen * linearS); + float linearBlue = (LToBlue * linearL) + (MToBlue * linearM) + (SToBlue * linearS); + + intensity = HeifTransferFunctions.ToGamma(parameters.TransferCharacteristics, linearRed); + protan = HeifTransferFunctions.ToGamma(parameters.TransferCharacteristics, linearGreen); + tritan = HeifTransferFunctions.ToGamma(parameters.TransferCharacteristics, linearBlue); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void ConvertToRgb( + ref Vector128 intensity, + ref Vector128 protan, + ref Vector128 tritan, + in HeifColorConversionParameters parameters) + { + Vector128 nonlinearL = Vector128.MultiplyAddEstimate( + Vector128.Create(TritanToL), + tritan, + Vector128.MultiplyAddEstimate(Vector128.Create(ProtanToL), protan, intensity)); + + Vector128 nonlinearM = Vector128.MultiplyAddEstimate( + Vector128.Create(TritanToM), + tritan, + Vector128.MultiplyAddEstimate(Vector128.Create(ProtanToM), protan, intensity)); + + Vector128 nonlinearS = Vector128.MultiplyAddEstimate( + Vector128.Create(TritanToS), + tritan, + Vector128.MultiplyAddEstimate(Vector128.Create(ProtanToS), protan, intensity)); + + Vector128 linearL = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, nonlinearL); + Vector128 linearM = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, nonlinearM); + Vector128 linearS = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, nonlinearS); + Vector128 linearRed = Vector128.MultiplyAddEstimate( + Vector128.Create(SToRed), linearS, Vector128.MultiplyAddEstimate(Vector128.Create(MToRed), linearM, Vector128.Create(LToRed) * linearL)); + + Vector128 linearGreen = Vector128.MultiplyAddEstimate( + Vector128.Create(SToGreen), linearS, Vector128.MultiplyAddEstimate(Vector128.Create(MToGreen), linearM, Vector128.Create(LToGreen) * linearL)); + + Vector128 linearBlue = Vector128.MultiplyAddEstimate( + Vector128.Create(SToBlue), linearS, Vector128.MultiplyAddEstimate(Vector128.Create(MToBlue), linearM, Vector128.Create(LToBlue) * linearL)); + + intensity = HeifTransferFunctions.ToGamma(parameters.TransferCharacteristics, linearRed); + protan = HeifTransferFunctions.ToGamma(parameters.TransferCharacteristics, linearGreen); + tritan = HeifTransferFunctions.ToGamma(parameters.TransferCharacteristics, linearBlue); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void ConvertToRgb( + ref Vector256 intensity, + ref Vector256 protan, + ref Vector256 tritan, + in HeifColorConversionParameters parameters) + { + Vector256 nonlinearL = Vector256.MultiplyAddEstimate( + Vector256.Create(TritanToL), + tritan, + Vector256.MultiplyAddEstimate(Vector256.Create(ProtanToL), protan, intensity)); + + Vector256 nonlinearM = Vector256.MultiplyAddEstimate( + Vector256.Create(TritanToM), + tritan, + Vector256.MultiplyAddEstimate(Vector256.Create(ProtanToM), protan, intensity)); + + Vector256 nonlinearS = Vector256.MultiplyAddEstimate( + Vector256.Create(TritanToS), + tritan, + Vector256.MultiplyAddEstimate(Vector256.Create(ProtanToS), protan, intensity)); + + Vector256 linearL = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, nonlinearL); + Vector256 linearM = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, nonlinearM); + Vector256 linearS = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, nonlinearS); + Vector256 linearRed = Vector256.MultiplyAddEstimate( + Vector256.Create(SToRed), linearS, Vector256.MultiplyAddEstimate(Vector256.Create(MToRed), linearM, Vector256.Create(LToRed) * linearL)); + + Vector256 linearGreen = Vector256.MultiplyAddEstimate( + Vector256.Create(SToGreen), linearS, Vector256.MultiplyAddEstimate(Vector256.Create(MToGreen), linearM, Vector256.Create(LToGreen) * linearL)); + + Vector256 linearBlue = Vector256.MultiplyAddEstimate( + Vector256.Create(SToBlue), linearS, Vector256.MultiplyAddEstimate(Vector256.Create(MToBlue), linearM, Vector256.Create(LToBlue) * linearL)); + + intensity = HeifTransferFunctions.ToGamma(parameters.TransferCharacteristics, linearRed); + protan = HeifTransferFunctions.ToGamma(parameters.TransferCharacteristics, linearGreen); + tritan = HeifTransferFunctions.ToGamma(parameters.TransferCharacteristics, linearBlue); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void ConvertToRgb( + ref Vector512 intensity, + ref Vector512 protan, + ref Vector512 tritan, + in HeifColorConversionParameters parameters) + { + Vector512 nonlinearL = Vector512.MultiplyAddEstimate( + Vector512.Create(TritanToL), + tritan, + Vector512.MultiplyAddEstimate(Vector512.Create(ProtanToL), protan, intensity)); + + Vector512 nonlinearM = Vector512.MultiplyAddEstimate( + Vector512.Create(TritanToM), + tritan, + Vector512.MultiplyAddEstimate(Vector512.Create(ProtanToM), protan, intensity)); + + Vector512 nonlinearS = Vector512.MultiplyAddEstimate( + Vector512.Create(TritanToS), + tritan, + Vector512.MultiplyAddEstimate(Vector512.Create(ProtanToS), protan, intensity)); + + Vector512 linearL = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, nonlinearL); + Vector512 linearM = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, nonlinearM); + Vector512 linearS = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, nonlinearS); + Vector512 linearRed = Vector512.MultiplyAddEstimate( + Vector512.Create(SToRed), linearS, Vector512.MultiplyAddEstimate(Vector512.Create(MToRed), linearM, Vector512.Create(LToRed) * linearL)); + + Vector512 linearGreen = Vector512.MultiplyAddEstimate( + Vector512.Create(SToGreen), linearS, Vector512.MultiplyAddEstimate(Vector512.Create(MToGreen), linearM, Vector512.Create(LToGreen) * linearL)); + + Vector512 linearBlue = Vector512.MultiplyAddEstimate( + Vector512.Create(SToBlue), linearS, Vector512.MultiplyAddEstimate(Vector512.Create(MToBlue), linearM, Vector512.Create(LToBlue) * linearL)); + + intensity = HeifTransferFunctions.ToGamma(parameters.TransferCharacteristics, linearRed); + protan = HeifTransferFunctions.ToGamma(parameters.TransferCharacteristics, linearGreen); + tritan = HeifTransferFunctions.ToGamma(parameters.TransferCharacteristics, linearBlue); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void ConvertFromRgb( + float red, + float green, + float blue, + in HeifColorConversionParameters parameters, + out float intensity, + out float protan, + out float tritan) + { + // The encoded RGB signal is linearized before the LMS matrix, then the signaled transfer function + // is reapplied to each LMS component before the fixed IPT-C2 opponent matrix. + float linearRed = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, red); + float linearGreen = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, green); + float linearBlue = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, blue); + float nonlinearL = HeifTransferFunctions.ToGamma( + parameters.TransferCharacteristics, + (RedToL * linearRed) + (GreenToL * linearGreen) + (BlueToL * linearBlue)); + + float nonlinearM = HeifTransferFunctions.ToGamma( + parameters.TransferCharacteristics, + (RedToM * linearRed) + (GreenToM * linearGreen) + (BlueToM * linearBlue)); + + float nonlinearS = HeifTransferFunctions.ToGamma( + parameters.TransferCharacteristics, + (RedToS * linearRed) + (GreenToS * linearGreen) + (BlueToS * linearBlue)); + + intensity = (LToIntensity * nonlinearL) + (MToIntensity * nonlinearM) + (SToIntensity * nonlinearS); + protan = (LToProtan * nonlinearL) + (MToProtan * nonlinearM) + (SToProtan * nonlinearS); + tritan = (LToTritan * nonlinearL) + (MToTritan * nonlinearM) + (SToTritan * nonlinearS); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void ConvertFromRgb( + Vector128 red, + Vector128 green, + Vector128 blue, + in HeifColorConversionParameters parameters, + out Vector128 intensity, + out Vector128 protan, + out Vector128 tritan) + { + Vector128 linearRed = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, red); + Vector128 linearGreen = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, green); + Vector128 linearBlue = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, blue); + Vector128 linearL = Vector128.MultiplyAddEstimate( + Vector128.Create(BlueToL), + linearBlue, + Vector128.MultiplyAddEstimate(Vector128.Create(GreenToL), linearGreen, Vector128.Create(RedToL) * linearRed)); + + Vector128 linearM = Vector128.MultiplyAddEstimate( + Vector128.Create(BlueToM), + linearBlue, + Vector128.MultiplyAddEstimate(Vector128.Create(GreenToM), linearGreen, Vector128.Create(RedToM) * linearRed)); + + Vector128 linearS = Vector128.MultiplyAddEstimate( + Vector128.Create(BlueToS), + linearBlue, + Vector128.MultiplyAddEstimate(Vector128.Create(GreenToS), linearGreen, Vector128.Create(RedToS) * linearRed)); + + Vector128 nonlinearL = HeifTransferFunctions.ToGamma(parameters.TransferCharacteristics, linearL); + Vector128 nonlinearM = HeifTransferFunctions.ToGamma(parameters.TransferCharacteristics, linearM); + Vector128 nonlinearS = HeifTransferFunctions.ToGamma(parameters.TransferCharacteristics, linearS); + intensity = Vector128.MultiplyAddEstimate( + Vector128.Create(SToIntensity), + nonlinearS, + Vector128.MultiplyAddEstimate(Vector128.Create(MToIntensity), nonlinearM, Vector128.Create(LToIntensity) * nonlinearL)); + + protan = Vector128.MultiplyAddEstimate( + Vector128.Create(SToProtan), + nonlinearS, + Vector128.MultiplyAddEstimate(Vector128.Create(MToProtan), nonlinearM, Vector128.Create(LToProtan) * nonlinearL)); + + tritan = Vector128.MultiplyAddEstimate( + Vector128.Create(SToTritan), + nonlinearS, + Vector128.MultiplyAddEstimate(Vector128.Create(MToTritan), nonlinearM, Vector128.Create(LToTritan) * nonlinearL)); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void ConvertFromRgb( + Vector256 red, + Vector256 green, + Vector256 blue, + in HeifColorConversionParameters parameters, + out Vector256 intensity, + out Vector256 protan, + out Vector256 tritan) + { + Vector256 linearRed = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, red); + Vector256 linearGreen = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, green); + Vector256 linearBlue = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, blue); + Vector256 linearL = Vector256.MultiplyAddEstimate( + Vector256.Create(BlueToL), + linearBlue, + Vector256.MultiplyAddEstimate(Vector256.Create(GreenToL), linearGreen, Vector256.Create(RedToL) * linearRed)); + + Vector256 linearM = Vector256.MultiplyAddEstimate( + Vector256.Create(BlueToM), + linearBlue, + Vector256.MultiplyAddEstimate(Vector256.Create(GreenToM), linearGreen, Vector256.Create(RedToM) * linearRed)); + + Vector256 linearS = Vector256.MultiplyAddEstimate( + Vector256.Create(BlueToS), + linearBlue, + Vector256.MultiplyAddEstimate(Vector256.Create(GreenToS), linearGreen, Vector256.Create(RedToS) * linearRed)); + + Vector256 nonlinearL = HeifTransferFunctions.ToGamma(parameters.TransferCharacteristics, linearL); + Vector256 nonlinearM = HeifTransferFunctions.ToGamma(parameters.TransferCharacteristics, linearM); + Vector256 nonlinearS = HeifTransferFunctions.ToGamma(parameters.TransferCharacteristics, linearS); + intensity = Vector256.MultiplyAddEstimate( + Vector256.Create(SToIntensity), + nonlinearS, + Vector256.MultiplyAddEstimate(Vector256.Create(MToIntensity), nonlinearM, Vector256.Create(LToIntensity) * nonlinearL)); + + protan = Vector256.MultiplyAddEstimate( + Vector256.Create(SToProtan), + nonlinearS, + Vector256.MultiplyAddEstimate(Vector256.Create(MToProtan), nonlinearM, Vector256.Create(LToProtan) * nonlinearL)); + + tritan = Vector256.MultiplyAddEstimate( + Vector256.Create(SToTritan), + nonlinearS, + Vector256.MultiplyAddEstimate(Vector256.Create(MToTritan), nonlinearM, Vector256.Create(LToTritan) * nonlinearL)); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void ConvertFromRgb( + Vector512 red, + Vector512 green, + Vector512 blue, + in HeifColorConversionParameters parameters, + out Vector512 intensity, + out Vector512 protan, + out Vector512 tritan) + { + Vector512 linearRed = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, red); + Vector512 linearGreen = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, green); + Vector512 linearBlue = HeifTransferFunctions.ToLinear(parameters.TransferCharacteristics, blue); + Vector512 linearL = Vector512.MultiplyAddEstimate( + Vector512.Create(BlueToL), + linearBlue, + Vector512.MultiplyAddEstimate(Vector512.Create(GreenToL), linearGreen, Vector512.Create(RedToL) * linearRed)); + + Vector512 linearM = Vector512.MultiplyAddEstimate( + Vector512.Create(BlueToM), + linearBlue, + Vector512.MultiplyAddEstimate(Vector512.Create(GreenToM), linearGreen, Vector512.Create(RedToM) * linearRed)); + + Vector512 linearS = Vector512.MultiplyAddEstimate( + Vector512.Create(BlueToS), + linearBlue, + Vector512.MultiplyAddEstimate(Vector512.Create(GreenToS), linearGreen, Vector512.Create(RedToS) * linearRed)); + + Vector512 nonlinearL = HeifTransferFunctions.ToGamma(parameters.TransferCharacteristics, linearL); + Vector512 nonlinearM = HeifTransferFunctions.ToGamma(parameters.TransferCharacteristics, linearM); + Vector512 nonlinearS = HeifTransferFunctions.ToGamma(parameters.TransferCharacteristics, linearS); + intensity = Vector512.MultiplyAddEstimate( + Vector512.Create(SToIntensity), + nonlinearS, + Vector512.MultiplyAddEstimate(Vector512.Create(MToIntensity), nonlinearM, Vector512.Create(LToIntensity) * nonlinearL)); + + protan = Vector512.MultiplyAddEstimate( + Vector512.Create(SToProtan), + nonlinearS, + Vector512.MultiplyAddEstimate(Vector512.Create(MToProtan), nonlinearM, Vector512.Create(LToProtan) * nonlinearL)); + + tritan = Vector512.MultiplyAddEstimate( + Vector512.Create(SToTritan), + nonlinearS, + Vector512.MultiplyAddEstimate(Vector512.Create(MToTritan), nonlinearM, Vector512.Create(LToTritan) * nonlinearL)); + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Components/ColorConverters/HeifColorConverter.Operator.cs b/src/ImageSharp/Formats/Heif/Components/ColorConverters/HeifColorConverter.Operator.cs new file mode 100644 index 0000000000..9b118a3112 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Components/ColorConverters/HeifColorConverter.Operator.cs @@ -0,0 +1,433 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.InteropServices; +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Components; + +/// +/// Provides the static operator contract and SIMD traversal used by HEIF color converters. Each lane carries one pixel +/// and the three vectors remain planar component rows throughout conversion. Descending vector widths consume a single +/// shared offset, preserving SIMD execution for the remainder without overlapping stores or requiring row padding. +/// +internal abstract partial class HeifColorConverterBase +{ + /// + /// Defines color-model arithmetic for scalar and SIMD lanes in both conversion directions. + /// + /// + /// Operator methods are lane-local and must preserve input order. The closed operator type lets the JIT bind the + /// matrix or lifting transform once per converter, keeping color-model dispatch outside every row loop. + /// + internal interface IHeifColorOperator + { + /// + /// Gets a value indicating whether chroma uses the luma range rather than the centered chroma range. + /// + public static abstract bool ChromaUsesLumaRange { get; } + + /// + /// Converts one normalized encoded sample to RGB. + /// + /// The first encoded component, replaced by red. + /// The second encoded component, replaced by green. + /// The third encoded component, replaced by blue. + /// The resolved H.273 conversion parameters. + public static abstract void ConvertToRgb( + ref float component0, + ref float component1, + ref float component2, + in HeifColorConversionParameters parameters); + + /// + /// Converts four normalized encoded samples to RGB. + /// + /// The first encoded component lanes, replaced by red. + /// The second encoded component lanes, replaced by green. + /// The third encoded component lanes, replaced by blue. + /// The resolved H.273 conversion parameters. + public static abstract void ConvertToRgb( + ref Vector128 component0, + ref Vector128 component1, + ref Vector128 component2, + in HeifColorConversionParameters parameters); + + /// + /// Converts eight normalized encoded samples to RGB. + /// + /// The first encoded component lanes, replaced by red. + /// The second encoded component lanes, replaced by green. + /// The third encoded component lanes, replaced by blue. + /// The resolved H.273 conversion parameters. + public static abstract void ConvertToRgb( + ref Vector256 component0, + ref Vector256 component1, + ref Vector256 component2, + in HeifColorConversionParameters parameters); + + /// + /// Converts sixteen normalized encoded samples to RGB. + /// + /// The first encoded component lanes, replaced by red. + /// The second encoded component lanes, replaced by green. + /// The third encoded component lanes, replaced by blue. + /// The resolved H.273 conversion parameters. + public static abstract void ConvertToRgb( + ref Vector512 component0, + ref Vector512 component1, + ref Vector512 component2, + in HeifColorConversionParameters parameters); + + /// + /// Converts one normalized RGB sample to encoded components. + /// + /// The normalized red component. + /// The normalized green component. + /// The normalized blue component. + /// The resolved H.273 conversion parameters. + /// The first converted component. + /// The second converted component. + /// The third converted component. + public static abstract void ConvertFromRgb( + float red, + float green, + float blue, + in HeifColorConversionParameters parameters, + out float component0, + out float component1, + out float component2); + + /// + /// Converts four normalized RGB samples to encoded components. + /// + /// The normalized red lanes. + /// The normalized green lanes. + /// The normalized blue lanes. + /// The resolved H.273 conversion parameters. + /// The first converted component lanes. + /// The second converted component lanes. + /// The third converted component lanes. + public static abstract void ConvertFromRgb( + Vector128 red, + Vector128 green, + Vector128 blue, + in HeifColorConversionParameters parameters, + out Vector128 component0, + out Vector128 component1, + out Vector128 component2); + + /// + /// Converts eight normalized RGB samples to encoded components. + /// + /// The normalized red lanes. + /// The normalized green lanes. + /// The normalized blue lanes. + /// The resolved H.273 conversion parameters. + /// The first converted component lanes. + /// The second converted component lanes. + /// The third converted component lanes. + public static abstract void ConvertFromRgb( + Vector256 red, + Vector256 green, + Vector256 blue, + in HeifColorConversionParameters parameters, + out Vector256 component0, + out Vector256 component1, + out Vector256 component2); + + /// + /// Converts sixteen normalized RGB samples to encoded components. + /// + /// The normalized red lanes. + /// The normalized green lanes. + /// The normalized blue lanes. + /// The resolved H.273 conversion parameters. + /// The first converted component lanes. + /// The second converted component lanes. + /// The third converted component lanes. + public static abstract void ConvertFromRgb( + Vector512 red, + Vector512 green, + Vector512 blue, + in HeifColorConversionParameters parameters, + out Vector512 component0, + out Vector512 component1, + out Vector512 component2); + } + + /// + /// Converts an HEIF color model using one operator-driven traversal for all SIMD widths. + /// + /// The color-model-specific arithmetic. + internal sealed class HeifColorConverter : HeifColorConverterBase + where TOperator : struct, IHeifColorOperator + { + /// + /// Initializes a new instance of the class. + /// + /// The resolved H.273 conversion parameters. + /// Whether the frame contains only luma samples. + public HeifColorConverter(in HeifColorConversionParameters parameters, bool isMonochrome) + : base(in parameters, isMonochrome) + { + } + + /// + public override float ChromaScale => TOperator.ChromaUsesLumaRange ? this.Parameters.LumaScale : this.Parameters.ChromaScale; + + /// + public override float ChromaBias => TOperator.ChromaUsesLumaRange ? this.Parameters.LumaBias : this.Parameters.ChromaBias; + + /// + public override void ConvertToRgbInPlace(Span component0, Span component1, Span component2) + { + HeifColorConversionParameters parameters = this.Parameters; + + // Row reconstruction owns equally sized planar buffers. As in JPEG, first-element byrefs let each + // SIMD width share one offset while the closed operator type keeps color-model dispatch out of the loop. + ref float component0Base = ref MemoryMarshal.GetReference(component0); + ref float component1Base = ref MemoryMarshal.GetReference(component1); + ref float component2Base = ref MemoryMarshal.GetReference(component2); + int length = component0.Length; + int i = 0; + + if (this.IsMonochrome) + { + // Monochrome has no operator arithmetic: expanding the luma range once and copying each SIMD + // vector to all three planes is cheaper than routing it through a three-component operator. + if (Vector512.IsHardwareAccelerated && i <= length - Vector512.Count) + { + Vector512 bias = Vector512.Create(parameters.LumaBias); + Vector512 scale = Vector512.Create(parameters.LumaScale); + int oneVectorFromEnd = length - Vector512.Count; + for (; i <= oneVectorFromEnd; i += Vector512.Count) + { + Vector512 value = (Unsafe.As>(ref Unsafe.Add(ref component0Base, i)) - bias) / scale; + Unsafe.As>(ref Unsafe.Add(ref component0Base, i)) = value; + Unsafe.As>(ref Unsafe.Add(ref component1Base, i)) = value; + Unsafe.As>(ref Unsafe.Add(ref component2Base, i)) = value; + } + } + + if (Vector256.IsHardwareAccelerated && i <= length - Vector256.Count) + { + Vector256 bias = Vector256.Create(parameters.LumaBias); + Vector256 scale = Vector256.Create(parameters.LumaScale); + int oneVectorFromEnd = length - Vector256.Count; + for (; i <= oneVectorFromEnd; i += Vector256.Count) + { + Vector256 value = (Unsafe.As>(ref Unsafe.Add(ref component0Base, i)) - bias) / scale; + Unsafe.As>(ref Unsafe.Add(ref component0Base, i)) = value; + Unsafe.As>(ref Unsafe.Add(ref component1Base, i)) = value; + Unsafe.As>(ref Unsafe.Add(ref component2Base, i)) = value; + } + } + + if (Vector128.IsHardwareAccelerated && i <= length - Vector128.Count) + { + Vector128 bias = Vector128.Create(parameters.LumaBias); + Vector128 scale = Vector128.Create(parameters.LumaScale); + int oneVectorFromEnd = length - Vector128.Count; + for (; i <= oneVectorFromEnd; i += Vector128.Count) + { + Vector128 value = (Unsafe.As>(ref Unsafe.Add(ref component0Base, i)) - bias) / scale; + Unsafe.As>(ref Unsafe.Add(ref component0Base, i)) = value; + Unsafe.As>(ref Unsafe.Add(ref component1Base, i)) = value; + Unsafe.As>(ref Unsafe.Add(ref component2Base, i)) = value; + } + } + + for (; i < length; i++) + { + float value = (Unsafe.Add(ref component0Base, i) - parameters.LumaBias) / parameters.LumaScale; + Unsafe.Add(ref component0Base, i) = value; + Unsafe.Add(ref component1Base, i) = value; + Unsafe.Add(ref component2Base, i) = value; + } + + return; + } + + float chromaBias = this.ChromaBias; + float chromaScale = this.ChromaScale; + + // Descending widths preserve vector execution for the remainder left by a wider register. Divide by the + // signaled ranges directly because multiplying by rounded reciprocals changes exact output-code boundaries. + if (Vector512.IsHardwareAccelerated && i <= length - Vector512.Count) + { + Vector512 lumaBias = Vector512.Create(parameters.LumaBias); + Vector512 lumaScale = Vector512.Create(parameters.LumaScale); + Vector512 chromaBiasVector = Vector512.Create(chromaBias); + Vector512 chromaScaleVector = Vector512.Create(chromaScale); + int oneVectorFromEnd = length - Vector512.Count; + for (; i <= oneVectorFromEnd; i += Vector512.Count) + { + ref Vector512 c0 = ref Unsafe.As>(ref Unsafe.Add(ref component0Base, i)); + ref Vector512 c1 = ref Unsafe.As>(ref Unsafe.Add(ref component1Base, i)); + ref Vector512 c2 = ref Unsafe.As>(ref Unsafe.Add(ref component2Base, i)); + c0 = (c0 - lumaBias) / lumaScale; + c1 = (c1 - chromaBiasVector) / chromaScaleVector; + c2 = (c2 - chromaBiasVector) / chromaScaleVector; + + TOperator.ConvertToRgb(ref c0, ref c1, ref c2, in parameters); + } + } + + if (Vector256.IsHardwareAccelerated && i <= length - Vector256.Count) + { + Vector256 lumaBias = Vector256.Create(parameters.LumaBias); + Vector256 lumaScale = Vector256.Create(parameters.LumaScale); + Vector256 chromaBiasVector = Vector256.Create(chromaBias); + Vector256 chromaScaleVector = Vector256.Create(chromaScale); + int oneVectorFromEnd = length - Vector256.Count; + for (; i <= oneVectorFromEnd; i += Vector256.Count) + { + ref Vector256 c0 = ref Unsafe.As>(ref Unsafe.Add(ref component0Base, i)); + ref Vector256 c1 = ref Unsafe.As>(ref Unsafe.Add(ref component1Base, i)); + ref Vector256 c2 = ref Unsafe.As>(ref Unsafe.Add(ref component2Base, i)); + c0 = (c0 - lumaBias) / lumaScale; + c1 = (c1 - chromaBiasVector) / chromaScaleVector; + c2 = (c2 - chromaBiasVector) / chromaScaleVector; + + TOperator.ConvertToRgb(ref c0, ref c1, ref c2, in parameters); + } + } + + if (Vector128.IsHardwareAccelerated && i <= length - Vector128.Count) + { + Vector128 lumaBias = Vector128.Create(parameters.LumaBias); + Vector128 lumaScale = Vector128.Create(parameters.LumaScale); + Vector128 chromaBiasVector = Vector128.Create(chromaBias); + Vector128 chromaScaleVector = Vector128.Create(chromaScale); + int oneVectorFromEnd = length - Vector128.Count; + for (; i <= oneVectorFromEnd; i += Vector128.Count) + { + ref Vector128 c0 = ref Unsafe.As>(ref Unsafe.Add(ref component0Base, i)); + ref Vector128 c1 = ref Unsafe.As>(ref Unsafe.Add(ref component1Base, i)); + ref Vector128 c2 = ref Unsafe.As>(ref Unsafe.Add(ref component2Base, i)); + c0 = (c0 - lumaBias) / lumaScale; + c1 = (c1 - chromaBiasVector) / chromaScaleVector; + c2 = (c2 - chromaBiasVector) / chromaScaleVector; + + TOperator.ConvertToRgb(ref c0, ref c1, ref c2, in parameters); + } + } + + // Scalar conversion is reserved for the zero-to-three samples left after the SIMD cascade. + for (; i < length; i++) + { + float c0 = (Unsafe.Add(ref component0Base, i) - parameters.LumaBias) / parameters.LumaScale; + float c1 = (Unsafe.Add(ref component1Base, i) - chromaBias) / chromaScale; + float c2 = (Unsafe.Add(ref component2Base, i) - chromaBias) / chromaScale; + TOperator.ConvertToRgb(ref c0, ref c1, ref c2, in parameters); + Unsafe.Add(ref component0Base, i) = c0; + Unsafe.Add(ref component1Base, i) = c1; + Unsafe.Add(ref component2Base, i) = c2; + } + } + + /// + public override void ConvertFromRgbInPlace( + Span component0, + Span component1, + Span component2, + float maximumValue) + { + HeifColorConversionParameters parameters = this.Parameters; + + // The unpacker supplies three planar RGB rows. These same buffers become the destination component + // rows after each operator call, so encoding retains JPEG's planar contract without another allocation. + ref float component0Base = ref MemoryMarshal.GetReference(component0); + ref float component1Base = ref MemoryMarshal.GetReference(component1); + ref float component2Base = ref MemoryMarshal.GetReference(component2); + int length = component0.Length; + int i = 0; + + // RGB normalization is part of the vector load, and each operator returns planar components through + // out parameters. This is the same input/output shape used by JPEG's encoder-side color operators. + if (Vector512.IsHardwareAccelerated) + { + int oneVectorFromEnd = length - Vector512.Count; + + if (i <= oneVectorFromEnd) + { + Vector512 inverseMaximum = Vector512.Create(1F / maximumValue); + + for (; i <= oneVectorFromEnd; i += Vector512.Count) + { + Vector512 red = Unsafe.As>(ref Unsafe.Add(ref component0Base, i)) * inverseMaximum; + Vector512 green = Unsafe.As>(ref Unsafe.Add(ref component1Base, i)) * inverseMaximum; + Vector512 blue = Unsafe.As>(ref Unsafe.Add(ref component2Base, i)) * inverseMaximum; + ref Vector512 c0 = ref Unsafe.As>(ref Unsafe.Add(ref component0Base, i)); + ref Vector512 c1 = ref Unsafe.As>(ref Unsafe.Add(ref component1Base, i)); + ref Vector512 c2 = ref Unsafe.As>(ref Unsafe.Add(ref component2Base, i)); + + TOperator.ConvertFromRgb(red, green, blue, in parameters, out c0, out c1, out c2); + } + } + } + + if (Vector256.IsHardwareAccelerated) + { + int oneVectorFromEnd = length - Vector256.Count; + + if (i <= oneVectorFromEnd) + { + Vector256 inverseMaximum = Vector256.Create(1F / maximumValue); + + for (; i <= oneVectorFromEnd; i += Vector256.Count) + { + Vector256 red = Unsafe.As>(ref Unsafe.Add(ref component0Base, i)) * inverseMaximum; + Vector256 green = Unsafe.As>(ref Unsafe.Add(ref component1Base, i)) * inverseMaximum; + Vector256 blue = Unsafe.As>(ref Unsafe.Add(ref component2Base, i)) * inverseMaximum; + ref Vector256 c0 = ref Unsafe.As>(ref Unsafe.Add(ref component0Base, i)); + ref Vector256 c1 = ref Unsafe.As>(ref Unsafe.Add(ref component1Base, i)); + ref Vector256 c2 = ref Unsafe.As>(ref Unsafe.Add(ref component2Base, i)); + + TOperator.ConvertFromRgb(red, green, blue, in parameters, out c0, out c1, out c2); + } + } + } + + if (Vector128.IsHardwareAccelerated) + { + int oneVectorFromEnd = length - Vector128.Count; + + if (i <= oneVectorFromEnd) + { + Vector128 inverseMaximum = Vector128.Create(1F / maximumValue); + + for (; i <= oneVectorFromEnd; i += Vector128.Count) + { + Vector128 red = Unsafe.As>(ref Unsafe.Add(ref component0Base, i)) * inverseMaximum; + Vector128 green = Unsafe.As>(ref Unsafe.Add(ref component1Base, i)) * inverseMaximum; + Vector128 blue = Unsafe.As>(ref Unsafe.Add(ref component2Base, i)) * inverseMaximum; + ref Vector128 c0 = ref Unsafe.As>(ref Unsafe.Add(ref component0Base, i)); + ref Vector128 c1 = ref Unsafe.As>(ref Unsafe.Add(ref component1Base, i)); + ref Vector128 c2 = ref Unsafe.As>(ref Unsafe.Add(ref component2Base, i)); + + TOperator.ConvertFromRgb(red, green, blue, in parameters, out c0, out c1, out c2); + } + } + } + + float inverseMaximumScalar = 1F / maximumValue; + + // The shared offset leaves at most three samples for the scalar fallback on SIMD-capable systems. + for (; i < length; i++) + { + float red = Unsafe.Add(ref component0Base, i) * inverseMaximumScalar; + float green = Unsafe.Add(ref component1Base, i) * inverseMaximumScalar; + float blue = Unsafe.Add(ref component2Base, i) * inverseMaximumScalar; + + TOperator.ConvertFromRgb(red, green, blue, in parameters, out float c0, out float c1, out float c2); + + Unsafe.Add(ref component0Base, i) = c0; + Unsafe.Add(ref component1Base, i) = c1; + Unsafe.Add(ref component2Base, i) = c2; + } + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Components/ColorConverters/HeifColorConverter.Smpte2085Operator.cs b/src/ImageSharp/Formats/Heif/Components/ColorConverters/HeifColorConverter.Smpte2085Operator.cs new file mode 100644 index 0000000000..451c167121 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Components/ColorConverters/HeifColorConverter.Smpte2085Operator.cs @@ -0,0 +1,146 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Components; + +/// +/// Provides SMPTE ST 2085 YDzDx conversion for scalar and SIMD lanes. +/// +internal abstract partial class HeifColorConverterBase +{ + /// + /// Implements SMPTE ST 2085 YDzDx conversion for scalar and SIMD lanes. + /// + internal readonly struct HeifSmpte2085ColorOperator : IHeifColorOperator + { + /// + /// The SMPTE ST 2085 blue primary normalization factor. + /// + public const float BlueNormalization = 0.986566F; + + /// + /// The SMPTE ST 2085 green contribution to the red primary. + /// + public const float RedGreenContribution = 0.991902F; + + /// + public static bool ChromaUsesLumaRange => false; + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void ConvertToRgb(ref float y, ref float dz, ref float dx, in HeifColorConversionParameters parameters) + { + // The encoded YDzDx planes carry green directly. The two difference planes restore blue and red. + float g = y; + float b = ((2F * dz) + y) / BlueNormalization; + float r = (2F * dx) + (RedGreenContribution * y); + y = r; + dz = g; + dx = b; + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void ConvertToRgb(ref Vector128 y, ref Vector128 dz, ref Vector128 dx, in HeifColorConversionParameters parameters) + { + Vector128 g = y; + Vector128 b = Vector128.MultiplyAddEstimate(Vector128.Create(2F), dz, y) / Vector128.Create(BlueNormalization); + Vector128 r = Vector128.MultiplyAddEstimate(Vector128.Create(2F), dx, Vector128.Create(RedGreenContribution) * y); + y = r; + dz = g; + dx = b; + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void ConvertToRgb(ref Vector256 y, ref Vector256 dz, ref Vector256 dx, in HeifColorConversionParameters parameters) + { + Vector256 g = y; + Vector256 b = Vector256.MultiplyAddEstimate(Vector256.Create(2F), dz, y) / Vector256.Create(BlueNormalization); + Vector256 r = Vector256.MultiplyAddEstimate(Vector256.Create(2F), dx, Vector256.Create(RedGreenContribution) * y); + y = r; + dz = g; + dx = b; + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void ConvertToRgb(ref Vector512 y, ref Vector512 dz, ref Vector512 dx, in HeifColorConversionParameters parameters) + { + Vector512 g = y; + Vector512 b = Vector512.MultiplyAddEstimate(Vector512.Create(2F), dz, y) / Vector512.Create(BlueNormalization); + Vector512 r = Vector512.MultiplyAddEstimate(Vector512.Create(2F), dx, Vector512.Create(RedGreenContribution) * y); + y = r; + dz = g; + dx = b; + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void ConvertFromRgb( + float r, + float g, + float b, + in HeifColorConversionParameters parameters, + out float y, + out float dz, + out float dx) + { + // Y is the green primary; Dz and Dx are half-scaled blue and red differences. + y = g; + dz = ((BlueNormalization * b) - y) * 0.5F; + dx = (r - (RedGreenContribution * y)) * 0.5F; + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void ConvertFromRgb( + Vector128 r, + Vector128 g, + Vector128 b, + in HeifColorConversionParameters parameters, + out Vector128 y, + out Vector128 dz, + out Vector128 dx) + { + y = g; + dz = Vector128.Create(0.5F) * Vector128.MultiplyAddEstimate(Vector128.Create(BlueNormalization), b, -y); + dx = Vector128.Create(0.5F) * Vector128.MultiplyAddEstimate(Vector128.Create(-RedGreenContribution), y, r); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void ConvertFromRgb( + Vector256 r, + Vector256 g, + Vector256 b, + in HeifColorConversionParameters parameters, + out Vector256 y, + out Vector256 dz, + out Vector256 dx) + { + y = g; + dz = Vector256.Create(0.5F) * Vector256.MultiplyAddEstimate(Vector256.Create(BlueNormalization), b, -y); + dx = Vector256.Create(0.5F) * Vector256.MultiplyAddEstimate(Vector256.Create(-RedGreenContribution), y, r); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void ConvertFromRgb( + Vector512 r, + Vector512 g, + Vector512 b, + in HeifColorConversionParameters parameters, + out Vector512 y, + out Vector512 dz, + out Vector512 dx) + { + y = g; + dz = Vector512.Create(0.5F) * Vector512.MultiplyAddEstimate(Vector512.Create(BlueNormalization), b, -y); + dx = Vector512.Create(0.5F) * Vector512.MultiplyAddEstimate(Vector512.Create(-RedGreenContribution), y, r); + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Components/ColorConverters/HeifColorConverter.YCgCoOperator.cs b/src/ImageSharp/Formats/Heif/Components/ColorConverters/HeifColorConverter.YCgCoOperator.cs new file mode 100644 index 0000000000..414985887b --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Components/ColorConverters/HeifColorConverter.YCgCoOperator.cs @@ -0,0 +1,144 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Components; + +/// +/// Provides YCgCo matrix conversion for scalar and SIMD lanes. +/// +internal abstract partial class HeifColorConverterBase +{ + /// + /// Implements YCgCo conversion for scalar and SIMD lanes. + /// + internal readonly struct HeifYCgCoColorOperator : IHeifColorOperator + { + /// + public static bool ChromaUsesLumaRange => false; + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void ConvertToRgb(ref float y, ref float cg, ref float co, in HeifColorConversionParameters parameters) + { + // Reusing Y - Cg for both outer primaries keeps the inverse transform to four additions. + float temporary = y - cg; + float r = temporary + co; + float g = y + cg; + float b = temporary - co; + y = r; + cg = g; + co = b; + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void ConvertToRgb(ref Vector128 y, ref Vector128 cg, ref Vector128 co, in HeifColorConversionParameters parameters) + { + Vector128 temporary = y - cg; + Vector128 r = temporary + co; + Vector128 g = y + cg; + Vector128 b = temporary - co; + y = r; + cg = g; + co = b; + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void ConvertToRgb(ref Vector256 y, ref Vector256 cg, ref Vector256 co, in HeifColorConversionParameters parameters) + { + Vector256 temporary = y - cg; + Vector256 r = temporary + co; + Vector256 g = y + cg; + Vector256 b = temporary - co; + y = r; + cg = g; + co = b; + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void ConvertToRgb(ref Vector512 y, ref Vector512 cg, ref Vector512 co, in HeifColorConversionParameters parameters) + { + Vector512 temporary = y - cg; + Vector512 r = temporary + co; + Vector512 g = y + cg; + Vector512 b = temporary - co; + y = r; + cg = g; + co = b; + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void ConvertFromRgb( + float r, + float g, + float b, + in HeifColorConversionParameters parameters, + out float y, + out float cg, + out float co) + { + // R + B is shared by Y and Cg, while Co is the half-scaled red/blue difference. + float sum = r + b; + y = (0.5F * g) + (0.25F * sum); + cg = (0.5F * g) - (0.25F * sum); + co = 0.5F * (r - b); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void ConvertFromRgb( + Vector128 r, + Vector128 g, + Vector128 b, + in HeifColorConversionParameters parameters, + out Vector128 y, + out Vector128 cg, + out Vector128 co) + { + Vector128 sum = r + b; + y = Vector128.MultiplyAddEstimate(Vector128.Create(0.5F), g, Vector128.Create(0.25F) * sum); + cg = Vector128.MultiplyAddEstimate(Vector128.Create(0.5F), g, Vector128.Create(-0.25F) * sum); + co = Vector128.Create(0.5F) * (r - b); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void ConvertFromRgb( + Vector256 r, + Vector256 g, + Vector256 b, + in HeifColorConversionParameters parameters, + out Vector256 y, + out Vector256 cg, + out Vector256 co) + { + Vector256 sum = r + b; + y = Vector256.MultiplyAddEstimate(Vector256.Create(0.5F), g, Vector256.Create(0.25F) * sum); + cg = Vector256.MultiplyAddEstimate(Vector256.Create(0.5F), g, Vector256.Create(-0.25F) * sum); + co = Vector256.Create(0.5F) * (r - b); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void ConvertFromRgb( + Vector512 r, + Vector512 g, + Vector512 b, + in HeifColorConversionParameters parameters, + out Vector512 y, + out Vector512 cg, + out Vector512 co) + { + Vector512 sum = r + b; + y = Vector512.MultiplyAddEstimate(Vector512.Create(0.5F), g, Vector512.Create(0.25F) * sum); + cg = Vector512.MultiplyAddEstimate(Vector512.Create(0.5F), g, Vector512.Create(-0.25F) * sum); + co = Vector512.Create(0.5F) * (r - b); + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Components/ColorConverters/HeifColorConverter.YCgCoReversibleOperator.cs b/src/ImageSharp/Formats/Heif/Components/ColorConverters/HeifColorConverter.YCgCoReversibleOperator.cs new file mode 100644 index 0000000000..015ffdc4c1 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Components/ColorConverters/HeifColorConverter.YCgCoReversibleOperator.cs @@ -0,0 +1,225 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Components; + +/// +/// Provides reversible YCgCo integer lifting for scalar and SIMD lanes. +/// +internal abstract partial class HeifColorConverterBase +{ + /// + /// Implements the YCgCo-Re and YCgCo-Ro integer lifting transforms for scalar and SIMD lanes. + /// + internal readonly struct HeifYCgCoReversibleColorOperator : IHeifColorOperator + { + /// + public static bool ChromaUsesLumaRange => false; + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void ConvertToRgb(ref float y, ref float cg, ref float co, in HeifColorConversionParameters parameters) + { + int yCode = (int)MathF.Floor((y * parameters.LumaSampleMaximum) + 0.5F); + int cgCode = (int)MathF.Floor((cg * parameters.LumaSampleMaximum) + 0.5F); + int coCode = (int)MathF.Floor((co * parameters.LumaSampleMaximum) + 0.5F); + + // Signed arithmetic shifts are part of the reversible lifting definition. In particular, they preserve + // the specified floor division for negative odd Cg and Co values instead of truncating toward zero. + int temporary = yCode - (cgCode >> 1); + int greenCode = Numerics.Clamp(temporary + cgCode, 0, (int)parameters.RgbSampleMaximum); + int blueCode = Numerics.Clamp(temporary - (coCode >> 1), 0, (int)parameters.RgbSampleMaximum); + int redCode = Numerics.Clamp(blueCode + coCode, 0, (int)parameters.RgbSampleMaximum); + float inverseRgbScale = 1F / parameters.RgbScale; + y = (redCode - parameters.RgbBias) * inverseRgbScale; + cg = (greenCode - parameters.RgbBias) * inverseRgbScale; + co = (blueCode - parameters.RgbBias) * inverseRgbScale; + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void ConvertToRgb( + ref Vector128 y, + ref Vector128 cg, + ref Vector128 co, + in HeifColorConversionParameters parameters) + { + Vector128 encodedMaximum = Vector128.Create(parameters.LumaSampleMaximum); + Vector128 half = Vector128.Create(0.5F); + Vector128 yCode = Vector128.ConvertToInt32(Vector128.Floor((y * encodedMaximum) + half)); + Vector128 cgCode = Vector128.ConvertToInt32(Vector128.Floor((cg * encodedMaximum) + half)); + Vector128 coCode = Vector128.ConvertToInt32(Vector128.Floor((co * encodedMaximum) + half)); + + // Integer lanes preserve the normative arithmetic shifts; converting the lifting stages back to + // floating point would change negative odd Cg and Co values and break reversibility. + Vector128 temporary = yCode - Vector128.ShiftRightArithmetic(cgCode, 1); + Vector128 zero = Vector128.Zero; + Vector128 rgbMaximum = Vector128.Create((int)parameters.RgbSampleMaximum); + Vector128 greenCode = Vector128.Min(Vector128.Max(temporary + cgCode, zero), rgbMaximum); + Vector128 blueCode = Vector128.Min(Vector128.Max(temporary - Vector128.ShiftRightArithmetic(coCode, 1), zero), rgbMaximum); + Vector128 redCode = Vector128.Min(Vector128.Max(blueCode + coCode, zero), rgbMaximum); + Vector128 rgbBias = Vector128.Create(parameters.RgbBias); + Vector128 inverseRgbScale = Vector128.Create(1F / parameters.RgbScale); + y = (Vector128.ConvertToSingle(redCode) - rgbBias) * inverseRgbScale; + cg = (Vector128.ConvertToSingle(greenCode) - rgbBias) * inverseRgbScale; + co = (Vector128.ConvertToSingle(blueCode) - rgbBias) * inverseRgbScale; + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void ConvertToRgb( + ref Vector256 y, + ref Vector256 cg, + ref Vector256 co, + in HeifColorConversionParameters parameters) + { + Vector256 encodedMaximum = Vector256.Create(parameters.LumaSampleMaximum); + Vector256 half = Vector256.Create(0.5F); + Vector256 yCode = Vector256.ConvertToInt32(Vector256.Floor((y * encodedMaximum) + half)); + Vector256 cgCode = Vector256.ConvertToInt32(Vector256.Floor((cg * encodedMaximum) + half)); + Vector256 coCode = Vector256.ConvertToInt32(Vector256.Floor((co * encodedMaximum) + half)); + Vector256 temporary = yCode - Vector256.ShiftRightArithmetic(cgCode, 1); + Vector256 zero = Vector256.Zero; + Vector256 rgbMaximum = Vector256.Create((int)parameters.RgbSampleMaximum); + Vector256 greenCode = Vector256.Min(Vector256.Max(temporary + cgCode, zero), rgbMaximum); + Vector256 blueCode = Vector256.Min(Vector256.Max(temporary - Vector256.ShiftRightArithmetic(coCode, 1), zero), rgbMaximum); + Vector256 redCode = Vector256.Min(Vector256.Max(blueCode + coCode, zero), rgbMaximum); + Vector256 rgbBias = Vector256.Create(parameters.RgbBias); + Vector256 inverseRgbScale = Vector256.Create(1F / parameters.RgbScale); + y = (Vector256.ConvertToSingle(redCode) - rgbBias) * inverseRgbScale; + cg = (Vector256.ConvertToSingle(greenCode) - rgbBias) * inverseRgbScale; + co = (Vector256.ConvertToSingle(blueCode) - rgbBias) * inverseRgbScale; + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void ConvertToRgb( + ref Vector512 y, + ref Vector512 cg, + ref Vector512 co, + in HeifColorConversionParameters parameters) + { + Vector512 encodedMaximum = Vector512.Create(parameters.LumaSampleMaximum); + Vector512 half = Vector512.Create(0.5F); + Vector512 yCode = Vector512.ConvertToInt32(Vector512.Floor((y * encodedMaximum) + half)); + Vector512 cgCode = Vector512.ConvertToInt32(Vector512.Floor((cg * encodedMaximum) + half)); + Vector512 coCode = Vector512.ConvertToInt32(Vector512.Floor((co * encodedMaximum) + half)); + Vector512 temporary = yCode - Vector512.ShiftRightArithmetic(cgCode, 1); + Vector512 zero = Vector512.Zero; + Vector512 rgbMaximum = Vector512.Create((int)parameters.RgbSampleMaximum); + Vector512 greenCode = Vector512.Min(Vector512.Max(temporary + cgCode, zero), rgbMaximum); + Vector512 blueCode = Vector512.Min(Vector512.Max(temporary - Vector512.ShiftRightArithmetic(coCode, 1), zero), rgbMaximum); + Vector512 redCode = Vector512.Min(Vector512.Max(blueCode + coCode, zero), rgbMaximum); + Vector512 rgbBias = Vector512.Create(parameters.RgbBias); + Vector512 inverseRgbScale = Vector512.Create(1F / parameters.RgbScale); + y = (Vector512.ConvertToSingle(redCode) - rgbBias) * inverseRgbScale; + cg = (Vector512.ConvertToSingle(greenCode) - rgbBias) * inverseRgbScale; + co = (Vector512.ConvertToSingle(blueCode) - rgbBias) * inverseRgbScale; + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void ConvertFromRgb( + float red, + float green, + float blue, + in HeifColorConversionParameters parameters, + out float y, + out float cg, + out float co) + { + int redCode = (int)MathF.Floor((red * parameters.RgbScale) + parameters.RgbBias + 0.5F); + int greenCode = (int)MathF.Floor((green * parameters.RgbScale) + parameters.RgbBias + 0.5F); + int blueCode = (int)MathF.Floor((blue * parameters.RgbScale) + parameters.RgbBias + 0.5F); + int coCode = redCode - blueCode; + int temporary = blueCode + (coCode >> 1); + int cgCode = greenCode - temporary; + int yCode = temporary + (cgCode >> 1); + float inverseEncodedMaximum = 1F / parameters.LumaSampleMaximum; + y = yCode * inverseEncodedMaximum; + cg = cgCode * inverseEncodedMaximum; + co = coCode * inverseEncodedMaximum; + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void ConvertFromRgb( + Vector128 red, + Vector128 green, + Vector128 blue, + in HeifColorConversionParameters parameters, + out Vector128 y, + out Vector128 cg, + out Vector128 co) + { + Vector128 rgbScale = Vector128.Create(parameters.RgbScale); + Vector128 quantizationOffset = Vector128.Create(parameters.RgbBias + 0.5F); + Vector128 redCode = Vector128.ConvertToInt32(Vector128.Floor((red * rgbScale) + quantizationOffset)); + Vector128 greenCode = Vector128.ConvertToInt32(Vector128.Floor((green * rgbScale) + quantizationOffset)); + Vector128 blueCode = Vector128.ConvertToInt32(Vector128.Floor((blue * rgbScale) + quantizationOffset)); + Vector128 coCode = redCode - blueCode; + Vector128 temporary = blueCode + Vector128.ShiftRightArithmetic(coCode, 1); + Vector128 cgCode = greenCode - temporary; + Vector128 yCode = temporary + Vector128.ShiftRightArithmetic(cgCode, 1); + Vector128 inverseEncodedMaximum = Vector128.Create(1F / parameters.LumaSampleMaximum); + y = Vector128.ConvertToSingle(yCode) * inverseEncodedMaximum; + cg = Vector128.ConvertToSingle(cgCode) * inverseEncodedMaximum; + co = Vector128.ConvertToSingle(coCode) * inverseEncodedMaximum; + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void ConvertFromRgb( + Vector256 red, + Vector256 green, + Vector256 blue, + in HeifColorConversionParameters parameters, + out Vector256 y, + out Vector256 cg, + out Vector256 co) + { + Vector256 rgbScale = Vector256.Create(parameters.RgbScale); + Vector256 quantizationOffset = Vector256.Create(parameters.RgbBias + 0.5F); + Vector256 redCode = Vector256.ConvertToInt32(Vector256.Floor((red * rgbScale) + quantizationOffset)); + Vector256 greenCode = Vector256.ConvertToInt32(Vector256.Floor((green * rgbScale) + quantizationOffset)); + Vector256 blueCode = Vector256.ConvertToInt32(Vector256.Floor((blue * rgbScale) + quantizationOffset)); + Vector256 coCode = redCode - blueCode; + Vector256 temporary = blueCode + Vector256.ShiftRightArithmetic(coCode, 1); + Vector256 cgCode = greenCode - temporary; + Vector256 yCode = temporary + Vector256.ShiftRightArithmetic(cgCode, 1); + Vector256 inverseEncodedMaximum = Vector256.Create(1F / parameters.LumaSampleMaximum); + y = Vector256.ConvertToSingle(yCode) * inverseEncodedMaximum; + cg = Vector256.ConvertToSingle(cgCode) * inverseEncodedMaximum; + co = Vector256.ConvertToSingle(coCode) * inverseEncodedMaximum; + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void ConvertFromRgb( + Vector512 red, + Vector512 green, + Vector512 blue, + in HeifColorConversionParameters parameters, + out Vector512 y, + out Vector512 cg, + out Vector512 co) + { + Vector512 rgbScale = Vector512.Create(parameters.RgbScale); + Vector512 quantizationOffset = Vector512.Create(parameters.RgbBias + 0.5F); + Vector512 redCode = Vector512.ConvertToInt32(Vector512.Floor((red * rgbScale) + quantizationOffset)); + Vector512 greenCode = Vector512.ConvertToInt32(Vector512.Floor((green * rgbScale) + quantizationOffset)); + Vector512 blueCode = Vector512.ConvertToInt32(Vector512.Floor((blue * rgbScale) + quantizationOffset)); + Vector512 coCode = redCode - blueCode; + Vector512 temporary = blueCode + Vector512.ShiftRightArithmetic(coCode, 1); + Vector512 cgCode = greenCode - temporary; + Vector512 yCode = temporary + Vector512.ShiftRightArithmetic(cgCode, 1); + Vector512 inverseEncodedMaximum = Vector512.Create(1F / parameters.LumaSampleMaximum); + y = Vector512.ConvertToSingle(yCode) * inverseEncodedMaximum; + cg = Vector512.ConvertToSingle(cgCode) * inverseEncodedMaximum; + co = Vector512.ConvertToSingle(coCode) * inverseEncodedMaximum; + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Components/ColorConverters/HeifColorConverter.cs b/src/ImageSharp/Formats/Heif/Components/ColorConverters/HeifColorConverter.cs new file mode 100644 index 0000000000..887d00b9c9 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Components/ColorConverters/HeifColorConverter.cs @@ -0,0 +1,138 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Components; + +/// +/// Identifies the H.273 matrix operation used between encoded planes and RGB components. +/// +internal enum HeifColorConversionMode +{ + /// + /// A coefficient-based YCbCr matrix conversion. + /// + Coefficients, + + /// + /// Direct G, B, and R component mapping from the Y, U, and V planes. + /// + Identity, + + /// + /// The reversible-style YCgCo color transform. + /// + YCgCo, + + /// + /// The SMPTE ST 2085 YDzDx color transform. + /// + Smpte2085, + + /// + /// A constant-luminance transform using the signaled transfer characteristics. + /// + ConstantLuminance, + + /// + /// The BT.2100 ICtCp color transform. + /// + ICtCp, + + /// + /// The IPT-C2 color transform. + /// + IptC2, + + /// + /// The YCgCo-Re or YCgCo-Ro reversible color transform. + /// + YCgCoReversible, +} + +/// +/// Converts normalized component planes between an encoded HEIF color model and RGB. +/// +internal abstract partial class HeifColorConverterBase +{ + /// + /// Initializes a new instance of the class. + /// + /// The resolved H.273 conversion parameters. + /// Whether the frame contains only luma samples. + protected HeifColorConverterBase(in HeifColorConversionParameters parameters, bool isMonochrome) + { + this.Parameters = parameters; + this.IsMonochrome = isMonochrome; + } + + /// + /// Gets the resolved H.273 conversion parameters. + /// + protected HeifColorConversionParameters Parameters { get; } + + /// + /// Gets a value indicating whether the frame contains only luma samples. + /// + protected bool IsMonochrome { get; } + + /// + /// Gets the scale used to encode luma components. + /// + public float LumaScale => this.Parameters.LumaScale; + + /// + /// Gets the bias used to encode luma components. + /// + public float LumaBias => this.Parameters.LumaBias; + + /// + /// Gets the scale used to encode chroma components. + /// + public abstract float ChromaScale { get; } + + /// + /// Gets the bias used to encode chroma components. + /// + public abstract float ChromaBias { get; } + + /// + /// Converts normalized encoded components to normalized RGB in place. + /// + /// The luma or first color component, replaced by red. + /// The first chroma or second color component, replaced by green. + /// The second chroma or third color component, replaced by blue. + public abstract void ConvertToRgbInPlace(Span component0, Span component1, Span component2); + + /// + /// Converts normalized RGB components to normalized encoded components in place. + /// + /// The red component, replaced by luma or the first color component. + /// The green component, replaced by the first chroma or second color component. + /// The blue component, replaced by the second chroma or third color component. + /// The largest value in the RGB component planes. + public abstract void ConvertFromRgbInPlace( + Span component0, + Span component1, + Span component2, + float maximumValue); + + /// + /// Creates the converter selected by the resolved H.273 matrix and transfer characteristics. + /// + /// The resolved matrix conversion mode. + /// The resolved H.273 conversion parameters. + /// Whether the frame contains only luma samples. + /// The selected converter. + public static HeifColorConverterBase Create(HeifColorConversionMode mode, in HeifColorConversionParameters parameters, bool isMonochrome) + => mode switch + { + HeifColorConversionMode.Identity => new HeifColorConverter(in parameters, isMonochrome), + HeifColorConversionMode.YCgCo => new HeifColorConverter(in parameters, isMonochrome), + HeifColorConversionMode.Smpte2085 => new HeifColorConverter(in parameters, isMonochrome), + HeifColorConversionMode.ConstantLuminance => new HeifColorConverter(in parameters, isMonochrome), + HeifColorConversionMode.ICtCp => new HeifColorConverter(in parameters, isMonochrome), + HeifColorConversionMode.IptC2 => new HeifColorConverter(in parameters, isMonochrome), + HeifColorConversionMode.YCgCoReversible => new HeifColorConverter(in parameters, isMonochrome), + _ => new HeifColorConverter(in parameters, isMonochrome), + }; +} diff --git a/src/ImageSharp/Formats/Heif/Components/ColorConverters/HeifPlanarColorConverter.cs b/src/ImageSharp/Formats/Heif/Components/ColorConverters/HeifPlanarColorConverter.cs new file mode 100644 index 0000000000..472abc44a3 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Components/ColorConverters/HeifPlanarColorConverter.cs @@ -0,0 +1,871 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Buffers; +using System.Runtime.InteropServices; +using SixLabors.ImageSharp.Memory; +using SixLabors.ImageSharp.PixelFormats; + +namespace SixLabors.ImageSharp.Formats.Heif.Components; + +/// +/// Converts between native HEIF component planes and packed ImageSharp pixels. +/// +internal static class HeifPlanarColorConverter +{ + /// + /// The number of planar color components retained for each source row. + /// + private const int ColorComponentCount = 3; + + /// + /// The number of source rows consumed together by vertically subsampled chroma. + /// + private const int VerticallySubsampledRowCount = 2; + + /// + /// The largest value represented by an eight-bit packed RGB component. + /// + private const float ByteMaximum = byte.MaxValue; + + /// + /// The largest value represented by a 16-bit packed RGB component. + /// + private const float UShortMaximum = ushort.MaxValue; + + /// + /// Converts complete native unsigned 16-bit component planes to packed pixels. + /// + public static void ConvertToRgb( + Configuration configuration, + TBuffer buffer, + ImageFrame image, + in HeifColorConversionParameters parameters, + HeifColorConversionMode mode) + where TPixel : unmanaged, IPixel + where TBuffer : struct, IHeifPlanarSampleBuffer + => ConvertToRgb(configuration, buffer, image, in parameters, mode, 0, 0); + + /// + /// Converts a region of native unsigned 16-bit component storage to packed pixels and selects eligible exact integer kernels. + /// + /// The destination pixel type. + /// The codec adapter exposing the native component planes. + /// The configuration used for allocation and pixel conversion. + /// The native component-plane buffer. + /// The destination image frame. + /// The resolved H.273 conversion parameters. + /// The resolved H.273 conversion mode. + /// The horizontal luma-sample offset of the first converted pixel. + /// The vertical luma-sample offset of the first converted pixel. + public static void ConvertToRgb( + Configuration configuration, + TBuffer buffer, + ImageFrame image, + in HeifColorConversionParameters parameters, + HeifColorConversionMode mode, + int sourceX, + int sourceY) + where TPixel : unmanaged, IPixel + where TBuffer : struct, IHeifPlanarSampleBuffer + { + if (HeifYuvToRgb8Converter.SupportsFixedPointConversion( + buffer.ChromaSubsamplingX, + buffer.ChromaSubsamplingY, + buffer.LumaBitDepth, + buffer.ChromaBitDepth, + parameters.IsFullRange, + parameters.MatrixCoefficients, + mode)) + { + // The fixed-point operator preserves the exact code-value rounding used by the verified eight-bit + // presentation path. Selection belongs here so no codec can acquire a private color-conversion route. + HeifYuvToRgb8Converter.ConvertFixedPoint(configuration, buffer, image, in parameters, sourceX, sourceY); + return; + } + + ConvertToRgb( + configuration, + buffer, + image, + in parameters, + mode, + sourceX, + sourceY); + } + + /// + /// Converts complete native component planes to packed pixels. + /// + public static void ConvertToRgb( + Configuration configuration, + TBuffer buffer, + ImageFrame image, + in HeifColorConversionParameters parameters, + HeifColorConversionMode mode) + where TPixel : unmanaged, IPixel + where TBuffer : struct, IHeifPlanarSampleBuffer + where TSample : unmanaged + where TLoader : struct, IHeifSampleConverter + => ConvertToRgb(configuration, buffer, image, in parameters, mode, 0, 0); + + /// + /// Converts a region of native component planes to packed pixels. + /// + /// The destination pixel type. + /// The codec adapter exposing the native component planes. + /// The native unsigned sample storage type. + /// The SIMD widening operations for the sample type. + /// The configuration used for allocation and pixel conversion. + /// The native component-plane buffer. + /// The destination image frame. + /// The resolved H.273 conversion parameters. + /// The resolved H.273 conversion mode. + /// The horizontal luma-sample offset of the first converted pixel. + /// The vertical luma-sample offset of the first converted pixel. + public static void ConvertToRgb( + Configuration configuration, + TBuffer buffer, + ImageFrame image, + in HeifColorConversionParameters parameters, + HeifColorConversionMode mode, + int sourceX, + int sourceY) + where TPixel : unmanaged, IPixel + where TBuffer : struct, IHeifPlanarSampleBuffer + where TSample : unmanaged + where TLoader : struct, IHeifSampleConverter + { + HeifColorConverterBase colorConverter = HeifColorConverterBase.Create(mode, in parameters, buffer.IsMonochrome); + YuvToRgbRowConverter converter = new( + configuration, + buffer, + image, + colorConverter, + sourceX, + sourceY); + + using IMemoryOwner scratchOwner = configuration.MemoryAllocator.Allocate(converter.BufferLength); + Span scratch = scratchOwner.GetSpan(); + for (int y = 0; y < image.Height; y++) + { + converter.Convert(y, scratch); + } + } + + /// + /// Converts packed pixels to native component planes. + /// + /// The source pixel type. + /// The codec adapter exposing the native component planes. + /// The native unsigned sample storage type. + /// The SIMD narrowing and storage operations for the sample type. + /// The configuration used for allocation and pixel conversion. + /// The source image frame. + /// The destination component-plane buffer. + /// The resolved H.273 conversion parameters. + /// The resolved H.273 conversion mode. + public static void ConvertFromRgb( + Configuration configuration, + ImageFrame image, + TBuffer buffer, + in HeifColorConversionParameters parameters, + HeifColorConversionMode mode) + where TPixel : unmanaged, IPixel + where TBuffer : struct, IHeifPlanarSampleBuffer + where TSample : unmanaged + where TStorer : struct, IHeifSampleConverter + { + Rectangle sourceRectangle = new(0, 0, image.Width, image.Height); + ConvertFromRgb( + configuration, + image, + sourceRectangle, + buffer, + in parameters, + mode); + } + + /// + /// Converts a rectangular packed-pixel region to native component planes. + /// + /// The source pixel type. + /// The codec adapter exposing the native component planes. + /// The native unsigned sample storage type. + /// The SIMD narrowing and storage operations for the sample type. + /// The configuration used for allocation and pixel conversion. + /// The source image frame. + /// The source region mapped to the complete destination buffer. + /// The destination component-plane buffer. + /// The resolved H.273 conversion parameters. + /// The resolved H.273 conversion mode. + public static void ConvertFromRgb( + Configuration configuration, + ImageFrame image, + Rectangle sourceRectangle, + TBuffer buffer, + in HeifColorConversionParameters parameters, + HeifColorConversionMode mode) + where TPixel : unmanaged, IPixel + where TBuffer : struct, IHeifPlanarSampleBuffer + where TSample : unmanaged + where TStorer : struct, IHeifSampleConverter + { + HeifColorConverterBase colorConverter = HeifColorConverterBase.Create(mode, in parameters, buffer.IsMonochrome); + RgbToYuvRowConverter converter = new( + configuration, + buffer, + image, + sourceRectangle, + colorConverter, + in parameters); + + using IMemoryOwner componentOwner = configuration.MemoryAllocator.Allocate(converter.ComponentBufferLength); + Span components = componentOwner.GetSpan(); + if (converter.UsesByteInput) + { + converter.Convert(Span.Empty, components); + return; + } + + using IMemoryOwner packedOwner = configuration.MemoryAllocator.Allocate(sourceRectangle.Width); + converter.Convert(packedOwner.GetSpan()[..sourceRectangle.Width], components); + } + + /// + /// Converts a rectangular packed-pixel region using a retained color converter and caller-owned row storage. + /// + /// The source pixel type. + /// The codec adapter exposing the native component planes. + /// The native unsigned sample storage type. + /// The SIMD narrowing and storage operations for the sample type. + /// The configuration used for pixel conversion. + /// The source image frame. + /// The source region mapped to the complete destination buffer. + /// The destination component-plane buffer. + /// The resolved H.273 conversion parameters. + /// The retained converter matching . + /// The reusable high-bit-depth packed RGB row, or an empty span for eight-bit input. + /// The reusable planar component rows. + public static void ConvertFromRgb( + Configuration configuration, + ImageFrame image, + Rectangle sourceRectangle, + TBuffer buffer, + in HeifColorConversionParameters parameters, + HeifColorConverterBase colorConverter, + Span packed, + Span components) + where TPixel : unmanaged, IPixel + where TBuffer : struct, IHeifPlanarSampleBuffer + where TSample : unmanaged + where TStorer : struct, IHeifSampleConverter + { + RgbToYuvRowConverter converter = new( + configuration, + buffer, + image, + sourceRectangle, + colorConverter, + in parameters); + + converter.Convert(packed, components); + } + + /// + /// Gets the planar float storage required to convert one row, or one vertically subsampled row pair. + /// + /// The source-row width. + /// Whether only luma is written. + /// The vertical chroma-subsampling shift. + /// The required number of float elements. + public static int GetRgbToYuvComponentBufferLength(int width, bool isMonochrome, int subsamplingY) + => width * ColorComponentCount * (subsamplingY == 0 || isMonochrome ? 1 : VerticallySubsampledRowCount); + + /// + /// Resolves the two chroma rows and quarter-sample weight surrounding a luma row. + /// + /// The luma row coordinate. + /// The vertical chroma subsampling shift. + /// The chroma offset in half-luma-sample units. + /// The last available chroma row. + /// The lower chroma row. + /// The upper chroma row. + /// The upper-row weight with a denominator of four. + private static void GetChromaCoordinates( + int coordinate, + int subsampling, + int position, + int maximum, + out int lower, + out int upper, + out int upperWeight) + { + if (subsampling == 0) + { + lower = coordinate; + upper = coordinate; + upperWeight = 0; + return; + } + + // Multiplication by two expresses the luma coordinate in half-sample units, while division by four + // addresses the subsampled plane. Floor division is required before the first centered chroma sample. + int quarterCoordinate = (coordinate * 2) - position; + int unclampedLower = quarterCoordinate >= 0 ? quarterCoordinate >> 2 : -((-quarterCoordinate + 3) >> 2); + int fraction = quarterCoordinate - (unclampedLower * 4); + lower = Numerics.Clamp(unclampedLower, 0, maximum); + upper = Numerics.Clamp(unclampedLower + 1, 0, maximum); + upperWeight = lower == upper ? 0 : fraction; + } + + /// + /// Converts one native component row using pooled planar and packed-pixel storage. + /// + /// The destination pixel type. + /// The codec adapter exposing the native component planes. + /// The native unsigned sample storage type. + /// The SIMD widening operations for the sample type. + private struct YuvToRgbRowConverter + where TPixel : unmanaged, IPixel + where TBuffer : struct, IHeifPlanarSampleBuffer + where TSample : unmanaged + where TLoader : struct, IHeifSampleConverter + { + /// + /// The configuration used for packed-pixel conversion. + /// + private readonly Configuration configuration; + + /// + /// The codec adapter exposing the reconstructed component planes. + /// + private TBuffer buffer; + + /// + /// The destination image frame. + /// + private readonly ImageFrame image; + + /// + /// The selected H.273 color converter. + /// + private readonly HeifColorConverterBase colorConverter; + + /// + /// The horizontal luma-sample offset of the output window. + /// + private readonly int sourceX; + + /// + /// The vertical luma-sample offset of the output window. + /// + private readonly int sourceY; + + /// + /// The full coded luma-plane width. + /// + private readonly int bufferWidth; + + /// + /// The chroma-plane width in samples. + /// + private readonly int chromaWidth; + + /// + /// The chroma-plane height in samples. + /// + private readonly int chromaHeight; + + /// + /// The horizontal chroma subsampling shift. + /// + private readonly int subsamplingX; + + /// + /// The vertical chroma subsampling shift. + /// + private readonly int subsamplingY; + + /// + /// The horizontal chroma position in half-luma-sample units. + /// + private readonly int chromaPositionX; + + /// + /// The vertical chroma position in half-luma-sample units. + /// + private readonly int chromaPositionY; + + /// + /// Whether the buffer contains only the luma plane. + /// + private readonly bool isMonochrome; + + /// + /// Whether the destination uses JPEG-compatible eight-bit RGB plane packing. + /// + private readonly bool usesBytePacking; + + /// + /// Whether chroma reconstruction must retain the complete coded row before selecting the output window. + /// + private readonly bool reconstructCompleteRow; + + /// + /// Initializes a new instance of the struct. + /// + /// The configuration used for pixel conversion. + /// The codec adapter exposing the reconstructed component planes. + /// The destination image frame. + /// The selected H.273 color converter. + /// The horizontal luma-sample offset of the output window. + /// The vertical luma-sample offset of the output window. + public YuvToRgbRowConverter( + Configuration configuration, + TBuffer buffer, + ImageFrame image, + HeifColorConverterBase colorConverter, + int sourceX, + int sourceY) + { + this.configuration = configuration; + this.buffer = buffer; + this.image = image; + this.colorConverter = colorConverter; + this.sourceX = sourceX; + this.sourceY = sourceY; + this.bufferWidth = buffer.Width; + this.subsamplingX = buffer.ChromaSubsamplingX; + this.subsamplingY = buffer.ChromaSubsamplingY; + this.chromaPositionX = buffer.ChromaPositionX; + this.chromaPositionY = buffer.ChromaPositionY; + this.isMonochrome = buffer.IsMonochrome; + this.usesBytePacking = buffer.LumaBitDepth == 8 && (buffer.IsMonochrome || buffer.ChromaBitDepth == 8); + this.chromaWidth = (buffer.Width + (1 << this.subsamplingX) - 1) >> this.subsamplingX; + this.chromaHeight = (buffer.Height + (1 << this.subsamplingY) - 1) >> this.subsamplingY; + this.reconstructCompleteRow = sourceX != 0 || image.Width != buffer.Width; + } + + /// + /// Gets a value indicating whether the destination uses JPEG-compatible eight-bit RGB plane packing. + /// + public readonly bool UsesBytePacking => this.usesBytePacking; + + /// + /// Gets the number of float elements required by the reusable row buffer. + /// + public readonly int BufferLength + { + get + { + int componentLength = this.image.Width * 3; + if (!this.isMonochrome && this.subsamplingX != 0) + { + // Full-image conversion reconstructs directly into the component rows. Cropped conversion + // retains the complete coded row so interpolation phase is preserved at the window boundary. + componentLength += (this.chromaWidth * 2) + (this.reconstructCompleteRow ? this.bufferWidth : 0); + } + + int packedRowCount = this.UsesBytePacking ? 1 : 2; + return componentLength + (this.image.Width * packedRowCount); + } + } + + /// + /// Converts one reconstructed row to packed pixels. + /// + /// The zero-based output row. + /// The reusable pooled row buffer. + public void Convert(int y, Span scratch) + { + int width = this.image.Width; + Span red = scratch[..width]; + Span green = scratch.Slice(width, width); + Span blue = scratch.Slice(width * 2, width); + int sourceY = y + this.sourceY; + ReadOnlySpan luma = this.buffer.GetLumaRowSpan(sourceY).Slice(this.sourceX, width); + HeifSampleConversion.ConvertSamplesToFloat(luma, red); + + int packedOffset = width * 3; + if (!this.isMonochrome) + { + GetChromaCoordinates( + sourceY, + this.subsamplingY, + this.chromaPositionY, + this.chromaHeight - 1, + out int y0, + out int y1, + out int y1Weight); + + ReadOnlySpan cb0 = this.buffer.GetChromaBlueRowSpan(y0); + ReadOnlySpan cb1 = this.buffer.GetChromaBlueRowSpan(y1); + ReadOnlySpan cr0 = this.buffer.GetChromaRedRowSpan(y0); + ReadOnlySpan cr1 = this.buffer.GetChromaRedRowSpan(y1); + if (this.subsamplingX == 0) + { + HeifSampleConversion.ConvertSamplesToFloat(cb0.Slice(this.sourceX, width), green); + HeifSampleConversion.ConvertSamplesToFloat(cr0.Slice(this.sourceX, width), blue); + } + else + { + Span chroma0 = scratch.Slice(packedOffset, this.chromaWidth); + Span chroma1 = scratch.Slice(packedOffset + this.chromaWidth, this.chromaWidth); + Span reconstructed = this.reconstructCompleteRow + ? scratch.Slice(packedOffset + (this.chromaWidth * 2), this.bufferWidth) + : green; + + bool isCenteredX = this.chromaPositionX == 1; + + HeifSampleConversion.ReconstructChromaRow( + cb0, + cb1, + y1Weight, + this.subsamplingX, + isCenteredX, + reconstructed, + chroma0, + chroma1); + + if (this.reconstructCompleteRow) + { + reconstructed.Slice(this.sourceX, width).CopyTo(green); + } + + reconstructed = this.reconstructCompleteRow ? reconstructed : blue; + HeifSampleConversion.ReconstructChromaRow( + cr0, + cr1, + y1Weight, + this.subsamplingX, + isCenteredX, + reconstructed, + chroma0, + chroma1); + + if (this.reconstructCompleteRow) + { + reconstructed.Slice(this.sourceX, width).CopyTo(blue); + } + + packedOffset += (this.chromaWidth * 2) + (this.reconstructCompleteRow ? this.bufferWidth : 0); + } + } + + this.colorConverter.ConvertToRgbInPlace(red, green, blue); + Span destination = this.image.PixelBuffer.DangerousGetRowSpan(y); + Span packedStorage = scratch[packedOffset..]; + if (this.UsesBytePacking) + { + // This is JPEG's planar packing contract. Existing pixel-specific SIMD packers therefore own the + // final RGB-to-TPixel conversion rather than a HEIF-codec-specific per-pixel implementation. + Span byteStorage = MemoryMarshal.AsBytes(packedStorage)[..(width * 3)]; + Span redBytes = byteStorage[..width]; + Span greenBytes = byteStorage.Slice(width, width); + Span blueBytes = byteStorage.Slice(width * 2, width); + SimdUtils.NormalizedFloatToByteSaturate(red, redBytes); + SimdUtils.NormalizedFloatToByteSaturate(green, greenBytes); + SimdUtils.NormalizedFloatToByteSaturate(blue, blueBytes); + + PixelOperations.Instance.PackFromRgbPlanes(redBytes, greenBytes, blueBytes, destination); + return; + } + + Span packed = MemoryMarshal.Cast(packedStorage)[..width]; + HeifSampleConversion.PackRgba64(red, green, blue, packed); + PixelOperations.Instance.FromRgba64(this.configuration, packed, destination); + } + } + + /// + /// Converts packed image rows to native component planes using pooled planar storage. + /// + /// The source pixel type. + /// The codec adapter exposing the native component planes. + /// The native unsigned sample storage type. + /// The SIMD narrowing and storage operations for the sample type. + private struct RgbToYuvRowConverter + where TPixel : unmanaged, IPixel + where TBuffer : struct, IHeifPlanarSampleBuffer + where TSample : unmanaged + where TStorer : struct, IHeifSampleConverter + { + /// + /// The configuration used for packed-pixel conversion. + /// + private readonly Configuration configuration; + + /// + /// The codec adapter exposing the destination component planes. + /// + private TBuffer buffer; + + /// + /// The source image frame. + /// + private readonly ImageFrame image; + + /// + /// The source region mapped to the complete destination planes. + /// + private readonly Rectangle sourceRectangle; + + /// + /// The selected H.273 color converter. + /// + private readonly HeifColorConverterBase colorConverter; + + /// + /// The largest encoded luma sample value. + /// + private readonly float lumaMaximum; + + /// + /// The largest encoded chroma sample value. + /// + private readonly float chromaMaximum; + + /// + /// The horizontal chroma subsampling shift. + /// + private readonly int subsamplingX; + + /// + /// The vertical chroma subsampling shift. + /// + private readonly int subsamplingY; + + /// + /// The horizontal chroma position in half-luma-sample units. + /// + private readonly int chromaPositionX; + + /// + /// The vertical chroma position in half-luma-sample units. + /// + private readonly int chromaPositionY; + + /// + /// Whether the destination buffer contains only the luma plane. + /// + private readonly bool isMonochrome; + + /// + /// Whether source pixels use JPEG-compatible eight-bit RGB plane unpacking. + /// + private readonly bool usesByteInput; + + /// + /// Initializes a new instance of the struct. + /// + /// The configuration used for pixel conversion. + /// The codec adapter exposing the destination component planes. + /// The source image frame. + /// The source region mapped to the complete destination planes. + /// The selected H.273 color converter. + /// The resolved H.273 component ranges. + public RgbToYuvRowConverter( + Configuration configuration, + TBuffer buffer, + ImageFrame image, + Rectangle sourceRectangle, + HeifColorConverterBase colorConverter, + in HeifColorConversionParameters parameters) + { + this.configuration = configuration; + this.buffer = buffer; + this.image = image; + this.sourceRectangle = sourceRectangle; + this.colorConverter = colorConverter; + this.lumaMaximum = parameters.LumaSampleMaximum; + this.chromaMaximum = parameters.ChromaSampleMaximum; + this.subsamplingX = buffer.ChromaSubsamplingX; + this.subsamplingY = buffer.ChromaSubsamplingY; + this.chromaPositionX = buffer.ChromaPositionX; + this.chromaPositionY = buffer.ChromaPositionY; + this.isMonochrome = buffer.IsMonochrome; + this.usesByteInput = buffer.LumaBitDepth == 8 && (buffer.IsMonochrome || buffer.ChromaBitDepth == 8); + } + + /// + /// Gets a value indicating whether source pixels use JPEG-compatible eight-bit RGB plane unpacking. + /// + public readonly bool UsesByteInput => this.usesByteInput; + + /// + /// Gets the number of float elements required by the reusable component buffer. + /// + public readonly int ComponentBufferLength + => GetRgbToYuvComponentBufferLength(this.sourceRectangle.Width, this.isMonochrome, this.subsamplingY); + + /// + /// Converts every packed source row to the destination component planes. + /// + /// The reusable high-bit-depth RGB staging row. + /// The reusable planar component buffer. + public void Convert(Span packed, Span components) + { + int width = this.sourceRectangle.Width; + Span luma0 = components[..width]; + Span blue0 = components.Slice(width, width); + Span red0 = components.Slice(width * 2, width); + if (this.subsamplingY == 0) + { + for (int y = 0; y < this.sourceRectangle.Height; y++) + { + this.ConvertSourceRow(y, packed, luma0, blue0, red0); + HeifSampleConversion.WriteSamples( + luma0, + this.buffer.GetLumaRowSpan(y), + this.colorConverter.LumaScale, + this.colorConverter.LumaBias, + this.lumaMaximum); + + if (!this.isMonochrome) + { + this.WriteChromaRows(y, blue0, red0, Span.Empty, Span.Empty, 0F); + } + } + + return; + } + + Span luma1 = this.isMonochrome ? luma0 : components.Slice(width * 3, width); + Span blue1 = this.isMonochrome ? blue0 : components.Slice(width * 4, width); + Span red1 = this.isMonochrome ? red0 : components.Slice(width * 5, width); + int chromaHeight = (this.sourceRectangle.Height + 1) >> 1; + for (int destinationY = 0; destinationY < chromaHeight; destinationY++) + { + // A vertically subsampled chroma row is owned by one two-row luma cell. Processing that cell as a + // unit removes per-row state and lets the selected chroma position choose or average the two rows. + int sourceY = destinationY << 1; + this.ConvertSourceRow(sourceY, packed, luma0, blue0, red0); + HeifSampleConversion.WriteSamples( + luma0, + this.buffer.GetLumaRowSpan(sourceY), + this.colorConverter.LumaScale, + this.colorConverter.LumaBias, + this.lumaMaximum); + + bool hasSecondRow = sourceY + 1 < this.sourceRectangle.Height; + if (hasSecondRow) + { + this.ConvertSourceRow(sourceY + 1, packed, luma1, blue1, red1); + HeifSampleConversion.WriteSamples( + luma1, + this.buffer.GetLumaRowSpan(sourceY + 1), + this.colorConverter.LumaScale, + this.colorConverter.LumaBias, + this.lumaMaximum); + } + + if (this.isMonochrome) + { + continue; + } + + if (this.chromaPositionY == 0 || !hasSecondRow) + { + this.WriteChromaRows(destinationY, blue0, red0, Span.Empty, Span.Empty, 0F); + } + else if (this.chromaPositionY == 1) + { + this.WriteChromaRows(destinationY, blue0, red0, blue1, red1, 0.5F); + } + else + { + this.WriteChromaRows(destinationY, blue1, red1, Span.Empty, Span.Empty, 0F); + } + } + } + + /// + /// Converts one packed source row into normalized planar components. + /// + /// The source row index. + /// The high-bit-depth RGB staging row. + /// The destination luma or first component values. + /// The destination blue-difference or second component values. + /// The destination red-difference or third component values. + private void ConvertSourceRow(int y, Span packed, Span luma, Span chromaBlue, Span chromaRed) + { + ReadOnlySpan source = this.image.PixelBuffer + .DangerousGetRowSpan(this.sourceRectangle.Y + y) + .Slice(this.sourceRectangle.X, this.sourceRectangle.Width); + + if (this.UsesByteInput) + { + // JPEG's planar unpack contract reaches the existing pixel-specific SIMD implementation before + // the closed H.273 operator transforms the component rows in place. + PixelOperations.Instance.UnpackIntoRgbPlanes(luma, chromaBlue, chromaRed, source); + this.colorConverter.ConvertFromRgbInPlace(luma, chromaBlue, chromaRed, ByteMaximum); + return; + } + + PixelOperations.Instance.ToRgb48(this.configuration, source, packed); + HeifSampleConversion.DeinterleaveRgb48(packed, luma, chromaBlue, chromaRed); + this.colorConverter.ConvertFromRgbInPlace(luma, chromaBlue, chromaRed, UShortMaximum); + } + + /// + /// Filters and writes one pair of chroma component rows. + /// + /// The destination chroma row. + /// The first blue-difference source row. + /// The first red-difference source row. + /// The optional second blue-difference source row. + /// The optional second red-difference source row. + /// The second-row contribution. + private void WriteChromaRows( + int destinationY, + ReadOnlySpan blue0, + ReadOnlySpan red0, + ReadOnlySpan blue1, + ReadOnlySpan red1, + float row1Weight) + { + Span blueDestination = this.buffer.GetChromaBlueRowSpan(destinationY); + Span redDestination = this.buffer.GetChromaRedRowSpan(destinationY); + if (this.subsamplingX == 0) + { + HeifSampleConversion.WriteSamples( + blue0, + blueDestination, + this.colorConverter.ChromaScale, + this.colorConverter.ChromaBias, + this.chromaMaximum); + + HeifSampleConversion.WriteSamples( + red0, + redDestination, + this.colorConverter.ChromaScale, + this.colorConverter.ChromaBias, + this.chromaMaximum); + + return; + } + + bool isCenteredX = this.chromaPositionX == 1; + HeifSampleConversion.WriteSubsampledSamples( + blue0, + blue1, + blueDestination, + isCenteredX, + row1Weight, + this.colorConverter.ChromaScale, + this.colorConverter.ChromaBias, + this.chromaMaximum); + + HeifSampleConversion.WriteSubsampledSamples( + red0, + red1, + redDestination, + isCenteredX, + row1Weight, + this.colorConverter.ChromaScale, + this.colorConverter.ChromaBias, + this.chromaMaximum); + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Components/ColorConverters/HeifSampleConversion.cs b/src/ImageSharp/Formats/Heif/Components/ColorConverters/HeifSampleConversion.cs new file mode 100644 index 0000000000..39bcbc4360 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Components/ColorConverters/HeifSampleConversion.cs @@ -0,0 +1,962 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Numerics; +using System.Runtime.CompilerServices; +using System.Runtime.InteropServices; +using System.Runtime.Intrinsics; +using SixLabors.ImageSharp.Common.Helpers; +using SixLabors.ImageSharp.PixelFormats; + +namespace SixLabors.ImageSharp.Formats.Heif.Components; + +/// +/// Provides SIMD sample widening, chroma reconstruction, planar storage, and packed output for HEIF color conversion. +/// +internal static class HeifSampleConversion +{ + /// + /// The largest value represented by a 16-bit packed RGB component. + /// + private const float UShortMaximum = ushort.MaxValue; + + /// + /// Widens reconstructed integer samples into a pooled float component row. + /// + /// The reconstructed sample type. + /// The widening operations for the sample type. + /// The reconstructed samples. + /// The destination component row. + public static void ConvertSamplesToFloat(ReadOnlySpan source, Span destination) + where TSample : unmanaged + where TLoader : struct, IHeifSampleConverter + { + ref TSample sourceBase = ref MemoryMarshal.GetReference(source); + ref float destinationBase = ref MemoryMarshal.GetReference(destination); + int length = destination.Length; + int i = 0; + + // Descending vector widths match the JPEG color-converter traversal. A wide-capable CPU processes + // complete wide batches first while short and irregular rows continue through narrower SIMD tails. + if (Vector512.IsHardwareAccelerated) + { + int oneVectorFromEnd = length - Vector512.Count; + for (; i <= oneVectorFromEnd; i += Vector512.Count) + { + Vector512 samples = TLoader.LoadVector512(ref Unsafe.Add(ref sourceBase, i)); + Unsafe.As>(ref Unsafe.Add(ref destinationBase, i)) = samples; + } + } + + if (Vector256.IsHardwareAccelerated) + { + int oneVectorFromEnd = length - Vector256.Count; + for (; i <= oneVectorFromEnd; i += Vector256.Count) + { + Vector256 samples = TLoader.LoadVector256(ref Unsafe.Add(ref sourceBase, i)); + Unsafe.As>(ref Unsafe.Add(ref destinationBase, i)) = samples; + } + } + + if (Vector128.IsHardwareAccelerated) + { + int oneVectorFromEnd = length - Vector128.Count; + for (; i <= oneVectorFromEnd; i += Vector128.Count) + { + Vector128 samples = TLoader.LoadVector128(ref Unsafe.Add(ref sourceBase, i)); + Unsafe.As>(ref Unsafe.Add(ref destinationBase, i)) = samples; + } + } + + for (; i < length; i++) + { + Unsafe.Add(ref destinationBase, i) = GetSample(source, i); + } + } + + /// + /// Reconstructs one full-width chroma row using the signaled vertical and horizontal sample positions. + /// + /// The reconstructed sample type. + /// The widening operations for the sample type. + /// The upper chroma row. + /// The lower chroma row. + /// The lower-row weight with a denominator of four. + /// The horizontal chroma subsampling shift. + /// Whether horizontally subsampled chroma is centered between luma samples. + /// The reconstructed full-width chroma row. + /// The first pooled chroma scratch row. + /// The second pooled chroma scratch row. + public static void ReconstructChromaRow( + ReadOnlySpan row0, + ReadOnlySpan row1, + int y1Weight, + int subX, + bool isCenteredX, + Span destination, + Span scratch0, + Span scratch1) + where TSample : unmanaged + where TLoader : struct, IHeifSampleConverter + { + int sourceLength = subX == 0 ? destination.Length : (destination.Length + 1) >> 1; + Span top = scratch0[..sourceLength]; + ConvertSamplesToFloat(row0, top); + + if (y1Weight != 0) + { + Span bottom = scratch1[..sourceLength]; + ConvertSamplesToFloat(row1, bottom); + InterpolateChromaRows(top, bottom, y1Weight); + } + + if (subX == 0) + { + top.CopyTo(destination); + return; + } + + UpsampleChromaHorizontal(top, destination, isCenteredX); + } + + /// + /// Interpolates two chroma rows in place using quarter-sample weights. + /// + /// The upper row, replaced by the interpolated values. + /// The lower row. + /// The lower-row weight with a denominator of four. + private static void InterpolateChromaRows(Span top, ReadOnlySpan bottom, int bottomWeight) + { + ref float topBase = ref MemoryMarshal.GetReference(top); + ref float bottomBase = ref MemoryMarshal.GetReference(bottom); + int length = top.Length; + int i = 0; + float topWeight = 4 - bottomWeight; + + if (Vector512.IsHardwareAccelerated) + { + Vector512 topWeightVector = Vector512.Create(topWeight); + Vector512 bottomWeightVector = Vector512.Create((float)bottomWeight); + Vector512 scale = Vector512.Create(0.25F); + int oneVectorFromEnd = length - Vector512.Count; + for (; i <= oneVectorFromEnd; i += Vector512.Count) + { + ref Vector512 topVector = ref Unsafe.As>(ref Unsafe.Add(ref topBase, i)); + Vector512 bottomVector = Unsafe.As>(ref Unsafe.Add(ref bottomBase, i)); + topVector = Vector512.MultiplyAddEstimate(bottomWeightVector, bottomVector, topWeightVector * topVector) * scale; + } + } + + if (Vector256.IsHardwareAccelerated) + { + Vector256 topWeightVector = Vector256.Create(topWeight); + Vector256 bottomWeightVector = Vector256.Create((float)bottomWeight); + Vector256 scale = Vector256.Create(0.25F); + int oneVectorFromEnd = length - Vector256.Count; + for (; i <= oneVectorFromEnd; i += Vector256.Count) + { + ref Vector256 topVector = ref Unsafe.As>(ref Unsafe.Add(ref topBase, i)); + Vector256 bottomVector = Unsafe.As>(ref Unsafe.Add(ref bottomBase, i)); + topVector = Vector256.MultiplyAddEstimate(bottomWeightVector, bottomVector, topWeightVector * topVector) * scale; + } + } + + if (Vector128.IsHardwareAccelerated) + { + Vector128 topWeightVector = Vector128.Create(topWeight); + Vector128 bottomWeightVector = Vector128.Create((float)bottomWeight); + Vector128 scale = Vector128.Create(0.25F); + int oneVectorFromEnd = length - Vector128.Count; + for (; i <= oneVectorFromEnd; i += Vector128.Count) + { + ref Vector128 topVector = ref Unsafe.As>(ref Unsafe.Add(ref topBase, i)); + Vector128 bottomVector = Unsafe.As>(ref Unsafe.Add(ref bottomBase, i)); + topVector = Vector128.MultiplyAddEstimate(bottomWeightVector, bottomVector, topWeightVector * topVector) * scale; + } + } + + for (; i < length; i++) + { + Unsafe.Add(ref topBase, i) = ((Unsafe.Add(ref topBase, i) * topWeight) + (Unsafe.Add(ref bottomBase, i) * bottomWeight)) * 0.25F; + } + } + + /// + /// Expands horizontally subsampled chroma to luma width using the selected sample-position rules. + /// + /// The subsampled chroma values. + /// The full-width chroma values. + /// Whether chroma lies between neighboring luma samples. + private static void UpsampleChromaHorizontal(ReadOnlySpan source, Span destination, bool isCentered) + { + ref float sourceBase = ref MemoryMarshal.GetReference(source); + ref float destinationBase = ref MemoryMarshal.GetReference(destination); + int sourceLength = source.Length; + int i = 0; + + if (isCentered) + { + // The first centered pair extends the left edge. Interior vectors can then read one real neighbor + // on each side and use the exact [1,3]/4 and [3,1]/4 interpolation weights. + StoreChromaPair(ref destinationBase, 0, source[0], ((3F * source[0]) + source[Math.Min(1, sourceLength - 1)]) * 0.25F, destination.Length); + i = 1; + } + + if (Vector512.IsHardwareAccelerated) + { + int oneVectorBeforeEnd = sourceLength - Vector512.Count - 1; + Vector512 quarter = Vector512.Create(0.25F); + Vector512 half = Vector512.Create(0.5F); + Vector512 three = Vector512.Create(3F); + for (; i <= oneVectorBeforeEnd; i += Vector512.Count) + { + Vector512 center = Unsafe.As>(ref Unsafe.Add(ref sourceBase, i)); + Vector512 next = Unsafe.As>(ref Unsafe.Add(ref sourceBase, i + 1)); + Vector512 even; + Vector512 odd; + if (isCentered) + { + Vector512 previous = Unsafe.As>(ref Unsafe.Add(ref sourceBase, i - 1)); + even = Vector512.MultiplyAddEstimate(three, center, previous) * quarter; + odd = Vector512.MultiplyAddEstimate(three, center, next) * quarter; + } + else + { + even = center; + odd = (center + next) * half; + } + + // Vector512 has no cross-platform unpack helper. The interpolation remains 512-bit; four + // established Vector128 unpack operations only transpose the final even/odd lanes for storage. + StoreInterleavedChroma(even.GetLower().GetLower(), odd.GetLower().GetLower(), ref Unsafe.Add(ref destinationBase, i * 2)); + StoreInterleavedChroma(even.GetLower().GetUpper(), odd.GetLower().GetUpper(), ref Unsafe.Add(ref destinationBase, (i * 2) + 8)); + StoreInterleavedChroma(even.GetUpper().GetLower(), odd.GetUpper().GetLower(), ref Unsafe.Add(ref destinationBase, (i * 2) + 16)); + StoreInterleavedChroma(even.GetUpper().GetUpper(), odd.GetUpper().GetUpper(), ref Unsafe.Add(ref destinationBase, (i * 2) + 24)); + } + } + + if (Vector256.IsHardwareAccelerated) + { + int oneVectorBeforeEnd = sourceLength - Vector256.Count - 1; + Vector256 quarter = Vector256.Create(0.25F); + Vector256 half = Vector256.Create(0.5F); + Vector256 three = Vector256.Create(3F); + for (; i <= oneVectorBeforeEnd; i += Vector256.Count) + { + Vector256 center = Unsafe.As>(ref Unsafe.Add(ref sourceBase, i)); + Vector256 next = Unsafe.As>(ref Unsafe.Add(ref sourceBase, i + 1)); + Vector256 even; + Vector256 odd; + if (isCentered) + { + Vector256 previous = Unsafe.As>(ref Unsafe.Add(ref sourceBase, i - 1)); + even = Vector256.MultiplyAddEstimate(three, center, previous) * quarter; + odd = Vector256.MultiplyAddEstimate(three, center, next) * quarter; + } + else + { + even = center; + odd = (center + next) * half; + } + + StoreInterleavedChroma(even.GetLower(), odd.GetLower(), ref Unsafe.Add(ref destinationBase, i * 2)); + StoreInterleavedChroma(even.GetUpper(), odd.GetUpper(), ref Unsafe.Add(ref destinationBase, (i * 2) + 8)); + } + } + + if (Vector128.IsHardwareAccelerated) + { + int oneVectorBeforeEnd = sourceLength - Vector128.Count - 1; + Vector128 quarter = Vector128.Create(0.25F); + Vector128 half = Vector128.Create(0.5F); + Vector128 three = Vector128.Create(3F); + for (; i <= oneVectorBeforeEnd; i += Vector128.Count) + { + Vector128 center = Unsafe.As>(ref Unsafe.Add(ref sourceBase, i)); + Vector128 next = Unsafe.As>(ref Unsafe.Add(ref sourceBase, i + 1)); + Vector128 even; + Vector128 odd; + if (isCentered) + { + Vector128 previous = Unsafe.As>(ref Unsafe.Add(ref sourceBase, i - 1)); + even = Vector128.MultiplyAddEstimate(three, center, previous) * quarter; + odd = Vector128.MultiplyAddEstimate(three, center, next) * quarter; + } + else + { + even = center; + odd = (center + next) * half; + } + + StoreInterleavedChroma(even, odd, ref Unsafe.Add(ref destinationBase, i * 2)); + } + } + + for (; i < sourceLength; i++) + { + float center = source[i]; + float next = source[Math.Min(i + 1, sourceLength - 1)]; + float even = isCentered ? (source[Math.Max(i - 1, 0)] + (3F * center)) * 0.25F : center; + float odd = isCentered ? ((3F * center) + next) * 0.25F : (center + next) * 0.5F; + StoreChromaPair(ref destinationBase, i * 2, even, odd, destination.Length); + } + } + + /// + /// Stores four even chroma lanes interleaved with their four odd lanes. + /// + /// The even luma-coordinate values. + /// The odd luma-coordinate values. + /// The first destination value. + private static void StoreInterleavedChroma(Vector128 even, Vector128 odd, ref float destination) + { + Vector128 lower = Vector128_.UnpackLow(even.AsInt32(), odd.AsInt32()).AsSingle(); + Vector128 upper = Vector128_.UnpackHigh(even.AsInt32(), odd.AsInt32()).AsSingle(); + Unsafe.As>(ref destination) = lower; + Unsafe.As>(ref Unsafe.Add(ref destination, Vector128.Count)) = upper; + } + + /// + /// Stores one reconstructed chroma pair without writing beyond an odd-width destination row. + /// + /// The first destination value. + /// The even destination index. + /// The even luma-coordinate value. + /// The odd luma-coordinate value. + /// The destination length. + private static void StoreChromaPair(ref float destination, int index, float even, float odd, int length) + { + Unsafe.Add(ref destination, index) = even; + if (index + 1 < length) + { + Unsafe.Add(ref destination, index + 1) = odd; + } + } + + /// + /// Deinterleaves high-bit-depth RGB pixels into planar component rows. + /// + /// The packed RGBA pixels. + /// The destination red components. + /// The destination green components. + /// The destination blue components. + public static void DeinterleaveRgb48(ReadOnlySpan source, Span red, Span green, Span blue) + { + ref float redBase = ref MemoryMarshal.GetReference(red); + ref float greenBase = ref MemoryMarshal.GetReference(green); + ref float blueBase = ref MemoryMarshal.GetReference(blue); + ref Rgb48 sourceBase = ref MemoryMarshal.GetReference(source); + int length = source.Length; + int i = 0; + + if (Vector128.IsHardwareAccelerated) + { + int oneVectorFromEnd = length - Vector128.Count; + for (; i <= oneVectorFromEnd; i += Vector128.Count) + { + ref Rgb48 pixel0 = ref Unsafe.Add(ref sourceBase, i); + ref Rgb48 pixel1 = ref Unsafe.Add(ref sourceBase, i + 1); + ref Rgb48 pixel2 = ref Unsafe.Add(ref sourceBase, i + 2); + ref Rgb48 pixel3 = ref Unsafe.Add(ref sourceBase, i + 3); + + // Widen each UInt16 channel to a UInt32 lane before converting to Single. This avoids + // reinterpreting adjacent 16-bit samples as one unrelated 32-bit integer. + Vector128 redVector = Vector128.ConvertToSingle( + Vector128.Create((uint)pixel0.R, pixel1.R, pixel2.R, pixel3.R)); + + Vector128 greenVector = Vector128.ConvertToSingle( + Vector128.Create((uint)pixel0.G, pixel1.G, pixel2.G, pixel3.G)); + + Vector128 blueVector = Vector128.ConvertToSingle( + Vector128.Create((uint)pixel0.B, pixel1.B, pixel2.B, pixel3.B)); + + Unsafe.As>(ref Unsafe.Add(ref redBase, i)) = redVector; + Unsafe.As>(ref Unsafe.Add(ref greenBase, i)) = greenVector; + Unsafe.As>(ref Unsafe.Add(ref blueBase, i)) = blueVector; + } + } + + for (; i < length; i++) + { + Rgb48 pixel = Unsafe.Add(ref sourceBase, i); + Unsafe.Add(ref redBase, i) = pixel.R; + Unsafe.Add(ref greenBase, i) = pixel.G; + Unsafe.Add(ref blueBase, i) = pixel.B; + } + } + + /// + /// Scales, quantizes, and stores one planar row using the widest available SIMD width. + /// + /// The encoded sample type. + /// The narrowing and storage operations for the sample type. + /// The normalized source values. + /// The encoded destination samples. + /// The encoded range scale. + /// The encoded range bias. + /// The largest encoded sample value. + public static void WriteSamples(ReadOnlySpan source, Span destination, float scale, float bias, float maximum) + where TSample : unmanaged + where TStorer : struct, IHeifSampleConverter + { + ref float sourceBase = ref MemoryMarshal.GetReference(source); + ref TSample destinationBase = ref MemoryMarshal.GetReference(destination); + int length = destination.Length; + int i = 0; + + if (Vector512.IsHardwareAccelerated) + { + int oneVectorFromEnd = length - Vector512.Count; + for (; i <= oneVectorFromEnd; i += Vector512.Count) + { + Vector512 values = Unsafe.As>(ref Unsafe.Add(ref sourceBase, i)); + TStorer.Store(ScaleBiasRoundAndClampToInt32(values, scale, bias, maximum), ref Unsafe.Add(ref destinationBase, i)); + } + } + + if (Vector256.IsHardwareAccelerated) + { + int oneVectorFromEnd = length - Vector256.Count; + for (; i <= oneVectorFromEnd; i += Vector256.Count) + { + Vector256 values = Unsafe.As>(ref Unsafe.Add(ref sourceBase, i)); + TStorer.Store(ScaleBiasRoundAndClampToInt32(values, scale, bias, maximum), ref Unsafe.Add(ref destinationBase, i)); + } + } + + if (Vector128.IsHardwareAccelerated) + { + int oneVectorFromEnd = length - Vector128.Count; + for (; i <= oneVectorFromEnd; i += Vector128.Count) + { + Vector128 values = Unsafe.As>(ref Unsafe.Add(ref sourceBase, i)); + TStorer.Store(ScaleBiasRoundAndClampToInt32(values, scale, bias, maximum), ref Unsafe.Add(ref destinationBase, i)); + } + } + + for (; i < length; i++) + { + Unsafe.Add(ref destinationBase, i) = ToSample((Unsafe.Add(ref sourceBase, i) * scale) + bias, maximum); + } + } + + /// + /// Filters one or two planar rows into horizontally subsampled encoded samples. + /// + /// The encoded sample type. + /// The narrowing and storage operations for the sample type. + /// The first normalized source row. + /// The optional second normalized source row. + /// The encoded subsampled destination row. + /// Whether each output sample is centered between two horizontal source samples. + /// The contribution of the second row, in the inclusive range zero through one. + /// The encoded range scale. + /// The encoded range bias. + /// The largest encoded sample value. + public static void WriteSubsampledSamples( + ReadOnlySpan row0, + ReadOnlySpan row1, + Span destination, + bool isCenteredX, + float row1Weight, + float scale, + float bias, + float maximum) + where TSample : unmanaged + where TStorer : struct, IHeifSampleConverter + { + ref float row0Base = ref MemoryMarshal.GetReference(row0); + ref float row1Base = ref MemoryMarshal.GetReference(row1); + ref TSample destinationBase = ref MemoryMarshal.GetReference(destination); + bool hasSecondRow = !row1.IsEmpty; + float effectiveRow1Weight = hasSecondRow ? row1Weight : 0F; + float row0Weight = 1F - effectiveRow1Weight; + int length = row0.Length; + int i = 0; + + if (Vector512.IsHardwareAccelerated) + { + Vector512 evenOdd = Vector512.Create(0, 2, 4, 6, 8, 10, 12, 14, 1, 3, 5, 7, 9, 11, 13, 15); + Vector512 row0WeightVector = Vector512.Create(row0Weight); + Vector512 row1WeightVector = Vector512.Create(effectiveRow1Weight); + int oneVectorFromEnd = length - Vector512.Count; + for (; i <= oneVectorFromEnd; i += Vector512.Count) + { + Vector512 filtered = Unsafe.As>(ref Unsafe.Add(ref row0Base, i)) * row0WeightVector; + if (hasSecondRow) + { + filtered = Vector512.MultiplyAddEstimate( + Unsafe.As>(ref Unsafe.Add(ref row1Base, i)), + row1WeightVector, + filtered); + } + + Vector512 shuffled = Vector512.Shuffle(filtered, evenOdd); + Vector256 samples = isCenteredX + ? (shuffled.GetLower() + shuffled.GetUpper()) * Vector256.Create(0.5F) + : shuffled.GetLower(); + + TStorer.Store(ScaleBiasRoundAndClampToInt32(samples, scale, bias, maximum), ref Unsafe.Add(ref destinationBase, i >> 1)); + } + } + + if (Vector256.IsHardwareAccelerated) + { + Vector256 evenOdd = Vector256.Create(0, 2, 4, 6, 1, 3, 5, 7); + Vector256 row0WeightVector = Vector256.Create(row0Weight); + Vector256 row1WeightVector = Vector256.Create(effectiveRow1Weight); + int oneVectorFromEnd = length - Vector256.Count; + for (; i <= oneVectorFromEnd; i += Vector256.Count) + { + Vector256 filtered = Unsafe.As>(ref Unsafe.Add(ref row0Base, i)) * row0WeightVector; + if (hasSecondRow) + { + filtered = Vector256.MultiplyAddEstimate( + Unsafe.As>(ref Unsafe.Add(ref row1Base, i)), + row1WeightVector, + filtered); + } + + Vector256 shuffled = Vector256.Shuffle(filtered, evenOdd); + Vector128 samples = isCenteredX + ? (shuffled.GetLower() + shuffled.GetUpper()) * Vector128.Create(0.5F) + : shuffled.GetLower(); + + TStorer.Store(ScaleBiasRoundAndClampToInt32(samples, scale, bias, maximum), ref Unsafe.Add(ref destinationBase, i >> 1)); + } + } + + if (Vector128.IsHardwareAccelerated) + { + Vector128 evenOdd = Vector128.Create(0, 2, 1, 3); + Vector128 row0WeightVector = Vector128.Create(row0Weight); + Vector128 row1WeightVector = Vector128.Create(effectiveRow1Weight); + int twoVectorsFromEnd = length - (Vector128.Count * 2); + for (; i <= twoVectorsFromEnd; i += Vector128.Count * 2) + { + Vector128 filtered0 = Unsafe.As>(ref Unsafe.Add(ref row0Base, i)) * row0WeightVector; + Vector128 filtered1 = Unsafe.As>(ref Unsafe.Add(ref row0Base, i + Vector128.Count)) * row0WeightVector; + if (hasSecondRow) + { + filtered0 = Vector128.MultiplyAddEstimate( + Unsafe.As>(ref Unsafe.Add(ref row1Base, i)), + row1WeightVector, + filtered0); + + filtered1 = Vector128.MultiplyAddEstimate( + Unsafe.As>(ref Unsafe.Add(ref row1Base, i + Vector128.Count)), + row1WeightVector, + filtered1); + } + + Vector128 shuffled0 = Vector128.Shuffle(filtered0, evenOdd); + Vector128 shuffled1 = Vector128.Shuffle(filtered1, evenOdd); + Vector64 samples0 = isCenteredX + ? (shuffled0.GetLower() + shuffled0.GetUpper()) * Vector64.Create(0.5F) + : shuffled0.GetLower(); + + Vector64 samples1 = isCenteredX + ? (shuffled1.GetLower() + shuffled1.GetUpper()) * Vector64.Create(0.5F) + : shuffled1.GetLower(); + + TStorer.Store( + ScaleBiasRoundAndClampToInt32(Vector128.Create(samples0, samples1), scale, bias, maximum), + ref Unsafe.Add(ref destinationBase, i >> 1)); + } + } + + for (; i < length; i += 2) + { + float row0Sample = Unsafe.Add(ref row0Base, i); + if (isCenteredX && i + 1 < length) + { + row0Sample = (row0Sample + Unsafe.Add(ref row0Base, i + 1)) * 0.5F; + } + + float sample = row0Sample * row0Weight; + if (hasSecondRow) + { + float row1Sample = Unsafe.Add(ref row1Base, i); + if (isCenteredX && i + 1 < length) + { + row1Sample = (row1Sample + Unsafe.Add(ref row1Base, i + 1)) * 0.5F; + } + + sample += row1Sample * effectiveRow1Weight; + } + + Unsafe.Add(ref destinationBase, i >> 1) = ToSample((sample * scale) + bias, maximum); + } + } + + /// + /// Packs 16-bit RGB component rows into opaque 16-bit RGBA pixels. + /// + /// The red components. + /// The green components. + /// The blue components. + /// The destination pixels. + public static void PackRgba64(ReadOnlySpan red, ReadOnlySpan green, ReadOnlySpan blue, Span destination) + { + ref ushort redBase = ref MemoryMarshal.GetReference(red); + ref ushort greenBase = ref MemoryMarshal.GetReference(green); + ref ushort blueBase = ref MemoryMarshal.GetReference(blue); + ref Rgba64 destinationBase = ref MemoryMarshal.GetReference(destination); + int length = destination.Length; + int i = 0; + + if (Vector512.IsHardwareAccelerated) + { + int oneVectorFromEnd = length - Vector512.Count; + for (; i <= oneVectorFromEnd; i += Vector512.Count) + { + (Vector256 redLower, Vector256 redUpper) = Vector256.Widen(Vector256.LoadUnsafe(ref Unsafe.Add(ref redBase, i))); + (Vector256 greenLower, Vector256 greenUpper) = Vector256.Widen(Vector256.LoadUnsafe(ref Unsafe.Add(ref greenBase, i))); + (Vector256 blueLower, Vector256 blueUpper) = Vector256.Widen(Vector256.LoadUnsafe(ref Unsafe.Add(ref blueBase, i))); + Vector512 r = Vector512.Create(redLower, redUpper).AsInt32(); + Vector512 g = Vector512.Create(greenLower, greenUpper).AsInt32(); + Vector512 b = Vector512.Create(blueLower, blueUpper).AsInt32(); + StoreRgba64Batch(r.GetLower().GetLower(), g.GetLower().GetLower(), b.GetLower().GetLower(), ref Unsafe.Add(ref destinationBase, i)); + StoreRgba64Batch(r.GetLower().GetUpper(), g.GetLower().GetUpper(), b.GetLower().GetUpper(), ref Unsafe.Add(ref destinationBase, i + 4)); + StoreRgba64Batch(r.GetUpper().GetLower(), g.GetUpper().GetLower(), b.GetUpper().GetLower(), ref Unsafe.Add(ref destinationBase, i + 8)); + StoreRgba64Batch(r.GetUpper().GetUpper(), g.GetUpper().GetUpper(), b.GetUpper().GetUpper(), ref Unsafe.Add(ref destinationBase, i + 12)); + } + } + + if (Vector256.IsHardwareAccelerated) + { + int oneVectorFromEnd = length - Vector256.Count; + for (; i <= oneVectorFromEnd; i += Vector256.Count) + { + Vector128 red16 = Vector128.LoadUnsafe(ref Unsafe.Add(ref redBase, i)); + Vector128 green16 = Vector128.LoadUnsafe(ref Unsafe.Add(ref greenBase, i)); + Vector128 blue16 = Vector128.LoadUnsafe(ref Unsafe.Add(ref blueBase, i)); + Vector256 r = Vector256.Create(Vector128.WidenLower(red16), Vector128.WidenUpper(red16)).AsInt32(); + Vector256 g = Vector256.Create(Vector128.WidenLower(green16), Vector128.WidenUpper(green16)).AsInt32(); + Vector256 b = Vector256.Create(Vector128.WidenLower(blue16), Vector128.WidenUpper(blue16)).AsInt32(); + StoreRgba64Batch(r.GetLower(), g.GetLower(), b.GetLower(), ref Unsafe.Add(ref destinationBase, i)); + StoreRgba64Batch(r.GetUpper(), g.GetUpper(), b.GetUpper(), ref Unsafe.Add(ref destinationBase, i + 4)); + } + } + + if (Vector128.IsHardwareAccelerated) + { + int oneVectorFromEnd = length - Vector128.Count; + for (; i <= oneVectorFromEnd; i += Vector128.Count) + { + ulong packedRed = Unsafe.ReadUnaligned(ref Unsafe.As(ref Unsafe.Add(ref redBase, i))); + ulong packedGreen = Unsafe.ReadUnaligned(ref Unsafe.As(ref Unsafe.Add(ref greenBase, i))); + ulong packedBlue = Unsafe.ReadUnaligned(ref Unsafe.As(ref Unsafe.Add(ref blueBase, i))); + Vector128 r = Vector128.WidenLower(Vector128.CreateScalarUnsafe(packedRed).AsUInt16()).AsInt32(); + Vector128 g = Vector128.WidenLower(Vector128.CreateScalarUnsafe(packedGreen).AsUInt16()).AsInt32(); + Vector128 b = Vector128.WidenLower(Vector128.CreateScalarUnsafe(packedBlue).AsUInt16()).AsInt32(); + StoreRgba64Batch(r, g, b, ref Unsafe.Add(ref destinationBase, i)); + } + } + + for (; i < length; i++) + { + Unsafe.Add(ref destinationBase, i) = new Rgba64( + Unsafe.Add(ref redBase, i), + Unsafe.Add(ref greenBase, i), + Unsafe.Add(ref blueBase, i), + ushort.MaxValue); + } + } + + /// + /// Packs normalized RGB component rows into opaque 16-bit RGBA pixels. + /// + /// The normalized red components. + /// The normalized green components. + /// The normalized blue components. + /// The destination pixels. + public static void PackRgba64(ReadOnlySpan red, ReadOnlySpan green, ReadOnlySpan blue, Span destination) + { + ref float redBase = ref MemoryMarshal.GetReference(red); + ref float greenBase = ref MemoryMarshal.GetReference(green); + ref float blueBase = ref MemoryMarshal.GetReference(blue); + ref Rgba64 destinationBase = ref MemoryMarshal.GetReference(destination); + int length = destination.Length; + int i = 0; + + if (Vector512.IsHardwareAccelerated) + { + int oneVectorFromEnd = length - Vector512.Count; + for (; i <= oneVectorFromEnd; i += Vector512.Count) + { + Vector512 r = ScaleRoundAndClampToInt32(Unsafe.As>(ref Unsafe.Add(ref redBase, i)), UShortMaximum); + Vector512 g = ScaleRoundAndClampToInt32(Unsafe.As>(ref Unsafe.Add(ref greenBase, i)), UShortMaximum); + Vector512 b = ScaleRoundAndClampToInt32(Unsafe.As>(ref Unsafe.Add(ref blueBase, i)), UShortMaximum); + StoreRgba64Batch(r.GetLower().GetLower(), g.GetLower().GetLower(), b.GetLower().GetLower(), ref Unsafe.Add(ref destinationBase, i)); + StoreRgba64Batch(r.GetLower().GetUpper(), g.GetLower().GetUpper(), b.GetLower().GetUpper(), ref Unsafe.Add(ref destinationBase, i + 4)); + StoreRgba64Batch(r.GetUpper().GetLower(), g.GetUpper().GetLower(), b.GetUpper().GetLower(), ref Unsafe.Add(ref destinationBase, i + 8)); + StoreRgba64Batch(r.GetUpper().GetUpper(), g.GetUpper().GetUpper(), b.GetUpper().GetUpper(), ref Unsafe.Add(ref destinationBase, i + 12)); + } + } + + if (Vector256.IsHardwareAccelerated) + { + int oneVectorFromEnd = length - Vector256.Count; + for (; i <= oneVectorFromEnd; i += Vector256.Count) + { + Vector256 r = ScaleRoundAndClampToInt32(Unsafe.As>(ref Unsafe.Add(ref redBase, i)), UShortMaximum); + Vector256 g = ScaleRoundAndClampToInt32(Unsafe.As>(ref Unsafe.Add(ref greenBase, i)), UShortMaximum); + Vector256 b = ScaleRoundAndClampToInt32(Unsafe.As>(ref Unsafe.Add(ref blueBase, i)), UShortMaximum); + StoreRgba64Batch(r.GetLower(), g.GetLower(), b.GetLower(), ref Unsafe.Add(ref destinationBase, i)); + StoreRgba64Batch(r.GetUpper(), g.GetUpper(), b.GetUpper(), ref Unsafe.Add(ref destinationBase, i + 4)); + } + } + + if (Vector128.IsHardwareAccelerated) + { + int oneVectorFromEnd = length - Vector128.Count; + for (; i <= oneVectorFromEnd; i += Vector128.Count) + { + Vector128 r = ScaleRoundAndClampToInt32(Unsafe.As>(ref Unsafe.Add(ref redBase, i)), UShortMaximum); + Vector128 g = ScaleRoundAndClampToInt32(Unsafe.As>(ref Unsafe.Add(ref greenBase, i)), UShortMaximum); + Vector128 b = ScaleRoundAndClampToInt32(Unsafe.As>(ref Unsafe.Add(ref blueBase, i)), UShortMaximum); + StoreRgba64Batch(r, g, b, ref Unsafe.Add(ref destinationBase, i)); + } + } + + for (; i < length; i++) + { + Unsafe.Add(ref destinationBase, i) = new Rgba64( + ToSample(Unsafe.Add(ref redBase, i) * UShortMaximum, UShortMaximum), + ToSample(Unsafe.Add(ref greenBase, i) * UShortMaximum, UShortMaximum), + ToSample(Unsafe.Add(ref blueBase, i) * UShortMaximum, UShortMaximum), + ushort.MaxValue); + } + } + + /// + /// Packs normalized monochrome samples into 16-bit luminance pixels. + /// + /// The normalized monochrome samples. + /// The destination luminance pixels. + public static void PackL16(ReadOnlySpan source, Span destination) + { + ref float sourceBase = ref MemoryMarshal.GetReference(source); + ref L16 destinationBase = ref MemoryMarshal.GetReference(destination); + int length = destination.Length; + int i = 0; + + if (Vector512.IsHardwareAccelerated) + { + Vector512 maximum = Vector512.Create(UShortMaximum); + Vector512 redWeight = Vector512.Create(0.2126F); + Vector512 greenWeight = Vector512.Create(0.7152F); + Vector512 blueWeight = Vector512.Create(0.0722F); + Vector512 roundingOffset = Vector512.Create(0.5F); + int oneVectorFromEnd = length - Vector512.Count; + for (; i <= oneVectorFromEnd; i += Vector512.Count) + { + Vector512 value = Vector512.Clamp( + Unsafe.As>(ref Unsafe.Add(ref sourceBase, i)), + Vector512.Zero, + Vector512.One) * maximum; + + // L16 uses its BT.709 luminance expression even when all three source components are equal. Preserve + // that exact arithmetic order so the SIMD path remains byte-identical to L16.FromScaledVector4. + Vector512 luminance = ((value * redWeight) + (value * greenWeight)) + (value * blueWeight); + Vector512 samples = Vector512.ConvertToInt32(luminance + roundingOffset); + Vector256 packed = Vector256.Narrow(samples.GetLower().AsUInt32(), samples.GetUpper().AsUInt32()); + packed.StoreUnsafe(ref Unsafe.As(ref Unsafe.Add(ref destinationBase, i))); + } + } + + if (Vector256.IsHardwareAccelerated) + { + Vector256 maximum = Vector256.Create(UShortMaximum); + Vector256 redWeight = Vector256.Create(0.2126F); + Vector256 greenWeight = Vector256.Create(0.7152F); + Vector256 blueWeight = Vector256.Create(0.0722F); + Vector256 roundingOffset = Vector256.Create(0.5F); + int oneVectorFromEnd = length - Vector256.Count; + for (; i <= oneVectorFromEnd; i += Vector256.Count) + { + Vector256 value = Vector256.Clamp( + Unsafe.As>(ref Unsafe.Add(ref sourceBase, i)), + Vector256.Zero, + Vector256.One) * maximum; + + Vector256 luminance = ((value * redWeight) + (value * greenWeight)) + (value * blueWeight); + Vector256 samples = Vector256.ConvertToInt32(luminance + roundingOffset); + Vector128 packed = Vector128.Narrow(samples.GetLower().AsUInt32(), samples.GetUpper().AsUInt32()); + packed.StoreUnsafe(ref Unsafe.As(ref Unsafe.Add(ref destinationBase, i))); + } + } + + if (Vector128.IsHardwareAccelerated) + { + Vector128 maximum = Vector128.Create(UShortMaximum); + Vector128 redWeight = Vector128.Create(0.2126F); + Vector128 greenWeight = Vector128.Create(0.7152F); + Vector128 blueWeight = Vector128.Create(0.0722F); + Vector128 roundingOffset = Vector128.Create(0.5F); + int oneVectorFromEnd = length - Vector128.Count; + for (; i <= oneVectorFromEnd; i += Vector128.Count) + { + Vector128 value = Vector128.Clamp( + Unsafe.As>(ref Unsafe.Add(ref sourceBase, i)), + Vector128.Zero, + Vector128.One) * maximum; + + Vector128 luminance = ((value * redWeight) + (value * greenWeight)) + (value * blueWeight); + Vector128 samples = Vector128.ConvertToInt32(luminance + roundingOffset); + Vector64 packed = Vector128.Narrow(samples.AsUInt32(), Vector128.Zero).GetLower(); + packed.StoreUnsafe(ref Unsafe.As(ref Unsafe.Add(ref destinationBase, i))); + } + } + + for (; i < length; i++) + { + Unsafe.Add(ref destinationBase, i) = L16.FromScaledVector4(new Vector4(Unsafe.Add(ref sourceBase, i))); + } + } + + /// + /// Reads an eight-bit or 16-bit unsigned sample without an intermediate conversion buffer. + /// + /// The encoded sample type. + /// The source samples. + /// The zero-based sample index. + /// The sample value. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static float GetSample(ReadOnlySpan source, int index) + where TSample : unmanaged + { + ref TSample sample = ref Unsafe.Add(ref MemoryMarshal.GetReference(source), index); + return typeof(TSample) == typeof(byte) + ? Unsafe.As(ref sample) + : Unsafe.As(ref sample); + } + + /// + /// Rounds and clamps a conversion result to the encoded sample range. + /// + /// The encoded sample type. + /// The conversion result. + /// The largest encoded sample value. + /// The bounded sample. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static TSample ToSample(float value, float maximum) + where TSample : unmanaged + { + int sample = Numerics.Clamp((int)MathF.Round(value, MidpointRounding.AwayFromZero), 0, (int)maximum); + if (typeof(TSample) == typeof(byte)) + { + byte result = (byte)sample; + return Unsafe.As(ref result); + } + + ushort highBitDepthResult = (ushort)sample; + return Unsafe.As(ref highBitDepthResult); + } + + /// + /// Applies the encoded component range and converts four lanes to bounded integer samples. + /// + /// The normalized component values. + /// The encoded range scale. + /// The encoded range bias. + /// The largest encoded sample value. + /// The bounded integer samples. + private static Vector128 ScaleBiasRoundAndClampToInt32(Vector128 value, float scale, float bias, float maximum) + { + Vector128 encoded = (value * Vector128.Create(scale)) + Vector128.Create(bias); + Vector128 bounded = Vector128.Clamp(encoded, Vector128.Zero, Vector128.Create(maximum)); + return Vector128.ConvertToInt32(Vector128.Round(bounded, MidpointRounding.AwayFromZero)); + } + + /// + /// Applies the encoded component range and converts eight lanes to bounded integer samples. + /// + /// The normalized component values. + /// The encoded range scale. + /// The encoded range bias. + /// The largest encoded sample value. + /// The bounded integer samples. + private static Vector256 ScaleBiasRoundAndClampToInt32(Vector256 value, float scale, float bias, float maximum) + { + Vector256 encoded = (value * Vector256.Create(scale)) + Vector256.Create(bias); + Vector256 bounded = Vector256.Clamp(encoded, Vector256.Zero, Vector256.Create(maximum)); + return Vector256.ConvertToInt32(Vector256.Round(bounded, MidpointRounding.AwayFromZero)); + } + + /// + /// Applies the encoded component range and converts sixteen lanes to bounded integer samples. + /// + /// The normalized component values. + /// The encoded range scale. + /// The encoded range bias. + /// The largest encoded sample value. + /// The bounded integer samples. + private static Vector512 ScaleBiasRoundAndClampToInt32(Vector512 value, float scale, float bias, float maximum) + { + Vector512 encoded = (value * Vector512.Create(scale)) + Vector512.Create(bias); + Vector512 bounded = Vector512.Clamp(encoded, Vector512.Zero, Vector512.Create(maximum)); + return Vector512.ConvertToInt32(Vector512.Round(bounded, MidpointRounding.AwayFromZero)); + } + + /// + /// Scales, rounds, and clamps four normalized components to integer storage values. + /// + /// The normalized component values. + /// The largest storage value. + /// The bounded integer values. + private static Vector128 ScaleRoundAndClampToInt32(Vector128 value, float maximum) + { + Vector128 scaled = value * Vector128.Create(maximum); + Vector128 bounded = Vector128.Min(Vector128.Max(scaled, Vector128.Zero), Vector128.Create(maximum)); + return Vector128.ConvertToInt32(Vector128.Round(bounded, MidpointRounding.AwayFromZero)); + } + + /// + /// Scales, rounds, and clamps eight normalized components to integer storage values. + /// + /// The normalized component values. + /// The largest storage value. + /// The bounded integer values. + private static Vector256 ScaleRoundAndClampToInt32(Vector256 value, float maximum) + { + Vector256 scaled = value * Vector256.Create(maximum); + Vector256 bounded = Vector256.Min(Vector256.Max(scaled, Vector256.Zero), Vector256.Create(maximum)); + return Vector256.ConvertToInt32(Vector256.Round(bounded, MidpointRounding.AwayFromZero)); + } + + /// + /// Scales, rounds, and clamps sixteen normalized components to integer storage values. + /// + /// The normalized component values. + /// The largest storage value. + /// The bounded integer values. + private static Vector512 ScaleRoundAndClampToInt32(Vector512 value, float maximum) + { + Vector512 scaled = value * Vector512.Create(maximum); + Vector512 bounded = Vector512.Min(Vector512.Max(scaled, Vector512.Zero), Vector512.Create(maximum)); + return Vector512.ConvertToInt32(Vector512.Round(bounded, MidpointRounding.AwayFromZero)); + } + + /// + /// Interleaves four red, green, and blue integer lanes into four opaque 16-bit RGBA pixels. + /// + /// The red component values. + /// The green component values. + /// The blue component values. + /// The first destination pixel. + private static void StoreRgba64Batch(Vector128 red, Vector128 green, Vector128 blue, ref Rgba64 destination) + { + Vector128 red16 = Vector128.Narrow(red.AsUInt32(), Vector128.Zero); + Vector128 green16 = Vector128.Narrow(green.AsUInt32(), Vector128.Zero); + Vector128 blue16 = Vector128.Narrow(blue.AsUInt32(), Vector128.Zero); + Vector128 alpha16 = Vector128.Create(ushort.MaxValue); + Vector128 redGreen = Vector128_.UnpackLow(red16.AsInt16(), green16.AsInt16()).AsUInt16(); + Vector128 blueAlpha = Vector128_.UnpackLow(blue16.AsInt16(), alpha16.AsInt16()).AsUInt16(); + Vector128 lower = Vector128_.UnpackLow(redGreen.AsInt32(), blueAlpha.AsInt32()).AsUInt32(); + Vector128 upper = Vector128_.UnpackHigh(redGreen.AsInt32(), blueAlpha.AsInt32()).AsUInt32(); + Unsafe.As>(ref destination) = lower; + Unsafe.As>(ref Unsafe.Add(ref destination, 2)) = upper; + } +} diff --git a/src/ImageSharp/Formats/Heif/Components/ColorConverters/HeifTransferFunctions.VectorOperations.cs b/src/ImageSharp/Formats/Heif/Components/ColorConverters/HeifTransferFunctions.VectorOperations.cs new file mode 100644 index 0000000000..d2f9a23843 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Components/ColorConverters/HeifTransferFunctions.VectorOperations.cs @@ -0,0 +1,807 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.Intrinsics; +using SixLabors.ImageSharp.Metadata.Profiles.Cicp; +using static SixLabors.ImageSharp.Formats.Heif.Components.HeifTransferVectorOperations; + +namespace SixLabors.ImageSharp.Formats.Heif.Components; + +/// +/// Provides fixed-width vector overloads and shared H.273 transfer operations for HEIF color conversion. One lane +/// represents one normalized color component. Closed vector operations implementations bind the 128-, 256-, or 512-bit implementation +/// once per row kernel, while conditional selection evaluates piecewise transfer curves without per-lane branches. +/// Inputs to logarithms and powers are bounded before evaluation because SIMD selection evaluates both branches. +/// +internal static partial class HeifTransferFunctions +{ + /// + /// Converts four nonlinear signal values to their H.273 linear-domain values. + /// + /// The signaled transfer characteristics. + /// The nonlinear signal values. + /// The corresponding linear-domain values. + public static Vector128 ToLinear(CicpTransferCharacteristics transferCharacteristics, Vector128 value) + => ToLinear, Vector128Operations>(transferCharacteristics, value); + + /// + /// Converts eight nonlinear signal values to their H.273 linear-domain values. + /// + /// The signaled transfer characteristics. + /// The nonlinear signal values. + /// The corresponding linear-domain values. + public static Vector256 ToLinear(CicpTransferCharacteristics transferCharacteristics, Vector256 value) + => ToLinear, Vector256Operations>(transferCharacteristics, value); + + /// + /// Converts sixteen nonlinear signal values to their H.273 linear-domain values. + /// + /// The signaled transfer characteristics. + /// The nonlinear signal values. + /// The corresponding linear-domain values. + public static Vector512 ToLinear(CicpTransferCharacteristics transferCharacteristics, Vector512 value) + => ToLinear, Vector512Operations>(transferCharacteristics, value); + + /// + /// Converts four linear signal values to their H.273 nonlinear-domain values. + /// + /// The signaled transfer characteristics. + /// The linear signal values. + /// The corresponding nonlinear-domain values. + public static Vector128 ToGamma(CicpTransferCharacteristics transferCharacteristics, Vector128 value) + => ToGamma, Vector128Operations>(transferCharacteristics, value); + + /// + /// Converts eight linear signal values to their H.273 nonlinear-domain values. + /// + /// The signaled transfer characteristics. + /// The linear signal values. + /// The corresponding nonlinear-domain values. + public static Vector256 ToGamma(CicpTransferCharacteristics transferCharacteristics, Vector256 value) + => ToGamma, Vector256Operations>(transferCharacteristics, value); + + /// + /// Converts sixteen linear signal values to their H.273 nonlinear-domain values. + /// + /// The signaled transfer characteristics. + /// The linear signal values. + /// The corresponding nonlinear-domain values. + public static Vector512 ToGamma(CicpTransferCharacteristics transferCharacteristics, Vector512 value) + => ToGamma, Vector512Operations>(transferCharacteristics, value); + + /// + /// Converts nonlinear signal values to their H.273 linear-domain values using the selected SIMD width. + /// + /// The SIMD vector type. + /// The operations for the SIMD vector type. + /// The signaled transfer characteristics. + /// The nonlinear signal values. + /// The corresponding linear-domain values. + private static TVector ToLinear(CicpTransferCharacteristics transferCharacteristics, TVector value) + where TVector : struct + where TOperations : struct, ITransferVectorOperations + { + TVector zero = TOperations.Create(0F); + TVector one = TOperations.Create(1F); + + switch (transferCharacteristics) + { + case CicpTransferCharacteristics.ItuRBt709_6: + case CicpTransferCharacteristics.ItuRBt601_7: + case CicpTransferCharacteristics.ItuRBt2020_2_10bit: + case CicpTransferCharacteristics.ItuRBt2020_2_12bit: + return ToLinearBt709(value); + case CicpTransferCharacteristics.Gamma2_2: + return Power(TOperations.Min(TOperations.Max(value, zero), one), 2.2F); + case CicpTransferCharacteristics.Gamma2_8: + return Power(TOperations.Min(TOperations.Max(value, zero), one), 2.8F); + case CicpTransferCharacteristics.SmpteSt240: + return ToLinearSmpte240(value); + case CicpTransferCharacteristics.Linear: + return TOperations.Min(TOperations.Max(value, zero), one); + case CicpTransferCharacteristics.Log100: + { + // H.273 assigns an interval to zero for logarithmic curves. The scalar midpoint convention is + // selected lane-wise after evaluating the positive branch, which keeps the hot path branchless. + TVector exponent = TOperations.Multiply(TOperations.Subtract(TOperations.Min(value, one), one), TOperations.Create(2F * 2.302585092994046F)); + TVector positive = TOperations.Exp(exponent); + return TOperations.ConditionalSelect(TOperations.LessThanOrEqual(value, zero), TOperations.Create(0.005F), positive); + } + + case CicpTransferCharacteristics.Log100Sqrt: + { + TVector exponent = TOperations.Multiply(TOperations.Subtract(TOperations.Min(value, one), one), TOperations.Create(2.5F * 2.302585092994046F)); + TVector positive = TOperations.Exp(exponent); + return TOperations.ConditionalSelect(TOperations.LessThanOrEqual(value, zero), TOperations.Create(0.00158113883F), positive); + } + + case CicpTransferCharacteristics.Iec61966_2_4: + return ToLinearIec61966(value); + case CicpTransferCharacteristics.ItuRBt1361_0: + return ToLinearBt1361(value); + case CicpTransferCharacteristics.Iec61966_2_1: + return ToLinearSrgb(value); + case CicpTransferCharacteristics.SmpteSt2084: + return ToLinearPq(value); + case CicpTransferCharacteristics.SmpteSt428_1: + return TOperations.Divide(Power(TOperations.Max(value, zero), 2.6F), TOperations.Create(Smpte428Scale)); + case CicpTransferCharacteristics.AribStdB67: + return ToLinearHlg(value); + default: + return ToLinearBt709(value); + } + } + + /// + /// Converts linear signal values to their H.273 nonlinear-domain values using the selected SIMD width. + /// + /// The SIMD vector type. + /// The operations for the SIMD vector type. + /// The signaled transfer characteristics. + /// The linear signal values. + /// The corresponding nonlinear-domain values. + private static TVector ToGamma(CicpTransferCharacteristics transferCharacteristics, TVector value) + where TVector : struct + where TOperations : struct, ITransferVectorOperations + { + TVector zero = TOperations.Create(0F); + TVector one = TOperations.Create(1F); + + switch (transferCharacteristics) + { + case CicpTransferCharacteristics.ItuRBt709_6: + case CicpTransferCharacteristics.ItuRBt601_7: + case CicpTransferCharacteristics.ItuRBt2020_2_10bit: + case CicpTransferCharacteristics.ItuRBt2020_2_12bit: + return ToGammaBt709(value); + case CicpTransferCharacteristics.Gamma2_2: + return Power(TOperations.Min(TOperations.Max(value, zero), one), 1F / 2.2F); + case CicpTransferCharacteristics.Gamma2_8: + return Power(TOperations.Min(TOperations.Max(value, zero), one), 1F / 2.8F); + case CicpTransferCharacteristics.SmpteSt240: + return ToGammaSmpte240(value); + case CicpTransferCharacteristics.Linear: + return TOperations.Min(TOperations.Max(value, zero), one); + case CicpTransferCharacteristics.Log100: + { + // Clamp inactive lanes to the threshold before Log. ConditionalSelect does not short-circuit, + // so this prevents negative input lanes from contaminating the vector operation with NaN values. + TVector threshold = TOperations.Create(0.01F); + TVector bounded = TOperations.Min(TOperations.Max(value, threshold), one); + TVector positive = TOperations.Add(one, TOperations.Divide(TOperations.Log(bounded), TOperations.Create(2F * 2.302585092994046F))); + return TOperations.ConditionalSelect(TOperations.LessThanOrEqual(value, threshold), zero, positive); + } + + case CicpTransferCharacteristics.Log100Sqrt: + { + TVector threshold = TOperations.Create(0.00316227766F); + TVector bounded = TOperations.Min(TOperations.Max(value, threshold), one); + TVector positive = TOperations.Add(one, TOperations.Divide(TOperations.Log(bounded), TOperations.Create(2.5F * 2.302585092994046F))); + return TOperations.ConditionalSelect(TOperations.LessThanOrEqual(value, threshold), zero, positive); + } + + case CicpTransferCharacteristics.Iec61966_2_4: + return ToGammaIec61966(value); + case CicpTransferCharacteristics.ItuRBt1361_0: + return ToGammaBt1361(value); + case CicpTransferCharacteristics.Iec61966_2_1: + return ToGammaSrgb(value); + case CicpTransferCharacteristics.SmpteSt2084: + return ToGammaPq(value); + case CicpTransferCharacteristics.SmpteSt428_1: + return Power(TOperations.Multiply(TOperations.Create(Smpte428Scale), TOperations.Max(value, zero)), 1F / 2.6F); + case CicpTransferCharacteristics.AribStdB67: + return ToGammaHlg(value); + default: + return ToGammaBt709(value); + } + } + + /// + /// Applies the inverse BT.709-family opto-electronic transfer function to a SIMD vector. + /// + /// The SIMD vector type. + /// The operations for the SIMD vector type. + /// The nonlinear signal values. + /// The linear signal values. + private static TVector ToLinearBt709(TVector value) + where TVector : struct + where TOperations : struct, ITransferVectorOperations + { + TVector zero = TOperations.Create(0F); + TVector one = TOperations.Create(1F); + TVector linear = TOperations.Divide(value, TOperations.Create(4.5F)); + TVector baseValue = TOperations.Divide(TOperations.Add(value, TOperations.Create(Bt709Alpha - 1F)), TOperations.Create(Bt709Alpha)); + TVector nonlinear = Power(TOperations.Max(baseValue, zero), 1F / 0.45F); + TVector belowOne = TOperations.ConditionalSelect(TOperations.LessThan(value, TOperations.Create(4.5F * Bt709Beta)), linear, nonlinear); + + // The comparisons deliberately mirror the scalar ordering. This preserves the H.273 lower and upper + // saturation rules while allowing all lanes to execute without data-dependent branches. + TVector bounded = TOperations.ConditionalSelect(TOperations.LessThan(value, one), belowOne, one); + return TOperations.ConditionalSelect(TOperations.LessThan(value, zero), zero, bounded); + } + + /// + /// Applies the BT.709-family opto-electronic transfer function to a SIMD vector. + /// + /// The SIMD vector type. + /// The operations for the SIMD vector type. + /// The linear signal values. + /// The nonlinear signal values. + private static TVector ToGammaBt709(TVector value) + where TVector : struct + where TOperations : struct, ITransferVectorOperations + { + TVector zero = TOperations.Create(0F); + TVector one = TOperations.Create(1F); + TVector linear = TOperations.Multiply(value, TOperations.Create(4.5F)); + TVector nonlinear = TOperations.Subtract( + TOperations.Multiply(TOperations.Create(Bt709Alpha), Power(TOperations.Max(value, zero), 0.45F)), + TOperations.Create(Bt709Alpha - 1F)); + TVector belowOne = TOperations.ConditionalSelect(TOperations.LessThan(value, TOperations.Create(Bt709Beta)), linear, nonlinear); + TVector bounded = TOperations.ConditionalSelect(TOperations.LessThan(value, one), belowOne, one); + return TOperations.ConditionalSelect(TOperations.LessThan(value, zero), zero, bounded); + } + + /// + /// Applies the inverse SMPTE ST 240 opto-electronic transfer function to a SIMD vector. + /// + /// The SIMD vector type. + /// The operations for the SIMD vector type. + /// The nonlinear signal values. + /// The linear signal values. + private static TVector ToLinearSmpte240(TVector value) + where TVector : struct + where TOperations : struct, ITransferVectorOperations + { + TVector zero = TOperations.Create(0F); + TVector one = TOperations.Create(1F); + TVector linear = TOperations.Divide(value, TOperations.Create(4F)); + TVector baseValue = TOperations.Divide(TOperations.Add(value, TOperations.Create(Smpte240Alpha - 1F)), TOperations.Create(Smpte240Alpha)); + TVector nonlinear = Power(TOperations.Max(baseValue, zero), 1F / 0.45F); + TVector belowOne = TOperations.ConditionalSelect(TOperations.LessThan(value, TOperations.Create(4F * Smpte240Beta)), linear, nonlinear); + TVector bounded = TOperations.ConditionalSelect(TOperations.LessThan(value, one), belowOne, one); + return TOperations.ConditionalSelect(TOperations.LessThan(value, zero), zero, bounded); + } + + /// + /// Applies the SMPTE ST 240 opto-electronic transfer function to a SIMD vector. + /// + /// The SIMD vector type. + /// The operations for the SIMD vector type. + /// The linear signal values. + /// The nonlinear signal values. + private static TVector ToGammaSmpte240(TVector value) + where TVector : struct + where TOperations : struct, ITransferVectorOperations + { + TVector zero = TOperations.Create(0F); + TVector one = TOperations.Create(1F); + TVector linear = TOperations.Multiply(value, TOperations.Create(4F)); + TVector nonlinear = TOperations.Subtract( + TOperations.Multiply(TOperations.Create(Smpte240Alpha), Power(TOperations.Max(value, zero), 0.45F)), + TOperations.Create(Smpte240Alpha - 1F)); + TVector belowOne = TOperations.ConditionalSelect(TOperations.LessThan(value, TOperations.Create(Smpte240Beta)), linear, nonlinear); + TVector bounded = TOperations.ConditionalSelect(TOperations.LessThan(value, one), belowOne, one); + return TOperations.ConditionalSelect(TOperations.LessThan(value, zero), zero, bounded); + } + + /// + /// Applies the inverse extended IEC 61966-2-4 transfer function to a SIMD vector. + /// + /// The SIMD vector type. + /// The operations for the SIMD vector type. + /// The nonlinear signal values. + /// The linear signal values. + private static TVector ToLinearIec61966(TVector value) + where TVector : struct + where TOperations : struct, ITransferVectorOperations + { + TVector negativeBase = TOperations.Divide(TOperations.Subtract(value, TOperations.Create(Bt709Alpha - 1F)), TOperations.Create(-Bt709Alpha)); + TVector negative = TOperations.Negate(Power(TOperations.Max(negativeBase, TOperations.Create(0F)), 1F / 0.45F)); + TVector linear = TOperations.Divide(value, TOperations.Create(4.5F)); + TVector positiveBase = TOperations.Divide(TOperations.Add(value, TOperations.Create(Bt709Alpha - 1F)), TOperations.Create(Bt709Alpha)); + TVector positive = Power(TOperations.Max(positiveBase, TOperations.Create(0F)), 1F / 0.45F); + TVector centerOrPositive = TOperations.ConditionalSelect(TOperations.LessThan(value, TOperations.Create(4.5F * Bt709Beta)), linear, positive); + return TOperations.ConditionalSelect(TOperations.LessThan(value, TOperations.Create(-4.5F * Bt709Beta)), negative, centerOrPositive); + } + + /// + /// Applies the extended IEC 61966-2-4 transfer function to a SIMD vector. + /// + /// The SIMD vector type. + /// The operations for the SIMD vector type. + /// The linear signal values. + /// The nonlinear signal values. + private static TVector ToGammaIec61966(TVector value) + where TVector : struct + where TOperations : struct, ITransferVectorOperations + { + TVector zero = TOperations.Create(0F); + TVector negative = TOperations.Add( + TOperations.Negate(TOperations.Multiply( + TOperations.Create(Bt709Alpha), + Power(TOperations.Max(TOperations.Negate(value), zero), 0.45F))), + TOperations.Create(Bt709Alpha - 1F)); + + TVector linear = TOperations.Multiply(value, TOperations.Create(4.5F)); + TVector positive = TOperations.Subtract( + TOperations.Multiply(TOperations.Create(Bt709Alpha), Power(TOperations.Max(value, zero), 0.45F)), + TOperations.Create(Bt709Alpha - 1F)); + TVector centerOrPositive = TOperations.ConditionalSelect(TOperations.LessThan(value, TOperations.Create(Bt709Beta)), linear, positive); + return TOperations.ConditionalSelect(TOperations.LessThan(value, TOperations.Create(-Bt709Beta)), negative, centerOrPositive); + } + + /// + /// Applies the inverse extended BT.1361 transfer function to a SIMD vector. + /// + /// The SIMD vector type. + /// The operations for the SIMD vector type. + /// The nonlinear signal values. + /// The linear signal values. + private static TVector ToLinearBt1361(TVector value) + where TVector : struct + where TOperations : struct, ITransferVectorOperations + { + TVector zero = TOperations.Create(0F); + TVector negativeBase = TOperations.Divide(TOperations.Subtract(value, TOperations.Create(0.02482420670236F)), TOperations.Create(-0.27482420670236F)); + TVector negative = TOperations.Divide(Power(TOperations.Max(negativeBase, zero), 1F / 0.45F), TOperations.Create(-4F)); + TVector negativeOrPositive = TOperations.ConditionalSelect(TOperations.LessThan(value, zero), negative, ToLinearBt709(value)); + return TOperations.ConditionalSelect(TOperations.LessThan(value, TOperations.Create(-0.25F)), TOperations.Create(-0.25F), negativeOrPositive); + } + + /// + /// Applies the extended BT.1361 transfer function to a SIMD vector. + /// + /// The SIMD vector type. + /// The operations for the SIMD vector type. + /// The linear signal values. + /// The nonlinear signal values. + private static TVector ToGammaBt1361(TVector value) + where TVector : struct + where TOperations : struct, ITransferVectorOperations + { + TVector zero = TOperations.Create(0F); + TVector negativePower = Power(TOperations.Max(TOperations.Multiply(TOperations.Create(-4F), value), zero), 0.45F); + TVector negative = TOperations.Add(TOperations.Multiply(TOperations.Create(-0.27482420670236F), negativePower), TOperations.Create(0.02482420670236F)); + TVector negativeOrPositive = TOperations.ConditionalSelect(TOperations.LessThan(value, zero), negative, ToGammaBt709(value)); + return TOperations.ConditionalSelect(TOperations.LessThan(value, TOperations.Create(-0.25F)), TOperations.Create(-0.25F), negativeOrPositive); + } + + /// + /// Applies the inverse extended IEC 61966-2-1 transfer function to a SIMD vector. + /// + /// The SIMD vector type. + /// The operations for the SIMD vector type. + /// The nonlinear signal values. + /// The linear signal values. + private static TVector ToLinearSrgb(TVector value) + where TVector : struct + where TOperations : struct, ITransferVectorOperations + { + TVector zero = TOperations.Create(0F); + TVector negativeBase = TOperations.Divide(TOperations.Subtract(value, TOperations.Create(SrgbAlpha - 1F)), TOperations.Create(-SrgbAlpha)); + TVector negative = TOperations.Negate(Power(TOperations.Max(negativeBase, zero), 2.4F)); + TVector linear = TOperations.Divide(value, TOperations.Create(12.92F)); + TVector positiveBase = TOperations.Divide(TOperations.Add(value, TOperations.Create(SrgbAlpha - 1F)), TOperations.Create(SrgbAlpha)); + TVector positive = Power(TOperations.Max(positiveBase, zero), 2.4F); + TVector centerOrPositive = TOperations.ConditionalSelect(TOperations.LessThan(value, TOperations.Create(12.92F * SrgbBeta)), linear, positive); + return TOperations.ConditionalSelect(TOperations.LessThan(value, TOperations.Create(-12.92F * SrgbBeta)), negative, centerOrPositive); + } + + /// + /// Applies the extended IEC 61966-2-1 transfer function to a SIMD vector. + /// + /// The SIMD vector type. + /// The operations for the SIMD vector type. + /// The linear signal values. + /// The nonlinear signal values. + private static TVector ToGammaSrgb(TVector value) + where TVector : struct + where TOperations : struct, ITransferVectorOperations + { + TVector zero = TOperations.Create(0F); + TVector negative = TOperations.Add( + TOperations.Negate(TOperations.Multiply( + TOperations.Create(SrgbAlpha), + Power(TOperations.Max(TOperations.Negate(value), zero), 1F / 2.4F))), + TOperations.Create(SrgbAlpha - 1F)); + + TVector linear = TOperations.Multiply(value, TOperations.Create(12.92F)); + TVector positive = TOperations.Subtract( + TOperations.Multiply(TOperations.Create(SrgbAlpha), Power(TOperations.Max(value, zero), 1F / 2.4F)), + TOperations.Create(SrgbAlpha - 1F)); + TVector centerOrPositive = TOperations.ConditionalSelect(TOperations.LessThan(value, TOperations.Create(SrgbBeta)), linear, positive); + return TOperations.ConditionalSelect(TOperations.LessThan(value, TOperations.Create(-SrgbBeta)), negative, centerOrPositive); + } + + /// + /// Applies the inverse SMPTE ST 2084 perceptual-quantizer transfer function to a SIMD vector. + /// + /// The SIMD vector type. + /// The operations for the SIMD vector type. + /// The nonlinear signal values. + /// The normalized linear signal values. + private static TVector ToLinearPq(TVector value) + where TVector : struct + where TOperations : struct, ITransferVectorOperations + { + TVector zero = TOperations.Create(0F); + TVector nonlinearPower = Power(TOperations.Min(TOperations.Max(value, zero), TOperations.Create(1F)), 1F / PqM); + TVector numerator = TOperations.Max(TOperations.Subtract(nonlinearPower, TOperations.Create(PqC1)), zero); + TVector denominator = TOperations.Subtract(TOperations.Create(PqC2), TOperations.Multiply(TOperations.Create(PqC3), nonlinearPower)); + TVector positive = TOperations.Min(Power(TOperations.Divide(numerator, denominator), 1F / PqN), TOperations.Create(1F)); + return TOperations.ConditionalSelect(TOperations.LessThanOrEqual(value, zero), zero, positive); + } + + /// + /// Applies the SMPTE ST 2084 perceptual-quantizer transfer function to a SIMD vector. + /// + /// The SIMD vector type. + /// The operations for the SIMD vector type. + /// The normalized linear signal values. + /// The nonlinear signal values. + private static TVector ToGammaPq(TVector value) + where TVector : struct + where TOperations : struct, ITransferVectorOperations + { + TVector zero = TOperations.Create(0F); + TVector linearPower = Power(TOperations.Min(TOperations.Max(value, zero), TOperations.Create(1F)), PqN); + TVector numerator = TOperations.Add(TOperations.Create(PqC1), TOperations.Multiply(TOperations.Create(PqC2), linearPower)); + TVector denominator = TOperations.Add(TOperations.Create(1F), TOperations.Multiply(TOperations.Create(PqC3), linearPower)); + TVector positive = TOperations.Min(Power(TOperations.Divide(numerator, denominator), PqM), TOperations.Create(1F)); + return TOperations.ConditionalSelect(TOperations.LessThanOrEqual(value, zero), zero, positive); + } + + /// + /// Applies the inverse HLG opto-electronic transfer function to a SIMD vector. + /// + /// The SIMD vector type. + /// The operations for the SIMD vector type. + /// The nonlinear signal values. + /// The normalized linear signal values. + private static TVector ToLinearHlg(TVector value) + where TVector : struct + where TOperations : struct, ITransferVectorOperations + { + TVector zero = TOperations.Create(0F); + TVector bounded = TOperations.Min(TOperations.Max(value, zero), TOperations.Create(1F)); + TVector linear = TOperations.Divide(TOperations.Multiply(bounded, bounded), TOperations.Create(3F)); + TVector exponent = TOperations.Divide(TOperations.Subtract(bounded, TOperations.Create(HlgC)), TOperations.Create(HlgA)); + TVector logarithmic = TOperations.Divide(TOperations.Add(TOperations.Exp(exponent), TOperations.Create(HlgB)), TOperations.Create(12F)); + TVector positive = TOperations.ConditionalSelect(TOperations.LessThanOrEqual(value, TOperations.Create(0.5F)), linear, logarithmic); + return TOperations.ConditionalSelect(TOperations.LessThanOrEqual(value, zero), zero, positive); + } + + /// + /// Applies the HLG opto-electronic transfer function to a SIMD vector. + /// + /// The SIMD vector type. + /// The operations for the SIMD vector type. + /// The normalized linear signal values. + /// The nonlinear signal values. + private static TVector ToGammaHlg(TVector value) + where TVector : struct + where TOperations : struct, ITransferVectorOperations + { + TVector zero = TOperations.Create(0F); + TVector bounded = TOperations.Min(TOperations.Max(value, zero), TOperations.Create(1F)); + TVector linear = TOperations.Sqrt(TOperations.Multiply(TOperations.Create(3F), bounded)); + + // Clamp the logarithm input for inactive lanes. SIMD conditional selection evaluates both branches, + // while the scalar definition evaluates Log only above the 1/12 transition. + TVector logarithmInput = TOperations.Max( + TOperations.Subtract(TOperations.Multiply(TOperations.Create(12F), bounded), TOperations.Create(HlgB)), + TOperations.Create(float.Epsilon)); + TVector logarithmic = TOperations.Add(TOperations.Multiply(TOperations.Create(HlgA), TOperations.Log(logarithmInput)), TOperations.Create(HlgC)); + TVector positive = TOperations.ConditionalSelect(TOperations.LessThanOrEqual(bounded, TOperations.Create(1F / 12F)), linear, logarithmic); + return TOperations.ConditionalSelect(TOperations.LessThanOrEqual(value, zero), zero, positive); + } + + /// + /// Raises nonnegative SIMD values to a scalar exponent. + /// + /// The SIMD vector type. + /// The operations for the SIMD vector type. + /// The nonnegative base values. + /// The exponent applied to every lane. + /// The exponentiated values. + private static TVector Power(TVector value, float exponent) + where TVector : struct + where TOperations : struct, ITransferVectorOperations + { + // System.Numerics.Tensors does not currently vectorize Pow. Expressing positive powers as Exp(Log(x) * y) + // uses the .NET 10 cross-platform vector math kernels and keeps all transfer-function lanes in SIMD. + return TOperations.Exp(TOperations.Multiply(TOperations.Log(value), TOperations.Create(exponent))); + } +} + +/// +/// Contains the vector-width operations used by the shared H.273 transfer-function formulas. +/// +internal static class HeifTransferVectorOperations +{ + /// + /// Defines the lane-wise operations required by the shared H.273 SIMD formulas. + /// + /// The SIMD vector type. + public interface ITransferVectorOperations + where TVector : struct + { + /// + /// Creates a vector whose lanes contain the specified value. + /// + /// The value copied to every lane. + /// The created vector. + public static abstract TVector Create(float value); + + /// + /// Adds corresponding vector lanes. + /// + /// The left operand. + /// The right operand. + /// The lane-wise sum. + public static abstract TVector Add(TVector left, TVector right); + + /// + /// Subtracts corresponding vector lanes. + /// + /// The left operand. + /// The right operand. + /// The lane-wise difference. + public static abstract TVector Subtract(TVector left, TVector right); + + /// + /// Multiplies corresponding vector lanes. + /// + /// The left operand. + /// The right operand. + /// The lane-wise product. + public static abstract TVector Multiply(TVector left, TVector right); + + /// + /// Multiplies corresponding vector lanes and adds an addend using the fastest supported estimate. + /// + /// The left multiplication operand. + /// The right multiplication operand. + /// The value added to the product. + /// The lane-wise multiply-add result. + public static abstract TVector MultiplyAddEstimate(TVector left, TVector right, TVector addend); + + /// + /// Divides corresponding vector lanes. + /// + /// The dividend. + /// The divisor. + /// The lane-wise quotient. + public static abstract TVector Divide(TVector left, TVector right); + + /// + /// Negates every vector lane. + /// + /// The input vector. + /// The negated vector. + public static abstract TVector Negate(TVector value); + + /// + /// Selects the smaller value in each pair of lanes. + /// + /// The left operand. + /// The right operand. + /// The lane-wise minimum. + public static abstract TVector Min(TVector left, TVector right); + + /// + /// Selects the larger value in each pair of lanes. + /// + /// The left operand. + /// The right operand. + /// The lane-wise maximum. + public static abstract TVector Max(TVector left, TVector right); + + /// + /// Compares whether each left lane is less than its right lane. + /// + /// The left operand. + /// The right operand. + /// The comparison mask. + public static abstract TVector LessThan(TVector left, TVector right); + + /// + /// Compares whether each left lane is less than or equal to its right lane. + /// + /// The left operand. + /// The right operand. + /// The comparison mask. + public static abstract TVector LessThanOrEqual(TVector left, TVector right); + + /// + /// Selects lanes from two vectors according to a comparison mask. + /// + /// The comparison mask. + /// The value selected for set mask lanes. + /// The value selected for clear mask lanes. + /// The selected lanes. + public static abstract TVector ConditionalSelect(TVector condition, TVector left, TVector right); + + /// + /// Computes the natural exponential of every vector lane. + /// + /// The input vector. + /// The lane-wise exponential. + public static abstract TVector Exp(TVector value); + + /// + /// Computes the natural logarithm of every vector lane. + /// + /// The input vector. + /// The lane-wise logarithm. + public static abstract TVector Log(TVector value); + + /// + /// Computes the square root of every vector lane. + /// + /// The input vector. + /// The lane-wise square root. + public static abstract TVector Sqrt(TVector value); + } + + /// + /// Maps the shared transfer-function formulas to 128-bit vector operations. + /// + public readonly struct Vector128Operations : ITransferVectorOperations> + { + /// + public static Vector128 Create(float value) => Vector128.Create(value); + + /// + public static Vector128 Add(Vector128 left, Vector128 right) => left + right; + + /// + public static Vector128 Subtract(Vector128 left, Vector128 right) => left - right; + + /// + public static Vector128 Multiply(Vector128 left, Vector128 right) => left * right; + + /// + public static Vector128 MultiplyAddEstimate(Vector128 left, Vector128 right, Vector128 addend) + => Vector128.MultiplyAddEstimate(left, right, addend); + + /// + public static Vector128 Divide(Vector128 left, Vector128 right) => left / right; + + /// + public static Vector128 Negate(Vector128 value) => -value; + + /// + public static Vector128 Min(Vector128 left, Vector128 right) => Vector128.Min(left, right); + + /// + public static Vector128 Max(Vector128 left, Vector128 right) => Vector128.Max(left, right); + + /// + public static Vector128 LessThan(Vector128 left, Vector128 right) => Vector128.LessThan(left, right); + + /// + public static Vector128 LessThanOrEqual(Vector128 left, Vector128 right) => Vector128.LessThanOrEqual(left, right); + + /// + public static Vector128 ConditionalSelect(Vector128 condition, Vector128 left, Vector128 right) + => Vector128.ConditionalSelect(condition, left, right); + + /// + public static Vector128 Exp(Vector128 value) => Vector128.Exp(value); + + /// + public static Vector128 Log(Vector128 value) => Vector128.Log(value); + + /// + public static Vector128 Sqrt(Vector128 value) => Vector128.Sqrt(value); + } + + /// + /// Maps the shared transfer-function formulas to 256-bit vector operations. + /// + public readonly struct Vector256Operations : ITransferVectorOperations> + { + /// + public static Vector256 Create(float value) => Vector256.Create(value); + + /// + public static Vector256 Add(Vector256 left, Vector256 right) => left + right; + + /// + public static Vector256 Subtract(Vector256 left, Vector256 right) => left - right; + + /// + public static Vector256 Multiply(Vector256 left, Vector256 right) => left * right; + + /// + public static Vector256 MultiplyAddEstimate(Vector256 left, Vector256 right, Vector256 addend) + => Vector256.MultiplyAddEstimate(left, right, addend); + + /// + public static Vector256 Divide(Vector256 left, Vector256 right) => left / right; + + /// + public static Vector256 Negate(Vector256 value) => -value; + + /// + public static Vector256 Min(Vector256 left, Vector256 right) => Vector256.Min(left, right); + + /// + public static Vector256 Max(Vector256 left, Vector256 right) => Vector256.Max(left, right); + + /// + public static Vector256 LessThan(Vector256 left, Vector256 right) => Vector256.LessThan(left, right); + + /// + public static Vector256 LessThanOrEqual(Vector256 left, Vector256 right) => Vector256.LessThanOrEqual(left, right); + + /// + public static Vector256 ConditionalSelect(Vector256 condition, Vector256 left, Vector256 right) + => Vector256.ConditionalSelect(condition, left, right); + + /// + public static Vector256 Exp(Vector256 value) => Vector256.Exp(value); + + /// + public static Vector256 Log(Vector256 value) => Vector256.Log(value); + + /// + public static Vector256 Sqrt(Vector256 value) => Vector256.Sqrt(value); + } + + /// + /// Maps the shared transfer-function formulas to 512-bit vector operations. + /// + public readonly struct Vector512Operations : ITransferVectorOperations> + { + /// + public static Vector512 Create(float value) => Vector512.Create(value); + + /// + public static Vector512 Add(Vector512 left, Vector512 right) => left + right; + + /// + public static Vector512 Subtract(Vector512 left, Vector512 right) => left - right; + + /// + public static Vector512 Multiply(Vector512 left, Vector512 right) => left * right; + + /// + public static Vector512 MultiplyAddEstimate(Vector512 left, Vector512 right, Vector512 addend) + => Vector512.MultiplyAddEstimate(left, right, addend); + + /// + public static Vector512 Divide(Vector512 left, Vector512 right) => left / right; + + /// + public static Vector512 Negate(Vector512 value) => -value; + + /// + public static Vector512 Min(Vector512 left, Vector512 right) => Vector512.Min(left, right); + + /// + public static Vector512 Max(Vector512 left, Vector512 right) => Vector512.Max(left, right); + + /// + public static Vector512 LessThan(Vector512 left, Vector512 right) => Vector512.LessThan(left, right); + + /// + public static Vector512 LessThanOrEqual(Vector512 left, Vector512 right) => Vector512.LessThanOrEqual(left, right); + + /// + public static Vector512 ConditionalSelect(Vector512 condition, Vector512 left, Vector512 right) + => Vector512.ConditionalSelect(condition, left, right); + + /// + public static Vector512 Exp(Vector512 value) => Vector512.Exp(value); + + /// + public static Vector512 Log(Vector512 value) => Vector512.Log(value); + + /// + public static Vector512 Sqrt(Vector512 value) => Vector512.Sqrt(value); + } +} diff --git a/src/ImageSharp/Formats/Heif/Components/ColorConverters/HeifTransferFunctions.cs b/src/ImageSharp/Formats/Heif/Components/ColorConverters/HeifTransferFunctions.cs new file mode 100644 index 0000000000..fb1edff970 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Components/ColorConverters/HeifTransferFunctions.cs @@ -0,0 +1,444 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Metadata.Profiles.Cicp; + +namespace SixLabors.ImageSharp.Formats.Heif.Components; + +/// +/// Applies the H.273 transfer characteristics used by HEIF color conversion. +/// +internal static partial class HeifTransferFunctions +{ + /// + /// The BT.709 and BT.2020 nonlinear scale factor. + /// + private const float Bt709Alpha = 1.09929682680944F; + + /// + /// The BT.709 and BT.2020 linear-domain transition point. + /// + private const float Bt709Beta = 0.018053968510807F; + + /// + /// The SMPTE ST 240 nonlinear scale factor. + /// + private const float Smpte240Alpha = 1.111572195921731F; + + /// + /// The SMPTE ST 240 linear-domain transition point. + /// + private const float Smpte240Beta = 0.022821585529445F; + + /// + /// The sRGB nonlinear scale factor. + /// + private const float SrgbAlpha = 1.0550107189475866F; + + /// + /// The sRGB linear-domain transition point. + /// + private const float SrgbBeta = 0.0030412825601275209F; + + /// + /// The SMPTE ST 2084 first rational constant. + /// + private const float PqC1 = 0.8359375F; + + /// + /// The SMPTE ST 2084 numerator scale. + /// + private const float PqC2 = 18.8515625F; + + /// + /// The SMPTE ST 2084 denominator scale. + /// + private const float PqC3 = 18.6875F; + + /// + /// The SMPTE ST 2084 outer exponent. + /// + private const float PqM = 78.84375F; + + /// + /// The SMPTE ST 2084 inner exponent. + /// + private const float PqN = 0.1593017578125F; + + /// + /// The SMPTE ST 428 luminance normalization factor. + /// + private const float Smpte428Scale = 0.91655527974030934F; + + /// + /// The HLG logarithmic scale. + /// + private const float HlgA = 0.17883277F; + + /// + /// The HLG logarithmic offset. + /// + private const float HlgB = 0.28466892F; + + /// + /// The HLG output offset. + /// + private const float HlgC = 0.55991073F; + + /// + /// Converts a nonlinear signal value to its H.273 linear-domain value. + /// + /// The signaled transfer characteristics. + /// The nonlinear signal value. + /// The corresponding linear-domain value. + public static float ToLinear(CicpTransferCharacteristics transferCharacteristics, float value) + { + switch (transferCharacteristics) + { + case CicpTransferCharacteristics.ItuRBt709_6: + case CicpTransferCharacteristics.ItuRBt601_7: + case CicpTransferCharacteristics.ItuRBt2020_2_10bit: + case CicpTransferCharacteristics.ItuRBt2020_2_12bit: + return ToLinearBt709(value); + case CicpTransferCharacteristics.Gamma2_2: + return MathF.Pow(Math.Clamp(value, 0F, 1F), 2.2F); + case CicpTransferCharacteristics.Gamma2_8: + return MathF.Pow(Math.Clamp(value, 0F, 1F), 2.8F); + case CicpTransferCharacteristics.SmpteSt240: + return ToLinearSmpte240(value); + case CicpTransferCharacteristics.Linear: + return Math.Clamp(value, 0F, 1F); + case CicpTransferCharacteristics.Log100: + // Zero represents an interval rather than one linear value. The midpoint matches libavif and + // minimizes the worst-case round-trip error when constant-luminance content is decoded. + return value <= 0F ? 0.005F : MathF.Pow(10F, 2F * (MathF.Min(value, 1F) - 1F)); + case CicpTransferCharacteristics.Log100Sqrt: + return value <= 0F ? 0.00158113883F : MathF.Pow(10F, 2.5F * (MathF.Min(value, 1F) - 1F)); + case CicpTransferCharacteristics.Iec61966_2_4: + return ToLinearIec61966(value); + case CicpTransferCharacteristics.ItuRBt1361_0: + return ToLinearBt1361(value); + case CicpTransferCharacteristics.Iec61966_2_1: + return ToLinearSrgb(value); + case CicpTransferCharacteristics.SmpteSt2084: + return ToLinearPq(value); + case CicpTransferCharacteristics.SmpteSt428_1: + return MathF.Pow(MathF.Max(value, 0F), 2.6F) / Smpte428Scale; + case CicpTransferCharacteristics.AribStdB67: + return ToLinearHlg(value); + default: + // H.273 leaves unspecified and reserved transfer values to the application. Match libavif's + // deterministic BT.709 fallback for still-image conversion. + return ToLinearBt709(value); + } + } + + /// + /// Converts a linear signal value to its H.273 nonlinear-domain value. + /// + /// The signaled transfer characteristics. + /// The linear signal value. + /// The corresponding nonlinear-domain value. + public static float ToGamma(CicpTransferCharacteristics transferCharacteristics, float value) + { + switch (transferCharacteristics) + { + case CicpTransferCharacteristics.ItuRBt709_6: + case CicpTransferCharacteristics.ItuRBt601_7: + case CicpTransferCharacteristics.ItuRBt2020_2_10bit: + case CicpTransferCharacteristics.ItuRBt2020_2_12bit: + return ToGammaBt709(value); + case CicpTransferCharacteristics.Gamma2_2: + return MathF.Pow(Math.Clamp(value, 0F, 1F), 1F / 2.2F); + case CicpTransferCharacteristics.Gamma2_8: + return MathF.Pow(Math.Clamp(value, 0F, 1F), 1F / 2.8F); + case CicpTransferCharacteristics.SmpteSt240: + return ToGammaSmpte240(value); + case CicpTransferCharacteristics.Linear: + return Math.Clamp(value, 0F, 1F); + case CicpTransferCharacteristics.Log100: + return value <= 0.01F ? 0F : 1F + (MathF.Log10(MathF.Min(value, 1F)) / 2F); + case CicpTransferCharacteristics.Log100Sqrt: + return value <= 0.00316227766F ? 0F : 1F + (MathF.Log10(MathF.Min(value, 1F)) / 2.5F); + case CicpTransferCharacteristics.Iec61966_2_4: + return ToGammaIec61966(value); + case CicpTransferCharacteristics.ItuRBt1361_0: + return ToGammaBt1361(value); + case CicpTransferCharacteristics.Iec61966_2_1: + return ToGammaSrgb(value); + case CicpTransferCharacteristics.SmpteSt2084: + return ToGammaPq(value); + case CicpTransferCharacteristics.SmpteSt428_1: + return MathF.Pow(Smpte428Scale * MathF.Max(value, 0F), 1F / 2.6F); + case CicpTransferCharacteristics.AribStdB67: + return ToGammaHlg(value); + default: + return ToGammaBt709(value); + } + } + + /// + /// Applies the inverse BT.709-family opto-electronic transfer function. + /// + /// The nonlinear signal value. + /// The linear signal value. + private static float ToLinearBt709(float value) + { + if (value < 0F) + { + return 0F; + } + + if (value < 4.5F * Bt709Beta) + { + return value / 4.5F; + } + + return value < 1F + ? MathF.Pow((value + (Bt709Alpha - 1F)) / Bt709Alpha, 1F / 0.45F) + : 1F; + } + + /// + /// Applies the BT.709-family opto-electronic transfer function. + /// + /// The linear signal value. + /// The nonlinear signal value. + private static float ToGammaBt709(float value) + { + if (value < 0F) + { + return 0F; + } + + if (value < Bt709Beta) + { + return value * 4.5F; + } + + return value < 1F + ? (Bt709Alpha * MathF.Pow(value, 0.45F)) - (Bt709Alpha - 1F) + : 1F; + } + + /// + /// Applies the inverse SMPTE ST 240 opto-electronic transfer function. + /// + /// The nonlinear signal value. + /// The linear signal value. + private static float ToLinearSmpte240(float value) + { + if (value < 0F) + { + return 0F; + } + + if (value < 4F * Smpte240Beta) + { + return value / 4F; + } + + return value < 1F + ? MathF.Pow((value + (Smpte240Alpha - 1F)) / Smpte240Alpha, 1F / 0.45F) + : 1F; + } + + /// + /// Applies the SMPTE ST 240 opto-electronic transfer function. + /// + /// The linear signal value. + /// The nonlinear signal value. + private static float ToGammaSmpte240(float value) + { + if (value < 0F) + { + return 0F; + } + + if (value < Smpte240Beta) + { + return value * 4F; + } + + return value < 1F + ? (Smpte240Alpha * MathF.Pow(value, 0.45F)) - (Smpte240Alpha - 1F) + : 1F; + } + + /// + /// Applies the inverse extended IEC 61966-2-4 transfer function. + /// + /// The nonlinear signal value. + /// The linear signal value. + private static float ToLinearIec61966(float value) + { + if (value < -4.5F * Bt709Beta) + { + return -MathF.Pow((value - (Bt709Alpha - 1F)) / -Bt709Alpha, 1F / 0.45F); + } + + return value < 4.5F * Bt709Beta + ? value / 4.5F + : MathF.Pow((value + (Bt709Alpha - 1F)) / Bt709Alpha, 1F / 0.45F); + } + + /// + /// Applies the extended IEC 61966-2-4 transfer function. + /// + /// The linear signal value. + /// The nonlinear signal value. + private static float ToGammaIec61966(float value) + { + if (value < -Bt709Beta) + { + return (-Bt709Alpha * MathF.Pow(-value, 0.45F)) + (Bt709Alpha - 1F); + } + + return value < Bt709Beta + ? value * 4.5F + : (Bt709Alpha * MathF.Pow(value, 0.45F)) - (Bt709Alpha - 1F); + } + + /// + /// Applies the inverse extended BT.1361 transfer function. + /// + /// The nonlinear signal value. + /// The linear signal value. + private static float ToLinearBt1361(float value) + { + if (value < -0.25F) + { + return -0.25F; + } + + if (value < 0F) + { + return MathF.Pow((value - 0.02482420670236F) / -0.27482420670236F, 1F / 0.45F) / -4F; + } + + return ToLinearBt709(value); + } + + /// + /// Applies the extended BT.1361 transfer function. + /// + /// The linear signal value. + /// The nonlinear signal value. + private static float ToGammaBt1361(float value) + { + if (value < -0.25F) + { + return -0.25F; + } + + if (value < 0F) + { + return (-0.27482420670236F * MathF.Pow(-4F * value, 0.45F)) + 0.02482420670236F; + } + + return ToGammaBt709(value); + } + + /// + /// Applies the inverse extended IEC 61966-2-1 transfer function. + /// + /// The nonlinear signal value. + /// The linear signal value. + private static float ToLinearSrgb(float value) + { + if (value < -12.92F * SrgbBeta) + { + return -MathF.Pow((value - (SrgbAlpha - 1F)) / -SrgbAlpha, 2.4F); + } + + return value < 12.92F * SrgbBeta + ? value / 12.92F + : MathF.Pow((value + (SrgbAlpha - 1F)) / SrgbAlpha, 2.4F); + } + + /// + /// Applies the extended IEC 61966-2-1 transfer function. + /// + /// The linear signal value. + /// The nonlinear signal value. + private static float ToGammaSrgb(float value) + { + if (value < -SrgbBeta) + { + return (-SrgbAlpha * MathF.Pow(-value, 1F / 2.4F)) + (SrgbAlpha - 1F); + } + + return value < SrgbBeta + ? value * 12.92F + : (SrgbAlpha * MathF.Pow(value, 1F / 2.4F)) - (SrgbAlpha - 1F); + } + + /// + /// Applies the inverse SMPTE ST 2084 perceptual-quantizer transfer function. + /// + /// The nonlinear signal value. + /// The normalized linear signal value. + private static float ToLinearPq(float value) + { + if (value <= 0F) + { + return 0F; + } + + float nonlinearPower = MathF.Pow(MathF.Min(value, 1F), 1F / PqM); + float numerator = MathF.Max(nonlinearPower - PqC1, 0F); + float denominator = PqC2 - (PqC3 * nonlinearPower); + return MathF.Pow(numerator / denominator, 1F / PqN); + } + + /// + /// Applies the SMPTE ST 2084 perceptual-quantizer transfer function. + /// + /// The normalized linear signal value. + /// The nonlinear signal value. + private static float ToGammaPq(float value) + { + if (value <= 0F) + { + return 0F; + } + + float linearPower = MathF.Pow(MathF.Min(value, 1F), PqN); + return MathF.Pow((PqC1 + (PqC2 * linearPower)) / (1F + (PqC3 * linearPower)), PqM); + } + + /// + /// Applies the inverse HLG opto-electronic transfer function. + /// + /// The nonlinear signal value. + /// The normalized linear signal value. + private static float ToLinearHlg(float value) + { + if (value <= 0F) + { + return 0F; + } + + return value <= 0.5F + ? (value * value) / 3F + : (MathF.Exp((MathF.Min(value, 1F) - HlgC) / HlgA) + HlgB) / 12F; + } + + /// + /// Applies the HLG opto-electronic transfer function. + /// + /// The normalized linear signal value. + /// The nonlinear signal value. + private static float ToGammaHlg(float value) + { + if (value <= 0F) + { + return 0F; + } + + float bounded = MathF.Min(value, 1F); + return bounded <= 1F / 12F + ? MathF.Sqrt(3F * bounded) + : (HlgA * MathF.Log((12F * bounded) - HlgB)) + HlgC; + } +} diff --git a/src/ImageSharp/Formats/Heif/Components/ColorConverters/HeifYuvToRgb8Converter.FixedPointCoefficientOperator.cs b/src/ImageSharp/Formats/Heif/Components/ColorConverters/HeifYuvToRgb8Converter.FixedPointCoefficientOperator.cs new file mode 100644 index 0000000000..82b6b7d8f8 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Components/ColorConverters/HeifYuvToRgb8Converter.FixedPointCoefficientOperator.cs @@ -0,0 +1,97 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Components; + +/// +/// Provides full-range fixed-point coefficient conversion. Each signed 32-bit lane carries one luma or duplicated +/// chroma sample. Matrix coefficients use a common fixed-point scale; every component rounds once after its complete +/// weighted sum, then clips before the enclosing row kernel narrows and packs the RGB result. +/// +internal static partial class HeifYuvToRgb8Converter +{ + /// + /// Implements full-range coefficient-based YCbCr conversion for scalar and SIMD lanes. + /// + private readonly struct FixedPointCoefficientOperator : IHeifYuvToRgb8Operator + { + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void Convert( + Vector512 y, + Vector512 cb, + Vector512 cr, + in ConversionParameters parameters, + out Vector512 r, + out Vector512 g, + out Vector512 b) + { + Vector512Parameters fixedPoint = parameters.FixedPointSixteenLane; + cb -= fixedPoint.ChromaMidpoint; + cr -= fixedPoint.ChromaMidpoint; + r = Vector512.Clamp(y + (((fixedPoint.RedCr * cr) + fixedPoint.RoundingBias) >> CoefficientShift), default, fixedPoint.Maximum); + g = Vector512.Clamp(y + (((fixedPoint.GreenCb * cb) + (fixedPoint.GreenCr * cr) + fixedPoint.RoundingBias) >> CoefficientShift), default, fixedPoint.Maximum); + b = Vector512.Clamp(y + (((fixedPoint.BlueCb * cb) + fixedPoint.RoundingBias) >> CoefficientShift), default, fixedPoint.Maximum); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void Convert( + Vector256 y, + Vector256 cb, + Vector256 cr, + in ConversionParameters parameters, + out Vector256 r, + out Vector256 g, + out Vector256 b) + { + Vector256Parameters fixedPoint = parameters.FixedPointEightLane; + cb -= fixedPoint.ChromaMidpoint; + cr -= fixedPoint.ChromaMidpoint; + r = Vector256.Clamp(y + (((fixedPoint.RedCr * cr) + fixedPoint.RoundingBias) >> CoefficientShift), default, fixedPoint.Maximum); + g = Vector256.Clamp(y + (((fixedPoint.GreenCb * cb) + (fixedPoint.GreenCr * cr) + fixedPoint.RoundingBias) >> CoefficientShift), default, fixedPoint.Maximum); + b = Vector256.Clamp(y + (((fixedPoint.BlueCb * cb) + fixedPoint.RoundingBias) >> CoefficientShift), default, fixedPoint.Maximum); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void Convert( + Vector128 y, + Vector128 cb, + Vector128 cr, + in ConversionParameters parameters, + out Vector128 r, + out Vector128 g, + out Vector128 b) + { + Vector128Parameters fixedPoint = parameters.FixedPointFourLane; + cb -= fixedPoint.ChromaMidpoint; + cr -= fixedPoint.ChromaMidpoint; + r = Vector128.Clamp(y + (((fixedPoint.RedCr * cr) + fixedPoint.RoundingBias) >> CoefficientShift), default, fixedPoint.Maximum); + g = Vector128.Clamp(y + (((fixedPoint.GreenCb * cb) + (fixedPoint.GreenCr * cr) + fixedPoint.RoundingBias) >> CoefficientShift), default, fixedPoint.Maximum); + b = Vector128.Clamp(y + (((fixedPoint.BlueCb * cb) + fixedPoint.RoundingBias) >> CoefficientShift), default, fixedPoint.Maximum); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void Convert(ushort y, ushort cb, ushort cr, in ConversionParameters parameters, out byte r, out byte g, out byte b) + { + FixedPointParameters fixedPoint = parameters.FixedPointScalar; + int centeredBlue = cb - ChromaMidpoint; + int centeredRed = cr - ChromaMidpoint; + + // All overloads preserve this term grouping and round once after the complete contribution for a + // component has been accumulated, so vector width cannot change an output code value. + int red = y + (((fixedPoint.RedCr * centeredRed) + RoundingBias) >> CoefficientShift); + int green = y + (((fixedPoint.GreenCb * centeredBlue) + (fixedPoint.GreenCr * centeredRed) + RoundingBias) >> CoefficientShift); + int blue = y + (((fixedPoint.BlueCb * centeredBlue) + RoundingBias) >> CoefficientShift); + + r = (byte)Numerics.Clamp(red, 0, byte.MaxValue); + g = (byte)Numerics.Clamp(green, 0, byte.MaxValue); + b = (byte)Numerics.Clamp(blue, 0, byte.MaxValue); + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Components/ColorConverters/HeifYuvToRgb8Converter.Operator.cs b/src/ImageSharp/Formats/Heif/Components/ColorConverters/HeifYuvToRgb8Converter.Operator.cs new file mode 100644 index 0000000000..63b3142b1e --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Components/ColorConverters/HeifYuvToRgb8Converter.Operator.cs @@ -0,0 +1,283 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.InteropServices; +using System.Runtime.Intrinsics; +using SixLabors.ImageSharp.Common.Helpers; + +namespace SixLabors.ImageSharp.Formats.Heif.Components; + +/// +/// Defines fixed-point operators and row traversal for eight-bit 4:2:0 conversion. Consecutive lanes represent output +/// pixels; each native chroma sample is duplicated into the two lanes covered by horizontal subsampling before the +/// closed color operator runs. All products remain in signed 32-bit lanes until clipped RGB values are narrowed. +/// +internal static partial class HeifYuvToRgb8Converter +{ + /// + /// Defines fixed-point color arithmetic for scalar and SIMD lanes. + /// + private interface IHeifYuvToRgb8Operator + { + /// + /// Converts sixteen YCbCr samples to eight-bit RGB lanes. + /// + /// The luma lanes. + /// The blue-difference lanes. + /// The red-difference lanes. + /// The image conversion parameters. + /// The converted red lanes. + /// The converted green lanes. + /// The converted blue lanes. + public static abstract void Convert( + Vector512 y, + Vector512 cb, + Vector512 cr, + in ConversionParameters parameters, + out Vector512 r, + out Vector512 g, + out Vector512 b); + + /// + /// Converts eight YCbCr samples to eight-bit RGB lanes. + /// + /// The luma lanes. + /// The blue-difference lanes. + /// The red-difference lanes. + /// The image conversion parameters. + /// The converted red lanes. + /// The converted green lanes. + /// The converted blue lanes. + public static abstract void Convert( + Vector256 y, + Vector256 cb, + Vector256 cr, + in ConversionParameters parameters, + out Vector256 r, + out Vector256 g, + out Vector256 b); + + /// + /// Converts four YCbCr samples to eight-bit RGB lanes. + /// + /// The luma lanes. + /// The blue-difference lanes. + /// The red-difference lanes. + /// The image conversion parameters. + /// The converted red lanes. + /// The converted green lanes. + /// The converted blue lanes. + public static abstract void Convert( + Vector128 y, + Vector128 cb, + Vector128 cr, + in ConversionParameters parameters, + out Vector128 r, + out Vector128 g, + out Vector128 b); + + /// + /// Converts one YCbCr sample to eight-bit RGB. + /// + /// The luma sample. + /// The blue-difference sample. + /// The red-difference sample. + /// The image conversion parameters. + /// The converted red sample. + /// The converted green sample. + /// The converted blue sample. + public static abstract void Convert(ushort y, ushort cb, ushort cr, in ConversionParameters parameters, out byte r, out byte g, out byte b); + } + + /// + /// Converts one luma row and its nearest native chroma row to planar eight-bit RGB. + /// + /// The fixed-point color arithmetic selected for the row. + /// The full-resolution luma samples. + /// The half-width blue-difference samples. + /// The half-width red-difference samples. + /// The destination red samples. + /// The destination green samples. + /// The destination blue samples. + /// The horizontal chroma subsampling shift. + /// The fixed-point matrix coefficients. + private static void ConvertRow( + ReadOnlySpan luma, + ReadOnlySpan chromaBlue, + ReadOnlySpan chromaRed, + Span red, + Span green, + Span blue, + int subsamplingX, + in ConversionParameters parameters) + where TOperator : struct, IHeifYuvToRgb8Operator + { + ref ushort lumaBase = ref MemoryMarshal.GetReference(luma); + ref ushort chromaBlueBase = ref MemoryMarshal.GetReference(chromaBlue); + ref ushort chromaRedBase = ref MemoryMarshal.GetReference(chromaRed); + ref byte redBase = ref MemoryMarshal.GetReference(red); + ref byte greenBase = ref MemoryMarshal.GetReference(green); + ref byte blueBase = ref MemoryMarshal.GetReference(blue); + int x = 0; + + // The shared offset lets the widest supported register consume the row first. Narrower widths then + // handle the complete remainder, leaving at most three pixels for the scalar fallback. + if (Vector512.IsHardwareAccelerated) + { + int oneVectorFromEnd = luma.Length - Vector512.Count; + + for (; x <= oneVectorFromEnd; x += Vector512.Count) + { + Vector512 y = LoadVector512(ref Unsafe.Add(ref lumaBase, x)); + Vector512 cb = subsamplingX == 0 + ? LoadVector512(ref Unsafe.Add(ref chromaBlueBase, x)) + : LoadRepeatedVector512(ref Unsafe.Add(ref chromaBlueBase, x >> 1)); + + Vector512 cr = subsamplingX == 0 + ? LoadVector512(ref Unsafe.Add(ref chromaRedBase, x)) + : LoadRepeatedVector512(ref Unsafe.Add(ref chromaRedBase, x >> 1)); + + TOperator.Convert(y, cb, cr, in parameters, out Vector512 r, out Vector512 g, out Vector512 b); + HeifByteSampleConverter.Store(r, ref Unsafe.Add(ref redBase, x)); + HeifByteSampleConverter.Store(g, ref Unsafe.Add(ref greenBase, x)); + HeifByteSampleConverter.Store(b, ref Unsafe.Add(ref blueBase, x)); + } + } + + if (Vector256.IsHardwareAccelerated) + { + int oneVectorFromEnd = luma.Length - Vector256.Count; + + for (; x <= oneVectorFromEnd; x += Vector256.Count) + { + Vector256 y = LoadVector256(ref Unsafe.Add(ref lumaBase, x)); + Vector256 cb = subsamplingX == 0 + ? LoadVector256(ref Unsafe.Add(ref chromaBlueBase, x)) + : LoadRepeatedVector256(ref Unsafe.Add(ref chromaBlueBase, x >> 1)); + + Vector256 cr = subsamplingX == 0 + ? LoadVector256(ref Unsafe.Add(ref chromaRedBase, x)) + : LoadRepeatedVector256(ref Unsafe.Add(ref chromaRedBase, x >> 1)); + + TOperator.Convert(y, cb, cr, in parameters, out Vector256 r, out Vector256 g, out Vector256 b); + HeifByteSampleConverter.Store(r, ref Unsafe.Add(ref redBase, x)); + HeifByteSampleConverter.Store(g, ref Unsafe.Add(ref greenBase, x)); + HeifByteSampleConverter.Store(b, ref Unsafe.Add(ref blueBase, x)); + } + } + + if (Vector128.IsHardwareAccelerated) + { + int oneVectorFromEnd = luma.Length - Vector128.Count; + + for (; x <= oneVectorFromEnd; x += Vector128.Count) + { + Vector128 y = LoadVector128(ref Unsafe.Add(ref lumaBase, x)); + Vector128 cb = subsamplingX == 0 + ? LoadVector128(ref Unsafe.Add(ref chromaBlueBase, x)) + : LoadRepeatedVector128(ref Unsafe.Add(ref chromaBlueBase, x >> 1)); + + Vector128 cr = subsamplingX == 0 + ? LoadVector128(ref Unsafe.Add(ref chromaRedBase, x)) + : LoadRepeatedVector128(ref Unsafe.Add(ref chromaRedBase, x >> 1)); + + TOperator.Convert(y, cb, cr, in parameters, out Vector128 r, out Vector128 g, out Vector128 b); + HeifByteSampleConverter.Store(r, ref Unsafe.Add(ref redBase, x)); + HeifByteSampleConverter.Store(g, ref Unsafe.Add(ref greenBase, x)); + HeifByteSampleConverter.Store(b, ref Unsafe.Add(ref blueBase, x)); + } + } + + for (; x < luma.Length; x++) + { + TOperator.Convert( + Unsafe.Add(ref lumaBase, x), + Unsafe.Add(ref chromaBlueBase, x >> subsamplingX), + Unsafe.Add(ref chromaRedBase, x >> subsamplingX), + in parameters, + out Unsafe.Add(ref redBase, x), + out Unsafe.Add(ref greenBase, x), + out Unsafe.Add(ref blueBase, x)); + } + } + + /// + /// Loads sixteen luma samples as signed 32-bit SIMD lanes. + /// + /// The first native luma sample. + /// The widened luma lanes. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector512 LoadVector512(ref ushort source) + { + (Vector256 lower, Vector256 upper) = Vector256.Widen(Vector256.LoadUnsafe(ref source)); + return Vector512.Create(lower, upper).AsInt32(); + } + + /// + /// Loads eight luma samples as signed 32-bit SIMD lanes. + /// + /// The first native luma sample. + /// The widened luma lanes. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector256 LoadVector256(ref ushort source) + { + Vector128 samples = Vector128.LoadUnsafe(ref source); + return Vector256.Create(Vector128.WidenLower(samples), Vector128.WidenUpper(samples)).AsInt32(); + } + + /// + /// Loads four luma samples as signed 32-bit SIMD lanes. + /// + /// The first native luma sample. + /// The widened luma lanes. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector128 LoadVector128(ref ushort source) + { + ulong packed = Unsafe.ReadUnaligned(ref Unsafe.As(ref source)); + return Vector128.WidenLower(Vector128.CreateScalarUnsafe(packed).AsUInt16()).AsInt32(); + } + + /// + /// Loads eight chroma samples and repeats each sample into two of sixteen 32-bit SIMD lanes. + /// + /// The first native chroma sample. + /// The horizontally replicated chroma lanes. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector512 LoadRepeatedVector512(ref ushort source) + { + Vector128 samples = Vector128.LoadUnsafe(ref source); + Vector128 lower = Vector128_.UnpackLow(samples.AsInt16(), samples.AsInt16()).AsUInt16(); + Vector128 upper = Vector128_.UnpackHigh(samples.AsInt16(), samples.AsInt16()).AsUInt16(); + (Vector256 widenedLower, Vector256 widenedUpper) = Vector256.Widen(Vector256.Create(lower, upper)); + return Vector512.Create(widenedLower, widenedUpper).AsInt32(); + } + + /// + /// Loads four chroma samples and repeats each sample into two of eight 32-bit SIMD lanes. + /// + /// The first native chroma sample. + /// The horizontally replicated chroma lanes. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector256 LoadRepeatedVector256(ref ushort source) + { + ulong packed = Unsafe.ReadUnaligned(ref Unsafe.As(ref source)); + Vector128 samples = Vector128.CreateScalarUnsafe(packed).AsUInt16(); + Vector128 repeated = Vector128_.UnpackLow(samples.AsInt16(), samples.AsInt16()).AsUInt16(); + return Vector256.Create(Vector128.WidenLower(repeated), Vector128.WidenUpper(repeated)).AsInt32(); + } + + /// + /// Loads two chroma samples and repeats each sample into two of four 32-bit SIMD lanes. + /// + /// The first native chroma sample. + /// The horizontally replicated chroma lanes. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector128 LoadRepeatedVector128(ref ushort source) + { + uint packed = Unsafe.ReadUnaligned(ref Unsafe.As(ref source)); + Vector128 samples = Vector128.CreateScalarUnsafe(packed).AsUInt16(); + Vector128 repeated = Vector128_.UnpackLow(samples.AsInt16(), samples.AsInt16()).AsUInt16(); + return Vector128.WidenLower(repeated).AsInt32(); + } +} diff --git a/src/ImageSharp/Formats/Heif/Components/ColorConverters/HeifYuvToRgb8Converter.Parameters.cs b/src/ImageSharp/Formats/Heif/Components/ColorConverters/HeifYuvToRgb8Converter.Parameters.cs new file mode 100644 index 0000000000..e5e95fb28f --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Components/ColorConverters/HeifYuvToRgb8Converter.Parameters.cs @@ -0,0 +1,261 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Components; + +/// +/// Provides fixed-point scalar and SIMD coefficient storage for eight-bit 4:2:0 conversion. +/// +internal static partial class HeifYuvToRgb8Converter +{ + /// + /// Stores every scalar and SIMD coefficient representation resolved once for an image. + /// + private readonly struct ConversionParameters + { + /// + /// The scalar fixed-point coefficients. + /// + public readonly FixedPointParameters FixedPointScalar; + + /// + /// The four-lane SIMD coefficients. + /// + public readonly Vector128Parameters FixedPointFourLane; + + /// + /// The eight-lane SIMD coefficients. + /// + public readonly Vector256Parameters FixedPointEightLane; + + /// + /// The sixteen-lane SIMD coefficients. + /// + public readonly Vector512Parameters FixedPointSixteenLane; + + /// + /// Initializes a new instance of the struct. + /// + /// The shared floating-point conversion parameters. + public ConversionParameters(in HeifColorConversionParameters parameters) + { + FixedPointParameters scalar = new(in parameters); + this.FixedPointScalar = scalar; + this.FixedPointFourLane = new(in scalar); + this.FixedPointEightLane = new(in scalar); + this.FixedPointSixteenLane = new(in scalar); + } + } + + /// + /// Stores the scalar fixed-point coefficients resolved for one image. + /// + private readonly struct FixedPointParameters + { + /// + /// Initializes a new instance of the struct. + /// + /// The shared floating-point conversion parameters. + public FixedPointParameters(in HeifColorConversionParameters parameters) + { + float scale = 1 << CoefficientShift; + + // Rounding each image-invariant coefficient once gives the integer kernel eight fractional bits. + // The signed green coefficients retain the exact addition and rounding order used by every SIMD lane. + this.RedCr = (int)MathF.Round(parameters.RedChromaScale * scale, MidpointRounding.AwayFromZero); + this.GreenCb = -(int)MathF.Round(parameters.GreenBlueChromaScale * scale, MidpointRounding.AwayFromZero); + this.GreenCr = -(int)MathF.Round(parameters.GreenRedChromaScale * scale, MidpointRounding.AwayFromZero); + this.BlueCb = (int)MathF.Round(parameters.BlueChromaScale * scale, MidpointRounding.AwayFromZero); + } + + /// + /// Gets the red contribution from centered Cr. + /// + public int RedCr { get; } + + /// + /// Gets the green contribution from centered Cb. + /// + public int GreenCb { get; } + + /// + /// Gets the green contribution from centered Cr. + /// + public int GreenCr { get; } + + /// + /// Gets the blue contribution from centered Cb. + /// + public int BlueCb { get; } + } + + /// + /// Broadcasts the fixed-point coefficients for four-lane conversion. + /// + private readonly struct Vector128Parameters + { + /// + /// Initializes a new instance of the struct. + /// + /// The scalar fixed-point coefficients. + public Vector128Parameters(in FixedPointParameters parameters) + { + this.ChromaMidpoint = Vector128.Create(HeifYuvToRgb8Converter.ChromaMidpoint); + this.RoundingBias = Vector128.Create(HeifYuvToRgb8Converter.RoundingBias); + this.Maximum = Vector128.Create((int)byte.MaxValue); + this.RedCr = Vector128.Create(parameters.RedCr); + this.GreenCb = Vector128.Create(parameters.GreenCb); + this.GreenCr = Vector128.Create(parameters.GreenCr); + this.BlueCb = Vector128.Create(parameters.BlueCb); + } + + /// + /// Gets the neutral chroma code-value lanes. + /// + public Vector128 ChromaMidpoint { get; } + + /// + /// Gets the fixed-point rounding-bias lanes. + /// + public Vector128 RoundingBias { get; } + + /// + /// Gets the maximum eight-bit sample lanes. + /// + public Vector128 Maximum { get; } + + /// + /// Gets the red Cr coefficient lanes. + /// + public Vector128 RedCr { get; } + + /// + /// Gets the green Cb coefficient lanes. + /// + public Vector128 GreenCb { get; } + + /// + /// Gets the green Cr coefficient lanes. + /// + public Vector128 GreenCr { get; } + + /// + /// Gets the blue Cb coefficient lanes. + /// + public Vector128 BlueCb { get; } + } + + /// + /// Broadcasts the fixed-point coefficients for eight-lane conversion. + /// + private readonly struct Vector256Parameters + { + /// + /// Initializes a new instance of the struct. + /// + /// The scalar fixed-point coefficients. + public Vector256Parameters(in FixedPointParameters parameters) + { + this.ChromaMidpoint = Vector256.Create(HeifYuvToRgb8Converter.ChromaMidpoint); + this.RoundingBias = Vector256.Create(HeifYuvToRgb8Converter.RoundingBias); + this.Maximum = Vector256.Create((int)byte.MaxValue); + this.RedCr = Vector256.Create(parameters.RedCr); + this.GreenCb = Vector256.Create(parameters.GreenCb); + this.GreenCr = Vector256.Create(parameters.GreenCr); + this.BlueCb = Vector256.Create(parameters.BlueCb); + } + + /// + /// Gets the neutral chroma code-value lanes. + /// + public Vector256 ChromaMidpoint { get; } + + /// + /// Gets the fixed-point rounding-bias lanes. + /// + public Vector256 RoundingBias { get; } + + /// + /// Gets the maximum eight-bit sample lanes. + /// + public Vector256 Maximum { get; } + + /// + /// Gets the red Cr coefficient lanes. + /// + public Vector256 RedCr { get; } + + /// + /// Gets the green Cb coefficient lanes. + /// + public Vector256 GreenCb { get; } + + /// + /// Gets the green Cr coefficient lanes. + /// + public Vector256 GreenCr { get; } + + /// + /// Gets the blue Cb coefficient lanes. + /// + public Vector256 BlueCb { get; } + } + + /// + /// Broadcasts the fixed-point coefficients for sixteen-lane conversion. + /// + private readonly struct Vector512Parameters + { + /// + /// Initializes a new instance of the struct. + /// + /// The scalar fixed-point coefficients. + public Vector512Parameters(in FixedPointParameters parameters) + { + this.ChromaMidpoint = Vector512.Create(HeifYuvToRgb8Converter.ChromaMidpoint); + this.RoundingBias = Vector512.Create(HeifYuvToRgb8Converter.RoundingBias); + this.Maximum = Vector512.Create((int)byte.MaxValue); + this.RedCr = Vector512.Create(parameters.RedCr); + this.GreenCb = Vector512.Create(parameters.GreenCb); + this.GreenCr = Vector512.Create(parameters.GreenCr); + this.BlueCb = Vector512.Create(parameters.BlueCb); + } + + /// + /// Gets the neutral chroma code-value lanes. + /// + public Vector512 ChromaMidpoint { get; } + + /// + /// Gets the fixed-point rounding-bias lanes. + /// + public Vector512 RoundingBias { get; } + + /// + /// Gets the maximum eight-bit sample lanes. + /// + public Vector512 Maximum { get; } + + /// + /// Gets the red Cr coefficient lanes. + /// + public Vector512 RedCr { get; } + + /// + /// Gets the green Cb coefficient lanes. + /// + public Vector512 GreenCb { get; } + + /// + /// Gets the green Cr coefficient lanes. + /// + public Vector512 GreenCr { get; } + + /// + /// Gets the blue Cb coefficient lanes. + /// + public Vector512 BlueCb { get; } + } +} diff --git a/src/ImageSharp/Formats/Heif/Components/ColorConverters/HeifYuvToRgb8Converter.cs b/src/ImageSharp/Formats/Heif/Components/ColorConverters/HeifYuvToRgb8Converter.cs new file mode 100644 index 0000000000..b900f49f51 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Components/ColorConverters/HeifYuvToRgb8Converter.cs @@ -0,0 +1,113 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Buffers; +using SixLabors.ImageSharp.Advanced; +using SixLabors.ImageSharp.Memory; +using SixLabors.ImageSharp.Metadata.Profiles.Cicp; +using SixLabors.ImageSharp.PixelFormats; + +namespace SixLabors.ImageSharp.Formats.Heif.Components; + +/// +/// Converts HEIF YUV planes directly to packed eight-bit RGB pixels. +/// +internal static partial class HeifYuvToRgb8Converter +{ + /// + /// The fixed-point precision used for H.273 matrix coefficients. + /// + private const int CoefficientShift = 8; + + /// + /// The half-unit bias used before fixed-point coefficient results are shifted to integer samples. + /// + private const int RoundingBias = 1 << (CoefficientShift - 1); + + /// + /// The neutral code value for full-range eight-bit chroma. + /// + private const int ChromaMidpoint = 128; + + /// + /// Determines whether the specialized fixed-point conversion supports the supplied plane and color description. + /// + /// The horizontal chroma subsampling shift. + /// The vertical chroma subsampling shift. + /// The luma sample precision in bits. + /// The chroma sample precision in bits. + /// Whether the samples use the complete numeric range. + /// The H.273 matrix-coefficient code point. + /// The resolved H.273 conversion operation. + /// when the planes can use this converter; otherwise, . + public static bool SupportsFixedPointConversion( + int subsamplingX, + int subsamplingY, + int lumaBitDepth, + int chromaBitDepth, + bool isFullRange, + CicpMatrixCoefficients matrixCoefficients, + HeifColorConversionMode mode) + => subsamplingX == 1 + && subsamplingY == 1 + && lumaBitDepth == 8 + && chromaBitDepth == 8 + && isFullRange + && matrixCoefficients == CicpMatrixCoefficients.Unspecified + && mode == HeifColorConversionMode.Coefficients; + + /// + /// Converts supported HEIF component planes to packed pixels using integer SIMD with a scalar tail. + /// + /// The destination pixel type. + /// The codec adapter that exposes reconstructed component rows. + /// The configuration used for allocation and pixel conversion. + /// The reconstructed component-plane buffer. + /// The destination image frame. + /// The resolved H.273 conversion parameters. + /// The horizontal luma-sample offset of the output window. + /// The vertical luma-sample offset of the output window. + public static void ConvertFixedPoint( + Configuration configuration, + TBuffer buffer, + ImageFrame image, + in HeifColorConversionParameters parameters, + int sourceX, + int sourceY) + where TPixel : unmanaged, IPixel + where TBuffer : struct, IHeifPlanarSampleBuffer + { + ConversionParameters conversionParameters = new(in parameters); + using IMemoryOwner componentOwner = configuration.MemoryAllocator.Allocate(image.Width * 3); + Span components = componentOwner.GetSpan(); + Span red = components[..image.Width]; + Span green = components.Slice(image.Width, image.Width); + Span blue = components.Slice(image.Width * 2, image.Width); + + // The value-type buffer closes the row-access contract at the call site. Constrained calls are therefore + // devirtualized without boxing while keeping codec-specific buffer ownership outside the color pipeline. + for (int y = 0; y < image.Height; y++) + { + int lumaY = sourceY + y; + + // The codec boundary validates 4:2:0 crop offsets in complete chroma-sample units. Each native chroma + // sample therefore covers one 2x2 luma cell without an alignment branch in the SIMD loop. + ReadOnlySpan luma = buffer.GetLumaRowSpan(lumaY).Slice(sourceX, image.Width); + ReadOnlySpan chromaBlue = buffer.GetChromaBlueRowSpan(lumaY >> 1).Slice(sourceX >> 1); + ReadOnlySpan chromaRed = buffer.GetChromaRedRowSpan(lumaY >> 1).Slice(sourceX >> 1); + + ConvertRow( + luma, + chromaBlue, + chromaRed, + red, + green, + blue, + 1, + in conversionParameters); + + Span destination = image.PixelBuffer.DangerousGetRowSpan(y); + PixelOperations.Instance.PackFromRgbPlanes(red, green, blue, destination); + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Components/ColorConverters/IHeifPlanarSampleBuffer.cs b/src/ImageSharp/Formats/Heif/Components/ColorConverters/IHeifPlanarSampleBuffer.cs new file mode 100644 index 0000000000..9d8a56699e --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Components/ColorConverters/IHeifPlanarSampleBuffer.cs @@ -0,0 +1,78 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Components; + +/// +/// Exposes the native planar sample layout shared by HEIF image codecs. +/// +/// The native unsigned sample storage type. +internal interface IHeifPlanarSampleBuffer + where TSample : unmanaged +{ + /// + /// Gets the full luma-plane width in samples. + /// + public int Width { get; } + + /// + /// Gets the full luma-plane height in samples. + /// + public int Height { get; } + + /// + /// Gets the luma sample precision in bits. + /// + public int LumaBitDepth { get; } + + /// + /// Gets the chroma sample precision in bits. + /// + public int ChromaBitDepth { get; } + + /// + /// Gets a value indicating whether the buffer contains only the luma plane. + /// + public bool IsMonochrome { get; } + + /// + /// Gets the horizontal chroma subsampling shift. + /// + public int ChromaSubsamplingX { get; } + + /// + /// Gets the vertical chroma subsampling shift. + /// + public int ChromaSubsamplingY { get; } + + /// + /// Gets the horizontal chroma position in half-luma-sample units. + /// + public int ChromaPositionX { get; } + + /// + /// Gets the vertical chroma position in half-luma-sample units. + /// + public int ChromaPositionY { get; } + + /// + /// Gets one complete writable row from the luma, green, or intensity plane. + /// + /// The zero-based row index in plane samples. + /// The native luma, green, or intensity samples. + public Span GetLumaRowSpan(int row); + + /// + /// Gets one complete writable row from the blue-difference, blue, or first opponent-color plane. + /// + /// The zero-based row index in plane samples. + /// The native blue-difference, blue, or first opponent-color samples. + public Span GetChromaBlueRowSpan(int row); + + /// + /// Gets one complete writable row from the red-difference, red, or second opponent-color plane. + /// + /// The zero-based row index in plane samples. + /// The native red-difference, red, or second opponent-color samples. + public Span GetChromaRedRowSpan(int row); +} diff --git a/src/ImageSharp/Formats/Heif/Components/ColorConverters/IHeifSampleConverter.cs b/src/ImageSharp/Formats/Heif/Components/ColorConverters/IHeifSampleConverter.cs new file mode 100644 index 0000000000..6ee0b02e2b --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Components/ColorConverters/IHeifSampleConverter.cs @@ -0,0 +1,172 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Components; + +/// +/// Defines SIMD widening and narrowing operations for one native HEIF sample representation. +/// +/// The native sample type. +/// +/// Every vector lane represents one consecutive planar sample. Implementations widen native integer storage to +/// normalized single-precision arithmetic and narrow only values already scaled and clipped for the destination sample +/// type. The containing row traversal selects vector width; these operators do not reorder pixels or cross row bounds. +/// +internal interface IHeifSampleConverter + where TSample : unmanaged +{ + /// + /// Loads and widens four samples to single-precision lanes. + /// + /// The first source sample. + /// The widened samples. + public static abstract Vector128 LoadVector128(ref TSample source); + + /// + /// Loads and widens eight samples to single-precision lanes. + /// + /// The first source sample. + /// The widened samples. + public static abstract Vector256 LoadVector256(ref TSample source); + + /// + /// Loads and widens sixteen samples to single-precision lanes. + /// + /// The first source sample. + /// The widened samples. + public static abstract Vector512 LoadVector512(ref TSample source); + + /// + /// Narrows and stores four integer samples. + /// + /// The integer samples. + /// The first destination sample. + public static abstract void Store(Vector128 source, ref TSample destination); + + /// + /// Narrows and stores eight integer samples. + /// + /// The integer samples. + /// The first destination sample. + public static abstract void Store(Vector256 source, ref TSample destination); + + /// + /// Narrows and stores sixteen integer samples. + /// + /// The integer samples. + /// The first destination sample. + public static abstract void Store(Vector512 source, ref TSample destination); +} + +/// +/// Converts between eight-bit native samples and the planar conversion pipeline. +/// +internal readonly struct HeifByteSampleConverter : IHeifSampleConverter +{ + /// + public static Vector128 LoadVector128(ref byte source) + { + uint packed = Unsafe.ReadUnaligned(ref source); + Vector128 samples16 = Vector128.WidenLower(Vector128.CreateScalarUnsafe(packed).AsByte()); + return Vector128.ConvertToSingle(Vector128.WidenLower(samples16)); + } + + /// + public static Vector256 LoadVector256(ref byte source) + { + ulong packed = Unsafe.ReadUnaligned(ref source); + Vector128 samples16 = Vector128.WidenLower(Vector128.CreateScalarUnsafe(packed).AsByte()); + Vector256 samples32 = Vector256.Create(Vector128.WidenLower(samples16), Vector128.WidenUpper(samples16)); + return Vector256.ConvertToSingle(samples32); + } + + /// + public static Vector512 LoadVector512(ref byte source) + { + Vector128 packed = Unsafe.ReadUnaligned>(ref source); + (Vector128 lower16, Vector128 upper16) = Vector128.Widen(packed); + Vector256 lower32 = Vector256.Create(Vector128.WidenLower(lower16), Vector128.WidenUpper(lower16)); + Vector256 upper32 = Vector256.Create(Vector128.WidenLower(upper16), Vector128.WidenUpper(upper16)); + return Vector512.ConvertToSingle(Vector512.Create(lower32, upper32)); + } + + /// + public static void Store(Vector128 source, ref byte destination) + { + Vector128 samples16 = Vector128.Narrow(source.AsUInt32(), Vector128.Zero); + Vector128 samples8 = Vector128.Narrow(samples16, Vector128.Zero); + + // The lower four bytes contain the four source lanes after the two narrowing stages. + Unsafe.WriteUnaligned(ref destination, samples8.AsUInt32().ToScalar()); + } + + /// + public static void Store(Vector256 source, ref byte destination) + { + Store(source.GetLower(), ref destination); + Store(source.GetUpper(), ref Unsafe.Add(ref destination, Vector128.Count)); + } + + /// + public static void Store(Vector512 source, ref byte destination) + { + Store(source.GetLower(), ref destination); + Store(source.GetUpper(), ref Unsafe.Add(ref destination, Vector256.Count)); + } +} + +/// +/// Converts between unsigned 16-bit native samples and the planar conversion pipeline. +/// +internal readonly struct HeifUShortSampleConverter : IHeifSampleConverter +{ + /// + public static Vector128 LoadVector128(ref ushort source) + { + ulong packed = Unsafe.ReadUnaligned(ref Unsafe.As(ref source)); + Vector128 samples16 = Vector128.CreateScalarUnsafe(packed).AsUInt16(); + return Vector128.ConvertToSingle(Vector128.WidenLower(samples16)); + } + + /// + public static Vector256 LoadVector256(ref ushort source) + { + Vector128 samples16 = Unsafe.ReadUnaligned>(ref Unsafe.As(ref source)); + Vector256 samples32 = Vector256.Create(Vector128.WidenLower(samples16), Vector128.WidenUpper(samples16)); + return Vector256.ConvertToSingle(samples32); + } + + /// + public static Vector512 LoadVector512(ref ushort source) + { + Vector256 samples16 = Unsafe.ReadUnaligned>(ref Unsafe.As(ref source)); + (Vector256 lower32, Vector256 upper32) = Vector256.Widen(samples16); + return Vector512.ConvertToSingle(Vector512.Create(lower32, upper32)); + } + + /// + public static void Store(Vector128 source, ref ushort destination) + { + Vector128 samples = Vector128.Narrow(source.AsUInt32(), Vector128.Zero); + + // The lower four UInt16 values are contiguous and can be committed with one unaligned store. + Unsafe.WriteUnaligned(ref Unsafe.As(ref destination), samples.AsUInt64().ToScalar()); + } + + /// + public static void Store(Vector256 source, ref ushort destination) + { + Store(source.GetLower(), ref destination); + Store(source.GetUpper(), ref Unsafe.Add(ref destination, Vector128.Count)); + } + + /// + public static void Store(Vector512 source, ref ushort destination) + { + Store(source.GetLower(), ref destination); + Store(source.GetUpper(), ref Unsafe.Add(ref destination, Vector256.Count)); + } +} diff --git a/src/ImageSharp/Formats/Heif/GridHeifItemDecoder.cs b/src/ImageSharp/Formats/Heif/GridHeifItemDecoder.cs new file mode 100644 index 0000000000..996893c1d6 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/GridHeifItemDecoder.cs @@ -0,0 +1,634 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Buffers; +using System.Buffers.Binary; +using SixLabors.ImageSharp.Formats.Heif.Av1; +using SixLabors.ImageSharp.Formats.Heif.Components.Alpha; +using SixLabors.ImageSharp.Memory; +using SixLabors.ImageSharp.Metadata.Profiles.Cicp; +using SixLabors.ImageSharp.PixelFormats; + +namespace SixLabors.ImageSharp.Formats.Heif; + +/// +/// Decodes the image items referenced by a HEIF grid derived-image item. +/// +/// The destination pixel type. +internal sealed class GridHeifItemDecoder : IHeifItemDecoder, IHeifAlphaItemDecoder + where TPixel : unmanaged, IPixel +{ + /// + /// The image-grid descriptor version defined by HEIF. + /// + private const byte GridDescriptorVersion = 0; + + /// + /// The descriptor flag that selects 32-bit output dimensions instead of 16-bit dimensions. + /// + private const byte LargeDimensionsFlag = 1; + + /// + /// The descriptor length when output dimensions use 16-bit fields. + /// + private const int ShortGridDescriptorLength = 8; + + /// + /// The descriptor length when output dimensions use 32-bit fields. + /// + private const int LongGridDescriptorLength = 12; + + /// + /// The minimum width and height of the first cell in a MIAF image grid. + /// + private const int MinimumGridCellDimension = 64; + + /// + /// The item definitions available to the grid, indexed by item identifier. + /// + private readonly Dictionary items; + + /// + /// The item-reference relationships used to locate the grid's tiles. + /// + private readonly IList itemLinks; + + /// + /// Reads one selected encoded item payload on demand. + /// + private readonly Func> itemDataReader; + + /// + /// The optional row-major tile identifiers supplied for an auxiliary grid plane. + /// + private readonly IReadOnlyList? tileItemIds; + + /// + /// Initializes a new instance of the class. + /// + /// The item definitions in the containing HEIF file. + /// The item-reference relationships in the containing HEIF file. + /// Reads one selected encoded image payload on demand. + /// + /// Optional row-major tile identifiers that replace the grid item's own derived-image references. + /// + public GridHeifItemDecoder( + IList items, + IList itemLinks, + Func> itemDataReader, + IReadOnlyList? tileItemIds = null) + { + Dictionary itemLookup = new(items.Count); + foreach (HeifItem item in items) + { + itemLookup.Add(item.Id, item); + } + + this.items = itemLookup; + this.itemLinks = itemLinks; + this.itemDataReader = itemDataReader; + this.tileItemIds = tileItemIds; + } + + /// + /// Gets the grid derived-image item type. + /// + public Heif4CharCode Type => Heif4CharCode.Grid; + + /// + /// Gets the compression method used by the decoded grid tiles. + /// + public HeifCompressionMethod CompressionMethod { get; private set; } + + /// + /// Decodes the tiles referenced by a grid derived-image item. + /// + /// The general options governing the containing HEIF decode. + /// The grid derived-image item. + /// The grid descriptor payload. + /// The container color description inherited by tiles that do not declare one. + /// The token used to cancel between tile payloads. + /// The image reconstructed from the referenced grid tiles. + public Image DecodeItemData( + DecoderOptions options, + HeifItem gridItem, + Span data, + CicpProfile? colorProfile, + CancellationToken cancellationToken) + { + GridDescriptor descriptor = ParseGridDescriptor(data); + IReadOnlyList linked = this.GetLinkedTileIds(gridItem, descriptor); + + Heif4CharCode tileType = default; + Av1CodecConfiguration? av1GridConfiguration = null; + Image result = this.CreateGridResult( + options, + descriptor, + linked[0], + colorProfile, + ref tileType, + ref av1GridConfiguration, + cancellationToken, + out int tileWidth, + out int tileHeight); + + try + { + for (int tileIndex = 1; tileIndex < linked.Count; tileIndex++) + { + cancellationToken.ThrowIfCancellationRequested(); + HeifItem item = this.items[linked[tileIndex]]; + using Image tile = this.DecodeGridTile( + options, + item, + colorProfile, + ref tileType, + ref av1GridConfiguration, + cancellationToken); + + Size copySize = GetGridTileCopySize( + descriptor, + tileWidth, + tileHeight, + tileIndex); + + if (!IsGridTileExtentValid(tile.Size, copySize, tileWidth, tileHeight)) + { + throw new InvalidImageContentException( + $"HEIF image grid tile {item.Id} has dimensions {tile.Size}, which cannot cover its {copySize} grid region."); + } + + CopyGridTile(tile, result, descriptor, tileIndex, tileWidth, tileHeight); + } + + return result; + } + catch + { + result.Dispose(); + throw; + } + } + + /// + /// Decodes the first validated grid tile, establishes the common tile geometry, and creates the output canvas. + /// + private Image CreateGridResult( + DecoderOptions options, + in GridDescriptor descriptor, + uint firstTileId, + CicpProfile? colorProfile, + ref Heif4CharCode tileType, + ref Av1CodecConfiguration? av1GridConfiguration, + CancellationToken cancellationToken, + out int tileWidth, + out int tileHeight) + { + cancellationToken.ThrowIfCancellationRequested(); + HeifItem item = this.items[firstTileId]; + using Image tile = this.DecodeGridTile( + options, + item, + colorProfile, + ref tileType, + ref av1GridConfiguration, + cancellationToken); + + tileWidth = tile.Width; + tileHeight = tile.Height; + ValidateGridCoverage(descriptor, tileWidth, tileHeight); + ValidateGridDimensions(descriptor, tile.Size, av1GridConfiguration); + Size copySize = GetGridTileCopySize(descriptor, tileWidth, tileHeight, 0); + if (!IsGridTileExtentValid(tile.Size, copySize, tileWidth, tileHeight)) + { + throw new InvalidImageContentException( + $"HEIF image grid tile {item.Id} has dimensions {tile.Size}, which cannot cover its {copySize} grid region."); + } + + Image result = new( + options.Configuration, + descriptor.OutputSize.Width, + descriptor.OutputSize.Height, + tile.Metadata.DeepClone()); + + try + { + CopyGridTile(tile, result, descriptor, 0, tileWidth, tileHeight); + return result; + } + catch + { + result.Dispose(); + throw; + } + } + + /// + /// Decodes and scales one grid tile while its encoded payload owner is active. + /// + private Image DecodeGridTile( + DecoderOptions options, + HeifItem item, + CicpProfile? colorProfile, + ref Heif4CharCode tileType, + ref Av1CodecConfiguration? av1GridConfiguration, + CancellationToken cancellationToken) + { + ValidateTileConfiguration(item, ref tileType, ref av1GridConfiguration); + IHeifItemDecoder? decoder = HeifCompressionFactory.GetDecoder(item.Type) + ?? throw new ImageFormatException($"The HEIF image grid uses unsupported tile type '{item.Type}'."); + + using IMemoryOwner itemMemory = this.itemDataReader(item); + this.CompressionMethod = decoder.CompressionMethod; + Image tile = decoder.DecodeItemData( + options, + item, + itemMemory.GetSpan(), + item.CicpProfile ?? colorProfile, + cancellationToken); + + try + { + HeifItemDecoderUtilities.ScaleToItemExtent(tile, item); + return tile; + } + catch + { + // Ownership transfers to the caller only after extent normalization succeeds. + tile.Dispose(); + throw; + } + } + + /// + /// Copies one decoded tile into its cropped row-major grid position. + /// + private static void CopyGridTile( + Image tile, + Image result, + in GridDescriptor descriptor, + int tileIndex, + int tileWidth, + int tileHeight) + { + int column = tileIndex % descriptor.Columns; + int row = tileIndex / descriptor.Columns; + int destinationX = column * tileWidth; + int destinationY = row * tileHeight; + int copyWidth = Math.Min(tileWidth, descriptor.OutputSize.Width - destinationX); + int copyHeight = Math.Min(tileHeight, descriptor.OutputSize.Height - destinationY); + ImageFrame source = tile.Frames.RootFrame; + ImageFrame destination = result.Frames.RootFrame; + + // Copy before disposing this decoded tile, keeping peak tile storage independent of grid cell count. + // The descriptor may crop only the rightmost column and bottom row. + for (int y = 0; y < copyHeight; y++) + { + Span destinationRow = destination.PixelBuffer + .DangerousGetRowSpan(destinationY + y) + .Slice(destinationX, copyWidth); + + source.PixelBuffer.DangerousGetRowSpan(y)[..copyWidth].CopyTo(destinationRow); + } + } + + /// + public void DecodeAlphaItemData( + DecoderOptions options, + HeifItem gridItem, + Span data, + ImageFrame destination, + Size outputSize, + Rectangle destinationRectangle, + bool premultiplied, + CancellationToken cancellationToken) + { + GridDescriptor descriptor = ParseGridDescriptor(data); + IReadOnlyList linked = this.GetLinkedTileIds(gridItem, descriptor); + Heif4CharCode tileType = default; + Av1CodecConfiguration? av1GridConfiguration = null; + HeifItem firstItem = this.items[linked[0]]; + if (firstItem.Extent == default) + { + throw new InvalidImageContentException($"HEIF alpha grid tile {firstItem.Id} has no spatial extent."); + } + + Size tileSize = firstItem.Extent; + ValidateGridCoverage(descriptor, tileSize.Width, tileSize.Height); + + // Validate the complete grid before mutating the color frame. IgnoreImageData can then omit a failed alpha + // grid without leaving a partially composed prefix in the returned image. + for (int tileIndex = 0; tileIndex < linked.Count; tileIndex++) + { + uint id = linked[tileIndex]; + HeifItem item = this.items[id]; + ValidateTileConfiguration(item, ref tileType, ref av1GridConfiguration); + if (HeifCompressionFactory.GetDecoder(item.Type) is not IHeifAlphaItemDecoder) + { + throw new ImageFormatException($"The HEIF alpha grid uses unsupported tile type '{item.Type}'."); + } + + if (item.Extent == default) + { + throw new InvalidImageContentException($"HEIF alpha grid tile {item.Id} has no spatial extent."); + } + + Size copySize = GetGridTileCopySize( + descriptor, + tileSize.Width, + tileSize.Height, + tileIndex); + + if (!IsGridTileExtentValid(item.Extent, copySize, tileSize.Width, tileSize.Height)) + { + throw new InvalidImageContentException( + $"HEIF alpha grid tile {item.Id} has dimensions {item.Extent}, which cannot cover its {copySize} grid region."); + } + } + + ValidateGridDimensions(descriptor, tileSize, av1GridConfiguration); + + int gridWidth = descriptor.OutputSize.Width; + int gridHeight = descriptor.OutputSize.Height; + if (descriptor.OutputSize != outputSize || destinationRectangle.Size != outputSize) + { + throw new InvalidImageContentException("The HEIF alpha grid dimensions do not match the color grid dimensions."); + } + + for (int tileIndex = 0; tileIndex < linked.Count; tileIndex++) + { + cancellationToken.ThrowIfCancellationRequested(); + HeifItem item = this.items[linked[tileIndex]]; + IHeifItemDecoder? itemDecoder = HeifCompressionFactory.GetDecoder(item.Type); + if (itemDecoder is not IHeifAlphaItemDecoder decoder) + { + throw new InvalidImageContentException($"HEIF alpha grid tile {item.Id} uses unsupported coding format '{item.Type}'."); + } + + using IMemoryOwner itemMemory = this.itemDataReader(item); + int column = tileIndex % descriptor.Columns; + int row = tileIndex / descriptor.Columns; + int destinationX = destinationRectangle.X + (column * tileSize.Width); + int destinationY = destinationRectangle.Y + (row * tileSize.Height); + Size copySize = GetGridTileCopySize( + descriptor, + tileSize.Width, + tileSize.Height, + tileIndex); + + Rectangle tileDestination = new(destinationX, destinationY, copySize.Width, copySize.Height); + + decoder.DecodeAlphaItemData( + options, + item, + itemMemory.GetSpan(), + destination, + item.Extent, + tileDestination, + premultiplied, + cancellationToken); + } + } + + /// + /// Validates that the first cell dimensions cover the grid while leaving a nonempty final row and column. + /// + private static void ValidateGridCoverage(in GridDescriptor descriptor, int tileWidth, int tileHeight) + { + if (((long)tileWidth * descriptor.Columns) < descriptor.OutputSize.Width || + ((long)tileHeight * descriptor.Rows) < descriptor.OutputSize.Height) + { + throw new InvalidImageContentException("The HEIF image grid tiles do not cover the output canvas."); + } + + if (((long)tileWidth * (descriptor.Columns - 1)) >= descriptor.OutputSize.Width || + ((long)tileHeight * (descriptor.Rows - 1)) >= descriptor.OutputSize.Height) + { + throw new InvalidImageContentException("The HEIF image grid edge tiles do not overlap the output canvas."); + } + } + + /// + /// Gets the portion of one cell that overlaps the output canvas. + /// + private static Size GetGridTileCopySize( + in GridDescriptor descriptor, + int tileWidth, + int tileHeight, + int tileIndex) + { + int column = tileIndex % descriptor.Columns; + int row = tileIndex / descriptor.Columns; + int copyWidth = column == descriptor.Columns - 1 + ? descriptor.OutputSize.Width - (tileWidth * (descriptor.Columns - 1)) + : tileWidth; + + int copyHeight = row == descriptor.Rows - 1 + ? descriptor.OutputSize.Height - (tileHeight * (descriptor.Rows - 1)) + : tileHeight; + + return new Size(copyWidth, copyHeight); + } + + /// + /// Determines whether a cell can cover its output region without exceeding the first cell's dimensions. + /// + private static bool IsGridTileExtentValid(Size extent, Size copySize, int tileWidth, int tileHeight) + => extent.Width >= copySize.Width + && extent.Width <= tileWidth + && extent.Height >= copySize.Height + && extent.Height <= tileHeight; + + /// + /// Validates the MIAF cell-size and chroma-alignment rules established by the first grid cell. + /// + private static void ValidateGridDimensions( + in GridDescriptor descriptor, + Size tileSize, + Av1CodecConfiguration? av1GridConfiguration) + { + if (tileSize.Width < MinimumGridCellDimension || tileSize.Height < MinimumGridCellDimension) + { + throw new InvalidImageContentException( + $"HEIF image grid cells must be at least {MinimumGridCellDimension} samples wide and high."); + } + + if (av1GridConfiguration is null || av1GridConfiguration.IsMonochrome) + { + return; + } + + if (av1GridConfiguration.ChromaSubsamplingX && + (((descriptor.OutputSize.Width & 1) != 0) || ((tileSize.Width & 1) != 0))) + { + throw new InvalidImageContentException( + "HEIF image grid widths must be even when AV1 chroma is horizontally subsampled."); + } + + if (av1GridConfiguration.ChromaSubsamplingY && + (((descriptor.OutputSize.Height & 1) != 0) || ((tileSize.Height & 1) != 0))) + { + throw new InvalidImageContentException( + "HEIF image grid heights must be even when AV1 chroma is vertically subsampled."); + } + } + + /// + /// Parses and validates the fixed HEIF image-grid descriptor fields used by both color and alpha composition. + /// + /// The complete image-grid descriptor payload. + /// The validated row, column, and output dimensions. + private static GridDescriptor ParseGridDescriptor(ReadOnlySpan data) + { + if (data.Length < ShortGridDescriptorLength) + { + throw new InvalidImageContentException("The HEIF image grid descriptor is truncated."); + } + + int offset = 0; + byte version = data[offset++]; + if (version != GridDescriptorVersion) + { + throw new InvalidImageContentException($"The HEIF image grid descriptor has unsupported version {version}."); + } + + byte flags = data[offset++]; + bool usesLargeDimensions = (flags & LargeDimensionsFlag) != 0; + int descriptorLength = usesLargeDimensions ? LongGridDescriptorLength : ShortGridDescriptorLength; + if (data.Length != descriptorLength) + { + throw new InvalidImageContentException("The HEIF image grid descriptor has an invalid length."); + } + + int rows = data[offset++] + 1; + int columns = data[offset++] + 1; + uint outputWidth = usesLargeDimensions + ? BinaryPrimitives.ReadUInt32BigEndian(data[offset..]) + : BinaryPrimitives.ReadUInt16BigEndian(data[offset..]); + + offset += usesLargeDimensions ? sizeof(uint) : sizeof(ushort); + uint outputHeight = usesLargeDimensions + ? BinaryPrimitives.ReadUInt32BigEndian(data[offset..]) + : BinaryPrimitives.ReadUInt16BigEndian(data[offset..]); + + if (outputWidth is 0 or > int.MaxValue || outputHeight is 0 or > int.MaxValue) + { + throw new InvalidImageContentException("The HEIF image grid descriptor has invalid output dimensions."); + } + + return new GridDescriptor(rows, columns, new Size((int)outputWidth, (int)outputHeight)); + } + + /// + /// Resolves and validates the row-major tile identifiers for a grid descriptor. + /// + /// The grid item whose derived-image references are being resolved. + /// The validated grid dimensions. + /// The exact row-major tile identifiers required by the descriptor. + private IReadOnlyList GetLinkedTileIds(HeifItem gridItem, in GridDescriptor descriptor) + { + IReadOnlyList linked; + if (this.tileItemIds is not null) + { + // Auxiliary grids already own an immutable row-major identifier list; no defensive list copy is needed. + linked = this.tileItemIds; + } + else + { + List resolved = []; + foreach (HeifItemLink link in this.itemLinks) + { + if (link.Type == Heif4CharCode.Dimg && link.SourceId == gridItem.Id) + { + // The order of dimg destinations is the normative row-major order of the grid cells. + resolved.AddRange(link.DestinationIds); + } + } + + linked = resolved; + } + + int tileCount = descriptor.Rows * descriptor.Columns; + if (linked.Count != tileCount) + { + string message = $"The HEIF image grid requires {tileCount} tiles, but its derived-image references contain {linked.Count}."; + throw new InvalidImageContentException(message); + } + + return linked; + } + + /// + /// Validates the coding format and common AV1 sample layout of one grid tile. + /// + /// The coded grid tile being validated. + /// The common coding type established by the first grid tile. + /// The common AV1 sample layout established by the first AV1 grid tile. + private static void ValidateTileConfiguration( + HeifItem item, + ref Heif4CharCode tileType, + ref Av1CodecConfiguration? av1GridConfiguration) + { + if (tileType == default) + { + tileType = item.Type; + } + else if (item.Type != tileType) + { + throw new InvalidImageContentException("All HEIF image grid tiles must use the same coding format."); + } + + if (item.Type != Heif4CharCode.Av01) + { + return; + } + + Av1CodecConfiguration itemConfiguration = item.Av1CodecConfiguration + ?? throw new InvalidImageContentException($"AV1 image grid tile {item.Id} has no codec configuration property."); + + if (av1GridConfiguration is null) + { + av1GridConfiguration = itemConfiguration; + } + else if (!av1GridConfiguration.HasMatchingImageConfiguration(itemConfiguration)) + { + // All grid cells share one output sample layout. Reject differing AV1 descriptions before allocating + // or composing tiles so channel precision and chroma geometry cannot change between cells. + throw new InvalidImageContentException("All AV1 image grid tiles must use matching codec configurations."); + } + } + + /// + /// Contains the bounded row, column, and output dimensions from one image-grid descriptor. + /// + private readonly struct GridDescriptor + { + /// + /// Initializes a new instance of the struct. + /// + /// The number of grid rows. + /// The number of grid columns. + /// The output canvas dimensions. + public GridDescriptor(int rows, int columns, Size outputSize) + { + this.Rows = rows; + this.Columns = columns; + this.OutputSize = outputSize; + } + + /// + /// Gets the number of grid rows. + /// + public int Rows { get; } + + /// + /// Gets the number of grid columns. + /// + public int Columns { get; } + + /// + /// Gets the output canvas dimensions. + /// + public Size OutputSize { get; } + } +} diff --git a/src/ImageSharp/Formats/Heif/Heif4CharCode.cs b/src/ImageSharp/Formats/Heif/Heif4CharCode.cs new file mode 100644 index 0000000000..8fc0fa93d0 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Heif4CharCode.cs @@ -0,0 +1,486 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +// + +using System.CodeDom.Compiler; + +namespace SixLabors.ImageSharp.Formats.Heif; + +/// +/// Supported 4 character codes for use in HEIF images. +/// +[GeneratedCode("TextTemplateFileGenerator", "")] +public enum Heif4CharCode : uint +{ + /// + /// File Type. + /// + Ftyp = 0x66747970U, + + /// + /// Metadata. + /// + Meta = 0x6D657461U, + + /// + /// Media Data. + /// + Mdat = 0x6D646174U, + + /// + /// Movie container for an image sequence. + /// + Moov = 0x6D6F6F76U, + + /// + /// Movie header for an image sequence. + /// + Mvhd = 0x6D766864U, + + /// + /// Image sequence track. + /// + Trak = 0x7472616BU, + + /// + /// Image sequence track header. + /// + Tkhd = 0x746B6864U, + + /// + /// Image sequence track references. + /// + Tref = 0x74726566U, + + /// + /// Image sequence edit container. + /// + Edts = 0x65647473U, + + /// + /// Image sequence edit list. + /// + Elst = 0x656C7374U, + + /// + /// Image sequence media container. + /// + Mdia = 0x6D646961U, + + /// + /// Image sequence media header. + /// + Mdhd = 0x6D646864U, + + /// + /// Image sequence media information. + /// + Minf = 0x6D696E66U, + + /// + /// Image sequence sample table. + /// + Stbl = 0x7374626CU, + + /// + /// Image sequence sample descriptions. + /// + Stsd = 0x73747364U, + + /// + /// Image sequence sample timing. + /// + Stts = 0x73747473U, + + /// + /// Image sequence composition offsets. + /// + Ctts = 0x63747473U, + + /// + /// Image sequence composition-to-decode timing. + /// + Cslg = 0x63736C67U, + + /// + /// Image sequence sample-to-chunk map. + /// + Stsc = 0x73747363U, + + /// + /// Image sequence sample sizes. + /// + Stsz = 0x7374737AU, + + /// + /// Compact image sequence sample sizes. + /// + Stz2 = 0x73747A32U, + + /// + /// Image sequence 32-bit chunk offsets. + /// + Stco = 0x7374636FU, + + /// + /// Image sequence 64-bit chunk offsets. + /// + Co64 = 0x636F3634U, + + /// + /// Image sequence sync samples. + /// + Stss = 0x73747373U, + + /// + /// Image sequence sample group descriptions. + /// + Sgpd = 0x73677064U, + + /// + /// Image sequence sample-to-group map. + /// + Sbgp = 0x73626770U, + + /// + /// Direct reference samples group type. + /// + Refs = 0x72656673U, + + /// + /// Image sequence coding constraints. + /// + Ccst = 0x63637374U, + + /// + /// Auxiliary image sequence type. + /// + Auxi = 0x61757869U, + + /// + /// Self-contained data location. + /// + Url = 0x75726C20U, + + /// + /// Item Information Entry. + /// + Infe = 0x696E6665U, + + /// + /// Item Data. + /// + Idat = 0x69646174U, + + /// + /// Item Location. + /// + Iloc = 0x696C6F63U, + + /// + /// EXIF metadata. + /// + Exif = 0x45786966U, + + /// + /// Data Reference. + /// + Dref = 0x64726566U, + + /// + /// Primary Item. + /// + Pitm = 0x7069746DU, + + /// + /// Item Spatial Extent. + /// + Ispe = 0x69737065U, + + /// + /// Alternative text. + /// + Altt = 0x616C7474U, + + /// + /// Colour information. + /// + Colr = 0x636F6C72U, + + /// + /// Content light level information. + /// + Clli = 0x636C6C69U, + + /// + /// Mastering display colour volume. + /// + Mdcv = 0x6D646376U, + + /// + /// Content colour volume. + /// + Cclv = 0x63636C76U, + + /// + /// Ambient viewing environment. + /// + Amve = 0x616D7665U, + + /// + /// Reference viewing environment. + /// + Reve = 0x72657665U, + + /// + /// Nominal diffuse white. + /// + Ndwt = 0x6E647774U, + + /// + /// AV1 configuration. + /// + Av1C = 0x61763143U, + + /// + /// AV1 operating-point selector. + /// + A1op = 0x61316F70U, + + /// + /// AV1 layer selector. + /// + Lsel = 0x6C73656CU, + + /// + /// AV1 layered-image indexing. + /// + A1lx = 0x61316C78U, + + /// + /// Image Mirror. + /// + Imir = 0x696D6972U, + + /// + /// Image Rotation. + /// + Irot = 0x69726F74U, + + /// + /// Clean Aperture. + /// + Clap = 0x636C6170U, + + /// + /// Image Scaling. + /// + Iscl = 0x6973636CU, + + /// + /// Pixel Aspect Ratio. + /// + Pasp = 0x70617370U, + + /// + /// Pixel Information. + /// + Pixi = 0x70697869U, + + /// + /// Auxiliary Type. + /// + AuxC = 0x61757843U, + + /// + /// Reference Location. + /// + Rloc = 0x726C6F63U, + + /// + /// User Description. + /// + Udes = 0x75646573U, + + /// + /// IPMP Control Box. + /// + Ipmc = 0x69706D63U, + + /// + /// Item Property Container. + /// + Ipco = 0x6970636FU, + + /// + /// Item Property Association. + /// + Ipma = 0x69706D61U, + + /// + /// High Efficient File brand. + /// + Mif1 = 0x6D696631U, + + /// + /// Multi-Image Application Format brand. + /// + Miaf = 0x6D696166U, + + /// + /// AVIF brand. + /// + Avif = 0x61766966U, + + /// + /// AVIF image sequence brand. + /// + Avis = 0x61766973U, + + /// + /// AVIF intra-only image sequence brand. + /// + Avio = 0x6176696FU, + + /// + /// HEIF image sequence structural brand. + /// + Msf1 = 0x6D736631U, + + /// + /// ISO base media version 8 structural brand. + /// + Iso8 = 0x69736F38U, + + /// + /// Legacy JPEG coded tile. + /// + Jpeg = 0x6A706567U, + + /// + /// JPEG image sequence brand. + /// + Jpgs = 0x6A706773U, + + /// + /// AOMedia Video Coding tile. + /// + Av01 = 0x61763031U, + + /// + /// Data Information. + /// + Dinf = 0x64696E66U, + + /// + /// Group list. + /// + Grpl = 0x6772706CU, + + /// + /// Handler. + /// + Hdlr = 0x68646C72U, + + /// + /// Item Information. + /// + Iinf = 0x69696E66U, + + /// + /// Item Property. + /// + Iprp = 0x69707270U, + + /// + /// Item Protection. + /// + Ipro = 0x6970726FU, + + /// + /// Item Reference. + /// + Iref = 0x69726566U, + + /// + /// Grid. + /// + Grid = 0x67726964U, + + /// + /// Derived Image. + /// + Dimg = 0x64696D67U, + + /// + /// Thumbnail. + /// + Thmb = 0x74686D62U, + + /// + /// Auxiliary Image. + /// + Auxl = 0x6175786CU, + + /// + /// Premultiplied Alpha. + /// + Prem = 0x7072656DU, + + /// + /// Content Description. + /// + Cdsc = 0x63647363U, + + /// + /// MIME type. + /// + Mime = 0x6D696D65U, + + /// + /// URI. + /// + Uri = 0x75726920U, + + /// + /// Picture handler type. + /// + Pict = 0x70696374U, + + /// + /// Auxiliary image sequence handler type. + /// + Auxv = 0x61757876U, + + /// + /// Video handler type. + /// + Vide = 0x76696465U, + + /// + /// Unique Identifier. + /// + Uuid = 0x75756964U, + + /// + /// Free space. + /// + Free = 0x66726565U, + + /// + /// CICP color information. + /// + Nclx = 0x6E636C78U, + + /// + /// Restricted ICC color profile. + /// + RICC = 0x72494343U, + + /// + /// Unrestricted ICC color profile. + /// + Prof = 0x70726F66U, + +} diff --git a/src/ImageSharp/Formats/Heif/Heif4CharCode.tt b/src/ImageSharp/Formats/Heif/Heif4CharCode.tt new file mode 100644 index 0000000000..a983a8ad5c --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Heif4CharCode.tt @@ -0,0 +1,147 @@ +<#@ template language="C#" #> +<#@ import namespace="System.Collections.Generic" #> +<#@ import namespace="System.Text" #> +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +// +<# + var codes = new []{ + "ftyp", "File Type", + "meta", "Metadata", + "mdat", "Media Data", + "moov", "Movie container for an image sequence", + "mvhd", "Movie header for an image sequence", + "trak", "Image sequence track", + "tkhd", "Image sequence track header", + "tref", "Image sequence track references", + "edts", "Image sequence edit container", + "elst", "Image sequence edit list", + "mdia", "Image sequence media container", + "mdhd", "Image sequence media header", + "minf", "Image sequence media information", + "stbl", "Image sequence sample table", + "stsd", "Image sequence sample descriptions", + "stts", "Image sequence sample timing", + "ctts", "Image sequence composition offsets", + "cslg", "Image sequence composition-to-decode timing", + "stsc", "Image sequence sample-to-chunk map", + "stsz", "Image sequence sample sizes", + "stz2", "Compact image sequence sample sizes", + "stco", "Image sequence 32-bit chunk offsets", + "co64", "Image sequence 64-bit chunk offsets", + "stss", "Image sequence sync samples", + "sgpd", "Image sequence sample group descriptions", + "sbgp", "Image sequence sample-to-group map", + "refs", "Direct reference samples group type", + "ccst", "Image sequence coding constraints", + "auxi", "Auxiliary image sequence type", + "url ", "Self-contained data location", + "infe", "Item Information Entry", + "idat", "Item Data", + "iloc", "Item Location", + "Exif", "EXIF metadata", + "dref", "Data Reference", + "pitm", "Primary Item", + "ispe", "Item Spatial Extent", + "altt", "Alternative text", + "colr", "Colour information", + "clli", "Content light level information", + "mdcv", "Mastering display colour volume", + "cclv", "Content colour volume", + "amve", "Ambient viewing environment", + "reve", "Reference viewing environment", + "ndwt", "Nominal diffuse white", + "av1C", "AV1 configuration", + "a1op", "AV1 operating-point selector", + "lsel", "AV1 layer selector", + "a1lx", "AV1 layered-image indexing", + "imir", "Image Mirror", + "irot", "Image Rotation", + "clap", "Clean Aperture", + "iscl", "Image Scaling", + "pasp", "Pixel Aspect Ratio", + "pixi", "Pixel Information", + "auxC", "Auxiliary Type", + "rloc", "Reference Location", + "udes", "User Description", + "ipmc", "IPMP Control Box", + "ipco", "Item Property Container", + "ipma", "Item Property Association", + "mif1", "High Efficient File brand", + "avif", "AVIF brand", + "avis", "AVIF image sequence brand", + "avio", "AVIF intra-only image sequence brand", + "msf1", "HEIF image sequence structural brand", + "iso8", "ISO base media version 8 structural brand", + "jpeg", "Legacy JPEG coded tile", + "jpgs", "JPEG image sequence brand", + "av01", "AOMedia Video Coding tile", + "dinf", "Data Information", + "grpl", "Group list", + "hdlr", "Handler", + "iinf", "Item Information", + "iprp", "Item Property", + "ipro", "Item Protection", + "iref", "Item Reference", + "grid", "Grid", + "dimg", "Derived Image", + "thmb", "Thumbnail", + "auxl", "Auxiliary Image", + "prem", "Premultiplied Alpha", + "cdsc", "Content Description", + "mime", "MIME type", + "uri ", "URI", + "pict", "Picture handler type", + "auxv", "Auxiliary image sequence handler type", + "vide", "Video handler type", + "uuid", "Unique Identifier", + "free", "Free space", + "nclx", "CICP color information", + "rICC", "Restricted ICC color profile", + "prof", "Unrestricted ICC color profile", + }; +#> + +using System.CodeDom.Compiler; + +namespace SixLabors.ImageSharp.Formats.Heif; + +/// +/// Supported 4 character codes for use in HEIF images. +/// +[GeneratedCode("TextTemplateFileGenerator", "")] +public enum Heif4CharCode : uint +{ +<# + for (int i = 0; i < codes.Length; i += 2) + { + string shortName = codes[i]; + string longName = codes[i + 1]; + string pascal = PascalCasing(shortName); + string hex = Code2Hex(shortName); +#> + /// + /// <#= longName #>. + /// + <#= pascal #> = <#= hex #>, + +<# + } +#> +} +<#+ + +private string PascalCasing(string code) +{ + char firstChar = char.ToUpper(code[0], System.Globalization.CultureInfo.InvariantCulture); + return string.Concat(firstChar, code.Substring(1)); +} + +private string Code2Hex(string code) +{ + byte[] b = Encoding.ASCII.GetBytes(code); + return String.Format("0x{0:X2}{1:X2}{2:X2}{3:X2}U", b[0], b[1], b[2], b[3]); +} + +#> diff --git a/src/ImageSharp/Formats/Heif/HeifAmbientViewingEnvironment.cs b/src/ImageSharp/Formats/Heif/HeifAmbientViewingEnvironment.cs new file mode 100644 index 0000000000..f5bf8d6a4a --- /dev/null +++ b/src/ImageSharp/Formats/Heif/HeifAmbientViewingEnvironment.cs @@ -0,0 +1,75 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.ColorProfiles; + +namespace SixLabors.ImageSharp.Formats.Heif; + +/// +/// Describes the nominal ambient environment intended for viewing a HEIF image. +/// +public readonly struct HeifAmbientViewingEnvironment : IEquatable +{ + /// + /// Initializes a new instance of the struct. + /// + /// The environmental illuminance in lux. + /// + /// The CIE 1931 chromaticity coordinates of the ambient light in the nominal viewing environment. + /// + public HeifAmbientViewingEnvironment(double illuminance, CieXyChromaticityCoordinates ambientLight) + { + this.Illuminance = illuminance; + this.AmbientLight = ambientLight; + } + + /// + /// Gets the environmental illuminance in lux. + /// + public double Illuminance { get; } + + /// + /// Gets the CIE 1931 chromaticity coordinates of the ambient light in the nominal viewing environment. + /// + public CieXyChromaticityCoordinates AmbientLight { get; } + + /// + /// Compares two ambient viewing environments for equality. + /// + /// The first ambient viewing environment. + /// The second ambient viewing environment. + /// when the illuminance and ambient-light coordinates are equal. + public static bool operator ==(HeifAmbientViewingEnvironment left, HeifAmbientViewingEnvironment right) + => left.Equals(right); + + /// + /// Compares two ambient viewing environments for inequality. + /// + /// The first ambient viewing environment. + /// The second ambient viewing environment. + /// when the illuminance or ambient-light coordinates differ. + public static bool operator !=(HeifAmbientViewingEnvironment left, HeifAmbientViewingEnvironment right) + => !left.Equals(right); + + /// + /// Determines whether the specified object is an ambient viewing environment with the same values. + /// + /// The object to compare with this value. + /// when contains the same environment values. + public override bool Equals(object? obj) + => obj is HeifAmbientViewingEnvironment other && this.Equals(other); + + /// + /// Determines whether the specified ambient viewing environment has the same values as this value. + /// + /// The ambient viewing environment to compare with this value. + /// when the illuminance and ambient-light coordinates are equal. + public bool Equals(HeifAmbientViewingEnvironment other) + => this.Illuminance.Equals(other.Illuminance) && this.AmbientLight.Equals(other.AmbientLight); + + /// + /// Returns a hash code for this ambient viewing environment. + /// + /// A hash code derived from the illuminance and ambient-light coordinates. + public override int GetHashCode() => HashCode.Combine(this.Illuminance, this.AmbientLight); +} diff --git a/src/ImageSharp/Formats/Heif/HeifBitDepth.cs b/src/ImageSharp/Formats/Heif/HeifBitDepth.cs new file mode 100644 index 0000000000..470e3ba874 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/HeifBitDepth.cs @@ -0,0 +1,25 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif; + +/// +/// Enumerates the supported HEIF image-component bit depths. +/// +public enum HeifBitDepth : byte +{ + /// + /// Eight bits per image component. + /// + Bit8 = 8, + + /// + /// Ten bits per image component. + /// + Bit10 = 10, + + /// + /// Twelve bits per image component. + /// + Bit12 = 12 +} diff --git a/src/ImageSharp/Formats/Heif/HeifBoxPayloadReader.cs b/src/ImageSharp/Formats/Heif/HeifBoxPayloadReader.cs new file mode 100644 index 0000000000..7e598e9be4 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/HeifBoxPayloadReader.cs @@ -0,0 +1,211 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Buffers.Binary; + +namespace SixLabors.ImageSharp.Formats.Heif; + +/// +/// Reads fixed-width values and null-terminated strings from one bounded HEIF box payload. +/// +internal ref struct HeifBoxPayloadReader +{ + /// + /// The source stream shared by the container parser. + /// + private readonly Stream stream; + + /// + /// The caller-owned buffer reused for sequential values. + /// + private readonly Span buffer; + + /// + /// The payload name used in malformed-image diagnostics. + /// + private readonly string name; + + /// + /// The number of bytes not yet loaded from the bounded payload. + /// + private long remaining; + + /// + /// The next unread byte in . + /// + private int offset; + + /// + /// The number of valid bytes currently stored in . + /// + private int count; + + /// + /// Initializes a new instance of the struct. + /// + /// The stream positioned at the bounded payload. + /// The exact number of payload bytes. + /// The caller-owned reusable buffer. + /// The payload name used in malformed-image diagnostics. + public HeifBoxPayloadReader(Stream stream, long length, Span buffer, string name) + { + this.stream = stream; + this.buffer = buffer; + this.name = name; + this.remaining = length; + this.offset = 0; + this.count = 0; + } + + /// + /// Gets the number of unread bytes in the bounded payload. + /// + public readonly long Remaining => this.remaining + this.count - this.offset; + + /// + /// Gets a value indicating whether the complete bounded payload has been consumed. + /// + public readonly bool IsComplete => this.Remaining == 0; + + /// + /// Reads one unsigned byte from the bounded payload. + /// + /// The next byte. + public byte ReadByte() + { + this.Ensure(1); + return this.buffer[this.offset++]; + } + + /// + /// Reads one big-endian unsigned 16-bit value from the bounded payload. + /// + /// The next 16-bit value. + public ushort ReadUInt16() + { + this.Ensure(2); + ushort value = BinaryPrimitives.ReadUInt16BigEndian(this.buffer[this.offset..]); + this.offset += 2; + return value; + } + + /// + /// Reads one big-endian unsigned 32-bit value from the bounded payload. + /// + /// The next 32-bit value. + public uint ReadUInt32() + { + this.Ensure(4); + uint value = BinaryPrimitives.ReadUInt32BigEndian(this.buffer[this.offset..]); + this.offset += 4; + return value; + } + + /// + /// Reads one big-endian unsigned 64-bit value from the bounded payload. + /// + /// The next 64-bit value. + public ulong ReadUInt64() + { + this.Ensure(8); + ulong value = BinaryPrimitives.ReadUInt64BigEndian(this.buffer[this.offset..]); + this.offset += 8; + return value; + } + + /// + /// Reads a zero-width, 32-bit, or 64-bit unsigned field. + /// + /// The field width in bytes. + /// The decoded unsigned value. + public ulong ReadVariableUInt(int size) => size switch + { + 0 => 0, + 4 => this.ReadUInt32(), + 8 => this.ReadUInt64(), + _ => throw new InvalidImageContentException($"The {this.name} payload uses an unsupported integer field size.") + }; + + /// + /// Consumes one null-terminated byte string without materializing it. + /// + public void SkipNullTerminatedString() + { + while (this.Remaining > 0) + { + if (this.ReadByte() == 0) + { + return; + } + } + + throw new InvalidImageContentException($"The {this.name} payload contains an unterminated string."); + } + + /// + /// Consumes one null-terminated byte string and compares it with an expected ASCII value. + /// + /// The expected ASCII bytes without a null terminator. + /// when the complete string matches . + public bool ReadNullTerminatedStringEquals(ReadOnlySpan expected) + { + int index = 0; + bool equals = true; + while (this.Remaining > 0) + { + byte value = this.ReadByte(); + if (value == 0) + { + return equals && index == expected.Length; + } + + if ((uint)index >= (uint)expected.Length || value != expected[index]) + { + equals = false; + } + + index++; + } + + throw new InvalidImageContentException($"The {this.name} payload contains an unterminated string."); + } + + /// + /// Refills the reusable buffer without reading beyond the bounded payload. + /// + /// The number of contiguous bytes required by the next value. + private void Ensure(int required) + { + int buffered = this.count - this.offset; + if (buffered >= required) + { + return; + } + + if (buffered > 0) + { + // Preserve an incomplete fixed-width value at the start of the buffer before the next read. + this.buffer.Slice(this.offset, buffered).CopyTo(this.buffer); + } + + this.offset = 0; + this.count = buffered; + while (this.count < required && this.remaining > 0) + { + int requested = (int)Math.Min(this.buffer.Length - this.count, this.remaining); + int read = this.stream.Read(this.buffer.Slice(this.count, requested)); + if (read == 0) + { + throw new InvalidImageContentException($"The {this.name} payload is truncated."); + } + + this.count += read; + this.remaining -= read; + } + + if (this.count < required) + { + throw new InvalidImageContentException($"The {this.name} payload is truncated."); + } + } +} diff --git a/src/ImageSharp/Formats/Heif/HeifBoxReader.cs b/src/ImageSharp/Formats/Heif/HeifBoxReader.cs new file mode 100644 index 0000000000..9904bdeb29 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/HeifBoxReader.cs @@ -0,0 +1,223 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Buffers; +using System.Buffers.Binary; +using SixLabors.ImageSharp.Memory; + +namespace SixLabors.ImageSharp.Formats.Heif; + +/// +/// Reads bounded ISO BMFF box headers and payloads used by the HEIF image container. +/// +internal readonly struct HeifBoxReader +{ + /// + /// The allocator used for payloads that must be materialized while parsing. + /// + private readonly MemoryAllocator allocator; + + /// + /// Initializes a new instance of the struct. + /// + /// The allocator used for bounded payload buffers. + public HeifBoxReader(MemoryAllocator allocator) => this.allocator = allocator; + + /// + /// Reads an ISO BMFF box header using caller-owned scratch and resolves its validated payload length. + /// + /// The stream positioned at the box size field. + /// The absolute end position of the containing box or file. + /// Caller-owned scratch containing at least eight bytes. + /// Receives the box four-character code. + /// Indicates whether a size-zero box may extend to the end of the file. + /// The number of payload bytes following the complete variable-length header. + public static long ReadHeader(Stream stream, long parentEndPosition, Span scratch, out Heif4CharCode boxType, bool topLevel = false) + { + if (parentEndPosition - stream.Position < 8) + { + throw new InvalidImageContentException("Not enough data to read the box header."); + } + + Span buffer = scratch[..8]; + ReadExactly(stream, buffer, "Not enough data to read the box header."); + + ulong boxSize = BinaryPrimitives.ReadUInt32BigEndian(buffer); + int headerSize = 8; + boxType = (Heif4CharCode)BinaryPrimitives.ReadUInt32BigEndian(buffer[4..]); + + if (boxSize == 1) + { + if (parentEndPosition - stream.Position < 8) + { + throw new InvalidImageContentException("Not enough data to read the extended box size."); + } + + ReadExactly(stream, buffer, "Not enough data to read the extended box size."); + boxSize = BinaryPrimitives.ReadUInt64BigEndian(buffer); + headerSize += 8; + } + + if (boxType == Heif4CharCode.Uuid) + { + if (parentEndPosition - stream.Position < 16) + { + throw new InvalidImageContentException("Not enough data to read the UUID box user type."); + } + + // The UUID user type belongs to the variable box header even though the bounded image parser does not + // interpret it. Advance here so every caller receives the actual payload start and length. + Skip(stream, 16); + headerSize += 16; + } + + if (boxSize == 0) + { + if (!topLevel) + { + throw new InvalidImageContentException("A nested box cannot extend to the end of the file."); + } + + return parentEndPosition - stream.Position; + } + + if (boxSize < (ulong)headerSize) + { + throw new InvalidImageContentException("Box size is smaller than its header."); + } + + ulong contentLength = boxSize - (ulong)headerSize; + if (contentLength > (ulong)(parentEndPosition - stream.Position)) + { + throw new InvalidImageContentException("Box size extends beyond its parent boundary."); + } + + return (long)contentLength; + } + + /// + /// Parses an ISO BMFF child-box header from a bounded parent payload. + /// + /// The remaining bytes in the parent payload, beginning at the child size field. + /// Receives the validated child payload length. + /// Receives the child box four-character code. + /// The number of bytes occupied by the complete child header. + public static int ParseHeader(ReadOnlySpan buffer, out long length, out Heif4CharCode boxType) + { + if (buffer.Length < 8) + { + throw new InvalidImageContentException("Not enough data to read the box header."); + } + + ulong boxSize = BinaryPrimitives.ReadUInt32BigEndian(buffer); + int bytesRead = 8; + boxType = (Heif4CharCode)BinaryPrimitives.ReadUInt32BigEndian(buffer[4..]); + if (boxSize == 1) + { + if (buffer.Length < 16) + { + throw new InvalidImageContentException("Not enough data to read the extended box size."); + } + + boxSize = BinaryPrimitives.ReadUInt64BigEndian(buffer[bytesRead..]); + bytesRead += 8; + } + + if (boxType == Heif4CharCode.Uuid) + { + if (buffer.Length - bytesRead < 16) + { + throw new InvalidImageContentException("Not enough data to read the UUID box user type."); + } + + bytesRead += 16; + } + + if (boxSize == 0) + { + throw new InvalidImageContentException("A nested box cannot extend to the end of the file."); + } + + if (boxSize < (ulong)bytesRead) + { + throw new InvalidImageContentException("Box size is smaller than its header."); + } + + ulong contentLength = boxSize - (ulong)bytesRead; + if (contentLength > (ulong)(buffer.Length - bytesRead)) + { + throw new InvalidImageContentException("Box size extends beyond its parent boundary."); + } + + length = (long)contentLength; + return bytesRead; + } + + /// + /// Reads a complete bounded box payload into allocator-owned memory. + /// + /// The stream positioned at the payload start. + /// The validated payload length. + /// An owner containing exactly the requested payload bytes. + public IMemoryOwner ReadPayload(Stream stream, long length) + { + if ((ulong)length > int.MaxValue) + { + throw new InvalidImageContentException("Box content is too large to buffer."); + } + + int bufferLength = (int)length; + IMemoryOwner memory = this.allocator.Allocate(bufferLength); + try + { + ReadExactly(stream, memory.GetSpan(), "Stream length is not sufficient for box content."); + return memory; + } + catch + { + memory.Dispose(); + throw; + } + } + + /// + /// Advances over a validated box payload without narrowing its 64-bit length. + /// + /// The seekable container stream. + /// The validated payload length. + public static void Skip(Stream stream, long length) => stream.Seek(length, SeekOrigin.Current); + + /// + /// Validates a child payload length against the bytes remaining in its parent. + /// + /// The declared child payload length. + /// The number of bytes remaining in the parent. + public static void EnsureInsideParent(long length, long parentLength) + { + if (length < 0 || parentLength < 0 || length > parentLength) + { + throw new InvalidImageContentException("Box size extends beyond its parent boundary."); + } + } + + /// + /// Reads exactly the requested number of bytes or rejects the truncated payload. + /// + /// The source stream. + /// The complete destination span. + /// The malformed-image message used when the stream ends early. + public static void ReadExactly(Stream stream, Span destination, string message) + { + int offset = 0; + while (offset < destination.Length) + { + int read = stream.Read(destination[offset..]); + if (read == 0) + { + throw new InvalidImageContentException(message); + } + + offset += read; + } + } +} diff --git a/src/ImageSharp/Formats/Heif/HeifChromaSubsampling.cs b/src/ImageSharp/Formats/Heif/HeifChromaSubsampling.cs new file mode 100644 index 0000000000..099430c7cf --- /dev/null +++ b/src/ImageSharp/Formats/Heif/HeifChromaSubsampling.cs @@ -0,0 +1,30 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif; + +/// +/// Enumerates the chroma sampling layouts supported for HEIF image encoding. +/// +public enum HeifChromaSubsampling : byte +{ + /// + /// A single luminance plane without chroma planes. + /// + Monochrome, + + /// + /// Chroma sampled at half the luma resolution horizontally and vertically. + /// + Yuv420, + + /// + /// Chroma sampled at half the luma resolution horizontally and full resolution vertically. + /// + Yuv422, + + /// + /// Chroma sampled at full luma resolution horizontally and vertically. + /// + Yuv444 +} diff --git a/src/ImageSharp/Formats/Heif/HeifCleanAperture.cs b/src/ImageSharp/Formats/Heif/HeifCleanAperture.cs new file mode 100644 index 0000000000..029a95a3f0 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/HeifCleanAperture.cs @@ -0,0 +1,160 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif; + +/// +/// Describes the fractional dimensions and center offsets of a HEIF clean-aperture property. +/// +internal readonly struct HeifCleanAperture : IEquatable +{ + /// + /// Initializes a new instance of the struct. + /// + /// The clean-aperture width numerator. + /// The clean-aperture width denominator. + /// The clean-aperture height numerator. + /// The clean-aperture height denominator. + /// The horizontal center-offset numerator. + /// The horizontal center-offset denominator. + /// The vertical center-offset numerator. + /// The vertical center-offset denominator. + public HeifCleanAperture( + int widthNumerator, + int widthDenominator, + int heightNumerator, + int heightDenominator, + int horizontalOffsetNumerator, + int horizontalOffsetDenominator, + int verticalOffsetNumerator, + int verticalOffsetDenominator) + { + this.WidthNumerator = widthNumerator; + this.WidthDenominator = widthDenominator; + this.HeightNumerator = heightNumerator; + this.HeightDenominator = heightDenominator; + this.HorizontalOffsetNumerator = horizontalOffsetNumerator; + this.HorizontalOffsetDenominator = horizontalOffsetDenominator; + this.VerticalOffsetNumerator = verticalOffsetNumerator; + this.VerticalOffsetDenominator = verticalOffsetDenominator; + } + + /// + /// Gets the clean-aperture width numerator. + /// + public int WidthNumerator { get; } + + /// + /// Gets the clean-aperture width denominator. + /// + public int WidthDenominator { get; } + + /// + /// Gets the clean-aperture height numerator. + /// + public int HeightNumerator { get; } + + /// + /// Gets the clean-aperture height denominator. + /// + public int HeightDenominator { get; } + + /// + /// Gets the horizontal center-offset numerator. + /// + public int HorizontalOffsetNumerator { get; } + + /// + /// Gets the horizontal center-offset denominator. + /// + public int HorizontalOffsetDenominator { get; } + + /// + /// Gets the vertical center-offset numerator. + /// + public int VerticalOffsetNumerator { get; } + + /// + /// Gets the vertical center-offset denominator. + /// + public int VerticalOffsetDenominator { get; } + + /// + /// Converts the clean aperture to an integer crop rectangle within the coded image extent. + /// + /// The coded image dimensions. + /// The clean-aperture crop rectangle. + /// + /// The clean-aperture fractions do not describe an integer rectangle inside the coded image extent. + /// + public Rectangle ToRectangle(Size imageExtent) + { + if (this.WidthNumerator <= 0 || this.HeightNumerator <= 0 || + this.WidthDenominator <= 0 || this.HeightDenominator <= 0 || + this.HorizontalOffsetDenominator <= 0 || this.VerticalOffsetDenominator <= 0) + { + throw new InvalidImageContentException("The clean aperture dimensions and denominators must be positive."); + } + + if ((this.WidthNumerator % this.WidthDenominator) != 0 || + (this.HeightNumerator % this.HeightDenominator) != 0) + { + throw new InvalidImageContentException("The clean aperture dimensions must resolve to integer pixels."); + } + + int width = this.WidthNumerator / this.WidthDenominator; + int height = this.HeightNumerator / this.HeightDenominator; + + // Express each top-left coordinate over twice the offset denominator. This is the exact + // center-plus-offset-minus-half-size equation without floating-point rounding. + long xNumerator = ((long)(imageExtent.Width - width) * this.HorizontalOffsetDenominator) + + (2L * this.HorizontalOffsetNumerator); + long xDenominator = 2L * this.HorizontalOffsetDenominator; + long yNumerator = ((long)(imageExtent.Height - height) * this.VerticalOffsetDenominator) + + (2L * this.VerticalOffsetNumerator); + long yDenominator = 2L * this.VerticalOffsetDenominator; + + if ((xNumerator % xDenominator) != 0 || (yNumerator % yDenominator) != 0) + { + throw new InvalidImageContentException("The clean aperture offsets must resolve to integer pixels."); + } + + long x = xNumerator / xDenominator; + long y = yNumerator / yDenominator; + if (x < 0 || y < 0 || x + width > imageExtent.Width || y + height > imageExtent.Height) + { + throw new InvalidImageContentException("The clean aperture lies outside the coded image extent."); + } + + return new Rectangle((int)x, (int)y, width, height); + } + + /// + public bool Equals(HeifCleanAperture other) + => this.WidthNumerator == other.WidthNumerator && + this.WidthDenominator == other.WidthDenominator && + this.HeightNumerator == other.HeightNumerator && + this.HeightDenominator == other.HeightDenominator && + this.HorizontalOffsetNumerator == other.HorizontalOffsetNumerator && + this.HorizontalOffsetDenominator == other.HorizontalOffsetDenominator && + this.VerticalOffsetNumerator == other.VerticalOffsetNumerator && + this.VerticalOffsetDenominator == other.VerticalOffsetDenominator; + + /// + public override bool Equals(object? obj) => obj is HeifCleanAperture other && this.Equals(other); + + /// + public override int GetHashCode() + { + HashCode hashCode = default; + hashCode.Add(this.WidthNumerator); + hashCode.Add(this.WidthDenominator); + hashCode.Add(this.HeightNumerator); + hashCode.Add(this.HeightDenominator); + hashCode.Add(this.HorizontalOffsetNumerator); + hashCode.Add(this.HorizontalOffsetDenominator); + hashCode.Add(this.VerticalOffsetNumerator); + hashCode.Add(this.VerticalOffsetDenominator); + return hashCode.ToHashCode(); + } +} diff --git a/src/ImageSharp/Formats/Heif/HeifCompressionFactory.cs b/src/ImageSharp/Formats/Heif/HeifCompressionFactory.cs new file mode 100644 index 0000000000..df00957036 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/HeifCompressionFactory.cs @@ -0,0 +1,26 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.PixelFormats; + +namespace SixLabors.ImageSharp.Formats.Heif; + +/// +/// Selects the still-image decoder for a compressed HEIF image item. +/// +internal static class HeifCompressionFactory +{ + /// + /// Gets a decoder for the specified compressed image item type. + /// + /// The destination pixel type. + /// The image item type. + /// A matching item decoder, or when the item type is not supported. + public static IHeifItemDecoder? GetDecoder(Heif4CharCode type) + where TPixel : unmanaged, IPixel => type switch + { + Heif4CharCode.Jpeg => new JpegHeifItemDecoder(), + Heif4CharCode.Av01 => new Av1HeifItemDecoder(), + _ => null + }; +} diff --git a/src/ImageSharp/Formats/Heif/HeifCompressionMethod.cs b/src/ImageSharp/Formats/Heif/HeifCompressionMethod.cs new file mode 100644 index 0000000000..2accbeec3c --- /dev/null +++ b/src/ImageSharp/Formats/Heif/HeifCompressionMethod.cs @@ -0,0 +1,20 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif; + +/// +/// Identifies the compression method used by a coded image item in a HEIF file. +/// +public enum HeifCompressionMethod +{ + /// + /// Legacy JPEG coding. + /// + LegacyJpeg, + + /// + /// AOMedia Video 1 (AV1) coding. + /// + Av1, +} diff --git a/src/ImageSharp/Formats/Heif/HeifConfigurationModule.cs b/src/ImageSharp/Formats/Heif/HeifConfigurationModule.cs new file mode 100644 index 0000000000..c4077cb29b --- /dev/null +++ b/src/ImageSharp/Formats/Heif/HeifConfigurationModule.cs @@ -0,0 +1,18 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif; + +/// +/// Configures HEIF image-format support. +/// +public sealed class HeifConfigurationModule : IImageFormatConfigurationModule +{ + /// + public void Configure(Configuration configuration) + { + configuration.ImageFormatsManager.SetEncoder(HeifFormat.Instance, new HeifEncoder()); + configuration.ImageFormatsManager.SetDecoder(HeifFormat.Instance, HeifDecoder.Instance); + configuration.ImageFormatsManager.AddImageFormatDetector(new HeifImageFormatDetector()); + } +} diff --git a/src/ImageSharp/Formats/Heif/HeifConstants.cs b/src/ImageSharp/Formats/Heif/HeifConstants.cs new file mode 100644 index 0000000000..ed184edcc1 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/HeifConstants.cs @@ -0,0 +1,120 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Buffers.Binary; + +namespace SixLabors.ImageSharp.Formats.Heif; + +/// +/// Contains HEIF constant values defined in the specification. +/// +internal static class HeifConstants +{ + /// + /// The auxiliary-type URN used by current HEIF alpha image items. + /// + public const string AlphaAuxiliaryType = "urn:mpeg:mpegB:cicp:systems:auxiliary:alpha"; + + /// + /// The MIME types recognized by this HEIF implementation. + /// + public static readonly IEnumerable MimeTypes = new[] { "image/heif", "image/avif" }; + + /// + /// The file extensions recognized by this HEIF implementation. + /// + public static readonly IEnumerable FileExtensions = new[] { "heif", "hif", "avif" }; + + /// + /// Determines the supported image presentation declared by a file-type box. + /// + /// + /// The file-type box payload, beginning with the major brand and minor version and followed by compatible brands. + /// + /// Receives the supported still-image or image-sequence presentation. + /// when the payload declares a supported HEIF image presentation. + public static bool TryGetFileType(ReadOnlySpan boxContent, out HeifFileType fileType) + { + fileType = HeifFileType.Unsupported; + + // Every brand is a four-character code. The payload must contain the major brand and minor version before + // any compatible brands, otherwise accepting a partial trailing code could produce a false detection. + if (boxContent.Length < 8 || (boxContent.Length & 3) != 0) + { + return false; + } + + Heif4CharCode majorBrand = (Heif4CharCode)BinaryPrimitives.ReadUInt32BigEndian(boxContent); + if (IsUnsupportedSequenceBrand(majorBrand)) + { + // A sequence major brand controls the presentation even if a still-image compatible brand is present. + return false; + } + + if (IsSupportedSequenceBrand(majorBrand)) + { + fileType = HeifFileType.ImageSequence; + return true; + } + + if (IsSupportedStillImageBrand(majorBrand)) + { + fileType = HeifFileType.StillImage; + return true; + } + + // The minor-version field follows the major brand; compatible brands start at byte eight. + bool hasStillImageBrand = false; + for (int offset = 8; offset < boxContent.Length; offset += 4) + { + Heif4CharCode compatibleBrand = (Heif4CharCode)BinaryPrimitives.ReadUInt32BigEndian(boxContent[offset..]); + if (IsSupportedSequenceBrand(compatibleBrand)) + { + fileType = HeifFileType.ImageSequence; + return true; + } + + if (IsSupportedStillImageBrand(compatibleBrand)) + { + hasStillImageBrand = true; + } + } + + fileType = hasStillImageBrand ? HeifFileType.StillImage : HeifFileType.Unsupported; + return hasStillImageBrand; + } + + /// + /// Determines whether an auxiliary-type property identifies an alpha image plane. + /// + /// The null-terminated auxiliary type decoded from an auxC property. + /// when the type is the registered HEIF alpha URN. + public static bool IsAlphaAuxiliaryType(string? auxiliaryType) + => auxiliaryType == AlphaAuxiliaryType; + + /// + /// Determines whether identifies a still-image container supported by this codec. + /// + /// The registered file-type brand. + /// when the brand identifies a supported still-image container. + private static bool IsSupportedStillImageBrand(Heif4CharCode brand) + => brand is Heif4CharCode.Mif1 + or Heif4CharCode.Avif + or Heif4CharCode.Jpeg; + + /// + /// Determines whether identifies a supported timed image sequence. + /// + /// The registered file-type brand. + /// when the brand identifies a supported timed image sequence. + private static bool IsSupportedSequenceBrand(Heif4CharCode brand) + => brand is Heif4CharCode.Avis; + + /// + /// Determines whether requires an image-sequence profile outside the implemented scope. + /// + /// The registered file-type brand. + /// when the major brand requires unsupported JPEG sequence support. + private static bool IsUnsupportedSequenceBrand(Heif4CharCode brand) + => brand is Heif4CharCode.Jpgs; +} diff --git a/src/ImageSharp/Formats/Heif/HeifContentColorVolume.cs b/src/ImageSharp/Formats/Heif/HeifContentColorVolume.cs new file mode 100644 index 0000000000..41f7793dde --- /dev/null +++ b/src/ImageSharp/Formats/Heif/HeifContentColorVolume.cs @@ -0,0 +1,117 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.ColorProfiles; + +namespace SixLabors.ImageSharp.Formats.Heif; + +/// +/// Describes the optional color-primary and luminance limits of the image content represented by a HEIF image. +/// +public readonly struct HeifContentColorVolume : IEquatable +{ + /// + /// Initializes a new instance of the struct. + /// + /// + /// The CIE 1931 chromaticity coordinates of the content color primaries, or when they + /// are not specified. + /// + /// + /// The normalized minimum content luminance, or when it is not specified. + /// + /// + /// The normalized maximum content luminance, or when it is not specified. + /// + /// + /// The normalized average content luminance, or when it is not specified. + /// + public HeifContentColorVolume( + RgbPrimariesChromaticityCoordinates? primaries, + double? minimumLuminance, + double? maximumLuminance, + double? averageLuminance) + { + this.Primaries = primaries; + this.MinimumLuminance = minimumLuminance; + this.MaximumLuminance = maximumLuminance; + this.AverageLuminance = averageLuminance; + } + + /// + /// Gets the CIE 1931 chromaticity coordinates of the content color primaries, or when + /// they are not specified. + /// + public RgbPrimariesChromaticityCoordinates? Primaries { get; } + + /// + /// Gets the normalized minimum content luminance, or when it is not specified. + /// + /// + /// The value is interpreted according to the transfer characteristics signaled for the image and does not + /// necessarily represent luminance in candelas per square metre. + /// + public double? MinimumLuminance { get; } + + /// + /// Gets the normalized maximum content luminance, or when it is not specified. + /// + /// + /// The value is interpreted according to the transfer characteristics signaled for the image and does not + /// necessarily represent luminance in candelas per square metre. + /// + public double? MaximumLuminance { get; } + + /// + /// Gets the normalized average content luminance, or when it is not specified. + /// + /// + /// The value is interpreted according to the transfer characteristics signaled for the image and does not + /// necessarily represent luminance in candelas per square metre. + /// + public double? AverageLuminance { get; } + + /// + /// Compares two content color volumes for equality. + /// + /// The first content color volume. + /// The second content color volume. + /// when every specified color-volume value is equal. + public static bool operator ==(HeifContentColorVolume left, HeifContentColorVolume right) + => left.Equals(right); + + /// + /// Compares two content color volumes for inequality. + /// + /// The first content color volume. + /// The second content color volume. + /// when any specified color-volume value differs. + public static bool operator !=(HeifContentColorVolume left, HeifContentColorVolume right) + => !left.Equals(right); + + /// + /// Determines whether the specified object is a content color volume with the same values. + /// + /// The object to compare with this value. + /// when contains the same color-volume values. + public override bool Equals(object? obj) + => obj is HeifContentColorVolume other && this.Equals(other); + + /// + /// Determines whether the specified content color volume has the same values as this value. + /// + /// The content color volume to compare with this value. + /// when every specified color-volume value is equal. + public bool Equals(HeifContentColorVolume other) + => this.Primaries.Equals(other.Primaries) + && this.MinimumLuminance.Equals(other.MinimumLuminance) + && this.MaximumLuminance.Equals(other.MaximumLuminance) + && this.AverageLuminance.Equals(other.AverageLuminance); + + /// + /// Returns a hash code for this content color volume. + /// + /// A hash code derived from the specified color-volume values. + public override int GetHashCode() + => HashCode.Combine(this.Primaries, this.MinimumLuminance, this.MaximumLuminance, this.AverageLuminance); +} diff --git a/src/ImageSharp/Formats/Heif/HeifContentLightLevel.cs b/src/ImageSharp/Formats/Heif/HeifContentLightLevel.cs new file mode 100644 index 0000000000..1c1c5f9c17 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/HeifContentLightLevel.cs @@ -0,0 +1,35 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif; + +/// +/// Describes the maximum content and picture-average light levels of a HEIF image. +/// +public readonly struct HeifContentLightLevel +{ + /// + /// Initializes a new instance of the struct. + /// + /// + /// The maximum light level of any individual sample, in candelas per square metre, or zero when unspecified. + /// + /// + /// The maximum average light level of any picture, in candelas per square metre, or zero when unspecified. + /// + public HeifContentLightLevel(ushort maximumContentLightLevel, ushort maximumPictureAverageLightLevel) + { + this.MaximumContentLightLevel = maximumContentLightLevel; + this.MaximumPictureAverageLightLevel = maximumPictureAverageLightLevel; + } + + /// + /// Gets the maximum light level of any individual sample, in candelas per square metre, or zero when unspecified. + /// + public ushort MaximumContentLightLevel { get; } + + /// + /// Gets the maximum average light level of any picture, in candelas per square metre, or zero when unspecified. + /// + public ushort MaximumPictureAverageLightLevel { get; } +} diff --git a/src/ImageSharp/Formats/Heif/HeifDecoder.cs b/src/ImageSharp/Formats/Heif/HeifDecoder.cs new file mode 100644 index 0000000000..8f7e7a2c3f --- /dev/null +++ b/src/ImageSharp/Formats/Heif/HeifDecoder.cs @@ -0,0 +1,50 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.PixelFormats; + +namespace SixLabors.ImageSharp.Formats.Heif; + +/// +/// Image decoder for reading HEIF images from a stream. +/// +public sealed class HeifDecoder : ImageDecoder +{ + /// + /// Initializes a new instance of the class. + /// + private HeifDecoder() + { + } + + /// + /// Gets the shared instance. + /// + public static HeifDecoder Instance { get; } = new(); + + /// + protected override ImageInfo Identify(DecoderOptions options, Stream stream, CancellationToken cancellationToken) + { + Guard.NotNull(options, nameof(options)); + Guard.NotNull(stream, nameof(stream)); + + return new HeifDecoderCore(options).Identify(options.Configuration, stream, cancellationToken); + } + + /// + protected override Image Decode(DecoderOptions options, Stream stream, CancellationToken cancellationToken) + { + Guard.NotNull(options, nameof(options)); + Guard.NotNull(stream, nameof(stream)); + + HeifDecoderCore decoder = new(options); + Image image = decoder.Decode(options.Configuration, stream, cancellationToken); + ScaleToTargetSize(options, image); + + return image; + } + + /// + protected override Image Decode(DecoderOptions options, Stream stream, CancellationToken cancellationToken) + => this.Decode(options, stream, cancellationToken); +} diff --git a/src/ImageSharp/Formats/Heif/HeifDecoderCore.cs b/src/ImageSharp/Formats/Heif/HeifDecoderCore.cs new file mode 100644 index 0000000000..468b83ecc2 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/HeifDecoderCore.cs @@ -0,0 +1,3152 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Buffers; +using System.Buffers.Binary; +using System.Text; +using SixLabors.ImageSharp.Common.Helpers; +using SixLabors.ImageSharp.Formats.Heif.Av1; +using SixLabors.ImageSharp.Formats.Heif.Components.Alpha; +using SixLabors.ImageSharp.IO; +using SixLabors.ImageSharp.Memory; +using SixLabors.ImageSharp.Metadata; +using SixLabors.ImageSharp.Metadata.Profiles.Cicp; +using SixLabors.ImageSharp.Metadata.Profiles.Exif; +using SixLabors.ImageSharp.Metadata.Profiles.Icc; +using SixLabors.ImageSharp.Metadata.Profiles.Xmp; +using SixLabors.ImageSharp.PixelFormats; +using SixLabors.ImageSharp.Processing; +using SixLabors.ImageSharp.Processing.Processors.Transforms; + +namespace SixLabors.ImageSharp.Formats.Heif; + +/// +/// Performs the HEIF decoding operation. +/// +internal sealed class HeifDecoderCore : ImageDecoderCore +{ + /// + /// Marks an item property whose box type is not understood by this decoder. + /// + private static readonly object UnknownProperty = new(); + + /// + /// Marks an understood item property whose value was skipped or discarded by decoder policy. + /// + private static readonly object IgnoredProperty = new(); + + /// + /// The general configuration. + /// + private readonly Configuration configuration; + + /// + /// The general options passed to nested coded-image decoders without presentation-level target scaling. + /// + private readonly DecoderOptions payloadOptions; + + /// + /// The decoded by this decoder instance. + /// + private readonly ImageMetadata metadata; + + /// + /// The shared bounded box reader used by the item and image-sequence container paths. + /// + private readonly HeifBoxReader boxReader; + + /// + /// The bounded image-sequence parser sharing the configured frame and metadata policy. + /// + private readonly HeifSequenceParser sequenceParser; + + /// + /// The fixed scratch buffer reused for all item-container box headers in this decode operation. + /// + private readonly byte[] boxHeaderScratch; + + /// + /// The item identifier selected by the primary-item box. + /// + private uint primaryItem; + + /// + /// The item declarations parsed from the item-information box. + /// + private readonly List items; + + /// + /// The typed relationships parsed from the item-reference box. + /// + private readonly List itemLinks; + + /// + /// The absolute stream position at which the current HEIF file begins. + /// + private long fileStartOffset; + + /// + /// The absolute stream offset of the item-data box payload, or -1 when no item-data box exists. + /// + private long itemDataOffset = -1; + + /// + /// The number of bytes in the item-data box payload. + /// + private long itemDataLength; + + /// + /// Initializes a new instance of the class. + /// + /// The decoder options. + public HeifDecoderCore(DecoderOptions options) + : base(options) + { + this.configuration = options.Configuration; + + // HEIF owns final presentation resizing and ICC conversion after item/grid composition and container-profile + // selection. Nested codecs retain every other general policy but must not apply either operation independently. + this.payloadOptions = options.TargetSize is null && options.ColorProfileHandling == ColorProfileHandling.Preserve + ? options + : new DecoderOptions + { + Configuration = options.Configuration, + Sampler = options.Sampler, + SkipMetadata = options.SkipMetadata, + MaxFrames = options.MaxFrames, + SegmentIntegrityHandling = options.SegmentIntegrityHandling, + ColorProfileHandling = ColorProfileHandling.Preserve + }; + + this.metadata = new ImageMetadata(); + this.boxReader = new HeifBoxReader(this.configuration.MemoryAllocator); + this.sequenceParser = new HeifSequenceParser(options); + this.boxHeaderScratch = new byte[8]; + this.items = []; + this.itemLinks = []; + } + + /// + /// Gets the dependency order in which recognized metadata children are interpreted. + /// + private static ReadOnlySpan MetadataParseOrder => + [ + Heif4CharCode.Hdlr, + Heif4CharCode.Iinf, + Heif4CharCode.Pitm, + Heif4CharCode.Iref, + Heif4CharCode.Iloc, + Heif4CharCode.Iprp, + Heif4CharCode.Idat + ]; + + /// + protected override Image Decode(BufferedReadStream stream, CancellationToken cancellationToken) + { + this.fileStartOffset = stream.Position; + HeifFileType fileType = this.ReadFileTypeBox(stream); + if (fileType == HeifFileType.Unsupported) + { + throw new ImageFormatException("Not an HEIF image."); + } + + if (fileType == HeifFileType.ImageSequence) + { + HeifSequence sequence = this.ParseImageSequence(stream); + return this.DecodeImageSequence(stream, sequence, cancellationToken); + } + + this.items.Clear(); + this.itemLinks.Clear(); + this.itemDataOffset = -1; + this.itemDataLength = 0; + + // Item locations are absolute file offsets or idat-relative offsets, so payload bytes need not be adjacent to + // the metadata box. Complete the top-level scan before resolving and decoding the primary item. + while (stream.Position < stream.Length) + { + long boxLength = HeifBoxReader.ReadHeader(stream, stream.Length, this.boxHeaderScratch, out Heif4CharCode boxType, true); + switch (boxType) + { + case Heif4CharCode.Meta: + this.ParseMetadata(stream, boxLength); + break; + case Heif4CharCode.Mdat: + case Heif4CharCode.Free: + HeifBoxReader.Skip(stream, boxLength); + break; + case 0U: + // Some files have trailing zeros, skiping to EOF. + HeifBoxReader.Skip(stream, stream.Length - stream.Position); + break; + default: + HeifBoxReader.Skip(stream, boxLength); + break; + } + } + + return this.DecodePrimaryItem(stream, cancellationToken); + } + + /// + protected override ImageInfo Identify(BufferedReadStream stream, CancellationToken cancellationToken) + { + this.fileStartOffset = stream.Position; + HeifFileType fileType = this.ReadFileTypeBox(stream); + if (fileType == HeifFileType.Unsupported) + { + throw new ImageFormatException("Not an HEIF image."); + } + + if (fileType == HeifFileType.ImageSequence) + { + return this.IdentifyImageSequence(this.ParseImageSequence(stream)); + } + + this.items.Clear(); + this.itemLinks.Clear(); + this.itemDataOffset = -1; + this.itemDataLength = 0; + + // Identification reads only the container model. Payload boxes remain skipped because dimensions and format + // metadata come from item declarations and associated properties rather than reconstructed pixels. + while (stream.Position < stream.Length) + { + long boxLength = HeifBoxReader.ReadHeader(stream, stream.Length, this.boxHeaderScratch, out Heif4CharCode boxType, true); + switch (boxType) + { + case Heif4CharCode.Meta: + this.ParseMetadata(stream, boxLength); + break; + default: + // Silently skip all other box types. + HeifBoxReader.Skip(stream, boxLength); + break; + } + } + + HeifItem? item = this.FindItemById(this.primaryItem); + if (item is null) + { + throw new ImageFormatException("No primary item found"); + } + + this.UpdateMetadata(this.metadata, item); + + Size presentationExtent = GetPresentationExtent(item); + return new ImageInfo(new(presentationExtent.Width, presentationExtent.Height), this.metadata); + } + + /// + /// Reads and validates the leading file-type box against the image presentations supported by this decoder. + /// + /// The container stream positioned at its first top-level box. + /// The declared supported image presentation, or . + private HeifFileType ReadFileTypeBox(BufferedReadStream stream) + { + long boxLength = HeifBoxReader.ReadHeader(stream, stream.Length, this.boxHeaderScratch, out Heif4CharCode boxType, true); + if (boxType != Heif4CharCode.Ftyp) + { + return HeifFileType.Unsupported; + } + + if (boxLength < 8 || boxLength > int.MaxValue || (boxLength & 3) != 0) + { + return HeifFileType.Unsupported; + } + + using IMemoryOwner boxMemory = this.boxReader.ReadPayload(stream, boxLength); + Span boxBuffer = boxMemory.GetSpan(); + return HeifConstants.TryGetFileType(boxBuffer, out HeifFileType fileType) ? fileType : HeifFileType.Unsupported; + } + + /// + /// Locates and parses the single movie box of a supported AVIF image sequence. + /// + /// The complete container stream positioned after its file-type box. + /// The bounded selected image-sequence model. + private HeifSequence ParseImageSequence(BufferedReadStream stream) + { + this.items.Clear(); + this.itemLinks.Clear(); + this.itemDataOffset = -1; + this.itemDataLength = 0; + + HeifSequence? sequence = null; + while (stream.Position < stream.Length) + { + long boxLength = HeifBoxReader.ReadHeader(stream, stream.Length, this.boxHeaderScratch, out Heif4CharCode boxType, true); + switch (boxType) + { + case Heif4CharCode.Meta: + this.ParseMetadata(stream, boxLength); + break; + case Heif4CharCode.Moov: + if (sequence is not null) + { + throw new InvalidImageContentException("The HEIF image sequence contains more than one movie box."); + } + + sequence = this.sequenceParser.Parse(stream, boxLength, this.fileStartOffset); + break; + default: + // Sequence samples and image items use file-relative offsets, so payload boxes never need buffering. + HeifBoxReader.Skip(stream, boxLength); + break; + } + } + + return sequence ?? throw new InvalidImageContentException("The HEIF image sequence contains no movie box."); + } + + /// + /// Creates image and frame metadata from a parsed AVIF image sequence without decoding its samples. + /// + /// The parsed selected image sequence. + /// The identified dimensions and bounded visible-frame metadata. + private ImageInfo IdentifyImageSequence(HeifSequence sequence) + { + HeifSequenceTrack colorTrack = sequence.ColorTrack; + HeifItem primaryItem = this.FindSequencePrimaryItem(); + bool animateRootFrame = IsPrimaryItemFirstSequenceSample(primaryItem, colorTrack); + Size sequenceExtent = GetSequencePresentationExtent(colorTrack); + this.Dimensions = animateRootFrame ? sequenceExtent : GetPresentationExtent(primaryItem); + if (!animateRootFrame && this.Dimensions != sequenceExtent) + { + throw new InvalidImageContentException("The primary image and image sequence have different presentation dimensions."); + } + + this.UpdateMetadata(this.metadata, primaryItem); + this.UpdateSequenceMetadata(this.metadata, sequence, animateRootFrame); + ImageFrameMetadata[] frameMetadata = CreateSequenceFrameMetadata(colorTrack, animateRootFrame); + return new ImageInfo(this.Dimensions, this.metadata, frameMetadata); + } + + /// + /// Decodes the retained visible samples of an AVIF image sequence into one multi-frame image. + /// + /// The destination pixel format. + /// The complete seekable HEIF stream. + /// The parsed selected image sequence. + /// The token used to cancel work between coded samples. + /// The decoded multi-frame image. + private Image DecodeImageSequence( + BufferedReadStream stream, + HeifSequence sequence, + CancellationToken cancellationToken) + where TPixel : unmanaged, IPixel + { + HeifSequenceTrack colorTrack = sequence.ColorTrack; + HeifItem primaryItem = this.FindSequencePrimaryItem(); + bool animateRootFrame = IsPrimaryItemFirstSequenceSample(primaryItem, colorTrack); + this.UpdateSequenceMetadata(this.metadata, sequence, animateRootFrame); + Size codedSize = new(colorTrack.CodedWidth, colorTrack.CodedHeight); + Rectangle sourceRectangle = colorTrack.CleanAperture is not null + ? colorTrack.CleanAperture.Value.ToRectangle(codedSize) + : new Rectangle(Point.Empty, codedSize); + + // HEIF stores counter-clockwise quarter turns; ImageSharp's exact modes are clockwise. + RotateMode rotation = colorTrack.RotationAngle switch + { + 1 => RotateMode.Rotate270, + 2 => RotateMode.Rotate180, + 3 => RotateMode.Rotate90, + _ => RotateMode.None + }; + + Size presentationSize = rotation is RotateMode.Rotate90 or RotateMode.Rotate270 + ? new Size(sourceRectangle.Height, sourceRectangle.Width) + : sourceRectangle.Size; + + // The returned image owns every presented frame from the outset. A separate primary becomes its root; + // otherwise the first successfully decoded timed sample fills the root allocated here. + Image image = animateRootFrame + ? new Image( + this.configuration, + presentationSize.Width, + presentationSize.Height, + this.metadata) + : this.DecodePrimaryItem(stream, cancellationToken); + + try + { + if (image.Size != presentationSize) + { + throw new InvalidImageContentException( + "The primary image and image sequence have different presentation dimensions."); + } + + using Av1Decoder colorDecoder = new(this.configuration); + + HeifSequenceTrack? alphaTrack = sequence.AlphaTrack; + (HeifSequenceTrack Track, Av1Decoder Decoder)? alphaState = alphaTrack is null + ? null + : (alphaTrack, new Av1Decoder(this.configuration)); + + using Av1Decoder? alphaDecoder = alphaState?.Decoder; + + // Quarter turns need source and destination frames with opposite dimensions. Reuse one source frame + // across the sequence, then rotate each completed color-and-alpha sample into its final owned frame. + using ImageFrame? rotationSource = rotation == RotateMode.None + ? null + : new ImageFrame( + this.configuration, + sourceRectangle.Width, + sourceRectangle.Height); + + int decodedFrameCount = 0; + for (int sampleIndex = 0; sampleIndex < colorTrack.Samples.Length; sampleIndex++) + { + cancellationToken.ThrowIfCancellationRequested(); + HeifSequenceSample colorSample = colorTrack.Samples[sampleIndex]; + if (colorSample.IsHidden) + { + this.ExecuteImageDataSegmentAction( + () => this.DecodeSequenceReference(stream, colorTrack, colorSample, colorDecoder)); + + if (alphaState is not null) + { + (HeifSequenceTrack Track, Av1Decoder Decoder) currentAlphaState = alphaState.Value; + HeifSequenceSample alphaSample = currentAlphaState.Track.Samples[sampleIndex]; + this.ExecuteImageDataSegmentAction( + () => this.DecodeSequenceReference( + stream, + currentAlphaState.Track, + alphaSample, + currentAlphaState.Decoder)); + } + + continue; + } + + bool appendedDestination = false; + ImageFrame decodedFrame; + if (rotationSource is not null) + { + decodedFrame = rotationSource; + } + else if (animateRootFrame && decodedFrameCount == 0) + { + decodedFrame = image.Frames.RootFrame; + } + else + { + decodedFrame = image.Frames.CreateFrame(); + appendedDestination = true; + } + + bool colorDecoded = false; + this.ExecuteImageDataSegmentAction( + () => + { + this.DecodeSequenceFrame( + stream, + colorTrack, + colorSample, + colorDecoder, + sourceRectangle, + decodedFrame); + + colorDecoded = true; + }); + + if (!colorDecoded) + { + if (alphaState is not null) + { + (HeifSequenceTrack Track, Av1Decoder Decoder) currentAlphaState = alphaState.Value; + HeifSequenceSample alphaSample = currentAlphaState.Track.Samples[sampleIndex]; + this.ExecuteImageDataSegmentAction( + () => this.DecodeSequenceReference( + stream, + currentAlphaState.Track, + alphaSample, + currentAlphaState.Decoder)); + } + + if (appendedDestination) + { + image.Frames.RemoveFrame(image.Frames.Count - 1); + } + + continue; + } + + bool alphaDecoded = true; + if (alphaState is not null) + { + alphaDecoded = false; + (HeifSequenceTrack Track, Av1Decoder Decoder) currentAlphaState = alphaState.Value; + HeifSequenceSample alphaSample = currentAlphaState.Track.Samples[sampleIndex]; + this.ExecuteImageDataSegmentAction( + () => + { + this.DecodeSequenceAlphaFrame( + stream, + currentAlphaState.Track, + alphaSample, + currentAlphaState.Decoder, + sourceRectangle, + decodedFrame, + colorTrack.IsPremultiplied); + + alphaDecoded = true; + }); + } + + if (!alphaDecoded) + { + if (appendedDestination) + { + image.Frames.RemoveFrame(image.Frames.Count - 1); + } + + continue; + } + + ImageFrame presentedFrame = decodedFrame; + if (rotationSource is not null) + { + presentedFrame = animateRootFrame && decodedFrameCount == 0 + ? image.Frames.RootFrame + : image.Frames.CreateFrame(); + + RotateProcessor.ApplyQuarterTurn( + rotation, + rotationSource, + presentedFrame, + this.configuration); + + presentedFrame.Metadata.CicpProfile = rotationSource.Metadata.CicpProfile; + } + + if (colorTrack.MirrorAxis is not null) + { + // Axis zero reflects top-to-bottom around the horizontal axis; axis one reflects left-to-right. + FlipMode flip = colorTrack.MirrorAxis.Value == 0 ? FlipMode.Vertical : FlipMode.Horizontal; + FlipProcessor.Apply(flip, presentedFrame, this.configuration); + } + + presentedFrame.Metadata.GetHeifMetadata().FrameDelay = new Rational( + colorSample.Duration, + colorTrack.MediaTimescale); + + if (!animateRootFrame && !this.Options.SkipMetadata) + { + presentedFrame.Metadata.IccProfile = colorTrack.IccProfile; + _ = this.TryConvertIccProfile(presentedFrame); + } + + decodedFrameCount++; + } + + if (decodedFrameCount == 0) + { + throw new InvalidImageContentException( + "The HEIF image sequence contains no decodable visible samples."); + } + + if (!this.Options.SkipMetadata) + { + if (animateRootFrame) + { + image.Metadata.CicpProfile ??= image.Frames.RootFrame.Metadata.CicpProfile?.DeepClone(); + _ = this.TryConvertIccProfile(image); + } + } + else + { + foreach (ImageFrame frame in image.Frames) + { + frame.Metadata.CicpProfile = null; + } + } + + HeifMetadata resultMetadata = image.Metadata.GetHeifMetadata(); + resultMetadata.RepeatCount = colorTrack.RepeatCount; + resultMetadata.AnimateRootFrame = animateRootFrame; + resultMetadata.HasAlpha |= alphaState is not null; + this.Dimensions = image.Size; + return image; + } + catch + { + image.Dispose(); + throw; + } + } + + /// + /// Reads and decodes one bounded coded sample without retaining its encoded byte buffer. + /// + /// The destination pixel format. + /// The complete seekable HEIF stream. + /// The track supplying the codec configuration and color description. + /// The validated sample range. + /// The decoder retaining earlier sequence references. + /// The clean-aperture region mapped to the destination frame. + /// The caller-owned frame receiving the presented sample. + private void DecodeSequenceFrame( + BufferedReadStream stream, + HeifSequenceTrack track, + HeifSequenceSample sample, + Av1Decoder decoder, + Rectangle sourceRectangle, + ImageFrame destination) + where TPixel : unmanaged, IPixel + { + if (track.CodecType != Heif4CharCode.Av01) + { + throw new ImageFormatException($"No decoder is available for image-sequence sample type '{track.CodecType}'."); + } + + Av1CodecConfiguration codecConfiguration = track.Av1CodecConfiguration + ?? throw new InvalidImageContentException("The AV1 image-sequence track has no codec configuration."); + + using IMemoryOwner sampleOwner = this.ReadSequenceSample(stream, track, sample); + Span sampleData = sampleOwner.GetSpan()[..sample.Length]; + + decoder.DecodeSequenceFrame( + sampleData, + track.CicpProfile, + codecConfiguration, + new Size(track.CodedWidth, track.CodedHeight), + sourceRectangle, + destination); + } + + /// + /// Decodes one AV1 auxiliary sample and composes its native luma plane directly into a color frame. + /// + /// The destination color pixel type. + /// The complete seekable HEIF stream. + /// The alpha track supplying the codec configuration and color description. + /// The validated alpha sample range. + /// The decoder retaining earlier alpha-sequence references. + /// The clean-aperture region mapped to the destination frame. + /// The decoded color frame receiving alpha values. + /// Whether stored color samples must be converted to unassociated alpha. + private void DecodeSequenceAlphaFrame( + BufferedReadStream stream, + HeifSequenceTrack track, + HeifSequenceSample sample, + Av1Decoder decoder, + Rectangle sourceRectangle, + ImageFrame destination, + bool premultiplied) + where TPixel : unmanaged, IPixel + { + if (track.CodecType != Heif4CharCode.Av01) + { + throw new ImageFormatException($"No decoder is available for image-sequence alpha sample type '{track.CodecType}'."); + } + + Av1CodecConfiguration codecConfiguration = track.Av1CodecConfiguration + ?? throw new InvalidImageContentException("The AV1 alpha image-sequence track has no codec configuration."); + + if (!codecConfiguration.IsMonochrome) + { + throw new InvalidImageContentException("An AV1 alpha image-sequence track must be encoded as monochrome."); + } + + using IMemoryOwner sampleOwner = this.ReadSequenceSample(stream, track, sample); + Span sampleData = sampleOwner.GetSpan()[..sample.Length]; + decoder.DecodeSequenceAlpha( + sampleData, + track.CicpProfile, + codecConfiguration, + new Size(track.CodedWidth, track.CodedHeight), + sourceRectangle, + destination, + destination.Size, + destination.Bounds, + premultiplied); + } + + /// + /// Decodes one non-presented sequence sample so later dependent samples can resolve its retained references. + /// + /// The complete seekable HEIF stream. + /// The track supplying the codec configuration and color description. + /// The validated non-presented sample. + /// The decoder retaining sequence reference state. + private void DecodeSequenceReference( + BufferedReadStream stream, + HeifSequenceTrack track, + HeifSequenceSample sample, + Av1Decoder decoder) + { + if (track.CodecType != Heif4CharCode.Av01) + { + throw new ImageFormatException($"No decoder is available for image-sequence sample type '{track.CodecType}'."); + } + + Av1CodecConfiguration codecConfiguration = track.Av1CodecConfiguration + ?? throw new InvalidImageContentException("The AV1 image-sequence track has no codec configuration."); + + using IMemoryOwner sampleOwner = this.ReadSequenceSample(stream, track, sample); + Span sampleData = sampleOwner.GetSpan()[..sample.Length]; + decoder.DecodeSequenceReference(sampleData, track.CicpProfile, codecConfiguration); + } + + /// + /// Reads and validates one bounded AV1 sequence sample into allocator-owned codec input storage. + /// + /// The complete seekable HEIF stream. + /// The track supplying the codec configuration and color description. + /// The validated sample range. + /// The allocator-owned buffer containing the validated coded sample. + private IMemoryOwner ReadSequenceSample( + BufferedReadStream stream, + HeifSequenceTrack track, + HeifSequenceSample sample) + { + Av1CodecConfiguration codecConfiguration = track.Av1CodecConfiguration + ?? throw new InvalidImageContentException("The AV1 image-sequence track has no codec configuration."); + + IMemoryOwner sampleOwner = this.configuration.MemoryAllocator.Allocate(sample.Length); + try + { + Span sampleData = sampleOwner.GetSpan()[..sample.Length]; + stream.Position = this.fileStartOffset + sample.Offset; + HeifBoxReader.ReadExactly(stream, sampleData, "The HEIF image-sequence sample is truncated."); + codecConfiguration.ValidateSampleData( + sampleData, + sample.IsSync, + track.ContentLightLevel, + track.MasteringDisplayColorVolume, + this.Options, + out _, + out _); + + return sampleOwner; + } + catch + { + sampleOwner.Dispose(); + throw; + } + } + + /// + /// Updates image-level metadata from the selected color and optional alpha sequence tracks. + /// + /// The image metadata receiving the sequence description. + /// The parsed selected image sequence. + /// Whether the primary image is also the first timed sample. + private void UpdateSequenceMetadata(ImageMetadata metadata, HeifSequence sequence, bool animateRootFrame) + { + HeifSequenceTrack colorTrack = sequence.ColorTrack; + HeifMetadata heifMetadata = metadata.GetHeifMetadata(); + heifMetadata.RepeatCount = colorTrack.RepeatCount; + heifMetadata.AnimateRootFrame = animateRootFrame; + heifMetadata.HasAlpha |= sequence.AlphaTrack is not null; + switch (colorTrack.CodecType) + { + case Heif4CharCode.Av01: + Av1CodecConfiguration av1Configuration = colorTrack.Av1CodecConfiguration + ?? throw new InvalidImageContentException("The AV1 image-sequence track has no codec configuration."); + + if (animateRootFrame) + { + heifMetadata.CompressionMethod = HeifCompressionMethod.Av1; + heifMetadata.BitDepth = av1Configuration.BitDepth; + heifMetadata.IsMonochrome = av1Configuration.IsMonochrome; + } + + break; + default: + throw new InvalidImageContentException($"The image-sequence sample entry '{colorTrack.CodecType}' is not supported."); + } + + if (this.Options.SkipMetadata || !animateRootFrame) + { + return; + } + + // The selected track is decoder-private and no longer mutates after parsing. Reuse its profiles so the + // returned metadata does not duplicate their storage. + metadata.IccProfile = colorTrack.IccProfile; + metadata.CicpProfile = colorTrack.CicpProfile; + heifMetadata.ContentLightLevel = colorTrack.ContentLightLevel; + heifMetadata.MasteringDisplayColorVolume = colorTrack.MasteringDisplayColorVolume; + heifMetadata.ContentColorVolume = colorTrack.ContentColorVolume; + heifMetadata.AmbientViewingEnvironment = colorTrack.AmbientViewingEnvironment; + heifMetadata.ReferenceViewingEnvironment = colorTrack.ReferenceViewingEnvironment; + heifMetadata.NominalDiffuseWhite = colorTrack.NominalDiffuseWhite; + ApplyPixelAspectRatioMetadata(metadata, colorTrack.PixelAspectRatio, colorTrack.RotationAngle); + + HeifSequenceMetadata? trackMetadata = colorTrack.Metadata; + if (trackMetadata?.ExifData is not null) + { + this.ExecuteAncillarySegmentAction(() => ApplyExifProfile(metadata, trackMetadata.ExifData)); + } + + if (trackMetadata?.XmpData is not null) + { + this.ExecuteAncillarySegmentAction(() => metadata.XmpProfile = new XmpProfile(trackMetadata.XmpData)); + } + } + + /// + /// Creates one HEIF frame-metadata entry for each visible retained sequence sample. + /// + /// The selected color track supplying sample durations. + /// Whether the first sequence sample occupies the root-frame slot. + /// The exact visible-frame metadata array in presentation order. + private static ImageFrameMetadata[] CreateSequenceFrameMetadata( + HeifSequenceTrack track, + bool animateRootFrame) + { + int visibleFrameCount = 0; + foreach (HeifSequenceSample sample in track.Samples) + { + visibleFrameCount += sample.IsHidden ? 0 : 1; + } + + int firstSequenceFrameIndex = animateRootFrame ? 0 : 1; + ImageFrameMetadata[] result = new ImageFrameMetadata[visibleFrameCount + firstSequenceFrameIndex]; + if (!animateRootFrame) + { + result[0] = new ImageFrameMetadata(); + } + + int frameIndex = firstSequenceFrameIndex; + foreach (HeifSequenceSample sample in track.Samples) + { + if (sample.IsHidden) + { + continue; + } + + ImageFrameMetadata frameMetadata = new(); + frameMetadata.GetHeifMetadata().FrameDelay = new Rational(sample.Duration, track.MediaTimescale); + result[frameIndex++] = frameMetadata; + } + + return result; + } + + /// + /// Gets the primary image item required alongside an AVIF image sequence. + /// + private HeifItem FindSequencePrimaryItem() + => this.FindItemById(this.primaryItem) + ?? throw new InvalidImageContentException("The HEIF image sequence contains no primary image item."); + + /// + /// Determines whether the primary image item reuses the first presented sequence sample. + /// + private static bool IsPrimaryItemFirstSequenceSample(HeifItem primaryItem, HeifSequenceTrack colorTrack) + { + if (primaryItem.DataLocations.Count != 1) + { + return false; + } + + HeifLocation location = primaryItem.DataLocations[0]; + foreach (HeifSequenceSample sample in colorTrack.Samples) + { + if (sample.IsHidden) + { + continue; + } + + return location.Origin == HeifLocationOffsetOrigin.FileOffset + && location.BaseOffset + location.Offset == sample.Offset + && location.Length == sample.Length; + } + + return false; + } + + /// + /// Updates identification metadata from the primary item or its decodable thumbnail fallback. + /// + /// The destination image metadata. + /// The primary item whose visible representation is being identified. + private void UpdateMetadata(ImageMetadata metadata, HeifItem item) + { + HeifItem presentationItem = item; + HeifItem metadataItem = item; + if (item.Type == Heif4CharCode.Grid) + { + // A grid is a derived image rather than a compression method. Its dimg references identify the coded + // tile items whose decoder determines the compression reported for the primary presentation. + HeifItem? gridTile = this.FindDecodableGridTile(item); + HeifItem? thumbnail = gridTile is null ? this.FindDecodableThumbnail(item) : null; + metadataItem = gridTile ?? thumbnail ?? item; + presentationItem = thumbnail ?? item; + if (gridTile is not null) + { + Av1CodecConfiguration? gridConfiguration = gridTile.Type == Heif4CharCode.Av01 + ? gridTile.Av1CodecConfiguration + ?? throw new InvalidImageContentException($"AV1 image grid tile {gridTile.Id} has no codec configuration property.") + : null; + + foreach (HeifItemLink link in this.itemLinks) + { + if (link.Type != Heif4CharCode.Dimg || link.SourceId != item.Id) + { + continue; + } + + foreach (uint tileId in link.DestinationIds) + { + HeifItem tile = this.FindRequiredItemById(tileId); + if (tile.Type != gridTile.Type) + { + throw new InvalidImageContentException("All HEIF image grid tiles must use the same coding format."); + } + + if (gridConfiguration is not null) + { + Av1CodecConfiguration tileConfiguration = tile.Av1CodecConfiguration + ?? throw new InvalidImageContentException($"AV1 image grid tile {tile.Id} has no codec configuration property."); + + // Identify never reads the derived-image descriptor or coded tile payloads, but it still + // validates the shared sample layout needed to describe the displayed grid accurately. + if (!gridConfiguration.HasMatchingImageConfiguration(tileConfiguration)) + { + throw new InvalidImageContentException("All AV1 image grid tiles must use matching codec configurations."); + } + } + } + } + } + } + else if (HeifCompressionFactory.GetDecoder(item.Type) is null) + { + HeifItem? thumbnail = this.FindDecodableThumbnail(item); + metadataItem = thumbnail ?? item; + presentationItem = thumbnail ?? item; + } + + HeifMetadata meta = metadata.GetHeifMetadata(); + HeifCompressionMethod compressionMethod; + if (metadataItem.Type == Heif4CharCode.Av01) + { + Av1CodecConfiguration codecConfiguration = metadataItem.Av1CodecConfiguration + ?? throw new InvalidImageContentException($"AV1 image item {metadataItem.Id} has no codec configuration property."); + + compressionMethod = HeifCompressionMethod.Av1; + meta.BitDepth = codecConfiguration.BitDepth; + meta.IsMonochrome = codecConfiguration.IsMonochrome; + } + else if (metadataItem.Type == Heif4CharCode.Jpeg) + { + compressionMethod = HeifCompressionMethod.LegacyJpeg; + } + else + { + throw new InvalidImageContentException($"Image item {metadataItem.Id} uses unsupported item type '{metadataItem.Type}'."); + } + + meta.CompressionMethod = compressionMethod; + meta.HasAlpha = this.FindAlphaItem(presentationItem) is not null + || (presentationItem.Type == Heif4CharCode.Grid && this.FindGridAlphaTiles(presentationItem) is not null); + + if (!this.Options.SkipMetadata) + { + this.ApplyItemColorMetadata(metadata, presentationItem); + this.ApplyItemHdrMetadata(metadata, presentationItem); + this.ApplyItemPixelAspectRatioMetadata(metadata, presentationItem); + } + } + + /// + /// Indexes and parses the recognized children of a metadata box. + /// + /// The stream positioned at the metadata full-box header. + /// The bounded metadata payload length. + private void ParseMetadata(BufferedReadStream stream, long boxLength) + { + if (boxLength < 4) + { + throw new InvalidImageContentException("The metadata box is missing its version and flags."); + } + + long endPosition = stream.Position + boxLength; + stream.Skip(4); + + // Physical child order is not a dependency order. Record bounded payload positions first, then parse item + // declarations before the locations, references, and properties that resolve those identifiers. + Dictionary boxes = []; + while (stream.Position < endPosition) + { + long length = HeifBoxReader.ReadHeader(stream, endPosition, this.boxHeaderScratch, out Heif4CharCode boxType); + if (MetadataParseOrder.Contains(boxType)) + { + // Association and location boxes can precede the item declarations they reference. + if (!boxes.TryAdd(boxType, (stream.Position, length))) + { + throw new InvalidImageContentException($"The metadata box contains duplicate '{boxType}' boxes."); + } + } + + HeifBoxReader.Skip(stream, length); + } + + foreach (Heif4CharCode boxType in MetadataParseOrder) + { + if (!boxes.TryGetValue(boxType, out (long Offset, long Length) box)) + { + continue; + } + + stream.Position = box.Offset; + switch (boxType) + { + case Heif4CharCode.Hdlr: + this.ParseHandler(stream, box.Length); + break; + case Heif4CharCode.Iinf: + this.ParseItemInfo(stream, box.Length); + break; + case Heif4CharCode.Pitm: + this.ParsePrimaryItem(stream, box.Length); + break; + case Heif4CharCode.Iref: + this.ParseItemReference(stream, box.Length); + break; + case Heif4CharCode.Iloc: + this.ParseItemLocation(stream, box.Length); + break; + case Heif4CharCode.Iprp: + this.ParseItemProperties(stream, box.Length); + break; + case Heif4CharCode.Idat: + if (box.Length == 0) + { + throw new InvalidImageContentException("The item data box is empty."); + } + + // iloc construction method one addresses bytes from the start of the idat payload, not its header. + this.itemDataOffset = box.Offset; + this.itemDataLength = box.Length; + break; + } + } + + stream.Position = endPosition; + } + + /// + /// Validates that the metadata handler describes picture items rather than a timed media track. + /// + /// The stream positioned at the handler full-box payload. + /// The bounded handler payload length. + private void ParseHandler(BufferedReadStream stream, long boxLength) + { + using IMemoryOwner boxMemory = this.boxReader.ReadPayload(stream, boxLength); + Span boxBuffer = boxMemory.GetSpan(); + + EnsureBufferRemaining(boxBuffer, 0, 12, "handler"); + + // The full-box header and pre_defined field precede the handler type. A picture + // handler keeps this bounded parser in the still-image metadata model. + int bytesRead = 8; + Heif4CharCode handlerType = (Heif4CharCode)BinaryPrimitives.ReadUInt32BigEndian(boxBuffer[bytesRead..]); + if (handlerType != Heif4CharCode.Pict) + { + throw new ImageFormatException("Not a picture file."); + } + } + + /// + /// Parses the item-information box and its item-information entries. + /// + /// The stream positioned at the item-information full-box payload. + /// The bounded item-information payload length. + private void ParseItemInfo(BufferedReadStream stream, long boxLength) + { + using IMemoryOwner boxMemory = this.boxReader.ReadPayload(stream, boxLength); + Span boxBuffer = boxMemory.GetSpan(); + EnsureBufferRemaining(boxBuffer, 0, 4, "item info"); + + int bytesRead = 0; + byte version = boxBuffer[bytesRead]; + if (version > 1) + { + throw new InvalidImageContentException($"The item info box has unsupported version {version}."); + } + + bytesRead += 4; + uint entryCount = ReadUInt16Or32(boxBuffer, version != 0, ref bytesRead); + + for (uint i = 0; i < entryCount; i++) + { + bytesRead += this.ParseItemInfoEntry(boxBuffer[bytesRead..]); + } + + if (bytesRead != boxBuffer.Length) + { + throw new InvalidImageContentException("The item info entry count does not consume the item info box."); + } + } + + /// + /// Parses one versioned item-information entry from a bounded item-information payload. + /// + /// The bytes beginning at the item-information-entry box header. + /// The complete item-information-entry box length. + private int ParseItemInfoEntry(Span buffer) + { + int headerLength = HeifBoxReader.ParseHeader(buffer, out long boxLength, out Heif4CharCode boxType); + if (boxType != Heif4CharCode.Infe) + { + throw new InvalidImageContentException($"The item info box contains unexpected child '{boxType}'."); + } + + int totalLength = checked(headerLength + (int)boxLength); + Span entryBuffer = buffer[..totalLength]; + int bytesRead = headerLength; + EnsureBufferRemaining(entryBuffer, bytesRead, 4, "item info entry"); + byte version = entryBuffer[bytesRead]; + if (version > 3) + { + throw new InvalidImageContentException($"The item info entry has unsupported version {version}."); + } + + bytesRead += 4; + HeifItem? item = null; + if (version is 0 or 1) + { + EnsureBufferRemaining(entryBuffer, bytesRead, 4, "item info entry"); + uint itemId = BinaryPrimitives.ReadUInt16BigEndian(entryBuffer[bytesRead..]); + bytesRead += 2; + item = new HeifItem(boxType, itemId); + + uint protectionIndex = BinaryPrimitives.ReadUInt16BigEndian(entryBuffer[bytesRead..]); + bytesRead += 2; + if (protectionIndex != 0) + { + throw new InvalidImageContentException($"Item {itemId} uses unsupported item protection."); + } + + item.Name = ReadNullTerminatedString(entryBuffer[bytesRead..], out int nameLength); + bytesRead += nameLength; + item.ContentType = ReadNullTerminatedString(entryBuffer[bytesRead..], out int contentTypeLength); + bytesRead += contentTypeLength; + + if (bytesRead < totalLength) + { + item.ContentEncoding = ReadNullTerminatedString(entryBuffer[bytesRead..], out int contentEncodingLength); + bytesRead += contentEncodingLength; + } + + if (version == 1) + { + if (bytesRead < totalLength) + { + EnsureBufferRemaining(entryBuffer, bytesRead, 4, "item info entry"); + item.ExtensionType = BinaryPrimitives.ReadUInt32BigEndian(entryBuffer[bytesRead..]); + bytesRead += 4; + } + + if (bytesRead < totalLength) + { + // Version-one extension payloads are outside the image item types currently + // consumed by this decoder, but remain bounded within this entry. + bytesRead = totalLength; + } + } + } + + if (version >= 2) + { + uint itemId = ReadUInt16Or32(entryBuffer, version == 3, ref bytesRead); + + EnsureBufferRemaining(entryBuffer, bytesRead, 6, "item info entry"); + + uint protectionIndex = BinaryPrimitives.ReadUInt16BigEndian(entryBuffer[bytesRead..]); + bytesRead += 2; + if (protectionIndex != 0) + { + throw new InvalidImageContentException($"Item {itemId} uses unsupported item protection."); + } + + Heif4CharCode itemType = (Heif4CharCode)BinaryPrimitives.ReadUInt32BigEndian(entryBuffer[bytesRead..]); + bytesRead += 4; + item = new HeifItem(itemType, itemId); + item.Name = ReadNullTerminatedString(entryBuffer[bytesRead..], out int nameLength); + bytesRead += nameLength; + if (item.Type == Heif4CharCode.Mime) + { + item.ContentType = ReadNullTerminatedString(entryBuffer[bytesRead..], out int contentTypeLength); + bytesRead += contentTypeLength; + + if (bytesRead < totalLength) + { + item.ContentEncoding = ReadNullTerminatedString(entryBuffer[bytesRead..], out int contentEncodingLength); + bytesRead += contentEncodingLength; + } + } + else if (item.Type == Heif4CharCode.Uri) + { + item.UriType = ReadNullTerminatedString(entryBuffer[bytesRead..], out int uriLength); + bytesRead += uriLength; + } + } + + if (item is not null) + { + if (this.FindItemById(item.Id) is not null) + { + throw new InvalidImageContentException($"The item info box contains duplicate item ID {item.Id}."); + } + + this.items.Add(item); + } + + if (bytesRead != totalLength) + { + throw new InvalidImageContentException("The item info entry contains unexpected trailing data."); + } + + return totalLength; + } + + /// + /// Parses typed relationships between source and destination items. + /// + /// The stream positioned at the item-reference full-box payload. + /// The bounded item-reference payload length. + private void ParseItemReference(BufferedReadStream stream, long boxLength) + { + using IMemoryOwner boxMemory = this.boxReader.ReadPayload(stream, boxLength); + Span boxBuffer = boxMemory.GetSpan(); + EnsureBufferRemaining(boxBuffer, 0, 4, "item reference"); + + int bytesRead = 0; + byte version = boxBuffer[bytesRead]; + if (version > 1) + { + throw new InvalidImageContentException($"The item reference box has unsupported version {version}."); + } + + bool largeIds = version == 1; + bytesRead += 4; + while (bytesRead < boxLength) + { + int referenceHeaderLength = HeifBoxReader.ParseHeader(boxBuffer[bytesRead..], out long referenceLength, out Heif4CharCode linkType); + int referenceEnd = checked(bytesRead + referenceHeaderLength + (int)referenceLength); + Span referenceBuffer = boxBuffer[..referenceEnd]; + bytesRead += referenceHeaderLength; + + if (linkType is not Heif4CharCode.Dimg + and not Heif4CharCode.Auxl + and not Heif4CharCode.Prem + and not Heif4CharCode.Thmb + and not Heif4CharCode.Cdsc) + { + // Unknown reference types do not participate in the bounded image model. Their child-box boundary + // was validated above, so skip the payload without imposing semantics from a general ISOBMFF reader. + bytesRead = referenceEnd; + continue; + } + + if (this.Options.SkipMetadata && linkType == Heif4CharCode.Cdsc) + { + // Descriptive metadata links have no effect when their payloads are not requested. Avoid validating + // their optional item graph while preserving the surrounding image relationships. + bytesRead = referenceEnd; + continue; + } + + try + { + uint sourceId = ReadUInt16Or32(referenceBuffer, largeIds, ref bytesRead); + if (this.FindItemById(sourceId) is null) + { + throw new InvalidImageContentException($"The item reference box references unknown source item ID {sourceId}."); + } + + HeifItemLink link = new(linkType, sourceId); + + EnsureBufferRemaining(referenceBuffer, bytesRead, 2, "item reference"); + int count = BinaryPrimitives.ReadUInt16BigEndian(referenceBuffer[bytesRead..]); + bytesRead += 2; + for (uint i = 0; i < count; i++) + { + uint destId = ReadUInt16Or32(referenceBuffer, largeIds, ref bytesRead); + if (this.FindItemById(destId) is null) + { + throw new InvalidImageContentException($"The item reference box references unknown destination item ID {destId}."); + } + + link.DestinationIds.Add(destId); + } + + if (bytesRead != referenceEnd) + { + throw new InvalidImageContentException($"The '{linkType}' item reference length does not match its entry count."); + } + + this.itemLinks.Add(link); + } + catch (Exception ex) when (linkType == Heif4CharCode.Cdsc && ImageDecoderCore.ShouldIgnoreAncillarySegmentError(this.Options, ex)) + { + // A malformed descriptive link cannot change reconstructed pixels, so non-strict modes omit it. + bytesRead = referenceEnd; + } + catch (Exception ex) when (linkType != Heif4CharCode.Cdsc && ImageDecoderCore.ShouldIgnoreImageDataSegmentError(this.Options, ex)) + { + // IgnoreImageData permits a malformed optional image relationship to be omitted while retaining + // independently reconstructable items and thumbnail fallbacks. + bytesRead = referenceEnd; + } + } + } + + /// + /// Reads the identifier of the presentation's primary item. + /// + /// The stream positioned at the primary-item full-box payload. + /// The bounded primary-item payload length. + private void ParsePrimaryItem(BufferedReadStream stream, long boxLength) + { + using IMemoryOwner boxMemory = this.boxReader.ReadPayload(stream, boxLength); + Span boxBuffer = boxMemory.GetSpan(); + EnsureBufferRemaining(boxBuffer, 0, 4, "primary item"); + + byte version = boxBuffer[0]; + if (version > 1) + { + throw new InvalidImageContentException($"The primary item box has unsupported version {version}."); + } + + int bytesRead = 4; + this.primaryItem = ReadUInt16Or32(boxBuffer, version == 1, ref bytesRead); + if (bytesRead != boxBuffer.Length) + { + throw new InvalidImageContentException("The primary item box has an invalid length."); + } + } + + /// + /// Parses the ordered item-property table and applies its item associations. + /// + /// The stream positioned at the item-properties payload. + /// The bounded item-properties payload length. + private void ParseItemProperties(BufferedReadStream stream, long boxLength) + { + // Property types may repeat, and ipma can physically precede ipco. Index the bounded + // children first so associations are always resolved after the ordered property table. + List> properties = new(); + long endBoxPosition = stream.Position + boxLength; + (long Offset, long Length)? propertyContainer = null; + List<(long Offset, long Length)> associations = []; + while (stream.Position < endBoxPosition) + { + long containerLength = HeifBoxReader.ReadHeader(stream, endBoxPosition, this.boxHeaderScratch, out Heif4CharCode containerType); + if (containerType == Heif4CharCode.Ipco) + { + if (propertyContainer.HasValue) + { + throw new InvalidImageContentException("The item properties box contains duplicate property containers."); + } + + propertyContainer = (stream.Position, containerLength); + } + else if (containerType == Heif4CharCode.Ipma) + { + associations.Add((stream.Position, containerLength)); + } + + // Unknown optional children remain bounded by iprp and do not expand the still-image model. + HeifBoxReader.Skip(stream, containerLength); + } + + if (!propertyContainer.HasValue) + { + throw new InvalidImageContentException("The item properties box does not contain a property container."); + } + + stream.Position = propertyContainer.Value.Offset; + this.ParsePropertyContainer(stream, propertyContainer.Value.Length, properties); + foreach ((long Offset, long Length) association in associations) + { + stream.Position = association.Offset; + this.ParsePropertyAssociation(stream, association.Length, properties); + } + + stream.Position = endBoxPosition; + } + + /// + /// Parses the ordered property boxes contained by an item-property container. + /// + /// The stream positioned at the first property box. + /// The bounded item-property-container payload length. + /// The one-based association table in physical property order. + private void ParsePropertyContainer(BufferedReadStream stream, long boxLength, List> properties) + { + long endPosition = stream.Position + boxLength; + while (stream.Position < endPosition) + { + long itemLength = HeifBoxReader.ReadHeader(stream, endPosition, this.boxHeaderScratch, out Heif4CharCode itemType); + if (this.Options.SkipMetadata && itemType is Heif4CharCode.Pasp + or Heif4CharCode.Clli + or Heif4CharCode.Mdcv + or Heif4CharCode.Cclv + or Heif4CharCode.Amve + or Heif4CharCode.Reve + or Heif4CharCode.Ndwt) + { + // These properties affect only exposed image metadata. Preserve their physical ipco positions while + // avoiding payload allocation and validation when the caller requested no metadata. + HeifBoxReader.Skip(stream, itemLength); + properties.Add(new KeyValuePair(itemType, IgnoredProperty)); + continue; + } + + if (this.Options.SkipMetadata && itemType == Heif4CharCode.Colr && itemLength >= 4) + { + Span profileTypeBuffer = this.boxHeaderScratch.AsSpan(0, 4); + HeifBoxReader.ReadExactly(stream, profileTypeBuffer, "The HEIF color-information property is truncated."); + Heif4CharCode profileType = (Heif4CharCode)BinaryPrimitives.ReadUInt32BigEndian(profileTypeBuffer); + if (profileType is Heif4CharCode.RICC or Heif4CharCode.Prof) + { + // ICC bytes cannot affect reconstruction when metadata is skipped. Retain only the property index + // and leave the potentially large profile payload out of the allocator entirely. + HeifBoxReader.Skip(stream, itemLength - 4); + properties.Add(new KeyValuePair(Heif4CharCode.Colr, IgnoredProperty)); + continue; + } + + stream.Position -= 4; + } + + if (!this.Options.SkipMetadata && itemType == Heif4CharCode.Colr && itemLength is >= 4 and <= int.MaxValue) + { + Span profileTypeBuffer = this.boxHeaderScratch.AsSpan(0, 4); + HeifBoxReader.ReadExactly(stream, profileTypeBuffer, "The HEIF color-information property is truncated."); + Heif4CharCode profileType = (Heif4CharCode)BinaryPrimitives.ReadUInt32BigEndian(profileTypeBuffer); + + if (profileType is Heif4CharCode.RICC or Heif4CharCode.Prof) + { + // Read directly into the array retained by IccProfile so the generic box buffer cannot create a + // second full-sized copy of the profile at this ownership boundary. + byte[] profileData = new byte[(int)itemLength - 4]; + HeifBoxReader.ReadExactly(stream, profileData, "Stream length is not sufficient for box content."); + IccProfile? iccProfile = null; + + try + { + iccProfile = HeifPropertyParser.ParseIccProfile(profileData); + } + catch (Exception ex) when (ImageDecoderCore.ShouldIgnoreAncillarySegmentError(this.Options, ex)) + { + // Keep the understood property index without retaining invalid ancillary metadata. + } + + // A malformed ancillary profile can be ignored by policy while the physical property still + // occupies its ipco index and remains understood for essential-association handling. + properties.Add(new KeyValuePair( + Heif4CharCode.Colr, + iccProfile ?? IgnoredProperty)); + + continue; + } + + stream.Position -= 4; + } + + using IMemoryOwner boxMemory = this.boxReader.ReadPayload(stream, itemLength); + Span boxBuffer = boxMemory.GetSpan(); + try + { + switch (itemType) + { + case Heif4CharCode.Ispe: + EnsureBufferRemaining(boxBuffer, 0, 12, "image spatial extents"); + + // The full-box header precedes the unsigned display width and height. + uint width = BinaryPrimitives.ReadUInt32BigEndian(boxBuffer[4..]); + uint height = BinaryPrimitives.ReadUInt32BigEndian(boxBuffer[8..]); + if (width is 0 or > int.MaxValue || height is 0 or > int.MaxValue) + { + throw new InvalidImageContentException("The image spatial extents property has invalid dimensions."); + } + + properties.Add(new KeyValuePair(Heif4CharCode.Ispe, new Size((int)width, (int)height))); + break; + case Heif4CharCode.Pasp: + object pixelAspectRatio = IgnoredProperty; + try + { + pixelAspectRatio = HeifPropertyParser.ParsePixelAspectRatio(boxBuffer); + } + catch (Exception ex) when (ImageDecoderCore.ShouldIgnoreAncillarySegmentError(this.Options, ex)) + { + // Keep the understood property index without retaining invalid ancillary metadata. + } + + properties.Add(new KeyValuePair(Heif4CharCode.Pasp, pixelAspectRatio)); + break; + case Heif4CharCode.Pixi: + EnsureBufferRemaining(boxBuffer, 0, 5, "pixel information"); + if (boxBuffer[0] != 0 || boxBuffer[1] != 0 || boxBuffer[2] != 0 || boxBuffer[3] != 0) + { + throw new InvalidImageContentException("The pixel information property has an unsupported version or flags."); + } + + // The full-box header precedes one bit-depth byte for each channel. + int channelCount = boxBuffer[4]; + if (channelCount == 0) + { + throw new InvalidImageContentException("The pixel information property has no channels."); + } + + int offset = 5; + EnsureBufferRemaining(boxBuffer, offset, channelCount, "pixel information"); + if (boxBuffer.Length != offset + channelCount) + { + throw new InvalidImageContentException("The pixel information property contains unexpected trailing data."); + } + + // Property associations are resolved after the pooled box buffer is reused, so retain the + // exact channel vector once at this ownership boundary. + byte[] channelBitDepths = new byte[channelCount]; + boxBuffer.Slice(offset, channelCount).CopyTo(channelBitDepths); + for (int i = 0; i < channelBitDepths.Length; i++) + { + if (channelBitDepths[i] == 0) + { + throw new InvalidImageContentException($"The pixel information property declares zero precision for channel {i}."); + } + } + + properties.Add(new KeyValuePair(Heif4CharCode.Pixi, channelBitDepths)); + + break; + case Heif4CharCode.AuxC: + EnsureBufferRemaining(boxBuffer, 0, 5, "auxiliary type"); + if (boxBuffer[0] != 0) + { + throw new InvalidImageContentException($"The auxiliary type property has unsupported version {boxBuffer[0]}."); + } + + // aux_type is a required null-terminated string. Any remaining bytes are the registered + // auxiliary subtype payload, which is not needed to identify an alpha image plane. + string auxiliaryType = ReadNullTerminatedString(boxBuffer[4..], out _); + properties.Add(new KeyValuePair(Heif4CharCode.AuxC, auxiliaryType)); + break; + case Heif4CharCode.Colr: + EnsureBufferRemaining(boxBuffer, 0, 4, "color information"); + Heif4CharCode profileType = (Heif4CharCode)BinaryPrimitives.ReadUInt32BigEndian(boxBuffer); + object colorInformation = UnknownProperty; + if (profileType == Heif4CharCode.Nclx) + { + colorInformation = HeifPropertyParser.ParseCicpProfile(boxBuffer[4..]); + } + + properties.Add(new KeyValuePair(Heif4CharCode.Colr, colorInformation)); + + break; + case Heif4CharCode.Clli: + object contentLightLevel = IgnoredProperty; + try + { + contentLightLevel = HeifPropertyParser.ParseContentLightLevel(boxBuffer); + } + catch (Exception ex) when (ImageDecoderCore.ShouldIgnoreAncillarySegmentError(this.Options, ex)) + { + // Keep the understood property index without retaining invalid ancillary metadata. + } + + properties.Add(new KeyValuePair(Heif4CharCode.Clli, contentLightLevel)); + break; + case Heif4CharCode.Mdcv: + object masteringDisplayColorVolume = IgnoredProperty; + try + { + masteringDisplayColorVolume = HeifPropertyParser.ParseMasteringDisplayColorVolume(boxBuffer); + } + catch (Exception ex) when (ImageDecoderCore.ShouldIgnoreAncillarySegmentError(this.Options, ex)) + { + // Keep the understood property index without retaining invalid ancillary metadata. + } + + properties.Add(new KeyValuePair(Heif4CharCode.Mdcv, masteringDisplayColorVolume)); + break; + case Heif4CharCode.Cclv: + object contentColorVolume = IgnoredProperty; + try + { + contentColorVolume = HeifPropertyParser.ParseContentColorVolume(boxBuffer); + } + catch (Exception ex) when (ImageDecoderCore.ShouldIgnoreAncillarySegmentError(this.Options, ex)) + { + // Keep the understood property index without retaining invalid ancillary metadata. + } + + properties.Add(new KeyValuePair(Heif4CharCode.Cclv, contentColorVolume)); + break; + case Heif4CharCode.Amve: + object ambientViewingEnvironment = IgnoredProperty; + try + { + ambientViewingEnvironment = HeifPropertyParser.ParseAmbientViewingEnvironment(boxBuffer); + } + catch (Exception ex) when (ImageDecoderCore.ShouldIgnoreAncillarySegmentError(this.Options, ex)) + { + // Keep the understood property index without retaining invalid ancillary metadata. + } + + properties.Add(new KeyValuePair(Heif4CharCode.Amve, ambientViewingEnvironment)); + break; + case Heif4CharCode.Reve: + object referenceViewingEnvironment = IgnoredProperty; + try + { + referenceViewingEnvironment = HeifPropertyParser.ParseReferenceViewingEnvironment(boxBuffer); + } + catch (Exception ex) when (ImageDecoderCore.ShouldIgnoreAncillarySegmentError(this.Options, ex)) + { + // Keep the understood property index without retaining invalid ancillary metadata. + } + + properties.Add(new KeyValuePair(Heif4CharCode.Reve, referenceViewingEnvironment)); + break; + case Heif4CharCode.Ndwt: + object nominalDiffuseWhite = IgnoredProperty; + try + { + nominalDiffuseWhite = HeifPropertyParser.ParseNominalDiffuseWhite(boxBuffer); + } + catch (Exception ex) when (ImageDecoderCore.ShouldIgnoreAncillarySegmentError(this.Options, ex)) + { + // Keep the understood property index without retaining invalid ancillary metadata. + } + + properties.Add(new KeyValuePair(Heif4CharCode.Ndwt, nominalDiffuseWhite)); + break; + case Heif4CharCode.Av1C: + EnsureBufferRemaining(boxBuffer, 0, 4, "AV1 codec configuration"); + properties.Add( + new KeyValuePair( + Heif4CharCode.Av1C, + new Av1CodecConfiguration(boxBuffer, this.Options))); + + break; + case Heif4CharCode.A1op: + properties.Add( + new KeyValuePair( + Heif4CharCode.A1op, + HeifPropertyParser.ParseAv1OperatingPointSelector(boxBuffer))); + + break; + case Heif4CharCode.Lsel: + properties.Add( + new KeyValuePair( + Heif4CharCode.Lsel, + HeifPropertyParser.ParseAv1LayerSelector(boxBuffer))); + + break; + case Heif4CharCode.A1lx: + properties.Add( + new KeyValuePair( + Heif4CharCode.A1lx, + HeifPropertyParser.ParseAv1LayeredImageIndex(boxBuffer))); + + break; + case Heif4CharCode.Clap: + properties.Add( + new KeyValuePair( + Heif4CharCode.Clap, + HeifPropertyParser.ParseCleanAperture(boxBuffer))); + + break; + case Heif4CharCode.Irot: + properties.Add(new KeyValuePair(Heif4CharCode.Irot, HeifPropertyParser.ParseRotation(boxBuffer))); + break; + case Heif4CharCode.Imir: + properties.Add(new KeyValuePair(Heif4CharCode.Imir, HeifPropertyParser.ParseMirrorAxis(boxBuffer))); + break; + case Heif4CharCode.Altt: + case Heif4CharCode.Iscl: + case Heif4CharCode.Rloc: + case Heif4CharCode.Udes: + // These registered image properties are not arbitrary unknown boxes. Preserve their indices so + // container identification remains available while their owning image stage handles the value. + properties.Add(new KeyValuePair(itemType, IgnoredProperty)); + break; + default: + // Unknown properties still occupy an ipco index and become an error only when marked essential. + properties.Add(new KeyValuePair(itemType, UnknownProperty)); + break; + } + } + catch (Exception ex) when (ImageDecoderCore.ShouldIgnoreImageDataSegmentError(this.Options, ex)) + { + // Invalid image properties retain their physical association index. Typed association handling ignores + // the placeholder so another decodable item or the coded-image defaults can remain usable. + properties.Add(new KeyValuePair(itemType, IgnoredProperty)); + } + } + } + + /// + /// Applies one-based property indices and essential flags to their referenced items. + /// + /// The stream positioned at the property-association full-box payload. + /// The bounded property-association payload length. + /// The properties in the order used by association indices. + private void ParsePropertyAssociation(BufferedReadStream stream, long boxLength, List> properties) + { + using IMemoryOwner boxMemory = this.boxReader.ReadPayload(stream, boxLength); + Span boxBuffer = boxMemory.GetSpan(); + EnsureBufferRemaining(boxBuffer, 0, 8, "item property association"); + byte version = boxBuffer[0]; + if (version > 1) + { + throw new InvalidImageContentException($"The item property association box has unsupported version {version}."); + } + + bool largePropertyIndex = (boxBuffer[3] & 1) != 0; + int bytesRead = 4; + uint entryCount = BinaryPrimitives.ReadUInt32BigEndian(boxBuffer[bytesRead..]); + bytesRead += 4; + for (uint entryIndex = 0; entryIndex < entryCount; entryIndex++) + { + uint itemId = ReadUInt16Or32(boxBuffer, version == 1, ref bytesRead); + HeifItem? item = this.FindItemById(itemId); + if (item is null) + { + throw new InvalidImageContentException($"Item property association references unknown item ID {itemId}."); + } + + EnsureBufferRemaining(boxBuffer, bytesRead, 1, "item property association"); + int associationCount = boxBuffer[bytesRead++]; + for (int i = 0; i < associationCount; i++) + { + uint association; + uint propertyIndexMask; + uint essentialMask; + if (largePropertyIndex) + { + EnsureBufferRemaining(boxBuffer, bytesRead, 2, "item property association"); + association = BinaryPrimitives.ReadUInt16BigEndian(boxBuffer[bytesRead..]); + bytesRead += 2; + propertyIndexMask = 0x7FFFU; + essentialMask = 0x8000U; + } + else + { + EnsureBufferRemaining(boxBuffer, bytesRead, 1, "item property association"); + association = boxBuffer[bytesRead++]; + propertyIndexMask = 0x7FU; + essentialMask = 0x80U; + } + + uint propertyIndex = association & propertyIndexMask; + bool essential = (association & essentialMask) != 0; + if (propertyIndex == 0) + { + if (essential) + { + throw new InvalidImageContentException($"Item {itemId} associates essential property index 0."); + } + + continue; + } + + propertyIndex--; + if (propertyIndex >= properties.Count) + { + throw new InvalidImageContentException( + $"Item {itemId} references property index {propertyIndex + 1}, but only {properties.Count} properties exist."); + } + + KeyValuePair prop = properties[(int)propertyIndex]; + if (essential && ReferenceEquals(prop.Value, UnknownProperty)) + { + throw new InvalidImageContentException($"Item {itemId} associates unknown essential property '{prop.Key}'."); + } + + if (!essential && prop.Key is Heif4CharCode.Clap or Heif4CharCode.Irot or Heif4CharCode.Imir) + { + this.ThrowOrIgnoreImageDataSegmentError( + $"Item {itemId} associates nonessential transformative property '{prop.Key}'."); + + continue; + } + + // AVIF 1.1 section 2.3.2.1.1 requires a1op to be essential, while HEIF section 6.5.11.1 + // imposes the same requirement on lsel because ignoring either selector changes the decoded image. + if (!essential && prop.Key is Heif4CharCode.A1op or Heif4CharCode.Lsel) + { + this.ThrowOrIgnoreImageDataSegmentError( + $"Item {itemId} associates AV1 selector property '{prop.Key}' without marking it essential."); + + continue; + } + + // AVIF 1.1 section 2.3.2.3.2 requires a1lx to be nonessential; decoders may consume the complete + // item payload without using its optional layer-boundary optimization. + if (essential && prop.Key == Heif4CharCode.A1lx) + { + this.ThrowOrIgnoreImageDataSegmentError( + $"Item {itemId} marks AV1 layered-image indexing property '{prop.Key}' as essential."); + + continue; + } + + switch (prop.Key) + { + case Heif4CharCode.Ispe: + if (prop.Value is Size extent) + { + item.SetExtent(extent); + } + + break; + case Heif4CharCode.Pasp: + if (prop.Value is HeifPixelAspectRatio pixelAspectRatio) + { + if (item.PixelAspectRatio is not null) + { + this.ThrowOrIgnoreNonStrictSegmentError( + $"Item {itemId} associates more than one pixel aspect ratio property."); + + break; + } + + item.PixelAspectRatio = pixelAspectRatio; + } + + break; + case Heif4CharCode.Pixi: + if (prop.Value is byte[] channelBitDepths) + { + if (item.ChannelBitDepths is not null) + { + this.ThrowOrIgnoreImageDataSegmentError( + $"Item {itemId} associates more than one pixel information property."); + + break; + } + + int bitsPerPixel = 0; + for (int channel = 0; channel < channelBitDepths.Length; channel++) + { + bitsPerPixel += channelBitDepths[channel]; + } + + item.ChannelCount = channelBitDepths.Length; + item.ChannelBitDepths = channelBitDepths; + item.BitsPerPixel = bitsPerPixel; + } + + break; + case Heif4CharCode.Av1C: + if (prop.Value is Av1CodecConfiguration av1CodecConfiguration) + { + if (item.Type != Heif4CharCode.Av01) + { + this.ThrowOrIgnoreImageDataSegmentError( + $"Item {itemId} associates an AV1 codec configuration with non-AV1 item type '{item.Type}'."); + + break; + } + + if (item.Av1CodecConfiguration is not null) + { + this.ThrowOrIgnoreImageDataSegmentError( + $"Item {itemId} associates more than one AV1 codec configuration property."); + + break; + } + + item.Av1CodecConfiguration = av1CodecConfiguration; + } + + break; + case Heif4CharCode.A1op: + if (prop.Value is Av1OperatingPointSelector operatingPointSelector) + { + if (item.Type != Heif4CharCode.Av01) + { + this.ThrowOrIgnoreImageDataSegmentError( + $"Item {itemId} associates an AV1 operating-point selector with non-AV1 item type '{item.Type}'."); + + break; + } + + if (item.Av1OperatingPointSelector is not null) + { + this.ThrowOrIgnoreImageDataSegmentError( + $"Item {itemId} associates more than one AV1 operating-point selector property."); + + break; + } + + item.Av1OperatingPointSelector = operatingPointSelector; + } + + break; + case Heif4CharCode.Lsel: + if (prop.Value is Av1LayerSelector layerSelector) + { + if (item.Type != Heif4CharCode.Av01) + { + this.ThrowOrIgnoreImageDataSegmentError( + $"Item {itemId} associates an AV1 layer selector with non-AV1 item type '{item.Type}'."); + + break; + } + + if (item.Av1LayerSelector is not null) + { + this.ThrowOrIgnoreImageDataSegmentError( + $"Item {itemId} associates more than one AV1 layer selector property."); + + break; + } + + item.Av1LayerSelector = layerSelector; + } + + break; + case Heif4CharCode.A1lx: + if (prop.Value is Av1LayeredImageIndex layeredImageIndex) + { + if (item.Type != Heif4CharCode.Av01) + { + this.ThrowOrIgnoreImageDataSegmentError( + $"Item {itemId} associates AV1 layered-image indexing with non-AV1 item type '{item.Type}'."); + + break; + } + + if (item.Av1LayeredImageIndex is not null) + { + this.ThrowOrIgnoreImageDataSegmentError( + $"Item {itemId} associates more than one AV1 layered-image indexing property."); + + break; + } + + item.Av1LayeredImageIndex = layeredImageIndex; + } + + break; + case Heif4CharCode.AuxC: + if (prop.Value is string auxiliaryType) + { + if (item.AuxiliaryType is not null) + { + this.ThrowOrIgnoreImageDataSegmentError( + $"Item {itemId} associates more than one auxiliary type property."); + + break; + } + + item.AuxiliaryType = auxiliaryType; + } + + break; + case Heif4CharCode.Colr: + if (prop.Value is IccProfile iccProfile) + { + if (item.IccProfile is not null) + { + this.ThrowOrIgnoreNonStrictSegmentError( + $"Item {itemId} associates more than one ICC color property."); + + break; + } + + item.IccProfile = iccProfile; + } + else if (prop.Value is CicpProfile cicpProfile) + { + if (item.CicpProfile is not null) + { + this.ThrowOrIgnoreImageDataSegmentError( + $"Item {itemId} associates more than one CICP color property."); + + break; + } + + item.CicpProfile = cicpProfile; + } + + break; + case Heif4CharCode.Clli: + if (prop.Value is HeifContentLightLevel contentLightLevel) + { + if (item.ContentLightLevel is not null) + { + this.ThrowOrIgnoreNonStrictSegmentError( + $"Item {itemId} associates more than one content light level property."); + + break; + } + + item.ContentLightLevel = contentLightLevel; + } + + break; + case Heif4CharCode.Mdcv: + if (prop.Value is HeifMasteringDisplayColorVolume masteringDisplayColorVolume) + { + if (item.MasteringDisplayColorVolume is not null) + { + this.ThrowOrIgnoreNonStrictSegmentError( + $"Item {itemId} associates more than one mastering display color-volume property."); + + break; + } + + item.MasteringDisplayColorVolume = masteringDisplayColorVolume; + } + + break; + case Heif4CharCode.Cclv: + if (prop.Value is HeifContentColorVolume contentColorVolume) + { + if (item.ContentColorVolume is not null) + { + this.ThrowOrIgnoreNonStrictSegmentError( + $"Item {itemId} associates more than one content color-volume property."); + + break; + } + + item.ContentColorVolume = contentColorVolume; + } + + break; + case Heif4CharCode.Amve: + if (prop.Value is HeifAmbientViewingEnvironment ambientViewingEnvironment) + { + if (item.AmbientViewingEnvironment is not null) + { + this.ThrowOrIgnoreNonStrictSegmentError( + $"Item {itemId} associates more than one ambient viewing-environment property."); + + break; + } + + item.AmbientViewingEnvironment = ambientViewingEnvironment; + } + + break; + case Heif4CharCode.Reve: + if (prop.Value is HeifReferenceViewingEnvironment referenceViewingEnvironment) + { + if (item.ReferenceViewingEnvironment is not null) + { + this.ThrowOrIgnoreNonStrictSegmentError( + $"Item {itemId} associates more than one reference viewing-environment property."); + + break; + } + + item.ReferenceViewingEnvironment = referenceViewingEnvironment; + } + + break; + case Heif4CharCode.Ndwt: + if (prop.Value is HeifNominalDiffuseWhite nominalDiffuseWhite) + { + if (item.NominalDiffuseWhite is not null) + { + this.ThrowOrIgnoreNonStrictSegmentError( + $"Item {itemId} associates more than one nominal diffuse-white property."); + + break; + } + + item.NominalDiffuseWhite = nominalDiffuseWhite; + } + + break; + case Heif4CharCode.Clap: + if (prop.Value is HeifCleanAperture cleanAperture) + { + if (item.CleanAperture is not null) + { + this.ThrowOrIgnoreImageDataSegmentError( + $"Item {itemId} associates more than one clean aperture property."); + + break; + } + + item.CleanAperture = cleanAperture; + } + + break; + case Heif4CharCode.Irot: + if (prop.Value is byte rotationAngle) + { + if (item.RotationAngle is not null) + { + this.ThrowOrIgnoreImageDataSegmentError( + $"Item {itemId} associates more than one image rotation property."); + + break; + } + + item.RotationAngle = rotationAngle; + } + + break; + case Heif4CharCode.Imir: + if (prop.Value is byte mirrorAxis) + { + if (item.MirrorAxis is not null) + { + this.ThrowOrIgnoreImageDataSegmentError( + $"Item {itemId} associates more than one image mirror property."); + + break; + } + + item.MirrorAxis = mirrorAxis; + } + + break; + } + } + } + + if (bytesRead != boxBuffer.Length) + { + throw new InvalidImageContentException("The item property association box contains unexpected trailing data."); + } + } + + /// + /// Parses the construction method, base offset, and ordered extents for every declared item. + /// + /// The stream positioned at the item-location full-box payload. + /// The bounded item-location payload length. + private void ParseItemLocation(BufferedReadStream stream, long boxLength) + { + using IMemoryOwner boxMemory = this.boxReader.ReadPayload(stream, boxLength); + Span boxBuffer = boxMemory.GetSpan(); + int bytesRead = 0; + EnsureBufferRemaining(boxBuffer, bytesRead, 6, "item location"); + byte version = boxBuffer[bytesRead]; + if (version > 2) + { + throw new InvalidImageContentException($"The item location box has unsupported version {version}."); + } + + bytesRead += 4; + + // The first two payload bytes pack four-bit integer widths for extent offset, extent length, base offset, + // and, for versions one and two, extent index. A zero width represents an implicit zero value. + byte b1 = boxBuffer[bytesRead]; + bytesRead++; + byte b2 = boxBuffer[bytesRead]; + bytesRead++; + int offsetSize = (b1 >> 4) & 0x0f; + int lengthSize = b1 & 0x0f; + int baseOffsetSize = (b2 >> 4) & 0x0f; + int indexSize = 0; + if (version is 1 or 2) + { + indexSize = b2 & 0x0f; + } + + if (!IsSupportedFieldSize(offsetSize) + || !IsSupportedFieldSize(lengthSize) + || !IsSupportedFieldSize(baseOffsetSize) + || !IsSupportedFieldSize(indexSize)) + { + throw new InvalidImageContentException("The item location box uses an invalid integer field size."); + } + + EnsureBufferRemaining(boxBuffer, bytesRead, version == 2 ? 4 : 2, "item location"); + uint itemCount = ReadUInt16Or32(boxBuffer, version == 2, ref bytesRead); + HashSet locatedItemIds = []; + for (uint i = 0; i < itemCount; i++) + { + EnsureBufferRemaining(boxBuffer, bytesRead, version == 2 ? 4 : 2, "item location"); + uint itemId = ReadUInt16Or32(boxBuffer, version == 2, ref bytesRead); + HeifItem? item = this.FindItemById(itemId); + if (item is null) + { + throw new InvalidImageContentException($"The item location box references unknown item ID {itemId}."); + } + + if (!locatedItemIds.Add(itemId)) + { + throw new InvalidImageContentException($"The item location box contains duplicate locations for item ID {itemId}."); + } + + HeifLocationOffsetOrigin constructionMethod = HeifLocationOffsetOrigin.FileOffset; + if (version is 1 or 2) + { + EnsureBufferRemaining(boxBuffer, bytesRead, 2, "item location"); + ushort constructionField = BinaryPrimitives.ReadUInt16BigEndian(boxBuffer[bytesRead..]); + bytesRead += 2; + if ((constructionField & 0xFFF0) != 0) + { + throw new InvalidImageContentException("The item location box has nonzero reserved construction bits."); + } + + constructionMethod = (HeifLocationOffsetOrigin)(constructionField & 0x0F); + if (constructionMethod is not HeifLocationOffsetOrigin.FileOffset and not HeifLocationOffsetOrigin.ItemDataOffset) + { + throw new InvalidImageContentException($"The item location box uses unsupported construction method {(int)constructionMethod}."); + } + } + + EnsureBufferRemaining(boxBuffer, bytesRead, 2, "item location"); + uint dataReferenceIndex = BinaryPrimitives.ReadUInt16BigEndian(boxBuffer[bytesRead..]); + bytesRead += 2; + if (dataReferenceIndex != 0) + { + throw new InvalidImageContentException("External item data references are not supported."); + } + + long baseOffset = ReadUIntVariable(boxBuffer, baseOffsetSize, ref bytesRead); + EnsureBufferRemaining(boxBuffer, bytesRead, 2, "item location"); + uint extentCount = BinaryPrimitives.ReadUInt16BigEndian(boxBuffer[bytesRead..]); + bytesRead += 2; + for (uint j = 0; j < extentCount; j++) + { + if (version is 1 or 2 && indexSize > 0) + { + // Extent indices select referenced-item extents only for construction method two. Methods zero + // and one still carry the field when configured, so consume it to preserve the following offsets. + ReadUIntVariable(boxBuffer, indexSize, ref bytesRead); + } + + long extentOffset = ReadUIntVariable(boxBuffer, offsetSize, ref bytesRead); + long extentLength = ReadUIntVariable(boxBuffer, lengthSize, ref bytesRead); + HeifLocation loc = new(constructionMethod, baseOffset, extentOffset, extentLength); + item.DataLocations.Add(loc); + } + } + + if (bytesRead != boxBuffer.Length) + { + throw new InvalidImageContentException("The item location box contains unexpected trailing data."); + } + } + + /// + /// Determines whether an item-location integer width can be represented by the supported reader primitives. + /// + /// The width in bytes from an item-location size nibble. + /// for the registered zero, 32-bit, and 64-bit widths. + private static bool IsSupportedFieldSize(int size) => size is 0 or 4 or 8; + + /// + /// Reads a version-selected 16-bit or 32-bit unsigned identifier or count. + /// + /// The bounded box payload. + /// Indicates that the field is 32 bits rather than 16 bits. + /// The running payload offset, advanced past the field. + /// The decoded unsigned value. + private static uint ReadUInt16Or32(Span buffer, bool isLarge, ref int bytesRead) + { + int fieldLength = isLarge ? 4 : 2; + EnsureBufferRemaining(buffer, bytesRead, fieldLength, "versioned integer field"); + + uint result; + if (isLarge) + { + result = BinaryPrimitives.ReadUInt32BigEndian(buffer[bytesRead..]); + bytesRead += 4; + } + else + { + result = BinaryPrimitives.ReadUInt16BigEndian(buffer[bytesRead..]); + bytesRead += 2; + } + + return result; + } + + /// + /// Reads a zero-width, 32-bit, or 64-bit unsigned item-location field into the supported stream range. + /// + /// The bounded item-location payload. + /// The field width selected by the item-location size nibble. + /// The running payload offset, advanced past the field. + /// The decoded nonnegative stream offset or length. + private static long ReadUIntVariable(Span buffer, int numBytes, ref int bytesRead) + { + EnsureBufferRemaining(buffer, bytesRead, numBytes, "item location"); + ulong result = numBytes switch + { + 0 => 0, + 4 => BinaryPrimitives.ReadUInt32BigEndian(buffer[bytesRead..]), + 8 => BinaryPrimitives.ReadUInt64BigEndian(buffer[bytesRead..]), + _ => throw new InvalidImageContentException("The item location box uses an invalid integer field size.") + }; + + if (result > long.MaxValue) + { + throw new InvalidImageContentException("An item location offset exceeds the supported stream range."); + } + + bytesRead += numBytes; + return (long)result; + } + + /// + /// Assembles one logical item from its declared extents. + /// + /// The complete seekable HEIF container stream. + /// The item whose extents are requested. + /// An owner containing the contiguous item payload. + private IMemoryOwner ReadItemData(BufferedReadStream stream, HeifItem item) + { + long itemLength = 0; + foreach (HeifLocation location in item.DataLocations) + { + if (location.Length < 0 || itemLength > int.MaxValue - location.Length) + { + throw new InvalidImageContentException($"Item {item.Id} data is too large to buffer."); + } + + itemLength += location.Length; + } + + if (itemLength == 0) + { + throw new InvalidImageContentException($"Item {item.Id} has no data extents."); + } + + int bufferLength = (int)itemLength; + IMemoryOwner itemMemory = this.configuration.MemoryAllocator.Allocate(bufferLength); + try + { + // One logical item is the concatenation of its extents in declared order. Only the selected item is + // materialized, and its owner is released as soon as the codec or metadata consumer has finished. + Span itemBuffer = itemMemory.GetSpan()[..bufferLength]; + int writeOffset = 0; + foreach (HeifLocation location in item.DataLocations) + { + if (location.BaseOffset < 0 + || location.Offset < 0 + || location.BaseOffset > long.MaxValue - location.Offset) + { + throw new InvalidImageContentException($"Item {item.Id} has an invalid extent offset."); + } + + long relativeOffset = location.BaseOffset + location.Offset; + long sourceOffset; + long sourceBytesRemaining; + if (location.Origin == HeifLocationOffsetOrigin.FileOffset) + { + // Construction method zero resolves base_offset + extent_offset from the start of the file. + long fileLength = stream.Length - this.fileStartOffset; + HeifBoxReader.EnsureInsideParent(relativeOffset, fileLength); + sourceOffset = this.fileStartOffset + relativeOffset; + sourceBytesRemaining = fileLength - relativeOffset; + } + else if (location.Origin == HeifLocationOffsetOrigin.ItemDataOffset) + { + if (this.itemDataOffset < 0 || relativeOffset > this.itemDataLength) + { + throw new InvalidImageContentException($"Item {item.Id} has an extent outside its item data box."); + } + + // Construction method one resolves the same relative value from the idat payload start. + sourceOffset = this.itemDataOffset + relativeOffset; + sourceBytesRemaining = this.itemDataLength - relativeOffset; + } + else + { + throw new InvalidImageContentException($"Item {item.Id} uses an unsupported location origin."); + } + + HeifBoxReader.EnsureInsideParent(location.Length, sourceBytesRemaining); + stream.Position = sourceOffset; + int extentLength = (int)location.Length; + int bytesRead = stream.Read(itemBuffer.Slice(writeOffset, extentLength)); + if (bytesRead != extentLength) + { + throw new InvalidImageContentException($"Item {item.Id} extent is truncated."); + } + + writeOffset += extentLength; + } + + return itemMemory; + } + catch + { + itemMemory.Dispose(); + throw; + } + } + + /// + /// Resolves item extents, selects the primary or supported thumbnail decoder, and reconstructs the image. + /// + /// The destination pixel format. + /// The complete seekable HEIF container stream. + /// The token used to cancel item assembly and payload decoding. + /// The image reconstructed from the selected item. + private Image DecodePrimaryItem(BufferedReadStream stream, CancellationToken cancellationToken) + where TPixel : unmanaged, IPixel + { + Func> itemDataReader = item => this.ReadItemData(stream, item); + + HeifItem? rootItem = this.FindItemById(this.primaryItem); + if (rootItem is null) + { + throw new ImageFormatException("No primary HEIF item defined."); + } + + Image? image = null; + HeifItem itemToDecode = rootItem; + IHeifItemDecoder? itemDecoder = this.GetItemDecoder(rootItem, itemDataReader); + bool supportedItemFound = itemDecoder is not null; + if (itemDecoder is not null) + { + this.ExecuteImageDataSegmentAction( + () => image = this.DecodeImageItem(rootItem, itemDecoder, itemDataReader, cancellationToken)); + } + + if (image is null) + { + // An unsupported primary item always permits its registered thumbnail fallback. IgnoreImageData also + // reaches this branch after a recoverable primary payload failure, matching other multi-image decoders. + HeifItem? thumbnailItem = this.FindDecodableThumbnail(rootItem); + if (thumbnailItem is not null) + { + itemDecoder = HeifCompressionFactory.GetDecoder(thumbnailItem.Type); + supportedItemFound |= itemDecoder is not null; + if (itemDecoder is not null) + { + itemToDecode = thumbnailItem; + this.ExecuteImageDataSegmentAction( + () => image = this.DecodeImageItem(thumbnailItem, itemDecoder, itemDataReader, cancellationToken)); + } + } + } + + if (image is null || itemDecoder is null) + { + if (!supportedItemFound) + { + throw new ImageFormatException("No supported image item was found inside this HEIF container."); + } + + throw new InvalidImageContentException("The HEIF container does not contain a decodable image item."); + } + + try + { + bool hasAlpha = false; + this.ExecuteImageDataSegmentAction( + () => hasAlpha = this.DecodeAlphaPlane(itemToDecode, itemDataReader, image.Frames.RootFrame, cancellationToken)); + + if (!this.Options.SkipMetadata) + { + this.ApplyItemColorMetadata(image.Metadata, itemToDecode); + this.ApplyItemHdrMetadata(image.Metadata, itemToDecode); + this.ApplyAssociatedMetadata(image.Metadata, rootItem, itemDataReader); + } + + // MIAF defines crop, rotation, and mirror as presentation operations in that order. Applying the + // container transforms after item composition keeps every composed plane in the same coordinate space. + ApplyPresentationTransforms(image, itemToDecode); + + if (!this.Options.SkipMetadata) + { + this.ApplyItemPixelAspectRatioMetadata(image.Metadata, itemToDecode); + + // ICC conversion belongs to the presented RGB image. Running it after alpha, grid composition, crop, + // rotation, and mirroring keeps still images aligned with the sequence path and avoids converting + // pixels removed by a clean-aperture crop. + _ = this.TryConvertIccProfile(image); + } + + // The decoder determines the compression of the pixels that were actually returned, including grid tiles + // and a thumbnail fallback when the primary image compression is not available. + HeifMetadata meta = image.Metadata.GetHeifMetadata(); + meta.CompressionMethod = itemDecoder.CompressionMethod; + meta.HasAlpha = hasAlpha; + if (this.Options.SkipMetadata) + { + // AV1 item decoders still parse encoded metadata to enforce codec/container equivalence and select + // the correct conversion. Remove the exposed values so the decoder option suppresses metadata. + image.Metadata.CicpProfile = null; + meta.ContentLightLevel = null; + meta.MasteringDisplayColorVolume = null; + meta.ContentColorVolume = null; + meta.AmbientViewingEnvironment = null; + meta.ReferenceViewingEnvironment = null; + meta.NominalDiffuseWhite = null; + } + + return image; + } + catch + { + // Ownership transfers to the caller only after every auxiliary plane has been composed successfully. + image.Dispose(); + throw; + } + } + + /// + /// Applies the color profiles associated with a presented still-image item. + /// + /// The image metadata receiving the profiles. + /// The color image item whose pixels are presented. + private void ApplyItemColorMetadata(ImageMetadata metadata, HeifItem colorItem) + { + // Color properties can be associated with the derived grid or its coded tile items. Prefer the presentation + // grid and use the first decodable tile only when the grid does not provide the corresponding profile. + HeifItem? gridTile = colorItem.Type == Heif4CharCode.Grid + ? this.FindDecodableGridTile(colorItem) + : null; + + // The associated item model is decoder-private and no longer mutates after property resolution. Reuse its + // profiles so the returned metadata does not duplicate their storage. + IccProfile? iccProfile = colorItem.IccProfile ?? gridTile?.IccProfile; + if (iccProfile is not null) + { + metadata.IccProfile = iccProfile; + } + + CicpProfile? cicpProfile = colorItem.CicpProfile ?? gridTile?.CicpProfile; + if (cicpProfile is not null) + { + metadata.CicpProfile = cicpProfile; + } + } + + /// + /// Applies high-dynamic-range metadata associated with a presented still-image item. + /// + /// The image metadata receiving the high-dynamic-range description. + /// The image item whose pixels are presented. + private void ApplyItemHdrMetadata(ImageMetadata metadata, HeifItem imageItem) + { + HeifItem? gridTile = imageItem.Type == Heif4CharCode.Grid + ? this.FindDecodableGridTile(imageItem) + : null; + + // A derived grid can describe the complete presentation. Fall back to the first coded tile only when the + // grid does not carry its own value, matching the precedence used for its color-profile properties. + HeifContentLightLevel? contentLightLevel = imageItem.ContentLightLevel ?? gridTile?.ContentLightLevel; + if (contentLightLevel is not null) + { + metadata.GetHeifMetadata().ContentLightLevel = contentLightLevel; + } + + HeifMasteringDisplayColorVolume? masteringDisplayColorVolume = imageItem.MasteringDisplayColorVolume + ?? gridTile?.MasteringDisplayColorVolume; + + if (masteringDisplayColorVolume is not null) + { + metadata.GetHeifMetadata().MasteringDisplayColorVolume = masteringDisplayColorVolume; + } + + HeifContentColorVolume? contentColorVolume = imageItem.ContentColorVolume ?? gridTile?.ContentColorVolume; + if (contentColorVolume is not null) + { + metadata.GetHeifMetadata().ContentColorVolume = contentColorVolume; + } + + HeifAmbientViewingEnvironment? ambientViewingEnvironment = imageItem.AmbientViewingEnvironment + ?? gridTile?.AmbientViewingEnvironment; + + if (ambientViewingEnvironment is not null) + { + metadata.GetHeifMetadata().AmbientViewingEnvironment = ambientViewingEnvironment; + } + + HeifReferenceViewingEnvironment? referenceViewingEnvironment = imageItem.ReferenceViewingEnvironment + ?? gridTile?.ReferenceViewingEnvironment; + + if (referenceViewingEnvironment is not null) + { + metadata.GetHeifMetadata().ReferenceViewingEnvironment = referenceViewingEnvironment; + } + + HeifNominalDiffuseWhite? nominalDiffuseWhite = imageItem.NominalDiffuseWhite ?? gridTile?.NominalDiffuseWhite; + if (nominalDiffuseWhite is not null) + { + metadata.GetHeifMetadata().NominalDiffuseWhite = nominalDiffuseWhite; + } + } + + /// + /// Applies the pixel aspect ratio associated with a presented still-image item. + /// + /// The image metadata receiving the aspect ratio. + /// The image item whose pixels are presented. + private void ApplyItemPixelAspectRatioMetadata(ImageMetadata metadata, HeifItem imageItem) + { + HeifItem? gridTile = imageItem.Type == Heif4CharCode.Grid + ? this.FindDecodableGridTile(imageItem) + : null; + + HeifPixelAspectRatio? pixelAspectRatio = imageItem.PixelAspectRatio ?? gridTile?.PixelAspectRatio; + if (pixelAspectRatio is null) + { + return; + } + + ApplyPixelAspectRatioMetadata(metadata, pixelAspectRatio, imageItem.RotationAngle); + } + + /// + /// Applies registered pixel spacing to ImageSharp's aspect-ratio resolution metadata. + /// + /// The image metadata receiving the aspect ratio. + /// The optional registered horizontal and vertical spacing. + /// The optional counter-clockwise quarter-turn count. + private static void ApplyPixelAspectRatioMetadata( + ImageMetadata metadata, + HeifPixelAspectRatio? pixelAspectRatio, + byte? rotationAngle) + { + if (pixelAspectRatio is null) + { + return; + } + + // ImageMetadata expresses pixel width:height as vertical-density:horizontal-density. A quarter-turn exchanges + // the displayed pixel axes, so it also exchanges which spacing value supplies each density. + bool swapsAxes = rotationAngle is 1 or 3; + metadata.HorizontalResolution = swapsAxes + ? pixelAspectRatio.HorizontalSpacing + : pixelAspectRatio.VerticalSpacing; + + metadata.VerticalResolution = swapsAxes + ? pixelAspectRatio.VerticalSpacing + : pixelAspectRatio.HorizontalSpacing; + + metadata.ResolutionUnits = PixelResolutionUnit.AspectRatio; + } + + /// + /// Applies Exif and XMP metadata items that describe a decoded color image item. + /// + /// The decoded image metadata receiving the profiles. + /// The color image item described by the metadata links. + /// Reads one associated item payload on demand. + private void ApplyAssociatedMetadata( + ImageMetadata metadata, + HeifItem colorItem, + Func> itemDataReader) + { + foreach (HeifItemLink link in this.itemLinks) + { + if (link.Type != Heif4CharCode.Cdsc || !link.DestinationIds.Contains(colorItem.Id)) + { + continue; + } + + HeifItem? metadataItem = this.FindItemById(link.SourceId); + if (metadataItem is null) + { + continue; + } + + if (metadataItem.Type == Heif4CharCode.Exif) + { + this.ExecuteAncillarySegmentAction(() => + { + using IMemoryOwner itemMemory = itemDataReader(metadataItem); + ApplyExifProfile(metadata, itemMemory.GetSpan()); + }); + } + else if (metadataItem.Type == Heif4CharCode.Mime && + string.Equals(metadataItem.ContentType, "application/rdf+xml", StringComparison.Ordinal)) + { + this.ExecuteAncillarySegmentAction(() => + { + using IMemoryOwner itemMemory = itemDataReader(metadataItem); + Span itemData = itemMemory.GetSpan(); + + // XmpProfile retains its input array after the assembled item buffer is returned to its pool. + byte[] ownedData = new byte[itemData.Length]; + itemData.CopyTo(ownedData); + metadata.XmpProfile = new XmpProfile(ownedData); + }); + } + } + } + + /// + /// Validates the HEIF Exif TIFF-header offset and applies the contained TIFF payload. + /// + /// The image metadata receiving the Exif profile. + /// The complete HEIF Exif item including its four-byte offset field. + private static void ApplyExifProfile(ImageMetadata metadata, ReadOnlySpan itemData) + { + if (itemData.Length < 8) + { + throw new InvalidImageContentException("The HEIF Exif item is truncated."); + } + + uint declaredTiffHeaderOffset = BinaryPrimitives.ReadUInt32BigEndian(itemData); + ReadOnlySpan exifData = itemData[4..]; + int actualTiffHeaderOffset = -1; + + // Annex A stores the offset to the first TIFF byte-order marker. Match libavif by finding the first valid + // TIFF signature and requiring the declared offset to identify that same header. + for (int i = 0; i <= exifData.Length - 4; i++) + { + bool isBigEndianTiff = exifData[i] == (byte)'M' && + exifData[i + 1] == (byte)'M' && + exifData[i + 2] == 0 && + exifData[i + 3] == 42; + + bool isLittleEndianTiff = exifData[i] == (byte)'I' && + exifData[i + 1] == (byte)'I' && + exifData[i + 2] == 42 && + exifData[i + 3] == 0; + + if (isBigEndianTiff || isLittleEndianTiff) + { + actualTiffHeaderOffset = i; + break; + } + } + + if (actualTiffHeaderOffset < 0 || declaredTiffHeaderOffset != (uint)actualTiffHeaderOffset) + { + throw new InvalidImageContentException("The HEIF Exif item has an invalid TIFF-header offset."); + } + + ReadOnlySpan tiffData = exifData[actualTiffHeaderOffset..]; + + // ExifProfile retains its input array after the assembled item buffers are disposed at the end of decode. + byte[] ownedData = new byte[tiffData.Length]; + tiffData.CopyTo(ownedData); + metadata.ExifProfile = new ExifProfile(ownedData); + } + + /// + /// Selects the registered coded-image or grid decoder for an image item. + /// + /// The destination pixel format. + /// The coded or derived image item. + /// Reads one selected item payload on demand. + /// The selected decoder, or when the item cannot be reconstructed. + private IHeifItemDecoder? GetItemDecoder( + HeifItem item, + Func> itemDataReader) + where TPixel : unmanaged, IPixel + => item.Type == Heif4CharCode.Grid && this.FindDecodableGridTile(item) is not null + ? new GridHeifItemDecoder(this.items, this.itemLinks, itemDataReader) + : HeifCompressionFactory.GetDecoder(item.Type); + + /// + /// Decodes one image item from its assembled payload. + /// + /// The destination pixel format. + /// The image item to decode. + /// The decoder selected for the item. + /// Reads the selected item payload. + /// The token used to cancel the payload decode. + /// The decoded image. + private Image DecodeImageItem( + HeifItem item, + IHeifItemDecoder decoder, + Func> itemDataReader, + CancellationToken cancellationToken) + where TPixel : unmanaged, IPixel + { + using IMemoryOwner itemMemory = itemDataReader(item); + Image image = decoder.DecodeItemData( + this.payloadOptions, + item, + itemMemory.GetSpan(), + item.CicpProfile, + cancellationToken); + + try + { + HeifItemDecoderUtilities.ScaleToItemExtent(image, item); + return image; + } + catch + { + image.Dispose(); + throw; + } + } + + /// + /// Gets the dimensions of an image item after its clean-aperture and rotation properties are applied. + /// + /// The image item whose presentation dimensions are requested. + /// The item dimensions after the optional crop and quarter-turn rotation. + private static Size GetPresentationExtent(HeifItem item) + { + Size extent = item.CleanAperture is not null ? item.CleanAperture.Value.ToRectangle(item.Extent).Size : item.Extent; + return item.RotationAngle is not null && (item.RotationAngle.Value & 1) != 0 + ? new Size(extent.Height, extent.Width) + : extent; + } + + /// + /// Gets the dimensions of a sequence sample after its clean-aperture and rotation properties are applied. + /// + /// The selected color track whose samples share the presentation properties. + /// The displayed frame dimensions. + private static Size GetSequencePresentationExtent(HeifSequenceTrack track) + { + Size codedExtent = new(track.CodedWidth, track.CodedHeight); + Size extent = track.CleanAperture is not null ? track.CleanAperture.Value.ToRectangle(codedExtent).Size : codedExtent; + return track.RotationAngle is not null && (track.RotationAngle.Value & 1) != 0 + ? new Size(extent.Height, extent.Width) + : extent; + } + + /// + /// Applies the clean-aperture, rotation, and mirror properties associated with an image item. + /// + /// The image pixel format. + /// The decoded image item. + /// The item carrying the presentation properties. + private static void ApplyPresentationTransforms(Image image, HeifItem item) + where TPixel : unmanaged, IPixel + => ApplyPresentationTransforms(image, item.CleanAperture, item.RotationAngle, item.MirrorAxis); + + /// + /// Applies shared clean-aperture, rotation, and mirror properties to every frame of an image presentation. + /// + /// The image pixel format. + /// The decoded image presentation. + /// The optional clean-aperture crop. + /// The optional counter-clockwise quarter-turn count. + /// The optional horizontal or vertical mirror axis. + private static void ApplyPresentationTransforms( + Image image, + HeifCleanAperture? cleanAperture, + byte? rotationAngle, + byte? mirrorAxis) + where TPixel : unmanaged, IPixel + { + if (cleanAperture is not null) + { + Rectangle cropRectangle = cleanAperture.Value.ToRectangle(image.Size); + if (cropRectangle != image.Bounds) + { + image.Mutate(context => context.Crop(cropRectangle)); + } + } + + if (rotationAngle is not null) + { + // HEIF angles count quarter turns counter-clockwise, while ImageSharp's optimized rotate modes are clockwise. + RotateMode rotation = rotationAngle.Value switch + { + 1 => RotateMode.Rotate270, + 2 => RotateMode.Rotate180, + 3 => RotateMode.Rotate90, + _ => RotateMode.None + }; + + if (rotation != RotateMode.None) + { + image.Mutate(context => context.Rotate(rotation)); + } + } + + if (mirrorAxis is not null) + { + // Axis zero reflects top-to-bottom around the horizontal axis; axis one reflects left-to-right. + FlipMode flip = mirrorAxis.Value == 0 ? FlipMode.Vertical : FlipMode.Horizontal; + image.Mutate(context => context.Flip(flip)); + } + } + + /// + /// Decodes and composes the direct or per-grid-tile alpha auxiliary associated with a color image item. + /// + /// The destination color pixel type. + /// The color image item whose alpha plane is requested. + /// Reads one selected item payload on demand. + /// The decoded color frame receiving alpha values. + /// The token used to cancel the auxiliary payload decode. + /// when an auxiliary alpha plane was decoded and composed. + private bool DecodeAlphaPlane( + HeifItem colorItem, + Func> itemDataReader, + ImageFrame destination, + CancellationToken cancellationToken) + where TPixel : unmanaged, IPixel + { + HeifItem? alphaItem = this.FindAlphaItem(colorItem); + if (alphaItem is not null) + { + // libavif releases through 1.3 omitted alpha transform associations, so accept complete absence for + // compatibility. If either property is present, it must match the color item before plane composition. + bool alphaHasTransforms = alphaItem.CleanAperture is not null || + alphaItem.RotationAngle is not null || + alphaItem.MirrorAxis is not null; + + bool cleanAperturesMatch = alphaItem.CleanAperture is null + ? colorItem.CleanAperture is null + : colorItem.CleanAperture is not null && alphaItem.CleanAperture.Value.Equals(colorItem.CleanAperture.Value); + + if (alphaHasTransforms && + (!cleanAperturesMatch || alphaItem.RotationAngle != colorItem.RotationAngle || alphaItem.MirrorAxis != colorItem.MirrorAxis)) + { + throw new ImageFormatException("The alpha auxiliary image and color image use different presentation transforms."); + } + + IHeifItemDecoder? itemDecoder = this.GetItemDecoder(alphaItem, itemDataReader); + if (itemDecoder is not IHeifAlphaItemDecoder decoder) + { + throw new ImageFormatException($"The alpha auxiliary item uses unsupported item type '{alphaItem.Type}'."); + } + + bool premultiplied = this.itemLinks.Any( + link => link.Type == Heif4CharCode.Prem + && link.SourceId == colorItem.Id + && link.DestinationIds.Contains(alphaItem.Id)); + + using IMemoryOwner itemMemory = itemDataReader(alphaItem); + decoder.DecodeAlphaItemData( + this.payloadOptions, + alphaItem, + itemMemory.GetSpan(), + destination, + destination.Size, + destination.Bounds, + premultiplied, + cancellationToken); + + return true; + } + + if (colorItem.Type != Heif4CharCode.Grid) + { + return false; + } + + List? alphaTileIds = this.FindGridAlphaTiles(colorItem); + if (alphaTileIds is null) + { + return false; + } + + // The color grid descriptor defines the same row/column layout and output canvas for per-tile alpha + // auxiliaries. Supplying their IDs lets the existing grid compositor preserve that normative ordering. + GridHeifItemDecoder gridDecoder = new( + this.items, + this.itemLinks, + itemDataReader, + alphaTileIds); + + using IMemoryOwner gridMemory = itemDataReader(colorItem); + gridDecoder.DecodeAlphaItemData( + this.payloadOptions, + colorItem, + gridMemory.GetSpan(), + destination, + destination.Size, + destination.Bounds, + false, + cancellationToken); + + return true; + } + + /// + /// Validates that a fixed-width field remains within a buffered box payload. + /// + /// The bounded box payload. + /// The zero-based field offset. + /// The field width in bytes. + /// The diagnostic name used for malformed input errors. + private static void EnsureBufferRemaining(ReadOnlySpan buffer, int offset, int count, string boxName) + { + if ((uint)offset > (uint)buffer.Length || (uint)count > (uint)(buffer.Length - offset)) + { + throw new InvalidImageContentException($"The {boxName} box is truncated."); + } + } + + /// + /// Finds an item by its file-defined identifier. + /// + /// The item identifier. + /// The matching item, or when it has not been declared. + private HeifItem? FindItemById(uint itemId) + => this.items.FirstOrDefault(item => item.Id == itemId); + + /// + /// Resolves an item identifier referenced by another parsed HEIF structure. + /// + /// The required item identifier. + /// The declared item. + /// No item with the referenced identifier was declared. + private HeifItem FindRequiredItemById(uint itemId) + => this.FindItemById(itemId) + ?? throw new InvalidImageContentException($"HEIF item reference targets undeclared item {itemId}."); + + /// + /// Finds the alpha auxiliary image linked to a color image item. + /// + /// The color image item. + /// The alpha auxiliary item, or when no registered alpha relationship exists. + private HeifItem? FindAlphaItem(HeifItem colorItem) + { + HeifItem? alphaItem = null; + foreach (HeifItemLink link in this.itemLinks) + { + if (link.Type != Heif4CharCode.Auxl || !link.DestinationIds.Contains(colorItem.Id)) + { + continue; + } + + HeifItem candidate = this.FindRequiredItemById(link.SourceId); + if (!HeifConstants.IsAlphaAuxiliaryType(candidate.AuxiliaryType)) + { + continue; + } + + if (alphaItem is not null && alphaItem.Id != candidate.Id) + { + throw new InvalidImageContentException($"Item {colorItem.Id} has more than one alpha auxiliary image."); + } + + alphaItem = candidate; + } + + return alphaItem; + } + + /// + /// Resolves one alpha auxiliary image for each tile of a color grid. + /// + /// The color grid whose tile order defines the alpha grid. + /// + /// The row-major alpha tile identifiers, or when any color tile has no alpha auxiliary. + /// + private List? FindGridAlphaTiles(HeifItem gridItem) + { + List colorTileIds = []; + foreach (HeifItemLink link in this.itemLinks) + { + if (link.Type == Heif4CharCode.Dimg && link.SourceId == gridItem.Id) + { + colorTileIds.AddRange(link.DestinationIds); + } + } + + if (colorTileIds.Count == 0) + { + return null; + } + + List alphaTileIds = new(colorTileIds.Count); + foreach (uint colorTileId in colorTileIds) + { + HeifItem colorTile = this.FindRequiredItemById(colorTileId); + HeifItem? alphaTile = this.FindAlphaItem(colorTile); + if (alphaTile is null) + { + // A partial set cannot describe an alpha plane for the complete grid. libavif treats this case as + // an opaque image rather than mixing opaque cells with auxiliary alpha cells. + return null; + } + + bool alphaIsDerivedTile = this.itemLinks.Any( + link => link.Type == Heif4CharCode.Dimg && link.DestinationIds.Contains(alphaTile.Id)); + + if (alphaIsDerivedTile) + { + throw new InvalidImageContentException($"Alpha auxiliary item {alphaTile.Id} is already a derived-image tile."); + } + + alphaTileIds.Add(alphaTile.Id); + } + + return alphaTileIds; + } + + /// + /// Finds the first tile of a grid when every referenced tile uses a registered still-image decoder. + /// + /// The destination pixel format used to select item decoders. + /// The grid derived-image item. + /// The first decodable grid tile, or when the grid has no tiles or any tile cannot be decoded. + private HeifItem? FindDecodableGridTile(HeifItem gridItem) + where TPixel : unmanaged, IPixel + { + HeifItem? firstTile = null; + foreach (HeifItemLink link in this.itemLinks) + { + if (link.Type != Heif4CharCode.Dimg || link.SourceId != gridItem.Id) + { + continue; + } + + foreach (uint itemId in link.DestinationIds) + { + HeifItem tile = this.FindRequiredItemById(itemId); + if (HeifCompressionFactory.GetDecoder(tile.Type) is null) + { + // A partially decodable grid cannot yield the requested canvas. Returning no tile lets the + // caller select a thumbnail of the complete primary presentation when one is available. + return null; + } + + firstTile ??= tile; + } + } + + return firstTile; + } + + /// + /// Finds a decodable thumbnail that represents the specified master image item. + /// + /// The destination pixel format used to select item decoders. + /// The master image item referenced by the thumbnail. + /// A decodable thumbnail item, or when no matching thumbnail is available. + private HeifItem? FindDecodableThumbnail(HeifItem masterItem) + where TPixel : unmanaged, IPixel + { + // A thumbnail reference points from the thumbnail item to the master image. Restrict fallback to this + // presentation rather than allowing an unrelated thumbnail elsewhere in the file to be selected. + HeifItemLink? thumbnailReference = this.itemLinks.FirstOrDefault( + link => link.Type == Heif4CharCode.Thmb && link.DestinationIds.Contains(masterItem.Id)); + + if (thumbnailReference is null) + { + return null; + } + + HeifItem thumbnailItem = this.FindRequiredItemById(thumbnailReference.SourceId); + if (HeifCompressionFactory.GetDecoder(thumbnailItem.Type) is null) + { + return null; + } + + return thumbnailItem; + } + + /// + /// Decodes the UTF-8 bytes preceding the first null terminator. + /// + /// The bytes beginning at a required null-terminated string. + /// The number of source bytes consumed, including the terminator. + /// The decoded string without its terminator. + private static string ReadNullTerminatedString(Span span, out int bytesRead) + { + int terminator = span.IndexOf((byte)0); + if (terminator < 0) + { + throw new InvalidImageContentException("A null-terminated item information string is truncated."); + } + + bytesRead = terminator + 1; + return Encoding.UTF8.GetString(span[..terminator]); + } +} diff --git a/src/ImageSharp/Formats/Heif/HeifEncoder.cs b/src/ImageSharp/Formats/Heif/HeifEncoder.cs new file mode 100644 index 0000000000..1b6472ee5a --- /dev/null +++ b/src/ImageSharp/Formats/Heif/HeifEncoder.cs @@ -0,0 +1,120 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif; + +/// +/// Image encoder for writing image data to a stream in a HEIF container. +/// +public sealed class HeifEncoder : AnimatedImageEncoder +{ + /// + /// Backing field for . + /// + private int? quality; + + /// + /// Backing field for . + /// + private int? alphaQuality; + + /// + /// Backing field for . + /// + private int effort = 5; + + /// + /// Gets the compression method used for the primary image item. + /// The default is . + /// + public HeifCompressionMethod CompressionMethod { get; init; } = HeifCompressionMethod.Av1; + + /// + /// Gets the lossy compression quality, or to use the compression method's default quality. + /// Valid values range from 0 for the lowest quality to 100 for the highest quality. A value of 100 does not + /// enable encoding. + /// + /// The quality is outside the range 0 to 100. + public int? Quality + { + get => this.quality; + init + { + if (value is < 0 or > 100) + { + throw new ArgumentException("Quality must be in the range [0..100]."); + } + + this.quality = value; + } + } + + /// + /// Gets the lossy compression quality for the auxiliary alpha image, or to use the + /// effective . Valid values range from 0 for the lowest quality to 100 for the highest + /// quality. This option has no effect when the encoded image does not require an auxiliary alpha image. + /// + /// The alpha quality is outside the range 0 to 100. + public int? AlphaQuality + { + get => this.alphaQuality; + init + { + if (value is < 0 or > 100) + { + throw new ArgumentException("Alpha quality must be in the range [0..100]."); + } + + this.alphaQuality = value; + } + } + + /// + /// Gets the encoding effort in the range 0 to 10. A value of 0 selects the fastest encoding and 10 selects the + /// slowest encoding with the greatest compression effort. The default is 5. Legacy JPEG image items use a fixed + /// encoding effort, so this option does not affect them. + /// + /// The effort is outside the range 0 to 10. + public int Effort + { + get => this.effort; + init + { + if (value is < 0 or > 10) + { + throw new ArgumentException("Effort must be in the range [0..10]."); + } + + this.effort = value; + } + } + + /// + /// Gets a value indicating whether the primary and auxiliary alpha images are encoded without loss. When + /// , and do not affect the encoded image. + /// This option has no effect on legacy JPEG image items. The default is . + /// + public bool Lossless { get; init; } + + /// + /// Gets the encoded precision of each image component, or to use the HEIF metadata bit + /// depth. Metadata that does not specify a bit depth defaults to . Legacy JPEG + /// image items are always encoded with . + /// + public HeifBitDepth? BitDepth { get; init; } + + /// + /// Gets the encoded chroma sampling, or to use + /// for lossy encoding and for lossless encoding. Oversized still + /// images use when a subsampled AVIF grid cannot represent an odd + /// output dimension. + /// + public HeifChromaSubsampling? ChromaSubsampling { get; init; } + + /// + protected override void Encode(Image image, Stream stream, CancellationToken cancellationToken) + { + HeifEncoderCore encoder = new(image.Configuration, this); + encoder.Encode(image, stream, cancellationToken); + } +} diff --git a/src/ImageSharp/Formats/Heif/HeifEncoderCore.Sequence.cs b/src/ImageSharp/Formats/Heif/HeifEncoderCore.Sequence.cs new file mode 100644 index 0000000000..22d3ff710b --- /dev/null +++ b/src/ImageSharp/Formats/Heif/HeifEncoderCore.Sequence.cs @@ -0,0 +1,1277 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Buffers; +using System.Buffers.Binary; +using System.Text; +using SixLabors.ImageSharp.Formats.Heif.Av1; +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline; +using SixLabors.ImageSharp.IO; +using SixLabors.ImageSharp.Metadata.Profiles.Cicp; +using SixLabors.ImageSharp.Metadata.Profiles.Icc; +using SixLabors.ImageSharp.PixelFormats; + +namespace SixLabors.ImageSharp.Formats.Heif; + +internal sealed partial class HeifEncoderCore +{ + /// + /// The millisecond media timescale used when every frame delay can be represented exactly. + /// + private const uint DefaultSequenceTimescale = 1000; + + /// + /// The microsecond fallback used when the exact common frame-delay timescale exceeds 32 bits. + /// + private const uint FallbackSequenceTimescale = 1000000; + + /// + /// The identity value for signed 16.16 movie and track matrix entries. + /// + private const uint UnityFixed16Point16 = 1U << 16; + + /// + /// The identity value for the signed 2.30 homogeneous movie and track matrix entry. + /// + private const uint UnityFixed2Point30 = 1U << 30; + + /// + /// The identity value for unsigned 8.8 track volume. + /// + private const ushort UnityFixed8Point8 = 1 << 8; + + /// + /// The packed ISO 639-2/T language code for undetermined content. + /// + private const ushort PackedUndeterminedLanguage = 0x55C4; + + /// + /// The coding-constraints flag stating that every reference picture is intra. + /// + private const uint AllReferencePicturesIntraMask = 1U << 31; + + /// + /// The coding-constraints flag stating that intra prediction is used. + /// + private const uint IntraPicturePredictionUsedMask = 1U << 30; + + /// + /// The conventional 72-dpi horizontal and vertical resolution stored as unsigned 16.16. + /// + private const uint DefaultVisualSampleResolution = 72U << 16; + + /// + /// The fixed visual-sample-entry compressor-name field length. + /// + private const int VisualSampleCompressorNameLength = 32; + + /// + /// The visual-sample-entry depth used for color pictures. + /// + private const ushort VisualSampleDepth = 24; + + private Av1EncodingSettings ResolveAv1Encoding(Image image) + where TPixel : unmanaged, IPixel + { + HeifMetadata metadata = image.Metadata.GetHeifMetadata(); + HeifBitDepth bitDepth = this.encoder.BitDepth ?? metadata.BitDepth; + Av1BitDepth av1BitDepth = bitDepth switch + { + HeifBitDepth.Bit8 => Av1BitDepth.EightBit, + HeifBitDepth.Bit10 => Av1BitDepth.TenBit, + HeifBitDepth.Bit12 => Av1BitDepth.TwelveBit, + _ => throw new NotSupportedException($"HEIF bit depth '{bitDepth}' is not supported.") + }; + + HeifChromaSubsampling defaultChromaSubsampling = this.encoder.Lossless + ? HeifChromaSubsampling.Yuv444 + : HeifChromaSubsampling.Yuv420; + + HeifChromaSubsampling chromaSubsampling = this.encoder.ChromaSubsampling ?? + (metadata.IsMonochrome ? HeifChromaSubsampling.Monochrome : defaultChromaSubsampling); + + if (image.Frames.Count == 1 + && (image.Width > Av1Constants.MaxFrameDimension || image.Height > Av1Constants.MaxFrameDimension) + && ((chromaSubsampling == HeifChromaSubsampling.Yuv420 + && (((image.Width & 1) != 0) || ((image.Height & 1) != 0))) + || (chromaSubsampling == HeifChromaSubsampling.Yuv422 && (image.Width & 1) != 0))) + { + // A derived grid requires even output dimensions on every subsampled axis. Resolve incompatible + // sampling through conversion so the complete source dimensions remain representable. + chromaSubsampling = HeifChromaSubsampling.Yuv444; + } + + (bool isMonochrome, bool subsamplingX, bool subsamplingY) = chromaSubsampling switch + { + HeifChromaSubsampling.Monochrome => (true, true, true), + HeifChromaSubsampling.Yuv420 => (false, true, true), + HeifChromaSubsampling.Yuv422 => (false, true, false), + HeifChromaSubsampling.Yuv444 => (false, false, false), + _ => throw new NotSupportedException($"HEIF chroma sampling '{chromaSubsampling}' is not supported.") + }; + + CicpProfile? sourceColorProfile = image.Metadata.CicpProfile; + CicpProfile colorProfile; + if (sourceColorProfile is null) + { + colorProfile = new CicpProfile( + (byte)CicpColorPrimaries.Unspecified, + (byte)CicpTransferCharacteristics.Unspecified, + (byte)CicpMatrixCoefficients.ItuRBt601_7_525, + false); + } + else + { + bool identityMatrix = sourceColorProfile.MatrixCoefficients == CicpMatrixCoefficients.Identity; + bool legalIdentityMatrix = !isMonochrome + && chromaSubsampling == HeifChromaSubsampling.Yuv444; + + bool reversibleMatrix = sourceColorProfile.MatrixCoefficients is CicpMatrixCoefficients.YCgCoRe or CicpMatrixCoefficients.YCgCoRo; + if (sourceColorProfile.MatrixCoefficients == CicpMatrixCoefficients.Unspecified + || (identityMatrix && !legalIdentityMatrix) + || (reversibleMatrix && !isMonochrome && chromaSubsampling != HeifChromaSubsampling.Yuv444)) + { + // Packed source pixels can be converted to the requested sampling even when their metadata + // describes a matrix that requires 4:4:4. Use and signal BT.601 without changing source metadata. + colorProfile = new CicpProfile( + (byte)sourceColorProfile.ColorPrimaries, + (byte)sourceColorProfile.TransferCharacteristics, + (byte)CicpMatrixCoefficients.ItuRBt601_7_525, + sourceColorProfile.FullRange); + } + else if (identityMatrix + && sourceColorProfile.ColorPrimaries == CicpColorPrimaries.ItuRBt709_6 + && sourceColorProfile.TransferCharacteristics == CicpTransferCharacteristics.Iec61966_2_1 + && !sourceColorProfile.FullRange) + { + // Only BT.709/sRGB identity omits the range bit and infers full range. Other identity + // descriptions carry that bit explicitly and can preserve limited-range sample conversion. + colorProfile = new CicpProfile( + (byte)sourceColorProfile.ColorPrimaries, + (byte)sourceColorProfile.TransferCharacteristics, + (byte)sourceColorProfile.MatrixCoefficients, + true); + } + else + { + colorProfile = sourceColorProfile; + } + } + + ObuColorConfig colorConfig = new() + { + IsColorDescriptionPresent = true, + IsMonochrome = isMonochrome, + ColorPrimaries = (ObuColorPrimaries)colorProfile.ColorPrimaries, + TransferCharacteristics = (ObuTransferCharacteristics)colorProfile.TransferCharacteristics, + MatrixCoefficients = (ObuMatrixCoefficients)colorProfile.MatrixCoefficients, + ColorRange = colorProfile.FullRange, + SubSamplingX = subsamplingX, + SubSamplingY = subsamplingY, + ChromaSamplePosition = ObuChromoSamplePosition.Unknown, + BitDepth = av1BitDepth + }; + + ObuColorConfig alphaConfig = new() + { + IsMonochrome = true, + ColorRange = true, + SubSamplingX = true, + SubSamplingY = true, + BitDepth = av1BitDepth + }; + + int quality = this.encoder.Quality ?? 75; + int colorQIndex = this.encoder.Lossless ? 0 : GetAv1QuantizerIndex(quality); + int alphaQuality = this.encoder.AlphaQuality ?? quality; + int alphaQIndex = this.encoder.Lossless ? 0 : GetAv1QuantizerIndex(alphaQuality); + bool hasAlpha = TPixel.GetPixelTypeInfo().AlphaRepresentation != PixelAlphaRepresentation.None; + return new Av1EncodingSettings( + bitDepth, + chromaSubsampling, + colorProfile, + colorConfig, + alphaConfig, + colorQIndex, + alphaQIndex, + hasAlpha); + } + + private HeifSequenceEncoding CompressAv1Sequence( + Image image, + ChunkedMemoryStream stream, + Av1EncodingSettings settings, + Memory samples, + int firstFrameIndex, + CancellationToken cancellationToken) + where TPixel : unmanaged, IPixel + { + byte[]? exifData = null; + uint tiffHeaderOffset = 0; + byte[]? xmpData = null; + if (!this.encoder.SkipMetadata) + { + exifData = GetExifData(image.Metadata, out tiffHeaderOffset); + byte[]? sourceXmpData = image.Metadata.XmpProfile?.Data; + if (sourceXmpData is not null && sourceXmpData.Length > 0) + { + xmpData = sourceXmpData; + } + } + + int frameCount = image.Frames.Count - firstFrameIndex; + uint timescale = GetSequenceTimescale(image, firstFrameIndex); + + // The container needs only offset, length, and duration after each frame is streamed. Color and alpha + // share one allocator-owned table, with each track occupying one contiguous slice until moov is written. + Span colorSamples = samples.Span[..frameCount]; + ImageFrame firstFrame = image.Frames[firstFrameIndex]; + ObuSequenceHeader colorHeader; + bool colorUsesInterPrediction = settings.ColorQIndex != 0; + using (Av1FrameEncoder.SequenceEncoder colorEncoder = Av1FrameEncoder.CreateColorSequenceEncoder( + this.configuration, + image.Width, + image.Height, + settings.ColorConfig, + settings.ColorQIndex, + this.encoder.Effort)) + { + cancellationToken.ThrowIfCancellationRequested(); + long colorOffset = stream.Length; + colorEncoder.EncodeKeyFrame(firstFrame, stream); + colorHeader = colorEncoder.SequenceHeader; + + colorSamples[0] = new HeifSequenceSampleInfo( + colorOffset, + checked((int)(stream.Length - colorOffset)), + GetSequenceSampleDuration(firstFrame.Metadata.GetHeifMetadata().FrameDelay, timescale), + isSyncSample: true); + + for (int sampleIndex = 1; sampleIndex < frameCount; sampleIndex++) + { + cancellationToken.ThrowIfCancellationRequested(); + int frameIndex = firstFrameIndex + sampleIndex; + ImageFrame frame = image.Frames[frameIndex]; + uint duration = GetSequenceSampleDuration(frame.Metadata.GetHeifMetadata().FrameDelay, timescale); + colorOffset = stream.Length; + if (colorUsesInterPrediction) + { + colorEncoder.EncodeInterFrame(frame, stream); + } + else + { + // Lossless AV1 requires 4x4 transforms. Until the inter path supports that reversible size, + // continuation samples remain independent key frames instead of weakening losslessness. + colorEncoder.EncodeKeyFrame(frame, stream); + } + + colorSamples[sampleIndex] = new HeifSequenceSampleInfo( + colorOffset, + checked((int)(stream.Length - colorOffset)), + duration, + isSyncSample: !colorUsesInterPrediction); + } + } + + HeifSequenceTrackEncoding colorTrack = new( + new Av1CodecConfiguration(colorHeader), + samples[..frameCount], + false); + + HeifSequenceTrackEncoding? alphaTrack = null; + if (settings.HasAlpha) + { + Memory alphaSampleMemory = samples.Slice(frameCount, frameCount); + Span alphaSamples = alphaSampleMemory.Span; + ObuSequenceHeader alphaHeader; + bool alphaUsesInterPrediction = settings.AlphaQIndex != 0; + using (Av1FrameEncoder.SequenceEncoder alphaEncoder = Av1FrameEncoder.CreateAlphaSequenceEncoder( + this.configuration, + image.Width, + image.Height, + settings.AlphaConfig, + settings.AlphaQIndex, + this.encoder.Effort)) + { + cancellationToken.ThrowIfCancellationRequested(); + long alphaOffset = stream.Length; + alphaEncoder.EncodeKeyFrame(firstFrame, stream); + alphaHeader = alphaEncoder.SequenceHeader; + + alphaSamples[0] = new HeifSequenceSampleInfo( + alphaOffset, + checked((int)(stream.Length - alphaOffset)), + colorSamples[0].Duration, + isSyncSample: true); + + for (int sampleIndex = 1; sampleIndex < frameCount; sampleIndex++) + { + cancellationToken.ThrowIfCancellationRequested(); + int frameIndex = firstFrameIndex + sampleIndex; + alphaOffset = stream.Length; + if (alphaUsesInterPrediction) + { + alphaEncoder.EncodeInterFrame(image.Frames[frameIndex], stream); + } + else + { + alphaEncoder.EncodeKeyFrame(image.Frames[frameIndex], stream); + } + + alphaSamples[sampleIndex] = new HeifSequenceSampleInfo( + alphaOffset, + checked((int)(stream.Length - alphaOffset)), + colorSamples[sampleIndex].Duration, + isSyncSample: !alphaUsesInterPrediction); + } + } + + alphaTrack = new HeifSequenceTrackEncoding( + new Av1CodecConfiguration(alphaHeader), + alphaSampleMemory, + true); + } + + ReadOnlyMemory iccProfileData = ReadOnlyMemory.Empty; + IccProfile? iccProfile = image.Metadata.IccProfile; + if (!this.encoder.SkipMetadata && iccProfile is not null) + { + iccProfileData = iccProfile.GetDataForWriting(); + } + + return new HeifSequenceEncoding( + image.Width, + image.Height, + this.encoder.RepeatCount ?? image.Metadata.GetHeifMetadata().RepeatCount, + timescale, + colorTrack, + alphaTrack, + settings.ColorProfile, + iccProfileData, + exifData, + tiffHeaderOffset, + xmpData); + } + + private int WriteSequenceFileTypeBox(Stream stream) + { + Span buffer = stackalloc byte[44]; + int bytesWritten = WriteBoxHeader(buffer, Heif4CharCode.Ftyp); + BinaryPrimitives.WriteUInt32BigEndian(buffer[bytesWritten..], (uint)Heif4CharCode.Avis); + bytesWritten += sizeof(uint); + BinaryPrimitives.WriteUInt32BigEndian(buffer[bytesWritten..], 0); + bytesWritten += sizeof(uint); + BinaryPrimitives.WriteUInt32BigEndian(buffer[bytesWritten..], (uint)Heif4CharCode.Avif); + bytesWritten += sizeof(uint); + BinaryPrimitives.WriteUInt32BigEndian(buffer[bytesWritten..], (uint)Heif4CharCode.Avio); + bytesWritten += sizeof(uint); + BinaryPrimitives.WriteUInt32BigEndian(buffer[bytesWritten..], (uint)Heif4CharCode.Avis); + bytesWritten += sizeof(uint); + BinaryPrimitives.WriteUInt32BigEndian(buffer[bytesWritten..], (uint)Heif4CharCode.Msf1); + bytesWritten += sizeof(uint); + BinaryPrimitives.WriteUInt32BigEndian(buffer[bytesWritten..], (uint)Heif4CharCode.Iso8); + bytesWritten += sizeof(uint); + BinaryPrimitives.WriteUInt32BigEndian(buffer[bytesWritten..], (uint)Heif4CharCode.Mif1); + bytesWritten += sizeof(uint); + BinaryPrimitives.WriteUInt32BigEndian(buffer[bytesWritten..], (uint)Heif4CharCode.Miaf); + bytesWritten += sizeof(uint); + BinaryPrimitives.WriteUInt32BigEndian(buffer, (uint)bytesWritten); + stream.Write(buffer[..bytesWritten]); + return bytesWritten; + } + + private void WriteSequenceMovieBox(HeifSequenceEncoding sequence, int precedingBoxLength, Stream stream) + { + int movieLength = GetSequenceMovieBoxLength(sequence); + using IMemoryOwner movieOwner = this.configuration.MemoryAllocator.Allocate(movieLength); + Span memory = movieOwner.Memory.Span[..movieLength]; + int offset = 0; + int movieStart = BeginSequenceBox(memory, ref offset, Heif4CharCode.Moov); + ulong mediaDuration = GetSequenceMediaDuration(sequence.ColorTrack.Samples); + ulong trackDuration = sequence.RepeatCount == 0 + ? ulong.MaxValue + : checked(mediaDuration * sequence.RepeatCount); + bool hasAlpha = sequence.AlphaTrack.HasValue; + + WriteSequenceMovieHeader( + memory, + ref offset, + sequence.Timescale, + trackDuration, + hasAlpha ? 3U : 2U); + int colorChunkOffsetPosition = WriteSequenceTrack( + memory, + ref offset, + sequence, + sequence.ColorTrack, + 1, + mediaDuration, + trackDuration); + + int alphaChunkOffsetPosition = -1; + long alphaPayloadOffset = 0; + if (hasAlpha) + { + HeifSequenceTrackEncoding alphaTrack = sequence.AlphaTrack.GetValueOrDefault(); + alphaPayloadOffset = alphaTrack.Samples[0].Offset; + alphaChunkOffsetPosition = WriteSequenceTrack( + memory, + ref offset, + sequence, + alphaTrack, + 2, + mediaDuration, + trackDuration); + } + + EndSequenceBox(memory, movieStart, offset); + ulong mediaDataOffset = checked((ulong)precedingBoxLength + (uint)offset + 8U); + BinaryPrimitives.WriteUInt64BigEndian( + memory[colorChunkOffsetPosition..], + checked(mediaDataOffset + (ulong)sequence.ColorTrack.Samples[0].Offset)); + + if (alphaChunkOffsetPosition >= 0) + { + BinaryPrimitives.WriteUInt64BigEndian( + memory[alphaChunkOffsetPosition..], + checked(mediaDataOffset + (ulong)alphaPayloadOffset)); + } + + stream.Write(memory); + } + + private static int GetSequenceMovieBoxLength(HeifSequenceEncoding sequence) + { + const int movieHeaderBoxLength = 120; + const int trackHeaderBoxLength = 104; + const int trackReferenceBoxLength = 20; + const int editListBoxLength = 44; + const int mediaBoxFixedLength = 129; + const int colorInformationBoxLength = 19; + const int codecConfigurationBoxLength = 12; + const int codingConstraintsBoxLength = 16; + const int visualSampleEntryLength = 86; + const int sampleDescriptionBoxLength = 16; + const int sampleTableBoxHeaderLength = 8; + const int timeToSampleBoxFixedLength = 16; + const int sampleToChunkBoxLength = 28; + const int sampleSizeBoxFixedLength = 20; + const int chunkOffsetBoxLength = 24; + const int syncSampleBoxFixedLength = 16; + const int timingRunLength = 8; + const int sampleSizeEntryLength = sizeof(uint); + const int syncSampleEntryLength = sizeof(uint); + const int sampleTableFixedLength = + sampleTableBoxHeaderLength + + sampleDescriptionBoxLength + + visualSampleEntryLength + + codecConfigurationBoxLength + + codingConstraintsBoxLength + + timeToSampleBoxFixedLength + + sampleToChunkBoxLength + + sampleSizeBoxFixedLength + + chunkOffsetBoxLength + + syncSampleBoxFixedLength; + + const int metadataFixedLength = 83; + const int metadataLocationLength = 16; + const int exifInformationLength = 25; + const int xmpInformationLength = 44; + const int exifOffsetLength = sizeof(uint); + + int repeatBoxLength = sequence.RepeatCount == 1 ? 0 : editListBoxLength; + int colorRunCount = GetSequenceTimingRunCount(sequence.ColorTrack.Samples); + int colorSyncSampleCount = GetSequenceSyncSampleCount(sequence.ColorTrack.Samples); + long colorSampleTableLength = + (long)sampleTableFixedLength + + (colorRunCount * timingRunLength) + + (sequence.ColorTrack.Samples.Length * sampleSizeEntryLength) + + (colorSyncSampleCount * syncSampleEntryLength) + + colorInformationBoxLength; + + if (!sequence.IccProfileData.IsEmpty) + { + colorSampleTableLength = colorSampleTableLength + + IccColorInformationPropertyBoxFixedLength + + sequence.IccProfileData.Length; + } + + byte[]? exifData = sequence.ExifData; + byte[]? xmpData = sequence.XmpData; + long metadataLength = 0; + if (exifData is not null || xmpData is not null) + { + int metadataItemCount = (exifData is not null ? 1 : 0) + (xmpData is not null ? 1 : 0); + metadataLength = (long)metadataFixedLength + + (metadataItemCount * metadataLocationLength) + + (exifData is not null ? (long)exifInformationLength + exifOffsetLength + exifData.Length : 0) + + (xmpData is not null ? (long)xmpInformationLength + xmpData.Length : 0); + } + + long colorTrackLength = + BasicBoxHeaderLength + + trackHeaderBoxLength + + repeatBoxLength + + metadataLength + + mediaBoxFixedLength + + colorSampleTableLength; + + long alphaTrackLength = 0; + if (sequence.AlphaTrack.HasValue) + { + HeifSequenceTrackEncoding alphaTrack = sequence.AlphaTrack.GetValueOrDefault(); + int alphaRunCount = GetSequenceTimingRunCount(alphaTrack.Samples); + int alphaSyncSampleCount = GetSequenceSyncSampleCount(alphaTrack.Samples); + int auxiliaryTypeBoxLength = + FullBoxHeaderLength + + Encoding.UTF8.GetByteCount(HeifConstants.AlphaAuxiliaryType) + + 1; + + long alphaSampleTableLength = + (long)sampleTableFixedLength + + (alphaRunCount * timingRunLength) + + (alphaTrack.Samples.Length * sampleSizeEntryLength) + + (alphaSyncSampleCount * syncSampleEntryLength) + + auxiliaryTypeBoxLength; + + alphaTrackLength = + BasicBoxHeaderLength + + trackHeaderBoxLength + + trackReferenceBoxLength + + repeatBoxLength + + mediaBoxFixedLength + + alphaSampleTableLength; + } + + // The movie box contains one header and one or two tracks. Every nested variable-length field above is + // resolved before this exact allocation, so container writing cannot re-rent or copy its buffer. + long movieLength = BasicBoxHeaderLength + movieHeaderBoxLength + colorTrackLength + alphaTrackLength; + return checked((int)movieLength); + } + + private static void WriteSequenceMovieHeader( + Span memory, + ref int offset, + uint timescale, + ulong duration, + uint nextTrackId) + { + int movieHeaderStart = BeginSequenceBox(memory, ref offset, Heif4CharCode.Mvhd); + WriteSequenceFullBoxHeader(memory, ref offset, 1, 0); + WriteSequenceUInt64(memory, ref offset, 0); + WriteSequenceUInt64(memory, ref offset, 0); + WriteSequenceUInt32(memory, ref offset, timescale); + WriteSequenceUInt64(memory, ref offset, duration); + WriteSequenceUInt32(memory, ref offset, UnityFixed16Point16); + WriteSequenceUInt16(memory, ref offset, UnityFixed8Point8); + WriteSequenceUInt16(memory, ref offset, 0); + WriteSequenceZeros(memory, ref offset, 2 * sizeof(uint)); + WriteSequenceIdentityMatrix(memory, ref offset); + WriteSequenceZeros(memory, ref offset, 6 * sizeof(uint)); + WriteSequenceUInt32(memory, ref offset, nextTrackId); + EndSequenceBox(memory, movieHeaderStart, offset); + } + + private static int WriteSequenceTrack( + Span memory, + ref int offset, + HeifSequenceEncoding sequence, + HeifSequenceTrackEncoding track, + uint trackId, + ulong mediaDuration, + ulong trackDuration) + { + int trackStart = BeginSequenceBox(memory, ref offset, Heif4CharCode.Trak); + WriteSequenceTrackHeader( + memory, + ref offset, + sequence.Width, + sequence.Height, + trackId, + trackDuration); + + if (track.IsAlpha) + { + WriteSequenceTrackReference(memory, ref offset, Heif4CharCode.Auxl, 1); + } + + if (sequence.RepeatCount != 1) + { + WriteSequenceEditList(memory, ref offset, mediaDuration); + } + + if (!track.IsAlpha && (sequence.ExifData is not null || sequence.XmpData is not null)) + { + WriteSequenceTrackMetadata(memory, ref offset, sequence); + } + + int mediaStart = BeginSequenceBox(memory, ref offset, Heif4CharCode.Mdia); + WriteSequenceMediaHeader(memory, ref offset, sequence.Timescale, mediaDuration); + WriteSequenceHandler(memory, ref offset, track.IsAlpha ? Heif4CharCode.Auxv : Heif4CharCode.Pict); + int mediaInformationStart = BeginSequenceBox(memory, ref offset, Heif4CharCode.Minf); + WriteSequenceDataInformation(memory, ref offset); + int chunkOffsetPosition = WriteSequenceSampleTable(memory, ref offset, sequence, track); + EndSequenceBox(memory, mediaInformationStart, offset); + EndSequenceBox(memory, mediaStart, offset); + EndSequenceBox(memory, trackStart, offset); + return chunkOffsetPosition; + } + + private static void WriteSequenceTrackHeader( + Span memory, + ref int offset, + int width, + int height, + uint trackId, + ulong duration) + { + int trackHeaderStart = BeginSequenceBox(memory, ref offset, Heif4CharCode.Tkhd); + WriteSequenceFullBoxHeader(memory, ref offset, 1, 1); + WriteSequenceUInt64(memory, ref offset, 0); + WriteSequenceUInt64(memory, ref offset, 0); + WriteSequenceUInt32(memory, ref offset, trackId); + WriteSequenceUInt32(memory, ref offset, 0); + WriteSequenceUInt64(memory, ref offset, duration); + WriteSequenceZeros(memory, ref offset, (2 * sizeof(uint)) + (4 * sizeof(ushort))); + WriteSequenceIdentityMatrix(memory, ref offset); + WriteSequenceUInt32(memory, ref offset, (uint)width << 16); + WriteSequenceUInt32(memory, ref offset, (uint)height << 16); + EndSequenceBox(memory, trackHeaderStart, offset); + } + + private static void WriteSequenceTrackReference( + Span memory, + ref int offset, + Heif4CharCode referenceType, + uint referencedTrackId) + { + int referencesStart = BeginSequenceBox(memory, ref offset, Heif4CharCode.Tref); + int referenceStart = BeginSequenceBox(memory, ref offset, referenceType); + WriteSequenceUInt32(memory, ref offset, referencedTrackId); + EndSequenceBox(memory, referenceStart, offset); + EndSequenceBox(memory, referencesStart, offset); + } + + private static void WriteSequenceEditList( + Span memory, + ref int offset, + ulong mediaDuration) + { + int editStart = BeginSequenceBox(memory, ref offset, Heif4CharCode.Edts); + int editListStart = BeginSequenceBox(memory, ref offset, Heif4CharCode.Elst); + WriteSequenceFullBoxHeader(memory, ref offset, 1, 1); + WriteSequenceUInt32(memory, ref offset, 1); + WriteSequenceUInt64(memory, ref offset, mediaDuration); + WriteSequenceUInt64(memory, ref offset, 0); + WriteSequenceUInt16(memory, ref offset, 1); + WriteSequenceUInt16(memory, ref offset, 0); + EndSequenceBox(memory, editListStart, offset); + EndSequenceBox(memory, editStart, offset); + } + + private static void WriteSequenceTrackMetadata( + Span memory, + ref int offset, + HeifSequenceEncoding sequence) + { + const byte fourByteOffsetAndLengthSizes = 0x44; + byte[]? exifData = sequence.ExifData; + byte[]? xmpData = sequence.XmpData; + ushort itemCount = (ushort)((exifData is not null ? 1 : 0) + (xmpData is not null ? 1 : 0)); + int metadataStart = BeginSequenceBox(memory, ref offset, Heif4CharCode.Meta); + WriteSequenceFullBoxHeader(memory, ref offset, 0, 0); + WriteSequenceHandler(memory, ref offset, Heif4CharCode.Pict); + + // Construction method one makes each extent relative to the local idat payload, keeping metadata independent + // of the final file and movie-box offsets. + int locationsStart = BeginSequenceBox(memory, ref offset, Heif4CharCode.Iloc); + WriteSequenceFullBoxHeader(memory, ref offset, 1, 0); + memory[offset++] = fourByteOffsetAndLengthSizes; + memory[offset++] = 0; + WriteSequenceUInt16(memory, ref offset, itemCount); + ushort itemId = 1; + uint itemDataOffset = 0; + if (exifData is not null) + { + uint exifLength = (uint)exifData.Length + sizeof(uint); + WriteSequenceTrackMetadataLocation(memory, ref offset, itemId++, itemDataOffset, exifLength); + itemDataOffset += exifLength; + } + + if (xmpData is not null) + { + WriteSequenceTrackMetadataLocation( + memory, + ref offset, + itemId, + itemDataOffset, + (uint)xmpData.Length); + } + + EndSequenceBox(memory, locationsStart, offset); + + int informationStart = BeginSequenceBox(memory, ref offset, Heif4CharCode.Iinf); + WriteSequenceFullBoxHeader(memory, ref offset, 0, 0); + WriteSequenceUInt16(memory, ref offset, itemCount); + itemId = 1; + if (exifData is not null) + { + WriteSequenceTrackMetadataItem(memory, ref offset, itemId++, Heif4CharCode.Exif); + } + + if (xmpData is not null) + { + WriteSequenceTrackMetadataItem(memory, ref offset, itemId, Heif4CharCode.Mime); + } + + EndSequenceBox(memory, informationStart, offset); + + int itemDataStart = BeginSequenceBox(memory, ref offset, Heif4CharCode.Idat); + if (exifData is not null) + { + WriteSequenceUInt32(memory, ref offset, sequence.ExifTiffHeaderOffset); + WriteSequenceBytes(memory, ref offset, exifData); + } + + if (xmpData is not null) + { + WriteSequenceBytes(memory, ref offset, xmpData); + } + + EndSequenceBox(memory, itemDataStart, offset); + EndSequenceBox(memory, metadataStart, offset); + } + + private static void WriteSequenceTrackMetadataLocation( + Span memory, + ref int offset, + ushort itemId, + uint itemDataOffset, + uint itemLength) + { + WriteSequenceUInt16(memory, ref offset, itemId); + WriteSequenceUInt16(memory, ref offset, 1); + WriteSequenceUInt16(memory, ref offset, 0); + WriteSequenceUInt16(memory, ref offset, 1); + WriteSequenceUInt32(memory, ref offset, itemDataOffset); + WriteSequenceUInt32(memory, ref offset, itemLength); + } + + private static void WriteSequenceTrackMetadataItem( + Span memory, + ref int offset, + ushort itemId, + Heif4CharCode itemType) + { + int itemStart = BeginSequenceBox(memory, ref offset, Heif4CharCode.Infe); + WriteSequenceFullBoxHeader(memory, ref offset, 2, 0); + WriteSequenceUInt16(memory, ref offset, itemId); + WriteSequenceUInt16(memory, ref offset, 0); + WriteSequenceUInt32(memory, ref offset, (uint)itemType); + ReadOnlySpan itemName = itemType == Heif4CharCode.Exif ? "Exif"u8 : "XMP"u8; + WriteSequenceBytes(memory, ref offset, itemName); + memory[offset++] = 0; + if (itemType == Heif4CharCode.Mime) + { + WriteSequenceBytes(memory, ref offset, "application/rdf+xml"u8); + memory[offset++] = 0; + } + + EndSequenceBox(memory, itemStart, offset); + } + + private static void WriteSequenceMediaHeader( + Span memory, + ref int offset, + uint timescale, + ulong mediaDuration) + { + int mediaHeaderStart = BeginSequenceBox(memory, ref offset, Heif4CharCode.Mdhd); + WriteSequenceFullBoxHeader(memory, ref offset, 1, 0); + WriteSequenceUInt64(memory, ref offset, 0); + WriteSequenceUInt64(memory, ref offset, 0); + WriteSequenceUInt32(memory, ref offset, timescale); + WriteSequenceUInt64(memory, ref offset, mediaDuration); + WriteSequenceUInt16(memory, ref offset, PackedUndeterminedLanguage); + WriteSequenceUInt16(memory, ref offset, 0); + EndSequenceBox(memory, mediaHeaderStart, offset); + } + + private static void WriteSequenceHandler( + Span memory, + ref int offset, + Heif4CharCode handlerType) + { + int handlerStart = BeginSequenceBox(memory, ref offset, Heif4CharCode.Hdlr); + WriteSequenceFullBoxHeader(memory, ref offset, 0, 0); + WriteSequenceUInt32(memory, ref offset, 0); + WriteSequenceUInt32(memory, ref offset, (uint)handlerType); + WriteSequenceZeros(memory, ref offset, 12); + memory[offset++] = 0; + EndSequenceBox(memory, handlerStart, offset); + } + + private static void WriteSequenceDataInformation(Span memory, ref int offset) + { + int dataInformationStart = BeginSequenceBox(memory, ref offset, Heif4CharCode.Dinf); + int dataReferenceStart = BeginSequenceBox(memory, ref offset, Heif4CharCode.Dref); + WriteSequenceFullBoxHeader(memory, ref offset, 0, 0); + WriteSequenceUInt32(memory, ref offset, 1); + int locationStart = BeginSequenceBox(memory, ref offset, Heif4CharCode.Url); + WriteSequenceFullBoxHeader(memory, ref offset, 0, 1); + EndSequenceBox(memory, locationStart, offset); + EndSequenceBox(memory, dataReferenceStart, offset); + EndSequenceBox(memory, dataInformationStart, offset); + } + + private static int WriteSequenceSampleTable( + Span memory, + ref int offset, + HeifSequenceEncoding sequence, + HeifSequenceTrackEncoding track) + { + int sampleTableStart = BeginSequenceBox(memory, ref offset, Heif4CharCode.Stbl); + WriteSequenceSampleDescription(memory, ref offset, sequence, track); + WriteSequenceSampleTiming(memory, ref offset, track.Samples); + + // Payloads are emitted contiguously per track, so one chunk maps directly to every sample in that track. + int sampleToChunkStart = BeginSequenceBox(memory, ref offset, Heif4CharCode.Stsc); + WriteSequenceFullBoxHeader(memory, ref offset, 0, 0); + WriteSequenceUInt32(memory, ref offset, 1); + WriteSequenceUInt32(memory, ref offset, 1); + WriteSequenceUInt32(memory, ref offset, (uint)track.Samples.Length); + WriteSequenceUInt32(memory, ref offset, 1); + EndSequenceBox(memory, sampleToChunkStart, offset); + + int sampleSizesStart = BeginSequenceBox(memory, ref offset, Heif4CharCode.Stsz); + WriteSequenceFullBoxHeader(memory, ref offset, 0, 0); + WriteSequenceUInt32(memory, ref offset, 0); + WriteSequenceUInt32(memory, ref offset, (uint)track.Samples.Length); + foreach (HeifSequenceSampleInfo sample in track.Samples) + { + WriteSequenceUInt32(memory, ref offset, (uint)sample.Length); + } + + EndSequenceBox(memory, sampleSizesStart, offset); + + int chunkOffsetsStart = BeginSequenceBox(memory, ref offset, Heif4CharCode.Co64); + WriteSequenceFullBoxHeader(memory, ref offset, 0, 0); + WriteSequenceUInt32(memory, ref offset, 1); + int chunkOffsetPosition = offset; + WriteSequenceUInt64(memory, ref offset, 0); + EndSequenceBox(memory, chunkOffsetsStart, offset); + + int syncSampleCount = GetSequenceSyncSampleCount(track.Samples); + int syncSamplesStart = BeginSequenceBox(memory, ref offset, Heif4CharCode.Stss); + WriteSequenceFullBoxHeader(memory, ref offset, 0, 0); + WriteSequenceUInt32(memory, ref offset, (uint)syncSampleCount); + uint sampleNumber = 1; + foreach (HeifSequenceSampleInfo sample in track.Samples) + { + if (sample.IsSyncSample) + { + WriteSequenceUInt32(memory, ref offset, sampleNumber); + } + + sampleNumber++; + } + + EndSequenceBox(memory, syncSamplesStart, offset); + EndSequenceBox(memory, sampleTableStart, offset); + return chunkOffsetPosition; + } + + private static void WriteSequenceSampleDescription( + Span memory, + ref int offset, + HeifSequenceEncoding sequence, + HeifSequenceTrackEncoding track) + { + int descriptionStart = BeginSequenceBox(memory, ref offset, Heif4CharCode.Stsd); + WriteSequenceFullBoxHeader(memory, ref offset, 0, 0); + WriteSequenceUInt32(memory, ref offset, 1); + int sampleEntryStart = BeginSequenceBox(memory, ref offset, Heif4CharCode.Av01); + WriteSequenceZeros(memory, ref offset, 6); + WriteSequenceUInt16(memory, ref offset, 1); + WriteSequenceZeros(memory, ref offset, (2 * sizeof(ushort)) + (3 * sizeof(uint))); + WriteSequenceUInt16(memory, ref offset, (ushort)sequence.Width); + WriteSequenceUInt16(memory, ref offset, (ushort)sequence.Height); + WriteSequenceUInt32(memory, ref offset, DefaultVisualSampleResolution); + WriteSequenceUInt32(memory, ref offset, DefaultVisualSampleResolution); + WriteSequenceUInt32(memory, ref offset, 0); + WriteSequenceUInt16(memory, ref offset, 1); + WriteSequenceZeros(memory, ref offset, VisualSampleCompressorNameLength); + WriteSequenceUInt16(memory, ref offset, VisualSampleDepth); + WriteSequenceUInt16(memory, ref offset, ushort.MaxValue); + + int configurationStart = BeginSequenceBox(memory, ref offset, Heif4CharCode.Av1C); + track.Configuration.WriteFixedHeader(memory.Slice(offset, Av1CodecConfiguration.FixedHeaderSize)); + offset += Av1CodecConfiguration.FixedHeaderSize; + EndSequenceBox(memory, configurationStart, offset); + + if (track.IsAlpha) + { + int auxiliaryTypeStart = BeginSequenceBox(memory, ref offset, Heif4CharCode.Auxi); + WriteSequenceFullBoxHeader(memory, ref offset, 0, 0); + int auxiliaryTypeLength = Encoding.UTF8.GetByteCount(HeifConstants.AlphaAuxiliaryType); + Span auxiliaryType = memory.Slice(offset, auxiliaryTypeLength + 1); + offset += Encoding.UTF8.GetBytes(HeifConstants.AlphaAuxiliaryType, auxiliaryType); + memory[offset++] = 0; + EndSequenceBox(memory, auxiliaryTypeStart, offset); + } + else + { + if (!sequence.IccProfileData.IsEmpty) + { + offset += WriteIccColorInformationPropertyBox(memory, offset, sequence.IccProfileData); + } + + offset += WriteColorInformationPropertyBox(memory, offset, sequence.ColorProfile); + } + + int codingConstraintsStart = BeginSequenceBox(memory, ref offset, Heif4CharCode.Ccst); + WriteSequenceFullBoxHeader(memory, ref offset, 0, 0); + + uint codingConstraints = IntraPicturePredictionUsedMask; + if (GetSequenceSyncSampleCount(track.Samples) == track.Samples.Length) + { + codingConstraints |= AllReferencePicturesIntraMask; + } + + // Sync samples are key frames in this encoder. The all-intra flag is therefore valid when every sample + // is independently decodable, including lossless sequences that deliberately avoid inter transforms. + WriteSequenceUInt32(memory, ref offset, codingConstraints); + EndSequenceBox(memory, codingConstraintsStart, offset); + EndSequenceBox(memory, sampleEntryStart, offset); + EndSequenceBox(memory, descriptionStart, offset); + } + + private static void WriteSequenceSampleTiming( + Span memory, + ref int offset, + ReadOnlySpan samples) + { + // The time-to-sample table stores runs, not one entry per frame. Preserve exact resolved durations while + // combining only adjacent frames whose delays are equal. + int runCount = GetSequenceTimingRunCount(samples); + + int timingStart = BeginSequenceBox(memory, ref offset, Heif4CharCode.Stts); + WriteSequenceFullBoxHeader(memory, ref offset, 0, 0); + WriteSequenceUInt32(memory, ref offset, (uint)runCount); + uint runDuration = samples[0].Duration; + uint runLength = 1; + for (int sampleIndex = 1; sampleIndex <= samples.Length; sampleIndex++) + { + if (sampleIndex < samples.Length && samples[sampleIndex].Duration == runDuration) + { + runLength++; + continue; + } + + WriteSequenceUInt32(memory, ref offset, runLength); + WriteSequenceUInt32(memory, ref offset, runDuration); + if (sampleIndex < samples.Length) + { + runDuration = samples[sampleIndex].Duration; + runLength = 1; + } + } + + EndSequenceBox(memory, timingStart, offset); + } + + private static int GetSequenceTimingRunCount(ReadOnlySpan samples) + { + int runCount = 1; + for (int sampleIndex = 1; sampleIndex < samples.Length; sampleIndex++) + { + runCount += samples[sampleIndex].Duration == samples[sampleIndex - 1].Duration ? 0 : 1; + } + + return runCount; + } + + private static int GetSequenceSyncSampleCount(ReadOnlySpan samples) + { + int count = 0; + foreach (HeifSequenceSampleInfo sample in samples) + { + if (sample.IsSyncSample) + { + count++; + } + } + + return count; + } + + private static uint GetSequenceSampleDuration(Rational delay, uint timescale) + { + // HEIF metadata uses either a zero numerator or a zero denominator for an unspecified duration. + // BMFF samples still require a finite positive duration, so encode the smallest representable value. + if (delay.Numerator == 0 || delay.Denominator == 0) + { + return 1; + } + + ulong scaledDuration = ((ulong)delay.Numerator * timescale) + (delay.Denominator / 2U); + return checked((uint)Math.Max(1UL, scaledDuration / delay.Denominator)); + } + + private static uint GetSequenceTimescale(Image image, int firstFrameIndex) + where TPixel : unmanaged, IPixel + { + uint timescale = DefaultSequenceTimescale; + for (int frameIndex = firstFrameIndex; frameIndex < image.Frames.Count; frameIndex++) + { + ImageFrame frame = image.Frames[frameIndex]; + Rational delay = frame.Metadata.GetHeifMetadata().FrameDelay; + if (delay.Numerator == 0 || delay.Denominator == 0) + { + continue; + } + + uint commonDivisor = GetGreatestCommonDivisor(timescale, delay.Denominator); + ulong commonTimescale = ((ulong)timescale / commonDivisor) * delay.Denominator; + if (commonTimescale > uint.MaxValue) + { + // A media timescale is a 32-bit field. Microsecond fallback retains bounded timing precision when + // the exact least common multiple of caller-provided rational delays cannot be represented. + return FallbackSequenceTimescale; + } + + timescale = (uint)commonTimescale; + } + + return timescale; + } + + private static uint GetGreatestCommonDivisor(uint left, uint right) + { + while (right != 0) + { + uint remainder = left % right; + left = right; + right = remainder; + } + + return left; + } + + private static ulong GetSequenceMediaDuration(ReadOnlySpan samples) + { + ulong duration = 0; + foreach (HeifSequenceSampleInfo sample in samples) + { + duration = checked(duration + sample.Duration); + } + + return duration; + } + + private static int BeginSequenceBox( + Span memory, + ref int offset, + Heif4CharCode type) + { + // Reserve the size field now and patch it at the matching EndSequenceBox call after nested boxes expand. + int start = offset; + offset += WriteBoxHeader(memory[offset..], type); + return start; + } + + private static void EndSequenceBox(Span memory, int start, int offset) + => BinaryPrimitives.WriteUInt32BigEndian( + memory.Slice(start, sizeof(uint)), + (uint)(offset - start)); + + private static void WriteSequenceFullBoxHeader( + Span memory, + ref int offset, + byte version, + uint flags) + { + Span destination = memory.Slice(offset, sizeof(uint)); + BinaryPrimitives.WriteUInt32BigEndian(destination, flags); + destination[0] = version; + offset += sizeof(uint); + } + + private static void WriteSequenceIdentityMatrix(Span memory, ref int offset) + { + WriteSequenceUInt32(memory, ref offset, UnityFixed16Point16); + WriteSequenceUInt32(memory, ref offset, 0); + WriteSequenceUInt32(memory, ref offset, 0); + WriteSequenceUInt32(memory, ref offset, 0); + WriteSequenceUInt32(memory, ref offset, UnityFixed16Point16); + WriteSequenceUInt32(memory, ref offset, 0); + WriteSequenceUInt32(memory, ref offset, 0); + WriteSequenceUInt32(memory, ref offset, 0); + WriteSequenceUInt32(memory, ref offset, UnityFixed2Point30); + } + + private static void WriteSequenceZeros(Span memory, ref int offset, int length) + { + memory.Slice(offset, length).Clear(); + offset += length; + } + + private static void WriteSequenceBytes( + Span memory, + ref int offset, + ReadOnlySpan source) + { + source.CopyTo(memory[offset..]); + offset += source.Length; + } + + private static void WriteSequenceUInt16(Span memory, ref int offset, ushort value) + { + BinaryPrimitives.WriteUInt16BigEndian(memory[offset..], value); + offset += sizeof(ushort); + } + + private static void WriteSequenceUInt32(Span memory, ref int offset, uint value) + { + BinaryPrimitives.WriteUInt32BigEndian(memory[offset..], value); + offset += sizeof(uint); + } + + private static void WriteSequenceUInt64(Span memory, ref int offset, ulong value) + { + BinaryPrimitives.WriteUInt64BigEndian(memory[offset..], value); + offset += sizeof(ulong); + } + + private readonly struct Av1EncodingSettings + { + public Av1EncodingSettings( + HeifBitDepth bitDepth, + HeifChromaSubsampling chromaSubsampling, + CicpProfile colorProfile, + ObuColorConfig colorConfig, + ObuColorConfig alphaConfig, + int colorQIndex, + int alphaQIndex, + bool hasAlpha) + { + this.BitDepth = bitDepth; + this.ChromaSubsampling = chromaSubsampling; + this.ColorProfile = colorProfile; + this.ColorConfig = colorConfig; + this.AlphaConfig = alphaConfig; + this.ColorQIndex = colorQIndex; + this.AlphaQIndex = alphaQIndex; + this.HasAlpha = hasAlpha; + } + + public HeifBitDepth BitDepth { get; } + + public HeifChromaSubsampling ChromaSubsampling { get; } + + public CicpProfile ColorProfile { get; } + + public ObuColorConfig ColorConfig { get; } + + public ObuColorConfig AlphaConfig { get; } + + public int ColorQIndex { get; } + + public int AlphaQIndex { get; } + + public bool HasAlpha { get; } + } + + private readonly struct HeifSequenceSampleInfo + { + public HeifSequenceSampleInfo(long offset, int length, uint duration, bool isSyncSample) + { + this.Offset = offset; + this.Length = length; + this.Duration = duration; + this.IsSyncSample = isSyncSample; + } + + public long Offset { get; } + + public int Length { get; } + + public uint Duration { get; } + + public bool IsSyncSample { get; } + } + + private readonly struct HeifSequenceEncoding + { + public HeifSequenceEncoding( + int width, + int height, + ushort repeatCount, + uint timescale, + HeifSequenceTrackEncoding colorTrack, + HeifSequenceTrackEncoding? alphaTrack, + CicpProfile colorProfile, + ReadOnlyMemory iccProfileData, + byte[]? exifData, + uint exifTiffHeaderOffset, + byte[]? xmpData) + { + this.Width = width; + this.Height = height; + this.RepeatCount = repeatCount; + this.Timescale = timescale; + this.ColorTrack = colorTrack; + this.AlphaTrack = alphaTrack; + this.ColorProfile = colorProfile; + this.IccProfileData = iccProfileData; + this.ExifData = exifData; + this.ExifTiffHeaderOffset = exifTiffHeaderOffset; + this.XmpData = xmpData; + } + + public int Width { get; } + + public int Height { get; } + + public ushort RepeatCount { get; } + + public uint Timescale { get; } + + public HeifSequenceTrackEncoding ColorTrack { get; } + + public HeifSequenceTrackEncoding? AlphaTrack { get; } + + public CicpProfile ColorProfile { get; } + + public ReadOnlyMemory IccProfileData { get; } + + public byte[]? ExifData { get; } + + public uint ExifTiffHeaderOffset { get; } + + public byte[]? XmpData { get; } + } + + private readonly struct HeifSequenceTrackEncoding + { + private readonly ReadOnlyMemory samples; + + public HeifSequenceTrackEncoding( + Av1CodecConfiguration configuration, + ReadOnlyMemory samples, + bool isAlpha) + { + this.Configuration = configuration; + this.samples = samples; + this.IsAlpha = isAlpha; + } + + public Av1CodecConfiguration Configuration { get; } + + public ReadOnlySpan Samples + => this.samples.Span; + + public bool IsAlpha { get; } + } +} diff --git a/src/ImageSharp/Formats/Heif/HeifEncoderCore.cs b/src/ImageSharp/Formats/Heif/HeifEncoderCore.cs new file mode 100644 index 0000000000..09b8cabfbb --- /dev/null +++ b/src/ImageSharp/Formats/Heif/HeifEncoderCore.cs @@ -0,0 +1,1618 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Buffers; +using System.Buffers.Binary; +using System.Text; +using SixLabors.ImageSharp.Formats.Heif.Av1; +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline; +using SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.Quantizers; +using SixLabors.ImageSharp.Formats.Jpeg; +using SixLabors.ImageSharp.IO; +using SixLabors.ImageSharp.Metadata; +using SixLabors.ImageSharp.Metadata.Profiles.Cicp; +using SixLabors.ImageSharp.Metadata.Profiles.Icc; +using SixLabors.ImageSharp.PixelFormats; + +namespace SixLabors.ImageSharp.Formats.Heif; + +/// +/// Image encoder for writing an image to a stream as a HEIF image. +/// +internal sealed partial class HeifEncoderCore +{ + // ISO BMFF box lengths include their size and type fields. Full boxes also include version and flags. + private const int BasicBoxHeaderLength = 8; + private const int FullBoxHeaderLength = 12; + private const int HandlerBoxLength = 33; + private const int PrimaryItemBoxLength = 14; + private const int ItemInformationBoxFixedLength = 14; + private const int ItemInformationEntryFixedLength = 21; + private const int ItemReferenceBoxFixedLength = 12; + private const int ItemReferenceEntryFixedLength = 12; + private const int ItemPropertiesBoxFixedLength = 32; + private const int PropertyAssociationEntryFixedLength = 3; + private const int ItemLocationBoxFixedLength = 16; + private const int ItemLocationEntryFixedLength = 8; + private const int ItemExtentLength = 12; + private const int SpatialExtentPropertyBoxLength = 20; + private const int PixelInformationPropertyBoxFixedLength = 13; + private const int Av1CodecConfigurationPropertyBoxLength = BasicBoxHeaderLength + Av1CodecConfiguration.FixedHeaderSize; + private const int AuxiliaryTypePropertyBoxFixedLength = 13; + private const int IccColorInformationPropertyBoxFixedLength = 12; + private const int CicpColorInformationPropertyBoxLength = 19; + private const int MaximumCompactPropertyIndex = 0x7F; + private const ushort EssentialPropertyFlag = 0x8000; + private const byte CompactEssentialPropertyFlag = 0x80; + private const uint HiddenImageItemFlag = 1; + + /// + /// The version defined for the AVIF grid item payload. + /// + private const byte GridDescriptorVersion = 0; + + /// + /// The largest row or column count representable by a grid descriptor. + /// + private const int MaximumGridAxisCellCount = byte.MaxValue + 1; + + /// + /// The minimum width and height permitted for the first cell of an AVIF grid. + /// + private const int MinimumGridCellDimension = 64; + + /// + /// The grid descriptor length when output dimensions use 32-bit fields. + /// + private const int LongGridDescriptorLength = 12; + + /// + /// Selects 32-bit output dimensions in a grid descriptor. + /// + private const byte LargeGridDimensionsFlag = 1; + + /// + /// The global configuration. + /// + private readonly Configuration configuration; + + /// + /// The encoder with options. + /// + private readonly HeifEncoder encoder; + + /// + /// Initializes a new instance of the class. + /// + /// The configuration. + /// The encoder with options. + public HeifEncoderCore(Configuration configuration, HeifEncoder encoder) + { + this.configuration = configuration; + this.encoder = encoder; + } + + /// + /// Encodes the image to the specified stream from the . + /// + /// The pixel format. + /// The to encode from. + /// The to encode the image data to. + /// The token to request cancellation. + public void Encode(Image image, Stream stream, CancellationToken cancellationToken) + where TPixel : unmanaged, IPixel + { + Guard.NotNull(image, nameof(image)); + Guard.NotNull(stream, nameof(stream)); + + switch (this.encoder.CompressionMethod) + { + case HeifCompressionMethod.LegacyJpeg: + break; + case HeifCompressionMethod.Av1: + if (image.Frames.Count > 1) + { + if (image.Width > ushort.MaxValue || image.Height > ushort.MaxValue) + { + throw new NotSupportedException("AV1 image-sequence dimensions cannot exceed 65535 pixels."); + } + } + + break; + default: + throw new NotSupportedException($"HEIF compression method '{this.encoder.CompressionMethod}' is not supported."); + } + + using ChunkedMemoryStream compressedPixels = new(this.configuration.MemoryAllocator); + if (this.encoder.CompressionMethod == HeifCompressionMethod.Av1 && image.Frames.Count > 1) + { + Av1EncodingSettings settings = this.ResolveAv1Encoding(image); + bool animateRootFrame = this.encoder.AnimateRootFrame + ?? image.Metadata.GetHeifMetadata().AnimateRootFrame; + + int firstFrameIndex = animateRootFrame ? 0 : 1; + int sequenceFrameCount = image.Frames.Count - firstFrameIndex; + int sampleCount = sequenceFrameCount * (settings.HasAlpha ? 2 : 1); + using IMemoryOwner samplesOwner = + this.configuration.MemoryAllocator.Allocate(sampleCount); + + List sequenceItems = new(); + List sequenceLinks = new(); + if (!animateRootFrame) + { + Av1ImageItemEncoding primaryImage = this.CompressAv1ImageItem( + image.Frames.RootFrame, + compressedPixels, + settings, + cancellationToken); + + this.WriteAv1ImageItems( + image, + compressedPixels, + settings, + primaryImage, + sequenceItems, + sequenceLinks); + } + + Memory samples = samplesOwner.Memory[..sampleCount]; + HeifSequenceEncoding sequence = this.CompressAv1Sequence( + image, + compressedPixels, + settings, + samples, + firstFrameIndex, + cancellationToken); + + if (animateRootFrame) + { + HeifSequenceSampleInfo colorSample = sequence.ColorTrack.Samples[0]; + HeifSequenceTrackEncoding? alphaTrack = sequence.AlphaTrack; + Av1ImageItemEncoding primaryImage = new( + sequence.ColorTrack.Configuration, + colorSample.Offset, + colorSample.Length, + alphaTrack?.Configuration, + alphaTrack?.Samples[0].Offset ?? 0, + alphaTrack?.Samples[0].Length ?? 0); + + // The primary image item and the first track sample describe the same sync sample. Sharing its + // extent matches libavif and avoids encoding or storing the root frame twice. + this.WriteAv1ImageItems( + image, + compressedPixels, + settings, + primaryImage, + sequenceItems, + sequenceLinks); + } + + int fileTypeLength = this.WriteSequenceFileTypeBox(stream); + int metadataLength = GetMetadataBoxLength(sequenceItems, sequenceLinks); + int movieLength = GetSequenceMovieBoxLength(sequence); + this.WriteMetadataBox(sequenceItems, sequenceLinks, fileTypeLength, movieLength, stream); + this.WriteSequenceMovieBox(sequence, fileTypeLength + metadataLength, stream); + this.WriteMediaDataBox(compressedPixels, stream); + stream.Flush(); + return; + } + + List items = new(); + List links = new(); + switch (this.encoder.CompressionMethod) + { + case HeifCompressionMethod.LegacyJpeg: + this.CompressPixels(image, compressedPixels, cancellationToken); + GenerateLegacyJpegItem(image, compressedPixels.Length, items); + break; + case HeifCompressionMethod.Av1: + this.CompressAv1Pixels(image, compressedPixels, items, links, cancellationToken); + break; + } + + // Write out the generated header and pixels. + long metadataBoxOffset = this.WriteFileTypeBox(stream); + this.WriteMetadataBox(items, links, metadataBoxOffset, 0, stream); + this.WriteMediaDataBox(compressedPixels, stream); + stream.Flush(); + } + + /// + /// Builds the item declarations and relationships for the encoded image payload. + /// + /// The source pixel format. + /// The source image. + /// The encoded primary-item payload length. + /// The destination item collection. + private static void GenerateLegacyJpegItem(Image image, long pixelDataLength, List items) + where TPixel : unmanaged, IPixel + { + HeifItem primaryItem = new(Heif4CharCode.Jpeg, 1u); + primaryItem.DataLocations.Add(new HeifLocation(HeifLocationOffsetOrigin.FileOffset, 0L, 0L, pixelDataLength)); + primaryItem.BitsPerPixel = 24; + primaryItem.ChannelCount = 3; + primaryItem.SetExtent(image.Size); + items.Add(primaryItem); + + // No item relationship is emitted until the writer has a distinct derived image, + // thumbnail, auxiliary image, or metadata item to reference. + } + + /// + /// Writes an eight-byte ISO BMFF basic box header with a placeholder size. + /// + /// The destination beginning at the box size field. + /// The box four-character code. + /// The number of header bytes written. + private static int WriteBoxHeader(Span buffer, Heif4CharCode type) + { + int bytesWritten = 0; + BinaryPrimitives.WriteUInt32BigEndian(buffer[bytesWritten..], 8U); + bytesWritten += 4; + BinaryPrimitives.WriteUInt32BigEndian(buffer[bytesWritten..], (uint)type); + bytesWritten += 4; + + return bytesWritten; + } + + /// + /// Writes a 12-byte ISO BMFF full-box header with a placeholder size. + /// + /// The destination beginning at the box size field. + /// The box four-character code. + /// The full-box syntax version. + /// The 24-bit full-box flags value. + /// The number of header bytes written. + private static int WriteBoxHeader(Span buffer, Heif4CharCode type, byte version, uint flags) + { + int bytesWritten = 0; + BinaryPrimitives.WriteUInt32BigEndian(buffer[bytesWritten..], 12); + bytesWritten += 4; + BinaryPrimitives.WriteUInt32BigEndian(buffer[bytesWritten..], (uint)type); + bytesWritten += 4; + + // Writing the 24-bit flags as a big-endian 32-bit value establishes the three flag bytes, after which the + // version overwrites the leading byte to form the full-box version-and-flags word. + BinaryPrimitives.WriteUInt32BigEndian(buffer[bytesWritten..], flags); + buffer[bytesWritten] = version; + bytesWritten += 4; + + return bytesWritten; + } + + /// + /// Writes the major brand, minor version, and compatible brands for the current HEIF output. + /// + /// The destination stream. + /// The number of bytes written. + private int WriteFileTypeBox(Stream stream) + { + Span buffer = stackalloc byte[28]; + int bytesWritten = WriteBoxHeader(buffer, Heif4CharCode.Ftyp); + Heif4CharCode majorBrand = this.encoder.CompressionMethod == HeifCompressionMethod.Av1 + ? Heif4CharCode.Avif + : Heif4CharCode.Mif1; + + BinaryPrimitives.WriteUInt32BigEndian(buffer[bytesWritten..], (uint)majorBrand); + bytesWritten += 4; + BinaryPrimitives.WriteUInt32BigEndian(buffer[bytesWritten..], 0); + bytesWritten += 4; + if (majorBrand == Heif4CharCode.Avif) + { + // A still AVIF is also a MIAF image collection, so advertise both structural brands with the codec brand. + BinaryPrimitives.WriteUInt32BigEndian(buffer[bytesWritten..], (uint)Heif4CharCode.Avif); + bytesWritten += 4; + BinaryPrimitives.WriteUInt32BigEndian(buffer[bytesWritten..], (uint)Heif4CharCode.Mif1); + bytesWritten += 4; + BinaryPrimitives.WriteUInt32BigEndian(buffer[bytesWritten..], (uint)Heif4CharCode.Miaf); + bytesWritten += 4; + } + + BinaryPrimitives.WriteUInt32BigEndian(buffer, (uint)bytesWritten); + stream.Write(buffer[..bytesWritten]); + + return bytesWritten; + } + + /// + /// Writes the metadata box containing item declarations, relationships, properties, and file locations. + /// + /// The declared image and metadata items. + /// The typed relationships between items. + /// The metadata box offset from the start of the encoded file. + /// The number of bytes between this box and the media-data box. + /// The destination stream positioned after the file-type box. + private void WriteMetadataBox( + List items, + List links, + long metadataBoxOffset, + int followingBoxLength, + Stream stream) + { + int metadataLength = GetMetadataBoxLength(items, links); + using IMemoryOwner metadataOwner = this.configuration.MemoryAllocator.Allocate(metadataLength); + Span memory = metadataOwner.Memory.Span[..metadataLength]; + Span buffer = memory[..FullBoxHeaderLength]; + int bytesWritten = WriteBoxHeader(buffer, Heif4CharCode.Meta, 0, 0); + bytesWritten += WriteHandlerBox(memory, bytesWritten); + bytesWritten += WritePrimaryItemBox(memory, bytesWritten); + bytesWritten += WriteItemInfoBox(memory, bytesWritten, items); + if (links.Count > 0) + { + // iref is optional and has no meaning without at least one typed item relationship. + bytesWritten += WriteItemReferenceBox(memory, bytesWritten, links); + } + + bytesWritten += WriteItemPropertiesBox(memory, bytesWritten, items); + + // iloc needs the absolute mdat payload position, but that position depends on the final meta length. Emit it + // once to establish the stable box size, calculate the following mdat position, then patch the same bytes. + int itemLocationOffset = bytesWritten; + bytesWritten += WriteItemLocationBox(memory, bytesWritten, items, 0); + + // The mdat payload immediately follows the completed meta box and its own eight-byte header. + long mediaDataOffset = checked(metadataBoxOffset + bytesWritten + followingBoxLength + BasicBoxHeaderLength); + WriteItemLocationBox(memory, itemLocationOffset, items, mediaDataOffset); + + buffer = memory[..bytesWritten]; + BinaryPrimitives.WriteUInt32BigEndian(buffer, (uint)bytesWritten); + stream.Write(buffer); + } + + private static int GetMetadataBoxLength(List items, List links) + { + // All variable-length strings, profiles, relationships, properties, and extents are resolved before + // allocating the metadata box, so writing it never needs to re-rent or copy a backing buffer. + return checked( + FullBoxHeaderLength + + HandlerBoxLength + + PrimaryItemBoxLength + + GetItemInformationBoxLength(items) + + (links.Count == 0 ? 0 : GetItemReferenceBoxLength(links)) + + GetItemPropertiesBoxLength(items) + + GetItemLocationBoxLength(items)); + } + + private static int GetItemInformationBoxLength(List items) + { + long length = ItemInformationBoxFixedLength; + foreach (HeifItem item in items) + { + length += ItemInformationEntryFixedLength + Encoding.UTF8.GetByteCount(item.Name ?? string.Empty); + if (item.Type == Heif4CharCode.Mime) + { + length += 1 + Encoding.UTF8.GetByteCount(item.ContentType ?? string.Empty); + if (item.ContentEncoding is not null) + { + length += 1 + Encoding.UTF8.GetByteCount(item.ContentEncoding); + } + } + } + + return checked((int)length); + } + + private static int GetItemReferenceBoxLength(List links) + { + long length = ItemReferenceBoxFixedLength; + foreach (HeifItemLink link in links) + { + length += ItemReferenceEntryFixedLength + ((long)link.DestinationIds.Count * sizeof(ushort)); + } + + return checked((int)length); + } + + /// + /// Gets the exact number of bytes required for the item-properties box. + /// + /// The items whose properties and associations are counted. + /// The complete item-properties-box length. + public static int GetItemPropertiesBoxLength(List items) + { + long propertyCount = 0; + long associationItemCount = 0; + long associationPropertyCount = 0; + long propertyBytes = 0; + foreach (HeifItem item in items) + { + HeifItem propertyItem = item.PropertySource ?? item; + int itemPropertyCount = GetPropertyCount(propertyItem); + associationItemCount += itemPropertyCount == 0 ? 0 : 1; + associationPropertyCount += itemPropertyCount; + + if (item.PropertySource is not null) + { + continue; + } + + propertyCount += itemPropertyCount; + propertyBytes += item.Extent == default ? 0 : SpatialExtentPropertyBoxLength; + if (item.ChannelBitDepths is not null) + { + propertyBytes += PixelInformationPropertyBoxFixedLength + item.ChannelBitDepths.Length; + } + else if (item.UniformChannelBitDepth is not null) + { + propertyBytes += PixelInformationPropertyBoxFixedLength + item.ChannelCount; + } + + propertyBytes += item.Av1CodecConfiguration is null + ? 0 + : Av1CodecConfigurationPropertyBoxLength; + propertyBytes += item.AuxiliaryType is null + ? 0 + : AuxiliaryTypePropertyBoxFixedLength + Encoding.UTF8.GetByteCount(item.AuxiliaryType); + propertyBytes += item.IccProfile is null + ? 0 + : IccColorInformationPropertyBoxFixedLength + item.GetIccProfileDataForWriting().Length; + propertyBytes += item.CicpProfile is null ? 0 : CicpColorInformationPropertyBoxLength; + } + + int associationSize = propertyCount > MaximumCompactPropertyIndex ? sizeof(ushort) : sizeof(byte); + long length = ItemPropertiesBoxFixedLength + + propertyBytes + + (associationItemCount * PropertyAssociationEntryFixedLength) + + (associationPropertyCount * associationSize); + + return checked((int)length); + } + + private static int GetItemLocationBoxLength(List items) + { + long extentCount = 0; + foreach (HeifItem item in items) + { + extentCount += item.DataLocations.Count; + } + + long length = + ItemLocationBoxFixedLength + + ((long)items.Count * ItemLocationEntryFixedLength) + + (extentCount * ItemExtentLength); + + return checked((int)length); + } + + /// + /// Writes the picture metadata handler box. + /// + /// The preallocated metadata buffer. + /// The destination offset within the metadata box. + /// The complete handler-box length. + private static int WriteHandlerBox(Span memory, int memoryOffset) + { + Span buffer = memory.Slice(memoryOffset, HandlerBoxLength); + int bytesWritten = WriteBoxHeader(buffer, Heif4CharCode.Hdlr, 0, 0); + BinaryPrimitives.WriteUInt32BigEndian(buffer[bytesWritten..], 0); + bytesWritten += 4; + BinaryPrimitives.WriteUInt32BigEndian(buffer[bytesWritten..], (uint)Heif4CharCode.Pict); + bytesWritten += 4; + for (int i = 0; i < 13; i++) + { + buffer[bytesWritten++] = 0; + } + + BinaryPrimitives.WriteUInt32BigEndian(buffer, (uint)bytesWritten); + return bytesWritten; + } + + /// + /// Writes the identifier of the primary presentation item. + /// + /// The preallocated metadata buffer. + /// The destination offset within the metadata box. + /// The complete primary-item-box length. + private static int WritePrimaryItemBox(Span memory, int memoryOffset) + { + Span buffer = memory.Slice(memoryOffset, PrimaryItemBoxLength); + int bytesWritten = WriteBoxHeader(buffer, Heif4CharCode.Pitm, 0, 0); + BinaryPrimitives.WriteUInt16BigEndian(buffer[bytesWritten..], 1); + bytesWritten += 2; + + BinaryPrimitives.WriteUInt32BigEndian(buffer, (uint)bytesWritten); + return bytesWritten; + } + + /// + /// Writes the item-information box and one version-two entry for each item. + /// + /// The preallocated metadata buffer. + /// The destination offset within the metadata box. + /// The items to declare. + /// The complete item-information-box length. + private static int WriteItemInfoBox(Span memory, int memoryOffset, List items) + { + Span buffer = memory.Slice(memoryOffset, GetItemInformationBoxLength(items)); + int bytesWritten = WriteBoxHeader(buffer, Heif4CharCode.Iinf, 0, 0); + BinaryPrimitives.WriteUInt16BigEndian(buffer[bytesWritten..], (ushort)items.Count); + bytesWritten += 2; + foreach (HeifItem item in items) + { + int itemLengthOffset = bytesWritten; + bytesWritten += WriteBoxHeader( + buffer[bytesWritten..], + Heif4CharCode.Infe, + 2, + item.IsHidden ? HiddenImageItemFlag : 0); + + BinaryPrimitives.WriteUInt16BigEndian(buffer[bytesWritten..], (ushort)item.Id); + bytesWritten += 2; + BinaryPrimitives.WriteUInt16BigEndian(buffer[bytesWritten..], 0); + bytesWritten += 2; + BinaryPrimitives.WriteUInt32BigEndian(buffer[bytesWritten..], (uint)item.Type); + bytesWritten += 4; + bytesWritten += Encoding.UTF8.GetBytes(item.Name ?? string.Empty, buffer[bytesWritten..]); + buffer[bytesWritten++] = 0; + if (item.Type == Heif4CharCode.Mime) + { + bytesWritten += Encoding.UTF8.GetBytes(item.ContentType ?? string.Empty, buffer[bytesWritten..]); + buffer[bytesWritten++] = 0; + if (item.ContentEncoding is not null) + { + bytesWritten += Encoding.UTF8.GetBytes(item.ContentEncoding, buffer[bytesWritten..]); + buffer[bytesWritten++] = 0; + } + } + + BinaryPrimitives.WriteUInt32BigEndian(buffer[itemLengthOffset..], (uint)(bytesWritten - itemLengthOffset)); + } + + BinaryPrimitives.WriteUInt32BigEndian(buffer, (uint)bytesWritten); + return bytesWritten; + } + + /// + /// Writes typed item-reference child boxes using 16-bit item identifiers. + /// + /// The preallocated metadata buffer. + /// The destination offset within the metadata box. + /// The relationships to write. + /// The complete item-reference-box length. + private static int WriteItemReferenceBox(Span memory, int memoryOffset, List links) + { + Span buffer = memory.Slice(memoryOffset, GetItemReferenceBoxLength(links)); + int bytesWritten = WriteBoxHeader(buffer, Heif4CharCode.Iref, 0, 0); + foreach (HeifItemLink link in links) + { + int itemLengthOffset = bytesWritten; + bytesWritten += WriteBoxHeader(buffer[bytesWritten..], link.Type); + BinaryPrimitives.WriteUInt16BigEndian(buffer[bytesWritten..], (ushort)link.SourceId); + bytesWritten += 2; + BinaryPrimitives.WriteUInt16BigEndian(buffer[bytesWritten..], (ushort)link.DestinationIds.Count); + bytesWritten += 2; + foreach (uint destId in link.DestinationIds) + { + BinaryPrimitives.WriteUInt16BigEndian(buffer[bytesWritten..], (ushort)destId); + bytesWritten += 2; + } + + BinaryPrimitives.WriteUInt32BigEndian(buffer[itemLengthOffset..], (uint)(bytesWritten - itemLengthOffset)); + } + + BinaryPrimitives.WriteUInt32BigEndian(buffer, (uint)bytesWritten); + return bytesWritten; + } + + /// + /// Writes spatial-extent properties and their one-based item associations. + /// + /// The preallocated metadata buffer. + /// The destination offset within the metadata box. + /// The items whose dimensions are written and associated. + /// The complete item-properties-box length. + public static int WriteItemPropertiesBox(Span memory, int memoryOffset, List items) + { + Span buffer = memory.Slice(memoryOffset, GetItemPropertiesBoxLength(items)); + int bytesWritten = WriteBoxHeader(buffer, Heif4CharCode.Iprp); + + // ipco order defines the one-based property indices written later in ipma. + int ipcoLengthOffset = bytesWritten; + bytesWritten += WriteBoxHeader(buffer[bytesWritten..], Heif4CharCode.Ipco); + ushort nextPropertyIndex = 1; + foreach (HeifItem item in items) + { + if (item.PropertySource is not null) + { + continue; + } + + item.FirstPropertyIndex = nextPropertyIndex; + if (item.Extent != default) + { + bytesWritten += WriteSpatialExtentPropertyBox(memory, memoryOffset + bytesWritten, item); + nextPropertyIndex++; + } + + byte[]? channelBitDepths = item.ChannelBitDepths; + if (channelBitDepths is not null) + { + bytesWritten += WritePixelInformationPropertyBox(memory, memoryOffset + bytesWritten, channelBitDepths); + nextPropertyIndex++; + } + else + { + byte? uniformChannelBitDepth = item.UniformChannelBitDepth; + if (uniformChannelBitDepth is not null) + { + bytesWritten += WritePixelInformationPropertyBox( + memory, + memoryOffset + bytesWritten, + item.ChannelCount, + uniformChannelBitDepth.Value); + + nextPropertyIndex++; + } + } + + Av1CodecConfiguration? codecConfiguration = item.Av1CodecConfiguration; + if (codecConfiguration is not null) + { + bytesWritten += WriteAv1CodecConfigurationPropertyBox(memory, memoryOffset + bytesWritten, codecConfiguration); + nextPropertyIndex++; + } + + string? auxiliaryType = item.AuxiliaryType; + if (auxiliaryType is not null) + { + bytesWritten += WriteAuxiliaryTypePropertyBox(memory, memoryOffset + bytesWritten, auxiliaryType); + nextPropertyIndex++; + } + + IccProfile? iccProfile = item.IccProfile; + if (iccProfile is not null) + { + bytesWritten += WriteIccColorInformationPropertyBox(memory, memoryOffset + bytesWritten, item.GetIccProfileDataForWriting()); + nextPropertyIndex++; + } + + CicpProfile? cicpProfile = item.CicpProfile; + if (cicpProfile is not null) + { + bytesWritten += WriteColorInformationPropertyBox(memory, memoryOffset + bytesWritten, cicpProfile); + nextPropertyIndex++; + } + } + + BinaryPrimitives.WriteUInt32BigEndian(buffer[ipcoLengthOffset..], (uint)(bytesWritten - ipcoLengthOffset)); + int propertyCount = nextPropertyIndex - 1; + int associationItemCount = 0; + foreach (HeifItem item in items) + { + HeifItem propertyItem = item.PropertySource ?? item; + int itemPropertyCount = GetPropertyCount(propertyItem); + if (itemPropertyCount == 0) + { + continue; + } + + associationItemCount++; + } + + bool largePropertyIndex = propertyCount > MaximumCompactPropertyIndex; + + // ipma uses a 15-bit index only when the property table cannot fit in the compact seven-bit form. + int ipmaLengthOffset = bytesWritten; + bytesWritten += WriteBoxHeader(buffer[bytesWritten..], Heif4CharCode.Ipma, 0, largePropertyIndex ? 1U : 0U); + BinaryPrimitives.WriteUInt32BigEndian(buffer[bytesWritten..], (uint)associationItemCount); + bytesWritten += 4; + foreach (HeifItem item in items) + { + HeifItem propertyItem = item.PropertySource ?? item; + int itemPropertyCount = GetPropertyCount(propertyItem); + if (itemPropertyCount == 0) + { + continue; + } + + BinaryPrimitives.WriteUInt16BigEndian(buffer[bytesWritten..], (ushort)item.Id); + bytesWritten += 2; + + buffer[bytesWritten++] = (byte)itemPropertyCount; + ushort propertyIndex = propertyItem.FirstPropertyIndex; + if (propertyItem.Extent != default) + { + WritePropertyAssociation(buffer, ref bytesWritten, propertyIndex++, largePropertyIndex, false); + } + + if (propertyItem.ChannelBitDepths is not null || propertyItem.UniformChannelBitDepth is not null) + { + WritePropertyAssociation(buffer, ref bytesWritten, propertyIndex++, largePropertyIndex, false); + } + + if (propertyItem.Av1CodecConfiguration is not null) + { + WritePropertyAssociation(buffer, ref bytesWritten, propertyIndex++, largePropertyIndex, true); + } + + if (propertyItem.AuxiliaryType is not null) + { + WritePropertyAssociation(buffer, ref bytesWritten, propertyIndex++, largePropertyIndex, false); + } + + if (propertyItem.IccProfile is not null) + { + WritePropertyAssociation(buffer, ref bytesWritten, propertyIndex++, largePropertyIndex, false); + } + + if (propertyItem.CicpProfile is not null) + { + WritePropertyAssociation(buffer, ref bytesWritten, propertyIndex++, largePropertyIndex, false); + } + } + + BinaryPrimitives.WriteUInt32BigEndian(buffer[ipmaLengthOffset..], (uint)(bytesWritten - ipmaLengthOffset)); + + // Update size of enclosing 'iprp' box. + BinaryPrimitives.WriteUInt32BigEndian(buffer, (uint)bytesWritten); + return bytesWritten; + } + + /// + /// Gets the number of properties emitted for an item. + /// + /// The item whose populated properties are counted. + /// The number of emitted properties. + private static int GetPropertyCount(HeifItem item) + { + int count = item.Extent != default ? 1 : 0; + count += item.ChannelBitDepths is not null || item.UniformChannelBitDepth is not null ? 1 : 0; + count += item.Av1CodecConfiguration is not null ? 1 : 0; + count += item.AuxiliaryType is not null ? 1 : 0; + count += item.IccProfile is not null ? 1 : 0; + count += item.CicpProfile is not null ? 1 : 0; + return count; + } + + /// + /// Writes one compact or extended property association. + /// + /// The item-property-association destination. + /// The current destination offset, advanced past the association. + /// The one-based property index. + /// Whether the association uses a 15-bit property index. + /// Whether decoding the item requires understanding this property. + private static void WritePropertyAssociation( + Span buffer, + ref int offset, + ushort propertyIndex, + bool largePropertyIndex, + bool essential) + { + if (largePropertyIndex) + { + ushort association = essential ? (ushort)(propertyIndex | EssentialPropertyFlag) : propertyIndex; + BinaryPrimitives.WriteUInt16BigEndian(buffer[offset..], association); + offset += 2; + } + else + { + buffer[offset++] = essential ? (byte)(propertyIndex | CompactEssentialPropertyFlag) : (byte)propertyIndex; + } + } + + /// + /// Writes the encoded precision of each image channel. + /// + /// The preallocated metadata buffer. + /// The destination offset within the property container. + /// The encoded precision of each channel. + /// The complete pixel-information-box length. + private static int WritePixelInformationPropertyBox( + Span memory, + int memoryOffset, + ReadOnlySpan channelBitDepths) + { + Span buffer = memory.Slice(memoryOffset, PixelInformationPropertyBoxFixedLength + channelBitDepths.Length); + int bytesWritten = WriteBoxHeader(buffer, Heif4CharCode.Pixi, 0, 0); + buffer[bytesWritten++] = (byte)channelBitDepths.Length; + channelBitDepths.CopyTo(buffer[bytesWritten..]); + bytesWritten += channelBitDepths.Length; + + BinaryPrimitives.WriteUInt32BigEndian(buffer, (uint)bytesWritten); + return bytesWritten; + } + + /// + /// Writes one common encoded precision for every image channel. + /// + /// The preallocated metadata buffer. + /// The destination offset within the property container. + /// The number of encoded image channels. + /// The common encoded precision. + /// The complete pixel-information-box length. + private static int WritePixelInformationPropertyBox( + Span memory, + int memoryOffset, + int channelCount, + byte channelBitDepth) + { + Span buffer = memory.Slice(memoryOffset, PixelInformationPropertyBoxFixedLength + channelCount); + int bytesWritten = WriteBoxHeader(buffer, Heif4CharCode.Pixi, 0, 0); + buffer[bytesWritten++] = (byte)channelCount; + buffer.Slice(bytesWritten, channelCount).Fill(channelBitDepth); + bytesWritten += channelCount; + + BinaryPrimitives.WriteUInt32BigEndian(buffer, (uint)bytesWritten); + return bytesWritten; + } + + /// + /// Writes an AV1 codec-configuration property. + /// + /// The preallocated metadata buffer. + /// The destination offset within the property container. + /// The fixed image configuration. + /// The complete AV1 codec-configuration-box length. + private static int WriteAv1CodecConfigurationPropertyBox( + Span memory, + int memoryOffset, + Av1CodecConfiguration configuration) + { + Span buffer = memory.Slice(memoryOffset, Av1CodecConfigurationPropertyBoxLength); + int bytesWritten = WriteBoxHeader(buffer, Heif4CharCode.Av1C); + configuration.WriteFixedHeader(buffer.Slice(bytesWritten, Av1CodecConfiguration.FixedHeaderSize)); + bytesWritten += Av1CodecConfiguration.FixedHeaderSize; + + BinaryPrimitives.WriteUInt32BigEndian(buffer, (uint)bytesWritten); + return bytesWritten; + } + + /// + /// Writes the registered type of an auxiliary image item. + /// + /// The preallocated metadata buffer. + /// The destination offset within the property container. + /// The null-terminated registered auxiliary type. + /// The complete auxiliary-type-box length. + private static int WriteAuxiliaryTypePropertyBox( + Span memory, + int memoryOffset, + string auxiliaryType) + { + int auxiliaryTypeLength = Encoding.UTF8.GetByteCount(auxiliaryType); + Span buffer = memory.Slice(memoryOffset, AuxiliaryTypePropertyBoxFixedLength + auxiliaryTypeLength); + int bytesWritten = WriteBoxHeader(buffer, Heif4CharCode.AuxC, 0, 0); + bytesWritten += Encoding.UTF8.GetBytes(auxiliaryType, buffer[bytesWritten..]); + buffer[bytesWritten++] = 0; + + BinaryPrimitives.WriteUInt32BigEndian(buffer, (uint)bytesWritten); + return bytesWritten; + } + + /// + /// Writes an unrestricted ICC color profile for a color image item. + /// + /// The preallocated metadata buffer. + /// The destination offset within the property container. + /// The serialized ICC profile to write. + /// The complete color-information-box length. + private static int WriteIccColorInformationPropertyBox( + Span memory, + int memoryOffset, + ReadOnlyMemory profileData) + { + Span buffer = memory.Slice(memoryOffset, IccColorInformationPropertyBoxFixedLength + profileData.Length); + int bytesWritten = WriteBoxHeader(buffer, Heif4CharCode.Colr); + BinaryPrimitives.WriteUInt32BigEndian(buffer[bytesWritten..], (uint)Heif4CharCode.Prof); + bytesWritten += 4; + profileData.Span.CopyTo(buffer[bytesWritten..]); + bytesWritten += profileData.Length; + + BinaryPrimitives.WriteUInt32BigEndian(buffer, (uint)bytesWritten); + return bytesWritten; + } + + /// + /// Writes an H.273 color description for a color image item. + /// + /// The preallocated metadata buffer. + /// The destination offset within the property container. + /// The color description to write. + /// The complete color-information-box length. + private static int WriteColorInformationPropertyBox( + Span memory, + int memoryOffset, + CicpProfile profile) + { + Span buffer = memory.Slice(memoryOffset, CicpColorInformationPropertyBoxLength); + int bytesWritten = WriteBoxHeader(buffer, Heif4CharCode.Colr); + BinaryPrimitives.WriteUInt32BigEndian(buffer[bytesWritten..], (uint)Heif4CharCode.Nclx); + bytesWritten += 4; + BinaryPrimitives.WriteUInt16BigEndian(buffer[bytesWritten..], (ushort)profile.ColorPrimaries); + bytesWritten += 2; + BinaryPrimitives.WriteUInt16BigEndian(buffer[bytesWritten..], (ushort)profile.TransferCharacteristics); + bytesWritten += 2; + BinaryPrimitives.WriteUInt16BigEndian(buffer[bytesWritten..], (ushort)profile.MatrixCoefficients); + bytesWritten += 2; + buffer[bytesWritten++] = profile.FullRange ? (byte)0x80 : (byte)0; + + BinaryPrimitives.WriteUInt32BigEndian(buffer, (uint)bytesWritten); + return bytesWritten; + } + + /// + /// Writes an item's display width and height as an image-spatial-extents property. + /// + /// The preallocated metadata buffer. + /// The destination offset within the property container. + /// The item whose extent is written. + /// The complete image-spatial-extents-box length. + private static int WriteSpatialExtentPropertyBox(Span memory, int memoryOffset, HeifItem item) + { + Span buffer = memory.Slice(memoryOffset, SpatialExtentPropertyBoxLength); + int bytesWritten = WriteBoxHeader(buffer, Heif4CharCode.Ispe, 0, 0); + BinaryPrimitives.WriteUInt32BigEndian(buffer[bytesWritten..], (uint)item.Extent.Width); + bytesWritten += 4; + BinaryPrimitives.WriteUInt32BigEndian(buffer[bytesWritten..], (uint)item.Extent.Height); + bytesWritten += 4; + + BinaryPrimitives.WriteUInt32BigEndian(buffer, (uint)bytesWritten); + return bytesWritten; + } + + /// + /// Writes version-one file-relative locations for every ordered item extent. + /// + /// The preallocated metadata buffer. + /// The destination offset within the metadata box. + /// The items and relative payload extents to locate. + /// The absolute stream offset of the media-data payload. + /// The complete item-location-box length. + private static int WriteItemLocationBox(Span memory, int memoryOffset, List items, long mediaDataOffset) + { + Span buffer = memory.Slice(memoryOffset, GetItemLocationBoxLength(items)); + int bytesWritten = WriteBoxHeader(buffer, Heif4CharCode.Iloc, 1, 0); + + // The high and low nibbles select eight-byte offsets and four-byte lengths. Base offsets and extent indices + // are omitted, because every generated extent is written as one absolute file offset into mdat. + buffer[bytesWritten++] = 0x84; + buffer[bytesWritten++] = 0; + BinaryPrimitives.WriteUInt16BigEndian(buffer[bytesWritten..], (ushort)items.Count); + bytesWritten += 2; + foreach (HeifItem item in items) + { + BinaryPrimitives.WriteUInt16BigEndian(buffer[bytesWritten..], (ushort)item.Id); + bytesWritten += 2; + + // Version 1 stores twelve reserved bits followed by the four-bit construction method. + BinaryPrimitives.WriteUInt16BigEndian(buffer[bytesWritten..], (ushort)HeifLocationOffsetOrigin.FileOffset); + bytesWritten += 2; + BinaryPrimitives.WriteUInt16BigEndian(buffer[bytesWritten..], 0); + bytesWritten += 2; + BinaryPrimitives.WriteUInt16BigEndian(buffer[bytesWritten..], (ushort)item.DataLocations.Count); + bytesWritten += 2; + foreach (HeifLocation loc in item.DataLocations) + { + // Generated locations are relative to the mdat payload until the enclosing meta size is known. + long absoluteOffset = checked(mediaDataOffset + loc.BaseOffset + loc.Offset); + BinaryPrimitives.WriteUInt64BigEndian(buffer[bytesWritten..], (ulong)absoluteOffset); + bytesWritten += 8; + BinaryPrimitives.WriteUInt32BigEndian(buffer[bytesWritten..], (uint)loc.Length); + bytesWritten += 4; + } + } + + BinaryPrimitives.WriteUInt32BigEndian(buffer, (uint)bytesWritten); + return bytesWritten; + } + + /// + /// Writes the encoded primary-item bytes in a media-data box. + /// + /// The encoded item payload stream. + /// The destination stream. + private void WriteMediaDataBox(ChunkedMemoryStream data, Stream stream) + { + Span buf = stackalloc byte[12]; + int bytesWritten = WriteBoxHeader(buf, Heif4CharCode.Mdat); + BinaryPrimitives.WriteUInt32BigEndian(buf, checked((uint)(data.Length + bytesWritten))); + stream.Write(buf[..bytesWritten]); + + data.WriteTo(stream); + } + + /// + /// Maps the public lossy quality scale through libaom's external quantizer scale to its internal quantizer index. + /// + /// The lossy quality in the inclusive range zero through one hundred. + /// The AV1 quantizer index. + public static int GetAv1QuantizerIndex(int quality) + { + int scaledQuality = (100 - quality) * 63; + int quantizer = (scaledQuality + 50) / 100; + + // External quantizer zero maps to the codec's lossless qindex. Keep quality 100 lossy as its public contract requires. + quantizer = Math.Max(quantizer, 1); + return Av1QuantizationLookup.GetQIndex(quantizer); + } + + /// + /// Encodes the source root frame into AV1 color and optional auxiliary-alpha item payloads. + /// + /// The source pixel format. + /// The source image. + /// The shared destination for consecutive item payloads. + /// The destination item declarations. + /// The destination item relationships. + /// The token used to cancel payload encoding. + private void CompressAv1Pixels( + Image image, + ChunkedMemoryStream stream, + List items, + List links, + CancellationToken cancellationToken) + where TPixel : unmanaged, IPixel + { + Av1EncodingSettings settings = this.ResolveAv1Encoding(image); + if (image.Width > Av1Constants.MaxFrameDimension || image.Height > Av1Constants.MaxFrameDimension) + { + this.CompressAv1GridPixels(image, stream, settings, items, links, cancellationToken); + return; + } + + Av1ImageItemEncoding encoding = this.CompressAv1ImageItem( + image.Frames.RootFrame, + stream, + settings, + cancellationToken); + + this.WriteAv1ImageItems(image, stream, settings, encoding, items, links); + } + + /// + /// Encodes a still image as independently coded AV1 cells referenced by one derived grid item. + /// + private void CompressAv1GridPixels( + Image image, + ChunkedMemoryStream stream, + Av1EncodingSettings settings, + List items, + List links, + CancellationToken cancellationToken) + where TPixel : unmanaged, IPixel + { + bool isSubsampledX = !settings.ColorConfig.IsMonochrome && settings.ColorConfig.SubSamplingX; + bool isSubsampledY = !settings.ColorConfig.IsMonochrome && settings.ColorConfig.SubSamplingY; + int columns = GetGridCellCount(image.Width, Av1Constants.MaxFrameDimension); + int rows = GetGridCellCount(image.Height, Av1Constants.MaxFrameDimension); + if (columns > MaximumGridAxisCellCount || rows > MaximumGridAxisCellCount) + { + throw new NotSupportedException( + $"AVIF grids support at most {MaximumGridAxisCellCount} columns and rows."); + } + + int cellWidth = GetGridCellSize(image.Width, columns, isSubsampledX); + int cellHeight = GetGridCellSize(image.Height, rows, isSubsampledY); + Size encodedCellSize = new( + Math.Max(cellWidth, MinimumGridCellDimension), + Math.Max(cellHeight, MinimumGridCellDimension)); + + long cellCount = (long)columns * rows; + long itemCount = 1 + cellCount; + if (settings.HasAlpha) + { + itemCount += 1 + cellCount; + } + + if (!this.encoder.SkipMetadata) + { + itemCount += image.Metadata.ExifProfile is null ? 0 : 1; + itemCount += image.Metadata.XmpProfile is null ? 0 : 1; + } + + if (itemCount > ushort.MaxValue) + { + throw new NotSupportedException( + $"The encoded AVIF grid requires {itemCount} items, but this container supports at most {ushort.MaxValue}."); + } + + byte channelBitDepth = (byte)settings.BitDepth; + long descriptorOffset = stream.Length; + int descriptorLength = WriteGridDescriptor(stream, rows, columns, image.Size); + HeifItem colorGrid = new(Heif4CharCode.Grid, 1) + { + ChannelCount = settings.ColorConfig.IsMonochrome ? 1 : 3, + UniformChannelBitDepth = channelBitDepth, + BitsPerPixel = channelBitDepth * (settings.ColorConfig.IsMonochrome ? 1 : 3), + IccProfile = this.encoder.SkipMetadata ? null : image.Metadata.IccProfile, + CicpProfile = settings.ColorProfile + }; + + colorGrid.DataLocations.Add( + new HeifLocation( + HeifLocationOffsetOrigin.FileOffset, + 0L, + descriptorOffset, + descriptorLength)); + + colorGrid.SetExtent(image.Size); + items.Add(colorGrid); + HeifItemLink colorGridLink = new(Heif4CharCode.Dimg, colorGrid.Id); + links.Add(colorGridLink); + HeifItem? colorPropertySource = null; + ImageFrame rootFrame = image.Frames.RootFrame; + for (int row = 0; row < rows; row++) + { + int y = row * cellHeight; + int height = Math.Min(cellHeight, image.Height - y); + for (int column = 0; column < columns; column++) + { + cancellationToken.ThrowIfCancellationRequested(); + int x = column * cellWidth; + int width = Math.Min(cellWidth, image.Width - x); + Rectangle sourceRectangle = new(x, y, width, height); + long colorOffset = stream.Length; + ObuSequenceHeader colorHeader = Av1FrameEncoder.EncodeGridCell( + this.configuration, + rootFrame, + sourceRectangle, + encodedCellSize, + stream, + settings.ColorConfig, + settings.ColorQIndex, + this.encoder.Effort); + + long colorLength = stream.Length - colorOffset; + HeifItem colorCell = new(Heif4CharCode.Av01, (uint)items.Count + 1) + { + IsHidden = true, + ChannelCount = colorGrid.ChannelCount, + UniformChannelBitDepth = channelBitDepth, + BitsPerPixel = colorGrid.BitsPerPixel, + Av1CodecConfiguration = new Av1CodecConfiguration(colorHeader), + IccProfile = colorGrid.IccProfile, + CicpProfile = settings.ColorProfile + }; + + colorCell.DataLocations.Add( + new HeifLocation( + HeifLocationOffsetOrigin.FileOffset, + 0L, + colorOffset, + colorLength)); + + colorCell.SetExtent(encodedCellSize); + ShareGridCellProperties(colorCell, ref colorPropertySource); + + items.Add(colorCell); + colorGridLink.DestinationIds.Add(colorCell.Id); + } + } + + if (settings.HasAlpha) + { + descriptorOffset = stream.Length; + descriptorLength = WriteGridDescriptor(stream, rows, columns, image.Size); + HeifItem alphaGrid = new(Heif4CharCode.Grid, (uint)items.Count + 1) + { + ChannelCount = 1, + UniformChannelBitDepth = channelBitDepth, + BitsPerPixel = channelBitDepth, + AuxiliaryType = HeifConstants.AlphaAuxiliaryType + }; + + alphaGrid.DataLocations.Add( + new HeifLocation( + HeifLocationOffsetOrigin.FileOffset, + 0L, + descriptorOffset, + descriptorLength)); + + alphaGrid.SetExtent(image.Size); + items.Add(alphaGrid); + HeifItemLink alphaGridLink = new(Heif4CharCode.Dimg, alphaGrid.Id); + links.Add(alphaGridLink); + HeifItemLink alphaLink = new(Heif4CharCode.Auxl, alphaGrid.Id); + alphaLink.DestinationIds.Add(colorGrid.Id); + links.Add(alphaLink); + HeifItem? alphaPropertySource = null; + for (int row = 0; row < rows; row++) + { + int y = row * cellHeight; + int height = Math.Min(cellHeight, image.Height - y); + for (int column = 0; column < columns; column++) + { + cancellationToken.ThrowIfCancellationRequested(); + int x = column * cellWidth; + int width = Math.Min(cellWidth, image.Width - x); + Rectangle sourceRectangle = new(x, y, width, height); + long alphaOffset = stream.Length; + ObuSequenceHeader alphaHeader = Av1FrameEncoder.EncodeAlphaGridCell( + this.configuration, + rootFrame, + sourceRectangle, + encodedCellSize, + stream, + settings.AlphaConfig, + settings.AlphaQIndex, + this.encoder.Effort); + + long alphaLength = stream.Length - alphaOffset; + HeifItem alphaCell = new(Heif4CharCode.Av01, (uint)items.Count + 1) + { + IsHidden = true, + ChannelCount = 1, + UniformChannelBitDepth = channelBitDepth, + BitsPerPixel = channelBitDepth, + Av1CodecConfiguration = new Av1CodecConfiguration(alphaHeader), + AuxiliaryType = HeifConstants.AlphaAuxiliaryType + }; + + alphaCell.DataLocations.Add( + new HeifLocation( + HeifLocationOffsetOrigin.FileOffset, + 0L, + alphaOffset, + alphaLength)); + + alphaCell.SetExtent(encodedCellSize); + ShareGridCellProperties(alphaCell, ref alphaPropertySource); + + items.Add(alphaCell); + alphaGridLink.DestinationIds.Add(alphaCell.Id); + } + } + } + + this.WriteMetadataItems(image, stream, colorGrid, items, links); + } + + /// + /// Gets the minimum number of independently coded cells needed along one grid axis. + /// + /// The complete output dimension along the axis. + /// The largest permitted nominal cell dimension. + private static int GetGridCellCount(int dimension, int maximumCellDimension) + => (int)(((long)dimension + maximumCellDimension - 1) / maximumCellDimension); + + /// + /// Gets the nominal cell size while preserving chroma alignment for every non-edge cell. + /// + private static int GetGridCellSize(int dimension, int cellCount, bool isSubsampled) + { + int cellSize = (int)(((long)dimension + cellCount - 1) / cellCount); + if (isSubsampled && (cellSize & 1) != 0) + { + cellSize++; + } + + return cellSize; + } + + /// + /// Writes the fixed grid item payload and returns its exact length. + /// + private static int WriteGridDescriptor(Stream stream, int rows, int columns, Size outputSize) + { + bool usesLargeDimensions = outputSize.Width > ushort.MaxValue || outputSize.Height > ushort.MaxValue; + Span descriptor = stackalloc byte[LongGridDescriptorLength]; + int descriptorLength = 0; + descriptor[descriptorLength++] = GridDescriptorVersion; + descriptor[descriptorLength++] = usesLargeDimensions ? LargeGridDimensionsFlag : (byte)0; + descriptor[descriptorLength++] = (byte)(rows - 1); + descriptor[descriptorLength++] = (byte)(columns - 1); + if (usesLargeDimensions) + { + BinaryPrimitives.WriteUInt32BigEndian(descriptor[descriptorLength..], (uint)outputSize.Width); + descriptorLength += sizeof(uint); + BinaryPrimitives.WriteUInt32BigEndian(descriptor[descriptorLength..], (uint)outputSize.Height); + descriptorLength += sizeof(uint); + } + else + { + BinaryPrimitives.WriteUInt16BigEndian(descriptor[descriptorLength..], (ushort)outputSize.Width); + descriptorLength += sizeof(ushort); + BinaryPrimitives.WriteUInt16BigEndian(descriptor[descriptorLength..], (ushort)outputSize.Height); + descriptorLength += sizeof(ushort); + } + + stream.Write(descriptor[..descriptorLength]); + return descriptorLength; + } + + /// + /// Reuses the common property set emitted for the first cell in one grid plane. + /// + private static void ShareGridCellProperties(HeifItem item, ref HeifItem? source) + { + if (source is null) + { + source = item; + return; + } + + // Every cell in one plane is coded to the same extent and configuration so current AVIF readers can + // share one property set. Only the source rectangle differs for cells clipped by the output canvas. + item.PropertySource = source; + } + + /// + /// Encodes one frame as the color and optional alpha payloads used by a primary AV1 image item. + /// + private Av1ImageItemEncoding CompressAv1ImageItem( + ImageFrame frame, + ChunkedMemoryStream stream, + Av1EncodingSettings settings, + CancellationToken cancellationToken) + where TPixel : unmanaged, IPixel + { + cancellationToken.ThrowIfCancellationRequested(); + long colorOffset = stream.Length; + ObuSequenceHeader colorHeader = Av1FrameEncoder.Encode( + this.configuration, + frame, + stream, + settings.ColorConfig, + settings.ColorQIndex, + this.encoder.Effort); + + long colorLength = stream.Length - colorOffset; + Av1CodecConfiguration? alphaConfiguration = null; + long alphaOffset = 0; + long alphaLength = 0; + + if (settings.HasAlpha) + { + cancellationToken.ThrowIfCancellationRequested(); + alphaOffset = stream.Length; + ObuSequenceHeader alphaHeader = Av1FrameEncoder.EncodeAlpha( + this.configuration, + frame, + stream, + settings.AlphaConfig, + settings.AlphaQIndex, + this.encoder.Effort); + + alphaLength = stream.Length - alphaOffset; + alphaConfiguration = new Av1CodecConfiguration(alphaHeader); + } + + return new Av1ImageItemEncoding( + new Av1CodecConfiguration(colorHeader), + colorOffset, + colorLength, + alphaConfiguration, + alphaOffset, + alphaLength); + } + + /// + /// Declares a primary AV1 image item over existing payload extents and appends its associated metadata payloads. + /// + private void WriteAv1ImageItems( + Image image, + ChunkedMemoryStream stream, + Av1EncodingSettings settings, + Av1ImageItemEncoding encoding, + List items, + List links) + where TPixel : unmanaged, IPixel + { + byte channelBitDepth = (byte)settings.BitDepth; + HeifItem colorItem = new(Heif4CharCode.Av01, 1) + { + ChannelCount = settings.ColorConfig.IsMonochrome ? 1 : 3, + UniformChannelBitDepth = channelBitDepth, + BitsPerPixel = channelBitDepth * (settings.ColorConfig.IsMonochrome ? 1 : 3), + Av1CodecConfiguration = encoding.ColorConfiguration, + IccProfile = this.encoder.SkipMetadata ? null : image.Metadata.IccProfile, + CicpProfile = settings.ColorProfile + }; + + colorItem.DataLocations.Add( + new HeifLocation( + HeifLocationOffsetOrigin.FileOffset, + 0L, + encoding.ColorOffset, + encoding.ColorLength)); + + colorItem.SetExtent(image.Size); + items.Add(colorItem); + + Av1CodecConfiguration? alphaConfiguration = encoding.AlphaConfiguration; + if (alphaConfiguration is not null) + { + HeifItem alphaItem = new(Heif4CharCode.Av01, 2) + { + ChannelCount = 1, + UniformChannelBitDepth = channelBitDepth, + BitsPerPixel = channelBitDepth, + Av1CodecConfiguration = alphaConfiguration, + AuxiliaryType = HeifConstants.AlphaAuxiliaryType + }; + + alphaItem.DataLocations.Add( + new HeifLocation( + HeifLocationOffsetOrigin.FileOffset, + 0L, + encoding.AlphaOffset, + encoding.AlphaLength)); + + alphaItem.SetExtent(image.Size); + items.Add(alphaItem); + HeifItemLink alphaLink = new(Heif4CharCode.Auxl, alphaItem.Id); + alphaLink.DestinationIds.Add(colorItem.Id); + links.Add(alphaLink); + } + + this.WriteMetadataItems(image, stream, colorItem, items, links); + } + + /// + /// Appends Exif and XMP payload items associated with the primary presentation item. + /// + private void WriteMetadataItems( + Image image, + ChunkedMemoryStream stream, + HeifItem primaryItem, + List items, + List links) + where TPixel : unmanaged, IPixel + { + if (this.encoder.SkipMetadata) + { + return; + } + + byte[]? exifData = GetExifData(image.Metadata, out uint tiffHeaderOffset); + if (exifData is not null) + { + long exifOffset = stream.Length; + Span offsetBuffer = stackalloc byte[sizeof(uint)]; + BinaryPrimitives.WriteUInt32BigEndian(offsetBuffer, tiffHeaderOffset); + stream.Write(offsetBuffer); + stream.Write(exifData); + + HeifItem exifItem = new(Heif4CharCode.Exif, (uint)items.Count + 1) + { + Name = "Exif" + }; + + exifItem.DataLocations.Add( + new HeifLocation( + HeifLocationOffsetOrigin.FileOffset, + 0L, + exifOffset, + sizeof(uint) + (long)exifData.Length)); + + items.Add(exifItem); + HeifItemLink exifLink = new(Heif4CharCode.Cdsc, exifItem.Id); + exifLink.DestinationIds.Add(primaryItem.Id); + links.Add(exifLink); + } + + byte[]? xmpData = image.Metadata.XmpProfile?.Data; + if (xmpData is not null && xmpData.Length > 0) + { + long xmpOffset = stream.Length; + stream.Write(xmpData); + HeifItem xmpItem = new(Heif4CharCode.Mime, (uint)items.Count + 1) + { + Name = "XMP", + ContentType = "application/rdf+xml" + }; + + xmpItem.DataLocations.Add( + new HeifLocation( + HeifLocationOffsetOrigin.FileOffset, + 0L, + xmpOffset, + xmpData.Length)); + + items.Add(xmpItem); + HeifItemLink xmpLink = new(Heif4CharCode.Cdsc, xmpItem.Id); + xmpLink.DestinationIds.Add(primaryItem.Id); + links.Add(xmpLink); + } + } + + /// + /// Materializes the caller's Exif profile once and locates the TIFF header addressed by HEIF's four-byte prefix. + /// + /// The source image metadata. + /// The byte offset of the TIFF header within the returned profile. + /// The serialized profile, or when the source has no Exif payload. + private static byte[]? GetExifData(ImageMetadata metadata, out uint tiffHeaderOffset) + { + byte[]? exifData = metadata.ExifProfile?.ToByteArray(); + if (exifData is null || exifData.Length == 0) + { + tiffHeaderOffset = 0; + return null; + } + + // A directly supplied profile can retain the optional Exif identifier before its TIFF byte-order marker. + for (int i = 0; i <= exifData.Length - 4; i++) + { + bool isBigEndianTiff = exifData[i] == (byte)'M' + && exifData[i + 1] == (byte)'M' + && exifData[i + 2] == 0 + && exifData[i + 3] == 42; + + bool isLittleEndianTiff = exifData[i] == (byte)'I' + && exifData[i + 1] == (byte)'I' + && exifData[i + 2] == 42 + && exifData[i + 3] == 0; + + if (isBigEndianTiff || isLittleEndianTiff) + { + tiffHeaderOffset = (uint)i; + return exifData; + } + } + + throw new ImageFormatException("The Exif profile does not contain a TIFF header."); + } + + /// + /// Encodes the source pixels as the current legacy JPEG item payload. + /// + /// The source pixel format. + /// The source image. + /// The destination for the encoded JPEG item bytes. + /// The token used to cancel payload encoding. + private void CompressPixels( + Image image, + ChunkedMemoryStream stream, + CancellationToken cancellationToken) + where TPixel : unmanaged, IPixel + { + JpegColorType colorType = this.encoder.ChromaSubsampling switch + { + null or HeifChromaSubsampling.Yuv420 => JpegColorType.YCbCrRatio420, + HeifChromaSubsampling.Yuv422 => JpegColorType.YCbCrRatio422, + HeifChromaSubsampling.Yuv444 => JpegColorType.YCbCrRatio444, + HeifChromaSubsampling.Monochrome => JpegColorType.Luminance, + _ => throw new NotSupportedException($"HEIF chroma sampling '{this.encoder.ChromaSubsampling}' is not supported.") + }; + + JpegEncoder encoder = new() + { + // The HEIF quality scale includes zero while the JPEG payload encoder starts at one. + // Map the lowest HEIF setting to the lowest representable JPEG setting. + Quality = this.encoder.Quality == 0 ? 1 : this.encoder.Quality, + ColorType = colorType, + SkipMetadata = this.encoder.SkipMetadata + }; + + // ImageEncoder is a synchronous contract. Wait for the cancellable JPEG operation so HEIF encoding + // cannot return while its pooled item payload is still being produced. + image.SaveAsJpegAsync(stream, encoder, cancellationToken).GetAwaiter().GetResult(); + } + + /// + /// Describes the already-written color and optional alpha extents backing one AV1 image item. + /// + private readonly struct Av1ImageItemEncoding + { + public Av1ImageItemEncoding( + Av1CodecConfiguration colorConfiguration, + long colorOffset, + long colorLength, + Av1CodecConfiguration? alphaConfiguration, + long alphaOffset, + long alphaLength) + { + this.ColorConfiguration = colorConfiguration; + this.ColorOffset = colorOffset; + this.ColorLength = colorLength; + this.AlphaConfiguration = alphaConfiguration; + this.AlphaOffset = alphaOffset; + this.AlphaLength = alphaLength; + } + + public Av1CodecConfiguration ColorConfiguration { get; } + + public long ColorOffset { get; } + + public long ColorLength { get; } + + public Av1CodecConfiguration? AlphaConfiguration { get; } + + public long AlphaOffset { get; } + + public long AlphaLength { get; } + } +} diff --git a/src/ImageSharp/Formats/Heif/HeifFileType.cs b/src/ImageSharp/Formats/Heif/HeifFileType.cs new file mode 100644 index 0000000000..3031e30fa7 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/HeifFileType.cs @@ -0,0 +1,25 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif; + +/// +/// Identifies the supported image presentation declared by a HEIF file-type box. +/// +internal enum HeifFileType +{ + /// + /// The file-type box does not declare a supported HEIF image presentation. + /// + Unsupported, + + /// + /// The container presents a primary image item. + /// + StillImage, + + /// + /// The container presents a timed AVIF image sequence. + /// + ImageSequence +} diff --git a/src/ImageSharp/Formats/Heif/HeifFormat.cs b/src/ImageSharp/Formats/Heif/HeifFormat.cs new file mode 100644 index 0000000000..55e57207ec --- /dev/null +++ b/src/ImageSharp/Formats/Heif/HeifFormat.cs @@ -0,0 +1,40 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif; + +/// +/// Represents the HEIF image format. +/// +public sealed class HeifFormat : IImageFormat +{ + /// + /// Initializes a new instance of the class. + /// + private HeifFormat() + { + } + + /// + /// Gets the shared instance. + /// + public static HeifFormat Instance { get; } = new(); + + /// + public string Name => "HEIF"; + + /// + public string DefaultMimeType => "image/heif"; + + /// + public IEnumerable MimeTypes => HeifConstants.MimeTypes; + + /// + public IEnumerable FileExtensions => HeifConstants.FileExtensions; + + /// + public HeifMetadata CreateDefaultFormatMetadata() => new(); + + /// + public HeifFrameMetadata CreateDefaultFormatFrameMetadata() => new(); +} diff --git a/src/ImageSharp/Formats/Heif/HeifFrameMetadata.cs b/src/ImageSharp/Formats/Heif/HeifFrameMetadata.cs new file mode 100644 index 0000000000..48294eb6d4 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/HeifFrameMetadata.cs @@ -0,0 +1,71 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Numerics; +using SixLabors.ImageSharp.PixelFormats; + +namespace SixLabors.ImageSharp.Formats.Heif; + +/// +/// Provides HEIF-specific metadata for an image frame. +/// +public class HeifFrameMetadata : IFormatFrameMetadata +{ + /// + /// Initializes a new instance of the class. + /// + public HeifFrameMetadata() + { + } + + /// + /// Initializes a new instance of the class by copying another instance. + /// + /// The metadata to copy. + private HeifFrameMetadata(HeifFrameMetadata other) => this.FrameDelay = other.FrameDelay; + + /// + /// Gets or sets the frame display duration in seconds. The numerator contains duration units and the denominator + /// contains units per second. Zero indicates that no explicit duration is available. + /// + public Rational FrameDelay { get; set; } = new(0); + + /// + public static HeifFrameMetadata FromFormatConnectingFrameMetadata(FormatConnectingFrameMetadata metadata) + => new() + { + FrameDelay = new Rational(metadata.Duration.TotalSeconds) + }; + + /// + public FormatConnectingFrameMetadata ToFormatConnectingFrameMetadata() + { + double seconds = this.FrameDelay.ToDouble(); + + // Rational permits a zero denominator for metadata roundtripping. ImageSharp's connecting metadata cannot + // represent an infinite duration, so preserve it as the same unspecified zero duration used by other formats. + if (!double.IsFinite(seconds)) + { + seconds = 0; + } + + return new FormatConnectingFrameMetadata + { + BlendMode = FrameBlendMode.Source, + DisposalMode = FrameDisposalMode.DoNotDispose, + Duration = TimeSpan.FromSeconds(seconds) + }; + } + + /// + public void AfterFrameApply(ImageFrame source, ImageFrame destination, Matrix4x4 matrix) + where TPixel : unmanaged, IPixel + { + } + + /// + IDeepCloneable IDeepCloneable.DeepClone() => this.DeepClone(); + + /// + public HeifFrameMetadata DeepClone() => new(this); +} diff --git a/src/ImageSharp/Formats/Heif/HeifImageFormatDetector.cs b/src/ImageSharp/Formats/Heif/HeifImageFormatDetector.cs new file mode 100644 index 0000000000..968f810e0a --- /dev/null +++ b/src/ImageSharp/Formats/Heif/HeifImageFormatDetector.cs @@ -0,0 +1,70 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Buffers.Binary; +using System.Diagnostics.CodeAnalysis; + +namespace SixLabors.ImageSharp.Formats.Heif; + +/// +/// Detects HEIF file headers. +/// +public sealed class HeifImageFormatDetector : IImageFormatDetector +{ + /// + public int HeaderSize => 32; + + /// + public bool TryDetectFormat(ReadOnlySpan header, [NotNullWhen(true)] out IImageFormat? format) + { + format = IsSupportedFileFormat(header) ? HeifFormat.Instance : null; + return format is not null; + } + + /// + /// Determines whether the available header begins with a supported HEIF image file-type box. + /// + /// The fixed-size header prefix supplied by format detection. + /// when the prefix declares a supported image or image-sequence brand. + private static bool IsSupportedFileFormat(ReadOnlySpan header) + { + // Detection is intentionally limited to files beginning with ftyp. Other valid top-level boxes can precede + // ftyp in ISO BMFF, but scanning arbitrary input is outside the fixed-header detector contract. + if (header.Length < 16 || BinaryPrimitives.ReadUInt32BigEndian(header[4..]) != (uint)Heif4CharCode.Ftyp) + { + return false; + } + + uint compactBoxSize = BinaryPrimitives.ReadUInt32BigEndian(header); + int boxHeaderSize = 8; + ulong boxSize = compactBoxSize; + if (compactBoxSize == 1) + { + // An extended-size box inserts its 64-bit size before the normal ftyp payload. + if (header.Length < 24) + { + return false; + } + + boxHeaderSize = 16; + boxSize = BinaryPrimitives.ReadUInt64BigEndian(header[8..]); + } + + if (boxSize < (uint)(boxHeaderSize + 8)) + { + return false; + } + + ulong boxContentLength = boxSize - (uint)boxHeaderSize; + if ((boxContentLength & 3) != 0) + { + return false; + } + + // HeaderSize may expose only a prefix of a longer ftyp box. Whole compatible-brand codes in that prefix are + // sufficient for detection; the decoder validates the complete box before reading the rest of the container. + int availableContentLength = (int)Math.Min(boxContentLength, (ulong)(header.Length - boxHeaderSize)); + availableContentLength &= ~3; + return HeifConstants.TryGetFileType(header.Slice(boxHeaderSize, availableContentLength), out _); + } +} diff --git a/src/ImageSharp/Formats/Heif/HeifItem.cs b/src/ImageSharp/Formats/Heif/HeifItem.cs new file mode 100644 index 0000000000..e1c1ec4c3a --- /dev/null +++ b/src/ImageSharp/Formats/Heif/HeifItem.cs @@ -0,0 +1,257 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1; +using SixLabors.ImageSharp.Metadata.Profiles.Cicp; +using SixLabors.ImageSharp.Metadata.Profiles.Icc; + +namespace SixLabors.ImageSharp.Formats.Heif; + +/// +/// Describes a metadata or image item in a HEIF still-image container. +/// +/// The four-character item type. +/// The item identifier used by locations, properties, and references. +internal sealed class HeifItem(Heif4CharCode type, uint id) +{ + private IccProfile? iccProfile; + + private ReadOnlyMemory serializedIccProfile; + + /// + /// Gets the ID of this Item. + /// + public uint Id { get; } = id; + + /// + /// Gets the type of this Item. + /// + public Heif4CharCode Type { get; } = type; + + /// + /// Gets or sets a value indicating whether this item is excluded from primary-item discovery. + /// + public bool IsHidden { get; set; } + + /// + /// Gets or sets an earlier item whose identical property associations are reused by this item. + /// + public HeifItem? PropertySource { get; set; } + + /// + /// Gets or sets the first one-based property index assigned while writing the property container. + /// + public ushort FirstPropertyIndex { get; set; } + + /// + /// Gets or sets the name of this item. + /// + public string? Name { get; set; } + + /// + /// Gets or sets the Content Type of this item. + /// + public string? ContentType { get; set; } + + /// + /// Gets or sets the Content Encoding of this item. + /// + public string? ContentEncoding { get; set; } + + /// + /// Gets or sets the type of extension of this item. + /// + public uint ExtensionType { get; set; } + + /// + /// Gets or sets the URI of this item. + /// + public string? UriType { get; set; } + + /// + /// Gets or sets the registered auxiliary type associated with this image item. + /// + public string? AuxiliaryType { get; set; } + + /// + /// Gets or sets the ICC profile associated with this color image item, or when the item + /// has no restricted or unrestricted ICC color-information property. + /// + public IccProfile? IccProfile + { + get => this.iccProfile; + set + { + this.iccProfile = value; + this.serializedIccProfile = default; + } + } + + /// + /// Gets or sets the CICP color description associated with this color image item, or + /// when the item has no nclx color-information property. + /// + public CicpProfile? CicpProfile { get; set; } + + /// + /// Gets or sets the content light-level information associated with this image item, or + /// when the item has no content light-level property. + /// + public HeifContentLightLevel? ContentLightLevel { get; set; } + + /// + /// Gets or sets the mastering-display color volume associated with this image item, or + /// when the item has no mastering-display property. + /// + public HeifMasteringDisplayColorVolume? MasteringDisplayColorVolume { get; set; } + + /// + /// Gets or sets the content color volume associated with this image item, or when the + /// item has no content color-volume property. + /// + public HeifContentColorVolume? ContentColorVolume { get; set; } + + /// + /// Gets or sets the nominal ambient viewing environment associated with this image item, or + /// when the item has no ambient viewing-environment property. + /// + public HeifAmbientViewingEnvironment? AmbientViewingEnvironment { get; set; } + + /// + /// Gets or sets the reference mastering environment associated with this image item, or + /// when the item has no reference viewing-environment property. + /// + public HeifReferenceViewingEnvironment? ReferenceViewingEnvironment { get; set; } + + /// + /// Gets or sets the nominal diffuse-white description associated with this image item, or + /// when the item has no nominal diffuse-white property. + /// + public HeifNominalDiffuseWhite? NominalDiffuseWhite { get; set; } + + /// + /// Gets or sets the AV1 codec configuration associated with this coded image item, or + /// when the item has no AV1 codec-configuration property. + /// + public Av1CodecConfiguration? Av1CodecConfiguration { get; set; } + + /// + /// Gets or sets the operating-point selector associated with this AV1 image item, or when + /// the item uses the default operating-point index zero. + /// + public Av1OperatingPointSelector? Av1OperatingPointSelector { get; set; } + + /// + /// Gets or sets the spatial-layer selector associated with this AV1 image item, or when + /// no explicit layer selection is present. + /// + public Av1LayerSelector? Av1LayerSelector { get; set; } + + /// + /// Gets or sets the layered-image payload index associated with this AV1 image item, or + /// when the payload does not provide explicit layer boundaries. + /// + public Av1LayeredImageIndex? Av1LayeredImageIndex { get; set; } + + /// + /// Gets or sets the relative pixel spacing associated with this image item, or when the + /// item has no pixel-aspect-ratio property. + /// + public HeifPixelAspectRatio? PixelAspectRatio { get; set; } + + /// + /// Gets or sets the clean-aperture crop applied before image rotation and mirroring, or + /// when no clean-aperture property is associated with the item. + /// + public HeifCleanAperture? CleanAperture { get; set; } + + /// + /// Gets or sets the number of 90-degree counter-clockwise rotations applied to the image, or + /// when no image-rotation property is associated with the item. + /// + public byte? RotationAngle { get; set; } + + /// + /// Gets or sets the image-mirror axis, where zero is the horizontal axis and one is the vertical axis, or + /// when no image-mirror property is associated with the item. + /// + public byte? MirrorAxis { get; set; } + + /// + /// Gets or sets the number of color channels in each pixel. + /// + public int ChannelCount { get; set; } + + /// + /// Gets or sets the encoded precision of each image channel, or when the item has no + /// pixel-information property. + /// + public byte[]? ChannelBitDepths { get; set; } + + /// + /// Gets or sets the common encoded precision of every image channel, or when channel + /// precision is absent or represented individually by . + /// + public byte? UniformChannelBitDepth { get; set; } + + /// + /// Gets or sets the number of bits in a single pixel. + /// + public int BitsPerPixel { get; set; } + + /// + /// Gets the spatial extent of this item. + /// + public Size Extent { get; private set; } + + /// + /// Gets the spatial extent of this grid cells in this item. + /// + public Size GridCellExtent { get; private set; } + + /// + /// Gets the list of data locations for this item. + /// + public List DataLocations { get; } = []; + + /// + /// Gets the serialized ICC payload used while sizing and writing an encoded item. + /// + /// The serialized profile data, or an empty memory when no profile is assigned. + public ReadOnlyMemory GetIccProfileDataForWriting() + { + if (this.serializedIccProfile.IsEmpty && this.iccProfile is not null) + { + // Exact-size container writing queries the payload length before copying it. Retaining the serialized + // view on this transient item prevents an entry-built profile from being serialized for both passes. + this.serializedIccProfile = this.iccProfile.GetDataForWriting(); + } + + return this.serializedIccProfile; + } + + /// + /// Set the image extent. + /// + /// The size to set the extent to. + /// + /// Might be called twice for a grid, in which case the second call is the cell extent. + /// + public void SetExtent(Size extent) + { + if (this.Extent == default) + { + this.Extent = extent; + } + else + { + this.GridCellExtent = extent; + } + } + + /// + /// Returns the item type and identifier. + /// + /// The item type and identifier separated by a colon. + public override string ToString() => $"{this.Type}:{this.Id}"; +} diff --git a/src/ImageSharp/Formats/Heif/HeifItemDecoderUtilities.cs b/src/ImageSharp/Formats/Heif/HeifItemDecoderUtilities.cs new file mode 100644 index 0000000000..53418a0fcd --- /dev/null +++ b/src/ImageSharp/Formats/Heif/HeifItemDecoderUtilities.cs @@ -0,0 +1,33 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.PixelFormats; +using SixLabors.ImageSharp.Processing; + +namespace SixLabors.ImageSharp.Formats.Heif; + +/// +/// Provides shared presentation operations for decoded HEIF image items. +/// +internal static class HeifItemDecoderUtilities +{ + /// + /// Scales a decoded image to the spatial extent associated with its image item. + /// + /// The decoded pixel format. + /// The decoded image. + /// The image item that defines the presented spatial extent. + public static void ScaleToItemExtent(Image image, HeifItem item) + where TPixel : unmanaged, IPixel + { + Size extent = item.Extent; + if (extent == default || (image.Width == extent.Width && image.Height == extent.Height)) + { + return; + } + + // libavif applies box filtering when coded dimensions differ from an item's ispe dimensions. Reusing the + // same ImageSharp resampler keeps direct images and grid tiles on one presentation path. + image.Mutate(context => context.Resize(extent.Width, extent.Height, KnownResamplers.Box)); + } +} diff --git a/src/ImageSharp/Formats/Heif/HeifItemLink.cs b/src/ImageSharp/Formats/Heif/HeifItemLink.cs new file mode 100644 index 0000000000..381a8c986a --- /dev/null +++ b/src/ImageSharp/Formats/Heif/HeifItemLink.cs @@ -0,0 +1,27 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif; + +/// +/// Link between instances within the same HEIF file. +/// +/// The four-character reference type. +/// The identifier of the item that owns the references. +internal sealed class HeifItemLink(Heif4CharCode type, uint sourceId) +{ + /// + /// Gets the type of link. + /// + public Heif4CharCode Type { get; } = type; + + /// + /// Gets the ID of the source item of this link. + /// + public uint SourceId { get; } = sourceId; + + /// + /// Gets the destination item IDs of this link. + /// + public List DestinationIds { get; } = new List(); +} diff --git a/src/ImageSharp/Formats/Heif/HeifLocation.cs b/src/ImageSharp/Formats/Heif/HeifLocation.cs new file mode 100644 index 0000000000..1485be5dc7 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/HeifLocation.cs @@ -0,0 +1,66 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif; + +/// +/// Describes one contiguous extent of an item's encoded data. +/// +/// The origin from which the base and extent offsets are measured. +/// The item-location base offset. +/// The extent offset relative to the base offset. +/// The length of the extent in bytes. +internal sealed class HeifLocation(HeifLocationOffsetOrigin origin, long baseOffset, long offset, long length) +{ + /// + /// Gets the origin of the offsets in this location. + /// + public HeifLocationOffsetOrigin Origin { get; } = origin; + + /// + /// Gets the item-location base offset in bytes. + /// + public long BaseOffset { get; } = baseOffset; + + /// + /// Gets the extent offset relative to in bytes. + /// + public long Offset { get; } = offset; + + /// + /// Gets the extent length in bytes. + /// + public long Length { get; } = length; + + /// + /// Resolves the absolute stream position of this extent. + /// + /// The absolute origin of the item-data payload. + /// The absolute origin of the referenced item payload. + /// The absolute byte position of the extent in the input stream. + public long GetStreamPosition(long positionOfMediaData, long positionOfItem) => this.Origin switch + { + HeifLocationOffsetOrigin.FileOffset => this.BaseOffset + this.Offset, + HeifLocationOffsetOrigin.ItemDataOffset => positionOfMediaData + this.BaseOffset + this.Offset, + _ => positionOfItem + this.BaseOffset + this.Offset + }; + + /// + public override int GetHashCode() => HashCode.Combine(this.Origin, this.Offset, this.Length, this.BaseOffset); + + /// + public override bool Equals(object? obj) + { + if (obj is not HeifLocation other) + { + return false; + } + + if (this.Origin != other.Origin || this.Length != other.Length) + { + return false; + } + + return this.Offset == other.Offset && this.BaseOffset == other.BaseOffset; + } +} diff --git a/src/ImageSharp/Formats/Heif/HeifLocationComparer.cs b/src/ImageSharp/Formats/Heif/HeifLocationComparer.cs new file mode 100644 index 0000000000..d20dccb98f --- /dev/null +++ b/src/ImageSharp/Formats/Heif/HeifLocationComparer.cs @@ -0,0 +1,61 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif; + +/// +/// Orders item extents by their resolved absolute stream position. +/// +internal sealed class HeifLocationComparer : IComparer +{ + /// + /// The absolute origin of item-data-relative extents. + /// + private readonly long positionOfMediaData; + + /// + /// The absolute origin of item-relative extents. + /// + private readonly long positionOfItem; + + /// + /// Initializes a new instance of the class. + /// + /// The absolute origin of the item-data payload. + /// The absolute origin of the referenced item payload. + public HeifLocationComparer(long positionOfMediaData, long positionOfItem) + { + this.positionOfMediaData = positionOfMediaData; + this.positionOfItem = positionOfItem; + } + + /// + /// Compares two extents by their resolved absolute stream positions. + /// + /// The first extent. + /// The second extent. + /// A negative value when precedes , zero when their positions match, or a positive value otherwise. + public int Compare(HeifLocation? x, HeifLocation? y) + { + if (x is null) + { + if (y is null) + { + return 0; + } + + return 1; + } + + if (y is null) + { + return -1; + } + + long xPos = x.GetStreamPosition(this.positionOfMediaData, this.positionOfItem); + long yPos = y.GetStreamPosition(this.positionOfMediaData, this.positionOfItem); + + // CompareTo avoids overflowing when valid 64-bit offsets lie near opposite numeric limits. + return xPos.CompareTo(yPos); + } +} diff --git a/src/ImageSharp/Formats/Heif/HeifLocationOffsetOrigin.cs b/src/ImageSharp/Formats/Heif/HeifLocationOffsetOrigin.cs new file mode 100644 index 0000000000..e3011d0229 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/HeifLocationOffsetOrigin.cs @@ -0,0 +1,25 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif; + +/// +/// Identifies the origin used to resolve an item-location extent offset. +/// +internal enum HeifLocationOffsetOrigin +{ + /// + /// The base and extent offsets are absolute file offsets. + /// + FileOffset = 0, + + /// + /// The base and extent offsets are relative to the item-data box payload. + /// + ItemDataOffset = 1, + + /// + /// The base and extent offsets are relative to another item payload. + /// + ItemOffset = 2 +} diff --git a/src/ImageSharp/Formats/Heif/HeifMasteringDisplayColorVolume.cs b/src/ImageSharp/Formats/Heif/HeifMasteringDisplayColorVolume.cs new file mode 100644 index 0000000000..90323f195d --- /dev/null +++ b/src/ImageSharp/Formats/Heif/HeifMasteringDisplayColorVolume.cs @@ -0,0 +1,84 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.ColorProfiles; + +namespace SixLabors.ImageSharp.Formats.Heif; + +/// +/// Describes the color volume of the display used to master a HEIF image. +/// +public readonly struct HeifMasteringDisplayColorVolume : IEquatable +{ + /// + /// Initializes a new instance of the struct. + /// + /// The CIE 1931 chromaticity coordinates of the mastering display primaries. + /// The CIE 1931 chromaticity coordinates of the mastering display white point. + /// The nominal maximum mastering-display luminance in candelas per square metre. + /// The nominal minimum mastering-display luminance in candelas per square metre. + public HeifMasteringDisplayColorVolume( + RgbPrimariesChromaticityCoordinates primaries, + CieXyChromaticityCoordinates whitePoint, + double maximumLuminance, + double minimumLuminance) + { + this.Primaries = primaries; + this.WhitePoint = whitePoint; + this.MaximumLuminance = maximumLuminance; + this.MinimumLuminance = minimumLuminance; + } + + /// + /// Gets the CIE 1931 chromaticity coordinates of the mastering display primaries. + /// + public RgbPrimariesChromaticityCoordinates Primaries { get; } + + /// + /// Gets the CIE 1931 chromaticity coordinates of the mastering display white point. + /// + public CieXyChromaticityCoordinates WhitePoint { get; } + + /// + /// Gets the nominal maximum mastering-display luminance in candelas per square metre. + /// + public double MaximumLuminance { get; } + + /// + /// Gets the nominal minimum mastering-display luminance in candelas per square metre. + /// + public double MinimumLuminance { get; } + + /// + /// Compares two mastering-display color volumes for equality. + /// + /// The first mastering-display color volume. + /// The second mastering-display color volume. + /// when every color-volume value is equal. + public static bool operator ==(HeifMasteringDisplayColorVolume left, HeifMasteringDisplayColorVolume right) + => left.Equals(right); + + /// + /// Compares two mastering-display color volumes for inequality. + /// + /// The first mastering-display color volume. + /// The second mastering-display color volume. + /// when any color-volume value differs. + public static bool operator !=(HeifMasteringDisplayColorVolume left, HeifMasteringDisplayColorVolume right) + => !left.Equals(right); + + /// + public override bool Equals(object? obj) + => obj is HeifMasteringDisplayColorVolume other && this.Equals(other); + + /// + public bool Equals(HeifMasteringDisplayColorVolume other) + => this.Primaries.Equals(other.Primaries) + && this.WhitePoint.Equals(other.WhitePoint) + && this.MaximumLuminance.Equals(other.MaximumLuminance) + && this.MinimumLuminance.Equals(other.MinimumLuminance); + + /// + public override int GetHashCode() + => HashCode.Combine(this.Primaries, this.WhitePoint, this.MaximumLuminance, this.MinimumLuminance); +} diff --git a/src/ImageSharp/Formats/Heif/HeifMetadata.cs b/src/ImageSharp/Formats/Heif/HeifMetadata.cs new file mode 100644 index 0000000000..c11f42ca14 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/HeifMetadata.cs @@ -0,0 +1,186 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Numerics; +using SixLabors.ImageSharp.PixelFormats; + +namespace SixLabors.ImageSharp.Formats.Heif; + +/// +/// Provides HEIF specific metadata information for the image. +/// +public class HeifMetadata : IFormatMetadata +{ + /// + /// Initializes a new instance of the class. + /// + public HeifMetadata() + { + } + + /// + /// Initializes a new instance of the class. + /// + /// The metadata to create an instance from. + private HeifMetadata(HeifMetadata other) + { + this.CompressionMethod = other.CompressionMethod; + this.BitDepth = other.BitDepth; + this.IsMonochrome = other.IsMonochrome; + this.HasAlpha = other.HasAlpha; + this.RepeatCount = other.RepeatCount; + this.AnimateRootFrame = other.AnimateRootFrame; + this.ContentLightLevel = other.ContentLightLevel; + this.MasteringDisplayColorVolume = other.MasteringDisplayColorVolume; + this.ContentColorVolume = other.ContentColorVolume; + this.AmbientViewingEnvironment = other.AmbientViewingEnvironment; + this.ReferenceViewingEnvironment = other.ReferenceViewingEnvironment; + this.NominalDiffuseWhite = other.NominalDiffuseWhite; + } + + /// + /// Gets or sets the compression method used for the primary frame. + /// + public HeifCompressionMethod CompressionMethod { get; set; } = HeifCompressionMethod.LegacyJpeg; + + /// + /// Gets or sets the encoded precision of each color component. The default is . + /// + public HeifBitDepth BitDepth { get; set; } = HeifBitDepth.Bit8; + + /// + /// Gets or sets a value indicating whether the primary image contains a single luminance component. + /// + public bool IsMonochrome { get; set; } + + /// + /// Gets or sets a value indicating whether the primary image has an alpha channel. + /// + public bool HasAlpha { get; set; } + + /// + /// Gets or sets the number of times the image sequence is played. A value of zero repeats indefinitely. + /// The default is one play. + /// + public ushort RepeatCount { get; set; } = 1; + + /// + /// Gets or sets a value indicating whether the root frame is included in the image sequence. + /// The default is . + /// + public bool AnimateRootFrame { get; set; } = true; + + /// + /// Gets or sets the content light-level information for the primary image, or when it is + /// not available. + /// + public HeifContentLightLevel? ContentLightLevel { get; set; } + + /// + /// Gets or sets the mastering-display color volume for the primary image, or when it is + /// not available. + /// + public HeifMasteringDisplayColorVolume? MasteringDisplayColorVolume { get; set; } + + /// + /// Gets or sets the content color volume for the primary image, or when it is not + /// available. + /// + public HeifContentColorVolume? ContentColorVolume { get; set; } + + /// + /// Gets or sets the nominal ambient viewing environment for the primary image, or + /// when it is not available. + /// + public HeifAmbientViewingEnvironment? AmbientViewingEnvironment { get; set; } + + /// + /// Gets or sets the reference mastering environment for the primary image, or when + /// it is not available. + /// + public HeifReferenceViewingEnvironment? ReferenceViewingEnvironment { get; set; } + + /// + /// Gets or sets the nominal diffuse-white description for the primary image, or when + /// it is not available. + /// + public HeifNominalDiffuseWhite? NominalDiffuseWhite { get; set; } + + /// + public static HeifMetadata FromFormatConnectingMetadata(FormatConnectingMetadata metadata) + { + int componentPrecision = metadata.PixelTypeInfo.ComponentInfo?.GetMaximumComponentPrecision() ?? 8; + HeifBitDepth bitDepth = componentPrecision switch + { + <= 8 => HeifBitDepth.Bit8, + <= 10 => HeifBitDepth.Bit10, + _ => HeifBitDepth.Bit12 + }; + + return new HeifMetadata + { + BitDepth = bitDepth, + IsMonochrome = metadata.PixelTypeInfo.ColorType.HasFlag(PixelColorType.Luminance) + && !metadata.PixelTypeInfo.ColorType.HasFlag(PixelColorType.ChrominanceBlue), + HasAlpha = metadata.PixelTypeInfo.AlphaRepresentation != PixelAlphaRepresentation.None, + RepeatCount = metadata.RepeatCount, + AnimateRootFrame = metadata.AnimateRootFrame + }; + } + + /// + public PixelTypeInfo GetPixelTypeInfo() + { + int bitDepth = (int)this.BitDepth; + int colorComponentCount = this.IsMonochrome ? 1 : 3; + int componentCount = colorComponentCount + (this.HasAlpha ? 1 : 0); + int bitsPerPixel = componentCount * bitDepth; + PixelColorType colorType = this.IsMonochrome ? PixelColorType.Luminance : PixelColorType.RGB; + PixelComponentInfo info; + if (this.IsMonochrome) + { + info = this.HasAlpha + ? PixelComponentInfo.Create(2, bitsPerPixel, bitDepth, bitDepth) + : PixelComponentInfo.Create(1, bitsPerPixel, bitDepth); + } + else + { + info = this.HasAlpha + ? PixelComponentInfo.Create(4, bitsPerPixel, bitDepth, bitDepth, bitDepth, bitDepth) + : PixelComponentInfo.Create(3, bitsPerPixel, bitDepth, bitDepth, bitDepth); + } + + if (this.HasAlpha) + { + colorType |= PixelColorType.Alpha; + } + + return new PixelTypeInfo(bitsPerPixel) + { + AlphaRepresentation = this.HasAlpha ? PixelAlphaRepresentation.Unassociated : PixelAlphaRepresentation.None, + ColorType = colorType, + ComponentInfo = info, + }; + } + + /// + public FormatConnectingMetadata ToFormatConnectingMetadata() + => new() + { + AnimateRootFrame = this.AnimateRootFrame, + PixelTypeInfo = this.GetPixelTypeInfo(), + RepeatCount = this.RepeatCount + }; + + /// + IDeepCloneable IDeepCloneable.DeepClone() => this.DeepClone(); + + /// + public HeifMetadata DeepClone() => new(this); + + /// + public void AfterImageApply(Image destination, Matrix4x4 matrix) + where TPixel : unmanaged, IPixel + { + } +} diff --git a/src/ImageSharp/Formats/Heif/HeifNominalDiffuseWhite.cs b/src/ImageSharp/Formats/Heif/HeifNominalDiffuseWhite.cs new file mode 100644 index 0000000000..3815e1551e --- /dev/null +++ b/src/ImageSharp/Formats/Heif/HeifNominalDiffuseWhite.cs @@ -0,0 +1,64 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif; + +/// +/// Describes the nominal diffuse-white luminance of a HEIF image. +/// +public readonly struct HeifNominalDiffuseWhite : IEquatable +{ + /// + /// Initializes a new instance of the struct. + /// + /// + /// The nominal diffuse-white luminance in candelas per square metre, or to use the + /// standard default. + /// + public HeifNominalDiffuseWhite(double? luminance) => this.Luminance = luminance; + + /// + /// Gets the nominal diffuse-white luminance in candelas per square metre, or when the + /// image requests the standard default. + /// + public double? Luminance { get; } + + /// + /// Compares two nominal diffuse-white descriptions for equality. + /// + /// The first nominal diffuse-white description. + /// The second nominal diffuse-white description. + /// when both descriptions specify the same luminance behavior. + public static bool operator ==(HeifNominalDiffuseWhite left, HeifNominalDiffuseWhite right) + => left.Equals(right); + + /// + /// Compares two nominal diffuse-white descriptions for inequality. + /// + /// The first nominal diffuse-white description. + /// The second nominal diffuse-white description. + /// when the descriptions specify different luminance behavior. + public static bool operator !=(HeifNominalDiffuseWhite left, HeifNominalDiffuseWhite right) + => !left.Equals(right); + + /// + /// Determines whether the specified object is a nominal diffuse-white description with the same value. + /// + /// The object to compare with this value. + /// when specifies the same luminance behavior. + public override bool Equals(object? obj) + => obj is HeifNominalDiffuseWhite other && this.Equals(other); + + /// + /// Determines whether the specified nominal diffuse-white description has the same value as this value. + /// + /// The nominal diffuse-white description to compare with this value. + /// when both descriptions specify the same luminance behavior. + public bool Equals(HeifNominalDiffuseWhite other) => this.Luminance.Equals(other.Luminance); + + /// + /// Returns a hash code for this nominal diffuse-white description. + /// + /// A hash code derived from the luminance behavior. + public override int GetHashCode() => this.Luminance.GetHashCode(); +} diff --git a/src/ImageSharp/Formats/Heif/HeifPixelAspectRatio.cs b/src/ImageSharp/Formats/Heif/HeifPixelAspectRatio.cs new file mode 100644 index 0000000000..a276fba576 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/HeifPixelAspectRatio.cs @@ -0,0 +1,22 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif; + +/// +/// Describes the relative horizontal and vertical spacing of pixels in a HEIF image. +/// +/// The relative horizontal pixel spacing. +/// The relative vertical pixel spacing. +internal sealed class HeifPixelAspectRatio(uint horizontalSpacing, uint verticalSpacing) +{ + /// + /// Gets the relative horizontal pixel spacing. + /// + public uint HorizontalSpacing { get; } = horizontalSpacing; + + /// + /// Gets the relative vertical pixel spacing. + /// + public uint VerticalSpacing { get; } = verticalSpacing; +} diff --git a/src/ImageSharp/Formats/Heif/HeifPropertyParser.cs b/src/ImageSharp/Formats/Heif/HeifPropertyParser.cs new file mode 100644 index 0000000000..31a431b0b8 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/HeifPropertyParser.cs @@ -0,0 +1,448 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Buffers.Binary; +using SixLabors.ImageSharp.ColorProfiles; +using SixLabors.ImageSharp.Formats.Heif.Av1; +using SixLabors.ImageSharp.Metadata.Profiles.Cicp; +using SixLabors.ImageSharp.Metadata.Profiles.Icc; + +namespace SixLabors.ImageSharp.Formats.Heif; + +/// +/// Parses the image properties shared by HEIF items and image-sequence sample entries. +/// +internal static class HeifPropertyParser +{ + /// + /// Parses relative horizontal and vertical pixel spacing. + /// + /// The complete pixel-aspect-ratio payload. + /// The validated relative pixel spacing. + public static HeifPixelAspectRatio ParsePixelAspectRatio(ReadOnlySpan data) + { + EnsureExactLength(data, 8, "pixel aspect ratio"); + uint horizontalSpacing = BinaryPrimitives.ReadUInt32BigEndian(data); + uint verticalSpacing = BinaryPrimitives.ReadUInt32BigEndian(data[4..]); + if (horizontalSpacing == 0 || verticalSpacing == 0) + { + throw new InvalidImageContentException("The pixel aspect ratio property has zero spacing."); + } + + return new HeifPixelAspectRatio(horizontalSpacing, verticalSpacing); + } + + /// + /// Parses and validates an embedded ICC profile. + /// + /// The complete ICC profile bytes. + /// The validated ICC profile. + public static IccProfile ParseIccProfile(byte[] data) + { + if (data.Length == 0) + { + throw new InvalidImageContentException("The HEIF ICC color property contains an empty profile."); + } + + // The HEIF parser allocates this exact array as the profile's final storage, so IccProfile can adopt it without + // copying the potentially large profile payload. + IccProfile profile = new(data); + if (!profile.CheckIsValid()) + { + throw new InvalidIccProfileException("Invalid HEIF ICC profile."); + } + + return profile; + } + + /// + /// Parses a CICP color description from an nclx color-information payload. + /// + /// The complete payload following the nclx color type. + /// The CICP color description. + public static CicpProfile ParseCicpProfile(ReadOnlySpan data) + { + EnsureExactLength(data, 7, "CICP color information"); + ushort colorPrimaries = BinaryPrimitives.ReadUInt16BigEndian(data); + ushort transferCharacteristics = BinaryPrimitives.ReadUInt16BigEndian(data[2..]); + ushort matrixCoefficients = BinaryPrimitives.ReadUInt16BigEndian(data[4..]); + byte rangeAndReserved = data[6]; + if ((rangeAndReserved & 0x7F) != 0) + { + throw new InvalidImageContentException("The HEIF CICP color property has nonzero reserved bits."); + } + + // The box fields are 16-bit so future registrations remain representable. ImageSharp's CICP profile exposes + // the currently registered byte-sized H.273 values and maps larger future values to unspecified. + byte colorPrimariesValue = colorPrimaries <= byte.MaxValue ? (byte)colorPrimaries : (byte)CicpColorPrimaries.Unspecified; + byte transferCharacteristicsValue = transferCharacteristics <= byte.MaxValue + ? (byte)transferCharacteristics + : (byte)CicpTransferCharacteristics.Unspecified; + + byte matrixCoefficientsValue = matrixCoefficients <= byte.MaxValue + ? (byte)matrixCoefficients + : (byte)CicpMatrixCoefficients.Unspecified; + + return new CicpProfile(colorPrimariesValue, transferCharacteristicsValue, matrixCoefficientsValue, (rangeAndReserved & 0x80) != 0); + } + + /// + /// Parses content light-level information. + /// + /// The complete content-light-level payload. + /// The content light-level information. + public static HeifContentLightLevel ParseContentLightLevel(ReadOnlySpan data) + { + EnsureExactLength(data, 4, "content light level"); + return new HeifContentLightLevel(BinaryPrimitives.ReadUInt16BigEndian(data), BinaryPrimitives.ReadUInt16BigEndian(data[2..])); + } + + /// + /// Parses mastering-display color-volume information. + /// + /// The complete mastering-display color-volume payload. + /// The mastering-display color-volume information. + public static HeifMasteringDisplayColorVolume ParseMasteringDisplayColorVolume(ReadOnlySpan data) + { + EnsureExactLength(data, 24, "mastering display color volume"); + const float chromaticityScale = 1F / 50000F; + const double luminanceScale = 1D / 10000D; + + // The registered payload inherits the G, B, R primary order used by its mastering-display source syntax. + // Reorder it into ImageSharp's existing RGB coordinate type at the shared container boundary. + CieXyChromaticityCoordinates greenPrimary = new( + BinaryPrimitives.ReadUInt16BigEndian(data) * chromaticityScale, + BinaryPrimitives.ReadUInt16BigEndian(data[2..]) * chromaticityScale); + + CieXyChromaticityCoordinates bluePrimary = new( + BinaryPrimitives.ReadUInt16BigEndian(data[4..]) * chromaticityScale, + BinaryPrimitives.ReadUInt16BigEndian(data[6..]) * chromaticityScale); + + CieXyChromaticityCoordinates redPrimary = new( + BinaryPrimitives.ReadUInt16BigEndian(data[8..]) * chromaticityScale, + BinaryPrimitives.ReadUInt16BigEndian(data[10..]) * chromaticityScale); + + return new HeifMasteringDisplayColorVolume( + new RgbPrimariesChromaticityCoordinates(redPrimary, greenPrimary, bluePrimary), + new CieXyChromaticityCoordinates( + BinaryPrimitives.ReadUInt16BigEndian(data[12..]) * chromaticityScale, + BinaryPrimitives.ReadUInt16BigEndian(data[14..]) * chromaticityScale), + BinaryPrimitives.ReadUInt32BigEndian(data[16..]) * luminanceScale, + BinaryPrimitives.ReadUInt32BigEndian(data[20..]) * luminanceScale); + } + + /// + /// Parses content color-volume information. + /// + /// The complete content color-volume payload. + /// The content color-volume information. + public static HeifContentColorVolume ParseContentColorVolume(ReadOnlySpan data) + { + if (data.IsEmpty) + { + throw new InvalidImageContentException("The content color-volume property is truncated."); + } + + byte flags = data[0]; + if ((flags & 0xC3) != 0) + { + throw new InvalidImageContentException("The content color-volume property has nonzero reserved flags."); + } + + bool primariesPresent = (flags & 0x20) != 0; + bool minimumLuminancePresent = (flags & 0x10) != 0; + bool maximumLuminancePresent = (flags & 0x08) != 0; + bool averageLuminancePresent = (flags & 0x04) != 0; + if (!primariesPresent && !minimumLuminancePresent && !maximumLuminancePresent && !averageLuminancePresent) + { + throw new InvalidImageContentException("The content color-volume property does not describe any values."); + } + + int expectedLength = 1 + + (primariesPresent ? 24 : 0) + + (minimumLuminancePresent ? 4 : 0) + + (maximumLuminancePresent ? 4 : 0) + + (averageLuminancePresent ? 4 : 0); + + EnsureExactLength(data, expectedLength, "content color volume"); + int offset = 1; + RgbPrimariesChromaticityCoordinates? primaries = null; + if (primariesPresent) + { + int greenX = BinaryPrimitives.ReadInt32BigEndian(data[offset..]); + int greenY = BinaryPrimitives.ReadInt32BigEndian(data[(offset + 4)..]); + int blueX = BinaryPrimitives.ReadInt32BigEndian(data[(offset + 8)..]); + int blueY = BinaryPrimitives.ReadInt32BigEndian(data[(offset + 12)..]); + int redX = BinaryPrimitives.ReadInt32BigEndian(data[(offset + 16)..]); + int redY = BinaryPrimitives.ReadInt32BigEndian(data[(offset + 20)..]); + const int maximumChromaticityValue = 5_000_000; + if (greenX is < -maximumChromaticityValue or > maximumChromaticityValue + || greenY is < -maximumChromaticityValue or > maximumChromaticityValue + || blueX is < -maximumChromaticityValue or > maximumChromaticityValue + || blueY is < -maximumChromaticityValue or > maximumChromaticityValue + || redX is < -maximumChromaticityValue or > maximumChromaticityValue + || redY is < -maximumChromaticityValue or > maximumChromaticityValue) + { + throw new InvalidImageContentException("The content color-volume property has an out-of-range primary coordinate."); + } + + const float chromaticityScale = 1F / 50000F; + + // Content-color-volume syntax also stores signed coordinates in G, B, R order. + primaries = new RgbPrimariesChromaticityCoordinates( + new CieXyChromaticityCoordinates(redX * chromaticityScale, redY * chromaticityScale), + new CieXyChromaticityCoordinates(greenX * chromaticityScale, greenY * chromaticityScale), + new CieXyChromaticityCoordinates(blueX * chromaticityScale, blueY * chromaticityScale)); + + offset += 24; + } + + uint? minimumLuminance = minimumLuminancePresent ? BinaryPrimitives.ReadUInt32BigEndian(data[offset..]) : null; + offset += minimumLuminancePresent ? 4 : 0; + uint? maximumLuminance = maximumLuminancePresent ? BinaryPrimitives.ReadUInt32BigEndian(data[offset..]) : null; + offset += maximumLuminancePresent ? 4 : 0; + uint? averageLuminance = averageLuminancePresent ? BinaryPrimitives.ReadUInt32BigEndian(data[offset..]) : null; + if ((minimumLuminance is not null && averageLuminance is not null && minimumLuminance.Value > averageLuminance.Value) + || (averageLuminance is not null && maximumLuminance is not null && averageLuminance.Value > maximumLuminance.Value) + || (minimumLuminance is not null && maximumLuminance is not null && minimumLuminance.Value > maximumLuminance.Value)) + { + throw new InvalidImageContentException("The content color-volume luminance values are not in ascending order."); + } + + const double luminanceScale = 1D / 10000000D; + + // These values are normalized according to the signaled transfer characteristics. Preserve that + // unitless meaning instead of presenting them as physical display luminance. + return new HeifContentColorVolume( + primaries, + minimumLuminance * luminanceScale, + maximumLuminance * luminanceScale, + averageLuminance * luminanceScale); + } + + /// + /// Parses an ambient viewing environment. + /// + /// The complete ambient viewing-environment payload. + /// The ambient viewing environment. + public static HeifAmbientViewingEnvironment ParseAmbientViewingEnvironment(ReadOnlySpan data) + { + EnsureExactLength(data, 8, "ambient viewing environment"); + uint illuminance = BinaryPrimitives.ReadUInt32BigEndian(data); + ushort lightX = BinaryPrimitives.ReadUInt16BigEndian(data[4..]); + ushort lightY = BinaryPrimitives.ReadUInt16BigEndian(data[6..]); + if (illuminance == 0) + { + throw new InvalidImageContentException("The ambient viewing-environment property has zero illuminance."); + } + + if (lightX > 50000 || lightY > 50000) + { + throw new InvalidImageContentException("The ambient viewing-environment property has an out-of-range chromaticity coordinate."); + } + + const double illuminanceScale = 1D / 10000D; + const float chromaticityScale = 1F / 50000F; + + // The property inherits H.274's fixed-point units: 0.0001 lux for illuminance and 0.00002 for each + // normalized CIE chromaticity coordinate. + return new HeifAmbientViewingEnvironment( + illuminance * illuminanceScale, + new CieXyChromaticityCoordinates(lightX * chromaticityScale, lightY * chromaticityScale)); + } + + /// + /// Parses a reference viewing environment. + /// + /// The complete reference viewing-environment payload. + /// The reference viewing environment. + public static HeifReferenceViewingEnvironment ParseReferenceViewingEnvironment(ReadOnlySpan data) + { + EnsureExactLength(data, 20, "reference viewing environment"); + if (BinaryPrimitives.ReadUInt32BigEndian(data) != 0) + { + throw new InvalidImageContentException("The reference viewing-environment property has an unsupported version or flags."); + } + + ushort surroundX = BinaryPrimitives.ReadUInt16BigEndian(data[8..]); + ushort surroundY = BinaryPrimitives.ReadUInt16BigEndian(data[10..]); + ushort peripheryX = BinaryPrimitives.ReadUInt16BigEndian(data[16..]); + ushort peripheryY = BinaryPrimitives.ReadUInt16BigEndian(data[18..]); + if (surroundX > 10000 || surroundY > 10000 || peripheryX > 10000 || peripheryY > 10000) + { + throw new InvalidImageContentException("The reference viewing-environment property has an out-of-range chromaticity coordinate."); + } + + const double luminanceScale = 1D / 10000D; + const float chromaticityScale = 1F / 10000F; + + // The full-box header is followed by display-surround and wider-periphery fields. Both groups use + // 0.0001 increments, but luminance is physical cd/m2 while the CIE coordinates are normalized. + return new HeifReferenceViewingEnvironment( + BinaryPrimitives.ReadUInt32BigEndian(data[4..]) * luminanceScale, + new CieXyChromaticityCoordinates(surroundX * chromaticityScale, surroundY * chromaticityScale), + BinaryPrimitives.ReadUInt32BigEndian(data[12..]) * luminanceScale, + new CieXyChromaticityCoordinates(peripheryX * chromaticityScale, peripheryY * chromaticityScale)); + } + + /// + /// Parses nominal diffuse-white information. + /// + /// The complete nominal diffuse-white payload. + /// The nominal diffuse-white information. + public static HeifNominalDiffuseWhite ParseNominalDiffuseWhite(ReadOnlySpan data) + { + EnsureExactLength(data, 8, "nominal diffuse white"); + if (BinaryPrimitives.ReadUInt32BigEndian(data) != 0) + { + throw new InvalidImageContentException("The nominal diffuse-white property has an unsupported version or flags."); + } + + uint luminance = BinaryPrimitives.ReadUInt32BigEndian(data[4..]); + const double luminanceScale = 1D / 10000D; + + // A zero coded value requests the standard-defined default rather than describing black diffuse white. + return new HeifNominalDiffuseWhite(luminance == 0 ? null : luminance * luminanceScale); + } + + /// + /// Parses the sequence-header operating point selected by an AV1 image item. + /// + /// The complete AV1 operating-point-selector payload. + /// The selected zero-based operating-point index. + public static Av1OperatingPointSelector ParseAv1OperatingPointSelector(ReadOnlySpan data) + { + EnsureExactLength(data, 1, "AV1 operating-point selector"); + byte index = data[0]; + if (index >= Av1Constants.MaxOperatingPointCount) + { + // AV1 signals operating_points_cnt_minus_1 in five bits, so a sequence header cannot contain + // an operating point whose zero-based index is greater than 31. + throw new InvalidImageContentException($"The AV1 operating-point selector requests unsupported index {index}."); + } + + return new Av1OperatingPointSelector(index); + } + + /// + /// Parses the spatial layer selected by an AV1 image item. + /// + /// The complete AV1 layer-selector payload. + /// The selected spatial-layer identifier. + public static Av1LayerSelector ParseAv1LayerSelector(ReadOnlySpan data) + { + EnsureExactLength(data, 2, "AV1 layer selector"); + ushort layerId = BinaryPrimitives.ReadUInt16BigEndian(data); + if (layerId != Av1LayerSelector.AllLayers && layerId >= Av1Constants.MaxSpatialLayerCount) + { + // AV1 OBU extension headers carry spatial_id in two bits. AVIF reserves 0xFFFF to request + // progressive exposure or final-layer decoding instead of selecting one of those four IDs. + throw new InvalidImageContentException($"The AV1 layer selector requests unsupported layer {layerId}."); + } + + return new Av1LayerSelector(layerId); + } + + /// + /// Parses the explicit payload boundaries of a layered AV1 image item. + /// + /// The complete AV1 layered-image-indexing payload. + /// The three explicit layer sizes. + public static Av1LayeredImageIndex ParseAv1LayeredImageIndex(ReadOnlySpan data) + { + if (data.IsEmpty) + { + throw new InvalidImageContentException("The AV1 layered-image indexing property has an invalid length."); + } + + byte sizeFlags = data[0]; + if ((sizeFlags & 0xFE) != 0) + { + throw new InvalidImageContentException("The AV1 layered-image indexing property has nonzero reserved bits."); + } + + bool usesLargeSizes = (sizeFlags & 1) != 0; + int layerSizeWidth = usesLargeSizes ? 4 : 2; + + // a1lx stores the first three sizes explicitly. A fourth layer, when present, consumes the + // remaining item payload and therefore has no stored size field. + EnsureExactLength(data, 1 + (3 * layerSizeWidth), "AV1 layered-image indexing"); + if (usesLargeSizes) + { + return new Av1LayeredImageIndex( + BinaryPrimitives.ReadUInt32BigEndian(data[1..]), + BinaryPrimitives.ReadUInt32BigEndian(data[5..]), + BinaryPrimitives.ReadUInt32BigEndian(data[9..])); + } + + return new Av1LayeredImageIndex( + BinaryPrimitives.ReadUInt16BigEndian(data[1..]), + BinaryPrimitives.ReadUInt16BigEndian(data[3..]), + BinaryPrimitives.ReadUInt16BigEndian(data[5..])); + } + + /// + /// Parses a clean-aperture crop description. + /// + /// The complete clean-aperture payload. + /// The clean-aperture crop description. + public static HeifCleanAperture ParseCleanAperture(ReadOnlySpan data) + { + EnsureExactLength(data, 32, "clean aperture"); + return new HeifCleanAperture( + BinaryPrimitives.ReadInt32BigEndian(data), + BinaryPrimitives.ReadInt32BigEndian(data[4..]), + BinaryPrimitives.ReadInt32BigEndian(data[8..]), + BinaryPrimitives.ReadInt32BigEndian(data[12..]), + BinaryPrimitives.ReadInt32BigEndian(data[16..]), + BinaryPrimitives.ReadInt32BigEndian(data[20..]), + BinaryPrimitives.ReadInt32BigEndian(data[24..]), + BinaryPrimitives.ReadInt32BigEndian(data[28..])); + } + + /// + /// Parses the number of counter-clockwise quarter turns applied to an image. + /// + /// The complete image-rotation payload. + /// The number of counter-clockwise quarter turns. + public static byte ParseRotation(ReadOnlySpan data) + { + EnsureExactLength(data, 1, "image rotation"); + if ((data[0] & 0xFC) != 0) + { + throw new InvalidImageContentException("The image rotation property has nonzero reserved bits."); + } + + return (byte)(data[0] & 3); + } + + /// + /// Parses the horizontal or vertical image-mirror axis. + /// + /// The complete image-mirror payload. + /// Zero for the horizontal axis or one for the vertical axis. + public static byte ParseMirrorAxis(ReadOnlySpan data) + { + EnsureExactLength(data, 1, "image mirror"); + if ((data[0] & 0xFE) != 0) + { + throw new InvalidImageContentException("The image mirror property has nonzero reserved bits."); + } + + return (byte)(data[0] & 1); + } + + /// + /// Requires a fixed-size image property to contain exactly its registered payload length. + /// + /// The complete property payload. + /// The registered payload length. + /// The property name used in malformed-image diagnostics. + private static void EnsureExactLength(ReadOnlySpan data, int length, string name) + { + if (data.Length != length) + { + throw new InvalidImageContentException($"The {name} property has an invalid length."); + } + } +} diff --git a/src/ImageSharp/Formats/Heif/HeifReferenceViewingEnvironment.cs b/src/ImageSharp/Formats/Heif/HeifReferenceViewingEnvironment.cs new file mode 100644 index 0000000000..899dd9c90a --- /dev/null +++ b/src/ImageSharp/Formats/Heif/HeifReferenceViewingEnvironment.cs @@ -0,0 +1,99 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.ColorProfiles; + +namespace SixLabors.ImageSharp.Formats.Heif; + +/// +/// Describes the display surround and periphery in which a HEIF image was mastered. +/// +public readonly struct HeifReferenceViewingEnvironment : IEquatable +{ + /// + /// Initializes a new instance of the struct. + /// + /// The luminance of the area immediately surrounding the display. + /// The CIE 1931 chromaticity coordinates of the surround light. + /// The luminance of the environment outside the display surround. + /// The CIE 1931 chromaticity coordinates of the periphery light. + public HeifReferenceViewingEnvironment( + double surroundLuminance, + CieXyChromaticityCoordinates surroundLight, + double peripheryLuminance, + CieXyChromaticityCoordinates peripheryLight) + { + this.SurroundLuminance = surroundLuminance; + this.SurroundLight = surroundLight; + this.PeripheryLuminance = peripheryLuminance; + this.PeripheryLight = peripheryLight; + } + + /// + /// Gets the luminance of the area immediately surrounding the display in candelas per square metre. + /// + public double SurroundLuminance { get; } + + /// + /// Gets the CIE 1931 chromaticity coordinates of the surround light. + /// + public CieXyChromaticityCoordinates SurroundLight { get; } + + /// + /// Gets the luminance of the environment outside the display surround in candelas per square metre. + /// + public double PeripheryLuminance { get; } + + /// + /// Gets the CIE 1931 chromaticity coordinates of the periphery light. + /// + public CieXyChromaticityCoordinates PeripheryLight { get; } + + /// + /// Compares two reference viewing environments for equality. + /// + /// The first reference viewing environment. + /// The second reference viewing environment. + /// when every surround and periphery value is equal. + public static bool operator ==(HeifReferenceViewingEnvironment left, HeifReferenceViewingEnvironment right) + => left.Equals(right); + + /// + /// Compares two reference viewing environments for inequality. + /// + /// The first reference viewing environment. + /// The second reference viewing environment. + /// when any surround or periphery value differs. + public static bool operator !=(HeifReferenceViewingEnvironment left, HeifReferenceViewingEnvironment right) + => !left.Equals(right); + + /// + /// Determines whether the specified object is a reference viewing environment with the same values. + /// + /// The object to compare with this value. + /// when contains the same environment values. + public override bool Equals(object? obj) + => obj is HeifReferenceViewingEnvironment other && this.Equals(other); + + /// + /// Determines whether the specified reference viewing environment has the same values as this value. + /// + /// The reference viewing environment to compare with this value. + /// when every surround and periphery value is equal. + public bool Equals(HeifReferenceViewingEnvironment other) + => this.SurroundLuminance.Equals(other.SurroundLuminance) + && this.SurroundLight.Equals(other.SurroundLight) + && this.PeripheryLuminance.Equals(other.PeripheryLuminance) + && this.PeripheryLight.Equals(other.PeripheryLight); + + /// + /// Returns a hash code for this reference viewing environment. + /// + /// A hash code derived from the surround and periphery values. + public override int GetHashCode() + => HashCode.Combine( + this.SurroundLuminance, + this.SurroundLight, + this.PeripheryLuminance, + this.PeripheryLight); +} diff --git a/src/ImageSharp/Formats/Heif/HeifSequence.cs b/src/ImageSharp/Formats/Heif/HeifSequence.cs new file mode 100644 index 0000000000..14938b3cee --- /dev/null +++ b/src/ImageSharp/Formats/Heif/HeifSequence.cs @@ -0,0 +1,38 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif; + +/// +/// Contains the selected color and optional alpha tracks of one HEIF image sequence. +/// +internal sealed class HeifSequence +{ + /// + /// Initializes a new instance of the class. + /// + /// The selected master image-sequence track. + /// The linked alpha image-sequence track, when present. + /// The movie time scale in units per second. + public HeifSequence(HeifSequenceTrack colorTrack, HeifSequenceTrack? alphaTrack, uint movieTimescale) + { + this.ColorTrack = colorTrack; + this.AlphaTrack = alphaTrack; + this.MovieTimescale = movieTimescale; + } + + /// + /// Gets the selected master image-sequence track. + /// + public HeifSequenceTrack ColorTrack { get; } + + /// + /// Gets the linked alpha image-sequence track, when present. + /// + public HeifSequenceTrack? AlphaTrack { get; } + + /// + /// Gets the movie time scale in units per second. + /// + public uint MovieTimescale { get; } +} diff --git a/src/ImageSharp/Formats/Heif/HeifSequenceMetadata.cs b/src/ImageSharp/Formats/Heif/HeifSequenceMetadata.cs new file mode 100644 index 0000000000..e192d2c96d --- /dev/null +++ b/src/ImageSharp/Formats/Heif/HeifSequenceMetadata.cs @@ -0,0 +1,31 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif; + +/// +/// Contains the bounded Exif and XMP item payloads implicitly associated with a HEIF image-sequence track. +/// +internal sealed class HeifSequenceMetadata +{ + /// + /// Initializes a new instance of the class. + /// + /// The complete HEIF Exif item payload, or . + /// The complete XMP packet, or . + public HeifSequenceMetadata(byte[]? exifData, byte[]? xmpData) + { + this.ExifData = exifData; + this.XmpData = xmpData; + } + + /// + /// Gets the complete HEIF Exif item payload, including its TIFF-header offset field. + /// + public byte[]? ExifData { get; } + + /// + /// Gets the raw UTF-8 XMP packet. + /// + public byte[]? XmpData { get; } +} diff --git a/src/ImageSharp/Formats/Heif/HeifSequenceParser.cs b/src/ImageSharp/Formats/Heif/HeifSequenceParser.cs new file mode 100644 index 0000000000..6fff6edf2f --- /dev/null +++ b/src/ImageSharp/Formats/Heif/HeifSequenceParser.cs @@ -0,0 +1,2451 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Buffers; +using System.Buffers.Binary; +using SixLabors.ImageSharp.Formats.Heif.Av1; +using SixLabors.ImageSharp.Memory; +using SixLabors.ImageSharp.Metadata.Profiles.Icc; + +namespace SixLabors.ImageSharp.Formats.Heif; + +/// +/// Parses the bounded track and sample syntax required to identify a HEIF image sequence. +/// +internal sealed class HeifSequenceParser +{ + /// + /// The reusable scratch size used by sequential table reads. + /// + private const int ScratchLength = 4096; + + /// + /// The configured allocator used for parser scratch and bounded table state. + /// + private readonly MemoryAllocator allocator; + + /// + /// The shared reader for variable-length HEIF box headers. + /// + private readonly HeifBoxReader boxReader; + + /// + /// The bounded parser for Exif and XMP items embedded in selected image tracks. + /// + private readonly HeifTrackMetadataParser metadataParser; + + /// + /// The maximum number of sample descriptors retained for decoding or identification. + /// + private readonly int maxFrames; + + /// + /// The general decoder options controlling frame limits, metadata loading, and recoverable segment errors. + /// + private readonly DecoderOptions options; + + /// + /// The absolute stream position at which the current HEIF file begins. + /// + private long fileStartOffset; + + /// + /// Initializes a new instance of the class. + /// + /// The general decoder options. + public HeifSequenceParser(DecoderOptions options) + { + this.options = options; + this.allocator = options.Configuration.MemoryAllocator; + this.boxReader = new HeifBoxReader(this.allocator); + this.metadataParser = new HeifTrackMetadataParser(this.allocator); + this.maxFrames = (int)options.MaxFrames; + } + + /// + /// Parses one movie box and selects its master image-sequence track and linked alpha track. + /// + /// The seekable HEIF stream positioned at the movie payload. + /// The validated movie payload length. + /// The absolute stream position at which the HEIF file begins. + /// The bounded image-sequence model required by the HEIF decoder. + public HeifSequence Parse(Stream stream, long boxLength, long fileStartOffset = 0) + { + this.fileStartOffset = fileStartOffset; + long movieStart = stream.Position; + long movieEnd = checked(movieStart + boxLength); + HeifBoxReader.EnsureInsideParent(boxLength, stream.Length - movieStart); + + using IMemoryOwner scratchOwner = this.allocator.Allocate(ScratchLength); + Span scratch = scratchOwner.GetSpan(); + BoxReference movieHeader = default; + uint colorTrackId = 0; + + // The first pass reads only fixed track identity fields. This prevents files containing unrelated media tracks + // from forcing codec configurations and sample tables into the image decoder's retained model. + while (stream.Position < movieEnd) + { + long childLength = HeifBoxReader.ReadHeader(stream, movieEnd, scratch, out Heif4CharCode childType); + long childStart = stream.Position; + if (childType == Heif4CharCode.Mvhd) + { + SetUnique(ref movieHeader, childStart, childLength, "movie", childType); + } + else if (childType == Heif4CharCode.Trak) + { + TrackIdentity identity = ScanTrackIdentity(stream, childLength, scratch); + if (colorTrackId == 0 + && identity.IsEnabled + && identity.HandlerType == Heif4CharCode.Pict + && identity.AuxiliaryForTrackId == 0) + { + colorTrackId = identity.Id; + } + } + + stream.Position = checked(childStart + childLength); + } + + if (!movieHeader.IsPresent) + { + throw new InvalidImageContentException("The HEIF image sequence has no movie header."); + } + + if (colorTrackId == 0) + { + throw new InvalidImageContentException("The HEIF image sequence has no enabled picture track."); + } + + stream.Position = movieHeader.Offset; + uint movieTimescale = ParseMovieHeader(stream, movieHeader.Length, scratch); + HeifSequenceTrack? colorTrack = null; + HeifSequenceTrack? alphaTrack = null; + + // Track references can precede the master track. Re-scan the bounded movie now that the selected master ID is + // known, and fully parse only that track and the one alpha auxiliary linked to it. + stream.Position = movieStart; + while (stream.Position < movieEnd) + { + long childLength = HeifBoxReader.ReadHeader(stream, movieEnd, scratch, out Heif4CharCode childType); + long childStart = stream.Position; + if (childType == Heif4CharCode.Trak) + { + TrackIdentity identity = ScanTrackIdentity(stream, childLength, scratch); + bool isColor = identity.Id == colorTrackId; + bool isLinkedAuxiliary = identity.AuxiliaryForTrackId == colorTrackId + && identity.HandlerType is Heif4CharCode.Auxv or Heif4CharCode.Pict; + + if (isColor || (isLinkedAuxiliary && alphaTrack is null)) + { + stream.Position = childStart; + HeifSequenceTrack track = this.ParseTrack(stream, childLength, identity, scratch); + if (isColor) + { + colorTrack = track; + } + else if (track.IsAlpha) + { + alphaTrack = track; + } + } + } + + stream.Position = checked(childStart + childLength); + } + + if (colorTrack is null) + { + throw new InvalidImageContentException("The selected HEIF picture track could not be parsed."); + } + + try + { + ValidateAlphaTrack(colorTrack, alphaTrack); + } + catch (Exception ex) when (ImageDecoderCore.ShouldIgnoreImageDataSegmentError(this.options, ex)) + { + // Alpha is optional image data. IgnoreImageData permits a malformed auxiliary sequence to be omitted while + // retaining the independently decodable color presentation. + alphaTrack = null; + colorTrack.IsPremultiplied = false; + } + + return new HeifSequence(colorTrack, alphaTrack, movieTimescale); + } + + /// + /// Reads the fixed identity fields used to select an image track without materializing its sample table. + /// + /// The stream positioned at the track payload. + /// The validated track payload length. + /// The parser-owned reusable scratch span. + /// The track identity and image relationship fields. + private static TrackIdentity ScanTrackIdentity(Stream stream, long boxLength, Span scratch) + { + long trackEnd = checked(stream.Position + boxLength); + BoxReference trackHeader = default; + BoxReference trackReferences = default; + BoxReference media = default; + + while (stream.Position < trackEnd) + { + long childLength = HeifBoxReader.ReadHeader(stream, trackEnd, scratch, out Heif4CharCode childType); + long childStart = stream.Position; + switch (childType) + { + case Heif4CharCode.Tkhd: + SetUnique(ref trackHeader, childStart, childLength, "track", childType); + break; + case Heif4CharCode.Tref: + SetUnique(ref trackReferences, childStart, childLength, "track", childType); + break; + case Heif4CharCode.Mdia: + SetUnique(ref media, childStart, childLength, "track", childType); + break; + } + + stream.Position = checked(childStart + childLength); + } + + if (!media.IsPresent) + { + throw new InvalidImageContentException("A HEIF track is missing its media box."); + } + + stream.Position = media.Offset; + Heif4CharCode handlerType = ScanMediaHandler(stream, media.Length, scratch); + if (handlerType is not Heif4CharCode.Pict and not Heif4CharCode.Auxv) + { + // Non-image tracks are outside this parser's retained ISOBMFF surface. Do not impose image dimensions, + // matrices, or reference semantics on audio and other unrelated media carried by a valid image file. + TrackIdentity unrelatedIdentity = default; + unrelatedIdentity.HandlerType = handlerType; + return unrelatedIdentity; + } + + if (!trackHeader.IsPresent) + { + throw new InvalidImageContentException("A HEIF image-sequence track is missing its track header."); + } + + stream.Position = trackHeader.Offset; + TrackIdentity identity = ParseTrackHeader(stream, trackHeader.Length, scratch); + if (trackReferences.IsPresent) + { + stream.Position = trackReferences.Offset; + ParseTrackReferences(stream, trackReferences.Length, ref identity, scratch); + } + + identity.HandlerType = handlerType; + return identity; + } + + /// + /// Parses the retained behavior of one selected image-sequence track. + /// + /// The stream positioned at the track payload. + /// The validated track payload length. + /// The fixed identity fields from the selection pass. + /// The parser-owned reusable scratch span. + /// The selected track with its validated sample descriptors. + private HeifSequenceTrack ParseTrack(Stream stream, long boxLength, TrackIdentity identity, Span scratch) + { + long trackEnd = checked(stream.Position + boxLength); + BoxReference edit = default; + BoxReference metadata = default; + BoxReference media = default; + + while (stream.Position < trackEnd) + { + long childLength = HeifBoxReader.ReadHeader(stream, trackEnd, scratch, out Heif4CharCode childType); + long childStart = stream.Position; + if (childType == Heif4CharCode.Edts) + { + SetUnique(ref edit, childStart, childLength, "track", childType); + } + else if (childType == Heif4CharCode.Meta) + { + SetUnique(ref metadata, childStart, childLength, "track", childType); + } + else if (childType == Heif4CharCode.Mdia) + { + SetUnique(ref media, childStart, childLength, "track", childType); + } + + stream.Position = checked(childStart + childLength); + } + + if (!media.IsPresent) + { + throw new InvalidImageContentException("A selected HEIF image-sequence track has no media box."); + } + + HeifSequenceTrack track = new() + { + Id = identity.Id, + Width = identity.Width, + Height = identity.Height, + HandlerType = identity.HandlerType, + TrackDuration = identity.TrackDuration, + AuxiliaryForTrackId = identity.AuxiliaryForTrackId, + PremultipliedByTrackId = identity.PremultipliedByTrackId + }; + + if (edit.IsPresent) + { + stream.Position = edit.Offset; + ParseEdit(stream, edit.Length, track, scratch); + } + + if (metadata.IsPresent && !this.options.SkipMetadata) + { + try + { + stream.Position = metadata.Offset; + track.Metadata = this.metadataParser.Parse(stream, metadata.Length, scratch); + } + catch (Exception ex) when (ImageDecoderCore.ShouldIgnoreAncillarySegmentError(this.options, ex)) + { + // The validated parent range lets decoding continue safely without this optional metadata box. + } + } + + stream.Position = media.Offset; + this.ParseMedia(stream, media.Length, track, scratch); + if (track.TotalSampleCount == 0 || track.Samples.Length == 0) + { + throw new InvalidImageContentException("The HEIF image-sequence track contains no retained image samples."); + } + + return track; + } + + /// + /// Parses the movie time scale and validates that no general movie-canvas transformation is required. + /// + /// The stream positioned at the movie-header payload. + /// The validated movie-header payload length. + /// The parser-owned reusable scratch span. + /// The nonzero movie time scale. + private static uint ParseMovieHeader(Stream stream, long boxLength, Span scratch) + { + ReadOnlySpan prefix = ReadPrefix(stream, boxLength, scratch, 4, "movie header"); + byte version = prefix[0]; + int requiredLength = version switch + { + 0 => 100, + 1 => 112, + _ => throw new InvalidImageContentException($"The movie header has unsupported version {version}.") + }; + + prefix = ReadPrefixFromStart(stream, boxLength, scratch, requiredLength, "movie header"); + EnsureZeroFlags(prefix, "movie header"); + uint timescale = BinaryPrimitives.ReadUInt32BigEndian(prefix[(version == 0 ? 12 : 20)..]); + if (timescale == 0) + { + throw new InvalidImageContentException("The movie header has a zero time scale."); + } + + int matrixOffset = version == 0 ? 36 : 48; + HeifTrackMatrix matrix = HeifTrackMatrix.Parse(prefix.Slice(matrixOffset, 36)); + if (!matrix.IsIdentity) + { + throw new NotSupportedException("The HEIF image sequence requires an unsupported movie presentation matrix."); + } + + return timescale; + } + + /// + /// Parses track identity, dimensions, duration, and transformation matrix. + /// + /// The stream positioned at the track-header payload. + /// The validated track-header payload length. + /// The parser-owned reusable scratch span. + /// The fixed track identity fields. + private static TrackIdentity ParseTrackHeader(Stream stream, long boxLength, Span scratch) + { + const int fullBoxHeaderLength = sizeof(uint); + const int trackIdAndReservedLength = 2 * sizeof(uint); + const int postDurationFieldsLength = (2 * sizeof(uint)) + (4 * sizeof(ushort)); + const int matrixLength = 9 * sizeof(int); + const int dimensionsLength = 2 * sizeof(uint); + const int fixedPointFractionalBits = 16; + const uint trackEnabledFlag = 1 << 0; + + ReadOnlySpan prefix = ReadPrefix(stream, boxLength, scratch, fullBoxHeaderLength, "track header"); + byte version = prefix[0]; + + // ISO/IEC 14496-12, Section 8.3.2 defines 'tkhd' as a FullBox followed by creation and modification + // times, the track identifier, a reserved field, and the duration. Version 1 widens each time and duration + // field from 32 to 64 bits, which shifts every field that follows them by twelve bytes. + int versionedFieldLength = version switch + { + 0 => sizeof(uint), + 1 => sizeof(ulong), + _ => throw new InvalidImageContentException($"The track header has unsupported version {version}.") + }; + + int trackIdOffset = fullBoxHeaderLength + (2 * versionedFieldLength); + int durationOffset = trackIdOffset + trackIdAndReservedLength; + + // The fields between duration and matrix are two reserved 32-bit values followed by the 16-bit layer, + // alternate_group, volume, and reserved values specified by Section 8.3.2. The matrix then contains nine + // 32-bit fixed-point coefficients, followed by the two 32-bit track dimensions. + int matrixOffset = durationOffset + versionedFieldLength + postDurationFieldsLength; + int widthOffset = matrixOffset + matrixLength; + int requiredLength = widthOffset + dimensionsLength; + + prefix = ReadPrefixFromStart(stream, boxLength, scratch, requiredLength, "track header"); + uint flags = ReadFlags(prefix); + uint id = BinaryPrimitives.ReadUInt32BigEndian(prefix[trackIdOffset..]); + + // Section 8.3.2 reserves track_ID zero, so accepting it would make track references ambiguous. + if (id == 0) + { + throw new InvalidImageContentException("A HEIF image-sequence track has identifier zero."); + } + + ulong duration = version == 0 + ? BinaryPrimitives.ReadUInt32BigEndian(prefix[durationOffset..]) + : BinaryPrimitives.ReadUInt64BigEndian(prefix[durationOffset..]); + + // The specification uses the all-ones value for an indefinite duration. Normalize the 32-bit version to + // the 64-bit sentinel used by the sequence model so both TrackHeaderBox versions follow the same path. + if (version == 0 && duration == uint.MaxValue) + { + duration = ulong.MaxValue; + } + + // Section 8.3.2 stores width and height as unsigned 16.16 fixed-point values. ImageSharp dimensions are + // integral pixels, matching libavif, so discard the fractional half before validating the display size. + uint fixedWidth = BinaryPrimitives.ReadUInt32BigEndian(prefix[widthOffset..]); + uint fixedHeight = BinaryPrimitives.ReadUInt32BigEndian(prefix[(widthOffset + sizeof(uint))..]); + int width = checked((int)(fixedWidth >> fixedPointFractionalBits)); + int height = checked((int)(fixedHeight >> fixedPointFractionalBits)); + + if (width == 0 || height == 0) + { + throw new InvalidImageContentException("A HEIF image-sequence track has zero dimensions."); + } + + HeifTrackMatrix matrix = HeifTrackMatrix.Parse(prefix.Slice(matrixOffset, matrixLength)); + + // The Section 8.3.2 matrix transforms the track into the movie presentation coordinate system. This image + // decoder currently emits the stored raster directly, so a non-unity matrix would produce incorrect pixels. + if (!matrix.IsIdentity) + { + throw new NotSupportedException("The HEIF image-sequence track requires an unsupported movie presentation matrix."); + } + + // ISO/IEC 14496-12 Section 8.3.2 assigns bit zero to track_enabled. Image sequences, including files written + // by libavif, do not require track_in_movie to be set, so only the enabled bit participates in selection. + bool isEnabled = (flags & trackEnabledFlag) != 0; + + return new TrackIdentity(id, isEnabled, width, height, duration); + } + + /// + /// Parses the image-specific track references used for alpha and premultiplication linkage. + /// + /// The stream positioned at the track-reference payload. + /// The validated track-reference payload length. + /// The track identity receiving image relationships. + /// The parser-owned reusable scratch span. + private static void ParseTrackReferences(Stream stream, long boxLength, ref TrackIdentity identity, Span scratch) + { + long referenceEnd = checked(stream.Position + boxLength); + while (stream.Position < referenceEnd) + { + long childLength = HeifBoxReader.ReadHeader(stream, referenceEnd, scratch, out Heif4CharCode childType); + long childStart = stream.Position; + if (childType is Heif4CharCode.Auxl or Heif4CharCode.Prem) + { + if (childLength < 4 || (childLength & 3) != 0) + { + throw new InvalidImageContentException($"The '{childType}' track reference has an invalid identifier list."); + } + + ReadOnlySpan data = ReadPrefix(stream, childLength, scratch, 4, $"{childType} track reference"); + uint referencedTrackId = BinaryPrimitives.ReadUInt32BigEndian(data); + if (referencedTrackId == 0) + { + throw new InvalidImageContentException($"The '{childType}' track reference contains identifier zero."); + } + + if (childType == Heif4CharCode.Auxl) + { + if (identity.AuxiliaryForTrackId != 0) + { + throw new InvalidImageContentException("The track contains duplicate alpha-auxiliary references."); + } + + identity.AuxiliaryForTrackId = referencedTrackId; + } + else + { + if (identity.PremultipliedByTrackId != 0) + { + throw new InvalidImageContentException("The track contains duplicate premultiplication references."); + } + + identity.PremultipliedByTrackId = referencedTrackId; + } + } + + stream.Position = checked(childStart + childLength); + } + } + + /// + /// Finds and parses the handler type from one media box. + /// + /// The stream positioned at the media payload. + /// The validated media payload length. + /// The parser-owned reusable scratch span. + /// The declared media handler type. + private static Heif4CharCode ScanMediaHandler(Stream stream, long boxLength, Span scratch) + { + long mediaEnd = checked(stream.Position + boxLength); + BoxReference handler = default; + while (stream.Position < mediaEnd) + { + long childLength = HeifBoxReader.ReadHeader(stream, mediaEnd, scratch, out Heif4CharCode childType); + long childStart = stream.Position; + if (childType == Heif4CharCode.Hdlr) + { + SetUnique(ref handler, childStart, childLength, "media", childType); + } + + stream.Position = checked(childStart + childLength); + } + + if (!handler.IsPresent) + { + throw new InvalidImageContentException("A HEIF image-sequence media box has no handler."); + } + + stream.Position = handler.Offset; + return ParseHandler(stream, handler.Length, scratch); + } + + /// + /// Parses the media header and sample table of a selected image track. + /// + /// The stream positioned at the media payload. + /// The validated media payload length. + /// The selected track receiving media state. + /// The parser-owned reusable scratch span. + private void ParseMedia(Stream stream, long boxLength, HeifSequenceTrack track, Span scratch) + { + long mediaEnd = checked(stream.Position + boxLength); + BoxReference mediaHeader = default; + BoxReference handler = default; + BoxReference mediaInformation = default; + + while (stream.Position < mediaEnd) + { + long childLength = HeifBoxReader.ReadHeader(stream, mediaEnd, scratch, out Heif4CharCode childType); + long childStart = stream.Position; + switch (childType) + { + case Heif4CharCode.Mdhd: + SetUnique(ref mediaHeader, childStart, childLength, "media", childType); + break; + case Heif4CharCode.Hdlr: + SetUnique(ref handler, childStart, childLength, "media", childType); + break; + case Heif4CharCode.Minf: + SetUnique(ref mediaInformation, childStart, childLength, "media", childType); + break; + } + + stream.Position = checked(childStart + childLength); + } + + if (!mediaHeader.IsPresent || !handler.IsPresent || !mediaInformation.IsPresent) + { + throw new InvalidImageContentException("A selected HEIF image-sequence media box is incomplete."); + } + + stream.Position = mediaHeader.Offset; + ParseMediaHeader(stream, mediaHeader.Length, track, scratch); + stream.Position = handler.Offset; + Heif4CharCode handlerType = ParseHandler(stream, handler.Length, scratch); + if (handlerType != track.HandlerType) + { + throw new InvalidImageContentException("The HEIF image-sequence track handler changed between parser passes."); + } + + stream.Position = mediaInformation.Offset; + this.ParseMediaInformation(stream, mediaInformation.Length, track, scratch); + } + + /// + /// Parses the media time scale and duration of a selected track. + /// + /// The stream positioned at the media-header payload. + /// The validated media-header payload length. + /// The selected track receiving timing state. + /// The parser-owned reusable scratch span. + private static void ParseMediaHeader(Stream stream, long boxLength, HeifSequenceTrack track, Span scratch) + { + ReadOnlySpan prefix = ReadPrefix(stream, boxLength, scratch, 4, "media header"); + byte version = prefix[0]; + int requiredLength = version switch + { + 0 => 24, + 1 => 36, + _ => throw new InvalidImageContentException($"The media header has unsupported version {version}.") + }; + + prefix = ReadPrefixFromStart(stream, boxLength, scratch, requiredLength, "media header"); + EnsureZeroFlags(prefix, "media header"); + int timescaleOffset = version == 0 ? 12 : 20; + track.MediaTimescale = BinaryPrimitives.ReadUInt32BigEndian(prefix[timescaleOffset..]); + track.MediaDuration = version == 0 + ? BinaryPrimitives.ReadUInt32BigEndian(prefix[(timescaleOffset + 4)..]) + : BinaryPrimitives.ReadUInt64BigEndian(prefix[(timescaleOffset + 4)..]); + + if (track.MediaTimescale == 0) + { + throw new InvalidImageContentException("The HEIF image-sequence media header has a zero time scale."); + } + } + + /// + /// Parses a media handler and returns its four-character handler type. + /// + /// The stream positioned at the handler payload. + /// The validated handler payload length. + /// The parser-owned reusable scratch span. + /// The declared handler type. + private static Heif4CharCode ParseHandler(Stream stream, long boxLength, Span scratch) + { + ReadOnlySpan prefix = ReadPrefix(stream, boxLength, scratch, 24, "handler"); + EnsureVersionAndFlags(prefix, 0, 0, "handler"); + if (BinaryPrimitives.ReadUInt32BigEndian(prefix[4..]) != 0) + { + throw new InvalidImageContentException("The HEIF image-sequence handler has a nonzero predefined field."); + } + + return (Heif4CharCode)BinaryPrimitives.ReadUInt32BigEndian(prefix[8..]); + } + + /// + /// Locates the self-contained data reference and sample table inside a selected track. + /// + /// The stream positioned at the media-information payload. + /// The validated media-information payload length. + /// The selected track receiving its sample table. + /// The parser-owned reusable scratch span. + private void ParseMediaInformation(Stream stream, long boxLength, HeifSequenceTrack track, Span scratch) + { + long informationEnd = checked(stream.Position + boxLength); + BoxReference dataInformation = default; + BoxReference sampleTable = default; + while (stream.Position < informationEnd) + { + long childLength = HeifBoxReader.ReadHeader(stream, informationEnd, scratch, out Heif4CharCode childType); + long childStart = stream.Position; + if (childType == Heif4CharCode.Dinf) + { + SetUnique(ref dataInformation, childStart, childLength, "media information", childType); + } + else if (childType == Heif4CharCode.Stbl) + { + SetUnique(ref sampleTable, childStart, childLength, "media information", childType); + } + + stream.Position = checked(childStart + childLength); + } + + if (!dataInformation.IsPresent || !sampleTable.IsPresent) + { + throw new InvalidImageContentException("A selected HEIF image-sequence track has no data reference or sample table."); + } + + stream.Position = dataInformation.Offset; + ParseDataInformation(stream, dataInformation.Length, scratch); + stream.Position = sampleTable.Offset; + this.ParseSampleTable(stream, sampleTable.Length, track, scratch); + } + + /// + /// Requires a selected image track to address sample bytes in the current HEIF file. + /// + /// The stream positioned at the data-information payload. + /// The validated data-information payload length. + /// The parser-owned reusable scratch span. + private static void ParseDataInformation(Stream stream, long boxLength, Span scratch) + { + long informationEnd = checked(stream.Position + boxLength); + BoxReference dataReference = default; + while (stream.Position < informationEnd) + { + long childLength = HeifBoxReader.ReadHeader(stream, informationEnd, scratch, out Heif4CharCode childType); + long childStart = stream.Position; + if (childType == Heif4CharCode.Dref) + { + SetUnique(ref dataReference, childStart, childLength, "data information", childType); + } + + stream.Position = checked(childStart + childLength); + } + + if (!dataReference.IsPresent) + { + throw new InvalidImageContentException("A selected HEIF image-sequence track has no data-reference box."); + } + + stream.Position = dataReference.Offset; + long referenceEnd = checked(stream.Position + dataReference.Length); + ReadOnlySpan prefix = ReadPrefix(stream, dataReference.Length, scratch, 8, "data reference"); + EnsureVersionAndFlags(prefix, 0, 0, "data reference"); + if (BinaryPrimitives.ReadUInt32BigEndian(prefix[4..]) != 1) + { + throw new InvalidImageContentException("A HEIF image-sequence track must contain exactly one data reference."); + } + + long locationLength = HeifBoxReader.ReadHeader(stream, referenceEnd, scratch, out Heif4CharCode locationType); + if (locationType != Heif4CharCode.Url || locationLength != 4) + { + throw new InvalidImageContentException("A HEIF image-sequence track uses an external data reference."); + } + + prefix = ReadPrefix(stream, locationLength, scratch, 4, "data location"); + EnsureVersionAndFlags(prefix, 0, 1, "data location"); + if (stream.Position != referenceEnd) + { + throw new InvalidImageContentException("The data-reference box contains undeclared entries."); + } + } + + /// + /// Indexes and resolves the bounded sample-table boxes required by an image sequence. + /// + /// The stream positioned at the sample-table payload. + /// The validated sample-table payload length. + /// The selected track receiving sample descriptors. + /// The parser-owned reusable scratch span. + private void ParseSampleTable(Stream stream, long boxLength, HeifSequenceTrack track, Span scratch) + { + long tableEnd = checked(stream.Position + boxLength); + BoxReference sampleDescription = default; + BoxReference sampleTiming = default; + BoxReference sampleToChunk = default; + BoxReference sampleSizes = default; + BoxReference chunkOffsets = default; + BoxReference syncSamples = default; + BoxReference compositionOffsets = default; + BoxReference compositionToDecode = default; + BoxReference sampleGroupDescriptions = default; + BoxReference sampleToGroup = default; + + while (stream.Position < tableEnd) + { + long childLength = HeifBoxReader.ReadHeader(stream, tableEnd, scratch, out Heif4CharCode childType); + long childStart = stream.Position; + switch (childType) + { + case Heif4CharCode.Stsd: + SetUnique(ref sampleDescription, childStart, childLength, "sample table", childType); + break; + case Heif4CharCode.Stts: + SetUnique(ref sampleTiming, childStart, childLength, "sample table", childType); + break; + case Heif4CharCode.Stsc: + SetUnique(ref sampleToChunk, childStart, childLength, "sample table", childType); + break; + case Heif4CharCode.Stsz: + case Heif4CharCode.Stz2: + SetUnique(ref sampleSizes, childStart, childLength, "sample table", childType); + sampleSizes.Type = childType; + break; + case Heif4CharCode.Stco: + case Heif4CharCode.Co64: + SetUnique(ref chunkOffsets, childStart, childLength, "sample table", childType); + chunkOffsets.Type = childType; + break; + case Heif4CharCode.Stss: + SetUnique(ref syncSamples, childStart, childLength, "sample table", childType); + break; + case Heif4CharCode.Ctts: + SetUnique(ref compositionOffsets, childStart, childLength, "sample table", childType); + break; + case Heif4CharCode.Cslg: + SetUnique(ref compositionToDecode, childStart, childLength, "sample table", childType); + break; + case Heif4CharCode.Sgpd: + if (ReadSampleGroupType(stream, childLength, scratch) == Heif4CharCode.Refs) + { + SetUnique(ref sampleGroupDescriptions, childStart, childLength, "sample table", childType); + } + + break; + case Heif4CharCode.Sbgp: + if (ReadSampleGroupType(stream, childLength, scratch) == Heif4CharCode.Refs) + { + SetUnique(ref sampleToGroup, childStart, childLength, "sample table", childType); + } + + break; + } + + stream.Position = checked(childStart + childLength); + } + + if (!sampleDescription.IsPresent + || !sampleTiming.IsPresent + || !sampleToChunk.IsPresent + || !sampleSizes.IsPresent + || !chunkOffsets.IsPresent) + { + throw new InvalidImageContentException("A selected HEIF image-sequence sample table is incomplete."); + } + + stream.Position = sampleDescription.Offset; + this.ParseSampleDescription(stream, sampleDescription.Length, track, scratch); + stream.Position = sampleSizes.Offset; + this.ParseSampleSizes(stream, sampleSizes.Length, sampleSizes.Type, track, scratch); + stream.Position = sampleTiming.Offset; + ulong decodedDuration = ParseSampleTiming(stream, sampleTiming.Length, track, scratch); + + if (track.MediaDuration != 0 && track.MediaDuration != ulong.MaxValue && track.MediaDuration != decodedDuration) + { + throw new InvalidImageContentException("The image-sequence sample durations do not match the media-header duration."); + } + + stream.Position = chunkOffsets.Offset; + uint chunkCount = ReadChunkCount(stream, chunkOffsets.Length, chunkOffsets.Type, scratch); + stream.Position = sampleToChunk.Offset; + using IMemoryOwner entries = this.ParseSampleToChunk( + stream, + sampleToChunk.Length, + chunkCount, + track, + scratch, + out int entryCount); + + stream.Position = chunkOffsets.Offset; + this.ResolveSampleLocations(stream, chunkOffsets.Length, chunkOffsets.Type, chunkCount, entries.GetSpan()[..entryCount], track, scratch); + if (syncSamples.IsPresent) + { + stream.Position = syncSamples.Offset; + ParseSyncSamples(stream, syncSamples.Length, track, scratch); + } + else + { + Span samples = track.Samples; + for (int i = 0; i < samples.Length; i++) + { + samples[i].IsSync = true; + } + } + + if (sampleGroupDescriptions.IsPresent != sampleToGroup.IsPresent) + { + throw new InvalidImageContentException("The direct-reference sample group is missing its description or sample map."); + } + + if (sampleGroupDescriptions.IsPresent) + { + this.ParseDirectReferences(stream, sampleGroupDescriptions, sampleToGroup, track, scratch); + } + + if (compositionOffsets.IsPresent || compositionToDecode.IsPresent) + { + // AV1-ISOBMFF defines AV1 sample composition time as decode time and explicitly prohibits both boxes. + throw new InvalidImageContentException("An AV1 image-sequence track contains prohibited composition timing boxes."); + } + + SetCompositionTimes(track); + } + + /// + /// Parses the single visual sample entry and its codec and coding-constraint children. + /// + /// The stream positioned at the sample-description payload. + /// The validated sample-description payload length. + /// The selected track receiving its codec configuration. + /// The parser-owned reusable scratch span. + private void ParseSampleDescription(Stream stream, long boxLength, HeifSequenceTrack track, Span scratch) + { + long descriptionEnd = checked(stream.Position + boxLength); + ReadOnlySpan prefix = ReadPrefix(stream, boxLength, scratch, 8, "sample description"); + byte version = prefix[0]; + if (version is not 0 and not 1 || ReadFlags(prefix) != 0) + { + throw new InvalidImageContentException("The sample-description box has an unsupported version or flags."); + } + + if (BinaryPrimitives.ReadUInt32BigEndian(prefix[4..]) != 1) + { + throw new InvalidImageContentException("A HEIF image-sequence track must contain exactly one sample description."); + } + + long entryLength = HeifBoxReader.ReadHeader(stream, descriptionEnd, scratch, out Heif4CharCode entryType); + if (entryType != Heif4CharCode.Av01 || entryLength < 78) + { + throw new InvalidImageContentException($"The image-sequence sample entry '{entryType}' is unsupported or truncated."); + } + + long entryEnd = checked(stream.Position + entryLength); + prefix = ReadPrefix(stream, entryLength, scratch, 78, "visual sample entry"); + if (BinaryPrimitives.ReadUInt16BigEndian(prefix[6..]) != 1) + { + throw new InvalidImageContentException("The image-sequence sample entry uses a nonlocal data reference."); + } + + int codedWidth = BinaryPrimitives.ReadUInt16BigEndian(prefix[24..]); + int codedHeight = BinaryPrimitives.ReadUInt16BigEndian(prefix[26..]); + if (codedWidth == 0 || codedHeight == 0) + { + throw new InvalidImageContentException("The image-sequence sample entry has zero coded dimensions."); + } + + track.CodecType = entryType; + track.CodedWidth = codedWidth; + track.CodedHeight = codedHeight; + bool configurationSeen = false; + bool codingConstraintsSeen = false; + bool auxiliaryTypeSeen = false; + while (stream.Position < entryEnd) + { + long childLength = HeifBoxReader.ReadHeader(stream, entryEnd, scratch, out Heif4CharCode childType); + long childStart = stream.Position; + switch (childType) + { + case Heif4CharCode.Av1C when entryType == Heif4CharCode.Av01: + if (configurationSeen) + { + throw new InvalidImageContentException("The AV1 image-sequence sample entry has duplicate codec configurations."); + } + + using (IMemoryOwner configuration = this.boxReader.ReadPayload(stream, childLength)) + { + track.Av1CodecConfiguration = new Av1CodecConfiguration(configuration.GetSpan(), this.options); + } + + configurationSeen = true; + break; + case Heif4CharCode.Ccst: + if (codingConstraintsSeen) + { + throw new InvalidImageContentException("The image-sequence sample entry has duplicate coding constraints."); + } + + ParseCodingConstraints(stream, childLength, track, scratch); + codingConstraintsSeen = true; + break; + case Heif4CharCode.Auxi: + if (auxiliaryTypeSeen) + { + throw new InvalidImageContentException("The image-sequence sample entry has duplicate auxiliary types."); + } + + track.IsAlpha = this.ParseAuxiliaryType(stream, childLength); + auxiliaryTypeSeen = true; + break; + case Heif4CharCode.Colr: + this.ParseTrackColorInformation(stream, childLength, track, scratch); + break; + case Heif4CharCode.Pasp: + case Heif4CharCode.Clli: + case Heif4CharCode.Mdcv: + case Heif4CharCode.Cclv: + case Heif4CharCode.Amve: + case Heif4CharCode.Reve: + case Heif4CharCode.Ndwt: + if (!this.options.SkipMetadata) + { + try + { + ParseTrackImageProperty(stream, childLength, childType, track, scratch); + } + catch (Exception ex) when (ImageDecoderCore.ShouldIgnoreAncillarySegmentError(this.options, ex)) + { + // The complete child range remains known, so optional metadata can be discarded safely. + } + } + + break; + case Heif4CharCode.Clap: + case Heif4CharCode.Irot: + case Heif4CharCode.Imir: + try + { + ParseTrackImageProperty(stream, childLength, childType, track, scratch); + } + catch (Exception ex) when (ImageDecoderCore.ShouldIgnoreImageDataSegmentError(this.options, ex)) + { + // IgnoreImageData permits a recoverable presentation property to be omitted. + } + + break; + } + + stream.Position = checked(childStart + childLength); + } + + if (stream.Position != descriptionEnd || !configurationSeen || !codingConstraintsSeen) + { + throw new InvalidImageContentException("The image-sequence sample description is incomplete or has trailing entries."); + } + } + + /// + /// Parses one presentation, color, or HDR property carried by a visual sample entry. + /// + /// The stream positioned at the property payload. + /// The validated property payload length. + /// The registered image property type. + /// The selected image track receiving the property. + /// The parser-owned reusable scratch span. + private static void ParseTrackImageProperty( + Stream stream, + long boxLength, + Heif4CharCode boxType, + HeifSequenceTrack track, + Span scratch) + { + ReadOnlySpan data = ReadPropertyPayload(stream, boxLength, scratch, boxType); + switch (boxType) + { + case Heif4CharCode.Pasp: + if (track.PixelAspectRatio is not null) + { + throw new InvalidImageContentException("The image-sequence sample entry has duplicate pixel-aspect-ratio properties."); + } + + track.PixelAspectRatio = HeifPropertyParser.ParsePixelAspectRatio(data); + break; + case Heif4CharCode.Clli: + if (track.ContentLightLevel is not null) + { + throw new InvalidImageContentException("The image-sequence sample entry has duplicate content-light-level properties."); + } + + track.ContentLightLevel = HeifPropertyParser.ParseContentLightLevel(data); + break; + case Heif4CharCode.Mdcv: + if (track.MasteringDisplayColorVolume is not null) + { + throw new InvalidImageContentException("The image-sequence sample entry has duplicate mastering-display properties."); + } + + track.MasteringDisplayColorVolume = HeifPropertyParser.ParseMasteringDisplayColorVolume(data); + break; + case Heif4CharCode.Cclv: + if (track.ContentColorVolume is not null) + { + throw new InvalidImageContentException("The image-sequence sample entry has duplicate content-color-volume properties."); + } + + track.ContentColorVolume = HeifPropertyParser.ParseContentColorVolume(data); + break; + case Heif4CharCode.Amve: + if (track.AmbientViewingEnvironment is not null) + { + throw new InvalidImageContentException("The image-sequence sample entry has duplicate ambient-viewing properties."); + } + + track.AmbientViewingEnvironment = HeifPropertyParser.ParseAmbientViewingEnvironment(data); + break; + case Heif4CharCode.Reve: + if (track.ReferenceViewingEnvironment is not null) + { + throw new InvalidImageContentException("The image-sequence sample entry has duplicate reference-viewing properties."); + } + + track.ReferenceViewingEnvironment = HeifPropertyParser.ParseReferenceViewingEnvironment(data); + break; + case Heif4CharCode.Ndwt: + if (track.NominalDiffuseWhite is not null) + { + throw new InvalidImageContentException("The image-sequence sample entry has duplicate nominal-diffuse-white properties."); + } + + track.NominalDiffuseWhite = HeifPropertyParser.ParseNominalDiffuseWhite(data); + break; + case Heif4CharCode.Clap: + if (track.CleanAperture is not null) + { + throw new InvalidImageContentException("The image-sequence sample entry has duplicate clean-aperture properties."); + } + + HeifCleanAperture cleanAperture = HeifPropertyParser.ParseCleanAperture(data); + _ = cleanAperture.ToRectangle(new Size(track.CodedWidth, track.CodedHeight)); + track.CleanAperture = cleanAperture; + break; + case Heif4CharCode.Irot: + if (track.RotationAngle is not null) + { + throw new InvalidImageContentException("The image-sequence sample entry has duplicate rotation properties."); + } + + track.RotationAngle = HeifPropertyParser.ParseRotation(data); + break; + case Heif4CharCode.Imir: + if (track.MirrorAxis is not null) + { + throw new InvalidImageContentException("The image-sequence sample entry has duplicate mirror properties."); + } + + track.MirrorAxis = HeifPropertyParser.ParseMirrorAxis(data); + break; + } + } + + /// + /// Parses one ICC or CICP color-information property from a visual sample entry. + /// + /// The stream positioned at the color-information payload. + /// The validated color-information payload length. + /// The selected image track receiving the color description. + /// The parser-owned reusable scratch span. + private void ParseTrackColorInformation(Stream stream, long boxLength, HeifSequenceTrack track, Span scratch) + { + ReadOnlySpan prefix = ReadPrefix(stream, boxLength, scratch, 4, "color information"); + Heif4CharCode profileType = (Heif4CharCode)BinaryPrimitives.ReadUInt32BigEndian(prefix); + if (profileType == Heif4CharCode.Nclx) + { + try + { + if (track.CicpProfile is not null) + { + throw new InvalidImageContentException("The image-sequence sample entry has duplicate CICP color properties."); + } + + if (boxLength != 11) + { + throw new InvalidImageContentException("The CICP color-information property has an invalid length."); + } + + prefix = ReadPrefixFromStart(stream, boxLength, scratch, 11, "color information"); + track.CicpProfile = HeifPropertyParser.ParseCicpProfile(prefix[4..]); + } + catch (Exception ex) when (ImageDecoderCore.ShouldIgnoreImageDataSegmentError(this.options, ex)) + { + // IgnoreImageData permits the decoder to fall back to the coded sequence's color description. + } + } + else if ((profileType is Heif4CharCode.RICC or Heif4CharCode.Prof) && !this.options.SkipMetadata) + { + try + { + if (track.IccProfile is not null) + { + throw new InvalidImageContentException("The image-sequence sample entry has duplicate ICC color properties."); + } + + if (boxLength <= 4 || boxLength > int.MaxValue) + { + throw new InvalidImageContentException("The ICC color-information property is empty or too large."); + } + + // Read directly into the array retained by IccProfile so the generic box buffer cannot create a + // second full-sized copy of the profile at this ownership boundary. + byte[] profileData = new byte[(int)boxLength - 4]; + HeifBoxReader.ReadExactly(stream, profileData, "Stream length is not sufficient for box content."); + track.IccProfile = HeifPropertyParser.ParseIccProfile(profileData); + } + catch (Exception ex) when (ImageDecoderCore.ShouldIgnoreAncillarySegmentError(this.options, ex)) + { + // A malformed optional ICC profile does not invalidate the coded image outside strict mode. + } + } + } + + /// + /// Reads a bounded fixed-size image property through the parser's reusable scratch buffer. + /// + /// The stream positioned at the property payload. + /// The validated property payload length. + /// The parser-owned reusable scratch span. + /// The property type used in malformed-image diagnostics. + /// The complete property payload within . + private static ReadOnlySpan ReadPropertyPayload(Stream stream, long boxLength, Span scratch, Heif4CharCode boxType) + { + if (boxLength > scratch.Length) + { + throw new InvalidImageContentException($"The '{boxType}' image-sequence property exceeds its registered bounded size."); + } + + return ReadPrefix(stream, boxLength, scratch, (int)boxLength, $"{boxType} image-sequence property"); + } + + /// + /// Parses coding constraints that bound inter-picture references for an image sequence. + /// + /// The stream positioned at the coding-constraints payload. + /// The validated coding-constraints payload length. + /// The selected track receiving coding constraints. + /// The parser-owned reusable scratch span. + private static void ParseCodingConstraints(Stream stream, long boxLength, HeifSequenceTrack track, Span scratch) + { + ReadOnlySpan data = ReadPrefix(stream, boxLength, scratch, 8, "coding constraints"); + if (boxLength != 8) + { + throw new InvalidImageContentException("The image-sequence coding-constraints box has an invalid length."); + } + + EnsureVersionAndFlags(data, 0, 0, "coding constraints"); + uint constraints = BinaryPrimitives.ReadUInt32BigEndian(data[4..]); + if ((constraints & 0x03FFFFFF) != 0) + { + throw new InvalidImageContentException("The image-sequence coding constraints contain nonzero reserved bits."); + } + + track.AllReferencePicturesIntra = (constraints & 0x80000000) != 0; + track.IntraPicturePredictionUsed = (constraints & 0x40000000) != 0; + track.MaximumReferencesPerPicture = (byte)((constraints >> 26) & 15); + } + + /// + /// Determines whether an auxiliary-track type identifies an alpha image sequence. + /// + /// The stream positioned at the auxiliary-type payload. + /// The validated auxiliary-type payload length. + /// when the payload contains the registered HEIF alpha URN. + private bool ParseAuxiliaryType(Stream stream, long boxLength) + { + if (boxLength < 5 || boxLength > int.MaxValue) + { + throw new InvalidImageContentException("The image-sequence auxiliary type is truncated or too large."); + } + + using IMemoryOwner payload = this.boxReader.ReadPayload(stream, boxLength); + ReadOnlySpan data = payload.GetSpan(); + EnsureVersionAndFlags(data, 0, 0, "auxiliary type"); + ReadOnlySpan type = data[4..]; + if (type[^1] != 0) + { + throw new InvalidImageContentException("The image-sequence auxiliary type is not null terminated."); + } + + type = type[..^1]; + return type.SequenceEqual("urn:mpeg:mpegB:cicp:systems:auxiliary:alpha"u8); + } + + /// + /// Parses either full-width or compact sample sizes into the retained descriptor array. + /// + /// The stream positioned at the sample-size payload. + /// The validated sample-size payload length. + /// The full-width or compact sample-size box type. + /// The selected track receiving retained sample lengths. + /// The parser-owned reusable scratch span. + private void ParseSampleSizes(Stream stream, long boxLength, Heif4CharCode boxType, HeifSequenceTrack track, Span scratch) + { + if (boxType == Heif4CharCode.Stsz) + { + this.ParseFullSampleSizes(stream, boxLength, track, scratch); + } + else + { + this.ParseCompactSampleSizes(stream, boxLength, track, scratch); + } + } + + /// + /// Parses a full-width sample-size table without retaining entries beyond . + /// + /// The stream positioned at the sample-size payload. + /// The validated sample-size payload length. + /// The selected track receiving retained sample lengths. + /// The parser-owned reusable scratch span. + private void ParseFullSampleSizes(Stream stream, long boxLength, HeifSequenceTrack track, Span scratch) + { + ReadOnlySpan prefix = ReadPrefix(stream, boxLength, scratch, 12, "sample sizes"); + EnsureVersionAndFlags(prefix, 0, 0, "sample sizes"); + uint constantSize = BinaryPrimitives.ReadUInt32BigEndian(prefix[4..]); + uint sampleCount = BinaryPrimitives.ReadUInt32BigEndian(prefix[8..]); + if (sampleCount == 0) + { + throw new InvalidImageContentException("The image-sequence sample-size table is empty."); + } + + long entryBytes = constantSize == 0 ? checked((long)sampleCount * 4) : 0; + if (boxLength != 12 + entryBytes) + { + throw new InvalidImageContentException("The image-sequence sample-size table length does not match its entry count."); + } + + int retainedCount = (int)Math.Min(sampleCount, (uint)this.maxFrames); + track.TotalSampleCount = sampleCount; + track.Samples = new HeifSequenceSample[retainedCount]; + if (constantSize != 0) + { + int size = ValidateSampleSize(constantSize); + Span samples = track.Samples; + for (int i = 0; i < samples.Length; i++) + { + samples[i].Length = size; + } + + return; + } + + HeifBoxPayloadReader reader = new(stream, entryBytes, scratch, "sample sizes"); + for (uint i = 0; i < sampleCount; i++) + { + int size = ValidateSampleSize(reader.ReadUInt32()); + if (i < retainedCount) + { + track.Samples[(int)i].Length = size; + } + } + } + + /// + /// Parses a compact sample-size table without expanding entries beyond . + /// + /// The stream positioned at the compact sample-size payload. + /// The validated compact sample-size payload length. + /// The selected track receiving retained sample lengths. + /// The parser-owned reusable scratch span. + private void ParseCompactSampleSizes(Stream stream, long boxLength, HeifSequenceTrack track, Span scratch) + { + ReadOnlySpan prefix = ReadPrefix(stream, boxLength, scratch, 12, "compact sample sizes"); + EnsureVersionAndFlags(prefix, 0, 0, "compact sample sizes"); + if (prefix[4] != 0 || prefix[5] != 0 || prefix[6] != 0 || prefix[7] is not 4 and not 8 and not 16) + { + throw new InvalidImageContentException("The compact sample-size table has invalid reserved fields or field width."); + } + + int fieldSize = prefix[7]; + uint sampleCount = BinaryPrimitives.ReadUInt32BigEndian(prefix[8..]); + if (sampleCount == 0) + { + throw new InvalidImageContentException("The compact sample-size table is empty."); + } + + long entryBytes = checked((((long)sampleCount * fieldSize) + 7) / 8); + if (boxLength != 12 + entryBytes) + { + throw new InvalidImageContentException("The compact sample-size table length does not match its entry count."); + } + + int retainedCount = (int)Math.Min(sampleCount, (uint)this.maxFrames); + track.TotalSampleCount = sampleCount; + track.Samples = new HeifSequenceSample[retainedCount]; + HeifBoxPayloadReader reader = new(stream, entryBytes, scratch, "compact sample sizes"); + for (uint i = 0; i < sampleCount; i++) + { + uint size; + if (fieldSize == 4) + { + byte packed = reader.ReadByte(); + size = (uint)(packed >> 4); + if (i < retainedCount) + { + track.Samples[(int)i].Length = ValidateSampleSize(size); + } + + i++; + if (i >= sampleCount) + { + if ((packed & 15) != 0) + { + throw new InvalidImageContentException("The compact sample-size table has nonzero padding bits."); + } + + break; + } + + size = (uint)(packed & 15); + } + else + { + size = fieldSize == 8 ? reader.ReadByte() : reader.ReadUInt16(); + } + + int validatedSize = ValidateSampleSize(size); + if (i < retainedCount) + { + track.Samples[(int)i].Length = validatedSize; + } + } + } + + /// + /// Expands retained sample durations and validates the complete timing run table. + /// + /// The stream positioned at the time-to-sample payload. + /// The validated time-to-sample payload length. + /// The selected track receiving retained durations. + /// The parser-owned reusable scratch span. + /// The total decoded duration in media-time-scale units. + private static ulong ParseSampleTiming(Stream stream, long boxLength, HeifSequenceTrack track, Span scratch) + { + ReadOnlySpan prefix = ReadPrefix(stream, boxLength, scratch, 8, "sample timing"); + EnsureVersionAndFlags(prefix, 0, 0, "sample timing"); + uint entryCount = BinaryPrimitives.ReadUInt32BigEndian(prefix[4..]); + long entryBytes = checked((long)entryCount * 8); + if (entryCount == 0 || boxLength != 8 + entryBytes) + { + throw new InvalidImageContentException("The image-sequence timing table is empty or has an invalid length."); + } + + HeifBoxPayloadReader reader = new(stream, entryBytes, scratch, "sample timing"); + ulong describedSamples = 0; + ulong decodedDuration = 0; + int retainedOffset = 0; + for (uint entry = 0; entry < entryCount; entry++) + { + uint sampleCount = reader.ReadUInt32(); + uint sampleDelta = reader.ReadUInt32(); + if (sampleCount == 0 || sampleDelta == 0) + { + throw new InvalidImageContentException("The image-sequence timing table contains a zero run or duration."); + } + + describedSamples = checked(describedSamples + sampleCount); + decodedDuration = checked(decodedDuration + ((ulong)sampleCount * sampleDelta)); + int retainedRun = Math.Min((int)Math.Min(sampleCount, int.MaxValue), track.Samples.Length - retainedOffset); + Span samples = track.Samples; + for (int i = 0; i < retainedRun; i++) + { + samples[retainedOffset + i].Duration = sampleDelta; + } + + retainedOffset += retainedRun; + } + + if (describedSamples != track.TotalSampleCount) + { + throw new InvalidImageContentException("The image-sequence timing table does not describe every sample."); + } + + return decodedDuration; + } + + /// + /// Reads and validates the declared chunk count without retaining chunk offsets. + /// + /// The stream positioned at a chunk-offset payload. + /// The validated chunk-offset payload length. + /// The 32-bit or 64-bit chunk-offset box type. + /// The parser-owned reusable scratch span. + /// The nonzero number of chunks. + private static uint ReadChunkCount(Stream stream, long boxLength, Heif4CharCode boxType, Span scratch) + { + ReadOnlySpan prefix = ReadPrefix(stream, boxLength, scratch, 8, "chunk offsets"); + EnsureVersionAndFlags(prefix, 0, 0, "chunk offsets"); + uint chunkCount = BinaryPrimitives.ReadUInt32BigEndian(prefix[4..]); + int entrySize = boxType == Heif4CharCode.Co64 ? 8 : 4; + if (chunkCount == 0 || boxLength != 8 + checked((long)chunkCount * entrySize)) + { + throw new InvalidImageContentException("The image-sequence chunk-offset table is empty or has an invalid length."); + } + + return chunkCount; + } + + /// + /// Parses sample-to-chunk runs into allocator-owned state bounded by the retained frame count. + /// + /// The stream positioned at the sample-to-chunk payload. + /// The validated sample-to-chunk payload length. + /// The validated number of chunks. + /// The selected track whose complete sample count is validated. + /// The parser-owned reusable scratch span. + /// Receives the number of retained mapping entries. + /// Allocator-owned sample-to-chunk runs that cover all retained samples. + private IMemoryOwner ParseSampleToChunk( + Stream stream, + long boxLength, + uint chunkCount, + HeifSequenceTrack track, + Span scratch, + out int retainedEntryCount) + { + ReadOnlySpan prefix = ReadPrefix(stream, boxLength, scratch, 8, "sample-to-chunk"); + EnsureVersionAndFlags(prefix, 0, 0, "sample-to-chunk"); + uint entryCount = BinaryPrimitives.ReadUInt32BigEndian(prefix[4..]); + long entryBytes = checked((long)entryCount * 12); + if (entryCount == 0 || entryCount > chunkCount || boxLength != 8 + entryBytes) + { + throw new InvalidImageContentException("The sample-to-chunk table is empty or has an invalid length or entry count."); + } + + int retainedCapacity = (int)Math.Min(entryCount, (uint)track.Samples.Length); + IMemoryOwner owner = this.allocator.Allocate(retainedCapacity); + Span retainedEntries = owner.GetSpan(); + HeifBoxPayloadReader reader = new(stream, entryBytes, scratch, "sample-to-chunk"); + uint previousFirstChunk = 0; + uint previousSamplesPerChunk = 0; + ulong describedSamples = 0; + retainedEntryCount = 0; + + try + { + for (uint i = 0; i < entryCount; i++) + { + uint firstChunk = reader.ReadUInt32(); + uint samplesPerChunk = reader.ReadUInt32(); + uint sampleDescriptionIndex = reader.ReadUInt32(); + if ((i == 0 && firstChunk != 1) || firstChunk <= previousFirstChunk || firstChunk > chunkCount + || samplesPerChunk == 0 || sampleDescriptionIndex != 1) + { + throw new InvalidImageContentException("The sample-to-chunk table contains an invalid run."); + } + + if (i != 0) + { + describedSamples = checked(describedSamples + ((ulong)(firstChunk - previousFirstChunk) * previousSamplesPerChunk)); + } + + if (retainedEntryCount < retainedEntries.Length) + { + retainedEntries[retainedEntryCount++] = new SampleToChunkEntry(firstChunk, samplesPerChunk); + } + + previousFirstChunk = firstChunk; + previousSamplesPerChunk = samplesPerChunk; + } + + describedSamples = checked(describedSamples + ((ulong)(chunkCount + 1U - previousFirstChunk) * previousSamplesPerChunk)); + if (describedSamples != track.TotalSampleCount) + { + throw new InvalidImageContentException("The sample-to-chunk table does not map every declared sample."); + } + + return owner; + } + catch + { + owner.Dispose(); + throw; + } + } + + /// + /// Resolves retained samples directly from sequential chunk offsets and compact mapping runs. + /// + /// The stream positioned at the chunk-offset payload. + /// The validated chunk-offset payload length. + /// The 32-bit or 64-bit chunk-offset box type. + /// The validated number of chunks. + /// The retained sample-to-chunk runs. + /// The selected track receiving absolute sample locations. + /// The parser-owned reusable scratch span. + private void ResolveSampleLocations( + Stream stream, + long boxLength, + Heif4CharCode boxType, + uint chunkCount, + ReadOnlySpan entries, + HeifSequenceTrack track, + Span scratch) + { + _ = ReadChunkCount(stream, boxLength, boxType, scratch); + int entrySize = boxType == Heif4CharCode.Co64 ? 8 : 4; + HeifBoxPayloadReader reader = new(stream, checked((long)chunkCount * entrySize), scratch, "chunk offsets"); + int retainedSample = 0; + int runIndex = 0; + long fileLength = stream.Length - this.fileStartOffset; + for (uint chunkIndex = 0; chunkIndex < chunkCount; chunkIndex++) + { + ulong chunkOffset = entrySize == 8 ? reader.ReadUInt64() : reader.ReadUInt32(); + if (chunkOffset > (ulong)fileLength) + { + throw new InvalidImageContentException("An image-sequence chunk offset extends beyond the file."); + } + + uint chunkNumber = chunkIndex + 1; + if (runIndex + 1 < entries.Length && entries[runIndex + 1].FirstChunk <= chunkNumber) + { + runIndex++; + } + + uint samplesPerChunk = entries[runIndex].SamplesPerChunk; + ulong sampleOffset = chunkOffset; + for (uint sampleInChunk = 0; sampleInChunk < samplesPerChunk && retainedSample < track.Samples.Length; sampleInChunk++) + { + ref HeifSequenceSample sample = ref track.Samples[retainedSample++]; + ulong sampleEnd = checked(sampleOffset + (uint)sample.Length); + if (sampleEnd > (ulong)fileLength || sampleOffset > long.MaxValue) + { + throw new InvalidImageContentException("An image-sequence sample extends beyond the file."); + } + + sample.Offset = (long)sampleOffset; + sampleOffset = sampleEnd; + } + } + + if (retainedSample != track.Samples.Length) + { + throw new InvalidImageContentException("The image-sequence chunk table does not locate every retained sample."); + } + } + + /// + /// Applies explicit one-based sync-sample declarations to retained sample descriptors. + /// + /// The stream positioned at the sync-sample payload. + /// The validated sync-sample payload length. + /// The selected track receiving random-access markers. + /// The parser-owned reusable scratch span. + private static void ParseSyncSamples(Stream stream, long boxLength, HeifSequenceTrack track, Span scratch) + { + ReadOnlySpan prefix = ReadPrefix(stream, boxLength, scratch, 8, "sync samples"); + EnsureVersionAndFlags(prefix, 0, 0, "sync samples"); + uint entryCount = BinaryPrimitives.ReadUInt32BigEndian(prefix[4..]); + long entryBytes = checked((long)entryCount * 4); + if (entryCount == 0 || boxLength != 8 + entryBytes) + { + throw new InvalidImageContentException("The sync-sample table is empty or has an invalid length."); + } + + HeifBoxPayloadReader reader = new(stream, entryBytes, scratch, "sync samples"); + uint previousSample = 0; + for (uint i = 0; i < entryCount; i++) + { + uint sampleNumber = reader.ReadUInt32(); + if (sampleNumber <= previousSample || sampleNumber > track.TotalSampleCount) + { + throw new InvalidImageContentException("The sync-sample table contains an invalid sample number."); + } + + if (sampleNumber <= track.Samples.Length) + { + track.Samples[(int)sampleNumber - 1].IsSync = true; + } + + previousSample = sampleNumber; + } + + if (!track.Samples[0].IsSync) + { + throw new InvalidImageContentException("The first retained image-sequence sample is not a random-access sample."); + } + } + + /// + /// Resolves the image-specific direct-reference sample group into compact zero-based sample indices. + /// + /// The seekable source stream. + /// The validated direct-reference group-description payload. + /// The validated direct-reference sample-map payload. + /// The selected image track receiving its dependency graph. + /// The parser-owned reusable scratch span. + private void ParseDirectReferences( + Stream stream, + BoxReference descriptions, + BoxReference sampleMap, + HeifSequenceTrack track, + Span scratch) + { + using IMemoryOwner assignmentOwner = this.allocator.Allocate(track.Samples.Length); + Span assignments = assignmentOwner.GetSpan()[..track.Samples.Length]; + stream.Position = sampleMap.Offset; + uint greatestGroupIndex = ParseSampleToGroup(stream, sampleMap.Length, track, assignments, scratch); + + // Sorting the retained value-type assignments lets each group description be applied in one sequential pass. + // This avoids a dictionary and prevents attacker-controlled group counts from causing quadratic lookup work. + assignments.Sort(); + stream.Position = descriptions.Offset; + int directReferenceCount = ParseDirectReferenceDescriptions( + stream, + descriptions.Length, + greatestGroupIndex, + track, + assignments, + [], + false, + scratch); + + using IMemoryOwner sampleIdOwner = this.allocator.Allocate(track.Samples.Length); + Span sampleIds = sampleIdOwner.GetSpan()[..track.Samples.Length]; + int sampleIdCount = 0; + Span samples = track.Samples; + for (int i = 0; i < samples.Length; i++) + { + uint sampleId = samples[i].SampleId; + if (sampleId != 0) + { + sampleIds[sampleIdCount++] = new SampleIdIndexEntry(sampleId, i); + } + } + + sampleIds = sampleIds[..sampleIdCount]; + sampleIds.Sort(); + for (int i = 1; i < sampleIds.Length; i++) + { + if (sampleIds[i - 1].Id == sampleIds[i].Id) + { + throw new InvalidImageContentException("The direct-reference sample group contains a duplicate positive sample identifier."); + } + } + + if (directReferenceCount == 0) + { + return; + } + + track.DirectReferenceSampleIndices = new int[directReferenceCount]; + stream.Position = descriptions.Offset; + _ = ParseDirectReferenceDescriptions( + stream, + descriptions.Length, + greatestGroupIndex, + track, + assignments, + sampleIds, + true, + scratch); + + if (track.AllReferencePicturesIntra) + { + ReadOnlySpan referenceIndices = track.DirectReferenceSampleIndices; + for (int i = 0; i < referenceIndices.Length; i++) + { + if (samples[referenceIndices[i]].DirectReferenceCount != 0) + { + throw new InvalidImageContentException("The direct-reference sample group contradicts its all-reference-pictures-intra constraint."); + } + } + } + } + + /// + /// Expands the run-length sample-to-group map only for samples retained by the decoder. + /// + /// The stream positioned at the sample-to-group payload. + /// The validated sample-to-group payload length. + /// The selected track whose complete sample count is validated. + /// The exact retained assignment span. + /// The parser-owned reusable scratch span. + /// The greatest group-description index used by any declared sample. + private static uint ParseSampleToGroup( + Stream stream, + long boxLength, + HeifSequenceTrack track, + Span assignments, + Span scratch) + { + long payloadStart = stream.Position; + ReadOnlySpan prefix = ReadPrefix(stream, boxLength, scratch, 8, "sample-to-group"); + byte version = prefix[0]; + if (version is not 0 and not 1 || ReadFlags(prefix) != 0 + || (Heif4CharCode)BinaryPrimitives.ReadUInt32BigEndian(prefix[4..]) != Heif4CharCode.Refs) + { + throw new InvalidImageContentException("The direct-reference sample-to-group box has an unsupported version, flags, or grouping type."); + } + + int headerLength = version == 0 ? 12 : 16; + stream.Position = payloadStart; + prefix = ReadPrefix(stream, boxLength, scratch, headerLength, "sample-to-group"); + if (version == 1 && BinaryPrimitives.ReadUInt32BigEndian(prefix[8..]) != 0) + { + throw new InvalidImageContentException("The direct-reference sample group has a nonzero grouping-type parameter."); + } + + int entryCountOffset = version == 0 ? 8 : 12; + uint entryCount = BinaryPrimitives.ReadUInt32BigEndian(prefix[entryCountOffset..]); + long entryBytes = checked((long)entryCount * 8); + if (entryCount == 0 || boxLength != headerLength + entryBytes) + { + throw new InvalidImageContentException("The direct-reference sample map is empty or has an invalid length."); + } + + HeifBoxPayloadReader reader = new(stream, entryBytes, scratch, "direct-reference sample map"); + ulong describedSamples = 0; + int retainedOffset = 0; + uint greatestGroupIndex = 0; + for (uint i = 0; i < entryCount; i++) + { + uint sampleCount = reader.ReadUInt32(); + uint groupDescriptionIndex = reader.ReadUInt32(); + if (sampleCount == 0) + { + throw new InvalidImageContentException("The direct-reference sample map contains a zero-length run."); + } + + describedSamples = checked(describedSamples + sampleCount); + greatestGroupIndex = Math.Max(greatestGroupIndex, groupDescriptionIndex); + int retainedRun = Math.Min((int)Math.Min(sampleCount, int.MaxValue), assignments.Length - retainedOffset); + for (int j = 0; j < retainedRun; j++) + { + assignments[retainedOffset + j] = new SampleGroupAssignment(groupDescriptionIndex, retainedOffset + j); + } + + retainedOffset += retainedRun; + } + + if (describedSamples != track.TotalSampleCount || retainedOffset != assignments.Length) + { + throw new InvalidImageContentException("The direct-reference sample map does not describe every sample."); + } + + return greatestGroupIndex; + } + + /// + /// Parses direct-reference descriptions, first sizing and then resolving the retained dependency graph. + /// + /// The stream positioned at the sample-group-description payload. + /// The validated sample-group-description payload length. + /// The greatest description index used by the complete sample map. + /// The selected track receiving sample identifiers and dependency indices. + /// The retained sample assignments sorted by group-description index. + /// The sorted positive sample identifiers, or an empty span during the sizing pass. + /// Whether this pass resolves reference identifiers into compact sample indices. + /// The parser-owned reusable scratch span. + /// The exact number of retained direct-reference indices. + private static int ParseDirectReferenceDescriptions( + Stream stream, + long boxLength, + uint greatestGroupIndex, + HeifSequenceTrack track, + ReadOnlySpan assignments, + ReadOnlySpan sampleIds, + bool resolveReferences, + Span scratch) + { + long payloadStart = stream.Position; + ReadOnlySpan prefix = ReadPrefix(stream, boxLength, scratch, 8, "sample-group descriptions"); + byte version = prefix[0]; + if (version is not 1 and not 2 || ReadFlags(prefix) != 0 + || (Heif4CharCode)BinaryPrimitives.ReadUInt32BigEndian(prefix[4..]) != Heif4CharCode.Refs) + { + throw new InvalidImageContentException("The direct-reference sample-group-description box has an unsupported version, flags, or grouping type."); + } + + int headerLength = version == 1 ? 16 : 20; + stream.Position = payloadStart; + prefix = ReadPrefix(stream, boxLength, scratch, headerLength, "sample-group descriptions"); + uint defaultLength = BinaryPrimitives.ReadUInt32BigEndian(prefix[8..]); + uint defaultGroupIndex = version == 2 ? BinaryPrimitives.ReadUInt32BigEndian(prefix[12..]) : 0; + int entryCountOffset = version == 1 ? 12 : 16; + uint entryCount = BinaryPrimitives.ReadUInt32BigEndian(prefix[entryCountOffset..]); + if (greatestGroupIndex > entryCount || defaultGroupIndex > entryCount) + { + throw new InvalidImageContentException("The direct-reference sample map uses an undefined group-description index."); + } + + long entryBytes = boxLength - headerLength; + HeifBoxPayloadReader reader = new(stream, entryBytes, scratch, "direct-reference descriptions"); + long consumedBytes = 0; + int assignmentOffset = 0; + int directReferenceCount = 0; + while (assignmentOffset < assignments.Length && assignments[assignmentOffset].GroupDescriptionIndex == 0) + { + assignmentOffset++; + } + + for (uint entry = 0; entry < entryCount; entry++) + { + uint descriptionIndex = entry + 1; + uint descriptionLength = defaultLength; + if (descriptionLength == 0) + { + if (entryBytes - consumedBytes < 4) + { + throw new InvalidImageContentException("The direct-reference sample-group description is truncated."); + } + + descriptionLength = reader.ReadUInt32(); + consumedBytes += 4; + } + + if (descriptionLength < 5 || descriptionLength > entryBytes - consumedBytes) + { + throw new InvalidImageContentException("The direct-reference sample-group description has an invalid length."); + } + + uint sampleId = reader.ReadUInt32(); + byte referenceCount = reader.ReadByte(); + uint requiredLength = 5U + ((uint)referenceCount * 4U); + if (descriptionLength != requiredLength) + { + throw new InvalidImageContentException("The direct-reference sample-group entry has an invalid length."); + } + + int firstAssignment = assignmentOffset; + while (assignmentOffset < assignments.Length && assignments[assignmentOffset].GroupDescriptionIndex == descriptionIndex) + { + int sampleIndex = assignments[assignmentOffset].SampleIndex; + ref HeifSequenceSample sample = ref track.Samples[sampleIndex]; + if (!resolveReferences) + { + if (referenceCount > track.MaximumReferencesPerPicture || directReferenceCount > int.MaxValue - referenceCount) + { + throw new InvalidImageContentException("The direct-reference sample group exceeds its coding constraints or supported size."); + } + + sample.SampleId = sampleId; + sample.DirectReferenceOffset = directReferenceCount; + sample.DirectReferenceCount = referenceCount; + directReferenceCount += referenceCount; + } + else if (sample.SampleId != sampleId || sample.DirectReferenceCount != referenceCount) + { + throw new InvalidImageContentException("The direct-reference sample group changed between parser passes."); + } + + assignmentOffset++; + } + + for (int reference = 0; reference < referenceCount; reference++) + { + uint referenceSampleId = reader.ReadUInt32(); + if (referenceSampleId == 0) + { + throw new InvalidImageContentException("The direct-reference sample group contains identifier zero in a reference list."); + } + + if (resolveReferences) + { + int low = 0; + int high = sampleIds.Length - 1; + while (low <= high) + { + int middle = low + ((high - low) >> 1); + uint candidate = sampleIds[middle].Id; + if (candidate < referenceSampleId) + { + low = middle + 1; + } + else if (candidate > referenceSampleId) + { + high = middle - 1; + } + else + { + low = middle; + break; + } + } + + if (low >= sampleIds.Length || sampleIds[low].Id != referenceSampleId) + { + throw new InvalidImageContentException("A direct-reference sample identifier does not name a retained sample."); + } + + int referencedSampleIndex = sampleIds[low].SampleIndex; + for (int assignment = firstAssignment; assignment < assignmentOffset; assignment++) + { + int sampleIndex = assignments[assignment].SampleIndex; + HeifSequenceSample sample = track.Samples[sampleIndex]; + if (sample.IsSync || referencedSampleIndex >= sampleIndex) + { + throw new InvalidImageContentException("A direct-reference sample is not earlier in decode order or is attached to a sync sample."); + } + + track.DirectReferenceSampleIndices[sample.DirectReferenceOffset + reference] = referencedSampleIndex; + } + } + } + + consumedBytes += descriptionLength; + } + + if (consumedBytes != entryBytes || assignmentOffset != assignments.Length) + { + throw new InvalidImageContentException("The direct-reference sample-group descriptions do not cover the retained sample map."); + } + + return directReferenceCount; + } + + /// + /// Reads the grouping type shared by a sample map or sample-group-description box. + /// + /// The stream positioned at the full-box payload. + /// The validated payload length. + /// The parser-owned reusable scratch span. + /// The declared grouping type. + private static Heif4CharCode ReadSampleGroupType(Stream stream, long boxLength, Span scratch) + { + ReadOnlySpan prefix = ReadPrefix(stream, boxLength, scratch, 8, "sample group"); + return (Heif4CharCode)BinaryPrimitives.ReadUInt32BigEndian(prefix[4..]); + } + + /// + /// Computes retained sample composition times while preserving decode-order storage. + /// + /// The selected track whose durations and offsets have been validated. + private static void SetCompositionTimes(HeifSequenceTrack track) + { + long decodeTime = 0; + Span samples = track.Samples; + for (int i = 0; i < samples.Length; i++) + { + ref HeifSequenceSample sample = ref samples[i]; + sample.CompositionTime = sample.IsHidden ? long.MinValue : checked(decodeTime + sample.CompositionOffset); + decodeTime = checked(decodeTime + sample.Duration); + } + } + + /// + /// Parses the single normal-rate edit list used to signal image-sequence repetition. + /// + /// The stream positioned at the edit-container payload. + /// The validated edit-container payload length. + /// The selected track receiving repetition behavior. + /// The parser-owned reusable scratch span. + private static void ParseEdit(Stream stream, long boxLength, HeifSequenceTrack track, Span scratch) + { + long editEnd = checked(stream.Position + boxLength); + BoxReference editList = default; + while (stream.Position < editEnd) + { + long childLength = HeifBoxReader.ReadHeader(stream, editEnd, scratch, out Heif4CharCode childType); + long childStart = stream.Position; + if (childType == Heif4CharCode.Elst) + { + SetUnique(ref editList, childStart, childLength, "edit", childType); + } + + stream.Position = checked(childStart + childLength); + } + + if (!editList.IsPresent) + { + throw new InvalidImageContentException("The image-sequence edit container has no edit list."); + } + + stream.Position = editList.Offset; + ReadOnlySpan prefix = ReadPrefix(stream, editList.Length, scratch, 8, "edit list"); + byte version = prefix[0]; + uint flags = ReadFlags(prefix); + int entryLength = version switch + { + 0 => 12, + 1 => 20, + _ => throw new InvalidImageContentException($"The edit list has unsupported version {version}.") + }; + + if ((flags & ~1U) != 0 || BinaryPrimitives.ReadUInt32BigEndian(prefix[4..]) != 1 || editList.Length != 8 + entryLength) + { + throw new InvalidImageContentException("The image-sequence edit list has unsupported flags, entries, or length."); + } + + prefix = ReadPrefix(stream, entryLength, scratch, entryLength, "edit-list entry"); + ulong segmentDuration; + long mediaTime; + int rateOffset; + if (version == 0) + { + segmentDuration = BinaryPrimitives.ReadUInt32BigEndian(prefix); + mediaTime = BinaryPrimitives.ReadInt32BigEndian(prefix[4..]); + rateOffset = 8; + } + else + { + segmentDuration = BinaryPrimitives.ReadUInt64BigEndian(prefix); + mediaTime = BinaryPrimitives.ReadInt64BigEndian(prefix[8..]); + rateOffset = 16; + } + + if (segmentDuration == 0 || mediaTime != 0 || BinaryPrimitives.ReadInt16BigEndian(prefix[rateOffset..]) != 1 + || BinaryPrimitives.ReadInt16BigEndian(prefix[(rateOffset + 2)..]) != 0) + { + throw new InvalidImageContentException("The image-sequence edit list requires unsupported splicing or playback rate behavior."); + } + + if ((flags & 1) == 0) + { + track.RepeatCount = 1; + } + else if (track.TrackDuration == ulong.MaxValue) + { + track.RepeatCount = 0; + } + else + { + if (track.TrackDuration == 0) + { + throw new InvalidImageContentException("A repeating image-sequence track has zero duration."); + } + + ulong plays = (track.TrackDuration / segmentDuration) + (track.TrackDuration % segmentDuration == 0 ? 0UL : 1UL); + + // The public metadata uses zero for indefinite repetition. Preserve extremely large finite edit counts as + // indefinite rather than wrapping the observable ushort play count. + track.RepeatCount = plays is 0 or > ushort.MaxValue ? (ushort)0 : (ushort)plays; + } + + track.HasEditList = true; + } + + /// + /// Validates that a linked alpha track can be matched frame-for-frame with its master color track. + /// + /// The selected master color track. + /// The optional linked alpha track. + private static void ValidateAlphaTrack(HeifSequenceTrack colorTrack, HeifSequenceTrack? alphaTrack) + { + if (colorTrack.PremultipliedByTrackId != 0) + { + if (alphaTrack is null || colorTrack.PremultipliedByTrackId != alphaTrack.Id) + { + throw new InvalidImageContentException("The color image-sequence track references an unrelated premultiplication track."); + } + + colorTrack.IsPremultiplied = true; + } + + if (alphaTrack is null) + { + return; + } + + if (alphaTrack.TotalSampleCount != colorTrack.TotalSampleCount || alphaTrack.Samples.Length != colorTrack.Samples.Length) + { + throw new InvalidImageContentException("The alpha and color image-sequence tracks contain different sample counts."); + } + + for (int i = 0; i < colorTrack.Samples.Length; i++) + { + HeifSequenceSample colorSample = colorTrack.Samples[i]; + HeifSequenceSample alphaSample = alphaTrack.Samples[i]; + ulong colorDuration = (ulong)colorSample.Duration * alphaTrack.MediaTimescale; + ulong alphaDuration = (ulong)alphaSample.Duration * colorTrack.MediaTimescale; + if (colorDuration != alphaDuration) + { + throw new InvalidImageContentException("The alpha and color image-sequence samples have different presentation durations."); + } + + if (colorSample.IsHidden != alphaSample.IsHidden) + { + throw new InvalidImageContentException("The alpha and color image-sequence samples have different presentation visibility."); + } + + if (!colorSample.IsHidden) + { + // Cross-multiplication retains exact signed presentation times without floating-point rounding or overflow. + Int128 colorCompositionTime = (Int128)colorSample.CompositionTime * alphaTrack.MediaTimescale; + Int128 alphaCompositionTime = (Int128)alphaSample.CompositionTime * colorTrack.MediaTimescale; + if (colorCompositionTime != alphaCompositionTime) + { + throw new InvalidImageContentException("The alpha and color image-sequence samples have different presentation times."); + } + } + } + + bool alphaHasPresentationProperties = alphaTrack.CleanAperture is not null || alphaTrack.RotationAngle is not null || alphaTrack.MirrorAxis is not null; + if (alphaHasPresentationProperties && + (!Nullable.Equals(colorTrack.CleanAperture, alphaTrack.CleanAperture) || + colorTrack.RotationAngle != alphaTrack.RotationAngle || + colorTrack.MirrorAxis != alphaTrack.MirrorAxis)) + { + // libavif accepts legacy alpha tracks with no transform properties, but requires exact equality when any + // alpha transform is declared because composition occurs before the shared color-track presentation step. + throw new NotSupportedException("The alpha and color image-sequence tracks use different presentation transforms."); + } + } + + /// + /// Validates and narrows a sample size to the decoder's contiguous-buffer length type. + /// + /// The file-defined unsigned sample size. + /// The positive sample size as an . + private static int ValidateSampleSize(uint size) + { + if (size == 0 || size > int.MaxValue) + { + throw new InvalidImageContentException("An image-sequence sample has an unsupported size."); + } + + return (int)size; + } + + /// + /// Records one unique child box while retaining only its stream range. + /// + /// The child reference owned by the bounded parent parser. + /// The absolute payload offset. + /// The validated payload length. + /// The parent name used in malformed-image diagnostics. + /// The unique child box type. + private static void SetUnique(ref BoxReference reference, long offset, long length, string parentName, Heif4CharCode boxType) + { + if (reference.IsPresent) + { + throw new InvalidImageContentException($"The {parentName} box contains duplicate '{boxType}' boxes."); + } + + reference = new BoxReference(offset, length, boxType); + } + + /// + /// Reads a fixed prefix from the stream's current position. + /// + /// The source stream. + /// The validated enclosing payload length. + /// The reusable destination scratch span. + /// The required prefix length. + /// The payload name used in malformed-image diagnostics. + /// The requested prefix within . + private static ReadOnlySpan ReadPrefix(Stream stream, long boxLength, Span scratch, int length, string name) + { + if (boxLength < length) + { + throw new InvalidImageContentException($"The {name} payload is truncated."); + } + + Span destination = scratch[..length]; + HeifBoxReader.ReadExactly(stream, destination, $"The {name} payload is truncated."); + return destination; + } + + /// + /// Rewinds to the start of a partially read payload and reads a larger fixed prefix. + /// + /// The source stream positioned after a four-byte prefix. + /// The validated enclosing payload length. + /// The reusable destination scratch span. + /// The required prefix length. + /// The payload name used in malformed-image diagnostics. + /// The requested prefix within . + private static ReadOnlySpan ReadPrefixFromStart(Stream stream, long boxLength, Span scratch, int length, string name) + { + stream.Position -= 4; + return ReadPrefix(stream, boxLength, scratch, length, name); + } + + /// + /// Reads the lower 24-bit flags field from a full-box prefix. + /// + /// The prefix beginning with version and flags. + /// The unsigned flags value. + private static uint ReadFlags(ReadOnlySpan data) => BinaryPrimitives.ReadUInt32BigEndian(data) & 0x00FFFFFF; + + /// + /// Requires a full-box prefix to use one supported version and flags value. + /// + /// The prefix beginning with version and flags. + /// The required version. + /// The required flags. + /// The box name used in malformed-image diagnostics. + private static void EnsureVersionAndFlags(ReadOnlySpan data, byte version, uint flags, string name) + { + if (data[0] != version || ReadFlags(data) != flags) + { + throw new InvalidImageContentException($"The {name} box has unsupported version or flags."); + } + } + + /// + /// Requires a full-box prefix to contain zero flags. + /// + /// The prefix beginning with version and flags. + /// The box name used in malformed-image diagnostics. + private static void EnsureZeroFlags(ReadOnlySpan data, string name) + { + if (ReadFlags(data) != 0) + { + throw new InvalidImageContentException($"The {name} box has unsupported flags."); + } + } + + /// + /// Identifies a selected track without retaining its codec or sample-table payloads. + /// + private struct TrackIdentity + { + /// + /// Initializes a new instance of the struct. + /// + /// The file-defined track identifier. + /// Whether the track is enabled. + /// The displayed track width. + /// The displayed track height. + /// The track duration in movie-time-scale units. + public TrackIdentity(uint id, bool isEnabled, int width, int height, ulong trackDuration) + { + this.Id = id; + this.IsEnabled = isEnabled; + this.Width = width; + this.Height = height; + this.TrackDuration = trackDuration; + this.HandlerType = default; + this.AuxiliaryForTrackId = 0; + this.PremultipliedByTrackId = 0; + } + + /// + /// Gets the file-defined track identifier. + /// + public uint Id { get; } + + /// + /// Gets a value indicating whether the track is enabled. + /// + public bool IsEnabled { get; } + + /// + /// Gets the displayed track width. + /// + public int Width { get; } + + /// + /// Gets the displayed track height. + /// + public int Height { get; } + + /// + /// Gets the track duration in movie-time-scale units. + /// + public ulong TrackDuration { get; } + + /// + /// Gets or sets the media handler type. + /// + public Heif4CharCode HandlerType { get; set; } + + /// + /// Gets or sets the master track served by this auxiliary track. + /// + public uint AuxiliaryForTrackId { get; set; } + + /// + /// Gets or sets the track identifier used by premultiplication signaling. + /// + public uint PremultipliedByTrackId { get; set; } + } + + /// + /// Retains one unique child payload range without creating a general box object model. + /// + private struct BoxReference + { + /// + /// Initializes a new instance of the struct. + /// + /// The absolute payload offset. + /// The validated payload length. + /// The child box type. + public BoxReference(long offset, long length, Heif4CharCode type) + { + this.Offset = offset; + this.Length = length; + this.Type = type; + this.IsPresent = true; + } + + /// + /// Gets the absolute payload offset. + /// + public long Offset { get; } + + /// + /// Gets the validated payload length. + /// + public long Length { get; } + + /// + /// Gets or sets the child box type when one logical slot accepts multiple concrete box types. + /// + public Heif4CharCode Type { get; set; } + + /// + /// Gets a value indicating whether the child was present. + /// + public bool IsPresent { get; } + } + + /// + /// Maps a one-based chunk run to its number of samples per chunk. + /// + private readonly struct SampleToChunkEntry + { + /// + /// Initializes a new instance of the struct. + /// + /// The one-based first chunk in the run. + /// The number of samples stored in each run chunk. + public SampleToChunkEntry(uint firstChunk, uint samplesPerChunk) + { + this.FirstChunk = firstChunk; + this.SamplesPerChunk = samplesPerChunk; + } + + /// + /// Gets the one-based first chunk in the run. + /// + public uint FirstChunk { get; } + + /// + /// Gets the number of samples stored in each run chunk. + /// + public uint SamplesPerChunk { get; } + } + + /// + /// Associates one retained sample with its one-based direct-reference group description. + /// + private readonly struct SampleGroupAssignment : IComparable + { + /// + /// Initializes a new instance of the struct. + /// + /// The one-based group-description index, or zero for no group. + /// The zero-based retained sample index. + public SampleGroupAssignment(uint groupDescriptionIndex, int sampleIndex) + { + this.GroupDescriptionIndex = groupDescriptionIndex; + this.SampleIndex = sampleIndex; + } + + /// + /// Gets the one-based group-description index, or zero for no group. + /// + public uint GroupDescriptionIndex { get; } + + /// + /// Gets the zero-based retained sample index. + /// + public int SampleIndex { get; } + + /// + /// Compares this assignment with another assignment in group-description and sample order. + /// + /// The other assignment. + /// A value indicating the relative sort order. + public int CompareTo(SampleGroupAssignment other) + { + int result = this.GroupDescriptionIndex.CompareTo(other.GroupDescriptionIndex); + return result != 0 ? result : this.SampleIndex.CompareTo(other.SampleIndex); + } + } + + /// + /// Maps one positive direct-reference identifier to its retained decode-order sample index. + /// + private readonly struct SampleIdIndexEntry : IComparable + { + /// + /// Initializes a new instance of the struct. + /// + /// The positive file-defined sample identifier. + /// The zero-based retained sample index. + public SampleIdIndexEntry(uint id, int sampleIndex) + { + this.Id = id; + this.SampleIndex = sampleIndex; + } + + /// + /// Gets the positive file-defined sample identifier. + /// + public uint Id { get; } + + /// + /// Gets the zero-based retained sample index. + /// + public int SampleIndex { get; } + + /// + /// Compares this entry with another entry by sample identifier. + /// + /// The other identifier entry. + /// A value indicating the relative sort order. + public int CompareTo(SampleIdIndexEntry other) => this.Id.CompareTo(other.Id); + } +} diff --git a/src/ImageSharp/Formats/Heif/HeifSequenceSample.cs b/src/ImageSharp/Formats/Heif/HeifSequenceSample.cs new file mode 100644 index 0000000000..3353ba8ebd --- /dev/null +++ b/src/ImageSharp/Formats/Heif/HeifSequenceSample.cs @@ -0,0 +1,60 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif; + +/// +/// Describes one retained coded sample in a HEIF image sequence. +/// +internal struct HeifSequenceSample +{ + /// + /// Gets or sets the absolute file offset of the coded sample. + /// + public long Offset { get; set; } + + /// + /// Gets or sets the coded sample length in bytes. + /// + public int Length { get; set; } + + /// + /// Gets or sets the sample duration in media-time-scale units. + /// + public uint Duration { get; set; } + + /// + /// Gets or sets the signed offset from decode time to composition time in media-time-scale units. + /// + public long CompositionOffset { get; set; } + + /// + /// Gets or sets the computed composition time in media-time-scale units. + /// + public long CompositionTime { get; set; } + + /// + /// Gets or sets a value indicating whether the sample is decoded only as a reference and is not presented. + /// + public bool IsHidden { get; set; } + + /// + /// Gets or sets a value indicating whether decoding can begin at this sample. + /// + public bool IsSync { get; set; } + + /// + /// Gets or sets the positive identifier used when another retained sample directly references this sample. + /// + public uint SampleId { get; set; } + + /// + /// Gets or sets the first direct reference in the owning track's compact reference-index array. + /// + public int DirectReferenceOffset { get; set; } + + /// + /// Gets or sets the number of direct reference indices belonging to this sample. + /// + public byte DirectReferenceCount { get; set; } +} diff --git a/src/ImageSharp/Formats/Heif/HeifSequenceTrack.cs b/src/ImageSharp/Formats/Heif/HeifSequenceTrack.cs new file mode 100644 index 0000000000..63f2ee7dd8 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/HeifSequenceTrack.cs @@ -0,0 +1,201 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1; +using SixLabors.ImageSharp.Metadata.Profiles.Cicp; +using SixLabors.ImageSharp.Metadata.Profiles.Icc; + +namespace SixLabors.ImageSharp.Formats.Heif; + +/// +/// Owns the bounded image behavior retained from one HEIF image-sequence track. +/// +internal sealed class HeifSequenceTrack +{ + /// + /// Initializes a new instance of the class. + /// + public HeifSequenceTrack() + { + } + + /// + /// Gets or sets the file-defined track identifier. + /// + public uint Id { get; set; } + + /// + /// Gets or sets the displayed track width in pixels. + /// + public int Width { get; set; } + + /// + /// Gets or sets the displayed track height in pixels. + /// + public int Height { get; set; } + + /// + /// Gets or sets the coded sample width in pixels before track presentation transforms. + /// + public int CodedWidth { get; set; } + + /// + /// Gets or sets the coded sample height in pixels before track presentation transforms. + /// + public int CodedHeight { get; set; } + + /// + /// Gets or sets the media time scale in units per second. + /// + public uint MediaTimescale { get; set; } + + /// + /// Gets or sets the declared media duration in media-time-scale units. + /// + public ulong MediaDuration { get; set; } + + /// + /// Gets or sets the total number of samples declared by the sample table. + /// + public uint TotalSampleCount { get; set; } + + /// + /// Gets or sets the coded sample-entry type. + /// + public Heif4CharCode CodecType { get; set; } + + /// + /// Gets or sets the parsed AV1 configuration when is . + /// + public Av1CodecConfiguration? Av1CodecConfiguration { get; set; } + + /// + /// Gets or sets the ICC profile associated with the image sequence. + /// + public IccProfile? IccProfile { get; set; } + + /// + /// Gets or sets the CICP color description associated with the image sequence. + /// + public CicpProfile? CicpProfile { get; set; } + + /// + /// Gets or sets the content light-level information associated with the image sequence. + /// + public HeifContentLightLevel? ContentLightLevel { get; set; } + + /// + /// Gets or sets the mastering-display color volume associated with the image sequence. + /// + public HeifMasteringDisplayColorVolume? MasteringDisplayColorVolume { get; set; } + + /// + /// Gets or sets the content color volume associated with the image sequence. + /// + public HeifContentColorVolume? ContentColorVolume { get; set; } + + /// + /// Gets or sets the nominal ambient viewing environment associated with the image sequence. + /// + public HeifAmbientViewingEnvironment? AmbientViewingEnvironment { get; set; } + + /// + /// Gets or sets the reference mastering environment associated with the image sequence. + /// + public HeifReferenceViewingEnvironment? ReferenceViewingEnvironment { get; set; } + + /// + /// Gets or sets the nominal diffuse-white description associated with the image sequence. + /// + public HeifNominalDiffuseWhite? NominalDiffuseWhite { get; set; } + + /// + /// Gets or sets the relative pixel spacing associated with the image sequence. + /// + public HeifPixelAspectRatio? PixelAspectRatio { get; set; } + + /// + /// Gets or sets the clean-aperture crop applied to each decoded sample. + /// + public HeifCleanAperture? CleanAperture { get; set; } + + /// + /// Gets or sets the number of 90-degree counter-clockwise rotations applied to each decoded sample. + /// + public byte? RotationAngle { get; set; } + + /// + /// Gets or sets the image-mirror axis, where zero is horizontal and one is vertical. + /// + public byte? MirrorAxis { get; set; } + + /// + /// Gets or sets the Exif and XMP item payloads implicitly associated with the image-sequence track. + /// + public HeifSequenceMetadata? Metadata { get; set; } + + /// + /// Gets or sets the retained sample descriptors in decode order. + /// + public HeifSequenceSample[] Samples { get; set; } = []; + + /// + /// Gets or sets the compact zero-based sample indices referenced by retained samples. + /// + public int[] DirectReferenceSampleIndices { get; set; } = []; + + /// + /// Gets or sets the identifier of the master track served by this auxiliary track, or zero for a master track. + /// + public uint AuxiliaryForTrackId { get; set; } + + /// + /// Gets or sets a value indicating whether the track is an alpha auxiliary image sequence. + /// + public bool IsAlpha { get; set; } + + /// + /// Gets or sets the identifier of the alpha track that premultiplies this color track, or zero when color is unassociated. + /// + public uint PremultipliedByTrackId { get; set; } + + /// + /// Gets or sets a value indicating whether the color track is premultiplied by this auxiliary alpha track. + /// + public bool IsPremultiplied { get; set; } + + /// + /// Gets or sets a value indicating whether every reference picture is intra coded. + /// + public bool AllReferencePicturesIntra { get; set; } + + /// + /// Gets or sets a value indicating whether coded pictures use intra-picture prediction. + /// + public bool IntraPicturePredictionUsed { get; set; } + + /// + /// Gets or sets the maximum number of reference pictures permitted for one coded picture. + /// + public byte MaximumReferencesPerPicture { get; set; } + + /// + /// Gets or sets the number of times the sequence is played. Zero indicates indefinite repetition. + /// + public ushort RepeatCount { get; set; } = 1; + + /// + /// Gets or sets the track handler type. + /// + public Heif4CharCode HandlerType { get; set; } + + /// + /// Gets or sets the track duration in movie-time-scale units. + /// + public ulong TrackDuration { get; set; } + + /// + /// Gets or sets a value indicating whether the track contains the edit list required for hidden samples. + /// + public bool HasEditList { get; set; } +} diff --git a/src/ImageSharp/Formats/Heif/HeifTrackMatrix.cs b/src/ImageSharp/Formats/Heif/HeifTrackMatrix.cs new file mode 100644 index 0000000000..989b4803ff --- /dev/null +++ b/src/ImageSharp/Formats/Heif/HeifTrackMatrix.cs @@ -0,0 +1,105 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Buffers.Binary; + +namespace SixLabors.ImageSharp.Formats.Heif; + +/// +/// Contains a fixed-point movie or track transformation matrix from a HEIF image sequence. +/// +internal readonly struct HeifTrackMatrix +{ + /// + /// Initializes a new instance of the struct. + /// + /// The horizontal scale and rotation coefficient in 16.16 fixed-point form. + /// The horizontal skew and rotation coefficient in 16.16 fixed-point form. + /// The first perspective coefficient in 2.30 fixed-point form. + /// The vertical skew and rotation coefficient in 16.16 fixed-point form. + /// The vertical scale and rotation coefficient in 16.16 fixed-point form. + /// The second perspective coefficient in 2.30 fixed-point form. + /// The horizontal translation in 16.16 fixed-point form. + /// The vertical translation in 16.16 fixed-point form. + /// The homogeneous scale coefficient in 2.30 fixed-point form. + public HeifTrackMatrix(int a, int b, int u, int c, int d, int v, int x, int y, int w) + { + this.A = a; + this.B = b; + this.U = u; + this.C = c; + this.D = d; + this.V = v; + this.X = x; + this.Y = y; + this.W = w; + } + + /// + /// Gets the horizontal scale and rotation coefficient in 16.16 fixed-point form. + /// + public int A { get; } + + /// + /// Gets the horizontal skew and rotation coefficient in 16.16 fixed-point form. + /// + public int B { get; } + + /// + /// Gets the first perspective coefficient in 2.30 fixed-point form. + /// + public int U { get; } + + /// + /// Gets the vertical skew and rotation coefficient in 16.16 fixed-point form. + /// + public int C { get; } + + /// + /// Gets the vertical scale and rotation coefficient in 16.16 fixed-point form. + /// + public int D { get; } + + /// + /// Gets the second perspective coefficient in 2.30 fixed-point form. + /// + public int V { get; } + + /// + /// Gets the horizontal translation in 16.16 fixed-point form. + /// + public int X { get; } + + /// + /// Gets the vertical translation in 16.16 fixed-point form. + /// + public int Y { get; } + + /// + /// Gets the homogeneous scale coefficient in 2.30 fixed-point form. + /// + public int W { get; } + + /// + /// Gets a value indicating whether the matrix leaves the image coordinate system unchanged. + /// + public bool IsIdentity => this.A == 0x00010000 && this.B == 0 && this.U == 0 && this.C == 0 && this.D == 0x00010000 && + this.V == 0 && this.X == 0 && this.Y == 0 && this.W == 0x40000000; + + /// + /// Reads the nine fixed-point coefficients from a matrix payload in file byte order. + /// + /// The 36-byte matrix payload. + /// The decoded transformation matrix. + public static HeifTrackMatrix Parse(ReadOnlySpan data) + => new( + BinaryPrimitives.ReadInt32BigEndian(data), + BinaryPrimitives.ReadInt32BigEndian(data[4..]), + BinaryPrimitives.ReadInt32BigEndian(data[8..]), + BinaryPrimitives.ReadInt32BigEndian(data[12..]), + BinaryPrimitives.ReadInt32BigEndian(data[16..]), + BinaryPrimitives.ReadInt32BigEndian(data[20..]), + BinaryPrimitives.ReadInt32BigEndian(data[24..]), + BinaryPrimitives.ReadInt32BigEndian(data[28..]), + BinaryPrimitives.ReadInt32BigEndian(data[32..])); +} diff --git a/src/ImageSharp/Formats/Heif/HeifTrackMetadataParser.cs b/src/ImageSharp/Formats/Heif/HeifTrackMetadataParser.cs new file mode 100644 index 0000000000..def4dbb774 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/HeifTrackMetadataParser.cs @@ -0,0 +1,718 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Buffers; +using System.Buffers.Binary; +using SixLabors.ImageSharp.Memory; + +namespace SixLabors.ImageSharp.Formats.Heif; + +/// +/// Parses the Exif and XMP items implicitly associated with one HEIF image-sequence track. +/// +internal sealed class HeifTrackMetadataParser +{ + /// + /// The allocator used for temporary item identifiers and extent descriptors. + /// + private readonly MemoryAllocator allocator; + + /// + /// Initializes a new instance of the class. + /// + /// The allocator used for bounded temporary parser state. + public HeifTrackMetadataParser(MemoryAllocator allocator) => this.allocator = allocator; + + /// + /// Gets the registered MIME content type for XMP metadata without allocating a managed byte array. + /// + private static ReadOnlySpan XmpContentType => "application/rdf+xml"u8; + + /// + /// Parses the bounded metadata items from one track-level metadata box. + /// + /// The seekable HEIF stream positioned at the metadata full-box header. + /// The validated metadata payload length. + /// The caller-owned reusable parser scratch. + /// The retained Exif and XMP payloads. + public HeifSequenceMetadata Parse(Stream stream, long boxLength, Span scratch) + { + long metadataStart = stream.Position; + long metadataEnd = checked(metadataStart + boxLength); + HeifBoxReader.EnsureInsideParent(boxLength, stream.Length - metadataStart); + ReadOnlySpan fullBoxHeader = ReadPrefix(stream, boxLength, scratch, 4, "track metadata"); + if (BinaryPrimitives.ReadUInt32BigEndian(fullBoxHeader) != 0) + { + throw new InvalidImageContentException("The track metadata box has unsupported version or flags."); + } + + BoxReference handler = default; + BoxReference itemInformation = default; + BoxReference itemLocations = default; + BoxReference itemData = default; + bool firstChild = true; + while (stream.Position < metadataEnd) + { + long childLength = HeifBoxReader.ReadHeader(stream, metadataEnd, scratch, out Heif4CharCode childType); + long childStart = stream.Position; + if (firstChild && childType != Heif4CharCode.Hdlr) + { + throw new InvalidImageContentException("The track metadata box does not begin with its picture handler."); + } + + switch (childType) + { + case Heif4CharCode.Hdlr: + SetUnique(ref handler, childStart, childLength, childType); + break; + case Heif4CharCode.Iinf: + SetUnique(ref itemInformation, childStart, childLength, childType); + break; + case Heif4CharCode.Iloc: + SetUnique(ref itemLocations, childStart, childLength, childType); + break; + case Heif4CharCode.Idat: + SetUnique(ref itemData, childStart, childLength, childType); + break; + } + + firstChild = false; + stream.Position = checked(childStart + childLength); + } + + if (!handler.IsPresent) + { + throw new InvalidImageContentException("The track metadata box has no picture handler."); + } + + stream.Position = handler.Offset; + if (ParseHandler(stream, handler.Length, scratch) != Heif4CharCode.Pict) + { + throw new InvalidImageContentException("The track metadata box does not use the picture handler."); + } + + if (!itemInformation.IsPresent && !itemLocations.IsPresent) + { + return new HeifSequenceMetadata(null, null); + } + + if (!itemInformation.IsPresent || !itemLocations.IsPresent) + { + throw new InvalidImageContentException("The track metadata box has incomplete item declarations or locations."); + } + + stream.Position = itemInformation.Offset; + MetadataItemIds itemIds = this.ParseItemInformation(stream, itemInformation.Length, scratch); + byte[]? exifData = itemIds.ExifItemId == 0 + ? null + : this.ReadItemPayload(stream, itemLocations, itemData, itemIds.ExifItemId, scratch); + + byte[]? xmpData = itemIds.XmpItemId == 0 + ? null + : this.ReadItemPayload(stream, itemLocations, itemData, itemIds.XmpItemId, scratch); + + return new HeifSequenceMetadata(exifData, xmpData); + } + + /// + /// Parses item declarations while retaining only the identifiers for Exif and unencoded XMP items. + /// + /// The stream positioned at the item-information full-box header. + /// The validated item-information payload length. + /// The caller-owned reusable parser scratch. + /// The recognized metadata item identifiers. + private MetadataItemIds ParseItemInformation(Stream stream, long boxLength, Span scratch) + { + long itemInformationStart = stream.Position; + long itemInformationEnd = checked(itemInformationStart + boxLength); + ReadOnlySpan prefix = ReadPrefix(stream, boxLength, scratch, 4, "track item information"); + byte version = prefix[0]; + int headerLength = version switch + { + 0 => 6, + 1 => 8, + _ => throw new InvalidImageContentException($"The track item-information box has unsupported version {version}.") + }; + + prefix = ReadPrefixFromStart(stream, boxLength, scratch, headerLength, "track item information"); + if ((BinaryPrimitives.ReadUInt32BigEndian(prefix) & 0x00FFFFFF) != 0) + { + throw new InvalidImageContentException("The track item-information box has unsupported flags."); + } + + uint entryCount = version == 0 + ? BinaryPrimitives.ReadUInt16BigEndian(prefix[4..]) + : BinaryPrimitives.ReadUInt32BigEndian(prefix[4..]); + + long entryBytes = itemInformationEnd - stream.Position; + if (entryCount > int.MaxValue || entryCount > (ulong)(entryBytes / 8)) + { + throw new InvalidImageContentException("The track item-information entry count exceeds its bounded payload."); + } + + if (entryCount == 0) + { + if (entryBytes != 0) + { + throw new InvalidImageContentException("The empty track item-information box contains trailing data."); + } + + return default; + } + + using IMemoryOwner identifierOwner = this.allocator.Allocate((int)entryCount); + Span identifiers = identifierOwner.GetSpan()[..(int)entryCount]; + MetadataItemIds result = default; + for (int i = 0; i < identifiers.Length; i++) + { + if (stream.Position >= itemInformationEnd) + { + throw new InvalidImageContentException("The track item-information entry count exceeds its bounded payload."); + } + + long entryLength = HeifBoxReader.ReadHeader(stream, itemInformationEnd, scratch, out Heif4CharCode entryType); + if (entryType != Heif4CharCode.Infe) + { + throw new InvalidImageContentException($"The track item-information box contains unexpected child '{entryType}'."); + } + + uint itemId = ParseItemInformationEntry(stream, entryLength, scratch, out Heif4CharCode itemType, out bool isXmp); + identifiers[i] = itemId; + if (itemType == Heif4CharCode.Exif) + { + if (result.ExifItemId != 0) + { + throw new InvalidImageContentException("The image-sequence track declares more than one Exif metadata item."); + } + + result.ExifItemId = itemId; + } + else if (isXmp) + { + if (result.XmpItemId != 0) + { + throw new InvalidImageContentException("The image-sequence track declares more than one XMP metadata item."); + } + + result.XmpItemId = itemId; + } + } + + if (stream.Position != itemInformationEnd) + { + throw new InvalidImageContentException("The track item-information entry count does not consume its bounded payload."); + } + + identifiers.Sort(); + if (identifiers[0] == 0) + { + throw new InvalidImageContentException("The track item-information box declares item identifier zero."); + } + + for (int i = 1; i < identifiers.Length; i++) + { + if (identifiers[i] == identifiers[i - 1]) + { + throw new InvalidImageContentException($"The track item-information box contains duplicate item ID {identifiers[i]}."); + } + } + + return result; + } + + /// + /// Parses one item-information entry without materializing its name or MIME strings. + /// + /// The stream positioned at the item-information-entry full-box header. + /// The validated item-information-entry payload length. + /// The caller-owned reusable parser scratch. + /// Receives the explicit item type for version two or three entries. + /// Receives whether the entry declares an unencoded XMP MIME item. + /// The positive item identifier. + private static uint ParseItemInformationEntry( + Stream stream, + long boxLength, + Span scratch, + out Heif4CharCode itemType, + out bool isXmp) + { + long entryStart = stream.Position; + long entryEnd = checked(entryStart + boxLength); + ReadOnlySpan prefix = ReadPrefix(stream, boxLength, scratch, 4, "track item-information entry"); + byte version = prefix[0]; + int fixedLength = version switch + { + 0 or 1 => 8, + 2 => 12, + 3 => 14, + _ => throw new InvalidImageContentException($"The track item-information entry has unsupported version {version}.") + }; + + prefix = ReadPrefixFromStart(stream, boxLength, scratch, fixedLength, "track item-information entry"); + uint flags = BinaryPrimitives.ReadUInt32BigEndian(prefix) & 0x00FFFFFF; + if ((flags & ~1U) != 0) + { + throw new InvalidImageContentException("The track item-information entry has unsupported flags."); + } + + int itemIdOffset = 4; + uint itemId = version == 3 + ? BinaryPrimitives.ReadUInt32BigEndian(prefix[itemIdOffset..]) + : BinaryPrimitives.ReadUInt16BigEndian(prefix[itemIdOffset..]); + + int protectionOffset = version == 3 ? 8 : 6; + if (BinaryPrimitives.ReadUInt16BigEndian(prefix[protectionOffset..]) != 0) + { + throw new InvalidImageContentException($"Track metadata item {itemId} uses unsupported item protection."); + } + + itemType = version >= 2 + ? (Heif4CharCode)BinaryPrimitives.ReadUInt32BigEndian(prefix[(protectionOffset + 2)..]) + : default; + + isXmp = false; + if (version < 2 || itemType is not Heif4CharCode.Exif and not Heif4CharCode.Mime) + { + stream.Position = entryEnd; + return itemId; + } + + HeifBoxPayloadReader reader = new(stream, entryEnd - stream.Position, scratch, "track item-information entry"); + reader.SkipNullTerminatedString(); + if (itemType == Heif4CharCode.Mime) + { + isXmp = reader.ReadNullTerminatedStringEquals(XmpContentType); + if (reader.Remaining > 0) + { + // ImageSharp cannot decode an encoded MIME payload. Retain XMP only when the optional encoding is empty. + isXmp &= reader.ReadNullTerminatedStringEquals(ReadOnlySpan.Empty); + } + } + + if (!reader.IsComplete) + { + throw new InvalidImageContentException($"Track metadata item {itemId} contains unexpected trailing data."); + } + + return itemId; + } + + /// + /// Resolves and reads one recognized metadata item while pooling only its transient extent descriptors. + /// + /// The seekable HEIF stream. + /// The item-location payload range. + /// The optional item-data payload range. + /// The recognized metadata item identifier. + /// The caller-owned reusable parser scratch. + /// The exact retained item payload, or for an item without data. + private byte[]? ReadItemPayload( + Stream stream, + BoxReference itemLocations, + BoxReference itemData, + uint itemId, + Span scratch) + { + stream.Position = itemLocations.Offset; + ItemLocationSummary summary = ParseItemLocation( + stream, + itemLocations.Length, + itemData, + itemId, + Span.Empty, + scratch); + + if (summary.ExtentCount == 0 || summary.TotalLength == 0) + { + return null; + } + + using IMemoryOwner extentOwner = this.allocator.Allocate(summary.ExtentCount); + Span extents = extentOwner.GetSpan()[..summary.ExtentCount]; + stream.Position = itemLocations.Offset; + ItemLocationSummary verifiedSummary = ParseItemLocation(stream, itemLocations.Length, itemData, itemId, extents, scratch); + if (verifiedSummary.TotalLength != summary.TotalLength) + { + throw new InvalidImageContentException($"Track metadata item {itemId} changed between location parser passes."); + } + + // The payload survives parser disposal and is handed directly to ImageSharp's metadata profile model. + byte[] data = new byte[summary.TotalLength]; + int destinationOffset = 0; + foreach (MetadataExtent extent in extents) + { + stream.Position = extent.Offset; + HeifBoxReader.ReadExactly( + stream, + data.AsSpan(destinationOffset, extent.Length), + $"Track metadata item {itemId} has a truncated extent."); + + destinationOffset += extent.Length; + } + + return data; + } + + /// + /// Parses one item-location box for a selected metadata item. + /// + /// The stream positioned at the item-location full-box header. + /// The validated item-location payload length. + /// The optional item-data payload range. + /// The metadata item identifier whose extents are retained. + /// The exact target extent span, or an empty span for the counting pass. + /// The caller-owned reusable parser scratch. + /// The target item's extent count and total payload length. + private static ItemLocationSummary ParseItemLocation( + Stream stream, + long boxLength, + BoxReference itemData, + uint targetItemId, + Span targetExtents, + Span scratch) + { + HeifBoxPayloadReader reader = new(stream, boxLength, scratch, "track item location"); + uint versionAndFlags = reader.ReadUInt32(); + byte version = (byte)(versionAndFlags >> 24); + if (version > 2 || (versionAndFlags & 0x00FFFFFF) != 0) + { + throw new InvalidImageContentException("The track item-location box has unsupported version or flags."); + } + + byte offsetAndLengthSizes = reader.ReadByte(); + byte baseAndIndexSizes = reader.ReadByte(); + int offsetSize = offsetAndLengthSizes >> 4; + int lengthSize = offsetAndLengthSizes & 15; + int baseOffsetSize = baseAndIndexSizes >> 4; + int indexSize = version is 1 or 2 ? baseAndIndexSizes & 15 : 0; + if (!IsSupportedFieldSize(offsetSize) + || !IsSupportedFieldSize(lengthSize) + || !IsSupportedFieldSize(baseOffsetSize) + || !IsSupportedFieldSize(indexSize)) + { + throw new InvalidImageContentException("The track item-location box uses an unsupported integer field size."); + } + + uint itemCount = version == 2 ? reader.ReadUInt32() : reader.ReadUInt16(); + int minimumItemLength = version == 0 ? 6 : 8; + if (itemCount > (ulong)(reader.Remaining / minimumItemLength)) + { + throw new InvalidImageContentException("The track item-location count exceeds its bounded payload."); + } + + bool targetFound = false; + int targetExtentIndex = 0; + int targetLength = 0; + for (uint i = 0; i < itemCount; i++) + { + uint itemId = version == 2 ? reader.ReadUInt32() : reader.ReadUInt16(); + int constructionMethod = 0; + if (version is 1 or 2) + { + ushort constructionField = reader.ReadUInt16(); + if ((constructionField & 0xFFF0) != 0) + { + throw new InvalidImageContentException("The track item-location box has nonzero reserved construction bits."); + } + + constructionMethod = constructionField & 15; + } + + if (constructionMethod is not 0 and not 1) + { + throw new InvalidImageContentException($"The track item-location box uses unsupported construction method {constructionMethod}."); + } + + if (reader.ReadUInt16() != 0) + { + throw new InvalidImageContentException("External track metadata data references are not supported."); + } + + ulong baseOffset = reader.ReadVariableUInt(baseOffsetSize); + int extentCount = reader.ReadUInt16(); + bool isTarget = itemId == targetItemId; + if (isTarget) + { + if (targetFound) + { + throw new InvalidImageContentException($"The track item-location box contains duplicate locations for item ID {itemId}."); + } + + if (!targetExtents.IsEmpty && targetExtents.Length != extentCount) + { + throw new InvalidImageContentException($"Track metadata item {itemId} changed between location parser passes."); + } + + targetFound = true; + } + + for (int j = 0; j < extentCount; j++) + { + if (version is 1 or 2 && indexSize > 0) + { + _ = reader.ReadVariableUInt(indexSize); + } + + ulong extentOffset = reader.ReadVariableUInt(offsetSize); + ulong extentLength = reader.ReadVariableUInt(lengthSize); + if (!isTarget) + { + continue; + } + + MetadataExtent extent = ResolveExtent(stream, itemData, constructionMethod, baseOffset, extentOffset, extentLength, itemId); + if (!targetExtents.IsEmpty) + { + targetExtents[targetExtentIndex] = extent; + } + + targetExtentIndex++; + if (extent.Length > int.MaxValue - targetLength) + { + throw new InvalidImageContentException($"Track metadata item {itemId} has an unsupported combined length."); + } + + targetLength += extent.Length; + } + } + + if (!reader.IsComplete) + { + throw new InvalidImageContentException("The track item-location box contains unexpected trailing data."); + } + + return targetFound ? new ItemLocationSummary(targetExtentIndex, targetLength) : default; + } + + /// + /// Resolves one file-relative or item-data-relative extent into an absolute stream range. + /// + /// The complete seekable HEIF stream. + /// The optional item-data payload range. + /// The item-location construction method. + /// The item-location base offset. + /// The extent offset relative to the base offset. + /// The extent length in bytes. + /// The metadata item identifier used in malformed-image diagnostics. + /// The validated absolute extent. + private static MetadataExtent ResolveExtent( + Stream stream, + BoxReference itemData, + int constructionMethod, + ulong baseOffset, + ulong extentOffset, + ulong extentLength, + uint itemId) + { + if (baseOffset > ulong.MaxValue - extentOffset || extentLength > int.MaxValue) + { + throw new InvalidImageContentException($"Track metadata item {itemId} has an unsupported extent range."); + } + + ulong relativeOffset = baseOffset + extentOffset; + ulong origin = 0; + if (constructionMethod == 1) + { + if (!itemData.IsPresent || relativeOffset > (ulong)itemData.Length || extentLength > (ulong)itemData.Length - relativeOffset) + { + throw new InvalidImageContentException($"Track metadata item {itemId} has an extent outside its item-data box."); + } + + origin = (ulong)itemData.Offset; + } + + if (relativeOffset > ulong.MaxValue - origin) + { + throw new InvalidImageContentException($"Track metadata item {itemId} has an unsupported extent offset."); + } + + ulong absoluteOffset = origin + relativeOffset; + if (absoluteOffset > (ulong)stream.Length || extentLength > (ulong)stream.Length - absoluteOffset) + { + throw new InvalidImageContentException($"Track metadata item {itemId} has an extent outside the HEIF stream."); + } + + return new MetadataExtent((long)absoluteOffset, (int)extentLength); + } + + /// + /// Parses one picture handler and returns its registered handler type. + /// + /// The stream positioned at the handler full-box header. + /// The validated handler payload length. + /// The caller-owned reusable parser scratch. + /// The registered handler type. + private static Heif4CharCode ParseHandler(Stream stream, long boxLength, Span scratch) + { + ReadOnlySpan prefix = ReadPrefix(stream, boxLength, scratch, 24, "track metadata handler"); + if (BinaryPrimitives.ReadUInt32BigEndian(prefix) != 0 || BinaryPrimitives.ReadUInt32BigEndian(prefix[4..]) != 0) + { + throw new InvalidImageContentException("The track metadata handler has unsupported fields."); + } + + return (Heif4CharCode)BinaryPrimitives.ReadUInt32BigEndian(prefix[8..]); + } + + /// + /// Determines whether an item-location integer width is supported by the bounded reader. + /// + /// The width in bytes. + /// for zero-width, 32-bit, or 64-bit fields. + private static bool IsSupportedFieldSize(int size) => size is 0 or 4 or 8; + + /// + /// Reads a fixed prefix from the stream's current position. + /// + /// The source stream. + /// The validated enclosing payload length. + /// The caller-owned reusable parser scratch. + /// The required prefix length. + /// The payload name used in malformed-image diagnostics. + /// The requested prefix within . + private static ReadOnlySpan ReadPrefix(Stream stream, long boxLength, Span scratch, int length, string name) + { + if (boxLength < length) + { + throw new InvalidImageContentException($"The {name} payload is truncated."); + } + + Span destination = scratch[..length]; + HeifBoxReader.ReadExactly(stream, destination, $"The {name} payload is truncated."); + return destination; + } + + /// + /// Rewinds to the start of a partially read payload and reads a larger fixed prefix. + /// + /// The source stream positioned after a four-byte prefix. + /// The validated enclosing payload length. + /// The caller-owned reusable parser scratch. + /// The required prefix length. + /// The payload name used in malformed-image diagnostics. + /// The requested prefix within . + private static ReadOnlySpan ReadPrefixFromStart(Stream stream, long boxLength, Span scratch, int length, string name) + { + stream.Position -= 4; + return ReadPrefix(stream, boxLength, scratch, length, name); + } + + /// + /// Records one unique metadata child while retaining only its stream range. + /// + /// The child reference owned by the metadata parser. + /// The absolute payload offset. + /// The validated payload length. + /// The unique child box type. + private static void SetUnique(ref BoxReference reference, long offset, long length, Heif4CharCode boxType) + { + if (reference.IsPresent) + { + throw new InvalidImageContentException($"The track metadata box contains duplicate '{boxType}' boxes."); + } + + reference = new BoxReference(offset, length); + } + + /// + /// Retains the recognized item identifiers from one track metadata box. + /// + private struct MetadataItemIds + { + /// + /// Gets or sets the Exif item identifier, or zero when absent. + /// + public uint ExifItemId { get; set; } + + /// + /// Gets or sets the XMP item identifier, or zero when absent. + /// + public uint XmpItemId { get; set; } + } + + /// + /// Retains one unique child payload range without creating a generic metadata box model. + /// + private readonly struct BoxReference + { + /// + /// Initializes a new instance of the struct. + /// + /// The absolute payload offset. + /// The validated payload length. + public BoxReference(long offset, long length) + { + this.Offset = offset; + this.Length = length; + this.IsPresent = true; + } + + /// + /// Gets the absolute payload offset. + /// + public long Offset { get; } + + /// + /// Gets the validated payload length. + /// + public long Length { get; } + + /// + /// Gets a value indicating whether the child was present. + /// + public bool IsPresent { get; } + } + + /// + /// Describes one absolute metadata item extent. + /// + private readonly struct MetadataExtent + { + /// + /// Initializes a new instance of the struct. + /// + /// The absolute stream offset. + /// The extent length in bytes. + public MetadataExtent(long offset, int length) + { + this.Offset = offset; + this.Length = length; + } + + /// + /// Gets the absolute stream offset. + /// + public long Offset { get; } + + /// + /// Gets the extent length in bytes. + /// + public int Length { get; } + } + + /// + /// Summarizes the retained extents for one metadata item. + /// + private readonly struct ItemLocationSummary + { + /// + /// Initializes a new instance of the struct. + /// + /// The number of retained extents. + /// The combined payload length. + public ItemLocationSummary(int extentCount, int totalLength) + { + this.ExtentCount = extentCount; + this.TotalLength = totalLength; + } + + /// + /// Gets the number of retained extents. + /// + public int ExtentCount { get; } + + /// + /// Gets the combined payload length. + /// + public int TotalLength { get; } + } +} diff --git a/src/ImageSharp/Formats/Heif/IHeifItemDecoder.cs b/src/ImageSharp/Formats/Heif/IHeifItemDecoder.cs new file mode 100644 index 0000000000..9835a47086 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/IHeifItemDecoder.cs @@ -0,0 +1,43 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Metadata.Profiles.Cicp; +using SixLabors.ImageSharp.PixelFormats; + +namespace SixLabors.ImageSharp.Formats.Heif; + +/// +/// Decodes the compressed payload of a single HEIF image item. +/// +/// The destination pixel type. +internal interface IHeifItemDecoder + where TPixel : unmanaged, IPixel +{ + /// + /// Gets the image item type decoded by this implementation. + /// + public Heif4CharCode Type { get; } + + /// + /// Gets the compression method used by the image item. + /// + public HeifCompressionMethod CompressionMethod { get; } + + /// + /// Decodes the compressed payload of an image item. + /// + /// The general options governing the containing HEIF decode. + /// The HEIF item whose encoded payload is being decoded. + /// The encoded image payload. + /// + /// The container color description that overrides matching color information in the encoded image payload. + /// + /// The token used to cancel the payload decode. + /// The decoded image. + public Image DecodeItemData( + DecoderOptions options, + HeifItem item, + Span data, + CicpProfile? colorProfile, + CancellationToken cancellationToken); +} diff --git a/src/ImageSharp/Formats/Heif/JpegHeifItemDecoder.cs b/src/ImageSharp/Formats/Heif/JpegHeifItemDecoder.cs new file mode 100644 index 0000000000..831f62c164 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/JpegHeifItemDecoder.cs @@ -0,0 +1,52 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Jpeg; +using SixLabors.ImageSharp.Metadata.Profiles.Cicp; +using SixLabors.ImageSharp.PixelFormats; + +namespace SixLabors.ImageSharp.Formats.Heif; + +/// +/// Decodes a single JPEG-coded HEIF image item. +/// +/// The destination pixel type. +internal class JpegHeifItemDecoder : IHeifItemDecoder + where TPixel : unmanaged, IPixel +{ + /// + /// Gets the JPEG-coded image item type. + /// + public Heif4CharCode Type => Heif4CharCode.Jpeg; + + /// + /// Gets the legacy JPEG compression method. + /// + public HeifCompressionMethod CompressionMethod => HeifCompressionMethod.LegacyJpeg; + + /// + /// Decodes the encoded JPEG payload of an image item. + /// + /// The general options governing the containing HEIF decode. + /// The HEIF item whose encoded payload is being decoded. + /// The encoded JPEG payload. + /// The container color description associated with the image item. + /// The token used to cancel the payload decode. + /// The decoded image. + public unsafe Image DecodeItemData( + DecoderOptions options, + HeifItem item, + Span data, + CicpProfile? colorProfile, + CancellationToken cancellationToken) + { + // The JPEG decoder owns the payload's JPEG color coding. The containing decoder attaches HEIF CICP as + // presentation metadata after payload decode, so it must not be mistaken for JPEG component-transform syntax. + fixed (byte* dataPointer = data) + { + using UnmanagedMemoryStream stream = new(dataPointer, data.Length); + using JpegDecoderCore decoder = new(new JpegDecoderOptions { GeneralOptions = options }); + return decoder.Decode(options.Configuration, stream, cancellationToken); + } + } +} diff --git a/src/ImageSharp/Formats/Heif/MetadataExtensions.cs b/src/ImageSharp/Formats/Heif/MetadataExtensions.cs new file mode 100644 index 0000000000..9165b9182c --- /dev/null +++ b/src/ImageSharp/Formats/Heif/MetadataExtensions.cs @@ -0,0 +1,42 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif; +using SixLabors.ImageSharp.Metadata; + +namespace SixLabors.ImageSharp; + +/// +/// Extension methods for the type. +/// +public static partial class MetadataExtensions +{ + /// + /// Gets the HEIF format specific metadata for the image. + /// + /// The metadata this method extends. + /// The . + public static HeifMetadata GetHeifMetadata(this ImageMetadata metadata) => metadata.GetFormatMetadata(HeifFormat.Instance); + + /// + /// Creates a new cloned instance of the HEIF metadata associated with the image. + /// + /// The metadata this method extends. + /// The cloned . + public static HeifMetadata CloneHeifMetadata(this ImageMetadata metadata) => metadata.CloneFormatMetadata(HeifFormat.Instance); + + /// + /// Gets the HEIF format-specific metadata for the image frame. If none is present, metadata is converted from + /// the decoded format or a default instance is created and associated with the frame. + /// + /// The metadata this method extends. + /// The . + public static HeifFrameMetadata GetHeifMetadata(this ImageFrameMetadata metadata) => metadata.GetFormatMetadata(HeifFormat.Instance); + + /// + /// Creates a new cloned instance of the HEIF metadata associated with the image frame. + /// + /// The metadata this method extends. + /// The cloned . + public static HeifFrameMetadata CloneHeifMetadata(this ImageFrameMetadata metadata) => metadata.CloneFormatMetadata(HeifFormat.Instance); +} diff --git a/src/ImageSharp/Formats/Heif/Readme.md b/src/ImageSharp/Formats/Heif/Readme.md new file mode 100644 index 0000000000..ee16813fc7 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Readme.md @@ -0,0 +1,7 @@ +# Implementation references + +[MPEG-4 register authority](https://mp4ra.org/) + +[AV1 bitstream and decoding reference implementation](https://aomedia.googlesource.com/aom/) + +[AV1 bitstream specification](https://aomediacodec.github.io/av1-spec/) diff --git a/src/ImageSharp/Formats/ImageDecoderCore.cs b/src/ImageSharp/Formats/ImageDecoderCore.cs index 3f36b8931f..a5b238c58d 100644 --- a/src/ImageSharp/Formats/ImageDecoderCore.cs +++ b/src/ImageSharp/Formats/ImageDecoderCore.cs @@ -39,7 +39,7 @@ protected ImageDecoderCore(DecoderOptions options) /// The action. protected void ExecuteAncillarySegmentAction(Action action) { - if (this.Options.SegmentIntegrityHandling is SegmentIntegrityHandling.Strict) + if (!ShouldIgnoreAncillarySegmentErrors(this.Options)) { action(); return; @@ -49,12 +49,7 @@ protected void ExecuteAncillarySegmentAction(Action action) { action(); } - catch (Exception ex) when (ex - is ImageFormatException - or InvalidIccProfileException - or InvalidImageContentException - or InvalidOperationException - or NotSupportedException) + catch (Exception ex) when (IsRecoverableSegmentError(ex)) { // Intentionally ignored in non-strict segment integrity modes. } @@ -66,7 +61,7 @@ or InvalidOperationException /// The action. protected void ExecuteImageDataSegmentAction(Action action) { - if (this.Options.SegmentIntegrityHandling is not SegmentIntegrityHandling.IgnoreImageData) + if (!ShouldIgnoreImageDataSegmentErrors(this.Options)) { action(); return; @@ -76,21 +71,62 @@ protected void ExecuteImageDataSegmentAction(Action action) { action(); } - catch (Exception ex) when (ex - is ImageFormatException - or InvalidIccProfileException - or InvalidImageContentException - or InvalidOperationException - or NotSupportedException) + catch (Exception ex) when (IsRecoverableSegmentError(ex)) { // Intentionally ignored when image data integrity handling is set to IgnoreImageData. } } + /// + /// Determines whether an exception represents a recoverable image segment error. + /// + /// The exception raised while processing an image segment. + /// when a segment integrity policy may ignore the exception. + public static bool IsRecoverableSegmentError(Exception exception) + => exception is ImageFormatException + or InvalidIccProfileException + or InvalidImageContentException + or InvalidOperationException + or NotSupportedException; + + /// + /// Determines whether the configured policy permits recoverable ancillary-segment errors to be ignored. + /// + /// The general decoder options. + /// when recoverable ancillary-segment errors may be ignored. + public static bool ShouldIgnoreAncillarySegmentErrors(DecoderOptions options) + => options.SegmentIntegrityHandling is not SegmentIntegrityHandling.Strict; + + /// + /// Determines whether the configured policy permits recoverable image-data-segment errors to be ignored. + /// + /// The general decoder options. + /// when recoverable image-data-segment errors may be ignored. + public static bool ShouldIgnoreImageDataSegmentErrors(DecoderOptions options) + => options.SegmentIntegrityHandling is SegmentIntegrityHandling.IgnoreImageData; + + /// + /// Determines whether an ancillary-segment exception may be ignored by the configured decoder policy. + /// + /// The general decoder options. + /// The exception raised while processing an ancillary segment. + /// when decoding may continue without the ancillary segment. + public static bool ShouldIgnoreAncillarySegmentError(DecoderOptions options, Exception exception) + => ShouldIgnoreAncillarySegmentErrors(options) && IsRecoverableSegmentError(exception); + + /// + /// Determines whether an image-data-segment exception may be ignored by the configured decoder policy. + /// + /// The general decoder options. + /// The exception raised while processing an image-data segment. + /// when decoding may continue without the image-data segment. + public static bool ShouldIgnoreImageDataSegmentError(DecoderOptions options, Exception exception) + => ShouldIgnoreImageDataSegmentErrors(options) && IsRecoverableSegmentError(exception); + /// /// Throws unless the decoder is running in a non-strict segment integrity mode. - /// Use this only from within when local control flow - /// must continue after the error. + /// Use this when ancillary parsing must continue locally after the error rather than returning through + /// . /// /// The exception message. protected void ThrowOrIgnoreNonStrictSegmentError(string message) @@ -101,6 +137,18 @@ protected void ThrowOrIgnoreNonStrictSegmentError(string message) } } + /// + /// Throws unless the decoder permits recoverable image-data segment errors to be ignored. + /// + /// The exception message. + protected void ThrowOrIgnoreImageDataSegmentError(string message) + { + if (!ShouldIgnoreImageDataSegmentErrors(this.Options)) + { + throw new InvalidImageContentException(message); + } + } + /// /// Reads the raw image information from the specified stream. /// diff --git a/src/ImageSharp/Formats/Jpeg/Components/ColorConverters/JpegColorConverter.Operator.cs b/src/ImageSharp/Formats/Jpeg/Components/ColorConverters/JpegColorConverter.Operator.cs index 3b2ab914b3..39dec7ba98 100644 --- a/src/ImageSharp/Formats/Jpeg/Components/ColorConverters/JpegColorConverter.Operator.cs +++ b/src/ImageSharp/Formats/Jpeg/Components/ColorConverters/JpegColorConverter.Operator.cs @@ -4,7 +4,6 @@ using System.Runtime.CompilerServices; using System.Runtime.InteropServices; using System.Runtime.Intrinsics; -using SixLabors.ImageSharp.Common.Helpers; namespace SixLabors.ImageSharp.Formats.Jpeg.Components; diff --git a/src/ImageSharp/Formats/Jpeg/Components/Decoder/SpectralConverter{TPixel}.cs b/src/ImageSharp/Formats/Jpeg/Components/Decoder/SpectralConverter{TPixel}.cs index 2bd4b95fdd..d70e85476e 100644 --- a/src/ImageSharp/Formats/Jpeg/Components/Decoder/SpectralConverter{TPixel}.cs +++ b/src/ImageSharp/Formats/Jpeg/Components/Decoder/SpectralConverter{TPixel}.cs @@ -42,11 +42,6 @@ internal class SpectralConverter : SpectralConverter, IDisposable /// private IMemoryOwner rgbBuffer; - /// - /// Proxy buffer used in packing from RGB to target TPixel pixels. - /// - private IMemoryOwner paddedProxyPixelRow; - /// /// Resulting 2D pixel buffer. /// @@ -163,19 +158,8 @@ private void ConvertStride(int spectralStep, IccProfile iccProfile) SimdUtils.NormalizedFloatToByteSaturate(values.Component1, g); SimdUtils.NormalizedFloatToByteSaturate(values.Component2, b); - // PackFromRgbPlanes expects the destination to be padded, so try to get padded span containing extra elements from the next row. - // If we can't get such a padded row because we are on a MemoryGroup boundary or at the last row, - // pack pixels to a temporary, padded proxy buffer, then copy the relevant values to the destination row. - if (this.pixelBuffer.DangerousTryGetPaddedRowSpan(yy, 3, out Span destRow)) - { - PixelOperations.Instance.PackFromRgbPlanes(r, g, b, destRow); - } - else - { - Span proxyRow = this.paddedProxyPixelRow.GetSpan(); - PixelOperations.Instance.PackFromRgbPlanes(r, g, b, proxyRow); - proxyRow[..width].CopyTo(this.pixelBuffer.DangerousGetRowSpan(yy)); - } + Span destination = this.pixelBuffer.DangerousGetRowSpan(yy); + PixelOperations.Instance.PackFromRgbPlanes(r, g, b, destination); } this.pixelRowCounter += this.pixelRowsPerStep; @@ -214,7 +198,6 @@ public override void PrepareForDecoding() pixelSize.Height, this.Configuration.PreferContiguousImageBuffers, AllocationOptions.Clean); - this.paddedProxyPixelRow = allocator.Allocate(pixelSize.Width + 3); // Component processors from spectral to RGB int bufferWidth = majorBlockWidth * blockPixelSize; @@ -275,7 +258,6 @@ public void Dispose() } this.rgbBuffer?.Dispose(); - this.paddedProxyPixelRow?.Dispose(); this.pixelBuffer?.Dispose(); } } diff --git a/src/ImageSharp/Formats/Jpeg/JpegEncoderCore.cs b/src/ImageSharp/Formats/Jpeg/JpegEncoderCore.cs index c9b684009e..47c8d45a63 100644 --- a/src/ImageSharp/Formats/Jpeg/JpegEncoderCore.cs +++ b/src/ImageSharp/Formats/Jpeg/JpegEncoderCore.cs @@ -90,11 +90,13 @@ public void Encode(Image image, Stream stream, CancellationToken this.WriteApp14Marker(frameConfig.AdobeColorTransformMarkerFlag.Value, buffer); } - // Write Exif, XMP, ICC and IPTC profiles - this.WriteProfiles(metadata, buffer); - - // Write comments - this.WriteComments(image.Configuration, jpegMetadata); + if (!this.encoder.SkipMetadata) + { + // Profiles and comments are optional application metadata and must not leak into a containing format + // when the containing encoder has requested a metadata-free embedded JPEG payload. + this.WriteProfiles(metadata, buffer); + this.WriteComments(image.Configuration, jpegMetadata); + } // Write the image dimensions. this.WriteStartOfFrame(image.Width, image.Height, frameConfig, buffer); diff --git a/src/ImageSharp/Formats/Png/PngEncoderCore.cs b/src/ImageSharp/Formats/Png/PngEncoderCore.cs index caf2393d89..fe1de830f0 100644 --- a/src/ImageSharp/Formats/Png/PngEncoderCore.cs +++ b/src/ImageSharp/Formats/Png/PngEncoderCore.cs @@ -1057,7 +1057,7 @@ private static void AppendHex(StringBuilder sb, ReadOnlySpan data) /// CICP matrix coefficients other than Identity are not supported in PNG. private void WriteCicpChunk(Stream stream, ImageMetadata metaData) { - if (metaData.CicpProfile is null) + if (this.chunkFilter == PngChunkFilter.ExcludeAll || metaData.CicpProfile is null) { return; } diff --git a/src/ImageSharp/ImageSharp.csproj b/src/ImageSharp/ImageSharp.csproj index 971d73b849..a5ff12924f 100644 --- a/src/ImageSharp/ImageSharp.csproj +++ b/src/ImageSharp/ImageSharp.csproj @@ -52,6 +52,11 @@ + + True + True + Heif4CharCode.tt + True True @@ -160,6 +165,10 @@ + + TextTemplatingFileGenerator + Heif4CharCode.cs + TextTemplatingFileGenerator InlineArray.cs diff --git a/src/ImageSharp/Metadata/Profiles/CICP/Enums/CicpMatrixCoefficients.cs b/src/ImageSharp/Metadata/Profiles/CICP/Enums/CicpMatrixCoefficients.cs index 931beac846..fdf044c6e9 100644 --- a/src/ImageSharp/Metadata/Profiles/CICP/Enums/CicpMatrixCoefficients.cs +++ b/src/ImageSharp/Metadata/Profiles/CICP/Enums/CicpMatrixCoefficients.cs @@ -6,7 +6,7 @@ namespace SixLabors.ImageSharp.Metadata.Profiles.Cicp; #pragma warning disable CA1707 // Underscores in enum members /// -/// Matrix coefficients according to ITU-T H.273 / ISO/IEC 23091-2_2019 subclause 8.3 +/// Matrix coefficients according to ITU-T H.273. /// public enum CicpMatrixCoefficients : byte { @@ -91,6 +91,21 @@ public enum CicpMatrixCoefficients : byte /// Rec. ITU-R BT.2100-2 ICtCp /// ICtCp = 14, + + /// + /// IPT-C2. + /// + IptC2 = 15, + + /// + /// Reversible YCgCo with two additional encoded bits. + /// + YCgCoRe = 16, + + /// + /// Reversible YCgCo with one additional encoded bit. + /// + YCgCoRo = 17, } #pragma warning restore CA1707 // Underscores in enum members diff --git a/src/ImageSharp/Metadata/Profiles/ICC/IccProfile.cs b/src/ImageSharp/Metadata/Profiles/ICC/IccProfile.cs index eaba0a045c..d337b77e14 100644 --- a/src/ImageSharp/Metadata/Profiles/ICC/IccProfile.cs +++ b/src/ImageSharp/Metadata/Profiles/ICC/IccProfile.cs @@ -177,6 +177,22 @@ public byte[] ToByteArray() return IccWriter.Write(this); } + /// + /// Gets the serialized profile storage for an encoder that consumes it before the operation returns. + /// + /// The existing raw profile storage, or a newly serialized profile when this instance was built from entries. + internal ReadOnlyMemory GetDataForWriting() + { + if (this.data is not null) + { + // Internal encoders only read this memory during the current operation, so the immutable view avoids + // cloning an already-owned profile before it is copied into format-specific output. + return this.data; + } + + return IccWriter.Write(this); + } + private void InitializeHeader() { if (this.header != null) diff --git a/src/ImageSharp/PixelFormats/PixelImplementations/PixelOperations/L16.PixelOperations.cs b/src/ImageSharp/PixelFormats/PixelImplementations/PixelOperations/L16.PixelOperations.cs index b79d2a5b21..348ff35c42 100644 --- a/src/ImageSharp/PixelFormats/PixelImplementations/PixelOperations/L16.PixelOperations.cs +++ b/src/ImageSharp/PixelFormats/PixelImplementations/PixelOperations/L16.PixelOperations.cs @@ -2,6 +2,10 @@ // Licensed under the Six Labors Split License. using System.Numerics; +using System.Runtime.CompilerServices; +using System.Runtime.InteropServices; +using System.Runtime.Intrinsics; +using SixLabors.ImageSharp.Common.Helpers; namespace SixLabors.ImageSharp.PixelFormats; @@ -18,9 +22,163 @@ internal partial class PixelOperations : PixelOperations // Alpha is implicitly one, so both outward representations already contain associated color components. /// - protected override void ToAssociatedVector4(Configuration configuration, ReadOnlySpan source, Span destination) => this.ToUnassociatedVector4(configuration, source, destination); + protected override void ToUnassociatedVector4( + Configuration configuration, + ReadOnlySpan source, + Span destination) + => ConvertToVector4(source, destination); /// - protected override void ToAssociatedScaledVector4(Configuration configuration, ReadOnlySpan source, Span destination) => this.ToUnassociatedScaledVector4(configuration, source, destination); + protected override void ToUnassociatedScaledVector4( + Configuration configuration, + ReadOnlySpan source, + Span destination) + => ConvertToVector4(source, destination); + + /// + protected override void ToAssociatedVector4( + Configuration configuration, + ReadOnlySpan source, + Span destination) + => this.ToUnassociatedVector4(configuration, source, destination); + + /// + protected override void ToAssociatedScaledVector4( + Configuration configuration, + ReadOnlySpan source, + Span destination) + => this.ToUnassociatedScaledVector4(configuration, source, destination); + + /// + /// Expands packed luminance samples into normalized RGB vectors with opaque alpha. + /// + /// The packed luminance samples. + /// The destination vectors. + private static void ConvertToVector4(ReadOnlySpan source, Span destination) + { + ref ushort sourceBase = ref Unsafe.As(ref MemoryMarshal.GetReference(source)); + ref Vector4 destinationBase = ref MemoryMarshal.GetReference(destination); + int length = source.Length; + int i = 0; + + if (Vector512.IsHardwareAccelerated) + { + Vector512 maximum = Vector512.Create((float)ushort.MaxValue); + int samplesPerVector = Vector512.Count; + int oneVectorFromEnd = length - samplesPerVector; + + for (; i <= oneVectorFromEnd; i += samplesPerVector) + { + Vector512 packed = Vector512.LoadUnsafe(ref sourceBase, (nuint)i); + (Vector512 lower, Vector512 upper) = Vector512.Widen(packed); + + StoreLuminanceVectors(Vector512.ConvertToSingle(lower.AsInt32()) / maximum, ref Unsafe.Add(ref destinationBase, (uint)i)); + StoreLuminanceVectors(Vector512.ConvertToSingle(upper.AsInt32()) / maximum, ref Unsafe.Add(ref destinationBase, (uint)(i + (samplesPerVector / 2)))); + } + } + + if (Vector256.IsHardwareAccelerated) + { + Vector256 maximum = Vector256.Create((float)ushort.MaxValue); + int samplesPerVector = Vector256.Count; + int oneVectorFromEnd = length - samplesPerVector; + + for (; i <= oneVectorFromEnd; i += samplesPerVector) + { + Vector256 packed = Vector256.LoadUnsafe(ref sourceBase, (nuint)i); + (Vector256 lower, Vector256 upper) = Vector256.Widen(packed); + + StoreLuminanceVectors(Vector256.ConvertToSingle(lower.AsInt32()) / maximum, ref Unsafe.Add(ref destinationBase, (uint)i)); + StoreLuminanceVectors(Vector256.ConvertToSingle(upper.AsInt32()) / maximum, ref Unsafe.Add(ref destinationBase, (uint)(i + (samplesPerVector / 2)))); + } + } + + if (Vector128.IsHardwareAccelerated) + { + Vector128 maximum = Vector128.Create((float)ushort.MaxValue); + int samplesPerVector = Vector128.Count; + int oneVectorFromEnd = length - samplesPerVector; + + for (; i <= oneVectorFromEnd; i += samplesPerVector) + { + Vector128 packed = Vector128.LoadUnsafe(ref sourceBase, (nuint)i); + (Vector128 lower, Vector128 upper) = Vector128.Widen(packed); + + StoreLuminanceVectors(Vector128.ConvertToSingle(lower.AsInt32()) / maximum, ref Unsafe.Add(ref destinationBase, (uint)i)); + StoreLuminanceVectors(Vector128.ConvertToSingle(upper.AsInt32()) / maximum, ref Unsafe.Add(ref destinationBase, (uint)(i + (samplesPerVector / 2)))); + } + } + + for (; i < length; i++) + { + Unsafe.Add(ref destinationBase, (uint)i) = Unsafe.As(ref Unsafe.Add(ref sourceBase, (uint)i)).ToVector4(); + } + } + + /// + /// Replicates sixteen normalized luminance samples into sixteen RGB vectors with opaque alpha. + /// + /// The normalized luminance samples. + /// The first destination vector. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static void StoreLuminanceVectors(Vector512 source, ref Vector4 destination) + { + Vector512 rgbMask = Vector512.Create(-1, -1, -1, 0, -1, -1, -1, 0, -1, -1, -1, 0, -1, -1, -1, 0); + Vector512 opaqueAlpha = Vector512.Create(0F, 0F, 0F, 1F, 0F, 0F, 0F, 1F, 0F, 0F, 0F, 1F, 0F, 0F, 0F, 1F); + Vector512 indices0 = Vector512.Create(0, 0, 0, 0, 1, 1, 1, 1, 2, 2, 2, 2, 3, 3, 3, 3); + Vector512 indices1 = Vector512.Create(4, 4, 4, 4, 5, 5, 5, 5, 6, 6, 6, 6, 7, 7, 7, 7); + Vector512 indices2 = Vector512.Create(8, 8, 8, 8, 9, 9, 9, 9, 10, 10, 10, 10, 11, 11, 11, 11); + Vector512 indices3 = Vector512.Create(12, 12, 12, 12, 13, 13, 13, 13, 14, 14, 14, 14, 15, 15, 15, 15); + ref Vector512 destinationBase = ref Unsafe.As>(ref destination); + + // Native indexed shuffles expand four luminance values per store. Clearing every fourth lane before + // inserting one preserves the implicit opaque alpha without scalar lane extraction. + destinationBase = (Vector512.ShuffleNative(source, indices0) & rgbMask.AsSingle()) | opaqueAlpha; + Unsafe.Add(ref destinationBase, 1) = (Vector512.ShuffleNative(source, indices1) & rgbMask.AsSingle()) | opaqueAlpha; + Unsafe.Add(ref destinationBase, 2) = (Vector512.ShuffleNative(source, indices2) & rgbMask.AsSingle()) | opaqueAlpha; + Unsafe.Add(ref destinationBase, 3) = (Vector512.ShuffleNative(source, indices3) & rgbMask.AsSingle()) | opaqueAlpha; + } + + /// + /// Replicates eight normalized luminance samples into eight RGB vectors with opaque alpha. + /// + /// The normalized luminance samples. + /// The first destination vector. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static void StoreLuminanceVectors(Vector256 source, ref Vector4 destination) + { + Vector256 rgbMask = Vector256.Create(-1, -1, -1, 0, -1, -1, -1, 0); + Vector256 opaqueAlpha = Vector256.Create(0F, 0F, 0F, 1F, 0F, 0F, 0F, 1F); + Vector256 indices0 = Vector256.Create(0, 0, 0, 0, 1, 1, 1, 1); + Vector256 indices1 = Vector256.Create(2, 2, 2, 2, 3, 3, 3, 3); + Vector256 indices2 = Vector256.Create(4, 4, 4, 4, 5, 5, 5, 5); + Vector256 indices3 = Vector256.Create(6, 6, 6, 6, 7, 7, 7, 7); + ref Vector256 destinationBase = ref Unsafe.As>(ref destination); + + destinationBase = (Vector256.ShuffleNative(source, indices0) & rgbMask.AsSingle()) | opaqueAlpha; + Unsafe.Add(ref destinationBase, 1) = (Vector256.ShuffleNative(source, indices1) & rgbMask.AsSingle()) | opaqueAlpha; + Unsafe.Add(ref destinationBase, 2) = (Vector256.ShuffleNative(source, indices2) & rgbMask.AsSingle()) | opaqueAlpha; + Unsafe.Add(ref destinationBase, 3) = (Vector256.ShuffleNative(source, indices3) & rgbMask.AsSingle()) | opaqueAlpha; + } + + /// + /// Replicates four normalized luminance samples into four RGB vectors with opaque alpha. + /// + /// The normalized luminance samples. + /// The first destination vector. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static void StoreLuminanceVectors(Vector128 source, ref Vector4 destination) + { + Vector128 rgbMask = Vector128.Create(-1, -1, -1, 0); + Vector128 opaqueAlpha = Vector128.Create(0F, 0F, 0F, 1F); + ref Vector128 destinationBase = ref Unsafe.As>(ref destination); + + // The immediate controls broadcast one source lane to RGB. The mask replaces the fourth lane with the + // implicit alpha value without extracting an individual sample from the SIMD register. + destinationBase = (Vector128_.ShuffleNative(source, 0b_00_00_00_00) & rgbMask.AsSingle()) | opaqueAlpha; + Unsafe.Add(ref destinationBase, 1) = (Vector128_.ShuffleNative(source, 0b_01_01_01_01) & rgbMask.AsSingle()) | opaqueAlpha; + Unsafe.Add(ref destinationBase, 2) = (Vector128_.ShuffleNative(source, 0b_10_10_10_10) & rgbMask.AsSingle()) | opaqueAlpha; + Unsafe.Add(ref destinationBase, 3) = (Vector128_.ShuffleNative(source, 0b_11_11_11_11) & rgbMask.AsSingle()) | opaqueAlpha; + } } } diff --git a/src/ImageSharp/PixelFormats/PixelOperations{TPixel}.cs b/src/ImageSharp/PixelFormats/PixelOperations{TPixel}.cs index 63ece41881..e5faf976cf 100644 --- a/src/ImageSharp/PixelFormats/PixelOperations{TPixel}.cs +++ b/src/ImageSharp/PixelFormats/PixelOperations{TPixel}.cs @@ -348,7 +348,6 @@ public virtual void To( /// /// Bulk operation that packs 3 separate RGB channels to . - /// The destination must have a padding of 3. /// /// A to the red values. /// A to the green values. @@ -424,6 +423,6 @@ internal static void GuardPackFromRgbPlanes(ReadOnlySpan greenChannel, Rea { Guard.IsTrue(greenChannel.Length == count, nameof(greenChannel), "Channels must be of same size!"); Guard.IsTrue(blueChannel.Length == count, nameof(blueChannel), "Channels must be of same size!"); - Guard.IsTrue(destination.Length > count + 2, nameof(destination), "'destination' must contain a padding of 3 elements!"); + Guard.IsTrue(destination.Length >= count, nameof(destination), "'destination' span should not be shorter than the source channels!"); } } diff --git a/src/ImageSharp/Primitives/Point.cs b/src/ImageSharp/Primitives/Point.cs index 99193e3bb0..8e9cadcbec 100644 --- a/src/ImageSharp/Primitives/Point.cs +++ b/src/ImageSharp/Primitives/Point.cs @@ -145,6 +145,24 @@ public Point(Size size) public static Point operator /(Point left, int right) => new(left.X / right, left.Y / right); + /// + /// Shift to the right by a amount producing . + /// + /// Shifted value of type . + /// Shifted amount of type . + /// Result of type . + public static Point operator >>(Point left, int right) + => new(left.X >> right, left.Y >> right); + + /// + /// Shift to the left by a amount producing . + /// + /// Shifted value of type . + /// Shifted amount of type . + /// Result of type . + public static Point operator <<(Point left, int right) + => new(left.X << right, left.Y << right); + /// /// Compares two objects for equality. /// @@ -280,6 +298,24 @@ public void Offset(int dx, int dy) [MethodImpl(MethodImplOptions.AggressiveInlining)] public void Offset(Point point) => this.Offset(point.X, point.Y); + /// + /// Shifts the coordinate value of this to the right with the specified amount. + /// + /// The point to shift. + /// The number of bits to shift to the right. + /// The . + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Point ShiftRight(Point point, int bitCount) => new(unchecked(point.X >> bitCount), unchecked(point.Y >> bitCount)); + + /// + /// Shifts the coordinate value of this to the left with the specified amount. + /// + /// The point to shift. + /// The number of bits to shift to the left. + /// The . + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Point ShiftLeft(Point point, int bitCount) => new(unchecked(point.X << bitCount), unchecked(point.Y << bitCount)); + /// public override readonly int GetHashCode() => HashCode.Combine(this.X, this.Y); diff --git a/src/ImageSharp/Processing/Processors/Transforms/Linear/FlipProcessor{TPixel}.cs b/src/ImageSharp/Processing/Processors/Transforms/Linear/FlipProcessor{TPixel}.cs index 86ba2f0f9a..d4999220f1 100644 --- a/src/ImageSharp/Processing/Processors/Transforms/Linear/FlipProcessor{TPixel}.cs +++ b/src/ImageSharp/Processing/Processors/Transforms/Linear/FlipProcessor{TPixel}.cs @@ -60,15 +60,23 @@ public FlipProcessor(Configuration configuration, FlipProcessor definition, Imag /// protected override void OnFrameApply(ImageFrame source) + => Apply(this.definition.FlipMode, source, this.Configuration); + + /// + /// Applies an exact axis-aligned reflection to an existing frame. + /// + /// The reflection direction. + /// The frame modified in place. + /// The configuration controlling row parallelism and scratch allocation. + internal static void Apply(FlipMode flipMode, ImageFrame source, Configuration configuration) { - switch (this.definition.FlipMode) + switch (flipMode) { - // No default needed as we have already set the pixels. case FlipMode.Vertical: - FlipX(source.PixelBuffer, this.Configuration); + FlipX(source.PixelBuffer, configuration); break; case FlipMode.Horizontal: - FlipY(source, this.Configuration); + FlipY(source, configuration); break; } } diff --git a/src/ImageSharp/Processing/Processors/Transforms/Linear/RotateProcessor{TPixel}.cs b/src/ImageSharp/Processing/Processors/Transforms/Linear/RotateProcessor{TPixel}.cs index e9d0ecf57d..ed376872a6 100644 --- a/src/ImageSharp/Processing/Processors/Transforms/Linear/RotateProcessor{TPixel}.cs +++ b/src/ImageSharp/Processing/Processors/Transforms/Linear/RotateProcessor{TPixel}.cs @@ -103,25 +103,52 @@ private bool OptimizedApply( if (MathF.Abs(degrees - 90) < Constants.Epsilon) { - Rotate90(source, destination, configuration); + ApplyQuarterTurn(RotateMode.Rotate90, source, destination, configuration); return true; } if (MathF.Abs(degrees - 180) < Constants.Epsilon) { - Rotate180(source, destination, configuration); + ApplyQuarterTurn(RotateMode.Rotate180, source, destination, configuration); return true; } if (MathF.Abs(degrees - 270) < Constants.Epsilon) { - Rotate270(source, destination, configuration); + ApplyQuarterTurn(RotateMode.Rotate270, source, destination, configuration); return true; } return false; } + /// + /// Applies an exact quarter-turn rotation between already allocated frames. + /// + /// The clockwise quarter-turn rotation. + /// The source frame. + /// The destination frame with the rotated dimensions. + /// The configuration controlling row parallelism. + internal static void ApplyQuarterTurn( + RotateMode rotation, + ImageFrame source, + ImageFrame destination, + Configuration configuration) + { + switch (rotation) + { + case RotateMode.Rotate90: + Rotate90(source, destination, configuration); + break; + case RotateMode.Rotate180: + Rotate180(source, destination, configuration); + break; + case RotateMode.Rotate270: + Rotate270(source, destination, configuration); + break; + } + } + /// /// Rotates the image 180 degrees clockwise at the centre point. /// diff --git a/tests/ImageSharp.Benchmarks/Codecs/Heif/Av1ChromaFromLumaBenchmarks.cs b/tests/ImageSharp.Benchmarks/Codecs/Heif/Av1ChromaFromLumaBenchmarks.cs new file mode 100644 index 0000000000..c1b621cef7 --- /dev/null +++ b/tests/ImageSharp.Benchmarks/Codecs/Heif/Av1ChromaFromLumaBenchmarks.cs @@ -0,0 +1,137 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using BenchmarkDotNet.Attributes; +using BenchmarkDotNet.Columns; +using BenchmarkDotNet.Configs; +using BenchmarkDotNet.Jobs; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.ChromaFromLuma; + +namespace SixLabors.ImageSharp.Benchmarks.Codecs.Heif; + +/// +/// Measures frame-wide AV1 chroma-from-luma prediction at each available intrinsic tier. +/// +[Config(typeof(Configuration))] +[MemoryDiagnoser(displayGenColumns: false)] +[GroupBenchmarksBy(BenchmarkLogicalGroupRule.ByCategory)] +[CategoriesColumn] +public class Av1ChromaFromLumaBenchmarks +{ + /// + /// The coded frame width, which is an exact multiple of the maximum CfL block side. + /// + private const int Width = 1920; + + /// + /// The coded frame height including the final padded coding-tree row for a 1080-line presentation. + /// + private const int Height = 1088; + + /// + /// The maximum CfL block side in chroma samples. + /// + private const int BlockSize = 32; + + /// + /// The fixed-stride Q3 luma residual surface reused by each benchmark block. + /// + private readonly short[] lumaQ3 = new short[BlockSize * BlockSize]; + + /// + /// The frame-wide 8-bit DC prediction surface. + /// + private readonly byte[] destination8 = new byte[Width * Height]; + + /// + /// The frame-wide 12-bit DC prediction surface. + /// + private readonly short[] destination12 = new short[Width * Height]; + + /// + /// Populates deterministic Q3 residuals and DC predictions outside the measured traversal. + /// + [GlobalSetup] + public void Setup() + { + for (int index = 0; index < this.lumaQ3.Length; index++) + { + this.lumaQ3[index] = (short)(((index * 4051) % 65521) - 32760); + } + + this.destination8.AsSpan().Fill(137); + this.destination12.AsSpan().Fill(2101); + } + + /// + /// Measures frame-wide 8-bit chroma-from-luma prediction. + /// + /// The final reconstructed sample, keeping the frame output observable. + [Benchmark] + [BenchmarkCategory("8Bit")] + public byte Predict8BitFrame() + { + for (int row = 0; row < Height; row += BlockSize) + { + for (int column = 0; column < Width; column += BlockSize) + { + Av1ChromaFromLumaPredictor.Predict(this.lumaQ3, this.destination8.AsSpan((row * Width) + column), Width, 11, BlockSize, BlockSize); + } + } + + return this.destination8[^1]; + } + + /// + /// Measures frame-wide 12-bit chroma-from-luma prediction. + /// + /// The final reconstructed sample, keeping the frame output observable. + [Benchmark] + [BenchmarkCategory("12Bit")] + public short Predict12BitFrame() + { + for (int row = 0; row < Height; row += BlockSize) + { + for (int column = 0; column < Width; column += BlockSize) + { + Av1ChromaFromLumaPredictor.Predict(this.lumaQ3, this.destination12.AsSpan((row * Width) + column), Width, 11, 12, BlockSize, BlockSize); + } + } + + return this.destination12[^1]; + } + + /// + /// Configures production-process measurements for the default, AVX2, Vector128, and scalar paths. + /// + public sealed class Configuration : ManualConfig + { + /// + /// Initializes a new instance of the class. + /// + public Configuration() + { + this.AddJob( + Job.ShortRun + .WithId("Hardware") + .AsBaseline()); + + this.AddJob( + Job.ShortRun + .WithId("Avx2") + .WithEnvironmentVariable("DOTNET_EnableAVX512F", "0")); + + this.AddJob( + Job.ShortRun + .WithId("Vector128") + .WithEnvironmentVariable("DOTNET_EnableAVX512F", "0") + .WithEnvironmentVariable("DOTNET_EnableAVX2", "0")); + + this.AddJob( + Job.ShortRun + .WithId("Scalar") + .WithEnvironmentVariable("DOTNET_EnableHWIntrinsic", "0")); + } + } +} diff --git a/tests/ImageSharp.Benchmarks/Codecs/Heif/Av1ColorConversionBenchmarks.cs b/tests/ImageSharp.Benchmarks/Codecs/Heif/Av1ColorConversionBenchmarks.cs new file mode 100644 index 0000000000..668ca4196e --- /dev/null +++ b/tests/ImageSharp.Benchmarks/Codecs/Heif/Av1ColorConversionBenchmarks.cs @@ -0,0 +1,134 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using BenchmarkDotNet.Attributes; +using SixLabors.ImageSharp.Formats.Heif.Av1; +using SixLabors.ImageSharp.Formats.Heif.Av1.Color; +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.PixelFormats; + +namespace SixLabors.ImageSharp.Benchmarks.Codecs.Heif; + +/// +/// Measures frame-wide AV1 YUV 4:2:0 color conversion in both directions. +/// +[MemoryDiagnoser(displayGenColumns: false)] +public class Av1ColorConversionBenchmarks +{ + /// + /// The benchmark frame width. + /// + private const int Width = 1920; + + /// + /// The benchmark frame height. + /// + private const int Height = 1080; + + /// + /// The source RGB image. + /// + private Image source; + + /// + /// The destination RGB image. + /// + private Image destination; + + /// + /// The reusable AV1 frame planes. + /// + private Av1FrameBuffer frameBuffer; + + /// + /// Gets or sets the encoded AV1 bit depth. + /// + [Params(8, 10, 12)] + public int BitDepth { get; set; } + + /// + /// Allocates and populates deterministic full-HD RGB and YUV frames outside the measured operations. + /// + [GlobalSetup] + public void Setup() + { + this.source = new Image(Width, Height); + this.destination = new Image(Width, Height); + for (int y = 0; y < Height; y++) + { + Span row = this.source.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(y); + for (int x = 0; x < Width; x++) + { + // The relatively prime channel steps avoid uniform rows while remaining deterministic. + row[x] = new Rgb48( + (ushort)((x * 1879) + (y * 791)), + (ushort)((x * 977) + (y * 3251)), + (ushort)((x * 613) + (y * 4987))); + } + } + + ObuSequenceHeader sequenceHeader = new() + { + MaxFrameWidth = Width, + MaxFrameHeight = Height, + ColorConfig = new ObuColorConfig + { + BitDepth = this.BitDepth switch + { + 10 => Av1BitDepth.TenBit, + 12 => Av1BitDepth.TwelveBit, + _ => Av1BitDepth.EightBit, + }, + ColorPrimaries = ObuColorPrimaries.Bt709, + TransferCharacteristics = ObuTransferCharacteristics.Bt709, + MatrixCoefficients = ObuMatrixCoefficients.Bt709, + ColorRange = false, + SubSamplingX = true, + SubSamplingY = true, + ChromaSamplePosition = ObuChromoSamplePosition.Unknown, + }, + }; + + this.frameBuffer = new Av1FrameBuffer(Configuration.Default, sequenceHeader, Av1ColorFormat.Yuv420, false); + Av1YuvConverter.ConvertFromRgb(Configuration.Default, this.source.Frames.RootFrame, this.frameBuffer); + } + + /// + /// Releases the benchmark images and reconstructed planes. + /// + [GlobalCleanup] + public void Cleanup() + { + this.frameBuffer?.Dispose(); + this.destination?.Dispose(); + this.source?.Dispose(); + } + + /// + /// Measures full-frame YUV-to-RGB conversion, including chroma reconstruction and packed-pixel conversion. + /// + /// A converted pixel that keeps the frame result observable. + [Benchmark] + public Rgb48 ConvertToRgb() + { + Av1FrameBuffer frameBuffer = this.frameBuffer; + Image destination = this.destination; + Av1YuvConverter.ConvertToRgb(Configuration.Default, frameBuffer, destination.Frames.RootFrame); + return destination.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(Height - 1)[Width - 1]; + } + + /// + /// Measures full-frame RGB-to-YUV conversion, including planar unpacking and chroma downsampling. + /// + /// An encoded luma sample that keeps the frame result observable. + [Benchmark] + public int ConvertFromRgb() + { + Image source = this.source; + Av1FrameBuffer frameBuffer = this.frameBuffer; + Av1YuvConverter.ConvertFromRgb(Configuration.Default, source.Frames.RootFrame, frameBuffer); + return this.BitDepth == 8 + ? frameBuffer.DeriveBlockPointer(Av1Plane.Y, 0, 0).DangerousGetRowSpan(Height - 1)[Width - 1] + : frameBuffer.GetHighBitDepthRowSpan(Av1Plane.Y, Height - 1, 0, 0)[Width - 1]; + } +} diff --git a/tests/ImageSharp.Benchmarks/Codecs/Heif/Av1DeblockingFilterBenchmarks.cs b/tests/ImageSharp.Benchmarks/Codecs/Heif/Av1DeblockingFilterBenchmarks.cs new file mode 100644 index 0000000000..bb066331d2 --- /dev/null +++ b/tests/ImageSharp.Benchmarks/Codecs/Heif/Av1DeblockingFilterBenchmarks.cs @@ -0,0 +1,145 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using BenchmarkDotNet.Attributes; +using BenchmarkDotNet.Columns; +using BenchmarkDotNet.Configs; +using BenchmarkDotNet.Jobs; +using SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.LoopFilter; + +namespace SixLabors.ImageSharp.Benchmarks.Codecs.Heif; + +/// +/// Measures AV1 deblocking across a padded full-HD reconstruction surface. +/// +[Config(typeof(Configuration))] +[MemoryDiagnoser(displayGenColumns: false)] +[GroupBenchmarksBy(BenchmarkLogicalGroupRule.ByCategory)] +[CategoriesColumn] +public class Av1DeblockingFilterBenchmarks +{ + /// + /// The visible coded frame width in samples. + /// + private const int Width = 1920; + + /// + /// The coded frame height including the final padded coding-tree row for a 1080-line presentation. + /// + private const int Height = 1088; + + /// + /// The border reserved around the visible reconstruction surface. + /// + private const int Padding = 16; + + /// + /// The number of samples between adjacent padded rows. + /// + private const int Stride = Width + (2 * Padding); + + /// + /// The eight-bit padded reconstruction surface. + /// + private readonly byte[] samples8 = new byte[Stride * (Height + (2 * Padding))]; + + /// + /// The twelve-bit padded reconstruction surface. + /// + private readonly ushort[] samples12 = new ushort[Stride * (Height + (2 * Padding))]; + + /// + /// Populates smooth deterministic samples that exercise enabled narrow and wide filter masks. + /// + [GlobalSetup] + public void Setup() + { + for (int row = 0; row < Height + (2 * Padding); row++) + { + for (int column = 0; column < Stride; column++) + { + int value = 96 + ((row + (2 * column)) & 15); + int offset = (row * Stride) + column; + this.samples8[offset] = (byte)value; + this.samples12[offset] = (ushort)(value << 4); + } + } + } + + /// + /// Measures both AV1 deblocking passes over an eight-bit full-HD frame. + /// + /// A filtered sample that keeps the output observable. + [Benchmark] + [BenchmarkCategory("8Bit")] + public byte Filter8BitFrame() + { + for (int y = 0; y < Height; y += 4) + { + for (int x = 8; x < Width; x += 8) + { + int q0Offset = ((Padding + y) * Stride) + Padding + x; + Av1DeblockingFilter.FilterVertical(this.samples8, q0Offset, Stride, 14, 20, 60, 3); + } + } + + for (int y = 8; y < Height; y += 8) + { + for (int x = 0; x < Width; x += 4) + { + int q0Offset = ((Padding + y) * Stride) + Padding + x; + Av1DeblockingFilter.FilterHorizontal(this.samples8, q0Offset, Stride, 14, 20, 60, 3); + } + } + + return this.samples8[(Padding * Stride) + Padding]; + } + + /// + /// Measures both AV1 deblocking passes over a twelve-bit full-HD frame. + /// + /// A filtered sample that keeps the output observable. + [Benchmark] + [BenchmarkCategory("12Bit")] + public ushort Filter12BitFrame() + { + for (int y = 0; y < Height; y += 4) + { + for (int x = 8; x < Width; x += 8) + { + int q0Offset = ((Padding + y) * Stride) + Padding + x; + Av1DeblockingFilter.FilterVertical(this.samples12, q0Offset, Stride, 14, 20, 60, 3, 12); + } + } + + for (int y = 8; y < Height; y += 8) + { + for (int x = 0; x < Width; x += 4) + { + int q0Offset = ((Padding + y) * Stride) + Padding + x; + Av1DeblockingFilter.FilterHorizontal(this.samples12, q0Offset, Stride, 14, 20, 60, 3, 12); + } + } + + return this.samples12[(Padding * Stride) + Padding]; + } + + /// + /// Configures production-process measurements for hardware and scalar filtering. + /// + public sealed class Configuration : ManualConfig + { + /// + /// Initializes a new instance of the class. + /// + public Configuration() + { + this.AddJob(Job.ShortRun.WithId("Hardware").AsBaseline()); + + this.AddJob( + Job.ShortRun + .WithId("Scalar") + .WithEnvironmentVariable("DOTNET_EnableHWIntrinsic", "0")); + } + } +} diff --git a/tests/ImageSharp.Benchmarks/Codecs/Heif/Av1FilmGrainBenchmarks.cs b/tests/ImageSharp.Benchmarks/Codecs/Heif/Av1FilmGrainBenchmarks.cs new file mode 100644 index 0000000000..3c20be940f --- /dev/null +++ b/tests/ImageSharp.Benchmarks/Codecs/Heif/Av1FilmGrainBenchmarks.cs @@ -0,0 +1,272 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using BenchmarkDotNet.Attributes; +using BenchmarkDotNet.Columns; +using BenchmarkDotNet.Configs; +using BenchmarkDotNet.Jobs; +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.FilmGrain; + +namespace SixLabors.ImageSharp.Benchmarks.Codecs.Heif; + +/// +/// Measures AV1 film-grain application across full-HD-equivalent 4:2:0 component planes. +/// +[Config(typeof(Configuration))] +[MemoryDiagnoser(displayGenColumns: false)] +[GroupBenchmarksBy(BenchmarkLogicalGroupRule.ByCategory)] +[CategoriesColumn] +public class Av1FilmGrainBenchmarks +{ + /// + /// The aligned full-HD luma width. + /// + private const int Width = 1920; + + /// + /// The aligned full-HD luma height. + /// + private const int Height = 1088; + + /// + /// The luma width and height of one selected grain block. + /// + private const int BlockSize = 32; + + /// + /// The 4:2:0 chroma-plane width. + /// + private const int ChromaWidth = Width / 2; + + /// + /// The 4:2:0 chroma-plane height. + /// + private const int ChromaHeight = Height / 2; + + /// + /// The 4:2:0 chroma width and height of one selected grain block. + /// + private const int ChromaBlockSize = BlockSize / 2; + + /// + /// The deterministic eight-bit luma plane. + /// + private readonly byte[] luma8 = new byte[Width * Height]; + + /// + /// The deterministic eight-bit first chroma plane. + /// + private readonly byte[] cb8 = new byte[ChromaWidth * ChromaHeight]; + + /// + /// The deterministic eight-bit second chroma plane. + /// + private readonly byte[] cr8 = new byte[ChromaWidth * ChromaHeight]; + + /// + /// The deterministic twelve-bit luma plane. + /// + private readonly ushort[] luma12 = new ushort[Width * Height]; + + /// + /// The deterministic twelve-bit first chroma plane. + /// + private readonly ushort[] cb12 = new ushort[ChromaWidth * ChromaHeight]; + + /// + /// The deterministic twelve-bit second chroma plane. + /// + private readonly ushort[] cr12 = new ushort[ChromaWidth * ChromaHeight]; + + /// + /// The expanded luma scaling function. + /// + private readonly int[] scalingY = new int[256]; + + /// + /// The expanded first chroma scaling function. + /// + private readonly int[] scalingCb = new int[256]; + + /// + /// The expanded second chroma scaling function. + /// + private readonly int[] scalingCr = new int[256]; + + /// + /// The selected luma grain block. + /// + private readonly int[] lumaGrain = new int[BlockSize * BlockSize]; + + /// + /// The selected first chroma grain block. + /// + private readonly int[] cbGrain = new int[ChromaBlockSize * ChromaBlockSize]; + + /// + /// The selected second chroma grain block. + /// + private readonly int[] crGrain = new int[ChromaBlockSize * ChromaBlockSize]; + + /// + /// The active grain parameters shared by both measured sample precisions. + /// + private readonly ObuFilmGrainParameters parameters = new() + { + NumYPoints = 2, + ChromaScalingFromLuma = true, + GrainScalingMinus8 = 3 + }; + + /// + /// Populates deterministic source planes and grain blocks outside the measured traversal. + /// + [GlobalSetup] + public void Setup() + { + for (int index = 0; index < this.luma8.Length; index++) + { + int value = ((index * 37) + 113) & byte.MaxValue; + this.luma8[index] = (byte)value; + this.luma12[index] = (ushort)(value << 4); + } + + for (int index = 0; index < this.cb8.Length; index++) + { + int cb = ((index * 53) + 97) & byte.MaxValue; + int cr = ((index * 71) + 41) & byte.MaxValue; + this.cb8[index] = (byte)cb; + this.cr8[index] = (byte)cr; + this.cb12[index] = (ushort)(cb << 4); + this.cr12[index] = (ushort)(cr << 4); + } + + for (int index = 0; index < this.lumaGrain.Length; index++) + { + this.lumaGrain[index] = ((index * 29) & byte.MaxValue) - 128; + } + + for (int index = 0; index < this.cbGrain.Length; index++) + { + this.cbGrain[index] = ((index * 43) & byte.MaxValue) - 128; + this.crGrain[index] = ((index * 61) & byte.MaxValue) - 128; + } + + // A zero scaling function keeps every invocation's source planes stable. The measured code still performs the + // production lookup, interpolation, grain multiplication, clipping, and native sample packing for every lane. + this.scalingY.AsSpan().Clear(); + this.scalingCb.AsSpan().Clear(); + this.scalingCr.AsSpan().Clear(); + } + + /// + /// Applies eight-bit grain blocks across full-HD-equivalent 4:2:0 planes. + /// + /// The final luma sample, keeping the output observable. + [Benchmark] + [BenchmarkCategory("8Bit")] + public byte Apply8Bit() + { + for (int y = 0; y < Height; y += BlockSize) + { + for (int x = 0; x < Width; x += BlockSize) + { + int lumaOffset = (y * Width) + x; + int chromaOffset = ((y / 2) * ChromaWidth) + (x / 2); + + Av1FilmGrainNoise.Apply( + this.parameters, + this.scalingY, + this.scalingCb, + this.scalingCr, + this.luma8.AsSpan(lumaOffset), + this.cb8.AsSpan(chromaOffset), + this.cr8.AsSpan(chromaOffset), + Width, + ChromaWidth, + this.lumaGrain, + this.cbGrain, + this.crGrain, + BlockSize, + ChromaBlockSize, + BlockSize / 2, + BlockSize / 2, + 8, + 1, + 1, + isMonochrome: false, + isIdentityMatrix: false); + } + } + + return this.luma8[^1]; + } + + /// + /// Applies twelve-bit grain blocks across full-HD-equivalent 4:2:0 planes. + /// + /// The final luma sample, keeping the output observable. + [Benchmark] + [BenchmarkCategory("12Bit")] + public ushort Apply12Bit() + { + for (int y = 0; y < Height; y += BlockSize) + { + for (int x = 0; x < Width; x += BlockSize) + { + int lumaOffset = (y * Width) + x; + int chromaOffset = ((y / 2) * ChromaWidth) + (x / 2); + + Av1FilmGrainNoise.Apply( + this.parameters, + this.scalingY, + this.scalingCb, + this.scalingCr, + this.luma12.AsSpan(lumaOffset), + this.cb12.AsSpan(chromaOffset), + this.cr12.AsSpan(chromaOffset), + Width, + ChromaWidth, + this.lumaGrain, + this.cbGrain, + this.crGrain, + BlockSize, + ChromaBlockSize, + BlockSize / 2, + BlockSize / 2, + 12, + 1, + 1, + isMonochrome: false, + isIdentityMatrix: false); + } + } + + return this.luma12[^1]; + } + + /// + /// Configures production-process measurements for hardware, no-AVX, and scalar grain application. + /// + public sealed class Configuration : ManualConfig + { + /// + /// Initializes a new instance of the class. + /// + public Configuration() + { + this.AddJob(Job.ShortRun.WithId("Hardware").AsBaseline()); + + this.AddJob( + Job.ShortRun + .WithId("NoAvx") + .WithEnvironmentVariable("DOTNET_EnableAVX", "0")); + + this.AddJob( + Job.ShortRun + .WithId("Scalar") + .WithEnvironmentVariable("DOTNET_EnableHWIntrinsic", "0")); + } + } +} diff --git a/tests/ImageSharp.Benchmarks/Codecs/Heif/Av1LoopRestorationBenchmarks.cs b/tests/ImageSharp.Benchmarks/Codecs/Heif/Av1LoopRestorationBenchmarks.cs new file mode 100644 index 0000000000..a2c335cbb7 --- /dev/null +++ b/tests/ImageSharp.Benchmarks/Codecs/Heif/Av1LoopRestorationBenchmarks.cs @@ -0,0 +1,169 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using BenchmarkDotNet.Attributes; +using BenchmarkDotNet.Columns; +using BenchmarkDotNet.Configs; +using BenchmarkDotNet.Jobs; +using SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.LoopRestoration; + +namespace SixLabors.ImageSharp.Benchmarks.Codecs.Heif; + +/// +/// Measures normative AV1 self-guided restoration across a full-HD-equivalent luma workload. +/// +[Config(typeof(Configuration))] +[MemoryDiagnoser(displayGenColumns: false)] +[GroupBenchmarksBy(BenchmarkLogicalGroupRule.ByCategory)] +[CategoriesColumn] +public class Av1LoopRestorationBenchmarks +{ + /// + /// The width of one normative self-guided processing unit. + /// + private const int Width = 64; + + /// + /// The height of one normative self-guided processing unit. + /// + private const int Height = 64; + + /// + /// The three source samples required on each side of a processing unit. + /// + private const int Border = 3; + + /// + /// The number of processing units covering a 1920 by 1080 luma plane. + /// + private const int ProcessingUnitCount = 30 * 17; + + /// + /// The bordered source-row stride. + /// + private const int SourceStride = Width + (Border * 2); + + /// + /// The self-guided parameter set activating both radius-two and radius-one filtering. + /// + private const int ParameterSetIndex = 0; + + /// + /// The deterministic bordered eight-bit source block. + /// + private readonly ushort[] source8 = new ushort[SourceStride * (Height + (Border * 2))]; + + /// + /// The deterministic bordered twelve-bit source block. + /// + private readonly ushort[] source12 = new ushort[SourceStride * (Height + (Border * 2))]; + + /// + /// The restored processing-unit destination. + /// + private readonly ushort[] destination = new ushort[Width * Height]; + + /// + /// The caller-owned self-guided work storage. + /// + private readonly int[] scratch = new int[Av1SelfGuidedFilter.GetScratchLength(Width, Height)]; + + /// + /// Gets the two transmitted projection coefficients used by the measured parameter set. + /// + private static ReadOnlySpan ProjectionCoefficients => [31, -7]; + + /// + /// Populates deterministic bordered source blocks outside the measured traversal. + /// + [GlobalSetup] + public void Setup() + { + for (int row = 0; row < Height + (Border * 2); row++) + { + for (int column = 0; column < SourceStride; column++) + { + int sample = ((row * 4051) + (column * 7919) + 127) & byte.MaxValue; + int offset = (row * SourceStride) + column; + this.source8[offset] = (ushort)sample; + this.source12[offset] = (ushort)(sample << 4); + } + } + } + + /// + /// Measures eight-bit self-guided restoration for a full-HD-equivalent luma plane. + /// + /// The final restored sample, keeping the output observable. + [Benchmark] + [BenchmarkCategory("8Bit")] + public ushort Restore8BitPlane() + { + for (int unit = 0; unit < ProcessingUnitCount; unit++) + { + Av1SelfGuidedFilter.FilterBlock( + this.source8, + SourceStride, + this.destination, + Width, + Width, + Height, + 8, + ParameterSetIndex, + ProjectionCoefficients, + this.scratch); + } + + return this.destination[^1]; + } + + /// + /// Measures twelve-bit self-guided restoration for a full-HD-equivalent luma plane. + /// + /// The final restored sample, keeping the output observable. + [Benchmark] + [BenchmarkCategory("12Bit")] + public ushort Restore12BitPlane() + { + for (int unit = 0; unit < ProcessingUnitCount; unit++) + { + Av1SelfGuidedFilter.FilterBlock( + this.source12, + SourceStride, + this.destination, + Width, + Width, + Height, + 12, + ParameterSetIndex, + ProjectionCoefficients, + this.scratch); + } + + return this.destination[^1]; + } + + /// + /// Configures production-process measurements for hardware, 128-bit, and scalar filtering. + /// + public sealed class Configuration : ManualConfig + { + /// + /// Initializes a new instance of the class. + /// + public Configuration() + { + this.AddJob(Job.ShortRun.WithId("Hardware").AsBaseline()); + + this.AddJob( + Job.ShortRun + .WithId("Vector128") + .WithEnvironmentVariable("DOTNET_EnableAVX", "0")); + + this.AddJob( + Job.ShortRun + .WithId("Scalar") + .WithEnvironmentVariable("DOTNET_EnableHWIntrinsic", "0")); + } + } +} diff --git a/tests/ImageSharp.Benchmarks/Codecs/Heif/Av1LosslessTransformBenchmarks.cs b/tests/ImageSharp.Benchmarks/Codecs/Heif/Av1LosslessTransformBenchmarks.cs new file mode 100644 index 0000000000..c2103a3018 --- /dev/null +++ b/tests/ImageSharp.Benchmarks/Codecs/Heif/Av1LosslessTransformBenchmarks.cs @@ -0,0 +1,138 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using BenchmarkDotNet.Attributes; +using BenchmarkDotNet.Columns; +using BenchmarkDotNet.Configs; +using BenchmarkDotNet.Jobs; +using SixLabors.ImageSharp.Formats.Heif.Av1; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +namespace SixLabors.ImageSharp.Benchmarks.Codecs.Heif; + +/// +/// Measures frame-wide AV1 lossless inverse-transform reconstruction. +/// +[Config(typeof(Configuration))] +[MemoryDiagnoser(displayGenColumns: false)] +[GroupBenchmarksBy(BenchmarkLogicalGroupRule.ByCategory)] +[CategoriesColumn] +public class Av1LosslessTransformBenchmarks +{ + /// + /// The coded frame width in samples. + /// + private const int Width = 1920; + + /// + /// The coded frame height including the final padded coding-tree row for a 1080-line presentation. + /// + private const int Height = 1088; + + /// + /// The lossless AV1 transform side in samples. + /// + private const int TransformSize = 4; + + /// + /// The dense dequantized coefficient block used by every transform. + /// + private readonly int[] coefficients = + [ + 320, -192, 64, -448, + 128, 256, -320, 96, + -224, 160, 384, -128, + 448, -64, -256, 192 + ]; + + /// + /// The caller-owned transform workspace reused across the frame. + /// + private readonly int[] workspace = new int[Av1TransformWorkspace.MaximumLength]; + + /// + /// The frame-wide eight-bit reconstruction surface. + /// + private readonly byte[] destination8 = new byte[Width * Height]; + + /// + /// The frame-wide twelve-bit reconstruction surface. + /// + private readonly short[] destination12 = new short[Width * Height]; + + /// + /// Measures dense eight-bit lossless reconstruction across a padded 1920-by-1088 frame. + /// + /// The final reconstructed sample, keeping the frame output observable. + [Benchmark] + [BenchmarkCategory("8Bit")] + public byte Reconstruct8BitFrame() + { + for (int row = 0; row < Height; row += TransformSize) + { + for (int column = 0; column < Width; column += TransformSize) + { + Av1InverseTransformer.Reconstruct8Bit( + this.coefficients, + this.destination8.AsSpan((row * Width) + column), + Width, + Av1TransformSize.Size4x4, + Av1TransformType.DctDct, + 0, + this.coefficients.Length, + true, + this.workspace); + } + } + + return this.destination8[^1]; + } + + /// + /// Measures dense twelve-bit lossless reconstruction across a padded 1920-by-1088 frame. + /// + /// The final reconstructed sample, keeping the frame output observable. + [Benchmark] + [BenchmarkCategory("12Bit")] + public short Reconstruct12BitFrame() + { + for (int row = 0; row < Height; row += TransformSize) + { + for (int column = 0; column < Width; column += TransformSize) + { + Av1InverseTransformer.ReconstructHighBitDepth( + this.coefficients, + this.destination12.AsSpan((row * Width) + column), + Width, + Av1TransformSize.Size4x4, + Av1TransformType.DctDct, + 0, + this.coefficients.Length, + true, + Av1BitDepth.TwelveBit, + this.workspace); + } + } + + return this.destination12[^1]; + } + + /// + /// Configures production-process measurements for hardware and scalar reconstruction. + /// + public sealed class Configuration : ManualConfig + { + /// + /// Initializes a new instance of the class. + /// + public Configuration() + { + this.AddJob(Job.ShortRun.WithId("Hardware").AsBaseline()); + + this.AddJob( + Job.ShortRun + .WithId("Scalar") + .WithEnvironmentVariable("DOTNET_EnableHWIntrinsic", "0")); + } + } +} diff --git a/tests/ImageSharp.Benchmarks/Codecs/Heif/Av1PalettePredictionBenchmarks.cs b/tests/ImageSharp.Benchmarks/Codecs/Heif/Av1PalettePredictionBenchmarks.cs new file mode 100644 index 0000000000..52e205d782 --- /dev/null +++ b/tests/ImageSharp.Benchmarks/Codecs/Heif/Av1PalettePredictionBenchmarks.cs @@ -0,0 +1,164 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using BenchmarkDotNet.Attributes; +using BenchmarkDotNet.Columns; +using BenchmarkDotNet.Configs; +using BenchmarkDotNet.Jobs; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; +using SixLabors.ImageSharp.Memory; + +namespace SixLabors.ImageSharp.Benchmarks.Codecs.Heif; + +/// +/// Measures frame-wide AV1 palette reconstruction at each available intrinsic tier. +/// +[Config(typeof(Configuration))] +[MemoryDiagnoser(displayGenColumns: false)] +[GroupBenchmarksBy(BenchmarkLogicalGroupRule.ByCategory)] +[CategoriesColumn] +public class Av1PalettePredictionBenchmarks +{ + /// + /// The coded frame width, which is an exact multiple of the maximum palette block side. + /// + private const int Width = 1920; + + /// + /// The coded frame height including the final padded coding-tree row for a 1080-line presentation. + /// + private const int Height = 1088; + + /// + /// The maximum AV1 palette block side in samples. + /// + private const int BlockSize = 64; + + /// + /// The eight-entry 8-bit palette reused by each benchmark block. + /// + private readonly ushort[] palette8 = [3, 37, 71, 109, 143, 181, 217, 251]; + + /// + /// The eight-entry 12-bit palette reused by each benchmark block. + /// + private readonly ushort[] palette12 = [17, 509, 1001, 1493, 1985, 2477, 2969, 4095]; + + /// + /// The decoded color-index map for one maximum-size palette block. + /// + private Buffer2D colorIndexMapBuffer; + + /// + /// The row-addressable view of . + /// + private Buffer2DRegion colorIndexMap; + + /// + /// The frame-wide 8-bit reconstruction surface. + /// + private readonly byte[] destination8 = new byte[Width * Height]; + + /// + /// The frame-wide 12-bit reconstruction surface. + /// + private readonly short[] destination12 = new short[Width * Height]; + + /// + /// Populates a deterministic, spatially varying color-index map outside the measured traversal. + /// + [GlobalSetup] + public void Setup() + { + this.colorIndexMapBuffer = SixLabors.ImageSharp.Configuration.Default.MemoryAllocator.Allocate2D(BlockSize, BlockSize); + this.colorIndexMap = new Buffer2DRegion(this.colorIndexMapBuffer); + for (int row = 0; row < BlockSize; row++) + { + Span colorIndexRow = this.colorIndexMap.DangerousGetRowSpan(row); + for (int column = 0; column < BlockSize; column++) + { + colorIndexRow[column] = (byte)(((row * 5) + (column * 3)) & 7); + } + } + } + + /// + /// Releases the row-addressable color-index map after the benchmark run. + /// + [GlobalCleanup] + public void Cleanup() => this.colorIndexMapBuffer?.Dispose(); + + /// + /// Measures frame-wide 8-bit palette reconstruction. + /// + /// The final reconstructed sample, keeping the frame output observable. + [Benchmark] + [BenchmarkCategory("8Bit")] + public byte Predict8BitFrame() + { + for (int row = 0; row < Height; row += BlockSize) + { + for (int column = 0; column < Width; column += BlockSize) + { + Av1PalettePredictor.Predict(this.palette8, this.colorIndexMap, this.destination8.AsSpan((row * Width) + column), Width, BlockSize, BlockSize); + } + } + + return this.destination8[^1]; + } + + /// + /// Measures frame-wide 12-bit palette reconstruction. + /// + /// The final reconstructed sample, keeping the frame output observable. + [Benchmark] + [BenchmarkCategory("12Bit")] + public short Predict12BitFrame() + { + for (int row = 0; row < Height; row += BlockSize) + { + for (int column = 0; column < Width; column += BlockSize) + { + Av1PalettePredictor.Predict(this.palette12, this.colorIndexMap, this.destination12.AsSpan((row * Width) + column), Width, BlockSize, BlockSize); + } + } + + return this.destination12[^1]; + } + + /// + /// Configures production-process measurements for hardware, forced Vector512, AVX2, Vector128, and scalar paths. + /// + public sealed class Configuration : ManualConfig + { + /// + /// Initializes a new instance of the class. + /// + public Configuration() + { + this.AddJob(Job.ShortRun.WithId("Hardware").AsBaseline()); + + this.AddJob( + Job.ShortRun + .WithId("Vector512") + .WithEnvironmentVariable("DOTNET_PreferredVectorBitWidth", "512") + .WithEnvironmentVariable("COMPlus_PreferredVectorBitWidth", "512")); + + this.AddJob( + Job.ShortRun + .WithId("Avx2") + .WithEnvironmentVariable("DOTNET_EnableAVX512F", "0")); + + this.AddJob( + Job.ShortRun + .WithId("Vector128") + .WithEnvironmentVariable("DOTNET_EnableAVX512F", "0") + .WithEnvironmentVariable("DOTNET_EnableAVX2", "0")); + + this.AddJob( + Job.ShortRun + .WithId("Scalar") + .WithEnvironmentVariable("DOTNET_EnableHWIntrinsic", "0")); + } + } +} diff --git a/tests/ImageSharp.Benchmarks/Codecs/Heif/Av1SequenceEncoderBenchmarks.cs b/tests/ImageSharp.Benchmarks/Codecs/Heif/Av1SequenceEncoderBenchmarks.cs new file mode 100644 index 0000000000..619e3b30cd --- /dev/null +++ b/tests/ImageSharp.Benchmarks/Codecs/Heif/Av1SequenceEncoderBenchmarks.cs @@ -0,0 +1,201 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Numerics; +using BenchmarkDotNet.Attributes; +using SixLabors.ImageSharp.Formats.Heif.Av1; +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline; +using SixLabors.ImageSharp.Formats.Heif.Components; +using SixLabors.ImageSharp.Memory; +using SixLabors.ImageSharp.PixelFormats; +using SixLabors.ImageSharp.Processing; +using SixLabors.ImageSharp.Tests; + +namespace SixLabors.ImageSharp.Benchmarks.Codecs.Heif; + +/// +/// Measures encoding a photographic sequence with fractional motion at the interpolation-search effort boundaries. +/// +[MemoryDiagnoser] +public class Av1SequenceEncoderBenchmarks +{ + /// + /// The number of displayed pictures in each independently encoded sequence. + /// + private const int FrameCount = 3; + + /// + /// The native AV1 quantizer index corresponding to libaom's public constant-quality level 30. + /// + private const int QIndex = 120; + + /// + /// The fixed native speed baseline, independent of ImageSharp's effort scale. + /// + private const int NativeCpuUsed = 6; + + /// + /// The native public quantizer corresponding to , also used for both rate-control bounds. + /// + private const int NativeQuality = 30; + + private Image sequence; + private Configuration configuration; + private ObuColorConfig colorConfig; + private MemoryStream output; + private string outputDirectory; + + /// + /// Gets or sets the square frame dimension. + /// + [Params(256, 512)] + public int Dimension { get; set; } + + /// + /// Gets or sets the effort controlling fixed, common switchable, or independently switchable filters. + /// + [Params(7, 8, 9)] + public int Effort { get; set; } + + /// + /// Prepares identical photographic RGB frames and a planar source file for checking reconstructed output quality. + /// + [GlobalSetup] + public void Setup() + { + this.configuration = Configuration.Default.Clone(); + this.configuration.MaxDegreeOfParallelism = 1; + this.colorConfig = new ObuColorConfig + { + BitDepth = Av1BitDepth.EightBit, + IsColorDescriptionPresent = true, + ColorPrimaries = ObuColorPrimaries.Bt601, + TransferCharacteristics = ObuTransferCharacteristics.Bt601, + MatrixCoefficients = ObuMatrixCoefficients.Bt601, + ColorRange = true, + SubSamplingX = true, + SubSamplingY = true, + ChromaSamplePosition = ObuChromoSamplePosition.Unknown + }; + + this.output = new MemoryStream(); + this.outputDirectory = TestEnvironment.CreateOutputDirectory("Heif", "Av1", nameof(Av1SequenceEncoderBenchmarks)); + string inputPath = Path.Combine(TestEnvironment.InputImagesDirectoryFullPath, TestImages.Png.Bike); + using Image photograph = Image.Load(inputPath); + + // Leave a source margin for the half-pixel translations. Resampling is setup work, not encoder time; + // every invocation consumes the same three images rather than repeatedly translating a previous result. + photograph.Mutate(context => context.Resize(new ResizeOptions + { + Size = new Size(this.Dimension + FrameCount, this.Dimension + FrameCount), + Mode = ResizeMode.Crop + })); + + Rectangle sourceBounds = new(0, 0, photograph.Width, photograph.Height); + Size targetSize = new(this.Dimension, this.Dimension); + this.sequence = photograph.Clone(context => context.Crop(new Rectangle(Point.Empty, targetSize))); + for (int frameIndex = 1; frameIndex < FrameCount; frameIndex++) + { + Matrix3x2 translation = Matrix3x2.CreateTranslation(-0.5F * frameIndex, -0.5F * frameIndex); + using Image translated = photograph.Clone(context => + context.Transform(sourceBounds, translation, targetSize, KnownResamplers.Bicubic)); + + this.sequence.Frames.AddFrame(translated.Frames.RootFrame); + } + + // Export the production-converted source planes only for checking reconstructed output quality. + // Neither timed encoder reads this file: both convert the original RGB frames during each operation. + using Av1EncoderFrameBuffer planar = new(this.configuration, this.Dimension, this.Dimension, 8, Av1ColorFormat.Yuv420, 0, 0); + using FileStream raw = File.Create(Path.Combine(this.outputDirectory, $"bike-{this.Dimension}-3frames.source.yuv")); + foreach (ImageFrame frame in this.sequence.Frames) + { + Av1FrameEncoder.PrepareSource(this.configuration, frame, planar.Frame, this.colorConfig); + for (int planeIndex = 0; planeIndex < this.colorConfig.PlaneCount; planeIndex++) + { + Buffer2DRegion plane = planar.Frame.View.GetPlane((Av1Plane)planeIndex); + for (int y = 0; y < plane.Height; y++) + { + raw.Write(plane.DangerousGetRowSpan(y)); + } + } + } + } + + /// + /// Encodes one key picture and two dependent pictures, returning the complete OBU payload length. + /// + /// The encoded sequence length. + [Benchmark] + public long ImageSharp() + { + this.output.SetLength(0); + using Av1FrameEncoder.SequenceEncoder encoder = Av1FrameEncoder.CreateColorSequenceEncoder( + this.configuration, this.Dimension, this.Dimension, this.colorConfig, QIndex, this.Effort); + + // One operation owns the real sequence lifetime: allocation, conversion, key/inter coding, and disposal. + // The caller's destination is reused, excluding filesystem and MemoryStream growth from steady-state timing. + encoder.EncodeKeyFrame(this.sequence.Frames.RootFrame, this.output); + for (int frameIndex = 1; frameIndex < FrameCount; frameIndex++) + { + encoder.EncodeInterFrame(this.sequence.Frames[frameIndex], this.output); + } + + return this.output.Length; + } + + /// + /// Encodes the same RGB sequence with current-main libaom, including conversion, allocation, output, and disposal. + /// + /// The encoded sequence length. + [Benchmark(Baseline = true)] + public long Libaom() + { + // cpu-used is a separate speed scale, not an ImageSharp effort mapping. Keep the reference at + // good-quality speed six while comparing the three managed interpolation-search boundaries. + this.output.SetLength(0); + using LibaomBenchmarkEncoder encoder = LibaomBenchmarkEncoder.Open(this.Dimension, this.Dimension, NativeQuality, NativeCpuUsed); + using Av1EncoderFrameBuffer planar = new(this.configuration, this.Dimension, this.Dimension, 8, Av1ColorFormat.Yuv420, 0, 0); + using Av1FrameEncoder.Av1EncoderConversionWorkspace conversion = new(this.configuration, this.Dimension, this.colorConfig, false, false); + Rectangle bounds = new(0, 0, this.Dimension, this.Dimension); + for (int frameIndex = 0; frameIndex < FrameCount; frameIndex++) + { + // Conversion belongs inside both measured paths. Reuse the same row workspace and SIMD converter + // as the managed sequence encoder, writing directly into the planes passed to native libaom. + conversion.Convert.PlanarView, byte, HeifByteSampleConverter>( + this.configuration, this.sequence.Frames[frameIndex], bounds, planar.Frame.View); + + encoder.Encode(planar.Frame, frameIndex, this.output); + } + + encoder.Finish(this.output); + return this.output.Length; + } + + /// + /// Retains the measured managed encoder output and releases the input images and destination stream. + /// + [GlobalCleanup(Target = nameof(ImageSharp))] + public void CleanupImageSharp() => this.Cleanup($"bike-{this.Dimension}-q{QIndex}-effort{this.Effort}.obu"); + + /// + /// Retains the measured reference encoder output and releases the input images and destination stream. + /// + [GlobalCleanup(Target = nameof(Libaom))] + public void CleanupLibaom() => this.Cleanup($"bike-{this.Dimension}-q{QIndex}-libaom-cpu{NativeCpuUsed}.obu"); + + /// + /// Writes the measured payload without another encoding pass and releases the shared benchmark resources. + /// + /// The codec-specific payload file name. + private void Cleanup(string outputName) + { + // Output validation and quality measurement use the actual measured payload, with no encode or decode + // hidden inside the timed operation and no file-sized ToArray copy. + using FileStream encoded = File.Create(Path.Combine(this.outputDirectory, outputName)); + this.output.Position = 0; + this.output.CopyTo(encoded); + this.output.Dispose(); + this.sequence.Dispose(); + } +} diff --git a/tests/ImageSharp.Benchmarks/Codecs/Heif/Av1SuperResolutionBenchmarks.cs b/tests/ImageSharp.Benchmarks/Codecs/Heif/Av1SuperResolutionBenchmarks.cs new file mode 100644 index 0000000000..84c6181b0e --- /dev/null +++ b/tests/ImageSharp.Benchmarks/Codecs/Heif/Av1SuperResolutionBenchmarks.cs @@ -0,0 +1,145 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using BenchmarkDotNet.Attributes; +using BenchmarkDotNet.Columns; +using BenchmarkDotNet.Configs; +using BenchmarkDotNet.Jobs; +using SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.SuperResolution; + +namespace SixLabors.ImageSharp.Benchmarks.Codecs.Heif; + +/// +/// Measures normative AV1 super-resolution filtering across a full-HD luma plane. +/// +[Config(typeof(Configuration))] +[MemoryDiagnoser(displayGenColumns: false)] +[GroupBenchmarksBy(BenchmarkLogicalGroupRule.ByCategory)] +[CategoriesColumn] +public class Av1SuperResolutionBenchmarks +{ + /// + /// The displayed luma width after normative upscaling. + /// + private const int UpscaledWidth = 1920; + + /// + /// The coded luma width produced by a fixed super-resolution denominator of twelve. + /// + private const int CodedWidth = 1280; + + /// + /// The number of visible rows in the measured luma plane. + /// + private const int Height = 1080; + + /// + /// The fixed-point source-position increment for the measured scale ratio. + /// + private readonly int step = Av1SuperResolutionFilter.GetConvolveStep(CodedWidth, UpscaledWidth); + + /// + /// The initial fixed-point source position for the measured scale ratio. + /// + private readonly int initialSubpixel; + + /// + /// The replicated-edge eight-bit source row. + /// + private readonly byte[] source8 = new byte[CodedWidth + (Av1SuperResolutionFilter.SourceBorder * 2)]; + + /// + /// The filtered eight-bit output row. + /// + private readonly byte[] destination8 = new byte[UpscaledWidth]; + + /// + /// The replicated-edge twelve-bit source row. + /// + private readonly ushort[] source12 = new ushort[CodedWidth + (Av1SuperResolutionFilter.SourceBorder * 2)]; + + /// + /// The filtered twelve-bit output row. + /// + private readonly ushort[] destination12 = new ushort[UpscaledWidth]; + + /// + /// Initializes a new instance of the class. + /// + public Av1SuperResolutionBenchmarks() + { + this.initialSubpixel = Av1SuperResolutionFilter.GetInitialSubpixel(CodedWidth, UpscaledWidth, this.step); + } + + /// + /// Populates deterministic source samples and their replicated frame edges outside the measured traversal. + /// + [GlobalSetup] + public void Setup() + { + Span reconstructed8 = this.source8.AsSpan(Av1SuperResolutionFilter.SourceBorder, CodedWidth); + Span reconstructed12 = this.source12.AsSpan(Av1SuperResolutionFilter.SourceBorder, CodedWidth); + for (int column = 0; column < CodedWidth; column++) + { + int sample = ((column * 4051) + 127) & byte.MaxValue; + reconstructed8[column] = (byte)sample; + reconstructed12[column] = (ushort)(sample << 4); + } + + this.source8.AsSpan(0, Av1SuperResolutionFilter.SourceBorder).Fill(reconstructed8[0]); + this.source8.AsSpan(Av1SuperResolutionFilter.SourceBorder + CodedWidth).Fill(reconstructed8[^1]); + this.source12.AsSpan(0, Av1SuperResolutionFilter.SourceBorder).Fill(reconstructed12[0]); + this.source12.AsSpan(Av1SuperResolutionFilter.SourceBorder + CodedWidth).Fill(reconstructed12[^1]); + } + + /// + /// Measures normative eight-bit filtering for a full-HD luma plane. + /// + /// The final filtered sample, keeping the output observable. + [Benchmark] + [BenchmarkCategory("8Bit")] + public byte Filter8BitPlane() + { + for (int row = 0; row < Height; row++) + { + Av1SuperResolutionFilter.UpscaleRow(this.source8, this.destination8, this.step, this.initialSubpixel); + } + + return this.destination8[^1]; + } + + /// + /// Measures normative twelve-bit filtering for a full-HD luma plane. + /// + /// The final filtered sample, keeping the output observable. + [Benchmark] + [BenchmarkCategory("12Bit")] + public ushort Filter12BitPlane() + { + for (int row = 0; row < Height; row++) + { + Av1SuperResolutionFilter.UpscaleRow(this.source12, this.destination12, this.step, this.initialSubpixel, 12); + } + + return this.destination12[^1]; + } + + /// + /// Configures production-process measurements for hardware and scalar filtering. + /// + public sealed class Configuration : ManualConfig + { + /// + /// Initializes a new instance of the class. + /// + public Configuration() + { + this.AddJob(Job.ShortRun.WithId("Hardware").AsBaseline()); + + this.AddJob( + Job.ShortRun + .WithId("Scalar") + .WithEnvironmentVariable("DOTNET_EnableHWIntrinsic", "0")); + } + } +} diff --git a/tests/ImageSharp.Benchmarks/Codecs/Heif/Av1TransformBenchmarks.cs b/tests/ImageSharp.Benchmarks/Codecs/Heif/Av1TransformBenchmarks.cs new file mode 100644 index 0000000000..9649282c13 --- /dev/null +++ b/tests/ImageSharp.Benchmarks/Codecs/Heif/Av1TransformBenchmarks.cs @@ -0,0 +1,147 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using BenchmarkDotNet.Attributes; +using BenchmarkDotNet.Columns; +using BenchmarkDotNet.Configs; +using BenchmarkDotNet.Jobs; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +namespace SixLabors.ImageSharp.Benchmarks.Codecs.Heif; + +/// +/// Measures the production AV1 forward-transform pipeline at the runtime's selected vector width. +/// +[Config(typeof(Configuration))] +[MemoryDiagnoser(displayGenColumns: false)] +[GroupBenchmarksBy(BenchmarkLogicalGroupRule.ByCategory)] +[CategoriesColumn] +public class Av1TransformBenchmarks +{ + private readonly short[] spatial = new short[64 * 64]; + private readonly int[] coefficients = new int[32 * 32]; + private readonly int[] workspace = new int[Av1TransformWorkspace.MaximumLength]; + + /// + /// Gets or sets the coded sample bit depth used by the transform. + /// + [Params(8, 12)] + public int BitDepth { get; set; } + + /// + /// Initializes deterministic residual data outside the measured operations. + /// + [GlobalSetup] + public void Setup() + { + for (int index = 0; index < this.spatial.Length; index++) + { + this.spatial[index] = (short)(((index * 73) % 511) - 255); + } + } + + /// + /// Measures an eight-by-eight forward DCT block. + /// + /// The last coefficient written by the transform. + [Benchmark] + [BenchmarkCategory("Forward8x8")] + public int Forward8x8() + { + Av1ForwardTransformer.Transform2d( + this.spatial, + this.coefficients, + 8, + Av1TransformType.DctDct, + Av1TransformSize.Size8x8, + this.BitDepth, + this.workspace); + + return this.coefficients[63]; + } + + /// + /// Measures a thirty-two-by-thirty-two forward DCT block. + /// + /// The last coefficient written by the transform. + [Benchmark] + [BenchmarkCategory("Forward32x32")] + public int Forward32x32() + { + Av1ForwardTransformer.Transform2d( + this.spatial, + this.coefficients, + 32, + Av1TransformType.DctDct, + Av1TransformSize.Size32x32, + this.BitDepth, + this.workspace); + + return this.coefficients[^1]; + } + + /// + /// Measures the packed-to-expanded boundary of a thirty-two-by-sixty-four forward DCT block. + /// + /// The last coded coefficient written by the transform. + [Benchmark] + [BenchmarkCategory("Forward32x64")] + public int Forward32x64() + { + Av1ForwardTransformer.Transform2d( + this.spatial, + this.coefficients, + 32, + Av1TransformType.DctDct, + Av1TransformSize.Size32x64, + this.BitDepth, + this.workspace); + + return this.coefficients[^1]; + } + + /// + /// Measures a sixty-four-by-sixty-four forward DCT block with the normative coefficient truncation. + /// + /// The last coded coefficient written by the transform. + [Benchmark] + [BenchmarkCategory("Forward64x64")] + public int Forward64x64() + { + Av1ForwardTransformer.Transform2d( + this.spatial, + this.coefficients, + 64, + Av1TransformType.DctDct, + Av1TransformSize.Size64x64, + this.BitDepth, + this.workspace); + + return this.coefficients[^1]; + } + + /// + /// Configures separate production-process measurements for preferred 256-bit and 512-bit vectors. + /// + public sealed class Configuration : ManualConfig + { + /// + /// Initializes a new instance of the class. + /// + public Configuration() + { + this.AddJob( + Job.ShortRun + .WithId("Vector256") + .WithEnvironmentVariable("DOTNET_PreferredVectorBitWidth", "256") + .WithEnvironmentVariable("COMPlus_PreferredVectorBitWidth", "256") + .AsBaseline()); + + this.AddJob( + Job.ShortRun + .WithId("Vector512") + .WithEnvironmentVariable("DOTNET_PreferredVectorBitWidth", "512") + .WithEnvironmentVariable("COMPlus_PreferredVectorBitWidth", "512")); + } + } +} diff --git a/tests/ImageSharp.Benchmarks/Codecs/Heif/LibaomBenchmarkEncoder.cs b/tests/ImageSharp.Benchmarks/Codecs/Heif/LibaomBenchmarkEncoder.cs new file mode 100644 index 0000000000..a0007d0a24 --- /dev/null +++ b/tests/ImageSharp.Benchmarks/Codecs/Heif/LibaomBenchmarkEncoder.cs @@ -0,0 +1,135 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.InteropServices; +using Microsoft.Win32.SafeHandles; +using SixLabors.ImageSharp.Formats.Heif.Av1; +using SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline; +using SixLabors.ImageSharp.Memory; + +namespace SixLabors.ImageSharp.Benchmarks.Codecs.Heif; + +/// +/// Owns a benchmark-only libaom encoder through a C adapter compiled against the reference's actual headers. +/// +internal sealed unsafe partial class LibaomBenchmarkEncoder : SafeHandleZeroOrMinusOneIsInvalid +{ + /// + /// The benchmark adapter's platform-independent library name. + /// + private const string LibraryName = "imagesharp_aom_benchmark"; + + /// + /// Initializes an empty handle for the native create call's generated marshaller. + /// + public LibaomBenchmarkEncoder() + : base(ownsHandle: true) + { + } + + /// + /// Opens one native sequence with a single coding thread, no lookahead, and the requested quality and speed. + /// + public static LibaomBenchmarkEncoder Open(int width, int height, int quality, int speed) + { + CheckStatus(Create((uint)width, (uint)height, (uint)quality, speed, out LibaomBenchmarkEncoder encoder)); + return encoder; + } + + /// + /// Encodes the converted source planes and consumes every output packet before those bytes can be invalidated. + /// + public void Encode(Av1EncoderFrame frame, long frameIndex, Stream output) + { + Buffer2DRegion y = frame.View.GetPlane(Av1Plane.Y); + Buffer2DRegion u = frame.View.GetPlane(Av1Plane.U); + Buffer2DRegion v = frame.View.GetPlane(Av1Plane.V); + fixed (byte* yPointer = y.DangerousGetRowSpan(0), uPointer = u.DangerousGetRowSpan(0), vPointer = v.DangerousGetRowSpan(0)) + { + // The native call is synchronous. Its input descriptors borrow these pinned rows only until + // EncodeFrame returns; the encoder owns any retained reference and lookahead storage itself. + CheckStatus(EncodeFrame(this, yPointer, uPointer, vPointer, y.Stride, u.Stride, frameIndex)); + } + + this.WritePackets(output); + } + + /// + /// Finishes the sequence and writes any remaining coded packets. + /// + public void Finish(Stream output) + { + do + { + CheckStatus(Flush(this)); + } + while (this.WritePackets(output)); + } + + /// + protected override bool ReleaseHandle() => Destroy(this.handle) == 0; + + /// + /// Writes borrowed packet memory before the next call into the native codec invalidates it. + /// + private bool WritePackets(Stream output) + { + bool wrotePacket = false; + while (NextPacket(this, out byte* data, out nuint length) != 0) + { + // Packet lengths are bounded by the benchmark's image dimensions. Stream.Write consumes the + // borrowed bytes synchronously, without retaining a native pointer or creating a managed array. + output.Write(new ReadOnlySpan(data, (int)length)); + wrotePacket = true; + } + + return wrotePacket; + } + + /// + /// Converts a native codec error into a managed benchmark failure instead of accepting invalid timing data. + /// + private static void CheckStatus(int status) + { + if (status != 0) + { + throw new InvalidOperationException(Marshal.PtrToStringUTF8(ErrorString(status))); + } + } + + /// + /// Creates an owned opaque context; no libaom structure layout crosses the managed boundary. + /// + [LibraryImport(LibraryName, EntryPoint = "benchmark_create")] + private static partial int Create(uint width, uint height, uint quality, int speed, out LibaomBenchmarkEncoder encoder); + + /// + /// Borrows three pinned planes for one synchronous native encode call. + /// + [LibraryImport(LibraryName, EntryPoint = "benchmark_encode")] + private static partial int EncodeFrame(LibaomBenchmarkEncoder encoder, byte* y, byte* u, byte* v, int yStride, int uvStride, long frameIndex); + + /// + /// Signals the end of the native sequence. + /// + [LibraryImport(LibraryName, EntryPoint = "benchmark_flush")] + private static partial int Flush(LibaomBenchmarkEncoder encoder); + + /// + /// Returns borrowed native packet storage and its pointer-sized length. + /// + [LibraryImport(LibraryName, EntryPoint = "benchmark_next_packet")] + private static partial int NextPacket(LibaomBenchmarkEncoder encoder, out byte* data, out nuint length); + + /// + /// Destroys the context through the same native library that allocated it. + /// + [LibraryImport(LibraryName, EntryPoint = "benchmark_destroy")] + private static partial int Destroy(nint encoder); + + /// + /// Returns a static UTF-8 error message owned by libaom. + /// + [LibraryImport(LibraryName, EntryPoint = "benchmark_error_string")] + private static partial nint ErrorString(int status); +} diff --git a/tests/ImageSharp.Benchmarks/Codecs/Heif/Native/CMakeLists.txt b/tests/ImageSharp.Benchmarks/Codecs/Heif/Native/CMakeLists.txt new file mode 100644 index 0000000000..f19a785265 --- /dev/null +++ b/tests/ImageSharp.Benchmarks/Codecs/Heif/Native/CMakeLists.txt @@ -0,0 +1,12 @@ +cmake_minimum_required(VERSION 3.20) +project(imagesharp_aom_benchmark LANGUAGES C CXX) + +# Link the explicitly supplied current-main Release reference build. This adapter is benchmark-only; +# it is never referenced by ImageSharp or copied by the production project. +set(AOM_SOURCE_DIRECTORY "" CACHE PATH "Verified official libaom source directory") +set(AOM_BUILD_DIRECTORY "" CACHE PATH "Matching optimized Release libaom build directory") +add_library(imagesharp_aom_benchmark SHARED aom_benchmark.c) +target_include_directories(imagesharp_aom_benchmark PRIVATE "${AOM_SOURCE_DIRECTORY}") +find_library(AOM_LIBRARY NAMES aom PATHS "${AOM_BUILD_DIRECTORY}" NO_DEFAULT_PATH REQUIRED) +target_link_libraries(imagesharp_aom_benchmark PRIVATE "${AOM_LIBRARY}") +set_target_properties(imagesharp_aom_benchmark PROPERTIES C_STANDARD 11 LINKER_LANGUAGE CXX) diff --git a/tests/ImageSharp.Benchmarks/Codecs/Heif/Native/aom_benchmark.c b/tests/ImageSharp.Benchmarks/Codecs/Heif/Native/aom_benchmark.c new file mode 100644 index 0000000000..ba5f00e154 --- /dev/null +++ b/tests/ImageSharp.Benchmarks/Codecs/Heif/Native/aom_benchmark.c @@ -0,0 +1,131 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +#include +#include +#include "aom/aom_encoder.h" +#include "aom/aomcx.h" + +#if defined(_WIN32) +#define BENCHMARK_API __declspec(dllexport) +#else +#define BENCHMARK_API __attribute__((visibility("default"))) +#endif + +// Keep libaom's version-dependent structures and variadic controls entirely on the C side. +// The managed benchmark exchanges only opaque ownership, fixed-width integers, and borrowed buffers. +typedef struct benchmark_encoder { + aom_codec_ctx_t codec; + aom_codec_iter_t iterator; + unsigned int width; + unsigned int height; +} benchmark_encoder; + +// Creates one single-threaded, unlagged, constant-quality sequence using the reference's normal tools. +// A successful result belongs to the caller and must be released by benchmark_destroy. +BENCHMARK_API int benchmark_create(unsigned int width, unsigned int height, + unsigned int quality, int speed, + benchmark_encoder **result) { + aom_codec_enc_cfg_t config; + aom_codec_iface_t *iface = aom_codec_av1_cx(); + aom_codec_err_t status = aom_codec_enc_config_default(iface, &config, AOM_USAGE_GOOD_QUALITY); + *result = NULL; + if (status != AOM_CODEC_OK) return status; + + benchmark_encoder *encoder = calloc(1, sizeof(*encoder)); + if (encoder == NULL) return AOM_CODEC_MEM_ERROR; + + config.g_w = width; + config.g_h = height; + config.g_threads = 1; + config.g_lag_in_frames = 0; + config.g_timebase.num = 1; + config.g_timebase.den = 30; + config.rc_end_usage = AOM_Q; + // Fix both bounds to the requested quantizer. CQ alone permits frame-quality boosts, whereas + // the managed sequence uses this same base quantizer for every frame in the comparison. + config.rc_min_quantizer = quality; + config.rc_max_quantizer = quality; + config.kf_mode = AOM_KF_DISABLED; + status = aom_codec_enc_init(&encoder->codec, iface, &config, 0); + if (status != AOM_CODEC_OK) { + free(encoder); + return status; + } + + // These calls remain type-checked against the current libaom headers, including each control's argument type. + status = aom_codec_control(&encoder->codec, AOME_SET_CPUUSED, speed); + if (status == AOM_CODEC_OK) status = aom_codec_control(&encoder->codec, AOME_SET_CQ_LEVEL, quality); + if (status == AOM_CODEC_OK) status = aom_codec_control(&encoder->codec, AV1E_SET_ROW_MT, 0u); + if (status == AOM_CODEC_OK) status = aom_codec_control(&encoder->codec, AV1E_SET_COLOR_PRIMARIES, AOM_CICP_CP_BT_601); + if (status == AOM_CODEC_OK) status = aom_codec_control(&encoder->codec, AV1E_SET_TRANSFER_CHARACTERISTICS, AOM_CICP_TC_BT_601); + if (status == AOM_CODEC_OK) status = aom_codec_control(&encoder->codec, AV1E_SET_MATRIX_COEFFICIENTS, AOM_CICP_MC_BT_601); + if (status == AOM_CODEC_OK) status = aom_codec_control(&encoder->codec, AV1E_SET_COLOR_RANGE, AOM_CR_FULL_RANGE); + if (status != AOM_CODEC_OK) { + aom_codec_destroy(&encoder->codec); + free(encoder); + return status; + } + + encoder->width = width; + encoder->height = height; + *result = encoder; + return AOM_CODEC_OK; +} + +// Borrows the already converted planes for this synchronous encode call. The caller pins all three +// pointers until it returns; libaom retains its own reference pictures, never these managed input pointers. +BENCHMARK_API int benchmark_encode(benchmark_encoder *encoder, unsigned char *y, + unsigned char *u, unsigned char *v, + int y_stride, int uv_stride, int64_t frame_index) { + aom_image_t image; + if (aom_img_wrap(&image, AOM_IMG_FMT_I420, encoder->width, encoder->height, 1, y) == NULL) { + return AOM_CODEC_INVALID_PARAM; + } + + // ImageSharp's source planes include aligned borders. Describe those existing rows directly instead + // of flattening them into another contiguous YUV allocation before calling the reference encoder. + image.planes[AOM_PLANE_Y] = y; + image.planes[AOM_PLANE_U] = u; + image.planes[AOM_PLANE_V] = v; + image.stride[AOM_PLANE_Y] = y_stride; + image.stride[AOM_PLANE_U] = uv_stride; + image.stride[AOM_PLANE_V] = uv_stride; + encoder->iterator = NULL; + return aom_codec_encode(&encoder->codec, &image, frame_index, 1, 0); +} + +// Drains delayed output at the end of the sequence even though lookahead is disabled. +BENCHMARK_API int benchmark_flush(benchmark_encoder *encoder) { + encoder->iterator = NULL; + return aom_codec_encode(&encoder->codec, NULL, -1, 1, 0); +} + +// The returned packet storage belongs to libaom and is valid only until the next codec call. +// The managed side copies it directly into the same kind of destination stream as its own encoder. +BENCHMARK_API int benchmark_next_packet(benchmark_encoder *encoder, const void **data, size_t *length) { + const aom_codec_cx_pkt_t *packet; + while ((packet = aom_codec_get_cx_data(&encoder->codec, &encoder->iterator)) != NULL) { + if (packet->kind == AOM_CODEC_CX_FRAME_PKT) { + *data = packet->data.frame.buf; + *length = packet->data.frame.sz; + return 1; + } + } + + *data = NULL; + *length = 0; + return 0; +} + +// Releases exactly the native context allocated by benchmark_create. +BENCHMARK_API int benchmark_destroy(benchmark_encoder *encoder) { + aom_codec_err_t status = aom_codec_destroy(&encoder->codec); + free(encoder); + return status; +} + +// Libaom owns this static UTF-8 error string; the caller must not free it. +BENCHMARK_API const char *benchmark_error_string(int status) { + return aom_codec_err_to_string((aom_codec_err_t)status); +} diff --git a/tests/ImageSharp.Benchmarks/Codecs/Heif/README.md b/tests/ImageSharp.Benchmarks/Codecs/Heif/README.md new file mode 100644 index 0000000000..5bc9e1d613 --- /dev/null +++ b/tests/ImageSharp.Benchmarks/Codecs/Heif/README.md @@ -0,0 +1,137 @@ +# AV1 sequence encoder comparison + +`Av1SequenceEncoderBenchmarks` compares the managed AV1 sequence encoder with an optimized build of official libaom `main`. +The native adapter belongs only to this benchmark project. ImageSharp production code remains fully managed. + +## Measurement boundary + +Both methods start with the same three `Rgb24` photographic frames and finish with the complete raw AV1 OBU sequence in a reused `MemoryStream`. +The measured operation includes encoder/workspace construction, RGB-to-YUV conversion for every frame, encoding, output writes, flushing where required, and disposal. +Both use the existing ImageSharp SIMD color converter and write directly into the aligned source planes consumed by their encoder. +Libaom receives pinned plane pointers for each synchronous encode call; it does not read a preconverted input file. + +Image loading, resizing, half-pixel source translations, validation-file writes, decoding, and quality calculations are outside both measured methods. +MemoryStream capacity is retained between operations for both methods. These are RGB-to-OBU measurements, not public HEIF container-save measurements. +Do not compare them with `aomenc`'s internal encode-time report, which excludes the conversion boundary. + +The source is the existing `TestImages.Png.Bike` photograph. Frames one and two are independently translated by half a pixel and one pixel on both axes. +The parameter matrix contains 256x256 and 512x512 frames at ImageSharp efforts seven, eight, and nine. +Input is full-range BT.601, eight-bit 4:2:0; each sequence contains one key frame and two dependent frames. +Both codecs use one coding thread. Libaom uses good-quality mode, no lookahead, disabled automatic key frames, and `cpu-used=6`. +That speed is an independent reference setting, not a mapping from the ImageSharp effort scale. + +ImageSharp's base quantizer index is 120. Libaom's public quantizer 30 maps to that same index in the pinned reference. +The native minimum and maximum quantizers are both fixed to 30, in addition to the CQ setting, so frame-level CQ boosts cannot lower the base quantizer. +Other native coding tools remain enabled. Identical quantizers do not imply identical quality or bitrate; always retain size and decoded-quality results with timings. + +BenchmarkDotNet's allocation column measures managed allocations only. It does not measure libaom's native allocations, pooled native memory, or total peak memory. +Do not interpret its allocation ratio as a whole-encoder memory comparison. + +## Build the reference and benchmark adapter + +The verified source revision is `d565eec60f084421fa34fc0534b760c6452b6a6c`, libaom 3.15.0, exported at `D:\GitHub\ynse01\aom-d565eec6-source`. +The x64 Release build is `D:\GitHub\ynse01\aom-d565eec6-build-x64-release`. +It uses MSVC 19.51.36256.0 and NASM 3.02, runtime CPU detection, and SSE2, SSE4.1, AVX2, and AVX512 kernels. +The generic reference decoder build is suitable for conformance but must not be used as the timing reference. + +Run from the repository root in an x64 Visual Studio developer PowerShell with the existing CMake, Ninja, NASM, and Perl installations available: + +```powershell +$aomSourceDirectory = 'D:/GitHub/ynse01/aom-d565eec6-source' +$aomBuildDirectory = 'D:/GitHub/ynse01/aom-d565eec6-build-x64-release' +$adapterBuildDirectory = 'artifacts/av1-native-benchmark-release' + +cmake -S $aomSourceDirectory -B $aomBuildDirectory -G Ninja ` + -DCMAKE_BUILD_TYPE=Release -DAOM_TARGET_CPU=x86_64 -DENABLE_NASM=ON ` + -DENABLE_TESTS=OFF -DENABLE_EXAMPLES=ON -DENABLE_TOOLS=ON ` + -DCONFIG_WEBM_IO=OFF -DCONFIG_LIBYUV=OFF +if ($LASTEXITCODE -ne 0) { throw 'Reference configuration failed.' } + +cmake --build $aomBuildDirectory --target aomenc aomdec --parallel 4 +if ($LASTEXITCODE -ne 0) { throw 'Reference build failed.' } + +cmake -S tests/ImageSharp.Benchmarks/Codecs/Heif/Native -B $adapterBuildDirectory -G Ninja ` + -DCMAKE_BUILD_TYPE=Release "-DAOM_SOURCE_DIRECTORY=$aomSourceDirectory" "-DAOM_BUILD_DIRECTORY=$aomBuildDirectory" +if ($LASTEXITCODE -ne 0) { throw 'Adapter configuration failed.' } + +cmake --build $adapterBuildDirectory --parallel 1 +if ($LASTEXITCODE -ne 0) { throw 'Adapter build failed.' } + +dotnet build tests/ImageSharp.Benchmarks/ImageSharp.Benchmarks.csproj -c Release -f net11.0 ` + --no-restore --disable-build-servers -m:1 -p:UseSharedCompilation=false ` + -p:SIXLABORS_TESTING_PREVIEW=true -p:SIXLABORS_DISABLE_CONFIG_COPY=true +if ($LASTEXITCODE -ne 0) { throw 'Managed build failed.' } + +Copy-Item -LiteralPath "$adapterBuildDirectory/imagesharp_aom_benchmark.dll" ` + -Destination artifacts/bin/tests/ImageSharp.Benchmarks/Release/net11.0/imagesharp_aom_benchmark.dll +``` + +The adapter links the explicit reference build. It neither downloads tools nor adds a native production dependency. +Rebuild it whenever the source headers or reference library change; record the exact source revision and optimized build configuration with every report. + +## Validate, then measure + +Run one pair first, in the existing benchmark host with the in-process toolchain. Do not run builds or tests concurrently with timing. + +```powershell +$benchmarkAssembly = 'artifacts/bin/tests/ImageSharp.Benchmarks/Release/net11.0/ImageSharp.Benchmarks.dll' +$benchmarkFilter = '*Av1SequenceEncoderBenchmarks.*(Dimension: 256, Effort: 7)' + +dotnet $benchmarkAssembly --inProcess --job Dry --filter $benchmarkFilter ` + --stopOnFirstError --noOverwrite --artifacts artifacts/BenchmarkDotNet/av1-sequence-rgb-dry +``` + +The actual last measured payload for each method and the independently exported source planes are retained under +`tests/Images/ActualOutput/Heif/Av1/Av1SequenceEncoderBenchmarks`. +Both timed methods convert RGB on every invocation; the exported source file exists only for offline quality calculation. +Decode both payloads using the matching current-main `aomdec`, and require the expected complete three-frame YUV length: + +```powershell +$outputDirectory = 'tests/Images/ActualOutput/Heif/Av1/Av1SequenceEncoderBenchmarks' +foreach ($payloadName in 'bike-256-q120-effort7.obu', 'bike-256-q120-libaom-cpu6.obu') { + $payloadPath = Join-Path $outputDirectory $payloadName + & "$aomBuildDirectory/aomdec.exe" --codec=av1 --threads=1 --rawvideo -o "$payloadPath.yuv" $payloadPath + if ($LASTEXITCODE -ne 0) { throw "Reference decode failed: $payloadName" } + + if ((Get-Item -LiteralPath "$payloadPath.yuv").Length -ne (3 * 256 * 256 * 3 / 2)) { + throw "Incomplete decoded sequence: $payloadName" + } +} + +dotnet $benchmarkAssembly --inProcess --job Short --filter $benchmarkFilter ` + --stopOnFirstError --noOverwrite --artifacts artifacts/BenchmarkDotNet/av1-sequence-rgb-short +``` + +Repeat output validation after the measured run. Compute per-plane PSNR against the exported source using +`10 * log10(255^2 * sampleCount / squaredErrorSum)`, accumulated over all three frames. +Aggregate YUV PSNR uses the total squared error and sample count, not the arithmetic mean of the three PSNR values. +Keep bitstream length, quality, absolute time, runtime, CPU information, reference settings, and any environment warnings together. +The short job is an initial diagnostic checkpoint, not a substitute for the complete size/quality/performance matrix or an equal-quality rate-distortion comparison. + +## Verified checkpoint: 2026-09-05 + +The final fixed-quantizer pair ran with BenchmarkDotNet 0.15.8, .NET 11.0.0-preview.7.26381.103, x64 RyuJIT, and AVX512 available. +The in-process Short job used three warmup iterations and three measured iterations. Each operation encoded all three 256x256 frames. + +| Encoder setting | Mean sequence time | OBU size | Y PSNR | U PSNR | V PSNR | Aggregate YUV PSNR | +| --- | ---: | ---: | ---: | ---: | ---: | ---: | +| ImageSharp effort 7, base index 120 | 2,053.31 ms | 11.54 KiB | 39.040 dB | 38.630 dB | 32.778 dB | 37.124 dB | +| Libaom cpu-used 6, fixed public quantizer 30 | 70.67 ms | 8.27 KiB | 39.275 dB | 39.637 dB | 37.351 dB | 38.942 dB | + +Both final measured payloads decode to exactly three complete native YUV frames using the optimized current-main reference decoder. +ImageSharp is about 29 times slower for this case while producing a larger, lower-PSNR sequence. The performance exit gate remains open. +The managed allocation counter reports 9.38 MiB per ImageSharp operation; its source still needs attribution, and it is not comparable with libaom's unmeasured native footprint. + +Evidence is retained in `artifacts/BenchmarkDotNet/av1-sequence-rgb-fixed-q-short-20260905/20260905-131149`. +The initial Dry run and earlier unrestricted-CQ Short run are preliminary checks, not the final comparison above. +BenchmarkDotNet could not change the power plan or query CPU model information in this environment; those warnings remain in the log. +This result is a local diagnostic, not a controlled-hardware or equal-quality performance claim. + +Final OBU SHA-256 values: + +- ImageSharp: `AB0327D33405FB7A6EC1895D465C9EE08E0B218DDA9D7E8EE7E0DCA6C5F8A73B` +- Libaom: `0C44A3475849762B5A09937397DEA4397264B83E79DAF930A06B70734977E4A9` + +The final Release benchmark build has zero errors and 39 existing benchmark warnings outside the new files. +Roslyn compiler and analyzer diagnostics contain no errors or warnings in the new benchmark files. +No production file changed in this benchmark checkpoint; the preceding 2,524-case encoder/entropy verification remains the latest production test run. diff --git a/tests/ImageSharp.Benchmarks/General/PixelConversion/PixelConversion_PackFromRgbPlanes.cs b/tests/ImageSharp.Benchmarks/General/PixelConversion/PixelConversion_PackFromRgbPlanes.cs index 226dcc7775..636a9c4a16 100644 --- a/tests/ImageSharp.Benchmarks/General/PixelConversion/PixelConversion_PackFromRgbPlanes.cs +++ b/tests/ImageSharp.Benchmarks/General/PixelConversion/PixelConversion_PackFromRgbPlanes.cs @@ -244,7 +244,7 @@ public void Rgb24_Avx2_Bytes() ReadOnlySpan g = this.rBuf; ReadOnlySpan b = this.rBuf; Span rgb = this.rgbBuf; - SimdUtils.HwIntrinsics.PackFromRgbPlanesAvx2Reduce(ref r, ref g, ref b, ref rgb); + SimdUtils.HwIntrinsics.PackFromRgbPlanesReduce(ref r, ref g, ref b, ref rgb); } [Benchmark] @@ -254,7 +254,7 @@ public void Rgba32_Avx2_Bytes() ReadOnlySpan g = this.rBuf; ReadOnlySpan b = this.rBuf; Span rgb = this.rgbaBuf; - SimdUtils.HwIntrinsics.PackFromRgbPlanesAvx2Reduce(ref r, ref g, ref b, ref rgb); + SimdUtils.HwIntrinsics.PackFromRgbPlanesReduce(ref r, ref g, ref b, ref rgb); } #pragma warning disable SA1132 diff --git a/tests/ImageSharp.Benchmarks/General/PixelConversion/PixelConversion_PackFromRgbPlanesExact.cs b/tests/ImageSharp.Benchmarks/General/PixelConversion/PixelConversion_PackFromRgbPlanesExact.cs new file mode 100644 index 0000000000..45f476493a --- /dev/null +++ b/tests/ImageSharp.Benchmarks/General/PixelConversion/PixelConversion_PackFromRgbPlanesExact.cs @@ -0,0 +1,64 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using BenchmarkDotNet.Attributes; +using SixLabors.ImageSharp.PixelFormats; + +namespace SixLabors.ImageSharp.Benchmarks.General.PixelConversion; + +/// +/// Measures planar RGB packing for padded and exact-length decoder rows. +/// +public class PixelConversion_PackFromRgbPlanesExact +{ + private byte[] red; + private byte[] green; + private byte[] blue; + private Rgb24[] exactDestination; + private Rgb24[] paddedDestination; + + /// + /// Gets or sets the decoded row width. + /// + [Params(1920, 4242)] + public int Count { get; set; } + + /// + /// Creates deterministic component planes and destination rows outside the measured operation. + /// + [GlobalSetup] + public void Setup() + { + this.red = new byte[this.Count]; + this.green = new byte[this.Count]; + this.blue = new byte[this.Count]; + this.exactDestination = new Rgb24[this.Count]; + this.paddedDestination = new Rgb24[this.Count + 3]; + + new Random(42).NextBytes(this.red); + new Random(43).NextBytes(this.green); + new Random(44).NextBytes(this.blue); + } + + /// + /// Packs a row using the legacy decoder contract with three writable destination pixels beyond the row. + /// + [Benchmark(Baseline = true)] + public void PaddedDestination() => SimdUtils.PackFromRgbPlanes(this.red, this.green, this.blue, this.paddedDestination); + + /// + /// Packs through a padded proxy and copies the completed row, matching the former decoder fallback. + /// + [Benchmark] + public void PaddedProxyAndCopy() + { + SimdUtils.PackFromRgbPlanes(this.red, this.green, this.blue, this.paddedDestination); + this.paddedDestination.AsSpan(0, this.Count).CopyTo(this.exactDestination); + } + + /// + /// Packs a row directly into the exact-length destination exposed by an image frame. + /// + [Benchmark] + public void ExactDestination() => SimdUtils.PackFromRgbPlanes(this.red, this.green, this.blue, this.exactDestination); +} diff --git a/tests/ImageSharp.Tests/Common/SimdUtilsTests.cs b/tests/ImageSharp.Tests/Common/SimdUtilsTests.cs index b4fbf5fd80..7b7b1b371b 100644 --- a/tests/ImageSharp.Tests/Common/SimdUtilsTests.cs +++ b/tests/ImageSharp.Tests/Common/SimdUtilsTests.cs @@ -272,7 +272,7 @@ public void PackFromRgbPlanes_Rgba32(int count) => TestPackFromRgbPlanes SimdUtils.PackFromRgbPlanes(r, g, b, actual)); [Fact] - public void PackFromRgbPlanesAvx2Reduce_Rgb24() + public void PackFromRgbPlanesReduce_Rgb24() { if (!Avx2.IsSupported) { @@ -282,15 +282,14 @@ public void PackFromRgbPlanesAvx2Reduce_Rgb24() byte[] r = [.. Enumerable.Range(0, 32).Select(x => (byte)x)]; byte[] g = [.. Enumerable.Range(100, 32).Select(x => (byte)x)]; byte[] b = [.. Enumerable.Range(200, 32).Select(x => (byte)x)]; - const int padding = 4; - Rgb24[] d = new Rgb24[32 + padding]; + Rgb24[] d = new Rgb24[32]; ReadOnlySpan rr = r.AsSpan(); ReadOnlySpan gg = g.AsSpan(); ReadOnlySpan bb = b.AsSpan(); Span dd = d.AsSpan(); - SimdUtils.HwIntrinsics.PackFromRgbPlanesAvx2Reduce(ref rr, ref gg, ref bb, ref dd); + SimdUtils.HwIntrinsics.PackFromRgbPlanesReduce(ref rr, ref gg, ref bb, ref dd); for (int i = 0; i < 32; i++) { @@ -302,11 +301,11 @@ public void PackFromRgbPlanesAvx2Reduce_Rgb24() Assert.Equal(0, rr.Length); Assert.Equal(0, gg.Length); Assert.Equal(0, bb.Length); - Assert.Equal(padding, dd.Length); + Assert.Equal(0, dd.Length); } [Fact] - public void PackFromRgbPlanesAvx2Reduce_Rgba32() + public void PackFromRgbPlanesReduce_Rgba32() { if (!Avx2.IsSupported) { @@ -324,7 +323,7 @@ public void PackFromRgbPlanesAvx2Reduce_Rgba32() ReadOnlySpan bb = b.AsSpan(); Span dd = d.AsSpan(); - SimdUtils.HwIntrinsics.PackFromRgbPlanesAvx2Reduce(ref rr, ref gg, ref bb, ref dd); + SimdUtils.HwIntrinsics.PackFromRgbPlanesReduce(ref rr, ref gg, ref bb, ref dd); for (int i = 0; i < 32; i++) { @@ -354,10 +353,10 @@ internal static void TestPackFromRgbPlanes(int count, Action (uint)i).ToArray(); + int bufferLength = (int)Numerics.DivideCeil((uint)(values.Length * bitCount), 8); + byte[] buffer = new byte[bufferLength]; + Av1BitStreamWriter writer = new(buffer); + for (int i = 0; i < values.Length; i++) + { + writer.WriteLiteral(values[i], bitCount); + } + + writer.Flush(); + + // Read the written value back. + Av1BitStreamReader reader = new(buffer); + uint[] actuals = new uint[values.Length]; + for (int i = 0; i < values.Length; i++) + { + uint actual = reader.ReadLiteral(bitCount); + actuals[i] = actual; + } + + Assert.Equal(values, actuals); + } + + [Theory] + [InlineData(4, 6, 4, 9, 14)] + [InlineData(8, 42, 8, 189, 63)] + [InlineData(8, 52, 18, 255, 241)] + [InlineData(16, 4050, 16003, 503, 814)] + public void ReadWriteAsLiteralArray(int bitCount, uint val1, uint val2, uint val3, uint val4) + { + uint[] values = [val1, val2, val3, val4]; + int bufferLength = (int)Numerics.DivideCeil((uint)(values.Length * bitCount), 8); + byte[] buffer = new byte[bufferLength]; + Av1BitStreamWriter writer = new(buffer); + for (int i = 0; i < values.Length; i++) + { + writer.WriteLiteral(values[i], bitCount); + } + + writer.Flush(); + + // Read the written value back. + Av1BitStreamReader reader = new(buffer); + for (int i = 0; i < values.Length; i++) + { + uint actual = reader.ReadLiteral(bitCount); + Assert.NotEqual(0U, actual); + Assert.Equal(values[i], actual); + } + } + + [Theory] + + [InlineData(4, 0, 1, 2, 3)] + [InlineData(5, 0, 1, 2, 3)] + [InlineData(5, 1, 2, 3, 4)] + [InlineData(8, 0, 1, 2, 3)] + [InlineData(8, 4, 5, 6, 7)] + [InlineData(16, 15, 0, 5, 8)] + public void ReadWriteAsNonSymmetricArray(uint numberOfSymbols, uint val1, uint val2, uint val3, uint val4) + { + uint[] values = [val1, val2, val3, val4]; + byte[] buffer = new byte[values.Length * sizeof(uint)]; + Av1BitStreamWriter writer = new(buffer); + for (int i = 0; i < values.Length; i++) + { + writer.WriteNonSymmetric(values[i], numberOfSymbols); + } + + writer.Flush(); + + // Read the written value back. + Av1BitStreamReader reader = new(buffer); + uint[] actuals = new uint[4]; + for (int i = 0; i < values.Length; i++) + { + ulong actual = reader.ReadNonSymmetric(numberOfSymbols); + actuals[i] = (uint)actual; + } + + Assert.Equal(values, actuals); + } + + [Fact] + public void SignedReferenceSubexponentialMatchesFiniteRecentering() + { + const int ValueMagnitude = 5; + const int GroupBitCount = 3; + byte[] buffer = new byte[2]; + Av1BitStreamWriter writer = new(buffer); + writer.WriteSignedReferenceSubexponential(-4, ValueMagnitude, GroupBitCount, -4); + writer.WriteSignedReferenceSubexponential(4, ValueMagnitude, GroupBitCount, -4); + writer.WriteSignedReferenceSubexponential(0, ValueMagnitude, GroupBitCount, 0); + writer.Flush(); + + Assert.Equal([0x1e, 0x00], buffer); + + Av1BitStreamReader reader = new(buffer); + Assert.Equal(-4, reader.ReadSignedReferenceSubexponential(ValueMagnitude, GroupBitCount, -4)); + Assert.Equal(4, reader.ReadSignedReferenceSubexponential(ValueMagnitude, GroupBitCount, -4)); + Assert.Equal(0, reader.ReadSignedReferenceSubexponential(ValueMagnitude, GroupBitCount, 0)); + } + + [Theory] + [InlineData(3)] + [InlineData(4)] + [InlineData(5)] + [InlineData(7)] + [InlineData(8)] + public void ReadSignedRainbowArray(int bitCount) + { + int maxValue = (1 << (bitCount - 1)) - 1; + int[] values = Enumerable.Range(-maxValue, maxValue).ToArray(); + int bufferLength = (int)Numerics.DivideCeil((uint)(values.Length * bitCount), 8); + byte[] buffer = new byte[bufferLength]; + Av1BitStreamWriter writer = new(buffer); + for (int i = 0; i < values.Length; i++) + { + writer.WriteSignedFromUnsigned(values[i], bitCount); + } + + writer.Flush(); + + // Read the written value back. + Av1BitStreamReader reader = new(buffer); + int[] actuals = new int[values.Length]; + for (int i = 0; i < values.Length; i++) + { + int actual = reader.ReadSignedFromUnsigned(bitCount); + actuals[i] = actual; + } + + Assert.Equal(values, actuals); + } + + [Fact] + public void ReadSignedFromUnsigned() + { + // arrange + byte[] buffer = { 0xd2, 0xa4 }; + Av1BitStreamReader reader = new(buffer); + int expected0 = -23; + int expected1 = 41; + + // act + int actual0 = reader.ReadSignedFromUnsigned(7); + int actual1 = reader.ReadSignedFromUnsigned(7); + + Assert.Equal(expected0, actual0); + Assert.Equal(expected1, actual1); + } + + [Theory] + [InlineData(new byte[] { 0x01 }, 1, 1, 8)] + [InlineData(new byte[] { 0x01, 0x00, 0x00, 0x00 }, 1, 4, 32)] // One byte value with leading bytes. + [InlineData(new byte[] { 0xD9, 0x01 }, 473, 2, 16)] // Two bytes. + [InlineData(new byte[] { 0xD9, 0x01, 0x00, 0x00 }, 473, 4, 32)] // Two byte value with leading bytes. + public void ReadLittleEndian(byte[] buffer, uint expected, int n, int expectedBitPosition) + { + // arrange + Av1BitStreamReader reader = new(buffer); + + // act + uint actual = reader.ReadLittleEndian(n); + + Assert.Equal(expected, actual); + Assert.Equal(expectedBitPosition, reader.BitPosition); + } + + [Theory] + [InlineData(new byte[] { 0x80 }, 0, 1)] // Zero bit value. + [InlineData(new byte[] { 0x60 }, 2, 3)] // One bit value, 011. + [InlineData(new byte[] { 0x38 }, 6, 5)] // Two bit value, 00111. + [InlineData(new byte[] { 0x00, 0x00, 0x00, 0x01, 0xFF, 0xFF, 0xFF, 0xFF, 0xFE }, uint.MaxValue - 1, 63)] // 31 bit value. + public void ReadUnsignedVariableLength(byte[] buffer, uint expected, int expectedBitPosition) + { + // arrange + Av1BitStreamReader reader = new(buffer); + + // act + uint actual = reader.ReadUnsignedVariableLength(); + + // assert + Assert.Equal(expected, actual); + Assert.Equal(expectedBitPosition, reader.BitPosition); + } + + [Theory] + [InlineData(5, 6, 4, -7, -2)] + [InlineData(7, 26, -8, -19, -26)] + [InlineData(8, 52, 127, -127, -21)] + [InlineData(16, -4050, -16003, -503, 8414)] + public void ReadWriteSignedArray(int bitCount, int val1, int val2, int val3, int val4) + { + int[] values = [val1, val2, val3, val4]; + int bufferLength = (int)Numerics.DivideCeil((uint)(values.Length * bitCount), 8); + byte[] buffer = new byte[bufferLength]; + Av1BitStreamWriter writer = new(buffer); + for (int i = 0; i < values.Length; i++) + { + writer.WriteSignedFromUnsigned(values[i], bitCount); + } + + writer.Flush(); + + // Read the written value back. + Av1BitStreamReader reader = new(buffer); + int[] actuals = new int[4]; + for (int i = 0; i < values.Length; i++) + { + int actual = reader.ReadSignedFromUnsigned(bitCount); + actuals[i] = actual; + } + + Assert.Equal(values, actuals); + } + + [Theory] + [InlineData(new byte[] { 0x01 }, 1, 8)] // One byte value. + [InlineData(new byte[] { 0x81, 0x80, 0x80, 0x00 }, 1, 32)] // One byte value with trailing bytes. + [InlineData(new byte[] { 0xD9, 0x01 }, 217, 16)] // Two byte value. + [InlineData(new byte[] { 0xD9, 0x81, 0x80, 0x80, 0x00 }, 217, 40)] // Two byte value with trailing bytes. + public void ReadLittleEndianBytes128(byte[] buffer, ulong expected, int expectedBitPosition) + { + // arrange + Av1BitStreamReader reader = new(buffer); + + // act + ulong actual = reader.ReadLittleEndianBytes128(out int length); + + // assert + Assert.Equal(expected, actual); + Assert.Equal(expectedBitPosition, reader.BitPosition); + Assert.NotEqual(0UL, actual); + } + + [Theory] + [InlineData(4, 6, 7, 9, 14)] + [InlineData(8, 42, 8, 189, 63)] + [InlineData(8, 52, 18, 255, 241)] + [InlineData(16, 4050, 16003, 503, 8414)] + public void ReadWriteLittleEndianBytes128Array(uint val0, uint val1, uint val2, uint val3, uint val4) + { + uint[] values = [val0, val1, val2, val3, val4]; + const int MaximumEncodedUInt32Length = 5; + byte[] buffer = new byte[values.Length * MaximumEncodedUInt32Length]; + Av1BitStreamWriter writer = new(buffer); + for (int i = 0; i < values.Length; i++) + { + writer.WriteLittleEndianBytes128(values[i]); + } + + writer.Flush(); + + // Read the written value back. + Av1BitStreamReader reader = new(buffer); + uint[] actuals = new uint[5]; + for (int i = 0; i < values.Length; i++) + { + ulong actual = reader.ReadLittleEndianBytes128(out int length); + actuals[i] = (uint)actual; + Assert.NotEqual(0UL, actual); + } + + Assert.Equal(values, actuals); + } +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1BlockSizeTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1BlockSizeTests.cs new file mode 100644 index 0000000000..7e3506aaa2 --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1BlockSizeTests.cs @@ -0,0 +1,121 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1; + +[Trait("Format", "Avif")] +public class Av1BlockSizeTests +{ + [Theory] + [MemberData(nameof(GetAllSizes))] + internal void GetWidthReturnsCorrectWidth(int s) + { + // Assign + Av1BlockSize blockSize = (Av1BlockSize)s; + int expectedWidth = blockSize switch + { + Av1BlockSize.Block4x4 or Av1BlockSize.Block4x8 or Av1BlockSize.Block4x16 => 4, + Av1BlockSize.Block8x4 or Av1BlockSize.Block8x8 or Av1BlockSize.Block8x16 or Av1BlockSize.Block8x32 => 8, + Av1BlockSize.Block16x4 or Av1BlockSize.Block16x8 or Av1BlockSize.Block16x16 or Av1BlockSize.Block16x32 or Av1BlockSize.Block16x64 => 16, + Av1BlockSize.Block32x8 or Av1BlockSize.Block32x16 or Av1BlockSize.Block32x32 or Av1BlockSize.Block32x64 => 32, + Av1BlockSize.Block64x16 or Av1BlockSize.Block64x32 or Av1BlockSize.Block64x64 or Av1BlockSize.Block64x128 => 64, + Av1BlockSize.Block128x64 or Av1BlockSize.Block128x128 => 128, + _ => -1 + }; + + // Act + int actualWidth = blockSize.GetWidth(); + + // Assert + Assert.Equal(expectedWidth, actualWidth); + } + + [Theory] + [MemberData(nameof(GetAllSizes))] + internal void GetHeightReturnsCorrectHeight(int s) + { + // Assign + Av1BlockSize blockSize = (Av1BlockSize)s; + int expectedHeight = blockSize switch + { + Av1BlockSize.Block4x4 or Av1BlockSize.Block8x4 or Av1BlockSize.Block16x4 => 4, + Av1BlockSize.Block4x8 or Av1BlockSize.Block8x8 or Av1BlockSize.Block16x8 or Av1BlockSize.Block32x8 => 8, + Av1BlockSize.Block4x16 or Av1BlockSize.Block8x16 or Av1BlockSize.Block16x16 or Av1BlockSize.Block32x16 or Av1BlockSize.Block64x16 => 16, + Av1BlockSize.Block8x32 or Av1BlockSize.Block16x32 or Av1BlockSize.Block32x32 or Av1BlockSize.Block64x32 => 32, + Av1BlockSize.Block16x64 or Av1BlockSize.Block32x64 or Av1BlockSize.Block64x64 or Av1BlockSize.Block128x64 => 64, + Av1BlockSize.Block64x128 or Av1BlockSize.Block128x128 => 128, + _ => -1 + }; + + // Act + int actualHeight = blockSize.GetHeight(); + + // Assert + Assert.Equal(expectedHeight, actualHeight); + } + + [Theory] + [MemberData(nameof(GetAllSizes))] + internal void GetSubSampledReturnsCorrectSize(int s) + { + if (s is 0 or 1 or 2 or 16 or 17) + { + // Exceptional values, skip for this generic test. + return; + } + + // Assign + Av1BlockSize blockSize = (Av1BlockSize)s; + int originalWidth = blockSize.GetWidth(); + int originalHeight = blockSize.GetHeight(); + int halfWidth = originalWidth / 2; + int halfHeight = originalHeight / 2; + + // Act + Av1BlockSize actualNoNo = blockSize.GetSubsampled(false, false); + Av1BlockSize actualYesNo = blockSize.GetSubsampled(true, false); + Av1BlockSize actualNoYes = blockSize.GetSubsampled(false, true); + Av1BlockSize actualYesYes = blockSize.GetSubsampled(true, true); + + // Assert + Assert.Equal(originalWidth, actualNoNo.GetWidth()); + Assert.Equal(originalHeight, actualNoNo.GetHeight()); + + if (actualYesNo != Av1BlockSize.Invalid) + { + Assert.Equal(halfWidth, actualYesNo.GetWidth()); + Assert.Equal(originalHeight, actualYesNo.GetHeight()); + } + + if (actualNoYes != Av1BlockSize.Invalid) + { + Assert.Equal(originalWidth, actualNoYes.GetWidth()); + Assert.Equal(halfHeight, actualNoYes.GetHeight()); + } + + Assert.Equal(halfWidth, actualYesYes.GetWidth()); + Assert.Equal(halfHeight, actualYesYes.GetHeight()); + } + + public static TheoryData GetAllSizes() + { + TheoryData combinations = []; + for (int s = 0; s < (int)Av1BlockSize.AllSizes; s++) + { + combinations.Add(s); + } + + return combinations; + } + + private static int GetRatio(Av1BlockSize blockSize) + { + int width = blockSize.GetWidth(); + int height = blockSize.GetHeight(); + int ratio = width >= height ? width / height : -height / width; + return ratio; + } +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1CdefFilterTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1CdefFilterTests.cs new file mode 100644 index 0000000000..46b69f9551 --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1CdefFilterTests.cs @@ -0,0 +1,692 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Numerics; +using SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.Cdef; +using SixLabors.ImageSharp.Tests.TestUtilities; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1; + +/// +/// Verifies AV1 constrained directional enhancement filtering across sample precision, block geometry, and intrinsic tiers. +/// +[Trait("Format", "Avif")] +public class Av1CdefFilterTests +{ + /// + /// The hardware configurations required to exercise packed filtering and the scalar fallback. + /// + private const HwIntrinsics Configurations = HwIntrinsics.AllowAll | HwIntrinsics.DisableAVX2 | HwIntrinsics.DisableHWIntrinsic; + + /// + /// The row stride of the bordered source plane used by the filter tests. + /// + private const int SourceStride = 16; + + /// + /// The number of unavailable samples surrounding the test image. + /// + private const int SourceBorder = 2; + + /// + /// Verifies direction selection and variance against an independent scalar definition. + /// + [Fact] + public void FindDirectionMatchesReference() + => FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidateDirections, Configurations); + + /// + /// Verifies every CDEF block geometry and strength mode against an independent scalar definition. + /// + [Fact] + public void FilterBlockMatchesReference() + => FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidateFilters, Configurations); + + /// + /// Verifies eight-bit widening and 16-bit copying across packed and scalar execution tiers. + /// + [Fact] + public void CopyPlaneMatchesReference() + => FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidatePlaneCopies, Configurations); + + /// + /// Verifies the complete asymmetric chroma direction mappings and the unchanged symmetric mappings. + /// + [Fact] + public void ConvertDirectionMatchesSubsamplingGeometry() + { + int[] horizontalSubsampling = [7, 0, 2, 4, 5, 6, 6, 6]; + int[] verticalSubsampling = [1, 2, 2, 2, 3, 4, 6, 0]; + for (int direction = 0; direction < 8; direction++) + { + Assert.Equal(horizontalSubsampling[direction], Av1CdefFilter.ConvertDirection(direction, 1, 0)); + Assert.Equal(verticalSubsampling[direction], Av1CdefFilter.ConvertDirection(direction, 0, 1)); + Assert.Equal(direction, Av1CdefFilter.ConvertDirection(direction, 0, 0)); + Assert.Equal(direction, Av1CdefFilter.ConvertDirection(direction, 1, 1)); + } + } + + /// + /// Verifies luma strength adjustment at zero, logarithmic-class boundaries, and the capped variance class. + /// + [Fact] + public void AdjustStrengthMatchesIndependentDefinition() + { + foreach (int strength in new[] { 0, 4, 15, 60 }) + { + foreach (int variance in new[] { 0, 1, 63, 64, 255, 4096, 1 << 20 }) + { + int varianceClass = variance >> 6; + int adjustment = varianceClass == 0 ? 0 : Math.Min(BitOperations.Log2((uint)varianceClass), 12); + int expected = variance == 0 ? 0 : ((strength * (4 + adjustment)) + 8) >> 4; + Assert.Equal(expected, Av1CdefFilter.AdjustStrength(strength, variance)); + } + } + } + + /// + /// Verifies that the deterministic direction corpus exercises every selected-direction branch. + /// + [Fact] + public void DirectionCorpusCoversEveryDirection() + { + const int stride = 32; + const int sourceOffset = (4 * stride) + 5; + int secondSourceOffset = sourceOffset + 8; + HashSet observedDirections = []; + for (int pattern = 0; pattern < 8; pattern++) + { + ushort[] source = new ushort[stride * 16]; + PopulateDirectionSource(source, sourceOffset, stride, pattern, 8); + observedDirections.Add(FindDirectionReference(source, sourceOffset, stride, 0, out _)); + observedDirections.Add(FindDirectionReference(source, secondSourceOffset, stride, 0, out _)); + } + + Assert.Equal(Enumerable.Range(0, 8), observedDirections.Order()); + } + + /// + /// Exercises direction search with multiple source patterns at every supported sample precision. + /// + private static void ValidateDirections() + { + const int stride = 32; + const int sourceOffset = (4 * stride) + 5; + + foreach (int bitDepth in new[] { 8, 10, 12 }) + { + int coefficientShift = bitDepth - 8; + for (int pattern = 0; pattern < 8; pattern++) + { + ushort[] source = new ushort[stride * 16]; + int secondSourceOffset = sourceOffset + 8; + PopulateDirectionSource(source, sourceOffset, stride, pattern, bitDepth); + + int expectedDirection = FindDirectionReference(source, sourceOffset, stride, coefficientShift, out int expectedVariance); + int actualDirection = Av1CdefFilter.FindDirection(source, sourceOffset, stride, coefficientShift, out int actualVariance); + Assert.Equal(expectedDirection, actualDirection); + Assert.Equal(expectedVariance, actualVariance); + + int secondExpectedDirection = FindDirectionReference(source, secondSourceOffset, stride, coefficientShift, out int secondExpectedVariance); + Av1CdefFilter.FindDirections( + source, + sourceOffset, + secondSourceOffset, + stride, + coefficientShift, + out int firstActualDirection, + out int firstActualVariance, + out int secondActualDirection, + out int secondActualVariance); + + Assert.Equal(expectedDirection, firstActualDirection); + Assert.Equal(expectedVariance, firstActualVariance); + Assert.Equal(secondExpectedDirection, secondActualDirection); + Assert.Equal(secondExpectedVariance, secondActualVariance); + } + } + } + + /// + /// Exercises CDEF source-plane preparation across vector boundaries and row padding. + /// + private static void ValidatePlaneCopies() + { + const int height = 6; + + foreach (int width in new[] { 4, 7, 8, 15, 16, 31, 32, 37 }) + { + int sourceStride = width + 9; + int destinationStride = width + 11; + int sourceOffset = sourceStride + 3; + int destinationOffset = destinationStride + 5; + byte[] byteSource = new byte[(height + 2) * sourceStride]; + ushort[] ushortSource = new ushort[byteSource.Length]; + ushort[] expectedByteDestination = Enumerable.Repeat((ushort)0x7A7A, (height + 2) * destinationStride).ToArray(); + ushort[] expectedUShortDestination = Enumerable.Repeat((ushort)0x5A5A, expectedByteDestination.Length).ToArray(); + + for (int row = 0; row < height; row++) + { + for (int column = 0; column < width; column++) + { + int index = sourceOffset + (row * sourceStride) + column; + byteSource[index] = (byte)((row * 47) + (column * 13)); + ushortSource[index] = (ushort)(byteSource[index] * 17); + expectedByteDestination[destinationOffset + (row * destinationStride) + column] = byteSource[index]; + expectedUShortDestination[destinationOffset + (row * destinationStride) + column] = ushortSource[index]; + } + } + + ushort[] actualByteDestination = Enumerable.Repeat((ushort)0x7A7A, expectedByteDestination.Length).ToArray(); + ushort[] actualUShortDestination = Enumerable.Repeat((ushort)0x5A5A, expectedUShortDestination.Length).ToArray(); + + Av1CdefFilter.CopyPlane( + byteSource, + sourceOffset, + sourceStride, + actualByteDestination, + destinationOffset, + destinationStride, + width, + height); + + Av1CdefFilter.CopyPlane( + ushortSource, + sourceOffset, + sourceStride, + actualUShortDestination, + destinationOffset, + destinationStride, + width, + height); + + Assert.Equal(expectedByteDestination, actualByteDestination); + Assert.Equal(expectedUShortDestination, actualUShortDestination); + } + } + + /// + /// Populates two adjacent 8x8 blocks with deterministic directional samples. + /// + /// The destination source plane. + /// The first populated sample. + /// The source row stride. + /// The deterministic pattern index. + /// The sample precision. + private static void PopulateDirectionSource(Span source, int sourceOffset, int sourceStride, int pattern, int bitDepth) + { + int coefficientShift = bitDepth - 8; + int maximum = (1 << bitDepth) - 1; + for (int row = 0; row < 8; row++) + { + for (int column = 0; column < 16; column++) + { + int localColumn = column & 7; + int direction = column < 8 ? pattern : (pattern + 4) & 7; + int line = GetDirectionLineReference(direction, row, localColumn); + + // Samples are constant along the requested geometric line and vary between lines. Direction search + // therefore minimizes reconstruction error in that direction while still exercising nonuniform values. + int value = (24 + (line * 14)) << coefficientShift; + source[sourceOffset + (row * sourceStride) + column] = (ushort)(value & maximum); + } + } + } + + /// + /// Maps one source coordinate to its line in an AV1 direction independently of the production implementation. + /// + /// The zero-based AV1 direction index. + /// The source row. + /// The source column. + /// The zero-based line index. + private static int GetDirectionLineReference(int direction, int row, int column) => direction switch + { + 0 => row + column, + 1 => row + (column / 2), + 2 => row, + 3 => 3 + row - (column / 2), + 4 => 7 + row - column, + 5 => 3 - (row / 2) + column, + 6 => column, + _ => (row / 2) + column + }; + + /// + /// Exercises all normative block dimensions, directions, strength combinations, output types, and coded precisions. + /// + private static void ValidateFilters() + { + (int Width, int Height)[] dimensions = [(4, 4), (4, 8), (8, 4), (8, 8)]; + foreach (int bitDepth in new[] { 8, 10, 12 }) + { + int coefficientShift = bitDepth - 8; + int scale = 1 << coefficientShift; + ushort[] source = CreateBorderedSource(bitDepth); + int sourceOffset = (SourceBorder * SourceStride) + SourceBorder; + (int Primary, int Secondary)[] strengths = [(0, 0), (4 * scale, 0), (0, 2 * scale), (5 * scale, 2 * scale)]; + + foreach ((int blockWidth, int blockHeight) in dimensions) + { + foreach (int direction in Enumerable.Range(0, 8)) + { + foreach ((int primaryStrength, int secondaryStrength) in strengths) + { + if (bitDepth == 8) + { + AssertByteFilter( + source, + sourceOffset, + blockWidth, + blockHeight, + primaryStrength, + secondaryStrength, + direction, + coefficientShift); + } + + AssertUInt16Filter( + source, + sourceOffset, + blockWidth, + blockHeight, + primaryStrength, + secondaryStrength, + direction, + coefficientShift); + } + } + } + } + } + + /// + /// Creates a deterministic image whose top-left output block touches the unavailable-neighbor border. + /// + /// The source sample precision. + /// The bordered 16-bit source plane. + private static ushort[] CreateBorderedSource(int bitDepth) + { + ushort[] source = Enumerable.Repeat(Av1CdefFilter.VeryLarge, SourceStride * SourceStride).ToArray(); + int maximum = (1 << bitDepth) - 1; + int scale = 1 << (bitDepth - 8); + for (int row = SourceBorder; row < SourceStride - SourceBorder; row++) + { + for (int column = SourceBorder; column < SourceStride - SourceBorder; column++) + { + int localRow = row - SourceBorder; + int localColumn = column - SourceBorder; + int value = (72 + (localRow * 9) + (localColumn * 5) + ((localRow * localColumn) & 15)) * scale; + source[(row * SourceStride) + column] = (ushort)Math.Min(value, maximum); + } + } + + return source; + } + + /// + /// Verifies one eight-bit output block while retaining untouched destination padding in the comparison. + /// + /// The bordered source plane. + /// The first source sample in the block. + /// The output block width. + /// The output block height. + /// The primary filter strength. + /// The secondary filter strength. + /// The primary filter direction. + /// The source precision shift. + private static void AssertByteFilter( + ushort[] source, + int sourceOffset, + int blockWidth, + int blockHeight, + int primaryStrength, + int secondaryStrength, + int direction, + int coefficientShift) + { + const int destinationStride = 12; + const int destinationOffset = destinationStride + 1; + byte[] expected = Enumerable.Repeat((byte)231, destinationStride * 10).ToArray(); + byte[] actual = (byte[])expected.Clone(); + int damping = 5 + coefficientShift; + + FilterReference( + source, + sourceOffset, + SourceStride, + expected, + destinationOffset, + destinationStride, + primaryStrength, + secondaryStrength, + direction, + damping, + coefficientShift, + blockWidth, + blockHeight); + + Av1CdefFilter.FilterBlock( + source, + sourceOffset, + SourceStride, + actual, + destinationOffset, + destinationStride, + primaryStrength, + secondaryStrength, + direction, + damping, + damping, + coefficientShift, + blockWidth, + blockHeight); + + Assert.Equal(expected, actual); + } + + /// + /// Verifies one 16-bit output block while retaining untouched destination padding in the comparison. + /// + /// The bordered source plane. + /// The first source sample in the block. + /// The output block width. + /// The output block height. + /// The primary filter strength. + /// The secondary filter strength. + /// The primary filter direction. + /// The source precision shift. + private static void AssertUInt16Filter( + ushort[] source, + int sourceOffset, + int blockWidth, + int blockHeight, + int primaryStrength, + int secondaryStrength, + int direction, + int coefficientShift) + { + const int destinationStride = 12; + const int destinationOffset = destinationStride + 1; + ushort[] expected = Enumerable.Repeat((ushort)60000, destinationStride * 10).ToArray(); + ushort[] actual = (ushort[])expected.Clone(); + int damping = 5 + coefficientShift; + + FilterReference( + source, + sourceOffset, + SourceStride, + expected, + destinationOffset, + destinationStride, + primaryStrength, + secondaryStrength, + direction, + damping, + coefficientShift, + blockWidth, + blockHeight); + + Av1CdefFilter.FilterBlock( + source, + sourceOffset, + SourceStride, + actual, + destinationOffset, + destinationStride, + primaryStrength, + secondaryStrength, + direction, + damping, + damping, + coefficientShift, + blockWidth, + blockHeight); + + Assert.Equal(expected, actual); + } + + /// + /// Finds one direction and variance using the scalar AV1 definition independently of the production layouts. + /// + /// The source plane. + /// The first sample in the 8x8 block. + /// The source row stride. + /// The source precision shift. + /// Receives the directional variance. + /// The selected direction. + private static int FindDirectionReference( + ReadOnlySpan source, + int sourceOffset, + int sourceStride, + int coefficientShift, + out int variance) + { + int[,] partials = new int[8, 15]; + int[] costs = new int[8]; + int[] divisions = [0, 840, 420, 280, 210, 168, 140, 120, 105]; + for (int row = 0; row < 8; row++) + { + for (int column = 0; column < 8; column++) + { + int value = (source[sourceOffset + (row * sourceStride) + column] >> coefficientShift) - 128; + partials[0, row + column] += value; + partials[1, row + (column / 2)] += value; + partials[2, row] += value; + partials[3, 3 + row - (column / 2)] += value; + partials[4, 7 + row - column] += value; + partials[5, 3 - (row / 2) + column] += value; + partials[6, column] += value; + partials[7, (row / 2) + column] += value; + } + } + + for (int line = 0; line < 8; line++) + { + costs[2] += partials[2, line] * partials[2, line]; + costs[6] += partials[6, line] * partials[6, line]; + } + + costs[2] *= divisions[8]; + costs[6] *= divisions[8]; + for (int line = 0; line < 7; line++) + { + costs[0] += ((partials[0, line] * partials[0, line]) + (partials[0, 14 - line] * partials[0, 14 - line])) * divisions[line + 1]; + costs[4] += ((partials[4, line] * partials[4, line]) + (partials[4, 14 - line] * partials[4, 14 - line])) * divisions[line + 1]; + } + + costs[0] += partials[0, 7] * partials[0, 7] * divisions[8]; + costs[4] += partials[4, 7] * partials[4, 7] * divisions[8]; + for (int direction = 1; direction < 8; direction += 2) + { + for (int line = 0; line < 5; line++) + { + costs[direction] += partials[direction, 3 + line] * partials[direction, 3 + line]; + } + + costs[direction] *= divisions[8]; + for (int line = 0; line < 3; line++) + { + costs[direction] += ((partials[direction, line] * partials[direction, line]) + + (partials[direction, 10 - line] * partials[direction, 10 - line])) * divisions[(2 * line) + 2]; + } + } + + int bestCost = 0; + int bestDirection = 0; + for (int direction = 0; direction < 8; direction++) + { + if (costs[direction] > bestCost) + { + bestCost = costs[direction]; + bestDirection = direction; + } + } + + variance = (bestCost - costs[(bestDirection + 4) & 7]) >> 10; + return bestDirection; + } + + /// + /// Applies the independent scalar filter definition to eight-bit output storage. + /// + private static void FilterReference( + ReadOnlySpan source, + int sourceOffset, + int sourceStride, + Span destination, + int destinationOffset, + int destinationStride, + int primaryStrength, + int secondaryStrength, + int direction, + int damping, + int coefficientShift, + int blockWidth, + int blockHeight) + { + for (int row = 0; row < blockHeight; row++) + { + for (int column = 0; column < blockWidth; column++) + { + destination[destinationOffset + (row * destinationStride) + column] = (byte)FilterSampleReference( + source, + sourceOffset + (row * sourceStride) + column, + sourceStride, + primaryStrength, + secondaryStrength, + direction, + damping, + coefficientShift); + } + } + } + + /// + /// Applies the independent scalar filter definition to 16-bit output storage. + /// + private static void FilterReference( + ReadOnlySpan source, + int sourceOffset, + int sourceStride, + Span destination, + int destinationOffset, + int destinationStride, + int primaryStrength, + int secondaryStrength, + int direction, + int damping, + int coefficientShift, + int blockWidth, + int blockHeight) + { + for (int row = 0; row < blockHeight; row++) + { + for (int column = 0; column < blockWidth; column++) + { + destination[destinationOffset + (row * destinationStride) + column] = (ushort)FilterSampleReference( + source, + sourceOffset + (row * sourceStride) + column, + sourceStride, + primaryStrength, + secondaryStrength, + direction, + damping, + coefficientShift); + } + } + } + + /// + /// Computes one independently filtered sample from its primary and secondary neighbors. + /// + private static int FilterSampleReference( + ReadOnlySpan source, + int sourceIndex, + int sourceStride, + int primaryStrength, + int secondaryStrength, + int direction, + int damping, + int coefficientShift) + { + bool enablePrimary = primaryStrength != 0; + bool enableSecondary = secondaryStrength != 0; + bool clippingRequired = enablePrimary && enableSecondary; + int primaryTapSet = (primaryStrength >> coefficientShift) & 1; + int sample = source[sourceIndex]; + int sum = 0; + int minimum = sample; + int maximum = sample; + + for (int tap = 0; tap < 2; tap++) + { + if (enablePrimary) + { + int offset = GetDirectionOffsetReference(direction, tap, sourceStride); + int neighbor0 = source[sourceIndex + offset]; + int neighbor1 = source[sourceIndex - offset]; + int weight = primaryTapSet == 0 ? (tap == 0 ? 4 : 2) : 3; + sum += weight * ConstrainReference(neighbor0 - sample, primaryStrength, damping); + sum += weight * ConstrainReference(neighbor1 - sample, primaryStrength, damping); + + if (clippingRequired) + { + maximum = neighbor0 != Av1CdefFilter.VeryLarge ? Math.Max(maximum, neighbor0) : maximum; + maximum = neighbor1 != Av1CdefFilter.VeryLarge ? Math.Max(maximum, neighbor1) : maximum; + minimum = Math.Min(minimum, Math.Min(neighbor0, neighbor1)); + } + } + + if (enableSecondary) + { + int offset0 = GetDirectionOffsetReference((direction + 2) & 7, tap, sourceStride); + int offset1 = GetDirectionOffsetReference((direction + 6) & 7, tap, sourceStride); + int neighbor0 = source[sourceIndex + offset0]; + int neighbor1 = source[sourceIndex - offset0]; + int neighbor2 = source[sourceIndex + offset1]; + int neighbor3 = source[sourceIndex - offset1]; + int weight = tap == 0 ? 2 : 1; + sum += weight * ConstrainReference(neighbor0 - sample, secondaryStrength, damping); + sum += weight * ConstrainReference(neighbor1 - sample, secondaryStrength, damping); + sum += weight * ConstrainReference(neighbor2 - sample, secondaryStrength, damping); + sum += weight * ConstrainReference(neighbor3 - sample, secondaryStrength, damping); + + if (clippingRequired) + { + maximum = neighbor0 != Av1CdefFilter.VeryLarge ? Math.Max(maximum, neighbor0) : maximum; + maximum = neighbor1 != Av1CdefFilter.VeryLarge ? Math.Max(maximum, neighbor1) : maximum; + maximum = neighbor2 != Av1CdefFilter.VeryLarge ? Math.Max(maximum, neighbor2) : maximum; + maximum = neighbor3 != Av1CdefFilter.VeryLarge ? Math.Max(maximum, neighbor3) : maximum; + minimum = Math.Min(minimum, Math.Min(Math.Min(neighbor0, neighbor1), Math.Min(neighbor2, neighbor3))); + } + } + } + + int filtered = sample + ((8 + sum - (sum < 0 ? 1 : 0)) >> 4); + return clippingRequired ? Math.Clamp(filtered, minimum, maximum) : filtered; + } + + /// + /// Applies the scalar AV1 constrain equation independently of the production implementation. + /// + private static int ConstrainReference(int difference, int threshold, int damping) + { + int shift = Math.Max(0, damping - BitOperations.Log2((uint)threshold)); + int magnitude = Math.Abs(difference); + int constrained = Math.Clamp(threshold - (magnitude >> shift), 0, magnitude); + return difference < 0 ? -constrained : constrained; + } + + /// + /// Converts a direction and tap to a signed source offset independently of the production implementation. + /// + private static int GetDirectionOffsetReference(int direction, int tap, int stride) + { + int[] x0 = [1, 1, 1, 1, 1, 0, 0, 0]; + int[] y0 = [-1, 0, 0, 0, 1, 1, 1, 1]; + int[] x1 = [2, 2, 2, 2, 2, 1, 0, -1]; + int[] y1 = [-2, -1, 0, 1, 2, 2, 2, 2]; + return tap == 0 ? (y0[direction] * stride) + x0[direction] : (y1[direction] * stride) + x1[direction]; + } +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ChromaFromLumaTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ChromaFromLumaTests.cs new file mode 100644 index 0000000000..2d760e9659 --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ChromaFromLumaTests.cs @@ -0,0 +1,494 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Numerics; +using SixLabors.ImageSharp.Formats.Heif.Av1; +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.ChromaFromLuma; +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; +using SixLabors.ImageSharp.Tests.TestUtilities; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1; + +/// +/// Verifies AV1 chroma-from-luma storage, parameter derivation, and prediction. +/// +[Trait("Format", "Avif")] +public class Av1ChromaFromLumaTests +{ + /// + /// The hardware configurations required to exercise each SIMD tier and the complete scalar fallback. + /// + private const HwIntrinsics PredictorConfigurations = + HwIntrinsics.AllowAll | HwIntrinsics.DisableAVX512F | HwIntrinsics.DisableAVX | HwIntrinsics.DisableHWIntrinsic; + + /// + /// Verifies Q3 luma storage for every AV1 chroma-subsampling layout. + /// + [Theory] + [InlineData(false, false, new short[] { 8, 16, 24, 32, 40, 48, 56, 64, 72, 80, 88, 96, 104, 112, 120, 128 })] + [InlineData(true, false, new short[] { 12, 28, 44, 60, 76, 92, 108, 124 })] + [InlineData(true, true, new short[] { 28, 44, 92, 108 })] + public void Store8BitMatchesReference(bool subX, bool subY, short[] expected) + { + ObuColorConfig colorConfig = new() { SubSamplingX = subX, SubSamplingY = subY }; + Av1ChromaFromLumaContext context = new(colorConfig); + byte[] input = Enumerable.Range(1, 16).Select(x => (byte)x).ToArray(); + + context.Store(input, 4, 0, 0, Av1TransformSize.Size4x4, Av1BlockSize.Block4x4, 0, 0); + + int width = 4 >> (subX ? 1 : 0); + int height = 4 >> (subY ? 1 : 0); + Assert.Equal(expected, GetBlock(context.Q3Buffer, width, height)); + } + + /// + /// Verifies that Q3 storage retains the complete 12-bit sample range. + /// + [Fact] + public void StoreHighBitDepthPreservesTwelveBitQ3Range() + { + ObuColorConfig colorConfig = new(); + Av1ChromaFromLumaContext context = new(colorConfig); + short[] input = Enumerable.Repeat((short)4095, 16).ToArray(); + + context.Store(input, 4, 0, 0, Av1TransformSize.Size4x4, Av1BlockSize.Block4x4, 0, 0); + + Assert.All(GetBlock(context.Q3Buffer, 4, 4), value => Assert.Equal(32760, value)); + } + + /// + /// Verifies that sub-8-by-8 luma blocks are combined before the shared average is removed. + /// + [Fact] + public void StoreCombinesSub8x8LumaBeforeSubtractingAverage() + { + ObuColorConfig colorConfig = new() { SubSamplingX = true, SubSamplingY = true }; + Av1ChromaFromLumaContext context = new(colorConfig); + + context.Store(Enumerable.Repeat((byte)10, 16).ToArray(), 4, 0, 0, Av1TransformSize.Size4x4, Av1BlockSize.Block4x4, 0, 0); + context.Store(Enumerable.Repeat((byte)20, 16).ToArray(), 4, 0, 0, Av1TransformSize.Size4x4, Av1BlockSize.Block4x4, 0, 1); + context.Store(Enumerable.Repeat((byte)30, 16).ToArray(), 4, 0, 0, Av1TransformSize.Size4x4, Av1BlockSize.Block4x4, 1, 0); + context.Store(Enumerable.Repeat((byte)40, 16).ToArray(), 4, 0, 0, Av1TransformSize.Size4x4, Av1BlockSize.Block4x4, 1, 1); + + context.ComputeParameters(Av1TransformSize.Size4x4); + + short[] expected = + [ + -120, -120, -40, -40, + -120, -120, -40, -40, + 40, 40, 120, 120, + 40, 40, 120, 120 + ]; + + Assert.Equal(expected, GetBlock(context.Q3Buffer, 4, 4)); + } + + /// + /// Verifies that frame-edge extension precedes average subtraction. + /// + [Fact] + public void ComputeParametersPadsFrameEdgeBeforeSubtractingAverage() + { + ObuColorConfig colorConfig = new(); + Av1ChromaFromLumaContext context = new(colorConfig); + byte[] input = Enumerable.Range(1, 16).Select(x => (byte)x).ToArray(); + context.Store(input, 4, 0, 0, Av1TransformSize.Size4x4, Av1BlockSize.Block4x4, 0, 0); + + context.ComputeParameters(Av1TransformSize.Size8x8); + + short[] actual = GetBlock(context.Q3Buffer, 8, 8); + Assert.Equal(-90, actual[0]); + Assert.Equal(-66, actual[7]); + Assert.Equal(6, actual[56]); + Assert.Equal(30, actual[63]); + Assert.Equal(0, actual.Sum(x => x)); + } + + /// + /// Verifies 8-bit CfL scaling and clipping with known values. + /// + [Fact] + public void Predict8BitAddsScaledLumaAndClips() + { + short[] lumaQ3 = new short[32 * 32]; + new short[] { -64, -32, 64, 64 }.CopyTo(lumaQ3, 0); + byte[] destination = [128, 128, 128, 128]; + + Av1ChromaFromLumaPredictor.Predict(lumaQ3, destination, 4, 8, 4, 1); + + Assert.Equal(new byte[] { 120, 124, 136, 136 }, destination); + } + + /// + /// Verifies high-bit-depth CfL scaling and clipping with known values. + /// + [Theory] + [InlineData((int)Av1BitDepth.TenBit, 1023)] + [InlineData((int)Av1BitDepth.TwelveBit, 4095)] + public void PredictHighBitDepthAddsScaledLumaAndClips(int bitDepthIndex, short maximum) + { + short[] lumaQ3 = new short[32 * 32]; + new short[] { -128, -64, 64, 128 }.CopyTo(lumaQ3, 0); + short dc = (short)(maximum / 2); + short[] destination = [dc, dc, dc, dc]; + + Av1ChromaFromLumaPredictor.Predict(lumaQ3, destination, 4, 16, ((Av1BitDepth)bitDepthIndex).GetBitCount(), 4, 1); + + Assert.Equal(new short[] { (short)(dc - 32), (short)(dc - 16), (short)(dc + 16), (short)(dc + 32) }, destination); + } + + /// + /// Verifies exact 8-, 10-, and 12-bit CfL output and padding preservation across all intrinsic tiers. + /// + [Fact] + public void PredictMatchesReference() + => FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidatePredictors, PredictorConfigurations); + + /// + /// Verifies exact luma subsampling and average subtraction across all intrinsic tiers. + /// + [Fact] + public void ContextOperationsMatchReference() + => FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidateContextOperations, PredictorConfigurations); + + /// + /// Verifies the plane-specific alpha magnitude and joint-sign mapping. + /// + [Theory] + [InlineData((int)Av1Plane.U, 3)] + [InlineData((int)Av1Plane.V, 4)] + public void AlphaIndexSelectsMagnitudeForRequestedChromaPlane(int planeIndex, int expected) + { + // U occupies the high nibble and V occupies the low nibble in the packed AV1 alpha index. + const int alphaIndex = 0x23; + const int bothPositiveJointSign = 7; + + int actual = Av1PredictionDecoder.ChromaFromLumaIndexToAlpha( + alphaIndex, + bothPositiveJointSign, + (Av1Plane)planeIndex); + + Assert.Equal(expected, actual); + } + + /// + /// Exercises every valid CfL width, both alpha signs, clipping boundaries, and padded destination rows. + /// + private static void ValidatePredictors() + { + int[] widths = [4, 8, 16, 32]; + int[] alphaValues = [-16, -9, 0, 7, 16]; + foreach (int width in widths) + { + const int height = 8; + int stride = width + 5; + short[] lumaQ3 = CreateLumaSurface(height); + + foreach (int alphaQ3 in alphaValues) + { + byte[] expected = CreateByteDestination(stride, height, 137); + byte[] actual = (byte[])expected.Clone(); + ApplyReference(lumaQ3, expected, stride, alphaQ3, width, height); + Av1ChromaFromLumaPredictor.Predict(lumaQ3, actual, stride, alphaQ3, width, height); + Assert.Equal(expected, actual); + + foreach (int bitDepth in new[] { 10, 12 }) + { + short dc = (short)((1 << (bitDepth - 1)) + 53); + short[] expectedHigh = CreateHighBitDepthDestination(stride, height, dc); + short[] actualHigh = (short[])expectedHigh.Clone(); + ApplyReference(lumaQ3, expectedHigh, stride, alphaQ3, bitDepth, width, height); + Av1ChromaFromLumaPredictor.Predict(lumaQ3, actualHigh, stride, alphaQ3, bitDepth, width, height); + Assert.Equal(expectedHigh, actualHigh); + } + } + } + } + + /// + /// Exercises every CfL sampling layout, source width, sample precision, and padded minimum block extent. + /// + private static void ValidateContextOperations() + { + int[] sourceSizes = [4, 8, 16, 32]; + int[] samplingLayouts = [0, 1, 3]; + foreach (int sampling in samplingLayouts) + { + bool subX = (sampling & 1) != 0; + bool subY = (sampling & 2) != 0; + ObuColorConfig colorConfig = new() { SubSamplingX = subX, SubSamplingY = subY }; + + foreach (int sourceSize in sourceSizes) + { + int stride = sourceSize + 3; + Av1TransformSize sourceTransform = GetTransformSize(sourceSize, sourceSize); + int activeWidth = sourceSize >> (subX ? 1 : 0); + int activeHeight = sourceSize >> (subY ? 1 : 0); + int targetWidth = Math.Max(activeWidth, 4); + int targetHeight = Math.Max(activeHeight, 4); + Av1TransformSize targetTransform = GetTransformSize(targetWidth, targetHeight); + byte[] input = CreateByteInput(stride, sourceSize); + short[] expected = CreateStoredReference(input, stride, sourceSize, subX, subY); + PadAndSubtractAverage(expected, activeWidth, activeHeight, targetWidth, targetHeight); + Av1ChromaFromLumaContext context = new(colorConfig); + context.Store(input, stride, 0, 0, sourceTransform, sourceTransform.ToBlockSize(), 0, 0); + context.ComputeParameters(targetTransform); + Assert.Equal(expected, context.Q3Buffer.ToArray()); + + short[] highInput = CreateHighBitDepthInput(stride, sourceSize); + expected = CreateStoredReference(highInput, stride, sourceSize, subX, subY); + PadAndSubtractAverage(expected, activeWidth, activeHeight, targetWidth, targetHeight); + context = new Av1ChromaFromLumaContext(colorConfig); + context.Store(highInput, stride, 0, 0, sourceTransform, sourceTransform.ToBlockSize(), 0, 0); + context.ComputeParameters(targetTransform); + Assert.Equal(expected, context.Q3Buffer.ToArray()); + } + } + } + + /// + /// Maps dimensions used by the CfL operation matrix to their AV1 transform identifier. + /// + private static Av1TransformSize GetTransformSize(int width, int height) + => (width, height) switch + { + (4, 4) => Av1TransformSize.Size4x4, + (8, 8) => Av1TransformSize.Size8x8, + (16, 16) => Av1TransformSize.Size16x16, + (32, 32) => Av1TransformSize.Size32x32, + (4, 8) => Av1TransformSize.Size4x8, + (8, 4) => Av1TransformSize.Size8x4, + (8, 16) => Av1TransformSize.Size8x16, + (16, 8) => Av1TransformSize.Size16x8, + (16, 32) => Av1TransformSize.Size16x32, + (32, 16) => Av1TransformSize.Size32x16, + _ => throw new InvalidOperationException(), + }; + + /// + /// Creates deterministic 8-bit luma rows with padding that must not contribute to CfL. + /// + private static byte[] CreateByteInput(int stride, int height) + { + byte[] result = Enumerable.Repeat((byte)251, stride * height).ToArray(); + for (int row = 0; row < height; row++) + { + for (int column = 0; column < stride - 3; column++) + { + result[(row * stride) + column] = (byte)(((row * 67) + (column * 29) + 11) & byte.MaxValue); + } + } + + return result; + } + + /// + /// Creates deterministic 12-bit luma rows with padding that must not contribute to CfL. + /// + private static short[] CreateHighBitDepthInput(int stride, int height) + { + short[] result = Enumerable.Repeat((short)4095, stride * height).ToArray(); + for (int row = 0; row < height; row++) + { + for (int column = 0; column < stride - 3; column++) + { + result[(row * stride) + column] = (short)(((row * 977) + (column * 353) + 101) & 4095); + } + } + + return result; + } + + /// + /// Produces the normative Q3 luma surface for 8-bit input. + /// + private static short[] CreateStoredReference(byte[] input, int stride, int size, bool subX, bool subY) + { + short[] result = new short[32 * 32]; + int rowStep = subY ? 2 : 1; + int columnStep = subX ? 2 : 1; + int shift = subX ? (subY ? 1 : 2) : 3; + for (int row = 0; row < size; row += rowStep) + { + for (int column = 0; column < size; column += columnStep) + { + int sum = input[(row * stride) + column]; + if (subX) + { + sum += input[(row * stride) + column + 1]; + } + + if (subY) + { + sum += input[((row + 1) * stride) + column] + input[((row + 1) * stride) + column + 1]; + } + + result[((row / rowStep) * 32) + (column / columnStep)] = (short)(sum << shift); + } + } + + return result; + } + + /// + /// Produces the normative Q3 luma surface for high-bit-depth input. + /// + private static short[] CreateStoredReference(short[] input, int stride, int size, bool subX, bool subY) + { + short[] result = new short[32 * 32]; + int rowStep = subY ? 2 : 1; + int columnStep = subX ? 2 : 1; + int shift = subX ? (subY ? 1 : 2) : 3; + for (int row = 0; row < size; row += rowStep) + { + for (int column = 0; column < size; column += columnStep) + { + int sum = input[(row * stride) + column]; + if (subX) + { + sum += input[(row * stride) + column + 1]; + } + + if (subY) + { + sum += input[((row + 1) * stride) + column] + input[((row + 1) * stride) + column + 1]; + } + + result[((row / rowStep) * 32) + (column / columnStep)] = (short)(sum << shift); + } + } + + return result; + } + + /// + /// Applies CfL edge extension and rounded average subtraction to an independently stored Q3 surface. + /// + private static void PadAndSubtractAverage(short[] buffer, int activeWidth, int activeHeight, int width, int height) + { + for (int row = 0; row < activeHeight; row++) + { + buffer.AsSpan((row * 32) + activeWidth, width - activeWidth).Fill(buffer[(row * 32) + activeWidth - 1]); + } + + for (int row = activeHeight; row < height; row++) + { + buffer.AsSpan((row - 1) * 32, width).CopyTo(buffer.AsSpan(row * 32, width)); + } + + int sum = (width * height) >> 1; + for (int row = 0; row < height; row++) + { + for (int column = 0; column < width; column++) + { + sum += buffer[(row * 32) + column]; + } + } + + short average = (short)(sum >> (BitOperations.Log2((uint)width) + BitOperations.Log2((uint)height))); + for (int row = 0; row < height; row++) + { + for (int column = 0; column < width; column++) + { + buffer[(row * 32) + column] -= average; + } + } + } + + /// + /// Creates a deterministic Q3 surface spanning the legal signed 12-bit CfL range. + /// + private static short[] CreateLumaSurface(int height) + { + short[] result = new short[32 * height]; + for (int row = 0; row < height; row++) + { + for (int column = 0; column < 32; column++) + { + result[(row * 32) + column] = (short)((((row * 7919) + (column * 4051)) % 65521) - 32760); + } + } + + return result; + } + + /// + /// Creates an 8-bit DC prediction with non-image padding sentinels. + /// + private static byte[] CreateByteDestination(int stride, int height, byte dc) + { + byte[] result = Enumerable.Repeat((byte)203, stride * height).ToArray(); + for (int row = 0; row < height; row++) + { + result.AsSpan(row * stride, stride - 5).Fill(dc); + } + + return result; + } + + /// + /// Creates a high-bit-depth DC prediction with non-image padding sentinels. + /// + private static short[] CreateHighBitDepthDestination(int stride, int height, short dc) + { + short[] result = Enumerable.Repeat((short)-1, stride * height).ToArray(); + for (int row = 0; row < height; row++) + { + result.AsSpan(row * stride, stride - 5).Fill(dc); + } + + return result; + } + + /// + /// Applies the AV1 signed Q3 rounding definition to an 8-bit destination. + /// + private static void ApplyReference(short[] lumaQ3, byte[] destination, int stride, int alphaQ3, int width, int height) + { + int dc = destination[0]; + for (int row = 0; row < height; row++) + { + for (int column = 0; column < width; column++) + { + int product = alphaQ3 * lumaQ3[(row * 32) + column]; + int scaled = product < 0 ? -((-product + 32) >> 6) : (product + 32) >> 6; + destination[(row * stride) + column] = (byte)Math.Clamp(dc + scaled, byte.MinValue, byte.MaxValue); + } + } + } + + /// + /// Applies the AV1 signed Q3 rounding definition to a high-bit-depth destination. + /// + private static void ApplyReference(short[] lumaQ3, short[] destination, int stride, int alphaQ3, int bitDepth, int width, int height) + { + int dc = destination[0]; + int maximum = (1 << bitDepth) - 1; + for (int row = 0; row < height; row++) + { + for (int column = 0; column < width; column++) + { + int product = alphaQ3 * lumaQ3[(row * 32) + column]; + int scaled = product < 0 ? -((-product + 32) >> 6) : (product + 32) >> 6; + destination[(row * stride) + column] = (short)Math.Clamp(dc + scaled, 0, maximum); + } + } + } + + /// + /// Extracts the active rows from the fixed-stride CfL buffer. + /// + private static short[] GetBlock(ReadOnlySpan buffer, int width, int height) + { + short[] result = new short[width * height]; + for (int y = 0; y < height; y++) + { + buffer.Slice(y * 32, width).CopyTo(result.AsSpan(y * width, width)); + } + + return result; + } +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1CoefficientsEntropyTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1CoefficientsEntropyTests.cs new file mode 100644 index 0000000000..f304fe83cd --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1CoefficientsEntropyTests.cs @@ -0,0 +1,1624 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Buffers; +using System.Runtime.CompilerServices; +using SixLabors.ImageSharp.Formats.Heif.Av1; +using SixLabors.ImageSharp.Formats.Heif.Av1.Entropy; +using SixLabors.ImageSharp.Formats.Heif.Av1.Motion; +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.Quantizers; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; +using SixLabors.ImageSharp.Memory; +using SixLabors.ImageSharp.Tests.Memory; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1; + +[Trait("Format", "Avif")] +public class Av1CoefficientsEntropyTests +{ + private const int BaseQIndex = 23; + + // These tests encode at most one 8x8 transform with generated magnitudes no greater than 64. + private const int CoefficientSyntaxBufferLength = 256; + + [Fact] + public void NeighborArrayWritesEveryCoveredFourByFourEdgeUnit() + { + using Av1NeighborArrayUnit neighbors = new( + Configuration.Default, + leftSize: 8, + topSize: 8) + { + GranularityNormalLog2 = 2 + }; + + neighbors.UnitModeWrite( + 37, + new Point(8, 4), + new Size(16, 8), + Av1NeighborArrayUnit.UnitMask.Top | Av1NeighborArrayUnit.UnitMask.Left); + + Assert.Equal(new byte[] { 0, 0, 37, 37, 37, 37, 0, 0 }, neighbors.Top.ToArray()); + Assert.Equal(new byte[] { 0, 37, 37, 0, 0, 0, 0, 0 }, neighbors.Left.ToArray()); + } + + [Fact] + public void NeighborArrayOwnsOnlyLeftAndTopContexts() + { + TestMemoryAllocator allocator = new(); + allocator.EnableNonThreadSafeLogging(); + Configuration configuration = Configuration.Default.Clone(); + configuration.MemoryAllocator = allocator; + + TestMemoryAllocator.AllocationRequest allocation; + using (Av1NeighborArrayUnit neighbors = new(configuration, leftSize: 8, topSize: 12) + { + GranularityNormalLog2 = Av1Constants.ModeInfoSizeLog2 + }) + { + allocation = Assert.Single(allocator.AllocationLog); + Assert.Empty(allocator.ReturnLog); + Assert.Equal(20, allocation.Length); + Assert.Equal(AllocationOptions.Clean, allocation.AllocationOptions); + Assert.Equal(8, neighbors.Left.Length); + Assert.Equal(12, neighbors.Top.Length); + } + + TestMemoryAllocator.ReturnRequest returned = Assert.Single(allocator.ReturnLog); + Assert.Equal(allocation.AllocationId, returned.AllocationId); + } + + [Theory] + [InlineData((int)Av1ComponentType.Luminance, 5)] + [InlineData((int)Av1ComponentType.Chroma, 12)] + public void WriterDerivesTransformContextFromCompleteFourByFourEdges( + int componentType, + int expectedSkipContext) + { + using Av1NeighborArrayUnit neighbors = new( + Configuration.Default, + leftSize: 8, + topSize: 8) + { + GranularityNormalLog2 = 2 + }; + + // The high bits carry positive, positive, and negative DC signs. The low bits select + // the high-above and low-left coefficient classes used by the luma skip-context table. + neighbors.Top[2] = (2 << Av1Constants.CoefficientContextBitCount) | 4; + neighbors.Top[3] = 2 << Av1Constants.CoefficientContextBitCount; + neighbors.Left[1] = (1 << Av1Constants.CoefficientContextBitCount) | 1; + Av1TransformBlockContext context = Av1TileWriter.GetTransformBlockContexts( + (Av1ComponentType)componentType, + neighbors, + new Point(8, 4), + Av1BlockSize.Block16x16, + Av1TransformSize.Size8x8); + + Assert.Equal(2, context.DcSignContext); + Assert.Equal(expectedSkipContext, context.SkipContext); + } + + [Theory] + [InlineData(false, 2, 1, 6, 6)] + [InlineData(true, 2, 2, 10, 3)] + public void PictureControlSetMapsModeInfoAllocationByIndex( + bool disallow4x4, + int column, + int row, + int gridOffset, + int allocationOffset) + { + Av1MacroBlockModeInfo expected = CreateModeInfo(Av1PredictionMode.Paeth); + Av1MacroBlockModeInfo[] allocation = new Av1MacroBlockModeInfo[16]; + allocation[allocationOffset] = expected; + int[] grid = new int[16]; + Av1PictureControlSet picture = new() + { + PartitionContexts = [], + LuminanceDcSignLevelCoefficientNeighbors = [], + CrDcSignLevelCoefficientNeighbors = [], + CbDcSignLevelCoefficientNeighbors = [], + TransformFunctionContexts = [], + Sequence = new Av1SequenceControlSet { SequenceHeader = new ObuSequenceHeader() }, + Parent = new Av1PictureParentControlSet + { + Common = new Av1EncoderCommon + { + ModeInfoColumnCount = 4, + ModeInfoRowCount = 4, + ModeInfoStride = 4, + FrameSize = new ObuFrameSize(), + TilesInfo = new ObuTileGroupHeader() + }, + FrameHeader = new ObuFrameHeader(), + PreviousQIndex = Memory.Empty + }, + SegmentationNeighborMap = Memory.Empty, + ModeInfoGrid = grid, + ModeInfoAllocation = allocation, + ModeInfoStride = 4, + Disallow4x4AllFrames = disallow4x4, + CdefPreset = Memory.Empty, + TileDataOffsets = Memory.Empty, + TileDataLengths = Memory.Empty + }; + + Point position = new(column, row); + ref Av1MacroBlockModeInfo result = ref picture.GetMacroBlockModeInfo(position); + result.Block.Mode = Av1PredictionMode.Smooth; + picture.MapModeInfoBlock(position, Av1BlockSize.Block8x8); + + Assert.Equal(Av1PredictionMode.Smooth, allocation[allocationOffset].Block.Mode); + Assert.Equal(allocationOffset, grid[gridOffset]); + Assert.Equal(allocationOffset, grid[gridOffset + 1]); + Assert.Equal(allocationOffset, grid[gridOffset + 4]); + Assert.Equal(allocationOffset, grid[gridOffset + 5]); + } + + [Fact] + public void MacroBlockReadsNeighborsRelativeToCurrentGridEntry() + { + Av1MacroBlockModeInfo[] allocation = + [ + CreateModeInfo(Av1PredictionMode.Vertical), + CreateModeInfo(Av1PredictionMode.Horizontal), + CreateModeInfo(Av1PredictionMode.DC) + ]; + + int[] grid = new int[9]; + grid[1] = 0; + grid[3] = 1; + grid[4] = 2; + Av1MacroBlockD macroBlock = CreateMacroBlock(); + macroBlock.SetModeInfoGrid(grid, allocation, 4); + + Assert.Equal(Av1PredictionMode.Horizontal, macroBlock.GetRelativeModeInfo(-1).Block.Mode); + Assert.Equal(Av1PredictionMode.Vertical, macroBlock.GetRelativeModeInfo(-3).Block.Mode); + Assert.Equal(Av1PredictionMode.DC, macroBlock.GetRelativeModeInfo(0).Block.Mode); + } + + [Fact] + public void EncoderBlockModeInfoStoresSelectedSyntax() + { + Av1EncoderBlockModeInfo modeInfo = default; + + Assert.False(modeInfo.Skip); + Assert.False(modeInfo.SkipMode); + Assert.False(modeInfo.UseIntraBlockCopy); + + modeInfo.Skip = true; + modeInfo.SkipMode = true; + modeInfo.UseIntraBlockCopy = true; + modeInfo.BlockSize = Av1BlockSize.Block16x16; + modeInfo.PartitionType = Av1PartitionType.Split; + modeInfo.SegmentId = 3; + modeInfo.Mode = Av1PredictionMode.Smooth; + modeInfo.UvMode = Av1ChromaPredictionMode.Smooth; + + Assert.True(modeInfo.Skip); + Assert.True(modeInfo.SkipMode); + Assert.True(modeInfo.UseIntraBlockCopy); + Assert.Equal(Av1BlockSize.Block16x16, modeInfo.BlockSize); + Assert.Equal(Av1PartitionType.Split, modeInfo.PartitionType); + Assert.Equal(3, modeInfo.SegmentId); + Assert.Equal(Av1PredictionMode.Smooth, modeInfo.Mode); + Assert.Equal(Av1ChromaPredictionMode.Smooth, modeInfo.UvMode); + + modeInfo.SkipMode = false; + + Assert.True(modeInfo.Skip); + Assert.False(modeInfo.SkipMode); + Assert.True(modeInfo.UseIntraBlockCopy); + } + + [Fact] + public void EncoderModeInfoUsesPackedValueStorage() + { + Assert.Equal(7, Unsafe.SizeOf()); + Assert.Equal(8, Unsafe.SizeOf()); + } + + /// + /// Verifies that segment, reference, filter, and flag updates preserve adjacent packed values. + /// + [Fact] + public void EncoderBlockModeInfoPackedFieldsRemainIndependent() + { + for (int segment = 0; segment < 8; segment++) + { + for (int reference = 0; reference < 8; reference++) + { + for (int vertical = 0; vertical < 4; vertical++) + { + for (int horizontal = 0; horizontal < 4; horizontal++) + { + Av1EncoderBlockModeInfo modeInfo = new() + { + Skip = true, + SkipMode = true, + UseIntraBlockCopy = true, + SegmentId = segment, + ReferenceFrame = (Av1ReferenceFrameType)reference, + VerticalInterpolationFilter = (Av1InterpolationFilter)vertical, + HorizontalInterpolationFilter = (Av1InterpolationFilter)horizontal + }; + + Assert.Equal(segment, modeInfo.SegmentId); + Assert.Equal((Av1ReferenceFrameType)reference, modeInfo.ReferenceFrame); + Assert.Equal((Av1InterpolationFilter)vertical, modeInfo.VerticalInterpolationFilter); + Assert.Equal((Av1InterpolationFilter)horizontal, modeInfo.HorizontalInterpolationFilter); + Assert.True(modeInfo.Skip); + Assert.True(modeInfo.SkipMode); + Assert.True(modeInfo.UseIntraBlockCopy); + + // Overwrite every bit in each shared region after the adjacent value has been populated. + modeInfo.SegmentId = segment ^ 7; + Assert.Equal((Av1ReferenceFrameType)reference, modeInfo.ReferenceFrame); + modeInfo.ReferenceFrame = (Av1ReferenceFrameType)(reference ^ 7); + Assert.Equal(segment ^ 7, modeInfo.SegmentId); + modeInfo.Skip = false; + modeInfo.SkipMode = false; + modeInfo.UseIntraBlockCopy = false; + Assert.Equal((Av1InterpolationFilter)vertical, modeInfo.VerticalInterpolationFilter); + Assert.Equal((Av1InterpolationFilter)horizontal, modeInfo.HorizontalInterpolationFilter); + modeInfo.VerticalInterpolationFilter = (Av1InterpolationFilter)(vertical ^ 3); + modeInfo.HorizontalInterpolationFilter = (Av1InterpolationFilter)(horizontal ^ 3); + Assert.False(modeInfo.Skip); + Assert.False(modeInfo.SkipMode); + Assert.False(modeInfo.UseIntraBlockCopy); + } + } + } + } + } + + /// + /// Verifies both filter directions for matching references, mismatches, and unavailable tile neighbors. + /// + [Theory] + [InlineData(false, false)] + [InlineData(false, true)] + [InlineData(true, false)] + [InlineData(true, true)] + public void EncoderInterpolationContextUsesTileNeighbors(bool aboveAvailable, bool leftAvailable) + { + // Rows are the above filter and columns the left filter; index three means a nonmatching reference. + ReadOnlySpan expectedContexts = [0, 3, 3, 0, 3, 1, 3, 1, 3, 3, 2, 2, 0, 1, 2, 3]; + Av1MacroBlockModeInfo[] allocation = new Av1MacroBlockModeInfo[3]; + int[] grid = new int[9]; + grid[1] = 0; + grid[3] = 1; + grid[4] = 2; + Av1MacroBlockD macroBlock = CreateMacroBlock(); + macroBlock.ModeInfoStride = 3; + macroBlock.IsUpAvailable = aboveAvailable; + macroBlock.IsLeftAvailable = leftAvailable; + macroBlock.SetModeInfoGrid(grid, allocation, 4); + Av1EncoderBlockModeInfo current = new() { ReferenceFrame = Av1ReferenceFrameType.Last }; + for (int above = 0; above < 4; above++) + { + for (int left = 0; left < 4; left++) + { + allocation[0].Block.ReferenceFrame = above == 3 ? Av1ReferenceFrameType.Golden : Av1ReferenceFrameType.Last; + allocation[0].Block.VerticalInterpolationFilter = (Av1InterpolationFilter)(above % 3); + allocation[0].Block.HorizontalInterpolationFilter = (Av1InterpolationFilter)((above + 1) % 3); + allocation[1].Block.ReferenceFrame = left == 3 ? Av1ReferenceFrameType.Golden : Av1ReferenceFrameType.Last; + allocation[1].Block.VerticalInterpolationFilter = (Av1InterpolationFilter)(left % 3); + allocation[1].Block.HorizontalInterpolationFilter = (Av1InterpolationFilter)((left + 1) % 3); + int aboveVertical = aboveAvailable ? above : 3; + int leftVertical = leftAvailable ? left : 3; + int aboveHorizontal = aboveVertical == 3 ? 3 : (above + 1) % 3; + int leftHorizontal = leftVertical == 3 ? 3 : (left + 1) % 3; + + Assert.Equal(expectedContexts[(aboveVertical * 4) + leftVertical], Av1SymbolContextHelper.GetSwitchableInterpolationContext(current, macroBlock, 0)); + Assert.Equal(8 + expectedContexts[(aboveHorizontal * 4) + leftHorizontal], Av1SymbolContextHelper.GetSwitchableInterpolationContext(current, macroBlock, 1)); + } + } + } + + /// + /// Verifies which inter modes signal filters and distinguishes residual skip from compound skip mode. + /// + [Theory] + [InlineData(Av1GlobalMotionType.Identity, Av1PredictionMode.GlobalMotionVector, false)] + [InlineData(Av1GlobalMotionType.Translation, Av1PredictionMode.GlobalMotionVector, true)] + [InlineData(Av1GlobalMotionType.RotationZoom, Av1PredictionMode.GlobalMotionVector, false)] + [InlineData(Av1GlobalMotionType.Affine, Av1PredictionMode.GlobalMotionVector, false)] + [InlineData(Av1GlobalMotionType.Identity, Av1PredictionMode.NewMotionVector, true)] + [InlineData(Av1GlobalMotionType.Identity, Av1PredictionMode.NearestMotionVector, true)] + public void EncoderInterpolationSyntaxMatchesModeEligibility(int globalType, int predictionMode, bool expected) + { + ObuFrameHeader frameHeader = new() { InterpolationFilter = Av1InterpolationFilter.Switchable }; + frameHeader.GetGlobalMotionParameters()[0].Type = (Av1GlobalMotionType)globalType; + Av1EncoderBlockModeInfo modeInfo = new() + { + BlockSize = Av1BlockSize.Block8x8, + ReferenceFrame = Av1ReferenceFrameType.Last, + Mode = (Av1PredictionMode)predictionMode + }; + + Assert.Equal(expected, Av1TileWriter.UsesSwitchableInterpolation(frameHeader, modeInfo)); + modeInfo.Skip = true; + Assert.Equal(expected, Av1TileWriter.UsesSwitchableInterpolation(frameHeader, modeInfo)); + modeInfo.BlockSize = Av1BlockSize.Block4x8; + Assert.True(Av1TileWriter.UsesSwitchableInterpolation(frameHeader, modeInfo)); + modeInfo.SkipMode = true; + Assert.False(Av1TileWriter.UsesSwitchableInterpolation(frameHeader, modeInfo)); + modeInfo.SkipMode = false; + frameHeader.InterpolationFilter = Av1InterpolationFilter.Regular; + Assert.False(Av1TileWriter.UsesSwitchableInterpolation(frameHeader, modeInfo)); + } + + [Fact] + public void EncoderSuperblockWorkspaceUsesOneExactSizeOwner() + { + TestMemoryAllocator allocator = new(); + allocator.EnableNonThreadSafeLogging(); + Configuration configuration = Configuration.Default.Clone(); + configuration.MemoryAllocator = allocator; + + TestMemoryAllocator.AllocationRequest allocation; + using (Av1EncoderSuperblockWorkspace workspace = new(configuration)) + { + allocation = Assert.Single(allocator.AllocationLog); + Assert.Empty(allocator.ReturnLog); + Assert.Equal(typeof(byte), allocation.ElementType); + Assert.Equal(AllocationOptions.None, allocation.AllocationOptions); + Assert.Equal(Av1EncoderSuperblockWorkspace.StorageByteLength, allocation.Length); + Assert.Equal(Av1EncoderSuperblockWorkspace.MaximumFinalBlockCount, workspace.FinalBlocks.Length); + Assert.Equal(Av1EncoderSuperblockWorkspace.MaximumPartitionCount, workspace.PartitionTypes.Length); + Assert.Equal(Av1EncoderBlockStruct.StorageSize, Unsafe.SizeOf()); + Assert.Equal(Av1EncoderPaletteInfo.StorageSize, Unsafe.SizeOf()); + Assert.Equal(0, workspace.PaletteInfo.PaletteSizes[0]); + Assert.Equal(0, workspace.FinalBlocks[^1].QuantizationIndex); + Assert.Equal(Av1FilterIntraMode.AllFilterIntraModes, workspace.FinalBlocks[0].FilterIntraMode); + Assert.Equal(Av1FilterIntraMode.AllFilterIntraModes, workspace.FinalBlocks[^1].FilterIntraMode); + Assert.Equal(0, workspace.PartitionTypes[^1]); + + workspace.PaletteInfo.PaletteSizes[0] = 7; + workspace.FinalBlocks[0].FilterIntraMode = Av1FilterIntraMode.DC; + workspace.FinalBlocks[^1].QuantizationIndex = 255; + workspace.FinalBlocks[^1].FilterIntraMode = Av1FilterIntraMode.Paeth; + workspace.PartitionTypes.Fill(byte.MaxValue); + workspace.Reset(); + + Assert.Equal(0, workspace.PaletteInfo.PaletteSizes[0]); + Assert.Equal(0, workspace.FinalBlocks[^1].QuantizationIndex); + Assert.Equal(Av1FilterIntraMode.AllFilterIntraModes, workspace.FinalBlocks[0].FilterIntraMode); + Assert.Equal(Av1FilterIntraMode.AllFilterIntraModes, workspace.FinalBlocks[^1].FilterIntraMode); + for (int index = 0; index < workspace.PartitionTypes.Length; index++) + { + Assert.Equal(0, workspace.PartitionTypes[index]); + } + } + + TestMemoryAllocator.ReturnRequest returned = Assert.Single(allocator.ReturnLog); + Assert.Equal(allocation.AllocationId, returned.AllocationId); + } + + [Fact] + public void EncoderBlocksKeepInlineModeStateWithoutPerBlockAllocations() + { + Av1EncoderBlockStruct[] blocks = new Av1EncoderBlockStruct[2]; + Av1EncoderPaletteInfo[] palettes = new Av1EncoderPaletteInfo[2]; + + // Exercise the inline-array accessors before measuring so one-time runtime generic initialization is + // excluded from the steady-state allocation contract used for every encoded block. + ref Av1EncoderBlockStruct warmupBlock = ref blocks[0]; + ref Av1EncoderPaletteInfo warmupPalette = ref palettes[0]; + warmupPalette.PaletteSizes[0] = 1; + warmupBlock.PredictionUnit.AngleDelta[(int)Av1PlaneType.Y] = 1; + + long before = GC.GetAllocatedBytesForCurrentThread(); + ref Av1EncoderBlockStruct block = ref blocks[1]; + ref Av1EncoderPaletteInfo palette = ref palettes[1]; + palette.PaletteSizes[0] = 3; + palette.PaletteSizes[1] = 5; + block.PredictionUnit.AngleDelta[(int)Av1PlaneType.Y] = -2; + block.PredictionUnit.AngleDelta[(int)Av1PlaneType.Uv] = 3; + long allocated = GC.GetAllocatedBytesForCurrentThread() - before; + + Assert.Equal(3, palettes[1].PaletteSizes[0]); + Assert.Equal(5, palettes[1].PaletteSizes[1]); + Assert.Equal(-2, blocks[1].PredictionUnit.AngleDelta[(int)Av1PlaneType.Y]); + Assert.Equal(3, blocks[1].PredictionUnit.AngleDelta[(int)Av1PlaneType.Uv]); + Assert.Equal(0, allocated); + } + + [Fact] + public void EncoderSuperblockWorkspaceExposesReusablePaletteMaps() + { + TestMemoryAllocator allocator = new(); + allocator.EnableNonThreadSafeLogging(); + Configuration configuration = Configuration.Default.Clone(); + configuration.MemoryAllocator = allocator; + + TestMemoryAllocator.AllocationRequest allocation; + using (Av1EncoderSuperblockWorkspace workspace = new(configuration)) + { + allocation = Assert.Single(allocator.AllocationLog); + Assert.Empty(allocator.ReturnLog); + + Av1EncoderPaletteMapBuffer maps = workspace.GetPaletteMaps(); + Assert.Same(maps, workspace.GetPaletteMaps()); + Assert.Single(allocator.AllocationLog); + + Buffer2DRegion luma = maps.GetMap(Av1PlaneType.Y, 64, 64); + Buffer2DRegion chroma = maps.GetMap(Av1PlaneType.Uv, 32, 32); + luma.DangerousGetRowSpan(0)[0] = 3; + chroma.DangerousGetRowSpan(0)[0] = 5; + Av1EncoderPaletteMapBuffer retainedMaps = workspace.GetPaletteMaps(); + Assert.Equal(3, retainedMaps.GetMap(Av1PlaneType.Y, 64, 64).DangerousGetRowSpan(0)[0]); + Assert.Equal(5, retainedMaps.GetMap(Av1PlaneType.Uv, 32, 32).DangerousGetRowSpan(0)[0]); + } + + TestMemoryAllocator.ReturnRequest returned = Assert.Single(allocator.ReturnLog); + Assert.Equal(allocation.AllocationId, returned.AllocationId); + } + + [Fact] + public void PaletteModeWriterMatchesColorCacheBoundaryAndRoundTrips() + { + const int Width = 16; + const int Height = 72; + const int BlockSizeContext = 0; + Point blockOrigin = new(8, 64); + ObuColorConfig colorConfig = new() + { + IsMonochrome = false, + SubSamplingX = false, + SubSamplingY = false, + BitDepth = Av1BitDepth.EightBit + }; + + ObuTileGroupHeader tiles = new() + { + TileColumnCount = 1, + TileRowCount = 1 + }; + + tiles.TileColumnStartModeInfo[1] = Width >> Av1Constants.ModeInfoSizeLog2; + tiles.TileRowStartModeInfo[1] = Height >> Av1Constants.ModeInfoSizeLog2; + ObuSequenceHeader sequenceHeader = new() + { + ColorConfig = colorConfig + }; + + ObuFrameHeader frameHeader = new() + { + AllowScreenContentTools = true, + ModeInfoColumnCount = Width >> Av1Constants.ModeInfoSizeLog2, + ModeInfoRowCount = Height >> Av1Constants.ModeInfoSizeLog2, + TilesInfo = tiles + }; + + using Av1EncoderPictureBuffer pictureBuffer = new( + Configuration.Default, + sequenceHeader, + frameHeader, + Width, + Height, + disallow4x4AllFrames: true); + + Av1PictureControlSet picture = pictureBuffer.Picture; + Av1NeighborArrayUnit paletteContexts = Assert.Single(picture.PaletteContexts); + ref Av1EncoderPaletteInfo above = ref paletteContexts.Top[paletteContexts.GetTopIndex(blockOrigin)]; + above.PaletteSizes[0] = 2; + above.PaletteSizes[1] = 2; + above.SetColors(Av1Plane.Y, [10, 30]); + above.SetColors(Av1Plane.U, [15, 35]); + ref Av1EncoderPaletteInfo left = ref paletteContexts.Left[paletteContexts.GetLeftIndex(blockOrigin)]; + left.PaletteSizes[0] = 2; + left.PaletteSizes[1] = 2; + left.SetColors(Av1Plane.Y, [20, 40]); + left.SetColors(Av1Plane.U, [25, 45]); + + Av1EncoderPaletteInfo current = default; + current.PaletteSizes[0] = 3; + current.PaletteSizes[1] = 3; + current.SetColors(Av1Plane.Y, [20, 50, 70]); + current.SetColors(Av1Plane.U, [25, 55, 80]); + current.SetColors(Av1Plane.V, [10, 12, 9]); + Av1MacroBlockModeInfo modeInfo = default; + modeInfo.Block = new Av1EncoderBlockModeInfo + { + BlockSize = Av1BlockSize.Block8x8, + Mode = Av1PredictionMode.DC, + UvMode = Av1ChromaPredictionMode.DC + }; + + Av1MacroBlockD macroBlock = new() + { + Tile = new Av1TileInfo(0, 0, frameHeader), + IsUpAvailable = true, + IsLeftAvailable = true + }; + + using Av1SymbolEncoder encoder = new(Configuration.Default, 128, BaseQIndex, updateCdf: true); + Av1TileWriter.WritePaletteModeInfo( + picture.Sequence, + picture, + encoder, + macroBlock, + modeInfo, + ref current, + Av1BlockSize.Block8x8, + blockOrigin, + tileIndex: 0, + hasChroma: true); + + using IMemoryOwner encoded = encoder.Exit(); + Av1SymbolDecoder decoder = new(Configuration.Default, encoded.GetSpan(), BaseQIndex); + Assert.True(decoder.ReadPaletteYMode(BlockSizeContext, neighborContext: 2)); + Assert.Equal(3, decoder.ReadPaletteSize(BlockSizeContext, Av1PlaneType.Y)); + Span decodedY = stackalloc ushort[3]; + decoder.ReadPaletteYColors([20, 40], 3, bitDepth: 8, decodedY); + Assert.Equal([20, 50, 70], decodedY.ToArray()); + Assert.True(decoder.ReadPaletteUvMode(hasLumaPalette: true)); + Assert.Equal(3, decoder.ReadPaletteSize(BlockSizeContext, Av1PlaneType.Uv)); + Span decodedU = stackalloc ushort[3]; + Span decodedV = stackalloc ushort[3]; + decoder.ReadPaletteUvColors([25, 45], 3, bitDepth: 8, decodedU, decodedV); + Assert.Equal([25, 55, 80], decodedU.ToArray()); + Assert.Equal([10, 12, 9], decodedV.ToArray()); + decoder.ValidateTrailingBits(); + } + + [Theory] + [InlineData(false, 6, 4096, 1024, 6144, 256, 64, 384, 36864L)] + [InlineData(true, 2, 16384, 4096, 24576, 1024, 256, 1536, 49152L)] + public void EncoderCoefficientBufferMatchesLibaom420SuperblockLayout( + bool use128x128Superblock, + int expectedSuperblockCount, + int expectedLumaCount, + int expectedChromaCount, + int expectedCoefficientsPerSuperblock, + int expectedLumaTransformBlockCount, + int expectedChromaTransformBlockCount, + int expectedTransformBlocksPerSuperblock, + long expectedTotalCoefficientCount) + { + ObuSequenceHeader sequenceHeader = new() { Use128x128Superblock = use128x128Superblock }; + sequenceHeader.ColorConfig.IsMonochrome = false; + sequenceHeader.ColorConfig.SubSamplingX = true; + sequenceHeader.ColorConfig.SubSamplingY = true; + + using Av1EncoderCoefficientBuffer coefficients = new( + Configuration.Default, + sequenceHeader, + width: 129, + height: 65); + + Assert.Equal(expectedSuperblockCount, coefficients.SuperblockCount); + Assert.Equal(expectedLumaCount, coefficients.LumaCoefficientCount); + Assert.Equal(expectedChromaCount, coefficients.ChromaCoefficientCount); + Assert.Equal(expectedCoefficientsPerSuperblock, coefficients.CoefficientsPerSuperblock); + Assert.Equal(expectedLumaTransformBlockCount, coefficients.LumaTransformBlockCount); + Assert.Equal(expectedChromaTransformBlockCount, coefficients.ChromaTransformBlockCount); + Assert.Equal(expectedTransformBlocksPerSuperblock, coefficients.TransformBlocksPerSuperblock); + Assert.Equal(expectedTotalCoefficientCount, coefficients.TotalCoefficientCount); + Assert.Equal(expectedLumaCount, coefficients.GetPlaneSpan(0, Av1Plane.Y).Length); + Assert.Equal(expectedChromaCount, coefficients.GetPlaneSpan(0, Av1Plane.U).Length); + Assert.Equal(expectedChromaCount, coefficients.GetPlaneSpan(0, Av1Plane.V).Length); + Assert.Equal(expectedLumaTransformBlockCount, coefficients.GetTransformBlockSpan(0, Av1Plane.Y).Length); + Assert.Equal(expectedChromaTransformBlockCount, coefficients.GetTransformBlockSpan(0, Av1Plane.U).Length); + Assert.Equal(expectedChromaTransformBlockCount, coefficients.GetTransformBlockSpan(0, Av1Plane.V).Length); + } + + [Fact] + public void EncoderCoefficientBufferKeepsEveryPlaneAndSuperblockDisjoint() + { + ObuSequenceHeader sequenceHeader = new() { Use128x128Superblock = true }; + sequenceHeader.ColorConfig.IsMonochrome = false; + sequenceHeader.ColorConfig.SubSamplingX = true; + sequenceHeader.ColorConfig.SubSamplingY = true; + + using Av1EncoderCoefficientBuffer coefficients = new( + Configuration.Default, + sequenceHeader, + width: 129, + height: 65); + + coefficients.GetPlaneSpan(0, Av1Plane.Y)[0] = 11; + coefficients.GetPlaneSpan(0, Av1Plane.U)[0] = 22; + coefficients.GetPlaneSpan(0, Av1Plane.V)[0] = 33; + coefficients.GetPlaneSpan(1, Av1Plane.Y)[0] = 44; + coefficients.GetTransformBlockSpan(0, Av1Plane.Y)[0].EndOfBlock = 55; + coefficients.GetTransformBlockSpan(0, Av1Plane.U)[0].EndOfBlock = 66; + coefficients.GetTransformBlockSpan(0, Av1Plane.V)[0].EndOfBlock = 77; + coefficients.GetTransformBlockSpan(1, Av1Plane.Y)[0].EndOfBlock = 88; + + Assert.Equal(11, coefficients.GetPlaneSpan(0, Av1Plane.Y)[0]); + Assert.Equal(22, coefficients.GetPlaneSpan(0, Av1Plane.U)[0]); + Assert.Equal(33, coefficients.GetPlaneSpan(0, Av1Plane.V)[0]); + Assert.Equal(44, coefficients.GetPlaneSpan(1, Av1Plane.Y)[0]); + Assert.Equal(55, coefficients.GetTransformBlockSpan(0, Av1Plane.Y)[0].EndOfBlock); + Assert.Equal(66, coefficients.GetTransformBlockSpan(0, Av1Plane.U)[0].EndOfBlock); + Assert.Equal(77, coefficients.GetTransformBlockSpan(0, Av1Plane.V)[0].EndOfBlock); + Assert.Equal(88, coefficients.GetTransformBlockSpan(1, Av1Plane.Y)[0].EndOfBlock); + } + + [Fact] + public void EncoderLumaTraversalRepresentsAllTransformsIn128x128Block() + { + Av1PictureControlSet picture = CreateEncoderPicture(32, 32, use128x128Superblock: true); + ref Av1MacroBlockModeInfo modeInfo = ref picture.ModeInfoAllocation.Span[0]; + modeInfo.Block.BlockSize = Av1BlockSize.Block128x128; + modeInfo.Block.TransformSize = Av1TransformSize.Size16x16; + modeInfo.Block.SegmentId = 0; + Av1TileInfo tile = new(0, 0, picture.Parent.FrameHeader); + Av1TileWriter.Av1EntropyCodingContext context = new() + { + MacroBlock = new Av1MacroBlockD { Tile = tile }, + MacroBlockModeInfo = modeInfo, + SuperblockOrigin = Point.Empty + }; + + using Av1NeighborArrayUnit luma = new( + Configuration.Default, + leftSize: 128, + topSize: 128) + { + GranularityNormalLog2 = Av1Constants.ModeInfoSizeLog2 + }; + + using Av1EncoderCoefficientBuffer coefficients = new( + Configuration.Default, + picture.Sequence.SequenceHeader, + width: 128, + height: 128); + + Span transformBlocks = + coefficients.GetTransformBlockSpan(0, Av1Plane.Y); + transformBlocks.Fill(new Av1EncoderTransformBlockState { TransformType = Av1TransformType.Identity }); + + Av1EncoderBlockStruct block = default; + using Av1SymbolEncoder writer = new(Configuration.Default, 4096, BaseQIndex, updateCdf: true); + Av1TileWriter.EncodeTransformCoefficientsY( + picture, + context, + writer, + ref block, + Point.Empty, + Av1PredictionMode.DC, + Av1BlockSize.Block128x128, + coefficients, + superblockIndex: 0, + luma); + + writer.Dispose(); + + int visitedTransformCount = 0; + for (int index = 0; index < transformBlocks.Length; index++) + { + if ((index % 16) == 0) + { + Assert.Equal(Av1TransformType.DctDct, transformBlocks[index].TransformType); + visitedTransformCount++; + } + else + { + Assert.Equal(Av1TransformType.Identity, transformBlocks[index].TransformType); + } + } + + Assert.Equal(64, visitedTransformCount); + Assert.Equal(16384, context.CodedAreaSuperblock); + } + + [Fact] + public void SegmentationUpdateUsesModeInfoUnits() + { + Av1PictureControlSet picture = CreateEncoderPicture(8, 8); + picture.SegmentationNeighborMap = new byte[64]; + + picture.UpdateSegmentation(Av1BlockSize.Block16x8, new Point(8, 12), segmentId: 5); + + for (int row = 0; row < 8; row++) + { + for (int column = 0; column < 8; column++) + { + byte expected = row is 3 or 4 && column >= 2 && column < 6 ? (byte)5 : (byte)0; + Assert.Equal(expected, picture.SegmentationNeighborMap.Span[(row * 8) + column]); + } + } + } + + [Fact] + public void TransformSizeContextUsesIntraBlockCopyNeighborExtents() + { + Av1PictureControlSet picture = CreateEncoderPicture(16, 16); + Point blockOrigin = new(16, 16); + Av1MacroBlockD macroBlock = new() + { + Tile = new Av1TileInfo(0, 0, picture.Parent.FrameHeader), + IsUpAvailable = true, + IsLeftAvailable = true + }; + + int modeInfoIndex = + ((blockOrigin.Y >> Av1Constants.ModeInfoSizeLog2) * picture.ModeInfoStride) + + (blockOrigin.X >> Av1Constants.ModeInfoSizeLog2); + + macroBlock.ModeInfoStride = picture.ModeInfoStride; + macroBlock.SetModeInfoGrid(picture.ModeInfoGrid, picture.ModeInfoAllocation, modeInfoIndex); + + ref Av1MacroBlockModeInfo aboveModeInfo = ref macroBlock.GetRelativeModeInfo(-macroBlock.ModeInfoStride); + aboveModeInfo.Block.BlockSize = Av1BlockSize.Block16x8; + aboveModeInfo.Block.UseIntraBlockCopy = true; + ref Av1MacroBlockModeInfo leftModeInfo = ref macroBlock.GetRelativeModeInfo(-1); + leftModeInfo.Block.BlockSize = Av1BlockSize.Block8x16; + leftModeInfo.Block.UseIntraBlockCopy = true; + + using Av1NeighborArrayUnit transforms = new( + Configuration.Default, + leftSize: 64, + topSize: 64) + { + GranularityNormalLog2 = Av1Constants.ModeInfoSizeLog2 + }; + + // Residual contexts report 8x8, but libaom derives 16x16 availability from the IBC coding blocks. + transforms.Top[transforms.GetTopIndex(blockOrigin)] = 8; + transforms.Left[transforms.GetLeftIndex(blockOrigin)] = 8; + + Assert.Equal( + 2, + Av1TileWriter.GetTransformSizeContext( + transforms, + macroBlock, + blockOrigin, + Av1BlockSize.Block16x16)); + } + + [Fact] + public void SelectedTransformSizeRoundTripsAndPublishesRectangularEdgeContexts() + { + Av1PictureControlSet picture = CreateEncoderPicture(16, 16); + picture.Parent.FrameHeader.TransformMode = Av1TransformMode.Select; + ref Av1MacroBlockModeInfo modeInfo = ref picture.ModeInfoAllocation.Span[0]; + modeInfo.Block.BlockSize = Av1BlockSize.Block16x32; + modeInfo.Block.TransformSize = Av1TransformSize.Size8x8; + modeInfo.Block.SegmentId = 0; + Point blockOrigin = new(16, 16); + Av1MacroBlockD macroBlock = new() + { + Tile = new Av1TileInfo(0, 0, picture.Parent.FrameHeader), + IsUpAvailable = true, + IsLeftAvailable = true + }; + + int modeInfoIndex = + ((blockOrigin.Y >> Av1Constants.ModeInfoSizeLog2) * picture.ModeInfoStride) + + (blockOrigin.X >> Av1Constants.ModeInfoSizeLog2); + + // Uniform-size context substitutes coding-block extents for inter neighbors, so mirror production mode-info setup. + macroBlock.ModeInfoStride = picture.ModeInfoStride; + macroBlock.SetModeInfoGrid(picture.ModeInfoGrid, picture.ModeInfoAllocation, modeInfoIndex); + + using Av1NeighborArrayUnit transforms = new( + Configuration.Default, + leftSize: 64, + topSize: 64) + { + GranularityNormalLog2 = Av1Constants.ModeInfoSizeLog2 + }; + + int topIndex = transforms.GetTopIndex(blockOrigin); + int leftIndex = transforms.GetLeftIndex(blockOrigin); + transforms.Top[topIndex] = 16; + transforms.Left[leftIndex] = 16; + picture.TransformFunctionContexts = [transforms]; + + using Av1SymbolEncoder writer = new(Configuration.Default, 64, BaseQIndex, updateCdf: true); + Av1TileWriter.WriteTransformSize( + picture, + writer, + ref modeInfo, + macroBlock, + modeInfo.Block.BlockSize, + blockOrigin, + tileIndex: 0); + + using IMemoryOwner encoded = writer.Exit(); + writer.Dispose(); + + Av1SymbolDecoder reader = new(Configuration.Default, encoded.GetSpan(), BaseQIndex); + Assert.Equal( + Av1TransformSize.Size8x8, + reader.ReadTransformSize(Av1BlockSize.Block16x32, context: 1)); + + for (int index = 0; index < transforms.Top.Length; index++) + { + byte expected = index >= topIndex && index < topIndex + 4 ? (byte)8 : (byte)0; + Assert.Equal(expected, transforms.Top[index]); + } + + for (int index = 0; index < transforms.Left.Length; index++) + { + byte expected = index >= leftIndex && index < leftIndex + 8 ? (byte)8 : (byte)0; + Assert.Equal(expected, transforms.Left[index]); + } + } + + [Theory] + [InlineData((int)Av1PartitionType.None, 24, 24)] + [InlineData((int)Av1PartitionType.Horizontal, 24, 28)] + [InlineData((int)Av1PartitionType.Vertical, 28, 24)] + [InlineData((int)Av1PartitionType.Split, 0, 0)] + [InlineData((int)Av1PartitionType.HorizontalA, 24, 28)] + [InlineData((int)Av1PartitionType.HorizontalB, 28, 28)] + [InlineData((int)Av1PartitionType.VerticalA, 28, 24)] + [InlineData((int)Av1PartitionType.VerticalB, 28, 28)] + [InlineData((int)Av1PartitionType.Horizontal4, 24, 30)] + [InlineData((int)Av1PartitionType.Vertical4, 30, 24)] + public void PartitionContextUpdatesMatchLibaomExtendedPartitionRules( + int partitionValue, + byte expectedAbove, + byte expectedLeft) + { + using Av1NeighborArrayUnit neighbors = new( + Configuration.Default, + leftSize: 16, + topSize: 16) + { + GranularityNormalLog2 = 2 + }; + + Av1PartitionType partition = (Av1PartitionType)partitionValue; + Av1BlockSize blockSize = Av1BlockSize.Block32x32; + Av1BlockSize subSize = partition.GetBlockSubSize(blockSize); + + Av1TileWriter.UpdatePartitionContexts( + neighbors, + new Point(8, 12), + subSize, + blockSize, + partition); + + for (int index = 0; index < 16; index++) + { + byte above = index is >= 2 and < 10 ? expectedAbove : (byte)0; + byte left = index is >= 3 and < 11 ? expectedLeft : (byte)0; + Assert.Equal(above, neighbors.Top[index].Above); + Assert.Equal(left, neighbors.Left[index].Left); + } + } + + [Fact] + public void EightByEightSplitPublishesFourByFourPartitionContexts() + { + using Av1NeighborArrayUnit neighbors = new( + Configuration.Default, + leftSize: 4, + topSize: 4) + { + GranularityNormalLog2 = 2 + }; + + Av1TileWriter.UpdatePartitionContexts( + neighbors, + new Point(4, 4), + Av1BlockSize.Block4x4, + Av1BlockSize.Block8x8, + Av1PartitionType.Split); + + Assert.Equal(31, neighbors.Top[1].Above); + Assert.Equal(31, neighbors.Top[2].Above); + Assert.Equal(31, neighbors.Left[1].Left); + Assert.Equal(31, neighbors.Left[2].Left); + } + + [Fact] + public void EncoderModeInfoEdgesUseFourByFourUnits() + { + Av1PictureControlSet picture = CreateEncoderPicture(6, 5); + Av1TileInfo tile = new(0, 0, picture.Parent.FrameHeader); + Av1MacroBlockD macroBlock = new() { Tile = tile }; + Point position = new(2, 3); + picture.GetMacroBlockModeInfo(position).Block.Mode = Av1PredictionMode.Paeth; + picture.MapModeInfoBlock(position, Av1BlockSize.Block16x8); + + Av1TileWriter.SetModeInfoRowAndColumn( + picture, + macroBlock, + tile, + position, + Av1BlockSize.Block16x8, + picture.ModeInfoStride, + picture.Parent.Common.ModeInfoRowCount, + picture.Parent.Common.ModeInfoColumnCount); + + Assert.Equal(-96, macroBlock.ToTopEdge); + Assert.Equal(0, macroBlock.ToBottomEdge); + Assert.Equal(-64, macroBlock.ToLeftEdge); + Assert.Equal(0, macroBlock.ToRightEdge); + Assert.Equal( + picture.GetFromModeInfoGrid(new Point(2, 2)).Block.Mode, + macroBlock.GetRelativeModeInfo(-picture.ModeInfoStride).Block.Mode); + Assert.Equal( + picture.GetFromModeInfoGrid(new Point(1, 3)).Block.Mode, + macroBlock.GetRelativeModeInfo(-1).Block.Mode); + + for (int row = 0; row < picture.Parent.Common.ModeInfoRowCount; row++) + { + for (int column = 0; column < picture.Parent.Common.ModeInfoColumnCount; column++) + { + Av1PredictionMode expected = row >= 3 && column >= 2 ? Av1PredictionMode.Paeth : Av1PredictionMode.DC; + Assert.Equal(expected, picture.GetFromModeInfoGrid(new Point(column, row)).Block.Mode); + } + } + } + + [Fact] + public void CdefUsesLibaomUnitIndexAndFirstBlockStrength() + { + Av1PictureControlSet picture = CreateEncoderPicture(32, 32, use128x128Superblock: true); + picture.Parent.FrameHeader.CdefParameters.BitCount = 2; + picture.ModeInfoAllocation.Span[16].CdefStrength = 3; + picture.ModeInfoAllocation.Span[20].CdefStrength = 1; + using Av1SymbolEncoder writer = new(Configuration.Default, 16, BaseQIndex, updateCdf: true); + + Av1TileWriter.WriteCdef( + picture.Sequence, + picture, + writer, + tileIndex: 0, + skip: false, + modeInfoPosition: new Point(20, 4)); + + Assert.True(picture.CdefPreset.Span.SequenceEqual([-1, 3, -1, -1])); + } + + [Fact] + public void SuperblockWriterTraversesSplitTreeFromAbsoluteOrigin() + { + Av1PictureControlSet picture = CreateEncoderPicture(32, 16); + picture.Sequence.SequenceHeader.ColorConfig.IsMonochrome = true; + picture.Parent.FrameHeader.CodedLossless = true; + using Av1NeighborArrayUnit partitions = new( + Configuration.Default, + leftSize: 16, + topSize: 32) + { + GranularityNormalLog2 = 2 + }; + + using Av1NeighborArrayUnit luma = new( + Configuration.Default, + leftSize: 16, + topSize: 32) + { + GranularityNormalLog2 = 2 + }; + + using Av1NeighborArrayUnit red = new( + Configuration.Default, + leftSize: 16, + topSize: 32) + { + GranularityNormalLog2 = 2 + }; + + using Av1NeighborArrayUnit blue = new( + Configuration.Default, + leftSize: 16, + topSize: 32) + { + GranularityNormalLog2 = 2 + }; + + using Av1NeighborArrayUnit transforms = new( + Configuration.Default, + leftSize: 16, + topSize: 32) + { + GranularityNormalLog2 = 2 + }; + + picture.PartitionContexts = [partitions]; + picture.LuminanceDcSignLevelCoefficientNeighbors = [luma]; + picture.CrDcSignLevelCoefficientNeighbors = [red]; + picture.CbDcSignLevelCoefficientNeighbors = [blue]; + picture.TransformFunctionContexts = [transforms]; + Av1TileInfo tile = new(0, 0, picture.Parent.FrameHeader); + Point[] modeInfoPositions = [new(16, 0), new(24, 0), new(16, 8), new(24, 8)]; + using Av1EncoderSuperblockWorkspace workspace = new(Configuration.Default); + for (int index = 0; index < modeInfoPositions.Length; index++) + { + Point position = modeInfoPositions[index]; + ref Av1EncoderBlockModeInfo blockMode = ref picture.ModeInfoAllocation.Span[ + (position.Y * picture.ModeInfoStride) + position.X].Block; + + blockMode.BlockSize = Av1BlockSize.Block32x32; + blockMode.Skip = true; + blockMode.Mode = Av1PredictionMode.DC; + blockMode.UvMode = Av1ChromaPredictionMode.DC; + workspace.FinalBlocks[index].HasChroma = false; + } + + ReadOnlySpan partitionTypes = + [ + (byte)Av1PartitionType.Split, + (byte)Av1PartitionType.None, + (byte)Av1PartitionType.None, + (byte)Av1PartitionType.None, + (byte)Av1PartitionType.None + ]; + + partitionTypes.CopyTo(workspace.PartitionTypes); + Av1Superblock superblock = new() + { + Workspace = workspace, + TileInfo = tile, + Index = 1 + }; + Av1TileWriter.Av1EntropyCodingContext context = new() + { + MacroBlock = new Av1MacroBlockD { Tile = tile }, + MacroBlockModeInfo = picture.ModeInfoAllocation.Span[16], + SuperblockOrigin = new Point(64, 0) + }; + using Av1EncoderCoefficientBuffer coefficients = new( + Configuration.Default, + picture.Sequence.SequenceHeader, + width: 128, + height: 64); + + using Av1SymbolEncoder writer = new(Configuration.Default, 512, BaseQIndex, updateCdf: true); + + Av1TileWriter.WriteSuperblock( + picture, + context, + writer, + superblock, + coefficients, + tileIndex: 0); + + Assert.Equal(Av1TransformSize.Size32x32, context.MacroBlockModeInfo.Block.TransformSize); + foreach (Point position in modeInfoPositions) + { + Assert.Equal( + Av1TransformSize.Size32x32, + picture.GetMacroBlockModeInfo(position).Block.TransformSize); + } + + writer.Dispose(); + + Assert.Equal(4096, context.CodedAreaSuperblock); + Assert.Equal(0, context.CodedAreaSuperblockUv); + for (int index = 0; index < partitions.Top.Length; index++) + { + Assert.Equal(index < 16 ? 0 : 24, partitions.Top[index].Above); + } + + for (int index = 0; index < partitions.Left.Length; index++) + { + Assert.Equal(24, partitions.Left[index].Left); + } + + for (int index = 0; index < transforms.Top.Length; index++) + { + Assert.Equal(index < 16 ? 0 : 32, transforms.Top[index]); + } + + for (int index = 0; index < transforms.Left.Length; index++) + { + Assert.Equal(32, transforms.Left[index]); + } + } + + [Theory] + [InlineData(true)] + [InlineData(false)] + public void PartitionWriterUsesMatchingFrameEdgeDistribution(bool bottomEdge) + { + const int partitionContext = 8; + Av1BlockSize blockSize = Av1BlockSize.Block32x32; + int modeInfoColumnCount = bottomEdge ? 8 : 4; + int modeInfoRowCount = bottomEdge ? 4 : 8; + Av1PictureControlSet picture = CreateEncoderPicture(modeInfoColumnCount, modeInfoRowCount); + using Av1NeighborArrayUnit neighbors = new( + Configuration.Default, + leftSize: 1, + topSize: 1) + { + GranularityNormalLog2 = 2 + }; + + Av1PartitionType nonSplitPartition = bottomEdge ? Av1PartitionType.Horizontal : Av1PartitionType.Vertical; + ReadOnlySpan decisions = + [ + nonSplitPartition, + Av1PartitionType.Split, + nonSplitPartition, + nonSplitPartition, + Av1PartitionType.Split, + Av1PartitionType.Split, + nonSplitPartition, + Av1PartitionType.Split + ]; + + using Av1SymbolEncoder actualWriter = new(Configuration.Default, 16, BaseQIndex, updateCdf: true); + using Av1SymbolEncoder expectedWriter = new(Configuration.Default, 16, BaseQIndex, updateCdf: true); + foreach (Av1PartitionType decision in decisions) + { + Av1TileWriter.EncodePartition( + picture, + actualWriter, + blockSize, + decision, + Point.Empty, + neighbors); + + if (bottomEdge) + { + expectedWriter.WriteSplitOrHorizontal(decision, blockSize, partitionContext); + } + else + { + expectedWriter.WriteSplitOrVertical(decision, blockSize, partitionContext); + } + } + + using IMemoryOwner actual = actualWriter.Exit(); + using IMemoryOwner expected = expectedWriter.Exit(); + + Assert.True(expected.GetSpan().SequenceEqual(actual.GetSpan())); + } + + [Theory] + [InlineData((int)Av1BlockSize.Block4x4, false, false, true, true)] + [InlineData((int)Av1BlockSize.Block8x8, true, true, true, true)] + [InlineData((int)Av1BlockSize.Block8x8, false, false, true, false)] + [InlineData((int)Av1BlockSize.Block16x16, true, true, true, false)] + [InlineData((int)Av1BlockSize.Block32x32, true, true, false, true)] + [InlineData((int)Av1BlockSize.Block64x64, true, true, false, false)] + public void ChromaFromLumaAvailabilityUsesLosslessPlaneGeometry( + int blockSize, + bool subSamplingX, + bool subSamplingY, + bool isLossless, + bool expected) + => Assert.Equal( + expected, + ((Av1BlockSize)blockSize).AllowsChromaFromLuma(isLossless, subSamplingX, subSamplingY)); + + [Fact] + public void LosslessChromaModeUsesPlaneSizedChromaFromLumaAlphabet() + { + ObuFrameHeader frameHeader = new(); + frameHeader.LosslessArray[0] = true; + ObuColorConfig colorConfig = new() + { + SubSamplingX = true, + SubSamplingY = true + }; + + Av1MacroBlockModeInfo modeInfo = default; + modeInfo.Block.SegmentId = 0; + Av1EncoderBlockStruct block = default; + Av1BlockSize blockSize = Av1BlockSize.Block16x16; + ReadOnlySpan decisions = + [ + Av1ChromaPredictionMode.DC, + Av1ChromaPredictionMode.Smooth, + Av1ChromaPredictionMode.Paeth, + Av1ChromaPredictionMode.SmoothVertical, + Av1ChromaPredictionMode.DC, + Av1ChromaPredictionMode.SmoothHorizontal + ]; + + using Av1SymbolEncoder actualWriter = new(Configuration.Default, 16, BaseQIndex, updateCdf: true); + using Av1SymbolEncoder expectedWriter = new(Configuration.Default, 16, BaseQIndex, updateCdf: true); + foreach (Av1ChromaPredictionMode decision in decisions) + { + Av1TileWriter.EncodeIntraChromaMode( + actualWriter, + frameHeader, + colorConfig, + modeInfo, + ref block, + blockSize, + Av1PredictionMode.DC, + decision); + + expectedWriter.WriteChromaMode( + decision, + isChromaFromLumaAllowed: false, + Av1PredictionMode.DC); + } + + using IMemoryOwner actual = actualWriter.Exit(); + using IMemoryOwner expected = expectedWriter.Exit(); + + Assert.True(expected.GetSpan().SequenceEqual(actual.GetSpan())); + } + + [Fact] + public void RoundTripZeroEndOfBlock() + { + // Assign + Av1BlockSize blockSize = Av1BlockSize.Block4x4; + Av1TransformSize transformSize = Av1TransformSize.Size4x4; + Av1TransformType transformType = Av1TransformType.Identity; + Av1PredictionMode intraDirection = Av1PredictionMode.DC; + Av1ComponentType componentType = Av1ComponentType.Luminance; + Av1FilterIntraMode filterIntraMode = Av1FilterIntraMode.DC; + ushort endOfBlock = 0; + Av1BlockModeInfo modeInfo = new(blockSize, new Point(0, 0)); + Av1TransformInfo transformInfo = new(transformSize, 0, 0); + int[] aboveContexts = new int[1]; + int[] leftContexts = new int[1]; + Av1TransformBlockContext transformBlockContext = default; + Configuration configuration = Configuration.Default; + using Av1SymbolEncoder encoder = new(configuration, CoefficientSyntaxBufferLength, BaseQIndex, updateCdf: true); + Span coefficientsBuffer = [1, 2, 3, 4, 5]; + Span expected = new int[16]; + Span actuals = new int[16]; + + // Act + encoder.WriteCoefficients(transformSize, transformType, intraDirection, coefficientsBuffer, componentType, transformBlockContext, endOfBlock, true, filterIntraMode, usesInterTransformSet: false); + + using IMemoryOwner encoded = encoder.Exit(); + + Av1SymbolDecoder decoder = new(Configuration.Default, encoded.GetSpan(), BaseQIndex); + using Av1LevelBuffer levels = new(Configuration.Default); + decoder.ReadCoefficients( + modeInfo, + new Point(0, 0), + aboveContexts, + leftContexts, + 0, + 0, + 0, + 1, + 1, + transformBlockContext, + transformSize, + false, + true, + transformType, + ref transformInfo, + 0, + 0, + levels, + actuals, + CreateInverseQuantizer()); + + // Assert + Assert.Equal(endOfBlock, transformInfo.EndOfBlock); + Assert.Equal(expected, actuals); + } + + [Theory] + [InlineData(1)] + [InlineData(2)] + [InlineData(3)] + [InlineData(4)] + [InlineData(5)] + [InlineData(6)] + [InlineData(7)] + [InlineData(8)] + [InlineData(9)] + [InlineData(10)] + [InlineData(11)] + [InlineData(12)] + [InlineData(13)] + [InlineData(14)] + [InlineData(15)] + [InlineData(16)] + public void RoundTripFullBlock(ushort endOfBlock) + { + // Assign + const Av1BlockSize blockSize = Av1BlockSize.Block4x4; + const Av1TransformSize transformSize = Av1TransformSize.Size4x4; + const Av1TransformType transformType = Av1TransformType.Identity; + const Av1PredictionMode intraDirection = Av1PredictionMode.DC; + const Av1ComponentType componentType = Av1ComponentType.Luminance; + const Av1FilterIntraMode filterIntraMode = Av1FilterIntraMode.DC; + Av1BlockModeInfo modeInfo = new(blockSize, new Point(0, 0)); + Av1TransformInfo transformInfo = new(transformSize, 0, 0); + int[] aboveContexts = new int[1]; + int[] leftContexts = new int[1]; + Av1TransformBlockContext transformBlockContext = default; + Configuration configuration = Configuration.Default; + using Av1SymbolEncoder encoder = new(configuration, CoefficientSyntaxBufferLength, BaseQIndex, updateCdf: true); + Span coefficientsBuffer = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16]; + ReadOnlySpan scan = Av1ScanOrderConstants.GetScanOrder(transformSize, transformType).Scan; + for (int scanIndex = endOfBlock; scanIndex < scan.Length; scanIndex++) + { + coefficientsBuffer[scan[scanIndex]] = 0; + } + + Span actuals = new int[16]; + + // Act + encoder.WriteCoefficients(transformSize, transformType, intraDirection, coefficientsBuffer, componentType, transformBlockContext, endOfBlock, true, filterIntraMode, usesInterTransformSet: false); + + using IMemoryOwner encoded = encoder.Exit(); + + Av1SymbolDecoder decoder = new(Configuration.Default, encoded.GetSpan(), BaseQIndex); + using Av1LevelBuffer levels = new(Configuration.Default); + int plane = Math.Min((int)componentType, 1); + decoder.ReadCoefficients( + modeInfo, + new Point(0, 0), + aboveContexts, + leftContexts, + 0, + 0, + plane, + 1, + 1, + transformBlockContext, + transformSize, + false, + true, + transformType, + ref transformInfo, + 0, + 0, + levels, + actuals, + CreateInverseQuantizer()); + + decoder.ValidateTrailingBits(); + + // Assert + Assert.Equal(endOfBlock, transformInfo.EndOfBlock); + } + + [Theory] + [MemberData(nameof(GetTransformTypes))] + public void RoundTripFullCoefficientsYSize4x4(int txType) + { + // Assign + const ushort endOfBlock = 16; + const Av1ComponentType componentType = Av1ComponentType.Luminance; + Av1BlockSize blockSize = Av1BlockSize.Block4x4; + Av1TransformSize transformSize = blockSize.GetMaximumTransformSize(); + Av1TransformType transformType = (Av1TransformType)txType; + Av1PredictionMode intraDirection = Av1PredictionMode.DC; + Av1FilterIntraMode filterIntraMode = Av1FilterIntraMode.DC; + RoundTripCoefficientsCore(endOfBlock, componentType, blockSize, transformSize, transformType, intraDirection, filterIntraMode, true, false); + } + + [Theory] + [MemberData(nameof(GetTransformTypes))] + public void RoundTripFullCoefficientsUvSize4x4(int txType) + { + // Assign + const ushort endOfBlock = 16; + const Av1ComponentType componentType = Av1ComponentType.Chroma; + Av1BlockSize blockSize = Av1BlockSize.Block4x4; + Av1TransformSize transformSize = blockSize.GetMaxUvTransformSize(true, true); + Av1TransformType transformType = (Av1TransformType)txType; + Av1PredictionMode intraDirection = Av1PredictionMode.DC; + Av1FilterIntraMode filterIntraMode = Av1FilterIntraMode.DC; + RoundTripCoefficientsCore(endOfBlock, componentType, blockSize, transformSize, transformType, intraDirection, filterIntraMode, true, false); + } + + [Theory] + [InlineData(1)] + [InlineData(2)] + [InlineData(3)] + [InlineData(17)] + [InlineData(33)] + [InlineData(63)] + [InlineData(64)] + public void RoundTripCoefficientsYSize8x8(ushort endOfBlock) + { + const Av1ComponentType componentType = Av1ComponentType.Luminance; + const Av1BlockSize blockSize = Av1BlockSize.Block8x8; + const Av1TransformSize transformSize = Av1TransformSize.Size8x8; + const Av1TransformType transformType = Av1TransformType.DctDct; + const Av1PredictionMode intraDirection = Av1PredictionMode.DC; + const Av1FilterIntraMode filterIntraMode = Av1FilterIntraMode.DC; + RoundTripCoefficientsCore(endOfBlock, componentType, blockSize, transformSize, transformType, intraDirection, filterIntraMode, false, true); + } + + private static Av1InverseQuantizer CreateInverseQuantizer() + { + ObuSequenceHeader sequenceHeader = new() + { + ColorConfig = new ObuColorConfig { BitDepth = Av1BitDepth.EightBit } + }; + + ObuFrameHeader frameHeader = new(); + frameHeader.QuantizationParameters.BaseQIndex = BaseQIndex; + return new Av1InverseQuantizer(sequenceHeader, frameHeader); + } + + private static void RoundTripCoefficientsCore( + ushort endOfBlock, + Av1ComponentType componentType, + Av1BlockSize blockSize, + Av1TransformSize transformSize, + Av1TransformType transformType, + Av1PredictionMode intraDirection, + Av1FilterIntraMode filterIntraMode, + bool useReducedTransformSet, + bool useSparseCoefficients) + { + Av1BlockModeInfo modeInfo = new(blockSize, new Point(0, 0)); + Av1TransformInfo transformInfo = new(transformSize, 0, 0); + int[] aboveContexts = new int[transformSize.Get4x4WideCount()]; + int[] leftContexts = new int[transformSize.Get4x4HighCount()]; + Av1TransformBlockContext transformBlockContext = default; + Configuration configuration = Configuration.Default; + using Av1SymbolEncoder encoder = new(configuration, CoefficientSyntaxBufferLength, BaseQIndex, updateCdf: true); + int coefficientCount = blockSize.GetHeight() * blockSize.GetWidth(); + ReadOnlySpan scan = Av1ScanOrderConstants.GetScanOrder(transformSize, transformType).Scan; + Span coefficientsBuffer = new int[coefficientCount]; + + for (int scanIndex = 0; scanIndex < endOfBlock; scanIndex++) + { + if (!useSparseCoefficients || scanIndex == endOfBlock - 1 || scanIndex % 4 == 0) + { + int level = scanIndex + 1; + + // Signed levels prove encoder context derivation uses magnitude; sparse cases also cover zero-map runs. + coefficientsBuffer[scan[scanIndex]] = (scanIndex & 1) == 0 ? -level : level; + } + } + + Span actuals = new int[coefficientCount]; + + // Act + encoder.WriteCoefficients( + transformSize, + transformType, + intraDirection, + coefficientsBuffer, + componentType, + transformBlockContext, + endOfBlock, + useReducedTransformSet, + filterIntraMode, + usesInterTransformSet: false); + + using IMemoryOwner encoded = encoder.Exit(); + + Av1SymbolDecoder decoder = new(Configuration.Default, encoded.GetSpan(), BaseQIndex); + using Av1LevelBuffer levels = new(Configuration.Default); + int plane = Math.Min((int)componentType, 1); + decoder.ReadCoefficients( + modeInfo, + new Point(0, 0), + aboveContexts, + leftContexts, + 0, + 0, + plane, + 1, + 1, + transformBlockContext, + transformSize, + false, + useReducedTransformSet, + transformType, + ref transformInfo, + 0, + 0, + levels, + actuals, + CreateInverseQuantizer()); + + decoder.ValidateTrailingBits(); + + // Assert + Assert.Equal(endOfBlock, transformInfo.EndOfBlock); + + // Reference quant_common.c defines 8-bit qindex 23 as DC=26 and AC=30. Entropy output now publishes + // dequantized raster values, including zero runs and positions beyond EOB, rather than packed raw levels. + for (int coefficientIndex = 0; coefficientIndex < coefficientCount; coefficientIndex++) + { + int dequant = coefficientIndex == 0 ? 26 : 30; + Assert.Equal(coefficientsBuffer[coefficientIndex] * dequant, actuals[coefficientIndex]); + } + } + + private static Av1MacroBlockModeInfo CreateModeInfo(Av1PredictionMode mode) + { + Av1MacroBlockModeInfo result = default; + result.Block.Mode = mode; + return result; + } + + private static Av1PictureControlSet CreateEncoderPicture( + int modeInfoColumnCount, + int modeInfoRowCount, + bool use128x128Superblock = false) + { + ObuTileGroupHeader tiles = new() + { + TileColumnCount = 1, + TileRowCount = 1 + }; + + tiles.TileColumnStartModeInfo[1] = modeInfoColumnCount; + tiles.TileRowStartModeInfo[1] = modeInfoRowCount; + ObuSequenceHeader sequenceHeader = new() { Use128x128Superblock = use128x128Superblock }; + ObuFrameHeader frameHeader = new() + { + ModeInfoColumnCount = modeInfoColumnCount, + ModeInfoRowCount = modeInfoRowCount, + TilesInfo = tiles + }; + + Av1MacroBlockModeInfo[] modeInfoAllocation = new Av1MacroBlockModeInfo[modeInfoColumnCount * modeInfoRowCount]; + int[] modeInfoGrid = new int[modeInfoAllocation.Length]; + for (int index = 0; index < modeInfoAllocation.Length; index++) + { + modeInfoAllocation[index] = CreateModeInfo(Av1PredictionMode.DC); + modeInfoGrid[index] = index; + } + + return new Av1PictureControlSet + { + PartitionContexts = [], + LuminanceDcSignLevelCoefficientNeighbors = [], + CrDcSignLevelCoefficientNeighbors = [], + CbDcSignLevelCoefficientNeighbors = [], + TransformFunctionContexts = [], + Sequence = new Av1SequenceControlSet { SequenceHeader = sequenceHeader }, + Parent = new Av1PictureParentControlSet + { + Common = new Av1EncoderCommon + { + ModeInfoColumnCount = modeInfoColumnCount, + ModeInfoRowCount = modeInfoRowCount, + ModeInfoStride = modeInfoColumnCount, + FrameSize = new ObuFrameSize(), + TilesInfo = tiles + }, + FrameHeader = frameHeader, + PreviousQIndex = Memory.Empty + }, + SegmentationNeighborMap = Memory.Empty, + ModeInfoGrid = modeInfoGrid, + ModeInfoAllocation = modeInfoAllocation, + ModeInfoStride = modeInfoColumnCount, + CdefPreset = new int[] { -1, -1, -1, -1 }, + TileDataOffsets = Memory.Empty, + TileDataLengths = Memory.Empty + }; + } + + private static Av1MacroBlockD CreateMacroBlock() + { + ObuTileGroupHeader tiles = new() + { + TileColumnCount = 1, + TileRowCount = 1 + }; + + tiles.TileColumnStartModeInfo[1] = 3; + tiles.TileRowStartModeInfo[1] = 3; + ObuFrameHeader frameHeader = new() + { + ModeInfoColumnCount = 3, + ModeInfoRowCount = 3, + TilesInfo = tiles + }; + + return new Av1MacroBlockD { Tile = new Av1TileInfo(0, 0, frameHeader) }; + } + + public static TheoryData GetTransformTypes() + { + TheoryData result = []; + for (Av1TransformType transformType = Av1TransformType.DctDct; transformType < Av1TransformType.VerticalDct; transformType++) + { + result.Add((int)transformType); + } + + return result; + } +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1CompoundBlockDecoderTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1CompoundBlockDecoderTests.cs new file mode 100644 index 0000000000..2771176831 --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1CompoundBlockDecoderTests.cs @@ -0,0 +1,1682 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1; +using SixLabors.ImageSharp.Formats.Heif.Av1.Motion; +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.LoopFilter; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; +using SixLabors.ImageSharp.Formats.Heif.Av1.ReferenceFrames; +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; +using SixLabors.ImageSharp.Tests.TestUtilities; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1; + +/// +/// Verifies compound prediction through the production block-reconstruction branch. +/// +[Trait("Format", "Avif")] +public class Av1CompoundBlockDecoderTests +{ + /// + /// The hardware configurations covering the warped predictor's vector and scalar paths. + /// + private const HwIntrinsics GlobalWarpConfigurations = HwIntrinsics.AllowAll | HwIntrinsics.DisableHWIntrinsic; + + /// + /// The hardware configurations covering every compound-prediction vector width and the scalar fallback. + /// + private const HwIntrinsics CompoundPredictionConfigurations = + HwIntrinsics.AllowAll | HwIntrinsics.DisableAVX512F | HwIntrinsics.DisableAVX | HwIntrinsics.DisableHWIntrinsic; + + /// + /// Verifies that two retained reference planes are predicted and averaged before residual reconstruction. + /// + /// The native sample depth. + [Theory] + [InlineData((int)Av1BitDepth.EightBit)] + [InlineData((int)Av1BitDepth.TenBit)] + [InlineData((int)Av1BitDepth.TwelveBit)] + public void DecodeBlockWithAverageCompound(int bitDepthValue) + { + Av1BitDepth bitDepth = (Av1BitDepth)bitDepthValue; + ushort firstValue = bitDepth == Av1BitDepth.EightBit ? (ushort)20 : (ushort)100; + ushort secondValue = bitDepth switch + { + Av1BitDepth.EightBit => 41, + Av1BitDepth.TenBit => 701, + _ => 3001, + }; + + ushort expected = (ushort)((firstValue + secondValue + 1) >> 1); + ObuSequenceHeader sequenceHeader = CreateSequenceHeader(bitDepth); + ObuFrameHeader frameHeader = CreateFrameHeader(); + frameHeader.GetReferenceFrameIndices()[0] = 0; + frameHeader.GetReferenceFrameIndices()[1] = 1; + + using Av1ReferenceFrameStore referenceFrames = new(); + Assert.True(referenceFrames.Commit(1, CreateReferenceFrame(sequenceHeader, firstValue), showFrame: false)); + Assert.True(referenceFrames.Commit(2, CreateReferenceFrame(sequenceHeader, secondValue), showFrame: false)); + + using Av1FrameBuffer frameBuffer = new( + Configuration.Default, + sequenceHeader, + Av1ColorFormat.Yuv400, + false); + + using Av1FrameInfo frameInfo = new(sequenceHeader); + Av1SuperblockInfo superblockInfo = frameInfo.GetSuperblock(Point.Empty); + superblockInfo.GetTransformInfoY()[0] = new Av1TransformInfo(Av1TransformSize.Size8x8, 0, 0); + + Av1BlockModeInfo modeInfo = new(Av1BlockSize.Block8x8, Point.Empty) + { + Skip = true, + YMode = Av1PredictionMode.NearestNearestMotionVector, + CompoundIndex = true, + CompoundType = Av1CompoundType.Average, + }; + + modeInfo.ReferenceFrames[0] = Av1ReferenceFrameType.Last; + modeInfo.ReferenceFrames[1] = Av1ReferenceFrameType.Last2; + modeInfo.InterpolationFilters.Clear(); + modeInfo.SetTransformUnitCount(Av1PlaneType.Y, 1); + + using Av1LoopFilterContext loopFilterContext = + new(frameBuffer.MemoryAllocator, sequenceHeader, frameHeader); + + using Av1BlockDecoder decoder = new( + sequenceHeader, + frameHeader, + frameBuffer, + loopFilterContext, + referenceFrames); + + decoder.UpdateSuperblock(superblockInfo); + decoder.DecodeBlock( + modeInfo, + Point.Empty, + Av1BlockSize.Block8x8, + superblockInfo, + new Av1TileInfo(0, 0, frameHeader)); + + for (int row = 0; row < 8; row++) + { + if (bitDepth == Av1BitDepth.EightBit) + { + Span samples = frameBuffer.DeriveBlockPointer(Av1Plane.Y, 0, 0).DangerousGetRowSpan(row); + for (int column = 0; column < 8; column++) + { + Assert.Equal((byte)expected, samples[column]); + } + } + else + { + Span samples = frameBuffer.GetHighBitDepthRowSpan(Av1Plane.Y, row, 0, 0); + for (int column = 0; column < 8; column++) + { + Assert.Equal(expected, samples[column]); + } + } + } + } + + /// + /// Verifies that high-bit-depth subpixel predictors retain their no-round precision until the compound average. + /// + [Fact] + public void DecodeBlockWithHighBitDepthAverageCompound() + => FeatureTestRunner.RunWithHwIntrinsicsFeature( + ValidateSubpixelHighBitDepthEqualAverageCompoundPrediction, + CompoundPredictionConfigurations); + + /// + /// Verifies that high-bit-depth subpixel predictors retain no-round precision until distance weighting. + /// + [Fact] + public void DecodeBlockWithHighBitDepthDistanceWeightedCompound() + => FeatureTestRunner.RunWithHwIntrinsicsFeature( + ValidateSubpixelHighBitDepthDistanceWeightedCompoundPrediction, + CompoundPredictionConfigurations); + + /// + /// Verifies that high-bit-depth subpixel predictors retain no-round precision until wedge blending. + /// + [Fact] + public void DecodeBlockWithHighBitDepthWedgeCompound() + => FeatureTestRunner.RunWithHwIntrinsicsFeature( + ValidateSubpixelHighBitDepthWedgeCompoundPrediction, + CompoundPredictionConfigurations); + + /// + /// Verifies that high-bit-depth subpixel predictors retain no-round precision through difference masking and blending. + /// + [Fact] + public void DecodeBlockWithHighBitDepthDifferenceWeightedCompound() + => FeatureTestRunner.RunWithHwIntrinsicsFeature( + ValidateSubpixelHighBitDepthDifferenceWeightedCompoundPrediction, + CompoundPredictionConfigurations); + + /// + /// Verifies that both references of a GLOBAL_GLOBALMV block use their complete matrix before compound averaging. + /// + [Fact] + public void DecodeBlockWithCompoundGlobalWarp() + => FeatureTestRunner.RunWithHwIntrinsicsFeature( + ValidateCompoundGlobalWarpPrediction, + GlobalWarpConfigurations); + + /// + /// Verifies that the production block branch maps a smaller current frame into a larger retained reference. + /// + /// The native sample depth. + [Theory] + [InlineData((int)Av1BitDepth.EightBit)] + [InlineData((int)Av1BitDepth.TenBit)] + [InlineData((int)Av1BitDepth.TwelveBit)] + public void DecodeBlockWithScaledReference(int bitDepthValue) + { + Av1BitDepth bitDepth = (Av1BitDepth)bitDepthValue; + ObuSequenceHeader sequenceHeader = CreateSequenceHeader(bitDepth, 16); + ObuFrameHeader frameHeader = CreateFrameHeader(); + frameHeader.GetReferenceFrameIndices()[0] = 0; + + using Av1ReferenceFrameStore referenceFrames = new(); + Assert.True(referenceFrames.Commit(1, CreateScaledPatternReferenceFrame(sequenceHeader), showFrame: false)); + + using Av1FrameBuffer frameBuffer = new( + Configuration.Default, + sequenceHeader, + Av1ColorFormat.Yuv400, + false); + + frameBuffer.Width = 8; + frameBuffer.Height = 8; + using Av1FrameInfo frameInfo = new(sequenceHeader); + Av1SuperblockInfo superblockInfo = frameInfo.GetSuperblock(Point.Empty); + superblockInfo.GetTransformInfoY()[0] = new Av1TransformInfo(Av1TransformSize.Size8x8, 0, 0); + + Av1BlockModeInfo modeInfo = CreateSingleReferenceModeInfo(Av1BlockSize.Block8x8, Point.Empty); + modeInfo.InterpolationFilters.Fill(Av1InterpolationFilter.Bilinear); + modeInfo.SetTransformUnitCount(Av1PlaneType.Y, 1); + + using Av1LoopFilterContext loopFilterContext = + new(frameBuffer.MemoryAllocator, sequenceHeader, frameHeader); + + using Av1BlockDecoder decoder = new( + sequenceHeader, + frameHeader, + frameBuffer, + loopFilterContext, + referenceFrames); + + decoder.UpdateSuperblock(superblockInfo); + decoder.DecodeBlock( + modeInfo, + Point.Empty, + Av1BlockSize.Block8x8, + superblockInfo, + new Av1TileInfo(0, 0, frameHeader)); + + for (int row = 0; row < 8; row++) + { + for (int column = 0; column < 8; column++) + { + ushort expected = (ushort)(5 + (column * 4) + (row * 16)); + if (bitDepth == Av1BitDepth.EightBit) + { + Span samples = frameBuffer.DeriveBlockPointer(Av1Plane.Y, 0, 0).DangerousGetRowSpan(row); + Assert.Equal((byte)expected, samples[column]); + } + else + { + Span samples = frameBuffer.GetHighBitDepthRowSpan(Av1Plane.Y, row, 0, 0); + Assert.Equal(expected, samples[column]); + } + } + } + } + + /// + /// Verifies that scaled predictors retain their no-round precision until compound averaging. + /// + [Fact] + public void DecodeBlockWithScaledCompound() + => FeatureTestRunner.RunWithHwIntrinsicsFeature( + ValidateScaledCompoundPrediction, + CompoundPredictionConfigurations); + + /// + /// Verifies selectable compound reconstruction through the production block branch at every supported bit depth. + /// + /// The native sample depth. + /// The selected compound operation. + [Theory] + [InlineData((int)Av1BitDepth.EightBit, (int)Av1CompoundType.DistanceWeighted)] + [InlineData((int)Av1BitDepth.TenBit, (int)Av1CompoundType.DistanceWeighted)] + [InlineData((int)Av1BitDepth.TwelveBit, (int)Av1CompoundType.DistanceWeighted)] + [InlineData((int)Av1BitDepth.EightBit, (int)Av1CompoundType.Wedge)] + [InlineData((int)Av1BitDepth.TenBit, (int)Av1CompoundType.Wedge)] + [InlineData((int)Av1BitDepth.TwelveBit, (int)Av1CompoundType.Wedge)] + [InlineData((int)Av1BitDepth.EightBit, (int)Av1CompoundType.DifferenceWeighted)] + [InlineData((int)Av1BitDepth.TenBit, (int)Av1CompoundType.DifferenceWeighted)] + [InlineData((int)Av1BitDepth.TwelveBit, (int)Av1CompoundType.DifferenceWeighted)] + public void DecodeBlockWithSelectableCompound(int bitDepthValue, int compoundTypeValue) + { + Av1BitDepth bitDepth = (Av1BitDepth)bitDepthValue; + Av1CompoundType compoundType = (Av1CompoundType)compoundTypeValue; + ushort firstValue = bitDepth == Av1BitDepth.EightBit ? (ushort)20 : (ushort)100; + ushort secondValue = bitDepth switch + { + Av1BitDepth.EightBit => 41, + Av1BitDepth.TenBit => 701, + _ => 3001, + }; + + ReadOnlySpan wedgeMask = + [ + 0, 0, 0, 1, 1, 2, 4, 6, + 0, 1, 1, 2, 4, 6, 11, 18, + 1, 2, 4, 6, 11, 18, 27, 37, + 4, 6, 11, 18, 27, 37, 46, 53, + 11, 18, 27, 37, 46, 53, 58, 60, + 27, 37, 46, 53, 58, 60, 62, 63, + 46, 53, 58, 60, 62, 63, 63, 64, + 58, 60, 62, 63, 63, 64, 64, 64, + ]; + + ObuSequenceHeader sequenceHeader = CreateSequenceHeader(bitDepth); + sequenceHeader.OrderHintInfo.EnableOrderHint = true; + sequenceHeader.OrderHintInfo.OrderHintBits = 5; + ObuFrameHeader frameHeader = CreateFrameHeader(); + frameHeader.OrderHint = 10; + frameHeader.GetReferenceFrameIndices()[0] = 0; + frameHeader.GetReferenceFrameIndices()[1] = 1; + frameHeader.GetReferenceOrderHints()[0] = 9; + frameHeader.GetReferenceOrderHints()[1] = 5; + + using Av1ReferenceFrameStore referenceFrames = new(); + Assert.True(referenceFrames.Commit(1, CreateReferenceFrame(sequenceHeader, firstValue), showFrame: false)); + Assert.True(referenceFrames.Commit(2, CreateReferenceFrame(sequenceHeader, secondValue), showFrame: false)); + + using Av1FrameBuffer frameBuffer = new( + Configuration.Default, + sequenceHeader, + Av1ColorFormat.Yuv400, + false); + + using Av1FrameInfo frameInfo = new(sequenceHeader); + Av1SuperblockInfo superblockInfo = frameInfo.GetSuperblock(Point.Empty); + superblockInfo.GetTransformInfoY()[0] = new Av1TransformInfo(Av1TransformSize.Size8x8, 0, 0); + + Av1BlockModeInfo modeInfo = new(Av1BlockSize.Block8x8, Point.Empty) + { + Skip = true, + YMode = Av1PredictionMode.NearestNearestMotionVector, + CompoundIndex = compoundType != Av1CompoundType.DistanceWeighted, + CompoundType = compoundType, + CompoundWedgeIndex = 0, + CompoundWedgeSign = true, + DifferenceWeightedMaskType = Av1DifferenceWeightedMaskType.Type38, + }; + + modeInfo.ReferenceFrames[0] = Av1ReferenceFrameType.Last; + modeInfo.ReferenceFrames[1] = Av1ReferenceFrameType.Last2; + modeInfo.InterpolationFilters.Clear(); + modeInfo.SetTransformUnitCount(Av1PlaneType.Y, 1); + + using Av1LoopFilterContext loopFilterContext = + new(frameBuffer.MemoryAllocator, sequenceHeader, frameHeader); + + using Av1BlockDecoder decoder = new( + sequenceHeader, + frameHeader, + frameBuffer, + loopFilterContext, + referenceFrames); + + decoder.UpdateSuperblock(superblockInfo); + decoder.DecodeBlock( + modeInfo, + Point.Empty, + Av1BlockSize.Block8x8, + superblockInfo, + new Av1TileInfo(0, 0, frameHeader)); + + int differenceShift = bitDepth.GetBitCount() - 8 + 4; + int differenceAlpha = Math.Min(64, 38 + (Math.Abs(firstValue - secondValue) >> differenceShift)); + for (int row = 0; row < 8; row++) + { + for (int column = 0; column < 8; column++) + { + int alpha = compoundType switch + { + Av1CompoundType.Wedge => wedgeMask[(row * 8) + column], + Av1CompoundType.DifferenceWeighted => differenceAlpha, + _ => 52, + }; + + ushort expected = (ushort)(((alpha * firstValue) + ((64 - alpha) * secondValue) + 32) >> 6); + if (bitDepth == Av1BitDepth.EightBit) + { + Span samples = frameBuffer.DeriveBlockPointer(Av1Plane.Y, 0, 0).DangerousGetRowSpan(row); + Assert.Equal((byte)expected, samples[column]); + } + else + { + Span samples = frameBuffer.GetHighBitDepthRowSpan(Av1Plane.Y, row, 0, 0); + Assert.Equal(expected, samples[column]); + } + } + } + } + + /// + /// Verifies smooth inter-intra reconstruction through the production block branch at every supported bit depth. + /// + /// The native sample depth. + [Theory] + [InlineData((int)Av1BitDepth.EightBit)] + [InlineData((int)Av1BitDepth.TenBit)] + [InlineData((int)Av1BitDepth.TwelveBit)] + public void DecodeBlockWithSmoothInterIntra(int bitDepthValue) + { + Av1BitDepth bitDepth = (Av1BitDepth)bitDepthValue; + ushort interValue = bitDepth == Av1BitDepth.EightBit ? (ushort)20 : (ushort)100; + ushort intraValue = (ushort)(1 << (bitDepth.GetBitCount() - 1)); + ushort expected = (ushort)((interValue + intraValue + 1) >> 1); + ObuSequenceHeader sequenceHeader = CreateSequenceHeader(bitDepth); + ObuFrameHeader frameHeader = CreateFrameHeader(); + frameHeader.GetReferenceFrameIndices()[0] = 0; + + using Av1ReferenceFrameStore referenceFrames = new(); + Assert.True(referenceFrames.Commit(1, CreateReferenceFrame(sequenceHeader, interValue), showFrame: false)); + + using Av1FrameBuffer frameBuffer = new( + Configuration.Default, + sequenceHeader, + Av1ColorFormat.Yuv400, + false); + + using Av1FrameInfo frameInfo = new(sequenceHeader); + Av1SuperblockInfo superblockInfo = frameInfo.GetSuperblock(Point.Empty); + superblockInfo.GetTransformInfoY()[0] = new Av1TransformInfo(Av1TransformSize.Size8x8, 0, 0); + + Av1BlockModeInfo modeInfo = new(Av1BlockSize.Block8x8, Point.Empty) + { + Skip = true, + YMode = Av1PredictionMode.NearestMotionVector, + InterIntraMode = Av1InterIntraMode.DC, + UseInterIntraWedge = false, + }; + + modeInfo.ReferenceFrames[0] = Av1ReferenceFrameType.Last; + modeInfo.ReferenceFrames[1] = Av1ReferenceFrameType.Intra; + modeInfo.InterpolationFilters.Clear(); + modeInfo.SetTransformUnitCount(Av1PlaneType.Y, 1); + + using Av1LoopFilterContext loopFilterContext = + new(frameBuffer.MemoryAllocator, sequenceHeader, frameHeader); + + using Av1BlockDecoder decoder = new( + sequenceHeader, + frameHeader, + frameBuffer, + loopFilterContext, + referenceFrames); + + decoder.UpdateSuperblock(superblockInfo); + decoder.DecodeBlock( + modeInfo, + Point.Empty, + Av1BlockSize.Block8x8, + superblockInfo, + new Av1TileInfo(0, 0, frameHeader)); + + for (int row = 0; row < 8; row++) + { + if (bitDepth == Av1BitDepth.EightBit) + { + Span samples = frameBuffer.DeriveBlockPointer(Av1Plane.Y, 0, 0).DangerousGetRowSpan(row); + for (int column = 0; column < 8; column++) + { + Assert.Equal((byte)expected, samples[column]); + } + } + else + { + Span samples = frameBuffer.GetHighBitDepthRowSpan(Av1Plane.Y, row, 0, 0); + for (int column = 0; column < 8; column++) + { + Assert.Equal(expected, samples[column]); + } + } + } + } + + /// + /// Verifies above-then-left OBMC reconstruction through the production block branch at every supported bit depth. + /// + /// The native sample depth. + [Theory] + [InlineData((int)Av1BitDepth.EightBit)] + [InlineData((int)Av1BitDepth.TenBit)] + [InlineData((int)Av1BitDepth.TwelveBit)] + public void DecodeBlockWithObmc(int bitDepthValue) + { + const int frameSize = 24; + const int blockOrigin = 8; + Av1BitDepth bitDepth = (Av1BitDepth)bitDepthValue; + ObuSequenceHeader sequenceHeader = CreateSequenceHeader(bitDepth, frameSize); + ObuFrameHeader frameHeader = CreateFrameHeader(frameSize); + frameHeader.GetReferenceFrameIndices()[0] = 0; + + using Av1ReferenceFrameStore referenceFrames = new(); + Assert.True(referenceFrames.Commit(1, CreatePatternReferenceFrame(sequenceHeader, frameHeader), showFrame: false)); + + using Av1FrameBuffer frameBuffer = new( + Configuration.Default, + sequenceHeader, + Av1ColorFormat.Yuv400, + false); + + using Av1FrameInfo frameInfo = new(sequenceHeader); + Av1SuperblockInfo superblockInfo = frameInfo.GetSuperblock(Point.Empty); + superblockInfo.GetTransformInfoY()[0] = new Av1TransformInfo(Av1TransformSize.Size8x8, 0, 0); + + Av1BlockModeInfo above = CreateSingleReferenceModeInfo(Av1BlockSize.Block8x8, new Point(2, 0)); + above.MotionVectors[0] = new Av1MotionVector(0, 8); + frameInfo.UpdateModeInfo(above, superblockInfo); + + Av1BlockModeInfo left = CreateSingleReferenceModeInfo(Av1BlockSize.Block8x8, new Point(0, 2)); + left.MotionVectors[0] = new Av1MotionVector(8, 0); + frameInfo.UpdateModeInfo(left, superblockInfo); + + Av1BlockModeInfo current = CreateSingleReferenceModeInfo(Av1BlockSize.Block8x8, new Point(2, 2)); + current.MotionMode = Av1MotionMode.Obmc; + current.SetTransformUnitCount(Av1PlaneType.Y, 1); + frameInfo.UpdateModeInfo(current, superblockInfo); + + using Av1LoopFilterContext loopFilterContext = + new(frameBuffer.MemoryAllocator, sequenceHeader, frameHeader); + + using Av1BlockDecoder decoder = new( + sequenceHeader, + frameHeader, + frameBuffer, + loopFilterContext, + referenceFrames); + + decoder.UpdateSuperblock(superblockInfo); + decoder.DecodeBlock( + current, + new Point(2, 2), + Av1BlockSize.Block8x8, + superblockInfo, + new Av1TileInfo(0, 0, frameHeader)); + + ReadOnlySpan mask = [39, 50, 59, 64]; + for (int row = 0; row < 8; row++) + { + for (int column = 0; column < 8; column++) + { + int first = GetPatternValue(blockOrigin + column, blockOrigin + row); + if (row < mask.Length) + { + int aboveValue = GetPatternValue(blockOrigin + column + 1, blockOrigin + row); + first = ((mask[row] * first) + ((64 - mask[row]) * aboveValue) + 32) >> 6; + } + + if (column < mask.Length) + { + int leftValue = GetPatternValue(blockOrigin + column, blockOrigin + row + 1); + first = ((mask[column] * first) + ((64 - mask[column]) * leftValue) + 32) >> 6; + } + + if (bitDepth == Av1BitDepth.EightBit) + { + Span samples = frameBuffer.DeriveBlockPointer(Av1Plane.Y, 0, 0).DangerousGetRowSpan(blockOrigin + row); + Assert.Equal((byte)first, samples[blockOrigin + column]); + } + else + { + Span samples = frameBuffer.GetHighBitDepthRowSpan(Av1Plane.Y, blockOrigin + row, 0, 0); + Assert.Equal((ushort)first, samples[blockOrigin + column]); + } + } + } + } + + /// + /// Verifies the above and left OBMC rectangles on horizontally and vertically subsampled chroma planes. + /// + /// The chroma-subsampling layout to reconstruct. + [Theory] + [InlineData((int)Av1ColorFormat.Yuv420)] + [InlineData((int)Av1ColorFormat.Yuv422)] + public void DecodeBlockWithSubsampledObmc(int colorFormatValue) + { + const int frameSize = 48; + const int blockOrigin = 16; + Av1ColorFormat colorFormat = (Av1ColorFormat)colorFormatValue; + ObuSequenceHeader sequenceHeader = CreateSequenceHeader(Av1BitDepth.EightBit, frameSize, colorFormat); + ObuFrameHeader frameHeader = CreateFrameHeader(frameSize); + frameHeader.GetReferenceFrameIndices()[0] = 0; + + using Av1ReferenceFrameStore referenceFrames = new(); + Assert.True(referenceFrames.Commit(1, CreatePatternReferenceFrame(sequenceHeader, frameHeader, colorFormat), showFrame: false)); + + using Av1FrameBuffer frameBuffer = new( + Configuration.Default, + sequenceHeader, + colorFormat, + false); + + using Av1FrameInfo frameInfo = new(sequenceHeader); + Av1SuperblockInfo superblockInfo = frameInfo.GetSuperblock(Point.Empty); + superblockInfo.GetTransformInfoY()[0] = new Av1TransformInfo(Av1TransformSize.Size16x16, 0, 0); + superblockInfo.GetTransformInfoUv()[0] = new Av1TransformInfo(Av1TransformSize.Size8x8, 0, 0); + superblockInfo.GetTransformInfoUv()[1] = new Av1TransformInfo(Av1TransformSize.Size8x8, 0, 0); + + Av1BlockModeInfo above = CreateSingleReferenceModeInfo(Av1BlockSize.Block16x16, new Point(4, 0)); + above.MotionVectors[0] = new Av1MotionVector(0, 16); + frameInfo.UpdateModeInfo(above, superblockInfo); + + Av1BlockModeInfo left = CreateSingleReferenceModeInfo(Av1BlockSize.Block16x16, new Point(0, 4)); + left.MotionVectors[0] = new Av1MotionVector(16, 0); + frameInfo.UpdateModeInfo(left, superblockInfo); + + Av1BlockModeInfo current = CreateSingleReferenceModeInfo(Av1BlockSize.Block16x16, new Point(4, 4)); + current.MotionMode = Av1MotionMode.Obmc; + current.SetTransformUnitCount(Av1PlaneType.Y, 1); + current.SetTransformUnitCount(Av1PlaneType.Uv, 1); + frameInfo.UpdateModeInfo(current, superblockInfo); + + using Av1LoopFilterContext loopFilterContext = + new(frameBuffer.MemoryAllocator, sequenceHeader, frameHeader); + + using Av1BlockDecoder decoder = new( + sequenceHeader, + frameHeader, + frameBuffer, + loopFilterContext, + referenceFrames); + + decoder.UpdateSuperblock(superblockInfo); + decoder.DecodeBlock( + current, + new Point(4, 4), + Av1BlockSize.Block16x16, + superblockInfo, + new Av1TileInfo(0, 0, frameHeader)); + + ReadOnlySpan mask4 = [39, 50, 59, 64]; + + ReadOnlySpan mask8 = [36, 42, 48, 53, 57, 61, 64, 64]; + + for (int plane = (int)Av1Plane.U; plane <= (int)Av1Plane.V; plane++) + { + int subY = colorFormat == Av1ColorFormat.Yuv420 ? 1 : 0; + int planeOriginX = blockOrigin >> 1; + int planeOriginY = blockOrigin >> subY; + int predictionWidth = Av1BlockSize.Block16x16.GetWidth() >> 1; + int predictionHeight = Av1BlockSize.Block16x16.GetHeight() >> subY; + int leftSourceRowOffset = 2 >> subY; + ReadOnlySpan verticalMask = subY == 0 ? mask8 : mask4; + + for (int row = 0; row < predictionHeight; row++) + { + Span samples = frameBuffer.DeriveBlockPointer((Av1Plane)plane, 1, subY).DangerousGetRowSpan(planeOriginY + row); + for (int column = 0; column < predictionWidth; column++) + { + int expected = GetPlanePatternValue(plane, planeOriginX + column, planeOriginY + row); + if (row < verticalMask.Length) + { + int aboveValue = GetPlanePatternValue(plane, planeOriginX + column + 1, planeOriginY + row); + expected = ((verticalMask[row] * expected) + ((64 - verticalMask[row]) * aboveValue) + 32) >> 6; + } + + if (column < mask4.Length) + { + int leftValue = GetPlanePatternValue(plane, planeOriginX + column, planeOriginY + row + leftSourceRowOffset); + expected = ((mask4[column] * expected) + ((64 - mask4[column]) * leftValue) + 32) >> 6; + } + + Assert.Equal((byte)expected, samples[planeOriginX + column]); + } + } + } + } + + /// + /// Creates one skipped single-reference mode record for direct block-reconstruction tests. + /// + private static Av1BlockModeInfo CreateSingleReferenceModeInfo(Av1BlockSize blockSize, Point position) + { + Av1BlockModeInfo modeInfo = new(blockSize, position) + { + Skip = true, + YMode = Av1PredictionMode.NearestMotionVector, + }; + + modeInfo.ReferenceFrames[0] = Av1ReferenceFrameType.Last; + modeInfo.ReferenceFrames[1] = Av1ReferenceFrameType.None; + modeInfo.InterpolationFilters.Clear(); + return modeInfo; + } + + /// + /// Reconstructs the scaled compound regression at every supported source precision. + /// + private static void ValidateScaledCompoundPrediction() + { + foreach (Av1BitDepth bitDepth in new[] { Av1BitDepth.EightBit, Av1BitDepth.TenBit, Av1BitDepth.TwelveBit }) + { + ValidateScaledCompoundPredictionAtBitDepth(bitDepth); + } + } + + /// + /// Reconstructs one scaled compound block and compares the production branch with the no-round pipeline. + /// + /// The native sample depth. + private static void ValidateScaledCompoundPredictionAtBitDepth(Av1BitDepth bitDepth) + { + const int currentSize = 8; + const int referenceSize = 16; + ObuSequenceHeader sequenceHeader = CreateSequenceHeader(bitDepth, referenceSize); + ObuFrameHeader frameHeader = CreateFrameHeader(currentSize); + frameHeader.GetReferenceFrameIndices()[0] = 0; + frameHeader.GetReferenceFrameIndices()[1] = 1; + + using Av1ReferenceFrameStore referenceFrames = new(); + Assert.True(referenceFrames.Commit( + 1, + CreateScaledPatternReferenceFrame(sequenceHeader), + showFrame: false)); + + Assert.True(referenceFrames.Commit( + 2, + CreateScaledPatternReferenceFrame( + sequenceHeader, + horizontalScale: 3, + verticalScale: 7, + nonlinearScale: 3), + showFrame: false)); + + Av1BlockModeInfo modeInfo = new(Av1BlockSize.Block8x8, Point.Empty) + { + Skip = true, + YMode = Av1PredictionMode.NearestNearestMotionVector, + CompoundIndex = true, + CompoundType = Av1CompoundType.Average, + }; + + modeInfo.ReferenceFrames[0] = Av1ReferenceFrameType.Last; + modeInfo.ReferenceFrames[1] = Av1ReferenceFrameType.Last2; + modeInfo.InterpolationFilters.Fill(Av1InterpolationFilter.Bilinear); + modeInfo.SetTransformUnitCount(Av1PlaneType.Y, 1); + + Av1ReferenceScale scale = new(referenceSize, referenceSize, currentSize, currentSize); + int sourceColumnQ10 = scale.ScaleHorizontal(0) + Av1ReferenceScale.ExtraOffset; + int sourceRowQ10 = scale.ScaleVertical(0) + Av1ReferenceScale.ExtraOffset; + int horizontalPhase = sourceColumnQ10 & Av1ReferenceScale.SubpixelMask; + int verticalPhase = sourceRowQ10 & Av1ReferenceScale.SubpixelMask; + ushort[] firstIntermediate = new ushort[currentSize * currentSize]; + ushort[] secondIntermediate = new ushort[currentSize * currentSize]; + byte[] firstRounded8 = new byte[currentSize * currentSize]; + byte[] secondRounded8 = new byte[currentSize * currentSize]; + ushort[] firstRoundedHigh = new ushort[currentSize * currentSize]; + ushort[] secondRoundedHigh = new ushort[currentSize * currentSize]; + short[] predictionScratch = new short[ + Av1ScaledInterPredictor.GetScaledScratchLength( + currentSize, + currentSize, + verticalPhase, + scale.VerticalStep)]; + + for (int referenceIndex = 0; referenceIndex < 2; referenceIndex++) + { + Av1FrameBuffer reference = referenceFrames.ResolveRequired(referenceIndex).FrameBuffer; + Span intermediate = referenceIndex == 0 ? firstIntermediate : secondIntermediate; + int sourceIndex; + if (bitDepth == Av1BitDepth.EightBit) + { + Span source = reference.GetPaddedPlaneSpan( + Av1Plane.Y, + 0, + 0, + out int sourceStride, + out Point sourceOrigin); + + sourceIndex = + ((sourceOrigin.Y + (sourceRowQ10 >> Av1ReferenceScale.SubpixelBits)) * sourceStride) + + sourceOrigin.X + + (sourceColumnQ10 >> Av1ReferenceScale.SubpixelBits); + + Av1ScaledInterPredictor.PredictScaledCompound( + source, + sourceStride, + sourceIndex, + intermediate, + currentSize, + currentSize, + currentSize, + Av1InterpolationFilter.Bilinear, + Av1InterpolationFilter.Bilinear, + horizontalPhase, + scale.HorizontalStep, + verticalPhase, + scale.VerticalStep, + predictionScratch); + + Av1ScaledInterPredictor.PredictScaled( + source, + sourceStride, + sourceIndex, + referenceIndex == 0 ? firstRounded8 : secondRounded8, + currentSize, + currentSize, + currentSize, + Av1InterpolationFilter.Bilinear, + Av1InterpolationFilter.Bilinear, + horizontalPhase, + scale.HorizontalStep, + verticalPhase, + scale.VerticalStep, + predictionScratch); + } + else + { + Span source = reference.GetPaddedPlaneSpan16( + Av1Plane.Y, + 0, + 0, + out int sourceStride, + out Point sourceOrigin); + + sourceIndex = + ((sourceOrigin.Y + (sourceRowQ10 >> Av1ReferenceScale.SubpixelBits)) * sourceStride) + + sourceOrigin.X + + (sourceColumnQ10 >> Av1ReferenceScale.SubpixelBits); + + Av1ScaledInterPredictor.PredictScaledCompound( + source, + sourceStride, + sourceIndex, + intermediate, + currentSize, + currentSize, + currentSize, + Av1InterpolationFilter.Bilinear, + Av1InterpolationFilter.Bilinear, + horizontalPhase, + scale.HorizontalStep, + verticalPhase, + scale.VerticalStep, + bitDepth.GetBitCount(), + predictionScratch); + + Av1ScaledInterPredictor.PredictScaled( + source, + sourceStride, + sourceIndex, + referenceIndex == 0 ? firstRoundedHigh : secondRoundedHigh, + currentSize, + currentSize, + currentSize, + Av1InterpolationFilter.Bilinear, + Av1InterpolationFilter.Bilinear, + horizontalPhase, + scale.HorizontalStep, + verticalPhase, + scale.VerticalStep, + bitDepth.GetBitCount(), + predictionScratch); + } + } + + using Av1FrameBuffer frameBuffer = new( + Configuration.Default, + sequenceHeader, + Av1ColorFormat.Yuv400, + false); + + frameBuffer.Width = currentSize; + frameBuffer.Height = currentSize; + using Av1FrameInfo frameInfo = new(sequenceHeader); + Av1SuperblockInfo superblockInfo = frameInfo.GetSuperblock(Point.Empty); + superblockInfo.GetTransformInfoY()[0] = new Av1TransformInfo(Av1TransformSize.Size8x8, 0, 0); + using Av1LoopFilterContext loopFilterContext = + new(frameBuffer.MemoryAllocator, sequenceHeader, frameHeader); + + using Av1BlockDecoder decoder = new( + sequenceHeader, + frameHeader, + frameBuffer, + loopFilterContext, + referenceFrames); + + decoder.UpdateSuperblock(superblockInfo); + decoder.DecodeBlock( + modeInfo, + Point.Empty, + Av1BlockSize.Block8x8, + superblockInfo, + new Av1TileInfo(0, 0, frameHeader)); + + if (bitDepth == Av1BitDepth.EightBit) + { + byte[] expected = new byte[currentSize * currentSize]; + Av1CompoundIntermediateAveragePredictor.AverageIntermediate( + expected, + currentSize, + firstIntermediate, + currentSize, + secondIntermediate, + currentSize, + currentSize, + currentSize, + bitDepth.GetBitCount()); + + byte[] prematurelyRounded = new byte[currentSize * currentSize]; + for (int index = 0; index < prematurelyRounded.Length; index++) + { + prematurelyRounded[index] = (byte)((firstRounded8[index] + secondRounded8[index] + 1) >> 1); + } + + Assert.False(expected.AsSpan().SequenceEqual(prematurelyRounded)); + for (int row = 0; row < currentSize; row++) + { + Span actual = frameBuffer.DeriveBlockPointer(Av1Plane.Y, 0, 0).DangerousGetRowSpan(row); + Assert.Equal(expected.AsSpan(row * currentSize, currentSize), actual[..currentSize]); + } + } + else + { + ushort[] expected = new ushort[currentSize * currentSize]; + Av1CompoundIntermediateAveragePredictor.AverageIntermediate( + expected, + currentSize, + firstIntermediate, + currentSize, + secondIntermediate, + currentSize, + currentSize, + currentSize, + bitDepth.GetBitCount()); + + ushort[] prematurelyRounded = new ushort[currentSize * currentSize]; + for (int index = 0; index < prematurelyRounded.Length; index++) + { + prematurelyRounded[index] = (ushort)((firstRoundedHigh[index] + secondRoundedHigh[index] + 1) >> 1); + } + + Assert.False(expected.AsSpan().SequenceEqual(prematurelyRounded)); + for (int row = 0; row < currentSize; row++) + { + Span actual = frameBuffer.GetHighBitDepthRowSpan(Av1Plane.Y, row, 0, 0); + Assert.Equal(expected.AsSpan(row * currentSize, currentSize), actual[..currentSize]); + } + } + } + + /// + /// Reconstructs a compound global-warp block at every supported native sample depth. + /// + private static void ValidateCompoundGlobalWarpPrediction() + { + foreach (Av1BitDepth bitDepth in new[] { Av1BitDepth.EightBit, Av1BitDepth.TenBit, Av1BitDepth.TwelveBit }) + { + ValidateCompoundGlobalWarpPredictionAtBitDepth(bitDepth); + } + } + + /// + /// Reconstructs the high-bit-depth subpixel compound regression at every supported source precision. + /// + private static void ValidateSubpixelHighBitDepthEqualAverageCompoundPrediction() + { + foreach (Av1BitDepth bitDepth in new[] { Av1BitDepth.TenBit, Av1BitDepth.TwelveBit }) + { + ValidateSubpixelHighBitDepthCompoundPredictionAtBitDepth(bitDepth, Av1CompoundType.Average); + } + } + + /// + /// Reconstructs the high-bit-depth subpixel distance-weighted regression at every supported source precision. + /// + private static void ValidateSubpixelHighBitDepthDistanceWeightedCompoundPrediction() + { + foreach (Av1BitDepth bitDepth in new[] { Av1BitDepth.TenBit, Av1BitDepth.TwelveBit }) + { + ValidateSubpixelHighBitDepthCompoundPredictionAtBitDepth(bitDepth, Av1CompoundType.DistanceWeighted); + } + } + + /// + /// Reconstructs the high-bit-depth subpixel wedge regression at every supported source precision. + /// + private static void ValidateSubpixelHighBitDepthWedgeCompoundPrediction() + { + foreach (Av1BitDepth bitDepth in new[] { Av1BitDepth.TenBit, Av1BitDepth.TwelveBit }) + { + ValidateSubpixelHighBitDepthCompoundPredictionAtBitDepth(bitDepth, Av1CompoundType.Wedge); + } + } + + /// + /// Reconstructs both high-bit-depth subpixel difference-mask orientations at every supported source precision. + /// + private static void ValidateSubpixelHighBitDepthDifferenceWeightedCompoundPrediction() + { + foreach (Av1BitDepth bitDepth in new[] { Av1BitDepth.TenBit, Av1BitDepth.TwelveBit }) + { + foreach (Av1DifferenceWeightedMaskType maskType in Enum.GetValues()) + { + ValidateSubpixelHighBitDepthCompoundPredictionAtBitDepth( + bitDepth, + Av1CompoundType.DifferenceWeighted, + maskType); + } + } + } + + /// + /// Reconstructs one high-bit-depth half-sample compound block and compares it with the scalar no-round pipeline. + /// + /// The native sample depth. + /// The final compound operation. + /// The difference-mask orientation when that compound operation is selected. + private static void ValidateSubpixelHighBitDepthCompoundPredictionAtBitDepth( + Av1BitDepth bitDepth, + Av1CompoundType compoundType, + Av1DifferenceWeightedMaskType differenceWeightedMaskType = Av1DifferenceWeightedMaskType.Type38) + { + const int frameSize = 32; + const int blockOrigin = 8; + const int blockSize = 8; + ObuSequenceHeader sequenceHeader = CreateSequenceHeader(bitDepth, frameSize); + sequenceHeader.OrderHintInfo.EnableOrderHint = true; + sequenceHeader.OrderHintInfo.OrderHintBits = 5; + ObuFrameHeader frameHeader = CreateFrameHeader(frameSize); + frameHeader.OrderHint = 10; + frameHeader.GetReferenceFrameIndices()[0] = 0; + frameHeader.GetReferenceFrameIndices()[1] = 1; + frameHeader.GetReferenceOrderHints()[0] = 9; + frameHeader.GetReferenceOrderHints()[1] = 5; + + using Av1ReferenceFrameStore referenceFrames = new(); + Assert.True(referenceFrames.Commit( + 1, + CreatePatternReferenceFrame(sequenceHeader, CreateFrameHeader(frameSize)), + showFrame: false)); + + Assert.True(referenceFrames.Commit( + 2, + CreatePatternReferenceFrame(sequenceHeader, CreateFrameHeader(frameSize), sampleOffset: 40), + showFrame: false)); + + Av1BlockModeInfo modeInfo = new(Av1BlockSize.Block8x8, new Point(2, 2)) + { + Skip = true, + YMode = Av1PredictionMode.NearestNearestMotionVector, + CompoundIndex = compoundType != Av1CompoundType.DistanceWeighted, + CompoundType = compoundType, + CompoundWedgeIndex = 0, + CompoundWedgeSign = true, + DifferenceWeightedMaskType = differenceWeightedMaskType, + }; + + modeInfo.ReferenceFrames[0] = Av1ReferenceFrameType.Last; + modeInfo.ReferenceFrames[1] = Av1ReferenceFrameType.Last2; + modeInfo.MotionVectors[0] = new Av1MotionVector(0, 0); + + // The second predictor lands exactly halfway between horizontal samples. Rounding it before combining the + // references changes every result by one, so this vector distinguishes the required no-round production path. + modeInfo.MotionVectors[1] = new Av1MotionVector(0, 4); + modeInfo.InterpolationFilters.Fill(Av1InterpolationFilter.Bilinear); + modeInfo.SetTransformUnitCount(Av1PlaneType.Y, 1); + + ushort[] expectedFirst = new ushort[blockSize * blockSize]; + ushort[] expectedSecond = new ushort[blockSize * blockSize]; + Span expectedPredictions = expectedFirst; + ushort[] actualFirst = new ushort[blockSize * blockSize]; + ushort[] actualSecond = new ushort[blockSize * blockSize]; + Span actualPredictions = actualFirst; + short[] predictionScratch = new short[128 * (blockSize + 8)]; + for (int referenceIndex = 0; referenceIndex < 2; referenceIndex++) + { + Av1FrameBuffer reference = referenceFrames.ResolveRequired(referenceIndex).FrameBuffer; + Span source = reference.GetPaddedPlaneSpan16( + Av1Plane.Y, + 0, + 0, + out int sourceStride, + out Point sourceOrigin); + + Av1MotionVector motionVector = modeInfo.MotionVectors[referenceIndex]; + int sourceColumnQ4 = (blockOrigin << 4) + (motionVector.Column << 1); + int sourceRowQ4 = (blockOrigin << 4) + (motionVector.Row << 1); + int sourceIndex = + ((sourceOrigin.Y + (sourceRowQ4 >> 4)) * sourceStride) + sourceOrigin.X + (sourceColumnQ4 >> 4); + + Av1CompoundInterPredictor.PredictCompoundScalar( + source, + sourceStride, + sourceIndex, + expectedPredictions, + blockSize, + blockSize, + blockSize, + Av1InterpolationFilter.Bilinear, + Av1InterpolationFilter.Bilinear, + sourceColumnQ4 & 15, + sourceRowQ4 & 15, + bitDepth.GetBitCount(), + predictionScratch); + + Av1CompoundInterPredictor.PredictCompound( + source, + sourceStride, + sourceIndex, + actualPredictions, + blockSize, + blockSize, + blockSize, + Av1InterpolationFilter.Bilinear, + Av1InterpolationFilter.Bilinear, + sourceColumnQ4 & 15, + sourceRowQ4 & 15, + bitDepth.GetBitCount(), + predictionScratch); + + expectedPredictions = expectedSecond; + actualPredictions = actualSecond; + } + + Assert.Equal(expectedFirst, actualFirst); + Assert.Equal(expectedSecond, actualSecond); + + ushort[] expected = new ushort[blockSize * blockSize]; + if (compoundType == Av1CompoundType.DistanceWeighted) + { + Av1CompoundDistanceWeights.Derive( + sequenceHeader.OrderHintInfo, + frameHeader, + modeInfo.ReferenceFrames[0], + modeInfo.ReferenceFrames[1], + out int firstWeight, + out int secondWeight); + + Av1CompoundIntermediateDistanceWeightedPredictor.DistanceWeightedIntermediate( + expected, + blockSize, + expectedFirst, + blockSize, + expectedSecond, + blockSize, + blockSize, + blockSize, + firstWeight, + secondWeight, + bitDepth.GetBitCount()); + + Assert.NotEqual((ushort)60, expected[0]); + } + else if (compoundType == Av1CompoundType.DifferenceWeighted) + { + int bitCount = bitDepth.GetBitCount(); + int intermediateRange = bitCount + 7 - 3 + 2; + int round0 = 3 + Math.Max(intermediateRange - 16, 0); + int roundBits = 14 - round0 - 7; + int offsetBits = bitCount + 14 - round0; + int roundOffset = (1 << (offsetBits - 7)) + (1 << (offsetBits - 8)); + int differenceRound = roundBits + bitCount - 8; + int maximum = (1 << bitCount) - 1; + byte[] expectedMask = new byte[blockSize * blockSize]; + byte[] actualMask = new byte[blockSize * blockSize]; + + for (int index = 0; index < expected.Length; index++) + { + int difference = Math.Abs(expectedFirst[index] - expectedSecond[index]); + difference = (difference + (1 << (differenceRound - 1))) >> differenceRound; + int alpha = Math.Min(64, 38 + (difference >> 4)); + if (differenceWeightedMaskType == Av1DifferenceWeightedMaskType.Type38Inverse) + { + alpha = 64 - alpha; + } + + expectedMask[index] = (byte)alpha; + int result = ((alpha * expectedFirst[index]) + ((64 - alpha) * expectedSecond[index])) >> 6; + result -= roundOffset; + if (roundBits != 0) + { + result = (result + (1 << (roundBits - 1))) >> roundBits; + } + + expected[index] = (ushort)Math.Clamp(result, 0, maximum); + } + + Av1CompoundIntermediateDifferenceWeightedMaskBuilder.FillDifferenceWeightedIntermediateMask( + actualMask, + blockSize, + expectedFirst, + blockSize, + expectedSecond, + blockSize, + blockSize, + blockSize, + bitCount, + differenceWeightedMaskType); + + Assert.Equal(expectedMask, actualMask); + } + else if (compoundType == Av1CompoundType.Wedge) + { + ReadOnlySpan wedgeMask = + [ + 0, 0, 0, 1, 1, 2, 4, 6, + 0, 1, 1, 2, 4, 6, 11, 18, + 1, 2, 4, 6, 11, 18, 27, 37, + 4, 6, 11, 18, 27, 37, 46, 53, + 11, 18, 27, 37, 46, 53, 58, 60, + 27, 37, 46, 53, 58, 60, 62, 63, + 46, 53, 58, 60, 62, 63, 63, 64, + 58, 60, 62, 63, 63, 64, 64, 64, + ]; + + byte[] generatedWedgeMask = new byte[blockSize * blockSize]; + Av1WedgeMask.Fill( + generatedWedgeMask, + blockSize, + Av1BlockSize.Block8x8, + wedgeIndex: 0, + wedgeSign: true, + subX: 0, + subY: 0, + invert: false); + + Assert.Equal(wedgeMask, generatedWedgeMask); + + Av1CompoundIntermediateMaskBlendPredictor.BlendIntermediate( + expected, + blockSize, + expectedFirst, + blockSize, + expectedSecond, + blockSize, + wedgeMask, + blockSize, + blockSize, + blockSize, + subX: 0, + subY: 0, + bitDepth.GetBitCount()); + + Assert.NotEqual((ushort)60, expected[0]); + } + else + { + Av1CompoundIntermediateAveragePredictor.AverageIntermediate( + expected, + blockSize, + expectedFirst, + blockSize, + expectedSecond, + blockSize, + blockSize, + blockSize, + bitDepth.GetBitCount()); + + Assert.Equal((ushort)60, expected[0]); + } + + using Av1FrameBuffer frameBuffer = new( + Configuration.Default, + sequenceHeader, + Av1ColorFormat.Yuv400, + false); + + using Av1FrameInfo frameInfo = new(sequenceHeader); + Av1SuperblockInfo superblockInfo = frameInfo.GetSuperblock(Point.Empty); + superblockInfo.GetTransformInfoY()[0] = new Av1TransformInfo(Av1TransformSize.Size8x8, 0, 0); + using Av1LoopFilterContext loopFilterContext = + new(frameBuffer.MemoryAllocator, sequenceHeader, frameHeader); + + using Av1BlockDecoder decoder = new( + sequenceHeader, + frameHeader, + frameBuffer, + loopFilterContext, + referenceFrames); + + decoder.UpdateSuperblock(superblockInfo); + decoder.DecodeBlock( + modeInfo, + new Point(2, 2), + Av1BlockSize.Block8x8, + superblockInfo, + new Av1TileInfo(0, 0, frameHeader)); + + for (int row = 0; row < blockSize; row++) + { + Span actual = frameBuffer.GetHighBitDepthRowSpan(Av1Plane.Y, blockOrigin + row, 0, 0); + Assert.Equal( + expected.AsSpan(row * blockSize, blockSize), + actual.Slice(blockOrigin, blockSize)); + } + } + + /// + /// Reconstructs one compound global-warp block and compares it with direct scalar compound predictors. + /// + /// The native sample depth. + private static void ValidateCompoundGlobalWarpPredictionAtBitDepth(Av1BitDepth bitDepth) + { + const int frameSize = 32; + const int blockOrigin = 8; + const int blockSize = 8; + int bitDepthValue = bitDepth.GetBitCount(); + int intermediateRange = bitDepthValue + Av1TranslationalInterPredictor.FilterBits - Av1TranslationalInterPredictor.Round0Bits + 2; + int round0 = Av1TranslationalInterPredictor.Round0Bits + Math.Max(intermediateRange - 16, 0); + int compoundRoundBits = + (2 * Av1TranslationalInterPredictor.FilterBits) - round0 - Av1CompoundInterPredictor.CompoundRound1Bits; + + int compoundOffsetBits = + bitDepthValue + + (2 * Av1TranslationalInterPredictor.FilterBits) - + round0 - + Av1CompoundInterPredictor.CompoundRound1Bits; + + int compoundRoundOffset = (1 << compoundOffsetBits) + (1 << (compoundOffsetBits - 1)); + int maximum = (1 << bitDepthValue) - 1; + ObuSequenceHeader sequenceHeader = CreateSequenceHeader(bitDepth, frameSize); + ObuFrameHeader frameHeader = CreateFrameHeader(frameSize); + frameHeader.GetReferenceFrameIndices()[0] = 0; + frameHeader.GetReferenceFrameIndices()[1] = 1; + + Av1GlobalMotionParameters globalMotionParameters = Av1GlobalMotionParameters.Identity; + globalMotionParameters.Type = Av1GlobalMotionType.RotationZoom; + globalMotionParameters[0] = -357376; + globalMotionParameters[1] = 372736; + globalMotionParameters[2] = 65468; + globalMotionParameters[3] = 2856; + globalMotionParameters[4] = -2856; + globalMotionParameters[5] = 65468; + globalMotionParameters.UpdateShearParameters(); + Assert.False(globalMotionParameters.IsInvalid); + frameHeader.GetGlobalMotionParameters()[0] = globalMotionParameters; + frameHeader.GetGlobalMotionParameters()[1] = globalMotionParameters; + + using Av1ReferenceFrameStore referenceFrames = new(); + Assert.True(referenceFrames.Commit( + 1, + CreatePatternReferenceFrame(sequenceHeader, CreateFrameHeader(frameSize)), + showFrame: false)); + + Assert.True(referenceFrames.Commit( + 2, + CreatePatternReferenceFrame(sequenceHeader, CreateFrameHeader(frameSize), sampleOffset: 40), + showFrame: false)); + + Av1FrameBuffer firstReference = referenceFrames.ResolveRequired(0).FrameBuffer; + Av1FrameBuffer secondReference = referenceFrames.ResolveRequired(1).FrameBuffer; + ushort[] firstHighBitDepthPrediction = new ushort[blockSize * blockSize]; + ushort[] secondHighBitDepthPrediction = new ushort[blockSize * blockSize]; + short[] firstScratch = new short[Av1WarpedInterPredictor.WarpedScratchLength]; + short[] secondScratch = new short[Av1WarpedInterPredictor.WarpedScratchLength]; + Point blockPosition = new(blockOrigin, blockOrigin); + if (bitDepth == Av1BitDepth.EightBit) + { + Span firstSource = firstReference.GetPaddedPlaneSpan( + Av1Plane.Y, + 0, + 0, + out int firstStride, + out Point firstOrigin); + + Span secondSource = secondReference.GetPaddedPlaneSpan( + Av1Plane.Y, + 0, + 0, + out int secondStride, + out Point secondOrigin); + + Av1WarpedInterPredictor.PredictWarpedCompoundScalar( + firstSource, + firstStride, + firstOrigin, + frameSize, + frameSize, + firstHighBitDepthPrediction, + blockSize, + blockPosition, + blockSize, + blockSize, + 0, + 0, + globalMotionParameters, + firstScratch); + + Av1WarpedInterPredictor.PredictWarpedCompoundScalar( + secondSource, + secondStride, + secondOrigin, + frameSize, + frameSize, + secondHighBitDepthPrediction, + blockSize, + blockPosition, + blockSize, + blockSize, + 0, + 0, + globalMotionParameters, + secondScratch); + } + else + { + Span firstSource = firstReference.GetPaddedPlaneSpan16( + Av1Plane.Y, + 0, + 0, + out int firstStride, + out Point firstOrigin); + + Span secondSource = secondReference.GetPaddedPlaneSpan16( + Av1Plane.Y, + 0, + 0, + out int secondStride, + out Point secondOrigin); + + Av1WarpedInterPredictor.PredictWarpedCompoundScalar( + firstSource, + firstStride, + firstOrigin, + frameSize, + frameSize, + firstHighBitDepthPrediction, + blockSize, + blockPosition, + blockSize, + blockSize, + 0, + 0, + bitDepthValue, + globalMotionParameters, + firstScratch); + + Av1WarpedInterPredictor.PredictWarpedCompoundScalar( + secondSource, + secondStride, + secondOrigin, + frameSize, + frameSize, + secondHighBitDepthPrediction, + blockSize, + blockPosition, + blockSize, + blockSize, + 0, + 0, + bitDepthValue, + globalMotionParameters, + secondScratch); + } + + using Av1FrameBuffer frameBuffer = new( + Configuration.Default, + sequenceHeader, + Av1ColorFormat.Yuv400, + false); + + using Av1FrameInfo frameInfo = new(sequenceHeader); + Av1SuperblockInfo superblockInfo = frameInfo.GetSuperblock(Point.Empty); + superblockInfo.GetTransformInfoY()[0] = new Av1TransformInfo(Av1TransformSize.Size8x8, 0, 0); + + Av1BlockModeInfo modeInfo = new(Av1BlockSize.Block8x8, new Point(2, 2)) + { + Skip = true, + YMode = Av1PredictionMode.GlobalGlobalMotionVector, + CompoundIndex = true, + CompoundType = Av1CompoundType.Average, + }; + + modeInfo.ReferenceFrames[0] = Av1ReferenceFrameType.Last; + modeInfo.ReferenceFrames[1] = Av1ReferenceFrameType.Last2; + modeInfo.InterpolationFilters.Clear(); + modeInfo.SetTransformUnitCount(Av1PlaneType.Y, 1); + + using Av1LoopFilterContext loopFilterContext = + new(frameBuffer.MemoryAllocator, sequenceHeader, frameHeader); + + using Av1BlockDecoder decoder = new( + sequenceHeader, + frameHeader, + frameBuffer, + loopFilterContext, + referenceFrames); + + decoder.UpdateSuperblock(superblockInfo); + decoder.DecodeBlock( + modeInfo, + new Point(2, 2), + Av1BlockSize.Block8x8, + superblockInfo, + new Av1TileInfo(0, 0, frameHeader)); + + for (int row = 0; row < blockSize; row++) + { + for (int column = 0; column < blockSize; column++) + { + int predictionIndex = (row * blockSize) + column; + + // The reference decoder truncates the equal average before removing the compound bias, then performs the sole final + // rounding step. Reconstructing each reference to native pixels first can differ from this result. + int intermediate = ((firstHighBitDepthPrediction[predictionIndex] + + secondHighBitDepthPrediction[predictionIndex]) >> 1) - compoundRoundOffset; + + int expected = Math.Clamp( + (intermediate + (1 << (compoundRoundBits - 1))) >> compoundRoundBits, + 0, + maximum); + + if (bitDepth == Av1BitDepth.EightBit) + { + Span samples = frameBuffer.DeriveBlockPointer(Av1Plane.Y, 0, 0).DangerousGetRowSpan(blockOrigin + row); + Assert.Equal((byte)expected, samples[blockOrigin + column]); + } + else + { + Span samples = frameBuffer.GetHighBitDepthRowSpan(Av1Plane.Y, blockOrigin + row, 0, 0); + Assert.Equal((ushort)expected, samples[blockOrigin + column]); + } + } + } + } + + /// + /// Creates one retained frame whose integer-coordinate luma samples make both OBMC axes observable. + /// + private static Av1ReferenceFrame CreatePatternReferenceFrame( + ObuSequenceHeader sequenceHeader, + ObuFrameHeader frameHeader, + Av1ColorFormat colorFormat = Av1ColorFormat.Yuv400, + int sampleOffset = 0) + { + Av1FrameBuffer frameBuffer = new( + Configuration.Default, + sequenceHeader, + colorFormat, + false); + + for (int plane = 0; plane < sequenceHeader.ColorConfig.PlaneCount; plane++) + { + int subX = plane > 0 && sequenceHeader.ColorConfig.SubSamplingX ? 1 : 0; + int subY = plane > 0 && sequenceHeader.ColorConfig.SubSamplingY ? 1 : 0; + int planeWidth = sequenceHeader.MaxFrameWidth >> subX; + int planeHeight = sequenceHeader.MaxFrameHeight >> subY; + for (int row = 0; row < planeHeight; row++) + { + if (sequenceHeader.ColorConfig.BitDepth == Av1BitDepth.EightBit) + { + Span samples = frameBuffer.DeriveBlockPointer((Av1Plane)plane, subX, subY).DangerousGetRowSpan(row); + for (int column = 0; column < planeWidth; column++) + { + samples[column] = (byte)(GetPlanePatternValue(plane, column, row) + sampleOffset); + } + } + else + { + Span samples = frameBuffer.GetHighBitDepthRowSpan((Av1Plane)plane, row, subX, subY); + for (int column = 0; column < planeWidth; column++) + { + samples[column] = (ushort)(GetPlanePatternValue(plane, column, row) + sampleOffset); + } + } + } + } + + Av1ReferenceFrameBorder.Extend(frameBuffer); + using Av1FrameInfo frameInfo = new(sequenceHeader); + return new Av1ReferenceFrame(frameBuffer, frameHeader, frameInfo); + } + + /// + /// Creates a 16x16 retained frame whose deterministic pattern exposes scaled bilinear precision. + /// + /// The sequence dimensions and sample precision. + /// The horizontal linear contribution. + /// The vertical linear contribution. + /// The contribution that makes neighboring interpolation errors differ. + private static Av1ReferenceFrame CreateScaledPatternReferenceFrame( + ObuSequenceHeader sequenceHeader, + int horizontalScale = 2, + int verticalScale = 8, + int nonlinearScale = 0) + { + Av1FrameBuffer frameBuffer = new( + Configuration.Default, + sequenceHeader, + Av1ColorFormat.Yuv400, + false); + + frameBuffer.Width = 16; + frameBuffer.Height = 16; + for (int row = 0; row < 16; row++) + { + if (sequenceHeader.ColorConfig.BitDepth == Av1BitDepth.EightBit) + { + Span samples = frameBuffer.DeriveBlockPointer(Av1Plane.Y, 0, 0).DangerousGetRowSpan(row); + for (int column = 0; column < 16; column++) + { + samples[column] = (byte)( + (column * horizontalScale) + + (row * verticalScale) + + (((column * row) & 7) * nonlinearScale)); + } + } + else + { + Span samples = frameBuffer.GetHighBitDepthRowSpan(Av1Plane.Y, row, 0, 0); + for (int column = 0; column < 16; column++) + { + samples[column] = (ushort)( + (column * horizontalScale) + + (row * verticalScale) + + (((column * row) & 7) * nonlinearScale)); + } + } + } + + Av1ReferenceFrameBorder.Extend(frameBuffer); + using Av1FrameInfo frameInfo = new(sequenceHeader); + return new Av1ReferenceFrame(frameBuffer, CreateFrameHeader(16), frameInfo); + } + + /// + /// Gets the deterministic luma value stored at one reference-frame coordinate. + /// + private static int GetPatternValue(int column, int row) => column + (row * 4); + + /// + /// Gets the deterministic plane value stored at one reference-frame coordinate. + /// + private static int GetPlanePatternValue(int plane, int column, int row) => GetPatternValue(column, row) + (plane * 20); + + /// + /// Creates one independently owned retained frame filled with a constant visible luma value. + /// + private static Av1ReferenceFrame CreateReferenceFrame(ObuSequenceHeader sequenceHeader, ushort value) + { + Av1FrameBuffer frameBuffer = new( + Configuration.Default, + sequenceHeader, + Av1ColorFormat.Yuv400, + false); + + if (sequenceHeader.ColorConfig.BitDepth == Av1BitDepth.EightBit) + { + for (int row = 0; row < 8; row++) + { + frameBuffer.DeriveBlockPointer(Av1Plane.Y, 0, 0).DangerousGetRowSpan(row).Fill((byte)value); + } + } + else + { + for (int row = 0; row < 8; row++) + { + frameBuffer.GetHighBitDepthRowSpan(Av1Plane.Y, row, 0, 0).Fill(value); + } + } + + using Av1FrameInfo frameInfo = new(sequenceHeader); + return new Av1ReferenceFrame(frameBuffer, CreateFrameHeader(), frameInfo); + } + + /// + /// Creates the monochrome 8x8 sequence used by direct reconstruction tests. + /// + private static ObuSequenceHeader CreateSequenceHeader( + Av1BitDepth bitDepth, + int frameSize = 8, + Av1ColorFormat colorFormat = Av1ColorFormat.Yuv400) + => new() + { + MaxFrameWidth = frameSize, + MaxFrameHeight = frameSize, + Use128x128Superblock = false, + ColorConfig = new ObuColorConfig + { + IsMonochrome = colorFormat == Av1ColorFormat.Yuv400, + BitDepth = bitDepth, + SubSamplingX = colorFormat is Av1ColorFormat.Yuv420 or Av1ColorFormat.Yuv422, + SubSamplingY = colorFormat == Av1ColorFormat.Yuv420, + }, + }; + + /// + /// Creates an unscaled 8x8 inter-frame header with one complete tile. + /// + private static ObuFrameHeader CreateFrameHeader(int frameSize = 8) + { + ObuFrameHeader frameHeader = new() + { + FrameType = ObuFrameType.InterFrame, + ModeInfoColumnCount = frameSize >> Av1Constants.ModeInfoSizeLog2, + ModeInfoRowCount = frameSize >> Av1Constants.ModeInfoSizeLog2, + FrameSize = new ObuFrameSize + { + FrameWidth = frameSize, + FrameHeight = frameSize, + }, + }; + + frameHeader.TilesInfo.TileColumnStartModeInfo[1] = frameHeader.ModeInfoColumnCount; + frameHeader.TilesInfo.TileRowStartModeInfo[1] = frameHeader.ModeInfoRowCount; + return frameHeader; + } +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1CompoundInterPredictorTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1CompoundInterPredictorTests.cs new file mode 100644 index 0000000000..7a4178b226 --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1CompoundInterPredictorTests.cs @@ -0,0 +1,911 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; +using SixLabors.ImageSharp.Tests.TestUtilities; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1; + +/// +/// Verifies AV1 compound prediction and blending across every hardware-intrinsic tier. +/// +[Trait("Format", "Avif")] +public class Av1CompoundInterPredictorTests +{ + /// + /// Exercises the native vector width, 256-bit and 128-bit paths, and the complete scalar fallback. + /// + private const HwIntrinsics PredictorConfigurations = + HwIntrinsics.AllowAll | HwIntrinsics.DisableAVX512F | HwIntrinsics.DisableAVX | HwIntrinsics.DisableHWIntrinsic; + + /// + /// Verifies rounded 8-bit averaging, scalar tails, and untouched row padding under every SIMD configuration. + /// + [Fact] + public void ByteAverageMatchesReference() + => FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidateByteAverage, PredictorConfigurations); + + /// + /// Verifies rounded 10/12-bit averaging, scalar tails, and untouched row padding under every SIMD configuration. + /// + [Fact] + public void HighBitDepthAverageMatchesReference() + => FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidateHighBitDepthAverage, PredictorConfigurations); + + /// + /// Verifies 10/12-bit no-round prediction and compound finalization across every intrinsic width. + /// + [Fact] + public void HighBitDepthIntermediatesMatchReference() + => FeatureTestRunner.RunWithHwIntrinsicsFeature( + ValidateHighBitDepthCompoundIntermediates, + PredictorConfigurations); + + /// + /// Verifies every display-distance quantization class in both temporal directions. + /// + /// The first reference order hint. + /// The second reference order hint. + /// The expected first predictor weight. + /// The expected second predictor weight. + [Theory] + [InlineData(13, 20, 9, 7)] + [InlineData(15, 18, 11, 5)] + [InlineData(15, 19, 12, 4)] + [InlineData(15, 20, 13, 3)] + [InlineData(12, 19, 7, 9)] + [InlineData(14, 17, 5, 11)] + [InlineData(13, 17, 4, 12)] + [InlineData(12, 17, 3, 13)] + [InlineData(12, 16, 3, 13)] + [InlineData(16, 20, 13, 3)] + public void DistanceWeightsMatchReference( + int firstOrderHint, + int secondOrderHint, + int expectedFirstWeight, + int expectedSecondWeight) + { + ObuOrderHintInfo orderHintInfo = new() + { + EnableOrderHint = true, + OrderHintBits = 5, + }; + + ObuFrameHeader frameHeader = new() { OrderHint = 16 }; + frameHeader.GetReferenceFrameIndices()[0] = 0; + frameHeader.GetReferenceFrameIndices()[1] = 1; + frameHeader.GetReferenceOrderHints()[0] = (uint)firstOrderHint; + frameHeader.GetReferenceOrderHints()[1] = (uint)secondOrderHint; + + Av1CompoundDistanceWeights.Derive( + orderHintInfo, + frameHeader, + Av1ReferenceFrameType.Last, + Av1ReferenceFrameType.Last2, + out int firstWeight, + out int secondWeight); + + Assert.Equal(expectedFirstWeight, firstWeight); + Assert.Equal(expectedSecondWeight, secondWeight); + } + + /// + /// Verifies 8-bit distance and per-sample mask blending across every intrinsic width and scalar tail. + /// + [Fact] + public void ByteSelectableBlendsMatchReference() + => FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidateByteSelectableBlends, PredictorConfigurations); + + /// + /// Verifies 10/12-bit distance and per-sample mask blending across every intrinsic width and scalar tail. + /// + [Fact] + public void HighBitDepthSelectableBlendsMatchReference() + => FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidateHighBitDepthSelectableBlends, PredictorConfigurations); + + /// + /// Verifies the four smooth inter-intra modes and their complemented destination orientation. + /// + [Fact] + public void SmoothInterIntraMasksMatchReference() + { + ReadOnlySpan weights = [60, 34, 19, 11, 6, 4, 2, 1]; + + foreach (Av1InterIntraMode mode in Enum.GetValues()) + { + const int width = 8; + const int height = 4; + const int stride = 11; + byte[] mask = new byte[stride * height]; + byte[] inverted = new byte[stride * height]; + mask.AsSpan().Fill(0xA5); + inverted.AsSpan().Fill(0xA5); + + Av1InterIntraMaskBuilder.FillInterIntraMask(mask, stride, width, height, mode, invert: false); + Av1InterIntraMaskBuilder.FillInterIntraMask(inverted, stride, width, height, mode, invert: true); + + for (int row = 0; row < height; row++) + { + for (int column = 0; column < width; column++) + { + byte expected = mode switch + { + Av1InterIntraMode.Vertical => weights[row], + Av1InterIntraMode.Horizontal => weights[column], + Av1InterIntraMode.Smooth => weights[Math.Min(row, column)], + _ => 32, + }; + + Assert.Equal(expected, mask[(row * stride) + column]); + Assert.Equal((byte)(64 - expected), inverted[(row * stride) + column]); + } + + for (int column = width; column < stride; column++) + { + Assert.Equal(0xA5, mask[(row * stride) + column]); + Assert.Equal(0xA5, inverted[(row * stride) + column]); + } + } + } + } + + /// + /// Verifies the reference horizontal curve at the index exercised by a 32-by-16 inter-intra block. + /// + [Fact] + public void HorizontalInterIntraMaskMatchesReference() + { + const int width = 32; + const int height = 16; + byte[] mask = new byte[width * height]; + byte[] inverted = new byte[width * height]; + + Av1InterIntraMaskBuilder.FillInterIntraMask(mask, width, width, height, Av1InterIntraMode.Horizontal, invert: false); + Av1InterIntraMaskBuilder.FillInterIntraMask(inverted, width, width, height, Av1InterIntraMode.Horizontal, invert: true); + + Assert.Equal(2, mask[23]); + Assert.Equal(62, inverted[23]); + } + + /// + /// Verifies the reference difference-mask formula in both orientations at each supported bit depth. + /// + [Fact] + public void DifferenceWeightedMasksMatchReference() + => FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidateDifferenceWeightedMasks, PredictorConfigurations); + + /// + /// Applies the independent difference-mask formula at every bit depth and intrinsic width. + /// + private static void ValidateDifferenceWeightedMasks() + { + ReadOnlySpan widths = [9, 16, 23, 32, 47, 64, 127]; + + foreach (int width in widths) + { + const int height = 3; + int firstStride = width + 4; + int secondStride = width + 2; + int maskStride = width + 3; + foreach (int bitDepth in new[] { 8, 10, 12 }) + { + int sampleMask = (1 << bitDepth) - 1; + ushort[] first = new ushort[firstStride * height]; + ushort[] second = new ushort[secondStride * height]; + for (int row = 0; row < height; row++) + { + for (int column = 0; column < width; column++) + { + first[(row * firstStride) + column] = (ushort)(((row * 911) + (column * 521)) & sampleMask); + second[(row * secondStride) + column] = (ushort)(((row * 307) + (column * 997) + 31) & sampleMask); + } + } + + foreach (Av1DifferenceWeightedMaskType maskType in Enum.GetValues()) + { + byte[] actual = new byte[maskStride * height]; + actual.AsSpan().Fill(0xA5); + + if (bitDepth == 8) + { + byte[] firstByte = Array.ConvertAll(first, value => (byte)value); + byte[] secondByte = Array.ConvertAll(second, value => (byte)value); + Av1DifferenceWeightedMaskBuilder.FillDifferenceWeightedMask( + actual, + maskStride, + firstByte, + firstStride, + secondByte, + secondStride, + width, + height, + maskType); + } + else + { + Av1DifferenceWeightedMaskBuilder.FillDifferenceWeightedMask( + actual, + maskStride, + first, + firstStride, + second, + secondStride, + width, + height, + bitDepth, + maskType); + } + + for (int row = 0; row < height; row++) + { + for (int column = 0; column < width; column++) + { + int difference = Math.Abs(first[(row * firstStride) + column] - second[(row * secondStride) + column]); + int alpha = Math.Min(64, 38 + ((difference >> (bitDepth - 8)) / 16)); + byte expected = (byte)(maskType == Av1DifferenceWeightedMaskType.Type38Inverse ? 64 - alpha : alpha); + + Assert.Equal(expected, actual[(row * maskStride) + column]); + } + + for (int column = width; column < maskStride; column++) + { + Assert.Equal(0xA5, actual[(row * maskStride) + column]); + } + } + } + } + } + } + + /// + /// Applies independent byte arithmetic to block widths that cross every vector and scalar boundary. + /// + private static void ValidateByteAverage() + { + ReadOnlySpan widths = [4, 7, 8, 15, 16, 23, 31, 32, 47, 64, 127, 128]; + + foreach (int width in widths) + { + const int height = 5; + int destinationStride = width + 11; + int secondStride = width + 7; + byte[] expected = new byte[destinationStride * height]; + byte[] actual = new byte[destinationStride * height]; + byte[] scalar = new byte[destinationStride * height]; + byte[] second = new byte[secondStride * height]; + + FillByteInputs(expected, second, destinationStride, secondStride, width, height); + expected.CopyTo(actual, 0); + expected.CopyTo(scalar, 0); + + for (int row = 0; row < height; row++) + { + for (int column = 0; column < width; column++) + { + int destinationIndex = (row * destinationStride) + column; + int secondIndex = (row * secondStride) + column; + expected[destinationIndex] = (byte)((expected[destinationIndex] + second[secondIndex] + 1) >> 1); + } + } + + Av1CompoundAveragePredictor.Average(actual, destinationStride, second, secondStride, width, height); + Av1CompoundAveragePredictor.AverageScalar(scalar, destinationStride, second, secondStride, width, height); + + Assert.Equal(expected, actual); + Assert.Equal(expected, scalar); + } + } + + /// + /// Applies independent ushort arithmetic at both supported high-bit-depth limits. + /// + private static void ValidateHighBitDepthAverage() + { + ReadOnlySpan widths = [4, 7, 8, 15, 16, 23, 31, 32, 47, 64, 127, 128]; + + foreach (int bitDepth in new[] { 10, 12 }) + { + foreach (int width in widths) + { + const int height = 5; + int destinationStride = width + 9; + int secondStride = width + 5; + ushort[] expected = new ushort[destinationStride * height]; + ushort[] actual = new ushort[destinationStride * height]; + ushort[] scalar = new ushort[destinationStride * height]; + ushort[] second = new ushort[secondStride * height]; + + FillHighBitDepthInputs(expected, second, destinationStride, secondStride, width, height, bitDepth); + expected.CopyTo(actual, 0); + expected.CopyTo(scalar, 0); + + for (int row = 0; row < height; row++) + { + for (int column = 0; column < width; column++) + { + int destinationIndex = (row * destinationStride) + column; + int secondIndex = (row * secondStride) + column; + expected[destinationIndex] = (ushort)((expected[destinationIndex] + second[secondIndex] + 1) >> 1); + } + } + + Av1CompoundAveragePredictor.Average(actual, destinationStride, second, secondStride, width, height); + Av1CompoundAveragePredictor.AverageScalar(scalar, destinationStride, second, secondStride, width, height); + + Assert.Equal(expected, actual); + Assert.Equal(expected, scalar); + } + } + } + + /// + /// Applies the high-bit-depth no-round convolution equations independently of the production operators. + /// + private static void ValidateHighBitDepthCompoundIntermediates() + { + ReadOnlySpan widths = [9, 17, 33, 65]; + ReadOnlySpan<(int Horizontal, int Vertical)> phases = + [(0, 0), (5, 0), (0, 9), (5, 9)]; + + foreach (int bitDepth in new[] { 10, 12 }) + { + int maximum = (1 << bitDepth) - 1; + int intermediateRange = bitDepth + 7 - 3 + 2; + int round0 = 3 + Math.Max(intermediateRange - 16, 0); + int roundBits = 14 - round0 - 7; + int offsetBits = bitDepth + 14 - round0; + int roundOffset = (1 << (offsetBits - 7)) + (1 << (offsetBits - 8)); + + foreach (int width in widths) + { + const int height = 3; + int sourceStride = width + 5; + int intermediateStride = width + 3; + int destinationStride = width + 7; + ushort[] firstSource = new ushort[sourceStride * (height + 1)]; + ushort[] secondSource = new ushort[sourceStride * (height + 1)]; + + for (int row = 0; row <= height; row++) + { + for (int column = 0; column < sourceStride; column++) + { + firstSource[(row * sourceStride) + column] = + (ushort)(((row * 613) + (column * 349) + 17) & maximum); + + secondSource[(row * sourceStride) + column] = + (ushort)(((row * 947) + (column * 181) + 71) & maximum); + } + } + + foreach ((int horizontalPhase, int verticalPhase) in phases) + { + int horizontal0 = 128 - (horizontalPhase * 8); + int horizontal1 = horizontalPhase * 8; + int vertical0 = 128 - (verticalPhase * 8); + int vertical1 = verticalPhase * 8; + ushort[] expectedFirst = new ushort[intermediateStride * height]; + ushort[] expectedSecond = new ushort[intermediateStride * height]; + ushort[] actualFirst = new ushort[intermediateStride * height]; + ushort[] actualSecond = new ushort[intermediateStride * height]; + ushort[] scalarFirst = new ushort[intermediateStride * height]; + ushort[] scalarSecond = new ushort[intermediateStride * height]; + expectedFirst.AsSpan().Fill(0xA5A5); + expectedSecond.AsSpan().Fill(0xA5A5); + actualFirst.AsSpan().Fill(0xA5A5); + actualSecond.AsSpan().Fill(0xA5A5); + scalarFirst.AsSpan().Fill(0xA5A5); + scalarSecond.AsSpan().Fill(0xA5A5); + + for (int predictorIndex = 0; predictorIndex < 2; predictorIndex++) + { + ReadOnlySpan source = predictorIndex == 0 ? firstSource : secondSource; + Span expected = predictorIndex == 0 ? expectedFirst : expectedSecond; + + for (int row = 0; row < height; row++) + { + for (int column = 0; column < width; column++) + { + int sourceIndex = (row * sourceStride) + column; + int result; + if (horizontalPhase == 0 && verticalPhase == 0) + { + result = (source[sourceIndex] << roundBits) + roundOffset; + } + else if (verticalPhase == 0) + { + int sum = (horizontal0 * source[sourceIndex]) + + (horizontal1 * source[sourceIndex + 1]); + + result = ((sum + (1 << (round0 - 1))) >> round0) + roundOffset; + } + else if (horizontalPhase == 0) + { + int sum = (vertical0 * source[sourceIndex]) + + (vertical1 * source[sourceIndex + sourceStride]); + + int shifted = sum << (7 - round0); + result = ((shifted + 64) >> 7) + roundOffset; + } + else + { + int horizontalBias = 1 << (bitDepth + 6); + int firstHorizontal = horizontalBias + + (horizontal0 * source[sourceIndex]) + + (horizontal1 * source[sourceIndex + 1]); + + int secondHorizontal = horizontalBias + + (horizontal0 * source[sourceIndex + sourceStride]) + + (horizontal1 * source[sourceIndex + sourceStride + 1]); + + firstHorizontal = (firstHorizontal + (1 << (round0 - 1))) >> round0; + secondHorizontal = (secondHorizontal + (1 << (round0 - 1))) >> round0; + int verticalBias = 1 << (bitDepth + 14 - round0); + int vertical = verticalBias + + (vertical0 * firstHorizontal) + + (vertical1 * secondHorizontal); + + result = (vertical + 64) >> 7; + } + + expected[(row * intermediateStride) + column] = (ushort)result; + } + } + } + + int scratchStride = Math.Max(width, 128); + short[] scratch = new short[scratchStride * (height + 8)]; + Av1CompoundInterPredictor.PredictCompound( + firstSource, + sourceStride, + sourceOrigin: 0, + actualFirst, + intermediateStride, + width, + height, + Av1InterpolationFilter.Bilinear, + Av1InterpolationFilter.Bilinear, + horizontalPhase, + verticalPhase, + bitDepth, + scratch); + + Av1CompoundInterPredictor.PredictCompound( + secondSource, + sourceStride, + sourceOrigin: 0, + actualSecond, + intermediateStride, + width, + height, + Av1InterpolationFilter.Bilinear, + Av1InterpolationFilter.Bilinear, + horizontalPhase, + verticalPhase, + bitDepth, + scratch); + + Av1CompoundInterPredictor.PredictCompoundScalar( + firstSource, + sourceStride, + sourceOrigin: 0, + scalarFirst, + intermediateStride, + width, + height, + Av1InterpolationFilter.Bilinear, + Av1InterpolationFilter.Bilinear, + horizontalPhase, + verticalPhase, + bitDepth, + scratch); + + Av1CompoundInterPredictor.PredictCompoundScalar( + secondSource, + sourceStride, + sourceOrigin: 0, + scalarSecond, + intermediateStride, + width, + height, + Av1InterpolationFilter.Bilinear, + Av1InterpolationFilter.Bilinear, + horizontalPhase, + verticalPhase, + bitDepth, + scratch); + + Assert.Equal(expectedFirst, actualFirst); + Assert.Equal(expectedSecond, actualSecond); + Assert.Equal(expectedFirst, scalarFirst); + Assert.Equal(expectedSecond, scalarSecond); + + ushort[] expectedDestination = new ushort[destinationStride * height]; + ushort[] actualDestination = new ushort[destinationStride * height]; + expectedDestination.AsSpan().Fill(0xA5A5); + actualDestination.AsSpan().Fill(0xA5A5); + + for (int row = 0; row < height; row++) + { + for (int column = 0; column < width; column++) + { + int intermediateIndex = (row * intermediateStride) + column; + int result = ((expectedFirst[intermediateIndex] + expectedSecond[intermediateIndex]) >> 1) - + roundOffset; + + result = (result + (1 << (roundBits - 1))) >> roundBits; + expectedDestination[(row * destinationStride) + column] = + (ushort)Math.Clamp(result, 0, maximum); + } + } + + Av1CompoundIntermediateAveragePredictor.AverageIntermediate( + actualDestination, + destinationStride, + actualFirst, + intermediateStride, + actualSecond, + intermediateStride, + width, + height, + bitDepth); + + Assert.Equal(expectedDestination, actualDestination); + + ReadOnlySpan distanceWeights = [9, 7, 11, 5, 12, 4, 13, 3]; + for (int weightIndex = 0; weightIndex < distanceWeights.Length; weightIndex += 2) + { + ushort[] expectedWeighted = new ushort[destinationStride * height]; + ushort[] actualWeighted = new ushort[destinationStride * height]; + expectedWeighted.AsSpan().Fill(0xA5A5); + actualWeighted.AsSpan().Fill(0xA5A5); + int firstWeight = distanceWeights[weightIndex]; + int secondWeight = distanceWeights[weightIndex + 1]; + + for (int row = 0; row < height; row++) + { + for (int column = 0; column < width; column++) + { + int intermediateIndex = (row * intermediateStride) + column; + int result = ((expectedFirst[intermediateIndex] * firstWeight) + + (expectedSecond[intermediateIndex] * secondWeight)) >> 4; + + result -= roundOffset; + result = (result + (1 << (roundBits - 1))) >> roundBits; + expectedWeighted[(row * destinationStride) + column] = + (ushort)Math.Clamp(result, 0, maximum); + } + } + + Av1CompoundIntermediateDistanceWeightedPredictor.DistanceWeightedIntermediate( + actualWeighted, + destinationStride, + actualFirst, + intermediateStride, + actualSecond, + intermediateStride, + width, + height, + firstWeight, + secondWeight, + bitDepth); + + Assert.Equal(expectedWeighted, actualWeighted); + } + + int maskStride = width + 5; + byte[] mask = new byte[maskStride * height]; + ushort[] expectedMasked = new ushort[destinationStride * height]; + ushort[] actualMasked = new ushort[destinationStride * height]; + expectedMasked.AsSpan().Fill(0xA5A5); + actualMasked.AsSpan().Fill(0xA5A5); + + for (int row = 0; row < height; row++) + { + for (int column = 0; column < width; column++) + { + byte alpha = (byte)(((row * 29) + (column * 17) + 3) % 65); + mask[(row * maskStride) + column] = alpha; + int intermediateIndex = (row * intermediateStride) + column; + int result = ((alpha * expectedFirst[intermediateIndex]) + + ((64 - alpha) * expectedSecond[intermediateIndex])) >> 6; + + result -= roundOffset; + result = (result + (1 << (roundBits - 1))) >> roundBits; + expectedMasked[(row * destinationStride) + column] = + (ushort)Math.Clamp(result, 0, maximum); + } + } + + Av1CompoundIntermediateMaskBlendPredictor.BlendIntermediate( + actualMasked, + destinationStride, + actualFirst, + intermediateStride, + actualSecond, + intermediateStride, + mask, + maskStride, + width, + height, + subX: 0, + subY: 0, + bitDepth); + + Assert.Equal(expectedMasked, actualMasked); + + int differenceRound = roundBits + bitDepth - 8; + foreach (Av1DifferenceWeightedMaskType maskType in Enum.GetValues()) + { + byte[] expectedDifferenceMask = new byte[maskStride * height]; + byte[] actualDifferenceMask = new byte[maskStride * height]; + expectedDifferenceMask.AsSpan().Fill(0xA5); + actualDifferenceMask.AsSpan().Fill(0xA5); + + for (int row = 0; row < height; row++) + { + for (int column = 0; column < width; column++) + { + int intermediateIndex = (row * intermediateStride) + column; + int difference = Math.Abs( + expectedFirst[intermediateIndex] - expectedSecond[intermediateIndex]); + + difference = (difference + (1 << (differenceRound - 1))) >> differenceRound; + int alpha = Math.Min(64, 38 + (difference >> 4)); + if (maskType == Av1DifferenceWeightedMaskType.Type38Inverse) + { + alpha = 64 - alpha; + } + + expectedDifferenceMask[(row * maskStride) + column] = (byte)alpha; + } + } + + Av1CompoundIntermediateDifferenceWeightedMaskBuilder.FillDifferenceWeightedIntermediateMask( + actualDifferenceMask, + maskStride, + actualFirst, + intermediateStride, + actualSecond, + intermediateStride, + width, + height, + bitDepth, + maskType); + + Assert.Equal(expectedDifferenceMask, actualDifferenceMask); + } + } + } + } + } + + /// + /// Applies independent byte arithmetic to every selectable compound blend. + /// + private static void ValidateByteSelectableBlends() + { + ReadOnlySpan widths = [4, 7, 8, 15, 16, 23, 31, 32, 47, 64, 127, 128]; + ReadOnlySpan distanceWeights = [9, 7, 11, 5, 12, 4, 13, 3]; + + foreach (int width in widths) + { + const int height = 5; + int destinationStride = width + 11; + int secondStride = width + 7; + int maskStride = width + 5; + byte[] first = new byte[destinationStride * height]; + byte[] second = new byte[secondStride * height]; + byte[] mask = new byte[maskStride * height]; + + FillByteInputs(first, second, destinationStride, secondStride, width, height); + FillMask(mask, maskStride, width, height); + + for (int weightIndex = 0; weightIndex < distanceWeights.Length; weightIndex += 2) + { + byte[] expected = (byte[])first.Clone(); + byte[] actual = (byte[])first.Clone(); + int firstWeight = distanceWeights[weightIndex]; + int secondWeight = distanceWeights[weightIndex + 1]; + + for (int row = 0; row < height; row++) + { + for (int column = 0; column < width; column++) + { + int destinationIndex = (row * destinationStride) + column; + int secondIndex = (row * secondStride) + column; + expected[destinationIndex] = (byte)(((expected[destinationIndex] * firstWeight) + + (second[secondIndex] * secondWeight) + 8) >> 4); + } + } + + Av1CompoundDistanceWeightedPredictor.DistanceWeighted( + actual, + destinationStride, + second, + secondStride, + width, + height, + firstWeight, + secondWeight); + + Assert.Equal(expected, actual); + } + + byte[] maskedExpected = (byte[])first.Clone(); + byte[] maskedActual = (byte[])first.Clone(); + for (int row = 0; row < height; row++) + { + for (int column = 0; column < width; column++) + { + int destinationIndex = (row * destinationStride) + column; + int secondIndex = (row * secondStride) + column; + int alpha = mask[(row * maskStride) + column]; + maskedExpected[destinationIndex] = (byte)(((alpha * maskedExpected[destinationIndex]) + + ((64 - alpha) * second[secondIndex]) + 32) >> 6); + } + } + + Av1CompoundMaskBlendPredictor.Blend( + maskedActual, + destinationStride, + second, + secondStride, + mask, + maskStride, + width, + height); + + Assert.Equal(maskedExpected, maskedActual); + } + } + + /// + /// Applies independent high-bit-depth arithmetic to every selectable compound blend. + /// + private static void ValidateHighBitDepthSelectableBlends() + { + ReadOnlySpan widths = [4, 7, 8, 15, 16, 23, 31, 32, 47, 64, 127, 128]; + ReadOnlySpan distanceWeights = [9, 7, 11, 5, 12, 4, 13, 3]; + + foreach (int bitDepth in new[] { 10, 12 }) + { + foreach (int width in widths) + { + const int height = 5; + int destinationStride = width + 9; + int secondStride = width + 5; + int maskStride = width + 3; + ushort[] first = new ushort[destinationStride * height]; + ushort[] second = new ushort[secondStride * height]; + byte[] mask = new byte[maskStride * height]; + + FillHighBitDepthInputs(first, second, destinationStride, secondStride, width, height, bitDepth); + FillMask(mask, maskStride, width, height); + + for (int weightIndex = 0; weightIndex < distanceWeights.Length; weightIndex += 2) + { + ushort[] expected = (ushort[])first.Clone(); + ushort[] actual = (ushort[])first.Clone(); + int firstWeight = distanceWeights[weightIndex]; + int secondWeight = distanceWeights[weightIndex + 1]; + + for (int row = 0; row < height; row++) + { + for (int column = 0; column < width; column++) + { + int destinationIndex = (row * destinationStride) + column; + int secondIndex = (row * secondStride) + column; + expected[destinationIndex] = (ushort)(((expected[destinationIndex] * firstWeight) + + (second[secondIndex] * secondWeight) + 8) >> 4); + } + } + + Av1CompoundDistanceWeightedPredictor.DistanceWeighted( + actual, + destinationStride, + second, + secondStride, + width, + height, + firstWeight, + secondWeight); + + Assert.Equal(expected, actual); + } + + ushort[] maskedExpected = (ushort[])first.Clone(); + ushort[] maskedActual = (ushort[])first.Clone(); + for (int row = 0; row < height; row++) + { + for (int column = 0; column < width; column++) + { + int destinationIndex = (row * destinationStride) + column; + int secondIndex = (row * secondStride) + column; + int alpha = mask[(row * maskStride) + column]; + maskedExpected[destinationIndex] = (ushort)(((alpha * maskedExpected[destinationIndex]) + + ((64 - alpha) * second[secondIndex]) + 32) >> 6); + } + } + + Av1CompoundMaskBlendPredictor.Blend( + maskedActual, + destinationStride, + second, + secondStride, + mask, + maskStride, + width, + height); + + Assert.Equal(maskedExpected, maskedActual); + } + } + } + + /// + /// Fills active byte samples while assigning different sentinels to the unused row tails. + /// + private static void FillByteInputs( + Span destination, + Span second, + int destinationStride, + int secondStride, + int width, + int height) + { + destination.Fill(0xD3); + second.Fill(0xA7); + + for (int row = 0; row < height; row++) + { + for (int column = 0; column < width; column++) + { + destination[(row * destinationStride) + column] = (byte)((row * 47) + (column * 29) + 3); + second[(row * secondStride) + column] = (byte)((row * 31) + (column * 53) + 11); + } + } + } + + /// + /// Fills active ushort samples across the requested precision while preserving guarded row tails. + /// + private static void FillHighBitDepthInputs( + Span destination, + Span second, + int destinationStride, + int secondStride, + int width, + int height, + int bitDepth) + { + destination.Fill(0xDEAD); + second.Fill(0xBEEF); + int mask = (1 << bitDepth) - 1; + + for (int row = 0; row < height; row++) + { + for (int column = 0; column < width; column++) + { + destination[(row * destinationStride) + column] = (ushort)(((row * 947) + (column * 613) + 17) & mask); + second[(row * secondStride) + column] = (ushort)(((row * 541) + (column * 887) + 23) & mask); + } + } + } + + /// + /// Fills active mask samples across the complete AV1 alpha range while guarding every row tail. + /// + private static void FillMask(Span mask, int maskStride, int width, int height) + { + mask.Fill(0xA5); + for (int row = 0; row < height; row++) + { + for (int column = 0; column < width; column++) + { + mask[(row * maskStride) + column] = (byte)(((row * 19) + (column * 37)) % 65); + } + } + } +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1CompoundReferenceEntropyTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1CompoundReferenceEntropyTests.cs new file mode 100644 index 0000000000..0a69afe44b --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1CompoundReferenceEntropyTests.cs @@ -0,0 +1,302 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Buffers; +using SixLabors.ImageSharp.Formats.Heif.Av1; +using SixLabors.ImageSharp.Formats.Heif.Av1.Entropy; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1; + +/// +/// Verifies AV1 compound-reference selection and compound inter-mode entropy against the current AV1 reference. +/// +[Trait("Format", "Avif")] +public class Av1CompoundReferenceEntropyTests +{ + /// + /// Verifies every binary compound-reference default against the reference decoder's forward Q15 tables. + /// + [Fact] + public void CompoundReferenceDefaultsMatchReference() + { + AssertBinaryDefaults( + [1198, 2070, 9166, 7499, 22475], + Av1DefaultDistributions.CompoundReferenceType); + + AssertBinaryDefaults( + [ + [5284, 3865, 3128], + [23152, 14173, 15270], + [31774, 25120, 26710], + ], + Av1DefaultDistributions.UnidirectionalCompoundReference); + + AssertBinaryDefaults( + [ + [4946, 9468, 1503], + [19891, 22441, 15160], + [30731, 31059, 27544], + ], + Av1DefaultDistributions.CompoundReference); + + AssertBinaryDefaults( + [ + [2235, 1423], + [17182, 15175], + [30606, 30489], + ], + Av1DefaultDistributions.CompoundBackwardReference); + } + + /// + /// Verifies all eight compound inter-mode defaults against the reference decoder's forward Q15 tables. + /// + [Fact] + public void InterCompoundModeDefaultsMatchReference() + { + uint[][] expected = + [ + [7760, 13823, 15808, 17641, 19156, 20666, 26891], + [10730, 19452, 21145, 22749, 24039, 25131, 28724], + [10664, 20221, 21588, 22906, 24295, 25387, 28436], + [13298, 16984, 20471, 24182, 25067, 25736, 26422], + [18904, 23325, 25242, 27432, 27898, 28258, 30758], + [10725, 17454, 20124, 22820, 24195, 25168, 26046], + [17125, 24273, 25814, 27492, 28214, 28704, 30592], + [13046, 23214, 24505, 25942, 27435, 28442, 29330], + ]; + + Av1Distribution[] actual = Av1DefaultDistributions.InterCompoundMode; + + Assert.Equal(expected.Length, actual.Length); + for (int context = 0; context < expected.Length; context++) + { + Assert.Equal(8, actual[context].NumberOfSymbols); + for (int threshold = 0; threshold < expected[context].Length; threshold++) + { + Assert.Equal((uint)Av1Distribution.ProbabilityTop - expected[context][threshold], actual[context][threshold]); + } + } + } + + /// + /// Verifies that each semantic reference reader selects its requested context row and tree decision. + /// + [Fact] + public void CompoundReferenceReadersUseRequestedDistributions() + { + bool[] values = [false, true, true, false, true, false]; + + for (int context = 0; context < 5; context++) + { + AssertBinaryReader( + Av1DefaultDistributions.CompoundReferenceType[context], + values, + (ref Av1SymbolDecoder decoder) => decoder.ReadCompoundReferenceIsBidirectional(context)); + } + + for (int context = 0; context < 3; context++) + { + for (int decision = 0; decision < 3; decision++) + { + AssertBinaryReader( + Av1DefaultDistributions.UnidirectionalCompoundReference[context][decision], + values, + (ref Av1SymbolDecoder decoder) => decoder.ReadUnidirectionalCompoundReference(context, decision)); + + AssertBinaryReader( + Av1DefaultDistributions.CompoundReference[context][decision], + values, + (ref Av1SymbolDecoder decoder) => decoder.ReadCompoundForwardReference(context, decision)); + } + + for (int decision = 0; decision < 2; decision++) + { + AssertBinaryReader( + Av1DefaultDistributions.CompoundBackwardReference[context][decision], + values, + (ref Av1SymbolDecoder decoder) => decoder.ReadCompoundBackwardReference(context, decision)); + } + } + } + + /// + /// Verifies the packed-mode-context mapping and all eight compound mode symbols. + /// + [Fact] + public void CompoundModeReaderUsesMappedDistribution() + { + ReadOnlySpan packedContexts = [0, 1, 33, 34, 35, 66, 67, 68]; + + for (int context = 0; context < packedContexts.Length; context++) + { + using Av1SymbolWriter writer = new(Configuration.Default, 3, updateCdf: true); + Av1Distribution writerDistribution = Av1DefaultDistributions.InterCompoundMode[context]; + writer.WriteSymbol(0, writerDistribution); + writer.WriteSymbol(7, writerDistribution); + writer.WriteSymbol(3, writerDistribution); + + using IMemoryOwner encoded = writer.Exit(); + Av1SymbolDecoder decoder = new(Configuration.Default, encoded.Memory.Span, 0, updateCdf: true); + + Assert.Equal(Av1PredictionMode.NearestNearestMotionVector, decoder.ReadInterCompoundMode(packedContexts[context])); + Assert.Equal(Av1PredictionMode.NewNewMotionVector, decoder.ReadInterCompoundMode(packedContexts[context])); + Assert.Equal(Av1PredictionMode.NewNearestMotionVector, decoder.ReadInterCompoundMode(packedContexts[context])); + } + } + + /// + /// Verifies the compound-reference type context across intra, single, bidirectional, and unidirectional neighbors. + /// + [Fact] + public void CompoundReferenceTypeContextMatchesReference() + { + Av1BlockModeInfo intra = CreateModeInfo(Av1ReferenceFrameType.Intra, Av1ReferenceFrameType.None); + Av1BlockModeInfo singleForward = CreateModeInfo(Av1ReferenceFrameType.Last, Av1ReferenceFrameType.None); + Av1BlockModeInfo singleBackward = CreateModeInfo(Av1ReferenceFrameType.Backward, Av1ReferenceFrameType.None); + Av1BlockModeInfo bidirectional = CreateModeInfo(Av1ReferenceFrameType.Last, Av1ReferenceFrameType.Backward); + Av1BlockModeInfo forwardUnidirectional = CreateModeInfo(Av1ReferenceFrameType.Last, Av1ReferenceFrameType.Last2); + Av1BlockModeInfo backwardUnidirectional = CreateModeInfo(Av1ReferenceFrameType.Backward, Av1ReferenceFrameType.Alternate); + + Assert.Equal(2, Av1SymbolContextHelper.GetCompoundReferenceTypeContext(null, null)); + Assert.Equal(2, Av1SymbolContextHelper.GetCompoundReferenceTypeContext(intra, null)); + Assert.Equal(2, Av1SymbolContextHelper.GetCompoundReferenceTypeContext(singleForward, null)); + Assert.Equal(0, Av1SymbolContextHelper.GetCompoundReferenceTypeContext(bidirectional, null)); + Assert.Equal(4, Av1SymbolContextHelper.GetCompoundReferenceTypeContext(forwardUnidirectional, null)); + Assert.Equal(2, Av1SymbolContextHelper.GetCompoundReferenceTypeContext(intra, singleForward)); + Assert.Equal(1, Av1SymbolContextHelper.GetCompoundReferenceTypeContext(intra, bidirectional)); + Assert.Equal(3, Av1SymbolContextHelper.GetCompoundReferenceTypeContext(intra, forwardUnidirectional)); + Assert.Equal(3, Av1SymbolContextHelper.GetCompoundReferenceTypeContext(singleForward, singleForward)); + Assert.Equal(1, Av1SymbolContextHelper.GetCompoundReferenceTypeContext(singleForward, singleBackward)); + Assert.Equal(0, Av1SymbolContextHelper.GetCompoundReferenceTypeContext(bidirectional, bidirectional)); + Assert.Equal(2, Av1SymbolContextHelper.GetCompoundReferenceTypeContext(bidirectional, forwardUnidirectional)); + Assert.Equal(4, Av1SymbolContextHelper.GetCompoundReferenceTypeContext(forwardUnidirectional, forwardUnidirectional)); + Assert.Equal(3, Av1SymbolContextHelper.GetCompoundReferenceTypeContext(forwardUnidirectional, backwardUnidirectional)); + } + + /// + /// Verifies the exact neighboring-vote groups used by every compound reference-tree decision. + /// + [Fact] + public void CompoundReferenceContextsAggregateNormativeGroups() + { + InlineArray8 referenceCountStorage = default; + Span referenceCounts = referenceCountStorage; + referenceCounts[(int)Av1ReferenceFrameType.Last] = 5; + referenceCounts[(int)Av1ReferenceFrameType.Last2] = 1; + referenceCounts[(int)Av1ReferenceFrameType.Last3] = 2; + referenceCounts[(int)Av1ReferenceFrameType.Golden] = 2; + referenceCounts[(int)Av1ReferenceFrameType.Backward] = 3; + referenceCounts[(int)Av1ReferenceFrameType.Alternate2] = 3; + referenceCounts[(int)Av1ReferenceFrameType.Alternate] = 6; + + Assert.Equal(0, Av1SymbolContextHelper.GetUnidirectionalCompoundBackwardContext(referenceCounts)); + Assert.Equal(0, Av1SymbolContextHelper.GetUnidirectionalCompoundLast3OrGoldenContext(referenceCounts)); + Assert.Equal(1, Av1SymbolContextHelper.GetUnidirectionalCompoundGoldenContext(referenceCounts)); + Assert.Equal(2, Av1SymbolContextHelper.GetCompoundForwardLast3OrGoldenContext(referenceCounts)); + Assert.Equal(2, Av1SymbolContextHelper.GetCompoundForwardLast2Context(referenceCounts)); + Assert.Equal(1, Av1SymbolContextHelper.GetCompoundForwardGoldenContext(referenceCounts)); + Assert.Equal(1, Av1SymbolContextHelper.GetCompoundBackwardAlternateContext(referenceCounts)); + Assert.Equal(1, Av1SymbolContextHelper.GetCompoundBackwardAlternate2Context(referenceCounts)); + } + + /// + /// Verifies compound CDF copying and snapshot update-count reset without sharing mutable state. + /// + [Fact] + public void FrameEntropyLifecycleIncludesCompoundDistributions() + { + const int updateCount = 20; + Av1FrameEntropyContext source = new(0); + Av1FrameEntropyContext copy = new(0); + Av1FrameEntropyContext snapshot = new(0); + + for (int i = 0; i < updateCount; i++) + { + source.CompoundReferenceType[4].Update(1); + source.UnidirectionalCompoundReference[2][2].Update(1); + source.CompoundReference[1][1].Update(1); + source.CompoundBackwardReference[0][1].Update(1); + source.InterCompoundMode[7].Update(6); + } + + copy.CopyFrom(source); + source.SnapshotTo(snapshot); + + Assert.Equal(source.CompoundReferenceType[4][0], copy.CompoundReferenceType[4][0]); + Assert.Equal(source.UnidirectionalCompoundReference[2][2][0], copy.UnidirectionalCompoundReference[2][2][0]); + Assert.Equal(source.CompoundReference[1][1][0], copy.CompoundReference[1][1][0]); + Assert.Equal(source.CompoundBackwardReference[0][1][0], copy.CompoundBackwardReference[0][1][0]); + Assert.Equal(source.InterCompoundMode[7][6], copy.InterCompoundMode[7][6]); + + source.CompoundReferenceType[4].Update(0); + snapshot.CompoundReferenceType[4].Update(0); + source.InterCompoundMode[7].Update(0); + snapshot.InterCompoundMode[7].Update(0); + + Assert.NotEqual(source.CompoundReferenceType[4][0], snapshot.CompoundReferenceType[4][0]); + Assert.NotEqual(source.InterCompoundMode[7][0], snapshot.InterCompoundMode[7][0]); + } + + /// + /// Verifies one binary reader against a separately adapted writer distribution. + /// + private static void AssertBinaryReader(Av1Distribution distribution, ReadOnlySpan values, SymbolReader reader) + { + using Av1SymbolWriter writer = new(Configuration.Default, values.Length, updateCdf: true); + foreach (bool value in values) + { + writer.WriteSymbol(value, distribution); + } + + using IMemoryOwner encoded = writer.Exit(); + Av1SymbolDecoder decoder = new(Configuration.Default, encoded.Memory.Span, 0, updateCdf: true); + foreach (bool value in values) + { + Assert.Equal(value, reader(ref decoder)); + } + } + + /// + /// Verifies one array of binary defaults stored in inverse-cumulative form. + /// + private static void AssertBinaryDefaults(ReadOnlySpan expected, Av1Distribution[] actual) + { + Assert.Equal(expected.Length, actual.Length); + for (int i = 0; i < expected.Length; i++) + { + Assert.Equal((uint)Av1Distribution.ProbabilityTop - expected[i], actual[i][0]); + Assert.Equal(2, actual[i].NumberOfSymbols); + } + } + + /// + /// Verifies a matrix of binary defaults stored in inverse-cumulative form. + /// + private static void AssertBinaryDefaults(uint[][] expected, Av1Distribution[][] actual) + { + Assert.Equal(expected.Length, actual.Length); + for (int row = 0; row < expected.Length; row++) + { + AssertBinaryDefaults(expected[row], actual[row]); + } + } + + /// + /// Creates decoded block-mode state with the requested primary and secondary references. + /// + private static Av1BlockModeInfo CreateModeInfo(Av1ReferenceFrameType primary, Av1ReferenceFrameType secondary) + { + Av1BlockModeInfo modeInfo = new(Av1BlockSize.Block8x8, Point.Empty); + modeInfo.ReferenceFrames[0] = primary; + modeInfo.ReferenceFrames[1] = secondary; + return modeInfo; + } + + /// + /// Invokes one semantic binary symbol reader. + /// + private delegate bool SymbolReader(ref Av1SymbolDecoder decoder); +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1DeblockingFilterTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1DeblockingFilterTests.cs new file mode 100644 index 0000000000..df8237eb2c --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1DeblockingFilterTests.cs @@ -0,0 +1,656 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1; +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.LoopFilter; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; +using SixLabors.ImageSharp.Tests.TestUtilities; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1; + +/// +/// Verifies AV1 deblocking across every filter width, sample precision, orientation, and intrinsic tier. +/// +[Trait("Format", "Avif")] +public class Av1DeblockingFilterTests +{ + /// + /// The hardware configurations required to exercise packed filtering and the scalar fallback. + /// + private const HwIntrinsics Configurations = HwIntrinsics.AllowAll | HwIntrinsics.DisableHWIntrinsic; + + /// + /// The padded plane width used to expose horizontal and vertical edge traversal. + /// + private const int Stride = 32; + + /// + /// The first Q-side coordinate, leaving the widest kernel addressable on every side. + /// + private const int EdgeCoordinate = 12; + + /// + /// Verifies the AV1 reference-category default deltas used to derive frame-edge filter levels. + /// + [Fact] + public void LoopFilterReferenceDeltasMatchAv1Defaults() + { + ObuLoopFilterParameters parameters = new(); + + Assert.Equal([1, 0, 0, 0, -1, 0, -1, -1], parameters.ReferenceDeltas); + } + + /// + /// Verifies exact filtering and untouched padding against an independent scalar definition. + /// + [Fact] + public void FilterMatchesIndependentDefinitionAcrossIntrinsicTiers() + => FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidateFilters, Configurations); + + /// + /// Verifies skipped inter-edge decisions and reference and mode level deltas through the production frame filter. + /// + [Fact] + public void DecodeFrameMatchesReference() + { + ValidateInterEdgeAndDeltaDecisions( + Av1PredictionMode.GlobalMotionVector, + Av1ReferenceFrameType.Last, + 20, + -3, + 4, + false, + 17); + + ValidateInterEdgeAndDeltaDecisions( + Av1PredictionMode.NewMotionVector, + Av1ReferenceFrameType.Last, + 20, + -3, + 4, + false, + 21); + + ValidateInterEdgeAndDeltaDecisions( + Av1PredictionMode.GlobalMotionVector, + Av1ReferenceFrameType.Golden, + 20, + 2, + 4, + false, + 22); + } + + /// + /// Verifies that opposite reference and mode adjustments cancel before the final filter-level clamp. + /// + [Theory] + [InlineData(1, -63, 63, false)] + [InlineData(63, 63, -63, false)] + [InlineData(1, -63, 63, true)] + [InlineData(63, 63, -63, true)] + public void DecodeFrameCombinesDeltasBeforeClipping(int baseLevel, int referenceDelta, int modeDelta, bool deltaLoopFilterPresent) + { + // Both native paths, per-block delta-LF and the precomputed frame table, clamp only after adding + // reference and mode adjustments. These equal and opposite deltas leave the base level unchanged. + ValidateInterEdgeAndDeltaDecisions( + Av1PredictionMode.NewMotionVector, + Av1ReferenceFrameType.Last, + baseLevel, + referenceDelta, + modeDelta, + deltaLoopFilterPresent, + baseLevel); + } + + /// + /// Exercises mixed flatness, high-edge-variance, disabled-mask, direction, and bit-depth cases. + /// + private static void ValidateFilters() + { + int[] filterLengths = [4, 6, 8, 14]; + foreach (int bitDepth in new[] { 8, 10, 12 }) + { + int scale = 1 << (bitDepth - 8); + int[][] mixedWindows = CreateMixedWindows(scale); + int[][] disabledWindows = CreateDisabledWindows(bitDepth); + + foreach (bool vertical in new[] { true, false }) + { + foreach (int filterLength in filterLengths) + { + if (bitDepth == 8) + { + AssertByteFilter(vertical, filterLength, mixedWindows); + AssertByteFilter(vertical, filterLength, disabledWindows); + } + else + { + AssertUInt16Filter(vertical, filterLength, bitDepth, mixedWindows); + AssertUInt16Filter(vertical, filterLength, bitDepth, disabledWindows); + } + } + } + } + } + + /// + /// Filters two adjacent skipped inter blocks and compares the visible luma plane with the scalar definition. + /// + private static void ValidateInterEdgeAndDeltaDecisions( + Av1PredictionMode mode, + Av1ReferenceFrameType referenceFrame, + int baseLevel, + int referenceDelta, + int modeDelta, + bool deltaLoopFilterPresent, + int expectedLevel) + { + const int width = Stride; + const int height = 8; + const int edge = 16; + ObuSequenceHeader sequenceHeader = new() + { + MaxFrameWidth = width, + MaxFrameHeight = height, + Use128x128Superblock = false, + ColorConfig = new ObuColorConfig + { + IsMonochrome = true, + BitDepth = Av1BitDepth.EightBit, + }, + }; + + ObuFrameHeader frameHeader = new() + { + FrameType = ObuFrameType.InterFrame, + ModeInfoColumnCount = width >> Av1Constants.ModeInfoSizeLog2, + ModeInfoRowCount = height >> Av1Constants.ModeInfoSizeLog2, + FrameSize = new ObuFrameSize + { + FrameWidth = width, + FrameHeight = height, + }, + }; + + ObuLoopFilterParameters filterParameters = frameHeader.LoopFilterParameters; + frameHeader.DeltaQParameters.IsPresent = deltaLoopFilterPresent; + frameHeader.DeltaLoopFilterParameters.IsPresent = deltaLoopFilterPresent; + filterParameters.FilterLevel[0] = baseLevel; + filterParameters.ReferenceDeltaModeEnabled = true; + filterParameters.ReferenceDeltas[(int)referenceFrame] = referenceDelta; + filterParameters.ModeDeltas[1] = modeDelta; + + using Av1FrameBuffer frameBuffer = new( + Configuration.Default, + sequenceHeader, + Av1ColorFormat.Yuv400, + false); + + using Av1FrameInfo frameInfo = new(sequenceHeader); + Av1SuperblockInfo superblock = frameInfo.GetSuperblock(Point.Empty); + Av1BlockModeInfo leftModeInfo = new(Av1BlockSize.Block16x8, Point.Empty) + { + Skip = true, + YMode = mode, + }; + + Av1BlockModeInfo rightModeInfo = new(Av1BlockSize.Block16x8, new Point(4, 0)) + { + Skip = true, + YMode = mode, + }; + + leftModeInfo.ReferenceFrames[0] = referenceFrame; + rightModeInfo.ReferenceFrames[0] = referenceFrame; + frameInfo.UpdateModeInfo(leftModeInfo, superblock); + frameInfo.UpdateModeInfo(rightModeInfo, superblock); + + using Av1LoopFilterContext loopFilterContext = + new(frameBuffer.MemoryAllocator, sequenceHeader, frameHeader); + + loopFilterContext.SetTransformSize(Av1Plane.Y, Point.Empty, Av1TransformSize.Size8x8); + loopFilterContext.SetTransformSize(Av1Plane.Y, new Point(2, 0), Av1TransformSize.Size8x8); + loopFilterContext.SetTransformSize(Av1Plane.Y, new Point(4, 0), Av1TransformSize.Size8x8); + loopFilterContext.SetTransformSize(Av1Plane.Y, new Point(6, 0), Av1TransformSize.Size8x8); + + byte[] expected = new byte[width * height]; + for (int row = 0; row < height; row++) + { + Span expectedRow = expected.AsSpan(row * width, width); + Span actualRow = frameBuffer.DeriveBlockPointer(Av1Plane.Y, 0, 0).DangerousGetRowSpan(row); + expectedRow[..edge].Fill(100); + expectedRow[edge..].Fill(130); + actualRow[..edge].Fill(100); + actualRow[edge..].Fill(130); + } + + int limit = expectedLevel; + int boundaryLimit = (2 * (expectedLevel + 2)) + limit; + int highEdgeVarianceThreshold = expectedLevel >> 4; + ApplyReference(expected, true, edge, 8, limit, boundaryLimit, highEdgeVarianceThreshold, 8); + ApplyReference(expected, true, (4 * width) + edge, 8, limit, boundaryLimit, highEdgeVarianceThreshold, 8); + + Av1LoopFilterDecoder decoder = new(sequenceHeader, frameHeader, frameInfo, frameBuffer, loopFilterContext); + decoder.DecodeFrame(); + + for (int row = 0; row < height; row++) + { + ReadOnlySpan expectedRow = expected.AsSpan(row * width, width); + ReadOnlySpan actualRow = frameBuffer.DeriveBlockPointer(Av1Plane.Y, 0, 0).DangerousGetRowSpan(row); + Assert.Equal(expectedRow, actualRow); + } + } + + /// + /// Verifies one eight-bit filter configuration against the independent definition. + /// + private static void AssertByteFilter(bool vertical, int filterLength, int[][] windows) + { + byte[] expected = Enumerable.Repeat((byte)231, Stride * Stride).ToArray(); + Populate(expected, vertical, windows); + byte[] actual = (byte[])expected.Clone(); + int q0Offset = (EdgeCoordinate * Stride) + EdgeCoordinate; + + ApplyReference(expected, vertical, q0Offset, filterLength, 20, 60, 3, 8); + if (vertical) + { + Av1DeblockingFilter.FilterVertical(actual, q0Offset, Stride, filterLength, 20, 60, 3); + } + else + { + Av1DeblockingFilter.FilterHorizontal(actual, q0Offset, Stride, filterLength, 20, 60, 3); + } + + Assert.Equal(expected, actual); + } + + /// + /// Verifies one high-bit-depth filter configuration against the independent definition. + /// + private static void AssertUInt16Filter(bool vertical, int filterLength, int bitDepth, int[][] windows) + { + ushort[] expected = Enumerable.Repeat((ushort)60000, Stride * Stride).ToArray(); + Populate(expected, vertical, windows); + ushort[] actual = (ushort[])expected.Clone(); + int q0Offset = (EdgeCoordinate * Stride) + EdgeCoordinate; + + ApplyReference(expected, vertical, q0Offset, filterLength, 20, 60, 3, bitDepth); + if (vertical) + { + Av1DeblockingFilter.FilterVertical(actual, q0Offset, Stride, filterLength, 20, 60, 3, bitDepth); + } + else + { + Av1DeblockingFilter.FilterHorizontal(actual, q0Offset, Stride, filterLength, 20, 60, 3, bitDepth); + } + + Assert.Equal(expected, actual); + } + + /// + /// Creates four lanes that independently select the wide, shorter-wide, narrow, and high-variance results. + /// + private static int[][] CreateMixedWindows(int scale) + => + [ + Scale([100, 100, 100, 100, 100, 100, 100, 101, 101, 101, 101, 101, 101, 101], scale), + Scale([94, 94, 94, 100, 100, 100, 100, 101, 101, 101, 101, 107, 107, 107], scale), + Scale([86, 88, 90, 92, 94, 96, 98, 102, 104, 106, 108, 110, 112, 114], scale), + Scale([85, 85, 85, 85, 85, 85, 100, 104, 119, 119, 119, 119, 119, 119], scale) + ]; + + /// + /// Creates four lanes whose cross-edge discontinuity disables every filter width. + /// + private static int[][] CreateDisabledWindows(int bitDepth) + { + int maximum = (1 << bitDepth) - 1; + int[] window = [0, 0, 0, 0, 0, 0, 0, maximum, maximum, maximum, maximum, maximum, maximum, maximum]; + return [(int[])window.Clone(), (int[])window.Clone(), (int[])window.Clone(), (int[])window.Clone()]; + } + + /// + /// Scales an eight-bit-domain sample window to the requested coded precision. + /// + private static int[] Scale(int[] values, int scale) + { + for (int index = 0; index < values.Length; index++) + { + values[index] *= scale; + } + + return values; + } + + /// + /// Places four eight-bit p6..q6 windows along one padded edge. + /// + private static void Populate(Span samples, bool vertical, int[][] windows) + { + for (int index = 0; index < 4; index++) + { + for (int sample = 0; sample < 14; sample++) + { + int distance = sample - 7; + int offset = vertical + ? ((EdgeCoordinate + index) * Stride) + EdgeCoordinate + distance + : ((EdgeCoordinate + distance) * Stride) + EdgeCoordinate + index; + + samples[offset] = (byte)windows[index][sample]; + } + } + } + + /// + /// Places four 16-bit p6..q6 windows along one padded edge. + /// + private static void Populate(Span samples, bool vertical, int[][] windows) + { + for (int index = 0; index < 4; index++) + { + for (int sample = 0; sample < 14; sample++) + { + int distance = sample - 7; + int offset = vertical + ? ((EdgeCoordinate + index) * Stride) + EdgeCoordinate + distance + : ((EdgeCoordinate + distance) * Stride) + EdgeCoordinate + index; + + samples[offset] = (ushort)windows[index][sample]; + } + } + } + + /// + /// Applies the scalar AV1 definition to four eight-bit samples along one edge. + /// + private static void ApplyReference( + Span samples, + bool vertical, + int q0Offset, + int filterLength, + int limit, + int boundaryLimit, + int highEdgeVarianceThreshold, + int bitDepth) + { + for (int index = 0; index < 4; index++) + { + int[] window = LoadWindow(samples, vertical, q0Offset, index); + FilterReference(window, filterLength, limit, boundaryLimit, highEdgeVarianceThreshold, bitDepth); + StoreWindow(samples, vertical, q0Offset, index, filterLength, window); + } + } + + /// + /// Applies the scalar AV1 definition to four 16-bit samples along one edge. + /// + private static void ApplyReference( + Span samples, + bool vertical, + int q0Offset, + int filterLength, + int limit, + int boundaryLimit, + int highEdgeVarianceThreshold, + int bitDepth) + { + for (int index = 0; index < 4; index++) + { + int[] window = LoadWindow(samples, vertical, q0Offset, index); + FilterReference(window, filterLength, limit, boundaryLimit, highEdgeVarianceThreshold, bitDepth); + StoreWindow(samples, vertical, q0Offset, index, filterLength, window); + } + } + + /// + /// Loads one eight-bit p6..q6 window independently of the production edge operators. + /// + private static int[] LoadWindow(ReadOnlySpan samples, bool vertical, int q0Offset, int index) + { + int[] result = new int[14]; + for (int sample = 0; sample < result.Length; sample++) + { + int distance = sample - 7; + int offset = vertical ? q0Offset + (index * Stride) + distance : q0Offset + (distance * Stride) + index; + result[sample] = samples[offset]; + } + + return result; + } + + /// + /// Loads one 16-bit p6..q6 window independently of the production edge operators. + /// + private static int[] LoadWindow(ReadOnlySpan samples, bool vertical, int q0Offset, int index) + { + int[] result = new int[14]; + for (int sample = 0; sample < result.Length; sample++) + { + int distance = sample - 7; + int offset = vertical ? q0Offset + (index * Stride) + distance : q0Offset + (distance * Stride) + index; + result[sample] = samples[offset]; + } + + return result; + } + + /// + /// Stores every potentially modified sample from one eight-bit reference window. + /// + private static void StoreWindow(Span samples, bool vertical, int q0Offset, int index, int filterLength, ReadOnlySpan window) + { + int radius = filterLength switch + { + 4 or 6 => 2, + 8 => 3, + _ => 6 + }; + + for (int distance = -radius; distance < radius; distance++) + { + int offset = vertical ? q0Offset + (index * Stride) + distance : q0Offset + (distance * Stride) + index; + samples[offset] = (byte)window[distance + 7]; + } + } + + /// + /// Stores every potentially modified sample from one 16-bit reference window. + /// + private static void StoreWindow(Span samples, bool vertical, int q0Offset, int index, int filterLength, ReadOnlySpan window) + { + int radius = filterLength switch + { + 4 or 6 => 2, + 8 => 3, + _ => 6 + }; + + for (int distance = -radius; distance < radius; distance++) + { + int offset = vertical ? q0Offset + (index * Stride) + distance : q0Offset + (distance * Stride) + index; + samples[offset] = (ushort)window[distance + 7]; + } + } + + /// + /// Selects and applies the normative scalar kernel for one p6..q6 window. + /// + private static void FilterReference( + Span samples, + int filterLength, + int limit, + int boundaryLimit, + int highEdgeVarianceThreshold, + int bitDepth) + { + int scale = 1 << (bitDepth - 8); + int scaledLimit = limit * scale; + int scaledBoundaryLimit = boundaryLimit * scale; + bool filterEnabled = IsFilterEnabled(samples, filterLength, scaledLimit, scaledBoundaryLimit); + + if (filterLength == 6 && filterEnabled && IsFlat(samples, filterLength, scale)) + { + int p2 = samples[4]; + int p1 = samples[5]; + int p0 = samples[6]; + int q0 = samples[7]; + int q1 = samples[8]; + int q2 = samples[9]; + samples[5] = ((3 * p2) + (2 * p1) + (2 * p0) + q0 + 4) >> 3; + samples[6] = (p2 + (2 * p1) + (2 * p0) + (2 * q0) + q1 + 4) >> 3; + samples[7] = (p1 + (2 * p0) + (2 * q0) + (2 * q1) + q2 + 4) >> 3; + samples[8] = (p0 + (2 * q0) + (2 * q1) + (3 * q2) + 4) >> 3; + return; + } + + bool flat = filterLength >= 8 && IsFlat(samples, filterLength, scale); + if (filterLength == 14 && filterEnabled && flat && IsOuterFlat(samples, scale)) + { + ApplyWideReference(samples); + return; + } + + if (filterLength >= 8 && filterEnabled && flat) + { + int p3 = samples[3]; + int p2 = samples[4]; + int p1 = samples[5]; + int p0 = samples[6]; + int q0 = samples[7]; + int q1 = samples[8]; + int q2 = samples[9]; + int q3 = samples[10]; + samples[4] = ((3 * p3) + (2 * p2) + p1 + p0 + q0 + 4) >> 3; + samples[5] = ((2 * p3) + p2 + (2 * p1) + p0 + q0 + q1 + 4) >> 3; + samples[6] = (p3 + p2 + p1 + (2 * p0) + q0 + q1 + q2 + 4) >> 3; + samples[7] = (p2 + p1 + p0 + (2 * q0) + q1 + q2 + q3 + 4) >> 3; + samples[8] = (p1 + p0 + q0 + (2 * q1) + q2 + (2 * q3) + 4) >> 3; + samples[9] = (p0 + q0 + q1 + (2 * q2) + (3 * q3) + 4) >> 3; + return; + } + + ApplyNarrowReference(samples, filterEnabled, highEdgeVarianceThreshold * scale, bitDepth); + } + + /// + /// Evaluates the AV1 filter mask for the selected reference width. + /// + private static bool IsFilterEnabled(ReadOnlySpan samples, int filterLength, int limit, int boundaryLimit) + { + bool enabled = Math.Abs(samples[5] - samples[6]) <= limit + && Math.Abs(samples[8] - samples[7]) <= limit + && ((2 * Math.Abs(samples[6] - samples[7])) + (Math.Abs(samples[5] - samples[8]) >> 1)) <= boundaryLimit; + + if (filterLength >= 6) + { + enabled = enabled + && Math.Abs(samples[4] - samples[5]) <= limit + && Math.Abs(samples[9] - samples[8]) <= limit; + } + + if (filterLength >= 8) + { + enabled = enabled + && Math.Abs(samples[3] - samples[4]) <= limit + && Math.Abs(samples[10] - samples[9]) <= limit; + } + + return enabled; + } + + /// + /// Evaluates the AV1 inner flatness mask for the selected reference width. + /// + private static bool IsFlat(ReadOnlySpan samples, int filterLength, int threshold) + { + bool flat = Math.Abs(samples[5] - samples[6]) <= threshold + && Math.Abs(samples[8] - samples[7]) <= threshold + && Math.Abs(samples[4] - samples[6]) <= threshold + && Math.Abs(samples[9] - samples[7]) <= threshold; + + return filterLength == 6 + ? flat + : flat && Math.Abs(samples[3] - samples[6]) <= threshold && Math.Abs(samples[10] - samples[7]) <= threshold; + } + + /// + /// Evaluates the AV1 outer flatness mask for the fourteen-tap reference kernel. + /// + private static bool IsOuterFlat(ReadOnlySpan samples, int threshold) + => Math.Abs(samples[0] - samples[6]) <= threshold + && Math.Abs(samples[1] - samples[6]) <= threshold + && Math.Abs(samples[2] - samples[6]) <= threshold + && Math.Abs(samples[11] - samples[7]) <= threshold + && Math.Abs(samples[12] - samples[7]) <= threshold + && Math.Abs(samples[13] - samples[7]) <= threshold; + + /// + /// Applies the signed-saturating four-tap reference equations. + /// + private static void ApplyNarrowReference(Span samples, bool filterEnabled, int highEdgeVarianceThreshold, int bitDepth) + { + if (!filterEnabled) + { + return; + } + + int offset = 128 << (bitDepth - 8); + int minimum = -offset; + int maximum = offset - 1; + int p1 = samples[5] - offset; + int p0 = samples[6] - offset; + int q0 = samples[7] - offset; + int q1 = samples[8] - offset; + bool highVariance = Math.Abs(samples[5] - samples[6]) > highEdgeVarianceThreshold + || Math.Abs(samples[8] - samples[7]) > highEdgeVarianceThreshold; + + int filter = highVariance ? Math.Clamp(p1 - q1, minimum, maximum) : 0; + filter = Math.Clamp(filter + (3 * (q0 - p0)), minimum, maximum); + int filter1 = Math.Clamp(filter + 4, minimum, maximum) >> 3; + int filter2 = Math.Clamp(filter + 3, minimum, maximum) >> 3; + samples[7] = Math.Clamp(q0 - filter1, minimum, maximum) + offset; + samples[6] = Math.Clamp(p0 + filter2, minimum, maximum) + offset; + + int outerFilter = highVariance ? 0 : (filter1 + 1) >> 1; + samples[8] = Math.Clamp(q1 - outerFilter, minimum, maximum) + offset; + samples[5] = Math.Clamp(p1 + outerFilter, minimum, maximum) + offset; + } + + /// + /// Applies the thirteen-tap reference equations to the twelve modifiable samples. + /// + private static void ApplyWideReference(Span samples) + { + int p6 = samples[0]; + int p5 = samples[1]; + int p4 = samples[2]; + int p3 = samples[3]; + int p2 = samples[4]; + int p1 = samples[5]; + int p0 = samples[6]; + int q0 = samples[7]; + int q1 = samples[8]; + int q2 = samples[9]; + int q3 = samples[10]; + int q4 = samples[11]; + int q5 = samples[12]; + int q6 = samples[13]; + + samples[1] = ((7 * p6) + (2 * p5) + (2 * p4) + p3 + p2 + p1 + p0 + q0 + 8) >> 4; + samples[2] = ((5 * p6) + (2 * p5) + (2 * p4) + (2 * p3) + p2 + p1 + p0 + q0 + q1 + 8) >> 4; + samples[3] = ((4 * p6) + p5 + (2 * p4) + (2 * p3) + (2 * p2) + p1 + p0 + q0 + q1 + q2 + 8) >> 4; + samples[4] = ((3 * p6) + p5 + p4 + (2 * p3) + (2 * p2) + (2 * p1) + p0 + q0 + q1 + q2 + q3 + 8) >> 4; + samples[5] = ((2 * p6) + p5 + p4 + p3 + (2 * p2) + (2 * p1) + (2 * p0) + q0 + q1 + q2 + q3 + q4 + 8) >> 4; + samples[6] = (p6 + p5 + p4 + p3 + p2 + (2 * p1) + (2 * p0) + (2 * q0) + q1 + q2 + q3 + q4 + q5 + 8) >> 4; + samples[7] = (p5 + p4 + p3 + p2 + p1 + (2 * p0) + (2 * q0) + (2 * q1) + q2 + q3 + q4 + q5 + q6 + 8) >> 4; + samples[8] = (p4 + p3 + p2 + p1 + p0 + (2 * q0) + (2 * q1) + (2 * q2) + q3 + q4 + q5 + (2 * q6) + 8) >> 4; + samples[9] = (p3 + p2 + p1 + p0 + q0 + (2 * q1) + (2 * q2) + (2 * q3) + q4 + q5 + (3 * q6) + 8) >> 4; + samples[10] = (p2 + p1 + p0 + q0 + q1 + (2 * q2) + (2 * q3) + (2 * q4) + q5 + (4 * q6) + 8) >> 4; + samples[11] = (p1 + p0 + q0 + q1 + q2 + (2 * q3) + (2 * q4) + (2 * q5) + (5 * q6) + 8) >> 4; + samples[12] = (p0 + q0 + q1 + q2 + q3 + (2 * q4) + (2 * q5) + (7 * q6) + 8) >> 4; + } +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1EncoderFrameTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1EncoderFrameTests.cs new file mode 100644 index 0000000000..6cec574eaf --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1EncoderFrameTests.cs @@ -0,0 +1,2020 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.InteropServices; +using SixLabors.ImageSharp.Formats; +using SixLabors.ImageSharp.Formats.Heif.Av1; +using SixLabors.ImageSharp.Formats.Heif.Av1.Motion; +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; +using SixLabors.ImageSharp.Formats.Heif.Components; +using SixLabors.ImageSharp.Formats.Heif.Components.Alpha; +using SixLabors.ImageSharp.Memory; +using SixLabors.ImageSharp.PixelFormats; +using SixLabors.ImageSharp.Tests.Memory; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1; + +public class Av1EncoderFrameTests +{ + private const int EightBit = (int)Av1BitDepth.EightBit; + private const int TenBit = (int)Av1BitDepth.TenBit; + private const int TwelveBit = (int)Av1BitDepth.TwelveBit; + private const int Yuv400 = (int)Av1ColorFormat.Yuv400; + private const int Yuv420 = (int)Av1ColorFormat.Yuv420; + private const int Yuv422 = (int)Av1ColorFormat.Yuv422; + private const int Yuv444 = (int)Av1ColorFormat.Yuv444; + + [Theory] + [InlineData(EightBit, false, false, false)] + [InlineData(EightBit, false, true, false)] + [InlineData(EightBit, true, false, false)] + [InlineData(EightBit, true, true, false)] + [InlineData(TenBit, false, false, false)] + [InlineData(TenBit, false, true, false)] + [InlineData(TenBit, true, false, false)] + [InlineData(TenBit, true, true, false)] + [InlineData(TwelveBit, false, false, false)] + [InlineData(TwelveBit, false, true, false)] + [InlineData(TwelveBit, true, false, false)] + [InlineData(TwelveBit, true, true, false)] + [InlineData(EightBit, false, true, true)] + [InlineData(EightBit, true, true, true)] + [InlineData(TenBit, false, true, true)] + [InlineData(TenBit, true, true, true)] + [InlineData(TwelveBit, false, true, true)] + [InlineData(TwelveBit, true, true, true)] + public void RectangularIntraReferencesExtendTheLastAvailableSample(int bitDepthValue, bool transpose, bool extensionAvailable, bool limitedExtent) + { + Av1BitDepth bitDepth = (Av1BitDepth)bitDepthValue; + if (bitDepth == Av1BitDepth.EightBit) + { + AssertRectangularIntraReferences(bitDepth, transpose, extensionAvailable, limitedExtent); + } + else + { + AssertRectangularIntraReferences(bitDepth, transpose, extensionAvailable, limitedExtent); + } + } + + private static void AssertRectangularIntraReferences( + Av1BitDepth bitDepth, + bool transpose, + bool extensionAvailable, + bool limitedExtent) + where TSample : unmanaged + where TOperator : struct, Av1IntraSuperblockEncoder.IBlockEncodingOperator + { + int width = transpose ? 16 : 4; + int height = transpose ? 4 : 16; + int scale = 1 << (bitDepth.GetBitCount() - 8); + using Buffer2D plane = Configuration.Default.MemoryAllocator.Allocate2D(33, 33); + for (int i = 0; i < 32; i++) + { + plane.DangerousGetRowSpan(0)[i + 1] = TOperator.CreateSample((10 + i) * scale); + plane.DangerousGetRowSpan(i + 1)[0] = TOperator.CreateSample((50 + i) * scale); + } + + plane.DangerousGetRowSpan(0)[0] = TOperator.CreateSample(100 * scale); + + // Native reconintra.c extends a four-sample edge through its four-sample neighbor, then + // repeats sample seven to cover the twenty samples required by a 4x16 directional ray. + // A clipped frame leaves only two adjacent samples; backing-buffer values beyond the region + // must not contribute. These explicit offsets distinguish all three extension cases. + int[] shortEdge = limitedExtent + ? [0, 1, 2, 3, 4, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5] + : extensionAvailable + ? [0, 1, 2, 3, 4, 5, 6, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7] + : [0, 1, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3]; + + int[] longEdge = limitedExtent + ? [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 17, 17] + : extensionAvailable + ? [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19] + : [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 15, 15, 15, 15]; + + int[] expectedAbove = transpose ? longEdge : shortEdge; + int[] expectedLeft = transpose ? shortEdge : longEdge; + TSample poison = TOperator.CreateSample((1 << bitDepth.GetBitCount()) - 1); + TSample[] above = new TSample[23]; + TSample[] left = new TSample[23]; + above.AsSpan().Fill(poison); + left.AsSpan().Fill(poison); + + // The exact-sized interior includes the corner and twenty projected samples. Sentinel samples + // on either side detect writes outside the reference view, including the former 2*long-edge span. + Av1IntraSuperblockEncoder.ModeDecision.PrepareReferenceSamples( + plane.GetRegion(0, 0, limitedExtent ? width + 3 : 33, limitedExtent ? height + 3 : 33), + new Point(1, 1), + width, + height, + true, + true, + extensionAvailable, + extensionAvailable, + bitDepth, + above.AsSpan(1, 21), + left.AsSpan(1, 21)); + + Assert.Equal(poison, above[0]); + Assert.Equal(poison, left[0]); + Assert.Equal(poison, above[^1]); + Assert.Equal(poison, left[^1]); + Assert.Equal(TOperator.CreateSample(100 * scale), above[1]); + Assert.Equal(TOperator.CreateSample(100 * scale), left[1]); + for (int i = 0; i < 20; i++) + { + Assert.Equal(TOperator.CreateSample((10 + expectedAbove[i]) * scale), above[i + 2]); + Assert.Equal(TOperator.CreateSample((50 + expectedLeft[i]) * scale), left[i + 2]); + } + } + + [Fact] + public void EncodeUsesMultipleTilesWhenSingleTileWidthLimitIsExceeded() + { + const int Width = Av1Constants.MaxTileWidth + 1; + const int SuperblockSize = 1 << (Av1Constants.MaxSuperBlockSizeLog2 - 1); + const int Height = SuperblockSize; + int superblockColumns = (Width + SuperblockSize - 1) / SuperblockSize; + int secondTileStart = ((superblockColumns + 1) / 2) * SuperblockSize; + using Image source = new(Width, Height, new L8(128)); + source[0, 0] = new L8(1); + source[secondTileStart - 1, 0] = new L8(17); + source[secondTileStart, 0] = new L8(241); + source[Width - 1, Height - 1] = new L8(255); + using MemoryStream stream = new(); + ObuColorConfig colorConfig = CreateColorConfig(Av1BitDepth.EightBit, Av1ColorFormat.Yuv400); + + Av1FrameEncoder.Encode( + Configuration.Default, + source.Frames.RootFrame, + stream, + colorConfig, + qIndex: 0, + effort: 0); + + using Av1Decoder decoder = new(Configuration.Default); + using Image decoded = decoder.Decode(stream.ToArray()); + + Assert.Equal(2, decoder.FrameHeader.TilesInfo.TileColumnCount); + Assert.Equal(1, decoder.FrameHeader.TilesInfo.TileRowCount); + Assert.Equal(source.Size, decoded.Size); + Assert.Equal(source[0, 0], decoded[0, 0]); + Assert.Equal(source[secondTileStart - 1, 0], decoded[secondTileStart - 1, 0]); + Assert.Equal(source[secondTileStart, 0], decoded[secondTileStart, 0]); + Assert.Equal(source[Width - 1, Height - 1], decoded[Width - 1, Height - 1]); + } + + [Theory] + [InlineData(8, 8, false, EightBit, Yuv400)] + [InlineData(8, 8, true, EightBit, Yuv400)] + [InlineData(16, 16, false, EightBit, Yuv400)] + [InlineData(16, 16, true, EightBit, Yuv400)] + [InlineData(8, 8, false, TenBit, Yuv400)] + [InlineData(8, 8, true, TenBit, Yuv400)] + [InlineData(8, 8, false, TwelveBit, Yuv400)] + [InlineData(8, 8, true, TwelveBit, Yuv400)] + [InlineData(16, 16, false, EightBit, Yuv420)] + [InlineData(16, 16, true, EightBit, Yuv420)] + [InlineData(13, 11, true, EightBit, Yuv420)] + [InlineData(16, 16, false, TenBit, Yuv420)] + [InlineData(16, 16, true, TenBit, Yuv420)] + [InlineData(16, 16, false, TwelveBit, Yuv420)] + [InlineData(16, 16, true, TwelveBit, Yuv420)] + [InlineData(16, 16, false, EightBit, Yuv422)] + [InlineData(16, 16, true, EightBit, Yuv422)] + [InlineData(13, 11, true, EightBit, Yuv422)] + [InlineData(16, 16, false, TenBit, Yuv422)] + [InlineData(16, 16, true, TenBit, Yuv422)] + [InlineData(16, 16, false, TwelveBit, Yuv422)] + [InlineData(16, 16, true, TwelveBit, Yuv422)] + [InlineData(16, 16, false, EightBit, Yuv444)] + [InlineData(16, 16, true, EightBit, Yuv444)] + [InlineData(13, 11, true, EightBit, Yuv444)] + [InlineData(16, 16, false, TenBit, Yuv444)] + [InlineData(16, 16, true, TenBit, Yuv444)] + [InlineData(16, 16, false, TwelveBit, Yuv444)] + [InlineData(16, 16, true, TwelveBit, Yuv444)] + public void EncodeWritesReducedStillPictureConsumedByProductionDecoder(int width, int height, bool hasGradient, int bitDepthValue, int colorFormatValue) + { + Av1BitDepth bitDepth = (Av1BitDepth)bitDepthValue; + Av1ColorFormat colorFormat = (Av1ColorFormat)colorFormatValue; + using Image source = new(width, height); + for (int y = 0; y < height; y++) + { + Span row = source.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(y); + for (int x = 0; x < width; x++) + { + if (colorFormat == Av1ColorFormat.Yuv400) + { + byte value = hasGradient ? (byte)((x * 13) + (y * 17)) : (byte)128; + row[x] = new Rgba32(value, value, value); + } + else + { + byte red = hasGradient ? (byte)((x * 13) + (y * 17)) : (byte)192; + byte green = hasGradient ? (byte)((x * 7) + (y * 5)) : (byte)64; + byte blue = hasGradient ? (byte)((x * 3) + (y * 11)) : (byte)32; + row[x] = new Rgba32(red, green, blue); + } + } + } + + ObuColorConfig colorConfig = CreateColorConfig(bitDepth, colorFormat); + using MemoryStream stream = new(); + ObuSequenceHeader encodedHeader = Av1FrameEncoder.Encode( + Configuration.Default, + source.Frames.RootFrame, + stream, + colorConfig, + qIndex: 37, + effort: 5); + + byte[] payload = stream.ToArray(); + string outputDirectory = Path.Combine( + TestEnvironment.ActualOutputDirectoryFullPath, + "Formats", + "Heif", + "Av1"); + + Directory.CreateDirectory(outputDirectory); + string contentName = hasGradient ? "gradient" : "constant"; + int bitCount = bitDepth.GetBitCount(); + string colorName = colorFormat.ToString()[3..]; + string fileName = $"encoder-frame-{width}x{height}-{bitCount}b-{colorName}-{contentName}.obu"; + File.WriteAllBytes(Path.Combine(outputDirectory, fileName), payload); + + using Av1Decoder decoder = new(Configuration.Default); + using Image decoded = decoder.Decode(payload); + + Assert.Equal(width, decoded.Width); + Assert.Equal(height, decoded.Height); + ObuSequenceProfile expectedProfile = bitDepth == Av1BitDepth.TwelveBit || colorFormat == Av1ColorFormat.Yuv422 + ? ObuSequenceProfile.Professional + : colorFormat == Av1ColorFormat.Yuv444 + ? ObuSequenceProfile.High + : ObuSequenceProfile.Main; + + Assert.Equal(expectedProfile, encodedHeader.SequenceProfile); + Assert.True(encodedHeader.IsReducedStillPictureHeader); + + Rgba32 first = decoded[0, 0]; + Assert.Equal(byte.MaxValue, first.A); + if (colorFormat == Av1ColorFormat.Yuv400) + { + Assert.Equal(first.R, first.G); + Assert.Equal(first.R, first.B); + } + else + { + Rgba32 center = decoded[width / 2, height / 2]; + Assert.True(center.R != center.G || center.G != center.B); + } + + if (hasGradient) + { + Assert.NotEqual(first, decoded[width - 1, height - 1]); + } + else + { + if (colorFormat == Av1ColorFormat.Yuv400) + { + Assert.InRange(first.R, 120, 136); + + for (int y = 0; y < height; y++) + { + foreach (Rgba32 pixel in decoded.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(y)) + { + Assert.Equal(first, pixel); + } + } + } + } + } + + [Theory] + [InlineData(false)] + [InlineData(true)] + public void EncodeSequenceFrameWritesNonReducedHeaderConsumedByProductionDecoder(bool encodeAlpha) + { + const int Width = 16; + const int Height = 16; + using Image source = new(Width, Height, new Rgba32(48, 96, 192)); + using MemoryStream stream = new(); + ObuColorConfig colorConfig = encodeAlpha + ? CreateColorConfig(Av1BitDepth.EightBit) + : CreateColorConfig(Av1BitDepth.EightBit, Av1ColorFormat.Yuv420); + + using Av1FrameEncoder.SequenceEncoder encoder = encodeAlpha + ? Av1FrameEncoder.CreateAlphaSequenceEncoder( + Configuration.Default, + Width, + Height, + colorConfig, + qIndex: 37, + effort: 5) + : Av1FrameEncoder.CreateColorSequenceEncoder( + Configuration.Default, + Width, + Height, + colorConfig, + qIndex: 37, + effort: 5); + + encoder.EncodeKeyFrame(source.Frames.RootFrame, stream); + ObuSequenceHeader encodedHeader = encoder.SequenceHeader; + byte[] payload = stream.ToArray(); + using Av1Decoder decoder = new(Configuration.Default); + using Image decoded = decoder.Decode(payload); + ObuSequenceHeader decodedHeader = decoder.SequenceHeader; + + Assert.False(encodedHeader.IsStillPicture); + Assert.False(encodedHeader.IsReducedStillPictureHeader); + Assert.Equal(encodeAlpha, encodedHeader.ColorConfig.IsMonochrome); + Assert.NotNull(decodedHeader); + Assert.False(decodedHeader.IsStillPicture); + Assert.False(decodedHeader.IsReducedStillPictureHeader); + Assert.Equal(new Size(Width, Height), decoded.Size); + } + + /// + /// Verifies dependent color samples with odd visible dimensions and motion across subsampled chroma phases. + /// + [Theory] + [InlineData(EightBit, Yuv420, 8)] + [InlineData(TenBit, Yuv420, 8)] + [InlineData(TwelveBit, Yuv420, 8)] + [InlineData(EightBit, Yuv420, 9)] + [InlineData(TenBit, Yuv420, 9)] + [InlineData(TwelveBit, Yuv420, 9)] + [InlineData(EightBit, Yuv422, 9)] + [InlineData(TenBit, Yuv422, 9)] + [InlineData(TwelveBit, Yuv422, 9)] + [InlineData(EightBit, Yuv444, 9)] + [InlineData(TenBit, Yuv444, 9)] + [InlineData(TwelveBit, Yuv444, 9)] + public void SequenceEncoderPreservesNativeColorPlanesWithSubpixelMotion(int bitDepthValue, int colorFormatValue, int effort) + { + const int Width = 23; + const int Height = 19; + const int QIndex = 17; + const int ByteToUInt16Scale = ushort.MaxValue / byte.MaxValue; + Av1BitDepth bitDepth = (Av1BitDepth)bitDepthValue; + Av1ColorFormat colorFormat = (Av1ColorFormat)colorFormatValue; + ObuColorConfig colorConfig = CreateColorConfig(bitDepth, colorFormat); + ReadOnlySpan period = [0, 28, 40, 28, 0, -28, -40, -12]; + using Image source = new(Width, Height); + using Av1FrameEncoder.SequenceEncoder encoder = Av1FrameEncoder.CreateColorSequenceEncoder( + Configuration.Default, Width, Height, colorConfig, QIndex, effort); + + string outputDirectory = TestEnvironment.CreateOutputDirectory("Heif", "Av1", nameof(this.SequenceEncoderPreservesNativeColorPlanesWithSubpixelMotion)); + string outputName = $"{bitDepth.GetBitCount()}-{colorFormat}-effort{effort}"; + using FileStream output = File.Create(Path.Combine(outputDirectory, outputName + ".obu")); + using BinaryWriter rawOutput = new(File.Create(Path.Combine(outputDirectory, outputName + ".managed.yuv"))); + using Av1Decoder decoder = new(Configuration.Default); + using MemoryStream sample = new(); + for (int frameIndex = 0; frameIndex < 2; frameIndex++) + { + // The second source translates all three channels by one luma sample on each axis. Chroma is + // converted independently by the production converter, so 4:2:0 and 4:2:2 cannot hide behind + // constant neutral planes. Odd dimensions also exercise each plane's visible-edge clipping. + for (int y = 0; y < Height; y++) + { + Span row = source.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(y); + int referenceY = Math.Min(y + frameIndex, Height - 1); + for (int x = 0; x < Width; x++) + { + int referenceX = Math.Min(x + frameIndex, Width - 1); + row[x] = new Rgb48( + (ushort)((128 + period[referenceX % period.Length]) * ByteToUInt16Scale), + (ushort)((128 + period[referenceY % period.Length]) * ByteToUInt16Scale), + (ushort)((128 + period[(referenceX + referenceY) % period.Length]) * ByteToUInt16Scale)); + } + } + + sample.SetLength(0); + if (frameIndex == 0) + { + encoder.EncodeKeyFrame(source.Frames.RootFrame, sample); + } + else + { + encoder.EncodeInterFrame(source.Frames.RootFrame, sample); + } + + sample.Position = 0; + sample.CopyTo(output); + decoder.DecodeSequenceReference(sample.ToArray(), null, null); + Assert.True(Assert.IsType(decoder.SequenceHeader).EnableIntraEdgeFilter); + Av1FrameBuffer decoded = Assert.IsType>(decoder.FrameBuffer); + Assert.Equal(Width, decoded.Width); + Assert.Equal(Height, decoded.Height); + Assert.Equal(bitDepth, decoded.BitDepth); + Av1FrameInfo decodedFrameInfo = Assert.IsType(decoder.FrameInfo); + foreach (Av1BlockModeInfo mode in decodedFrameInfo.GetModeInfos(Point.Empty, decodedFrameInfo.GetModeInfoCount(Point.Empty))) + { + // The same ordinary-intra policy applies in key and inter frames. Read the emitted syntax, + // rather than infer the skip flag from pixel agreement between encoder and decoder. + if (mode.ReferenceFrames[0] == Av1ReferenceFrameType.Intra && !mode.UseIntraBlockCopy) + { + Assert.False(mode.Skip); + } + } + + for (int planeIndex = 0; planeIndex < 3; planeIndex++) + { + Av1Plane plane = (Av1Plane)planeIndex; + int subsamplingX = plane == Av1Plane.Y || !colorConfig.SubSamplingX ? 0 : 1; + int subsamplingY = plane == Av1Plane.Y || !colorConfig.SubSamplingY ? 0 : 1; + int planeHeight = (Height + subsamplingY) >> subsamplingY; + if (bitDepth == Av1BitDepth.EightBit) + { + Buffer2DRegion planeSamples = decoded.DeriveBlockPointer(plane, subsamplingX, subsamplingY); + for (int y = 0; y < planeHeight; y++) + { + rawOutput.Write(planeSamples.DangerousGetRowSpan(y)); + } + } + else + { + for (int y = 0; y < planeHeight; y++) + { + foreach (ushort value in decoded.GetHighBitDepthRowSpan(plane, y, subsamplingX, subsamplingY)) + { + // Raw high-bit-depth output uses explicit little-endian samples on every host. + rawOutput.Write(value); + } + } + } + } + } + + ObuFrameHeader frameHeader = Assert.IsType(decoder.FrameHeader); + Assert.Equal(ObuFrameType.InterFrame, frameHeader.FrameType); + Assert.Equal(Av1InterpolationFilter.Switchable, frameHeader.InterpolationFilter); + Av1FrameInfo frameInfo = Assert.IsType(decoder.FrameInfo); + bool hasMotion = false; + bool hasFractionalChromaMotion = false; + foreach (Av1BlockModeInfo mode in frameInfo.GetModeInfos(Point.Empty, frameInfo.GetModeInfoCount(Point.Empty))) + { + if (mode.ReferenceFrames[0] == Av1ReferenceFrameType.Last) + { + Av1MotionVector vector = mode.MotionVectors[0]; + hasMotion |= vector.Column != 0 || vector.Row != 0; + + // A subsampled chroma phase repeats every two luma pixels, or sixteen Q3 motion units. + int chromaPhaseMask = (Av1MotionVector.SubpixelScale << 1) - 1; + hasFractionalChromaMotion |= + (colorConfig.SubSamplingX && (vector.Column & chromaPhaseMask) != 0) || + (colorConfig.SubSamplingY && (vector.Row & chromaPhaseMask) != 0); + } + } + + Assert.True(hasMotion); + if (colorConfig.SubSamplingX || colorConfig.SubSamplingY) + { + Assert.True(hasFractionalChromaMotion); + } + } + + /// + /// Verifies retained reference reconstruction and effort-dependent filter signaling through production sequence decoding. + /// + [Theory] + [InlineData(5, false, false)] + [InlineData(7, false, false)] + [InlineData(8, true, false)] + [InlineData(9, true, true)] + public void SequenceEncoderUsesRetainedReconstructionForInterFrame(int effort, bool switchableFilters, bool dualFilters) + { + const int Width = 16; + const int Height = 16; + Rgba32 sourceColor = new(48, 96, 192); + using Image source = new(Width, Height, sourceColor); + using MemoryStream firstSample = new(); + using MemoryStream secondSample = new(); + ObuColorConfig colorConfig = CreateColorConfig(Av1BitDepth.EightBit, Av1ColorFormat.Yuv420); + using Av1FrameEncoder.SequenceEncoder encoder = Av1FrameEncoder.CreateColorSequenceEncoder( + Configuration.Default, + Width, + Height, + colorConfig, + qIndex: 37, + effort); + + encoder.EncodeKeyFrame(source.Frames.RootFrame, firstSample); + encoder.EncodeInterFrame(source.Frames.RootFrame, secondSample); + + // Retain the exact two-sample elementary stream for independent reference-decoder acceptance. + string outputDirectory = TestEnvironment.CreateOutputDirectory("Heif", "Av1", nameof(this.SequenceEncoderUsesRetainedReconstructionForInterFrame)); + using (FileStream output = File.Create(Path.Combine(outputDirectory, $"effort-{effort}.obu"))) + { + firstSample.Position = 0; + firstSample.CopyTo(output); + secondSample.Position = 0; + secondSample.CopyTo(output); + } + + using Av1Decoder decoder = new(Configuration.Default); + using ImageFrame decodedFirst = decoder.DecodeSequenceFrame( + firstSample.ToArray(), + null, + null); + + Av1FrameInfo firstFrameInfo = Assert.IsType(decoder.FrameInfo); + foreach (Av1BlockModeInfo mode in firstFrameInfo.GetModeInfos(Point.Empty, firstFrameInfo.GetModeInfoCount(Point.Empty))) + { + Assert.Equal(Av1ReferenceFrameType.Intra, mode.ReferenceFrames[0]); + Assert.False(mode.UseIntraBlockCopy); + Assert.False(mode.Skip); + } + + using ImageFrame decodedSecond = decoder.DecodeSequenceFrame( + secondSample.ToArray(), + null, + null); + + ObuFrameHeader frameHeader = decoder.FrameHeader; + Assert.Equal(ObuFrameType.InterFrame, frameHeader.FrameType); + Assert.False(frameHeader.SegmentationParameters.Enabled); + Assert.False(frameHeader.AllowScreenContentTools); + Assert.False(frameHeader.ForceIntegerMotionVector); + Assert.Equal(effort >= 8, frameHeader.AllowHighPrecisionMotionVector); + Assert.Equal(37, frameHeader.QuantizationParameters.BaseQIndex); + Assert.Equal(switchableFilters ? Av1InterpolationFilter.Switchable : Av1InterpolationFilter.Regular, frameHeader.InterpolationFilter); + Assert.Equal(dualFilters, decoder.SequenceHeader.EnableDualFilter); + Av1FrameInfo secondFrameInfo = Assert.IsType(decoder.FrameInfo); + bool hasSkippedInterBlock = false; + foreach (Av1BlockModeInfo mode in secondFrameInfo.GetModeInfos(Point.Empty, secondFrameInfo.GetModeInfoCount(Point.Empty))) + { + hasSkippedInterBlock |= mode.ReferenceFrames[0] == Av1ReferenceFrameType.Last && mode.Skip; + } + + // Repeated frames still use the inter skip alternative when prediction supplies the retained samples. + Assert.True(hasSkippedInterBlock); + for (int y = 0; y < Height; y++) + { + Assert.Equal( + decodedFirst.PixelBuffer.DangerousGetRowSpan(y), + decodedSecond.PixelBuffer.DangerousGetRowSpan(y)); + } + } + + [Fact] + public void SequenceEncoderWritesSelectedGlobalTranslation() + { + const int Width = 64; + const int Height = 64; + const int HorizontalOffset = 4; + using Image first = new(Width, Height); + using Image second = new(Width, Height); + for (int y = 0; y < Height; y++) + { + Span firstRow = first.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(y); + for (int x = 0; x < Width; x++) + { + byte value = (byte)(((x * 37) + (y * 53) + ((x * y) * 11)) & byte.MaxValue); + firstRow[x] = new Rgba32(value, value, value); + } + } + + for (int y = 0; y < Height; y++) + { + ReadOnlySpan firstRow = first.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(y); + Span secondRow = second.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(y); + for (int x = 0; x < Width; x++) + { + secondRow[x] = firstRow[Math.Min(x + HorizontalOffset, Width - 1)]; + } + } + + using MemoryStream firstSample = new(); + using MemoryStream secondSample = new(); + ObuColorConfig colorConfig = CreateColorConfig(Av1BitDepth.EightBit, Av1ColorFormat.Yuv420); + using Av1FrameEncoder.SequenceEncoder encoder = Av1FrameEncoder.CreateColorSequenceEncoder( + Configuration.Default, + Width, + Height, + colorConfig, + qIndex: 4, + effort: 6); + + encoder.EncodeKeyFrame(first.Frames.RootFrame, firstSample); + encoder.EncodeInterFrame(second.Frames.RootFrame, secondSample); + + using Av1Decoder decoder = new(Configuration.Default); + using ImageFrame decodedFirst = decoder.DecodeSequenceFrame( + firstSample.ToArray(), + null, + null); + + using ImageFrame decodedSecond = decoder.DecodeSequenceFrame( + secondSample.ToArray(), + null, + null); + + ObuFrameHeader frameHeader = decoder.FrameHeader; + Av1GlobalMotionParameters globalMotion = frameHeader.GetGlobalMotionParameters()[0]; + Av1MotionVector vector = globalMotion.GetMotionVector( + frameHeader.AllowHighPrecisionMotionVector, + Av1BlockSize.Block8x8, + default, + frameHeader.ForceIntegerMotionVector); + + Assert.Equal(Av1GlobalMotionType.RotationZoom, globalMotion.Type); + Assert.False(frameHeader.AllowScreenContentTools); + Assert.False(frameHeader.ForceIntegerMotionVector); + Assert.Equal(0, vector.Row); + Assert.Equal(HorizontalOffset * 8, vector.Column); + Assert.Equal(first.Size, decodedFirst.Size); + Assert.Equal(second.Size, decodedSecond.Size); + } + + [Fact] + public void SequenceEncoderRejectsInvalidConversionBeforeAllocatingStorage() + { + ObuColorConfig colorConfig = CreateColorConfig(Av1BitDepth.TenBit, Av1ColorFormat.Yuv420); + colorConfig.MatrixCoefficients = ObuMatrixCoefficients.YCgCoRe; + Configuration configuration = Configuration.Default.Clone(); + TestMemoryAllocator allocator = new(); + allocator.EnableNonThreadSafeLogging(); + configuration.MemoryAllocator = allocator; + + // This internal factory receives resolved AV1 settings. The shared converter already rejects a + // reversible matrix with subsampling; that rejection must occur before any owner can be stranded. + Assert.Throws(() => + { + using Av1FrameEncoder.SequenceEncoder encoder = Av1FrameEncoder.CreateColorSequenceEncoder( + configuration, 32, 32, colorConfig, 17, 9); + }); + + Assert.Empty(allocator.AllocationLog); + Assert.Empty(allocator.ReturnLog); + } + + [Theory] + [InlineData(false, EightBit)] + [InlineData(false, TenBit)] + [InlineData(false, TwelveBit)] + [InlineData(true, EightBit)] + [InlineData(true, TenBit)] + [InlineData(true, TwelveBit)] + public void SequenceEncoderConstructionFailureReturnsEveryAllocation(bool encodeAlpha, int bitDepthValue) + { + ObuColorConfig colorConfig = CreateColorConfig( + (Av1BitDepth)bitDepthValue, + encodeAlpha ? Av1ColorFormat.Yuv400 : Av1ColorFormat.Yuv420); + + Configuration configuration = Configuration.Default.Clone(); + TestMemoryAllocator successfulAllocator = new(); + successfulAllocator.EnableNonThreadSafeLogging(); + configuration.MemoryAllocator = successfulAllocator; + using (Av1FrameEncoder.SequenceEncoder encoder = encodeAlpha + ? Av1FrameEncoder.CreateAlphaSequenceEncoder(configuration, 32, 32, colorConfig, 17, 9) + : Av1FrameEncoder.CreateColorSequenceEncoder(configuration, 32, 32, colorConfig, 17, 9)) + { + Assert.NotEmpty(successfulAllocator.AllocationLog); + } + + Assert.Equal(successfulAllocator.AllocationLog.Count, successfulAllocator.ReturnLog.Count); + for (int failureIndex = 0; failureIndex < successfulAllocator.AllocationLog.Count; failureIndex++) + { + FailingSequenceAllocator allocator = new(failureIndex); + configuration.MemoryAllocator = allocator; + + // Fail each real allocator request, including those made inside nested constructors. A constructor + // that throws never reaches the caller's using statement, so its completed owners must unwind there. + InvalidMemoryOperationException exception = Assert.Throws(() => + { + using Av1FrameEncoder.SequenceEncoder encoder = encodeAlpha + ? Av1FrameEncoder.CreateAlphaSequenceEncoder(configuration, 32, 32, colorConfig, 17, 9) + : Av1FrameEncoder.CreateColorSequenceEncoder(configuration, 32, 32, colorConfig, 17, 9); + }); + + Assert.Equal("Sequence allocation failure.", exception.Message); + Assert.Equal(failureIndex, allocator.AllocationLog.Count); + Assert.All( + allocator.AllocationLog, + allocation => Assert.Single(allocator.ReturnLog, returned => returned.AllocationId == allocation.AllocationId)); + + Assert.Equal(allocator.AllocationLog.Count, allocator.ReturnLog.Count); + } + } + + [Theory] + [InlineData(false, EightBit, Yuv420, 384)] + [InlineData(false, TwelveBit, Yuv444, 288)] + [InlineData(true, EightBit, Yuv400, 192)] + [InlineData(true, TwelveBit, Yuv400, 192)] + public void SequenceEncoderReusesAllocatorOwnedRowStorage( + bool encodeAlpha, + int bitDepthValue, + int colorFormatValue, + int expectedRowStorageLength) + { + const int Width = 64; + const int Height = 64; + Av1BitDepth bitDepth = (Av1BitDepth)bitDepthValue; + Av1ColorFormat colorFormat = (Av1ColorFormat)colorFormatValue; + using Image source = new( + Width, + Height, + new Rgba64(ushort.MaxValue, 32768, 16384, 49152)); + + TestMemoryAllocator allocator = new(); + allocator.EnableNonThreadSafeLogging(); + Configuration configuration = Configuration.Default.Clone(); + configuration.MemoryAllocator = allocator; + ObuColorConfig colorConfig = CreateColorConfig(bitDepth, colorFormat); + TestMemoryAllocator.AllocationRequest rowStorage; + int allocationCount; + using (Av1FrameEncoder.SequenceEncoder encoder = encodeAlpha + ? Av1FrameEncoder.CreateAlphaSequenceEncoder( + configuration, + Width, + Height, + colorConfig, + qIndex: 37, + effort: 6) + : Av1FrameEncoder.CreateColorSequenceEncoder( + configuration, + Width, + Height, + colorConfig, + qIndex: 37, + effort: 6)) + { + rowStorage = Assert.Single( + allocator.AllocationLog, + allocation => allocation.ElementType == typeof(float)); + + allocationCount = allocator.AllocationLog.Count; + using MemoryStream output = new(256 * 1024); + encoder.EncodeKeyFrame(source.Frames.RootFrame, output); + encoder.EncodeInterFrame(source.Frames.RootFrame, output); + + // Fixed sequence geometry lets libaom retain its frame-sized compressor data. The ImageSharp + // sequence encoder must likewise perform every sample conversion and coding pass without another rent. + Assert.Equal(allocationCount, allocator.AllocationLog.Count); + } + + Assert.Equal(expectedRowStorageLength, rowStorage.Length); + Assert.Contains( + allocator.ReturnLog, + returned => returned.AllocationId == rowStorage.AllocationId); + } + + [Theory] + [InlineData(TenBit, 8, 8, 0)] + [InlineData(TwelveBit, 8, 8, 0)] + [InlineData(TenBit, 24, 16, 9)] + [InlineData(TwelveBit, 24, 16, 9)] + [InlineData(TenBit, 16, 24, 10)] + [InlineData(TwelveBit, 16, 24, 10)] + public void LosslessHighBitDepthEncodingPreservesNativePlanes(int bitDepthValue, int width, int height, int effort) + { + Av1BitDepth bitDepth = (Av1BitDepth)bitDepthValue; + using Image source = new(width, height); + for (int row = 0; row < height; row++) + { + Span pixels = source.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(row); + for (int column = 0; column < width; column++) + { + pixels[column] = new Rgb48( + (ushort)(((column * 7001) + (row * 997)) & ushort.MaxValue), + (ushort)(((row * 6007) + (column * 1231)) & ushort.MaxValue), + (ushort)(((column * 4001) + (row * 3001)) & ushort.MaxValue)); + } + } + + ObuColorConfig colorConfig = new() + { + IsColorDescriptionPresent = true, + ColorPrimaries = ObuColorPrimaries.Bt709, + TransferCharacteristics = ObuTransferCharacteristics.Srgb, + MatrixCoefficients = ObuMatrixCoefficients.Identity, + ColorRange = true, + BitDepth = bitDepth + }; + + using Av1EncoderFrameBuffer expected = new( + Configuration.Default, + width, + height, + bitDepth.GetBitCount(), + Av1ColorFormat.Yuv444, + 1, + 1); + + Av1FrameEncoder.PrepareSource( + Configuration.Default, + source.Frames.RootFrame, + expected.Frame, + colorConfig); + + using MemoryStream stream = new(); + Av1FrameEncoder.Encode( + Configuration.Default, + source.Frames.RootFrame, + stream, + colorConfig, + qIndex: 0, + effort); + + byte[] payload = stream.ToArray(); + string outputDirectory = Path.Combine( + TestEnvironment.ActualOutputDirectoryFullPath, + "Formats", + "Heif", + "Av1"); + + Directory.CreateDirectory(outputDirectory); + string outputName = $"encoder-frame-{width}x{height}-{bitDepth.GetBitCount()}b-444-lossless-effort{effort}"; + File.WriteAllBytes( + Path.Combine(outputDirectory, outputName + ".obu"), + payload); + + using Av1Decoder decoder = new(Configuration.Default); + using Av1FrameBuffer actual = decoder.DecodeFrameBuffer(payload, null, null, out _); + ObuFrameHeader frameHeader = Assert.IsType(decoder.FrameHeader); + + Assert.Equal(width, actual.Width); + Assert.Equal(height, actual.Height); + Assert.True(frameHeader.CodedLossless); + Assert.True(frameHeader.AllLossless); + Assert.Equal(Av1TransformMode.Only4x4, frameHeader.TransformMode); + + // Lossless native planes are the oracle for external decoding, not the packed RGB conversion on return. + // UInt16 raw samples are explicitly little-endian even when these tests run on a different host byte order. + using BinaryWriter rawOutput = new(File.Create(Path.Combine(outputDirectory, outputName + ".source.yuv"))); + foreach (Av1Plane plane in new[] { Av1Plane.Y, Av1Plane.U, Av1Plane.V }) + { + Buffer2DRegion expectedPlane = expected.Frame.View.GetPlane(plane); + for (int row = 0; row < height; row++) + { + ReadOnlySpan expectedRow = expectedPlane.DangerousGetRowSpan(row); + Assert.Equal(expectedRow, actual.GetHighBitDepthRowSpan(plane, row, 0, 0)); + foreach (ushort sample in expectedRow) + { + rawOutput.Write(sample); + } + } + } + } + + /// + /// Verifies that live partition search preserves lossless syntax across clipped parent nodes and superblocks. + /// + [Theory] + [InlineData(48, 24, 9)] + [InlineData(24, 48, 9)] + [InlineData(80, 24, 9)] + [InlineData(24, 80, 9)] + [InlineData(96, 24, 10)] + [InlineData(24, 96, 10)] + public void EncodeLosslessPartitionSearchAcrossClippedSuperblocks(int width, int height, int effort) + { + ReadOnlySpan period = [0, 28, 40, 28, 0, -28, -40, -12]; + using Image source = new(width, height); + for (int y = 0; y < height; y++) + { + Span row = source.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(y); + for (int x = 0; x < width; x++) + { + row[x] = new L8((byte)(128 + period[x % period.Length] + period[y % period.Length])); + } + } + + // The repeated surface favors larger early leaves. Later clipped parents must still split from their + // own geometry instead of reading a stale position in the original fixed-eight partition preorder. + ObuColorConfig colorConfig = CreateColorConfig(Av1BitDepth.EightBit, Av1ColorFormat.Yuv400); + colorConfig.ColorRange = true; + using MemoryStream stream = new(); + Av1FrameEncoder.Encode(Configuration.Default, source.Frames.RootFrame, stream, colorConfig, qIndex: 0, effort); + byte[] payload = stream.ToArray(); + string outputDirectory = TestEnvironment.CreateOutputDirectory("Heif", "Av1", nameof(this.EncodeLosslessPartitionSearchAcrossClippedSuperblocks)); + string outputName = $"{width}x{height}-effort{effort}"; + File.WriteAllBytes(Path.Combine(outputDirectory, outputName + ".obu"), payload); + using Av1Decoder decoder = new(Configuration.Default); + using Av1FrameBuffer decoded = decoder.DecodeFrameBuffer(payload, null, null, out _); + Assert.Equal(width, decoded.Width); + Assert.Equal(height, decoded.Height); + Buffer2DRegion actual = decoded.DeriveBlockPointer(Av1Plane.Y, 0, 0); + using FileStream rawOutput = File.Create(Path.Combine(outputDirectory, outputName + ".source.yuv")); + for (int y = 0; y < height; y++) + { + ReadOnlySpan expectedRow = MemoryMarshal.AsBytes(source.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(y)); + Assert.Equal(expectedRow, actual.DangerousGetRowSpan(y)); + rawOutput.Write(expectedRow); + } + } + + [Fact] + public void EncodeEffortNineSelectsSubEightPartition() + { + const int Size = 16; + using Image source = new(Size, Size); + for (int y = 0; y < Size; y++) + { + Span row = source.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(y); + for (int x = 0; x < Size; x++) + { + // The bottom-right 8x8 uses horizontal prediction on its left half and vertical prediction + // on its right half. Twelve source values keep a parent palette from reproducing both halves. + byte value; + if (x < 8 && y < 8) + { + value = 128; + } + else if (y < 8) + { + value = (byte)(16 + ((x - 8) * 20)); + } + else + { + value = x < 12 + ? (byte)(176 + ((y - 8) * 9)) + : (byte)(16 + ((x - 8) * 20)); + } + + row[x] = new Rgba32(value, value, value); + } + } + + using MemoryStream stream = new(); + _ = Av1FrameEncoder.Encode( + Configuration.Default, + source.Frames.RootFrame, + stream, + CreateColorConfig(Av1BitDepth.EightBit, Av1ColorFormat.Yuv400), + qIndex: 4, + effort: 9); + + byte[] payload = stream.ToArray(); + using Av1Decoder decoder = new(Configuration.Default); + using Image decoded = decoder.Decode(payload); + Av1FrameInfo frameInfo = Assert.IsType(decoder.FrameInfo); + Point[] leafPositions = + [ + new(2, 2), + new(3, 2), + new(2, 3), + new(3, 3) + ]; + + foreach (Point leafPosition in leafPositions) + { + Assert.Equal( + Av1BlockSize.Block4x8, + frameInfo.GetModeInfoAt(leafPosition).BlockSize); + } + + Assert.Equal(new Size(Size, Size), decoded.Size); + } + + [Fact] + public void EncodeEffortNineSelectsSixteenBySixteenVerticalPartition() + { + const int Size = 32; + using Image source = new(Size, Size); + for (int y = 0; y < Size; y++) + { + Span row = source.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(y); + for (int x = 0; x < Size; x++) + { + byte value = 128; + if (x == 15 && y >= 16) + { + value = (byte)(24 + ((y - 16) * 13)); + } + else if (y == 15 && x >= 16) + { + value = (byte)(16 + ((x - 16) * 15)); + } + else if (x >= 16 && y >= 16) + { + // The left 8x16 half repeats its external left edge, while the right half repeats + // its external top edge. One 16x16 predictor cannot reproduce both surfaces. + value = x < 24 + ? (byte)(24 + ((y - 16) * 13)) + : (byte)(16 + ((x - 16) * 15)); + } + + row[x] = new Rgba32(value, value, value); + } + } + + using MemoryStream stream = new(); + _ = Av1FrameEncoder.Encode( + Configuration.Default, + source.Frames.RootFrame, + stream, + CreateColorConfig(Av1BitDepth.EightBit, Av1ColorFormat.Yuv400), + qIndex: 4, + effort: 9); + + byte[] payload = stream.ToArray(); + using Av1Decoder decoder = new(Configuration.Default); + using Image decoded = decoder.Decode(payload); + Av1FrameInfo frameInfo = Assert.IsType(decoder.FrameInfo); + for (int modeInfoY = 4; modeInfoY < 8; modeInfoY++) + { + for (int modeInfoX = 4; modeInfoX < 8; modeInfoX++) + { + Assert.Equal( + Av1BlockSize.Block8x16, + frameInfo.GetModeInfoAt(new Point(modeInfoX, modeInfoY)).BlockSize); + } + } + + Assert.Equal(new Size(Size, Size), decoded.Size); + } + + [Fact] + public void EncodeEffortTenSelectsThirtyTwoByThirtyTwoBlocks() + { + const int Size = 32; + using Image source = new(Size, Size); + for (int y = 0; y < Size; y++) + { + Span row = source.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(y); + for (int x = 0; x < Size; x++) + { + row[x] = new Rgba32(128, 128, 128); + } + } + + using MemoryStream stream = new(); + _ = Av1FrameEncoder.Encode( + Configuration.Default, + source.Frames.RootFrame, + stream, + CreateColorConfig(Av1BitDepth.EightBit, Av1ColorFormat.Yuv400), + qIndex: 4, + effort: 10); + + byte[] payload = stream.ToArray(); + using Av1Decoder decoder = new(Configuration.Default); + using Image decoded = decoder.Decode(payload); + Av1FrameInfo frameInfo = Assert.IsType(decoder.FrameInfo); + for (int modeInfoY = 0; modeInfoY < 8; modeInfoY++) + { + for (int modeInfoX = 0; modeInfoX < 8; modeInfoX++) + { + Assert.Equal( + Av1BlockSize.Block32x32, + frameInfo.GetModeInfoAt(new Point(modeInfoX, modeInfoY)).BlockSize); + } + } + + Assert.Equal(new Size(Size, Size), decoded.Size); + } + + [Theory] + [InlineData(Yuv400)] + [InlineData(Yuv444)] + public void EncodeEffortTenSelectsSixtyFourBySixtyFourBlock(int colorFormatValue) + { + const int Size = 64; + Av1ColorFormat colorFormat = (Av1ColorFormat)colorFormatValue; + using Image source = new(Size, Size); + for (int y = 0; y < Size; y++) + { + Span row = source.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(y); + for (int x = 0; x < Size; x++) + { + row[x] = new Rgba32(180, 64, 220); + } + } + + using MemoryStream stream = new(); + _ = Av1FrameEncoder.Encode( + Configuration.Default, + source.Frames.RootFrame, + stream, + CreateColorConfig(Av1BitDepth.EightBit, colorFormat), + qIndex: 4, + effort: 10); + + byte[] payload = stream.ToArray(); + using Av1Decoder decoder = new(Configuration.Default); + using Image decoded = decoder.Decode(payload); + Av1FrameInfo frameInfo = Assert.IsType(decoder.FrameInfo); + for (int modeInfoY = 0; modeInfoY < 16; modeInfoY++) + { + for (int modeInfoX = 0; modeInfoX < 16; modeInfoX++) + { + Assert.Equal( + Av1BlockSize.Block64x64, + frameInfo.GetModeInfoAt(new Point(modeInfoX, modeInfoY)).BlockSize); + } + } + + Assert.Equal(new Size(Size, Size), decoded.Size); + } + + [Theory] + [InlineData(Yuv400)] + [InlineData(Yuv444)] + public void EncodeEffortTenSelectsOneHundredTwentyEightByOneHundredTwentyEightBlock(int colorFormatValue) + { + const int Size = 128; + Av1ColorFormat colorFormat = (Av1ColorFormat)colorFormatValue; + using Image source = new(Size, Size); + for (int y = 0; y < Size; y++) + { + Span row = source.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(y); + for (int x = 0; x < Size; x++) + { + row[x] = new Rgba32(180, 64, 220); + } + } + + using MemoryStream stream = new(); + ObuSequenceHeader sequenceHeader = Av1FrameEncoder.Encode( + Configuration.Default, + source.Frames.RootFrame, + stream, + CreateColorConfig(Av1BitDepth.EightBit, colorFormat), + qIndex: 4, + effort: 10); + + Assert.True(sequenceHeader.Use128x128Superblock); + byte[] payload = stream.ToArray(); + using Av1Decoder decoder = new(Configuration.Default); + using Image decoded = decoder.Decode(payload); + Av1FrameInfo frameInfo = Assert.IsType(decoder.FrameInfo); + for (int modeInfoY = 0; modeInfoY < 32; modeInfoY++) + { + for (int modeInfoX = 0; modeInfoX < 32; modeInfoX++) + { + Assert.Equal( + Av1BlockSize.Block128x128, + frameInfo.GetModeInfoAt(new Point(modeInfoX, modeInfoY)).BlockSize); + } + } + + Assert.Equal(new Size(Size, Size), decoded.Size); + } + + [Fact] + public void EncodeEffortTenSearchesHighBitDepthOneHundredTwentyEightRoot() + { + const int Size = 128; + using Image source = new(Size, Size); + for (int y = 0; y < Size; y++) + { + Span row = source.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(y); + for (int x = 0; x < Size; x++) + { + row[x] = new Rgba32(180, 64, 220); + } + } + + using MemoryStream stream = new(); + ObuSequenceHeader sequenceHeader = Av1FrameEncoder.Encode( + Configuration.Default, + source.Frames.RootFrame, + stream, + CreateColorConfig(Av1BitDepth.TwelveBit, Av1ColorFormat.Yuv444), + qIndex: 4, + effort: 10); + + Assert.True(sequenceHeader.Use128x128Superblock); + byte[] payload = stream.ToArray(); + using Av1Decoder decoder = new(Configuration.Default); + using Image decoded = decoder.Decode(payload); + Assert.Equal(new Size(Size, Size), decoded.Size); + } + + [Theory] + [InlineData(EightBit)] + [InlineData(TenBit)] + [InlineData(TwelveBit)] + public void EncodeAlphaWritesMonochromeReducedStillPicture(int bitDepthValue) + { + const int Width = 16; + const int Height = 16; + Av1BitDepth bitDepth = (Av1BitDepth)bitDepthValue; + using Image source = new(Width, Height); + for (int y = 0; y < Height; y++) + { + Span row = source.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(y); + for (int x = 0; x < Width; x++) + { + ushort alpha = (ushort)(((x + y) * ushort.MaxValue) / (Width + Height - 2)); + row[x] = new Rgba64(ushort.MaxValue, 0, 0, alpha); + } + } + + using MemoryStream stream = new(); + ObuSequenceHeader encodedHeader = Av1FrameEncoder.EncodeAlpha( + Configuration.Default, + source.Frames.RootFrame, + stream, + CreateColorConfig(bitDepth), + qIndex: 37, + effort: 5); + + byte[] payload = stream.ToArray(); + using Av1Decoder decoder = new(Configuration.Default); + using Image decoded = decoder.Decode(payload); + + Assert.True(encodedHeader.ColorConfig.IsMonochrome); + Assert.Equal( + bitDepth == Av1BitDepth.TwelveBit ? ObuSequenceProfile.Professional : ObuSequenceProfile.Main, + encodedHeader.SequenceProfile); + + Assert.Equal(new Size(Width, Height), decoded.Size); + Assert.True(decoded[0, 0].R < decoded[Width - 1, Height - 1].R); + Assert.Equal(decoded[0, 0].R, decoded[0, 0].G); + Assert.Equal(decoded[0, 0].R, decoded[0, 0].B); + Assert.Equal(ushort.MaxValue, decoded[0, 0].A); + + string outputDirectory = Path.Combine( + TestEnvironment.ActualOutputDirectoryFullPath, + "Formats", + "Heif", + "Av1"); + + Directory.CreateDirectory(outputDirectory); + File.WriteAllBytes( + Path.Combine(outputDirectory, $"encoder-alpha-{Width}x{Height}-{bitDepth.GetBitCount()}b.obu"), + payload); + } + + [Fact] + public void AlphaConversionUsesOnePooledRowAndPreservesTwelveBitPrecision() + { + const int Width = 19; + const int Border = Av1EncoderFrame.LumaBorder; + using Image image = new(Width, 1); + ushort[] expected = new ushort[Width]; + for (int x = 0; x < Width; x++) + { + ushort alpha = (ushort)((x * (long)ushort.MaxValue) / (Width - 1)); + image[x, 0] = new Rgba64(0, 0, 0, alpha); + expected[x] = (ushort)(((alpha * 4095L) + (ushort.MaxValue / 2)) / ushort.MaxValue); + } + + using Av1EncoderFrameBuffer frameBuffer = new( + Configuration.Default, + Width, + 1, + 12, + Av1ColorFormat.Yuv400, + 0, + 0); + + TestMemoryAllocator allocator = new(); + allocator.EnableNonThreadSafeLogging(); + Configuration configuration = Configuration.Default.Clone(); + configuration.MemoryAllocator = allocator; + + HeifPlanarAlphaEncoder.Convert< + Rgba64, + Av1EncoderFrame.PlanarView, + ushort, + HeifUShortSampleConverter>( + configuration, + image.Frames.RootFrame, + frameBuffer.Frame.View); + + frameBuffer.Frame.ExtendBorders(); + + AssertReplicatedSingleRow(frameBuffer.Luma, Border, expected); + TestMemoryAllocator.AllocationRequest allocation = Assert.Single(allocator.AllocationLog); + Assert.Equal(typeof(float), allocation.ElementType); + Assert.Equal(Width * 3, allocation.Length); + TestMemoryAllocator.ReturnRequest returned = Assert.Single(allocator.ReturnLog); + Assert.Equal(allocation.AllocationId, returned.AllocationId); + } + + [Theory] + [InlineData(EightBit, Yuv400, 0x1F, 0x1C)] + [InlineData(TenBit, Yuv420, 0x1F, 0x4C)] + [InlineData(TenBit, Yuv444, 0x3F, 0x40)] + [InlineData(TwelveBit, Yuv422, 0x5F, 0x68)] + public void CodecConfigurationWritesFixedHeaderFromEncodedSequenceHeader( + int bitDepthValue, + int colorFormatValue, + byte expectedProfileAndLevel, + byte expectedColorFlags) + { + Av1BitDepth bitDepth = (Av1BitDepth)bitDepthValue; + Av1ColorFormat colorFormat = (Av1ColorFormat)colorFormatValue; + using Image source = new(8, 8); + using MemoryStream stream = new(); + ObuSequenceHeader sequenceHeader = Av1FrameEncoder.Encode( + Configuration.Default, + source.Frames.RootFrame, + stream, + CreateColorConfig(bitDepth, colorFormat), + qIndex: 37, + effort: 5); + + Av1CodecConfiguration configuration = new(sequenceHeader); + byte[] fixedHeader = new byte[Av1CodecConfiguration.FixedHeaderSize]; + configuration.WriteFixedHeader(fixedHeader); + + Assert.Equal([0x81, expectedProfileAndLevel, expectedColorFlags, 0x00], fixedHeader); + Av1CodecConfiguration parsed = new(fixedHeader, new DecoderOptions()); + Assert.True(configuration.HasMatchingImageConfiguration(parsed)); + parsed.Validate(sequenceHeader); + } + + [Fact] + public void ScreenContentDetectorMatchesLibaomFeatureThresholds() + { + const int width = 160; + const int height = 16; + using Av1EncoderFrameBuffer byteFrame = new( + Configuration.Default, + width, + height, + 8, + Av1ColorFormat.Yuv400, + 0, + 0); + + for (int row = 0; row < height; row++) + { + Span samples = byteFrame.Frame.View.GetLumaRowSpan(row)[..width]; + samples.Fill(96); + for (int column = 0; column < 16; column++) + { + samples[column] = column < 8 ? (byte)32 : (byte)224; + } + } + + // One qualifying block is exactly ten percent of this frame, and the reference threshold is strict. + Assert.False(Av1ScreenContentDetector.IsPaletteLikely(byteFrame.Frame)); + Av1ScreenContentDetector.Detect( + byteFrame.Frame, + out bool allowScreenContentTools, + out bool allowIntraBlockCopy); + + Assert.False(allowScreenContentTools); + Assert.False(allowIntraBlockCopy); + for (int row = 0; row < height; row++) + { + Span samples = byteFrame.Frame.View.GetLumaRowSpan(row); + for (int column = 16; column < 32; column++) + { + samples[column] = column < 24 ? (byte)48 : (byte)208; + } + } + + Assert.True(Av1ScreenContentDetector.IsPaletteLikely(byteFrame.Frame)); + Av1ScreenContentDetector.Detect( + byteFrame.Frame, + out allowScreenContentTools, + out allowIntraBlockCopy); + + Assert.True(allowScreenContentTools); + Assert.True(allowIntraBlockCopy); + using Av1EncoderFrameBuffer highBitDepthFrame = new( + Configuration.Default, + 16, + 16, + 10, + Av1ColorFormat.Yuv400, + 0, + 0); + + for (int row = 0; row < 16; row++) + { + Span samples = highBitDepthFrame.Frame.View.GetLumaRowSpan(row); + for (int column = 0; column < 16; column++) + { + samples[column] = column < 8 ? (ushort)128 : (ushort)131; + } + } + + Assert.False(Av1ScreenContentDetector.IsPaletteLikely(highBitDepthFrame.Frame)); + Av1ScreenContentDetector.Detect( + highBitDepthFrame.Frame, + out allowScreenContentTools, + out allowIntraBlockCopy); + + Assert.False(allowScreenContentTools); + Assert.False(allowIntraBlockCopy); + for (int row = 0; row < 16; row++) + { + Span samples = highBitDepthFrame.Frame.View.GetLumaRowSpan(row); + samples[8..16].Fill(640); + } + + Assert.True(Av1ScreenContentDetector.IsPaletteLikely(highBitDepthFrame.Frame)); + Av1ScreenContentDetector.Detect( + highBitDepthFrame.Frame, + out allowScreenContentTools, + out allowIntraBlockCopy); + + Assert.True(allowScreenContentTools); + Assert.True(allowIntraBlockCopy); + for (int row = 0; row < 16; row++) + { + Span samples = highBitDepthFrame.Frame.View.GetLumaRowSpan(row); + for (int column = 0; column < 16; column++) + { + samples[column] = (ushort)((column % 5) * 200); + } + } + + Assert.False(Av1ScreenContentDetector.IsPaletteLikely(highBitDepthFrame.Frame)); + Av1ScreenContentDetector.Detect( + highBitDepthFrame.Frame, + out allowScreenContentTools, + out allowIntraBlockCopy); + + Assert.False(allowScreenContentTools); + Assert.False(allowIntraBlockCopy); + } + + [Fact] + public void ScreenContentDetectorMatchesLibaomIntraBlockCopyVarianceThreshold() + { + const int Width = 16; + const int Height = 16; + using Av1EncoderFrameBuffer frame = new( + Configuration.Default, + Width, + Height, + 8, + Av1ColorFormat.Yuv400, + 0, + 0); + + Buffer2DRegion luma = frame.Frame.View.GetPlane(Av1Plane.Y); + for (int row = 0; row < Height; row++) + { + luma.DangerousGetRowSpan(row).Fill(96); + } + + // A single delta of eleven leaves total variance below half a sample after per-pixel rounding. + luma.DangerousGetRowSpan(0)[0] = 107; + Av1ScreenContentDetector.Detect( + frame.Frame, + out bool allowScreenContentTools, + out bool allowIntraBlockCopy); + + Assert.True(allowScreenContentTools); + Assert.False(allowIntraBlockCopy); + + // Raising that delta to twelve crosses the exact integer rounding boundary used by libaom. + luma.DangerousGetRowSpan(0)[0] = 108; + Av1ScreenContentDetector.Detect( + frame.Frame, + out allowScreenContentTools, + out allowIntraBlockCopy); + + Assert.True(allowScreenContentTools); + Assert.True(allowIntraBlockCopy); + } + + [Fact] + public void EncodeActivatesScreenContentTools() + { + const int width = 16; + const int height = 16; + using Image source = new(width, height); + for (int row = 0; row < height; row++) + { + Span pixels = source.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(row); + for (int column = 0; column < width; column++) + { + pixels[column] = (((column >> 2) + (row >> 2)) & 1) == 0 + ? new Rgba32(224, 32, 32) + : new Rgba32(32, 32, 224); + } + } + + ObuColorConfig colorConfig = CreateColorConfig(Av1BitDepth.EightBit, Av1ColorFormat.Yuv444); + using MemoryStream stream = new(); + _ = Av1FrameEncoder.Encode( + Configuration.Default, + source.Frames.RootFrame, + stream, + colorConfig, + qIndex: 37, + effort: 5); + + byte[] payload = stream.ToArray(); + Av1BitStreamReader reader = new(payload); + Av1TileDecoderStub tileReader = new(); + ObuReader obuReader = new(); + obuReader.ReadAll(ref reader, payload.Length, () => tileReader); + ObuFrameHeader frameHeader = Assert.IsType(obuReader.FrameHeader); + Assert.True(frameHeader.AllowScreenContentTools); + Assert.True(frameHeader.AllowIntraBlockCopy); + using Av1Decoder decoder = new(Configuration.Default); + using Image decoded = decoder.Decode(payload); + Assert.Equal(new Size(width, height), decoded.Size); + + string outputDirectory = Path.Combine( + TestEnvironment.ActualOutputDirectoryFullPath, + "Formats", + "Heif", + "Av1"); + + Directory.CreateDirectory(outputDirectory); + File.WriteAllBytes(Path.Combine(outputDirectory, "encoder-frame-16x16-8b-444-palette.obu"), payload); + } + + [Theory] + [InlineData(0, false, false, false)] + [InlineData(1, false, false, false)] + [InlineData(2, false, false, false)] + [InlineData(3, false, false, false)] + [InlineData(4, true, false, false)] + [InlineData(5, true, true, false)] + [InlineData(6, true, true, true)] + [InlineData(7, true, true, true)] + [InlineData(8, true, true, true)] + [InlineData(10, true, true, true)] + public void EncodeEffortControlsSearchFeatures( + int effort, + bool enableFilterIntra, + bool enableScreenContentTools, + bool selectTransformSize) + { + const int width = 16; + const int height = 16; + using Image source = new(width, height); + for (int row = 0; row < height; row++) + { + Span pixels = source.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(row); + for (int column = 0; column < width; column++) + { + pixels[column] = (((column >> 2) + (row >> 2)) & 1) == 0 + ? new Rgba32(224, 32, 32) + : new Rgba32(32, 32, 224); + } + } + + ObuColorConfig colorConfig = CreateColorConfig(Av1BitDepth.EightBit, Av1ColorFormat.Yuv444); + using MemoryStream stream = new(); + _ = Av1FrameEncoder.Encode( + Configuration.Default, + source.Frames.RootFrame, + stream, + colorConfig, + qIndex: 37, + effort); + + byte[] payload = stream.ToArray(); + using Av1Decoder decoder = new(Configuration.Default); + using Image decoded = decoder.Decode(payload); + ObuSequenceHeader sequenceHeader = Assert.IsType(decoder.SequenceHeader); + ObuFrameHeader frameHeader = Assert.IsType(decoder.FrameHeader); + Av1FrameInfo frameInfo = Assert.IsType(decoder.FrameInfo); + Assert.Equal(enableFilterIntra, sequenceHeader.EnableFilterIntra); + Assert.Equal(enableScreenContentTools, frameHeader.AllowScreenContentTools); + Assert.Equal(enableScreenContentTools, frameHeader.AllowIntraBlockCopy); + Assert.Equal( + selectTransformSize ? Av1TransformMode.Select : Av1TransformMode.Largest, + frameHeader.TransformMode); + Assert.Equal(new Size(width, height), decoded.Size); + + int modeCount = 0; + foreach (Av1BlockModeInfo modeInfo in frameInfo.GetSuperblock(Point.Empty).GetModeInfos()) + { + modeCount++; + if (effort == 0) + { + Assert.Equal(Av1PredictionMode.DC, modeInfo.YMode); + Assert.Equal(Av1ChromaPredictionMode.DC, modeInfo.UvMode); + } + + if (effort <= 1) + { + Assert.Equal(0, modeInfo.GetAngleDelta(Av1Plane.Y)); + Assert.Equal(0, modeInfo.GetAngleDelta(Av1Plane.U)); + } + + if (effort < 4) + { + Assert.False(modeInfo.UseFilterIntra); + } + + if (effort < 5) + { + Assert.False(modeInfo.UseIntraBlockCopy); + Assert.Equal(0, modeInfo.GetPaletteSize(Av1Plane.Y)); + Assert.Equal(0, modeInfo.GetPaletteSize(Av1Plane.U)); + } + } + + Assert.NotEqual(0, modeCount); + + string outputDirectory = Path.Combine( + TestEnvironment.ActualOutputDirectoryFullPath, + "Formats", + "Heif", + "Av1"); + + Directory.CreateDirectory(outputDirectory); + File.WriteAllBytes(Path.Combine(outputDirectory, $"encoder-frame-16x16-8b-444-effort-{effort}.obu"), payload); + } + + [Fact] + public void EncodeEffortSixSelectsFourByFourLumaTransforms() + { + const int Width = 16; + const int Height = 16; + using Image source = new(Width, Height); + for (int row = 0; row < Height; row++) + { + Span pixels = source.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(row); + for (int column = 0; column < Width; column++) + { + byte value = (byte)(16 + ((((row >> 2) * 4) + (column >> 2)) * 14)); + pixels[column] = new Rgba32(value, value, value); + } + } + + ObuColorConfig colorConfig = CreateColorConfig(Av1BitDepth.EightBit, Av1ColorFormat.Yuv400); + using MemoryStream stream = new(); + _ = Av1FrameEncoder.Encode( + Configuration.Default, + source.Frames.RootFrame, + stream, + colorConfig, + qIndex: 37, + effort: 6); + + byte[] payload = stream.ToArray(); + using Av1Decoder decoder = new(Configuration.Default); + using Image decoded = decoder.Decode(payload); + Assert.NotNull(decoder.FrameHeader); + Assert.Equal(Av1TransformMode.Select, decoder.FrameHeader.TransformMode); + Assert.NotNull(decoder.FrameInfo); + bool foundSplitTransform = false; + foreach (Av1BlockModeInfo modeInfo in decoder.FrameInfo.GetSuperblock(Point.Empty).GetModeInfos()) + { + foundSplitTransform |= modeInfo.GetTransformUnitCount(Av1Plane.Y) == 4; + } + + Assert.True(foundSplitTransform); + Assert.Equal(new Size(Width, Height), decoded.Size); + + string outputDirectory = Path.Combine( + TestEnvironment.ActualOutputDirectoryFullPath, + "Formats", + "Heif", + "Av1"); + + Directory.CreateDirectory(outputDirectory); + File.WriteAllBytes( + Path.Combine(outputDirectory, "encoder-frame-16x16-8b-400-transform-size-select.obu"), + payload); + } + + [Theory] + [InlineData(5, false)] + [InlineData(6, true)] + public void EncodeSelectsIntraBlockCopyForRepeatedScreenContent( + int effort, + bool selectTransformSize) + { + const int Width = 328; + const int Height = 16; + const ulong Pattern = 0xD6A5_3C97_E18B_4F20UL; + using Image source = new(Width, Height); + for (int row = 0; row < Height; row++) + { + Span pixels = source.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(row); + for (int column = 0; column < Width; column++) + { + int patternIndex = ((row & 7) * 8) + (column & 7); + pixels[column] = ((Pattern >> patternIndex) & 1) == 0 + ? new Rgba32(224, 32, 32) + : new Rgba32(32, 32, 224); + } + } + + ObuColorConfig colorConfig = CreateColorConfig(Av1BitDepth.EightBit, Av1ColorFormat.Yuv444); + using MemoryStream stream = new(); + _ = Av1FrameEncoder.Encode( + Configuration.Default, + source.Frames.RootFrame, + stream, + colorConfig, + qIndex: 37, + effort); + + byte[] payload = stream.ToArray(); + using Av1Decoder decoder = new(Configuration.Default); + using Image decoded = decoder.Decode(payload); + Assert.NotNull(decoder.FrameHeader); + Assert.True(decoder.FrameHeader.AllowScreenContentTools); + Assert.True(decoder.FrameHeader.AllowIntraBlockCopy); + Assert.Equal( + selectTransformSize ? Av1TransformMode.Select : Av1TransformMode.Largest, + decoder.FrameHeader.TransformMode); + Assert.NotNull(decoder.FrameInfo); + Av1SuperblockInfo targetSuperblock = decoder.FrameInfo.GetSuperblock(new Point(5, 0)); + bool usesIntraBlockCopy = false; + foreach (Av1BlockModeInfo modeInfo in targetSuperblock.GetModeInfos()) + { + usesIntraBlockCopy |= modeInfo.UseIntraBlockCopy; + } + + Assert.True(usesIntraBlockCopy); + Assert.Equal(new Size(Width, Height), decoded.Size); + + string outputDirectory = Path.Combine( + TestEnvironment.ActualOutputDirectoryFullPath, + "Formats", + "Heif", + "Av1"); + + Directory.CreateDirectory(outputDirectory); + string fileName = effort == 5 + ? "encoder-frame-328x16-8b-444-intrabc.obu" + : "encoder-frame-328x16-8b-444-intrabc-effort-6.obu"; + + File.WriteAllBytes(Path.Combine(outputDirectory, fileName), payload); + } + + [Fact] + public void PrepareSourceConvertsRgba32DirectlyIntoBorderedEightBitPlane() + { + const int width = 4; + const int height = 1; + const int border = Av1EncoderFrame.LumaBorder; + + using Image image = new(width, height); + image[0, 0] = new Rgba32(byte.MaxValue, 0, 0, 0); + image[1, 0] = new Rgba32(0, byte.MaxValue, 0); + image[2, 0] = new Rgba32(0, 0, byte.MaxValue); + image[3, 0] = new Rgba32(byte.MaxValue, byte.MaxValue, byte.MaxValue); + + using Av1EncoderFrameBuffer frameBuffer = new( + Configuration.Default, + width, + height, + 8, + Av1ColorFormat.Yuv400, + 0, + 0); + + ObuColorConfig colorConfig = CreateColorConfig(Av1BitDepth.EightBit); + + Av1FrameEncoder.PrepareSource(Configuration.Default, image.Frames.RootFrame, frameBuffer.Frame, colorConfig); + + byte[] expected = [76, 150, 29, 255]; + AssertReplicatedSingleRow(frameBuffer.Luma, border, expected); + } + + [Fact] + public void PrepareSourcePreservesHighBitDepthPrecision() + { + const int width = 4; + const int height = 1; + const int border = Av1EncoderFrame.LumaBorder; + + using Image image = new(width, height); + image[0, 0] = new Rgba64(ushort.MaxValue, 0, 0, 0); + image[1, 0] = new Rgba64(0, ushort.MaxValue, 0, ushort.MaxValue); + image[2, 0] = new Rgba64(0, 0, ushort.MaxValue, ushort.MaxValue); + image[3, 0] = new Rgba64(ushort.MaxValue, ushort.MaxValue, ushort.MaxValue, ushort.MaxValue); + + using Av1EncoderFrameBuffer frameBuffer = new( + Configuration.Default, + width, + height, + 10, + Av1ColorFormat.Yuv400, + 0, + 0); + + ObuColorConfig colorConfig = CreateColorConfig(Av1BitDepth.TenBit); + + Av1FrameEncoder.PrepareSource(Configuration.Default, image.Frames.RootFrame, frameBuffer.Frame, colorConfig); + + ushort[] expected = [306, 601, 117, 1023]; + AssertReplicatedSingleRow(frameBuffer.Luma, border, expected); + } + + [Fact] + public void ExtendBordersReplicatesEveryPhysicalPlaneEdge() + { + const int visibleWidth = 5; + const int visibleHeight = 3; + const int lumaBorder = Av1EncoderFrame.LumaBorder; + const int chromaBorder = lumaBorder / 2; + + using Av1EncoderFrameBuffer frameBuffer = new( + Configuration.Default, + visibleWidth, + visibleHeight, + 8, + Av1ColorFormat.Yuv420, + 1, + 1); + + Buffer2D luma = frameBuffer.Luma; + Buffer2D chromaBlue = Assert.IsType>(frameBuffer.ChromaBlue); + Buffer2D chromaRed = Assert.IsType>(frameBuffer.ChromaRed); + + FillVisible(luma, lumaBorder, lumaBorder, visibleWidth, visibleHeight, 10); + FillVisible(chromaBlue, chromaBorder, chromaBorder, (visibleWidth + 1) / 2, (visibleHeight + 1) / 2, 80); + FillVisible(chromaRed, chromaBorder, chromaBorder, (visibleWidth + 1) / 2, (visibleHeight + 1) / 2, 120); + + frameBuffer.Frame.ExtendBorders(); + + AssertReplicatedPlane(luma, lumaBorder, lumaBorder, visibleWidth, visibleHeight, 10); + AssertReplicatedPlane(chromaBlue, chromaBorder, chromaBorder, (visibleWidth + 1) / 2, (visibleHeight + 1) / 2, 80); + AssertReplicatedPlane(chromaRed, chromaBorder, chromaBorder, (visibleWidth + 1) / 2, (visibleHeight + 1) / 2, 120); + } + + [Theory] + [InlineData(5, 3, 0, 0, 160, 136)] + [InlineData(5, 3, 1, 0, 80, 136)] + [InlineData(5, 3, 1, 1, 80, 68)] + [InlineData(1921, 1081, 0, 0, 2080, 1216)] + [InlineData(1921, 1081, 1, 1, 1040, 608)] + public void GetPlaneBufferSizeMatchesLibaomLayout( + int width, + int height, + int subsamplingX, + int subsamplingY, + int expectedWidth, + int expectedHeight) + { + Size actual = Av1EncoderFrame.GetPlaneBufferSize(width, height, subsamplingX, subsamplingY); + + Assert.Equal(new Size(expectedWidth, expectedHeight), actual); + } + + [Fact] + public void FrameBufferUsesOneExactSizeOwnerForAllPlanes() + { + TestMemoryAllocator allocator = new(); + allocator.EnableNonThreadSafeLogging(); + Configuration configuration = Configuration.Default.Clone(); + configuration.MemoryAllocator = allocator; + + TestMemoryAllocator.AllocationRequest allocation; + using (Av1EncoderFrameBuffer frameBuffer = new( + configuration, + 64, + 64, + 8, + Av1ColorFormat.Yuv420, + 1, + 1)) + { + allocation = Assert.Single(allocator.AllocationLog); + Assert.Empty(allocator.ReturnLog); + Assert.Equal(typeof(byte), allocation.ElementType); + Assert.Equal(55_296, allocation.Length); + Assert.Single(frameBuffer.Luma.MemoryGroup); + Assert.Single(Assert.IsType>(frameBuffer.ChromaBlue).MemoryGroup); + Assert.Single(Assert.IsType>(frameBuffer.ChromaRed).MemoryGroup); + } + + TestMemoryAllocator.ReturnRequest returned = Assert.Single(allocator.ReturnLog); + Assert.Equal(allocation.AllocationId, returned.AllocationId); + } + + [Fact] + public void EncodeReturnsEveryOperationAllocationAndUsesOneLibaomSizedTileReservation() + { + const int Width = 64; + const int Height = 64; + const int ExpectedTileOutputLength = 60 * 1024; + + using Image source = new(Width, Height); + for (int y = 0; y < Height; y++) + { + Span row = source.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(y); + for (int x = 0; x < Width; x++) + { + row[x] = new Rgba32( + (byte)((x * 3) + y), + (byte)(x + (y * 5)), + (byte)((x * 7) + (y * 11))); + } + } + + TestMemoryAllocator allocator = new(); + allocator.EnableNonThreadSafeLogging(); + Configuration configuration = Configuration.Default.Clone(); + configuration.MemoryAllocator = allocator; + using MemoryStream storage = new(); + using NonSeekableStream destination = new(storage); + + _ = Av1FrameEncoder.Encode( + configuration, + source.Frames.RootFrame, + destination, + CreateColorConfig(Av1BitDepth.TwelveBit, Av1ColorFormat.Yuv444), + qIndex: 37, + effort: 5); + + Assert.False(destination.CanSeek); + Assert.NotEqual(0, storage.Length); + TestMemoryAllocator.AllocationRequest tileOutput = Assert.Single( + allocator.AllocationLog, + allocation => allocation.ElementType == typeof(byte) && allocation.Length == ExpectedTileOutputLength); + + Assert.Equal(ExpectedTileOutputLength, tileOutput.Length); + Assert.Equal(allocator.AllocationLog.Count, allocator.ReturnLog.Count); + Assert.Equal( + allocator.AllocationLog.Select(allocation => allocation.AllocationId).Order(), + allocator.ReturnLog.Select(returned => returned.AllocationId).Order()); + } + + private static ObuColorConfig CreateColorConfig( + Av1BitDepth bitDepth, + Av1ColorFormat colorFormat = Av1ColorFormat.Yuv400) + => new() + { + IsColorDescriptionPresent = true, + IsMonochrome = colorFormat == Av1ColorFormat.Yuv400, + ColorPrimaries = ObuColorPrimaries.Bt601, + TransferCharacteristics = ObuTransferCharacteristics.Bt601, + MatrixCoefficients = ObuMatrixCoefficients.Bt601, + ColorRange = true, + SubSamplingX = colorFormat != Av1ColorFormat.Yuv444, + SubSamplingY = colorFormat == Av1ColorFormat.Yuv400 || colorFormat == Av1ColorFormat.Yuv420, + ChromaSamplePosition = ObuChromoSamplePosition.Unknown, + BitDepth = bitDepth + }; + + private static void FillVisible(Buffer2D plane, int originX, int originY, int width, int height, int seed) + { + for (int y = 0; y < height; y++) + { + Span row = plane.DangerousGetRowSpan(originY + y); + for (int x = 0; x < width; x++) + { + row[originX + x] = (byte)(seed + (y * width) + x); + } + } + } + + private static void AssertReplicatedPlane(Buffer2D plane, int originX, int originY, int width, int height, int seed) + { + for (int y = 0; y < plane.Height; y++) + { + ReadOnlySpan row = plane.DangerousGetRowSpan(y); + int sourceY = Math.Clamp(y - originY, 0, height - 1); + for (int x = 0; x < row.Length; x++) + { + int sourceX = Math.Clamp(x - originX, 0, width - 1); + Assert.Equal((byte)(seed + (sourceY * width) + sourceX), row[x]); + } + } + } + + private static void AssertReplicatedSingleRow( + Buffer2D plane, + int originX, + ReadOnlySpan expected) + where TSample : unmanaged, IEquatable + { + for (int y = 0; y < plane.Height; y++) + { + ReadOnlySpan row = plane.DangerousGetRowSpan(y); + for (int x = 0; x < row.Length; x++) + { + int sourceX = Math.Clamp(x - originX, 0, expected.Length - 1); + Assert.Equal(expected[sourceX], row[x]); + } + } + } + + private sealed class FailingSequenceAllocator : TestMemoryAllocator + { + private readonly int failureIndex; + + /// + /// Initializes a new instance of the class. + /// + /// The zero-based allocation request that fails. + public FailingSequenceAllocator(int failureIndex) + { + this.failureIndex = failureIndex; + this.EnableNonThreadSafeLogging(); + } + + /// + protected override AllocationTrackedMemoryManager AllocateCore(int length, AllocationOptions options) + { + if (this.AllocationLog.Count == this.failureIndex) + { + throw new InvalidMemoryOperationException("Sequence allocation failure."); + } + + return base.AllocateCore(length, options); + } + } +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1EncoderModeInfoBufferTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1EncoderModeInfoBufferTests.cs new file mode 100644 index 0000000000..726a7c3b80 --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1EncoderModeInfoBufferTests.cs @@ -0,0 +1,440 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1; +using SixLabors.ImageSharp.Formats.Heif.Av1.Motion; +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; +using SixLabors.ImageSharp.Memory; +using SixLabors.ImageSharp.Tests.Memory; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1; + +public class Av1EncoderModeInfoBufferTests +{ + [Theory] + [InlineData(false, 1024, 1024, 12_288)] + [InlineData(true, 1024, 256, 6_144)] + public void ConstructorMatchesLibaomAlignedModeInfoGeometry( + bool disallow4x4, + int expectedGridLength, + int expectedAllocationLength, + int expectedStorageLength) + { + TestMemoryAllocator allocator = new(); + allocator.EnableNonThreadSafeLogging(); + Configuration configuration = Configuration.Default.Clone(); + configuration.MemoryAllocator = allocator; + + TestMemoryAllocator.AllocationRequest allocation; + using (Av1EncoderModeInfoBuffer buffer = new(configuration, 65, 33, disallow4x4)) + { + allocation = Assert.Single(allocator.AllocationLog); + Assert.Empty(allocator.ReturnLog); + Assert.Equal(typeof(byte), allocation.ElementType); + Assert.Equal(AllocationOptions.Clean, allocation.AllocationOptions); + Assert.Equal(expectedStorageLength, allocation.Length); + Assert.Equal(18, buffer.ModeInfoColumnCount); + Assert.Equal(10, buffer.ModeInfoRowCount); + Assert.Equal(32, buffer.ModeInfoStride); + Assert.Equal(disallow4x4, buffer.Disallow4x4AllFrames); + Assert.Equal(expectedGridLength, buffer.Grid.Length); + Assert.Equal(expectedAllocationLength, buffer.Allocation.Length); + } + + TestMemoryAllocator.ReturnRequest returned = Assert.Single(allocator.ReturnLog); + Assert.Equal(allocation.AllocationId, returned.AllocationId); + } + + [Theory] + [InlineData(false, false, 336)] + [InlineData(true, false, 3_536)] + [InlineData(true, true, 6_416)] + public unsafe void PictureBufferPacksAllPictureStateIntoTwoAllocatorOwners( + bool allowScreenContentTools, + bool allowIntraBlockCopy, + int expectedContextStorageLength) + { + const int Width = 16; + const int Height = 16; + + // Four CDEF presets, the preceding quantizer, and two payload bounds follow the context regions. + const int TileStateStorageLength = 7 * sizeof(int); + TestMemoryAllocator allocator = new(); + allocator.EnableNonThreadSafeLogging(); + Configuration configuration = Configuration.Default.Clone(); + configuration.MemoryAllocator = allocator; + ObuColorConfig colorConfig = new() + { + IsMonochrome = false, + SubSamplingX = true, + SubSamplingY = true, + BitDepth = Av1BitDepth.EightBit + }; + + ObuTileGroupHeader tiles = new() + { + TileColumnCount = 1, + TileRowCount = 1 + }; + + tiles.TileColumnStartModeInfo[1] = Width >> Av1Constants.ModeInfoSizeLog2; + tiles.TileRowStartModeInfo[1] = Height >> Av1Constants.ModeInfoSizeLog2; + ObuSequenceHeader sequenceHeader = new() + { + Use128x128Superblock = true, + ColorConfig = colorConfig + }; + + ObuFrameHeader frameHeader = new() + { + AllowScreenContentTools = allowScreenContentTools, + AllowIntraBlockCopy = allowIntraBlockCopy, + ModeInfoColumnCount = Width >> Av1Constants.ModeInfoSizeLog2, + ModeInfoRowCount = Height >> Av1Constants.ModeInfoSizeLog2, + TilesInfo = tiles + }; + + TestMemoryAllocator.AllocationRequest[] allocations; + using (Av1EncoderPictureBuffer buffer = new( + configuration, + sequenceHeader, + frameHeader, + Width, + Height, + disallow4x4AllFrames: true)) + { + allocations = allocator.AllocationLog.ToArray(); + Assert.Equal(2, allocations.Length); + Assert.Equal(typeof(byte), allocations[0].ElementType); + Assert.Equal(6_144, allocations[0].Length); + Assert.Equal(AllocationOptions.Clean, allocations[0].AllocationOptions); + Assert.Equal(typeof(byte), allocations[1].ElementType); + Assert.Equal(expectedContextStorageLength + TileStateStorageLength, allocations[1].Length); + Assert.Equal(AllocationOptions.Clean, allocations[1].AllocationOptions); + Assert.Empty(allocator.ReturnLog); + + Av1PictureControlSet picture = buffer.Picture; + Assert.Equal(16, picture.SegmentationNeighborMap.Length); + Assert.Equal(32, picture.PartitionContexts[0].Left.Length); + Assert.Equal(32, picture.PartitionContexts[0].Top.Length); + Assert.Equal(32, picture.LuminanceDcSignLevelCoefficientNeighbors[0].Left.Length); + Assert.Equal(32, picture.LuminanceDcSignLevelCoefficientNeighbors[0].Top.Length); + Assert.Equal(16, picture.CbDcSignLevelCoefficientNeighbors[0].Left.Length); + Assert.Equal(16, picture.CbDcSignLevelCoefficientNeighbors[0].Top.Length); + Assert.Equal(32, picture.TransformFunctionContexts[0].Left.Length); + Assert.Equal(32, picture.TransformFunctionContexts[0].Top.Length); + Assert.Equal(4, picture.CdefPreset.Length); + Assert.Equal(1, picture.Parent.PreviousQIndex.Length); + Assert.Equal(1, picture.TileDataOffsets.Length); + Assert.Equal(1, picture.TileDataLengths.Length); + + // Exact offsets prove that all four typed views occupy the trailing region of the same owner, + // without gaps, overlapping fields, or a separate allocation hidden behind a memory manager. + fixed (byte* state = picture.SegmentationNeighborMap.Span) + { + fixed (int* cdef = picture.CdefPreset.Span, + quantizer = picture.Parent.PreviousQIndex.Span, + offsets = picture.TileDataOffsets.Span, + lengths = picture.TileDataLengths.Span) + { + Assert.Equal((nuint)0, (nuint)cdef % (nuint)sizeof(int)); + Assert.Equal(expectedContextStorageLength, (byte*)cdef - state); + Assert.Equal(4, quantizer - cdef); + Assert.Equal(1, offsets - quantizer); + Assert.Equal(1, lengths - offsets); + Assert.Equal(allocations[1].Length, (byte*)(lengths + 1) - state); + } + } + + if (allowScreenContentTools) + { + Av1NeighborArrayUnit paletteContext = Assert.Single(picture.PaletteContexts); + Assert.Equal(32, paletteContext.Left.Length); + Assert.Equal(32, paletteContext.Top.Length); + Assert.Equal(Av1Constants.ModeInfoSizeLog2, paletteContext.GranularityNormalLog2); + Assert.Equal(0, paletteContext.Left[0].PaletteSizes[0]); + Assert.Equal(0, paletteContext.Top[^1].PaletteSizes[1]); + + // The typed palette region starts at its natural 16-bit alignment inside the shared byte owner. + fixed (Av1EncoderPaletteInfo* pointer = paletteContext.Left) + { + Assert.Equal((nuint)0, (nuint)pointer % (nuint)sizeof(ushort)); + } + + paletteContext.Left[0].PaletteSizes[0] = 3; + Assert.Equal(0, paletteContext.Top[0].PaletteSizes[0]); + } + else + { + Assert.Empty(picture.PaletteContexts); + } + + if (allowIntraBlockCopy) + { + Assert.Equal(256, picture.DisplacementVectors.Length); + Assert.Equal(4, sizeof(Av1EncoderDisplacementVector)); + Assert.Equal(9, picture.IntraBlockCopySearch.OriginWidth); + Assert.Equal(9, picture.IntraBlockCopySearch.OriginHeight); + + // The packed vector region starts at its natural 16-bit alignment inside the shared byte owner. + fixed (Av1EncoderDisplacementVector* pointer = picture.DisplacementVectors.Span) + { + Assert.Equal((nuint)0, (nuint)pointer % (nuint)sizeof(short)); + } + + Point position = new(2, 2); + Av1MotionVector displacement = new(-16376, 16376); + picture.MapModeInfoBlock(position, Av1BlockSize.Block8x8); + picture.SetDisplacementVector(position, displacement); + Assert.Equal(displacement, picture.GetDisplacementVector(new Point(3, 3))); + } + else + { + Assert.Equal(0, picture.DisplacementVectors.Length); + } + } + + Assert.Equal(2, allocator.ReturnLog.Count); + Assert.Equal( + allocations.Select(x => x.AllocationId).Order(), + allocator.ReturnLog.Select(x => x.AllocationId).Order()); + } + + [Fact] + public void InterPictureBufferExposesPackedMotionVectorStorage() + { + const int Width = 16; + const int Height = 16; + ObuColorConfig colorConfig = new() + { + IsMonochrome = false, + SubSamplingX = true, + SubSamplingY = true, + BitDepth = Av1BitDepth.EightBit + }; + + ObuTileGroupHeader tiles = new() + { + TileColumnCount = 1, + TileRowCount = 1 + }; + + tiles.TileColumnStartModeInfo[1] = Width >> Av1Constants.ModeInfoSizeLog2; + tiles.TileRowStartModeInfo[1] = Height >> Av1Constants.ModeInfoSizeLog2; + ObuSequenceHeader sequenceHeader = new() { ColorConfig = colorConfig }; + ObuFrameHeader frameHeader = new() + { + FrameType = ObuFrameType.InterFrame, + ModeInfoColumnCount = Width >> Av1Constants.ModeInfoSizeLog2, + ModeInfoRowCount = Height >> Av1Constants.ModeInfoSizeLog2, + TilesInfo = tiles + }; + + using Av1EncoderPictureBuffer buffer = new( + Configuration.Default, + sequenceHeader, + frameHeader, + Width, + Height, + disallow4x4AllFrames: true); + + Av1PictureControlSet picture = buffer.Picture; + Assert.Equal(256, picture.DisplacementVectors.Length); + Assert.Equal(0, picture.IntraBlockCopySearch.OriginWidth); + + Point position = new(2, 2); + Av1MotionVector vector = new(-32, 40); + picture.MapModeInfoBlock(position, Av1BlockSize.Block8x8); + picture.SetDisplacementVector(position, vector); + Assert.Equal(vector, picture.GetDisplacementVector(new Point(3, 3))); + } + + [Fact] + public void PictureBufferResetReusesStorageAndRestoresFrameState() + { + const int Width = 16; + const int Height = 16; + const int InitialQIndex = 37; + const int NextQIndex = 91; + TestMemoryAllocator allocator = new(); + allocator.EnableNonThreadSafeLogging(); + Configuration configuration = Configuration.Default.Clone(); + configuration.MemoryAllocator = allocator; + ObuColorConfig colorConfig = new() + { + IsMonochrome = false, + SubSamplingX = true, + SubSamplingY = true, + BitDepth = Av1BitDepth.EightBit + }; + + ObuTileGroupHeader initialTiles = new() + { + TileColumnCount = 1, + TileRowCount = 1 + }; + + initialTiles.TileColumnStartModeInfo[1] = Width >> Av1Constants.ModeInfoSizeLog2; + initialTiles.TileRowStartModeInfo[1] = Height >> Av1Constants.ModeInfoSizeLog2; + ObuSequenceHeader sequenceHeader = new() + { + Use128x128Superblock = true, + ColorConfig = colorConfig + }; + + ObuFrameHeader initialFrameHeader = new() + { + FrameType = ObuFrameType.KeyFrame, + ModeInfoColumnCount = Width >> Av1Constants.ModeInfoSizeLog2, + ModeInfoRowCount = Height >> Av1Constants.ModeInfoSizeLog2, + TilesInfo = initialTiles + }; + + initialFrameHeader.QuantizationParameters.BaseQIndex = InitialQIndex; + using Av1EncoderPictureBuffer buffer = new( + configuration, + sequenceHeader, + initialFrameHeader, + Width, + Height, + disallow4x4AllFrames: true, + allocateScreenContentState: true, + allocateMotionVectorState: true, + allocateIntraBlockCopySearch: true); + + Av1PictureControlSet picture = buffer.Picture; + int allocationCount = allocator.AllocationLog.Count; + picture.ModeInfoGrid.Span[0] = 7; + picture.ModeInfoAllocation.Span[0].Block.Mode = Av1PredictionMode.Paeth; + picture.SegmentationNeighborMap.Span[0] = 3; + picture.PartitionContexts[0].Left[0] = new Av1PartitionContext(5, 7); + picture.TransformFunctionContexts[0].Top[0] = 8; + picture.PaletteContexts[0].Left[0].PaletteSizes[0] = 2; + picture.DisplacementVectors.Span[0] = new Av1EncoderDisplacementVector { Row = -8, Column = 16 }; + picture.CdefPreset.Span[0] = 2; + picture.Parent.PreviousQIndex.Span[0] = InitialQIndex + 1; + picture.TileDataOffsets.Span[0] = 11; + picture.TileDataLengths.Span[0] = 13; + ObuTileGroupHeader nextTiles = new() + { + TileColumnCount = 1, + TileRowCount = 1 + }; + + nextTiles.TileColumnStartModeInfo[1] = Width >> Av1Constants.ModeInfoSizeLog2; + nextTiles.TileRowStartModeInfo[1] = Height >> Av1Constants.ModeInfoSizeLog2; + ObuFrameHeader nextFrameHeader = new() + { + FrameType = ObuFrameType.InterFrame, + ModeInfoColumnCount = Width >> Av1Constants.ModeInfoSizeLog2, + ModeInfoRowCount = Height >> Av1Constants.ModeInfoSizeLog2, + TilesInfo = nextTiles + }; + + nextFrameHeader.QuantizationParameters.BaseQIndex = NextQIndex; + buffer.Reset(nextFrameHeader); + + Assert.Equal(allocationCount, allocator.AllocationLog.Count); + Assert.Empty(allocator.ReturnLog); + Assert.Equal(0, picture.ModeInfoGrid.Span[0]); + Assert.Equal(Av1PredictionMode.DC, picture.ModeInfoAllocation.Span[0].Block.Mode); + Assert.Equal(0, picture.SegmentationNeighborMap.Span[0]); + Assert.Equal(default, picture.PartitionContexts[0].Left[0]); + Assert.Equal(Av1Constants.MaxTransformSize, picture.TransformFunctionContexts[0].Top[0]); + Assert.Equal(0, picture.PaletteContexts[0].Left[0].PaletteSizes[0]); + Assert.Equal(default, picture.DisplacementVectors.Span[0]); + Assert.Equal(-1, picture.CdefPreset.Span[0]); + Assert.Equal(NextQIndex, picture.Parent.PreviousQIndex.Span[0]); + Assert.Equal(0, picture.TileDataOffsets.Span[0]); + Assert.Equal(0, picture.TileDataLengths.Span[0]); + Assert.Same(nextFrameHeader, picture.Parent.FrameHeader); + Assert.Same(nextTiles, picture.Parent.Common.TilesInfo); + } + + [Theory] + [InlineData(false, 2, 3, 98)] + [InlineData(true, 2, 2, 17)] + public void PictureMappingUsesPackedAlignedStorage( + bool disallow4x4, + int column, + int row, + int expectedAllocationOffset) + { + using Av1EncoderModeInfoBuffer buffer = new(Configuration.Default, 65, 33, disallow4x4); + Av1PictureControlSet picture = CreatePicture(buffer); + Point position = new(column, row); + ref Av1MacroBlockModeInfo modeInfo = ref picture.GetMacroBlockModeInfo(position); + modeInfo.Block.Mode = Av1PredictionMode.Paeth; + + picture.MapModeInfoBlock(position, Av1BlockSize.Block8x8); + + Assert.Equal(Av1PredictionMode.Paeth, buffer.Allocation.Span[expectedAllocationOffset].Block.Mode); + int alignedRowCount = buffer.Grid.Length / buffer.ModeInfoStride; + for (int y = 0; y < alignedRowCount; y++) + { + for (int x = 0; x < buffer.ModeInfoStride; x++) + { + int gridOffset = (y * buffer.ModeInfoStride) + x; + bool isMapped = y >= row && y < row + 2 && x >= column && x < column + 2; + Assert.Equal(isMapped ? expectedAllocationOffset : 0, buffer.Grid.Span[gridOffset]); + + if (isMapped) + { + Assert.Equal(Av1PredictionMode.Paeth, picture.GetFromModeInfoGrid(new Point(x, y)).Block.Mode); + } + } + } + } + + private static Av1PictureControlSet CreatePicture(Av1EncoderModeInfoBuffer buffer) + { + ObuTileGroupHeader tiles = new() + { + TileColumnCount = 1, + TileRowCount = 1 + }; + + tiles.TileColumnStartModeInfo[1] = buffer.ModeInfoColumnCount; + tiles.TileRowStartModeInfo[1] = buffer.ModeInfoRowCount; + ObuSequenceHeader sequenceHeader = new(); + ObuFrameHeader frameHeader = new() + { + ModeInfoColumnCount = buffer.ModeInfoColumnCount, + ModeInfoRowCount = buffer.ModeInfoRowCount, + TilesInfo = tiles + }; + + return new Av1PictureControlSet + { + PartitionContexts = [], + LuminanceDcSignLevelCoefficientNeighbors = [], + CrDcSignLevelCoefficientNeighbors = [], + CbDcSignLevelCoefficientNeighbors = [], + TransformFunctionContexts = [], + Sequence = new Av1SequenceControlSet { SequenceHeader = sequenceHeader }, + Parent = new Av1PictureParentControlSet + { + Common = new Av1EncoderCommon + { + ModeInfoColumnCount = buffer.ModeInfoColumnCount, + ModeInfoRowCount = buffer.ModeInfoRowCount, + ModeInfoStride = buffer.ModeInfoStride, + FrameSize = new ObuFrameSize(), + TilesInfo = tiles + }, + FrameHeader = frameHeader, + PreviousQIndex = Memory.Empty + }, + SegmentationNeighborMap = Memory.Empty, + ModeInfoGrid = buffer.Grid, + ModeInfoAllocation = buffer.Allocation, + ModeInfoStride = buffer.ModeInfoStride, + Disallow4x4AllFrames = buffer.Disallow4x4AllFrames, + CdefPreset = Memory.Empty, + TileDataOffsets = Memory.Empty, + TileDataLengths = Memory.Empty + }; + } +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1EntropyTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1EntropyTests.cs new file mode 100644 index 0000000000..b9187859a9 --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1EntropyTests.cs @@ -0,0 +1,2435 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Buffers; +using System.Numerics; +using SixLabors.ImageSharp.Formats.Heif.Av1; +using SixLabors.ImageSharp.Formats.Heif.Av1.Entropy; +using SixLabors.ImageSharp.Formats.Heif.Av1.Motion; +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.ChromaFromLuma; +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; +using SixLabors.ImageSharp.Memory; +using SixLabors.ImageSharp.Tests.Memory; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1; + +[Trait("Format", "Avif")] +public class Av1EntropyTests +{ + private const int BaseQIndex = 23; + + // Short syntax round trips encode only their small in-method symbol vectors. + private const int ShortSyntaxBufferLength = 64; + + [Theory] + [InlineData(4, true)] + [InlineData(2, true)] + [InlineData(1, true)] + [InlineData(4, false)] + public void DeltaLoopFilterChannelsAdaptIndependently(int channelCount, bool updateCdf) + { + ReadOnlySpan deltas = [0, 1, -2, 3, -1, 0, 17, -3, 2, -9, 0, 1]; + Av1Distribution[] channels = new Av1Distribution[channelCount]; + for (int channel = 0; channel < channelCount; channel++) + { + // libaom entropymode.c gives each multi-delta channel this independent initial CDF. + channels[channel] = new(28160, 32120, 32677); + } + + using Av1SymbolWriter writer = new(Configuration.Default, 512, updateCdf); + for (int index = 0; index < 96; index++) + { + int delta = deltas[index % deltas.Length]; + int magnitude = Math.Abs(delta); + writer.WriteSymbol(Math.Min(magnitude, 3), channels[index % channelCount]); + if (magnitude >= 3) + { + // The escape magnitude is 2^bits + 1 plus the transmitted remainder, followed by its sign. + int bits = BitOperations.Log2((uint)(magnitude - 1)); + writer.WriteLiteral((uint)(bits - 1), 3); + writer.WriteLiteral((uint)(magnitude - (1 << bits) - 1), bits); + } + + if (magnitude != 0) + { + writer.WriteLiteral(delta < 0 ? 1U : 0U, 1); + } + } + + using IMemoryOwner payload = writer.Exit(); + Av1SymbolDecoder reader = new(Configuration.Default, payload.Memory.Span, BaseQIndex, updateCdf); + for (int index = 0; index < 96; index++) + { + Assert.Equal(deltas[index % deltas.Length], reader.ReadDeltaLoopFilter(channelCount > 1, index % channelCount)); + } + } + + [Fact] + public void FrameEntropyLifecyclePreservesIndependentDeltaLoopFilterChannels() + { + Av1FrameEntropyContext source = new(BaseQIndex); + Av1FrameEntropyContext copy = new(BaseQIndex); + Av1FrameEntropyContext snapshot = new(BaseQIndex); + Av1Distribution defaults = new(28160, 32120, 32677); + for (int channel = 0; channel < 4; channel++) + { + for (int observation = 0; observation < 20; observation++) + { + source.DeltaLoopFilterMultiAbsolute[channel].Update((channel % 3) + 1); + } + } + + copy.CopyFrom(source); + source.SnapshotTo(snapshot); + for (int channel = 0; channel < 4; channel++) + { + Av1Distribution original = source.DeltaLoopFilterMultiAbsolute[channel]; + Av1Distribution copied = copy.DeltaLoopFilterMultiAbsolute[channel]; + Av1Distribution published = snapshot.DeltaLoopFilterMultiAbsolute[channel]; + Assert.NotSame(original, copied); + Assert.NotSame(original, published); + for (int symbol = 0; symbol < 4; symbol++) + { + Assert.Equal(original[symbol], copied[symbol]); + Assert.Equal(original[symbol], published[symbol]); + Assert.Equal(defaults[symbol], source.DeltaLoopFilterAbsolute[symbol]); + } + + // A published CDF preserves probabilities but restarts its observation history. The next + // identical symbol must therefore move its threshold further than in the twenty-count source. + original.Update(0); + published.Update(0); + Assert.NotEqual(original[0], published[0]); + Assert.NotEqual(original[0], copied[0]); + } + + copy.ResetToDefaults(255); + Av1FrameEntropyContext fresh = new(BaseQIndex); + for (int channel = 0; channel < 4; channel++) + { + for (int symbol = 0; symbol < 4; symbol++) + { + Assert.Equal(defaults[symbol], copy.DeltaLoopFilterMultiAbsolute[channel][symbol]); + Assert.Equal(defaults[symbol], fresh.DeltaLoopFilterMultiAbsolute[channel][symbol]); + } + } + } + + [Fact] + public void ProbabilityCostTableMatchesDefinition() + { + for (int normalizedProbability = 128; normalizedProbability < 256; normalizedProbability++) + { + int expected = (int)Math.Round( + -Math.Log2(normalizedProbability / 256D) * (1 << Av1ProbabilityCost.CostShift), + MidpointRounding.AwayFromZero); + + int actual = Av1ProbabilityCost.GetProbabilityCost(normalizedProbability << 7); + + Assert.Equal(expected, actual); + } + } + + [Theory] + [InlineData(0, 7680)] + [InlineData(1, 7680)] + [InlineData(4, 6656)] + [InlineData(8192, 1024)] + [InlineData(10000, 878)] + [InlineData(16384, 512)] + [InlineData(24576, 212)] + [InlineData(32767, 3)] + [InlineData(32768, 3)] + public void ProbabilityCostMatchesCurrentLibaom(int probability, int expected) + => Assert.Equal(expected, Av1ProbabilityCost.GetProbabilityCost(probability)); + + [Fact] + public void SymbolCostUsesDistributionIntervals() + { + Av1Distribution distribution = new(8192, 24576); + + Assert.Equal(1024, Av1ProbabilityCost.GetSymbolCost(distribution, 0)); + Assert.Equal(512, Av1ProbabilityCost.GetSymbolCost(distribution, 1)); + Assert.Equal(1024, Av1ProbabilityCost.GetSymbolCost(distribution, 2)); + } + + [Theory] + [InlineData(0, 6656)] + [InlineData(1, 6656)] + [InlineData(2, 6656)] + [InlineData(3, 6656)] + [InlineData(4, 6656)] + [InlineData(8, 6144)] + public void SymbolCostUsesRangeCoderMinimumProbability(uint probability, int expected) + { + // A middle interval can collapse during adaptation. Libaom cost.c floors its mass at EC_MIN_PROB=4, + // giving 13 * 512 rate units at and below that floor, while mass 8 costs 12 * 512 units. + Av1Distribution distribution = new(16384, 16384 + probability); + Assert.Equal(expected, Av1ProbabilityCost.GetSymbolCost(distribution, 1)); + Assert.Equal(expected, Av1ProbabilityCost.GetSymbolCost((int)probability)); + } + + [Theory] + [InlineData(0, 0)] + [InlineData(1, 512)] + [InlineData(7, 3584)] + public void LiteralCostUsesProbabilityCostPrecision(int bitCount, int expected) + => Assert.Equal(expected, Av1ProbabilityCost.GetLiteralCost(bitCount)); + + [Fact] + public void ProbabilityCostDoesNotAllocate() + { + Av1Distribution distribution = new(8192, 24576); + _ = Av1ProbabilityCost.GetSymbolCost(distribution, 0); + long before = GC.GetAllocatedBytesForCurrentThread(); + + for (int i = 0; i < 1000; i++) + { + _ = Av1ProbabilityCost.GetSymbolCost(distribution, i % distribution.NumberOfSymbols); + } + + long after = GC.GetAllocatedBytesForCurrentThread(); + Assert.Equal(before, after); + } + + [Fact] + public void SymbolEncoderSyntaxCostsMatchCurrentDistributions() + { + const byte TopContext = 0; + const byte LeftContext = 0; + const int SkipContext = 0; + const Av1BlockSize BlockSize = Av1BlockSize.Block8x8; + const Av1TransformSize TransformSize = Av1TransformSize.Size8x8; + const Av1PredictionMode LumaMode = Av1PredictionMode.DC; + const Av1ChromaPredictionMode ChromaMode = Av1ChromaPredictionMode.DC; + const Av1FilterIntraMode FilterMode = Av1FilterIntraMode.DC; + + using Av1SymbolEncoder encoder = new(Configuration.Default, 64, BaseQIndex, updateCdf: false); + + Av1Distribution luma = Av1DefaultDistributions.KeyFrameYMode[TopContext][LeftContext]; + Assert.Equal( + Av1ProbabilityCost.GetSymbolCost(luma, (int)LumaMode), + encoder.GetLumaModeCost(LumaMode, TopContext, LeftContext)); + + Av1Distribution angle = Av1DefaultDistributions.AngleDelta[0]; + Assert.Equal( + Av1ProbabilityCost.GetSymbolCost(angle, Av1Constants.MaxAngleDelta), + encoder.GetAngleDeltaCost(Av1Constants.MaxAngleDelta, Av1PredictionMode.Vertical)); + + Av1Distribution chroma = Av1DefaultDistributions.UvMode[0][(int)LumaMode]; + Assert.Equal( + Av1ProbabilityCost.GetSymbolCost(chroma, (int)ChromaMode), + encoder.GetChromaModeCost(ChromaMode, false, LumaMode)); + + Av1Distribution filterEnable = Av1DefaultDistributions.FilterIntra[(int)BlockSize]; + Av1Distribution filterMode = Av1DefaultDistributions.FilterIntraMode; + int expectedFilterCost = Av1ProbabilityCost.GetSymbolCost(filterEnable, 1) + + Av1ProbabilityCost.GetSymbolCost(filterMode, (int)FilterMode); + + Assert.Equal(expectedFilterCost, encoder.GetFilterIntraModeCost(FilterMode, BlockSize)); + Assert.Equal( + Av1ProbabilityCost.GetSymbolCost(filterEnable, 0), + encoder.GetFilterIntraModeCost(Av1FilterIntraMode.AllFilterIntraModes, BlockSize)); + + Av1Distribution skip = Av1DefaultDistributions.Skip[SkipContext]; + Assert.Equal( + Av1ProbabilityCost.GetSymbolCost(skip, 1), + encoder.GetSkipCost(true, SkipContext)); + + Av1Distribution transformSize = Av1DefaultDistributions.TransformSize[0][SkipContext]; + Assert.Equal( + Av1ProbabilityCost.GetSymbolCost(transformSize, 0), + encoder.GetTransformSizeCost(BlockSize, TransformSize, SkipContext)); + + Assert.Equal( + Av1ProbabilityCost.GetSymbolCost(transformSize, 1), + encoder.GetTransformSizeCost(BlockSize, Av1TransformSize.Size4x4, SkipContext)); + + Av1Distribution transformPartition = Av1DefaultDistributions.TransformPartition[SkipContext]; + Assert.Equal( + Av1ProbabilityCost.GetSymbolCost(transformPartition, 0), + encoder.GetTransformPartitionCost(false, SkipContext)); + + Assert.Equal( + Av1ProbabilityCost.GetSymbolCost(transformPartition, 1), + encoder.GetTransformPartitionCost(true, SkipContext)); + + Av1Distribution transformSkip = Av1DefaultDistributions + .GetTransformBlockSkip(BaseQIndex)[(int)TransformSize][SkipContext]; + + Assert.Equal( + Av1ProbabilityCost.GetSymbolCost(transformSkip, 1), + encoder.GetTransformBlockSkipCost(true, TransformSize, SkipContext)); + } + + [Fact] + public void SymbolEncoderResetRestoresNormativeDistributionState() + { + const byte TopContext = 0; + const byte LeftContext = 0; + const Av1PredictionMode LumaMode = Av1PredictionMode.DC; + using Av1SymbolEncoder encoder = new(Configuration.Default, 64, BaseQIndex, updateCdf: true); + int initialCost = encoder.GetLumaModeCost(LumaMode, TopContext, LeftContext); + + for (int i = 0; i < 8; i++) + { + encoder.WriteLumaMode(LumaMode, TopContext, LeftContext); + } + + Assert.NotEqual(initialCost, encoder.GetLumaModeCost(LumaMode, TopContext, LeftContext)); + + encoder.Reset(); + + Assert.Equal(initialCost, encoder.GetLumaModeCost(LumaMode, TopContext, LeftContext)); + } + + [Fact] + public void BlockSkipDecisionUsesAdaptedRatesForEmptyTransforms() + { + const int QIndex = 73; + const int BlockSkipContext = 2; + const int TransformSkipContext = 0; + const Av1TransformSize TransformSize = Av1TransformSize.Size8x8; + using Av1SymbolEncoder encoder = new(Configuration.Default, 256, QIndex, updateCdf: true); + int emptyTransformRate = encoder.GetTransformBlockSkipCost( + true, + TransformSize, + TransformSkipContext); + + Assert.True( + Av1TileWriter.ShouldSkipCoefficients( + encoder, + BlockSkipContext, + emptyTransformRate)); + + // Repeated non-skip symbols make another block skip more expensive while the empty-transform + // rate remains unchanged, proving the decision reads the adapted live distribution. + for (int index = 0; index < 256; index++) + { + encoder.WriteSkip(false, BlockSkipContext); + } + + Assert.False( + Av1TileWriter.ShouldSkipCoefficients( + encoder, + BlockSkipContext, + emptyTransformRate)); + } + + [Theory] + [InlineData(-16, 16)] + [InlineData(0, 8)] + [InlineData(-4, 0)] + public void ChromaFromLumaCostMatchesCurrentDistributions(int alphaU, int alphaV) + { + int signU = Av1ChromaFromLumaMath.AlphaToSign(alphaU); + int signV = Av1ChromaFromLumaMath.AlphaToSign(alphaV); + int jointSign = Av1ChromaFromLumaMath.JointSign(signU, signV); + int indexU = Av1ChromaFromLumaMath.AlphaToMagnitudeIndex(alphaU); + int indexV = Av1ChromaFromLumaMath.AlphaToMagnitudeIndex(alphaV); + int packedIndex = Av1ChromaFromLumaMath.PackIndices(indexU, indexV); + int expected = Av1ProbabilityCost.GetSymbolCost(Av1DefaultDistributions.ChromaFromLumaSign, jointSign); + if (signU != Av1ChromaFromLumaMath.SignZero) + { + expected += Av1ProbabilityCost.GetSymbolCost( + Av1DefaultDistributions.ChromaFromLumaAlpha[Av1ChromaFromLumaMath.ContextU(jointSign)], + indexU); + } + + if (signV != Av1ChromaFromLumaMath.SignZero) + { + expected += Av1ProbabilityCost.GetSymbolCost( + Av1DefaultDistributions.ChromaFromLumaAlpha[Av1ChromaFromLumaMath.ContextV(jointSign)], + indexV); + } + + using Av1SymbolEncoder encoder = new(Configuration.Default, 64, BaseQIndex, updateCdf: false); + Assert.Equal(expected, encoder.GetChromaFromLumaCost(packedIndex, jointSign)); + } + + /// + /// Verifies that live luma rate accounting includes the selected signed directional adjustment. + /// + /// The signed AV1 directional adjustment. + [Theory] + [InlineData(-3)] + [InlineData(0)] + [InlineData(3)] + public void LumaModeCostIncludesSelectedAngleDelta(int angleDelta) + { + const Av1PredictionMode Mode = Av1PredictionMode.Directional135Degrees; + using Av1SymbolEncoder encoder = new(Configuration.Default, 64, BaseQIndex, updateCdf: false); + Av1MacroBlockD macroBlock = new() + { + Tile = new Av1TileInfo(0, 0, new ObuFrameHeader()) + }; + + int expected = encoder.GetLumaModeCost(Mode, 0, 0) + + encoder.GetAngleDeltaCost(angleDelta + Av1Constants.MaxAngleDelta, Mode); + + Assert.Equal( + expected, + Av1TileWriter.GetLumaModeCost( + encoder, + macroBlock, + Av1BlockSize.Block8x8, + Mode, + angleDelta, + isIntraFrame: true)); + } + + /// + /// Verifies that live chroma rate accounting includes the selected signed directional adjustment. + /// + /// The signed AV1 directional adjustment. + [Theory] + [InlineData(-3)] + [InlineData(0)] + [InlineData(3)] + public void ChromaModeCostIncludesSelectedAngleDelta(int angleDelta) + { + const Av1PredictionMode LumaMode = Av1PredictionMode.DC; + const Av1ChromaPredictionMode ChromaMode = Av1ChromaPredictionMode.Directional135Degrees; + const Av1BlockSize BlockSize = Av1BlockSize.Block8x8; + ObuFrameHeader frameHeader = new(); + ObuColorConfig colorConfig = new() + { + IsMonochrome = false, + SubSamplingX = true, + SubSamplingY = true, + BitDepth = Av1BitDepth.EightBit + }; + + Av1MacroBlockModeInfo modeInfo = default; + using Av1SymbolEncoder encoder = new(Configuration.Default, 64, BaseQIndex, updateCdf: false); + bool isChromaFromLumaAllowed = BlockSize.AllowsChromaFromLuma( + frameHeader.LosslessArray[0], + colorConfig.SubSamplingX, + colorConfig.SubSamplingY); + + int expected = encoder.GetChromaModeCost(ChromaMode, isChromaFromLumaAllowed, LumaMode) + + encoder.GetAngleDeltaCost(angleDelta + Av1Constants.MaxAngleDelta, ChromaMode.ToLumaMode()); + + Assert.Equal( + expected, + Av1TileWriter.GetChromaModeCost( + encoder, + frameHeader, + colorConfig, + modeInfo, + BlockSize, + LumaMode, + ChromaMode, + angleDelta)); + } + + [Fact] + public void SymbolEncoderCostTracksWrittenLumaMode() + { + using Av1SymbolEncoder encoder = new(Configuration.Default, 64, BaseQIndex, updateCdf: true); + Av1Distribution expected = Av1DefaultDistributions.KeyFrameYMode[0][0]; + + Assert.Equal( + Av1ProbabilityCost.GetSymbolCost(expected, (int)Av1PredictionMode.DC), + encoder.GetLumaModeCost(Av1PredictionMode.DC, 0, 0)); + + encoder.WriteLumaMode(Av1PredictionMode.DC, 0, 0); + expected.Update((int)Av1PredictionMode.DC); + + Assert.Equal( + Av1ProbabilityCost.GetSymbolCost(expected, (int)Av1PredictionMode.DC), + encoder.GetLumaModeCost(Av1PredictionMode.DC, 0, 0)); + } + + [Fact] + public void CoefficientCostMatchesCurrentLibaomForEmptyAndDcBlocks() + { + const int qIndex = 0; + const Av1TransformSize transformSize = Av1TransformSize.Size4x4; + const Av1ComponentType componentType = Av1ComponentType.Luminance; + Av1TransformBlockContext transformBlockContext = default; + Span coefficients = stackalloc int[16]; + using Av1SymbolEncoder encoder = new(Configuration.Default, 64, qIndex, updateCdf: false); + Av1TransformSize transformSizeContext = Av1SymbolContextHelper.GetTransformSizeContext(transformSize); + Av1Distribution transformSkip = Av1DefaultDistributions + .GetTransformBlockSkip(qIndex)[(int)transformSizeContext][transformBlockContext.SkipContext]; + + int emptyCost = encoder.GetCoefficientCost( + transformSize, + Av1TransformType.DctDct, + Av1PredictionMode.DC, + coefficients, + componentType, + transformBlockContext, + 0, + false, + Av1FilterIntraMode.AllFilterIntraModes, + usesInterTransformSet: false); + + Assert.Equal(Av1ProbabilityCost.GetSymbolCost(transformSkip, 1), emptyCost); + + // Prime every active level with nonzero data so the one-coefficient path proves its EOB-only + // context derivation does not depend on clearing or rebuilding the forward-neighbor map. + coefficients.Fill(7); + _ = encoder.GetCoefficientCost( + transformSize, + Av1TransformType.DctDct, + Av1PredictionMode.DC, + coefficients, + componentType, + transformBlockContext, + 16, + false, + Av1FilterIntraMode.AllFilterIntraModes, + usesInterTransformSet: false); + + coefficients.Clear(); + coefficients[0] = 1; + Av1Distribution endOfBlock = Av1DefaultDistributions + .GetEndOfBlockFlag(qIndex)[transformSize.GetLog2Minus4()][(int)componentType][0]; + Av1Distribution coefficientBaseEnd = Av1DefaultDistributions + .GetBaseEndOfBlock(qIndex)[(int)transformSizeContext][(int)componentType][0]; + Av1Distribution dcSign = Av1DefaultDistributions + .GetDcSign(qIndex)[(int)componentType][transformBlockContext.DcSignContext]; + int expectedDcCost = + Av1ProbabilityCost.GetSymbolCost(transformSkip, 0) + + Av1ProbabilityCost.GetSymbolCost(endOfBlock, 0) + + Av1ProbabilityCost.GetSymbolCost(coefficientBaseEnd, 0) + + Av1ProbabilityCost.GetSymbolCost(dcSign, 0); + + int dcCost = encoder.GetCoefficientCost( + transformSize, + Av1TransformType.DctDct, + Av1PredictionMode.DC, + coefficients, + componentType, + transformBlockContext, + 1, + false, + Av1FilterIntraMode.AllFilterIntraModes, + usesInterTransformSet: false); + + Assert.Equal(expectedDcCost, dcCost); + } + + [Fact] + public void CoefficientCostMatchesCurrentLibaomBaseRangeAndGolomb() + { + const int qIndex = 0; + const int level = 25; + const Av1TransformSize transformSize = Av1TransformSize.Size4x4; + const Av1ComponentType componentType = Av1ComponentType.Luminance; + Av1TransformBlockContext transformBlockContext = default; + Span coefficients = stackalloc int[16]; + coefficients[0] = -level; + using Av1SymbolEncoder encoder = new(Configuration.Default, 64, qIndex, updateCdf: false); + Av1TransformSize transformSizeContext = Av1SymbolContextHelper.GetTransformSizeContext(transformSize); + Av1Distribution transformSkip = Av1DefaultDistributions + .GetTransformBlockSkip(qIndex)[(int)transformSizeContext][transformBlockContext.SkipContext]; + Av1Distribution endOfBlock = Av1DefaultDistributions + .GetEndOfBlockFlag(qIndex)[transformSize.GetLog2Minus4()][(int)componentType][0]; + Av1Distribution coefficientBaseEnd = Av1DefaultDistributions + .GetBaseEndOfBlock(qIndex)[(int)transformSizeContext][(int)componentType][0]; + Av1Distribution coefficientBaseRange = Av1DefaultDistributions + .GetCoefficientsBaseRange(qIndex)[(int)transformSizeContext][(int)componentType][0]; + Av1Distribution dcSign = Av1DefaultDistributions + .GetDcSign(qIndex)[(int)componentType][transformBlockContext.DcSignContext]; + + // Level 25 consumes all four three-symbol base-range chunks, followed by the seven-bit code for Golomb value 10. + int expected = + Av1ProbabilityCost.GetSymbolCost(transformSkip, 0) + + Av1ProbabilityCost.GetSymbolCost(endOfBlock, 0) + + Av1ProbabilityCost.GetSymbolCost(coefficientBaseEnd, 2) + + (4 * Av1ProbabilityCost.GetSymbolCost(coefficientBaseRange, 3)) + + Av1ProbabilityCost.GetSymbolCost(dcSign, 1) + + Av1ProbabilityCost.GetLiteralCost(7); + + int actual = encoder.GetCoefficientCost( + transformSize, + Av1TransformType.DctDct, + Av1PredictionMode.DC, + coefficients, + componentType, + transformBlockContext, + 1, + false, + Av1FilterIntraMode.AllFilterIntraModes, + usesInterTransformSet: false); + + Assert.Equal(expected, actual); + } + + [Fact] + public void CoefficientCostMatchesCurrentLibaomCoefficientTraversal() + { + const ushort endOfBlock = 4; + const Av1TransformSize transformSize = Av1TransformSize.Size4x4; + const Av1TransformType transformType = Av1TransformType.DctDct; + const Av1ComponentType componentType = Av1ComponentType.Luminance; + const Av1PredictionMode intraDirection = Av1PredictionMode.DC; + const Av1FilterIntraMode filterIntraMode = Av1FilterIntraMode.AllFilterIntraModes; + Av1TransformBlockContext transformBlockContext = default; + ReadOnlySpan scan = Av1ScanOrderConstants.GetScanOrder(transformSize, transformType).Scan; + Span coefficients = stackalloc int[16]; + coefficients[scan[0]] = -25; + coefficients[scan[2]] = 3; + coefficients[scan[3]] = -4; + using Av1LevelBuffer levels = new(Configuration.Default, new Size(4, 4)); + levels.Initialize(coefficients); + Span coefficientContexts = stackalloc sbyte[16]; + Av1TransformClass transformClass = transformType.ToClass(); + Av1SymbolContextHelper.GetNzMapContexts( + levels, + scan, + endOfBlock, + transformSize, + transformClass, + coefficientContexts); + + Av1TransformSize transformSizeContext = Av1SymbolContextHelper.GetTransformSizeContext(transformSize); + Av1Distribution transformSkip = Av1DefaultDistributions + .GetTransformBlockSkip(BaseQIndex)[(int)transformSizeContext][transformBlockContext.SkipContext]; + Av1Distribution endOfBlockFlag = Av1DefaultDistributions + .GetEndOfBlockFlag(BaseQIndex)[transformSize.GetLog2Minus4()][(int)componentType][0]; + Av1Distribution[][][] coefficientBase = Av1DefaultDistributions.GetCoefficientsBase(BaseQIndex); + Av1Distribution[][][] coefficientBaseEnd = Av1DefaultDistributions.GetBaseEndOfBlock(BaseQIndex); + Av1Distribution[][][] coefficientBaseRange = Av1DefaultDistributions.GetCoefficientsBaseRange(BaseQIndex); + Av1Distribution dcSign = Av1DefaultDistributions + .GetDcSign(BaseQIndex)[(int)componentType][transformBlockContext.DcSignContext]; + int expected = Av1ProbabilityCost.GetSymbolCost(transformSkip, 0); + + Av1TransformSetType transformSetType = Av1SymbolContextHelper.GetExtendedTransformSetType( + transformSize, + false); + + int extendedSet = Av1SymbolContextHelper.GetExtendedTransformSet(transformSetType); + int transformTypeIndex = Av1SymbolContextHelper.GetExtendedTransformIndex(transformSetType, transformType); + expected += Av1ProbabilityCost.GetSymbolCost( + Av1DefaultDistributions.IntraExtendedTransform[extendedSet][(int)transformSize.GetSquareSize()][(int)intraDirection], + transformTypeIndex); + + short endOfBlockPosition = Av1SymbolContextHelper.GetEndOfBlockPosition(endOfBlock, out int endOfBlockExtra); + expected += Av1ProbabilityCost.GetSymbolCost(endOfBlockFlag, endOfBlockPosition - 1); + int endOfBlockOffsetBitCount = Av1SymbolContextHelper.EndOfBlockOffsetBits[endOfBlockPosition]; + int endOfBlockBit = Av1Math.GetBit(endOfBlockExtra, endOfBlockOffsetBitCount - 1); + expected += Av1ProbabilityCost.GetSymbolCost( + Av1DefaultDistributions.GetEndOfBlockExtra(BaseQIndex)[(int)transformSizeContext][(int)componentType][endOfBlockPosition], + endOfBlockBit); + expected += Av1ProbabilityCost.GetLiteralCost(endOfBlockOffsetBitCount - 1); + + int eobPosition = scan[3]; + int eobContext = coefficientContexts[eobPosition]; + int eobBaseRangeContext = Av1SymbolContextHelper.GetBaseRangeContextEndOfBlock( + levels.GetPosition(eobPosition), + transformClass); + + expected += Av1ProbabilityCost.GetSymbolCost( + coefficientBaseEnd[(int)transformSizeContext][(int)componentType][eobContext], + 2); + expected += Av1ProbabilityCost.GetSymbolCost( + coefficientBaseRange[(int)transformSizeContext][(int)componentType][eobBaseRangeContext], + 1); + expected += Av1ProbabilityCost.GetLiteralCost(1); + + int acPosition = scan[2]; + int acContext = coefficientContexts[acPosition]; + int acBaseRangeContext = Av1SymbolContextHelper.GetBaseRangeContext( + levels, + levels.GetPosition(acPosition), + transformClass); + + expected += Av1ProbabilityCost.GetSymbolCost( + coefficientBase[(int)transformSizeContext][(int)componentType][acContext], + 3); + expected += Av1ProbabilityCost.GetSymbolCost( + coefficientBaseRange[(int)transformSizeContext][(int)componentType][acBaseRangeContext], + 0); + expected += Av1ProbabilityCost.GetLiteralCost(1); + + int zeroPosition = scan[1]; + expected += Av1ProbabilityCost.GetSymbolCost( + coefficientBase[(int)transformSizeContext][(int)componentType][coefficientContexts[zeroPosition]], + 0); + + int dcPosition = scan[0]; + int dcContext = coefficientContexts[dcPosition]; + int dcBaseRangeContext = Av1SymbolContextHelper.GetBaseRangeContext( + levels, + levels.GetPosition(dcPosition), + transformClass); + + expected += Av1ProbabilityCost.GetSymbolCost( + coefficientBase[(int)transformSizeContext][(int)componentType][dcContext], + 3); + expected += 4 * Av1ProbabilityCost.GetSymbolCost( + coefficientBaseRange[(int)transformSizeContext][(int)componentType][dcBaseRangeContext], + 3); + expected += Av1ProbabilityCost.GetLiteralCost(7); + expected += Av1ProbabilityCost.GetSymbolCost(dcSign, 1); + + using Av1SymbolEncoder encoder = new(Configuration.Default, 64, BaseQIndex, updateCdf: false); + int actual = encoder.GetCoefficientCost( + transformSize, + transformType, + intraDirection, + coefficients, + componentType, + transformBlockContext, + endOfBlock, + false, + filterIntraMode, + usesInterTransformSet: false); + + Assert.Equal(expected, actual); + } + + [Fact] + public void CoefficientCostDoesNotChangeWriterOrLiveDistributions() + { + const Av1TransformSize transformSize = Av1TransformSize.Size4x4; + const Av1TransformType transformType = Av1TransformType.DctDct; + const Av1ComponentType componentType = Av1ComponentType.Luminance; + const ushort endOfBlock = 4; + Av1TransformBlockContext transformBlockContext = default; + Span coefficients = stackalloc int[16]; + ReadOnlySpan scan = Av1ScanOrderConstants.GetScanOrder(transformSize, transformType).Scan; + coefficients[scan[0]] = -25; + coefficients[scan[2]] = 3; + coefficients[scan[3]] = 1; + using Av1SymbolEncoder actualEncoder = new(Configuration.Default, 64, BaseQIndex, updateCdf: true); + using Av1SymbolEncoder expectedEncoder = new(Configuration.Default, 64, BaseQIndex, updateCdf: true); + + int initialCost = actualEncoder.GetCoefficientCost( + transformSize, + transformType, + Av1PredictionMode.DC, + coefficients, + componentType, + transformBlockContext, + endOfBlock, + false, + Av1FilterIntraMode.AllFilterIntraModes, + usesInterTransformSet: false); + + Assert.Equal( + initialCost, + actualEncoder.GetCoefficientCost( + transformSize, + transformType, + Av1PredictionMode.DC, + coefficients, + componentType, + transformBlockContext, + endOfBlock, + false, + Av1FilterIntraMode.AllFilterIntraModes, + usesInterTransformSet: false)); + + int actualContext = actualEncoder.WriteCoefficients( + transformSize, + transformType, + Av1PredictionMode.DC, + coefficients, + componentType, + transformBlockContext, + endOfBlock, + false, + Av1FilterIntraMode.AllFilterIntraModes, + usesInterTransformSet: false); + + int adaptedCost = actualEncoder.GetCoefficientCost( + transformSize, + transformType, + Av1PredictionMode.DC, + coefficients, + componentType, + transformBlockContext, + endOfBlock, + false, + Av1FilterIntraMode.AllFilterIntraModes, + usesInterTransformSet: false); + + int expectedContext = expectedEncoder.WriteCoefficients( + transformSize, + transformType, + Av1PredictionMode.DC, + coefficients, + componentType, + transformBlockContext, + endOfBlock, + false, + Av1FilterIntraMode.AllFilterIntraModes, + usesInterTransformSet: false); + + using IMemoryOwner actual = actualEncoder.Exit(); + using IMemoryOwner expected = expectedEncoder.Exit(); + + Assert.NotEqual(initialCost, adaptedCost); + Assert.Equal(expectedContext, actualContext); + Assert.True(expected.GetSpan().SequenceEqual(actual.GetSpan())); + } + + [Fact] + public void CoefficientCostDoesNotAllocateAfterScratchInitialization() + { + const Av1TransformSize transformSize = Av1TransformSize.Size4x4; + const Av1TransformType transformType = Av1TransformType.DctDct; + const ushort endOfBlock = 4; + ReadOnlySpan scan = Av1ScanOrderConstants.GetScanOrder(transformSize, transformType).Scan; + Span coefficients = stackalloc int[16]; + coefficients[scan[0]] = -25; + coefficients[scan[2]] = 3; + coefficients[scan[3]] = 1; + using Av1SymbolEncoder encoder = new(Configuration.Default, 64, BaseQIndex, updateCdf: false); + + _ = encoder.GetCoefficientCost( + transformSize, + transformType, + Av1PredictionMode.DC, + coefficients, + Av1ComponentType.Luminance, + default, + endOfBlock, + false, + Av1FilterIntraMode.AllFilterIntraModes, + usesInterTransformSet: false); + + long before = GC.GetAllocatedBytesForCurrentThread(); + for (int i = 0; i < 1000; i++) + { + _ = encoder.GetCoefficientCost( + transformSize, + transformType, + Av1PredictionMode.DC, + coefficients, + Av1ComponentType.Luminance, + default, + endOfBlock, + false, + Av1FilterIntraMode.AllFilterIntraModes, + usesInterTransformSet: false); + } + + long after = GC.GetAllocatedBytesForCurrentThread(); + Assert.Equal(before, after); + } + + [Theory] + [InlineData(128, 2, 0L, 2, 0L, 1L)] + [InlineData(128, 1, 0L, 1, 0L, 1L)] + [InlineData(64, 512, 2_000_000_000L, 512, 2_000_000_000L, 512_000_000_128L)] + public void RateDistortionStatisticsRoundCombinedRate( + int multiplier, + int firstRate, + long firstDistortion, + int secondRate, + long secondDistortion, + long expectedCost) + { + // The first two cases cross opposite sides of the half-unit boundary: independently rounded + // child costs would be two and zero, while the combined reference cost is one in both cases. + Av1RateDistortionStatistics combined = new(multiplier, firstRate, firstDistortion); + Av1RateDistortionStatistics second = new(multiplier, secondRate, secondDistortion); + combined.Add(multiplier, in second); + + Assert.Equal(firstRate + secondRate, combined.Rate); + Assert.Equal(firstDistortion + secondDistortion, combined.Distortion); + Assert.Equal(expectedCost, combined.Cost); + } + + [Theory] + [InlineData(1, 255, 0L, 0L)] + [InlineData(1, 256, 0L, 1L)] + [InlineData(128, 512, 1000L, 128_128L)] + [InlineData(512, 512, 1000L, 128_512L)] + [InlineData(64, 1024, 4_000_000_000L, 512_000_000_128L)] + public void RateDistortionCostMatchesCurrentLibaom( + int rateMultiplier, + int rate, + long distortion, + long expected) + => Assert.Equal(expected, Av1RateDistortion.GetCost(rateMultiplier, rate, distortion)); + + /// + /// Verifies fixed curve samples, interpolation, error categories, native quantizer normalization, and skip selection. + /// + [Theory] + [InlineData(Av1BlockSize.Block4x4, 16L, 16, 8, Av1BitDepth.EightBit, 1, 13243, 17L)] + [InlineData(Av1BlockSize.Block8x8, 64L, 64, 8, Av1BitDepth.EightBit, 1, 45715, 66L)] + [InlineData(Av1BlockSize.Block16x16, 256L, 256, 8, Av1BitDepth.EightBit, 1, 154928, 265L)] + [InlineData(Av1BlockSize.Block32x32, 1024L, 1024, 8, Av1BitDepth.EightBit, 1, 410224, 1061L)] + [InlineData(Av1BlockSize.Block8x8, 96L, 64, 8, Av1BitDepth.EightBit, 1, 53253, 71L)] + [InlineData(Av1BlockSize.Block8x8, 1024L, 64, 8, Av1BitDepth.EightBit, 1, 96672, 95L)] + [InlineData(Av1BlockSize.Block8x8, 1056L, 64, 8, Av1BitDepth.EightBit, 1, 97253, 95L)] + [InlineData(Av1BlockSize.Block8x8, 64L, 64, 32, Av1BitDepth.TenBit, 1, 45715, 66L)] + [InlineData(Av1BlockSize.Block8x8, 64L, 64, 128, Av1BitDepth.TwelveBit, 1, 45715, 66L)] + [InlineData(Av1BlockSize.Block8x8, 64L, 64, 8, Av1BitDepth.EightBit, 1000000, 0, 1024L)] + [InlineData(Av1BlockSize.Block8x8, 0L, 64, 8, Av1BitDepth.EightBit, 1, 0, 0L)] + [InlineData(Av1BlockSize.Block128x128, 1L, 16384, 21387, Av1BitDepth.TwelveBit, 1, 0, 16L)] + public void PredictionErrorModelMatchesReferenceCurveSamples( + int blockSize, + long squaredError, + int sampleCount, + int acQuantizer, + int bitDepth, + int rateMultiplier, + int expectedRate, + long expectedDistortion) + { + // Expectations come from the published curve samples and cubic polynomial, not from an encode/decode + // round trip. Unit normalized error and unit quantizer hit rate column 31 in each block-size category. + Av1RateDistortion.ModelPredictionError( + (Av1BlockSize)blockSize, + squaredError, + sampleCount, + acQuantizer, + (Av1BitDepth)bitDepth, + rateMultiplier, + out int rate, + out long distortion); + + Assert.Equal(expectedRate, rate); + Assert.Equal(expectedDistortion, distortion); + } + + [Theory] + [InlineData(1, 8191, 100, 100)] + [InlineData(1, 8192, 100, 101)] + [InlineData(128, 4096, 100, 101)] + [InlineData(512, 3072, 100, 102)] + public void MotionSearchCostMatchesCurrentLibaom( + int rateMultiplier, + int motionVectorRate, + int variance, + int expected) + => Assert.Equal(expected, Av1RateDistortion.GetMotionSearchCost(rateMultiplier, motionVectorRate, variance)); + + [Theory] + [InlineData(0, 0, 2)] + [InlineData(255, 0, 21)] + [InlineData(255, 1, 21)] + [InlineData(255, 2, 21)] + public void MotionSearchSadPerBitMatchesCurrentLibaom(int qIndex, int bitDepth, int expected) + => Assert.Equal(expected, Av1RateDistortion.GetMotionSearchSadPerBit(qIndex, (Av1BitDepth)bitDepth)); + + [Theory] + [InlineData(2, 255, 100, 101)] + [InlineData(21, 512, 100, 121)] + [InlineData(21, 1000, 100, 141)] + public void MotionSearchSadCostMatchesCurrentLibaom( + int sadPerBit, + int motionVectorRate, + int sumOfAbsoluteDifferences, + int expected) + => Assert.Equal( + expected, + Av1RateDistortion.GetMotionSearchSadCost(sadPerBit, motionVectorRate, sumOfAbsoluteDifferences)); + + [Theory] + [InlineData(0, 0, 52)] + [InlineData(0, 1, 3)] + [InlineData(0, 2, 1)] + [InlineData(255, 0, 9_467_088)] + [InlineData(255, 1, 20_228_608)] + [InlineData(255, 2, 63_215_524)] + public void KeyFrameRateMultiplierMatchesCurrentLibaom( + int qIndex, + int bitDepth, + int expected) + => Assert.Equal(expected, Av1RateDistortion.GetKeyFrameRateMultiplier(qIndex, (Av1BitDepth)bitDepth)); + + [Theory] + [InlineData(0, 0, 51)] + [InlineData(0, 1, 3)] + [InlineData(0, 2, 1)] + [InlineData(255, 0, 9_288_598)] + [InlineData(255, 1, 20_049_918)] + [InlineData(255, 2, 63_036_850)] + public void InterFrameRateMultiplierMatchesCurrentLibaom( + int qIndex, + int bitDepth, + int expected) + => Assert.Equal(expected, Av1RateDistortion.GetInterFrameRateMultiplier(qIndex, (Av1BitDepth)bitDepth)); + + [Theory] + [InlineData(0)] + [InlineData(1)] + [InlineData(ShortSyntaxBufferLength)] + public void SymbolWriterMatchesCurrentLibaomCarryRegression(int initialCapacity) + { + using Av1SymbolWriter writer = new(Configuration.Default, initialCapacity, updateCdf: false); + writer.WriteBoolean(false, 16_384); + writer.WriteBoolean(false, 16_384); + writer.WriteBoolean(true, 512); + writer.WriteBoolean(false, 8_192); + using IMemoryOwner encoded = writer.Exit(); + + Assert.Equal(2, encoded.Memory.Length); + Assert.Equal(63, encoded.Memory.Span[0]); + } + + [Fact] + public void SymbolWriterRentsFixedOutputBuffer() + { + const int bufferLength = 257; + TestMemoryAllocator allocator = new(); + allocator.EnableNonThreadSafeLogging(); + Configuration configuration = Configuration.Default.Clone(); + configuration.MemoryAllocator = allocator; + TestMemoryAllocator.AllocationRequest allocation; + + using (Av1SymbolWriter writer = new(configuration, bufferLength, updateCdf: false)) + { + writer.WriteLiteral(false); + allocation = Assert.Single(allocator.AllocationLog); + + Assert.Equal(typeof(byte), allocation.ElementType); + Assert.Equal(bufferLength, allocation.Length); + } + + TestMemoryAllocator.ReturnRequest returned = Assert.Single(allocator.ReturnLog); + Assert.Equal(allocation.HashCodeOfBuffer, returned.HashCodeOfBuffer); + } + + [Theory] + [InlineData(0, false)] + [InlineData(1, true)] + [InlineData(7, false)] + [InlineData(8, true)] + [InlineData(17, true)] + public void SymbolWriterGrowthPreservesConsecutiveTiles(int initialCapacity, bool updateCdf) + { + TestMemoryAllocator allocator = new(); + allocator.EnableNonThreadSafeLogging(); + Configuration configuration = Configuration.Default.Clone(); + configuration.MemoryAllocator = allocator; + + using (Av1SymbolWriter writer = new(configuration, initialCapacity, updateCdf)) + using (Av1SymbolWriter expected = new(Configuration.Default, 8192, updateCdf)) + { + int outputLength = 0; + for (int tile = 0; tile < 3; tile++) + { + writer.Reset(outputLength); + expected.Reset(outputLength); + Av1Distribution distribution = new(100, 16000, 32000); + Av1Distribution expectedDistribution = new(100, 16000, 32000); + for (int index = 0; index < 257; index++) + { + // Small intervals provoke carries while literals cross repeated word-flush boundaries. + int symbol = (index + tile) & 3; + uint literal = (uint)((index * 73) + tile); + writer.WriteSymbol(symbol, distribution); + writer.WriteLiteral(literal, 8); + expected.WriteSymbol(symbol, expectedDistribution); + expected.WriteLiteral(literal, 8); + } + + ReadOnlyMemory actualTile = writer.Exit(out int length); + ReadOnlyMemory expectedTile = expected.Exit(out int expectedLength); + Assert.Equal(expectedLength, length); + Assert.True(expectedTile.Span.SequenceEqual(actualTile.Span)); + outputLength += length; + + // Growth in a later tile must preserve all earlier finalized tile bytes too. + Assert.True(expected.GetOutput(outputLength).Span.SequenceEqual(writer.GetOutput(outputLength).Span)); + } + + Assert.True(allocator.AllocationLog.Count > 1); + Assert.Equal(allocator.AllocationLog.Count - 1, allocator.ReturnLog.Count); + int allocations = allocator.AllocationLog.Count; + writer.Reset(); + writer.WriteLiteral(false); + _ = writer.Exit(out _); + Assert.Equal(allocations, allocator.AllocationLog.Count); + } + + Assert.Equal( + allocator.AllocationLog.Select(x => x.AllocationId).Order(), + allocator.ReturnLog.Select(x => x.AllocationId).Order()); + } + + [Theory] + [InlineData(false)] + [InlineData(true)] + public void SymbolWriterGrowthFailureRetainsItsOwner(bool failDuringFinalization) + { + int initialCapacity = failDuringFinalization ? 1 : 8; + OutputLimitedAllocator allocator = new(initialCapacity); + allocator.EnableNonThreadSafeLogging(); + Configuration configuration = Configuration.Default.Clone(); + configuration.MemoryAllocator = allocator; + + using (Av1SymbolWriter writer = new(configuration, initialCapacity, updateCdf: false)) + { + Assert.Throws(() => + { + if (failDuringFinalization) + { + writer.WriteBoolean(false, 16_384); + writer.WriteBoolean(false, 16_384); + writer.WriteBoolean(true, 512); + writer.WriteBoolean(false, 8_192); + _ = writer.Exit(out _); + } + else + { + for (int index = 0; index < 32; index++) + { + writer.WriteLiteral((uint)index, 8); + } + } + }); + + Assert.Single(allocator.AllocationLog); + Assert.Empty(allocator.ReturnLog); + } + + Assert.Equal( + Assert.Single(allocator.AllocationLog).AllocationId, + Assert.Single(allocator.ReturnLog).AllocationId); + } + + [Fact] + public void SymbolWriterResetReusesExistingOutputAllocation() + { + const int bufferLength = 257; + TestMemoryAllocator allocator = new(); + allocator.EnableNonThreadSafeLogging(); + Configuration configuration = Configuration.Default.Clone(); + configuration.MemoryAllocator = allocator; + TestMemoryAllocator.AllocationRequest allocation; + + using (Av1SymbolWriter writer = new(configuration, bufferLength, updateCdf: false)) + { + writer.WriteBoolean(false, 16_384); + writer.WriteBoolean(false, 16_384); + writer.WriteBoolean(true, 512); + writer.WriteBoolean(false, 8_192); + allocation = Assert.Single(allocator.AllocationLog); + ReadOnlyMemory encoded = writer.Exit(out int length); + + Assert.Equal(2, length); + Assert.Equal(length, encoded.Length); + Assert.Equal(63, encoded.Span[0]); + Assert.Single(allocator.AllocationLog); + Assert.Empty(allocator.ReturnLog); + + int firstLength = length; + writer.Reset(firstLength); + writer.WriteBoolean(false, 16_384); + writer.WriteBoolean(false, 16_384); + writer.WriteBoolean(true, 512); + writer.WriteBoolean(false, 8_192); + encoded = writer.Exit(out length); + + Assert.Equal(2, length); + Assert.Equal(length, encoded.Length); + Assert.Equal(63, encoded.Span[0]); + ReadOnlyMemory output = writer.GetOutput(firstLength + length); + Assert.True(output.Span[..firstLength].SequenceEqual(output.Span[firstLength..])); + Assert.Single(allocator.AllocationLog); + Assert.Empty(allocator.ReturnLog); + + writer.Reset(); + writer.WriteBoolean(false, 16_384); + writer.WriteBoolean(false, 16_384); + writer.WriteBoolean(true, 512); + writer.WriteBoolean(false, 8_192); + encoded = writer.Exit(out length); + + Assert.Equal(2, length); + Assert.Equal(length, encoded.Length); + Assert.Equal(63, encoded.Span[0]); + Assert.Single(allocator.AllocationLog); + Assert.Empty(allocator.ReturnLog); + } + + TestMemoryAllocator.ReturnRequest returned = Assert.Single(allocator.ReturnLog); + Assert.Equal(allocation.AllocationId, returned.AllocationId); + } + + /// + /// Verifies bounded coefficient scratch is rented at construction and reused by costing, coding, and frame resets. + /// + [Fact] + public void SymbolEncoderReusesConstructorOwnedCoefficientScratchAcrossFrames() + { + TestMemoryAllocator allocator = new(); + allocator.EnableNonThreadSafeLogging(); + Configuration configuration = Configuration.Default.Clone(); + configuration.MemoryAllocator = allocator; + Span coefficients = stackalloc int[16]; + + using (Av1SymbolEncoder encoder = new(configuration, 64, BaseQIndex, updateCdf: true)) + { + Assert.Equal(3, allocator.AllocationLog.Count); + TestMemoryAllocator.AllocationRequest levelScratch = allocator.AllocationLog[0]; + TestMemoryAllocator.AllocationRequest contextScratch = allocator.AllocationLog[1]; + TestMemoryAllocator.AllocationRequest outputScratch = allocator.AllocationLog[2]; + int maximumTransformDimension = Av1Constants.MaxTransformSize / 2; + int expectedLevelLength = + (Av1Constants.TransformPadHorizontal + maximumTransformDimension) * + (Av1Constants.TransformPadTop + maximumTransformDimension + Av1Constants.TransformPadBottom); + + Assert.Equal(typeof(byte), levelScratch.ElementType); + Assert.Equal(expectedLevelLength, levelScratch.Length); + Assert.Equal(AllocationOptions.Clean, levelScratch.AllocationOptions); + Assert.Equal(typeof(sbyte), contextScratch.ElementType); + Assert.Equal(maximumTransformDimension * maximumTransformDimension, contextScratch.Length); + Assert.Equal(typeof(byte), outputScratch.ElementType); + Assert.Equal(64, outputScratch.Length); + + // Exercise both an empty and a coded transform on each side of a frame reset. The second pass must + // reuse every constructor-owned buffer even after nonzero levels and probability updates exist. + for (int frame = 0; frame < 2; frame++) + { + coefficients.Clear(); + int emptyContext = encoder.WriteCoefficients( + Av1TransformSize.Size4x4, + Av1TransformType.DctDct, + Av1PredictionMode.DC, + coefficients, + Av1ComponentType.Luminance, + default, + 0, + false, + Av1FilterIntraMode.DC, + usesInterTransformSet: false); + + Assert.Equal(0, emptyContext); + Assert.Equal(3, allocator.AllocationLog.Count); + + coefficients[0] = 1; + _ = encoder.GetCoefficientCost( + Av1TransformSize.Size4x4, + Av1TransformType.DctDct, + Av1PredictionMode.DC, + coefficients, + Av1ComponentType.Luminance, + default, + 1, + false, + Av1FilterIntraMode.DC, + usesInterTransformSet: false); + + encoder.WriteCoefficients( + Av1TransformSize.Size4x4, + Av1TransformType.DctDct, + Av1PredictionMode.DC, + coefficients, + Av1ComponentType.Luminance, + default, + 1, + false, + Av1FilterIntraMode.DC, + usesInterTransformSet: false); + + encoder.Exit(out _); + encoder.Reset(); + Assert.Equal(3, allocator.AllocationLog.Count); + Assert.Empty(allocator.ReturnLog); + } + } + + Assert.Equal(3, allocator.ReturnLog.Count); + Assert.Equal( + allocator.AllocationLog.Select(x => x.AllocationId).Order(), + allocator.ReturnLog.Select(x => x.AllocationId).Order()); + } + + [Fact] + public void ReadRandomLiteral() + { + // Assign + const int bitCount = 4; + Random rand = new(bitCount); + byte[] values = Enumerable.Range(0, 100).Select(x => (byte)rand.Next(1 << bitCount)).ToArray(); + Av1SymbolReader reader = new(values); + List actuals = []; + + // Act + for (int i = 0; i < values.Length; i++) + { + actuals.Add(reader.ReadLiteral(bitCount)); + } + + // Assert + Assert.True(values.Length > bitCount); + } + + [Theory] + [InlineData(0, 0, 128)] + [InlineData(1, 255, 128)] + public void RawBytesFromWriteLiteral1Bit(uint value, byte exp0, byte exp1) + { + byte[] expected = [exp0, exp1]; + AssertRawBytesWritten(1, value, expected); + } + + [Theory] + [InlineData(0, 0, 0, 128)] + [InlineData(1, 85, 118, 192)] + [InlineData(2, 170, 165, 128)] + [InlineData(3, 255, 255, 128)] + public void RawBytesFromWriteLiteral2Bits(uint value, byte exp0, byte exp1, byte exp2) + { + byte[] expected = [exp0, exp1, exp2]; + AssertRawBytesWritten(2, value, expected); + } + + [Theory] + [InlineData(0, 0, 0, 0, 128)] + [InlineData(1, 36, 198, 146, 128)] + [InlineData(2, 73, 81, 182, 192)] + [InlineData(3, 109, 192, 146, 64)] + [InlineData(4, 146, 66, 73, 128)] + [InlineData(5, 182, 214, 219, 128)] + [InlineData(6, 219, 107, 109, 128)] + [InlineData(7, 255, 255, 255, 128)] + public void RawBytesFromWriteLiteral3Bits(uint value, byte exp0, byte exp1, byte exp2, byte exp3) + { + byte[] expected = [exp0, exp1, exp2, exp3]; + AssertRawBytesWritten(3, value, expected); + } + + [Theory] + [InlineData(0, 0, 0, 0, 0, 128)] + [InlineData(1, 17, 68, 34, 34, 128)] + [InlineData(2, 34, 86, 68, 68, 128)] + [InlineData(3, 51, 104, 102, 102, 128)] + [InlineData(4, 68, 118, 34, 34, 64)] + [InlineData(5, 85, 118, 170, 170, 192)] + [InlineData(6, 102, 119, 51, 51, 64)] + [InlineData(7, 119, 119, 187, 187, 192)] + [InlineData(8, 136, 129, 17, 17, 128)] + [InlineData(9, 153, 147, 51, 51, 128)] + [InlineData(10, 170, 165, 85, 85, 128)] + [InlineData(11, 187, 183, 119, 119, 128)] + [InlineData(12, 204, 201, 153, 153, 128)] + [InlineData(13, 221, 219, 187, 187, 128)] + [InlineData(14, 238, 237, 221, 221, 128)] + [InlineData(15, 255, 255, 255, 255, 128)] + public void RawBytesFromWriteLiteral4Bits(uint value, byte exp0, byte exp1, byte exp2, byte exp3, byte exp4) + { + byte[] expected = [exp0, exp1, exp2, exp3, exp4]; + AssertRawBytesWritten(4, value, expected); + } + + private static void AssertRawBytesWritten(int bitCount, uint value, byte[] expected) + { + // Assign + const int writeCount = 8; + uint[] values = new uint[writeCount]; + Array.Fill(values, value); + Configuration configuration = Configuration.Default; + using Av1SymbolWriter writer = new(configuration, ShortSyntaxBufferLength, updateCdf: true); + + // Act + for (int i = 0; i < writeCount; i++) + { + writer.WriteLiteral(value, bitCount); + } + + using IMemoryOwner actual = writer.Exit(); + + // Assert + Assert.Equal(expected, actual.GetSpan().ToArray()); + } + + [Theory] + [InlineData(0, 0, 128)] + [InlineData(1, 255, 128)] + public void RawBytesReadLiteral1Bit(int value, byte exp0, byte exp1) + { + byte[] buffer = [exp0, exp1]; + AssertRawBytesRead(1, buffer, value); + } + + [Theory] + [InlineData(0, 0, 0, 128)] + [InlineData(1, 85, 118, 192)] + [InlineData(2, 170, 165, 128)] + [InlineData(3, 255, 255, 128)] + public void RawBytesReadLiteral2Bits(int value, byte exp0, byte exp1, byte exp2) + { + byte[] buffer = [exp0, exp1, exp2]; + AssertRawBytesRead(2, buffer, value); + } + + [Theory] + [InlineData(0, 0, 0, 0, 128)] + [InlineData(1, 36, 198, 146, 128)] + [InlineData(2, 73, 81, 182, 192)] + [InlineData(3, 109, 192, 146, 64)] + [InlineData(4, 146, 66, 73, 128)] + [InlineData(5, 182, 214, 219, 128)] + [InlineData(6, 219, 107, 109, 128)] + [InlineData(7, 255, 255, 255, 128)] + public void RawBytesReadLiteral3Bits(int value, byte exp0, byte exp1, byte exp2, byte exp3) + { + byte[] buffer = [exp0, exp1, exp2, exp3]; + AssertRawBytesRead(3, buffer, value); + } + + [Theory] + [InlineData(0, 0, 0, 0, 0, 128)] + [InlineData(1, 17, 68, 34, 34, 128)] + [InlineData(2, 34, 86, 68, 68, 128)] + [InlineData(3, 51, 104, 102, 102, 128)] + [InlineData(4, 68, 118, 34, 34, 64)] + [InlineData(5, 85, 118, 170, 170, 192)] + [InlineData(6, 102, 119, 51, 51, 64)] + [InlineData(7, 119, 119, 187, 187, 192)] + [InlineData(8, 136, 129, 17, 17, 128)] + [InlineData(9, 153, 147, 51, 51, 128)] + [InlineData(10, 170, 165, 85, 85, 128)] + [InlineData(11, 187, 183, 119, 119, 128)] + [InlineData(12, 204, 201, 153, 153, 128)] + [InlineData(13, 221, 219, 187, 187, 128)] + [InlineData(14, 238, 237, 221, 221, 128)] + [InlineData(15, 255, 255, 255, 255, 128)] + public void RawBytesReadLiteral4Bits(int value, byte exp0, byte exp1, byte exp2, byte exp3, byte exp4) + { + byte[] buffer = [exp0, exp1, exp2, exp3, exp4]; + AssertRawBytesRead(4, buffer, value); + } + + private static void AssertRawBytesRead(int bitCount, byte[] buffer, int expected) + { + // Assign + int[] values = new int[8]; + int[] expectedValues = new int[8]; + Array.Fill(expectedValues, expected); + Av1SymbolReader reader = new(buffer); + + // Act + for (int i = 0; i < 8; i++) + { + values[i] = reader.ReadLiteral(bitCount); + } + + // Assert + Assert.Equal(expectedValues, values); + } + + [Fact] + public void RoundTripUniformPaletteIndices() + { + Configuration configuration = Configuration.Default; + using Av1SymbolEncoder encoder = new(configuration, 64, BaseQIndex, updateCdf: true); + + for (int valueCount = 2; valueCount <= Av1Constants.PaletteMaxSize; valueCount++) + { + for (int value = 0; value < valueCount; value++) + { + encoder.WriteUniform(valueCount, value); + } + } + + using IMemoryOwner encoded = encoder.Exit(); + Av1SymbolDecoder decoder = new(configuration, encoded.GetSpan(), BaseQIndex); + for (int valueCount = 2; valueCount <= Av1Constants.PaletteMaxSize; valueCount++) + { + for (int value = 0; value < valueCount; value++) + { + Assert.Equal(value, decoder.ReadUniform(valueCount)); + } + } + } + + [Fact] + public void RoundTripPaletteSymbols() + { + Configuration configuration = Configuration.Default; + using Av1SymbolEncoder encoder = new(configuration, 256, BaseQIndex, updateCdf: true); + + for (int blockSizeContext = 0; blockSizeContext < 7; blockSizeContext++) + { + for (int neighborContext = 0; neighborContext < 3; neighborContext++) + { + encoder.WritePaletteYMode( + ((blockSizeContext + neighborContext) & 1) != 0, + blockSizeContext, + neighborContext); + } + + int paletteSize = blockSizeContext + 2; + encoder.WritePaletteSize(paletteSize, blockSizeContext, Av1PlaneType.Y); + encoder.WritePaletteSize(Av1Constants.PaletteMaxSize - blockSizeContext, blockSizeContext, Av1PlaneType.Uv); + } + + encoder.WritePaletteUvMode(false, false); + encoder.WritePaletteUvMode(true, false); + encoder.WritePaletteUvMode(false, true); + encoder.WritePaletteUvMode(true, true); + for (int paletteSize = 2; paletteSize <= Av1Constants.PaletteMaxSize; paletteSize++) + { + for (int colorContext = 0; colorContext < 5; colorContext++) + { + int colorOrderIndex = (paletteSize + colorContext - 1) % paletteSize; + encoder.WritePaletteColorIndex(colorOrderIndex, paletteSize, colorContext, Av1PlaneType.Y); + encoder.WritePaletteColorIndex(colorOrderIndex, paletteSize, colorContext, Av1PlaneType.Uv); + } + } + + using IMemoryOwner encoded = encoder.Exit(); + Av1SymbolDecoder decoder = new(configuration, encoded.GetSpan(), BaseQIndex); + for (int blockSizeContext = 0; blockSizeContext < 7; blockSizeContext++) + { + for (int neighborContext = 0; neighborContext < 3; neighborContext++) + { + Assert.Equal( + ((blockSizeContext + neighborContext) & 1) != 0, + decoder.ReadPaletteYMode(blockSizeContext, neighborContext)); + } + + Assert.Equal(blockSizeContext + 2, decoder.ReadPaletteSize(blockSizeContext, Av1PlaneType.Y)); + Assert.Equal( + Av1Constants.PaletteMaxSize - blockSizeContext, + decoder.ReadPaletteSize(blockSizeContext, Av1PlaneType.Uv)); + } + + Assert.False(decoder.ReadPaletteUvMode(false)); + Assert.True(decoder.ReadPaletteUvMode(false)); + Assert.False(decoder.ReadPaletteUvMode(true)); + Assert.True(decoder.ReadPaletteUvMode(true)); + for (int paletteSize = 2; paletteSize <= Av1Constants.PaletteMaxSize; paletteSize++) + { + for (int colorContext = 0; colorContext < 5; colorContext++) + { + int expected = (paletteSize + colorContext - 1) % paletteSize; + Assert.Equal( + expected, + decoder.ReadPaletteColorIndex(paletteSize, colorContext, Av1PlaneType.Y)); + + Assert.Equal( + expected, + decoder.ReadPaletteColorIndex(paletteSize, colorContext, Av1PlaneType.Uv)); + } + } + } + + [Fact] + public void PaletteSyntaxCostsMatchCurrentDistributions() + { + using Av1SymbolEncoder encoder = new(Configuration.Default, 64, BaseQIndex, updateCdf: false); + Av1Distribution[][] yMode = Av1DefaultDistributions.PaletteYMode; + Av1Distribution[] uvMode = Av1DefaultDistributions.PaletteUvMode; + Av1Distribution[] ySize = Av1DefaultDistributions.PaletteYSize; + Av1Distribution[] uvSize = Av1DefaultDistributions.PaletteUvSize; + Av1Distribution[][] yColorIndex = Av1DefaultDistributions.PaletteYColorIndex; + Av1Distribution[][] uvColorIndex = Av1DefaultDistributions.PaletteUvColorIndex; + + Assert.Equal( + Av1ProbabilityCost.GetSymbolCost(yMode[4][2], 1), + encoder.GetPaletteYModeCost(true, 4, 2)); + + Assert.Equal( + Av1ProbabilityCost.GetSymbolCost(uvMode[1], 0), + encoder.GetPaletteUvModeCost(false, true)); + + Assert.Equal( + Av1ProbabilityCost.GetSymbolCost(ySize[3], 4), + encoder.GetPaletteSizeCost(6, 3, Av1PlaneType.Y)); + + Assert.Equal( + Av1ProbabilityCost.GetSymbolCost(uvSize[5], 1), + encoder.GetPaletteSizeCost(3, 5, Av1PlaneType.Uv)); + + Assert.Equal( + Av1ProbabilityCost.GetSymbolCost(yColorIndex[6][4], 7), + encoder.GetPaletteColorIndexCost(7, 8, 4, Av1PlaneType.Y)); + + Assert.Equal( + Av1ProbabilityCost.GetSymbolCost(uvColorIndex[3][2], 4), + encoder.GetPaletteColorIndexCost(4, 5, 2, Av1PlaneType.Uv)); + + Assert.Equal(Av1ProbabilityCost.GetLiteralCost(2), Av1SymbolEncoder.GetUniformCost(5, 2)); + Assert.Equal(Av1ProbabilityCost.GetLiteralCost(3), Av1SymbolEncoder.GetUniformCost(5, 3)); + } + + [Theory] + [InlineData(8)] + [InlineData(10)] + [InlineData(12)] + public void RoundTripPaletteColors(int bitDepth) + { + ushort[] colorCache = [4, 17, 23, 51]; + ushort[] yColors = [4, 23, 90]; + ushort[] uColors = [17, 51, 100]; + ushort[] deltaVColors = [1, 2, 1]; + ushort[] rawVColors = [0, (ushort)(1 << (bitDepth - 1)), 0]; + using Av1SymbolEncoder encoder = new(Configuration.Default, 128, BaseQIndex, updateCdf: true); + encoder.WritePaletteYColors(colorCache, yColors, bitDepth); + encoder.WritePaletteUvColors(colorCache, uColors, deltaVColors, bitDepth); + encoder.WritePaletteUvColors(colorCache, uColors, rawVColors, bitDepth); + + using IMemoryOwner encoded = encoder.Exit(); + Av1SymbolDecoder decoder = new(Configuration.Default, encoded.GetSpan(), BaseQIndex); + ushort[] decodedYColors = new ushort[yColors.Length]; + ushort[] decodedUColors = new ushort[uColors.Length]; + ushort[] decodedDeltaVColors = new ushort[deltaVColors.Length]; + ushort[] decodedRawVColors = new ushort[rawVColors.Length]; + decoder.ReadPaletteYColors(colorCache, yColors.Length, bitDepth, decodedYColors); + decoder.ReadPaletteUvColors( + colorCache, + uColors.Length, + bitDepth, + decodedUColors, + decodedDeltaVColors); + + Assert.Equal(yColors, decodedYColors); + Assert.Equal(uColors, decodedUColors); + Assert.Equal(deltaVColors, decodedDeltaVColors); + + decodedUColors.AsSpan().Clear(); + decoder.ReadPaletteUvColors( + colorCache, + uColors.Length, + bitDepth, + decodedUColors, + decodedRawVColors); + + Assert.Equal(uColors, decodedUColors); + Assert.Equal(rawVColors, decodedRawVColors); + } + + [Fact] + public void PaletteColorCostsMatchCurrentLibaomBitCounts() + { + ushort[] colorCache = [5, 10, 20]; + + Assert.Equal( + Av1ProbabilityCost.GetLiteralCost(20), + Av1SymbolEncoder.GetPaletteYColorCost([], [10, 20, 21], 8)); + + Assert.Equal( + Av1ProbabilityCost.GetLiteralCost(11), + Av1SymbolEncoder.GetPaletteYColorCost(colorCache, [5, 20, 30], 8)); + + Assert.Equal( + Av1ProbabilityCost.GetLiteralCost(32), + Av1SymbolEncoder.GetPaletteUvColorCost(colorCache, [5, 20, 30], [20, 21, 20], 8)); + + Assert.Equal( + Av1ProbabilityCost.GetLiteralCost(36), + Av1SymbolEncoder.GetPaletteUvColorCost(colorCache, [5, 20, 30], [0, 128, 0], 8)); + } + + [Fact] + public void PaletteColorCostDoesNotAllocate() + { + ushort[] colorCache = [5, 10, 20]; + ushort[] yColors = [5, 20, 30]; + ushort[] uColors = [5, 20, 30]; + ushort[] vColors = [20, 21, 20]; + _ = Av1SymbolEncoder.GetPaletteYColorCost(colorCache, yColors, 8); + long before = GC.GetAllocatedBytesForCurrentThread(); + + for (int i = 0; i < 1000; i++) + { + _ = Av1SymbolEncoder.GetPaletteYColorCost(colorCache, yColors, 8); + _ = Av1SymbolEncoder.GetPaletteUvColorCost(colorCache, uColors, vColors, 8); + } + + long after = GC.GetAllocatedBytesForCurrentThread(); + Assert.Equal(before, after); + } + + [Fact] + public void PaletteColorMapCostMatchesCurrentDistributions() + { + using Buffer2D map = Configuration.Default.MemoryAllocator.Allocate2D(2, 2, AllocationOptions.Clean); + map.DangerousGetRowSpan(0)[0] = 2; + map.DangerousGetRowSpan(0)[1] = 0; + map.DangerousGetRowSpan(1)[0] = 1; + map.DangerousGetRowSpan(1)[1] = 2; + Buffer2DRegion region = new(map); + Av1Distribution[][] distributions = Av1DefaultDistributions.PaletteYColorIndex; + int expected = Av1SymbolEncoder.GetUniformCost(3, 2); + expected += Av1ProbabilityCost.GetSymbolCost(distributions[1][0], 1); + expected += Av1ProbabilityCost.GetSymbolCost(distributions[1][0], 2); + expected += Av1ProbabilityCost.GetSymbolCost(distributions[1][1], 2); + using Av1SymbolEncoder encoder = new(Configuration.Default, 64, BaseQIndex, updateCdf: false); + + Assert.Equal( + expected, + encoder.GetPaletteColorMapCost(3, Av1PlaneType.Y, 2, 2, region)); + } + + [Fact] + public void PaletteCacheMergesSortedNeighborColorsWithoutDuplicates() + { + ReadOnlySpan above = [1, 3, 5, 7]; + ReadOnlySpan left = [2, 3, 6, 7]; + Span cache = stackalloc ushort[2 * Av1Constants.PaletteMaxSize]; + + int count = Av1PaletteCache.Merge(above, left, cache); + + Assert.Equal([1, 2, 3, 5, 6, 7], cache[..count].ToArray()); + } + + [Theory] + [InlineData(1, 1, 0, 0, 0, 1, 4, 1)] + [InlineData(1, 1, 0, 1, 0, 1, 3, 1)] + [InlineData(1, 1, 0, 0, 1, 1, 2, 1)] + [InlineData(1, 1, 0, 2, 1, 2, 1, 2)] + [InlineData(0, 1, 0, 2, 0, 0, 0, 1)] + public void PaletteColorMapContextMatchesCurrentLibaom( + int row, + int column, + byte left, + byte upperLeft, + byte above, + byte current, + int expectedContext, + int expectedOrderIndex) + { + using Buffer2D map = Configuration.Default.MemoryAllocator.Allocate2D(2, 2, AllocationOptions.Clean); + map.DangerousGetRowSpan(0)[0] = upperLeft; + map.DangerousGetRowSpan(0)[1] = above; + map.DangerousGetRowSpan(1)[0] = left; + map.DangerousGetRowSpan(row)[column] = current; + Span colorOrder = stackalloc byte[Av1Constants.PaletteMaxSize]; + + int actualContext = Av1PaletteColorMap.GetContext( + new Buffer2DRegion(map), + row, + column, + paletteSize: 4, + current, + colorOrder, + out int actualOrderIndex); + + Assert.Equal(expectedContext, actualContext); + Assert.Equal(expectedOrderIndex, actualOrderIndex); + } + + [Fact] + public void RoundTripPaletteColorMaps() + { + const int Rows = 5; + const int Columns = 7; + const int Width = 9; + const int Height = 6; + Configuration configuration = Configuration.Default; + using Buffer2D source = configuration.MemoryAllocator.Allocate2D(Width, Height); + using Buffer2D decoded = configuration.MemoryAllocator.Allocate2D(Width, Height); + Buffer2DRegion sourceRegion = new(source); + Buffer2DRegion decodedRegion = new(decoded); + using Av1SymbolEncoder encoder = new(configuration, 512, BaseQIndex, updateCdf: true); + for (int paletteSize = 2; paletteSize <= Av1Constants.PaletteMaxSize; paletteSize++) + { + for (int plane = 0; plane < 2; plane++) + { + for (int row = 0; row < Rows; row++) + { + Span sourceRow = source.DangerousGetRowSpan(row); + for (int column = 0; column < Columns; column++) + { + sourceRow[column] = (byte)(((row * 3) + (column * 5) + plane) % paletteSize); + } + } + + encoder.WritePaletteColorMap( + paletteSize, + (Av1PlaneType)plane, + Rows, + Columns, + sourceRegion); + } + } + + using IMemoryOwner encoded = encoder.Exit(); + Av1SymbolDecoder decoder = new(configuration, encoded.GetSpan(), BaseQIndex); + for (int paletteSize = 2; paletteSize <= Av1Constants.PaletteMaxSize; paletteSize++) + { + for (int plane = 0; plane < 2; plane++) + { + for (int row = 0; row < Height; row++) + { + decoded.DangerousGetRowSpan(row).Fill(byte.MaxValue); + } + + decoder.ReadPaletteColorMap( + paletteSize, + (Av1PlaneType)plane, + Rows, + Columns, + decodedRegion); + + for (int row = 0; row < Rows; row++) + { + ReadOnlySpan decodedRow = decoded.DangerousGetRowSpan(row); + for (int column = 0; column < Columns; column++) + { + Assert.Equal( + (byte)(((row * 3) + (column * 5) + plane) % paletteSize), + decodedRow[column]); + } + + for (int column = Columns; column < Width; column++) + { + Assert.Equal(byte.MaxValue, decodedRow[column]); + } + } + + for (int row = Rows; row < Height; row++) + { + ReadOnlySpan decodedRow = decoded.DangerousGetRowSpan(row); + for (int column = 0; column < Width; column++) + { + Assert.Equal(byte.MaxValue, decodedRow[column]); + } + } + } + } + + decoder.ValidateTrailingBits(); + } + + [Fact] + public void PaletteColorMapCostDoesNotAllocateAfterEntropyInitialization() + { + using Buffer2D map = Configuration.Default.MemoryAllocator.Allocate2D(7, 5, AllocationOptions.Clean); + for (int row = 0; row < map.Height; row++) + { + Span mapRow = map.DangerousGetRowSpan(row); + for (int column = 0; column < map.Width; column++) + { + mapRow[column] = (byte)(((row * 3) + (column * 5)) % 4); + } + } + + using Av1SymbolEncoder encoder = new(Configuration.Default, 64, BaseQIndex, updateCdf: false); + Buffer2DRegion region = new(map); + + // Run past tiered-compilation thresholds before measuring the established steady-state allocation contract. + for (int i = 0; i < 1000; i++) + { + _ = encoder.GetPaletteColorMapCost(4, Av1PlaneType.Y, map.Height, map.Width, region); + } + + long before = GC.GetAllocatedBytesForCurrentThread(); + + for (int i = 0; i < 1000; i++) + { + _ = encoder.GetPaletteColorMapCost(4, Av1PlaneType.Y, map.Height, map.Width, region); + } + + long after = GC.GetAllocatedBytesForCurrentThread(); + Assert.Equal(before, after); + } + + [Theory] + [MemberData(nameof(GetRangeData), 20)] + public void RoundTripPartitionType(int context) + { + // Assign + Configuration configuration = Configuration.Default; + using Av1SymbolEncoder encoder = new(configuration, ShortSyntaxBufferLength, BaseQIndex, updateCdf: true); + Av1PartitionType[] values = [ + Av1PartitionType.Split, Av1PartitionType.Split, Av1PartitionType.Split, Av1PartitionType.None, + Av1PartitionType.Split, Av1PartitionType.Split, Av1PartitionType.None, Av1PartitionType.None]; + Av1PartitionType[] actuals = new Av1PartitionType[values.Length]; + + // Act + foreach (Av1PartitionType value in values) + { + encoder.WritePartitionType(value, context); + } + + using IMemoryOwner encoded = encoder.Exit(); + + Av1SymbolDecoder decoder = new(Configuration.Default, encoded.GetSpan(), BaseQIndex); + for (int i = 0; i < values.Length; i++) + { + actuals[i] = decoder.ReadPartitionType(context); + } + + // Assert + Assert.Equal(values, actuals); + } + + [Theory] + [MemberData(nameof(GetSplitPartitionTypeData))] + public void RoundTripSplitOrHorizontalPartitionType(int size, int context) + { + // Assign + Av1BlockSize blockSize = (Av1BlockSize)size; + Configuration configuration = Configuration.Default; + using Av1SymbolEncoder encoder = new(configuration, ShortSyntaxBufferLength, BaseQIndex, updateCdf: true); + Av1PartitionType[] values = [ + Av1PartitionType.Split, Av1PartitionType.Split, Av1PartitionType.Split, Av1PartitionType.Horizontal, + Av1PartitionType.Split, Av1PartitionType.Split, Av1PartitionType.Horizontal, Av1PartitionType.Horizontal]; + Av1PartitionType[] actuals = new Av1PartitionType[values.Length]; + + // Act + foreach (Av1PartitionType value in values) + { + encoder.WriteSplitOrHorizontal(value, blockSize, context); + } + + using IMemoryOwner encoded = encoder.Exit(); + + Av1SymbolDecoder decoder = new(Configuration.Default, encoded.GetSpan(), BaseQIndex); + for (int i = 0; i < values.Length; i++) + { + actuals[i] = decoder.ReadSplitOrHorizontal(blockSize, context); + } + + // Assert + Assert.Equal(values, actuals); + } + + [Theory] + [MemberData(nameof(GetSplitPartitionTypeData))] + public void RoundTripSplitOrVerticalPartitionType(int size, int context) + { + // Assign + Av1BlockSize blockSize = (Av1BlockSize)size; + Configuration configuration = Configuration.Default; + using Av1SymbolEncoder encoder = new(configuration, ShortSyntaxBufferLength, BaseQIndex, updateCdf: true); + Av1PartitionType[] values = [ + Av1PartitionType.Split, Av1PartitionType.Split, Av1PartitionType.Split, Av1PartitionType.Vertical, + Av1PartitionType.Split, Av1PartitionType.Split, Av1PartitionType.Vertical, Av1PartitionType.Vertical]; + Av1PartitionType[] actuals = new Av1PartitionType[values.Length]; + + // Act + foreach (Av1PartitionType value in values) + { + encoder.WriteSplitOrVertical(value, blockSize, context); + } + + using IMemoryOwner encoded = encoder.Exit(); + + Av1SymbolDecoder decoder = new(Configuration.Default, encoded.GetSpan(), BaseQIndex); + for (int i = 0; i < values.Length; i++) + { + actuals[i] = decoder.ReadSplitOrVertical(blockSize, context); + } + + // Assert + Assert.Equal(values, actuals); + } + + [Theory] + [InlineData(0)] + [InlineData(1)] + [InlineData(2)] + public void RoundTripSkip(int context) + { + // Assign + Configuration configuration = Configuration.Default; + using Av1SymbolEncoder encoder = new(configuration, ShortSyntaxBufferLength, BaseQIndex, updateCdf: true); + bool[] values = [true, true, false, false, false, false, false, false, true]; + bool[] actuals = new bool[values.Length]; + + // Act + foreach (bool value in values) + { + encoder.WriteSkip(value, context); + } + + using IMemoryOwner encoded = encoder.Exit(); + + Av1SymbolDecoder decoder = new(Configuration.Default, encoded.GetSpan(), BaseQIndex); + for (int i = 0; i < values.Length; i++) + { + actuals[i] = decoder.ReadSkip(context); + } + + // Assert + Assert.Equal(values, actuals); + } + + [Theory] + [MemberData(nameof(GetTransformBlockSkipData))] + internal void RoundTripTransformBlockSkip(int transformContext, int skipContext) + { + // Assign + Av1TransformSize transformSizeContext = (Av1TransformSize)transformContext; + Configuration configuration = Configuration.Default; + using Av1SymbolEncoder encoder = new(configuration, ShortSyntaxBufferLength, BaseQIndex, updateCdf: true); + bool[] values = [true, true, false, false, false, false, false, false, true]; + bool[] actuals = new bool[values.Length]; + + // Act + foreach (bool value in values) + { + encoder.WriteTransformBlockSkip(value, transformSizeContext, skipContext); + } + + using IMemoryOwner encoded = encoder.Exit(); + + Av1SymbolDecoder decoder = new(Configuration.Default, encoded.GetSpan(), BaseQIndex); + for (int i = 0; i < values.Length; i++) + { + actuals[i] = decoder.ReadTransformBlockSkip(transformSizeContext, skipContext); + } + + // Assert + Assert.Equal(values, actuals); + } + + [Theory] + [MemberData(nameof(GetTransformTypeData))] + + // [InlineData(2, 0, 1)] + public void RoundTripTransformType(int txSizeContext, int intraMode, int intraDir) + { + // Assign + Av1TransformSize transformSizeContext = (Av1TransformSize)txSizeContext; + Av1FilterIntraMode filterIntraMode = (Av1FilterIntraMode)intraMode; + Av1PredictionMode intraDirection = (Av1PredictionMode)intraDir; + Configuration configuration = Configuration.Default; + using Av1SymbolEncoder encoder = new(configuration, ShortSyntaxBufferLength, BaseQIndex, updateCdf: true); + + // TODO: Include AdstFlipAdst, which is currently mapped to Identity. + Av1TransformType[] values = [ + Av1TransformType.DctDct, Av1TransformType.DctDct, Av1TransformType.Identity, Av1TransformType.AdstDct, + Av1TransformType.DctDct, Av1TransformType.AdstAdst, Av1TransformType.Identity, Av1TransformType.DctAdst + ]; + Av1TransformType[] actuals = new Av1TransformType[values.Length]; + + // Act + foreach (Av1TransformType value in values) + { + encoder.WriteTransformType(value, transformSizeContext, true, BaseQIndex, filterIntraMode, intraDirection, usesInterTransformSet: false); + } + + using IMemoryOwner encoded = encoder.Exit(); + + Av1SymbolDecoder decoder = new(Configuration.Default, encoded.GetSpan(), BaseQIndex); + for (int i = 0; i < values.Length; i++) + { + actuals[i] = decoder.ReadTransformType(transformSizeContext, true, false, false, false, filterIntraMode, intraDirection); + } + + // Assert + Assert.Equal(values, actuals); + } + + [Theory] + [MemberData(nameof(GetInterTransformTypeData))] + public void InterTransformTypeUsesExpectedCostAndRoundTrips(int txSize, bool useReducedTransformSet) + { + Av1TransformSize transformSize = (Av1TransformSize)txSize; + Av1TransformSetType transformSetType = Av1SymbolContextHelper.GetExtendedTransformSetType( + transformSize, + isInter: true, + useReducedTransformSet); + + int extendedSet = Av1SymbolContextHelper.GetExtendedTransformSet(transformSetType, isInter: true); + Av1TransformSize squareTransformSize = transformSize.GetSquareSize(); + Av1Distribution expectedDistribution = + Av1DefaultDistributions.InterExtendedTransform[extendedSet][(int)squareTransformSize]; + + Configuration configuration = Configuration.Default; + using Av1SymbolEncoder costEncoder = new(configuration, ShortSyntaxBufferLength, BaseQIndex, updateCdf: false); + using Av1SymbolEncoder encoder = new(configuration, ShortSyntaxBufferLength, BaseQIndex, updateCdf: true); + int transformTypeCount = Av1SymbolContextHelper.GetExtendedTransformTypeCount(transformSetType); + + for (int symbol = 0; symbol < transformTypeCount; symbol++) + { + Av1TransformType transformType = Av1SymbolContextHelper.GetExtendedTransformType(transformSetType, symbol); + int expectedCost = Av1ProbabilityCost.GetSymbolCost(expectedDistribution, symbol); + int actualCost = costEncoder.GetTransformTypeCost( + transformType, + transformSize, + useReducedTransformSet, + BaseQIndex, + Av1FilterIntraMode.AllFilterIntraModes, + Av1PredictionMode.DC, + usesInterTransformSet: true); + + Assert.Equal(expectedCost, actualCost); + encoder.WriteTransformType( + transformType, + transformSize, + useReducedTransformSet, + BaseQIndex, + Av1FilterIntraMode.AllFilterIntraModes, + Av1PredictionMode.DC, + usesInterTransformSet: true); + } + + using IMemoryOwner encoded = encoder.Exit(); + Av1SymbolDecoder decoder = new(configuration, encoded.GetSpan(), BaseQIndex); + for (int symbol = 0; symbol < transformTypeCount; symbol++) + { + Av1TransformType expected = Av1SymbolContextHelper.GetExtendedTransformType(transformSetType, symbol); + Av1TransformType actual = decoder.ReadTransformType( + transformSize, + useReducedTransformSet, + isInter: true, + useFilterIntra: false, + isLossless: false, + Av1FilterIntraMode.AllFilterIntraModes, + Av1PredictionMode.DC); + + Assert.Equal(expected, actual); + } + } + + [Theory] + [MemberData(nameof(GetEndOfBlockPositionData))] + public void RoundTripEndOfBlockPosition(int txSize, int txSizeContext, int plane, int txClass) + { + // Assign + Av1TransformSize transformSize = (Av1TransformSize)txSize; + Av1TransformSize transformSizeContext = (Av1TransformSize)txSizeContext; + Av1ComponentType componentType = (Av1ComponentType)plane; + Av1PlaneType planeType = (Av1PlaneType)plane; + Av1TransformClass transformClass = (Av1TransformClass)txClass; + Configuration configuration = Configuration.Default; + using Av1SymbolEncoder encoder = new(configuration, ShortSyntaxBufferLength, BaseQIndex, updateCdf: true); + + int[] values = [1, 2, 3, 4, 5]; + int[] actuals = new int[values.Length]; + + // Act + foreach (int value in values) + { + encoder.WriteEndOfBlockPosition((ushort)value, componentType, transformClass, transformSize, transformSizeContext); + } + + using IMemoryOwner encoded = encoder.Exit(); + + Av1SymbolDecoder decoder = new(Configuration.Default, encoded.GetSpan(), BaseQIndex); + for (int i = 0; i < values.Length; i++) + { + actuals[i] = decoder.ReadEndOfBlockPosition(transformSize, transformClass, transformSizeContext, planeType); + } + + // Assert + Assert.Equal(values, actuals); + } + + [Fact] + public void RoundTripGolomb() + { + // Assign + Configuration configuration = Configuration.Default; + int[] values = Enumerable.Range(0, 16384).ToArray(); + int[] actuals = new int[values.Length]; + + // Reserve the longest code for every value so this broad corpus cannot exhaust the fixed entropy output. + int maximumCodeBitCount = (BitOperations.Log2((uint)values.Length) * 2) + 1; + int bufferLength = (int)Numerics.DivideCeil((uint)(values.Length * maximumCodeBitCount), 8); + using Av1SymbolEncoder encoder = new(configuration, bufferLength, BaseQIndex, updateCdf: true); + + // Act + foreach (int value in values) + { + encoder.WriteGolomb(value); + } + + using IMemoryOwner encoded = encoder.Exit(); + + Av1SymbolDecoder decoder = new(Configuration.Default, encoded.GetSpan(), BaseQIndex); + for (int i = 0; i < values.Length; i++) + { + actuals[i] = decoder.ReadGolomb(); + } + + // Assert + Assert.Equal(values, actuals); + } + + [Theory] + [InlineData(0)] + [InlineData(1)] + [InlineData(2)] + public void RoundTripSegmentId(int context) + { + // Assign + int[] values = [3, 6, 7, 0, 2, 0, 2, 1, 1]; + Configuration configuration = Configuration.Default; + using Av1SymbolEncoder encoder = new(configuration, ShortSyntaxBufferLength, BaseQIndex, updateCdf: true); + int[] actuals = new int[values.Length]; + + // Act + foreach (int value in values) + { + encoder.WriteSegmentId(value, context); + } + + using IMemoryOwner encoded = encoder.Exit(); + + Av1SymbolDecoder decoder = new(Configuration.Default, encoded.GetSpan(), BaseQIndex); + for (int i = 0; i < values.Length; i++) + { + actuals[i] = decoder.ReadSegmentId(context); + } + + // Assert + Assert.Equal(values, actuals); + } + + [Fact] + public void RoundTripDeltaQuantizerIndex() + { + // Assign + int[] values = [3, 6, -7, -8, -2, 0, 2, 1, -1]; + Configuration configuration = Configuration.Default; + using Av1SymbolEncoder encoder = new(configuration, ShortSyntaxBufferLength, BaseQIndex, updateCdf: true); + int[] actuals = new int[values.Length]; + + // Act + foreach (int value in values) + { + encoder.WriteDeltaQuantizerIndex(value); + } + + using IMemoryOwner encoded = encoder.Exit(); + + Av1SymbolDecoder decoder = new(Configuration.Default, encoded.GetSpan(), BaseQIndex); + for (int i = 0; i < values.Length; i++) + { + actuals[i] = decoder.ReadDeltaQuantizerIndex(); + } + + // Assert + Assert.Equal(values, actuals); + } + + [Theory] + [MemberData(nameof(GetRangeData), (int)Av1BlockSize.AllSizes)] + public void RoundTripFilterIntraMode(int bSize) + { + // Assign + Av1BlockSize blockSize = (Av1BlockSize)bSize; + Av1FilterIntraMode[] values = [ + Av1FilterIntraMode.DC, Av1FilterIntraMode.Vertical, Av1FilterIntraMode.DC, Av1FilterIntraMode.Paeth, + Av1FilterIntraMode.AllFilterIntraModes, Av1FilterIntraMode.Directional157, Av1FilterIntraMode.DC, Av1FilterIntraMode.Directional157]; + Configuration configuration = Configuration.Default; + using Av1SymbolEncoder encoder = new(configuration, ShortSyntaxBufferLength, BaseQIndex, updateCdf: true); + Av1FilterIntraMode[] actuals = new Av1FilterIntraMode[values.Length]; + + // Act + foreach (Av1FilterIntraMode value in values) + { + encoder.WriteFilterIntraMode(value, blockSize); + } + + using IMemoryOwner encoded = encoder.Exit(); + + Av1SymbolDecoder decoder = new(Configuration.Default, encoded.GetSpan(), BaseQIndex); + for (int i = 0; i < values.Length; i++) + { + actuals[i] = decoder.ReadFilterUltraMode(blockSize); + } + + // Assert + Assert.Equal(values, actuals); + } + + [Fact] + public void RoundTripUseIntraBlockCopy() + { + // Assign + bool[] values = [true, true, false, true, false, false, false]; + Configuration configuration = Configuration.Default; + using Av1SymbolEncoder encoder = new(configuration, ShortSyntaxBufferLength, BaseQIndex, updateCdf: true); + bool[] actuals = new bool[values.Length]; + + Assert.Equal(51, encoder.GetUseIntraBlockCopyCost(false)); + Assert.Equal(1982, encoder.GetUseIntraBlockCopyCost(true)); + + // Act + foreach (bool value in values) + { + encoder.WriteUseIntraBlockCopy(value); + } + + using IMemoryOwner encoded = encoder.Exit(); + + Av1SymbolDecoder decoder = new(Configuration.Default, encoded.GetSpan(), BaseQIndex); + for (int i = 0; i < values.Length; i++) + { + actuals[i] = decoder.ReadUseIntraBlockCopy(); + } + + // Assert + Assert.Equal(values, actuals); + } + + /// + /// Verifies integer displacement-vector joints, signs, magnitude classes, and adaptive offset bits. + /// + [Fact] + public void RoundTripIntraBlockCopyDisplacementVectors() + { + Av1MotionVector[] references = + [ + new(-4096, 0), + new(0, -4096), + new(-1024, 0), + new(256, -256), + new(0, 0), + new(-2048, 2048), + ]; + + Av1MotionVector[] values = + [ + new(-4096, 0), + new(8, -4096), + new(-1040, 24), + new(256, -336), + new(512, 1024), + new(6144, -6144), + ]; + + int[] expectedCosts = [1440, 1661, 5231, 5807, 16955, 31656]; + Configuration configuration = Configuration.Default; + using Av1SymbolEncoder encoder = new(configuration, 64, BaseQIndex, updateCdf: true); + + // These current-libaom costs cover every joint, both signs, class zero, and large-class offset bits. + for (int i = 0; i < values.Length; i++) + { + Assert.Equal(expectedCosts[i], encoder.GetDisplacementVectorCost(values[i], references[i])); + } + + for (int i = 0; i < values.Length; i++) + { + encoder.WriteDisplacementVector(values[i], references[i]); + } + + using IMemoryOwner encoded = encoder.Exit(); + Av1SymbolDecoder decoder = new(configuration, encoded.GetSpan(), BaseQIndex); + Av1MotionVector[] actual = new Av1MotionVector[values.Length]; + + for (int i = 0; i < actual.Length; i++) + { + actual[i] = decoder.ReadDisplacementVector(references[i]); + } + + Assert.Equal(values, actual); + } + + public static TheoryData GetRangeData(int count) + { + TheoryData result = []; + for (int i = 0; i < count; i++) + { + result.Add(i); + } + + return result; + } + + public static TheoryData GetTransformBlockSkipData() + { + TheoryData result = []; + for (Av1TransformSize transformSizeContext = Av1TransformSize.Size4x4; transformSizeContext <= Av1TransformSize.Size64x64; transformSizeContext++) + { + for (int skipContext = 0; skipContext < 5; skipContext++) + { + result.Add((int)transformSizeContext, skipContext); + } + } + + return result; + } + + public static TheoryData GetSplitPartitionTypeData() + { + TheoryData result = []; + for (Av1BlockSize blockSize = Av1BlockSize.Block4x4; blockSize < Av1BlockSize.AllSizes; blockSize++) + { + for (int context = 4; context < 16; context++) + { + result.Add((int)blockSize, context); + } + } + + return result; + } + + public static TheoryData GetTransformTypeData() + { + TheoryData result = []; + for (Av1TransformSize transformSize = Av1TransformSize.Size4x4; transformSize < Av1TransformSize.AllSizes; transformSize++) + { + if (transformSize == Av1TransformSize.Size16x16) + { + for (Av1PredictionMode intraDirection = Av1PredictionMode.IntraModeStart; intraDirection < Av1PredictionMode.IntraModeEnd; intraDirection++) + { + result.Add((int)transformSize, (int)Av1FilterIntraMode.AllFilterIntraModes, (int)intraDirection); + } + + if (transformSize == Av1TransformSize.Size16x16) + { + result.Add((int)transformSize, 0, 0); + result.Add((int)transformSize, 1, 1); + result.Add((int)transformSize, 2, 2); + result.Add((int)transformSize, 3, 6); + result.Add((int)transformSize, 4, 0); + } + + continue; + } + + if (transformSize.GetSquareSize() >= Av1TransformSize.Size16x16 || transformSize is Av1TransformSize.Size32x8 or Av1TransformSize.Size8x32) + { + // DctOnly, doesn't make sense to test. + continue; + } + + for (Av1FilterIntraMode filterIntraMode = Av1FilterIntraMode.DC; filterIntraMode <= Av1FilterIntraMode.AllFilterIntraModes; filterIntraMode++) + { + for (Av1PredictionMode intraDirection = Av1PredictionMode.IntraModeStart; intraDirection < Av1PredictionMode.IntraModeEnd; intraDirection++) + { + result.Add((int)transformSize, (int)filterIntraMode, (int)intraDirection); + } + } + } + + return result; + } + + // Exercise the allocator's actual contiguous-buffer boundary while retaining the existing owner log. + private sealed class OutputLimitedAllocator : TestMemoryAllocator + { + public OutputLimitedAllocator(int limit) => this.SingleBufferAllocationLimitBytes = limit; + } + + public static TheoryData GetInterTransformTypeData() + { + TheoryData result = []; + for (Av1TransformSize transformSize = Av1TransformSize.Size4x4; + transformSize <= Av1TransformSize.Size32x32; + transformSize++) + { + result.Add((int)transformSize, false); + result.Add((int)transformSize, true); + } + + return result; + } + + public static TheoryData GetEndOfBlockPositionData() + { + TheoryData result = []; + for (Av1TransformSize transformSize = Av1TransformSize.Size4x4; transformSize < Av1TransformSize.AllSizes; transformSize++) + { + for (Av1TransformSize transformSizeContext = Av1TransformSize.Size4x4; transformSizeContext <= Av1TransformSize.Size64x64; transformSizeContext++) + { + for (int componentType = 0; componentType < 2; componentType++) + { + for (Av1TransformClass transformClass = Av1TransformClass.Class2D; transformClass <= Av1TransformClass.ClassVertical; transformClass++) + { + result.Add((int)transformSize, (int)transformSizeContext, componentType, (int)transformClass); + } + } + } + } + + return result; + } +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ForwardQuantizerTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ForwardQuantizerTests.cs new file mode 100644 index 0000000000..1fcce27ff2 --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ForwardQuantizerTests.cs @@ -0,0 +1,245 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1; +using SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.Quantizers; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; +using SixLabors.ImageSharp.Tests.TestUtilities; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1; + +/// +/// Verifies AV1 forward quantization against current libaom's fast no-matrix arithmetic. +/// +[Trait("Format", "Avif")] +public class Av1ForwardQuantizerTests +{ + /// + /// The hardware configurations covering every quantizer vector tier and the scalar fallback. + /// + private const HwIntrinsics QuantizerConfigurations = + HwIntrinsics.AllowAll | HwIntrinsics.DisableAVX512F | HwIntrinsics.DisableAVX | HwIntrinsics.DisableHWIntrinsic; + + /// + /// Verifies raster quantization and scan-order EOB selection at every SIMD tier. + /// + [Fact] + public void FastQuantizerMatchesLibaomReferenceAcrossHardwareWidths() + => FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidateQuantizer, QuantizerConfigurations); + + /// + /// Verifies that repeated transform quantization uses only caller-owned buffers. + /// + [Fact] + public void QuantizerDoesNotAllocatePerTransform() + { + const int coefficientCount = 64; + int[] coefficients = new int[coefficientCount]; + int[] quantized = new int[coefficientCount]; + int[] dequantized = new int[coefficientCount]; + FillCoefficients(coefficients, 73); + + Av1ForwardQuantizer.QuantizeLossy( + coefficients, + quantized, + dequantized, + Av1TransformSize.Size8x8, + Av1TransformType.DctDct, + 73, + -1, + 3, + Av1BitDepth.TenBit); + + long before = GC.GetAllocatedBytesForCurrentThread(); + for (int iteration = 0; iteration < 32; iteration++) + { + Av1ForwardQuantizer.QuantizeLossy( + coefficients, + quantized, + dequantized, + Av1TransformSize.Size8x8, + Av1TransformType.DctDct, + 73, + -1, + 3, + Av1BitDepth.TenBit); + } + + Assert.Equal(0, GC.GetAllocatedBytesForCurrentThread() - before); + } + + /// + /// Verifies that lossless quantization removes only the reversible transform scale. + /// + [Fact] + public void LosslessQuantizerRetainsExactReconstructionCoefficients() + { + int[] coefficients = + [ + 4, -8, 12, -16, + 20, -24, 28, -32, + 36, -40, 44, -48, + 52, -56, 60, -64 + ]; + + int[] quantized = new int[coefficients.Length]; + int[] dequantized = new int[coefficients.Length]; + + ushort endOfBlock = Av1ForwardQuantizer.QuantizeLossless( + coefficients, + quantized, + dequantized, + Av1BitDepth.TwelveBit); + + Assert.Equal((ushort)16, endOfBlock); + Assert.Equal(coefficients.Select(x => x / 4), quantized); + Assert.Equal(coefficients, dequantized); + } + + /// + /// Exercises each transform-scale category, coded 64-point layout, quantizer range, and sample precision. + /// + private static void ValidateQuantizer() + { + ReadOnlySpan transformSizes = + [ + Av1TransformSize.Size4x4, + Av1TransformSize.Size8x8, + Av1TransformSize.Size16x16, + Av1TransformSize.Size32x32, + Av1TransformSize.Size64x16, + Av1TransformSize.Size64x64, + ]; + + ReadOnlySpan quantizerIndices = [1, 73, 173, 255]; + ReadOnlySpan bitDepths = [Av1BitDepth.EightBit, Av1BitDepth.TenBit, Av1BitDepth.TwelveBit]; + + foreach (Av1TransformSize transformSize in transformSizes) + { + int coefficientCount = transformSize.GetAdjusted().GetSize2d(); + int[] coefficients = new int[coefficientCount]; + int[] expectedQuantized = new int[coefficientCount]; + int[] expectedDequantized = new int[coefficientCount]; + int[] actualQuantized = new int[coefficientCount]; + int[] actualDequantized = new int[coefficientCount]; + + foreach (int qIndex in quantizerIndices) + { + FillCoefficients(coefficients, qIndex); + + foreach (Av1BitDepth bitDepth in bitDepths) + { + ushort expectedEndOfBlock = QuantizeReference( + coefficients, + expectedQuantized, + expectedDequantized, + transformSize, + Av1TransformType.DctDct, + qIndex, + -1, + 3, + bitDepth); + + ushort actualEndOfBlock = Av1ForwardQuantizer.QuantizeLossy( + coefficients, + actualQuantized, + actualDequantized, + transformSize, + Av1TransformType.DctDct, + qIndex, + -1, + 3, + bitDepth); + + Assert.Equal(expectedEndOfBlock, actualEndOfBlock); + Assert.Equal(expectedQuantized, actualQuantized); + Assert.Equal(expectedDequantized, actualDequantized); + } + } + } + } + + /// + /// Fills one transform with deterministic signed values spanning threshold, rounding, and clamp behavior. + /// + private static void FillCoefficients(Span coefficients, int seed) + { + for (int i = 0; i < coefficients.Length; i++) + { + coefficients[i] = (((i * 7919) + (seed * 313)) % 90001) - 45000; + } + + coefficients[0] = 0; + coefficients[1] = 1; + coefficients[2] = -1; + coefficients[3] = short.MaxValue; + coefficients[4] = -short.MaxValue; + } + + /// + /// Mirrors av1_quantize_fp_no_qmatrix from current libaom without sharing the production traversal. + /// + private static ushort QuantizeReference( + ReadOnlySpan coefficients, + Span quantizedCoefficients, + Span dequantizedCoefficients, + Av1TransformSize transformSize, + Av1TransformType transformType, + int qIndex, + int dcDeltaQ, + int acDeltaQ, + Av1BitDepth bitDepth) + { + quantizedCoefficients.Clear(); + dequantizedCoefficients.Clear(); + + int logScale = transformSize.GetScale(); + int dcDequantizer = Av1QuantizationLookup.GetDcQuant(qIndex, dcDeltaQ, bitDepth); + int acDequantizer = Av1QuantizationLookup.GetAcQuant(qIndex, acDeltaQ, bitDepth); + int dcQuantizer = (1 << 16) / dcDequantizer; + int acQuantizer = (1 << 16) / acDequantizer; + int dcRounding = RoundPowerOfTwo((64 * dcDequantizer) >> 7, logScale); + int acRounding = RoundPowerOfTwo((64 * acDequantizer) >> 7, logScale); + ReadOnlySpan scan = Av1ScanOrderConstants.GetScanOrder(transformSize, transformType).Scan; + ushort endOfBlock = 0; + + for (int scanIndex = 0; scanIndex < scan.Length; scanIndex++) + { + int coefficientIndex = scan[scanIndex]; + int coefficient = coefficients[coefficientIndex]; + int coefficientSign = coefficient >> 31; + long magnitude = ((long)coefficient ^ coefficientSign) - coefficientSign; + int dequantizer = coefficientIndex == 0 ? dcDequantizer : acDequantizer; + int quantizer = coefficientIndex == 0 ? dcQuantizer : acQuantizer; + int rounding = coefficientIndex == 0 ? dcRounding : acRounding; + int quantizedMagnitude = 0; + + if ((magnitude << (1 + logScale)) >= dequantizer) + { + magnitude += rounding; + if (bitDepth == Av1BitDepth.EightBit) + { + magnitude = Math.Min(magnitude, short.MaxValue); + } + + quantizedMagnitude = (int)((magnitude * quantizer) >> (16 - logScale)); + } + + if (quantizedMagnitude != 0) + { + quantizedCoefficients[coefficientIndex] = (quantizedMagnitude ^ coefficientSign) - coefficientSign; + int dequantizedMagnitude = (quantizedMagnitude * dequantizer) >> logScale; + dequantizedCoefficients[coefficientIndex] = (dequantizedMagnitude ^ coefficientSign) - coefficientSign; + endOfBlock = (ushort)(scanIndex + 1); + } + } + + return endOfBlock; + } + + /// + /// Applies libaom's positive round-power-of-two operation. + /// + private static int RoundPowerOfTwo(int value, int shift) + => shift == 0 ? value : (value + (1 << (shift - 1))) >> shift; +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ForwardTransformTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ForwardTransformTests.cs new file mode 100644 index 0000000000..7cd8097fd7 --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ForwardTransformTests.cs @@ -0,0 +1,877 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.Intrinsics; +using System.Runtime.Intrinsics.X86; +using SixLabors.ImageSharp.Formats.Heif.Av1; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform.Forward; +using SixLabors.ImageSharp.Tests.TestUtilities; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1; + +/// +/// Verifies AV1 forward transform arithmetic, dispatch, layout, and allocation behavior. +/// +[Trait("Format", "Avif")] +public class Av1ForwardTransformTests +{ + /// + /// The hardware configurations covering every transform vector tier and the scalar fallback. + /// + private const HwIntrinsics TransformConfigurations = + HwIntrinsics.AllowAll | HwIntrinsics.DisableAVX512F | HwIntrinsics.DisableAVX | HwIntrinsics.DisableHWIntrinsic; + + /// + /// Gets every normative transform size, type, and bit-depth combination shared with the inverse suite. + /// + public static TheoryData ValidTransformCases { get; } = CreateValidTransformCases(); + + /// + /// Verifies every one-dimensional stage network across its scalar and available vector representations. + /// + [Fact] + public void OneDimensionalOperatorsMatchAcrossHardwareWidths() + => FeatureTestRunner.RunWithHwIntrinsicsFeature(AssertOneDimensionalOperators, TransformConfigurations); + + /// + /// Verifies every one-dimensional stage network against the independent analytical transform definition. + /// + [Fact] + public void OneDimensionalOperatorsMatchAnalyticalReference() + { + AssertOperatorAccuracy(Av1TransformType1d.Dct, 4); + AssertOperatorAccuracy(Av1TransformType1d.Dct, 8); + AssertOperatorAccuracy(Av1TransformType1d.Dct, 16); + AssertOperatorAccuracy(Av1TransformType1d.Dct, 32); + AssertOperatorAccuracy(Av1TransformType1d.Dct, 64); + AssertOperatorAccuracy(Av1TransformType1d.Adst, 4); + AssertOperatorAccuracy(Av1TransformType1d.Adst, 8); + AssertOperatorAccuracy(Av1TransformType1d.Adst, 16); + AssertOperatorAccuracy(Av1TransformType1d.Identity, 4); + AssertOperatorAccuracy(Av1TransformType1d.Identity, 8); + AssertOperatorAccuracy(Av1TransformType1d.Identity, 16); + AssertOperatorAccuracy(Av1TransformType1d.Identity, 32); + } + + /// + /// Verifies every permitted size, type, and bit-depth combination against the direct scalar two-axis definition. + /// + [Fact] + public void TwoDimensionalPipelineMatchesReference() + => FeatureTestRunner.RunWithHwIntrinsicsFeature(AssertTwoDimensionalPipeline, TransformConfigurations); + + /// + /// Verifies the reversible transform against the independent scalar operation order at every hardware tier. + /// + [Fact] + public void LosslessTransformMatchesLibaomReferenceAcrossHardwareWidths() + => FeatureTestRunner.RunWithHwIntrinsicsFeature(AssertLosslessTransform, TransformConfigurations); + + /// + /// Verifies that the complete transform dispatcher reuses caller-owned workspace. + /// + [Fact] + public void TransformDispatchDoesNotAllocatePerBlock() + { + const int width = 8; + short[] input = new short[width * width]; + int[] output = new int[input.Length]; + int[] workspace = new int[Av1TransformWorkspace.MaximumLength]; + + Av1ForwardTransformer.Transform2d(input, output, width, Av1TransformType.DctDct, Av1TransformSize.Size8x8, 8, workspace); + long before = GC.GetAllocatedBytesForCurrentThread(); + + for (int iteration = 0; iteration < 32; iteration++) + { + Av1ForwardTransformer.Transform2d(input, output, width, Av1TransformType.DctDct, Av1TransformSize.Size8x8, 8, workspace); + } + + Assert.Equal(0, GC.GetAllocatedBytesForCurrentThread() - before); + } + + private static void AssertLosslessTransform() + { + const int stride = 7; + short[] input = new short[stride * 4]; + for (int row = 0; row < 4; row++) + { + for (int column = 0; column < 4; column++) + { + input[(row * stride) + column] = (short)((row * 1003) - (column * 499) + (row * column * 71) - 1024); + } + } + + int[] expected = new int[16]; + int[] actual = new int[16]; + TransformLosslessReference(input, expected, stride); + Av1ForwardTransformer.TransformLossless4x4(input, actual, stride); + + Assert.Equal(expected, actual); + + long before = GC.GetAllocatedBytesForCurrentThread(); + for (int iteration = 0; iteration < 32; iteration++) + { + Av1ForwardTransformer.TransformLossless4x4(input, actual, stride); + } + + Assert.Equal(0, GC.GetAllocatedBytesForCurrentThread() - before); + } + + private static void TransformLosslessReference(ReadOnlySpan input, Span output, int stride) + { + // The first pass traverses columns and writes their four transformed values contiguously. The second + // pass consumes that transposed layout in place, matching the normative reversible operation order. + for (int column = 0; column < 4; column++) + { + int a = input[column]; + int b = input[stride + column]; + int c = input[(2 * stride) + column]; + int d = input[(3 * stride) + column]; + + a += b; + d -= c; + int e = (a - d) >> 1; + b = e - b; + c = e - c; + a -= c; + d += b; + + int offset = column * 4; + output[offset] = a; + output[offset + 1] = c; + output[offset + 2] = d; + output[offset + 3] = b; + } + + for (int column = 0; column < 4; column++) + { + int a = output[column]; + int b = output[4 + column]; + int c = output[8 + column]; + int d = output[12 + column]; + + a += b; + d -= c; + int e = (a - d) >> 1; + b = e - b; + c = e - c; + a -= c; + d += b; + + output[column] = a * 4; + output[4 + column] = c * 4; + output[8 + column] = d * 4; + output[12 + column] = b * 4; + } + + // Encoder coefficient storage is row-major, so exchange the reference walk's final frequency axes. + (output[1], output[4]) = (output[4], output[1]); + (output[2], output[8]) = (output[8], output[2]); + (output[3], output[12]) = (output[12], output[3]); + (output[6], output[9]) = (output[9], output[6]); + (output[7], output[13]) = (output[13], output[7]); + (output[11], output[14]) = (output[14], output[11]); + } + + /// + /// Exercises every DCT, ADST, and identity stage network using both Int16 and Int32 lane arithmetic. + /// + private static void AssertOneDimensionalOperators() + { + AssertOperator(4); + AssertOperator(8); + AssertOperator(16); + AssertOperator(32); + AssertOperator(64); + AssertOperator(4); + AssertOperator(8); + AssertOperator(16); + AssertOperator(4); + AssertOperator(8); + AssertOperator(16); + AssertOperator(32); + } + + /// + /// Compares one stage network across all available scalar and vector representations. + /// + /// The transform operator. + /// The transform length. + private static void AssertOperator(int length) + where TOperator : struct, Av1ForwardTransformer.IAv1ForwardTransform1dOperator + { + const int cosBit = 12; + + AssertInt32Vector128Operator(length, cosBit); + + if (Vector256.IsHardwareAccelerated) + { + AssertInt32Vector256Operator(length, cosBit); + } + + if (Vector512.IsHardwareAccelerated) + { + AssertInt32Vector512Operator(length, cosBit); + } + + AssertInt16Vector128Operator(length, cosBit); + + if (Avx2.IsSupported) + { + AssertInt16Vector256Operator(length, cosBit); + } + + if (Avx512BW.IsSupported) + { + AssertInt16Vector512Operator(length, cosBit); + } + } + + /// + /// Compares one integer stage network with the analytical reference transform. + /// + /// The transform operator. + /// The analytical transform definition. + /// The transform length. + private static void AssertOperatorAccuracy(Av1TransformType1d transformType, int length) + where TOperator : struct, Av1ForwardTransformer.IAv1ForwardTransform1dOperator + { + const int cosBit = 13; + const int testBlockCount = 500; + const int maximumCoefficientError = 7; + Random random = new(0); + double[] referenceInput = new double[length]; + double[] referenceOutput = new double[length]; + Av1TransformVector values = default; + Av1TransformVector buffer0 = default; + Av1TransformVector buffer1 = default; + + for (int block = 0; block < testBlockCount; block++) + { + for (int index = 0; index < length; index++) + { + int input = random.Next(1024) - random.Next(1024); + values[index] = input; + referenceInput[index] = input; + } + + ref byte valuesBase = ref System.Runtime.CompilerServices.Unsafe.As, byte>(ref values); + + TOperator.Transform(ref valuesBase, sizeof(int), sizeof(int), ref buffer0, ref buffer1, cosBit); + Av1ReferenceTransform.ReferenceTransform1d(transformType, referenceInput, referenceOutput, length); + + // the reference decoder permits seven integer coefficient units because each fixed-point butterfly rounds independently. + for (int index = 0; index < length; index++) + { + int expected = (int)Math.Round(referenceOutput[index], MidpointRounding.AwayFromZero); + int error = Math.Abs(values[index] - expected); + + Assert.True( + error <= maximumCoefficientError, + $"{typeof(TOperator).Name} coefficient {index}: expected {expected}, actual {values[index]}, error {error}."); + } + } + } + + /// + /// Compares the Vector128 Int32 representation with the scalar stage network lane by lane. + /// + /// The transform operator. + /// The transform length. + /// The fixed-point precision of the cosine constants. + private static void AssertInt32Vector128Operator(int length, int cosBit) + where TOperator : struct, Av1ForwardTransformer.IAv1ForwardTransform1dOperator + { + int laneCount = System.Runtime.CompilerServices.Unsafe.SizeOf>() / sizeof(int); + Av1TransformVector> vectorValues = default; + Av1TransformVector> vectorBuffer0 = default; + Av1TransformVector> vectorBuffer1 = default; + + for (int index = 0; index < length; index++) + { + ref int firstLane = ref System.Runtime.CompilerServices.Unsafe.As, int>(ref vectorValues[index]); + + for (int lane = 0; lane < laneCount; lane++) + { + System.Runtime.CompilerServices.Unsafe.Add(ref firstLane, lane) = GetInputValue(index, lane); + } + } + + ref byte vectorValuesBase = ref System.Runtime.CompilerServices.Unsafe.As>, byte>(ref vectorValues); + nint vectorStride = System.Runtime.CompilerServices.Unsafe.SizeOf>(); + + TOperator.Transform(ref vectorValuesBase, vectorStride, vectorStride, ref vectorBuffer0, ref vectorBuffer1, cosBit); + + for (int lane = 0; lane < laneCount; lane++) + { + Av1TransformVector scalarValues = default; + Av1TransformVector scalarBuffer0 = default; + Av1TransformVector scalarBuffer1 = default; + + for (int index = 0; index < length; index++) + { + scalarValues[index] = GetInputValue(index, lane); + } + + ref byte scalarValuesBase = ref System.Runtime.CompilerServices.Unsafe.As, byte>(ref scalarValues); + + TOperator.Transform(ref scalarValuesBase, sizeof(int), sizeof(int), ref scalarBuffer0, ref scalarBuffer1, cosBit); + + for (int index = 0; index < length; index++) + { + ref int firstLane = ref System.Runtime.CompilerServices.Unsafe.As, int>(ref vectorValues[index]); + Assert.Equal(scalarValues[index], System.Runtime.CompilerServices.Unsafe.Add(ref firstLane, lane)); + } + } + } + + /// + /// Compares the Vector256 Int32 representation with the scalar stage network lane by lane. + /// + /// The transform operator. + /// The transform length. + /// The fixed-point precision of the cosine constants. + private static void AssertInt32Vector256Operator(int length, int cosBit) + where TOperator : struct, Av1ForwardTransformer.IAv1ForwardTransform1dOperator + { + int laneCount = System.Runtime.CompilerServices.Unsafe.SizeOf>() / sizeof(int); + Av1TransformVector> vectorValues = default; + Av1TransformVector> vectorBuffer0 = default; + Av1TransformVector> vectorBuffer1 = default; + + for (int index = 0; index < length; index++) + { + ref int firstLane = ref System.Runtime.CompilerServices.Unsafe.As, int>(ref vectorValues[index]); + + for (int lane = 0; lane < laneCount; lane++) + { + System.Runtime.CompilerServices.Unsafe.Add(ref firstLane, lane) = GetInputValue(index, lane); + } + } + + ref byte vectorValuesBase = ref System.Runtime.CompilerServices.Unsafe.As>, byte>(ref vectorValues); + nint vectorStride = System.Runtime.CompilerServices.Unsafe.SizeOf>(); + + TOperator.Transform(ref vectorValuesBase, vectorStride, vectorStride, ref vectorBuffer0, ref vectorBuffer1, cosBit); + + for (int lane = 0; lane < laneCount; lane++) + { + Av1TransformVector scalarValues = default; + Av1TransformVector scalarBuffer0 = default; + Av1TransformVector scalarBuffer1 = default; + + for (int index = 0; index < length; index++) + { + scalarValues[index] = GetInputValue(index, lane); + } + + ref byte scalarValuesBase = ref System.Runtime.CompilerServices.Unsafe.As, byte>(ref scalarValues); + + TOperator.Transform(ref scalarValuesBase, sizeof(int), sizeof(int), ref scalarBuffer0, ref scalarBuffer1, cosBit); + + for (int index = 0; index < length; index++) + { + ref int firstLane = ref System.Runtime.CompilerServices.Unsafe.As, int>(ref vectorValues[index]); + Assert.Equal(scalarValues[index], System.Runtime.CompilerServices.Unsafe.Add(ref firstLane, lane)); + } + } + } + + /// + /// Compares the Vector512 Int32 representation with the scalar stage network lane by lane. + /// + /// The transform operator. + /// The transform length. + /// The fixed-point precision of the cosine constants. + private static void AssertInt32Vector512Operator(int length, int cosBit) + where TOperator : struct, Av1ForwardTransformer.IAv1ForwardTransform1dOperator + { + int laneCount = System.Runtime.CompilerServices.Unsafe.SizeOf>() / sizeof(int); + Av1TransformVector> vectorValues = default; + Av1TransformVector> vectorBuffer0 = default; + Av1TransformVector> vectorBuffer1 = default; + + for (int index = 0; index < length; index++) + { + ref int firstLane = ref System.Runtime.CompilerServices.Unsafe.As, int>(ref vectorValues[index]); + + for (int lane = 0; lane < laneCount; lane++) + { + System.Runtime.CompilerServices.Unsafe.Add(ref firstLane, lane) = GetInputValue(index, lane); + } + } + + ref byte vectorValuesBase = ref System.Runtime.CompilerServices.Unsafe.As>, byte>(ref vectorValues); + nint vectorStride = System.Runtime.CompilerServices.Unsafe.SizeOf>(); + + TOperator.Transform(ref vectorValuesBase, vectorStride, vectorStride, ref vectorBuffer0, ref vectorBuffer1, cosBit); + + for (int lane = 0; lane < laneCount; lane++) + { + Av1TransformVector scalarValues = default; + Av1TransformVector scalarBuffer0 = default; + Av1TransformVector scalarBuffer1 = default; + + for (int index = 0; index < length; index++) + { + scalarValues[index] = GetInputValue(index, lane); + } + + ref byte scalarValuesBase = ref System.Runtime.CompilerServices.Unsafe.As, byte>(ref scalarValues); + + TOperator.Transform(ref scalarValuesBase, sizeof(int), sizeof(int), ref scalarBuffer0, ref scalarBuffer1, cosBit); + + for (int index = 0; index < length; index++) + { + ref int firstLane = ref System.Runtime.CompilerServices.Unsafe.As, int>(ref vectorValues[index]); + Assert.Equal(scalarValues[index], System.Runtime.CompilerServices.Unsafe.Add(ref firstLane, lane)); + } + } + } + + /// + /// Compares the Vector128 Int16 representation with the scalar stage network lane by lane. + /// + /// The transform operator. + /// The transform length. + /// The fixed-point precision of the cosine constants. + private static void AssertInt16Vector128Operator(int length, int cosBit) + where TOperator : struct, Av1ForwardTransformer.IAv1ForwardTransform1dOperator + { + int laneCount = System.Runtime.CompilerServices.Unsafe.SizeOf>() / sizeof(short); + Av1TransformVector> vectorValues = default; + Av1TransformVector> vectorBuffer0 = default; + Av1TransformVector> vectorBuffer1 = default; + + for (int index = 0; index < length; index++) + { + ref short firstLane = ref System.Runtime.CompilerServices.Unsafe.As, short>(ref vectorValues[index]); + + for (int lane = 0; lane < laneCount; lane++) + { + System.Runtime.CompilerServices.Unsafe.Add(ref firstLane, lane) = GetPackedInputValue(index, lane); + } + } + + ref byte vectorValuesBase = ref System.Runtime.CompilerServices.Unsafe.As>, byte>(ref vectorValues); + nint vectorStride = System.Runtime.CompilerServices.Unsafe.SizeOf>(); + + TOperator.Transform(ref vectorValuesBase, vectorStride, vectorStride, ref vectorBuffer0, ref vectorBuffer1, cosBit); + + for (int lane = 0; lane < laneCount; lane++) + { + Av1TransformVector scalarValues = default; + Av1TransformVector scalarBuffer0 = default; + Av1TransformVector scalarBuffer1 = default; + + for (int index = 0; index < length; index++) + { + scalarValues[index] = GetPackedInputValue(index, lane); + } + + ref byte scalarValuesBase = ref System.Runtime.CompilerServices.Unsafe.As, byte>(ref scalarValues); + + TOperator.Transform(ref scalarValuesBase, sizeof(short), sizeof(short), ref scalarBuffer0, ref scalarBuffer1, cosBit); + + for (int index = 0; index < length; index++) + { + ref short firstLane = ref System.Runtime.CompilerServices.Unsafe.As, short>(ref vectorValues[index]); + Assert.Equal(scalarValues[index], System.Runtime.CompilerServices.Unsafe.Add(ref firstLane, lane)); + } + } + } + + /// + /// Compares the Vector256 Int16 representation with the scalar stage network lane by lane. + /// + /// The transform operator. + /// The transform length. + /// The fixed-point precision of the cosine constants. + private static void AssertInt16Vector256Operator(int length, int cosBit) + where TOperator : struct, Av1ForwardTransformer.IAv1ForwardTransform1dOperator + { + int laneCount = System.Runtime.CompilerServices.Unsafe.SizeOf>() / sizeof(short); + Av1TransformVector> vectorValues = default; + Av1TransformVector> vectorBuffer0 = default; + Av1TransformVector> vectorBuffer1 = default; + + for (int index = 0; index < length; index++) + { + ref short firstLane = ref System.Runtime.CompilerServices.Unsafe.As, short>(ref vectorValues[index]); + + for (int lane = 0; lane < laneCount; lane++) + { + System.Runtime.CompilerServices.Unsafe.Add(ref firstLane, lane) = GetPackedInputValue(index, lane); + } + } + + ref byte vectorValuesBase = ref System.Runtime.CompilerServices.Unsafe.As>, byte>(ref vectorValues); + nint vectorStride = System.Runtime.CompilerServices.Unsafe.SizeOf>(); + + TOperator.Transform(ref vectorValuesBase, vectorStride, vectorStride, ref vectorBuffer0, ref vectorBuffer1, cosBit); + + for (int lane = 0; lane < laneCount; lane++) + { + Av1TransformVector scalarValues = default; + Av1TransformVector scalarBuffer0 = default; + Av1TransformVector scalarBuffer1 = default; + + for (int index = 0; index < length; index++) + { + scalarValues[index] = GetPackedInputValue(index, lane); + } + + ref byte scalarValuesBase = ref System.Runtime.CompilerServices.Unsafe.As, byte>(ref scalarValues); + + TOperator.Transform(ref scalarValuesBase, sizeof(short), sizeof(short), ref scalarBuffer0, ref scalarBuffer1, cosBit); + + for (int index = 0; index < length; index++) + { + ref short firstLane = ref System.Runtime.CompilerServices.Unsafe.As, short>(ref vectorValues[index]); + Assert.Equal(scalarValues[index], System.Runtime.CompilerServices.Unsafe.Add(ref firstLane, lane)); + } + } + } + + /// + /// Compares the Vector512 Int16 representation with the scalar stage network lane by lane. + /// + /// The transform operator. + /// The transform length. + /// The fixed-point precision of the cosine constants. + private static void AssertInt16Vector512Operator(int length, int cosBit) + where TOperator : struct, Av1ForwardTransformer.IAv1ForwardTransform1dOperator + { + int laneCount = System.Runtime.CompilerServices.Unsafe.SizeOf>() / sizeof(short); + Av1TransformVector> vectorValues = default; + Av1TransformVector> vectorBuffer0 = default; + Av1TransformVector> vectorBuffer1 = default; + + for (int index = 0; index < length; index++) + { + ref short firstLane = ref System.Runtime.CompilerServices.Unsafe.As, short>(ref vectorValues[index]); + + for (int lane = 0; lane < laneCount; lane++) + { + System.Runtime.CompilerServices.Unsafe.Add(ref firstLane, lane) = GetPackedInputValue(index, lane); + } + } + + ref byte vectorValuesBase = ref System.Runtime.CompilerServices.Unsafe.As>, byte>(ref vectorValues); + nint vectorStride = System.Runtime.CompilerServices.Unsafe.SizeOf>(); + + TOperator.Transform(ref vectorValuesBase, vectorStride, vectorStride, ref vectorBuffer0, ref vectorBuffer1, cosBit); + + for (int lane = 0; lane < laneCount; lane++) + { + Av1TransformVector scalarValues = default; + Av1TransformVector scalarBuffer0 = default; + Av1TransformVector scalarBuffer1 = default; + + for (int index = 0; index < length; index++) + { + scalarValues[index] = GetPackedInputValue(index, lane); + } + + ref byte scalarValuesBase = ref System.Runtime.CompilerServices.Unsafe.As, byte>(ref scalarValues); + + TOperator.Transform(ref scalarValuesBase, sizeof(short), sizeof(short), ref scalarBuffer0, ref scalarBuffer1, cosBit); + + for (int index = 0; index < length; index++) + { + ref short firstLane = ref System.Runtime.CompilerServices.Unsafe.As, short>(ref vectorValues[index]); + Assert.Equal(scalarValues[index], System.Runtime.CompilerServices.Unsafe.Add(ref firstLane, lane)); + } + } + } + + /// + /// Exercises the complete normative transform matrix for the active hardware configuration. + /// + private static void AssertTwoDimensionalPipeline() + { + for (Av1TransformSize transformSize = 0; transformSize < Av1TransformSize.AllSizes; transformSize++) + { + for (Av1TransformType transformType = 0; transformType < Av1TransformType.AllTransformTypes; transformType++) + { + Av1Transform2dFlipConfiguration config = Av1Transform2dFlipConfiguration.CreateForward(transformType, transformSize, 8); + + if (!config.IsAllowed()) + { + continue; + } + + for (int bitDepth = 8; bitDepth <= 12; bitDepth += 2) + { + AssertTwoDimensionalCase(transformType, transformSize, bitDepth); + } + } + } + } + + /// + /// Compares one complete transform with the direct scalar two-axis definition. + /// + /// The compound transform type. + /// The transform-block dimensions. + /// The source sample bit depth. + private static void AssertTwoDimensionalCase(Av1TransformType transformType, Av1TransformSize transformSize, int bitDepth) + { + int width = transformSize.GetWidth(); + int height = transformSize.GetHeight(); + int inputStride = width + 3; + Av1TransformSize adjustedSize = transformSize.GetAdjusted(); + int coefficientCount = adjustedSize.GetWidth() * adjustedSize.GetHeight(); + short[] input = new short[inputStride * height]; + int sampleMaximum = (1 << bitDepth) - 1; + + for (int row = 0; row < height; row++) + { + for (int column = 0; column < width; column++) + { + int index = (row * width) + column; + input[(row * inputStride) + column] = (short)((index & 3) switch + { + 0 => sampleMaximum, + 1 => -sampleMaximum, + 2 => ((index * 73) % ((2 * sampleMaximum) + 1)) - sampleMaximum, + _ => 0, + }); + } + } + + int[] expected = new int[coefficientCount]; + int[] actual = new int[coefficientCount]; + int[] workspace = new int[Av1TransformWorkspace.MaximumLength]; + Av1Transform2dFlipConfiguration config = Av1Transform2dFlipConfiguration.CreateForward(transformType, transformSize, bitDepth); + + DispatchReferenceColumn(input, inputStride, expected, ref config); + Av1ForwardTransformer.Transform2d(input, actual, (uint)inputStride, transformType, transformSize, bitDepth, workspace); + Assert.Equal(expected, actual); + } + + /// + /// Selects the scalar reference column operator. + /// + /// The spatial residual samples. + /// The number of input samples between rows. + /// The destination reference coefficients. + /// The resolved transform functions, shifts, and axis orientation. + private static void DispatchReferenceColumn(ReadOnlySpan input, int stride, Span output, ref Av1Transform2dFlipConfiguration config) + { + switch (config.TransformFunctionTypeColumn) + { + case Av1TransformFunctionType.Dct4: + DispatchReferenceRow(input, stride, output, ref config); + break; + case Av1TransformFunctionType.Dct8: + DispatchReferenceRow(input, stride, output, ref config); + break; + case Av1TransformFunctionType.Dct16: + DispatchReferenceRow(input, stride, output, ref config); + break; + case Av1TransformFunctionType.Dct32: + DispatchReferenceRow(input, stride, output, ref config); + break; + case Av1TransformFunctionType.Dct64: + DispatchReferenceRow(input, stride, output, ref config); + break; + case Av1TransformFunctionType.Adst4: + DispatchReferenceRow(input, stride, output, ref config); + break; + case Av1TransformFunctionType.Adst8: + DispatchReferenceRow(input, stride, output, ref config); + break; + case Av1TransformFunctionType.Adst16: + DispatchReferenceRow(input, stride, output, ref config); + break; + case Av1TransformFunctionType.Identity4: + DispatchReferenceRow(input, stride, output, ref config); + break; + case Av1TransformFunctionType.Identity8: + DispatchReferenceRow(input, stride, output, ref config); + break; + case Av1TransformFunctionType.Identity16: + DispatchReferenceRow(input, stride, output, ref config); + break; + case Av1TransformFunctionType.Identity32: + DispatchReferenceRow(input, stride, output, ref config); + break; + } + } + + /// + /// Selects the scalar reference row operator. + /// + /// The column transform operator. + /// The spatial residual samples. + /// The number of input samples between rows. + /// The destination reference coefficients. + /// The resolved transform functions, shifts, and axis orientation. + private static void DispatchReferenceRow(ReadOnlySpan input, int stride, Span output, ref Av1Transform2dFlipConfiguration config) + where TColumnOperator : struct, Av1ForwardTransformer.IAv1ForwardTransform1dOperator + { + switch (config.TransformFunctionTypeRow) + { + case Av1TransformFunctionType.Dct4: + TransformReference(input, stride, output, ref config); + break; + case Av1TransformFunctionType.Dct8: + TransformReference(input, stride, output, ref config); + break; + case Av1TransformFunctionType.Dct16: + TransformReference(input, stride, output, ref config); + break; + case Av1TransformFunctionType.Dct32: + TransformReference(input, stride, output, ref config); + break; + case Av1TransformFunctionType.Dct64: + TransformReference(input, stride, output, ref config); + break; + case Av1TransformFunctionType.Adst4: + TransformReference(input, stride, output, ref config); + break; + case Av1TransformFunctionType.Adst8: + TransformReference(input, stride, output, ref config); + break; + case Av1TransformFunctionType.Adst16: + TransformReference(input, stride, output, ref config); + break; + case Av1TransformFunctionType.Identity4: + TransformReference(input, stride, output, ref config); + break; + case Av1TransformFunctionType.Identity8: + TransformReference(input, stride, output, ref config); + break; + case Av1TransformFunctionType.Identity16: + TransformReference(input, stride, output, ref config); + break; + case Av1TransformFunctionType.Identity32: + TransformReference(input, stride, output, ref config); + break; + } + } + + /// + /// Applies the direct scalar column and row transform definition used as the layout and dispatch oracle. + /// + /// The column transform operator. + /// The row transform operator. + /// The spatial residual samples. + /// The number of input samples between rows. + /// The destination reference coefficients. + /// The resolved transform functions, shifts, and axis orientation. + private static void TransformReference( + ReadOnlySpan input, + int stride, + Span output, + ref Av1Transform2dFlipConfiguration config) + where TColumnOperator : struct, Av1ForwardTransformer.IAv1ForwardTransform1dOperator + where TRowOperator : struct, Av1ForwardTransformer.IAv1ForwardTransform1dOperator + { + int width = config.TransformSize.GetWidth(); + int height = config.TransformSize.GetHeight(); + int outputWidth = Math.Min(width, 32); + int outputHeight = Math.Min(height, 32); + int[] intermediate = new int[width * height]; + Av1TransformVector values = default; + Av1TransformVector buffer0 = default; + Av1TransformVector buffer1 = default; + + for (int column = 0; column < width; column++) + { + for (int row = 0; row < height; row++) + { + int sourceRow = config.FlipUpsideDown ? height - row - 1 : row; + values[row] = input[(sourceRow * stride) + column] << config.Shift0; + } + + ref byte valuesBase = ref System.Runtime.CompilerServices.Unsafe.As, byte>(ref values); + + TColumnOperator.Transform(ref valuesBase, sizeof(int), sizeof(int), ref buffer0, ref buffer1, config.CosBitColumn); + int destinationColumn = config.FlipLeftToRight ? width - column - 1 : column; + + for (int row = 0; row < height; row++) + { + intermediate[(row * width) + destinationColumn] = Av1Math.RoundShift(values[row], -config.Shift1); + } + } + + bool normalizeRectangle = Math.Abs(config.TransformSize.GetRectangleLogRatio()) == 1; + + for (int row = 0; row < outputHeight; row++) + { + for (int column = 0; column < width; column++) + { + values[column] = intermediate[(row * width) + column]; + } + + ref byte valuesBase = ref System.Runtime.CompilerServices.Unsafe.As, byte>(ref values); + + TRowOperator.Transform(ref valuesBase, sizeof(int), sizeof(int), ref buffer0, ref buffer1, config.CosBitRow); + + for (int column = 0; column < outputWidth; column++) + { + int value = Av1Math.RoundShift(values[column], -config.Shift2); + output[(row * outputWidth) + column] = normalizeRectangle + ? Av1Transform1dMath.HalfButterfly(Av1Transform1dMath.NewSqrt2, value, 0, 0, Av1Transform1dMath.NewSqrt2Bits) + : value; + } + } + } + + /// + /// Gets a deterministic signed thirty-two-bit transform input. + /// + /// The transform position. + /// The independent SIMD lane. + /// The deterministic input value. + private static int GetInputValue(int index, int lane) + => (((index * 73) + (lane * 151)) % 8191) - 4095; + + /// + /// Gets a deterministic signed sixteen-bit input including overflow-sensitive edge values. + /// + /// The transform position. + /// The independent SIMD lane. + /// The deterministic packed input value. + private static short GetPackedInputValue(int index, int lane) + => (short)((index + lane) % 5 switch + { + 0 => short.MaxValue, + 1 => short.MinValue, + 2 => 255, + 3 => -255, + _ => (((index * 73) + (lane * 151)) % 511) - 255, + }); + + /// + /// Creates the complete normative transform matrix shared by the forward and inverse tests. + /// + /// Every permitted transform type, transform size, and AV1 image bit depth. + private static TheoryData CreateValidTransformCases() + { + TheoryData cases = []; + + for (Av1TransformSize transformSize = 0; transformSize < Av1TransformSize.AllSizes; transformSize++) + { + for (Av1TransformType transformType = 0; transformType < Av1TransformType.AllTransformTypes; transformType++) + { + Av1Transform2dFlipConfiguration config = Av1Transform2dFlipConfiguration.CreateForward(transformType, transformSize, 8); + + if (!config.IsAllowed()) + { + continue; + } + + for (int bitDepth = 8; bitDepth <= 12; bitDepth += 2) + { + cases.Add((int)transformType, (int)transformSize, bitDepth); + } + } + } + + return cases; + } +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1FrameBufferTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1FrameBufferTests.cs new file mode 100644 index 0000000000..188ff5e400 --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1FrameBufferTests.cs @@ -0,0 +1,381 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1; +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.LoopFilter; +using SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.Quantizers; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.ChromaFromLuma; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; +using SixLabors.ImageSharp.Formats.Heif.Av1.ReferenceFrames; +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; +using SixLabors.ImageSharp.Memory; +using SixLabors.ImageSharp.Tests.Memory; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1; + +/// +/// Verifies AV1 frame-plane allocation contracts. +/// +[Trait("Format", "Avif")] +public class Av1FrameBufferTests +{ + /// + /// Verifies that padded frame planes remain contiguous when the allocator would otherwise split the buffer. + /// + [Fact] + public void ConstructorRequestsContiguousPaddedPlanes() + { + TestMemoryAllocator allocator = new() { BufferCapacityInBytes = 10_000 }; + Configuration configuration = Configuration.Default.Clone(); + configuration.MemoryAllocator = allocator; + ObuSequenceHeader sequenceHeader = new() + { + MaxFrameWidth = 64, + MaxFrameHeight = 64, + ColorConfig = new ObuColorConfig + { + IsMonochrome = true, + BitDepth = Av1BitDepth.EightBit + } + }; + + using Av1FrameBuffer frameBuffer = new( + configuration, + sequenceHeader, + Av1ColorFormat.Yuv400, + false); + + MemoryGroup memoryGroup = frameBuffer.GetPlaneBuffer(Av1Plane.Y).FastMemoryGroup; + Assert.Equal(1, memoryGroup.Count); + Assert.True(memoryGroup.TotalLength > allocator.BufferCapacityInBytes); + } + + /// + /// Verifies that an external frame geometry cannot make the padded-plane owner fall back to multiple groups. + /// + [Fact] + public void ConstructorRejectsPaddedPlaneThatCannotBeContiguous() + { + TestMemoryAllocator allocator = new(); + allocator.EnableNonThreadSafeLogging(); + Configuration configuration = Configuration.Default.Clone(); + configuration.MemoryAllocator = allocator; + ObuSequenceHeader sequenceHeader = new() + { + MaxFrameWidth = 65_536, + MaxFrameHeight = 65_536, + ColorConfig = new ObuColorConfig + { + IsMonochrome = true, + BitDepth = Av1BitDepth.EightBit + } + }; + + Assert.Throws( + () => new Av1FrameBuffer(configuration, sequenceHeader, Av1ColorFormat.Yuv400, false)); + + Assert.Empty(allocator.AllocationLog); + } + + /// + /// Verifies that all padded component planes share one frame owner. + /// + [Fact] + public void ConstructorUsesOneFrameOwnerForAllPaddedPlanes() + { + TestMemoryAllocator allocator = new(); + allocator.EnableNonThreadSafeLogging(); + Configuration configuration = Configuration.Default.Clone(); + configuration.MemoryAllocator = allocator; + ObuSequenceHeader sequenceHeader = new() + { + MaxFrameWidth = 64, + MaxFrameHeight = 64, + ColorConfig = new ObuColorConfig + { + IsMonochrome = false, + SubSamplingX = true, + SubSamplingY = true, + BitDepth = Av1BitDepth.EightBit + } + }; + + TestMemoryAllocator.AllocationRequest allocation; + using (Av1FrameBuffer frameBuffer = new( + configuration, + sequenceHeader, + Av1ColorFormat.Yuv420, + false)) + { + allocation = Assert.Single(allocator.AllocationLog); + Assert.Empty(allocator.ReturnLog); + Assert.Equal(typeof(byte), allocation.ElementType); + Assert.Equal(614_400, allocation.Length); + Assert.Single(frameBuffer.GetPlaneBuffer(Av1Plane.Y).MemoryGroup); + Assert.Single(frameBuffer.GetPlaneBuffer(Av1Plane.U).MemoryGroup); + Assert.Single(frameBuffer.GetPlaneBuffer(Av1Plane.V).MemoryGroup); + } + + TestMemoryAllocator.ReturnRequest returned = Assert.Single(allocator.ReturnLog); + Assert.Equal(allocation.AllocationId, returned.AllocationId); + } + + /// + /// Verifies that block reconstruction uses one exact-size owner across monochrome and chroma plane layouts. + /// + [Theory] + [InlineData(true, false, false)] + [InlineData(false, true, true)] + [InlineData(false, true, false)] + [InlineData(false, false, false)] + public void BlockDecoderUsesOneContiguousWorkspaceOwner( + bool isMonochrome, + bool subsamplingX, + bool subsamplingY) + { + TestMemoryAllocator allocator = new(); + Configuration configuration = Configuration.Default.Clone(); + configuration.MemoryAllocator = allocator; + ObuSequenceHeader sequenceHeader = new() + { + MaxFrameWidth = 64, + MaxFrameHeight = 64, + Use128x128Superblock = false, + ColorConfig = new ObuColorConfig + { + IsMonochrome = isMonochrome, + SubSamplingX = subsamplingX, + SubSamplingY = subsamplingY, + BitDepth = Av1BitDepth.EightBit + } + }; + + Av1ColorFormat colorFormat = isMonochrome + ? Av1ColorFormat.Yuv400 + : subsamplingX + ? subsamplingY ? Av1ColorFormat.Yuv420 : Av1ColorFormat.Yuv422 + : Av1ColorFormat.Yuv444; + + using Av1FrameBuffer frameBuffer = new(configuration, sequenceHeader, colorFormat, false); + ObuFrameHeader frameHeader = new() + { + ModeInfoColumnCount = 16, + ModeInfoRowCount = 16 + }; + + using Av1LoopFilterContext loopFilterContext = + new(Configuration.Default.MemoryAllocator, sequenceHeader, frameHeader); + + using Av1ReferenceFrameStore referenceFrames = new(); + + // Reset the frame-plane logs so the following assertions describe only the block decoder's scratch owner. + allocator.EnableNonThreadSafeLogging(); + + int maximumBlockLength = 1 << sequenceHeader.SuperblockSizeLog2; + int maximumBlockArea = maximumBlockLength * maximumBlockLength; + int predictorWorkingLength = Math.Max( + Av1PredictionDecoder.ScratchLength, + Math.Max( + Av1TranslationalInterPredictor.GetScratchLength(maximumBlockLength, maximumBlockLength), + Av1ScaledInterPredictor.GetMaximumScaledScratchLength(maximumBlockLength, maximumBlockLength))); + + int predictionScratchLength = + (2 * maximumBlockArea) + + ((maximumBlockArea + 1) >> 1) + + predictorWorkingLength + + Av1ChromaFromLumaContext.BufferLength; + + int expectedWorkspaceLength = + (Av1TransformWorkspace.MaximumLength * 2) + + predictionScratchLength; + + TestMemoryAllocator.AllocationRequest workspaceAllocation; + using (Av1BlockDecoder blockDecoder = new( + sequenceHeader, + frameHeader, + frameBuffer, + loopFilterContext, + referenceFrames)) + { + workspaceAllocation = Assert.Single(allocator.AllocationLog); + Assert.Empty(allocator.ReturnLog); + Assert.Equal(typeof(short), workspaceAllocation.ElementType); + Assert.Equal(expectedWorkspaceLength, workspaceAllocation.Length); + } + + TestMemoryAllocator.ReturnRequest returned = Assert.Single(allocator.ReturnLog); + Assert.Equal(workspaceAllocation.AllocationId, returned.AllocationId); + } + + /// + /// Verifies that failure to allocate the active chroma transform map releases the preceding luma map. + /// + [Fact] + public void LoopFilterContextAllocationFailureReleasesLumaMap() + { + FailingTestMemoryAllocator allocator = new(failureAllocationNumber: 2); + ObuSequenceHeader sequenceHeader = new() + { + MaxFrameWidth = 64, + MaxFrameHeight = 64, + Use128x128Superblock = false, + ColorConfig = new ObuColorConfig + { + IsMonochrome = false, + SubSamplingX = true, + SubSamplingY = true, + BitDepth = Av1BitDepth.EightBit + } + }; + + ObuFrameHeader frameHeader = new() + { + ModeInfoColumnCount = 16, + ModeInfoRowCount = 16 + }; + + Assert.Throws( + () => new Av1LoopFilterContext(allocator, sequenceHeader, frameHeader)); + + TestMemoryAllocator.AllocationRequest allocation = Assert.Single(allocator.AllocationLog); + TestMemoryAllocator.ReturnRequest returned = Assert.Single(allocator.ReturnLog); + + Assert.Equal(2, allocator.AllocationAttemptCount); + Assert.Equal(allocation.HashCodeOfBuffer, returned.HashCodeOfBuffer); + } + + /// + /// Verifies that active-superblock coefficient scratch uses one configured allocator lease and omits unused + /// chroma storage for a monochrome frame. + /// + [Fact] + [ValidateDisposedMemoryAllocations] + public void FrameInfoCoefficientScratchUsesConfiguredAllocator() + { + TestMemoryAllocator allocator = new(); + allocator.EnableNonThreadSafeLogging(); + Configuration configuration = Configuration.Default.Clone(); + configuration.MemoryAllocator = allocator; + ObuSequenceHeader sequenceHeader = new() + { + MaxFrameWidth = 64, + MaxFrameHeight = 64, + Use128x128Superblock = false, + ColorConfig = new ObuColorConfig + { + IsMonochrome = true, + SubSamplingX = true, + SubSamplingY = true, + BitDepth = Av1BitDepth.EightBit + } + }; + + ObuFrameHeader frameHeader = new() + { + FrameSize = new ObuFrameSize + { + FrameWidth = 64, + FrameHeight = 64 + } + }; + + Av1FrameInfo frameInfo = new(configuration, sequenceHeader, frameHeader); + int expectedCoefficientCount = 16 * 16 * Av1FrameInfo.CoefficientCountPerModeInfo; + TestMemoryAllocator.AllocationRequest coefficientScratch = Assert.Single( + allocator.AllocationLog, + request => request.ElementType == typeof(int) && request.Length == expectedCoefficientCount); + + Assert.Equal(expectedCoefficientCount, frameInfo.GetCoefficientsY().Length); + Assert.Equal(0, frameInfo.GetCoefficientsU().Length); + Assert.Equal(0, frameInfo.GetCoefficientsV().Length); + + frameInfo.Dispose(); + + Assert.Single( + allocator.ReturnLog, + returned => returned.HashCodeOfBuffer == coefficientScratch.HashCodeOfBuffer); + } + + /// + /// Verifies that a later frame-state allocation failure returns the coefficient scratch rented first. + /// + [Fact] + public void FrameInfoConstructorFailureReleasesCoefficientScratch() + { + FailingTestMemoryAllocator allocator = new(failureAllocationNumber: 2); + Configuration configuration = Configuration.Default.Clone(); + configuration.MemoryAllocator = allocator; + ObuSequenceHeader sequenceHeader = new() + { + MaxFrameWidth = 64, + MaxFrameHeight = 64, + Use128x128Superblock = false, + ColorConfig = new ObuColorConfig + { + IsMonochrome = true, + SubSamplingX = true, + SubSamplingY = true, + BitDepth = Av1BitDepth.EightBit + } + }; + + ObuFrameHeader frameHeader = new() + { + FrameSize = new ObuFrameSize + { + FrameWidth = 64, + FrameHeight = 64 + } + }; + + Assert.Throws(() => new Av1FrameInfo(configuration, sequenceHeader, frameHeader)); + + TestMemoryAllocator.AllocationRequest coefficientScratch = Assert.Single(allocator.AllocationLog); + TestMemoryAllocator.ReturnRequest returned = Assert.Single(allocator.ReturnLog); + Assert.Equal(typeof(int), coefficientScratch.ElementType); + Assert.Equal(2, allocator.AllocationAttemptCount); + Assert.Equal(coefficientScratch.HashCodeOfBuffer, returned.HashCodeOfBuffer); + } + + /// + /// Provides tracked plane owners until the configured allocation attempt fails. + /// + private sealed class FailingTestMemoryAllocator : TestMemoryAllocator + { + private readonly int failureAllocationNumber; + private int allocationAttemptCount; + + /// + /// Initializes a new instance of the class. + /// + /// The one-based allocation attempt that must fail. + public FailingTestMemoryAllocator(int failureAllocationNumber) + { + this.failureAllocationNumber = failureAllocationNumber; + this.EnableNonThreadSafeLogging(); + } + + /// + /// Gets the number of backing-owner allocation attempts made through this allocator. + /// + public int AllocationAttemptCount => this.allocationAttemptCount; + + /// + protected override AllocationTrackedMemoryManager AllocateCore(int length, AllocationOptions options = AllocationOptions.None) + { + this.allocationAttemptCount++; + + if (this.allocationAttemptCount == this.failureAllocationNumber) + { + // Fail before delegating so this attempt never creates or tracks an owner. Diagnostic counts then + // describe only the successfully published owners that the constructor is responsible for releasing. + throw new InvalidMemoryOperationException("The configured AV1 allocation failed."); + } + + return base.AllocateCore(length, options); + } + } +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1FrameDecoderStub.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1FrameDecoderStub.cs new file mode 100644 index 0000000000..c7378c14af --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1FrameDecoderStub.cs @@ -0,0 +1,17 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline; +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1; + +internal class Av1FrameDecoderStub : IAv1FrameDecoder +{ + private readonly List superblocks = []; + + public void DecodeSuperblock(Point modeInfoPosition, Av1SuperblockInfo superblockInfo, Av1TileInfo tileInfo) + => this.superblocks.Add(superblockInfo); + + public int SuperblockCount => this.superblocks.Count; +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1GlobalMotionParametersTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1GlobalMotionParametersTests.cs new file mode 100644 index 0000000000..19d69f47ca --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1GlobalMotionParametersTests.cs @@ -0,0 +1,115 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1; +using SixLabors.ImageSharp.Formats.Heif.Av1.Motion; +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1; + +/// +/// Verifies AV1 global-motion-vector derivation against the fixed-point rules used by the reference decoder. +/// +[Trait("Format", "Avif")] +public class Av1GlobalMotionParametersTests +{ + /// + /// Verifies that an identity model produces no displacement at every block position. + /// + [Fact] + public void IdentityModelProducesZeroMotionVector() + { + Av1GlobalMotionParameters parameters = Av1GlobalMotionParameters.Identity; + + Av1MotionVector actual = parameters.GetMotionVector( + allowHighPrecisionMotionVector: true, + Av1BlockSize.Block128x128, + new Point(31, 17), + forceIntegerMotionVector: false); + + Assert.Equal(default, actual); + } + + /// + /// Verifies the published AV1 translation-component ordering and optional integer precision reduction. + /// + [Theory] + [InlineData(false, 19, -21)] + [InlineData(true, 16, -24)] + public void TranslationModelMatchesNormativeComponentOrdering(bool forceIntegerMotionVector, int expectedRow, int expectedColumn) + { + Av1GlobalMotionParameters parameters = Av1GlobalMotionParameters.Identity; + parameters.Type = Av1GlobalMotionType.Translation; + + // Translation parameters retain sixteen fractional bits; the derived vector retains three. + parameters[0] = 19 << 13; + parameters[1] = -21 << 13; + + Av1MotionVector actual = parameters.GetMotionVector( + allowHighPrecisionMotionVector: true, + Av1BlockSize.Block16x16, + new Point(4, 7), + forceIntegerMotionVector); + + Assert.Equal(new Av1MotionVector(expectedRow, expectedColumn), actual); + } + + /// + /// Verifies affine evaluation at the AV1 block center for high- and low-precision vector output. + /// + [Theory] + [InlineData(true, 1, 3)] + [InlineData(false, 0, 2)] + public void AffineModelEvaluatesBlockCenter(bool allowHighPrecisionMotionVector, int expectedRow, int expectedColumn) + { + Av1GlobalMotionParameters parameters = Av1GlobalMotionParameters.Identity; + parameters.Type = Av1GlobalMotionType.Affine; + + // The 8x8 block at mode-info position (2, 3) has center (11, 15). These deltas produce horizontal and + // vertical fixed-point offsets that exercise signed rounding at the selected output precision. + parameters[0] = 2048; + parameters[1] = -1024; + parameters[2] = Av1GlobalMotionParameters.ModelScale + 1024; + parameters[3] = 512; + parameters[4] = -256; + parameters[5] = Av1GlobalMotionParameters.ModelScale + 768; + + Av1MotionVector actual = parameters.GetMotionVector( + allowHighPrecisionMotionVector, + Av1BlockSize.Block8x8, + new Point(2, 3), + forceIntegerMotionVector: false); + + Assert.Equal(new Av1MotionVector(expectedRow, expectedColumn), actual); + } + + /// + /// Verifies local least-squares projection against a multi-sample model traced from the reference decoder. + /// + [Fact] + public void LocalProjectionMatchesReference() + { + Point[] sourcePoints = [new(24, -40), new(-40, 24), new(-24, -24), new(72, -24)]; + Point[] referencePoints = [new(-16, -8), new(-72, 64), new(-64, 16), new(32, 8)]; + + Av1GlobalMotionParameters parameters = Av1GlobalMotionParameters.DeriveLocalProjection( + sourcePoints, + referencePoints, + Av1BlockSize.Block8x8, + new Av1MotionVector(32, -40), + new Point(8, 6)); + + Assert.Equal(Av1GlobalMotionType.Affine, parameters.Type); + Assert.False(parameters.IsInvalid); + Assert.Equal(-191565, parameters[0]); + Assert.Equal(599107, parameters[1]); + Assert.Equal(61755, parameters[2]); + Assert.Equal(-140, parameters[3]); + Assert.Equal(-6909, parameters[4]); + Assert.Equal(62012, parameters[5]); + Assert.Equal(-3776, parameters.Alpha); + Assert.Equal(-128, parameters.Beta); + Assert.Equal(-7360, parameters.Gamma); + Assert.Equal(-3520, parameters.Delta); + } +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1InterFrameIntraEntropyTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1InterFrameIntraEntropyTests.cs new file mode 100644 index 0000000000..548e0a00ac --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1InterFrameIntraEntropyTests.cs @@ -0,0 +1,257 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Buffers; +using SixLabors.ImageSharp.Formats.Heif.Av1; +using SixLabors.ImageSharp.Formats.Heif.Av1.Entropy; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; +using SixLabors.ImageSharp.Memory; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1; + +/// +/// Verifies the entropy state and spatial contexts used by intra-coded blocks inside AV1 inter frames. +/// +[Trait("Format", "Avif")] +public class Av1InterFrameIntraEntropyTests +{ + /// + /// Gets the reference decoder's four forward Q15 luma-mode CDF rows in block-size-group order. + /// + private static ReadOnlySpan FrameYModeForwardThresholds => + [ + 22801, 23489, 24293, 24756, 25601, 26123, 26606, 27418, 27945, 29228, 29685, 30349, + 18673, 19845, 22631, 23318, 23950, 24649, 25527, 27364, 28152, 29701, 29984, 30852, + 19770, 20979, 23396, 23939, 24241, 24654, 25136, 27073, 27830, 29360, 29730, 30659, + 20155, 21301, 22838, 23178, 23261, 23533, 23703, 24804, 25352, 26575, 27016, 28049, + ]; + + /// + /// Verifies the four normative intra/inter distributions against the reference decoder's forward Q15 defaults. + /// + [Fact] + public void IntraInterDefaultsMatchReference() + { + uint[] forwardThresholds = [806, 16662, 20186, 26538]; + Av1Distribution[] distributions = Av1DefaultDistributions.IntraInter; + + Assert.Equal(forwardThresholds.Length, distributions.Length); + for (int context = 0; context < distributions.Length; context++) + { + // Av1Distribution stores inverse cumulative thresholds, so compare each forward default after the same + // forward-to-inverse conversion performed by its constructor. + Assert.Equal((uint)Av1Distribution.ProbabilityTop - forwardThresholds[context], distributions[context][0]); + Assert.Equal(2, distributions[context].NumberOfSymbols); + } + } + + /// + /// Verifies every inter-frame intra luma-mode threshold against the reference decoder's forward Q15 defaults. + /// + [Fact] + public void FrameYModeDefaultsMatchReference() + { + const int thresholdsPerGroup = 12; + ReadOnlySpan forwardThresholds = FrameYModeForwardThresholds; + Av1Distribution[] distributions = Av1DefaultDistributions.FrameYMode; + + Assert.Equal(4, distributions.Length); + for (int group = 0; group < distributions.Length; group++) + { + Assert.Equal(thresholdsPerGroup + 1, distributions[group].NumberOfSymbols); + + for (int threshold = 0; threshold < thresholdsPerGroup; threshold++) + { + uint expected = (uint)Av1Distribution.ProbabilityTop - forwardThresholds[(group * thresholdsPerGroup) + threshold]; + Assert.Equal(expected, distributions[group][threshold]); + } + } + } + + /// + /// Verifies that the intra/inter reader selects and adapts each of the four spatial-context distributions. + /// + /// The intra/inter spatial context. + [Theory] + [InlineData(0)] + [InlineData(1)] + [InlineData(2)] + [InlineData(3)] + public void ReadIsInterUsesRequestedContext(int context) + { + bool[] expected = [false, true, true, false, true, false, false, true]; + Av1Distribution writerDistribution = Av1DefaultDistributions.IntraInter[context]; + using Av1SymbolWriter writer = new(Configuration.Default, 8, updateCdf: true); + + foreach (bool value in expected) + { + writer.WriteSymbol(value, writerDistribution); + } + + using IMemoryOwner encoded = writer.Exit(); + Av1SymbolDecoder decoder = new(Configuration.Default, encoded.GetSpan(), 0, updateCdf: true); + + foreach (bool value in expected) + { + Assert.Equal(value, decoder.ReadIsInter(context)); + } + } + + /// + /// Verifies that inter-frame intra luma modes use the normative size group for every AV1 block size. + /// + /// The AV1 block-size enumeration value. + /// The normative size group from AV1 section 9.3. + [Theory] + [MemberData(nameof(GetBlockSizeGroups))] + public void ReadInterFrameYModeUsesNormativeSizeGroup(int blockSizeValue, int sizeGroup) + { + Av1BlockSize blockSize = (Av1BlockSize)blockSizeValue; + Av1PredictionMode[] expected = + [ + Av1PredictionMode.DC, + Av1PredictionMode.Directional45Degrees, + Av1PredictionMode.Smooth, + Av1PredictionMode.Paeth, + Av1PredictionMode.Horizontal, + Av1PredictionMode.Directional157Degrees, + ]; + + Av1Distribution writerDistribution = Av1DefaultDistributions.FrameYMode[sizeGroup]; + using Av1SymbolWriter writer = new(Configuration.Default, 8, updateCdf: true); + + foreach (Av1PredictionMode mode in expected) + { + writer.WriteSymbol((int)mode, writerDistribution); + } + + using IMemoryOwner encoded = writer.Exit(); + Av1SymbolDecoder decoder = new(Configuration.Default, encoded.GetSpan(), 0, updateCdf: true); + + foreach (Av1PredictionMode mode in expected) + { + Assert.Equal(mode, decoder.ReadInterFrameYMode(blockSize)); + } + } + + /// + /// Verifies that all four intra/inter contexts follow the normative above-and-left neighbor classification. + /// + /// Whether the above block is available. + /// Whether the available above block uses inter prediction. + /// Whether the left block is available. + /// Whether the available left block uses inter prediction. + /// The expected intra/inter context. + [Theory] + [InlineData(false, false, false, false, 0)] + [InlineData(true, true, false, false, 0)] + [InlineData(true, false, false, false, 2)] + [InlineData(false, false, true, true, 0)] + [InlineData(false, false, true, false, 2)] + [InlineData(true, true, true, true, 0)] + [InlineData(true, false, true, true, 1)] + [InlineData(true, true, true, false, 1)] + [InlineData(true, false, true, false, 3)] + public void IntraInterContextMatchesNeighborPredictionTypes( + bool hasAbove, + bool aboveIsInter, + bool hasLeft, + bool leftIsInter, + int expected) + { + Av1BlockModeInfo? above = hasAbove ? CreateModeInfo(aboveIsInter) : null; + Av1BlockModeInfo? left = hasLeft ? CreateModeInfo(leftIsInter) : null; + + int actual = Av1SymbolContextHelper.GetIntraInterContext(above, left); + + Assert.Equal(expected, actual); + } + + /// + /// Verifies that frame-context copies retain adapted inter-frame intra state without sharing mutable distributions. + /// + [Fact] + public void FrameEntropyCopyRetainsIndependentInterFrameIntraState() + { + Av1FrameEntropyContext source = new(0); + Av1FrameEntropyContext destination = new(0); + source.FrameYMode[2].Update((int)Av1PredictionMode.Smooth); + source.IntraInter[3].Update(1); + + destination.CopyFrom(source); + + Assert.Equal(source.FrameYMode[2][0], destination.FrameYMode[2][0]); + Assert.Equal(source.IntraInter[3][0], destination.IntraInter[3][0]); + + source.FrameYMode[2].Update((int)Av1PredictionMode.Paeth); + source.IntraInter[3].Update(0); + + Assert.NotEqual(source.FrameYMode[2][0], destination.FrameYMode[2][0]); + Assert.NotEqual(source.IntraInter[3][0], destination.IntraInter[3][0]); + } + + /// + /// Verifies that a published frame snapshot preserves adapted thresholds but resets their update-rate history. + /// + [Fact] + public void FrameEntropySnapshotResetsInterFrameIntraUpdateCounts() + { + const int updateCount = 20; + Av1FrameEntropyContext source = new(0); + Av1FrameEntropyContext snapshot = new(0); + + for (int i = 0; i < updateCount; i++) + { + source.FrameYMode[1].Update((int)Av1PredictionMode.Vertical); + source.IntraInter[1].Update(1); + } + + source.SnapshotTo(snapshot); + + Assert.Equal(source.FrameYMode[1][0], snapshot.FrameYMode[1][0]); + Assert.Equal(source.IntraInter[1][0], snapshot.IntraInter[1][0]); + + // The source retains twenty observations while the published snapshot restarts at zero. Applying the same + // symbol therefore moves identical thresholds by different update rates only when reset wiring is complete. + source.FrameYMode[1].Update((int)Av1PredictionMode.DC); + snapshot.FrameYMode[1].Update((int)Av1PredictionMode.DC); + source.IntraInter[1].Update(0); + snapshot.IntraInter[1].Update(0); + + Assert.NotEqual(source.FrameYMode[1][0], snapshot.FrameYMode[1][0]); + Assert.NotEqual(source.IntraInter[1][0], snapshot.IntraInter[1][0]); + } + + /// + /// Provides the normative AV1 size-group table in block-size enumeration order. + /// + /// Every decoded block size paired with its luma-mode size group. + public static TheoryData GetBlockSizeGroups() + { + // This is size_group_lookup from AV1 section 9.3 and the normative lookup table. Keeping expected values explicit + // ensures that the test does not reproduce the production formula it is intended to verify. + int[] sizeGroups = [0, 0, 0, 1, 1, 1, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 0, 0, 1, 1, 2, 2]; + TheoryData result = []; + + for (int blockSize = 0; blockSize < sizeGroups.Length; blockSize++) + { + result.Add(blockSize, sizeGroups[blockSize]); + } + + return result; + } + + /// + /// Creates decoded neighbor state with either an intra or inter primary reference. + /// + /// Whether the neighbor uses inter prediction. + /// The initialized block mode state. + private static Av1BlockModeInfo CreateModeInfo(bool isInter) + { + Av1BlockModeInfo modeInfo = new(Av1BlockSize.Block4x4, Point.Empty); + modeInfo.ReferenceFrames[0] = isInter ? Av1ReferenceFrameType.Last : Av1ReferenceFrameType.Intra; + modeInfo.ReferenceFrames[1] = Av1ReferenceFrameType.None; + return modeInfo; + } +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1InterFrameModeInfoTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1InterFrameModeInfoTests.cs new file mode 100644 index 0000000000..316c9ebe94 --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1InterFrameModeInfoTests.cs @@ -0,0 +1,441 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Buffers; +using SixLabors.ImageSharp.Formats.Heif.Av1; +using SixLabors.ImageSharp.Formats.Heif.Av1.Entropy; +using SixLabors.ImageSharp.Formats.Heif.Av1.Motion; +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; +using SixLabors.ImageSharp.Memory; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1; + +/// +/// Verifies inter-frame block-prefix, reference selection, motion-mode, and interpolation-filter syntax. +/// +[Trait("Format", "Avif")] +public class Av1InterFrameModeInfoTests +{ + /// + /// Verifies that an inter frame can select an intra-coded block using the block-size luma distribution. + /// + [Fact] + public void ReadInterFrameModeInfoReadsIntraCodedBlock() + { + ObuSequenceHeader sequenceHeader = CreateSequenceHeader(); + ObuFrameHeader frameHeader = CreateFrameHeader(); + using Av1TileReader tileReader = new(Configuration.Default, sequenceHeader, frameHeader); + Av1BlockModeInfo modeInfo = new(Av1BlockSize.Block8x8, Point.Empty); + Av1SuperblockInfo superblockInfo = new(tileReader.FrameInfo, Point.Empty); + Av1PartitionInfo partitionInfo = new(modeInfo, superblockInfo, false, Av1PartitionType.None); + + Av1Distribution skip = Av1DefaultDistributions.Skip[0]; + Av1Distribution intraInter = Av1DefaultDistributions.IntraInter[0]; + Av1Distribution yMode = Av1DefaultDistributions.FrameYMode[1]; + using Av1SymbolWriter writer = new(Configuration.Default, 1, updateCdf: true); + writer.WriteSymbol(false, skip); + writer.WriteSymbol(false, intraInter); + writer.WriteSymbol((int)Av1PredictionMode.DC, yMode); + using IMemoryOwner encoded = writer.Exit(); + Av1SymbolDecoder decoder = new(Configuration.Default, encoded.Memory.Span, 0, updateCdf: true); + + tileReader.ReadInterFrameModeInfo(ref decoder, ref partitionInfo, new Av1TileInfo(0, 0, frameHeader)); + modeInfo = partitionInfo.ModeInfo; + + Assert.False(modeInfo.SkipMode); + Assert.False(modeInfo.Skip); + Assert.Equal(Av1ReferenceFrameType.Intra, modeInfo.ReferenceFrames[0]); + Assert.Equal(Av1ReferenceFrameType.None, modeInfo.ReferenceFrames[1]); + Assert.Equal(Av1PredictionMode.DC, modeInfo.YMode); + Assert.Equal(Av1ChromaPredictionMode.DC, modeInfo.UvMode); + } + + /// + /// Verifies that skip mode omits the residual-skip and intra-inter symbols and marks the block as inter coded. + /// + [Fact] + public void SkipModeForcesInterAndSkipsResidual() + { + ObuSequenceHeader sequenceHeader = CreateSequenceHeader(); + sequenceHeader.OrderHintInfo.EnableOrderHint = true; + sequenceHeader.OrderHintInfo.OrderHintBits = 3; + ObuFrameHeader frameHeader = CreateFrameHeader(); + frameHeader.ReferenceMode = ObuReferenceMode.ReferenceModeSelect; + frameHeader.OrderHint = 4; + for (int index = 0; index < Av1Constants.ReferencesPerFrame; index++) + { + frameHeader.GetReferenceFrameIndices()[index] = (uint)index; + } + + Span referenceOrderHints = frameHeader.GetReferenceOrderHints(); + referenceOrderHints[0] = 3; + referenceOrderHints[1] = 2; + referenceOrderHints[2] = 1; + referenceOrderHints[3] = 0; + referenceOrderHints[4] = 5; + referenceOrderHints[5] = 6; + referenceOrderHints[6] = 7; + frameHeader.SkipModeParameters.Derive(sequenceHeader.OrderHintInfo, frameHeader); + frameHeader.SkipModeParameters.SkipModeFlag = true; + using Av1TileReader tileReader = new(Configuration.Default, sequenceHeader, frameHeader); + Av1BlockModeInfo aboveModeInfo = new(Av1BlockSize.Block8x8, Point.Empty) { SkipMode = true }; + Av1BlockModeInfo modeInfo = new(Av1BlockSize.Block8x8, Point.Empty); + + Av1Distribution skipMode = Av1DefaultDistributions.SkipMode[1]; + using Av1SymbolWriter writer = new(Configuration.Default, 1, updateCdf: true); + writer.WriteSymbol(true, skipMode); + using IMemoryOwner encoded = writer.Exit(); + Memory encodedMemory = encoded.Memory; + + modeInfo = ReadInterFrameModeInfo(tileReader, encodedMemory, modeInfo, aboveModeInfo); + + Assert.True(modeInfo.SkipMode); + Assert.True(modeInfo.Skip); + Assert.Equal(Av1PredictionMode.NearestNearestMotionVector, modeInfo.YMode); + Assert.Equal(Av1ReferenceFrameType.Last, modeInfo.ReferenceFrames[0]); + Assert.Equal(Av1ReferenceFrameType.Backward, modeInfo.ReferenceFrames[1]); + Assert.Equal(Av1CompoundType.Average, modeInfo.CompoundType); + } + + /// + /// Verifies switchable interpolation-filter decoding with shared and independent axis selections. + /// + /// Whether the horizontal axis carries an independent filter symbol. + /// The expected horizontal interpolation filter. + [Theory] + [InlineData(false, (int)Av1InterpolationFilter.Smooth)] + [InlineData(true, (int)Av1InterpolationFilter.Sharp)] + public void ReadInterFrameModeInfoReadsInterpolationFilters( + bool enableDualFilter, + int expectedHorizontalFilter) + { + ObuSequenceHeader sequenceHeader = CreateSequenceHeader(); + sequenceHeader.EnableDualFilter = enableDualFilter; + ObuFrameHeader frameHeader = CreateFrameHeader(); + frameHeader.InterpolationFilter = Av1InterpolationFilter.Switchable; + + // Forcing segment zero to GLOBALMV removes reference and inter-mode symbols from this focused fixture. A + // translational global model still requires interpolation, leaving only the filter branch under test. + ObuSegmentationParameters segmentationParameters = frameHeader.SegmentationParameters; + segmentationParameters.Enabled = true; + segmentationParameters.SetFeatureEnabled(0, (int)ObuSegmentationLevelFeature.GlobalMotionVector, true); + frameHeader.GetGlobalMotionParameters()[0].Type = Av1GlobalMotionType.Translation; + + using Av1TileReader tileReader = new(Configuration.Default, sequenceHeader, frameHeader); + Av1BlockModeInfo modeInfo = new(Av1BlockSize.Block8x8, Point.Empty); + Av1SuperblockInfo superblockInfo = new(tileReader.FrameInfo, Point.Empty); + Av1PartitionInfo partitionInfo = new(modeInfo, superblockInfo, false, Av1PartitionType.None); + using Av1SymbolWriter writer = new(Configuration.Default, 2, updateCdf: true); + writer.WriteSymbol(false, Av1DefaultDistributions.Skip[0]); + writer.WriteSymbol((int)Av1InterpolationFilter.Smooth, Av1DefaultDistributions.SwitchableInterpolation[3]); + if (enableDualFilter) + { + writer.WriteSymbol((int)Av1InterpolationFilter.Sharp, Av1DefaultDistributions.SwitchableInterpolation[11]); + } + + using IMemoryOwner encoded = writer.Exit(); + Av1SymbolDecoder decoder = new(Configuration.Default, encoded.GetSpan(), 0, updateCdf: true); + + tileReader.ReadInterFrameModeInfo(ref decoder, ref partitionInfo, new Av1TileInfo(0, 0, frameHeader)); + modeInfo = partitionInfo.ModeInfo; + + Assert.Equal(Av1InterpolationFilter.Smooth, modeInfo.InterpolationFilters[0]); + Assert.Equal((Av1InterpolationFilter)expectedHorizontalFilter, modeInfo.InterpolationFilters[1]); + } + + /// + /// Verifies that an identity global-motion block omits switchable interpolation-filter symbols. + /// + [Fact] + public void IdentityGlobalMotionOmitsInterpolationFilters() + { + ObuSequenceHeader sequenceHeader = CreateSequenceHeader(); + sequenceHeader.EnableDualFilter = true; + ObuFrameHeader frameHeader = CreateFrameHeader(); + frameHeader.InterpolationFilter = Av1InterpolationFilter.Switchable; + ObuSegmentationParameters segmentationParameters = frameHeader.SegmentationParameters; + segmentationParameters.Enabled = true; + segmentationParameters.SetFeatureEnabled(0, (int)ObuSegmentationLevelFeature.GlobalMotionVector, true); + + using Av1TileReader tileReader = new(Configuration.Default, sequenceHeader, frameHeader); + Av1BlockModeInfo modeInfo = new(Av1BlockSize.Block8x8, Point.Empty); + Av1SuperblockInfo superblockInfo = new(tileReader.FrameInfo, Point.Empty); + Av1PartitionInfo partitionInfo = new(modeInfo, superblockInfo, false, Av1PartitionType.None); + using Av1SymbolWriter writer = new(Configuration.Default, 3, updateCdf: true); + writer.WriteSymbol(false, Av1DefaultDistributions.Skip[0]); + + // Identity is distinct from Translation for this syntax gate. These sentinel symbols must remain unread even + // though the separate global-motion-block classification requires a model greater than Translation. + writer.WriteSymbol((int)Av1InterpolationFilter.Smooth, Av1DefaultDistributions.SwitchableInterpolation[3]); + writer.WriteSymbol((int)Av1InterpolationFilter.Sharp, Av1DefaultDistributions.SwitchableInterpolation[11]); + using IMemoryOwner encoded = writer.Exit(); + Av1SymbolDecoder decoder = new(Configuration.Default, encoded.GetSpan(), 0, updateCdf: true); + + tileReader.ReadInterFrameModeInfo(ref decoder, ref partitionInfo, new Av1TileInfo(0, 0, frameHeader)); + modeInfo = partitionInfo.ModeInfo; + + Assert.Equal(Av1InterpolationFilter.Regular, modeInfo.InterpolationFilters[0]); + Assert.Equal(Av1InterpolationFilter.Regular, modeInfo.InterpolationFilters[1]); + } + + /// + /// Verifies every unidirectional and bidirectional compound reference-tree leaf through paired motion parsing. + /// + /// The zero-based normative compound reference pair. + /// The expected primary retained-reference label. + /// The expected secondary retained-reference label. + [Theory] + [InlineData(0, (int)Av1ReferenceFrameType.Backward, (int)Av1ReferenceFrameType.Alternate)] + [InlineData(1, (int)Av1ReferenceFrameType.Last, (int)Av1ReferenceFrameType.Last2)] + [InlineData(2, (int)Av1ReferenceFrameType.Last, (int)Av1ReferenceFrameType.Last3)] + [InlineData(3, (int)Av1ReferenceFrameType.Last, (int)Av1ReferenceFrameType.Golden)] + [InlineData(4, (int)Av1ReferenceFrameType.Last, (int)Av1ReferenceFrameType.Backward)] + [InlineData(5, (int)Av1ReferenceFrameType.Last2, (int)Av1ReferenceFrameType.Alternate2)] + [InlineData(6, (int)Av1ReferenceFrameType.Last3, (int)Av1ReferenceFrameType.Alternate)] + [InlineData(7, (int)Av1ReferenceFrameType.Golden, (int)Av1ReferenceFrameType.Alternate)] + public void ReadInterFrameModeInfoReadsCompoundReferencePair( + int pairIndex, + int expectedPrimary, + int expectedSecondary) + { + ObuSequenceHeader sequenceHeader = CreateSequenceHeader(); + ObuFrameHeader frameHeader = CreateFrameHeader(); + frameHeader.ReferenceMode = ObuReferenceMode.ReferenceModeSelect; + using Av1TileReader tileReader = new(Configuration.Default, sequenceHeader, frameHeader); + Av1BlockModeInfo modeInfo = new(Av1BlockSize.Block8x8, Point.Empty); + using Av1SymbolWriter writer = new(Configuration.Default, 8, updateCdf: true); + writer.WriteSymbol(false, Av1DefaultDistributions.Skip[0]); + writer.WriteSymbol(true, Av1DefaultDistributions.IntraInter[0]); + writer.WriteSymbol(true, Av1DefaultDistributions.CompInter[1]); + WriteCompoundReferencePair(writer, pairIndex); + writer.WriteSymbol(0, Av1DefaultDistributions.InterCompoundMode[0]); + + using IMemoryOwner encoded = writer.Exit(); + Memory encodedMemory = encoded.Memory; + + modeInfo = ReadInterFrameModeInfo(tileReader, encodedMemory, modeInfo); + + Assert.Equal((Av1ReferenceFrameType)expectedPrimary, modeInfo.ReferenceFrames[0]); + Assert.Equal((Av1ReferenceFrameType)expectedSecondary, modeInfo.ReferenceFrames[1]); + Assert.Equal(Av1PredictionMode.NearestNearestMotionVector, modeInfo.YMode); + Assert.Equal(default(Av1MotionVector), modeInfo.MotionVectors[0]); + Assert.Equal(default(Av1MotionVector), modeInfo.MotionVectors[1]); + Assert.Equal(Av1CompoundType.Average, modeInfo.CompoundType); + } + + /// + /// Verifies selectable compound syntax in its normative position before interpolation filtering. + /// + /// The selected compound operation. + [Theory] + [InlineData((int)Av1CompoundType.DistanceWeighted)] + [InlineData((int)Av1CompoundType.Wedge)] + [InlineData((int)Av1CompoundType.DifferenceWeighted)] + public void ReadsSelectableCompoundBeforeInterpolation(int compoundTypeValue) + { + Av1CompoundType compoundType = (Av1CompoundType)compoundTypeValue; + ObuSequenceHeader sequenceHeader = CreateSequenceHeader(); + sequenceHeader.EnableMaskedCompound = true; + sequenceHeader.EnableDualFilter = false; + sequenceHeader.OrderHintInfo.EnableOrderHint = true; + sequenceHeader.OrderHintInfo.EnableJointCompound = true; + sequenceHeader.OrderHintInfo.OrderHintBits = 3; + ObuFrameHeader frameHeader = CreateFrameHeader(); + frameHeader.ReferenceMode = ObuReferenceMode.ReferenceModeSelect; + frameHeader.InterpolationFilter = Av1InterpolationFilter.Switchable; + frameHeader.OrderHint = 4; + frameHeader.GetReferenceFrameIndices()[0] = 0; + frameHeader.GetReferenceFrameIndices()[1] = 1; + frameHeader.GetReferenceOrderHints()[0] = 3; + frameHeader.GetReferenceOrderHints()[1] = 5; + + using Av1TileReader tileReader = new(Configuration.Default, sequenceHeader, frameHeader); + Av1BlockModeInfo modeInfo = new(Av1BlockSize.Block8x8, Point.Empty); + using Av1SymbolWriter writer = new(Configuration.Default, 12, updateCdf: true); + writer.WriteSymbol(false, Av1DefaultDistributions.Skip[0]); + writer.WriteSymbol(true, Av1DefaultDistributions.IntraInter[0]); + writer.WriteSymbol(true, Av1DefaultDistributions.CompInter[1]); + WriteCompoundReferencePair(writer, pairIndex: 1); + writer.WriteSymbol(0, Av1DefaultDistributions.InterCompoundMode[0]); + + bool masked = compoundType is Av1CompoundType.Wedge or Av1CompoundType.DifferenceWeighted; + writer.WriteSymbol(masked, Av1DefaultDistributions.CompoundGroupIndex[0]); + if (masked) + { + writer.WriteSymbol( + compoundType == Av1CompoundType.Wedge ? 0 : 1, + Av1DefaultDistributions.CompoundType[(int)Av1BlockSize.Block8x8]); + + if (compoundType == Av1CompoundType.Wedge) + { + writer.WriteSymbol(13, Av1DefaultDistributions.WedgeIndex[(int)Av1BlockSize.Block8x8]); + writer.WriteLiteral(true); + } + else + { + writer.WriteLiteral(true); + } + } + else + { + // Equal reference distances select context three; false chooses distance weighting. + writer.WriteSymbol(false, Av1DefaultDistributions.CompoundIndex[3]); + } + + writer.WriteSymbol((int)Av1InterpolationFilter.Sharp, Av1DefaultDistributions.SwitchableInterpolation[3]); + + using IMemoryOwner encoded = writer.Exit(); + modeInfo = ReadInterFrameModeInfo(tileReader, encoded.Memory, modeInfo); + + Assert.Equal(Av1ReferenceFrameType.Last, modeInfo.ReferenceFrames[0]); + Assert.Equal(Av1ReferenceFrameType.Last2, modeInfo.ReferenceFrames[1]); + Assert.Equal(masked, modeInfo.CompoundGroupIndex); + Assert.Equal(compoundType != Av1CompoundType.DistanceWeighted, modeInfo.CompoundIndex); + Assert.Equal(compoundType, modeInfo.CompoundType); + Assert.Equal(compoundType == Av1CompoundType.Wedge ? 13 : 0, modeInfo.CompoundWedgeIndex); + Assert.Equal(compoundType == Av1CompoundType.Wedge, modeInfo.CompoundWedgeSign); + Assert.Equal( + compoundType == Av1CompoundType.DifferenceWeighted + ? Av1DifferenceWeightedMaskType.Type38Inverse + : Av1DifferenceWeightedMaskType.Type38, + modeInfo.DifferenceWeightedMaskType); + + Assert.Equal(Av1InterpolationFilter.Sharp, modeInfo.InterpolationFilters[0]); + Assert.Equal(Av1InterpolationFilter.Sharp, modeInfo.InterpolationFilters[1]); + } + + /// + /// Invokes the ref-struct mode parser with one available above neighbor. + /// + /// The tile reader. + /// The range-coded block-prefix symbols. + /// The current coding block. + /// The available above block supplying skip-mode context. + /// The decoded block mode information. + private static Av1BlockModeInfo ReadInterFrameModeInfo( + Av1TileReader tileReader, + Memory encoded, + Av1BlockModeInfo modeInfo, + Av1BlockModeInfo aboveModeInfo) + { + Av1SuperblockInfo superblockInfo = new(tileReader.FrameInfo, Point.Empty); + Av1PartitionInfo partitionInfo = new(modeInfo, superblockInfo, false, Av1PartitionType.None) + { + AvailableAbove = true, + AboveModeInfo = aboveModeInfo, + }; + + Av1SymbolDecoder decoder = new(Configuration.Default, encoded.Span, 0, updateCdf: true); + tileReader.ReadInterFrameModeInfo(ref decoder, ref partitionInfo, new Av1TileInfo(0, 0, tileReader.FrameHeader)); + return partitionInfo.ModeInfo; + } + + /// + /// Invokes the ref-struct mode parser without spatial neighbors. + /// + /// The decoded block mode information. + private static Av1BlockModeInfo ReadInterFrameModeInfo( + Av1TileReader tileReader, + Memory encoded, + Av1BlockModeInfo modeInfo) + { + Av1SuperblockInfo superblockInfo = new(tileReader.FrameInfo, Point.Empty); + Av1PartitionInfo partitionInfo = new(modeInfo, superblockInfo, false, Av1PartitionType.None); + Av1SymbolDecoder decoder = new(Configuration.Default, encoded.Span, 0, updateCdf: true); + tileReader.ReadInterFrameModeInfo(ref decoder, ref partitionInfo, new Av1TileInfo(0, 0, tileReader.FrameHeader)); + return partitionInfo.ModeInfo; + } + + /// + /// Writes one complete compound-reference tree leaf using the neutral no-neighbor contexts. + /// + private static void WriteCompoundReferencePair(Av1SymbolWriter writer, int pairIndex) + { + bool bidirectional = pairIndex >= 4; + writer.WriteSymbol(bidirectional, Av1DefaultDistributions.CompoundReferenceType[2]); + if (!bidirectional) + { + bool backwardPair = pairIndex == 0; + writer.WriteSymbol(backwardPair, Av1DefaultDistributions.UnidirectionalCompoundReference[1][0]); + if (!backwardPair) + { + bool last3OrGolden = pairIndex >= 2; + writer.WriteSymbol(last3OrGolden, Av1DefaultDistributions.UnidirectionalCompoundReference[1][1]); + if (last3OrGolden) + { + writer.WriteSymbol(pairIndex == 3, Av1DefaultDistributions.UnidirectionalCompoundReference[1][2]); + } + } + + return; + } + + bool last3OrGoldenForward = pairIndex >= 6; + writer.WriteSymbol(last3OrGoldenForward, Av1DefaultDistributions.CompoundReference[1][0]); + if (last3OrGoldenForward) + { + writer.WriteSymbol(pairIndex == 7, Av1DefaultDistributions.CompoundReference[1][2]); + } + else + { + writer.WriteSymbol(pairIndex == 5, Av1DefaultDistributions.CompoundReference[1][1]); + } + + bool alternateBackward = pairIndex >= 6; + writer.WriteSymbol(alternateBackward, Av1DefaultDistributions.CompoundBackwardReference[1][0]); + if (!alternateBackward) + { + writer.WriteSymbol(pairIndex == 5, Av1DefaultDistributions.CompoundBackwardReference[1][1]); + } + } + + /// + /// Creates the monochrome 64x64 sequence geometry used by direct mode-prefix tests. + /// + /// The initialized sequence header. + private static ObuSequenceHeader CreateSequenceHeader() + => new() + { + MaxFrameWidth = 64, + MaxFrameHeight = 64, + Use128x128Superblock = false, + EnableCdef = false, + EnableFilterIntra = false, + ColorConfig = new ObuColorConfig + { + IsMonochrome = true, + BitDepth = Av1BitDepth.EightBit, + }, + }; + + /// + /// Creates an inter-frame header whose optional block-prefix tools are disabled. + /// + /// The initialized frame header. + private static ObuFrameHeader CreateFrameHeader() + { + ObuFrameHeader frameHeader = new() + { + FrameType = ObuFrameType.InterFrame, + ModeInfoColumnCount = 16, + ModeInfoRowCount = 16, + CodedLossless = true, + AllowScreenContentTools = false, + FrameSize = new ObuFrameSize + { + FrameWidth = 64, + FrameHeight = 64, + SuperResolutionUpscaledWidth = 64, + RenderWidth = 64, + RenderHeight = 64, + }, + }; + + frameHeader.TilesInfo.TileColumnStartModeInfo[1] = frameHeader.ModeInfoColumnCount; + frameHeader.TilesInfo.TileRowStartModeInfo[1] = frameHeader.ModeInfoRowCount; + return frameHeader; + } +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1InterIntraEntropyTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1InterIntraEntropyTests.cs new file mode 100644 index 0000000000..4aa4ab975d --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1InterIntraEntropyTests.cs @@ -0,0 +1,170 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Buffers; +using SixLabors.ImageSharp.Formats.Heif.Av1; +using SixLabors.ImageSharp.Formats.Heif.Av1.Entropy; +using SixLabors.ImageSharp.Memory; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1; + +/// +/// Verifies the entropy state and block-size groups used by the AV1 inter-intra prediction flag. +/// +[Trait("Format", "Avif")] +public class Av1InterIntraEntropyTests +{ + /// + /// Verifies the four block-size-group distributions against the reference decoder's forward Q15 defaults. + /// + [Fact] + public void DefaultsMatchReference() + { + ReadOnlySpan forwardThresholds = [16384, 26887, 27597, 30237]; + Av1Distribution[] distributions = Av1DefaultDistributions.InterIntra; + + Assert.Equal(forwardThresholds.Length, distributions.Length); + for (int group = 0; group < distributions.Length; group++) + { + // Av1Distribution stores inverse cumulative thresholds, so convert the reference decoder's forward threshold before + // comparing the exact Q15 state consumed by the range decoder. + uint expected = (uint)Av1Distribution.ProbabilityTop - forwardThresholds[group]; + + Assert.Equal(expected, distributions[group][0]); + Assert.Equal(2, distributions[group].NumberOfSymbols); + } + } + + /// + /// Verifies every AV1 block size against the normative size-group conversion table. + /// + /// The AV1 block-size enumeration value. + /// The normative zero-based size group. + [Theory] + [MemberData(nameof(GetBlockSizeGroups))] + public void GetSizeGroupMatchesNormativeTable(int blockSizeValue, int expectedGroup) + { + Av1BlockSize blockSize = (Av1BlockSize)blockSizeValue; + + Assert.Equal(expectedGroup, blockSize.GetSizeGroup()); + } + + /// + /// Verifies that the inter-intra flag reader selects and adapts the distribution for each size group. + /// + /// A block-size enumeration value representing one size group. + /// The expected zero-based size group. + [Theory] + [InlineData((int)Av1BlockSize.Block4x4, 0)] + [InlineData((int)Av1BlockSize.Block8x8, 1)] + [InlineData((int)Av1BlockSize.Block16x16, 2)] + [InlineData((int)Av1BlockSize.Block32x32, 3)] + public void ReaderUsesBlockSizeGroup(int blockSizeValue, int sizeGroup) + { + bool[] expected = [false, true, true, false, true, false, false, true]; + Av1Distribution writerDistribution = Av1DefaultDistributions.InterIntra[sizeGroup]; + using Av1SymbolWriter writer = new(Configuration.Default, expected.Length, updateCdf: true); + + foreach (bool value in expected) + { + writer.WriteSymbol(value, writerDistribution); + } + + using IMemoryOwner encoded = writer.Exit(); + Av1SymbolDecoder decoder = new(Configuration.Default, encoded.GetSpan(), 0, updateCdf: true); + Av1BlockSize blockSize = (Av1BlockSize)blockSizeValue; + + foreach (bool value in expected) + { + Assert.Equal(value, decoder.ReadIsInterIntra(blockSize)); + } + } + + /// + /// Verifies that frame-context copies retain adapted inter-intra state without sharing mutable distributions. + /// + [Fact] + public void FrameEntropyCopyRetainsIndependentState() + { + Av1FrameEntropyContext source = new(0); + Av1FrameEntropyContext destination = new(0); + source.InterIntra[2].Update(1); + + destination.CopyFrom(source); + + Assert.NotSame(source.InterIntra[2], destination.InterIntra[2]); + Assert.Equal(source.InterIntra[2][0], destination.InterIntra[2][0]); + + source.InterIntra[2].Update(0); + + Assert.NotEqual(source.InterIntra[2][0], destination.InterIntra[2][0]); + } + + /// + /// Verifies that resetting a frame context restores the default threshold and adaptation state. + /// + [Fact] + public void FrameEntropyResetRestoresDefaultState() + { + Av1FrameEntropyContext context = new(0); + Av1FrameEntropyContext expected = new(0); + context.InterIntra[3].Update(1); + + context.ResetToDefaults(0); + + Assert.Equal(expected.InterIntra[3][0], context.InterIntra[3][0]); + + // Applying the same next observation proves that reset restored the update-rate history as well as the visible + // threshold; otherwise two equal thresholds would diverge because their adaptation rates differ. + context.InterIntra[3].Update(0); + expected.InterIntra[3].Update(0); + + Assert.Equal(expected.InterIntra[3][0], context.InterIntra[3][0]); + } + + /// + /// Verifies that a published frame snapshot preserves adapted thresholds and resets their update-rate history. + /// + [Fact] + public void FrameEntropySnapshotResetsUpdateCount() + { + const int updateCount = 20; + Av1FrameEntropyContext source = new(0); + Av1FrameEntropyContext snapshot = new(0); + + for (int i = 0; i < updateCount; i++) + { + source.InterIntra[1].Update(1); + } + + source.SnapshotTo(snapshot); + + Assert.Equal(source.InterIntra[1][0], snapshot.InterIntra[1][0]); + + // The source retains twenty observations while the published snapshot restarts at zero. Their next identical + // observation must therefore move the shared starting threshold by different update rates. + source.InterIntra[1].Update(0); + snapshot.InterIntra[1].Update(0); + + Assert.NotEqual(source.InterIntra[1][0], snapshot.InterIntra[1][0]); + } + + /// + /// Provides the normative AV1 size-group table in block-size enumeration order. + /// + /// Every decoded block size paired with its size group. + public static TheoryData GetBlockSizeGroups() + { + // These are the explicit Size_Group values from AV1 section 9.3 and the normative lookup table. The test keeps the + // expected table independent from the production geometry formula so a shared calculation cannot mask errors. + int[] sizeGroups = [0, 0, 0, 1, 1, 1, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 0, 0, 1, 1, 2, 2]; + TheoryData result = []; + + for (int blockSize = 0; blockSize < sizeGroups.Length; blockSize++) + { + result.Add(blockSize, sizeGroups[blockSize]); + } + + return result; + } +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1InterModeEntropyTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1InterModeEntropyTests.cs new file mode 100644 index 0000000000..0340d42657 --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1InterModeEntropyTests.cs @@ -0,0 +1,223 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Buffers; +using SixLabors.ImageSharp.Formats.Heif.Av1.Entropy; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1; + +/// +/// Verifies the adaptive distributions and packed contexts used to select an AV1 single-reference inter mode. +/// +[Trait("Format", "Avif")] +public class Av1InterModeEntropyTests +{ + /// + /// Verifies the normative single-reference inter-mode distributions against the reference decoder's forward Q15 defaults. + /// + [Fact] + public void InterModeDefaultsMatchReference() + { + AssertBinaryDefaults(Av1DefaultDistributions.NewMv, [24035, 16630, 15339, 8386, 12222, 4676]); + AssertBinaryDefaults(Av1DefaultDistributions.ZeroMv, [2175, 1054]); + AssertBinaryDefaults(Av1DefaultDistributions.RefMv, [23974, 24188, 17848, 28622, 24312, 19923]); + AssertBinaryDefaults(Av1DefaultDistributions.Drl, [13104, 24560, 18945]); + } + + /// + /// Verifies that each inter-mode context occupies the normative field in the packed mode context. + /// + /// The packed mode context. + /// The expected newly decoded motion-vector context. + /// The expected global-motion context. + /// The expected spatial reference-motion-vector context. + [Theory] + [InlineData(0, 0, 0, 0)] + [InlineData(77, 5, 1, 4)] + [InlineData(93, 5, 1, 5)] + public void PackedInterModeContextMatchesReference(int modeContext, int expectedNewMv, int expectedZeroMv, int expectedRefMv) + { + Assert.Equal(expectedNewMv, Av1SymbolContextHelper.GetNewMvContext(modeContext)); + Assert.Equal(expectedZeroMv, Av1SymbolContextHelper.GetZeroMvContext(modeContext)); + Assert.Equal(expectedRefMv, Av1SymbolContextHelper.GetRefMvContext(modeContext)); + } + + /// + /// Verifies the exact short-circuit order and symbol polarity of the single-reference inter-mode tree. + /// + /// The expected prediction mode. + /// The new-motion-vector decision. + /// The global-motion decision, or negative when the leaf precedes it. + /// The spatial reference-motion-vector decision, or negative when the leaf precedes it. + [Theory] + [InlineData((int)Av1PredictionMode.NewMotionVector, 0, -1, -1)] + [InlineData((int)Av1PredictionMode.GlobalMotionVector, 1, 0, -1)] + [InlineData((int)Av1PredictionMode.NearestMotionVector, 1, 1, 0)] + [InlineData((int)Av1PredictionMode.NearMotionVector, 1, 1, 1)] + public void ReadInterModeMatchesReference(int expectedMode, int newMvSymbol, int zeroMvSymbol, int refMvSymbol) + { + const int modeContext = 77; + Av1Distribution newMv = Av1DefaultDistributions.NewMv[5]; + Av1Distribution zeroMv = Av1DefaultDistributions.ZeroMv[1]; + Av1Distribution refMv = Av1DefaultDistributions.RefMv[4]; + Av1Distribution drl = Av1DefaultDistributions.Drl[2]; + using Av1SymbolWriter writer = new(Configuration.Default, 8, updateCdf: true); + + writer.WriteSymbol(newMvSymbol, newMv); + if (zeroMvSymbol >= 0) + { + writer.WriteSymbol(zeroMvSymbol, zeroMv); + } + + if (refMvSymbol >= 0) + { + writer.WriteSymbol(refMvSymbol, refMv); + } + + // A symbol after the selected leaf proves that the decoder consumed exactly the decisions on that branch. + writer.WriteSymbol(true, drl); + + using IMemoryOwner encoded = writer.Exit(); + Av1SymbolDecoder decoder = new(Configuration.Default, encoded.Memory.Span, 0, updateCdf: true); + + Assert.Equal((Av1PredictionMode)expectedMode, decoder.ReadInterMode(modeContext)); + Assert.True(decoder.ReadDrl(2)); + } + + /// + /// Verifies that the dynamic reference-list reader selects each requested context distribution. + /// + /// The dynamic reference-list context. + [Theory] + [InlineData(0)] + [InlineData(1)] + [InlineData(2)] + public void ReadDrlUsesRequestedContext(int context) + { + bool[] expected = [false, true, true, false, true, false, false, true]; + Av1Distribution writerDistribution = Av1DefaultDistributions.Drl[context]; + using Av1SymbolWriter writer = new(Configuration.Default, 8, updateCdf: true); + + foreach (bool value in expected) + { + writer.WriteSymbol(value, writerDistribution); + } + + using IMemoryOwner encoded = writer.Exit(); + Av1SymbolDecoder decoder = new(Configuration.Default, encoded.Memory.Span, 0, updateCdf: true); + + foreach (bool value in expected) + { + Assert.Equal(value, decoder.ReadDrl(context)); + } + } + + /// + /// Verifies the four candidate-weight pairings used to select a dynamic reference-list context. + /// + /// The current candidate's weight. + /// The next candidate's weight. + /// The expected dynamic reference-list context. + [Theory] + [InlineData(640, 640, 0)] + [InlineData(640, 639, 1)] + [InlineData(639, 639, 2)] + [InlineData(639, 640, 0)] + public void DrlContextMatchesCandidateWeightCategories(ushort currentWeight, ushort nextWeight, int expected) + { + ushort[] referenceWeights = [currentWeight, nextWeight]; + + Assert.Equal(expected, Av1SymbolContextHelper.GetDrlContext(referenceWeights, 0)); + } + + /// + /// Verifies that frame-context copies retain inter-mode adaptation without sharing mutable distributions. + /// + [Fact] + public void FrameEntropyCopyRetainsIndependentInterModeState() + { + Av1FrameEntropyContext source = new(0); + Av1FrameEntropyContext destination = new(0); + source.NewMv[5].Update(1); + source.ZeroMv[1].Update(1); + source.RefMv[4].Update(1); + source.Drl[2].Update(1); + + destination.CopyFrom(source); + + Assert.Equal(source.NewMv[5][0], destination.NewMv[5][0]); + Assert.Equal(source.ZeroMv[1][0], destination.ZeroMv[1][0]); + Assert.Equal(source.RefMv[4][0], destination.RefMv[4][0]); + Assert.Equal(source.Drl[2][0], destination.Drl[2][0]); + + source.NewMv[5].Update(0); + source.ZeroMv[1].Update(0); + source.RefMv[4].Update(0); + source.Drl[2].Update(0); + + Assert.NotEqual(source.NewMv[5][0], destination.NewMv[5][0]); + Assert.NotEqual(source.ZeroMv[1][0], destination.ZeroMv[1][0]); + Assert.NotEqual(source.RefMv[4][0], destination.RefMv[4][0]); + Assert.NotEqual(source.Drl[2][0], destination.Drl[2][0]); + } + + /// + /// Verifies that publishing frame state resets the inter-mode distributions' update-rate history. + /// + [Fact] + public void FrameEntropySnapshotResetsInterModeUpdateCounts() + { + const int updateCount = 20; + Av1FrameEntropyContext source = new(0); + Av1FrameEntropyContext snapshot = new(0); + + for (int i = 0; i < updateCount; i++) + { + source.NewMv[5].Update(1); + source.ZeroMv[1].Update(1); + source.RefMv[4].Update(1); + source.Drl[2].Update(1); + } + + source.SnapshotTo(snapshot); + + Assert.Equal(source.NewMv[5][0], snapshot.NewMv[5][0]); + Assert.Equal(source.ZeroMv[1][0], snapshot.ZeroMv[1][0]); + Assert.Equal(source.RefMv[4][0], snapshot.RefMv[4][0]); + Assert.Equal(source.Drl[2][0], snapshot.Drl[2][0]); + + // The source retains twenty observations while the snapshot restarts at zero. Applying the same next symbol + // therefore moves identical thresholds by different amounts only when the new distributions participate in reset. + source.NewMv[5].Update(0); + snapshot.NewMv[5].Update(0); + source.ZeroMv[1].Update(0); + snapshot.ZeroMv[1].Update(0); + source.RefMv[4].Update(0); + snapshot.RefMv[4].Update(0); + source.Drl[2].Update(0); + snapshot.Drl[2].Update(0); + + Assert.NotEqual(source.NewMv[5][0], snapshot.NewMv[5][0]); + Assert.NotEqual(source.ZeroMv[1][0], snapshot.ZeroMv[1][0]); + Assert.NotEqual(source.RefMv[4][0], snapshot.RefMv[4][0]); + Assert.NotEqual(source.Drl[2][0], snapshot.Drl[2][0]); + } + + /// + /// Verifies binary distribution defaults after their conversion to the inverse cumulative representation. + /// + /// The distributions under test. + /// The normative forward Q15 thresholds. + private static void AssertBinaryDefaults(Av1Distribution[] distributions, ReadOnlySpan forwardThresholds) + { + Assert.Equal(forwardThresholds.Length, distributions.Length); + for (int context = 0; context < distributions.Length; context++) + { + uint expected = (uint)Av1Distribution.ProbabilityTop - forwardThresholds[context]; + + Assert.Equal(expected, distributions[context][0]); + Assert.Equal(2, distributions[context].NumberOfSymbols); + } + } +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1InterpolationFilterEntropyTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1InterpolationFilterEntropyTests.cs new file mode 100644 index 0000000000..cfeadfcee0 --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1InterpolationFilterEntropyTests.cs @@ -0,0 +1,358 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Buffers; +using SixLabors.ImageSharp.Formats.Heif.Av1; +using SixLabors.ImageSharp.Formats.Heif.Av1.Entropy; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1; + +/// +/// Verifies the adaptive distributions and spatial contexts used by AV1 switchable interpolation filters. +/// +[Trait("Format", "Avif")] +public class Av1InterpolationFilterEntropyTests +{ + /// + /// The number of reference, direction, and neighbor-state combinations represented by the distribution table. + /// + private const int SwitchableInterpolationContextCount = 16; + + /// + /// The interpolation-filter direction index for vertical prediction. + /// + private const int VerticalDirection = 0; + + /// + /// The interpolation-filter direction index for horizontal prediction. + /// + private const int HorizontalDirection = 1; + + /// + /// Gets the reference decoder's forward Q15 switchable interpolation-filter thresholds in context order. + /// + private static ReadOnlySpan ForwardThresholds => + [ + 31935, 32720, + 5568, 32719, + 422, 2938, + 28244, 32608, + 31206, 31953, + 4862, 32121, + 770, 1152, + 20889, 25637, + 31910, 32724, + 4120, 32712, + 305, 2247, + 27403, 32636, + 31022, 32009, + 2963, 32093, + 601, 943, + 14969, 21398, + ]; + + /// + /// Verifies every normative switchable interpolation-filter distribution against the reference decoder's forward Q15 defaults. + /// + [Fact] + public void DefaultsMatchReference() + { + const int thresholdCount = 2; + ReadOnlySpan forwardThresholds = ForwardThresholds; + Av1Distribution[] distributions = Av1DefaultDistributions.SwitchableInterpolation; + + Assert.Equal(SwitchableInterpolationContextCount, distributions.Length); + for (int context = 0; context < distributions.Length; context++) + { + Assert.Equal(thresholdCount + 1, distributions[context].NumberOfSymbols); + + for (int threshold = 0; threshold < thresholdCount; threshold++) + { + // Av1Distribution stores inverse cumulative thresholds. Complement each published forward value by + // the same Q15 probability top used during production construction before comparing exact state. + uint expected = (uint)Av1Distribution.ProbabilityTop - forwardThresholds[(context * thresholdCount) + threshold]; + + Assert.Equal(expected, distributions[context][threshold]); + } + } + } + + /// + /// Verifies that the symbol reader selects and adapts each of the sixteen switchable interpolation contexts. + /// + /// The reference, direction, and neighbor filter context. + [Theory] + [MemberData(nameof(GetContexts))] + public void ReaderUsesRequestedContext(int context) + { + Av1InterpolationFilter[] expected = + [ + Av1InterpolationFilter.Regular, + Av1InterpolationFilter.Sharp, + Av1InterpolationFilter.Smooth, + Av1InterpolationFilter.Regular, + Av1InterpolationFilter.Smooth, + Av1InterpolationFilter.Sharp, + ]; + + Av1Distribution writerDistribution = Av1DefaultDistributions.SwitchableInterpolation[context]; + using Av1SymbolWriter writer = new(Configuration.Default, expected.Length, updateCdf: true); + + foreach (Av1InterpolationFilter filter in expected) + { + writer.WriteSymbol((int)filter, writerDistribution); + } + + using IMemoryOwner encoded = writer.Exit(); + Av1SymbolDecoder decoder = new(Configuration.Default, encoded.Memory.Span, 0, updateCdf: true); + + foreach (Av1InterpolationFilter filter in expected) + { + Assert.Equal(filter, decoder.ReadSwitchableInterpolationFilter(context)); + } + } + + /// + /// Verifies the encoder's context selection, read-only costing, and adaptive output against independently seeded distributions. + /// + [Theory] + [MemberData(nameof(GetContexts))] + public void EncoderUsesRequestedContextAndLiveCosts(int context) + { + ReadOnlySpan filters = + [ + Av1InterpolationFilter.Sharp, + Av1InterpolationFilter.Smooth, + Av1InterpolationFilter.Regular, + Av1InterpolationFilter.Sharp, + Av1InterpolationFilter.Regular, + Av1InterpolationFilter.Smooth, + ]; + + // The constructor converts forward thresholds to inverse CDF storage. Supply the published forward + // values directly, independently of the production context factory. + Av1Distribution distribution = new( + ForwardThresholds[context * 2], + ForwardThresholds[(context * 2) + 1]); + + using Av1SymbolWriter expectedWriter = new(Configuration.Default, 64, updateCdf: true); + using Av1SymbolEncoder encoder = new(Configuration.Default, 64, qIndex: 0, updateCdf: true); + foreach (Av1InterpolationFilter filter in filters) + { + int expectedCost = Av1ProbabilityCost.GetSymbolCost(distribution, (int)filter); + Assert.Equal(expectedCost, encoder.GetSwitchableInterpolationFilterCost(filter, context)); + Assert.Equal(expectedCost, encoder.GetSwitchableInterpolationFilterCost(filter, context)); + expectedWriter.WriteSymbol((int)filter, distribution); + encoder.WriteSwitchableInterpolationFilter(filter, context); + } + + using IMemoryOwner expected = expectedWriter.Exit(); + using IMemoryOwner actual = encoder.Exit(); + Assert.Equal(expected.Memory.Span, actual.Memory.Span); + Av1SymbolDecoder decoder = new(Configuration.Default, actual.Memory.Span, 0, updateCdf: true); + foreach (Av1InterpolationFilter filter in filters) + { + Assert.Equal(filter, decoder.ReadSwitchableInterpolationFilter(context)); + } + } + + /// + /// Verifies all sixteen combinations of reference type, direction, and contributing neighbor filter state. + /// + [Fact] + public void ContextLayoutMatchesReference() + { + Av1BlockModeInfo single = CreateModeInfo( + Av1ReferenceFrameType.Last, + Av1ReferenceFrameType.None, + Av1InterpolationFilter.Regular, + Av1InterpolationFilter.Regular); + + Av1BlockModeInfo compound = CreateModeInfo( + Av1ReferenceFrameType.Last, + Av1ReferenceFrameType.Backward, + Av1InterpolationFilter.Regular, + Av1InterpolationFilter.Regular); + + Av1BlockModeInfo regular = CreateModeInfo( + Av1ReferenceFrameType.Last, + Av1ReferenceFrameType.None, + Av1InterpolationFilter.Regular, + Av1InterpolationFilter.Regular); + + Av1BlockModeInfo smooth = CreateModeInfo( + Av1ReferenceFrameType.Last, + Av1ReferenceFrameType.None, + Av1InterpolationFilter.Smooth, + Av1InterpolationFilter.Smooth); + + Av1BlockModeInfo sharp = CreateModeInfo( + Av1ReferenceFrameType.Last, + Av1ReferenceFrameType.None, + Av1InterpolationFilter.Sharp, + Av1InterpolationFilter.Sharp); + + // Contexts zero through three are single-reference vertical contexts. Compound prediction adds four, while + // horizontal prediction adds eight. The mixed Regular/Smooth pair selects the fourth neighbor state. + Assert.Equal(0, Av1SymbolContextHelper.GetSwitchableInterpolationContext(single, regular, null, VerticalDirection)); + Assert.Equal(1, Av1SymbolContextHelper.GetSwitchableInterpolationContext(single, smooth, null, VerticalDirection)); + Assert.Equal(2, Av1SymbolContextHelper.GetSwitchableInterpolationContext(single, sharp, null, VerticalDirection)); + Assert.Equal(3, Av1SymbolContextHelper.GetSwitchableInterpolationContext(single, regular, smooth, VerticalDirection)); + Assert.Equal(4, Av1SymbolContextHelper.GetSwitchableInterpolationContext(compound, regular, null, VerticalDirection)); + Assert.Equal(5, Av1SymbolContextHelper.GetSwitchableInterpolationContext(compound, smooth, null, VerticalDirection)); + Assert.Equal(6, Av1SymbolContextHelper.GetSwitchableInterpolationContext(compound, sharp, null, VerticalDirection)); + Assert.Equal(7, Av1SymbolContextHelper.GetSwitchableInterpolationContext(compound, regular, smooth, VerticalDirection)); + Assert.Equal(8, Av1SymbolContextHelper.GetSwitchableInterpolationContext(single, regular, null, HorizontalDirection)); + Assert.Equal(9, Av1SymbolContextHelper.GetSwitchableInterpolationContext(single, smooth, null, HorizontalDirection)); + Assert.Equal(10, Av1SymbolContextHelper.GetSwitchableInterpolationContext(single, sharp, null, HorizontalDirection)); + Assert.Equal(11, Av1SymbolContextHelper.GetSwitchableInterpolationContext(single, regular, smooth, HorizontalDirection)); + Assert.Equal(12, Av1SymbolContextHelper.GetSwitchableInterpolationContext(compound, regular, null, HorizontalDirection)); + Assert.Equal(13, Av1SymbolContextHelper.GetSwitchableInterpolationContext(compound, smooth, null, HorizontalDirection)); + Assert.Equal(14, Av1SymbolContextHelper.GetSwitchableInterpolationContext(compound, sharp, null, HorizontalDirection)); + Assert.Equal(15, Av1SymbolContextHelper.GetSwitchableInterpolationContext(compound, regular, smooth, HorizontalDirection)); + } + + /// + /// Verifies that only neighbors sharing the current primary reference contribute their directional filter. + /// + [Fact] + public void ContextUsesMatchingPrimaryOrSecondaryNeighborReference() + { + Av1BlockModeInfo current = CreateModeInfo( + Av1ReferenceFrameType.Last, + Av1ReferenceFrameType.None, + Av1InterpolationFilter.Regular, + Av1InterpolationFilter.Regular); + + Av1BlockModeInfo secondaryMatch = CreateModeInfo( + Av1ReferenceFrameType.Golden, + Av1ReferenceFrameType.Last, + Av1InterpolationFilter.Smooth, + Av1InterpolationFilter.Sharp); + + Av1BlockModeInfo mismatch = CreateModeInfo( + Av1ReferenceFrameType.Golden, + Av1ReferenceFrameType.None, + Av1InterpolationFilter.Regular, + Av1InterpolationFilter.Regular); + + Assert.Equal(1, Av1SymbolContextHelper.GetSwitchableInterpolationContext(current, secondaryMatch, mismatch, VerticalDirection)); + Assert.Equal(10, Av1SymbolContextHelper.GetSwitchableInterpolationContext(current, secondaryMatch, mismatch, HorizontalDirection)); + Assert.Equal(3, Av1SymbolContextHelper.GetSwitchableInterpolationContext(current, mismatch, null, VerticalDirection)); + Assert.Equal(11, Av1SymbolContextHelper.GetSwitchableInterpolationContext(current, null, null, HorizontalDirection)); + } + + /// + /// Verifies that frame-context copies retain interpolation adaptation without sharing mutable distributions. + /// + [Fact] + public void FrameEntropyCopyRetainsIndependentInterpolationState() + { + Av1FrameEntropyContext source = new(0); + Av1FrameEntropyContext destination = new(0); + source.SwitchableInterpolation[15].Update((int)Av1InterpolationFilter.Sharp); + + destination.CopyFrom(source); + + Assert.Equal(source.SwitchableInterpolation[15][0], destination.SwitchableInterpolation[15][0]); + + source.SwitchableInterpolation[15].Update((int)Av1InterpolationFilter.Regular); + + Assert.NotEqual(source.SwitchableInterpolation[15][0], destination.SwitchableInterpolation[15][0]); + } + + /// + /// Verifies that restoring frame defaults replaces adapted interpolation thresholds and update history. + /// + [Fact] + public void FrameEntropyResetRestoresInterpolationDefaults() + { + const int updateCount = 20; + Av1FrameEntropyContext context = new(0); + + for (int i = 0; i < updateCount; i++) + { + context.SwitchableInterpolation[5].Update((int)Av1InterpolationFilter.Sharp); + } + + context.ResetToDefaults(0); + + Av1Distribution expected = Av1DefaultDistributions.SwitchableInterpolation[5]; + + Assert.Equal(expected[0], context.SwitchableInterpolation[5][0]); + Assert.Equal(expected[1], context.SwitchableInterpolation[5][1]); + + expected.Update((int)Av1InterpolationFilter.Smooth); + context.SwitchableInterpolation[5].Update((int)Av1InterpolationFilter.Smooth); + + Assert.Equal(expected[0], context.SwitchableInterpolation[5][0]); + Assert.Equal(expected[1], context.SwitchableInterpolation[5][1]); + } + + /// + /// Verifies that publishing frame state resets interpolation update history while retaining adapted thresholds. + /// + [Fact] + public void FrameEntropySnapshotResetsInterpolationUpdateCounts() + { + const int updateCount = 20; + Av1FrameEntropyContext source = new(0); + Av1FrameEntropyContext snapshot = new(0); + + for (int i = 0; i < updateCount; i++) + { + source.SwitchableInterpolation[7].Update((int)Av1InterpolationFilter.Smooth); + } + + source.SnapshotTo(snapshot); + + Assert.Equal(source.SwitchableInterpolation[7][0], snapshot.SwitchableInterpolation[7][0]); + + // The source retains its observations while the snapshot restarts at zero. Applying the same next symbol moves + // identical thresholds by different amounts only when the new distribution participates in snapshot reset. + source.SwitchableInterpolation[7].Update((int)Av1InterpolationFilter.Regular); + snapshot.SwitchableInterpolation[7].Update((int)Av1InterpolationFilter.Regular); + + Assert.NotEqual(source.SwitchableInterpolation[7][0], snapshot.SwitchableInterpolation[7][0]); + } + + /// + /// Provides every switchable interpolation-filter context. + /// + /// The sixteen zero-based context indices. + public static TheoryData GetContexts() + { + TheoryData result = []; + + for (int context = 0; context < SwitchableInterpolationContextCount; context++) + { + result.Add(context); + } + + return result; + } + + /// + /// Creates decoded block state with the requested references and directional interpolation filters. + /// + /// The primary reference label. + /// The optional secondary reference label. + /// The vertical interpolation filter. + /// The horizontal interpolation filter. + /// The initialized block mode state. + private static Av1BlockModeInfo CreateModeInfo( + Av1ReferenceFrameType primaryReference, + Av1ReferenceFrameType secondaryReference, + Av1InterpolationFilter verticalFilter, + Av1InterpolationFilter horizontalFilter) + { + Av1BlockModeInfo modeInfo = new(Av1BlockSize.Block8x8, Point.Empty); + modeInfo.ReferenceFrames[0] = primaryReference; + modeInfo.ReferenceFrames[1] = secondaryReference; + modeInfo.InterpolationFilters[0] = verticalFilter; + modeInfo.InterpolationFilters[1] = horizontalFilter; + return modeInfo; + } +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1IntraBlockCopyPredictorTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1IntraBlockCopyPredictorTests.cs new file mode 100644 index 0000000000..06bec5fa60 --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1IntraBlockCopyPredictorTests.cs @@ -0,0 +1,235 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.IntraBlockCopy; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; +using SixLabors.ImageSharp.Tests.TestUtilities; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1; + +/// +/// Verifies AV1 intra-block-copy interpolation across the supported hardware-intrinsic configurations. +/// +[Trait("Format", "Heif")] +public class Av1IntraBlockCopyPredictorTests +{ + /// + /// Exercises each SIMD register-width tier and the complete scalar fallback. + /// + private const HwIntrinsics PredictorConfigurations = + HwIntrinsics.AllowAll | HwIntrinsics.DisableAVX512F | HwIntrinsics.DisableAVX | HwIntrinsics.DisableHWIntrinsic; + + /// + /// Verifies all four source phases for 8-bit samples at every AV1 transform size. + /// + [Fact] + public void EightBitPredictionMatchesScalar() + => FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidateEightBitPrediction, PredictorConfigurations); + + /// + /// Verifies all four source phases for high-bit-depth samples at every AV1 transform size. + /// + [Fact] + public void HighBitDepthPredictionMatchesScalar() + => FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidateHighBitDepthPrediction, PredictorConfigurations); + + /// + /// Verifies the four normative interpolation equations against independently calculated sample blocks. + /// + [Fact] + public void PredictionMatchesKnownInterpolationValues() + => FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidateKnownInterpolationValues, PredictorConfigurations); + + /// + /// Compares the SIMD-first 8-bit implementation with its scalar definition and verifies that row padding is unchanged. + /// + private static void ValidateEightBitPrediction() + { + for (int sizeIndex = 0; sizeIndex < (int)Av1TransformSize.AllSizes; sizeIndex++) + { + Av1TransformSize transformSize = (Av1TransformSize)sizeIndex; + int width = transformSize.GetWidth(); + int height = transformSize.GetHeight(); + int sourceStride = width + 17; + int destinationStride = width + 7; + byte[] source = new byte[sourceStride * (height + 1)]; + + for (int i = 0; i < source.Length; i++) + { + source[i] = (byte)((i * 29) + 17); + } + + for (int phase = 0; phase < 4; phase++) + { + byte[] expected = Enumerable.Repeat((byte)0xA5, destinationStride * height).ToArray(); + byte[] actual = Enumerable.Repeat((byte)0xA5, destinationStride * height).ToArray(); + bool halfX = (phase & 1) != 0; + bool halfY = (phase & 2) != 0; + + Av1IntraBlockCopyPredictor.PredictScalar( + source, + sourceStride, + expected, + destinationStride, + width, + height, + halfX, + halfY); + + Av1IntraBlockCopyPredictor.Predict( + source, + sourceStride, + actual, + destinationStride, + width, + height, + halfX, + halfY); + + Assert.Equal(expected, actual); + } + } + } + + /// + /// Compares the SIMD-first high-bit-depth implementation with its scalar definition and verifies exact-width stores. + /// + private static void ValidateHighBitDepthPrediction() + { + for (int sizeIndex = 0; sizeIndex < (int)Av1TransformSize.AllSizes; sizeIndex++) + { + Av1TransformSize transformSize = (Av1TransformSize)sizeIndex; + int width = transformSize.GetWidth(); + int height = transformSize.GetHeight(); + int sourceStride = width + 9; + int destinationStride = width + 5; + short[] source = new short[sourceStride * (height + 1)]; + + for (int i = 0; i < source.Length; i++) + { + source[i] = (short)(((i * 53) + 31) & 0xFFF); + } + + for (int phase = 0; phase < 4; phase++) + { + short[] expected = Enumerable.Repeat((short)0x5A5A, destinationStride * height).ToArray(); + short[] actual = Enumerable.Repeat((short)0x5A5A, destinationStride * height).ToArray(); + bool halfX = (phase & 1) != 0; + bool halfY = (phase & 2) != 0; + + Av1IntraBlockCopyPredictor.PredictScalar( + source, + sourceStride, + expected, + destinationStride, + width, + height, + halfX, + halfY); + + Av1IntraBlockCopyPredictor.Predict( + source, + sourceStride, + actual, + destinationStride, + width, + height, + halfX, + halfY); + + Assert.Equal(expected, actual); + } + } + } + + /// + /// Applies each source phase to a four-by-four block whose expected results are simple arithmetic progressions. + /// + private static void ValidateKnownInterpolationValues() + { + const int sourceStride = 21; + byte[] source = new byte[sourceStride * 5]; + for (int row = 0; row < 5; row++) + { + for (int column = 0; column < 5; column++) + { + source[(row * sourceStride) + column] = (byte)((row * 20) + (column * 4)); + } + } + + ReadOnlySpan copied = + [ + 0, 4, 8, 12, + 20, 24, 28, 32, + 40, 44, 48, 52, + 60, 64, 68, 72, + ]; + + ReadOnlySpan horizontal = + [ + 2, 6, 10, 14, + 22, 26, 30, 34, + 42, 46, 50, 54, + 62, 66, 70, 74, + ]; + + ReadOnlySpan vertical = + [ + 10, 14, 18, 22, + 30, 34, 38, 42, + 50, 54, 58, 62, + 70, 74, 78, 82, + ]; + + ReadOnlySpan bilinear = + [ + 12, 16, 20, 24, + 32, 36, 40, 44, + 52, 56, 60, 64, + 72, 76, 80, 84, + ]; + + ValidateKnownPhase(source, sourceStride, copied, false, false); + ValidateKnownPhase(source, sourceStride, horizontal, true, false); + ValidateKnownPhase(source, sourceStride, vertical, false, true); + ValidateKnownPhase(source, sourceStride, bilinear, true, true); + } + + /// + /// Verifies one four-by-four source phase for both 8-bit and translated high-bit-depth samples. + /// + /// The five-by-five 8-bit source region. + /// The number of source samples per row. + /// The independently calculated four-by-four prediction. + /// Indicates whether the horizontal phase is one half-sample. + /// Indicates whether the vertical phase is one half-sample. + private static void ValidateKnownPhase( + ReadOnlySpan source, + int sourceStride, + ReadOnlySpan expected, + bool halfX, + bool halfY) + { + byte[] actual = new byte[16]; + Av1IntraBlockCopyPredictor.Predict(source, sourceStride, actual, 4, 4, 4, halfX, halfY); + Assert.Equal(expected, actual); + + const int highBitDepthOffset = 1024; + short[] highBitDepthSource = new short[source.Length]; + short[] highBitDepthExpected = new short[expected.Length]; + short[] highBitDepthActual = new short[16]; + + for (int i = 0; i < source.Length; i++) + { + highBitDepthSource[i] = (short)(source[i] + highBitDepthOffset); + } + + for (int i = 0; i < expected.Length; i++) + { + highBitDepthExpected[i] = (short)(expected[i] + highBitDepthOffset); + } + + Av1IntraBlockCopyPredictor.Predict(highBitDepthSource, sourceStride, highBitDepthActual, 4, 4, 4, halfX, halfY); + Assert.Equal(highBitDepthExpected, highBitDepthActual); + } +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1IntraBlockCopyTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1IntraBlockCopyTests.cs new file mode 100644 index 0000000000..0fcaba5cdb --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1IntraBlockCopyTests.cs @@ -0,0 +1,623 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1; +using SixLabors.ImageSharp.Formats.Heif.Av1.Entropy; +using SixLabors.ImageSharp.Formats.Heif.Av1.Motion; +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline; +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; +using SixLabors.ImageSharp.Memory; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1; + +/// +/// Verifies AV1 intra-block-copy reference derivation and displacement-vector legality rules. +/// +[Trait("Format", "Heif")] +public class Av1IntraBlockCopyTests +{ + /// + /// Verifies the horizontal fallback used in the tile's first superblock row. + /// + [Fact] + public void FindReferenceUsesFirstRowFallback() + { + ObuSequenceHeader sequenceHeader = CreateSequenceHeader(); + using Av1FrameInfo frameInfo = new(sequenceHeader); + Av1SuperblockInfo superblockInfo = frameInfo.GetSuperblock(new Point(5, 0)); + Av1BlockModeInfo modeInfo = new(Av1BlockSize.Block16x16, Point.Empty); + Av1PartitionInfo partitionInfo = new(modeInfo, superblockInfo, true, Av1PartitionType.None) + { + ColumnIndex = 80, + RowIndex = 0, + }; + + Av1TileInfo tileInfo = CreateTileInfo(); + Av1MotionVector[] candidates = new Av1MotionVector[8]; + int[] weights = new int[8]; + + Av1MotionVector actual = Av1IntraBlockCopy.FindReference( + ref partitionInfo, + tileInfo, + sequenceHeader.SuperblockModeInfoSize, + candidates, + weights); + + Assert.Equal(new Av1MotionVector(0, -2560), actual); + } + + /// + /// Verifies the vertical fallback used after the tile's first superblock row when spatial candidates are absent. + /// + [Fact] + public void FindReferenceUsesPreviousSuperblockRowFallback() + { + ObuSequenceHeader sequenceHeader = CreateSequenceHeader(); + using Av1FrameInfo frameInfo = new(sequenceHeader); + Av1SuperblockInfo aboveSuperblock = frameInfo.GetSuperblock(new Point(5, 0)); + Av1BlockModeInfo aboveModeInfo = new(Av1BlockSize.Block64x64, Point.Empty); + frameInfo.UpdateModeInfo(aboveModeInfo, aboveSuperblock); + + Av1SuperblockInfo superblockInfo = frameInfo.GetSuperblock(new Point(5, 1)); + Av1BlockModeInfo modeInfo = new(Av1BlockSize.Block16x16, Point.Empty); + Av1PartitionInfo partitionInfo = new(modeInfo, superblockInfo, true, Av1PartitionType.None) + { + AvailableAbove = true, + ColumnIndex = 80, + RowIndex = 16, + }; + + Av1TileInfo tileInfo = CreateTileInfo(); + Av1MotionVector[] candidates = new Av1MotionVector[8]; + int[] weights = new int[8]; + + Av1MotionVector actual = Av1IntraBlockCopy.FindReference( + ref partitionInfo, + tileInfo, + sequenceHeader.SuperblockModeInfoSize, + candidates, + weights); + + Assert.Equal(new Av1MotionVector(-512, 0), actual); + } + + /// + /// Verifies that encoder mode aliases and packed displacement storage feed the shared spatial ranking. + /// + [Fact] + public void EncoderReferenceUsesMappedIntraBlockCopyNeighbor() + { + const int Width = 640; + const int Height = 256; + ObuSequenceHeader sequenceHeader = CreateSequenceHeader(); + ObuFrameHeader frameHeader = CreateFrameHeader(); + frameHeader.AllowScreenContentTools = true; + frameHeader.AllowIntraBlockCopy = true; + + using Av1EncoderPictureBuffer buffer = new( + Configuration.Default, + sequenceHeader, + frameHeader, + Width, + Height, + disallow4x4AllFrames: true); + + Av1PictureControlSet picture = buffer.Picture; + Point candidatePosition = new(80, 12); + ref Av1MacroBlockModeInfo candidate = ref picture.GetMacroBlockModeInfo(candidatePosition); + candidate.Block = new Av1EncoderBlockModeInfo + { + BlockSize = Av1BlockSize.Block16x16, + PartitionType = Av1PartitionType.None, + UseIntraBlockCopy = true + }; + + Av1MotionVector displacement = new(0, -2560); + picture.MapModeInfoBlock(candidatePosition, candidate.Block.BlockSize); + picture.SetDisplacementVector(candidatePosition, displacement); + + Point currentPosition = new(80, 16); + Av1TileInfo tileInfo = new(0, 0, frameHeader); + Av1MacroBlockD macroBlock = new() { Tile = tileInfo }; + Av1TileWriter.SetModeInfoRowAndColumn( + picture, + macroBlock, + tileInfo, + currentPosition, + Av1BlockSize.Block16x16, + picture.ModeInfoStride, + picture.Parent.Common.ModeInfoRowCount, + picture.Parent.Common.ModeInfoColumnCount); + + Span candidates = stackalloc Av1MotionVector[8]; + Span weights = stackalloc int[8]; + Av1MotionVector actual = Av1IntraBlockCopy.FindReference( + picture, + macroBlock, + currentPosition, + Av1BlockSize.Block16x16, + Av1PartitionType.None, + candidates, + weights); + + Assert.Equal(displacement, actual); + } + + /// + /// Verifies that the tile writer derives the encoder reference and emits the retained displacement. + /// + [Fact] + public void TileWriterEmitsRetainedDisplacementAgainstDerivedReference() + { + const int Width = 640; + const int Height = 256; + ObuSequenceHeader sequenceHeader = CreateSequenceHeader(); + ObuFrameHeader frameHeader = CreateFrameHeader(); + frameHeader.AllowScreenContentTools = true; + frameHeader.AllowIntraBlockCopy = true; + + using Av1EncoderPictureBuffer buffer = new( + Configuration.Default, + sequenceHeader, + frameHeader, + Width, + Height, + disallow4x4AllFrames: true); + + Av1PictureControlSet picture = buffer.Picture; + Point modeInfoPosition = new(80, 0); + Av1MacroBlockModeInfo modeInfo = default; + modeInfo.Block = new Av1EncoderBlockModeInfo + { + BlockSize = Av1BlockSize.Block16x16, + PartitionType = Av1PartitionType.None, + UseIntraBlockCopy = true + }; + + Av1MotionVector displacement = new(0, -3072); + picture.MapModeInfoBlock(modeInfoPosition, modeInfo.Block.BlockSize); + picture.SetDisplacementVector(modeInfoPosition, displacement); + Av1TileInfo tileInfo = new(0, 0, frameHeader); + Av1MacroBlockD macroBlock = new() { Tile = tileInfo }; + Av1TileWriter.SetModeInfoRowAndColumn( + picture, + macroBlock, + tileInfo, + modeInfoPosition, + modeInfo.Block.BlockSize, + picture.ModeInfoStride, + picture.Parent.Common.ModeInfoRowCount, + picture.Parent.Common.ModeInfoColumnCount); + + using Av1SymbolEncoder writer = new(Configuration.Default, 64, 0, updateCdf: true); + Av1TileWriter.WriteIntraBlockCopyInfo( + picture, + writer, + macroBlock, + modeInfoPosition, + modeInfo); + + using var encoded = writer.Exit(); + Av1SymbolDecoder decoder = new(Configuration.Default, encoded.GetSpan(), 0); + Assert.True(decoder.ReadUseIntraBlockCopy()); + Assert.Equal( + displacement, + decoder.ReadDisplacementVector(new Av1MotionVector(0, -2560))); + } + + /// + /// Verifies tile bounds, whole-sample precision, the four-block delay, and wavefront ordering. + /// + [Fact] + public void IsValidEnforcesIntraBlockCopySourceRestrictions() + { + ObuSequenceHeader sequenceHeader = CreateSequenceHeader(); + using Av1FrameInfo frameInfo = new(sequenceHeader); + Av1SuperblockInfo superblockInfo = frameInfo.GetSuperblock(new Point(8, 2)); + Av1BlockModeInfo modeInfo = new(Av1BlockSize.Block16x16, Point.Empty); + Av1PartitionInfo partitionInfo = new(modeInfo, superblockInfo, true, Av1PartitionType.None) + { + ColumnIndex = 128, + RowIndex = 32, + }; + + Av1TileInfo tileInfo = CreateTileInfo(); + + // A source five 64-sample columns earlier satisfies both the four-column delay and same-row wavefront limit. + Assert.True(Av1IntraBlockCopy.IsValid(new Av1MotionVector(0, -2560), ref partitionInfo, tileInfo, sequenceHeader)); + + // Moving the source one 64-sample column to the right reaches the forbidden delay boundary exactly. + Assert.False(Av1IntraBlockCopy.IsValid(new Av1MotionVector(0, -2048), ref partitionInfo, tileInfo, sequenceHeader)); + Assert.False(Av1IntraBlockCopy.IsValid(new Av1MotionVector(0, -2559), ref partitionInfo, tileInfo, sequenceHeader)); + Assert.False(Av1IntraBlockCopy.IsValid(new Av1MotionVector(0, -4608), ref partitionInfo, tileInfo, sequenceHeader)); + Assert.False(Av1IntraBlockCopy.IsValid(new Av1MotionVector(512, -2560), ref partitionInfo, tileInfo, sequenceHeader)); + } + + /// + /// Verifies exact hash matches at unaligned origins in both normative search regions. + /// + [Fact] + public void SearchIndexFindsUnalignedAboveAndLeftMatches() + { + const int Width = 640; + const int Height = 256; + const int QIndex = 23; + Point blockOrigin = new(512, 128); + Point aboveOrigin = new(515, 57); + Point leftOrigin = new(191, 131); + ObuSequenceHeader sequenceHeader = CreateSequenceHeader(); + ObuFrameHeader frameHeader = CreateFrameHeader(); + frameHeader.AllowScreenContentTools = true; + frameHeader.AllowIntraBlockCopy = true; + + using Av1EncoderPictureBuffer pictureBuffer = new( + Configuration.Default, + sequenceHeader, + frameHeader, + Width, + Height, + disallow4x4AllFrames: true); + + using Av1EncoderFrameBuffer source = new( + Configuration.Default, + Width, + Height, + 8, + Av1ColorFormat.Yuv400, + 0, + 0); + + using Av1EncoderFrameBuffer reconstruction = new( + Configuration.Default, + Width, + Height, + 8, + Av1ColorFormat.Yuv400, + 0, + 0); + + Buffer2DRegion sourceLuma = source.Frame.View.GetPlane(Av1Plane.Y); + Buffer2DRegion reconstructionLuma = reconstruction.Frame.View.GetPlane(Av1Plane.Y); + uint randomState = 0x8F3A21C5; + for (int row = 0; row < Height; row++) + { + Span sourceRow = sourceLuma.DangerousGetRowSpan(row); + reconstructionLuma.DangerousGetRowSpan(row).Clear(); + for (int column = 0; column < Width; column++) + { + randomState = unchecked((randomState * 1_664_525) + 1_013_904_223); + sourceRow[column] = (byte)(randomState >> 24); + } + } + + for (int row = 0; row < 8; row++) + { + ReadOnlySpan blockRow = sourceLuma.DangerousGetRowSpan(blockOrigin.Y + row).Slice(blockOrigin.X, 8); + blockRow.CopyTo(sourceLuma.DangerousGetRowSpan(aboveOrigin.Y + row)[aboveOrigin.X..]); + blockRow.CopyTo(sourceLuma.DangerousGetRowSpan(leftOrigin.Y + row)[leftOrigin.X..]); + blockRow.CopyTo(reconstructionLuma.DangerousGetRowSpan(aboveOrigin.Y + row)[aboveOrigin.X..]); + blockRow.CopyTo(reconstructionLuma.DangerousGetRowSpan(leftOrigin.Y + row)[leftOrigin.X..]); + } + + Av1PictureControlSet picture = pictureBuffer.Picture; + picture.IntraBlockCopySearch.Initialize(sourceLuma); + using Av1SymbolEncoder writer = new(Configuration.Default, 64, QIndex, updateCdf: true); + Span candidates = stackalloc Av1MotionVector[2]; + Av1MotionVector reference = new(0, -2560); + int candidateCount = picture.IntraBlockCopySearch.FindCandidates( + source.Frame.CodedView.GetPlane(Av1Plane.Y), + reconstruction.Frame.CodedView.GetPlane(Av1Plane.Y), + blockOrigin, + new Av1TileInfo(0, 0, frameHeader), + sequenceHeader, + writer, + reference, + Av1RateDistortion.GetKeyFrameRateMultiplier(QIndex, Av1BitDepth.EightBit), + candidates); + + Assert.Equal(2, candidateCount); + Assert.Equal(new Av1MotionVector(-568, 24), candidates[0]); + Assert.Equal(new Av1MotionVector(24, -2568), candidates[1]); + } + + /// + /// Verifies that NSTEP pixel search reaches an unaligned reconstructed match which has no exact source hash match. + /// + [Fact] + public void PixelSearchFindsUnalignedNonHashMatch() + { + const int Width = 640; + const int Height = 256; + const int QIndex = 23; + Point blockOrigin = new(0, 128); + Point predictionOrigin = new(15, 80); + ObuSequenceHeader sequenceHeader = CreateSequenceHeader(); + ObuFrameHeader frameHeader = CreateFrameHeader(); + frameHeader.AllowScreenContentTools = true; + frameHeader.AllowIntraBlockCopy = true; + + using Av1EncoderPictureBuffer pictureBuffer = new( + Configuration.Default, + sequenceHeader, + frameHeader, + Width, + Height, + disallow4x4AllFrames: true); + + using Av1EncoderFrameBuffer source = new( + Configuration.Default, + Width, + Height, + 8, + Av1ColorFormat.Yuv400, + 0, + 0); + + using Av1EncoderFrameBuffer reconstruction = new( + Configuration.Default, + Width, + Height, + 8, + Av1ColorFormat.Yuv400, + 0, + 0); + + Buffer2DRegion sourceLuma = source.Frame.View.GetPlane(Av1Plane.Y); + Buffer2DRegion reconstructionLuma = reconstruction.Frame.View.GetPlane(Av1Plane.Y); + for (int row = 0; row < Height; row++) + { + sourceLuma.DangerousGetRowSpan(row).Clear(); + reconstructionLuma.DangerousGetRowSpan(row).Clear(); + } + + for (int row = 0; row < 8; row++) + { + byte value = (byte)(100 + (row * 10)); + sourceLuma.DangerousGetRowSpan(blockOrigin.Y + row).Slice(blockOrigin.X, 8).Fill(value); + reconstructionLuma.DangerousGetRowSpan(predictionOrigin.Y + row).Slice(predictionOrigin.X, 8).Fill(value); + } + + Buffer2DRegion codedSourceLuma = source.Frame.CodedView.GetPlane(Av1Plane.Y); + Buffer2DRegion codedReconstructionLuma = reconstruction.Frame.CodedView.GetPlane(Av1Plane.Y); + Assert.False(Av1IntraSuperblockEncoder.ByteOperator.BlocksEqual(codedSourceLuma, blockOrigin, predictionOrigin)); + Assert.Equal( + 0, + Av1IntraSuperblockEncoder.ByteOperator.GetSumOfAbsoluteDifferences( + codedSourceLuma, + blockOrigin, + codedReconstructionLuma, + predictionOrigin)); + + Assert.Equal( + 8_640, + Av1IntraSuperblockEncoder.ByteOperator.GetSumOfAbsoluteDifferences( + codedSourceLuma, + blockOrigin, + codedReconstructionLuma, + new Point(15, 120))); + + using Av1SymbolEncoder writer = new(Configuration.Default, 64, QIndex, updateCdf: true); + Span candidates = stackalloc Av1MotionVector[2]; + int candidateCount = pictureBuffer.Picture.IntraBlockCopySearch + .FindPixelCandidates( + codedSourceLuma, + codedReconstructionLuma, + blockOrigin, + new Av1TileInfo(0, 0, frameHeader), + sequenceHeader, + writer, + new Av1MotionVector(-64, 120), + QIndex, + Av1RateDistortion.GetKeyFrameRateMultiplier(QIndex, Av1BitDepth.EightBit), + candidates); + + Assert.Equal(1, candidateCount); + Assert.Equal(-384, candidates[0].Row); + Assert.Equal(120, candidates[0].Column); + } + + /// + /// Verifies that the exhaustive mesh recovers an exact match outside every centered NSTEP search site. + /// + [Fact] + public void PixelSearchFallsBackToExhaustiveMesh() + { + const int Width = 640; + const int Height = 256; + const int QIndex = 23; + Point blockOrigin = new(0, 128); + Point predictionOrigin = new(256, 8); + ObuSequenceHeader sequenceHeader = CreateSequenceHeader(); + ObuFrameHeader frameHeader = CreateFrameHeader(); + frameHeader.AllowScreenContentTools = true; + frameHeader.AllowIntraBlockCopy = true; + + using Av1EncoderPictureBuffer pictureBuffer = new( + Configuration.Default, + sequenceHeader, + frameHeader, + Width, + Height, + disallow4x4AllFrames: true); + + using Av1EncoderFrameBuffer source = new( + Configuration.Default, + Width, + Height, + 8, + Av1ColorFormat.Yuv400, + 0, + 0); + + using Av1EncoderFrameBuffer reconstruction = new( + Configuration.Default, + Width, + Height, + 8, + Av1ColorFormat.Yuv400, + 0, + 0); + + Buffer2DRegion sourceLuma = source.Frame.View.GetPlane(Av1Plane.Y); + Buffer2DRegion reconstructionLuma = reconstruction.Frame.View.GetPlane(Av1Plane.Y); + for (int row = 0; row < Height; row++) + { + sourceLuma.DangerousGetRowSpan(row).Clear(); + reconstructionLuma.DangerousGetRowSpan(row).Clear(); + } + + for (int row = 0; row < 8; row++) + { + Span sourceRow = sourceLuma.DangerousGetRowSpan(blockOrigin.Y + row).Slice(blockOrigin.X, 8); + Span predictionRow = reconstructionLuma.DangerousGetRowSpan(predictionOrigin.Y + row).Slice(predictionOrigin.X, 8); + for (int column = 0; column < 8; column++) + { + byte value = (byte)((row + column) % 2 == 0 ? 255 : 0); + sourceRow[column] = value; + predictionRow[column] = value; + } + } + + Buffer2DRegion codedSourceLuma = source.Frame.CodedView.GetPlane(Av1Plane.Y); + Buffer2DRegion codedReconstructionLuma = reconstruction.Frame.CodedView.GetPlane(Av1Plane.Y); + using Av1SymbolEncoder writer = new(Configuration.Default, 64, QIndex, updateCdf: true); + Span candidates = stackalloc Av1MotionVector[2]; + int candidateCount = pictureBuffer.Picture.IntraBlockCopySearch + .FindPixelCandidates( + codedSourceLuma, + codedReconstructionLuma, + blockOrigin, + new Av1TileInfo(0, 0, frameHeader), + sequenceHeader, + writer, + new Av1MotionVector(-64, 0), + QIndex, + Av1RateDistortion.GetKeyFrameRateMultiplier(QIndex, Av1BitDepth.EightBit), + candidates); + + Assert.Equal(1, candidateCount); + Assert.Equal(-960, candidates[0].Row); + Assert.Equal(2048, candidates[0].Column); + } + + /// + /// Verifies high-bit-depth SIMD variance normalization against the eight-bit search domain. + /// + [Fact] + public void SearchVarianceMatchesTwelveBitReference() + { + const int Width = 11; + using Av1EncoderFrameBuffer source = new( + Configuration.Default, + Width, + 8, + 12, + Av1ColorFormat.Yuv400, + 0, + 0); + + using Av1EncoderFrameBuffer reconstruction = new( + Configuration.Default, + Width, + 8, + 12, + Av1ColorFormat.Yuv400, + 0, + 0); + + Buffer2DRegion sourceLuma = source.Frame.View.GetPlane(Av1Plane.Y); + Buffer2DRegion reconstructionLuma = reconstruction.Frame.View.GetPlane(Av1Plane.Y); + for (int row = 0; row < 8; row++) + { + Span sourceRow = sourceLuma.DangerousGetRowSpan(row); + Span reconstructionRow = reconstructionLuma.DangerousGetRowSpan(row); + for (int column = 0; column < 8; column++) + { + sourceRow[column] = 1000; + reconstructionRow[column] = (ushort)(1000 + (((row * 8) + column) % 2 == 0 ? 17 : 33)); + } + } + + int sumOfAbsoluteDifferences = Av1IntraSuperblockEncoder.UInt16Operator.GetSumOfAbsoluteDifferences( + sourceLuma, + Point.Empty, + reconstructionLuma, + Point.Empty); + + int variance = Av1IntraSuperblockEncoder.UInt16Operator.GetVariance( + sourceLuma, + Point.Empty, + reconstructionLuma, + Point.Empty, + Av1BitDepth.TwelveBit); + + Span fourSumsOfAbsoluteDifferences = stackalloc int[4]; + Av1IntraSuperblockEncoder.UInt16Operator.GetFourSumsOfAbsoluteDifferences( + sourceLuma, + Point.Empty, + reconstructionLuma, + Point.Empty, + fourSumsOfAbsoluteDifferences); + + for (int candidate = 0; candidate < fourSumsOfAbsoluteDifferences.Length; candidate++) + { + int expected = Av1IntraSuperblockEncoder.UInt16Operator.GetSumOfAbsoluteDifferences( + sourceLuma, + Point.Empty, + reconstructionLuma, + new Point(candidate, 0)); + + Assert.Equal(expected, fourSumsOfAbsoluteDifferences[candidate]); + } + + Assert.Equal(1600, sumOfAbsoluteDifferences); + Assert.Equal(16, variance); + } + + /// + /// Creates the 640-by-256, 4:2:0 sequence geometry shared by the displacement tests. + /// + private static ObuSequenceHeader CreateSequenceHeader() + => new() + { + MaxFrameWidth = 640, + MaxFrameHeight = 256, + Use128x128Superblock = false, + ColorConfig = new ObuColorConfig + { + IsMonochrome = false, + SubSamplingX = true, + SubSamplingY = true, + BitDepth = Av1BitDepth.EightBit, + }, + }; + + /// + /// Creates one tile covering the complete test frame. + /// + private static Av1TileInfo CreateTileInfo() + => new(0, 0, CreateFrameHeader()); + + /// + /// Creates the frame and tile geometry shared by reference and validity tests. + /// + private static ObuFrameHeader CreateFrameHeader() + { + ObuTileGroupHeader tilesInfo = new() + { + TileColumnCount = 1, + TileRowCount = 1, + }; + + tilesInfo.TileColumnStartModeInfo[1] = 160; + tilesInfo.TileRowStartModeInfo[1] = 64; + + return new ObuFrameHeader + { + ModeInfoColumnCount = 160, + ModeInfoRowCount = 64, + TilesInfo = tilesInfo, + }; + } +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1IntraSuperblockEncoderTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1IntraSuperblockEncoderTests.cs new file mode 100644 index 0000000000..36f25efbb1 --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1IntraSuperblockEncoderTests.cs @@ -0,0 +1,3952 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Buffers; +using System.Numerics; +using System.Runtime.InteropServices; +using SixLabors.ImageSharp.Formats.Heif.Av1; +using SixLabors.ImageSharp.Formats.Heif.Av1.Entropy; +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline; +using SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.Quantizers; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.ChromaFromLuma; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; +using SixLabors.ImageSharp.Memory; +using SixLabors.ImageSharp.PixelFormats; +using SixLabors.ImageSharp.Tests.Memory; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1; + +/// +/// Verifies live intra superblock mode decisions, traversal, and reconstruction. +/// +[Trait("Format", "Avif")] +public class Av1IntraSuperblockEncoderTests +{ + /// + /// Verifies non-regular filter selection, retained reconstruction, and allocation-free inter tile coding. + /// + [Theory] + [InlineData((int)Av1InterpolationFilter.Smooth, false)] + [InlineData((int)Av1InterpolationFilter.Sharp, false)] + [InlineData((int)Av1InterpolationFilter.Smooth, true)] + [InlineData((int)Av1InterpolationFilter.Sharp, true)] + public void ProductionTileSelectsNonRegularInterpolation(int filterValue, bool dualFilter) + { + const int Width = 32; + const int Height = 8; + const int TargetColumn = 8; + const int BlockWidth = 8; + const int QIndex = 37; + const int TileBufferLength = 4096; + Av1InterpolationFilter filter = (Av1InterpolationFilter)filterValue; + int effort = dualFilter ? 9 : 8; + ReadOnlySpan referencePeriod = [128, 184, 208, 184, 128, 72, 48, 72]; + + // These are fixed half-sample responses of the reference's eight-tap smooth and sharp kernels. + // The horizontal pass rounds first by three bits and then by four; edge samples are replicated. + // Keeping the results literal avoids using the predictor under test to manufacture its own target. + ReadOnlySpan targetRow = filter == Av1InterpolationFilter.Smooth + ? [159, 189, 190, 154, 102, 66, 66, 102, 154, 190, 190, 154, 102, 66, 66, 102, + 154, 190, 190, 154, 102, 66, 66, 102, 154, 190, 190, 154, 102, 67, 61, 69] + : [153, 204, 200, 158, 98, 55, 55, 98, 158, 202, 202, 158, 98, 55, 55, 98, + 158, 202, 202, 158, 98, 55, 55, 98, 158, 202, 202, 158, 100, 53, 59, 75]; + + TestMemoryAllocator allocator = new(); + allocator.EnableNonThreadSafeLogging(); + Configuration configuration = Configuration.Default.Clone(); + configuration.MemoryAllocator = allocator; + ObuColorConfig colorConfig = new() + { + IsMonochrome = true, + ColorRange = true, + SubSamplingX = true, + SubSamplingY = true, + BitDepth = Av1BitDepth.EightBit + }; + + using Image referenceImage = new(Width, Height); + using Av1EncoderFrameBuffer reference = new(configuration, Width, Height, 8, Av1ColorFormat.Yuv400, 0, 0); + using Av1EncoderFrameBuffer source = new(configuration, Width, Height, 8, Av1ColorFormat.Yuv400, 0, 0); + using Av1EncoderFrameBuffer reconstruction = new(configuration, Width, Height, 8, Av1ColorFormat.Yuv400, 0, 0); + for (int y = 0; y < Height; y++) + { + Span pixels = referenceImage.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(y); + Span referenceRow = reference.Frame.CodedView.GetPlane(Av1Plane.Y).DangerousGetRowSpan(y); + for (int x = 0; x < Width; x++) + { + byte sample = referencePeriod[x % referencePeriod.Length]; + referenceRow[x] = sample; + pixels[x] = new L8(sample); + } + + targetRow.CopyTo(source.Frame.CodedView.GetPlane(Av1Plane.Y).DangerousGetRowSpan(y)); + } + + reference.Frame.ExtendBorders(); + source.Frame.ExtendBorders(); + ClearPlane(reconstruction.Luma); + + // A lossless key frame gives an independent decoder exactly the reference samples used by tile search. + using MemoryStream firstSample = new(); + using Av1FrameEncoder.SequenceEncoder keyEncoder = Av1FrameEncoder.CreateColorSequenceEncoder( + configuration, + Width, + Height, + colorConfig, + qIndex: 0, + effort); + + keyEncoder.EncodeKeyFrame(referenceImage.Frames.RootFrame, firstSample); + ObuSequenceHeader sequenceHeader = keyEncoder.SequenceHeader; + using Av1EncoderModeInfoBuffer modeInfo = new(configuration, Width, Height, disallow4x4AllFrames: true); + Av1PictureControlSet template = CreatePicture(modeInfo, colorConfig, use128x128Superblock: false, QIndex); + ObuFrameHeader frameHeader = template.Parent.FrameHeader; + frameHeader.FrameType = ObuFrameType.InterFrame; + frameHeader.ShowFrame = true; + frameHeader.ErrorResilientMode = true; + frameHeader.RefreshFrameFlags = byte.MaxValue; + frameHeader.DisableFrameEndUpdateCdf = true; + frameHeader.ReferenceMode = ObuReferenceMode.SingleReference; + frameHeader.InterpolationFilter = Av1InterpolationFilter.Switchable; + frameHeader.AllowHighPrecisionMotionVector = true; + frameHeader.TransformMode = Av1TransformMode.Select; + frameHeader.FrameSize.FrameWidth = Width; + frameHeader.FrameSize.FrameHeight = Height; + frameHeader.FrameSize.SuperResolutionUpscaledWidth = Width; + frameHeader.FrameSize.RenderWidth = Width; + frameHeader.FrameSize.RenderHeight = Height; + frameHeader.TilesInfo.HasUniformTileSpacing = true; + Av1QuantizationLookup.UpdateFrameQuantizationState(frameHeader); + + using Av1EncoderPictureBuffer picture = new(configuration, sequenceHeader, frameHeader, Width, Height, disallow4x4AllFrames: true); + using Av1EncoderCoefficientBuffer coefficients = new(configuration, sequenceHeader, Width, Height); + using Av1EncoderSuperblockWorkspace superblockWorkspace = new(configuration); + using Av1EncoderBlockWorkspace blockWorkspace = new(configuration); + using Av1SymbolEncoder symbolEncoder = new(configuration, TileBufferLength, QIndex, updateCdf: true); + Av1EncoderTileWorkspace tileWorkspace = new(frameHeader, superblockWorkspace); + int allocationCount = allocator.AllocationLog.Count; + Av1TileEncoder tileWriter = new( + symbolEncoder, + source.Frame, + reference.Frame, + reconstruction.Frame, + picture.Picture, + coefficients, + tileWorkspace, + blockWorkspace, + effort); + + Assert.Equal(allocationCount, allocator.AllocationLog.Count); + Point targetPosition = new(TargetColumn >> Av1Constants.ModeInfoSizeLog2, 0); + ref Av1MacroBlockModeInfo targetMode = ref picture.Picture.GetMacroBlockModeInfo(targetPosition); + Assert.Equal(Av1ReferenceFrameType.Last, targetMode.Block.ReferenceFrame); + Assert.Equal(filter, targetMode.Block.HorizontalInterpolationFilter); + Assert.Equal(dualFilter ? Av1InterpolationFilter.Regular : filter, targetMode.Block.VerticalInterpolationFilter); + Assert.Equal(4, picture.Picture.GetDisplacementVector(targetPosition).Column); + Assert.Equal(0, picture.Picture.GetDisplacementVector(targetPosition).Row); + for (int y = 0; y < Height; y++) + { + Assert.Equal( + targetRow.Slice(TargetColumn, BlockWidth), + reconstruction.Frame.View.GetPlane(Av1Plane.Y).DangerousGetRowSpan(y).Slice(TargetColumn, BlockWidth)); + } + + using MemoryStream secondSample = new(); + using ObuWriter obuWriter = new(configuration); + obuWriter.WriteFrame(secondSample, sequenceHeader, frameHeader, tileWriter); + using Av1Decoder decoder = new(configuration); + + // Sequence decoding retains the first frame's reference slots. The still-image transfer API deliberately + // releases those slots, so it cannot be used between dependent samples. Full-range monochrome L8 is exact. + using ImageFrame decodedFirst = decoder.DecodeSequenceFrame(firstSample.ToArray(), null, null); + using ImageFrame decodedSecond = decoder.DecodeSequenceFrame(secondSample.ToArray(), null, null); + + // Preserve both the production stream and every managed reconstructed luma sample for exact libaom comparison. + string outputDirectory = TestEnvironment.CreateOutputDirectory("Heif", "Av1", nameof(this.ProductionTileSelectsNonRegularInterpolation)); + string outputName = $"{filter}-{dualFilter}"; + using FileStream output = File.Create(Path.Combine(outputDirectory, outputName + ".obu")); + firstSample.Position = 0; + firstSample.CopyTo(output); + secondSample.Position = 0; + secondSample.CopyTo(output); + using FileStream rawOutput = File.Create(Path.Combine(outputDirectory, outputName + ".managed.yuv")); + for (int y = 0; y < Height; y++) + { + ReadOnlySpan expected = reference.Frame.View.GetPlane(Av1Plane.Y).DangerousGetRowSpan(y); + ReadOnlySpan actual = MemoryMarshal.AsBytes(decodedFirst.PixelBuffer.DangerousGetRowSpan(y)); + Assert.Equal(expected, actual); + rawOutput.Write(actual); + } + + for (int y = 0; y < Height; y++) + { + ReadOnlySpan expected = reconstruction.Frame.View.GetPlane(Av1Plane.Y).DangerousGetRowSpan(y); + ReadOnlySpan actual = MemoryMarshal.AsBytes(decodedSecond.PixelBuffer.DangerousGetRowSpan(y)); + Assert.Equal(expected, actual); + rawOutput.Write(actual); + } + } + + /// + /// Verifies independent half-sample filters on both axes without discarding native sample precision. + /// + [Theory] + [InlineData(10, false)] + [InlineData(10, true)] + [InlineData(12, false)] + [InlineData(12, true)] + public void ProductionTileSelectsDualAxisInterpolationHighBitDepth(int bitDepth, bool reverseFilters) + { + const int Width = 48; + const int Height = 24; + const int TargetColumn = 16; + const int TargetRow = 8; + const int BlockSize = 8; + const int QIndex = 1; + const int Effort = 9; + const int TileBufferLength = 8192; + const int FilterScale = 128; + int sampleScale = 1 << (bitDepth - 8); + int maximumSample = (1 << bitDepth) - 1; + Av1InterpolationFilter horizontalFilter = reverseFilters ? Av1InterpolationFilter.Sharp : Av1InterpolationFilter.Smooth; + Av1InterpolationFilter verticalFilter = reverseFilters ? Av1InterpolationFilter.Smooth : Av1InterpolationFilter.Sharp; + ReadOnlySpan referencePeriod = [0, 28, 40, 28, 0, -28, -40, -12]; + + // These Q7 sums are the fixed half-sample responses of the periodic reference to libaom's eight-tap + // kernels. The source is separable: 128 + horizontal period + vertical period. Each horizontal sum + // is divisible by the first-pass rounding unit (including the five-bit shift at 12 bits), so the + // two-axis result is the sum of these responses with one final Q7 rounding, not two rounded pixels. + // The asymmetric final phase separates sharp from regular after eight-bit error normalization. A low + // quantizer makes retaining the exact two-axis predictor preferable to saving a filter symbol. + ReadOnlySpan smoothResponse = [1872, 3952, 3984, 1648, -1680, -3760, -3152, -816]; + ReadOnlySpan sharpResponse = [1536, 4896, 4640, 1856, -1728, -5088, -3424, -640]; + ReadOnlySpan horizontalResponse = reverseFilters ? sharpResponse : smoothResponse; + ReadOnlySpan verticalResponse = reverseFilters ? smoothResponse : sharpResponse; + TestMemoryAllocator allocator = new(); + allocator.EnableNonThreadSafeLogging(); + Configuration configuration = Configuration.Default.Clone(); + configuration.MemoryAllocator = allocator; + ObuColorConfig colorConfig = new() + { + IsMonochrome = true, + ColorRange = true, + SubSamplingX = true, + SubSamplingY = true, + BitDepth = (Av1BitDepth)((bitDepth - 8) / 2) + }; + + using Image referenceImage = new(Width, Height); + using Av1EncoderFrameBuffer reference = new(configuration, Width, Height, bitDepth, Av1ColorFormat.Yuv400, 0, 0); + using Av1EncoderFrameBuffer source = new(configuration, Width, Height, bitDepth, Av1ColorFormat.Yuv400, 0, 0); + using Av1EncoderFrameBuffer reconstruction = new(configuration, Width, Height, bitDepth, Av1ColorFormat.Yuv400, 0, 0); + for (int y = 0; y < Height; y++) + { + Span pixels = referenceImage.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(y); + Span referenceRow = reference.Frame.CodedView.GetPlane(Av1Plane.Y).DangerousGetRowSpan(y); + Span sourceRow = source.Frame.CodedView.GetPlane(Av1Plane.Y).DangerousGetRowSpan(y); + for (int x = 0; x < Width; x++) + { + int sample = (128 + referencePeriod[x % BlockSize] + referencePeriod[y % BlockSize]) * sampleScale; + referenceRow[x] = (ushort)sample; + + // Map native samples to L16's complete range. The lossless key-frame comparison below proves + // that the public pixel conversion recovers every original 10/12-bit reference sample. + pixels[x] = new L16((ushort)(((sample * ushort.MaxValue) + (maximumSample / 2)) / maximumSample)); + int response = (128 * FilterScale) + horizontalResponse[x % BlockSize] + verticalResponse[y % BlockSize]; + sourceRow[x] = (ushort)(((response * sampleScale) + (FilterScale / 2)) / FilterScale); + } + } + + reference.Frame.ExtendBorders(); + source.Frame.ExtendBorders(); + ClearPlane(reconstruction.Luma); + using MemoryStream firstSample = new(); + using Av1FrameEncoder.SequenceEncoder keyEncoder = Av1FrameEncoder.CreateColorSequenceEncoder( + configuration, Width, Height, colorConfig, qIndex: 0, Effort); + + keyEncoder.EncodeKeyFrame(referenceImage.Frames.RootFrame, firstSample); + ObuSequenceHeader sequenceHeader = keyEncoder.SequenceHeader; + using Av1EncoderModeInfoBuffer modeInfo = new(configuration, Width, Height, disallow4x4AllFrames: true); + Av1PictureControlSet template = CreatePicture(modeInfo, colorConfig, use128x128Superblock: false, QIndex); + ObuFrameHeader frameHeader = template.Parent.FrameHeader; + frameHeader.FrameType = ObuFrameType.InterFrame; + frameHeader.ShowFrame = true; + frameHeader.ErrorResilientMode = true; + frameHeader.RefreshFrameFlags = byte.MaxValue; + frameHeader.DisableFrameEndUpdateCdf = true; + frameHeader.ReferenceMode = ObuReferenceMode.SingleReference; + frameHeader.InterpolationFilter = Av1InterpolationFilter.Switchable; + frameHeader.AllowHighPrecisionMotionVector = true; + frameHeader.TransformMode = Av1TransformMode.Select; + frameHeader.FrameSize.FrameWidth = Width; + frameHeader.FrameSize.FrameHeight = Height; + frameHeader.FrameSize.SuperResolutionUpscaledWidth = Width; + frameHeader.FrameSize.RenderWidth = Width; + frameHeader.FrameSize.RenderHeight = Height; + frameHeader.TilesInfo.HasUniformTileSpacing = true; + Av1QuantizationLookup.UpdateFrameQuantizationState(frameHeader); + + using Av1EncoderPictureBuffer picture = new(configuration, sequenceHeader, frameHeader, Width, Height, disallow4x4AllFrames: true); + using Av1EncoderCoefficientBuffer coefficients = new(configuration, sequenceHeader, Width, Height); + using Av1EncoderSuperblockWorkspace superblockWorkspace = new(configuration); + using Av1EncoderBlockWorkspace blockWorkspace = new(configuration); + using Av1SymbolEncoder symbolEncoder = new(configuration, TileBufferLength, QIndex, updateCdf: true); + Av1EncoderTileWorkspace tileWorkspace = new(frameHeader, superblockWorkspace); + int allocationCount = allocator.AllocationLog.Count; + Av1TileEncoder tileWriter = new( + symbolEncoder, source.Frame, reference.Frame, reconstruction.Frame, picture.Picture, coefficients, tileWorkspace, blockWorkspace, Effort); + + Assert.Equal(allocationCount, allocator.AllocationLog.Count); + + // This block is at least three reference taps from every frame edge. It must retain two genuinely + // fractional axes, not a zero-phase filter alias. + Point targetPosition = new(TargetColumn >> Av1Constants.ModeInfoSizeLog2, TargetRow >> Av1Constants.ModeInfoSizeLog2); + ref Av1MacroBlockModeInfo targetMode = ref picture.Picture.GetMacroBlockModeInfo(targetPosition); + Assert.Equal(Av1ReferenceFrameType.Last, targetMode.Block.ReferenceFrame); + Assert.Equal(horizontalFilter, targetMode.Block.HorizontalInterpolationFilter); + Assert.Equal(verticalFilter, targetMode.Block.VerticalInterpolationFilter); + Assert.Equal(4, picture.Picture.GetDisplacementVector(targetPosition).Column); + Assert.Equal(4, picture.Picture.GetDisplacementVector(targetPosition).Row); + for (int y = TargetRow; y < TargetRow + BlockSize; y++) + { + Assert.Equal( + source.Frame.View.GetPlane(Av1Plane.Y).DangerousGetRowSpan(y).Slice(TargetColumn, BlockSize), + reconstruction.Frame.View.GetPlane(Av1Plane.Y).DangerousGetRowSpan(y).Slice(TargetColumn, BlockSize)); + } + + using MemoryStream secondSample = new(); + using ObuWriter obuWriter = new(configuration); + obuWriter.WriteFrame(secondSample, sequenceHeader, frameHeader, tileWriter); + string outputDirectory = TestEnvironment.CreateOutputDirectory("Heif", "Av1", nameof(this.ProductionTileSelectsDualAxisInterpolationHighBitDepth)); + string outputName = $"{bitDepth}-{horizontalFilter}-{verticalFilter}"; + using FileStream output = File.Create(Path.Combine(outputDirectory, outputName + ".obu")); + firstSample.Position = 0; + firstSample.CopyTo(output); + secondSample.Position = 0; + secondSample.CopyTo(output); + using BinaryWriter rawOutput = new(File.Create(Path.Combine(outputDirectory, outputName + ".managed.yuv"))); + using Av1Decoder decoder = new(configuration); + for (int frameIndex = 0; frameIndex < 2; frameIndex++) + { + // Consume native retained planes before the next sample can replace them. BinaryWriter emits explicit + // little-endian UInt16 samples, matching the raw reference-decoder output independently of host byte order. + decoder.DecodeSequenceReference((frameIndex == 0 ? firstSample : secondSample).ToArray(), null, null); + Av1FrameBuffer decoded = Assert.IsType>(decoder.FrameBuffer); + Buffer2DRegion expected = (frameIndex == 0 ? reference : reconstruction).Frame.View.GetPlane(Av1Plane.Y); + for (int y = 0; y < Height; y++) + { + ReadOnlySpan actualRow = decoded.GetHighBitDepthRowSpan(Av1Plane.Y, y, 0, 0); + Assert.Equal(expected.DangerousGetRowSpan(y), actualRow); + foreach (ushort sample in actualRow) + { + rawOutput.Write(sample); + } + } + } + } + + /// + /// Gets the normative eight-sample weights used to build independent smooth-mode fixtures. + /// + private static ReadOnlySpan Smooth8Weights => [255, 197, 146, 105, 73, 50, 37, 32]; + + [Fact] + public void EncodesClipped128SuperblockInWriterPreorderWithoutAllocation() + { + const int Width = 16; + const int Height = 16; + ObuColorConfig colorConfig = new() + { + IsMonochrome = false, + SubSamplingX = true, + SubSamplingY = true, + BitDepth = Av1BitDepth.EightBit + }; + + using Av1EncoderFrameBuffer source = new( + Configuration.Default, + Width, + Height, + 8, + Av1ColorFormat.Yuv420, + 1, + 1); + + using Av1EncoderFrameBuffer reconstruction = new( + Configuration.Default, + Width, + Height, + 8, + Av1ColorFormat.Yuv420, + 1, + 1); + + FillPlane(source.Frame.CodedView.GetPlane(Av1Plane.Y), (byte)128); + FillPlane(source.Frame.CodedView.GetPlane(Av1Plane.U), (byte)128); + FillPlane(source.Frame.CodedView.GetPlane(Av1Plane.V), (byte)128); + ClearPlane(reconstruction.Luma); + ClearPlane(Assert.IsType>(reconstruction.ChromaBlue)); + ClearPlane(Assert.IsType>(reconstruction.ChromaRed)); + + using Av1EncoderModeInfoBuffer modeInfo = new(Configuration.Default, Width, Height, disallow4x4AllFrames: true); + Av1PictureControlSet picture = CreatePicture(modeInfo, colorConfig, use128x128Superblock: true, qIndex: 73); + picture.Sequence.SequenceHeader.EnableFilterIntra = true; + using Av1EncoderCoefficientBuffer coefficients = new( + Configuration.Default, + picture.Sequence.SequenceHeader, + Width, + Height); + + using Av1EncoderSuperblockWorkspace superblockWorkspace = new(Configuration.Default); + using Av1EncoderBlockWorkspace blockWorkspace = new(Configuration.Default); + Av1Superblock superblock = new() + { + Workspace = superblockWorkspace, + TileInfo = new Av1TileInfo(0, 0, picture.Parent.FrameHeader), + Index = 0 + }; + + Av1IntraSuperblockEncoder.Encode( + source.Frame, + reconstruction.Frame, + picture, + superblock, + coefficients, + blockWorkspace); + + long before = GC.GetAllocatedBytesForCurrentThread(); + for (int iteration = 0; iteration < 8; iteration++) + { + Av1IntraSuperblockEncoder.Encode( + source.Frame, + reconstruction.Frame, + picture, + superblock, + coefficients, + blockWorkspace); + } + + Assert.Equal(0, GC.GetAllocatedBytesForCurrentThread() - before); + Av1PartitionType[] expectedPartitions = + [ + Av1PartitionType.Split, + Av1PartitionType.Split, + Av1PartitionType.Split, + Av1PartitionType.Split, + Av1PartitionType.None, + Av1PartitionType.None, + Av1PartitionType.None, + Av1PartitionType.None + ]; + + for (int index = 0; index < expectedPartitions.Length; index++) + { + Assert.Equal(expectedPartitions[index], (Av1PartitionType)superblock.CodingUnitPartitionTypes[index]); + } + + for (int index = 0; index < 4; index++) + { + Assert.True(superblock.FinalBlocks[index].HasChroma); + Assert.Equal(73, superblock.FinalBlocks[index].QuantizationIndex); + Assert.Equal(Av1FilterIntraMode.AllFilterIntraModes, superblock.FinalBlocks[index].FilterIntraMode); + } + + Point[] modeInfoPositions = [new(0, 0), new(2, 0), new(0, 2), new(2, 2)]; + foreach (Point position in modeInfoPositions) + { + ref Av1MacroBlockModeInfo block = ref picture.GetMacroBlockModeInfo(position); + Assert.Equal(Av1BlockSize.Block8x8, block.Block.BlockSize); + Assert.Equal(Av1TransformSize.Size8x8, block.Block.TransformSize); + Assert.Equal(Av1PredictionMode.DC, block.Block.Mode); + Assert.Equal(Av1ChromaPredictionMode.DC, block.Block.UvMode); + Assert.False(block.Block.Skip); + } + + Span lumaStates = coefficients.GetTransformBlockSpan(0, Av1Plane.Y); + Span blueStates = coefficients.GetTransformBlockSpan(0, Av1Plane.U); + Span redStates = coefficients.GetTransformBlockSpan(0, Av1Plane.V); + int[] lumaStateIndices = [0, 4, 8, 12]; + for (int index = 0; index < 4; index++) + { + Assert.Equal((ushort)0, lumaStates[lumaStateIndices[index]].EndOfBlock); + Assert.Equal(Av1TransformType.DctDct, lumaStates[lumaStateIndices[index]].TransformType); + Assert.Equal((ushort)0, blueStates[index].EndOfBlock); + Assert.Equal((ushort)0, redStates[index].EndOfBlock); + } + + AssertContainsNonzero(reconstruction.Frame.CodedView.GetPlane(Av1Plane.Y)); + AssertContainsNonzero(reconstruction.Frame.CodedView.GetPlane(Av1Plane.U)); + AssertContainsNonzero(reconstruction.Frame.CodedView.GetPlane(Av1Plane.V)); + + // Edge contexts cover the complete 128x128 superblock because partition updates retain the coded geometry + // even when most of the superblock lies beyond this deliberately clipped frame. + const int ContextUnitCount = 128 >> Av1Constants.ModeInfoSizeLog2; + using Av1NeighborArrayUnit partitions = new( + Configuration.Default, + ContextUnitCount, + ContextUnitCount) + { + GranularityNormalLog2 = Av1Constants.ModeInfoSizeLog2 + }; + + using Av1NeighborArrayUnit lumaContexts = new( + Configuration.Default, + ContextUnitCount, + ContextUnitCount) + { + GranularityNormalLog2 = Av1Constants.ModeInfoSizeLog2 + }; + + using Av1NeighborArrayUnit blueContexts = new( + Configuration.Default, + ContextUnitCount, + ContextUnitCount) + { + GranularityNormalLog2 = Av1Constants.ModeInfoSizeLog2 + }; + + using Av1NeighborArrayUnit redContexts = new( + Configuration.Default, + ContextUnitCount, + ContextUnitCount) + { + GranularityNormalLog2 = Av1Constants.ModeInfoSizeLog2 + }; + + using Av1NeighborArrayUnit transformContexts = new( + Configuration.Default, + ContextUnitCount, + ContextUnitCount) + { + GranularityNormalLog2 = Av1Constants.ModeInfoSizeLog2 + }; + + picture.PartitionContexts = [partitions]; + picture.LuminanceDcSignLevelCoefficientNeighbors = [lumaContexts]; + picture.CbDcSignLevelCoefficientNeighbors = [blueContexts]; + picture.CrDcSignLevelCoefficientNeighbors = [redContexts]; + picture.TransformFunctionContexts = [transformContexts]; + Av1TileWriter.Av1EntropyCodingContext entropyContext = new() + { + MacroBlock = new Av1MacroBlockD { Tile = superblock.TileInfo }, + MacroBlockModeInfo = picture.GetMacroBlockModeInfo(default), + SuperblockOrigin = default + }; + + using Av1SymbolEncoder writer = new(Configuration.Default, 512, 73, updateCdf: true); + Av1TileWriter.WriteSuperblock( + picture, + entropyContext, + writer, + superblock, + coefficients, + tileIndex: 0); + + using IMemoryOwner encoded = writer.Exit(); + + // The writer must consume exactly the transform areas populated above, proving both traversals stay synchronized. + Assert.Equal(256, entropyContext.CodedAreaSuperblock); + Assert.Equal(64, entropyContext.CodedAreaSuperblockUv); + Assert.NotEqual(0, encoded.GetSpan().Length); + + using Av1EncoderFrameBuffer tileReconstruction = new( + Configuration.Default, + Width, + Height, + 8, + Av1ColorFormat.Yuv420, + 1, + 1); + + ClearPlane(tileReconstruction.Luma); + ClearPlane(Assert.IsType>(tileReconstruction.ChromaBlue)); + ClearPlane(Assert.IsType>(tileReconstruction.ChromaRed)); + using Av1EncoderPictureBuffer tilePicture = new( + Configuration.Default, + picture.Sequence.SequenceHeader, + picture.Parent.FrameHeader, + Width, + Height, + disallow4x4AllFrames: true); + + using Av1EncoderCoefficientBuffer tileCoefficients = new( + Configuration.Default, + picture.Sequence.SequenceHeader, + Width, + Height); + + using Av1EncoderSuperblockWorkspace tileSuperblockWorkspace = new(Configuration.Default); + using Av1EncoderBlockWorkspace tileBlockWorkspace = new(Configuration.Default); + using Av1SymbolEncoder tileSymbolEncoder = CreateTileSymbolEncoder( + tilePicture.Picture, + 512); + + Av1TileEncoder tileWriter = new( + tileSymbolEncoder, + source.Frame, + tileReconstruction.Frame, + tilePicture.Picture, + tileCoefficients, + tileSuperblockWorkspace, + tileBlockWorkspace, + effort: 5); + + // The production tile traversal must be byte-identical to the explicit analyze-then-write composition above. + Assert.True(encoded.GetSpan().SequenceEqual(tileWriter.GetTileData(0))); + } + + [Theory] + [InlineData(true)] + [InlineData(false)] + public void PreservesIntraNonSkipForAllZeroTransforms(bool isMonochrome) + { + const int Width = 8; + const int Height = 8; + Av1ColorFormat colorFormat = isMonochrome ? Av1ColorFormat.Yuv400 : Av1ColorFormat.Yuv420; + ObuColorConfig colorConfig = new() + { + IsMonochrome = isMonochrome, + SubSamplingX = true, + SubSamplingY = true, + BitDepth = Av1BitDepth.EightBit + }; + + using Av1EncoderFrameBuffer source = new( + Configuration.Default, + Width, + Height, + 8, + colorFormat, + 1, + 1); + + using Av1EncoderFrameBuffer reconstruction = new( + Configuration.Default, + Width, + Height, + 8, + colorFormat, + 1, + 1); + + FillPlane(source.Frame.CodedView.GetPlane(Av1Plane.Y), (byte)128); + ClearPlane(reconstruction.Luma); + if (!isMonochrome) + { + FillPlane(source.Frame.CodedView.GetPlane(Av1Plane.U), (byte)128); + FillPlane(source.Frame.CodedView.GetPlane(Av1Plane.V), (byte)128); + ClearPlane(Assert.IsType>(reconstruction.ChromaBlue)); + ClearPlane(Assert.IsType>(reconstruction.ChromaRed)); + } + + using Av1EncoderModeInfoBuffer modeInfo = new(Configuration.Default, Width, Height, disallow4x4AllFrames: true); + Av1PictureControlSet pictureTemplate = CreatePicture(modeInfo, colorConfig, use128x128Superblock: false, qIndex: 37); + using Av1EncoderPictureBuffer pictureBuffer = new( + Configuration.Default, + pictureTemplate.Sequence.SequenceHeader, + pictureTemplate.Parent.FrameHeader, + Width, + Height, + disallow4x4AllFrames: true); + + Av1PictureControlSet picture = pictureBuffer.Picture; + using Av1EncoderCoefficientBuffer coefficients = new( + Configuration.Default, + pictureTemplate.Sequence.SequenceHeader, + Width, + Height); + + using Av1EncoderSuperblockWorkspace superblockWorkspace = new(Configuration.Default); + using Av1EncoderBlockWorkspace blockWorkspace = new(Configuration.Default); + Av1Superblock superblock = new() + { + Workspace = superblockWorkspace, + TileInfo = new Av1TileInfo(0, 0, picture.Parent.FrameHeader), + Index = 0 + }; + + Av1IntraSuperblockEncoder.Encode( + source.Frame, + reconstruction.Frame, + picture, + superblock, + coefficients, + blockWorkspace); + + ref Av1MacroBlockModeInfo block = ref picture.GetMacroBlockModeInfo(default); + + // Ordinary intra blocks retain the non-skip flag and empty transform symbols. Libaom applies + // this policy before final coding even when skipping would reconstruct the same samples. + Assert.False(block.Block.Skip); + Assert.Equal((ushort)0, coefficients.GetTransformBlockSpan(0, Av1Plane.Y)[0].EndOfBlock); + if (!isMonochrome) + { + Assert.Equal((ushort)0, coefficients.GetTransformBlockSpan(0, Av1Plane.U)[0].EndOfBlock); + Assert.Equal((ushort)0, coefficients.GetTransformBlockSpan(0, Av1Plane.V)[0].EndOfBlock); + } + + Av1TileWriter.Av1EntropyCodingContext entropyContext = new() + { + MacroBlock = new Av1MacroBlockD { Tile = superblock.TileInfo }, + MacroBlockModeInfo = picture.GetMacroBlockModeInfo(default), + SuperblockOrigin = default + }; + + using Av1SymbolEncoder writer = new(Configuration.Default, 256, 37, updateCdf: true); + Av1TileWriter.WriteSuperblock( + picture, + entropyContext, + writer, + superblock, + coefficients, + tileIndex: 0); + + using IMemoryOwner precomputedTile = writer.Exit(); + using Av1EncoderFrameBuffer liveReconstruction = new( + Configuration.Default, + Width, + Height, + 8, + colorFormat, + 1, + 1); + + ClearPlane(liveReconstruction.Luma); + if (!isMonochrome) + { + ClearPlane(Assert.IsType>(liveReconstruction.ChromaBlue)); + ClearPlane(Assert.IsType>(liveReconstruction.ChromaRed)); + } + + using Av1EncoderPictureBuffer livePicture = new( + Configuration.Default, + pictureTemplate.Sequence.SequenceHeader, + pictureTemplate.Parent.FrameHeader, + Width, + Height, + disallow4x4AllFrames: true); + + using Av1EncoderCoefficientBuffer liveCoefficients = new( + Configuration.Default, + pictureTemplate.Sequence.SequenceHeader, + Width, + Height); + + using Av1EncoderSuperblockWorkspace liveSuperblockWorkspace = new(Configuration.Default); + using Av1EncoderBlockWorkspace liveBlockWorkspace = new(Configuration.Default); + using Av1SymbolEncoder liveSymbolEncoder = CreateTileSymbolEncoder( + livePicture.Picture, + 256); + + Av1TileEncoder liveTileWriter = new( + liveSymbolEncoder, + source.Frame, + liveReconstruction.Frame, + livePicture.Picture, + liveCoefficients, + liveSuperblockWorkspace, + liveBlockWorkspace, + effort: 5); + + Assert.True(precomputedTile.GetSpan().SequenceEqual(liveTileWriter.GetTileData(0))); + } + + [Fact] + public void PreservesTwelveBitMonochromeReconstructionPrecision() + { + const int Width = 8; + const int Height = 8; + ObuColorConfig colorConfig = new() + { + IsMonochrome = true, + SubSamplingX = true, + SubSamplingY = true, + BitDepth = Av1BitDepth.TwelveBit + }; + + using Av1EncoderFrameBuffer source = new( + Configuration.Default, + Width, + Height, + 12, + Av1ColorFormat.Yuv400, + 0, + 0); + + using Av1EncoderFrameBuffer reconstruction = new( + Configuration.Default, + Width, + Height, + 12, + Av1ColorFormat.Yuv400, + 0, + 0); + + Buffer2DRegion sourcePlane = source.Frame.CodedView.GetPlane(Av1Plane.Y); + for (int y = 0; y < sourcePlane.Height; y++) + { + Span row = sourcePlane.DangerousGetRowSpan(y); + for (int x = 0; x < row.Length; x++) + { + row[x] = (ushort)(3000 + (((x * 71) + (y * 113)) % 1000)); + } + } + + ClearPlane(reconstruction.Luma); + using Av1EncoderModeInfoBuffer modeInfo = new(Configuration.Default, Width, Height, disallow4x4AllFrames: true); + Av1PictureControlSet picture = CreatePicture(modeInfo, colorConfig, use128x128Superblock: false, qIndex: 37); + using Av1EncoderCoefficientBuffer coefficients = new( + Configuration.Default, + picture.Sequence.SequenceHeader, + Width, + Height); + + using Av1EncoderSuperblockWorkspace superblockWorkspace = new(Configuration.Default); + using Av1EncoderBlockWorkspace blockWorkspace = new(Configuration.Default); + Av1Superblock superblock = new() + { + Workspace = superblockWorkspace, + TileInfo = new Av1TileInfo(0, 0, picture.Parent.FrameHeader), + Index = 0 + }; + + Av1IntraSuperblockEncoder.Encode( + source.Frame, + reconstruction.Frame, + picture, + superblock, + coefficients, + blockWorkspace); + + Buffer2DRegion reconstructionPlane = reconstruction.Frame.CodedView.GetPlane(Av1Plane.Y); + ushort maximum = 0; + for (int y = 0; y < reconstructionPlane.Height; y++) + { + foreach (ushort sample in reconstructionPlane.DangerousGetRowSpan(y)) + { + maximum = Math.Max(maximum, sample); + Assert.InRange(sample, (ushort)0, (ushort)4095); + } + } + + Assert.InRange(maximum, (ushort)(byte.MaxValue + 1), (ushort)4095); + Assert.False(superblock.FinalBlocks[0].HasChroma); + Assert.Equal(37, superblock.FinalBlocks[0].QuantizationIndex); + Assert.NotEqual((ushort)0, coefficients.GetTransformBlockSpan(0, Av1Plane.Y)[0].EndOfBlock); + Assert.Equal(0, coefficients.GetPlaneSpan(0, Av1Plane.U).Length); + Assert.Equal(0, coefficients.GetPlaneSpan(0, Av1Plane.V).Length); + + using Av1EncoderFrameBuffer tileReconstruction = new( + Configuration.Default, + Width, + Height, + 12, + Av1ColorFormat.Yuv400, + 0, + 0); + + ClearPlane(tileReconstruction.Luma); + using Av1EncoderPictureBuffer tilePicture = new( + Configuration.Default, + picture.Sequence.SequenceHeader, + picture.Parent.FrameHeader, + Width, + Height, + disallow4x4AllFrames: true); + + using Av1EncoderCoefficientBuffer tileCoefficients = new( + Configuration.Default, + picture.Sequence.SequenceHeader, + Width, + Height); + + using Av1EncoderSuperblockWorkspace tileSuperblockWorkspace = new(Configuration.Default); + using Av1EncoderBlockWorkspace tileBlockWorkspace = new(Configuration.Default); + using Av1SymbolEncoder tileSymbolEncoder = CreateTileSymbolEncoder( + tilePicture.Picture, + 256); + + Av1TileEncoder tileWriter = new( + tileSymbolEncoder, + source.Frame, + tileReconstruction.Frame, + tilePicture.Picture, + tileCoefficients, + tileSuperblockWorkspace, + tileBlockWorkspace, + effort: 5); + + Assert.NotEqual(0, tileWriter.GetTileData(0).Length); + ushort reconstructedSample = tileReconstruction.Frame.CodedView + .GetPlane(Av1Plane.Y) + .DangerousGetRowSpan(0)[0]; + + Assert.InRange(reconstructedSample, (ushort)(byte.MaxValue + 1), (ushort)4095); + } + + [Theory] + [InlineData(true, false)] + [InlineData(false, false)] + [InlineData(true, true)] + [InlineData(false, true)] + public void BlockDecisionRetainsUnroundedRateAndDistortion(bool isMonochrome, bool textured) + { + const int Width = 8; + const int Height = 8; + const int QIndex = 37; + Av1ColorFormat colorFormat = isMonochrome ? Av1ColorFormat.Yuv400 : Av1ColorFormat.Yuv420; + ObuColorConfig colorConfig = new() + { + IsMonochrome = isMonochrome, + SubSamplingX = true, + SubSamplingY = true, + BitDepth = Av1BitDepth.EightBit + }; + + using Av1EncoderFrameBuffer source = new(Configuration.Default, Width, Height, 8, colorFormat, 1, 1); + using Av1EncoderFrameBuffer reconstruction = new(Configuration.Default, Width, Height, 8, colorFormat, 1, 1); + int planeCount = isMonochrome ? 1 : 3; + for (int planeIndex = 0; planeIndex < planeCount; planeIndex++) + { + Buffer2DRegion plane = source.Frame.CodedView.GetPlane((Av1Plane)planeIndex); + int length = planeIndex == 0 ? 8 : 4; + for (int y = 0; y < length; y++) + { + Span row = plane.DangerousGetRowSpan(y); + for (int x = 0; x < length; x++) + { + row[x] = textured ? (byte)(114 + (((x * 13) + (y * 7) + (planeIndex * 5)) % 29)) : (byte)128; + } + } + } + + using Av1EncoderModeInfoBuffer modeInfoBuffer = new(Configuration.Default, Width, Height, disallow4x4AllFrames: true); + Av1PictureControlSet template = CreatePicture(modeInfoBuffer, colorConfig, use128x128Superblock: false, QIndex); + template.Parent.FrameHeader.TransformMode = Av1TransformMode.Largest; + using Av1EncoderPictureBuffer pictureBuffer = new( + Configuration.Default, + template.Sequence.SequenceHeader, + template.Parent.FrameHeader, + Width, + Height, + disallow4x4AllFrames: true); + + Av1PictureControlSet picture = pictureBuffer.Picture; + using Av1EncoderCoefficientBuffer coefficients = new(Configuration.Default, template.Sequence.SequenceHeader, Width, Height); + using Av1EncoderSuperblockWorkspace superblockWorkspace = new(Configuration.Default); + using Av1EncoderBlockWorkspace blockWorkspace = new(Configuration.Default); + Av1Superblock superblock = new() + { + Workspace = superblockWorkspace, + TileInfo = new Av1TileInfo(0, 0, picture.Parent.FrameHeader), + Index = 0 + }; + + Av1IntraSuperblockEncoder.Prepare(picture, superblock, Point.Empty); + Av1MacroBlockD macroBlock = new() { Tile = superblock.TileInfo }; + Av1TileWriter.SetModeInfoRowAndColumn( + picture, + macroBlock, + superblock.TileInfo, + Point.Empty, + Av1BlockSize.Block8x8, + picture.Parent.Common.ModeInfoStride, + picture.Parent.Common.ModeInfoRowCount, + picture.Parent.Common.ModeInfoColumnCount); + + Av1IntraSuperblockEncoder.ModeDecision decision = new( + source.Frame, + reconstruction.Frame, + reconstruction.Frame, + picture, + superblock, + coefficients, + blockWorkspace, + effort: 0); + + ref Av1MacroBlockModeInfo modeInfo = ref picture.GetMacroBlockModeInfo(Point.Empty); + Av1EncoderBlockStruct block = default; + Av1EncoderPaletteInfo palette = default; + using Av1SymbolEncoder writer = new(Configuration.Default, 256, QIndex, updateCdf: true); + decision.EncodeBlock(writer, macroBlock, Point.Empty, 0, ref modeInfo, ref block, ref palette); + + Assert.Equal(Av1PredictionMode.DC, modeInfo.Block.Mode); + Assert.Equal(Av1TransformSize.Size8x8, modeInfo.Block.TransformSize); + Assert.False(modeInfo.Block.Skip); + int expectedRate = writer.GetSkipCost(false, Av1TileWriter.GetSkipContext(macroBlock)); + expectedRate += Av1TileWriter.GetLumaModeCost( + writer, + macroBlock, + Av1BlockSize.Block8x8, + Av1PredictionMode.DC, + 0, + isIntraFrame: true); + + if (!isMonochrome) + { + Assert.Equal(Av1ChromaPredictionMode.DC, modeInfo.Block.UvMode); + expectedRate += Av1TileWriter.GetChromaModeCost( + writer, + picture.Parent.FrameHeader, + colorConfig, + modeInfo, + Av1BlockSize.Block8x8, + Av1PredictionMode.DC, + Av1ChromaPredictionMode.DC, + 0); + } + + long squaredError = 0; + for (int planeIndex = 0; planeIndex < planeCount; planeIndex++) + { + Av1Plane plane = (Av1Plane)planeIndex; + Av1TransformSize transformSize = planeIndex == 0 ? Av1TransformSize.Size8x8 : Av1TransformSize.Size4x4; + Av1BlockSize blockSize = planeIndex == 0 ? Av1BlockSize.Block8x8 : Av1BlockSize.Block4x4; + Av1ComponentType component = planeIndex == 0 ? Av1ComponentType.Luminance : Av1ComponentType.Chroma; + Av1NeighborArrayUnit neighbors = planeIndex switch + { + 0 => picture.LuminanceDcSignLevelCoefficientNeighbors[0], + 1 => picture.CbDcSignLevelCoefficientNeighbors[0], + _ => picture.CrDcSignLevelCoefficientNeighbors[0] + }; + + Av1EncoderTransformBlockState state = coefficients.GetTransformBlockSpan(0, plane)[0]; + expectedRate += writer.GetCoefficientCost( + transformSize, + state.TransformType, + Av1PredictionMode.DC, + coefficients.GetPlaneSpan(0, plane)[..transformSize.GetSize2d()], + component, + Av1TileWriter.GetTransformBlockContexts(component, neighbors, Point.Empty, blockSize, transformSize), + state.EndOfBlock, + picture.Parent.FrameHeader.UseReducedTransformSet, + Av1FilterIntraMode.AllFilterIntraModes, + usesInterTransformSet: false); + + Buffer2DRegion sourcePlane = source.Frame.CodedView.GetPlane(plane); + Buffer2DRegion reconstructedPlane = reconstruction.Frame.CodedView.GetPlane(plane); + int length = planeIndex == 0 ? 8 : 4; + for (int y = 0; y < length; y++) + { + ReadOnlySpan sourceRow = sourcePlane.DangerousGetRowSpan(y); + ReadOnlySpan reconstructedRow = reconstructedPlane.DangerousGetRowSpan(y); + for (int x = 0; x < length; x++) + { + int difference = sourceRow[x] - reconstructedRow[x]; + squaredError += difference * difference; + } + } + } + + // Pixel-domain SSE uses the reference's four fractional distortion bits. The probability rate + // is rounded after all planes and block syntax have been counted, before adding scaled distortion. + long expectedDistortion = squaredError * 16; + int multiplier = Av1RateDistortion.GetKeyFrameRateMultiplier(QIndex, Av1BitDepth.EightBit); + long expectedCost = ((((long)expectedRate * multiplier) + 256) / 512) + (expectedDistortion * 128); + Assert.Equal(textured, squaredError > 0); + Assert.Equal(expectedRate, decision.SelectedBlockStatistics.Rate); + Assert.Equal(expectedDistortion, decision.SelectedBlockStatistics.Distortion); + Assert.Equal(expectedCost, decision.SelectedBlockStatistics.Cost); + } + + [Fact] + public void BlockDecisionObservesLiveCdfInWriterOrder() + { + const int Width = 16; + const int Height = 8; + const int QIndex = 37; + ObuColorConfig colorConfig = new() + { + IsMonochrome = true, + SubSamplingX = true, + SubSamplingY = true, + BitDepth = Av1BitDepth.EightBit, + }; + + using Av1EncoderModeInfoBuffer modeInfo = new(Configuration.Default, Width, Height, disallow4x4AllFrames: true); + Av1PictureControlSet pictureTemplate = CreatePicture( + modeInfo, + colorConfig, + use128x128Superblock: false, + QIndex); + + using Av1EncoderPictureBuffer picture = new( + Configuration.Default, + pictureTemplate.Sequence.SequenceHeader, + pictureTemplate.Parent.FrameHeader, + Width, + Height, + disallow4x4AllFrames: true); + + using Av1EncoderCoefficientBuffer coefficients = new( + Configuration.Default, + pictureTemplate.Sequence.SequenceHeader, + Width, + Height); + + using Av1EncoderSuperblockWorkspace superblockWorkspace = new(Configuration.Default); + Av1Superblock superblock = new() + { + Workspace = superblockWorkspace, + TileInfo = new Av1TileInfo(0, 0, picture.Picture.Parent.FrameHeader), + Index = 0, + }; + + Av1IntraSuperblockEncoder.Prepare(picture.Picture, superblock, Point.Empty); + Av1TileWriter.Av1EntropyCodingContext entropyContext = new() + { + MacroBlock = new Av1MacroBlockD { Tile = superblock.TileInfo }, + MacroBlockModeInfo = picture.Picture.GetMacroBlockModeInfo(default), + SuperblockOrigin = default, + }; + + int[] costs = new int[2]; + BlockCostRecorder blockEncoder = new(costs, QIndex); + using Av1SymbolEncoder writer = new(Configuration.Default, 256, QIndex, updateCdf: true); + Av1TileWriter.WriteSuperblock( + picture.Picture, + entropyContext, + writer, + superblock, + coefficients, + tileIndex: 0, + ref blockEncoder); + + using IMemoryOwner encoded = writer.Exit(); + Assert.Equal(2, blockEncoder.Count); + Assert.True(costs[1] < costs[0]); + Assert.NotEqual(0, encoded.GetSpan().Length); + } + + [Fact] + public void ProductionWriterConsumesPaletteMapAndPublishesPaletteEdges() + { + const int Width = 8; + const int Height = 8; + const int QIndex = 23; + ObuColorConfig colorConfig = new() + { + IsMonochrome = true, + SubSamplingX = true, + SubSamplingY = true, + BitDepth = Av1BitDepth.EightBit + }; + + ObuTileGroupHeader tiles = new() + { + TileColumnCount = 1, + TileRowCount = 1 + }; + + tiles.TileColumnStartModeInfo[1] = Width >> Av1Constants.ModeInfoSizeLog2; + tiles.TileRowStartModeInfo[1] = Height >> Av1Constants.ModeInfoSizeLog2; + ObuSequenceHeader sequenceHeader = new() + { + ColorConfig = colorConfig + }; + + ObuFrameHeader frameHeader = new() + { + AllowScreenContentTools = true, + ModeInfoColumnCount = Width >> Av1Constants.ModeInfoSizeLog2, + ModeInfoRowCount = Height >> Av1Constants.ModeInfoSizeLog2, + FrameSize = new ObuFrameSize + { + FrameWidth = Width, + FrameHeight = Height + }, + TilesInfo = tiles + }; + + frameHeader.QuantizationParameters.BaseQIndex = QIndex; + frameHeader.QuantizationParameters.QIndex.Fill(QIndex); + byte[][] payloads = new byte[2][]; + for (int mapVariant = 0; mapVariant < payloads.Length; mapVariant++) + { + using Av1EncoderPictureBuffer pictureBuffer = new( + Configuration.Default, + sequenceHeader, + frameHeader, + Width, + Height, + disallow4x4AllFrames: true); + + using Av1EncoderCoefficientBuffer coefficients = new( + Configuration.Default, + sequenceHeader, + Width, + Height); + + using Av1EncoderSuperblockWorkspace workspace = new(Configuration.Default); + Av1Superblock superblock = new() + { + Workspace = workspace, + TileInfo = new Av1TileInfo(0, 0, frameHeader), + Index = 0 + }; + + Av1PictureControlSet picture = pictureBuffer.Picture; + Av1IntraSuperblockEncoder.Prepare(picture, superblock, Point.Empty); + Av1TileWriter.Av1EntropyCodingContext entropyContext = new() + { + MacroBlock = new Av1MacroBlockD { Tile = superblock.TileInfo }, + MacroBlockModeInfo = picture.GetMacroBlockModeInfo(default), + SuperblockOrigin = default + }; + + PaletteBlockEncoder blockEncoder = new(workspace, QIndex, mapVariant); + using Av1SymbolEncoder writer = new(Configuration.Default, 128, QIndex, updateCdf: true); + Av1TileWriter.WriteSuperblock( + picture, + entropyContext, + writer, + superblock, + coefficients, + tileIndex: 0, + ref blockEncoder); + + using IMemoryOwner encoded = writer.Exit(); + payloads[mapVariant] = encoded.GetSpan().ToArray(); + Assert.Equal(1, blockEncoder.Count); + Av1NeighborArrayUnit paletteContext = Assert.Single(picture.PaletteContexts); + for (int index = 0; index < 2; index++) + { + Assert.Equal(3, paletteContext.Top[index].PaletteSizes[0]); + Assert.Equal(3, paletteContext.Left[index].PaletteSizes[0]); + Assert.Equal([16, 128, 240], paletteContext.Top[index].GetColors(Av1Plane.Y).ToArray()); + Assert.Equal([16, 128, 240], paletteContext.Left[index].GetColors(Av1Plane.Y).ToArray()); + } + + Assert.Equal(0, paletteContext.Top[2].PaletteSizes[0]); + Assert.Equal(0, paletteContext.Left[2].PaletteSizes[0]); + } + + // Changing only the selected color indices must change the range-coded tile payload. + Assert.False(payloads[0].SequenceEqual(payloads[1])); + } + + [Fact] + public void ProductionTileSelectsExactLumaPaletteAtFullAndClippedSizes() + { + AssertProductionTileSelectsExactLumaPalette( + Av1BitDepth.EightBit, + 8, + 8, + 8, + false, + (byte)32, + (byte)224, + 32, + 224, + static (writer, source, reconstruction, picture, coefficients, superblockWorkspace, blockWorkspace) => + new Av1TileEncoder( + writer, + source, + reconstruction, + picture, + coefficients, + superblockWorkspace, + blockWorkspace, + effort: 5)); + + AssertProductionTileSelectsExactLumaPalette( + Av1BitDepth.TwelveBit, + 12, + 8, + 8, + false, + (ushort)512, + (ushort)3584, + 512, + 3584, + static (writer, source, reconstruction, picture, coefficients, superblockWorkspace, blockWorkspace) => + new Av1TileEncoder( + writer, + source, + reconstruction, + picture, + coefficients, + superblockWorkspace, + blockWorkspace, + effort: 5)); + + AssertProductionTileSelectsExactLumaPalette( + Av1BitDepth.EightBit, + 8, + 5, + 3, + false, + (byte)48, + (byte)208, + 48, + 208, + static (writer, source, reconstruction, picture, coefficients, superblockWorkspace, blockWorkspace) => + new Av1TileEncoder( + writer, + source, + reconstruction, + picture, + coefficients, + superblockWorkspace, + blockWorkspace, + effort: 5)); + } + + [Theory] + [InlineData(5, 3)] + [InlineData(3, 5)] + [InlineData(1, 5)] + public void ProductionTileSelectsExactLumaPaletteAtClippedHighBitDepths(int width, int height) + { + AssertProductionTileSelectsExactLumaPalette( + Av1BitDepth.TenBit, + 10, + width, + height, + false, + (ushort)128, + (ushort)896, + 128, + 896, + static (writer, source, reconstruction, picture, coefficients, superblockWorkspace, blockWorkspace) => + new Av1TileEncoder( + writer, source, reconstruction, picture, coefficients, superblockWorkspace, blockWorkspace, effort: 5)); + + AssertProductionTileSelectsExactLumaPalette( + Av1BitDepth.TwelveBit, + 12, + width, + height, + false, + (ushort)512, + (ushort)3584, + 512, + 3584, + static (writer, source, reconstruction, picture, coefficients, superblockWorkspace, blockWorkspace) => + new Av1TileEncoder( + writer, source, reconstruction, picture, coefficients, superblockWorkspace, blockWorkspace, effort: 5)); + } + + [Fact] + public void ProductionTileSelectsLumaPaletteWithFourByFourTransforms() + { + AssertProductionTileSelectsExactLumaPalette( + Av1BitDepth.EightBit, + 8, + 8, + 8, + true, + (byte)64, + (byte)192, + 64, + 192, + static (writer, source, reconstruction, picture, coefficients, superblockWorkspace, blockWorkspace) => + new Av1TileEncoder( + writer, + source, + reconstruction, + picture, + coefficients, + superblockWorkspace, + blockWorkspace, + effort: 6)); + + AssertProductionTileSelectsExactLumaPalette( + Av1BitDepth.TwelveBit, + 12, + 8, + 8, + true, + (ushort)1024, + (ushort)3072, + 1024, + 3072, + static (writer, source, reconstruction, picture, coefficients, superblockWorkspace, blockWorkspace) => + new Av1TileEncoder( + writer, + source, + reconstruction, + picture, + coefficients, + superblockWorkspace, + blockWorkspace, + effort: 6)); + } + + [Fact] + public void ProductionTileSelectsExactPairedChromaPalette() + { + AssertProductionTileSelectsExactPairedChromaPalette(false, 8, 8, false, false); + AssertProductionTileSelectsExactPairedChromaPalette(true, 8, 8, false, false); + } + + [Theory] + [InlineData(5, 3, false, false)] + [InlineData(3, 5, false, false)] + [InlineData(5, 3, true, false)] + [InlineData(3, 5, true, false)] + [InlineData(5, 3, true, true)] + [InlineData(3, 5, true, true)] + [InlineData(1, 5, true, false)] + [InlineData(5, 1, true, false)] + [InlineData(1, 5, true, true)] + [InlineData(5, 1, true, true)] + public void ProductionTileSelectsExactPairedChromaPaletteAtClippedSizes(int width, int height, bool subX, bool subY) + => AssertProductionTileSelectsExactPairedChromaPalette(false, width, height, subX, subY); + + private static void AssertProductionTileSelectsExactPairedChromaPalette( + bool useLumaPalette, + int width, + int height, + bool subX, + bool subY) + { + const int QIndex = 37; + ObuColorConfig colorConfig = new() + { + IsMonochrome = false, + SubSamplingX = subX, + SubSamplingY = subY, + BitDepth = Av1BitDepth.EightBit + }; + + Av1ColorFormat colorFormat = subY ? Av1ColorFormat.Yuv420 : subX ? Av1ColorFormat.Yuv422 : Av1ColorFormat.Yuv444; + using Av1EncoderFrameBuffer source = new( + Configuration.Default, + width, + height, + 8, + colorFormat, + 0, + 0); + + using Av1EncoderFrameBuffer reconstruction = new( + Configuration.Default, + width, + height, + 8, + colorFormat, + 0, + 0); + + Buffer2DRegion lumaSource = source.Frame.CodedView.GetPlane(Av1Plane.Y); + Buffer2DRegion blueSource = source.Frame.CodedView.GetPlane(Av1Plane.U); + Buffer2DRegion redSource = source.Frame.CodedView.GetPlane(Av1Plane.V); + for (int row = 0; row < lumaSource.Height; row++) + { + Span lumaRow = lumaSource.DangerousGetRowSpan(row); + if (useLumaPalette) + { + for (int column = 0; column < lumaRow.Length; column++) + { + lumaRow[column] = column < width / 2 ? (byte)64 : (byte)192; + } + } + else + { + lumaRow.Fill(128); + } + } + + int chromaWidth = (width + (subX ? 1 : 0)) >> (subX ? 1 : 0); + int chromaHeight = (height + (subY ? 1 : 0)) >> (subY ? 1 : 0); + for (int row = 0; row < blueSource.Height; row++) + { + for (int column = 0; column < blueSource.Width; column++) + { + // Repeat the last visible sample into coded alignment, including one-pixel source axes. + bool firstColor = chromaHeight > 1 + ? Math.Min(row, chromaHeight - 1) < chromaHeight / 2 + : Math.Min(column, chromaWidth - 1) < chromaWidth / 2; + + blueSource.DangerousGetRowSpan(row)[column] = firstColor ? (byte)32 : (byte)224; + redSource.DangerousGetRowSpan(row)[column] = firstColor ? (byte)200 : (byte)40; + } + } + + ClearPlane(reconstruction.Luma); + ClearPlane(Assert.IsType>(reconstruction.ChromaBlue)); + ClearPlane(Assert.IsType>(reconstruction.ChromaRed)); + using Av1EncoderModeInfoBuffer modeInfo = new( + Configuration.Default, + width, + height, + disallow4x4AllFrames: true); + + Av1PictureControlSet pictureTemplate = CreatePicture( + modeInfo, + colorConfig, + use128x128Superblock: false, + QIndex); + + pictureTemplate.Parent.FrameHeader.AllowScreenContentTools = true; + pictureTemplate.Parent.FrameHeader.FrameSize.FrameWidth = width; + pictureTemplate.Parent.FrameHeader.FrameSize.FrameHeight = height; + using Av1EncoderPictureBuffer picture = new( + Configuration.Default, + pictureTemplate.Sequence.SequenceHeader, + pictureTemplate.Parent.FrameHeader, + width, + height, + disallow4x4AllFrames: true); + + using Av1EncoderCoefficientBuffer coefficients = new( + Configuration.Default, + pictureTemplate.Sequence.SequenceHeader, + width, + height); + + using Av1EncoderSuperblockWorkspace superblockWorkspace = new(Configuration.Default); + using Av1EncoderBlockWorkspace blockWorkspace = new(Configuration.Default); + using Av1SymbolEncoder symbolEncoder = CreateTileSymbolEncoder( + picture.Picture, + 256); + + Av1TileEncoder tileWriter = new( + symbolEncoder, + source.Frame, + reconstruction.Frame, + picture.Picture, + coefficients, + superblockWorkspace, + blockWorkspace, + effort: 5); + + ref Av1MacroBlockModeInfo mode = ref picture.Picture.GetMacroBlockModeInfo(default); + Assert.Equal(Av1ChromaPredictionMode.DC, mode.Block.UvMode); + Assert.Equal(useLumaPalette ? 2 : 0, superblockWorkspace.PaletteInfo.PaletteSizes[0]); + Assert.Equal(2, superblockWorkspace.PaletteInfo.PaletteSizes[1]); + Assert.Equal([32, 224], superblockWorkspace.PaletteInfo.GetColors(Av1Plane.U).ToArray()); + Assert.Equal([200, 40], superblockWorkspace.PaletteInfo.GetColors(Av1Plane.V).ToArray()); + Assert.Equal((ushort)0, coefficients.GetTransformBlockSpan(0, Av1Plane.U)[0].EndOfBlock); + Assert.Equal((ushort)0, coefficients.GetTransformBlockSpan(0, Av1Plane.V)[0].EndOfBlock); + + Buffer2DRegion colorIndexMap = superblockWorkspace + .GetPaletteMaps() + .GetMap(Av1PlaneType.Uv, blueSource.Width, blueSource.Height); + + Buffer2DRegion blueReconstruction = reconstruction.Frame.CodedView.GetPlane(Av1Plane.U); + Buffer2DRegion redReconstruction = reconstruction.Frame.CodedView.GetPlane(Av1Plane.V); + for (int row = 0; row < blueSource.Height; row++) + { + for (int column = 0; column < blueSource.Width; column++) + { + byte expectedIndex = blueSource.DangerousGetRowSpan(row)[column] == 32 ? (byte)0 : (byte)1; + Assert.Equal(expectedIndex, colorIndexMap.DangerousGetRowSpan(row)[column]); + } + + Assert.True(blueSource.DangerousGetRowSpan(row).SequenceEqual(blueReconstruction.DangerousGetRowSpan(row))); + Assert.True(redSource.DangerousGetRowSpan(row).SequenceEqual(redReconstruction.DangerousGetRowSpan(row))); + } + + byte[] payload = WriteCompleteTileObu(pictureTemplate, tileWriter, width, height); + using Av1Decoder decoder = new(Configuration.Default); + using Av1FrameBuffer decodedFrame = decoder.DecodeFrameBuffer(payload, null, null, out _); + Assert.Equal(width, decodedFrame.Width); + Assert.Equal(height, decodedFrame.Height); + Assert.NotNull(decoder.FrameInfo); + Assert.Equal(2, decoder.FrameInfo.GetModeInfoAt(default).GetPaletteSize(Av1PlaneType.Uv)); + for (int plane = 0; plane < 3; plane++) + { + int planeSubX = plane > 0 && subX ? 1 : 0; + int planeSubY = plane > 0 && subY ? 1 : 0; + Buffer2DRegion actual = decodedFrame.DeriveBlockPointer((Av1Plane)plane, planeSubX, planeSubY); + Buffer2DRegion expected = reconstruction.Frame.View.GetPlane((Av1Plane)plane); + for (int row = 0; row < expected.Height; row++) + { + Assert.Equal(expected.DangerousGetRowSpan(row), actual.DangerousGetRowSpan(row)); + } + } + + string outputDirectory = Path.Combine(TestEnvironment.ActualOutputDirectoryFullPath, "Formats", "Heif", "Av1"); + string outputName = $"encoder-palette-chroma-{width}x{height}-{subX}-{subY}-{useLumaPalette}"; + Directory.CreateDirectory(outputDirectory); + File.WriteAllBytes(Path.Combine(outputDirectory, outputName + ".obu"), payload); + using FileStream raw = File.Create(Path.Combine(outputDirectory, outputName + ".retained.yuv")); + for (int plane = 0; plane < 3; plane++) + { + Buffer2DRegion retained = reconstruction.Frame.View.GetPlane((Av1Plane)plane); + for (int row = 0; row < retained.Height; row++) + { + raw.Write(retained.DangerousGetRowSpan(row)); + } + } + + Assert.NotEqual(0, tileWriter.GetTileData(0).Length); + } + + [Theory] + [InlineData((int)Av1PredictionMode.Vertical, 0)] + [InlineData((int)Av1PredictionMode.Horizontal, 0)] + [InlineData((int)Av1PredictionMode.Smooth, 0)] + [InlineData((int)Av1PredictionMode.Paeth, 0)] + [InlineData((int)Av1PredictionMode.SmoothVertical, 0)] + [InlineData((int)Av1PredictionMode.SmoothHorizontal, 0)] + [InlineData((int)Av1PredictionMode.Directional135Degrees, 0)] + [InlineData((int)Av1PredictionMode.Directional203Degrees, 0)] + [InlineData((int)Av1PredictionMode.Directional157Degrees, 0)] + [InlineData((int)Av1PredictionMode.Directional67Degrees, 0)] + [InlineData((int)Av1PredictionMode.Directional113Degrees, 0)] + [InlineData((int)Av1PredictionMode.Directional45Degrees, 0)] + [InlineData((int)Av1PredictionMode.Directional45Degrees, -3)] + [InlineData((int)Av1PredictionMode.Directional45Degrees, 3)] + [InlineData((int)Av1PredictionMode.Directional135Degrees, -3)] + [InlineData((int)Av1PredictionMode.Directional135Degrees, 3)] + [InlineData((int)Av1PredictionMode.Directional203Degrees, -3)] + [InlineData((int)Av1PredictionMode.Directional203Degrees, 3)] + public void ProductionTileSelectsModeFromCurrentReconstruction(int expectedModeValue, int expectedAngleDelta) + { + const int Width = 16; + const int Height = 16; + const byte TopReference = 48; + const byte LeftReference = 208; + const int QIndex = 1; + Av1PredictionMode expectedMode = (Av1PredictionMode)expectedModeValue; + bool isDiagonal = expectedMode is >= Av1PredictionMode.Directional45Degrees and <= Av1PredictionMode.Directional67Degrees; + int cornerReference = expectedMode == Av1PredictionMode.Horizontal + ? LeftReference + : expectedMode == Av1PredictionMode.Vertical ? TopReference : 128; + + Span directionalTarget = stackalloc byte[64]; + if (isDiagonal) + { + Span aboveStorage = stackalloc byte[17]; + Span above = aboveStorage[1..]; + Span leftStorage = stackalloc byte[17]; + Span left = leftStorage[1..]; + aboveStorage[0] = 128; + leftStorage[0] = 128; + for (int i = 0; i < 8; i++) + { + above[i] = (byte)(32 + (i * 24)); + left[i] = (byte)(224 - (i * 24)); + } + + above[8..].Fill(above[7]); + left[8..].Fill(left[7]); + + // Directional arithmetic has separate byte-exact reference coverage. This fixture uses its scalar + // path only to isolate production mode traversal, reference gathering, and rate-distortion selection. + Av1DirectionalIntraPredictor.PredictScalar( + directionalTarget, + 8, + Av1TransformSize.Size8x8, + above, + left, + false, + false, + expectedMode.ToAngle() + (expectedAngleDelta * Av1Constants.AngleStep)); + } + + ObuColorConfig colorConfig = new() + { + IsMonochrome = true, + SubSamplingX = true, + SubSamplingY = true, + BitDepth = Av1BitDepth.EightBit + }; + + using Av1EncoderFrameBuffer source = new( + Configuration.Default, + Width, + Height, + 8, + Av1ColorFormat.Yuv400, + 0, + 0); + + using Av1EncoderFrameBuffer reconstruction = new( + Configuration.Default, + Width, + Height, + 8, + Av1ColorFormat.Yuv400, + 0, + 0); + + Buffer2DRegion sourcePlane = source.Frame.CodedView.GetPlane(Av1Plane.Y); + + // The first three 8x8 blocks establish the corner, top, and left reconstruction consumed by + // the bottom-right target. This makes the assertion exercise production traversal and live state. + for (int y = 0; y < Height; y++) + { + Span row = sourcePlane.DangerousGetRowSpan(y); + for (int x = 0; x < Width; x++) + { + int rowIndex = y - 8; + int columnIndex = x - 8; + int value; + if (y < 8) + { + value = x < 8 + ? cornerReference + : isDiagonal + ? 32 + (columnIndex * 24) + : expectedMode == Av1PredictionMode.Paeth ? 40 + (columnIndex * 20) : TopReference; + } + else if (x < 8) + { + value = isDiagonal + ? 224 - (rowIndex * 24) + : expectedMode == Av1PredictionMode.Paeth ? 200 - (rowIndex * 20) : LeftReference; + } + else if (isDiagonal) + { + value = directionalTarget[(rowIndex * 8) + columnIndex]; + } + else if (expectedMode == Av1PredictionMode.Paeth) + { + // Build the target from the nearest of left, top, and corner without calling the production predictor. + int top = 40 + (columnIndex * 20); + int left = 200 - (rowIndex * 20); + int predictor = top + left - 128; + int leftDistance = Math.Abs(predictor - left); + int topDistance = Math.Abs(predictor - top); + int cornerDistance = Math.Abs(predictor - 128); + + value = leftDistance <= topDistance && leftDistance <= cornerDistance + ? left + : topDistance <= cornerDistance ? top : 128; + } + else + { + // Apply the normative interpolation directly so a production predictor cannot generate its own fixture. + int rowWeight = Smooth8Weights[rowIndex]; + int columnWeight = Smooth8Weights[columnIndex]; + value = expectedMode switch + { + Av1PredictionMode.Horizontal => LeftReference, + Av1PredictionMode.Vertical => TopReference, + Av1PredictionMode.SmoothVertical => ((rowWeight * TopReference) + ((256 - rowWeight) * LeftReference) + 128) >> 8, + Av1PredictionMode.SmoothHorizontal => ((columnWeight * LeftReference) + ((256 - columnWeight) * TopReference) + 128) >> 8, + _ => ((rowWeight * TopReference) + ((256 - rowWeight) * LeftReference) + + (columnWeight * LeftReference) + ((256 - columnWeight) * TopReference) + 256) >> 9 + }; + } + + row[x] = (byte)value; + } + } + + ClearPlane(reconstruction.Luma); + using Av1EncoderModeInfoBuffer modeInfo = new(Configuration.Default, Width, Height, disallow4x4AllFrames: true); + Av1PictureControlSet pictureTemplate = CreatePicture(modeInfo, colorConfig, use128x128Superblock: false, QIndex); + using Av1EncoderPictureBuffer picture = new( + Configuration.Default, + pictureTemplate.Sequence.SequenceHeader, + pictureTemplate.Parent.FrameHeader, + Width, + Height, + disallow4x4AllFrames: true); + + using Av1EncoderCoefficientBuffer coefficients = new( + Configuration.Default, + pictureTemplate.Sequence.SequenceHeader, + Width, + Height); + + using Av1EncoderSuperblockWorkspace superblockWorkspace = new(Configuration.Default); + using Av1EncoderBlockWorkspace blockWorkspace = new(Configuration.Default); + using Av1SymbolEncoder symbolEncoder = CreateTileSymbolEncoder( + picture.Picture, + 512); + + Av1TileEncoder tileWriter = new( + symbolEncoder, + source.Frame, + reconstruction.Frame, + picture.Picture, + coefficients, + superblockWorkspace, + blockWorkspace, + effort: 5); + + ref Av1MacroBlockModeInfo targetBlock = ref picture.Picture.GetMacroBlockModeInfo(new Point(2, 2)); + Assert.Equal(expectedMode, targetBlock.Block.Mode); + Assert.Equal( + expectedAngleDelta, + superblockWorkspace.FinalBlocks[3].PredictionUnit.AngleDelta[(int)Av1PlaneType.Y]); + + Av1EncoderTransformBlockState targetState = + coefficients.GetTransformBlockSpan(0, Av1Plane.Y)[12]; + + // Every transform has the same skip cost for this exact-prediction target, so reference enum order + // requires DCT-DCT to win even when the mode-derived first pass used another transform. + Assert.Equal((ushort)0, targetState.EndOfBlock); + Assert.Equal(Av1TransformType.DctDct, targetState.TransformType); + Assert.NotEqual(0, tileWriter.GetTileData(0).Length); + } + + [Theory] + [InlineData((int)Av1ChromaPredictionMode.Vertical, 0, (int)Av1TransformType.AdstDct, (int)Av1ColorFormat.Yuv444)] + [InlineData((int)Av1ChromaPredictionMode.Horizontal, 0, (int)Av1TransformType.DctAdst, (int)Av1ColorFormat.Yuv420)] + [InlineData((int)Av1ChromaPredictionMode.Paeth, 0, (int)Av1TransformType.AdstAdst, (int)Av1ColorFormat.Yuv422)] + [InlineData((int)Av1ChromaPredictionMode.Directional45Degrees, -3, (int)Av1TransformType.DctDct, (int)Av1ColorFormat.Yuv420)] + [InlineData((int)Av1ChromaPredictionMode.Directional135Degrees, 3, (int)Av1TransformType.AdstAdst, (int)Av1ColorFormat.Yuv422)] + [InlineData((int)Av1ChromaPredictionMode.Directional203Degrees, -3, (int)Av1TransformType.DctAdst, (int)Av1ColorFormat.Yuv444)] + public void ProductionTileSelectsChromaModeFromCurrentReconstruction( + int expectedModeValue, + int expectedAngleDelta, + int expectedTransformTypeValue, + int colorFormatValue) + { + const int Width = 16; + const int Height = 16; + const int QIndex = 1; + Av1ChromaPredictionMode expectedMode = (Av1ChromaPredictionMode)expectedModeValue; + Av1TransformType expectedTransformType = (Av1TransformType)expectedTransformTypeValue; + Av1ColorFormat colorFormat = (Av1ColorFormat)colorFormatValue; + bool subsamplingX = colorFormat is Av1ColorFormat.Yuv420 or Av1ColorFormat.Yuv422; + bool subsamplingY = colorFormat == Av1ColorFormat.Yuv420; + int chromaSubsamplingX = subsamplingX ? 1 : 0; + int chromaSubsamplingY = subsamplingY ? 1 : 0; + Av1TransformSize transformSize = Av1BlockSize.Block8x8.GetMaxUvTransformSize( + subsamplingX, + subsamplingY); + + ObuColorConfig colorConfig = new() + { + IsMonochrome = false, + SubSamplingX = subsamplingX, + SubSamplingY = subsamplingY, + BitDepth = Av1BitDepth.EightBit + }; + + using Av1EncoderFrameBuffer source = new( + Configuration.Default, + Width, + Height, + 8, + colorFormat, + chromaSubsamplingX, + chromaSubsamplingY); + + using Av1EncoderFrameBuffer reconstruction = new( + Configuration.Default, + Width, + Height, + 8, + colorFormat, + chromaSubsamplingX, + chromaSubsamplingY); + + FillPlane(source.Frame.CodedView.GetPlane(Av1Plane.Y), (byte)128); + FillChromaModeSelectionPlane( + source.Frame.CodedView.GetPlane(Av1Plane.U), + transformSize, + expectedMode, + expectedAngleDelta); + + FillChromaModeSelectionPlane( + source.Frame.CodedView.GetPlane(Av1Plane.V), + transformSize, + expectedMode, + expectedAngleDelta); + + ClearPlane(reconstruction.Luma); + ClearPlane(Assert.IsType>(reconstruction.ChromaBlue)); + ClearPlane(Assert.IsType>(reconstruction.ChromaRed)); + using Av1EncoderModeInfoBuffer modeInfo = new(Configuration.Default, Width, Height, disallow4x4AllFrames: true); + Av1PictureControlSet pictureTemplate = CreatePicture(modeInfo, colorConfig, use128x128Superblock: false, QIndex); + using Av1EncoderPictureBuffer picture = new( + Configuration.Default, + pictureTemplate.Sequence.SequenceHeader, + pictureTemplate.Parent.FrameHeader, + Width, + Height, + disallow4x4AllFrames: true); + + using Av1EncoderCoefficientBuffer coefficients = new( + Configuration.Default, + pictureTemplate.Sequence.SequenceHeader, + Width, + Height); + + using Av1EncoderSuperblockWorkspace superblockWorkspace = new(Configuration.Default); + using Av1EncoderBlockWorkspace blockWorkspace = new(Configuration.Default); + using Av1SymbolEncoder symbolEncoder = CreateTileSymbolEncoder( + picture.Picture, + 512); + + Av1TileEncoder tileWriter = new( + symbolEncoder, + source.Frame, + reconstruction.Frame, + picture.Picture, + coefficients, + superblockWorkspace, + blockWorkspace, + effort: 5); + + ref Av1MacroBlockModeInfo targetBlock = ref picture.Picture.GetMacroBlockModeInfo(new Point(2, 2)); + Assert.Equal(expectedMode, targetBlock.Block.UvMode); + Assert.Equal( + expectedAngleDelta, + superblockWorkspace.FinalBlocks[3].PredictionUnit.AngleDelta[(int)Av1PlaneType.Uv]); + + int targetTransformIndex = (3 * transformSize.GetSize2d()) / + Av1EncoderCoefficientBuffer.TransformBlockUnitCoefficientCount; + + Av1EncoderTransformBlockState blueState = + coefficients.GetTransformBlockSpan(0, Av1Plane.U)[targetTransformIndex]; + + Av1EncoderTransformBlockState redState = + coefficients.GetTransformBlockSpan(0, Av1Plane.V)[targetTransformIndex]; + + Assert.NotEqual((ushort)0, blueState.EndOfBlock); + Assert.NotEqual((ushort)0, redState.EndOfBlock); + Assert.Equal(expectedTransformType, blueState.TransformType); + Assert.Equal(expectedTransformType, redState.TransformType); + Assert.NotEqual(0, tileWriter.GetTileData(0).Length); + } + + [Theory] + [InlineData((int)Av1ColorFormat.Yuv420)] + [InlineData((int)Av1ColorFormat.Yuv422)] + [InlineData((int)Av1ColorFormat.Yuv444)] + public void ProductionTileSelectsChromaFromReconstructedLuma(int colorFormatValue) + => VerifyProductionTileSelectsChromaFromReconstructedLuma( + colorFormatValue, + 8, + static (writer, source, reconstruction, picture, coefficients, superblockWorkspace, blockWorkspace) => + new( + writer, + source, + reconstruction, + picture, + coefficients, + superblockWorkspace, + blockWorkspace, + effort: 5)); + + [Theory] + [InlineData((int)Av1ColorFormat.Yuv420, 10)] + [InlineData((int)Av1ColorFormat.Yuv420, 12)] + [InlineData((int)Av1ColorFormat.Yuv422, 10)] + [InlineData((int)Av1ColorFormat.Yuv422, 12)] + [InlineData((int)Av1ColorFormat.Yuv444, 10)] + [InlineData((int)Av1ColorFormat.Yuv444, 12)] + public void ProductionTileSelectsChromaFromReconstructedLumaHighBitDepth( + int colorFormatValue, + int bitDepth) + => VerifyProductionTileSelectsChromaFromReconstructedLuma( + colorFormatValue, + bitDepth, + static (writer, source, reconstruction, picture, coefficients, superblockWorkspace, blockWorkspace) => + new( + writer, + source, + reconstruction, + picture, + coefficients, + superblockWorkspace, + blockWorkspace, + effort: 5)); + + private static void VerifyProductionTileSelectsChromaFromReconstructedLuma( + int colorFormatValue, + int bitDepth, + TileWriterFactory createWriter) + where TSample : unmanaged, IBinaryInteger + { + const int Width = 16; + const int Height = 16; + const int QIndex = 1; + const int TileBufferLength = 512; + const int AlphaU = 16; + const int AlphaV = -16; + Av1ColorFormat colorFormat = (Av1ColorFormat)colorFormatValue; + bool subsamplingX = colorFormat is Av1ColorFormat.Yuv420 or Av1ColorFormat.Yuv422; + bool subsamplingY = colorFormat == Av1ColorFormat.Yuv420; + int chromaSubsamplingX = subsamplingX ? 1 : 0; + int chromaSubsamplingY = subsamplingY ? 1 : 0; + int sampleScale = 1 << (bitDepth - 8); + int midpoint = 1 << (bitDepth - 1); + int maxSample = (1 << bitDepth) - 1; + Av1TransformSize transformSize = Av1BlockSize.Block8x8.GetMaxUvTransformSize( + subsamplingX, + subsamplingY); + + ObuColorConfig colorConfig = new() + { + IsMonochrome = false, + SubSamplingX = subsamplingX, + SubSamplingY = subsamplingY, + BitDepth = (Av1BitDepth)((bitDepth - 8) / 2) + }; + + using Av1EncoderFrameBuffer pilotSource = new( + Configuration.Default, + Width, + Height, + bitDepth, + colorFormat, + chromaSubsamplingX, + chromaSubsamplingY); + + using Av1EncoderFrameBuffer pilotReconstruction = new( + Configuration.Default, + Width, + Height, + bitDepth, + colorFormat, + chromaSubsamplingX, + chromaSubsamplingY); + + Buffer2DRegion pilotLuma = pilotSource.Frame.CodedView.GetPlane(Av1Plane.Y); + for (int y = 0; y < pilotLuma.Height; y++) + { + Span row = pilotLuma.DangerousGetRowSpan(y); + for (int x = 0; x < row.Length; x++) + { + row[x] = TSample.CreateChecked( + (96 + (((x * 29) + (y * 47) + (((x ^ y) & 1) * 53)) & 63)) * sampleScale); + } + } + + FillPlane(pilotSource.Frame.CodedView.GetPlane(Av1Plane.U), TSample.CreateChecked(midpoint)); + FillPlane(pilotSource.Frame.CodedView.GetPlane(Av1Plane.V), TSample.CreateChecked(midpoint)); + ClearPlane(pilotReconstruction.Luma); + ClearPlane(Assert.IsType>(pilotReconstruction.ChromaBlue)); + ClearPlane(Assert.IsType>(pilotReconstruction.ChromaRed)); + using Av1EncoderModeInfoBuffer pilotModeInfo = new(Configuration.Default, Width, Height, disallow4x4AllFrames: true); + Av1PictureControlSet pilotTemplate = CreatePicture(pilotModeInfo, colorConfig, use128x128Superblock: false, QIndex); + using Av1EncoderPictureBuffer pilotPicture = new( + Configuration.Default, + pilotTemplate.Sequence.SequenceHeader, + pilotTemplate.Parent.FrameHeader, + Width, + Height, + disallow4x4AllFrames: true); + + using Av1EncoderCoefficientBuffer pilotCoefficients = new( + Configuration.Default, + pilotTemplate.Sequence.SequenceHeader, + Width, + Height); + + using Av1EncoderSuperblockWorkspace pilotSuperblockWorkspace = new(Configuration.Default); + using Av1EncoderBlockWorkspace pilotBlockWorkspace = new(Configuration.Default); + using Av1SymbolEncoder pilotSymbolEncoder = CreateTileSymbolEncoder( + pilotPicture.Picture, + TileBufferLength); + + Av1TileEncoder pilotWriter = createWriter( + pilotSymbolEncoder, + pilotSource.Frame, + pilotReconstruction.Frame, + pilotPicture.Picture, + pilotCoefficients, + pilotSuperblockWorkspace, + pilotBlockWorkspace); + + Buffer2DRegion reconstructedLuma = pilotReconstruction.Frame.CodedView.GetPlane(Av1Plane.Y); + int chromaWidth = transformSize.GetWidth(); + int chromaHeight = transformSize.GetHeight(); + int sampleCount = transformSize.GetSize2d(); + int lumaScaleShift = 3 - chromaSubsamplingX - chromaSubsamplingY; + Span lumaQ3 = stackalloc short[64]; + int sumQ3 = sampleCount >> 1; + for (int row = 0; row < chromaHeight; row++) + { + for (int column = 0; column < chromaWidth; column++) + { + int lumaSum = 0; + int lumaX = 8 + (column << chromaSubsamplingX); + int lumaY = 8 + (row << chromaSubsamplingY); + for (int offsetY = 0; offsetY <= chromaSubsamplingY; offsetY++) + { + ReadOnlySpan lumaRow = reconstructedLuma.DangerousGetRowSpan(lumaY + offsetY); + for (int offsetX = 0; offsetX <= chromaSubsamplingX; offsetX++) + { + lumaSum += int.CreateChecked(lumaRow[lumaX + offsetX]); + } + } + + short sampleQ3 = (short)(lumaSum << lumaScaleShift); + lumaQ3[(row * chromaWidth) + column] = sampleQ3; + sumQ3 += sampleQ3; + } + } + + int averageQ3 = sumQ3 >> (transformSize.GetBlockWidthLog2() + transformSize.GetBlockHeightLog2()); + using Av1EncoderFrameBuffer source = new( + Configuration.Default, + Width, + Height, + bitDepth, + colorFormat, + chromaSubsamplingX, + chromaSubsamplingY); + + using Av1EncoderFrameBuffer reconstruction = new( + Configuration.Default, + Width, + Height, + bitDepth, + colorFormat, + chromaSubsamplingX, + chromaSubsamplingY); + + for (int y = 0; y < pilotLuma.Height; y++) + { + pilotLuma.DangerousGetRowSpan(y).CopyTo(source.Frame.CodedView.GetPlane(Av1Plane.Y).DangerousGetRowSpan(y)); + } + + Buffer2DRegion blue = source.Frame.CodedView.GetPlane(Av1Plane.U); + Buffer2DRegion red = source.Frame.CodedView.GetPlane(Av1Plane.V); + FillPlane(blue, TSample.CreateChecked(midpoint)); + FillPlane(red, TSample.CreateChecked(midpoint)); + for (int row = 0; row < chromaHeight; row++) + { + Span blueRow = blue.DangerousGetRowSpan(chromaHeight + row); + Span redRow = red.DangerousGetRowSpan(chromaHeight + row); + for (int column = 0; column < chromaWidth; column++) + { + int acQ3 = lumaQ3[(row * chromaWidth) + column] - averageQ3; + int blueProduct = AlphaU * acQ3; + int redProduct = AlphaV * acQ3; + int blueAdjustment = (blueProduct + 32 + (blueProduct >> 31)) >> 6; + int redAdjustment = (redProduct + 32 + (redProduct >> 31)) >> 6; + blueRow[chromaWidth + column] = TSample.CreateChecked(Math.Clamp(midpoint + blueAdjustment, 0, maxSample)); + redRow[chromaWidth + column] = TSample.CreateChecked(Math.Clamp(midpoint + redAdjustment, 0, maxSample)); + } + } + + ClearPlane(reconstruction.Luma); + ClearPlane(Assert.IsType>(reconstruction.ChromaBlue)); + ClearPlane(Assert.IsType>(reconstruction.ChromaRed)); + using Av1EncoderModeInfoBuffer modeInfo = new(Configuration.Default, Width, Height, disallow4x4AllFrames: true); + Av1PictureControlSet pictureTemplate = CreatePicture(modeInfo, colorConfig, use128x128Superblock: false, QIndex); + using Av1EncoderPictureBuffer picture = new( + Configuration.Default, + pictureTemplate.Sequence.SequenceHeader, + pictureTemplate.Parent.FrameHeader, + Width, + Height, + disallow4x4AllFrames: true); + + using Av1EncoderCoefficientBuffer coefficients = new( + Configuration.Default, + pictureTemplate.Sequence.SequenceHeader, + Width, + Height); + + using Av1EncoderSuperblockWorkspace superblockWorkspace = new(Configuration.Default); + using Av1EncoderBlockWorkspace blockWorkspace = new(Configuration.Default); + using Av1SymbolEncoder symbolEncoder = CreateTileSymbolEncoder( + picture.Picture, + TileBufferLength); + + Av1TileEncoder tileWriter = createWriter( + symbolEncoder, + source.Frame, + reconstruction.Frame, + picture.Picture, + coefficients, + superblockWorkspace, + blockWorkspace); + + Buffer2DRegion actualLuma = reconstruction.Frame.CodedView.GetPlane(Av1Plane.Y); + for (int y = 0; y < reconstructedLuma.Height; y++) + { + Assert.Equal(reconstructedLuma.DangerousGetRowSpan(y), actualLuma.DangerousGetRowSpan(y)); + } + + ref Av1MacroBlockModeInfo targetBlock = ref picture.Picture.GetMacroBlockModeInfo(new Point(2, 2)); + Assert.Equal(Av1ChromaPredictionMode.ChromaFromLuma, targetBlock.Block.UvMode); + Assert.Equal( + Av1ChromaFromLumaMath.JointSign( + Av1ChromaFromLumaMath.SignPositive, + Av1ChromaFromLumaMath.SignNegative), + superblockWorkspace.FinalBlocks[3].PredictionUnit.ChromaFromLumaSigns); + + Assert.Equal( + Av1ChromaFromLumaMath.PackIndices( + Av1ChromaFromLumaMath.AlphaToMagnitudeIndex(AlphaU), + Av1ChromaFromLumaMath.AlphaToMagnitudeIndex(AlphaV)), + superblockWorkspace.FinalBlocks[3].PredictionUnit.ChromaFromLumaIndex); + + int targetTransformIndex = (3 * sampleCount) / + Av1EncoderCoefficientBuffer.TransformBlockUnitCoefficientCount; + + Av1EncoderTransformBlockState blueState = + coefficients.GetTransformBlockSpan(0, Av1Plane.U)[targetTransformIndex]; + + Av1EncoderTransformBlockState redState = + coefficients.GetTransformBlockSpan(0, Av1Plane.V)[targetTransformIndex]; + + Assert.Equal((ushort)0, blueState.EndOfBlock); + Assert.Equal((ushort)0, redState.EndOfBlock); + Assert.Equal(Av1TransformType.DctDct, blueState.TransformType); + Assert.Equal(Av1TransformType.DctDct, redState.TransformType); + Assert.NotEqual(0, pilotWriter.GetTileData(0).Length); + Assert.NotEqual(0, tileWriter.GetTileData(0).Length); + } + + [Theory] + [InlineData((int)Av1FilterIntraMode.DC)] + [InlineData((int)Av1FilterIntraMode.Vertical)] + [InlineData((int)Av1FilterIntraMode.Horizontal)] + [InlineData((int)Av1FilterIntraMode.Directional157)] + [InlineData((int)Av1FilterIntraMode.Paeth)] + public void ProductionTileSelectsFilterIntraMode(int filterIntraModeValue) + => VerifyProductionTileSelectsFilterIntraMode( + filterIntraModeValue, + 8, + false, + static (writer, source, reconstruction, picture, coefficients, superblockWorkspace, blockWorkspace) => + new( + writer, + source, + reconstruction, + picture, + coefficients, + superblockWorkspace, + blockWorkspace, + effort: 5), + static (mode, destination, stride, above, left, width, height, _, scratch) => + Av1FilterIntraPredictorBase.GetPredictor(mode) + .Predict(destination, stride, above, left, width, height, scratch)); + + [Theory] + [InlineData((int)Av1FilterIntraMode.DC, 10)] + [InlineData((int)Av1FilterIntraMode.DC, 12)] + [InlineData((int)Av1FilterIntraMode.Vertical, 10)] + [InlineData((int)Av1FilterIntraMode.Vertical, 12)] + [InlineData((int)Av1FilterIntraMode.Horizontal, 10)] + [InlineData((int)Av1FilterIntraMode.Horizontal, 12)] + [InlineData((int)Av1FilterIntraMode.Directional157, 10)] + [InlineData((int)Av1FilterIntraMode.Directional157, 12)] + [InlineData((int)Av1FilterIntraMode.Paeth, 10)] + [InlineData((int)Av1FilterIntraMode.Paeth, 12)] + public void ProductionTileSelectsFilterIntraModeHighBitDepth( + int filterIntraModeValue, + int bitDepth) + => VerifyProductionTileSelectsFilterIntraMode( + filterIntraModeValue, + bitDepth, + false, + static (writer, source, reconstruction, picture, coefficients, superblockWorkspace, blockWorkspace) => + new( + writer, + source, + reconstruction, + picture, + coefficients, + superblockWorkspace, + blockWorkspace, + effort: 5), + static (mode, destination, stride, above, left, width, height, sampleBitDepth, scratch) => + Av1FilterIntraPredictorBase.GetPredictor(mode) + .Predict( + MemoryMarshal.Cast(destination), + stride, + MemoryMarshal.Cast(above), + MemoryMarshal.Cast(left), + width, + height, + sampleBitDepth, + MemoryMarshal.Cast(scratch))); + + [Fact] + public void ProductionTileSelectsFilterIntraWithFourByFourTransforms() + => VerifyProductionTileSelectsFilterIntraMode( + (int)Av1FilterIntraMode.DC, + 8, + true, + static (writer, source, reconstruction, picture, coefficients, superblockWorkspace, blockWorkspace) => + new( + writer, + source, + reconstruction, + picture, + coefficients, + superblockWorkspace, + blockWorkspace, + effort: 6), + static (mode, destination, stride, above, left, width, height, _, scratch) => + Av1FilterIntraPredictorBase.GetPredictor(mode) + .Predict(destination, stride, above, left, width, height, scratch)); + + [Theory] + [InlineData(10)] + [InlineData(12)] + public void ProductionTileSelectsFilterIntraWithFourByFourTransformsHighBitDepth(int bitDepth) + => VerifyProductionTileSelectsFilterIntraMode( + (int)Av1FilterIntraMode.DC, + bitDepth, + true, + static (writer, source, reconstruction, picture, coefficients, superblockWorkspace, blockWorkspace) => + new( + writer, + source, + reconstruction, + picture, + coefficients, + superblockWorkspace, + blockWorkspace, + effort: 6), + static (mode, destination, stride, above, left, width, height, sampleBitDepth, scratch) => + Av1FilterIntraPredictorBase.GetPredictor(mode) + .Predict( + MemoryMarshal.Cast(destination), + stride, + MemoryMarshal.Cast(above), + MemoryMarshal.Cast(left), + width, + height, + sampleBitDepth, + MemoryMarshal.Cast(scratch))); + + private static void VerifyProductionTileSelectsFilterIntraMode( + int filterIntraModeValue, + int bitDepth, + bool useSplitTransform, + TileWriterFactory createWriter, + FilterPrediction predictFilter) + where TSample : unmanaged, IBinaryInteger + { + const int Width = 16; + const int Height = 16; + const int QIndex = 37; + const int TileBufferLength = 512; + const int TargetX = 8; + const int TargetY = 8; + const Av1TransformSize TransformSize = Av1TransformSize.Size8x8; + Av1FilterIntraMode filterIntraMode = (Av1FilterIntraMode)filterIntraModeValue; + int sampleScale = 1 << (bitDepth - 8); + ObuColorConfig colorConfig = new() + { + IsMonochrome = true, + SubSamplingX = true, + SubSamplingY = true, + BitDepth = (Av1BitDepth)((bitDepth - 8) / 2) + }; + + using Av1EncoderFrameBuffer pilotSource = new( + Configuration.Default, + Width, + Height, + bitDepth, + Av1ColorFormat.Yuv400, + 1, + 1); + + using Av1EncoderFrameBuffer pilotReconstruction = new( + Configuration.Default, + Width, + Height, + bitDepth, + Av1ColorFormat.Yuv400, + 1, + 1); + + Buffer2DRegion pilotLuma = pilotSource.Frame.CodedView.GetPlane(Av1Plane.Y); + for (int y = 0; y < pilotLuma.Height; y++) + { + Span row = pilotLuma.DangerousGetRowSpan(y); + for (int x = 0; x < row.Length; x++) + { + row[x] = TSample.CreateChecked( + (64 + (((x * 71) + (y * 109) + (((x ^ y) & 3) * 37)) & 127)) * sampleScale); + } + } + + ClearPlane(pilotReconstruction.Luma); + using Av1EncoderModeInfoBuffer pilotModeInfo = new(Configuration.Default, Width, Height, disallow4x4AllFrames: true); + Av1PictureControlSet pilotTemplate = CreatePicture(pilotModeInfo, colorConfig, use128x128Superblock: false, QIndex); + pilotTemplate.Sequence.SequenceHeader.EnableFilterIntra = true; + pilotTemplate.Parent.FrameHeader.TransformMode = useSplitTransform + ? Av1TransformMode.Select + : Av1TransformMode.Largest; + + using Av1EncoderPictureBuffer pilotPicture = new( + Configuration.Default, + pilotTemplate.Sequence.SequenceHeader, + pilotTemplate.Parent.FrameHeader, + Width, + Height, + disallow4x4AllFrames: true); + + using Av1EncoderCoefficientBuffer pilotCoefficients = new( + Configuration.Default, + pilotTemplate.Sequence.SequenceHeader, + Width, + Height); + + using Av1EncoderSuperblockWorkspace pilotSuperblockWorkspace = new(Configuration.Default); + using Av1EncoderBlockWorkspace pilotBlockWorkspace = new(Configuration.Default); + using Av1SymbolEncoder pilotSymbolEncoder = CreateTileSymbolEncoder( + pilotPicture.Picture, + TileBufferLength); + + Av1TileEncoder pilotWriter = createWriter( + pilotSymbolEncoder, + pilotSource.Frame, + pilotReconstruction.Frame, + pilotPicture.Picture, + pilotCoefficients, + pilotSuperblockWorkspace, + pilotBlockWorkspace); + + Buffer2DRegion reconstructedLuma = pilotReconstruction.Frame.CodedView.GetPlane(Av1Plane.Y); + Span aboveStorage = stackalloc TSample[9]; + Span above = aboveStorage[1..]; + Span left = stackalloc TSample[8]; + ReadOnlySpan reconstructedAbove = reconstructedLuma.DangerousGetRowSpan(TargetY - 1); + aboveStorage[0] = reconstructedAbove[TargetX - 1]; + reconstructedAbove.Slice(TargetX, 8).CopyTo(above); + for (int row = 0; row < 8; row++) + { + left[row] = reconstructedLuma.DangerousGetRowSpan(TargetY + row)[TargetX - 1]; + } + + Span target = stackalloc TSample[TransformSize.GetSize2d()]; + Span filterScratch = stackalloc TSample[Av1FilterIntraPredictorBase.ScratchLength]; + if (useSplitTransform) + { + Span transformAboveStorage = stackalloc TSample[5]; + Span transformAbove = transformAboveStorage[1..]; + Span transformLeft = stackalloc TSample[4]; + for (int transformRow = 0; transformRow < 2; transformRow++) + { + int rowOffset = transformRow * 4; + for (int transformColumn = 0; transformColumn < 2; transformColumn++) + { + int columnOffset = transformColumn * 4; + ReadOnlySpan availableAbove = transformRow == 0 + ? above.Slice(columnOffset, 4) + : target.Slice(((rowOffset - 1) * 8) + columnOffset, 4); + + // The predictor consumes the corner through the element immediately before the top-edge span. + // Later transforms therefore use already reconstructed samples from the same 8-by-8 block. + transformAboveStorage[0] = transformRow == 0 + ? transformColumn == 0 ? aboveStorage[0] : above[columnOffset - 1] + : transformColumn == 0 ? left[rowOffset - 1] : target[((rowOffset - 1) * 8) + columnOffset - 1]; + + availableAbove.CopyTo(transformAbove); + for (int row = 0; row < 4; row++) + { + transformLeft[row] = transformColumn == 0 + ? left[rowOffset + row] + : target[((rowOffset + row) * 8) + columnOffset - 1]; + } + + int destinationOffset = (rowOffset * 8) + columnOffset; + predictFilter( + filterIntraMode, + target[destinationOffset..], + 8, + transformAbove, + transformLeft, + 4, + 4, + bitDepth, + filterScratch); + } + } + } + else + { + predictFilter(filterIntraMode, target, 8, above, left, 8, 8, bitDepth, filterScratch); + } + + if (useSplitTransform) + { + for (int transformRow = 0; transformRow < 2; transformRow++) + { + for (int transformColumn = 0; transformColumn < 2; transformColumn++) + { + int transformIndex = (transformRow * 2) + transformColumn; + + // Distinct transform-local frequency patterns remain compact in separate 4-by-4 bases but spread + // across coefficients when a single 8-by-8 transform spans the discontinuities between quadrants. + for (int row = 0; row < 4; row++) + { + Span targetRow = target.Slice( + (((transformRow * 4) + row) * 8) + (transformColumn * 4), + 4); + + for (int column = 0; column < targetRow.Length; column++) + { + int residualSign = transformIndex switch + { + 0 => row < 2 ? -1 : 1, + 1 => column < 2 ? -1 : 1, + 2 => (row < 2) == (column < 2) ? -1 : 1, + _ => ((row + column) & 1) == 0 ? -1 : 1 + }; + + targetRow[column] = TSample.CreateChecked( + int.CreateChecked(targetRow[column]) + (residualSign * 40 * sampleScale)); + } + } + } + } + } + + using Av1EncoderFrameBuffer source = new( + Configuration.Default, + Width, + Height, + bitDepth, + Av1ColorFormat.Yuv400, + 1, + 1); + + using Av1EncoderFrameBuffer reconstruction = new( + Configuration.Default, + Width, + Height, + bitDepth, + Av1ColorFormat.Yuv400, + 1, + 1); + + Buffer2DRegion sourceLuma = source.Frame.CodedView.GetPlane(Av1Plane.Y); + for (int y = 0; y < pilotLuma.Height; y++) + { + pilotLuma.DangerousGetRowSpan(y).CopyTo(sourceLuma.DangerousGetRowSpan(y)); + } + + for (int row = 0; row < 8; row++) + { + target.Slice(row * 8, 8).CopyTo(sourceLuma.DangerousGetRowSpan(TargetY + row).Slice(TargetX, 8)); + } + + ClearPlane(reconstruction.Luma); + using Av1EncoderModeInfoBuffer modeInfo = new(Configuration.Default, Width, Height, disallow4x4AllFrames: true); + Av1PictureControlSet pictureTemplate = CreatePicture(modeInfo, colorConfig, use128x128Superblock: false, QIndex); + pictureTemplate.Sequence.SequenceHeader.EnableFilterIntra = true; + pictureTemplate.Parent.FrameHeader.TransformMode = useSplitTransform + ? Av1TransformMode.Select + : Av1TransformMode.Largest; + + using Av1EncoderPictureBuffer picture = new( + Configuration.Default, + pictureTemplate.Sequence.SequenceHeader, + pictureTemplate.Parent.FrameHeader, + Width, + Height, + disallow4x4AllFrames: true); + + using Av1EncoderCoefficientBuffer coefficients = new( + Configuration.Default, + pictureTemplate.Sequence.SequenceHeader, + Width, + Height); + + using Av1EncoderSuperblockWorkspace superblockWorkspace = new(Configuration.Default); + using Av1EncoderBlockWorkspace blockWorkspace = new(Configuration.Default); + using Av1SymbolEncoder symbolEncoder = CreateTileSymbolEncoder( + picture.Picture, + TileBufferLength); + + Av1TileEncoder tileWriter = createWriter( + symbolEncoder, + source.Frame, + reconstruction.Frame, + picture.Picture, + coefficients, + superblockWorkspace, + blockWorkspace); + + ref Av1MacroBlockModeInfo targetBlock = ref picture.Picture.GetMacroBlockModeInfo(new Point(2, 2)); + Assert.Equal(Av1PredictionMode.DC, targetBlock.Block.Mode); + Assert.Equal(filterIntraMode, superblockWorkspace.FinalBlocks[3].FilterIntraMode); + Assert.Equal( + useSplitTransform ? Av1TransformSize.Size4x4 : Av1TransformSize.Size8x8, + targetBlock.Block.TransformSize); + + int targetTransformIndex = (3 * TransformSize.GetSize2d()) / + Av1EncoderCoefficientBuffer.TransformBlockUnitCoefficientCount; + + int targetTransformCount = useSplitTransform ? 4 : 1; + Span targetStates = coefficients + .GetTransformBlockSpan(0, Av1Plane.Y) + .Slice(targetTransformIndex, targetTransformCount); + + foreach (Av1EncoderTransformBlockState targetState in targetStates) + { + if (useSplitTransform) + { + Assert.NotEqual((ushort)0, targetState.EndOfBlock); + } + else + { + Assert.Equal((ushort)0, targetState.EndOfBlock); + Assert.Equal(Av1TransformType.DctDct, targetState.TransformType); + } + } + + Buffer2DRegion actualLuma = reconstruction.Frame.CodedView.GetPlane(Av1Plane.Y); + Assert.Equal(above, actualLuma.DangerousGetRowSpan(TargetY - 1).Slice(TargetX, 8)); + long reconstructionError = 0; + for (int row = 0; row < 8; row++) + { + Assert.Equal(left[row], actualLuma.DangerousGetRowSpan(TargetY + row)[TargetX - 1]); + ReadOnlySpan targetRow = target.Slice(row * 8, 8); + ReadOnlySpan actualRow = actualLuma.DangerousGetRowSpan(TargetY + row).Slice(TargetX, 8); + if (useSplitTransform) + { + for (int column = 0; column < targetRow.Length; column++) + { + long difference = long.CreateChecked(targetRow[column]) - long.CreateChecked(actualRow[column]); + reconstructionError += difference * difference; + } + } + else + { + Assert.Equal(targetRow, actualRow); + } + } + + if (useSplitTransform) + { + // The chosen transforms must reduce the source error below leaving the known residual entirely uncoded. + long predictionOnlyError = 64L * 40 * 40 * sampleScale * sampleScale; + Assert.InRange(reconstructionError, 1, predictionOnlyError - 1); + + byte[] payload = WriteCompleteTileObu(pictureTemplate, tileWriter, Width, Height); + using Av1Decoder decoder = new(Configuration.Default); + using Image decoded = decoder.Decode(payload); + Assert.NotNull(decoder.FrameInfo); + Av1BlockModeInfo decodedBlock = decoder.FrameInfo.GetModeInfoAt(new Point(2, 2)); + Assert.True(decodedBlock.UseFilterIntra); + Assert.Equal(filterIntraMode, decodedBlock.FilterIntraMode); + Assert.Equal(4, decodedBlock.GetTransformUnitCount(Av1Plane.Y)); + Assert.Equal(new Size(Width, Height), decoded.Size); + + string outputDirectory = Path.Combine( + TestEnvironment.ActualOutputDirectoryFullPath, + "Formats", + "Heif", + "Av1"); + + Directory.CreateDirectory(outputDirectory); + File.WriteAllBytes( + Path.Combine(outputDirectory, $"encoder-filter-intra-transform-size-select-{bitDepth}b.obu"), + payload); + } + + Assert.NotEqual(0, pilotWriter.GetTileData(0).Length); + Assert.NotEqual(0, tileWriter.GetTileData(0).Length); + } + + [Fact] + public void ProductionDirectionalModesConsumeAvailableExtendedEdges() + { + const int Width = 72; + const int Height = 16; + const int QIndex = 1; + ObuColorConfig colorConfig = new() + { + IsMonochrome = true, + SubSamplingX = true, + SubSamplingY = true, + BitDepth = Av1BitDepth.EightBit + }; + + using Av1EncoderFrameBuffer source = new( + Configuration.Default, + Width, + Height, + 8, + Av1ColorFormat.Yuv400, + 0, + 0); + + using Av1EncoderFrameBuffer reconstruction = new( + Configuration.Default, + Width, + Height, + 8, + Av1ColorFormat.Yuv400, + 0, + 0); + + Buffer2DRegion sourcePlane = source.Frame.CodedView.GetPlane(Av1Plane.Y); + FillPlane(sourcePlane, (byte)128); + + Span aboveStorage = stackalloc byte[17]; + Span above = aboveStorage[1..]; + Span leftStorage = stackalloc byte[17]; + Span left = leftStorage[1..]; + aboveStorage[0] = 128; + leftStorage[0] = 128; + for (int i = 0; i < 16; i++) + { + above[i] = (byte)(32 + (i * 12)); + left[i] = (byte)(224 - (i * 12)); + } + + Span topRightTarget = stackalloc byte[64]; + Span bottomLeftTarget = stackalloc byte[64]; + Span predictionScratch = stackalloc byte[64]; + Av1DirectionalIntraPredictor.Predict( + topRightTarget, + 8, + Av1TransformSize.Size8x8, + above, + left, + false, + false, + 45, + predictionScratch); + + Av1DirectionalIntraPredictor.Predict( + bottomLeftTarget, + 8, + Av1TransformSize.Size8x8, + above, + left, + false, + false, + 203, + predictionScratch); + + // The lower-left target consumes top-right samples from the already reconstructed row above. + // The upper-right superblock target consumes bottom-left samples from the completed superblock to its left. + for (int y = 0; y < Height; y++) + { + Span row = sourcePlane.DangerousGetRowSpan(y); + if (y < 8) + { + above.CopyTo(row[..16]); + bottomLeftTarget.Slice(y * 8, 8).CopyTo(row.Slice(64, 8)); + } + else + { + topRightTarget.Slice((y - 8) * 8, 8).CopyTo(row[..8]); + } + + row.Slice(56, 8).Fill(left[y]); + } + + ClearPlane(reconstruction.Luma); + using Av1EncoderModeInfoBuffer modeInfo = new(Configuration.Default, Width, Height, disallow4x4AllFrames: true); + Av1PictureControlSet pictureTemplate = CreatePicture(modeInfo, colorConfig, use128x128Superblock: false, QIndex); + using Av1EncoderPictureBuffer picture = new( + Configuration.Default, + pictureTemplate.Sequence.SequenceHeader, + pictureTemplate.Parent.FrameHeader, + Width, + Height, + disallow4x4AllFrames: true); + + using Av1EncoderCoefficientBuffer coefficients = new( + Configuration.Default, + pictureTemplate.Sequence.SequenceHeader, + Width, + Height); + + using Av1EncoderSuperblockWorkspace superblockWorkspace = new(Configuration.Default); + using Av1EncoderBlockWorkspace blockWorkspace = new(Configuration.Default); + using Av1SymbolEncoder symbolEncoder = CreateTileSymbolEncoder( + picture.Picture, + 2048); + + Av1TileEncoder tileWriter = new( + symbolEncoder, + source.Frame, + reconstruction.Frame, + picture.Picture, + coefficients, + superblockWorkspace, + blockWorkspace, + effort: 5); + + ref Av1MacroBlockModeInfo topRightBlock = ref picture.Picture.GetMacroBlockModeInfo(new Point(0, 2)); + ref Av1MacroBlockModeInfo bottomLeftBlock = ref picture.Picture.GetMacroBlockModeInfo(new Point(16, 0)); + Assert.Equal(Av1PredictionMode.Directional45Degrees, topRightBlock.Block.Mode); + Assert.Equal(Av1PredictionMode.Directional203Degrees, bottomLeftBlock.Block.Mode); + Assert.NotEqual(0, tileWriter.GetTileData(0).Length); + } + + [Fact] + public void ProductionTileSelectsIntraBlockCopyByFullRateDistortion() + { + VerifyProductionTileSelectsIntraBlockCopy( + Av1BitDepth.EightBit, + 8, + static value => (byte)value, + static (writer, source, reconstruction, picture, coefficients, superblockWorkspace, blockWorkspace) => + new Av1TileEncoder( + writer, + source, + reconstruction, + picture, + coefficients, + superblockWorkspace, + blockWorkspace, + effort: 5)); + + VerifyProductionTileSelectsIntraBlockCopy( + Av1BitDepth.TwelveBit, + 12, + static value => (ushort)(value << 4), + static (writer, source, reconstruction, picture, coefficients, superblockWorkspace, blockWorkspace) => + new Av1TileEncoder( + writer, + source, + reconstruction, + picture, + coefficients, + superblockWorkspace, + blockWorkspace, + effort: 5)); + } + + private static void VerifyProductionTileSelectsIntraBlockCopy( + Av1BitDepth bitDepth, + int bitDepthValue, + SampleFactory createSample, + TileWriterFactory createTileWriter) + where TSample : unmanaged + { + const int Width = 328; + const int Height = 8; + const int QIndex = 1; + const int TileBufferLength = 4096; + const int ReferenceColumn = 0; + const int TargetColumn = 320; + ObuColorConfig colorConfig = new() + { + IsMonochrome = true, + SubSamplingX = true, + SubSamplingY = true, + BitDepth = bitDepth + }; + + using Av1EncoderFrameBuffer source = new( + Configuration.Default, + Width, + Height, + bitDepthValue, + Av1ColorFormat.Yuv400, + 0, + 0); + + using Av1EncoderFrameBuffer reconstruction = new( + Configuration.Default, + Width, + Height, + bitDepthValue, + Av1ColorFormat.Yuv400, + 0, + 0); + + Buffer2DRegion sourcePlane = source.Frame.CodedView.GetPlane(Av1Plane.Y); + for (int row = 0; row < Height; row++) + { + Span sourceRow = sourcePlane.DangerousGetRowSpan(row); + for (int column = 0; column < Width; column++) + { + sourceRow[column] = createSample(17 + (((column * 29) + (row * 43)) % 211)); + } + + for (int column = 0; column < 8; column++) + { + // The repeated high-contrast block has one legal hash match five completed 64-pixel regions earlier. + TSample sample = createSample(((column * 73) + (row * 109) + (((column + row) & 1) * 127)) & 255); + sourceRow[ReferenceColumn + column] = sample; + sourceRow[TargetColumn + column] = sample; + } + } + + ClearPlane(reconstruction.Luma); + using Av1EncoderModeInfoBuffer modeInfo = new( + Configuration.Default, + Width, + Height, + disallow4x4AllFrames: true); + + Av1PictureControlSet pictureTemplate = CreatePicture( + modeInfo, + colorConfig, + use128x128Superblock: false, + QIndex); + + pictureTemplate.Parent.FrameHeader.AllowScreenContentTools = true; + pictureTemplate.Parent.FrameHeader.AllowIntraBlockCopy = true; + pictureTemplate.Parent.FrameHeader.FrameSize.FrameWidth = Width; + pictureTemplate.Parent.FrameHeader.FrameSize.FrameHeight = Height; + using Av1EncoderPictureBuffer picture = new( + Configuration.Default, + pictureTemplate.Sequence.SequenceHeader, + pictureTemplate.Parent.FrameHeader, + Width, + Height, + disallow4x4AllFrames: true); + + using Av1EncoderCoefficientBuffer coefficients = new( + Configuration.Default, + pictureTemplate.Sequence.SequenceHeader, + Width, + Height); + + using Av1EncoderSuperblockWorkspace superblockWorkspace = new(Configuration.Default); + using Av1EncoderBlockWorkspace blockWorkspace = new(Configuration.Default); + using Av1SymbolEncoder symbolEncoder = CreateTileSymbolEncoder( + picture.Picture, + TileBufferLength); + + Av1TileEncoder tileWriter = createTileWriter( + symbolEncoder, + source.Frame, + reconstruction.Frame, + picture.Picture, + coefficients, + superblockWorkspace, + blockWorkspace); + + Point targetModeInfoPosition = new(TargetColumn >> Av1Constants.ModeInfoSizeLog2, 0); + ref Av1MacroBlockModeInfo targetMode = ref picture.Picture.GetMacroBlockModeInfo(targetModeInfoPosition); + Assert.True(targetMode.Block.UseIntraBlockCopy); + Assert.Equal(Av1PredictionMode.DC, targetMode.Block.Mode); + Assert.Equal(Av1ChromaPredictionMode.DC, targetMode.Block.UvMode); + var displacementVector = picture.Picture.GetDisplacementVector(targetModeInfoPosition); + Assert.Equal(0, displacementVector.Row); + Assert.Equal((ReferenceColumn - TargetColumn) * 8, displacementVector.Column); + Assert.Equal(Av1FilterIntraMode.AllFilterIntraModes, superblockWorkspace.FinalBlocks[0].FilterIntraMode); + Assert.Equal(0, superblockWorkspace.PaletteInfo.PaletteSizes[0]); + Assert.NotEqual(0, tileWriter.GetTileData(0).Length); + } + + [Fact] + public void ProductionTileRetainsHalfSampleChromaIntraBlockCopy() + { + const int Width = 328; + const int Height = 8; + const int QIndex = 1; + const int ReferenceColumn = 1; + const int TargetColumn = 320; + ObuColorConfig colorConfig = new() + { + IsMonochrome = false, + SubSamplingX = true, + SubSamplingY = true, + BitDepth = Av1BitDepth.EightBit + }; + + using Av1EncoderFrameBuffer source = new( + Configuration.Default, + Width, + Height, + 8, + Av1ColorFormat.Yuv420, + 1, + 1); + + using Av1EncoderFrameBuffer reconstruction = new( + Configuration.Default, + Width, + Height, + 8, + Av1ColorFormat.Yuv420, + 1, + 1); + + Buffer2DRegion lumaSource = source.Frame.CodedView.GetPlane(Av1Plane.Y); + for (int row = 0; row < Height; row++) + { + Span lumaRow = lumaSource.DangerousGetRowSpan(row); + for (int column = 0; column < Width; column++) + { + lumaRow[column] = (byte)(23 + (((column * 31) + (row * 47)) % 197)); + } + + for (int column = 0; column < 8; column++) + { + byte sample = (byte)(((column * 79) + (row * 113) + (((column + row) & 1) * 127)) & 255); + lumaRow[ReferenceColumn + column] = sample; + lumaRow[TargetColumn + column] = sample; + } + } + + int chromaTargetColumn = TargetColumn >> 1; + Buffer2DRegion blueSource = source.Frame.CodedView.GetPlane(Av1Plane.U); + Buffer2DRegion redSource = source.Frame.CodedView.GetPlane(Av1Plane.V); + for (int row = 0; row < Height >> 1; row++) + { + Span blueRow = blueSource.DangerousGetRowSpan(row); + Span redRow = redSource.DangerousGetRowSpan(row); + for (int column = 0; column < Width >> 1; column++) + { + blueRow[column] = (byte)(32 + (((column * 17) + (row * 29)) % 160)); + redRow[column] = (byte)(40 + (((column * 23) + (row * 37)) % 152)); + } + + for (int column = 0; column < 4; column++) + { + // An odd luma displacement maps 4:2:0 chroma between adjacent reference samples. + blueRow[chromaTargetColumn + column] = (byte)((blueRow[column] + blueRow[column + 1] + 1) >> 1); + redRow[chromaTargetColumn + column] = (byte)((redRow[column] + redRow[column + 1] + 1) >> 1); + } + } + + ClearPlane(reconstruction.Luma); + ClearPlane(Assert.IsType>(reconstruction.ChromaBlue)); + ClearPlane(Assert.IsType>(reconstruction.ChromaRed)); + using Av1EncoderModeInfoBuffer modeInfo = new( + Configuration.Default, + Width, + Height, + disallow4x4AllFrames: true); + + Av1PictureControlSet pictureTemplate = CreatePicture( + modeInfo, + colorConfig, + use128x128Superblock: false, + QIndex); + + pictureTemplate.Parent.FrameHeader.AllowScreenContentTools = true; + pictureTemplate.Parent.FrameHeader.AllowIntraBlockCopy = true; + pictureTemplate.Parent.FrameHeader.FrameSize.FrameWidth = Width; + pictureTemplate.Parent.FrameHeader.FrameSize.FrameHeight = Height; + using Av1EncoderPictureBuffer picture = new( + Configuration.Default, + pictureTemplate.Sequence.SequenceHeader, + pictureTemplate.Parent.FrameHeader, + Width, + Height, + disallow4x4AllFrames: true); + + using Av1EncoderCoefficientBuffer coefficients = new( + Configuration.Default, + pictureTemplate.Sequence.SequenceHeader, + Width, + Height); + + using Av1EncoderSuperblockWorkspace superblockWorkspace = new(Configuration.Default); + using Av1EncoderBlockWorkspace blockWorkspace = new(Configuration.Default); + using Av1SymbolEncoder symbolEncoder = CreateTileSymbolEncoder( + picture.Picture, + 4096); + + Av1TileEncoder tileWriter = new( + symbolEncoder, + source.Frame, + reconstruction.Frame, + picture.Picture, + coefficients, + superblockWorkspace, + blockWorkspace, + effort: 5); + + Point targetModeInfoPosition = new(TargetColumn >> Av1Constants.ModeInfoSizeLog2, 0); + ref Av1MacroBlockModeInfo targetMode = ref picture.Picture.GetMacroBlockModeInfo(targetModeInfoPosition); + Assert.True(targetMode.Block.UseIntraBlockCopy); + var displacementVector = picture.Picture.GetDisplacementVector(targetModeInfoPosition); + Assert.Equal(0, displacementVector.Row); + Assert.Equal((ReferenceColumn - TargetColumn) * 8, displacementVector.Column); + Assert.Equal(8, displacementVector.Column & 15); + Av1TransformSetType interTransformSet = Av1SymbolContextHelper.GetExtendedTransformSetType( + Av1TransformSize.Size4x4, + isInter: true, + useReducedSet: false); + + Assert.True(coefficients.GetTransformBlockSpan(5, Av1Plane.U)[0].TransformType.IsExtendedSetUsed(interTransformSet)); + Assert.True(coefficients.GetTransformBlockSpan(5, Av1Plane.V)[0].TransformType.IsExtendedSetUsed(interTransformSet)); + Assert.NotEqual( + (byte)0, + reconstruction.Frame.CodedView.GetPlane(Av1Plane.U).DangerousGetRowSpan(0)[chromaTargetColumn]); + + Assert.NotEqual( + (byte)0, + reconstruction.Frame.CodedView.GetPlane(Av1Plane.V).DangerousGetRowSpan(0)[chromaTargetColumn]); + + Assert.NotEqual(0, tileWriter.GetTileData(0).Length); + } + + [Fact] + public void TileWriterMapsClippedRasterTraversalToEverySuperblockCoefficientSegment() + { + const int Width = 72; + const int Height = 72; + const int QIndex = 53; + ObuColorConfig colorConfig = new() + { + IsMonochrome = true, + SubSamplingX = true, + SubSamplingY = true, + BitDepth = Av1BitDepth.EightBit + }; + + ObuTileGroupHeader tiles = new() + { + TileColumnCount = 1, + TileRowCount = 1 + }; + + int modeInfoColumnCount = Width >> Av1Constants.ModeInfoSizeLog2; + int modeInfoRowCount = Height >> Av1Constants.ModeInfoSizeLog2; + tiles.TileColumnStartModeInfo[1] = modeInfoColumnCount; + tiles.TileRowStartModeInfo[1] = modeInfoRowCount; + ObuSequenceHeader sequenceHeader = new() + { + Use128x128Superblock = false, + ColorConfig = colorConfig + }; + + ObuFrameHeader frameHeader = new() + { + ModeInfoColumnCount = modeInfoColumnCount, + ModeInfoRowCount = modeInfoRowCount, + TilesInfo = tiles + }; + + frameHeader.QuantizationParameters.BaseQIndex = QIndex; + frameHeader.QuantizationParameters.QIndex.Fill(QIndex); + using Av1EncoderFrameBuffer source = new( + Configuration.Default, + Width, + Height, + 8, + Av1ColorFormat.Yuv400, + 0, + 0); + + using Av1EncoderFrameBuffer reconstruction = new( + Configuration.Default, + Width, + Height, + 8, + Av1ColorFormat.Yuv400, + 0, + 0); + + FillPlane(source.Frame.CodedView.GetPlane(Av1Plane.Y), 251, 29); + ClearPlane(reconstruction.Luma); + using Av1EncoderPictureBuffer picture = new( + Configuration.Default, + sequenceHeader, + frameHeader, + Width, + Height, + disallow4x4AllFrames: true); + + using Av1EncoderCoefficientBuffer coefficients = new( + Configuration.Default, + sequenceHeader, + Width, + Height); + + using Av1EncoderSuperblockWorkspace superblockWorkspace = new(Configuration.Default); + using Av1EncoderBlockWorkspace blockWorkspace = new(Configuration.Default); + using Av1SymbolEncoder symbolEncoder = CreateTileSymbolEncoder( + picture.Picture, + 4096); + + Av1TileEncoder tileWriter = new( + symbolEncoder, + source.Frame, + reconstruction.Frame, + picture.Picture, + coefficients, + superblockWorkspace, + blockWorkspace, + effort: 5); + + Assert.Equal(4, coefficients.SuperblockCount); + bool usesNonDctTransform = false; + for (int superblockIndex = 0; superblockIndex < coefficients.SuperblockCount; superblockIndex++) + { + Span transformBlocks = + coefficients.GetTransformBlockSpan(superblockIndex, Av1Plane.Y); + + Assert.NotEqual((ushort)0, transformBlocks[0].EndOfBlock); + foreach (Av1EncoderTransformBlockState transformBlock in transformBlocks) + { + usesNonDctTransform |= + transformBlock.EndOfBlock > 0 && transformBlock.TransformType != Av1TransformType.DctDct; + } + } + + Assert.True(usesNonDctTransform); + Assert.NotEqual( + (byte)0, + reconstruction.Frame.CodedView.GetPlane(Av1Plane.Y).DangerousGetRowSpan(Height - 1)[Width - 1]); + + ref Av1MacroBlockModeInfo bottomRight = ref picture.Picture.GetMacroBlockModeInfo(new Point(16, 16)); + Assert.Equal(Av1BlockSize.Block8x8, bottomRight.Block.BlockSize); + Assert.NotEqual(0, tileWriter.GetTileData(0).Length); + } + + /// + /// Verifies that mixed partition trials and final writing retain the decoder's reconstruction order. + /// + [Theory] + [InlineData(false, 32, false)] + [InlineData(true, 32, false)] + [InlineData(false, 56, false)] + [InlineData(true, 56, false)] + [InlineData(false, 32, true)] + [InlineData(true, 32, true)] + [InlineData(false, 56, true)] + [InlineData(true, 56, true)] + public void ProductionMixedPartitionsPreserveReconstructionOrder(bool transpose, int size, bool enableIntraEdgeFilter) + { + const int QIndex = 4; + ObuColorConfig colorConfig = new() + { + IsMonochrome = true, + SubSamplingX = true, + SubSamplingY = true, + BitDepth = Av1BitDepth.EightBit + }; + + using Av1EncoderFrameBuffer source = new(Configuration.Default, size, size, 8, Av1ColorFormat.Yuv400, 0, 0); + using Av1EncoderFrameBuffer reconstruction = new(Configuration.Default, size, size, 8, Av1ColorFormat.Yuv400, 0, 0); + Buffer2DRegion sourcePlane = source.Frame.CodedView.GetPlane(Av1Plane.Y); + for (int y = 0; y < size; y++) + { + for (int x = 0; x < size; x++) + { + // The lower-right quadrant contains two different square surfaces beside one vertical + // surface. Transposition exercises the corresponding horizontal reconstruction order. + int value = x < 16 && y < 16 ? 128 + : y < 16 ? 16 + ((x - 16) * 12) + : x < 16 ? 16 + ((y - 16) * 12) + : x >= 24 ? 16 + ((x - 16) * 12) + : y < 24 ? 16 + ((x + y - 31) * 12) + : 16 + ((x - 8) * 12); + + sourcePlane.DangerousGetRowSpan(transpose ? x : y)[transpose ? y : x] = (byte)value; + } + } + + ClearPlane(reconstruction.Luma); + using Av1EncoderModeInfoBuffer modeInfo = new(Configuration.Default, size, size, disallow4x4AllFrames: false); + Av1PictureControlSet template = CreatePicture(modeInfo, colorConfig, use128x128Superblock: false, QIndex); + template.Sequence.SequenceHeader.EnableIntraEdgeFilter = enableIntraEdgeFilter; + using Av1EncoderPictureBuffer picture = new( + Configuration.Default, template.Sequence.SequenceHeader, template.Parent.FrameHeader, size, size, disallow4x4AllFrames: false); + + using Av1EncoderCoefficientBuffer coefficients = new(Configuration.Default, template.Sequence.SequenceHeader, size, size); + using Av1EncoderSuperblockWorkspace superblockWorkspace = new(Configuration.Default); + using Av1EncoderBlockWorkspace blockWorkspace = new(Configuration.Default); + using Av1SymbolEncoder symbolEncoder = CreateTileSymbolEncoder(picture.Picture, 8192); + Av1TileEncoder tileWriter = new( + symbolEncoder, source.Frame, reconstruction.Frame, picture.Picture, coefficients, superblockWorkspace, blockWorkspace, effort: 9); + + byte[] payload = WriteCompleteTileObu(picture.Picture, tileWriter, size, size); + using Av1Decoder decoder = new(Configuration.Default); + decoder.DecodeSequenceReference(payload, null, null); + Assert.Equal(enableIntraEdgeFilter, Assert.IsType(decoder.SequenceHeader).EnableIntraEdgeFilter); + Av1FrameInfo decodedInfo = Assert.IsType(decoder.FrameInfo); + Av1FrameBuffer decodedFrame = Assert.IsType>(decoder.FrameBuffer); + Buffer2DRegion decodedPlane = decodedFrame.DeriveBlockPointer(Av1Plane.Y, 0, 0); + Buffer2DRegion retainedPlane = reconstruction.Frame.CodedView.GetPlane(Av1Plane.Y); + bool hasMixedPartition = false; + for (int y = 0; y < size; y++) + { + Assert.Equal(retainedPlane.DangerousGetRowSpan(y).ToArray(), decodedPlane.DangerousGetRowSpan(y).ToArray()); + for (int x = 0; x < size; x += 4) + { + Point position = new(x >> 2, y >> 2); + Av1PartitionType partition = decodedInfo.GetModeInfoAt(position).PartitionType; + + // Interior 4x4 entries alias the block origin through the live grid; unused allocation + // slots may still contain rejected trial data and are not retained block state. + int allocationIndex = picture.Picture.ModeInfoGrid.Span[(position.Y * picture.Picture.ModeInfoStride) + position.X]; + Assert.Equal(partition, picture.Picture.ModeInfoAllocation.Span[allocationIndex].Block.PartitionType); + hasMixedPartition |= partition is Av1PartitionType.HorizontalA or Av1PartitionType.HorizontalB + or Av1PartitionType.VerticalA or Av1PartitionType.VerticalB; + } + } + + Assert.True(hasMixedPartition); + string directory = Path.Combine( + TestEnvironment.ActualOutputDirectoryFullPath, "Heif", "Av1", nameof(this.ProductionMixedPartitionsPreserveReconstructionOrder)); + + Directory.CreateDirectory(directory); + File.WriteAllBytes(Path.Combine(directory, $"{size}-{transpose}-{enableIntraEdgeFilter}.obu"), payload); + using FileStream raw = File.Create(Path.Combine(directory, $"{size}-{transpose}-{enableIntraEdgeFilter}.retained.yuv")); + for (int y = 0; y < size; y++) + { + raw.Write(retainedPlane.DangerousGetRowSpan(y)); + } + } + + private static byte[] WriteCompleteTileObu( + Av1PictureControlSet pictureTemplate, + IAv1TileWriter tileWriter, + int width, + int height) + { + // Tile fixtures initialize only entropy state. Complete the same still-picture headers as the frame + // encoder before serializing so independent decoders validate the real OBU syntax. + ObuSequenceHeader sequenceHeader = pictureTemplate.Sequence.SequenceHeader; + ObuColorConfig colorConfig = sequenceHeader.ColorConfig; + Av1ColorFormat colorFormat = colorConfig.GetColorFormat(); + sequenceHeader.IsStillPicture = true; + sequenceHeader.IsReducedStillPictureHeader = true; + sequenceHeader.SequenceProfile = colorConfig.BitDepth == Av1BitDepth.TwelveBit || + colorFormat == Av1ColorFormat.Yuv422 + ? ObuSequenceProfile.Professional + : colorFormat == Av1ColorFormat.Yuv444 + ? ObuSequenceProfile.High + : ObuSequenceProfile.Main; + + sequenceHeader.OperatingPoint = [new ObuOperatingPoint { SequenceLevelIndex = 31 }]; + sequenceHeader.FrameWidthBits = width > 1 ? Av1Math.MostSignificantBit((uint)(width - 1)) + 1 : 1; + sequenceHeader.FrameHeightBits = height > 1 ? Av1Math.MostSignificantBit((uint)(height - 1)) + 1 : 1; + sequenceHeader.MaxFrameWidth = width; + sequenceHeader.MaxFrameHeight = height; + sequenceHeader.ForceScreenContentTools = 2; + sequenceHeader.ForceIntegerMotionVector = 2; + + ObuFrameHeader frameHeader = pictureTemplate.Parent.FrameHeader; + frameHeader.FrameType = ObuFrameType.KeyFrame; + frameHeader.ShowFrame = true; + frameHeader.ErrorResilientMode = true; + frameHeader.RefreshFrameFlags = byte.MaxValue; + frameHeader.DisableFrameEndUpdateCdf = true; + frameHeader.FrameSize = new ObuFrameSize + { + FrameWidth = width, + FrameHeight = height, + SuperResolutionDenominator = Av1Constants.ScaleNumerator, + SuperResolutionUpscaledWidth = width, + RenderWidth = width, + RenderHeight = height + }; + + frameHeader.TilesInfo.HasUniformTileSpacing = true; + using MemoryStream stream = new(); + using ObuWriter obuWriter = new(Configuration.Default); + obuWriter.WriteSequenceFrame( + stream, + sequenceHeader, + frameHeader, + tileWriter); + + return stream.ToArray(); + } + + private static Av1PictureControlSet CreatePicture( + Av1EncoderModeInfoBuffer modeInfo, + ObuColorConfig colorConfig, + bool use128x128Superblock, + int qIndex) + { + ObuTileGroupHeader tiles = new() + { + TileColumnCount = 1, + TileRowCount = 1 + }; + + tiles.TileColumnStartModeInfo[1] = modeInfo.ModeInfoColumnCount; + tiles.TileRowStartModeInfo[1] = modeInfo.ModeInfoRowCount; + ObuSequenceHeader sequenceHeader = new() + { + Use128x128Superblock = use128x128Superblock, + ColorConfig = colorConfig + }; + + ObuFrameHeader frameHeader = new() + { + ModeInfoColumnCount = modeInfo.ModeInfoColumnCount, + ModeInfoRowCount = modeInfo.ModeInfoRowCount, + TilesInfo = tiles + }; + + frameHeader.QuantizationParameters.BaseQIndex = qIndex; + frameHeader.QuantizationParameters.QIndex.Fill(qIndex); + return new Av1PictureControlSet + { + PartitionContexts = [], + LuminanceDcSignLevelCoefficientNeighbors = [], + CrDcSignLevelCoefficientNeighbors = [], + CbDcSignLevelCoefficientNeighbors = [], + TransformFunctionContexts = [], + Sequence = new Av1SequenceControlSet { SequenceHeader = sequenceHeader }, + Parent = new Av1PictureParentControlSet + { + Common = new Av1EncoderCommon + { + ModeInfoColumnCount = modeInfo.ModeInfoColumnCount, + ModeInfoRowCount = modeInfo.ModeInfoRowCount, + ModeInfoStride = modeInfo.ModeInfoStride, + TilesInfo = tiles, + FrameSize = new ObuFrameSize() + }, + FrameHeader = frameHeader, + PreviousQIndex = new int[] { qIndex } + }, + SegmentationNeighborMap = new byte[modeInfo.ModeInfoColumnCount * modeInfo.ModeInfoRowCount], + ModeInfoGrid = modeInfo.Grid, + ModeInfoAllocation = modeInfo.Allocation, + ModeInfoStride = modeInfo.ModeInfoStride, + Disallow4x4AllFrames = modeInfo.Disallow4x4AllFrames, + CdefPreset = new int[] { -1, -1, -1, -1 }, + TileDataOffsets = Memory.Empty, + TileDataLengths = Memory.Empty + }; + } + + private static void AssertProductionTileSelectsExactLumaPalette( + Av1BitDepth bitDepth, + int bitDepthValue, + int width, + int height, + bool useSplitTransform, + TSample lowerColor, + TSample upperColor, + ushort expectedLowerColor, + ushort expectedUpperColor, + TileWriterFactory createTileWriter) + where TSample : unmanaged, IBinaryInteger + { + const int QIndex = 37; + const int TileBufferLength = 256; + ObuColorConfig colorConfig = new() + { + IsMonochrome = true, + SubSamplingX = true, + SubSamplingY = true, + BitDepth = bitDepth + }; + + using Av1EncoderFrameBuffer source = new( + Configuration.Default, + width, + height, + bitDepthValue, + Av1ColorFormat.Yuv400, + 0, + 0); + + using Av1EncoderFrameBuffer reconstruction = new( + Configuration.Default, + width, + height, + bitDepthValue, + Av1ColorFormat.Yuv400, + 0, + 0); + + Buffer2DRegion sourcePlane = source.Frame.CodedView.GetPlane(Av1Plane.Y); + for (int row = 0; row < sourcePlane.Height; row++) + { + int visibleRow = Math.Min(row, height - 1); + Span sourceRow = sourcePlane.DangerousGetRowSpan(row); + if (!useSplitTransform) + { + sourceRow.Fill(visibleRow < height / 2 ? lowerColor : upperColor); + continue; + } + + int transformRow = visibleRow >> 2; + int localRow = visibleRow & 3; + for (int column = 0; column < sourceRow.Length; column++) + { + int transformColumn = column >> 2; + int localColumn = column & 3; + int transformIndex = (transformRow * 2) + transformColumn; + int residual = transformIndex switch + { + 0 => (localRow * 2) - 3, + 1 => (localColumn * 2) - 3, + 2 => (localRow + localColumn) - 3, + _ => localRow - localColumn + }; + + int baseColor = int.CreateChecked(visibleRow < height / 2 ? lowerColor : upperColor); + sourceRow[column] = TSample.CreateChecked( + baseColor + (residual * 4 * (1 << (bitDepthValue - 8)))); + } + } + + ClearPlane(reconstruction.Luma); + using Av1EncoderModeInfoBuffer modeInfo = new( + Configuration.Default, + width, + height, + disallow4x4AllFrames: true); + + Av1PictureControlSet pictureTemplate = CreatePicture( + modeInfo, + colorConfig, + use128x128Superblock: false, + QIndex); + + pictureTemplate.Parent.FrameHeader.AllowScreenContentTools = true; + pictureTemplate.Parent.FrameHeader.TransformMode = useSplitTransform + ? Av1TransformMode.Select + : Av1TransformMode.Largest; + + pictureTemplate.Parent.FrameHeader.FrameSize.FrameWidth = width; + pictureTemplate.Parent.FrameHeader.FrameSize.FrameHeight = height; + using Av1EncoderPictureBuffer picture = new( + Configuration.Default, + pictureTemplate.Sequence.SequenceHeader, + pictureTemplate.Parent.FrameHeader, + width, + height, + disallow4x4AllFrames: true); + + using Av1EncoderCoefficientBuffer coefficients = new( + Configuration.Default, + pictureTemplate.Sequence.SequenceHeader, + width, + height); + + using Av1EncoderSuperblockWorkspace superblockWorkspace = new(Configuration.Default); + using Av1EncoderBlockWorkspace blockWorkspace = new(Configuration.Default); + using Av1SymbolEncoder symbolEncoder = CreateTileSymbolEncoder( + picture.Picture, + TileBufferLength); + + Av1TileEncoder tileWriter = createTileWriter( + symbolEncoder, + source.Frame, + reconstruction.Frame, + picture.Picture, + coefficients, + superblockWorkspace, + blockWorkspace); + + ref Av1MacroBlockModeInfo mode = ref picture.Picture.GetMacroBlockModeInfo(default); + Assert.Equal(Av1PredictionMode.DC, mode.Block.Mode); + Assert.Equal(Av1FilterIntraMode.AllFilterIntraModes, superblockWorkspace.FinalBlocks[0].FilterIntraMode); + Assert.Equal( + useSplitTransform ? Av1TransformSize.Size4x4 : Av1TransformSize.Size8x8, + mode.Block.TransformSize); + + Span transformStates = coefficients + .GetTransformBlockSpan(0, Av1Plane.Y)[..(useSplitTransform ? 4 : 1)]; + + if (useSplitTransform) + { + int coefficientBearingTransformCount = 0; + foreach (Av1EncoderTransformBlockState transformState in transformStates) + { + if (transformState.EndOfBlock > 0) + { + coefficientBearingTransformCount++; + } + } + + Assert.InRange(coefficientBearingTransformCount, 1, transformStates.Length); + Assert.InRange(superblockWorkspace.PaletteInfo.PaletteSizes[0], 2, Av1Constants.PaletteMaxSize); + } + else + { + Assert.Equal((ushort)0, transformStates[0].EndOfBlock); + Assert.Equal(2, superblockWorkspace.PaletteInfo.PaletteSizes[0]); + Assert.Equal( + [expectedLowerColor, expectedUpperColor], + superblockWorkspace.PaletteInfo.GetColors(Av1Plane.Y).ToArray()); + } + + Buffer2DRegion colorIndexMap = superblockWorkspace + .GetPaletteMaps() + .GetMap(Av1PlaneType.Y, 8, 8); + + Buffer2DRegion reconstructionPlane = reconstruction.Frame.CodedView.GetPlane(Av1Plane.Y); + ReadOnlySpan selectedPaletteColors = superblockWorkspace.PaletteInfo.GetColors(Av1Plane.Y); + long predictionOnlyError = 0; + long reconstructionError = 0; + for (int row = 0; row < reconstructionPlane.Height; row++) + { + if (useSplitTransform) + { + ReadOnlySpan sourceRow = sourcePlane.DangerousGetRowSpan(row); + ReadOnlySpan reconstructionRow = reconstructionPlane.DangerousGetRowSpan(row); + ReadOnlySpan mapRow = colorIndexMap.DangerousGetRowSpan(row); + for (int column = 0; column < reconstructionRow.Length; column++) + { + long sourceSample = long.CreateChecked(sourceRow[column]); + long predictionDifference = sourceSample - selectedPaletteColors[mapRow[column]]; + long reconstructionDifference = sourceSample - long.CreateChecked(reconstructionRow[column]); + predictionOnlyError += predictionDifference * predictionDifference; + reconstructionError += reconstructionDifference * reconstructionDifference; + } + } + else + { + int visibleRow = Math.Min(row, height - 1); + byte expectedIndex = (byte)(visibleRow < height / 2 ? 0 : 1); + foreach (byte index in colorIndexMap.DangerousGetRowSpan(row)) + { + Assert.Equal(expectedIndex, index); + } + + Assert.True(sourcePlane.DangerousGetRowSpan(row).SequenceEqual(reconstructionPlane.DangerousGetRowSpan(row))); + } + } + + // Decode every payload, including clipped maps: retained reconstruction alone cannot reveal missing map symbols. + byte[] payload = WriteCompleteTileObu(pictureTemplate, tileWriter, width, height); + using Av1Decoder decoder = new(Configuration.Default); + using Image decoded = decoder.Decode(payload); + Assert.NotNull(decoder.FrameInfo); + Av1BlockModeInfo decodedBlock = decoder.FrameInfo.GetModeInfoAt(default); + Assert.True(decodedBlock.GetPaletteSize(Av1Plane.Y) > 0); + Assert.Equal(new Size(width, height), decoded.Size); + using Av1FrameBuffer decodedFrame = decoder.DecodeFrameBuffer(payload, null, null, out _); + Buffer2DRegion decodedPlane = decodedFrame.DeriveBlockPointer(Av1Plane.Y, 0, 0); + for (int row = 0; row < height; row++) + { + ReadOnlySpan decodedSamples = MemoryMarshal.Cast(decodedPlane.DangerousGetRowSpan(row)); + Assert.Equal(reconstructionPlane.DangerousGetRowSpan(row)[..width], decodedSamples); + } + + if (useSplitTransform) + { + Assert.True(predictionOnlyError > 0); + Assert.True(reconstructionError < predictionOnlyError); + Assert.Equal(4, decodedBlock.GetTransformUnitCount(Av1Plane.Y)); + } + + string outputDirectory = Path.Combine(TestEnvironment.ActualOutputDirectoryFullPath, "Formats", "Heif", "Av1"); + string outputName = useSplitTransform + ? $"encoder-palette-transform-size-select-{bitDepthValue}b" + : $"encoder-palette-luma-{bitDepthValue}b-{width}x{height}"; + + Directory.CreateDirectory(outputDirectory); + File.WriteAllBytes(Path.Combine(outputDirectory, outputName + ".obu"), payload); + using FileStream raw = File.Create(Path.Combine(outputDirectory, outputName + ".retained.yuv")); + for (int row = 0; row < height; row++) + { + raw.Write(MemoryMarshal.AsBytes(reconstructionPlane.DangerousGetRowSpan(row)[..width])); + } + + Assert.NotEqual(0, tileWriter.GetTileData(0).Length); + } + + private static void FillChromaModeSelectionPlane( + Buffer2DRegion plane, + Av1TransformSize transformSize, + Av1ChromaPredictionMode expectedMode, + int expectedAngleDelta) + { + int width = transformSize.GetWidth(); + int height = transformSize.GetHeight(); + Span aboveStorage = stackalloc byte[17]; + Span above = aboveStorage.Slice(1, width * 2); + Span leftStorage = stackalloc byte[17]; + Span left = leftStorage.Slice(1, height * 2); + aboveStorage[0] = 128; + leftStorage[0] = 128; + for (int column = 0; column < width; column++) + { + above[column] = (byte)(32 + ((192 * column) / (width - 1))); + } + + for (int row = 0; row < height; row++) + { + left[row] = (byte)(224 - ((192 * row) / (height - 1))); + } + + above[width..].Fill(above[width - 1]); + left[height..].Fill(left[height - 1]); + Span target = stackalloc byte[64]; + int sampleCount = transformSize.GetSize2d(); + if (expectedMode.IsDirectional()) + { + // Directional arithmetic has separate byte-exact reference coverage. This fixture uses its scalar + // path only to isolate chroma traversal, joint U/V rate-distortion selection, and packed mode state. + Av1DirectionalIntraPredictor.PredictScalar( + target[..sampleCount], + width, + transformSize, + above, + left, + false, + false, + expectedMode.ToLumaMode().ToAngle() + (expectedAngleDelta * Av1Constants.AngleStep)); + } + else + { + // Build the supported non-directional targets directly so production prediction cannot self-validate. + for (int row = 0; row < height; row++) + { + for (int column = 0; column < width; column++) + { + int top = above[column]; + int leftSample = left[row]; + int predictor = top + leftSample - 128; + int leftDistance = Math.Abs(predictor - leftSample); + int topDistance = Math.Abs(predictor - top); + int cornerDistance = Math.Abs(predictor - 128); + target[(row * width) + column] = expectedMode switch + { + Av1ChromaPredictionMode.Vertical => (byte)top, + Av1ChromaPredictionMode.Horizontal => (byte)leftSample, + _ => (byte)(leftDistance <= topDistance && leftDistance <= cornerDistance + ? leftSample + : topDistance <= cornerDistance ? top : 128) + }; + } + } + } + + // The first three transform-sized quadrants establish the references consumed by the bottom-right + // target. Its checkerboard offset keeps coefficients nonzero so the implicit transform affects the stream. + for (int row = 0; row < plane.Height; row++) + { + Span destination = plane.DangerousGetRowSpan(row); + for (int column = 0; column < plane.Width; column++) + { + destination[column] = row < height + ? column < width ? (byte)128 : above[column - width] + : column < width + ? left[row - height] + : (byte)Math.Clamp( + target[((row - height) * width) + column - width] + + ((((row - height) + column - width) & 1) == 0 ? 5 : -5), + 0, + 255); + } + } + } + + private static void FillPlane(Buffer2DRegion plane, int modulus, int seed) + { + for (int y = 0; y < plane.Height; y++) + { + Span row = plane.DangerousGetRowSpan(y); + for (int x = 0; x < row.Length; x++) + { + row[x] = (byte)(1 + ((seed + (x * 43) + (y * 79)) % modulus)); + } + } + } + + private static void FillPlane(Buffer2DRegion plane, TSample value) + where TSample : unmanaged + { + for (int y = 0; y < plane.Height; y++) + { + plane.DangerousGetRowSpan(y).Fill(value); + } + } + + /// + /// Creates the operation owner for a production tile's entropy state and bounded output memory. + /// + /// The picture supplying quantization and CDF-update settings. + /// The bounded output allocation length in bytes. + /// The symbol encoder that must remain alive while the tile output is consumed. + private static Av1SymbolEncoder CreateTileSymbolEncoder(Av1PictureControlSet picture, int bufferLength) + { + ObuFrameHeader frameHeader = picture.Parent.FrameHeader; + return new Av1SymbolEncoder( + Configuration.Default, + bufferLength, + frameHeader.QuantizationParameters.BaseQIndex, + updateCdf: !frameHeader.DisableCdfUpdate); + } + + private delegate Av1TileEncoder TileWriterFactory( + Av1SymbolEncoder writer, + Av1EncoderFrame source, + Av1EncoderFrame reconstruction, + Av1PictureControlSet picture, + Av1EncoderCoefficientBuffer coefficients, + Av1EncoderSuperblockWorkspace superblockWorkspace, + Av1EncoderBlockWorkspace blockWorkspace) + where TSample : unmanaged; + + private delegate TSample SampleFactory(int value) + where TSample : unmanaged; + + private delegate void FilterPrediction( + Av1FilterIntraMode mode, + Span destination, + int destinationStride, + ReadOnlySpan above, + ReadOnlySpan left, + int width, + int height, + int bitDepth, + Span scratch) + where TSample : unmanaged; + + private static void ClearPlane(Buffer2D plane) + where TSample : unmanaged + { + for (int y = 0; y < plane.Height; y++) + { + plane.DangerousGetRowSpan(y).Clear(); + } + } + + private static void AssertContainsNonzero(Buffer2DRegion plane) + where TSample : unmanaged, IEquatable + { + bool containsNonzero = false; + for (int y = 0; y < plane.Height; y++) + { + foreach (TSample sample in plane.DangerousGetRowSpan(y)) + { + containsNonzero |= !sample.Equals(default); + } + } + + Assert.True(containsNonzero); + } + + /// + /// Records the live luma-mode cost while supplying an all-skipped final block. + /// + private struct BlockCostRecorder : Av1TileWriter.IBlockEncodingHandler + { + private readonly int[] costs; + private readonly int qIndex; + + /// + /// Initializes a new instance of the struct. + /// + /// The destination for costs observed in writer order. + /// The block quantizer index. + public BlockCostRecorder(int[] costs, int qIndex) + { + this.costs = costs; + this.qIndex = qIndex; + this.Count = 0; + } + + /// + /// Gets the number of final blocks visited by the writer. + /// + public int Count { get; private set; } + + /// + public readonly Av1PartitionType SelectPartition( + Av1SymbolEncoder writer, + Av1MacroBlockD macroBlock, + Point blockOrigin, + ushort tileIndex, + Av1BlockSize blockSize, + Av1PartitionType preparedPartition) + => preparedPartition; + + /// + public void EncodeBlock( + Av1SymbolEncoder writer, + Av1MacroBlockD macroBlock, + Point blockOrigin, + ushort tileIndex, + ref Av1MacroBlockModeInfo modeInfo, + ref Av1EncoderBlockStruct block, + ref Av1EncoderPaletteInfo paletteInfo) + { + this.costs[this.Count++] = Av1TileWriter.GetLumaModeCost( + writer, + macroBlock, + Av1BlockSize.Block8x8, + Av1PredictionMode.DC, + 0, + isIntraFrame: true); + + modeInfo.Block = new Av1EncoderBlockModeInfo + { + BlockSize = Av1BlockSize.Block8x8, + PartitionType = Av1PartitionType.None, + SegmentId = 0, + Skip = true, + TransformSize = Av1TransformSize.Size8x8, + Mode = Av1PredictionMode.DC, + UvMode = Av1ChromaPredictionMode.DC, + }; + + block.HasChroma = false; + block.QuantizationIndex = this.qIndex; + block.SegmentId = 0; + } + } + + /// + /// Supplies one skipped monochrome palette block to the production tile writer. + /// + private struct PaletteBlockEncoder : Av1TileWriter.IBlockEncodingHandler + { + private readonly Av1EncoderSuperblockWorkspace workspace; + private readonly int qIndex; + private readonly int mapVariant; + + /// + /// Initializes a new instance of the struct. + /// + /// The workspace that owns the palette index map. + /// The block quantizer index. + /// The map pattern selected by the test. + public PaletteBlockEncoder( + Av1EncoderSuperblockWorkspace workspace, + int qIndex, + int mapVariant) + { + this.workspace = workspace; + this.qIndex = qIndex; + this.mapVariant = mapVariant; + this.Count = 0; + } + + /// + /// Gets the number of final blocks visited by the writer. + /// + public int Count { get; private set; } + + /// + public readonly Av1PartitionType SelectPartition( + Av1SymbolEncoder writer, + Av1MacroBlockD macroBlock, + Point blockOrigin, + ushort tileIndex, + Av1BlockSize blockSize, + Av1PartitionType preparedPartition) + => preparedPartition; + + /// + public void EncodeBlock( + Av1SymbolEncoder writer, + Av1MacroBlockD macroBlock, + Point blockOrigin, + ushort tileIndex, + ref Av1MacroBlockModeInfo modeInfo, + ref Av1EncoderBlockStruct block, + ref Av1EncoderPaletteInfo paletteInfo) + { + this.Count++; + modeInfo.Block = new Av1EncoderBlockModeInfo + { + BlockSize = Av1BlockSize.Block8x8, + PartitionType = Av1PartitionType.None, + SegmentId = 0, + Skip = true, + TransformSize = Av1TransformSize.Size8x8, + Mode = Av1PredictionMode.DC, + UvMode = Av1ChromaPredictionMode.DC + }; + + block.HasChroma = false; + block.QuantizationIndex = this.qIndex; + block.SegmentId = 0; + paletteInfo.PaletteSizes[0] = 3; + paletteInfo.SetColors(Av1Plane.Y, [16, 128, 240]); + Buffer2DRegion map = this.workspace + .GetPaletteMaps() + .GetMap(Av1PlaneType.Y, 8, 8); + + for (int row = 0; row < map.Height; row++) + { + Span mapRow = map.DangerousGetRowSpan(row); + for (int column = 0; column < map.Width; column++) + { + mapRow[column] = this.mapVariant == 0 + ? (byte)0 + : (byte)((row + column) % 3); + } + } + } + } +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1InverseQuantizationTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1InverseQuantizationTests.cs new file mode 100644 index 0000000000..cd46dabeb3 --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1InverseQuantizationTests.cs @@ -0,0 +1,125 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1; +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.Quantizers; +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1; + +[Trait("Format", "Avif")] +public class Av1InverseQuantizationTests +{ + [Theory] + [InlineData((int)Av1BitDepth.EightBit, 26, 30, 40, 32767, -32768)] + [InlineData((int)Av1BitDepth.TenBit, 75, 83, 112, 131071, -131072)] + [InlineData((int)Av1BitDepth.TwelveBit, 266, 297, 399, 524287, -524288)] + public void DequantizationMatchesReferenceMatrixAndPrecisionValues( + int bitDepthValue, + int dc, + int ac, + int weightedAc, + int maximum, + int minimum) + { + ObuSequenceHeader sequenceHeader = new() + { + ColorConfig = new ObuColorConfig { BitDepth = (Av1BitDepth)bitDepthValue } + }; + + ObuFrameHeader frameHeader = new(); + frameHeader.QuantizationParameters.BaseQIndex = 23; + frameHeader.QuantizationParameters.IsUsingQMatrix = true; + frameHeader.SegmentationParameters.QMLevel[0][0] = 0; + Av1InverseQuantizer quantizer = new(sequenceHeader, frameHeader); + Av1BlockModeInfo mode = new(Av1BlockSize.Block4x4, Point.Empty); + Av1InverseQuantizer.TransformParameters matrix = new( + quantizer, mode, Av1TransformType.DctDct, Av1TransformSize.Size4x4, Av1Plane.Y); + + // quant_common.c's qindex-23 tables supply the three DC/AC pairs above. Its level-zero luma matrix + // begins with weights 32 and 43; the rounded AC values are independently fixed in the theory data. + Assert.Equal(7 * dc, matrix.Dequantize(7, 0, false)); + Assert.Equal(11 * weightedAc, matrix.Dequantize(11, 1, false)); + Assert.Equal(-11 * weightedAc, matrix.Dequantize(11, 1, true)); + Assert.Equal(maximum, matrix.Dequantize(0xfffff, 1, false)); + Assert.Equal(minimum, matrix.Dequantize(0xfffff, 1, true)); + + // Identity and one-dimensional transforms bypass matrix weighting even when the frame enables it. + Av1InverseQuantizer.TransformParameters identity = new( + quantizer, mode, Av1TransformType.Identity, Av1TransformSize.Size4x4, Av1Plane.Y); + + Av1InverseQuantizer.TransformParameters horizontal = new( + quantizer, mode, Av1TransformType.HorizontalAdst, Av1TransformSize.Size4x4, Av1Plane.Y); + + Assert.Equal(11 * ac, identity.Dequantize(11, 1, false)); + Assert.Equal(11 * ac, horizontal.Dequantize(11, 1, false)); + } + + [Fact] + public void DequantizationPreservesProductMaskAndTransformRounding() + { + ObuSequenceHeader sequenceHeader = new() + { + ColorConfig = new ObuColorConfig { BitDepth = Av1BitDepth.EightBit } + }; + + ObuFrameHeader frameHeader = new(); + frameHeader.QuantizationParameters.BaseQIndex = 23; + frameHeader.QuantizationParameters.IsUsingQMatrix = true; + frameHeader.SegmentationParameters.QMLevel[0][0] = 0; + Av1InverseQuantizer quantizer = new(sequenceHeader, frameHeader); + Av1BlockModeInfo mode = new(Av1BlockSize.Block64x64, Point.Empty); + Av1InverseQuantizer.TransformParameters matrix = new( + quantizer, mode, Av1TransformType.DctDct, Av1TransformSize.Size4x4, Av1Plane.Y); + + // The final 4x4 matrix weight is 200, giving AC=188. Its product with 89241 is 2^24 + 92: + // retaining the 24-bit intermediate must produce 92, rather than saturating the unmasked product. + Assert.Equal(92, matrix.Dequantize(89241, 15, false)); + Assert.Equal(-92, matrix.Dequantize(89241, 15, true)); + + frameHeader.QuantizationParameters.IsUsingQMatrix = false; + Av1InverseQuantizer.TransformParameters scaled32 = new( + quantizer, mode, Av1TransformType.DctDct, Av1TransformSize.Size32x32, Av1Plane.Y); + + Av1InverseQuantizer.TransformParameters scaled64 = new( + quantizer, mode, Av1TransformType.DctDct, Av1TransformSize.Size64x64, Av1Plane.Y); + + // A magnitude of 11 with AC=30 gives 330 before scaling. Sign follows truncation of the positive + // magnitude, so the negative 64x64 result is -82 rather than the arithmetic-right-shift result -83. + Assert.Equal(165, scaled32.Dequantize(11, 1, false)); + Assert.Equal(82, scaled64.Dequantize(11, 1, false)); + Assert.Equal(-82, scaled64.Dequantize(11, 1, true)); + + frameHeader.QuantizationParameters.BaseQIndex = 0; + frameHeader.QuantizationParameters.IsUsingQMatrix = true; + frameHeader.LosslessArray[0] = true; + Av1InverseQuantizer losslessQuantizer = new(sequenceHeader, frameHeader); + Av1InverseQuantizer.TransformParameters lossless = new( + losslessQuantizer, mode, Av1TransformType.DctDct, Av1TransformSize.Size4x4, Av1Plane.Y); + + Assert.Equal(44, lossless.Dequantize(11, 1, false)); + } + + [Fact] + public void MatricesCoverAllLevelsPlanesAndTransformSizes() + { + for (int level = 0; level < Av1Constants.QuantificationMatrixLevelCount; level++) + { + for (Av1Plane plane = Av1Plane.Y; (int)plane < Av1Constants.MaxPlanes; plane++) + { + for (int transformSizeIndex = 0; transformSizeIndex < (int)Av1TransformSize.AllSizes; transformSizeIndex++) + { + Av1TransformSize transformSize = (Av1TransformSize)transformSizeIndex; + Av1TransformSize adjustedSize = transformSize.GetAdjusted(); + int expectedLength = adjustedSize.GetWidth() * adjustedSize.GetHeight(); + + ReadOnlySpan matrix = Av1InverseQuantizationLookup.GetQuantizationMatrix(level, plane, transformSize); + + Assert.Equal(expectedLength, matrix.Length); + } + } + } + } +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1InverseTransformTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1InverseTransformTests.cs new file mode 100644 index 0000000000..9230be5b44 --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1InverseTransformTests.cs @@ -0,0 +1,1014 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.Intrinsics; +using SixLabors.ImageSharp.Formats.Heif.Av1; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform.Forward; +using SixLabors.ImageSharp.Tests.TestUtilities; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1; + +/// +/// Verifies AV1 forward and inverse transform reconstruction across supported sizes, precisions, and intrinsic tiers. +/// +[Trait("Format", "Avif")] +public class Av1InverseTransformTests +{ + /// + /// The hardware configurations covering every transform SIMD tier and the scalar fallback. + /// + private const HwIntrinsics TransformConfigurations = + HwIntrinsics.AllowAll | HwIntrinsics.DisableAVX512F | HwIntrinsics.DisableAVX | HwIntrinsics.DisableHWIntrinsic; + + /// + /// Verifies DCT operator parity across the supported hardware feature levels. + /// + [Fact] + public void DctOperatorsProduceIdenticalScalarAndSimdResults() + => FeatureTestRunner.RunWithHwIntrinsicsFeature(AssertDctOperatorParity, TransformConfigurations); + + /// + /// Verifies ADST operator parity across the supported hardware feature levels. + /// + [Fact] + public void AdstOperatorsProduceIdenticalScalarAndSimdResults() + => FeatureTestRunner.RunWithHwIntrinsicsFeature(AssertAdstOperatorParity, TransformConfigurations); + + /// + /// Verifies identity operator parity across the supported hardware feature levels. + /// + [Fact] + public void IdentityOperatorsProduceIdenticalScalarAndSimdResults() + => FeatureTestRunner.RunWithHwIntrinsicsFeature(AssertIdentityOperatorParity, TransformConfigurations); + + /// + /// Verifies the reference widened operations at the twelve-bit inverse row-stage bounds. + /// + [Fact] + public void TwelveBitWideIntermediatesMatchReference() + => FeatureTestRunner.RunWithHwIntrinsicsFeature(AssertTwelveBitWideIntermediateParity, TransformConfigurations); + + /// + /// Verifies the inverse DCT operators against their scalar implementations. + /// + private static void AssertDctOperatorParity() + { + AssertOperatorParity(4); + AssertOperatorParity(8); + AssertOperatorParity(16); + AssertOperatorParity(32); + AssertOperatorParity(64); + } + + /// + /// Verifies the inverse ADST operators against their scalar implementations. + /// + private static void AssertAdstOperatorParity() + { + AssertOperatorParity(4); + AssertOperatorParity(8); + AssertOperatorParity(16); + } + + /// + /// Verifies the inverse identity operators against their scalar implementations. + /// + private static void AssertIdentityOperatorParity() + { + AssertOperatorParity(4); + AssertOperatorParity(8); + AssertOperatorParity(16); + AssertOperatorParity(32); + } + + /// + /// Exercises the exact ADST4 rounding and identity-product overflows that are possible at a twenty-bit row range. + /// + private static void AssertTwelveBitWideIntermediateParity() + { + const int cosBit = 12; + InlineArray12 stageRange = default; + for (int index = 0; index < Av1Transform2dFlipConfiguration.MaxStageNumber; index++) + { + stageRange[index] = 20; + } + + Av1TransformVector> adstInput128 = default; + adstInput128.V0 = Vector128.Create(196_118, -196_118, 196_117, -196_117); + adstInput128.V1 = Vector128.Create(196_117, -196_117, 196_117, -196_117); + adstInput128.V2 = Vector128.Create(196_117, -196_117, 196_117, -196_117); + adstInput128.V3 = Vector128.Create(196_117, -196_117, 196_117, -196_117); + Av1TransformVector> adstInput256 = default; + adstInput256.V0 = Vector256.Create(196_118, -196_118, 196_117, -196_117, 196_118, -196_118, 196_117, -196_117); + adstInput256.V1 = Vector256.Create(196_117, -196_117, 196_117, -196_117, 196_117, -196_117, 196_117, -196_117); + adstInput256.V2 = adstInput256.V1; + adstInput256.V3 = adstInput256.V1; + Av1TransformVector> adstOutput128 = default; + Av1TransformVector> adstStep128 = default; + Av1TransformVector> adstOutput256 = default; + Av1TransformVector> adstStep256 = default; + + Av1Inverse2dTransformer.Adst4Operator.Transform( + ref adstInput128, + ref adstOutput128, + ref adstStep128, + cosBit, + stageRange); + + Av1Inverse2dTransformer.Adst4Operator.Transform( + ref adstInput256, + ref adstOutput256, + ref adstStep256, + cosBit, + stageRange); + + // These are the exact outputs of the reference decoder's signed Int64 terminal round. The first positive lane has an + // Int32 fixed-point sum of 2,147,482,471, so adding the 2,048 rounding bias in Int32 would wrap. + Vector128 adstExpected0 = Vector128.Create(524_288, -524_288, 524_287, -524_287); + Vector128 adstExpected1 = Vector128.Create(33_612, -33_612, 33_612, -33_612); + Vector128 adstExpected2 = Vector128.Create(160_112, -160_112, 160_111, -160_111); + Vector128 adstExpected3 = Vector128.Create(77_567, -77_567, 77_566, -77_566); + Assert.Equal(adstExpected0, adstOutput128.V0); + Assert.Equal(adstExpected1, adstOutput128.V1); + Assert.Equal(adstExpected2, adstOutput128.V2); + Assert.Equal(adstExpected3, adstOutput128.V3); + Assert.Equal(Vector256.Create(adstExpected0, adstExpected0), adstOutput256.V0); + Assert.Equal(Vector256.Create(adstExpected1, adstExpected1), adstOutput256.V1); + Assert.Equal(Vector256.Create(adstExpected2, adstExpected2), adstOutput256.V2); + Assert.Equal(Vector256.Create(adstExpected3, adstExpected3), adstOutput256.V3); + + Vector128 identityInput128 = Vector128.Create(524_287, -524_288, 524_286, -524_287); + Vector256 identityInput256 = Vector256.Create( + 524_287, + -524_288, + 524_286, + -524_287, + 370_727, + -370_728, + 262_143, + -262_144); + + AssertWidenedIdentityOperator( + 4, + identityInput128, + Vector128.Create(741_503, -741_504, 741_501, -741_503), + identityInput256, + Vector256.Create(741_503, -741_504, 741_501, -741_503, 524_322, -524_323, 370_751, -370_752), + stageRange); + + AssertWidenedIdentityOperator( + 16, + identityInput128, + Vector128.Create(1_483_005, -1_483_008, 1_483_002, -1_483_005), + identityInput256, + Vector256.Create(1_483_005, -1_483_008, 1_483_002, -1_483_005, 1_048_643, -1_048_646, 741_501, -741_504), + stageRange); + } + + /// + /// Verifies one identity operator against exact reference widened fixed-point results. + /// + /// The inverse identity operator. + /// The identity-transform length. + /// The four-lane bounded input. + /// The exact four-lane result. + /// The eight-lane bounded input. + /// The exact eight-lane result. + /// The twelve-bit inverse row-stage range. + private static void AssertWidenedIdentityOperator( + int length, + Vector128 input128, + Vector128 expected128, + Vector256 input256, + Vector256 expected256, + InlineArray12 stageRange) + where TOperator : struct, Av1Inverse2dTransformer.IAv1Transform1dOperator + { + const int cosBit = 12; + Av1TransformVector> values128 = default; + Av1TransformVector> output128 = default; + Av1TransformVector> step128 = default; + Av1TransformVector> values256 = default; + Av1TransformVector> output256 = default; + Av1TransformVector> step256 = default; + + for (int index = 0; index < length; index++) + { + values128[index] = input128; + values256[index] = input256; + } + + TOperator.Transform(ref values128, ref output128, ref step128, cosBit, stageRange); + TOperator.Transform(ref values256, ref output256, ref step256, cosBit, stageRange); + + for (int index = 0; index < length; index++) + { + Assert.Equal(expected128, output128[index]); + Assert.Equal(expected256, output256[index]); + } + } + + [Theory] + [InlineData((int)Av1TransformSize.Size4x4, 0, -4)] + [InlineData((int)Av1TransformSize.Size8x8, -1, -4)] + [InlineData((int)Av1TransformSize.Size16x16, -2, -4)] + [InlineData((int)Av1TransformSize.Size32x32, -2, -4)] + [InlineData((int)Av1TransformSize.Size64x64, -2, -4)] + [InlineData((int)Av1TransformSize.Size4x8, 0, -4)] + [InlineData((int)Av1TransformSize.Size8x4, 0, -4)] + [InlineData((int)Av1TransformSize.Size8x16, -1, -4)] + [InlineData((int)Av1TransformSize.Size16x8, -1, -4)] + [InlineData((int)Av1TransformSize.Size16x32, -1, -4)] + [InlineData((int)Av1TransformSize.Size32x16, -1, -4)] + [InlineData((int)Av1TransformSize.Size32x64, -1, -4)] + [InlineData((int)Av1TransformSize.Size64x32, -1, -4)] + [InlineData((int)Av1TransformSize.Size4x16, -1, -4)] + [InlineData((int)Av1TransformSize.Size16x4, -1, -4)] + [InlineData((int)Av1TransformSize.Size8x32, -2, -4)] + [InlineData((int)Av1TransformSize.Size32x8, -2, -4)] + [InlineData((int)Av1TransformSize.Size16x64, -2, -4)] + [InlineData((int)Av1TransformSize.Size64x16, -2, -4)] + public void InverseConfigurationUsesNormativeShifts(int transformSizeValue, int firstShift, int secondShift) + { + Av1TransformSize transformSize = (Av1TransformSize)transformSizeValue; + Av1Transform2dFlipConfiguration config = Av1Transform2dFlipConfiguration.CreateInverse(Av1TransformType.DctDct, transformSize, 8); + + Assert.Equal(firstShift, config.Shift0); + Assert.Equal(secondShift, config.Shift1); + Assert.Equal(0, config.Shift2); + Assert.Equal(12, config.CosBitColumn); + Assert.Equal(12, config.CosBitRow); + } + + [Theory] + [InlineData(8, 16, 16)] + [InlineData(10, 18, 16)] + [InlineData(12, 20, 18)] + public void InverseConfigurationUsesNormativeStageRanges(int bitDepth, byte rowRange, byte columnRange) + { + Av1Transform2dFlipConfiguration config = Av1Transform2dFlipConfiguration.CreateInverse( + Av1TransformType.AdstAdst, + Av1TransformSize.Size16x16, + bitDepth); + + InlineArray12 configuredRowRange = config.StageRangeRow; + InlineArray12 configuredColumnRange = config.StageRangeColumn; + + for (int index = 0; index < config.StageNumberRow; index++) + { + Assert.Equal(rowRange, configuredRowRange[index]); + } + + for (int index = 0; index < config.StageNumberColumn; index++) + { + Assert.Equal(columnRange, configuredColumnRange[index]); + } + } + + [Fact] + public void ForwardAndInverseOperatorPairsReconstructTheirInput() + { + AssertRoundTrip(Av1TransformType.DctDct, Av1TransformSize.Size4x4, 1, 1); + AssertRoundTrip(Av1TransformType.DctDct, Av1TransformSize.Size8x8, 2, 2); + AssertRoundTrip(Av1TransformType.DctDct, Av1TransformSize.Size16x16, 3, 3); + AssertRoundTrip(Av1TransformType.DctDct, Av1TransformSize.Size32x32, 4, 4); + AssertRoundTrip(Av1TransformType.DctDct, Av1TransformSize.Size64x64, 5, 5); + AssertRoundTrip(Av1TransformType.AdstAdst, Av1TransformSize.Size4x4, 1, 1); + AssertRoundTrip(Av1TransformType.AdstAdst, Av1TransformSize.Size8x8, 2, 2); + AssertRoundTrip(Av1TransformType.AdstAdst, Av1TransformSize.Size16x16, 3, 3); + AssertRoundTrip(Av1TransformType.Identity, Av1TransformSize.Size4x4, 1, 1); + AssertRoundTrip(Av1TransformType.Identity, Av1TransformSize.Size8x8, 2, 1); + AssertRoundTrip(Av1TransformType.Identity, Av1TransformSize.Size16x16, 3, 1); + AssertRoundTrip(Av1TransformType.Identity, Av1TransformSize.Size32x32, 4, 1); + } + + /// + /// Verifies that every applicable SIMD traversal reconstructs the same samples as the scalar traversal. + /// + /// The integral value. + /// The integral value. + /// The coded sample bit depth. + [Theory] + [MemberData(nameof(Av1ForwardTransformTests.ValidTransformCases), MemberType = typeof(Av1ForwardTransformTests))] + public void TwoDimensionalKernelsMatchReference( + int transformTypeValue, + int transformSizeValue, + int bitDepth) + { + Av1TransformType transformType = (Av1TransformType)transformTypeValue; + Av1TransformSize transformSize = (Av1TransformSize)transformSizeValue; + Av1Transform2dFlipConfiguration config = Av1Transform2dFlipConfiguration.CreateInverse(transformType, transformSize, bitDepth); + DispatchColumn(transformType, transformSize, bitDepth, ref config); + } + + /// + /// Verifies lossless inverse Walsh-Hadamard reconstruction against an independent definition. + /// + [Fact] + public void LosslessWalshHadamardMatchesReference() + => FeatureTestRunner.RunWithHwIntrinsicsFeature(AssertLosslessWalshHadamardParity, TransformConfigurations); + + /// + /// Exercises DC-only and complete lossless blocks at every supported sample precision. + /// + private static void AssertLosslessWalshHadamardParity() + { + const int stride = 7; + int[] workspace = new int[Av1TransformWorkspace.MaximumLength]; + int[][] coefficientCases = + [ + [512, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], + [-516, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], + [320, -192, 64, -448, 128, 256, -320, 96, -224, 160, 384, -128, 448, -64, -256, 192] + ]; + + for (int coefficientCase = 0; coefficientCase < coefficientCases.Length; coefficientCase++) + { + int[] coefficients = coefficientCases[coefficientCase]; + int coefficientCount = coefficientCase < 2 ? 1 : coefficients.Length; + byte[] expectedBytes = new byte[stride * 4]; + + Array.Fill(expectedBytes, (byte)233); + + PopulatePrediction(expectedBytes, stride, byte.MaxValue); + byte[] actualBytes = (byte[])expectedBytes.Clone(); + + ApplyWalshHadamardReference(coefficients, expectedBytes, stride, coefficientCount, 8); + Av1InverseTransformer.Reconstruct8Bit( + coefficients, + actualBytes, + stride, + Av1TransformSize.Size4x4, + Av1TransformType.DctDct, + 0, + coefficientCount, + true, + workspace); + + Assert.Equal(expectedBytes, actualBytes); + + foreach (int bitDepth in new[] { 10, 12 }) + { + int maximum = (1 << bitDepth) - 1; + short[] expected = new short[stride * 4]; + + Array.Fill(expected, (short)-1); + + PopulatePrediction(expected, stride, maximum); + short[] actual = (short[])expected.Clone(); + + ApplyWalshHadamardReference(coefficients, expected, stride, coefficientCount, bitDepth); + Av1InverseTransformer.ReconstructHighBitDepth( + coefficients, + actual, + stride, + Av1TransformSize.Size4x4, + Av1TransformType.DctDct, + 0, + coefficientCount, + true, + bitDepth == 10 ? Av1BitDepth.TenBit : Av1BitDepth.TwelveBit, + workspace); + + Assert.Equal(expected, actual); + } + } + } + + [Fact] + public void ReconstructionDispatchDoesNotAllocatePerBlock() + { + const int width = 8; + int[] coefficients = new int[width * width]; + byte[] reconstruction = new byte[coefficients.Length]; + int[] workspace = new int[Av1TransformWorkspace.MaximumLength]; + + Av1InverseTransformer.Reconstruct8Bit( + coefficients, reconstruction, width, Av1TransformSize.Size8x8, Av1TransformType.DctDct, 0, coefficients.Length, false, workspace); + + long before = GC.GetAllocatedBytesForCurrentThread(); + + for (int iteration = 0; iteration < 32; iteration++) + { + Av1InverseTransformer.Reconstruct8Bit( + coefficients, reconstruction, width, Av1TransformSize.Size8x8, Av1TransformType.DctDct, 0, coefficients.Length, false, workspace); + } + + long allocated = GC.GetAllocatedBytesForCurrentThread() - before; + Assert.Equal(0, allocated); + } + + [Theory] + [InlineData((int)Av1BitDepth.TenBit, 1023)] + [InlineData((int)Av1BitDepth.TwelveBit, 4095)] + public void HighBitDepthReconstructionClipsPositiveValues(int bitDepthIndex, short maximum) + { + const int width = 4; + int[] coefficients = new int[width * width]; + coefficients[0] = 64; + short[] reconstruction = new short[width * width]; + Array.Fill(reconstruction, (short)(maximum - 1)); + int[] workspace = new int[Av1TransformWorkspace.MaximumLength]; + + Av1InverseTransformer.ReconstructHighBitDepth( + coefficients, + reconstruction, + width, + Av1TransformSize.Size4x4, + Av1TransformType.DctDct, + 0, + 1, + false, + (Av1BitDepth)bitDepthIndex, + workspace); + + Assert.All(reconstruction, value => Assert.Equal(maximum, value)); + } + + [Theory] + [InlineData((int)Av1BitDepth.TenBit)] + [InlineData((int)Av1BitDepth.TwelveBit)] + public void HighBitDepthReconstructionClipsNegativeValues(int bitDepthIndex) + { + const int width = 4; + int[] coefficients = new int[width * width]; + coefficients[0] = -64; + short[] reconstruction = new short[width * width]; + Array.Fill(reconstruction, (short)1); + int[] workspace = new int[Av1TransformWorkspace.MaximumLength]; + + Av1InverseTransformer.ReconstructHighBitDepth( + coefficients, + reconstruction, + width, + Av1TransformSize.Size4x4, + Av1TransformType.DctDct, + 0, + 1, + false, + (Av1BitDepth)bitDepthIndex, + workspace); + + Assert.All(reconstruction, value => Assert.Equal((short)0, value)); + } + + /// + /// Populates active eight-bit prediction samples while preserving row-padding sentinels. + /// + private static void PopulatePrediction(Span prediction, int stride, int maximum) + { + for (int row = 0; row < 4; row++) + { + for (int column = 0; column < 4; column++) + { + prediction[(row * stride) + column] = (byte)(((row * 101) + (column * 67) + 19) & maximum); + } + } + } + + /// + /// Populates active high-bit-depth prediction samples while preserving row-padding sentinels. + /// + private static void PopulatePrediction(Span prediction, int stride, int maximum) + { + for (int row = 0; row < 4; row++) + { + for (int column = 0; column < 4; column++) + { + prediction[(row * stride) + column] = (short)(((row * 911) + (column * 593) + 37) & maximum); + } + } + } + + /// + /// Applies the normative inverse Walsh-Hadamard definition to an eight-bit prediction block. + /// + private static void ApplyWalshHadamardReference(ReadOnlySpan coefficients, Span destination, int stride, int coefficientCount, int bitDepth) + { + int[] residuals = CalculateWalshHadamardReference(coefficients, coefficientCount); + int maximum = (1 << bitDepth) - 1; + + for (int row = 0; row < 4; row++) + { + for (int column = 0; column < 4; column++) + { + int offset = (row * stride) + column; + destination[offset] = (byte)Math.Clamp(destination[offset] + residuals[(row * 4) + column], 0, maximum); + } + } + } + + /// + /// Applies the normative inverse Walsh-Hadamard definition to a high-bit-depth prediction block. + /// + private static void ApplyWalshHadamardReference(ReadOnlySpan coefficients, Span destination, int stride, int coefficientCount, int bitDepth) + { + int[] residuals = CalculateWalshHadamardReference(coefficients, coefficientCount); + int maximum = (1 << bitDepth) - 1; + + for (int row = 0; row < 4; row++) + { + for (int column = 0; column < 4; column++) + { + int offset = (row * stride) + column; + destination[offset] = (short)Math.Clamp(destination[offset] + residuals[(row * 4) + column], 0, maximum); + } + } + } + + /// + /// Calculates the exact four-by-four residual matrix defined by AV1's reversible transform. + /// + private static int[] CalculateWalshHadamardReference(ReadOnlySpan coefficients, int coefficientCount) + { + int[] residuals = new int[16]; + + if (coefficientCount == 1) + { + int first = coefficients[0] >> 2; + int half = first >> 1; + int firstIntermediate = first - half; + + for (int column = 0; column < 4; column++) + { + int intermediate = column == 0 ? firstIntermediate : half; + int repeatedResidual = intermediate >> 1; + residuals[column] = intermediate - repeatedResidual; + residuals[4 + column] = repeatedResidual; + residuals[8 + column] = repeatedResidual; + residuals[12 + column] = repeatedResidual; + } + + return residuals; + } + + int[] intermediateValues = new int[16]; + for (int row = 0; row < 4; row++) + { + int coefficientOffset = row * 4; + int a = coefficients[coefficientOffset] >> 2; + int c = coefficients[coefficientOffset + 1] >> 2; + int d = coefficients[coefficientOffset + 2] >> 2; + int b = coefficients[coefficientOffset + 3] >> 2; + + ApplyWalshHadamardReference(ref a, ref b, ref c, ref d); + intermediateValues[row] = a; + intermediateValues[4 + row] = b; + intermediateValues[8 + row] = c; + intermediateValues[12 + row] = d; + } + + for (int column = 0; column < 4; column++) + { + int offset = column * 4; + int a = intermediateValues[offset]; + int c = intermediateValues[offset + 1]; + int d = intermediateValues[offset + 2]; + int b = intermediateValues[offset + 3]; + + ApplyWalshHadamardReference(ref a, ref b, ref c, ref d); + residuals[column] = a; + residuals[4 + column] = b; + residuals[8 + column] = c; + residuals[12 + column] = d; + } + + return residuals; + } + + /// + /// Applies one scalar four-point reversible Walsh-Hadamard dimension for the independent test definition. + /// + private static void ApplyWalshHadamardReference(ref int a, ref int b, ref int c, ref int d) + { + a += c; + d -= b; + int middle = (a - d) >> 1; + b = middle - b; + c = middle - c; + a -= b; + d += c; + } + + /// + /// Compares one inverse transform operator across scalar and the supported SIMD lane widths. + /// + /// The inverse transform operator. + /// The transform length. + private static void AssertOperatorParity(int length) + where TOperator : struct, Av1Inverse2dTransformer.IAv1Transform1dOperator + { + const int cosBit = 12; + InlineArray12 stageRange = default; + + for (int index = 0; index < Av1Transform2dFlipConfiguration.MaxStageNumber; index++) + { + stageRange[index] = 24; + } + + Av1TransformVector> input128 = default; + Av1TransformVector> output128 = default; + Av1TransformVector> step128 = default; + Av1TransformVector> input256 = default; + Av1TransformVector> output256 = default; + Av1TransformVector> step256 = default; + + for (int index = 0; index < length; index++) + { + input128[index] = Vector128.Create( + GetInputValue(index, 0), + GetInputValue(index, 1), + GetInputValue(index, 2), + GetInputValue(index, 3)); + + input256[index] = Vector256.Create( + GetInputValue(index, 0), + GetInputValue(index, 1), + GetInputValue(index, 2), + GetInputValue(index, 3), + GetInputValue(index, 4), + GetInputValue(index, 5), + GetInputValue(index, 6), + GetInputValue(index, 7)); + } + + TOperator.Transform(ref input128, ref output128, ref step128, cosBit, stageRange); + TOperator.Transform(ref input256, ref output256, ref step256, cosBit, stageRange); + + int[] scalarInput = new int[length]; + int[] scalarOutput = new int[length]; + int[] scalarStep = new int[length]; + + for (int lane = 0; lane < Vector256.Count; lane++) + { + for (int index = 0; index < length; index++) + { + scalarInput[index] = GetInputValue(index, lane); + } + + TOperator.Transform(scalarInput, scalarOutput, scalarStep, cosBit, stageRange); + + for (int index = 0; index < length; index++) + { + Assert.Equal(scalarOutput[index], output256[index].GetElement(lane)); + + if (lane < Vector128.Count) + { + Assert.Equal(scalarOutput[index], output128[index].GetElement(lane)); + } + } + } + } + + /// + /// Verifies that a matching one-dimensional forward and inverse operator pair reconstructs bounded input. + /// + /// The forward transform operator. + /// The inverse transform operator. + /// The compound transform type. + /// The transform-block dimensions. + /// The power-of-two scale applied by the operator pair. + /// The maximum permitted reconstruction error. + private static void AssertRoundTrip(Av1TransformType transformType, Av1TransformSize transformSize, int scaleLog2, int allowedError) + where TForwardOperator : struct, Av1ForwardTransformer.IAv1ForwardTransform1dOperator + where TInverseOperator : struct, Av1Inverse2dTransformer.IAv1Transform1dOperator + { + const int bitDepth = 10; + const int testBlockCount = 30; + Av1Transform2dFlipConfiguration forwardConfig = Av1Transform2dFlipConfiguration.CreateForward(transformType, transformSize, bitDepth); + Av1Transform2dFlipConfiguration inverseConfig = Av1Transform2dFlipConfiguration.CreateInverse(transformType, transformSize, bitDepth); + int length = transformSize.GetWidth(); + Random random = new(0); + int[] input = new int[length]; + int[] forward = new int[length]; + int[] inverse = new int[length]; + int[] step = new int[length]; + Av1TransformVector values = default; + Av1TransformVector buffer0 = default; + Av1TransformVector buffer1 = default; + + for (int block = 0; block < testBlockCount; block++) + { + for (int index = 0; index < length; index++) + { + input[index] = random.Next((1 << bitDepth) - 1); + values[index] = input[index]; + } + + ref byte valuesBase = ref System.Runtime.CompilerServices.Unsafe.As, byte>(ref values); + + TForwardOperator.Transform(ref valuesBase, sizeof(int), sizeof(int), ref buffer0, ref buffer1, forwardConfig.CosBitColumn); + + for (int index = 0; index < length; index++) + { + forward[index] = values[index]; + } + + TInverseOperator.Transform(forward, inverse, step, inverseConfig.CosBitColumn, inverseConfig.StageRangeColumn); + + for (int index = 0; index < length; index++) + { + int reconstructed = inverse[index] >> scaleLog2; + Assert.InRange(Math.Abs(input[index] - reconstructed), 0, allowedError); + } + } + } + + /// + /// Closes the static-generic inverse column operator selected by a transform configuration. + /// + /// The compound transform type. + /// The transform-block dimensions. + /// The coded sample bit depth. + /// The inverse transform configuration. + private static void DispatchColumn( + Av1TransformType transformType, + Av1TransformSize transformSize, + int bitDepth, + ref Av1Transform2dFlipConfiguration config) + { + switch (config.TransformFunctionTypeColumn) + { + case Av1TransformFunctionType.Dct4: + DispatchRow(transformType, transformSize, bitDepth, ref config); + break; + case Av1TransformFunctionType.Dct8: + DispatchRow(transformType, transformSize, bitDepth, ref config); + break; + case Av1TransformFunctionType.Dct16: + DispatchRow(transformType, transformSize, bitDepth, ref config); + break; + case Av1TransformFunctionType.Dct32: + DispatchRow(transformType, transformSize, bitDepth, ref config); + break; + case Av1TransformFunctionType.Dct64: + DispatchRow(transformType, transformSize, bitDepth, ref config); + break; + case Av1TransformFunctionType.Adst4: + DispatchRow(transformType, transformSize, bitDepth, ref config); + break; + case Av1TransformFunctionType.Adst8: + DispatchRow(transformType, transformSize, bitDepth, ref config); + break; + case Av1TransformFunctionType.Adst16: + DispatchRow(transformType, transformSize, bitDepth, ref config); + break; + case Av1TransformFunctionType.Identity4: + DispatchRow(transformType, transformSize, bitDepth, ref config); + break; + case Av1TransformFunctionType.Identity8: + DispatchRow(transformType, transformSize, bitDepth, ref config); + break; + case Av1TransformFunctionType.Identity16: + DispatchRow(transformType, transformSize, bitDepth, ref config); + break; + case Av1TransformFunctionType.Identity32: + DispatchRow(transformType, transformSize, bitDepth, ref config); + break; + default: + Assert.Fail($"Unexpected column function {config.TransformFunctionTypeColumn} for {transformType} {transformSize}."); + break; + } + } + + /// + /// Closes the static-generic inverse row operator after the column operator has been selected. + /// + /// The selected inverse column operator. + /// The compound transform type. + /// The transform-block dimensions. + /// The coded sample bit depth. + /// The inverse transform configuration. + private static void DispatchRow( + Av1TransformType transformType, + Av1TransformSize transformSize, + int bitDepth, + ref Av1Transform2dFlipConfiguration config) + where TColumnOperator : struct, Av1Inverse2dTransformer.IAv1Transform1dOperator + { + switch (config.TransformFunctionTypeRow) + { + case Av1TransformFunctionType.Dct4: + AssertTransform2dParity(transformType, transformSize, bitDepth, ref config); + break; + case Av1TransformFunctionType.Dct8: + AssertTransform2dParity(transformType, transformSize, bitDepth, ref config); + break; + case Av1TransformFunctionType.Dct16: + AssertTransform2dParity(transformType, transformSize, bitDepth, ref config); + break; + case Av1TransformFunctionType.Dct32: + AssertTransform2dParity(transformType, transformSize, bitDepth, ref config); + break; + case Av1TransformFunctionType.Dct64: + AssertTransform2dParity(transformType, transformSize, bitDepth, ref config); + break; + case Av1TransformFunctionType.Adst4: + AssertTransform2dParity(transformType, transformSize, bitDepth, ref config); + break; + case Av1TransformFunctionType.Adst8: + AssertTransform2dParity(transformType, transformSize, bitDepth, ref config); + break; + case Av1TransformFunctionType.Adst16: + AssertTransform2dParity(transformType, transformSize, bitDepth, ref config); + break; + case Av1TransformFunctionType.Identity4: + AssertTransform2dParity(transformType, transformSize, bitDepth, ref config); + break; + case Av1TransformFunctionType.Identity8: + AssertTransform2dParity(transformType, transformSize, bitDepth, ref config); + break; + case Av1TransformFunctionType.Identity16: + AssertTransform2dParity(transformType, transformSize, bitDepth, ref config); + break; + case Av1TransformFunctionType.Identity32: + AssertTransform2dParity(transformType, transformSize, bitDepth, ref config); + break; + default: + Assert.Fail($"Unexpected row function {config.TransformFunctionTypeRow} for {transformType} {transformSize}."); + break; + } + } + + /// + /// Produces bounded conformant coefficients and selects byte or high-bit-depth reconstruction verification. + /// + /// The selected inverse column operator. + /// The selected inverse row operator. + /// The compound transform type. + /// The transform-block dimensions. + /// The coded sample bit depth. + /// The inverse transform configuration. + private static void AssertTransform2dParity( + Av1TransformType transformType, + Av1TransformSize transformSize, + int bitDepth, + ref Av1Transform2dFlipConfiguration config) + where TColumnOperator : struct, Av1Inverse2dTransformer.IAv1Transform1dOperator + where TRowOperator : struct, Av1Inverse2dTransformer.IAv1Transform1dOperator + { + int width = transformSize.GetWidth(); + int height = transformSize.GetHeight(); + int inputStride = width + 5; + int maximum = (1 << bitDepth) - 1; + short[] residual = new short[inputStride * height]; + + for (int row = 0; row < height; row++) + { + for (int column = 0; column < width; column++) + { + int index = (row * width) + column; + residual[(row * inputStride) + column] = (short)((index & 3) switch + { + 0 => maximum, + 1 => -maximum, + 2 => ((index * 73) % ((maximum * 2) + 1)) - maximum, + _ => 0, + }); + } + } + + // A conformant forward transform supplies coefficient magnitudes at the exact fixed-point bounds expected by + // the inverse kernels. This is stronger than arbitrary small coefficients and avoids impossible stress inputs. + int[] coefficients = new int[width * height]; + int[] forwardWorkspace = new int[Av1TransformWorkspace.GetRequiredLength(transformSize)]; + Av1ForwardTransformer.Transform2d(residual, coefficients, (uint)inputStride, transformType, transformSize, bitDepth, forwardWorkspace); + + if (bitDepth == 8) + { + AssertByteTransform2dParity(coefficients, transformSize, ref config); + return; + } + + AssertHighBitDepthTransform2dParity(coefficients, transformSize, bitDepth, ref config); + } + + /// + /// Compares eight-bit scalar and SIMD reconstruction with independently padded read and write rows. + /// + /// The selected inverse column operator. + /// The selected inverse row operator. + /// The conformant forward-transform coefficients. + /// The transform-block dimensions. + /// The inverse transform configuration. + private static void AssertByteTransform2dParity( + int[] coefficients, + Av1TransformSize transformSize, + ref Av1Transform2dFlipConfiguration config) + where TColumnOperator : struct, Av1Inverse2dTransformer.IAv1Transform1dOperator + where TRowOperator : struct, Av1Inverse2dTransformer.IAv1Transform1dOperator + { + const int bitDepth = 8; + int width = transformSize.GetWidth(); + int height = transformSize.GetHeight(); + int readStride = width + 3; + int writeStride = width + 7; + int workspaceLength = Av1TransformWorkspace.GetRequiredLength(transformSize); + byte[] prediction = new byte[readStride * height]; + + for (int row = 0; row < height; row++) + { + for (int column = 0; column < width; column++) + { + prediction[(row * readStride) + column] = (byte)(((row * width) + column) * 29); + } + } + + byte[] scalar = new byte[writeStride * height]; + byte[] vector128 = new byte[writeStride * height]; + int[] scalarWorkspace = new int[workspaceLength]; + int[] vector128Workspace = new int[workspaceLength]; + Array.Fill(scalar, byte.MaxValue); + Array.Fill(vector128, byte.MaxValue); + + Av1Inverse2dTransformer.Transform2dScalar( + coefficients, prediction, readStride, scalar, writeStride, ref config, scalarWorkspace, bitDepth); + + Av1Inverse2dTransformer.Transform2dVector128( + coefficients, prediction, readStride, vector128, writeStride, ref config, vector128Workspace, bitDepth); + + Assert.Equal(scalar, vector128); + + if (width >= Vector256.Count && height >= Vector256.Count) + { + byte[] vector256 = new byte[writeStride * height]; + int[] vector256Workspace = new int[workspaceLength]; + Array.Fill(vector256, byte.MaxValue); + + Av1Inverse2dTransformer.Transform2dVector256( + coefficients, prediction, readStride, vector256, writeStride, ref config, vector256Workspace, bitDepth); + + Assert.Equal(scalar, vector256); + } + } + + /// + /// Compares high-bit-depth scalar and SIMD reconstruction with independently padded read and write rows. + /// + /// The selected inverse column operator. + /// The selected inverse row operator. + /// The conformant forward-transform coefficients. + /// The transform-block dimensions. + /// The coded sample bit depth. + /// The inverse transform configuration. + private static void AssertHighBitDepthTransform2dParity( + int[] coefficients, + Av1TransformSize transformSize, + int bitDepth, + ref Av1Transform2dFlipConfiguration config) + where TColumnOperator : struct, Av1Inverse2dTransformer.IAv1Transform1dOperator + where TRowOperator : struct, Av1Inverse2dTransformer.IAv1Transform1dOperator + { + int width = transformSize.GetWidth(); + int height = transformSize.GetHeight(); + int readStride = width + 3; + int writeStride = width + 7; + int maximum = (1 << bitDepth) - 1; + int workspaceLength = Av1TransformWorkspace.GetRequiredLength(transformSize); + short[] prediction = new short[readStride * height]; + + for (int row = 0; row < height; row++) + { + for (int column = 0; column < width; column++) + { + prediction[(row * readStride) + column] = (short)((((row * width) + column) * 47) & maximum); + } + } + + short[] scalar = new short[writeStride * height]; + short[] vector128 = new short[writeStride * height]; + int[] scalarWorkspace = new int[workspaceLength]; + int[] vector128Workspace = new int[workspaceLength]; + Array.Fill(scalar, short.MinValue); + Array.Fill(vector128, short.MinValue); + + Av1Inverse2dTransformer.Transform2dScalar( + coefficients, prediction, readStride, scalar, writeStride, ref config, scalarWorkspace, bitDepth); + + Av1Inverse2dTransformer.Transform2dVector128( + coefficients, prediction, readStride, vector128, writeStride, ref config, vector128Workspace, bitDepth); + + Assert.Equal(scalar, vector128); + + if (width >= Vector256.Count && height >= Vector256.Count) + { + short[] vector256 = new short[writeStride * height]; + int[] vector256Workspace = new int[workspaceLength]; + Array.Fill(vector256, short.MinValue); + + Av1Inverse2dTransformer.Transform2dVector256( + coefficients, prediction, readStride, vector256, writeStride, ref config, vector256Workspace, bitDepth); + + Assert.Equal(scalar, vector256); + } + } + + /// + /// Produces deterministic bounded input for one transform position and SIMD lane. + /// + /// The position within the transform. + /// The SIMD lane index. + /// The input value. + private static int GetInputValue(int index, int lane) => (((index * 73) + (lane * 151)) % 1023) - 511; +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1LevelBufferTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1LevelBufferTests.cs new file mode 100644 index 0000000000..5b0556510b --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1LevelBufferTests.cs @@ -0,0 +1,106 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1; + +[Trait("Format", "Avif")] +public class Av1LevelBufferTests +{ + [Theory] + [InlineData(4, 4, 4, 1)] + [InlineData(4, 4, 5, 1)] + [InlineData(4, 4, 6, 1)] + [InlineData(4, 4, 7, 1)] + [InlineData(8, 4, 7, 0)] + [InlineData(8, 8, 16, 2)] + [InlineData(8, 4, 16, 2)] + public void TestGetPaddedRow(int width, int height, int index, byte expected) + { + // Arrange + Size size = new(width, height); + using Av1LevelBuffer levels = new(Configuration.Default, size); + for (byte i = 0; i < 4; i++) + { + levels.GetRow(i).Fill(i); + } + + // Act + Point pos = levels.GetPosition(index); + + // Assert + Assert.Equal(expected, pos.Y); + Assert.Equal(expected, levels.GetRow(pos)[0]); + } + + [Theory] + [InlineData(4, 4)] + [InlineData(8, 4)] + [InlineData(8, 8)] + [InlineData(16, 4)] + public void TestGetRow(int width, int height) + { + // Arrange + Size size = new(width, height); + using Av1LevelBuffer levels = new(Configuration.Default, size); + for (byte i = 0; i < height; i++) + { + levels.GetRow(i).Fill(i); + } + + for (int j = 0; j < height; j++) + { + // Act + Span actual = levels.GetRow(j); + + // Assert + Assert.Equal(j, actual[0]); + Assert.True(actual.Length >= width); + } + } + + [Fact] + public void InitializeStoresAbsoluteSaturatedLevels() + { + // Arrange + using Av1LevelBuffer levels = new(Configuration.Default, new Size(2, 2)); + Span coefficients = [-300, -1, 1, 300]; + + // Act + levels.Initialize(coefficients); + + // Assert + Assert.Equal([127, 1], levels.GetRow(0)[..2].ToArray()); + Assert.Equal([1, 127], levels.GetRow(1)[..2].ToArray()); + } + + [Theory] + [InlineData(4, 4)] + [InlineData(8, 4)] + [InlineData(8, 8)] + [InlineData(16, 4)] + public void TestClear(int width, int height) + { + // Arrange + Size size = new(width, height); + using Av1LevelBuffer levels = new(Configuration.Default, size); + for (byte i = 0; i < height; i++) + { + levels.GetRow(i).Fill(i); + } + + // Act + levels.Clear(); + + // Assert + for (int j = 0; j < height; j++) + { + Span rowSpan = levels.GetRow(j); + for (int k = 0; k < width; k++) + { + Assert.Equal(0, rowSpan[k]); + } + } + } +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1MathTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1MathTests.cs new file mode 100644 index 0000000000..a68f3d0367 --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1MathTests.cs @@ -0,0 +1,123 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1; + +[Trait("Format", "Avif")] +public class Av1MathTests +{ + [Theory] + [InlineData(1, 0)] + [InlineData(2, 1)] + [InlineData(3, 1)] + [InlineData(4, 2)] + [InlineData(5, 2)] + [InlineData(7, 2)] + [InlineData(8, 3)] + [InlineData(9, 3)] + public void TestMostSignificantBit(uint value, int expected) + { + int actual = Av1Math.MostSignificantBit(value); + Assert.Equal(expected, actual); + } + + [Theory] + [InlineData(1, 0)] + [InlineData(2, 1)] + [InlineData(3, 1)] + [InlineData(4, 2)] + [InlineData(5, 2)] + [InlineData(7, 2)] + [InlineData(8, 3)] + [InlineData(9, 3)] + public void TestLog2(int value, int expected) + { + int actual = Av1Math.Log2(value); + Assert.Equal(expected, actual); + } + + [Theory] + [InlineData(1, 0)] + [InlineData(2, 1)] + [InlineData(3, 1)] + [InlineData(4, 2)] + [InlineData(5, 2)] + [InlineData(7, 2)] + [InlineData(8, 3)] + [InlineData(9, 3)] + public void TestFloorLog2(uint value, uint expected) + { + uint actual = Av1Math.FloorLog2(value); + Assert.Equal(expected, actual); + } + + [Theory] + [InlineData(1, 0)] + [InlineData(2, 1)] + [InlineData(3, 1)] + [InlineData(4, 2)] + [InlineData(5, 2)] + [InlineData(7, 2)] + [InlineData(8, 3)] + [InlineData(9, 3)] + public void TestLog2_32(uint value, uint expected) + { + uint actual = Av1Math.Log2_32(value); + Assert.Equal(expected, actual); + } + + [Theory] + [InlineData(1, 0)] + [InlineData(2, 1)] + [InlineData(3, 2)] + [InlineData(4, 2)] + [InlineData(5, 3)] + [InlineData(7, 3)] + [InlineData(8, 3)] + [InlineData(9, 4)] + public void TestLog2Ceiling(uint value, uint expected) + { + uint actual = Av1Math.CeilLog2(value); + Assert.Equal(expected, actual); + } + + [Theory] + [InlineData(4, 2, 1)] + [InlineData(4, 3, 0)] + [InlineData(5, 3, 0)] + [InlineData(8, 3, 1)] + [InlineData(9, 3, 1)] + [InlineData(9, 0, 1)] + [InlineData(8, 0, 0)] + public void TestGetBitSet(int value, int n, int expected) + { + int actual = Av1Math.GetBit(value, n); + Assert.Equal(expected, actual); + } + + [Theory] + [InlineData(4, 2, 4)] + [InlineData(0, 2, 4)] + [InlineData(0, 3, 8)] + [InlineData(4, 3, 12)] + public void TestSetBitSet(int value, int n, int expected) + { + int actual = value; + Av1Math.SetBit(ref actual, n); + Assert.Equal(expected, actual); + } + + [Theory] + [InlineData(255, 4, 4)] + [InlineData(255, -1, 0)] + [InlineData(255, 255, 255)] + [InlineData(255, 256, 255)] + [InlineData(255, 1000, 255)] + public void TestClip3(int max, int value, int expected) + { + int actual = Av1Math.Clip3(0, max, value); + Assert.Equal(expected, actual); + } +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1MotionModeEntropyTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1MotionModeEntropyTests.cs new file mode 100644 index 0000000000..90a670add8 --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1MotionModeEntropyTests.cs @@ -0,0 +1,223 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Buffers; +using SixLabors.ImageSharp.Formats.Heif.Av1; +using SixLabors.ImageSharp.Formats.Heif.Av1.Entropy; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1; + +/// +/// Verifies the entropy defaults, lifecycle, and range-decoder alignment used by AV1 motion-mode syntax. +/// +[Trait("Format", "Avif")] +public class Av1MotionModeEntropyTests +{ + /// + /// Gets the reference decoder's forward Q15 Simple Translation, OBMC, and Warped thresholds in block-size order. + /// + private static ReadOnlySpan MotionModeForwardThresholds => + [ + 10923, 21845, + 10923, 21845, + 10923, 21845, + 7651, 24760, + 4738, 24765, + 5391, 25528, + 19419, 26810, + 5123, 23606, + 11606, 24308, + 26260, 29116, + 20360, 28062, + 21679, 26830, + 29516, 30701, + 28898, 30397, + 30878, 31335, + 32507, 32558, + 10923, 21845, + 10923, 21845, + 28799, 31390, + 26431, 30774, + 28973, 31594, + 29742, 31203, + ]; + + /// + /// Gets the reference decoder's forward Q15 Simple Translation and OBMC thresholds in block-size order. + /// + private static ReadOnlySpan ObmcForwardThresholds => + [ + 16384, 16384, 16384, 10437, 9371, 9301, 17432, 14423, 15142, 25817, 22823, + 22083, 30128, 31014, 31560, 32638, 16384, 16384, 23664, 20901, 24008, 26879, + ]; + + /// + /// Verifies all twenty-two ternary and binary motion-mode distributions against the reference decoder's forward Q15 defaults. + /// + [Fact] + public void DefaultsMatchReference() + { + const int blockSizeCount = (int)Av1BlockSize.AllSizes; + const int ternaryThresholdCount = 2; + ReadOnlySpan motionModeForwardThresholds = MotionModeForwardThresholds; + ReadOnlySpan obmcForwardThresholds = ObmcForwardThresholds; + Av1Distribution[] motionMode = Av1DefaultDistributions.MotionMode; + Av1Distribution[] obmc = Av1DefaultDistributions.Obmc; + + Assert.Equal(blockSizeCount * ternaryThresholdCount, motionModeForwardThresholds.Length); + Assert.Equal(blockSizeCount, obmcForwardThresholds.Length); + Assert.Equal(blockSizeCount, motionMode.Length); + Assert.Equal(blockSizeCount, obmc.Length); + + for (int blockSize = 0; blockSize < blockSizeCount; blockSize++) + { + Assert.Equal(3, motionMode[blockSize].NumberOfSymbols); + Assert.Equal(2, obmc[blockSize].NumberOfSymbols); + + for (int threshold = 0; threshold < ternaryThresholdCount; threshold++) + { + // Av1Distribution stores inverse cumulative thresholds, so complement the reference decoder's published forward + // Q15 values before comparing the exact state consumed by the range decoder. + uint expected = (uint)Av1Distribution.ProbabilityTop - + motionModeForwardThresholds[(blockSize * ternaryThresholdCount) + threshold]; + + Assert.Equal(expected, motionMode[blockSize][threshold]); + } + + uint expectedObmc = (uint)Av1Distribution.ProbabilityTop - obmcForwardThresholds[blockSize]; + + Assert.Equal(expectedObmc, obmc[blockSize][0]); + } + } + + /// + /// Verifies that newly created frame contexts and explicit copies own independent motion-mode distributions. + /// + [Fact] + public void FrameEntropyContextsDeepCopyAndCopyFromMotionModeState() + { + int blockSize = (int)Av1BlockSize.Block16x16; + Av1FrameEntropyContext source = new(0); + Av1FrameEntropyContext destination = new(0); + + Assert.NotSame(source.MotionMode[blockSize], destination.MotionMode[blockSize]); + Assert.NotSame(source.Obmc[blockSize], destination.Obmc[blockSize]); + + source.MotionMode[blockSize].Update((int)Av1MotionMode.Warped); + source.Obmc[blockSize].Update((int)Av1MotionMode.Obmc); + + Assert.NotEqual(source.MotionMode[blockSize][0], destination.MotionMode[blockSize][0]); + Assert.NotEqual(source.Obmc[blockSize][0], destination.Obmc[blockSize][0]); + + destination.CopyFrom(source); + + Assert.Equal(source.MotionMode[blockSize][0], destination.MotionMode[blockSize][0]); + Assert.Equal(source.MotionMode[blockSize][1], destination.MotionMode[blockSize][1]); + Assert.Equal(source.Obmc[blockSize][0], destination.Obmc[blockSize][0]); + + source.MotionMode[blockSize].Update((int)Av1MotionMode.SimpleTranslation); + source.Obmc[blockSize].Update((int)Av1MotionMode.SimpleTranslation); + + Assert.NotEqual(source.MotionMode[blockSize][0], destination.MotionMode[blockSize][0]); + Assert.NotEqual(source.Obmc[blockSize][0], destination.Obmc[blockSize][0]); + } + + /// + /// Verifies that resetting a frame context restores both motion-mode thresholds and adaptation history. + /// + [Fact] + public void FrameEntropyResetRestoresMotionModeState() + { + const int updateCount = 20; + int blockSize = (int)Av1BlockSize.Block32x16; + Av1FrameEntropyContext context = new(0); + Av1FrameEntropyContext expected = new(0); + + for (int update = 0; update < updateCount; update++) + { + context.MotionMode[blockSize].Update((int)Av1MotionMode.Warped); + context.Obmc[blockSize].Update((int)Av1MotionMode.Obmc); + } + + context.ResetToDefaults(0); + + Assert.Equal(expected.MotionMode[blockSize][0], context.MotionMode[blockSize][0]); + Assert.Equal(expected.MotionMode[blockSize][1], context.MotionMode[blockSize][1]); + Assert.Equal(expected.Obmc[blockSize][0], context.Obmc[blockSize][0]); + + // Equal thresholds can still carry different observation counts. Applying the same next symbols proves that + // ResetToDefaults restored the update-rate history as well as the visible probability thresholds. + context.MotionMode[blockSize].Update((int)Av1MotionMode.Obmc); + expected.MotionMode[blockSize].Update((int)Av1MotionMode.Obmc); + context.Obmc[blockSize].Update((int)Av1MotionMode.SimpleTranslation); + expected.Obmc[blockSize].Update((int)Av1MotionMode.SimpleTranslation); + + Assert.Equal(expected.MotionMode[blockSize][0], context.MotionMode[blockSize][0]); + Assert.Equal(expected.MotionMode[blockSize][1], context.MotionMode[blockSize][1]); + Assert.Equal(expected.Obmc[blockSize][0], context.Obmc[blockSize][0]); + } + + /// + /// Verifies that a published frame snapshot retains adapted motion-mode thresholds but resets update counts. + /// + [Fact] + public void FrameEntropySnapshotResetsMotionModeUpdateCounts() + { + const int updateCount = 20; + int blockSize = (int)Av1BlockSize.Block16x32; + Av1FrameEntropyContext source = new(0); + Av1FrameEntropyContext snapshot = new(0); + + for (int update = 0; update < updateCount; update++) + { + source.MotionMode[blockSize].Update((int)Av1MotionMode.Warped); + source.Obmc[blockSize].Update((int)Av1MotionMode.Obmc); + } + + source.SnapshotTo(snapshot); + + Assert.Equal(source.MotionMode[blockSize][0], snapshot.MotionMode[blockSize][0]); + Assert.Equal(source.MotionMode[blockSize][1], snapshot.MotionMode[blockSize][1]); + Assert.Equal(source.Obmc[blockSize][0], snapshot.Obmc[blockSize][0]); + + // The source retains twenty observations while the snapshot restarts at zero. Identical next observations + // therefore move their equal starting thresholds by different update rates. + source.MotionMode[blockSize].Update((int)Av1MotionMode.SimpleTranslation); + snapshot.MotionMode[blockSize].Update((int)Av1MotionMode.SimpleTranslation); + source.Obmc[blockSize].Update((int)Av1MotionMode.SimpleTranslation); + snapshot.Obmc[blockSize].Update((int)Av1MotionMode.SimpleTranslation); + + Assert.NotEqual(source.MotionMode[blockSize][0], snapshot.MotionMode[blockSize][0]); + Assert.NotEqual(source.Obmc[blockSize][0], snapshot.Obmc[blockSize][0]); + } + + /// + /// Verifies that both motion-mode alphabets leave the range decoder aligned for the immediately following filter symbol. + /// + /// Whether the motion-mode symbol uses the ternary rather than binary distribution. + [Theory] + [InlineData(false)] + [InlineData(true)] + public void PreservesFollowingInterpolationSymbol(bool allowWarpedMotion) + { + Av1BlockSize blockSize = Av1BlockSize.Block16x16; + const int interpolationContext = 3; + Av1Distribution motionModeDistribution = allowWarpedMotion + ? Av1DefaultDistributions.MotionMode[(int)blockSize] + : Av1DefaultDistributions.Obmc[(int)blockSize]; + + using Av1SymbolWriter writer = new(Configuration.Default, 2, updateCdf: true); + writer.WriteSymbol((int)Av1MotionMode.SimpleTranslation, motionModeDistribution); + writer.WriteSymbol( + (int)Av1InterpolationFilter.Sharp, + Av1DefaultDistributions.SwitchableInterpolation[interpolationContext]); + + using IMemoryOwner encoded = writer.Exit(); + Av1SymbolDecoder decoder = new(Configuration.Default, encoded.Memory.Span, 0, updateCdf: true); + + Assert.Equal(Av1MotionMode.SimpleTranslation, decoder.ReadMotionMode(blockSize, allowWarpedMotion)); + Assert.Equal(Av1InterpolationFilter.Sharp, decoder.ReadSwitchableInterpolationFilter(interpolationContext)); + } +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1MotionModeInfoTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1MotionModeInfoTests.cs new file mode 100644 index 0000000000..984b3f6dd0 --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1MotionModeInfoTests.cs @@ -0,0 +1,270 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Buffers; +using SixLabors.ImageSharp.Formats.Heif.Av1; +using SixLabors.ImageSharp.Formats.Heif.Av1.Entropy; +using SixLabors.ImageSharp.Formats.Heif.Av1.Motion; +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; +using SixLabors.ImageSharp.Formats.Heif.Av1.ReferenceFrames; +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1; + +/// +/// Verifies inter-intra, motion-mode, and interpolation-filter syntax ordering in inter-frame mode parsing. +/// +[Trait("Format", "Avif")] +public class Av1MotionModeInfoTests +{ + /// + /// Verifies that an extended 8x32 rectangle omits inter-intra syntax and reads the following interpolation filter. + /// + [Fact] + public void OmitsInterIntraForExtendedRectangle() + { + ObuSequenceHeader sequenceHeader = CreateSequenceHeader(); + sequenceHeader.EnableInterIntraCompound = true; + ObuFrameHeader frameHeader = CreateFrameHeader(); + ConfigureForcedTranslationalGlobalMotion(frameHeader); + + using Av1TileReader tileReader = new(Configuration.Default, sequenceHeader, frameHeader); + Av1BlockModeInfo modeInfo = new(Av1BlockSize.Block8x32, Point.Empty); + Av1SuperblockInfo superblockInfo = new(tileReader.FrameInfo, Point.Empty); + Av1PartitionInfo partitionInfo = new(modeInfo, superblockInfo, false, Av1PartitionType.None); + + using Av1SymbolWriter writer = new(Configuration.Default, 2, updateCdf: true); + writer.WriteSymbol(false, Av1DefaultDistributions.Skip[0]); + + // With no matching above or left filter, a single-reference vertical filter uses context three. Writing the + // filter immediately after Skip makes any accidental extended-rectangle inter-intra read desynchronize it. + writer.WriteSymbol((int)Av1InterpolationFilter.Sharp, Av1DefaultDistributions.SwitchableInterpolation[3]); + + using IMemoryOwner encoded = writer.Exit(); + Av1SymbolDecoder decoder = new(Configuration.Default, encoded.Memory.Span, 0, updateCdf: true); + + tileReader.ReadInterFrameModeInfo(ref decoder, ref partitionInfo, new Av1TileInfo(0, 0, frameHeader)); + modeInfo = partitionInfo.ModeInfo; + + Assert.Equal(Av1MotionMode.SimpleTranslation, modeInfo.MotionMode); + Assert.Equal(Av1InterpolationFilter.Sharp, modeInfo.InterpolationFilters[0]); + Assert.Equal(Av1InterpolationFilter.Sharp, modeInfo.InterpolationFilters[1]); + } + + /// + /// Verifies that a false inter-intra flag continues through omitted, binary, and ternary motion-mode syntax into interpolation. + /// + /// Whether the frame enables per-block motion-mode syntax. + /// Whether the eligible block uses the ternary rather than binary motion-mode distribution. + /// The motion mode written when syntax is present. + [Theory] + [InlineData(false, false, (int)Av1MotionMode.SimpleTranslation)] + [InlineData(true, false, (int)Av1MotionMode.SimpleTranslation)] + [InlineData(true, true, (int)Av1MotionMode.SimpleTranslation)] + [InlineData(true, false, (int)Av1MotionMode.Obmc)] + [InlineData(true, true, (int)Av1MotionMode.Obmc)] + public void ContinuesFromInterIntraToInterpolation( + bool isMotionModeSwitchable, + bool allowWarpedMotion, + int selectedMotionModeValue) + { + ObuSequenceHeader sequenceHeader = CreateSequenceHeader(); + sequenceHeader.EnableInterIntraCompound = true; + ObuFrameHeader frameHeader = CreateFrameHeader(); + frameHeader.IsMotionModeSwitchable = isMotionModeSwitchable; + frameHeader.AllowWarpedMotion = allowWarpedMotion; + ConfigureForcedTranslationalGlobalMotion(frameHeader); + + using Av1ReferenceFrameStore referenceFrames = new(); + using Av1FrameInfo retainedFrameInfo = new(sequenceHeader); + Av1ReferenceFrame retainedFrame = new( + new Av1FrameBuffer(Configuration.Default, sequenceHeader, Av1ColorFormat.Yuv400, false), + CreateFrameHeader(), + retainedFrameInfo); + + Assert.True(referenceFrames.Commit(1, retainedFrame, showFrame: false)); + + Av1FrameEntropyContexts entropyContexts = new(0); + using Av1TileReader tileReader = new( + Configuration.Default, + sequenceHeader, + frameHeader, + entropyContexts, + null, + referenceFrames); + + Av1SuperblockInfo superblockInfo = tileReader.FrameInfo.GetSuperblock(Point.Empty); + Av1BlockModeInfo aboveModeInfo = new(Av1BlockSize.Block8x8, Point.Empty) + { + YMode = Av1PredictionMode.NearestMotionVector, + }; + + aboveModeInfo.ReferenceFrames[0] = Av1ReferenceFrameType.Last; + aboveModeInfo.ReferenceFrames[1] = Av1ReferenceFrameType.None; + aboveModeInfo.InterpolationFilters.Clear(); + tileReader.FrameInfo.UpdateModeInfo(aboveModeInfo, superblockInfo); + + Av1BlockModeInfo modeInfo = new(Av1BlockSize.Block8x8, new Point(0, 2)); + Av1PartitionInfo partitionInfo = new(modeInfo, superblockInfo, false, Av1PartitionType.None) + { + ColumnIndex = 0, + RowIndex = 2, + AvailableAbove = true, + AboveModeInfo = aboveModeInfo, + }; + + using Av1SymbolWriter writer = new(Configuration.Default, 4, updateCdf: true); + writer.WriteSymbol(false, Av1DefaultDistributions.Skip[0]); + writer.WriteSymbol(false, Av1DefaultDistributions.InterIntra[Av1BlockSize.Block8x8.GetSizeGroup()]); + + if (isMotionModeSwitchable) + { + Av1Distribution motionModeDistribution = allowWarpedMotion + ? Av1DefaultDistributions.MotionMode[(int)Av1BlockSize.Block8x8] + : Av1DefaultDistributions.Obmc[(int)Av1BlockSize.Block8x8]; + + writer.WriteSymbol(selectedMotionModeValue, motionModeDistribution); + } + + // The matching regular above neighbor selects vertical context zero. Sharp is deliberately non-default so the + // assertion proves that every preceding conditional symbol consumed exactly its own range-coded interval. + writer.WriteSymbol((int)Av1InterpolationFilter.Sharp, Av1DefaultDistributions.SwitchableInterpolation[0]); + + using IMemoryOwner encoded = writer.Exit(); + Av1SymbolDecoder decoder = new(Configuration.Default, encoded.Memory.Span, 0, updateCdf: true); + + tileReader.ReadInterFrameModeInfo(ref decoder, ref partitionInfo, new Av1TileInfo(0, 0, frameHeader)); + modeInfo = partitionInfo.ModeInfo; + + Assert.Equal(Av1ReferenceFrameType.Last, modeInfo.ReferenceFrames[0]); + Assert.Equal(Av1ReferenceFrameType.None, modeInfo.ReferenceFrames[1]); + Av1MotionMode expectedMotionMode = isMotionModeSwitchable + ? (Av1MotionMode)selectedMotionModeValue + : Av1MotionMode.SimpleTranslation; + + Assert.Equal(expectedMotionMode, modeInfo.MotionMode); + Assert.Equal(Av1InterpolationFilter.Sharp, modeInfo.InterpolationFilters[0]); + Assert.Equal(Av1InterpolationFilter.Sharp, modeInfo.InterpolationFilters[1]); + } + + /// + /// Verifies selected smooth and wedge inter-intra syntax before switchable interpolation. + /// + /// Whether the block selects an inter-intra wedge. + [Theory] + [InlineData(false)] + [InlineData(true)] + public void ReadsInterIntraBeforeInterpolation(bool useWedge) + { + ObuSequenceHeader sequenceHeader = CreateSequenceHeader(); + sequenceHeader.EnableInterIntraCompound = true; + ObuFrameHeader frameHeader = CreateFrameHeader(); + ConfigureForcedTranslationalGlobalMotion(frameHeader); + + using Av1TileReader tileReader = new(Configuration.Default, sequenceHeader, frameHeader); + Av1BlockModeInfo modeInfo = new(Av1BlockSize.Block8x8, Point.Empty); + Av1SuperblockInfo superblockInfo = new(tileReader.FrameInfo, Point.Empty); + Av1PartitionInfo partitionInfo = new(modeInfo, superblockInfo, false, Av1PartitionType.None); + + using Av1SymbolWriter writer = new(Configuration.Default, 6, updateCdf: true); + writer.WriteSymbol(false, Av1DefaultDistributions.Skip[0]); + writer.WriteSymbol(true, Av1DefaultDistributions.InterIntra[Av1BlockSize.Block8x8.GetSizeGroup()]); + writer.WriteSymbol( + (int)Av1InterIntraMode.Smooth, + Av1DefaultDistributions.InterIntraMode[Av1BlockSize.Block8x8.GetSizeGroup()]); + + writer.WriteSymbol(useWedge, Av1DefaultDistributions.WedgeInterIntra[(int)Av1BlockSize.Block8x8]); + if (useWedge) + { + writer.WriteSymbol(13, Av1DefaultDistributions.WedgeIndex[(int)Av1BlockSize.Block8x8]); + } + + writer.WriteSymbol((int)Av1InterpolationFilter.Sharp, Av1DefaultDistributions.SwitchableInterpolation[3]); + + using IMemoryOwner encoded = writer.Exit(); + Av1SymbolDecoder decoder = new(Configuration.Default, encoded.Memory.Span, 0, updateCdf: true); + + tileReader.ReadInterFrameModeInfo(ref decoder, ref partitionInfo, new Av1TileInfo(0, 0, frameHeader)); + modeInfo = partitionInfo.ModeInfo; + + Assert.Equal(Av1ReferenceFrameType.Last, modeInfo.ReferenceFrames[0]); + Assert.Equal(Av1ReferenceFrameType.Intra, modeInfo.ReferenceFrames[1]); + Assert.Equal(Av1InterIntraMode.Smooth, modeInfo.InterIntraMode); + Assert.Equal(useWedge, modeInfo.UseInterIntraWedge); + Assert.Equal(useWedge ? 13 : 0, modeInfo.InterIntraWedgeIndex); + Assert.Equal(Av1MotionMode.SimpleTranslation, modeInfo.MotionMode); + Assert.Equal(Av1InterpolationFilter.Sharp, modeInfo.InterpolationFilters[0]); + Assert.Equal(Av1InterpolationFilter.Sharp, modeInfo.InterpolationFilters[1]); + } + + /// + /// Creates the monochrome 64x64 sequence geometry used by direct inter-mode syntax tests. + /// + /// The initialized sequence header. + private static ObuSequenceHeader CreateSequenceHeader() + => new() + { + MaxFrameWidth = 64, + MaxFrameHeight = 64, + Use128x128Superblock = false, + EnableDualFilter = false, + EnableCdef = false, + EnableFilterIntra = false, + ColorConfig = new ObuColorConfig + { + IsMonochrome = true, + BitDepth = Av1BitDepth.EightBit, + }, + }; + + /// + /// Creates an inter-frame header whose one tile and coded dimensions cover the complete test frame. + /// + /// The initialized frame header. + private static ObuFrameHeader CreateFrameHeader() + { + ObuTileGroupHeader tilesInfo = new() + { + TileColumnCount = 1, + TileRowCount = 1, + }; + + tilesInfo.TileColumnStartModeInfo[1] = 16; + tilesInfo.TileRowStartModeInfo[1] = 16; + + return new() + { + FrameType = ObuFrameType.InterFrame, + ModeInfoColumnCount = 16, + ModeInfoRowCount = 16, + CodedLossless = true, + AllowScreenContentTools = false, + InterpolationFilter = Av1InterpolationFilter.Switchable, + FrameSize = new ObuFrameSize + { + FrameWidth = 64, + FrameHeight = 64, + SuperResolutionUpscaledWidth = 64, + RenderWidth = 64, + RenderHeight = 64, + }, + TilesInfo = tilesInfo, + }; + } + + /// + /// Forces segment zero to a translational global-motion mode that omits reference and inter-mode symbols but still carries interpolation. + /// + /// The frame header to configure. + private static void ConfigureForcedTranslationalGlobalMotion(ObuFrameHeader frameHeader) + { + ObuSegmentationParameters segmentationParameters = frameHeader.SegmentationParameters; + segmentationParameters.Enabled = true; + segmentationParameters.SetFeatureEnabled(0, (int)ObuSegmentationLevelFeature.GlobalMotionVector, true); + frameHeader.GetGlobalMotionParameters()[0].Type = Av1GlobalMotionType.Translation; + frameHeader.GetReferenceFrameIndices()[0] = 0; + } +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1MotionVariationCandidatesTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1MotionVariationCandidatesTests.cs new file mode 100644 index 0000000000..bae07daf76 --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1MotionVariationCandidatesTests.cs @@ -0,0 +1,396 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1; +using SixLabors.ImageSharp.Formats.Heif.Av1.Motion; +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1; + +/// +/// Verifies the spatial neighbor and projection-sample rules used to select an AV1 motion mode. +/// +[Trait("Format", "Avif")] +public class Av1MotionVariationCandidatesTests +{ + /// + /// Verifies that overlap detection exhausts the complete above edge before falling back to the complete left edge. + /// + [Fact] + public void BuildScansCompleteAboveEdgeThenFallsBackToLeftEdge() + { + ObuSequenceHeader sequenceHeader = CreateSequenceHeader(); + ObuFrameHeader frameHeader = CreateFrameHeader(); + using Av1FrameInfo frameInfo = new(sequenceHeader); + + for (int offset = 0; offset < 8; offset += 2) + { + AddModeInfo( + frameInfo, + sequenceHeader, + new Point(4 + offset, 2), + Av1BlockSize.Block8x8, + Av1ReferenceFrameType.Intra, + Av1ReferenceFrameType.None, + default); + + Av1ReferenceFrameType leftReference = offset == 6 + ? Av1ReferenceFrameType.Last + : Av1ReferenceFrameType.Intra; + + AddModeInfo( + frameInfo, + sequenceHeader, + new Point(2, 4 + offset), + Av1BlockSize.Block8x8, + leftReference, + Av1ReferenceFrameType.None, + default); + } + + Av1PartitionInfo partitionInfo = CreatePartitionInfo( + frameInfo, + sequenceHeader, + new Point(4, 4), + Av1BlockSize.Block32x32, + availableAbove: true, + availableLeft: true); + + Av1MotionVariationCandidates candidates = new(); + + candidates.Build( + ref partitionInfo, + new Av1TileInfo(0, 0, frameHeader), + sequenceHeader, + frameHeader, + Av1ReferenceFrameType.Last); + + Assert.True(candidates.HasOverlappableNeighbor); + Assert.Equal(1, candidates.Count); + } + + /// + /// Verifies that 4x4 neighbors use the second mode record of each horizontal or vertical 8-sample pair. + /// + [Fact] + public void BuildUsesSecondCellForFourSampleNeighborPairs() + { + ObuSequenceHeader sequenceHeader = CreateSequenceHeader(); + ObuFrameHeader frameHeader = CreateFrameHeader(); + using Av1FrameInfo horizontalFrameInfo = new(sequenceHeader); + AddModeInfo( + horizontalFrameInfo, + sequenceHeader, + new Point(4, 3), + Av1BlockSize.Block4x4, + Av1ReferenceFrameType.Intra, + Av1ReferenceFrameType.None, + default); + + AddModeInfo( + horizontalFrameInfo, + sequenceHeader, + new Point(5, 3), + Av1BlockSize.Block4x4, + Av1ReferenceFrameType.Last, + Av1ReferenceFrameType.None, + default); + + Av1PartitionInfo horizontalPartition = CreatePartitionInfo( + horizontalFrameInfo, + sequenceHeader, + new Point(4, 4), + Av1BlockSize.Block8x8, + availableAbove: true, + availableLeft: false); + + Av1MotionVariationCandidates horizontalCandidates = new(); + horizontalCandidates.Build( + ref horizontalPartition, + new Av1TileInfo(0, 0, frameHeader), + sequenceHeader, + frameHeader, + Av1ReferenceFrameType.Last); + + using Av1FrameInfo verticalFrameInfo = new(sequenceHeader); + AddModeInfo( + verticalFrameInfo, + sequenceHeader, + new Point(3, 4), + Av1BlockSize.Block4x4, + Av1ReferenceFrameType.Intra, + Av1ReferenceFrameType.None, + default); + + AddModeInfo( + verticalFrameInfo, + sequenceHeader, + new Point(3, 5), + Av1BlockSize.Block4x4, + Av1ReferenceFrameType.Last, + Av1ReferenceFrameType.None, + default); + + Av1PartitionInfo verticalPartition = CreatePartitionInfo( + verticalFrameInfo, + sequenceHeader, + new Point(4, 4), + Av1BlockSize.Block8x8, + availableAbove: false, + availableLeft: true); + + Av1MotionVariationCandidates verticalCandidates = new(); + verticalCandidates.Build( + ref verticalPartition, + new Av1TileInfo(0, 0, frameHeader), + sequenceHeader, + frameHeader, + Av1ReferenceFrameType.Last); + + Assert.True(horizontalCandidates.HasOverlappableNeighbor); + Assert.True(verticalCandidates.HasOverlappableNeighbor); + } + + /// + /// Verifies that projection samples require a matching single reference and stop at the normative capacity of eight. + /// + [Fact] + public void BuildCollectsMatchingSingleReferenceSamplesUpToCapacity() + { + ObuSequenceHeader sequenceHeader = CreateSequenceHeader(); + ObuFrameHeader frameHeader = CreateFrameHeader(); + using Av1FrameInfo frameInfo = new(sequenceHeader); + + // The first candidate has the wrong primary reference and the second is compound. Ten following candidates + // are eligible, so the retained range must begin at offset two and stop after eight samples at offset nine. + for (int offset = 0; offset < 12; offset++) + { + Av1ReferenceFrameType primaryReference = offset == 0 + ? Av1ReferenceFrameType.Golden + : Av1ReferenceFrameType.Last; + + Av1ReferenceFrameType secondaryReference = offset == 1 + ? Av1ReferenceFrameType.Golden + : Av1ReferenceFrameType.None; + + AddModeInfo( + frameInfo, + sequenceHeader, + new Point(8 + offset, 15), + Av1BlockSize.Block4x4, + primaryReference, + secondaryReference, + new Av1MotionVector(offset, offset + 1)); + } + + Av1PartitionInfo partitionInfo = CreatePartitionInfo( + frameInfo, + sequenceHeader, + new Point(8, 16), + Av1BlockSize.Block64x64, + availableAbove: true, + availableLeft: false); + + Av1MotionVariationCandidates candidates = new(); + + candidates.Build( + ref partitionInfo, + new Av1TileInfo(0, 0, frameHeader), + sequenceHeader, + frameHeader, + Av1ReferenceFrameType.Last); + + Assert.Equal(8, candidates.Count); + + // Positions are Q3 neighbor centers relative to the current block. Reference points add the corresponding + // Q3 motion vector without rounding, which also proves that the rejected first two candidates were skipped. + Assert.Equal(new Point(72, -24), candidates.SourcePoints[0]); + Assert.Equal(new Point(75, -22), candidates.ReferencePoints[0]); + Assert.Equal(new Point(296, -24), candidates.SourcePoints[7]); + Assert.Equal(new Point(306, -15), candidates.ReferencePoints[7]); + } + + /// + /// Verifies that eligible top-left and top-right diagonal blocks contribute after the direct edge neighbors. + /// + [Fact] + public void BuildIncludesEligibleTopLeftAndTopRightSamples() + { + ObuSequenceHeader sequenceHeader = CreateSequenceHeader(); + ObuFrameHeader frameHeader = CreateFrameHeader(); + using Av1FrameInfo frameInfo = new(sequenceHeader); + AddModeInfo(frameInfo, sequenceHeader, new Point(4, 2), Av1BlockSize.Block8x8, Av1ReferenceFrameType.Last, Av1ReferenceFrameType.None, default); + AddModeInfo(frameInfo, sequenceHeader, new Point(2, 4), Av1BlockSize.Block8x8, Av1ReferenceFrameType.Last, Av1ReferenceFrameType.None, default); + AddModeInfo(frameInfo, sequenceHeader, new Point(2, 2), Av1BlockSize.Block8x8, Av1ReferenceFrameType.Last, Av1ReferenceFrameType.None, default); + AddModeInfo(frameInfo, sequenceHeader, new Point(6, 2), Av1BlockSize.Block8x8, Av1ReferenceFrameType.Last, Av1ReferenceFrameType.None, default); + + Av1PartitionInfo partitionInfo = CreatePartitionInfo( + frameInfo, + sequenceHeader, + new Point(4, 4), + Av1BlockSize.Block8x8, + availableAbove: true, + availableLeft: true); + + Av1MotionVariationCandidates candidates = new(); + + candidates.Build( + ref partitionInfo, + new Av1TileInfo(0, 0, frameHeader), + sequenceHeader, + frameHeader, + Av1ReferenceFrameType.Last); + + Assert.Equal(4, candidates.Count); + Assert.Equal(new Point(24, -40), candidates.SourcePoints[0]); + Assert.Equal(new Point(-40, 24), candidates.SourcePoints[1]); + Assert.Equal(new Point(-40, -40), candidates.SourcePoints[2]); + Assert.Equal(new Point(88, -40), candidates.SourcePoints[3]); + } + + /// + /// Verifies that edge blocks already covering the diagonal positions suppress duplicate top-left and top-right samples. + /// + [Fact] + public void BuildSuppressesDiagonalSamplesCoveredByEdgeNeighbors() + { + ObuSequenceHeader sequenceHeader = CreateSequenceHeader(); + ObuFrameHeader frameHeader = CreateFrameHeader(); + using Av1FrameInfo frameInfo = new(sequenceHeader); + + // The aligned 16x8 above block covers the top-right position. The 8x16 left block begins two mode-info rows + // above the current block and therefore covers its top-left position. + AddModeInfo(frameInfo, sequenceHeader, new Point(4, 4), Av1BlockSize.Block16x8, Av1ReferenceFrameType.Last, Av1ReferenceFrameType.None, default); + AddModeInfo(frameInfo, sequenceHeader, new Point(2, 4), Av1BlockSize.Block8x16, Av1ReferenceFrameType.Last, Av1ReferenceFrameType.None, default); + + Av1PartitionInfo partitionInfo = CreatePartitionInfo( + frameInfo, + sequenceHeader, + new Point(4, 6), + Av1BlockSize.Block8x8, + availableAbove: true, + availableLeft: true); + + Av1MotionVariationCandidates candidates = new(); + + candidates.Build( + ref partitionInfo, + new Av1TileInfo(0, 0, frameHeader), + sequenceHeader, + frameHeader, + Av1ReferenceFrameType.Last); + + Assert.Equal(2, candidates.Count); + } + + /// + /// Creates the monochrome 128x128 sequence geometry used by spatial motion-mode tests. + /// + /// The initialized sequence header. + private static ObuSequenceHeader CreateSequenceHeader() + => new() + { + MaxFrameWidth = 128, + MaxFrameHeight = 128, + Use128x128Superblock = false, + ColorConfig = new ObuColorConfig + { + IsMonochrome = true, + BitDepth = Av1BitDepth.EightBit, + }, + }; + + /// + /// Creates a single-tile frame covering the complete sequence geometry. + /// + /// The initialized frame header. + private static ObuFrameHeader CreateFrameHeader() + { + ObuTileGroupHeader tilesInfo = new() + { + TileColumnCount = 1, + TileRowCount = 1, + }; + + tilesInfo.TileColumnStartModeInfo[1] = 32; + tilesInfo.TileRowStartModeInfo[1] = 32; + + return new() + { + FrameType = ObuFrameType.InterFrame, + ModeInfoColumnCount = 32, + ModeInfoRowCount = 32, + TilesInfo = tilesInfo, + }; + } + + /// + /// Creates one current partition at a frame-relative mode-information position. + /// + /// The frame map containing the neighboring mode records. + /// The sequence geometry defining superblock-relative addressing. + /// The frame-relative block origin in 4x4 units. + /// The current block geometry. + /// Whether the above edge is available. + /// Whether the left edge is available. + /// The initialized partition state. + private static Av1PartitionInfo CreatePartitionInfo( + Av1FrameInfo frameInfo, + ObuSequenceHeader sequenceHeader, + Point position, + Av1BlockSize blockSize, + bool availableAbove, + bool availableLeft) + { + int superblockSize = sequenceHeader.SuperblockModeInfoSize; + Point superblockPosition = new(position.X / superblockSize, position.Y / superblockSize); + Point relativePosition = new(position.X % superblockSize, position.Y % superblockSize); + Av1BlockModeInfo modeInfo = new(blockSize, relativePosition); + + return new Av1PartitionInfo(modeInfo, frameInfo.GetSuperblock(superblockPosition), false, Av1PartitionType.None) + { + ColumnIndex = position.X, + RowIndex = position.Y, + AvailableAbove = availableAbove, + AvailableLeft = availableLeft, + }; + } + + /// + /// Creates and maps one decoded neighbor at a frame-relative mode-information position. + /// + /// The frame map that owns the neighbor. + /// The sequence geometry defining superblock-relative addressing. + /// The frame-relative block origin in 4x4 units. + /// The neighboring block geometry. + /// The primary prediction reference. + /// The optional secondary prediction reference. + /// The primary motion vector in one-eighth-sample units. + private static void AddModeInfo( + Av1FrameInfo frameInfo, + ObuSequenceHeader sequenceHeader, + Point position, + Av1BlockSize blockSize, + Av1ReferenceFrameType primaryReference, + Av1ReferenceFrameType secondaryReference, + Av1MotionVector motionVector) + { + int superblockSize = sequenceHeader.SuperblockModeInfoSize; + Point superblockPosition = new(position.X / superblockSize, position.Y / superblockSize); + Point relativePosition = new(position.X % superblockSize, position.Y % superblockSize); + Av1SuperblockInfo superblockInfo = frameInfo.GetSuperblock(superblockPosition); + Av1BlockModeInfo modeInfo = new(blockSize, relativePosition) + { + YMode = primaryReference == Av1ReferenceFrameType.Intra + ? Av1PredictionMode.DC + : Av1PredictionMode.NearestMotionVector, + }; + + modeInfo.ReferenceFrames[0] = primaryReference; + modeInfo.ReferenceFrames[1] = secondaryReference; + modeInfo.MotionVectors[0] = motionVector; + frameInfo.UpdateModeInfo(modeInfo, superblockInfo); + } +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1MotionVectorEntropyTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1MotionVectorEntropyTests.cs new file mode 100644 index 0000000000..9040b66c10 --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1MotionVectorEntropyTests.cs @@ -0,0 +1,376 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Buffers; +using SixLabors.ImageSharp.Formats.Heif.Av1.Entropy; +using SixLabors.ImageSharp.Formats.Heif.Av1.Motion; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1; + +/// +/// Verifies the adaptive AV1 normal and displacement motion-vector entropy contexts. +/// +[Trait("Format", "Avif")] +public class Av1MotionVectorEntropyTests +{ + /// + /// Verifies both motion-vector contexts against every normative forward Q15 default from the reference decoder. + /// + [Fact] + public void MotionVectorDefaultsMatchReference() + { + Av1FrameEntropyContext context = new(0); + + Assert.NotSame(context.MotionVector, context.DisplacementVector); + AssertContextDefaults(context.MotionVector); + AssertContextDefaults(context.DisplacementVector); + } + + /// + /// Verifies integer, quarter-sample, and eighth-sample syntax consumption and reconstruction. + /// + /// The numeric motion-vector precision. + /// Indicates whether the coded delta occupies the horizontal component. + /// The coded magnitude class. + /// The coded integer magnitude offset. + /// The coded fractional symbol, or negative when omitted. + /// The coded eighth-sample symbol, or negative when omitted. + /// The expected positive component in one-eighth-sample units. + [Theory] + [InlineData((int)Av1MotionVectorPrecision.Integer, false, 0, 1, -1, -1, 16)] + [InlineData((int)Av1MotionVectorPrecision.QuarterSample, true, 0, 0, 2, -1, 6)] + [InlineData((int)Av1MotionVectorPrecision.EighthSample, false, 0, 1, 1, 0, 11)] + [InlineData((int)Av1MotionVectorPrecision.EighthSample, true, 1, 1, 3, 1, 32)] + public void ReadMotionVectorUsesRequestedPrecision( + int precisionValue, + bool horizontal, + int magnitudeClass, + int integerOffset, + int fractional, + int highPrecision, + int expectedMagnitude) + { + Av1MotionVectorPrecision precision = (Av1MotionVectorPrecision)precisionValue; + Av1MotionVectorContext writerContext = new(); + Av1MotionVectorContext.Component component = horizontal ? writerContext.Horizontal : writerContext.Vertical; + Av1Distribution trailingDistribution = Av1DefaultDistributions.Drl[1]; + using Av1SymbolWriter writer = new(Configuration.Default, 8, updateCdf: true); + + writer.WriteSymbol(horizontal ? 1 : 2, writerContext.Joint); + writer.WriteSymbol(false, component.Sign); + writer.WriteSymbol(magnitudeClass, component.MagnitudeClass); + + if (magnitudeClass == 0) + { + writer.WriteSymbol(integerOffset, component.ClassZero); + } + else + { + // CLASS0_BITS is one, so a nonzero class transmits exactly magnitudeClass integer-offset bits. + for (int bit = 0; bit < magnitudeClass; bit++) + { + writer.WriteSymbol((integerOffset >> bit) & 1, component.OffsetBits[bit]); + } + } + + if (precision != Av1MotionVectorPrecision.Integer) + { + Av1Distribution fractionalDistribution = magnitudeClass == 0 + ? component.ClassZeroFractional[integerOffset] + : component.Fractional; + + writer.WriteSymbol(fractional, fractionalDistribution); + } + + if (precision == Av1MotionVectorPrecision.EighthSample) + { + Av1Distribution highPrecisionDistribution = magnitudeClass == 0 + ? component.ClassZeroHighPrecision + : component.HighPrecision; + + writer.WriteSymbol(highPrecision, highPrecisionDistribution); + } + + // The trailing decision detects either an omitted precision symbol being consumed or a required one being skipped. + writer.WriteSymbol(true, trailingDistribution); + + using IMemoryOwner encoded = writer.Exit(); + Av1FrameEntropyContext decoderContext = new(0); + uint normalJoint = decoderContext.MotionVector.Joint[0]; + uint displacementJoint = decoderContext.DisplacementVector.Joint[0]; + Av1SymbolDecoder decoder = new(Configuration.Default, encoded.Memory.Span, decoderContext, updateCdf: true); + Av1MotionVector reference = new(27, -11); + + Av1MotionVector actual = decoder.ReadMotionVector(reference, precision); + Av1MotionVector expected = horizontal + ? new Av1MotionVector(reference.Row, reference.Column + expectedMagnitude) + : new Av1MotionVector(reference.Row + expectedMagnitude, reference.Column); + + Assert.Equal(expected, actual); + Assert.NotEqual(normalJoint, decoderContext.MotionVector.Joint[0]); + Assert.Equal(displacementJoint, decoderContext.DisplacementVector.Joint[0]); + Assert.True(decoder.ReadDrl(1)); + } + + [Theory] + [InlineData((int)Av1MotionVectorPrecision.Integer, 16, -24)] + [InlineData((int)Av1MotionVectorPrecision.QuarterSample, 6, -10)] + [InlineData((int)Av1MotionVectorPrecision.EighthSample, 11, -17)] + public void WriteMotionVectorRoundTripsRequestedPrecision( + int precisionValue, + int rowDelta, + int columnDelta) + { + Av1MotionVectorPrecision precision = (Av1MotionVectorPrecision)precisionValue; + Av1MotionVector reference = new(27, -11); + Av1MotionVector value = reference + new Av1MotionVector(rowDelta, columnDelta); + Av1MotionVectorContext writerContext = new(); + Av1Distribution trailingDistribution = Av1DefaultDistributions.Drl[1]; + using Av1SymbolWriter writer = new(Configuration.Default, 32, updateCdf: true); + + writerContext.Write(writer, value, reference, precision); + writer.WriteSymbol(true, trailingDistribution); + + using IMemoryOwner encoded = writer.Exit(); + Av1FrameEntropyContext decoderContext = new(0); + Av1SymbolDecoder decoder = new( + Configuration.Default, + encoded.Memory.Span, + decoderContext, + updateCdf: true); + + Assert.Equal(value, decoder.ReadMotionVector(reference, precision)); + Assert.True(decoder.ReadDrl(1)); + } + + /// + /// Verifies that normal and displacement motion vectors never share adaptive distribution state. + /// + [Fact] + public void NormalAndDisplacementContextsAdaptIndependently() + { + Av1FrameEntropyContext context = new(0); + uint displacementJoint = context.DisplacementVector.Joint[0]; + uint normalFractional = context.MotionVector.Vertical.Fractional[0]; + + context.MotionVector.Joint.Update(3); + context.DisplacementVector.Vertical.Fractional.Update(2); + + Assert.NotEqual(displacementJoint, context.MotionVector.Joint[0]); + Assert.Equal(displacementJoint, context.DisplacementVector.Joint[0]); + Assert.Equal(normalFractional, context.MotionVector.Vertical.Fractional[0]); + Assert.NotEqual(normalFractional, context.DisplacementVector.Vertical.Fractional[0]); + } + + /// + /// Verifies that frame-context copies retain complete motion-vector state without sharing it. + /// + [Fact] + public void FrameEntropyCopyRetainsIndependentMotionVectorState() + { + Av1FrameEntropyContext source = new(0); + Av1FrameEntropyContext destination = new(0); + UpdateState(source.MotionVector, 1, 5); + UpdateState(source.DisplacementVector, 1, 7); + + destination.CopyFrom(source); + + AssertStateEqual(source.MotionVector, destination.MotionVector); + AssertStateEqual(source.DisplacementVector, destination.DisplacementVector); + + UpdateState(source.MotionVector, 0, 1); + UpdateState(source.DisplacementVector, 0, 1); + + AssertStateNotEqual(source.MotionVector, destination.MotionVector); + AssertStateNotEqual(source.DisplacementVector, destination.DisplacementVector); + } + + /// + /// Verifies that publishing frame state resets every motion-vector update count. + /// + [Fact] + public void FrameEntropySnapshotResetsMotionVectorUpdateCounts() + { + Av1FrameEntropyContext source = new(0); + Av1FrameEntropyContext snapshot = new(0); + UpdateState(source.MotionVector, 1, 20); + UpdateState(source.DisplacementVector, 1, 20); + + source.SnapshotTo(snapshot); + + AssertStateEqual(source.MotionVector, snapshot.MotionVector); + AssertStateEqual(source.DisplacementVector, snapshot.DisplacementVector); + + // The source retains twenty observations while the snapshot restarts at zero. The same next symbol therefore + // moves identical thresholds by different amounts only when every new distribution participates in reset. + UpdateState(source.MotionVector, 0, 1); + UpdateState(snapshot.MotionVector, 0, 1); + UpdateState(source.DisplacementVector, 0, 1); + UpdateState(snapshot.DisplacementVector, 0, 1); + + AssertStateNotEqual(source.MotionVector, snapshot.MotionVector); + AssertStateNotEqual(source.DisplacementVector, snapshot.DisplacementVector); + } + + /// + /// Verifies one complete motion-vector context against the normative defaults. + /// + /// The context under test. + private static void AssertContextDefaults(Av1MotionVectorContext context) + { + Assert.NotSame(context.Vertical, context.Horizontal); + AssertDistribution(context.Joint, [4096, 11264, 19328]); + AssertComponentDefaults(context.Vertical); + AssertComponentDefaults(context.Horizontal); + } + + /// + /// Verifies one component's complete set of normative defaults. + /// + /// The component under test. + private static void AssertComponentDefaults(Av1MotionVectorContext.Component component) + { + AssertDistribution(component.MagnitudeClass, [28672, 30976, 31858, 32320, 32551, 32656, 32740, 32757, 32762, 32767]); + Assert.Equal(2, component.ClassZeroFractional.Length); + AssertDistribution(component.ClassZeroFractional[0], [16384, 24576, 26624]); + AssertDistribution(component.ClassZeroFractional[1], [12288, 21248, 24128]); + AssertDistribution(component.Fractional, [8192, 17408, 21248]); + AssertDistribution(component.Sign, [16384]); + AssertDistribution(component.ClassZeroHighPrecision, [20480]); + AssertDistribution(component.HighPrecision, [16384]); + AssertDistribution(component.ClassZero, [27648]); + + ReadOnlySpan offsetThresholds = [17408, 17920, 18944, 20480, 22528, 24576, 28672, 29952, 29952, 30720]; + + Assert.Equal(offsetThresholds.Length, component.OffsetBits.Length); + for (int bit = 0; bit < offsetThresholds.Length; bit++) + { + uint expected = (uint)Av1Distribution.ProbabilityTop - offsetThresholds[bit]; + + Assert.Equal(2, component.OffsetBits[bit].NumberOfSymbols); + Assert.Equal(expected, component.OffsetBits[bit][0]); + } + } + + /// + /// Verifies one distribution after conversion from forward to inverse cumulative thresholds. + /// + /// The distribution under test. + /// The normative forward Q15 thresholds. + private static void AssertDistribution(Av1Distribution distribution, ReadOnlySpan forwardThresholds) + { + Assert.Equal(forwardThresholds.Length + 1, distribution.NumberOfSymbols); + for (int threshold = 0; threshold < forwardThresholds.Length; threshold++) + { + uint expected = (uint)Av1Distribution.ProbabilityTop - forwardThresholds[threshold]; + + Assert.Equal(expected, distribution[threshold]); + } + } + + /// + /// Applies the same observations to every distribution in a motion-vector context. + /// + /// The context to adapt. + /// The coded symbol used for each observation. + /// The number of observations. + private static void UpdateState(Av1MotionVectorContext context, int symbol, int count) + { + for (int update = 0; update < count; update++) + { + context.Joint.Update(symbol); + UpdateState(context.Vertical, symbol); + UpdateState(context.Horizontal, symbol); + } + } + + /// + /// Applies one observation to every distribution in one motion-vector component. + /// + /// The component to adapt. + /// The coded symbol used for the observation. + private static void UpdateState(Av1MotionVectorContext.Component component, int symbol) + { + component.MagnitudeClass.Update(symbol); + component.ClassZeroFractional[0].Update(symbol); + component.ClassZeroFractional[1].Update(symbol); + component.Fractional.Update(symbol); + component.Sign.Update(symbol); + component.ClassZeroHighPrecision.Update(symbol); + component.HighPrecision.Update(symbol); + component.ClassZero.Update(symbol); + + for (int bit = 0; bit < component.OffsetBits.Length; bit++) + { + component.OffsetBits[bit].Update(symbol); + } + } + + /// + /// Verifies equal adaptive thresholds across two motion-vector contexts. + /// + /// The expected context. + /// The actual context. + private static void AssertStateEqual(Av1MotionVectorContext expected, Av1MotionVectorContext actual) + { + Assert.Equal(expected.Joint[0], actual.Joint[0]); + AssertStateEqual(expected.Vertical, actual.Vertical); + AssertStateEqual(expected.Horizontal, actual.Horizontal); + } + + /// + /// Verifies equal adaptive thresholds across two motion-vector components. + /// + /// The expected component. + /// The actual component. + private static void AssertStateEqual(Av1MotionVectorContext.Component expected, Av1MotionVectorContext.Component actual) + { + Assert.Equal(expected.MagnitudeClass[0], actual.MagnitudeClass[0]); + Assert.Equal(expected.ClassZeroFractional[0][0], actual.ClassZeroFractional[0][0]); + Assert.Equal(expected.ClassZeroFractional[1][0], actual.ClassZeroFractional[1][0]); + Assert.Equal(expected.Fractional[0], actual.Fractional[0]); + Assert.Equal(expected.Sign[0], actual.Sign[0]); + Assert.Equal(expected.ClassZeroHighPrecision[0], actual.ClassZeroHighPrecision[0]); + Assert.Equal(expected.HighPrecision[0], actual.HighPrecision[0]); + Assert.Equal(expected.ClassZero[0], actual.ClassZero[0]); + + for (int bit = 0; bit < expected.OffsetBits.Length; bit++) + { + Assert.Equal(expected.OffsetBits[bit][0], actual.OffsetBits[bit][0]); + } + } + + /// + /// Verifies independently adaptive thresholds across two motion-vector contexts. + /// + /// The independently adapted context. + /// The copied or reset context. + private static void AssertStateNotEqual(Av1MotionVectorContext expected, Av1MotionVectorContext actual) + { + Assert.NotEqual(expected.Joint[0], actual.Joint[0]); + AssertStateNotEqual(expected.Vertical, actual.Vertical); + AssertStateNotEqual(expected.Horizontal, actual.Horizontal); + } + + /// + /// Verifies independently adaptive thresholds across two motion-vector components. + /// + /// The independently adapted component. + /// The copied or reset component. + private static void AssertStateNotEqual(Av1MotionVectorContext.Component expected, Av1MotionVectorContext.Component actual) + { + Assert.NotEqual(expected.MagnitudeClass[0], actual.MagnitudeClass[0]); + Assert.NotEqual(expected.ClassZeroFractional[0][0], actual.ClassZeroFractional[0][0]); + Assert.NotEqual(expected.ClassZeroFractional[1][0], actual.ClassZeroFractional[1][0]); + Assert.NotEqual(expected.Fractional[0], actual.Fractional[0]); + Assert.NotEqual(expected.Sign[0], actual.Sign[0]); + Assert.NotEqual(expected.ClassZeroHighPrecision[0], actual.ClassZeroHighPrecision[0]); + Assert.NotEqual(expected.HighPrecision[0], actual.HighPrecision[0]); + Assert.NotEqual(expected.ClassZero[0], actual.ClassZero[0]); + + for (int bit = 0; bit < expected.OffsetBits.Length; bit++) + { + Assert.NotEqual(expected.OffsetBits[bit][0], actual.OffsetBits[bit][0]); + } + } +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1MotionVectorTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1MotionVectorTests.cs new file mode 100644 index 0000000000..7aaaae5381 --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1MotionVectorTests.cs @@ -0,0 +1,158 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1.Motion; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1; + +/// +/// Verifies AV1 motion-vector precision, range, spatial-clamp, and temporal-projection semantics. +/// +[Trait("Format", "Avif")] +public class Av1MotionVectorTests +{ + /// + /// Verifies that high-precision vectors retain their one-eighth-sample components unchanged. + /// + [Fact] + public void LowerPrecisionRetainsHighPrecisionComponents() + { + Av1MotionVector vector = new(15, -15); + + Assert.Equal(vector, vector.LowerPrecision(allowHighPrecision: true, forceInteger: false)); + } + + /// + /// Verifies that low precision removes odd one-eighth-sample components toward zero. + /// + /// The original vertical component. + /// The original horizontal component. + /// The expected low-precision vertical component. + /// The expected low-precision horizontal component. + [Theory] + [InlineData(15, -15, 14, -14)] + [InlineData(14, -14, 14, -14)] + [InlineData(1, -1, 0, 0)] + public void LowerPrecisionReducesOddComponentsTowardZero(int row, int column, int expectedRow, int expectedColumn) + { + Av1MotionVector actual = new Av1MotionVector(row, column).LowerPrecision(allowHighPrecision: false, forceInteger: false); + + Assert.Equal(new Av1MotionVector(expectedRow, expectedColumn), actual); + } + + /// + /// Verifies AV1 integer-sample rounding, including half-sample ties toward zero on both signs. + /// + /// The original component in one-eighth-sample units. + /// The expected integer-precision component. + [Theory] + [InlineData(3, 0)] + [InlineData(4, 0)] + [InlineData(5, 8)] + [InlineData(11, 8)] + [InlineData(12, 8)] + [InlineData(13, 16)] + [InlineData(16, 16)] + [InlineData(-3, 0)] + [InlineData(-4, 0)] + [InlineData(-5, -8)] + [InlineData(-11, -8)] + [InlineData(-12, -8)] + [InlineData(-13, -16)] + [InlineData(-16, -16)] + public void LowerPrecisionRoundsIntegerHalfTiesTowardZero(int component, int expected) + { + Av1MotionVector actual = new Av1MotionVector(component, -component).LowerPrecision( + allowHighPrecision: true, + forceInteger: true); + + Assert.Equal(new Av1MotionVector(expected, -expected), actual); + } + + /// + /// Verifies that AV1 reserves both signed motion-vector endpoints. + /// + /// The vertical component. + /// The horizontal component. + /// The expected validity. + [Theory] + [InlineData(-16383, 16383, true)] + [InlineData(-16384, 0, false)] + [InlineData(-16385, 0, false)] + [InlineData(16384, 0, false)] + [InlineData(16385, 0, false)] + [InlineData(0, -16384, false)] + [InlineData(0, 16384, false)] + public void IsValidUsesExclusiveMotionVectorEndpoints(int row, int column, bool expected) + => Assert.Equal(expected, new Av1MotionVector(row, column).IsValid); + + /// + /// Verifies the complete-block and sixteen-sample borders used to clamp spatial reference candidates. + /// + [Fact] + public void ClampReferenceMatchesSpatialLimits() + { + const int blockWidth = 16; + const int blockHeight = 8; + const int blockToLeftEdge = -256; + const int blockToRightEdge = 512; + const int blockToTopEdge = -128; + const int blockToBottomEdge = 384; + + Av1MotionVector upper = new Av1MotionVector(1000, 1000).ClampReference( + blockWidth, + blockHeight, + blockToLeftEdge, + blockToRightEdge, + blockToTopEdge, + blockToBottomEdge); + + Av1MotionVector lower = new Av1MotionVector(-1000, -1000).ClampReference( + blockWidth, + blockHeight, + blockToLeftEdge, + blockToRightEdge, + blockToTopEdge, + blockToBottomEdge); + + Av1MotionVector inside = new Av1MotionVector(48, -64).ClampReference( + blockWidth, + blockHeight, + blockToLeftEdge, + blockToRightEdge, + blockToTopEdge, + blockToBottomEdge); + + Assert.Equal(new Av1MotionVector(576, 768), upper); + Assert.Equal(new Av1MotionVector(-320, -512), lower); + Assert.Equal(new Av1MotionVector(48, -64), inside); + } + + /// + /// Verifies AV1 fixed-point temporal projection, distance limiting, symmetric rounding, and endpoint clamping. + /// + /// The source vertical component. + /// The source horizontal component. + /// The signed source-to-target frame distance. + /// The positive source-to-reference frame distance. + /// The expected projected vertical component. + /// The expected projected horizontal component. + [Theory] + [InlineData(64, -96, 2, 4, 32, -48)] + [InlineData(64, -96, -2, 4, -32, 48)] + [InlineData(2, -2, 1, 3, 1, -1)] + [InlineData(31, -31, 40, 40, 31, -31)] + [InlineData(4095, -4095, 31, 1, 16383, -16383)] + public void ProjectTemporalMatchesReference( + int row, + int column, + int numerator, + int denominator, + int expectedRow, + int expectedColumn) + { + Av1MotionVector actual = new Av1MotionVector(row, column).ProjectTemporal(numerator, denominator); + + Assert.Equal(new Av1MotionVector(expectedRow, expectedColumn), actual); + } +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1PaletteKMeans2DTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1PaletteKMeans2DTests.cs new file mode 100644 index 0000000000..a06c90b7de --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1PaletteKMeans2DTests.cs @@ -0,0 +1,131 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline; +using SixLabors.ImageSharp.Tests.TestUtilities; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1; + +/// +/// Verifies paired AV1 palette clustering against independent scalar results at every intrinsic tier. +/// +[Trait("Format", "Avif")] +public class Av1PaletteKMeans2DTests +{ + private const HwIntrinsics Configurations = + HwIntrinsics.AllowAll | HwIntrinsics.DisableAVX512F | HwIntrinsics.DisableAVX | HwIntrinsics.DisableHWIntrinsic; + + [Fact] + public void AssignIndicesMatchesScalarAtEveryIntrinsicTier() + => FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidateAssignment, Configurations); + + [Fact] + public void ClusterMatchesReferenceFixture() + { + short[] firstSamples = [0, 2, 0, 100, 102, 100, 200, 202, 200]; + short[] secondSamples = [10, 10, 12, 110, 110, 112, 210, 210, 212]; + short[] firstCentroids = [20, 100, 180]; + short[] secondCentroids = [30, 110, 190]; + byte[] indices = new byte[firstSamples.Length]; + short[] alternateFirstCentroids = new short[firstCentroids.Length]; + short[] alternateSecondCentroids = new short[secondCentroids.Length]; + byte[] alternateIndices = new byte[firstSamples.Length]; + + long distortion = Av1PaletteKMeans2D.Cluster( + firstSamples, + secondSamples, + firstCentroids, + secondCentroids, + indices, + alternateFirstCentroids, + alternateSecondCentroids, + alternateIndices); + + Assert.Equal([1, 101, 201], firstCentroids); + Assert.Equal([11, 111, 211], secondCentroids); + Assert.Equal([0, 0, 0, 1, 1, 1, 2, 2, 2], indices); + Assert.Equal(18, distortion); + } + + [Fact] + public void InitializeCentroidsMatchesReferenceIntegerOrder() + { + short[] firstCentroids = new short[3]; + short[] secondCentroids = new short[3]; + + Av1PaletteKMeans2D.InitializeCentroids(10, 250, 20, 260, firstCentroids, secondCentroids); + + Assert.Equal([50, 130, 210], firstCentroids); + Assert.Equal([60, 140, 220], secondCentroids); + } + + private static void ValidateAssignment() + { + const int SampleCount = 95; + short[] firstCentroids = [0, 512, 1024, 2048, 3072, 4095]; + short[] secondCentroids = [4094, 3072, 2048, 1024, 512, 0]; + short[] firstSamples = new short[SampleCount]; + short[] secondSamples = new short[SampleCount]; + for (int index = 0; index < SampleCount; index++) + { + firstSamples[index] = (short)(((index * 977) + (index * index * 17)) & 4095); + secondSamples[index] = (short)(((index * 619) + (index * index * 29)) & 4095); + } + + // The first sample is equidistant from the first two colors and must retain the first palette index. + firstSamples[0] = 256; + secondSamples[0] = 3583; + byte[] expected = new byte[SampleCount]; + long expectedDistortion = AssignReference( + firstSamples, + secondSamples, + firstCentroids, + secondCentroids, + expected); + + byte[] actual = Enumerable.Repeat(byte.MaxValue, SampleCount + 7).ToArray(); + long actualDistortion = Av1PaletteKMeans2D.AssignIndices( + firstSamples, + secondSamples, + firstCentroids, + secondCentroids, + actual); + + Assert.Equal(expectedDistortion, actualDistortion); + Assert.Equal(expected, actual.AsSpan(..SampleCount).ToArray()); + Assert.All(actual[SampleCount..], value => Assert.Equal(byte.MaxValue, value)); + } + + private static long AssignReference( + ReadOnlySpan firstSamples, + ReadOnlySpan secondSamples, + ReadOnlySpan firstCentroids, + ReadOnlySpan secondCentroids, + Span indices) + { + long distortion = 0; + for (int sampleIndex = 0; sampleIndex < firstSamples.Length; sampleIndex++) + { + int firstDifference = firstSamples[sampleIndex] - firstCentroids[0]; + int secondDifference = secondSamples[sampleIndex] - secondCentroids[0]; + int bestDistance = (firstDifference * firstDifference) + (secondDifference * secondDifference); + int bestIndex = 0; + for (int centroidIndex = 1; centroidIndex < firstCentroids.Length; centroidIndex++) + { + firstDifference = firstSamples[sampleIndex] - firstCentroids[centroidIndex]; + secondDifference = secondSamples[sampleIndex] - secondCentroids[centroidIndex]; + int distance = (firstDifference * firstDifference) + (secondDifference * secondDifference); + if (distance < bestDistance) + { + bestDistance = distance; + bestIndex = centroidIndex; + } + } + + indices[sampleIndex] = (byte)bestIndex; + distortion += bestDistance; + } + + return distortion; + } +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1PaletteKMeansTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1PaletteKMeansTests.cs new file mode 100644 index 0000000000..f23932752a --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1PaletteKMeansTests.cs @@ -0,0 +1,120 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline; +using SixLabors.ImageSharp.Tests.TestUtilities; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1; + +/// +/// Verifies AV1 palette clustering against independent scalar results at every intrinsic tier. +/// +[Trait("Format", "Avif")] +public class Av1PaletteKMeansTests +{ + /// + /// The hardware configurations covering every descending SIMD width and the scalar fallback. + /// + private const HwIntrinsics Configurations = + HwIntrinsics.AllowAll | HwIntrinsics.DisableAVX512F | HwIntrinsics.DisableAVX | HwIntrinsics.DisableHWIntrinsic; + + /// + /// Verifies nearest-color indices, squared distortion, tie order, and destination bounds. + /// + [Fact] + public void AssignIndicesMatchesScalarAtEveryIntrinsicTier() + => FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidateAssignment, Configurations); + + /// + /// Verifies the deterministic centroid sequence on three separated sample groups. + /// + [Fact] + public void ClusterMatchesReferenceFixture() + { + short[] samples = [0, 1, 2, 100, 101, 102, 200, 201, 202]; + short[] centroids = [33, 100, 167]; + byte[] indices = new byte[samples.Length]; + short[] alternateCentroids = new short[centroids.Length]; + byte[] alternateIndices = new byte[samples.Length]; + + long distortion = Av1PaletteKMeans.Cluster( + samples, + centroids, + indices, + alternateCentroids, + alternateIndices); + + Assert.Equal([1, 101, 201], centroids); + Assert.Equal([0, 0, 0, 1, 1, 1, 2, 2, 2], indices); + Assert.Equal(6, distortion); + } + + /// + /// Verifies the integer interval midpoints used to seed palette refinement. + /// + [Fact] + public void InitializeCentroidsMatchesReferenceIntegerOrder() + { + short[] centroids = new short[3]; + + Av1PaletteKMeans.InitializeCentroids(10, 250, centroids); + + Assert.Equal([50, 130, 210], centroids); + } + + /// + /// Compares production assignment with a scalar equation over a length that exercises every available remainder path. + /// + private static void ValidateAssignment() + { + const int SampleCount = 95; + short[] centroids = [0, 512, 1024, 2048, 3072, 4095]; + short[] samples = new short[SampleCount]; + for (int index = 0; index < samples.Length; index++) + { + samples[index] = (short)(((index * 977) + (index * index * 17)) & 4095); + } + + // This sample is equidistant from the first two colors and must retain the lower palette index. + samples[0] = 256; + byte[] expected = new byte[SampleCount]; + long expectedDistortion = AssignReference(samples, centroids, expected); + byte[] actual = Enumerable.Repeat(byte.MaxValue, SampleCount + 7).ToArray(); + + long actualDistortion = Av1PaletteKMeans.AssignIndices(samples, centroids, actual); + + Assert.Equal(expectedDistortion, actualDistortion); + Assert.Equal(expected, actual.AsSpan(..SampleCount).ToArray()); + Assert.All(actual[SampleCount..], value => Assert.Equal(byte.MaxValue, value)); + } + + /// + /// Applies the scalar nearest-color rule independently of the production SIMD traversal. + /// + private static long AssignReference( + ReadOnlySpan samples, + ReadOnlySpan centroids, + Span indices) + { + long distortion = 0; + for (int sampleIndex = 0; sampleIndex < samples.Length; sampleIndex++) + { + int bestDistance = Math.Abs(samples[sampleIndex] - centroids[0]); + int bestIndex = 0; + for (int centroidIndex = 1; centroidIndex < centroids.Length; centroidIndex++) + { + int distance = Math.Abs(samples[sampleIndex] - centroids[centroidIndex]); + if (distance < bestDistance) + { + bestDistance = distance; + bestIndex = centroidIndex; + } + } + + indices[sampleIndex] = (byte)bestIndex; + distortion += (long)bestDistance * bestDistance; + } + + return distortion; + } +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1PalettePredictorTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1PalettePredictorTests.cs new file mode 100644 index 0000000000..cc432f5fd2 --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1PalettePredictorTests.cs @@ -0,0 +1,146 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; +using SixLabors.ImageSharp.Memory; +using SixLabors.ImageSharp.Tests.TestUtilities; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1; + +/// +/// Verifies AV1 palette reconstruction across every supported sample precision and intrinsic tier. +/// +[Trait("Format", "Avif")] +public class Av1PalettePredictorTests +{ + /// + /// The hardware configurations required to exercise each packed width and the scalar fallback. + /// + private const HwIntrinsics Configurations = + HwIntrinsics.AllowAll | HwIntrinsics.DisableAVX512F | HwIntrinsics.DisableAVX | HwIntrinsics.DisableHWIntrinsic; + + /// + /// Verifies exact indexed reconstruction and destination-padding preservation for every palette size. + /// + [Fact] + public void PredictMatchesReference() + => FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidatePredictors, Configurations); + + /// + /// Exercises all palette sizes and transform widths at 8, 10, and 12 bits. + /// + private static void ValidatePredictors() + { + int[] widths = [4, 8, 16, 32, 64]; + foreach (int paletteSize in Enumerable.Range(2, Av1Constants.PaletteMaxSize - 1)) + { + foreach (int width in widths) + { + int height = width == 64 ? 16 : width; + int mapStride = width + 5; + int destinationStride = width + 9; + byte[] colorIndexMap = CreateColorIndexMap(mapStride, height, width, paletteSize); + using Buffer2D colorIndexMapBuffer = Configuration.Default.MemoryAllocator.Allocate2D(mapStride, height); + for (int row = 0; row < height; row++) + { + colorIndexMap.AsSpan(row * mapStride, mapStride).CopyTo(colorIndexMapBuffer.DangerousGetRowSpan(row)); + } + + Buffer2DRegion colorIndexMapRegion = new(colorIndexMapBuffer); + ushort[] bytePalette = CreatePalette(paletteSize, 8); + byte[] expectedBytes = Enumerable.Repeat((byte)251, destinationStride * height).ToArray(); + byte[] actualBytes = (byte[])expectedBytes.Clone(); + + ApplyReference(bytePalette, colorIndexMap, mapStride, expectedBytes, destinationStride, width, height); + Av1PalettePredictor.Predict(bytePalette, colorIndexMapRegion, actualBytes, destinationStride, width, height); + Assert.Equal(expectedBytes, actualBytes); + + foreach (int bitDepth in new[] { 10, 12 }) + { + ushort[] palette = CreatePalette(paletteSize, bitDepth); + short[] expected = Enumerable.Repeat((short)-1, destinationStride * height).ToArray(); + short[] actual = (short[])expected.Clone(); + + ApplyReference(palette, colorIndexMap, mapStride, expected, destinationStride, width, height); + Av1PalettePredictor.Predict(palette, colorIndexMapRegion, actual, destinationStride, width, height); + Assert.Equal(expected, actual); + } + } + } + } + + /// + /// Creates a deterministic palette spanning the legal range for the requested bit depth. + /// + private static ushort[] CreatePalette(int paletteSize, int bitDepth) + { + ushort[] result = new ushort[paletteSize]; + int maximum = (1 << bitDepth) - 1; + for (int index = 0; index < result.Length; index++) + { + result[index] = (ushort)(((index * 977) + 37) & maximum); + } + + return result; + } + + /// + /// Creates deterministic active indices and invalid padding indices for each map row. + /// + private static byte[] CreateColorIndexMap(int stride, int height, int width, int paletteSize) + { + byte[] result = Enumerable.Repeat((byte)Av1Constants.PaletteMaxSize, stride * height).ToArray(); + for (int row = 0; row < height; row++) + { + for (int column = 0; column < width; column++) + { + result[(row * stride) + column] = (byte)(((row * 5) + (column * 3)) % paletteSize); + } + } + + return result; + } + + /// + /// Applies independent scalar palette lookup to an 8-bit destination. + /// + private static void ApplyReference( + ReadOnlySpan palette, + ReadOnlySpan colorIndexMap, + int mapStride, + Span destination, + int destinationStride, + int width, + int height) + { + for (int row = 0; row < height; row++) + { + for (int column = 0; column < width; column++) + { + destination[(row * destinationStride) + column] = (byte)palette[colorIndexMap[(row * mapStride) + column]]; + } + } + } + + /// + /// Applies independent scalar palette lookup to a high-bit-depth destination. + /// + private static void ApplyReference( + ReadOnlySpan palette, + ReadOnlySpan colorIndexMap, + int mapStride, + Span destination, + int destinationStride, + int width, + int height) + { + for (int row = 0; row < height; row++) + { + for (int column = 0; column < width; column++) + { + destination[(row * destinationStride) + column] = (short)palette[colorIndexMap[(row * mapStride) + column]]; + } + } + } +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1PartitionTypeTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1PartitionTypeTests.cs new file mode 100644 index 0000000000..e7b073264e --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1PartitionTypeTests.cs @@ -0,0 +1,61 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1; + +[Trait("Format", "Avif")] +public class Av1PartitionTypeTests +{ + [Theory] + [MemberData(nameof(GetAllCombinations))] + internal void GetSubBlockSizeReturnsCorrectRatio(int t, int s) + { + // Assign + Av1PartitionType partitionType = (Av1PartitionType)t; + Av1BlockSize blockSize = (Av1BlockSize)s; + int expectedRatio = partitionType switch + { + Av1PartitionType.None or Av1PartitionType.Split => 1, + Av1PartitionType.HorizontalA or Av1PartitionType.HorizontalB or Av1PartitionType.Horizontal => 2, + Av1PartitionType.VerticalA or Av1PartitionType.VerticalB or Av1PartitionType.Vertical => -2, + Av1PartitionType.Horizontal4 => 4, + Av1PartitionType.Vertical4 => -4, + _ => -1 + }; + + // Act + Av1BlockSize subBlockSize = partitionType.GetBlockSubSize(blockSize); + + // Assert + if (subBlockSize != Av1BlockSize.Invalid) + { + int actualRatio = GetRatio(subBlockSize); + Assert.Equal(expectedRatio, actualRatio); + } + } + + public static TheoryData GetAllCombinations() + { + TheoryData combinations = []; + for (int t = 0; t <= (int)Av1PartitionType.Vertical4; t++) + { + for (int s = 0; s < (int)Av1BlockSize.AllSizes; s++) + { + combinations.Add(t, s); + } + } + + return combinations; + } + + private static int GetRatio(Av1BlockSize blockSize) + { + int width = blockSize.GetWidth(); + int height = blockSize.GetHeight(); + int ratio = width >= height ? width / height : -height / width; + return ratio; + } +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1PredictorTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1PredictorTests.cs new file mode 100644 index 0000000000..222fdde671 --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1PredictorTests.cs @@ -0,0 +1,898 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Numerics; +using SixLabors.ImageSharp.Formats.Heif.Av1; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; +using SixLabors.ImageSharp.Tests.TestUtilities; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1; + +/// +/// Verifies AV1 intra prediction against scalar definitions across the supported hardware-intrinsic configurations. +/// +[Trait("Format", "Heif")] +public class Av1PredictorTests +{ + /// + /// The offset within directional reference storage that leaves readable samples before both edge origins. + /// + private const int ReferenceOrigin = 128; + + /// + /// The hardware configurations required to exercise each SIMD tier and the complete scalar fallback. + /// + private const HwIntrinsics PredictorConfigurations = + HwIntrinsics.AllowAll | HwIntrinsics.DisableAVX512F | HwIntrinsics.DisableAVX | HwIntrinsics.DisableHWIntrinsic; + + /// + /// Gets the cardinal, base, and adjusted angles covering every directional projection zone. + /// + private static ReadOnlySpan DirectionalAngles => [36, 45, 54, 67, 90, 104, 113, 126, 135, 148, 157, 166, 180, 194, 203, 212]; + + /// + /// Gets the complete set of AV1 filter-intra coefficient modes. + /// + private static ReadOnlySpan FilterIntraModes => + [ + Av1FilterIntraMode.DC, + Av1FilterIntraMode.Vertical, + Av1FilterIntraMode.Horizontal, + Av1FilterIntraMode.Directional157, + Av1FilterIntraMode.Paeth, + ]; + + /// + /// Verifies DC prediction with each register-width tier and the scalar fallback. + /// + [Fact] + public void DcPredictorsMatchReference() + => FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidateDcPredictors, PredictorConfigurations); + + /// + /// Verifies horizontal prediction with each register-width tier and the scalar fallback. + /// + [Fact] + public void HorizontalPredictorMatchesReference() + => FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidateHorizontalPredictor, PredictorConfigurations); + + /// + /// Verifies vertical prediction with each register-width tier and the scalar fallback. + /// + [Fact] + public void VerticalPredictorMatchesReference() + => FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidateVerticalPredictor, PredictorConfigurations); + + /// + /// Verifies Paeth prediction with each register-width tier and the scalar fallback. + /// + [Fact] + public void PaethPredictorMatchesReference() + => FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidatePaethPredictor, PredictorConfigurations); + + /// + /// Verifies smooth prediction with each register-width tier and the scalar fallback. + /// + [Fact] + public void SmoothPredictorMatchesReference() + => FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidateSmoothPredictor, PredictorConfigurations); + + /// + /// Verifies horizontal smooth prediction with each register-width tier and the scalar fallback. + /// + [Fact] + public void SmoothHorizontalPredictorMatchesReference() + => FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidateSmoothHorizontalPredictor, PredictorConfigurations); + + /// + /// Verifies vertical smooth prediction with each register-width tier and the scalar fallback. + /// + [Fact] + public void SmoothVerticalPredictorMatchesReference() + => FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidateSmoothVerticalPredictor, PredictorConfigurations); + + /// + /// Verifies directional prediction with each register-width tier and the scalar fallback. + /// + [Fact] + public void DirectionalPredictorsMatchReference() + => FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidateDirectionalPredictors, PredictorConfigurations); + + /// + /// Verifies filter-intra prediction with each register-width tier and the scalar fallback. + /// + [Fact] + public void FilterIntraPredictorsMatchReference() + => FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidateFilterIntraPredictors, PredictorConfigurations); + + /// + /// Verifies intra-edge upsampling with each register-width tier and the scalar fallback. + /// + [Fact] + public void EdgeUpsamplingMatchesReference() + => FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidateEdgeUpsampling, PredictorConfigurations); + + /// + /// Verifies intra-edge filtering with each register-width tier and the scalar fallback. + /// + [Fact] + public void EdgeFilteringMatchesReference() + => FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidateEdgeFiltering, PredictorConfigurations); + + /// + /// Verifies the different edge preparation selected by a smooth neighbor on a 4x8 directional block. + /// + [Theory] + [InlineData(false, false)] + [InlineData(false, true)] + [InlineData(true, false)] + [InlineData(true, true)] + public void EdgePreparationUsesSmoothNeighborThresholds(bool transpose, bool smoothNeighbor) + { + const int Count = 12; + byte[] edge = CreateUpsampleByteEdge(Count); + byte[] expected = (byte[])edge.Clone(); + byte[] unusedEdge = CreateUpsampleByteEdge(Count); + byte[] expectedUnused = (byte[])unusedEdge.Clone(); + byte[] scratch = new byte[Av1IntraEdgeFilter.ScratchLength]; + + // At 23 degrees from the cardinal direction and width + height = 12, an ordinary neighbor + // selects unfiltered half samples. A smooth neighbor selects strength one without upsampling. + // The expected samples use the independent scalar kernels, never the production selector. + if (smoothNeighbor) + { + byte[] source = edge.AsSpan(1, Count + 1).ToArray(); + byte[] filtered = (byte[])source.Clone(); + FilterEdgeScalar(source, filtered, 1); + filtered.CopyTo(expected, 1); + } + else + { + UpsampleEdgeScalar(expected, Count, 8); + } + + Av1IntraEdgePreparation.Prepare( + (transpose ? unusedEdge : edge).AsSpan(2), + (transpose ? edge : unusedEdge).AsSpan(2), + transpose ? 8 : 4, + transpose ? 4 : 8, + transpose ? 203 : 67, + transpose ? 8 : 4, + transpose ? 4 : 8, + smoothNeighbor, + 8, + scratch, + out bool upsampleAbove, + out bool upsampleLeft); + + Assert.Equal(!transpose && !smoothNeighbor, upsampleAbove); + Assert.Equal(transpose && !smoothNeighbor, upsampleLeft); + Assert.Equal(expected, edge); + Assert.Equal(expectedUnused, unusedEdge); + } + + /// + /// Verifies that an unavailable sole directional edge retains its constant prediction and distinct corner. + /// + [Theory] + [InlineData(false, false)] + [InlineData(false, true)] + [InlineData(true, false)] + [InlineData(true, true)] + public void EdgePreparationPreservesUnavailableSoleEdge(bool transpose, bool perpendicularAvailable) + { + byte[] edge = CreateUpsampleByteEdge(12); + byte[] perpendicular = CreateUpsampleByteEdge(12); + edge.AsSpan(2, 12).Fill(perpendicularAvailable ? perpendicular[2] : transpose ? (byte)129 : (byte)127); + byte[] expected = (byte[])edge.Clone(); + byte[] expectedPerpendicular = (byte[])perpendicular.Clone(); + byte[] scratch = new byte[Av1IntraEdgeFilter.ScratchLength]; + + // These angles normally enable half-sample interpolation. Native prediction exits before that + // stage when its sole edge is unavailable; the corner must not introduce a nonconstant sample. + Av1IntraEdgePreparation.Prepare( + (transpose ? perpendicular : edge).AsSpan(2), + (transpose ? edge : perpendicular).AsSpan(2), + transpose ? 8 : 4, + transpose ? 4 : 8, + transpose ? 203 : 67, + transpose && perpendicularAvailable ? 8 : 0, + !transpose && perpendicularAvailable ? 8 : 0, + false, + 8, + scratch, + out bool upsampleAbove, + out bool upsampleLeft); + + Assert.False(upsampleAbove); + Assert.False(upsampleLeft); + Assert.Equal(expected, edge); + Assert.Equal(expectedPerpendicular, perpendicular); + } + + /// + /// Verifies the traversal-order bits that distinguish current libaom's mixed-vertical square tables. + /// + [Fact] + public void MixedVerticalAvailabilityUsesDedicatedSquareTables() + { + Assert.True(Av1BottomRightTopLeftConstants.HasTopRight(Av1PartitionType.Split, Av1BlockSize.Block8x8, 16)); + Assert.False(Av1BottomRightTopLeftConstants.HasTopRight(Av1PartitionType.VerticalA, Av1BlockSize.Block8x8, 16)); + Assert.False(Av1BottomRightTopLeftConstants.HasTopRight(Av1PartitionType.VerticalB, Av1BlockSize.Block8x8, 16)); + + Assert.False(Av1BottomRightTopLeftConstants.HasBottomLeft(Av1PartitionType.Split, Av1BlockSize.Block8x8, 1)); + Assert.True(Av1BottomRightTopLeftConstants.HasBottomLeft(Av1PartitionType.VerticalA, Av1BlockSize.Block8x8, 1)); + Assert.True(Av1BottomRightTopLeftConstants.HasBottomLeft(Av1PartitionType.VerticalB, Av1BlockSize.Block8x8, 1)); + } + + /// + /// Verifies that mixed-vertical rectangles use current libaom's ordinary rectangle tables. + /// + [Theory] + [InlineData((int)Av1BlockSize.Block4x8)] + [InlineData((int)Av1BlockSize.Block8x16)] + [InlineData((int)Av1BlockSize.Block16x32)] + [InlineData((int)Av1BlockSize.Block32x64)] + [InlineData((int)Av1BlockSize.Block64x128)] + public void MixedVerticalAvailabilityReusesVerticalRectangleTables(int blockSizeValue) + { + Av1BlockSize blockSize = (Av1BlockSize)blockSizeValue; + bool expectedTopRight = Av1BottomRightTopLeftConstants.HasTopRight(Av1PartitionType.Split, blockSize, 0); + bool expectedBottomLeft = Av1BottomRightTopLeftConstants.HasBottomLeft(Av1PartitionType.Split, blockSize, 0); + + Assert.Equal(expectedTopRight, Av1BottomRightTopLeftConstants.HasTopRight(Av1PartitionType.VerticalA, blockSize, 0)); + Assert.Equal(expectedTopRight, Av1BottomRightTopLeftConstants.HasTopRight(Av1PartitionType.VerticalB, blockSize, 0)); + Assert.Equal(expectedBottomLeft, Av1BottomRightTopLeftConstants.HasBottomLeft(Av1PartitionType.VerticalA, blockSize, 0)); + Assert.Equal(expectedBottomLeft, Av1BottomRightTopLeftConstants.HasBottomLeft(Av1PartitionType.VerticalB, blockSize, 0)); + } + + /// + /// Verifies all four DC neighbor-availability combinations at every AV1 transform size. + /// + private static void ValidateDcPredictors() + { + for (int sizeIndex = 0; sizeIndex < (int)Av1TransformSize.AllSizes; sizeIndex++) + { + Av1TransformSize transformSize = (Av1TransformSize)sizeIndex; + int width = transformSize.GetWidth(); + int height = transformSize.GetHeight(); + int stride = width + 5; + byte[] above = CreateByteSamples(width, 17); + byte[] left = CreateByteSamples(height, 43); + short[] aboveHigh = CreateHighBitDepthSamples(width, 17); + short[] leftHigh = CreateHighBitDepthSamples(height, 43); + + for (int availability = 0; availability < 4; availability++) + { + bool hasLeft = (availability & 1) != 0; + bool hasAbove = (availability & 2) != 0; + byte[] expected = CreateByteDestination(stride, height); + byte[] actual = CreateByteDestination(stride, height); + short[] expectedHigh = CreateHighBitDepthDestination(stride, height); + short[] actualHigh = CreateHighBitDepthDestination(stride, height); + + Av1DcIntraPredictor.PredictScalar(hasLeft, hasAbove, expected, stride, above, left, width, height); + Av1DcIntraPredictor.Predict(hasLeft, hasAbove, actual, stride, above, left, width, height); + Av1DcIntraPredictor.PredictScalar(hasLeft, hasAbove, expectedHigh, stride, aboveHigh, leftHigh, width, height, 12); + Av1DcIntraPredictor.Predict(hasLeft, hasAbove, actualHigh, stride, aboveHigh, leftHigh, width, height, 12); + + Assert.Equal(expected, actual); + Assert.Equal(expectedHigh, actualHigh); + } + } + } + + /// + /// Verifies horizontal prediction at every AV1 transform size and sample precision. + /// + private static void ValidateHorizontalPredictor() => ValidateNonDirectionalPredictor(Av1PredictionMode.Horizontal); + + /// + /// Verifies vertical prediction at every AV1 transform size and sample precision. + /// + private static void ValidateVerticalPredictor() => ValidateNonDirectionalPredictor(Av1PredictionMode.Vertical); + + /// + /// Verifies Paeth prediction at every AV1 transform size and sample precision. + /// + private static void ValidatePaethPredictor() => ValidateNonDirectionalPredictor(Av1PredictionMode.Paeth); + + /// + /// Verifies smooth prediction at every AV1 transform size and sample precision. + /// + private static void ValidateSmoothPredictor() => ValidateNonDirectionalPredictor(Av1PredictionMode.Smooth); + + /// + /// Verifies horizontal smooth prediction at every AV1 transform size and sample precision. + /// + private static void ValidateSmoothHorizontalPredictor() => ValidateNonDirectionalPredictor(Av1PredictionMode.SmoothHorizontal); + + /// + /// Verifies vertical smooth prediction at every AV1 transform size and sample precision. + /// + private static void ValidateSmoothVerticalPredictor() => ValidateNonDirectionalPredictor(Av1PredictionMode.SmoothVertical); + + /// + /// Verifies one closed non-directional operator at every AV1 transform size and sample precision. + /// + /// The prediction mode to verify. + private static void ValidateNonDirectionalPredictor(Av1PredictionMode mode) + { + Av1NonDirectionalIntraPredictorBase predictor = Av1NonDirectionalIntraPredictorBase.GetPredictor(mode); + for (int sizeIndex = 0; sizeIndex < (int)Av1TransformSize.AllSizes; sizeIndex++) + { + Av1TransformSize transformSize = (Av1TransformSize)sizeIndex; + int width = transformSize.GetWidth(); + int height = transformSize.GetHeight(); + int stride = width + 5; + byte[] aboveStorage = CreateByteSamples(width + 1, 19); + byte[] left = CreateByteSamples(height, 71); + short[] aboveHighStorage = CreateHighBitDepthSamples(width + 1, 19); + short[] leftHigh = CreateHighBitDepthSamples(height, 71); + byte[] expected = CreateByteDestination(stride, height); + byte[] actual = CreateByteDestination(stride, height); + short[] expectedHigh = CreateHighBitDepthDestination(stride, height); + short[] actualHigh = CreateHighBitDepthDestination(stride, height); + + predictor.PredictScalar(expected, stride, aboveStorage.AsSpan(1), left, width, height); + predictor.Predict(actual, stride, aboveStorage.AsSpan(1), left, width, height); + predictor.PredictScalar(expectedHigh, stride, aboveHighStorage.AsSpan(1), leftHigh, width, height); + predictor.Predict(actualHigh, stride, aboveHighStorage.AsSpan(1), leftHigh, width, height); + + Assert.Equal(expected, actual); + Assert.Equal(expectedHigh, actualHigh); + } + + ValidateKnownNonDirectionalVector(mode, predictor); + } + + /// + /// Verifies one non-directional operator against a byte-exact reference block and its translated high-bit-depth equivalent. + /// + /// The prediction mode being verified. + /// The closed operator-driven predictor. + private static void ValidateKnownNonDirectionalVector(Av1PredictionMode mode, Av1NonDirectionalIntraPredictorBase predictor) + { + byte[] aboveStorage; + byte[] left; + byte[] expected; + + if (mode == Av1PredictionMode.Paeth) + { + aboveStorage = [50, 60, 10, 90, 40]; + left = [20, 80, 30, 100]; + expected = + [ + 20, 10, 50, 20, + 80, 50, 90, 80, + 30, 10, 90, 30, + 100, 50, 100, 100, + ]; + } + else + { + aboveStorage = [0, 20, 40, 60, 80]; + left = [20, 40, 60, 80]; + expected = mode switch + { + Av1PredictionMode.Horizontal => + [ + 20, 20, 20, 20, + 40, 40, 40, 40, + 60, 60, 60, 60, + 80, 80, 80, 80, + ], + Av1PredictionMode.Vertical => + [ + 20, 40, 60, 80, + 20, 40, 60, 80, + 20, 40, 60, 80, + 20, 40, 60, 80, + ], + Av1PredictionMode.Smooth => + [ + 20, 43, 60, 73, + 43, 57, 68, 75, + 60, 68, 73, 78, + 73, 75, 78, 80, + ], + Av1PredictionMode.SmoothHorizontal => + [ + 20, 45, 60, 65, + 40, 57, 67, 70, + 60, 68, 73, 75, + 80, 80, 80, 80, + ], + _ => + [ + 20, 40, 60, 80, + 45, 57, 68, 80, + 60, 67, 73, 80, + 65, 70, 75, 80, + ], + }; + } + + byte[] actual = new byte[16]; + predictor.Predict(actual, 4, aboveStorage.AsSpan(1), left, 4, 4); + + Assert.Equal(expected, actual); + + const int offset = 512; + short[] aboveHighStorage = new short[aboveStorage.Length]; + short[] leftHigh = new short[left.Length]; + short[] expectedHigh = new short[expected.Length]; + short[] actualHigh = new short[16]; + + for (int i = 0; i < aboveStorage.Length; i++) + { + aboveHighStorage[i] = (short)(aboveStorage[i] + offset); + } + + for (int i = 0; i < left.Length; i++) + { + leftHigh[i] = (short)(left[i] + offset); + } + + for (int i = 0; i < expected.Length; i++) + { + expectedHigh[i] = (short)(expected[i] + offset); + } + + predictor.Predict(actualHigh, 4, aboveHighStorage.AsSpan(1), leftHigh, 4, 4); + + Assert.Equal(expectedHigh, actualHigh); + } + + /// + /// Verifies every directional zone, rectangular transpose, and edge-upsampling index rule. + /// + private static void ValidateDirectionalPredictors() + { + byte[] aboveStorage = CreateByteSamples(512, 23); + byte[] leftStorage = CreateByteSamples(512, 89); + short[] aboveHighStorage = CreateHighBitDepthSamples(512, 23); + short[] leftHighStorage = CreateHighBitDepthSamples(512, 89); + ReadOnlySpan above = aboveStorage.AsSpan(ReferenceOrigin); + ReadOnlySpan left = leftStorage.AsSpan(ReferenceOrigin); + ReadOnlySpan aboveHigh = aboveHighStorage.AsSpan(ReferenceOrigin); + ReadOnlySpan leftHigh = leftHighStorage.AsSpan(ReferenceOrigin); + + foreach (int angle in DirectionalAngles) + { + for (int sizeIndex = 0; sizeIndex < (int)Av1TransformSize.AllSizes; sizeIndex++) + { + ValidateDirectionalCase((Av1TransformSize)sizeIndex, angle, false, false, above, left, aboveHigh, leftHigh); + } + } + + // Edge upsampling is permitted only for small blocks. These cases exercise top-only, both-edge, + // and left-only indexing without asking an invalid large transform to consume an upsampled edge. + ValidateDirectionalCase(Av1TransformSize.Size4x4, 45, true, false, above, left, aboveHigh, leftHigh); + ValidateDirectionalCase(Av1TransformSize.Size4x4, 135, true, true, above, left, aboveHigh, leftHigh); + ValidateDirectionalCase(Av1TransformSize.Size4x4, 203, false, true, above, left, aboveHigh, leftHigh); + ValidateKnownDirectionalVectors(); + } + + /// + /// Verifies all three projection zones against byte-exact reference blocks. + /// + private static void ValidateKnownDirectionalVectors() + { + ValidateKnownDirectionalVector( + 45, + [0, 10, 20, 30, 40, 50, 60, 70, 80, 90], + [0, 0, 0, 0, 0, 0, 0, 0, 0, 0], + [ + 20, 30, 40, 50, + 30, 40, 50, 60, + 40, 50, 60, 70, + 50, 60, 70, 80, + ]); + + ValidateKnownDirectionalVector( + 135, + [5, 10, 20, 30, 40, 50, 60, 70, 80, 90], + [5, 50, 60, 70, 80, 90, 100, 110, 120, 130], + [ + 5, 10, 20, 30, + 50, 5, 10, 20, + 60, 50, 5, 10, + 70, 60, 50, 5, + ]); + + ValidateKnownDirectionalVector( + 203, + [0, 0, 0, 0, 0, 0, 0, 0, 0, 0], + [0, 10, 20, 30, 40, 50, 60, 70, 80, 90], + [ + 14, 18, 23, 27, + 24, 28, 33, 37, + 34, 38, 43, 47, + 44, 48, 53, 57, + ]); + } + + /// + /// Verifies one directional projection and its translated high-bit-depth equivalent. + /// + /// The adjusted directional angle. + /// The top-left prefix followed by the top reference. + /// The top-left prefix followed by the left reference. + /// The byte-exact predicted block. + private static void ValidateKnownDirectionalVector( + int angle, + ReadOnlySpan aboveStorage, + ReadOnlySpan leftStorage, + ReadOnlySpan expected) + { + ReadOnlySpan above = aboveStorage[1..]; + ReadOnlySpan left = leftStorage[1..]; + + byte[] actual = new byte[16]; + byte[] scratch = new byte[Av1DirectionalIntraPredictor.ScratchLength]; + Av1DirectionalIntraPredictor.Predict(actual, 4, Av1TransformSize.Size4x4, above, left, false, false, angle, scratch); + + Assert.Equal(expected, actual); + + const int offset = 512; + short[] aboveHighStorage = new short[aboveStorage.Length]; + short[] leftHighStorage = new short[leftStorage.Length]; + short[] expectedHigh = new short[expected.Length]; + short[] actualHigh = new short[16]; + short[] scratchHigh = new short[Av1DirectionalIntraPredictor.ScratchLength]; + + for (int i = 0; i < aboveStorage.Length; i++) + { + aboveHighStorage[i] = (short)(aboveStorage[i] + offset); + } + + for (int i = 0; i < leftStorage.Length; i++) + { + leftHighStorage[i] = (short)(leftStorage[i] + offset); + } + + for (int i = 0; i < expected.Length; i++) + { + expectedHigh[i] = (short)(expected[i] + offset); + } + + Av1DirectionalIntraPredictor.Predict( + actualHigh, + 4, + Av1TransformSize.Size4x4, + aboveHighStorage.AsSpan(1), + leftHighStorage.AsSpan(1), + false, + false, + angle, + scratchHigh); + + Assert.Equal(expectedHigh, actualHigh); + } + + /// + /// Verifies one directional prediction configuration for both native sample representations. + /// + private static void ValidateDirectionalCase( + Av1TransformSize transformSize, + int angle, + bool upsampleAbove, + bool upsampleLeft, + ReadOnlySpan above, + ReadOnlySpan left, + ReadOnlySpan aboveHigh, + ReadOnlySpan leftHigh) + { + int width = transformSize.GetWidth(); + int height = transformSize.GetHeight(); + int stride = width + 5; + byte[] expected = CreateByteDestination(stride, height); + byte[] actual = CreateByteDestination(stride, height); + short[] expectedHigh = CreateHighBitDepthDestination(stride, height); + short[] actualHigh = CreateHighBitDepthDestination(stride, height); + byte[] scratch = new byte[Av1DirectionalIntraPredictor.ScratchLength]; + short[] scratchHigh = new short[Av1DirectionalIntraPredictor.ScratchLength]; + + Av1DirectionalIntraPredictor.PredictScalar(expected, stride, transformSize, above, left, upsampleAbove, upsampleLeft, angle); + Av1DirectionalIntraPredictor.Predict(actual, stride, transformSize, above, left, upsampleAbove, upsampleLeft, angle, scratch); + Av1DirectionalIntraPredictor.PredictScalar(expectedHigh, stride, transformSize, aboveHigh, leftHigh, upsampleAbove, upsampleLeft, angle); + Av1DirectionalIntraPredictor.Predict(actualHigh, stride, transformSize, aboveHigh, leftHigh, upsampleAbove, upsampleLeft, angle, scratchHigh); + + Assert.Equal(expected, actual); + Assert.Equal(expectedHigh, actualHigh); + } + + /// + /// Verifies every filter-intra operator at each transform size permitted by the AV1 syntax. + /// + private static void ValidateFilterIntraPredictors() + { + foreach (Av1FilterIntraMode mode in FilterIntraModes) + { + Av1FilterIntraPredictorBase predictor = Av1FilterIntraPredictorBase.GetPredictor(mode); + for (int sizeIndex = 0; sizeIndex < (int)Av1TransformSize.AllSizes; sizeIndex++) + { + Av1TransformSize transformSize = (Av1TransformSize)sizeIndex; + int width = transformSize.GetWidth(); + int height = transformSize.GetHeight(); + if (width > 32 || height > 32) + { + continue; + } + + int stride = width + 5; + byte[] aboveStorage = CreateByteSamples(width + 1, 29); + byte[] left = CreateByteSamples(height, 97); + short[] aboveHighStorage = CreateHighBitDepthSamples(width + 1, 29); + short[] leftHigh = CreateHighBitDepthSamples(height, 97); + byte[] expected = CreateByteDestination(stride, height); + byte[] actual = CreateByteDestination(stride, height); + short[] expectedHigh = CreateHighBitDepthDestination(stride, height); + short[] actualHigh = CreateHighBitDepthDestination(stride, height); + byte[] expectedScratch = new byte[Av1FilterIntraPredictorBase.ScratchLength]; + byte[] actualScratch = new byte[Av1FilterIntraPredictorBase.ScratchLength]; + short[] expectedHighScratch = new short[Av1FilterIntraPredictorBase.ScratchLength]; + short[] actualHighScratch = new short[Av1FilterIntraPredictorBase.ScratchLength]; + + predictor.PredictScalar(expected, stride, aboveStorage.AsSpan(1), left, width, height, expectedScratch); + predictor.Predict(actual, stride, aboveStorage.AsSpan(1), left, width, height, actualScratch); + predictor.PredictScalar(expectedHigh, stride, aboveHighStorage.AsSpan(1), leftHigh, width, height, 12, expectedHighScratch); + predictor.Predict(actualHigh, stride, aboveHighStorage.AsSpan(1), leftHigh, width, height, 12, actualHighScratch); + + Assert.Equal(expected, actual); + Assert.Equal(expectedHigh, actualHigh); + } + } + + ValidateKnownFilterIntraVectors(); + } + + /// + /// Retains byte-exact reference vectors so scalar and SIMD code cannot share the same mistranslation unnoticed. + /// + private static void ValidateKnownFilterIntraVectors() + { + byte[][] expectedByMode = + [ + [42, 65, 89, 123, 72, 77, 91, 110, 105, 100, 104, 112, 142, 128, 123, 124], + [44, 79, 116, 153, 69, 94, 126, 158, 94, 109, 136, 163, 119, 124, 146, 168], + [47, 67, 87, 107, 83, 93, 103, 113, 122, 127, 132, 137, 161, 163, 166, 168], + [38, 55, 81, 111, 64, 62, 73, 92, 97, 83, 81, 86, 134, 113, 103, 100], + [49, 81, 114, 148, 82, 105, 132, 159, 117, 132, 153, 174, 152, 159, 177, 190], + ]; + + // These edge values are the input to the five reference vectors above. The leading top value is the + // shared top-left sample addressed through above[-1] by the normative recursive filter process. + byte[] aboveStorage = [17, 30, 70, 110, 150]; + byte[] left = [40, 80, 120, 160]; + short[] aboveHighStorage = [529, 542, 582, 622, 662]; + short[] leftHigh = [552, 592, 632, 672]; + + for (int modeIndex = 0; modeIndex < FilterIntraModes.Length; modeIndex++) + { + byte[] actual = new byte[16]; + byte[] scratch = new byte[Av1FilterIntraPredictorBase.ScratchLength]; + short[] actualHigh = new short[16]; + short[] expectedHigh = new short[16]; + short[] scratchHigh = new short[Av1FilterIntraPredictorBase.ScratchLength]; + Av1FilterIntraPredictorBase predictor = Av1FilterIntraPredictorBase.GetPredictor(FilterIntraModes[modeIndex]); + + predictor.Predict(actual, 4, aboveStorage.AsSpan(1), left, 4, 4, scratch); + + for (int i = 0; i < expectedHigh.Length; i++) + { + expectedHigh[i] = (short)(expectedByMode[modeIndex][i] + 512); + } + + predictor.Predict(actualHigh, 4, aboveHighStorage.AsSpan(1), leftHigh, 4, 4, 10, scratchHigh); + + Assert.Equal(expectedByMode[modeIndex], actual); + Assert.Equal(expectedHigh, actualHigh); + } + } + + /// + /// Verifies vector interleaving, endpoint extension, clamping, and scalar tails in edge upsampling. + /// + private static void ValidateEdgeUpsampling() + { + ReadOnlySpan counts = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16]; + foreach (int count in counts) + { + byte[] actual = CreateUpsampleByteEdge(count); + byte[] expected = (byte[])actual.Clone(); + byte[] scratch = new byte[Av1IntraEdgeUpsampler.ScratchLength]; + + UpsampleEdgeScalar(expected, count, 8); + Av1IntraEdgeUpsampler.Apply(actual.AsSpan(2), count, scratch); + + Assert.Equal(expected, actual); + + ReadOnlySpan bitDepths = [10, 12]; + + foreach (int bitDepth in bitDepths) + { + short[] actualHigh = CreateUpsampleHighBitDepthEdge(count, bitDepth); + short[] expectedHigh = (short[])actualHigh.Clone(); + short[] scratchHigh = new short[Av1IntraEdgeUpsampler.ScratchLength]; + + UpsampleEdgeScalar(expectedHigh, count, bitDepth); + Av1IntraEdgeUpsampler.Apply(actualHigh.AsSpan(2), count, bitDepth, scratchHigh); + + Assert.Equal(expectedHigh, actualHigh); + } + } + } + + /// + /// Verifies all three edge-filter kernels across vector boundaries and the maximum normative edge length. + /// + private static void ValidateEdgeFiltering() + { + ReadOnlySpan counts = [4, 8, 9, 16, 17, 31, 32, 33, 64, 65, 129]; + foreach (int count in counts) + { + for (int strength = 1; strength <= 3; strength++) + { + byte[] actual = CreateByteSamples(count, 31); + byte[] expected = (byte[])actual.Clone(); + byte[] source = (byte[])actual.Clone(); + byte[] scratch = new byte[Av1IntraEdgeFilter.ScratchLength]; + + FilterEdgeScalar(source, expected, strength); + Av1IntraEdgeFilter.Apply(ref actual[0], count, strength, scratch); + + Assert.Equal(expected, actual); + + short[] actualHigh = CreateHighBitDepthSamples(count, 31); + short[] expectedHigh = (short[])actualHigh.Clone(); + short[] sourceHigh = (short[])actualHigh.Clone(); + short[] scratchHigh = new short[Av1IntraEdgeFilter.ScratchLength]; + + FilterEdgeScalar(sourceHigh, expectedHigh, strength); + Av1IntraEdgeFilter.Apply(ref actualHigh[0], count, strength, scratchHigh); + + Assert.Equal(expectedHigh, actualHigh); + } + } + } + + /// + /// Creates deterministic 8-bit samples with enough variation to expose lane-order mistakes. + /// + private static byte[] CreateByteSamples(int length, int seed) + { + byte[] samples = new byte[length]; + for (int i = 0; i < samples.Length; i++) + { + samples[i] = (byte)(((i * 73) + (seed * 29) + ((i * i) * 7)) & 255); + } + + return samples; + } + + /// + /// Creates deterministic 12-bit samples with values spanning the full reconstructed range. + /// + private static short[] CreateHighBitDepthSamples(int length, int seed) + { + short[] samples = new short[length]; + for (int i = 0; i < samples.Length; i++) + { + samples[i] = (short)(((i * 977) + (seed * 131) + ((i * i) * 37)) & 4095); + } + + return samples; + } + + /// + /// Creates a strided byte destination initialized with a padding sentinel. + /// + private static byte[] CreateByteDestination(int stride, int height) + { + byte[] destination = new byte[stride * height]; + Array.Fill(destination, (byte)0xCD); + return destination; + } + + /// + /// Creates a strided high-bit-depth destination initialized with a padding sentinel. + /// + private static short[] CreateHighBitDepthDestination(int stride, int height) + { + short[] destination = new short[stride * height]; + Array.Fill(destination, (short)-1234); + return destination; + } + + /// + /// Creates an 8-bit edge with two prefix samples and room for all interleaved outputs. + /// + private static byte[] CreateUpsampleByteEdge(int count) + { + byte[] edge = new byte[(2 * count) + 4]; + Array.Fill(edge, (byte)0xA5); + edge[1] = 231; + for (int i = 0; i < count; i++) + { + edge[i + 2] = (byte)(((i * 97) + 41) & 255); + } + + return edge; + } + + /// + /// Creates a high-bit-depth edge containing extrema that exercise interpolation clamping. + /// + private static short[] CreateUpsampleHighBitDepthEdge(int count, int bitDepth) + { + int maximum = (1 << bitDepth) - 1; + short[] edge = new short[(2 * count) + 4]; + Array.Fill(edge, (short)-1); + edge[1] = (short)maximum; + for (int i = 0; i < count; i++) + { + edge[i + 2] = (short)((i & 1) == 0 ? 0 : maximum); + } + + return edge; + } + + /// + /// Applies the normative four-tap upsampling formula to an edge stored at index two. + /// + private static void UpsampleEdgeScalar(T[] edge, int count, int bitDepth) + where T : unmanaged, IBinaryInteger + { + T[] input = new T[count + 3]; + input[0] = edge[1]; + input[1] = edge[1]; + for (int i = 0; i < count; i++) + { + input[i + 2] = edge[i + 2]; + } + + input[count + 2] = input[count + 1]; + edge[0] = input[0]; + int maximum = (1 << bitDepth) - 1; + for (int i = 0; i < count; i++) + { + int value = -int.CreateChecked(input[i]) + + (9 * int.CreateChecked(input[i + 1])) + + (9 * int.CreateChecked(input[i + 2])) + - int.CreateChecked(input[i + 3]); + + edge[(2 * i) + 1] = T.CreateChecked(Math.Clamp((value + 8) >> 4, 0, maximum)); + edge[(2 * i) + 2] = input[i + 2]; + } + } + + /// + /// Applies the normative AV1 edge-filter definition to an independent source copy. + /// + private static void FilterEdgeScalar(T[] source, T[] destination, int strength) + where T : unmanaged, IBinaryInteger + { + ReadOnlySpan kernel = strength switch + { + 1 => [0, 4, 8, 4, 0], + 2 => [0, 5, 6, 5, 0], + _ => [2, 4, 4, 4, 2], + }; + + for (int i = 1; i < source.Length; i++) + { + int sum = 0; + for (int tap = 0; tap < kernel.Length; tap++) + { + int sourceIndex = Math.Clamp(i - 2 + tap, 0, source.Length - 1); + sum += int.CreateChecked(source[sourceIndex]) * kernel[tap]; + } + + destination[i] = T.CreateChecked((sum + 8) >> 4); + } + } +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReconstructionConformanceTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReconstructionConformanceTests.cs new file mode 100644 index 0000000000..52fbcdda2a --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReconstructionConformanceTests.cs @@ -0,0 +1,4285 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Buffers.Binary; +using System.Globalization; +using System.Text; +using SixLabors.ImageSharp.Formats; +using SixLabors.ImageSharp.Formats.Heif; +using SixLabors.ImageSharp.Formats.Heif.Av1; +using SixLabors.ImageSharp.Formats.Heif.Av1.Motion; +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; +using SixLabors.ImageSharp.Formats.Png; +using SixLabors.ImageSharp.Memory; +using SixLabors.ImageSharp.Metadata.Profiles.Cicp; +using SixLabors.ImageSharp.PixelFormats; +using SixLabors.ImageSharp.Tests.Memory; +using SixLabors.ImageSharp.Tests.TestUtilities; +using SixLabors.ImageSharp.Tests.TestUtilities.ImageComparison; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1; + +/// +/// Validates complete AV1 reconstruction against independently decoded native component planes. +/// +[Trait("Format", "Avif")] +public class Av1ReconstructionConformanceTests +{ + /// + /// The width and height of one CDEF unit in 4x4 luma mode-information units. + /// + private const int CdefUnitModeInfoSize = 16; + + /// + /// The hardware configurations covering normal SIMD dispatch and the scalar fallback. + /// + private const HwIntrinsics ReconstructionConfigurations = HwIntrinsics.AllowAll | HwIntrinsics.DisableHWIntrinsic; + + /// + /// The hardware configurations covering the 256-bit, 128-bit, and scalar palette-reconstruction paths. + /// + private const HwIntrinsics PaletteConfigurations = + HwIntrinsics.AllowAll | HwIntrinsics.DisableAVX | HwIntrinsics.DisableHWIntrinsic; + + /// + /// The hardware configurations covering normal dispatch, narrower vector fallbacks, and scalar intra-block copy. + /// + private const HwIntrinsics IntraBlockCopyConfigurations = + HwIntrinsics.AllowAll | HwIntrinsics.DisableAVX512F | HwIntrinsics.DisableAVX | HwIntrinsics.DisableHWIntrinsic; + + /// + /// The hardware configurations covering the narrower vector widths and scalar fallback for the profile matrix. + /// + private const HwIntrinsics ProfileFallbackConfigurations = + HwIntrinsics.DisableAVX512F | HwIntrinsics.DisableAVX | HwIntrinsics.DisableHWIntrinsic; + + /// + /// The hardware configurations covering the 128-bit and scalar lossless inverse-transform paths. + /// + private const HwIntrinsics LosslessConfigurations = HwIntrinsics.AllowAll | HwIntrinsics.DisableHWIntrinsic; + + /// + /// The hardware configurations covering the 256-bit, 128-bit, and scalar loop-restoration paths. + /// + private const HwIntrinsics LoopRestorationConfigurations = + HwIntrinsics.AllowAll | HwIntrinsics.DisableAVX | HwIntrinsics.DisableHWIntrinsic; + + /// + /// The coverage bit representing an active Wiener restoration unit. + /// + private const int WienerRestorationCoverage = 1 << (int)Av1RestorationFilterType.Wiener; + + /// + /// The coverage bit representing an active self-guided restoration unit. + /// + private const int SelfGuidedRestorationCoverage = 1 << (int)Av1RestorationFilterType.SgrProjection; + + /// + /// The coverage bit representing luma palette prediction. + /// + private const int LumaPaletteCoverage = 1 << 0; + + /// + /// The coverage bit representing chroma palette prediction. + /// + private const int ChromaPaletteCoverage = 1 << 1; + + /// + /// The luma and chroma syntax coverage required from the independent palette fixture. + /// + private const int RequiredPaletteCoverage = LumaPaletteCoverage | ChromaPaletteCoverage; + + /// + /// The bit mask containing every AV1 partition type defined for a coding block. + /// + private const int RequiredPartitionCoverage = (1 << ((int)Av1PartitionType.Vertical4 + 1)) - 1; + + /// + /// The displayed width shared by the independent lossless fixtures. + /// + private const int LosslessFixtureWidth = 100; + + /// + /// The displayed height shared by the independent lossless fixtures. + /// + private const int LosslessFixtureHeight = 60; + + /// + /// The displayed width shared by the independent AV1 profile fixtures. + /// + private const int ProfileFixtureWidth = 512; + + /// + /// The displayed height shared by the independent AV1 profile fixtures. + /// + private const int ProfileFixtureHeight = 256; + + /// + /// The displayed width of the independent two-layer progressive fixture. + /// + private const int ProgressiveFixtureWidth = 33; + + /// + /// The displayed height of the independent two-layer progressive fixture. + /// + private const int ProgressiveFixtureHeight = 11; + + /// + /// The byte length of the fixture's base layer as declared by its a1lx property. + /// + private const int ProgressiveFirstLayerSize = 55; + + /// + /// The displayed width and height of the independent scaled-reference fixture. + /// + private const int ScaledReferenceFixtureSize = 80; + + /// + /// The retained base-layer width and height of the independent scaled-reference fixture. + /// + private const int ScaledReferenceBaseLayerSize = 40; + + /// + /// The byte length of the scaled-reference fixture's base layer as declared by its a1lx property. + /// + private const int ScaledReferenceFirstLayerSize = 701; + + /// + /// The displayed width and height of the independent compound image sequence. + /// + private const int AverageCompoundFixtureSize = 80; + + /// + /// The number of presented frames in the independent compound image sequence. + /// + private const int AverageCompoundFixtureFrameCount = 19; + + /// + /// The number of shown frames in the official AV1 all-intra sequence. + /// + private const int OfficialAllIntraFixtureFrameCount = 39; + + /// + /// The number of shown frames in the official AV1 CDF-update sequence. + /// + private const int OfficialCdfUpdateFixtureFrameCount = 2; + + /// + /// The number of shown frames in the official AV1 temporal motion-field sequence. + /// + private const int OfficialMotionFieldFixtureFrameCount = 4; + + /// + /// The displayed width of the official AV1 extreme-displacement intra-block-copy sequence. + /// + private const int OfficialIntraBlockCopyFixtureWidth = 1920; + + /// + /// The displayed height of the official AV1 extreme-displacement intra-block-copy sequence. + /// + private const int OfficialIntraBlockCopyFixtureHeight = 1080; + + /// + /// The number of shown frames in the official AV1 extreme-displacement intra-block-copy sequence. + /// + private const int OfficialIntraBlockCopyFixtureFrameCount = 2; + + /// + /// The displayed width of the official AV1 two-spatial-layer sequence. + /// + private const int OfficialTwoSpatialLayerFixtureWidth = 1280; + + /// + /// The displayed height of the official AV1 two-spatial-layer sequence. + /// + private const int OfficialTwoSpatialLayerFixtureHeight = 720; + + /// + /// The number of default-operating-point frames in the official AV1 two-spatial-layer sequence. + /// + private const int OfficialTwoSpatialLayerFixtureFrameCount = 8; + + /// + /// The displayed width of the official AV1 two-temporal-layer sequence. + /// + private const int OfficialTwoTemporalLayerFixtureWidth = 640; + + /// + /// The displayed height of the official AV1 two-temporal-layer sequence. + /// + private const int OfficialTwoTemporalLayerFixtureHeight = 360; + + /// + /// The number of default-operating-point frames in the official AV1 two-temporal-layer sequence. + /// + private const int OfficialTwoTemporalLayerFixtureFrameCount = 8; + + /// + /// The displayed width of the official AV1 spatial-and-temporal-layer sequence. + /// + private const int OfficialSpatialTemporalLayerFixtureWidth = 1280; + + /// + /// The displayed height of the official AV1 spatial-and-temporal-layer sequence. + /// + private const int OfficialSpatialTemporalLayerFixtureHeight = 720; + + /// + /// The number of default-operating-point frames in the official AV1 spatial-and-temporal-layer sequence. + /// + private const int OfficialSpatialTemporalLayerFixtureFrameCount = 8; + + /// + /// The number of frames in the official AV1 active-film-grain sequence. + /// + private const int OfficialFilmGrainFixtureFrameCount = 10; + + /// + /// The width of the official AV1 eight-bit monochrome sequence. + /// + private const int OfficialMonochromeFixtureWidth = 320; + + /// + /// The height of the official AV1 eight-bit monochrome sequence. + /// + private const int OfficialMonochromeFixtureHeight = 180; + + /// + /// The number of frames in the official AV1 eight-bit monochrome sequence. + /// + private const int OfficialMonochromeFixtureFrameCount = 10; + + /// + /// The width of the official AV1 eight-bit quantizer-boundary sequences. + /// + private const int OfficialEightBitQuantizerFixtureWidth = 352; + + /// + /// The height of the official AV1 eight-bit quantizer-boundary sequences. + /// + private const int OfficialEightBitQuantizerFixtureHeight = 288; + + /// + /// The width of the official AV1 ten-bit quantizer-boundary sequences. + /// + private const int OfficialTenBitQuantizerFixtureWidth = 640; + + /// + /// The height of the official AV1 ten-bit quantizer-boundary sequences. + /// + private const int OfficialTenBitQuantizerFixtureHeight = 360; + + /// + /// The number of frames in each official AV1 quantizer-boundary sequence. + /// + private const int OfficialQuantizerFixtureFrameCount = 2; + + /// + /// The minimum dimension retained from the official AV1 frame-size matrix. + /// + private const int OfficialFrameSizeFixtureMinimumDimension = 196; + + /// + /// The maximum dimension retained from the official AV1 frame-size matrix. + /// + private const int OfficialFrameSizeFixtureMaximumDimension = 226; + + /// + /// The number of frames in each official AV1 frame-size sequence. + /// + private const int OfficialFrameSizeFixtureFrameCount = 2; + + /// + /// The coverage bit representing tile-local adaptive CDF updates. + /// + private const int TileCdfUpdateCoverage = 1 << 0; + + /// + /// The coverage bit representing publication of the selected frame-end CDF. + /// + private const int FrameEndCdfUpdateCoverage = 1 << 1; + + /// + /// The coverage bit representing temporal reference-motion-vector projection. + /// + private const int ReferenceFrameMotionVectorCoverage = 1 << 2; + + /// + /// The coverage bit representing displayed film-grain synthesis. + /// + private const int FilmGrainCoverage = 1 << 3; + + /// + /// The coverage bit representing a validated inter-frame identifier transition. + /// + private const int FrameIdentifierCoverage = 1 << 4; + + /// + /// The bit mask containing every intra prediction mode. + /// + private const int RequiredIntraModeCoverage = (1 << (int)Av1PredictionMode.IntraModes) - 1; + + /// + /// The transform types selected by the official all-intra conformance sequence. + /// + private const int RequiredAllIntraTransformTypeCoverage = + (1 << (int)Av1TransformType.DctDct) | + (1 << (int)Av1TransformType.AdstDct) | + (1 << (int)Av1TransformType.DctAdst) | + (1 << (int)Av1TransformType.AdstAdst) | + (1 << (int)Av1TransformType.Identity) | + (1 << (int)Av1TransformType.VerticalDct) | + (1 << (int)Av1TransformType.HorizontalDct); + + /// + /// The displayed width of the official AV1 motion-vector sequence. + /// + private const int OfficialMotionVectorFixtureWidth = 352; + + /// + /// The displayed height of the official AV1 motion-vector sequence. + /// + private const int OfficialMotionVectorFixtureHeight = 288; + + /// + /// The number of shown frames in the official AV1 motion-vector sequence. + /// + private const int OfficialMotionVectorFixtureFrameCount = 4; + + /// + /// The bit mask containing every single-reference and compound inter prediction mode. + /// + private const int RequiredInterModeCoverage = + (1 << ((int)Av1PredictionMode.InterModeEnd - (int)Av1PredictionMode.InterModeStart)) - 1; + + /// + /// The bit mask containing every simple, OBMC, and locally warped motion mode. + /// + private const int RequiredMotionModeCoverage = (1 << 3) - 1; + + /// + /// The bit mask containing every regular, smooth, and sharp vertical/horizontal filter pair. + /// + private const int RequiredSwitchableFilterPairCoverage = (1 << 9) - 1; + + /// + /// The coverage bit representing distance-weighted compound prediction. + /// + private const int DistanceWeightedCompoundCoverage = 1 << 0; + + /// + /// The coverage bit representing a non-inverted wedge compound mask. + /// + private const int WedgeCompoundCoverage = 1 << 1; + + /// + /// The coverage bit representing an inverted wedge compound mask. + /// + private const int InvertedWedgeCompoundCoverage = 1 << 2; + + /// + /// The coverage bit representing the first difference-weighted mask orientation. + /// + private const int DifferenceWeightedCompoundCoverage = 1 << 3; + + /// + /// The coverage bit representing the inverted difference-weighted mask orientation. + /// + private const int InvertedDifferenceWeightedCompoundCoverage = 1 << 4; + + /// + /// The coverage bit representing smooth inter-intra prediction. + /// + private const int SmoothInterIntraCoverage = 1 << 5; + + /// + /// The coverage bit representing wedge inter-intra prediction. + /// + private const int WedgeInterIntraCoverage = 1 << 6; + + /// + /// The coverage bit representing overlapping motion compensation. + /// + private const int ObmcCoverage = 1 << 7; + + /// + /// The coverage bit representing local warped-motion prediction. + /// + private const int LocalWarpCoverage = 1 << 8; + + /// + /// The coverage bit representing non-translational global warped-motion prediction. + /// + private const int GlobalWarpCoverage = 1 << 9; + + /// + /// The hardware configurations covering the available vector widths and the scalar color-conversion fallback. + /// + private const HwIntrinsics PresentationConfigurations = + HwIntrinsics.AllowAll | HwIntrinsics.DisableAVX512F | HwIntrinsics.DisableAVX | HwIntrinsics.DisableHWIntrinsic; + + /// + /// Verifies deblocking syntax, filter activation, component traversal, and presentation for real eight-, ten-, + /// and twelve-bit AV1 and AVIF content. + /// + [Fact] + public void DecodeDeblockingMatchesReference() + { + ValidateFixture( + TestImages.Heif.Av1Deblocking8BitAvif, + TestImages.Heif.Av1Deblocking8BitPayload, + TestImages.Heif.Av1Deblocking8BitReference, + 768, + 512, + Av1BitDepth.EightBit, + Av1ColorFormat.Yuv420, + HeifBitDepth.Bit8); + + ValidateFixture( + TestImages.Heif.Av1Deblocking10BitAvif, + TestImages.Heif.Av1Deblocking10BitPayload, + TestImages.Heif.Av1Deblocking10BitReference, + 1024, + 428, + Av1BitDepth.TenBit, + Av1ColorFormat.Yuv444, + HeifBitDepth.Bit10); + + ValidateNativeFixture( + TestImages.Heif.Av1Deblocking12BitPayload, + TestImages.Heif.Av1Deblocking12BitReference, + 1024, + 428, + Av1BitDepth.TwelveBit, + Av1ColorFormat.Yuv444, + requireActiveCdef: false); + + ValidatePresentedImage(TestImages.Heif.Av1Deblocking12BitAvif, 64, 64, HeifBitDepth.Bit12); + } + + /// + /// Verifies active CDEF syntax, strength selection, unit traversal, subsampling, frame edges, and final native + /// samples against the independent scalar reference for independently encoded eight-, ten-, and twelve-bit still-picture streams + /// under normal SIMD dispatch and with hardware intrinsics disabled. + /// + [Fact] + public void DecodeCdefNativeMatchesReference() + => FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidateActiveCdefFixtures, ReconstructionConfigurations); + + /// + /// Verifies exact presented pixels and public metadata for independently encoded eight-, ten-, and twelve-bit + /// active-CDEF AVIF images across the available vector widths and the scalar fallback. + /// + /// The AVIF input and matching reference-output naming context. + /// The expected presented width. + /// The expected presented height. + /// The expected public sample precision. + [Theory] + [WithFile(TestImages.Heif.Av1Cdef8BitAvif, PixelTypes.Rgba32, 768, 512, HeifBitDepth.Bit8)] + [WithFile(TestImages.Heif.Av1Cdef10BitAvif, PixelTypes.Rgba32, 1024, 428, HeifBitDepth.Bit10)] + [WithFile(TestImages.Heif.Av1Cdef12BitAvif, PixelTypes.Rgba32, 1024, 428, HeifBitDepth.Bit12)] + public void DecodeCdefMatchesReference( + TestImageProvider provider, + int width, + int height, + HeifBitDepth bitDepth) + { + AssertPresentedMetadata(provider, width, height, bitDepth); + + FeatureTestRunner.RunWithHwIntrinsicsFeature( + ValidatePresentedFixture, + PresentationConfigurations, + provider); + } + + /// + /// Verifies exact native reconstruction for every valid AV1 profile, bit-depth, and chroma-format combination + /// supported by AVIF across every available vector width and the scalar fallback. + /// + [Fact] + public void DecodeProfileMatrixNativeMatchesReference() + => ValidateProfileNativeFixtures(); + + /// + /// Verifies exact native reconstruction for every valid AV1 profile, bit-depth, and chroma-format combination + /// under each narrower vector width and the scalar fallback. + /// + [Fact] + public void DecodeProfileMatrixFallbacksMatchReference() + => FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidateProfileNativeFixtures, ProfileFallbackConfigurations); + + /// + /// Verifies exact presented pixels, public bit-depth metadata, and CICP signaling for every valid AV1 profile, + /// bit-depth, and chroma-format combination supported by AVIF. + /// + /// The AVIF input and matching reference-output naming context. + /// The expected public sample precision. + [Theory] + [WithFile(TestImages.Heif.Av1Profile8BitMonochromeAvif, PixelTypes.Rgba32, HeifBitDepth.Bit8)] + [WithFile(TestImages.Heif.Av1Profile8Bit420Avif, PixelTypes.Rgba32, HeifBitDepth.Bit8)] + [WithFile(TestImages.Heif.Av1Profile8Bit422Avif, PixelTypes.Rgba32, HeifBitDepth.Bit8)] + [WithFile(TestImages.Heif.Av1Profile8Bit444Avif, PixelTypes.Rgba32, HeifBitDepth.Bit8)] + [WithFile(TestImages.Heif.Av1Profile10BitMonochromeAvif, PixelTypes.Rgba32, HeifBitDepth.Bit10)] + [WithFile(TestImages.Heif.Av1Profile10Bit420Avif, PixelTypes.Rgba32, HeifBitDepth.Bit10)] + [WithFile(TestImages.Heif.Av1Profile10Bit422Avif, PixelTypes.Rgba32, HeifBitDepth.Bit10)] + [WithFile(TestImages.Heif.Av1Profile10Bit444Avif, PixelTypes.Rgba32, HeifBitDepth.Bit10)] + [WithFile(TestImages.Heif.Av1Profile12BitMonochromeAvif, PixelTypes.Rgba32, HeifBitDepth.Bit12)] + [WithFile(TestImages.Heif.Av1Profile12Bit420Avif, PixelTypes.Rgba32, HeifBitDepth.Bit12)] + [WithFile(TestImages.Heif.Av1Profile12Bit422Avif, PixelTypes.Rgba32, HeifBitDepth.Bit12)] + [WithFile(TestImages.Heif.Av1Profile12Bit444Avif, PixelTypes.Rgba32, HeifBitDepth.Bit12)] + public void DecodeProfileMatrixMatchesReference( + TestImageProvider provider, + HeifBitDepth bitDepth) + { + using Image image = provider.GetImage(); + HeifMetadata metadata = image.Metadata.GetHeifMetadata(); + Assert.Equal(HeifCompressionMethod.Av1, metadata.CompressionMethod); + Assert.Equal(bitDepth, metadata.BitDepth); + + CicpProfile colorProfile = Assert.IsType(image.Metadata.CicpProfile); + Assert.Equal(CicpColorPrimaries.ItuRBt709_6, colorProfile.ColorPrimaries); + Assert.Equal(CicpTransferCharacteristics.Iec61966_2_1, colorProfile.TransferCharacteristics); + Assert.Equal(CicpMatrixCoefficients.ItuRBt601_7_525, colorProfile.MatrixCoefficients); + Assert.True(colorProfile.FullRange); + + FeatureTestRunner.RunWithHwIntrinsicsFeature( + ValidatePresentedFixture, + PresentationConfigurations, + provider); + } + + /// + /// Verifies decoded luma and chroma palette syntax and exact native samples for an independently encoded AV1 + /// still-picture stream. + /// + [Fact] + public void DecodePaletteMatchesReference() + => FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidatePaletteNativeFixture, PaletteConfigurations); + + /// + /// Verifies exact palette reconstruction through bounded reusable superblock maps and tracked disposal. + /// + [Fact] + [ValidateDisposedMemoryAllocations] + public void DecodePaletteWithConstrainedAllocator() + { + TestMemoryAllocator allocator = new() { BufferCapacityInBytes = 1_024 }; + allocator.EnableNonThreadSafeLogging(); + Configuration configuration = Configuration.Default.Clone(); + configuration.MemoryAllocator = allocator; + byte[] payload = TestFile.Create(TestImages.Heif.Av1Palette8BitPayload).Bytes; + byte[] reference = TestFile.Create(TestImages.Heif.Av1Palette8BitReference).Bytes; + + using (Av1Decoder decoder = new(configuration)) + { + int paletteMapLength = 1 << Av1Constants.MaxSuperBlockSizeLog2; + TestMemoryAllocator.AllocationRequest paletteMaps = Assert.Single(allocator.AllocationLog); + Assert.Equal(2 * paletteMapLength * paletteMapLength, paletteMaps.Length); + int paletteAllocationId = paletteMaps.AllocationId; + + using Av1FrameBuffer frameBuffer = decoder.DecodeFrameBuffer(payload, null, null, out _); + + Assert.Equal(RequiredPaletteCoverage, GetPaletteCoverage(decoder)); + AssertNativePlanesEqual(decoder, frameBuffer, reference); + Assert.DoesNotContain(allocator.ReturnLog, returned => returned.AllocationId == paletteAllocationId); + } + + Assert.Equal(allocator.AllocationLog.Count, allocator.ReturnLog.Count); + Assert.All( + allocator.AllocationLog, + allocation => Assert.Single( + allocator.ReturnLog, + returned => returned.AllocationId == allocation.AllocationId)); + } + + /// + /// Verifies that a real palette frame whose tile entropy payload ends early is rejected instead of being decoded + /// from the range decoder's implicit zero padding. + /// + [Fact] + public void DecodeFrameBufferRejectsTruncatedPaletteTileEntropy() + { + const int TruncatedTileByteCount = 8; + byte[] validPayload = TestFile.Create(TestImages.Heif.Av1Palette8BitPayload).Bytes; + int obuOffset = 0; + int finalObuOffset = 0; + int finalSizeFieldOffset = 0; + int finalSizeFieldLength = 0; + ulong finalPayloadLength = 0; + while (obuOffset < validPayload.Length) + { + byte obuHeader = validPayload[obuOffset]; + Assert.NotEqual(0, obuHeader & 0x02); + + int headerLength = 1 + ((obuHeader >> 2) & 1); + int sizeFieldOffset = obuOffset + headerLength; + Av1BitStreamReader sizeReader = new(validPayload.AsSpan(sizeFieldOffset)); + ulong payloadLength = sizeReader.ReadLittleEndianBytes128(out int sizeFieldLength); + int nextObuOffset = checked(sizeFieldOffset + sizeFieldLength + (int)payloadLength); + if (nextObuOffset == validPayload.Length) + { + finalObuOffset = obuOffset; + finalSizeFieldOffset = sizeFieldOffset; + finalSizeFieldLength = sizeFieldLength; + finalPayloadLength = payloadLength; + } + + obuOffset = nextObuOffset; + } + + Assert.Equal(ObuType.Frame, (ObuType)((validPayload[finalObuOffset] >> 3) & 0x0F)); + Assert.Equal(1, finalSizeFieldLength); + Assert.InRange(finalPayloadLength, (ulong)(TruncatedTileByteCount + 1), 0x7FUL); + + byte[] truncatedPayload = validPayload[..^TruncatedTileByteCount]; + truncatedPayload[finalSizeFieldOffset] = (byte)(finalPayloadLength - TruncatedTileByteCount); + + using Av1Decoder decoder = new(Configuration.Default); + + Assert.Throws( + () => decoder.DecodeFrameBuffer(truncatedPayload, null, null, out _).Dispose()); + + Assert.Null(decoder.SequenceHeader); + Assert.Null(decoder.FrameHeader); + Assert.Null(decoder.FrameInfo); + } + + /// + /// Verifies decoded luma and chroma palette syntax and exact presented pixels against the retained reference image + /// across the available vector widths and the scalar fallback. + /// + /// The AVIF input and matching reference-output naming context. + [Theory] + [WithFile(TestImages.Heif.Av1Palette8BitAvif, PixelTypes.Rgba32)] + public void DecodePalettePresentationMatchesReference(TestImageProvider provider) + => FeatureTestRunner.RunWithHwIntrinsicsFeature( + ValidatePresentedFixture, + PresentationConfigurations, + provider); + + /// + /// Verifies that malformed data following a decoded palette tile releases its frame state before the same decoder + /// processes another payload. + /// + [Fact] + public void DecodeFrameBufferRecoversAfterMalformedFollowingObu() + { + byte[] validPayload = TestFile.Create(TestImages.Heif.Av1Palette8BitPayload).Bytes; + + // The palette fixture ends with one combined-frame OBU containing one tile, so the intact prefix creates and + // completes a real Av1TileReader. The appended padding OBU declares one zero byte; AV1 padding requires a + // trailing-one bit, making this later bounded-payload failure deterministic without corrupting tile entropy. + byte[] malformedPayload = + [ + .. validPayload, + 0x7A, // Padding OBU with an explicit payload-size field. + 0x01, // LEB128 payload length of one byte. + 0x00, // Invalid padding payload with no trailing-one bit. + ]; + + using Av1Decoder decoder = new(Configuration.Default); + + Assert.Throws( + () => decoder.DecodeFrameBuffer(malformedPayload, null, null, out _).Dispose()); + + Assert.Null(decoder.SequenceHeader); + Assert.Null(decoder.FrameHeader); + Assert.Null(decoder.FrameInfo); + + using Av1FrameBuffer recoveredFrameBuffer = decoder.DecodeFrameBuffer(validPayload, null, null, out _); + + Assert.Equal(33, recoveredFrameBuffer.Width); + Assert.Equal(11, recoveredFrameBuffer.Height); + Assert.Equal(RequiredPaletteCoverage, GetPaletteCoverage(decoder)); + } + + /// + /// Verifies selected intra-block-copy prediction and exact native samples against the independent scalar reference for an + /// independently encoded AV1 still pictures across every available vector width and the scalar fallback. + /// + [Fact] + public void DecodeIntraBlockCopyNativeMatchesReference() + => FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidateIntraBlockCopyNativeFixtures, IntraBlockCopyConfigurations); + + /// + /// Verifies exact presented pixels for independently encoded intra-block-copy AVIF images across the available + /// vector widths and the scalar fallback. + /// + /// The AVIF input and matching reference-output naming context. + [Theory] + [WithFile(TestImages.Heif.Av1IntraBlockCopy8BitAvif, PixelTypes.Rgba32)] + [WithFile(TestImages.Heif.Av1IntraBlockCopy10BitAvif, PixelTypes.Rgba32)] + [WithFile(TestImages.Heif.Av1IntraBlockCopy12BitAvif, PixelTypes.Rgba32)] + public void DecodeIntraBlockCopyMatchesReference( + TestImageProvider provider) + => FeatureTestRunner.RunWithHwIntrinsicsFeature( + ValidatePresentedFixture, + PresentationConfigurations, + provider); + + /// + /// Verifies the production single-reference inter-reconstruction path against exact native and presentation + /// references across the available vector widths and scalar fallback. + /// + /// The AVIF input and matching reference-output naming context. + [Theory] + [WithFile(TestImages.Heif.Av1Progressive8BitAvif, PixelTypes.Rgba32)] + public void DecodeProgressiveSingleMatchesReference( + TestImageProvider provider) + => FeatureTestRunner.RunWithHwIntrinsicsFeature( + ValidateProgressiveSingleReferenceFixtureWithDefaultConfiguration, + PresentationConfigurations, + provider); + + /// + /// Verifies production single-reference inter reconstruction with a constrained allocator. + /// + [Fact] + [ValidateDisposedMemoryAllocations] + public void DecodeProgressiveSingleWithConstrainedAllocator() + { + TestMemoryAllocator allocator = new() { BufferCapacityInBytes = 1_024 }; + allocator.EnableNonThreadSafeLogging(); + Configuration configuration = Configuration.Default.Clone(); + configuration.MemoryAllocator = allocator; + + ValidateProgressiveSingleReferenceFixture(configuration); + + Assert.Contains(allocator.AllocationLog, allocation => allocation.ElementType == typeof(Av1BlockModeInfo)); + Assert.Contains(allocator.AllocationLog, allocation => allocation.ElementType == typeof(Av1TransformInfo)); + Assert.Equal(allocator.AllocationLog.Count, allocator.ReturnLog.Count); + Assert.All( + allocator.AllocationLog, + allocation => Assert.Single( + allocator.ReturnLog, + returned => returned.AllocationId == allocation.AllocationId)); + } + + /// + /// Verifies that an essential lsel property returns the selected base spatial layer rather than the final + /// progressive layer, with exact reference native planes and the retained exact presentation reference. + /// + /// The selected-layer AVIF input and matching reference-output naming context. + [Theory] + [WithFile(TestImages.Heif.Av1ScaledReferenceSelectedLayerAvif, PixelTypes.Rgba32)] + public void DecodeSelectedProgressiveLayerMatchesReference( + TestImageProvider provider) + => FeatureTestRunner.RunWithHwIntrinsicsFeature( + ValidateSelectedProgressiveSpatialLayerWithDefaultConfiguration, + PresentationConfigurations, + provider); + + /// + /// Verifies selected-layer native reconstruction and public presentation with constrained tracked allocation. + /// + [Fact] + [ValidateDisposedMemoryAllocations] + public void DecodeSelectedProgressiveLayerWithConstrainedAllocator() + { + TestMemoryAllocator allocator = new() { BufferCapacityInBytes = 1_024 }; + allocator.EnableNonThreadSafeLogging(); + Configuration configuration = Configuration.Default.Clone(); + configuration.MemoryAllocator = allocator; + + ValidateSelectedProgressiveSpatialLayer(configuration); + + Assert.Equal(allocator.AllocationLog.Count, allocator.ReturnLog.Count); + Assert.All( + allocator.AllocationLog, + allocation => Assert.Single( + allocator.ReturnLog, + returned => returned.AllocationId == allocation.AllocationId)); + } + + /// + /// Verifies an independently encoded 40x40 retained layer scaled into an 80x80 dependent layer against exact + /// reference native planes and the retained exact presentation reference. + /// + /// The AVIF input and matching reference-output naming context. + [Theory] + [WithFile(TestImages.Heif.Av1ScaledReferenceAvif, PixelTypes.Rgba32)] + public void DecodeScaledReferenceMatchesReference(TestImageProvider provider) + => FeatureTestRunner.RunWithHwIntrinsicsFeature( + ValidateScaledReferenceFixtureWithDefaultConfiguration, + ReconstructionConfigurations, + provider); + + /// + /// Verifies scaled-reference reconstruction with constrained tracked allocation and contiguous frame planes. + /// + [Fact] + [ValidateDisposedMemoryAllocations] + public void DecodeScaledReferenceWithConstrainedAllocator() + { + TestMemoryAllocator allocator = new() { BufferCapacityInBytes = 1_024 }; + allocator.EnableNonThreadSafeLogging(); + Configuration configuration = Configuration.Default.Clone(); + configuration.MemoryAllocator = allocator; + + ValidateScaledReferenceFixture(configuration); + + Assert.Contains(allocator.AllocationLog, request => request.ElementType.Name == "RetainedMotionFieldEntry"); + Assert.Contains(allocator.AllocationLog, request => request.ElementType.Name == "TemporalMotionFieldEntry"); + Assert.Equal(allocator.AllocationLog.Count, allocator.ReturnLog.Count); + Assert.All( + allocator.AllocationLog, + allocation => Assert.Single( + allocator.ReturnLog, + returned => returned.AllocationId == allocation.AllocationId)); + } + + /// + /// Verifies that the production dependent-frame result owns its motion-field storage until decoder disposal. + /// + [Fact] + public void DecodeProgressiveSingleTracksMotionFieldOwnership() + { + TestMemoryAllocator allocator = new(); + allocator.EnableNonThreadSafeLogging(); + Configuration configuration = Configuration.Default.Clone(); + configuration.MemoryAllocator = allocator; + byte[] payload = TestFile.Create(TestImages.Heif.Av1Progressive8BitPayload).Bytes; + + using Av1Decoder decoder = new(configuration); + using Av1FrameBuffer frameBuffer = decoder.DecodeFrameBuffer( + payload, + null, + null, + out _, + new Av1LayeredImageIndex(ProgressiveFirstLayerSize, 0, 0)); + + TestMemoryAllocator.AllocationRequest retainedMotionField = Assert.Single( + allocator.AllocationLog, + request => request.ElementType.Name == "RetainedMotionFieldEntry"); + + TestMemoryAllocator.AllocationRequest temporalMotionField = Assert.Single( + allocator.AllocationLog, + request => request.ElementType.Name == "TemporalMotionFieldEntry"); + + // Reference-slot and presentation owners are released while DecodeFrameBuffer transfers the native planes. + // The decoder's inspectable FrameInfo result remains the final motion-field owner until decoder disposal. + Assert.DoesNotContain( + allocator.ReturnLog, + returned => returned.AllocationId == retainedMotionField.AllocationId); + + Assert.DoesNotContain( + allocator.ReturnLog, + returned => returned.AllocationId == temporalMotionField.AllocationId); + + frameBuffer.Dispose(); + + Assert.DoesNotContain( + allocator.ReturnLog, + returned => returned.AllocationId == retainedMotionField.AllocationId); + + Assert.DoesNotContain( + allocator.ReturnLog, + returned => returned.AllocationId == temporalMotionField.AllocationId); + + decoder.Dispose(); + decoder.Dispose(); + + Assert.Single( + allocator.ReturnLog, + returned => returned.AllocationId == retainedMotionField.AllocationId); + + Assert.Single(allocator.ReturnLog, returned => returned.AllocationId == temporalMotionField.AllocationId); + } + + /// + /// Verifies exact native reconstruction and presentation for an image sequence that exercises equal-weight + /// compound prediction. The native reference has been reverified against the current AV1 reference. + /// + [Theory] + [WithFile(TestImages.Heif.Av1AverageCompoundSequenceAvif, PixelTypes.Rgba32)] + public void DecodeEqualAverageCompoundMatchesReference( + TestImageProvider provider) + + => FeatureTestRunner.RunWithHwIntrinsicsFeature( + ValidateAverageCompoundSequenceWithDefaultConfiguration, + ReconstructionConfigurations, + provider); + + /// + /// Verifies the complete compound sequence through a constrained allocator. + /// + [Fact] + [ValidateDisposedMemoryAllocations] + public void DecodeEqualAverageCompoundWithConstrainedAllocator() + { + TestMemoryAllocator allocator = new() { BufferCapacityInBytes = 1_024 }; + allocator.EnableNonThreadSafeLogging(); + Configuration configuration = Configuration.Default.Clone(); + configuration.MemoryAllocator = allocator; + + ValidateAverageCompoundSequence(configuration); + + Assert.Contains(allocator.AllocationLog, request => request.ElementType.Name == "RetainedMotionFieldEntry"); + Assert.Contains(allocator.AllocationLog, request => request.ElementType.Name == "TemporalMotionFieldEntry"); + Assert.Equal(allocator.AllocationLog.Count, allocator.ReturnLog.Count); + Assert.All( + allocator.AllocationLog, + allocation => Assert.Single( + allocator.ReturnLog, + returned => returned.AllocationId == allocation.AllocationId)); + } + + /// + /// Runs the exact compound-sequence comparisons with the default configuration. + /// + /// The serialized input provider and reference-output naming context. + private static void ValidateAverageCompoundSequenceWithDefaultConfiguration(string providerDump) + { + ValidateAverageCompoundSequence(Configuration.Default); + ValidateFinalSequencePresentation(providerDump); + } + + /// + /// Validates the complete compound sequence with the requested allocator. + /// + /// The decoder configuration. + private static void ValidateAverageCompoundSequence(Configuration configuration) + { + byte[] fileBytes = TestFile.Create(TestImages.Heif.Av1AverageCompoundSequenceAvif).Bytes; + byte[] referenceBytes = TestFile.Create(TestImages.Heif.Av1AverageCompoundSequenceNativeReference).Bytes; + ReadOnlySpan fileHeader = + "YUV4MPEG2 W80 H80 F25:1 Ip A0:0 C444 XYSCSS=444 XCOLORRANGE=LIMITED\n"u8; + + ReadOnlySpan frameHeader = "FRAME\n"u8; + + ReadOnlySpan nativeReference = referenceBytes; + Assert.True(nativeReference.StartsWith(fileHeader)); + nativeReference = nativeReference[fileHeader.Length..]; + Assert.True(nativeReference.StartsWith(frameHeader)); + nativeReference = nativeReference[frameHeader.Length..]; + Assert.Equal(AverageCompoundFixtureSize * AverageCompoundFixtureSize * 3, nativeReference.Length); + + HeifSequence sequence = ParseImageSequence(fileBytes); + HeifSequenceTrack track = sequence.ColorTrack; + int compoundBlockCount = 0; + int visibleFrameCount = 0; + bool nativeCompared = false; + + using Av1Decoder decoder = new(configuration); + for (int sampleIndex = 0; sampleIndex < track.Samples.Length; sampleIndex++) + { + HeifSequenceSample sample = track.Samples[sampleIndex]; + Span sampleData = fileBytes.AsSpan((int)sample.Offset, sample.Length); + if (sample.IsHidden) + { + decoder.DecodeSequenceReference( + sampleData, + track.CicpProfile, + track.Av1CodecConfiguration); + + continue; + } + + ImageFrame decodedFrame; + try + { + decodedFrame = decoder.DecodeSequenceFrame( + sampleData, + track.CicpProfile, + track.Av1CodecConfiguration); + } + catch (InvalidImageContentException exception) + { + throw new InvalidImageContentException($"The verified compound fixture failed at sample {sampleIndex}.", exception); + } + + using ImageFrame frame = decodedFrame; + + _ = Assert.IsType(decoder.FrameHeader); + Av1FrameBuffer frameBuffer = Assert.IsType>(decoder.FrameBuffer); + Av1FrameInfo frameInfo = decoder.FrameInfo; + + // Inter prediction addresses padding with one base span and a logical row stride. The frame owner must + // preserve that contract even when the configured allocator would ordinarily split a large buffer. + Assert.Equal(1, frameBuffer.GetPlaneBuffer(Av1Plane.Y).FastMemoryGroup.Count); + Assert.Equal(1, frameBuffer.GetPlaneBuffer(Av1Plane.U).FastMemoryGroup.Count); + Assert.Equal(1, frameBuffer.GetPlaneBuffer(Av1Plane.V).FastMemoryGroup.Count); + + // A pure show_existing_frame payload presents a retained reference without decoding new block syntax. + if (frameInfo is not null) + { + ObuSequenceHeader sequenceHeader = Assert.IsType(decoder.SequenceHeader); + int superblockSizeLog2 = sequenceHeader.SuperblockSizeLog2; + int superblockColumnCount = Av1Math.AlignPowerOf2(sequenceHeader.MaxFrameWidth, superblockSizeLog2) >> superblockSizeLog2; + int superblockRowCount = Av1Math.AlignPowerOf2(sequenceHeader.MaxFrameHeight, superblockSizeLog2) >> superblockSizeLog2; + + for (int superblockRow = 0; superblockRow < superblockRowCount; superblockRow++) + { + for (int superblockColumn = 0; superblockColumn < superblockColumnCount; superblockColumn++) + { + Av1SuperblockInfo superblockInfo = frameInfo.GetSuperblock(new Point(superblockColumn, superblockRow)); + foreach (Av1BlockModeInfo modeInfo in superblockInfo.GetModeInfos()) + { + if (modeInfo.ReferenceFrames[1] <= Av1ReferenceFrameType.Intra) + { + continue; + } + + Assert.Equal(Av1CompoundType.Average, modeInfo.CompoundType); + compoundBlockCount++; + } + } + } + } + + if (visibleFrameCount == AverageCompoundFixtureFrameCount - 1) + { + Assert.Equal(AverageCompoundFixtureSize, frameBuffer.Width); + Assert.Equal(AverageCompoundFixtureSize, frameBuffer.Height); + Assert.Equal(Av1BitDepth.EightBit, frameBuffer.BitDepth); + Assert.Equal(Av1ColorFormat.Yuv444, frameBuffer.ColorFormat); + AssertNativePlanesEqual(decoder, frameBuffer, nativeReference); + nativeCompared = true; + } + + visibleFrameCount++; + } + + Assert.Equal(AverageCompoundFixtureFrameCount, visibleFrameCount); + Assert.NotEqual(0, compoundBlockCount); + Assert.True(nativeCompared); + } + + /// + /// Verifies every selectable compound and inter-intra production branch against retained native and presentation references. + /// + [Theory] + [WithFile(TestImages.Heif.Av1DistanceWeightedCompoundSequenceAvif, PixelTypes.Rgba32)] + [WithFile(TestImages.Heif.Av1WedgeCompoundSequenceAvif, PixelTypes.Rgba32)] + [WithFile(TestImages.Heif.Av1DifferenceWeightedCompoundSequenceAvif, PixelTypes.Rgba32)] + [WithFile(TestImages.Heif.Av1InterIntraSequenceAvif, PixelTypes.Rgba32)] + public void DecodeSelectableCompoundMatchesReference( + TestImageProvider provider) + + => FeatureTestRunner.RunWithHwIntrinsicsFeature( + ValidateSelectableCompoundSequenceWithDefaultConfiguration, + ReconstructionConfigurations, + provider); + + /// + /// Verifies selectable compound and inter-intra reconstruction through a constrained allocator. + /// + [Fact] + [ValidateDisposedMemoryAllocations] + public void DecodeSelectableCompoundWithConstrainedAllocator() + { + ValidateInterPredictionSequenceWithConstrainedAllocator( + TestImages.Heif.Av1DistanceWeightedCompoundSequenceAvif, + TestImages.Heif.Av1DistanceWeightedCompoundSequenceNativeReference, + DistanceWeightedCompoundCoverage, + AverageCompoundFixtureSize, + AverageCompoundFixtureFrameCount); + + ValidateInterPredictionSequenceWithConstrainedAllocator( + TestImages.Heif.Av1WedgeCompoundSequenceAvif, + TestImages.Heif.Av1WedgeCompoundSequenceNativeReference, + WedgeCompoundCoverage | InvertedWedgeCompoundCoverage, + AverageCompoundFixtureSize, + AverageCompoundFixtureFrameCount); + + ValidateInterPredictionSequenceWithConstrainedAllocator( + TestImages.Heif.Av1DifferenceWeightedCompoundSequenceAvif, + TestImages.Heif.Av1DifferenceWeightedCompoundSequenceNativeReference, + DifferenceWeightedCompoundCoverage | InvertedDifferenceWeightedCompoundCoverage, + AverageCompoundFixtureSize, + AverageCompoundFixtureFrameCount); + + ValidateInterPredictionSequenceWithConstrainedAllocator( + TestImages.Heif.Av1InterIntraSequenceAvif, + TestImages.Heif.Av1InterIntraSequenceNativeReference, + SmoothInterIntraCoverage | WedgeInterIntraCoverage, + AverageCompoundFixtureSize, + AverageCompoundFixtureFrameCount); + } + + /// + /// Verifies production OBMC reconstruction against exact native and presentation references. The native reference + /// has been reverified against the current AV1 reference. + /// + [Theory] + [WithFile(TestImages.Heif.Av1ObmcSequenceAvif, PixelTypes.Rgba32)] + public void DecodeObmcMatchesReference(TestImageProvider provider) + => FeatureTestRunner.RunWithHwIntrinsicsFeature( + ValidateObmcSequenceWithDefaultConfiguration, + ReconstructionConfigurations, + provider); + + /// + /// Verifies production OBMC reconstruction through a constrained allocator. + /// + [Fact] + [ValidateDisposedMemoryAllocations] + public void DecodeObmcWithConstrainedAllocator() + => ValidateInterPredictionSequenceWithConstrainedAllocator( + TestImages.Heif.Av1ObmcSequenceAvif, + TestImages.Heif.Av1ObmcSequenceNativeReference, + ObmcCoverage, + AverageCompoundFixtureSize, + AverageCompoundFixtureFrameCount); + + /// + /// Verifies production local warped-motion reconstruction against reference native and retained presentation references. + /// + [Theory] + [WithFile(TestImages.Heif.Av1LocalWarpSequenceAvif, PixelTypes.Rgba32)] + public void DecodeLocalWarpMatchesReference(TestImageProvider provider) + => FeatureTestRunner.RunWithHwIntrinsicsFeature( + ValidateLocalWarpSequenceWithDefaultConfiguration, + ReconstructionConfigurations, + provider); + + /// + /// Verifies production local warped-motion reconstruction through a constrained allocator. + /// + [Fact] + [ValidateDisposedMemoryAllocations] + public void DecodeLocalWarpWithConstrainedAllocator() + { + TestMemoryAllocator allocator = new() { BufferCapacityInBytes = 1_024 }; + allocator.EnableNonThreadSafeLogging(); + Configuration configuration = Configuration.Default.Clone(); + configuration.MemoryAllocator = allocator; + + ValidateInterPredictionSequence( + configuration, + TestImages.Heif.Av1LocalWarpSequenceAvif, + TestImages.Heif.Av1LocalWarpSequenceNativeReference, + LocalWarpCoverage, + 256, + 2); + + Assert.Contains(allocator.AllocationLog, request => request.ElementType.Name == "RetainedMotionFieldEntry"); + Assert.Contains(allocator.AllocationLog, request => request.ElementType.Name == "TemporalMotionFieldEntry"); + Assert.Equal(allocator.AllocationLog.Count, allocator.ReturnLog.Count); + Assert.All( + allocator.AllocationLog, + allocation => Assert.Single( + allocator.ReturnLog, + returned => returned.AllocationId == allocation.AllocationId)); + } + + /// + /// Verifies production non-translational global motion against reference native and retained presentation references. + /// + [Theory] + [WithFile(TestImages.Heif.Av1GlobalWarpSequenceAvif, PixelTypes.Rgba32)] + public void DecodeGlobalWarpMatchesReference(TestImageProvider provider) + => FeatureTestRunner.RunWithHwIntrinsicsFeature( + ValidateGlobalWarpSequenceWithDefaultConfiguration, + ReconstructionConfigurations, + provider); + + /// + /// Verifies production non-translational global-motion reconstruction through a constrained allocator. + /// + [Fact] + [ValidateDisposedMemoryAllocations] + public void DecodeGlobalWarpWithConstrainedAllocator() + => ValidateInterPredictionSequenceWithConstrainedAllocator( + TestImages.Heif.Av1GlobalWarpSequenceAvif, + TestImages.Heif.Av1GlobalWarpSequenceNativeReference, + GlobalWarpCoverage, + 256, + 2); + + /// + /// Verifies every intra prediction mode and the fixture's seven transform types against the official + /// reference all-intra conformance sequence and its exact native output. + /// + [Fact] + public void DecodeAllIntraMatchesReference() => ValidateOfficialAllIntraFixture(); + + /// + /// Decodes every all-intra IVF sample in one session, compares each frame exactly, and records the syntax + /// selections that make the fixture authoritative for prediction and transform coverage. + /// + private static void ValidateOfficialAllIntraFixture() + { + byte[] ivf = TestFile.Create(TestImages.Heif.Av1OfficialAllIntraSequence).Bytes; + byte[] nativeReference = TestFile.Create(TestImages.Heif.Av1OfficialAllIntraSequenceNativeReference).Bytes; + ReadOnlySpan y4mFileHeader = "YUV4MPEG2 W352 H288 F3:1 Ip C420jpeg\n"u8; + ReadOnlySpan y4mFrameHeader = "FRAME\n"u8; + + Assert.True(ivf.AsSpan(0, 4).SequenceEqual("DKIF"u8)); + Assert.Equal(0, BinaryPrimitives.ReadUInt16LittleEndian(ivf.AsSpan(4, 2))); + Assert.Equal(32, BinaryPrimitives.ReadUInt16LittleEndian(ivf.AsSpan(6, 2))); + Assert.True(ivf.AsSpan(8, 4).SequenceEqual("AV01"u8)); + Assert.Equal(OfficialMotionVectorFixtureWidth, BinaryPrimitives.ReadUInt16LittleEndian(ivf.AsSpan(12, 2))); + Assert.Equal(OfficialMotionVectorFixtureHeight, BinaryPrimitives.ReadUInt16LittleEndian(ivf.AsSpan(14, 2))); + Assert.Equal( + OfficialAllIntraFixtureFrameCount, + checked((int)BinaryPrimitives.ReadUInt32LittleEndian(ivf.AsSpan(24, 4)))); + + Assert.True(nativeReference.AsSpan().StartsWith(y4mFileHeader)); + + int ivfOffset = 32; + int nativeOffset = y4mFileHeader.Length; + int nativeFrameLength = + (OfficialMotionVectorFixtureWidth * OfficialMotionVectorFixtureHeight) + + (2 * (OfficialMotionVectorFixtureWidth >> 1) * (OfficialMotionVectorFixtureHeight >> 1)); + + int intraModeCoverage = 0; + int transformTypeCoverage = 0; + using Av1Decoder decoder = new(Configuration.Default); + for (int frameIndex = 0; frameIndex < OfficialAllIntraFixtureFrameCount; frameIndex++) + { + int payloadLength = checked((int)BinaryPrimitives.ReadUInt32LittleEndian(ivf.AsSpan(ivfOffset, 4))); + ivfOffset += 12; + using ImageFrame frame = decoder.DecodeSequenceFrame( + ivf.AsSpan(ivfOffset, payloadLength), + null, + null); + + ivfOffset += payloadLength; + Assert.Equal(OfficialMotionVectorFixtureWidth, frame.Width); + Assert.Equal(OfficialMotionVectorFixtureHeight, frame.Height); + Assert.True(nativeReference.AsSpan(nativeOffset).StartsWith(y4mFrameHeader)); + nativeOffset += y4mFrameHeader.Length; + + Av1FrameBuffer frameBuffer = Assert.IsType>(decoder.FrameBuffer); + Assert.Equal(OfficialMotionVectorFixtureWidth, frameBuffer.Width); + Assert.Equal(OfficialMotionVectorFixtureHeight, frameBuffer.Height); + Assert.Equal(Av1BitDepth.EightBit, frameBuffer.BitDepth); + Assert.Equal(Av1ColorFormat.Yuv420, frameBuffer.ColorFormat); + AssertNativePlanesEqual( + decoder, + frameBuffer, + nativeReference.AsSpan(nativeOffset, nativeFrameLength)); + + nativeOffset += nativeFrameLength; + + ObuSequenceHeader sequenceHeader = Assert.IsType(decoder.SequenceHeader); + Av1FrameInfo frameInfo = Assert.IsType(decoder.FrameInfo); + transformTypeCoverage |= frameInfo.LumaTransformTypeCoverage; + int superblockColumnCount = Av1Math.AlignPowerOf2(sequenceHeader.MaxFrameWidth, sequenceHeader.SuperblockSizeLog2) + >> sequenceHeader.SuperblockSizeLog2; + int superblockRowCount = Av1Math.AlignPowerOf2(sequenceHeader.MaxFrameHeight, sequenceHeader.SuperblockSizeLog2) + >> sequenceHeader.SuperblockSizeLog2; + + for (int superblockRow = 0; superblockRow < superblockRowCount; superblockRow++) + { + for (int superblockColumn = 0; superblockColumn < superblockColumnCount; superblockColumn++) + { + Av1SuperblockInfo superblockInfo = frameInfo.GetSuperblock(new Point(superblockColumn, superblockRow)); + foreach (Av1BlockModeInfo modeInfo in superblockInfo.GetModeInfos()) + { + if (modeInfo.YMode is >= Av1PredictionMode.IntraModeStart and < Av1PredictionMode.IntraModeEnd) + { + intraModeCoverage |= 1 << ((int)modeInfo.YMode - (int)Av1PredictionMode.IntraModeStart); + } + } + } + } + } + + Assert.Equal(ivf.Length, ivfOffset); + Assert.Equal(nativeReference.Length, nativeOffset); + Assert.Equal(RequiredIntraModeCoverage, intraModeCoverage); + Assert.Equal(RequiredAllIntraTransformTypeCoverage, transformTypeCoverage); + } + + /// + /// Verifies adaptive tile and frame-end CDF updates against the official reference sequence and exact native + /// output under normal and scalar dispatch. + /// + [Fact] + public void DecodeCdfUpdateMatchesReference() + => FeatureTestRunner.RunWithHwIntrinsicsFeature( + ValidateOfficialCdfUpdateFixture, + ReconstructionConfigurations); + + /// + /// Verifies temporal reference-motion-vector projection against the official reference sequence and exact + /// native output under normal and scalar dispatch. + /// + [Fact] + public void DecodeMotionFieldMatchesReference() + => FeatureTestRunner.RunWithHwIntrinsicsFeature( + ValidateOfficialMotionFieldFixture, + ReconstructionConfigurations); + + /// + /// Verifies extreme intra-block-copy displacement vectors against the official reference sequence and exact + /// native output under normal and scalar dispatch. + /// + [Fact] + public void DecodeIntraBlockCopySequenceMatchesReference() + => FeatureTestRunner.RunWithHwIntrinsicsFeature( + ValidateOfficialIntraBlockCopyFixture, + ReconstructionConfigurations); + + /// + /// Decodes the official intra-block-copy sequence and proves that the active copied blocks reconstruct exactly. + /// + private static void ValidateOfficialIntraBlockCopyFixture() + { + byte[] ivf = TestFile.Create(TestImages.Heif.Av1OfficialIntraBlockCopySequence).Bytes; + byte[] nativeReference = TestFile.Create(TestImages.Heif.Av1OfficialIntraBlockCopySequenceNativeReference).Bytes; + ReadOnlySpan y4mFileHeader = "YUV4MPEG2 W1920 H1080 F30:1 Ip C420jpeg\n"u8; + ReadOnlySpan y4mFrameHeader = "FRAME\n"u8; + + Assert.True(ivf.AsSpan(0, 4).SequenceEqual("DKIF"u8)); + Assert.Equal(0, BinaryPrimitives.ReadUInt16LittleEndian(ivf.AsSpan(4, 2))); + Assert.Equal(32, BinaryPrimitives.ReadUInt16LittleEndian(ivf.AsSpan(6, 2))); + Assert.True(ivf.AsSpan(8, 4).SequenceEqual("AV01"u8)); + Assert.Equal(OfficialIntraBlockCopyFixtureWidth, BinaryPrimitives.ReadUInt16LittleEndian(ivf.AsSpan(12, 2))); + Assert.Equal(OfficialIntraBlockCopyFixtureHeight, BinaryPrimitives.ReadUInt16LittleEndian(ivf.AsSpan(14, 2))); + Assert.Equal( + OfficialIntraBlockCopyFixtureFrameCount, + checked((int)BinaryPrimitives.ReadUInt32LittleEndian(ivf.AsSpan(24, 4)))); + + Assert.True(nativeReference.AsSpan().StartsWith(y4mFileHeader)); + + int ivfOffset = 32; + int nativeOffset = y4mFileHeader.Length; + int nativeFrameLength = + (OfficialIntraBlockCopyFixtureWidth * OfficialIntraBlockCopyFixtureHeight) + + (2 * (OfficialIntraBlockCopyFixtureWidth >> 1) * (OfficialIntraBlockCopyFixtureHeight >> 1)); + + int intraBlockCopyBlockCount = 0; + bool allowIntraBlockCopy = false; + using Av1Decoder decoder = new(Configuration.Default); + for (int frameIndex = 0; frameIndex < OfficialIntraBlockCopyFixtureFrameCount; frameIndex++) + { + int payloadLength = checked((int)BinaryPrimitives.ReadUInt32LittleEndian(ivf.AsSpan(ivfOffset, 4))); + ivfOffset += 12; + using ImageFrame frame = decoder.DecodeSequenceFrame( + ivf.AsSpan(ivfOffset, payloadLength), + null, + null); + + ivfOffset += payloadLength; + Assert.Equal(OfficialIntraBlockCopyFixtureWidth, frame.Width); + Assert.Equal(OfficialIntraBlockCopyFixtureHeight, frame.Height); + Assert.True(nativeReference.AsSpan(nativeOffset).StartsWith(y4mFrameHeader)); + nativeOffset += y4mFrameHeader.Length; + + Av1FrameBuffer frameBuffer = Assert.IsType>(decoder.FrameBuffer); + Assert.Equal(OfficialIntraBlockCopyFixtureWidth, frameBuffer.Width); + Assert.Equal(OfficialIntraBlockCopyFixtureHeight, frameBuffer.Height); + Assert.Equal(Av1BitDepth.EightBit, frameBuffer.BitDepth); + Assert.Equal(Av1ColorFormat.Yuv420, frameBuffer.ColorFormat); + AssertNativePlanesEqual( + decoder, + frameBuffer, + nativeReference.AsSpan(nativeOffset, nativeFrameLength)); + + nativeOffset += nativeFrameLength; + ObuFrameHeader frameHeader = Assert.IsType(decoder.FrameHeader); + allowIntraBlockCopy |= frameHeader.AllowIntraBlockCopy; + intraBlockCopyBlockCount += GetIntraBlockCopyBlockCount(decoder); + } + + Assert.Equal(ivf.Length, ivfOffset); + Assert.Equal(nativeReference.Length, nativeOffset); + Assert.True(allowIntraBlockCopy); + Assert.NotEqual(0, intraBlockCopyBlockCount); + } + + /// + /// Verifies exact temporal motion-field reconstruction and balanced ownership with a constrained allocator. + /// + [Fact] + [ValidateDisposedMemoryAllocations] + public void DecodeMotionFieldWithConstrainedAllocator() + { + TestMemoryAllocator allocator = new() { BufferCapacityInBytes = 2_048 }; + allocator.EnableNonThreadSafeLogging(); + Configuration configuration = Configuration.Default.Clone(); + configuration.MemoryAllocator = allocator; + + int coverage = ValidateCompactSequence( + configuration, + TestImages.Heif.Av1OfficialMotionFieldSequence, + TestImages.Heif.Av1OfficialMotionFieldSequenceNativeReference, + OfficialMotionFieldFixtureFrameCount); + + Assert.NotEqual(0, coverage & ReferenceFrameMotionVectorCoverage); + Assert.Contains(allocator.AllocationLog, request => request.ElementType.Name == "RetainedMotionFieldEntry"); + Assert.Contains(allocator.AllocationLog, request => request.ElementType.Name == "TemporalMotionFieldEntry"); + Assert.Equal(allocator.AllocationLog.Count, allocator.ReturnLog.Count); + Assert.All( + allocator.AllocationLog, + allocation => Assert.Single( + allocator.ReturnLog, + returned => returned.AllocationId == allocation.AllocationId)); + } + + /// + /// Validates that the official CDF-update fixture selects both adaptive update boundaries. + /// + private static void ValidateOfficialCdfUpdateFixture() + { + int coverage = ValidateCompactSequence( + Configuration.Default, + TestImages.Heif.Av1OfficialCdfUpdateSequence, + TestImages.Heif.Av1OfficialCdfUpdateSequenceNativeReference, + OfficialCdfUpdateFixtureFrameCount); + + Assert.Equal(TileCdfUpdateCoverage | FrameEndCdfUpdateCoverage, coverage & 3); + } + + /// + /// Validates that the official temporal motion-field fixture enables projected reference motion vectors. + /// + private static void ValidateOfficialMotionFieldFixture() + { + int coverage = ValidateCompactSequence( + Configuration.Default, + TestImages.Heif.Av1OfficialMotionFieldSequence, + TestImages.Heif.Av1OfficialMotionFieldSequenceNativeReference, + OfficialMotionFieldFixtureFrameCount); + + Assert.NotEqual(0, coverage & ReferenceFrameMotionVectorCoverage); + } + + /// + /// Verifies the default operating point of an official two-spatial-layer sequence against exact reference + /// native output under normal and scalar dispatch. + /// + [Fact] + public void DecodeTwoSpatialLayersMatchesReference() + => FeatureTestRunner.RunWithHwIntrinsicsFeature( + ValidateOfficialTwoSpatialLayerFixture, + ReconstructionConfigurations); + + /// + /// Verifies the official two-spatial-layer sequence through constrained tracked allocation. + /// + [Fact] + [ValidateDisposedMemoryAllocations] + public void DecodeTwoSpatialLayersWithConstrainedAllocator() + { + TestMemoryAllocator allocator = new() { BufferCapacityInBytes = 8_192 }; + allocator.EnableNonThreadSafeLogging(); + Configuration configuration = Configuration.Default.Clone(); + configuration.MemoryAllocator = allocator; + + ValidateCompactSequence( + configuration, + TestImages.Heif.Av1OfficialTwoSpatialLayerSequence, + TestImages.Heif.Av1OfficialTwoSpatialLayerSequenceNativeReference, + OfficialTwoSpatialLayerFixtureFrameCount, + OfficialTwoSpatialLayerFixtureWidth, + OfficialTwoSpatialLayerFixtureHeight); + + Assert.Equal(allocator.AllocationLog.Count, allocator.ReturnLog.Count); + Assert.All( + allocator.AllocationLog, + allocation => Assert.Single( + allocator.ReturnLog, + returned => returned.AllocationId == allocation.AllocationId)); + } + + /// + /// Decodes the default operating point of the official two-spatial-layer sequence. + /// + private static void ValidateOfficialTwoSpatialLayerFixture() + => ValidateCompactSequence( + Configuration.Default, + TestImages.Heif.Av1OfficialTwoSpatialLayerSequence, + TestImages.Heif.Av1OfficialTwoSpatialLayerSequenceNativeReference, + OfficialTwoSpatialLayerFixtureFrameCount, + OfficialTwoSpatialLayerFixtureWidth, + OfficialTwoSpatialLayerFixtureHeight); + + /// + /// Verifies the default operating point of an official two-temporal-layer sequence against exact reference + /// native output under normal and scalar dispatch. + /// + [Fact] + public void DecodeTwoTemporalLayersMatchesReference() + => FeatureTestRunner.RunWithHwIntrinsicsFeature( + ValidateOfficialTwoTemporalLayerFixture, + ReconstructionConfigurations); + + /// + /// Verifies the official two-temporal-layer sequence through constrained tracked allocation. + /// + [Fact] + [ValidateDisposedMemoryAllocations] + public void DecodeTwoTemporalLayersWithConstrainedAllocator() + { + TestMemoryAllocator allocator = new() { BufferCapacityInBytes = 8_192 }; + allocator.EnableNonThreadSafeLogging(); + Configuration configuration = Configuration.Default.Clone(); + configuration.MemoryAllocator = allocator; + + ValidateCompactSequence( + configuration, + TestImages.Heif.Av1OfficialTwoTemporalLayerSequence, + TestImages.Heif.Av1OfficialTwoTemporalLayerSequenceNativeReference, + OfficialTwoTemporalLayerFixtureFrameCount, + OfficialTwoTemporalLayerFixtureWidth, + OfficialTwoTemporalLayerFixtureHeight); + + Assert.Equal(allocator.AllocationLog.Count, allocator.ReturnLog.Count); + Assert.All( + allocator.AllocationLog, + allocation => Assert.Single( + allocator.ReturnLog, + returned => returned.AllocationId == allocation.AllocationId)); + } + + /// + /// Decodes the default operating point of the official two-temporal-layer sequence. + /// + private static void ValidateOfficialTwoTemporalLayerFixture() + => ValidateCompactSequence( + Configuration.Default, + TestImages.Heif.Av1OfficialTwoTemporalLayerSequence, + TestImages.Heif.Av1OfficialTwoTemporalLayerSequenceNativeReference, + OfficialTwoTemporalLayerFixtureFrameCount, + OfficialTwoTemporalLayerFixtureWidth, + OfficialTwoTemporalLayerFixtureHeight); + + /// + /// Verifies the default operating point of an official spatial-and-temporal-layer sequence against exact + /// reference native output under normal and scalar dispatch. + /// + [Fact] + public void DecodeSpatialTemporalLayersMatchesReference() + => FeatureTestRunner.RunWithHwIntrinsicsFeature( + ValidateOfficialSpatialTemporalLayerFixture, + ReconstructionConfigurations); + + /// + /// Verifies the official spatial-and-temporal-layer sequence through constrained tracked allocation. + /// + [Fact] + [ValidateDisposedMemoryAllocations] + public void DecodeSpatialTemporalLayersWithConstrainedAllocator() + { + TestMemoryAllocator allocator = new() { BufferCapacityInBytes = 8_192 }; + allocator.EnableNonThreadSafeLogging(); + Configuration configuration = Configuration.Default.Clone(); + configuration.MemoryAllocator = allocator; + + ValidateCompactSequence( + configuration, + TestImages.Heif.Av1OfficialSpatialTemporalLayerSequence, + TestImages.Heif.Av1OfficialSpatialTemporalLayerSequenceNativeReference, + OfficialSpatialTemporalLayerFixtureFrameCount, + OfficialSpatialTemporalLayerFixtureWidth, + OfficialSpatialTemporalLayerFixtureHeight); + + Assert.Equal(allocator.AllocationLog.Count, allocator.ReturnLog.Count); + Assert.All( + allocator.AllocationLog, + allocation => Assert.Single( + allocator.ReturnLog, + returned => returned.AllocationId == allocation.AllocationId)); + } + + /// + /// Decodes the default operating point of the official spatial-and-temporal-layer sequence. + /// + private static void ValidateOfficialSpatialTemporalLayerFixture() + => ValidateCompactSequence( + Configuration.Default, + TestImages.Heif.Av1OfficialSpatialTemporalLayerSequence, + TestImages.Heif.Av1OfficialSpatialTemporalLayerSequenceNativeReference, + OfficialSpatialTemporalLayerFixtureFrameCount, + OfficialSpatialTemporalLayerFixtureWidth, + OfficialSpatialTemporalLayerFixtureHeight); + + /// + /// Verifies active film-grain presentation and dependent-frame reconstruction against exact reference native + /// output under normal and scalar dispatch. + /// + [Fact] + public void DecodeFilmGrainSequenceMatchesReference() + => FeatureTestRunner.RunWithHwIntrinsicsFeature( + ValidateOfficialFilmGrainFixture, + ReconstructionConfigurations); + + /// + /// Verifies the official film-grain sequence through constrained tracked allocation. + /// + [Fact] + [ValidateDisposedMemoryAllocations] + public void DecodeFilmGrainWithConstrainedAllocator() + { + TestMemoryAllocator allocator = new() { BufferCapacityInBytes = 2_048 }; + allocator.EnableNonThreadSafeLogging(); + Configuration configuration = Configuration.Default.Clone(); + configuration.MemoryAllocator = allocator; + + int coverage = ValidateCompactSequence( + configuration, + TestImages.Heif.Av1OfficialFilmGrainSequence, + TestImages.Heif.Av1OfficialFilmGrainSequenceNativeReference, + OfficialFilmGrainFixtureFrameCount); + + Assert.NotEqual(0, coverage & FilmGrainCoverage); + Assert.Equal(allocator.AllocationLog.Count, allocator.ReturnLog.Count); + Assert.All( + allocator.AllocationLog, + allocation => Assert.Single( + allocator.ReturnLog, + returned => returned.AllocationId == allocation.AllocationId)); + } + + /// + /// Decodes the official film-grain sequence and verifies that synthesis is active. + /// + private static void ValidateOfficialFilmGrainFixture() + { + int coverage = ValidateCompactSequence( + Configuration.Default, + TestImages.Heif.Av1OfficialFilmGrainSequence, + TestImages.Heif.Av1OfficialFilmGrainSequenceNativeReference, + OfficialFilmGrainFixtureFrameCount); + + Assert.NotEqual(0, coverage & FilmGrainCoverage); + } + + /// + /// Verifies the official ten-bit film-grain sequence against exact reference native output. + /// + [Fact] + public void DecodeTenBitFilmGrainSequenceMatchesReference() + => FeatureTestRunner.RunWithHwIntrinsicsFeature( + ValidateOfficialTenBitFilmGrainFixture, + ReconstructionConfigurations); + + /// + /// Decodes the official ten-bit film-grain sequence and verifies that synthesis is active. + /// + private static void ValidateOfficialTenBitFilmGrainFixture() + { + int coverage = ValidateCompactSequence( + Configuration.Default, + TestImages.Heif.Av1OfficialTenBitFilmGrainSequence, + TestImages.Heif.Av1OfficialTenBitFilmGrainSequenceNativeReference, + OfficialFilmGrainFixtureFrameCount, + OfficialMotionVectorFixtureWidth, + OfficialMotionVectorFixtureHeight, + Av1ColorFormat.Yuv420, + Av1BitDepth.TenBit); + + Assert.NotEqual(0, coverage & FilmGrainCoverage); + } + + /// + /// Verifies the official eight-bit monochrome sequence against exact reference native output under normal and + /// scalar dispatch. + /// + [Fact] + public void DecodeMonochromeSequenceMatchesReference() + => FeatureTestRunner.RunWithHwIntrinsicsFeature( + ValidateOfficialMonochromeFixture, + ReconstructionConfigurations); + + /// + /// Verifies the official eight-bit monochrome sequence through constrained tracked allocation. + /// + [Fact] + [ValidateDisposedMemoryAllocations] + public void DecodeMonochromeWithConstrainedAllocator() + { + TestMemoryAllocator allocator = new() { BufferCapacityInBytes = 2_048 }; + allocator.EnableNonThreadSafeLogging(); + Configuration configuration = Configuration.Default.Clone(); + configuration.MemoryAllocator = allocator; + + ValidateCompactSequence( + configuration, + TestImages.Heif.Av1OfficialMonochromeSequence, + TestImages.Heif.Av1OfficialMonochromeSequenceNativeReference, + OfficialMonochromeFixtureFrameCount, + OfficialMonochromeFixtureWidth, + OfficialMonochromeFixtureHeight, + Av1ColorFormat.Yuv400); + + Assert.Equal(allocator.AllocationLog.Count, allocator.ReturnLog.Count); + Assert.All( + allocator.AllocationLog, + allocation => Assert.Single( + allocator.ReturnLog, + returned => returned.AllocationId == allocation.AllocationId)); + } + + /// + /// Decodes the official eight-bit monochrome sequence. + /// + private static void ValidateOfficialMonochromeFixture() + => ValidateCompactSequence( + Configuration.Default, + TestImages.Heif.Av1OfficialMonochromeSequence, + TestImages.Heif.Av1OfficialMonochromeSequenceNativeReference, + OfficialMonochromeFixtureFrameCount, + OfficialMonochromeFixtureWidth, + OfficialMonochromeFixtureHeight, + Av1ColorFormat.Yuv400); + + /// + /// Verifies the official ten-bit monochrome sequence against exact reference native output. + /// + [Fact] + public void DecodeTenBitMonochromeSequenceMatchesReference() + => FeatureTestRunner.RunWithHwIntrinsicsFeature( + ValidateOfficialTenBitMonochromeFixture, + ReconstructionConfigurations); + + /// + /// Decodes the official ten-bit monochrome sequence. + /// + private static void ValidateOfficialTenBitMonochromeFixture() + => ValidateCompactSequence( + Configuration.Default, + TestImages.Heif.Av1OfficialTenBitMonochromeSequence, + TestImages.Heif.Av1OfficialTenBitMonochromeSequenceNativeReference, + OfficialMonochromeFixtureFrameCount, + OfficialMonochromeFixtureWidth, + OfficialMonochromeFixtureHeight, + Av1ColorFormat.Yuv400, + Av1BitDepth.TenBit); + + /// + /// Verifies both official ten-bit sequences through constrained tracked allocation. + /// + [Fact] + [ValidateDisposedMemoryAllocations] + public void DecodeTenBitWithConstrainedAllocator() + { + TestMemoryAllocator allocator = new() { BufferCapacityInBytes = 2_048 }; + allocator.EnableNonThreadSafeLogging(); + Configuration configuration = Configuration.Default.Clone(); + configuration.MemoryAllocator = allocator; + + int filmGrainCoverage = ValidateCompactSequence( + configuration, + TestImages.Heif.Av1OfficialTenBitFilmGrainSequence, + TestImages.Heif.Av1OfficialTenBitFilmGrainSequenceNativeReference, + OfficialFilmGrainFixtureFrameCount, + OfficialMotionVectorFixtureWidth, + OfficialMotionVectorFixtureHeight, + Av1ColorFormat.Yuv420, + Av1BitDepth.TenBit); + + ValidateCompactSequence( + configuration, + TestImages.Heif.Av1OfficialTenBitMonochromeSequence, + TestImages.Heif.Av1OfficialTenBitMonochromeSequenceNativeReference, + OfficialMonochromeFixtureFrameCount, + OfficialMonochromeFixtureWidth, + OfficialMonochromeFixtureHeight, + Av1ColorFormat.Yuv400, + Av1BitDepth.TenBit); + + Assert.NotEqual(0, filmGrainCoverage & FilmGrainCoverage); + Assert.Equal(allocator.AllocationLog.Count, allocator.ReturnLog.Count); + Assert.All( + allocator.AllocationLog, + allocation => Assert.Single( + allocator.ReturnLog, + returned => returned.AllocationId == allocation.AllocationId)); + } + + /// + /// Verifies the official eight-bit quantizer boundaries against exact reference native output. + /// + [Fact] + public void DecodeEightBitQuantizerBoundariesMatchReference() + => FeatureTestRunner.RunWithHwIntrinsicsFeature( + ValidateOfficialEightBitQuantizerBoundaryFixtures, + ReconstructionConfigurations); + + /// + /// Decodes the official eight-bit quantizer boundaries. + /// + private static void ValidateOfficialEightBitQuantizerBoundaryFixtures() + => ValidateOfficialEightBitQuantizerBoundaryFixturesWithConfiguration(Configuration.Default); + + /// + /// Verifies the official ten-bit quantizer boundaries against exact reference native output. + /// + [Fact] + public void DecodeTenBitQuantizerBoundariesMatchReference() + => FeatureTestRunner.RunWithHwIntrinsicsFeature( + ValidateOfficialTenBitQuantizerBoundaryFixtures, + ReconstructionConfigurations); + + /// + /// Decodes the official ten-bit quantizer boundaries. + /// + private static void ValidateOfficialTenBitQuantizerBoundaryFixtures() + => ValidateOfficialTenBitQuantizerBoundaryFixturesWithConfiguration(Configuration.Default); + + /// + /// Verifies the official quantizer boundaries through constrained tracked allocation. + /// + [Fact] + [ValidateDisposedMemoryAllocations] + public void DecodeQuantizerBoundariesWithConstrainedAllocator() + { + TestMemoryAllocator allocator = new() { BufferCapacityInBytes = 2_560 }; + allocator.EnableNonThreadSafeLogging(); + Configuration configuration = Configuration.Default.Clone(); + configuration.MemoryAllocator = allocator; + + ValidateOfficialEightBitQuantizerBoundaryFixturesWithConfiguration(configuration); + ValidateOfficialTenBitQuantizerBoundaryFixturesWithConfiguration(configuration); + + // Each decoded frame owns one maximum-sized coefficient-context scratch rent. An allocation per transform + // would produce hundreds of identically typed smaller rents for these deliberately dense fixtures. + int coefficientScratchLength = + ((Av1Constants.MaxTransformSize / 2) + Av1Constants.TransformPadHorizontal) * + (Av1Constants.TransformPadTop + (Av1Constants.MaxTransformSize / 2) + Av1Constants.TransformPadBottom); + + int coefficientScratchAllocations = allocator.AllocationLog.Count( + allocation => allocation.ElementType == typeof(byte) && allocation.Length == coefficientScratchLength); + + Assert.Equal(4 * OfficialQuantizerFixtureFrameCount, coefficientScratchAllocations); + Assert.Equal(allocator.AllocationLog.Count, allocator.ReturnLog.Count); + Assert.All( + allocator.AllocationLog, + allocation => Assert.Single( + allocator.ReturnLog, + returned => returned.AllocationId == allocation.AllocationId)); + } + + /// + /// Decodes both retained official eight-bit quantizer-boundary fixtures. + /// + private static void ValidateOfficialEightBitQuantizerBoundaryFixturesWithConfiguration(Configuration configuration) + { + ValidateCompactSequence( + configuration, + TestImages.Heif.Av1OfficialEightBitMinimumQuantizerSequence, + TestImages.Heif.Av1OfficialEightBitMinimumQuantizerSequenceNativeReference, + OfficialQuantizerFixtureFrameCount, + OfficialEightBitQuantizerFixtureWidth, + OfficialEightBitQuantizerFixtureHeight); + + ValidateCompactSequence( + configuration, + TestImages.Heif.Av1OfficialEightBitMaximumQuantizerSequence, + TestImages.Heif.Av1OfficialEightBitMaximumQuantizerSequenceNativeReference, + OfficialQuantizerFixtureFrameCount, + OfficialEightBitQuantizerFixtureWidth, + OfficialEightBitQuantizerFixtureHeight); + } + + /// + /// Decodes both retained official ten-bit quantizer-boundary fixtures. + /// + private static void ValidateOfficialTenBitQuantizerBoundaryFixturesWithConfiguration(Configuration configuration) + { + ValidateCompactSequence( + configuration, + TestImages.Heif.Av1OfficialTenBitMinimumQuantizerSequence, + TestImages.Heif.Av1OfficialTenBitMinimumQuantizerSequenceNativeReference, + OfficialQuantizerFixtureFrameCount, + OfficialTenBitQuantizerFixtureWidth, + OfficialTenBitQuantizerFixtureHeight, + Av1ColorFormat.Yuv420, + Av1BitDepth.TenBit, + "60:1"); + + ValidateCompactSequence( + configuration, + TestImages.Heif.Av1OfficialTenBitMaximumQuantizerSequence, + TestImages.Heif.Av1OfficialTenBitMaximumQuantizerSequenceNativeReference, + OfficialQuantizerFixtureFrameCount, + OfficialTenBitQuantizerFixtureWidth, + OfficialTenBitQuantizerFixtureHeight, + Av1ColorFormat.Yuv420, + Av1BitDepth.TenBit, + "60:1"); + } + + /// + /// Verifies all four corners of the official frame-size matrix against exact reference native output. + /// + [Fact] + public void DecodeFrameSizeCornersMatchReference() + => FeatureTestRunner.RunWithHwIntrinsicsFeature( + ValidateOfficialFrameSizeCornerFixtures, + ReconstructionConfigurations); + + /// + /// Decodes all four retained frame-size corners. + /// + private static void ValidateOfficialFrameSizeCornerFixtures() + => ValidateOfficialFrameSizeCornerFixturesWithConfiguration(Configuration.Default); + + /// + /// Verifies all four frame-size corners through constrained tracked allocation. + /// + [Fact] + [ValidateDisposedMemoryAllocations] + public void DecodeFrameSizeCornersWithConstrainedAllocator() + { + TestMemoryAllocator allocator = new() { BufferCapacityInBytes = 1_024 }; + allocator.EnableNonThreadSafeLogging(); + Configuration configuration = Configuration.Default.Clone(); + configuration.MemoryAllocator = allocator; + + ValidateOfficialFrameSizeCornerFixturesWithConfiguration(configuration); + + Assert.Equal(allocator.AllocationLog.Count, allocator.ReturnLog.Count); + Assert.All( + allocator.AllocationLog, + allocation => Assert.Single( + allocator.ReturnLog, + returned => returned.AllocationId == allocation.AllocationId)); + } + + /// + /// Decodes every retained official frame-size fixture and compares every native sample. + /// + private static void ValidateOfficialFrameSizeCornerFixturesWithConfiguration(Configuration configuration) + { + ValidateCompactSequence( + configuration, + TestImages.Heif.Av1OfficialMinimumFrameSizeSequence, + TestImages.Heif.Av1OfficialMinimumFrameSizeSequenceNativeReference, + OfficialFrameSizeFixtureFrameCount, + OfficialFrameSizeFixtureMinimumDimension, + OfficialFrameSizeFixtureMinimumDimension); + + ValidateCompactSequence( + configuration, + TestImages.Heif.Av1OfficialMinimumWidthMaximumHeightSequence, + TestImages.Heif.Av1OfficialMinimumWidthMaximumHeightSequenceNativeReference, + OfficialFrameSizeFixtureFrameCount, + OfficialFrameSizeFixtureMinimumDimension, + OfficialFrameSizeFixtureMaximumDimension); + + ValidateCompactSequence( + configuration, + TestImages.Heif.Av1OfficialMaximumWidthMinimumHeightSequence, + TestImages.Heif.Av1OfficialMaximumWidthMinimumHeightSequenceNativeReference, + OfficialFrameSizeFixtureFrameCount, + OfficialFrameSizeFixtureMaximumDimension, + OfficialFrameSizeFixtureMinimumDimension); + + ValidateCompactSequence( + configuration, + TestImages.Heif.Av1OfficialMaximumFrameSizeSequence, + TestImages.Heif.Av1OfficialMaximumFrameSizeSequenceNativeReference, + OfficialFrameSizeFixtureFrameCount, + OfficialFrameSizeFixtureMaximumDimension, + OfficialFrameSizeFixtureMaximumDimension); + } + + /// + /// Verifies modulo frame identifiers across a key/inter sequence with exact native output under every dispatch. + /// + [Fact] + public void DecodeFrameIdentifiersMatchReference() + => FeatureTestRunner.RunWithHwIntrinsicsFeature( + ValidateFrameIdentifierFixture, + ReconstructionConfigurations); + + /// + /// Decodes the frame-identifier fixture and requires a validated identifier transition on its inter frame. + /// + private static void ValidateFrameIdentifierFixture() + { + int coverage = ValidateCompactSequence( + Configuration.Default, + TestImages.Heif.Av1FrameIdentifierSequence, + TestImages.Heif.Av1FrameIdentifierSequenceNativeReference, + OfficialFrameSizeFixtureFrameCount, + OfficialFrameSizeFixtureMinimumDimension, + OfficialFrameSizeFixtureMinimumDimension); + + Assert.Equal(FrameIdentifierCoverage, coverage & FrameIdentifierCoverage); + } + + /// + /// Decodes one compact IVF sequence, compares every native sample, and returns its active frame-state + /// coverage mask. + /// + private static int ValidateCompactSequence( + Configuration configuration, + string fixturePath, + string nativeReferencePath, + int expectedFrameCount, + int expectedWidth = OfficialMotionVectorFixtureWidth, + int expectedHeight = OfficialMotionVectorFixtureHeight, + Av1ColorFormat expectedColorFormat = Av1ColorFormat.Yuv420, + Av1BitDepth expectedBitDepth = Av1BitDepth.EightBit, + string expectedFrameRate = "30:1") + { + byte[] ivf = TestFile.Create(fixturePath).Bytes; + byte[] nativeReference = TestFile.Create(nativeReferencePath).Bytes; + bool hasY4mHeaders = nativeReference.AsSpan().StartsWith("YUV4MPEG2 "u8); + int bitDepth = expectedBitDepth switch + { + Av1BitDepth.EightBit => 8, + Av1BitDepth.TenBit => 10, + Av1BitDepth.TwelveBit => 12, + _ => throw new InvalidOperationException("The compact official sequence oracle requires a valid AV1 bit depth.") + }; + + string y4mColorSpace = expectedColorFormat switch + { + Av1ColorFormat.Yuv400 => expectedBitDepth == Av1BitDepth.EightBit + ? "Cmono" + : $"Cmono{bitDepth}", + Av1ColorFormat.Yuv420 => expectedBitDepth == Av1BitDepth.EightBit + ? "C420jpeg" + : $"C420p{bitDepth} XYSCSS=420P{bitDepth}", + _ => throw new InvalidOperationException("The compact official sequence oracle supports YUV400 and YUV420 references.") + }; + + ReadOnlySpan y4mFileHeader = hasY4mHeaders + ? Encoding.ASCII.GetBytes( + $"YUV4MPEG2 W{expectedWidth} H{expectedHeight} F{expectedFrameRate} Ip {y4mColorSpace}\n") + : []; + + ReadOnlySpan y4mFrameHeader = "FRAME\n"u8; + + Assert.True(ivf.AsSpan(0, 4).SequenceEqual("DKIF"u8)); + Assert.Equal(0, BinaryPrimitives.ReadUInt16LittleEndian(ivf.AsSpan(4, 2))); + Assert.Equal(32, BinaryPrimitives.ReadUInt16LittleEndian(ivf.AsSpan(6, 2))); + Assert.True(ivf.AsSpan(8, 4).SequenceEqual("AV01"u8)); + Assert.Equal(expectedWidth, BinaryPrimitives.ReadUInt16LittleEndian(ivf.AsSpan(12, 2))); + Assert.Equal(expectedHeight, BinaryPrimitives.ReadUInt16LittleEndian(ivf.AsSpan(14, 2))); + Assert.Equal( + expectedFrameCount, + checked((int)BinaryPrimitives.ReadUInt32LittleEndian(ivf.AsSpan(24, 4)))); + + Assert.True(nativeReference.AsSpan().StartsWith(y4mFileHeader)); + + int ivfOffset = 32; + int nativeOffset = y4mFileHeader.Length; + int nativeSampleCount = expectedColorFormat == Av1ColorFormat.Yuv400 + ? expectedWidth * expectedHeight + : (expectedWidth * expectedHeight) + + (2 * GetSubsampledSize(expectedWidth, 1) * GetSubsampledSize(expectedHeight, 1)); + + int nativeFrameLength = nativeSampleCount * (expectedBitDepth == Av1BitDepth.EightBit ? 1 : sizeof(ushort)); + + int coverage = 0; + uint previousFrameIdentifier = 0; + bool hasPreviousFrameIdentifier = false; + using Av1Decoder decoder = new(configuration); + for (int frameIndex = 0; frameIndex < expectedFrameCount; frameIndex++) + { + int payloadLength = checked((int)BinaryPrimitives.ReadUInt32LittleEndian(ivf.AsSpan(ivfOffset, 4))); + ivfOffset += 12; + using ImageFrame frame = decoder.DecodeSequenceFrame( + ivf.AsSpan(ivfOffset, payloadLength), + null, + null); + + ivfOffset += payloadLength; + Assert.Equal(expectedWidth, frame.Width); + Assert.Equal(expectedHeight, frame.Height); + if (hasY4mHeaders) + { + Assert.True(nativeReference.AsSpan(nativeOffset).StartsWith(y4mFrameHeader)); + nativeOffset += y4mFrameHeader.Length; + } + + Av1FrameBuffer frameBuffer = Assert.IsType>(decoder.FrameBuffer); + Assert.Equal(expectedBitDepth, frameBuffer.BitDepth); + Assert.Equal(expectedColorFormat, frameBuffer.ColorFormat); + AssertNativePlanesEqual( + decoder, + frameBuffer, + nativeReference.AsSpan(nativeOffset, nativeFrameLength), + frameIndex); + + nativeOffset += nativeFrameLength; + + ObuFrameHeader frameHeader = Assert.IsType(decoder.FrameHeader); + coverage |= frameHeader.DisableCdfUpdate ? 0 : TileCdfUpdateCoverage; + coverage |= frameHeader.DisableFrameEndUpdateCdf ? 0 : FrameEndCdfUpdateCoverage; + coverage |= frameHeader.UseReferenceFrameMotionVectors ? ReferenceFrameMotionVectorCoverage : 0; + coverage |= frameHeader.FilmGrainParameters.ApplyGrain ? FilmGrainCoverage : 0; + + ObuSequenceHeader sequenceHeader = Assert.IsType(decoder.SequenceHeader); + if (sequenceHeader.IsFrameIdNumbersPresent) + { + if (hasPreviousFrameIdentifier && !frameHeader.IsIntra && + previousFrameIdentifier != frameHeader.CurrentFrameId) + { + coverage |= FrameIdentifierCoverage; + } + + previousFrameIdentifier = frameHeader.CurrentFrameId; + hasPreviousFrameIdentifier = true; + } + } + + Assert.Equal(ivf.Length, ivfOffset); + Assert.Equal(nativeReference.Length, nativeOffset); + return coverage; + } + + /// + /// Verifies every ordinary inter mode, motion mode, and switchable dual-filter pair against the official + /// reference motion-vector conformance sequence and its exact native output. + /// + [Fact] + public void DecodeMotionVectorSequenceMatchesReference() + => FeatureTestRunner.RunWithHwIntrinsicsFeature( + ValidateOfficialMotionVectorFixtureWithDefaultConfiguration, + ReconstructionConfigurations); + + /// + /// Verifies the official motion-vector conformance sequence through constrained tracked allocation. + /// + [Fact] + [ValidateDisposedMemoryAllocations] + public void DecodeMotionVectorsWithConstrainedAllocator() + { + TestMemoryAllocator allocator = new() { BufferCapacityInBytes = 2_048 }; + allocator.EnableNonThreadSafeLogging(); + Configuration configuration = Configuration.Default.Clone(); + configuration.MemoryAllocator = allocator; + + ValidateOfficialMotionVectorFixture(configuration); + + Assert.Contains(allocator.AllocationLog, request => request.ElementType.Name == "RetainedMotionFieldEntry"); + Assert.Contains(allocator.AllocationLog, request => request.ElementType.Name == "TemporalMotionFieldEntry"); + Assert.Equal(allocator.AllocationLog.Count, allocator.ReturnLog.Count); + Assert.All( + allocator.AllocationLog, + allocation => Assert.Single( + allocator.ReturnLog, + returned => returned.AllocationId == allocation.AllocationId)); + } + + /// + /// Runs the official motion-vector fixture with the default decoder configuration. + /// + private static void ValidateOfficialMotionVectorFixtureWithDefaultConfiguration() + => ValidateOfficialMotionVectorFixture(Configuration.Default); + + /// + /// Decodes every IVF sample in one retained session, compares each shown frame exactly, and records the + /// syntax selections that make the vector authoritative for ordinary inter-mode and filter coverage. + /// + private static void ValidateOfficialMotionVectorFixture(Configuration configuration) + { + byte[] ivf = TestFile.Create(TestImages.Heif.Av1OfficialMotionVectorSequence).Bytes; + byte[] nativeReference = TestFile.Create(TestImages.Heif.Av1OfficialMotionVectorSequenceNativeReference).Bytes; + ReadOnlySpan y4mFileHeader = "YUV4MPEG2 W352 H288 F30:1 Ip C420jpeg\n"u8; + ReadOnlySpan y4mFrameHeader = "FRAME\n"u8; + + Assert.True(ivf.AsSpan(0, 4).SequenceEqual("DKIF"u8)); + Assert.Equal(0, BinaryPrimitives.ReadUInt16LittleEndian(ivf.AsSpan(4, 2))); + Assert.Equal(32, BinaryPrimitives.ReadUInt16LittleEndian(ivf.AsSpan(6, 2))); + Assert.True(ivf.AsSpan(8, 4).SequenceEqual("AV01"u8)); + Assert.Equal(OfficialMotionVectorFixtureWidth, BinaryPrimitives.ReadUInt16LittleEndian(ivf.AsSpan(12, 2))); + Assert.Equal(OfficialMotionVectorFixtureHeight, BinaryPrimitives.ReadUInt16LittleEndian(ivf.AsSpan(14, 2))); + Assert.Equal( + OfficialMotionVectorFixtureFrameCount, + checked((int)BinaryPrimitives.ReadUInt32LittleEndian(ivf.AsSpan(24, 4)))); + + Assert.True(nativeReference.AsSpan().StartsWith(y4mFileHeader)); + + int ivfOffset = 32; + int nativeOffset = y4mFileHeader.Length; + int nativeFrameLength = + (OfficialMotionVectorFixtureWidth * OfficialMotionVectorFixtureHeight) + + (2 * (OfficialMotionVectorFixtureWidth >> 1) * (OfficialMotionVectorFixtureHeight >> 1)); + + int interModeCoverage = 0; + int motionModeCoverage = 0; + int switchableFilterPairCoverage = 0; + using Av1Decoder decoder = new(configuration); + for (int frameIndex = 0; frameIndex < OfficialMotionVectorFixtureFrameCount; frameIndex++) + { + int payloadLength = checked((int)BinaryPrimitives.ReadUInt32LittleEndian(ivf.AsSpan(ivfOffset, 4))); + ivfOffset += 12; + ImageFrame decodedFrame = decoder.DecodeSequenceFrame( + ivf.AsSpan(ivfOffset, payloadLength), + null, + null); + + using ImageFrame frame = decodedFrame; + + ivfOffset += payloadLength; + Assert.Equal(OfficialMotionVectorFixtureWidth, frame.Width); + Assert.Equal(OfficialMotionVectorFixtureHeight, frame.Height); + Assert.True(nativeReference.AsSpan(nativeOffset).StartsWith(y4mFrameHeader)); + nativeOffset += y4mFrameHeader.Length; + + Av1FrameBuffer frameBuffer = Assert.IsType>(decoder.FrameBuffer); + Assert.Equal(OfficialMotionVectorFixtureWidth, frameBuffer.Width); + Assert.Equal(OfficialMotionVectorFixtureHeight, frameBuffer.Height); + Assert.Equal(Av1BitDepth.EightBit, frameBuffer.BitDepth); + Assert.Equal(Av1ColorFormat.Yuv420, frameBuffer.ColorFormat); + AssertNativePlanesEqual( + decoder, + frameBuffer, + nativeReference.AsSpan(nativeOffset, nativeFrameLength)); + + nativeOffset += nativeFrameLength; + + Av1FrameInfo frameInfo = decoder.FrameInfo; + if (frameInfo is null) + { + // show_existing_frame contributes no new mode or interpolation-filter syntax. + continue; + } + + ObuSequenceHeader sequenceHeader = Assert.IsType(decoder.SequenceHeader); + int superblockColumnCount = Av1Math.AlignPowerOf2(sequenceHeader.MaxFrameWidth, sequenceHeader.SuperblockSizeLog2) + >> sequenceHeader.SuperblockSizeLog2; + int superblockRowCount = Av1Math.AlignPowerOf2(sequenceHeader.MaxFrameHeight, sequenceHeader.SuperblockSizeLog2) + >> sequenceHeader.SuperblockSizeLog2; + + for (int superblockRow = 0; superblockRow < superblockRowCount; superblockRow++) + { + for (int superblockColumn = 0; superblockColumn < superblockColumnCount; superblockColumn++) + { + Av1SuperblockInfo superblockInfo = frameInfo.GetSuperblock(new Point(superblockColumn, superblockRow)); + foreach (Av1BlockModeInfo modeInfo in superblockInfo.GetModeInfos()) + { + if (modeInfo.YMode is < Av1PredictionMode.InterModeStart or >= Av1PredictionMode.InterModeEnd) + { + continue; + } + + interModeCoverage |= 1 << ((int)modeInfo.YMode - (int)Av1PredictionMode.InterModeStart); + motionModeCoverage |= 1 << (int)modeInfo.MotionMode; + int verticalFilter = (int)modeInfo.InterpolationFilters[0]; + int horizontalFilter = (int)modeInfo.InterpolationFilters[1]; + switchableFilterPairCoverage |= 1 << ((verticalFilter * 3) + horizontalFilter); + } + } + } + } + + Assert.Equal(ivf.Length, ivfOffset); + Assert.Equal(nativeReference.Length, nativeOffset); + Assert.Equal(RequiredInterModeCoverage, interModeCoverage); + Assert.Equal(RequiredMotionModeCoverage, motionModeCoverage); + Assert.Equal(RequiredSwitchableFilterPairCoverage, switchableFilterPairCoverage); + } + + /// + /// Verifies one complete inter-prediction sequence with a separately tracked constrained allocator. + /// + private static void ValidateInterPredictionSequenceWithConstrainedAllocator( + string imagePath, + string nativeReferencePath, + int requiredCoverage, + int fixtureSize, + int visibleFrameCount) + { + TestMemoryAllocator allocator = new() { BufferCapacityInBytes = 1_024 }; + allocator.EnableNonThreadSafeLogging(); + Configuration configuration = Configuration.Default.Clone(); + configuration.MemoryAllocator = allocator; + + ValidateInterPredictionSequence( + configuration, + imagePath, + nativeReferencePath, + requiredCoverage, + fixtureSize, + visibleFrameCount); + + Assert.Contains(allocator.AllocationLog, request => request.ElementType.Name == "RetainedMotionFieldEntry"); + Assert.Contains(allocator.AllocationLog, request => request.ElementType.Name == "TemporalMotionFieldEntry"); + Assert.Equal(allocator.AllocationLog.Count, allocator.ReturnLog.Count); + Assert.All( + allocator.AllocationLog, + allocation => Assert.Single( + allocator.ReturnLog, + returned => returned.AllocationId == allocation.AllocationId)); + } + + /// + /// Runs one selectable compound fixture with exact native and final presentation comparisons. + /// + /// The serialized input provider and reference-output naming context. + private static void ValidateSelectableCompoundSequenceWithDefaultConfiguration(string providerDump) + { + TestImageProvider provider = + FeatureTestRunner.DeserializeForXunit>(providerDump); + + (string NativeReferencePath, int RequiredCoverage) expected = + provider.SourceFileOrDescription switch + { + TestImages.Heif.Av1DistanceWeightedCompoundSequenceAvif => + (TestImages.Heif.Av1DistanceWeightedCompoundSequenceNativeReference, DistanceWeightedCompoundCoverage), + TestImages.Heif.Av1WedgeCompoundSequenceAvif => + (TestImages.Heif.Av1WedgeCompoundSequenceNativeReference, WedgeCompoundCoverage | InvertedWedgeCompoundCoverage), + TestImages.Heif.Av1DifferenceWeightedCompoundSequenceAvif => + (TestImages.Heif.Av1DifferenceWeightedCompoundSequenceNativeReference, DifferenceWeightedCompoundCoverage | InvertedDifferenceWeightedCompoundCoverage), + TestImages.Heif.Av1InterIntraSequenceAvif => + (TestImages.Heif.Av1InterIntraSequenceNativeReference, SmoothInterIntraCoverage | WedgeInterIntraCoverage), + _ => throw new InvalidOperationException($"Unexpected selectable-compound fixture: {provider.SourceFileOrDescription}.") + }; + + ValidateInterPredictionSequence( + Configuration.Default, + provider.SourceFileOrDescription, + expected.NativeReferencePath, + expected.RequiredCoverage, + AverageCompoundFixtureSize, + AverageCompoundFixtureFrameCount); + + ValidateFinalSequencePresentation(providerDump); + } + + /// + /// Runs the OBMC sequence with exact final presentation comparison. + /// + /// The serialized input provider and reference-output naming context. + private static void ValidateObmcSequenceWithDefaultConfiguration(string providerDump) + { + ValidateInterPredictionSequence( + Configuration.Default, + TestImages.Heif.Av1ObmcSequenceAvif, + TestImages.Heif.Av1ObmcSequenceNativeReference, + ObmcCoverage, + AverageCompoundFixtureSize, + AverageCompoundFixtureFrameCount); + + ValidateFinalSequencePresentation(providerDump); + } + + /// + /// Runs the local warped-motion sequence with exact final presentation comparison. + /// + /// The serialized input provider and reference-output naming context. + private static void ValidateLocalWarpSequenceWithDefaultConfiguration(string providerDump) + { + ValidateInterPredictionSequence( + Configuration.Default, + TestImages.Heif.Av1LocalWarpSequenceAvif, + TestImages.Heif.Av1LocalWarpSequenceNativeReference, + LocalWarpCoverage, + 256, + 2); + + ValidateFinalSequencePresentation(providerDump); + } + + /// + /// Runs the non-translational global-motion sequence with exact final presentation comparison. + /// + /// The serialized input provider and reference-output naming context. + private static void ValidateGlobalWarpSequenceWithDefaultConfiguration(string providerDump) + { + ValidateInterPredictionSequence( + Configuration.Default, + TestImages.Heif.Av1GlobalWarpSequenceAvif, + TestImages.Heif.Av1GlobalWarpSequenceNativeReference, + GlobalWarpCoverage, + 256, + 2); + + ValidateFinalSequencePresentation(providerDump); + } + + /// + /// Decodes one complete retained-reference sequence and compares its final native samples exactly. + /// + private static void ValidateInterPredictionSequence( + Configuration configuration, + string imagePath, + string nativeReferencePath, + int requiredCoverage, + int fixtureSize, + int visibleFrameCount) + { + byte[] fileBytes = TestFile.Create(imagePath).Bytes; + byte[] referenceBytes = TestFile.Create(nativeReferencePath).Bytes; + string fileHeaderText = + $"YUV4MPEG2 W{fixtureSize} H{fixtureSize} F25:1 Ip A0:0 C444 XYSCSS=444 XCOLORRANGE=LIMITED\n"; + + ReadOnlySpan fileHeader = Encoding.ASCII.GetBytes(fileHeaderText); + + ReadOnlySpan frameHeader = "FRAME\n"u8; + + ReadOnlySpan nativeReference = referenceBytes; + Assert.True(nativeReference.StartsWith(fileHeader)); + nativeReference = nativeReference[fileHeader.Length..]; + Assert.True(nativeReference.StartsWith(frameHeader)); + nativeReference = nativeReference[frameHeader.Length..]; + Assert.Equal(fixtureSize * fixtureSize * 3, nativeReference.Length); + + HeifSequence sequence = ParseImageSequence(fileBytes); + HeifSequenceTrack track = sequence.ColorTrack; + int coverage = 0; + int decodedVisibleFrameCount = 0; + bool nativeCompared = false; + + using Av1Decoder decoder = new(configuration); + for (int sampleIndex = 0; sampleIndex < track.Samples.Length; sampleIndex++) + { + HeifSequenceSample sample = track.Samples[sampleIndex]; + Span sampleData = fileBytes.AsSpan((int)sample.Offset, sample.Length); + if (sample.IsHidden) + { + decoder.DecodeSequenceReference( + sampleData, + track.CicpProfile, + track.Av1CodecConfiguration); + + coverage |= GetInterPredictionCoverage(decoder); + continue; + } + + using ImageFrame frame = decoder.DecodeSequenceFrame( + sampleData, + track.CicpProfile, + track.Av1CodecConfiguration); + + Av1FrameBuffer frameBuffer = Assert.IsType>(decoder.FrameBuffer); + coverage |= GetInterPredictionCoverage(decoder); + + // Every inter-prediction branch retains the same row-addressed plane contract under constrained allocators. + Assert.Equal(1, frameBuffer.GetPlaneBuffer(Av1Plane.Y).FastMemoryGroup.Count); + Assert.Equal(1, frameBuffer.GetPlaneBuffer(Av1Plane.U).FastMemoryGroup.Count); + Assert.Equal(1, frameBuffer.GetPlaneBuffer(Av1Plane.V).FastMemoryGroup.Count); + + if (decodedVisibleFrameCount == visibleFrameCount - 1) + { + AssertNativePlanesEqual(decoder, frameBuffer, nativeReference); + nativeCompared = true; + } + + decodedVisibleFrameCount++; + } + + Assert.Equal(visibleFrameCount, decodedVisibleFrameCount); + Assert.Equal(requiredCoverage, coverage & requiredCoverage); + Assert.True(nativeCompared); + } + + /// + /// Collects the compound, inter-intra, OBMC, and warped modes completed in one bounded payload. + /// + private static int GetInterPredictionCoverage(Av1Decoder decoder) => + (int)decoder.DecodedInterPredictionFeatures; + + /// + /// Verifies lossless syntax, residual reconstruction, and exact native samples against the independent scalar reference for + /// independently encoded eight-, ten-, and twelve-bit AVIF images. + /// + [Fact] + public void DecodeLosslessNativeMatchesReference() + => FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidateLosslessFixtures, LosslessConfigurations); + + /// + /// Verifies exact presented pixels for independently encoded lossless eight-, ten-, and twelve-bit AVIF images + /// across the available vector widths and the scalar fallback. + /// + /// The AVIF input and matching reference-output naming context. + /// The expected public sample precision. + [Theory] + [WithFile(TestImages.Heif.Av1Lossless8BitAvif, PixelTypes.Rgba32, HeifBitDepth.Bit8)] + [WithFile(TestImages.Heif.Av1Lossless10BitAvif, PixelTypes.Rgba32, HeifBitDepth.Bit10)] + [WithFile(TestImages.Heif.Av1Lossless12BitAvif, PixelTypes.Rgba32, HeifBitDepth.Bit12)] + public void DecodeLosslessMatchesReference( + TestImageProvider provider, + HeifBitDepth bitDepth) + { + AssertPresentedMetadata(provider, LosslessFixtureWidth, LosslessFixtureHeight, bitDepth); + + FeatureTestRunner.RunWithHwIntrinsicsFeature( + ValidatePresentedFixture, + PresentationConfigurations, + provider); + } + + /// + /// Verifies active normative super-resolution, chroma-width rounding, replicated edges, and exact native samples + /// against the independent scalar reference for independently encoded eight-, ten-, and twelve-bit still-picture streams. + /// + [Fact] + public void DecodeSuperResolutionNativeMatchesReference() + => FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidateSuperResolutionFixtures, ReconstructionConfigurations); + + /// + /// Verifies exact presented pixels and public metadata for independently packaged eight-, ten-, and twelve-bit + /// active-super-resolution AVIF images across the available vector widths and the scalar fallback. + /// + /// The AVIF input and matching reference-output naming context. + /// The expected presented width. + /// The expected presented height. + /// The expected public sample precision. + [Theory] + [WithFile(TestImages.Heif.Av1SuperResolution8BitAvif, PixelTypes.Rgba32, 768, 512, HeifBitDepth.Bit8)] + [WithFile(TestImages.Heif.Av1SuperResolution10BitAvif, PixelTypes.Rgba32, 1024, 428, HeifBitDepth.Bit10)] + [WithFile(TestImages.Heif.Av1SuperResolution12BitAvif, PixelTypes.Rgba32, 1024, 428, HeifBitDepth.Bit12)] + public void DecodeSuperResolutionMatchesReference( + TestImageProvider provider, + int width, + int height, + HeifBitDepth bitDepth) + { + AssertPresentedMetadata(provider, width, height, bitDepth); + + FeatureTestRunner.RunWithHwIntrinsicsFeature( + ValidatePresentedFixture, + PresentationConfigurations, + provider); + } + + /// + /// Verifies active normative loop restoration and exact native samples against the independent scalar reference for independently + /// encoded eight-, ten-, and twelve-bit still-picture streams. + /// + [Fact] + public void DecodeLoopRestorationNativeMatchesReference() + => FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidateLoopRestorationFixtures, LoopRestorationConfigurations); + + /// + /// Verifies combined super-resolution and loop-restoration geometry for independently encoded 8-bit 4:2:0 content. + /// + [Fact] + public void DecodeRestorationSuperResolution8Bit420MatchesReference() + => FeatureTestRunner.RunWithHwIntrinsicsFeature( + ValidateLoopRestorationAndSuperResolution8Bit420, + LoopRestorationConfigurations); + + /// + /// Verifies combined super-resolution and loop-restoration geometry for independently encoded 10-bit 4:2:2 content. + /// + [Fact] + public void DecodeRestorationSuperRes10Bit422MatchesReference() + => FeatureTestRunner.RunWithHwIntrinsicsFeature( + ValidateLoopRestorationAndSuperResolution10Bit422, + LoopRestorationConfigurations); + + /// + /// Verifies combined super-resolution and loop-restoration geometry for independently encoded 12-bit 4:4:4 content. + /// + [Fact] + public void DecodeRestorationSuperRes12Bit444MatchesReference() + => FeatureTestRunner.RunWithHwIntrinsicsFeature( + ValidateLoopRestorationAndSuperResolution12Bit444, + LoopRestorationConfigurations); + + /// + /// Verifies exact presented pixels and public metadata for independently encoded eight-, ten-, and twelve-bit + /// active-restoration AVIF images across the available vector widths and the scalar fallback. + /// + /// The AVIF input and matching reference-output naming context. + /// The expected presented width. + /// The expected presented height. + /// The expected public sample precision. + [Theory] + [WithFile(TestImages.Heif.Av1Restoration8BitAvif, PixelTypes.Rgba32, 768, 512, HeifBitDepth.Bit8)] + [WithFile(TestImages.Heif.Av1Restoration10BitAvif, PixelTypes.Rgba32, 1024, 428, HeifBitDepth.Bit10)] + [WithFile(TestImages.Heif.Av1Restoration12BitAvif, PixelTypes.Rgba32, 1024, 428, HeifBitDepth.Bit12)] + public void DecodeLoopRestorationMatchesReference( + TestImageProvider provider, + int width, + int height, + HeifBitDepth bitDepth) + { + AssertPresentedMetadata(provider, width, height, bitDepth); + + FeatureTestRunner.RunWithHwIntrinsicsFeature( + ValidatePresentedFixture, + PresentationConfigurations, + provider); + } + + /// + /// Verifies that the independently encoded AVIF presentation fixtures collectively select both restoration algorithms. + /// + [Fact] + public void LoopRestorationPresentationFixturesSelectBothAlgorithms() + { + int restorationCoverage = GetRestorationCoverageFromAvif(TestFile.Create(TestImages.Heif.Av1Restoration8BitAvif).Bytes); + restorationCoverage |= GetRestorationCoverageFromAvif(TestFile.Create(TestImages.Heif.Av1Restoration10BitAvif).Bytes); + restorationCoverage |= GetRestorationCoverageFromAvif(TestFile.Create(TestImages.Heif.Av1Restoration12BitAvif).Bytes); + + int requiredCoverage = WienerRestorationCoverage | SelfGuidedRestorationCoverage; + Assert.Equal(requiredCoverage, restorationCoverage & requiredCoverage); + } + + /// + /// Verifies film-grain template generation, block selection, overlap, chroma scaling, subsampling, high-bit-depth + /// arithmetic, and exact native presentation samples against the independent scalar reference. + /// + [Fact] + public void DecodeFilmGrainMatchesReference() + => FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidateFilmGrainFixtures, LoopRestorationConfigurations); + + /// + /// Verifies that independently encoded AV1 streams exercise every normative coding-block partition shape. + /// + [Fact] + public void IndependentFixturesCoverEveryPartitionType() + { + int coverage = GetPartitionCoverage(TestImages.Heif.Av1Cdef8BitPayload); + coverage |= GetPartitionCoverage(TestImages.Heif.Av1Cdef10BitPayload); + coverage |= GetPartitionCoverage(TestImages.Heif.Av1Cdef12BitPayload); + + Assert.Equal(RequiredPartitionCoverage, coverage & RequiredPartitionCoverage); + } + + /// + /// Validates every native profile fixture under the hardware configuration selected by + /// . + /// + private static void ValidateProfileNativeFixtures() + { + ValidateProfileNativeFixture( + TestImages.Heif.Av1Profile8BitMonochromeAvif, + TestImages.Heif.Av1Profile8BitMonochromeReference, + Av1BitDepth.EightBit, + Av1ColorFormat.Yuv400, + ObuSequenceProfile.Main); + + ValidateProfileNativeFixture( + TestImages.Heif.Av1Profile8Bit420Avif, + TestImages.Heif.Av1Profile8Bit420Reference, + Av1BitDepth.EightBit, + Av1ColorFormat.Yuv420, + ObuSequenceProfile.Main); + + ValidateProfileNativeFixture( + TestImages.Heif.Av1Profile8Bit422Avif, + TestImages.Heif.Av1Profile8Bit422Reference, + Av1BitDepth.EightBit, + Av1ColorFormat.Yuv422, + ObuSequenceProfile.Professional); + + ValidateProfileNativeFixture( + TestImages.Heif.Av1Profile8Bit444Avif, + TestImages.Heif.Av1Profile8Bit444Reference, + Av1BitDepth.EightBit, + Av1ColorFormat.Yuv444, + ObuSequenceProfile.High); + + ValidateProfileNativeFixture( + TestImages.Heif.Av1Profile10BitMonochromeAvif, + TestImages.Heif.Av1Profile10BitMonochromeReference, + Av1BitDepth.TenBit, + Av1ColorFormat.Yuv400, + ObuSequenceProfile.Main); + + ValidateProfileNativeFixture( + TestImages.Heif.Av1Profile10Bit420Avif, + TestImages.Heif.Av1Profile10Bit420Reference, + Av1BitDepth.TenBit, + Av1ColorFormat.Yuv420, + ObuSequenceProfile.Main); + + ValidateProfileNativeFixture( + TestImages.Heif.Av1Profile10Bit422Avif, + TestImages.Heif.Av1Profile10Bit422Reference, + Av1BitDepth.TenBit, + Av1ColorFormat.Yuv422, + ObuSequenceProfile.Professional); + + ValidateProfileNativeFixture( + TestImages.Heif.Av1Profile10Bit444Avif, + TestImages.Heif.Av1Profile10Bit444Reference, + Av1BitDepth.TenBit, + Av1ColorFormat.Yuv444, + ObuSequenceProfile.High); + + ValidateProfileNativeFixture( + TestImages.Heif.Av1Profile12BitMonochromeAvif, + TestImages.Heif.Av1Profile12BitMonochromeReference, + Av1BitDepth.TwelveBit, + Av1ColorFormat.Yuv400, + ObuSequenceProfile.Professional); + + ValidateProfileNativeFixture( + TestImages.Heif.Av1Profile12Bit420Avif, + TestImages.Heif.Av1Profile12Bit420Reference, + Av1BitDepth.TwelveBit, + Av1ColorFormat.Yuv420, + ObuSequenceProfile.Professional); + + ValidateProfileNativeFixture( + TestImages.Heif.Av1Profile12Bit422Avif, + TestImages.Heif.Av1Profile12Bit422Reference, + Av1BitDepth.TwelveBit, + Av1ColorFormat.Yuv422, + ObuSequenceProfile.Professional); + + ValidateProfileNativeFixture( + TestImages.Heif.Av1Profile12Bit444Avif, + TestImages.Heif.Av1Profile12Bit444Reference, + Av1BitDepth.TwelveBit, + Av1ColorFormat.Yuv444, + ObuSequenceProfile.Professional); + } + + /// + /// Validates one independently encoded AVIF against its native Y4M reference and signaled sequence profile. + /// + /// The complete AVIF container. + /// The independently decoded native Y4M output. + /// The expected AV1 sample precision. + /// The expected native chroma-sampling layout. + /// The AV1 profile required by the bit-depth and chroma-format combination. + private static void ValidateProfileNativeFixture( + string imagePath, + string referencePath, + Av1BitDepth bitDepth, + Av1ColorFormat colorFormat, + ObuSequenceProfile sequenceProfile) + { + byte[] imageBytes = TestFile.Create(imagePath).Bytes; + byte[] referenceBytes = TestFile.Create(referencePath).Bytes; + (string chromaTag, string extendedChromaTag) = GetY4mColorSpace(bitDepth, colorFormat); + string expectedHeader = + $"YUV4MPEG2 W{ProfileFixtureWidth} H{ProfileFixtureHeight} F25:1 Ip A0:0 C{chromaTag} XYSCSS={extendedChromaTag} XCOLORRANGE=FULL\n"; + + int headerTerminator = referenceBytes.AsSpan().IndexOf((byte)'\n'); + Assert.NotEqual(-1, headerTerminator); + int fileHeaderLength = headerTerminator + 1; + Assert.Equal(expectedHeader, Encoding.ASCII.GetString(referenceBytes, 0, fileHeaderLength)); + + ReadOnlySpan nativeReference = referenceBytes.AsSpan(fileHeaderLength); + ReadOnlySpan frameHeader = "FRAME\n"u8; + Assert.True(nativeReference.StartsWith(frameHeader)); + nativeReference = nativeReference[frameHeader.Length..]; + + Span payload = GetSoleAv1ItemPayload(imageBytes); + using Av1Decoder decoder = new(Configuration.Default); + using Av1FrameBuffer frameBuffer = decoder.DecodeFrameBuffer(payload, null, null, out _); + + Assert.Equal(ProfileFixtureWidth, frameBuffer.Width); + Assert.Equal(ProfileFixtureHeight, frameBuffer.Height); + Assert.Equal(bitDepth, frameBuffer.BitDepth); + Assert.Equal(colorFormat, frameBuffer.ColorFormat); + + ObuSequenceHeader sequenceHeader = Assert.IsType(decoder.SequenceHeader); + ObuColorConfig colorConfig = sequenceHeader.ColorConfig; + Assert.Equal(sequenceProfile, sequenceHeader.SequenceProfile); + Assert.Equal(bitDepth, colorConfig.BitDepth); + Assert.Equal(colorFormat, colorConfig.GetColorFormat()); + Assert.Equal(colorFormat == Av1ColorFormat.Yuv400, colorConfig.IsMonochrome); + Assert.True(colorConfig.IsColorDescriptionPresent); + Assert.Equal(ObuColorPrimaries.Bt709, colorConfig.ColorPrimaries); + Assert.Equal(ObuTransferCharacteristics.Srgb, colorConfig.TransferCharacteristics); + Assert.Equal(ObuMatrixCoefficients.Bt601, colorConfig.MatrixCoefficients); + Assert.True(colorConfig.ColorRange); + AssertNativePlanesEqual(decoder, frameBuffer, nativeReference); + } + + /// + /// Gets the Y4M chroma tags that encode one AV1 bit-depth and sampling-layout combination. + /// + /// The encoded AV1 sample precision. + /// The encoded AV1 chroma-sampling layout. + /// The Y4M C tag and extended XYSCSS tag. + private static (string ChromaTag, string ExtendedChromaTag) GetY4mColorSpace(Av1BitDepth bitDepth, Av1ColorFormat colorFormat) + { + // Y4M uses a legacy 420jpeg name at eight bits, lowercase p in high-depth C tags, and uppercase P in the + // corresponding XYSCSS tags. Keeping the exact spellings detects a reference generated with different layout. + return (bitDepth, colorFormat) switch + { + (Av1BitDepth.EightBit, Av1ColorFormat.Yuv400) => ("mono", "400"), + (Av1BitDepth.EightBit, Av1ColorFormat.Yuv420) => ("420jpeg", "420JPEG"), + (Av1BitDepth.EightBit, Av1ColorFormat.Yuv422) => ("422", "422"), + (Av1BitDepth.EightBit, Av1ColorFormat.Yuv444) => ("444", "444"), + (Av1BitDepth.TenBit, Av1ColorFormat.Yuv400) => ("mono10", "400"), + (Av1BitDepth.TenBit, Av1ColorFormat.Yuv420) => ("420p10", "420P10"), + (Av1BitDepth.TenBit, Av1ColorFormat.Yuv422) => ("422p10", "422P10"), + (Av1BitDepth.TenBit, Av1ColorFormat.Yuv444) => ("444p10", "444P10"), + (Av1BitDepth.TwelveBit, Av1ColorFormat.Yuv400) => ("mono12", "400"), + (Av1BitDepth.TwelveBit, Av1ColorFormat.Yuv420) => ("420p12", "420P12"), + (Av1BitDepth.TwelveBit, Av1ColorFormat.Yuv422) => ("422p12", "422P12"), + _ => ("444p12", "444P12") + }; + } + + /// + /// Validates every active-CDEF fixture under the hardware configuration selected by . + /// + private static void ValidateActiveCdefFixtures() + { + ValidateActiveCdefFixture( + TestImages.Heif.Av1Cdef8BitPayload, + TestImages.Heif.Av1Cdef8BitReference, + 768, + 512, + Av1BitDepth.EightBit, + Av1ColorFormat.Yuv420); + + ValidateActiveCdefFixture( + TestImages.Heif.Av1Cdef10BitPayload, + TestImages.Heif.Av1Cdef10BitReference, + 1024, + 428, + Av1BitDepth.TenBit, + Av1ColorFormat.Yuv444); + + ValidateActiveCdefFixture( + TestImages.Heif.Av1Cdef12BitPayload, + TestImages.Heif.Av1Cdef12BitReference, + 1024, + 428, + Av1BitDepth.TwelveBit, + Av1ColorFormat.Yuv444); + } + + /// + /// Validates the active-palette native fixture under the hardware configuration selected by + /// . + /// + private static void ValidatePaletteNativeFixture() + => ValidateNativeFixture( + TestImages.Heif.Av1Palette8BitPayload, + TestImages.Heif.Av1Palette8BitReference, + 33, + 11, + Av1BitDepth.EightBit, + Av1ColorFormat.Yuv444, + requireActiveCdef: false, + requireActiveLoopFilter: false, + requirePalette: true); + + /// + /// Validates every active intra-block-copy native fixture under the hardware configuration selected by + /// . + /// + private static void ValidateIntraBlockCopyNativeFixtures() + { + ValidateIntraBlockCopyNativeFixture( + TestImages.Heif.Av1IntraBlockCopy8BitAvif, + TestImages.Heif.Av1IntraBlockCopy8BitReference, + Av1BitDepth.EightBit); + + ValidateIntraBlockCopyNativeFixture( + TestImages.Heif.Av1IntraBlockCopy10BitAvif, + TestImages.Heif.Av1IntraBlockCopy10BitReference, + Av1BitDepth.TenBit); + + ValidateIntraBlockCopyNativeFixture( + TestImages.Heif.Av1IntraBlockCopy12BitAvif, + TestImages.Heif.Av1IntraBlockCopy12BitReference, + Av1BitDepth.TwelveBit); + } + + /// + /// Validates one independently encoded intra-block-copy AVIF against its native Y4M reference. + /// + /// The complete AVIF container. + /// The independently decoded native Y4M output. + /// The expected AV1 sample precision. + private static void ValidateIntraBlockCopyNativeFixture(string imagePath, string referencePath, Av1BitDepth bitDepth) + { + byte[] imageBytes = TestFile.Create(imagePath).Bytes; + byte[] referenceBytes = TestFile.Create(referencePath).Bytes; + ReadOnlySpan fileHeader = bitDepth switch + { + Av1BitDepth.EightBit => "YUV4MPEG2 W512 H256 F25:1 Ip A0:0 C444 XYSCSS=444 XCOLORRANGE=FULL\n"u8, + Av1BitDepth.TenBit => "YUV4MPEG2 W512 H256 F25:1 Ip A0:0 C444p10 XYSCSS=444P10 XCOLORRANGE=FULL\n"u8, + _ => "YUV4MPEG2 W512 H256 F25:1 Ip A0:0 C444p12 XYSCSS=444P12 XCOLORRANGE=FULL\n"u8 + }; + + ReadOnlySpan frameHeader = "FRAME\n"u8; + + // The retained Y4M header locks the independently decoded reference to the expected dimensions, sampling, + // bit depth, and full range. Only the following frame payload contains the planar Y, U, and V samples. + ReadOnlySpan nativeReference = referenceBytes; + Assert.True(nativeReference.StartsWith(fileHeader)); + nativeReference = nativeReference[fileHeader.Length..]; + Assert.True(nativeReference.StartsWith(frameHeader)); + nativeReference = nativeReference[frameHeader.Length..]; + + Span payload = GetSoleAv1ItemPayload(imageBytes); + using Av1Decoder decoder = new(Configuration.Default); + using Av1FrameBuffer frameBuffer = decoder.DecodeFrameBuffer(payload, null, null, out _); + + Assert.Equal(512, frameBuffer.Width); + Assert.Equal(256, frameBuffer.Height); + Assert.Equal(bitDepth, frameBuffer.BitDepth); + Assert.Equal(Av1ColorFormat.Yuv444, frameBuffer.ColorFormat); + Assert.NotNull(decoder.FrameHeader); + Assert.True(decoder.FrameHeader.AllowIntraBlockCopy); + Assert.NotEqual(0, GetIntraBlockCopyBlockCount(decoder)); + AssertNativePlanesEqual(decoder, frameBuffer, nativeReference); + } + + /// + /// Runs the exact final-layer native and presentation comparisons with the default configuration. + /// + /// The serialized AVIF input provider and reference-output naming context. + private static void ValidateProgressiveSingleReferenceFixtureWithDefaultConfiguration(string providerDump) + { + ValidateProgressiveSingleReferenceFixture(Configuration.Default); + ValidatePresentedFixture(providerDump); + } + + /// + /// Runs the selected-spatial-layer native and presentation comparisons with the default configuration. + /// + /// The serialized selected-layer provider and reference-output naming context. + private static void ValidateSelectedProgressiveSpatialLayerWithDefaultConfiguration(string providerDump) + { + ValidateSelectedProgressiveSpatialLayer(Configuration.Default); + ValidatePresentedFixture(providerDump); + } + + /// + /// Runs the exact scaled-reference native and presentation comparison with the default configuration. + /// + /// The serialized AVIF input provider and reference-output naming context. + private static void ValidateScaledReferenceFixtureWithDefaultConfiguration(string providerDump) + { + ValidateScaledReferenceFixture(Configuration.Default); + ValidatePresentedFixture(providerDump); + } + + /// + /// Verifies the genuine size-changing layered fixture with the requested allocator. + /// + /// The decoder configuration. + private static void ValidateScaledReferenceFixture(Configuration configuration) + { + byte[] payload = TestFile.Create(TestImages.Heif.Av1ScaledReferencePayload).Bytes; + byte[] baseReferenceBytes = TestFile.Create(TestImages.Heif.Av1ScaledReferenceBaseNativeReference).Bytes; + byte[] referenceBytes = TestFile.Create(TestImages.Heif.Av1ScaledReferenceNativeReference).Bytes; + ReadOnlySpan fileHeader = + "YUV4MPEG2 W80 H80 F25:1 Ip A0:0 C444 XYSCSS=444 XCOLORRANGE=LIMITED\n"u8; + + ReadOnlySpan frameHeader = "FRAME\n"u8; + ReadOnlySpan nativeReference = referenceBytes; + Assert.True(nativeReference.StartsWith(fileHeader)); + nativeReference = nativeReference[fileHeader.Length..]; + Assert.True(nativeReference.StartsWith(frameHeader)); + nativeReference = nativeReference[frameHeader.Length..]; + Assert.Equal(ScaledReferenceFixtureSize * ScaledReferenceFixtureSize * 3, nativeReference.Length); + + // Decode the independently declared base extent alone to prove that the retained reference is 40x40 rather + // than relying on the 80x80 item presentation dimensions recorded by the container. + using (Av1Decoder baseDecoder = new(configuration)) + using (Av1FrameBuffer baseFrameBuffer = baseDecoder.DecodeFrameBuffer( + payload.AsSpan(0, ScaledReferenceFirstLayerSize), + null, + null, + out _)) + { + Assert.Equal(ScaledReferenceBaseLayerSize, baseFrameBuffer.Width); + Assert.Equal(ScaledReferenceBaseLayerSize, baseFrameBuffer.Height); + Assert.Equal(ScaledReferenceBaseLayerSize * ScaledReferenceBaseLayerSize * 3, baseReferenceBytes.Length); + AssertNativePlanesEqual(baseDecoder, baseFrameBuffer, baseReferenceBytes); + } + + // Exercise the same retained owner across two calls so the independently verified base samples are checked + // in the exact decoder session that supplies the size-changing reference to the dependent frame. + using (Av1Decoder sequenceDecoder = new(configuration)) + { + sequenceDecoder.DecodeSequenceReference( + payload.AsSpan(0, ScaledReferenceFirstLayerSize), + null, + null); + + Av1FrameBuffer retainedBaseFrameBuffer = Assert.IsType>(sequenceDecoder.FrameBuffer); + AssertNativePlanesEqual(sequenceDecoder, retainedBaseFrameBuffer, baseReferenceBytes); + using Av1FrameBuffer sequenceFrameBuffer = sequenceDecoder.DecodeFrameBuffer( + payload.AsSpan(ScaledReferenceFirstLayerSize), + null, + null, + out _); + + AssertNativePlanesEqual(sequenceDecoder, sequenceFrameBuffer, nativeReference); + } + + using Av1Decoder decoder = new(configuration); + using Av1FrameBuffer frameBuffer = decoder.DecodeFrameBuffer( + payload, + null, + null, + out _, + new Av1LayeredImageIndex(ScaledReferenceFirstLayerSize, 0, 0)); + + Assert.Equal(ScaledReferenceFixtureSize, frameBuffer.Width); + Assert.Equal(ScaledReferenceFixtureSize, frameBuffer.Height); + Assert.Equal(Av1BitDepth.EightBit, frameBuffer.BitDepth); + Assert.Equal(Av1ColorFormat.Yuv444, frameBuffer.ColorFormat); + Assert.Equal(1, frameBuffer.GetPlaneBuffer(Av1Plane.Y).FastMemoryGroup.Count); + Assert.Equal(1, frameBuffer.GetPlaneBuffer(Av1Plane.U).FastMemoryGroup.Count); + Assert.Equal(1, frameBuffer.GetPlaneBuffer(Av1Plane.V).FastMemoryGroup.Count); + + ObuSequenceHeader sequenceHeader = Assert.IsType(decoder.SequenceHeader); + ObuFrameHeader finalFrameHeader = Assert.IsType(decoder.FrameHeader); + Av1FrameInfo frameInfo = Assert.IsType(decoder.FrameInfo); + + Assert.Equal(ScaledReferenceFixtureSize, sequenceHeader.MaxFrameWidth); + Assert.Equal(ScaledReferenceFixtureSize, sequenceHeader.MaxFrameHeight); + Assert.Equal(ObuFrameType.InterFrame, finalFrameHeader.FrameType); + Assert.True(finalFrameHeader.LoopFilterParameters.ReferenceDeltaModeEnabled); + Assert.NotEqual(0, finalFrameHeader.LoopFilterParameters.FilterLevelU); + Assert.NotEqual(0, finalFrameHeader.LoopFilterParameters.FilterLevelV); + int interBlockCount = 0; + int intraBlockCount = 0; + int skippedInterBlockCount = 0; + int superblockSizeLog2 = sequenceHeader.SuperblockSizeLog2; + int superblockColumnCount = Av1Math.AlignPowerOf2(sequenceHeader.MaxFrameWidth, superblockSizeLog2) >> superblockSizeLog2; + int superblockRowCount = Av1Math.AlignPowerOf2(sequenceHeader.MaxFrameHeight, superblockSizeLog2) >> superblockSizeLog2; + for (int superblockRow = 0; superblockRow < superblockRowCount; superblockRow++) + { + for (int superblockColumn = 0; superblockColumn < superblockColumnCount; superblockColumn++) + { + Av1SuperblockInfo superblock = frameInfo.GetSuperblock(new Point(superblockColumn, superblockRow)); + foreach (Av1BlockModeInfo modeInfo in superblock.GetModeInfos()) + { + if (modeInfo.ReferenceFrames[0] >= Av1ReferenceFrameType.Last) + { + interBlockCount++; + if (modeInfo.Skip) + { + skippedInterBlockCount++; + } + } + else + { + intraBlockCount++; + } + } + } + } + + Assert.NotEqual(0, interBlockCount); + Assert.NotEqual(0, intraBlockCount); + Assert.NotEqual(0, skippedInterBlockCount); + AssertNativePlanesEqual(decoder, frameBuffer, nativeReference); + } + + /// + /// Verifies the selected base spatial layer with the requested allocator. + /// + /// The decoder configuration. + private static void ValidateSelectedProgressiveSpatialLayer(Configuration configuration) + { + byte[] payload = TestFile.Create(TestImages.Heif.Av1ScaledReferencePayload).Bytes; + byte[] nativeReference = + TestFile.Create(TestImages.Heif.Av1ScaledReferenceBaseNativeReference).Bytes; + + Assert.Equal(ScaledReferenceBaseLayerSize * ScaledReferenceBaseLayerSize * 3, nativeReference.Length); + + Av1LayeredImageIndex layeredImageIndex = new(ScaledReferenceFirstLayerSize, 0, 0); + int selectedPayloadLength = layeredImageIndex.GetPayloadLength( + payload.Length, + new Av1LayerSelector(0)); + + Assert.Equal(ScaledReferenceFirstLayerSize, selectedPayloadLength); + + using Av1Decoder decoder = new(configuration); + using Av1FrameBuffer frameBuffer = decoder.DecodeFrameBuffer( + payload.AsSpan(0, selectedPayloadLength), + null, + null, + out _, + layeredImageIndex); + + Assert.Equal(ScaledReferenceBaseLayerSize, frameBuffer.Width); + Assert.Equal(ScaledReferenceBaseLayerSize, frameBuffer.Height); + Assert.Equal(Av1BitDepth.EightBit, frameBuffer.BitDepth); + Assert.Equal(Av1ColorFormat.Yuv444, frameBuffer.ColorFormat); + Assert.Equal(ObuFrameType.KeyFrame, Assert.IsType(decoder.FrameHeader).FrameType); + AssertNativePlanesEqual(decoder, frameBuffer, nativeReference); + } + + /// + /// Verifies the final dependent layer with the requested allocator. + /// + /// The decoder configuration. + private static void ValidateProgressiveSingleReferenceFixture(Configuration configuration) + { + byte[] payload = TestFile.Create(TestImages.Heif.Av1Progressive8BitPayload).Bytes; + byte[] referenceBytes = TestFile.Create(TestImages.Heif.Av1Progressive8BitReference).Bytes; + ReadOnlySpan fileHeader = + "YUV4MPEG2 W33 H11 F25:1 Ip A0:0 C444alpha XYSCSS=444 XCOLORRANGE=FULL\n"u8; + + ReadOnlySpan frameHeader = "FRAME\n"u8; + + int planeSampleCount = ProgressiveFixtureWidth * ProgressiveFixtureHeight; + int frameSampleCount = planeSampleCount * 4; + + // The reference stores both progressive YUV444-alpha outputs in decode order. Select the second frame so this + // assertion cannot pass by comparing only the independently decodable base layer. + ReadOnlySpan nativeReference = referenceBytes; + Assert.True(nativeReference.StartsWith(fileHeader)); + nativeReference = nativeReference[fileHeader.Length..]; + Assert.True(nativeReference.StartsWith(frameHeader)); + int storedFrameSize = frameHeader.Length + frameSampleCount; + Assert.Equal(storedFrameSize * 2, nativeReference.Length); + + ReadOnlySpan finalFrameReference = nativeReference[storedFrameSize..]; + Assert.True(finalFrameReference.StartsWith(frameHeader)); + finalFrameReference = finalFrameReference[frameHeader.Length..]; + Assert.Equal(frameSampleCount, finalFrameReference.Length); + + // The Y4M stores the color item's Y, U, and V planes before the auxiliary alpha plane. Native AV1 reconstruction + // is compared with exactly those first three planes of the final dependent frame. + ReadOnlySpan colorReference = finalFrameReference[..(planeSampleCount * 3)]; + + using Av1Decoder decoder = new(configuration); + using Av1FrameBuffer frameBuffer = decoder.DecodeFrameBuffer( + payload, + null, + null, + out _, + new Av1LayeredImageIndex(ProgressiveFirstLayerSize, 0, 0)); + + Assert.Equal(ProgressiveFixtureWidth, frameBuffer.Width); + Assert.Equal(ProgressiveFixtureHeight, frameBuffer.Height); + Assert.Equal(Av1BitDepth.EightBit, frameBuffer.BitDepth); + Assert.Equal(Av1ColorFormat.Yuv444, frameBuffer.ColorFormat); + Assert.Equal(1, frameBuffer.GetPlaneBuffer(Av1Plane.Y).FastMemoryGroup.Count); + Assert.Equal(1, frameBuffer.GetPlaneBuffer(Av1Plane.U).FastMemoryGroup.Count); + Assert.Equal(1, frameBuffer.GetPlaneBuffer(Av1Plane.V).FastMemoryGroup.Count); + + ObuSequenceHeader sequenceHeader = Assert.IsType(decoder.SequenceHeader); + ObuFrameHeader finalFrameHeader = Assert.IsType(decoder.FrameHeader); + Av1FrameInfo frameInfo = Assert.IsType(decoder.FrameInfo); + + Assert.Equal(ObuFrameType.InterFrame, finalFrameHeader.FrameType); + + int superblockSizeLog2 = sequenceHeader.SuperblockSizeLog2; + int superblockColumnCount = Av1Math.AlignPowerOf2(sequenceHeader.MaxFrameWidth, superblockSizeLog2) >> superblockSizeLog2; + int superblockRowCount = Av1Math.AlignPowerOf2(sequenceHeader.MaxFrameHeight, superblockSizeLog2) >> superblockSizeLog2; + int interBlockCount = 0; + + // Traverse the final coding-block records once rather than revisiting every 4x4 map cell covered by each + // block. The syntax assertions ensure that this fixture reaches only the completed single-reference path. + for (int superblockRow = 0; superblockRow < superblockRowCount; superblockRow++) + { + for (int superblockColumn = 0; superblockColumn < superblockColumnCount; superblockColumn++) + { + Av1SuperblockInfo superblock = frameInfo.GetSuperblock(new Point(superblockColumn, superblockRow)); + foreach (Av1BlockModeInfo modeInfo in superblock.GetModeInfos()) + { + if (modeInfo.ReferenceFrames[0] < Av1ReferenceFrameType.Last) + { + continue; + } + + Assert.Equal(Av1ReferenceFrameType.None, modeInfo.ReferenceFrames[1]); + Assert.Equal(Av1MotionMode.SimpleTranslation, modeInfo.MotionMode); + interBlockCount++; + } + } + } + + Assert.NotEqual(0, interBlockCount); + AssertNativePlanesEqual(decoder, frameBuffer, colorReference); + + DecoderOptions options = new() { Configuration = configuration, MaxFrames = 1 }; + byte[] imageBytes = TestFile.Create(TestImages.Heif.Av1Progressive8BitAvif).Bytes; + using Image image = Image.Load(options, imageBytes); + + Assert.Equal(ProgressiveFixtureWidth, image.Width); + Assert.Equal(ProgressiveFixtureHeight, image.Height); + Assert.Single(image.Frames); + Assert.Equal(HeifBitDepth.Bit8, image.Metadata.GetHeifMetadata().BitDepth); + } + + /// + /// Parses the selected image-sequence tracks from a complete HEIF fixture. + /// + /// The complete HEIF file. + /// The bounded image-sequence model. + private static HeifSequence ParseImageSequence(byte[] fileBytes) + { + using MemoryStream stream = new(fileBytes, false); + Span scratch = stackalloc byte[32]; + while (stream.Position < stream.Length) + { + long boxLength = HeifBoxReader.ReadHeader( + stream, + stream.Length, + scratch, + out Heif4CharCode boxType, + topLevel: true); + + long boxStart = stream.Position; + if (boxType == Heif4CharCode.Moov) + { + HeifSequenceParser parser = new(new DecoderOptions { MaxFrames = 32 }); + return parser.Parse(stream, boxLength); + } + + stream.Position = checked(boxStart + boxLength); + } + + throw new InvalidImageContentException("The HEIF fixture contains no image sequence."); + } + + /// + /// Validates every lossless native fixture under the hardware configuration selected by + /// . + /// + private static void ValidateLosslessFixtures() + { + ValidateLosslessFixture( + TestImages.Heif.Av1Lossless8BitAvif, + TestImages.Heif.Av1Lossless8BitReference, + Av1BitDepth.EightBit); + + ValidateLosslessFixture( + TestImages.Heif.Av1Lossless10BitAvif, + TestImages.Heif.Av1Lossless10BitReference, + Av1BitDepth.TenBit); + + ValidateLosslessFixture( + TestImages.Heif.Av1Lossless12BitAvif, + TestImages.Heif.Av1Lossless12BitReference, + Av1BitDepth.TwelveBit); + } + + /// + /// Validates every active super-resolution fixture under the hardware configuration selected by + /// . + /// + private static void ValidateSuperResolutionFixtures() + { + ValidateSuperResolutionFixture( + TestImages.Heif.Av1SuperResolution8BitPayload, + TestImages.Heif.Av1SuperResolution8BitReference, + 768, + 512, + Av1BitDepth.EightBit, + Av1ColorFormat.Yuv420); + + ValidateSuperResolutionFixture( + TestImages.Heif.Av1SuperResolution10BitPayload, + TestImages.Heif.Av1SuperResolution10BitReference, + 1024, + 428, + Av1BitDepth.TenBit, + Av1ColorFormat.Yuv444); + + ValidateSuperResolutionFixture( + TestImages.Heif.Av1SuperResolution12BitPayload, + TestImages.Heif.Av1SuperResolution12BitReference, + 1024, + 428, + Av1BitDepth.TwelveBit, + Av1ColorFormat.Yuv444); + } + + /// + /// Validates every active loop-restoration fixture under the hardware configuration selected by + /// . + /// + private static void ValidateLoopRestorationFixtures() + { + int restorationCoverage = ValidateLoopRestorationFixture( + TestImages.Heif.Av1Restoration8BitPayload, + TestImages.Heif.Av1Restoration8BitReference, + 768, + 512, + Av1BitDepth.EightBit, + Av1ColorFormat.Yuv420); + + restorationCoverage |= ValidateLoopRestorationFixture( + TestImages.Heif.Av1Restoration10BitPayload, + TestImages.Heif.Av1Restoration10BitReference, + 1024, + 428, + Av1BitDepth.TenBit, + Av1ColorFormat.Yuv444); + + restorationCoverage |= ValidateLoopRestorationFixture( + TestImages.Heif.Av1Restoration12BitPayload, + TestImages.Heif.Av1Restoration12BitReference, + 1024, + 428, + Av1BitDepth.TwelveBit, + Av1ColorFormat.Yuv444); + + // Exact output only proves both restoration algorithms when the independent fixture set + // actually selects at least one unit of each type during every feature-runner invocation. + int requiredCoverage = WienerRestorationCoverage | SelfGuidedRestorationCoverage; + Assert.Equal(requiredCoverage, restorationCoverage & requiredCoverage); + } + + /// + /// Validates active restoration after super-resolution for 8-bit 4:2:0 content. + /// + private static void ValidateLoopRestorationAndSuperResolution8Bit420() + => ValidateLoopRestorationFixture( + TestImages.Heif.Av1RestorationSuperResolution8BitPayload, + TestImages.Heif.Av1RestorationSuperResolution8BitReference, + 768, + 512, + Av1BitDepth.EightBit, + Av1ColorFormat.Yuv420, + requireSuperResolution: true); + + /// + /// Validates active restoration after super-resolution for 10-bit 4:2:2 content. + /// + private static void ValidateLoopRestorationAndSuperResolution10Bit422() + => ValidateLoopRestorationFixture( + TestImages.Heif.Av1RestorationSuperResolution10BitPayload, + TestImages.Heif.Av1RestorationSuperResolution10BitReference, + 512, + 256, + Av1BitDepth.TenBit, + Av1ColorFormat.Yuv422, + requireSuperResolution: true); + + /// + /// Validates active restoration after super-resolution for 12-bit 4:4:4 content. + /// + private static void ValidateLoopRestorationAndSuperResolution12Bit444() + => ValidateLoopRestorationFixture( + TestImages.Heif.Av1RestorationSuperResolution12BitPayload, + TestImages.Heif.Av1RestorationSuperResolution12BitReference, + 1024, + 428, + Av1BitDepth.TwelveBit, + Av1ColorFormat.Yuv444, + requireSuperResolution: true); + + /// + /// Validates every active film-grain fixture under the hardware configuration selected by + /// . + /// + private static void ValidateFilmGrainFixtures() + { + ValidateFilmGrainFixture( + TestImages.Heif.Av1FilmGrain8BitPayload, + TestImages.Heif.Av1FilmGrain8BitReference, + 100, + 60, + Av1BitDepth.EightBit, + Av1ColorFormat.Yuv420); + + ValidateFilmGrainFixture( + TestImages.Heif.Av1FilmGrain10BitPayload, + TestImages.Heif.Av1FilmGrain10BitReference, + 100, + 60, + Av1BitDepth.TenBit, + Av1ColorFormat.Yuv422); + + ValidateFilmGrainFixture( + TestImages.Heif.Av1FilmGrain12BitPayload, + TestImages.Heif.Av1FilmGrain12BitReference, + 100, + 60, + Av1BitDepth.TwelveBit, + Av1ColorFormat.Yuv444); + + ValidateFilmGrainFixture( + TestImages.Heif.Av1FilmGrain8BitRestrictedPayload, + TestImages.Heif.Av1FilmGrain8BitRestrictedReference, + 100, + 60, + Av1BitDepth.EightBit, + Av1ColorFormat.Yuv420, + requireRestrictedRange: true); + + ValidateFilmGrainFixture( + TestImages.Heif.Av1FilmGrain8BitMonochromePayload, + TestImages.Heif.Av1FilmGrain8BitMonochromeReference, + 100, + 60, + Av1BitDepth.EightBit, + Av1ColorFormat.Yuv400, + requireRestrictedRange: true); + + ValidateFilmGrainFixture( + TestImages.Heif.Av1FilmGrain12BitIdentityPayload, + TestImages.Heif.Av1FilmGrain12BitIdentityReference, + 100, + 60, + Av1BitDepth.TwelveBit, + Av1ColorFormat.Yuv444, + requireRestrictedRange: true, + requireIdentityMatrix: true); + + ValidateFilmGrainFixture( + TestImages.Heif.Av1FilmGrainOddDimensionsPayload, + TestImages.Heif.Av1FilmGrainOddDimensionsReference, + 33, + 11, + Av1BitDepth.EightBit, + Av1ColorFormat.Yuv420); + } + + /// + /// Validates one elementary-stream sample and its containing AVIF image. + /// + /// The complete AVIF container. + /// The AV1 elementary-stream sample extracted from the container. + /// The retained native planar output. + /// The expected displayed width. + /// The expected displayed height. + /// The expected AV1 sample precision. + /// The expected native chroma-sampling layout. + /// The expected public HEIF sample precision. + private static void ValidateFixture( + string imagePath, + string payloadPath, + string referencePath, + int width, + int height, + Av1BitDepth bitDepth, + Av1ColorFormat colorFormat, + HeifBitDepth metadataBitDepth) + { + ValidateNativeFixture(payloadPath, referencePath, width, height, bitDepth, colorFormat, false); + ValidatePresentedImage(imagePath, width, height, metadataBitDepth); + } + + /// + /// Validates complete native-plane reconstruction for one AV1 elementary-stream sample. + /// + /// The AV1 elementary-stream sample. + /// The retained native planar output. + /// The expected reconstructed width. + /// The expected reconstructed height. + /// The expected AV1 sample precision. + /// The expected native chroma-sampling layout. + /// Indicates whether the stream must signal and select nonzero CDEF strengths. + /// Indicates whether the stream must use normative horizontal upscaling. + /// Indicates whether the stream must select at least one loop-restoration unit. + /// Indicates whether the displayed frame must synthesize signaled film grain. + /// Indicates whether film grain must clip every plane to its restricted range. + /// Indicates whether restricted chroma clipping must use the luma endpoints. + /// Indicates whether the stream must signal a nonzero deblocking strength. + /// Indicates whether the stream must select palette prediction for luma and chroma. + /// A bit mask containing every selected loop-restoration filter type. + private static int ValidateNativeFixture( + string payloadPath, + string referencePath, + int width, + int height, + Av1BitDepth bitDepth, + Av1ColorFormat colorFormat, + bool requireActiveCdef, + bool requireSuperResolution = false, + bool requireLoopRestoration = false, + bool requireFilmGrain = false, + bool requireRestrictedRange = false, + bool requireIdentityMatrix = false, + bool requireActiveLoopFilter = true, + bool requirePalette = false) + { + int restorationCoverage = 0; + byte[] payload = TestFile.Create(payloadPath).Bytes; + byte[] reference = TestFile.Create(referencePath).Bytes; + using Av1Decoder decoder = new(Configuration.Default); + using Av1FrameBuffer frameBuffer = decoder.DecodeFrameBuffer(payload, null, null, out _); + + Assert.Equal(width, frameBuffer.Width); + Assert.Equal(height, frameBuffer.Height); + Assert.Equal(bitDepth, frameBuffer.BitDepth); + Assert.Equal(colorFormat, frameBuffer.ColorFormat); + Assert.NotNull(decoder.FrameHeader); + + if (requireSuperResolution) + { + ObuFrameSize frameSize = decoder.FrameHeader.FrameSize; + Assert.True(frameSize.FrameWidth < frameSize.SuperResolutionUpscaledWidth); + Assert.Equal(width, frameSize.SuperResolutionUpscaledWidth); + if (!requireLoopRestoration) + { + // The original super-resolution fixtures isolate upscaling by disabling restoration. + Assert.False(decoder.FrameHeader.LoopRestorationParameters.UsesLoopRestoration); + } + } + + if (requireActiveLoopFilter) + { + ObuLoopFilterParameters filterParameters = decoder.FrameHeader.LoopFilterParameters; + Assert.True( + filterParameters.FilterLevel[0] != 0 + || filterParameters.FilterLevel[1] != 0 + || filterParameters.FilterLevelU != 0 + || filterParameters.FilterLevelV != 0); + } + + if (requireActiveCdef) + { + Assert.NotNull(decoder.SequenceHeader); + Assert.True(decoder.SequenceHeader.EnableCdef); + Assert.False(decoder.FrameHeader.LoopRestorationParameters.UsesLoopRestoration); + Assert.NotNull(decoder.FrameInfo); + ObuConstraintDirectionalEnhancementFilterParameters parameters = decoder.FrameHeader.CdefParameters; + bool hasActiveStrength = false; + int superblockSizeLog2 = decoder.SequenceHeader.SuperblockSizeLog2; + int superblockColumnCount = Av1Math.AlignPowerOf2(decoder.SequenceHeader.MaxFrameWidth, superblockSizeLog2) >> superblockSizeLog2; + int superblockRowCount = Av1Math.AlignPowerOf2(decoder.SequenceHeader.MaxFrameHeight, superblockSizeLog2) >> superblockSizeLog2; + for (int superblockRow = 0; superblockRow < superblockRowCount && !hasActiveStrength; superblockRow++) + { + for (int superblockColumn = 0; superblockColumn < superblockColumnCount && !hasActiveStrength; superblockColumn++) + { + Span selectedStrengths = decoder.FrameInfo.GetCdefStrength(new Point(superblockColumn, superblockRow)); + + // Unassigned entries belong to completely skipped units. Every assigned index must resolve through + // the signaled table before the exact output can establish that CDEF changed reconstructed samples. + foreach (int selectedStrength in selectedStrengths) + { + if (selectedStrength >= 0 + && (parameters.YStrength[selectedStrength] != 0 || parameters.UvStrength[selectedStrength] != 0)) + { + hasActiveStrength = true; + break; + } + } + } + } + + // The independent output only proves CDEF when the encoded frame selects at least one nonzero strength. + Assert.True(hasActiveStrength); + } + + if (requireLoopRestoration) + { + Assert.True(decoder.FrameHeader.LoopRestorationParameters.UsesLoopRestoration); + Assert.NotNull(decoder.FrameInfo); + restorationCoverage = GetRestorationCoverage(decoder); + Assert.NotEqual(0, restorationCoverage); + } + + if (requireFilmGrain) + { + Assert.True(decoder.FrameHeader.FilmGrainParameters.ApplyGrain); + } + + if (requireRestrictedRange) + { + Assert.True(decoder.FrameHeader.FilmGrainParameters.ClipToRestrictedRange); + } + + if (requireIdentityMatrix) + { + Assert.NotNull(decoder.SequenceHeader); + Assert.Equal(ObuMatrixCoefficients.Identity, decoder.SequenceHeader.ColorConfig.MatrixCoefficients); + } + + if (requirePalette) + { + Assert.Equal(RequiredPaletteCoverage, GetPaletteCoverage(decoder)); + } + + AssertNativePlanesEqual(decoder, frameBuffer, reference); + return restorationCoverage; + } + + /// + /// Validates lossless frame syntax and complete native reconstruction for one AVIF image. + /// + /// The independently encoded AVIF container. + /// The independently decoded raw planar output. + /// The expected AV1 sample precision. + private static void ValidateLosslessFixture(string imagePath, string referencePath, Av1BitDepth bitDepth) + { + byte[] imageBytes = TestFile.Create(imagePath).Bytes; + byte[] referenceBytes = TestFile.Create(referencePath).Bytes; + Span payload = GetSoleAv1ItemPayload(imageBytes); + ReadOnlySpan nativeReference = referenceBytes; + using Av1Decoder decoder = new(Configuration.Default); + using Av1FrameBuffer frameBuffer = decoder.DecodeFrameBuffer(payload, null, null, out _); + + Assert.Equal(LosslessFixtureWidth, frameBuffer.Width); + Assert.Equal(LosslessFixtureHeight, frameBuffer.Height); + Assert.Equal(bitDepth, frameBuffer.BitDepth); + Assert.Equal(Av1ColorFormat.Yuv444, frameBuffer.ColorFormat); + Assert.NotNull(decoder.SequenceHeader); + Assert.NotNull(decoder.FrameHeader); + Assert.NotNull(decoder.FrameInfo); + Assert.True(decoder.FrameHeader.CodedLossless); + Assert.True(decoder.FrameHeader.AllLossless); + Assert.Equal(0, decoder.FrameHeader.QuantizationParameters.BaseQIndex); + Assert.Equal(ObuMatrixCoefficients.Identity, decoder.SequenceHeader.ColorConfig.MatrixCoefficients); + Assert.False(decoder.FrameHeader.AllowIntraBlockCopy); + Assert.Equal(0, GetPaletteCoverage(decoder)); + + // Exact equality with independently decoded native planes proves the complete residual reconstruction contract. + AssertNativePlanesEqual(decoder, frameBuffer, nativeReference); + } + + /// + /// Validates one independently encoded stream that activates constrained directional enhancement filtering. + /// + /// The AV1 elementary-stream sample. + /// The independently decoded native planar output. + /// The expected reconstructed width. + /// The expected reconstructed height. + /// The expected AV1 sample precision. + /// The expected native chroma-sampling layout. + private static void ValidateActiveCdefFixture( + string payloadPath, + string referencePath, + int width, + int height, + Av1BitDepth bitDepth, + Av1ColorFormat colorFormat) + => ValidateNativeFixture(payloadPath, referencePath, width, height, bitDepth, colorFormat, requireActiveCdef: true); + + /// + /// Validates one independently encoded stream that activates normative super-resolution. + /// + /// The AV1 elementary-stream sample. + /// The independently decoded native planar output. + /// The expected upscaled width. + /// The expected reconstructed height. + /// The expected AV1 sample precision. + /// The expected native chroma-sampling layout. + private static void ValidateSuperResolutionFixture( + string payloadPath, + string referencePath, + int width, + int height, + Av1BitDepth bitDepth, + Av1ColorFormat colorFormat) + => ValidateNativeFixture( + payloadPath, + referencePath, + width, + height, + bitDepth, + colorFormat, + requireActiveCdef: false, + requireSuperResolution: true); + + /// + /// Validates one independently encoded stream that activates normative loop restoration. + /// + /// The AV1 elementary-stream sample. + /// The independently decoded native planar output. + /// The expected reconstructed width. + /// The expected reconstructed height. + /// The expected AV1 sample precision. + /// The expected native chroma-sampling layout. + /// Whether the stream must upscale from a narrower coded frame. + /// A bit mask containing every selected loop-restoration filter type. + private static int ValidateLoopRestorationFixture( + string payloadPath, + string referencePath, + int width, + int height, + Av1BitDepth bitDepth, + Av1ColorFormat colorFormat, + bool requireSuperResolution = false) + => ValidateNativeFixture( + payloadPath, + referencePath, + width, + height, + bitDepth, + colorFormat, + requireActiveCdef: false, + requireSuperResolution: requireSuperResolution, + requireLoopRestoration: true); + + /// + /// Validates one independently encoded stream that applies film grain to the displayed samples. + /// + /// The AV1 elementary-stream sample. + /// The independently decoded native planar output. + /// The expected displayed width. + /// The expected displayed height. + /// The expected AV1 sample precision. + /// The expected native chroma-sampling layout. + /// Whether film grain must clip every plane to its restricted range. + /// Whether restricted chroma clipping must use the luma endpoints. + private static void ValidateFilmGrainFixture( + string payloadPath, + string referencePath, + int width, + int height, + Av1BitDepth bitDepth, + Av1ColorFormat colorFormat, + bool requireRestrictedRange = false, + bool requireIdentityMatrix = false) + => ValidateNativeFixture( + payloadPath, + referencePath, + width, + height, + bitDepth, + colorFormat, + requireActiveCdef: false, + requireFilmGrain: true, + requireRestrictedRange: requireRestrictedRange, + requireIdentityMatrix: requireIdentityMatrix, + requireActiveLoopFilter: false); + + /// + /// Validates the public presentation and metadata produced from one complete AVIF container. + /// + /// The complete AVIF container. + /// The expected displayed width. + /// The expected displayed height. + /// The expected public HEIF sample precision. + private static void ValidatePresentedImage(string imagePath, int width, int height, HeifBitDepth metadataBitDepth) + { + DecoderOptions options = new() { MaxFrames = 1 }; + byte[] imageBytes = TestFile.Create(imagePath).Bytes; + using Image image = Image.Load(options, imageBytes); + + Assert.Equal(width, image.Width); + Assert.Equal(height, image.Height); + Assert.Single(image.Frames); + HeifMetadata metadata = image.Metadata.GetHeifMetadata(); + Assert.Equal(HeifCompressionMethod.Av1, metadata.CompressionMethod); + Assert.Equal(metadataBitDepth, metadata.BitDepth); + + if (metadataBitDepth != HeifBitDepth.Bit8) + { + bool containsPrecisionBeyondEightBits = false; + image.ProcessPixelRows(accessor => + { + for (int y = 0; y < accessor.Height && !containsPrecisionBeyondEightBits; y++) + { + Span row = accessor.GetRowSpan(y); + foreach (Rgba64 pixel in row) + { + // Expanding an eight-bit channel to ushort always produces a multiple of 257. At least one + // source-derived RGB channel must fall between those values to prove native precision survived. + if ((pixel.R % 257) != 0 || (pixel.G % 257) != 0 || (pixel.B % 257) != 0) + { + containsPrecisionBeyondEightBits = true; + break; + } + } + } + }); + + Assert.True(containsPrecisionBeyondEightBits, "The high-bit-depth presentation contains only eight-bit-expanded RGB samples."); + } + } + + /// + /// Verifies the public dimensions, frame count, compression method, and sample precision of one AVIF input. + /// + /// The AVIF input provider. + /// The expected displayed width. + /// The expected displayed height. + /// The expected public HEIF sample precision. + private static void AssertPresentedMetadata( + TestImageProvider provider, + int width, + int height, + HeifBitDepth bitDepth) + { + using Image image = provider.GetImage(); + Assert.Equal(width, image.Width); + Assert.Equal(height, image.Height); + Assert.Single(image.Frames); + + HeifMetadata metadata = image.Metadata.GetHeifMetadata(); + Assert.Equal(HeifCompressionMethod.Av1, metadata.CompressionMethod); + Assert.Equal(bitDepth, metadata.BitDepth); + } + + /// + /// Compares one AVIF presentation with its retained output through the repository reference-image contract. + /// + /// The serialized input provider and reference-output naming context. + private static void ValidatePresentedFixture(string providerDump) + { + TestImageProvider provider = + FeatureTestRunner.DeserializeForXunit>(providerDump); + + using Image image = provider.GetImage(); + + // CICP records the AVIF source component layout, but PNG permits only the identity matrix. The debug image + // is a pixel artifact; the test verifies source metadata independently where that is part of the contract. + image.DebugSave(provider, new PngEncoder { SkipMetadata = true }); + + image.CompareToReferenceOutput(ImageComparer.Exact, provider); + } + + /// + /// Compares the final visible frame of one AVIF sequence through the repository reference-image contract. + /// + /// The serialized input provider and reference-output naming context. + private static void ValidateFinalSequencePresentation(string providerDump) + { + TestImageProvider provider = + FeatureTestRunner.DeserializeForXunit>(providerDump); + + using Image sequence = provider.GetImage(); + using Image finalFrame = sequence.Frames.CloneFrame(sequence.Frames.Count - 1); + + // The retained source CICP matrix cannot be represented in a PNG cICP chunk. Omit metadata only from the + // diagnostic output; the exact reference comparison below still consumes the original decoded image. + finalFrame.DebugSave(provider, new PngEncoder { SkipMetadata = true }); + + finalFrame.CompareToReferenceOutput(ImageComparer.Exact, provider); + } + + /// + /// Verifies that the sole AV1 image item in an independently packaged AVIF uses normative super-resolution. + /// + /// The complete AVIF file. + private static void AssertUsesSuperResolution(Span imageBytes) + { + Span payload = GetSoleAv1ItemPayload(imageBytes); + using Av1Decoder decoder = new(Configuration.Default); + using Av1FrameBuffer frameBuffer = decoder.DecodeFrameBuffer(payload, null, null, out _); + + Assert.NotNull(decoder.FrameHeader); + ObuFrameSize frameSize = decoder.FrameHeader.FrameSize; + Assert.True(frameSize.FrameWidth < frameSize.SuperResolutionUpscaledWidth); + Assert.Equal(frameBuffer.Width, frameSize.SuperResolutionUpscaledWidth); + } + + /// + /// Verifies that the sole AV1 image item in an independently encoded AVIF selects luma and chroma palettes. + /// + /// The complete AVIF file. + private static void AssertUsesPalette(Span imageBytes) + { + Span payload = GetSoleAv1ItemPayload(imageBytes); + using Av1Decoder decoder = new(Configuration.Default); + using Av1FrameBuffer frameBuffer = decoder.DecodeFrameBuffer(payload, null, null, out _); + + Assert.Equal(RequiredPaletteCoverage, GetPaletteCoverage(decoder)); + } + + /// + /// Verifies that the sole AV1 image item in an independently encoded AVIF selects intra-block-copy prediction. + /// + /// The complete AVIF file. + private static void AssertUsesIntraBlockCopy(Span imageBytes) + { + Span payload = GetSoleAv1ItemPayload(imageBytes); + using Av1Decoder decoder = new(Configuration.Default); + using Av1FrameBuffer frameBuffer = decoder.DecodeFrameBuffer(payload, null, null, out _); + + Assert.NotNull(decoder.FrameHeader); + Assert.True(decoder.FrameHeader.AllowIntraBlockCopy); + Assert.NotEqual(0, GetIntraBlockCopyBlockCount(decoder)); + } + + /// + /// Decodes the sole image item in an independently generated AVIF fixture and returns its restoration coverage. + /// + /// The complete AVIF file. + /// A bit mask containing every selected loop-restoration filter type. + private static int GetRestorationCoverageFromAvif(Span imageBytes) + { + Span payload = GetSoleAv1ItemPayload(imageBytes); + using Av1Decoder decoder = new(Configuration.Default); + using Av1FrameBuffer frameBuffer = decoder.DecodeFrameBuffer(payload, null, null, out _); + + Assert.NotNull(decoder.FrameHeader); + Assert.True(decoder.FrameHeader.LoopRestorationParameters.UsesLoopRestoration); + Assert.NotNull(decoder.FrameInfo); + int restorationCoverage = GetRestorationCoverage(decoder); + Assert.NotEqual(0, restorationCoverage); + return restorationCoverage; + } + + /// + /// Gets the partition types selected by one independently encoded AV1 elementary stream. + /// + /// The AV1 elementary-stream sample. + /// A bit mask containing every selected partition type. + private static int GetPartitionCoverage(string payloadPath) + { + byte[] payload = TestFile.Create(payloadPath).Bytes; + using Av1Decoder decoder = new(Configuration.Default); + using Av1FrameBuffer frameBuffer = decoder.DecodeFrameBuffer(payload, null, null, out _); + + Assert.NotNull(decoder.SequenceHeader); + Assert.NotNull(decoder.FrameInfo); + int superblockSizeLog2 = decoder.SequenceHeader.SuperblockSizeLog2; + int superblockColumnCount = Av1Math.AlignPowerOf2(decoder.SequenceHeader.MaxFrameWidth, superblockSizeLog2) >> superblockSizeLog2; + int superblockRowCount = Av1Math.AlignPowerOf2(decoder.SequenceHeader.MaxFrameHeight, superblockSizeLog2) >> superblockSizeLog2; + int halfSuperblockSize = 1 << (superblockSizeLog2 - 1); + int coverage = 0; + + // Mode records retain their bitstream traversal order and store each final coding block once, so iterating + // the parsed count observes every selected leaf partition without repeatedly visiting its covered 4x4 cells. + // Split itself creates no mode record. All other partition types are terminal, so a leaf below half the + // superblock size on both axes proves that the parser reached it through at least one recursive split. + for (int superblockRow = 0; superblockRow < superblockRowCount; superblockRow++) + { + for (int superblockColumn = 0; superblockColumn < superblockColumnCount; superblockColumn++) + { + Av1SuperblockInfo superblock = decoder.FrameInfo.GetSuperblock(new Point(superblockColumn, superblockRow)); + foreach (Av1BlockModeInfo modeInfo in superblock.GetModeInfos()) + { + coverage |= 1 << (int)modeInfo.PartitionType; + if (modeInfo.BlockSize.GetWidth() < halfSuperblockSize && modeInfo.BlockSize.GetHeight() < halfSuperblockSize) + { + coverage |= 1 << (int)Av1PartitionType.Split; + } + } + } + } + + return coverage; + } + + /// + /// Gets the complete media-data payload from a single-item AVIF conformance fixture. + /// + /// The complete AVIF file. + /// The sole AV1 image-item payload. + private static Span GetSoleAv1ItemPayload(Span imageBytes) + { + int offset = 0; + while (offset < imageBytes.Length) + { + int headerLength = HeifBoxReader.ParseHeader(imageBytes[offset..], out long payloadLength, out Heif4CharCode boxType); + Assert.InRange(payloadLength, 0, int.MaxValue); + int payloadLength32 = (int)payloadLength; + + if (boxType == Heif4CharCode.Mdat) + { + // Every conformance container passed here deliberately stores its sole AV1 item as the complete + // mdat payload, so feature assertions inspect the exact bytes used by public presentation decoding. + return imageBytes.Slice(offset + headerLength, payloadLength32); + } + + offset = checked(offset + headerLength + payloadLength32); + } + + Assert.Fail("The AVIF fixture does not contain a media-data box."); + return []; + } + + /// + /// Returns the luma and chroma palette classes selected by a decoded frame. + /// + /// The decoder after tile parsing and reconstruction. + /// A bit mask containing the selected plane classes. + private static int GetPaletteCoverage(Av1Decoder decoder) + { + Assert.NotNull(decoder.FrameHeader); + Assert.NotNull(decoder.FrameInfo); + int modeInfoWidth = Av1Math.DivideLog2Ceiling(decoder.FrameHeader.FrameSize.FrameWidth, Av1Constants.ModeInfoSizeLog2); + int modeInfoHeight = Av1Math.DivideLog2Ceiling(decoder.FrameHeader.FrameSize.FrameHeight, Av1Constants.ModeInfoSizeLog2); + int paletteCoverage = 0; + for (int y = 0; y < modeInfoHeight; y++) + { + for (int x = 0; x < modeInfoWidth; x++) + { + Av1BlockModeInfo modeInfo = decoder.FrameInfo.GetModeInfoAt(new Point(x, y)); + if (modeInfo.GetPaletteSize(Av1PlaneType.Y) != 0) + { + paletteCoverage |= LumaPaletteCoverage; + } + + if (modeInfo.GetPaletteSize(Av1PlaneType.Uv) != 0) + { + paletteCoverage |= ChromaPaletteCoverage; + } + } + } + + return paletteCoverage; + } + + /// + /// Counts the final coding blocks that select intra-block-copy prediction. + /// + /// The decoder after tile parsing and reconstruction. + /// The number of selected intra-block-copy coding blocks. + private static int GetIntraBlockCopyBlockCount(Av1Decoder decoder) + { + Assert.NotNull(decoder.SequenceHeader); + Assert.NotNull(decoder.FrameInfo); + int superblockSizeLog2 = decoder.SequenceHeader.SuperblockSizeLog2; + int superblockColumnCount = Av1Math.AlignPowerOf2(decoder.SequenceHeader.MaxFrameWidth, superblockSizeLog2) >> superblockSizeLog2; + int superblockRowCount = Av1Math.AlignPowerOf2(decoder.SequenceHeader.MaxFrameHeight, superblockSizeLog2) >> superblockSizeLog2; + int blockCount = 0; + + // Mode records retain final coding blocks in bitstream order. Traversing each record once counts selected + // intra-block-copy operations without repeatedly visiting the 4x4 cells covered by a larger block. + for (int superblockRow = 0; superblockRow < superblockRowCount; superblockRow++) + { + for (int superblockColumn = 0; superblockColumn < superblockColumnCount; superblockColumn++) + { + Av1SuperblockInfo superblock = decoder.FrameInfo.GetSuperblock(new Point(superblockColumn, superblockRow)); + foreach (Av1BlockModeInfo modeInfo in superblock.GetModeInfos()) + { + if (modeInfo.UseIntraBlockCopy) + { + blockCount++; + } + } + } + } + + return blockCount; + } + + /// + /// Returns the restoration algorithms selected by the decoded frame's unit grids. + /// + /// The decoder after tile parsing and reconstruction. + /// A bit mask containing every selected loop-restoration filter type. + private static int GetRestorationCoverage(Av1Decoder decoder) + { + ObuSequenceHeader sequenceHeader = Assert.IsType(decoder.SequenceHeader); + Av1FrameInfo frameInfo = Assert.IsType(decoder.FrameInfo); + int restorationCoverage = 0; + for (int plane = 0; plane < sequenceHeader.ColorConfig.PlaneCount; plane++) + { + int rowCount = frameInfo.GetLoopRestorationUnitRowCount(plane); + int columnCount = frameInfo.GetLoopRestorationUnitColumnCount(plane); + for (int row = 0; row < rowCount; row++) + { + for (int column = 0; column < columnCount; column++) + { + Av1RestorationFilterType filterType = frameInfo.GetLoopRestorationUnit(plane, row, column).FilterType; + if (filterType != Av1RestorationFilterType.None) + { + restorationCoverage |= 1 << (int)filterType; + } + } + } + } + + return restorationCoverage; + } + + /// + /// Compares every visible native component sample with the independent planar reference. + /// + /// The decoder state used to identify the coded block containing a mismatch. + /// The reconstructed AV1 component planes. + /// The independently decoded planar Y, U, and V samples. + /// The zero-based sequence-frame index, or -1 for a standalone sample. + private static void AssertNativePlanesEqual( + Av1Decoder decoder, + Av1FrameBuffer frameBuffer, + ReadOnlySpan reference, + int frameIndex = -1) + { + (int chromaSubsamplingX, int chromaSubsamplingY) = frameBuffer.ColorFormat switch + { + Av1ColorFormat.Yuv420 => (1, 1), + Av1ColorFormat.Yuv422 => (1, 0), + _ => (0, 0) + }; + + int referenceOffset = 0; + ReadOnlySpan planes = frameBuffer.ColorFormat == Av1ColorFormat.Yuv400 + ? [Av1Plane.Y] + : [Av1Plane.Y, Av1Plane.U, Av1Plane.V]; + int mismatchCount = 0; + Av1Plane largestMismatchPlane = default; + int largestMismatchX = 0; + int largestMismatchY = 0; + ushort largestExpected = 0; + ushort largestActual = 0; + StringBuilder mismatchDescription = null; + + foreach (Av1Plane plane in planes) + { + int subsamplingX = plane == Av1Plane.Y ? 0 : chromaSubsamplingX; + int subsamplingY = plane == Av1Plane.Y ? 0 : chromaSubsamplingY; + int planeWidth = GetSubsampledSize(frameBuffer.Width, subsamplingX); + int planeHeight = GetSubsampledSize(frameBuffer.Height, subsamplingY); + + if (frameBuffer.BitDepth == Av1BitDepth.EightBit) + { + Buffer2DRegion actualPlane = frameBuffer.DeriveBlockPointer(plane, subsamplingX, subsamplingY); + for (int y = 0; y < planeHeight; y++) + { + Span actualRow = actualPlane.DangerousGetRowSpan(y)[..planeWidth]; + ReadOnlySpan expectedRow = reference.Slice(referenceOffset, planeWidth); + for (int x = 0; x < planeWidth; x++) + { + if (expectedRow[x] != actualRow[x]) + { + if (mismatchCount < 16) + { + mismatchDescription ??= new StringBuilder(); + mismatchDescription.Append(CultureInfo.InvariantCulture, $" {plane}({x},{y})={expectedRow[x]}/{actualRow[x]}"); + } + + if (mismatchCount == 0 || Math.Abs(expectedRow[x] - actualRow[x]) > Math.Abs(largestExpected - largestActual)) + { + largestMismatchPlane = plane; + largestMismatchX = x; + largestMismatchY = y; + largestExpected = expectedRow[x]; + largestActual = actualRow[x]; + } + + mismatchCount++; + } + } + + referenceOffset += planeWidth; + } + } + else + { + // aomdec writes high-bit-depth YUV as little-endian 16-bit values, independently of host endianness. + for (int y = 0; y < planeHeight; y++) + { + Span actualRow = frameBuffer.GetHighBitDepthRowSpan(plane, y, subsamplingX, subsamplingY); + for (int x = 0; x < planeWidth; x++) + { + ushort expected = BinaryPrimitives.ReadUInt16LittleEndian(reference.Slice(referenceOffset, sizeof(ushort))); + if (expected != actualRow[x]) + { + if (mismatchCount < 16) + { + mismatchDescription ??= new StringBuilder(); + mismatchDescription.Append(CultureInfo.InvariantCulture, $" {plane}({x},{y})={expected}/{actualRow[x]}"); + } + + if (mismatchCount == 0 || Math.Abs(expected - actualRow[x]) > Math.Abs(largestExpected - largestActual)) + { + largestMismatchPlane = plane; + largestMismatchX = x; + largestMismatchY = y; + largestExpected = expected; + largestActual = actualRow[x]; + } + + mismatchCount++; + } + + referenceOffset += sizeof(ushort); + } + } + } + } + + Assert.Equal(reference.Length, referenceOffset); + AssertSampleEqual( + decoder, + largestMismatchPlane, + largestMismatchX, + largestMismatchY, + largestExpected, + largestActual, + mismatchCount, + mismatchDescription?.ToString() ?? string.Empty, + frameIndex); + } + + /// + /// Calculates a component dimension after chroma subsampling with the AV1 rounding rule. + /// + /// The luma dimension. + /// The component subsampling shift. + /// The subsampled component dimension. + private static int GetSubsampledSize(int size, int subsampling) + => (size + (1 << subsampling) - 1) >> subsampling; + + /// + /// Reports the exact component coordinate when independently decoded samples differ. + /// + /// The decoder state used to identify the coded block containing the sample. + /// The compared component plane. + /// The sample X coordinate. + /// The sample Y coordinate. + /// The reference sample. + /// The reconstructed sample. + /// The total number of unequal native samples. + /// The first unequal samples in plane traversal order. + /// The zero-based sequence-frame index, or -1 for a standalone sample. + private static void AssertSampleEqual( + Av1Decoder decoder, + Av1Plane plane, + int x, + int y, + ushort expected, + ushort actual, + int mismatchCount, + string mismatchDescription, + int frameIndex) + { + if (expected != actual) + { + ObuSequenceHeader sequenceHeader = Assert.IsType(decoder.SequenceHeader); + int subsamplingX = plane == Av1Plane.Y || !sequenceHeader.ColorConfig.SubSamplingX ? 0 : 1; + int subsamplingY = plane == Av1Plane.Y || !sequenceHeader.ColorConfig.SubSamplingY ? 0 : 1; + Av1FrameInfo frameInfo = Assert.IsType(decoder.FrameInfo); + int modeInfoColumn = (x << subsamplingX) >> Av1Constants.ModeInfoSizeLog2; + int modeInfoRow = (y << subsamplingY) >> Av1Constants.ModeInfoSizeLog2; + Av1BlockModeInfo modeInfo = frameInfo.GetModeInfoAt(new Point(modeInfoColumn, modeInfoRow)); + int blockColumn = modeInfoColumn; + while (blockColumn > 0 && + frameInfo.GetModeInfoAt(new Point(blockColumn - 1, modeInfoRow)).ModeInfoIndex == modeInfo.ModeInfoIndex) + { + blockColumn--; + } + + int blockRow = modeInfoRow; + while (blockRow > 0 && + frameInfo.GetModeInfoAt(new Point(modeInfoColumn, blockRow - 1)).ModeInfoIndex == modeInfo.ModeInfoIndex) + { + blockRow--; + } + + int superblockSize = frameInfo.SuperblockModeInfoSize; + Av1SuperblockInfo superblock = frameInfo.GetSuperblock(new Point(blockColumn / superblockSize, blockRow / superblockSize)); + + ObuFrameHeader frameHeader = Assert.IsType(decoder.FrameHeader); + int cdefUnitColumn = (modeInfoColumn % superblockSize) / CdefUnitModeInfoSize; + int cdefUnitRow = (modeInfoRow % superblockSize) / CdefUnitModeInfoSize; + int cdefStrengthIndex = frameInfo.GetCdefStrength(superblock.Position)[cdefUnitColumn + (cdefUnitRow << 1)]; + int cdefStrength = cdefStrengthIndex < 0 ? -1 : frameHeader.CdefParameters.YStrength[cdefStrengthIndex]; + int nextModeInfoRow = Math.Min(modeInfoRow + 1, frameHeader.ModeInfoRowCount - 1); + Av1BlockModeInfo nextRowModeInfo = frameInfo.GetModeInfoAt(new Point(modeInfoColumn, nextModeInfoRow)); + Av1BlockModeInfo aboveModeInfo = frameInfo.GetModeInfoAt(new Point(modeInfoColumn, Math.Max(blockRow - 1, 0))); + Av1BlockModeInfo leftModeInfo = frameInfo.GetModeInfoAt(new Point(Math.Max(blockColumn - 1, 0), modeInfoRow)); + string frameDescription = frameIndex < 0 ? string.Empty : $"Frame {frameIndex}, "; + StringBuilder filmGrainCoefficientDescription = new(); + ReadOnlySpan filmGrainCoefficients = frameHeader.FilmGrainParameters.ArCoeffsYPlus128; + int filmGrainCoefficientCount = 2 * (int)frameHeader.FilmGrainParameters.ArCoeffLag * + ((int)frameHeader.FilmGrainParameters.ArCoeffLag + 1); + + for (int coefficientIndex = 0; coefficientIndex < filmGrainCoefficientCount; coefficientIndex++) + { + if (coefficientIndex != 0) + { + filmGrainCoefficientDescription.Append(','); + } + + filmGrainCoefficientDescription.Append((int)filmGrainCoefficients[coefficientIndex] - 128); + } + + // Exact conformance failures need the owning syntax state. A coordinate alone does not distinguish + // prediction, residual reconstruction, and in-loop filtering failures inside a large coded frame. + Assert.Fail( + $"{frameDescription}plane {plane} differs at ({x}, {y}): expected {expected}, actual {actual}. " + + $"Total unequal samples={mismatchCount}:{mismatchDescription}. " + + $"Block={modeInfo.BlockSize}, mode={modeInfo.YMode}, partition={modeInfo.PartitionType}, skip={modeInfo.Skip}, " + + $"refs={modeInfo.ReferenceFrames[0]}/{modeInfo.ReferenceFrames[1]}, " + + $"mvs={modeInfo.MotionVectors[0].Row},{modeInfo.MotionVectors[0].Column}/" + + $"{modeInfo.MotionVectors[1].Row},{modeInfo.MotionVectors[1].Column}, compound={modeInfo.CompoundType}, " + + $"filters={modeInfo.InterpolationFilters[0]}/{modeInfo.InterpolationFilters[1]}, motion={modeInfo.MotionMode}, " + + $"filter-intra={modeInfo.UseFilterIntra}/{modeInfo.FilterIntraMode}, angle-delta={modeInfo.GetAngleDelta(plane)}, " + + $"palette-size={modeInfo.GetPaletteSize(plane)}, transforms={modeInfo.GetTransformUnitCount(plane)}, " + + $"block-origin=({blockColumn}, {blockRow}). " + + $"Loop-filter={frameHeader.LoopFilterParameters.FilterLevel[0]}/{frameHeader.LoopFilterParameters.FilterLevel[1]}, " + + $"sharpness={frameHeader.LoopFilterParameters.SharpnessLevel}, delta-q={frameHeader.DeltaQParameters.IsPresent}, " + + $"superblock-q={superblock.SuperblockQuantizerIndex}, " + + $"delta-lf={frameHeader.DeltaLoopFilterParameters.IsPresent}/{frameHeader.DeltaLoopFilterParameters.IsMulti}, " + + $"CDEF={cdefStrengthIndex}/{cdefStrength}, restoration={frameHeader.LoopRestorationParameters.Items[0].Type}, " + + $"film-grain={frameHeader.FilmGrainParameters.ApplyGrain}/" + + $"overlap={frameHeader.FilmGrainParameters.OverlapFlag}/" + + $"update={frameHeader.FilmGrainParameters.UpdateGrain}/seed={frameHeader.FilmGrainParameters.GrainSeed}, " + + $"grain-points={frameHeader.FilmGrainParameters.NumYPoints}/" + + $"{frameHeader.FilmGrainParameters.NumCbPoints}/{frameHeader.FilmGrainParameters.NumCrPoints}, " + + $"grain-ar={frameHeader.FilmGrainParameters.ArCoeffLag}/" + + $"{frameHeader.FilmGrainParameters.ArCoeffShiftMinus6}, " + + $"grain-y-coefficients=[{filmGrainCoefficientDescription}], " + + $"grain-scale={frameHeader.FilmGrainParameters.GrainScalingMinus8}/" + + $"{frameHeader.FilmGrainParameters.GrainScaleShift}, " + + $"tiles={frameHeader.TilesInfo.TileColumnCount}x{frameHeader.TilesInfo.TileRowCount}, " + + $"first-tile-end=({frameHeader.TilesInfo.TileColumnStartModeInfo[1]}, {frameHeader.TilesInfo.TileRowStartModeInfo[1]}). " + + $"Neighbors: above={aboveModeInfo.BlockSize}/{aboveModeInfo.YMode}/skip={aboveModeInfo.Skip}, " + + $"left={leftModeInfo.BlockSize}/{leftModeInfo.YMode}/skip={leftModeInfo.Skip}, " + + $"next-row={nextRowModeInfo.BlockSize}/{nextRowModeInfo.YMode}/skip={nextRowModeInfo.Skip}/" + + $"angle-delta={nextRowModeInfo.GetAngleDelta(plane)}/transforms={nextRowModeInfo.GetTransformUnitCount(plane)}."); + } + } +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReferenceFrameStoreTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReferenceFrameStoreTests.cs new file mode 100644 index 0000000000..dad60ee05a --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReferenceFrameStoreTests.cs @@ -0,0 +1,857 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Buffers; +using System.Runtime.InteropServices; +using SixLabors.ImageSharp.Formats.Heif.Av1; +using SixLabors.ImageSharp.Formats.Heif.Av1.Entropy; +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.FilmGrain; +using SixLabors.ImageSharp.Formats.Heif.Av1.ReferenceFrames; +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; +using SixLabors.ImageSharp.Memory; +using SixLabors.ImageSharp.Tests.Memory; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1; + +/// +/// Verifies AV1 frame ownership, padded-buffer copying, and reference-border preservation. +/// +[Trait("Format", "Avif")] +[ValidateDisposedMemoryAllocations] +public class Av1ReferenceFrameStoreTests +{ + /// + /// Verifies that a zero refresh mask leaves the slot map and caller ownership unchanged. + /// + [Fact] + public void ApplyZeroRefreshMaskPreservesCallerOwnership() + { + using Av1ReferenceFrameStore store = new(); + using Av1ReferenceFrame frame = CreateFrame(); + + Assert.False(store.Commit(0, frame, showFrame: false)); + Assert.Null(store.Resolve(0)); + Assert.NotNull(frame.FrameBuffer.BufferY); + } + + /// + /// Verifies that every selected slot shares the one transferred frame owner. + /// + [Fact] + public void ApplyRefreshMaskStoresFrameInEverySelectedSlot() + { + using Av1ReferenceFrameStore store = new(); + Av1ReferenceFrame frame = CreateFrame(); + + Assert.True(store.Commit(0b1000_0101, frame, showFrame: false)); + Assert.Same(frame, store.Resolve(0)); + Assert.Null(store.Resolve(1)); + Assert.Same(frame, store.Resolve(2)); + Assert.Same(frame, store.Resolve(7)); + } + + /// + /// Verifies that a shown frame transfers to presentation ownership even when it refreshes no reference slot. + /// + [Fact] + public void ShownFrameWithZeroRefreshMaskTransfersOwnership() + { + using Av1ReferenceFrameStore store = new(); + Av1ReferenceFrame frame = CreateFrame(); + + Assert.True(store.Commit(0, frame, showFrame: true)); + Assert.Same(frame, store.OutputFrame); + Assert.Null(store.Resolve(0)); + } + + /// + /// Verifies that replacing the shown output does not release its predecessor while a reference slot retains it. + /// + [Fact] + public void ReplacingOutputPreservesReferencedPredecessor() + { + using Av1ReferenceFrameStore store = new(); + Av1ReferenceFrame firstOutput = CreateFrame(); + Av1ReferenceFrame secondOutput = CreateFrame(); + Av1ReferenceFrame replacementReference = CreateFrame(); + Av1FrameBuffer firstOutputBuffer = firstOutput.FrameBuffer; + store.Commit(0b0000_0001, firstOutput, showFrame: true); + + store.Commit(0, secondOutput, showFrame: true); + + Assert.NotNull(firstOutputBuffer.BufferY); + Assert.Same(firstOutput, store.Resolve(0)); + + store.Commit(0b0000_0001, replacementReference, showFrame: false); + + Assert.Null(firstOutputBuffer.BufferY); + } + + /// + /// Verifies that an independently committed presentation output does not release the ungrained owner retained by a reference slot. + /// + [Fact] + public void CommitOutputPreservesReferencedPreviousOutput() + { + using Av1ReferenceFrameStore store = new(); + Av1ReferenceFrame retainedReference = CreateFrame(); + Av1ReferenceFrame presentation = CreateFrame(); + Av1ReferenceFrame nextPresentation = CreateFrame(); + Av1ReferenceFrame replacementReference = CreateFrame(); + Av1FrameBuffer retainedReferenceBuffer = retainedReference.FrameBuffer; + Av1FrameBuffer presentationBuffer = presentation.FrameBuffer; + store.Commit(0b0000_0001, retainedReference, showFrame: true); + + store.CommitOutput(presentation); + + Assert.Same(presentation, store.OutputFrame); + Assert.Same(retainedReference, store.Resolve(0)); + Assert.NotNull(retainedReferenceBuffer.BufferY); + + store.CommitOutput(nextPresentation); + + Assert.Same(nextPresentation, store.OutputFrame); + Assert.Null(presentationBuffer.BufferY); + Assert.NotNull(retainedReferenceBuffer.BufferY); + + store.Commit(0b0000_0001, replacementReference, showFrame: false); + + Assert.Null(retainedReferenceBuffer.BufferY); + } + + /// + /// Verifies that replacing one alias does not release a frame retained by another slot. + /// + [Fact] + public void PartialReplacementPreservesSharedOwner() + { + using Av1ReferenceFrameStore store = new(); + Av1ReferenceFrame sharedFrame = CreateFrame(); + Av1ReferenceFrame replacement = CreateFrame(); + store.Commit(0b0000_0011, sharedFrame, showFrame: false); + + store.Commit(0b0000_0001, replacement, showFrame: false); + + Assert.Same(replacement, store.Resolve(0)); + Assert.Same(sharedFrame, store.Resolve(1)); + Assert.NotNull(sharedFrame.FrameBuffer.BufferY); + } + + /// + /// Verifies that replacing the final alias releases the displaced frame planes. + /// + [Fact] + public void FinalReplacementReleasesDisplacedOwner() + { + using Av1ReferenceFrameStore store = new(); + Av1ReferenceFrame displacedFrame = CreateFrame(); + Av1ReferenceFrame firstReplacement = CreateFrame(); + Av1ReferenceFrame secondReplacement = CreateFrame(); + Av1FrameBuffer displacedBuffer = displacedFrame.FrameBuffer; + store.Commit(0b0000_0011, displacedFrame, showFrame: false); + store.Commit(0b0000_0001, firstReplacement, showFrame: false); + + store.Commit(0b0000_0010, secondReplacement, showFrame: false); + + Assert.Null(displacedBuffer.BufferY); + Assert.Same(firstReplacement, store.Resolve(0)); + Assert.Same(secondReplacement, store.Resolve(1)); + } + + /// + /// Verifies that resetting the map releases one multiply referenced owner and clears every slot. + /// + [Fact] + public void ResetReleasesUniqueOwnersAndClearsSlots() + { + using Av1ReferenceFrameStore store = new(); + Av1ReferenceFrame frame = CreateFrame(); + Av1FrameBuffer frameBuffer = frame.FrameBuffer; + store.Commit(byte.MaxValue, frame, showFrame: false); + + store.Reset(); + + Assert.Null(frameBuffer.BufferY); + for (int slot = 0; slot < Av1Constants.ReferenceFrameCount; slot++) + { + Assert.Null(store.Resolve(slot)); + } + } + + /// + /// Verifies that taking the final output transfers its planes without copying and releases unrelated references. + /// + [Fact] + public void TakeOutputTransfersPlanesAndReleasesOtherReferences() + { + using Av1ReferenceFrameStore store = new(); + Av1ReferenceFrame reference = CreateFrame(); + Av1ReferenceFrame output = CreateFrame(); + Av1FrameBuffer referenceBuffer = reference.FrameBuffer; + store.Commit(0b0000_0001, reference, showFrame: false); + store.Commit(0b0000_0010, output, showFrame: true); + + using Av1ReferenceFrame selectedOutput = store.TakeOutput(); + using Av1FrameBuffer frameBuffer = selectedOutput.TakeFrameBuffer(); + + Assert.Same(output, selectedOutput); + Assert.Null(referenceBuffer.BufferY); + Assert.NotNull(frameBuffer.BufferY); + Assert.Null(store.OutputFrame); + Assert.Null(store.Resolve(0)); + Assert.Null(store.Resolve(1)); + } + + /// + /// Verifies motion-field ownership across frame initialization, reference aliases, shown output, and final disposal. + /// + [Fact] + public void MotionFieldsFollowAliasesAndPresentationOwnership() + { + TestMemoryAllocator allocator = new(); + allocator.EnableNonThreadSafeLogging(); + Configuration configuration = Configuration.Default.Clone(); + configuration.MemoryAllocator = allocator; + ObuSequenceHeader sequenceHeader = CreateSequenceHeader(64, 64, Av1BitDepth.EightBit, true, false, false); + sequenceHeader.OrderHintInfo.EnableOrderHint = true; + sequenceHeader.OrderHintInfo.EnableReferenceFrameMotionVectors = true; + + using Av1ReferenceFrameStore sourceReferences = new(); + using Av1FrameInfo sourceFrameInfo = new(sequenceHeader); + ObuFrameHeader sourceHeader = new() + { + FrameType = ObuFrameType.KeyFrame, + ShowFrame = true, + OrderHint = 0, + ModeInfoColumnCount = 16, + ModeInfoRowCount = 16 + }; + + Av1ReferenceFrame sourceFrame = new( + new Av1FrameBuffer(Configuration.Default, sequenceHeader, Av1ColorFormat.Yuv400, false), + sourceHeader, + sourceFrameInfo); + + Assert.True(sourceReferences.Commit(byte.MaxValue, sourceFrame, showFrame: false)); + + ObuFrameHeader frameHeader = new() + { + FrameType = ObuFrameType.InterFrame, + OrderHint = 1, + ModeInfoColumnCount = 16, + ModeInfoRowCount = 16, + UseReferenceFrameMotionVectors = true + }; + + using Av1FrameInfo frameInfo = new(sequenceHeader); + frameInfo.InitializeMotionField(configuration, sequenceHeader, frameHeader, sourceReferences); + + Assert.Equal(2, allocator.AllocationLog.Count); + TestMemoryAllocator.AllocationRequest retainedMotionField = + Assert.Single(allocator.AllocationLog, request => request.ElementType.Name == "RetainedMotionFieldEntry"); + + TestMemoryAllocator.AllocationRequest temporalMotionField = + Assert.Single(allocator.AllocationLog, request => request.ElementType.Name == "TemporalMotionFieldEntry"); + + using Av1ReferenceFrameStore store = new(); + Av1ReferenceFrame frame = new( + new Av1FrameBuffer(Configuration.Default, sequenceHeader, Av1ColorFormat.Yuv400, false), + frameHeader, + frameInfo); + + Assert.True(store.Commit(byte.MaxValue, frame, showFrame: true)); + + // Full current-frame state owns the projected temporal field. The retained frame owns only the source field + // needed by later projections, irrespective of how many map and output aliases identify the same frame. + frameInfo.Dispose(); + Assert.Single(allocator.ReturnLog, returned => returned.AllocationId == temporalMotionField.AllocationId); + + Av1ReferenceFrame output = store.TakeOutput(); + Assert.DoesNotContain(allocator.ReturnLog, returned => returned.AllocationId == retainedMotionField.AllocationId); + + output.Dispose(); + output.Dispose(); + store.Dispose(); + + Assert.All( + allocator.AllocationLog, + allocation => Assert.Single( + allocator.ReturnLog, + returned => returned.AllocationId == allocation.AllocationId)); + + Assert.Equal(2, allocator.ReturnLog.Count); + } + + /// + /// Verifies that tile-reader construction unwinds every successful allocation when temporal-field allocation fails. + /// + [Fact] + public void MotionFieldAllocationFailureUnwindsTileReaderOwnership() + { + FailingTemporalMotionFieldAllocator allocator = new(); + Configuration configuration = Configuration.Default.Clone(); + configuration.MemoryAllocator = allocator; + ObuSequenceHeader sequenceHeader = CreateSequenceHeader(64, 64, Av1BitDepth.EightBit, true, false, false); + sequenceHeader.OrderHintInfo.EnableOrderHint = true; + sequenceHeader.OrderHintInfo.EnableReferenceFrameMotionVectors = true; + + using Av1ReferenceFrameStore referenceFrames = new(); + using Av1FrameInfo retainedFrameInfo = new(sequenceHeader); + ObuFrameHeader retainedHeader = new() + { + FrameType = ObuFrameType.KeyFrame, + ShowFrame = true, + ModeInfoColumnCount = 16, + ModeInfoRowCount = 16 + }; + + Av1ReferenceFrame retainedFrame = new( + new Av1FrameBuffer(Configuration.Default, sequenceHeader, Av1ColorFormat.Yuv400, false), + retainedHeader, + retainedFrameInfo); + + Assert.True(referenceFrames.Commit(byte.MaxValue, retainedFrame, showFrame: false)); + + ObuFrameHeader frameHeader = new() + { + FrameType = ObuFrameType.InterFrame, + OrderHint = 1, + FrameSize = new ObuFrameSize + { + FrameWidth = 64, + FrameHeight = 64 + }, + ModeInfoColumnCount = 16, + ModeInfoRowCount = 16, + UseReferenceFrameMotionVectors = true + }; + + Av1FrameEntropyContexts entropyContexts = new(0); + + Assert.Throws( + () => new Av1TileReader( + configuration, + sequenceHeader, + frameHeader, + entropyContexts, + null, + referenceFrames)); + + Assert.NotEmpty(allocator.AllocationLog); + Assert.All( + allocator.AllocationLog, + allocation => Assert.Single( + allocator.ReturnLog, + returned => returned.AllocationId == allocation.AllocationId)); + + Assert.Equal(allocator.AllocationLog.Count, allocator.ReturnLog.Count); + } + + /// + /// Verifies that an eight-bit presentation copy contains each visible plane without copying decoder padding. + /// + [Fact] + public void CopyVisibleToCopiesEightBitPictureWithoutPadding() + => ValidateVisibleFrameCopy(Av1BitDepth.EightBit); + + /// + /// Verifies that a high-bit-depth presentation copy contains each visible plane without copying decoder padding. + /// + [Fact] + public void CopyVisibleToCopiesHighBitDepthPictureWithoutPadding() + => ValidateVisibleFrameCopy(Av1BitDepth.TwelveBit); + + /// + /// Verifies that luma and subsampled chroma allocations cover the greatest legal unscaled UMV prediction extent. + /// + [Fact] + public void PaddedPlanesCoverMaximumUnscaledMotionVectorExtent() + { + const int maximumLumaExtent = 135; + const int maximumSubsampledExtent = 71; + ObuSequenceHeader sequenceHeader = CreateSequenceHeader(128, 128, Av1BitDepth.EightBit, false, true, true); + using Av1FrameBuffer frameBuffer = new(Configuration.Default, sequenceHeader, Av1ColorFormat.Yuv420, false); + + Span luma = frameBuffer.GetPaddedPlaneSpan(Av1Plane.Y, 0, 0, out int lumaStride, out Point lumaOrigin); + int lumaHeight = luma.Length / lumaStride; + + Assert.True(lumaOrigin.X >= maximumLumaExtent); + Assert.True(lumaOrigin.Y >= maximumLumaExtent); + Assert.True(lumaStride - lumaOrigin.X - frameBuffer.Width >= maximumLumaExtent); + Assert.True(lumaHeight - lumaOrigin.Y - frameBuffer.Height >= maximumLumaExtent); + + Span chroma = frameBuffer.GetPaddedPlaneSpan(Av1Plane.U, 1, 1, out int chromaStride, out Point chromaOrigin); + int chromaWidth = Av1Math.DivideLog2Ceiling(frameBuffer.Width, 1); + int chromaHeight = Av1Math.DivideLog2Ceiling(frameBuffer.Height, 1); + int chromaAllocationHeight = chroma.Length / chromaStride; + + Assert.True(chromaOrigin.X >= maximumSubsampledExtent); + Assert.True(chromaOrigin.Y >= maximumSubsampledExtent); + Assert.True(chromaStride - chromaOrigin.X - chromaWidth >= maximumSubsampledExtent); + Assert.True(chromaAllocationHeight - chromaOrigin.Y - chromaHeight >= maximumSubsampledExtent); + } + + /// + /// Verifies that AV1 reference-border extension repeats the nearest visible edge across every allocated plane sample. + /// + /// The coded sample precision. + /// Whether the frame contains only luma. + /// Whether chroma is horizontally subsampled. + /// Whether chroma is vertically subsampled. + [Theory] + [InlineData(Av1BitDepth.EightBit, true, false, false)] + [InlineData(Av1BitDepth.EightBit, false, true, true)] + [InlineData(Av1BitDepth.TenBit, true, false, false)] + [InlineData(Av1BitDepth.TenBit, false, true, false)] + [InlineData(Av1BitDepth.TwelveBit, true, false, false)] + [InlineData(Av1BitDepth.TwelveBit, false, true, true)] + public void ExtendRepeatsVisibleEdgesAcrossCompletePadding( + int bitDepth, + bool isMonochrome, + bool subsamplingX, + bool subsamplingY) + { + ObuSequenceHeader sequenceHeader = CreateSequenceHeader( + 5, + 3, + (Av1BitDepth)bitDepth, + isMonochrome, + subsamplingX, + subsamplingY); + + using Av1FrameBuffer frameBuffer = new( + Configuration.Default, + sequenceHeader, + sequenceHeader.ColorConfig.GetColorFormat(), + false); + + InitializeVisiblePlane( + frameBuffer, + frameBuffer.GetPlaneBuffer(Av1Plane.Y), + frameBuffer.OriginX, + frameBuffer.OriginY, + frameBuffer.Width, + frameBuffer.Height, + 0); + + if (!isMonochrome) + { + int subX = subsamplingX ? 1 : 0; + int subY = subsamplingY ? 1 : 0; + int chromaOriginX = frameBuffer.OriginX >> subX; + int chromaOriginY = frameBuffer.OriginY >> subY; + int chromaWidth = Av1Math.DivideLog2Ceiling(frameBuffer.Width, subX); + int chromaHeight = Av1Math.DivideLog2Ceiling(frameBuffer.Height, subY); + + InitializeVisiblePlane(frameBuffer, frameBuffer.GetPlaneBuffer(Av1Plane.U), chromaOriginX, chromaOriginY, chromaWidth, chromaHeight, 1); + InitializeVisiblePlane(frameBuffer, frameBuffer.GetPlaneBuffer(Av1Plane.V), chromaOriginX, chromaOriginY, chromaWidth, chromaHeight, 2); + } + + Av1ReferenceFrameBorder.Extend(frameBuffer); + + AssertExtendedPlane( + frameBuffer, + frameBuffer.GetPlaneBuffer(Av1Plane.Y), + frameBuffer.OriginX, + frameBuffer.OriginY, + frameBuffer.Width, + frameBuffer.Height, + 0); + + if (!isMonochrome) + { + int subX = subsamplingX ? 1 : 0; + int subY = subsamplingY ? 1 : 0; + int chromaOriginX = frameBuffer.OriginX >> subX; + int chromaOriginY = frameBuffer.OriginY >> subY; + int chromaWidth = Av1Math.DivideLog2Ceiling(frameBuffer.Width, subX); + int chromaHeight = Av1Math.DivideLog2Ceiling(frameBuffer.Height, subY); + + AssertExtendedPlane(frameBuffer, frameBuffer.GetPlaneBuffer(Av1Plane.U), chromaOriginX, chromaOriginY, chromaWidth, chromaHeight, 1); + AssertExtendedPlane(frameBuffer, frameBuffer.GetPlaneBuffer(Av1Plane.V), chromaOriginX, chromaOriginY, chromaWidth, chromaHeight, 2); + } + } + + /// + /// Verifies that a refreshed shown frame retains its ungrained reconstruction while exposing an independently grained output. + /// + [Fact] + public void GrainedPresentationPreservesUngrainedReference() + { + ObuSequenceHeader sequenceHeader = CreateSequenceHeader(8, 8, Av1BitDepth.EightBit, true, false, false); + ObuFrameHeader frameHeader = new() + { + ShowFrame = true, + RefreshFrameFlags = 1, + FilmGrainParameters = new ObuFilmGrainParameters + { + ApplyGrain = true, + GrainSeed = 7391, + NumYPoints = 2, + GrainScalingMinus8 = 0, + ArCoeffLag = 0, + ArCoeffShiftMinus6 = 0, + GrainScaleShift = 0 + } + }; + + frameHeader.FilmGrainParameters.PointYValue[0] = 0; + frameHeader.FilmGrainParameters.PointYValue[1] = 255; + frameHeader.FilmGrainParameters.PointYScaling[0] = 255; + frameHeader.FilmGrainParameters.PointYScaling[1] = 255; + + Av1FrameBuffer reconstructed = new(Configuration.Default, sequenceHeader, Av1ColorFormat.Yuv400, false); + InitializeVisiblePlane( + reconstructed, + reconstructed.GetPlaneBuffer(Av1Plane.Y), + reconstructed.OriginX, + reconstructed.OriginY, + reconstructed.Width, + reconstructed.Height, + 0); + + Av1ReferenceFrameBorder.Extend(reconstructed); + Span reconstructedSamples = reconstructed.GetPlaneBuffer(Av1Plane.Y).DangerousGetSingleSpan(); + byte[] ungrainedSamples = new byte[reconstructedSamples.Length]; + reconstructedSamples.CopyTo(ungrainedSamples); + Av1FrameBuffer presentation = new(Configuration.Default, sequenceHeader, Av1ColorFormat.Yuv400, false); + reconstructed.CopyVisibleTo(presentation); + + Av1FilmGrainDecoder filmGrainDecoder = new(sequenceHeader, frameHeader, presentation); + filmGrainDecoder.DecodeFrame(); + + using Av1FrameInfo frameInfo = new(sequenceHeader); + Av1ReferenceFrame retainedReference = new(reconstructed, frameHeader, frameInfo); + Av1ReferenceFrame grainedOutput = new(presentation, frameHeader); + Av1FrameBuffer retainedReferenceBuffer = retainedReference.FrameBuffer; + using Av1ReferenceFrameStore store = new(); + store.Commit(frameHeader.RefreshFrameFlags, retainedReference, showFrame: false); + store.CommitOutput(grainedOutput); + + Assert.Same(retainedReference, store.Resolve(0)); + Assert.Same(grainedOutput, store.OutputFrame); + Assert.NotSame(retainedReference.FrameBuffer, grainedOutput.FrameBuffer); + Assert.True(ungrainedSamples.AsSpan().SequenceEqual(retainedReference.FrameBuffer.GetPlaneBuffer(Av1Plane.Y).DangerousGetSingleSpan())); + Assert.False(ungrainedSamples.AsSpan().SequenceEqual(grainedOutput.FrameBuffer.GetPlaneBuffer(Av1Plane.Y).DangerousGetSingleSpan())); + + using Av1ReferenceFrame selectedOutput = store.TakeOutput(); + + Assert.Same(grainedOutput, selectedOutput); + Assert.Null(retainedReferenceBuffer.BufferY); + Assert.NotNull(selectedOutput.FrameBuffer.BufferY); + Assert.Null(store.OutputFrame); + Assert.Null(store.Resolve(0)); + } + + /// + /// Verifies a visible-plane copy for one native AV1 sample precision. + /// + /// The coded sample precision. + private static void ValidateVisibleFrameCopy(Av1BitDepth bitDepth) + { + ObuSequenceHeader sequenceHeader = CreateSequenceHeader(5, 3, bitDepth, false, true, true); + using Av1FrameBuffer source = new(Configuration.Default, sequenceHeader, Av1ColorFormat.Yuv420, false); + using Av1FrameBuffer destination = new(Configuration.Default, sequenceHeader, Av1ColorFormat.Yuv420, false); + + source.OriginX--; + source.OriginY--; + source.Width = 4; + source.Height = 2; + source.MaxWidth = 4; + source.MaxHeight = 2; + Buffer2D sourceY = source.GetPlaneBuffer(Av1Plane.Y); + Buffer2D sourceCb = source.GetPlaneBuffer(Av1Plane.U); + Buffer2D sourceCr = source.GetPlaneBuffer(Av1Plane.V); + Buffer2D destinationY = destination.GetPlaneBuffer(Av1Plane.Y); + Buffer2D destinationCb = destination.GetPlaneBuffer(Av1Plane.U); + Buffer2D destinationCr = destination.GetPlaneBuffer(Av1Plane.V); + FillCompletePlane(source, sourceY, 17); + FillCompletePlane(source, sourceCb, 53); + FillCompletePlane(source, sourceCr, 89); + destinationY.DangerousGetSingleSpan().Fill(0xA5); + destinationCb.DangerousGetSingleSpan().Fill(0xA5); + destinationCr.DangerousGetSingleSpan().Fill(0xA5); + + source.CopyVisibleTo(destination); + + AssertVisiblePlaneCopy(source, destination, Av1Plane.Y, 0, 0); + AssertVisiblePlaneCopy(source, destination, Av1Plane.U, 1, 1); + AssertVisiblePlaneCopy(source, destination, Av1Plane.V, 1, 1); + Assert.Equal(source.StartPosition, destination.StartPosition); + Assert.Equal(source.OriginX, destination.OriginX); + Assert.Equal(source.OriginY, destination.OriginY); + Assert.Equal(source.Width, destination.Width); + Assert.Equal(source.Height, destination.Height); + Assert.Equal(source.MaxWidth, destination.MaxWidth); + Assert.Equal(source.MaxHeight, destination.MaxHeight); + Assert.Equal(source.BitDepth, destination.BitDepth); + Assert.Equal(source.ColorFormat, destination.ColorFormat); + + int visibleStorageOffset = (source.OriginY * sourceY.Width) + (source.OriginX * source.BytesPerSample); + byte sourceFirstVisibleByte = sourceY.DangerousGetSingleSpan()[visibleStorageOffset]; + + // Mutating a copied visible sample proves presentation ownership, not merely the already-untouched padding. + destinationY.DangerousGetSingleSpan()[visibleStorageOffset] ^= byte.MaxValue; + Assert.Equal(sourceFirstVisibleByte, sourceY.DangerousGetSingleSpan()[visibleStorageOffset]); + } + + /// + /// Verifies the copied visible rectangle and the untouched destination padding for one plane. + /// + /// The source frame. + /// The copied frame. + /// The plane to inspect. + /// The horizontal chroma subsampling shift. + /// The vertical chroma subsampling shift. + private static void AssertVisiblePlaneCopy( + Av1FrameBuffer source, + Av1FrameBuffer destination, + Av1Plane plane, + int subX, + int subY) + { + Buffer2D sourceBuffer = source.GetPlaneBuffer(plane); + Buffer2D destinationBuffer = destination.GetPlaneBuffer(plane); + int originX = (source.OriginX >> subX) * source.BytesPerSample; + int originY = source.OriginY >> subY; + int width = Av1Math.DivideLog2Ceiling(source.Width, subX) * source.BytesPerSample; + int height = Av1Math.DivideLog2Ceiling(source.Height, subY); + + for (int row = 0; row < destinationBuffer.Height; row++) + { + ReadOnlySpan sourceRow = sourceBuffer.DangerousGetRowSpan(row); + ReadOnlySpan destinationRow = destinationBuffer.DangerousGetRowSpan(row); + + for (int column = 0; column < destinationRow.Length; column++) + { + bool isVisible = row >= originY && row < originY + height && + column >= originX && column < originX + width; + + Assert.Equal(isVisible ? sourceRow[column] : (byte)0xA5, destinationRow[column]); + } + } + } + + /// + /// Fills every storage element of one padded plane with deterministic native sample data. + /// + /// The frame that defines the native sample size. + /// The complete padded plane. + /// The plane-specific value mixed into each sample. + private static void FillCompletePlane(Av1FrameBuffer frameBuffer, Buffer2D buffer, int seed) + { + if (frameBuffer.BytesPerSample == 1) + { + Span samples = buffer.DangerousGetSingleSpan(); + + for (int i = 0; i < samples.Length; i++) + { + samples[i] = (byte)((seed + (i * 17)) & byte.MaxValue); + } + + return; + } + + Span highBitDepthSamples = MemoryMarshal.Cast(buffer.DangerousGetSingleSpan()); + + for (int i = 0; i < highBitDepthSamples.Length; i++) + { + highBitDepthSamples[i] = (ushort)((seed + (i * 29)) & 0xFFF); + } + } + + /// + /// Initializes one visible plane with unique samples while leaving a distinct sentinel throughout its padding. + /// + /// The frame that defines the native sample size. + /// The padded plane to initialize. + /// The horizontal visible origin in plane samples. + /// The vertical visible origin in rows. + /// The visible plane width. + /// The visible plane height. + /// The zero-based plane index mixed into the visible samples. + private static void InitializeVisiblePlane( + Av1FrameBuffer frameBuffer, + Buffer2D buffer, + int originX, + int originY, + int width, + int height, + int planeIndex) + { + if (frameBuffer.BytesPerSample == 1) + { + Span samples = buffer.DangerousGetSingleSpan(); + samples.Fill(byte.MaxValue); + int stride = buffer.Width; + + for (int row = 0; row < height; row++) + { + for (int column = 0; column < width; column++) + { + samples[((originY + row) * stride) + originX + column] = (byte)GetVisibleSample(planeIndex, row, column); + } + } + + return; + } + + Span highBitDepthSamples = MemoryMarshal.Cast(buffer.DangerousGetSingleSpan()); + highBitDepthSamples.Fill(ushort.MaxValue); + int highBitDepthStride = buffer.Width >> 1; + + for (int row = 0; row < height; row++) + { + for (int column = 0; column < width; column++) + { + highBitDepthSamples[((originY + row) * highBitDepthStride) + originX + column] = + (ushort)GetVisibleSample(planeIndex, row, column); + } + } + } + + /// + /// Verifies every sample in one padded plane against nearest-edge replication of the initialized visible rectangle. + /// + /// The frame that defines the native sample size. + /// The padded plane to verify. + /// The horizontal visible origin in plane samples. + /// The vertical visible origin in rows. + /// The visible plane width. + /// The visible plane height. + /// The zero-based plane index mixed into the visible samples. + private static void AssertExtendedPlane( + Av1FrameBuffer frameBuffer, + Buffer2D buffer, + int originX, + int originY, + int width, + int height, + int planeIndex) + { + int stride = buffer.Width / frameBuffer.BytesPerSample; + int allocatedHeight = buffer.Height; + + if (frameBuffer.BytesPerSample == 1) + { + ReadOnlySpan samples = buffer.DangerousGetSingleSpan(); + + for (int row = 0; row < allocatedHeight; row++) + { + int visibleRow = Math.Clamp(row - originY, 0, height - 1); + + for (int column = 0; column < stride; column++) + { + int visibleColumn = Math.Clamp(column - originX, 0, width - 1); + byte expected = (byte)GetVisibleSample(planeIndex, visibleRow, visibleColumn); + byte actual = samples[(row * stride) + column]; + + if (actual != expected) + { + Assert.Equal(expected, actual); + } + } + } + + return; + } + + ReadOnlySpan highBitDepthSamples = MemoryMarshal.Cast(buffer.DangerousGetSingleSpan()); + + for (int row = 0; row < allocatedHeight; row++) + { + int visibleRow = Math.Clamp(row - originY, 0, height - 1); + + for (int column = 0; column < stride; column++) + { + int visibleColumn = Math.Clamp(column - originX, 0, width - 1); + ushort expected = (ushort)GetVisibleSample(planeIndex, visibleRow, visibleColumn); + ushort actual = highBitDepthSamples[(row * stride) + column]; + + if (actual != expected) + { + Assert.Equal(expected, actual); + } + } + } + } + + /// + /// Computes the deterministic visible sample used by the border-extension oracle. + /// + /// The zero-based plane index. + /// The visible row. + /// The visible column. + /// The native sample value. + private static int GetVisibleSample(int planeIndex, int row, int column) => ((planeIndex + 1) * 31) + (row * 11) + (column * 3); + + /// + /// Creates the smallest valid monochrome reference-frame owner for slot-lifecycle tests. + /// + /// A reference frame whose sample planes are owned by the caller. + private static Av1ReferenceFrame CreateFrame() + { + ObuSequenceHeader sequenceHeader = CreateSequenceHeader(1, 1, Av1BitDepth.EightBit, true, false, false); + Av1FrameBuffer frameBuffer = new(Configuration.Default, sequenceHeader, Av1ColorFormat.Yuv400, false); + using Av1FrameInfo frameInfo = new(sequenceHeader); + + return new Av1ReferenceFrame(frameBuffer, new ObuFrameHeader(), frameInfo); + } + + /// + /// Fails the temporal motion-field rent after allowing every earlier tile-reader allocation to succeed. + /// + private sealed class FailingTemporalMotionFieldAllocator : TestMemoryAllocator + { + /// + /// Initializes a new instance of the class. + /// + public FailingTemporalMotionFieldAllocator() => this.EnableNonThreadSafeLogging(); + + /// + protected override AllocationTrackedMemoryManager AllocateCore( + int length, + AllocationOptions options = AllocationOptions.None) + { + if (typeof(T).Name == "TemporalMotionFieldEntry") + { + // Fail before the owner is published so the allocation log contains only resources that the + // Av1TileReader constructor must unwind. + throw new InvalidMemoryOperationException("The configured temporal motion-field allocation failed."); + } + + return base.AllocateCore(length, options); + } + } + + /// + /// Creates the sequence geometry and color configuration used by direct frame-buffer tests. + /// + /// The maximum coded width. + /// The maximum coded height. + /// The coded sample precision. + /// Whether the sequence contains only luma. + /// Whether chroma is horizontally subsampled. + /// Whether chroma is vertically subsampled. + /// The initialized sequence header. + private static ObuSequenceHeader CreateSequenceHeader( + int width, + int height, + Av1BitDepth bitDepth, + bool isMonochrome, + bool subsamplingX, + bool subsamplingY) + => new() + { + MaxFrameWidth = width, + MaxFrameHeight = height, + ColorConfig = new ObuColorConfig + { + IsMonochrome = isMonochrome, + SubSamplingX = subsamplingX, + SubSamplingY = subsamplingY, + BitDepth = bitDepth + } + }; +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReferenceMotionVectorsTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReferenceMotionVectorsTests.cs new file mode 100644 index 0000000000..e8211599f4 --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReferenceMotionVectorsTests.cs @@ -0,0 +1,641 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1; +using SixLabors.ImageSharp.Formats.Heif.Av1.Motion; +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; +using SixLabors.ImageSharp.Formats.Heif.Av1.ReferenceFrames; +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1; + +/// +/// Verifies the spatial, temporal, global, and extension rules used to derive single- and compound-reference AV1 motion vectors. +/// +[Trait("Format", "Avif")] +public class Av1ReferenceMotionVectorsTests +{ + /// + /// Verifies adjacent-direction counting, duplicate weighting, stable ordering, and the nearest, near, and new-reference accessors. + /// + [Fact] + public void BuildOrdersAdjacentCandidatesAndPacksModeContext() + { + ObuSequenceHeader sequenceHeader = CreateSequenceHeader(enableTemporalMotionVectors: false); + ObuFrameHeader frameHeader = CreateFrameHeader(orderHint: 0, useReferenceFrameMotionVectors: false); + using Av1FrameInfo frameInfo = new(sequenceHeader); + FillFrameWithIntraBlocks(frameInfo, sequenceHeader); + + Av1MotionVector above = new(24, -10); + Av1MotionVector left = new(-14, 30); + AddModeInfo(frameInfo, sequenceHeader, new Point(8, 4), Av1BlockSize.Block16x16, Av1ReferenceFrameType.Last, above, Av1PredictionMode.NewMotionVector); + AddModeInfo( + frameInfo, + sequenceHeader, + new Point(4, 8), + Av1BlockSize.Block16x16, + Av1ReferenceFrameType.Last, + left, + Av1PredictionMode.NearestMotionVector); + + AddModeInfo( + frameInfo, + sequenceHeader, + new Point(12, 7), + Av1BlockSize.Block4x4, + Av1ReferenceFrameType.Last, + above, + Av1PredictionMode.NearestMotionVector); + + Av1SuperblockInfo superblockInfo = frameInfo.GetSuperblock(Point.Empty); + Av1BlockModeInfo modeInfo = new(Av1BlockSize.Block16x16, new Point(8, 8)); + Av1PartitionInfo partitionInfo = new(modeInfo, superblockInfo, true, Av1PartitionType.None) + { + ColumnIndex = 8, + RowIndex = 8, + }; + + Av1TileInfo tileInfo = new(0, 0, frameHeader); + partitionInfo.ComputeBoundaryOffsets(sequenceHeader, frameHeader, tileInfo); + Av1ReferenceMotionVectors referenceMotionVectors = new(); + + referenceMotionVectors.Build( + ref partitionInfo, + tileInfo, + frameInfo, + sequenceHeader, + frameHeader, + Av1ReferenceFrameType.Last); + + Assert.Equal(2, referenceMotionVectors.Count); + Assert.Equal(84, referenceMotionVectors.ModeContext); + Assert.Equal(above, referenceMotionVectors.Candidates[0]); + Assert.Equal(left, referenceMotionVectors.Candidates[1]); + Assert.Equal((ushort)660, referenceMotionVectors.Weights[0]); + Assert.Equal((ushort)656, referenceMotionVectors.Weights[1]); + Assert.Equal(above, referenceMotionVectors.Nearest); + Assert.Equal(left, referenceMotionVectors.GetNearReference(0)); + Assert.Equal(above, referenceMotionVectors.GetNewReference(0)); + } + + /// + /// Verifies that outer candidates are weight-sorted independently without crossing the nearest-region boundary. + /// + [Fact] + public void BuildSortsOuterCandidatesInsideTheirOwnRegion() + { + ObuSequenceHeader sequenceHeader = CreateSequenceHeader(enableTemporalMotionVectors: false); + ObuFrameHeader frameHeader = CreateFrameHeader(orderHint: 0, useReferenceFrameMotionVectors: false); + using Av1FrameInfo frameInfo = new(sequenceHeader); + FillFrameWithIntraBlocks(frameInfo, sequenceHeader); + + Av1MotionVector nearest = new(8, 16); + Av1MotionVector topLeft = new(24, 32); + Av1MotionVector outerRow = new(40, 48); + AddModeInfo( + frameInfo, + sequenceHeader, + new Point(8, 7), + Av1BlockSize.Block4x4, + Av1ReferenceFrameType.Last, + nearest, + Av1PredictionMode.NearestMotionVector); + + // A 4x4 intra neighbor keeps the adjacent scan from marking the deeper row as covered by a large background block. + AddModeInfo( + frameInfo, + sequenceHeader, + new Point(9, 7), + Av1BlockSize.Block4x4, + Av1ReferenceFrameType.Intra, + default, + Av1PredictionMode.DC); + + AddModeInfo( + frameInfo, + sequenceHeader, + new Point(7, 7), + Av1BlockSize.Block4x4, + Av1ReferenceFrameType.Last, + topLeft, + Av1PredictionMode.NearestMotionVector); + + AddModeInfo( + frameInfo, + sequenceHeader, + new Point(9, 3), + Av1BlockSize.Block8x16, + Av1ReferenceFrameType.Last, + outerRow, + Av1PredictionMode.NearestMotionVector); + + Av1SuperblockInfo superblockInfo = frameInfo.GetSuperblock(Point.Empty); + Av1BlockModeInfo modeInfo = new(Av1BlockSize.Block8x8, new Point(8, 8)); + Av1PartitionInfo partitionInfo = new(modeInfo, superblockInfo, true, Av1PartitionType.None) + { + ColumnIndex = 8, + RowIndex = 8, + }; + + Av1TileInfo tileInfo = new(0, 0, frameHeader); + partitionInfo.ComputeBoundaryOffsets(sequenceHeader, frameHeader, tileInfo); + Av1ReferenceMotionVectors referenceMotionVectors = new(); + + referenceMotionVectors.Build( + ref partitionInfo, + tileInfo, + frameInfo, + sequenceHeader, + frameHeader, + Av1ReferenceFrameType.Last); + + Assert.Equal(3, referenceMotionVectors.Count); + Assert.Equal(nearest, referenceMotionVectors.Candidates[0]); + Assert.Equal(outerRow, referenceMotionVectors.Candidates[1]); + Assert.Equal(topLeft, referenceMotionVectors.Candidates[2]); + Assert.Equal((ushort)642, referenceMotionVectors.Weights[0]); + Assert.Equal((ushort)8, referenceMotionVectors.Weights[1]); + Assert.Equal((ushort)4, referenceMotionVectors.Weights[2]); + Assert.Equal(51, referenceMotionVectors.ModeContext); + } + + /// + /// Verifies that an affine global-motion neighbor contributes the current block's global vector while extension retains its decoded vector. + /// + [Fact] + public void BuildSubstitutesAffineGlobalMotionForDirectCandidate() + { + ObuSequenceHeader sequenceHeader = CreateSequenceHeader(enableTemporalMotionVectors: false); + ObuFrameHeader frameHeader = CreateFrameHeader(orderHint: 0, useReferenceFrameMotionVectors: false); + Av1GlobalMotionParameters globalMotion = Av1GlobalMotionParameters.Identity; + globalMotion.Type = Av1GlobalMotionType.Affine; + globalMotion[0] = 4096; + globalMotion[1] = -2048; + globalMotion[2] = Av1GlobalMotionParameters.ModelScale + 512; + globalMotion[5] = Av1GlobalMotionParameters.ModelScale; + frameHeader.GetGlobalMotionParameters()[0] = globalMotion; + + using Av1FrameInfo frameInfo = new(sequenceHeader); + FillFrameWithIntraBlocks(frameInfo, sequenceHeader); + Av1MotionVector decoded = new(40, -24); + AddModeInfo( + frameInfo, + sequenceHeader, + new Point(8, 4), + Av1BlockSize.Block16x16, + Av1ReferenceFrameType.Last, + decoded, + Av1PredictionMode.GlobalMotionVector); + + Av1SuperblockInfo superblockInfo = frameInfo.GetSuperblock(Point.Empty); + Av1BlockModeInfo modeInfo = new(Av1BlockSize.Block16x16, new Point(8, 8)); + Av1PartitionInfo partitionInfo = new(modeInfo, superblockInfo, true, Av1PartitionType.None) + { + ColumnIndex = 8, + RowIndex = 8, + }; + + Av1TileInfo tileInfo = new(0, 0, frameHeader); + partitionInfo.ComputeBoundaryOffsets(sequenceHeader, frameHeader, tileInfo); + Av1ReferenceMotionVectors referenceMotionVectors = new(); + + referenceMotionVectors.Build( + ref partitionInfo, + tileInfo, + frameInfo, + sequenceHeader, + frameHeader, + Av1ReferenceFrameType.Last); + + Av1MotionVector expectedGlobal = globalMotion.GetMotionVector( + frameHeader.AllowHighPrecisionMotionVector, + modeInfo.BlockSize, + new Point(partitionInfo.ColumnIndex, partitionInfo.RowIndex), + frameHeader.ForceIntegerMotionVector); + + Assert.Equal(2, referenceMotionVectors.Count); + Assert.Equal(expectedGlobal, referenceMotionVectors.Candidates[0]); + Assert.Equal(decoded, referenceMotionVectors.Candidates[1]); + Assert.Equal((ushort)656, referenceMotionVectors.Weights[0]); + Assert.Equal((ushort)2, referenceMotionVectors.Weights[1]); + } + + /// + /// Verifies that stack extension reverses an opposite-side vector without reweighting a candidate already in the + /// direct stack. + /// + [Fact] + public void BuildReversesOppositeDirectionExtensionCandidate() + { + ObuSequenceHeader sequenceHeader = CreateSequenceHeader(enableTemporalMotionVectors: false); + sequenceHeader.OrderHintInfo.EnableOrderHint = true; + sequenceHeader.OrderHintInfo.OrderHintBits = 5; + ObuFrameHeader frameHeader = CreateFrameHeader(orderHint: 10, useReferenceFrameMotionVectors: false); + frameHeader.GetReferenceFrameIndices()[0] = 0; + frameHeader.GetReferenceFrameIndices()[4] = 1; + + using Av1ReferenceFrameStore referenceFrames = new(); + Av1ReferenceFrame past = CreateReferenceFrame(sequenceHeader, orderHint: 8); + Av1ReferenceFrame future = CreateReferenceFrame(sequenceHeader, orderHint: 12); + Assert.True(referenceFrames.Commit(1, past, showFrame: false)); + Assert.True(referenceFrames.Commit(2, future, showFrame: false)); + + using Av1FrameInfo frameInfo = new(sequenceHeader); + frameInfo.InitializeMotionField(Configuration.Default, sequenceHeader, frameHeader, referenceFrames); + FillFrameWithIntraBlocks(frameInfo, sequenceHeader); + Av1MotionVector direct = new(16, 24); + AddModeInfo( + frameInfo, + sequenceHeader, + new Point(8, 4), + Av1BlockSize.Block16x16, + Av1ReferenceFrameType.Last, + direct, + Av1PredictionMode.NearestMotionVector, + Av1ReferenceFrameType.Backward, + new Av1MotionVector(40, -24)); + + // The direct scan adds the first reference with its normative adjacent weight. Extension visits both entries: + // it must ignore that duplicate and append only the sign-corrected backward-reference vector. + Av1SuperblockInfo superblockInfo = frameInfo.GetSuperblock(Point.Empty); + Av1BlockModeInfo modeInfo = new(Av1BlockSize.Block16x16, new Point(8, 8)); + Av1PartitionInfo partitionInfo = new(modeInfo, superblockInfo, true, Av1PartitionType.None) + { + ColumnIndex = 8, + RowIndex = 8, + }; + + Av1TileInfo tileInfo = new(0, 0, frameHeader); + partitionInfo.ComputeBoundaryOffsets(sequenceHeader, frameHeader, tileInfo); + Av1ReferenceMotionVectors referenceMotionVectors = new(); + + referenceMotionVectors.Build( + ref partitionInfo, + tileInfo, + frameInfo, + sequenceHeader, + frameHeader, + Av1ReferenceFrameType.Last); + + Av1MotionVector expected = new(-40, 24); + Assert.Equal(2, referenceMotionVectors.Count); + Assert.Equal(direct, referenceMotionVectors.Candidates[0]); + Assert.Equal(expected, referenceMotionVectors.Candidates[1]); + Assert.Equal((ushort)656, referenceMotionVectors.Weights[0]); + Assert.Equal((ushort)2, referenceMotionVectors.Weights[1]); + Assert.Equal(direct, referenceMotionVectors.Nearest); + Assert.Equal(direct, referenceMotionVectors.GetNewReference(0)); + Assert.Equal(expected, referenceMotionVectors.GetNearReference(0)); + } + + /// + /// Verifies temporal field sampling, candidate deduplication, accumulated weight, and the global-motion context bit. + /// + [Fact] + public void BuildAccumulatesProjectedTemporalCandidates() + { + ObuSequenceHeader sequenceHeader = CreateSequenceHeader(enableTemporalMotionVectors: true); + sequenceHeader.OrderHintInfo.EnableOrderHint = true; + sequenceHeader.OrderHintInfo.OrderHintBits = 5; + + using Av1ReferenceFrameStore priorReferences = new(); + Av1ReferenceFrame prior = CreateReferenceFrame(sequenceHeader, orderHint: 6); + Assert.True(priorReferences.Commit(1, prior, showFrame: false)); + + ObuFrameHeader sourceHeader = CreateFrameHeader(orderHint: 8, useReferenceFrameMotionVectors: false); + using Av1FrameInfo sourceFrameInfo = new(sequenceHeader); + sourceFrameInfo.InitializeMotionField(Configuration.Default, sequenceHeader, sourceHeader, priorReferences); + FillFrameWithInterBlocks(sourceFrameInfo, sequenceHeader, Av1ReferenceFrameType.Last, default); + + using Av1ReferenceFrameStore sourceReferences = new(); + Av1ReferenceFrame source = new( + new Av1FrameBuffer(Configuration.Default, sequenceHeader, Av1ColorFormat.Yuv400, false), + sourceHeader, + sourceFrameInfo); + + Assert.True(sourceReferences.Commit(1, source, showFrame: false)); + + ObuFrameHeader frameHeader = CreateFrameHeader(orderHint: 10, useReferenceFrameMotionVectors: true); + using Av1FrameInfo frameInfo = new(sequenceHeader); + frameInfo.InitializeMotionField(Configuration.Default, sequenceHeader, frameHeader, sourceReferences); + FillFrameWithIntraBlocks(frameInfo, sequenceHeader); + + Av1SuperblockInfo superblockInfo = frameInfo.GetSuperblock(Point.Empty); + Av1BlockModeInfo modeInfo = new(Av1BlockSize.Block16x16, new Point(8, 8)); + Av1PartitionInfo partitionInfo = new(modeInfo, superblockInfo, true, Av1PartitionType.None) + { + ColumnIndex = 8, + RowIndex = 8, + }; + + Av1TileInfo tileInfo = new(0, 0, frameHeader); + partitionInfo.ComputeBoundaryOffsets(sequenceHeader, frameHeader, tileInfo); + Av1ReferenceMotionVectors referenceMotionVectors = new(); + + referenceMotionVectors.Build( + ref partitionInfo, + tileInfo, + frameInfo, + sequenceHeader, + frameHeader, + Av1ReferenceFrameType.Last); + + Assert.Equal(1, referenceMotionVectors.Count); + Assert.Equal(default, referenceMotionVectors.Candidates[0]); + Assert.Equal((ushort)14, referenceMotionVectors.Weights[0]); + Assert.Equal(0, referenceMotionVectors.ModeContext); + } + + /// + /// Verifies that compound candidates retain their primary and secondary vectors through weighting, sorting, and DRL access. + /// + [Fact] + public void BuildRetainsPairedCompoundCandidates() + { + ObuSequenceHeader sequenceHeader = CreateSequenceHeader(enableTemporalMotionVectors: false); + ObuFrameHeader frameHeader = CreateFrameHeader(orderHint: 0, useReferenceFrameMotionVectors: false); + using Av1FrameInfo frameInfo = new(sequenceHeader); + FillFrameWithIntraBlocks(frameInfo, sequenceHeader); + + Av1MotionVector abovePrimary = new(8, 16); + Av1MotionVector aboveSecondary = new(24, 32); + AddModeInfo( + frameInfo, + sequenceHeader, + new Point(8, 4), + Av1BlockSize.Block16x16, + Av1ReferenceFrameType.Last, + abovePrimary, + Av1PredictionMode.NewNewMotionVector, + Av1ReferenceFrameType.Backward, + aboveSecondary); + + Av1MotionVector leftPrimary = new(40, 48); + Av1MotionVector leftSecondary = new(56, 64); + AddModeInfo( + frameInfo, + sequenceHeader, + new Point(4, 8), + Av1BlockSize.Block16x16, + Av1ReferenceFrameType.Last, + leftPrimary, + Av1PredictionMode.NearestNearestMotionVector, + Av1ReferenceFrameType.Backward, + leftSecondary); + + Av1SuperblockInfo superblockInfo = frameInfo.GetSuperblock(Point.Empty); + Av1BlockModeInfo modeInfo = new(Av1BlockSize.Block16x16, new Point(8, 8)); + Av1PartitionInfo partitionInfo = new(modeInfo, superblockInfo, true, Av1PartitionType.None) + { + ColumnIndex = 8, + RowIndex = 8, + }; + + Av1TileInfo tileInfo = new(0, 0, frameHeader); + partitionInfo.ComputeBoundaryOffsets(sequenceHeader, frameHeader, tileInfo); + Av1ReferenceMotionVectors referenceMotionVectors = new(); + + referenceMotionVectors.Build( + ref partitionInfo, + tileInfo, + frameInfo, + sequenceHeader, + frameHeader, + Av1ReferenceFrameType.Last, + Av1ReferenceFrameType.Backward); + + Assert.Equal(2, referenceMotionVectors.Count); + Assert.Equal(abovePrimary, referenceMotionVectors.Candidates[0]); + Assert.Equal(aboveSecondary, referenceMotionVectors.CompoundCandidates[0]); + Assert.Equal(leftPrimary, referenceMotionVectors.Candidates[1]); + Assert.Equal(leftSecondary, referenceMotionVectors.CompoundCandidates[1]); + Assert.Equal((ushort)656, referenceMotionVectors.Weights[0]); + Assert.Equal((ushort)656, referenceMotionVectors.Weights[1]); + Assert.Equal(abovePrimary, referenceMotionVectors.GetCompoundNearestReference(0)); + Assert.Equal(aboveSecondary, referenceMotionVectors.GetCompoundNearestReference(1)); + Assert.Equal(leftPrimary, referenceMotionVectors.GetCompoundNearReference(0, 0)); + Assert.Equal(leftSecondary, referenceMotionVectors.GetCompoundNearReference(0, 1)); + Assert.Equal(abovePrimary, referenceMotionVectors.GetCompoundNewReference(0, 0)); + Assert.Equal(aboveSecondary, referenceMotionVectors.GetCompoundNewReference(0, 1)); + } + + /// + /// Verifies the positional compound fallback assembled from independent exact-reference neighbor lists. + /// + [Fact] + public void BuildExtendsCompoundStackWithPairedFallbacks() + { + ObuSequenceHeader sequenceHeader = CreateSequenceHeader(enableTemporalMotionVectors: false); + ObuFrameHeader frameHeader = CreateFrameHeader(orderHint: 0, useReferenceFrameMotionVectors: false); + using Av1FrameInfo frameInfo = new(sequenceHeader); + FillFrameWithIntraBlocks(frameInfo, sequenceHeader); + + Av1MotionVector above = new(8, 16); + Av1MotionVector left = new(24, 32); + AddModeInfo( + frameInfo, + sequenceHeader, + new Point(8, 4), + Av1BlockSize.Block16x16, + Av1ReferenceFrameType.Last, + above, + Av1PredictionMode.NearestMotionVector); + + AddModeInfo( + frameInfo, + sequenceHeader, + new Point(4, 8), + Av1BlockSize.Block16x16, + Av1ReferenceFrameType.Backward, + left, + Av1PredictionMode.NearestMotionVector); + + Av1SuperblockInfo superblockInfo = frameInfo.GetSuperblock(Point.Empty); + Av1BlockModeInfo modeInfo = new(Av1BlockSize.Block16x16, new Point(8, 8)); + Av1PartitionInfo partitionInfo = new(modeInfo, superblockInfo, true, Av1PartitionType.None) + { + ColumnIndex = 8, + RowIndex = 8, + }; + + Av1TileInfo tileInfo = new(0, 0, frameHeader); + partitionInfo.ComputeBoundaryOffsets(sequenceHeader, frameHeader, tileInfo); + Av1ReferenceMotionVectors referenceMotionVectors = new(); + + referenceMotionVectors.Build( + ref partitionInfo, + tileInfo, + frameInfo, + sequenceHeader, + frameHeader, + Av1ReferenceFrameType.Last, + Av1ReferenceFrameType.Backward); + + Assert.Equal(2, referenceMotionVectors.Count); + Assert.Equal(above, referenceMotionVectors.Candidates[0]); + Assert.Equal(left, referenceMotionVectors.CompoundCandidates[0]); + Assert.Equal(left, referenceMotionVectors.Candidates[1]); + Assert.Equal(above, referenceMotionVectors.CompoundCandidates[1]); + Assert.Equal((ushort)2, referenceMotionVectors.Weights[0]); + Assert.Equal((ushort)2, referenceMotionVectors.Weights[1]); + } + + /// + /// Creates the monochrome 128-by-128 sequence geometry shared by reference-motion-vector tests. + /// + /// Whether projected reference-frame motion vectors are enabled. + /// The configured sequence header. + private static ObuSequenceHeader CreateSequenceHeader(bool enableTemporalMotionVectors) + => new() + { + MaxFrameWidth = 128, + MaxFrameHeight = 128, + Use128x128Superblock = false, + ColorConfig = new ObuColorConfig + { + IsMonochrome = true, + BitDepth = Av1BitDepth.EightBit, + }, + OrderHintInfo = new ObuOrderHintInfo + { + EnableReferenceFrameMotionVectors = enableTemporalMotionVectors, + }, + }; + + /// + /// Creates one inter-frame header whose single tile covers the complete test frame. + /// + /// The frame's modulo display-order hint. + /// Whether this frame consumes its projected temporal motion field. + /// The configured frame header. + private static ObuFrameHeader CreateFrameHeader(uint orderHint, bool useReferenceFrameMotionVectors) + { + ObuTileGroupHeader tilesInfo = new() + { + TileColumnCount = 1, + TileRowCount = 1, + }; + + tilesInfo.TileColumnStartModeInfo[1] = 32; + tilesInfo.TileRowStartModeInfo[1] = 32; + + return new() + { + FrameType = ObuFrameType.InterFrame, + OrderHint = orderHint, + ModeInfoColumnCount = 32, + ModeInfoRowCount = 32, + AllowHighPrecisionMotionVector = true, + UseReferenceFrameMotionVectors = useReferenceFrameMotionVectors, + TilesInfo = tilesInfo, + }; + } + + /// + /// Maps one intra block over each 64-by-64 superblock so every spatial search position has initialized mode information. + /// + /// The frame map to initialize. + /// The sequence geometry defining the superblock grid. + private static void FillFrameWithIntraBlocks(Av1FrameInfo frameInfo, ObuSequenceHeader sequenceHeader) + { + for (int row = 0; row < 32; row += sequenceHeader.SuperblockModeInfoSize) + { + for (int column = 0; column < 32; column += sequenceHeader.SuperblockModeInfoSize) + { + AddModeInfo( + frameInfo, + sequenceHeader, + new Point(column, row), + Av1BlockSize.Block64x64, + Av1ReferenceFrameType.Intra, + default, + Av1PredictionMode.DC); + } + } + } + + /// + /// Maps one inter block over each 64-by-64 superblock and publishes its vector to the retained motion field. + /// + /// The frame map and retained field to initialize. + /// The sequence geometry defining the superblock grid. + /// The canonical reference selected by each block. + /// The retained motion vector. + private static void FillFrameWithInterBlocks( + Av1FrameInfo frameInfo, + ObuSequenceHeader sequenceHeader, + Av1ReferenceFrameType referenceFrame, + Av1MotionVector motionVector) + { + for (int row = 0; row < 32; row += sequenceHeader.SuperblockModeInfoSize) + { + for (int column = 0; column < 32; column += sequenceHeader.SuperblockModeInfoSize) + { + AddModeInfo( + frameInfo, + sequenceHeader, + new Point(column, row), + Av1BlockSize.Block64x64, + referenceFrame, + motionVector, + Av1PredictionMode.NearestMotionVector); + } + } + } + + /// + /// Creates and maps one mode-information block at a frame-relative position. + /// + /// The frame map that owns the block. + /// The sequence geometry defining superblock-relative addressing. + /// The block origin in frame-relative 4x4 units. + /// The block geometry. + /// The primary prediction reference. + /// The primary motion vector. + /// The decoded luma or inter prediction mode. + /// The optional secondary prediction reference. + /// The optional secondary motion vector. + /// The mapped mode-information block. + private static Av1BlockModeInfo AddModeInfo( + Av1FrameInfo frameInfo, + ObuSequenceHeader sequenceHeader, + Point position, + Av1BlockSize blockSize, + Av1ReferenceFrameType referenceFrame, + Av1MotionVector motionVector, + Av1PredictionMode predictionMode, + Av1ReferenceFrameType secondaryReferenceFrame = Av1ReferenceFrameType.None, + Av1MotionVector secondaryMotionVector = default) + { + int superblockSize = sequenceHeader.SuperblockModeInfoSize; + Point superblockPosition = new(position.X / superblockSize, position.Y / superblockSize); + Point relativePosition = new(position.X % superblockSize, position.Y % superblockSize); + Av1SuperblockInfo superblockInfo = frameInfo.GetSuperblock(superblockPosition); + Av1BlockModeInfo modeInfo = new(blockSize, relativePosition) + { + YMode = predictionMode, + }; + + modeInfo.ReferenceFrames[0] = referenceFrame; + modeInfo.ReferenceFrames[1] = secondaryReferenceFrame; + modeInfo.MotionVectors[0] = motionVector; + modeInfo.MotionVectors[1] = secondaryMotionVector; + frameInfo.UpdateModeInfo(modeInfo, superblockInfo); + return modeInfo; + } + + /// + /// Creates a retained monochrome frame at one display-order hint. + /// + /// The sequence geometry used by the retained frame. + /// The retained frame's modulo display-order hint. + /// A frame owner whose sample buffer and mode state are ready for reference-map ownership. + private static Av1ReferenceFrame CreateReferenceFrame(ObuSequenceHeader sequenceHeader, uint orderHint) + { + ObuFrameHeader frameHeader = CreateFrameHeader(orderHint, useReferenceFrameMotionVectors: false); + Av1FrameBuffer frameBuffer = new(Configuration.Default, sequenceHeader, Av1ColorFormat.Yuv400, false); + using Av1FrameInfo frameInfo = new(sequenceHeader); + return new Av1ReferenceFrame(frameBuffer, frameHeader, frameInfo); + } +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReferenceTransform.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReferenceTransform.cs new file mode 100644 index 0000000000..bf461db805 --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReferenceTransform.cs @@ -0,0 +1,271 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1; + +internal class Av1ReferenceTransform +{ + /// + /// Gets the analytical amplification used by the reference decoder's forward-transform tests. + /// + /// The transform configuration. + /// The two-dimensional transform amplification. + public static double GetScaleFactor(Av1Transform2dFlipConfiguration config) + { + int transformWidth = config.TransformSize.GetWidth(); + int transformHeight = config.TransformSize.GetHeight(); + int amplifyBit = config.Shift0 + config.Shift1 + config.Shift2; + double scaleFactor = + amplifyBit >= 0 ? (1 << amplifyBit) : (1.0 / (1 << -amplifyBit)); + + // For rectangular transforms, we need to multiply by an extra factor. + int rectType = config.TransformSize.GetRectangleLogRatio(); + if (Math.Abs(rectType) == 1) + { + scaleFactor *= Math.Sqrt(2); + } + + return scaleFactor; + } + + /// + /// Applies the analytical two-dimensional transform used by the reference decoder's + /// test/av1_txfm_test.cc. + /// + /// The raster input samples. + /// The raster output coefficients. + /// The two-dimensional transform type. + /// The transform dimensions. + /// The configured two-dimensional amplification. + public static void ReferenceTransformFunction2d(Span input, Span output, Av1TransformType transformType, Av1TransformSize transformSize, double scaleFactor) + { + // Get transform type and size of each dimension. + Av1Transform2dFlipConfiguration config = Av1Transform2dFlipConfiguration.CreateForward(transformType, transformSize, 8); + Av1TransformType1d columnType = GetTransformType1d(config.TransformFunctionTypeColumn); + Av1TransformType1d rowType = GetTransformType1d(config.TransformFunctionTypeRow); + int transformWidth = transformSize.GetWidth(); + int transformHeight = transformSize.GetHeight(); + Span tmpInput = new double[transformWidth * transformHeight]; + Span tmpOutput = new double[transformWidth * transformHeight]; + + // second forward transform with row_type + for (int r = 0; r < transformHeight; ++r) + { + ReferenceTransform1d(rowType, input[(r * transformWidth)..], output[(r * transformWidth)..], transformWidth); + } + + // matrix transposition + for (int r = 0; r < transformHeight; ++r) + { + for (int c = 0; c < transformWidth; ++c) + { + tmpInput[(c * transformHeight) + r] = output[(r * transformWidth) + c]; + } + } + + // first forward transform with column_type + for (int c = 0; c < transformWidth; ++c) + { + ReferenceTransform1d( + columnType, + tmpInput[(c * transformHeight)..], + tmpOutput[(c * transformHeight)..], + transformHeight); + } + + // matrix transposition + for (int r = 0; r < transformHeight; ++r) + { + for (int c = 0; c < transformWidth; ++c) + { + output[(c * transformHeight) + r] = tmpOutput[(r * transformWidth) + c]; + } + } + + // appropriate scale + for (int r = 0; r < transformHeight; ++r) + { + for (int c = 0; c < transformWidth; ++c) + { + output[(r * transformWidth) + c] *= scaleFactor; + } + } + } + + private static void Adst4Reference(ReadOnlySpan input, Span output) + { + // 16384 * sqrt(2) * sin(kPi/9) * 2 / 3 + const long sinPi19 = 5283; + const long sinPi29 = 9929; + const long sinPi39 = 13377; + const long sinPi49 = 15212; + + long x0, x1, x2, x3; + long s0, s1, s2, s3, s4, s5, s6, s7; + x0 = input[0]; + x1 = input[1]; + x2 = input[2]; + x3 = input[3]; + + if ((x0 | x1 | x2 | x3) == 0L) + { + output[0] = output[1] = output[2] = output[3] = 0; + return; + } + + s0 = sinPi19 * x0; + s1 = sinPi49 * x0; + s2 = sinPi29 * x1; + s3 = sinPi19 * x1; + s4 = sinPi39 * x2; + s5 = sinPi49 * x3; + s6 = sinPi29 * x3; + s7 = x0 + x1 - x3; + + x0 = s0 + s2 + s5; + x1 = sinPi39 * s7; + x2 = s1 - s3 + s6; + x3 = s4; + + s0 = x0 + x3; + s1 = x1; + s2 = x2 - x3; + s3 = x2 - x0 + x3; + + // 1-D transform scaling factor is sqrt(2). + output[0] = Av1Math.RoundShift(s0, 14); + output[1] = Av1Math.RoundShift(s1, 14); + output[2] = Av1Math.RoundShift(s2, 14); + output[3] = Av1Math.RoundShift(s3, 14); + } + + private static void ReferenceIdentity1d(ReadOnlySpan input, Span output, int size) + { + const double sqrt2 = 1.4142135623730950488016887242097f; + double scale = 0; + switch (size) + { + case 4: + scale = sqrt2; + break; + case 8: + scale = 2; + break; + case 16: + scale = 2 * sqrt2; + break; + case 32: + scale = 4; + break; + case 64: + scale = 4 * sqrt2; + break; + default: + Assert.Fail(); + break; + } + + for (int k = 0; k < size; ++k) + { + output[k] = input[k] * scale; + } + } + + private static void ReferenceDct1d(ReadOnlySpan input, Span output, int size) + { + const double kInvSqrt2 = 0.707106781186547524400844362104f; + for (int k = 0; k < size; ++k) + { + output[k] = 0; + for (int n = 0; n < size; ++n) + { + output[k] += input[n] * Math.Cos(Math.PI * ((2 * n) + 1) * k / (2 * size)); + } + + if (k == 0) + { + output[k] = output[k] * kInvSqrt2; + } + } + } + + private static void ReferenceAdst1d(ReadOnlySpan input, Span output, int size) + { + if (size == 4) + { + // Special case. + int[] int_input = new int[4]; + for (int i = 0; i < 4; ++i) + { + int_input[i] = (int)Math.Round(input[i]); + } + + int[] int_output = new int[4]; + Adst4Reference(int_input, int_output); + for (int i = 0; i < 4; ++i) + { + output[i] = int_output[i]; + } + + return; + } + + for (int k = 0; k < size; ++k) + { + output[k] = 0; + for (int n = 0; n < size; ++n) + { + output[k] += input[n] * Math.Sin(Math.PI * ((2 * n) + 1) * ((2 * k) + 1) / (4 * size)); + } + } + } + + internal static void ReferenceTransform1d(Av1TransformType1d type, ReadOnlySpan input, Span output, int size) + { + switch (type) + { + case Av1TransformType1d.Dct: + ReferenceDct1d(input, output, size); + break; + case Av1TransformType1d.Adst: + case Av1TransformType1d.FlipAdst: + ReferenceAdst1d(input, output, size); + break; + case Av1TransformType1d.Identity: + ReferenceIdentity1d(input, output, size); + break; + default: + Assert.Fail(); + break; + } + } + + private static Av1TransformType1d GetTransformType1d(Av1TransformFunctionType transformFunctionType) + { + switch (transformFunctionType) + { + case Av1TransformFunctionType.Dct4: + case Av1TransformFunctionType.Dct8: + case Av1TransformFunctionType.Dct16: + case Av1TransformFunctionType.Dct32: + case Av1TransformFunctionType.Dct64: + return Av1TransformType1d.Dct; + case Av1TransformFunctionType.Adst4: + case Av1TransformFunctionType.Adst8: + case Av1TransformFunctionType.Adst16: + return Av1TransformType1d.Adst; + case Av1TransformFunctionType.Identity4: + case Av1TransformFunctionType.Identity8: + case Av1TransformFunctionType.Identity16: + case Av1TransformFunctionType.Identity32: + return Av1TransformType1d.Identity; + case Av1TransformFunctionType.Invalid: + default: + Assert.Fail(); + return (Av1TransformType1d)5; + } + } +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReferenceYuvConverter.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReferenceYuvConverter.cs new file mode 100644 index 0000000000..12d56d9346 --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReferenceYuvConverter.cs @@ -0,0 +1,165 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Numerics; +using SixLabors.ImageSharp.Formats.Heif.Av1; +using SixLabors.ImageSharp.PixelFormats; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1; + +/// +/// This simulates converting 24-bit RGB values to YUV, then back to 24-bit RGB. +/// Using BT.709 transfer functions: https://en.wikipedia.org/wiki/Rec._709 +/// +/// It demonstrates that converting to 30-bit YUV then back to 24-bit RGB is lossy. +/// Using 10 bits per YUV value appears to be lossless. +/// +/// Converting RGB (24-bit) -> YUV(64-bit floats per channel, normalized[0 - 1]) -> RGB(24-bit) +/// Found 0 inaccurate conversions out of 16581375 RGB values +/// +/// Converting RGB(24-bit) -> YUV(30-bit) -> RGB(24-bit) +/// Found 0 inaccurate conversions out of 16581375 RGB values +/// +/// Converting RGB(24-bit) -> YUV(24-bit) -> RGB(24-bit) +/// Found 4058422 accurate conversions out of 16581375 RGB values +/// Found 12522953 inaccurate conversions out of 16581375 RGB values +/// Off by: {1: 8786792, 2: 3727753, 3: 8408} +/// +/// Ported from Python to C# from: https://gist.github.com/linrock/5be4f365c9c9e61eee9e8984ba13cb25. +internal class Av1ReferenceYuvConverter +{ + // The range of UV values in BT.709 is [-Umax, Umax] and [-Vmax, Vmax] + private const double Umax = 0.436; + private const double Vmax = 0.615; + + // Constants used in BT.709 + private const double Wr = 0.2126; + private const double Wb = 0.0722; + + // Constants used in BT.601 + // private const double Wr = 0.299; + // private const double Wb = 0.114; + private const double Wg = 1 - Wr - Wb; + + public static Span RgbToYuv(ReadOnlySpan row, bool normalized) + { + Rgb24[] result = new Rgb24[row.Length]; + for (int i = 0; i < row.Length; i++) + { + double[] current = RgbToYuv(row[i], normalized, true, false); + double y = Math.Max(0, Math.Min(255, Math.Round(current[0]))); + double u = Math.Max(0, Math.Min(255, Math.Round(current[1]))); + double v = Math.Max(0, Math.Min(255, Math.Round(current[2]))); + + result[i] = new Rgb24((byte)y, (byte)u, (byte)v); + } + + return result; + } + + public static double[] RgbToYuv(Rgb24 rgb, bool normalize = false, bool is_8bit = false, bool is_10bit = false) + { + double r = rgb.R / 255.0; + double g = rgb.G / 255.0; + double b = rgb.B / 255.0; + double y = (Wr * r) + (Wg * g) + (Wb * b); + double u = Umax * (b - y) / (1 - Wb); + double v = Vmax * (r - y) / (1 - Wr); + + // y[0, 1] u[-Umax, Umax] v[-Vmax, Vmax] + if (normalize) + { + u = (u + Umax) / (2 * Umax); + v = (v + Vmax) / (2 * Vmax); + + // y[0, 1] u[0, 1] v[0, 1] + } + + if (is_8bit) + { + y = Math.Round(y * 255); + u = Math.Round(u * 255); + v = Math.Round(v * 255); + + // y[0, 255] u[0, 255] v[0, 255] + } + + if (is_10bit) + { + y = Math.Round(y * 1023); + u = Math.Round(u * 1023); + v = Math.Round(v * 1023); + + // y[0, 1023] u[0, 1023] v[0, 1023] + } + + return [y, u, v]; + } + + public static Span YuvToRgb(Av1FrameBuffer frameBuffer, bool normalized) + { + Point pixelPosition = new Point(0, 1); + Span yRow = frameBuffer.DeriveBlockPointer(Av1Plane.Y, pixelPosition, 0, 0, out int _); + Span uRow = frameBuffer.DeriveBlockPointer(Av1Plane.U, pixelPosition, 0, 0, out int _); + Span vRow = frameBuffer.DeriveBlockPointer(Av1Plane.V, pixelPosition, 0, 0, out int _); + Rgb24[] result = new Rgb24[frameBuffer.Width]; + double[] yuv = new double[3]; + for (int i = 0; i < frameBuffer.Width; i++) + { + yuv[0] = yRow[i]; + yuv[1] = uRow[i]; + yuv[2] = vRow[i]; + double[] rgb = YuvToRgb(yuv, normalized, true, false); + double r = rgb[0] * 255; + double g = rgb[1] * 255; + double b = rgb[2] * 255; + byte redByte = (byte)Math.Max(0, Math.Min(255, Math.Round(r))); + byte greenByte = (byte)Math.Max(0, Math.Min(255, Math.Round(g))); + byte blueByte = (byte)Math.Max(0, Math.Min(255, Math.Round(b))); + + result[i] = new Rgb24(redByte, greenByte, blueByte); + } + + return result; + } + + public static double[] YuvToRgb(double[] yuv, bool normalized = false, bool is_8bit = false, bool is_10bit = false) + { + double y = yuv[0]; + double u = yuv[1]; + double v = yuv[2]; + if (is_8bit) + { + // y[0, 255] u[0, 255] v[0, 255] + y /= 255.0; + u /= 255.0; + v /= 255.0; + } + + if (is_10bit) + { + // y[0, 1023] u[0, 1023] v[0, 1023] + y /= 1023.0; + u /= 1023.0; + v /= 1023.0; + } + + if (normalized) + { + // y [0, 1], u [0, 1], v[0, 1] + u = (u - 0.5) * 2 * Umax; + v = (v - 0.5) * 2 * Vmax; + + // y [0, 1], u [-Umax, Umax], v[-Vmax, Vmax] + } + + // r = y + 1.28033 * v + // g = y - 0.21482 * u - 0.38059 * v + // b = y + 2.12798 * u + double r = y + (v * (1 - Wr) / Vmax); + double g = y - (u * Wb * (1 - Wb) / (Umax * Wg)) - (v * Wr * (1 - Wr) / (Vmax * Wg)); + double b = y + (u * (1 - Wb) / Umax); + + return [r, g, b]; + } +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ResidualBuilderTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ResidualBuilderTests.cs new file mode 100644 index 0000000000..64ac67a3e6 --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ResidualBuilderTests.cs @@ -0,0 +1,504 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.InteropServices; +using System.Runtime.Intrinsics; +using SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline; +using SixLabors.ImageSharp.Tests.TestUtilities; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1; + +/// +/// Verifies AV1 residual construction against source-minus-prediction reference arithmetic. +/// +[Trait("Format", "Avif")] +public class Av1ResidualBuilderTests +{ + /// + /// Verifies precision limits, independent strides, candidate order, and exact final-row bounds. + /// + [Theory] + [InlineData(255, false)] + [InlineData(255, true)] + public void ByteSearchMetricsMatchKnownMoments(int maximum, bool negative) + { + const int SourceStride = 13; + const int PredictionStride = 17; + const int SourceOffset = 1; + const int PredictionOffset = 3; + byte[] sourceBuffer = new byte[SourceOffset + (7 * SourceStride) + 8]; + byte[] predictionBuffer = new byte[PredictionOffset + (7 * PredictionStride) + 11]; + sourceBuffer.AsSpan().Fill((byte)maximum); + predictionBuffer.AsSpan().Fill((byte)maximum); + Span source = sourceBuffer.AsSpan(SourceOffset); + Span prediction = predictionBuffer.AsSpan(PredictionOffset); + int[] sums = [-1, 0, 0, 0, 0, -1]; + + // The last source row contains exactly eight samples, and the four prediction windows require + // exactly eleven. Distinct nonzero padding catches accidental participation of neighboring rows. + for (int row = 0; row < 8; row++) + { + source.Slice(row * SourceStride, 8).Fill((byte)(negative ? 0 : maximum)); + prediction.Slice(row * PredictionStride, 11).Fill((byte)(negative ? maximum : 0)); + } + + Assert.Equal(64 * maximum, Av1ResidualBuilder.SumAbsoluteDifferences8x8(source, SourceStride, prediction, PredictionStride)); + Av1ResidualBuilder.GetMoments8x8(source, SourceStride, prediction, PredictionStride, out int sum, out int squaredSum); + Assert.Equal((negative ? -64 : 64) * maximum, sum); + Assert.Equal(64 * maximum * maximum, squaredSum); + Av1ResidualBuilder.SumFourAbsoluteDifferences8x8(source, SourceStride, prediction, PredictionStride, sums.AsSpan(1, 4)); + Assert.Equal(new[] { -1, 64 * maximum, 64 * maximum, 64 * maximum, 64 * maximum, -1 }, sums); + + // Source (2*y + x) and prediction (y + 2*x) give residual (y - x) * step, exercising both signs + // and distinct rows. Across the 8x8 square its signed sum is zero, absolute sum 168, and squared sum 672. + // Offsetting prediction by 1, 2, or 3 columns gives absolute sums 198, 276, and 386 respectively. + int step = maximum / 32; + for (int row = 0; row < 8; row++) + { + for (int column = 0; column < 8; column++) + { + source[(row * SourceStride) + column] = (byte)(((2 * row) + column) * step); + } + + for (int column = 0; column < 11; column++) + { + prediction[(row * PredictionStride) + column] = (byte)((row + (2 * column)) * step); + } + } + + Assert.Equal(168 * step, Av1ResidualBuilder.SumAbsoluteDifferences8x8(source, SourceStride, prediction, PredictionStride)); + Av1ResidualBuilder.GetMoments8x8(source, SourceStride, prediction, PredictionStride, out sum, out squaredSum); + Assert.Equal(0, sum); + Assert.Equal(672 * step * step, squaredSum); + Av1ResidualBuilder.SumFourAbsoluteDifferences8x8(source, SourceStride, prediction, PredictionStride, sums.AsSpan(1, 4)); + Assert.Equal(new[] { -1, 168 * step, 198 * step, 276 * step, 386 * step, -1 }, sums); + } + + /// + /// Verifies precision limits, independent strides, candidate order, and exact final-row bounds. + /// + [Theory] + [InlineData(1023, false)] + [InlineData(1023, true)] + [InlineData(4095, false)] + [InlineData(4095, true)] + public void HighBitDepthSearchMetricsMatchKnownMoments(int maximum, bool negative) + { + const int SourceStride = 13; + const int PredictionStride = 17; + const int SourceOffset = 1; + const int PredictionOffset = 3; + ushort[] sourceBuffer = new ushort[SourceOffset + (7 * SourceStride) + 8]; + ushort[] predictionBuffer = new ushort[PredictionOffset + (7 * PredictionStride) + 11]; + sourceBuffer.AsSpan().Fill((ushort)maximum); + predictionBuffer.AsSpan().Fill((ushort)maximum); + Span source = sourceBuffer.AsSpan(SourceOffset); + Span prediction = predictionBuffer.AsSpan(PredictionOffset); + int[] sums = [-1, 0, 0, 0, 0, -1]; + + // The last source row contains exactly eight samples, and the four prediction windows require + // exactly eleven. Distinct nonzero padding catches accidental participation of neighboring rows. + for (int row = 0; row < 8; row++) + { + source.Slice(row * SourceStride, 8).Fill((ushort)(negative ? 0 : maximum)); + prediction.Slice(row * PredictionStride, 11).Fill((ushort)(negative ? maximum : 0)); + } + + Assert.Equal(64 * maximum, Av1ResidualBuilder.SumAbsoluteDifferences8x8(source, SourceStride, prediction, PredictionStride)); + Av1ResidualBuilder.GetMoments8x8(source, SourceStride, prediction, PredictionStride, out int sum, out int squaredSum); + Assert.Equal((negative ? -64 : 64) * maximum, sum); + Assert.Equal(64 * maximum * maximum, squaredSum); + Av1ResidualBuilder.SumFourAbsoluteDifferences8x8(source, SourceStride, prediction, PredictionStride, sums.AsSpan(1, 4)); + Assert.Equal(new[] { -1, 64 * maximum, 64 * maximum, 64 * maximum, 64 * maximum, -1 }, sums); + + // Source (2*y + x) and prediction (y + 2*x) give residual (y - x) * step, exercising both signs + // and distinct rows. Across the 8x8 square its signed sum is zero, absolute sum 168, and squared sum 672. + // Offsetting prediction by 1, 2, or 3 columns gives absolute sums 198, 276, and 386 respectively. + int step = maximum / 32; + for (int row = 0; row < 8; row++) + { + for (int column = 0; column < 8; column++) + { + source[(row * SourceStride) + column] = (ushort)(((2 * row) + column) * step); + } + + for (int column = 0; column < 11; column++) + { + prediction[(row * PredictionStride) + column] = (ushort)((row + (2 * column)) * step); + } + } + + Assert.Equal(168 * step, Av1ResidualBuilder.SumAbsoluteDifferences8x8(source, SourceStride, prediction, PredictionStride)); + Av1ResidualBuilder.GetMoments8x8(source, SourceStride, prediction, PredictionStride, out sum, out squaredSum); + Assert.Equal(0, sum); + Assert.Equal(672 * step * step, squaredSum); + Av1ResidualBuilder.SumFourAbsoluteDifferences8x8(source, SourceStride, prediction, PredictionStride, sums.AsSpan(1, 4)); + Assert.Equal(new[] { -1, 168 * step, 198 * step, 276 * step, 386 * step, -1 }, sums); + } + + private const HwIntrinsics ResidualConfigurations = + HwIntrinsics.AllowAll | HwIntrinsics.DisableAVX512F | HwIntrinsics.DisableAVX | HwIntrinsics.DisableHWIntrinsic; + + /// + /// Verifies 8-bit, 10-bit, and 12-bit residuals across misaligned planes, independent strides, and SIMD tails. + /// + [Fact] + public void ResidualsMatchReferenceAcrossHardwareWidths() + => FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidateResiduals, ResidualConfigurations); + + /// + /// Verifies every width-specific operator even when the current processor cannot select that width in the driver. + /// + [Fact] + public void ResidualOperatorsMatchReferenceAtEveryVectorWidth() + { + byte[] byteSource = new byte[Vector512.Count]; + byte[] bytePrediction = new byte[Vector512.Count]; + short[] byteExpected = new short[Vector512.Count]; + short[] byteActual = new short[Vector512.Count]; + FillBytePlanes(byteSource, byteSource.Length, bytePrediction, bytePrediction.Length, byteSource.Length, 1); + FillReference(byteSource, byteSource.Length, bytePrediction, bytePrediction.Length, byteExpected, byteExpected.Length, byteExpected.Length, 1); + + ref byte byteSourceBase = ref MemoryMarshal.GetArrayDataReference(byteSource); + ref byte bytePredictionBase = ref MemoryMarshal.GetArrayDataReference(bytePrediction); + Vector128 byteLower128 = Av1ResidualBuilder.ByteOperator.Subtract( + Unsafe.As>(ref byteSourceBase), + Unsafe.As>(ref bytePredictionBase), + out Vector128 byteUpper128); + + byteLower128.CopyTo(byteActual); + byteUpper128.CopyTo(byteActual.AsSpan(Vector128.Count)); + AssertEqual(byteExpected, byteActual, Vector128.Count); + + Vector256 byteLower256 = Av1ResidualBuilder.ByteOperator.Subtract( + Unsafe.As>(ref byteSourceBase), + Unsafe.As>(ref bytePredictionBase), + out Vector256 byteUpper256); + + byteLower256.CopyTo(byteActual); + byteUpper256.CopyTo(byteActual.AsSpan(Vector256.Count)); + AssertEqual(byteExpected, byteActual, Vector256.Count); + + Vector512 byteLower512 = Av1ResidualBuilder.ByteOperator.Subtract( + Unsafe.As>(ref byteSourceBase), + Unsafe.As>(ref bytePredictionBase), + out Vector512 byteUpper512); + + byteLower512.CopyTo(byteActual); + byteUpper512.CopyTo(byteActual.AsSpan(Vector512.Count)); + AssertEqual(byteExpected, byteActual, Vector512.Count); + + ushort[] uint16Source = new ushort[Vector512.Count]; + ushort[] uint16Prediction = new ushort[Vector512.Count]; + short[] uint16Expected = new short[Vector512.Count]; + short[] uint16Actual = new short[Vector512.Count]; + FillUInt16Planes(uint16Source, uint16Source.Length, uint16Prediction, uint16Prediction.Length, uint16Source.Length, 1, 4095); + FillReference(uint16Source, uint16Source.Length, uint16Prediction, uint16Prediction.Length, uint16Expected, uint16Expected.Length, uint16Expected.Length, 1); + + ref ushort uint16SourceBase = ref MemoryMarshal.GetArrayDataReference(uint16Source); + ref ushort uint16PredictionBase = ref MemoryMarshal.GetArrayDataReference(uint16Prediction); + Av1ResidualBuilder.UInt16Operator.Subtract( + Unsafe.As>(ref uint16SourceBase), + Unsafe.As>(ref uint16PredictionBase), + out _).CopyTo(uint16Actual); + + AssertEqual(uint16Expected, uint16Actual, Vector128.Count); + + Av1ResidualBuilder.UInt16Operator.Subtract( + Unsafe.As>(ref uint16SourceBase), + Unsafe.As>(ref uint16PredictionBase), + out _).CopyTo(uint16Actual); + + AssertEqual(uint16Expected, uint16Actual, Vector256.Count); + + Av1ResidualBuilder.UInt16Operator.Subtract( + Unsafe.As>(ref uint16SourceBase), + Unsafe.As>(ref uint16PredictionBase), + out _).CopyTo(uint16Actual); + + AssertEqual(uint16Expected, uint16Actual, Vector512.Count); + } + + /// + /// Verifies exact 12-bit residual energy through every hardware-selected vector width and the scalar tail. + /// + [Fact] + public void SumSquaresMatchesScalarAcrossHardwareWidths() + => FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidateSumSquares, ResidualConfigurations); + + /// + /// Verifies that repeated maximum-transform residual and squared-error processing uses only caller-owned buffers. + /// + [Fact] + public void ResidualConstructionDoesNotAllocate() + { + const int width = 64; + const int height = 64; + byte[] source = new byte[width * height]; + byte[] prediction = new byte[width * height]; + short[] residual = new short[width * height]; + ushort[] highBitDepthSource = new ushort[width * height]; + ushort[] highBitDepthPrediction = new ushort[width * height]; + short[] highBitDepthResidual = new short[width * height]; + FillBytePlanes(source, width, prediction, width, width, height); + FillUInt16Planes(highBitDepthSource, width, highBitDepthPrediction, width, width, height, 4095); + + Av1ResidualBuilder.Subtract(source, width, prediction, width, residual, width, width, height); + Av1ResidualBuilder.Subtract(highBitDepthSource, width, highBitDepthPrediction, width, highBitDepthResidual, width, width, height); + _ = Av1ResidualBuilder.SumSquaredError(source, width, prediction, width, width, height); + _ = Av1ResidualBuilder.SumSquaredError( + highBitDepthSource, + width, + highBitDepthPrediction, + width, + width, + height); + + int[] candidateSums = new int[4]; + _ = Av1ResidualBuilder.SumAbsoluteDifferences8x8(source, width, prediction, width); + _ = Av1ResidualBuilder.SumAbsoluteDifferences8x8(highBitDepthSource, width, highBitDepthPrediction, width); + Av1ResidualBuilder.SumFourAbsoluteDifferences8x8(source, width, prediction, width, candidateSums); + Av1ResidualBuilder.SumFourAbsoluteDifferences8x8(highBitDepthSource, width, highBitDepthPrediction, width, candidateSums); + Av1ResidualBuilder.GetMoments8x8(source, width, prediction, width, out _, out _); + Av1ResidualBuilder.GetMoments8x8(highBitDepthSource, width, highBitDepthPrediction, width, out _, out _); + + long sum = 0; + long before = GC.GetAllocatedBytesForCurrentThread(); + for (int iteration = 0; iteration < 32; iteration++) + { + Av1ResidualBuilder.Subtract(source, width, prediction, width, residual, width, width, height); + Av1ResidualBuilder.Subtract(highBitDepthSource, width, highBitDepthPrediction, width, highBitDepthResidual, width, width, height); + sum += Av1ResidualBuilder.SumSquaredError(source, width, prediction, width, width, height); + sum += Av1ResidualBuilder.SumSquaredError( + highBitDepthSource, + width, + highBitDepthPrediction, + width, + width, + height); + + sum += Av1ResidualBuilder.SumAbsoluteDifferences8x8(source, width, prediction, width); + sum += Av1ResidualBuilder.SumAbsoluteDifferences8x8(highBitDepthSource, width, highBitDepthPrediction, width); + Av1ResidualBuilder.SumFourAbsoluteDifferences8x8(source, width, prediction, width, candidateSums); + sum += candidateSums[0]; + Av1ResidualBuilder.SumFourAbsoluteDifferences8x8(highBitDepthSource, width, highBitDepthPrediction, width, candidateSums); + sum += candidateSums[3]; + Av1ResidualBuilder.GetMoments8x8(source, width, prediction, width, out int byteSum, out int byteSquares); + Av1ResidualBuilder.GetMoments8x8(highBitDepthSource, width, highBitDepthPrediction, width, out int wordSum, out int wordSquares); + sum += byteSum + byteSquares + wordSum + wordSquares; + } + + long allocated = GC.GetAllocatedBytesForCurrentThread() - before; + Assert.True(sum > 0); + Assert.Equal(0, allocated); + } + + private static void ValidateSumSquares() + { + short[] residual = new short[127]; + long expected = 0; + for (int index = 0; index < residual.Length; index++) + { + int magnitude = (index * 193) & 4095; + short value = (short)((index & 1) == 0 ? magnitude : -magnitude); + residual[index] = value; + expected += (long)value * value; + } + + residual[0] = -4095; + expected += 4095L * 4095; + Assert.Equal(expected, Av1ResidualBuilder.SumSquares(residual)); + } + + private static void ValidateResiduals() + { + ValidateByteResiduals(); + ValidateUInt16Residuals(1023); + ValidateUInt16Residuals(4095); + } + + private static void ValidateByteResiduals() + { + const int width = 127; + const int height = 3; + const int sourceStride = 131; + const int predictionStride = 137; + const int residualStride = 139; + const int sourceOffset = 1; + const int predictionOffset = 2; + const int residualOffset = 3; + byte[] source = new byte[sourceOffset + (sourceStride * height)]; + byte[] prediction = new byte[predictionOffset + (predictionStride * height)]; + short[] expected = new short[residualOffset + (residualStride * height)]; + short[] actual = new short[expected.Length]; + Array.Fill(expected, short.MinValue); + Array.Fill(actual, short.MinValue); + + Span sourcePlane = source.AsSpan(sourceOffset); + Span predictionPlane = prediction.AsSpan(predictionOffset); + Span expectedPlane = expected.AsSpan(residualOffset); + Span actualPlane = actual.AsSpan(residualOffset); + FillBytePlanes(sourcePlane, sourceStride, predictionPlane, predictionStride, width, height); + FillReference(sourcePlane, sourceStride, predictionPlane, predictionStride, expectedPlane, residualStride, width, height); + + Av1ResidualBuilder.Subtract(sourcePlane, sourceStride, predictionPlane, predictionStride, actualPlane, residualStride, width, height); + + Assert.Equal(expected, actual); + long expectedSquaredError = CalculateReferenceSquaredError(expectedPlane, residualStride, width, height); + + Assert.Equal( + expectedSquaredError, + Av1ResidualBuilder.SumSquaredError( + sourcePlane, + sourceStride, + predictionPlane, + predictionStride, + width, + height)); + } + + private static void ValidateUInt16Residuals(int maximumSample) + { + const int width = 127; + const int height = 3; + const int sourceStride = 131; + const int predictionStride = 137; + const int residualStride = 139; + const int sourceOffset = 1; + const int predictionOffset = 2; + const int residualOffset = 3; + ushort[] source = new ushort[sourceOffset + (sourceStride * height)]; + ushort[] prediction = new ushort[predictionOffset + (predictionStride * height)]; + short[] expected = new short[residualOffset + (residualStride * height)]; + short[] actual = new short[expected.Length]; + Array.Fill(expected, short.MinValue); + Array.Fill(actual, short.MinValue); + + Span sourcePlane = source.AsSpan(sourceOffset); + Span predictionPlane = prediction.AsSpan(predictionOffset); + Span expectedPlane = expected.AsSpan(residualOffset); + Span actualPlane = actual.AsSpan(residualOffset); + FillUInt16Planes(sourcePlane, sourceStride, predictionPlane, predictionStride, width, height, maximumSample); + FillReference(sourcePlane, sourceStride, predictionPlane, predictionStride, expectedPlane, residualStride, width, height); + + Av1ResidualBuilder.Subtract(sourcePlane, sourceStride, predictionPlane, predictionStride, actualPlane, residualStride, width, height); + + Assert.Equal(expected, actual); + long expectedSquaredError = CalculateReferenceSquaredError(expectedPlane, residualStride, width, height); + + Assert.Equal( + expectedSquaredError, + Av1ResidualBuilder.SumSquaredError( + sourcePlane, + sourceStride, + predictionPlane, + predictionStride, + width, + height)); + } + + private static void FillBytePlanes(Span source, int sourceStride, Span prediction, int predictionStride, int width, int height) + { + for (int y = 0; y < height; y++) + { + for (int x = 0; x < width; x++) + { + source[(y * sourceStride) + x] = (byte)(((x * 37) + (y * 19) + 251) & byte.MaxValue); + prediction[(y * predictionStride) + x] = (byte)(((x * 11) + (y * 43) + 127) & byte.MaxValue); + } + } + + source[0] = byte.MaxValue; + prediction[0] = 0; + source[1] = 0; + prediction[1] = byte.MaxValue; + } + + private static void FillUInt16Planes(Span source, int sourceStride, Span prediction, int predictionStride, int width, int height, int maximumSample) + { + for (int y = 0; y < height; y++) + { + for (int x = 0; x < width; x++) + { + source[(y * sourceStride) + x] = (ushort)(((x * 197) + (y * 389) + maximumSample) & maximumSample); + prediction[(y * predictionStride) + x] = (ushort)(((x * 283) + (y * 151) + (maximumSample / 2)) & maximumSample); + } + } + + source[0] = (ushort)maximumSample; + prediction[0] = 0; + source[1] = 0; + prediction[1] = (ushort)maximumSample; + } + + private static void FillReference( + ReadOnlySpan source, + int sourceStride, + ReadOnlySpan prediction, + int predictionStride, + Span residual, + int residualStride, + int width, + int height) + { + for (int y = 0; y < height; y++) + { + for (int x = 0; x < width; x++) + { + residual[(y * residualStride) + x] = (short)(source[(y * sourceStride) + x] - prediction[(y * predictionStride) + x]); + } + } + } + + private static void FillReference( + ReadOnlySpan source, + int sourceStride, + ReadOnlySpan prediction, + int predictionStride, + Span residual, + int residualStride, + int width, + int height) + { + for (int y = 0; y < height; y++) + { + for (int x = 0; x < width; x++) + { + residual[(y * residualStride) + x] = (short)(source[(y * sourceStride) + x] - prediction[(y * predictionStride) + x]); + } + } + } + + /// + /// Calculates residual energy with scalar arithmetic independently of the vectorized implementation under test. + /// + private static long CalculateReferenceSquaredError( + ReadOnlySpan residual, + int residualStride, + int width, + int height) + { + long squaredError = 0; + + // Only coded samples contribute to the error metric; stride padding must remain outside the calculation. + for (int y = 0; y < height; y++) + { + for (int x = 0; x < width; x++) + { + int value = residual[(y * residualStride) + x]; + squaredError += value * value; + } + } + + return squaredError; + } + + private static void AssertEqual(ReadOnlySpan expected, ReadOnlySpan actual, int count) + { + for (int i = 0; i < count; i++) + { + Assert.Equal(expected[i], actual[i]); + } + } +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ScaledInterPredictorTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ScaledInterPredictorTests.cs new file mode 100644 index 0000000000..04472139c6 --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ScaledInterPredictorTests.cs @@ -0,0 +1,819 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; +using SixLabors.ImageSharp.Tests.TestUtilities; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1; + +/// +/// Verifies AV1 reference scaling and variable-phase inter convolution against the reference decoder arithmetic. +/// +[Trait("Format", "Avif")] +public class Av1ScaledInterPredictorTests +{ + /// + /// The number of fractional bits in each interpolation coefficient. + /// + private const int FilterBits = 7; + + /// + /// The ordinary first-pass rounding distance. + /// + private const int Round0Bits = 3; + + /// + /// The vertical shift that preserves AV1's compound intermediate precision. + /// + private const int CompoundRound1Bits = 7; + + /// + /// The number of samples in every stored interpolation row. + /// + private const int FilterTapCount = 8; + + /// + /// The source border retained around the independently generated active coordinates. + /// + private const int SourcePadding = 16; + + /// + /// The guarded destination elements before the active block. + /// + private const int DestinationPrefix = 11; + + /// + /// The guarded destination elements after each active row. + /// + private const int DestinationRowPadding = 9; + + /// + /// The guarded destination elements after the final row. + /// + private const int DestinationSuffix = 17; + + /// + /// The byte value used to detect writes outside the active destination block. + /// + private const byte ByteSentinel = 0xD3; + + /// + /// The ushort value used to detect writes outside the active destination block. + /// + private const ushort UInt16Sentinel = 0xDEAD; + + /// + /// Exercises the native vector path and the complete scalar fallback in separate processes. + /// + private const HwIntrinsics PredictorConfigurations = HwIntrinsics.AllowAll | HwIntrinsics.DisableHWIntrinsic; + + /// + /// Verifies the reference decoder's Q14 scale factors, Q10 steps, and signed coordinate rounding. + /// + [Fact] + public void ReferenceScaleMatchesFixedPointRules() + { + Av1ReferenceScale downscaledReference = new(40, 24, 64, 48); + + Assert.True(downscaledReference.IsScaled); + Assert.Equal(10240, downscaledReference.HorizontalScale); + Assert.Equal(8192, downscaledReference.VerticalScale); + Assert.Equal(640, downscaledReference.HorizontalStep); + Assert.Equal(512, downscaledReference.VerticalStep); + Assert.Equal(ScaleCoordinate(37, 10240), downscaledReference.ScaleHorizontal(37)); + Assert.Equal(ScaleCoordinate(-37, 10240), downscaledReference.ScaleHorizontal(-37)); + + Av1ReferenceScale enlargedReference = new(96, 72, 64, 48); + + Assert.Equal(24576, enlargedReference.HorizontalScale); + Assert.Equal(24576, enlargedReference.VerticalScale); + Assert.Equal(1536, enlargedReference.HorizontalStep); + Assert.Equal(1536, enlargedReference.VerticalStep); + + Av1ReferenceScale identity = new(64, 48, 64, 48); + + Assert.False(identity.IsScaled); + Assert.Equal(1024, identity.HorizontalStep); + Assert.Equal(1024, identity.VerticalStep); + } + + /// + /// Verifies exact scaled 8-bit output, variable filter phases, vector tails, and untouched destination padding. + /// + [Fact] + public void BytePredictionMatchesReference() + => FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidateBytePredictions, PredictorConfigurations); + + /// + /// Verifies exact scaled 8-, 10-, and 12-bit output under the native vector and scalar configurations. + /// + [Fact] + public void HighBitDepthPredictionMatchesReference() + => FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidateHighBitDepthPredictions, PredictorConfigurations); + + /// + /// Verifies exact scaled 8-bit no-round compound intermediates under native vector and scalar configurations. + /// + [Fact] + public void ByteCompoundPredictionMatchesReference() + => FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidateByteCompoundPredictions, PredictorConfigurations); + + /// + /// Verifies exact scaled 8-, 10-, and 12-bit no-round compound intermediates under native vector and scalar configurations. + /// + [Fact] + public void HighBitDepthCompoundPredictionMatchesReference() + => FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidateHighBitDepthCompoundPredictions, PredictorConfigurations); + + /// + /// Applies each scaled-prediction scenario to byte storage. + /// + private static void ValidateBytePredictions() + { + foreach (ScaledPredictionCase testCase in CreatePredictionCases()) + { + byte[] source = CreateByteSource(testCase, out int sourceStride, out int sourceOrigin); + int destinationStride = testCase.Width + DestinationRowPadding; + byte[] expected = CreateByteDestination(testCase, destinationStride); + byte[] actual = (byte[])expected.Clone(); + short[] scratch = new short[ + Av1ScaledInterPredictor.GetScaledScratchLength( + testCase.Width, + testCase.Height, + testCase.VerticalPhase, + testCase.VerticalStep)]; + + ApplyReference(source, sourceStride, sourceOrigin, expected, destinationStride, testCase, 8); + + Av1ScaledInterPredictor.PredictScaled( + source, + sourceStride, + sourceOrigin, + actual.AsSpan(DestinationPrefix), + destinationStride, + testCase.Width, + testCase.Height, + testCase.HorizontalFilter, + testCase.VerticalFilter, + testCase.HorizontalPhase, + testCase.HorizontalStep, + testCase.VerticalPhase, + testCase.VerticalStep, + scratch); + + Assert.Equal(expected, actual); + } + } + + /// + /// Applies each scaled-prediction scenario to every supported high-bit-depth precision. + /// + private static void ValidateHighBitDepthPredictions() + { + int[] bitDepths = [8, 10, 12]; + foreach (int bitDepth in bitDepths) + { + foreach (ScaledPredictionCase testCase in CreatePredictionCases()) + { + ushort[] source = CreateUInt16Source(testCase, bitDepth, out int sourceStride, out int sourceOrigin); + int destinationStride = testCase.Width + DestinationRowPadding; + ushort[] expected = CreateUInt16Destination(testCase, destinationStride); + ushort[] actual = (ushort[])expected.Clone(); + short[] scratch = new short[ + Av1ScaledInterPredictor.GetScaledScratchLength( + testCase.Width, + testCase.Height, + testCase.VerticalPhase, + testCase.VerticalStep)]; + + ApplyReference(source, sourceStride, sourceOrigin, expected, destinationStride, testCase, bitDepth); + + Av1ScaledInterPredictor.PredictScaled( + source, + sourceStride, + sourceOrigin, + actual.AsSpan(DestinationPrefix), + destinationStride, + testCase.Width, + testCase.Height, + testCase.HorizontalFilter, + testCase.VerticalFilter, + testCase.HorizontalPhase, + testCase.HorizontalStep, + testCase.VerticalPhase, + testCase.VerticalStep, + bitDepth, + scratch); + + Assert.Equal(expected, actual); + } + } + } + + /// + /// Applies each scaled compound-prediction scenario to byte source storage. + /// + private static void ValidateByteCompoundPredictions() + { + foreach (ScaledPredictionCase testCase in CreatePredictionCases()) + { + byte[] source = CreateByteSource(testCase, out int sourceStride, out int sourceOrigin); + int destinationStride = testCase.Width + DestinationRowPadding; + ushort[] expected = CreateUInt16Destination(testCase, destinationStride); + ushort[] actual = (ushort[])expected.Clone(); + short[] scratch = new short[ + Av1ScaledInterPredictor.GetScaledScratchLength( + testCase.Width, + testCase.Height, + testCase.VerticalPhase, + testCase.VerticalStep)]; + + ApplyCompoundReference(source, sourceStride, sourceOrigin, expected, destinationStride, testCase, 8); + + Av1ScaledInterPredictor.PredictScaledCompound( + source, + sourceStride, + sourceOrigin, + actual.AsSpan(DestinationPrefix), + destinationStride, + testCase.Width, + testCase.Height, + testCase.HorizontalFilter, + testCase.VerticalFilter, + testCase.HorizontalPhase, + testCase.HorizontalStep, + testCase.VerticalPhase, + testCase.VerticalStep, + scratch); + + Assert.Equal(expected, actual); + } + } + + /// + /// Applies each scaled compound-prediction scenario to every supported high-bit-depth precision. + /// + private static void ValidateHighBitDepthCompoundPredictions() + { + int[] bitDepths = [8, 10, 12]; + foreach (int bitDepth in bitDepths) + { + foreach (ScaledPredictionCase testCase in CreatePredictionCases()) + { + ushort[] source = CreateUInt16Source(testCase, bitDepth, out int sourceStride, out int sourceOrigin); + int destinationStride = testCase.Width + DestinationRowPadding; + ushort[] expected = CreateUInt16Destination(testCase, destinationStride); + ushort[] actual = (ushort[])expected.Clone(); + short[] scratch = new short[ + Av1ScaledInterPredictor.GetScaledScratchLength( + testCase.Width, + testCase.Height, + testCase.VerticalPhase, + testCase.VerticalStep)]; + + ApplyCompoundReference(source, sourceStride, sourceOrigin, expected, destinationStride, testCase, bitDepth); + + Av1ScaledInterPredictor.PredictScaledCompound( + source, + sourceStride, + sourceOrigin, + actual.AsSpan(DestinationPrefix), + destinationStride, + testCase.Width, + testCase.Height, + testCase.HorizontalFilter, + testCase.VerticalFilter, + testCase.HorizontalPhase, + testCase.HorizontalStep, + testCase.VerticalPhase, + testCase.VerticalStep, + bitDepth, + scratch); + + Assert.Equal(expected, actual); + } + } + } + + /// + /// Creates cases covering variable phases, every filter family, reduced kernels, and vector tails. + /// + private static ScaledPredictionCase[] CreatePredictionCases() => + [ + new("fixture-regular-8x8", 8, 8, Av1InterpolationFilter.Regular, Av1InterpolationFilter.Regular, 800, 512, 800, 512), + new("fixture-regular-4x8", 4, 8, Av1InterpolationFilter.Regular, Av1InterpolationFilter.Regular, 800, 512, 800, 512), + new("fixture-regular-8x4", 8, 4, Av1InterpolationFilter.Regular, Av1InterpolationFilter.Regular, 800, 512, 800, 512), + new("bilinear-variable-phase", 13, 9, Av1InterpolationFilter.Bilinear, Av1InterpolationFilter.Bilinear, 192, 1536, 512, 640), + new("regular-smooth-wide", 20, 8, Av1InterpolationFilter.Regular, Av1InterpolationFilter.Smooth, 64, 2048, 448, 2048), + new("regular-sharp-all-widths", 37, 7, Av1InterpolationFilter.Regular, Av1InterpolationFilter.Sharp, 64, 2048, 512, 2048), + new("sharp-bilinear-tail", 12, 5, Av1InterpolationFilter.Sharp, Av1InterpolationFilter.Bilinear, 512, 2048, 192, 2048), + new("reduced-regular", 4, 8, Av1InterpolationFilter.Regular, Av1InterpolationFilter.Smooth, 192, 2048, 448, 2048), + new("reduced-sharp-maps-to-regular", 4, 8, Av1InterpolationFilter.Sharp, Av1InterpolationFilter.Smooth, 192, 2048, 448, 2048), + new("reduced-smooth", 8, 4, Av1InterpolationFilter.Regular, Av1InterpolationFilter.Smooth, 64, 2048, 832, 2048) + ]; + + /// + /// Creates deterministic padded byte source storage for one prediction case. + /// + private static byte[] CreateByteSource(ScaledPredictionCase testCase, out int stride, out int origin) + { + GetSourceGeometry(testCase, out int width, out int height); + stride = width; + origin = (SourcePadding * stride) + SourcePadding; + byte[] source = new byte[width * height]; + for (int row = 0; row < height; row++) + { + for (int column = 0; column < width; column++) + { + source[(row * stride) + column] = (byte)(((row * 29) + (column * 47) + (row * column * 3)) & byte.MaxValue); + } + } + + return source; + } + + /// + /// Creates deterministic padded ushort source storage for one prediction case. + /// + private static ushort[] CreateUInt16Source(ScaledPredictionCase testCase, int bitDepth, out int stride, out int origin) + { + GetSourceGeometry(testCase, out int width, out int height); + stride = width; + origin = (SourcePadding * stride) + SourcePadding; + int maximum = (1 << bitDepth) - 1; + ushort[] source = new ushort[width * height]; + for (int row = 0; row < height; row++) + { + for (int column = 0; column < width; column++) + { + source[(row * stride) + column] = (ushort)(((row * 269) + (column * 443) + (row * column * 31)) & maximum); + } + } + + return source; + } + + /// + /// Computes storage dimensions that keep every requested eight-tap read inside the test source. + /// + private static void GetSourceGeometry(ScaledPredictionCase testCase, out int width, out int height) + { + int maximumHorizontalPosition = testCase.HorizontalPhase + ((testCase.Width - 1) * testCase.HorizontalStep); + int maximumVerticalPosition = testCase.VerticalPhase + ((testCase.Height - 1) * testCase.VerticalStep); + width = (2 * SourcePadding) + (maximumHorizontalPosition >> Av1ReferenceScale.SubpixelBits) + FilterTapCount; + height = (2 * SourcePadding) + (maximumVerticalPosition >> Av1ReferenceScale.SubpixelBits) + FilterTapCount; + } + + /// + /// Creates a guarded byte destination initialized to its sentinel. + /// + private static byte[] CreateByteDestination(ScaledPredictionCase testCase, int stride) + { + byte[] destination = new byte[DestinationPrefix + (stride * testCase.Height) + DestinationSuffix]; + Array.Fill(destination, ByteSentinel); + return destination; + } + + /// + /// Creates a guarded ushort destination initialized to its sentinel. + /// + private static ushort[] CreateUInt16Destination(ScaledPredictionCase testCase, int stride) + { + ushort[] destination = new ushort[DestinationPrefix + (stride * testCase.Height) + DestinationSuffix]; + Array.Fill(destination, UInt16Sentinel); + return destination; + } + + /// + /// Applies the independent variable-phase two-pass reference convolution to byte storage. + /// + private static void ApplyReference( + byte[] source, + int sourceStride, + int sourceOrigin, + byte[] destination, + int destinationStride, + ScaledPredictionCase testCase, + int bitDepth) + { + short[] intermediate = CreateIntermediate(testCase); + int intermediateStride = testCase.Width; + int round0 = GetRound0Bits(bitDepth); + int horizontalBias = 1 << (bitDepth + FilterBits - 1); + Span coefficients = stackalloc short[FilterTapCount]; + + for (int row = 0; row < intermediate.Length / intermediateStride; row++) + { + for (int column = 0; column < testCase.Width; column++) + { + int position = testCase.HorizontalPhase + (column * testCase.HorizontalStep); + int sourceColumn = (position >> Av1ReferenceScale.SubpixelBits) - 3; + FillCoefficients(testCase.HorizontalFilter, (position & Av1ReferenceScale.SubpixelMask) >> 6, testCase.Width <= 4, coefficients); + int sourceIndex = sourceOrigin + ((row - 3) * sourceStride) + sourceColumn; + int sum = horizontalBias + Convolve(source, sourceIndex, coefficients); + intermediate[(row * intermediateStride) + column] = (short)RoundPowerOfTwo(sum, round0); + } + } + + WriteReference(intermediate, intermediateStride, destination, destinationStride, testCase, bitDepth); + } + + /// + /// Applies the independent variable-phase two-pass reference convolution to ushort storage. + /// + private static void ApplyReference( + ushort[] source, + int sourceStride, + int sourceOrigin, + ushort[] destination, + int destinationStride, + ScaledPredictionCase testCase, + int bitDepth) + { + short[] intermediate = CreateIntermediate(testCase); + int intermediateStride = testCase.Width; + int round0 = GetRound0Bits(bitDepth); + int horizontalBias = 1 << (bitDepth + FilterBits - 1); + Span coefficients = stackalloc short[FilterTapCount]; + + for (int row = 0; row < intermediate.Length / intermediateStride; row++) + { + for (int column = 0; column < testCase.Width; column++) + { + int position = testCase.HorizontalPhase + (column * testCase.HorizontalStep); + int sourceColumn = (position >> Av1ReferenceScale.SubpixelBits) - 3; + FillCoefficients(testCase.HorizontalFilter, (position & Av1ReferenceScale.SubpixelMask) >> 6, testCase.Width <= 4, coefficients); + int sourceIndex = sourceOrigin + ((row - 3) * sourceStride) + sourceColumn; + int sum = horizontalBias + Convolve(source, sourceIndex, coefficients); + intermediate[(row * intermediateStride) + column] = (short)RoundPowerOfTwo(sum, round0); + } + } + + WriteReference(intermediate, intermediateStride, destination, destinationStride, testCase, bitDepth); + } + + /// + /// Applies independent scaled compound convolution to byte source storage. + /// + private static void ApplyCompoundReference( + byte[] source, + int sourceStride, + int sourceOrigin, + ushort[] destination, + int destinationStride, + ScaledPredictionCase testCase, + int bitDepth) + { + short[] intermediate = CreateIntermediate(testCase); + int intermediateStride = testCase.Width; + int round0 = GetRound0Bits(bitDepth); + int horizontalBias = 1 << (bitDepth + FilterBits - 1); + Span coefficients = stackalloc short[FilterTapCount]; + + for (int row = 0; row < intermediate.Length / intermediateStride; row++) + { + for (int column = 0; column < testCase.Width; column++) + { + int position = testCase.HorizontalPhase + (column * testCase.HorizontalStep); + int sourceColumn = (position >> Av1ReferenceScale.SubpixelBits) - 3; + FillCoefficients(testCase.HorizontalFilter, (position & Av1ReferenceScale.SubpixelMask) >> 6, testCase.Width <= 4, coefficients); + int sourceIndex = sourceOrigin + ((row - 3) * sourceStride) + sourceColumn; + int sum = horizontalBias + Convolve(source, sourceIndex, coefficients); + intermediate[(row * intermediateStride) + column] = (short)RoundPowerOfTwo(sum, round0); + } + } + + WriteCompoundReference(intermediate, intermediateStride, destination, destinationStride, testCase, bitDepth); + } + + /// + /// Applies independent scaled compound convolution to ushort source storage. + /// + private static void ApplyCompoundReference( + ushort[] source, + int sourceStride, + int sourceOrigin, + ushort[] destination, + int destinationStride, + ScaledPredictionCase testCase, + int bitDepth) + { + short[] intermediate = CreateIntermediate(testCase); + int intermediateStride = testCase.Width; + int round0 = GetRound0Bits(bitDepth); + int horizontalBias = 1 << (bitDepth + FilterBits - 1); + Span coefficients = stackalloc short[FilterTapCount]; + + for (int row = 0; row < intermediate.Length / intermediateStride; row++) + { + for (int column = 0; column < testCase.Width; column++) + { + int position = testCase.HorizontalPhase + (column * testCase.HorizontalStep); + int sourceColumn = (position >> Av1ReferenceScale.SubpixelBits) - 3; + FillCoefficients(testCase.HorizontalFilter, (position & Av1ReferenceScale.SubpixelMask) >> 6, testCase.Width <= 4, coefficients); + int sourceIndex = sourceOrigin + ((row - 3) * sourceStride) + sourceColumn; + int sum = horizontalBias + Convolve(source, sourceIndex, coefficients); + intermediate[(row * intermediateStride) + column] = (short)RoundPowerOfTwo(sum, round0); + } + } + + WriteCompoundReference(intermediate, intermediateStride, destination, destinationStride, testCase, bitDepth); + } + + /// + /// Allocates the oracle's independently shaped intermediate block. + /// + private static short[] CreateIntermediate(ScaledPredictionCase testCase) + { + int height = ((((testCase.Height - 1) * testCase.VerticalStep) + testCase.VerticalPhase) >> Av1ReferenceScale.SubpixelBits) + FilterTapCount; + return new short[testCase.Width * height]; + } + + /// + /// Completes byte output from the horizontally filtered intermediate block. + /// + private static void WriteReference( + short[] intermediate, + int intermediateStride, + byte[] destination, + int destinationStride, + ScaledPredictionCase testCase, + int bitDepth) + { + int maximum = byte.MaxValue; + Span coefficients = stackalloc short[FilterTapCount]; + for (int row = 0; row < testCase.Height; row++) + { + int position = testCase.VerticalPhase + (row * testCase.VerticalStep); + int sourceRow = position >> Av1ReferenceScale.SubpixelBits; + FillCoefficients(testCase.VerticalFilter, (position & Av1ReferenceScale.SubpixelMask) >> 6, testCase.Height <= 4, coefficients); + for (int column = 0; column < testCase.Width; column++) + { + int value = FinishConvolution(intermediate, (sourceRow * intermediateStride) + column, intermediateStride, coefficients, bitDepth); + destination[DestinationPrefix + (row * destinationStride) + column] = (byte)Math.Clamp(value, 0, maximum); + } + } + } + + /// + /// Completes high-bit-depth output from the horizontally filtered intermediate block. + /// + private static void WriteReference( + short[] intermediate, + int intermediateStride, + ushort[] destination, + int destinationStride, + ScaledPredictionCase testCase, + int bitDepth) + { + int maximum = (1 << bitDepth) - 1; + Span coefficients = stackalloc short[FilterTapCount]; + for (int row = 0; row < testCase.Height; row++) + { + int position = testCase.VerticalPhase + (row * testCase.VerticalStep); + int sourceRow = position >> Av1ReferenceScale.SubpixelBits; + FillCoefficients(testCase.VerticalFilter, (position & Av1ReferenceScale.SubpixelMask) >> 6, testCase.Height <= 4, coefficients); + for (int column = 0; column < testCase.Width; column++) + { + int value = FinishConvolution(intermediate, (sourceRow * intermediateStride) + column, intermediateStride, coefficients, bitDepth); + destination[DestinationPrefix + (row * destinationStride) + column] = (ushort)Math.Clamp(value, 0, maximum); + } + } + } + + /// + /// Writes the biased no-round compound output from the horizontally filtered intermediate block. + /// + private static void WriteCompoundReference( + short[] intermediate, + int intermediateStride, + ushort[] destination, + int destinationStride, + ScaledPredictionCase testCase, + int bitDepth) + { + int round0 = GetRound0Bits(bitDepth); + int offsetBits = bitDepth + (2 * FilterBits) - round0; + int verticalBias = 1 << offsetBits; + Span coefficients = stackalloc short[FilterTapCount]; + for (int row = 0; row < testCase.Height; row++) + { + int position = testCase.VerticalPhase + (row * testCase.VerticalStep); + int sourceRow = position >> Av1ReferenceScale.SubpixelBits; + FillCoefficients(testCase.VerticalFilter, (position & Av1ReferenceScale.SubpixelMask) >> 6, testCase.Height <= 4, coefficients); + for (int column = 0; column < testCase.Width; column++) + { + int sum = verticalBias + Convolve( + intermediate, + (sourceRow * intermediateStride) + column, + intermediateStride, + coefficients); + + destination[DestinationPrefix + (row * destinationStride) + column] = + (ushort)RoundPowerOfTwo(sum, CompoundRound1Bits); + } + } + } + + /// + /// Removes both normative convolution biases after the vertical pass. + /// + private static int FinishConvolution( + short[] intermediate, + int sourceIndex, + int sourceStride, + ReadOnlySpan coefficients, + int bitDepth) + { + int round0 = GetRound0Bits(bitDepth); + int round1 = (2 * FilterBits) - round0; + int offsetBits = bitDepth + (2 * FilterBits) - round0; + int verticalBias = 1 << offsetBits; + int roundOffset = (1 << (offsetBits - round1)) + (1 << (offsetBits - round1 - 1)); + int sum = verticalBias + Convolve(intermediate, sourceIndex, sourceStride, coefficients); + return RoundPowerOfTwo(sum, round1) - roundOffset; + } + + /// + /// Computes one byte convolution sum. + /// + private static int Convolve(byte[] source, int sourceIndex, ReadOnlySpan coefficients) + { + int sum = 0; + for (int tap = 0; tap < FilterTapCount; tap++) + { + sum += source[sourceIndex + tap] * coefficients[tap]; + } + + return sum; + } + + /// + /// Computes one ushort convolution sum. + /// + private static int Convolve(ushort[] source, int sourceIndex, ReadOnlySpan coefficients) + { + int sum = 0; + for (int tap = 0; tap < FilterTapCount; tap++) + { + sum += source[sourceIndex + tap] * coefficients[tap]; + } + + return sum; + } + + /// + /// Computes one vertical convolution sum from the biased intermediate block. + /// + private static int Convolve(short[] source, int sourceIndex, int sourceStride, ReadOnlySpan coefficients) + { + int sum = 0; + for (int tap = 0; tap < FilterTapCount; tap++) + { + sum += source[sourceIndex + (tap * sourceStride)] * coefficients[tap]; + } + + return sum; + } + + /// + /// Selects one reference coefficient row without reading production filter storage. + /// + private static void FillCoefficients(Av1InterpolationFilter filter, int phase, bool reduced, Span destination) + { + destination.Clear(); + if (filter == Av1InterpolationFilter.Bilinear) + { + destination[3] = (short)(128 - (phase * 8)); + destination[4] = (short)(phase * 8); + return; + } + + if (reduced && filter == Av1InterpolationFilter.Sharp) + { + filter = Av1InterpolationFilter.Regular; + } + + ReadOnlySpan source = (filter, reduced, phase) switch + { + (Av1InterpolationFilter.Regular, false, 1) => [0, 2, -6, 126, 8, -2, 0, 0], + (Av1InterpolationFilter.Regular, false, 4) => [0, 2, -14, 110, 38, -10, 2, 0], + (Av1InterpolationFilter.Regular, false, 12) => [0, 2, -10, 38, 110, -14, 2, 0], + (Av1InterpolationFilter.Smooth, false, 7) => [0, -2, 16, 54, 48, 12, 0, 0], + (Av1InterpolationFilter.Sharp, false, 8) => [-4, 12, -24, 80, 80, -24, 12, -4], + (Av1InterpolationFilter.Regular, true, 3) => [0, 0, -10, 116, 28, -6, 0, 0], + (Av1InterpolationFilter.Regular, true, 4) => [0, 0, -12, 110, 38, -8, 0, 0], + (Av1InterpolationFilter.Regular, true, 12) => [0, 0, -8, 38, 110, -12, 0, 0], + (Av1InterpolationFilter.Smooth, true, 13) => [0, 0, 4, 40, 62, 22, 0, 0], + _ => throw new InvalidOperationException($"The scaled oracle has no row for {filter}, phase {phase}, reduced {reduced}.") + }; + + source.CopyTo(destination); + } + + /// + /// Computes the reference decoder's bit-depth-dependent first-pass shift. + /// + private static int GetRound0Bits(int bitDepth) + { + int intermediateRange = bitDepth + FilterBits - Round0Bits + 2; + return Round0Bits + Math.Max(intermediateRange - 16, 0); + } + + /// + /// Applies integer power-of-two rounding. + /// + private static int RoundPowerOfTwo(int value, int bits) => (value + (1 << (bits - 1))) >> bits; + + /// + /// Independently applies the reference decoder's signed Q14-to-Q10 scale conversion. + /// + private static int ScaleCoordinate(int value, int scale) + { + long scaled = ((long)value * scale) + ((scale - (1 << 14)) * 8L); + const int shift = 8; + const long rounding = 1L << (shift - 1); + return scaled < 0 + ? (int)-((-scaled + rounding) >> shift) + : (int)((scaled + rounding) >> shift); + } + + /// + /// Describes one scaled prediction case. + /// + private readonly struct ScaledPredictionCase + { + /// + /// Initializes a new instance of the struct. + /// + public ScaledPredictionCase( + string name, + int width, + int height, + Av1InterpolationFilter horizontalFilter, + Av1InterpolationFilter verticalFilter, + int horizontalPhase, + int horizontalStep, + int verticalPhase, + int verticalStep) + { + this.Name = name; + this.Width = width; + this.Height = height; + this.HorizontalFilter = horizontalFilter; + this.VerticalFilter = verticalFilter; + this.HorizontalPhase = horizontalPhase; + this.HorizontalStep = horizontalStep; + this.VerticalPhase = verticalPhase; + this.VerticalStep = verticalStep; + } + + /// + /// Gets the diagnostic case name. + /// + public string Name { get; } + + /// + /// Gets the output width. + /// + public int Width { get; } + + /// + /// Gets the output height. + /// + public int Height { get; } + + /// + /// Gets the horizontal interpolation filter. + /// + public Av1InterpolationFilter HorizontalFilter { get; } + + /// + /// Gets the vertical interpolation filter. + /// + public Av1InterpolationFilter VerticalFilter { get; } + + /// + /// Gets the initial horizontal Q10 position. + /// + public int HorizontalPhase { get; } + + /// + /// Gets the horizontal Q10 source step. + /// + public int HorizontalStep { get; } + + /// + /// Gets the initial vertical Q10 position. + /// + public int VerticalPhase { get; } + + /// + /// Gets the vertical Q10 source step. + /// + public int VerticalStep { get; } + + /// + public override string ToString() => this.Name; + } +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ScanOrderTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ScanOrderTests.cs new file mode 100644 index 0000000000..c8c59ce21e --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ScanOrderTests.cs @@ -0,0 +1,120 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1; + +[Trait("Format", "Avif")] +public class Av1ScanOrderTests +{ + [Theory] + [MemberData(nameof(GetCombinations))] + internal void AllIndicesScannedExactlyOnce(int s, int t) + { + // Assign + HashSet visitedScans = []; + Av1TransformSize transformSize = (Av1TransformSize)s; + Av1TransformType transformType = (Av1TransformType)t; + + // Act + Av1ScanOrder scanOrder = Av1ScanOrderConstants.GetScanOrder(transformSize, transformType); + + // Assert + foreach (short scan in scanOrder.Scan) + { + Assert.False(visitedScans.Contains(scan), $"Scan {scan} already visited before."); + visitedScans.Add(scan); + } + } + + [Theory] + [MemberData(nameof(GetCombinations))] + internal void AllIndicesScannedAreWithinRange(int s, int t) + { + // Assign + Av1TransformSize transformSize = (Av1TransformSize)s; + Av1TransformType transformType = (Av1TransformType)t; + int lowValue = 0; + + // Act + Av1ScanOrder scanOrder = Av1ScanOrderConstants.GetScanOrder(transformSize, transformType); + int highValue = scanOrder.Scan.Length - 1; + + // Assert + foreach (short scan in scanOrder.Scan) + { + Assert.InRange(scan, lowValue, highValue); + } + } + + [Theory] + [MemberData(nameof(GetCombinations))] + internal void CorrectNumberOfIndicesScanned(int s, int t) + { + // Assign + Av1TransformSize transformSize = (Av1TransformSize)s; + Av1TransformType transformType = (Av1TransformType)t; + int width = Math.Min(transformSize.GetWidth(), 32); + int height = Math.Min(transformSize.GetHeight(), 32); + + // Act + Av1ScanOrder scanOrder = Av1ScanOrderConstants.GetScanOrder(transformSize, transformType); + + // Assert + Assert.Equal(width * height, scanOrder.Scan.Length); + } + + [Theory] + [MemberData(nameof(GetCombinations))] + internal void AllIndicesAreInDiagonalOrder(int s, int t) + { + // Assign + Av1TransformSize transformSize = (Av1TransformSize)s; + Av1TransformType transformType = (Av1TransformType)t; + int width = Math.Min(transformSize.GetWidth(), 32); + int height = Math.Min(transformSize.GetHeight(), 32); + + // Act + Av1ScanOrder scanOrder = Av1ScanOrderConstants.GetScanOrder(transformSize, transformType); + + // Assert + HashSet visited = []; + ReadOnlySpan scan = scanOrder.Scan; + + // In reverse order, the indiced used in + // must already be known. + for (int i = scanOrder.Scan.Length - 1; i >= 0; i--) + { + visited.Add(scan[i]); + if (scan.Length > i + 1) + { + Assert.Contains(scan[i + 1], visited); + } + + if (scan.Length > i + width) + { + Assert.Contains(scan[i + width], visited); + } + + if (scan.Length > i + width + 1) + { + Assert.Contains(scan[i + width + 1], visited); + } + } + } + + public static TheoryData GetCombinations() + { + TheoryData combinations = []; + for (int s = 0; s < (int)Av1TransformSize.AllSizes; s++) + { + for (int t = 0; t < (int)Av1TransformType.AllTransformTypes; t++) + { + combinations.Add(s, t); + } + } + + return combinations; + } +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1SelectableCompoundEntropyTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1SelectableCompoundEntropyTests.cs new file mode 100644 index 0000000000..ad67d9365f --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1SelectableCompoundEntropyTests.cs @@ -0,0 +1,154 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Buffers; +using SixLabors.ImageSharp.Formats.Heif.Av1; +using SixLabors.ImageSharp.Formats.Heif.Av1.Entropy; +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; +using SixLabors.ImageSharp.Memory; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1; + +/// +/// Verifies the adaptive syntax distributions used by selectable compound and inter-intra prediction. +/// +[Trait("Format", "Avif")] +public class Av1SelectableCompoundEntropyTests +{ + /// + /// Verifies representative and complete multi-symbol defaults against the reference decoder's forward Q15 tables. + /// + [Fact] + public void DefaultsMatchReference() + { + AssertForwardThresholds(Av1DefaultDistributions.InterIntraMode[1], [1875, 11082, 27332]); + AssertForwardThresholds(Av1DefaultDistributions.WedgeInterIntra[(int)Av1BlockSize.Block8x8], [20036]); + AssertForwardThresholds(Av1DefaultDistributions.CompoundType[(int)Av1BlockSize.Block8x8], [23431]); + AssertForwardThresholds( + Av1DefaultDistributions.WedgeIndex[(int)Av1BlockSize.Block8x8], + [2438, 4440, 6599, 8663, 11005, 12874, 15751, 18094, 20359, 22362, 24127, 25702, 27752, 29450, 31171]); + + ReadOnlySpan compoundIndex = [18244, 12865, 7053, 13259, 9334, 4644]; + ReadOnlySpan compoundGroupIndex = [26607, 22891, 18840, 24594, 19934, 22674]; + + for (int context = 0; context < compoundIndex.Length; context++) + { + AssertForwardThresholds(Av1DefaultDistributions.CompoundIndex[context], [compoundIndex[context]]); + AssertForwardThresholds(Av1DefaultDistributions.CompoundGroupIndex[context], [compoundGroupIndex[context]]); + } + } + + /// + /// Verifies that every new reader selects and adapts its intended context without consuming adjacent syntax. + /// + [Fact] + public void ReadersRoundTripInNormativeOrder() + { + const int groupContext = 4; + const int compoundIndexContext = 2; + Av1BlockSize blockSize = Av1BlockSize.Block8x8; + using Av1SymbolWriter writer = new(Configuration.Default, 32, updateCdf: true); + writer.WriteSymbol((int)Av1InterIntraMode.Horizontal, Av1DefaultDistributions.InterIntraMode[blockSize.GetSizeGroup()]); + writer.WriteSymbol(true, Av1DefaultDistributions.WedgeInterIntra[(int)blockSize]); + writer.WriteSymbol(13, Av1DefaultDistributions.WedgeIndex[(int)blockSize]); + writer.WriteSymbol(true, Av1DefaultDistributions.CompoundGroupIndex[groupContext]); + writer.WriteSymbol(false, Av1DefaultDistributions.CompoundIndex[compoundIndexContext]); + writer.WriteSymbol(1, Av1DefaultDistributions.CompoundType[(int)blockSize]); + + using IMemoryOwner encoded = writer.Exit(); + Av1SymbolDecoder decoder = new(Configuration.Default, encoded.GetSpan(), 0, updateCdf: true); + + Assert.Equal(Av1InterIntraMode.Horizontal, decoder.ReadInterIntraMode(blockSize)); + Assert.True(decoder.ReadUseInterIntraWedge(blockSize)); + Assert.Equal(13, decoder.ReadWedgeIndex(blockSize)); + Assert.True(decoder.ReadCompoundGroupIndex(groupContext)); + Assert.False(decoder.ReadCompoundIndex(compoundIndexContext)); + Assert.Equal(Av1CompoundType.DifferenceWeighted, decoder.ReadMaskedCompoundType(blockSize)); + } + + /// + /// Verifies copying, resetting, and snapshot publication for every selectable-compound distribution family. + /// + [Fact] + public void FrameEntropyLifecycleIncludesSelectableCompoundFamilies() + { + Av1FrameEntropyContext source = new(0); + Av1FrameEntropyContext copy = new(0); + Av1FrameEntropyContext snapshot = new(0); + Av1FrameEntropyContext defaults = new(0); + Av1Distribution[] sourceDistributions = + [ + source.InterIntraMode[1], + source.WedgeInterIntra[(int)Av1BlockSize.Block8x8], + source.CompoundType[(int)Av1BlockSize.Block8x8], + source.WedgeIndex[(int)Av1BlockSize.Block8x8], + source.CompoundIndex[2], + source.CompoundGroupIndex[4], + ]; + + Av1Distribution[] copyDistributions = + [ + copy.InterIntraMode[1], + copy.WedgeInterIntra[(int)Av1BlockSize.Block8x8], + copy.CompoundType[(int)Av1BlockSize.Block8x8], + copy.WedgeIndex[(int)Av1BlockSize.Block8x8], + copy.CompoundIndex[2], + copy.CompoundGroupIndex[4], + ]; + + Av1Distribution[] snapshotDistributions = + [ + snapshot.InterIntraMode[1], + snapshot.WedgeInterIntra[(int)Av1BlockSize.Block8x8], + snapshot.CompoundType[(int)Av1BlockSize.Block8x8], + snapshot.WedgeIndex[(int)Av1BlockSize.Block8x8], + snapshot.CompoundIndex[2], + snapshot.CompoundGroupIndex[4], + ]; + + Av1Distribution[] defaultDistributions = + [ + defaults.InterIntraMode[1], + defaults.WedgeInterIntra[(int)Av1BlockSize.Block8x8], + defaults.CompoundType[(int)Av1BlockSize.Block8x8], + defaults.WedgeIndex[(int)Av1BlockSize.Block8x8], + defaults.CompoundIndex[2], + defaults.CompoundGroupIndex[4], + ]; + + foreach (Av1Distribution distribution in sourceDistributions) + { + distribution.Update(distribution.NumberOfSymbols - 1); + } + + copy.CopyFrom(source); + source.SnapshotTo(snapshot); + + for (int index = 0; index < sourceDistributions.Length; index++) + { + Assert.NotSame(sourceDistributions[index], copyDistributions[index]); + Assert.NotSame(sourceDistributions[index], snapshotDistributions[index]); + Assert.Equal(sourceDistributions[index][0], copyDistributions[index][0]); + Assert.Equal(sourceDistributions[index][0], snapshotDistributions[index][0]); + } + + source.ResetToDefaults(0); + + for (int index = 0; index < sourceDistributions.Length; index++) + { + Assert.Equal(defaultDistributions[index][0], sourceDistributions[index][0]); + } + } + + /// + /// Compares one inverse-cumulative distribution with pinned forward thresholds. + /// + private static void AssertForwardThresholds(Av1Distribution distribution, ReadOnlySpan forwardThresholds) + { + Assert.Equal(forwardThresholds.Length + 1, distribution.NumberOfSymbols); + for (int index = 0; index < forwardThresholds.Length; index++) + { + Assert.Equal((uint)Av1Distribution.ProbabilityTop - forwardThresholds[index], distribution[index]); + } + } +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1SelfGuidedFilterTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1SelfGuidedFilterTests.cs new file mode 100644 index 0000000000..44b874bc0f --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1SelfGuidedFilterTests.cs @@ -0,0 +1,426 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.LoopRestoration; +using SixLabors.ImageSharp.Tests.TestUtilities; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1; + +/// +/// Verifies AV1 self-guided restoration against a direct-window definition across the supported hardware-intrinsic configurations. +/// +[Trait("Format", "Heif")] +public class Av1SelfGuidedFilterTests +{ + /// + /// The number of source samples required on every side of a filtered processing unit. + /// + private const int Border = 3; + + /// + /// The number of fractional bits retained by each self-guided filter result. + /// + private const int RestorationBits = 4; + + /// + /// The number of fractional bits used by the projection coefficients. + /// + private const int ProjectionBits = 7; + + /// + /// The hardware configurations required to exercise AVX2-assisted 256-bit, portable 256-bit, 128-bit, and scalar execution. + /// + private const HwIntrinsics Configurations = + HwIntrinsics.AllowAll | HwIntrinsics.DisableAVX2 | HwIntrinsics.DisableAVX | HwIntrinsics.DisableHWIntrinsic; + + /// + /// Gets the radii selected by each of the sixteen normative parameter sets. + /// + private static ReadOnlySpan ParameterRadii => + [ + 2, 1, 2, 1, 2, 1, 2, 1, + 2, 1, 2, 1, 2, 1, 2, 1, + 2, 1, 2, 1, 0, 1, 0, 1, + 0, 1, 0, 1, 2, 0, 2, 0, + ]; + + /// + /// Gets the variance scales selected by each of the sixteen normative parameter sets. + /// + private static ReadOnlySpan ParameterScales => + [ + 140, 3236, 112, 2158, 93, 1618, 80, 1438, + 70, 1295, 58, 1177, 47, 1079, 37, 996, + 30, 925, 25, 863, -1, 2589, -1, 1618, + -1, 1177, -1, 925, 56, -1, 22, -1, + ]; + + /// + /// Gets processing-unit dimensions covering narrow chroma units, odd frame edges, and both vector remainder widths. + /// + private static ReadOnlySpan ProcessingUnitDimensions => + [ + 1, 1, + 3, 5, + 7, 4, + 13, 9, + 29, 6, + ]; + + /// + /// Verifies every normative parameter set, sample precision, and processing-unit tail against the direct-window definition. + /// + [Fact] + public void FilterMatchesReference() + => FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidateFilters, Configurations); + + /// + /// Validates the complete self-guided parameter matrix in the active hardware-intrinsic configuration. + /// + private static void ValidateFilters() + { + foreach (int bitDepth in new[] { 8, 10, 12 }) + { + int maximumSample = (1 << bitDepth) - 1; + ReadOnlySpan processingUnitDimensions = ProcessingUnitDimensions; + for (int dimensionIndex = 0; dimensionIndex < processingUnitDimensions.Length; dimensionIndex += 2) + { + int width = processingUnitDimensions[dimensionIndex]; + int height = processingUnitDimensions[dimensionIndex + 1]; + int sourceStride = width + (Border * 2) + 5; + int destinationStride = width + 7; + ushort[] source = new ushort[sourceStride * (height + (Border * 2))]; + ushort[] expected = new ushort[destinationStride * height]; + ushort[] actual = new ushort[destinationStride * height]; + int[] scratch = new int[Av1SelfGuidedFilter.GetScratchLength(width, height)]; + int[] projectionCoefficients = new int[2]; + + FillSource(source, sourceStride, maximumSample); + + for (int parameterSetIndex = 0; parameterSetIndex < 16; parameterSetIndex++) + { + expected.AsSpan().Fill(ushort.MaxValue); + actual.AsSpan().Fill(ushort.MaxValue); + projectionCoefficients[0] = -96 + ((parameterSetIndex * 17) & 127); + projectionCoefficients[1] = -32 + ((parameterSetIndex * 29) & 127); + + FilterReference( + source, + sourceStride, + expected, + destinationStride, + width, + height, + bitDepth, + parameterSetIndex, + projectionCoefficients); + + Av1SelfGuidedFilter.FilterBlock( + source, + sourceStride, + actual, + destinationStride, + width, + height, + bitDepth, + parameterSetIndex, + projectionCoefficients, + scratch); + + AssertBlockEqual(expected, actual, destinationStride, width, height, bitDepth, parameterSetIndex); + } + } + } + } + + /// + /// Populates the bordered source with deterministic values spanning the selected sample precision. + /// + /// The complete bordered source storage. + /// The number of samples between source rows. + /// The largest encoded sample value. + private static void FillSource(Span source, int sourceStride, int maximumSample) + { + int rowCount = source.Length / sourceStride; + for (int row = 0; row < rowCount; row++) + { + for (int column = 0; column < sourceStride; column++) + { + int value = (row * 239) + (column * 101) + (row * column * 17) + (((row + column) & 3) * (maximumSample / 3)); + source[(row * sourceStride) + column] = (ushort)(value & maximumSample); + } + } + + // Exact endpoints make clipping and the full local-variance range observable without depending on random input. + source[0] = 0; + source[^1] = (ushort)maximumSample; + } + + /// + /// Applies the normative projection to direct-window self-guided results. + /// + /// The source rectangle beginning three samples above and left of the processing unit. + /// The number of samples between source rows. + /// The destination storage beginning at the restored processing-unit origin. + /// The number of samples between destination rows. + /// The processing-unit width. + /// The processing-unit height. + /// The encoded sample bit depth. + /// The self-guided parameter-set index. + /// The two transmitted projection coefficients. + private static void FilterReference( + ReadOnlySpan source, + int sourceStride, + Span destination, + int destinationStride, + int width, + int height, + int bitDepth, + int parameterSetIndex, + ReadOnlySpan projectionCoefficients) + { + int parameterOffset = parameterSetIndex * 2; + int radius0 = ParameterRadii[parameterOffset]; + int radius1 = ParameterRadii[parameterOffset + 1]; + int scale0 = ParameterScales[parameterOffset]; + int scale1 = ParameterScales[parameterOffset + 1]; + int projection0; + int projection1; + + if (radius0 == 0) + { + projection0 = 0; + projection1 = (1 << ProjectionBits) - projectionCoefficients[1]; + } + else if (radius1 == 0) + { + projection0 = projectionCoefficients[0]; + projection1 = 0; + } + else + { + projection0 = projectionCoefficients[0]; + projection1 = (1 << ProjectionBits) - projection0 - projectionCoefficients[1]; + } + + int maximumSample = (1 << bitDepth) - 1; + for (int row = 0; row < height; row++) + { + int sourceRowOffset = (row + Border) * sourceStride; + int destinationRowOffset = row * destinationStride; + for (int column = 0; column < width; column++) + { + int unfiltered = source[sourceRowOffset + column + Border] << RestorationBits; + int projected = unfiltered << ProjectionBits; + + if (radius0 > 0) + { + int filtered0 = CalculateFilteredSample(source, sourceStride, column, row, bitDepth, radius0, scale0); + projected += projection0 * (filtered0 - unfiltered); + } + + if (radius1 > 0) + { + int filtered1 = CalculateFilteredSample(source, sourceStride, column, row, bitDepth, radius1, scale1); + projected += projection1 * (filtered1 - unfiltered); + } + + destination[destinationRowOffset + column] = + (ushort)Math.Clamp(RoundPowerOfTwo(projected, ProjectionBits + RestorationBits), 0, maximumSample); + } + } + } + + /// + /// Calculates one fixed-point filtered sample directly from its local coefficient windows. + /// + /// The bordered source rectangle. + /// The number of samples between source rows. + /// The processing-unit column. + /// The processing-unit row. + /// The encoded sample bit depth. + /// The selected filter radius. + /// The selected variance scale. + /// The filtered sample with four fractional bits. + private static int CalculateFilteredSample( + ReadOnlySpan source, + int sourceStride, + int column, + int row, + int bitDepth, + int radius, + int scale) + { + int blendFactor = 0; + int localMean = 0; + int roundingBits; + + if (radius == 2 && (row & 1) == 0) + { + for (int coefficientRow = row - 1; coefficientRow <= row + 1; coefficientRow += 2) + { + for (int coefficientColumn = column - 1; coefficientColumn <= column + 1; coefficientColumn++) + { + int weight = coefficientColumn == column ? 6 : 5; + (int localBlendFactor, int localMeanValue) = + CalculateCoefficient(source, sourceStride, coefficientColumn, coefficientRow, bitDepth, radius, scale); + + blendFactor += weight * localBlendFactor; + localMean += weight * localMeanValue; + } + } + + roundingBits = 9; + } + else if (radius == 2) + { + for (int coefficientColumn = column - 1; coefficientColumn <= column + 1; coefficientColumn++) + { + int weight = coefficientColumn == column ? 6 : 5; + (int localBlendFactor, int localMeanValue) = + CalculateCoefficient(source, sourceStride, coefficientColumn, row, bitDepth, radius, scale); + + blendFactor += weight * localBlendFactor; + localMean += weight * localMeanValue; + } + + roundingBits = 8; + } + else + { + for (int coefficientRow = row - 1; coefficientRow <= row + 1; coefficientRow++) + { + for (int coefficientColumn = column - 1; coefficientColumn <= column + 1; coefficientColumn++) + { + int weight = coefficientRow == row || coefficientColumn == column ? 4 : 3; + (int localBlendFactor, int localMeanValue) = + CalculateCoefficient(source, sourceStride, coefficientColumn, coefficientRow, bitDepth, radius, scale); + + blendFactor += weight * localBlendFactor; + localMean += weight * localMeanValue; + } + } + + roundingBits = 9; + } + + int sample = source[((row + Border) * sourceStride) + column + Border]; + return RoundPowerOfTwo((blendFactor * sample) + localMean, roundingBits); + } + + /// + /// Calculates the blend factor and scaled local mean for one coefficient location by visiting every window sample. + /// + /// The bordered source rectangle. + /// The number of samples between source rows. + /// The coefficient column relative to the processing unit. + /// The coefficient row relative to the processing unit. + /// The encoded sample bit depth. + /// The square-window radius. + /// The variance scale. + /// The local blend factor and scaled mean. + private static (int BlendFactor, int LocalMean) CalculateCoefficient( + ReadOnlySpan source, + int sourceStride, + int column, + int row, + int bitDepth, + int radius, + int scale) + { + int centerX = column + Border; + int centerY = row + Border; + int sum = 0; + int squareSum = 0; + + for (int windowY = centerY - radius; windowY <= centerY + radius; windowY++) + { + int sourceRowOffset = windowY * sourceStride; + for (int windowX = centerX - radius; windowX <= centerX + radius; windowX++) + { + int sample = source[sourceRowOffset + windowX]; + sum += sample; + squareSum += sample * sample; + } + } + + int diameter = (radius * 2) + 1; + int windowArea = diameter * diameter; + int normalizedSquareSum = RoundPowerOfTwo(squareSum, 2 * (bitDepth - 8)); + int normalizedSum = RoundPowerOfTwo(sum, bitDepth - 8); + uint squareOfSum = (uint)normalizedSum * (uint)normalizedSum; + uint scaledSquareSum = (uint)normalizedSquareSum * (uint)windowArea; + uint variance = scaledSquareSum < squareOfSum ? 0 : scaledSquareSum - squareOfSum; + uint varianceIndex = Math.Min(RoundPowerOfTwo(variance * (uint)scale, 20), 255U); + + // The endpoint exceptions are part of the normative table. The middle values are the rounded x / (x + 1) ratio in Q8. + int blendFactor = varianceIndex switch + { + 0 => 1, + 255 => 256, + _ => (int)(((varianceIndex << 8) + ((varianceIndex + 1) >> 1)) / (varianceIndex + 1)), + }; + + uint reciprocal = radius == 1 ? 455U : 164U; + uint meanProduct = (uint)(256 - blendFactor) * (uint)sum * reciprocal; + int localMean = (int)RoundPowerOfTwo(meanProduct, 12); + return (blendFactor, localMean); + } + + /// + /// Verifies visible samples and confirms that the filter does not overwrite destination-row padding. + /// + /// The direct-window output. + /// The production output. + /// The number of samples between destination rows. + /// The processing-unit width. + /// The processing-unit height. + /// The encoded sample bit depth. + /// The self-guided parameter-set index. + private static void AssertBlockEqual( + ReadOnlySpan expected, + ReadOnlySpan actual, + int stride, + int width, + int height, + int bitDepth, + int parameterSetIndex) + { + for (int row = 0; row < height; row++) + { + int rowOffset = row * stride; + for (int column = 0; column < width; column++) + { + if (expected[rowOffset + column] != actual[rowOffset + column]) + { + Assert.Fail( + $"Self-guided parameter {parameterSetIndex}, {bitDepth}-bit block differs at ({column}, {row}): " + + $"expected {expected[rowOffset + column]}, actual {actual[rowOffset + column]}."); + } + } + + for (int column = width; column < stride; column++) + { + Assert.Equal(ushort.MaxValue, actual[rowOffset + column]); + } + } + } + + /// + /// Rounds a signed fixed-point value to the requested lower precision. + /// + /// The signed fixed-point value. + /// The number of low bits to discard. + /// The rounded signed value. + private static int RoundPowerOfTwo(int value, int bitCount) + => bitCount == 0 ? value : (value + (1 << (bitCount - 1))) >> bitCount; + + /// + /// Rounds an unsigned fixed-point value to the requested lower precision. + /// + /// The unsigned fixed-point value. + /// The number of low bits to discard. + /// The rounded unsigned value. + private static uint RoundPowerOfTwo(uint value, int bitCount) + => bitCount == 0 ? value : (value + (1U << (bitCount - 1))) >> bitCount; +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1SingleReferenceEntropyTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1SingleReferenceEntropyTests.cs new file mode 100644 index 0000000000..a5efee5fae --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1SingleReferenceEntropyTests.cs @@ -0,0 +1,407 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Buffers; +using SixLabors.ImageSharp.Formats.Heif.Av1; +using SixLabors.ImageSharp.Formats.Heif.Av1.Entropy; +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1; + +/// +/// Verifies the adaptive distributions and spatial contexts used to select an AV1 inter block's reference mode and frame. +/// +[Trait("Format", "Avif")] +public class Av1SingleReferenceEntropyTests +{ + /// + /// The quantizer index used to initialize an otherwise unrelated tile entropy encoder. + /// + private const int BaseQIndex = 128; + + /// + /// Verifies all eighteen normative single-reference distributions against the reference decoder's forward Q15 defaults. + /// + [Fact] + public void SingleReferenceDefaultsMatchReference() + { + uint[][] forwardThresholds = + [ + [4897, 1555, 4236, 8650, 904, 1444], + [16973, 16751, 19647, 24773, 11014, 15087], + [29744, 30279, 31194, 31895, 26875, 30304], + ]; + + Av1Distribution[][] distributions = Av1DefaultDistributions.SingleReference; + + Assert.Equal(forwardThresholds.Length, distributions.Length); + for (int context = 0; context < distributions.Length; context++) + { + Assert.Equal(forwardThresholds[context].Length, distributions[context].Length); + for (int decision = 0; decision < distributions[context].Length; decision++) + { + // Av1Distribution stores inverse cumulative thresholds. Convert each published forward default by the + // same Q15 complement used by production construction before comparing the exact value. + uint expected = (uint)Av1Distribution.ProbabilityTop - forwardThresholds[context][decision]; + + Assert.Equal(expected, distributions[context][decision][0]); + Assert.Equal(2, distributions[context][decision].NumberOfSymbols); + } + } + } + + /// + /// Verifies the five normative block reference-mode distributions against the reference decoder's forward Q15 defaults. + /// + [Fact] + public void CompInterDefaultsMatchReference() + { + uint[] forwardThresholds = [26828, 24035, 12031, 10640, 2901]; + Av1Distribution[] distributions = Av1DefaultDistributions.CompInter; + + Assert.Equal(forwardThresholds.Length, distributions.Length); + for (int context = 0; context < distributions.Length; context++) + { + uint expected = (uint)Av1Distribution.ProbabilityTop - forwardThresholds[context]; + + Assert.Equal(expected, distributions[context][0]); + Assert.Equal(2, distributions[context].NumberOfSymbols); + } + } + + /// + /// Verifies that every semantic reader selects its exact context row and single-reference tree column. + /// + /// The zero-based single-reference tree decision. + /// The neighboring reference-vote context. + [Theory] + [MemberData(nameof(GetReaderCases))] + public void SingleReferenceReadersUseRequestedDistribution(int decision, int context) + { + bool[] expected = [false, true, true, false, true, false, false, true]; + Av1Distribution writerDistribution = Av1DefaultDistributions.SingleReference[context][decision]; + using Av1SymbolWriter writer = new(Configuration.Default, 8, updateCdf: true); + + foreach (bool value in expected) + { + writer.WriteSymbol(value, writerDistribution); + } + + using IMemoryOwner encoded = writer.Exit(); + Av1SymbolDecoder decoder = new(Configuration.Default, encoded.Memory.Span, 0, updateCdf: true); + + foreach (bool value in expected) + { + Assert.Equal(value, ReadDecision(ref decoder, decision, context)); + } + } + + /// + /// Verifies that the reference-mode reader selects each of the five spatial-context distributions. + /// + /// The block reference-mode context. + [Theory] + [InlineData(0)] + [InlineData(1)] + [InlineData(2)] + [InlineData(3)] + [InlineData(4)] + public void ReferenceModeReaderUsesRequestedContext(int context) + { + bool[] expected = [false, true, true, false, true, false, false, true]; + Av1Distribution writerDistribution = Av1DefaultDistributions.CompInter[context]; + using Av1SymbolWriter writer = new(Configuration.Default, 8, updateCdf: true); + + foreach (bool value in expected) + { + writer.WriteSymbol(value, writerDistribution); + } + + using IMemoryOwner encoded = writer.Exit(); + Av1SymbolDecoder decoder = new(Configuration.Default, encoded.Memory.Span, 0, updateCdf: true); + + foreach (bool value in expected) + { + Assert.Equal(value, decoder.ReadIsCompoundReference(context)); + } + } + + /// + /// Verifies that the production writer emits every single-reference branch consumed by the decoder. + /// + /// The encoded reference-frame label. + [Theory] + [InlineData((int)Av1ReferenceFrameType.Last)] + [InlineData((int)Av1ReferenceFrameType.Last2)] + [InlineData((int)Av1ReferenceFrameType.Last3)] + [InlineData((int)Av1ReferenceFrameType.Golden)] + [InlineData((int)Av1ReferenceFrameType.Backward)] + [InlineData((int)Av1ReferenceFrameType.Alternate2)] + [InlineData((int)Av1ReferenceFrameType.Alternate)] + public void SingleReferenceWriterRoundTripsEveryReference(int referenceFrameValue) + { + Av1ReferenceFrameType referenceFrame = (Av1ReferenceFrameType)referenceFrameValue; + InlineArray8 referenceCountStorage = default; + Span referenceCounts = referenceCountStorage; + referenceCounts[(int)Av1ReferenceFrameType.Last] = 5; + referenceCounts[(int)Av1ReferenceFrameType.Last2] = 1; + referenceCounts[(int)Av1ReferenceFrameType.Last3] = 2; + referenceCounts[(int)Av1ReferenceFrameType.Golden] = 2; + referenceCounts[(int)Av1ReferenceFrameType.Backward] = 3; + referenceCounts[(int)Av1ReferenceFrameType.Alternate2] = 3; + referenceCounts[(int)Av1ReferenceFrameType.Alternate] = 6; + + using Av1SymbolEncoder encoder = new(Configuration.Default, 8, BaseQIndex, updateCdf: true); + encoder.WriteSingleReference(referenceFrame, referenceCounts); + + using IMemoryOwner encoded = encoder.Exit(); + Av1SymbolDecoder decoder = new(Configuration.Default, encoded.Memory.Span, BaseQIndex, updateCdf: true); + Av1ReferenceFrameType decodedReference = ReadSingleReference(ref decoder, referenceCounts); + + Assert.Equal(referenceFrame, decodedReference); + } + + /// + /// Verifies one-pass neighbor collection, compound-neighbor votes, clearing, and intra-neighbor exclusion. + /// + [Fact] + public void CollectNeighborReferenceCountsMatchesReference() + { + Av1BlockModeInfo above = CreateModeInfo(Av1ReferenceFrameType.Last, Av1ReferenceFrameType.None); + Av1BlockModeInfo left = CreateModeInfo(Av1ReferenceFrameType.Backward, Av1ReferenceFrameType.Alternate); + InlineArray8 referenceCountStorage = default; + Span referenceCounts = referenceCountStorage; + referenceCounts.Fill(7); + + Av1SymbolContextHelper.CollectNeighborReferenceCounts(above, left, referenceCounts); + + ReadOnlySpan expected = [0, 1, 0, 0, 0, 1, 0, 1]; + + for (int reference = 0; reference < referenceCounts.Length; reference++) + { + Assert.Equal(expected[reference], referenceCounts[reference]); + } + + Av1BlockModeInfo intra = CreateModeInfo(Av1ReferenceFrameType.Intra, Av1ReferenceFrameType.None); + Av1SymbolContextHelper.CollectNeighborReferenceCounts(intra, null, referenceCounts); + + for (int reference = 0; reference < referenceCounts.Length; reference++) + { + Assert.Equal((byte)0, referenceCounts[reference]); + } + } + + /// + /// Verifies that the six context functions aggregate the exact reference groups used by the reference decoder. + /// + [Fact] + public void ContextsAggregateReferenceGroups() + { + InlineArray8 referenceCountStorage = default; + Span referenceCounts = referenceCountStorage; + referenceCounts[(int)Av1ReferenceFrameType.Last] = 5; + referenceCounts[(int)Av1ReferenceFrameType.Last2] = 1; + referenceCounts[(int)Av1ReferenceFrameType.Last3] = 2; + referenceCounts[(int)Av1ReferenceFrameType.Golden] = 2; + referenceCounts[(int)Av1ReferenceFrameType.Backward] = 3; + referenceCounts[(int)Av1ReferenceFrameType.Alternate2] = 3; + referenceCounts[(int)Av1ReferenceFrameType.Alternate] = 6; + + Assert.Equal(0, Av1SymbolContextHelper.GetSingleReferenceBackwardContext(referenceCounts)); + Assert.Equal(1, Av1SymbolContextHelper.GetSingleReferenceAlternateContext(referenceCounts)); + Assert.Equal(2, Av1SymbolContextHelper.GetSingleReferenceLast3OrGoldenContext(referenceCounts)); + Assert.Equal(2, Av1SymbolContextHelper.GetSingleReferenceLast2Context(referenceCounts)); + Assert.Equal(1, Av1SymbolContextHelper.GetSingleReferenceGoldenContext(referenceCounts)); + Assert.Equal(1, Av1SymbolContextHelper.GetSingleReferenceAlternate2Context(referenceCounts)); + } + + /// + /// Verifies every branch of the reference decoder's five-state single-versus-compound reference-mode context. + /// + [Fact] + public void ReferenceModeContextMatchesReference() + { + Av1BlockModeInfo singleForward = CreateModeInfo(Av1ReferenceFrameType.Last, Av1ReferenceFrameType.None); + Av1BlockModeInfo singleBackward = CreateModeInfo(Av1ReferenceFrameType.Backward, Av1ReferenceFrameType.None); + Av1BlockModeInfo intra = CreateModeInfo(Av1ReferenceFrameType.Intra, Av1ReferenceFrameType.None); + Av1BlockModeInfo compound = CreateModeInfo(Av1ReferenceFrameType.Last, Av1ReferenceFrameType.Backward); + Av1BlockModeInfo secondCompound = CreateModeInfo(Av1ReferenceFrameType.Last2, Av1ReferenceFrameType.Alternate); + + Assert.Equal(1, Av1SymbolContextHelper.GetReferenceModeContext(null, null)); + Assert.Equal(0, Av1SymbolContextHelper.GetReferenceModeContext(singleForward, null)); + Assert.Equal(1, Av1SymbolContextHelper.GetReferenceModeContext(singleBackward, null)); + Assert.Equal(3, Av1SymbolContextHelper.GetReferenceModeContext(compound, null)); + Assert.Equal(0, Av1SymbolContextHelper.GetReferenceModeContext(singleForward, singleForward)); + Assert.Equal(1, Av1SymbolContextHelper.GetReferenceModeContext(singleForward, singleBackward)); + Assert.Equal(2, Av1SymbolContextHelper.GetReferenceModeContext(singleForward, compound)); + Assert.Equal(3, Av1SymbolContextHelper.GetReferenceModeContext(intra, compound)); + Assert.Equal(2, Av1SymbolContextHelper.GetReferenceModeContext(compound, singleForward)); + Assert.Equal(3, Av1SymbolContextHelper.GetReferenceModeContext(compound, singleBackward)); + Assert.Equal(4, Av1SymbolContextHelper.GetReferenceModeContext(compound, secondCompound)); + } + + /// + /// Verifies the tied, symbol-one-majority, and symbol-zero-majority context states. + /// + /// The votes for the forward branch represented by symbol zero. + /// The votes for the backward branch represented by symbol one. + /// The expected context. + [Theory] + [InlineData(1, 1, 1)] + [InlineData(1, 2, 0)] + [InlineData(2, 1, 2)] + public void SingleReferenceContextReflectsNeighborVoteBalance(byte forwardCount, byte backwardCount, int expected) + { + InlineArray8 referenceCountStorage = default; + Span referenceCounts = referenceCountStorage; + referenceCounts[(int)Av1ReferenceFrameType.Last] = forwardCount; + referenceCounts[(int)Av1ReferenceFrameType.Backward] = backwardCount; + + int actual = Av1SymbolContextHelper.GetSingleReferenceBackwardContext(referenceCounts); + + Assert.Equal(expected, actual); + } + + /// + /// Verifies that frame-context copies retain reference-selection adaptation without sharing mutable distributions. + /// + [Fact] + public void EntropyCopyRetainsReferenceSelectionState() + { + Av1FrameEntropyContext source = new(0); + Av1FrameEntropyContext destination = new(0); + source.SingleReference[2][5].Update(1); + source.CompInter[4].Update(1); + + destination.CopyFrom(source); + + Assert.Equal(source.SingleReference[2][5][0], destination.SingleReference[2][5][0]); + Assert.Equal(source.CompInter[4][0], destination.CompInter[4][0]); + + source.SingleReference[2][5].Update(0); + source.CompInter[4].Update(0); + + Assert.NotEqual(source.SingleReference[2][5][0], destination.SingleReference[2][5][0]); + Assert.NotEqual(source.CompInter[4][0], destination.CompInter[4][0]); + } + + /// + /// Verifies that publishing frame state resets the reference-selection distributions' update-rate history. + /// + [Fact] + public void EntropySnapshotResetsReferenceSelectionCounts() + { + const int updateCount = 20; + Av1FrameEntropyContext source = new(0); + Av1FrameEntropyContext snapshot = new(0); + + for (int i = 0; i < updateCount; i++) + { + source.SingleReference[1][3].Update(1); + source.CompInter[2].Update(1); + } + + source.SnapshotTo(snapshot); + + Assert.Equal(source.SingleReference[1][3][0], snapshot.SingleReference[1][3][0]); + Assert.Equal(source.CompInter[2][0], snapshot.CompInter[2][0]); + + // The source retains twenty observations while the snapshot restarts at zero. The same next symbol therefore + // moves identical thresholds by different amounts only when the new distribution participates in reset. + source.SingleReference[1][3].Update(0); + snapshot.SingleReference[1][3].Update(0); + source.CompInter[2].Update(0); + snapshot.CompInter[2].Update(0); + + Assert.NotEqual(source.SingleReference[1][3][0], snapshot.SingleReference[1][3][0]); + Assert.NotEqual(source.CompInter[2][0], snapshot.CompInter[2][0]); + } + + /// + /// Provides every context and decision pairing in the single-reference distribution matrix. + /// + /// The eighteen context and decision combinations. + public static TheoryData GetReaderCases() + { + TheoryData result = []; + + for (int decision = 0; decision < 6; decision++) + { + for (int context = 0; context < 3; context++) + { + result.Add(decision, context); + } + } + + return result; + } + + /// + /// Reads one semantic single-reference decision through its production entry point. + /// + /// The tile symbol decoder. + /// The zero-based single-reference tree decision. + /// The neighboring reference-vote context. + /// The decoded binary decision. + private static bool ReadDecision(ref Av1SymbolDecoder decoder, int decision, int context) + => decision switch + { + 0 => decoder.ReadSingleReferenceIsBackward(context), + 1 => decoder.ReadSingleReferenceIsAlternate(context), + 2 => decoder.ReadSingleReferenceIsLast3OrGolden(context), + 3 => decoder.ReadSingleReferenceIsLast2(context), + 4 => decoder.ReadSingleReferenceIsGolden(context), + _ => decoder.ReadSingleReferenceIsAlternate2(context), + }; + + /// + /// Reads one complete single-reference branch through the production semantic entry points. + /// + private static Av1ReferenceFrameType ReadSingleReference( + ref Av1SymbolDecoder decoder, + scoped ReadOnlySpan referenceCounts) + { + int context = Av1SymbolContextHelper.GetSingleReferenceBackwardContext(referenceCounts); + if (decoder.ReadSingleReferenceIsBackward(context)) + { + context = Av1SymbolContextHelper.GetSingleReferenceAlternateContext(referenceCounts); + if (decoder.ReadSingleReferenceIsAlternate(context)) + { + return Av1ReferenceFrameType.Alternate; + } + + context = Av1SymbolContextHelper.GetSingleReferenceAlternate2Context(referenceCounts); + return decoder.ReadSingleReferenceIsAlternate2(context) + ? Av1ReferenceFrameType.Alternate2 + : Av1ReferenceFrameType.Backward; + } + + context = Av1SymbolContextHelper.GetSingleReferenceLast3OrGoldenContext(referenceCounts); + if (decoder.ReadSingleReferenceIsLast3OrGolden(context)) + { + context = Av1SymbolContextHelper.GetSingleReferenceGoldenContext(referenceCounts); + return decoder.ReadSingleReferenceIsGolden(context) + ? Av1ReferenceFrameType.Golden + : Av1ReferenceFrameType.Last3; + } + + context = Av1SymbolContextHelper.GetSingleReferenceLast2Context(referenceCounts); + return decoder.ReadSingleReferenceIsLast2(context) + ? Av1ReferenceFrameType.Last2 + : Av1ReferenceFrameType.Last; + } + + /// + /// Creates decoded block-mode state with the requested primary and secondary reference labels. + /// + /// The primary reference label. + /// The optional secondary reference label. + /// The initialized block mode state. + private static Av1BlockModeInfo CreateModeInfo(Av1ReferenceFrameType primary, Av1ReferenceFrameType secondary) + { + Av1BlockModeInfo modeInfo = new(Av1BlockSize.Block4x4, Point.Empty); + modeInfo.ReferenceFrames[0] = primary; + modeInfo.ReferenceFrames[1] = secondary; + return modeInfo; + } +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1SymbolContextTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1SymbolContextTests.cs new file mode 100644 index 0000000000..4d2eaaef79 --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1SymbolContextTests.cs @@ -0,0 +1,175 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using Microsoft.Diagnostics.Symbols; +using SixLabors.ImageSharp.Formats.Heif.Av1; +using SixLabors.ImageSharp.Formats.Heif.Av1.Entropy; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1; + +[Trait("Format", "Avif")] +public class Av1SymbolContextTests +{ + [Theory] + [MemberData(nameof(GetLowLevelContextEndOfBlockData))] + public void TestLowLevelContextEndOfBlockAccuracy(int width, int height, int index) + { + // Arrange + Size size = new(width, height); + Av1LevelBuffer levels = new(Configuration.Default, size); + Point position = levels.GetPosition(index); + int blockWidthLog2 = Av1Math.Log2(width); + int expectedContext = GetExpectedLowerLevelContextEndOfBlock(blockWidthLog2, height, index); + + // Act + int actualContext = Av1SymbolContextHelper.GetLowerLevelContextEndOfBlock(levels, position); + + // Assert + Assert.Equal(expectedContext, actualContext); + } + + [Theory] + [MemberData(nameof(GetExtendedTransformIndicesData))] + public void RoundTripExtendedTransformIndices(int setType, int index) + { + // Arrange + Av1TransformSetType transformSetType = (Av1TransformSetType)setType; + + // Act + Av1TransformType transformType = Av1SymbolContextHelper.GetExtendedTransformType(transformSetType, index); + int actualIndex = Av1SymbolContextHelper.GetExtendedTransformIndex(transformSetType, transformType); + + // Assert + Assert.Equal(actualIndex, index); + } + + [Theory] + [InlineData((int)Av1PredictionMode.DC, (int)Av1TransformSize.Size8x8, false, (int)Av1TransformType.DctDct)] + [InlineData((int)Av1PredictionMode.Vertical, (int)Av1TransformSize.Size8x8, false, (int)Av1TransformType.AdstDct)] + [InlineData((int)Av1PredictionMode.Horizontal, (int)Av1TransformSize.Size8x8, false, (int)Av1TransformType.DctAdst)] + [InlineData((int)Av1PredictionMode.Directional45Degrees, (int)Av1TransformSize.Size8x8, false, (int)Av1TransformType.DctDct)] + [InlineData((int)Av1PredictionMode.Directional135Degrees, (int)Av1TransformSize.Size8x8, false, (int)Av1TransformType.AdstAdst)] + [InlineData((int)Av1PredictionMode.Directional113Degrees, (int)Av1TransformSize.Size8x8, false, (int)Av1TransformType.AdstDct)] + [InlineData((int)Av1PredictionMode.Directional157Degrees, (int)Av1TransformSize.Size8x8, false, (int)Av1TransformType.DctAdst)] + [InlineData((int)Av1PredictionMode.Directional203Degrees, (int)Av1TransformSize.Size8x8, false, (int)Av1TransformType.DctAdst)] + [InlineData((int)Av1PredictionMode.Directional67Degrees, (int)Av1TransformSize.Size8x8, false, (int)Av1TransformType.AdstDct)] + [InlineData((int)Av1PredictionMode.Smooth, (int)Av1TransformSize.Size8x8, false, (int)Av1TransformType.AdstAdst)] + [InlineData((int)Av1PredictionMode.SmoothVertical, (int)Av1TransformSize.Size8x8, false, (int)Av1TransformType.AdstDct)] + [InlineData((int)Av1PredictionMode.SmoothHorizontal, (int)Av1TransformSize.Size8x8, false, (int)Av1TransformType.DctAdst)] + [InlineData((int)Av1PredictionMode.Paeth, (int)Av1TransformSize.Size8x8, false, (int)Av1TransformType.AdstAdst)] + [InlineData((int)Av1PredictionMode.Directional135Degrees, (int)Av1TransformSize.Size8x8, true, (int)Av1TransformType.AdstAdst)] + [InlineData((int)Av1PredictionMode.Directional135Degrees, (int)Av1TransformSize.Size32x32, false, (int)Av1TransformType.DctDct)] + public void DefaultIntraTransformTypeMatchesCurrentLibaom( + int modeValue, + int transformSizeValue, + bool useReducedSet, + int expectedValue) + { + Av1TransformType actual = Av1SymbolContextHelper.GetDefaultIntraTransformType( + (Av1PredictionMode)modeValue, + (Av1TransformSize)transformSizeValue, + useReducedSet); + + Assert.Equal((Av1TransformType)expectedValue, actual); + } + + [Theory] + [InlineData(0, 0, 0)] + [InlineData(-1, 4, 12)] + [InlineData(1, 4, 20)] + public void CoefficientContextMatchesCurrentLibaom(int dcCoefficient, ushort endOfBlock, byte expected) + { + Span coefficients = stackalloc int[16]; + coefficients.Fill(1); + coefficients[0] = dcCoefficient; + + byte actual = Av1SymbolContextHelper.GetCoefficientContext( + coefficients, + Av1TransformSize.Size4x4, + Av1TransformType.DctDct, + endOfBlock); + + Assert.Equal(expected, actual); + } + + [Theory] + [InlineData(8, 8, (int)Av1BlockSize.Block8x8, (int)Av1TransformSize.Size8x8, 18)] + [InlineData(4, 8, (int)Av1BlockSize.Block8x8, (int)Av1TransformSize.Size8x8, 19)] + [InlineData(4, 4, (int)Av1BlockSize.Block8x8, (int)Av1TransformSize.Size8x8, 20)] + [InlineData(4, 4, (int)Av1BlockSize.Block16x16, (int)Av1TransformSize.Size8x8, 17)] + [InlineData(0, 0, (int)Av1BlockSize.Block8x8, (int)Av1TransformSize.Size4x4, 0)] + public void TransformPartitionContextMatchesCurrentLibaom( + byte aboveWidth, + byte leftHeight, + int blockSizeValue, + int transformSizeValue, + int expected) + { + int actual = Av1SymbolContextHelper.GetTransformPartitionContext( + aboveWidth, + leftHeight, + (Av1BlockSize)blockSizeValue, + (Av1TransformSize)transformSizeValue); + + Assert.Equal(expected, actual); + } + + public static TheoryData GetLowLevelContextEndOfBlockData() + { + TheoryData result = []; + for (int y = 1; y < 6; y++) + { + for (int x = 1; x < 6; x++) + { + int total = (1 << x) * (1 << y); + for (int i = 0; i < total; i++) + { + result.Add(1 << x, 1 << y, i); + } + } + } + + return result; + } + + public static TheoryData GetExtendedTransformIndicesData() + { + TheoryData result = []; + for (Av1TransformSetType setType = Av1TransformSetType.DctOnly; setType < Av1TransformSetType.AllSets; setType++) + { + int count = Av1SymbolContextHelper.GetExtendedTransformTypeCount(setType); + for (int index = 0; index < count; index++) + { + result.Add((int)setType, index); + } + } + + return result; + } + + /// + /// Computes the expected lower-level coefficient context at the end of a transform block. + /// + internal static int GetExpectedLowerLevelContextEndOfBlock(int blockWidthLog2, int height, int scanIndex) + { + if (scanIndex == 0) + { + return 0; + } + + if (scanIndex <= height << blockWidthLog2 >> 3) + { + return 1; + } + + if (scanIndex <= height << blockWidthLog2 >> 2) + { + return 2; + } + + return 3; + } +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1TemporalSegmentationTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1TemporalSegmentationTests.cs new file mode 100644 index 0000000000..c219738d5b --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1TemporalSegmentationTests.cs @@ -0,0 +1,318 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Buffers; +using SixLabors.ImageSharp.Formats.Heif.Av1; +using SixLabors.ImageSharp.Formats.Heif.Av1.Entropy; +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Formats.Heif.Av1.ReferenceFrames; +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; +using SixLabors.ImageSharp.Memory; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1; + +/// +/// Verifies AV1 temporal segment-map prediction against independent decoder results. +/// +[Trait("Format", "Avif")] +public class Av1TemporalSegmentationTests +{ + /// + /// Verifies temporal segment-map prediction symbols through each of AV1's three neighbor contexts. + /// + /// The sum of predicted above and left neighbors. + [Theory] + [InlineData(0)] + [InlineData(1)] + [InlineData(2)] + public void SegmentIdPredictedRoundTrips(int context) + { + bool[] expected = [false, true, true, false, true, false]; + using Av1SymbolWriter writer = new(Configuration.Default, 1, updateCdf: true); + Av1Distribution writerDistribution = Av1DefaultDistributions.SegmentIdPredicted[context]; + + foreach (bool value in expected) + { + writer.WriteSymbol(value, writerDistribution); + } + + using IMemoryOwner encoded = writer.Exit(); + Av1SymbolDecoder decoder = new(Configuration.Default, encoded.GetSpan(), 0, updateCdf: true); + + foreach (bool value in expected) + { + Assert.Equal(value, decoder.ReadSegmentIdPredicted(context)); + } + } + + /// + /// Verifies that the frame entropy graph copies adapted temporal-prediction state instead of restoring defaults. + /// + [Fact] + public void FrameEntropyCopyRetainsAdaptedSegmentPrediction() + { + Av1FrameEntropyContext source = new(0); + Av1FrameEntropyContext destination = new(0); + source.SegmentIdPredicted[2].Update(1); + + destination.CopyFrom(source); + + Assert.Equal(source.SegmentIdPredicted[2][0], destination.SegmentIdPredicted[2][0]); + Assert.NotEqual(16384U, destination.SegmentIdPredicted[2][0]); + } + + /// + /// Verifies that a spatial segment symbol cannot select an identifier above the frame's last active segment. + /// + [Fact] + public void SpatialSegmentIdOutsideActiveRangeIsRejected() + { + ObuSequenceHeader sequenceHeader = CreateSequenceHeader(64, 64); + ObuFrameHeader frameHeader = CreateFrameHeader(16, 16, segmentationUpdateMap: 1, segmentationTemporalUpdate: 0); + frameHeader.SegmentationParameters.LastActiveSegmentId = 0; + using Av1TileReader tileReader = new(Configuration.Default, sequenceHeader, frameHeader); + using Av1SymbolWriter writer = new(Configuration.Default, 1, updateCdf: true); + writer.WriteSymbol(Av1Constants.MaxSegmentCount - 1, Av1DefaultDistributions.SegmentId[0]); + using IMemoryOwner encoded = writer.Exit(); + + Assert.Throws( + () => + { + Av1BlockModeInfo modeInfo = new(Av1BlockSize.Block8x8, Point.Empty); + Av1SuperblockInfo superblockInfo = new(tileReader.FrameInfo, Point.Empty); + Av1PartitionInfo partitionInfo = new(modeInfo, superblockInfo, false, Av1PartitionType.None); + Av1SymbolDecoder decoder = new(Configuration.Default, encoded.GetSpan(), 0, updateCdf: true); + + tileReader.ReadInterSegmentId(ref decoder, ref partitionInfo, beforeSkip: false); + }); + } + + /// + /// Verifies that only neighboring blocks which selected temporal prediction contribute to the binary CDF context. + /// + /// Whether an above block is available. + /// Whether the available above block selected temporal prediction. + /// Whether a left block is available. + /// Whether the available left block selected temporal prediction. + /// The expected context in the inclusive range zero through two. + [Theory] + [InlineData(false, false, false, false, 0)] + [InlineData(true, false, true, false, 0)] + [InlineData(true, true, false, false, 1)] + [InlineData(false, false, true, true, 1)] + [InlineData(true, true, true, true, 2)] + public void SegmentPredictionContextCountsPredictedNeighbors( + bool hasAbove, + bool abovePredicted, + bool hasLeft, + bool leftPredicted, + int expected) + { + Av1BlockModeInfo? aboveModeInfo = hasAbove ? CreateModeInfo(abovePredicted) : null; + Av1BlockModeInfo? leftModeInfo = hasLeft ? CreateModeInfo(leftPredicted) : null; + + int actual = Av1SymbolContextHelper.GetSegmentIdPredictedContext(aboveModeInfo, leftModeInfo); + + Assert.Equal(expected, actual); + } + + /// + /// Verifies that a temporal-prediction symbol selects the minimum retained segment across the complete block and writes it to the current map. + /// + /// Whether segment syntax precedes the residual-skip flag. + [Theory] + [InlineData(false)] + [InlineData(true)] + public void ReadInterSegmentIdUsesRetainedPrimaryMap(bool segmentIdPrecedesSkip) + { + const int modeInfoSize = 16; + ObuSequenceHeader sequenceHeader = CreateSequenceHeader(64, 64); + ObuFrameHeader primaryHeader = CreateFrameHeader(modeInfoSize, modeInfoSize, segmentationUpdateMap: 1, segmentationTemporalUpdate: 0); + using Av1FrameInfo primaryFrameInfo = new(sequenceHeader); + primaryFrameInfo.InitializeSegmentIds(primaryHeader, null); + primaryFrameInfo.SetSegmentId(Av1BlockSize.Block64x64, Point.Empty, 6); + + // The target 16x16 block covers sixteen 4x4 cells. One lower retained value proves that prediction scans the + // complete clipped coverage rather than reading only the block origin. + Point lowSegmentPosition = new(4, 4); + primaryFrameInfo.SetSegmentId(Av1BlockSize.Block4x4, lowSegmentPosition, 2); + + // The production reference store owns complete reconstructed frames. A minimal monochrome frame buffer keeps + // this test on the real ownership path while the assertions remain confined to retained segmentation state. + Av1FrameBuffer primaryBuffer = new(Configuration.Default, sequenceHeader, Av1ColorFormat.Yuv400, false); + Av1ReferenceFrame primaryFrame = new(primaryBuffer, primaryHeader, primaryFrameInfo); + using Av1ReferenceFrameStore referenceFrames = new(); + referenceFrames.Commit(1, primaryFrame, showFrame: false); + + ObuFrameHeader currentHeader = CreateFrameHeader(modeInfoSize, modeInfoSize, segmentationUpdateMap: 1, segmentationTemporalUpdate: 1); + currentHeader.FrameType = ObuFrameType.InterFrame; + currentHeader.PrimaryReferenceFrame = 0; + currentHeader.PrimaryReferenceSlot = 0; + currentHeader.SegmentationParameters.SegmentIdPrecedesSkip = segmentIdPrecedesSkip; + Av1FrameEntropyContexts entropyContexts = new(0); + using Av1TileReader tileReader = new(Configuration.Default, sequenceHeader, currentHeader, entropyContexts, null, referenceFrames); + + Point blockPosition = new(2, 2); + Av1BlockModeInfo modeInfo = new(Av1BlockSize.Block16x16, blockPosition); + Av1SuperblockInfo superblockInfo = new(tileReader.FrameInfo, Point.Empty); + Av1PartitionInfo partitionInfo = new(modeInfo, superblockInfo, false, Av1PartitionType.None) + { + ColumnIndex = blockPosition.X, + RowIndex = blockPosition.Y, + AvailableAbove = true, + AvailableLeft = true, + AboveModeInfo = CreateModeInfo(predicted: true), + LeftModeInfo = CreateModeInfo(predicted: false) + }; + + const int predictionContext = 1; + using Av1SymbolWriter writer = new(Configuration.Default, 1, updateCdf: true); + writer.WriteSymbol(true, Av1DefaultDistributions.SegmentIdPredicted[predictionContext]); + using IMemoryOwner encoded = writer.Exit(); + Av1SymbolDecoder decoder = new(Configuration.Default, encoded.GetSpan(), 0, updateCdf: true); + + tileReader.ReadInterSegmentId(ref decoder, ref partitionInfo, beforeSkip: segmentIdPrecedesSkip); + modeInfo = partitionInfo.ModeInfo; + + Assert.True(modeInfo.SegmentIdPredicted); + Assert.Equal(2, modeInfo.SegmentId); + for (int row = blockPosition.Y; row < blockPosition.Y + modeInfo.BlockSize.Get4x4HighCount(); row++) + { + for (int column = blockPosition.X; column < blockPosition.X + modeInfo.BlockSize.Get4x4WideCount(); column++) + { + Assert.Equal(2, tileReader.FrameInfo.GetSegmentId(row, column)); + } + } + } + + /// + /// Verifies that a skipped inter block uses the spatial predictor without reading a temporal-prediction symbol. + /// + [Fact] + public void SkippedInterUsesSpatialSegment() + { + ObuSequenceHeader sequenceHeader = CreateSequenceHeader(64, 64); + ObuFrameHeader frameHeader = CreateFrameHeader(16, 16, segmentationUpdateMap: 1, segmentationTemporalUpdate: 1); + using Av1TileReader tileReader = new(Configuration.Default, sequenceHeader, frameHeader); + Point blockPosition = new(2, 2); + + // Three equal spatial neighbors select segment three without consuming a spatial segment symbol. The block is + // initialized as predicted to prove that the normative skipped-block branch explicitly clears the stale flag. + tileReader.FrameInfo.SetSegmentId(Av1BlockSize.Block4x4, new Point(1, 1), 3); + tileReader.FrameInfo.SetSegmentId(Av1BlockSize.Block4x4, new Point(2, 1), 3); + tileReader.FrameInfo.SetSegmentId(Av1BlockSize.Block4x4, new Point(1, 2), 3); + Av1BlockModeInfo modeInfo = new(Av1BlockSize.Block8x8, blockPosition) + { + Skip = true, + SegmentIdPredicted = true + }; + + Av1SuperblockInfo superblockInfo = new(tileReader.FrameInfo, Point.Empty); + Av1PartitionInfo partitionInfo = new(modeInfo, superblockInfo, false, Av1PartitionType.None) + { + ColumnIndex = blockPosition.X, + RowIndex = blockPosition.Y, + AvailableAbove = true, + AvailableLeft = true + }; + + using Av1SymbolWriter writer = new(Configuration.Default, 1, updateCdf: true); + using IMemoryOwner encoded = writer.Exit(); + Av1SymbolDecoder decoder = new(Configuration.Default, encoded.GetSpan(), 0, updateCdf: true); + + tileReader.ReadInterSegmentId(ref decoder, ref partitionInfo, beforeSkip: false); + modeInfo = partitionInfo.ModeInfo; + + Assert.False(modeInfo.SegmentIdPredicted); + Assert.Equal(3, modeInfo.SegmentId); + Assert.Equal(3, tileReader.FrameInfo.GetSegmentId(blockPosition.Y, blockPosition.X)); + } + + /// + /// Verifies that retained segmentation maps with different mode-info geometry are unavailable for temporal prediction. + /// + [Fact] + public void PredictedSegmentIdIsZeroForMismatchedPrimaryGeometry() + { + ObuSequenceHeader sequenceHeader = CreateSequenceHeader(64, 64); + ObuFrameHeader currentHeader = CreateFrameHeader(16, 16, segmentationUpdateMap: 1, segmentationTemporalUpdate: 1); + ObuFrameHeader primaryHeader = CreateFrameHeader(8, 16, segmentationUpdateMap: 1, segmentationTemporalUpdate: 0); + using Av1FrameInfo currentFrameInfo = new(sequenceHeader); + using Av1FrameInfo primaryFrameInfo = new(sequenceHeader); + currentFrameInfo.InitializeSegmentIds(currentHeader, null); + primaryFrameInfo.InitializeSegmentIds(primaryHeader, null); + primaryFrameInfo.SetSegmentId(Av1BlockSize.Block32x64, Point.Empty, 5); + + Av1FrameBuffer primaryBuffer = + new(Configuration.Default, sequenceHeader, Av1ColorFormat.Yuv400, false); + + using Av1ReferenceFrame primaryFrame = new(primaryBuffer, primaryHeader, primaryFrameInfo); + int actual = currentFrameInfo.GetPredictedSegmentId( + primaryFrame.ReferenceState, + Av1BlockSize.Block16x16, + Point.Empty); + + Assert.Equal(0, actual); + } + + /// + /// Creates block mode state with the requested temporal segment-prediction flag. + /// + /// Whether the block selected its segment identifier from the retained map. + /// The initialized block mode state. + private static Av1BlockModeInfo CreateModeInfo(bool predicted) + => new(Av1BlockSize.Block4x4, Point.Empty) { SegmentIdPredicted = predicted }; + + /// + /// Creates the fixed 64x64-superblock sequence geometry used by segmentation-map tests. + /// + /// The maximum coded width in pixels. + /// The maximum coded height in pixels. + /// The initialized monochrome sequence header. + private static ObuSequenceHeader CreateSequenceHeader(int width, int height) + => new() + { + MaxFrameWidth = width, + MaxFrameHeight = height, + Use128x128Superblock = false, + ColorConfig = new ObuColorConfig + { + IsMonochrome = true, + BitDepth = Av1BitDepth.EightBit + } + }; + + /// + /// Creates the frame geometry and segmentation controls used by direct map tests. + /// + /// The active width in 4x4 mode-info units. + /// The active height in 4x4 mode-info units. + /// Whether the frame updates its segment map. + /// Whether map updates may select the retained primary map. + /// The initialized frame header. + private static ObuFrameHeader CreateFrameHeader( + int modeInfoColumnCount, + int modeInfoRowCount, + int segmentationUpdateMap, + int segmentationTemporalUpdate) + => new() + { + ModeInfoColumnCount = modeInfoColumnCount, + ModeInfoRowCount = modeInfoRowCount, + FrameSize = new ObuFrameSize + { + FrameWidth = modeInfoColumnCount << Av1Constants.ModeInfoSizeLog2, + FrameHeight = modeInfoRowCount << Av1Constants.ModeInfoSizeLog2, + SuperResolutionUpscaledWidth = modeInfoColumnCount << Av1Constants.ModeInfoSizeLog2, + RenderWidth = modeInfoColumnCount << Av1Constants.ModeInfoSizeLog2, + RenderHeight = modeInfoRowCount << Av1Constants.ModeInfoSizeLog2 + }, + SegmentationParameters = new ObuSegmentationParameters + { + Enabled = true, + LastActiveSegmentId = Av1Constants.MaxSegmentCount - 1, + SegmentationUpdateMap = segmentationUpdateMap, + SegmentationTemporalUpdate = segmentationTemporalUpdate + } + }; +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1TileDecoderStub.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1TileDecoderStub.cs new file mode 100644 index 0000000000..b96216b2ef --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1TileDecoderStub.cs @@ -0,0 +1,22 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1; + +internal class Av1TileDecoderStub : IAv1TileReader, IAv1TileWriter +{ + private readonly Dictionary tileDatas = []; + + public void ReadTile(Span tileData, int tileNum) + => this.tileDatas.Add(tileNum, tileData.ToArray()); + + /// + public void CompleteFrame() + { + } + + public ReadOnlySpan GetTileData(int tileNum) + => this.tileDatas[tileNum]; +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1TilingTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1TilingTests.cs new file mode 100644 index 0000000000..61f9aa57e2 --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1TilingTests.cs @@ -0,0 +1,439 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Buffers; +using SixLabors.ImageSharp.Formats.Heif.Av1; +using SixLabors.ImageSharp.Formats.Heif.Av1.Entropy; +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; +using SixLabors.ImageSharp.Formats.Heif.Av1.ReferenceFrames; +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; +using SixLabors.ImageSharp.Memory; +using SixLabors.ImageSharp.PixelFormats; +using SixLabors.ImageSharp.Tests.Memory; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1; + +[Trait("Format", "Avif")] +public class Av1TilingTests +{ + [Theory] + [InlineData(false, false)] + [InlineData(false, true)] + [InlineData(true, false)] + [InlineData(true, true)] + public void ConstructionFailureReturnsEveryAllocation(bool use128x128Superblock, bool monochrome) + { + ObuSequenceHeader sequenceHeader = new() + { + MaxFrameWidth = 128, + MaxFrameHeight = 128, + Use128x128Superblock = use128x128Superblock, + ColorConfig = new ObuColorConfig + { + BitDepth = Av1BitDepth.EightBit, + IsMonochrome = monochrome, + SubSamplingX = true, + SubSamplingY = true + } + }; + ObuFrameHeader frameHeader = new() + { + FrameSize = new ObuFrameSize + { + FrameWidth = 128, + FrameHeight = 128, + SuperResolutionUpscaledWidth = 128, + RenderWidth = 128, + RenderHeight = 128 + }, + ModeInfoColumnCount = 32, + ModeInfoRowCount = 32, + ModeInfoStride = 32 + }; + + Configuration configuration = Configuration.Default.Clone(); + TestMemoryAllocator successfulAllocator = new(); + successfulAllocator.EnableNonThreadSafeLogging(); + configuration.MemoryAllocator = successfulAllocator; + using (Av1TileReader reader = new(configuration, sequenceHeader, frameHeader)) + { + Assert.NotEmpty(successfulAllocator.AllocationLog); + } + + Assert.Equal(successfulAllocator.AllocationLog.Count, successfulAllocator.ReturnLog.Count); + for (int failureIndex = 0; failureIndex < successfulAllocator.AllocationLog.Count; failureIndex++) + { + FailingTileAllocator allocator = new(failureIndex); + configuration.MemoryAllocator = allocator; + + // Fail each actual rent, including nested frame-state and neighbor-context constructors. No reader + // reaches the caller's using statement on failure, so construction must return every completed owner. + InvalidMemoryOperationException exception = Assert.Throws(() => + { + using Av1TileReader reader = new(configuration, sequenceHeader, frameHeader); + }); + + Assert.Equal("Tile allocation failure.", exception.Message); + Assert.Equal(failureIndex, allocator.AllocationLog.Count); + Assert.All( + allocator.AllocationLog, + allocation => Assert.Single(allocator.ReturnLog, returned => returned.AllocationId == allocation.AllocationId)); + + Assert.Equal(allocator.AllocationLog.Count, allocator.ReturnLog.Count); + } + } + + /// + /// Verifies that frame mode-information indices do not wrap at the unsigned 16-bit boundary. + /// + [Fact] + public void ModeInfoMapSupportsMoreThanUShortMaxBlocks() + { + const int blockCount = ushort.MaxValue + 2; + Av1FrameInfo.Av1FrameModeInfoMap map = new(new Size(blockCount, 1)); + for (int index = 0; index < blockCount; index++) + { + map.Update(new Point(index, 0), Av1BlockSize.Block4x4); + } + + Assert.Equal(blockCount, map.NextIndex); + Assert.Equal(blockCount - 1, map[new Point(blockCount - 1, 0)]); + } + + /// + /// Verifies the decoded block geometry and prediction modes against the reference decoder inspection output for a real AVIF image item. + /// + [Fact] + public void ParsedAvifModeMapMatchesReference() + { + string filePath = Path.Combine(TestEnvironment.InputImagesDirectoryFullPath, TestImages.Heif.ParisIccExifXmpAvif); + byte[] content = File.ReadAllBytes(filePath); + + // The fixture's iloc box identifies item 1 as the AV1 payload at offset 0x17A8 with length 0x3AE4. + const int codedItemOffset = 0x17A8; + const int codedItemLength = 0x3AE4; + using Av1Decoder decoder = new(Configuration.Default); + using Image image = decoder.Decode(content.AsSpan(codedItemOffset, codedItemLength)); + Av1FrameInfo frameInfo = Assert.IsType(decoder.FrameInfo); + ObuFrameHeader frameHeader = Assert.IsType(decoder.FrameHeader); + Span blockSizeCounts = stackalloc int[(int)Av1BlockSize.AllSizes]; + Span modeCounts = stackalloc int[(int)Av1PredictionMode.IntraModes]; + + for (int row = 0; row < frameHeader.ModeInfoRowCount; row++) + { + for (int column = 0; column < frameHeader.ModeInfoColumnCount; column++) + { + Av1BlockModeInfo modeInfo = frameInfo.GetModeInfoAt(new Point(column, row)); + blockSizeCounts[(int)modeInfo.BlockSize]++; + modeCounts[(int)modeInfo.YMode]++; + } + } + + // These counts come from independently inspected 102 by 76 mode-info maps. + int[] expectedBlockSizeCounts = new int[(int)Av1BlockSize.AllSizes]; + expectedBlockSizeCounts[(int)Av1BlockSize.Block8x8] = 3176; + expectedBlockSizeCounts[(int)Av1BlockSize.Block8x16] = 48; + expectedBlockSizeCounts[(int)Av1BlockSize.Block16x16] = 4080; + expectedBlockSizeCounts[(int)Av1BlockSize.Block32x32] = 448; + + int[] expectedModeCounts = new int[(int)Av1PredictionMode.IntraModes]; + expectedModeCounts[(int)Av1PredictionMode.DC] = 3020; + expectedModeCounts[(int)Av1PredictionMode.Vertical] = 228; + expectedModeCounts[(int)Av1PredictionMode.Horizontal] = 2360; + expectedModeCounts[(int)Av1PredictionMode.Smooth] = 2144; + + Assert.Equal(expectedBlockSizeCounts, blockSizeCounts.ToArray()); + Assert.Equal(expectedModeCounts, modeCounts.ToArray()); + } + + [Fact] + public void DecoderReadsFirstTile() + { + string filePath = Path.Combine(TestEnvironment.InputImagesDirectoryFullPath, TestImages.Heif.Orange4x4); + byte[] content = File.ReadAllBytes(filePath); + Av1Decoder decoder = new(Configuration.Default); + + using Image image = decoder.Decode(content.AsSpan(0x010E, 0x001D)); + + Assert.Equal(4, image.Width); + Assert.Equal(4, image.Height); + Assert.True(image.Frames.RootFrame.PixelBuffer.DangerousGetSingleSpan().ContainsAnyExcept(default(Rgba32))); + } + + /// + /// Verifies that partition syntax cannot produce a luma block with no valid 4:2:0 chroma representation. + /// + [Fact] + public void RejectsPartitionThatCannotRepresentSubsampledChroma() + { + ObuSequenceHeader sequenceHeader = new() + { + MaxFrameWidth = 64, + MaxFrameHeight = 64, + Use128x128Superblock = false, + ColorConfig = new ObuColorConfig + { + BitDepth = Av1BitDepth.EightBit, + SubSamplingX = true, + SubSamplingY = true + } + }; + ObuTileGroupHeader tileInfo = new() + { + TileColumnCount = 1, + TileRowCount = 1 + }; + tileInfo.TileColumnStartModeInfo[1] = sequenceHeader.SuperblockModeInfoSize; + tileInfo.TileRowStartModeInfo[1] = sequenceHeader.SuperblockModeInfoSize; + ObuFrameHeader frameHeader = new() + { + FrameSize = new ObuFrameSize + { + FrameWidth = 64, + FrameHeight = 64, + SuperResolutionUpscaledWidth = 64, + RenderWidth = 64, + RenderHeight = 64 + }, + ModeInfoColumnCount = sequenceHeader.SuperblockModeInfoSize, + ModeInfoRowCount = sequenceHeader.SuperblockModeInfoSize, + ModeInfoStride = sequenceHeader.SuperblockModeInfoSize, + TilesInfo = tileInfo, + DisableCdfUpdate = true, + DisableFrameEndUpdateCdf = true + }; + + using Av1SymbolWriter writer = new(Configuration.Default, 1, updateCdf: false); + Av1Distribution[] partitionTypes = Av1DefaultDistributions.PartitionTypes; + Av1BlockSize blockSize = sequenceHeader.SuperblockSize; + while (blockSize > Av1BlockSize.Block8x8) + { + int blockSizeLog = blockSize.Get4x4WidthLog2() - Av1BlockSize.Block8x8.Get4x4WidthLog2(); + int context = blockSizeLog * Av1Constants.PartitionProbabilitySet; + writer.WriteSymbol((int)Av1PartitionType.Split, partitionTypes[context]); + blockSize = Av1PartitionType.Split.GetBlockSubSize(blockSize); + } + + writer.WriteSymbol((int)Av1PartitionType.Horizontal, partitionTypes[0]); + using IMemoryOwner encoded = writer.Exit(); + using Av1TileReader tileReader = new(Configuration.Default, sequenceHeader, frameHeader); + + Assert.Throws(() => tileReader.ReadTile(encoded.GetSpan(), 0)); + } + + [Theory] + [InlineData(TestImages.Heif.XnConvert, 0x010E, 0x03CC, 18, 16)] + [InlineData(TestImages.Heif.Orange4x4, 0x010E, 0x001d, 21, 1)] + public void DecodePixelsFirstTile(string filename, int dataOffset, int dataSize, int tileOffset, int superblockCount) + { + // Assign + string filePath = Path.Combine(TestEnvironment.InputImagesDirectoryFullPath, filename); + byte[] content = File.ReadAllBytes(filePath); + Span headerSpan = content.AsSpan(dataOffset, dataSize); + Span tileSpan = content.AsSpan(dataOffset + tileOffset, dataSize - tileOffset); + Av1BitStreamReader bitStreamReader = new(headerSpan); + IAv1TileReader stub = new Av1TileDecoderStub(); + ObuReader obuReader = new(); + obuReader.ReadAll(ref bitStreamReader, dataSize, () => stub); + using Av1ReferenceFrameStore referenceFrames = new(); + using Av1FrameBuffer frameBuffer = new( + Configuration.Default, + obuReader.SequenceHeader, + Av1ColorFormat.Yuv444, + false); + + using Av1FrameInfo frameInfo = new(obuReader.SequenceHeader); + using Av1FrameDecoder frameDecoder = new( + obuReader.SequenceHeader, + obuReader.FrameHeader, + frameInfo, + frameBuffer, + referenceFrames); + + using Av1TileReader tileReader = new( + Configuration.Default, + obuReader.SequenceHeader, + obuReader.FrameHeader, + frameDecoder); + + // Act + tileReader.ReadTile(tileSpan, 0); + + // Assert + Assert.Equal(dataSize * 8, bitStreamReader.BitPosition); + Assert.False(frameBuffer.BufferY.Size.IsEmpty); + Assert.True(frameBuffer.BufferY.DangerousGetSingleSpan().ContainsAnyExcept(0)); + } + + [Theory] + [InlineData((int)Av1BitDepth.TenBit, 1023)] + [InlineData((int)Av1BitDepth.TwelveBit, 4095)] + public void DecodePixelsFirstTileThroughHighBitDepthPipeline(int bitDepthIndex, ushort maximum) + { + string filePath = Path.Combine(TestEnvironment.InputImagesDirectoryFullPath, TestImages.Heif.Orange4x4); + byte[] content = File.ReadAllBytes(filePath); + const int dataOffset = 0x010E; + const int dataSize = 0x001D; + const int tileOffset = 21; + Span headerSpan = content.AsSpan(dataOffset, dataSize); + Span tileSpan = content.AsSpan(dataOffset + tileOffset, dataSize - tileOffset); + Av1BitStreamReader bitStreamReader = new(headerSpan); + IAv1TileReader stub = new Av1TileDecoderStub(); + ObuReader obuReader = new(); + obuReader.ReadAll(ref bitStreamReader, dataSize, () => stub); + + // Reuse known-good tile syntax after parsing so this test isolates native high-bit prediction and reconstruction wiring. + obuReader.SequenceHeader.ColorConfig.BitDepth = (Av1BitDepth)bitDepthIndex; + using Av1ReferenceFrameStore referenceFrames = new(); + using Av1FrameBuffer frameBuffer = new(Configuration.Default, obuReader.SequenceHeader, Av1ColorFormat.Yuv444, false); + using Av1FrameInfo frameInfo = new(obuReader.SequenceHeader); + using Av1FrameDecoder frameDecoder = new( + obuReader.SequenceHeader, + obuReader.FrameHeader, + frameInfo, + frameBuffer, + referenceFrames); + + using Av1TileReader tileReader = new( + Configuration.Default, + obuReader.SequenceHeader, + obuReader.FrameHeader, + frameDecoder); + + tileReader.ReadTile(tileSpan, 0); + + Span yRow = frameBuffer.GetHighBitDepthRowSpan(Av1Plane.Y, 0, 0, 0); + Assert.True(yRow[..4].ContainsAnyExcept(0)); + Assert.All(yRow[..4].ToArray(), value => Assert.InRange(value, (ushort)0, maximum)); + } + + [Theory] + [InlineData(TestImages.Heif.XnConvert, 0x010E, 0x03CC, 18, 16)] + [InlineData(TestImages.Heif.Orange4x4, 0x010E, 0x001d, 21, 1)] + public void DecodePartitionsFirstTile(string filename, int dataOffset, int dataSize, int tileOffset, int superblockCount) + { + // Assign + string filePath = Path.Combine(TestEnvironment.InputImagesDirectoryFullPath, filename); + byte[] content = File.ReadAllBytes(filePath); + Span headerSpan = content.AsSpan(dataOffset, dataSize); + Span tileSpan = content.AsSpan(dataOffset + tileOffset, dataSize - tileOffset); + Av1BitStreamReader bitStreamReader = new(headerSpan); + IAv1TileReader stub = new Av1TileDecoderStub(); + ObuReader obuReader = new(); + obuReader.ReadAll(ref bitStreamReader, dataSize, () => stub); + Av1FrameDecoderStub frameDecoder = new(); + using Av1TileReader tileReader = new( + Configuration.Default, + obuReader.SequenceHeader, + obuReader.FrameHeader, + frameDecoder); + + // Act + tileReader.ReadTile(tileSpan, 0); + + // Assert + Assert.Equal(dataSize * 8, bitStreamReader.BitPosition); + Assert.Equal(superblockCount, frameDecoder.SuperblockCount); + } + + [Fact] + public void ParsedSuperblocksExposeEveryModeInfoInBitstreamOrder() + { + string filePath = Path.Combine(TestEnvironment.InputImagesDirectoryFullPath, TestImages.Heif.XnConvert); + byte[] content = File.ReadAllBytes(filePath); + const int dataOffset = 0x010E; + const int dataSize = 0x03CC; + const int tileOffset = 18; + Span headerSpan = content.AsSpan(dataOffset, dataSize); + Span tileSpan = content.AsSpan(dataOffset + tileOffset, dataSize - tileOffset); + Av1BitStreamReader bitStreamReader = new(headerSpan); + IAv1TileReader stub = new Av1TileDecoderStub(); + ObuReader obuReader = new(); + obuReader.ReadAll(ref bitStreamReader, dataSize, () => stub); + using Av1TileReader tileReader = new(Configuration.Default, obuReader.SequenceHeader, obuReader.FrameHeader); + + tileReader.ReadTile(tileSpan, 0); + + int parsedModeInfoCount = 0; + int superblockSize = obuReader.SequenceHeader.SuperblockModeInfoSize; + for (int row = 0; row < obuReader.FrameHeader.ModeInfoRowCount; row += superblockSize) + { + for (int column = 0; column < obuReader.FrameHeader.ModeInfoColumnCount; column += superblockSize) + { + Point superblockPosition = new(column / superblockSize, row / superblockSize); + Av1SuperblockInfo superblockInfo = tileReader.FrameInfo.GetSuperblock(superblockPosition); + Av1FrameInfo.ModeInfoCollection modeInfos = superblockInfo.GetModeInfos(); + + Assert.Equal(superblockInfo.BlockCount, modeInfos.Length); + Assert.Equal(modeInfos[0].ModeInfoIndex, tileReader.FrameInfo.GetModeInfo(superblockPosition).ModeInfoIndex); + + foreach (Av1BlockModeInfo modeInfo in modeInfos) + { + Point modeInfoPosition = new( + superblockInfo.ModeInfoPosition.X + modeInfo.PositionInSuperblock.X, + superblockInfo.ModeInfoPosition.Y + modeInfo.PositionInSuperblock.Y); + + for (int y = 0; y < modeInfo.BlockSize.Get4x4HighCount(); y++) + { + for (int x = 0; x < modeInfo.BlockSize.Get4x4WideCount(); x++) + { + Assert.Equal( + modeInfo.ModeInfoIndex, + tileReader.FrameInfo.GetModeInfoAt(new Point(modeInfoPosition.X + x, modeInfoPosition.Y + y)).ModeInfoIndex); + } + } + } + + parsedModeInfoCount += modeInfos.Length; + } + } + + Assert.True(parsedModeInfoCount > 16); + } + + [Theory] + [InlineData(TestImages.Heif.XnConvert, 0x010E, 0x03CC, 18, 16)] + [InlineData(TestImages.Heif.Orange4x4, 0x010E, 0x001d, 21, 1)] + public void ParseHeaderForFirstTile(string filename, int dataOffset, int dataSize, int tileOffset, int superblockCount) + { + // Assign + string filePath = Path.Combine(TestEnvironment.InputImagesDirectoryFullPath, filename); + byte[] content = File.ReadAllBytes(filePath); + Span headerSpan = content.AsSpan(dataOffset, dataSize); + Span tileSpan = content.AsSpan(dataOffset + tileOffset, dataSize - tileOffset); + Av1BitStreamReader bitStreamReader = new(headerSpan); + ObuReader obuReader = new(); + Av1FrameDecoderStub frameDecoder = new(); + + // Act + obuReader.ReadAll(ref bitStreamReader, dataSize, () => new Av1TileReader(Configuration.Default, obuReader.SequenceHeader, obuReader.FrameHeader, frameDecoder)); + + // Assert + Assert.Equal(dataSize * 8, bitStreamReader.BitPosition); + Assert.Equal(superblockCount, frameDecoder.SuperblockCount); + } + + private sealed class FailingTileAllocator : TestMemoryAllocator + { + private readonly int failureIndex; + + public FailingTileAllocator(int failureIndex) + { + this.failureIndex = failureIndex; + this.EnableNonThreadSafeLogging(); + } + + protected override AllocationTrackedMemoryManager AllocateCore(int length, AllocationOptions options) + { + if (this.AllocationLog.Count == this.failureIndex) + { + throw new InvalidMemoryOperationException("Tile allocation failure."); + } + + return base.AllocateCore(length, options); + } + } +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1TransformBlockEncoderTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1TransformBlockEncoderTests.cs new file mode 100644 index 0000000000..5d5beede63 --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1TransformBlockEncoderTests.cs @@ -0,0 +1,763 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.InteropServices; +using SixLabors.ImageSharp.Formats.Heif.Av1; +using SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline; +using SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.Quantizers; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.ChromaFromLuma; +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; +using SixLabors.ImageSharp.Memory; +using SixLabors.ImageSharp.Tests.Memory; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1; + +/// +/// Verifies the finalized prediction, transform, quantization, and reconstruction block boundary. +/// +[Trait("Format", "Avif")] +public class Av1TransformBlockEncoderTests +{ + /// + /// Verifies that the composed block path retains the exact outputs already established for its arithmetic stages. + /// + [Fact] + public void LossyBlockEncodingMatchesTransformAndQuantizerContracts() + { + ValidateBlock(Av1TransformSize.Size4x4, Av1TransformType.DctDct, Av1BitDepth.EightBit, 1); + ValidateBlock(Av1TransformSize.Size8x8, Av1TransformType.Identity, Av1BitDepth.TenBit, 73); + ValidateBlock(Av1TransformSize.Size32x64, Av1TransformType.DctDct, Av1BitDepth.TenBit, 173); + ValidateBlock(Av1TransformSize.Size64x64, Av1TransformType.DctDct, Av1BitDepth.TwelveBit, 255); + } + + /// + /// Verifies that the eight-bit block boundary preserves stage ordering, strides, padding, and retained syntax. + /// + [Fact] + public void EightBitIntraDcBlockEncodingMatchesStageContracts() + { + const int SourceStride = 13; + const int ReconstructionStride = 15; + const byte PaddingSentinel = 176; + Av1TransformSize transformSize = Av1TransformSize.Size8x8; + int width = transformSize.GetWidth(); + int height = transformSize.GetHeight(); + int coefficientCount = transformSize.GetAdjusted().GetSize2d(); + byte[] source = new byte[SourceStride * height]; + byte[] expectedReconstruction = new byte[ReconstructionStride * height]; + byte[] actualReconstruction = new byte[ReconstructionStride * height]; + byte[] above = new byte[width]; + byte[] left = new byte[height]; + int[] expectedQuantized = new int[coefficientCount + 7]; + int[] actualQuantized = new int[coefficientCount + 7]; + using Av1EncoderFrameBuffer sourceFrame = new( + Configuration.Default, + width, + height, + 8, + Av1ColorFormat.Yuv400, + 0, + 0); + + using Av1EncoderFrameBuffer reconstructionFrame = new( + Configuration.Default, + width, + height, + 8, + Av1ColorFormat.Yuv400, + 0, + 0); + + reconstructionFrame.Luma.DangerousGetSingleSpan().Fill(PaddingSentinel); + Buffer2DRegion sourcePlane = sourceFrame.Frame.CodedView.GetPlane(Av1Plane.Y); + Buffer2DRegion reconstructionPlane = reconstructionFrame.Frame.CodedView.GetPlane(Av1Plane.Y); + using Av1EncoderBlockWorkspace expectedWorkspace = new(Configuration.Default); + using Av1EncoderBlockWorkspace actualWorkspace = new(Configuration.Default); + FillSource(source, SourceStride, width, height, byte.MaxValue); + Array.Fill(expectedReconstruction, (byte)211); + Array.Fill(actualReconstruction, (byte)211); + Array.Fill(expectedQuantized, int.MinValue); + Array.Fill(actualQuantized, int.MinValue); + + for (int y = 0; y < height; y++) + { + source.AsSpan(y * SourceStride, width).CopyTo(sourcePlane.DangerousGetRowSpan(y)); + reconstructionPlane.DangerousGetRowSpan(y).Fill(211); + } + + for (int i = 0; i < above.Length; i++) + { + above[i] = (byte)(37 + (i * 11)); + } + + for (int i = 0; i < left.Length; i++) + { + left[i] = (byte)(19 + (i * 13)); + } + + Av1DcIntraPredictor.PredictScalar( + true, + true, + expectedReconstruction, + ReconstructionStride, + above, + left, + width, + height); + + for (int y = 0; y < height; y++) + { + for (int x = 0; x < width; x++) + { + expectedWorkspace.Residual[(y * width) + x] = + (short)(source[(y * SourceStride) + x] - expectedReconstruction[(y * ReconstructionStride) + x]); + } + } + + Av1EncoderTransformBlockState expectedState = default; + Av1TransformBlockEncoder.EncodeLossy( + expectedWorkspace, + expectedQuantized, + transformSize, + Av1TransformType.DctDct, + 73, + -1, + 3, + Av1BitDepth.EightBit, + ref expectedState); + + if (expectedState.EndOfBlock > 0) + { + Av1InverseTransformer.Reconstruct8Bit( + expectedWorkspace.DequantizedCoefficients, + expectedReconstruction, + ReconstructionStride, + transformSize, + Av1TransformType.DctDct, + (int)Av1Plane.Y, + expectedState.EndOfBlock, + false, + expectedWorkspace.TransformWorkspace); + } + + Av1EncoderTransformBlockState actualState = default; + Av1TransformBlockEncoder.EncodeIntraDcLossy( + actualWorkspace, + sourcePlane, + reconstructionPlane, + Point.Empty, + above, + left, + true, + true, + actualQuantized, + transformSize, + Av1TransformType.DctDct, + 73, + -1, + 3, + Av1Plane.Y, + ref actualState); + + for (int y = 0; y < height; y++) + { + reconstructionPlane.DangerousGetRowSpan(y).CopyTo( + actualReconstruction.AsSpan(y * ReconstructionStride, width)); + } + + int physicalRow = reconstructionPlane.Bounds.Y; + int physicalColumn = reconstructionPlane.Bounds.X; + ReadOnlySpan completeRow = reconstructionFrame.Luma.DangerousGetRowSpan(physicalRow); + Assert.Equal(expectedReconstruction, actualReconstruction); + Assert.Equal(expectedQuantized, actualQuantized); + Assert.Equal(expectedState.EndOfBlock, actualState.EndOfBlock); + Assert.Equal(expectedState.TransformType, actualState.TransformType); + Assert.Equal(PaddingSentinel, completeRow[physicalColumn - 1]); + Assert.Equal(PaddingSentinel, completeRow[physicalColumn + width]); + } + + /// + /// Verifies that the high-bit-depth block boundary preserves stage ordering, strides, padding, and retained syntax. + /// + [Fact] + public void HighBitDepthIntraDcBlockEncodingMatchesStageContracts() + { + const int SourceStride = 19; + const int ReconstructionStride = 23; + Av1TransformSize transformSize = Av1TransformSize.Size16x8; + Av1BitDepth bitDepth = Av1BitDepth.TenBit; + int width = transformSize.GetWidth(); + int height = transformSize.GetHeight(); + int coefficientCount = transformSize.GetAdjusted().GetSize2d(); + ushort[] source = new ushort[SourceStride * height]; + ushort[] expectedReconstruction = new ushort[ReconstructionStride * height]; + ushort[] actualReconstruction = new ushort[ReconstructionStride * height]; + ushort[] above = new ushort[width]; + ushort[] left = new ushort[height]; + int[] expectedQuantized = new int[coefficientCount + 7]; + int[] actualQuantized = new int[coefficientCount + 7]; + using Buffer2D sourceBuffer = Buffer2D.WrapMemory(source, SourceStride, height, SourceStride); + using Buffer2D reconstructionBuffer = + Buffer2D.WrapMemory(actualReconstruction, ReconstructionStride, height, ReconstructionStride); + + using Av1EncoderBlockWorkspace expectedWorkspace = new(Configuration.Default); + using Av1EncoderBlockWorkspace actualWorkspace = new(Configuration.Default); + FillSource(source, SourceStride, width, height, (1 << bitDepth.GetBitCount()) - 1); + Array.Fill(expectedReconstruction, (ushort)777); + Array.Fill(actualReconstruction, (ushort)777); + Array.Fill(expectedQuantized, int.MinValue); + Array.Fill(actualQuantized, int.MinValue); + + for (int i = 0; i < above.Length; i++) + { + above[i] = (ushort)(173 + (i * 17)); + } + + for (int i = 0; i < left.Length; i++) + { + left[i] = (ushort)(91 + (i * 29)); + } + + Span signedExpectedReconstruction = MemoryMarshal.Cast(expectedReconstruction.AsSpan()); + Av1DcIntraPredictor.PredictScalar( + true, + false, + signedExpectedReconstruction, + ReconstructionStride, + MemoryMarshal.Cast(above), + MemoryMarshal.Cast(left), + width, + height, + bitDepth.GetBitCount()); + + for (int y = 0; y < height; y++) + { + for (int x = 0; x < width; x++) + { + expectedWorkspace.Residual[(y * width) + x] = + (short)(source[(y * SourceStride) + x] - expectedReconstruction[(y * ReconstructionStride) + x]); + } + } + + Av1EncoderTransformBlockState expectedState = default; + Av1TransformBlockEncoder.EncodeLossy( + expectedWorkspace, + expectedQuantized, + transformSize, + Av1TransformType.DctDct, + 117, + -2, + 4, + bitDepth, + ref expectedState); + + if (expectedState.EndOfBlock > 0) + { + Av1InverseTransformer.ReconstructHighBitDepth( + expectedWorkspace.DequantizedCoefficients, + signedExpectedReconstruction, + ReconstructionStride, + transformSize, + Av1TransformType.DctDct, + (int)Av1Plane.U, + expectedState.EndOfBlock, + false, + bitDepth, + expectedWorkspace.TransformWorkspace); + } + + Av1EncoderTransformBlockState actualState = default; + Av1TransformBlockEncoder.EncodeIntraDcLossy( + actualWorkspace, + new Buffer2DRegion(sourceBuffer), + new Buffer2DRegion(reconstructionBuffer), + Point.Empty, + above, + left, + true, + false, + actualQuantized, + transformSize, + Av1TransformType.DctDct, + 117, + -2, + 4, + Av1Plane.U, + bitDepth, + ref actualState); + + Assert.Equal(expectedReconstruction, actualReconstruction); + Assert.Equal(expectedQuantized, actualQuantized); + Assert.Equal(expectedState.EndOfBlock, actualState.EndOfBlock); + Assert.Equal(expectedState.TransformType, actualState.TransformType); + } + + /// + /// Verifies that high-bit-depth candidate distortion follows the codec's pixel-domain normalization order. + /// + [Fact] + public void TwelveBitCandidateNormalizesSseBeforeTransformScaling() + { + const int Width = 8; + const int Height = 8; + ushort[] source = new ushort[Width * Height]; + ushort[] reconstruction = new ushort[Width * Height]; + ushort[] above = new ushort[Width]; + ushort[] left = new ushort[Height]; + int[] quantized = new int[Width * Height]; + for (int x = 0; x < Width; x++) + { + above[x] = (ushort)(1000 + (x * 113)); + } + + for (int y = 0; y < Height; y++) + { + for (int x = 0; x < Width; x++) + { + source[(y * Width) + x] = (ushort)(above[x] + 1); + } + } + + using Buffer2D sourceBuffer = Buffer2D.WrapMemory(source, Width, Height, Width); + using Av1EncoderBlockWorkspace workspace = new(Configuration.Default); + Av1EncoderTransformBlockState state = default; + long distortion = Av1TransformBlockEncoder.EncodeIntraLossyCandidate( + workspace, + new Buffer2DRegion(sourceBuffer), + Point.Empty, + reconstruction, + above, + left, + hasLeft: false, + hasAbove: true, + Av1PredictionMode.Vertical, + 0, + enableIntraEdgeFilter: false, + smoothIntraEdges: false, + quantized, + Av1TransformSize.Size8x8, + Av1TransformType.DctDct, + qIndex: 255, + dcDeltaQ: 0, + acDeltaQ: 0, + Av1Plane.Y, + Av1BitDepth.TwelveBit, + ref state); + + for (int y = 0; y < Height; y++) + { + Assert.True(above.AsSpan().SequenceEqual(reconstruction.AsSpan(y * Width, Width))); + } + + // Rounding the 64-sample SSE before the transform-domain scale is observably different from scaling first. + Assert.Equal((ushort)0, state.EndOfBlock); + Assert.Equal(0, distortion); + } + + /// + /// Verifies that high-bit-depth directional candidates apply the selected syntax adjustment. + /// + /// The signed AV1 directional adjustment. + [Theory] + [InlineData(-3)] + [InlineData(3)] + public void TwelveBitDirectionalCandidateAppliesAngleDelta(int angleDelta) + { + const int Width = 8; + const int Height = 8; + ushort[] source = new ushort[Width * Height]; + ushort[] reconstruction = new ushort[Width * Height]; + int[] quantized = new int[Width * Height]; + Span aboveStorage = stackalloc ushort[17]; + Span above = aboveStorage[1..]; + Span leftStorage = stackalloc ushort[17]; + Span left = leftStorage[1..]; + aboveStorage[0] = 2048; + leftStorage[0] = 2048; + for (int i = 0; i < 16; i++) + { + above[i] = (ushort)(512 + (i * 128)); + left[i] = (ushort)(3584 - (i * 128)); + } + + Span signedSource = MemoryMarshal.Cast(source.AsSpan()); + Span signedAbove = MemoryMarshal.Cast(above); + Span signedLeft = MemoryMarshal.Cast(left); + + // Directional arithmetic has independent scalar-oracle coverage. This isolates the high-bit-depth + // candidate boundary and proves that its signed syntax adjustment reaches prediction unchanged. + Av1DirectionalIntraPredictor.PredictScalar( + signedSource, + Width, + Av1TransformSize.Size8x8, + signedAbove, + signedLeft, + false, + false, + Av1PredictionMode.Directional135Degrees.ToAngle() + (angleDelta * Av1Constants.AngleStep)); + + using Buffer2D sourceBuffer = Buffer2D.WrapMemory(source, Width, Height, Width); + using Av1EncoderBlockWorkspace workspace = new(Configuration.Default); + Av1EncoderTransformBlockState state = default; + long distortion = Av1TransformBlockEncoder.EncodeIntraLossyCandidate( + workspace, + new Buffer2DRegion(sourceBuffer), + Point.Empty, + reconstruction, + above, + left, + hasLeft: true, + hasAbove: true, + Av1PredictionMode.Directional135Degrees, + angleDelta, + enableIntraEdgeFilter: false, + smoothIntraEdges: false, + quantized, + Av1TransformSize.Size8x8, + Av1TransformType.DctDct, + qIndex: 255, + dcDeltaQ: 0, + acDeltaQ: 0, + Av1Plane.Y, + Av1BitDepth.TwelveBit, + ref state); + + Assert.Equal(0, distortion); + Assert.Equal((ushort)0, state.EndOfBlock); + Assert.True(source.AsSpan().SequenceEqual(reconstruction)); + } + + /// + /// Verifies that complete eight-bit and high-bit-depth DC block encoding uses only caller-owned storage. + /// + [Fact] + public void IntraDcBlockEncodingDoesNotAllocate() + { + const int Stride = 8; + Av1TransformSize transformSize = Av1TransformSize.Size8x8; + int coefficientCount = transformSize.GetAdjusted().GetSize2d(); + byte[] source8 = new byte[Stride * Stride]; + byte[] reconstruction8 = new byte[Stride * Stride]; + byte[] above8 = new byte[Stride]; + byte[] left8 = new byte[Stride]; + ushort[] source10 = new ushort[Stride * Stride]; + ushort[] reconstruction10 = new ushort[Stride * Stride]; + ushort[] above10 = new ushort[Stride]; + ushort[] left10 = new ushort[Stride]; + int[] quantized = new int[coefficientCount]; + using Buffer2D sourceBuffer8 = Buffer2D.WrapMemory(source8, Stride, Stride); + using Buffer2D reconstructionBuffer8 = Buffer2D.WrapMemory(reconstruction8, Stride, Stride); + using Buffer2D sourceBuffer10 = Buffer2D.WrapMemory(source10, Stride, Stride); + using Buffer2D reconstructionBuffer10 = Buffer2D.WrapMemory(reconstruction10, Stride, Stride); + Buffer2DRegion sourcePlane8 = new(sourceBuffer8); + Buffer2DRegion reconstructionPlane8 = new(reconstructionBuffer8); + Buffer2DRegion sourcePlane10 = new(sourceBuffer10); + Buffer2DRegion reconstructionPlane10 = new(reconstructionBuffer10); + using Av1EncoderBlockWorkspace workspace = new(Configuration.Default); + FillSource(source8, Stride, Stride, Stride, byte.MaxValue); + FillSource(source10, Stride, Stride, Stride, 1023); + Array.Fill(above8, (byte)103); + Array.Fill(left8, (byte)127); + Array.Fill(above10, (ushort)503); + Array.Fill(left10, (ushort)527); + Av1EncoderTransformBlockState state = default; + + Av1TransformBlockEncoder.EncodeIntraDcLossy( + workspace, + sourcePlane8, + reconstructionPlane8, + Point.Empty, + above8, + left8, + true, + true, + quantized, + transformSize, + Av1TransformType.DctDct, + 73, + -1, + 3, + Av1Plane.Y, + ref state); + + Av1TransformBlockEncoder.EncodeIntraDcLossy( + workspace, + sourcePlane10, + reconstructionPlane10, + Point.Empty, + above10, + left10, + true, + true, + quantized, + transformSize, + Av1TransformType.DctDct, + 73, + -1, + 3, + Av1Plane.Y, + Av1BitDepth.TenBit, + ref state); + + long before = GC.GetAllocatedBytesForCurrentThread(); + for (int iteration = 0; iteration < 16; iteration++) + { + Av1TransformBlockEncoder.EncodeIntraDcLossy( + workspace, + sourcePlane8, + reconstructionPlane8, + Point.Empty, + above8, + left8, + true, + true, + quantized, + transformSize, + Av1TransformType.DctDct, + 73, + -1, + 3, + Av1Plane.Y, + ref state); + + Av1TransformBlockEncoder.EncodeIntraDcLossy( + workspace, + sourcePlane10, + reconstructionPlane10, + Point.Empty, + above10, + left10, + true, + true, + quantized, + transformSize, + Av1TransformType.DctDct, + 73, + -1, + 3, + Av1Plane.Y, + Av1BitDepth.TenBit, + ref state); + } + + Assert.Equal(0, GC.GetAllocatedBytesForCurrentThread() - before); + } + + /// + /// Verifies that repeated maximum-transform block encoding uses only caller-owned workspaces. + /// + [Fact] + public void LossyBlockEncodingDoesNotAllocate() + { + Av1TransformSize transformSize = Av1TransformSize.Size64x64; + int width = transformSize.GetWidth(); + int height = transformSize.GetHeight(); + int coefficientCount = transformSize.GetAdjusted().GetSize2d(); + int[] quantized = new int[coefficientCount]; + using Av1EncoderBlockWorkspace workspace = new(Configuration.Default); + FillResidual(workspace.Residual, width, height, 4095); + Av1EncoderTransformBlockState state = default; + + // Cross tiered-compilation call thresholds before measuring the steady-state transform kernel. + for (int iteration = 0; iteration < 64; iteration++) + { + Av1TransformBlockEncoder.EncodeLossy( + workspace, + quantized, + transformSize, + Av1TransformType.DctDct, + 73, + -1, + 3, + Av1BitDepth.TwelveBit, + ref state); + } + + long before = GC.GetAllocatedBytesForCurrentThread(); + for (int iteration = 0; iteration < 16; iteration++) + { + Av1TransformBlockEncoder.EncodeLossy( + workspace, + quantized, + transformSize, + Av1TransformType.DctDct, + 73, + -1, + 3, + Av1BitDepth.TwelveBit, + ref state); + } + + Assert.Equal(0, GC.GetAllocatedBytesForCurrentThread() - before); + } + + /// + /// Verifies that the block workspace uses one exact-size allocator owner and returns it exactly once. + /// + [Fact] + public void BlockWorkspaceUsesOneExactSizeOwner() + { + TestMemoryAllocator allocator = new(); + allocator.EnableNonThreadSafeLogging(); + Configuration configuration = Configuration.Default.Clone(); + configuration.MemoryAllocator = allocator; + + TestMemoryAllocator.AllocationRequest allocation; + using (Av1EncoderBlockWorkspace workspace = new(configuration)) + { + allocation = Assert.Single(allocator.AllocationLog); + Assert.Empty(allocator.ReturnLog); + Assert.Equal(typeof(int), allocation.ElementType); + Assert.Equal(Av1EncoderBlockWorkspace.StorageLength, allocation.Length); + Assert.Equal(Av1EncoderBlockWorkspace.MaximumResidualCount, workspace.Residual.Length); + Assert.Equal(Av1EncoderBlockWorkspace.MaximumCoefficientCount, workspace.TransformCoefficients.Length); + Assert.Equal(Av1EncoderBlockWorkspace.MaximumCoefficientCount, workspace.DequantizedCoefficients.Length); + Assert.Equal(Av1TransformWorkspace.MaximumLength, workspace.TransformWorkspace.Length); + + Av1EncoderModeDecisionWorkspace modeWorkspace = workspace.GetModeDecisionWorkspace(); + Av1EncoderPaletteWorkspace paletteWorkspace = modeWorkspace.Palette; + Av1EncoderInterPredictionWorkspace intraBlockCopyWorkspace = + workspace.GetInterPredictionWorkspace(); + + Assert.Equal( + (2 * Av1Constants.MaxTransformSize) + 1, + modeWorkspace.GetReferenceSamples(3).Length); + + Assert.Equal(Av1EncoderModeDecisionWorkspace.MaximumSampleCount, modeWorkspace.GetCandidateReconstruction(1).Length); + Assert.Equal(Av1EncoderModeDecisionWorkspace.MaximumSampleCount, modeWorkspace.GetCandidateCoefficients(1).Length); + Assert.Equal(Av1EncoderModeDecisionWorkspace.MaximumTransformSampleCount, modeWorkspace.Prediction.Length); + Assert.Equal(Av1EncoderModeDecisionWorkspace.MaximumTransformSampleCount, modeWorkspace.Residual.Length); + Assert.Equal(Av1EncoderModeDecisionWorkspace.MaximumCandidateTransformBlockCount, modeWorkspace.CandidateTransformBlocks.Length); + Assert.Equal(2048, modeWorkspace.CandidateTransformBlocks.Length); + + // CfL is unavailable above 32x32, so its scratch remains fixed while larger partitions are enabled. + Assert.Equal(Av1ChromaFromLumaContext.BufferLength, modeWorkspace.ChromaFromLumaSamples.Length); + + Assert.Equal(Av1ChromaFromLumaMath.AlphaCandidateCount, modeWorkspace.GetChromaFromLumaRates(1).Length); + Assert.Equal(Av1ChromaFromLumaMath.AlphaCandidateCount, modeWorkspace.GetChromaFromLumaDistortions(1).Length); + int maximumPaletteSampleCount = + Av1BlockSize.Block64x64.GetWidth() * Av1BlockSize.Block64x64.GetHeight(); + + Assert.Equal(maximumPaletteSampleCount, paletteWorkspace.GetPrediction(1).Length); + Assert.Equal(maximumPaletteSampleCount, paletteWorkspace.AlternateIndices.Length); + + // Conventional mode search and IBC are sequential, so their typed views intentionally alias one owner region. + modeWorkspace.GetReferenceSamples(0)[0] = 123; + Assert.Equal((ushort)123, intraBlockCopyWorkspace.SelectedLumaReconstruction[0]); + } + + TestMemoryAllocator.ReturnRequest returned = Assert.Single(allocator.ReturnLog); + Assert.Equal(allocation.AllocationId, returned.AllocationId); + } + + private static void ValidateBlock( + Av1TransformSize transformSize, + Av1TransformType transformType, + Av1BitDepth bitDepth, + int qIndex) + { + int width = transformSize.GetWidth(); + int height = transformSize.GetHeight(); + int coefficientCount = transformSize.GetAdjusted().GetSize2d(); + int sampleMaximum = (1 << bitDepth.GetBitCount()) - 1; + int[] expectedTransformed = new int[coefficientCount + 7]; + int[] expectedQuantized = new int[coefficientCount + 7]; + int[] expectedDequantized = new int[coefficientCount + 7]; + int[] actualQuantized = new int[coefficientCount + 7]; + int[] expectedWorkspace = new int[Av1TransformWorkspace.MaximumLength]; + using Av1EncoderBlockWorkspace blockWorkspace = new(Configuration.Default); + Array.Fill(expectedTransformed, int.MinValue); + Array.Fill(expectedQuantized, int.MinValue); + Array.Fill(expectedDequantized, int.MinValue); + Array.Fill(actualQuantized, int.MinValue); + FillResidual(blockWorkspace.Residual, width, height, sampleMaximum); + + Av1ForwardTransformer.Transform2d( + blockWorkspace.Residual, + expectedTransformed.AsSpan(0, coefficientCount), + (uint)width, + transformType, + transformSize, + bitDepth.GetBitCount(), + expectedWorkspace); + + ushort expectedEndOfBlock = Av1ForwardQuantizer.QuantizeLossy( + expectedTransformed, + expectedQuantized, + expectedDequantized, + transformSize, + transformType, + qIndex, + -1, + 3, + bitDepth); + + Av1EncoderTransformBlockState actualState = default; + Av1TransformBlockEncoder.EncodeLossy( + blockWorkspace, + actualQuantized, + transformSize, + transformType, + qIndex, + -1, + 3, + bitDepth, + ref actualState); + + AssertEqual(expectedTransformed, blockWorkspace.TransformCoefficients, coefficientCount); + Assert.Equal(expectedQuantized, actualQuantized); + AssertEqual(expectedDequantized, blockWorkspace.DequantizedCoefficients, coefficientCount); + Assert.Equal(expectedEndOfBlock, actualState.EndOfBlock); + Assert.Equal(transformType, actualState.TransformType); + } + + private static void FillResidual(Span residual, int width, int height, int sampleMaximum) + { + for (int y = 0; y < height; y++) + { + for (int x = 0; x < width; x++) + { + int index = (y * width) + x; + residual[index] = (short)((index & 3) switch + { + 0 => sampleMaximum, + 1 => -sampleMaximum, + 2 => ((index * 73) % ((2 * sampleMaximum) + 1)) - sampleMaximum, + _ => 0, + }); + } + } + } + + private static void FillSource(Span source, int stride, int width, int height, int sampleMaximum) + { + for (int y = 0; y < height; y++) + { + for (int x = 0; x < width; x++) + { + source[(y * stride) + x] = (byte)(((y * 43) + (x * 71) + 29) % (sampleMaximum + 1)); + } + } + } + + private static void FillSource(Span source, int stride, int width, int height, int sampleMaximum) + { + for (int y = 0; y < height; y++) + { + for (int x = 0; x < width; x++) + { + source[(y * stride) + x] = (ushort)(((y * 181) + (x * 313) + 97) % (sampleMaximum + 1)); + } + } + } + + private static void AssertEqual(ReadOnlySpan expected, ReadOnlySpan actual, int count) + { + for (int i = 0; i < count; i++) + { + Assert.Equal(expected[i], actual[i]); + } + } +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1TransformSizeTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1TransformSizeTests.cs new file mode 100644 index 0000000000..0d0a2b26ec --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1TransformSizeTests.cs @@ -0,0 +1,174 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1; + +[Trait("Format", "Avif")] +public class Av1TransformSizeTests +{ + [Theory] + [MemberData(nameof(GetAllSizes))] + internal void GetWidthReturnsCorrectWidth(int s) + { + // Assign + Av1TransformSize transformSize = (Av1TransformSize)s; + int expectedWidth = transformSize switch + { + Av1TransformSize.Size4x4 or Av1TransformSize.Size4x8 or Av1TransformSize.Size4x16 => 4, + Av1TransformSize.Size8x4 or Av1TransformSize.Size8x8 or Av1TransformSize.Size8x16 or Av1TransformSize.Size8x32 => 8, + Av1TransformSize.Size16x4 or Av1TransformSize.Size16x8 or Av1TransformSize.Size16x16 or Av1TransformSize.Size16x32 or Av1TransformSize.Size16x64 => 16, + Av1TransformSize.Size32x8 or Av1TransformSize.Size32x16 or Av1TransformSize.Size32x32 or Av1TransformSize.Size32x64 => 32, + Av1TransformSize.Size64x16 or Av1TransformSize.Size64x32 or Av1TransformSize.Size64x64 => 64, + _ => -1 + }; + + // Act + int actualWidth = transformSize.GetWidth(); + + // Assert + Assert.Equal(expectedWidth, actualWidth); + } + + [Theory] + [MemberData(nameof(GetAllSizes))] + internal void GetHeightReturnsCorrectHeight(int s) + { + // Assign + Av1TransformSize transformSize = (Av1TransformSize)s; + int expectedHeight = transformSize switch + { + Av1TransformSize.Size4x4 or Av1TransformSize.Size8x4 or Av1TransformSize.Size16x4 => 4, + Av1TransformSize.Size4x8 or Av1TransformSize.Size8x8 or Av1TransformSize.Size16x8 or Av1TransformSize.Size32x8 => 8, + Av1TransformSize.Size4x16 or Av1TransformSize.Size8x16 or Av1TransformSize.Size16x16 or Av1TransformSize.Size32x16 or Av1TransformSize.Size64x16 => 16, + Av1TransformSize.Size8x32 or Av1TransformSize.Size16x32 or Av1TransformSize.Size32x32 or Av1TransformSize.Size64x32 => 32, + Av1TransformSize.Size16x64 or Av1TransformSize.Size32x64 or Av1TransformSize.Size64x64 => 64, + _ => -1 + }; + + // Act + int actualHeight = transformSize.GetHeight(); + + // Assert + Assert.Equal(expectedHeight, actualHeight); + } + + [Theory] + [MemberData(nameof(GetAllSizes))] + internal void GetSubSizeReturnsCorrectRatio(int s) + { + // Assign + Av1TransformSize transformSize = (Av1TransformSize)s; + int ratio = GetRatio(transformSize); + int expectedRatio = (ratio == 4) ? 2 : 1; + + // Act + Av1TransformSize actual = transformSize.GetSubSize(); + int actualRatio = GetRatio(actual); + + // Assert + Assert.Equal(expectedRatio, actualRatio); + } + + [Theory] + [MemberData(nameof(GetAllSizes))] + internal void GetSquareSizeReturnsCorrectRatio(int s) + { + // Assign + Av1TransformSize transformSize = (Av1TransformSize)s; + int ratio = GetRatio(transformSize); + int expectedRatio = 1; + int expectedSize = Math.Min(transformSize.GetWidth(), transformSize.GetHeight()); + + // Act + Av1TransformSize actual = transformSize.GetSquareSize(); + int actualRatio = GetRatio(actual); + + // Assert + Assert.Equal(expectedRatio, actualRatio); + Assert.Equal(expectedSize, actual.GetWidth()); + Assert.Equal(expectedSize, actual.GetHeight()); + } + + [Theory] + [MemberData(nameof(GetAllSizes))] + internal void GetSquareUpSizeReturnsCorrectRatio(int s) + { + // Assign + Av1TransformSize transformSize = (Av1TransformSize)s; + int ratio = GetRatio(transformSize); + int expectedRatio = 1; + int expectedSize = Math.Max(transformSize.GetWidth(), transformSize.GetHeight()); + + // Act + Av1TransformSize actual = transformSize.GetSquareUpSize(); + int actualRatio = GetRatio(actual); + + // Assert + Assert.Equal(expectedRatio, actualRatio); + Assert.Equal(expectedSize, actual.GetWidth()); + Assert.Equal(expectedSize, actual.GetHeight()); + } + + [Theory] + [MemberData(nameof(GetAllSizes))] + internal void ToBlockSizeReturnsSameWidthAndHeight(int s) + { + // Assign + Av1TransformSize transformSize = (Av1TransformSize)s; + int transformWidth = transformSize.GetWidth(); + int transformHeight = transformSize.GetHeight(); + + // Act + Av1BlockSize blockSize = transformSize.ToBlockSize(); + int blockWidth = blockSize.GetWidth(); + int blockHeight = blockSize.GetHeight(); + + // Assert + Assert.Equal(transformWidth, blockWidth); + Assert.Equal(transformHeight, blockHeight); + } + + [Theory] + [MemberData(nameof(GetAllSizes))] + internal void LogMinus4ReturnsReferenceValues(int s) + { + // Assign + Av1TransformSize transformSize = (Av1TransformSize)s; + int expected = ReferenceLog2Minus4(transformSize); + + // Act + int actual = transformSize.GetLog2Minus4(); + + // Assert + Assert.Equal(expected, actual); + } + + public static TheoryData GetAllSizes() + { + TheoryData combinations = []; + for (int s = 0; s < (int)Av1TransformSize.AllSizes; s++) + { + combinations.Add(s); + } + + return combinations; + } + + private static int GetRatio(Av1TransformSize transformSize) + { + int width = transformSize.GetWidth(); + int height = transformSize.GetHeight(); + int ratio = width > height ? width / height : height / width; + return ratio; + } + + private static int ReferenceLog2Minus4(Av1TransformSize transformSize) + { + int widthLog2 = Av1Math.Log2(transformSize.GetWidth()); + int heightLog2 = Av1Math.Log2(transformSize.GetHeight()); + return Math.Min(widthLog2, 5) + Math.Min(heightLog2, 5) - 4; + } +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1TranslationalInterPredictorTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1TranslationalInterPredictorTests.cs new file mode 100644 index 0000000000..0ccd072a11 --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1TranslationalInterPredictorTests.cs @@ -0,0 +1,1000 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; +using SixLabors.ImageSharp.Tests.TestUtilities; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1; + +/// +/// Verifies AV1 translational inter prediction against an independent implementation of the normative fixed-point convolution rules. +/// +[Trait("Format", "Avif")] +public class Av1TranslationalInterPredictorTests +{ + /// + /// The number of fractional coefficient bits in AV1 interpolation kernels. + /// + private const int FilterBits = 7; + + /// + /// The default first convolution shift used for 8- and 10-bit predictions. + /// + private const int Round0Bits = 3; + + /// + /// The number of stored coefficient positions in every tested interpolation kernel. + /// + private const int FilterTapCount = 8; + + /// + /// The number of coefficient positions preceding the integer-position sample. + /// + private const int FilterCenterOffset = 3; + + /// + /// The source samples retained before the integer-position column. + /// + private const int SourceLeftPadding = 3; + + /// + /// The source samples retained after each active row for a complete 512-bit byte load. + /// + private const int SourceRightPadding = 64; + + /// + /// The source rows retained before the integer-position row. + /// + private const int SourceTopPadding = 3; + + /// + /// The source rows retained after the prediction block. + /// + private const int SourceBottomPadding = 4; + + /// + /// The guarded destination elements preceding the first active row. + /// + private const int DestinationPrefix = 11; + + /// + /// The guarded destination elements following the final padded row. + /// + private const int DestinationSuffix = 17; + + /// + /// The guarded destination elements following each active row. + /// + private const int DestinationRowPadding = 13; + + /// + /// The non-image value stored in every guarded 8-bit destination element. + /// + private const byte ByteDestinationSentinel = 0xD3; + + /// + /// The non-image value stored in every guarded ushort destination element. + /// + private const ushort HighBitDepthDestinationSentinel = 0xDEAD; + + /// + /// Exercises the native vector width, the 256-bit path, the 128-bit path, and the complete scalar fallback. + /// + /// + /// Disabling AVX also disables AVX2 and leaves the x86 128-bit vector tier enabled, which is the established + /// configuration used by the other AV1 SIMD tests. + /// + private const HwIntrinsics PredictorConfigurations = + HwIntrinsics.AllowAll | HwIntrinsics.DisableAVX512F | HwIntrinsics.DisableAVX | HwIntrinsics.DisableHWIntrinsic; + + /// + /// Verifies exact 8-bit copy and convolution output, scalar tails, and untouched destination padding under every SIMD configuration. + /// + [Fact] + public void BytePredictionMatchesReference() + => FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidateBytePredictions, PredictorConfigurations); + + /// + /// Verifies exact 8-, 10-, and 12-bit ushort output, scalar tails, and untouched destination padding under every SIMD configuration. + /// + [Fact] + public void HighBitDepthPredictionMatchesReference() + => FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidateHighBitDepthPredictions, PredictorConfigurations); + + /// + /// Verifies that SIMD compound intermediates retain scalar-equivalent values and untouched destination padding. + /// + [Fact] + public void CompoundPredictionMatchesScalarAcrossIntrinsicWidths() + => FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidateCompoundPredictions, PredictorConfigurations); + + /// + /// Applies every byte prediction scenario to the SIMD-first and explicitly scalar entry points. + /// + private static void ValidateBytePredictions() + { + foreach (PredictionCase testCase in CreatePredictionCases()) + { + byte[] source = CreateByteSource(testCase, out int sourceStride, out int sourceOrigin); + int destinationStride = testCase.Width + DestinationRowPadding; + byte[] expected = CreateByteDestination(testCase, destinationStride); + byte[] actual = (byte[])expected.Clone(); + byte[] scalar = (byte[])expected.Clone(); + short[] simdScratch = CreateScratch(testCase); + short[] scalarScratch = CreateScratch(testCase); + + ApplyReference(source, sourceStride, sourceOrigin, expected, DestinationPrefix, destinationStride, testCase, 8); + + Av1TranslationalInterPredictor.Predict( + source, + sourceStride, + sourceOrigin, + actual.AsSpan(DestinationPrefix), + destinationStride, + testCase.Width, + testCase.Height, + testCase.HorizontalFilter, + testCase.VerticalFilter, + testCase.HorizontalPhase, + testCase.VerticalPhase, + simdScratch); + + Av1TranslationalInterPredictor.PredictScalar( + source, + sourceStride, + sourceOrigin, + scalar.AsSpan(DestinationPrefix), + destinationStride, + testCase.Width, + testCase.Height, + testCase.HorizontalFilter, + testCase.VerticalFilter, + testCase.HorizontalPhase, + testCase.VerticalPhase, + scalarScratch); + + AssertEqual(expected, actual, testCase, "SIMD-first byte"); + AssertEqual(expected, scalar, testCase, "scalar byte"); + } + } + + /// + /// Applies every ushort prediction scenario at each supported sample precision to the SIMD-first and scalar entry points. + /// + private static void ValidateHighBitDepthPredictions() + { + int[] bitDepths = [8, 10, 12]; + foreach (int bitDepth in bitDepths) + { + foreach (PredictionCase testCase in CreatePredictionCases()) + { + ushort[] source = CreateHighBitDepthSource(testCase, bitDepth, out int sourceStride, out int sourceOrigin); + int destinationStride = testCase.Width + DestinationRowPadding; + ushort[] expected = CreateHighBitDepthDestination(testCase, destinationStride); + ushort[] actual = (ushort[])expected.Clone(); + ushort[] scalar = (ushort[])expected.Clone(); + short[] simdScratch = CreateScratch(testCase); + short[] scalarScratch = CreateScratch(testCase); + + ApplyReference(source, sourceStride, sourceOrigin, expected, DestinationPrefix, destinationStride, testCase, bitDepth); + + Av1TranslationalInterPredictor.Predict( + source, + sourceStride, + sourceOrigin, + actual.AsSpan(DestinationPrefix), + destinationStride, + testCase.Width, + testCase.Height, + testCase.HorizontalFilter, + testCase.VerticalFilter, + testCase.HorizontalPhase, + testCase.VerticalPhase, + bitDepth, + simdScratch); + + Av1TranslationalInterPredictor.PredictScalar( + source, + sourceStride, + sourceOrigin, + scalar.AsSpan(DestinationPrefix), + destinationStride, + testCase.Width, + testCase.Height, + testCase.HorizontalFilter, + testCase.VerticalFilter, + testCase.HorizontalPhase, + testCase.VerticalPhase, + bitDepth, + scalarScratch); + + AssertEqual(expected, actual, testCase, $"SIMD-first {bitDepth}-bit ushort"); + AssertEqual(expected, scalar, testCase, $"scalar {bitDepth}-bit ushort"); + } + } + } + + /// + /// Applies every byte prediction scenario to the SIMD-first and scalar compound-intermediate entry points. + /// + private static void ValidateCompoundPredictions() + { + foreach (PredictionCase testCase in CreatePredictionCases()) + { + byte[] source = CreateByteSource(testCase, out int sourceStride, out int sourceOrigin); + int destinationStride = testCase.Width + DestinationRowPadding; + ushort[] expected = CreateHighBitDepthDestination(testCase, destinationStride); + ushort[] actual = (ushort[])expected.Clone(); + short[] simdScratch = CreateScratch(testCase); + short[] scalarScratch = CreateScratch(testCase); + + Av1CompoundInterPredictor.PredictCompoundScalar( + source, + sourceStride, + sourceOrigin, + expected.AsSpan(DestinationPrefix), + destinationStride, + testCase.Width, + testCase.Height, + testCase.HorizontalFilter, + testCase.VerticalFilter, + testCase.HorizontalPhase, + testCase.VerticalPhase, + scalarScratch); + + Av1CompoundInterPredictor.PredictCompound( + source, + sourceStride, + sourceOrigin, + actual.AsSpan(DestinationPrefix), + destinationStride, + testCase.Width, + testCase.Height, + testCase.HorizontalFilter, + testCase.VerticalFilter, + testCase.HorizontalPhase, + testCase.VerticalPhase, + simdScratch); + + AssertEqual(expected, actual, testCase, "SIMD-first compound intermediate"); + } + } + + /// + /// Creates the named operation matrix covering copy, each one-dimensional direction, separable filtering, reduced kernels, and vector tails. + /// + /// The prediction scenarios. + private static PredictionCase[] CreatePredictionCases() => + [ + new("copy-sub8x8-chroma", 2, 4, Av1InterpolationFilter.Regular, Av1InterpolationFilter.Sharp, 0, 0), + new("regular-horizontal-sub8x8-chroma", 2, 4, Av1InterpolationFilter.Regular, Av1InterpolationFilter.Regular, 3, 0), + new("regular-vertical-sub8x8-chroma", 4, 2, Av1InterpolationFilter.Regular, Av1InterpolationFilter.Regular, 0, 3), + new("smooth-sharp-sub8x8-chroma", 2, 2, Av1InterpolationFilter.Smooth, Av1InterpolationFilter.Sharp, 7, 13), + new("copy-wide-tail", 68, 8, Av1InterpolationFilter.Regular, Av1InterpolationFilter.Sharp, 0, 0), + new("regular-horizontal-wide-tail", 68, 8, Av1InterpolationFilter.Regular, Av1InterpolationFilter.Regular, 1, 0), + new("smooth-horizontal-256-tail", 36, 8, Av1InterpolationFilter.Smooth, Av1InterpolationFilter.Regular, 7, 0), + new("sharp-horizontal-128-tail", 20, 8, Av1InterpolationFilter.Sharp, Av1InterpolationFilter.Regular, 8, 0), + new("bilinear-horizontal", 8, 8, Av1InterpolationFilter.Bilinear, Av1InterpolationFilter.Regular, 15, 0), + new("regular-horizontal-reduced", 4, 8, Av1InterpolationFilter.Regular, Av1InterpolationFilter.Regular, 3, 0), + new("smooth-horizontal-reduced", 4, 8, Av1InterpolationFilter.Smooth, Av1InterpolationFilter.Regular, 13, 0), + new("sharp-horizontal-reduced", 4, 8, Av1InterpolationFilter.Sharp, Av1InterpolationFilter.Regular, 5, 0), + new("regular-vertical-reduced", 68, 4, Av1InterpolationFilter.Regular, Av1InterpolationFilter.Regular, 0, 3), + new("smooth-vertical-reduced", 36, 4, Av1InterpolationFilter.Regular, Av1InterpolationFilter.Smooth, 0, 13), + new("sharp-vertical-reduced", 20, 4, Av1InterpolationFilter.Regular, Av1InterpolationFilter.Sharp, 0, 5), + new("bilinear-vertical", 8, 8, Av1InterpolationFilter.Regular, Av1InterpolationFilter.Bilinear, 0, 8), + new("regular-smooth-two-dimensional", 68, 8, Av1InterpolationFilter.Regular, Av1InterpolationFilter.Smooth, 1, 15), + new("sharp-bilinear-two-dimensional", 36, 8, Av1InterpolationFilter.Sharp, Av1InterpolationFilter.Bilinear, 8, 3), + new("smooth-sharp-reduced-two-dimensional", 4, 4, Av1InterpolationFilter.Smooth, Av1InterpolationFilter.Sharp, 7, 13), + new("bilinear-regular-small-height", 20, 4, Av1InterpolationFilter.Bilinear, Av1InterpolationFilter.Regular, 11, 5), + new("sharp-smooth-small-width", 4, 8, Av1InterpolationFilter.Sharp, Av1InterpolationFilter.Smooth, 5, 7) + ]; + + /// + /// Creates an 8-bit padded reference plane and returns the integer-position source origin within that plane. + /// + /// The prediction geometry used to size the plane. + /// Receives the padded source-row stride. + /// Receives the integer-position sample index. + /// The complete padded source plane. + private static byte[] CreateByteSource(PredictionCase testCase, out int sourceStride, out int sourceOrigin) + { + sourceStride = SourceLeftPadding + testCase.Width + SourceRightPadding; + int sourceHeight = SourceTopPadding + testCase.Height + SourceBottomPadding; + byte[] source = new byte[sourceStride * sourceHeight]; + + for (int row = 0; row < sourceHeight; row++) + { + for (int column = 0; column < sourceStride; column++) + { + // Distinct row, column, and cross-term multipliers prevent a wrong stride or tap direction from + // producing the same arithmetic progression as the correctly addressed source window. + source[(row * sourceStride) + column] = (byte)(((row * 59) + (column * 37) + (row * column * 11) + 17) & byte.MaxValue); + } + } + + sourceOrigin = (SourceTopPadding * sourceStride) + SourceLeftPadding; + return source; + } + + /// + /// Creates a padded high-bit-depth reference plane spanning the legal range for the requested precision. + /// + /// The prediction geometry used to size the plane. + /// The decoded sample precision. + /// Receives the padded source-row stride. + /// Receives the integer-position sample index. + /// The complete padded source plane. + private static ushort[] CreateHighBitDepthSource(PredictionCase testCase, int bitDepth, out int sourceStride, out int sourceOrigin) + { + sourceStride = SourceLeftPadding + testCase.Width + SourceRightPadding; + int sourceHeight = SourceTopPadding + testCase.Height + SourceBottomPadding; + int maximum = (1 << bitDepth) - 1; + ushort[] source = new ushort[sourceStride * sourceHeight]; + + for (int row = 0; row < sourceHeight; row++) + { + for (int column = 0; column < sourceStride; column++) + { + // The high-bit-depth pattern uses different coprime multipliers and spans the complete requested + // range, exercising negative-lobe clipping as well as low and high sample values. + source[(row * sourceStride) + column] = (ushort)(((row * 977) + (column * 353) + (row * column * 29) + 101) & maximum); + } + } + + sourceOrigin = (SourceTopPadding * sourceStride) + SourceLeftPadding; + return source; + } + + /// + /// Creates an 8-bit destination whose prefix, row padding, and suffix expose stores outside the prediction block. + /// + /// The prediction geometry used to size the destination. + /// The padded destination-row stride. + /// The guarded destination storage. + private static byte[] CreateByteDestination(PredictionCase testCase, int destinationStride) + => Enumerable.Repeat(ByteDestinationSentinel, DestinationPrefix + (destinationStride * testCase.Height) + DestinationSuffix).ToArray(); + + /// + /// Creates a ushort destination whose prefix, row padding, and suffix expose stores outside the prediction block. + /// + /// The prediction geometry used to size the destination. + /// The padded destination-row stride. + /// The guarded destination storage. + private static ushort[] CreateHighBitDepthDestination(PredictionCase testCase, int destinationStride) + => Enumerable + .Repeat(HighBitDepthDestinationSentinel, DestinationPrefix + (destinationStride * testCase.Height) + DestinationSuffix) + .ToArray(); + + /// + /// Creates caller-owned two-dimensional intermediate storage using AV1's eight-tap vertical extent. + /// + /// The prediction geometry and fractional phases. + /// The required scratch storage, or an empty array for copy and one-dimensional predictions. + private static short[] CreateScratch(PredictionCase testCase) + => testCase.HorizontalPhase == 0 || testCase.VerticalPhase == 0 + ? [] + : new short[Math.Max(testCase.Width, 16) * (testCase.Height + FilterTapCount - 1)]; + + /// + /// Applies the reference decoder's single-reference copy or convolution equations to an 8-bit prediction block. + /// + /// The complete padded reference plane. + /// The source-row stride. + /// The integer-position sample index. + /// The guarded destination storage. + /// The first active destination index. + /// The destination-row stride. + /// The prediction filters, phases, and geometry. + /// The decoded sample precision. + private static void ApplyReference( + byte[] source, + int sourceStride, + int sourceOrigin, + byte[] destination, + int destinationOrigin, + int destinationStride, + PredictionCase testCase, + int bitDepth) + { + if (testCase.HorizontalPhase == 0 && testCase.VerticalPhase == 0) + { + for (int row = 0; row < testCase.Height; row++) + { + source.AsSpan(sourceOrigin + (row * sourceStride), testCase.Width) + .CopyTo(destination.AsSpan(destinationOrigin + (row * destinationStride), testCase.Width)); + } + + return; + } + + if (testCase.VerticalPhase == 0) + { + ReadOnlySpan horizontal = GetCoefficients(testCase.HorizontalFilter, testCase.HorizontalPhase, testCase.Width <= 4); + int round0 = GetRound0Bits(bitDepth); + + for (int row = 0; row < testCase.Height; row++) + { + for (int column = 0; column < testCase.Width; column++) + { + int sourceIndex = sourceOrigin + (row * sourceStride) + column - FilterCenterOffset; + int sum = Convolve(source, sourceIndex, 1, horizontal); + int value = RoundPowerOfTwo(RoundPowerOfTwo(sum, round0), FilterBits - round0); + destination[destinationOrigin + (row * destinationStride) + column] = (byte)Math.Clamp(value, 0, byte.MaxValue); + } + } + + return; + } + + if (testCase.HorizontalPhase == 0) + { + ReadOnlySpan vertical = GetCoefficients(testCase.VerticalFilter, testCase.VerticalPhase, testCase.Height <= 4); + + for (int row = 0; row < testCase.Height; row++) + { + for (int column = 0; column < testCase.Width; column++) + { + int sourceIndex = sourceOrigin + ((row - FilterCenterOffset) * sourceStride) + column; + int sum = Convolve(source, sourceIndex, sourceStride, vertical); + int value = RoundPowerOfTwo(sum, FilterBits); + destination[destinationOrigin + (row * destinationStride) + column] = (byte)Math.Clamp(value, 0, byte.MaxValue); + } + } + + return; + } + + ReadOnlySpan horizontalCoefficients = GetCoefficients(testCase.HorizontalFilter, testCase.HorizontalPhase, testCase.Width <= 4); + ReadOnlySpan verticalCoefficients = GetCoefficients(testCase.VerticalFilter, testCase.VerticalPhase, testCase.Height <= 4); + + ApplyTwoDimensionalReference( + source, + sourceStride, + sourceOrigin, + destination, + destinationOrigin, + destinationStride, + testCase, + horizontalCoefficients, + verticalCoefficients, + bitDepth); + } + + /// + /// Applies the reference decoder's single-reference copy or convolution equations to a high-bit-depth prediction block. + /// + /// The complete padded reference plane. + /// The source-row stride. + /// The integer-position sample index. + /// The guarded destination storage. + /// The first active destination index. + /// The destination-row stride. + /// The prediction filters, phases, and geometry. + /// The decoded sample precision. + private static void ApplyReference( + ushort[] source, + int sourceStride, + int sourceOrigin, + ushort[] destination, + int destinationOrigin, + int destinationStride, + PredictionCase testCase, + int bitDepth) + { + if (testCase.HorizontalPhase == 0 && testCase.VerticalPhase == 0) + { + for (int row = 0; row < testCase.Height; row++) + { + source.AsSpan(sourceOrigin + (row * sourceStride), testCase.Width) + .CopyTo(destination.AsSpan(destinationOrigin + (row * destinationStride), testCase.Width)); + } + + return; + } + + int maximum = (1 << bitDepth) - 1; + if (testCase.VerticalPhase == 0) + { + ReadOnlySpan horizontal = GetCoefficients(testCase.HorizontalFilter, testCase.HorizontalPhase, testCase.Width <= 4); + int round0 = GetRound0Bits(bitDepth); + + for (int row = 0; row < testCase.Height; row++) + { + for (int column = 0; column < testCase.Width; column++) + { + int sourceIndex = sourceOrigin + (row * sourceStride) + column - FilterCenterOffset; + int sum = Convolve(source, sourceIndex, 1, horizontal); + int value = RoundPowerOfTwo(RoundPowerOfTwo(sum, round0), FilterBits - round0); + destination[destinationOrigin + (row * destinationStride) + column] = (ushort)Math.Clamp(value, 0, maximum); + } + } + + return; + } + + if (testCase.HorizontalPhase == 0) + { + ReadOnlySpan vertical = GetCoefficients(testCase.VerticalFilter, testCase.VerticalPhase, testCase.Height <= 4); + + for (int row = 0; row < testCase.Height; row++) + { + for (int column = 0; column < testCase.Width; column++) + { + int sourceIndex = sourceOrigin + ((row - FilterCenterOffset) * sourceStride) + column; + int sum = Convolve(source, sourceIndex, sourceStride, vertical); + int value = RoundPowerOfTwo(sum, FilterBits); + destination[destinationOrigin + (row * destinationStride) + column] = (ushort)Math.Clamp(value, 0, maximum); + } + } + + return; + } + + ReadOnlySpan horizontalCoefficients = GetCoefficients(testCase.HorizontalFilter, testCase.HorizontalPhase, testCase.Width <= 4); + ReadOnlySpan verticalCoefficients = GetCoefficients(testCase.VerticalFilter, testCase.VerticalPhase, testCase.Height <= 4); + + ApplyTwoDimensionalReference( + source, + sourceStride, + sourceOrigin, + destination, + destinationOrigin, + destinationStride, + testCase, + horizontalCoefficients, + verticalCoefficients, + bitDepth); + } + + /// + /// Applies the reference decoder's biased two-pass 8-bit convolution and removes both intermediate bias terms after vertical filtering. + /// + /// The complete padded reference plane. + /// The source-row stride. + /// The integer-position sample index. + /// The guarded destination storage. + /// The first active destination index. + /// The destination-row stride. + /// The prediction geometry and phases. + /// The horizontal Q7 coefficient row. + /// The vertical Q7 coefficient row. + /// The decoded sample precision. + private static void ApplyTwoDimensionalReference( + byte[] source, + int sourceStride, + int sourceOrigin, + byte[] destination, + int destinationOrigin, + int destinationStride, + PredictionCase testCase, + ReadOnlySpan horizontalCoefficients, + ReadOnlySpan verticalCoefficients, + int bitDepth) + { + short[] intermediate = new short[(testCase.Height + FilterTapCount - 1) * testCase.Width]; + int horizontalBias = 1 << (bitDepth + FilterBits - 1); + int round0 = GetRound0Bits(bitDepth); + + for (int row = 0; row < testCase.Height + FilterTapCount - 1; row++) + { + for (int column = 0; column < testCase.Width; column++) + { + int sourceIndex = sourceOrigin + ((row - FilterCenterOffset) * sourceStride) + column - FilterCenterOffset; + int sum = horizontalBias + Convolve(source, sourceIndex, 1, horizontalCoefficients); + intermediate[(row * testCase.Width) + column] = (short)RoundPowerOfTwo(sum, round0); + } + } + + WriteTwoDimensionalReference(intermediate, destination, destinationOrigin, destinationStride, testCase, verticalCoefficients, bitDepth); + } + + /// + /// Applies the reference decoder's biased two-pass high-bit-depth convolution and removes both intermediate bias terms after vertical filtering. + /// + /// The complete padded reference plane. + /// The source-row stride. + /// The integer-position sample index. + /// The guarded destination storage. + /// The first active destination index. + /// The destination-row stride. + /// The prediction geometry and phases. + /// The horizontal Q7 coefficient row. + /// The vertical Q7 coefficient row. + /// The decoded sample precision. + private static void ApplyTwoDimensionalReference( + ushort[] source, + int sourceStride, + int sourceOrigin, + ushort[] destination, + int destinationOrigin, + int destinationStride, + PredictionCase testCase, + ReadOnlySpan horizontalCoefficients, + ReadOnlySpan verticalCoefficients, + int bitDepth) + { + short[] intermediate = new short[(testCase.Height + FilterTapCount - 1) * testCase.Width]; + int horizontalBias = 1 << (bitDepth + FilterBits - 1); + int round0 = GetRound0Bits(bitDepth); + + for (int row = 0; row < testCase.Height + FilterTapCount - 1; row++) + { + for (int column = 0; column < testCase.Width; column++) + { + int sourceIndex = sourceOrigin + ((row - FilterCenterOffset) * sourceStride) + column - FilterCenterOffset; + int sum = horizontalBias + Convolve(source, sourceIndex, 1, horizontalCoefficients); + intermediate[(row * testCase.Width) + column] = (short)RoundPowerOfTwo(sum, round0); + } + } + + WriteTwoDimensionalReference(intermediate, destination, destinationOrigin, destinationStride, testCase, verticalCoefficients, bitDepth); + } + + /// + /// Completes an 8-bit two-dimensional prediction from the independently generated biased intermediate block. + /// + /// The horizontally filtered signed intermediate block. + /// The guarded destination storage. + /// The first active destination index. + /// The destination-row stride. + /// The prediction geometry. + /// The vertical Q7 coefficient row. + /// The decoded sample precision. + private static void WriteTwoDimensionalReference( + short[] intermediate, + byte[] destination, + int destinationOrigin, + int destinationStride, + PredictionCase testCase, + ReadOnlySpan verticalCoefficients, + int bitDepth) + { + int round0 = GetRound0Bits(bitDepth); + int round1 = (2 * FilterBits) - round0; + int offsetBits = bitDepth + (2 * FilterBits) - round0; + int verticalBias = 1 << offsetBits; + int roundOffset = (1 << (offsetBits - round1)) + (1 << (offsetBits - round1 - 1)); + + for (int row = 0; row < testCase.Height; row++) + { + for (int column = 0; column < testCase.Width; column++) + { + int sum = verticalBias + Convolve(intermediate, (row * testCase.Width) + column, testCase.Width, verticalCoefficients); + int value = RoundPowerOfTwo(sum, round1) - roundOffset; + destination[destinationOrigin + (row * destinationStride) + column] = (byte)Math.Clamp(value, 0, byte.MaxValue); + } + } + } + + /// + /// Completes a high-bit-depth two-dimensional prediction from the independently generated biased intermediate block. + /// + /// The horizontally filtered signed intermediate block. + /// The guarded destination storage. + /// The first active destination index. + /// The destination-row stride. + /// The prediction geometry. + /// The vertical Q7 coefficient row. + /// The decoded sample precision. + private static void WriteTwoDimensionalReference( + short[] intermediate, + ushort[] destination, + int destinationOrigin, + int destinationStride, + PredictionCase testCase, + ReadOnlySpan verticalCoefficients, + int bitDepth) + { + int round0 = GetRound0Bits(bitDepth); + int round1 = (2 * FilterBits) - round0; + int offsetBits = bitDepth + (2 * FilterBits) - round0; + int verticalBias = 1 << offsetBits; + int roundOffset = (1 << (offsetBits - round1)) + (1 << (offsetBits - round1 - 1)); + int maximum = (1 << bitDepth) - 1; + + for (int row = 0; row < testCase.Height; row++) + { + for (int column = 0; column < testCase.Width; column++) + { + int sum = verticalBias + Convolve(intermediate, (row * testCase.Width) + column, testCase.Width, verticalCoefficients); + int value = RoundPowerOfTwo(sum, round1) - roundOffset; + destination[destinationOrigin + (row * destinationStride) + column] = (ushort)Math.Clamp(value, 0, maximum); + } + } + } + + /// + /// Computes one eight-tap Q7 convolution from 8-bit samples. + /// + /// The complete source storage. + /// The first coefficient's source index. + /// The source-element distance between taps. + /// The eight Q7 coefficients. + /// The unrounded convolution sum. + private static int Convolve(byte[] source, int sourceIndex, int sourceStep, ReadOnlySpan coefficients) + { + int sum = 0; + for (int tap = 0; tap < FilterTapCount; tap++) + { + sum += source[sourceIndex + (tap * sourceStep)] * coefficients[tap]; + } + + return sum; + } + + /// + /// Computes one eight-tap Q7 convolution from high-bit-depth samples. + /// + /// The complete source storage. + /// The first coefficient's source index. + /// The source-element distance between taps. + /// The eight Q7 coefficients. + /// The unrounded convolution sum. + private static int Convolve(ushort[] source, int sourceIndex, int sourceStep, ReadOnlySpan coefficients) + { + int sum = 0; + for (int tap = 0; tap < FilterTapCount; tap++) + { + sum += source[sourceIndex + (tap * sourceStep)] * coefficients[tap]; + } + + return sum; + } + + /// + /// Computes one eight-tap Q7 convolution from signed biased intermediate samples. + /// + /// The complete intermediate storage. + /// The first coefficient's source index. + /// The source-element distance between taps. + /// The eight Q7 coefficients. + /// The unrounded convolution sum. + private static int Convolve(short[] source, int sourceIndex, int sourceStep, ReadOnlySpan coefficients) + { + int sum = 0; + for (int tap = 0; tap < FilterTapCount; tap++) + { + sum += source[sourceIndex + (tap * sourceStep)] * coefficients[tap]; + } + + return sum; + } + + /// + /// Selects one normative Q7 coefficient row independently of the production filter storage. + /// + /// The interpolation-filter family. + /// The one-sixteenth-sample phase. + /// A value indicating whether the dimension is four samples. + /// The eight-position Q7 coefficient row. + private static ReadOnlySpan GetCoefficients(Av1InterpolationFilter filter, int phase, bool useReducedFilter) + { + if (filter == Av1InterpolationFilter.Bilinear) + { + return phase switch + { + 3 => BilinearPhase3, + 8 => BilinearPhase8, + 11 => BilinearPhase11, + 15 => BilinearPhase15, + _ => throw new InvalidOperationException($"The test oracle has no bilinear coefficient row for phase {phase}.") + }; + } + + if (useReducedFilter && filter == Av1InterpolationFilter.Sharp) + { + // AV1 maps sharp filtering on a four-sample dimension to the reduced regular table before convolution. + filter = Av1InterpolationFilter.Regular; + } + + return (filter, useReducedFilter, phase) switch + { + (Av1InterpolationFilter.Regular, false, 1) => RegularEightTapPhase1, + (Av1InterpolationFilter.Smooth, false, 7) => SmoothEightTapPhase7, + (Av1InterpolationFilter.Smooth, false, 15) => SmoothEightTapPhase15, + (Av1InterpolationFilter.Sharp, false, 8) => SharpEightTapPhase8, + (Av1InterpolationFilter.Regular, true, 3) => RegularFourTapPhase3, + (Av1InterpolationFilter.Regular, true, 5) => RegularFourTapPhase5, + (Av1InterpolationFilter.Regular, true, 13) => RegularFourTapPhase13, + (Av1InterpolationFilter.Smooth, true, 7) => SmoothFourTapPhase7, + (Av1InterpolationFilter.Smooth, true, 13) => SmoothFourTapPhase13, + _ => throw new InvalidOperationException( + $"The test oracle has no coefficient row for {filter}, phase {phase}, reduced {useReducedFilter}.") + }; + } + + /// + /// Gets the regular eight-tap Q7 kernel for phase 1 from AOM's normative decoder table. + /// + private static ReadOnlySpan RegularEightTapPhase1 => [0, 2, -6, 126, 8, -2, 0, 0]; + + /// + /// Gets the smooth eight-tap Q7 kernel for phase 7 from AOM's normative decoder table. + /// + private static ReadOnlySpan SmoothEightTapPhase7 => [0, -2, 16, 54, 48, 12, 0, 0]; + + /// + /// Gets the smooth eight-tap Q7 kernel for phase 15 from AOM's normative decoder table. + /// + private static ReadOnlySpan SmoothEightTapPhase15 => [0, 0, 2, 34, 62, 28, 2, 0]; + + /// + /// Gets the sharp eight-tap Q7 kernel for phase 8 from AOM's normative decoder table. + /// + private static ReadOnlySpan SharpEightTapPhase8 => [-4, 12, -24, 80, 80, -24, 12, -4]; + + /// + /// Gets the bilinear Q7 kernel for phase 3 from AOM's normative decoder table. + /// + private static ReadOnlySpan BilinearPhase3 => [0, 0, 0, 104, 24, 0, 0, 0]; + + /// + /// Gets the bilinear Q7 kernel for phase 8 from AOM's normative decoder table. + /// + private static ReadOnlySpan BilinearPhase8 => [0, 0, 0, 64, 64, 0, 0, 0]; + + /// + /// Gets the bilinear Q7 kernel for phase 11 from AOM's normative decoder table. + /// + private static ReadOnlySpan BilinearPhase11 => [0, 0, 0, 40, 88, 0, 0, 0]; + + /// + /// Gets the bilinear Q7 kernel for phase 15 from AOM's normative decoder table. + /// + private static ReadOnlySpan BilinearPhase15 => [0, 0, 0, 8, 120, 0, 0, 0]; + + /// + /// Gets the reduced regular Q7 kernel for phase 3 from AOM's normative decoder table. + /// + private static ReadOnlySpan RegularFourTapPhase3 => [0, 0, -10, 116, 28, -6, 0, 0]; + + /// + /// Gets the reduced regular Q7 kernel for phase 5 from AOM's normative decoder table. + /// + private static ReadOnlySpan RegularFourTapPhase5 => [0, 0, -12, 102, 48, -10, 0, 0]; + + /// + /// Gets the reduced regular Q7 kernel for phase 13 from AOM's normative decoder table. + /// + private static ReadOnlySpan RegularFourTapPhase13 => [0, 0, -6, 28, 116, -10, 0, 0]; + + /// + /// Gets the reduced smooth Q7 kernel for phase 7 from AOM's normative decoder table. + /// + private static ReadOnlySpan SmoothFourTapPhase7 => [0, 0, 14, 54, 48, 12, 0, 0]; + + /// + /// Gets the reduced smooth Q7 kernel for phase 13 from AOM's normative decoder table. + /// + private static ReadOnlySpan SmoothFourTapPhase13 => [0, 0, 4, 40, 62, 22, 0, 0]; + + /// + /// Gets AOM's first convolution shift while keeping the biased intermediate within sixteen signed bits. + /// + /// The decoded sample precision. + /// The first convolution shift. + private static int GetRound0Bits(int bitDepth) + { + int round0 = Round0Bits; + int intermediateBitCount = bitDepth + FilterBits - round0 + 2; + if (intermediateBitCount > 16) + { + round0 += intermediateBitCount - 16; + } + + return round0; + } + + /// + /// Applies AOM's integer power-of-two rounding rule. + /// + /// The signed integer to divide. + /// The base-2 divisor exponent. + /// The rounded quotient. + private static int RoundPowerOfTwo(int value, int bits) => (value + (1 << (bits - 1))) >> bits; + + /// + /// Reports the first differing byte, including guarded padding, for one named prediction path. + /// + /// The independently generated destination storage. + /// The production destination storage. + /// The prediction scenario. + /// The production execution path. + private static void AssertEqual(byte[] expected, byte[] actual, PredictionCase testCase, string path) + { + for (int i = 0; i < expected.Length; i++) + { + if (expected[i] != actual[i]) + { + Assert.Fail($"{path} prediction '{testCase.Name}' differs at storage index {i}: expected {expected[i]}, actual {actual[i]}."); + } + } + } + + /// + /// Reports the first differing ushort, including guarded padding, for one named prediction path. + /// + /// The independently generated destination storage. + /// The production destination storage. + /// The prediction scenario. + /// The production execution path. + private static void AssertEqual(ushort[] expected, ushort[] actual, PredictionCase testCase, string path) + { + for (int i = 0; i < expected.Length; i++) + { + if (expected[i] != actual[i]) + { + Assert.Fail($"{path} prediction '{testCase.Name}' differs at storage index {i}: expected {expected[i]}, actual {actual[i]}."); + } + } + } + + /// + /// Describes one prediction path, filter pair, phase pair, and block geometry. + /// + private readonly struct PredictionCase + { + /// + /// Initializes a new instance of the struct. + /// + /// The diagnostic scenario name. + /// The active prediction width. + /// The active prediction height. + /// The horizontal interpolation-filter family. + /// The vertical interpolation-filter family. + /// The horizontal one-sixteenth-sample phase. + /// The vertical one-sixteenth-sample phase. + public PredictionCase( + string name, + int width, + int height, + Av1InterpolationFilter horizontalFilter, + Av1InterpolationFilter verticalFilter, + int horizontalPhase, + int verticalPhase) + { + this.Name = name; + this.Width = width; + this.Height = height; + this.HorizontalFilter = horizontalFilter; + this.VerticalFilter = verticalFilter; + this.HorizontalPhase = horizontalPhase; + this.VerticalPhase = verticalPhase; + } + + /// + /// Gets the diagnostic scenario name. + /// + public string Name { get; } + + /// + /// Gets the active prediction width. + /// + public int Width { get; } + + /// + /// Gets the active prediction height. + /// + public int Height { get; } + + /// + /// Gets the horizontal interpolation-filter family. + /// + public Av1InterpolationFilter HorizontalFilter { get; } + + /// + /// Gets the vertical interpolation-filter family. + /// + public Av1InterpolationFilter VerticalFilter { get; } + + /// + /// Gets the horizontal one-sixteenth-sample phase. + /// + public int HorizontalPhase { get; } + + /// + /// Gets the vertical one-sixteenth-sample phase. + /// + public int VerticalPhase { get; } + } +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1WarpedInterPredictorTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1WarpedInterPredictorTests.cs new file mode 100644 index 0000000000..d251b72650 --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1WarpedInterPredictorTests.cs @@ -0,0 +1,610 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Numerics; +using SixLabors.ImageSharp.Formats.Heif.Av1.Motion; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; +using SixLabors.ImageSharp.Tests.TestUtilities; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1; + +/// +/// Verifies affine warped-motion prediction through native SIMD and scalar execution. +/// +[Trait("Format", "Avif")] +public class Av1WarpedInterPredictorTests +{ + /// + /// The hardware configurations covering every descending SIMD width and the scalar fallback. + /// + private const HwIntrinsics PredictorConfigurations = + HwIntrinsics.AllowAll | HwIntrinsics.DisableAVX512F | HwIntrinsics.DisableAVX | HwIntrinsics.DisableHWIntrinsic; + + /// + /// Verifies exact 8-bit native and compound prediction against independent scalar equations. + /// + [Fact] + public void BytePredictionMatchesReference() + => FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidateBytePrediction, PredictorConfigurations); + + /// + /// Verifies exact 8-, 10-, and 12-bit native and compound prediction against independent scalar equations. + /// + [Fact] + public void HighBitDepthPredictionMatchesReference() + => FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidateHighBitDepthPrediction, PredictorConfigurations); + + /// + /// Applies the reference multi-sample affine model to deterministic byte storage. + /// + private static void ValidateBytePrediction() + { + const int activeSize = 96; + const int padding = 24; + const int sourceStride = activeSize + (2 * padding); + const int width = 13; + const int height = 11; + const int destinationStride = width + 7; + byte[] source = new byte[sourceStride * sourceStride]; + for (int row = 0; row < sourceStride; row++) + { + for (int column = 0; column < sourceStride; column++) + { + source[(row * sourceStride) + column] = (byte)(((row * 29) + (column * 47) + (row * column * 3)) & byte.MaxValue); + } + } + + Av1GlobalMotionParameters parameters = CreateReferenceParameters(); + for (int subsampling = 0; subsampling <= 1; subsampling++) + { + byte[] expected = new byte[destinationStride * height]; + byte[] actual = new byte[destinationStride * height]; + Array.Fill(expected, (byte)0xD3); + Array.Fill(actual, (byte)0xD3); + short[] actualScratch = new short[Av1WarpedInterPredictor.WarpedScratchLength]; + Point destinationPosition = subsampling == 0 ? new Point(32, 24) : new Point(16, 12); + + PredictReference( + source, + sourceStride, + new Point(padding, padding), + activeSize, + activeSize, + expected, + destinationStride, + destinationPosition, + width, + height, + subsampling, + subsampling, + 8, + parameters, + compound: false); + + Av1WarpedInterPredictor.PredictWarped( + source, + sourceStride, + new Point(padding, padding), + activeSize, + activeSize, + actual, + destinationStride, + destinationPosition, + width, + height, + subsampling, + subsampling, + parameters, + actualScratch); + + Assert.Equal(expected, actual); + + ushort[] expectedCompound = new ushort[destinationStride * height]; + ushort[] actualCompound = new ushort[destinationStride * height]; + Array.Fill(expectedCompound, (ushort)0xDEAD); + Array.Fill(actualCompound, (ushort)0xDEAD); + + PredictReference( + source, + sourceStride, + new Point(padding, padding), + activeSize, + activeSize, + expectedCompound, + destinationStride, + destinationPosition, + width, + height, + subsampling, + subsampling, + 8, + parameters, + compound: true); + + Av1WarpedInterPredictor.PredictWarpedCompound( + source, + sourceStride, + new Point(padding, padding), + activeSize, + activeSize, + actualCompound, + destinationStride, + destinationPosition, + width, + height, + subsampling, + subsampling, + parameters, + actualScratch); + + Assert.Equal(expectedCompound, actualCompound); + } + } + + /// + /// Applies the multi-sample affine model to every supported high-bit-depth precision. + /// + private static void ValidateHighBitDepthPrediction() + { + const int activeSize = 96; + const int padding = 24; + const int sourceStride = activeSize + (2 * padding); + const int width = 13; + const int height = 11; + const int destinationStride = width + 7; + Av1GlobalMotionParameters parameters = CreateReferenceParameters(); + foreach (int bitDepth in new[] { 8, 10, 12 }) + { + int maximum = (1 << bitDepth) - 1; + ushort[] source = new ushort[sourceStride * sourceStride]; + for (int row = 0; row < sourceStride; row++) + { + for (int column = 0; column < sourceStride; column++) + { + source[(row * sourceStride) + column] = + (ushort)(((row * 269) + (column * 443) + (row * column * 31)) & maximum); + } + } + + for (int subsampling = 0; subsampling <= 1; subsampling++) + { + ushort[] expected = new ushort[destinationStride * height]; + ushort[] actual = new ushort[destinationStride * height]; + Array.Fill(expected, (ushort)0xDEAD); + Array.Fill(actual, (ushort)0xDEAD); + short[] actualScratch = new short[Av1WarpedInterPredictor.WarpedScratchLength]; + Point destinationPosition = subsampling == 0 ? new Point(32, 24) : new Point(16, 12); + + PredictReference( + source, + sourceStride, + new Point(padding, padding), + activeSize, + activeSize, + expected, + destinationStride, + destinationPosition, + width, + height, + subsampling, + subsampling, + bitDepth, + parameters, + compound: false); + + Av1WarpedInterPredictor.PredictWarped( + source, + sourceStride, + new Point(padding, padding), + activeSize, + activeSize, + actual, + destinationStride, + destinationPosition, + width, + height, + subsampling, + subsampling, + bitDepth, + parameters, + actualScratch); + + Assert.Equal(expected, actual); + + ushort[] expectedCompound = new ushort[destinationStride * height]; + ushort[] actualCompound = new ushort[destinationStride * height]; + Array.Fill(expectedCompound, (ushort)0xDEAD); + Array.Fill(actualCompound, (ushort)0xDEAD); + + PredictReference( + source, + sourceStride, + new Point(padding, padding), + activeSize, + activeSize, + expectedCompound, + destinationStride, + destinationPosition, + width, + height, + subsampling, + subsampling, + bitDepth, + parameters, + compound: true); + + Av1WarpedInterPredictor.PredictWarpedCompound( + source, + sourceStride, + new Point(padding, padding), + activeSize, + activeSize, + actualCompound, + destinationStride, + destinationPosition, + width, + height, + subsampling, + subsampling, + bitDepth, + parameters, + actualScratch); + + Assert.Equal(expectedCompound, actualCompound); + } + } + } + + /// + /// Reconstructs one warped block with scalar affine loops. + /// + private static void PredictReference( + ReadOnlySpan source, + int sourceStride, + Point sourceOrigin, + int sourceWidth, + int sourceHeight, + Span destination, + int destinationStride, + Point destinationPosition, + int width, + int height, + int subsamplingX, + int subsamplingY, + int bitDepth, + Av1GlobalMotionParameters parameters, + bool compound) + where TSource : unmanaged, IBinaryInteger + where TDestination : unmanaged, IBinaryInteger + { + const int filterBits = 7; + const int filterTaps = 8; + const int tileSize = 8; + const int modelPrecisionBits = 16; + const int phasePrecisionBits = 10; + const int pixelPrecisionShifts = 64; + Span intermediate = stackalloc int[15 * tileSize]; + + // The reference decoder raises round0 by two for 12-bit sources so the biased horizontal intermediate + // remains representable in 16 bits, then removes those two bits from the vertical rounding. + int intermediateRange = bitDepth + filterBits - 3 + 2; + int round0 = 3 + Math.Max(intermediateRange - 16, 0); + int verticalRound = compound ? 7 : (2 * filterBits) - round0; + int horizontalBias = 1 << (bitDepth + filterBits - 1); + int verticalBias = 1 << (bitDepth + (2 * filterBits) - round0); + int maximum = (1 << bitDepth) - 1; + + for (int tileRow = destinationPosition.Y; tileRow < destinationPosition.Y + height; tileRow += tileSize) + { + for (int tileColumn = destinationPosition.X; tileColumn < destinationPosition.X + width; tileColumn += tileSize) + { + int centerX = (tileColumn + 4) << subsamplingX; + int centerY = (tileRow + 4) << subsamplingY; + long projectedX = + ((long)parameters[2] * centerX) + ((long)parameters[3] * centerY) + parameters[0]; + + long projectedY = + ((long)parameters[4] * centerX) + ((long)parameters[5] * centerY) + parameters[1]; + + long planeX = projectedX >> subsamplingX; + long planeY = projectedY >> subsamplingY; + int integerX = (int)(planeX >> modelPrecisionBits); + int integerY = (int)(planeY >> modelPrecisionBits); + int phaseX = (int)planeX & ((1 << modelPrecisionBits) - 1); + int phaseY = (int)planeY & ((1 << modelPrecisionBits) - 1); + phaseX += (-4 * parameters.Alpha) + (-4 * parameters.Beta); + phaseY += (-4 * parameters.Gamma) + (-4 * parameters.Delta); + phaseX &= -1 << 6; + phaseY &= -1 << 6; + + for (int row = -7; row < 8; row++) + { + int sourceY = Math.Clamp(integerY + row, 0, sourceHeight - 1); + int phase = phaseX + (parameters.Beta * (row + 4)); + for (int column = -4; column < 4; column++) + { + int sourceX = integerX + column - 3; + int filterIndex = RoundPowerOfTwo(phase, phasePrecisionBits) + pixelPrecisionShifts; + int coefficientOffset = filterIndex * filterTaps; + int sum = horizontalBias; + for (int tap = 0; tap < filterTaps; tap++) + { + int sampleX = Math.Clamp(sourceX + tap, 0, sourceWidth - 1); + int sourceIndex = + ((sourceOrigin.Y + sourceY) * sourceStride) + sourceOrigin.X + sampleX; + + int sample = int.CreateChecked(source[sourceIndex]); + sum += sample * ReferenceWarpedFilter[coefficientOffset + tap]; + } + + intermediate[((row + 7) * tileSize) + column + 4] = RoundPowerOfTwo(sum, round0); + phase += parameters.Alpha; + } + } + + int lastRow = Math.Min(4, destinationPosition.Y + height - tileRow - 4); + for (int row = -4; row < lastRow; row++) + { + int phase = phaseY + (parameters.Delta * (row + 4)); + int lastColumn = Math.Min(4, destinationPosition.X + width - tileColumn - 4); + for (int column = -4; column < lastColumn; column++) + { + int filterIndex = RoundPowerOfTwo(phase, phasePrecisionBits) + pixelPrecisionShifts; + int coefficientOffset = filterIndex * filterTaps; + int sum = verticalBias; + for (int tap = 0; tap < filterTaps; tap++) + { + int intermediateIndex = ((row + tap + 4) * tileSize) + column + 4; + sum += intermediate[intermediateIndex] * ReferenceWarpedFilter[coefficientOffset + tap]; + } + + int value = RoundPowerOfTwo(sum, verticalRound); + if (!compound) + { + value -= (1 << (bitDepth - 1)) + (1 << bitDepth); + value = Math.Clamp(value, 0, maximum); + } + + int destinationIndex = + ((tileRow - destinationPosition.Y + row + 4) * destinationStride) + + tileColumn - + destinationPosition.X + + column + + 4; + + destination[destinationIndex] = TDestination.CreateChecked(value); + phase += parameters.Gamma; + } + } + } + } + } + + /// + /// Gets the 193 reference eight-tap warped-filter phases used by the independent scalar oracle. + /// + private static ReadOnlySpan ReferenceWarpedFilter => + [ + 0, 0, 127, 1, 0, 0, 0, 0, + 0, -1, 127, 2, 0, 0, 0, 0, + 1, -3, 127, 4, -1, 0, 0, 0, + 1, -4, 126, 6, -2, 1, 0, 0, + 1, -5, 126, 8, -3, 1, 0, 0, + 1, -6, 125, 11, -4, 1, 0, 0, + 1, -7, 124, 13, -4, 1, 0, 0, + 2, -8, 123, 15, -5, 1, 0, 0, + 2, -9, 122, 18, -6, 1, 0, 0, + 2, -10, 121, 20, -6, 1, 0, 0, + 2, -11, 120, 22, -7, 2, 0, 0, + 2, -12, 119, 25, -8, 2, 0, 0, + 3, -13, 117, 27, -8, 2, 0, 0, + 3, -13, 116, 29, -9, 2, 0, 0, + 3, -14, 114, 32, -10, 3, 0, 0, + 3, -15, 113, 35, -10, 2, 0, 0, + 3, -15, 111, 37, -11, 3, 0, 0, + 3, -16, 109, 40, -11, 3, 0, 0, + 3, -16, 108, 42, -12, 3, 0, 0, + 4, -17, 106, 45, -13, 3, 0, 0, + 4, -17, 104, 47, -13, 3, 0, 0, + 4, -17, 102, 50, -14, 3, 0, 0, + 4, -17, 100, 52, -14, 3, 0, 0, + 4, -18, 98, 55, -15, 4, 0, 0, + 4, -18, 96, 58, -15, 3, 0, 0, + 4, -18, 94, 60, -16, 4, 0, 0, + 4, -18, 91, 63, -16, 4, 0, 0, + 4, -18, 89, 65, -16, 4, 0, 0, + 4, -18, 87, 68, -17, 4, 0, 0, + 4, -18, 85, 70, -17, 4, 0, 0, + 4, -18, 82, 73, -17, 4, 0, 0, + 4, -18, 80, 75, -17, 4, 0, 0, + 4, -18, 78, 78, -18, 4, 0, 0, + 4, -17, 75, 80, -18, 4, 0, 0, + 4, -17, 73, 82, -18, 4, 0, 0, + 4, -17, 70, 85, -18, 4, 0, 0, + 4, -17, 68, 87, -18, 4, 0, 0, + 4, -16, 65, 89, -18, 4, 0, 0, + 4, -16, 63, 91, -18, 4, 0, 0, + 4, -16, 60, 94, -18, 4, 0, 0, + 3, -15, 58, 96, -18, 4, 0, 0, + 4, -15, 55, 98, -18, 4, 0, 0, + 3, -14, 52, 100, -17, 4, 0, 0, + 3, -14, 50, 102, -17, 4, 0, 0, + 3, -13, 47, 104, -17, 4, 0, 0, + 3, -13, 45, 106, -17, 4, 0, 0, + 3, -12, 42, 108, -16, 3, 0, 0, + 3, -11, 40, 109, -16, 3, 0, 0, + 3, -11, 37, 111, -15, 3, 0, 0, + 2, -10, 35, 113, -15, 3, 0, 0, + 3, -10, 32, 114, -14, 3, 0, 0, + 2, -9, 29, 116, -13, 3, 0, 0, + 2, -8, 27, 117, -13, 3, 0, 0, + 2, -8, 25, 119, -12, 2, 0, 0, + 2, -7, 22, 120, -11, 2, 0, 0, + 1, -6, 20, 121, -10, 2, 0, 0, + 1, -6, 18, 122, -9, 2, 0, 0, + 1, -5, 15, 123, -8, 2, 0, 0, + 1, -4, 13, 124, -7, 1, 0, 0, + 1, -4, 11, 125, -6, 1, 0, 0, + 1, -3, 8, 126, -5, 1, 0, 0, + 1, -2, 6, 126, -4, 1, 0, 0, + 0, -1, 4, 127, -3, 1, 0, 0, + 0, 0, 2, 127, -1, 0, 0, 0, + 0, 0, 0, 127, 1, 0, 0, 0, + 0, 0, -1, 127, 2, 0, 0, 0, + 0, 1, -3, 127, 4, -2, 1, 0, + 0, 1, -5, 127, 6, -2, 1, 0, + 0, 2, -6, 126, 8, -3, 1, 0, + -1, 2, -7, 126, 11, -4, 2, -1, + -1, 3, -8, 125, 13, -5, 2, -1, + -1, 3, -10, 124, 16, -6, 3, -1, + -1, 4, -11, 123, 18, -7, 3, -1, + -1, 4, -12, 122, 20, -7, 3, -1, + -1, 4, -13, 121, 23, -8, 3, -1, + -2, 5, -14, 120, 25, -9, 4, -1, + -1, 5, -15, 119, 27, -10, 4, -1, + -1, 5, -16, 118, 30, -11, 4, -1, + -2, 6, -17, 116, 33, -12, 5, -1, + -2, 6, -17, 114, 35, -12, 5, -1, + -2, 6, -18, 113, 38, -13, 5, -1, + -2, 7, -19, 111, 41, -14, 6, -2, + -2, 7, -19, 110, 43, -15, 6, -2, + -2, 7, -20, 108, 46, -15, 6, -2, + -2, 7, -20, 106, 49, -16, 6, -2, + -2, 7, -21, 104, 51, -16, 7, -2, + -2, 7, -21, 102, 54, -17, 7, -2, + -2, 8, -21, 100, 56, -18, 7, -2, + -2, 8, -22, 98, 59, -18, 7, -2, + -2, 8, -22, 96, 62, -19, 7, -2, + -2, 8, -22, 94, 64, -19, 7, -2, + -2, 8, -22, 91, 67, -20, 8, -2, + -2, 8, -22, 89, 69, -20, 8, -2, + -2, 8, -22, 87, 72, -21, 8, -2, + -2, 8, -21, 84, 74, -21, 8, -2, + -2, 8, -22, 82, 77, -21, 8, -2, + -2, 8, -21, 79, 79, -21, 8, -2, + -2, 8, -21, 77, 82, -22, 8, -2, + -2, 8, -21, 74, 84, -21, 8, -2, + -2, 8, -21, 72, 87, -22, 8, -2, + -2, 8, -20, 69, 89, -22, 8, -2, + -2, 8, -20, 67, 91, -22, 8, -2, + -2, 7, -19, 64, 94, -22, 8, -2, + -2, 7, -19, 62, 96, -22, 8, -2, + -2, 7, -18, 59, 98, -22, 8, -2, + -2, 7, -18, 56, 100, -21, 8, -2, + -2, 7, -17, 54, 102, -21, 7, -2, + -2, 7, -16, 51, 104, -21, 7, -2, + -2, 6, -16, 49, 106, -20, 7, -2, + -2, 6, -15, 46, 108, -20, 7, -2, + -2, 6, -15, 43, 110, -19, 7, -2, + -2, 6, -14, 41, 111, -19, 7, -2, + -1, 5, -13, 38, 113, -18, 6, -2, + -1, 5, -12, 35, 114, -17, 6, -2, + -1, 5, -12, 33, 116, -17, 6, -2, + -1, 4, -11, 30, 118, -16, 5, -1, + -1, 4, -10, 27, 119, -15, 5, -1, + -1, 4, -9, 25, 120, -14, 5, -2, + -1, 3, -8, 23, 121, -13, 4, -1, + -1, 3, -7, 20, 122, -12, 4, -1, + -1, 3, -7, 18, 123, -11, 4, -1, + -1, 3, -6, 16, 124, -10, 3, -1, + -1, 2, -5, 13, 125, -8, 3, -1, + -1, 2, -4, 11, 126, -7, 2, -1, + 0, 1, -3, 8, 126, -6, 2, 0, + 0, 1, -2, 6, 127, -5, 1, 0, + 0, 1, -2, 4, 127, -3, 1, 0, + 0, 0, 0, 2, 127, -1, 0, 0, + 0, 0, 0, 1, 127, 0, 0, 0, + 0, 0, 0, -1, 127, 2, 0, 0, + 0, 0, 1, -3, 127, 4, -1, 0, + 0, 0, 1, -4, 126, 6, -2, 1, + 0, 0, 1, -5, 126, 8, -3, 1, + 0, 0, 1, -6, 125, 11, -4, 1, + 0, 0, 1, -7, 124, 13, -4, 1, + 0, 0, 2, -8, 123, 15, -5, 1, + 0, 0, 2, -9, 122, 18, -6, 1, + 0, 0, 2, -10, 121, 20, -6, 1, + 0, 0, 2, -11, 120, 22, -7, 2, + 0, 0, 2, -12, 119, 25, -8, 2, + 0, 0, 3, -13, 117, 27, -8, 2, + 0, 0, 3, -13, 116, 29, -9, 2, + 0, 0, 3, -14, 114, 32, -10, 3, + 0, 0, 3, -15, 113, 35, -10, 2, + 0, 0, 3, -15, 111, 37, -11, 3, + 0, 0, 3, -16, 109, 40, -11, 3, + 0, 0, 3, -16, 108, 42, -12, 3, + 0, 0, 4, -17, 106, 45, -13, 3, + 0, 0, 4, -17, 104, 47, -13, 3, + 0, 0, 4, -17, 102, 50, -14, 3, + 0, 0, 4, -17, 100, 52, -14, 3, + 0, 0, 4, -18, 98, 55, -15, 4, + 0, 0, 4, -18, 96, 58, -15, 3, + 0, 0, 4, -18, 94, 60, -16, 4, + 0, 0, 4, -18, 91, 63, -16, 4, + 0, 0, 4, -18, 89, 65, -16, 4, + 0, 0, 4, -18, 87, 68, -17, 4, + 0, 0, 4, -18, 85, 70, -17, 4, + 0, 0, 4, -18, 82, 73, -17, 4, + 0, 0, 4, -18, 80, 75, -17, 4, + 0, 0, 4, -18, 78, 78, -18, 4, + 0, 0, 4, -17, 75, 80, -18, 4, + 0, 0, 4, -17, 73, 82, -18, 4, + 0, 0, 4, -17, 70, 85, -18, 4, + 0, 0, 4, -17, 68, 87, -18, 4, + 0, 0, 4, -16, 65, 89, -18, 4, + 0, 0, 4, -16, 63, 91, -18, 4, + 0, 0, 4, -16, 60, 94, -18, 4, + 0, 0, 3, -15, 58, 96, -18, 4, + 0, 0, 4, -15, 55, 98, -18, 4, + 0, 0, 3, -14, 52, 100, -17, 4, + 0, 0, 3, -14, 50, 102, -17, 4, + 0, 0, 3, -13, 47, 104, -17, 4, + 0, 0, 3, -13, 45, 106, -17, 4, + 0, 0, 3, -12, 42, 108, -16, 3, + 0, 0, 3, -11, 40, 109, -16, 3, + 0, 0, 3, -11, 37, 111, -15, 3, + 0, 0, 2, -10, 35, 113, -15, 3, + 0, 0, 3, -10, 32, 114, -14, 3, + 0, 0, 2, -9, 29, 116, -13, 3, + 0, 0, 2, -8, 27, 117, -13, 3, + 0, 0, 2, -8, 25, 119, -12, 2, + 0, 0, 2, -7, 22, 120, -11, 2, + 0, 0, 1, -6, 20, 121, -10, 2, + 0, 0, 1, -6, 18, 122, -9, 2, + 0, 0, 1, -5, 15, 123, -8, 2, + 0, 0, 1, -4, 13, 124, -7, 1, + 0, 0, 1, -4, 11, 125, -6, 1, + 0, 0, 1, -3, 8, 126, -5, 1, + 0, 0, 1, -2, 6, 126, -4, 1, + 0, 0, 0, -1, 4, 127, -3, 1, + 0, 0, 0, 0, 2, 127, -1, 0, + 0, 0, 0, 0, 2, 127, -1, 0, + ]; + + /// + /// Divides a signed fixed-point value by a power of two with the reference decoder's rounding. + /// + private static int RoundPowerOfTwo(int value, int bitCount) + => (value + (1 << (bitCount - 1))) >> bitCount; + + /// + /// Creates one nontrivial affine model traced from the independently verified two-frame local-warp fixture. + /// + private static Av1GlobalMotionParameters CreateReferenceParameters() + { + Av1GlobalMotionParameters parameters = Av1GlobalMotionParameters.Identity; + parameters.Type = Av1GlobalMotionType.Affine; + parameters[0] = -191565; + parameters[1] = 599107; + parameters[2] = 61755; + parameters[3] = -140; + parameters[4] = -6909; + parameters[5] = 62012; + parameters.UpdateShearParameters(); + Assert.False(parameters.IsInvalid); + Assert.Equal(-3776, parameters.Alpha); + Assert.Equal(-128, parameters.Beta); + Assert.Equal(-7360, parameters.Gamma); + Assert.Equal(-3520, parameters.Delta); + return parameters; + } +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1YuvConverterTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1YuvConverterTests.cs new file mode 100644 index 0000000000..0f0be7062e --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1YuvConverterTests.cs @@ -0,0 +1,1386 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System; +using System.Numerics; +using SixLabors.ImageSharp.Formats.Heif.Av1; +using SixLabors.ImageSharp.Formats.Heif.Av1.Color; +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Memory; +using SixLabors.ImageSharp.PixelFormats; +using SixLabors.ImageSharp.Processing; +using SixLabors.ImageSharp.Tests.TestUtilities; +using SixLabors.ImageSharp.Tests.TestUtilities.ImageComparison; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1; + +/// +/// Verifies AV1 color conversion, sample-range handling, chroma reconstruction, and encoder downsampling. +/// +[Trait("Format", "Avif")] +public class Av1YuvConverterTests +{ + /// + /// The hardware configurations covering 512-bit, 256-bit, 128-bit, and scalar conversion paths. + /// + private const HwIntrinsics AlphaConfigurations = + HwIntrinsics.AllowAll | HwIntrinsics.DisableAVX512F | HwIntrinsics.DisableAVX | HwIntrinsics.DisableHWIntrinsic; + + /// + /// Verifies known RGB-to-YUV values across coefficient, identity, and YCgCo matrices and sample ranges. + /// + /// The source red component. + /// The source green component. + /// The source blue component. + /// The expected luma or first encoded component. + /// The expected first chroma or second encoded component. + /// The expected second chroma or third encoded component. + /// Whether the encoded samples use the full range. + /// The matrix coefficients used for conversion. + [Theory] + [InlineData(255, 255, 255, 255, 128, 128, true, ObuMatrixCoefficients.Bt709)] + [InlineData(0, 0, 0, 0, 128, 128, true, ObuMatrixCoefficients.Bt709)] + [InlineData(42, 42, 42, 42, 128, 128, true, ObuMatrixCoefficients.Bt709)] + [InlineData(150, 100, 50, 107, 97, 155, true, ObuMatrixCoefficients.Bt709)] + [InlineData(150, 100, 50, 100, 50, 150, true, ObuMatrixCoefficients.Identity)] + [InlineData(150, 100, 50, 110, 95, 157, true, ObuMatrixCoefficients.Fcc)] + [InlineData(150, 100, 50, 109, 95, 157, true, ObuMatrixCoefficients.Bt470BG)] + [InlineData(150, 100, 50, 109, 95, 157, true, ObuMatrixCoefficients.Bt601)] + [InlineData(150, 100, 50, 109, 95, 157, true, ObuMatrixCoefficients.Unspecified)] + [InlineData(150, 100, 50, 106, 97, 156, true, ObuMatrixCoefficients.Smpte240)] + [InlineData(150, 100, 50, 100, 128, 178, true, ObuMatrixCoefficients.SmpteYCgCo)] + [InlineData(150, 100, 50, 110, 96, 155, true, ObuMatrixCoefficients.Bt2020NonConstantLuminance)] + [InlineData(255, 255, 255, 235, 128, 128, false, ObuMatrixCoefficients.Bt709)] + [InlineData(0, 0, 0, 16, 128, 128, false, ObuMatrixCoefficients.Bt709)] + [InlineData(42, 42, 42, 52, 128, 128, false, ObuMatrixCoefficients.Bt709)] + [InlineData(150, 100, 50, 108, 101, 152, false, ObuMatrixCoefficients.Bt709)] + public void RgbToYuvSinglePixel(byte r, byte g, byte b, int y, int u, int v, bool fullRange, int matrixCoefficients) + { + // Assign + using Image image = new(1, 1); + ImageFrame frame = image.Frames.RootFrame; + frame.DangerousTryGetSinglePixelMemory(out Memory memory); + memory.Span[0] = new Rgb24(r, g, b); + ObuSequenceHeader sequenceHeader = CreateSequenceHeader(1, 1, fullRange, (ObuMatrixCoefficients)matrixCoefficients); + using Av1FrameBuffer frameBuffer = new(Configuration.Default, sequenceHeader, Av1ColorFormat.Yuv444, false); + + // Act + Av1YuvConverter.ConvertFromRgb(Configuration.Default, frame, frameBuffer); + + // Assert + byte actualY = frameBuffer.DeriveBlockPointer(Av1Plane.Y, 0, 0).DangerousGetRowSpan(0)[0]; + byte actualU = frameBuffer.DeriveBlockPointer(Av1Plane.U, 0, 0).DangerousGetRowSpan(0)[0]; + byte actualV = frameBuffer.DeriveBlockPointer(Av1Plane.V, 0, 0).DangerousGetRowSpan(0)[0]; + Assert.Equal(y, actualY); + Assert.Equal(u, actualU); + Assert.Equal(v, actualV); + } + + /// + /// Verifies known YUV-to-RGB values across coefficient, identity, and YCgCo matrices and sample ranges. + /// + /// The expected red component. + /// The expected green component. + /// The expected blue component. + /// The source luma or first encoded component. + /// The source first chroma or second encoded component. + /// The source second chroma or third encoded component. + /// Whether the encoded samples use the full range. + /// The matrix coefficients used for conversion. + [Theory] + [InlineData(255, 255, 255, 255, 128, 128, true, ObuMatrixCoefficients.Bt709)] + [InlineData(0, 0, 0, 0, 128, 128, true, ObuMatrixCoefficients.Bt709)] + [InlineData(42, 42, 42, 42, 128, 128, true, ObuMatrixCoefficients.Bt709)] + [InlineData(150, 100, 50, 107, 97, 155, true, ObuMatrixCoefficients.Bt709)] + [InlineData(150, 100, 50, 100, 50, 150, true, ObuMatrixCoefficients.Identity)] + [InlineData(150, 100, 50, 110, 95, 157, true, ObuMatrixCoefficients.Fcc)] + [InlineData(150, 100, 50, 109, 95, 157, true, ObuMatrixCoefficients.Bt470BG)] + [InlineData(150, 100, 50, 109, 95, 157, true, ObuMatrixCoefficients.Bt601)] + [InlineData(150, 100, 50, 109, 95, 157, true, ObuMatrixCoefficients.Unspecified)] + [InlineData(150, 100, 50, 106, 97, 156, true, ObuMatrixCoefficients.Smpte240)] + [InlineData(150, 100, 50, 100, 128, 178, true, ObuMatrixCoefficients.SmpteYCgCo)] + [InlineData(150, 100, 50, 110, 96, 155, true, ObuMatrixCoefficients.Bt2020NonConstantLuminance)] + [InlineData(255, 255, 255, 235, 128, 128, false, ObuMatrixCoefficients.Bt709)] + [InlineData(0, 0, 0, 16, 128, 128, false, ObuMatrixCoefficients.Bt709)] + [InlineData(42, 42, 42, 52, 128, 128, false, ObuMatrixCoefficients.Bt709)] + [InlineData(150, 100, 50, 108, 101, 152, false, ObuMatrixCoefficients.Bt709)] + public void YuvToRgbSinglePixel(byte r, byte g, byte b, int y, int u, int v, bool fullRange, int matrixCoefficients) + { + // Assign + using Image image = new(1, 1); + ImageFrame frame = image.Frames.RootFrame; + ObuSequenceHeader sequenceHeader = CreateSequenceHeader(1, 1, fullRange, (ObuMatrixCoefficients)matrixCoefficients); + using Av1FrameBuffer frameBuffer = new(Configuration.Default, sequenceHeader, Av1ColorFormat.Yuv444, false); + frameBuffer.DeriveBlockPointer(Av1Plane.Y, 0, 0).DangerousGetRowSpan(0)[0] = (byte)y; + frameBuffer.DeriveBlockPointer(Av1Plane.U, 0, 0).DangerousGetRowSpan(0)[0] = (byte)u; + frameBuffer.DeriveBlockPointer(Av1Plane.V, 0, 0).DangerousGetRowSpan(0)[0] = (byte)v; + + // Act + Av1YuvConverter.ConvertToRgb(Configuration.Default, frameBuffer, frame); + + // Assert + frame.DangerousTryGetSinglePixelMemory(out Memory memory); + Rgb24 actual = memory.Span[0]; + Assert.Equal(r, actual.R, 1d); + Assert.Equal(g, actual.G, 1d); + Assert.Equal(b, actual.B, 1d); + } + + /// + /// Verifies that limited-range monochrome samples expand to the complete RGB output range. + /// + [Fact] + public void Yuv400ToRgbExpandsLimitedRangeLuma() + { + // Assign + using Image image = new(2, 1); + ObuSequenceHeader sequenceHeader = CreateSequenceHeader( + 2, + 1, + false, + ObuMatrixCoefficients.Identity, + Av1ColorFormat.Yuv400); + + using Av1FrameBuffer frameBuffer = new(Configuration.Default, sequenceHeader, Av1ColorFormat.Yuv400, false); + Span yRow = frameBuffer.DeriveBlockPointer(Av1Plane.Y, 0, 0).DangerousGetRowSpan(0); + yRow[0] = 16; + yRow[1] = 235; + + // Act + Av1YuvConverter.ConvertToRgb(Configuration.Default, frameBuffer, image.Frames.RootFrame); + + // Assert + Span actual = image.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(0); + Assert.Equal(new Rgb24(0, 0, 0), actual[0]); + Assert.Equal(new Rgb24(255, 255, 255), actual[1]); + } + + /// + /// Verifies RGB-to-monochrome conversion and range quantization for every supported AV1 bit depth. + /// + /// The encoded AV1 bit depth. + /// Whether the luma samples use the full range. + /// The expected encoded luma sample. + [Theory] + [InlineData(Av1BitDepth.EightBit, true, 107)] + [InlineData(Av1BitDepth.EightBit, false, 108)] + [InlineData(Av1BitDepth.TenBit, true, 429)] + [InlineData(Av1BitDepth.TenBit, false, 432)] + [InlineData(Av1BitDepth.TwelveBit, true, 1719)] + [InlineData(Av1BitDepth.TwelveBit, false, 1727)] + public void RgbToYuv400WritesQuantizedLuma(int bitDepth, bool fullRange, int expectedLuma) + { + // Rgb48 values scaled from eight-bit components exercise the precision-preserving high-bit-depth path. + using Image image = new(1, 1); + image.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(0)[0] = new Rgb48(150 * 257, 100 * 257, 50 * 257); + ObuSequenceHeader sequenceHeader = CreateSequenceHeader( + 1, + 1, + fullRange, + colorFormat: Av1ColorFormat.Yuv400, + bitDepth: (Av1BitDepth)bitDepth); + + using Av1FrameBuffer frameBuffer = new(Configuration.Default, sequenceHeader, Av1ColorFormat.Yuv400, false); + + Av1YuvConverter.ConvertFromRgb(Configuration.Default, image.Frames.RootFrame, frameBuffer); + + Assert.Equal(expectedLuma, GetPlaneSample(frameBuffer, Av1Plane.Y, 0, 0, 0, 0)); + Assert.Null(frameBuffer.BufferCb); + Assert.Null(frameBuffer.BufferCr); + } + + /// + /// Verifies full- and limited-range expansion for 10-bit and 12-bit reconstructed samples. + /// + /// The reconstructed AV1 bit depth. + /// Whether the samples use the full range. + /// The encoded black luma sample. + /// The encoded white luma sample. + /// The neutral encoded chroma sample. + [Theory] + [InlineData(Av1BitDepth.TenBit, true, 0, 1023, 512)] + [InlineData(Av1BitDepth.TenBit, false, 64, 940, 512)] + [InlineData(Av1BitDepth.TwelveBit, true, 0, 4095, 2048)] + [InlineData(Av1BitDepth.TwelveBit, false, 256, 3760, 2048)] + public void HighBitDepthYuvToRgbExpandsSignaledRange( + int bitDepth, + bool fullRange, + ushort black, + ushort white, + ushort neutralChroma) + { + // Assign + using Image image = new(2, 1); + ObuSequenceHeader sequenceHeader = CreateSequenceHeader(2, 1, fullRange, bitDepth: (Av1BitDepth)bitDepth); + using Av1FrameBuffer frameBuffer = new(Configuration.Default, sequenceHeader, Av1ColorFormat.Yuv444, false); + Span yRow = frameBuffer.GetHighBitDepthRowSpan(Av1Plane.Y, 0, 0, 0); + yRow[0] = black; + yRow[1] = white; + frameBuffer.GetHighBitDepthRowSpan(Av1Plane.U, 0, 0, 0).Fill(neutralChroma); + frameBuffer.GetHighBitDepthRowSpan(Av1Plane.V, 0, 0, 0).Fill(neutralChroma); + + // Act + Av1YuvConverter.ConvertToRgb(Configuration.Default, frameBuffer, image.Frames.RootFrame); + + // Assert + Span actual = image.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(0); + Assert.Equal(new Rgb24(0, 0, 0), actual[0]); + Assert.Equal(new Rgb24(255, 255, 255), actual[1]); + } + + /// + /// Verifies that high-bit-depth frame strides and row access use 16-bit sample units consistently. + /// + /// The reconstructed AV1 bit depth. + [Theory] + [InlineData(Av1BitDepth.TenBit)] + [InlineData(Av1BitDepth.TwelveBit)] + public void HighBitDepthFrameBufferUsesSampleUnitStrides(int bitDepth) + { + // Assign + ObuSequenceHeader sequenceHeader = CreateSequenceHeader( + 3, + 3, + colorFormat: Av1ColorFormat.Yuv420, + bitDepth: (Av1BitDepth)bitDepth); + + using Av1FrameBuffer frameBuffer = new(Configuration.Default, sequenceHeader, Av1ColorFormat.Yuv420, false); + + // Act + Span block = frameBuffer.DeriveBlockPointer16(Av1Plane.Y, Point.Empty, 0, 0, out int stride); + block[stride] = 321; + Span chromaRow = frameBuffer.GetHighBitDepthRowSpan(Av1Plane.U, 0, 1, 1); + + // Assert + Assert.Equal(2, frameBuffer.BytesPerSample); + Assert.Equal(608, stride); + Assert.Equal(stride * 2, frameBuffer.GetPlaneBuffer(Av1Plane.Y).Width); + Assert.Equal(321, frameBuffer.GetHighBitDepthRowSpan(Av1Plane.Y, 0, 0, 0)[0]); + Assert.Equal(2, chromaRow.Length); + } + + /// + /// Verifies libyuv's native two-times presentation filter at byte and twelve-bit precision across every + /// available intrinsic width and the scalar fallback. + /// + [Fact] + public void ScaleSelectedSpatialLayerMatchesReference() + => FeatureTestRunner.RunWithHwIntrinsicsFeature( + ValidateSelectedSpatialLayerScaling, + AlphaConfigurations); + + /// + /// Verifies the exact edge extension, quarter-sample weights, and rounding of the native presentation scaler. + /// + private static void ValidateSelectedSpatialLayerScaling() + { + byte[][] expectedByteRows = + [ + [0, 25, 75, 125, 175, 200], + [13, 38, 88, 138, 188, 213], + [38, 63, 113, 163, 213, 238], + [50, 75, 125, 175, 225, 250], + ]; + + ushort[][] expectedHighBitDepthRows = + [ + [0, 250, 750, 1250, 1750, 2000], + [125, 375, 875, 1375, 1875, 2125], + [375, 625, 1125, 1625, 2125, 2375], + [500, 750, 1250, 1750, 2250, 2500], + ]; + + ObuSequenceHeader byteSequenceHeader = CreateSequenceHeader( + 3, + 2, + colorFormat: Av1ColorFormat.Yuv400); + + using (Av1FrameBuffer frameBuffer = new( + Configuration.Default, + byteSequenceHeader, + Av1ColorFormat.Yuv400, + false)) + { + byte[] sourceSamples = [0, 100, 200, 50, 150, 250]; + for (int y = 0; y < byteSequenceHeader.MaxFrameHeight; y++) + { + sourceSamples.AsSpan(y * byteSequenceHeader.MaxFrameWidth, byteSequenceHeader.MaxFrameWidth).CopyTo( + frameBuffer.DeriveBlockPointer(Av1Plane.Y, 0, 0).DangerousGetRowSpan(y)); + } + + Av1PlanarSampleBuffer source = new(frameBuffer); + using Av1PresentationSampleBuffer> presentation = new( + Configuration.Default, + source, + 6, + 4); + + for (int y = 0; y < expectedByteRows.Length; y++) + { + Assert.True(presentation.View.GetLumaRowSpan(y).SequenceEqual(expectedByteRows[y])); + } + } + + ObuSequenceHeader highBitDepthSequenceHeader = CreateSequenceHeader( + 3, + 2, + colorFormat: Av1ColorFormat.Yuv400, + bitDepth: Av1BitDepth.TwelveBit); + + using Av1FrameBuffer highBitDepthFrameBuffer = new( + Configuration.Default, + highBitDepthSequenceHeader, + Av1ColorFormat.Yuv400, + false); + + ushort[] highBitDepthSourceSamples = [0, 1000, 2000, 500, 1500, 2500]; + for (int y = 0; y < highBitDepthSequenceHeader.MaxFrameHeight; y++) + { + highBitDepthSourceSamples.AsSpan( + y * highBitDepthSequenceHeader.MaxFrameWidth, + highBitDepthSequenceHeader.MaxFrameWidth).CopyTo( + highBitDepthFrameBuffer.GetHighBitDepthRowSpan(Av1Plane.Y, y, 0, 0)); + } + + Av1PlanarSampleBuffer highBitDepthSource = new(highBitDepthFrameBuffer); + using Av1PresentationSampleBuffer> highBitDepthPresentation = new( + Configuration.Default, + highBitDepthSource, + 6, + 4); + + for (int y = 0; y < expectedHighBitDepthRows.Length; y++) + { + Assert.True(highBitDepthPresentation.View.GetLumaRowSpan(y).SequenceEqual(expectedHighBitDepthRows[y])); + } + } + + /// + /// Verifies centered horizontal chroma reconstruction for a YUV 4:2:2 frame. + /// + [Fact] + public void Yuv422ToRgbBilinearlyUpsamplesCenteredChroma() + { + // Assign + using Image image = new(4, 1); + ObuSequenceHeader sequenceHeader = CreateSequenceHeader(4, 1, colorFormat: Av1ColorFormat.Yuv422); + using Av1FrameBuffer frameBuffer = new(Configuration.Default, sequenceHeader, Av1ColorFormat.Yuv422, false); + frameBuffer.DeriveBlockPointer(Av1Plane.Y, 0, 0).DangerousGetRowSpan(0).Fill(128); + Span uRow = frameBuffer.DeriveBlockPointer(Av1Plane.U, 1, 0).DangerousGetRowSpan(0); + uRow[0] = 128; + uRow[1] = 192; + frameBuffer.DeriveBlockPointer(Av1Plane.V, 1, 0).DangerousGetRowSpan(0).Fill(128); + + // Act + Av1YuvConverter.ConvertToRgb(Configuration.Default, frameBuffer, image.Frames.RootFrame); + + // Assert + Span actual = image.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(0); + Assert.Equal(new Rgb24(128, 128, 128), actual[0]); + Assert.Equal(new Rgb24(128, 125, 158), actual[1]); + Assert.Equal(new Rgb24(128, 119, 217), actual[2]); + Assert.Equal(new Rgb24(128, 116, 247), actual[3]); + } + + /// + /// Verifies vertical and horizontal YUV 4:2:0 reconstruction at every AV1 chroma sample position. + /// + /// The signaled AV1 chroma sample position. + /// The expected blue component in the top-row probe pixel. + /// The expected blue component in the left-column probe pixel. + [Theory] + [InlineData(ObuChromoSamplePosition.Unknown, 158, 98)] + [InlineData(ObuChromoSamplePosition.Vertical, 187, 98)] + [InlineData(ObuChromoSamplePosition.Colocated, 187, 69)] + public void Yuv420ToRgbUsesChromaSamplePosition(int chromaSamplePosition, byte expectedTopBlue, byte expectedLeftBlue) + { + // Assign + using Image image = new(4, 4); + ObuSequenceHeader sequenceHeader = CreateSequenceHeader( + 4, + 4, + colorFormat: Av1ColorFormat.Yuv420, + chromaSamplePosition: (ObuChromoSamplePosition)chromaSamplePosition); + + using Av1FrameBuffer frameBuffer = new(Configuration.Default, sequenceHeader, Av1ColorFormat.Yuv420, false); + Buffer2DRegion yPlane = frameBuffer.DeriveBlockPointer(Av1Plane.Y, 0, 0); + Buffer2DRegion uPlane = frameBuffer.DeriveBlockPointer(Av1Plane.U, 1, 1); + Buffer2DRegion vPlane = frameBuffer.DeriveBlockPointer(Av1Plane.V, 1, 1); + for (int y = 0; y < yPlane.Height; y++) + { + yPlane.DangerousGetRowSpan(y).Fill(128); + } + + uPlane.DangerousGetRowSpan(0)[0] = 128; + uPlane.DangerousGetRowSpan(0)[1] = 192; + uPlane.DangerousGetRowSpan(1)[0] = 64; + uPlane.DangerousGetRowSpan(1)[1] = 255; + vPlane.DangerousGetRowSpan(0).Fill(128); + vPlane.DangerousGetRowSpan(1).Fill(128); + + // Act + Av1YuvConverter.ConvertToRgb(Configuration.Default, frameBuffer, image.Frames.RootFrame); + + // Assert + Assert.Equal(expectedTopBlue, image.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(0)[1].B); + Assert.Equal(expectedLeftBlue, image.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(1)[0].B); + } + + /// + /// Verifies that encoding selects the horizontal and vertical sample coordinates defined by each AV1 chroma position. + /// + /// The signaled AV1 chroma sample position. + /// The expected encoded blue-difference sample. + /// The expected encoded red-difference sample. + [Theory] + [InlineData(ObuChromoSamplePosition.Unknown, 128, 128)] + [InlineData(ObuChromoSamplePosition.Vertical, 96, 192)] + [InlineData(ObuChromoSamplePosition.Colocated, 128, 128)] + public void RgbToYuv420UsesChromaSamplePosition(int chromaSamplePosition, byte expectedChromaBlue, byte expectedChromaRed) + { + using Image source = new(2, 2); + + // The four distinct YCgCo samples make left, centered, and vertically averaged selection observable as + // exact integer chroma values without introducing transfer-function or coefficient-rounding tolerances. + source[0, 0] = new Rgba32(0, 0, 0); + source[1, 0] = new Rgba32(0, byte.MaxValue, 0); + source[0, 1] = new Rgba32(byte.MaxValue, 0, 0); + source[1, 1] = new Rgba32(0, 0, byte.MaxValue); + ObuSequenceHeader sequenceHeader = CreateSequenceHeader( + 2, + 2, + matrixCoefficients: ObuMatrixCoefficients.SmpteYCgCo, + colorFormat: Av1ColorFormat.Yuv420, + chromaSamplePosition: (ObuChromoSamplePosition)chromaSamplePosition); + + using Av1FrameBuffer frameBuffer = new(Configuration.Default, sequenceHeader, Av1ColorFormat.Yuv420, false); + Av1YuvConverter.ConvertFromRgb(Configuration.Default, source.Frames.RootFrame, frameBuffer); + + Assert.Equal(expectedChromaBlue, frameBuffer.DeriveBlockPointer(Av1Plane.U, 1, 1).DangerousGetRowSpan(0)[0]); + Assert.Equal(expectedChromaRed, frameBuffer.DeriveBlockPointer(Av1Plane.V, 1, 1).DangerousGetRowSpan(0)[0]); + } + + /// + /// Verifies that odd image dimensions retain the final YUV 4:2:0 chroma row and column. + /// + [Fact] + public void Yuv420UsesCeilingChromaPlaneDimensions() + { + // Assign + ObuSequenceHeader sequenceHeader = CreateSequenceHeader(3, 3, colorFormat: Av1ColorFormat.Yuv420); + using Av1FrameBuffer frameBuffer = new(Configuration.Default, sequenceHeader, Av1ColorFormat.Yuv420, false); + + // Act + Buffer2DRegion uPlane = frameBuffer.DeriveBlockPointer(Av1Plane.U, 1, 1); + Buffer2DRegion vPlane = frameBuffer.DeriveBlockPointer(Av1Plane.V, 1, 1); + + // Assert + Assert.Equal(new Size(2, 2), uPlane.Size); + Assert.Equal(new Size(2, 2), vPlane.Size); + } + + /// + /// Compares SIMD-first RGB-to-YUV conversion with the independent scalar reference over randomized pixels. + /// + [Fact] + public void RgbToYuvCompareToReferenceRandomPixels() + { + const int sampleCount = 1000; + + // Assign + using Image image = new(sampleCount, 1); + ImageFrame frame = image.Frames.RootFrame; + frame.DangerousTryGetSinglePixelMemory(out Memory memory); + Random rnd = new(42); + Span input = new byte[sampleCount * 3]; + CreateTestData(rnd, input); + PixelOperations.Instance.FromBgr24Bytes(Configuration.Default, input, memory.Span, image.Width); + ObuSequenceHeader sequenceHeader = CreateSequenceHeader(image.Width, image.Height); + using Av1FrameBuffer frameBuffer = new(Configuration.Default, sequenceHeader, Av1ColorFormat.Yuv444, false); + + // Act + Av1YuvConverter.ConvertFromRgb(Configuration.Default, frame, frameBuffer); + Span referenceOutput = Av1ReferenceYuvConverter.RgbToYuv(memory.Span, true); + + // Assert + Span actual = new Rgb24[frameBuffer.Width]; + Span yRow = frameBuffer.DeriveBlockPointer(Av1Plane.Y, 0, 0).DangerousGetRowSpan(0); + Span uRow = frameBuffer.DeriveBlockPointer(Av1Plane.U, 0, 0).DangerousGetRowSpan(0); + Span vRow = frameBuffer.DeriveBlockPointer(Av1Plane.V, 0, 0).DangerousGetRowSpan(0); + for (int i = 0; i < frameBuffer.Width; i++) + { + Rgb24 pixel = default; + pixel.R = yRow[i]; + pixel.G = uRow[i]; + pixel.B = vRow[i]; + actual[i] = pixel; + } + + Compare(referenceOutput, actual, 3); + } + + /// + /// Compares SIMD-first YUV-to-RGB conversion with the independent scalar reference over randomized samples. + /// + [Fact] + public void YuvToRgbCompareToReferenceRandomPixels() + { + const int sampleCount = 1000; + + // Assign + using Image image = new(sampleCount, 1); + ImageFrame frame = image.Frames.RootFrame; + ObuSequenceHeader sequenceHeader = CreateSequenceHeader(image.Width, image.Height); + using Av1FrameBuffer frameBuffer = new(Configuration.Default, sequenceHeader, Av1ColorFormat.Yuv444, false); + Random rnd = new(42); + CreateTestData(rnd, frameBuffer, Av1Plane.Y); + CreateTestData(rnd, frameBuffer, Av1Plane.U); + CreateTestData(rnd, frameBuffer, Av1Plane.V); + + // Act + Av1YuvConverter.ConvertToRgb(Configuration.Default, frameBuffer, frame); + Span referenceOutput = Av1ReferenceYuvConverter.YuvToRgb(frameBuffer, true); + + // Assert + frame.DangerousTryGetSinglePixelMemory(out Memory memory); + Span actual = memory.Span; + Compare(referenceOutput, actual, 3); + } + + /// + /// Compares packed RGB rows within the permitted per-component tolerance. + /// + /// The independently converted reference pixels. + /// The pixels produced by the implementation under test. + /// The permitted absolute component difference. + private static void Compare(ReadOnlySpan referenceOutput, ReadOnlySpan actual, int allowedDifference) + { + for (int i = 0; i < actual.Length; i++) + { + if (Math.Abs(referenceOutput[i].R - actual[i].R) > allowedDifference || + Math.Abs(referenceOutput[i].G - actual[i].G) > allowedDifference || + Math.Abs(referenceOutput[i].B - actual[i].B) > allowedDifference) + { + Assert.Fail($"Difference at index {i}, expected: {referenceOutput[i]} but was {actual[i]}"); + } + } + } + + /// + /// Fills one reconstructed plane with deterministic pseudo-random test samples. + /// + /// The deterministic random number generator. + /// The frame containing the destination plane. + /// The destination plane. + private static void CreateTestData(Random rnd, Av1FrameBuffer frameBuffer, Av1Plane plane) + { + const int bitCount = 8; + Buffer2DRegion region = frameBuffer.DeriveBlockPointer(plane, 0, 0); + for (int y = 0; y < region.Height; y++) + { + CreateTestData(rnd, region.DangerousGetRowSpan(y), bitCount); + } + } + + /// + /// Fills an eight-bit sample span with deterministic pseudo-random values. + /// + /// The deterministic random number generator. + /// The destination sample span. + /// The number of significant sample bits. + private static void CreateTestData(Random rnd, Span span, int bitCount = 8) + { + int max = (1 << bitCount) - 1; + for (int i = 0; i < span.Length; i++) + { + byte current = (byte)rnd.Next(max); + span[i] = current; + } + } + + /// + /// Fills a high-bit-depth sample span with deterministic pseudo-random values. + /// + /// The deterministic random number generator. + /// The destination sample span. + /// The number of significant sample bits. + private static void CreateTestData(Random rnd, Span span, int bitCount) + { + int max = (1 << bitCount) - 1; + for (int i = 0; i < span.Length; i++) + { + ushort current = (ushort)rnd.Next(max); + span[i] = current; + } + } + + /// + /// Verifies RGB-to-YUV-to-RGB conversion for representative single-pixel colors. + /// + /// The source red component. + /// The source green component. + /// The source blue component. + [Theory] + [InlineData(255, 255, 255)] + [InlineData(0, 0, 0)] + [InlineData(42, 42, 42)] + [InlineData(42, 0, 0)] + [InlineData(42, 42, 0)] + [InlineData(42, 0, 42)] + [InlineData(0, 42, 42)] + [InlineData(0, 0, 42)] + [InlineData(150, 100, 50)] + public void RoundTripSinglePixel(byte r, byte g, byte b) + { + // Assign + using Image image = new(1, 1); + ImageFrame frame = image.Frames.RootFrame; + frame.DangerousTryGetSinglePixelMemory(out Memory memory); + memory.Span[0] = new Rgb24(r, g, b); + ObuSequenceHeader sequenceHeader = CreateSequenceHeader(1, 1); + using Av1FrameBuffer frameBuffer = new(Configuration.Default, sequenceHeader, Av1ColorFormat.Yuv444, false); + using Image actual = new(image.Width, image.Height); + + // Act + Av1YuvConverter.ConvertFromRgb(Configuration.Default, frame, frameBuffer); + Av1YuvConverter.ConvertToRgb(Configuration.Default, frameBuffer, actual.Frames.RootFrame); + + // Assert + actual.Frames.RootFrame.DangerousTryGetSinglePixelMemory(out Memory actualMemory); + Rgb24 actualPixel = actualMemory.Span[0]; + Assert.Equal(r, actualPixel.R, 2d); + Assert.Equal(g, actualPixel.G, 2d); + Assert.Equal(b, actualPixel.B, 2d); + } + + /// + /// Verifies 10-bit and 12-bit round trips for coefficient, identity, and YCgCo matrices. + /// + /// The encoded AV1 bit depth. + /// The matrix coefficients used for conversion. + [Theory] + [InlineData(Av1BitDepth.TenBit, ObuMatrixCoefficients.Bt709)] + [InlineData(Av1BitDepth.TenBit, ObuMatrixCoefficients.Identity)] + [InlineData(Av1BitDepth.TenBit, ObuMatrixCoefficients.SmpteYCgCo)] + [InlineData(Av1BitDepth.TenBit, ObuMatrixCoefficients.IptC2)] + [InlineData(Av1BitDepth.TenBit, ObuMatrixCoefficients.YCgCoRe)] + [InlineData(Av1BitDepth.TenBit, ObuMatrixCoefficients.YCgCoRo)] + [InlineData(Av1BitDepth.TwelveBit, ObuMatrixCoefficients.Bt709)] + [InlineData(Av1BitDepth.TwelveBit, ObuMatrixCoefficients.Identity)] + [InlineData(Av1BitDepth.TwelveBit, ObuMatrixCoefficients.SmpteYCgCo)] + [InlineData(Av1BitDepth.TwelveBit, ObuMatrixCoefficients.IptC2)] + [InlineData(Av1BitDepth.TwelveBit, ObuMatrixCoefficients.YCgCoRe)] + [InlineData(Av1BitDepth.TwelveBit, ObuMatrixCoefficients.YCgCoRo)] + public void HighBitDepthRoundTrip(int bitDepth, int matrixCoefficients) + { + // Assign + using Image image = new(3, 1); + Span source = image.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(0); + source[0] = new Rgb24(0, 0, 0); + source[1] = new Rgb24(150, 100, 50); + source[2] = new Rgb24(255, 255, 255); + ObuSequenceHeader sequenceHeader = CreateSequenceHeader( + image.Width, + image.Height, + matrixCoefficients: (ObuMatrixCoefficients)matrixCoefficients, + bitDepth: (Av1BitDepth)bitDepth); + + using Av1FrameBuffer frameBuffer = new(Configuration.Default, sequenceHeader, Av1ColorFormat.Yuv444, false); + using Image actual = new(image.Width, image.Height); + + // Act + Av1YuvConverter.ConvertFromRgb(Configuration.Default, image.Frames.RootFrame, frameBuffer); + Av1YuvConverter.ConvertToRgb(Configuration.Default, frameBuffer, actual.Frames.RootFrame); + + // Assert + Span actualPixels = actual.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(0); + for (int x = 0; x < source.Length; x++) + { + Assert.Equal(source[x].R, actualPixels[x].R, 1D); + Assert.Equal(source[x].G, actualPixels[x].G, 1D); + Assert.Equal(source[x].B, actualPixels[x].B, 1D); + } + } + + /// + /// Verifies the H.273 IPT-C2 matrices in both directions against independently calculated code values. + /// + [Fact] + public void IptC2MatchesKnownLinearTransferValuesInBothDirections() + { + // Assign + // The linear transfer characteristic isolates the two normative IPT-C2 matrices from transfer-curve error. + using Image source = new(1, 1); + source.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(0)[0] = new Rgb24(150, 100, 50); + ObuSequenceHeader sequenceHeader = CreateSequenceHeader( + 1, + 1, + matrixCoefficients: ObuMatrixCoefficients.IptC2, + transferCharacteristics: ObuTransferCharacteristics.Linear); + + using Av1FrameBuffer frameBuffer = new(Configuration.Default, sequenceHeader, Av1ColorFormat.Yuv444, false); + + // Act + Av1YuvConverter.ConvertFromRgb(Configuration.Default, source.Frames.RootFrame, frameBuffer); + + // Assert + Assert.Equal(100, frameBuffer.DeriveBlockPointer(Av1Plane.Y, 0, 0).DangerousGetRowSpan(0)[0]); + Assert.Equal(178, frameBuffer.DeriveBlockPointer(Av1Plane.U, 0, 0).DangerousGetRowSpan(0)[0]); + Assert.Equal(198, frameBuffer.DeriveBlockPointer(Av1Plane.V, 0, 0).DangerousGetRowSpan(0)[0]); + + using Image destination = new(1, 1); + Av1YuvConverter.ConvertToRgb(Configuration.Default, frameBuffer, destination.Frames.RootFrame); + + Assert.Equal(new Rgb24(150, 100, 51), destination.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(0)[0]); + } + + /// + /// Verifies the reversible YCgCo lifting stages against known pure-red code values in both directions. + /// + /// The encoded AV1 bit depth. + /// The reversible YCgCo variant. + /// The expected encoded luma value. + /// The expected encoded Cg value. + /// The expected encoded Co value. + [Theory] + [InlineData(Av1BitDepth.EightBit, ObuMatrixCoefficients.YCgCoRe, 15, 97, 191)] + [InlineData(Av1BitDepth.TenBit, ObuMatrixCoefficients.YCgCoRe, 63, 385, 767)] + [InlineData(Av1BitDepth.TwelveBit, ObuMatrixCoefficients.YCgCoRe, 255, 1537, 3071)] + [InlineData(Av1BitDepth.EightBit, ObuMatrixCoefficients.YCgCoRo, 31, 65, 255)] + [InlineData(Av1BitDepth.TenBit, ObuMatrixCoefficients.YCgCoRo, 127, 257, 1023)] + [InlineData(Av1BitDepth.TwelveBit, ObuMatrixCoefficients.YCgCoRo, 511, 1025, 4095)] + public void ReversibleYCgCoMatchesPureRed( + int bitDepth, + int matrixCoefficients, + int expectedY, + int expectedU, + int expectedV) + { + // Assign + // Pure red exercises positive odd Co and negative odd Cg. The expected samples come directly from + // the H.273 integer lifting equations at the logical RGB precision selected by each matrix code point. + using Image source = new(1, 1); + source.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(0)[0] = new Rgb48(ushort.MaxValue, 0, 0); + ObuSequenceHeader sequenceHeader = CreateSequenceHeader( + 1, + 1, + matrixCoefficients: (ObuMatrixCoefficients)matrixCoefficients, + bitDepth: (Av1BitDepth)bitDepth); + + using Av1FrameBuffer frameBuffer = new(Configuration.Default, sequenceHeader, Av1ColorFormat.Yuv444, false); + + // Act + Av1YuvConverter.ConvertFromRgb(Configuration.Default, source.Frames.RootFrame, frameBuffer); + + // Assert + int actualY; + int actualU; + int actualV; + if ((Av1BitDepth)bitDepth == Av1BitDepth.EightBit) + { + actualY = frameBuffer.DeriveBlockPointer(Av1Plane.Y, 0, 0).DangerousGetRowSpan(0)[0]; + actualU = frameBuffer.DeriveBlockPointer(Av1Plane.U, 0, 0).DangerousGetRowSpan(0)[0]; + actualV = frameBuffer.DeriveBlockPointer(Av1Plane.V, 0, 0).DangerousGetRowSpan(0)[0]; + } + else + { + actualY = frameBuffer.GetHighBitDepthRowSpan(Av1Plane.Y, 0, 0, 0)[0]; + actualU = frameBuffer.GetHighBitDepthRowSpan(Av1Plane.U, 0, 0, 0)[0]; + actualV = frameBuffer.GetHighBitDepthRowSpan(Av1Plane.V, 0, 0, 0)[0]; + } + + Assert.Equal(expectedY, actualY); + Assert.Equal(expectedU, actualU); + Assert.Equal(expectedV, actualV); + + using Image destination = new(1, 1); + Av1YuvConverter.ConvertToRgb(Configuration.Default, frameBuffer, destination.Frames.RootFrame); + + Assert.Equal(new Rgb48(ushort.MaxValue, 0, 0), destination.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(0)[0]); + } + + /// + /// Verifies that limited-range reversible YCgCo applies range adjustment to RGB code values before lifting. + /// + /// The encoded AV1 bit depth. + /// The reversible YCgCo variant. + /// The expected black luma code value. + /// The expected white luma code value. + [Theory] + [InlineData(Av1BitDepth.EightBit, ObuMatrixCoefficients.YCgCoRe, 4, 59)] + [InlineData(Av1BitDepth.TenBit, ObuMatrixCoefficients.YCgCoRe, 16, 235)] + [InlineData(Av1BitDepth.TwelveBit, ObuMatrixCoefficients.YCgCoRe, 64, 940)] + [InlineData(Av1BitDepth.EightBit, ObuMatrixCoefficients.YCgCoRo, 8, 118)] + [InlineData(Av1BitDepth.TenBit, ObuMatrixCoefficients.YCgCoRo, 32, 470)] + [InlineData(Av1BitDepth.TwelveBit, ObuMatrixCoefficients.YCgCoRo, 128, 1880)] + public void ReversibleYCgCoAppliesLimitedRangeBeforeLifting( + int bitDepth, + int matrixCoefficients, + int expectedBlack, + int expectedWhite) + { + // Assign + // Black and white have zero Cg and Co, exposing the RGB-domain range mapping without opponent-axis noise. + using Image source = new(2, 1); + Span sourcePixels = source.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(0); + sourcePixels[0] = new Rgb48(0, 0, 0); + sourcePixels[1] = new Rgb48(ushort.MaxValue, ushort.MaxValue, ushort.MaxValue); + ObuSequenceHeader sequenceHeader = CreateSequenceHeader( + 2, + 1, + fullRange: false, + matrixCoefficients: (ObuMatrixCoefficients)matrixCoefficients, + bitDepth: (Av1BitDepth)bitDepth); + + using Av1FrameBuffer frameBuffer = new(Configuration.Default, sequenceHeader, Av1ColorFormat.Yuv444, false); + + // Act + Av1YuvConverter.ConvertFromRgb(Configuration.Default, source.Frames.RootFrame, frameBuffer); + + // Assert + int expectedChromaBias = 1 << (((Av1BitDepth)bitDepth).GetBitCount() - 1); + if ((Av1BitDepth)bitDepth == Av1BitDepth.EightBit) + { + Span y = frameBuffer.DeriveBlockPointer(Av1Plane.Y, 0, 0).DangerousGetRowSpan(0); + Span u = frameBuffer.DeriveBlockPointer(Av1Plane.U, 0, 0).DangerousGetRowSpan(0); + Span v = frameBuffer.DeriveBlockPointer(Av1Plane.V, 0, 0).DangerousGetRowSpan(0); + Assert.Equal(expectedBlack, y[0]); + Assert.Equal(expectedWhite, y[1]); + Assert.Equal(expectedChromaBias, u[0]); + Assert.Equal(expectedChromaBias, u[1]); + Assert.Equal(expectedChromaBias, v[0]); + Assert.Equal(expectedChromaBias, v[1]); + } + else + { + Span y = frameBuffer.GetHighBitDepthRowSpan(Av1Plane.Y, 0, 0, 0); + Span u = frameBuffer.GetHighBitDepthRowSpan(Av1Plane.U, 0, 0, 0); + Span v = frameBuffer.GetHighBitDepthRowSpan(Av1Plane.V, 0, 0, 0); + Assert.Equal(expectedBlack, y[0]); + Assert.Equal(expectedWhite, y[1]); + Assert.Equal(expectedChromaBias, u[0]); + Assert.Equal(expectedChromaBias, u[1]); + Assert.Equal(expectedChromaBias, v[0]); + Assert.Equal(expectedChromaBias, v[1]); + } + + using Image destination = new(2, 1); + Av1YuvConverter.ConvertToRgb(Configuration.Default, frameBuffer, destination.Frames.RootFrame); + + Span destinationPixels = destination.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(0); + Assert.Equal(new Rgb48(0, 0, 0), destinationPixels[0]); + Assert.Equal(new Rgb48(ushort.MaxValue, ushort.MaxValue, ushort.MaxValue), destinationPixels[1]); + } + + /// + /// Verifies that reversible YCgCo rejects chroma subsampling in both conversion directions. + /// + /// The reversible YCgCo variant. + [Theory] + [InlineData(ObuMatrixCoefficients.YCgCoRe)] + [InlineData(ObuMatrixCoefficients.YCgCoRo)] + public void ReversibleYCgCoRequiresFullChroma(int matrixCoefficients) + { + // Assign + using Image image = new(2, 2); + ObuSequenceHeader sequenceHeader = CreateSequenceHeader( + 2, + 2, + matrixCoefficients: (ObuMatrixCoefficients)matrixCoefficients, + colorFormat: Av1ColorFormat.Yuv420); + + using Av1FrameBuffer frameBuffer = new(Configuration.Default, sequenceHeader, Av1ColorFormat.Yuv420, false); + + // Act and assert + Assert.Throws( + () => Av1YuvConverter.ConvertFromRgb(Configuration.Default, image.Frames.RootFrame, frameBuffer)); + + Assert.Throws( + () => Av1YuvConverter.ConvertToRgb(Configuration.Default, frameBuffer, image.Frames.RootFrame)); + } + + /// + /// Verifies that every H.273 operator produces the same result in SIMD batches and the scalar row tail. + /// + /// The matrix coefficients selecting the color operator. + /// The transfer characteristics used by nonlinear operators. + [Theory] + [InlineData(ObuMatrixCoefficients.Bt709, ObuTransferCharacteristics.Bt709)] + [InlineData(ObuMatrixCoefficients.Identity, ObuTransferCharacteristics.Bt709)] + [InlineData(ObuMatrixCoefficients.SmpteYCgCo, ObuTransferCharacteristics.Bt709)] + [InlineData(ObuMatrixCoefficients.Bt2020ConstantLuminance, ObuTransferCharacteristics.Bt202010Bit)] + [InlineData(ObuMatrixCoefficients.Smpte2085, ObuTransferCharacteristics.Bt709)] + [InlineData(ObuMatrixCoefficients.ChromaticityDerivedNonConstantLuminance, ObuTransferCharacteristics.Bt709)] + [InlineData(ObuMatrixCoefficients.ChromaticityDerivedConstantLuminance, ObuTransferCharacteristics.Bt709)] + [InlineData(ObuMatrixCoefficients.Bt2100ICtCp, ObuTransferCharacteristics.Smpte2084)] + [InlineData(ObuMatrixCoefficients.Bt2100ICtCp, ObuTransferCharacteristics.Hlg)] + [InlineData(ObuMatrixCoefficients.IptC2, ObuTransferCharacteristics.Bt709)] + [InlineData(ObuMatrixCoefficients.YCgCoRe, ObuTransferCharacteristics.Bt709)] + [InlineData(ObuMatrixCoefficients.YCgCoRo, ObuTransferCharacteristics.Bt709)] + public void ColorOperatorSimdBatchesMatchScalarTail(int matrixCoefficients, int transferCharacteristics) + { + const int width = 31; + + // Thirty-one samples exercise Vector512, Vector256, Vector128, and scalar stages on AVX-512 hardware. + // The same row still reaches the widest available stages and scalar tail on narrower SIMD hardware. + using Image source = new(width, 1); + source.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(0).Fill(new Rgb48(39999, 27777, 12345)); + ObuSequenceHeader sequenceHeader = CreateSequenceHeader( + width, + 1, + matrixCoefficients: (ObuMatrixCoefficients)matrixCoefficients, + bitDepth: Av1BitDepth.TwelveBit, + transferCharacteristics: (ObuTransferCharacteristics)transferCharacteristics, + colorPrimaries: ObuColorPrimaries.Bt2020); + + using Av1FrameBuffer frameBuffer = new(Configuration.Default, sequenceHeader, Av1ColorFormat.Yuv444, false); + using Image destination = new(width, 1); + + Av1YuvConverter.ConvertFromRgb(Configuration.Default, source.Frames.RootFrame, frameBuffer); + Av1YuvConverter.ConvertToRgb(Configuration.Default, frameBuffer, destination.Frames.RootFrame); + + AssertPlaneContainsRepeatedSample(frameBuffer, Av1Plane.Y, 0, 0); + AssertPlaneContainsRepeatedSample(frameBuffer, Av1Plane.U, 0, 0); + AssertPlaneContainsRepeatedSample(frameBuffer, Av1Plane.V, 0, 0); + + Span pixels = destination.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(0); + for (int x = 1; x < pixels.Length; x++) + { + Assert.Equal(pixels[0], pixels[x]); + } + } + + /// + /// Verifies horizontal and vertical chroma downsampling against full-resolution encoded components. + /// + /// The subsampled AV1 color format. + /// The encoded AV1 bit depth. + [Theory] + [InlineData(Av1ColorFormat.Yuv422, Av1BitDepth.EightBit)] + [InlineData(Av1ColorFormat.Yuv422, Av1BitDepth.TwelveBit)] + [InlineData(Av1ColorFormat.Yuv420, Av1BitDepth.EightBit)] + [InlineData(Av1ColorFormat.Yuv420, Av1BitDepth.TwelveBit)] + public void RgbToYuvSubsamplingAveragesFullResolutionChroma(int colorFormat, int bitDepth) + { + const int width = 35; + const int height = 3; + + using Image source = new(width, height); + for (int y = 0; y < height; y++) + { + Span row = source.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(y); + for (int x = 0; x < width; x++) + { + row[x] = new Rgb48( + (ushort)((x * 1879) + (y * 791)), + (ushort)((x * 977) + (y * 3251)), + (ushort)((x * 613) + (y * 4987))); + } + } + + ObuSequenceHeader fullResolutionHeader = CreateSequenceHeader( + width, + height, + colorFormat: Av1ColorFormat.Yuv444, + bitDepth: (Av1BitDepth)bitDepth); + + ObuSequenceHeader subsampledHeader = CreateSequenceHeader( + width, + height, + colorFormat: (Av1ColorFormat)colorFormat, + bitDepth: (Av1BitDepth)bitDepth); + + using Av1FrameBuffer fullResolution = new(Configuration.Default, fullResolutionHeader, Av1ColorFormat.Yuv444, false); + using Av1FrameBuffer subsampled = new(Configuration.Default, subsampledHeader, (Av1ColorFormat)colorFormat, false); + + Av1YuvConverter.ConvertFromRgb(Configuration.Default, source.Frames.RootFrame, fullResolution); + Av1YuvConverter.ConvertFromRgb(Configuration.Default, source.Frames.RootFrame, subsampled); + + AssertSubsampledPlaneMatchesAverage(fullResolution, subsampled, Av1Plane.U); + AssertSubsampledPlaneMatchesAverage(fullResolution, subsampled, Av1Plane.V); + } + + /// + /// Verifies an image-wide RGB-to-YUV-to-RGB conversion against the configured similarity tolerance. + /// + /// The source test-image provider. + // [Theory] + // [WithFile(TestImages.Jpeg.Baseline.Winter444_Interleaved, PixelTypes.Rgb24)] + public static void RoundTrip(TestImageProvider provider) + { + // Assign + using Image image = provider.GetImage(); + ImageFrame frame = image.Frames.RootFrame; + ObuSequenceHeader sequenceHeader = CreateSequenceHeader(image.Width, image.Height); + using Av1FrameBuffer frameBuffer = new(Configuration.Default, sequenceHeader, Av1ColorFormat.Yuv444, false); + using Image actual = new(image.Width, image.Height); + + // Act + Av1YuvConverter.ConvertFromRgb(Configuration.Default, frame, frameBuffer); + Av1YuvConverter.ConvertToRgb(Configuration.Default, frameBuffer, actual.Frames.RootFrame); + + // Assert + ImageComparer.Tolerant(0.002F).VerifySimilarity(image, actual); + } + + /// + /// Verifies that same-sized AV1 alpha composition preserves color and maps full-range luma exactly with and + /// without hardware intrinsics. + /// + [Fact] + public void ComposeAlphaMapsEightBitLumaExactly() + => FeatureTestRunner.RunWithHwIntrinsicsFeature( + ValidateEightBitAlphaComposition, + AlphaConfigurations); + + /// + /// Verifies that scaled 10-bit and 12-bit AV1 alpha composition matches ImageSharp's established box resampler + /// with and without hardware intrinsics. + /// + [Fact] + public void ComposeAlphaScalesHighBitDepthLuma() + => FeatureTestRunner.RunWithHwIntrinsicsFeature( + ValidateHighBitDepthAlphaScaling, + AlphaConfigurations); + + /// + /// Verifies that alpha scaling remains exact when the bounded working buffer must advance through multiple + /// source-row windows. + /// + [Fact] + public void ComposeAlphaScalesAcrossMultipleWorkingWindows() + => FeatureTestRunner.RunWithHwIntrinsicsFeature( + ValidateSlidingWindowAlphaScaling, + AlphaConfigurations); + + /// + /// Verifies exact limited-range endpoints and out-of-range clamping for every supported AV1 alpha bit depth. + /// + [Fact] + public void ComposeAlphaExpandsLimitedRange() + => FeatureTestRunner.RunWithHwIntrinsicsFeature( + ValidateLimitedRangeAlphaComposition, + AlphaConfigurations); + + /// + /// Exercises direct full-range byte alpha composition against exact code-value expansion. + /// + private static void ValidateEightBitAlphaComposition() + { + const int width = 19; + const int height = 5; + + ObuSequenceHeader sequenceHeader = CreateSequenceHeader(width, height, colorFormat: Av1ColorFormat.Yuv400); + using Av1FrameBuffer frameBuffer = new(Configuration.Default, sequenceHeader, Av1ColorFormat.Yuv400, false); + using Image destination = new(width, height); + Buffer2DRegion luma = frameBuffer.DeriveBlockPointer(Av1Plane.Y, 0, 0); + for (int y = 0; y < height; y++) + { + Span sourceRow = luma.DangerousGetRowSpan(y); + Span destinationRow = destination.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(y); + for (int x = 0; x < width; x++) + { + sourceRow[x] = (byte)((x * 11) + (y * 7)); + destinationRow[x] = new Rgba64((ushort)(1000 + x), (ushort)(2000 + y), 3000, ushort.MaxValue); + } + } + + Av1YuvConverter.ComposeAlpha( + Configuration.Default, + frameBuffer, + destination.Frames.RootFrame, + destination.Size, + destination.Bounds, + false); + + for (int y = 0; y < height; y++) + { + Span sourceRow = luma.DangerousGetRowSpan(y); + Span actualRow = destination.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(y); + for (int x = 0; x < width; x++) + { + Assert.Equal((ushort)(1000 + x), actualRow[x].R); + Assert.Equal((ushort)(2000 + y), actualRow[x].G); + Assert.Equal((ushort)3000, actualRow[x].B); + Assert.Equal((ushort)(sourceRow[x] * 257), actualRow[x].A); + } + } + } + + /// + /// Exercises the direct box-resize path for both supported high-bit-depth sample layouts. + /// + private static void ValidateHighBitDepthAlphaScaling() + { + const int sourceWidth = 5; + const int sourceHeight = 3; + const int destinationWidth = 9; + const int destinationHeight = 7; + + foreach (Av1BitDepth bitDepth in new[] { Av1BitDepth.TenBit, Av1BitDepth.TwelveBit }) + { + ObuSequenceHeader sequenceHeader = CreateSequenceHeader( + sourceWidth, + sourceHeight, + colorFormat: Av1ColorFormat.Yuv400, + bitDepth: bitDepth); + + using Av1FrameBuffer frameBuffer = new(Configuration.Default, sequenceHeader, Av1ColorFormat.Yuv400, false); + using Image expected = new(sourceWidth, sourceHeight); + using Image destination = new(destinationWidth, destinationHeight, new Rgba64(1000, 2000, 3000, ushort.MaxValue)); + ushort maximum = (ushort)((1 << bitDepth.GetBitCount()) - 1); + for (int y = 0; y < sourceHeight; y++) + { + Span sourceRow = frameBuffer.GetHighBitDepthRowSpan(Av1Plane.Y, y, 0, 0); + Span expectedRow = expected.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(y); + for (int x = 0; x < sourceWidth; x++) + { + sourceRow[x] = (ushort)(((x * 223) + (y * 151)) & maximum); + expectedRow[x] = L16.FromScaledVector4(new Vector4((float)sourceRow[x] / maximum)); + } + } + + expected.Mutate(context => context.Resize(destinationWidth, destinationHeight, KnownResamplers.Box)); + Av1YuvConverter.ComposeAlpha( + Configuration.Default, + frameBuffer, + destination.Frames.RootFrame, + destination.Size, + destination.Bounds, + false); + + for (int y = 0; y < destinationHeight; y++) + { + Span expectedRow = expected.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(y); + Span actualRow = destination.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(y); + for (int x = 0; x < destinationWidth; x++) + { + Assert.Equal((ushort)1000, actualRow[x].R); + Assert.Equal((ushort)2000, actualRow[x].G); + Assert.Equal((ushort)3000, actualRow[x].B); + Assert.Equal(expectedRow[x].PackedValue, actualRow[x].A); + } + } + } + } + + /// + /// Exercises overlapping box kernels across multiple transposed source-row windows. + /// + private static void ValidateSlidingWindowAlphaScaling() + { + const int sourceWidth = 13; + const int sourceHeight = 41; + const int destinationWidth = 23; + const int destinationHeight = 17; + + Configuration configuration = Configuration.CreateDefaultInstance(); + configuration.WorkingBufferSizeHintInBytes = 1; + ObuSequenceHeader sequenceHeader = CreateSequenceHeader( + sourceWidth, + sourceHeight, + colorFormat: Av1ColorFormat.Yuv400, + bitDepth: Av1BitDepth.TwelveBit); + + using Av1FrameBuffer frameBuffer = new(configuration, sequenceHeader, Av1ColorFormat.Yuv400, false); + using Image expected = new(configuration, sourceWidth, sourceHeight); + using Image destination = new(configuration, destinationWidth, destinationHeight, new Rgba64(1000, 2000, 3000, ushort.MaxValue)); + for (int y = 0; y < sourceHeight; y++) + { + Span sourceRow = frameBuffer.GetHighBitDepthRowSpan(Av1Plane.Y, y, 0, 0); + Span expectedRow = expected.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(y); + for (int x = 0; x < sourceWidth; x++) + { + sourceRow[x] = (ushort)(((x * 277) + (y * 193)) & 4095); + expectedRow[x] = L16.FromScaledVector4(new Vector4(sourceRow[x] / 4095F)); + } + } + + expected.Mutate(context => context.Resize(destinationWidth, destinationHeight, KnownResamplers.Box)); + Av1YuvConverter.ComposeAlpha( + configuration, + frameBuffer, + destination.Frames.RootFrame, + destination.Size, + destination.Bounds, + false); + + for (int y = 0; y < destinationHeight; y++) + { + Span expectedRow = expected.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(y); + Span actualRow = destination.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(y); + for (int x = 0; x < destinationWidth; x++) + { + Assert.Equal((ushort)1000, actualRow[x].R); + Assert.Equal((ushort)2000, actualRow[x].G); + Assert.Equal((ushort)3000, actualRow[x].B); + Assert.Equal(expectedRow[x].PackedValue, actualRow[x].A); + } + } + } + + /// + /// Exercises luma-range expansion and clamping for 8-bit, 10-bit, and 12-bit alpha samples. + /// + private static void ValidateLimitedRangeAlphaComposition() + { + foreach (Av1BitDepth bitDepth in new[] { Av1BitDepth.EightBit, Av1BitDepth.TenBit, Av1BitDepth.TwelveBit }) + { + int bitCount = bitDepth.GetBitCount(); + ushort minimum = (ushort)(16 << (bitCount - 8)); + ushort maximum = (ushort)(235 << (bitCount - 8)); + ushort storageMaximum = (ushort)((1 << bitCount) - 1); + ObuSequenceHeader sequenceHeader = CreateSequenceHeader( + 4, + 1, + fullRange: false, + colorFormat: Av1ColorFormat.Yuv400, + bitDepth: bitDepth); + + using Av1FrameBuffer frameBuffer = new(Configuration.Default, sequenceHeader, Av1ColorFormat.Yuv400, false); + using Image destination = new(4, 1, new Rgba64(1000, 2000, 3000, ushort.MaxValue)); + if (bitDepth == Av1BitDepth.EightBit) + { + Span luma = frameBuffer.DeriveBlockPointer(Av1Plane.Y, 0, 0).DangerousGetRowSpan(0); + luma[0] = 0; + luma[1] = (byte)minimum; + luma[2] = (byte)maximum; + luma[3] = byte.MaxValue; + } + else + { + Span luma = frameBuffer.GetHighBitDepthRowSpan(Av1Plane.Y, 0, 0, 0); + luma[0] = 0; + luma[1] = minimum; + luma[2] = maximum; + luma[3] = storageMaximum; + } + + Av1YuvConverter.ComposeAlpha( + Configuration.Default, + frameBuffer, + destination.Frames.RootFrame, + destination.Size, + destination.Bounds, + false); + + Span actual = destination.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(0); + Assert.Equal((ushort)0, actual[0].A); + Assert.Equal((ushort)0, actual[1].A); + Assert.Equal(ushort.MaxValue, actual[2].A); + Assert.Equal(ushort.MaxValue, actual[3].A); + } + } + + /// + /// Creates a sequence header containing the color signaling required by a conversion test. + /// + /// The frame width. + /// The frame height. + /// Whether encoded samples use the full range. + /// The matrix coefficients used for conversion. + /// The encoded plane layout. + /// The signaled chroma sample position. + /// The encoded sample bit depth. + /// The transfer characteristics used by nonlinear matrices. + /// The color primaries used by derived matrices. + /// The configured sequence header. + private static ObuSequenceHeader CreateSequenceHeader( + int width, + int height, + bool fullRange = true, + ObuMatrixCoefficients matrixCoefficients = ObuMatrixCoefficients.Bt709, + Av1ColorFormat colorFormat = Av1ColorFormat.Yuv444, + ObuChromoSamplePosition chromaSamplePosition = ObuChromoSamplePosition.Unknown, + Av1BitDepth bitDepth = Av1BitDepth.EightBit, + ObuTransferCharacteristics transferCharacteristics = ObuTransferCharacteristics.Bt709, + ObuColorPrimaries colorPrimaries = ObuColorPrimaries.Bt709) + => new() + { + MaxFrameWidth = width, + MaxFrameHeight = height, + ColorConfig = new ObuColorConfig + { + IsMonochrome = colorFormat == Av1ColorFormat.Yuv400, + BitDepth = bitDepth, + MatrixCoefficients = matrixCoefficients, + TransferCharacteristics = transferCharacteristics, + ColorPrimaries = colorPrimaries, + ColorRange = fullRange, + SubSamplingX = colorFormat is Av1ColorFormat.Yuv400 or Av1ColorFormat.Yuv420 or Av1ColorFormat.Yuv422, + SubSamplingY = colorFormat is Av1ColorFormat.Yuv400 or Av1ColorFormat.Yuv420, + ChromaSamplePosition = chromaSamplePosition, + }, + }; + + /// + /// Verifies that every high-bit-depth sample in a plane matches its first sample. + /// + /// The encoded frame buffer. + /// The plane to inspect. + /// The horizontal subsampling shift. + /// The vertical subsampling shift. + private static void AssertPlaneContainsRepeatedSample(Av1FrameBuffer frameBuffer, Av1Plane plane, int subX, int subY) + { + Span samples = frameBuffer.GetHighBitDepthRowSpan(plane, 0, subX, subY); + for (int x = 1; x < samples.Length; x++) + { + Assert.Equal(samples[0], samples[x]); + } + } + + /// + /// Verifies that a subsampled plane contains the rounded mean of the corresponding full-resolution samples. + /// + /// The full-resolution encoded frame. + /// The subsampled encoded frame. + /// The chroma plane to compare. + private static void AssertSubsampledPlaneMatchesAverage( + Av1FrameBuffer fullResolution, + Av1FrameBuffer subsampled, + Av1Plane plane) + { + int subY = subsampled.ColorConfig.SubSamplingY ? 1 : 0; + int chromaHeight = (subsampled.Height + subY) >> subY; + int chromaWidth = (subsampled.Width + 1) >> 1; + for (int y = 0; y < chromaHeight; y++) + { + int sourceY = y << subY; + int rowCount = subY == 0 ? 1 : Math.Min(2, fullResolution.Height - sourceY); + for (int x = 0; x < chromaWidth; x++) + { + int sourceX = x << 1; + int columnCount = Math.Min(2, fullResolution.Width - sourceX); + int sum = 0; + for (int row = 0; row < rowCount; row++) + { + for (int column = 0; column < columnCount; column++) + { + sum += GetPlaneSample(fullResolution, plane, sourceX + column, sourceY + row, 0, 0); + } + } + + int expected = (int)MathF.Round((float)sum / (rowCount * columnCount), MidpointRounding.AwayFromZero); + int actual = GetPlaneSample(subsampled, plane, x, y, 1, subY); + Assert.True( + actual >= expected - 1 && actual <= expected + 1, + $"Plane {plane}, sample ({x}, {y}): expected {expected} +/- 1 from sum {sum} over {rowCount * columnCount} samples but found {actual}."); + } + } + } + + /// + /// Gets one encoded sample from an eight-bit or high-bit-depth frame plane. + /// + /// The encoded frame buffer. + /// The plane containing the sample. + /// The horizontal sample coordinate. + /// The vertical sample coordinate. + /// The horizontal subsampling shift. + /// The vertical subsampling shift. + /// The encoded sample value. + private static int GetPlaneSample(Av1FrameBuffer frameBuffer, Av1Plane plane, int x, int y, int subX, int subY) + => frameBuffer.BitDepth == Av1BitDepth.EightBit + ? frameBuffer.DeriveBlockPointer(plane, subX, subY).DangerousGetRowSpan(y)[x] + : frameBuffer.GetHighBitDepthRowSpan(plane, y, subX, subY)[x]; +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/ObuFrameHeaderTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/ObuFrameHeaderTests.cs new file mode 100644 index 0000000000..e4daab57b4 --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/ObuFrameHeaderTests.cs @@ -0,0 +1,1117 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats; +using SixLabors.ImageSharp.Formats.Heif.Av1; +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Memory; +using SixLabors.ImageSharp.Tests.Memory; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1; + +[Trait("Format", "Avif")] +public class ObuFrameHeaderTests +{ + /// + /// Identifies one reference sequence-header conformance condition used by the malformed-input theory. + /// + public enum InvalidSequenceHeaderCase + { + /// + /// Encodes the otherwise valid baseline. + /// + None, + + /// + /// Encodes an unassigned sequence-level index. + /// + UndefinedSequenceLevel, + + /// + /// Encodes an initial display delay greater than ten frames. + /// + InitialDisplayDelayAboveTen, + + /// + /// Encodes a frame identifier wider than sixteen bits. + /// + FrameIdentifierLengthAboveSixteen, + + /// + /// Encodes a zero display-tick unit. + /// + ZeroDisplayTick, + + /// + /// Encodes a zero time scale. + /// + ZeroTimeScale, + + /// + /// Encodes the unsigned-variable-length overflow sentinel. + /// + OverflowingTicksPerPicture, + + /// + /// Encodes the sRGB identity-matrix tuple with the main profile. + /// + MainProfileSrgbIdentity, + + /// + /// Encodes an identity matrix with subsampled components. + /// + SubsampledIdentityMatrix + } + + private static readonly byte[] DefaultSequenceHeaderBitStream = + [0x0a, 0x06, 0b001_1_1_000, 0b00_1000_01, 0b11_110101, 0b001_11101, 0b111_1_1_1_0_1, 0b1_0_0_1_1_1_10]; + + // This complete temporal-delimiter and sequence-header prefix comes from the color item in libavif's + // draw_points_idat_progressive.avif. Its operating points select spatial layers 0+1 and layer 0 respectively. + private static ReadOnlySpan ProgressiveSequenceHeaderBitStream => + [ + 0x12, 0x00, + 0x0A, 0x0F, 0x20, 0x13, 0x01, 0x00, 0x80, 0x81, 0x4E, 0x0A, 0x36, 0xBE, 0x48, 0x08, 0x20, 0x34, 0x80 + ]; + + // Bits Syntax element Value + // 1 obu_forbidden_bit 0 + // 4 obu_type 2 (OBU_TEMPORAL_DELIMITER) + // 1 obu_extension_flag 0 + // 1 obu_has_size_field 1 + // 1 obu_reserved_1bit 0 + // 8 obu_size 0 + private static readonly byte[] DefaultTemporalDelimiterBitStream = [0x12, 0x00]; + + [Theory] + + // [InlineData(TestImages.Heif.IrvineAvif, 0x0198, 0x6bd1)] + [InlineData(TestImages.Heif.XnConvert, 0x010e, 0x03cc)] + [InlineData(TestImages.Heif.Orange4x4, 0x010e, 0x001d)] + public void ReadFrameHeader(string filename, int fileOffset, int blockSize) + { + // Assign + string filePath = Path.Combine(TestEnvironment.InputImagesDirectoryFullPath, filename); + byte[] content = File.ReadAllBytes(filePath); + Span span = content.AsSpan(fileOffset, blockSize); + Av1BitStreamReader reader = new(span); + IAv1TileReader decoder = new Av1TileDecoderStub(); + ObuReader obuReader = new(); + + // Act + obuReader.ReadAll(ref reader, blockSize, () => decoder); + + // Assert + Assert.NotNull(obuReader.SequenceHeader); + Assert.NotNull(obuReader.FrameHeader); + Assert.NotNull(obuReader.FrameHeader.TilesInfo); + Assert.Equal(reader.Length * 8, reader.BitPosition); + Assert.Equal(reader.Length, blockSize); + } + + [Theory] + [InlineData(TestImages.Heif.Orange4x4, 0x010e, 0x001d)] + [InlineData(TestImages.Heif.XnConvert, 0x010e, 0x03cc)] + public void BinaryIdenticalRoundTripFrameHeader(string filename, int fileOffset, int blockSize) + { + // Assign + string filePath = Path.Combine(TestEnvironment.InputImagesDirectoryFullPath, filename); + byte[] content = File.ReadAllBytes(filePath); + Span span = content.AsSpan(fileOffset, blockSize); + Av1TileDecoderStub tileStub = new(); + Av1BitStreamReader reader = new(span); + ObuReader obuReader = new(); + + // Act 1 + obuReader.ReadAll(ref reader, blockSize, () => tileStub); + + // Assign 2 + MemoryStream encoded = new(); + + // Act 2 + using ObuWriter obuWriter = new(Configuration.Default); + obuWriter.WriteSequenceFrame(encoded, obuReader.SequenceHeader, obuReader.FrameHeader, tileStub); + + // Assert + byte[] encodedArray = encoded.ToArray(); + Assert.Equal((ReadOnlySpan)span, encodedArray); + } + + [Theory] + [InlineData(TestImages.Heif.Orange4x4, 0x010e, 0x001d)] + [InlineData(TestImages.Heif.XnConvert, 0x010e, 0x03cc)] + public void ThreeTimeRoundTripFrameHeader(string filename, int fileOffset, int blockSize) + { + // Assign + string filePath = Path.Combine(TestEnvironment.InputImagesDirectoryFullPath, filename); + byte[] content = File.ReadAllBytes(filePath); + Span span = content.AsSpan(fileOffset, blockSize); + Av1TileDecoderStub tileStub = new(); + Av1BitStreamReader reader = new(span); + ObuReader obuReader1 = new(); + + // Act 1 + obuReader1.ReadAll(ref reader, blockSize, () => tileStub); + + // Assign 2 + MemoryStream encoded = new(); + + // Act 2 + using ObuWriter obuWriter = new(Configuration.Default); + obuWriter.WriteSequenceFrame(encoded, obuReader1.SequenceHeader, obuReader1.FrameHeader, tileStub); + + // Assign 2 + Span encodedBuffer = encoded.ToArray(); + IAv1TileReader tileDecoder2 = new Av1TileDecoderStub(); + Av1BitStreamReader reader2 = new(encodedBuffer); + ObuReader obuReader2 = new(); + + // Act 2 + obuReader2.ReadAll(ref reader2, encodedBuffer.Length, () => tileDecoder2); + + // Assert + Assert.Equal(ObuPrettyPrint.PrettyPrintProperties(obuReader1.SequenceHeader.ColorConfig), ObuPrettyPrint.PrettyPrintProperties(obuReader2.SequenceHeader.ColorConfig)); + Assert.Equal(ObuPrettyPrint.PrettyPrintProperties(obuReader1.SequenceHeader), ObuPrettyPrint.PrettyPrintProperties(obuReader2.SequenceHeader)); + Assert.Equal(ObuPrettyPrint.PrettyPrintProperties(obuReader1.FrameHeader), ObuPrettyPrint.PrettyPrintProperties(obuReader2.FrameHeader)); + Assert.Equal(ObuPrettyPrint.PrettyPrintProperties(obuReader1.FrameHeader.TilesInfo), ObuPrettyPrint.PrettyPrintProperties(obuReader2.FrameHeader.TilesInfo)); + } + + [Fact] + public void ReadTemporalDelimiter() + { + // Arrange + Av1BitStreamReader reader = new(DefaultTemporalDelimiterBitStream); + ObuReader obuReader = new(); + IAv1TileReader tileDecoder = new Av1TileDecoderStub(); + + // Act + obuReader.ReadAll(ref reader, DefaultTemporalDelimiterBitStream.Length, () => tileDecoder); + + // Assert + Assert.Null(obuReader.SequenceHeader); + Assert.Null(obuReader.FrameHeader); + } + + [Fact] + public void ReadAnnexBHeaderWithoutSizeField() + { + // Arrange + // Annex B's outer obu_length is one byte and covers the size-less temporal-delimiter header. + byte[] bitStream = [0x01, 0x10]; + Av1BitStreamReader reader = new(bitStream); + ObuReader obuReader = new(); + IAv1TileReader tileDecoder = new Av1TileDecoderStub(); + + // Act + obuReader.ReadAll(ref reader, bitStream.Length, () => tileDecoder, isAnnexB: true); + + // Assert + Assert.Null(obuReader.SequenceHeader); + Assert.Null(obuReader.FrameHeader); + } + + /// + /// Verifies that a final low-overhead frame OBU may use the enclosing image-item boundary instead of an OBU size field. + /// + [Fact] + public void ReadFinalLowOverheadFrameWithoutSizeField() + { + const int itemDataOffset = 0x010E; + const int itemDataLength = 0x001D; + string filePath = Path.Combine(TestEnvironment.InputImagesDirectoryFullPath, TestImages.Heif.Orange4x4); + byte[] fileContent = File.ReadAllBytes(filePath); + byte[] sizedBitStream = fileContent.AsSpan(itemDataOffset, itemDataLength).ToArray(); + int frameOffset = DefaultTemporalDelimiterBitStream.Length; + Av1BitStreamReader sequenceSizeReader = new(sizedBitStream.AsSpan(frameOffset + 1)); + ulong sequencePayloadLength = sequenceSizeReader.ReadLittleEndianBytes128(out int sequenceSizeLength); + frameOffset += 1 + sequenceSizeLength + (int)sequencePayloadLength; + + Av1BitStreamReader sizeReader = new(sizedBitStream.AsSpan(frameOffset + 1)); + ulong framePayloadLength = sizeReader.ReadLittleEndianBytes128(out int encodedSizeLength); + byte[] bitStream = new byte[sizedBitStream.Length - encodedSizeLength]; + + // Preserve the independently encoded frame payload while changing only the final OBU's legal boundary form. + sizedBitStream.AsSpan(0, frameOffset).CopyTo(bitStream); + bitStream[frameOffset] = (byte)(sizedBitStream[frameOffset] & ~0x02); + sizedBitStream.AsSpan(frameOffset + 1 + encodedSizeLength).CopyTo(bitStream.AsSpan(frameOffset + 1)); + + Assert.Equal(framePayloadLength, (ulong)(bitStream.Length - frameOffset - 1)); + ReadObuStream(bitStream); + } + + [Fact] + public void ReadSequenceHeader() + { + // Arrange + byte[] bitStream = DefaultSequenceHeaderBitStream; + Av1BitStreamReader reader = new(bitStream); + ObuReader obuReader = new(); + IAv1TileReader tileDecoder = new Av1TileDecoderStub(); + ObuSequenceHeader expected = GetDefaultSequenceHeader(); + + // Act + obuReader.ReadAll(ref reader, bitStream.Length, () => tileDecoder); + + // Assert + Assert.NotNull(obuReader.SequenceHeader); + Assert.Null(obuReader.FrameHeader); + Assert.Equal(ObuPrettyPrint.PrettyPrintProperties(expected), ObuPrettyPrint.PrettyPrintProperties(obuReader.SequenceHeader)); + } + + /// + /// Verifies reference sequence-header conformance failures through the complete bounded OBU parser. + /// + /// The single invalid syntax condition encoded into an otherwise valid sequence header. + [Theory] + [InlineData(InvalidSequenceHeaderCase.UndefinedSequenceLevel)] + [InlineData(InvalidSequenceHeaderCase.InitialDisplayDelayAboveTen)] + [InlineData(InvalidSequenceHeaderCase.FrameIdentifierLengthAboveSixteen)] + [InlineData(InvalidSequenceHeaderCase.ZeroDisplayTick)] + [InlineData(InvalidSequenceHeaderCase.ZeroTimeScale)] + [InlineData(InvalidSequenceHeaderCase.OverflowingTicksPerPicture)] + [InlineData(InvalidSequenceHeaderCase.MainProfileSrgbIdentity)] + [InlineData(InvalidSequenceHeaderCase.SubsampledIdentityMatrix)] + public void ReadSequenceHeaderRejectsConformanceFailure(InvalidSequenceHeaderCase invalidCase) + { + byte[] bitStream = CreateNonReducedSequenceHeaderObu(invalidCase); + + Assert.Throws(() => ReadObuStream(bitStream)); + } + + /// + /// Verifies that reserved OBU header fields are ignored consistently by container validation and syntax parsing. + /// + /// Whether the sequence header also carries nonzero reserved extension bits. + [Theory] + [InlineData(false)] + [InlineData(true)] + public void ReadSequenceHeaderIgnoresReservedObuHeaderBits(bool hasExtension) + { + byte[] bitStream; + if (hasExtension) + { + bitStream = new byte[DefaultSequenceHeaderBitStream.Length + 1]; + bitStream[0] = (byte)(DefaultSequenceHeaderBitStream[0] | 0x05); + bitStream[1] = 0x07; + DefaultSequenceHeaderBitStream.AsSpan(1).CopyTo(bitStream.AsSpan(2)); + } + else + { + bitStream = [.. DefaultSequenceHeaderBitStream]; + bitStream[0] |= 0x01; + } + + Av1CodecConfiguration configuration = new([0x81, 0x00, 0x00, 0x00], new DecoderOptions()); + configuration.ValidateItemData( + bitStream, + null, + null, + new DecoderOptions(), + out _, + out _); + + ReadObuStream(bitStream); + } + + /// + /// Verifies that metadata-type LEB128 values obey the reference decoder's shared unsigned 32-bit limit. + /// + [Fact] + public void ValidateItemDataRejectsMetadataTypeAboveLimit() + { + byte[] bitStream = + [ + .. DefaultSequenceHeaderBitStream, + + // Metadata type 2^32 followed by valid byte-aligned trailing bits. + 0x2A, 0x06, 0x80, 0x80, 0x80, 0x80, 0x10, 0x80 + ]; + + DecoderOptions options = new() { SegmentIntegrityHandling = SegmentIntegrityHandling.Strict }; + Av1CodecConfiguration configuration = new([0x81, 0x00, 0x00, 0x00], options); + + Assert.Throws( + () => configuration.ValidateItemData( + bitStream, + null, + null, + options, + out _, + out _)); + } + + /// + /// Verifies that an item cannot select an operating-point index absent from its sequence header. + /// + [Fact] + public void ReadOperatingPointRejectsIndexOutsideSequenceHeader() + { + byte[] bitStream = [.. ProgressiveSequenceHeaderBitStream]; + + Assert.Throws(() => ReadObuStream(bitStream, 2)); + } + + /// + /// Verifies that an extended OBU belongs to an operating point only when both of its layer identifiers are selected. + /// + /// The temporal-layer identifier carried by the test OBU. + /// The spatial-layer identifier carried by the test OBU. + /// Whether operating point one selects both identifiers. + [Theory] + [InlineData(0, 0, true)] + [InlineData(0, 1, false)] + [InlineData(1, 0, false)] + public void ReadOperatingPointRequiresBothLayerBits(byte temporalId, byte spatialId, bool isIncluded) + { + byte extension = (byte)((temporalId << 5) | (spatialId << 3)); + byte[] bitStream = [.. ProgressiveSequenceHeaderBitStream, 0x2E, extension, 0x01, 0x00]; + + // A selected metadata OBU reaches ignored-payload validation, where an all-zero payload is invalid. A filtered + // OBU has nevertheless had its complete header, size and payload boundary consumed before syntax is skipped. + if (isIncluded) + { + Assert.Throws(() => ReadObuStream(bitStream, 1)); + } + else + { + ReadObuStream(bitStream, 1); + } + } + + /// + /// Verifies that an all-zero operating-point mask includes every extended OBU. + /// + [Fact] + public void ReadZeroOperatingPointMaskIncludesExtendedObu() + { + byte[] bitStream = [.. DefaultSequenceHeaderBitStream, 0x2E, 0x08, 0x01, 0x00]; + + Assert.Throws(() => ReadObuStream(bitStream)); + } + + /// + /// Verifies that an OBU without an extension header applies to every operating point. + /// + [Fact] + public void ReadOperatingPointIncludesUnextendedObu() + { + byte[] bitStream = [.. ProgressiveSequenceHeaderBitStream, 0x2A, 0x01, 0x00]; + + Assert.Throws(() => ReadObuStream(bitStream, 1)); + } + + /// + /// Verifies that temporal delimiters remain part of stream framing even when their extension is outside the selected mask. + /// + [Fact] + public void ReadOperatingPointDoesNotFilterTemporalDelimiter() + { + byte[] bitStream = [.. ProgressiveSequenceHeaderBitStream, 0x16, 0x08, 0x01, 0x01]; + + Assert.Throws(() => ReadObuStream(bitStream, 1)); + } + + /// + /// Verifies that an excluded OBU cannot escape validation of its declared payload boundary. + /// + [Fact] + public void ReadFilteredOperatingPointObuStillValidatesBoundary() + { + byte[] bitStream = [.. ProgressiveSequenceHeaderBitStream, 0x2E, 0x08, 0x02, 0x80]; + + Assert.Throws(() => ReadObuStream(bitStream, 1)); + } + + /// + /// Verifies that the reduced sequence syntax cannot be used without declaring a still picture. + /// + [Fact] + public void ReducedHeaderWithoutStillPictureThrows() + { + const int sequenceHeaderPayloadOffset = 2; + byte[] bitStream = [.. DefaultSequenceHeaderBitStream]; + + // The first payload byte stores the three profile bits followed by still_picture and + // reduced_still_picture_header. Clear only still_picture so the remaining syntax stays reduced. + bitStream[sequenceHeaderPayloadOffset] &= 0b1110_1111; + + Assert.Throws(() => ReadObuStream(bitStream)); + } + + /// + /// Verifies that a declared still-picture sequence rejects frame prefixes which do not describe a shown key frame. + /// + /// The high nibble containing show-existing, frame-type, and show-frame syntax. + [Theory] + [InlineData(0b1000_0000)] // show_existing_frame = 1 + [InlineData(0b0101_0000)] // frame_type = INTRA_ONLY_FRAME, show_frame = 1 + [InlineData(0b0000_0000)] // frame_type = KEY_FRAME, show_frame = 0 + public void InvalidStillPicturePrefixThrows(int invalidFramePrefix) + { + ObuSequenceHeader sequenceHeader = GetDefaultSequenceHeader(); + sequenceHeader.IsReducedStillPictureHeader = false; + ObuFrameHeader frameHeader = GetKeyFrameHeader(); + Av1TileDecoderStub tileStub = new(); + byte[] emptyTile = []; + tileStub.ReadTile(emptyTile, 0); + + using MemoryStream stream = new(); + using ObuWriter writer = new(Configuration.Default); + writer.WriteSequenceFrame(stream, sequenceHeader, frameHeader, tileStub); + byte[] bitStream = stream.ToArray(); + + int sequenceObuOffset = DefaultTemporalDelimiterBitStream.Length; + Av1BitStreamReader sequenceSizeReader = new(bitStream.AsSpan(sequenceObuOffset + 1)); + ulong sequencePayloadLength = sequenceSizeReader.ReadLittleEndianBytes128(out int sequenceSizeLength); + int frameObuOffset = sequenceObuOffset + 1 + sequenceSizeLength + (int)sequencePayloadLength; + Av1BitStreamReader frameSizeReader = new(bitStream.AsSpan(frameObuOffset + 1)); + _ = frameSizeReader.ReadLittleEndianBytes128(out int frameSizeLength); + int framePayloadOffset = frameObuOffset + 1 + frameSizeLength; + + // Preserve syntax after the frame prefix so each malformed stream differs from a valid writer result only in + // show_existing_frame or the still-picture frame-type/show-frame conformance condition under test. + bitStream[framePayloadOffset] = (byte)((bitStream[framePayloadOffset] & 0x0F) | invalidFramePrefix); + + Assert.Throws(() => ReadObuStream(bitStream)); + } + + /// + /// Verifies that ignored OBU payloads are bounded and skipped before parsing the following sequence header. + /// + [Fact] + public void ReadIgnoredObusSkipsEachDeclaredPayload() + { + // 0x7A identifies padding and 0x4A identifies reserved OBU type 9. Both carry explicit sizes so their payload + // bytes must never be interpreted as another OBU header. + byte[] bitStream = + [ + 0x7A, 0x02, 0x80, 0x00, + 0x4A, 0x01, 0x80, + .. DefaultSequenceHeaderBitStream + ]; + + Av1BitStreamReader reader = new(bitStream); + ObuReader obuReader = new(); + IAv1TileReader tileDecoder = new Av1TileDecoderStub(); + + obuReader.ReadAll(ref reader, bitStream.Length, () => tileDecoder); + + Assert.NotNull(obuReader.SequenceHeader); + Assert.Equal(bitStream.Length * 8, reader.BitPosition); + } + + /// + /// Verifies that content-light metadata is parsed from the OBU and retained for image metadata transfer. + /// + [Fact] + public void ReadMetadataRetainsContentLightLevel() + { + // Metadata type 1 is followed by big-endian MaxCLL and MaxFALL values and byte-aligned trailing bits. + byte[] bitStream = [0x2A, 0x06, 0x01, 0x03, 0xE8, 0x01, 0x90, 0x80]; + Av1BitStreamReader reader = new(bitStream); + ObuReader obuReader = new(); + IAv1TileReader tileDecoder = new Av1TileDecoderStub(); + + obuReader.ReadAll(ref reader, bitStream.Length, tileDecoder); + + Assert.True(obuReader.ContentLightLevel.HasValue); + Assert.Equal((ushort)1_000, obuReader.ContentLightLevel.Value.MaximumContentLightLevel); + Assert.Equal((ushort)400, obuReader.ContentLightLevel.Value.MaximumPictureAverageLightLevel); + Assert.Equal(bitStream.Length * 8, reader.BitPosition); + } + + /// + /// Verifies that fixed-length metadata without its required trailing one bit is rejected. + /// + [Fact] + public void ReadMetadataRejectsMissingTrailingBits() + { + byte[] bitStream = [0x2A, 0x05, 0x01, 0x03, 0xE8, 0x01, 0x90]; + + Assert.Throws(() => ReadObuStream(bitStream)); + } + + /// + /// Verifies that the unsupported tile-list OBU is rejected instead of being treated as ignorable data. + /// + [Fact] + public void ReadTileListRejectsUnsupportedSyntax() + { + byte[] bitStream = [0x42, 0x01, 0x80]; + + Assert.Throws(() => ReadObuStream(bitStream)); + } + + /// + /// Verifies that a four-byte tile size cannot wrap into an empty first tile. + /// + [Fact] + public void ReadTileGroupRejectsFourByteTileSizeOverflow() + { + byte[] bitStream = CreateTwoTileFrame(GetDefaultSequenceHeader(), GetKeyFrameHeader(), 4); + Span encodedTileSize = bitStream.AsSpan(bitStream.Length - 6, 4); + encodedTileSize.Fill(byte.MaxValue); + + Assert.Throws(() => ReadObuStream(bitStream)); + } + + /// + /// Verifies the reference decoder's doubled minimum inner-tile width for a super-resolution-scaled frame. + /// + [Fact] + public void ReadTileInfoRejectsNarrowSuperResolutionInnerTile() + { + ObuSequenceHeader sequenceHeader = GetDefaultSequenceHeader(); + sequenceHeader.Use128x128Superblock = false; + sequenceHeader.EnableSuperResolution = true; + sequenceHeader.FrameWidthBits = 8; + sequenceHeader.MaxFrameWidth = 192; + + ObuFrameHeader frameHeader = GetKeyFrameHeader(); + frameHeader.FrameSize.FrameWidth = 96; + frameHeader.FrameSize.SuperResolutionUpscaledWidth = 192; + frameHeader.FrameSize.RenderWidth = 192; + frameHeader.FrameSize.SuperResolutionDenominator = 16; + frameHeader.ModeInfoColumnCount = 24; + + byte[] bitStream = CreateTwoTileFrame(sequenceHeader, frameHeader, 1); + + Assert.Throws(() => ReadObuStream(bitStream)); + } + + /// + /// Verifies that an intra-only frame cannot signal the all-slots refresh mask reserved for key and switch frames. + /// + [Fact] + public void ReadFrameHeaderRejectsIntraOnlyAllSlotsRefresh() + { + byte[] sequenceHeader = CreateNonReducedSequenceHeaderObu(default); + byte[] framePayload = new byte[2]; + Av1BitStreamWriter frameWriter = new(framePayload); + + frameWriter.WriteBoolean(false); + frameWriter.WriteLiteral((uint)ObuFrameType.IntraOnlyFrame, 2); + frameWriter.WriteBoolean(true); + frameWriter.WriteBoolean(true); + frameWriter.WriteBoolean(false); + frameWriter.WriteBoolean(false); + frameWriter.WriteLiteral(byte.MaxValue, 8); + + int framePayloadLength = (frameWriter.BitPosition + 7) >> 3; + frameWriter.Flush(); + + byte[] bitStream = new byte[sequenceHeader.Length + 2 + framePayloadLength]; + sequenceHeader.CopyTo(bitStream, 0); + int frameObuOffset = sequenceHeader.Length; + bitStream[frameObuOffset] = (byte)(((byte)ObuType.FrameHeader << 3) | 0x02); + bitStream[frameObuOffset + 1] = (byte)framePayloadLength; + framePayload.AsSpan(0, framePayloadLength).CopyTo(bitStream.AsSpan(frameObuOffset + 2)); + + Assert.Throws(() => ReadObuStream(bitStream)); + } + + /// + /// Verifies that invalid OBU boundaries, size fields, and trailing bytes are rejected. + /// + /// The malformed OBU stream. + [Theory] + [InlineData(new byte[] { 0x7A, 0x02, 0x11 })] + [InlineData(new byte[] { 0x7A, 0x01, 0x00 })] + [InlineData(new byte[] { 0x12, 0x01, 0x01 })] + [InlineData(new byte[] { 0x7A, 0x80, 0x80, 0x80, 0x80, 0x80, 0x80, 0x80, 0x80 })] + public void ReadInvalidObuBoundaryThrows(byte[] bitStream) + => Assert.Throws(() => ReadObuStream(bitStream)); + + /// + /// Verifies that an empty temporal delimiter may occupy a payload containing only zero padding bytes. + /// + [Fact] + public void ReadTemporalDelimiterAllowsZeroPayloadPadding() + { + byte[] bitStream = [0x12, 0x02, 0x00, 0x00]; + Exception exception = Record.Exception(() => ReadObuStream(bitStream)); + + Assert.Null(exception); + } + + [Fact] + public void WriteTemporalDelimiter() + { + // Arrange + using MemoryStream stream = new(2); + + // Act + ObuWriter.WriteTemporalDelimiter(stream); + byte[] actual = stream.GetBuffer(); + + // Assert + Assert.Equal(DefaultTemporalDelimiterBitStream, actual); + } + + [Fact] + public void WriteSequenceHeader() + { + // Arrange + using MemoryStream stream = new(10); + ObuSequenceHeader input = GetDefaultSequenceHeader(); + + // Act + ObuWriter.WriteSequenceHeader(Configuration.Default, stream, input); + byte[] buffer = stream.GetBuffer(); + + // Assert + // Skip over Temporal Delimiter header. + byte[] actual = buffer.AsSpan()[DefaultTemporalDelimiterBitStream.Length..].ToArray(); + Assert.Equal(DefaultSequenceHeaderBitStream, actual); + } + + [Fact] + public void WriteNonReducedSequenceHeaderPreservesTimingAndDecoderModel() + { + ObuSequenceHeader input = GetDefaultSequenceHeader(); + input.IsStillPicture = false; + input.IsReducedStillPictureHeader = false; + input.TimingInfoPresentFlag = true; + input.TimingInfo = new ObuTimingInfo + { + NumUnitsInDisplayTick = 1001, + TimeScale = 60000, + EqualPictureInterval = true, + NumTicksPerPicture = 2 + }; + + input.DecoderModelInfoPresentFlag = true; + input.DecoderModelInfo = new ObuDecoderModelInfo + { + BufferDelayLength = 12, + NumUnitsInDecodingTick = 1000, + BufferRemovalTimeLength = 10, + FramePresentationTimeLength = 9 + }; + + input.InitialDisplayDelayPresentFlag = true; + ObuOperatingPoint inputOperatingPoint = input.OperatingPoint[0]; + inputOperatingPoint.SequenceLevelIndex = Av1Constants.SequenceTierMinimumLevelIndex; + inputOperatingPoint.SequenceTier = 1; + inputOperatingPoint.IsDecoderModelInfoPresent = true; + inputOperatingPoint.DecoderBufferDelay = 137; + inputOperatingPoint.EncoderBufferDelay = 211; + inputOperatingPoint.LowDelayMode = true; + inputOperatingPoint.IsInitialDisplayDelayPresent = true; + inputOperatingPoint.InitialDisplayDelay = 4; + + using MemoryStream stream = new(); + ObuWriter.WriteSequenceHeader(Configuration.Default, stream, input); + byte[] bitStream = stream.ToArray(); + Av1BitStreamReader reader = new(bitStream); + ObuReader obuReader = new(); + obuReader.ReadAll(ref reader, bitStream.Length, () => new Av1TileDecoderStub()); + + ObuSequenceHeader output = obuReader.SequenceHeader; + ObuTimingInfo outputTiming = output.GetTimingInfo(); + ObuDecoderModelInfo outputDecoderModel = output.GetDecoderModelInfo(); + ObuOperatingPoint outputOperatingPoint = output.OperatingPoint[0]; + Assert.False(output.IsStillPicture); + Assert.False(output.IsReducedStillPictureHeader); + Assert.Equal(input.TimingInfo.NumUnitsInDisplayTick, outputTiming.NumUnitsInDisplayTick); + Assert.Equal(input.TimingInfo.TimeScale, outputTiming.TimeScale); + Assert.Equal(input.TimingInfo.NumTicksPerPicture, outputTiming.NumTicksPerPicture); + Assert.Equal(input.DecoderModelInfo.BufferDelayLength, outputDecoderModel.BufferDelayLength); + Assert.Equal(input.DecoderModelInfo.NumUnitsInDecodingTick, outputDecoderModel.NumUnitsInDecodingTick); + Assert.Equal(input.DecoderModelInfo.BufferRemovalTimeLength, outputDecoderModel.BufferRemovalTimeLength); + Assert.Equal(input.DecoderModelInfo.FramePresentationTimeLength, outputDecoderModel.FramePresentationTimeLength); + Assert.Equal(inputOperatingPoint.SequenceTier, outputOperatingPoint.SequenceTier); + Assert.Equal(inputOperatingPoint.DecoderBufferDelay, outputOperatingPoint.DecoderBufferDelay); + Assert.Equal(inputOperatingPoint.EncoderBufferDelay, outputOperatingPoint.EncoderBufferDelay); + Assert.Equal(inputOperatingPoint.LowDelayMode, outputOperatingPoint.LowDelayMode); + Assert.Equal(inputOperatingPoint.InitialDisplayDelay, outputOperatingPoint.InitialDisplayDelay); + } + + /// + /// Verifies that the combined frame OBU declares exactly the payload bytes emitted by the writer. + /// + [Fact] + public void WriteFrameHeader() + { + // Arrange + using MemoryStream stream = new(10); + ObuSequenceHeader sequenceInput = GetDefaultSequenceHeader(); + ObuFrameHeader frameInput = GetKeyFrameHeader(); + Av1TileDecoderStub tileStub = new(); + byte[] tileData = [0x80]; + tileStub.ReadTile(tileData, 0); + using ObuWriter obuWriter = new(Configuration.Default); + + // Act + obuWriter.WriteSequenceFrame(stream, sequenceInput, frameInput, tileStub); + byte[] bitStream = stream.ToArray(); + + // Assert + int frameOffset = DefaultTemporalDelimiterBitStream.Length + DefaultSequenceHeaderBitStream.Length; + Span frameObu = bitStream.AsSpan(frameOffset); + byte expectedHeader = (byte)(((byte)ObuType.Frame << 3) | 0x02); + Assert.Equal(expectedHeader, frameObu[0]); + + Av1BitStreamReader sizeReader = new(frameObu[1..]); + ulong declaredPayloadSize = sizeReader.ReadLittleEndianBytes128(out int encodedSizeLength); + Assert.Equal(frameObu.Length - 1 - encodedSizeLength, (int)declaredPayloadSize); + + Av1BitStreamReader reader = new(bitStream); + ObuReader obuReader = new(); + obuReader.ReadAll(ref reader, bitStream.Length, () => new Av1TileDecoderStub()); + + Assert.NotNull(obuReader.SequenceHeader); + Assert.NotNull(obuReader.FrameHeader); + Assert.Equal(bitStream.Length * 8, reader.BitPosition); + } + + /// + /// Verifies that the OBU writer reuses header scratch and streams encoded tiles without a payload-sized copy. + /// + [Fact] + public void WriterReusesHeaderScratchAndStreamsTilePayloadWithoutCopy() + { + const int TilePayloadLength = 64 * 1024; + + TestMemoryAllocator allocator = new(); + allocator.EnableNonThreadSafeLogging(); + Configuration configuration = new(); + configuration.MemoryAllocator = allocator; + ObuSequenceHeader sequenceHeader = GetDefaultSequenceHeader(); + ObuFrameHeader frameHeader = GetKeyFrameHeader(); + byte[] tileData = new byte[TilePayloadLength]; + tileData.AsSpan().Fill(0x80); + Av1TileDecoderStub tileStub = new(); + tileStub.ReadTile(tileData, 0); + + using MemoryStream stream = new(); + using (ObuWriter writer = new(configuration)) + { + writer.WriteSequenceFrame(stream, sequenceHeader, frameHeader, tileStub); + + // A second complete write must reuse the same bounded header owner retained by the writer. + writer.WriteSequenceFrame(stream, sequenceHeader, frameHeader, tileStub); + } + + TestMemoryAllocator.AllocationRequest headerScratch = Assert.Single(allocator.AllocationLog); + Assert.Equal(typeof(byte), headerScratch.ElementType); + Assert.InRange(headerScratch.Length, 1, TilePayloadLength - 1); + TestMemoryAllocator.ReturnRequest returned = Assert.Single(allocator.ReturnLog); + Assert.Equal(headerScratch.AllocationId, returned.AllocationId); + Assert.True(stream.GetBuffer().AsSpan((int)stream.Length - TilePayloadLength, TilePayloadLength).SequenceEqual(tileData)); + } + + /// + /// Verifies non-uniform tile boundaries use the next stored boundary and retain the clipped final mode-info edge. + /// + [Fact] + public void WriteNonUniformTileBoundariesRoundTrip() + { + ObuSequenceHeader sequenceHeader = GetDefaultSequenceHeader(); + ObuFrameHeader frameHeader = GetKeyFrameHeader(); + ObuTileGroupHeader tileInfo = frameHeader.TilesInfo; + tileInfo.HasUniformTileSpacing = false; + tileInfo.TileColumnCount = 2; + tileInfo.TileRowCount = 1; + tileInfo.TileSizeBytes = 1; + tileInfo.TileColumnStartModeInfo[0] = 0; + tileInfo.TileColumnStartModeInfo[1] = 64; + tileInfo.TileColumnStartModeInfo[2] = frameHeader.ModeInfoColumnCount; + tileInfo.TileRowStartModeInfo[0] = 0; + tileInfo.TileRowStartModeInfo[1] = frameHeader.ModeInfoRowCount; + + Av1TileDecoderStub sourceTiles = new(); + sourceTiles.ReadTile([0x80], 0); + sourceTiles.ReadTile([0x80], 1); + + using MemoryStream stream = new(); + using ObuWriter writer = new(Configuration.Default); + writer.WriteSequenceFrame(stream, sequenceHeader, frameHeader, sourceTiles); + byte[] bitStream = stream.ToArray(); + Assert.Equal([0x00, 0x80, 0x80], bitStream[^3..]); + + Av1BitStreamReader reader = new(bitStream); + ObuReader obuReader = new(); + Av1TileDecoderStub decodedTiles = new(); + + obuReader.ReadAll(ref reader, bitStream.Length, () => decodedTiles); + + ObuTileGroupHeader actual = obuReader.FrameHeader.TilesInfo; + Assert.False(actual.HasUniformTileSpacing); + Assert.Equal(2, actual.TileColumnCount); + Assert.Equal(1, actual.TileRowCount); + Assert.Equal(1, actual.TileSizeBytes); + Assert.Equal(0, actual.TileColumnStartModeInfo[0]); + Assert.Equal(64, actual.TileColumnStartModeInfo[1]); + Assert.Equal(frameHeader.ModeInfoColumnCount, actual.TileColumnStartModeInfo[2]); + Assert.Equal(0, actual.TileRowStartModeInfo[0]); + Assert.Equal(frameHeader.ModeInfoRowCount, actual.TileRowStartModeInfo[1]); + + ReadOnlySpan expectedTileData = [0x80]; + + Assert.True(decodedTiles.GetTileData(0).SequenceEqual(expectedTileData)); + Assert.True(decodedTiles.GetTileData(1).SequenceEqual(expectedTileData)); + Assert.Equal(bitStream.Length * 8, reader.BitPosition); + } + + /// + /// Encodes one non-reduced sequence header with a single selected conformance failure. + /// + /// The syntax condition to make invalid, or . + /// The complete explicitly sized sequence-header OBU. + private static byte[] CreateNonReducedSequenceHeaderObu(InvalidSequenceHeaderCase invalidCase) + { + const int SequenceHeaderBufferLength = 32; + + bool hasTimingInfo = invalidCase is + InvalidSequenceHeaderCase.ZeroDisplayTick or + InvalidSequenceHeaderCase.ZeroTimeScale or + InvalidSequenceHeaderCase.OverflowingTicksPerPicture; + + bool hasInitialDisplayDelay = invalidCase == InvalidSequenceHeaderCase.InitialDisplayDelayAboveTen; + bool hasFrameIdentifiers = invalidCase == InvalidSequenceHeaderCase.FrameIdentifierLengthAboveSixteen; + bool hasColorDescription = invalidCase is + InvalidSequenceHeaderCase.MainProfileSrgbIdentity or + InvalidSequenceHeaderCase.SubsampledIdentityMatrix; + + byte[] payloadBuffer = new byte[SequenceHeaderBufferLength]; + Av1BitStreamWriter writer = new(payloadBuffer); + writer.WriteLiteral((uint)ObuSequenceProfile.Main, 3); + writer.WriteBoolean(false); + writer.WriteBoolean(false); + writer.WriteBoolean(hasTimingInfo); + if (hasTimingInfo) + { + writer.WriteLiteral(invalidCase == InvalidSequenceHeaderCase.ZeroDisplayTick ? 0U : 1U, 32); + writer.WriteLiteral(invalidCase == InvalidSequenceHeaderCase.ZeroTimeScale ? 0U : 1U, 32); + + bool overflowingTicksPerPicture = invalidCase == InvalidSequenceHeaderCase.OverflowingTicksPerPicture; + writer.WriteBoolean(overflowingTicksPerPicture); + if (overflowingTicksPerPicture) + { + // Thirty-two leading zeros are the UVLC sentinel which the reference decoder rejects as UINT32_MAX. + writer.WriteLiteral(0U, 32); + } + + writer.WriteBoolean(false); + } + + writer.WriteBoolean(hasInitialDisplayDelay); + writer.WriteLiteral(0U, 5); + writer.WriteLiteral(0U, 12); + + uint sequenceLevel = invalidCase == InvalidSequenceHeaderCase.UndefinedSequenceLevel ? 24U : 0U; + writer.WriteLiteral(sequenceLevel, 5); + if (sequenceLevel > 7) + { + writer.WriteBoolean(false); + } + + if (hasInitialDisplayDelay) + { + writer.WriteBoolean(true); + writer.WriteLiteral(10U, 4); + } + + writer.WriteLiteral(7U, 4); + writer.WriteLiteral(7U, 4); + writer.WriteLiteral(63U, 8); + writer.WriteLiteral(63U, 8); + writer.WriteBoolean(hasFrameIdentifiers); + if (hasFrameIdentifiers) + { + writer.WriteLiteral(15U, 4); + writer.WriteLiteral(0U, 3); + } + + // Disable superblock and intra-edge tools. + writer.WriteBoolean(false); + writer.WriteBoolean(false); + writer.WriteBoolean(false); + + // Disable the inter compound, warped, dual-filter, and order-hint tools. + writer.WriteBoolean(false); + writer.WriteBoolean(false); + writer.WriteBoolean(false); + writer.WriteBoolean(false); + writer.WriteBoolean(false); + + // Select fixed disabled screen-content and integer-motion-vector behavior. + writer.WriteBoolean(false); + writer.WriteBoolean(false); + + // Disable super resolution, CDEF, and restoration in the otherwise valid baseline. + writer.WriteBoolean(false); + writer.WriteBoolean(false); + writer.WriteBoolean(false); + + // Encode an 8-bit, non-monochrome color configuration. + writer.WriteBoolean(false); + writer.WriteBoolean(false); + writer.WriteBoolean(hasColorDescription); + if (hasColorDescription) + { + bool isSrgbIdentity = invalidCase == InvalidSequenceHeaderCase.MainProfileSrgbIdentity; + writer.WriteLiteral((uint)(isSrgbIdentity ? ObuColorPrimaries.Bt709 : ObuColorPrimaries.Unspecified), 8); + writer.WriteLiteral((uint)(isSrgbIdentity ? ObuTransferCharacteristics.Srgb : ObuTransferCharacteristics.Unspecified), 8); + writer.WriteLiteral((uint)ObuMatrixCoefficients.Identity, 8); + } + + if (invalidCase != InvalidSequenceHeaderCase.MainProfileSrgbIdentity) + { + writer.WriteBoolean(false); + writer.WriteLiteral((uint)ObuChromoSamplePosition.Unknown, 2); + } + + writer.WriteBoolean(false); + writer.WriteBoolean(false); + + int trailingBitCount = 8 - (writer.BitPosition & 0x07); + writer.WriteLiteral(1U << (trailingBitCount - 1), trailingBitCount); + + int payloadLength = (writer.BitPosition + 7) >> 3; + writer.Flush(); + + byte[] obu = new byte[payloadLength + 2]; + obu[0] = (byte)(((byte)ObuType.SequenceHeader << 3) | 0x02); + obu[1] = (byte)payloadLength; + payloadBuffer.AsSpan(0, payloadLength).CopyTo(obu.AsSpan(2)); + return obu; + } + + /// + /// Encodes one valid reduced frame with two one-byte tile payloads. + /// + /// The sequence syntax to encode. + /// The frame syntax to encode. + /// The number of bytes used for the first tile's size field. + /// The complete temporal delimiter, sequence header, and combined frame OBU stream. + private static byte[] CreateTwoTileFrame( + ObuSequenceHeader sequenceHeader, + ObuFrameHeader frameHeader, + int tileSizeBytes) + { + frameHeader.TilesInfo.HasUniformTileSpacing = true; + frameHeader.TilesInfo.TileColumnCount = 2; + frameHeader.TilesInfo.TileRowCount = 1; + frameHeader.TilesInfo.TileSizeBytes = tileSizeBytes; + + Av1TileDecoderStub tileStub = new(); + tileStub.ReadTile([0x80], 0); + tileStub.ReadTile([0x80], 1); + + using MemoryStream stream = new(); + using ObuWriter writer = new(Configuration.Default); + writer.WriteSequenceFrame(stream, sequenceHeader, frameHeader, tileStub); + return stream.ToArray(); + } + + private static ObuSequenceHeader GetDefaultSequenceHeader() + + // Offset Bits Syntax element Value + // 0 3 seq_profile 1 + // 3 1 still_picture 1 + // 4 1 reduced_still_picture_header 1 + // 5 5 seq_level_idx[ 0 ] 0 + // 10 4 frame_width_bits_minus_1 8 + // 14 4 frame_height_bits_minus_1 7 + // 18 9 max_frame_width_minus_1 425 + // 27 8 max_frame_height_minus_1 239 + // 35 1 use_128x128_superblock 1 + // 36 1 enable_filter_intra 1 + // 37 1 enable_intra_edge_filter 1 + // 38 1 enable_superres 0 + // 39 1 enable_cdef 1 + // 40 1 enable_restoration 1 + // 41 1 ColorConfig.BitDepth.HasHighBit 0 + // 42 1 ColorConfig.IsDescriptionPresent 0 + // 43 1 ColorConfig.ColorRange 1 + // 44 1 ColorConfig.HasSeparateUVDelta 1 + // 45 1 film_grain_present 1 + // 47 2 Trailing bits 2 + => new() + { + SequenceProfile = ObuSequenceProfile.High, + IsStillPicture = true, + IsReducedStillPictureHeader = true, + TimingInfoPresentFlag = false, + InitialDisplayDelayPresentFlag = false, + FrameWidthBits = 8 + 1, + FrameHeightBits = 7 + 1, + MaxFrameWidth = 425 + 1, + MaxFrameHeight = 239 + 1, + IsFrameIdNumbersPresent = false, + Use128x128Superblock = true, + EnableFilterIntra = true, + EnableIntraEdgeFilter = true, + EnableInterIntraCompound = false, + EnableMaskedCompound = false, + EnableWarpedMotion = false, + EnableDualFilter = false, + EnableOrderHint = false, + OperatingPoint = [new()], + + // EnableJountCompound = true, + // EnableReferenceFrameMotionVectors = true, + ForceScreenContentTools = 2, + ForceIntegerMotionVector = 2, + EnableSuperResolution = false, + EnableCdef = true, + EnableRestoration = true, + ColorConfig = new() + { + IsMonochrome = false, + ColorPrimaries = ObuColorPrimaries.Unspecified, + TransferCharacteristics = ObuTransferCharacteristics.Unspecified, + MatrixCoefficients = ObuMatrixCoefficients.Unspecified, + SubSamplingX = false, + SubSamplingY = false, + BitDepth = Av1BitDepth.EightBit, + HasSeparateUvDelta = true, + ColorRange = true, + }, + AreFilmGrainingParametersPresent = true, + }; + + /// + /// Reads one complete OBU stream for malformed-input assertions that cannot capture a ref-struct reader. + /// + /// The complete encoded OBU stream. + private static void ReadObuStream(byte[] bitStream, byte operatingPointIndex = 0) + { + Av1BitStreamReader reader = new(bitStream); + ObuReader obuReader = new(operatingPointIndex); + IAv1TileReader tileDecoder = new Av1TileDecoderStub(); + + obuReader.ReadAll(ref reader, bitStream.Length, () => tileDecoder); + } + + private static ObuFrameHeader GetKeyFrameHeader() + => new() + { + FrameType = ObuFrameType.KeyFrame, + ShowFrame = true, + ShowableFrame = false, + DisableFrameEndUpdateCdf = false, + FrameSize = new() + { + FrameWidth = 426, + FrameHeight = 240, + RenderWidth = 426, + RenderHeight = 240, + SuperResolutionUpscaledWidth = 426, + }, + PrimaryReferenceFrame = 7, + ModeInfoRowCount = 60, + ModeInfoColumnCount = 108, + RefreshFrameFlags = 0xff, + ErrorResilientMode = true, + ForceIntegerMotionVector = true, + TilesInfo = new ObuTileGroupHeader() + { + HasUniformTileSpacing = true, + TileColumnCount = 1, + TileRowCount = 1, + } + }; +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/ObuFrameLifecycleTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/ObuFrameLifecycleTests.cs new file mode 100644 index 0000000000..727061d785 --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/ObuFrameLifecycleTests.cs @@ -0,0 +1,454 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1; +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Formats.Heif.Av1.ReferenceFrames; +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; +using SixLabors.ImageSharp.PixelFormats; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1; + +/// +/// Verifies frame ownership and completion while parsing layered AV1 OBU payloads. +/// +[Trait("Format", "Avif")] +public class ObuFrameLifecycleTests +{ + private const int ProgressiveSequencePrefixLength = 19; + private const int FirstProgressiveLayerLength = 55; + private const int NoFailingReaderIndex = -1; + private const int SecondFrameReaderIndex = 1; + private const byte ForbiddenObuHeader = 0x80; + private const byte InvalidTrailingByte = 1; + + /// + /// The operating point in the fixture sequence header that selects both spatial layers. + /// + private const byte ProgressiveOperatingPointIndex = 0; + + /// + /// The source and displayed width recorded by libavif for both progressive layers. + /// + private const int ProgressiveImageWidth = 33; + + /// + /// The source and displayed height recorded by libavif for both progressive layers. + /// + private const int ProgressiveImageHeight = 11; + + // This is the complete 72-byte color-item payload assembled from both extents of libavif's + // draw_points_idat_progressive.avif. It contains one sequence header followed by two coded + // spatial layers, so the lifecycle test exercises real progressive item framing. + private static ReadOnlySpan ProgressiveTwoFrameObuStream => + [ + + // Temporal delimiter and progressive sequence header. + 0x12, 0x00, + 0x0A, 0x0F, 0x20, 0x13, 0x01, 0x00, 0x80, 0x81, 0x4E, 0x0A, 0x36, 0xBE, 0x48, 0x08, 0x20, 0x34, 0x80, + + // Base spatial layer: unextended combined-frame OBU with a 34-byte payload. + 0x32, 0x22, 0x14, 0x00, 0x27, 0xC0, 0x00, 0x00, 0x80, 0x00, 0x20, 0xF4, 0xAC, 0x60, 0x4B, 0x59, + 0xB6, 0x97, 0xB3, 0xD1, 0xF9, 0x22, 0xB7, 0x5B, 0xAC, 0xD5, 0xD1, 0x99, 0x8C, 0x5F, 0x30, 0x67, + 0xB4, 0x6C, 0x99, 0x80, + + // Enhancement spatial layer: extended combined-frame OBU with temporal_id 0 and spatial_id 1. + 0x36, 0x08, 0x0E, 0x33, 0x01, 0xC0, 0x20, 0x00, 0x00, 0x06, 0x80, 0x01, 0x00, 0xF3, 0xA2, 0xD4, 0x38 + ]; + + /// + /// Verifies that a bounded progressive payload is consumed completely and assigns fresh tile state to each coded frame. + /// + [Fact] + public void ReadAllConsumesBothProgressiveFrames() + { + const int temporalId = 0; + const int baseSpatialId = 0; + const int enhancementSpatialId = 1; + byte[] bitStream = [.. ProgressiveTwoFrameObuStream]; + Av1BitStreamReader reader = new(bitStream); + using Av1ReferenceFrameStore referenceFrames = new(); + ObuReader obuReader = new(ProgressiveOperatingPointIndex, referenceFrames); + LifecycleTileReaderFactory factory = new(obuReader, referenceFrames, NoFailingReaderIndex); + + obuReader.ReadAll(ref reader, bitStream.Length, factory.Create); + + ObuFrameHeader secondFrameHeader = Assert.IsType(factory.Readers[1].CompletedFrameHeader); + + Assert.Equal(bitStream.Length * 8, reader.BitPosition); + Assert.Equal(2, factory.Readers.Count); + Assert.NotSame(factory.Readers[0], factory.Readers[1]); + Assert.All(factory.Readers, frameReader => Assert.Equal(1, frameReader.TileCount)); + Assert.All(factory.Readers, frameReader => Assert.Equal(1, frameReader.CompletionCount)); + Assert.All(factory.Readers, frameReader => Assert.Equal(1, frameReader.TileCountAtCompletion)); + Assert.Equal(temporalId, factory.Readers[0].CompletedTemporalId); + Assert.Equal(baseSpatialId, factory.Readers[0].CompletedSpatialId); + Assert.Equal(temporalId, factory.Readers[1].CompletedTemporalId); + Assert.Equal(enhancementSpatialId, factory.Readers[1].CompletedSpatialId); + Assert.True(factory.Readers[1].SelectedReferencesWereRetained); + Assert.Equal(ObuFrameType.InterFrame, secondFrameHeader.FrameType); + Assert.Equal(ProgressiveImageWidth, secondFrameHeader.FrameSize.SuperResolutionUpscaledWidth); + Assert.Equal(ProgressiveImageHeight, secondFrameHeader.FrameSize.FrameHeight); + Assert.Equal(ProgressiveImageWidth, secondFrameHeader.FrameSize.RenderWidth); + Assert.Equal(ProgressiveImageHeight, secondFrameHeader.FrameSize.RenderHeight); + } + + /// + /// Verifies that production existing-frame presentation needs no tile reader, frame buffer, or reconstruction graph. + /// + [Fact] + public void SequenceDecoderPresentsExistingFrameWithoutTileState() + { + byte[] bitStream = [.. ProgressiveTwoFrameObuStream]; + + // A standalone frame-header OBU selects retained slot zero. The payload contains show_existing_frame, the + // three-bit slot index, and the required trailing-one bit; no tile-group OBU follows it. + byte[] showExistingFrame = [0x1A, 0x01, 0x88]; + using Av1Decoder decoder = new(Configuration.Default, ProgressiveOperatingPointIndex); + using ImageFrame reconstructed = decoder.DecodeSequenceFrame(bitStream, null, null); + using ImageFrame existing = decoder.DecodeSequenceFrame(showExistingFrame, null, null); + + Assert.Equal(reconstructed.Size, existing.Size); + for (int row = 0; row < reconstructed.Height; row++) + { + Assert.True( + reconstructed.PixelBuffer.DangerousGetRowSpan(row) + .SequenceEqual(existing.PixelBuffer.DangerousGetRowSpan(row))); + } + + Assert.Null(decoder.FrameInfo); + } + + /// + /// Verifies that invalid trailing data in a subsequent sequence header is rejected before that sequence can create frame state. + /// + [Fact] + public void RejectsMalformedFollowingSequenceBeforeFrameReader() + { + byte[] firstLayer = ProgressiveTwoFrameObuStream[..FirstProgressiveLayerLength].ToArray(); + byte[] malformedSequence = AddInvalidSequenceHeaderTrailingByte(ProgressiveTwoFrameObuStream[..ProgressiveSequencePrefixLength].ToArray()); + byte[] bitStream = [.. firstLayer, .. malformedSequence]; + using Av1ReferenceFrameStore referenceFrames = new(); + ObuReader obuReader = new(ProgressiveOperatingPointIndex, referenceFrames); + LifecycleTileReaderFactory factory = new(obuReader, referenceFrames, NoFailingReaderIndex); + + Assert.Throws(() => ReadObuStream(bitStream, obuReader, factory.Create)); + AssertParserSessionReset(obuReader, referenceFrames); + LifecycleTileReader completedFrame = Assert.Single(factory.Readers); + + Assert.Equal(1, completedFrame.TileCount); + Assert.Equal(1, completedFrame.CompletionCount); + + ReadObuStream(ProgressiveTwoFrameObuStream.ToArray(), obuReader, factory.Create); + + Assert.Equal(3, factory.Readers.Count); + Assert.Equal(1, factory.Readers[1].CompletionCount); + Assert.Equal(1, factory.Readers[2].CompletionCount); + } + + /// + /// Verifies that synchronous tile validation failure prevents completion of the affected coded frame. + /// + [Fact] + public void ReadAllDoesNotCompleteFrameWhenTileValidationFails() + { + byte[] bitStream = [.. ProgressiveTwoFrameObuStream]; + using Av1ReferenceFrameStore referenceFrames = new(); + ObuReader obuReader = new(ProgressiveOperatingPointIndex, referenceFrames); + LifecycleTileReaderFactory factory = new(obuReader, referenceFrames, SecondFrameReaderIndex); + + Assert.Throws(() => ReadObuStream(bitStream, obuReader, factory.Create)); + Assert.Equal(2, factory.Readers.Count); + Assert.Equal(1, factory.Readers[0].CompletionCount); + Assert.Equal(1, factory.Readers[1].TileCount); + Assert.Equal(0, factory.Readers[1].CompletionCount); + AssertParserSessionReset(obuReader, referenceFrames); + + ReadObuStream(bitStream, obuReader, factory.Create); + + Assert.Equal(4, factory.Readers.Count); + Assert.Equal(1, factory.Readers[2].CompletionCount); + Assert.Equal(1, factory.Readers[3].CompletionCount); + } + + /// + /// Verifies that malformed data after a completed real frame invalidates retained state without preventing reuse of the parser. + /// + [Fact] + public void ClearsCompletedFrameStateAfterInvalidObuHeader() + { + byte[] bitStream = [.. ProgressiveTwoFrameObuStream[..FirstProgressiveLayerLength], ForbiddenObuHeader]; + using Av1ReferenceFrameStore referenceFrames = new(); + ObuReader obuReader = new(ProgressiveOperatingPointIndex, referenceFrames); + LifecycleTileReaderFactory factory = new(obuReader, referenceFrames, NoFailingReaderIndex); + + Assert.Throws(() => ReadObuStream(bitStream, obuReader, factory.Create)); + AssertParserSessionReset(obuReader, referenceFrames); + Assert.Equal(1, Assert.Single(factory.Readers).CompletionCount); + + ReadObuStream(ProgressiveTwoFrameObuStream.ToArray(), obuReader, factory.Create); + + Assert.Equal(3, factory.Readers.Count); + Assert.Equal(1, factory.Readers[1].CompletionCount); + Assert.Equal(1, factory.Readers[2].CompletionCount); + } + + /// + /// Verifies that a combined frame OBU cannot use the header-only retained-frame presentation form. + /// + [Fact] + public void ReadAllRejectsShowExistingFrameInCombinedFrameObu() + { + byte[] bitStream = + [ + .. ProgressiveTwoFrameObuStream[..FirstProgressiveLayerLength], + + // A one-byte combined-frame payload selecting retained slot zero. The reference decoder rejects this form + // because show_existing_frame is permitted only in a standalone frame-header OBU. + 0x32, 0x01, 0x80 + ]; + + using Av1ReferenceFrameStore referenceFrames = new(); + ObuReader obuReader = new(ProgressiveOperatingPointIndex, referenceFrames); + LifecycleTileReaderFactory factory = new(obuReader, referenceFrames, NoFailingReaderIndex); + + Assert.Throws(() => ReadObuStream(bitStream, obuReader, factory.Create)); + AssertParserSessionReset(obuReader, referenceFrames); + Assert.Equal(1, Assert.Single(factory.Readers).CompletionCount); + } + + /// + /// Extends the sequence-header OBU by one nonzero byte while retaining all following encoded frame bytes. + /// + /// A temporal-delimiter and sequence-header OBU prefix. + /// A stream whose sequence-header syntax has invalid nonzero trailing data. + private static byte[] AddInvalidSequenceHeaderTrailingByte(byte[] stream) + { + int sequenceObuOffset = GetNextObuOffset(stream, 0); + Av1BitStreamReader sizeReader = new(stream.AsSpan(sequenceObuOffset + 1)); + uint sequencePayloadLength = (uint)sizeReader.ReadLittleEndianBytes128(out int oldSizeLength); + int sequencePayloadOffset = sequenceObuOffset + 1 + oldSizeLength; + int frameObuOffset = sequencePayloadOffset + (int)sequencePayloadLength; + Span encodedSize = stackalloc byte[5]; + int newSizeLength = Av1BitStreamWriter.GetLittleEndianBytes128(sequencePayloadLength + 1, encodedSize); + byte[] malformed = new byte[stream.Length + 1 + newSizeLength - oldSizeLength]; + + stream.AsSpan(0, sequenceObuOffset + 1).CopyTo(malformed); + encodedSize[..newSizeLength].CopyTo(malformed.AsSpan(sequenceObuOffset + 1)); + stream.AsSpan(sequencePayloadOffset, (int)sequencePayloadLength) + .CopyTo(malformed.AsSpan(sequenceObuOffset + 1 + newSizeLength)); + + int trailingByteOffset = sequenceObuOffset + 1 + newSizeLength + (int)sequencePayloadLength; + malformed[trailingByteOffset] = InvalidTrailingByte; + stream.AsSpan(frameObuOffset).CopyTo(malformed.AsSpan(trailingByteOffset + 1)); + return malformed; + } + + /// + /// Gets the byte offset immediately following one explicitly sized OBU. + /// + /// The complete OBU stream. + /// The fixed-header offset of the current OBU. + /// The fixed-header offset of the following OBU. + private static int GetNextObuOffset(byte[] stream, int obuOffset) + { + Av1BitStreamReader sizeReader = new(stream.AsSpan(obuOffset + 1)); + ulong payloadLength = sizeReader.ReadLittleEndianBytes128(out int sizeLength); + return obuOffset + 1 + sizeLength + (int)payloadLength; + } + + /// + /// Reads a complete OBU stream through reference-type state so malformed-input assertions do not capture a ref struct. + /// + /// The complete bounded OBU stream. + /// The stateful OBU parser. + /// Creates the tile reader for each coded frame. + private static void ReadObuStream(byte[] stream, ObuReader obuReader, Func creator) + { + Av1BitStreamReader reader = new(stream); + obuReader.ReadAll(ref reader, stream.Length, creator); + } + + /// + /// Verifies that an unsuccessful bounded parse removed all state that could refer to the rejected session. + /// + /// The parser whose published header state must be empty. + /// The reference map whose retained frame owners must be empty. + private static void AssertParserSessionReset(ObuReader obuReader, Av1ReferenceFrameStore referenceFrames) + { + Assert.Null(obuReader.SequenceHeader); + Assert.Null(obuReader.FrameHeader); + + for (int slot = 0; slot < Av1Constants.ReferenceFrameCount; slot++) + { + Assert.Null(referenceFrames.Resolve(slot)); + } + } + + /// + /// Creates and retains one recording tile reader for every coded frame requested by the OBU parser. + /// + private sealed class LifecycleTileReaderFactory + { + private readonly ObuReader obuReader; + private readonly Av1ReferenceFrameStore referenceFrames; + private readonly int failingReaderIndex; + + /// + /// Initializes a new instance of the class. + /// + /// The parser that owns the current frame-header state. + /// The reconstructed reference map shared with the parser. + /// The zero-based reader index whose tile validation should fail. + public LifecycleTileReaderFactory( + ObuReader obuReader, + Av1ReferenceFrameStore referenceFrames, + int failingReaderIndex) + { + this.obuReader = obuReader; + this.referenceFrames = referenceFrames; + this.failingReaderIndex = failingReaderIndex; + } + + /// + /// Gets the tile readers created in coded-frame order. + /// + public List Readers { get; } = []; + + /// + /// Creates a fresh recording tile reader for the parser's current coded frame. + /// + /// The fresh tile reader. + public LifecycleTileReader Create() + { + LifecycleTileReader reader = new( + this.obuReader, + this.referenceFrames, + this.Readers.Count == this.failingReaderIndex); + + this.Readers.Add(reader); + return reader; + } + } + + /// + /// Records tile-reader ownership and the frame-header state observable at completion. + /// + private sealed class LifecycleTileReader : IAv1TileReader + { + private readonly ObuReader obuReader; + private readonly Av1ReferenceFrameStore referenceFrames; + private readonly bool failTileValidation; + + /// + /// Initializes a new instance of the class. + /// + /// The parser whose current frame header is captured at completion. + /// The reconstructed reference map shared with the parser. + /// A value indicating whether tile validation should fail. + public LifecycleTileReader( + ObuReader obuReader, + Av1ReferenceFrameStore referenceFrames, + bool failTileValidation) + { + this.obuReader = obuReader; + this.referenceFrames = referenceFrames; + this.failTileValidation = failTileValidation; + } + + /// + /// Gets the number of tile payloads delivered to this frame reader. + /// + public int TileCount { get; private set; } + + /// + /// Gets the number of frame-completion notifications delivered to this frame reader. + /// + public int CompletionCount { get; private set; } + + /// + /// Gets the number of delivered tile payloads observed when the frame was completed. + /// + public int TileCountAtCompletion { get; private set; } + + /// + /// Gets the primary frame-header OBU temporal identifier observed at completion. + /// + public int CompletedTemporalId { get; private set; } + + /// + /// Gets the primary frame-header OBU spatial identifier observed at completion. + /// + public int CompletedSpatialId { get; private set; } + + /// + /// Gets the frame header observed at successful completion. + /// + public ObuFrameHeader CompletedFrameHeader { get; private set; } + + /// + /// Gets a value indicating whether every inter-reference role resolved to a retained reconstructed owner before + /// the completed frame changed the reference map. + /// + public bool SelectedReferencesWereRetained { get; private set; } + + /// + public void ReadTile(Span tileData, int tileNum) + { + this.TileCount++; + if (this.failTileValidation) + { + // The final tile owns the remaining declared OBU payload. A trailing-symbol or entropy validation + // failure therefore originates at this boundary and must prevent the later completion callback. + throw new InvalidImageContentException("The test tile payload failed validation."); + } + } + + /// + public void CompleteFrame() + { + ObuSequenceHeader sequenceHeader = Assert.IsType(this.obuReader.SequenceHeader); + ObuFrameHeader frameHeader = Assert.IsType(this.obuReader.FrameHeader); + bool selectedReferencesWereRetained = true; + + if (!frameHeader.IsIntra) + { + Span referenceFrameIndices = frameHeader.GetReferenceFrameIndices(); + + // Resolve all seven roles before committing the current frame. The refresh mask may replace those + // slots, so checking after Commit would verify the new owner instead of the references just parsed. + for (int reference = 0; reference < Av1Constants.ReferencesPerFrame; reference++) + { + selectedReferencesWereRetained &= this.referenceFrames.Resolve((int)referenceFrameIndices[reference]) is not null; + } + } + + using Av1FrameInfo frameInfo = new(sequenceHeader); + Av1FrameBuffer frameBuffer = new( + Configuration.Default, + sequenceHeader, + sequenceHeader.ColorConfig.GetColorFormat(), + is16BitPipeline: false); + + Av1ReferenceFrame referenceFrame = new(frameBuffer, frameHeader, frameInfo); + + // A retained AV1 buffer exposes the visible post-super-resolution geometry rather than the sequence maxima + // used for allocation. frame_size_with_refs reads these exact dimensions for the following coded layer. + frameBuffer.Width = frameHeader.FrameSize.SuperResolutionUpscaledWidth; + frameBuffer.Height = frameHeader.FrameSize.FrameHeight; + + if (!this.referenceFrames.Commit(frameHeader.RefreshFrameFlags, referenceFrame, showFrame: frameHeader.ShowFrame)) + { + // A hidden frame with no refresh role remains caller-owned; match production by releasing it immediately. + referenceFrame.Dispose(); + } + + this.TileCountAtCompletion = this.TileCount; + this.CompletedTemporalId = frameHeader.TemporalId; + this.CompletedSpatialId = frameHeader.SpatialId; + this.CompletedFrameHeader = frameHeader; + this.SelectedReferencesWereRetained = selectedReferencesWereRetained; + this.CompletionCount++; + } + } +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/ObuPrettyPrint.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/ObuPrettyPrint.cs new file mode 100644 index 0000000000..23bf5b11c1 --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/ObuPrettyPrint.cs @@ -0,0 +1,73 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Collections; +using System.Reflection; +using System.Text; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1; + +internal class ObuPrettyPrint +{ + private static readonly char[] Spaces = " ".ToCharArray(); + + public static string PrettyPrintProperties(object obj, int indent = 0) + { + StringBuilder builder = new(); + builder.Append(obj.GetType().Name); + builder.AppendLine("{"); + indent += 2; + MemberInfo[] properties = obj.GetType().FindMembers(MemberTypes.Property, BindingFlags.Instance | BindingFlags.Public, null, null); + foreach (MemberInfo member in properties) + { + if (member is not PropertyInfo property || property.PropertyType.IsByRefLike) + { + // Reflection cannot box Span or ReadOnlySpan; their owning scalar syntax remains comparable. + continue; + } + + builder.Append(Spaces, 0, indent); + builder.Append(property.Name); + builder.Append(" = "); + object value = property.GetValue(obj) ?? "NULL"; + PrettyPrintValue(builder, value, indent); + } + + indent -= 2; + builder.Append(Spaces, 0, indent); + builder.AppendLine("}"); + return builder.ToString(); + } + + private static void PrettyPrintValue(StringBuilder builder, object value, int indent) + { + if (value.GetType() == typeof(string)) + { + builder.AppendLine(value.ToString()); + } + else if (value.GetType().IsArray) + { + builder.AppendLine("["); + indent += 2; + builder.Append(Spaces, 0, indent); + Type elementType = value.GetType().GetElementType(); + IList list = value as IList; + foreach (object item in list) + { + PrettyPrintValue(builder, item, indent); + } + + indent -= 2; + builder.Append(Spaces, 0, indent); + builder.AppendLine("]"); + } + else if (value.GetType().IsClass) + { + builder.AppendLine(PrettyPrintProperties(value, indent)); + } + else + { + builder.AppendLine(value.ToString()); + } + } +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/ObuSkipModeParametersTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/ObuSkipModeParametersTests.cs new file mode 100644 index 0000000000..b5de74558d --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/ObuSkipModeParametersTests.cs @@ -0,0 +1,246 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1; +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1; + +/// +/// Verifies the frame-level derivation of AV1 skip-mode reference pairs. +/// +[Trait("Format", "Avif")] +public class ObuSkipModeParametersTests +{ + /// + /// The public theory-data representation of . + /// + private const int KeyFrameValue = (int)ObuFrameType.KeyFrame; + + /// + /// The public theory-data representation of . + /// + private const int InterFrameValue = (int)ObuFrameType.InterFrame; + + /// + /// The public theory-data representation of . + /// + private const int SingleReferenceValue = (int)ObuReferenceMode.SingleReference; + + /// + /// The public theory-data representation of . + /// + private const int ReferenceModeSelectValue = (int)ObuReferenceMode.ReferenceModeSelect; + + /// + /// Verifies signed order-hint distances across the modulo-domain boundary. + /// + [Fact] + public void GetRelativeDistanceWrapsWithinConfiguredDomain() + { + ObuOrderHintInfo orderHintInfo = CreateOrderHintInfo(); + + Assert.Equal(-2, orderHintInfo.GetRelativeDistance(15, 1)); + Assert.Equal(2, orderHintInfo.GetRelativeDistance(1, 15)); + } + + /// + /// Verifies that disabled order hints have no temporal ordering. + /// + [Fact] + public void GetRelativeDistanceReturnsZeroWhenOrderHintsAreDisabled() + { + ObuOrderHintInfo orderHintInfo = new(); + + Assert.Equal(0, orderHintInfo.GetRelativeDistance(15, 1)); + } + + /// + /// Verifies that skip mode selects the nearest past and future canonical reference roles. + /// + [Fact] + public void DeriveSelectsNearestForwardAndBackwardReferences() + { + ObuOrderHintInfo orderHintInfo = CreateOrderHintInfo(); + ObuFrameHeader frameHeader = CreateInterFrame(8, [7, 3, 6, 2, 10, 12, 15]); + + frameHeader.SkipModeParameters.Derive(orderHintInfo, frameHeader); + + Assert.True(frameHeader.SkipModeParameters.SkipModeAllowed); + Assert.Equal(Av1ReferenceFrameType.Last, frameHeader.SkipModeParameters.FirstReferenceFrame); + Assert.Equal(Av1ReferenceFrameType.Backward, frameHeader.SkipModeParameters.SecondReferenceFrame); + } + + /// + /// Verifies that the derived pair identifies canonical roles rather than their physical reference-map slots. + /// + [Fact] + public void DeriveOrdersCanonicalRolesIndependentlyOfMappedSlots() + { + ObuOrderHintInfo orderHintInfo = CreateOrderHintInfo(); + ObuFrameHeader frameHeader = CreateInterFrame(8, [7, 3, 6, 2, 10, 12, 15]); + Span referenceFrameIndices = frameHeader.GetReferenceFrameIndices(); + Span referenceOrderHints = frameHeader.GetReferenceOrderHints(); + + // Several canonical roles deliberately share physical slot seven. The first matching role remains LAST, while + // the future BWDREF role maps to slot four; neither physical slot number becomes part of the derived pair. + referenceFrameIndices.Fill(7); + referenceFrameIndices[(int)Av1ReferenceFrameType.Backward - 1] = 4; + referenceOrderHints[7] = 7; + + frameHeader.SkipModeParameters.Derive(orderHintInfo, frameHeader); + + Assert.True(frameHeader.SkipModeParameters.SkipModeAllowed); + Assert.Equal(Av1ReferenceFrameType.Last, frameHeader.SkipModeParameters.FirstReferenceFrame); + Assert.Equal(Av1ReferenceFrameType.Backward, frameHeader.SkipModeParameters.SecondReferenceFrame); + } + + /// + /// Verifies that a frame with only future references cannot use skip mode. + /// + [Fact] + public void DeriveDisallowsSkipModeWithoutForwardReference() + { + ObuOrderHintInfo orderHintInfo = CreateOrderHintInfo(); + ObuFrameHeader frameHeader = CreateInterFrame(8, [9, 10, 11, 12, 13, 14, 15]); + + frameHeader.SkipModeParameters.Derive(orderHintInfo, frameHeader); + + Assert.False(frameHeader.SkipModeParameters.SkipModeAllowed); + Assert.Equal(Av1ReferenceFrameType.None, frameHeader.SkipModeParameters.FirstReferenceFrame); + Assert.Equal(Av1ReferenceFrameType.None, frameHeader.SkipModeParameters.SecondReferenceFrame); + } + + /// + /// Verifies that a forward-only frame selects the two closest distinct past reference orders. + /// + [Fact] + public void SelectsTwoForwardReferencesWithoutBackwardReference() + { + ObuOrderHintInfo orderHintInfo = CreateOrderHintInfo(); + ObuFrameHeader frameHeader = CreateInterFrame(8, [7, 3, 6, 2, 1, 5, 4]); + + frameHeader.SkipModeParameters.Derive(orderHintInfo, frameHeader); + + Assert.True(frameHeader.SkipModeParameters.SkipModeAllowed); + Assert.Equal(Av1ReferenceFrameType.Last, frameHeader.SkipModeParameters.FirstReferenceFrame); + Assert.Equal(Av1ReferenceFrameType.Last3, frameHeader.SkipModeParameters.SecondReferenceFrame); + } + + /// + /// Verifies that modulo wraparound participates in nearest-reference selection. + /// + [Fact] + public void DeriveSelectsReferencesAcrossOrderHintWraparound() + { + ObuOrderHintInfo orderHintInfo = CreateOrderHintInfo(); + ObuFrameHeader frameHeader = CreateInterFrame(1, [12, 15, 11, 10, 2, 5, 7]); + + frameHeader.SkipModeParameters.Derive(orderHintInfo, frameHeader); + + Assert.True(frameHeader.SkipModeParameters.SkipModeAllowed); + Assert.Equal(Av1ReferenceFrameType.Last2, frameHeader.SkipModeParameters.FirstReferenceFrame); + Assert.Equal(Av1ReferenceFrameType.Backward, frameHeader.SkipModeParameters.SecondReferenceFrame); + } + + /// + /// Verifies that skip mode remains unavailable without two temporally distinct usable reference orders. + /// + [Fact] + public void DeriveDisallowsSkipModeWithoutReferencePair() + { + ObuOrderHintInfo orderHintInfo = CreateOrderHintInfo(); + ObuFrameHeader frameHeader = CreateInterFrame(8, [7, 8, 8, 8, 8, 8, 8]); + + frameHeader.SkipModeParameters.Derive(orderHintInfo, frameHeader); + + Assert.False(frameHeader.SkipModeParameters.SkipModeAllowed); + Assert.Equal(Av1ReferenceFrameType.None, frameHeader.SkipModeParameters.FirstReferenceFrame); + Assert.Equal(Av1ReferenceFrameType.None, frameHeader.SkipModeParameters.SecondReferenceFrame); + } + + /// + /// Verifies that deriving an ineligible frame clears a reference pair retained by an earlier derivation. + /// + [Fact] + public void DeriveClearsPreviousReferencePair() + { + ObuOrderHintInfo orderHintInfo = CreateOrderHintInfo(); + ObuFrameHeader frameHeader = CreateInterFrame(8, [7, 3, 6, 2, 10, 12, 15]); + frameHeader.SkipModeParameters.Derive(orderHintInfo, frameHeader); + + frameHeader.ReferenceMode = ObuReferenceMode.SingleReference; + frameHeader.SkipModeParameters.Derive(orderHintInfo, frameHeader); + + Assert.False(frameHeader.SkipModeParameters.SkipModeAllowed); + Assert.Equal(Av1ReferenceFrameType.None, frameHeader.SkipModeParameters.FirstReferenceFrame); + Assert.Equal(Av1ReferenceFrameType.None, frameHeader.SkipModeParameters.SecondReferenceFrame); + } + + /// + /// Verifies the frame modes for which the AV1 syntax forbids skip-mode signaling. + /// + /// Whether the sequence enables order hints. + /// The numeric coded-frame-type value. + /// The numeric frame-level reference-mode value. + [Theory] + [InlineData(false, InterFrameValue, ReferenceModeSelectValue)] + [InlineData(true, KeyFrameValue, ReferenceModeSelectValue)] + [InlineData(true, InterFrameValue, SingleReferenceValue)] + public void DeriveDisallowsSkipModeForIneligibleFrameSyntax( + bool enableOrderHint, + int frameTypeValue, + int referenceModeValue) + { + ObuOrderHintInfo orderHintInfo = CreateOrderHintInfo(); + orderHintInfo.EnableOrderHint = enableOrderHint; + ObuFrameHeader frameHeader = CreateInterFrame(8, [7, 3, 6, 2, 10, 12, 15]); + frameHeader.FrameType = (ObuFrameType)frameTypeValue; + frameHeader.ReferenceMode = (ObuReferenceMode)referenceModeValue; + + frameHeader.SkipModeParameters.Derive(orderHintInfo, frameHeader); + + Assert.False(frameHeader.SkipModeParameters.SkipModeAllowed); + Assert.Equal(Av1ReferenceFrameType.None, frameHeader.SkipModeParameters.FirstReferenceFrame); + Assert.Equal(Av1ReferenceFrameType.None, frameHeader.SkipModeParameters.SecondReferenceFrame); + } + + /// + /// Creates the four-bit modulo order-hint configuration used by the derivation scenarios. + /// + /// The enabled order-hint configuration. + private static ObuOrderHintInfo CreateOrderHintInfo() + => new() + { + EnableOrderHint = true, + OrderHintBits = 4, + }; + + /// + /// Creates an inter frame whose seven canonical roles map directly to slots zero through six. + /// + /// The current frame order hint. + /// The order hint selected by each canonical role. + /// The initialized inter-frame header. + private static ObuFrameHeader CreateInterFrame(uint currentOrderHint, ReadOnlySpan referenceOrderHints) + { + ObuFrameHeader frameHeader = new() + { + FrameType = ObuFrameType.InterFrame, + OrderHint = currentOrderHint, + ReferenceMode = ObuReferenceMode.ReferenceModeSelect, + }; + + Span referenceFrameIndices = frameHeader.GetReferenceFrameIndices(); + Span referenceMapOrderHints = frameHeader.GetReferenceOrderHints(); + + for (int referenceIndex = 0; referenceIndex < Av1Constants.ReferencesPerFrame; referenceIndex++) + { + referenceFrameIndices[referenceIndex] = (uint)referenceIndex; + referenceMapOrderHints[referenceIndex] = referenceOrderHints[referenceIndex]; + } + + return frameHeader; + } +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/Color/HeifTransferFunctionsTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Color/HeifTransferFunctionsTests.cs new file mode 100644 index 0000000000..569b77604b --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/Color/HeifTransferFunctionsTests.cs @@ -0,0 +1,157 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.Intrinsics; +using SixLabors.ImageSharp.Formats.Heif.Components; +using SixLabors.ImageSharp.Metadata.Profiles.Cicp; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif.Color; + +/// +/// Verifies scalar and SIMD parity for every H.273 transfer characteristic consumed by HEIF color conversion. +/// +[Trait("Format", "Heif")] +public class HeifTransferFunctionsTests +{ + private static readonly float[] SignalValues = + [ + -0.5F, + -0.25F, + -0.081247F, + -0.01F, + 0F, + 0.0031308F, + 0.01F, + 0.04045F, + 1F / 12F, + 0.18F, + 0.25F, + 0.5F, + 0.75F, + 1F, + 1.25F, + 2F, + ]; + + /// + /// Gets every defined AV1-signallable transfer characteristic, including the deterministic unspecified fallback. + /// + public static TheoryData TransferCharacteristics { get; } = new() + { + (int)CicpTransferCharacteristics.ItuRBt709_6, + (int)CicpTransferCharacteristics.Unspecified, + (int)CicpTransferCharacteristics.Gamma2_2, + (int)CicpTransferCharacteristics.Gamma2_8, + (int)CicpTransferCharacteristics.ItuRBt601_7, + (int)CicpTransferCharacteristics.SmpteSt240, + (int)CicpTransferCharacteristics.Linear, + (int)CicpTransferCharacteristics.Log100, + (int)CicpTransferCharacteristics.Log100Sqrt, + (int)CicpTransferCharacteristics.Iec61966_2_4, + (int)CicpTransferCharacteristics.ItuRBt1361_0, + (int)CicpTransferCharacteristics.Iec61966_2_1, + (int)CicpTransferCharacteristics.ItuRBt2020_2_10bit, + (int)CicpTransferCharacteristics.ItuRBt2020_2_12bit, + (int)CicpTransferCharacteristics.SmpteSt2084, + (int)CicpTransferCharacteristics.SmpteSt428_1, + (int)CicpTransferCharacteristics.AribStdB67, + }; + + /// + /// Verifies that every SIMD width matches the scalar inverse transfer function at curve transitions, extrema, and extended-range values. + /// + /// The transfer-characteristic code point under test. + [Theory] + [MemberData(nameof(TransferCharacteristics))] + public void ToLinearSimdMatchesScalar(int transferCharacteristicsValue) + { + CicpTransferCharacteristics transferCharacteristics = (CicpTransferCharacteristics)transferCharacteristicsValue; + float[] expected = SignalValues.Select(value => HeifTransferFunctions.ToLinear(transferCharacteristics, value)).ToArray(); + + Vector128 vector128 = HeifTransferFunctions.ToLinear(transferCharacteristics, Vector128.Create(SignalValues.AsSpan(0, Vector128.Count))); + Vector256 vector256 = HeifTransferFunctions.ToLinear(transferCharacteristics, Vector256.Create(SignalValues.AsSpan(0, Vector256.Count))); + Vector512 vector512 = HeifTransferFunctions.ToLinear(transferCharacteristics, Vector512.Create(SignalValues)); + + AssertVectorMatchesScalar(expected, vector128, transferCharacteristics); + AssertVectorMatchesScalar(expected, vector256, transferCharacteristics); + AssertVectorMatchesScalar(expected, vector512, transferCharacteristics); + } + + /// + /// Verifies that every SIMD width matches the scalar forward transfer function at curve transitions, extrema, and extended-range values. + /// + /// The transfer-characteristic code point under test. + [Theory] + [MemberData(nameof(TransferCharacteristics))] + public void ToGammaSimdMatchesScalar(int transferCharacteristicsValue) + { + CicpTransferCharacteristics transferCharacteristics = (CicpTransferCharacteristics)transferCharacteristicsValue; + float[] expected = SignalValues.Select(value => HeifTransferFunctions.ToGamma(transferCharacteristics, value)).ToArray(); + + Vector128 vector128 = HeifTransferFunctions.ToGamma(transferCharacteristics, Vector128.Create(SignalValues.AsSpan(0, Vector128.Count))); + Vector256 vector256 = HeifTransferFunctions.ToGamma(transferCharacteristics, Vector256.Create(SignalValues.AsSpan(0, Vector256.Count))); + Vector512 vector512 = HeifTransferFunctions.ToGamma(transferCharacteristics, Vector512.Create(SignalValues)); + + AssertVectorMatchesScalar(expected, vector128, transferCharacteristics); + AssertVectorMatchesScalar(expected, vector256, transferCharacteristics); + AssertVectorMatchesScalar(expected, vector512, transferCharacteristics); + } + + /// + /// Compares four SIMD lanes with their scalar results. + /// + /// The scalar results. + /// The SIMD results. + /// The transfer characteristic under test. + private static void AssertVectorMatchesScalar(ReadOnlySpan expected, Vector128 actual, CicpTransferCharacteristics transferCharacteristics) + { + for (int i = 0; i < Vector128.Count; i++) + { + AssertClose(expected[i], actual.GetElement(i), transferCharacteristics, i, 128); + } + } + + /// + /// Compares eight SIMD lanes with their scalar results. + /// + /// The scalar results. + /// The SIMD results. + /// The transfer characteristic under test. + private static void AssertVectorMatchesScalar(ReadOnlySpan expected, Vector256 actual, CicpTransferCharacteristics transferCharacteristics) + { + for (int i = 0; i < Vector256.Count; i++) + { + AssertClose(expected[i], actual.GetElement(i), transferCharacteristics, i, 256); + } + } + + /// + /// Compares sixteen SIMD lanes with their scalar results. + /// + /// The scalar results. + /// The SIMD results. + /// The transfer characteristic under test. + private static void AssertVectorMatchesScalar(ReadOnlySpan expected, Vector512 actual, CicpTransferCharacteristics transferCharacteristics) + { + for (int i = 0; i < Vector512.Count; i++) + { + AssertClose(expected[i], actual.GetElement(i), transferCharacteristics, i, 512); + } + } + + /// + /// Verifies one SIMD lane within the tolerance of the runtime vector exponential and logarithm kernels. + /// + /// The scalar result. + /// The SIMD result. + /// The transfer characteristic under test. + /// The SIMD lane index. + /// The SIMD register width. + private static void AssertClose(float expected, float actual, CicpTransferCharacteristics transferCharacteristics, int lane, int width) + { + float tolerance = MathF.Max(2E-5F, MathF.Abs(expected) * 2E-5F); + Assert.True( + MathF.Abs(expected - actual) <= tolerance, + $"{transferCharacteristics} at {width}-bit lane {lane}: expected {expected:R}, actual {actual:R}, tolerance {tolerance:R}."); + } +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/HeifDecoderTests.cs b/tests/ImageSharp.Tests/Formats/Heif/HeifDecoderTests.cs new file mode 100644 index 0000000000..e43090a06e --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/HeifDecoderTests.cs @@ -0,0 +1,1293 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Buffers.Binary; +using SixLabors.ImageSharp.ColorProfiles; +using SixLabors.ImageSharp.ColorProfiles.Icc; +using SixLabors.ImageSharp.Formats; +using SixLabors.ImageSharp.Formats.Heif; +using SixLabors.ImageSharp.Formats.Png; +using SixLabors.ImageSharp.Metadata; +using SixLabors.ImageSharp.Metadata.Profiles.Icc; +using SixLabors.ImageSharp.PixelFormats; +using SixLabors.ImageSharp.Processing; +using SixLabors.ImageSharp.Tests.ColorProfiles.Icc; +using SixLabors.ImageSharp.Tests.TestUtilities.ImageComparison; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif; + +[Trait("Format", "Heif")] +[ValidateDisposedMemoryAllocations] +public class HeifDecoderTests +{ + private const uint UnknownBoxType = 0x74657374U; + + private static ReadOnlySpan MalformedJpegApp13 => + [ + 0xFF, 0xED, + 0x00, 0x1D, + (byte)'P', (byte)'h', (byte)'o', (byte)'t', (byte)'o', (byte)'s', (byte)'h', (byte)'o', (byte)'p', (byte)' ', (byte)'3', (byte)'.', + (byte)'0', 0x00, + (byte)'B', (byte)'a', (byte)'d', (byte)'R', (byte)'e', (byte)'s', (byte)'o', (byte)'u', (byte)'r', (byte)'c', (byte)'e', (byte)'!', + (byte)'!' + ]; + + [Theory] + [InlineData(TestImages.Heif.IrvineAvif, HeifCompressionMethod.Av1, HeifBitDepth.Bit8, 480, 640)] + public void Identify(string imagePath, HeifCompressionMethod compressionMethod, HeifBitDepth bitDepth, int width, int height) + { + TestFile testFile = TestFile.Create(imagePath); + using MemoryStream stream = new(testFile.Bytes, false); + + ImageInfo imageInfo = Image.Identify(stream); + HeifMetadata heifMetadata = imageInfo.Metadata.GetHeifMetadata(); + + Assert.NotNull(imageInfo); + Assert.Equal(HeifFormat.Instance, imageInfo.Metadata.DecodedImageFormat); + Assert.Equal(compressionMethod, heifMetadata.CompressionMethod); + Assert.Equal(bitDepth, heifMetadata.BitDepth); + Assert.Equal(width, imageInfo.Width); + Assert.Equal(height, imageInfo.Height); + } + + [Theory] + [InlineData(TestImages.Heif.Orange4x4, DecoderStreamKind.File, 1, 4, 4)] + [InlineData(TestImages.Heif.Orange4x4, DecoderStreamKind.Memory, 1, 4, 4)] + [InlineData(TestImages.Heif.Orange4x4, DecoderStreamKind.NonSeekable, 1, 4, 4)] + [InlineData(TestImages.Heif.Orange4x4, DecoderStreamKind.ShortRead, 1, 4, 4)] + [InlineData(TestImages.Heif.Animated8Bit, DecoderStreamKind.File, 5, 150, 150)] + [InlineData(TestImages.Heif.Animated8Bit, DecoderStreamKind.Memory, 5, 150, 150)] + [InlineData(TestImages.Heif.Animated8Bit, DecoderStreamKind.NonSeekable, 5, 150, 150)] + [InlineData(TestImages.Heif.Animated8Bit, DecoderStreamKind.ShortRead, 5, 150, 150)] + public void DecodeStillAndBoundedSequenceFromSupportedStream( + string imagePath, + DecoderStreamKind streamKind, + int expectedFrameCount, + int expectedWidth, + int expectedHeight) + { + TestFile testFile = TestFile.Create(imagePath); + using Image expected = Image.Load(testFile.Bytes); + using Stream stream = streamKind switch + { + DecoderStreamKind.File => File.OpenRead(testFile.FullPath), + DecoderStreamKind.Memory => new MemoryStream(testFile.Bytes, false), + DecoderStreamKind.NonSeekable => new NonSeekableStream(new MemoryStream(testFile.Bytes, false)), + DecoderStreamKind.ShortRead => new ShortReadMemoryStream(testFile.Bytes), + _ => throw new InvalidOperationException() + }; + + using Image actual = Image.Load(stream); + + Assert.Equal(new Size(expectedWidth, expectedHeight), actual.Size); + Assert.Equal(expectedFrameCount, actual.Frames.Count); + Assert.Equal(expected.Frames.Count, actual.Frames.Count); + for (int frameIndex = 0; frameIndex < actual.Frames.Count; frameIndex++) + { + for (int y = 0; y < actual.Height; y++) + { + Assert.True( + expected.Frames[frameIndex].PixelBuffer.DangerousGetRowSpan(y) + .SequenceEqual(actual.Frames[frameIndex].PixelBuffer.DangerousGetRowSpan(y))); + } + } + } + + [Theory] + [InlineData(TestImages.Heif.Orange4x4, 1, 4, 4)] + [InlineData(TestImages.Heif.Animated8Bit, 5, 150, 150)] + public void DecodeFromCurrentStreamPosition( + string imagePath, + int expectedFrameCount, + int expectedWidth, + int expectedHeight) + { + TestFile testFile = TestFile.Create(imagePath); + using MemoryStream stream = new(); + stream.Write([1, 2, 3, 4]); + long fileStart = stream.Position; + stream.Write(testFile.Bytes); + stream.Position = fileStart; + + using Image image = Image.Load(stream); + + Assert.Equal(new Size(expectedWidth, expectedHeight), image.Size); + Assert.Equal(expectedFrameCount, image.Frames.Count); + } + + /// + /// Verifies that AVIF decoding preserves the exact embedded ICC profile bytes. + /// + [Theory] + [WithFile(TestImages.Heif.ParisIccExifXmpAvif, PixelTypes.Rgba32)] + public void DecodeAvifPreservesEmbeddedIccProfile(TestImageProvider provider) + where TPixel : unmanaged, IPixel + { + DecoderOptions preserveOptions = new() { ColorProfileHandling = ColorProfileHandling.Preserve }; + + using Image preserved = provider.GetImage(HeifDecoder.Instance, preserveOptions); + using Image expectedPreserved = Image.Load(preserveOptions, TestFile.Create(TestImages.Heif.ParisIccExifXmpPng).Bytes); + + Assert.NotNull(preserved.Metadata.IccProfile); + Assert.NotNull(expectedPreserved.Metadata.IccProfile); + Assert.Equal(expectedPreserved.Metadata.IccProfile.ToByteArray(), preserved.Metadata.IccProfile.ToByteArray()); + } + + /// + /// Verifies that AVIF decoding converts pixels from an embedded non-sRGB ICC profile to sRGB. + /// + [Theory] + [WithFile(TestImages.Heif.PerceptualIccAvif, PixelTypes.Rgba32)] + public void DecodeAvifConvertsEmbeddedNonSrgbIccProfile(TestImageProvider provider) + where TPixel : unmanaged, IPixel + { + DecoderOptions preserveOptions = new() { ColorProfileHandling = ColorProfileHandling.Preserve }; + DecoderOptions convertOptions = new() { ColorProfileHandling = ColorProfileHandling.Convert }; + + using Image preserved = provider.GetImage(HeifDecoder.Instance, preserveOptions); + using Image converted = provider.GetImage(HeifDecoder.Instance, convertOptions); + using Image expected = Image.Load(convertOptions, TestFile.Create(TestImages.Png.Icc.Perceptual).Bytes); + + Assert.NotNull(preserved.Metadata.IccProfile); + Assert.Null(converted.Metadata.IccProfile); + Assert.NotEmpty(ImageComparer.Exact.CompareImages(preserved, converted)); + + // The decoded metadata retains the AVIF source matrix, which PNG cannot represent. The debug output exists + // only to inspect converted pixels, so omit metadata without altering the image under test. + converted.DebugSave( + provider, + new PngEncoder { SkipMetadata = true }, + testOutputDetails: "IccConverted"); + + // The PNG is the independent RGB source used by libavif's avifenc. A tolerant comparison accounts for the + // AV1 loss while proving the AVIF ICC stage produces the same target-profile interpretation. + ImageComparer.TolerantPercentage(1F, 20).VerifySimilarity(expected, converted); + } + + /// + /// Verifies that AVIF grid composition retains and converts the presented image's non-sRGB ICC profile. + /// + [Theory] + [WithFile(TestImages.Heif.PerceptualIccGridAvif, PixelTypes.Rgba32)] + public void DecodeAvifGridConvertsEmbeddedNonSrgbIccProfile(TestImageProvider provider) + where TPixel : unmanaged, IPixel + { + DecoderOptions preserveOptions = new() { ColorProfileHandling = ColorProfileHandling.Preserve }; + DecoderOptions convertOptions = new() { ColorProfileHandling = ColorProfileHandling.Convert }; + + using Image preserved = provider.GetImage(HeifDecoder.Instance, preserveOptions); + using Image converted = provider.GetImage(HeifDecoder.Instance, convertOptions); + using Image expectedPreserved = Image.Load(preserveOptions, TestFile.Create(TestImages.Png.Icc.Perceptual).Bytes); + using Image expected = Image.Load(convertOptions, TestFile.Create(TestImages.Png.Icc.Perceptual).Bytes); + + IccProfile preservedIccProfile = Assert.IsType(preserved.Metadata.IccProfile); + IccProfile expectedIccProfile = Assert.IsType(expectedPreserved.Metadata.IccProfile); + Assert.Null(converted.Metadata.IccProfile); + Assert.Equal(expectedIccProfile.ToByteArray(), preservedIccProfile.ToByteArray()); + Assert.NotEmpty(ImageComparer.Exact.CompareImages(preserved, converted)); + ImageComparer.TolerantPercentage(1F, 20).VerifySimilarity(expected, converted); + } + + /// + /// Verifies that AVIF sequence ICC conversion is applied to every presented frame. + /// + [Theory] + [WithFile(TestImages.Heif.PerceptualIccSequenceAvif, PixelTypes.Rgba32)] + public void DecodeAvifSequenceConvertsEveryFrameWithEmbeddedNonSrgbIccProfile(TestImageProvider provider) + where TPixel : unmanaged, IPixel + { + DecoderOptions preserveOptions = new() { ColorProfileHandling = ColorProfileHandling.Preserve }; + DecoderOptions convertOptions = new() { ColorProfileHandling = ColorProfileHandling.Convert }; + + using Image preserved = provider.GetImage(HeifDecoder.Instance, preserveOptions); + using Image converted = provider.GetImage(HeifDecoder.Instance, convertOptions); + using Image expectedPreserved = Image.Load(preserveOptions, TestFile.Create(TestImages.Png.Icc.Perceptual).Bytes); + using Image expected = Image.Load(convertOptions, TestFile.Create(TestImages.Png.Icc.Perceptual).Bytes); + + Assert.Equal(2, preserved.Frames.Count); + Assert.Equal(preserved.Frames.Count, converted.Frames.Count); + IccProfile preservedIccProfile = Assert.IsType(preserved.Metadata.IccProfile); + IccProfile expectedIccProfile = Assert.IsType(expectedPreserved.Metadata.IccProfile); + Assert.Null(converted.Metadata.IccProfile); + Assert.Equal(expectedIccProfile.ToByteArray(), preservedIccProfile.ToByteArray()); + + for (int i = 0; i < converted.Frames.Count; i++) + { + Assert.False(ImageComparer.Exact.CompareImagesOrFrames(i, preserved.Frames[i], converted.Frames[i]).IsEmpty); + Assert.True(ImageComparer.TolerantPercentage(1F, 20).CompareImagesOrFrames(i, expected.Frames.RootFrame, converted.Frames[i]).IsEmpty); + } + } + + /// + /// Verifies that non-sRGB ICC conversion follows auxiliary-alpha composition and preserves the composed alpha values. + /// + [Fact] + public void DecodeAvifAlphaImageConvertsEmbeddedIccProfileWithoutChangingAlpha() + { + DecoderOptions preserveOptions = new() { ColorProfileHandling = ColorProfileHandling.Preserve }; + DecoderOptions convertOptions = new() { ColorProfileHandling = ColorProfileHandling.Convert }; + byte[] encoded = TestFile.Create(TestImages.Heif.DuckyRommIccAlphaAvif).Bytes; + + using Image preserved = Image.Load(preserveOptions, encoded); + using Image converted = Image.Load(convertOptions, encoded); + using Image expected = preserved.Clone(); + + ColorProfileConverter converter = new(new ColorConversionOptions + { + SourceIccProfile = expected.Metadata.IccProfile, + TargetIccProfile = CompactSrgbV4Profile.Profile, + MemoryAllocator = expected.Configuration.MemoryAllocator, + }); + + // Build the oracle from the fully composed preserved decode so that only ICC ordering and alpha retention + // are under test; the independently encoded AV1 color and alpha payloads remain identical in both paths. + converter.Convert(expected); + + Assert.NotNull(preserved.Metadata.IccProfile); + Assert.Null(converted.Metadata.IccProfile); + Assert.Equal(TestIccProfiles.GetProfile(TestIccProfiles.RommRgb).ToByteArray(), preserved.Metadata.IccProfile.ToByteArray()); + Assert.NotEmpty(ImageComparer.Exact.CompareImages(preserved, converted)); + + for (int y = 0; y < converted.Height; y++) + { + Span preservedRow = preserved.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(y); + Span convertedRow = converted.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(y); + + for (int x = 0; x < convertedRow.Length; x++) + { + Assert.Equal(preservedRow[x].A, convertedRow[x].A); + } + } + + ImageComparer.Exact.VerifySimilarity(expected, converted); + } + + /// + /// Verifies that compact profile handling retains non-sRGB ICC profiles and leaves their pixels unconverted. + /// + [Theory] + [WithFile(TestImages.Heif.PerceptualIccAvif, PixelTypes.Rgba32)] + [WithFile(TestImages.Heif.PerceptualIccGridAvif, PixelTypes.Rgba32)] + [WithFile(TestImages.Heif.PerceptualIccSequenceAvif, PixelTypes.Rgba32)] + [WithFile(TestImages.Heif.DuckyRommIccAlphaAvif, PixelTypes.Rgba32)] + public void DecodeAvifRetainsNonSrgbIccProfileWhenCompacting(TestImageProvider provider) + where TPixel : unmanaged, IPixel + { + DecoderOptions preserveOptions = new() { ColorProfileHandling = ColorProfileHandling.Preserve }; + DecoderOptions compactOptions = new() { ColorProfileHandling = ColorProfileHandling.Compact }; + + using Image preserved = provider.GetImage(HeifDecoder.Instance, preserveOptions); + using Image compact = provider.GetImage(HeifDecoder.Instance, compactOptions); + + Assert.NotNull(preserved.Metadata.IccProfile); + Assert.NotNull(compact.Metadata.IccProfile); + Assert.Equal(preserved.Metadata.IccProfile.ToByteArray(), compact.Metadata.IccProfile.ToByteArray()); + Assert.Empty(ImageComparer.Exact.CompareImages(preserved, compact)); + } + + /// + /// Verifies that compact profile handling removes a canonical sRGB ICC profile without changing pixels. + /// + [Theory] + [WithFile(TestImages.Heif.ParisIccExifXmpAvif, PixelTypes.Rgba32)] + public void DecodeAvifCompactsCanonicalSrgbIccProfile(TestImageProvider provider) + where TPixel : unmanaged, IPixel + { + DecoderOptions preserveOptions = new() { ColorProfileHandling = ColorProfileHandling.Preserve }; + DecoderOptions compactOptions = new() { ColorProfileHandling = ColorProfileHandling.Compact }; + + using Image preserved = provider.GetImage(HeifDecoder.Instance, preserveOptions); + using Image compact = provider.GetImage(HeifDecoder.Instance, compactOptions); + + Assert.NotNull(preserved.Metadata.IccProfile); + Assert.Null(compact.Metadata.IccProfile); + Assert.Empty(ImageComparer.Exact.CompareImages(preserved, compact)); + } + + /// + /// Verifies that metadata skipping omits the embedded AVIF ICC profile. + /// + [Theory] + [WithFile(TestImages.Heif.ParisIccExifXmpAvif, PixelTypes.Rgba32)] + [WithFile(TestImages.Heif.PerceptualIccGridAvif, PixelTypes.Rgba32)] + [WithFile(TestImages.Heif.PerceptualIccSequenceAvif, PixelTypes.Rgba32)] + [WithFile(TestImages.Heif.DuckyRommIccAlphaAvif, PixelTypes.Rgba32)] + public void DecodeAvifSkipsEmbeddedIccProfileWithMetadata(TestImageProvider provider) + where TPixel : unmanaged, IPixel + { + DecoderOptions options = new() + { + ColorProfileHandling = ColorProfileHandling.Preserve, + SkipMetadata = true + }; + + using Image image = provider.GetImage(HeifDecoder.Instance, options); + + Assert.Null(image.Metadata.IccProfile); + } + + [Fact] + public void DecodeIgnoresUnknownTopLevelBox() + { + byte[] data = CreateLegacyJpegContainer(); + data = InsertBytes(data, data.Length, CreateUnknownBox()); + + using Image image = Image.Load(data); + + Assert.Equal(new Size(2, 3), image.Size); + } + + [Fact] + public void DecodeAppliesTargetSizeOnceToThePresentedHeifImage() + { + using Image source = new(64, 48); + for (int y = 0; y < source.Height; y++) + { + Span row = source.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(y); + for (int x = 0; x < row.Length; x++) + { + row[x] = new Rgba32((byte)(x * 3), (byte)(y * 5), (byte)((x * 7) + (y * 11))); + } + } + + using MemoryStream stream = new(); + source.Save(stream, new HeifEncoder()); + byte[] data = stream.ToArray(); + Size targetSize = new(17, 17); + DecoderOptions options = new() { TargetSize = targetSize }; + + using Image expected = Image.Load(data); + expected.Mutate(context => context.Resize(new ResizeOptions { Size = targetSize, Mode = ResizeMode.Max, Sampler = options.Sampler })); + + using Image image = Image.Load(options, data); + + Assert.Equal(expected.Size, image.Size); + for (int y = 0; y < image.Height; y++) + { + Assert.True(image.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(y).SequenceEqual( + expected.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(y))); + } + } + + [Fact] + public void DecodePropagatesStrictValidationToLegacyJpegItems() + { + byte[] data = CreateContainerWithMalformedJpegMetadata(); + DecoderOptions options = new() { SegmentIntegrityHandling = SegmentIntegrityHandling.Strict }; + + Assert.Throws(() => + { + using Image image = Image.Load(options, data); + }); + } + + [Theory] + [InlineData(SegmentIntegrityHandling.IgnoreAncillary)] + [InlineData(SegmentIntegrityHandling.IgnoreImageData)] + public void DecodePropagatesRecoverableMetadataValidationToLegacyJpegItems(SegmentIntegrityHandling handling) + { + byte[] data = CreateContainerWithMalformedJpegMetadata(); + DecoderOptions options = new() { SegmentIntegrityHandling = handling }; + + using Image image = Image.Load(options, data); + + Assert.Equal(new Size(2, 3), image.Size); + } + + [Fact] + public void DecodePropagatesSkipMetadataToLegacyJpegItems() + { + byte[] data = CreateContainerWithMalformedJpegMetadata(); + DecoderOptions options = new() + { + SkipMetadata = true, + SegmentIntegrityHandling = SegmentIntegrityHandling.Strict + }; + + using Image image = Image.Load(options, data); + + Assert.Equal(new Size(2, 3), image.Size); + } + + [Fact] + public void DecodePropagatesConfigurationToLegacyJpegItems() + { + byte[] data = CreateLegacyJpegContainer(); + Configuration configuration = Configuration.CreateDefaultInstance(); + DecoderOptions options = new() { Configuration = configuration }; + + using Image image = Image.Load(options, data); + + Assert.Same(configuration, image.Configuration); + } + + /// + /// Verifies that invalid optional alpha payloads remain fatal when image-data errors cannot be ignored. + /// + [Theory] + [InlineData(SegmentIntegrityHandling.Strict)] + [InlineData(SegmentIntegrityHandling.IgnoreAncillary)] + public void DecodeRejectsInvalidAlphaPayloadUnlessImageDataErrorsAreIgnored(SegmentIntegrityHandling handling) + { + byte[] data = [.. TestFile.Create(TestImages.Heif.DuckyRommIccAlphaAvif).Bytes]; + uint alphaItemId = FindFirstItemReferenceSourceId(data, Heif4CharCode.Auxl); + ClearItemPayload(data, alphaItemId); + DecoderOptions options = new() { SegmentIntegrityHandling = handling }; + + Assert.ThrowsAny(() => + { + using Image image = Image.Load(options, data); + }); + } + + /// + /// Verifies that omits a corrupt optional alpha item while + /// retaining the independently decodable color item. + /// + [Fact] + public void DecodeOmitsInvalidAlphaPayloadWhenImageDataErrorsAreIgnored() + { + byte[] source = TestFile.Create(TestImages.Heif.DuckyRommIccAlphaAvif).Bytes; + byte[] data = [.. source]; + uint alphaItemId = FindFirstItemReferenceSourceId(data, Heif4CharCode.Auxl); + ClearItemPayload(data, alphaItemId); + DecoderOptions options = new() { SegmentIntegrityHandling = SegmentIntegrityHandling.IgnoreImageData }; + + using Image expected = Image.Load(source); + using Image actual = Image.Load(options, data); + + AssertOpaqueRgbMatches(expected, actual); + } + + /// + /// Verifies that a malformed alpha relationship remains fatal when image-data errors cannot be ignored. + /// + [Theory] + [InlineData(SegmentIntegrityHandling.Strict)] + [InlineData(SegmentIntegrityHandling.IgnoreAncillary)] + public void DecodeRejectsMalformedAlphaReferenceUnlessImageDataErrorsAreIgnored(SegmentIntegrityHandling handling) + { + byte[] data = [.. TestFile.Create(TestImages.Heif.DuckyRommIccAlphaAvif).Bytes]; + InvalidateFirstItemReferenceSource(data, Heif4CharCode.Auxl); + DecoderOptions options = new() { SegmentIntegrityHandling = handling }; + + Assert.Throws(() => + { + using Image image = Image.Load(options, data); + }); + } + + /// + /// Verifies that omits a malformed optional alpha + /// relationship while retaining the independently decodable color item. + /// + [Fact] + public void DecodeOmitsMalformedAlphaReferenceWhenImageDataErrorsAreIgnored() + { + byte[] source = TestFile.Create(TestImages.Heif.DuckyRommIccAlphaAvif).Bytes; + byte[] data = [.. source]; + InvalidateFirstItemReferenceSource(data, Heif4CharCode.Auxl); + DecoderOptions options = new() { SegmentIntegrityHandling = SegmentIntegrityHandling.IgnoreImageData }; + + using Image expected = Image.Load(source); + using Image actual = Image.Load(options, data); + + AssertOpaqueRgbMatches(expected, actual); + } + + /// + /// Verifies that strict validation rejects a malformed descriptive metadata relationship. + /// + [Fact] + public void DecodeRejectsMalformedMetadataReferenceInStrictMode() + { + byte[] data = [.. TestFile.Create(TestImages.Heif.ParisIccExifXmpAvif).Bytes]; + InvalidateFirstItemReferenceSource(data, Heif4CharCode.Cdsc); + DecoderOptions options = new() { SegmentIntegrityHandling = SegmentIntegrityHandling.Strict }; + + Assert.Throws(() => + { + using Image image = Image.Load(options, data); + }); + } + + /// + /// Verifies that non-strict validation omits a malformed descriptive relationship without weakening image-data + /// validation. + /// + [Theory] + [InlineData(SegmentIntegrityHandling.IgnoreAncillary)] + [InlineData(SegmentIntegrityHandling.IgnoreImageData)] + public void DecodeOmitsMalformedMetadataReferenceWhenAncillaryErrorsAreIgnored(SegmentIntegrityHandling handling) + { + byte[] data = [.. TestFile.Create(TestImages.Heif.ParisIccExifXmpAvif).Bytes]; + InvalidateFirstItemReferenceSource(data, Heif4CharCode.Cdsc); + DecoderOptions options = new() { SegmentIntegrityHandling = handling }; + + using Image image = Image.Load(options, data); + + Assert.Null(image.Metadata.ExifProfile); + Assert.NotNull(image.Metadata.XmpProfile); + Assert.NotNull(image.Metadata.IccProfile); + } + + /// + /// Verifies that skipped metadata is neither retained nor validated through its optional descriptive links. + /// + [Fact] + public void DecodeDoesNotValidateSkippedMetadataReference() + { + byte[] data = [.. TestFile.Create(TestImages.Heif.ParisIccExifXmpAvif).Bytes]; + InvalidateFirstItemReferenceSource(data, Heif4CharCode.Cdsc); + DecoderOptions options = new() + { + SkipMetadata = true, + SegmentIntegrityHandling = SegmentIntegrityHandling.Strict + }; + + using Image image = Image.Load(options, data); + + Assert.Null(image.Metadata.ExifProfile); + Assert.Null(image.Metadata.XmpProfile); + Assert.Null(image.Metadata.IccProfile); + } + + [Fact] + public void IdentifyIgnoresUnknownMetadataBox() + { + byte[] data = CreateLegacyJpegContainer(); + int metaOffset = FindBoxOffset(data, Heif4CharCode.Meta, 0, data.Length); + int metaSize = (int)BinaryPrimitives.ReadUInt32BigEndian(data.AsSpan(metaOffset)); + data = InsertBytes(data, metaOffset + metaSize, CreateUnknownBox()); + IncrementBoxSize(data, metaOffset, 8); + + ImageInfo imageInfo = Image.Identify(data); + + Assert.Equal(new Size(2, 3), imageInfo.Size); + } + + [Fact] + public void IdentifyIgnoresUnknownNonEssentialProperty() + { + byte[] data = CreateContainerWithUnknownProperty(false); + + ImageInfo imageInfo = Image.Identify(data); + + Assert.Equal(new Size(2, 3), imageInfo.Size); + } + + [Fact] + public void IdentifyRejectsUnknownEssentialProperty() + { + byte[] data = CreateContainerWithUnknownProperty(true); + + InvalidImageContentException exception = Assert.Throws(() => Image.Identify(data)); + + Assert.Contains("essential", exception.Message, StringComparison.OrdinalIgnoreCase); + } + + [Fact] + public void IdentifyRejectsMalformedAncillaryPropertyInStrictMode() + { + byte[] data = CreateContainerWithProperty(CreateEmptyBox(Heif4CharCode.Pasp), false); + DecoderOptions options = new() { SegmentIntegrityHandling = SegmentIntegrityHandling.Strict }; + + Assert.Throws(() => Image.Identify(options, data)); + } + + [Theory] + [InlineData(SegmentIntegrityHandling.IgnoreAncillary)] + [InlineData(SegmentIntegrityHandling.IgnoreImageData)] + public void IdentifyIgnoresMalformedAncillaryPropertyWhenPermitted(SegmentIntegrityHandling handling) + { + byte[] data = CreateContainerWithProperty(CreateEmptyBox(Heif4CharCode.Pasp), false); + DecoderOptions options = new() { SegmentIntegrityHandling = handling }; + + ImageInfo imageInfo = Image.Identify(options, data); + + Assert.Equal(new Size(2, 3), imageInfo.Size); + Assert.Equal(PixelResolutionUnit.PixelsPerInch, imageInfo.Metadata.ResolutionUnits); + Assert.Equal(96D, imageInfo.Metadata.HorizontalResolution); + Assert.Equal(96D, imageInfo.Metadata.VerticalResolution); + } + + [Fact] + public void IdentifyDoesNotValidateSkippedAncillaryPropertyMetadata() + { + byte[] data = CreateContainerWithProperty(CreateEmptyBox(Heif4CharCode.Pasp), false); + DecoderOptions options = new() + { + SkipMetadata = true, + SegmentIntegrityHandling = SegmentIntegrityHandling.Strict + }; + + ImageInfo imageInfo = Image.Identify(options, data); + + Assert.Equal(new Size(2, 3), imageInfo.Size); + } + + [Theory] + [InlineData(SegmentIntegrityHandling.Strict)] + [InlineData(SegmentIntegrityHandling.IgnoreAncillary)] + public void IdentifyRejectsMalformedImagePropertyUnlessImageDataErrorsAreIgnored(SegmentIntegrityHandling handling) + { + byte[] data = CreateContainerWithProperty(CreateEmptyBox(Heif4CharCode.Irot), true); + DecoderOptions options = new() { SegmentIntegrityHandling = handling }; + + Assert.Throws(() => Image.Identify(options, data)); + } + + [Fact] + public void IdentifyIgnoresMalformedImagePropertyWhenImageDataErrorsAreIgnored() + { + byte[] data = CreateContainerWithProperty(CreateEmptyBox(Heif4CharCode.Irot), true); + DecoderOptions options = new() { SegmentIntegrityHandling = SegmentIntegrityHandling.IgnoreImageData }; + + ImageInfo imageInfo = Image.Identify(options, data); + + Assert.Equal(new Size(2, 3), imageInfo.Size); + } + + [Fact] + public void IdentifyRejectsDuplicateAncillaryPropertyAssociationInStrictMode() + { + byte[] data = CreateContainerWithDuplicatePropertyAssociation(CreatePixelAspectRatioBox(), false); + DecoderOptions options = new() { SegmentIntegrityHandling = SegmentIntegrityHandling.Strict }; + + Assert.Throws(() => Image.Identify(options, data)); + } + + [Theory] + [InlineData(SegmentIntegrityHandling.IgnoreAncillary)] + [InlineData(SegmentIntegrityHandling.IgnoreImageData)] + public void IdentifyIgnoresDuplicateAncillaryPropertyAssociationWhenPermitted(SegmentIntegrityHandling handling) + { + byte[] data = CreateContainerWithDuplicatePropertyAssociation(CreatePixelAspectRatioBox(), false); + DecoderOptions options = new() { SegmentIntegrityHandling = handling }; + + ImageInfo imageInfo = Image.Identify(options, data); + + Assert.Equal(new Size(2, 3), imageInfo.Size); + Assert.Equal(PixelResolutionUnit.AspectRatio, imageInfo.Metadata.ResolutionUnits); + Assert.Equal(1D, imageInfo.Metadata.HorizontalResolution); + Assert.Equal(2D, imageInfo.Metadata.VerticalResolution); + } + + [Theory] + [InlineData(SegmentIntegrityHandling.Strict)] + [InlineData(SegmentIntegrityHandling.IgnoreAncillary)] + public void IdentifyRejectsDuplicateImagePropertyAssociationUnlessImageDataErrorsAreIgnored(SegmentIntegrityHandling handling) + { + byte[] data = CreateContainerWithDuplicatePropertyAssociation(CreateRotationBox(), true); + DecoderOptions options = new() { SegmentIntegrityHandling = handling }; + + Assert.Throws(() => Image.Identify(options, data)); + } + + [Fact] + public void IdentifyIgnoresDuplicateImagePropertyAssociationWhenImageDataErrorsAreIgnored() + { + byte[] data = CreateContainerWithDuplicatePropertyAssociation(CreateRotationBox(), true); + DecoderOptions options = new() { SegmentIntegrityHandling = SegmentIntegrityHandling.IgnoreImageData }; + + ImageInfo imageInfo = Image.Identify(options, data); + + Assert.Equal(new Size(3, 2), imageInfo.Size); + } + + [Theory] + [InlineData(Heif4CharCode.Mif1)] + [InlineData(Heif4CharCode.Avif)] + [InlineData(Heif4CharCode.Jpeg)] + public void DetectorRecognizesSupportedStillImageMajorBrand(Heif4CharCode brand) + { + byte[] data = CreateLegacyJpegContainer(); + BinaryPrimitives.WriteUInt32BigEndian(data.AsSpan(8), (uint)brand); + HeifImageFormatDetector detector = new(); + + bool detected = detector.TryDetectFormat(data.AsSpan(0, detector.HeaderSize), out IImageFormat format); + + Assert.True(detected); + Assert.Same(HeifFormat.Instance, format); + } + + [Theory] + [InlineData(Heif4CharCode.Avis)] + public void DetectorRecognizesSupportedSequenceMajorBrand(Heif4CharCode brand) + { + byte[] data = CreateLegacyJpegContainer(); + BinaryPrimitives.WriteUInt32BigEndian(data.AsSpan(8), (uint)brand); + HeifImageFormatDetector detector = new(); + + bool detected = detector.TryDetectFormat(data.AsSpan(0, detector.HeaderSize), out IImageFormat format); + + Assert.True(detected); + Assert.Same(HeifFormat.Instance, format); + } + + [Fact] + public void DetectorRecognizesExtendedSizeFileTypeBox() + { + byte[] data = new byte[24]; + BinaryPrimitives.WriteUInt32BigEndian(data, 1); + BinaryPrimitives.WriteUInt32BigEndian(data.AsSpan(4), (uint)Heif4CharCode.Ftyp); + BinaryPrimitives.WriteUInt64BigEndian(data.AsSpan(8), (ulong)data.Length); + BinaryPrimitives.WriteUInt32BigEndian(data.AsSpan(16), (uint)Heif4CharCode.Avif); + HeifImageFormatDetector detector = new(); + + bool detected = detector.TryDetectFormat(data, out IImageFormat format); + + Assert.True(detected); + Assert.Same(HeifFormat.Instance, format); + } + + [Theory] + [InlineData(Heif4CharCode.Jpgs)] + public void DetectorRejectsUnsupportedSequenceMajorBrand(Heif4CharCode brand) + { + byte[] data = CreateLegacyJpegContainer(); + BinaryPrimitives.WriteUInt32BigEndian(data.AsSpan(8), (uint)brand); + HeifImageFormatDetector detector = new(); + + Assert.False(detector.TryDetectFormat(data.AsSpan(0, detector.HeaderSize), out _)); + } + + [Fact] + public void IdentifyRejectsUnsupportedBrands() + { + byte[] data = CreateLegacyJpegContainer(); + BinaryPrimitives.WriteUInt32BigEndian(data.AsSpan(8), UnknownBoxType); + BinaryPrimitives.WriteUInt32BigEndian(data.AsSpan(16), UnknownBoxType); + BinaryPrimitives.WriteUInt32BigEndian(data.AsSpan(20), UnknownBoxType); + using MemoryStream stream = new(data, false); + + Assert.Throws(() => HeifDecoder.Instance.Identify(DecoderOptions.Default, stream)); + } + + [Fact] + public void IdentifyAcceptsExtendedSizeTopLevelBox() + { + byte[] data = CreateLegacyJpegContainer(); + byte[] box = new byte[16]; + BinaryPrimitives.WriteUInt32BigEndian(box, 1); + BinaryPrimitives.WriteUInt32BigEndian(box.AsSpan(4), UnknownBoxType); + BinaryPrimitives.WriteUInt64BigEndian(box.AsSpan(8), (ulong)box.Length); + data = InsertBytes(data, data.Length, box); + + ImageInfo imageInfo = Image.Identify(data); + + Assert.Equal(new Size(2, 3), imageInfo.Size); + } + + [Fact] + public void IdentifyAcceptsUuidTopLevelBox() + { + byte[] data = CreateLegacyJpegContainer(); + byte[] box = new byte[24]; + BinaryPrimitives.WriteUInt32BigEndian(box, (uint)box.Length); + BinaryPrimitives.WriteUInt32BigEndian(box.AsSpan(4), (uint)Heif4CharCode.Uuid); + data = InsertBytes(data, data.Length, box); + + ImageInfo imageInfo = Image.Identify(data); + + Assert.Equal(new Size(2, 3), imageInfo.Size); + } + + [Fact] + public void IdentifyAcceptsSizeZeroTopLevelBox() + { + byte[] data = CreateLegacyJpegContainer(); + byte[] box = CreateUnknownBox(); + BinaryPrimitives.WriteUInt32BigEndian(box, 0); + data = InsertBytes(data, data.Length, box); + + ImageInfo imageInfo = Image.Identify(data); + + Assert.Equal(new Size(2, 3), imageInfo.Size); + } + + [Fact] + public void IdentifyAcceptsExtendedSizeItemInfoEntry() + { + byte[] data = CreateLegacyJpegContainer(); + int metaOffset = FindBoxOffset(data, Heif4CharCode.Meta, 0, data.Length); + int metaSize = (int)BinaryPrimitives.ReadUInt32BigEndian(data.AsSpan(metaOffset)); + int iinfOffset = FindBoxOffset(data, Heif4CharCode.Iinf, metaOffset + 12, metaSize - 12); + int infeOffset = iinfOffset + 14; + uint infeSize = BinaryPrimitives.ReadUInt32BigEndian(data.AsSpan(infeOffset)); + data = InsertBytes(data, infeOffset + 8, new byte[8]); + BinaryPrimitives.WriteUInt32BigEndian(data.AsSpan(infeOffset), 1); + BinaryPrimitives.WriteUInt64BigEndian(data.AsSpan(infeOffset + 8), infeSize + 8); + IncrementBoxSize(data, metaOffset, 8); + IncrementBoxSize(data, iinfOffset, 8); + + ImageInfo imageInfo = Image.Identify(data); + + Assert.Equal(new Size(2, 3), imageInfo.Size); + } + + [Fact] + public void IdentifyRejectsSizeZeroMetadataChild() + { + byte[] data = CreateLegacyJpegContainer(); + int metaOffset = FindBoxOffset(data, Heif4CharCode.Meta, 0, data.Length); + int metaSize = (int)BinaryPrimitives.ReadUInt32BigEndian(data.AsSpan(metaOffset)); + byte[] box = CreateUnknownBox(); + BinaryPrimitives.WriteUInt32BigEndian(box, 0); + data = InsertBytes(data, metaOffset + metaSize, box); + IncrementBoxSize(data, metaOffset, box.Length); + + Assert.Throws(() => Image.Identify(data)); + } + + [Fact] + public void IdentifyRejectsMetadataChildBeyondParent() + { + byte[] data = CreateLegacyJpegContainer(); + int metaOffset = FindBoxOffset(data, Heif4CharCode.Meta, 0, data.Length); + int metaSize = (int)BinaryPrimitives.ReadUInt32BigEndian(data.AsSpan(metaOffset)); + byte[] box = CreateUnknownBox(); + BinaryPrimitives.WriteUInt32BigEndian(box, 16); + data = InsertBytes(data, metaOffset + metaSize, box); + IncrementBoxSize(data, metaOffset, box.Length); + + Assert.Throws(() => Image.Identify(data)); + } + + [Fact] + public void IdentifyRejectsItemInfoEntryBeyondParent() + { + byte[] data = CreateLegacyJpegContainer(); + int metaOffset = FindBoxOffset(data, Heif4CharCode.Meta, 0, data.Length); + int metaSize = (int)BinaryPrimitives.ReadUInt32BigEndian(data.AsSpan(metaOffset)); + int iinfOffset = FindBoxOffset(data, Heif4CharCode.Iinf, metaOffset + 12, metaSize - 12); + uint iinfSize = BinaryPrimitives.ReadUInt32BigEndian(data.AsSpan(iinfOffset)); + int infeOffset = iinfOffset + 14; + BinaryPrimitives.WriteUInt32BigEndian(data.AsSpan(infeOffset), iinfSize); + + Assert.Throws(() => Image.Identify(data)); + } + + [Fact] + public void IdentifyRejectsBoxSmallerThanHeader() + { + byte[] data = CreateLegacyJpegContainer(); + byte[] box = CreateUnknownBox(); + BinaryPrimitives.WriteUInt32BigEndian(box, 4); + data = InsertBytes(data, data.Length, box); + + Assert.Throws(() => Image.Identify(data)); + } + + [Fact] + public void IdentifyRejectsTruncatedExtendedSizeHeader() + { + byte[] data = CreateLegacyJpegContainer(); + byte[] box = new byte[12]; + BinaryPrimitives.WriteUInt32BigEndian(box, 1); + BinaryPrimitives.WriteUInt32BigEndian(box.AsSpan(4), UnknownBoxType); + data = InsertBytes(data, data.Length, box); + + Assert.Throws(() => Image.Identify(data)); + } + + [Fact] + public void IdentifyRejectsTruncatedUuidHeader() + { + byte[] data = CreateLegacyJpegContainer(); + byte[] box = new byte[16]; + BinaryPrimitives.WriteUInt32BigEndian(box, 24); + BinaryPrimitives.WriteUInt32BigEndian(box.AsSpan(4), (uint)Heif4CharCode.Uuid); + data = InsertBytes(data, data.Length, box); + + Assert.Throws(() => Image.Identify(data)); + } + + [Fact] + public void IdentifyAcceptsItemPropertiesBeforeItemInfo() + { + byte[] data = CreateLegacyJpegContainer(); + int metaOffset = FindBoxOffset(data, Heif4CharCode.Meta, 0, data.Length); + int metaSize = (int)BinaryPrimitives.ReadUInt32BigEndian(data.AsSpan(metaOffset)); + int iinfOffset = FindBoxOffset(data, Heif4CharCode.Iinf, metaOffset + 12, metaSize - 12); + int iprpOffset = FindBoxOffset(data, Heif4CharCode.Iprp, metaOffset + 12, metaSize - 12); + int iprpSize = (int)BinaryPrimitives.ReadUInt32BigEndian(data.AsSpan(iprpOffset)); + data = MoveBoxBefore(data, iprpOffset, iprpSize, iinfOffset); + + ImageInfo imageInfo = Image.Identify(data); + + Assert.Equal(new Size(2, 3), imageInfo.Size); + } + + [Fact] + public void IdentifyAcceptsItemLocationBeforeItemInfo() + { + byte[] data = CreateLegacyJpegContainer(); + int metaOffset = FindBoxOffset(data, Heif4CharCode.Meta, 0, data.Length); + int metaSize = (int)BinaryPrimitives.ReadUInt32BigEndian(data.AsSpan(metaOffset)); + int iinfOffset = FindBoxOffset(data, Heif4CharCode.Iinf, metaOffset + 12, metaSize - 12); + int ilocOffset = FindBoxOffset(data, Heif4CharCode.Iloc, metaOffset + 12, metaSize - 12); + int ilocSize = (int)BinaryPrimitives.ReadUInt32BigEndian(data.AsSpan(ilocOffset)); + data = MoveBoxBefore(data, ilocOffset, ilocSize, iinfOffset); + + ImageInfo imageInfo = Image.Identify(data); + + Assert.Equal(new Size(2, 3), imageInfo.Size); + } + + [Fact] + public void IdentifyRejectsDuplicateUniqueMetadataBox() + { + byte[] data = CreateLegacyJpegContainer(); + int metaOffset = FindBoxOffset(data, Heif4CharCode.Meta, 0, data.Length); + int metaSize = (int)BinaryPrimitives.ReadUInt32BigEndian(data.AsSpan(metaOffset)); + int pitmOffset = FindBoxOffset(data, Heif4CharCode.Pitm, metaOffset + 12, metaSize - 12); + int pitmSize = (int)BinaryPrimitives.ReadUInt32BigEndian(data.AsSpan(pitmOffset)); + data = InsertBytes(data, metaOffset + metaSize, data.AsSpan(pitmOffset, pitmSize)); + IncrementBoxSize(data, metaOffset, pitmSize); + + Assert.Throws(() => Image.Identify(data)); + } + + private static byte[] CreateLegacyJpegContainer() + { + using Image image = new(2, 3); + using MemoryStream stream = new(); + image.Save(stream, new HeifEncoder { CompressionMethod = HeifCompressionMethod.LegacyJpeg }); + return stream.ToArray(); + } + + private static byte[] CreateContainerWithUnknownProperty(bool essential) + => CreateContainerWithProperty(CreateUnknownBox(), essential); + + private static byte[] CreateContainerWithProperty(ReadOnlySpan property, bool essential) + { + byte[] data = CreateLegacyJpegContainer(); + int metaOffset = FindBoxOffset(data, Heif4CharCode.Meta, 0, data.Length); + int metaSize = (int)BinaryPrimitives.ReadUInt32BigEndian(data.AsSpan(metaOffset)); + int iprpOffset = FindBoxOffset(data, Heif4CharCode.Iprp, metaOffset + 12, metaSize - 12); + int iprpSize = (int)BinaryPrimitives.ReadUInt32BigEndian(data.AsSpan(iprpOffset)); + int ipcoOffset = FindBoxOffset(data, Heif4CharCode.Ipco, iprpOffset + 8, iprpSize - 8); + int ipcoSize = (int)BinaryPrimitives.ReadUInt32BigEndian(data.AsSpan(ipcoOffset)); + int ipmaOffset = FindBoxOffset(data, Heif4CharCode.Ipma, iprpOffset + 8, iprpSize - 8); + + // Insert the property before ipma so its one-based index is 2 and all parent box sizes remain explicit. + data = InsertBytes(data, ipcoOffset + ipcoSize, property); + IncrementBoxSize(data, metaOffset, property.Length); + IncrementBoxSize(data, iprpOffset, property.Length); + IncrementBoxSize(data, ipcoOffset, property.Length); + ipmaOffset += property.Length; + + // The generated container has one item with one property association; append the inserted property to that entry. + int associationCountOffset = ipmaOffset + 18; + data[associationCountOffset]++; + int associationOffset = ipmaOffset + (int)BinaryPrimitives.ReadUInt32BigEndian(data.AsSpan(ipmaOffset)); + byte association = (byte)(2 | (essential ? 0x80 : 0)); + data = InsertBytes(data, associationOffset, new byte[] { association }); + IncrementBoxSize(data, metaOffset, 1); + IncrementBoxSize(data, iprpOffset, 1); + IncrementBoxSize(data, ipmaOffset, 1); + return data; + } + + private static byte[] CreateContainerWithMalformedJpegMetadata() + { + byte[] data = CreateLegacyJpegContainer(); + int metaOffset = FindBoxOffset(data, Heif4CharCode.Meta, 0, data.Length); + int metaSize = (int)BinaryPrimitives.ReadUInt32BigEndian(data.AsSpan(metaOffset)); + int itemLocationOffset = FindBoxOffset(data, Heif4CharCode.Iloc, metaOffset + 12, metaSize - 12); + int mediaDataOffset = FindBoxOffset(data, Heif4CharCode.Mdat, 0, data.Length); + + // The generated item uses one file-relative extent. Insert the malformed JPEG application segment after its + // start-of-image marker, then update the enclosing media-data size and the exact declared extent length. + data = InsertBytes(data, mediaDataOffset + 10, MalformedJpegApp13); + IncrementBoxSize(data, mediaDataOffset, MalformedJpegApp13.Length); + uint extentLength = BinaryPrimitives.ReadUInt32BigEndian(data.AsSpan(itemLocationOffset + 32)); + BinaryPrimitives.WriteUInt32BigEndian(data.AsSpan(itemLocationOffset + 32), extentLength + (uint)MalformedJpegApp13.Length); + return data; + } + + private static byte[] CreateContainerWithDuplicatePropertyAssociation(ReadOnlySpan property, bool essential) + { + byte[] data = CreateContainerWithProperty(property, essential); + int metaOffset = FindBoxOffset(data, Heif4CharCode.Meta, 0, data.Length); + int metaSize = (int)BinaryPrimitives.ReadUInt32BigEndian(data.AsSpan(metaOffset)); + int iprpOffset = FindBoxOffset(data, Heif4CharCode.Iprp, metaOffset + 12, metaSize - 12); + int iprpSize = (int)BinaryPrimitives.ReadUInt32BigEndian(data.AsSpan(iprpOffset)); + int ipmaOffset = FindBoxOffset(data, Heif4CharCode.Ipma, iprpOffset + 8, iprpSize - 8); + int associationCountOffset = ipmaOffset + 18; + + // Repeat the inserted property's one-based index in the existing item entry without changing box structure. + data[associationCountOffset]++; + int associationOffset = ipmaOffset + (int)BinaryPrimitives.ReadUInt32BigEndian(data.AsSpan(ipmaOffset)); + byte association = (byte)(2 | (essential ? 0x80 : 0)); + data = InsertBytes(data, associationOffset, new byte[] { association }); + IncrementBoxSize(data, metaOffset, 1); + IncrementBoxSize(data, iprpOffset, 1); + IncrementBoxSize(data, ipmaOffset, 1); + return data; + } + + private static byte[] CreateUnknownBox() + => CreateEmptyBox((Heif4CharCode)UnknownBoxType); + + private static byte[] CreateEmptyBox(Heif4CharCode type) + => CreateBox(type, []); + + private static byte[] CreatePixelAspectRatioBox() + { + byte[] payload = new byte[8]; + BinaryPrimitives.WriteUInt32BigEndian(payload, 2); + BinaryPrimitives.WriteUInt32BigEndian(payload.AsSpan(4), 1); + return CreateBox(Heif4CharCode.Pasp, payload); + } + + private static byte[] CreateRotationBox() => CreateBox(Heif4CharCode.Irot, [1]); + + private static byte[] CreateBox(Heif4CharCode type, ReadOnlySpan payload) + { + byte[] box = new byte[8 + payload.Length]; + BinaryPrimitives.WriteUInt32BigEndian(box, (uint)box.Length); + BinaryPrimitives.WriteUInt32BigEndian(box.AsSpan(4), (uint)type); + payload.CopyTo(box.AsSpan(8)); + return box; + } + + private static int FindBoxOffset(ReadOnlySpan data, Heif4CharCode type, int offset, int length) + { + int endOffset = offset + length; + while (offset < endOffset) + { + int boxSize = (int)BinaryPrimitives.ReadUInt32BigEndian(data[offset..]); + Heif4CharCode boxType = (Heif4CharCode)BinaryPrimitives.ReadUInt32BigEndian(data[(offset + 4)..]); + if (boxType == type) + { + return offset; + } + + offset += boxSize; + } + + return -1; + } + + /// + /// Reads the source item identifier from the first registered relationship of the requested type. + /// + /// The complete HEIF container. + /// The item-reference child type. + /// The source item identifier. + private static uint FindFirstItemReferenceSourceId(ReadOnlySpan data, Heif4CharCode referenceType) + { + int metaOffset = FindBoxOffset(data, Heif4CharCode.Meta, 0, data.Length); + Assert.True(metaOffset >= 0); + + int metaSize = (int)BinaryPrimitives.ReadUInt32BigEndian(data[metaOffset..]); + int itemReferenceOffset = FindBoxOffset(data, Heif4CharCode.Iref, metaOffset + 12, metaSize - 12); + Assert.True(itemReferenceOffset >= 0); + + int itemReferenceSize = (int)BinaryPrimitives.ReadUInt32BigEndian(data[itemReferenceOffset..]); + int relationshipOffset = FindBoxOffset(data, referenceType, itemReferenceOffset + 12, itemReferenceSize - 12); + Assert.True(relationshipOffset >= 0); + + byte version = data[itemReferenceOffset + 8]; + + Assert.InRange(version, (byte)0, (byte)1); + + return version == 0 + ? BinaryPrimitives.ReadUInt16BigEndian(data[(relationshipOffset + 8)..]) + : BinaryPrimitives.ReadUInt32BigEndian(data[(relationshipOffset + 8)..]); + } + + /// + /// Replaces the source item identifier of the first requested relationship with an undeclared value. + /// + /// The complete mutable HEIF container. + /// The item-reference child type. + private static void InvalidateFirstItemReferenceSource(Span data, Heif4CharCode referenceType) + { + int metaOffset = FindBoxOffset(data, Heif4CharCode.Meta, 0, data.Length); + Assert.True(metaOffset >= 0); + + int metaSize = (int)BinaryPrimitives.ReadUInt32BigEndian(data[metaOffset..]); + int itemReferenceOffset = FindBoxOffset(data, Heif4CharCode.Iref, metaOffset + 12, metaSize - 12); + Assert.True(itemReferenceOffset >= 0); + + int itemReferenceSize = (int)BinaryPrimitives.ReadUInt32BigEndian(data[itemReferenceOffset..]); + int relationshipOffset = FindBoxOffset(data, referenceType, itemReferenceOffset + 12, itemReferenceSize - 12); + Assert.True(relationshipOffset >= 0); + + byte version = data[itemReferenceOffset + 8]; + + Assert.InRange(version, (byte)0, (byte)1); + + if (version == 0) + { + BinaryPrimitives.WriteUInt16BigEndian(data[(relationshipOffset + 8)..], ushort.MaxValue); + } + else + { + BinaryPrimitives.WriteUInt32BigEndian(data[(relationshipOffset + 8)..], uint.MaxValue); + } + } + + /// + /// Clears every file-relative extent belonging to the requested item while retaining the container structure. + /// + /// The complete mutable HEIF container. + /// The item whose coded payload is cleared. + private static void ClearItemPayload(Span data, uint itemId) + { + int metaOffset = FindBoxOffset(data, Heif4CharCode.Meta, 0, data.Length); + Assert.True(metaOffset >= 0); + + int metaSize = (int)BinaryPrimitives.ReadUInt32BigEndian(data[metaOffset..]); + int itemLocationOffset = FindBoxOffset(data, Heif4CharCode.Iloc, metaOffset + 12, metaSize - 12); + Assert.True(itemLocationOffset >= 0); + + int offset = itemLocationOffset + 8; + byte version = data[offset]; + offset += 4; + + int extentOffsetSize = data[offset] >> 4; + int extentLengthSize = data[offset] & 0x0F; + offset++; + int baseOffsetSize = data[offset] >> 4; + int extentIndexSize = version is 1 or 2 ? data[offset] & 0x0F : 0; + offset++; + + uint itemCount = version == 2 + ? BinaryPrimitives.ReadUInt32BigEndian(data[offset..]) + : BinaryPrimitives.ReadUInt16BigEndian(data[offset..]); + + offset += version == 2 ? 4 : 2; + bool found = false; + for (uint itemIndex = 0; itemIndex < itemCount; itemIndex++) + { + uint currentItemId = version == 2 + ? BinaryPrimitives.ReadUInt32BigEndian(data[offset..]) + : BinaryPrimitives.ReadUInt16BigEndian(data[offset..]); + + offset += version == 2 ? 4 : 2; + if (version is 1 or 2) + { + ushort constructionMethod = BinaryPrimitives.ReadUInt16BigEndian(data[offset..]); + Assert.Equal(0, constructionMethod & 0x0F); + offset += 2; + } + + // The data-reference index is zero for the self-contained image items used by the fixture. + Assert.Equal(0, BinaryPrimitives.ReadUInt16BigEndian(data[offset..])); + offset += 2; + ulong baseOffset = ReadVariableUnsigned(data, baseOffsetSize, ref offset); + int extentCount = BinaryPrimitives.ReadUInt16BigEndian(data[offset..]); + offset += 2; + for (int extentIndex = 0; extentIndex < extentCount; extentIndex++) + { + _ = ReadVariableUnsigned(data, extentIndexSize, ref offset); + ulong extentOffset = ReadVariableUnsigned(data, extentOffsetSize, ref offset); + ulong extentLength = ReadVariableUnsigned(data, extentLengthSize, ref offset); + if (currentItemId == itemId) + { + data.Slice(checked((int)(baseOffset + extentOffset)), checked((int)extentLength)).Clear(); + found = true; + } + } + } + + Assert.True(found); + } + + /// + /// Reads one zero-width, 32-bit, or 64-bit unsigned item-location field. + /// + /// The complete HEIF container. + /// The field width in bytes. + /// The current read offset, advanced past the field. + /// The decoded field value. + private static ulong ReadVariableUnsigned(ReadOnlySpan data, int size, ref int offset) + { + ulong value = size switch + { + 0 => 0, + 4 => BinaryPrimitives.ReadUInt32BigEndian(data[offset..]), + 8 => BinaryPrimitives.ReadUInt64BigEndian(data[offset..]), + _ => throw new InvalidOperationException($"Unexpected item-location field width {size} in the test fixture.") + }; + + offset += size; + return value; + } + + /// + /// Verifies that omitting an invalid alpha item preserves color channels and produces opaque output. + /// + /// The image decoded with its valid alpha item. + /// The image decoded after the alpha item or relationship was invalidated. + private static void AssertOpaqueRgbMatches(Image expected, Image actual) + { + Assert.False(actual.Metadata.GetHeifMetadata().HasAlpha); + Assert.Equal(expected.Size, actual.Size); + for (int y = 0; y < actual.Height; y++) + { + Span expectedRow = expected.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(y); + Span actualRow = actual.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(y); + for (int x = 0; x < actualRow.Length; x++) + { + Assert.Equal(expectedRow[x].R, actualRow[x].R); + Assert.Equal(expectedRow[x].G, actualRow[x].G); + Assert.Equal(expectedRow[x].B, actualRow[x].B); + Assert.Equal(byte.MaxValue, actualRow[x].A); + } + } + } + + private static byte[] InsertBytes(byte[] data, int offset, ReadOnlySpan inserted) + { + byte[] result = new byte[data.Length + inserted.Length]; + data.AsSpan(0, offset).CopyTo(result); + inserted.CopyTo(result.AsSpan(offset)); + data.AsSpan(offset).CopyTo(result.AsSpan(offset + inserted.Length)); + return result; + } + + private static byte[] MoveBoxBefore(byte[] data, int boxOffset, int boxSize, int beforeOffset) + { + byte[] result = new byte[data.Length]; + data.AsSpan(0, beforeOffset).CopyTo(result); + data.AsSpan(boxOffset, boxSize).CopyTo(result.AsSpan(beforeOffset)); + data.AsSpan(beforeOffset, boxOffset - beforeOffset).CopyTo(result.AsSpan(beforeOffset + boxSize)); + data.AsSpan(boxOffset + boxSize).CopyTo(result.AsSpan(boxOffset + boxSize)); + return result; + } + + private static void IncrementBoxSize(byte[] data, int offset, int increment) + { + uint size = BinaryPrimitives.ReadUInt32BigEndian(data.AsSpan(offset)); + BinaryPrimitives.WriteUInt32BigEndian(data.AsSpan(offset), size + (uint)increment); + } + + public enum DecoderStreamKind + { + File, + Memory, + NonSeekable, + ShortRead + } + + private sealed class ShortReadMemoryStream : MemoryStream + { + private const int MaximumReadLength = 3; + + public ShortReadMemoryStream(byte[] data) + : base(data, false) + { + } + + public override int Read(byte[] buffer, int offset, int count) + => base.Read(buffer, offset, Math.Min(count, MaximumReadLength)); + + public override int Read(Span buffer) + => base.Read(buffer[..Math.Min(buffer.Length, MaximumReadLength)]); + } +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/HeifEncoderTests.cs b/tests/ImageSharp.Tests/Formats/Heif/HeifEncoderTests.cs new file mode 100644 index 0000000000..2fec37aaf6 --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/HeifEncoderTests.cs @@ -0,0 +1,1835 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Buffers; +using System.Buffers.Binary; +using System.Text; +using SixLabors.ImageSharp.Formats; +using SixLabors.ImageSharp.Formats.Heif; +using SixLabors.ImageSharp.Formats.Heif.Av1; +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; +using SixLabors.ImageSharp.Memory; +using SixLabors.ImageSharp.Metadata.Profiles.Cicp; +using SixLabors.ImageSharp.Metadata.Profiles.Exif; +using SixLabors.ImageSharp.Metadata.Profiles.Icc; +using SixLabors.ImageSharp.Metadata.Profiles.Xmp; +using SixLabors.ImageSharp.PixelFormats; +using SixLabors.ImageSharp.Tests.TestDataIcc; +using SixLabors.ImageSharp.Tests.TestUtilities.ImageComparison; +using SixLabors.ImageSharp.Tests.TestUtilities.ReferenceCodecs; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif; + +[Trait("Format", "Heif")] +[ValidateDisposedMemoryAllocations] +public class HeifEncoderTests +{ + private const int Av1EightBit = (int)Av1BitDepth.EightBit; + private const int Av1TenBit = (int)Av1BitDepth.TenBit; + private const int Av1TwelveBit = (int)Av1BitDepth.TwelveBit; + private const int Yuv400 = (int)Av1ColorFormat.Yuv400; + private const int Yuv420 = (int)Av1ColorFormat.Yuv420; + private const int Yuv422 = (int)Av1ColorFormat.Yuv422; + private const int Yuv444 = (int)Av1ColorFormat.Yuv444; + + [Fact] + public void OptionsHaveExpectedDefaults() + { + HeifEncoder encoder = new(); + + Assert.Equal(HeifCompressionMethod.Av1, encoder.CompressionMethod); + Assert.Null(encoder.Quality); + Assert.Null(encoder.AlphaQuality); + Assert.Equal(5, encoder.Effort); + Assert.False(encoder.Lossless); + Assert.Null(encoder.BitDepth); + Assert.Null(encoder.ChromaSubsampling); + Assert.Null(encoder.RepeatCount); + Assert.True(encoder.AnimateRootFrame); + } + + [Theory] + [InlineData(-1)] + [InlineData(101)] + public void QualityOutsideRangeThrows(int quality) + => Assert.Throws(() => new HeifEncoder { Quality = quality }); + + [Theory] + [InlineData(-1)] + [InlineData(101)] + public void AlphaQualityOutsideRangeThrows(int quality) + => Assert.Throws(() => new HeifEncoder { AlphaQuality = quality }); + + [Theory] + [InlineData(-1)] + [InlineData(11)] + public void EffortOutsideRangeThrows(int effort) + => Assert.Throws(() => new HeifEncoder { Effort = effort }); + + [Theory] + [InlineData(0, 0, 0)] + [InlineData(100, 100, 10)] + public void OptionRangeBoundariesAreAccepted(int quality, int alphaQuality, int effort) + { + HeifEncoder encoder = new() + { + Quality = quality, + AlphaQuality = alphaQuality, + Effort = effort + }; + + Assert.Equal(quality, encoder.Quality); + Assert.Equal(alphaQuality, encoder.AlphaQuality); + Assert.Equal(effort, encoder.Effort); + } + + [Fact] + public void LegacyJpegAcceptsZeroQuality() + { + using Image image = new(1, 1); + image[0, 0] = new Rgba32(10, 20, 30); + using MemoryStream stream = new(); + HeifEncoder encoder = new() + { + CompressionMethod = HeifCompressionMethod.LegacyJpeg, + Quality = 0 + }; + + image.Save(stream, encoder); + + Assert.NotEqual(0, stream.Length); + stream.Position = 0; + using Image decoded = Image.Load(stream); + Assert.Equal(image.Size, decoded.Size); + } + + [Fact] + public void LegacyJpegWritesNonSeekableStream() + { + using Image image = new(1, 1); + image[0, 0] = new Rgba32(10, 20, 30); + using MemoryStream storage = new(); + using NonSeekableStream destination = new(storage); + + image.Save( + destination, + new HeifEncoder { CompressionMethod = HeifCompressionMethod.LegacyJpeg }); + + Assert.NotEqual(0, storage.Length); + storage.Position = 0; + using Image decoded = Image.Load(storage); + Assert.Equal(image.Size, decoded.Size); + } + + [Fact] + public void LegacyJpegWritesAtCurrentStreamPosition() + { + using Image image = new(1, 1); + image[0, 0] = new Rgba32(10, 20, 30); + using MemoryStream stream = new(); + stream.Write([1, 2, 3, 4]); + long fileStart = stream.Position; + + image.Save( + stream, + new HeifEncoder { CompressionMethod = HeifCompressionMethod.LegacyJpeg }); + + stream.Position = fileStart; + using Image decoded = Image.Load(stream); + Assert.Equal(image.Size, decoded.Size); + } + + [Theory] + [InlineData(false, true)] + [InlineData(true, false)] + public void LegacyJpegHonorsSkipMetadataForEmbeddedProfiles(bool skipMetadata, bool expectedIccProfile) + { + using Image image = new(8, 8); + image.Metadata.IccProfile = new IccProfile(IccTestDataProfiles.ProfileRandomArray); + using MemoryStream stream = new(); + HeifEncoder encoder = new() + { + CompressionMethod = HeifCompressionMethod.LegacyJpeg, + SkipMetadata = skipMetadata + }; + + image.Save(stream, encoder); + + DecoderOptions preserveOptions = new() { ColorProfileHandling = ColorProfileHandling.Preserve }; + stream.Position = 0; + using Image decoded = Image.Load(preserveOptions, stream); + Assert.Equal(expectedIccProfile, decoded.Metadata.IccProfile is not null); + if (expectedIccProfile) + { + Assert.Equal( + IccTestDataProfiles.ProfileRandomArray, + Assert.IsType(decoded.Metadata.IccProfile).ToByteArray()); + } + } + + [Fact] + public void LegacyJpegIgnoresLosslessOption() + { + using Image image = new(1, 1); + using MemoryStream stream = new(); + HeifEncoder encoder = new() + { + CompressionMethod = HeifCompressionMethod.LegacyJpeg, + Lossless = true + }; + + image.Save(stream, encoder); + stream.Position = 0; + using Image decoded = Image.Load(stream); + Assert.Equal(image.Size, decoded.Size); + Assert.Equal( + HeifCompressionMethod.LegacyJpeg, + decoded.Metadata.GetHeifMetadata().CompressionMethod); + } + + [Theory] + [InlineData(HeifBitDepth.Bit10)] + [InlineData(HeifBitDepth.Bit12)] + public void LegacyJpegNormalizesHighBitDepthToEightBit(HeifBitDepth bitDepth) + { + using Image image = new(1, 1); + using MemoryStream stream = new(); + HeifEncoder encoder = new() + { + CompressionMethod = HeifCompressionMethod.LegacyJpeg, + BitDepth = bitDepth + }; + + image.Save(stream, encoder); + stream.Position = 0; + using Image decoded = Image.Load(stream); + Assert.Equal(HeifBitDepth.Bit8, decoded.Metadata.GetHeifMetadata().BitDepth); + } + + [Fact] + public void LegacyJpegEncodesRootFrameFromImageSequence() + { + using Image image = new(1, 1); + image.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(0)[0] = new Rgba32(255, 255, 255); + image.Frames.AddFrame(image.Frames.RootFrame); + image.Frames[1].PixelBuffer.DangerousGetRowSpan(0)[0] = new Rgba32(0, 0, 0); + using MemoryStream stream = new(); + HeifEncoder encoder = new() + { + CompressionMethod = HeifCompressionMethod.LegacyJpeg, + Quality = 100 + }; + + image.Save(stream, encoder); + stream.Position = 0; + using Image decoded = Image.Load(stream); + Assert.Single(decoded.Frames); + Assert.Equal(new Rgba32(255, 255, 255), decoded.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(0)[0]); + } + + [Fact] + public void Av1ImageSequencePreservesSeparateRootFrame() + { + const int width = 8; + const int height = 8; + using Image image = new(width, height); + for (int row = 0; row < height; row++) + { + image.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(row).Fill(new Rgb24(255, 255, 255)); + } + + image.Frames.AddFrame(image.Frames.RootFrame); + for (int row = 0; row < height; row++) + { + image.Frames[1].PixelBuffer.DangerousGetRowSpan(row).Fill(new Rgb24(0, 0, 0)); + } + + image.Frames[1].Metadata.GetHeifMetadata().FrameDelay = new Rational(1, 20); + using MemoryStream stream = new(); + HeifEncoder encoder = new() + { + CompressionMethod = HeifCompressionMethod.Av1, + AnimateRootFrame = false, + Lossless = true, + Effort = 0 + }; + + image.Save(stream, encoder); + byte[] file = stream.ToArray(); + Span fileType = GetTopLevelBox(file, Heif4CharCode.Ftyp); + Assert.Equal(Heif4CharCode.Miaf, (Heif4CharCode)BinaryPrimitives.ReadUInt32BigEndian(fileType[^sizeof(uint)..])); + Assert.Equal(1, BinaryPrimitives.ReadUInt16BigEndian(GetMetadataChild(file, Heif4CharCode.Pitm)[12..])); + + stream.Position = 0; + using Image decoded = Image.Load(stream); + Assert.Equal(2, decoded.Frames.Count); + Assert.False(decoded.Metadata.GetHeifMetadata().AnimateRootFrame); + Assert.Empty(ImageComparer.Exact.CompareImages(image, decoded)); + Assert.Equal( + image.Frames[1].Metadata.GetHeifMetadata().FrameDelay, + decoded.Frames[1].Metadata.GetHeifMetadata().FrameDelay); + } + + [Fact] + public void GridDecoderAcceptsSmallerRightAndBottomColorAndAlphaCells() + { + const int tileWidth = 64; + const int tileHeight = 64; + const int outputWidth = 96; + const int outputHeight = 96; + Size[] tileSizes = + [ + new(tileWidth, tileHeight), + new(outputWidth - tileWidth, tileHeight), + new(tileWidth, outputHeight - tileHeight), + new(outputWidth - tileWidth, outputHeight - tileHeight) + ]; + + Rgba32[] tileColors = + [ + new(32, 32, 32), + new(64, 64, 64), + new(96, 96, 96), + new(128, 128, 128) + ]; + + ObuColorConfig colorConfig = new() + { + IsColorDescriptionPresent = true, + ColorPrimaries = ObuColorPrimaries.Bt709, + TransferCharacteristics = ObuTransferCharacteristics.Srgb, + MatrixCoefficients = ObuMatrixCoefficients.Bt709, + ColorRange = true, + IsMonochrome = true, + SubSamplingX = true, + SubSamplingY = true, + BitDepth = Av1BitDepth.EightBit + }; + + List items = []; + Dictionary payloads = []; + HeifItem gridItem = new(Heif4CharCode.Grid, 1); + gridItem.SetExtent(new Size(outputWidth, outputHeight)); + items.Add(gridItem); + HeifItemLink gridLink = new(Heif4CharCode.Dimg, gridItem.Id); + for (int tileIndex = 0; tileIndex < tileSizes.Length; tileIndex++) + { + Size tileSize = tileSizes[tileIndex]; + using Image tile = new(tileSize.Width, tileSize.Height, tileColors[tileIndex]); + using MemoryStream payload = new(); + ObuSequenceHeader header = Av1FrameEncoder.Encode( + Configuration.Default, + tile.Frames.RootFrame, + payload, + colorConfig, + qIndex: 0, + effort: 0); + + uint itemId = (uint)tileIndex + 2; + HeifItem tileItem = new(Heif4CharCode.Av01, itemId) + { + Av1CodecConfiguration = new Av1CodecConfiguration(header) + }; + + tileItem.SetExtent(tileSize); + items.Add(tileItem); + gridLink.DestinationIds.Add(itemId); + payloads.Add(itemId, payload.ToArray()); + } + + List links = [gridLink]; + GridHeifItemDecoder decoder = new(items, links, ReadItem); + Span descriptor = [0, 0, 1, 1, 0, outputWidth, 0, outputHeight]; + using Image result = decoder.DecodeItemData( + new DecoderOptions { Configuration = Configuration.Default }, + gridItem, + descriptor, + null, + TestContext.Current.CancellationToken); + + for (int y = 0; y < outputHeight; y++) + { + for (int x = 0; x < outputWidth; x++) + { + int tileIndex = (y < tileHeight ? 0 : 2) + (x < tileWidth ? 0 : 1); + Assert.Equal(tileColors[tileIndex], result[x, y]); + } + } + + Rgba32 opaqueColor = new(7, 11, 13); + using Image alphaResult = new(outputWidth, outputHeight, opaqueColor); + decoder.DecodeAlphaItemData( + new DecoderOptions { Configuration = Configuration.Default }, + gridItem, + descriptor, + alphaResult.Frames.RootFrame, + alphaResult.Size, + new Rectangle(Point.Empty, alphaResult.Size), + false, + TestContext.Current.CancellationToken); + + for (int y = 0; y < outputHeight; y++) + { + for (int x = 0; x < outputWidth; x++) + { + int tileIndex = (y < tileHeight ? 0 : 2) + (x < tileWidth ? 0 : 1); + Rgba32 expected = opaqueColor; + expected.A = tileColors[tileIndex].R; + Assert.Equal(expected, alphaResult[x, y]); + } + } + + IMemoryOwner ReadItem(HeifItem item) + { + byte[] payload = payloads[item.Id]; + IMemoryOwner owner = Configuration.Default.MemoryAllocator.Allocate(payload.Length); + payload.CopyTo(owner.Memory.Span); + return owner; + } + } + + [Fact] + public void GridDecoderRejectsCellSmallerThanMiafMinimum() + { + ObuColorConfig colorConfig = new() + { + IsColorDescriptionPresent = true, + ColorPrimaries = ObuColorPrimaries.Bt709, + TransferCharacteristics = ObuTransferCharacteristics.Srgb, + MatrixCoefficients = ObuMatrixCoefficients.Bt709, + ColorRange = true, + IsMonochrome = true, + SubSamplingX = true, + SubSamplingY = true, + BitDepth = Av1BitDepth.EightBit + }; + + InvalidImageContentException exception = Assert.Throws( + () => + { + using Image decoded = DecodeSingleCellGrid(63, 64, colorConfig); + }); + + Assert.Contains("grid cells must be at least 64 samples", exception.Message, StringComparison.Ordinal); + } + + /// + /// Verifies grid dimensions preserve chroma alignment for AV1's 4:2:2 and 4:2:0 layouts. + /// + [Theory] + [InlineData(65, 64, true, false)] + [InlineData(65, 64, true, true)] + [InlineData(64, 65, true, true)] + public void GridDecoderRejectsOddSubsampledDimension( + int width, + int height, + bool subsamplingX, + bool subsamplingY) + { + ObuColorConfig colorConfig = new() + { + IsColorDescriptionPresent = true, + ColorPrimaries = ObuColorPrimaries.Bt709, + TransferCharacteristics = ObuTransferCharacteristics.Srgb, + MatrixCoefficients = ObuMatrixCoefficients.Bt709, + ColorRange = true, + BitDepth = Av1BitDepth.EightBit, + SubSamplingX = subsamplingX, + SubSamplingY = subsamplingY + }; + + InvalidImageContentException exception = Assert.Throws( + () => + { + using Image decoded = DecodeSingleCellGrid(width, height, colorConfig); + }); + + Assert.Contains("must be even", exception.Message, StringComparison.Ordinal); + } + + [Theory] + [InlineData(null)] + [InlineData(HeifChromaSubsampling.Yuv420)] + [InlineData(HeifChromaSubsampling.Yuv422)] + [InlineData(HeifChromaSubsampling.Yuv444)] + public void Av1OversizedStillImageWritesAndDecodesGrid(HeifChromaSubsampling? chromaSubsampling) + { + const int width = 65537; + using Image image = new(width, 1); + image[0, 0] = new Rgb24(1, 2, 3); + image[32768, 0] = new Rgb24(11, 13, 17); + image[32769, 0] = new Rgb24(19, 23, 29); + image[width - 1, 0] = new Rgb24(31, 37, 41); + + // Identity-matrix 4:4:4 makes the lossless AV1 cells preserve the packed RGB channels exactly. + CicpProfile sourceProfile = new(1, 13, 0, true); + image.Metadata.CicpProfile = sourceProfile; + using MemoryStream stream = new(); + HeifEncoder encoder = new() + { + CompressionMethod = HeifCompressionMethod.Av1, + Lossless = true, + ChromaSubsampling = chromaSubsampling, + Effort = 0 + }; + + image.Save(stream, encoder); + byte[] file = stream.ToArray(); + Assert.Equal( + [0, 1, 0, 1, 0, 1, 0, 1, 0, 0, 0, 1], + GetItemPayload(file, 1).ToArray()); + + using Av1Decoder firstCellDecoder = new(Configuration.Default); + using Image firstCell = firstCellDecoder.Decode(GetItemPayload(file, 2)); + using Av1Decoder secondCellDecoder = new(Configuration.Default); + using Image secondCell = secondCellDecoder.Decode(GetItemPayload(file, 3)); + + // Odd grid dimensions require full-resolution chroma, including when subsampling was explicitly requested. + // The conversion must retain the source profile and signal the resolved sampling on every coded cell. + ObuSequenceHeader firstHeader = Assert.IsType(firstCellDecoder.SequenceHeader); + ObuSequenceHeader secondHeader = Assert.IsType(secondCellDecoder.SequenceHeader); + Assert.False(firstHeader.ColorConfig.SubSamplingX); + Assert.False(firstHeader.ColorConfig.SubSamplingY); + Assert.False(secondHeader.ColorConfig.SubSamplingX); + Assert.False(secondHeader.ColorConfig.SubSamplingY); + Assert.Equal(ObuMatrixCoefficients.Identity, firstHeader.ColorConfig.MatrixCoefficients); + Assert.Equal(ObuMatrixCoefficients.Identity, secondHeader.ColorConfig.MatrixCoefficients); + Assert.Same(sourceProfile, image.Metadata.CicpProfile); + + // AVIF requires the first grid cell to be at least 64 samples on both axes. The derived grid trims + // the replicated right and bottom edges back to the presentation encoded in its descriptor. + Assert.Equal(new Size(32769, 64), firstCell.Size); + Assert.Equal(new Size(32769, 64), secondCell.Size); + Assert.Equal(image[32768, 0], firstCell[32768, 0]); + Assert.Equal(image[32769, 0], secondCell[0, 0]); + Assert.Equal(image[width - 1, 0], secondCell[32767, 0]); + Assert.Equal(secondCell[32767, 0], secondCell[32768, 0]); + Assert.Equal(secondCell[0, 0], secondCell[0, 63]); + Assert.Equal(1U, GetItemInfoFlags(file, 2)); + Assert.Equal(1U, GetItemInfoFlags(file, 3)); + + ReadOnlySpan references = GetMetadataChild(file, Heif4CharCode.Iref); + const int FirstReferenceOffset = 12; + Assert.Equal( + Heif4CharCode.Dimg, + (Heif4CharCode)BinaryPrimitives.ReadUInt32BigEndian(references[(FirstReferenceOffset + 4)..])); + + Assert.Equal(1, BinaryPrimitives.ReadUInt16BigEndian(references[(FirstReferenceOffset + 8)..])); + Assert.Equal(2, BinaryPrimitives.ReadUInt16BigEndian(references[(FirstReferenceOffset + 10)..])); + Assert.Equal(2, BinaryPrimitives.ReadUInt16BigEndian(references[(FirstReferenceOffset + 12)..])); + Assert.Equal(3, BinaryPrimitives.ReadUInt16BigEndian(references[(FirstReferenceOffset + 14)..])); + + stream.Position = 0; + using Image decoded = Image.Load(stream); + Assert.Empty(ImageComparer.Exact.CompareImages(image, decoded)); + } + + [Fact] + public void Av1LosslessRoundTripPreservesColorAndAlpha() + { + const int width = 8; + const int height = 8; + using Image image = new(width, height); + for (int row = 0; row < height; row++) + { + Span pixels = image.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(row); + for (int column = 0; column < width; column++) + { + pixels[column] = new Rgba32( + (byte)((column * 31) + row), + (byte)((row * 29) + column), + (byte)((column * 17) + (row * 11)), + (byte)((column * 23) + (row * 7))); + } + } + + // Identity-matrix 4:4:4 maps the packed RGB channels directly onto AV1 planes, so codec losslessness + // can be asserted against the original pixels without a separate color-conversion tolerance. + image.Metadata.CicpProfile = new CicpProfile(1, 13, 0, true); + using MemoryStream stream = new(); + HeifEncoder encoder = new() + { + CompressionMethod = HeifCompressionMethod.Av1, + Lossless = true, + Effort = 0 + }; + + image.Save(stream, encoder); + byte[] file = stream.ToArray(); + Span colorPayload = GetItemPayload(file, 1); + using Av1Decoder colorDecoder = new(Configuration.Default); + using Image colorImage = colorDecoder.Decode(colorPayload); + ObuSequenceHeader colorSequenceHeader = Assert.IsType(colorDecoder.SequenceHeader); + ObuFrameHeader colorFrameHeader = Assert.IsType(colorDecoder.FrameHeader); + + Assert.Equal(Av1ColorFormat.Yuv444, colorSequenceHeader.ColorConfig.GetColorFormat()); + Assert.Equal(0, colorFrameHeader.QuantizationParameters.BaseQIndex); + Assert.True(colorFrameHeader.CodedLossless); + Assert.True(colorFrameHeader.AllLossless); + Assert.Equal(Av1TransformMode.Only4x4, colorFrameHeader.TransformMode); + + Span alphaPayload = GetItemPayload(file, 2); + using Av1Decoder alphaDecoder = new(Configuration.Default); + using Image alphaImage = alphaDecoder.Decode(alphaPayload); + ObuFrameHeader alphaFrameHeader = Assert.IsType(alphaDecoder.FrameHeader); + Assert.Equal(0, alphaFrameHeader.QuantizationParameters.BaseQIndex); + Assert.True(alphaFrameHeader.CodedLossless); + + stream.Position = 0; + using Image decoded = Image.Load(stream); + Assert.Empty(ImageComparer.Exact.CompareImages(image, decoded)); + + string outputDirectory = Path.Combine( + TestEnvironment.ActualOutputDirectoryFullPath, + "Formats", + "Heif", + "Av1"); + + Directory.CreateDirectory(outputDirectory); + File.WriteAllBytes(Path.Combine(outputDirectory, "encoder-public-lossless-color.obu"), colorPayload.ToArray()); + File.WriteAllBytes(Path.Combine(outputDirectory, "encoder-public-lossless-alpha.obu"), alphaPayload.ToArray()); + } + + [Theory] + [InlineData(HeifBitDepth.Bit10)] + [InlineData(HeifBitDepth.Bit12)] + public void Av1LosslessRoundTripPreservesHighBitDepthSourcePixels(HeifBitDepth bitDepth) + { + const int width = 8; + const int height = 8; + int codedMaximum = (1 << (int)bitDepth) - 1; + using Image image = new(width, height); + for (int row = 0; row < height; row++) + { + Span pixels = image.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(row); + for (int column = 0; column < width; column++) + { + int red = ((column * 131) + (row * 37) + 1) & codedMaximum; + int green = ((column * 61) + (row * 173) + 3) & codedMaximum; + int blue = ((column * 211) + (row * 47) + 5) & codedMaximum; + int alpha = ((column * 127) + (row * 89)) & codedMaximum; + pixels[column] = new Rgba64( + ExpandToUShort(red, codedMaximum), + ExpandToUShort(green, codedMaximum), + ExpandToUShort(blue, codedMaximum), + ExpandToUShort(alpha, codedMaximum)); + } + } + + // Full-range identity 4:4:4 preserves the requested sample lattice, isolating source precision from a + // deliberately lossy color matrix or chroma subsampling step. + image.Metadata.CicpProfile = new CicpProfile(1, 13, 0, true); + using MemoryStream stream = new(); + HeifEncoder encoder = new() + { + CompressionMethod = HeifCompressionMethod.Av1, + BitDepth = bitDepth, + ChromaSubsampling = HeifChromaSubsampling.Yuv444, + Lossless = true, + Effort = 0 + }; + + image.Save(stream, encoder); + byte[] file = stream.ToArray(); + Span colorPayload = GetItemPayload(file, 1); + Span alphaPayload = GetItemPayload(file, 2); + stream.Position = 0; + using Image decoded = Image.Load(stream); + Assert.Empty(ImageComparer.Exact.CompareImages(image, decoded)); + + string outputDirectory = Path.Combine( + TestEnvironment.ActualOutputDirectoryFullPath, + "Formats", + "Heif", + "Av1"); + + Directory.CreateDirectory(outputDirectory); + File.WriteAllBytes( + Path.Combine(outputDirectory, $"encoder-public-lossless-{(int)bitDepth}b-color.obu"), + colorPayload.ToArray()); + + File.WriteAllBytes( + Path.Combine(outputDirectory, $"encoder-public-lossless-{(int)bitDepth}b-alpha.obu"), + alphaPayload.ToArray()); + } + + private static ushort ExpandToUShort(int sample, int maximum) + => (ushort)(((sample * (long)ushort.MaxValue) + (maximum / 2)) / maximum); + + [Theory] + [InlineData((ushort)0, null, (ushort)0)] + [InlineData((ushort)3, null, (ushort)3)] + [InlineData((ushort)3, (ushort)7, (ushort)7)] + public void Av1LosslessImageSequencePreservesFramesTimingAndAlpha( + ushort metadataRepeatCount, + ushort? encoderRepeatCount, + ushort expectedRepeatCount) + { + const int width = 8; + const int height = 8; + const int frameCount = 3; + using Image image = new(width, height); + image.Frames.AddFrame(image.Frames.RootFrame); + image.Frames.AddFrame(image.Frames.RootFrame); + for (int frameIndex = 0; frameIndex < frameCount; frameIndex++) + { + ImageFrame frame = image.Frames[frameIndex]; + frame.Metadata.GetHeifMetadata().FrameDelay = frameIndex switch + { + 0 => new Rational(1, 24), + 1 => new Rational(1, 25), + _ => new Rational(1, 30) + }; + + for (int row = 0; row < height; row++) + { + Span pixels = frame.PixelBuffer.DangerousGetRowSpan(row); + for (int column = 0; column < width; column++) + { + pixels[column] = new Rgba32( + (byte)((frameIndex * 53) + (column * 19) + row), + (byte)((frameIndex * 31) + (row * 23) + column), + (byte)((frameIndex * 71) + (column * 7) + (row * 13)), + (byte)((frameIndex * 47) + (column * 17) + (row * 11))); + } + } + } + + image.Metadata.CicpProfile = new CicpProfile(1, 13, 0, true); + image.Metadata.IccProfile = new IccProfile(IccTestDataProfiles.ProfileRandomArray); + ExifProfile exifProfile = new(); + exifProfile.SetValue(ExifTag.Software, "ImageSharp AV1 sequence"); + image.Metadata.ExifProfile = exifProfile; + byte[] xmpData = Encoding.UTF8.GetBytes("ImageSharp AV1 sequence"); + image.Metadata.XmpProfile = new XmpProfile(xmpData); + image.Metadata.GetHeifMetadata().RepeatCount = metadataRepeatCount; + using MemoryStream stream = new(); + HeifEncoder encoder = new() + { + CompressionMethod = HeifCompressionMethod.Av1, + Lossless = true, + Effort = 0, + RepeatCount = encoderRepeatCount + }; + + image.Save(stream, encoder); + byte[] file = stream.ToArray(); + Assert.Equal((uint)Heif4CharCode.Avis, BinaryPrimitives.ReadUInt32BigEndian(file.AsSpan(8))); + Assert.Equal(1, BinaryPrimitives.ReadUInt16BigEndian(GetMetadataChild(file, Heif4CharCode.Pitm)[12..])); + + using (Av1Decoder sampleDecoder = new(Configuration.Default)) + using (Image decodedSample = sampleDecoder.Decode(GetItemPayload(file, 1))) + { + ObuSequenceHeader sampleHeader = sampleDecoder.SequenceHeader; + Assert.NotNull(sampleHeader); + Assert.False(sampleHeader.IsStillPicture); + Assert.False(sampleHeader.IsReducedStillPictureHeader); + } + + stream.Position = 0; + DecoderOptions preserveOptions = new() { ColorProfileHandling = ColorProfileHandling.Preserve }; + using Image decoded = Image.Load(preserveOptions, stream); + Assert.Equal(frameCount, decoded.Frames.Count); + Assert.Equal(expectedRepeatCount, decoded.Metadata.GetHeifMetadata().RepeatCount); + Assert.True(decoded.Metadata.GetHeifMetadata().AnimateRootFrame); + Assert.Empty(ImageComparer.Exact.CompareImages(image, decoded)); + Assert.Equal( + IccTestDataProfiles.ProfileRandomArray, + Assert.IsType(decoded.Metadata.IccProfile).ToByteArray()); + + ExifProfile decodedExif = Assert.IsType(decoded.Metadata.ExifProfile); + Assert.True(decodedExif.TryGetValue(ExifTag.Software, out IExifValue software)); + Assert.Equal("ImageSharp AV1 sequence", software.Value); + Assert.Equal(xmpData, Assert.IsType(decoded.Metadata.XmpProfile).ToByteArray()); + for (int frameIndex = 0; frameIndex < frameCount; frameIndex++) + { + Assert.Equal( + image.Frames[frameIndex].Metadata.GetHeifMetadata().FrameDelay, + decoded.Frames[frameIndex].Metadata.GetHeifMetadata().FrameDelay); + } + } + + [Fact] + public void Av1ImageSequenceWritesToPrefixedNonSeekableStream() + { + using Image image = new(8, 8); + image.Frames.AddFrame(image.Frames.RootFrame); + image.Frames.RootFrame.Metadata.GetHeifMetadata().FrameDelay = new Rational(1, 10); + image.Frames[1].Metadata.GetHeifMetadata().FrameDelay = new Rational(1, 20); + image.Metadata.IccProfile = new IccProfile(IccTestDataProfiles.ProfileRandomArray); + image.Metadata.ExifProfile = new ExifProfile(); + image.Metadata.ExifProfile.SetValue(ExifTag.Software, "suppressed"); + image.Metadata.XmpProfile = new XmpProfile(Encoding.UTF8.GetBytes("suppressed")); + using MemoryStream storage = new(); + storage.Write([1, 2, 3, 4]); + long fileStart = storage.Position; + using NonSeekableStream destination = new(storage); + HeifEncoder encoder = new() + { + CompressionMethod = HeifCompressionMethod.Av1, + Effort = 0, + SkipMetadata = true + }; + + image.Save(destination, encoder); + storage.Position = fileStart; + using Image decoded = Image.Load(storage); + Assert.Equal(image.Size, decoded.Size); + Assert.Equal(image.Frames.Count, decoded.Frames.Count); + Assert.Null(decoded.Metadata.IccProfile); + Assert.Null(decoded.Metadata.ExifProfile); + Assert.Null(decoded.Metadata.XmpProfile); + } + + [Theory] + [InlineData(0, 255)] + [InlineData(1, 249)] + [InlineData(2, 249)] + [InlineData(50, 128)] + [InlineData(60, 100)] + [InlineData(75, 64)] + [InlineData(99, 4)] + [InlineData(100, 4)] + public void Av1QualityMapsThroughLibaomQuantizers(int quality, int expectedQIndex) + => Assert.Equal(expectedQIndex, HeifEncoderCore.GetAv1QuantizerIndex(quality)); + + [Fact] + public void Av1WritesStillImageWithRequiredBrandsAndProductionPayload() + { + const int width = 16; + const int height = 16; + using Image image = new(width, height); + for (int row = 0; row < height; row++) + { + Span pixels = image.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(row); + for (int column = 0; column < width; column++) + { + pixels[column] = new Rgb24( + (byte)(column * 11), + (byte)(row * 13), + (byte)((column + row) * 7)); + } + } + + using MemoryStream stream = new(); + HeifEncoder encoder = new() + { + CompressionMethod = HeifCompressionMethod.Av1, + Quality = 75, + Effort = 0 + }; + + image.Save(stream, encoder); + byte[] file = stream.ToArray(); + ReadOnlySpan fileType = GetTopLevelBox(file, Heif4CharCode.Ftyp); + Assert.Equal(28, BinaryPrimitives.ReadInt32BigEndian(fileType)); + Assert.Equal(Heif4CharCode.Ftyp, (Heif4CharCode)BinaryPrimitives.ReadUInt32BigEndian(fileType[4..])); + Assert.Equal(Heif4CharCode.Avif, (Heif4CharCode)BinaryPrimitives.ReadUInt32BigEndian(fileType[8..])); + Assert.Equal(0, BinaryPrimitives.ReadInt32BigEndian(fileType[12..])); + Assert.Equal(Heif4CharCode.Avif, (Heif4CharCode)BinaryPrimitives.ReadUInt32BigEndian(fileType[16..])); + Assert.Equal(Heif4CharCode.Mif1, (Heif4CharCode)BinaryPrimitives.ReadUInt32BigEndian(fileType[20..])); + Assert.Equal(Heif4CharCode.Miaf, (Heif4CharCode)BinaryPrimitives.ReadUInt32BigEndian(fileType[24..])); + + Span payload = GetItemPayload(file, 1); + using Av1Decoder payloadDecoder = new(Configuration.Default); + using Image payloadImage = payloadDecoder.Decode(payload); + ObuFrameHeader frameHeader = Assert.IsType(payloadDecoder.FrameHeader); + Assert.Equal(64, frameHeader.QuantizationParameters.BaseQIndex); + Assert.Equal(image.Size, payloadImage.Size); + + stream.Position = 0; + using Image decoded = Image.Load(stream); + HeifMetadata metadata = decoded.Metadata.GetHeifMetadata(); + Assert.Equal(image.Size, decoded.Size); + Assert.Equal(HeifCompressionMethod.Av1, metadata.CompressionMethod); + Assert.Equal(HeifBitDepth.Bit8, metadata.BitDepth); + Assert.False(metadata.IsMonochrome); + Assert.False(metadata.HasAlpha); + CicpProfile colorProfile = Assert.IsType(decoded.Metadata.CicpProfile); + Assert.Equal(CicpMatrixCoefficients.ItuRBt601_7_525, colorProfile.MatrixCoefficients); + Assert.False(colorProfile.FullRange); + + string outputDirectory = Path.Combine( + TestEnvironment.ActualOutputDirectoryFullPath, + "Formats", + "Heif", + "Av1"); + + Directory.CreateDirectory(outputDirectory); + File.WriteAllBytes(Path.Combine(outputDirectory, "encoder-public-16x16-8b-420.avif"), file); + File.WriteAllBytes(Path.Combine(outputDirectory, "encoder-public-16x16-8b-420.obu"), payload.ToArray()); + } + + [Fact] + public void Av1WritesAuxiliaryAlphaFromSourcePixelType() + { + const int width = 16; + const int height = 8; + using Image image = new(width, height); + for (int row = 0; row < height; row++) + { + Span pixels = image.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(row); + for (int column = 0; column < width; column++) + { + pixels[column] = column < 8 + ? new Rgba32(40, 80, 120, 0) + : new Rgba32(40, 80, 120, 255); + } + } + + using MemoryStream stream = new(); + HeifEncoder encoder = new() + { + CompressionMethod = HeifCompressionMethod.Av1, + Quality = 75, + AlphaQuality = 100, + Effort = 0 + }; + + image.Save(stream, encoder); + byte[] file = stream.ToArray(); + Span colorPayload = GetItemPayload(file, 1); + Span alphaPayload = GetItemPayload(file, 2); + using Av1Decoder colorDecoder = new(Configuration.Default); + using Image colorImage = colorDecoder.Decode(colorPayload); + ObuFrameHeader colorFrameHeader = Assert.IsType(colorDecoder.FrameHeader); + Assert.Equal(64, colorFrameHeader.QuantizationParameters.BaseQIndex); + + using Av1Decoder alphaDecoder = new(Configuration.Default); + using Image alphaImage = alphaDecoder.Decode(alphaPayload); + ObuSequenceHeader alphaSequenceHeader = Assert.IsType(alphaDecoder.SequenceHeader); + ObuFrameHeader alphaFrameHeader = Assert.IsType(alphaDecoder.FrameHeader); + Assert.True(alphaSequenceHeader.ColorConfig.IsMonochrome); + Assert.Equal(4, alphaFrameHeader.QuantizationParameters.BaseQIndex); + + stream.Position = 0; + using Image decoded = Image.Load(stream); + HeifMetadata metadata = decoded.Metadata.GetHeifMetadata(); + Assert.True(metadata.HasAlpha); + Assert.InRange(decoded[0, 0].A, (byte)0, (byte)8); + Assert.InRange(decoded[width - 1, 0].A, (byte)247, byte.MaxValue); + + string outputDirectory = Path.Combine( + TestEnvironment.ActualOutputDirectoryFullPath, + "Formats", + "Heif", + "Av1"); + + Directory.CreateDirectory(outputDirectory); + File.WriteAllBytes(Path.Combine(outputDirectory, "encoder-public-alpha-color.obu"), colorPayload.ToArray()); + File.WriteAllBytes(Path.Combine(outputDirectory, "encoder-public-alpha-auxiliary.obu"), alphaPayload.ToArray()); + } + + [Theory] + [InlineData(HeifBitDepth.Bit8, HeifChromaSubsampling.Monochrome, Av1EightBit, Yuv400)] + [InlineData(HeifBitDepth.Bit10, HeifChromaSubsampling.Yuv420, Av1TenBit, Yuv420)] + [InlineData(HeifBitDepth.Bit10, HeifChromaSubsampling.Yuv422, Av1TenBit, Yuv422)] + [InlineData(HeifBitDepth.Bit12, HeifChromaSubsampling.Yuv444, Av1TwelveBit, Yuv444)] + public void Av1ExplicitPrecisionAndSamplingReachPayload( + HeifBitDepth bitDepth, + HeifChromaSubsampling chromaSubsampling, + int expectedAv1BitDepthValue, + int expectedColorFormatValue) + { + Av1BitDepth expectedAv1BitDepth = (Av1BitDepth)expectedAv1BitDepthValue; + Av1ColorFormat expectedColorFormat = (Av1ColorFormat)expectedColorFormatValue; + using Image image = new(8, 8); + for (int row = 0; row < image.Height; row++) + { + Span pixels = image.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(row); + for (int column = 0; column < image.Width; column++) + { + pixels[column] = new Rgb24( + (byte)(column * 29), + (byte)(row * 29), + (byte)((column + row) * 13)); + } + } + + image.Metadata.GetHeifMetadata().BitDepth = HeifBitDepth.Bit10; + using MemoryStream stream = new(); + HeifEncoder encoder = new() + { + CompressionMethod = HeifCompressionMethod.Av1, + BitDepth = bitDepth, + ChromaSubsampling = chromaSubsampling, + Effort = 0 + }; + + image.Save(stream, encoder); + byte[] file = stream.ToArray(); + Span payload = GetItemPayload(file, 1); + using Av1Decoder payloadDecoder = new(Configuration.Default); + using Image payloadImage = payloadDecoder.Decode(payload); + ObuSequenceHeader sequenceHeader = Assert.IsType(payloadDecoder.SequenceHeader); + Assert.Equal(expectedAv1BitDepth, sequenceHeader.ColorConfig.BitDepth); + Assert.Equal(expectedColorFormat, sequenceHeader.ColorConfig.GetColorFormat()); + Assert.Equal(image.Size, payloadImage.Size); + + stream.Position = 0; + using Image decoded = Image.Load(stream); + HeifMetadata metadata = decoded.Metadata.GetHeifMetadata(); + Assert.Equal(bitDepth, metadata.BitDepth); + Assert.Equal(chromaSubsampling == HeifChromaSubsampling.Monochrome, metadata.IsMonochrome); + + string outputDirectory = Path.Combine( + TestEnvironment.ActualOutputDirectoryFullPath, + "Formats", + "Heif", + "Av1"); + + Directory.CreateDirectory(outputDirectory); + File.WriteAllBytes( + Path.Combine(outputDirectory, $"encoder-public-8x8-{(int)bitDepth}b-{chromaSubsampling}.obu"), + payload.ToArray()); + } + + [Fact] + public void Av1UsesMetadataBitDepthWhenOptionIsNull() + { + using Image image = new(8, 8); + image.Metadata.GetHeifMetadata().BitDepth = HeifBitDepth.Bit10; + using MemoryStream stream = new(); + HeifEncoder encoder = new() + { + CompressionMethod = HeifCompressionMethod.Av1, + ChromaSubsampling = HeifChromaSubsampling.Yuv444, + Effort = 0 + }; + + image.Save(stream, encoder); + byte[] file = stream.ToArray(); + Span payload = GetItemPayload(file, 1); + using Av1Decoder payloadDecoder = new(Configuration.Default); + using Image payloadImage = payloadDecoder.Decode(payload); + ObuSequenceHeader sequenceHeader = Assert.IsType(payloadDecoder.SequenceHeader); + Assert.Equal(Av1BitDepth.TenBit, sequenceHeader.ColorConfig.BitDepth); + } + + [Theory] + [InlineData(HeifBitDepth.Bit8, false, false, false)] + [InlineData(HeifBitDepth.Bit8, true, false, false)] + [InlineData(HeifBitDepth.Bit10, false, false, false)] + [InlineData(HeifBitDepth.Bit10, true, false, false)] + [InlineData(HeifBitDepth.Bit12, false, false, false)] + [InlineData(HeifBitDepth.Bit12, true, false, false)] + [InlineData(HeifBitDepth.Bit8, false, true, false)] + [InlineData(HeifBitDepth.Bit8, true, true, false)] + [InlineData(HeifBitDepth.Bit10, false, true, false)] + [InlineData(HeifBitDepth.Bit10, true, true, false)] + [InlineData(HeifBitDepth.Bit12, false, true, false)] + [InlineData(HeifBitDepth.Bit12, true, true, false)] + [InlineData(HeifBitDepth.Bit8, false, false, true)] + [InlineData(HeifBitDepth.Bit12, false, true, true)] + public void Av1PreservesIdentityMatrixColorDescription( + HeifBitDepth bitDepth, + bool fullRange, + bool sequence, + bool srgb) + { + const int Width = 8; + const int Height = 8; + using Image image = new(Width, Height); + if (sequence) + { + image.Frames.AddFrame(image.Frames.RootFrame); + } + + for (int frameIndex = 0; frameIndex < image.Frames.Count; frameIndex++) + { + ImageFrame frame = image.Frames[frameIndex]; + frame.Metadata.GetHeifMetadata().FrameDelay = new Rational(1, 25); + for (int y = 0; y < Height; y++) + { + Span row = frame.PixelBuffer.DangerousGetRowSpan(y); + for (int x = 0; x < Width; x++) + { + row[x] = new Rgb24( + (byte)((x * 31) + y + frameIndex), + (byte)((y * 29) + x + frameIndex), + (byte)((x * 17) + (y * 11) + frameIndex)); + } + } + } + + // BT.2020/PQ identity uses explicit range syntax. Only the BT.709/sRGB identity combination + // infers full range, so its limited-range metadata must be normalized before pixel conversion. + CicpProfile profile = srgb ? new(1, 13, 0, fullRange) : new(9, 16, 0, fullRange); + image.Metadata.CicpProfile = profile; + bool expectedFullRange = fullRange || srgb; + using MemoryStream stream = new(); + image.Save(stream, new HeifEncoder + { + CompressionMethod = HeifCompressionMethod.Av1, + BitDepth = bitDepth, + ChromaSubsampling = HeifChromaSubsampling.Yuv444, + Lossless = true, + Effort = 0 + }); + + Assert.Same(profile, image.Metadata.CicpProfile); + Assert.Equal(fullRange, profile.FullRange); + byte[] file = stream.ToArray(); + using Av1Decoder sampleDecoder = new(Configuration.Default); + using Image sample = sampleDecoder.Decode(GetItemPayload(file, 1)); + ObuSequenceHeader header = Assert.IsType(sampleDecoder.SequenceHeader); + Assert.Equal(ObuMatrixCoefficients.Identity, header.ColorConfig.MatrixCoefficients); + Assert.Equal((byte)profile.ColorPrimaries, (byte)header.ColorConfig.ColorPrimaries); + Assert.Equal((byte)profile.TransferCharacteristics, (byte)header.ColorConfig.TransferCharacteristics); + Assert.Equal(expectedFullRange, header.ColorConfig.ColorRange); + Assert.Equal(Av1ColorFormat.Yuv444, header.ColorConfig.GetColorFormat()); + + stream.Position = 0; + DecoderOptions options = new() { ColorProfileHandling = ColorProfileHandling.Preserve }; + using Image decoded = Image.Load(options, stream); + CicpProfile decodedProfile = Assert.IsType(decoded.Metadata.CicpProfile); + Assert.Equal(profile.ColorPrimaries, decodedProfile.ColorPrimaries); + Assert.Equal(profile.TransferCharacteristics, decodedProfile.TransferCharacteristics); + Assert.Equal(CicpMatrixCoefficients.Identity, decodedProfile.MatrixCoefficients); + Assert.Equal(expectedFullRange, decodedProfile.FullRange); + Assert.Equal(image.Frames.Count, decoded.Frames.Count); + for (int frameIndex = 0; frameIndex < image.Frames.Count; frameIndex++) + { + for (int y = 0; y < Height; y++) + { + ReadOnlySpan expectedRow = image.Frames[frameIndex].PixelBuffer.DangerousGetRowSpan(y); + ReadOnlySpan actualRow = decoded.Frames[frameIndex].PixelBuffer.DangerousGetRowSpan(y); + for (int x = 0; x < Width; x++) + { + // Limited-range conversion rounds onto 219 codes before the lossless codec stage. + Assert.InRange((int)actualRow[x].R - expectedRow[x].R, -1, 1); + Assert.InRange((int)actualRow[x].G - expectedRow[x].G, -1, 1); + Assert.InRange((int)actualRow[x].B - expectedRow[x].B, -1, 1); + } + } + } + + string directory = TestEnvironment.CreateOutputDirectory("Heif", "Av1", nameof(this.Av1PreservesIdentityMatrixColorDescription)); + string name = $"{(int)bitDepth}-{fullRange}-{sequence}-{srgb}"; + File.WriteAllBytes(Path.Combine(directory, name + ".obu"), GetTopLevelBox(file, Heif4CharCode.Mdat)[8..].ToArray()); + using BinaryWriter expectedSamples = new(File.Create(Path.Combine(directory, name + ".expected.yuv"))); + int depthScale = 1 << ((int)bitDepth - 8); + int bias = expectedFullRange ? 0 : 16 * depthScale; + int range = expectedFullRange ? (1 << (int)bitDepth) - 1 : 219 * depthScale; + for (int frameIndex = 0; frameIndex < image.Frames.Count; frameIndex++) + { + for (int plane = 0; plane < 3; plane++) + { + for (int y = 0; y < Height; y++) + { + ReadOnlySpan row = image.Frames[frameIndex].PixelBuffer.DangerousGetRowSpan(y); + for (int x = 0; x < Width; x++) + { + // The independent reference is G, B, R with the luma range on every plane. + // These integer sample expectations do not call the production color converter. + int channel = plane == 0 ? row[x].G : plane == 1 ? row[x].B : row[x].R; + int value = bias + (((channel * range) + 127) / 255); + if (bitDepth == HeifBitDepth.Bit8) + { + expectedSamples.Write((byte)value); + } + else + { + expectedSamples.Write((ushort)value); + } + } + } + } + } + } + + [Theory] + [InlineData(CicpMatrixCoefficients.Identity, HeifChromaSubsampling.Yuv420)] + [InlineData(CicpMatrixCoefficients.YCgCoRe, null)] + [InlineData(CicpMatrixCoefficients.YCgCoRe, HeifChromaSubsampling.Yuv420)] + [InlineData(CicpMatrixCoefficients.YCgCoRe, HeifChromaSubsampling.Yuv422)] + [InlineData(CicpMatrixCoefficients.YCgCoRo, null)] + [InlineData(CicpMatrixCoefficients.YCgCoRo, HeifChromaSubsampling.Yuv420)] + [InlineData(CicpMatrixCoefficients.YCgCoRo, HeifChromaSubsampling.Yuv422)] + public void Av1SanitizesIncompatibleMatrixWithoutMutatingSourceMetadata( + CicpMatrixCoefficients matrix, + HeifChromaSubsampling? subsampling) + { + using Image image = new(8, 8, new Rgb24(32, 96, 192)); + CicpProfile sourceProfile = new(1, 13, (byte)matrix, false); + image.Metadata.CicpProfile = sourceProfile; + using MemoryStream stream = new(); + HeifEncoder encoder = new() + { + CompressionMethod = HeifCompressionMethod.Av1, + ChromaSubsampling = subsampling, + Effort = 0 + }; + + image.Save(stream, encoder); + Assert.Same(sourceProfile, image.Metadata.CicpProfile); + Assert.Equal(matrix, sourceProfile.MatrixCoefficients); + Assert.False(sourceProfile.FullRange); + + stream.Position = 0; + using Image decoded = Image.Load(stream); + CicpProfile decodedProfile = Assert.IsType(decoded.Metadata.CicpProfile); + Assert.Equal(CicpMatrixCoefficients.ItuRBt601_7_525, decodedProfile.MatrixCoefficients); + Assert.False(decodedProfile.FullRange); + + // A fallback must change the actual encoded conversion as well as its metadata. Compare with the + // same packed pixels explicitly encoded using that fallback matrix and the requested sampling. + using Image explicitConversion = image.Clone(); + explicitConversion.Metadata.CicpProfile = new CicpProfile(1, 13, (byte)CicpMatrixCoefficients.ItuRBt601_7_525, false); + using MemoryStream expected = new(); + explicitConversion.Save(expected, encoder); + Assert.Equal(expected.ToArray(), stream.ToArray()); + } + + [Fact] + public void Av1PreservesIccExifAndXmpMetadata() + { + using Image image = new(8, 8); + image.Metadata.IccProfile = new IccProfile(IccTestDataProfiles.ProfileRandomArray); + + ExifProfile generatedExif = new(); + generatedExif.SetValue(ExifTag.Software, "ImageSharp HEIF"); + byte[] exifData = generatedExif.ToByteArray(); + image.Metadata.ExifProfile = generatedExif; + + byte[] xmpData = Encoding.UTF8.GetBytes("ImageSharp HEIF"); + image.Metadata.XmpProfile = new XmpProfile(xmpData); + + using MemoryStream stream = new(); + HeifEncoder encoder = new() + { + CompressionMethod = HeifCompressionMethod.Av1, + Effort = 0 + }; + + image.Save(stream, encoder); + byte[] file = stream.ToArray(); + + Span itemInfo = GetMetadataChild(file, Heif4CharCode.Iinf); + Assert.Equal(3, BinaryPrimitives.ReadUInt16BigEndian(itemInfo[12..])); + int entryOffset = 14; + + int colorEntryLength = BinaryPrimitives.ReadInt32BigEndian(itemInfo[entryOffset..]); + Assert.Equal(1, BinaryPrimitives.ReadUInt16BigEndian(itemInfo[(entryOffset + 12)..])); + Assert.Equal(Heif4CharCode.Av01, (Heif4CharCode)BinaryPrimitives.ReadUInt32BigEndian(itemInfo[(entryOffset + 16)..])); + Assert.Equal([0], itemInfo.Slice(entryOffset + 20, colorEntryLength - 20).ToArray()); + entryOffset += colorEntryLength; + + int exifEntryLength = BinaryPrimitives.ReadInt32BigEndian(itemInfo[entryOffset..]); + Assert.Equal(2, BinaryPrimitives.ReadUInt16BigEndian(itemInfo[(entryOffset + 12)..])); + Assert.Equal(Heif4CharCode.Exif, (Heif4CharCode)BinaryPrimitives.ReadUInt32BigEndian(itemInfo[(entryOffset + 16)..])); + Assert.Equal("Exif\0", Encoding.UTF8.GetString(itemInfo.Slice(entryOffset + 20, exifEntryLength - 20))); + entryOffset += exifEntryLength; + + int xmpEntryLength = BinaryPrimitives.ReadInt32BigEndian(itemInfo[entryOffset..]); + Assert.Equal(3, BinaryPrimitives.ReadUInt16BigEndian(itemInfo[(entryOffset + 12)..])); + Assert.Equal(Heif4CharCode.Mime, (Heif4CharCode)BinaryPrimitives.ReadUInt32BigEndian(itemInfo[(entryOffset + 16)..])); + Assert.Equal( + "XMP\0application/rdf+xml\0", + Encoding.UTF8.GetString(itemInfo.Slice(entryOffset + 20, xmpEntryLength - 20))); + + entryOffset += xmpEntryLength; + Assert.Equal(itemInfo.Length, entryOffset); + + Span itemReferences = GetMetadataChild(file, Heif4CharCode.Iref); + int referenceOffset = 12; + for (ushort sourceId = 2; sourceId <= 3; sourceId++) + { + int referenceLength = BinaryPrimitives.ReadInt32BigEndian(itemReferences[referenceOffset..]); + Assert.Equal(14, referenceLength); + Assert.Equal( + Heif4CharCode.Cdsc, + (Heif4CharCode)BinaryPrimitives.ReadUInt32BigEndian(itemReferences[(referenceOffset + 4)..])); + + Assert.Equal(sourceId, BinaryPrimitives.ReadUInt16BigEndian(itemReferences[(referenceOffset + 8)..])); + Assert.Equal(1, BinaryPrimitives.ReadUInt16BigEndian(itemReferences[(referenceOffset + 10)..])); + Assert.Equal(1, BinaryPrimitives.ReadUInt16BigEndian(itemReferences[(referenceOffset + 12)..])); + referenceOffset += referenceLength; + } + + Assert.Equal(itemReferences.Length, referenceOffset); + + Span encodedExif = GetItemPayload(file, 2); + Assert.Equal(0U, BinaryPrimitives.ReadUInt32BigEndian(encodedExif)); + Assert.Equal(exifData, encodedExif[4..].ToArray()); + Assert.Equal(xmpData, GetItemPayload(file, 3).ToArray()); + + DecoderOptions preserveOptions = new() { ColorProfileHandling = ColorProfileHandling.Preserve }; + using Image decoded = Image.Load(preserveOptions, file); + Assert.Equal( + IccTestDataProfiles.ProfileRandomArray, + Assert.IsType(decoded.Metadata.IccProfile).ToByteArray()); + + ExifProfile decodedExif = Assert.IsType(decoded.Metadata.ExifProfile); + Assert.True(decodedExif.TryGetValue(ExifTag.Software, out IExifValue software)); + Assert.Equal("ImageSharp HEIF", software.Value); + Assert.Equal(xmpData, Assert.IsType(decoded.Metadata.XmpProfile).ToByteArray()); + } + + [Fact] + public void Av1SkipMetadataSuppressesIccExifAndXmp() + { + using Image image = new(8, 8); + image.Metadata.IccProfile = new IccProfile(IccTestDataProfiles.ProfileRandomArray); + image.Metadata.ExifProfile = new ExifProfile(); + image.Metadata.ExifProfile.SetValue(ExifTag.Software, "ImageSharp HEIF"); + image.Metadata.XmpProfile = new XmpProfile(Encoding.UTF8.GetBytes("ImageSharp HEIF")); + + using MemoryStream stream = new(); + HeifEncoder encoder = new() + { + CompressionMethod = HeifCompressionMethod.Av1, + Effort = 0, + SkipMetadata = true + }; + + image.Save(stream, encoder); + byte[] file = stream.ToArray(); + Span itemInfo = GetMetadataChild(file, Heif4CharCode.Iinf); + Assert.Equal(1, BinaryPrimitives.ReadUInt16BigEndian(itemInfo[12..])); + + Span itemProperties = GetMetadataChild(file, Heif4CharCode.Iprp); + const int IpcoOffset = 8; + int ipcoEnd = IpcoOffset + BinaryPrimitives.ReadInt32BigEndian(itemProperties[IpcoOffset..]); + int propertyOffset = IpcoOffset + 8; + while (propertyOffset < ipcoEnd) + { + int propertyLength = BinaryPrimitives.ReadInt32BigEndian(itemProperties[propertyOffset..]); + Heif4CharCode propertyType = (Heif4CharCode)BinaryPrimitives.ReadUInt32BigEndian(itemProperties[(propertyOffset + 4)..]); + if (propertyType == Heif4CharCode.Colr) + { + Assert.Equal( + Heif4CharCode.Nclx, + (Heif4CharCode)BinaryPrimitives.ReadUInt32BigEndian(itemProperties[(propertyOffset + 8)..])); + } + + propertyOffset += propertyLength; + } + + Assert.Equal(ipcoEnd, propertyOffset); + + DecoderOptions preserveOptions = new() { ColorProfileHandling = ColorProfileHandling.Preserve }; + using Image decoded = Image.Load(preserveOptions, file); + Assert.Null(decoded.Metadata.IccProfile); + Assert.Null(decoded.Metadata.ExifProfile); + Assert.Null(decoded.Metadata.XmpProfile); + } + + [Fact] + public void Av1WritesNonSeekableStream() + { + using Image image = new(8, 8); + using MemoryStream storage = new(); + using NonSeekableStream destination = new(storage); + HeifEncoder encoder = new() + { + CompressionMethod = HeifCompressionMethod.Av1, + Effort = 0 + }; + + image.Save(destination, encoder); + Assert.NotEqual(0, storage.Length); + storage.Position = 0; + using Image decoded = Image.Load(storage); + Assert.Equal(image.Size, decoded.Size); + } + + [Fact] + public void Av1WritesAtCurrentStreamPosition() + { + using Image image = new(8, 8); + using MemoryStream stream = new(); + stream.Write([1, 2, 3, 4]); + long fileStart = stream.Position; + HeifEncoder encoder = new() + { + CompressionMethod = HeifCompressionMethod.Av1, + Effort = 0 + }; + + image.Save(stream, encoder); + stream.Position = fileStart; + using Image decoded = Image.Load(stream); + Assert.Equal(image.Size, decoded.Size); + } + + [Fact] + public void Av1ItemPropertiesWriteRequiredTypesAndEssentialConfiguration() + { + ObuSequenceHeader colorHeader = new() + { + SequenceProfile = ObuSequenceProfile.Main, + OperatingPoint = [new ObuOperatingPoint { SequenceLevelIndex = 31 }], + ColorConfig = new ObuColorConfig + { + BitDepth = Av1BitDepth.TenBit, + SubSamplingX = true, + SubSamplingY = true + } + }; + + ObuSequenceHeader alphaHeader = new() + { + SequenceProfile = ObuSequenceProfile.Main, + OperatingPoint = [new ObuOperatingPoint { SequenceLevelIndex = 31 }], + ColorConfig = new ObuColorConfig + { + BitDepth = Av1BitDepth.TenBit, + IsMonochrome = true, + SubSamplingX = true, + SubSamplingY = true + } + }; + + HeifItem colorItem = new(Heif4CharCode.Av01, 1) + { + ChannelBitDepths = [10, 10, 10], + Av1CodecConfiguration = new Av1CodecConfiguration(colorHeader), + CicpProfile = new CicpProfile(1, 13, 6, true) + }; + + colorItem.SetExtent(new Size(64, 48)); + HeifItem alphaItem = new(Heif4CharCode.Av01, 2) + { + ChannelBitDepths = [10], + Av1CodecConfiguration = new Av1CodecConfiguration(alphaHeader), + AuxiliaryType = HeifConstants.AlphaAuxiliaryType + }; + + alphaItem.SetExtent(new Size(64, 48)); + List items = [colorItem, alphaItem]; + int expectedLength = HeifEncoderCore.GetItemPropertiesBoxLength(items); + using IMemoryOwner owner = Configuration.Default.MemoryAllocator.Allocate(expectedLength); + Span propertyBox = owner.Memory.Span[..expectedLength]; + int length = HeifEncoderCore.WriteItemPropertiesBox(propertyBox, 0, items); + + Assert.Equal(length, BinaryPrimitives.ReadInt32BigEndian(propertyBox)); + Assert.Equal(Heif4CharCode.Iprp, (Heif4CharCode)BinaryPrimitives.ReadUInt32BigEndian(propertyBox[4..])); + + const int IpcoOffset = 8; + int ipcoSize = BinaryPrimitives.ReadInt32BigEndian(propertyBox[IpcoOffset..]); + Assert.Equal(Heif4CharCode.Ipco, (Heif4CharCode)BinaryPrimitives.ReadUInt32BigEndian(propertyBox[(IpcoOffset + 4)..])); + + Heif4CharCode[] expectedTypes = + [ + Heif4CharCode.Ispe, + Heif4CharCode.Pixi, + Heif4CharCode.Av1C, + Heif4CharCode.Colr, + Heif4CharCode.Ispe, + Heif4CharCode.Pixi, + Heif4CharCode.Av1C, + Heif4CharCode.AuxC + ]; + + int propertyOffset = IpcoOffset + 8; + int ipcoEnd = IpcoOffset + ipcoSize; + int propertyIndex = 0; + while (propertyOffset < ipcoEnd) + { + int propertySize = BinaryPrimitives.ReadInt32BigEndian(propertyBox[propertyOffset..]); + Heif4CharCode propertyType = (Heif4CharCode)BinaryPrimitives.ReadUInt32BigEndian(propertyBox[(propertyOffset + 4)..]); + ReadOnlySpan payload = propertyBox.Slice(propertyOffset + 8, propertySize - 8); + Assert.Equal(expectedTypes[propertyIndex], propertyType); + switch (propertyIndex) + { + case 0: + case 4: + Assert.Equal(0, BinaryPrimitives.ReadInt32BigEndian(payload)); + Assert.Equal(64, BinaryPrimitives.ReadInt32BigEndian(payload[4..])); + Assert.Equal(48, BinaryPrimitives.ReadInt32BigEndian(payload[8..])); + break; + case 1: + Assert.Equal([0, 0, 0, 0, 3, 10, 10, 10], payload.ToArray()); + break; + case 2: + Assert.Equal([0x81, 0x1F, 0x4C, 0], payload.ToArray()); + break; + case 3: + Assert.Equal(Heif4CharCode.Nclx, (Heif4CharCode)BinaryPrimitives.ReadUInt32BigEndian(payload)); + Assert.Equal(1, BinaryPrimitives.ReadUInt16BigEndian(payload[4..])); + Assert.Equal(13, BinaryPrimitives.ReadUInt16BigEndian(payload[6..])); + Assert.Equal(6, BinaryPrimitives.ReadUInt16BigEndian(payload[8..])); + Assert.Equal(0x80, payload[10]); + break; + case 5: + Assert.Equal([0, 0, 0, 0, 1, 10], payload.ToArray()); + break; + case 6: + Assert.Equal([0x81, 0x1F, 0x5C, 0], payload.ToArray()); + break; + case 7: + Assert.Equal(0, BinaryPrimitives.ReadInt32BigEndian(payload)); + Assert.Equal(HeifConstants.AlphaAuxiliaryType, Encoding.UTF8.GetString(payload[4..^1])); + Assert.Equal(0, payload[^1]); + break; + } + + propertyOffset += propertySize; + propertyIndex++; + } + + Assert.Equal(expectedTypes.Length, propertyIndex); + Assert.Equal(ipcoEnd, propertyOffset); + + int ipmaOffset = ipcoEnd; + int ipmaSize = BinaryPrimitives.ReadInt32BigEndian(propertyBox[ipmaOffset..]); + Assert.Equal(length - ipmaOffset, ipmaSize); + Assert.Equal(Heif4CharCode.Ipma, (Heif4CharCode)BinaryPrimitives.ReadUInt32BigEndian(propertyBox[(ipmaOffset + 4)..])); + ReadOnlySpan ipmaPayload = propertyBox.Slice(ipmaOffset + 8, ipmaSize - 8); + Assert.Equal(0, BinaryPrimitives.ReadInt32BigEndian(ipmaPayload)); + Assert.Equal(2, BinaryPrimitives.ReadInt32BigEndian(ipmaPayload[4..])); + Assert.Equal( + [0, 1, 4, 1, 2, 0x83, 4, 0, 2, 4, 5, 6, 0x87, 8], + ipmaPayload[8..].ToArray()); + } + + [Fact] + public void ItemPropertiesReuseAnEarlierIdenticalCellPropertySet() + { + ObuSequenceHeader sequenceHeader = new() + { + SequenceProfile = ObuSequenceProfile.Main, + OperatingPoint = [new ObuOperatingPoint { SequenceLevelIndex = 31 }], + ColorConfig = new ObuColorConfig + { + BitDepth = Av1BitDepth.EightBit, + SubSamplingX = false, + SubSamplingY = false + } + }; + + HeifItem firstCell = new(Heif4CharCode.Av01, 1) + { + ChannelBitDepths = [8, 8, 8], + Av1CodecConfiguration = new Av1CodecConfiguration(sequenceHeader) + }; + + firstCell.SetExtent(new Size(64, 64)); + HeifItem secondCell = new(Heif4CharCode.Av01, 2) + { + PropertySource = firstCell + }; + + secondCell.SetExtent(firstCell.Extent); + List items = [firstCell, secondCell]; + int expectedLength = HeifEncoderCore.GetItemPropertiesBoxLength(items); + using IMemoryOwner owner = Configuration.Default.MemoryAllocator.Allocate(expectedLength); + Span propertyBox = owner.Memory.Span[..expectedLength]; + int length = HeifEncoderCore.WriteItemPropertiesBox(propertyBox, 0, items); + + Assert.Equal(92, length); + const int IpcoOffset = 8; + int ipmaOffset = IpcoOffset + BinaryPrimitives.ReadInt32BigEndian(propertyBox[IpcoOffset..]); + ReadOnlySpan ipmaPayload = propertyBox[(ipmaOffset + 8)..]; + Assert.Equal( + [0, 0, 0, 0, 0, 0, 0, 2, 0, 1, 3, 1, 2, 0x83, 0, 2, 3, 1, 2, 0x83], + ipmaPayload.ToArray()); + } + + [Fact] + public void Av1ItemPropertiesWriteIccBeforeCicpAndExcludeMetadataItemsFromAssociations() + { + IccProfile iccProfile = new(IccTestDataProfiles.ProfileRandomArray); + HeifItem colorItem = new(Heif4CharCode.Av01, 1) + { + IccProfile = iccProfile, + CicpProfile = new CicpProfile(1, 13, 6, true) + }; + + colorItem.SetExtent(new Size(64, 48)); + List items = + [ + colorItem, + new HeifItem(Heif4CharCode.Exif, 2), + new HeifItem(Heif4CharCode.Mime, 3) + ]; + + int expectedLength = HeifEncoderCore.GetItemPropertiesBoxLength(items); + using IMemoryOwner owner = Configuration.Default.MemoryAllocator.Allocate(expectedLength); + Span propertyBox = owner.Memory.Span[..expectedLength]; + int length = HeifEncoderCore.WriteItemPropertiesBox(propertyBox, 0, items); + const int IpcoOffset = 8; + int ipcoEnd = IpcoOffset + BinaryPrimitives.ReadInt32BigEndian(propertyBox[IpcoOffset..]); + int propertyOffset = IpcoOffset + 8; + + Assert.Equal(Heif4CharCode.Ispe, (Heif4CharCode)BinaryPrimitives.ReadUInt32BigEndian(propertyBox[(propertyOffset + 4)..])); + propertyOffset += BinaryPrimitives.ReadInt32BigEndian(propertyBox[propertyOffset..]); + + int iccPropertyLength = BinaryPrimitives.ReadInt32BigEndian(propertyBox[propertyOffset..]); + Assert.Equal(Heif4CharCode.Colr, (Heif4CharCode)BinaryPrimitives.ReadUInt32BigEndian(propertyBox[(propertyOffset + 4)..])); + Assert.Equal(Heif4CharCode.Prof, (Heif4CharCode)BinaryPrimitives.ReadUInt32BigEndian(propertyBox[(propertyOffset + 8)..])); + Assert.Equal( + IccTestDataProfiles.ProfileRandomArray, + propertyBox.Slice(propertyOffset + 12, iccPropertyLength - 12).ToArray()); + + propertyOffset += iccPropertyLength; + + int cicpPropertyLength = BinaryPrimitives.ReadInt32BigEndian(propertyBox[propertyOffset..]); + Assert.Equal(Heif4CharCode.Colr, (Heif4CharCode)BinaryPrimitives.ReadUInt32BigEndian(propertyBox[(propertyOffset + 4)..])); + Assert.Equal(Heif4CharCode.Nclx, (Heif4CharCode)BinaryPrimitives.ReadUInt32BigEndian(propertyBox[(propertyOffset + 8)..])); + propertyOffset += cicpPropertyLength; + Assert.Equal(ipcoEnd, propertyOffset); + + int ipmaOffset = ipcoEnd; + int ipmaLength = BinaryPrimitives.ReadInt32BigEndian(propertyBox[ipmaOffset..]); + ReadOnlySpan ipmaPayload = propertyBox.Slice(ipmaOffset + 8, ipmaLength - 8); + Assert.Equal(0, BinaryPrimitives.ReadInt32BigEndian(ipmaPayload)); + Assert.Equal(1, BinaryPrimitives.ReadInt32BigEndian(ipmaPayload[4..])); + Assert.Equal([0, 1, 3, 1, 2, 3], ipmaPayload[8..].ToArray()); + } + + [Fact] + public void ItemPropertiesUseLargeAssociationsWhenPropertyCountExceedsCompactRange() + { + const int ItemCount = 43; + ObuSequenceHeader sequenceHeader = new() + { + SequenceProfile = ObuSequenceProfile.Main, + OperatingPoint = [new ObuOperatingPoint { SequenceLevelIndex = 31 }], + ColorConfig = new ObuColorConfig + { + BitDepth = Av1BitDepth.EightBit, + SubSamplingX = true, + SubSamplingY = true + } + }; + + Av1CodecConfiguration codecConfiguration = new(sequenceHeader); + byte[] channelBitDepths = [8, 8, 8]; + List items = new(ItemCount); + for (uint itemId = 1; itemId <= ItemCount; itemId++) + { + HeifItem item = new(Heif4CharCode.Av01, itemId) + { + ChannelBitDepths = channelBitDepths, + Av1CodecConfiguration = codecConfiguration + }; + + item.SetExtent(new Size(1, 1)); + items.Add(item); + } + + int expectedLength = HeifEncoderCore.GetItemPropertiesBoxLength(items); + using IMemoryOwner owner = Configuration.Default.MemoryAllocator.Allocate(expectedLength); + Span propertyBox = owner.Memory.Span[..expectedLength]; + int length = HeifEncoderCore.WriteItemPropertiesBox(propertyBox, 0, items); + const int IpcoOffset = 8; + int ipcoSize = BinaryPrimitives.ReadInt32BigEndian(propertyBox[IpcoOffset..]); + int ipmaOffset = IpcoOffset + ipcoSize; + + Assert.Equal(1, BinaryPrimitives.ReadInt32BigEndian(propertyBox[(ipmaOffset + 8)..])); + Assert.Equal(ItemCount, BinaryPrimitives.ReadInt32BigEndian(propertyBox[(ipmaOffset + 12)..])); + + const int AssociationEntrySize = 9; + int finalEntryOffset = ipmaOffset + 16 + ((ItemCount - 1) * AssociationEntrySize); + Assert.Equal(ItemCount, BinaryPrimitives.ReadUInt16BigEndian(propertyBox[finalEntryOffset..])); + Assert.Equal(3, propertyBox[finalEntryOffset + 2]); + Assert.Equal(127, BinaryPrimitives.ReadUInt16BigEndian(propertyBox[(finalEntryOffset + 3)..])); + Assert.Equal(128, BinaryPrimitives.ReadUInt16BigEndian(propertyBox[(finalEntryOffset + 5)..])); + Assert.Equal(0x8081, BinaryPrimitives.ReadUInt16BigEndian(propertyBox[(finalEntryOffset + 7)..])); + } + + [Fact] + public void LegacyJpegEncodingDoesNotMutateSourceHeifMetadata() + { + using Image image = new(1, 1); + image[0, 0] = new Rgba32(10, 20, 30, 255); + HeifMetadata metadata = image.Metadata.GetHeifMetadata(); + metadata.CompressionMethod = HeifCompressionMethod.Av1; + using MemoryStream stream = new(); + HeifEncoder encoder = new() { CompressionMethod = HeifCompressionMethod.LegacyJpeg }; + + image.Save(stream, encoder); + + Assert.Same(metadata, image.Metadata.GetHeifMetadata()); + Assert.Equal(HeifCompressionMethod.Av1, metadata.CompressionMethod); + } + + [Theory] + [WithFile(TestImages.Heif.IrvineAvif, PixelTypes.Rgba32, HeifCompressionMethod.LegacyJpeg)] + public static void Encode(TestImageProvider provider, HeifCompressionMethod compressionMethod) + where TPixel : unmanaged, IPixel + { + using Image image = provider.GetImage(new MagickReferenceDecoder(HeifFormat.Instance)); + using MemoryStream stream = new(); + HeifEncoder encoder = new() { CompressionMethod = compressionMethod }; + image.Save(stream, encoder); + stream.Position = 0; + + ImageInfo imageInfo = Image.Identify(stream); + Assert.Equal(image.Size, imageInfo.Size); + + stream.Position = 0; + using Image encodedImage = Image.Load(stream); + HeifMetadata heifMetadata = encodedImage.Metadata.GetHeifMetadata(); + + ImageComparer.Exact.CompareImages(image, encodedImage); + Assert.Equal(compressionMethod, heifMetadata.CompressionMethod); + } + + private static Span GetItemPayload(Span file, ushort itemId) + { + int offset = 0; + while (offset < file.Length) + { + int boxSize = BinaryPrimitives.ReadInt32BigEndian(file[offset..]); + Heif4CharCode boxType = (Heif4CharCode)BinaryPrimitives.ReadUInt32BigEndian(file[(offset + 4)..]); + if (boxType == Heif4CharCode.Meta) + { + int childOffset = offset + 12; + int boxEnd = offset + boxSize; + while (childOffset < boxEnd) + { + int childSize = BinaryPrimitives.ReadInt32BigEndian(file[childOffset..]); + Heif4CharCode childType = (Heif4CharCode)BinaryPrimitives.ReadUInt32BigEndian(file[(childOffset + 4)..]); + if (childType == Heif4CharCode.Iloc) + { + int locationOffset = childOffset + 14; + int itemCount = BinaryPrimitives.ReadUInt16BigEndian(file[locationOffset..]); + locationOffset += 2; + for (int itemIndex = 0; itemIndex < itemCount; itemIndex++) + { + ushort currentItemId = BinaryPrimitives.ReadUInt16BigEndian(file[locationOffset..]); + locationOffset += 6; + int extentCount = BinaryPrimitives.ReadUInt16BigEndian(file[locationOffset..]); + locationOffset += 2; + for (int extentIndex = 0; extentIndex < extentCount; extentIndex++) + { + int itemOffset = checked((int)BinaryPrimitives.ReadUInt64BigEndian(file[locationOffset..])); + locationOffset += 8; + int itemLength = BinaryPrimitives.ReadInt32BigEndian(file[locationOffset..]); + locationOffset += 4; + if (currentItemId == itemId) + { + return file.Slice(itemOffset, itemLength); + } + } + } + } + + childOffset += childSize; + } + } + + offset += boxSize; + } + + throw new InvalidImageContentException($"The encoded file has no payload for item {itemId}."); + } + + private static uint GetItemInfoFlags(Span file, ushort itemId) + { + ReadOnlySpan itemInformation = GetMetadataChild(file, Heif4CharCode.Iinf); + int entryOffset = 14; + while (entryOffset < itemInformation.Length) + { + int entrySize = BinaryPrimitives.ReadInt32BigEndian(itemInformation[entryOffset..]); + ReadOnlySpan entry = itemInformation.Slice(entryOffset, entrySize); + ushort currentItemId = BinaryPrimitives.ReadUInt16BigEndian(entry[12..]); + if (currentItemId == itemId) + { + return (uint)((entry[9] << 16) | (entry[10] << 8) | entry[11]); + } + + entryOffset += entrySize; + } + + throw new InvalidImageContentException($"The encoded file has no item-information entry for item {itemId}."); + } + + private static Image DecodeSingleCellGrid(int width, int height, ObuColorConfig colorConfig) + { + using Image tile = new(width, height, new Rgba32(127, 127, 127)); + using MemoryStream payloadStream = new(); + ObuSequenceHeader header = Av1FrameEncoder.Encode( + Configuration.Default, + tile.Frames.RootFrame, + payloadStream, + colorConfig, + qIndex: 0, + effort: 0); + + byte[] payload = payloadStream.ToArray(); + HeifItem gridItem = new(Heif4CharCode.Grid, 1); + gridItem.SetExtent(new Size(width, height)); + HeifItem tileItem = new(Heif4CharCode.Av01, 2) + { + Av1CodecConfiguration = new Av1CodecConfiguration(header) + }; + + tileItem.SetExtent(new Size(width, height)); + HeifItemLink gridLink = new(Heif4CharCode.Dimg, gridItem.Id); + gridLink.DestinationIds.Add(tileItem.Id); + GridHeifItemDecoder decoder = new([gridItem, tileItem], [gridLink], ReadItem); + byte[] descriptor = new byte[8]; + BinaryPrimitives.WriteUInt16BigEndian(descriptor.AsSpan(4), (ushort)width); + BinaryPrimitives.WriteUInt16BigEndian(descriptor.AsSpan(6), (ushort)height); + return decoder.DecodeItemData( + new DecoderOptions { Configuration = Configuration.Default }, + gridItem, + descriptor, + null, + TestContext.Current.CancellationToken); + + IMemoryOwner ReadItem(HeifItem item) + { + Assert.Equal(tileItem.Id, item.Id); + IMemoryOwner owner = Configuration.Default.MemoryAllocator.Allocate(payload.Length); + payload.CopyTo(owner.Memory.Span); + return owner; + } + } + + private static Span GetTopLevelBox(Span file, Heif4CharCode requestedType) + { + int offset = 0; + while (offset < file.Length) + { + int boxSize = BinaryPrimitives.ReadInt32BigEndian(file[offset..]); + Heif4CharCode boxType = (Heif4CharCode)BinaryPrimitives.ReadUInt32BigEndian(file[(offset + sizeof(uint))..]); + if (boxType == requestedType) + { + return file.Slice(offset, boxSize); + } + + offset += boxSize; + } + + throw new InvalidImageContentException($"The encoded file has no {requestedType} top-level box."); + } + + private static Span GetMetadataChild(Span file, Heif4CharCode childType) + { + int offset = 0; + while (offset < file.Length) + { + int boxSize = BinaryPrimitives.ReadInt32BigEndian(file[offset..]); + Heif4CharCode boxType = (Heif4CharCode)BinaryPrimitives.ReadUInt32BigEndian(file[(offset + 4)..]); + if (boxType == Heif4CharCode.Meta) + { + int childOffset = offset + 12; + int boxEnd = offset + boxSize; + while (childOffset < boxEnd) + { + int childSize = BinaryPrimitives.ReadInt32BigEndian(file[childOffset..]); + Heif4CharCode currentChildType = + (Heif4CharCode)BinaryPrimitives.ReadUInt32BigEndian(file[(childOffset + 4)..]); + + if (currentChildType == childType) + { + return file.Slice(childOffset, childSize); + } + + childOffset += childSize; + } + } + + offset += boxSize; + } + + throw new InvalidImageContentException($"The encoded file has no {childType} metadata child."); + } +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/HeifFrameMetadataTests.cs b/tests/ImageSharp.Tests/Formats/Heif/HeifFrameMetadataTests.cs new file mode 100644 index 0000000000..789e2ad00f --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/HeifFrameMetadataTests.cs @@ -0,0 +1,69 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats; +using SixLabors.ImageSharp.Formats.Heif; +using SixLabors.ImageSharp.PixelFormats; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif; + +[Trait("Format", "Heif")] +public class HeifFrameMetadataTests +{ + [Fact] + public void FormatCreatesFrameMetadata() + { + HeifFrameMetadata metadata = HeifFormat.Instance.CreateDefaultFormatFrameMetadata(); + + Assert.Equal(new Rational(0), metadata.FrameDelay); + } + + [Fact] + public void DeepCloneCopiesFrameDelay() + { + HeifFrameMetadata metadata = new() { FrameDelay = new Rational(1, 4) }; + + HeifFrameMetadata clone = metadata.DeepClone(); + clone.FrameDelay = new Rational(1, 2); + + Assert.Equal(new Rational(1, 4), metadata.FrameDelay); + Assert.Equal(new Rational(1, 2), clone.FrameDelay); + } + + [Fact] + public void DurationRoundTripsFormatConnectingMetadata() + { + FormatConnectingFrameMetadata connectingMetadata = new() { Duration = TimeSpan.FromMilliseconds(250) }; + + HeifFrameMetadata metadata = HeifFrameMetadata.FromFormatConnectingFrameMetadata(connectingMetadata); + FormatConnectingFrameMetadata result = metadata.ToFormatConnectingFrameMetadata(); + + Assert.Equal(0.25, metadata.FrameDelay.ToDouble(), 10); + Assert.Equal(TimeSpan.FromMilliseconds(250), result.Duration); + Assert.Equal(FrameBlendMode.Source, result.BlendMode); + Assert.Equal(FrameDisposalMode.DoNotDispose, result.DisposalMode); + } + + [Fact] + public void ZeroDenominatorConvertsToUnspecifiedDuration() + { + HeifFrameMetadata metadata = new() { FrameDelay = new Rational(1, 0) }; + + FormatConnectingFrameMetadata result = metadata.ToFormatConnectingFrameMetadata(); + + Assert.Equal(TimeSpan.Zero, result.Duration); + } + + [Fact] + public void ImageFrameMetadataExtensionsReturnAndCloneHeifMetadata() + { + using Image image = new(1, 1); + HeifFrameMetadata metadata = image.Frames.RootFrame.Metadata.GetHeifMetadata(); + metadata.FrameDelay = new Rational(1, 5); + + HeifFrameMetadata clone = image.Frames.RootFrame.Metadata.CloneHeifMetadata(); + + Assert.NotSame(metadata, clone); + Assert.Equal(metadata.FrameDelay, clone.FrameDelay); + } +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/HeifLocationTests.cs b/tests/ImageSharp.Tests/Formats/Heif/HeifLocationTests.cs new file mode 100644 index 0000000000..b1877c7b5a --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/HeifLocationTests.cs @@ -0,0 +1,116 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif; + +[Trait("Format", "Heif")] +public class HeifLocationTests +{ + [Fact] + public void CheckSameLocationFromDifferentOrigin() + { + // Arrange + const int dataPosition = 50; + const int itemPosition = 100; + HeifLocation fromItem = new(HeifLocationOffsetOrigin.ItemOffset, 0, 10, 42); + HeifLocation fromFile = new(HeifLocationOffsetOrigin.FileOffset, 0, 110, 42); + HeifLocation fromData = new(HeifLocationOffsetOrigin.ItemDataOffset, 0, 60, 42); + + // Act + long itemActual = fromItem.GetStreamPosition(dataPosition, itemPosition); + long fileActual = fromFile.GetStreamPosition(dataPosition, itemPosition); + long dataActual = fromData.GetStreamPosition(dataPosition, itemPosition); + + // Assert + Assert.Equal(110, itemActual); + Assert.Equal(110, fileActual); + Assert.Equal(110, dataActual); + } + + [Fact] + public void CheckSameLocationFromDifferentOriginWithBaseOffset() + { + // Arrange + const int dataPosition = 50; + const int itemPosition = 100; + HeifLocation fromItem = new(HeifLocationOffsetOrigin.ItemOffset, 40, 10, 42); + HeifLocation fromFile = new(HeifLocationOffsetOrigin.FileOffset, 40, 110, 42); + HeifLocation fromData = new(HeifLocationOffsetOrigin.ItemDataOffset, 40, 60, 42); + + // Act + long itemActual = fromItem.GetStreamPosition(dataPosition, itemPosition); + long fileActual = fromFile.GetStreamPosition(dataPosition, itemPosition); + long dataActual = fromData.GetStreamPosition(dataPosition, itemPosition); + + // Assert + Assert.Equal(150, itemActual); + Assert.Equal(150, fileActual); + Assert.Equal(150, dataActual); + } + + [Fact] + public void CheckComparerOnSameLocation() + { + // Arrange + const int dataPosition = 50; + const int itemPosition = 100; + HeifLocation fromItem = new(HeifLocationOffsetOrigin.ItemOffset, 0, 50, 42); + HeifLocation fromFile = new(HeifLocationOffsetOrigin.FileOffset, 30, 120, 42); + HeifLocation fromData = new(HeifLocationOffsetOrigin.ItemDataOffset, 40, 60, 42); + HeifLocationComparer comparer = new(dataPosition, itemPosition); + + // Act + int item2Data = comparer.Compare(fromItem, fromData); + int item2File = comparer.Compare(fromItem, fromFile); + int file2Data = comparer.Compare(fromFile, fromData); + int data2File = comparer.Compare(fromData, fromFile); + + // Assert + Assert.Equal(0, item2Data); + Assert.Equal(0, item2File); + Assert.Equal(0, file2Data); + Assert.Equal(0, data2File); + } + + [Fact] + public void CheckComparerOnLowerLocation() + { + // Arrange + const int dataPosition = 50; + const int itemPosition = 100; + HeifLocation fromItem = new(HeifLocationOffsetOrigin.ItemOffset, 0, 50, 42); + HeifLocation fromFile = new(HeifLocationOffsetOrigin.FileOffset, 30, 150, 42); + HeifLocation fromData = new(HeifLocationOffsetOrigin.ItemDataOffset, 40, 80, 42); + HeifLocationComparer comparer = new(dataPosition, itemPosition); + + // Act + int item2Data = comparer.Compare(fromItem, fromData); + int item2File = comparer.Compare(fromItem, fromFile); + + // Assert + Assert.Equal(-1, item2Data); + Assert.Equal(-1, item2File); + } + + [Fact] + public void CheckComparerOnHigherLocation() + { + // Arrange + const int dataPosition = 50; + const int itemPosition = 100; + HeifLocation fromItem = new(HeifLocationOffsetOrigin.ItemOffset, 10, 50, 42); + HeifLocation fromFile = new(HeifLocationOffsetOrigin.FileOffset, 30, 120, 42); + HeifLocation fromData = new(HeifLocationOffsetOrigin.ItemDataOffset, 40, 60, 42); + HeifLocationComparer comparer = new(dataPosition, itemPosition); + + // Act + int item2Data = comparer.Compare(fromItem, fromData); + int item2File = comparer.Compare(fromItem, fromFile); + + // Assert + Assert.Equal(1, item2Data); + Assert.Equal(1, item2File); + } +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/HeifMetadataTests.cs b/tests/ImageSharp.Tests/Formats/Heif/HeifMetadataTests.cs new file mode 100644 index 0000000000..7c75dfea09 --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/HeifMetadataTests.cs @@ -0,0 +1,115 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats; +using SixLabors.ImageSharp.Formats.Heif; +using SixLabors.ImageSharp.PixelFormats; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif; + +[Trait("Format", "Heif")] +public class HeifMetadataTests +{ + [Fact] + public void DefaultsMatchLegacyEightBitHeif() + { + HeifMetadata metadata = new(); + + Assert.Equal(HeifCompressionMethod.LegacyJpeg, metadata.CompressionMethod); + Assert.Equal(HeifBitDepth.Bit8, metadata.BitDepth); + Assert.False(metadata.IsMonochrome); + Assert.False(metadata.HasAlpha); + Assert.Equal(1, metadata.RepeatCount); + Assert.True(metadata.AnimateRootFrame); + } + + [Fact] + public void DeepCloneCopiesImageDescription() + { + HeifMetadata metadata = new() + { + CompressionMethod = HeifCompressionMethod.Av1, + BitDepth = HeifBitDepth.Bit12, + IsMonochrome = true, + HasAlpha = true, + RepeatCount = 3, + AnimateRootFrame = false + }; + + HeifMetadata clone = metadata.DeepClone(); + + Assert.Equal(metadata.CompressionMethod, clone.CompressionMethod); + Assert.Equal(metadata.BitDepth, clone.BitDepth); + Assert.Equal(metadata.IsMonochrome, clone.IsMonochrome); + Assert.Equal(metadata.HasAlpha, clone.HasAlpha); + Assert.Equal(metadata.RepeatCount, clone.RepeatCount); + Assert.Equal(metadata.AnimateRootFrame, clone.AnimateRootFrame); + } + + [Fact] + public void SequenceStateRoundTripsFormatConnectingMetadata() + { + FormatConnectingMetadata connectingMetadata = new() + { + AnimateRootFrame = false, + PixelTypeInfo = new PixelTypeInfo(24), + RepeatCount = 7 + }; + + HeifMetadata metadata = HeifMetadata.FromFormatConnectingMetadata(connectingMetadata); + FormatConnectingMetadata result = metadata.ToFormatConnectingMetadata(); + + Assert.False(result.AnimateRootFrame); + Assert.Equal(7, result.RepeatCount); + } + + [Theory] + [InlineData(1, HeifBitDepth.Bit8)] + [InlineData(8, HeifBitDepth.Bit8)] + [InlineData(9, HeifBitDepth.Bit10)] + [InlineData(10, HeifBitDepth.Bit10)] + [InlineData(11, HeifBitDepth.Bit12)] + [InlineData(16, HeifBitDepth.Bit12)] + public void FromFormatConnectingMetadataSelectsSupportedBitDepth(int componentPrecision, HeifBitDepth expected) + { + FormatConnectingMetadata connectingMetadata = new() + { + PixelTypeInfo = new PixelTypeInfo(componentPrecision) + { + ComponentInfo = PixelComponentInfo.Create(1, componentPrecision, componentPrecision) + } + }; + + HeifMetadata metadata = HeifMetadata.FromFormatConnectingMetadata(connectingMetadata); + + Assert.Equal(expected, metadata.BitDepth); + } + + [Theory] + [InlineData(HeifBitDepth.Bit8, false, false, 24, 3)] + [InlineData(HeifBitDepth.Bit10, false, true, 40, 4)] + [InlineData(HeifBitDepth.Bit12, true, false, 12, 1)] + [InlineData(HeifBitDepth.Bit12, true, true, 24, 2)] + public void GetPixelTypeInfoUsesComponentBitDepth( + HeifBitDepth bitDepth, + bool isMonochrome, + bool hasAlpha, + int expectedBitsPerPixel, + int expectedComponentCount) + { + HeifMetadata metadata = new() + { + BitDepth = bitDepth, + IsMonochrome = isMonochrome, + HasAlpha = hasAlpha + }; + + PixelTypeInfo pixelTypeInfo = metadata.GetPixelTypeInfo(); + PixelComponentInfo componentInfo = pixelTypeInfo.ComponentInfo.Value; + + Assert.Equal(expectedBitsPerPixel, pixelTypeInfo.BitsPerPixel); + Assert.Equal(expectedComponentCount, componentInfo.ComponentCount); + Assert.Equal((int)bitDepth, componentInfo.GetMaximumComponentPrecision()); + } + +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/HeifPropertyParserTests.cs b/tests/ImageSharp.Tests/Formats/Heif/HeifPropertyParserTests.cs new file mode 100644 index 0000000000..50ca1d93a0 --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/HeifPropertyParserTests.cs @@ -0,0 +1,133 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif; +using SixLabors.ImageSharp.Formats.Heif.Av1; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif; + +/// +/// Verifies parsing and interpretation of HEIF image-item property payloads. +/// +[Trait("Format", "Heif")] +public class HeifPropertyParserTests +{ + /// + /// Verifies that every AV1 operating-point index representable by a sequence header is accepted. + /// + /// The zero-based operating-point index. + [Theory] + [InlineData(0)] + [InlineData(31)] + public void ParseAv1OperatingPointSelectorAcceptsSequenceHeaderRange(byte index) + { + Av1OperatingPointSelector selector = HeifPropertyParser.ParseAv1OperatingPointSelector([index]); + + Assert.Equal(index, selector.Index); + } + + /// + /// Verifies that an AV1 operating-point index outside the sequence-header range is rejected. + /// + [Fact] + public void ParseAv1OperatingPointSelectorRejectsOutOfRangeIndex() + => Assert.Throws(() => HeifPropertyParser.ParseAv1OperatingPointSelector([32])); + + /// + /// Verifies the four AV1 spatial-layer identifiers and the progressive final-layer selector. + /// + /// The most-significant selector byte. + /// The least-significant selector byte. + /// The parsed layer identifier. + [Theory] + [InlineData(0, 0, 0)] + [InlineData(0, 3, 3)] + [InlineData(255, 255, Av1LayerSelector.AllLayers)] + public void ParseAv1LayerSelectorAcceptsDefinedValues(byte highByte, byte lowByte, ushort expected) + { + Av1LayerSelector selector = HeifPropertyParser.ParseAv1LayerSelector([highByte, lowByte]); + + Assert.Equal(expected, selector.LayerId); + } + + /// + /// Verifies that an AV1 spatial-layer identifier wider than the OBU extension field is rejected. + /// + [Fact] + public void ParseAv1LayerSelectorRejectsOutOfRangeLayer() + => Assert.Throws(() => HeifPropertyParser.ParseAv1LayerSelector([0, 4])); + + /// + /// Verifies the compact 16-bit representation of the three explicit layered-image boundaries. + /// + [Fact] + public void ParseAv1LayeredImageIndexReadsSmallSizes() + { + Av1LayeredImageIndex index = HeifPropertyParser.ParseAv1LayeredImageIndex([0, 0, 55, 0, 17, 1, 2]); + + Assert.Equal(55U, index.FirstLayerSize); + Assert.Equal(17U, index.SecondLayerSize); + Assert.Equal(258U, index.ThirdLayerSize); + } + + /// + /// Verifies the 32-bit representation used when a layered-image boundary exceeds 16 bits. + /// + [Fact] + public void ParseAv1LayeredImageIndexReadsLargeSizes() + { + Av1LayeredImageIndex index = HeifPropertyParser.ParseAv1LayeredImageIndex( + [1, 0, 1, 0, 0, 0, 2, 0, 0, 0, 3, 0, 0]); + + Assert.Equal(65536U, index.FirstLayerSize); + Assert.Equal(131072U, index.SecondLayerSize); + Assert.Equal(196608U, index.ThirdLayerSize); + } + + /// + /// Verifies that reserved layered-image flag bits cannot alter the payload interpretation. + /// + [Fact] + public void ParseAv1LayeredImageIndexRejectsReservedBits() + => Assert.Throws(() => HeifPropertyParser.ParseAv1LayeredImageIndex([2, 0, 1, 0, 0, 0, 0])); + + /// + /// Verifies cumulative selection through the two-layer payload used by the libavif progressive fixture. + /// + /// The selected spatial layer. + /// The cumulative payload length through that layer. + [Theory] + [InlineData(0, 55)] + [InlineData(1, 72)] + [InlineData(Av1LayerSelector.AllLayers, 72)] + public void GetPayloadLengthSelectsCumulativeLayerBytes(ushort layerId, int expectedLength) + { + Av1LayeredImageIndex index = new(55, 0, 0); + Av1LayerSelector selector = new(layerId); + + Assert.Equal(expectedLength, index.GetPayloadLength(72, selector)); + } + + /// + /// Verifies that a selector cannot address a layer absent from the indexed item payload. + /// + [Fact] + public void GetPayloadLengthRejectsAbsentLayer() + { + Av1LayeredImageIndex index = new(55, 0, 0); + Av1LayerSelector selector = new(2); + + Assert.Throws(() => index.GetPayloadLength(72, selector)); + } + + /// + /// Verifies that every explicit layer boundary leaves bytes for the following implicit layer. + /// + [Fact] + public void GetPayloadLengthRejectsBoundaryAtItemEnd() + { + Av1LayeredImageIndex index = new(72, 0, 0); + + Assert.Throws(() => index.GetPayloadLength(72, null)); + } +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/HeifSequenceParserTests.cs b/tests/ImageSharp.Tests/Formats/Heif/HeifSequenceParserTests.cs new file mode 100644 index 0000000000..65e75cbb50 --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/HeifSequenceParserTests.cs @@ -0,0 +1,1839 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Buffers.Binary; +using System.Text; +using SixLabors.ImageSharp.Formats; +using SixLabors.ImageSharp.Formats.Heif; +using SixLabors.ImageSharp.Metadata; +using SixLabors.ImageSharp.Metadata.Profiles.Cicp; +using SixLabors.ImageSharp.PixelFormats; +using SixLabors.ImageSharp.Processing; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif; + +/// +/// Verifies HEIF image-sequence parsing with upstream libavif files and narrowly constructed invalid containers. +/// +[Trait("Format", "Heif")] +[ValidateDisposedMemoryAllocations] +public class HeifSequenceParserTests +{ + private const int BoxHeaderLength = 8; + private const int FileTypeBoxLength = 24; + private const int SyntheticFileLength = 2048; + private const uint SyntheticChunkOffset = 1024; + private const uint SyntheticMovieTimescale = 1000; + private const uint SyntheticTrackDuration = 600; + private const uint SyntheticMediaDuration = 200; + private const uint SyntheticSampleDuration = 100; + private const uint SyntheticSampleCount = 2; + private const int SyntheticWidth = 320; + private const int SyntheticHeight = 240; + private const int FirstSyntheticSampleLength = 10; + private const int SecondSyntheticSampleLength = 12; + private const uint ColorTrackId = 1; + private const uint AlphaTrackId = 2; + private const uint UnrelatedTrackId = 3; + private const uint MismatchedAlphaTimescale = 2000; + private const uint UnityFixed16Point16 = 1U << 16; + private const uint DoubleFixed16Point16 = 2U << 16; + private const uint UnityFixed2Point30 = 1U << 30; + private const ushort UnityFixed8Point8 = 1 << 8; + private const ushort PackedUndeterminedLanguage = 0x55C4; + private const uint SyntheticHorizontalResolution = 72U << 16; + private const ushort SyntheticPixelDepth = 24; + private const ushort SyntheticMaximumContentLightLevel = 1000; + private const ushort SyntheticMaximumFrameAverageLightLevel = 400; + private const uint SyntheticHorizontalPixelSpacing = 4; + private const uint SyntheticVerticalPixelSpacing = 3; + private const ushort TrackExifItemId = 1; + private const ushort TrackXmpItemId = 2; + private const int OrangeAv1ConfigurationOffset = 0xC7; + private const int OrangeAv1ConfigurationLength = 4; + private const int OrangeAv1SampleOffset = 0x10E; + private const int OrangeAv1SampleLength = 0x1D; + private const int TrackExifOffset = 1800; + private const int TrackXmpOffset = 1840; + private const int LibavifAnimationFrameCount = 5; + private const int LibavifAnimationSize = 150; + private const int LibavifKeyframeAnimationSize = 64; + private const int FinitePlayCount = 1; + private const int InfinitePlayCount = 0; + private const byte InvalidAv1SampleByte = 0x80; + + /// + /// Gets the minimal little-endian TIFF payload stored in the synthetic track-level Exif item. + /// + private static ReadOnlySpan TrackExifData => [0, 0, 0, 0, 0x49, 0x49, 0x2A, 0, 8, 0, 0, 0]; + + /// + /// Gets the minimal XMP packet stored in the synthetic track-level MIME item. + /// + private static ReadOnlySpan TrackXmpData => ""u8; + + /// + /// Gets the minimal AV1CodecConfigurationBox payload for profile zero, level zero, and an absent initial + /// presentation-delay field. The high marker and version bits encode marker one and configuration version one. + /// + private static ReadOnlySpan DefaultAv1Configuration => [0x81, 0, 0, 0]; + + /// + /// Verifies that genuine libavif animation files are identified from their image-sequence tracks rather than + /// from the fallback primary item. The audio variant must produce the same image description because non-image + /// tracks are deliberately outside the decoder's retained ISOBMFF surface. + /// + /// The libavif animation fixture to identify. + [Theory] + [InlineData(TestImages.Heif.Animated8Bit)] + [InlineData(TestImages.Heif.Animated8BitWithAudio)] + public void IdentifyReadsRealLibavifSequence(string imagePath) + { + TestFile file = TestFile.Create(imagePath); + + ImageInfo info = Image.Identify(file.Bytes); + HeifMetadata metadata = info.Metadata.GetHeifMetadata(); + + Assert.Equal(new Size(LibavifAnimationSize, LibavifAnimationSize), info.Size); + Assert.Equal(LibavifAnimationFrameCount, info.FrameCount); + Assert.Equal(HeifCompressionMethod.Av1, metadata.CompressionMethod); + Assert.Equal(HeifBitDepth.Bit8, metadata.BitDepth); + Assert.Equal(FinitePlayCount, metadata.RepeatCount); + Assert.False(metadata.HasAlpha); + } + + /// + /// Verifies that a genuine libavif animation carries its linked alpha track, infinite repetition, and metadata + /// items through the public sequence metadata boundary. + /// + [Fact] + public void IdentifyReadsRealLibavifSequenceWithAlphaAndMetadata() + { + TestFile file = TestFile.Create(TestImages.Heif.Animated8BitWithAlphaExifXmp); + + ImageInfo info = Image.Identify(file.Bytes); + HeifMetadata metadata = info.Metadata.GetHeifMetadata(); + + Assert.Equal(new Size(LibavifAnimationSize, LibavifAnimationSize), info.Size); + Assert.Equal(LibavifAnimationFrameCount, info.FrameCount); + Assert.Equal(HeifCompressionMethod.Av1, metadata.CompressionMethod); + Assert.Equal(HeifBitDepth.Bit8, metadata.BitDepth); + Assert.Equal(InfinitePlayCount, metadata.RepeatCount); + Assert.True(metadata.HasAlpha); + Assert.NotNull(info.Metadata.ExifProfile); + Assert.NotNull(info.Metadata.XmpProfile); + } + + /// + /// Verifies that a genuine 12-bit libavif sequence with inter-frame dependencies is identified as all five frames + /// instead of falling back to its primary image item. + /// + [Fact] + public void IdentifyReadsReal12BitLibavifSequence() + { + TestFile file = TestFile.Create(TestImages.Heif.Animated12BitWithKeyframes); + + ImageInfo info = Image.Identify(file.Bytes); + HeifMetadata metadata = info.Metadata.GetHeifMetadata(); + + Assert.Equal(new Size(LibavifKeyframeAnimationSize, LibavifKeyframeAnimationSize), info.Size); + Assert.Equal(LibavifAnimationFrameCount, info.FrameCount); + Assert.Equal(HeifCompressionMethod.Av1, metadata.CompressionMethod); + Assert.Equal(HeifBitDepth.Bit12, metadata.BitDepth); + } + + /// + /// Verifies that identification transfers bounded sequence, track property, metadata-item, and frame-timing + /// information from the parser into the public image metadata model. + /// + [Fact] + public void IdentifyReturnsBoundedSequenceAndFrameMetadata() + { + byte[] data = CreateSequenceContainer(trackProperties: true, trackMetadata: true); + + ImageInfo info = Image.Identify(data); + HeifMetadata metadata = info.Metadata.GetHeifMetadata(); + + Assert.Equal(new Size(SyntheticHeight, SyntheticWidth), info.Size); + Assert.Equal(2, info.FrameCount); + Assert.Equal(HeifCompressionMethod.Av1, metadata.CompressionMethod); + Assert.Equal(HeifBitDepth.Bit8, metadata.BitDepth); + Assert.Equal(3, metadata.RepeatCount); + Assert.False(metadata.HasAlpha); + Assert.NotNull(info.Metadata.CicpProfile); + Assert.NotNull(info.Metadata.ExifProfile); + Assert.NotNull(info.Metadata.XmpProfile); + Assert.NotNull(metadata.ContentLightLevel); + Assert.Equal(4D, info.Metadata.HorizontalResolution); + Assert.Equal(3D, info.Metadata.VerticalResolution); + Assert.All( + info.FrameMetadataCollection, + frame => Assert.Equal(new Rational(SyntheticSampleDuration, SyntheticMovieTimescale), frame.GetHeifMetadata().FrameDelay)); + } + + /// + /// Verifies that omits ancillary sequence profiles and item metadata + /// without discarding structural codec, repetition, or frame-timing information required to describe the image. + /// + [Fact] + public void IdentifySkipsAncillarySequenceMetadataWithoutDroppingImageMetadata() + { + byte[] data = CreateSequenceContainer(trackProperties: true, trackMetadata: true); + using MemoryStream stream = new(data, false); + DecoderOptions options = new() { SkipMetadata = true }; + + ImageInfo info = Image.Identify(options, stream); + HeifMetadata metadata = info.Metadata.GetHeifMetadata(); + + Assert.Equal(HeifCompressionMethod.Av1, metadata.CompressionMethod); + Assert.Equal(HeifBitDepth.Bit8, metadata.BitDepth); + Assert.Equal(3, metadata.RepeatCount); + Assert.Null(info.Metadata.CicpProfile); + Assert.Null(info.Metadata.IccProfile); + Assert.Null(info.Metadata.ExifProfile); + Assert.Null(info.Metadata.XmpProfile); + Assert.Null(metadata.ContentLightLevel); + Assert.All( + info.FrameMetadataCollection, + frame => Assert.Equal(new Rational(SyntheticSampleDuration, SyntheticMovieTimescale), frame.GetHeifMetadata().FrameDelay)); + } + + /// + /// Verifies that two independently addressable AV1 samples become two complete ImageSharp frames with the + /// expected pixels and per-frame duration. + /// + [Fact] + public void DecodeAdoptsIndependentAv1SamplesAsImageFrames() + { + byte[] source = TestFile.Create(TestImages.Heif.Orange4x4).Bytes; + byte[] data = CreateDecodableAv1SequenceContainer( + source.AsSpan(OrangeAv1SampleOffset, OrangeAv1SampleLength), + source.AsSpan(OrangeAv1ConfigurationOffset, OrangeAv1ConfigurationLength)); + + using Image expected = Image.Load(source); + using Image actual = Image.Load(data); + + Assert.Equal(new Size(4, 4), actual.Size); + Assert.Equal(2, actual.Frames.Count); + foreach (ImageFrame frame in actual.Frames) + { + Assert.Equal( + new Rational(SyntheticSampleDuration, SyntheticMovieTimescale), + frame.Metadata.GetHeifMetadata().FrameDelay); + + for (int y = 0; y < frame.Height; y++) + { + Assert.True( + frame.PixelBuffer.DangerousGetRowSpan(y) + .SequenceEqual(expected.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(y))); + } + } + } + + [Fact] + public void DecodeAppliesTrackPresentationPropertiesToEveryFrame() + { + byte[] source = TestFile.Create(TestImages.Heif.Orange4x4).Bytes; + byte[] data = CreateDecodableAv1SequenceContainer( + source.AsSpan(OrangeAv1SampleOffset, OrangeAv1SampleLength), + source.AsSpan(OrangeAv1ConfigurationOffset, OrangeAv1ConfigurationLength), + trackProperties: true); + + int cleanApertureTypeOffset = data.AsSpan().IndexOf("clap"u8); + Assert.True(cleanApertureTypeOffset >= 0); + + // Narrow the synthetic full-frame aperture to its centered 2x2 region without changing the coded AV1 sample. + BinaryPrimitives.WriteUInt32BigEndian(data.AsSpan(cleanApertureTypeOffset + 4), 2); + BinaryPrimitives.WriteUInt32BigEndian(data.AsSpan(cleanApertureTypeOffset + 12), 2); + + using Image expected = Image.Load(source); + expected.Mutate(context => context + .Crop(new Rectangle(1, 1, 2, 2)) + .Rotate(RotateMode.Rotate270) + .Flip(FlipMode.Horizontal)); + + using Image actual = Image.Load(data); + + Assert.Equal(expected.Size, actual.Size); + Assert.Equal(2, actual.Frames.Count); + Assert.Equal(4D, actual.Metadata.HorizontalResolution); + Assert.Equal(3D, actual.Metadata.VerticalResolution); + Assert.Equal(PixelResolutionUnit.AspectRatio, actual.Metadata.ResolutionUnits); + Assert.NotNull(actual.Metadata.CicpProfile); + for (int frameIndex = 0; frameIndex < actual.Frames.Count; frameIndex++) + { + Assert.NotNull(actual.Frames[frameIndex].Metadata.CicpProfile); + for (int y = 0; y < actual.Height; y++) + { + Assert.True( + expected.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(y) + .SequenceEqual(actual.Frames[frameIndex].PixelBuffer.DangerousGetRowSpan(y))); + } + } + } + + /// + /// Verifies that strict and ancillary-tolerant decoding both reject corrupt coded image data because neither + /// integrity mode permits recovery from errors in a retained AV1 sample. + /// + /// The segment-integrity policy applied at the decoder boundary. + [Theory] + [InlineData(SegmentIntegrityHandling.Strict)] + [InlineData(SegmentIntegrityHandling.IgnoreAncillary)] + public void DecodeRejectsInvalidAv1SampleUnlessImageDataErrorsAreIgnored(SegmentIntegrityHandling handling) + { + byte[] source = TestFile.Create(TestImages.Heif.Orange4x4).Bytes; + byte[] data = CreateDecodableAv1SequenceContainer( + source.AsSpan(OrangeAv1SampleOffset, OrangeAv1SampleLength), + [InvalidAv1SampleByte], + source.AsSpan(OrangeAv1ConfigurationOffset, OrangeAv1ConfigurationLength), + false); + + DecoderOptions options = new() { SegmentIntegrityHandling = handling }; + + Assert.Throws(() => Image.Load(options, data)); + } + + /// + /// Verifies that image-data tolerance drops an invalid non-root AV1 sample while preserving the valid frame. + /// + [Fact] + public void DecodeSkipsInvalidAv1SampleWhenImageDataErrorsAreIgnored() + { + byte[] source = TestFile.Create(TestImages.Heif.Orange4x4).Bytes; + byte[] data = CreateDecodableAv1SequenceContainer( + source.AsSpan(OrangeAv1SampleOffset, OrangeAv1SampleLength), + [InvalidAv1SampleByte], + source.AsSpan(OrangeAv1ConfigurationOffset, OrangeAv1ConfigurationLength), + false); + + DecoderOptions options = new() { SegmentIntegrityHandling = SegmentIntegrityHandling.IgnoreImageData }; + + using Image image = Image.Load(options, data); + + Assert.Equal(new Size(4, 4), image.Size); + Assert.Single(image.Frames); + } + + /// + /// Verifies that an AV1 alpha track whose sequence header is not monochrome is rejected at the codec boundary. + /// + [Fact] + public void DecodeRejectsNonMonochromeAv1AlphaSamples() + { + byte[] source = TestFile.Create(TestImages.Heif.Orange4x4).Bytes; + byte[] data = CreateAv1SequenceWithNonMonochromeAlphaContainer( + source.AsSpan(OrangeAv1SampleOffset, OrangeAv1SampleLength), + source.AsSpan(OrangeAv1ConfigurationOffset, OrangeAv1ConfigurationLength)); + + Assert.Throws(() => + { + using Image image = Image.Load(data); + }); + } + + /// + /// Verifies that a genuine libavif alpha sequence composes every retained frame from the linked monochrome + /// auxiliary track instead of returning any color frame as opaque. + /// + [Fact] + public void DecodeComposesEveryRealLibavifAlphaSequenceFrame() + { + TestFile file = TestFile.Create(TestImages.Heif.Animated8BitWithAlphaExifXmp); + + using Image image = Image.Load(file.Bytes); + + Assert.Equal(LibavifAnimationFrameCount, image.Frames.Count); + Assert.True(image.Metadata.GetHeifMetadata().HasAlpha); + Assert.NotNull(image.Metadata.ExifProfile); + Assert.NotNull(image.Metadata.XmpProfile); + foreach (ImageFrame frame in image.Frames) + { + bool hasNonOpaqueSample = false; + for (int y = 0; y < frame.Height && !hasNonOpaqueSample; y++) + { + foreach (Rgba32 pixel in frame.PixelBuffer.DangerousGetRowSpan(y)) + { + if (pixel.A != byte.MaxValue) + { + hasNonOpaqueSample = true; + break; + } + } + } + + Assert.True(hasNonOpaqueSample); + Assert.True(frame.Metadata.GetHeifMetadata().FrameDelay.Numerator > 0); + Assert.True(frame.Metadata.GetHeifMetadata().FrameDelay.Denominator > 0); + } + } + + /// + /// Verifies the complete libavif-shaped sample-table mapping, including timing, chunk offsets, sync status, + /// coding constraints, and the normalized sequence play count. + /// + [Fact] + public void ParseResolvesLibavifShapedSampleTable() + { + byte[] data = CreateSequenceFile(SyntheticChunkOffset); + using MemoryStream stream = new(data, false); + HeifSequenceParser parser = CreateParser(SyntheticSampleCount); + stream.Position = BoxHeaderLength; + + HeifSequence sequence = parser.Parse(stream, GetMoviePayloadLength(data)); + + Assert.Equal(SyntheticMovieTimescale, sequence.MovieTimescale); + Assert.Null(sequence.AlphaTrack); + Assert.Equal(ColorTrackId, sequence.ColorTrack.Id); + Assert.Equal(SyntheticWidth, sequence.ColorTrack.Width); + Assert.Equal(SyntheticHeight, sequence.ColorTrack.Height); + Assert.Equal(Heif4CharCode.Av01, sequence.ColorTrack.CodecType); + Assert.NotNull(sequence.ColorTrack.Av1CodecConfiguration); + Assert.Equal(SyntheticSampleCount, sequence.ColorTrack.TotalSampleCount); + Assert.Equal(3, sequence.ColorTrack.RepeatCount); + Assert.False(sequence.ColorTrack.AllReferencePicturesIntra); + Assert.True(sequence.ColorTrack.IntraPicturePredictionUsed); + Assert.Equal(15, sequence.ColorTrack.MaximumReferencesPerPicture); + Assert.Collection( + sequence.ColorTrack.Samples, + sample => + { + Assert.Equal(SyntheticChunkOffset, sample.Offset); + Assert.Equal(FirstSyntheticSampleLength, sample.Length); + Assert.Equal(SyntheticSampleDuration, sample.Duration); + Assert.True(sample.IsSync); + }, + sample => + { + Assert.Equal((long)SyntheticChunkOffset + FirstSyntheticSampleLength, sample.Offset); + Assert.Equal(SecondSyntheticSampleLength, sample.Length); + Assert.Equal(SyntheticSampleDuration, sample.Duration); + Assert.False(sample.IsSync); + }); + } + + /// + /// Verifies that the parser does not select a picture track whose TrackHeaderBox clears the enabled flag. + /// + [Fact] + public void ParseRejectsDisabledPictureTrack() + { + byte[] data = CreateSequenceFile(SyntheticChunkOffset, trackEnabled: false); + using MemoryStream stream = new(data, false); + HeifSequenceParser parser = CreateParser(SyntheticSampleCount); + stream.Position = BoxHeaderLength; + + Assert.Throws(() => parser.Parse(stream, GetMoviePayloadLength(data))); + } + + /// + /// Verifies that the parser retains only the configured maximum number of frames while preserving the track's + /// declared total sample count. + /// + [Fact] + public void ParseRetainsOnlyConfiguredFrameCount() + { + const uint retainedFrameLimit = 1; + + byte[] data = CreateSequenceFile(SyntheticChunkOffset); + using MemoryStream stream = new(data, false); + HeifSequenceParser parser = CreateParser(retainedFrameLimit); + stream.Position = BoxHeaderLength; + + HeifSequence sequence = parser.Parse(stream, GetMoviePayloadLength(data)); + + Assert.Equal(SyntheticSampleCount, sequence.ColorTrack.TotalSampleCount); + HeifSequenceSample sample = Assert.Single(sequence.ColorTrack.Samples); + Assert.Equal(SyntheticChunkOffset, sample.Offset); + Assert.Equal(FirstSyntheticSampleLength, sample.Length); + Assert.Equal(SyntheticSampleDuration, sample.Duration); + } + + /// + /// Verifies that a retained sample whose declared byte range extends beyond the source stream is rejected. + /// + [Fact] + public void ParseRejectsRetainedSampleBeyondFile() + { + uint truncatedChunkOffset = SyntheticFileLength - BoxHeaderLength; + byte[] data = CreateSequenceFile(truncatedChunkOffset); + using MemoryStream stream = new(data, false); + HeifSequenceParser parser = CreateParser(SyntheticSampleCount); + stream.Position = BoxHeaderLength; + + Assert.Throws(() => parser.Parse(stream, GetMoviePayloadLength(data))); + } + + /// + /// Verifies that presentation transforms at either the movie or track level are rejected until the decoder can + /// apply those matrices to the emitted raster. + /// + /// Whether the movie header contains a non-unity matrix. + /// Whether the track header contains a non-unity matrix. + [Theory] + [InlineData(true, false)] + [InlineData(false, true)] + public void ParseRejectsNonIdentityMoviePresentationMatrix(bool nonIdentityMovieMatrix, bool nonIdentityTrackMatrix) + { + byte[] data = CreateSequenceFile( + SyntheticChunkOffset, + nonIdentityMovieMatrix: nonIdentityMovieMatrix, + nonIdentityTrackMatrix: nonIdentityTrackMatrix); + + using MemoryStream stream = new(data, false); + HeifSequenceParser parser = CreateParser(SyntheticSampleCount); + stream.Position = BoxHeaderLength; + + Assert.Throws(() => parser.Parse(stream, GetMoviePayloadLength(data))); + } + + /// + /// Verifies that auxiliary-track and premultiplication references are resolved by track identifier and remain + /// valid when matching presentation properties are present on either track. + /// + /// Whether the color sample entry carries presentation properties. + /// Whether the alpha sample entry carries matching presentation properties. + [Theory] + [InlineData(false, false)] + [InlineData(true, false)] + [InlineData(true, true)] + public void ParseMatchesAlphaTrackAndPremultiplicationByTrackId(bool colorTransforms, bool alphaTransforms) + { + byte[] data = CreateSequenceFileWithAlpha( + SyntheticChunkOffset, + SyntheticMovieTimescale, + AlphaTrackId, + colorTransforms, + alphaTransforms); + + using MemoryStream stream = new(data, false); + HeifSequenceParser parser = CreateParser(SyntheticSampleCount); + stream.Position = BoxHeaderLength; + + HeifSequence sequence = parser.Parse(stream, GetMoviePayloadLength(data)); + + Assert.NotNull(sequence.AlphaTrack); + Assert.Equal(AlphaTrackId, sequence.AlphaTrack.Id); + Assert.True(sequence.AlphaTrack.IsAlpha); + Assert.True(sequence.ColorTrack.IsPremultiplied); + } + + /// + /// Verifies that an alpha track with a different media timescale is rejected under policies that do not permit + /// recovery from image-data inconsistencies. + /// + /// The segment-integrity policy applied at the parser boundary. + [Theory] + [InlineData(SegmentIntegrityHandling.Strict)] + [InlineData(SegmentIntegrityHandling.IgnoreAncillary)] + public void ParseRejectsAlphaTrackWithDifferentDecodeTiming(SegmentIntegrityHandling handling) + { + byte[] data = CreateSequenceFileWithAlpha(SyntheticChunkOffset, MismatchedAlphaTimescale, 0); + using MemoryStream stream = new(data, false); + HeifSequenceParser parser = CreateParser(SyntheticSampleCount, segmentIntegrityHandling: handling); + stream.Position = BoxHeaderLength; + + Assert.Throws(() => parser.Parse(stream, GetMoviePayloadLength(data))); + } + + /// + /// Verifies that image-data tolerance drops a timing-incompatible alpha track and clears the color track's + /// premultiplication state. + /// + [Fact] + public void ParseDropsAlphaTrackWithDifferentDecodeTimingWhenImageDataErrorsAreIgnored() + { + byte[] data = CreateSequenceFileWithAlpha(SyntheticChunkOffset, MismatchedAlphaTimescale, 0); + using MemoryStream stream = new(data, false); + HeifSequenceParser parser = CreateParser( + SyntheticSampleCount, + segmentIntegrityHandling: SegmentIntegrityHandling.IgnoreImageData); + + stream.Position = BoxHeaderLength; + + HeifSequence sequence = parser.Parse(stream, GetMoviePayloadLength(data)); + + Assert.Null(sequence.AlphaTrack); + Assert.False(sequence.ColorTrack.IsPremultiplied); + } + + /// + /// Verifies that a premultiplication reference cannot name a track other than the selected linked alpha track. + /// + [Fact] + public void ParseRejectsPremultiplicationReferenceToUnrelatedTrack() + { + byte[] data = CreateSequenceFileWithAlpha(SyntheticChunkOffset, SyntheticMovieTimescale, UnrelatedTrackId); + using MemoryStream stream = new(data, false); + HeifSequenceParser parser = CreateParser(SyntheticSampleCount); + stream.Position = BoxHeaderLength; + + Assert.Throws(() => parser.Parse(stream, GetMoviePayloadLength(data))); + } + + /// + /// Verifies that linked color and alpha tracks with different presentation properties are rejected because they + /// cannot be composed frame-for-frame into one image sequence. + /// + [Fact] + public void ParseRejectsMismatchedAlphaPresentationTransforms() + { + byte[] data = CreateSequenceFileWithAlpha(SyntheticChunkOffset, SyntheticMovieTimescale, 0, false, true); + using MemoryStream stream = new(data, false); + HeifSequenceParser parser = CreateParser(SyntheticSampleCount); + stream.Position = BoxHeaderLength; + + Assert.Throws(() => parser.Parse(stream, GetMoviePayloadLength(data))); + } + + /// + /// Verifies that AV1 image-sequence tracks reject prohibited composition timing boxes. + /// + [Fact] + public void ParseRejectsCompositionOffsetsForAv1() + { + byte[] data = CreateSequenceFile(SyntheticChunkOffset, compositionOffsets: true); + using MemoryStream stream = new(data, false); + HeifSequenceParser parser = CreateParser(SyntheticSampleCount); + stream.Position = BoxHeaderLength; + + Assert.Throws(() => parser.Parse(stream, GetMoviePayloadLength(data))); + } + + /// + /// Verifies that AV1 direct-reference sample groups resolve file-defined sample identifiers into compact + /// zero-based indices retained by each dependent sample. + /// + [Fact] + public void ParseResolvesDirectReferenceSamples() + { + byte[] data = CreateSequenceFile(SyntheticChunkOffset, directReferences: true); + using MemoryStream stream = new(data, false); + HeifSequenceParser parser = CreateParser(SyntheticSampleCount); + stream.Position = BoxHeaderLength; + + HeifSequence sequence = parser.Parse(stream, GetMoviePayloadLength(data)); + + Assert.Equal(new[] { 0 }, sequence.ColorTrack.DirectReferenceSampleIndices); + Assert.Equal(1U, sequence.ColorTrack.Samples[0].SampleId); + Assert.Equal(0, sequence.ColorTrack.Samples[0].DirectReferenceCount); + Assert.Equal(0U, sequence.ColorTrack.Samples[1].SampleId); + Assert.Equal(0, sequence.ColorTrack.Samples[1].DirectReferenceOffset); + Assert.Equal(1, sequence.ColorTrack.Samples[1].DirectReferenceCount); + } + + /// + /// Verifies that a direct-reference group cannot name a sample identifier absent from the retained description + /// table. + /// + [Fact] + public void ParseRejectsUnknownDirectReferenceSampleId() + { + byte[] data = CreateSequenceFile( + SyntheticChunkOffset, + directReferences: true, + directReferenceSampleId: AlphaTrackId); + + using MemoryStream stream = new(data, false); + HeifSequenceParser parser = CreateParser(SyntheticSampleCount); + stream.Position = BoxHeaderLength; + + Assert.Throws(() => parser.Parse(stream, GetMoviePayloadLength(data))); + } + + /// + /// Verifies that image properties nested in the visual sample entry are retained with their specified color, + /// geometry, orientation, light-level, and viewing-environment values. + /// + [Fact] + public void ParseRetainsTrackImageProperties() + { + byte[] data = CreateSequenceFile(SyntheticChunkOffset, trackProperties: true); + using MemoryStream stream = new(data, false); + HeifSequenceParser parser = CreateParser(SyntheticSampleCount); + stream.Position = BoxHeaderLength; + + HeifSequenceTrack track = parser.Parse(stream, GetMoviePayloadLength(data)).ColorTrack; + + Assert.NotNull(track.CicpProfile); + HeifPixelAspectRatio pixelAspectRatio = Assert.IsType(track.PixelAspectRatio); + Assert.Equal(SyntheticHorizontalPixelSpacing, pixelAspectRatio.HorizontalSpacing); + Assert.Equal(SyntheticVerticalPixelSpacing, pixelAspectRatio.VerticalSpacing); + Size codedSize = new(SyntheticWidth, SyntheticHeight); + + Assert.True(track.CleanAperture.HasValue); + HeifCleanAperture cleanAperture = track.CleanAperture.GetValueOrDefault(); + Assert.Equal(new Rectangle(Point.Empty, codedSize), cleanAperture.ToRectangle(codedSize)); + Assert.Equal((byte)1, track.RotationAngle); + Assert.Equal((byte)1, track.MirrorAxis); + Assert.True(track.ContentLightLevel.HasValue); + HeifContentLightLevel contentLightLevel = track.ContentLightLevel.GetValueOrDefault(); + Assert.Equal(SyntheticMaximumContentLightLevel, contentLightLevel.MaximumContentLightLevel); + Assert.NotNull(track.MasteringDisplayColorVolume); + Assert.NotNull(track.ContentColorVolume); + Assert.NotNull(track.AmbientViewingEnvironment); + Assert.NotNull(track.ReferenceViewingEnvironment); + Assert.NotNull(track.NominalDiffuseWhite); + } + + /// + /// Verifies that track-level Exif and XMP items are bounded and retained whether their extents address the file + /// or the metadata box's item-data payload. + /// + /// Whether metadata extents use construction method one and address the item-data box. + [Theory] + [InlineData(false)] + [InlineData(true)] + public void ParseRetainsBoundedTrackMetadata(bool useItemData) + { + byte[] data = CreateSequenceFile(SyntheticChunkOffset, trackMetadata: true, metadataInItemData: useItemData); + using MemoryStream stream = new(data, false); + HeifSequenceParser parser = CreateParser(SyntheticSampleCount); + stream.Position = BoxHeaderLength; + + HeifSequenceTrack track = parser.Parse(stream, GetMoviePayloadLength(data)).ColorTrack; + + Assert.NotNull(track.Metadata); + Assert.Equal(TrackExifData.ToArray(), track.Metadata.ExifData); + Assert.Equal(TrackXmpData.ToArray(), track.Metadata.XmpData); + } + + /// + /// Verifies that metadata skipping avoids both validation and retention of malformed optional track metadata + /// while leaving image samples available. + /// + [Fact] + public void ParseDoesNotValidateOrRetainSkippedTrackMetadata() + { + byte[] data = CreateSequenceFile(SyntheticChunkOffset, trackMetadata: true, invalidTrackMetadata: true); + using MemoryStream stream = new(data, false); + HeifSequenceParser parser = CreateParser(SyntheticSampleCount, skipMetadata: true); + stream.Position = BoxHeaderLength; + + HeifSequenceTrack track = parser.Parse(stream, GetMoviePayloadLength(data)).ColorTrack; + + Assert.Null(track.Metadata); + Assert.Equal(SyntheticSampleCount, (uint)track.Samples.Length); + } + + /// + /// Verifies that malformed track metadata is fatal under strict validation but is omitted under ancillary-error + /// tolerance without affecting the retained image samples. + /// + [Fact] + public void ParseUsesAncillaryIntegrityPolicyForTrackMetadata() + { + byte[] data = CreateSequenceFile(SyntheticChunkOffset, trackMetadata: true, invalidTrackMetadata: true); + using MemoryStream strictStream = new(data, false); + HeifSequenceParser strictParser = CreateParser(SyntheticSampleCount); + strictStream.Position = BoxHeaderLength; + + Assert.Throws(() => strictParser.Parse(strictStream, GetMoviePayloadLength(data))); + + using MemoryStream tolerantStream = new(data, false); + HeifSequenceParser tolerantParser = CreateParser( + SyntheticSampleCount, + segmentIntegrityHandling: SegmentIntegrityHandling.IgnoreAncillary); + + tolerantStream.Position = BoxHeaderLength; + + HeifSequenceTrack track = tolerantParser.Parse(tolerantStream, GetMoviePayloadLength(data)).ColorTrack; + + Assert.Null(track.Metadata); + Assert.Equal(SyntheticSampleCount, (uint)track.Samples.Length); + } + + /// + /// Verifies that malformed presentation properties remain image-data errors under strict and ancillary-tolerant + /// policies because they affect the rendered image geometry. + /// + /// The segment-integrity policy applied at the parser boundary. + [Theory] + [InlineData(SegmentIntegrityHandling.Strict)] + [InlineData(SegmentIntegrityHandling.IgnoreAncillary)] + public void ParseRejectsInvalidPresentationPropertyUnlessImageDataErrorsAreIgnored(SegmentIntegrityHandling handling) + { + byte[] data = CreateSequenceFile(SyntheticChunkOffset, trackProperties: true, invalidRotation: true); + using MemoryStream stream = new(data, false); + HeifSequenceParser parser = CreateParser(SyntheticSampleCount, segmentIntegrityHandling: handling); + stream.Position = BoxHeaderLength; + + Assert.Throws(() => parser.Parse(stream, GetMoviePayloadLength(data))); + } + + /// + /// Verifies that image-data tolerance omits only the malformed presentation property while retaining independent + /// valid properties and all image samples. + /// + [Fact] + public void ParseOmitsInvalidPresentationPropertyWhenImageDataErrorsAreIgnored() + { + byte[] data = CreateSequenceFile(SyntheticChunkOffset, trackProperties: true, invalidRotation: true); + using MemoryStream stream = new(data, false); + HeifSequenceParser parser = CreateParser( + SyntheticSampleCount, + segmentIntegrityHandling: SegmentIntegrityHandling.IgnoreImageData); + + stream.Position = BoxHeaderLength; + + HeifSequenceTrack track = parser.Parse(stream, GetMoviePayloadLength(data)).ColorTrack; + + Assert.Null(track.RotationAngle); + Assert.NotNull(track.PixelAspectRatio); + Assert.Equal(SyntheticSampleCount, (uint)track.Samples.Length); + } + + /// + /// Builds a bounded movie box containing one configurable image-sequence track and leaves sample payload space + /// outside the movie so tests can independently control sample offsets and source-length validation. + /// + /// The absolute file offset of the track's single sample chunk. + /// Whether to write prohibited composition timing boxes. + /// Whether to write AV1 direct-reference sample groups. + /// The sample identifier named by the dependent sample. + /// Whether to write image presentation properties in the sample entry. + /// Whether to write track-level Exif and XMP metadata items. + /// Whether metadata extents address an item-data box instead of file offsets. + /// Whether the metadata handler is intentionally invalid. + /// Whether the rotation property contains reserved high bits. + /// The displayed and coded sample width in pixels. + /// The displayed and coded sample height in pixels. + /// The AV1CodecConfigurationBox payload, or the valid default payload. + /// The first sample size, or the synthetic default size. + /// The second sample size, or the first/default sample size. + /// Whether both samples are declared as sync samples. + /// The alpha track identifier named by the premultiplication reference. + /// Whether the movie header matrix contains horizontal scaling. + /// Whether the track header matrix contains horizontal scaling. + /// Whether the picture track is eligible for sequence presentation. + /// The fixed-length synthetic file containing the serialized movie box. + private static byte[] CreateSequenceFile( + uint chunkOffset, + bool compositionOffsets = false, + bool directReferences = false, + uint directReferenceSampleId = ColorTrackId, + bool trackProperties = false, + bool trackMetadata = false, + bool metadataInItemData = false, + bool invalidTrackMetadata = false, + bool invalidRotation = false, + int width = SyntheticWidth, + int height = SyntheticHeight, + byte[] av1Configuration = null, + int? sampleSize = null, + int? secondSampleSize = null, + bool allSamplesSync = false, + uint premultipliedByTrackId = 0, + bool nonIdentityMovieMatrix = false, + bool nonIdentityTrackMatrix = false, + bool trackEnabled = true) + { + using MemoryStream stream = new(); + using BinaryWriter writer = new(stream, Encoding.UTF8, true); + long movie = BeginBox(writer, Heif4CharCode.Moov); + + long movieHeader = BeginBox(writer, Heif4CharCode.Mvhd); + + // ISO/IEC 14496-12 Section 8.2.2 orders the version-zero fields as creation time, modification time, + // timescale, duration, preferred 16.16 rate, preferred 8.8 volume, reserved words, matrix, predefined words, + // and the next available track identifier. + WriteFullBoxHeader(writer, 0, 0); + WriteUInt32(writer, 0); + WriteUInt32(writer, 0); + WriteUInt32(writer, SyntheticMovieTimescale); + WriteUInt32(writer, SyntheticTrackDuration); + WriteUInt32(writer, UnityFixed16Point16); + WriteUInt16(writer, UnityFixed8Point8); + WriteUInt16(writer, 0); + WriteZeros(writer, 2 * sizeof(uint)); + WritePresentationMatrix(writer, nonIdentityMovieMatrix); + WriteZeros(writer, 6 * sizeof(uint)); + WriteUInt32(writer, UnrelatedTrackId); + EndBox(writer, movieHeader); + + long track = BeginBox(writer, Heif4CharCode.Trak); + WriteTrackHeader(writer, width, height, ColorTrackId, nonIdentityTrackMatrix, trackEnabled); + if (premultipliedByTrackId != 0) + { + WriteTrackReference(writer, Heif4CharCode.Prem, premultipliedByTrackId); + } + + WriteEditList(writer); + if (trackMetadata) + { + WriteTrackMetadata(writer, metadataInItemData, invalidTrackMetadata); + } + + long media = BeginBox(writer, Heif4CharCode.Mdia); + WriteMediaHeader(writer); + WriteHandler(writer, Heif4CharCode.Pict); + + long mediaInformation = BeginBox(writer, Heif4CharCode.Minf); + WriteDataInformation(writer); + WriteSampleTable( + writer, + chunkOffset, + compositionOffsets, + directReferences, + directReferenceSampleId, + trackProperties, + invalidRotation, + width, + height, + av1Configuration, + sampleSize, + secondSampleSize, + allSamplesSync); + + EndBox(writer, mediaInformation); + EndBox(writer, media); + EndBox(writer, track); + EndBox(writer, movie); + + byte[] movieBytes = stream.ToArray(); + + // The fixed outer length leaves deterministic space for file-addressed metadata and sample extents while + // allowing individual tests to place an extent deliberately beyond the source boundary. + byte[] file = new byte[SyntheticFileLength]; + + movieBytes.CopyTo(file, 0); + if (trackMetadata && !metadataInItemData) + { + TrackExifData.CopyTo(file.AsSpan(TrackExifOffset)); + TrackXmpData.CopyTo(file.AsSpan(TrackXmpOffset)); + } + + return file; + } + + /// + /// Appends an auxiliary alpha track to a synthetic color-track movie and links it by track identifier. + /// + /// The absolute file offset of the color sample chunk. + /// The alpha track's media time scale in units per second. + /// The track identifier named by the color premultiplication reference. + /// Whether the color sample entry carries image presentation properties. + /// Whether the alpha sample entry carries image presentation properties. + /// The absolute alpha sample-chunk offset, or the color chunk offset when omitted. + /// The displayed and coded width of both tracks in pixels. + /// The displayed and coded height of both tracks in pixels. + /// The AV1CodecConfigurationBox payload shared by the tracks. + /// The size of each sample in both tracks. + /// Whether all color and alpha samples are sync samples. + /// The fixed-length synthetic file containing the color and alpha tracks. + private static byte[] CreateSequenceFileWithAlpha( + uint chunkOffset, + uint alphaTimescale, + uint premultipliedByTrackId, + bool colorTransforms = false, + bool alphaTransforms = false, + uint? alphaChunkOffset = null, + int width = SyntheticWidth, + int height = SyntheticHeight, + byte[] av1Configuration = null, + int? sampleSize = null, + bool allSamplesSync = false) + { + byte[] colorFile = CreateSequenceFile( + chunkOffset, + trackProperties: colorTransforms, + width: width, + height: height, + av1Configuration: av1Configuration, + sampleSize: sampleSize, + allSamplesSync: allSamplesSync, + premultipliedByTrackId: premultipliedByTrackId); + + int movieLength = (int)BinaryPrimitives.ReadUInt32BigEndian(colorFile); + using MemoryStream stream = new(); + using BinaryWriter writer = new(stream, Encoding.UTF8, true); + long track = BeginBox(writer, Heif4CharCode.Trak); + WriteTrackHeader(writer, width, height, AlphaTrackId, false, true); + WriteTrackReference(writer, Heif4CharCode.Auxl, ColorTrackId); + + long media = BeginBox(writer, Heif4CharCode.Mdia); + WriteMediaHeader(writer, alphaTimescale); + WriteHandler(writer, Heif4CharCode.Auxv); + + long mediaInformation = BeginBox(writer, Heif4CharCode.Minf); + WriteDataInformation(writer); + WriteSampleTable( + writer, + alphaChunkOffset ?? chunkOffset, + false, + false, + ColorTrackId, + alphaTransforms, + false, + width, + height, + av1Configuration, + sampleSize, + null, + allSamplesSync, + true); + + EndBox(writer, mediaInformation); + EndBox(writer, media); + EndBox(writer, track); + + byte[] alphaTrack = stream.ToArray(); + byte[] file = new byte[SyntheticFileLength]; + + // The alpha TrackBox is appended inside the existing MovieBox, so patch the movie size after concatenation. + colorFile.AsSpan(0, movieLength).CopyTo(file); + alphaTrack.CopyTo(file, movieLength); + BinaryPrimitives.WriteUInt32BigEndian(file, (uint)(movieLength + alphaTrack.Length)); + return file; + } + + /// + /// Prefixes a synthetic movie with the AVIF image-sequence FileTypeBox used by the public decoder entry points. + /// + /// Whether the sequence carries image presentation properties. + /// Whether the sequence carries track-level Exif and XMP items. + /// The complete synthetic AVIF byte stream. + private static byte[] CreateSequenceContainer(bool trackProperties, bool trackMetadata) + { + byte[] movie = CreateSequenceFile( + SyntheticChunkOffset, + trackProperties: trackProperties, + trackMetadata: trackMetadata, + metadataInItemData: true); + + byte[] data = new byte[movie.Length + FileTypeBoxLength]; + WriteSequenceFileTypeBox(data); + movie.CopyTo(data, FileTypeBoxLength); + return data; + } + + /// + /// Builds a two-frame AVIF sequence that stores the same independently decodable AV1 sample in both frames. + /// + /// The complete AV1 sample payload. + /// The AV1CodecConfigurationBox payload describing the sample. + /// The complete synthetic AVIF byte stream. + private static byte[] CreateDecodableAv1SequenceContainer( + ReadOnlySpan sample, + ReadOnlySpan configuration, + bool trackProperties = false) + => CreateDecodableAv1SequenceContainer(sample, sample, configuration, true, trackProperties); + + /// + /// Builds a two-frame AVIF sequence with caller-provided AV1 samples so integrity tests can corrupt one sample + /// without also corrupting the parser-owned container structures. + /// + /// The first AV1 sample payload. + /// The second AV1 sample payload. + /// The AV1CodecConfigurationBox payload describing both samples. + /// Whether both samples are marked independently decodable. + /// The complete synthetic AVIF byte stream. + private static byte[] CreateDecodableAv1SequenceContainer( + ReadOnlySpan firstSample, + ReadOnlySpan secondSample, + ReadOnlySpan configuration, + bool allSamplesSync, + bool trackProperties = false) + { + uint chunkOffset = FileTypeBoxLength + SyntheticFileLength; + byte[] movie = CreateSequenceFile( + chunkOffset, + trackProperties: trackProperties, + width: 4, + height: 4, + av1Configuration: configuration.ToArray(), + sampleSize: firstSample.Length, + secondSampleSize: secondSample.Length, + allSamplesSync: allSamplesSync); + + byte[] data = new byte[chunkOffset + firstSample.Length + secondSample.Length]; + WriteSequenceFileTypeBox(data); + movie.CopyTo(data, FileTypeBoxLength); + firstSample.CopyTo(data.AsSpan((int)chunkOffset)); + secondSample.CopyTo(data.AsSpan((int)chunkOffset + firstSample.Length)); + return data; + } + + /// + /// Builds two frame-aligned AV1 tracks that intentionally reuse a color sample for the declared alpha track. + /// + /// The AV1 sample payload stored in every color and alpha frame. + /// The AV1CodecConfigurationBox payload describing the sample. + /// The complete synthetic AVIF byte stream. + private static byte[] CreateAv1SequenceWithNonMonochromeAlphaContainer(ReadOnlySpan sample, ReadOnlySpan configuration) + { + uint colorChunkOffset = FileTypeBoxLength + SyntheticFileLength; + uint alphaChunkOffset = colorChunkOffset + (uint)(sample.Length * 2); + byte[] movie = CreateSequenceFileWithAlpha( + colorChunkOffset, + SyntheticMovieTimescale, + 0, + alphaChunkOffset: alphaChunkOffset, + width: 4, + height: 4, + av1Configuration: configuration.ToArray(), + sampleSize: sample.Length, + allSamplesSync: true); + + byte[] data = new byte[alphaChunkOffset + (sample.Length * 2)]; + WriteSequenceFileTypeBox(data); + movie.CopyTo(data, FileTypeBoxLength); + sample.CopyTo(data.AsSpan((int)colorChunkOffset)); + sample.CopyTo(data.AsSpan((int)colorChunkOffset + sample.Length)); + sample.CopyTo(data.AsSpan((int)alphaChunkOffset)); + sample.CopyTo(data.AsSpan((int)alphaChunkOffset + sample.Length)); + return data; + } + + /// + /// Writes the AVIF image-sequence FileTypeBox shared by all complete synthetic decoder inputs. + /// + /// The destination whose first 24 bytes receive the box. + private static void WriteSequenceFileTypeBox(Span destination) + { + int fieldOffset = 0; + + // ISO/IEC 14496-12 Section 4.3 stores the box size and type first, followed by the major brand, minor + // version, and compatible brands. 'avis' selects the sequence presentation while 'avif' and 'mif1' declare + // compatibility with the AVIF and HEIF image-item structures also present in these files. + BinaryPrimitives.WriteUInt32BigEndian(destination[fieldOffset..], FileTypeBoxLength); + fieldOffset += sizeof(uint); + BinaryPrimitives.WriteUInt32BigEndian(destination[fieldOffset..], (uint)Heif4CharCode.Ftyp); + fieldOffset += sizeof(uint); + BinaryPrimitives.WriteUInt32BigEndian(destination[fieldOffset..], (uint)Heif4CharCode.Avis); + fieldOffset += sizeof(uint); + BinaryPrimitives.WriteUInt32BigEndian(destination[fieldOffset..], 0); + fieldOffset += sizeof(uint); + BinaryPrimitives.WriteUInt32BigEndian(destination[fieldOffset..], (uint)Heif4CharCode.Avif); + fieldOffset += sizeof(uint); + BinaryPrimitives.WriteUInt32BigEndian(destination[fieldOffset..], (uint)Heif4CharCode.Mif1); + } + + /// + /// Writes a version-zero TrackHeaderBox with an enabled movie track, integral 16.16 dimensions, and a selectable + /// presentation matrix. + /// + /// The writer receiving big-endian box fields. + /// The displayed track width in pixels. + /// The displayed track height in pixels. + /// The nonzero file-defined track identifier. + /// Whether to encode horizontal scaling instead of the unity matrix. + /// Whether to set the TrackHeaderBox enabled flag. + private static void WriteTrackHeader( + BinaryWriter writer, + int width, + int height, + uint trackId, + bool nonIdentityMatrix, + bool isEnabled) + { + const uint trackEnabledFlag = 1U << 0; + const int fixedPointFractionalBits = 16; + + long trackHeader = BeginBox(writer, Heif4CharCode.Tkhd); + + // ISO/IEC 14496-12 Section 8.3.2 assigns bit zero to track_enabled. libavif sequence tracks set that bit without + // requiring track_in_movie. The following fields hold times, identifier, duration, matrix, and 16.16 dimensions. + WriteFullBoxHeader(writer, 0, isEnabled ? trackEnabledFlag : 0); + WriteUInt32(writer, 0); + WriteUInt32(writer, 0); + WriteUInt32(writer, trackId); + WriteUInt32(writer, 0); + WriteUInt32(writer, SyntheticTrackDuration); + WriteZeros(writer, (2 * sizeof(uint)) + (4 * sizeof(ushort))); + WritePresentationMatrix(writer, nonIdentityMatrix); + WriteUInt32(writer, (uint)width << fixedPointFractionalBits); + WriteUInt32(writer, (uint)height << fixedPointFractionalBits); + EndBox(writer, trackHeader); + } + + /// + /// Writes the nine fixed-point coefficients of an ISO base media presentation matrix. + /// + /// The writer receiving big-endian matrix coefficients. + /// Whether the horizontal 16.16 scale is two instead of one. + private static void WritePresentationMatrix(BinaryWriter writer, bool nonIdentityMatrix) + { + // The first six coefficients use 16.16 fixed point and the final perspective column uses 2.30. Altering only + // the horizontal scale gives matrix-validation tests one controlled departure from the unity matrix. + WriteUInt32(writer, nonIdentityMatrix ? DoubleFixed16Point16 : UnityFixed16Point16); + WriteUInt32(writer, 0); + WriteUInt32(writer, 0); + WriteUInt32(writer, 0); + WriteUInt32(writer, UnityFixed16Point16); + WriteUInt32(writer, 0); + WriteUInt32(writer, 0); + WriteUInt32(writer, 0); + WriteUInt32(writer, UnityFixed2Point30); + } + + /// + /// Writes one TrackReferenceBox child that links the owning track to a single referenced track identifier. + /// + /// The writer receiving the reference boxes. + /// The image-sequence reference relationship. + /// The referenced track identifier. + private static void WriteTrackReference(BinaryWriter writer, Heif4CharCode referenceType, uint trackId) + { + long references = BeginBox(writer, Heif4CharCode.Tref); + long reference = BeginBox(writer, referenceType); + WriteUInt32(writer, trackId); + EndBox(writer, reference); + EndBox(writer, references); + } + + /// + /// Writes a repeating single-entry EditListBox whose 200 movie-time-scale units are repeated to fill the + /// 600-unit track duration, producing three total plays. + /// + /// The writer receiving the edit boxes. + private static void WriteEditList(BinaryWriter writer) + { + const uint repeatEditListFlag = 1U << 0; + const uint editEntryCount = 1; + const uint mediaStartTime = 0; + const ushort unityMediaRateInteger = 1; + const ushort unityMediaRateFraction = 0; + + long edit = BeginBox(writer, Heif4CharCode.Edts); + long editList = BeginBox(writer, Heif4CharCode.Elst); + WriteFullBoxHeader(writer, 0, repeatEditListFlag); + WriteUInt32(writer, editEntryCount); + WriteUInt32(writer, SyntheticMediaDuration); + WriteUInt32(writer, mediaStartTime); + WriteUInt16(writer, unityMediaRateInteger); + WriteUInt16(writer, unityMediaRateFraction); + EndBox(writer, editList); + EndBox(writer, edit); + } + + /// + /// Writes a version-zero MediaHeaderBox for two 100-unit samples and the packed ISO-639 language code "und". + /// + /// The writer receiving the media header. + /// The media time scale in units per second. + private static void WriteMediaHeader(BinaryWriter writer, uint timescale = SyntheticMovieTimescale) + { + long mediaHeader = BeginBox(writer, Heif4CharCode.Mdhd); + WriteFullBoxHeader(writer, 0, 0); + WriteUInt32(writer, 0); + WriteUInt32(writer, 0); + WriteUInt32(writer, timescale); + WriteUInt32(writer, SyntheticMediaDuration); + WriteUInt16(writer, PackedUndeterminedLanguage); + WriteUInt16(writer, 0); + EndBox(writer, mediaHeader); + } + + /// + /// Writes a HandlerBox with the requested track role and an empty null-terminated handler name. + /// + /// The writer receiving the handler box. + /// The four-character handler role. + private static void WriteHandler(BinaryWriter writer, Heif4CharCode handlerType) + { + long handler = BeginBox(writer, Heif4CharCode.Hdlr); + WriteFullBoxHeader(writer, 0, 0); + WriteUInt32(writer, 0); + WriteUInt32(writer, (uint)handlerType); + WriteZeros(writer, 12); + writer.Write((byte)0); + EndBox(writer, handler); + } + + /// + /// Writes a self-contained DataInformationBox whose single DataEntryUrlBox resolves sample offsets in this file. + /// + /// The writer receiving the data-reference hierarchy. + private static void WriteDataInformation(BinaryWriter writer) + { + long dataInformation = BeginBox(writer, Heif4CharCode.Dinf); + long dataReference = BeginBox(writer, Heif4CharCode.Dref); + WriteFullBoxHeader(writer, 0, 0); + WriteUInt32(writer, 1); + long location = BeginBox(writer, Heif4CharCode.Url); + WriteFullBoxHeader(writer, 0, 1); + EndBox(writer, location); + EndBox(writer, dataReference); + EndBox(writer, dataInformation); + } + + /// + /// Writes a track-level MetaBox containing one Exif item and one XMP MIME item, with extents addressing either + /// the enclosing file or an ItemDataBox. + /// + /// The writer receiving the metadata hierarchy. + /// Whether item extents use construction method one. + /// Whether to write an invalid video handler instead of the picture handler. + private static void WriteTrackMetadata(BinaryWriter writer, bool useItemData, bool invalidHandler) + { + const byte fourByteOffsetAndLengthSizes = 0x44; + const ushort metadataItemCount = 2; + + long metadata = BeginBox(writer, Heif4CharCode.Meta); + WriteFullBoxHeader(writer, 0, 0); + WriteHandler(writer, invalidHandler ? Heif4CharCode.Vide : Heif4CharCode.Pict); + + long itemLocations = BeginBox(writer, Heif4CharCode.Iloc); + WriteFullBoxHeader(writer, useItemData ? (byte)1 : (byte)0, 0); + + // ISO/IEC 14496-12 Section 8.11.3 packs offset_size and length_size into the high and low nibbles. Four-byte + // fields cover the synthetic file while keeping the encoded records identical to normal HEIF metadata. + writer.Write(fourByteOffsetAndLengthSizes); + writer.Write((byte)0); + WriteUInt16(writer, metadataItemCount); + WriteTrackMetadataLocation( + writer, + TrackExifItemId, + useItemData, + useItemData ? 0U : TrackExifOffset, + (uint)TrackExifData.Length); + + WriteTrackMetadataLocation( + writer, + TrackXmpItemId, + useItemData, + useItemData ? (uint)TrackExifData.Length : TrackXmpOffset, + (uint)TrackXmpData.Length); + + EndBox(writer, itemLocations); + + long itemInformation = BeginBox(writer, Heif4CharCode.Iinf); + WriteFullBoxHeader(writer, 0, 0); + WriteUInt16(writer, metadataItemCount); + WriteTrackMetadataItem(writer, TrackExifItemId, Heif4CharCode.Exif); + WriteTrackMetadataItem(writer, TrackXmpItemId, Heif4CharCode.Mime); + EndBox(writer, itemInformation); + + if (useItemData) + { + long itemData = BeginBox(writer, Heif4CharCode.Idat); + writer.Write(TrackExifData); + writer.Write(TrackXmpData); + EndBox(writer, itemData); + } + + EndBox(writer, metadata); + } + + /// + /// Writes one ItemLocationBox record with a single extent encoded using four-byte offsets and lengths. + /// + /// The writer receiving the item-location record. + /// The item identifier linked to an ItemInfoEntry. + /// Whether the extent addresses ItemDataBox bytes. + /// The extent offset relative to the selected construction method. + /// The extent length in bytes. + private static void WriteTrackMetadataLocation(BinaryWriter writer, ushort itemId, bool useItemData, uint offset, uint length) + { + const ushort itemDataConstructionMethod = 1; + const ushort localDataReferenceIndex = 0; + const ushort extentCount = 1; + + WriteUInt16(writer, itemId); + if (useItemData) + { + WriteUInt16(writer, itemDataConstructionMethod); + } + + WriteUInt16(writer, localDataReferenceIndex); + WriteUInt16(writer, extentCount); + WriteUInt32(writer, offset); + WriteUInt32(writer, length); + } + + /// + /// Writes a version-two ItemInfoEntry for an Exif item or an XMP item using the registered RDF MIME type. + /// + /// The writer receiving the item information entry. + /// The identifier matched by the location record. + /// The item's four-character type. + private static void WriteTrackMetadataItem(BinaryWriter writer, ushort itemId, Heif4CharCode itemType) + { + const byte itemInfoVersion = 2; + const ushort noItemProtection = 0; + + long itemInformationEntry = BeginBox(writer, Heif4CharCode.Infe); + WriteFullBoxHeader(writer, itemInfoVersion, 0); + WriteUInt16(writer, itemId); + WriteUInt16(writer, noItemProtection); + WriteUInt32(writer, (uint)itemType); + writer.Write((byte)0); + if (itemType == Heif4CharCode.Mime) + { + writer.Write("application/rdf+xml"u8); + writer.Write((byte)0); + } + + EndBox(writer, itemInformationEntry); + } + + /// + /// Writes the timing, sample-to-chunk, size, chunk-offset, sync-sample, optional composition, and optional direct + /// reference boxes for exactly two image-sequence samples. + /// + /// The writer receiving the SampleTableBox. + /// The absolute file offset of the single sample chunk. + /// Whether to write prohibited composition timing boxes. + /// Whether to write AV1 direct-reference grouping. + /// The sample identifier referenced by the second sample. + /// Whether the visual sample entry contains presentation properties. + /// Whether the rotation property contains reserved high bits. + /// The coded sample width in pixels. + /// The coded sample height in pixels. + /// The AV1CodecConfigurationBox payload. + /// The first sample length, or the synthetic default. + /// The second sample length, or the first/default length. + /// Whether both samples are listed as sync samples. + /// Whether the sample entry describes an auxiliary alpha track. + private static void WriteSampleTable( + BinaryWriter writer, + uint chunkOffset, + bool compositionOffsets, + bool directReferences, + uint directReferenceSampleId, + bool trackProperties, + bool invalidRotation, + int width, + int height, + byte[] av1Configuration, + int? sampleSize, + int? secondSampleSize, + bool allSamplesSync, + bool alpha = false) + { + const uint singleEntry = 1; + const uint firstChunk = 1; + const uint sampleDescriptionIndex = 1; + const uint variableSampleSizes = 0; + + long sampleTable = BeginBox(writer, Heif4CharCode.Stbl); + WriteSampleDescription(writer, trackProperties, invalidRotation, width, height, av1Configuration, allSamplesSync, alpha); + + long timing = BeginBox(writer, Heif4CharCode.Stts); + WriteFullBoxHeader(writer, 0, 0); + WriteUInt32(writer, singleEntry); + WriteUInt32(writer, SyntheticSampleCount); + WriteUInt32(writer, SyntheticSampleDuration); + EndBox(writer, timing); + + long sampleToChunk = BeginBox(writer, Heif4CharCode.Stsc); + WriteFullBoxHeader(writer, 0, 0); + WriteUInt32(writer, singleEntry); + WriteUInt32(writer, firstChunk); + WriteUInt32(writer, SyntheticSampleCount); + WriteUInt32(writer, sampleDescriptionIndex); + EndBox(writer, sampleToChunk); + + long sampleSizes = BeginBox(writer, Heif4CharCode.Stsz); + WriteFullBoxHeader(writer, 0, 0); + WriteUInt32(writer, variableSampleSizes); + WriteUInt32(writer, SyntheticSampleCount); + WriteUInt32(writer, (uint)(sampleSize ?? FirstSyntheticSampleLength)); + WriteUInt32(writer, (uint)(secondSampleSize ?? sampleSize ?? SecondSyntheticSampleLength)); + EndBox(writer, sampleSizes); + + long chunkOffsets = BeginBox(writer, Heif4CharCode.Stco); + WriteFullBoxHeader(writer, 0, 0); + WriteUInt32(writer, singleEntry); + WriteUInt32(writer, chunkOffset); + EndBox(writer, chunkOffsets); + + long syncSamples = BeginBox(writer, Heif4CharCode.Stss); + WriteFullBoxHeader(writer, 0, 0); + WriteUInt32(writer, allSamplesSync ? SyntheticSampleCount : singleEntry); + WriteUInt32(writer, firstChunk); + if (allSamplesSync) + { + WriteUInt32(writer, SyntheticSampleCount); + } + + EndBox(writer, syncSamples); + + if (compositionOffsets) + { + const byte signedCompositionOffsetVersion = 1; + const uint compositionEntryCount = SyntheticSampleCount; + const uint oneSample = 1; + uint hiddenSampleOffset = unchecked((uint)int.MinValue); + + long offsets = BeginBox(writer, Heif4CharCode.Ctts); + WriteFullBoxHeader(writer, signedCompositionOffsetVersion, 0); + WriteUInt32(writer, compositionEntryCount); + WriteUInt32(writer, oneSample); + WriteUInt32(writer, hiddenSampleOffset); + WriteUInt32(writer, oneSample); + WriteUInt32(writer, 0); + EndBox(writer, offsets); + + // The CompositionToDecodeBox declares that visible presentation begins after the hidden sample's + // 100-unit slot and ends at the two-sample media duration. + long compositionToDecode = BeginBox(writer, Heif4CharCode.Cslg); + WriteFullBoxHeader(writer, 0, 0); + WriteUInt32(writer, 0); + WriteUInt32(writer, 0); + WriteUInt32(writer, 0); + WriteUInt32(writer, SyntheticSampleDuration); + WriteUInt32(writer, SyntheticMediaDuration); + EndBox(writer, compositionToDecode); + } + + if (directReferences) + { + WriteDirectReferenceSampleGroup(writer, directReferenceSampleId); + } + + EndBox(writer, sampleTable); + } + + /// + /// Writes AV1 "refs" sample-group descriptions and maps the first sample to an independent description and the + /// second sample to a description containing one direct reference. + /// + /// The writer receiving the sample-group boxes. + /// The sample identifier named by the dependent description. + private static void WriteDirectReferenceSampleGroup(BinaryWriter writer, uint directReferenceSampleId) + { + const byte variableLengthDescriptionVersion = 1; + const uint variableDescriptionLength = 0; + const uint descriptionCount = SyntheticSampleCount; + const uint independentDescriptionLength = sizeof(uint) + sizeof(byte); + const uint dependentDescriptionLength = independentDescriptionLength + sizeof(uint); + const uint independentSampleId = ColorTrackId; + const uint dependentSampleId = 0; + const byte noDirectReferences = 0; + const byte oneDirectReference = 1; + + long descriptions = BeginBox(writer, Heif4CharCode.Sgpd); + WriteFullBoxHeader(writer, variableLengthDescriptionVersion, 0); + WriteUInt32(writer, (uint)Heif4CharCode.Refs); + WriteUInt32(writer, variableDescriptionLength); + WriteUInt32(writer, descriptionCount); + WriteUInt32(writer, independentDescriptionLength); + WriteUInt32(writer, independentSampleId); + writer.Write(noDirectReferences); + WriteUInt32(writer, dependentDescriptionLength); + WriteUInt32(writer, dependentSampleId); + writer.Write(oneDirectReference); + WriteUInt32(writer, directReferenceSampleId); + EndBox(writer, descriptions); + + const uint sampleGroupRunCount = SyntheticSampleCount; + const uint oneSamplePerRun = 1; + const uint independentDescriptionIndex = 1; + const uint dependentDescriptionIndex = 2; + + long sampleMap = BeginBox(writer, Heif4CharCode.Sbgp); + WriteFullBoxHeader(writer, 0, 0); + WriteUInt32(writer, (uint)Heif4CharCode.Refs); + WriteUInt32(writer, sampleGroupRunCount); + WriteUInt32(writer, oneSamplePerRun); + WriteUInt32(writer, independentDescriptionIndex); + WriteUInt32(writer, oneSamplePerRun); + WriteUInt32(writer, dependentDescriptionIndex); + EndBox(writer, sampleMap); + } + + /// + /// Writes one VisualSampleEntry and its codec configuration, optional alpha role, optional presentation + /// properties, and CodingConstraintsBox. + /// + /// The writer receiving the SampleDescriptionBox. + /// Whether to append image presentation properties. + /// Whether the rotation property contains reserved high bits. + /// The coded sample width in pixels. + /// The coded sample height in pixels. + /// The AV1CodecConfigurationBox payload. + /// Whether coding constraints declare every reference picture intra coded. + /// Whether the entry carries the HEIF alpha auxiliary type. + private static void WriteSampleDescription( + BinaryWriter writer, + bool trackProperties, + bool invalidRotation, + int width, + int height, + byte[] av1Configuration, + bool allSamplesSync, + bool alpha) + { + const ushort localDataReferenceIndex = 1; + const ushort visualSampleFrameCount = 1; + const int compressorNameLength = 32; + const ushort noColorTable = ushort.MaxValue; + const uint allReferencePicturesIntraMask = 1U << 31; + const uint intraPicturePredictionUsedMask = 1U << 30; + const int maximumReferencesShift = 26; + const uint maximumReferencesPerPicture = 15; + + long description = BeginBox(writer, Heif4CharCode.Stsd); + WriteFullBoxHeader(writer, 0, 0); + WriteUInt32(writer, 1); + long sampleEntry = BeginBox(writer, Heif4CharCode.Av01); + + // ISO/IEC 14496-12 Section 12.1.3 defines six reserved bytes and a data-reference index before the visual + // sample entry's predefined words, dimensions, 16.16 resolution, frame count, fixed compressor-name field, + // pixel depth, and the -1 sentinel indicating that no color table is present. + WriteZeros(writer, 6); + WriteUInt16(writer, localDataReferenceIndex); + WriteZeros(writer, (2 * sizeof(ushort)) + (3 * sizeof(uint))); + WriteUInt16(writer, (ushort)width); + WriteUInt16(writer, (ushort)height); + WriteUInt32(writer, SyntheticHorizontalResolution); + WriteUInt32(writer, SyntheticHorizontalResolution); + WriteUInt32(writer, 0); + WriteUInt16(writer, visualSampleFrameCount); + WriteZeros(writer, compressorNameLength); + WriteUInt16(writer, SyntheticPixelDepth); + WriteUInt16(writer, noColorTable); + + long configuration = BeginBox(writer, Heif4CharCode.Av1C); + ReadOnlySpan configurationPayload = av1Configuration is null ? DefaultAv1Configuration : av1Configuration; + + writer.Write(configurationPayload); + EndBox(writer, configuration); + + if (alpha) + { + long auxiliaryType = BeginBox(writer, Heif4CharCode.Auxi); + WriteFullBoxHeader(writer, 0, 0); + writer.Write(Encoding.UTF8.GetBytes(HeifConstants.AlphaAuxiliaryType)); + writer.Write((byte)0); + EndBox(writer, auxiliaryType); + } + + if (trackProperties) + { + WriteTrackImageProperties(writer, invalidRotation, width, height); + } + + long codingConstraintsBox = BeginBox(writer, Heif4CharCode.Ccst); + WriteFullBoxHeader(writer, 0, 0); + + // HEIF CodingConstraintsBox places the two boolean constraints in bits 31 and 30 and the four-bit maximum + // reference count in bits 29..26. The synthetic dependency path deliberately permits the maximum of fifteen. + uint codingConstraints = intraPicturePredictionUsedMask | (maximumReferencesPerPicture << maximumReferencesShift); + if (allSamplesSync) + { + codingConstraints |= allReferencePicturesIntraMask; + } + + WriteUInt32(writer, codingConstraints); + EndBox(writer, codingConstraintsBox); + EndBox(writer, sampleEntry); + EndBox(writer, description); + } + + /// + /// Writes representative color, aspect-ratio, clean-aperture, orientation, HDR, and viewing-environment + /// properties inside a visual sample entry. + /// + /// The writer receiving the property boxes. + /// Whether the rotation byte contains reserved high bits. + /// The clean-aperture width in pixels. + /// The clean-aperture height in pixels. + private static void WriteTrackImageProperties(BinaryWriter writer, bool invalidRotation, int width, int height) + { + const byte fullRangeFlag = 1 << 7; + const byte reservedRotationBits = 0xFC; + const byte rotateCounterClockwise90Degrees = 1; + const byte verticalMirrorAxis = 1; + + long color = BeginBox(writer, Heif4CharCode.Colr); + WriteUInt32(writer, (uint)Heif4CharCode.Nclx); + WriteUInt16(writer, (ushort)CicpColorPrimaries.ItuRBt709_6); + WriteUInt16(writer, (ushort)CicpTransferCharacteristics.Iec61966_2_1); + WriteUInt16(writer, (ushort)CicpMatrixCoefficients.ItuRBt601_7_525); + writer.Write(fullRangeFlag); + EndBox(writer, color); + + long pixelAspectRatio = BeginBox(writer, Heif4CharCode.Pasp); + WriteUInt32(writer, SyntheticHorizontalPixelSpacing); + WriteUInt32(writer, SyntheticVerticalPixelSpacing); + EndBox(writer, pixelAspectRatio); + + const uint cleanApertureDenominator = 1; + const uint centeredCleanApertureOffset = 0; + + long cleanAperture = BeginBox(writer, Heif4CharCode.Clap); + WriteUInt32(writer, (uint)width); + WriteUInt32(writer, cleanApertureDenominator); + WriteUInt32(writer, (uint)height); + WriteUInt32(writer, cleanApertureDenominator); + WriteUInt32(writer, centeredCleanApertureOffset); + WriteUInt32(writer, cleanApertureDenominator); + WriteUInt32(writer, centeredCleanApertureOffset); + WriteUInt32(writer, cleanApertureDenominator); + EndBox(writer, cleanAperture); + + long rotation = BeginBox(writer, Heif4CharCode.Irot); + writer.Write(invalidRotation ? reservedRotationBits : rotateCounterClockwise90Degrees); + EndBox(writer, rotation); + + long mirror = BeginBox(writer, Heif4CharCode.Imir); + writer.Write(verticalMirrorAxis); + EndBox(writer, mirror); + + long contentLightLevel = BeginBox(writer, Heif4CharCode.Clli); + WriteUInt16(writer, SyntheticMaximumContentLightLevel); + WriteUInt16(writer, SyntheticMaximumFrameAverageLightLevel); + EndBox(writer, contentLightLevel); + + // MasteringDisplayColorVolume stores chromaticity in 0.00002 increments and luminance in 0.0001 cd/m2. + const ushort redPrimaryX = 15_000; + const ushort redPrimaryY = 30_000; + const ushort greenPrimaryX = 7_500; + const ushort greenPrimaryY = 3_000; + const ushort bluePrimaryX = 34_000; + const ushort bluePrimaryY = 16_000; + const ushort whitePointX = 15_635; + const ushort whitePointY = 16_450; + const uint maximumDisplayLuminance = 10_000_000; + const uint minimumDisplayLuminance = 50; + + long masteringDisplay = BeginBox(writer, Heif4CharCode.Mdcv); + WriteUInt16(writer, redPrimaryX); + WriteUInt16(writer, redPrimaryY); + WriteUInt16(writer, greenPrimaryX); + WriteUInt16(writer, greenPrimaryY); + WriteUInt16(writer, bluePrimaryX); + WriteUInt16(writer, bluePrimaryY); + WriteUInt16(writer, whitePointX); + WriteUInt16(writer, whitePointY); + WriteUInt32(writer, maximumDisplayLuminance); + WriteUInt32(writer, minimumDisplayLuminance); + EndBox(writer, masteringDisplay); + + const byte minimumLuminancePresentMask = 1 << 4; + const byte maximumLuminancePresentMask = 1 << 3; + const byte averageLuminancePresentMask = 1 << 2; + const uint minimumLuminanceValue = 1_000_000; + const uint maximumLuminanceValue = 10_000_000; + const uint averageLuminanceValue = 5_000_000; + + long contentColorVolume = BeginBox(writer, Heif4CharCode.Cclv); + + // ISO/IEC 23000-22 declares the optional content-color-volume fields through bits 5..2. This payload carries + // all three luminance values and deliberately omits the much larger primary-chromaticity field set. + writer.Write((byte)(minimumLuminancePresentMask | maximumLuminancePresentMask | averageLuminancePresentMask)); + WriteUInt32(writer, minimumLuminanceValue); + WriteUInt32(writer, maximumLuminanceValue); + WriteUInt32(writer, averageLuminanceValue); + EndBox(writer, contentColorVolume); + + const uint ambientIlluminance = 10_000; + + long ambientViewing = BeginBox(writer, Heif4CharCode.Amve); + WriteUInt32(writer, ambientIlluminance); + WriteUInt16(writer, whitePointX); + WriteUInt16(writer, whitePointY); + EndBox(writer, ambientViewing); + + const uint referenceViewingIlluminance = 10_000; + const ushort referenceWhiteX = 3_127; + const ushort referenceWhiteY = 3_290; + const uint referenceBlackLuminance = 5_000; + + long referenceViewing = BeginBox(writer, Heif4CharCode.Reve); + WriteFullBoxHeader(writer, 0, 0); + WriteUInt32(writer, referenceViewingIlluminance); + WriteUInt16(writer, referenceWhiteX); + WriteUInt16(writer, referenceWhiteY); + WriteUInt32(writer, referenceBlackLuminance); + WriteUInt16(writer, referenceWhiteX); + WriteUInt16(writer, referenceWhiteY); + EndBox(writer, referenceViewing); + + const uint nominalDiffuseWhiteLuminance = 2_030_000; + + long nominalDiffuseWhite = BeginBox(writer, Heif4CharCode.Ndwt); + WriteFullBoxHeader(writer, 0, 0); + WriteUInt32(writer, nominalDiffuseWhiteLuminance); + EndBox(writer, nominalDiffuseWhite); + } + + /// + /// Starts a small ISO base media box with a placeholder 32-bit size that patches later. + /// + /// The writer receiving the box header. + /// The box's four-character type. + /// The stream position of the size field. + private static long BeginBox(BinaryWriter writer, Heif4CharCode type) + { + long start = writer.BaseStream.Position; + WriteUInt32(writer, 0); + WriteUInt32(writer, (uint)type); + return start; + } + + /// + /// Completes a box by patching its total byte length and restoring the writer to the end of the payload. + /// + /// The writer containing the box. + /// The stream position returned by . + private static void EndBox(BinaryWriter writer, long start) + { + long end = writer.BaseStream.Position; + writer.BaseStream.Position = start; + WriteUInt32(writer, checked((uint)(end - start))); + writer.BaseStream.Position = end; + } + + /// + /// Writes the FullBox version byte and low 24 flag bits as one big-endian word. + /// + /// The writer receiving the FullBox header. + /// The box syntax version. + /// The box-specific low 24 flag bits. + private static void WriteFullBoxHeader(BinaryWriter writer, byte version, uint flags) + => WriteUInt32(writer, ((uint)version << 24) | flags); + + /// + /// Writes an unsigned 16-bit ISO base media field in network byte order. + /// + /// The little-endian binary writer receiving the field. + /// The host-order value. + private static void WriteUInt16(BinaryWriter writer, ushort value) + => writer.Write(BinaryPrimitives.ReverseEndianness(value)); + + /// + /// Writes an unsigned 32-bit ISO base media field in network byte order. + /// + /// The little-endian binary writer receiving the field. + /// The host-order value. + private static void WriteUInt32(BinaryWriter writer, uint value) + => writer.Write(BinaryPrimitives.ReverseEndianness(value)); + + /// + /// Writes reserved bytes whose governing box syntax requires all bits to be zero. + /// + /// The writer receiving the reserved bytes. + /// The number of reserved bytes. + private static void WriteZeros(BinaryWriter writer, int count) => writer.Write(new byte[count]); + + /// + /// Reads the synthetic movie box size and removes its standard eight-byte header to obtain the parser payload + /// boundary expected by . + /// + /// The synthetic file beginning with a MovieBox. + /// The validated movie payload length. + private static int GetMoviePayloadLength(byte[] data) + => checked((int)BinaryPrimitives.ReadUInt32BigEndian(data) - BoxHeaderLength); + + /// + /// Creates a sequence parser with the decoder limits and integrity policy exercised by a test. + /// + /// The maximum number of sequence frames to retain. + /// Whether optional metadata parsing is disabled. + /// The malformed-segment recovery policy. + /// The configured parser. + private static HeifSequenceParser CreateParser( + uint maxFrames, + bool skipMetadata = false, + SegmentIntegrityHandling segmentIntegrityHandling = SegmentIntegrityHandling.Strict) + => new(new DecoderOptions + { + MaxFrames = maxFrames, + SkipMetadata = skipMetadata, + SegmentIntegrityHandling = segmentIntegrityHandling + }); +} diff --git a/tests/ImageSharp.Tests/Formats/ImageFormatManagerTests.cs b/tests/ImageSharp.Tests/Formats/ImageFormatManagerTests.cs index 1cd03e0d5d..146898b46f 100644 --- a/tests/ImageSharp.Tests/Formats/ImageFormatManagerTests.cs +++ b/tests/ImageSharp.Tests/Formats/ImageFormatManagerTests.cs @@ -6,6 +6,7 @@ using SixLabors.ImageSharp.Formats.Ani; using SixLabors.ImageSharp.Formats.Bmp; using SixLabors.ImageSharp.Formats.Gif; +using SixLabors.ImageSharp.Formats.Heif; using SixLabors.ImageSharp.Formats.Jpeg; using SixLabors.ImageSharp.Formats.Pbm; using SixLabors.ImageSharp.Formats.Png; @@ -37,6 +38,7 @@ public void IfAutoLoadWellKnownFormatsIsTrueAllFormatsAreLoaded() Assert.Equal(1, this.DefaultFormatsManager.ImageEncoders.Select(item => item.Value).OfType().Count()); Assert.Equal(1, this.DefaultFormatsManager.ImageEncoders.Select(item => item.Value).OfType().Count()); Assert.Equal(1, this.DefaultFormatsManager.ImageEncoders.Select(item => item.Value).OfType().Count()); + Assert.Equal(1, this.DefaultFormatsManager.ImageEncoders.Select(item => item.Value).OfType().Count()); Assert.Equal(1, this.DefaultFormatsManager.ImageEncoders.Select(item => item.Value).OfType().Count()); Assert.Equal(1, this.DefaultFormatsManager.ImageEncoders.Select(item => item.Value).OfType().Count()); Assert.Equal(1, this.DefaultFormatsManager.ImageEncoders.Select(item => item.Value).OfType().Count()); @@ -47,6 +49,7 @@ public void IfAutoLoadWellKnownFormatsIsTrueAllFormatsAreLoaded() Assert.Equal(1, this.DefaultFormatsManager.ImageDecoders.Select(item => item.Value).OfType().Count()); Assert.Equal(1, this.DefaultFormatsManager.ImageDecoders.Select(item => item.Value).OfType().Count()); Assert.Equal(1, this.DefaultFormatsManager.ImageDecoders.Select(item => item.Value).OfType().Count()); + Assert.Equal(1, this.DefaultFormatsManager.ImageDecoders.Select(item => item.Value).OfType().Count()); Assert.Equal(1, this.DefaultFormatsManager.ImageDecoders.Select(item => item.Value).OfType().Count()); Assert.Equal(1, this.DefaultFormatsManager.ImageDecoders.Select(item => item.Value).OfType().Count()); Assert.Equal(1, this.DefaultFormatsManager.ImageDecoders.Select(item => item.Value).OfType().Count()); diff --git a/tests/ImageSharp.Tests/Image/ImageTests.Decode_Cancellation.cs b/tests/ImageSharp.Tests/Image/ImageTests.Decode_Cancellation.cs index 368048a99e..f7fe9874d8 100644 --- a/tests/ImageSharp.Tests/Image/ImageTests.Decode_Cancellation.cs +++ b/tests/ImageSharp.Tests/Image/ImageTests.Decode_Cancellation.cs @@ -16,6 +16,7 @@ public class Decode_Cancellation : ImageLoadTestBase public static readonly string[] TestFileForEachCodec = [ TestImages.Jpeg.Baseline.Snake, + TestImages.Heif.IrvineAvif, // TODO: Figure out Unix cancellation failures, and validate cancellation for each decoder. //TestImages.Bmp.Car, diff --git a/tests/ImageSharp.Tests/Primitives/PointTests.cs b/tests/ImageSharp.Tests/Primitives/PointTests.cs index d2192f53bb..b740cbb28d 100644 --- a/tests/ImageSharp.Tests/Primitives/PointTests.cs +++ b/tests/ImageSharp.Tests/Primitives/PointTests.cs @@ -4,7 +4,7 @@ using System.Globalization; using System.Numerics; -namespace SixLabors.ImageSharp.Tests; +namespace SixLabors.ImageSharp.Tests.Primitives; public class PointTests { @@ -112,6 +112,27 @@ public void ArithmeticTest(int x, int y) Assert.Equal(subExpected, Point.Subtract(p, s)); } + [Theory] + [InlineData(int.MaxValue, int.MaxValue, 5)] + [InlineData(int.MinValue, int.MinValue, 4)] + [InlineData(int.MaxValue, int.MaxValue, 2)] + [InlineData(0, 0, 3)] + public void ShiftTest(int x, int y, int s) + { + Point rightExpected, leftExpected, p = new Point(x, y); + + unchecked + { + rightExpected = new Point(x >> s, y >> s); + leftExpected = new Point(x << s, y << s); + } + + Assert.Equal(rightExpected, p >> s); + Assert.Equal(leftExpected, p << s); + Assert.Equal(rightExpected, Point.ShiftRight(p, s)); + Assert.Equal(leftExpected, Point.ShiftLeft(p, s)); + } + [Theory] [InlineData(float.MaxValue, float.MinValue)] [InlineData(float.MinValue, float.MinValue)] diff --git a/tests/ImageSharp.Tests/TestImages.cs b/tests/ImageSharp.Tests/TestImages.cs index b7ef0d43f8..b3baf6a3d7 100644 --- a/tests/ImageSharp.Tests/TestImages.cs +++ b/tests/ImageSharp.Tests/TestImages.cs @@ -1277,6 +1277,202 @@ public static class Qoi public const string Wikipedia008 = "Qoi/wikipedia_008.qoi"; } + public static class Heif + { + // Downloaded from: https://github.com/AOMediaCodec/av1-avif/blob/master/testFiles/Microsoft/Irvine_CA.avif + public const string IrvineAvif = "Heif/Irvine_CA.avif"; + + public const string XnConvert = "Heif/jpeg444_xnconvert.avif"; + + // Extremely small image, 4x4 pixels with a single solid color. + public const string Orange4x4 = "Heif/Orange4x4.avif"; + + // Sourced from libavif/tests/data under libavif's BSD-2-Clause license. The AVIF was independently + // encoded from the paired PNG with avifenc and carries the same ICC, Exif, and XMP metadata. + public const string ParisIccExifXmpAvif = "Heif/paris_icc_exif_xmp.avif"; + public const string ParisIccExifXmpPng = "Heif/paris_icc_exif_xmp.png"; + + // Independently encoded from Png.Icc.Perceptual. Unlike the + // canonical-sRGB Paris profile, this perceptual profile requires an observable color conversion. + public const string PerceptualIccAvif = "Heif/perceptual_icc.avif"; + public const string PerceptualIccGridAvif = "Heif/perceptual_icc_grid.avif"; + public const string PerceptualIccSequenceAvif = "Heif/perceptual_icc_sequence.avif"; + + // Independently encoded from Png.Ducky. The ROMM RGB profile is + // deliberately non-sRGB so that color conversion is observable while the auxiliary alpha item is composed. + public const string DuckyRommIccAlphaAvif = "Heif/ducky_romm_icc_alpha.avif"; + + // Sourced from libavif/tests/data under libavif's BSD-2-Clause license. + public const string Animated8Bit = "Heif/colors-animated-8bpc.avif"; + public const string Animated8BitWithAudio = "Heif/colors-animated-8bpc-audio.avif"; + public const string Animated8BitWithAlphaExifXmp = "Heif/colors-animated-8bpc-alpha-exif-xmp.avif"; + public const string Animated12BitWithKeyframes = "Heif/colors-animated-12bpc-keyframes-0-2-3.avif"; + + // Genuine AVIF inputs, independently encoded AV1 samples, and retained exact native-plane output. The + // fixture README records provenance and the current-main verification status for each checkpoint. + public const string Av1Deblocking8BitAvif = "Heif/Av1/Conformance/libavif-kodim23-8b.avif"; + public const string Av1Deblocking8BitPayload = "Heif/Av1/Conformance/libavif-kodim23-8b.bit"; + public const string Av1Deblocking8BitReference = "Heif/Av1/Conformance/libavif-kodim23-8b-libaom.yuv"; + public const string Av1Deblocking10BitAvif = "Heif/Av1/Conformance/libavif-cosmos1650-10b.avif"; + public const string Av1Deblocking10BitPayload = "Heif/Av1/Conformance/libavif-cosmos1650-10b.bit"; + public const string Av1Deblocking10BitReference = "Heif/Av1/Conformance/libavif-cosmos1650-10b-libaom.yuv"; + public const string Av1Deblocking12BitAvif = "Heif/Av1/Conformance/libavif-colors-12b.avif"; + public const string Av1Deblocking12BitPayload = "Heif/Av1/Conformance/libaom-cosmos1650-12b.bit"; + public const string Av1Deblocking12BitReference = "Heif/Av1/Conformance/libaom-cosmos1650-12b-libaom.yuv"; + public const string Av1Progressive8BitAvif = "Heif/Av1/Conformance/libavif-progressive-draw-points-8b.avif"; + public const string Av1Progressive8BitPayload = "Heif/Av1/Conformance/libavif-progressive-draw-points-8b.bit"; + public const string Av1Progressive8BitReference = "Heif/Av1/Conformance/libavif-progressive-draw-points-8b-libaom-y4m.yuv"; + public const string Av1ScaledReferenceAvif = "Heif/Av1/Conformance/libavif-webp-logo-scaled-reference.avif"; + public const string Av1ScaledReferenceSelectedLayerAvif = "Heif/Av1/Conformance/libavif-webp-logo-scaled-reference-lsel0.avif"; + public const string Av1ScaledReferencePayload = "Heif/Av1/Conformance/libavif-webp-logo-scaled-reference.bit"; + public const string Av1ScaledReferenceBaseNativeReference = "Heif/Av1/Conformance/libavif-webp-logo-scaled-reference-base-libaom.yuv"; + public const string Av1ScaledReferenceNativeReference = "Heif/Av1/Conformance/libavif-webp-logo-scaled-reference-libaom.y4m"; + public const string Av1AverageCompoundSequenceAvif = "Heif/Av1/Conformance/libavif-webp-logo-average-compound.avif"; + public const string Av1AverageCompoundSequenceNativeReference = "Heif/Av1/Conformance/libavif-webp-logo-average-compound-libaom.y4m"; + public const string Av1DistanceWeightedCompoundSequenceAvif = "Heif/Av1/Conformance/libavif-webp-logo-distance-weighted-compound.avif"; + public const string Av1DistanceWeightedCompoundSequenceNativeReference = "Heif/Av1/Conformance/libavif-webp-logo-distance-weighted-compound-libaom.y4m"; + public const string Av1WedgeCompoundSequenceAvif = "Heif/Av1/Conformance/libavif-webp-logo-wedge-compound.avif"; + public const string Av1WedgeCompoundSequenceNativeReference = "Heif/Av1/Conformance/libavif-webp-logo-wedge-compound-libaom.y4m"; + public const string Av1DifferenceWeightedCompoundSequenceAvif = "Heif/Av1/Conformance/libavif-webp-logo-difference-weighted-compound.avif"; + public const string Av1DifferenceWeightedCompoundSequenceNativeReference = "Heif/Av1/Conformance/libavif-webp-logo-difference-weighted-compound-libaom.y4m"; + public const string Av1InterIntraSequenceAvif = "Heif/Av1/Conformance/libavif-webp-logo-inter-intra.avif"; + public const string Av1InterIntraSequenceNativeReference = "Heif/Av1/Conformance/libavif-webp-logo-inter-intra-libaom.y4m"; + public const string Av1ObmcSequenceAvif = "Heif/Av1/Conformance/libavif-webp-logo-obmc.avif"; + public const string Av1ObmcSequenceNativeReference = "Heif/Av1/Conformance/libavif-webp-logo-obmc-libaom.y4m"; + public const string Av1LocalWarpSequenceAvif = "Heif/Av1/Conformance/libavif-rotating-grid-local-warp.avif"; + public const string Av1LocalWarpSequenceNativeReference = "Heif/Av1/Conformance/libavif-rotating-grid-local-warp-libaom.y4m"; + public const string Av1GlobalWarpSequenceAvif = "Heif/Av1/Conformance/libavif-rotating-grid-global-warp.avif"; + public const string Av1GlobalWarpSequenceNativeReference = "Heif/Av1/Conformance/libavif-rotating-grid-global-warp-libaom.y4m"; + public const string Av1OfficialAllIntraSequence = "Heif/Av1/Conformance/libaom-av1-1-b8-02-allintra.ivf"; + public const string Av1OfficialAllIntraSequenceNativeReference = "Heif/Av1/Conformance/libaom-av1-1-b8-02-allintra-libaom.y4m"; + public const string Av1OfficialEightBitMinimumQuantizerSequence = "Heif/Av1/Conformance/libaom-av1-1-b8-00-quantizer-00.ivf"; + public const string Av1OfficialEightBitMinimumQuantizerSequenceNativeReference = "Heif/Av1/Conformance/libaom-av1-1-b8-00-quantizer-00-libaom.y4m"; + public const string Av1OfficialEightBitMaximumQuantizerSequence = "Heif/Av1/Conformance/libaom-av1-1-b8-00-quantizer-63.ivf"; + public const string Av1OfficialEightBitMaximumQuantizerSequenceNativeReference = "Heif/Av1/Conformance/libaom-av1-1-b8-00-quantizer-63-libaom.y4m"; + public const string Av1OfficialTenBitMinimumQuantizerSequence = "Heif/Av1/Conformance/libaom-av1-1-b10-00-quantizer-00.ivf"; + public const string Av1OfficialTenBitMinimumQuantizerSequenceNativeReference = "Heif/Av1/Conformance/libaom-av1-1-b10-00-quantizer-00-libaom.y4m"; + public const string Av1OfficialTenBitMaximumQuantizerSequence = "Heif/Av1/Conformance/libaom-av1-1-b10-00-quantizer-63.ivf"; + public const string Av1OfficialTenBitMaximumQuantizerSequenceNativeReference = "Heif/Av1/Conformance/libaom-av1-1-b10-00-quantizer-63-libaom.y4m"; + public const string Av1OfficialMinimumFrameSizeSequence = "Heif/Av1/Conformance/libaom-av1-1-b8-01-size-196x196.ivf"; + public const string Av1OfficialMinimumFrameSizeSequenceNativeReference = "Heif/Av1/Conformance/libaom-av1-1-b8-01-size-196x196-libaom.y4m"; + public const string Av1OfficialMinimumWidthMaximumHeightSequence = "Heif/Av1/Conformance/libaom-av1-1-b8-01-size-196x226.ivf"; + public const string Av1OfficialMinimumWidthMaximumHeightSequenceNativeReference = "Heif/Av1/Conformance/libaom-av1-1-b8-01-size-196x226-libaom.y4m"; + public const string Av1OfficialMaximumWidthMinimumHeightSequence = "Heif/Av1/Conformance/libaom-av1-1-b8-01-size-226x196.ivf"; + public const string Av1OfficialMaximumWidthMinimumHeightSequenceNativeReference = "Heif/Av1/Conformance/libaom-av1-1-b8-01-size-226x196-libaom.y4m"; + public const string Av1OfficialMaximumFrameSizeSequence = "Heif/Av1/Conformance/libaom-av1-1-b8-01-size-226x226.ivf"; + public const string Av1OfficialMaximumFrameSizeSequenceNativeReference = "Heif/Av1/Conformance/libaom-av1-1-b8-01-size-226x226-libaom.y4m"; + public const string Av1FrameIdentifierSequence = "Heif/Av1/Conformance/libaom-frame-id-196x196-8b.ivf"; + public const string Av1FrameIdentifierSequenceNativeReference = "Heif/Av1/Conformance/libaom-frame-id-196x196-8b-libaom.y4m"; + public const string Av1OfficialCdfUpdateSequence = "Heif/Av1/Conformance/libaom-av1-1-b8-04-cdfupdate.ivf"; + public const string Av1OfficialCdfUpdateSequenceNativeReference = "Heif/Av1/Conformance/libaom-av1-1-b8-04-cdfupdate-libaom.y4m"; + public const string Av1OfficialMotionVectorSequence = "Heif/Av1/Conformance/libaom-av1-1-b8-05-mv.ivf"; + public const string Av1OfficialMotionVectorSequenceNativeReference = "Heif/Av1/Conformance/libaom-av1-1-b8-05-mv-libaom.y4m"; + public const string Av1OfficialMotionFieldSequence = "Heif/Av1/Conformance/libaom-av1-1-b8-06-mfmv.ivf"; + public const string Av1OfficialMotionFieldSequenceNativeReference = "Heif/Av1/Conformance/libaom-av1-1-b8-06-mfmv-libaom.y4m"; + public const string Av1OfficialIntraBlockCopySequence = "Heif/Av1/Conformance/libaom-av1-1-b8-16-intra_only-intrabc-extreme-dv.ivf"; + public const string Av1OfficialIntraBlockCopySequenceNativeReference = "Heif/Av1/Conformance/libaom-av1-1-b8-16-intra_only-intrabc-extreme-dv-libaom.y4m"; + public const string Av1OfficialTwoSpatialLayerSequence = "Heif/Av1/Conformance/libaom-av1-1-b8-22-svc-L2T1.ivf"; + public const string Av1OfficialTwoSpatialLayerSequenceNativeReference = "Heif/Av1/Conformance/libaom-av1-1-b8-22-svc-L2T1-libaom.y4m"; + public const string Av1OfficialTwoTemporalLayerSequence = "Heif/Av1/Conformance/libaom-av1-1-b8-22-svc-L1T2.ivf"; + public const string Av1OfficialTwoTemporalLayerSequenceNativeReference = "Heif/Av1/Conformance/libaom-av1-1-b8-22-svc-L1T2-libaom.y4m"; + public const string Av1OfficialSpatialTemporalLayerSequence = "Heif/Av1/Conformance/libaom-av1-1-b8-22-svc-L2T2.ivf"; + public const string Av1OfficialSpatialTemporalLayerSequenceNativeReference = "Heif/Av1/Conformance/libaom-av1-1-b8-22-svc-L2T2-libaom.y4m"; + public const string Av1OfficialFilmGrainSequence = "Heif/Av1/Conformance/libaom-av1-1-b8-23-film-grain-50.ivf"; + public const string Av1OfficialFilmGrainSequenceNativeReference = "Heif/Av1/Conformance/libaom-av1-1-b8-23-film-grain-50-libaom.y4m"; + public const string Av1OfficialTenBitFilmGrainSequence = "Heif/Av1/Conformance/libaom-av1-1-b10-23-film-grain-50.ivf"; + public const string Av1OfficialTenBitFilmGrainSequenceNativeReference = "Heif/Av1/Conformance/libaom-av1-1-b10-23-film-grain-50-libaom.y4m"; + public const string Av1OfficialMonochromeSequence = "Heif/Av1/Conformance/libaom-av1-1-b8-24-monochrome.ivf"; + public const string Av1OfficialMonochromeSequenceNativeReference = "Heif/Av1/Conformance/libaom-av1-1-b8-24-monochrome-libaom.yuv"; + public const string Av1OfficialTenBitMonochromeSequence = "Heif/Av1/Conformance/libaom-av1-1-b10-24-monochrome.ivf"; + public const string Av1OfficialTenBitMonochromeSequenceNativeReference = "Heif/Av1/Conformance/libaom-av1-1-b10-24-monochrome-libaom.y4m"; + public const string Av1Cdef8BitPayload = "Heif/Av1/Conformance/libaom-cdef-kodim23-8b.bit"; + public const string Av1Cdef8BitReference = "Heif/Av1/Conformance/libaom-cdef-kodim23-8b-libaom.yuv"; + public const string Av1Cdef8BitAvif = "Heif/Av1/Conformance/libavif-cdef-kodim23-8b.avif"; + public const string Av1Cdef10BitPayload = "Heif/Av1/Conformance/libaom-cdef-cosmos-10b.bit"; + public const string Av1Cdef10BitReference = "Heif/Av1/Conformance/libaom-cdef-cosmos-10b-libaom.yuv"; + public const string Av1Cdef10BitAvif = "Heif/Av1/Conformance/libavif-cdef-cosmos-10b.avif"; + public const string Av1Cdef12BitPayload = "Heif/Av1/Conformance/libaom-cdef-cosmos-12b.bit"; + public const string Av1Cdef12BitReference = "Heif/Av1/Conformance/libaom-cdef-cosmos-12b-libaom.yuv"; + public const string Av1Cdef12BitAvif = "Heif/Av1/Conformance/libavif-cdef-cosmos-12b.avif"; + public const string Av1Profile8BitMonochromeAvif = "Heif/Av1/Conformance/libavif-profile-8b-400.avif"; + public const string Av1Profile8BitMonochromeReference = "Heif/Av1/Conformance/libavif-profile-8b-400-libaom-y4m.yuv"; + public const string Av1Profile8Bit420Avif = "Heif/Av1/Conformance/libavif-profile-8b-420.avif"; + public const string Av1Profile8Bit420Reference = "Heif/Av1/Conformance/libavif-profile-8b-420-libaom-y4m.yuv"; + public const string Av1Profile8Bit422Avif = "Heif/Av1/Conformance/libavif-profile-8b-422.avif"; + public const string Av1Profile8Bit422Reference = "Heif/Av1/Conformance/libavif-profile-8b-422-libaom-y4m.yuv"; + public const string Av1Profile8Bit444Avif = "Heif/Av1/Conformance/libavif-profile-8b-444.avif"; + public const string Av1Profile8Bit444Reference = "Heif/Av1/Conformance/libavif-profile-8b-444-libaom-y4m.yuv"; + public const string Av1Profile10BitMonochromeAvif = "Heif/Av1/Conformance/libavif-profile-10b-400.avif"; + public const string Av1Profile10BitMonochromeReference = "Heif/Av1/Conformance/libavif-profile-10b-400-libaom-y4m.yuv"; + public const string Av1Profile10Bit420Avif = "Heif/Av1/Conformance/libavif-profile-10b-420.avif"; + public const string Av1Profile10Bit420Reference = "Heif/Av1/Conformance/libavif-profile-10b-420-libaom-y4m.yuv"; + public const string Av1Profile10Bit422Avif = "Heif/Av1/Conformance/libavif-profile-10b-422.avif"; + public const string Av1Profile10Bit422Reference = "Heif/Av1/Conformance/libavif-profile-10b-422-libaom-y4m.yuv"; + public const string Av1Profile10Bit444Avif = "Heif/Av1/Conformance/libavif-profile-10b-444.avif"; + public const string Av1Profile10Bit444Reference = "Heif/Av1/Conformance/libavif-profile-10b-444-libaom-y4m.yuv"; + public const string Av1Profile12BitMonochromeAvif = "Heif/Av1/Conformance/libavif-profile-12b-400.avif"; + public const string Av1Profile12BitMonochromeReference = "Heif/Av1/Conformance/libavif-profile-12b-400-libaom-y4m.yuv"; + public const string Av1Profile12Bit420Avif = "Heif/Av1/Conformance/libavif-profile-12b-420.avif"; + public const string Av1Profile12Bit420Reference = "Heif/Av1/Conformance/libavif-profile-12b-420-libaom-y4m.yuv"; + public const string Av1Profile12Bit422Avif = "Heif/Av1/Conformance/libavif-profile-12b-422.avif"; + public const string Av1Profile12Bit422Reference = "Heif/Av1/Conformance/libavif-profile-12b-422-libaom-y4m.yuv"; + public const string Av1Profile12Bit444Avif = "Heif/Av1/Conformance/libavif-profile-12b-444.avif"; + public const string Av1Profile12Bit444Reference = "Heif/Av1/Conformance/libavif-profile-12b-444-libaom-y4m.yuv"; + public const string Av1Palette8BitPayload = "Heif/Av1/Conformance/libaom-palette-draw-points-8b-444.bit"; + public const string Av1Palette8BitReference = "Heif/Av1/Conformance/libaom-palette-draw-points-8b-444-libaom.yuv"; + public const string Av1Palette8BitAvif = "Heif/Av1/Conformance/libavif-palette-draw-points-8b.avif"; + public const string Av1IntraBlockCopy8BitAvif = "Heif/Av1/Conformance/libavif-intrabc-abc-8b-444.avif"; + public const string Av1IntraBlockCopy8BitReference = "Heif/Av1/Conformance/libavif-intrabc-abc-8b-444-libaom-y4m.yuv"; + public const string Av1IntraBlockCopy10BitAvif = "Heif/Av1/Conformance/libavif-intrabc-abc-10b-444.avif"; + public const string Av1IntraBlockCopy10BitReference = "Heif/Av1/Conformance/libavif-intrabc-abc-10b-444-libaom-y4m.yuv"; + public const string Av1IntraBlockCopy12BitAvif = "Heif/Av1/Conformance/libavif-intrabc-abc-12b-444.avif"; + public const string Av1IntraBlockCopy12BitReference = "Heif/Av1/Conformance/libavif-intrabc-abc-12b-444-libaom-y4m.yuv"; + public const string Av1Lossless8BitAvif = "Heif/Av1/Conformance/libavif-lossless-circle-8b-444.avif"; + public const string Av1Lossless8BitReference = "Heif/Av1/Conformance/libavif-lossless-circle-8b-444-libaom.yuv"; + public const string Av1Lossless10BitAvif = "Heif/Av1/Conformance/libavif-lossless-circle-10b-444.avif"; + public const string Av1Lossless10BitReference = "Heif/Av1/Conformance/libavif-lossless-circle-10b-444-libaom.yuv"; + public const string Av1Lossless12BitAvif = "Heif/Av1/Conformance/libavif-lossless-circle-12b-444.avif"; + public const string Av1Lossless12BitReference = "Heif/Av1/Conformance/libavif-lossless-circle-12b-444-libaom.yuv"; + public const string Av1SuperResolution8BitPayload = "Heif/Av1/Conformance/libaom-superres-kodim23-8b.bit"; + public const string Av1SuperResolution8BitReference = "Heif/Av1/Conformance/libaom-superres-kodim23-8b-libaom.yuv"; + public const string Av1SuperResolution8BitAvif = "Heif/Av1/Conformance/libavif-superres-kodim23-8b.avif"; + public const string Av1SuperResolution10BitPayload = "Heif/Av1/Conformance/libaom-superres-cosmos-10b.bit"; + public const string Av1SuperResolution10BitReference = "Heif/Av1/Conformance/libaom-superres-cosmos-10b-libaom.yuv"; + public const string Av1SuperResolution10BitAvif = "Heif/Av1/Conformance/libavif-superres-cosmos-10b.avif"; + public const string Av1SuperResolution12BitPayload = "Heif/Av1/Conformance/libaom-superres-cosmos-12b.bit"; + public const string Av1SuperResolution12BitReference = "Heif/Av1/Conformance/libaom-superres-cosmos-12b-libaom.yuv"; + public const string Av1SuperResolution12BitAvif = "Heif/Av1/Conformance/libavif-superres-cosmos-12b.avif"; + public const string Av1Restoration8BitPayload = "Heif/Av1/Conformance/libaom-restoration-kodim23-8b.bit"; + public const string Av1Restoration8BitReference = "Heif/Av1/Conformance/libaom-restoration-kodim23-8b-libaom.yuv"; + public const string Av1Restoration8BitAvif = "Heif/Av1/Conformance/libavif-restoration-kodim23-8b.avif"; + public const string Av1Restoration10BitPayload = "Heif/Av1/Conformance/libaom-restoration-cosmos-10b.bit"; + public const string Av1Restoration10BitReference = "Heif/Av1/Conformance/libaom-restoration-cosmos-10b-libaom.yuv"; + public const string Av1Restoration10BitAvif = "Heif/Av1/Conformance/libavif-restoration-cosmos-10b.avif"; + public const string Av1Restoration12BitPayload = "Heif/Av1/Conformance/libaom-restoration-cosmos-12b.bit"; + public const string Av1Restoration12BitReference = "Heif/Av1/Conformance/libaom-restoration-cosmos-12b-libaom.yuv"; + public const string Av1Restoration12BitAvif = "Heif/Av1/Conformance/libavif-restoration-cosmos-12b.avif"; + public const string Av1RestorationSuperResolution8BitPayload = "Heif/Av1/Conformance/libaom-restoration-superres-kodim23-8b.bit"; + public const string Av1RestorationSuperResolution8BitReference = "Heif/Av1/Conformance/libaom-restoration-superres-kodim23-8b-libaom.yuv"; + public const string Av1RestorationSuperResolution10BitPayload = "Heif/Av1/Conformance/libaom-restoration-superres-abc-10b-422.bit"; + public const string Av1RestorationSuperResolution10BitReference = "Heif/Av1/Conformance/libaom-restoration-superres-abc-10b-422-libaom.yuv"; + public const string Av1RestorationSuperResolution12BitPayload = "Heif/Av1/Conformance/libaom-restoration-superres-cosmos-12b.bit"; + public const string Av1RestorationSuperResolution12BitReference = "Heif/Av1/Conformance/libaom-restoration-superres-cosmos-12b-libaom.yuv"; + public const string Av1FilmGrain8BitPayload = "Heif/Av1/Conformance/libaom-film-grain-circle-8b-420.bit"; + public const string Av1FilmGrain8BitReference = "Heif/Av1/Conformance/libaom-film-grain-circle-8b-420-libaom.yuv"; + public const string Av1FilmGrain10BitPayload = "Heif/Av1/Conformance/libaom-film-grain-circle-10b-422.bit"; + public const string Av1FilmGrain10BitReference = "Heif/Av1/Conformance/libaom-film-grain-circle-10b-422-libaom.yuv"; + public const string Av1FilmGrain12BitPayload = "Heif/Av1/Conformance/libaom-film-grain-circle-12b-444.bit"; + public const string Av1FilmGrain12BitReference = "Heif/Av1/Conformance/libaom-film-grain-circle-12b-444-libaom.yuv"; + public const string Av1FilmGrain8BitRestrictedPayload = "Heif/Av1/Conformance/libaom-film-grain-circle-8b-420-limited.bit"; + public const string Av1FilmGrain8BitRestrictedReference = "Heif/Av1/Conformance/libaom-film-grain-circle-8b-420-limited-libaom.yuv"; + public const string Av1FilmGrain8BitMonochromePayload = "Heif/Av1/Conformance/libaom-film-grain-circle-8b-400-limited.bit"; + public const string Av1FilmGrain8BitMonochromeReference = "Heif/Av1/Conformance/libaom-film-grain-circle-8b-400-limited-libaom.yuv"; + public const string Av1FilmGrain12BitIdentityPayload = "Heif/Av1/Conformance/libaom-film-grain-circle-12b-444-identity-limited.bit"; + public const string Av1FilmGrain12BitIdentityReference = "Heif/Av1/Conformance/libaom-film-grain-circle-12b-444-identity-limited-libaom.yuv"; + public const string Av1FilmGrainOddDimensionsPayload = "Heif/Av1/Conformance/libaom-film-grain-draw-points-8b-420-odd.bit"; + public const string Av1FilmGrainOddDimensionsReference = "Heif/Av1/Conformance/libaom-film-grain-draw-points-8b-420-odd-libaom.yuv"; + } + public static class Ico { public const string Flutter = "Icon/flutter.ico"; diff --git a/tests/ImageSharp.Tests/TestUtilities/TestEnvironment.Formats.cs b/tests/ImageSharp.Tests/TestUtilities/TestEnvironment.Formats.cs index 8a40bd6720..ba2a2f3797 100644 --- a/tests/ImageSharp.Tests/TestUtilities/TestEnvironment.Formats.cs +++ b/tests/ImageSharp.Tests/TestUtilities/TestEnvironment.Formats.cs @@ -5,6 +5,7 @@ using SixLabors.ImageSharp.Formats.Bmp; using SixLabors.ImageSharp.Formats.Exr; using SixLabors.ImageSharp.Formats.Gif; +using SixLabors.ImageSharp.Formats.Heif; using SixLabors.ImageSharp.Formats.Jpeg; using SixLabors.ImageSharp.Formats.Pbm; using SixLabors.ImageSharp.Formats.Png; @@ -60,6 +61,7 @@ private static Configuration CreateDefaultConfiguration() Configuration cfg = new( new JpegConfigurationModule(), new GifConfigurationModule(), + new HeifConfigurationModule(), new PbmConfigurationModule(), new TgaConfigurationModule(), new WebpConfigurationModule(), diff --git a/tests/ImageSharp.Tests/TestUtilities/TestMemoryAllocator.cs b/tests/ImageSharp.Tests/TestUtilities/TestMemoryAllocator.cs index 6672249124..099be25570 100644 --- a/tests/ImageSharp.Tests/TestUtilities/TestMemoryAllocator.cs +++ b/tests/ImageSharp.Tests/TestUtilities/TestMemoryAllocator.cs @@ -3,6 +3,7 @@ using System.Buffers; using System.Numerics; +using System.Runtime.CompilerServices; using System.Runtime.InteropServices; using SixLabors.ImageSharp.Memory; @@ -12,6 +13,7 @@ internal class TestMemoryAllocator : MemoryAllocator { private List allocationLog; private List returnLog; + private int nextAllocationId; public TestMemoryAllocator(byte dirtyValue = 42) { @@ -39,15 +41,16 @@ public void EnableNonThreadSafeLogging() protected override AllocationTrackedMemoryManager AllocateCore(int length, AllocationOptions options = AllocationOptions.None) { - T[] array = this.AllocateArray(length, options); - return new BasicArrayBuffer(array, length, this); + int allocationId = ++this.nextAllocationId; + T[] array = this.AllocateArray(length, options, allocationId); + return new BasicArrayBuffer(array, length, this, allocationId); } - private T[] AllocateArray(int length, AllocationOptions options) + private T[] AllocateArray(int length, AllocationOptions options, int allocationId) where T : struct { T[] array = new T[length + 42]; - this.allocationLog?.Add(AllocationRequest.Create(options, length, array)); + this.allocationLog?.Add(AllocationRequest.Create(options, length, array, allocationId)); if (options == AllocationOptions.None) { @@ -61,17 +64,24 @@ private T[] AllocateArray(int length, AllocationOptions options) private void Return(BasicArrayBuffer buffer) where T : struct { - this.returnLog?.Add(new ReturnRequest(buffer.Array.GetHashCode())); + this.returnLog?.Add(new ReturnRequest(buffer.AllocationId, buffer.Array.GetHashCode())); } public struct AllocationRequest { - private AllocationRequest(Type elementType, AllocationOptions allocationOptions, int length, int lengthInBytes, int hashCodeOfBuffer) + private AllocationRequest( + Type elementType, + AllocationOptions allocationOptions, + int length, + int lengthInBytes, + int allocationId, + int hashCodeOfBuffer) { this.ElementType = elementType; this.AllocationOptions = allocationOptions; this.Length = length; this.LengthInBytes = lengthInBytes; + this.AllocationId = allocationId; this.HashCodeOfBuffer = hashCodeOfBuffer; if (elementType == typeof(Vector4)) @@ -79,11 +89,11 @@ private AllocationRequest(Type elementType, AllocationOptions allocationOptions, } } - public static AllocationRequest Create(AllocationOptions allocationOptions, int length, T[] buffer) + public static AllocationRequest Create(AllocationOptions allocationOptions, int length, T[] buffer, int allocationId) { Type type = typeof(T); - int elementSize = Marshal.SizeOf(type); - return new AllocationRequest(type, allocationOptions, length, length * elementSize, buffer.GetHashCode()); + int elementSize = Unsafe.SizeOf(); + return new AllocationRequest(type, allocationOptions, length, length * elementSize, allocationId, buffer.GetHashCode()); } public Type ElementType { get; } @@ -94,16 +104,21 @@ public static AllocationRequest Create(AllocationOptions allocationOptions, i public int LengthInBytes { get; } + public int AllocationId { get; } + public int HashCodeOfBuffer { get; } } public struct ReturnRequest { - public ReturnRequest(int hashCodeOfBuffer) + public ReturnRequest(int allocationId, int hashCodeOfBuffer) { + this.AllocationId = allocationId; this.HashCodeOfBuffer = hashCodeOfBuffer; } + public int AllocationId { get; } + public int HashCodeOfBuffer { get; } } @@ -116,16 +131,17 @@ private class BasicArrayBuffer : AllocationTrackedMemoryManager private readonly TestMemoryAllocator allocator; private GCHandle pinHandle; - public BasicArrayBuffer(T[] array, int length, TestMemoryAllocator allocator) + public BasicArrayBuffer(T[] array, int length, TestMemoryAllocator allocator, int allocationId) { this.allocator = allocator; DebugGuard.MustBeLessThanOrEqualTo(length, array.Length, nameof(length)); this.Array = array; this.Length = length; + this.AllocationId = allocationId; } public BasicArrayBuffer(T[] array, TestMemoryAllocator allocator) - : this(array, array.Length, allocator) + : this(array, array.Length, allocator, 0) { } @@ -139,6 +155,11 @@ public BasicArrayBuffer(T[] array, TestMemoryAllocator allocator) /// public int Length { get; } + /// + /// Gets the stable identity recorded for this allocation. + /// + public int AllocationId { get; } + /// public override Span GetSpan() => this.Array.AsSpan(0, this.Length); diff --git a/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeCdefMatchesReference_Rgba32_libavif-cdef-cosmos-10b.png b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeCdefMatchesReference_Rgba32_libavif-cdef-cosmos-10b.png new file mode 100644 index 0000000000..4a018c986a --- /dev/null +++ b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeCdefMatchesReference_Rgba32_libavif-cdef-cosmos-10b.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3a70c1dfd57fb1464719c2c0140a99503e7818c50036acf3d152a7d71ff5a03e +size 473983 diff --git a/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeCdefMatchesReference_Rgba32_libavif-cdef-cosmos-12b.png b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeCdefMatchesReference_Rgba32_libavif-cdef-cosmos-12b.png new file mode 100644 index 0000000000..e944926abc --- /dev/null +++ b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeCdefMatchesReference_Rgba32_libavif-cdef-cosmos-12b.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:19a4e29fa8df389dc8d5120a6278036154e412ea2ca86fef83eb7ef0efa0a27a +size 420742 diff --git a/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeCdefMatchesReference_Rgba32_libavif-cdef-kodim23-8b.png b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeCdefMatchesReference_Rgba32_libavif-cdef-kodim23-8b.png new file mode 100644 index 0000000000..7ab2e6b201 --- /dev/null +++ b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeCdefMatchesReference_Rgba32_libavif-cdef-kodim23-8b.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:19f3e0d5357df5dacf16dc73501e5bbe280f04e0f0e0831bfc8f2511497dccf4 +size 408944 diff --git a/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeEqualAverageCompoundMatchesReference_Rgba32_libavif-webp-logo-average-compound.png b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeEqualAverageCompoundMatchesReference_Rgba32_libavif-webp-logo-average-compound.png new file mode 100644 index 0000000000..046e57854b --- /dev/null +++ b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeEqualAverageCompoundMatchesReference_Rgba32_libavif-webp-logo-average-compound.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bcfabc1e1c7e17d8ecb40569849a04ffac6ca1fcdf613f217b33816ca47337ac +size 4728 diff --git a/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeGlobalWarpMatchesReference_Rgba32_libavif-rotating-grid-global-warp.png b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeGlobalWarpMatchesReference_Rgba32_libavif-rotating-grid-global-warp.png new file mode 100644 index 0000000000..edbbc15eb2 --- /dev/null +++ b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeGlobalWarpMatchesReference_Rgba32_libavif-rotating-grid-global-warp.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f7d27abf79450dfa311f72106fd1da80997eabc0937f2f5578ef627119ff83b0 +size 6101 diff --git a/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeIntraBlockCopyMatchesReference_Rgba32_libavif-intrabc-abc-10b-444.png b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeIntraBlockCopyMatchesReference_Rgba32_libavif-intrabc-abc-10b-444.png new file mode 100644 index 0000000000..18a7d748dc --- /dev/null +++ b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeIntraBlockCopyMatchesReference_Rgba32_libavif-intrabc-abc-10b-444.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1c8025d05ee94764db4dc52bf2b19a4b2a9478c7cb33bd5ad57ad5414417efe0 +size 6433 diff --git a/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeIntraBlockCopyMatchesReference_Rgba32_libavif-intrabc-abc-12b-444.png b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeIntraBlockCopyMatchesReference_Rgba32_libavif-intrabc-abc-12b-444.png new file mode 100644 index 0000000000..18a7d748dc --- /dev/null +++ b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeIntraBlockCopyMatchesReference_Rgba32_libavif-intrabc-abc-12b-444.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1c8025d05ee94764db4dc52bf2b19a4b2a9478c7cb33bd5ad57ad5414417efe0 +size 6433 diff --git a/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeIntraBlockCopyMatchesReference_Rgba32_libavif-intrabc-abc-8b-444.png b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeIntraBlockCopyMatchesReference_Rgba32_libavif-intrabc-abc-8b-444.png new file mode 100644 index 0000000000..4f1d508cc6 --- /dev/null +++ b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeIntraBlockCopyMatchesReference_Rgba32_libavif-intrabc-abc-8b-444.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4cd476ce81e54eed777e4218f9b0ad38cc509d60d1c38bb0ba2ccc013d4ee053 +size 6462 diff --git a/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeLocalWarpMatchesReference_Rgba32_libavif-rotating-grid-local-warp.png b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeLocalWarpMatchesReference_Rgba32_libavif-rotating-grid-local-warp.png new file mode 100644 index 0000000000..a9473e9c78 --- /dev/null +++ b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeLocalWarpMatchesReference_Rgba32_libavif-rotating-grid-local-warp.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4490d62fb6679378e92caca48427359091ad2106be49fc1a3848f78be03beeb1 +size 10084 diff --git a/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeLoopRestorationMatchesReference_Rgba32_libavif-restoration-cosmos-10b.png b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeLoopRestorationMatchesReference_Rgba32_libavif-restoration-cosmos-10b.png new file mode 100644 index 0000000000..309de9c1ab --- /dev/null +++ b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeLoopRestorationMatchesReference_Rgba32_libavif-restoration-cosmos-10b.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:24ff2fe2ab28f044945df7b35d4ed9213ae7e054e33b45fb92e4924566f3e944 +size 454939 diff --git a/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeLoopRestorationMatchesReference_Rgba32_libavif-restoration-cosmos-12b.png b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeLoopRestorationMatchesReference_Rgba32_libavif-restoration-cosmos-12b.png new file mode 100644 index 0000000000..5cdc002e5f --- /dev/null +++ b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeLoopRestorationMatchesReference_Rgba32_libavif-restoration-cosmos-12b.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4835eabb73ca6e51db7f6779a76499a9f2d07274116b816487681a51883f074f +size 436999 diff --git a/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeLoopRestorationMatchesReference_Rgba32_libavif-restoration-kodim23-8b.png b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeLoopRestorationMatchesReference_Rgba32_libavif-restoration-kodim23-8b.png new file mode 100644 index 0000000000..f457ec6dbb --- /dev/null +++ b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeLoopRestorationMatchesReference_Rgba32_libavif-restoration-kodim23-8b.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2098b78efb22b27473802e4e9e074d4948eda6d6bb06510069c9497e63f37bf5 +size 383051 diff --git a/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeLosslessMatchesReference_Rgba32_libavif-lossless-circle-10b-444.png b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeLosslessMatchesReference_Rgba32_libavif-lossless-circle-10b-444.png new file mode 100644 index 0000000000..4190c0bfe1 --- /dev/null +++ b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeLosslessMatchesReference_Rgba32_libavif-lossless-circle-10b-444.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f4ef1f4a117431ac32f887af4db38d4a90a910ddc5c120f8a716a4934a766c67 +size 1128 diff --git a/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeLosslessMatchesReference_Rgba32_libavif-lossless-circle-12b-444.png b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeLosslessMatchesReference_Rgba32_libavif-lossless-circle-12b-444.png new file mode 100644 index 0000000000..208184b7eb --- /dev/null +++ b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeLosslessMatchesReference_Rgba32_libavif-lossless-circle-12b-444.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:624d6bbc99ad1fb9a4bd658959f102c94aeeaab054d86901274832b0f8a12a95 +size 1152 diff --git a/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeLosslessMatchesReference_Rgba32_libavif-lossless-circle-8b-444.png b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeLosslessMatchesReference_Rgba32_libavif-lossless-circle-8b-444.png new file mode 100644 index 0000000000..53ef7ef638 --- /dev/null +++ b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeLosslessMatchesReference_Rgba32_libavif-lossless-circle-8b-444.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a9b2d70a65299c6da81c3b495c54e1eab99c1932e44add57cff2ae00c1f36072 +size 834 diff --git a/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeObmcMatchesReference_Rgba32_libavif-webp-logo-obmc.png b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeObmcMatchesReference_Rgba32_libavif-webp-logo-obmc.png new file mode 100644 index 0000000000..c94b799dc6 --- /dev/null +++ b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeObmcMatchesReference_Rgba32_libavif-webp-logo-obmc.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d2cb388c9092ef17c4f0382c0150dd30d6f9d0ee247ff45ab5d7d4d312ceb23c +size 4544 diff --git a/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodePalettePresentationMatchesReference_Rgba32_libavif-palette-draw-points-8b.png b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodePalettePresentationMatchesReference_Rgba32_libavif-palette-draw-points-8b.png new file mode 100644 index 0000000000..d140ba6fdb --- /dev/null +++ b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodePalettePresentationMatchesReference_Rgba32_libavif-palette-draw-points-8b.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1148ebf6aa4b0f2d069d5e9b9605f6fb2a315e525f18016cdcae23efdd81da84 +size 157 diff --git a/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeProfileMatrixMatchesReference_Rgba32_libavif-profile-10b-400.png b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeProfileMatrixMatchesReference_Rgba32_libavif-profile-10b-400.png new file mode 100644 index 0000000000..25a8163e49 --- /dev/null +++ b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeProfileMatrixMatchesReference_Rgba32_libavif-profile-10b-400.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5ed0fd0f21b2c5620b7b6289cf3ebdd6001e8498ce7b0daec17d5657051f9cea +size 5224 diff --git a/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeProfileMatrixMatchesReference_Rgba32_libavif-profile-10b-420.png b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeProfileMatrixMatchesReference_Rgba32_libavif-profile-10b-420.png new file mode 100644 index 0000000000..30a54877bb --- /dev/null +++ b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeProfileMatrixMatchesReference_Rgba32_libavif-profile-10b-420.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:649c9fef1d87858cc6b49b7fcf51ad691ea22f0a513cd69ab61a3bf1943dd1bf +size 9635 diff --git a/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeProfileMatrixMatchesReference_Rgba32_libavif-profile-10b-422.png b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeProfileMatrixMatchesReference_Rgba32_libavif-profile-10b-422.png new file mode 100644 index 0000000000..90ecec2d0d --- /dev/null +++ b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeProfileMatrixMatchesReference_Rgba32_libavif-profile-10b-422.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4c1a70c907c1040fcb7b2a6618fb0f9c84a4ddfdf441238138f9f3349fb79696 +size 9573 diff --git a/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeProfileMatrixMatchesReference_Rgba32_libavif-profile-10b-444.png b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeProfileMatrixMatchesReference_Rgba32_libavif-profile-10b-444.png new file mode 100644 index 0000000000..30a54877bb --- /dev/null +++ b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeProfileMatrixMatchesReference_Rgba32_libavif-profile-10b-444.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:649c9fef1d87858cc6b49b7fcf51ad691ea22f0a513cd69ab61a3bf1943dd1bf +size 9635 diff --git a/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeProfileMatrixMatchesReference_Rgba32_libavif-profile-12b-400.png b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeProfileMatrixMatchesReference_Rgba32_libavif-profile-12b-400.png new file mode 100644 index 0000000000..2c50dec739 --- /dev/null +++ b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeProfileMatrixMatchesReference_Rgba32_libavif-profile-12b-400.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:53c0c2e538c85323fceee54d30b7ba29d2af5fd9770a617faceadb9f2a502b2c +size 5459 diff --git a/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeProfileMatrixMatchesReference_Rgba32_libavif-profile-12b-420.png b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeProfileMatrixMatchesReference_Rgba32_libavif-profile-12b-420.png new file mode 100644 index 0000000000..b3b82c26c1 --- /dev/null +++ b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeProfileMatrixMatchesReference_Rgba32_libavif-profile-12b-420.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:589a61003f368ce59defd0e2334b5d50bfa24119e488bea19c56542bcfc4e930 +size 10122 diff --git a/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeProfileMatrixMatchesReference_Rgba32_libavif-profile-12b-422.png b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeProfileMatrixMatchesReference_Rgba32_libavif-profile-12b-422.png new file mode 100644 index 0000000000..d06bbba677 --- /dev/null +++ b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeProfileMatrixMatchesReference_Rgba32_libavif-profile-12b-422.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4c86a0e37069641c3e077428f2a0632081ef3b8fe997e10721347377ad98d14f +size 10142 diff --git a/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeProfileMatrixMatchesReference_Rgba32_libavif-profile-12b-444.png b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeProfileMatrixMatchesReference_Rgba32_libavif-profile-12b-444.png new file mode 100644 index 0000000000..5658d5ae11 --- /dev/null +++ b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeProfileMatrixMatchesReference_Rgba32_libavif-profile-12b-444.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:13b3e95816e3b98016fc7b8c11e161a3204e0dcf9a38008254d1468ec7e73769 +size 9942 diff --git a/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeProfileMatrixMatchesReference_Rgba32_libavif-profile-8b-400.png b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeProfileMatrixMatchesReference_Rgba32_libavif-profile-8b-400.png new file mode 100644 index 0000000000..e55b20c57f --- /dev/null +++ b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeProfileMatrixMatchesReference_Rgba32_libavif-profile-8b-400.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:260a860d1f01e56c0154a31a6f4802fd27b5f507f724d55084f5278b388003c2 +size 5294 diff --git a/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeProfileMatrixMatchesReference_Rgba32_libavif-profile-8b-420.png b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeProfileMatrixMatchesReference_Rgba32_libavif-profile-8b-420.png new file mode 100644 index 0000000000..fae45e031a --- /dev/null +++ b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeProfileMatrixMatchesReference_Rgba32_libavif-profile-8b-420.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2b39c31c012ad7e194870fcdd074a83f225dc0dfc40731f0c72fa8847821e0fd +size 9631 diff --git a/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeProfileMatrixMatchesReference_Rgba32_libavif-profile-8b-422.png b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeProfileMatrixMatchesReference_Rgba32_libavif-profile-8b-422.png new file mode 100644 index 0000000000..abdbafc004 --- /dev/null +++ b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeProfileMatrixMatchesReference_Rgba32_libavif-profile-8b-422.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:253291bc0bcebde6cc0ef6729c52ec025cba4a7ed046a3231297da7c63df3f4a +size 9748 diff --git a/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeProfileMatrixMatchesReference_Rgba32_libavif-profile-8b-444.png b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeProfileMatrixMatchesReference_Rgba32_libavif-profile-8b-444.png new file mode 100644 index 0000000000..3e0b2f65d6 --- /dev/null +++ b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeProfileMatrixMatchesReference_Rgba32_libavif-profile-8b-444.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:96423c89e68f6d4e6a349e0693b1c0b106055654681d51c822ab43fd12b86223 +size 9703 diff --git a/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeProgressiveSingleMatchesReference_Rgba32_libavif-progressive-draw-points-8b.png b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeProgressiveSingleMatchesReference_Rgba32_libavif-progressive-draw-points-8b.png new file mode 100644 index 0000000000..4e753f21b5 --- /dev/null +++ b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeProgressiveSingleMatchesReference_Rgba32_libavif-progressive-draw-points-8b.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0758c17dc36e38aee9f4389a335c2bf332ab91e4c79d7b0b22994fddd0fd1605 +size 186 diff --git a/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeScaledReferenceMatchesReference_Rgba32_libavif-webp-logo-scaled-reference.png b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeScaledReferenceMatchesReference_Rgba32_libavif-webp-logo-scaled-reference.png new file mode 100644 index 0000000000..5ac780b8aa --- /dev/null +++ b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeScaledReferenceMatchesReference_Rgba32_libavif-webp-logo-scaled-reference.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:dc4c6dbe6bd92c5fce1e3e23700afa603ef04ed02edd336213ebba1e3bd84ba0 +size 5759 diff --git a/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeSelectableCompoundMatchesReference_Rgba32_libavif-webp-logo-difference-weighted-compound.png b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeSelectableCompoundMatchesReference_Rgba32_libavif-webp-logo-difference-weighted-compound.png new file mode 100644 index 0000000000..c94b799dc6 --- /dev/null +++ b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeSelectableCompoundMatchesReference_Rgba32_libavif-webp-logo-difference-weighted-compound.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d2cb388c9092ef17c4f0382c0150dd30d6f9d0ee247ff45ab5d7d4d312ceb23c +size 4544 diff --git a/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeSelectableCompoundMatchesReference_Rgba32_libavif-webp-logo-distance-weighted-compound.png b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeSelectableCompoundMatchesReference_Rgba32_libavif-webp-logo-distance-weighted-compound.png new file mode 100644 index 0000000000..c94b799dc6 --- /dev/null +++ b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeSelectableCompoundMatchesReference_Rgba32_libavif-webp-logo-distance-weighted-compound.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d2cb388c9092ef17c4f0382c0150dd30d6f9d0ee247ff45ab5d7d4d312ceb23c +size 4544 diff --git a/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeSelectableCompoundMatchesReference_Rgba32_libavif-webp-logo-inter-intra.png b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeSelectableCompoundMatchesReference_Rgba32_libavif-webp-logo-inter-intra.png new file mode 100644 index 0000000000..9b72f7f9c1 --- /dev/null +++ b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeSelectableCompoundMatchesReference_Rgba32_libavif-webp-logo-inter-intra.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f0de4ccdfb6d95a400e69b69fa4c57f0beeeee75825722c31613385f0b3fd9fc +size 4551 diff --git a/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeSelectableCompoundMatchesReference_Rgba32_libavif-webp-logo-wedge-compound.png b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeSelectableCompoundMatchesReference_Rgba32_libavif-webp-logo-wedge-compound.png new file mode 100644 index 0000000000..c94b799dc6 --- /dev/null +++ b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeSelectableCompoundMatchesReference_Rgba32_libavif-webp-logo-wedge-compound.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d2cb388c9092ef17c4f0382c0150dd30d6f9d0ee247ff45ab5d7d4d312ceb23c +size 4544 diff --git a/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeSelectedProgressiveLayerMatchesReference_Rgba32_libavif-webp-logo-scaled-reference-lsel0.png b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeSelectedProgressiveLayerMatchesReference_Rgba32_libavif-webp-logo-scaled-reference-lsel0.png new file mode 100644 index 0000000000..1b0c1e4769 --- /dev/null +++ b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeSelectedProgressiveLayerMatchesReference_Rgba32_libavif-webp-logo-scaled-reference-lsel0.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:678c5e5d4650ea6f0c590302e7db9e3c6608851bc577453da4a6837bdb4d3af3 +size 6453 diff --git a/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeSuperResolutionMatchesReference_Rgba32_libavif-superres-cosmos-10b.png b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeSuperResolutionMatchesReference_Rgba32_libavif-superres-cosmos-10b.png new file mode 100644 index 0000000000..a8bb1d0399 --- /dev/null +++ b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeSuperResolutionMatchesReference_Rgba32_libavif-superres-cosmos-10b.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8a6b5185f4ae89a886c02cd4cf1419815cd8f8ab7f298a1ee3103cefe3ac54c0 +size 429957 diff --git a/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeSuperResolutionMatchesReference_Rgba32_libavif-superres-cosmos-12b.png b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeSuperResolutionMatchesReference_Rgba32_libavif-superres-cosmos-12b.png new file mode 100644 index 0000000000..5c6e3d0bf9 --- /dev/null +++ b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeSuperResolutionMatchesReference_Rgba32_libavif-superres-cosmos-12b.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d9951d83761717ed6652217d14eec411a922b78c0668db9c9fe441be0a5438fa +size 412629 diff --git a/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeSuperResolutionMatchesReference_Rgba32_libavif-superres-kodim23-8b.png b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeSuperResolutionMatchesReference_Rgba32_libavif-superres-kodim23-8b.png new file mode 100644 index 0000000000..1d5e6b5146 --- /dev/null +++ b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeSuperResolutionMatchesReference_Rgba32_libavif-superres-kodim23-8b.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:cd995bb10a9c10eba89584ea49c6a1c853b0168aa3fac324d3d935d9f985851f +size 383407 diff --git a/tests/Images/Input/Heif/Av1/Conformance/README.md b/tests/Images/Input/Heif/Av1/Conformance/README.md new file mode 100644 index 0000000000..a57ca19527 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/README.md @@ -0,0 +1,401 @@ +# AV1 reconstruction conformance fixtures + +These fixtures provide independent reference output for AV1 reconstruction and AVIF presentation tests. ImageSharp output is compared exactly with the retained native YUV planes and presented PNG files; the tests do not use a tolerance. Native fixtures remain beside their inputs here, while the PNG files use the repository reference-output naming contract under `tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests`. + +## Provenance + +The source images and original AVIF files come from `libavif/tests/data` at commit `062e582e8afda88e6baf988fdcf046a801efa0f5`. Their licenses are recorded in libavif's `tests/data/README.md` and continue to apply to the derived fixtures. This includes the unrestricted Kodak image, the CC BY 3.0 Cosmos Laundromat frame, and files distributed under libavif's BSD-2-Clause license. + +Original fixture-generation records identify libavif 1.4.2 from commit +`062e582e8afda88e6baf988fdcf046a801efa0f5` as the container tool. This does not pin the current +verification checkout. Current AV1 algorithm, arithmetic, syntax, and native-output verification uses only +the clean official libaom `main` checkout. Libavif commands record container and presentation provenance +only and are not used as an AV1 implementation reference. + +## File conventions + +- `.avif` files exercise the complete container and presentation path. +- `.bit` files contain the exact AV1 elementary-stream payload used by reconstruction tests. +- `-libaom.yuv` files contain headerless planar Y, U, and V reference samples. Samples above eight bits are stored as little-endian 16-bit values. +- `-libaom-y4m.yuv` files retain the Y4M header together with the native planar frame. +- `-libaom.y4m` files retain the Y4M header together with the native sequence frames selected for comparison. +- Reference-output `.png` files contain retained eight-bit RGBA presentations. Their names combine the public test method, `Rgba32`, and the input AVIF basename so `CompareToReferenceOutput` resolves them directly. + +## Coverage + +| Fixture family | Coverage | +| --- | --- | +| `libavif-kodim23`, `libavif-cosmos1650`, `libaom-cosmos1650` | Baseline 8-, 10-, and 12-bit reconstruction, chroma subsampling, and active deblocking | +| `*-cdef-*` | Active CDEF with loop restoration disabled | +| `*-superres-*` | Active horizontal super-resolution with CDEF and restoration disabled | +| `*-restoration-*` | Wiener and self-guided loop restoration | +| `*-restoration-superres-*` | Restoration after super-resolution, including 10-bit 4:2:2 clipped-edge transform coverage | +| `libavif-profile-*` | The 8-, 10-, and 12-bit matrix across monochrome, 4:2:0, 4:2:2, and 4:4:4 | +| `*-palette-*` | Luma and chroma palette prediction | +| `*-intrabc-*` | Intra-block copy at every supported bit depth | +| `*-lossless-*` | Lossless quantization, reversible transforms, and exact presentation | +| `*-film-grain-*` | Full and restricted range, monochrome, identity matrix, 8/10/12-bit synthesis, overlap, and odd frame dimensions | +| `libaom-av1-1-b8-00-quantizer-*`, `libaom-av1-1-b10-00-quantizer-*` | Official minimum- and maximum-quantizer dependent-frame reconstruction | +| `libaom-av1-1-b8-01-size-*` | Official frame-size matrix corner reconstruction | +| `libaom-frame-id-196x196-8b` | Key/inter frame-identifier signaling and retained-reference validation | +| `libaom-av1-1-b10-23`, `libaom-av1-1-b10-24` | Official ten-bit dependent-frame film grain and monochrome sequence reconstruction | +| `libavif-progressive-draw-points-8b` | A real two-layer color item whose final frame uses single-reference inter reconstruction, plus its progressive auxiliary alpha item | +| `libavif-webp-logo-average-compound` | A 19-frame YUV444 image sequence whose retained references reach equal-weight compound inter reconstruction | +| `libavif-webp-logo-distance-weighted-compound` | Selectable distance-weighted compound prediction | +| `libavif-webp-logo-wedge-compound` | Wedge compound prediction with both signaled mask orientations | +| `libavif-webp-logo-difference-weighted-compound` | Difference-weighted compound prediction with both mask orientations | +| `libavif-webp-logo-inter-intra` | Smooth and wedge inter-intra prediction | +| `libavif-webp-logo-obmc` | Overlapping motion compensation through a 19-frame dependent sequence | +| `libavif-webp-logo-scaled-reference` | A 40x40 retained reference scaled into an 80x80 dependent frame | +| `libavif-rotating-grid-local-warp` | Multi-sample local affine projection and warped prediction through a two-frame dependent sequence | +| `libavif-rotating-grid-global-warp` | Non-translational rotation/zoom GLOBALMV prediction through a two-frame dependent sequence | + +The corresponding tests also assert the syntax required by each family before comparing output. This prevents an inactive tool or an incorrectly substituted stream from passing solely because its final pixels happen to match. + +## Frame identifier fixture + +On 2026-08-31 current official libaom `main` at observed revision +`441c439b9916474cac15d2822af47a9ad70674a8` encoded the two 196x196 YUV420 frames from the retained +minimum-frame-size native reference with error resilience and frame identifiers enabled: + +```text +aomenc --ivf --passes=1 --limit=2 --threads=1 --row-mt=0 --lag-in-frames=0 --auto-alt-ref=0 --error-resilient=1 --kf-min-dist=9999 --kf-max-dist=9999 --lossless=1 --cpu-used=6 -o libaom-frame-id-196x196-8b.ivf libaom-av1-1-b8-01-size-196x196-libaom.y4m +aomdec --threads=1 --row-mt=0 --output-bit-depth=8 --all-layers --output=libaom-frame-id-196x196-8b-libaom.y4m libaom-frame-id-196x196-8b.ivf +``` + +The generated native reference contains both decoded frames. Every Y, Cb, and Cr sample matches the source +frames exactly. The production test additionally requires an enabled frame-identifier sequence header and a +validated identifier change on the dependent inter frame. + +## Baseline deblocking fixtures + +On 2026-08-31 current official libaom `main` at observed revision +`441c439b9916474cac15d2822af47a9ad70674a8` decoded the baseline 8-, 10-, and 12-bit +elementary streams using one thread, row threading disabled, raw output, and the corresponding native +output depth: + +```text +aomdec --codec=av1 --threads=1 --row-mt=0 --rawvideo --output-bit-depth=8 -o deblocking-8b.yuv libavif-kodim23-8b.bit +aomdec --codec=av1 --threads=1 --row-mt=0 --rawvideo --output-bit-depth=10 -o deblocking-10b.yuv libavif-cosmos1650-10b.bit +aomdec --codec=av1 --threads=1 --row-mt=0 --rawvideo --output-bit-depth=12 -o deblocking-12b.yuv libaom-cosmos1650-12b.bit +``` + +The payloads contain 20,750, 37,169, and 23,769 bytes. Their SHA-256 values are +`B7B1D3F85A870475ACF579FBB7A0B59FF94C30F84B3F3A066411A50F9BA1BD20`, +`F930BF11EB2F61BF4EE0FE61853CD387A4C3DBE53503707DB7B2C59D9D273FF3`, and +`A45F9653255C1A660906554BAFCC13FDEE04F88CC6D0E08E1EFA169CF0C6DDD2`. The generated +native outputs contain 589,824, 2,629,632, and 2,629,632 bytes and match the retained references exactly. +Their SHA-256 values are +`8DDE2EEC742C39F0579C29AE84CBA0FE01522A9008ADCB2CFFCCEC0295D18141`, +`9A59DD92A0C579F942ACCA8281EBD0465DC848BE200A4D2FF57EAFF589445F6C`, and +`EF712BE32AF7CF0A95C5C41BDCC51AFC05A4AB7C047383F5F65EDAD2BB986712`. + +The focused production test covers active 8-, 10-, and 12-bit deblocking. A separate current-main +dependent sequence exercises inter, intra, and skipped-inter blocks with reference/mode deltas enabled, +compares native reconstruction exactly, compares final presentation through the established reference-output +API, and repeats under constrained tracked allocation. The direct production filter test independently +distinguishes global and non-global mode-delta classes, LAST and GOLDEN reference deltas, internal transform +edges, and skipped prediction-unit boundaries. + +## Official ten-bit sequence fixtures + +The `libaom-av1-1-b10-23-film-grain-50.ivf` and `libaom-av1-1-b10-24-monochrome.ivf` streams are the official files from libaom's test-data bucket. Their SHA-1 values are `2F883C7E11C21A31F79BD9C809541BE90B0C7C4A` and `03A8D002594CCC51932332002BB6F9837EF46D0F`, exactly matching the current official libaom `main` manifest. Their SHA-256 values are `C36CF5AB6A2E9E27C212C06863759B60791E3FA681A0800B5D57FD4192EF29CB` and `6A1B0729305A167F10737A5375F0570139F055BCD7916DF260B653AB2210ADC1`. + +The retained native references were generated from the generic libaom build with: + +```text +aomdec --threads=1 --output=libaom-av1-1-b10-23-film-grain-50-libaom.y4m libaom-av1-1-b10-23-film-grain-50.ivf +aomdec --threads=1 --output=libaom-av1-1-b10-24-monochrome-libaom.y4m libaom-av1-1-b10-24-monochrome.ivf +``` + +The film-grain Y4M SHA-256 is `A1B553BE140F48ABDDB2A6D39917AB714BA03AC7FFD6359EAA1CB0D89C985A3B`, and the monochrome Y4M SHA-256 is `7394BC8146485D200BFDEC62E170482F8B1A85A64D21FAC62D10116FB1BB140D`. The tests decode and compare all ten frames from each sequence exactly under normal and scalar dispatch. The film-grain stream retains dependent ungrained references while applying ten-bit grain to each displayed 352x288 YUV420 frame; the monochrome stream verifies every 320x180 ten-bit luma sample without manufacturing chroma in ImageSharp. Both sequences also run through a constrained tracked allocator. + +## Official quantizer-boundary fixtures + +The retained `quantizer-00` and `quantizer-63` streams are the minimum- and maximum-quantizer boundaries from libaom's official eight- and ten-bit test matrices. Their SHA-1 values are `C2E1EC9936B95254187A359E94AA32A9F3DAD1B7`, `2A8AA33513D8E01AE9410C4BF5FE1E471B775482`, `9BBE8499796AA588FF02E313FB0D4349940D2FEA`, and `8B6EB3FFF2E0DB7EAC775B08C745250CA591E2D9`, exactly matching the current official libaom `main` manifest. Their SHA-256 values, in the same order, are `6382DBD2BEFBBC93D4EA283586F4FB43FEA5F1C52400E3D2C5281A46B1104C00`, `0E4EC80680F7AF8DE9621B016E0F2D7C0858B2951DEBC173DDA50C6A051547D3`, `FE6053CE4EE20A1C0EC6F7FE35DB097E92AD25D8A3505598BD89162C74D7944F`, and `39759AB77483E1D11049DC38B5F5262158FD9C3CBC9D1F82A02462FC5DF30E0C`. + +The native references were originally generated with the historical generic `aomdec --threads=1` build. On 2026-08-31 current official libaom `main` at observed revision `441c439b9916474cac15d2822af47a9ad70674a8` reproduced all four references byte for byte. Their SHA-256 values are `D499028E0606DB70CD56A72F151E04F36C09F300A448CCCD8430DD920D3589C5`, `4CC9892B3EE3399B293E31014B9F566C21E0C7A4765FC5F444528769C33E6D67`, `78373C28F401EB95D3E563D146622ED6C714ED96661E5E57C539CE71D7BED599`, and `A9DF86F671B8CF01EFC130660556412D4EBAF31A81D6F26FBDAEB0A7E839D8EA`. Each reference's two raw-frame MD5 values also match the corresponding official `.ivf.md5` file exactly. The tests compare every native sample under normal and scalar `FeatureTestRunner` dispatch and run all four sequences through a 2,560-byte row-aligned constrained tracked allocator. + +## Palette reconstruction fixture + +The 42-byte `libaom-palette-draw-points-8b-444.bit` payload has SHA-256 +`F412A9E7F19D1C009D0329B993BB503D74CCDA58505C54BAE8FB3C16142181DC`. On 2026-08-31 +current official libaom `main` at observed revision `441c439b9916474cac15d2822af47a9ad70674a8` +decoded it with one thread, row threading disabled, raw output, and eight-bit output depth. The resulting +1,089-byte YUV444 output matches the retained native reference exactly at SHA-256 +`E05F7C0DF06ECCF0E43869D1D7B03DAA1D635ACD26A766F8940899BE18D53251`. + +The production tests require active luma and chroma palette syntax, compare every native sample under +`FeatureTestRunner`, and compare the final AVIF presentation through the established reference-output API. +The retained PNG has SHA-256 +`1148EBF6AA4B0F2D069D5E9B9605F6FB2A315E525F18016CDCAE23EFDD81DA84`. A 1 KiB +constrained tracked allocator forces both frame-owned palette map surfaces across multiple memory groups; +the test verifies exact reconstruction and exactly one return for every recorded allocation. + +## Official all-intra fixture + +On 2026-08-31 current official libaom `main` at observed revision +`441c439b9916474cac15d2822af47a9ad70674a8` reproduced the retained 39-frame all-intra Y4M byte for +byte. The IVF SHA-256 is `5FCD265FD9F9BDD0D3179340B4C4532F1422CA5E5D97741C7481B84CB5DC122F`; +the native reference SHA-256 is +`1211EBEFBC9CCEF9ED19BE4CCE3F807D69FFFE338E95CCA1B5F4CA8023482175`. The production test +decodes all 39 frames in one decoder session, compares every native sample exactly, and requires coverage +of every intra prediction mode and all seven transform types selected by the fixture. + +## Adaptive reference-state fixtures + +On 2026-08-31 current official libaom `main` at observed revision +`441c439b9916474cac15d2822af47a9ad70674a8` decoded the `cdfupdate`, `mfmv`, `svc-L2T1`, +`svc-L1T2`, and `svc-L2T2` IVF streams with one thread, row threading disabled, and eight-bit output +depth. Every generated Y4M matches its retained native reference byte for byte. Their output SHA-256 +values, in that order, are: + +- `4FBFF73FF0DE2D9084DAE557D1D4BD677B0486516525BF4D327D2D795D5A7779`; +- `F7DB607694818C19E62FD9A27F53E1A3E2D00B72C39C0430C1B26399CC76777D`; +- `7A427631ECBF144F435AA4612F1201415FB1A9BCF9A67BA010AEF830B0C3AB81`; +- `4012DE2D4AFD095E7BB68EAE18B50B0674781BB4971CECABC0E5471E63373ED3`; +- `1ABB981CFF76BA9557DA437B258D8A95FCA755DED8E3949D857E8388AB1D6AE3`. + +The production tests compare every native sample under `FeatureTestRunner`. They require both adaptive +tile and frame-end CDF updates, active projected reference motion vectors, and the default operating point +of each spatial or temporal layer fixture. The motion-field and layer sequences also run through constrained +tracked allocators. Direct ownership tests cover initialization, retained-slot aliases, allocation-failure +unwinding, presentation ownership, decoder-result ownership, and exactly-once final disposal. + +## Official frame-size corner fixtures + +The retained 196x196, 196x226, 226x196, and 226x226 streams are the four corners of libaom's official eight-bit frame-size matrix. Their SHA-1 values are `9F386D19C87DBFD6AC84A06D2393DD88863AC003`, `5525F7E312EC073F480ED5A2BE5BDC4F0CE51A09`, `1A57B913443B267F4A31A6925C39F5B58022F550`, and `40DD208EB525CD90D7C0674CF787097FB909AFAE`, exactly matching the current official libaom `main` manifest. Their SHA-256 values, in the same order, are `ECACF9C2065EEC1A02248395412BE5E03ED7E3FEDFD6653622E09E73D1CAC747`, `2A2ECDDE60546FA8280039228B4BE541825EFAEE30F53EE50B91B11F2952BDA7`, `999522FBF3FCD9F8CC8DAC865BEF874B7BF655A4DAD93C397F50A7A1EC2C9027`, and `B5D9A30F24E33F8FA6A655961645650542DC2545FB7D557E2D55A043DAC69F50`. + +The native references were generated with the pinned generic `aomdec --threads=1` build. Their SHA-256 values are `4479030861DD9D6AB9B00FA8CF77A34712BCECA06C31131927D3E7E9BF5DDA70`, `817FF76E70946763C000E19FCCD9F0258CF9358201B6771AA68FB8C84490D26F`, `F4A70BA358E8B4ED558B589BFE56354C9552469FDABD01B3F4D5754334B0CBD9`, and `44FC32FB1D24CE4A33830B67D927F038E0E1379C6B480C598F018382EC78E9CE`. Each reference's two raw-frame MD5 values also match the corresponding official `.ivf.md5` file exactly. The tests compare every native sample under normal and scalar `FeatureTestRunner` dispatch and run all four sequences through a 1 KiB constrained tracked allocator. + +## Progressive dependent-frame fixture + +The `libavif-progressive-draw-points-8b.avif` fixture is the unmodified `tests/data/draw_points_idat_progressive.avif` file from the pinned libavif tree. Its SHA-256 is `077AB2AD1E46DD912A973E4F024CB1EB242A08298BE2DBF1A52A058E88C48A4A`. It was generated with: + +```text +./avifenc -q 100 --progressive ../tests/data/draw_points.png ../tests/data/draw_points_idat_progressive.avif +``` + +The primary color item's `a1lx` property divides its logical 72-byte AV1 payload into a 55-byte base layer and a 17-byte dependent layer. The container stores those layers in separate `iloc` extents at AVIF offsets 511 and 583. The `.bit` fixture concatenates those two logical color extents; it does not copy the physically adjacent auxiliary-alpha extent between them. + +Current official libaom `main` decodes the corrected logical payload into two 33x11 YUV444 frames. Both +frames' 1,089 color samples match the corresponding first three planes of the retained YUV444-alpha +outputs exactly. The alpha and PNG record the original fixture's presentation provenance; they are not AV1 +implementation references. The production-path test selects the second native frame, requires inter-coded +blocks in the final ImageSharp frame, and compares both native color and final presentation without a +tolerance. + +## Equal-average compound fixture + +The `libavif-webp-logo-average-compound.avif` file is retained solely as interoperability input. It was +created from `tests/data/webp_logo_animated.y4m` with the following command; libavif is not used as an +AV1 implementation or reconstruction reference: + +```text +./avifenc -j 1 -c aom -s 4 -q 80 -a enable-dist-wtd-comp=0 -a enable-masked-comp=0 -a enable-interintra-comp=0 -a enable-obmc=0 -a enable-warped-motion=0 -a enable-global-motion=0 tests/data/webp_logo_animated.y4m libavif-webp-logo-average-compound.avif +``` + +On 2026-08-31 the clean official libaom `main` checkout was refreshed from its upstream remote. At the +observed revision `441c439b9916474cac15d2822af47a9ad70674a8`, current `aomdec` decoded the 5,465-byte +`mdat` payload at file offset 1,065 as 19 shown 80x80 YUV444 frames: + +```text +aomdec --codec=av1 --threads=1 --row-mt=0 --output-bit-depth=8 -o compound-current-main.y4m compound-current-main.obu +``` + +All 19 frames decoded successfully. The final frame's 19,200 native samples have SHA-256 +`E79D2F49C260B1AC9B1B9BBBB2D611126AFD3B241DA389EB9E7BD4EA0ED42080` and match the stored Y4M's +Y, U, and V samples exactly with zero differences. The observed revision records the source used for +this verification; it does not pin the libaom checkout. + +The production test decodes every preceding sample to establish the retained-reference state, requires +actual equal-average compound blocks, and compares the final native planes exactly. It then compares the +final RGBA output through ImageSharp's established reference-output API. The PNG is presentation evidence +only and is not used to establish AV1 reconstruction arithmetic. + +## Selectable compound and inter-intra fixtures + +The four AVIF files are retained solely as interoperability inputs. Their original creation commands +used the same `tests/data/webp_logo_animated.y4m` source and encoded at speed zero after disabling later +inter-mode checkpoints. These commands record container-fixture provenance only; they are not codec +implementation references. Each command also disables competing prediction tools that would prevent the +resulting stream from isolating its named mode: + +```text +./avifenc -j 1 -c aom -s 0 -q 80 -a enable-obmc=0 -a enable-warped-motion=0 -a enable-global-motion=0 -a enable-masked-comp=0 -a enable-interintra-comp=0 tests/data/webp_logo_animated.y4m libavif-webp-logo-distance-weighted-compound.avif +./avifenc -j 1 -c aom -s 0 -q 80 -a enable-obmc=0 -a enable-warped-motion=0 -a enable-global-motion=0 -a enable-dist-wtd-comp=0 -a enable-diff-wtd-comp=0 -a enable-interintra-comp=0 tests/data/webp_logo_animated.y4m libavif-webp-logo-wedge-compound.avif +./avifenc -j 1 -c aom -s 0 -q 80 -a enable-obmc=0 -a enable-warped-motion=0 -a enable-global-motion=0 -a enable-dist-wtd-comp=0 -a enable-interinter-wedge=0 -a enable-interintra-comp=0 tests/data/webp_logo_animated.y4m libavif-webp-logo-difference-weighted-compound.avif +./avifenc -j 1 -c aom -s 0 -q 80 -a enable-obmc=0 -a enable-warped-motion=0 -a enable-global-motion=0 -a enable-dist-wtd-comp=0 -a enable-masked-comp=0 tests/data/webp_logo_animated.y4m libavif-webp-logo-inter-intra.avif +``` + +| Fixture | AVIF SHA-256 | Retained frame-18 Y4M SHA-256 | Retained frame-18 PNG SHA-256 | +| --- | --- | --- | --- | +| `libavif-webp-logo-distance-weighted-compound` | `DA710D11C60F03EEA209E4360E2FC807B89C49AD50671F0DFB1BCF4AD5EF76DD` | `904D1B5B3E7F334CE8D44040F9A7BDCAC1F7773122FF1C5F06A5B4DD31A62A97` | `D2CB388C9092EF17C4F0382C0150DD30D6F9D0EE247FF45AB5D7D4D312CEB23C` | +| `libavif-webp-logo-wedge-compound` | `98640640A445055FEA3D9E2F4A78FEEF54E97F8171B472CF57D99156E1C553B2` | `904D1B5B3E7F334CE8D44040F9A7BDCAC1F7773122FF1C5F06A5B4DD31A62A97` | `D2CB388C9092EF17C4F0382C0150DD30D6F9D0EE247FF45AB5D7D4D312CEB23C` | +| `libavif-webp-logo-difference-weighted-compound` | `FC6459CD334762D74D9D2654640221E80C463CC01B82B29A5866C9E725ABE273` | `904D1B5B3E7F334CE8D44040F9A7BDCAC1F7773122FF1C5F06A5B4DD31A62A97` | `D2CB388C9092EF17C4F0382C0150DD30D6F9D0EE247FF45AB5D7D4D312CEB23C` | +| `libavif-webp-logo-inter-intra` | `71DF22E63626B5BC9001FF1E88076B90F11BB47D18089750853E66F0CBBB084B` | `502265688138641A7B12C8C4190B66C76CD4808D9AED05B06486056B39D7E9A0` | `F0DE4CCDFB6D95A400E69B69FA4C57F0BEEEEE75825722C31613385F0B3FD9FC` | + +The inter-intra fixture's 5,327-byte AV1 `mdat` payload was decoded directly with the current official +libaom `main` checkout observed at `441c439b9916474cac15d2822af47a9ad70674a8`, using one thread and +with row threading disabled. Current `aomdec` produced all 19 YUV444 frames. The final frame's 19,200 +native samples have SHA-256 `E8B776C2751DC30CA838931A4B74535FC6E681179568A1278747A38CFF2E5BFA` +and match the retained Y4M with zero differing samples. + +The distance-weighted fixture's 5,372-byte AV1 `mdat` payload was decoded under the same current-libaom +conditions. Current `aomdec` produced all 19 YUV444 frames. The final frame's 19,200 native samples have +SHA-256 `E8CAA650F1571C5B9CACAF8C06E1DDF5F5D2ED35F65F1C34377076C573425899` and match the retained +Y4M with zero differing samples. + +The wedge fixture's 5,374-byte AV1 `mdat` payload was decoded under the same current-libaom conditions. +Current `aomdec` produced all 19 YUV444 frames. The final frame's 19,200 native samples have SHA-256 +`E8CAA650F1571C5B9CACAF8C06E1DDF5F5D2ED35F65F1C34377076C573425899` and match the retained Y4M +with zero differing samples. + +The difference-weighted fixture's 5,358-byte AV1 `mdat` payload was decoded under the same current-libaom +conditions. Current `aomdec` produced all 19 YUV444 frames. The final frame's 19,200 native samples have +SHA-256 `E8CAA650F1571C5B9CACAF8C06E1DDF5F5D2ED35F65F1C34377076C573425899` and match the retained +Y4M with zero differing samples. + +The production tests independently require their decoded mode states. The distance-weighted input must +exercise distance-weighted compound prediction, the wedge and difference-weighted inputs must each +exercise both mask orientations, and the inter-intra input must exercise both smooth and wedge inter-intra +prediction. The tests decode all preceding samples, compare final native Y, Cb, and Cr planes exactly, +compare final RGBA presentation through ImageSharp's established reference-output API, and repeat +reconstruction with constrained tracked allocation. The retained PNG files are presentation evidence only +and are not AV1 reconstruction references. + +## Overlapping motion-compensation fixture + +The `libavif-webp-logo-obmc.avif` fixture is retained interoperability input. It was originally +packaged by libavif from the same `tests/data/webp_logo_animated.y4m` source as the compound fixtures, +with libaom's encoder selected and competing compound, inter-intra, warped, and global prediction modes +disabled: + +```text +./avifenc -j 1 -c aom -s 0 -q 80 -a max-reference-frames=3 -a enable-dist-wtd-comp=0 -a enable-masked-comp=0 -a enable-interintra-comp=0 -a enable-warped-motion=0 -a enable-global-motion=0 tests/data/webp_logo_animated.y4m libavif-webp-logo-obmc.avif +``` + +On 2026-08-31 the fixture's 5,387-byte AV1 `mdat` payload at AVIF offset 1,065 was decoded with +`aomdec` from a freshly updated clean checkout of current official libaom `main`. One decode thread +was used and row threading remained disabled. All 19 frames decoded. The final frame's 19,200 native +YUV444 samples have SHA-256 +`E8CAA650F1571C5B9CACAF8C06E1DDF5F5D2ED35F65F1C34377076C573425899` and match +`libavif-webp-logo-obmc-libaom.y4m` with zero differing samples. + +The production test requires decoded OBMC mode state, decodes every retained-reference dependency, +compares final Y, Cb, and Cr planes exactly, compares final RGBA presentation through ImageSharp's +established reference-output API under normal and scalar FeatureTestRunner dispatch, and repeats the +decode with constrained tracked allocation. Direct production-branch tests separately cover +above-then-left blending at 8/10/12-bit and 4:2:0 and 4:2:2 overlap geometry. The retained presentation +PNG's SHA-256 is +`D2CB388C9092EF17C4F0382C0150DD30D6F9D0EE247FF45AB5D7D4D312CEB23C`; its pixels were not changed +when its contract-derived filename was updated with the test name. + +## Scaled-reference fixture + +The `libavif-webp-logo-scaled-reference.avif` and +`libavif-webp-logo-scaled-reference-lsel0.avif` files are retained solely as interoperability inputs. +Their shared 2,195-byte `.bit` payload contains a 701-byte independent base layer followed by the +dependent layer; no container implementation is used as an AV1 arithmetic reference. + +On 2026-08-31 the payload was decoded with `aomdec` from the freshly updated clean checkout of current +official libaom `main`, observed at `441c439b9916474cac15d2822af47a9ad70674a8`: + +```text +aomdec --codec=av1 --threads=1 --row-mt=0 --all-layers --rawvideo --output-bit-depth=8 -o current-libaom-scaled-reference-all.yuv libavif-webp-logo-scaled-reference.bit +``` + +Current libaom produced a 40x40 YUV444 base frame and an 80x80 YUV444 dependent frame. The combined +24,000 native samples have SHA-256 +`DD219E41B52C6C9343A92CD0A2D451DF57B73B25F10124811675B4CB2F8D666F`. The base frame matches +`libavif-webp-logo-scaled-reference-base-libaom.yuv` exactly, and the dependent frame matches the native +planes in `libavif-webp-logo-scaled-reference-libaom.y4m` exactly, with zero differing samples. + +The production tests require the retained 40x40 base and the 80x80 dependent reconstruction, compare both +native frames exactly, compare the selected base layer and final RGBA presentation through ImageSharp's +established reference-output API, and repeat both layer selections with constrained tracked allocation. +The scaled-prediction FeatureTestRunner tests cover native and no-round compound output at 8, 10, and +12 bits, and a complete `Av1BlockDecoder.DecodeBlock()` test proves that scaled compound references remain +in the no-round intermediate domain until the final blend. + +The two presentation PNGs were renamed with their current-libaom test contracts without changing their +bytes. Their SHA-256 values remain +`DC4C6DBE6BD92C5FCE1E3E23700AFA603EF04ED02EDD336213EBBA1E3BD84BA0` and +`678C5E5D4650EA6F0C590302E7DB9E3C6608851BC577453DA4A6837BDB4D3AF3`. + +## Local warped-motion fixture + +The `libavif-rotating-grid-local-warp.avif` file is retained solely as interoperability input. It was +originally encoded from a deterministic two-frame 256x256 limited-range YUV444 source whose second frame +rotates the first by 2.5 degrees. The original packaging command records fixture provenance only; libavif +is not used as an AV1 implementation or reconstruction reference: + +```text +./avifenc -j 1 -s 0 -q 60 -a color:enable-warped-motion=1 -a color:enable-global-motion=0 -a color:enable-obmc=0 rotating-grid-256-two-frame.y4m libavif-rotating-grid-local-warp.avif +``` + +On 2026-08-31 the clean official libaom `main` checkout was refreshed from its upstream remote. At the +observed revision `441c439b9916474cac15d2822af47a9ad70674a8`, the fixture's 2,310-byte AV1 `mdat` +payload at AVIF offset 997 has SHA-256 +`644D04FE1D1A32BB7A3856AD7EB49CF1EFDE0AC845E55BEAC4170F72353F2391` and was decoded with: + +```text +aomdec --codec=av1 --threads=1 --row-mt=0 --all-layers --output-bit-depth=8 -o local-warp-current-main.y4m local-warp-current-main.obu +``` + +Current libaom produced two 256x256 YUV444 frames. The complete Y4M has SHA-256 +`8FDC5D46014F5E5A7455A83643AB6F0DA66FC5A984E72A43F8C75BAD8271C299`. Its final +frame's 196,608 native samples have SHA-256 +`47B2AB39BF3B9DA15C1EC59840F964DFDF227760947F6E1295FB38A84555F75C` and match the +retained `libavif-rotating-grid-local-warp-libaom.y4m` reference with zero differing samples. + +The multi-sample model at mode-information row 6, column 8 derives matrix +`[-191565, 599107, 61755, -140, -6909, 62012]` and reduced shear +`[-3776, -128, -7360, -3520]` from four retained spatial neighbor samples. The production test requires +decoded `WARPED_CAUSAL` state through `Av1BlockDecoder.DecodeBlock()`, compares every final native Y, +U, and V sample exactly, compares the retained final presentation through the established image-reference +API, runs intrinsic and scalar dispatch through `FeatureTestRunner`, and repeats reconstruction with a +constrained tracked allocator. Separate 8-, 10-, and 12-bit operator tests use an independent scalar +transcription of current libaom's affine loops and all 1,544 current filter coefficients across AVX-512, +AVX, 128-bit, and scalar execution. + +## Global warped-motion fixture + +The `libavif-rotating-grid-global-warp.avif` file is retained solely as interoperability input. It uses +the same deterministic two-frame 256x256 limited-range YUV444 source as the local-warp fixture. The original +packaging command records fixture provenance only; libavif is not used as an AV1 implementation or +reconstruction reference: + +```text +./avifenc -j 1 -s 0 -q 100 -a color:enable-warped-motion=0 -a color:enable-global-motion=1 -a color:enable-obmc=0 rotating-grid-256-two-frame.y4m libavif-rotating-grid-global-warp.avif +``` + +On 2026-08-31 the clean official libaom `main` checkout was refreshed from its upstream remote. At the +observed revision `441c439b9916474cac15d2822af47a9ad70674a8`, the fixture's 38,475-byte AV1 +`mdat` payload at AVIF offset 997 has SHA-256 +`6AC7EC9984B1FF5C00403D7E3858441E9CEE75128F7414101D06DEEE59A351D0` and was decoded with: + +```text +aomdec --codec=av1 --threads=1 --row-mt=0 --all-layers --output-bit-depth=8 -o global-warp-current-main.y4m global-warp-current-main.obu +``` + +Current libaom produced two 256x256 YUV444 frames. The complete Y4M has SHA-256 +`84754DE0B9FABC4F3F8F344C848183EC17B625BFD87E4519C3D8AD7DEFD20F2C`. Its final +frame's 196,608 native samples have SHA-256 +`FEC89E2DE7496980389806B194425042F3800C7BAA817249D1A51D44A2B37A8E` and match the +retained `libavif-rotating-grid-global-warp-libaom.y4m` reference with zero differing samples. + +The decoded stream contains non-translational `GLOBALMV` blocks using rotation/zoom matrix +`[-357376, 372736, 65468, 2856, -2856, 65468]` and reduced shear +`[-64, 2880, -2880, 64]`. The production sequence test requires that decoded model and mode state, +compares every final native Y, U, and V sample exactly, compares the retained final presentation through +the established image-reference API, runs intrinsic and scalar dispatch through `FeatureTestRunner`, and +repeats reconstruction with constrained tracked allocation. A direct `Av1BlockDecoder.DecodeBlock()` +test drives both references of `GLOBAL_GLOBALMV` through matrix prediction and no-round compound averaging +at 8, 10, and 12 bits. High-bit-depth warped references remain in current libaom's unsigned +`CONV_BUF_TYPE` domain until the final blend; 12-bit prediction uses the corresponding adjusted round0 and +two-bit final rounding. + +## Updating fixtures + +Do not create conformance references with ImageSharp. Generate both the native-plane and presentation references with an independent decoder, record the exact upstream revisions and source license, and preserve exact comparisons. A new tool-specific fixture should demonstrate that the relevant syntax is active and should be no larger than required to cover that behavior. diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b10-00-quantizer-00-libaom.y4m b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b10-00-quantizer-00-libaom.y4m new file mode 100644 index 0000000000..99b2109e9c --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b10-00-quantizer-00-libaom.y4m @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:78373c28f401eb95d3e563d146622ed6c714ed96661e5e57c539ce71d7bed599 +size 1382463 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b10-00-quantizer-00.ivf b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b10-00-quantizer-00.ivf new file mode 100644 index 0000000000..f377dd42ff --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b10-00-quantizer-00.ivf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fe6053ce4ee20a1c0ec6f7fe35db097e92ad25d8a3505598bd89162c74d7944f +size 416797 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b10-00-quantizer-63-libaom.y4m b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b10-00-quantizer-63-libaom.y4m new file mode 100644 index 0000000000..98c134b9f2 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b10-00-quantizer-63-libaom.y4m @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a9df86f671b8cf01efc130660556412d4ebaf31a81d6f26fbdaeb0a7e839d8ea +size 1382463 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b10-00-quantizer-63.ivf b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b10-00-quantizer-63.ivf new file mode 100644 index 0000000000..fad7c11ac8 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b10-00-quantizer-63.ivf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:39759ab77483e1d11049dc38b5f5262158fd9c3cbc9d1f82a02462fc5df30e0c +size 1243 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b10-23-film-grain-50-libaom.y4m b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b10-23-film-grain-50-libaom.y4m new file mode 100644 index 0000000000..1a1dfd0036 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b10-23-film-grain-50-libaom.y4m @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a1b553be140f48abddb2a6d39917ab714ba03ac7ffd6359eaa1cb0d89c985a3b +size 3041391 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b10-23-film-grain-50.ivf b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b10-23-film-grain-50.ivf new file mode 100644 index 0000000000..5294eb9dc1 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b10-23-film-grain-50.ivf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c36cf5ab6a2e9e27c212c06863759b60791e3fa681a0800b5d57fd4192ef29cb +size 28906 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b10-24-monochrome-libaom.y4m b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b10-24-monochrome-libaom.y4m new file mode 100644 index 0000000000..b20fdcb28f --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b10-24-monochrome-libaom.y4m @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7394bc8146485d200bfdec62e170482f8b1a85a64d21fac62d10116fb1bb140d +size 1152097 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b10-24-monochrome.ivf b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b10-24-monochrome.ivf new file mode 100644 index 0000000000..48229b74b0 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b10-24-monochrome.ivf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6a1b0729305a167f10737a5375f0570139f055bcd7916df260b653ab2210adc1 +size 16044 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-00-quantizer-00-libaom.y4m b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-00-quantizer-00-libaom.y4m new file mode 100644 index 0000000000..8743cda66e --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-00-quantizer-00-libaom.y4m @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d499028e0606db70cd56a72f151e04f36c09f300a448cccd8430dd920d3589c5 +size 304178 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-00-quantizer-00.ivf b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-00-quantizer-00.ivf new file mode 100644 index 0000000000..da9ba4be1c --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-00-quantizer-00.ivf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6382dbd2befbbc93d4ea283586f4fb43fea5f1c52400e3d2c5281a46b1104c00 +size 128943 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-00-quantizer-63-libaom.y4m b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-00-quantizer-63-libaom.y4m new file mode 100644 index 0000000000..677670a531 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-00-quantizer-63-libaom.y4m @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4cc9892b3ee3399b293e31014b9f566c21e0c7a4765fc5f444528769c33e6d67 +size 304178 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-00-quantizer-63.ivf b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-00-quantizer-63.ivf new file mode 100644 index 0000000000..c92c90284c --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-00-quantizer-63.ivf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0e4ec80680f7af8de9621b016e0f2d7c0858b2951debc173dda50c6a051547d3 +size 606 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-01-size-196x196-libaom.y4m b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-01-size-196x196-libaom.y4m new file mode 100644 index 0000000000..b33d9189f4 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-01-size-196x196-libaom.y4m @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4479030861dd9d6ab9b00fa8cf77a34712bceca06c31131927d3e7e9bf5dda70 +size 115298 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-01-size-196x196.ivf b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-01-size-196x196.ivf new file mode 100644 index 0000000000..cb60d56eaf --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-01-size-196x196.ivf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ecacf9c2065eec1a02248395412be5e03ed7e3fedfd6653622e09e73d1cac747 +size 821 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-01-size-196x226-libaom.y4m b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-01-size-196x226-libaom.y4m new file mode 100644 index 0000000000..06e4af8f63 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-01-size-196x226-libaom.y4m @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:817ff76e70946763c000e19fccd9f0258cf9358201b6771aa68fb8c84490d26f +size 132938 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-01-size-196x226.ivf b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-01-size-196x226.ivf new file mode 100644 index 0000000000..048e34ffc1 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-01-size-196x226.ivf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2a2ecdde60546fa8280039228b4be541825efaee30f53ee50b91b11f2952bda7 +size 841 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-01-size-226x196-libaom.y4m b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-01-size-226x196-libaom.y4m new file mode 100644 index 0000000000..4ada371413 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-01-size-226x196-libaom.y4m @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f4a70ba358e8b4ed558b589bfe56354c9552469fdabd01b3f4d5754334b0cbd9 +size 132938 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-01-size-226x196.ivf b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-01-size-226x196.ivf new file mode 100644 index 0000000000..a5404f55a8 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-01-size-226x196.ivf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:999522fbf3fcd9f8cc8dac865bef874b7bf655a4dad93c397f50a7a1ec2c9027 +size 1053 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-01-size-226x226-libaom.y4m b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-01-size-226x226-libaom.y4m new file mode 100644 index 0000000000..12d380aab0 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-01-size-226x226-libaom.y4m @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:44fc32fb1d24ce4a33830b67d927f038e0e1379c6b480c598f018382ec78e9ce +size 153278 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-01-size-226x226.ivf b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-01-size-226x226.ivf new file mode 100644 index 0000000000..19197890b0 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-01-size-226x226.ivf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b5d9a30f24e33f8fa6a655961645650542dc2545fb7d557e2d55a043dac69f50 +size 1079 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-02-allintra-libaom.y4m b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-02-allintra-libaom.y4m new file mode 100644 index 0000000000..c52d566940 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-02-allintra-libaom.y4m @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1211ebefbc9ccef9ed19be4cce3f807d69fffe338e95cca1b5f4ca8023482175 +size 5930767 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-02-allintra.ivf b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-02-allintra.ivf new file mode 100644 index 0000000000..20073178ed --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-02-allintra.ivf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5fcd265fd9f9bdd0d3179340b4c4532f1422ca5e5d97741c7481b84cb5dc122f +size 1488725 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-04-cdfupdate-libaom.y4m b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-04-cdfupdate-libaom.y4m new file mode 100644 index 0000000000..6624f077e0 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-04-cdfupdate-libaom.y4m @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4fbff73ff0de2d9084dae557d1d4bd677b0486516525bf4d327d2d795d5a7779 +size 304178 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-04-cdfupdate.ivf b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-04-cdfupdate.ivf new file mode 100644 index 0000000000..a1923b3725 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-04-cdfupdate.ivf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:14a3dbf537b6bf15efc003182d9916d61438c93624a8bd26e6e3ae7eaf33ea82 +size 36088 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-05-mv-libaom.y4m b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-05-mv-libaom.y4m new file mode 100644 index 0000000000..487c5e08c4 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-05-mv-libaom.y4m @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d97ac78c81782cf1507549368047769dc677dbe205706458d1ee9c807de6ec78 +size 608318 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-05-mv.ivf b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-05-mv.ivf new file mode 100644 index 0000000000..952608615b --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-05-mv.ivf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:222a9050059b254dab17cfb802ff829c778e3f93af18961a622c8268576c1395 +size 55223 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-06-mfmv-libaom.y4m b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-06-mfmv-libaom.y4m new file mode 100644 index 0000000000..30415cd73f --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-06-mfmv-libaom.y4m @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f7db607694818c19e62fd9a27f53e1a3e2d00b72c39c0430c1b26399cc76777d +size 608318 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-06-mfmv.ivf b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-06-mfmv.ivf new file mode 100644 index 0000000000..38d15c7e92 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-06-mfmv.ivf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b59bf9586d8546dfda81dfec4ee4e32ceb502c9d22412ab0b63a2abb534a1f14 +size 44964 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-16-intra_only-intrabc-extreme-dv-libaom.y4m b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-16-intra_only-intrabc-extreme-dv-libaom.y4m new file mode 100644 index 0000000000..2a5797cc27 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-16-intra_only-intrabc-extreme-dv-libaom.y4m @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8cf7851e112172c62799f8bfdf895505742a3ca77e339c12e547ad2c47af41b3 +size 6220852 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-16-intra_only-intrabc-extreme-dv.ivf b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-16-intra_only-intrabc-extreme-dv.ivf new file mode 100644 index 0000000000..26856e218f --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-16-intra_only-intrabc-extreme-dv.ivf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0252716fe7a7c7f1ebb1eb23920a6737f45c10dfba942304bd339ae42b5c576c +size 339274 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-22-svc-L1T2-libaom.y4m b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-22-svc-L1T2-libaom.y4m new file mode 100644 index 0000000000..070bd31716 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-22-svc-L1T2-libaom.y4m @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4012de2d4afd095e7bb68eae18b50b0674781bb4971cecabc0e5471e63373ed3 +size 2764886 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-22-svc-L1T2.ivf b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-22-svc-L1T2.ivf new file mode 100644 index 0000000000..eb6e5e1e26 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-22-svc-L1T2.ivf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:eade696ab60415d476e1d2a489ec2cccd24d5eaaadfa4559c6488b7932274c5e +size 48883 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-22-svc-L2T1-libaom.y4m b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-22-svc-L2T1-libaom.y4m new file mode 100644 index 0000000000..2753e5b371 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-22-svc-L2T1-libaom.y4m @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7a427631ecbf144f435aa4612f1201415fb1a9bcf9a67ba010aef830b0c3ab81 +size 11059287 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-22-svc-L2T1.ivf b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-22-svc-L2T1.ivf new file mode 100644 index 0000000000..7a5bbf50c5 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-22-svc-L2T1.ivf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d65824e8e7c19bc958db810d728b2132b513b5a93cb5f99ca3d7995e19a226e1 +size 159037 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-22-svc-L2T2-libaom.y4m b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-22-svc-L2T2-libaom.y4m new file mode 100644 index 0000000000..ed4fc5dd73 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-22-svc-L2T2-libaom.y4m @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1abb981cff76ba9557da437b258d8a95fca755ded8e3949d857e8388ab1d6ae3 +size 11059287 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-22-svc-L2T2.ivf b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-22-svc-L2T2.ivf new file mode 100644 index 0000000000..ed262689a1 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-22-svc-L2T2.ivf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:14257d5fd3901581dbeb88133378cd79d50ea32506adb729909ab56b4a52af3c +size 119907 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-23-film-grain-50-libaom.y4m b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-23-film-grain-50-libaom.y4m new file mode 100644 index 0000000000..e312db6b52 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-23-film-grain-50-libaom.y4m @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c71958de496a949d12e819190f240bd56010e5ad57d8a0d55903bf380a9e0c26 +size 1520738 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-23-film-grain-50.ivf b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-23-film-grain-50.ivf new file mode 100644 index 0000000000..e298d56c53 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-23-film-grain-50.ivf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ba3edd82a58414f009e1c821b947ec8de4e0420f33803ca7bfea39af6aeea155 +size 28634 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-24-monochrome-libaom.yuv b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-24-monochrome-libaom.yuv new file mode 100644 index 0000000000..fef4d46114 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-24-monochrome-libaom.yuv @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6a4f8a68366e801d60a14c00be6ea88a861b1b631d86def8a8db65efb6b06fa8 +size 576000 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-24-monochrome.ivf b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-24-monochrome.ivf new file mode 100644 index 0000000000..5737877378 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-24-monochrome.ivf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3dcc2a3c181f74788bd9575d94f0862d9e914415ac358a4da4ed7d904f0535b7 +size 10000 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-cdef-cosmos-10b-libaom.yuv b/tests/Images/Input/Heif/Av1/Conformance/libaom-cdef-cosmos-10b-libaom.yuv new file mode 100644 index 0000000000..4ec4135012 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-cdef-cosmos-10b-libaom.yuv @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a3e95419a967bb552cd6f80365490f195fdad645114004771d8bb2662ea8f446 +size 2629632 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-cdef-cosmos-10b.bit b/tests/Images/Input/Heif/Av1/Conformance/libaom-cdef-cosmos-10b.bit new file mode 100644 index 0000000000..264de911c1 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-cdef-cosmos-10b.bit @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:138a1b4a8e4a9d36115bcb6a0e4f21fee245e574e4f5b775d9205cce436c1727 +size 26733 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-cdef-cosmos-12b-libaom.yuv b/tests/Images/Input/Heif/Av1/Conformance/libaom-cdef-cosmos-12b-libaom.yuv new file mode 100644 index 0000000000..fdd7ccf021 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-cdef-cosmos-12b-libaom.yuv @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e0e246a511272743c1a3a28fd58f66d3b17e741e999143d65d23f4ea773407df +size 2629632 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-cdef-cosmos-12b.bit b/tests/Images/Input/Heif/Av1/Conformance/libaom-cdef-cosmos-12b.bit new file mode 100644 index 0000000000..bafc276c4a --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-cdef-cosmos-12b.bit @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:862c9e01e379204e348fe5dec62caffe4b0c9275416240b899a0459fe70de95e +size 23157 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-cdef-kodim23-8b-libaom.yuv b/tests/Images/Input/Heif/Av1/Conformance/libaom-cdef-kodim23-8b-libaom.yuv new file mode 100644 index 0000000000..c0ae6066d8 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-cdef-kodim23-8b-libaom.yuv @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c46b370f221dfdc2db1cffc824961e7594659c3e2634656813459d9f058c7ab1 +size 589824 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-cdef-kodim23-8b.bit b/tests/Images/Input/Heif/Av1/Conformance/libaom-cdef-kodim23-8b.bit new file mode 100644 index 0000000000..63d7889aed --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-cdef-kodim23-8b.bit @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a1a140b806c7b2570fa7f9e2572eaecd62ac3fd76b59830eb508337e87919dd9 +size 16469 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-cosmos1650-12b-libaom.yuv b/tests/Images/Input/Heif/Av1/Conformance/libaom-cosmos1650-12b-libaom.yuv new file mode 100644 index 0000000000..d3f0731df5 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-cosmos1650-12b-libaom.yuv @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ef712be32af7cf0a95c5c41bdcc51afc05a4ab7c047383f5f65edad2bb986712 +size 2629632 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-cosmos1650-12b.bit b/tests/Images/Input/Heif/Av1/Conformance/libaom-cosmos1650-12b.bit new file mode 100644 index 0000000000..546c68d06f --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-cosmos1650-12b.bit @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a45f9653255c1a660906554bafcc13fdee04f88cc6d0e08e1efa169cf0c6ddd2 +size 23769 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-10b-422-libaom.yuv b/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-10b-422-libaom.yuv new file mode 100644 index 0000000000..d42e8992cf --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-10b-422-libaom.yuv @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2bcf2a42e9f01044a32557e45ac19d0077c4bb3f2791cb3d01e22d86d9d9c7c5 +size 24000 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-10b-422.bit b/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-10b-422.bit new file mode 100644 index 0000000000..24ee79ebb4 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-10b-422.bit @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9fa3da33ba476f569c7825e134f7727571463416814486e95656806756d91b69 +size 407 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-12b-444-identity-limited-libaom.yuv b/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-12b-444-identity-limited-libaom.yuv new file mode 100644 index 0000000000..166ab6fa8f --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-12b-444-identity-limited-libaom.yuv @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7970ad0150245766dddc2b6ce480e1b0be68c734d0d15fd1f38a0d7b87ec42c9 +size 36000 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-12b-444-identity-limited.bit b/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-12b-444-identity-limited.bit new file mode 100644 index 0000000000..8077dd619c --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-12b-444-identity-limited.bit @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d295643fb78a8705a2ff2de6c9b40db9d94fb92a510038b00d5fc9ebee83d9a5 +size 364 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-12b-444-libaom.yuv b/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-12b-444-libaom.yuv new file mode 100644 index 0000000000..d42a192cd6 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-12b-444-libaom.yuv @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:583077d4dc530c57271ab4c990708dbe5765622978878f4305b5f64a6512fc37 +size 36000 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-12b-444.bit b/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-12b-444.bit new file mode 100644 index 0000000000..cd7dc04fe3 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-12b-444.bit @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:48fe85c152a0ad35adfe963a62164f8f105cbc88e88af6e7e0a6c6dc4e124f79 +size 475 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-8b-400-limited-libaom.yuv b/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-8b-400-limited-libaom.yuv new file mode 100644 index 0000000000..91854a7120 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-8b-400-limited-libaom.yuv @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bb0afbff82dc67937b7c3411b11eb8ed73033bf68194c01bf947750258ea00b1 +size 6000 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-8b-400-limited.bit b/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-8b-400-limited.bit new file mode 100644 index 0000000000..2c8cf0cc9f --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-8b-400-limited.bit @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e16477e1ce47f0b4c30c21168c0e33e1bfb3c48d67e37ea4cddee8a2c0478457 +size 229 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-8b-420-libaom.yuv b/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-8b-420-libaom.yuv new file mode 100644 index 0000000000..c40a3b31f4 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-8b-420-libaom.yuv @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6433d14c6b863eca1b244d1e3834832de1cc231e2f3733633c88aef3cb326132 +size 9000 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-8b-420-limited-libaom.yuv b/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-8b-420-limited-libaom.yuv new file mode 100644 index 0000000000..554223c41f --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-8b-420-limited-libaom.yuv @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c1b6dc8bba3a944a9abd041e3f8f253806f4689412ffab8fb71b74bc99281480 +size 9000 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-8b-420-limited.bit b/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-8b-420-limited.bit new file mode 100644 index 0000000000..3ffccd10ef --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-8b-420-limited.bit @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:aba642e2258e7da69ce54f333bd9a30037e22cbca191ad7674e5fc341bb154df +size 379 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-8b-420.bit b/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-8b-420.bit new file mode 100644 index 0000000000..ee73138810 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-8b-420.bit @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0f082a3cbbe9f4d16ba8735ae48d95fa9a06fdca56658dd9e937a1c661d0e45b +size 381 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-draw-points-8b-420-odd-libaom.yuv b/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-draw-points-8b-420-odd-libaom.yuv new file mode 100644 index 0000000000..0a0e99a7c3 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-draw-points-8b-420-odd-libaom.yuv @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b86740f4707d30d3835504b5b39622ed8dd88418bb052a88e3edd84258a6230a +size 567 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-draw-points-8b-420-odd.bit b/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-draw-points-8b-420-odd.bit new file mode 100644 index 0000000000..5aee2e6722 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-draw-points-8b-420-odd.bit @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ae77f08e0c30ada187e7a630b4b502cd8349227f9128d449b5d1e3badcc5dd1a +size 153 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-frame-id-196x196-8b-libaom.y4m b/tests/Images/Input/Heif/Av1/Conformance/libaom-frame-id-196x196-8b-libaom.y4m new file mode 100644 index 0000000000..b33d9189f4 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-frame-id-196x196-8b-libaom.y4m @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4479030861dd9d6ab9b00fa8cf77a34712bceca06c31131927d3e7e9bf5dda70 +size 115298 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-frame-id-196x196-8b.ivf b/tests/Images/Input/Heif/Av1/Conformance/libaom-frame-id-196x196-8b.ivf new file mode 100644 index 0000000000..ee76f67f8e --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-frame-id-196x196-8b.ivf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:90db15e7d272e6bd3b8ea798c94a94c38cab07f2148f4d5c79dc3f5e0b28d00d +size 19963 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-palette-draw-points-8b-444-libaom.yuv b/tests/Images/Input/Heif/Av1/Conformance/libaom-palette-draw-points-8b-444-libaom.yuv new file mode 100644 index 0000000000..3d09aae3aa --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-palette-draw-points-8b-444-libaom.yuv @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e05f7c0df06eccf0e43869d1d7b03daa1d635acd26a766f8940899be18d53251 +size 1089 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-palette-draw-points-8b-444.bit b/tests/Images/Input/Heif/Av1/Conformance/libaom-palette-draw-points-8b-444.bit new file mode 100644 index 0000000000..af645073b0 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-palette-draw-points-8b-444.bit @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f412a9e7f19d1c009d0329b993bb503d74ccda58505c54bae8fb3c16142181dc +size 42 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-cosmos-10b-libaom.yuv b/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-cosmos-10b-libaom.yuv new file mode 100644 index 0000000000..fc9bf3c737 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-cosmos-10b-libaom.yuv @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:609d2dbdee3660b466b846b2269a8c061665ae7d52737bcf6c0004094c88dc80 +size 2629632 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-cosmos-10b.bit b/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-cosmos-10b.bit new file mode 100644 index 0000000000..d7eff8ae7a --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-cosmos-10b.bit @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:cbdd03657e8e21ae26769a1e1e59920dba779c47cd1d03b18425148310f10a0f +size 26746 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-cosmos-12b-libaom.yuv b/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-cosmos-12b-libaom.yuv new file mode 100644 index 0000000000..beb3e324e6 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-cosmos-12b-libaom.yuv @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:61971cbe96f1cfc86b42b0608b91bebc7cc67bf758acf7cfdf7e17cd36f66beb +size 2629632 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-cosmos-12b.bit b/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-cosmos-12b.bit new file mode 100644 index 0000000000..05d854575d --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-cosmos-12b.bit @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7444a3f87497ed68cad2f5af2363b80381523f976a982de0fa1f58db8533b23b +size 23182 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-kodim23-8b-libaom.yuv b/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-kodim23-8b-libaom.yuv new file mode 100644 index 0000000000..80b0c70802 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-kodim23-8b-libaom.yuv @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6305858ad4204af4df42ac4bb207e94e1a7f155d97241e2cb335cb46db8d6925 +size 589824 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-kodim23-8b.bit b/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-kodim23-8b.bit new file mode 100644 index 0000000000..bbf554513d --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-kodim23-8b.bit @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0e605f0783daf8f6c530fe73572468b20f9834e7a8699a3e4e0f846741306711 +size 16460 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-superres-abc-10b-422-libaom.yuv b/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-superres-abc-10b-422-libaom.yuv new file mode 100644 index 0000000000..cd0cb43191 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-superres-abc-10b-422-libaom.yuv @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:86771fc87add9f179f327c0c9d44ae9ee0b373f977c23585e8bd0c322a95a215 +size 524288 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-superres-abc-10b-422.bit b/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-superres-abc-10b-422.bit new file mode 100644 index 0000000000..d93077ee24 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-superres-abc-10b-422.bit @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f3c3d6ed68a69a0a689b333ac71f016b5f60cbd48941faf0dcec707f8e2d6c9c +size 1131 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-superres-cosmos-12b-libaom.yuv b/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-superres-cosmos-12b-libaom.yuv new file mode 100644 index 0000000000..c8b8de7fde --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-superres-cosmos-12b-libaom.yuv @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:84666f72dd19ae549b7c5b77b07063e3a52c6a7e7d8798be126c0cdfcc8af5e4 +size 2629632 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-superres-cosmos-12b.bit b/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-superres-cosmos-12b.bit new file mode 100644 index 0000000000..35af9e9d94 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-superres-cosmos-12b.bit @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bed0130a163d9ad7c3068f6964c3944871d66da5efd2f37e435467183dba2402 +size 16125 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-superres-kodim23-8b-libaom.yuv b/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-superres-kodim23-8b-libaom.yuv new file mode 100644 index 0000000000..1b35172262 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-superres-kodim23-8b-libaom.yuv @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5c6e3642a399f59b0aea75261a69f0f31e6fae07998631831b17b7338efa3d9e +size 589824 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-superres-kodim23-8b.bit b/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-superres-kodim23-8b.bit new file mode 100644 index 0000000000..222d235824 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-superres-kodim23-8b.bit @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f7b5ec685eb17237354dd7c7d08dddbe41bf408e5eb1c0fb697ceb4a0dec81e4 +size 12322 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-superres-cosmos-10b-libaom.yuv b/tests/Images/Input/Heif/Av1/Conformance/libaom-superres-cosmos-10b-libaom.yuv new file mode 100644 index 0000000000..eb3ad9033b --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-superres-cosmos-10b-libaom.yuv @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5ab5a53eccebfa727a249c04b8ff81486dc8473b1aaea39e07d91e6e53d97a10 +size 2629632 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-superres-cosmos-10b.bit b/tests/Images/Input/Heif/Av1/Conformance/libaom-superres-cosmos-10b.bit new file mode 100644 index 0000000000..22e946a332 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-superres-cosmos-10b.bit @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8ff2272ed9d8d40b925d9711f0a5af93638b979a8e7c4930336b583d9e9db837 +size 18427 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-superres-cosmos-12b-libaom.yuv b/tests/Images/Input/Heif/Av1/Conformance/libaom-superres-cosmos-12b-libaom.yuv new file mode 100644 index 0000000000..d2398caf0f --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-superres-cosmos-12b-libaom.yuv @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6a50d88223eb46bb6079bb28032e682676fe4376a0cc11137ec1f6a56cdb85c7 +size 2629632 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-superres-cosmos-12b.bit b/tests/Images/Input/Heif/Av1/Conformance/libaom-superres-cosmos-12b.bit new file mode 100644 index 0000000000..e4b3fc6781 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-superres-cosmos-12b.bit @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5d5bb84dc57f28aae953196464729ba99ba83d6510f15ebaabe23f523722a82d +size 16085 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-superres-kodim23-8b-libaom.yuv b/tests/Images/Input/Heif/Av1/Conformance/libaom-superres-kodim23-8b-libaom.yuv new file mode 100644 index 0000000000..49515f6de6 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-superres-kodim23-8b-libaom.yuv @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6a6ae4b088d7e193dad3057e53da9664dd30bee6bbf2938a2caefee2da1c6b5b +size 589824 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-superres-kodim23-8b.bit b/tests/Images/Input/Heif/Av1/Conformance/libaom-superres-kodim23-8b.bit new file mode 100644 index 0000000000..0dd8351ca8 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-superres-kodim23-8b.bit @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:532bcf722b067a9583dc849c9a48a2404124e9357877c72284c61e9b54794272 +size 12302 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-cdef-cosmos-10b.avif b/tests/Images/Input/Heif/Av1/Conformance/libavif-cdef-cosmos-10b.avif new file mode 100644 index 0000000000..30d7052273 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-cdef-cosmos-10b.avif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4046de65fd6534fef9b82bec5ca188268c9b8a969c7a6ff3699b45e4a555b2b2 +size 35092 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-cdef-cosmos-12b.avif b/tests/Images/Input/Heif/Av1/Conformance/libavif-cdef-cosmos-12b.avif new file mode 100644 index 0000000000..331c565fd6 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-cdef-cosmos-12b.avif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:46bc4f5d6a199e085c5a0e17a9b6f92cc7360115fa2a877ca5e8a623dd8aa546 +size 26258 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-cdef-kodim23-8b.avif b/tests/Images/Input/Heif/Av1/Conformance/libavif-cdef-kodim23-8b.avif new file mode 100644 index 0000000000..47f9f07f07 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-cdef-kodim23-8b.avif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9149704cf62a07bb8375066149b3d588d4bf3f8c335355de4f6dce4236acfb5b +size 22882 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-colors-12b.avif b/tests/Images/Input/Heif/Av1/Conformance/libavif-colors-12b.avif new file mode 100644 index 0000000000..1550d842f4 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-colors-12b.avif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3bf9f91da471749e7df639ba7945d4d94c1c3e3968c26f3619fbbcfc92790576 +size 2267 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-cosmos1650-10b-libaom.yuv b/tests/Images/Input/Heif/Av1/Conformance/libavif-cosmos1650-10b-libaom.yuv new file mode 100644 index 0000000000..b15f31bf19 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-cosmos1650-10b-libaom.yuv @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9a59dd92a0c579f942acca8281ebd0465dc848be200a4d2ff57eaff589445f6c +size 2629632 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-cosmos1650-10b.avif b/tests/Images/Input/Heif/Av1/Conformance/libavif-cosmos1650-10b.avif new file mode 100644 index 0000000000..dc79231cb7 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-cosmos1650-10b.avif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1c3db1867051ae23ba61ed217f6b7372a4248e6322d76a239feab14bc4c55ab5 +size 37451 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-cosmos1650-10b.bit b/tests/Images/Input/Heif/Av1/Conformance/libavif-cosmos1650-10b.bit new file mode 100644 index 0000000000..980c072965 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-cosmos1650-10b.bit @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f930bf11eb2f61bf4ee0fe61853cd387a4c3dbe53503707db7b2c59d9d273ff3 +size 37169 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-intrabc-abc-10b-444-libaom-y4m.yuv b/tests/Images/Input/Heif/Av1/Conformance/libavif-intrabc-abc-10b-444-libaom-y4m.yuv new file mode 100644 index 0000000000..6b158dc65a --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-intrabc-abc-10b-444-libaom-y4m.yuv @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c6c5b5be6295bbe45a24480d3959fcb6f97bcec2b7d572361d93a6373c99c971 +size 786511 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-intrabc-abc-10b-444.avif b/tests/Images/Input/Heif/Av1/Conformance/libavif-intrabc-abc-10b-444.avif new file mode 100644 index 0000000000..fc246fcb61 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-intrabc-abc-10b-444.avif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8620d552eee66e4f99606a509cc6af1a7d365025493d483a7bde4fb4c03e5c14 +size 14778 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-intrabc-abc-12b-444-libaom-y4m.yuv b/tests/Images/Input/Heif/Av1/Conformance/libavif-intrabc-abc-12b-444-libaom-y4m.yuv new file mode 100644 index 0000000000..ef8fb6146a --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-intrabc-abc-12b-444-libaom-y4m.yuv @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:89368788da9af213ed16d4e9b9bcae4f4a9f8460497e19542e8aa84e781391ae +size 786511 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-intrabc-abc-12b-444.avif b/tests/Images/Input/Heif/Av1/Conformance/libavif-intrabc-abc-12b-444.avif new file mode 100644 index 0000000000..4076167e71 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-intrabc-abc-12b-444.avif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b5fc8307611313acd6b8df5fb91cf7a98199b3df1d88a1971e543472f6485fdd +size 20247 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-intrabc-abc-8b-444-libaom-y4m.yuv b/tests/Images/Input/Heif/Av1/Conformance/libavif-intrabc-abc-8b-444-libaom-y4m.yuv new file mode 100644 index 0000000000..d1fcc73af9 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-intrabc-abc-8b-444-libaom-y4m.yuv @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:938814c6efdfecb4d3c630787f6435208ab03fd96e1505a2fa190571ab2298fc +size 393289 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-intrabc-abc-8b-444.avif b/tests/Images/Input/Heif/Av1/Conformance/libavif-intrabc-abc-8b-444.avif new file mode 100644 index 0000000000..612f45ddbe --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-intrabc-abc-8b-444.avif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1fd33226d8c0fd011f7a39ff91e34eebb088fadd46c491c55110b6ae70ef7418 +size 10068 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-kodim23-8b-libaom.yuv b/tests/Images/Input/Heif/Av1/Conformance/libavif-kodim23-8b-libaom.yuv new file mode 100644 index 0000000000..c858043687 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-kodim23-8b-libaom.yuv @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8dde2eec742c39f0579c29ae84cba0fe01522a9008adcb2cffccec0295d18141 +size 589824 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-kodim23-8b.avif b/tests/Images/Input/Heif/Av1/Conformance/libavif-kodim23-8b.avif new file mode 100644 index 0000000000..b3570628be --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-kodim23-8b.avif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fa97744b1bad2137cac0f81fb653175362e16c6d8d20613d0769a847f2769bd1 +size 21032 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-kodim23-8b.bit b/tests/Images/Input/Heif/Av1/Conformance/libavif-kodim23-8b.bit new file mode 100644 index 0000000000..d2e35fb7a9 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-kodim23-8b.bit @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b7b1d3f85a870475acf579fbb7a0b59ff94c30f84b3f3a066411a50f9ba1bd20 +size 20750 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-lossless-circle-10b-444-libaom.yuv b/tests/Images/Input/Heif/Av1/Conformance/libavif-lossless-circle-10b-444-libaom.yuv new file mode 100644 index 0000000000..d1fe0f33fb --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-lossless-circle-10b-444-libaom.yuv @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:da9e962783072a3b2e8c0055125df1c909276ea5483345059e22706bdeea8121 +size 36000 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-lossless-circle-10b-444.avif b/tests/Images/Input/Heif/Av1/Conformance/libavif-lossless-circle-10b-444.avif new file mode 100644 index 0000000000..95b70d52f2 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-lossless-circle-10b-444.avif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:dd55dd965f2023d15038a32f118c0c21d98053804f58d78d1e8f67365f1dcdf4 +size 2604 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-lossless-circle-12b-444-libaom.yuv b/tests/Images/Input/Heif/Av1/Conformance/libavif-lossless-circle-12b-444-libaom.yuv new file mode 100644 index 0000000000..8af8c4f3b2 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-lossless-circle-12b-444-libaom.yuv @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ad216d8499c02716a63d83ae8ce5f4aa0a211fb05b717dcc3b4467c649313a79 +size 36000 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-lossless-circle-12b-444.avif b/tests/Images/Input/Heif/Av1/Conformance/libavif-lossless-circle-12b-444.avif new file mode 100644 index 0000000000..6ac1a71e28 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-lossless-circle-12b-444.avif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f4663d229c3f4d14abf60160acdd24846ceca2334615e5bc09f0f5f4128f0d0c +size 3493 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-lossless-circle-8b-444-libaom.yuv b/tests/Images/Input/Heif/Av1/Conformance/libavif-lossless-circle-8b-444-libaom.yuv new file mode 100644 index 0000000000..9fdf1a7790 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-lossless-circle-8b-444-libaom.yuv @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0daf2592a6e05243f46269f23d7cc4dcdaf24582d2b544e1dc3892d4c591f575 +size 18000 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-lossless-circle-8b-444.avif b/tests/Images/Input/Heif/Av1/Conformance/libavif-lossless-circle-8b-444.avif new file mode 100644 index 0000000000..ebcd129015 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-lossless-circle-8b-444.avif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:63c0976280061543aeb6e0ca91bfdd982dbf1fff49ffe19a3fd967a83603e100 +size 1663 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-palette-draw-points-8b.avif b/tests/Images/Input/Heif/Av1/Conformance/libavif-palette-draw-points-8b.avif new file mode 100644 index 0000000000..365eb14153 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-palette-draw-points-8b.avif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9186a477e80fd98d6c064e67f53b03e307241c376dc27e395079a8018f75dc80 +size 317 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-10b-400-libaom-y4m.yuv b/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-10b-400-libaom-y4m.yuv new file mode 100644 index 0000000000..d1c20f8290 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-10b-400-libaom-y4m.yuv @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1c71bca8da6ee62c42e1ebdcfcba89e2829f8fd9569d3bf8107c8321ed085b06 +size 262220 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-10b-400.avif b/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-10b-400.avif new file mode 100644 index 0000000000..428648e41c --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-10b-400.avif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:78d25037c2c41f387e74a88ed844fe9888f64e88f8155af31ccf994f5566c23d +size 1631 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-10b-420-libaom-y4m.yuv b/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-10b-420-libaom-y4m.yuv new file mode 100644 index 0000000000..aee84e102f --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-10b-420-libaom-y4m.yuv @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:39b2dca5c922a09a085b798fdf9cad0ce218418eb0fd33a08388ff05e6dd3114 +size 393295 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-10b-420.avif b/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-10b-420.avif new file mode 100644 index 0000000000..4aadeb766b --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-10b-420.avif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5df732dc3c72684f35dfa9c69781133e6383ae8406e66e57d8cc3fa5e0dd852d +size 1682 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-10b-422-libaom-y4m.yuv b/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-10b-422-libaom-y4m.yuv new file mode 100644 index 0000000000..51aa38edc4 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-10b-422-libaom-y4m.yuv @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:79c6e4560af81fe427bc38f1fa64ee43622d1ec02a7876a533ccb6be72955a9e +size 524367 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-10b-422.avif b/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-10b-422.avif new file mode 100644 index 0000000000..1dd877072f --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-10b-422.avif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3420ea3f3fb2430ffa502a85190e3aab9aafdeb24880748d023b3a5091943c6d +size 1685 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-10b-444-libaom-y4m.yuv b/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-10b-444-libaom-y4m.yuv new file mode 100644 index 0000000000..3cd001634e --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-10b-444-libaom-y4m.yuv @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ad9f1b3318f9795f325a2f8d4e6dd5e5ea2c4ea51713f6aa9dab302a970b25bb +size 786511 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-10b-444.avif b/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-10b-444.avif new file mode 100644 index 0000000000..e7f3f99313 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-10b-444.avif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:354e339eaf0e6e2889659c2a889d65b9bdf6022543fb9f9522d11301251e0075 +size 1682 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-12b-400-libaom-y4m.yuv b/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-12b-400-libaom-y4m.yuv new file mode 100644 index 0000000000..70b3404a26 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-12b-400-libaom-y4m.yuv @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a020ff72013124975caae5823eaaa1abd3ed66e19cf6ccb31ff96fcd0b63a084 +size 262220 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-12b-400.avif b/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-12b-400.avif new file mode 100644 index 0000000000..d7d93a454c --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-12b-400.avif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:72af196f86395af9b5939598be97c45cd065ad4b62c276a0486bf76a6568020b +size 1643 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-12b-420-libaom-y4m.yuv b/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-12b-420-libaom-y4m.yuv new file mode 100644 index 0000000000..c40c8ac495 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-12b-420-libaom-y4m.yuv @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0b3b67cc8fff91e5302f02f4718967b9a2871b321109defdafb85dd46b558876 +size 393295 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-12b-420.avif b/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-12b-420.avif new file mode 100644 index 0000000000..f0a41f80c0 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-12b-420.avif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:063d1980df27ae7ff996c437d962ca51d4615d6f745eca6cc55a5c57a05c761d +size 1688 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-12b-422-libaom-y4m.yuv b/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-12b-422-libaom-y4m.yuv new file mode 100644 index 0000000000..79d63b266f --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-12b-422-libaom-y4m.yuv @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:efbb14ce47121971afca860b811b34da9414fb961a73e2f16fd403d835cf89d1 +size 524367 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-12b-422.avif b/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-12b-422.avif new file mode 100644 index 0000000000..f87357b03a --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-12b-422.avif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:80d606728693f5f890fc1ffb7a2bcff1b2128b783143445fdbed89da30b1353f +size 1686 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-12b-444-libaom-y4m.yuv b/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-12b-444-libaom-y4m.yuv new file mode 100644 index 0000000000..b802feb305 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-12b-444-libaom-y4m.yuv @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:43907add76f881850cfdc4ed23655dc71443c8d40ef16093b51745650dc66307 +size 786511 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-12b-444.avif b/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-12b-444.avif new file mode 100644 index 0000000000..ae98187a54 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-12b-444.avif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f35443d16382c2d184adc3e518230ce4df497a429d101c92b96bb592b50c1a36 +size 1683 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-8b-400-libaom-y4m.yuv b/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-8b-400-libaom-y4m.yuv new file mode 100644 index 0000000000..33c5a69c78 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-8b-400-libaom-y4m.yuv @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:da90f960e2297224b621ff866353d89ddb0dd9ed0ffcb201d4b5e49c250c5c07 +size 131146 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-8b-400.avif b/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-8b-400.avif new file mode 100644 index 0000000000..6452bda077 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-8b-400.avif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1b33f3d2f20a4d1d828ee3ed44e94e0ac5a13caa3971e0279ba740581437c194 +size 1665 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-8b-420-libaom-y4m.yuv b/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-8b-420-libaom-y4m.yuv new file mode 100644 index 0000000000..1257f0c682 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-8b-420-libaom-y4m.yuv @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:98342115d81519b685c3c3d4077e74896b5c3ebc6eb44ccfe11dd20f43fc06df +size 196689 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-8b-420.avif b/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-8b-420.avif new file mode 100644 index 0000000000..d6934fac62 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-8b-420.avif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:16353df2ed0b1e5f3b5111e6f514e8ba56b81be8b20aae5b4d07ccf71f203538 +size 1718 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-8b-422-libaom-y4m.yuv b/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-8b-422-libaom-y4m.yuv new file mode 100644 index 0000000000..c398ab5335 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-8b-422-libaom-y4m.yuv @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:dd25de31e7c3175b8854cdf63ce233d2657b07ba038158286763c19ab4ca60cc +size 262217 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-8b-422.avif b/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-8b-422.avif new file mode 100644 index 0000000000..40f000b051 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-8b-422.avif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b1f8d6ae7131d2596f43ac9a67c0afce6f4cb4eacd323383a22073bcb8afc05a +size 1736 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-8b-444-libaom-y4m.yuv b/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-8b-444-libaom-y4m.yuv new file mode 100644 index 0000000000..418816b6d0 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-8b-444-libaom-y4m.yuv @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:dae8878737e0afcc7084a252f5963bc8915e0febbf7a92284bad14bb8723e826 +size 393289 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-8b-444.avif b/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-8b-444.avif new file mode 100644 index 0000000000..573da92774 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-profile-8b-444.avif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ef6c09e4c7b3ecfcc7fef52abe3eab6afe422ec6bf9fd0fd3af00db445384a6f +size 1722 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-progressive-draw-points-8b-libaom-y4m.yuv b/tests/Images/Input/Heif/Av1/Conformance/libavif-progressive-draw-points-8b-libaom-y4m.yuv new file mode 100644 index 0000000000..24494db844 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-progressive-draw-points-8b-libaom-y4m.yuv @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:cc70e15d31a25289492469bb637ca7e00f11de24889de5a9c6b9bd3f20be4b7a +size 2986 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-progressive-draw-points-8b.avif b/tests/Images/Input/Heif/Av1/Conformance/libavif-progressive-draw-points-8b.avif new file mode 100644 index 0000000000..bc5d0a0d82 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-progressive-draw-points-8b.avif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:077ab2ad1e46dd912a973e4f024cb1eb242a08298be2dbf1a52a058e88c48a4a +size 600 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-progressive-draw-points-8b.bit b/tests/Images/Input/Heif/Av1/Conformance/libavif-progressive-draw-points-8b.bit new file mode 100644 index 0000000000..a6409ce406 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-progressive-draw-points-8b.bit @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:93ff341e1a7a4c849c3f713c4d20589fce93ddd7154106bce574106395e1498e +size 72 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-restoration-cosmos-10b.avif b/tests/Images/Input/Heif/Av1/Conformance/libavif-restoration-cosmos-10b.avif new file mode 100644 index 0000000000..d1f0acbbf9 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-restoration-cosmos-10b.avif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d3215c84d33f1cb37f9c8998ed4c1982837b9207c7a3018cac0df1659691ed8b +size 27021 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-restoration-cosmos-12b.avif b/tests/Images/Input/Heif/Av1/Conformance/libavif-restoration-cosmos-12b.avif new file mode 100644 index 0000000000..ecf55471ee --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-restoration-cosmos-12b.avif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fc446537fb7954656379384232857a23c5eab25187e63352097d276d8034869e +size 23453 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-restoration-kodim23-8b.avif b/tests/Images/Input/Heif/Av1/Conformance/libavif-restoration-kodim23-8b.avif new file mode 100644 index 0000000000..927b6c8fb0 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-restoration-kodim23-8b.avif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fb86fc23e29c19566600fa8e5230038598c37836971feff35976b3f720fd5df0 +size 16735 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-rotating-grid-global-warp-libaom.y4m b/tests/Images/Input/Heif/Av1/Conformance/libavif-rotating-grid-global-warp-libaom.y4m new file mode 100644 index 0000000000..f49908b53d --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-rotating-grid-global-warp-libaom.y4m @@ -0,0 +1,3 @@ +YUV4MPEG2 W256 H256 F25:1 Ip A0:0 C444 XYSCSS=444 XCOLORRANGE=LIMITED +FRAME +-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------뾾---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------뾾뾾----------------------------------------------------------------------------------------------------------뾾---------------------------------------------------------------------------------------------------------뾾--------------------------------------------------------------------------------------------------------------------------뾾----------------------------------------------------------------------------------------------------------뾾-----------------------------------------------------------------------------------------------------------뾾----------------------------------------------------------------------------------------------------------------------뾾---------------------------------------------------------------------------------------------------------------뾾--------------------------------------------------------------------------------------------------------------뾾-----------------------------------------------------------------------뾾---------------------------------------뾾----------------------------------------------------------------------뾾--------------------------------------뾾---------------------------------------------------------------------뾾-----------------------------뾾---------------------------------------------------------------------------------------------------------뾾----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------뾾--------------------------------------------------------------------------------------------뾾-----------------------------------------------------------------------------------------------------뾾------------------------------------------------------------------------------------------------------뾾--------------------------------------------------------------------------------------------------------뾾-------------------------------------------------------------------------------------------------------------뾾--------------------------------------------------------------------------------------------------------뾾---------------------------------------------------------------------------------------------------------------------뾾--------------------------------------------------------------------------------------------------------------------뾾----------------------------------------------------------------뾾---------------------------------------------뾾----------------------------------------------------------------뾾------------------------------------------------뾾---------------------------------------------------------------뾾------------------------------------------------------------뾾--------------------------------------------------------------뾾-------------------------------------------------------뾾------------------------------------------------------------------------------------------------------------------뾾-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------뾾-----------------------------------------------------------------------------------------------------------------뾾-----------------------------------------------------------------------------------------------------------------------뾾------------------------------------------------------------------뾾----------------------------------------------------뾾-------------------------------------------------------뾾----------------------------------------------------뾾-----------------------------------------------------------뾾----------------------------------------------------뾾------------------------------------------------------------------뾾---------------------------------------------------뾾---------------------------------------------------------------------------------------------------------------------뾾--------------------------------------------------------------------------------------------------------------뾾---------------------------------------------------------------------------------------------------------------------뾾-------------------------------------------------------------------------------------------------------------------뾾------------------------------------------------------------------------------------------------------------------뾾--------------------------------------------------------------------------------------------------------------------뾾------------------------------------------------------------------------------------------------------------뾾----------------------------------------------------------------------------------------------------------------------뾾-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------뾾--------------------------------------------------------------------------------------------------------------------------뾾----------------------------------------------------------------------------------------------------------------------------------뾾--------------------------------------------------------------------------------------------------------------------------뾾----------------------------------------------------------------------------------------------------------------------------뾾----------------------------------------------------------------------------------------------------------------------------뾾----------------------------------------------------------------------------------------------------------------------------뾾----------------------------------------------------------------------------------------------------------------------------뾾---------------------------------------------------------------------------------------------------------------------------뾾---------------------------------------------------------------------------------------------------------------------------------뾾------------------------------------------------------------------------------------------------------------------------뾾---------------------------------------------------뾾----------------------------------------------------------------뾾---------------------------------------------------뾾------------------------------------------------------------------------뾾---------------------------------------------------뾾-----------------------------------------------------------------------뾾---------------------------------------------------뾾------------------------------------------------------------뾾----------------------------------------------------뾾----------------------------------------------------------------뾾----------------------------------------------------뾾----------------------------------------------------------------------뾾----------------------------------------------------뾾--------------------------------------------------------------------------------------------------------------------뾾------------------------------------------------------------------------------------------------------------뾾--------------------------------------------------------------------------------------------------------------------뾾------------------------------------------------------------------------------------------------------------------뾾-------------------------------------------------------------------------------------------------------------------뾾---------------------------------------------------------------------------------------------------------------------뾾--------------------------------------------------------------------------------------------------------------뾾-----------------------------------------------------------------------------------------------------------------------뾾---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------뾾---------------------------------------------------------------------------------------------------------------뾾----------------------------------------------------------------------------------------------------------------------뾾---------------------------------------------------------------------------------------------------------------뾾----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------뾾----------------------------------------------------------------------------------------------------------------------뾾-----------------------------------------------------------------------------------------------------------------뾾-----------------------------------------------------------------------------------------------------------------뾾---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------뾾-----------------------------------------------------------------------------------------------------------------뾾------------------------------------------------------------------------------------------------------뾾-----------------------------------------------------------------------------------------------------------뾾------------------------------------------------------------------------------------------------------뾾----------------------------------------------------------------------------------------------------뾾---------------------------------------------------------------------------------------------------뾾------------------------------------------------------뾾-----------------------------------뾾--------------------------------------------------------------뾾----------------------------------뾾------------------------------------------------------------------------뾾---------------------------------뾾---------------------------------------------------------------------------------------------------------뾾----------------------------------------------------------------------------------------------------뾾-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------뾾----------------------------------------------------------------------------------------------------------뾾-----------------------------------------------------------------------------------------------------------뾾------------------------------------------------------------------------------------------------------------------뾾----------------------------------------------------------------------------------뾾---------------------뾾-------------------------------------------------------------------------------------뾾-------------------뾾--------------------------------------------------------------------------------------뾾-----------------뾾----------------------------------------------------------------------------------------뾾-----------------뾾-----------------------------------------------------------------------------------------뾾-----------------뾾뾾--------------------------------------------------------------------------------------------뾾----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------뾾---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------PPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX``````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX``````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX``````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhppppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX``````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhppppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX``````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhppppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX``````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhppppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhppppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhppppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhppppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX``````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhppppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX``````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX``````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPXXXXPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXPPPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX````XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX```````````````````````````XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX``````````````````````````````````````````````````XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX````````````````````````````````````````````````````````````````````````XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX```````````````````````````````````````````````````````````````````````````````````````````````XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX``````````````````````````````````````````````````````````````````````````````````````````````````````````````````````XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX```````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX``````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````XXXXXXXXXXXXXXXXXXXXXXX````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````hhh`````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhh``````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh```````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh`````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh``````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh````````````````````````````````````````````````````````````````````````````````````````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh`````````````````````````````````````````````````````````````````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh``````````````````````````````````````````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh```````````````````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh`hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpphhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhppppppppppppppppppppppppphhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppppppppppppppppphhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppphhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppphhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppphhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppphhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppphhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppphhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppphhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppphhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppphhppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppxxppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxpppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxpppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxpppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxpppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxpppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxppppppppppppppppppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxpppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \ No newline at end of file diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-rotating-grid-global-warp.avif b/tests/Images/Input/Heif/Av1/Conformance/libavif-rotating-grid-global-warp.avif new file mode 100644 index 0000000000..a4283bb130 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-rotating-grid-global-warp.avif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ee8cdf6df36fb2999a17d2a86c41040d8be13b958a1d5ae53e37343c6fb49e0e +size 39472 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-rotating-grid-local-warp-libaom.y4m b/tests/Images/Input/Heif/Av1/Conformance/libavif-rotating-grid-local-warp-libaom.y4m new file mode 100644 index 0000000000..292d93e9b7 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-rotating-grid-local-warp-libaom.y4m @@ -0,0 +1,3 @@ +YUV4MPEG2 W256 H256 F25:1 Ip A0:0 C444 XYSCSS=444 XCOLORRANGE=LIMITED +FRAME +------------------------------------------------0----------------0----------------0---------------./-----------------------------------------------------------------------------------------------------0----------------0---------------.0-----------------------------------------------------------------------------------------------------.--------------------------------0------------------------------------------------------------------------------------------------.------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------.0------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------..---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------,--.------------..---------------------------------------------------------------------------------------------------------------/...------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------.-----------------------------------------------------------------------------------------------------------------------..---------------------------------------------------------------------------------------------------------------------------------../--------------------------------------------------------------------------------------------------------------------------.---------/------------------------------------------------------------------------------------------------------------------------.-----------------------------------------------------------------------------------------------------------/----------------.----------------------------------/-------------------.-----------------------------------------------------------------------------------------------------------------------------..--------------------------------------------------------------------------------------------------------------------------------------------.------./------------------------------------------------------------------------------------------------------------------------------.------------------------------------------------------------------------------------------------------------------------------,-----------------0--------------------------/----------------------------------------------------------------------------------------------------------------------------------------------/----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------1----------------------------------------------------------------------------------------------------------------------------------------------------------,-----------------------------------------------------------------------------------------------------------------,,-------------/----------------------------------------------------------------------------------------.--------------.---------------.--------------/--------------/-----------------------------------------------------------.--------------.,--------------.-------------------------------------------------------------------------/--------------.--------------.---------------.-----------------------------------------------------------0-----------------------------/--------------/-------------------------------------------------------------------------------..----------------------------/--------------------------------------------------------------------------------------..----------------------------.--------------------------------------------------------------------------(--------------.-----------------------------.------------------------------------------------------------------------------------.-----------------------------.-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------,--------------------------------------------------------------------------------------------------------뿾---------------------------------------------------------------------------------------------------,-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------/---------------------------------------------------------------------------------------------------------------------------------------------------------뾾뾾----------------------------------------------------------------------------------------------------------뾾---------------------------------------------------------------------------------------------------------뾾--------------------------------------------------------------------------------------------------------.-----------------뾾----------------------------------------------------------------------------------------.-----------------뾾---------------------------------------..----------------------------------------------.-------------------뾾-----------------------------------------------------../.---------------------------------------.---------------------뾾------------------------------------------------------.----/-----------------.---------.-----------------------뾾------------------------------------------------------------------------------------.-------------------------뾾---------------------------------------------------------------------0-뾾------------0----------------/---------뾾---------------------------------------------------------------------,뾾--------../---------------------------뾾---------------------------------/-----------------------------------뾾-----------------------------뾾---------------------------------------------------------------------------------------------------------뾾-------------------------------------------------------------------------------------------------------......----------------------------------------------------------------------------------------------------------/----------------------------------------------------------------------------------------뾾--------------------------------------------------------------------------------------------뿿------------------------------/----------------------------------------------------------------------뿾------------------------------------------------------------------------------------------------------뾾--------------------------------------------------------------------------------------------------------뾾------------------------------.//----------------------------------------------------------------------------뾾--------------------------------------------------------------------------------------------------------뾾---------------------------------------------------------------------------------------------------------------------뾾--------------------------------------------------------------------------------------------------------------------뾾----------------------------------------------------------------뾾---------------------------------------------뾾----------------------------------------------------------------뾾------------------------------------------------뾾---------------------------------------------------------------뾾------------------------------------------------------------뾾--------------------------------------------------------------뾾-------------------------------------------------------뿾------------------------------------------------------------------------------------------------------------------뾾----------------------------------------/--------------------------------------.---------------------------------------------------------------------------.-------------------------------------/--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------/----------------------------------------------------------------------------------------------------------------.---------------------------------------------------------------------------------------------------------------.-------------------------------------------------------------------------,---------------------------------------.-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------뾾.----------------------------------------------------------------------------------------------------------------뿾---------------....-----------------------------------------------,----------------------------------------------------뾾------------------------./----------------------------------------뾾----------------------------------------------------뾾-------------------------------------------------------뾾------------------------.---------------------------뾾---------------------------------------------/.------------뿾------------------------.---------------------------뾾-----------------------------------------------------------..-----뾾------------------------.--------------------------뾾-----------------------------------------------------------------.------------------------.--------------------------뾾---------------------------------------------------------------------------------.----------------------------뾾----------------------------------------------------------------------------------------//---------------------------뾾-------------------------------------------------------------------------------------------------------------------꾾------------------------------------------------------------------------------------------------------------------뾾----------------------------------------------------------------------------------------00--------------------------羾---------------------------------------------------------------------------------------..--------/.---------¾/--------------------------------------------.------------------------------------------.-----------------------------꿾--------------------------------------------.--------------------------------------------------------------------------------------------------------------------------0.--------------------------------------------------------------------------------,,++------------------------------/.--,,---------4------------------------------------------------------------------,,-,----------------------------..------------------------/-------------------------------------------------------------------------------------------------.--,,----------------------------------------------------------------------------------------------------------------------.--------------------------------------------------------------------------------------------------------------------------------1-----------------------------------------------------------------------------------------------------------------------------뾾--------------------------------------------------------------------------------------------------------------------------뾾----------------------------------------------------------------------------------------------------------------------------------뾾--------------------------------------------------------------------------------------------------------------------------뾾----------------------------------------------------------------------------------------------------------------------------뾾----------------------------------------------------------------------------------------------------------------------------뾾----------------------------------------------------------------------------------------------------------------------------뾾----------------------------------------------------------------------------------------------------------------------------뾾---------------------------------------------------------------------------------------------------------------------------뾾---------------------------------------------------------------------------------------------------------------------------------뾾------------------------------------------------------------------------------------------------------------------------뾾---------------------------------------------------뾾0---------------------------------------------------------------뾾---------------------------------------------------뾾------------------------------------------------------------------------뾾---------------------------------------/-----------뾾-----------------------------------------------------------------------뾾-------------------------/.------------------------뾾---------------.--------------------------------------------뾾-------------------------..-------------/-----------뿾/---------------------------------------------/...--------------뾾-------------------------..-------------.-----------뿾.--------------/------------------------------------------------------뾾-------------------------.--------------.-----------뿾.----------------------------------------------------------./-------------------------------------------------------뿾-----------------------------------------------------------------------0.-----------------------------------뾾-----------------------------------------------------------------------/..------------00----------------------------뾾-----------------------------------------------------------------------..-----------------------------------------뾾------------------------------------------------------------------------.------------------------------------------뾾------------------------------------------------------------------------.--------------00----------------------------뾾---------------------------------------------------------------------------------------..---------------------뿾---------------------------------------------------------------------------------------.------------------------.------뾾------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------,------------------------------------------------.0-------------------------------------------------------------------------------------------------------------------------------/----------------------------------------------------------------------------------------------------------------.-----뾾---------------------------------------------------------------------------------------------------------------뿾----------------------------------------------------------------------------------------------------------------------뾾---------------------------------------------------------------------------------------------------------------뾾----------------------------------------------------------------------------------------------------------------,-----------------------------------------------------3./----------------.----------------------------------------------------------------------------------------------------------------/-----------------------------------------------------------------------------------------------4-----------------/-----------------------------------------------------------------------------------------------뾾-----------------.----------------/-----------------------------------------------------------------------------------뾾----------------------------------.--------------------------------------/---------------------------------------뾾-----------------.-------------------------------------------------------/---------------------------------------뾾-------------------------------------------------------------------------.--------------------------------------------------------------------------------------------------------------------.------------------------------------------------------------------------------------------------------------------------.-------------------------------------------------------------------------------------------------------------.-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------뾾-----------------------------------------------------------------------------------------------------------------뿾---------------1--------------------------------------------------------------------------------------뾾------------------------------...--------------------------------------------------------------------------뾾------------------------------------------------------------------------------------------------------뾾----------------------------------------------------------------------------------------------------뾾------------------------------0--------------------------------------------------------------------뾾------------------------------------------------------뾾-----------------------------------뾾--------------------------------------.---------------.------,뾾----------------------------------뾾---------------------------------------------------------------------.0,뾾---------------------------------뾾---------------------------------------------------------------------------------------------------------뾾------------------------------------/---------------------------------------------------------------뿾-------------------------------------------------------------------------------./----------------1-----------.--------------------------------------------------------------------------------------------------------------.-----------------------------------------------------------------------------------------------/---------,/3뾾.-----------------------------------------------------------------------------------------------.---------뾾.----------------/---------------------------------------------------------------------------------.-------뾾.--------------------------------../.-----------------------------------------------------------------------.-----뾾.---------------------------------------------------------------------------------뾾-----------------.---뾾-------------------------------------------------------------------------------------뾾-------------------뾾--------------------------------------------------------------------------------------뾾-----------------뾾----------------------------------------------------------------------------------------뿾-----------------뾾---------------------------------------------------.-------------------------------------뿾-----------------뿿뾾---------------------------------------------------..---------------------------------------뾾-----------------------------------------------------------./------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------3---------------------------------------------------------------------------------------------------,---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------,------------------------------------------------------------------------------------------------------------뿾-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------.------------------------------------------------------------------------/--------------/-----------------------------.-------------------------------------------------------------------------------.--------------.--------------/-----------------------------------------------------------------------------------------.--------------.-----------------------------------------------------------/--------------/------------------------------.--------------.-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------/-----------------------------------------------------------------------------------------------------------------.------------------------------------/..------------------------------------------------------------------------------.------------------------------------------------------.-------------------------------------------------------------------------.----------------------------------------------------.---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------/.-.---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------0.-----------------------------------------------------------------------------------------------------------------------------------------------.-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------1.----------------------------------------------------------------------------------------------------------------------------------------------.---------1------------------------------------------------------------------------------------------------------------------------------------------------1-------------------------------------------------------------------------------------------------------------------------------,,,-,,------------------1------------------------------------------------------------------------------------------------------------------------------------------1--------------------------------------------------/-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------/----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------.-------------------------------------------------------------------------------------------------------------------/----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------..-------------------------------------------------------------------------.---------------------------------------------------/...----------------------------------------------------------------------/------------------------------------------------------------------------------------------------------------------------0---------------------------------------------------------------------------------------------------------------------------------/----------------0.---------------0.----------------------------------------------------------------------------------------------------/----------------0..--------------/..--------------0-----------------------------------------------PPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPQXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXY````````````````````````````````ahhhhhhhhhhhhhhhhhhhhhhhhhhhhhhippppppppppppppppppppppppppppppppqxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxyPPPPPPPPPPPPPPPPPPPPPRXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXZ````````````````````````````````bhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhjpppppppppppppppppppppppppppppppprxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxzPPPPPPPPPPPPPPPPPPPPPVXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX^````````````````````````````````fhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhnppppppppppppppppppppppppppppppppvxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx~PPPPPPPPPPPPPPPPPPPPPWXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX_````````````````````````````````ghhhhhhhhhhhhhhhhhhhhhhhhhhhhhhoppppppppppppppppppppppppppppppppwxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX_````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX_````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX_````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX_````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX_````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX_````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPQXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXY_```````````````````````````````ahhhhhhhhhhhhhhhhhhhhhhhhhhhhhhippppppppppppppppppppppppppppppppqxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxyPPPPPPPPPPPPPPPPPPPPRXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXZ_```````````````````````````````bhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhjpppppppppppppppppppppppppppppppprxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxzPPPPPPPPPPPPPPPPPPPPVXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX^````````````````````````````````fhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhnppppppppppppppppppppppppppppppppvxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx~PPPPPPPPPPPPPPPPPPPPWXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX_````````````````````````````````ghhhhhhhhhhhhhhhhhhhhhhhhhhhhhhoppppppppppppppppppppppppppppppppwxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPQXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXY````````````````````````````````ahhhhhhhhhhhhhhhhhhhhhhhhhhhhhhippppppppppppppppppppppppppppppppqxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxyPPPPPPPPPPPPPPPPPPPRXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXZ````````````````````````````````bhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhjpppppppppppppppppppppppppppppppprxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxzPPPPPPPPPPPPPPPPPPPVXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX^````````````````````````````````fhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhnppppppppppppppppppppppppppppppppvxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx~PPPPPPPPPPPPPPPPPPPWXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX_````````````````````````````````ghhhhhhhhhhhhhhhhhhhhhhhhhhhhhhoppppppppppppppppppppppppppppppppwxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPQXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXY````````````````````````````````ahhhhhhhhhhhhhhhhhhhhhhhhhhhhhhipppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPRXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXZ````````````````````````````````bhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhjpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPVXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX^````````````````````````````````fhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhnpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPWXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX_````````````````````````````````ghhhhhhhhhhhhhhhhhhhhhhhhhhhhhhopppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppqxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxyPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhppppppppppppppppppppppppppppppppprxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxzPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppvxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx~PPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppwxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPQXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPRXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPVXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPWXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPXXXXYXXXXXXXXXXXXXXXXXXXXXXXXXXY`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhipppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPXXXXYXXXXXXXXXXXXXXXXXXXXXXXXXXZ`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhjpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX^`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhnpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX_`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhopppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````ahhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppqxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxyPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````bhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhppppppppppppppppppppppppppppppppprxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxzPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````fhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppvxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx~PPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````ghhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppwxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPQXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPRXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPVXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPWXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXY````````````````````````````````ahhhhhhhhhhhhhhhhhhhhhhhhhhhhhhippppppppppppppppppppppppppppppppqxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxyPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXZ````````````````````````````````bhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhjpppppppppppppppppppppppppppppppprxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxzPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX^````````````````````````````````fhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhnppppppppppppppppppppppppppppppppvxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx~PPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX_````````````````````````````````ghhhhhhhhhhhhhhhhhhhhhhhhhhhhhhoppppppppppppppppppppppppppppppppwxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPQXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXY````````````````````````````````ahhhhhhhhhhhhhhhhhhhhhhhhhhhhhhipppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPRXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXZ````````````````````````````````bhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhjpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPVXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX^````````````````````````````````fhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhnpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPWXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX_````````````````````````````````ghhhhhhhhhhhhhhhhhhhhhhhhhhhhhhopppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhppppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhppppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhppppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhppppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppqxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxyPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhppppppppppppppppppppppppppppppppprxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxzPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppvxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx~PPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppwxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPQXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXY````````````````````````````````ahhhhhhhhhhhhhhhhhhhhhhhhhhhhhhipppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxyPPPPPPPPPPPPPPRXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXZ````````````````````````````````bhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhjpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxzPPPPPPPPPPPPPPVXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX^````````````````````````````````fhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhnpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx~PPPPPPPPPPPPPPWXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX_````````````````````````````````ghhhhhhhhhhhhhhhhhhhhhhhhhhhhhhopppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppqxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhppppppppppppppppppppppppppppppppprxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppvxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppwxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPQXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXY````````````````````````````````ahhhhhhhhhhhhhhhhhhhhhhhhhhhhhhippppppppppppppppppppppppppppppppqxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxyPPPPPPPPPPPPPRXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXZ````````````````````````````````bhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhjpppppppppppppppppppppppppppppppprxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxzPPPPPPPPPPPPPVXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX^````````````````````````````````fhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhnppppppppppppppppppppppppppppppppvxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx~PPPPPPPPPPPPPWXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX_````````````````````````````````ghhhhhhhhhhhhhhhhhhhhhhhhhhhhhhoppppppppppppppppppppppppppppppppwxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXY`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXZ`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX^`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX_`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPQXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````ahhhhhhhhhhhhhhhhhhhhhhhhhhhhhhippppppppppppppppppppppppppppppppqxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxyPPPPPPPPPPPPRXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````bhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhjpppppppppppppppppppppppppppppppprxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxzPPPPPPPPPPPPVXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````fhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhnppppppppppppppppppppppppppppppppvxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx~PPPPPPPPPPPPWXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````ghhhhhhhhhhhhhhhhhhhhhhhhhhhhhhoppppppppppppppppppppppppppppppppwxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPXXYXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPXXYXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPXXYXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPXXYXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPXXYXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPXXYXXXXXXXXXXXXXXXXXXXXXXXXXXXX`````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhpppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPXXWUSQPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXWUSQPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXWUSQPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXWUSQPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXWUSQPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXWUSQPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXWUSQPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXWUSQPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXWUSQPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXWUSQPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXWUSQPPPPPPPPPPPPPPPPPPPPPPXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX``_][YXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX``````````````````````````_][YXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX````````````````````````````````````````````````_][YYYXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX``````````````````````````````````````````````````````````````````````_][YXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX``````````````````````````````````````````````````````````````````````````````````````````````_][YXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX``````````````````````````````````````````````````````````````````````````````````````````````````````````````````````_][YXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX``````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````_][YXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX``````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````_][YXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX``````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````_][YXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX``````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````_][YXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX``````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````_][YXXXXXXXXXXXXXXXXXXXXXX````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````aaaaaaaaaaaaaaaaaaaaaaaa````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````hhhhhhhggggggggggggggggecaaaaaaaaaaaaaaaaaaaaaaa````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````gggggggggggggggggggggggggggggggggggggggggggggggecaaaaaaaaaaaaaaaaaaaaaaa````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````ggggggghhhhhhhhhhhhhhhhhgggggggggggggggggggggggggggggggggggggggggggggggecaaaaaaaaaaaaaaaaaaaaaaa````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhgggggggggggggggggggggggggggggggggggggggggggggggecaaaaaaaaaaaaaaaaaaaa```````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````iihhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhggggggggggggggggggggggggggggggggggggggggggggggfecaaaaaaaaaaaaaaa````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````iiiiihhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhgggggggggggggggggggggggggggggggggggggggggggggggecaaaaaaaaaaaaaaaaaaaaaaa````````````````````````````````````````````````````````````````````````````````````````````````iiiiihhhhhhhhhhhhhhhhhhhhhhhhhhiiiiiiiiiiiiiiiiihhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhggggggggggggggggggggggggggggggggggggggggecaaaaaaaaaaaaaaaaaaaaaaa````````````````````````````````````````````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhiiiiiiiiiiiiiiiihhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhgggggggggggggggggggggggggggggggggggggggggggggggecaaaaaaaaaaaaaaaaaaaaaaa````````````````````````````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhiiiiiiiiiiiiiiiihhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhgggggggggggggggggggggggggggggggggggggggggggggggecaaaaaaaaaaaaaaaaaaaaaaa````````````````````````hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhiiiiiiiiiiiiiiiiihhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhgggggggggggggggggggggggggggggggggggggggggggggggecaaaaaaaaaaaaaaaaaaaaaaahhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhiiiiiiiihhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhgggggggggggggggggggggggggggggggggggggggggggggggghhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhiiiiiiiihhhhhhhhiiiiiiiihhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhgggggggggggggggggggggggghhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhgggggggghhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhiiiiiiiiihhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhiiiiiiiiiiiiiiiihhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhiiiiiiiiiiiiiiiihhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhppppppppppppppppppppppppppomkihhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhppppppppppppppppppppppppppppppppppppppppppppppomkihhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppomkihhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppomkihhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppomkihhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppomkihhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppomkihhhhhiiiiihhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppomkihhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppomkihhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppomkihhhhhhhhhhhhhhhhhhhhhhppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxwusqppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxwusqppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxwusqppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxwusqppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxwusqppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxwusqppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxwusqppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxwusqppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxwusqppppppppppppppppppppppppppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxwusqppppppppppppppppppppppxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx}{yxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx}{yxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx}{yxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx}{yxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx}{yxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx}{yxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx}{yxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx}{yxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx}{yxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx}{yxxxxxxxxxxxxxxxxxxxxxx \ No newline at end of file diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-rotating-grid-local-warp.avif b/tests/Images/Input/Heif/Av1/Conformance/libavif-rotating-grid-local-warp.avif new file mode 100644 index 0000000000..dfd8a44293 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-rotating-grid-local-warp.avif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:990bac4ad443005c217b0da4fccfa9adfb3aa147ad06c85f9a655a4433e9e8a7 +size 3307 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-superres-cosmos-10b.avif b/tests/Images/Input/Heif/Av1/Conformance/libavif-superres-cosmos-10b.avif new file mode 100644 index 0000000000..7988633ebc --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-superres-cosmos-10b.avif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:068f67f4d5b009a4cf110dbc66533400ded511d34f2a50ee03c89edbd64e1031 +size 18702 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-superres-cosmos-12b.avif b/tests/Images/Input/Heif/Av1/Conformance/libavif-superres-cosmos-12b.avif new file mode 100644 index 0000000000..6300d457ac --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-superres-cosmos-12b.avif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:036cd5e4bdeeee903facb7de06d39a9fb58ad40a9bf01e290d0f1f2c0088d37c +size 16356 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-superres-kodim23-8b.avif b/tests/Images/Input/Heif/Av1/Conformance/libavif-superres-kodim23-8b.avif new file mode 100644 index 0000000000..d3f9bd0fbc --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-superres-kodim23-8b.avif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e242db0c147f8a893f28546e8ac951b1c823d862692b509940a9fd6bca0b9056 +size 12577 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-webp-logo-average-compound-libaom.y4m b/tests/Images/Input/Heif/Av1/Conformance/libavif-webp-logo-average-compound-libaom.y4m new file mode 100644 index 0000000000..37be945f1a --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-webp-logo-average-compound-libaom.y4m @@ -0,0 +1,3 @@ +YUV4MPEG2 W80 H80 F25:1 Ip A0:0 C444 XYSCSS=444 XCOLORRANGE=LIMITED +FRAME +?@????????UVTTTTTTTTUvyTTtyT룰;=AB=Tdtttttv_T>BBBBBBBBBBBBBBBBBBBBB\tsXT\}}}|zyyyttttqqqqqqqqqqqqqqqqqqqqqqqqqqqrqoWU>KTWnvvvvvvvvvvvvvvvvnouonojnmnnkjgggggggggggggggggggggggggggggggcWTJBTTTUTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTB?BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB>뀀!)*)#"")37673(""55?FFF?73"ffffffffff")5ALRWRLB7*"\[[[[[[[[[""5ALWbcaWK?3"\MBBBBBBO[")7FRbotj^QB7)"\NBBBBBBNZ+'")91),(!''((''('"().38612..))())))),3769BGGNTU[oo[^aJAAAAAAAAAAA@A@@@@@@@@@@D@DDDDDDDDDDDDDDD:>621*---.---33678AFGGMTU[oo[^aJAAAAAAAAAAA@@@@@@@@@@@DDDDDDDDDDDDDDDDDDDB>561113233187=8CGGGGNTU[oo[^aJAAAAAAAAAA@@@@@@@@@@D@DDDDDDDDDDDDDDDDDBBBB9==6667675=8BFGGGGGNTU[oo[^aJAAAAAAAA@@@@@@@@@@D@DDDDDDDDDDDDDDDDDDBBBBBBAB98=>=8BBFGGGGGFFMTU[oo[^aJAAAAA@A@@@@@@@@@@DDDDDDDDDDDDDDDDDCDBBBBBBBBBB@G@FAGHGGGGGGGFFMTU[oo[^_JAAA@A@@@@@@@@@@DDDDDDDDDDDDDDKNOKECBBBBBBBBB@@AGGGGGGGGGGGGGFFMTU[oo[^_JAAA@@@@@@@@@@D@DDDDDDDEEDFQXZ\[[\ZWQKBBBBA@@F@GGGGGGGGGFFFFFFFLTU[oo[^^J@@@@@@@@@@@D@DDDDDDDDDEFS[[\fgjigf[[[RCBC@GAGGGGGGGGGGFFFFFFCCLSU[oo[^^J@@@@@@@@@@DDDDDDDDDDDCH\[]q|~|q]Y\NF@FFGGGGGGGGFFFFFFCCAALRT[oo[^^J@@@@@@@@DDDDDDDDDDDDCM[Zm~mZZPGGGGGGGGGFFFFFFFCAAAALRT[oo[]]J@@@@@@D@DDDDDDDDDDDCI[]ss][MGGGGGGFFFFFFFCAAAAAALQT[oo[]]J@?@@D@DDDDDDDDDDDDDF][ut[]JGGGGFFFFFFCCAAAAAAALQS[oo[]]J@??DDDDDDDDDDDDDDDDT[m~m[TGGGGFFFFCCAAAAAAAAALPS[oo[]]J@D@DDDDDDDDDDDDDDDI[]~][HFFFFFFCAAAAAAAAAAGLPS[oo[]]J@DDDDDDDDDDDDDDDDDS[qq[SFFGHBBAAAAAAAAKJJJKOS[oo[]\JDDDDDDDDDDDDDDDCDCV[~~[\BFCBBAAAAAAAAAJJJJKNS[oo[]ZJDDDDDDDDDDDDDDDCCJ\ed[FCBBAAAAABBAJJJJJJKMS[oo[]ZJDDDDDDDDDDDDDBBBBH\kjZJBBAAAAABCIJJJIJJIJKS[oo[[ZJDDDDDDDDDDDDBBBBBN[nm[QBAAAAABIJJJJJIHHHJJS[oo[[YJDDDDDDDDDDBBBBBAAL[onZPBAAABBIJJJJIHHHHHIJS[oo[[YJDDDDDDDDBBBBBBBABM\jj[JAABBIJJJJIHHHHHHHHIS[oo[[YJDDDDDDDBBBBBBBBAAJ\edZMAKIIJJJJIHHHHHHFFHIS[oo[[ZIDDDCDBBBBBBBBBB@GG\\\WJIJJJJJJHHHHHHFFFFHHS[oo[[WIDDDCCBBBBBBBB@G@GGRZtq[SJIJJJJHHHHHHHFFFFFFHS[oo[[VIDCCBBBBBBBB@@@GGGGN\b~~b\JJIIIHHHHHHHFFFFFFFFHS[no[ZUIBBBBBBBBB@@G@GGGGGFT[q~qZYJJIIHHHHHHFFFFFFFFGFLSZoo[ZUIBBBBBBBB@G@GGGGGGGGIZ[xv][LIIHHHHHHFFFFFFFFFHHHGR[oo[ZUIBBBAAB@@@GGGGGGGGGGGKZ\wv]\QKHHHHHHHFFFFFFFFHHHHHHS[no[YUHBBBAA??GGGGGGGGFFFFFFS[\qq[\QKHHHHHHFFFFGGGGHHHHHHHIS[oo[YVGBBAAF@GGGGGGGGGFFFFFGFR[[atvb[[PHIHHHHFFFFFGGGGHHHHHHHHIS[oo[YVGB@G@GGGGGGGGGGFFFFFFCCAMY[Z]gpqqog]Z[YJHHHHHHFFFFFFFGHHHHHHHHHHIS[oo[ZVHA@GGGGGGGGGGFFFFFFCCAAAAAJS[Z[[[\\ZSQIHHHHHFFFFFFFFHHHHHHHHHHHHIS[oo[ZVHHHGGGGGGGGGFFFFFFCAAAAAAA?AIJPRRPJEHGHHHHFFFFFFFFHHHHHHHHHHHHHHIS[oo[ZUHHHGGGGGGFFFFFFFCCAAAAAAABBIJJJJJJIHHHHHFFFFFFFFHHHHHHHHHHHHHHHHIS[oo[ZTHHHGGGGGFFFFFFCCAAAAAAAAAAJJJJJJHHHHHHGGFFFFFFFHHHHHHHHHHHHHHHHHIS[oo[ZSGGGGGGGGFFFFFCAAAAAAAAAAIJJJJIIIHHHHGFFFFFFFFHHHHHHHHHHHHHHHHHHHIS[oo[YRGGGGGFFFFEECAAAAAAAAAJJJJJJJIHHHHGGFFFFFFFFHHHHHHHHHHHHHHHHHHHHHIS[oo[YQGGGFFFFFFCCAAAAAAAAAJJJJJJJHHHHHGFFFFFFFFFHHHHHHHHHHHHHHHHHHHHHHIS[op[VPGFFFFFFDAAAAAAAAAAAJJJJJIHHHHHHHFFFFFFFFHHHHHHHHHHHHHHHHHHHHHHHHHS[op[VPLFFFFFDAAAAAAAAAAJJJJJJHHHHHHHGFFFFFFFHHHHHHHHHHHHHHHHHHHHHHHHHHHY\r[YPLFFFCAAAAAAAAAKJJJJJJJHHHHHHFFFFFFFFGHHHHHHHHHHHHHHHHHHHHHHHHHHHHY\[[PQLLGFFGJAAKKKKKKKJJJJJHHHHHHFFFFFFFGHHHHHHHHHHHHHHHHHHHHHHHHHHHHK[[e[YQQQQQPKLLLLLLKKKKKKKKKJJIIHHGGFFFFHHHHHHHHHHHHHHHHHHHHHHHHHHHHHJW[f`[WRNRNMNSSLMMMMLSSSQQKQQQQQQQQQQQQPOOOOOOOOOOOOOOOOOOOOOOOOOOPOSV[`e[[[[[[Z[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[\[Z[cfcddcccccccccccccccccbbbccccccccbccccccccccccccccccccccccccccgvvvvvvvvvvpppppppppppnddddddlppkdddddelpuuuuupojjjjjnoottttttttttttttssssssuuy}}|vqqqqppppppppppppppppppppppppppppppppppu}}vputppqqrrrrrrrrrrrrrrssssssrrqqppppppppppppy~|vqppuvqrrrrrrrrrttttttttssssrrrrrrrrrrrrrrrrpprw~yvpnnpvtqsqojjggggggggggggggggggggggggggggggggghiqs|{tokplpspqsodddddddddddddddeehdhghhhhhhggggggggghhmrz|{vojimlnpzpqsidddddddddddddddefghhhhhhhhhgggggggggggglrz}{wsphemoopzzprsidddddddddddddddfgghhhhhhhhggggggggggggghmty|{wtplfeklopzzprtiddddddddddhdhdhghhhhhhhhggggggggggggggggmntyy~{ytspkfffklopzzprtiddddddddddhdhhhhhhhhhhgggggggggggggggggggilttyzz{{{|yutoplfffgklopzzprtiddddddddddhhhhhhhhhhhggggggggggggggggggggggkpptttusttnokeffffgjlopzzprtiddddddddhhhhhhhhhhhgggggggggggggggggggggggggglmpoooplkfffffghhklopzzprsidddddhdhhhhhhhhhhggggggggggggggggggggggggggggggfgigefffffgfiiiklopzzprridddhdhhhhhhhhhhgggggggggggggihlljihgggggggggfggfeeeefffgghgiiiklopzzprridddhhhhhhhhhhhgggggggggghjloopppoppligggggggfffffffffffjiiiiijlmopzzprrhddhhhhhhhhhhgggggggggggiloposvwwvtppoljgggffffffffffffiiiiiiiilmopzzprrhhhhhhhhhhhggggggggggggjpqsz~~zspplegefffffffffiiiiiiiijjklopzzprrhhhhhhhhhgggggggggggggkopxzqomeefffffffiiiiiiiijjjjkjopzzppphhhhhhhhgggggggggggggloqzzppjefffffiiiiiiiijjjjjjjiop{zppphhhhggggggggggggggggioq{zpphffffiiiiiiiijjjjjjijiop{zppphhhhggggggggggggggggopyypofghhiiiiiijjjjjjjjijjop{zppphgggggggggggggggggggortoliiiiiiijjjjjjjjjjhjlopzzppphgggggggggggggggggglpyyoliiiijjjjjjjjjjjiihilopzzppphggggggggggggggggggppopiijjjjjjjjjjjiihihklopzzppphggggggggggggggghgiotsqjjjjjjjjjjjihhhhhhllopzzppphggggggggggggggghgkowvqjjjjjjjjjjhhhhhhhillopzzppphggggggggggggggghhjqyypljjjjjjjhhhhhhhhhiklopzzppphgggggggggggggggggjpxypmjjjjjjhhhhhhhhhhhhlopzzppphgggggggggggggggggkpvwpkjjjjhhhhhhhhhhhhhhlopzzppphgggggggggggggggggipstpjiihhhhhhhhhhhhhiiilopzzppphggggggggggggggggeepqpqhhhhhhhhhhhhhhiiiiilopzzppphggggggggggggggffeelp{~|qohhhhhhhhhhhhhiiiiiikopzzppphggggggggggggggffeelpsspkhhhhhhhhhhhiiiiiiiijopzzppphgggggggggggffffffffoqzvpohhhhhhhhhhhhiiiiiiiijopzzppphgggggggggffffffffffhpq}}ppihhhhhhhhhiiiiiiiihhhiopzzppphgggggggggefffffffffikqr||spihhhhhhhhiiiiiiiihhhhhiopzzpppigggggggeeeeffffefhiiilppzzpqlhhhhhhhiiiiiiiihhhhhhhiopzzpppigggggggeeefffffiiihiiilpoqz|rppihhhhhhiiiiiiiihhhhhhhhhiopzzppohffffffffffffghhiiiiiiijjnoppuuz{uvqopnjhhhhhhiiiiiiiihhhhhhhhhhiopzzppmiggfffffffffgghhiiiiijjjjjlnpqppppponigihhhhiiiiiiiihhhhhhhhhhhhiopzzqpliffffffffffgiiiiiiijjjjjjjjjikjllljihhhhhhiiiiiiiihhhhhhhhhhhhhhiopzzqplifffffffghhiiiiiiijjjjjjjjjihhhhhhhhhhhiiiiiiiiihhhhhhhhhhhhhhhhiopzzqplifffffffhiiiiiiijjjjjjjjjiiihhhhhhhhhhiiiiiiiiihhhhhhhhhhhhhhhhhiopzzpplifffffhgiiiiiijjjjjjjjjjihhhhhhhhhhhiiiiiiiiihhhhhhhhhhhhhhhhhhhiopzzppmifffiiiiiiiijjjjjjjjjhhhhhhhhhhhhhhiiiiiiiihhhhhhhhhhhhhhhhhhhhhiopz{ppmiffiiiiiiijjjjjjjjjjhhhhhhhhhhhhhiiiiiiiiihhhhhhhhhhhhhhhhhhhhhhiopzzpoliiiiiiiijjjjjjjjjjjhhhhhhhhhhhhhiiiiiiiihhhhhhhhhhhhhhhhhhhhhhhhhopzzpolkiiiiiiijjjjjjjjjhhhhhhhhhhhhhiiiiiiiiiihhhhhhhhhhhhhhhhhhhhhhhhhnozpnlkjiiiiiijjjjjjhhhhhhhhhhhhhhiiiiiiiiihhhhhhhhhhhhhhhhhhhhhhhhhhhhnpppljkjiiiiijjhhhhhhhhhhhhhhhhhhiiiiiiihhhhhhhhhhhhhhhhhhhhhhhhhhhhhkopupniijjiijklllllllllllllllllglhhhijjiihhhhhhhhhhhhhhhhhhhhhhhhhhhhkppwspomnlnnmmlnnnnnnlllmmnmmmimmmmiimljjiiiiiiiiiiiiiiiiiiiiiiiiiihoppsupopqppppppppppppppppppppppppqppppppppppppppppppppppppppppppppppotwttttttttttttttttttttttsttttttttsttttttttttttttttttttttttttttv \ No newline at end of file diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-webp-logo-average-compound.avif b/tests/Images/Input/Heif/Av1/Conformance/libavif-webp-logo-average-compound.avif new file mode 100644 index 0000000000..2d6139d189 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-webp-logo-average-compound.avif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7919049d367eedb7c965e170309d6759660ddbfd4bb1aef9496f9d66e314846a +size 6530 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-webp-logo-difference-weighted-compound-libaom.y4m b/tests/Images/Input/Heif/Av1/Conformance/libavif-webp-logo-difference-weighted-compound-libaom.y4m new file mode 100644 index 0000000000..2d3f2f87e2 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-webp-logo-difference-weighted-compound-libaom.y4m @@ -0,0 +1,3 @@ +YUV4MPEG2 W80 H80 F25:1 Ip A0:0 C444 XYSCSS=444 XCOLORRANGE=LIMITED +FRAME +>>>>>>>>>>TTTTTTTTTTTv{UUs{U룱:>BB>Tctttttt^T?BCBBBBBBBBBBBBBBBBBBB\ssY>TTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTcrsdT>ASVxWTB:Ti½\T:ITǻTJTi˽\T-T˰jT--TɰjT--TȰjT--TȰjT--TȰjT--TȰjT--TȰjT--T{zjT--To\STTUUTXnjT--TgTUWz{VTUfjT--TzVUeeUVz~~jT--TtUTTTo~~~~jT--TzTVVTu~~~~~~cT--TVUTV|~~~~~~~cT--TbUT^~~~~~~~~~cT--TTdeTz~~~~~~~~~~}cT--TeUUf~~~~~~~~~}}|cT--TWVVV~~~~~~~~~~}|||cT--TT{{St~~~~~~~~~~~||||{cT--TyTTn~~~~~~~~~~}|||{|z}cT--TsTTo~~~~~~~}|||||{zzy}cT--TtTTn~~~~~~}|||||zyyyy|}^T--TzTTn~~~}}}||||{yyyyxxz}_T--T|T{|Tu~~}}|||||yyyyyyuuy}]T--TVVWW||||||||yyyyyyttttyz^T--TfUT^||||||yyyyyyytttttty^T--TyTlkTm||||yyyyyyytttttttty^T--T_TS\||||yyyyyyttttttttrtt^T--TwzUVVTt{{yyyyyytttttttttqqqs^T--TwnUddTj}yzyyyyyttttttttqqqqqq^T--TwgTVVTf|zyyyyyttttttttqqqqqqqq^T--TwnTTllTTjyyyyyyutttttttqqqqqqqqqq^T--Tw~t]TTWWST\nyyyyyyttttttttqqqqqqqqqqq^T--Tw~~~~}n_TTUUTTSS]kyyyyyyttttttttqqqqqqqqqqqqq^T--Tw~~~~~~~~~xojegmotyyyyyyttttttttqqqqqqqqqqqqqqq^T--Tw~~~~~~~~~|||||{{zyyyyyttttttttqqqqqqqqqqqqqqqqq^T--Tw~~~~~~~~~~}|||||yyyyyyyttttttttqqqqqqqqqqqqqqqqqq^T--Tw~~~~~~~~~~||||||yyyyyyyttttttttqqqqqqqqqqqqqqqqqqqq^T--Tw~~~~~~~~~}||||||yyyyyyttttttttqqqqqqqqqqqqqqqqqqqqqq^T--Tw~~~~~~~~~|||||||yyyyyytttttttttqqqqqqqqqqqqqqqqqqqqqqq^T-.Uo~~~~~~~~~~|||||{yyyyyyyttttttttqqqqqqqqqqqqqqqqqqqqqqqqq]T-,Uj~~~~~~~~~~||||||zyyyyyyttttttttrqqqqqqqqqqqqqqqqqqqqqqqqqq]T-T\~~~~~~~~~}}|||||zzyyyyttttttttqqqqqqqqqqqqqqqqqqqqqqqqqqqqq]TTT|||||zzyyyytttttttqqqqqqqqqqqqqqqqqqqqqqqqqqqqqnTT>T\}}||yyyyttttqqqqqqqqqqqqqqqqqqqqqqqqqqqqqnWT>JTWovwvvvvvvvvvvvvvwnnvnnnjnnnnjjfggggggggggggggggggggggggggggggcWTJBTUTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTUC>BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB>뀀")(("#")37883)$"37?FFE>72"efffffffff"*7BKRWRLB7)!\[[[[[[[[[""3?LWbcbXL?3#[MBBBBBBN[")7FSbouj_RB8)"[MBBBBBBN[+&#)6"$/7AIP\gd\\[[]_____________^^^^]]]]\\\\\\\\\\\\]UFB;6/(.:1)-))!!"'&"")*-3778@GPTU[oo[]bJAAAAAAAAAAAA@@@@@@@@@@@@@D@DDDDDDDDDDDDD>613.-**)))))*.3768BGGMTU[oo[^aJAAAAAAAAAAA@@@@@@@@@@@@D@DDDDDDDDDDDDDDD:=511*-------31678BGGGMTU[oo[^aJAAAAAAAAAA@@@@@@??????CDDDDDDDDDDDDDDDDDDB>661112222166=8BGGGHMTU[oo[^aJAAAAAAAA@@@@@@@?????C?DDDDDDDDDDDDDDDDDBBBB:==6666666<8BFGGGGGMTU[oo[^aJAAAAAAA@@@@@@@@@@@D@DDDDDDDDDDDDDDCDDDBBBBBBBB98=<=8BBHGGGGGFGMTU[oo[^aJAAAAA@A@@@@@@@@@@DDDDDDDDDDDDDDEDDDDBBBBBBBBBB?FAF@GGHGGGGGFFFMTU[oo[^^JAAA@A@@@@@@@@@@DDDDDDDDDDDDDDMMNKFBBBBBBBBBB??@GGGGGHGGGGFFFFFMTU[oo[^^JA@@@@@@@@@@@@D@DDDDDDDDDCGPZ[[[[[[XRJBBBBB@@HAHHHGGGGGGFFFFFFFLTU[oo[^^J@@@@@@@@@@@D@DDDDDDDDDDERZ[\fgjjgf[[\RDCB>H@GGHGGGGGGGFFFFFFCCLST[oo[^^J@@@@@@@@@@DDDDDDDDDDDDI\[\q||q\[\NG@GGGGGGGGGGFFFFFFCCAALST[oo[^^J@@@@@@@@DDDDDDDDDDCDDN[[n~n[[QFGGGGGGGGFFFFFFFCAAAALRT[oo[]^J@@@@@@D@DDDDDDDDDCDDH[[tt[[MGGGGGGFFFFFFFCAAAAAALRS[oo[]]J@@@@D@DDDDDDDDDDDDDF\[ut\]JGGGGFFFFFFCCAAAAAAALQS[oo[]]J@@@DDDDDDDDDDDDDDDDR\mm\SGGGEFFFFCCAAAAAAAAAKQSZoo[]]J@E@DDDDDDDDDDDDDDDK[^]\IFFGFFFCAAAAAAAAAAKKQSZoo[]^J@DDDDDDDDDDDDDDDDDS[qq[RGEFFCCAAAAAAAAJJJJJPSZoo[]\JDDDDDDDDDDDDDDDCDBW[~[]CFCCAAAAAAAAAAJJJJJOSZoo[\YJDDDDDDDDDDDDDDDBCK\ed[FCAAAAAAAAAAJJJJJJJMS[oo[\YJDDDDDDDDDDDDDBBBBG[kk\JAAAAAAAAAJJJJJJJIJLS[oo[[YJDDDDDDDDDDDDBBBBBM[nm[QAAAAAAAJJJJJJJHHHJKS[oo[[YJDDDDDDDDDDBBBBBBAM[om[QAAAAAAJJJJJJHHHHHJJS[oo[[YJDDDDDDDDBBBBBBBACN[kj[JABAAJJJJJJHHHHHHHHIS[oo[[YJDDDDDDDBBBBBBBBA?I[edZKBKJJJJJJJHHHHHHGGHIS[oo[[YIDDDCDBCBBBBBBBB@FH[\\WJIJJJJJJHHHHHHHGFFFHS[oo[[WIDDDBCBBBBBBBB@G@GGRZuq[TJJJJJJHHHHHHHGFFFFFHS[oo[[VIDCCBBBBBBBB@@@GFGGNZb~b[KJJIIHHHHHHHGFFFFFFGHS[oo[[UIBBBBBBBBB@@G@GGGGGGSZqqZYJJIIHHHHHHFFFFFFFGHHMS[oo[[UIBBBBBBBB@G@GGGGGGGGH[\vv\ZNIIHHHHHHFFFFFFFFGHHHGS[oo[ZUIBBCBAB@@@GGGGGGGGGGGJZ]ww][RJIHHHHHHFFFFFFFGHHHHHIS[op[ZVHBCBBA@?GGGGGGGGFFFFGFR[[qq[[RJHHHHHHFFFFFFFGHHHHHHHIS[op[ZVHBBBAF@GGGGGGGGGFFFFFGFR[[bttb[[QHIHHHHFFFFFFFFHHHHHHHHHIS[op[ZVHBAG@GGGGGGGGGGFFFFFFCCALZ[\\goqqog\[[YKHIHHHHFFFFFFFFHHHHHHHHHHIS[oo[ZUHAAGGGGGGGGGGFFFFFFCCAAAA@JS\\[[[[[[SPHHHHHHGFFFFFFFHHHHHHHHHHHHIS[oo[ZUHHGGGGGGGGGFFFFFFFCAAAAAAAA@JJRRRQIHHHHHHHFFFFFFFFHHHHHHHHHHHHHHIS[oo[ZUGGGGGGGGGFFFFFFFCBAAAAAAAAAJJJJJJIHHHHHHFFFFFFFGHHHHHHHHHHHHHHHHIS[oo[ZTGGGGGGGGFFFFFFBBAAAAAAAAAAJJJJJIIIHHHHHEFFFFFFGGHHHHHHHHHHHHHHHHIS[oo[ZRGGGGGGGFFFFFFBAAAAAAAAAAKJJJJIIIHHHHIEEFFFFFGGHHHHHHHHHHHHHHHHHHIS[oo[ZQGGGGGGFFFFFBAAAAAAAAAJJJJJJIIHHHHHIEEFFFFFGGHHHHHHHHHHHHHHHHHHHHIS[oo[ZQGGGGGGFFFBAAAAAAAAAAJJJJJJIIHHHHIEFFFFFFFGHHHHIHHHHHHHHHHHHHHHHHIS[op[VPFFFFFFFDAAAAAAAAAABKJJJIHHHHHHHHFFFFFFFFHHHHHHHHHHHHHHHHHHHHHHHHHS[pp[VPKGFFFFDAAAAAAAAAAJJJJJIIHHHHHHFFFFFFGGGGHHHHHHHHHHHHHHHHHHHHHHHHHW[p[YPMFFGCAAAAAAAAAKKJJJJJJIHHHHHFFFFFFFFHHHHHHHHHHHHHHHHHHHHHHHHHHHHHW[[[PQNLEFFGJAAKKKKKKKKJJJJIHHHHHFFFFFFFGHHHHHHHHHHHHHHHHHHHHHHHHHHHHH[[e[YQQQQQQLLLLLLKKKKKKKKKKIIIIHHHHFFFFHHHHHHHHHHHHHHHHHHHHHHHHHHHHHMW[f`[WRNSMLMSSMMMMMMSRRQQMQQQQQQQQQQQQPPPPPOOOOOOOOOOOOOOOOOOOOOOMMSW[ce[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[Z[[[[[[[[[[[[[[[[[[[[[[[[[[[[[cfcbbbccccccccccccccccccccccccccccccccccccccccccccccccccccccccfvwwwvvvvvvqqqqppppppqnddddddmppjddddddmpuuutvqpjjjjjmopttttttttttttttttttttttx}}}vpppppppppppppppppppppppppppppppppppppqw}}wpvtpqqqrrrrrrrrrrrrrrrrssssssrqpppppppppppqy}|vqpptvorqrrssssstttttttttssrrrrrrrrrrrrsssssporx~yvonnpvtptrojiggggggggggggggggggggggggggggggggggirq{{tplplptprsodddddddddddddddddheghhhhhhhhhggggggggglry|{vokklmopzprsidddddddddddddddddhhhhhhhhhhggggggggggggkrz}{vtqiemnopzzprsiddddddddddddeddhhhhhhhhhhghgggggggggggggmty{{vtqkfeklopzzprsiddddddddddfdhefhhhhhhhhhgggggggggggggggflotyy|{yttpkffgjlopzzprsiddddddddddhdfhhhhhhhhhgggggggggggggggggggjltuzyz{{{{zxtpqjffffklopzzprsiddddddddddfhhhhhhhhhggggggggggggggggggggggglopustttttppkefffffklopzzprtiddddddddghhhhhhhhhhggggggggggggggggggggggggggjlooooqkkeffffhhhllopzzprtidddddhdghhhhhhhhhgggggggggggggggggggggggggggggfgghgffffffgghiillopzzprsidddhdhhhhhhhhhhggggggggggggggjjjihgggggggggggggeeeeffffggigiihklopzzprsgdddhhhhhhhhhhggggggggggggimpppoopppmiggggggffffeeffffffiiiiiiiklopzzpqshddhhhhhhhhhhgggggggggggimppptvwwvtpppmigggfffffffffffghiiiiiiiklopzzpqrhhhhhhhhhhhggggggggggggjpprz~~zrpplfffffffffffghiiiiiiijjklopzzpqrhhhhhhhhhgggggggggggggjopyypomffffffffghiiiiiiijjjjjjopzzpprhhhhhhhggggggggggggggkppzzpojfffffggiiiiiiijjjjjjkjopzzppqhhhhhghgggggggggggggippzzqohffggiiiiiiiijjjjjjjkjopzzppphhhhggggggggggggggggppyxpogghhiiiiiijjjjjjjjjijopzzppphhghggggggggggggggggpssqlhhiiiiijjjjjjjjjjhhlopzzppphhggggggggggggggggglpyyplihhiiijjjjjjjjhhhhhlopzzppphggggggggggggggggggppppiiiiijjjjjjjjjhhhhllopzzppqhggggggggggggggghginttpjiijjjjjjjjjhhhhhhllopzzppqhggggggggggggggghgjowwpkjijjjjjjjhhhhhhhhllopzzppqhgggggggggggggggggjoxypljjjjjjjhhhhhhhhhhllopzzppqhgggggggggggggggggjpxypljjjjjjhhhhhhhhhhhhlopzzqpqhgggggggggggggggggjpvwpkjjjjhhhhhhhhhhhhhhlopzzqpphgggggggggggggggggiprtpjjihhhhhhhhhhhhhhihlopzzqpphggggggggggggggggfeqpoqihhhhhhhhhhhhhhiiiilopzzqpphggggggggggggggfffelp{{pohhhhhhhhhhhhhiiiiiikopzzqpphgggggggggggggfffffkqqspjhhhhhhhhhhhiiiiiiiikopzzqpphgggggggggggffeefffeqqy~wpohhhhhhhhhhiiiiiiiiihjopzzqpphgggggggggfgeeffffggiqp}}ppihhhhhhhhiiiiiiiiihhhjopzzqpphgggggggggeeefffffggijqr}}tpihhhhhhhhiiiiiiiihhhhhjopzzqpphgggggggeeeeffffefhihimppzzppkhhhhhhhiiiiiiiihhhhhhhiopzzqpphgggggggeeefffffiiiiiiimppr{{rppihhhhhhiiiiiiiihhhhhhhhhiopzzponiggfgfffffffffghiiiiiiijlnpppvvyyvvpppnlhhhhhhiiiiiiiihhhhhhhhhhiopzzpoliggfffffffffghhhiiiiijjjjjjoppppppppojhhhhhhiiiiiiiihhhhhhhhhhhhiopzzpolifffffffffghiiiiiiijjjjjjjjjikkkkkkhhhhhhhiiiiiiiihhhhhhhhhhhhhhiopzzqplifffffffghiiiiiiiijjjjjjjjjhhhhhhhhhhhhhiiiiiiiihhhhhhhhhhhhhhhhiopzzqplifffffffiiiiiiiijjjjjjjjjjhhhhhhhhhhhhhiiiiiiihhhhhhhhhhhhhhhhhhiopzzqplifffffifiiiiiijjjjjjjjjjhhhhhhhhhhhhhiiiiiiiihhhhhhhhhhhhhhhhhhhiopzzpplifffiiiiiiiijjjjjjjjjihhhhhhhhhhhhhiiiiiiiiihhhhhhhhhhhhhhhhhhhhiopzzppliffiiiiiiijjjjjjjjjjhhhhhhhhhhhhhiiiiiiiiiihhhhhhhhhhhhhhhhhhhhhiopzzpoliiiiiiiijjjjjjjjjjjhhhhhhhhhhhhhiiiiiiiihhhhhhhhhhhhhhhhhhhhhhhhiopzzpolkiiiiiiijjjjjjjjjjhhhhhhhhhhhhiiiiiiiiiihhhhhhhhhhhhhhhhhhhhhhhhiopzpolkjiiiiiijjjjjjihihhhhhhhhhhhiiiiiiiihhhhhhhhhhhhhhhhhhhhhhhhhhhhiopppmjkjiiiiiiihhhhhhhhhhhhhhhhhhiiiiiiihhhhhhhhhhhhhhhhhhhhhhhhhhhhhippupnjjiiiijkkkkklllllllllllllhlhhhiiiiihhhhhhhhhhhhhhhhhhhhhhhhhhhhjopvspomnmnmmmmnnnnnnlllllmmnnimmmmiimljiiiiiiiiiiiiiiiiiiiiiiiiiijjopptupppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppptvuuttttttttttttttttttttttttttttttttttttttttttttttttttttttttttv \ No newline at end of file diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-webp-logo-difference-weighted-compound.avif b/tests/Images/Input/Heif/Av1/Conformance/libavif-webp-logo-difference-weighted-compound.avif new file mode 100644 index 0000000000..c91aaa7010 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-webp-logo-difference-weighted-compound.avif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fc6459cd334762d74d9d2654640221e80c463cc01b82b29a5866c9e725abe273 +size 6423 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-webp-logo-distance-weighted-compound-libaom.y4m b/tests/Images/Input/Heif/Av1/Conformance/libavif-webp-logo-distance-weighted-compound-libaom.y4m new file mode 100644 index 0000000000..2d3f2f87e2 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-webp-logo-distance-weighted-compound-libaom.y4m @@ -0,0 +1,3 @@ +YUV4MPEG2 W80 H80 F25:1 Ip A0:0 C444 XYSCSS=444 XCOLORRANGE=LIMITED +FRAME +>>>>>>>>>>TTTTTTTTTTTv{UUs{U룱:>BB>Tctttttt^T?BCBBBBBBBBBBBBBBBBBBB\ssY>TTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTcrsdT>ASVxWTB:Ti½\T:ITǻTJTi˽\T-T˰jT--TɰjT--TȰjT--TȰjT--TȰjT--TȰjT--TȰjT--T{zjT--To\STTUUTXnjT--TgTUWz{VTUfjT--TzVUeeUVz~~jT--TtUTTTo~~~~jT--TzTVVTu~~~~~~cT--TVUTV|~~~~~~~cT--TbUT^~~~~~~~~~cT--TTdeTz~~~~~~~~~~}cT--TeUUf~~~~~~~~~}}|cT--TWVVV~~~~~~~~~~}|||cT--TT{{St~~~~~~~~~~~||||{cT--TyTTn~~~~~~~~~~}|||{|z}cT--TsTTo~~~~~~~}|||||{zzy}cT--TtTTn~~~~~~}|||||zyyyy|}^T--TzTTn~~~}}}||||{yyyyxxz}_T--T|T{|Tu~~}}|||||yyyyyyuuy}]T--TVVWW||||||||yyyyyyttttyz^T--TfUT^||||||yyyyyyytttttty^T--TyTlkTm||||yyyyyyytttttttty^T--T_TS\||||yyyyyyttttttttrtt^T--TwzUVVTt{{yyyyyytttttttttqqqs^T--TwnUddTj}yzyyyyyttttttttqqqqqq^T--TwgTVVTf|zyyyyyttttttttqqqqqqqq^T--TwnTTllTTjyyyyyyutttttttqqqqqqqqqq^T--Tw~t]TTWWST\nyyyyyyttttttttqqqqqqqqqqq^T--Tw~~~~}n_TTUUTTSS]kyyyyyyttttttttqqqqqqqqqqqqq^T--Tw~~~~~~~~~xojegmotyyyyyyttttttttqqqqqqqqqqqqqqq^T--Tw~~~~~~~~~|||||{{zyyyyyttttttttqqqqqqqqqqqqqqqqq^T--Tw~~~~~~~~~~}|||||yyyyyyyttttttttqqqqqqqqqqqqqqqqqq^T--Tw~~~~~~~~~~||||||yyyyyyyttttttttqqqqqqqqqqqqqqqqqqqq^T--Tw~~~~~~~~~}||||||yyyyyyttttttttqqqqqqqqqqqqqqqqqqqqqq^T--Tw~~~~~~~~~|||||||yyyyyytttttttttqqqqqqqqqqqqqqqqqqqqqqq^T-.Uo~~~~~~~~~~|||||{yyyyyyyttttttttqqqqqqqqqqqqqqqqqqqqqqqqq]T-,Uj~~~~~~~~~~||||||zyyyyyyttttttttrqqqqqqqqqqqqqqqqqqqqqqqqqq]T-T\~~~~~~~~~}}|||||zzyyyyttttttttqqqqqqqqqqqqqqqqqqqqqqqqqqqqq]TTT|||||zzyyyytttttttqqqqqqqqqqqqqqqqqqqqqqqqqqqqqnTT>T\}}||yyyyttttqqqqqqqqqqqqqqqqqqqqqqqqqqqqqnWT>JTWovwvvvvvvvvvvvvvwnnvnnnjnnnnjjfggggggggggggggggggggggggggggggcWTJBTUTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTUC>BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB>뀀")(("#")37883)$"37?FFE>72"efffffffff"*7BKRWRLB7)!\[[[[[[[[[""3?LWbcbXL?3#[MBBBBBBN[")7FSbouj_RB8)"[MBBBBBBN[+&#)6"$/7AIP\gd\\[[]_____________^^^^]]]]\\\\\\\\\\\\]UFB;6/(.:1)-))!!"'&"")*-3778@GPTU[oo[]bJAAAAAAAAAAAA@@@@@@@@@@@@@D@DDDDDDDDDDDDD>613.-**)))))*.3768BGGMTU[oo[^aJAAAAAAAAAAA@@@@@@@@@@@@D@DDDDDDDDDDDDDDD:=511*-------31678BGGGMTU[oo[^aJAAAAAAAAAA@@@@@@??????CDDDDDDDDDDDDDDDDDDB>661112222166=8BGGGHMTU[oo[^aJAAAAAAAA@@@@@@@?????C?DDDDDDDDDDDDDDDDDBBBB:==6666666<8BFGGGGGMTU[oo[^aJAAAAAAA@@@@@@@@@@@D@DDDDDDDDDDDDDDCDDDBBBBBBBB98=<=8BBHGGGGGFGMTU[oo[^aJAAAAA@A@@@@@@@@@@DDDDDDDDDDDDDDEDDDDBBBBBBBBBB?FAF@GGHGGGGGFFFMTU[oo[^^JAAA@A@@@@@@@@@@DDDDDDDDDDDDDDMMNKFBBBBBBBBBB??@GGGGGHGGGGFFFFFMTU[oo[^^JA@@@@@@@@@@@@D@DDDDDDDDDCGPZ[[[[[[XRJBBBBB@@HAHHHGGGGGGFFFFFFFLTU[oo[^^J@@@@@@@@@@@D@DDDDDDDDDDERZ[\fgjjgf[[\RDCB>H@GGHGGGGGGGFFFFFFCCLST[oo[^^J@@@@@@@@@@DDDDDDDDDDDDI\[\q||q\[\NG@GGGGGGGGGGFFFFFFCCAALST[oo[^^J@@@@@@@@DDDDDDDDDDCDDN[[n~n[[QFGGGGGGGGFFFFFFFCAAAALRT[oo[]^J@@@@@@D@DDDDDDDDDCDDH[[tt[[MGGGGGGFFFFFFFCAAAAAALRS[oo[]]J@@@@D@DDDDDDDDDDDDDF\[ut\]JGGGGFFFFFFCCAAAAAAALQS[oo[]]J@@@DDDDDDDDDDDDDDDDR\mm\SGGGEFFFFCCAAAAAAAAAKQSZoo[]]J@E@DDDDDDDDDDDDDDDK[^]\IFFGFFFCAAAAAAAAAAKKQSZoo[]^J@DDDDDDDDDDDDDDDDDS[qq[RGEFFCCAAAAAAAAJJJJJPSZoo[]\JDDDDDDDDDDDDDDDCDBW[~[]CFCCAAAAAAAAAAJJJJJOSZoo[\YJDDDDDDDDDDDDDDDBCK\ed[FCAAAAAAAAAAJJJJJJJMS[oo[\YJDDDDDDDDDDDDDBBBBG[kk\JAAAAAAAAAJJJJJJJIJLS[oo[[YJDDDDDDDDDDDDBBBBBM[nm[QAAAAAAAJJJJJJJHHHJKS[oo[[YJDDDDDDDDDDBBBBBBAM[om[QAAAAAAJJJJJJHHHHHJJS[oo[[YJDDDDDDDDBBBBBBBACN[kj[JABAAJJJJJJHHHHHHHHIS[oo[[YJDDDDDDDBBBBBBBBA?I[edZKBKJJJJJJJHHHHHHGGHIS[oo[[YIDDDCDBCBBBBBBBB@FH[\\WJIJJJJJJHHHHHHHGFFFHS[oo[[WIDDDBCBBBBBBBB@G@GGRZuq[TJJJJJJHHHHHHHGFFFFFHS[oo[[VIDCCBBBBBBBB@@@GFGGNZb~b[KJJIIHHHHHHHGFFFFFFGHS[oo[[UIBBBBBBBBB@@G@GGGGGGSZqqZYJJIIHHHHHHFFFFFFFGHHMS[oo[[UIBBBBBBBB@G@GGGGGGGGH[\vv\ZNIIHHHHHHFFFFFFFFGHHHGS[oo[ZUIBBCBAB@@@GGGGGGGGGGGJZ]ww][RJIHHHHHHFFFFFFFGHHHHHIS[op[ZVHBCBBA@?GGGGGGGGFFFFGFR[[qq[[RJHHHHHHFFFFFFFGHHHHHHHIS[op[ZVHBBBAF@GGGGGGGGGFFFFFGFR[[bttb[[QHIHHHHFFFFFFFFHHHHHHHHHIS[op[ZVHBAG@GGGGGGGGGGFFFFFFCCALZ[\\goqqog\[[YKHIHHHHFFFFFFFFHHHHHHHHHHIS[oo[ZUHAAGGGGGGGGGGFFFFFFCCAAAA@JS\\[[[[[[SPHHHHHHGFFFFFFFHHHHHHHHHHHHIS[oo[ZUHHGGGGGGGGGFFFFFFFCAAAAAAAA@JJRRRQIHHHHHHHFFFFFFFFHHHHHHHHHHHHHHIS[oo[ZUGGGGGGGGGFFFFFFFCBAAAAAAAAAJJJJJJIHHHHHHFFFFFFFGHHHHHHHHHHHHHHHHIS[oo[ZTGGGGGGGGFFFFFFBBAAAAAAAAAAJJJJJIIIHHHHHEFFFFFFGGHHHHHHHHHHHHHHHHIS[oo[ZRGGGGGGGFFFFFFBAAAAAAAAAAKJJJJIIIHHHHIEEFFFFFGGHHHHHHHHHHHHHHHHHHIS[oo[ZQGGGGGGFFFFFBAAAAAAAAAJJJJJJIIHHHHHIEEFFFFFGGHHHHHHHHHHHHHHHHHHHHIS[oo[ZQGGGGGGFFFBAAAAAAAAAAJJJJJJIIHHHHIEFFFFFFFGHHHHIHHHHHHHHHHHHHHHHHIS[op[VPFFFFFFFDAAAAAAAAAABKJJJIHHHHHHHHFFFFFFFFHHHHHHHHHHHHHHHHHHHHHHHHHS[pp[VPKGFFFFDAAAAAAAAAAJJJJJIIHHHHHHFFFFFFGGGGHHHHHHHHHHHHHHHHHHHHHHHHHW[p[YPMFFGCAAAAAAAAAKKJJJJJJIHHHHHFFFFFFFFHHHHHHHHHHHHHHHHHHHHHHHHHHHHHW[[[PQNLEFFGJAAKKKKKKKKJJJJIHHHHHFFFFFFFGHHHHHHHHHHHHHHHHHHHHHHHHHHHHH[[e[YQQQQQQLLLLLLKKKKKKKKKKIIIIHHHHFFFFHHHHHHHHHHHHHHHHHHHHHHHHHHHHHMW[f`[WRNSMLMSSMMMMMMSRRQQMQQQQQQQQQQQQPPPPPOOOOOOOOOOOOOOOOOOOOOOMMSW[ce[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[Z[[[[[[[[[[[[[[[[[[[[[[[[[[[[[cfcbbbccccccccccccccccccccccccccccccccccccccccccccccccccccccccfvwwwvvvvvvqqqqppppppqnddddddmppjddddddmpuuutvqpjjjjjmopttttttttttttttttttttttx}}}vpppppppppppppppppppppppppppppppppppppqw}}wpvtpqqqrrrrrrrrrrrrrrrrssssssrqpppppppppppqy}|vqpptvorqrrssssstttttttttssrrrrrrrrrrrrsssssporx~yvonnpvtptrojiggggggggggggggggggggggggggggggggggirq{{tplplptprsodddddddddddddddddheghhhhhhhhhggggggggglry|{vokklmopzprsidddddddddddddddddhhhhhhhhhhggggggggggggkrz}{vtqiemnopzzprsiddddddddddddeddhhhhhhhhhhghgggggggggggggmty{{vtqkfeklopzzprsiddddddddddfdhefhhhhhhhhhgggggggggggggggflotyy|{yttpkffgjlopzzprsiddddddddddhdfhhhhhhhhhgggggggggggggggggggjltuzyz{{{{zxtpqjffffklopzzprsiddddddddddfhhhhhhhhhggggggggggggggggggggggglopustttttppkefffffklopzzprtiddddddddghhhhhhhhhhggggggggggggggggggggggggggjlooooqkkeffffhhhllopzzprtidddddhdghhhhhhhhhgggggggggggggggggggggggggggggfgghgffffffgghiillopzzprsidddhdhhhhhhhhhhggggggggggggggjjjihgggggggggggggeeeeffffggigiihklopzzprsgdddhhhhhhhhhhggggggggggggimpppoopppmiggggggffffeeffffffiiiiiiiklopzzpqshddhhhhhhhhhhgggggggggggimppptvwwvtpppmigggfffffffffffghiiiiiiiklopzzpqrhhhhhhhhhhhggggggggggggjpprz~~zrpplfffffffffffghiiiiiiijjklopzzpqrhhhhhhhhhgggggggggggggjopyypomffffffffghiiiiiiijjjjjjopzzpprhhhhhhhggggggggggggggkppzzpojfffffggiiiiiiijjjjjjkjopzzppqhhhhhghgggggggggggggippzzqohffggiiiiiiiijjjjjjjkjopzzppphhhhggggggggggggggggppyxpogghhiiiiiijjjjjjjjjijopzzppphhghggggggggggggggggpssqlhhiiiiijjjjjjjjjjhhlopzzppphhggggggggggggggggglpyyplihhiiijjjjjjjjhhhhhlopzzppphggggggggggggggggggppppiiiiijjjjjjjjjhhhhllopzzppqhggggggggggggggghginttpjiijjjjjjjjjhhhhhhllopzzppqhggggggggggggggghgjowwpkjijjjjjjjhhhhhhhhllopzzppqhgggggggggggggggggjoxypljjjjjjjhhhhhhhhhhllopzzppqhgggggggggggggggggjpxypljjjjjjhhhhhhhhhhhhlopzzqpqhgggggggggggggggggjpvwpkjjjjhhhhhhhhhhhhhhlopzzqpphgggggggggggggggggiprtpjjihhhhhhhhhhhhhhihlopzzqpphggggggggggggggggfeqpoqihhhhhhhhhhhhhhiiiilopzzqpphggggggggggggggfffelp{{pohhhhhhhhhhhhhiiiiiikopzzqpphgggggggggggggfffffkqqspjhhhhhhhhhhhiiiiiiiikopzzqpphgggggggggggffeefffeqqy~wpohhhhhhhhhhiiiiiiiiihjopzzqpphgggggggggfgeeffffggiqp}}ppihhhhhhhhiiiiiiiiihhhjopzzqpphgggggggggeeefffffggijqr}}tpihhhhhhhhiiiiiiiihhhhhjopzzqpphgggggggeeeeffffefhihimppzzppkhhhhhhhiiiiiiiihhhhhhhiopzzqpphgggggggeeefffffiiiiiiimppr{{rppihhhhhhiiiiiiiihhhhhhhhhiopzzponiggfgfffffffffghiiiiiiijlnpppvvyyvvpppnlhhhhhhiiiiiiiihhhhhhhhhhiopzzpoliggfffffffffghhhiiiiijjjjjjoppppppppojhhhhhhiiiiiiiihhhhhhhhhhhhiopzzpolifffffffffghiiiiiiijjjjjjjjjikkkkkkhhhhhhhiiiiiiiihhhhhhhhhhhhhhiopzzqplifffffffghiiiiiiiijjjjjjjjjhhhhhhhhhhhhhiiiiiiiihhhhhhhhhhhhhhhhiopzzqplifffffffiiiiiiiijjjjjjjjjjhhhhhhhhhhhhhiiiiiiihhhhhhhhhhhhhhhhhhiopzzqplifffffifiiiiiijjjjjjjjjjhhhhhhhhhhhhhiiiiiiiihhhhhhhhhhhhhhhhhhhiopzzpplifffiiiiiiiijjjjjjjjjihhhhhhhhhhhhhiiiiiiiiihhhhhhhhhhhhhhhhhhhhiopzzppliffiiiiiiijjjjjjjjjjhhhhhhhhhhhhhiiiiiiiiiihhhhhhhhhhhhhhhhhhhhhiopzzpoliiiiiiiijjjjjjjjjjjhhhhhhhhhhhhhiiiiiiiihhhhhhhhhhhhhhhhhhhhhhhhiopzzpolkiiiiiiijjjjjjjjjjhhhhhhhhhhhhiiiiiiiiiihhhhhhhhhhhhhhhhhhhhhhhhiopzpolkjiiiiiijjjjjjihihhhhhhhhhhhiiiiiiiihhhhhhhhhhhhhhhhhhhhhhhhhhhhiopppmjkjiiiiiiihhhhhhhhhhhhhhhhhhiiiiiiihhhhhhhhhhhhhhhhhhhhhhhhhhhhhippupnjjiiiijkkkkklllllllllllllhlhhhiiiiihhhhhhhhhhhhhhhhhhhhhhhhhhhhjopvspomnmnmmmmnnnnnnlllllmmnnimmmmiimljiiiiiiiiiiiiiiiiiiiiiiiiiijjopptupppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppptvuuttttttttttttttttttttttttttttttttttttttttttttttttttttttttttv \ No newline at end of file diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-webp-logo-distance-weighted-compound.avif b/tests/Images/Input/Heif/Av1/Conformance/libavif-webp-logo-distance-weighted-compound.avif new file mode 100644 index 0000000000..d7ce8203b9 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-webp-logo-distance-weighted-compound.avif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:da710d11c60f03eea209e4360e2fc807b89c49ad50671f0dfb1bcf4ad5ef76dd +size 6437 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-webp-logo-inter-intra-libaom.y4m b/tests/Images/Input/Heif/Av1/Conformance/libavif-webp-logo-inter-intra-libaom.y4m new file mode 100644 index 0000000000..6d8fdcc925 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-webp-logo-inter-intra-libaom.y4m @@ -0,0 +1,3 @@ +YUV4MPEG2 W80 H80 F25:1 Ip A0:0 C444 XYSCSS=444 XCOLORRANGE=LIMITED +FRAME +>>>>>>>>>>TTTTTTTTTTTv{UUs{U룱:>BB>Tctttttt^T?BCBBBBBBBBBBBBBBBBBBB\ssY>TTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTcrsdT>ASVxWTB:Ti½\T:ITǻTJTi˽\T-T˰jT--TɰjT--TȰjT--TȰjT--TȰjT--TȰjT--TȰjT--T{zjT--To\STTUUTXnjT--TgTUWz{VTUfjT--TzVUeeUVz~~jT--TtUTTTo~~~~jT--TzTVVTu~~~~~~cT--TVUTV|~~~~~~~cT--TbUT^~~~~~~~~~cT--TTdeTz~~~~~~~~~~}cT--TeUUf~~~~~~~~~}}|cT--TWVVV~~~~~~~~~~}|||cT--TT{{St~~~~~~~~~~~||||{cT--TyTTn~~~~~~~~~~}|||{|z}cT--TsTTo~~~~~~~}|||||{zzy}cT--TtTTn~~~~~~}|||||zyyyy|}^T--TzTTn~~~}}}||||{yyyyxxz}_T--T|T{|Tu~~}}|||||yyyyyyuuy}]T--TVVWW||||||||yyyyyyttttyz^T--TfUT^||||||yyyyyyytttttty^T--TyTlkTm||||yyyyyyytttttttty^T--T_TS\||||yyyyyyttttttttrtt^T--TwzUVVTt{{yyyyyytttttttttqqqs^T--TwnUddTj}yzyyyyyttttttttqqqqqq^T--TwgTVVTf|zyyyyyttttttttqqqqqqqq^T--TwnTTllTTjyyyyyyutttttttqqqqqqqqqq^T--Tw~t]TTWWST\nyyyyyyttttttttqqqqqqqqqqq^T--Tw~~~~}n_TTUUTTSS]kyyyyyyttttttttqqqqqqqqqqqqq^T--Tw~~~~~~~~~xojegmotyyyyyyttttttttqqqqqqqqqqqqqqq^T--Tw~~~~~~~~~|||||{{zyyyyyttttttttqqqqqqqqqqqqqqqqq^T--Tw~~~~~~~~~~}|||||yyyyyyyttttttttqqqqqqqqqqqqqqqqqq^T--Tw~~~~~~~~~~||||||yyyyyyyttttttttqqqqqqqqqqqqqqqqqqqq^T--Tw~~~~~~~~~}||||||yyyyyyttttttttqqqqqqqqqqqqqqqqqqqqqq^T--Tw~~~~~~~~~|||||||yyyyyytttttttttqqqqqqqqqqqqqqqqqqqqqqq^T-.Uo~~~~~~~~~~|||||{yyyyyyyttttttttqqqqqqqqqqqqqqqqqqqqqqqqq]T-,Uj~~~~~~~~~~||||||zyyyyyyttttttttrqqqqqqqqqqqqqqqqqqqqqqqqqq]T-T\~~~~~~~~~}}|||||zzyyyyttttttttqqqqqqqqqqqqqqqqqqqqqqqqqqqqq]TTT|||||zzyyyytttttttqqqqqqqqqqqqqqqqqqqqqqqqqqqqqnTT>T\}}||yyyyttttqqqqqqqqqqqqqqqqqqqqqqqqqqqqqnWT>JTWovwvvvvvvvvvvvvvwnnvnnnjnnnnjjfggggggggggggggggggggggggggggggcWTJBTUTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTUC>BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB>뀀")(("#")37883)$"37?FFE>72"efffffffff"*7BKRWRLB7)!\[[[[[[[[[""3?LWbcbXL?3#[MBBBBBBN[")7FSbouj_RB8)"[MBBBBBBN[+&#)6"$/7AIP\gd\\[[]_____________^^^^]]]]\\\\\\\\\\\\]UFB;6/(.:1)-))!!"'&"")*-3778@GPTU[oo[]bJAAAAAAAAAAAA@@@@@@@@@@@@@D@DDDDDDDDDDDDD>613.-**)))))*.3768BGGMTU[oo[^aJAAAAAAAAAAA@@@@@@@@@@@@D@DDDDDDDDDDDDDDD:=511*-------31678BGGGMTU[oo[^aJAAAAAAAAAA@@@@@@??????CDDDDDDDDDDDDDDDDDDB>661112222166=8BGGGHMTU[oo[^aJAAAAAAAA@@@@@@@?????C?DDDDDDDDDDDDDDDDDBBBB:==6666666<8BFGGGGGMTU[oo[^aJAAAAAAA@@@@@@@@@@@D@DDDDDDDDDDDDDDCDDDBBBBBBBB98=<=8BBHGGGGGFGMTU[oo[^aJAAAAA@A@@@@@@@@@@DDDDDDDDDDDDDDEDDDDBBBBBBBBBB?FAF@GGHGGGGGFFFMTU[oo[__JAAA@A@@@@@@@@@@DDDDDDDDDDDDDDMMNKFBBBBBBBBBB??@GGGGGHGGGGFFFFFMTU[oo[__JA@@@@@@@@@@@@D@DDDDDDDDDCGPZ[[[[[[XRJBBBBB@@HAHHHGGGGGGFFFFFFFLTU[oo[__J@@@@@@@@@@@D@DDDDDDDDDDERZ[\fgjjgf[[\RDCB>H@GGHGGGGGGGFFFFFFCCLST[oo[^^J@@@@@@@@@@DDDDDDDDDDDDI\[\q||q\[\NG@GGGGGGGGGGFFFFFFCCAALST[oo[]^J@@@@@@@@DDDDDDDDDDCDDN[[n~n[[QFGGGGGGGGFFFFFFFCAAAALRT[oo[]]J@@@@@@D@DDDDDDDDDCDDH[[tt[[MGGGGGGFFFFFFFCAAAAAALRS[oo[]]J@@@@D@DDDDDDDDDDDDDF\[ut\]JGGGGFFFFFFCCAAAAAAALQS[oo[]]J@@@DDDDDDDDDDDDDDDDR\mm\SGGGEFFFFCCAAAAAAAAAKQSZoo[]^J@E@DDDDDDDDDDDDDDDK[^]\IFFGFFFCAAAAAAAAAAKKQSZoo[]^J@DDDDDDDDDDDDDDDDDS[qq[RGEFFCCAAAAAAAAJJJJJPSZoo[]\JDDDDDDDDDDDDDDDCDBW[~[]CFCCAAAAAAAAAAJJJJJOSZoo[\YJDDDDDDDDDDDDDDDBCK\ed[FCAAAAAAAAAAJJJJJJJMS[oo[\YJDDDDDDDDDDDDDBBBBG[kk\JAAAAAAAAAJJJJJJJIJLS[oo[[YJDDDDDDDDDDDDBBBBBM[nm[QAAAAAAAJJJJJJJHHHJKS[oo[[YJDDDDDDDDDDBBBBBBAM[om[QAAAAAAJJJJJJHHHHHJJS[oo[[YJDDDDDDDDBBBBBBBACN[kj[JABAAJJJJJJHHHHHHHHIS[oo[[YJDDDDDDDBBBBBBBBA?I[edZKBKJJJJJJJHHHHHHGGHIS[oo[[YIDDDCDBCBBBBBBBB@FH[\\WJIJJJJJJHHHHHHHGFFFHS[oo[[WIDDDBCBBBBBBBB@G@GGRZuq[TJJJJJJHHHHHHHGFFFFFHS[oo[[VIDCCBBBBBBBB@@@GFGGNZb~b[KJJIIHHHHHHHGFFFFFFGHS[oo[[UIBBBBBBBBB@@G@GGGGGGSZqqZYJJIIHHHHHHFFFFFFFGHHMS[oo[[UIBBBBBBBB@G@GGGGGGGGH[\vv\ZNIIHHHHHHFFFFFFFFGHHHGS[oo[ZUIBBCBAB@@@GGGGGGGGGGGJZ]ww][RJIHHHHHHFFFFFFFGHHHHHIS[op[ZVHBCBBA@?GGGGGGGGFFFFGFR[[qq[[RJHHHHHHFFFFFFFGHHHHHHHIS[op[ZVHBBBAF@GGGGGGGGGFFFFFGFR[[bttb[[QHIHHHHFFFFFFFFHHHHHHHHHIS[op[ZVHBAG@GGGGGGGGGGFFFFFFCCALZ[\\goqqog\[[YKHIHHHHFFFFFFFFHHHHHHHHHHIS[oo[ZUHAAGGGGGGGGGGFFFFFFCCAAAA@JS\\[[[[[[SPHHHHHHGFFFFFFFHHHHHHHHHHHHIS[oo[ZUHHGGGGGGGGGFFFFFFFCAAAAAAAA@JJRRRQIHHHHHHHFFFFFFFFHHHHHHHHHHHHHHIS[oo[ZUGGGGGGGGGFFFFFFFCBAAAAAAAAAJJJJJJIHHHHHHFFFFFFFGHHHHHHHHHHHHHHHHIS[oo[ZTGGGGGGGGFFFFFFBBAAAAAAAAAAJJJJJIIIHHHHHEFFFFFFGGHHHHHHHHHHHHHHHHIS[oo[ZRGGGGGGGFFFFFFBAAAAAAAAAAKJJJJIIIHHHHIEEFFFFFGGHHHHHHHHHHHHHHHHHHIS[oo[ZQGGGGGGFFFFFBAAAAAAAAAJJJJJJIIHHHHHIEEFFFFFGGHHHHHHHHHHHHHHHHHHHHIS[oo[ZQGGGGGGFFFBAAAAAAAAAAJJJJJJIIHHHHIEFFFFFFFGHHHHIHHHHHHHHHHHHHHHHHIS[op[VPFFFFFFFDAAAAAAAAAABKJJJIHHHHHHHHFFFFFFFFHHHHHHHHHHHHHHHHHHHHHHHHHS[pp[VPKGFFFFDAAAAAAAAAAJJJJJIIHHHHHHFFFFFFGGGGHHHHHHHHHHHHHHHHHHHHHHHHHW[p[YPMFFGCAAAAAAAAAKKJJJJJJIHHHHHFFFFFFFFHHHHHHHHHHHHHHHHHHHHHHHHHHHHHW[[[PQNLEFFGJAAKKKKKKKKJJJJIHHHHHFFFFFFFGHHHHHHHHHHHHHHHHHHHHHHHHHHHHH[[e[YQQQQQQLLLLLLKKKKKKKKKKIIIIHHHHFFFFHHHHHHHHHHHHHHHHHHHHHHHHHHHHHMW[f`[WRNSMLMSSMMMMMMSRRQQMQQQQQQQQQQQQPPPPPOOOOOOOOOOOOOOOOOOOOOOMMSW[ce[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[Z[[[[[[[[[[[[[[[[[[[[[[[[[[[[[cfcbbbccccccccccccccccccccccccccccccccccccccccccccccccccccccccfvwwwvvvvvvqqqqppppppqnddddddmppjddddddmpuuutvqpjjjjjmopttttttttttttttttttttttx}}}vpppppppppppppppppppppppppppppppppppppqw}}wpvtpqqqrrrrrrrrrrrrrrrrssssssrqpppppppppppqy}|vqpptvorqrrssssstttttttttssrrrrrrrrrrrrsssssporx~yvonnpvtptrojiggggggggggggggggggggggggggggggggggirq{{tplplptprsodddddddddddddddddheghhhhhhhhhggggggggglry|{vokklmopzprsidddddddddddddddddhhhhhhhhhhggggggggggggkrz}{vtqiemnopzzprsiddddddddddddeddhhhhhhhhhhghgggggggggggggmty{{vtqkfeklopzzprsiddddddddddfdhefhhhhhhhhhgggggggggggggggflotyy|{yttpkffgjlopzzprsiddddddddddhdfhhhhhhhhhgggggggggggggggggggjltuzyz{{{{zxtpqjffffklopzzprsiddddddddddfhhhhhhhhhggggggggggggggggggggggglopustttttppkefffffklopzzprtiddddddddghhhhhhhhhhggggggggggggggggggggggggggjlooooqkkeffffhhhllopzzprtidddddhdghhhhhhhhhgggggggggggggggggggggggggggggfgghgffffffgghiillopzzprtidddhdhhhhhhhhhhggggggggggggggjjjihgggggggggggggeeeeffffggigiihklopzzprtgdddhhhhhhhhhhggggggggggggimpppoopppmiggggggffffeeffffffiiiiiiiklopzzprshddhhhhhhhhhhgggggggggggimppptvwwvtpppmigggfffffffffffghiiiiiiiklopzzpqshhhhhhhhhhhggggggggggggjpprz~~zrpplfffffffffffghiiiiiiijjklopzzpqrhhhhhhhhhgggggggggggggjopyypomffffffffghiiiiiiijjjjjjopzzppqhhhhhhhggggggggggggggkppzzpojfffffggiiiiiiijjjjjjkjopzzppphhhhhghgggggggggggggippzzqohffggiiiiiiiijjjjjjjkjopzzppphhhhggggggggggggggggppyxpogghhiiiiiijjjjjjjjjijopzzppphhghggggggggggggggggpssqlhhiiiiijjjjjjjjjjhhlopzzppphhggggggggggggggggglpyyplihhiiijjjjjjjjhhhhhlopzzppphggggggggggggggggggppppiiiiijjjjjjjjjhhhhllopzzppqhggggggggggggggghginttpjiijjjjjjjjjhhhhhhllopzzppqhggggggggggggggghgjowwpkjijjjjjjjhhhhhhhhllopzzppqhgggggggggggggggggjoxypljjjjjjjhhhhhhhhhhllopzzppqhgggggggggggggggggjpxypljjjjjjhhhhhhhhhhhhlopzzqpqhgggggggggggggggggjpvwpkjjjjhhhhhhhhhhhhhhlopzzqpphgggggggggggggggggiprtpjjihhhhhhhhhhhhhhihlopzzqpphggggggggggggggggfeqpoqihhhhhhhhhhhhhhiiiilopzzqpphggggggggggggggfffelp{{pohhhhhhhhhhhhhiiiiiikopzzqpphgggggggggggggfffffkqqspjhhhhhhhhhhhiiiiiiiikopzzqpphgggggggggggffeefffeqqy~wpohhhhhhhhhhiiiiiiiiihjopzzqpphgggggggggfgeeffffggiqp}}ppihhhhhhhhiiiiiiiiihhhjopzzqpphgggggggggeeefffffggijqr}}tpihhhhhhhhiiiiiiiihhhhhjopzzqpphgggggggeeeeffffefhihimppzzppkhhhhhhhiiiiiiiihhhhhhhiopzzqpphgggggggeeefffffiiiiiiimppr{{rppihhhhhhiiiiiiiihhhhhhhhhiopzzponiggfgfffffffffghiiiiiiijlnpppvvyyvvpppnlhhhhhhiiiiiiiihhhhhhhhhhiopzzpoliggfffffffffghhhiiiiijjjjjjoppppppppojhhhhhhiiiiiiiihhhhhhhhhhhhiopzzpolifffffffffghiiiiiiijjjjjjjjjikkkkkkhhhhhhhiiiiiiiihhhhhhhhhhhhhhiopzzqplifffffffghiiiiiiiijjjjjjjjjhhhhhhhhhhhhhiiiiiiiihhhhhhhhhhhhhhhhiopzzqplifffffffiiiiiiiijjjjjjjjjjhhhhhhhhhhhhhiiiiiiihhhhhhhhhhhhhhhhhhiopzzqplifffffifiiiiiijjjjjjjjjjhhhhhhhhhhhhhiiiiiiiihhhhhhhhhhhhhhhhhhhiopzzpplifffiiiiiiiijjjjjjjjjihhhhhhhhhhhhhiiiiiiiiihhhhhhhhhhhhhhhhhhhhiopzzppliffiiiiiiijjjjjjjjjjhhhhhhhhhhhhhiiiiiiiiiihhhhhhhhhhhhhhhhhhhhhiopzzpoliiiiiiiijjjjjjjjjjjhhhhhhhhhhhhhiiiiiiiihhhhhhhhhhhhhhhhhhhhhhhhiopzzpolkiiiiiiijjjjjjjjjjhhhhhhhhhhhhiiiiiiiiiihhhhhhhhhhhhhhhhhhhhhhhhiopzpolkjiiiiiijjjjjjihihhhhhhhhhhhiiiiiiiihhhhhhhhhhhhhhhhhhhhhhhhhhhhiopppmjkjiiiiiiihhhhhhhhhhhhhhhhhhiiiiiiihhhhhhhhhhhhhhhhhhhhhhhhhhhhhippupnjjiiiijkkkkklllllllllllllhlhhhiiiiihhhhhhhhhhhhhhhhhhhhhhhhhhhhjopvspomnmnmmmmnnnnnnlllllmmnnimmmmiimljiiiiiiiiiiiiiiiiiiiiiiiiiijjopptupppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppptvuuttttttttttttttttttttttttttttttttttttttttttttttttttttttttttv \ No newline at end of file diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-webp-logo-inter-intra.avif b/tests/Images/Input/Heif/Av1/Conformance/libavif-webp-logo-inter-intra.avif new file mode 100644 index 0000000000..6311f8e22d --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-webp-logo-inter-intra.avif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:71df22e63626b5bc9001ff1e88076b90f11bb47d18089750853e66f0cbbb084b +size 6392 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-webp-logo-obmc-libaom.y4m b/tests/Images/Input/Heif/Av1/Conformance/libavif-webp-logo-obmc-libaom.y4m new file mode 100644 index 0000000000..2d3f2f87e2 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-webp-logo-obmc-libaom.y4m @@ -0,0 +1,3 @@ +YUV4MPEG2 W80 H80 F25:1 Ip A0:0 C444 XYSCSS=444 XCOLORRANGE=LIMITED +FRAME +>>>>>>>>>>TTTTTTTTTTTv{UUs{U룱:>BB>Tctttttt^T?BCBBBBBBBBBBBBBBBBBBB\ssY>TTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTcrsdT>ASVxWTB:Ti½\T:ITǻTJTi˽\T-T˰jT--TɰjT--TȰjT--TȰjT--TȰjT--TȰjT--TȰjT--T{zjT--To\STTUUTXnjT--TgTUWz{VTUfjT--TzVUeeUVz~~jT--TtUTTTo~~~~jT--TzTVVTu~~~~~~cT--TVUTV|~~~~~~~cT--TbUT^~~~~~~~~~cT--TTdeTz~~~~~~~~~~}cT--TeUUf~~~~~~~~~}}|cT--TWVVV~~~~~~~~~~}|||cT--TT{{St~~~~~~~~~~~||||{cT--TyTTn~~~~~~~~~~}|||{|z}cT--TsTTo~~~~~~~}|||||{zzy}cT--TtTTn~~~~~~}|||||zyyyy|}^T--TzTTn~~~}}}||||{yyyyxxz}_T--T|T{|Tu~~}}|||||yyyyyyuuy}]T--TVVWW||||||||yyyyyyttttyz^T--TfUT^||||||yyyyyyytttttty^T--TyTlkTm||||yyyyyyytttttttty^T--T_TS\||||yyyyyyttttttttrtt^T--TwzUVVTt{{yyyyyytttttttttqqqs^T--TwnUddTj}yzyyyyyttttttttqqqqqq^T--TwgTVVTf|zyyyyyttttttttqqqqqqqq^T--TwnTTllTTjyyyyyyutttttttqqqqqqqqqq^T--Tw~t]TTWWST\nyyyyyyttttttttqqqqqqqqqqq^T--Tw~~~~}n_TTUUTTSS]kyyyyyyttttttttqqqqqqqqqqqqq^T--Tw~~~~~~~~~xojegmotyyyyyyttttttttqqqqqqqqqqqqqqq^T--Tw~~~~~~~~~|||||{{zyyyyyttttttttqqqqqqqqqqqqqqqqq^T--Tw~~~~~~~~~~}|||||yyyyyyyttttttttqqqqqqqqqqqqqqqqqq^T--Tw~~~~~~~~~~||||||yyyyyyyttttttttqqqqqqqqqqqqqqqqqqqq^T--Tw~~~~~~~~~}||||||yyyyyyttttttttqqqqqqqqqqqqqqqqqqqqqq^T--Tw~~~~~~~~~|||||||yyyyyytttttttttqqqqqqqqqqqqqqqqqqqqqqq^T-.Uo~~~~~~~~~~|||||{yyyyyyyttttttttqqqqqqqqqqqqqqqqqqqqqqqqq]T-,Uj~~~~~~~~~~||||||zyyyyyyttttttttrqqqqqqqqqqqqqqqqqqqqqqqqqq]T-T\~~~~~~~~~}}|||||zzyyyyttttttttqqqqqqqqqqqqqqqqqqqqqqqqqqqqq]TTT|||||zzyyyytttttttqqqqqqqqqqqqqqqqqqqqqqqqqqqqqnTT>T\}}||yyyyttttqqqqqqqqqqqqqqqqqqqqqqqqqqqqqnWT>JTWovwvvvvvvvvvvvvvwnnvnnnjnnnnjjfggggggggggggggggggggggggggggggcWTJBTUTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTUC>BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB>뀀")(("#")37883)$"37?FFE>72"efffffffff"*7BKRWRLB7)!\[[[[[[[[[""3?LWbcbXL?3#[MBBBBBBN[")7FSbouj_RB8)"[MBBBBBBN[+&#)6"$/7AIP\gd\\[[]_____________^^^^]]]]\\\\\\\\\\\\]UFB;6/(.:1)-))!!"'&"")*-3778@GPTU[oo[]bJAAAAAAAAAAAA@@@@@@@@@@@@@D@DDDDDDDDDDDDD>613.-**)))))*.3768BGGMTU[oo[^aJAAAAAAAAAAA@@@@@@@@@@@@D@DDDDDDDDDDDDDDD:=511*-------31678BGGGMTU[oo[^aJAAAAAAAAAA@@@@@@??????CDDDDDDDDDDDDDDDDDDB>661112222166=8BGGGHMTU[oo[^aJAAAAAAAA@@@@@@@?????C?DDDDDDDDDDDDDDDDDBBBB:==6666666<8BFGGGGGMTU[oo[^aJAAAAAAA@@@@@@@@@@@D@DDDDDDDDDDDDDDCDDDBBBBBBBB98=<=8BBHGGGGGFGMTU[oo[^aJAAAAA@A@@@@@@@@@@DDDDDDDDDDDDDDEDDDDBBBBBBBBBB?FAF@GGHGGGGGFFFMTU[oo[^^JAAA@A@@@@@@@@@@DDDDDDDDDDDDDDMMNKFBBBBBBBBBB??@GGGGGHGGGGFFFFFMTU[oo[^^JA@@@@@@@@@@@@D@DDDDDDDDDCGPZ[[[[[[XRJBBBBB@@HAHHHGGGGGGFFFFFFFLTU[oo[^^J@@@@@@@@@@@D@DDDDDDDDDDERZ[\fgjjgf[[\RDCB>H@GGHGGGGGGGFFFFFFCCLST[oo[^^J@@@@@@@@@@DDDDDDDDDDDDI\[\q||q\[\NG@GGGGGGGGGGFFFFFFCCAALST[oo[^^J@@@@@@@@DDDDDDDDDDCDDN[[n~n[[QFGGGGGGGGFFFFFFFCAAAALRT[oo[]^J@@@@@@D@DDDDDDDDDCDDH[[tt[[MGGGGGGFFFFFFFCAAAAAALRS[oo[]]J@@@@D@DDDDDDDDDDDDDF\[ut\]JGGGGFFFFFFCCAAAAAAALQS[oo[]]J@@@DDDDDDDDDDDDDDDDR\mm\SGGGEFFFFCCAAAAAAAAAKQSZoo[]]J@E@DDDDDDDDDDDDDDDK[^]\IFFGFFFCAAAAAAAAAAKKQSZoo[]^J@DDDDDDDDDDDDDDDDDS[qq[RGEFFCCAAAAAAAAJJJJJPSZoo[]\JDDDDDDDDDDDDDDDCDBW[~[]CFCCAAAAAAAAAAJJJJJOSZoo[\YJDDDDDDDDDDDDDDDBCK\ed[FCAAAAAAAAAAJJJJJJJMS[oo[\YJDDDDDDDDDDDDDBBBBG[kk\JAAAAAAAAAJJJJJJJIJLS[oo[[YJDDDDDDDDDDDDBBBBBM[nm[QAAAAAAAJJJJJJJHHHJKS[oo[[YJDDDDDDDDDDBBBBBBAM[om[QAAAAAAJJJJJJHHHHHJJS[oo[[YJDDDDDDDDBBBBBBBACN[kj[JABAAJJJJJJHHHHHHHHIS[oo[[YJDDDDDDDBBBBBBBBA?I[edZKBKJJJJJJJHHHHHHGGHIS[oo[[YIDDDCDBCBBBBBBBB@FH[\\WJIJJJJJJHHHHHHHGFFFHS[oo[[WIDDDBCBBBBBBBB@G@GGRZuq[TJJJJJJHHHHHHHGFFFFFHS[oo[[VIDCCBBBBBBBB@@@GFGGNZb~b[KJJIIHHHHHHHGFFFFFFGHS[oo[[UIBBBBBBBBB@@G@GGGGGGSZqqZYJJIIHHHHHHFFFFFFFGHHMS[oo[[UIBBBBBBBB@G@GGGGGGGGH[\vv\ZNIIHHHHHHFFFFFFFFGHHHGS[oo[ZUIBBCBAB@@@GGGGGGGGGGGJZ]ww][RJIHHHHHHFFFFFFFGHHHHHIS[op[ZVHBCBBA@?GGGGGGGGFFFFGFR[[qq[[RJHHHHHHFFFFFFFGHHHHHHHIS[op[ZVHBBBAF@GGGGGGGGGFFFFFGFR[[bttb[[QHIHHHHFFFFFFFFHHHHHHHHHIS[op[ZVHBAG@GGGGGGGGGGFFFFFFCCALZ[\\goqqog\[[YKHIHHHHFFFFFFFFHHHHHHHHHHIS[oo[ZUHAAGGGGGGGGGGFFFFFFCCAAAA@JS\\[[[[[[SPHHHHHHGFFFFFFFHHHHHHHHHHHHIS[oo[ZUHHGGGGGGGGGFFFFFFFCAAAAAAAA@JJRRRQIHHHHHHHFFFFFFFFHHHHHHHHHHHHHHIS[oo[ZUGGGGGGGGGFFFFFFFCBAAAAAAAAAJJJJJJIHHHHHHFFFFFFFGHHHHHHHHHHHHHHHHIS[oo[ZTGGGGGGGGFFFFFFBBAAAAAAAAAAJJJJJIIIHHHHHEFFFFFFGGHHHHHHHHHHHHHHHHIS[oo[ZRGGGGGGGFFFFFFBAAAAAAAAAAKJJJJIIIHHHHIEEFFFFFGGHHHHHHHHHHHHHHHHHHIS[oo[ZQGGGGGGFFFFFBAAAAAAAAAJJJJJJIIHHHHHIEEFFFFFGGHHHHHHHHHHHHHHHHHHHHIS[oo[ZQGGGGGGFFFBAAAAAAAAAAJJJJJJIIHHHHIEFFFFFFFGHHHHIHHHHHHHHHHHHHHHHHIS[op[VPFFFFFFFDAAAAAAAAAABKJJJIHHHHHHHHFFFFFFFFHHHHHHHHHHHHHHHHHHHHHHHHHS[pp[VPKGFFFFDAAAAAAAAAAJJJJJIIHHHHHHFFFFFFGGGGHHHHHHHHHHHHHHHHHHHHHHHHHW[p[YPMFFGCAAAAAAAAAKKJJJJJJIHHHHHFFFFFFFFHHHHHHHHHHHHHHHHHHHHHHHHHHHHHW[[[PQNLEFFGJAAKKKKKKKKJJJJIHHHHHFFFFFFFGHHHHHHHHHHHHHHHHHHHHHHHHHHHHH[[e[YQQQQQQLLLLLLKKKKKKKKKKIIIIHHHHFFFFHHHHHHHHHHHHHHHHHHHHHHHHHHHHHMW[f`[WRNSMLMSSMMMMMMSRRQQMQQQQQQQQQQQQPPPPPOOOOOOOOOOOOOOOOOOOOOOMMSW[ce[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[Z[[[[[[[[[[[[[[[[[[[[[[[[[[[[[cfcbbbccccccccccccccccccccccccccccccccccccccccccccccccccccccccfvwwwvvvvvvqqqqppppppqnddddddmppjddddddmpuuutvqpjjjjjmopttttttttttttttttttttttx}}}vpppppppppppppppppppppppppppppppppppppqw}}wpvtpqqqrrrrrrrrrrrrrrrrssssssrqpppppppppppqy}|vqpptvorqrrssssstttttttttssrrrrrrrrrrrrsssssporx~yvonnpvtptrojiggggggggggggggggggggggggggggggggggirq{{tplplptprsodddddddddddddddddheghhhhhhhhhggggggggglry|{vokklmopzprsidddddddddddddddddhhhhhhhhhhggggggggggggkrz}{vtqiemnopzzprsiddddddddddddeddhhhhhhhhhhghgggggggggggggmty{{vtqkfeklopzzprsiddddddddddfdhefhhhhhhhhhgggggggggggggggflotyy|{yttpkffgjlopzzprsiddddddddddhdfhhhhhhhhhgggggggggggggggggggjltuzyz{{{{zxtpqjffffklopzzprsiddddddddddfhhhhhhhhhggggggggggggggggggggggglopustttttppkefffffklopzzprtiddddddddghhhhhhhhhhggggggggggggggggggggggggggjlooooqkkeffffhhhllopzzprtidddddhdghhhhhhhhhgggggggggggggggggggggggggggggfgghgffffffgghiillopzzprsidddhdhhhhhhhhhhggggggggggggggjjjihgggggggggggggeeeeffffggigiihklopzzprsgdddhhhhhhhhhhggggggggggggimpppoopppmiggggggffffeeffffffiiiiiiiklopzzpqshddhhhhhhhhhhgggggggggggimppptvwwvtpppmigggfffffffffffghiiiiiiiklopzzpqrhhhhhhhhhhhggggggggggggjpprz~~zrpplfffffffffffghiiiiiiijjklopzzpqrhhhhhhhhhgggggggggggggjopyypomffffffffghiiiiiiijjjjjjopzzpprhhhhhhhggggggggggggggkppzzpojfffffggiiiiiiijjjjjjkjopzzppqhhhhhghgggggggggggggippzzqohffggiiiiiiiijjjjjjjkjopzzppphhhhggggggggggggggggppyxpogghhiiiiiijjjjjjjjjijopzzppphhghggggggggggggggggpssqlhhiiiiijjjjjjjjjjhhlopzzppphhggggggggggggggggglpyyplihhiiijjjjjjjjhhhhhlopzzppphggggggggggggggggggppppiiiiijjjjjjjjjhhhhllopzzppqhggggggggggggggghginttpjiijjjjjjjjjhhhhhhllopzzppqhggggggggggggggghgjowwpkjijjjjjjjhhhhhhhhllopzzppqhgggggggggggggggggjoxypljjjjjjjhhhhhhhhhhllopzzppqhgggggggggggggggggjpxypljjjjjjhhhhhhhhhhhhlopzzqpqhgggggggggggggggggjpvwpkjjjjhhhhhhhhhhhhhhlopzzqpphgggggggggggggggggiprtpjjihhhhhhhhhhhhhhihlopzzqpphggggggggggggggggfeqpoqihhhhhhhhhhhhhhiiiilopzzqpphggggggggggggggfffelp{{pohhhhhhhhhhhhhiiiiiikopzzqpphgggggggggggggfffffkqqspjhhhhhhhhhhhiiiiiiiikopzzqpphgggggggggggffeefffeqqy~wpohhhhhhhhhhiiiiiiiiihjopzzqpphgggggggggfgeeffffggiqp}}ppihhhhhhhhiiiiiiiiihhhjopzzqpphgggggggggeeefffffggijqr}}tpihhhhhhhhiiiiiiiihhhhhjopzzqpphgggggggeeeeffffefhihimppzzppkhhhhhhhiiiiiiiihhhhhhhiopzzqpphgggggggeeefffffiiiiiiimppr{{rppihhhhhhiiiiiiiihhhhhhhhhiopzzponiggfgfffffffffghiiiiiiijlnpppvvyyvvpppnlhhhhhhiiiiiiiihhhhhhhhhhiopzzpoliggfffffffffghhhiiiiijjjjjjoppppppppojhhhhhhiiiiiiiihhhhhhhhhhhhiopzzpolifffffffffghiiiiiiijjjjjjjjjikkkkkkhhhhhhhiiiiiiiihhhhhhhhhhhhhhiopzzqplifffffffghiiiiiiiijjjjjjjjjhhhhhhhhhhhhhiiiiiiiihhhhhhhhhhhhhhhhiopzzqplifffffffiiiiiiiijjjjjjjjjjhhhhhhhhhhhhhiiiiiiihhhhhhhhhhhhhhhhhhiopzzqplifffffifiiiiiijjjjjjjjjjhhhhhhhhhhhhhiiiiiiiihhhhhhhhhhhhhhhhhhhiopzzpplifffiiiiiiiijjjjjjjjjihhhhhhhhhhhhhiiiiiiiiihhhhhhhhhhhhhhhhhhhhiopzzppliffiiiiiiijjjjjjjjjjhhhhhhhhhhhhhiiiiiiiiiihhhhhhhhhhhhhhhhhhhhhiopzzpoliiiiiiiijjjjjjjjjjjhhhhhhhhhhhhhiiiiiiiihhhhhhhhhhhhhhhhhhhhhhhhiopzzpolkiiiiiiijjjjjjjjjjhhhhhhhhhhhhiiiiiiiiiihhhhhhhhhhhhhhhhhhhhhhhhiopzpolkjiiiiiijjjjjjihihhhhhhhhhhhiiiiiiiihhhhhhhhhhhhhhhhhhhhhhhhhhhhiopppmjkjiiiiiiihhhhhhhhhhhhhhhhhhiiiiiiihhhhhhhhhhhhhhhhhhhhhhhhhhhhhippupnjjiiiijkkkkklllllllllllllhlhhhiiiiihhhhhhhhhhhhhhhhhhhhhhhhhhhhjopvspomnmnmmmmnnnnnnlllllmmnnimmmmiimljiiiiiiiiiiiiiiiiiiiiiiiiiijjopptupppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppptvuuttttttttttttttttttttttttttttttttttttttttttttttttttttttttttv \ No newline at end of file diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-webp-logo-obmc.avif b/tests/Images/Input/Heif/Av1/Conformance/libavif-webp-logo-obmc.avif new file mode 100644 index 0000000000..1926224905 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-webp-logo-obmc.avif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:765245f71bd398f7ad87bd83fd5c5c11172981b37a4fabf4f10f65d4e8aea537 +size 6452 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-webp-logo-scaled-reference-base-libaom.yuv b/tests/Images/Input/Heif/Av1/Conformance/libavif-webp-logo-scaled-reference-base-libaom.yuv new file mode 100644 index 0000000000..f1b5a51261 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-webp-logo-scaled-reference-base-libaom.yuv @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:873dc1ab5623910fbf8053cbf684399f4f3bddba4110492e3e11de86e01fec1b +size 4800 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-webp-logo-scaled-reference-libaom.y4m b/tests/Images/Input/Heif/Av1/Conformance/libavif-webp-logo-scaled-reference-libaom.y4m new file mode 100644 index 0000000000..fa02030df1 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-webp-logo-scaled-reference-libaom.y4m @@ -0,0 +1,3 @@ +YUV4MPEG2 W80 H80 F25:1 Ip A0:0 C444 XYSCSS=444 XCOLORRANGE=LIMITED +FRAME +??>>>>>>>>VUUTTTTTTTTuwUTuwU뤯???B=Vcuuuuvt`V=?ACBBBBBBBBBBBBBBBBDB]u©tZ;TTTTTTTTTTTTTTTTTTTTUUUUUUUUUUUUUUUUWSdqsfU>FTXyWSB;Tiž\S;FTĻUKTi˾]U,S˰½iV.,UʰjT-,WɲkT--WɱiT-/UɰkT-0UȲkT-.TůkT-,Tª{zjS--Tn\UUSVSXXljS--TfVUVzzVUVfjS--Tª{USeeSUz~jT--T©sTUURq~~jT--TzWVVRu~~~~~~dT--TUSTW|~~~~~~dT--TfUU^}~~~~~~~dU--TVcgRz}}~~~~~~~}dU--TfRTf~~}~}}|}cT--TXXWX~~}~~~~}}|~cU--TT|yTu~~~}}~}|}{}}|bU.-TwUTo}|}~~~~~}}}||||{z}bV.-TtRTl~{}}}~~~}|{|}zzyx}bV--TuSTm|~~}}|||}{z{yyyy}_T--T|UUp~~~~~|{{||zyzxyxwz}]S--TySy|Vt~}}||}|{{yxzzxutx~^T--TVVVU{~}|}|||yyyyyxustuwy^T--TeTR^|~|{||zxzyyxvuuutttz^T--TzSklTl|}{{zxyzyxvuuutttuty^T--T`QW[|{|{yyyyywuuttstsssru^T--TvySVVTt|{zxyyywuuuuuutssrqqs^T--TvmSgeUl{yyyzzyvttttttttrqrqqt^T--TwfVVVSe~{xwyzxuttttttsqqqrrqqr^T--TwnTTmlTTl{yyzzywvuttssssrrrqrrqqp^T--Tw~~t[QUXUSS]lyzyyyxvuuuutssrrrqqqqqqqp^T--Tw~~~laTUSUTRTU_jzzzzyxuutsssssrrrqqqqqqqqqp^T--Tx~}~~~}zmkfgmluwyyyxxwutssssrrrqqqqqqqqqqqqp^T--Tx~}~~~~~~~}}~}}}}y{yyywvustsssssrrrqqqqqqqqqqqqp^T--Tx~~~~~~~~~~}||}||zyyxxxxvtssssssrrrqqppppqqqqqqqqp^T--Tx~~~~~~}}}}|}{{{{{zzyywvuutssssrqqqqqqppppqqqqqqqqp^T--Tx~~~~}}}}}|}{{zzzzxwvvuttssrrrqqqqqqqppqqqqqqqqqqp^T--Tx~}~~~~}}}|}|{{zzyyxusqttttssrqqqqqqqqqppqqqqqqqqqqp^T-.Tm}}}}~~~~|}~|{{{{zzzzyyttttttssrrrqqqqqqqqqqqqqqqqqqqqqp^T,-Sk~~}}}~~~~}}}|||~{yxxyzyvtsttutsqprrqqqqqqqqqqqqqqqqqqqqqqp\T,V[~~~~~}{{{{|z{yzzxuutttttssrrqrqqqqqqqqqqqqqqqqqqqqqqqp]TST|z{||xzyzzxuutttttssrrqqqqqqqqqqqqqqqqqqqqqqqqqpTT=T]~~|zyyxusutrqppqrqqqqqqqqqqqqqqqqqqqqqtqoWV@LTWmuwwwvvvvvvvvvvvuoptooqjmloojkgggggggggggggggggggggggggggggggcVUKBSSUTTTTTTTTTTTTTTTTTUTTSTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTSSTSC>ABBBBBBBBBBBBBCAABBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB=뀀%&&&%%$%56665%%$45BCCCB55$ffffffffff$$4BQRRRQB5$!\\\\\\\\\\%$4AQQcccQQB4#\OCDDEFFS\"'5EP`mrl_RG8*#~\O@@AABCOX)%$,4"&/6@HPZj{f\\\\\\\]^_``````_^]\\\\\\\]]]]^^]]\\\\\TFC?7-*1=CKQRQKA7/*,29>FL]Zg}f\\\fffffedcbaaaaa``_________^^^^^]]\\\\\TKD@8,'*38BDED@80*',4;HMUS]Wl\\\f\OOCCCDEFFFFFFFGGGGGGGGGFFEEDDDEEEEEEE<;6/($%*,67665/)%+,+24>STS^[\\f\CCCCCCBBAAAAAAA@@@@@@@@@@@@AABCDEEEEEE<82,'$%&&+(')*'##+-57:CGVTW]o]\aNCCAABAAAA@@@@@@?????@@@A@??ABDEEDDDDDCC<82,,)'%('%$%')*/577;CHQTV[no]\aJ?AAAAAAAA@@@@@@@@@@AAAABBBBCDDCCCCDCCCCA<820.,*)(''')-/1579@EFNTV[no]\aJ@AAAAAAAA@@@@@@@@@@AAAABBBCDDDDDCDDCCCCCA<831/-.,++-/13879AFHGLTV[no]\aKABAAAAAAA@@@@@@@@@@AAAABBBCDEEDDCDDCCCCCCA<653021112467<9DFIIGLTV[no]\aKABAAAAAAA@@AAAABBBBBBBBCCCCDDDDDCDDCCCCBBBB=;:8766778:>=>>??@BDGGGGHGFMTV[no]\aK@AAAAAAAA@@AAABBCCCCCCCCCDDDEEEEDDDCBBBBBBBA@@ABBCDCFHJGGGGGFFMTV[no\]`K@AAAAAAAA@AAAABBCCCCDCCDFFDEGILLHFDCBBBBBBBBBBCDDDDEGHKGGGGGEFMTV[no\]_J@@@@@@AA@@ABBBBBBBCFE@CEEER[[[[[[[[REEDAAABBCDEEEEFFFGGGFFFFFFMTV[no[^^J@@@@@@AAAAABBBCCCCEEFEEER[[[hhhhhh[[[RECCDCDEGGFGGHHGGGFFFFFAALSU[no[^^J@@@@@@AABBCBBCCCDDEBFDI[[[pp[[[JFEFGHHHGGGHGGFFFFFAAAALRU[no[^^J@@@@@@AABCCBCCCCDDEECM^[pp[VNHHHHHHGGGGGFFEFFAAAAALRU[oo[]]J@@@@@@DCCCDDDEEEFFFCF[Zpp^\LGHHHGGFFFFEDCBAAAAAALQT[oo[]]J@@@@D@DCCDDDEEEEEEDG]\usY^LGGGGFFFEEDCCBAAAAAALQT[oo[]]J@@@DDDDDDDDEEEEEEDCS\o~oYUHGGFFEEDDCBBBAAAAAALQTZoo[]]J@D@DDDDDDDDEEEEDEFFWZ^]KGGFFDDCCCBBBAAAAAFKQTZoo[]]J@DDDDDDEEEDEDEECEDPZrq[SCEEJCBBBBBBBBBJJJJKPTZoo[][JDDDDDDDEEEDDDDDCECV\[YADCCBBBBBBBBBBJJJJJOSZoo[][JDDDDDDDDDDDDDDDBEH[ed[CDBBBBBBBBBBJJJJJJJNRZoo[][JDDDDDDDDDDDDDDDACD\kk\IDBBBBBBBBJIIIIIIIILRZoo[[[JDDDDDDDDDDDDDCCBBO\oj\KCCBBBBBJJJIIIIIIIILQZpo[[[JDDDDDDDDDDDCCCBBAL\pj\MBDBBBBJJJIIHHHHHHHKQZpo[[[JDDDDDDDDDCCCCCCB@O\jj[NAFBBJJJJJIIHHHHHHHJPZpo[[[JDDDDDDDCCCCCCCCB@J[d~i[M@FJJJJJJIIIHHGGGGGJPZpo[[ZJDDDDDDCCCBBCBBB@FE]^~}~[XKHGIIIIIIIHHHGGGGFFJPZpo[ZYJDDCCCCCBBBBCCCDCGEU]ssZTIHHIIIIIIHHHGGFFFFFJPZpp[YXJDCCCBBBBBBBCDDDFHGN\`|`ZOIIIIIHHHHHGGGFFFFFFIQ[op[YWJCCCBBBBBBBCDEEFFGGGSXo}n[\IIIIIHHHHHGGGGGGFFFGIQ[op[YVJCCBBBBBBCDEFFFFHFHEJY[vp]^KIIIHHHHHGGGGGGGGGHHHIQ[op[XUJCBBBAAACDEFGGGFGEGFFLYZxv]^KKIHHHHHGGGGGGGGGGHHHHIQ[op[XUJCBCABABDEFGIIHGGDGFFGQZ[qpZ^KKKIHHHHGGGGGGGGGHIIHHHIQ[op[XUICBDAC@@FGGHIIHGGFFFEHER[[ats`ZZKKJJHGGGGGGGGGGGHHHIIHHHIQ[op[XUHAAEAEGGGGGGGGGGFFFFFAAANY_[\hqopmg_]]WNIIIHHGGFGGGGGGHHHHHHHHHHHQ[op[XUGAAFGFGGGGGGGGGGFFFAAAAAABJT[\YW\[]]TKGHHHHGGFFFGGGGGGHHHHHHHHHHHR[op[XUGGGGGGGGGGGFEEGGFFAAAAAA@@@CHKORRNKHGHHHHHHGGGGGGGGGGGGGHHHHHHHHHR\op[XUGGGGGGGGGFFFFEEDCBAAAAAAABDEHIJJKJIHHHHHHGGGGGGGGHHHHGGGHHHHHHHHIR\op[XUGGGGGGGGFFEFGECBAAAAAAAA@ADGHJIHHGGGHHGGGGGGGGGHHHHHHHHHHHHHHHHHIR\op[XUGGGGGGGGGFEEDCBAAAAAAAAAEIKJHIIHHGGGHHHGGFGGGHHHHHHHHHHHHHHHHHHHIS\op[XTGGGGGGGGGEDCBAAAAAAAAJJJJJJIIIIHHGGFFFGGGFGGGHHHHHHHHHHHHHHHHHHHIS\op[XSGGGGGGGGECBA@ABBAAAAJJJJJJJIIIIHHGGFEEFFGGGGHHHHHHHHHHHHHHHHHHHHIS[op[XRIGGFFFFEDCBBAAAAAAAJJJIIIIIHHHHGGFFFFFFGGGGHHHHHHHHHHHHHHHHHHHHHHS[qp[XQJFFFEDBBBBBBAAAAAJJJJJIIIHHHGGGGFFFEFFGHHHHHHHHHHHHHHHHHHHHHHHHHIUZv[XPLGFGECAABBBBAAJJJJJJJJIIIHHGGFFFGGGGGHHHHHHHHHHHHHHHHHHHHHHHHHHHJX]|[[PNLJFFFFDCBKKJJJJJJJJJJIIIHHGGGGGGGGGGHHHHHHHHHHHHHHHHHHHHHHHHHHHKZ_c[XQPRPNMMMMLLLKKKKKKKKKKJJJJIHHGGGGHIIJJIIHHHHHHHHHHHHHHHHHHHHKIHIX]ec[WPNPOOOOTMMMLLLSSSQQLQPPPPPPPPPPOOOOOOOOOOOOOOOOOOOOOOOOOOOOONQX]acZY\^ZZ[[[[\\[[[[[[[[[[ZZZZ[[[[[ZZYYYYYYZZ[[[[[[[[[[[[[[[[[[\]]Y\edcgddddddddccccccccccbbbcccddddccccccccccccccccccccccccccccbcf~vvvvvvvvvvppppppppppqleeeeeeko~qleeeeeeko~~sssssqlkkkkkkmottttttttttttttttttttttt{{|uqqqqqqqppoooooooppppppppppppppppppppppt{}xryurqqqqqqqqqqqqqqqqrrrrrrrrrrrrrqqqppppppty~tsnqw~urrqssssssstttttttssrrrrrrrrrrrqqqqppppppt{|xxoolpyrrrsqlleeeffggggggggggggggggggfffffgggggggotz~~|solqnnurruqeeeeeeedddddddefggggggggggfffffgggggggosyzxolkmnos{yrqskeedddddddeeeeffgggghhhhggggggggggggggggoty{xtojiimpqyyrqsicddddddddeeeeffgggghhhhggggggggggggggggioty{{vtrkgghmppyyrqsicddddddeeeefffgghhhhhhhgggggggggggggggggiotxz}~}{rqqkgfgimppyyrqsicddddddeeeefffgghhhhhhhggggggggggggggggggiotux|{{{zyxwuoongeehkmppyyrqsicddddddeeffgggghhhhhhhhggggggggggggggggggggloprvvuussqphgggffikmppyyrqsicdddddeeffggggghhhhhhhhgggggffggggggggggggghjkmnonmmkjiggggghikmppyyrqsidddeeeeffgghhhhihhhhhhhggggffffffggggggggggghhijiihhfffghgghijkmppyyqqsieeeffffffgghhhhhhhhhhhhggghhiiiihhhggggggggfgggggffgfffghhhhhjkmppyzqrrifffffffgggghhhhhhgggghghhhlpppppppplhhhgggggfggffffggghhhiiiiikmppyzprrhggggggggggghhhhhggggghhhlpppvvvvvvppplhgghgggggffffggghhhiiijjklopyzprrhggggggggggggggggggggghkpppzzpppihghhfffffffggghhiijjjjkkopyzprrhgggggggggggggggggggghkopzzpnjhggfffgggghhhhiijjjjjkkopzzppphggggggghhgggggggggghkotzzqsjhggffgghhhhiijjjjjjjkjnpzzppphggggggghhgggggggggginsx|rnjhhgghiiiiijjjjjjjjjkjnpzzppphggggggghhggggggggghlqw|wrmihhhhiiiijjjjjjjjjjkjnozzppphggggggghhggggggggginuz~}wqjhhhhiiijjjjjjjijjijjnozzppphggggggggggggggggghkmxynmihhhijjjjjjjjjiiiijjnozzppphggggggggggggggggginmnpjiiijjjjjjjjjjhiiiikoozzppphgggggggggggggggggiorrpliiijjjjjjjiihhiiiiloozzppphgggggggggggggggggjpuvooiiijjjjjjhghhhiiiiloozzppphgggggggggggggggggipwvooiiijjjjhhhhhhhiiiimpoyzppphgggggggggggggggggipwvooiiijjjhhhhhhhhiiiimppyzppphggggggggggggggggghouvooiiijhhhhhhhhhhiiiimppyzppphggggggggggggggggfgnrvoohhhhhhhhhhhhhiiiiimppyzppphggggggghhgggggggeflq|nqhhghhhhiiiiiiiijjjjmppyzppphggggggghhgggggggdekr{{{nohghhhhhiiiiiiiijjjjlopyzppphggggggghhgggggggddjqtqnlhghhhhhiiiiiiiiiiiikopyzppphgggggggggffffffgeehkr{xnmhhhhhhhhiiiiiiiiiiiikopzzppphgggggggggggggffgeegfpp|yopkhhhhhhhhiiiiiiiihhhhjnpzzppphgggggggfffffffffffggkou|{opiihhhhhhhhiiiiiihhhhhhjnpzzppphgggggggffffffffggggjgkpnyxmpiiihhhhhhhhiiiiiihhhhhhjnozzppphgggggggfffffffgggghighmmnq{zpmkjiiihhhhhhhhiiiiihhhhhhhjnozzpppgggggggggggggggghhhiijjjlmnpsuvw|uyonomkiiiiiiiiiiiiiiiihhhhhhhhjnozzpooggggggggggggggghhiiijjjjjlmnppqppppoljhhiiiiiiiiiiiiiiiihhhhhhhhjoozzpooggggggggghhhhhhhiijjjjjjjjjkkkllkjjihhhhiiiiiiiihhhhiiiihhhhhhhhjopzzoongggggggghhhhhhhiijjjjjjjjihhighhhhihhhhhiiiiiiiihhhhiiiihhhhhhhhkopzzonnggggggghhiiiiiijjjjjjjjjjiiihhhhhhhhhhhhiiiiiiiihhhhhhhhhhhhhhhhkopzzonnggggggghhiiiiijjjjjjjjjjiihhhhhhhhhhhhhhiiiiiiiihhhhhhhhhhhhhhhhkppzzonnggghhhhhiiiijjjjjjjjjhhhhhhhhhhhiihhiiiiiiiiiiiihhhhhhhhhhhhhhhhkppzzonlhhhhhhhhijkkkkkkjjjihhhhhhhhhhhhiiihiiiiiiiiiiiihhhhhhhhhhhhhhhhkppzzpolihhhhhhijjjjjjjjjjjihhhhhhhhiiiiiiiiiihhhhhhhhhhhhhhhhhhhhhhhhhhjop{zpolihhhhhhijjjjjjjjjhhhhhhhhhiiiiijiiiiihiihhhhhhhhhhhhhhhhhhhhhhhhioq}poljijiiiiijjjjjjhhhhhhhhhhhhhiiiiiiiiihhhhhhhhhhhhhhhhhhhhhhhhhhhhinq~ppljjjiiiiijjhhhhhhhhhhhhhhhhhgggghhiihhhhhhhhhhhhhhhhhhhhhhhhhhhhhinq~tpohikkjjjklllllllllllllllllkkkjjjjiihiihhhhhhhhhhhhhhhhhhhhhhhiiikppvtpomklmmmmlnnnnnnlllmmnmmmmllllkkkkkkkjjjjiiiiiiiiiiiiiiiiiiiijenonstoprrqqqqpppppppppppppppppppppppppppoooooopppppppppppppppppppoonpu~}yuttttttttttttttttttttttttuuuuutttssssssttttttttttttttttttttux} \ No newline at end of file diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-webp-logo-scaled-reference-lsel0.avif b/tests/Images/Input/Heif/Av1/Conformance/libavif-webp-logo-scaled-reference-lsel0.avif new file mode 100644 index 0000000000..6b8111e840 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-webp-logo-scaled-reference-lsel0.avif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e6e1f53f8dd558c5309e0c80b0635fa4efcbb636541e4e38206985a91e0fba66 +size 2505 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-webp-logo-scaled-reference.avif b/tests/Images/Input/Heif/Av1/Conformance/libavif-webp-logo-scaled-reference.avif new file mode 100644 index 0000000000..e883c8c619 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-webp-logo-scaled-reference.avif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:eb239f31ec8dbf5e97ad6f52670fca6497ae2a933822cfe724c75f66aaa2520b +size 2494 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-webp-logo-scaled-reference.bit b/tests/Images/Input/Heif/Av1/Conformance/libavif-webp-logo-scaled-reference.bit new file mode 100644 index 0000000000..7824a8572f --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-webp-logo-scaled-reference.bit @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b7e30e04a935414a517baa2df06ab756da18ba7c291220d9d7c063297761ae82 +size 2195 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-webp-logo-wedge-compound-libaom.y4m b/tests/Images/Input/Heif/Av1/Conformance/libavif-webp-logo-wedge-compound-libaom.y4m new file mode 100644 index 0000000000..2d3f2f87e2 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-webp-logo-wedge-compound-libaom.y4m @@ -0,0 +1,3 @@ +YUV4MPEG2 W80 H80 F25:1 Ip A0:0 C444 XYSCSS=444 XCOLORRANGE=LIMITED +FRAME +>>>>>>>>>>TTTTTTTTTTTv{UUs{U룱:>BB>Tctttttt^T?BCBBBBBBBBBBBBBBBBBBB\ssY>TTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTcrsdT>ASVxWTB:Ti½\T:ITǻTJTi˽\T-T˰jT--TɰjT--TȰjT--TȰjT--TȰjT--TȰjT--TȰjT--T{zjT--To\STTUUTXnjT--TgTUWz{VTUfjT--TzVUeeUVz~~jT--TtUTTTo~~~~jT--TzTVVTu~~~~~~cT--TVUTV|~~~~~~~cT--TbUT^~~~~~~~~~cT--TTdeTz~~~~~~~~~~}cT--TeUUf~~~~~~~~~}}|cT--TWVVV~~~~~~~~~~}|||cT--TT{{St~~~~~~~~~~~||||{cT--TyTTn~~~~~~~~~~}|||{|z}cT--TsTTo~~~~~~~}|||||{zzy}cT--TtTTn~~~~~~}|||||zyyyy|}^T--TzTTn~~~}}}||||{yyyyxxz}_T--T|T{|Tu~~}}|||||yyyyyyuuy}]T--TVVWW||||||||yyyyyyttttyz^T--TfUT^||||||yyyyyyytttttty^T--TyTlkTm||||yyyyyyytttttttty^T--T_TS\||||yyyyyyttttttttrtt^T--TwzUVVTt{{yyyyyytttttttttqqqs^T--TwnUddTj}yzyyyyyttttttttqqqqqq^T--TwgTVVTf|zyyyyyttttttttqqqqqqqq^T--TwnTTllTTjyyyyyyutttttttqqqqqqqqqq^T--Tw~t]TTWWST\nyyyyyyttttttttqqqqqqqqqqq^T--Tw~~~~}n_TTUUTTSS]kyyyyyyttttttttqqqqqqqqqqqqq^T--Tw~~~~~~~~~xojegmotyyyyyyttttttttqqqqqqqqqqqqqqq^T--Tw~~~~~~~~~|||||{{zyyyyyttttttttqqqqqqqqqqqqqqqqq^T--Tw~~~~~~~~~~}|||||yyyyyyyttttttttqqqqqqqqqqqqqqqqqq^T--Tw~~~~~~~~~~||||||yyyyyyyttttttttqqqqqqqqqqqqqqqqqqqq^T--Tw~~~~~~~~~}||||||yyyyyyttttttttqqqqqqqqqqqqqqqqqqqqqq^T--Tw~~~~~~~~~|||||||yyyyyytttttttttqqqqqqqqqqqqqqqqqqqqqqq^T-.Uo~~~~~~~~~~|||||{yyyyyyyttttttttqqqqqqqqqqqqqqqqqqqqqqqqq]T-,Uj~~~~~~~~~~||||||zyyyyyyttttttttrqqqqqqqqqqqqqqqqqqqqqqqqqq]T-T\~~~~~~~~~}}|||||zzyyyyttttttttqqqqqqqqqqqqqqqqqqqqqqqqqqqqq]TTT|||||zzyyyytttttttqqqqqqqqqqqqqqqqqqqqqqqqqqqqqnTT>T\}}||yyyyttttqqqqqqqqqqqqqqqqqqqqqqqqqqqqqnWT>JTWovwvvvvvvvvvvvvvwnnvnnnjnnnnjjfggggggggggggggggggggggggggggggcWTJBTUTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTUC>BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB>뀀")(("#")37883)$"37?FFE>72"efffffffff"*7BKRWRLB7)!\[[[[[[[[[""3?LWbcbXL?3#[MBBBBBBN[")7FSbouj_RB8)"[MBBBBBBN[+&#)6"$/7AIP\gd\\[[]_____________^^^^]]]]\\\\\\\\\\\\]UFB;6/(.:1)-))!!"'&"")*-3778@GPTU[oo[]bJAAAAAAAAAAAA@@@@@@@@@@@@@D@DDDDDDDDDDDDD>613.-**)))))*.3768BGGMTU[oo[^aJAAAAAAAAAAA@@@@@@@@@@@@D@DDDDDDDDDDDDDDD:=511*-------31678BGGGMTU[oo[^aJAAAAAAAAAA@@@@@@??????CDDDDDDDDDDDDDDDDDDB>661112222166=8BGGGHMTU[oo[^aJAAAAAAAA@@@@@@@?????C?DDDDDDDDDDDDDDDDDBBBB:==6666666<8BFGGGGGMTU[oo[^aJAAAAAAA@@@@@@@@@@@D@DDDDDDDDDDDDDDCDDDBBBBBBBB98=<=8BBHGGGGGFGMTU[oo[^aJAAAAA@A@@@@@@@@@@DDDDDDDDDDDDDDEDDDDBBBBBBBBBB?FAF@GGHGGGGGFFFMTU[oo[^^JAAA@A@@@@@@@@@@DDDDDDDDDDDDDDMMNKFBBBBBBBBBB??@GGGGGHGGGGFFFFFMTU[oo[^^JA@@@@@@@@@@@@D@DDDDDDDDDCGPZ[[[[[[XRJBBBBB@@HAHHHGGGGGGFFFFFFFLTU[oo[^^J@@@@@@@@@@@D@DDDDDDDDDDERZ[\fgjjgf[[\RDCB>H@GGHGGGGGGGFFFFFFCCLST[oo[^^J@@@@@@@@@@DDDDDDDDDDDDI\[\q||q\[\NG@GGGGGGGGGGFFFFFFCCAALST[oo[^^J@@@@@@@@DDDDDDDDDDCDDN[[n~n[[QFGGGGGGGGFFFFFFFCAAAALRT[oo[]^J@@@@@@D@DDDDDDDDDCDDH[[tt[[MGGGGGGFFFFFFFCAAAAAALRS[oo[]]J@@@@D@DDDDDDDDDDDDDF\[ut\]JGGGGFFFFFFCCAAAAAAALQS[oo[]]J@@@DDDDDDDDDDDDDDDDR\mm\SGGGEFFFFCCAAAAAAAAAKQSZoo[]]J@E@DDDDDDDDDDDDDDDK[^]\IFFGFFFCAAAAAAAAAAKKQSZoo[]^J@DDDDDDDDDDDDDDDDDS[qq[RGEFFCCAAAAAAAAJJJJJPSZoo[]\JDDDDDDDDDDDDDDDCDBW[~[]CFCCAAAAAAAAAAJJJJJOSZoo[\YJDDDDDDDDDDDDDDDBCK\ed[FCAAAAAAAAAAJJJJJJJMS[oo[\YJDDDDDDDDDDDDDBBBBG[kk\JAAAAAAAAAJJJJJJJIJLS[oo[[YJDDDDDDDDDDDDBBBBBM[nm[QAAAAAAAJJJJJJJHHHJKS[oo[[YJDDDDDDDDDDBBBBBBAM[om[QAAAAAAJJJJJJHHHHHJJS[oo[[YJDDDDDDDDBBBBBBBACN[kj[JABAAJJJJJJHHHHHHHHIS[oo[[YJDDDDDDDBBBBBBBBA?I[edZKBKJJJJJJJHHHHHHGGHIS[oo[[YIDDDCDBCBBBBBBBB@FH[\\WJIJJJJJJHHHHHHHGFFFHS[oo[[WIDDDBCBBBBBBBB@G@GGRZuq[TJJJJJJHHHHHHHGFFFFFHS[oo[[VIDCCBBBBBBBB@@@GFGGNZb~b[KJJIIHHHHHHHGFFFFFFGHS[oo[[UIBBBBBBBBB@@G@GGGGGGSZqqZYJJIIHHHHHHFFFFFFFGHHMS[oo[[UIBBBBBBBB@G@GGGGGGGGH[\vv\ZNIIHHHHHHFFFFFFFFGHHHGS[oo[ZUIBBCBAB@@@GGGGGGGGGGGJZ]ww][RJIHHHHHHFFFFFFFGHHHHHIS[op[ZVHBCBBA@?GGGGGGGGFFFFGFR[[qq[[RJHHHHHHFFFFFFFGHHHHHHHIS[op[ZVHBBBAF@GGGGGGGGGFFFFFGFR[[bttb[[QHIHHHHFFFFFFFFHHHHHHHHHIS[op[ZVHBAG@GGGGGGGGGGFFFFFFCCALZ[\\goqqog\[[YKHIHHHHFFFFFFFFHHHHHHHHHHIS[oo[ZUHAAGGGGGGGGGGFFFFFFCCAAAA@JS\\[[[[[[SPHHHHHHGFFFFFFFHHHHHHHHHHHHIS[oo[ZUHHGGGGGGGGGFFFFFFFCAAAAAAAA@JJRRRQIHHHHHHHFFFFFFFFHHHHHHHHHHHHHHIS[oo[ZUGGGGGGGGGFFFFFFFCBAAAAAAAAAJJJJJJIHHHHHHFFFFFFFGHHHHHHHHHHHHHHHHIS[oo[ZTGGGGGGGGFFFFFFBBAAAAAAAAAAJJJJJIIIHHHHHEFFFFFFGGHHHHHHHHHHHHHHHHIS[oo[ZRGGGGGGGFFFFFFBAAAAAAAAAAKJJJJIIIHHHHIEEFFFFFGGHHHHHHHHHHHHHHHHHHIS[oo[ZQGGGGGGFFFFFBAAAAAAAAAJJJJJJIIHHHHHIEEFFFFFGGHHHHHHHHHHHHHHHHHHHHIS[oo[ZQGGGGGGFFFBAAAAAAAAAAJJJJJJIIHHHHIEFFFFFFFGHHHHIHHHHHHHHHHHHHHHHHIS[op[VPFFFFFFFDAAAAAAAAAABKJJJIHHHHHHHHFFFFFFFFHHHHHHHHHHHHHHHHHHHHHHHHHS[pp[VPKGFFFFDAAAAAAAAAAJJJJJIIHHHHHHFFFFFFGGGGHHHHHHHHHHHHHHHHHHHHHHHHHW[p[YPMFFGCAAAAAAAAAKKJJJJJJIHHHHHFFFFFFFFHHHHHHHHHHHHHHHHHHHHHHHHHHHHHW[[[PQNLEFFGJAAKKKKKKKKJJJJIHHHHHFFFFFFFGHHHHHHHHHHHHHHHHHHHHHHHHHHHHH[[e[YQQQQQQLLLLLLKKKKKKKKKKIIIIHHHHFFFFHHHHHHHHHHHHHHHHHHHHHHHHHHHHHMW[f`[WRNSMLMSSMMMMMMSRRQQMQQQQQQQQQQQQPPPPPOOOOOOOOOOOOOOOOOOOOOOMMSW[ce[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[Z[[[[[[[[[[[[[[[[[[[[[[[[[[[[[cfcbbbccccccccccccccccccccccccccccccccccccccccccccccccccccccccfvwwwvvvvvvqqqqppppppqnddddddmppjddddddmpuuutvqpjjjjjmopttttttttttttttttttttttx}}}vpppppppppppppppppppppppppppppppppppppqw}}wpvtpqqqrrrrrrrrrrrrrrrrssssssrqpppppppppppqy}|vqpptvorqrrssssstttttttttssrrrrrrrrrrrrsssssporx~yvonnpvtptrojiggggggggggggggggggggggggggggggggggirq{{tplplptprsodddddddddddddddddheghhhhhhhhhggggggggglry|{vokklmopzprsidddddddddddddddddhhhhhhhhhhggggggggggggkrz}{vtqiemnopzzprsiddddddddddddeddhhhhhhhhhhghgggggggggggggmty{{vtqkfeklopzzprsiddddddddddfdhefhhhhhhhhhgggggggggggggggflotyy|{yttpkffgjlopzzprsiddddddddddhdfhhhhhhhhhgggggggggggggggggggjltuzyz{{{{zxtpqjffffklopzzprsiddddddddddfhhhhhhhhhggggggggggggggggggggggglopustttttppkefffffklopzzprtiddddddddghhhhhhhhhhggggggggggggggggggggggggggjlooooqkkeffffhhhllopzzprtidddddhdghhhhhhhhhgggggggggggggggggggggggggggggfgghgffffffgghiillopzzprsidddhdhhhhhhhhhhggggggggggggggjjjihgggggggggggggeeeeffffggigiihklopzzprsgdddhhhhhhhhhhggggggggggggimpppoopppmiggggggffffeeffffffiiiiiiiklopzzpqshddhhhhhhhhhhgggggggggggimppptvwwvtpppmigggfffffffffffghiiiiiiiklopzzpqrhhhhhhhhhhhggggggggggggjpprz~~zrpplfffffffffffghiiiiiiijjklopzzpqrhhhhhhhhhgggggggggggggjopyypomffffffffghiiiiiiijjjjjjopzzpprhhhhhhhggggggggggggggkppzzpojfffffggiiiiiiijjjjjjkjopzzppqhhhhhghgggggggggggggippzzqohffggiiiiiiiijjjjjjjkjopzzppphhhhggggggggggggggggppyxpogghhiiiiiijjjjjjjjjijopzzppphhghggggggggggggggggpssqlhhiiiiijjjjjjjjjjhhlopzzppphhggggggggggggggggglpyyplihhiiijjjjjjjjhhhhhlopzzppphggggggggggggggggggppppiiiiijjjjjjjjjhhhhllopzzppqhggggggggggggggghginttpjiijjjjjjjjjhhhhhhllopzzppqhggggggggggggggghgjowwpkjijjjjjjjhhhhhhhhllopzzppqhgggggggggggggggggjoxypljjjjjjjhhhhhhhhhhllopzzppqhgggggggggggggggggjpxypljjjjjjhhhhhhhhhhhhlopzzqpqhgggggggggggggggggjpvwpkjjjjhhhhhhhhhhhhhhlopzzqpphgggggggggggggggggiprtpjjihhhhhhhhhhhhhhihlopzzqpphggggggggggggggggfeqpoqihhhhhhhhhhhhhhiiiilopzzqpphggggggggggggggfffelp{{pohhhhhhhhhhhhhiiiiiikopzzqpphgggggggggggggfffffkqqspjhhhhhhhhhhhiiiiiiiikopzzqpphgggggggggggffeefffeqqy~wpohhhhhhhhhhiiiiiiiiihjopzzqpphgggggggggfgeeffffggiqp}}ppihhhhhhhhiiiiiiiiihhhjopzzqpphgggggggggeeefffffggijqr}}tpihhhhhhhhiiiiiiiihhhhhjopzzqpphgggggggeeeeffffefhihimppzzppkhhhhhhhiiiiiiiihhhhhhhiopzzqpphgggggggeeefffffiiiiiiimppr{{rppihhhhhhiiiiiiiihhhhhhhhhiopzzponiggfgfffffffffghiiiiiiijlnpppvvyyvvpppnlhhhhhhiiiiiiiihhhhhhhhhhiopzzpoliggfffffffffghhhiiiiijjjjjjoppppppppojhhhhhhiiiiiiiihhhhhhhhhhhhiopzzpolifffffffffghiiiiiiijjjjjjjjjikkkkkkhhhhhhhiiiiiiiihhhhhhhhhhhhhhiopzzqplifffffffghiiiiiiiijjjjjjjjjhhhhhhhhhhhhhiiiiiiiihhhhhhhhhhhhhhhhiopzzqplifffffffiiiiiiiijjjjjjjjjjhhhhhhhhhhhhhiiiiiiihhhhhhhhhhhhhhhhhhiopzzqplifffffifiiiiiijjjjjjjjjjhhhhhhhhhhhhhiiiiiiiihhhhhhhhhhhhhhhhhhhiopzzpplifffiiiiiiiijjjjjjjjjihhhhhhhhhhhhhiiiiiiiiihhhhhhhhhhhhhhhhhhhhiopzzppliffiiiiiiijjjjjjjjjjhhhhhhhhhhhhhiiiiiiiiiihhhhhhhhhhhhhhhhhhhhhiopzzpoliiiiiiiijjjjjjjjjjjhhhhhhhhhhhhhiiiiiiiihhhhhhhhhhhhhhhhhhhhhhhhiopzzpolkiiiiiiijjjjjjjjjjhhhhhhhhhhhhiiiiiiiiiihhhhhhhhhhhhhhhhhhhhhhhhiopzpolkjiiiiiijjjjjjihihhhhhhhhhhhiiiiiiiihhhhhhhhhhhhhhhhhhhhhhhhhhhhiopppmjkjiiiiiiihhhhhhhhhhhhhhhhhhiiiiiiihhhhhhhhhhhhhhhhhhhhhhhhhhhhhippupnjjiiiijkkkkklllllllllllllhlhhhiiiiihhhhhhhhhhhhhhhhhhhhhhhhhhhhjopvspomnmnmmmmnnnnnnlllllmmnnimmmmiimljiiiiiiiiiiiiiiiiiiiiiiiiiijjopptupppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppptvuuttttttttttttttttttttttttttttttttttttttttttttttttttttttttttv \ No newline at end of file diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-webp-logo-wedge-compound.avif b/tests/Images/Input/Heif/Av1/Conformance/libavif-webp-logo-wedge-compound.avif new file mode 100644 index 0000000000..d48a785592 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-webp-logo-wedge-compound.avif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:98640640a445055fea3d9e2f4a78feef54e97f8171b472cf57d99156e1c553b2 +size 6439 diff --git a/tests/Images/Input/Heif/Irvine_CA.avif b/tests/Images/Input/Heif/Irvine_CA.avif new file mode 100644 index 0000000000..81cd4c05ed --- /dev/null +++ b/tests/Images/Input/Heif/Irvine_CA.avif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ee9b8544668ba71e584311be8eb590d0a92464aa24aa75ab05af92ab4c9ccf4c +size 28133 diff --git a/tests/Images/Input/Heif/Orange4x4.avif b/tests/Images/Input/Heif/Orange4x4.avif new file mode 100644 index 0000000000..82f255d118 --- /dev/null +++ b/tests/Images/Input/Heif/Orange4x4.avif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:de6d438ed2808d8a622ce5288f683f033eb15426aea0debfe0278f054f8eb2e3 +size 299 diff --git a/tests/Images/Input/Heif/colors-animated-12bpc-keyframes-0-2-3.avif b/tests/Images/Input/Heif/colors-animated-12bpc-keyframes-0-2-3.avif new file mode 100644 index 0000000000..1550d842f4 --- /dev/null +++ b/tests/Images/Input/Heif/colors-animated-12bpc-keyframes-0-2-3.avif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3bf9f91da471749e7df639ba7945d4d94c1c3e3968c26f3619fbbcfc92790576 +size 2267 diff --git a/tests/Images/Input/Heif/colors-animated-8bpc-alpha-exif-xmp.avif b/tests/Images/Input/Heif/colors-animated-8bpc-alpha-exif-xmp.avif new file mode 100644 index 0000000000..a05d5438b7 --- /dev/null +++ b/tests/Images/Input/Heif/colors-animated-8bpc-alpha-exif-xmp.avif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c2e38681057c15009c4b76ea08cea68cdde80806abd41d42a646f697bf5aabb2 +size 7506 diff --git a/tests/Images/Input/Heif/colors-animated-8bpc-audio.avif b/tests/Images/Input/Heif/colors-animated-8bpc-audio.avif new file mode 100644 index 0000000000..af83fa7792 --- /dev/null +++ b/tests/Images/Input/Heif/colors-animated-8bpc-audio.avif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:624f3bfe78b6bd75e9e12fe9b36c6132e3effaf82aa2a443f1b2a207a7d3561b +size 3505 diff --git a/tests/Images/Input/Heif/colors-animated-8bpc.avif b/tests/Images/Input/Heif/colors-animated-8bpc.avif new file mode 100644 index 0000000000..ac90131fc5 --- /dev/null +++ b/tests/Images/Input/Heif/colors-animated-8bpc.avif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2f8683d21725261f37f86e115f0c212cc52d0fefd3a2ddfcc4fa648c1859906d +size 1235 diff --git a/tests/Images/Input/Heif/ducky_romm_icc_alpha.avif b/tests/Images/Input/Heif/ducky_romm_icc_alpha.avif new file mode 100644 index 0000000000..21b44e40df --- /dev/null +++ b/tests/Images/Input/Heif/ducky_romm_icc_alpha.avif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9419a5e9893bf7236aed6ee1b0bd50578ff59d8e63a1e61aea0761142afa2b20 +size 5344 diff --git a/tests/Images/Input/Heif/jpeg444_xnconvert.avif b/tests/Images/Input/Heif/jpeg444_xnconvert.avif new file mode 100644 index 0000000000..5080be098d --- /dev/null +++ b/tests/Images/Input/Heif/jpeg444_xnconvert.avif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:73c0521c669759df918cd59865537cd9cef3b7502ce3da825d8d24d9997e0e0d +size 1242 diff --git a/tests/Images/Input/Heif/paris_icc_exif_xmp.avif b/tests/Images/Input/Heif/paris_icc_exif_xmp.avif new file mode 100644 index 0000000000..d46a633cb5 --- /dev/null +++ b/tests/Images/Input/Heif/paris_icc_exif_xmp.avif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:961bc38b61e60b7651fa20efa24269ae2f35e4958822a81c908c9bbf9b3f66e1 +size 21132 diff --git a/tests/Images/Input/Heif/paris_icc_exif_xmp.png b/tests/Images/Input/Heif/paris_icc_exif_xmp.png new file mode 100644 index 0000000000..2da25e38ae --- /dev/null +++ b/tests/Images/Input/Heif/paris_icc_exif_xmp.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:308bb51c3338ed9519cdaaa79186b934a21d1748d94120ed0348eb4f193d0988 +size 153217 diff --git a/tests/Images/Input/Heif/perceptual_icc.avif b/tests/Images/Input/Heif/perceptual_icc.avif new file mode 100644 index 0000000000..7258fb301b --- /dev/null +++ b/tests/Images/Input/Heif/perceptual_icc.avif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:438b5d2bc7b32206b62676d6d460fb0037b857aa51dd03049867f68755227b3b +size 205370 diff --git a/tests/Images/Input/Heif/perceptual_icc_grid.avif b/tests/Images/Input/Heif/perceptual_icc_grid.avif new file mode 100644 index 0000000000..9d183f1988 --- /dev/null +++ b/tests/Images/Input/Heif/perceptual_icc_grid.avif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:337e2cbd72043eda399b65189b563ce4935086a05f671414d1c1667abfd0a18f +size 205897 diff --git a/tests/Images/Input/Heif/perceptual_icc_sequence.avif b/tests/Images/Input/Heif/perceptual_icc_sequence.avif new file mode 100644 index 0000000000..ae7154dd30 --- /dev/null +++ b/tests/Images/Input/Heif/perceptual_icc_sequence.avif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6d444b3e616a51375777b1bf8a01606682850274dd2be691e393f2ed8cbbf2c1 +size 325372