Add intra-only 4:4:4 (ChromaArrayType 3) decoding — byte-exact - #1
Open
JensWalter wants to merge 1 commit into
Open
Add intra-only 4:4:4 (ChromaArrayType 3) decoding — byte-exact#1JensWalter wants to merge 1 commit into
JensWalter wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
I am currently working on a image gallery server (https://codeberg.org/JensWalter/everlock) and found that some iPhone generated pictures can be decoded, and some cannot. The distinction here was the selected Chromatype (Apple use one for Photos, and one for edited Photos).
So after digging around I could add some ChromaType 4:4:4 support, although my knowledge about this is rather superficial. So I tried to make the changes as least invasive as possible, because of this. I also added some tests, specifically with sample pictures.
If you are open to merge that would be great. If my PR does not meet your expectation, I am willing to refine whatever you see as not working or needs refinement.
Also here is some description from Claude about this change:
Summary
The decoder was 4:2:0-only. This adds intra 4:4:4 support (8-bit and
10-bit,
ChromaArrayType == 3), which is the format HEIF/HEIC still images use— in particular Apple "edited"/HDR photos, which are commonly 4:4:4 10-bit and
previously failed to decode with
Unsupported("only 4:2:0 … supported").Output is byte-exact against FFmpeg. The 4:2:0 path is untouched
(byte-identical), and every change is gated on
ChromaArrayType, so existingbehaviour is preserved. 4:4:4 inter (P/B) slices are explicitly rejected
(see Limitations) rather than reconstructed with the wrong chroma.
Motivation
HEVC-in-HEIF still images from recent iPhones are frequently coded as 4:4:4
10-bit (e.g. after on-device editing or HDR capture). Decoding one previously
returned an
Unsupportederror at SPS parse. This change makes those imagesdecodable while keeping the byte-exact 4:2:0 guarantees intact.
Testing
New byte-exact tests and fixtures (same style as the existing suite —
decode_and_hashagainst a committed.h265, with the generating commanddocumented above each test):
test_decode_444_intra_byte_exact— 96×96 8-bit 4:4:4 intra. Exercisesfull-resolution chroma TBs, chroma reference smoothing (PLANAR/angular/DC),
per-PU
intra_chroma_pred_modeforPART_NxN(62 such CUs in the fixture),chroma SAO/deblock, and the conformance-window crop (96 is not CTB-aligned at
CTU 64).
test_decode_444_intra_10bit_byte_exact— 96×96 10-bit 4:4:4 intra(the HEIC photo format).
test_decode_444_inter_rejected— asserts a 4:4:4 P/B slice returnsUnsupported.Verification performed:
reconstruction is byte-identical.
10-bit; SAO, deblock, sign-hiding, strong-intra-smoothing,
PART_NxN).0 / 11,358,585 samples differ,
max|Δ| = 0on Y, U, and V.cargo clippyclean.Limitations / follow-ups
Unsupported.Enabling it would require parameterizing the chroma motion-compensation call
sites in
inter_pred.rs(chroma block size, stride, coordinates, and themv << (1 - shift)scaling) and enlarging the internal chroma scratchbuffers (currently sized for 4:2:0). Luma inter already works for 4:4:4.
CTB-aligned picture dimensions for the reference luma stride; non-aligned
resolutions are decoded correctly for intra but not for inter. The existing
test fixtures all use CTB-aligned resolutions.