Consider relaxing alpha bit depth constraint

[y-guyon]

Section 4.1 says:

An AV1 Alpha Image Item (respectively an AV1 Alpha Image Sequence) shall be encoded with the same bit depth as the associated master AV1 Image Item (respectively AV1 Image Sequence).

This may have benefits for software or hardware implementations. However higher precision than 8 bits for alpha is rarely necessary.
I did not test whether the compression rate could be improved by sticking to 8-bit alpha samples along a 10+-bit color image, but restricting the range to useful values would not hurt on the file format design side.

[y-guyon]

Note that this was enforced in #74.

[leo-barnes]

Hmmm. That's a fair point. I have a vague memory that MIAF maybe says something about this. We should verify that. If it doesn't I don't really have any objections to relaxing it.

You're mainly arguing for allowing a smaller bit-depth for alpha. Do we want to keep the restriction that bit-depth of alpha shall be <= bit-depth of RGB?

[FooIbar]

This is more relevant with YCgCo-R.
8-bit RGBA will be encoded to 10-bit YCgCo-Re + 10-bit alpha, which unnecessarily increases the file size

[y-guyon]

This is more relevant with YCgCo-R.
8-bit RGBA will be encoded to 10-bit YCgCo-Re + 10-bit alpha, which unnecessarily increases the file size

Do you have any file size figures of 8-bit alpha samples encoded as 10-bit by any chance?

Note that with AVIF 1.2 one could use a sato item to encode alpha samples to 8-bit and shift them to 10-bit to comply with the constraint, but this is way too complex and backward-incompatible.

[FooIbar]

The image from https://github.com/AOMediaCodec/libavif/blob/main/tests/data/abc.png for example:

avifenc 1.3.0	--lossless	--lossless --cicp 2/2/16	Δ
Color Size	12367 B	4588 B	-63%
Alpha Size	3641 B	5484 B	+51%
Total Size	16438 B	10502 B	-36%

And I'm seeing a roughly 60% alpha size increase with my corpus (mainly sprites).

[FooIbar]

Since AV2 has native alpha support, can you check if there's such limitation as well? Its spec is currently internal.

[y-guyon]

I have a vague memory that MIAF maybe says something about this

I was unable to find anything related to the alpha bit depth in MIAF.
However I found this about dimensions: When the width or the height of the alpha plane differs from the width or the height of the master image, respectively, the alpha plane is resized to have the same width and height as those of the master image.

I don't really have any objections to relaxing it

It would be a backward-compatible change in the sense that all existing conformant files would stay compliant with current and future implementations.
It would be a backward-incompatible change in the sense that new files with different alpha bit depth could be rejected by current implementations.
I do not mind changing the specification to allow for it: #324

Do we want to keep the restriction that bit-depth of alpha shall be <= bit-depth of RGB?

For consistency with dimensions that can be larger or smaller, and for simplicity, I would NOT keep the restriction.

Since AV2 has native alpha support, can you check if there's such limitation as well?

I could not find such a limitation in the AV2 specification draft. Assuming the current AVIF specification supported AV2, it would not be constrained because it is not an auxiliary item but a native channel in the color item.

[FooIbar]

I could not find such a limitation in the AV2 specification draft. Assuming the current AVIF specification supported AV2, it would not be constrained because it is not an auxiliary item but a native channel in the color item.

Sorry, what I meant was: does AV2 support different bit depths for color and alpha channels in the same sequence header?
Or was that size increase caused by libavif's alpha implementation, and wouldn't happen with AV2's native alpha support so the bit depth for alpha doesn't matter?
I tried the ToT avmenc, but it doesn't seem to support 444A input.

[y-guyon]

Sorry, what I meant was: does AV2 support different bit depths for color and alpha channels in the same sequence header?

I understood that and answered that I could not find such a limitation in AV2 (or explicit permission). And assuming different bit depths are supported in AV2 for alpha and color samples in the same sequence, this would not be an issue for AVIF because there would be a single item or track, and the AVIF constraint only applies to auxiliary items.

Or was that size increase caused by libavif's alpha implementation, and wouldn't happen with AV2's native alpha support so the bit depth for alpha doesn't matter?

This is a possibility.