Complete H.264 reference marking and stream capability checks

.github/workflows/release-ci.yml

@@ -270,7 +270,7 @@ jobs:
           make -C toolchain/v
           echo "$GITHUB_WORKSPACE/toolchain/v" >> "$GITHUB_PATH"
       - name: Install native build dependencies
-        run: sudo apt-get update && sudo apt-get install -y build-essential cmake libglfw3-dev libvulkan-dev libvulkan-volk-dev pkg-config
+        run: sudo apt-get update && sudo apt-get install -y build-essential cmake libglfw3-dev libvulkan-dev libvulkan-volk-dev pkg-config mesa-vulkan-drivers xvfb
       - name: Install headers matching Vulkan master
         run: |
           set -euo pipefail
@@ -295,6 +295,9 @@ jobs:
           cd "$GITHUB_WORKSPACE/source/v_vulkan_video"
           v -new-compiler -cc tcc test .
           v -new-compiler -cc tcc -o "$RUNNER_TEMP/v_vulkan_video" .
+          timeout 20s xvfb-run -a "$RUNNER_TEMP/v_vulkan_video" --list-gpus res/20240917_095400.mp4 > "$RUNNER_TEMP/v3-gpus.log" 2>&1
+          cat "$RUNNER_TEMP/v3-gpus.log"
+          grep -F 'Vulkan devices for H.264' "$RUNNER_TEMP/v3-gpus.log"
 
   windows:
     name: Windows / ${{ matrix.compiler }}
@@ -419,6 +422,10 @@ jobs:
           if ($LASTEXITCODE -ne 0 -or $HelpText -notmatch 'Usage:') {
             throw 'The Windows player did not print its command-line help.'
           }
+          $GpuText = (& .\dist\vkvideo-windows-x64\v_vulkan_video.exe --list-gpus .\dist\vkvideo-windows-x64\res\20240917_095400.mp4 | Out-String)
+          if ($LASTEXITCODE -ne 0 -or $GpuText -notmatch 'Vulkan devices for H.264') {
+            throw 'The Windows player could not enumerate Vulkan devices.'
+          }
           $Flags = @('-cc', 'msvc', '-cflags', '/DWIN32_LEAN_AND_MEAN')
           if ($Compiler -eq 'v3') { $Flags += '-new-compiler' }
           v @Flags test .

BUILDING.md

@@ -62,10 +62,13 @@ a platform-independent archive.
 ./v_vulkan_video [--list-gpus] [--gpu INDEX] [--decode-output-mode MODE] [video.mp4]
 ```
 
-GPU compatibility is evaluated against the input stream's actual H.264
-profile. Without `--gpu`, the first fully compatible presentation/decode device
-is selected. Invalid media and unsupported or out-of-range devices return a
-clean non-zero exit status with a diagnostic instead of a panic.
+GPU compatibility is evaluated against the input stream's H.264 profile,
+coded extent, DPB slots, active references, output mode, and image formats.
+Without `--gpu`, the first fully compatible presentation/decode device
+is selected. Unsupported codecs and profiles, missing slice parameter-set
+references, truncated MP4 samples, and incompatible or out-of-range devices
+return a non-zero exit status with a diagnostic. Arbitrarily corrupted SPS/PPS
+bitstreams are not fully validated by the pinned H.264 parser.
 `--decode-output-mode auto|coincident|distinct` selects an advertised DPB and
 output-image mode; `auto` is the default.
 

PLATFORM_SUPPORT.md

@@ -12,9 +12,15 @@ A normal Vulkan graphics driver does not necessarily provide Vulkan Video.
 | macOS | Unsupported for video decode | The UI bindings can be built for macOS, but this application requires Vulkan Video H.264 decode. Do not treat a MoltenVK graphics-capable system as proof of Vulkan Video support. |
 
 Linux V3 compilation is tracked by CI as an experimental, non-release-gating
-job. With V master at `efae23e85b`, the complete player builds with TinyCC and its
-three software-only test files pass. Playback with V3 on supported hardware
-has not yet been validated, so stable V remains the release compiler.
+job. With V master at `b99970bd438a7bdcdfbe38f74d9364db801d5439`, the complete
+player builds with TinyCC and its three software-only test files pass. On the
+Linux GTX 1060, V3/TinyCC decoded the ID-7 fixture (5 frames) and multislice
+fixture (24 frames) byte-for-byte identically to FFmpeg and exited cleanly on
+Escape. The player uses the binding's loader initialization and passes the
+swapchain semaphore by value. Raw Volk initialization can collide with Linux
+TinyCC's exported dispatch variables; a mutable handle parameter was lowered
+to its address by this compiler. Stable V remains the release compiler while
+broader V3 hardware and platform coverage is completed.
 
 On Windows, source-built V master at `b99970bd438a7bdcdfbe38f74d9364db801d5439`
 passed the V3/MSVC package build, all three software test files, `--help`,
@@ -44,6 +50,27 @@ The application currently decodes H.264/AVC video carried in MP4. It supports
 8-bit 4:2:0 progressive Baseline, Main, and High profiles when the driver
 reports a compatible Vulkan Video profile. Other codecs, chroma formats,
 bit depths, and interlaced streams are rejected with an explanatory error.
+Picture-order-count types 0, 1, and 2 are calculated for progressive frames.
+Separate top and bottom order counts are supplied to Vulkan references.
+The DPB applies sliding-window marking and explicit MMCO 1–6, including
+long-term references. The bundled 360p stream exercises MMCO 1 on the tested
+Linux GPU. The external `MR2_TANDBERG_E` conformance stream exercises MMCO 5
+and long-term operations 3, 4, and 6; `FRExt_MMCO4_Sony_B` exercises long-term
+operations 2, 3, 4, and 6. On the Linux GTX 1060, decoded NV12 output matched
+FFmpeg byte for byte for all 300 Tandberg frames and all 60 Sony frames. The
+bundled four-slice and ID-7 fixtures also matched for all 24 and 5 frames.
+The Sony comparison exposed a scaling-list bug: the pinned H.264 parser set
+the SPS list-presence flags but left the list values at zero. The player now
+[fills the validated lists](h264_parameter_sets.v#L77) before creating
+[Vulkan session parameters](decoder_session.v#L303).
+
+AVC samples with 1, 2, or 4 byte NAL length prefixes are accepted. The parser
+skips metadata-only samples, checks every slice in a sample belongs to the
+same picture, and reports invalid slice references before creating a Vulkan
+device. SPS/PPS preflight checks truncated syntax and the pinned parser's
+fixed-array limits. The pinned H.264 module's weighted-prediction reader is
+corrected in this application's checked slice reader. Device selection also
+checks the stream's H.264 level against the GPU's reported maximum.
 
 B-frame streams are decoded in codec order and retained in a bounded image
 queue until they become next in presentation order. The queue size is derived
@@ -53,16 +80,19 @@ have completed. The bundled Big Buck Bunny fixtures cover this path at 360p,
 720p, and 1080p.
 
 Unsupported media, missing Vulkan Video extensions, and incompatible GPU
-profiles produce orderly diagnostics and a non-zero exit status. Unexpected
-failures after Vulkan device creation (for example, allocation, swapchain, or
-queue-submission failures) remain fatal because teardown from partially
-recorded or submitted command buffers is not yet modeled as recoverable. These
-driver/runtime failures are tracked as post-release lifecycle hardening rather
-than being conflated with malformed-input handling.
+profiles produce orderly diagnostics and a non-zero exit status. The pinned
+H.264 parser and this application's preflight do not validate every semantic
+relationship inside arbitrarily corrupted SPS/PPS bitstreams. Failures after
+Vulkan device creation (for example, allocation, swapchain, or queue-submission
+failures) remain fatal because teardown from
+partially recorded or submitted command buffers is not yet modeled as
+recoverable. These driver/runtime failures are tracked as post-release
+lifecycle hardening rather than being conflated with malformed-input handling.
 
 Hardware is selected by capability rather than vendor name: the device must
 provide graphics/presentation, the required Vulkan Video extensions, an H.264
-decode queue, and a supported decode output format. The decoded-picture-buffer
+decode queue, coded extent and reference limits, and output/DPB formats with
+the required image usages. The decoded-picture-buffer
 and output-image mode is chosen from the modes reported by the driver. Use
 `--decode-output-mode coincident` or `--decode-output-mode distinct` to force a
 specific advertised path during compatibility testing; `auto` remains the
@@ -82,6 +112,50 @@ synchronization, and presentation still require a real Vulkan Video device.
 
 ## Hardware validation checklist
 
+To repeat the H.264 reference-marking smoke check, download the
+[MR2 Tandberg stream](https://dev.gentoo.org/~lu_zero/fate/h264-conformance/MR2_TANDBERG_E.264)
+and the
+[FRExt Sony stream](https://dev.gentoo.org/~lu_zero/fate/h264-conformance/FRext/FRExt_MMCO4_Sony_B.264),
+then remux them to MP4 without transcoding:
+
+```sh
+ffmpeg -r 30 -i MR2_TANDBERG_E.264 -c:v copy MR2_TANDBERG_E.mp4
+ffmpeg -r 25 -i FRExt_MMCO4_Sony_B.264 -c:v copy FRExt_MMCO4_Sony_B.mp4
+./v_vulkan_video MR2_TANDBERG_E.mp4
+./v_vulkan_video FRExt_MMCO4_Sony_B.mp4
+```
+
+The files are external conformance media and are not included in the repository.
+Inspect `memory_management_control_operation` with FFmpeg's `trace_headers`
+bitstream filter to confirm which operations each stream contains.
+
+To compare decoded pixels, set `VV_DUMP_NV12_DIR` to an empty directory before
+running the player. The [readback path](frame_readback.v#L48) copies the first
+playback loop's decoded images to display-order `N.nv12` files. Run the
+[comparison script](scripts/compare_nv12.py#L25) against the original H.264
+elementary stream for the Sony case: FFmpeg's MP4 remux has nonmonotonic
+timestamps and drops frames when exporting raw video.
+
+```sh
+VV_DUMP_NV12_DIR=/tmp/sony-nv12 ./v_vulkan_video FRExt_MMCO4_Sony_B.mp4
+python3 scripts/compare_nv12.py FRExt_MMCO4_Sony_B.264 /tmp/sony-nv12
+```
+
+The readback waits for decode fences, invalidates mapped memory, and writes
+only the first playback loop. It costs extra GPU memory and stalls that loop;
+leave the environment variable unset for normal playback. An exact match
+checks decoded NV12 bytes and display order on this GPU. It does not verify
+the YCbCr-to-RGB rendering or another driver's decode implementation.
+
+The rendered window was checked separately on the same GPU. An X11 capture of
+the bundled ID-7 color-bar fixture (BT.601 limited-range fallback) was compared
+with FFmpeg's RGB output at nine interior pixels. A generated H.264
+`smptehdbars` clip signaling BT.709 and full range was compared at eight
+interior pixels, with FFmpeg's scale filter explicitly set to BT.709/full
+input. The largest per-channel difference was 2 in 8-bit RGB in both checks.
+These spot checks cover the two indicated conversion paths, not every output
+pixel, chroma edge, display compositor, or GPU driver.
+
 Before calling a platform supported for release, run at least:
 
 - playback through multiple loops;

QUICKSTART.md

@@ -32,9 +32,10 @@ Install the [V compiler](https://github.com/vlang/v), then from this checkout:
 ```
 
 Use `./scripts/build_linux.sh --compiler v3` to build the full player with V3.
-Current V master builds the complete player with V3 and TinyCC on Linux, but V3
-playback has not yet been validated on supported hardware. Use the stable
-compiler for release builds. The V3 build uses TinyCC by default; pass
+Current V master builds the complete player with V3 and TinyCC on Linux, and
+the ID-7 and multislice fixtures have passed decoded-frame comparisons with
+FFmpeg on a GTX 1060. Broader V3 hardware coverage remains incomplete. Use the
+stable compiler for release builds. The V3 build uses TinyCC by default; pass
 `--cc gcc` if more detailed C diagnostics are needed. The wrapper also offers
 `--linkage static` and `--glfw bundled --glfw-version 3.4`; run it with `--help`
 for all choices.

README.md

@@ -86,10 +86,10 @@ and follow the chapters that match the problem you are solving.
 v test .
 ```
 
-The software-only tests cover MP4 metadata and validation, H.264 picture order,
-malformed and truncated inputs, playback timing, looping, and command-line
-parsing. Vulkan decode, synchronization, resize, and presentation still
-require hardware with Vulkan Video support.
+The software-only tests cover MP4 metadata and validation, H.264 parameter sets,
+multi-slice pictures, picture order, malformed and truncated inputs, playback
+timing, looping, and command-line parsing. Vulkan decode, synchronization,
+resize, and presentation still require hardware with Vulkan Video support.
 
 ## License