Compare shader metrics: stock MaterialX vs Metashade Standard Surface

[ppenenko]

Goal

Quantitatively compare the stock MaterialX Standard Surface implementation (flattened nodegraph) against the Metashade reimplementation (factored BSDF function) across shader quality and performance metrics.

Background

The Metashade Standard Surface generates a standalone mx_metashade_standard_surface_bsdf function that is called twice (for INDIRECT and TRANSMISSION closure types), while the stock codegen flattens the entire nodegraph into the surfaceshader function, duplicating all BSDF node calls for each closure type. The Metashade approach also uses inline arithmetic instead of graph-level composition operators (mx_add_bsdf, mx_layer_bsdf, etc.).

We already have 171 comparable GLSL baselines committed in the repo:

• Stock: contrib/renders/{aswf,adsk}/.../*_ps.glsl (186 files)
• Metashade: contrib/tests/metashade_ref/renders/standard_surface/{aswf,adsk}/.../*_ps.glsl (66 files)

Phase 1: Offline shader analysis (no test run needed)

Run diff_shaders.py on the committed GLSL baselines:

python python/MaterialXTest/diff_test_runs/diff_shaders.py \
    contrib/renders \
    contrib/tests/metashade_ref/renders/standard_surface \
    -o stock_vs_metashade.html

Metrics:

• LOC (non-blank lines) — built-in
• SPIR-V binary size — requires glslangValidator in PATH
• SPIR-V compile time — requires glslangValidator in PATH
• spirv-opt output size and time — requires spirv-opt in PATH

These metrics are deterministic and cache-independent.

Phase 2: Runtime comparison via Python render tests

Use the existing Python render test infrastructure (test_render_metashade.py) to compare:

• GPU render time — via GL timer queries
• Online shader compilation time — via glBuildProgram timing
• Rendered image quality — FLIP scores (already verified in render tests)

Open questions for Phase 2

• Perfetto tracing from Python: The MaterialXTrace module is C++-only; no Python bindings exist for PerfettoSink/Dispatcher. Need to either expose tracing to Python or use time.perf_counter() as an alternative.
• Driver cache: GPU compilation time comparisons are only representative with an empty driver cache. A per-test-case cache purge would be ideal but is incompatible with xdist parallelism. Consider running sequentially for profiling runs.
• Codegen time: Can be measured at the Python level around MaterialX.GenShader calls.

Related

• Measurement infrastructure PRs: Extended tracing instrumentation for shader codegen and rendering AcademySoftwareFoundation/MaterialX#2820, Add envSampleCount and framesPerMaterial test suite options AcademySoftwareFoundation/MaterialX#2821, Python comparison scripts for test suite output (diff_test_runs) AcademySoftwareFoundation/MaterialX#2822, GPU async timing instrumentation via MX_TRACE_ASYNC AcademySoftwareFoundation/MaterialX#2824
• Lobe pruning optimization: Early lobe pruning during ShaderGraph construction AcademySoftwareFoundation/MaterialX#2844, Lobe Pruning optimization in MaterialX proper AcademySoftwareFoundation/MaterialX#2680
• Standard Surface extraction: Extract Standard Surface BSDF into metashade.mtlx.standard_surface metashade#227

[ppenenko]

Static register pressure analysis

Unlike lobe pruning (where spirv-opt produced identical binaries, making register analysis redundant), Metashade vs stock generates structurally different shaders — factored function call vs flattened inline code. Register allocation may meaningfully diverge, directly impacting GPU occupancy and throughput.

Available offline tools:

Tool	Vendor	Input	Key metrics
AMD RGA	AMD	GLSL, SPIR-V	VGPRs, SGPRs, ISA size, occupancy, live register analysis
Arm Mali Offline Compiler (malioc)	Arm	ESSL, SPIR-V	Work registers, uniform registers, stack spills, cycle counts
NVIDIA Nsight Graphics	NVIDIA	Live app only	Registers per thread, SASS profiling — no offline mode

RGA was previously in diff_shaders.py but removed since it was redundant for lobe pruning. It should be re-added for this comparison. CLI: rga -s vk-offline-spv --isa out.isa --livereg report.txt input.spv

malioc would add the mobile perspective. CLI: malioc -c Mali-G78 shader.frag

RGA also has a VS Code extension.