NaN/Inf safety: per-operator guards, output wrapping, and diagnostic environments

[ppenenko]

Summary

Define the full NaN/Inf risk surface in MaterialX shader math, and design Metashade-based mitigation strategies: per-operator safe math guards, per-node output wrapping, and a diagnostic environment that visualizes NaN-producing nodes.

Builds on the safe math concept from #14 and the multi-environment infrastructure from #13.

Full NaN/Inf Risk Surface

High-risk operations (produce NaN/Inf from finite inputs)

Operation	Risk	Safe wrapper
x / 0	Inf; 0/0 -> NaN	safe_div(a, b) -- clamp denominator
pow(negative, non-integer)	NaN (undefined in GLSL)	safe_pow -- pow(max(base, 0.0), exp)
sqrt(negative)	NaN	safe_sqrt -- sqrt(max(x, 0.0))
log(0) / log(negative)	-Inf / NaN	safe_log -- log(max(x, 1e-8))
asin(x), acos(x) where abs(x) > 1	NaN (common after interpolation)	Clamp input to [-1, 1]
inversesqrt(x) / rsqrt(x)	NaN for negative, Inf for zero	rsqrt(max(x, 1e-8))
exp(large_x)	Inf	Clamp input
normalize(zero_vector)	NaN (internally v * rsqrt(dot(v,v)))	Length check before normalize

Medium-risk (propagate NaN, amplify Inf)

• reflect(I, N) / refract(...) -- propagates NaN from un-normalized inputs
• atan(y, x) -- undefined at origin on some hardware (usually okay in practice)

Two Complementary Strategies

Strategy 1: Per-operator safe math guards

Intercept risky operations at the point of NaN/Inf creation.

Metashade injection points:

• ArithmeticType._rhs_binary_operator() in _clike/dtypes.py -- for /
• FloatIntrinsicsMixin in glsl/_intrinsics.py -- for pow, sqrt, log, asin, acos
• _RawVector._per_element_or_scalar() in _rtsl/dtypes.py -- vector division

Tradeoff: Prevents NaN creation at the root cause, but adds a branch/clamp on every risky operation.

Strategy 2: Per-node output wrapping

Intercept every node output assignment to validate the result.

float mx_validate(float x) {
    return (isinf(x) || isnan(x)) ? 0.0 : x;
}

Metashade injection point:

• Generator.__setattr__ -- every sh.result = expr goes through it, so all node output assignments can be intercepted and wrapped with validation.

Tradeoff: One check per node output (lower overhead than per-operator), but catches symptoms rather than root causes.

Diagnostic variant: NaN -> pink

Output wrapping becomes a powerful diagnostic tool when the fallback value is a visible signal instead of zero:

vec3 mx_validate_debug(vec3 x) {
    return any(isnan(x)) ? vec3(1.0, 0.0, 0.5) : x;
}

This mirrors the pink Schlick proof-of-concept from #13, but applied to error detection -- artists and TDs can visually identify which material regions produce NaN, making it far easier to track down render artifacts than inspecting pixel values.

Proposed Environments

Environment	Strategy	Purpose
safe_math/	Per-operator guards	Production: silently prevent NaN/Inf creation
nan_debug/	Output wrapping with pink fallback	Diagnostic: visualize NaN-producing nodes
safe_math_strict/	Both combined	Belt-and-suspenders for debugging tough cases

All three leverage the --library compatible multi-environment system from #13:

# Production safety
MaterialXView --library contrib/tests/metashade_envs/safe_math material.mtlx

# Diagnostic mode
MaterialXView --library contrib/tests/metashade_envs/nan_debug material.mtlx

Related

• Multi-environment Metashade overrides with --library compatibility #13 -- Multi-environment infrastructure
• Safe math experiment and source code node reimplementation strategy #14 -- Safe math experiment and node reimplementation strategy