ACE reader fails on denormal floats under numpy 2, existing missing-E recovery never runs

[shimwell]

Summary

_read_ascii in src/endf/ace.py cannot read any ACE file containing a denormal value, because the existing recovery for NJOY's missing-E floats never runs under numpy 2.

The code already knows about this class of malformed float. The guard at ace.py:414 handles it:

xss = np.fromstring(datastr, sep=' ')          # line 406

# When NJOY writes an ACE file, any values less than 1e-100 actually
# get written without the 'e'. ...
if xss.size != nxs[1] + 1:                      # line 414
    datastr = ENDF_FLOAT_RE.sub(r'\1e\2\3', datastr)
    xss = np.fromstring(datastr, sep=' ')

That check keys off a short array, which is what numpy 1 produced: np.fromstring(..., sep=' ') stopped at the first token it could not parse and returned what it had. numpy 2 raises instead, so line 406 throws and line 414 is never reached.

File "endf/ace.py", line 406, in _read_ascii
    xss = np.fromstring(datastr, sep=' ')
ValueError: string or file could not be read to its end due to unmatched data

Reproduction

Any TENDL-2025 evaluation whose ACE output contains a denormal. Five do:

import endf
endf.IncidentNeutron.from_njoy("n-Db262.tendl", njoy_exec="njoy")

Also fails for n-Db263, n-Db264, n-Sg272 and n-Sg273. NJOY 2016.79, numpy 2.5.1, Python 3.13.

The source ENDF is fine, endf.Material() reads all five without complaint. The malformed floats are in the ACE file ACER writes. In Db262 exactly 2 of the 772031 XSS tokens are affected:

idx 350955: '6.10562372605-318'
idx 350983: '5.34601261754-318'

Denormals near 1e-318, where the three-digit negative exponent overflows the Fortran field width and the E is dropped. ENDF_FLOAT_RE already handles these correctly once it gets the chance:

>>> ENDF_FLOAT_RE.sub(r'\1e\2\3', '6.10562372605-318')
'6.10562372605e-318'

Fix

Give the existing recovery the short array it expects rather than letting the read fail:

try:
    xss = np.fromstring(datastr, sep=' ')
except ValueError:
    xss = np.empty(0)

With that, all five convert (Db262 gives 60 reactions, Sg272 gives 62).

Related

ace.py:156, in ascii_to_binary, has the same bare np.fromstring(..., sep=' ') and no size check to fall back on, so it will fail the same way on these files. Not on the from_njoy path, so it did not surface here.

Worth sending upstream too: nothing about this is specific to this fork or to TENDL, it affects any ACE file with a denormal on numpy 2.

[shimwell]

Cross-checked the fix against OpenMC, since a change to float parsing is worth confirming against another reader before trusting it.

Generated a real ACE file with NJOY 2016.79 from the TENDL-2025 Db263 evaluation. It contains exactly two missing-E tokens:

5.14540714336-318
6.93171630787-318

Fed the same file to both readers:

Reader	Result
endf-python, this fix	OK, 710864 XSS values, denormals read as 6.931716e-318 and 5.145407e-318
OpenMC 0.15.3	ValueError: string or file could not be read to its end due to unmatched data

The two packages do not disagree about what the token means. openmc/data/ace.py has the same structure at lines 403 to 413, the same "any values less than 1e-100 actually get written without the 'e'" comment, and no try/except, so its recovery is unreachable under numpy 2 for the same reason. It also imports ENDF_FLOAT_RE from endf.records, so both packages are literally applying the same substitution.

This fix therefore restores the shared intent rather than diverging from it. The parsed values are denormals near 1e-318, well below the smallest normal double at about 2.2e-308, so they are numerically indistinguishable from zero in any case. What matters is that the read completes rather than what the two tokens hold.

OpenMC has the same latent failure and would benefit from the same change.

[shimwell]

Cross-checked against other readers: the token is valid, and the fix here is sound

Compared against endf-parserpy 0.17.0 and OpenMC (openmc-dev/openmc @ 9a62e43).

The token is well-formed ENDF and other readers take it without special-casing. endf-parserpy's fortstr2float inserts the exponent marker generically — any sign preceded by a digit becomes E, whatever the exponent width:

for i, c in enumerate(valstr):
    if i > 0 and (c == "+" or c == "-"):
        if valstr[i - 1].isdigit():
            valstr = valstr[:i] + "E" + valstr[i:]

Run against the tokens from the report:

' 6.10562372605-318' ->  6.105624e-318
'-6.10562372605-318' -> -6.105624e-318
' 1.23456700000-120' ->  1.234567e-120
' 1.234567+8'        ->  123456700.0

So there is no ambiguity to resolve: NJOY is writing a legal three-digit-exponent field and a reader is expected to cope. Not a data problem.

The difference is strategy, and that is what numpy 2 broke. endf-parserpy parses every field individually, so exponent width never arises. This package takes the fast path — hand the whole block to np.fromstring and only fall back to the regex when the array comes out short:

# ... we don't do this by default because it's expensive!
if xss.size != nxs[1] + 1:
    datastr = ENDF_FLOAT_RE.sub(r'\1e\2\3', datastr)

That is a reasonable optimisation, and the regex fallback was correct all along. What broke is only the trigger: numpy 1 stopped short and returned a short array, numpy 2 raises instead, so the short-array check never got the chance to fire.

The fix on local-develop restores the trigger rather than changing the parse:

except ValueError:
    # numpy 2 raises on a token it cannot parse where numpy 1 stopped
    # short and returned what it had. The recovery below keys off a
    # short array, so give it one rather than letting the read fail.
    xss = np.empty(0)

which keeps the fast path for the overwhelming majority of files and hands the denormal case to the code that already knew how to handle it. Confirmed correct.

_{OpenMC imports float_endf and py_float_endf from this package and has no separate ACE float path, so it inherits both the bug and the fix. The Rust port has no fast path — it parses each field and puts the e back unconditionally — and tests/synthetic-denormal.ace.xz now pins both readers to the same nine values.}

---

Generated by Claude Code

[shimwell]

This one bit real data: TENDL-2025's Db262 and its neighbours are unreadable because of it.

The cause is that NJOY drops the e when an exponent needs three digits, so a token reads 6.10562372605-318 rather than 6.10562372605e-318. numpy 2 rejects it, and the existing missing-E recovery does not run because the failure happens earlier.

A Rust port parses these directly: parse_float("6.10562372605-318") gives 6.10562372605e-318, and parse_float("1.234567-120") gives 1.234567e-120. Source crates/endf/src/ace.rs. There is also a synthetic whole-file fixture, synthetic-denormal.ace.xz, built to pin both readers on the exact tokens rather than on the parser in isolation, in case that is a useful shape for a regression test here.