Refactor parsing of numeric ASCII lists-of-lists

[RamogninoF]

Summary

Extends the ASCII numeric list-of-lists parser to support sub-lists of arbitrary length, fixing slow parsing of faceList fields in polyMesh files that contain polygonal faces with 5 or more vertices.

Addresses the reviewer concern about missing inline count validation by introducing a hybrid validation strategy that keeps the performance of the fast path for large production meshes.

---

Background

OpenFOAM stores face connectivity in ASCII format as a list-of-lists where each entry is prefixed by its element count:

80867
(
4(0 1 20587 20586)
5(21224 21223 21284 21285 21286)
6(21270 21271 21272 21273 21274 21275)
...
)

The previous parser only recognised sub-lists of fixed length 3 or 4 (matching vector and tensor shapes), causing any face with 5+ vertices to fall through to the generic slow-path parser — making even small poly meshes extremely slow to parse.

---

Changes

Arbitrary sub-list length (original fix, ae73c6f)

Replaced the hardcoded shape check with a general loop that reads each inline count n and slices the next n values from the flattened array.

Inline count validation (this update)

The reviewer correctly noted that the flat-array approach silently accepts malformed input like 2(1 2 3) (count says 2, but 3 values are present). After stripping parentheses, the extra value bleeds into the stream and is misread as the count of the next sub-list, potentially returning wrong data without raising an error.

The fix uses a hybrid strategy based on whether an outer count is present in the file:

count is None — per-sublist path (fully validating)

When no outer element count is written, the parser iterates with _SUBLIST_CAPTURE.finditer and validates each n(...) block individually:

for m in _SUBLIST_CAPTURE.finditer(data):
    n = int(m.group(1))
    if n < 0:
        raise ParseError(...)
    inner = np.array(m.group(2).decode("ascii").split(), dtype=np_dtype)
    if len(inner) != n:
        raise ParseError(...)
    ret.append(inner)

Any mismatch between the declared count and the actual content raises immediately.

count is not None — fast flat-array path (production path)

When an outer count is present, the original flat-array approach is kept for performance. Two in-loop guards cover crash scenarios; the outer count check acts as the primary correctness net:

while i < len(values):
    n = int(values[i])
    if n < 0 or i + n + 1 > len(values):   # crash guard
        raise ParseError(...)
    ret.append(values[i + 1 : i + n + 1])
    i += n + 1
# ...
elif len(ret) != count:
    raise ParseError(...)   # net mismatch

Rationale for the split

OpenFOAM omits the outer count only for short lists (roughly ≤ 20 elements). For all large production face lists the outer count is always present, so:

• Performance where it matters (large meshes): the fast flat-array loop is used, with no per-sublist Python overhead.
• Full correctness where performance is free (short uncounted lists): each sub-list is validated independently.
• Known residual limitation on the fast path: an overfull sub-list whose extra values happen to form valid phantom sub-lists that sum to the correct outer count would not be detected. Given that vertex indices are large unsigned integers, the probability of this occurring in practice in a real mesh file is negligible.

---

Tests added

Eleven new tests in tests/test_files/test_parsing/test_poly_face_list.py, split by path:

Test	Path	Expected
Correct arbitrary-length sub-lists (3/4/5 vertices)	both	success
Zero-count sub-list 0()	no outer count	success
Overfull: 2(1 2 3)	no outer count	ParseError
Underfull: 4(1 2 3)	no outer count	ParseError
Overfull first, correct second: 2(1 2 3) 4(10 20 30 40)	no outer count	ParseError
Negative sub-list count: -1(0 1 2)	no outer count	ParseError
Float list-of-lists overfull sub-list	no outer count	ParseError
Outer count mismatch (declared 3, got 2 sub-lists)	outer count	ParseError
Underfull sub-list exceeds available data (crash guard)	outer count	ParseError
Negative sub-list count (crash guard)	outer count	ParseError

[RamogninoF]

@gerlero I am not an expert and I wrote this with help of AI based on #727 where you previously helped me on a similar topic. I have added tests which seem to properly work, along with previous functionalities.

Let me know if this works correctly and does not break anything!

Thanks!!!

[gerlero]

@RamogninoF thanks! I'll take a look.

In principle this shouldn't be possible with regular expressions alone (which foamlib's parser uses to be fast enough), but I'll look at the code to see what it's doing

[gerlero]

@RamogninoF problem to me is that this doesn't actually check that the sublist is well-formed. E.g. this will readily accept a list with a wrong count like 2 (1 2 3)...

[RamogninoF]

Unfortunately I have no background in parsing logics etc. and all this is far beyond my capabilities, I just hope this can be a useful starting point for you. At the current state parsing of meshes in ascii format is just straight impossible due to the time required for parsing (I gave up even on a 10k cells mesh after it was taking more then 10 minutes parsing the faces file). I would like to be able to support handling also ascii meshes rather then only binary

[RamogninoF]

A simple alternative could be just to add hardcoded parser for up to 10-vertex faces or so, which I think would be more then enough for most cases

[RamogninoF]

Or what if the string is parsed twice via regex, one to retrieve the list and one to get the prefix marking it's length, and these quantities are compared to validate the parsed data before returning?

[RamogninoF]

@gerlero I tried to update the logic considering your comments. Nevertheless, I tried to preserve a balance between safety and performances. I think that for practical production meshes the chances to have a silent digestion with a faulty file are virtually zero. Moreover, OpenFOAM should always write the number of elements in the list (of lists) with 20+ elements, so the double check should always take place.

What do you think?