Skip to content

bench: neither allocation ceiling measures allocations #1

Description

@stealth-rklopper

max_allocs and max_bytes both read tracemalloc.get_traced_memory(), which answers
(current: int, peak: int) — two byte figures. bench.py unpacks that pair as
blocks, size and stores it as _peak_blocks, _peak_bytes. Neither field holds what its
name says, and neither ceiling compares the quantity it claims to.

  • max_allocs divides the current bytes at the end of the loop by the iteration count
    and compares the result to an allocation count. Wrong quantity and wrong unit.
  • max_bytes divides the peak live bytes by the iteration count. A peak is a
    high-water mark rather than a sum, so the quotient is not bytes per iteration, and it
    falls as the iteration count rises. Running more iterations loosens the ceiling.

Both figures are levels of live memory, so whatever a body allocates and frees inside one
iteration leaves no trace in either.

What the ceilings report

CPython 3.14.7, 100 iterations, both ceilings stated so both numbers get computed:

Body Truth per iteration max_allocs reports max_bytes reports
allocates nothing 0 allocations, 0 bytes 31.92 31.92
10 × bytearray(4096), freed each iteration 10 allocations, ~40960 bytes 9.28 425.86
1 × bytearray(4096), kept 1 allocation, 4096 bytes 4170.52 4170.52

The third row is the plainest: one allocation reports as 4170 allocations, because the
number is bytes. The first row allocates nothing and still reports 31.92 per iteration,
because the contract's own _each list grows inside the traced region.

The test cannot fail

test_an_exceeded_allocation_ceiling_reports drives bytearray(4096) against
max_allocs(0) and asserts the run fails. It does fail — and so does a body that allocates
nothing, for the reason in the first row. The test would pass unchanged against a
max_allocs that measured nothing whatever. No case asserts that a body inside its
allocation ceiling passes, which is what would have caught this.

What a fix has to work with

CPython's standard library has no cumulative allocation counter. Every primitive answers a
level:

Primitive Answers
tracemalloc.get_traced_memory() current and peak traced bytes
sys.getallocatedblocks() blocks currently allocated
tracemalloc.take_snapshot().statistics() count and size of blocks still live

Go reaches these ceilings through runtime.MemStats.Mallocs and TotalAlloc, which are
cumulative and never fall. Sampling a level around each iteration and summing the positive
deltas is the closest Python gets, and it counts what an iteration retains — a real
number, and not the one either ceiling names.

Three ways out. Choosing one is the decision this issue wants:

  1. Measure retention and rename both ceilings to say that is what they hold.
  2. Keep both names, measure retention, and declare the divergence in the Python overlay.
  3. Drop both ceilings in Python and declare them absent in the overlay.

The standard states no accuracy for these two and the corpus does not reach them, so
nothing outside this repository contradicts any of the three today. The Python overlay
currently declares nothing about either, so the library claims both and delivers neither.

excluding covers time alone for this same reason, which its docstring records.

References

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions