Skip to content

perf: the OSM district import's build step is super-linear in road count #502

Description

@JArmandoAnaya

Found by the scale harness #244 built for it — which is the point of building one.

The measurement

The harness imports a synthetic lattice district. Scaling the district scales the road count, and the build step (the composite apply: authoring roads, then generating junctions) does not scale with it:

District Roads built Build time
7 blocks 560 ~92 ms
29 blocks (50.4 km²) ~12 000 minutes

The road count grows about 18× between those two. Linear cost would be ~1.7 s. It is roughly two orders of magnitude more, so something on that path is at least quadratic in the number of roads.

Note the parse-and-plan step is not implicated: the harness's TheCostDoesNotGrowQuadraticallyWithTheDistrict measures exactly that ratio for parse+plan and gets 4.14 where linear is 4 and quadratic would be 16. Whatever this is, it is in the apply, not the planner.

The prime suspect, named but not confirmed

CompositeCommand::dirty() (core/src/edit/operations.cpp) unions its children's DirtySets and de-duplicates with a linear std::ranges::find per entry. Each road child names exactly one road, so a composite of N road stages does on the order of N²/2 comparisons — about 72 million at 12 000 children, and that is before the DirtySet vectors' own allocation traffic.

Confirm before fixing. A profile is one run; guessing here would be the same mistake #442 made twice before it measured.

Other candidates worth ruling in or out in the same pass:

  • edit::create_junction generating a connecting road per (incoming lane, outgoing lane) pair — that is expected growth (it is why 1 624 planned segments become ~12 000 roads) but its per-junction cost may itself scale with the network.
  • Arena growth/reallocation as ~12 000 roads are created one command at a time.

Why it matters

#244's acceptance is that "an OSM district imports into an editable network meeting the perf targets". It does import, and the network is correct — the harness asserts junction counts and byte-identical undo. But a user importing a real district waits minutes, and the scale job's ceiling had to be set around a number nobody should be proud of.

Scope

  • Profile the composite apply at 29 blocks. Report where the time actually goes before changing anything.
  • If CompositeCommand::dirty() is confirmed: a topology = true short-circuit is the cheap fix (the editor rebuilds the mesh wholesale on topology anyway, so the per-road list is not read in that case), or a hash set instead of the linear scan.
  • Re-measure and move the dev target in core/tests/scale/scale_bench.cpp in the same diff, per the runner-variance policy that file states.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    pillar:P7Road to Parity pillar P7

    Type

    No type

    Projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions