Skip to content

The shared walk can be asked for an output format, and a page it cannot read is refused - #85

Merged
ChelseaKR merged 1 commit into
mainfrom
feat/84-out-format-for-the-shared-walk
Sep 8, 2026
Merged

ChelseaKR merged 1 commit into
mainfrom
feat/84-out-format-for-the-shared-walk

Conversation

@ChelseaKR

Copy link
Copy Markdown
Owner

What was wrong

iter_features gained return_geometry and out_sr in #75, which closed most of the consuming project's acquisition gap. It kept "f": "json" hard-coded, and that was the argument the duplication stayed alive: wildfire-service-territory-overlap reads four layers and three of them as f=geojson, which this walk could not be asked for. So its own fetch_feature_pages stays, and with it a second copy of the offset rule this package exists to stop being copied, and a second copy of this module's refusals about somebody else's server.

Two smaller things were found while measuring it:

  • The walk is not format-agnostic, though it reads as though it is. It reads features and exceededTransferLimit out of the top level of every answer. A JSON-shaped format carrying neither would yield nothing on its first page and stop — with no exception, a clean hash, and a record count of zero. That is indistinguishable from a layer holding no records.
  • payload.get("features", []) made the same two facts one value. "The service answered something this walk cannot read" and "the layer holds no records" both produced an empty list, and the walk ended quietly on either. The record count copied out of the resulting file is published on both pages and pinned in sources.py.

What changed

  • iter_features(..., out_format: str = DEFAULT_OUT_FORMAT), threaded into the query dict beside out_sr. A caller asking for geojson receives the service's own Feature objects, whole: nothing merged, nothing renamed, no geometry converted.
  • PAGEABLE_OUT_FORMATS and UnpageableFormatError. A format the walk cannot page is refused before a socket opens, with a message naming what the walk reads and why an unpageable format would look like an empty layer.
  • A page with no features key raises AcquisitionFailed naming the offset and the keys that were there. An empty features list still ends the walk cleanly, and that side of the boundary has its own test so the refusal cannot widen into one that refuses honest emptiness.
  • fetch_layer deliberately takes no out_format, and its signature is pinned by a test saying why: it reads feature["attributes"], which a GeoJSON Feature does not carry.

The default request has not moved. test_the_default_request_is_the_one_this_project_has_always_made now names f=json alongside returnGeometry=false and the absent outSR.

How it was verified

make verify green in full, determinism gate included.

Three negative controls, each committed-baseline / sabotage / git hash-object re-assert, __pycache__ cleared between runs, and each fired on exactly the tests predicted and no others:

sabotage predicted measured
restore payload.get("features", []) — the state origin/main is in the missing-key refusal only 1 red, that test
"f": out_format back to "f": "json" — the parameter accepted and ignored both geojson tests, default-request test green 2 red, exactly those
the format refusal never runs the unpageable-format test only 1 red, that test

The second is the one worth naming: a parameter that is accepted and silently dropped looks exactly like one that works, and only the two tests that read f off the sent URL can tell them apart.

The paging rule is exercised under f=geojson rather than assumed to be shared — a capped layer of 5,000 records served 1,000 at a time, walked to completion. The reason to expose the walk at all is that the offset rule stops being copied; a non-default format reaching the same records by an untested path would leave the consumer with a second implementation of the subtle part after all.

The planted unpageable format is f-that-no-geoservices-layer-will-ever-publish, not the next plausible format name, and the test asserts it is absent from PAGEABLE_OUT_FORMATS — a planted value drawn from the namespace the code grows into has a shelf life nobody writes down.

Not in this change

No GeoJSON parsing, validation or conversion here, per the issue's scope. The caller asks for a format and gets what the service sent.

Closes #84

Prepared with AI assistance; reviewed before submission.

…ot read is refused

iter_features gained return_geometry and out_sr in #75 and left f hard-coded to
json. That is the argument the consuming project still reads three of its four
layers as f=geojson through its own copy of the offset loop, so a second copy of
the paging rule and of this module's refusals stays alive in its tree.

out_format defaults to json, so the default request is byte-for-byte the one
behind the hashes pinned in sources.py. The test that holds that now names f
beside returnGeometry and the absent outSR.

The walk is not format-agnostic and now says so. It reads features and
exceededTransferLimit out of the top level of every answer; a format carrying
neither would yield nothing on its first page and stop, which is what an empty
layer looks like. PAGEABLE_OUT_FORMATS names the ones that carry both and the
refusal happens before a socket opens.

For the same reason payload.get("features", []) is gone. A missing key and an
empty list were one value, and the walk stopped quietly on either; the record
count copied out of the resulting file is published on both pages. An empty list
still ends the walk, held by its own test so the refusal cannot widen.

fetch_layer takes no out_format: it reads feature["attributes"], which a GeoJSON
Feature does not carry.
@ChelseaKR
ChelseaKR merged commit aba6255 into main Sep 8, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

iter_features hard-codes f=json, so a caller needing GeoJSON still copies the offset loop

1 participant