The shared walk can be asked for an output format, and a page it cannot read is refused - #85
Merged
Merged
Conversation
…ot read is refused iter_features gained return_geometry and out_sr in #75 and left f hard-coded to json. That is the argument the consuming project still reads three of its four layers as f=geojson through its own copy of the offset loop, so a second copy of the paging rule and of this module's refusals stays alive in its tree. out_format defaults to json, so the default request is byte-for-byte the one behind the hashes pinned in sources.py. The test that holds that now names f beside returnGeometry and the absent outSR. The walk is not format-agnostic and now says so. It reads features and exceededTransferLimit out of the top level of every answer; a format carrying neither would yield nothing on its first page and stop, which is what an empty layer looks like. PAGEABLE_OUT_FORMATS names the ones that carry both and the refusal happens before a socket opens. For the same reason payload.get("features", []) is gone. A missing key and an empty list were one value, and the walk stopped quietly on either; the record count copied out of the resulting file is published on both pages. An empty list still ends the walk, held by its own test so the refusal cannot widen. fetch_layer takes no out_format: it reads feature["attributes"], which a GeoJSON Feature does not carry.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What was wrong
iter_featuresgainedreturn_geometryandout_srin #75, which closed most of the consuming project's acquisition gap. It kept"f": "json"hard-coded, and that was the argument the duplication stayed alive:wildfire-service-territory-overlapreads four layers and three of them asf=geojson, which this walk could not be asked for. So its ownfetch_feature_pagesstays, and with it a second copy of the offset rule this package exists to stop being copied, and a second copy of this module's refusals about somebody else's server.Two smaller things were found while measuring it:
featuresandexceededTransferLimitout of the top level of every answer. A JSON-shaped format carrying neither would yield nothing on its first page and stop — with no exception, a clean hash, and a record count of zero. That is indistinguishable from a layer holding no records.payload.get("features", [])made the same two facts one value. "The service answered something this walk cannot read" and "the layer holds no records" both produced an empty list, and the walk ended quietly on either. The record count copied out of the resulting file is published on both pages and pinned insources.py.What changed
iter_features(..., out_format: str = DEFAULT_OUT_FORMAT), threaded into the query dict besideout_sr. A caller asking forgeojsonreceives the service's ownFeatureobjects, whole: nothing merged, nothing renamed, no geometry converted.PAGEABLE_OUT_FORMATSandUnpageableFormatError. A format the walk cannot page is refused before a socket opens, with a message naming what the walk reads and why an unpageable format would look like an empty layer.featureskey raisesAcquisitionFailednaming the offset and the keys that were there. An emptyfeatureslist still ends the walk cleanly, and that side of the boundary has its own test so the refusal cannot widen into one that refuses honest emptiness.fetch_layerdeliberately takes noout_format, and its signature is pinned by a test saying why: it readsfeature["attributes"], which a GeoJSONFeaturedoes not carry.The default request has not moved.
test_the_default_request_is_the_one_this_project_has_always_madenow namesf=jsonalongsidereturnGeometry=falseand the absentoutSR.How it was verified
make verifygreen in full, determinism gate included.Three negative controls, each committed-baseline / sabotage /
git hash-objectre-assert,__pycache__cleared between runs, and each fired on exactly the tests predicted and no others:payload.get("features", [])— the stateorigin/mainis in"f": out_formatback to"f": "json"— the parameter accepted and ignoredThe second is the one worth naming: a parameter that is accepted and silently dropped looks exactly like one that works, and only the two tests that read
foff the sent URL can tell them apart.The paging rule is exercised under
f=geojsonrather than assumed to be shared — a capped layer of 5,000 records served 1,000 at a time, walked to completion. The reason to expose the walk at all is that the offset rule stops being copied; a non-default format reaching the same records by an untested path would leave the consumer with a second implementation of the subtle part after all.The planted unpageable format is
f-that-no-geoservices-layer-will-ever-publish, not the next plausible format name, and the test asserts it is absent fromPAGEABLE_OUT_FORMATS— a planted value drawn from the namespace the code grows into has a shelf life nobody writes down.Not in this change
No GeoJSON parsing, validation or conversion here, per the issue's scope. The caller asks for a format and gets what the service sent.
Closes #84
Prepared with AI assistance; reviewed before submission.