Skip to content

Burn down the invalid-argument-type diagnostics - #830

Merged
d-chambers merged 8 commits into
devfrom
ty-tests
Aug 7, 2026
Merged

Burn down the invalid-argument-type diagnostics#830
d-chambers merged 8 commits into
devfrom
ty-tests

Conversation

@d-chambers

@d-chambers d-chambers commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Description

Continues the ty burn-down. invalid-argument-type goes from 34 to 18 in dascore/; the rules already gating CI stay at zero throughout.

The point of this rule is that it is the one that checks callers against what a function claims to accept. Turning it on is also what makes bringing tests/ into ty's scope worthwhile — with the rule ignored, adding tests/ reports zero of these, so the test suite cannot act as an oracle for public API annotations until this reaches zero.

Most of what follows is not annotation tidying. In each case ty was flagging a caller because the annotation was false, and the interesting part was which side was wrong.

Two latent bugs

  • FrozenDict.new could not round-trip non-string keys. It rebuilt the mapping with self.__class__(**contents), so any key that is not a valid identifier raised TypeError: keywords must be strings. FrozenDict({1: "a"}).new() reproduces it. Now passed positionally. Test added.
  • drop_coords advertised Collection[str], but no collection form worked. A list or set raised TypeError on the set intersection, and a tuple silently dropped nothing:
    pa.drop_coords(("latitude",))  # returned a patch that still had latitude
    The annotation was the honest one here, so the implementation was fixed to match it rather than the other way round. Both Patch.drop_coords and CoordManager.drop_coords now flatten with the existing iterate utility, so a name, a sequence of names, or a mix of both works at either level. The bug was that iterate was applied to the varargs tuple as a whole, which is already iterable — so a sequence passed in survived as a single unhashable element. It is now applied per argument. Tests cover list/tuple/set at both levels, mixed args, and a dimension named inside a sequence still raising.

The coord-map input type was wrong three ways

CoordManagerInput and Patch.new's private copy of it disagreed with each other and with the runtime:

  • a tuple key was allowed, but get_coord_manager rejects any key that is not a dimension name, so no caller could ever use one;
  • only ndarray was allowed as data, but a list or range works — and both the dispersion and taup transforms pass one;
  • it used dict rather than Mapping, so a caller holding a narrower value type did not match even when every value was valid.

Patch.new now shares the alias instead of maintaining its own. A bare tuple stays reserved for the (dimension, data) form, which is why Sequence[Any] rather than something looser.

Annotations narrower than their callers

  • is_netcdf4_file / get_cf_version declared h5py.File but receive the managed handle a FiberIO caster produces — and their existing tests already pass plain duck types. Both read only attrs, so that is what they ask for now.
  • get_patch_names forwards straight to scan_to_df, so it accepts everything that scans, including the list of patches the DASDAE writer hands it.

Narrowings made visible

  • patch_function handles the bare-decorator form by re-entering itself. Doing that check before the wrapper is built rather than after leaves the rest of the function seeing required_dims as the tuple it actually is. All three decorator forms verified unchanged.
  • The assembly loop's first pass sets axis and dims alongside buffer; all three are now asserted rather than just the one.
  • _KEEP was a bare object(), so _view's order and ids were inferred as object and could not be passed on. A dedicated class carries the same meaning and narrows.
  • dascore.units.__getattr__ was untyped, so from dascore.units import m inferred Unknown and silently absorbed every downstream check on it. Now Quantity.

What was deliberately left

Three clusters were investigated and rejected, rather than forced:

  • to_int / to_float (2 diagnostics). These are singledispatch generics declared -> np.ndarray, which is false. Widening to the true union pushes the count up to 34, because callers genuinely rely on the input-dependent return type. @overload is the right tool, but applying it makes ty lose .register entirely (27 new errors) — the two decorators do not compose. Left alone.
  • numpy/pint stub gaps (3 diagnostics). memoryview(ndarray), blake2b.update(ndarray), and 1 / coord.units are all correct at runtime; the stubs are imprecise.
  • FrozenDict.__getitem__ (1 diagnostic). **kwargs forces str keys, so _dict really is dict[K | str, V], and SupportsKeysAndGetItem is invariant in its key. Not expressible without a cast that buys nothing.

Changelog

  • changed: Patch.drop_coords and CoordManager.drop_coords accept a sequence of names as well as bare names; a list or set previously raised TypeError and a tuple or generator was silently ignored, and a name that is a dimension raises ParameterError.

Checklist

I have (if applicable):

  • referenced the GitHub issue this PR closes. (no issue; part of the ongoing ty burn-down)
  • documented the new feature with docstrings and/or appropriate doc page.
  • included tests. See testing guidelines.
  • added the "ready_for_review" tag once the PR is ready to be reviewed.

Summary by CodeRabbit

  • Bug Fixes

    • Preserved non-string keys when creating updated immutable mappings.
    • Improved metadata normalization for scanned data sources.
    • Strengthened validation during streaming patch assembly.
  • Improvements

    • Expanded supported coordinate input formats and coordinate removal options.
    • Improved compatibility with NetCDF-compatible file-like objects.
    • Enabled more flexible decorator usage and clearer unit lookup behavior.
  • Tests

    • Added regression coverage for mappings and coordinate removal with sequence inputs.

from dascore.units import m gave Unknown, which silently absorbed every
check downstream. A name reaching __getattr__ is always a non-empty
identifier, so it resolves to a quantity or raises.
The coord map type was wrong in three separate ways, all of which ty
flagged as callers failing to match it:

- A tuple key was allowed, but get_coord_manager rejects any key that is
  not a dimension name, so no caller could ever use one.
- Only ndarray was allowed as data, but a list or range works and both
  the dispersion and taup transforms pass one.
- dict was used rather than Mapping, so a caller holding a narrower
  value type did not match even when every value was valid.

Patch.new declared its own version of this type; it now shares the alias.
A bare tuple stays reserved for the (dimension, data) form.

Patch.drop_coords advertised Collection[str], which never worked: a list
or set raises TypeError on the set intersection and a tuple silently
drops nothing. It takes plain strings, as every caller already passes.

_scan_payload_to_summary defaulted source_format and source_version to
None, which PatchSummary's before-validator maps to "". Default them to
the string the model stores so the annotation matches the field.
FrozenDict.new rebuilt the mapping with `self.__class__(**contents)`,
which raises `TypeError: keywords must be strings` for any key that is
not a valid identifier. It now passes the mapping positionally.

pydantic.validate_call declares a ConfigDict, so build one rather than a
plain dict.
is_netcdf4_file and get_cf_version declared h5py.File but receive the
managed handle a FiberIO caster produces, and their tests already pass
plain duck types. Both read only `attrs`, so that is what they now ask
for.

get_patch_names forwards its input straight to scan_to_df, so it accepts
everything that scans -- including the list of patches the DASDAE writer
hands it.
patch_function handles the bare-decorator form by re-entering itself, so
required_dims is only ever a callable on that path. Doing the check
before the wrapper is built rather than after leaves the rest of the
function seeing the tuple of dimension names it actually gets.

The assembly loop's first pass sets axis and dims alongside buffer;
assert all three rather than just the one.
_KEEP was a bare object(), so _view's order and ids parameters were
inferred as object and neither could be passed on to the constructor.
A dedicated class carries the same meaning and lets an isinstance check
narrow the parameter back to the spec type.
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, you can upgrade your account or add credits to your account and enable them for code reviews in your settings.

@d-chambers d-chambers added the ready_for_review PR is ready for review label Aug 6, 2026
@coderabbitai

coderabbitai Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Warning

Review limit reached

@d-chambers, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 24 minutes

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: ea1b4c07-c951-4216-bc3a-d4b68923a8b1

📥 Commits

Reviewing files that changed from the base of the PR and between 923105f and 54548af.

📒 Files selected for processing (9)
  • dascore/core/coordmanager.py
  • dascore/proc/coords.py
  • dascore/units.py
  • dascore/utils/patch.py
  • docs/changelog.qmd
  • tests/test_core/test_coordmanager.py
  • tests/test_proc/test_proc_coords.py
  • tests/test_units.py
  • tests/test_utils/test_patch_utils.py
📝 Walkthrough

Walkthrough

Changes

The change broadens coordinate input contracts, updates structural type annotations, improves patch decorator handling, normalizes I/O defaults, and adds utility correctness checks.

API contract refinements

Layer / File(s) Summary
Coordinate input contracts
dascore/core/coordmanager.py, dascore/proc/basic.py, dascore/proc/coords.py, tests/test_core/test_coordmanager.py, tests/test_proc/test_proc_coords.py
CoordInput supports broader sequence and tuple forms. update uses CoordManagerInput. drop_coords accepts coordinate-name collections and validates flattened arguments.
Boundary type contracts
dascore/io/netcdf/utils.py, dascore/units.py
NetCDF helpers use structural attribute typing. Unit attribute lookup declares typed inputs and outputs.
I/O state and summary defaults
dascore/io/core.py, dascore/io/index/catalog.py
Payload summaries default source metadata to empty strings. Catalog views use a typed _Keep sentinel for inherited state.
Patch decorator and input contracts
dascore/utils/patch.py
patch_function supports bare decorator usage and uses ConfigDict. get_patch_names accepts ScanInput.
Utility correctness checks
dascore/utils/mapping.py, dascore/utils/patch_assembly.py, tests/test_utils/test_mapping_utils.py
FrozenDict.new preserves non-string keys. Streaming patch assembly checks buffer, axis, and dimension initialization. A regression test covers integer keys.

Possibly related PRs

Suggested labels: bug, proc, patch

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed Docstring coverage is 81.82% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly identifies the main change: reducing invalid-argument-type diagnostics through type and caller corrections.
Description check ✅ Passed The description explains the purpose, key fixes, deliberate exclusions, documentation, tests, and checklist status in sufficient detail.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch ty-tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot added bug Something isn't working CI continuous integration patch related to Patch class proc Related to processing module labels Aug 6, 2026
@codecov

codecov Bot commented Aug 6, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00%. Comparing base (3f4c83b) to head (54548af).

Additional details and impacted files
@@            Coverage Diff            @@
##               dev      #830   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files          164       164           
  Lines        17916     17924    +8     
=========================================
+ Hits         17916     17924    +8     
Flag Coverage Δ
network 48.49% <66.66%> (+<0.01%) ⬆️
unittests 100.00% <100.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Both Patch.drop_coords and CoordManager.drop_coords advertised
Collection[str] but no collection form worked: a list or set raised
TypeError on the set intersection and a tuple silently dropped nothing.

iterate was applied to the varargs tuple as a whole, which is already
iterable, so anything passed as a sequence survived as a single
unhashable element. Applying it per argument flattens as intended and
makes the documented signature true at both levels.
@coderabbitai coderabbitai Bot removed the CI continuous integration label Aug 6, 2026
CoordManagerInput claimed more than it delivered. Its Sequence[Any] arm
subsumed every tuple, so the "bare tuple is reserved for the (dim, data)
form" comment was false and the nested tuple arm rejected nothing, while
the union still turned away int and Quantity values (partial coords) that
first-party callers pass. It now matches the Patch constructor's coords.

get_patch_names referenced io.core.ScanInput behind TYPE_CHECKING. The
import cycle is real, but the annotation then failed to resolve at
runtime, and the docs renderer falls back all-or-nothing to raw strings
for a signature when any name is undefined. Spelled out instead.

drop_coords accepts any iterable, not just a sized collection, since
iterate branches on Iterable; a generator worked but did not type check.

The drop_coords tests only asserted the named coords were gone, so an
implementation that dropped every coord passed them. They now compare
against the bare-name call.
@d-chambers
d-chambers merged commit f48876b into dev Aug 7, 2026
27 checks passed
@d-chambers
d-chambers deleted the ty-tests branch August 7, 2026 11:03
@d-chambers d-chambers removed the ready_for_review PR is ready for review label Aug 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working patch related to Patch class proc Related to processing module

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant