stubs for the compiled module, and a gate that keeps them true - #4
Merged
Merged
Conversation
An extension module is a shared object and nothing can read a signature out of one, so mypy, pyright and an editor's completion all had nothing to go on. This is `_zudb.pyi`: every function, class, property and method, with the parameter names and defaults the module actually has. It ships in the wheel beside `py.typed`, which was already there. `zudb.Value` comes with it, the union a row holds and a parameter takes, for code that passes rows around and wants to say so. Written as a real module rather than only in the stub, because a type alias a checker knows and the interpreter does not is a name that fails at the first `from zudb import Value`. A stub is a promise no interpreter checks. Nothing fails when it names a method the engine does not have or gives a parameter a name the engine does not answer to; what fails is the caller who believed the editor. So it is checked: griffe reads the stub as text, inspects the installed extension, and the two are compared on every public name, every kind, every parameter and every default. PyO3 writes a text signature for everything it exports, which is what makes the second half possible. Two things the comparison has to work around. griffe merges a stub into the extension beside it, so a merged stub would agree with the module by construction and a missing name would come back from the `.so`; the package is copied to a temporary directory without its binary first. And a constructor is exported as `__new__` with `(*args, **kwargs)`, with the real signature on the class, where `inspect` finds it and where the stub's `__init__` is checked against it. What is not compared is the types. No inspection of a compiled module can confirm that a column is a `list[str]`, and the tests that call it are what say so. 25 tests for the stub, 176 in the client. Verified by breaking it both ways: a method the module does not have and a parameter renamed, each caught by the test that should catch it.
20 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
An extension module is a shared object and nothing can read a signature out of one, so mypy, pyright and an editor's completion all had nothing to go on. This is
python/zudb/_zudb.pyi: every function, class, property and method, with the parameter names and defaults the module actually has. It ships in the wheel besidepy.typed, which was already there.zudb.Valuecomes with it, the union a row holds and a parameter takes, for code that passes rows around and wants to say so. It is written as a real module and not only in the stub, because a type alias a checker knows and the interpreter does not is a name that fails at the firstfrom zudb import Value.The gate
A stub is a promise no interpreter checks. Nothing fails when it names a method the engine does not have, or gives a parameter a name the engine does not answer to. What fails is the caller who believed the editor.
So
tests/test_stubs.pychecks it. griffe reads the stub as text and inspects the installed extension, and the two are compared on:PyO3 writes a text signature for everything it exports, which is what makes the inspection half work at all:
connectcomes back as(path, *, read_only=False, memory_limit=None, threads=None)out of the built module.Two things the comparison has to work around, both commented where they happen. griffe merges a stub into the extension beside it, so a merged stub would agree with the module by construction and a name missing from the stub would come back from the
.so; the package is copied to a temporary directory without its binary first. And a constructor is exported as__new__with(*args, **kwargs), with the real signature on the class, which is whereinspectfinds it and where the stub's__init__is checked against it.What is not compared is the types. No inspection of a compiled module can confirm that a column is a
list[str], and the tests that call it are what say so.Checked
Verified by breaking it both ways, a method the module does not have and a parameter renamed, each caught by the test that should catch it. Verified from the other end with mypy, which resolves the whole surface through the stub and reveals a row as
tuple[None | bool | int | float | str | date | time | datetime | timedelta | Duration | Node | Rel | Path | list[...] | dict[str, ...], ...]. The wheel was built and listed:zudb/_zudb.pyi,zudb/py.typedandzudb/types.pyare all in it.25 tests for the stub, 176 in the client. CI installs griffe beside pytest so the check runs against the installed wheel rather than against the checkout it was built from.
Ticks the
.pyistubs line of tamnd/zu#168.