The Python client: connect, run a statement, read the rows - #1
Merged
Merged
Conversation
This is the core of the Python SDK of dx/06. `zudb.connect` opens or creates a database, `execute` and `sql` run one statement with named parameters, and a result answers its columns, its rows, its length and its notices, iterates, and fetches one row at a time. The extension links the engine crates directly rather than going through `libzu`, which is ADR 0002 in the engine repository. The README said the opposite and now says what the code does, along with what of the client is built and what is still to come. Values cross both ways. Nulls, booleans, integers, floats, strings, lists and records go out as the Python objects they are and come back in as parameters; dates, times, datetimes with and without an offset, and durations map onto the `datetime` module, except for the one thing it cannot hold. A `timedelta` is microseconds and days, so it can carry neither a count of months nor a count of nanoseconds, and a `Duration` class carries both and converts to a `timedelta` when asked rather than silently. Graph values are classes: `Node`, `Rel` and `Path`, with the table named rather than left as the id a row carries. Every condition is an exception carrying the fields the error model promises. The classes live in `zudb/errors.py`, because a Python programmer catches by name and subclasses what they catch, and a class written in Python has a signature, a docstring and something a type checker can read. Rust owns the mapping instead: one class per GQLSTATUS class, so catching `zudb.DataError` catches every condition in class 22 without listing them, and `e.code`, `e.line`, `e.column`, `e.offset`, `e.excerpt`, `e.doc_url` and `e.retryable` are fields and never a regular expression over the message. The GIL goes down for the whole statement, waiting for the connection's own lock included. A thread that waited for the lock while holding the GIL would deadlock the thread inside the executor, which has to take the GIL back to return, and a statement that held the GIL would stop the signal handler running, which is what a `Ctrl-C` needs. Eighty-four tests cover the lot: opening and closing, the context manager, the rows, the parameters, the values, the conditions and the threads. Three of them are about the GIL, one by counting the turns the main thread gets while another runs statements. CI runs ruff, rustfmt and clippy, then installs the wheel and runs the suite on 3.11 and 3.14 across Linux, macOS and Windows. Two notes for whoever writes the next test. A table the graph has not got is made by an `INSERT` whose values are written out, so the first row of a table cannot be parameters, and a rel table cannot be made this way at all. And `id` is the name the v0 engine gives a node's offset, so a table carrying a property of its own by that name is one where `RETURN p.id` and `WHERE p.id = 1` disagree; the fixtures call it `uid`.
20 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The core of the Python SDK of dx/06, for DX2 (tamnd/zu#168).
zudb.connectopens or creates a database,executeandsqlrun one statement with named parameters, and a result answers its columns, its rows, its length and its notices, iterates, and fetches one row at a time.The extension links the engine crates directly rather than going through
libzu, which is ADR 0002 in the engine repository. The README said the opposite and now says what the code does, with a short section on what is built and what is still to come, since the list of what this client is for is longer than what it does today.Values cross both ways. Nulls, booleans, integers, floats, strings, lists and records go out as the Python objects they are and come back in as parameters; dates, times, datetimes with and without an offset, and durations map onto the
datetimemodule, except for the one thing it cannot hold. Atimedeltais days and microseconds, so it carries neither a count of months nor a count of nanoseconds, and aDurationclass carries both and converts to atimedeltawhen it is asked to rather than quietly. Graph values are classes,Node,RelandPath, with the table named rather than left as the id a row carries.Every condition is an exception carrying the fields the error model promises. The classes live in
zudb/errors.py, because a Python programmer catches by name and subclasses what they catch, and a class written in Python has a signature, a docstring and something a type checker can read. Rust owns the mapping instead: one class per GQLSTATUS class, so catchingzudb.DataErrorcatches every condition in class 22 without listing them, ande.code,e.line,e.column,e.offset,e.excerpt,e.doc_urlande.retryableare fields rather than a regular expression over the message. An io error reaching the file is aConnectionErrorand not anInternalError, because a path that is not there is the caller's typo and not a bug to report.The GIL goes down for the whole statement, waiting for the connection's own lock included. A thread that waited for the lock while holding the GIL would deadlock the thread inside the executor, which has to take the GIL back to return, and a statement that held the GIL would stop the signal handler running, which is what a
Ctrl-Cneeds.Eighty-four tests cover opening and closing, the context manager, the rows, the parameters, the values, the conditions and the threads. Three are about the GIL, one of them by counting the turns the main thread gets while another thread runs statements: with the GIL held for the length of a statement that count is a handful, and it is thousands. CI runs ruff, rustfmt and clippy, then installs the wheel with
pip install .and runs the suite on 3.11 and 3.14 across Linux, macOS and Windows, so what the tests import is what a person who installs the package gets.Local: ruff clean,
cargo fmt --all --checkclean,cargo clippy --all-targets -- -D warningsclean, 84 tests pass against a maturin build and again against the installedcp311-abi3wheel. A cold interpreter plusimport zudbis 33 ms on this machine, which is a note and not the gate that milestone line asks for.Two things this turned up in the engine, neither of them fixed here. A table the graph has not got is made by an
INSERTwhose values are written out, so the first row of a new table cannot be parameters, and a rel table cannot be made this way at all, which is why nothing here tests aRelor aPathagainst real data yet. Andidis the name the v0 engine gives a node's offset, so a table carrying a property of its own by that name is one whereRETURN p.idanswers the property andWHERE p.id = 1answers by offset; the fixtures call ituidand the disagreement is worth its own issue in the engine.