Skip to content

The Python client: connect, run a statement, read the rows - #1

Merged
tamnd merged 1 commit into
mainfrom
python-core
Aug 17, 2026
Merged

tamnd merged 1 commit into
mainfrom
python-core

Conversation

@tamnd

@tamnd tamnd commented Aug 17, 2026

Copy link
Copy Markdown
Owner

The core of the Python SDK of dx/06, for DX2 (tamnd/zu#168). zudb.connect opens or creates a database, execute and sql run one statement with named parameters, and a result answers its columns, its rows, its length and its notices, iterates, and fetches one row at a time.

The extension links the engine crates directly rather than going through libzu, which is ADR 0002 in the engine repository. The README said the opposite and now says what the code does, with a short section on what is built and what is still to come, since the list of what this client is for is longer than what it does today.

Values cross both ways. Nulls, booleans, integers, floats, strings, lists and records go out as the Python objects they are and come back in as parameters; dates, times, datetimes with and without an offset, and durations map onto the datetime module, except for the one thing it cannot hold. A timedelta is days and microseconds, so it carries neither a count of months nor a count of nanoseconds, and a Duration class carries both and converts to a timedelta when it is asked to rather than quietly. Graph values are classes, Node, Rel and Path, with the table named rather than left as the id a row carries.

Every condition is an exception carrying the fields the error model promises. The classes live in zudb/errors.py, because a Python programmer catches by name and subclasses what they catch, and a class written in Python has a signature, a docstring and something a type checker can read. Rust owns the mapping instead: one class per GQLSTATUS class, so catching zudb.DataError catches every condition in class 22 without listing them, and e.code, e.line, e.column, e.offset, e.excerpt, e.doc_url and e.retryable are fields rather than a regular expression over the message. An io error reaching the file is a ConnectionError and not an InternalError, because a path that is not there is the caller's typo and not a bug to report.

The GIL goes down for the whole statement, waiting for the connection's own lock included. A thread that waited for the lock while holding the GIL would deadlock the thread inside the executor, which has to take the GIL back to return, and a statement that held the GIL would stop the signal handler running, which is what a Ctrl-C needs.

Eighty-four tests cover opening and closing, the context manager, the rows, the parameters, the values, the conditions and the threads. Three are about the GIL, one of them by counting the turns the main thread gets while another thread runs statements: with the GIL held for the length of a statement that count is a handful, and it is thousands. CI runs ruff, rustfmt and clippy, then installs the wheel with pip install . and runs the suite on 3.11 and 3.14 across Linux, macOS and Windows, so what the tests import is what a person who installs the package gets.

Local: ruff clean, cargo fmt --all --check clean, cargo clippy --all-targets -- -D warnings clean, 84 tests pass against a maturin build and again against the installed cp311-abi3 wheel. A cold interpreter plus import zudb is 33 ms on this machine, which is a note and not the gate that milestone line asks for.

Two things this turned up in the engine, neither of them fixed here. A table the graph has not got is made by an INSERT whose values are written out, so the first row of a new table cannot be parameters, and a rel table cannot be made this way at all, which is why nothing here tests a Rel or a Path against real data yet. And id is the name the v0 engine gives a node's offset, so a table carrying a property of its own by that name is one where RETURN p.id answers the property and WHERE p.id = 1 answers by offset; the fixtures call it uid and the disagreement is worth its own issue in the engine.

This is the core of the Python SDK of dx/06. `zudb.connect` opens or
creates a database, `execute` and `sql` run one statement with named
parameters, and a result answers its columns, its rows, its length and
its notices, iterates, and fetches one row at a time.

The extension links the engine crates directly rather than going
through `libzu`, which is ADR 0002 in the engine repository. The README
said the opposite and now says what the code does, along with what of
the client is built and what is still to come.

Values cross both ways. Nulls, booleans, integers, floats, strings,
lists and records go out as the Python objects they are and come back
in as parameters; dates, times, datetimes with and without an offset,
and durations map onto the `datetime` module, except for the one thing
it cannot hold. A `timedelta` is microseconds and days, so it can carry
neither a count of months nor a count of nanoseconds, and a `Duration`
class carries both and converts to a `timedelta` when asked rather than
silently. Graph values are classes: `Node`, `Rel` and `Path`, with the
table named rather than left as the id a row carries.

Every condition is an exception carrying the fields the error model
promises. The classes live in `zudb/errors.py`, because a Python
programmer catches by name and subclasses what they catch, and a class
written in Python has a signature, a docstring and something a type
checker can read. Rust owns the mapping instead: one class per
GQLSTATUS class, so catching `zudb.DataError` catches every condition
in class 22 without listing them, and `e.code`, `e.line`, `e.column`,
`e.offset`, `e.excerpt`, `e.doc_url` and `e.retryable` are fields and
never a regular expression over the message.

The GIL goes down for the whole statement, waiting for the connection's
own lock included. A thread that waited for the lock while holding the
GIL would deadlock the thread inside the executor, which has to take
the GIL back to return, and a statement that held the GIL would stop
the signal handler running, which is what a `Ctrl-C` needs.

Eighty-four tests cover the lot: opening and closing, the context
manager, the rows, the parameters, the values, the conditions and the
threads. Three of them are about the GIL, one by counting the turns
the main thread gets while another runs statements. CI runs ruff,
rustfmt and clippy, then installs the wheel and runs the suite on
3.11 and 3.14 across Linux, macOS and Windows.

Two notes for whoever writes the next test. A table the graph has not
got is made by an `INSERT` whose values are written out, so the first
row of a table cannot be parameters, and a rel table cannot be made
this way at all. And `id` is the name the v0 engine gives a node's
offset, so a table carrying a property of its own by that name is one
where `RETURN p.id` and `WHERE p.id = 1` disagree; the fixtures call it
`uid`.
@tamnd
tamnd merged commit 20dccf6 into main Aug 17, 2026
7 checks passed
@tamnd
tamnd deleted the python-core branch August 17, 2026 23:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant