a result as columns, and the frames built on them

[tamnd]

A result is rows to iterate and columns to hand to something else. The rows were there already. This is the columns: __arrow_c_stream__ on Result, and to_arrow, to_pandas, to_polars and record_batches written on top of it.

The interface is the PyCapsule one, so the capsule is the whole of it and the four methods are conveniences. pyarrow.table(result) and polars.DataFrame(result) read a result with nothing here doing anything, which is the point of a protocol every reader implements. to_pandas asks for Arrow-backed dtypes, which is what pandas 3 wants anyway and what keeps a string column from becoming a column of Python strings on the way in. A requested schema is accepted and ignored, which the protocol allows and which is honest: the result is what it is.

The types

QueryResult carries no column types, only values, so the type of a column is inferred from what is in it. One type to a column, because that is what Arrow holds.

Value	Arrow
integer	int64
float	float64
string	string
boolean	bool
null	null
list	list<item>
record	struct of its fields
date	date32
local time	time64[ns]
local datetime	timestamp[ns]
zoned datetime	timestamp[ns, tz=+05:30]
day-time duration	duration[ns]
year-month duration	month_day_nano_interval
node	struct<table, offset>
rel	struct<table, src, dst, ord>
path	struct<nodes: list, rels: list>

Integers beside floats widen to floats, which is the only mixture that is not refused: anything else says which two types it mixed and at which row. Nulls do not decide anything and a column of nothing but nulls is Arrow's null type, which is also what an empty result's columns are, so an empty result still has its column names.

Two of these are choices rather than mappings. A year-month duration goes as a month-day-nano interval with the days and nanoseconds zero, because pyarrow has no array class for the year-month one and raises KeyError: 21 on the type id, and neither pandas nor polars accepts any interval type at all. A time with an offset is refused, because Arrow has no type for one.

The numbers

Batches are 65,536 rows and the copy runs with the GIL released. On this machine:

	Arrow	Python objects
300,000 rows, three columns	44 ms	67 ms
300,000 rows, one integer column	13.8 ms	44.5 ms

That is where the difference actually lives: a Python object per cell is the cost, and not the query. Table names are borrowed rather than cloned per row, which took the node column from 27 ms to 17.5 ms over the same 300,000 rows. The GIL test hands the copy to a thread and spins the main one, and saw 535,456 turns in 83 ms.

Tests

33 of them, skipped as a file when pyarrow is not installed, and the pandas and polars ones skip on their own. Every type above, nulls in every container, the widening and the refusals, the capsule and the protocol both, two batches over 70,000 rows, Arrow and objects agreeing on the same statement, and the GIL. The wheel still depends on nothing: arrow is asked for with default-features = false, features = ["ffi"], which is the C Data Interface and none of the readers and writers.

Closes nothing on its own. Ticks the to_arrow and to_pandas line of tamnd/zu#168.