Repository navigation
[Python] Conversion to/from numpy 2.0+ new StringDType #42018
Description
With the upcoming numpy 2.0 release, they introduce a new variable-length string dtype, StringDType (https://numpy.org/devdocs/release/2.0.0-notes.html#highlights).
Currently trying to convert that obviously gives an error:
>>> arr = np.array(["some", "strings"], dtype=np.dtypes.StringDType())
>>> arr
array(['some', 'strings'], dtype=StringDType())
>>> pa.array(arr)
...
ArrowNotImplementedError: Unsupported numpy type 2056But ideally we should support this dtype both for conversion from numpy -> arrow, as in the arrow -> numpy conversion (although here it should probably be opt-in and by default still return object dtype)
Numpy provides a C API to access the individual string elements: https://numpy.org/devdocs/reference/c-api/strings.html
- [Python] Conversion from Arrow string to numpy StringDtype
Activity
github-actions commentedon Nov 18, 2025
This issue has been marked as stale because it has had no activity in the past 365 days. Please remove the stale label or comment below, or this issue will be closed in 14 days. If this improvement is still desired but has no current owner, please add the 'Status: needs champion' label.
alippai commentedon Dec 2, 2025
I believe this is not stale, numpy is continuously improving the StringDType support.
@jorisvandenbossche would this be implemented on the C++ side or cython/python?
thisisnic commentedon Feb 18, 2026
This issue has been closed because there has been no activity for the 14 days since the stale warning. Feel free to reopen if this has been closed in error.
alippai commentedon Feb 18, 2026
@thisisnic there was clear activity (both comment and pending MR). Is the script broken?
thisisnic commentedon Feb 18, 2026
Doing it manually due to GitHub API limits on the CI job, and it's missed the logic for where the label has been left but someone has commented. Will reopen!
alippai commentedon Feb 18, 2026
Thanks for keeping the issue queue tidy
rok commentedon Jul 18, 2026
cc @ngoldbaum
16 remaining items
apacheGH-42018: [Python] Simplify the StringDType conversion loop
apacheGH-42018: [Python] Convert StringDType nulls with a string na_o…
apacheGH-42018: [Python] Name the requested type when rejecting a Str…
apacheGH-42018: [Python] Test list inference for StringDType arrays
apacheGH-42018: [Python][Docs] Document NumPy StringDType conversion
apacheGH-42018: [Python] Clean up error message and comments
apacheGH-42018: [Python] Clean up error message and comments
apacheGH-42018: [Python] Clean up error message and comments
lorentzenchr commentedon Sep 28, 2026
What about the conversion to numpy with
to_numpy?ngoldbaum commentedon Sep 28, 2026
That will be a bigger code change and also might require disentangling the NumPy and Pandas exporters.
lorentzenchr commentedon Sep 28, 2026
Is it agreed upon?
ngoldbaum commentedon Sep 28, 2026
I'm not working on it. 🤷♂️
jorisvandenbossche commentedon Sep 30, 2026
I think it is certainly something we want to support in theory, but it will require someone to dive into it
GH-51652: [Python] Add NumPy StringDType to Arrow conversion (#51157)