Skip to content

PERF: cascade Series.dt.components for ArrowDtype durations - #66635

Closed
aboderinsamuel wants to merge 1 commit into
pandas-dev:mainfrom
aboderinsamuel:perf-63283-dt-components-cascade
Closed

aboderinsamuel wants to merge 1 commit into
pandas-dev:mainfrom
aboderinsamuel:perf-63283-dt-components-cascade

Conversation

@aboderinsamuel

@aboderinsamuel aboderinsamuel commented Aug 6, 2026 •

Copy link
Copy Markdown
  • closes #xxxx (Replace xxxx with the GitHub issue number)
  • Tests added and passed if fixing a bug or adding a new feature
  • All code checks passed.
  • Added type annotations to new arguments/methods/functions.
  • Added an entry in the latest doc/source/whatsnew/vX.X.X.rst file.
  • I have reviewed and followed all the contribution guidelines.
  • If I used AI to develop this pull request, I prompted it to follow AGENTS.md.

Speeds up Series.dt.components for ArrowDtype durations with two changes:

  1. Sub-day fields are peeled off a running remainder from the coarsest unit down, so each is a single division with no modulo (PyArrow has no modulo kernel, so each % was emulated with divide/multiply/subtract).
  2. The day floor-division is cached once and shared by _dt_days and _dt_day_remainder, instead of being recomputed on every .dt.components call.

No behavior change, covered by the existing test_arrow duration-component tests.

Benchmark of TimedeltaComponents.time_components (duration[ns][pyarrow], best of 60, before = main / after = branch): Series.dt.components 34.5 ms -> 18.0 ms (~1.9x) at N=100k all-positive, and 320 ms -> 173 ms (~1.85x) at N=1M mixed-sign. Single-field accessors unchanged.

xref #63283, #63470

Comment thread doc/source/whatsnew/v3.1.0.rst Outdated
Comment thread pandas/core/arrays/arrow/array.py Outdated
Comment thread pandas/core/arrays/arrow/array.py Outdated
Comment thread pandas/core/arrays/arrow/array.py
Comment thread doc/source/whatsnew/v3.1.0.rst Outdated
@aboderinsamuel
aboderinsamuel force-pushed the perf-63283-dt-components-cascade branch from e975485 to 9da7eba Compare August 13, 2026 10:14
@jorisvandenbossche jorisvandenbossche added Performance Memory or execution speed performance Arrow pyarrow functionality Awaiting Review PR in the maintainers' court; exempt from staleness labels Aug 18, 2026
@github-actions github-actions Bot removed the Awaiting Review PR in the maintainers' court; exempt from staleness label Aug 19, 2026
@github-actions github-actions Bot added the Awaiting Review PR in the maintainers' court; exempt from staleness label Aug 20, 2026
@jbrockmendel

Copy link
Copy Markdown
Member

Can you check perf on ser.dt.days? I suspect this is non-trivially slower

@github-actions github-actions Bot removed the Awaiting Review PR in the maintainers' court; exempt from staleness label Aug 25, 2026
@jbrockmendel

Copy link
Copy Markdown
Member

Closing as stale. Can reopen when you're ready to address comments and continue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Arrow pyarrow functionality Performance Memory or execution speed performance

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants