Repository navigation
GH-37900: [Python][FlightRPC] Add cooperative payload admission example - #51701
shatrughnaupadhyay wants to merge 1 commit into
Conversation
|
Thanks for opening a pull request! This pull request has been automatically closed because you currently have 4 open pull requests, which is more than the limit of 3. Due to the increase in pull requests opened by AI bots, and in order to keep the review queue manageable, Apache Arrow limits contributors without repository access to at most 3 concurrently open pull requests. This helps make sure each pull request gets the attention it needs and that work in progress does not go stale. Once one of your other open pull requests has been merged or closed, you are welcome to reopen this one. See also: |
|
|
github-actions
Bot
added
Component: Python
Component: Documentation
awaiting review
|
|
2 participants
This Python DoExchange example announces each batch's logical size, waits for a shared-budget grant or refusal, then releases the reservation after synchronous consumption. It demonstrates the cooperative application workaround discussed in #37900, using one non-null int64 column and client deadlines.
The budget accounts for logical payload bytes. It does not control allocation inside gRPC or cap process RSS, and it assumes cooperating peers. Native buffers, read-ahead, schema/control messages, retained callback data and server-side lease enforcement remain outside its scope. It does not resolve #37900.
Validation: 30 tests passed with Python 3.12.12 and the PyArrow 25.0.1 wheel, covering localhost concurrency, refusal and reuse, cancellation, deadlines, malformed input and exception cleanup. Flake8, autopep8 and whitespace checks passed. A contention benchmark is included. Source-built Arrow and upstream CI have not been verified.
Draft: design agreement and human author review are pending. The author must review, understand and own the changes before marking this ready.