feat: add cursor-based pagination to list commands#328
Conversation
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
🔀 Interaction Flow DiagramHere's a sequence diagram showing how the new cursor-based pagination flows through the CLI components: sequenceDiagram
participant User
participant CLI as CLI Command<br>(get.go / drive.go)
participant Router as ListFnPaginated<br>(get.go)
participant Pagination as Pagination Module<br>(core/pagination.go)
participant API as Controlplane API
User->>CLI: bl get sandboxes --limit/--cursor/--all
CLI->>CLI: Parse flags (limit, cursor, all)
CLI->>Router: ListFnPaginated(resource, limit, cursor, fetchAll)
alt --all flag (no limit)
Router->>Pagination: ListAllPaginated(resource)
else --all + --limit OR limit > 200
Router->>Pagination: ListWithLimit(resource, maxItems)
else --cursor flag
Router->>Pagination: ListPaginated(resource, limit, cursor)
else default (single page)
Router->>Pagination: ListPaginated(resource, 200, "")
end
loop while hasMore && under limit
Pagination->>Pagination: fetchPage(client, apiPath, pageSize, cursor)
Pagination->>API: GET /resource?limit=N&cursor=X<br>Header: Blaxel-Version: 2026-04-28
API-->>Pagination: {data: [...], meta: {nextCursor, hasMore, total}}
Pagination->>Pagination: Append items, update cursor
Note over Pagination: Show progress on TTY<br>"Fetching... 150/500"
end
Pagination-->>Router: PaginatedResult{Items, Meta}
Router-->>CLI: items []any
CLI->>User: Output formatted results
opt hasMore && not --all
CLI->>User: stderr: "Showing X of Y. Next page:<br>bl get resource --cursor <next>"
end
Summary of the flowThis PR introduces a pagination strategy router (
All paginated requests include Note Posted by PR Sequence Diagram · Tag @mendral-app with feedback. |
🧪 Testing GuideWhat this PR addressesAdds cursor-based pagination to all list commands ( Steps to exercise the new behavior
What to verify (expected behavior)
Note Posted by PR Testing Guide · Tag @mendral-app with feedback. |
|
📋 Created Linear issue ENG-3151 — status: In Progress
Auto-created because no Linear reference was found in the PR title, description, or branch name. Note Posted by Linear Issue Enforcer · Tag @mendral-app with feedback. |
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
There was a problem hiding this comment.
LGTM
The new commit cleanly implements the --all --limit N cap by checking limit != core.DefaultPageLimit to detect an explicitly-set flag, then delegating to ListWithLimit. Previous URL-encoding comment was already addressed. No new issues.
Tag @mendral-app with feedback or questions. View session
There was a problem hiding this comment.
🚩 Watch mode still uses unpaginated ListFn, not ListFnPaginated
executeAndDisplayWatch at cli/get.go:814-815 calls ListFn(&resource) rather than ListFnPaginated. This means watch mode ignores the --limit, --cursor, and --all flags, always fetching a single page of 200 via ListExec → ListExecPaginated. This is likely intentional (watch mode shouldn't auto-paginate thousands of items every 2 seconds), but it's an inconsistency with the non-watch code path that could confuse users who combine --watch --all.
(Refers to line 815)
Was this helpful? React with 👍 or 👎 to provide feedback.
There was a problem hiding this comment.
🚩 No unit tests added for new pagination code paths
This PR adds ~180 lines of new code in pagination.go and ~100 lines in get.go (ListFnPaginated, ListExecPaginated) with no corresponding test files. The AGENTS.md guideline states: "Keep tests next to the code as *_test.go. Add a unit test for new code paths." Key paths that would benefit from testing: fetchPage response parsing, ListWithLimit multi-page accumulation, ListFnPaginated branch logic (--all, --cursor, --limit interactions), and edge cases like empty pages or maxItems=0.
Was this helpful? React with 👍 or 👎 to provide feedback.
|
|
||
| // --all: fetch everything, but respect --limit if explicitly set. | ||
| if fetchAll { | ||
| if limit != core.DefaultPageLimit { |
There was a problem hiding this comment.
🟡 --all --limit 200 silently fetches all pages instead of capping at 200
ListFnPaginated at cli/get.go:626 uses limit != core.DefaultPageLimit to detect whether --limit was explicitly provided. Since the flag default is core.DefaultPageLimit (200), the check can't distinguish "user passed --limit 200" from "user didn't pass --limit". When a user runs e.g. bl get agents --all --limit 200, the code falls through to ListAllPaginated and fetches every single page, potentially returning thousands of items instead of capping at 200. This is especially problematic in scripts like bl get agents --all --limit $LIMIT where LIMIT may resolve to 200.
The fix is to use Cobra's cmd.Flags().Changed("limit") to determine whether --limit was explicitly set, and pass that boolean into ListFnPaginated. This requires plumbing a limitExplicit bool from the Run closure.
Prompt for agents
In ListFnPaginated (cli/get.go:618), the check `limit != core.DefaultPageLimit` is used as a proxy for "was --limit explicitly passed?" This fails when the user explicitly passes `--limit 200` (the same value as the default). The fix is to pass a boolean indicating whether --limit was explicitly set, derived from `cmd.Flags().Changed("limit")` in the Run closures of both cli/get.go (GetCmd, around line 255) and cli/drive.go (DriveListCmd, around line 407). Then ListFnPaginated's signature should become `func ListFnPaginated(resource *core.Resource, limit int, limitExplicit bool, cursor string, fetchAll bool)` and the conditional should check `if limitExplicit` instead of `if limit != core.DefaultPageLimit`.
Was this helpful? React with 👍 or 👎 to provide feedback.
Summary
List commands (
bl get sandboxes,bl get volumes, etc.) now use the controlplane's cursor-based pagination API instead of fetching a bare array. The CLI sendsBlaxel-Version: 2026-04-28to opt into the{data, meta}response shape.New flags on every paginated resource subcommand (sandboxes, volumes, agents, functions, models, jobs, policies, drives):
Default behavior (no flags): fetches one page of up to 200 items. When more results exist, prints the next cursor to stderr:
--allloops through all pages, showingFetching sandboxes... 400/1423on stderr (TTY-only, cleared on completion).Key design decisions:
Resourcestruct gainsAPIPath string+Paginated boolfields — set for the 8 resources with backend pagination supportcli/core/pagination.go:fetchPage→ListPaginated(single page) /ListAllPaginated(all pages with progress)ListExec(used by watch mode, cache seeding) still fetches one page transparently; the newListFnPaginatedis the entry point for interactive list with cursor outputbl drive listalso gets the same--limit/--cursor/--allflagsLink to Devin session: https://app.devin.ai/sessions/0e2a50e0e8824d2a865484fee6b9dbf2
Requested by: @drappier-charles
Note
New commit adds logic so
--all --limit Ncaps fetched items at N instead of fetching everything then discarding. UsesListWithLimitwhen limit differs from the default page size.Written by Mendral for commit 21ace91.