Skip to content

feat: add cursor-based pagination to list commands#328

Merged
drappier-charles merged 4 commits into
mainfrom
cdrappier/devin/add-pagination-to-list-commands
Jun 16, 2026
Merged

feat: add cursor-based pagination to list commands#328
drappier-charles merged 4 commits into
mainfrom
cdrappier/devin/add-pagination-to-list-commands

Conversation

@devin-ai-integration

@devin-ai-integration devin-ai-integration Bot commented Jun 16, 2026

Copy link
Copy Markdown
Contributor

Summary

List commands (bl get sandboxes, bl get volumes, etc.) now use the controlplane's cursor-based pagination API instead of fetching a bare array. The CLI sends Blaxel-Version: 2026-04-28 to opt into the {data, meta} response shape.

New flags on every paginated resource subcommand (sandboxes, volumes, agents, functions, models, jobs, policies, drives):

--limit N      max items per page (default/max 200, matching API cap)
--cursor STR   resume from a previous page's next cursor
--all          auto-paginate through every page with a progress indicator

Default behavior (no flags): fetches one page of up to 200 items. When more results exist, prints the next cursor to stderr:

Showing 200 of 1423 sandboxes. To see the next page run:
  bl get sandboxes --cursor eyJTS...

--all loops through all pages, showing Fetching sandboxes... 400/1423 on stderr (TTY-only, cleared on completion).

Key design decisions:

  • Resource struct gains APIPath string + Paginated bool fields — set for the 8 resources with backend pagination support
  • cli/core/pagination.go: fetchPageListPaginated (single page) / ListAllPaginated (all pages with progress)
  • ListExec (used by watch mode, cache seeding) still fetches one page transparently; the new ListFnPaginated is the entry point for interactive list with cursor output
  • Resources without pagination support (IntegrationConnection, VolumeTemplate, Image) keep the existing unpaginated SDK path via reflection
  • bl drive list also gets the same --limit/--cursor/--all flags

Link to Devin session: https://app.devin.ai/sessions/0e2a50e0e8824d2a865484fee6b9dbf2
Requested by: @drappier-charles


Note

New commit adds logic so --all --limit N caps fetched items at N instead of fetching everything then discarding. Uses ListWithLimit when limit differs from the default page size.

Written by Mendral for commit 21ace91.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR that start with 'DevinAI' or '@devin'.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
mendral-app[bot]

This comment was marked as outdated.

@mendral-app

mendral-app Bot commented Jun 16, 2026

Copy link
Copy Markdown
Contributor

🔀 Interaction Flow Diagram

Here's a sequence diagram showing how the new cursor-based pagination flows through the CLI components:

sequenceDiagram
    participant User
    participant CLI as CLI Command<br>(get.go / drive.go)
    participant Router as ListFnPaginated<br>(get.go)
    participant Pagination as Pagination Module<br>(core/pagination.go)
    participant API as Controlplane API

    User->>CLI: bl get sandboxes --limit/--cursor/--all
    CLI->>CLI: Parse flags (limit, cursor, all)
    CLI->>Router: ListFnPaginated(resource, limit, cursor, fetchAll)

    alt --all flag (no limit)
        Router->>Pagination: ListAllPaginated(resource)
    else --all + --limit OR limit > 200
        Router->>Pagination: ListWithLimit(resource, maxItems)
    else --cursor flag
        Router->>Pagination: ListPaginated(resource, limit, cursor)
    else default (single page)
        Router->>Pagination: ListPaginated(resource, 200, "")
    end

    loop while hasMore && under limit
        Pagination->>Pagination: fetchPage(client, apiPath, pageSize, cursor)
        Pagination->>API: GET /resource?limit=N&cursor=X<br>Header: Blaxel-Version: 2026-04-28
        API-->>Pagination: {data: [...], meta: {nextCursor, hasMore, total}}
        Pagination->>Pagination: Append items, update cursor
        Note over Pagination: Show progress on TTY<br>"Fetching... 150/500"
    end

    Pagination-->>Router: PaginatedResult{Items, Meta}
    Router-->>CLI: items []any
    CLI->>User: Output formatted results

    opt hasMore && not --all
        CLI->>User: stderr: "Showing X of Y. Next page:<br>bl get resource --cursor <next>"
    end
Loading

Summary of the flow

This PR introduces a pagination strategy router (ListFnPaginated) that sits between CLI commands and a new core pagination module. The key interactions are:

Flag combination Strategy Behavior
Default (no flags) Single page Fetches up to 200 items
--cursor STR Resume Fetches one page starting from cursor
--limit N (> 200) Auto-paginate to limit Loops pages until N items collected
--all Fetch everything Loops all pages with progress indicator

All paginated requests include Blaxel-Version: 2026-04-28 to opt into the {data, meta} envelope response. Eight resource types (sandboxes, volumes, agents, functions, models, jobs, policies, drives) are wired up via a Paginated flag on the Resource struct in config.go.

Note

Posted by PR Sequence Diagram · Tag @mendral-app with feedback.

@mendral-app

mendral-app Bot commented Jun 16, 2026

Copy link
Copy Markdown
Contributor

🧪 Testing Guide

What this PR addresses

Adds cursor-based pagination to all list commands (bl get sandboxes, bl get volumes, bl get agents, etc.) and bl drive list. Previously these commands fetched a bare array from the API with no pagination support. Now the CLI sends Blaxel-Version: 2026-04-28 to opt into a {data, meta} response shape and exposes --limit, --cursor, and --all flags.

Steps to exercise the new behavior

  1. Default behavior (single page):

    bl get sandboxes
    

    Verify it returns up to 200 items. If more exist, stderr should print a message like:

    Showing 200 of N sandboxes. To see the next page run:
      bl get sandboxes --cursor eyJTS...
    
  2. Custom limit:

    bl get sandboxes --limit 10
    

    Verify only 10 items are returned and the next-cursor hint appears if more exist.

  3. Cursor-based resume:
    Copy the cursor from step 2 and run:

    bl get sandboxes --cursor <paste_cursor>
    

    Verify the next page of results is returned (different items from page 1).

  4. Auto-paginate all results:

    bl get sandboxes --all
    

    Verify all items are fetched with a progress indicator on stderr (TTY only), and the final output contains all resources.

  5. --all --limit N caps total items:

    bl get sandboxes --all --limit 50
    

    Verify at most 50 items are returned (not all pages).

  6. Other paginated resources: Repeat a quick smoke test for at least one other resource type:

    bl get agents --limit 5
    bl get volumes --all
    bl drive list --limit 10
    
  7. Non-paginated resources still work:

    bl get integrations
    

    Verify resources without pagination support (IntegrationConnection, VolumeTemplate, Image) still return results normally via the existing SDK path.

What to verify (expected behavior)

  • All 8 paginated resources (sandboxes, volumes, agents, functions, models, jobs, policies, drives) accept --limit, --cursor, and --all flags without error.
  • Default (no flags) returns one page of up to 200 items with a cursor hint when more exist.
  • --all fetches every page and displays a progress indicator in TTY mode.
  • --all --limit N caps total fetched items at N.
  • Non-paginated resources are unaffected and continue to work as before.
  • bl drive list behaves the same as other paginated resources with the new flags.
  • No regressions in watch mode or cache seeding (ListExec still fetches one page transparently).
  • Generated docs match the new flags.

Note

Posted by PR Testing Guide · Tag @mendral-app with feedback.

@mendral-app

mendral-app Bot commented Jun 16, 2026

Copy link
Copy Markdown
Contributor

📋 Created Linear issue ENG-3151 — status: In Progress

  • Assignee: Charles Drappier (reviewer — bot PR)
  • Labels: CLI, enhancement
  • Estimate: L (4 files, ~363 lines)
  • PR linked: ✅ Issue will auto-close when this PR merges

⚠️ Could not prepend Fixes ENG-3151 to the PR description automatically (GitHub API restriction on bot-authored PRs). Please add it manually to ensure the issue auto-transitions to Done on merge.

Auto-created because no Linear reference was found in the PR title, description, or branch name.

Note

Posted by Linear Issue Enforcer · Tag @mendral-app with feedback.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
mendral-app[bot]

This comment was marked as outdated.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

@mendral-app mendral-app Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

The new commit cleanly implements the --all --limit N cap by checking limit != core.DefaultPageLimit to detect an explicitly-set flag, then delegating to ListWithLimit. Previous URL-encoding comment was already addressed. No new issues.

Tag @mendral-app with feedback or questions. View session

@drappier-charles drappier-charles marked this pull request as ready for review June 16, 2026 09:09
@drappier-charles drappier-charles merged commit 3b71652 into main Jun 16, 2026
8 of 10 checks passed
@drappier-charles drappier-charles deleted the cdrappier/devin/add-pagination-to-list-commands branch June 16, 2026 09:09

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 3 potential issues.

Open in Devin Review

Comment thread cli/get.go

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚩 Watch mode still uses unpaginated ListFn, not ListFnPaginated

executeAndDisplayWatch at cli/get.go:814-815 calls ListFn(&resource) rather than ListFnPaginated. This means watch mode ignores the --limit, --cursor, and --all flags, always fetching a single page of 200 via ListExecListExecPaginated. This is likely intentional (watch mode shouldn't auto-paginate thousands of items every 2 seconds), but it's an inconsistency with the non-watch code path that could confuse users who combine --watch --all.

(Refers to line 815)

Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

Comment thread cli/core/pagination.go

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚩 No unit tests added for new pagination code paths

This PR adds ~180 lines of new code in pagination.go and ~100 lines in get.go (ListFnPaginated, ListExecPaginated) with no corresponding test files. The AGENTS.md guideline states: "Keep tests next to the code as *_test.go. Add a unit test for new code paths." Key paths that would benefit from testing: fetchPage response parsing, ListWithLimit multi-page accumulation, ListFnPaginated branch logic (--all, --cursor, --limit interactions), and edge cases like empty pages or maxItems=0.

Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

Comment thread cli/get.go

// --all: fetch everything, but respect --limit if explicitly set.
if fetchAll {
if limit != core.DefaultPageLimit {

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 --all --limit 200 silently fetches all pages instead of capping at 200

ListFnPaginated at cli/get.go:626 uses limit != core.DefaultPageLimit to detect whether --limit was explicitly provided. Since the flag default is core.DefaultPageLimit (200), the check can't distinguish "user passed --limit 200" from "user didn't pass --limit". When a user runs e.g. bl get agents --all --limit 200, the code falls through to ListAllPaginated and fetches every single page, potentially returning thousands of items instead of capping at 200. This is especially problematic in scripts like bl get agents --all --limit $LIMIT where LIMIT may resolve to 200.

The fix is to use Cobra's cmd.Flags().Changed("limit") to determine whether --limit was explicitly set, and pass that boolean into ListFnPaginated. This requires plumbing a limitExplicit bool from the Run closure.

Prompt for agents
In ListFnPaginated (cli/get.go:618), the check `limit != core.DefaultPageLimit` is used as a proxy for "was --limit explicitly passed?" This fails when the user explicitly passes `--limit 200` (the same value as the default). The fix is to pass a boolean indicating whether --limit was explicitly set, derived from `cmd.Flags().Changed("limit")` in the Run closures of both cli/get.go (GetCmd, around line 255) and cli/drive.go (DriveListCmd, around line 407). Then ListFnPaginated's signature should become `func ListFnPaginated(resource *core.Resource, limit int, limitExplicit bool, cursor string, fetchAll bool)` and the conditional should check `if limitExplicit` instead of `if limit != core.DefaultPageLimit`.
Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant