Skip to content

feat(cli): add metrics resources command for CPU, memory and GPU memory - #93

Open
jonoirwinrsa wants to merge 5 commits into
mainfrom
feat/cli-metrics
Open

feat(cli): add metrics resources command for CPU, memory and GPU memory#93
jonoirwinrsa wants to merge 5 commits into
mainfrom
feat/cli-metrics

Conversation

@jonoirwinrsa

@jonoirwinrsa jonoirwinrsa commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

Stacked on #92 — base is feat/cli-json-output. Review that one first; merging it will retarget this to main.

Why

An agent that has just run cerebrium deploy can now see whether its app is up (#92), but not whether the hardware in cerebrium.toml is the right size.

cerebrium metrics resources APP reports the three numbers that answer it — CPU, memory, and GPU memory (VRAM) — from the resource-metrics endpoint.

$ cerebrium metrics resources brainiac-h3-v4 --start 2026-08-06T17:30:00Z --end 2026-08-06T19:00:00Z
Peak usage for p-367e7969-brainiac-h3-v4 (app) from 2026-08-06T17:30:00Z to 2026-08-06T19:00:00Z

METRIC               Max       P50       P90  UNIT
CPU                 1.04      1.04      1.04  cores
Memory            185.17    185.17    185.17  GB
GPU Memory             -         -         -  GB

Defaults to the last hour. --since, --start/--end, --container-id and --resolution narrow it; --output json returns the summary plus the raw series so an agent can do its own analysis.

Deliberately narrow: the backend also exposes latency percentiles, startup time, cost, queue depth and per-container CPU/memory. Those can follow. (The /metrics/* percentile endpoints are also behind a per-project feature flag, which is its own conversation.)

Two things worth reviewing

The endpoint's data arrays are not numbers. It relays raw query values, and those arrive as JSON strings, with null for padded gaps. The web client's (number | null)[] type is wrong about this — JS coerces on the way into the chart, so it never surfaced. Go doesn't, so MetricValue decodes from string, number or null and marshals back out as a plain number. NaN/±Inf decode as absent rather than blowing up json.Marshal.

Absence vs zero. A series that never reported prints -, not 0.00. A GPU row of 0.00 would claim we measured zero VRAM in use, which is a different statement from "no samples". A genuine measured 0 is still shown as 0.00; there's a test pinning that.

Also lifts normalizeAppID — which had been copy-pasted into three files — into api.NormalizeAppID, with its test moved alongside. Adding a fourth copy for this command seemed like the wrong direction.

Testing

go build, go vet, go test ./... all clean. No golden files regenerated. New unit tests cover MetricValue decoding (string/number/null/empty/NaN/Inf/garbage), number-form marshalling, window resolution, and the summary/column logic.

Verified against prod on a real H100 app over a window when it was actually serving:

  • App scope returns Max/P50/P90 per metric. All three are equal here because the app ran a single replica — a quantile over one series is that series.
  • Container scope was wrong in my first pass and is fixed here: the API names each metric's sole series after the metric (CPU, Memory, GPU), so matching cells to columns by name made Memory and GPU render - despite having data. Cells are now matched by position and the header collapses to PEAK. There's a regression test.
  • GPU reads - for this app. The series are present but every sample is null, i.e. no GPU samples came back for that window — the web client sends identical params, so this is upstream, not a gap in the request. I could not verify a non-null GPU number, so if you have an app that reports VRAM it's worth a sanity check before merge.
  • Bad --resolution, --start and --output values all produce clean validation errors; -o json piped with stdin closed is parseable (no spinner bytes).

jonoirwinrsa and others added 2 commits August 10, 2026 15:59
Agents driving `cerebrium deploy` have to scrape fixed-width text tables to
learn anything about a deployment. Only `status` supported `--output json`.

Add a shared output-format helper in internal/ui and wire it through apps
list/get, containers list, runs list, projects list, secrets list and files ls.
JSON emits the api structs as-is, including the fields the tables drop.
`status` is refactored onto the helper rather than keeping its own copy.

`secrets list --output json` omits values unless --show-values is passed, so it
leaks no more than the table does.

The spinner is skipped for JSON output so no frames land in the payload stream.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The output plumbing shipped with no tests, verified only by hand against a live
account. That does not survive the next refactor, and one of the behaviours is
security relevant.

`secrets list --output json` omits values unless --show-values is passed. Nothing
pinned that, so collapsing the payload back to the raw map would start printing
every secret in the project while the table still hid them. Extracts the shaping
into newJSONSecrets and covers it: values hidden by default, present when asked,
an empty value kept distinguishable from a hidden one, key order preserved, and
no secrets encoding as [] rather than null.

Also covers ParseOutputFormat (defaults, both flag forms, unsupported values,
and that the error names the accepted formats), PrintJSON (indentation, trailing
newline, empty slice as [], nothing written when encoding fails, stable across
runs), and the nil-receiver spinner guard the JSON paths depend on.

Both behaviours were mutation checked: forcing values to always be included and
removing the nil guard each fail the corresponding test.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
An agent that has just deployed an app has no way to tell whether the hardware
in cerebrium.toml is the right size. Add `cerebrium metrics resources APP`,
backed by the resource-metrics endpoint.

Reports peak usage over a window: CPU in cores, memory and GPU memory (VRAM)
in GB. Defaults to the last hour; --since, --start/--end, --container-id and
--resolution narrow it. Table output is a summary rather than a thousand
datapoints; --output json adds the raw series alongside it.

The endpoint relays raw query values, which arrive as JSON strings, and pads
gaps with null, so MetricValue decodes from either form and marshals back out
as a number. NaN and infinities decode as absent. A metric that never reported
prints "-" rather than 0.00, which would read as a measurement rather than the
absence of one.

Scoped to a single container the API returns one series per metric, named after
the metric itself, so summary cells are matched to columns by position and the
header collapses to a single PEAK column.

Also lifts normalizeAppID, which had been copied into three files, into
api.NormalizeAppID, moving its test along with it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Base automatically changed from feat/cli-json-output to main August 10, 2026 20:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant