Skip to content

Show CPU, memory and disk usage per instance and per volume - #134

Open
traviswu-bigstack wants to merge 2 commits into
masterfrom
travis.wu/instance-usage-columns
Open

traviswu-bigstack wants to merge 2 commits into
masterfrom
travis.wu/instance-usage-columns

Conversation

@traviswu-bigstack

Copy link
Copy Markdown

What type of PR is this?

/kind feature

What this PR does / why we need it

The instance list (/compute/instance and /compute/instance-admin) gains a single Usage column holding three stacked gauges — CPU, MEM, DISK — and the volume list (/storage/volume and /storage/volume-admin) gains an Allocated column. Part of the customer request behind bigstack-oss/cubecos#1449: see per-VM space use in the GUI.

Both read Prometheus directly, one instant query per metric returning every row at once, so adding the columns costs a fixed number of requests per page rather than one per table row on every render.

Gauge colours are the disk alarm's thresholds (tpl_alert_vm_disk.tick): blue below 50, green to 75, yellow to 85, red above. Warn and crit collapse into one red band — a row at 90% and a row at 96% both need attention now. A row that is about to raise an alarm looks like it before the alarm exists.

Disk falls back when an instance has no qemu-guest-agent, rather than reading -. The fallback is block allocation, drawn grey and marked with *, deliberately not on the colour scale — see below.

Which issue(s) this PR fixes

Part of bigstack-oss/cubecos#1449. The series come from the collector in bigstack-oss/cubecos#1450, which also fixes the prometheus_endpoint in skyline.yaml.inwithout that fix every Skyline Prometheus query 404s and all of these columns render -.

Special notes for your reviewer

Why the fallback is grey and the volume column is called "Allocated". Block allocation and filesystem usage are not the same measurement and cannot be substituted for one another. Ceph keeps a block allocated once written unless the guest passes TRIM all the way through, so allocation runs ahead of filesystem usage and never comes back down. On a measured instance here the block layer reported 85% while the guest reported 14.8%, and running fstrim inside the guest — which reported 32.1 GiB trimmed, exit 0 — moved rbd du by nothing (confirmed by counting RADOS objects, not just trusting the object map: 4350 objects × 8 MiB). Ceph's rbd_skip_partial_discard frees an object only when the whole 8 MiB is discarded, and on an aged filesystem free space is too scattered for that.

So colouring a fallback row red would flag an idle instance, and those rows cannot raise the disk alarm anyway — it streams guest readings, which is precisely what they lack. On the volume page there is no guest reading to fall back from, since nothing maps a block device back to a guest mountpoint; naming that column "Usage" invites reading a fully allocated thin volume as a full one, so it says "Allocated" and the tooltip gives the absolute figures.

The shared UsageBar component and resources/prometheus/usage.js exist because two pages now draw the same gauge; the per-page modules only hold their own queries.

Additional documentation


The instance list gains one Usage column holding three stacked gauges -- CPU,
MEM, DISK -- and the volume list gains an Allocated column. Both read
Prometheus directly with one instant query per metric returning every row at
once, rather than a request per table row on each render.

Gauge colours are the thresholds tpl_alert_vm_disk.tick fires on: blue below
50, green to 75, yellow to 85, red above. Warn and crit collapse into one red
band because a row at 90% and a row at 96% both need attention now. A row
about to raise a disk alarm looks like it before the alarm exists.

Disk falls back when an instance has no qemu-guest-agent. The guest figure is
absent for those, so the column uses block allocation instead of reading "-".
It is drawn grey and marked, not coloured, because the two are not the same
measurement: Ceph keeps a block allocated once written unless the guest passes
TRIM through, so allocation runs far ahead of filesystem usage and never comes
back down. One measured instance reads 85% allocated against 14.8% used, and
fstrim freeing 32.1 GiB inside the guest moved rbd du by nothing. Colouring
that red would flag an idle instance, and the disk alarm cannot fire for it
anyway -- the alarm streams guest readings, which is exactly what these rows
lack.

The volume column is named Allocated rather than Usage for the same reason.
There is no guest reading to fall back from: a volume is a block device and
nothing maps one back to a guest mountpoint, so allocation is the only
per-volume figure there is, and calling it "usage" invites reading a fully
allocated thin volume as a full one.

Signed-off-by: Travis Wu <travis.wu@bigstack.co>
…ures

Both the instance list and the volume list now show what the guest reports,
and fall back to the block layer only when there is no guest reading. A
fallback row carries a "block level" tag after its percentage.

The volume list used to show allocation for every row. That is not usage:
blocks stay allocated once written, so a volume written end to end reads 100%
while its filesystem is nearly empty -- two volumes here read 85% and 100%
against guests using 14.8% and less. It now reads the per-volume guest rollup,
which the collector keys off the uuid nova writes into the disk's libvirt
<serial>.

A tag rather than a colour, because colour already carries the alert
thresholds. Two meanings on one channel is why the same 100% appeared grey on
one page and red on the other: grey meant "different measurement" on the
instance list, while the volume list had no fallback concept at all and used
the normal scale. One meaning per channel now -- colour is nearness to the
alert, the tag is which layer measured it.

The tag is a word rather than an icon or a fill pattern: those are only
discoverable by hovering, and the figure being marked is precisely the one
that has already been misread.

A row with neither reading renders "Not measured" and no bar at all. An empty
bar sits at zero, which reads as an empty disk rather than as an unanswered
question -- the state of a volume on a backend with no usage adapter attached
to an instance with no agent.

Strings land in en, zh-hant and zh-hans; the tag reads 區塊層級 in Traditional
and 区块层级 in Simplified.

Signed-off-by: Travis Wu <travis.wu@bigstack.co>
@traviswu-bigstack
traviswu-bigstack force-pushed the travis.wu/instance-usage-columns branch from 8cdd241 to c031c65 Compare September 16, 2026 06:19
@traviswu-bigstack
traviswu-bigstack marked this pull request as ready for review September 16, 2026 06:23
@traviswu-bigstack
traviswu-bigstack requested a review from a team as a code owner September 16, 2026 06:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant