Read the per-VM series from telegraf, not monasca - #656
Conversation
Twelve Flux queries still read from bucket "monasca" -- six history queries in internal/cubecos/metric.go and six Top-Instances rank queries in internal/apis/v1/handlers/metrics/stmt.go. bigstack-oss/cubecos#672 removed monasca, so on a freshly installed cluster that bucket does not exist and every per-VM chart and rank in the UI comes back empty. On an upgraded cluster it is worse than empty: the database is deliberately kept so an operator can still read history, so the queries succeed and silently return data that stops at the upgrade. hex_sdk instance_metrics_collect writes the replacement series, and it was written to be a drop-in: same measurement names, same tag names, same field name. Only the database moved. So this is a bucket rename and nothing else -- no measurement, tag, field or unit changes anywhere in the twelve queries. Verified against a live cluster rather than by inspection. Running the exact query these statements generate, against bucket "telegraf" on jim-1cc: _value 0.282257, _field value, _measurement vm.cpu.utilization_norm_perc, resource_id e1997ae2-..., tenant_id 6e6449b..., tenant_name admin, vm_name p3test-vm which is every tag the rank queries group by -- resource_id, vm_name -- plus the device tag the storage and network queries add, confirmed present on vm.io.* and vm.net.*. Two other leftovers go with it: the monasca module entry in the metrics group (module.go), which matches the group list in cubecos' cli_cluster.cpp -- prometheus and thanos already moved out to metricsDb, this is the rest of that change. MonascaDebugEnabled and its role mapping (tuning.go), a tuning for a service with nothing left to configure. go vet clean. The schema half is in cube-cos-openapi and arrives with the submodule bump. Signed-off-by: Jim Lin <jim.lin@bigstack.co> Co-authored-by: Eandalf <clinah@connect.ust.hk> Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
SekiXu
left a comment
There was a problem hiding this comment.
Approving. This is the cube-cos-api half of the blocker I raised on cubecos#1457, and it covers it completely.
Verified on this branch rather than from the write-up:
- All twelve are changed — six in
internal/apis/v1/handlers/metrics/stmt.goand six ininternal/cubecos/metric.go— andgit grep -i monascaover the branch returns nothing outside the submodule. - The
module.goandtuning.goremovals are symmetric with the product side: cubecos#1457 deletesconfig_monasca.cpp, which carriesCONFIG_TUNING_BOOL(MONASCA_DEBUG, "monasca.debug.enabled", ...)with it. So the tuning disappears from both ends together rather than leaving one side advertising a key the other no longer accepts. - The measurement, tag and field names are untouched, which is what makes this a bucket rename and nothing more.
- The submodule pins
3d9065db, which is cube-cos-openapi#112's head. That PR is a clean fast-forward from itsdevelop, so a true ff merge landsdevelopon exactly that SHA and this pin is already correct — no re-pin needed unless the merge is not a fast-forward.
One sequencing constraint beyond the three-repo order. This must not reach a build ahead of cubecos#1457. On cubecos/develop the six vm.* measurements are written only by monasca-agent's libvirt plugin, into the monasca bucket — instance_metrics_collect, which writes them to telegraf, exists only on that branch. An ISO carrying this without #1457 leaves all twelve endpoints reading a bucket nothing writes.
The 17.47% figure is the part of your verification that mattered: matching monasca's usable-based definition rather than ceilometer's ~26% is what shows the numbers kept their meaning and not merely their shape.
…e schema
Picks up cube-cos-openapi 6217dc4, which removes both retired services from
docs.yaml:
monasca the module entry in the metrics group, the two module-name enums,
and the monasca.debug.enabled tuning. Removed from CubeCOS by
bigstack-oss/cubecos#672 phase 4.
senlin the module entry in the businessLogic group and the
senlin.debug.enabled tuning. Removed from CubeCOS by d4550c91 back
on the yoga train; the schema was only half-updated at the time, so
senlin came out of the module enums but stayed in those two.
Pairs with the Go change in the previous commit. Without it the generated
api/docs.json still advertises monasca as a queryable module and a settable
tuning, while the code behind both has gone -- a client could ask the API to
repair a module it no longer knows about.
api/docs.json is not tracked; it is generated from the submodule by task
generateApiDocs and embedded into the binary by api/docs.go's go:embed, so the
bump does change the shipped artifact even though no tracked file moves.
Verified: after the bump, yq produces a docs.json with zero monasca and zero
senlin references.
The pointer targets the openapi branch rather than its develop, since that PR
has not merged yet; it wants re-bumping to the merged SHA before this lands.
Signed-off-by: Jim Lin <jim.lin@bigstack.co>
Co-authored-by: Eandalf <clinah@connect.ust.hk>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
066caf5 to
ee34714
Compare
What type of PR is this?
/kind bug
Which issue(s) this PR fixes?
Part of bigstack-oss/cubecos#772 — specifically bigstack-oss/cubecos#769 (COS UI > Home > Chart page after the OpenStack and InfluxDB upgrade) and bigstack-oss/cubecos#771 (COS UI > Home > Health page).
What this PR does?
Repoints twelve Flux queries from the
monascabucket totelegraf. Six history queries ininternal/cubecos/metric.goand six Top-Instances rank queries ininternal/apis/v1/handlers/metrics/stmt.go.bigstack-oss/cubecos#672 removed monasca. The consequence differs by install type, and the upgrade case is the worse one:
monascabucket does not exist, so every per-VM chart and rank in the UI comes back emptymigrate_monasca_retiredeliberately keeps the InfluxDB database so an operator can still read history. So the queries succeed and silently return data frozen at the upgrade momenthex_sdk instance_metrics_collectwrites the replacement series and was written as a drop-in: same measurement names, same tag names, same field name. Only the database moved. So this is a bucket rename and nothing else — no measurement, tag, field or unit changes in any of the twelve.Two other monasca leftovers go with it:
monascaentry in themetricsmodule group (module.go), matching the group list in CubeCOS'cli_cluster.cpp—prometheusandthanosalready moved out tometricsDb, this is the rest of that changeMonascaDebugEnabledand its role mapping (tuning.go), a tuning for a service with nothing left to configureTest results (optional)
1). make sure the api docs have been updated
api/docs.jsonregenerates from the bumped submodule with zero monasca and zero senlin references. It is not tracked, butapi/docs.goembeds it withgo:embed, so it does reach the binary — the two builds differ in size accordingly.2). make sure the api works properly
Verified against live clusters, not by inspection.
Query level — running the exact statement these builders generate against
bucket: "telegraf"on jim-1cc returns every tag the queries rely on:resource_idandvm_nameare what the rank queries group by;deviceis present onvm.io.*andvm.net.*for the storage and network queries.Endpoint level — binary built from this branch and staged on jim-1cc and all three accept-3cc controllers (
/usr/local/bin/cube-cos-api, service active on :8082). Each node independently reports 0 monasca/senlin strings and 12from(bucket: "telegraf")references — exactly the twelve queries changed. Calling the real endpoints with a keystone token:GET /api/v1/datacenters/cc1/metrics/cpuUsage/rank/vms200—p3test-vm, value0.3041, full history seriesGET /api/v1/datacenters/cc1/metrics/memoryUsage/rank/vms200—p3test-vm, value17.4731, full history seriesThe memory figure is worth noting: 17.47% matches monasca's
usable-based definition rather than the ceilometer one (~26%), so the numbers the UI draws keep their meaning, not just their shape.hex_cli -v -c cluster checkremains 28/28 OK on both labs after the swap.This PR needs
3d9065d); it wants re-pinning to the merged SHA before this lands.🤖 Generated with Claude Code