cubecosapi: the agent can read its own cluster

[traviswu-bigstack]

What

An authenticated cube-cos-api client, and the setting that wires it — ADR 0016 slice 4 plus the component it configures.

internal/cubecosapi performs the read; cube-cos-api.json in the agent's own directory says where the api is and what to call the cluster. 18 of the 21 catalogue paths now return real data from a live cluster; three that cannot work are held back with the reason.

Why

Lab validation on the 1cc R630 found every catalogue path answering cube-cos-api access is not configured on this agent. Two defects, not one:

• ConfigureCubeCOS had no non-test caller — the configuration half, which slice 4 covers.
• Nothing in this repository implemented a CubeCOSGetter at all. ADR 0016 scoped the client out deliberately: "finding 2 is two problems — no configuration surface and no client to configure."

Slice 4 alone would have wired nothing, so this does both.

Reviewer notes

Auth: the internal node-to-node path, verified from source and on hardware. verifyAuthToken checks it before the OpenStack and OIDC paths, and isValidInternalToken compares the bearer token against nodes.GenToken(<Node header>) = sha512(hostname + oidc client secret). cube-cos-api writes its own value to /var/run/cube-cos-api/node_token (flushDefaultNodeToken, mode 0644). On the R630, a datacenter-scoped read is 401 without those two headers and 200 with them — so this is necessary, not merely sufficient.

The token is read per request. That file is rewritten every time the api starts, so a cached value would keep working until the next api restart and then fail in a way that looks like a configuration error and is cured by restarting the agent. Its mode is deliberately not checked: cube-cos-api publishes it 0644 and owns that decision; refusing to read it would disable the agent over something it cannot change. That the token is readable by any local account is cube-cos-api's exposure, not one this introduces — worth someone's attention, but not fixable here.

TLS: there is none, and the note that said otherwise was wrong. cube-cos-api's newHttpServer passes no certificate, and the config template has TLS only under the SAML service provider. On the R630 it listens on 10.32.1.200:8082, https:// answers nothing and http:// answers 200. Nothing was disabled to make this work — there was nothing to verify. The client accepts https so a future api that offers it needs no change, and refuses any other scheme at construction where an operator sees it in a startup line. Talking to the node's own management address keeps the token off the wire.

The setting's mode policy is the instance profile's, not the credential's. Refused if group- or world-writable; readability unconstrained. The file holds no secret — a datacenter name and a URL on the customer's own network — but whoever can write it chooses where the agent sends its node token, and a token posted to an attacker's listener is the whole of the node's api authority. Write access is the escalation.

maxBytes reads maxBytes+1 so truncation is detected rather than inferred from an exactly-full buffer, then returns ErrOutputTruncated — the contract the command runner already follows, so fetch stays the one place that decides what a cut looks like. Verified end to end against a real 42 KB response: capped at 1024 with the marker present.

Three reads held back. events, events/abstract and events/rank are zero-parameter GETs in the OpenAPI document and return 400 'type' can't be null and should be one of 'system', 'host', or 'instance' on a real cluster. The catalogue deliberately cannot express query parameters, so admitting them offers the model three reads that can only ever fail. cube_cos_events shipped as a bare GET of the same path before the catalogue existed — this narrows nothing that ever worked, and the vendored-spec conformance check could not have caught it, because the path exists and the method is GET. events/filterConditions and events/predefined need no parameter and stay admitted.

Validated on the 1cc R630, read-only

unconfigured read refused: cube-cos-api access is not configured on this agent;
                           write cube-cos-api.json to enable it
OK   healths       6266 bytes    OK   volumes        6897 bytes
OK   nodes         2102 bytes    OK   tunings/specs 42061 bytes
OK   services      2673 bytes    OK   metrics        2253 bytes
OK   events/filterConditions 203 OK   events/predefined 81 bytes
OK   truncation    1082 bytes, marker present
held-back resource refused: value for resource is not one this tool reads

Nothing was left behind: the probe binary was removed, no /etc/cube/advisor-agent was created, and cluster check reports every group ok.

Break and restore

break	result
setting removed from the declared list	layer 1 fails naming the file an operator would write for nothing; layer 2 also fails
declared and loaded, apply dropped	layer 1 passes, layer 2 fails — the discriminating case
redaction dropped from String	all four verbs and the in-slice case fail

One test of my own was wrong before it was right: TestAnUnknownCubeCOSFieldIsRefused used dataCenter, which encoding/json matches case-insensitively, so the field lands correctly and DisallowUnknownFields never fires. Corrected to datacentre and baseUrl — genuinely different names — and the tolerated case-variant behaviour is now pinned by its own test rather than left to be discovered.

Docs

bigstack-handbook#<n> — the as-built beside slices 1 and 3.