Skip to content

feat(enterprise): allocate a LoadBalancer IP pool for the Advisor web console - #112

Merged
github-actions[bot] merged 1 commit into
developfrom
travis.wu/advisor-lb-ip-pool
Sep 15, 2026
Merged

github-actions[bot] merged 1 commit into
developfrom
travis.wu/advisor-lb-ip-pool

Conversation

@traviswu-bigstack

@traviswu-bigstack traviswu-bigstack commented Sep 15, 2026

Copy link
Copy Markdown
Contributor

What type of PR is this?

/kind feature

What this PR does / why we need it

The advisor was allocated exactly one LoadBalancer address. Its web console proxies an operator's browser to the cluster's own web UIs, each on its own browser origin — and in the offline deployment model, which cannot be assumed to have any DNS we can configure, each origin is a pinned IP address.

That is forced, not preferred. Browsers separate cookie jars by host and by nothing else: RFC 6265 ignores port entirely, and a path-based scheme needs URL rewriting that breaks the OIDC flow. So a cluster needs the advisor's own address plus one per distinct upstream the console reaches.

  • lbIPProbelbIPPoolProbe, returning up to N free addresses instead of the first one. The existing machinery was already the right shape — it walks the subnet's allocation pool skipping neutron-held addresses and ARP-probes the survivors — so this generalises without a redesign.
  • The install form is offered the whole pool (ClusterQuery.SuggestedAdvisorPool), from the same probe that produces the LB-IP suggestion.
  • Preflight ARP-probes every pool address, not just the framework's LB IP.
  • install-advisor.sh renders them as webConsole.origins[], and every pool address joins the certificate's SANs.

Which issue(s) this PR fixes

Fixes #105

Special notes for your reviewer

CMP and its Keycloak share one origin, deliberately. They sit behind the same framework ingress, so they share an upstream — and advisor-api refuses a config where two origins share one (webconsole: upstream %q is also used by origin with upstream %q). They would have to share an origin even if it allowed it: split across two, the OIDC state cookie is set on one origin and the callback arrives at the other, which is a broken login that reads as a proxy bug. The two constraints agree, which is a good sign the model is right.

Target names are the node's, not the chart's. The chart's commented example uses cmp-portal, cmp-idp, dashboard. Those are not the names cubecos allows — cube-cmp, app-fw-idp, cube-cos (bigstack-oss/cubecos#1482) — and a name the node does not allow is a target the agent refuses to dial. A test fails if the example names ever appear here.

Why the pool is asked for whole, up front. Every address becomes a SAN on the advisor's serving certificate, and advisor-api refuses to enable a console whose certificate does not cover an origin address, naming the uncovered one. Adding an address later therefore means reissuing that certificate, not editing a value — hence advisorPoolSize = 5 (floor is 3: advisor + ingress + the node's dashboard) rather than the minimum.

Backwards compatible. The pool is a 5th positional argument to the install script and is quoted, so an empty pool still occupies its slot rather than shifting every later argument — a test pins both the populated and empty forms. An install that passes no pool leaves the console off, exactly as today.

A reused certificate is refused when it does not cover the pool. #111 landed first and brought the TLS keypair's reuse-on-re-run behaviour with it, which interacts with this directly: a certificate issued before an address joined the pool does not cover it, and advisor-api would then refuse to enable the console. Rather than install a console that cannot start, the script now checks each pool address against the reused certificate and fails naming the address, telling the operator to delete cube-advisor-web-tls to reissue — and warning that every browser which has accepted this advisor will challenge it once more. That is the cost the pool's headroom exists to avoid paying.

Not verified on hardware. The probe and the rendering are covered by unit tests only. The lab advisor is installed with a single address and re-running the installer against it would reissue its serving certificate, which every browser that has accepted it would then challenge. Worth a clean-cluster run before this leaves draft.

Additional documentation

AdvisorPool — web-console origin addresses, one per distinct upstream the
              console reaches. Empty leaves the web console off.

@traviswu-bigstack
traviswu-bigstack force-pushed the travis.wu/advisor-lb-ip-pool branch from 364170f to c7d337e Compare September 15, 2026 01:13
… console

The advisor got one address. Its web console proxies a browser to the
cluster's own web UIs, and in the offline deployment model -- where no DNS
can be assumed -- each of those is a pinned IP rather than a name. That is
forced, not preferred: browsers separate cookie jars by host and by
nothing else (RFC 6265 ignores port), and a path scheme needs URL
rewriting that breaks the OIDC flow. So a cluster needs the advisor's own
address plus one per distinct upstream the console reaches.

lbIPProbe becomes lbIPPoolProbe and returns up to N free addresses rather
than the first one; the install form is offered the whole pool, and
preflight ARP-probes every address, not just the framework's LB IP.

install-advisor.sh renders them as webConsole origins. CMP and the app
framework's Keycloak share the ingress, so they share one origin --
advisor-api refuses two origins on one upstream, and splitting them would
set the OIDC state cookie on one origin and take the callback on the
other. Target names are the node's (cube-cmp, app-fw-idp, cube-cos), not
the chart's example names, which no cubecos node allows.

Every pool address joins the certificate's SANs. advisor-api refuses to
enable a console whose certificate does not cover an origin, so an address
added later means reissuing that certificate rather than editing a value
-- which is why the pool is asked for whole up front, with headroom.

An install that passes no pool leaves the console off, exactly as before.

Fixes #105

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PZ5umjjCedZwWtbAbiMjfj
Signed-off-by: Travis Wu <travis.wu@bigstack.co>
@traviswu-bigstack
traviswu-bigstack force-pushed the travis.wu/advisor-lb-ip-pool branch from c7d337e to ca4267c Compare September 15, 2026 01:50
@traviswu-bigstack
traviswu-bigstack marked this pull request as ready for review September 15, 2026 01:51
@traviswu-bigstack traviswu-bigstack added the done Merge the pull request label Sep 15, 2026
@github-actions
github-actions Bot merged commit ca4267c into develop Sep 15, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

done Merge the pull request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] Allocate a LoadBalancer IP pool for the Advisor web console

1 participant