feat(enterprise): allocate a LoadBalancer IP pool for the Advisor web console - #112
Merged
Merged
Conversation
traviswu-bigstack
force-pushed
the
travis.wu/advisor-lb-ip-pool
branch
from
September 15, 2026 01:13
364170f to
c7d337e
Compare
… console The advisor got one address. Its web console proxies a browser to the cluster's own web UIs, and in the offline deployment model -- where no DNS can be assumed -- each of those is a pinned IP rather than a name. That is forced, not preferred: browsers separate cookie jars by host and by nothing else (RFC 6265 ignores port), and a path scheme needs URL rewriting that breaks the OIDC flow. So a cluster needs the advisor's own address plus one per distinct upstream the console reaches. lbIPProbe becomes lbIPPoolProbe and returns up to N free addresses rather than the first one; the install form is offered the whole pool, and preflight ARP-probes every address, not just the framework's LB IP. install-advisor.sh renders them as webConsole origins. CMP and the app framework's Keycloak share the ingress, so they share one origin -- advisor-api refuses two origins on one upstream, and splitting them would set the OIDC state cookie on one origin and take the callback on the other. Target names are the node's (cube-cmp, app-fw-idp, cube-cos), not the chart's example names, which no cubecos node allows. Every pool address joins the certificate's SANs. advisor-api refuses to enable a console whose certificate does not cover an origin, so an address added later means reissuing that certificate rather than editing a value -- which is why the pool is asked for whole up front, with headroom. An install that passes no pool leaves the console off, exactly as before. Fixes #105 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PZ5umjjCedZwWtbAbiMjfj Signed-off-by: Travis Wu <travis.wu@bigstack.co>
traviswu-bigstack
force-pushed
the
travis.wu/advisor-lb-ip-pool
branch
from
September 15, 2026 01:50
c7d337e to
ca4267c
Compare
traviswu-bigstack
marked this pull request as ready for review
September 15, 2026 01:51
This was referenced Sep 15, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What type of PR is this?
/kind feature
What this PR does / why we need it
The advisor was allocated exactly one LoadBalancer address. Its web console proxies an operator's browser to the cluster's own web UIs, each on its own browser origin — and in the offline deployment model, which cannot be assumed to have any DNS we can configure, each origin is a pinned IP address.
That is forced, not preferred. Browsers separate cookie jars by host and by nothing else: RFC 6265 ignores port entirely, and a path-based scheme needs URL rewriting that breaks the OIDC flow. So a cluster needs the advisor's own address plus one per distinct upstream the console reaches.
lbIPProbe→lbIPPoolProbe, returning up to N free addresses instead of the first one. The existing machinery was already the right shape — it walks the subnet's allocation pool skipping neutron-held addresses and ARP-probes the survivors — so this generalises without a redesign.ClusterQuery.SuggestedAdvisorPool), from the same probe that produces the LB-IP suggestion.install-advisor.shrenders them aswebConsole.origins[], and every pool address joins the certificate's SANs.Which issue(s) this PR fixes
Fixes #105
Special notes for your reviewer
CMP and its Keycloak share one origin, deliberately. They sit behind the same framework ingress, so they share an upstream — and
advisor-apirefuses a config where two origins share one (webconsole: upstream %q is also used by origin with upstream %q). They would have to share an origin even if it allowed it: split across two, the OIDC state cookie is set on one origin and the callback arrives at the other, which is a broken login that reads as a proxy bug. The two constraints agree, which is a good sign the model is right.Target names are the node's, not the chart's. The chart's commented example uses
cmp-portal,cmp-idp,dashboard. Those are not the names cubecos allows —cube-cmp,app-fw-idp,cube-cos(bigstack-oss/cubecos#1482) — and a name the node does not allow is a target the agent refuses to dial. A test fails if the example names ever appear here.Why the pool is asked for whole, up front. Every address becomes a SAN on the advisor's serving certificate, and
advisor-apirefuses to enable a console whose certificate does not cover an origin address, naming the uncovered one. Adding an address later therefore means reissuing that certificate, not editing a value — henceadvisorPoolSize = 5(floor is 3: advisor + ingress + the node's dashboard) rather than the minimum.Backwards compatible. The pool is a 5th positional argument to the install script and is quoted, so an empty pool still occupies its slot rather than shifting every later argument — a test pins both the populated and empty forms. An install that passes no pool leaves the console off, exactly as today.
A reused certificate is refused when it does not cover the pool. #111 landed first and brought the TLS keypair's reuse-on-re-run behaviour with it, which interacts with this directly: a certificate issued before an address joined the pool does not cover it, and
advisor-apiwould then refuse to enable the console. Rather than install a console that cannot start, the script now checks each pool address against the reused certificate and fails naming the address, telling the operator to deletecube-advisor-web-tlsto reissue — and warning that every browser which has accepted this advisor will challenge it once more. That is the cost the pool's headroom exists to avoid paying.Not verified on hardware. The probe and the rendering are covered by unit tests only. The lab advisor is installed with a single address and re-running the installer against it would reissue its serving certificate, which every browser that has accepted it would then challenge. Worth a clean-cluster run before this leaves draft.
Additional documentation