Integrate the new okdp-server and okdp-ui into the sandbox

[alex-mabrouk]

Problem Statement

The roadmap lists New okdp-server & okdp-ui versions under the Sandbox section, but the sandbox still deploys the current console, and there is no tracked plan yet for how the new control-plane should land here.

I spent some time running the new control plane server and new control plane UI against the current sandbox (Keycloak, current contexts and releases) to understand what is still needed before they can replace the existing control-plane.

The good news: after a few adjustments, every catalog service deploys end-to-end from the new console: SeaweedFS, Hive Metastore, Polaris, Trino, Superset, JupyterHub, Spark History Server, Airflow and OKDP Examples.

The integration required a handful of changes across the sandbox and the server, so the migration needs a bit of coordination rather than a single PR.

Proposed Solution

Land the integration as a sequence of small PRs. From what the experiment showed, the work falls into four areas:

1. Additional metadata in the sandbox contexts consumed by the new server (catalog block, root-level ingress, identity provisioning settings).
2. Keycloak configuration for the new UI (client scopes, redirect URIs, roles).
3. Alignment of the Releases created by the new server with the KuboCD defaultContexts mechanism used by the sandbox (server-side change).
4. Packaging: new images and KuboCD packages for the server and the UI, then switching the okdp-server / okdp-ui releases to them.

I have all of this working on a fork and I'm happy to split it into small PRs,

starting wherever you prefer. Is there already a preferred sequence or an existing migration plan I should align with?

Alternatives Considered

A single big-bang PR replacing the console at once; I'd rather avoid it: the changes span the sandbox contexts, the server and the packaging, and each area deserves its own review.

Additional Context

Related: the new server's Identity API currently depends on kubauth CRDs, which are not installed on the sandbox; tracked separately in
issue 25 on control plane server.

Its outcome affects step 4 (the new console cannot fully replace the current one while that page targets kubauth).

[mlahouar]

This is exactly the validation this migration needed. A few notes:

• It confirms the identity direction discussed in okdp-control-plane-server#25: standard OIDC, kubauth optional.
• Part of item 4 is already done: server and UI images and charts are published. What remains is the KuboCD packages for both.
• Items 1 and 3 overlap with the catalog management work in flight (okdp-control-plane-server#24, okdp-control-plane-ui#40).

The sequence of small PRs works for me.

[alex-mabrouk]

Thanks Moncef, that's helpful.

I'll switch to the official images and charts and drop the temporary ones from my fork.

My plan would be:

1. Open a PR for the KuboCD packages of the new server and UI (using the official images/charts). This replaces the current okdp-server/okdp-ui package definitions. It's safe to land early, since the sandbox releases keep pinning the already-published legacy tags until the switch.
2. Open a PR for the remaining sandbox changes (root ingress, identity provisioning settings and Keycloak configuration for the new UI).
3. Finally, switch the okdp-server / okdp-ui releases once everything is in place. This one also needs a UI image rebuilt with the #41 fix (the
   published 0.6.0 predates it).

One point I'd like to clarify regarding okdp-control-plane-server#24: the sandbox currently relies on KuboCD defaultContexts, while okdp-control-plane-server#24 writes the catalog into a single default Context. Do you see the catalog living in the existing platform Context, or would you rather introduce a dedicated writable Context? Happy to continue that discussion on okdp-control-plane-server#24.

Also, for reference, both consoles are currently running side by side on my fork (legacy on https://okdp-ui.<suffix>, new one on https://console.<suffix>). That made it easy to validate the migration without replacing the existing console.