Skip to content

CS management clusters into cosmosdb#4513

Merged
openshift-merge-bot[bot] merged 8 commits into
mainfrom
mgmt-cluster-cosmosdb
May 12, 2026
Merged

CS management clusters into cosmosdb#4513
openshift-merge-bot[bot] merged 8 commits into
mainfrom
mgmt-cluster-cosmosdb

Conversation

@geoberle
Copy link
Copy Markdown
Collaborator

@geoberle geoberle commented Mar 18, 2026 •
edited by openshift-ci Bot
Loading

What

mirror CS management clusters into cosmosdb

  • introduce new cosmosdb types
    ** Stamp - the infra deployment unit for a management cluster (think capi Machine)
    ** ManagementCluster - the provision shards for HCPs (think k8s Node)
  • includes API types, CRUD interface, database lister, informer, OCM conversion logic
  • ... and a periodic sync controller for importing them from CS
  • CS is still the inventory for management clusters and source of truth
  • includes a controller to sync the mgmt cluster placement information from CS into the serviceprovidercluster document
"cosmosMetadata": {
    "resourceID": "/providers/microsoft.redhatopenshift/stamps/1"
},
"resourceId": "/providers/microsoft.redhatopenshift/stamps/1",
"spec": {},
"status": {
    "conditions": [
        {
            "type": "Approved",
            "status": "True",
            "lastTransitionTime": "2026-05-11T00:44:23Z",
            "reason": "AutoApproved",
            "message": "Synced from Cluster Service provision shard"
        }
    ]
}

{
  "cosmosMetadata": {
    "resourceID": /providers/microsoft.redhatopenshift/stamps/1/managementclusters/default
  },
  "spec": {
    "schedulingPolicy": "Schedulable"
  },
  "status": {
    "aksResourceID": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/hcp-underlay-westus3/providers/Microsoft.ContainerService/managedClusters/hcp-underlay-westus3-mgmt-1",
    "publicDNSZoneResourceID": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/global/providers/Microsoft.Network/dnsZones/westus3.aroapp.io",
    "hostedClustersSecretsKeyVaultURL": "https://hcp-underlay-westus3-mgmt-1-cx.vault.azure.net",
    "hostedClustersManagedIdentitiesKeyVaultURL": "https://hcp-underlay-westus3-mgmt-1-msi.vault.azure.net",
    "hostedClustersSecretsKeyVaultManagedIdentityClientID": "00000000-0000-0000-0000-000000000001",
    "maestroConsumerName": "hcp-underlay-westus3-mgmt-1",
    "maestroRESTAPIURL": "http://maestro.maestro.svc.cluster.local:8000",
    "maestroGRPCTarget": "maestro-grpc.maestro.svc.cluster.local:8090",
    "clusterServiceProvisionShardID": "/api/clusters_mgmt/v1/provision_shards/00000000-0000-0000-0000-000000000002",
    "conditions": [
      {
        "type": "Ready",
        "status": "True",
        "lastTransitionTime": "2026-04-01T10:00:00Z",
        "reason": "ProvisionShardActive",
        "message": "ClustersService provision shard is active"
      }
    ]
  }
}

implements phase 1 of https://github.com/openshift-online/aro-enhancements/pull/1

Jira

ARO-26914 ARO-26917 ARO-26918 ARO-26919 ARO-26920 ARO-26921 ARO-26922

@openshift-ci openshift-ci Bot requested review from bennerv and deads2k March 18, 2026 15:35
@machi1990
Copy link
Copy Markdown
Collaborator

machi1990 commented Mar 18, 2026

We could potentially make

ARO-HCP/backend/pkg/controllers/delete_orphaned_maestro_readonly_bundles_controller.go

Line 199 in be491ea

provisionShardsIterator := c.clusterServiceClient.ListProvisionShards()
read db content instead of making a CS call to enlist all clusters.

machi1990
machi1990 reviewed Mar 18, 2026
View reviewed changes
Comment thread internal/validation/validate_management_cluster_test.go Outdated
@geoberle geoberle marked this pull request as draft March 18, 2026 16:16
@geoberle geoberle force-pushed the mgmt-cluster-cosmosdb branch from be491ea to 9c07c65 Compare March 19, 2026 08:47
@geoberle geoberle marked this pull request as ready for review March 19, 2026 16:40
@openshift-ci openshift-ci Bot requested a review from mbarnes March 19, 2026 16:40
@geoberle geoberle force-pushed the mgmt-cluster-cosmosdb branch 2 times, most recently from eee29a6 to dec6c48 Compare March 19, 2026 21:57
machi1990
machi1990 suggested changes Mar 19, 2026
View reviewed changes
Comment thread internal/ocm/convert_test.go Outdated
Comment thread internal/api/types_management_cluster.go Outdated
Comment thread backend/pkg/controllers/managementclustercontrollers/management_cluster_sync_test.go Outdated
Comment thread internal/validation/validate_management_cluster.go Outdated
Comment thread backend/pkg/controllers/managementclustercontrollers/management_cluster_migration.go
Comment thread internal/ocm/convert.go Outdated
Comment thread internal/ocm/convert.go Outdated
Comment thread internal/validation/validate_management_cluster.go Outdated
Comment thread internal/validation/validate_management_cluster.go Outdated
Comment thread internal/validation/validate_management_cluster.go Outdated
@geoberle geoberle force-pushed the mgmt-cluster-cosmosdb branch from dec6c48 to 2d5c4c3 Compare March 20, 2026 10:13
@geoberle geoberle force-pushed the mgmt-cluster-cosmosdb branch from 2d5c4c3 to cac7b80 Compare March 20, 2026 10:14
@geoberle geoberle force-pushed the mgmt-cluster-cosmosdb branch 3 times, most recently from d2df91c to 77f2b6e Compare March 20, 2026 11:45
machi1990
machi1990 reviewed Mar 23, 2026
View reviewed changes
Comment thread docs/mgmt-cluster-fleet-management.md Outdated
machi1990
machi1990 reviewed Mar 23, 2026
View reviewed changes
Comment thread internal/api/types_serviceprovider_cluster.go Outdated
@geoberle geoberle force-pushed the mgmt-cluster-cosmosdb branch from e885710 to faaaa2b Compare March 24, 2026 15:26
Copilot AI reviewed May 7, 2026
View reviewed changes
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

tuxerrante
tuxerrante approved these changes May 8, 2026
View reviewed changes
Copy link
Copy Markdown

@tuxerrante tuxerrante left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@openshift-ci
Copy link
Copy Markdown

openshift-ci Bot commented May 8, 2026

@tuxerrante: changing LGTM is restricted to collaborators

Details

In response to this:

/lgtm

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

Copilot AI reviewed May 8, 2026
View reviewed changes
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

Copilot AI reviewed May 10, 2026
View reviewed changes
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

Copilot AI reviewed May 11, 2026
View reviewed changes
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 64 out of 64 changed files in this pull request and generated 6 comments.

Comment thread internal/database/crud_validating.go
Comment thread backend/pkg/controllers/managementclustercontrollers/management_cluster_migration.go
Comment thread backend/pkg/controllers/managementclustercontrollers/management_cluster_migration.go
Comment thread internal/database/informers/list_watch.go
Comment thread internal/api/fleet/types_management_cluster.go Outdated
Comment thread backend/pkg/controllers/managementclustercontrollers/management_cluster_placement_sync.go Outdated
Copilot AI reviewed May 11, 2026
View reviewed changes
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

geoberle
geoberle commented May 11, 2026
View reviewed changes
Comment thread hack/run-with-port-forward.sh
machi1990
machi1990 reviewed May 11, 2026
View reviewed changes
Comment thread internal/api/types_serviceprovider_cluster.go
machi1990
machi1990 reviewed May 11, 2026
View reviewed changes
Comment thread internal/api/fleet/types_management_cluster.go
machi1990
machi1990 reviewed May 11, 2026
View reviewed changes
Comment thread backend/pkg/app/cosmos_wiring.go Outdated
machi1990
machi1990 reviewed May 11, 2026
View reviewed changes
Comment thread backend/pkg/controllers/managementclustercontrollers/management_cluster_sync.go Outdated
machi1990
machi1990 approved these changes May 11, 2026
View reviewed changes
Copy link
Copy Markdown
Collaborator

@machi1990 machi1990 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@geoberle
Copy link
Copy Markdown
Collaborator Author

geoberle commented May 11, 2026

/test e2e-parallel

@openshift-merge-bot
Copy link
Copy Markdown
Contributor

openshift-merge-bot Bot commented May 11, 2026

/retest-required

Remaining retests: 0 against base HEAD bca1ae4 and 2 for PR HEAD f6ca56c in total

geoberle added 7 commits May 11, 2026 13:47
@geoberle
database: add shared Cosmos informer infrastructure
ca244c7 
Add the listWatchWithoutWatchListSemantics wrapper and expiringWatcher
to internal/database/informers/. These are type-agnostic building blocks
for Cosmos-backed SharedIndexInformers that do not support the native
Kubernetes watch protocol. Taken from the kube-applier PR (#5076) so
both fleet and kube-applier subsystems share the same infrastructure.
@geoberle
fleet: add ManagementCluster type
50f3784
@geoberle
backend: wire fleet informers and management cluster controllers
f68f9cf
@geoberle
validation layer
909ea73 
Signed-off-by: Gerd Oberlechner <goberlec@redhat.com>
@geoberle
controller docs
a94645e
@geoberle
add stamp types
6dcd3d4 
Signed-off-by: Gerd Oberlechner <goberlec@redhat.com>
@geoberle
* rename mgmt cluster sync controller go ManagementClusterMigrationCo…
bc77341 
…ntroller

* create cosmosdb client once for both clients
Copilot AI reviewed May 11, 2026
View reviewed changes
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 66 out of 66 changed files in this pull request and generated 3 comments.

Comment thread internal/validation/validators.go
Comment thread internal/api/types_internalid.go
Comment thread hack/run-with-port-forward.sh
machi1990
machi1990 approved these changes May 11, 2026
View reviewed changes
Copy link
Copy Markdown
Collaborator

@machi1990 machi1990 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@openshift-ci
Copy link
Copy Markdown

openshift-ci Bot commented May 11, 2026

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: geoberle, machi1990, tuxerrante

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci
Copy link
Copy Markdown

openshift-ci Bot commented May 11, 2026 •
edited
Loading

@geoberle: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/cspr b0d4c11 link true /test cspr
ci/prow/images-push b0d4c11 link true /test images-push

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@openshift-merge-bot
Copy link
Copy Markdown
Contributor

openshift-merge-bot Bot commented May 11, 2026

/retest-required

Remaining retests: 0 against base HEAD 4e66ca0 and 2 for PR HEAD 3e8fb30 in total

@geoberle
Copy link
Copy Markdown
Collaborator Author

geoberle commented May 11, 2026

/retest-required

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@deads2k deads2k deads2k left review comments

Copilot code review Copilot Copilot left review comments

@bennerv bennerv Awaiting requested review from bennerv

@mbarnes mbarnes Awaiting requested review from mbarnes

@janboll janboll Awaiting requested review from janboll

+2 more reviewers

@tuxerrante tuxerrante tuxerrante approved these changes

@machi1990 machi1990 machi1990 approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

@machi1990 machi1990

Labels

approved lgtm

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@geoberle @machi1990 @raelga @deads2k @tuxerrante