Worker observability: expose controller-connection health and last-seen

[gourishkb]

Summary

Add conditions, events, and metrics to the worker operator so that platform operators can observe controller-connection health and track failover progress in real time.

Depends on: #467 (worker reconnect implemented)

---

Conditions

Add the following conditions to the Cluster CR (or equivalent worker status CR — confirm against ADR-001 Decision 7):

status:
  conditions:
    - type: ControllerConnected
      status: "True"          # or "False" or "Unknown"
      reason: Connected         # see reason table below
      message: ""
      lastTransitionTime: "2026-06-24T10:00:00Z"
    - type: ControllerEndpointSynced
      status: "True"
      reason: EndpointUpToDate
      message: ""
      lastTransitionTime: "2026-06-24T10:00:00Z"

Reason codes for ControllerConnected:

Reason	Status	Meaning
Connected	True	Management-plane connection to hub is healthy
ReconnectedAfterFailover	True	Successfully reconnected after hub endpoint changed
DialFailed	False	Could not reach controller endpoint; retrying
CertVerificationFailed	False	TLS handshake failed — CA bundle mismatch
EndpointNotConfigured	Unknown	No controllerEndpoint field present (non-HA mode)
Reconnecting	Unknown	Endpoint changed; dial in progress

Condition transitions must be idempotent — calling SetCondition with the same values must not create a spurious update.

---

Events

Emit Events on the worker namespace for these transitions:

reason	type	When
ControllerEndpointChanged	Normal	Cluster CR controllerEndpoint field updated
ControllerConnected	Normal	Connection to hub established or re-established
ControllerConnectionLost	Warning	Connection dropped; reconnect in progress
CertVerificationFailed	Warning	TLS handshake failed with new endpoint

---

Metrics (optional, implement if time permits)

# HELP worker_controller_reconnect_attempts_total Number of connection attempts to hub controller
# TYPE worker_controller_reconnect_attempts_total counter
worker_controller_reconnect_attempts_total{result="success"} 1
worker_controller_reconnect_attempts_total{result="failure"} 3

# HELP worker_controller_last_sync_time_seconds Unix timestamp of last successful sync from hub
# TYPE worker_controller_last_sync_time_seconds gauge
worker_controller_last_sync_time_seconds 1750000000

---

Acceptance Criteria

• ControllerConnected condition present on Cluster CR after worker starts
• Condition transitions to False/DialFailed when hub is unreachable — verified by unit test with fake dialer
• Condition transitions to True/ReconnectedAfterFailover after reconnect — verified by unit test
• Condition transitions are idempotent (no spurious updates on no-op)
• All four Events above are emitted in the correct scenarios
• reason values are consistent with the table above — no ad hoc strings elsewhere in the code