Skip to content

mcs: use the configured lease for the service registry instead of a fixed value #11016

Description

@bufferflies

Enhancement Task

Background

When a microservice (TSO / Scheduling / Resource Manager / Router) registers itself into the service registry via discovery.Register, the registry lease TTL is hardcoded to discovery.DefaultLeaseInSeconds (5s):

serviceRegister := discovery.NewServiceRegister(s.Context(), s.GetEtcdClient(),
    serviceName, s.GetAdvertiseListenAddr(), serializedEntry,
    discovery.DefaultLeaseInSeconds)

Each of these services already exposes a configurable lease (LeaderLease, toml key lease, default constant.DefaultLease = 5), used for the primary/leader election. The registry lease, however, stays fixed and ignores it.

Problem

When the configured leader lease is larger than the fixed registry lease (config lease > DefaultLeaseInSeconds), the two leases expire on different timescales. Under network jitter or etcd pressure the shorter registry lease can fail/expire first, while the longer leader/primary lease is still held.

As a result, the PD server evicts the node whose registry lease has lapsed even though it still legitimately holds its primary lease. This tears down a healthy primary, causing the primary lease to fail — a self-inflicted failover that would not happen if the registry lease were at least as long as the configured lease.

Proposal

Reuse the configured leader lease as the registry lease TTL, with discovery.DefaultLeaseInSeconds as a lower bound so the registry entry never expires faster than the current default:

  • Add GetLeaderLease() int64 to the server interface in pkg/mcs/utils/util.go.
  • In Register, use lease := max(s.GetLeaderLease(), discovery.DefaultLeaseInSeconds).
  • Each service returns its configured lease; the Router service has no lease config and returns 0, so it falls back to the default.

Scope

  • Only the register lease path is affected. The primary/leader election lease is unchanged.
  • Backward compatible: all lease-bearing configs default to 5, so max(5, 5) = 5 keeps current behavior; only an explicitly configured larger lease changes the registry TTL.

Metadata

Metadata

Assignees

No one assigned

    Labels

    type/enhancementThe issue or PR belongs to an enhancement.

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions