[refactor] vpp-manager rework#853
Draft
sknat wants to merge 2 commits into
Draft
Conversation
sknat
force-pushed
the
refact-vpp-manager
branch
from
608e6ad to
2e2237b
Compare
aritrbas
reviewed
Dec 8, 2025
sknat
force-pushed
the
refact-vpp-manager
branch
3 times, most recently
from
0763f66 to
9d0a0d5
Compare
aritrbas
reviewed
May 29, 2026
aritrbas
reviewed
May 29, 2026
aritrbas
reviewed
May 29, 2026
- consolidate calicovpp/agent, calicovpp/vpp, calicovpp/multinet-monitor
and calicovpp/init-eks into a single unified calicovpp/vpp image.
- restructure source directories under pkg/ for cleaner organization.
- move executable entrypoints to top-level cmd/ to allow for building
all golang agents from the top-level Makefile.
- runtime behavior is unchanged - two separate daemons (agent and vpp)
run in separate containers from the same image using command overrides.
Changes:
- unified image: calicovpp/vpp (contains all binaries)
- single env var: CALICO_VPP_IMAGE (replaces CALICO_AGENT_IMAGE,
CALICO_VPP_IMAGE, MULTINET_MONITOR_IMAGE)
- unified dockerfiles: Dockerfile and Dockerfile.debug now contain all
binaries - calico-vpp-agent, felix-api-proxy, gobgp, multinet-monitor,
vpp-manager and init-eks scripts in images/ubuntu/bin/
- top-level Makefile delegates image build through vpp-manager
- vpp-manager Makefile builds all Go binaries into unified image
- image/push targets removed from agent and multinet-monitor Makefile
- manifests updated to calicovpp/vpp with explicit command overrides for
agent and multinet-monitor containers
- path references in test/ and docs/ updated for pkg/ move
- image contexts moved to top-level images/
- helper scripts moved to top-level scripts/
- VPP clone/build directory relocated to top-level vpp_build
Changes for VPP kube-test integration:
CalicoVPP now owns the kube-test manifest as a set of kustomize sources
so VPP kube-test (and any other consumer) does not need to know which
image to use, where the VPP build lives in this repo or any other
CalicoVPP-internal layout detail.
- yaml/components/kube-test/ - kustomize coimponent shared by the kind
and baremetal flavors. Defines the kube-test daemonset patch, the
kube-test configmap defaults and the tigera-operator snippet.
- yaml/components/kube-test/kind/ - KinD-flavor adjustments
(uplink eth0 via af_packet, no hugepages).
- yaml/components/kube-test/baremetal/ - baremetal-flavor adjustments
(DPDK uplink driven by ${CALICOVPP_INTERFACE}, hugepages, larger
CPU/buffer allocation, docker.io registry, IfNotPresent pull policy).
- yaml/overlays/kube-test-{kind,baremetal}/ - overlays wiring the
base daemonset and the kube-test components together.
- yaml/generated/calico-vpp-kubetest.yaml - regenerated from the
kind overlay; consumers see only the runtime placeholders
${CALICOVPP_VERSION}, ${HOME}, ${ADDITIONAL_VPP_CONFIG},
${CALICOVPP_ENABLE_MEMIF}, ${CALICO_NETWORK_CONFIG} and
${CALICOVPP_INTERFACE} (baremetal only).
- yaml/Makefile - regenerates the checked-in kubetest yaml for kind
and sanity-checks the baremetal overlay.
Top-level Makefile targets (the public API consumed by VPP kube-test):
- make vpp - delegates to pkg/vpp-manager with VPP_DIR/BASE pass-through;
consumers no longer need to know where VPP build sources live.
- make kube-test-template FLAVOR={kind,baremetal}
- runs `kubectl kustomize` on the requested overlay and appends
the tigera Installation/APIServer snippet (the snippet contains
the ${CALICO_NETWORK_CONFIG} placeholder, so it is not a
kustomize resource and is concatenated verbatim post-build).
- make kube-test-push-images
- pulls/tags/pushes each image in $KUBE_TEST_IMAGES (default:
calicovpp/vpp) into the local kind registry. Older split-image
release branches backport this target with ${KUBE_TEST_IMAGES}
as calicovpp/vpp calicovpp/agent calicovpp/multinet-monitor.
Images that remain separate: calicovpp/vclsidecar, calicovpp/ctl
Directory Restructure:
├── cmd
│ ├── api-proxy
│ ├── calico-vpp-agent
│ ├── calicovppctl
│ ├── multinet-monitor
│ ├── replay-trace
│ └── vpp-manager
├── common.mk
├── docs
├── images
├── Makefile
├── pkg
│ ├── calico-vpp-agent
│ ├── config
│ ├── multinet-monitor
│ ├── vpplink
│ └── vpp-manager
├── scripts
├── test
├── vpp_build
└── yaml
Benefits:
- Single image tag to manage across all dataplane components
- Reduced bandwidth and storage requirements
- No risk of image version drift between components
- Simplified CI/CD pipeline
- Easier version management and releases
Signed-off-by: Aritra Basu <aritrbas+gh@cisco.com>
aritrbas
force-pushed
the
refact-vpp-manager
branch
from
fa57744 to
ccd8feb
Compare
Restructure vpp-manager around an explicit VppRunner type and move shared configuration into `pkg/config` so vpp-manager and calico-vpp-agent reference a single source of truth. - `pkg/config` is now the canonical home for VRF constants (DefaultVRFIndex, PuntTableID, PodVRFIndex), interface state/spec, machine state, env-var helpers, PingCalicoVpp(), ClearVppManagerFiles() and a single CreateVppLink(socket, log) used by both vpp-manager and calico-vpp-agent. - `cmd/vpp-manager/main.go` is reduced to a thin entrypoint. Signal handling, uplink driver invocation and cleanup are centralized in `pkg/vpp-manager/vpp_runner.go` behind VppRunner, with deferred panic recovery that guarantees SIGINT to VPP and the VPP_ERRORED hook fires on any unexpected exit. - Replaced embedded `default_hook.sh` and bash shell-out with a native Go NetworkManagerHook (`pkg/vpp-manager/hooks`). User-provided CALICOVPP_HOOK_* scripts override; otherwise the native hook runs when `CALICOVPP_ENABLE_NETWORK_MANAGER_HOOK=true` (default), and is a no-op when disabled. - Removed SupportedUplinkDrivers auto-detect retry; uplink driver is chosen deterministically from the configured interface spec. - All operational loops over uplinks iterate `InterfacesByID` (ordered slice) instead of `Interfaces` (Go map) so template rendering and driver step ordering is deterministic run-to-run. Signed-off-by: Nathan Skrzypczak <nathan.skrzypczak@gmail.com> Signed-off-by: Aritra Basu <aritrbas+gh@cisco.com>
aritrbas
force-pushed
the
refact-vpp-manager
branch
from
ccd8feb to
c442364
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Structural refactor of
vpp-manager. No new networking features; the goal is to remove hidden global state, givevpp-managera testable lifecycle and unify configuration betweenvpp-managerandcalico-vpp-agent.1.
pkg/configis now the single source of truth:DefaultVRFIndex,PuntTableID,PodVRFIndex) moved out ofpkg/calico-vpp-agent/commonso both binaries reference identical values.CreateVppLink(socket, log)/CreateVppLinkInRetryLoop(socket, log, timeout, retry)now lives inpkg/config/utils.goand used by bothvpp-managerandcalico-vpp-agent.2.
vpp-managerlifecycle viaVppRunner:cmd/vpp-manager/main.goreduced to argument parsing andVppRunner.Run(ctx).pkg/vpp-manager/vpp_runner.goowns the entire lifecycle: signal handling, uplink driver invocation, VPP startup, hook orchestration and cleanup.deferrecovers panics, signals VPP and runs theVPP_ERROREDhook on any failure.3. Native Go hooks replace
default_hook.sh:pkg/vpp-manager/hooks/network_manager_hook.goimplements all five hook points in Go.CALICOVPP_HOOK_*script (highest)CALICOVPP_ENABLE_NETWORK_MANAGER_HOOK=true(default)systemd-networkd/NetworkManagerrestart with targeted Go logic.4. Deterministic uplink driver selection:
SupportedUplinkDriversauto-detect retry loop is removed. The configured driver is invoked directly.uplink/default.go::CreateMainVppInterfacenow resolves the interface viaSearchInterfaceWithTagfrom the VPP startup config.5. Deterministic interface iteration:
vpp-managernow iterateInterfacesByID(ordered slice) instead ofInterfaces(Go map) so template rendering, driverUpdateVppConfigFile/PreconfigureLinux/RestoreLinux, and hook per-interface processing produce identical results run-to-run.Behavior Changes:
vpp-managerexits and is restarted by the DaemonSet rather than silently iteratingvirtio → rdma → vmxnet3 → af_xdp → af_packet. Deployments that relied on auto-detection must setCALICOVPP_INTERFACES[*].vppDriver(or the legacyCALICOVPP_NATIVE_DRIVER) explicitly.IfConfigSavePathsave/reload fallback removed. The mechanism that persisted Linux interface state to a file and reloaded it when discovery failed at restart has been removed.NewLinuxInterfaceStatenow either succeeds or returns an error. Users who relied onifConfigSavePathshould remove that field from theirCalicoVppInitialConfig; failure to discover the configured uplink will now causevpp-managerto fail-fast and the DaemonSet to restart it (avoids booting against stale state).default_hook.shembedded fallback removed. WhenCALICOVPP_ENABLE_NETWORK_MANAGER_HOOK=falseand noCALICOVPP_HOOK_*user script is configured, the hook becomes a no-op (previously it ran the embedded bash script which always restartedsystemd-networkd/NetworkManager). Thepkg/config/default_hook.shfile is retained as a reference for users authoring their own hook scripts.vpp-managercontainer image no longer requiresbashfor hook execution.