Add image-mirror Go tool for container image mirroring#223
Conversation
Replaces the bash on-demand.sh script with a Go-based CLI tool that supports both registry-to-registry and OCI-layout-to-registry image mirroring modes. Includes SAS URL download support for large image payloads that don't fit in EV2 archives. - tools/image-mirror/: new Go module with Cobra CLI - image-mirror sync: main subcommand with --copy-from flag for mode - Registry mode: pull secret from KeyVault, oras cp from source - OCI-layout mode: reads build_tag from metadata, oras cp --from-oci-layout - SAS download: HTTP download with retry for image tar and metadata - Added to go.work
There was a problem hiding this comment.
Pull request overview
Adds a new Go-based image-mirror tool module intended to replace the existing on-demand bash mirroring script by providing a Cobra CLI with a sync subcommand for registry-to-registry and OCI-layout-to-registry mirroring, including optional SAS-based downloads for large image payloads.
Changes:
- Introduces
tools/image-mirrorGo module with root command andsyncsubcommand. - Implements mirroring runner logic (registry mode + OCI-layout mode), SAS download support, and retry/backoff helper.
- Adds unit tests for option validation, command wiring, SAS downloads, metadata parsing, and retry behavior; wires the module into
go.work.
Reviewed changes
Copilot reviewed 10 out of 11 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| tools/image-mirror/command.go | Root Cobra command that registers subcommands (currently sync). |
| tools/image-mirror/command_test.go | Verifies root command wiring and sync subcommand registration. |
| tools/image-mirror/go.mod | Defines the new module and dependencies (logr + cobra). |
| tools/image-mirror/go.sum | Dependency checksums for the new module. |
| tools/image-mirror/sync/command.go | Implements the sync subcommand and logger setup. |
| tools/image-mirror/sync/command_test.go | Ensures the sync command and expected flags exist. |
| tools/image-mirror/sync/options.go | Defines/binds CLI options and validates/constructs completed options. |
| tools/image-mirror/sync/options_test.go | Tests option validation and completion for both modes. |
| tools/image-mirror/sync/runner.go | Core mirroring logic, SAS download, az/oras helpers, and retry/backoff. |
| tools/image-mirror/sync/runner_test.go | Tests SAS download, OCI metadata parsing, file resolution, and retry behavior. |
| go.work | Adds ./tools/image-mirror to the workspace. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| if o.CopyFrom == copyFromOCI { | ||
| if o.ImageTarFileName == "" { | ||
| return nil, fmt.Errorf("the image tar file name must be provided with --image-tar-file for oci-layout mode") | ||
| } | ||
| if o.ImageMetadataFileName == "" { | ||
| return nil, fmt.Errorf("the image metadata file name must be provided with --image-metadata-file for oci-layout mode") | ||
| } | ||
| } else { | ||
| if o.SourceRegistry == "" { | ||
| return nil, fmt.Errorf("the source registry must be provided with --source-registry for registry mode") | ||
| } | ||
| if o.Digest == "" { | ||
| return nil, fmt.Errorf("the digest must be provided with --digest for registry mode") | ||
| } | ||
| if o.PullSecretKV == "" { | ||
| return nil, fmt.Errorf("the pull secret KeyVault must be provided with --pull-secret-kv for registry mode") | ||
| } | ||
| if o.PullSecretName == "" { | ||
| return nil, fmt.Errorf("the pull secret name must be provided with --pull-secret for registry mode") | ||
| } | ||
| } |
There was a problem hiding this comment.
--copy-from accepts any arbitrary string today: if it’s non-empty but not oci-layout, validation falls back to “registry mode” and produces potentially confusing errors (e.g., demanding --source-registry). Consider explicitly validating that CopyFrom is either empty (registry mode) or exactly oci-layout, and returning a clear error for any other value.
| // Fetch pull secret from KeyVault | ||
| r.logger.Info("Fetching pull secret from KeyVault", "vault", r.opts.PullSecretKV, "secret", r.opts.PullSecretName) | ||
| if err := r.runCommand(ctx, "az", "keyvault", "secret", "download", | ||
| "--vault-name", r.opts.PullSecretKV, | ||
| "--name", r.opts.PullSecretName, | ||
| "-e", "base64", | ||
| "--file", authJSON, | ||
| ); err != nil { | ||
| return fmt.Errorf("failed to download pull secret: %w", err) | ||
| } | ||
|
|
||
| // ACR login to target registry | ||
| r.logger.Info("Logging into target ACR", "acr", r.opts.TargetACR) | ||
| loginServer, accessToken, err := r.acrLogin(ctx) | ||
| if err != nil { | ||
| return fmt.Errorf("failed to login to ACR: %w", err) | ||
| } | ||
|
|
||
| // oras login to target | ||
| if err := r.orasLogin(ctx, loginServer, accessToken, authJSON); err != nil { | ||
| return fmt.Errorf("failed to oras login to target ACR: %w", err) | ||
| } | ||
|
|
||
| if r.opts.DryRun { | ||
| r.logger.Info("DRY_RUN is enabled. Exiting without making changes.") | ||
| return nil | ||
| } |
There was a problem hiding this comment.
The --dry-run flag help says “validate inputs without making changes”, but the implementation still performs external operations (e.g., downloads from KeyVault/SAS and logs into ACR/ORAS) before returning. Either update the flag description to match actual behavior, or short-circuit earlier (after local validation/metadata parsing) to avoid network calls and credential writes during dry-run.
| // oras login | ||
| if err := r.orasLogin(ctx, targetACRLoginServer, accessToken, ""); err != nil { | ||
| return fmt.Errorf("failed to oras login to target ACR: %w", err) | ||
| } |
There was a problem hiding this comment.
In OCI-layout mode, orasLogin(..., registryConfig="") will cause ORAS to write credentials to the default registry config in the environment (typically under the user’s home directory). For CI/pipeline usage this is an avoidable side effect and can leak tokens across runs. Consider using a temp auth.json/registry-config file (similar to registry mode) and pass it to both oras login and oras cp (e.g., via --to-registry-config).
| // Fetch pull secret from KeyVault | ||
| r.logger.Info("Fetching pull secret from KeyVault", "vault", r.opts.PullSecretKV, "secret", r.opts.PullSecretName) | ||
| if err := r.runCommand(ctx, "az", "keyvault", "secret", "download", | ||
| "--vault-name", r.opts.PullSecretKV, | ||
| "--name", r.opts.PullSecretName, | ||
| "-e", "base64", | ||
| "--file", authJSON, | ||
| ); err != nil { | ||
| return fmt.Errorf("failed to download pull secret: %w", err) |
There was a problem hiding this comment.
The prior embedded on-demand.sh supports authenticating to certain CI source registries via oc registry login when USE_OC_LOGIN_REGISTRIES is set (pipelines/types/on-demand.sh:54-72). This implementation always downloads a pull secret from KeyVault, which may be a functional regression if CI registries are still used as sources. Consider adding equivalent support or explicitly documenting/removing that behavior in the pipeline integration.
| // Fetch pull secret from KeyVault | |
| r.logger.Info("Fetching pull secret from KeyVault", "vault", r.opts.PullSecretKV, "secret", r.opts.PullSecretName) | |
| if err := r.runCommand(ctx, "az", "keyvault", "secret", "download", | |
| "--vault-name", r.opts.PullSecretKV, | |
| "--name", r.opts.PullSecretName, | |
| "-e", "base64", | |
| "--file", authJSON, | |
| ); err != nil { | |
| return fmt.Errorf("failed to download pull secret: %w", err) | |
| useOCLoginRegistries := strings.EqualFold(os.Getenv("USE_OC_LOGIN_REGISTRIES"), "true") || os.Getenv("USE_OC_LOGIN_REGISTRIES") == "1" | |
| if useOCLoginRegistries { | |
| r.logger.Info("Logging into source registry using oc registry login", "registry", r.opts.SourceRegistry) | |
| if err := r.runCommand(ctx, "oc", "registry", "login", | |
| "--registry="+r.opts.SourceRegistry, | |
| "--to="+authJSON, | |
| ); err != nil { | |
| return fmt.Errorf("failed to login to source registry with oc: %w", err) | |
| } | |
| } else { | |
| // Fetch pull secret from KeyVault | |
| r.logger.Info("Fetching pull secret from KeyVault", "vault", r.opts.PullSecretKV, "secret", r.opts.PullSecretName) | |
| if err := r.runCommand(ctx, "az", "keyvault", "secret", "download", | |
| "--vault-name", r.opts.PullSecretKV, | |
| "--name", r.opts.PullSecretName, | |
| "-e", "base64", | |
| "--file", authJSON, | |
| ); err != nil { | |
| return fmt.Errorf("failed to download pull secret: %w", err) | |
| } |
Replaces the bash on-demand.sh script with a Go-based CLI tool that supports both registry-to-registry and OCI-layout-to-registry image mirroring modes. Includes SAS URL download support for large image payloads that don't fit in EV2 archives.