Skip to content

[BUG] hauler store copy silently drops cosign signature artifacts for images stored by digest #667

Description

@Ansible-man

Environmental Info:
Target Registry: Harbor
Hauler Version:
GitVersion: v2.0.1
GitCommit: 4f47155
GitTreeState: clean
BuildDate: 2026-06-23T16:02:17Z
GoVersion: go1.26.4-X:boringcrypto
Compiler: gc
Platform: linux/amd64
Describe the Bug:

When an image is added to the store using a tag@digest (or digest-only) reference, hauler store sync correctly verifies the cosign signature and pulls the signature artifact into the store alongside the image. However, hauler store copy registry://... fails to rewrite the signature artifact's reference for the target registry, logs a WRN, skips the signature, and still reports overall success. The image lands in the target registry but its cosign signature is silently dropped

The malformed reference in the warning shows the rewrite appending the signature artifact's digest to a ref that already ends in the image's digest, after apparently truncating the original digest at the : separator:

myorg/myimage@sha256  +  @sha256:<sig-artifact-digest>
= myorg/myimage@sha256@sha256:<sig-artifact-digest>

This looks like the copy ref-mapper (the toMapper passed to Layout.CopyAll, which relocates refs via pkg/reference.Relocate) is stripping a "tag" by splitting on :, which mishandles refs whose stored form is repo@sha256:<digest> (no tag) leaving a dangling @sha256 before the sig digest is appended.

Two problems:

  1. Ref construction is wrong for signature artifacts attached to digest-referenced images.
  2. A dropped artifact is only a WRN copy exits successfully with copied artifacts to , so in CI the signature loss goes completely unnoticed. Given the entire point of carrying signatures is supply-chain assurance in the airgap, this should arguably be a hard error (or at minimum a non-zero exit).
    Note: the tag from the manifest (tag@digest form) is also not preserved on ingest the store entry becomes digest-only which may be what forces the rewrite down the broken digest-only path.

Steps to Reproduce:

hauler-manifest.yaml (image signed with a cosign key pair):

apiVersion: content.hauler.cattle.io/v1
kind: Images
metadata:
  name: example-images
  annotations:
    hauler.dev/key: cosign.pub
spec:
  images:
    - name: registry.example.com/myorg/myimage:v1.0.2@sha256:........
hauler store sync -f hauler-manifest.yaml
hauler store copy registry://registry.example.com/test

Expected Behavior:

  • The cosign signature artifact pulled during sync is copied to the target registry alongside the image (e.g. as sha256-<image-digest>.sig or via OCI 1.1 referrers), so cosign verify --key cosign.pub <target-ref> succeeds in the airgap.
  • If any artifact in the store cannot be copied, hauler store copy fails (non-zero exit) rather than warning and reporting success.
    Actual Behavior:

Sync (works signature verified and pulled)

Copy (image pushed, signature dropped with a warning, exit success) - Logs redacted

hauler store copy registry://registry.example.com/test:

2026-07-14 21:37:41 INF example-01/somecontainerimage@sha256:.....
2026-07-14 21:37:41 INF registry.example.com/test/example-01/somecontainerimage@sha256:.....: digest: sha256:..... size: 1671
2026-07-14 21:37:41 WRN failed to rewrite ref [example-01/somecontainerimage@sha256:.....]: parsing reference "example-01/somecontainerimage@sha256@sha256:.....": could not parse reference: example-01/somecontainerimage@sha256@sha256:.....
2026-07-14 21:37:41 INF copied artifacts to [registry.example.com/test]

Workaround: Adding the signature's tag only results in hauler working as expected

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Fields

No fields configured for issues without a type.

Projects

Status
Resolved

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions