Dependabot gomod: per-directory entries break the gen/go ↔ remote-worker replace coupling

[pdettori]

Problem

Dependabot's gomod config declares two separate entries, one per directory:

  - package-ecosystem: gomod
    directory: /gen/go
    groups:
      grpc-protobuf: ...

  - package-ecosystem: gomod
    directory: /remote-worker
    groups:
      grpc-protobuf: ...

Each directory: entry is an independent update universe, so Dependabot opens
one PR per module. But the two modules are coupled:

• remote-worker/go.mod has
  replace github.com/kagenti/serverless-harness/gen/go => ../gen/go
• the proto job in .github/workflows/ci.yml builds both modules in the
  same job (Build and test Go stubs + Build and test remote-worker)

So any bump to a gen/go dependency makes remote-worker/go.mod stale, and CI
fails with:

go: updates to go.mod needed; to update it:
	go mod tidy

Dependabot cannot fix this itself, because the tidy has to land in the same
commit as the gen/go bump — and its branches are not writable by maintainers.

Observed

First occurrence, on the very first pair of PRs produced after #208 enabled
gomod coverage:

• chore(deps): bump the grpc-protobuf group in /gen/go with 2 updates #210 (/gen/go: grpc 1.83.2, protobuf 1.36.12) — proto check failed
• chore(deps): bump google.golang.org/grpc from 1.83.1 to 1.83.2 in /remote-worker in the grpc-protobuf group #209 (/remote-worker: grpc 1.83.2) — green, but incomplete on its own
• Resolved manually by bundling both into chore(deps): bump grpc to 1.83.2 and protobuf to 1.36.12 across both Go modules #211

Merge order does not rescue it: #209 leaves protobuf v1.36.11 // indirect,
which #210 then invalidates, so #210 fails even after a rebase.

Proposed fix

Collapse the two entries into one using the plural directories: key, so a
single PR spans both modules and the grpc-protobuf group applies across them:

  - package-ecosystem: gomod
    directories:
      - /gen/go
      - /remote-worker
    schedule:
      interval: weekly
    groups:
      grpc-protobuf:
        patterns:
          - 'google.golang.org/grpc*'
          - 'google.golang.org/protobuf*'
          - 'google.golang.org/genproto*'

Worth verifying that Dependabot runs go mod tidy in the dependent module when
a replace target moves within the same PR. If it does not, the alternative is a
Go workspace (go.work) covering both modules, or a small CI autofix step that
runs go mod tidy and commits.

Impact if unfixed

Every future gen/go dependency bump produces a red PR needing manual bundling.
Given the weekly schedule and the grpc-protobuf group's churn, that is roughly
a recurring weekly chore.

Assisted-By: Claude Code

[pdettori]

Research findings

I reproduced the failure deterministically and tested the two candidate fixes locally.
Summary up front: the proposed directories: fix does not work, and go.work does —
verified, not inferred.

1. Reproduction

Reconstructed PR #210's exact tree (gen/go at grpc 1.83.2 / protobuf 1.36.12 from
main, remote-worker/{go.mod,go.sum} at the pre-#211 commit b1af7bb):

$ cd remote-worker && go vet ./...
go: updates to go.mod needed; to update it:
	go mod tidy
exit 1

Same error CI reported. Good deterministic test bed for any fix.

2. The proposed directories: fix is insufficient

Per the Dependabot options reference,
plural directories: does not by itself bundle directories into one PR — it is only a
shorthand (plus glob support) that still yields one PR per directory. Spanning directories
requires groups.<name>.group-by: dependency-name, which carries three limitations that
each break this use case:

Limitation (verbatim from docs)	Consequence here
"Dependabot will create a single pull request for each dependency update across all specified directories"	One PR per dependency, not per group — the deliberate grpc-protobuf cohesion is lost, and grpc/protobuf split back into separate PRs
"Applies to version updates only"	Security updates stay per-directory. Go CVEs were the entire reason for #208 (cf. CVE-2026-84304 in #203), so the worst case stays broken
"If directories have incompatible version constraints for a dependency, Dependabot will create separate pull requests"	Exactly our situation whenever the modules are momentarily out of sync

Two further problems, independent of the above:

• Whether patterns and group-by may coexist in one group is undocumented.
• remote-worker carries google.golang.org/protobuf as // indirect. Dependabot's
  default allow covers direct dependencies only, so a protobuf bump would not touch
  remote-worker at all, regardless of directory config.

3. go.work fixes it — verified

Against that same broken tree, adding a root go.work covering both modules:

Check (unchanged stale remote-worker/go.mod)	Before	With go.work
cd gen/go && go build ./... && go test ./...	pass	pass
cd remote-worker && go vet ./...	fail	pass
cd remote-worker && go test -race ./...	fail	pass
cd remote-worker && go mod tidy -diff	drift	drift still detected

In workspace mode the go command resolves one MVS across both modules, so a stale
consumer go.mod no longer breaks the build — while go mod tidy -diff still reports the
drift, so the hygiene signal is not lost. That makes CI green on any Dependabot layout,
which is the property directories: cannot give us.

flowchart TD
    subgraph now["Today — per-directory, no workspace"]
        A["PR: bump gen/go<br/>grpc 1.83.2, protobuf 1.36.12"] --> B["remote-worker/go.mod<br/>still pins protobuf 1.36.11"]
        B --> C["proto job:<br/>updates to go.mod needed"]:::bad
    end

    subgraph work["With root go.work"]
        D["PR: bump gen/go<br/>(any subset, any layout)"] --> E["workspace MVS spans<br/>gen/go + remote-worker"]
        E --> F["proto job green"]:::good
        E --> G["go mod tidy -diff<br/>still reports drift"]:::info
    end

    classDef bad fill:#ffdce0,stroke:#d1242f
    classDef good fill:#dafbe1,stroke:#1a7f37
    classDef info fill:#fff8c5,stroke:#9a6700

Loading

Implementation notes from testing:

• Pin the workspace toolchain: go work init writes the local toolchain line
  (go 1.26.5 on my machine). It must say go 1.25.0 to match both modules and CI's
  go-version-file: gen/go/go.mod.
• go.work.sum is optional — the checksum database still verifies downloads without it.
  Suggest .gitignore-ing it rather than adding a fourth sum file to keep in sync.
• One real gap: remote-worker/Dockerfile copies gen/go/ and remote-worker/ but not
  go.work, so that image build would still fail on a stale go.mod. It is not built in
  CI (build.yaml builds the root Dockerfile, sandbox.Dockerfile and echo-target
  only), so this is latent rather than breaking — fixed by one COPY go.work ./.

4. On the CI autofix alternative

Discounted. GITHUB_TOKEN is read-only on pull_request events raised by Dependabot, so
committing a tidy back to its branch needs a PAT with write access, triggered by dependency
PR content. For a repo that pins every action by digest and runs Scorecard, that trade seems
clearly wrong.

Proposed change

1. Add root go.work (go 1.25.0, use ./gen/go ./remote-worker); gitignore go.work.sum.
2. COPY go.work ./ in remote-worker/Dockerfile.
3. Collapse the two gomod entries to one plural directories: entry — not as the fix,
   but because it deduplicates the grpc-protobuf group block. Keeps behaviour identical
   (still one PR per directory), and go.work is what makes those PRs green.
4. Add go mod tidy -diff to the proto job as a hygiene check.

Question before I implement

On item 4 — should the tidy-drift check block or only warn?

• Blocking keeps both go.mod files honest, but turns Dependabot PRs red again on
  exactly the coupled bumps this issue is about, so the weekly chore returns in a new shape.
• Warning (annotation, non-fatal) keeps PRs green and lets drift be tidied whenever
  convenient. Since neither module is consumed externally — gen/go is reached only through
  the local replace — per-module tidiness is a hygiene property here, not a correctness one.

I lean warn, and would tidy opportunistically. Happy to go blocking if you would rather
the modules never drift.

Assisted-By: Claude Code