feat(rest)!: make SigV4 backend optional

[laskoviymishka]

Moves AWS SigV4 signing out of catalog/rest and into an optional catalog/rest/sigv4 backend, so plain REST users no longer pull in the AWS SDK. go list -deps ./catalog/rest drops from 595 packages to 510 and links no aws-sdk-go-v2.

This is a breaking change. Code that signs with SigV4 still compiles, but it now needs the backend to be imported, and the explicit-config entry point moved packages:

• Ambient or property credentials: keep WithSigV4 / WithSigV4RegionSvc (or the rest.sigv4-enabled property) and add a blank import of the backend, import _ "github.com/apache/iceberg-go/catalog/rest/sigv4". Without it, NewCatalog / catalog.Load return an error naming the import to add, rather than silently not signing.
• Explicit config: replace rest.WithAwsConfig(cfg) with sigv4.WithAwsConfig(cfg), still paired with WithSigV4 / WithSigV4RegionSvc; no blank import is needed in that case. The one-argument rest.WithAwsConfig(aws.Config) is removed.

WithSigV4 / WithSigV4RegionSvc stay the single source of the signing region and service, including any /v1/config overrides, so swapping the import path is the whole migration.

Rebased on #1999, whose property-credential signing is carried into the backend: the registered signer builds a static credentials provider from the s3.* keys (falling back to the rest.* aliases) and otherwise uses the AWS default chain. The cross-origin signing guard stays in core, where it is signer-agnostic and now also covers a signer installed via WithSigner.

Follows the driver pattern established for cloud IO in #696, and is somewhat related to #2044.

[zeroshade]

Thanks for this. The split achieves its main goal: go list -deps ./catalog/rest no longer includes any aws-sdk package, and go build ./..., go vet, and go test -race ./catalog/rest/... all pass. Requesting changes because sigv4.WithAwsConfig changes how signing region/service are resolved, and the most natural migration compiles but signs with the wrong scope.

Blocking

1. sigv4.WithAwsConfig bypasses WithSigV4RegionSvc and server-provided signing overrides

WithAwsConfig(cfg, region, service) builds the signer with region/service fixed at option-construction time and installs it via WithSigner. resolveSigner returns opts.signer before it ever consults opts.sigv4Region / opts.sigv4Service. On main, createSession applied both to the explicit aws.Config.

Observed with a scratch test against this branch (credential scope from the Authorization header):

Setup	Scope signed	main
rest.WithSigV4RegionSvc("us-west-2","s3tables") + sigv4.WithAwsConfig(cfg{Region: eu-central-1}, "", "")	eu-central-1/execute-api	us-west-2/s3tables
sigv4.WithAwsConfig(cfg, "us-east-1", "execute-api") + /v1/config override rest.signing-region=ap-south-1, rest.signing-name=s3tables	us-east-1/execute-api	ap-south-1/s3tables

The first row is what existing callers will write after migrating: keep WithSigV4RegionSvc, swap rest.WithAwsConfig(cfg) for sigv4.WithAwsConfig(cfg, "", ""). It compiles, and the server then rejects the requests (403) because they are signed for the wrong service.

2. The client.region hint for S3 is lost (from reading the code, not run)

toProps (rest.go ~763-777) only emits rest.sigv4-enabled, rest.signing-*, and the client.region fallback when o.enableSigv4 is set. Using sigv4.WithAwsConfig(cfg, "us-east-1", "s3tables") alone never sets it. r.props is cloned into every loaded table's config, so S3 Tables users lose the region hint for table FileIO.

Suggested fix

Hand the backend a factory rather than a pre-built signer:

• Add rest.WithSignerFactory(SignerFactory).
• In resolveSigner, when enableSigv4 is set, use opts.signerFactory if present, else the registry, and build with SignerConfig{Region: opts.sigv4Region, Service: opts.sigv4Service}. Server overrides are already merged into opts by fetchConfig at that point.
• Keep the original one-argument sigv4.WithAwsConfig(cfg aws.Config), returning rest.WithSignerFactory(...) that uses cfg instead of LoadDefaultConfig.

Migration then really is just changing the import path. WithSigV4 / WithSigV4RegionSvc stay the single source of region/service, and toProps behaves as it does today. Please also decide (and document) whether a raw WithSigner should bypass those settings.

Non-blocking

• Behavior change: on main, WithAwsConfig alone did not sign; WithSigV4 was also required. sigv4.WithAwsConfig now always signs. The fix above removes the difference; if the current design stays, please call it out in the description and configuration.md.
• Runtime break for catalog.Load users: programs loading a REST catalog with rest.sigv4-enabled=true (or sigv4-enabled in .iceberg-go.yaml) now fail at runtime until they add the blank import. The error message is good, but this belongs in the release notes next to the WithAwsConfig removal.
• Weaker concurrency coverage: the removed TestSigv4ConcurrentSigners sent 1 KiB POSTs through one shared catalog from many goroutines. TestConcurrentSignedCatalogRequests builds a catalog per goroutine, so the signer/transport are never shared, and TestSignRequestConcurrent only signs body-less GETs, so body hashing never runs concurrently. Suggest making TestSignRequestConcurrent sign POSTs with bodies on one shared signer.
• Low-value tests in rest_internal_test.go: TestSessionTransportNoSignerDoesNotSign asserts that core doesn't set x-amz-content-sha256, but core has no code that could set it, so it can't fail; suggest deleting it. TestSessionTransportConcurrentRoundTrip only counts calls on a fake signer.
• TestSigV4WithoutBackendReturnsHelpfulError: the registry save/restore is a no-op. The internal rest test binary can't import sigv4 (import cycle), so the registry is always empty there. Dropping it lets the test run with t.Parallel().
• RegisterSigner(name, nil) is accepted and only panics later in resolveSigner; suggest rejecting nil at registration.
• Unclosed catalogs: catalogs created in sigv4_internal_test.go are never closed; add t.Cleanup(func() { _ = cat.Close() }).
• Doc nit: configuration.md: "wires an rest.WithSigner" → "a".

Looks good

• catalog/rest no longer links the AWS SDK.
• The new GetBody == nil guard turns a nil-func panic into a clear error.
• The logic that closes the cloned body and reports close errors was carried over correctly, with its tests.
• The CLI blank import adds no weight, since catalog/glue already pulls in the SDK.
• The execute-api fallback for the properties-only path matches WithSigV4.

[zeroshade]

Both blocking items from my 09-28 review are addressed. sigv4.WithAwsConfig now goes through WithSignerFactory, resolveSigner builds SignerConfig from the opts after fetchConfig, and toProps behaves as on main again. TestWithAwsConfigUsesConfiguredRegionService and TestSignerFactoryReceivesResolvedRegionService pin this.
I merged the branch into current main locally: go build ./..., go vet, and the catalog/rest, catalog/rest/sigv4 (-race) and cmd/iceberg tests pass, and go list -deps ./catalog/rest has no aws-sdk packages (595 -> 510).
Before merge:

• Retitle to feat(rest)!: and rewrite the description as release-note migration text: rest.WithAwsConfig(cfg) -> sigv4.WithAwsConfig(cfg), still paired with WithSigV4*. Code using WithSigV4 / WithSigV4RegionSvc / rest.sigv4-enabled still compiles, but NewCatalog / catalog.Load now return an error until catalog/rest/sigv4 is imported.
• Sequencing: #1999 merges first, then this PR rebases on it. Add Props iceberg.Properties (the post-override opts.additionalProps) to SignerConfig, move staticCredsFromProps into catalog/rest/sigv4, and keep the signingOrigin guard in core RoundTrip. It needs no SDK and will then cover WithSigner too. Port the #1999 tests along with it. cc @iremcaginyurtturk
• Hit Update branch so CI runs with the modernize linter (#2072) and the Go 1.26 floor (#2077); the last run predates both.
  The inline comments are nits.

[zeroshade]

Nothing pins this precedence. No test calls WithSigner, so this first branch is never exercised through NewCatalog. Nothing asserts that WithSignerFactory on its own leaves requests unsigned either, and that is the main-parity property the factory design relies on (on main, WithAwsConfig without WithSigV4 did not sign). Please add one table-driven test over resolveSigner:

• WithSigner, SigV4 off: returns that signer
• factory, SigV4 off: returns nil
• factory, SigV4 on: calls the factory with the opts region/service
• no factory, SigV4 on, no backend registered: returns the import error

[zeroshade]

resolveSigner only ever looks up SignerNameSigV4 (line 113), so RegisterSigner("anything-else", f) is accepted and never consulted. The name parameter plus an exported constant read like a multi-scheme registry that does not exist. The doc is also off: rest.sigv4-enabled carries "true", not this name. Since this becomes public API, either drop the name (e.g. RegisterSigV4(factory SignerFactory)) and unexport the constant, or keep the shape and state on RegisterSigner that only SignerNameSigV4 is consulted.

[zeroshade]

These lines describe an intermediate state of this PR (a pre-built signer / three-argument WithAwsConfig) that never shipped, so they will not mean anything to readers on main. Please drop them, and the matching sentence in rest_internal_test.go:1890-1892; keep only what the test asserts. Also lines 266-267 below: TestConcurrentSignedCatalogRequests does not restore the old TestSigv4ConcurrentSigners coverage. It builds one catalog per goroutine and sends only body-less GET /v1/config. Shared-signer body hashing is now covered by TestSignRequestConcurrent, so reword or drop that claim.

[zeroshade]

WithSignerFactory is missing here. It is the hook sigv4.WithAwsConfig is built on, and the one to use for a custom signer that should keep following the signing region/service.

Suggested change

	| AWS SigV4 | `WithSigV4`, `WithSigV4RegionSvc`, `WithSigner` |
	| AWS SigV4 | `WithSigV4`, `WithSigV4RegionSvc`, `WithSignerFactory`, `WithSigner` |