Snapshot reference auto-resolution silently overrides other selectors in the same `selectors` array

[hereicq]

Expected behavior

Per the [reference doc](https://learn.microsoft.com/en-us/azure/azure-app-configuration/reference-kubernetes-provider#key-value-selection): "the key-values of the last selector take precedence and override any overlapping keys from the previous selectors."

The doc explicitly demonstrates this with a snapshotName selector followed by a keyFilter selector that overrides some of the snapshot's values:

configuration:
  selectors:
    - snapshotName: app1_common_configuration
    - keyFilter: app1*
      labelFilter: development

"In the following sample, you load key-values of common configuration from a snapshot and then override some of them with key-values for development."

A reasonable user expectation, given the v2.6.0 introduction of snapshot references (auto-resolved via the snapshot-ref content type), is that snapshot references participate in the chain identically to snapshotName selectors — so a later selector can override values pulled in via a reference.

Actual behavior

A snapshot reference resolved via configuration.selectors always wins over any other selector in the chain that returns the same App Config key, regardless of:

• Selector order (placing the snapshot-ref selector first vs. last produces the same result).
• Override pattern (same-key in live store + co-located key-filter selector).
• Prefix-trim pattern (live/<key> in live store + trimKeyPrefixes: ["live/"] on the CR).

The override is silently lost — no warning, status stays Complete.

Repro summary

Created an App Configuration store containing:

• operational/foo = "baseline" (label test)
• .appconfig.featureflag/PocFeatureA (label test)
• A snapshot capturing both above
• system/active-snapshot (label test) — snapshot-ref content-type, points at the snapshot
• Overwrote operational/foo = "override-A" in the live store after snapshot creation (snapshot's frozen value still "baseline")

Applied the following CR:

configuration:
  selectors:
    - keyFilter: "system/active-snapshot"   # 1st: snapshot reference, resolves to baseline
      labelFilter: test
    - keyFilter: "operational/*"             # 2nd: direct read from live store, returns override-A
      labelFilter: test

Expected ConfigMap (per the doc's last-wins contract): operational/foo = "override-A".

Actual ConfigMap: operational/foo = "baseline" (snapshot wins).

Swapping selector order has no effect. Trying the live/ + trimKeyPrefixes pattern also fails. Replacing the snapshot-ref selector with a snapshotName selector immediately makes the override win — confirming the behavior is specific to snapshot references.

Root cause (source-code citation)

internal/loader/configuration_setting_loader.go, CreateKeyValueSettings:

1. Lines 216–253: the selector chain runs. processSettings classifies entries:
   • Plain key-values → rawSettings.KeyValueSettings (last-wins within the chain).
   • Snapshot references → collected into processCtx.snapshotRefs, not merged into KeyValueSettings during chain processing.
2. Lines 258–265: after the chain, resolveSnapshotReferences runs. It loads each referenced snapshot and calls processSettings again on the snapshot's contents, reusing the same rawSettings.KeyValueSettings map (line 405). The snapshot's entries overwrite anything matching keys from the chain.

So the in-chain last-wins semantic is correct between non-reference selectors, but snapshot reference resolution always wins on key collision because it runs separately and last.

Why this matters for design

The runtime-flippable property of snapshot references is appealing for designs that want a stable baseline + per-key live overrides — exactly the pattern the doc's snapshotName example demonstrates. With snapshot references unable to participate in the override pattern, the design choice becomes:

• Use snapshotName pinning, gain override semantics, lose runtime-flip (snapshot promotion now requires CR re-apply per cluster).
• Use snapshot references, gain runtime-flip, lose the ability to do per-key overrides outside the snapshot.

Either way, you lose one of the two properties that make App Configuration attractive for runtime-config-platform use cases.

Suggested resolutions (pick one)

1. Treat snapshot references as in-chain selectors. Resolve them during processSettings rather than post-chain, so last-wins applies between them and key-filter selectors. Most user-friendly and consistent with snapshotName behavior.
2. Add a CR property to control precedence. E.g., configuration.snapshotReferencePosition: in-chain | post-chain (default to current behavior to avoid breaking changes; advertise in-chain as the recommended setting for new designs).
3. Document the current behavior prominently. Add a callout to the [reference docs](https://learn.microsoft.com/en-us/azure/azure-app-configuration/reference-kubernetes-provider#snapshot-reference) explaining that snapshot references are post-chain-resolved and always win over other selectors. Users can then design around it.

[linglingye001]

Hi @hereicq, thanks for the detailed write-up, I've reproduced the scenario on my end and can confirm this is a bug: snapshot references resolved from selectors should participate in the in-chain last-wins ordering, just like snapshotName selectors do today.

We'll work on a fix to keep behavior consistent with the document.