PropEr: union of #{} (empty-map) and a record fails JSON roundtrip — record decodes as #{}

[andreashasse]

What PropEr found

prop_json_roundtrip_from_data:prop_json_encode_decode_roundtrip/0 fails on a union of an empty-map type (#{}) and a record. When the record's JSON is decoded, the union picks the empty-map branch first — which accepts any map and silently drops all keys — so the record decodes back to #{} instead of the record. The encode→decode→encode roundtrip is not idempotent.

This is not flakiness: the bug is real and deterministic for the offending type. PropEr only surfaces it non-deterministically because the triggering type/value is randomly generated and rebar3 proper runs with no fixed seed.

Surfaced in CI

GitHub Actions run (attempt 1, all three OTP jobs failed): https://github.com/andreashasse/spectra/actions/runs/27403079104

Per-job logs (attempt 1):

• test (27, 3.25): https://github.com/andreashasse/spectra/actions/runs/27403079104/job/80985493549
• test (28, 3.25): https://github.com/andreashasse/spectra/actions/runs/27403079104/job/80985493606
• test (29, 3.25): https://github.com/andreashasse/spectra/actions/runs/27403079104/job/80985493553

A re-run of the same commit passed (different random seed), which is consistent with "real bug, rarely generated" rather than "non-deterministic codec".

Shrunk failing case (from the log)

Type:     {sp_union,
              [{sp_map,[],undefined,#{}},
               {sp_rec,atom4,
                   [{sp_rec_field,atom1,<<"atom1">>,
                        {sp_map,[],undefined,#{}},
                        undefined}],
                   2,#{}}],
              #{}}
Data in:  {atom4,#{}}      %% the record #atom4{atom1 = #{}}
Json:     {"atom1":{}}
Data out: #{}              %% decoded as empty map, NOT the record
Json2:    {}

The type is #{} | #atom4{atom1 :: #{}}. The record value encodes to {"atom1":{}}, but on decode the #{} branch matches first and discards the "atom1" key, producing #{}.

Root cause / history

The empty-map type #{} currently accepts any map on decode and silently drops extra keys. In a union it therefore greedily matches a record's (or any non-empty map's) JSON before the more specific branch is tried — incorrect union disambiguation.

This exact behavior was previously fixed and then reverted (see CHANGELOG):

• 0.11.4 — Fixed: "#{} (empty-map type) in JSON encode/decode now correctly rejects non-empty maps. Previously it silently accepted and discarded all keys, which caused incorrect union disambiguation — for example, a record's JSON object could decode as #{} instead of the record."
• 0.12.1 — Fixed: "#{} (empty-map type) in JSON encode/decode now accepts any map input again — additional fields are silently dropped... The overly strict rejection introduced in 0.12.0 caused valid use-cases (unconstrained map types) to fail unexpectedly."

So 0.12.1 reintroduced the union-disambiguation problem that 0.11.4 fixed. The tension is that a bare #{} is used both as "the empty map" and as "an unconstrained map" — the two want opposite decode semantics, especially inside unions.

Suggested follow-up

1. Add a deterministic unit test reproducing #{} | #record{} (and #{} | #{...fields...}) roundtrips, per the project guideline of turning PropEr findings into unit tests.
2. Decide the intended semantics for #{} inside a union (e.g. only accept a genuinely empty object for the #{} branch, while keeping unconstrained-map use-cases working through a distinct representation), so union disambiguation is correct without re-breaking the 0.12.1 use-cases.
3. Pin a PropEr seed in CI (or record/replay counterexamples) so this is reproducible rather than seed-dependent.

Found while working on #181 (OpenAPI security schemes); that PR is unrelated to this bug — it only touches spectra_openapi.