Skip to content

Release v1.1.1 — survive a never-written DAR collection, and stop stranding the sentinel - #11

Merged
kengio merged 1 commit into
mainfrom
fix/fresh-collection-boundary
Sep 3, 2026
Merged

Release v1.1.1 — survive a never-written DAR collection, and stop stranding the sentinel#11
kengio merged 1 commit into
mainfrom
fix/fresh-collection-boundary

Conversation

@kengio

@kengio kengio commented Sep 3, 2026

Copy link
Copy Markdown
Owner

What broke

A lakehouse whose OneLake security has never been written answers every dataAccessRoles read with the implicit DefaultReader under a freshly minted id, while the collection ETag and the role's content stay identical. ControlBoundary.snapshot_from hashed the whole role, id included, so the two reads begin() takes a second apart never matched — and every first-ever generate on a new lakehouse was refused with:

DAR state changed after the approved snapshot; refused instead of refreshing authorization

Deterministically, and only there: a collection apply has written once carries stable ids, which is why estates that had already deployed never saw it. Observed on a customer estate's first pipeline run on 2026-09-02 (three consecutive reads, three ids, one ETag).

The refusal then stranded its own marker. begin() creates the sentinel and only then re-reads the collection; it raised before any lease existed, so release_unwritten_leases() had nothing to hand back. The next run met control-data incident sentinel already exists for an incident nobody had — and clear_incident() could not clear it, because it performs the same two-read compare.

What changes

Where Change
ControlBoundary.snapshot_from roles_digest covers each role's content (name, rules, members) and excludes the server-assigned id and per-role etag. The collection ETag compared beside it still catches a real write between the reads. .roles keeps the full payload.
ControlBoundary.begin A refusal at the post-creation re-read removes the sentinel this call created, then re-raises. A marker inherited from an outer lease is left for that lease.
run_mode Unwritten leases are released on the error path as well as on blocked. The unwind still stops at the first lease that authorized a write.
docs/control-data-security.md Says what the snapshot compares and what it deliberately ignores.
__version__ 1.1.0 → 1.1.1, with the CHANGELOG section. Patch release: no API or parameter changes, warnings unchanged.

Security consequence

The gate is not weakened. What a role grants is still fingerprinted byte for byte, and a concurrent write in the read→write window still rotates the collection ETag and fails the compare. What no longer counts as a change is the service re-minting its own bookkeeping ids. The sentinel's incident semantics are untouched: a lease that authorized a write keeps its marker on every path (test_release_unwritten_refuses_any_lease_that_reached_a_write, test_release_stops_at_the_first_lease_that_wrote).

Tests

  • New: a never-written-collection fake (id re-minted per read), the content-change-under-same-ETag refusal, the begin() cleanup and cleanup-failure branches, the nested-lease case, and the error-path release.
  • Changed: test_snapshot_is_immutable_and_reconfirmation_does_not_refresh_after_change asserted the stranded marker as expected behaviour; it now asserts the refusal alone.
  • pytest --cov: 100 % branch coverage holds. scripts/lint.sh and scripts/check_public_release.py tree . pass.

Release steps after merge

Tag with the sanitized maintainer identity per CONTRIBUTING (git tag -a v1.1.1 -m 'release: OLAF v1.1.1', reading the identity off v1.1.0); this PR does not create the tag.

…anding the sentinel

A lakehouse whose OneLake security has never been written answers every
dataAccessRoles read with the implicit DefaultReader under a freshly minted id,
while the collection ETag and the role's content stay identical.
ControlBoundary.snapshot_from hashed the whole role, id included, so the two
reads begin() takes a second apart never matched, and every first-ever generate
on a new lakehouse was refused with "DAR state changed after the approved
snapshot" — deterministically, and only there, because a collection apply has
written once carries stable ids. Observed on a customer estate's first pipeline
run on 2026-09-02.

That refusal then stranded its own marker: begin() had created the sentinel and
raised before any lease existed, so release_unwritten_leases() had nothing to
hand back, and the next run met "control-data incident sentinel already exists"
for an incident nobody had. clear_incident() performs the same two-read compare,
so it was blocked too.

- snapshot_from: the digest covers each role's content (name, rules, members)
  and excludes the server-assigned id and per-role etag; the collection ETag
  compared beside it still catches a real write between the reads. .roles keeps
  the full payload.
- begin(): a refusal at the post-creation re-read removes the sentinel this call
  created before re-raising; a marker inherited from an outer lease is left
  alone.
- run_mode: unwritten leases are released on the error path as well as on
  blocked; the unwind still stops at the first lease that authorized a write.
- docs/control-data-security.md says what the snapshot compares and ignores.
- tests: a never-written-collection fake, the content-change and cleanup
  branches, and the error-path release; the immutability test no longer pins
  the stranded marker as expected behaviour.

Patch release: no public API or parameter changes; the warnings a caller sees
are unchanged.
@kengio
kengio force-pushed the fix/fresh-collection-boundary branch from ea4964c to 4f4ebfb Compare September 3, 2026 08:02
@kengio
kengio merged commit c977d5d into main Sep 3, 2026
6 checks passed
@kengio
kengio deleted the fix/fresh-collection-boundary branch September 3, 2026 08:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant