Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,19 @@ Notable changes to OLAF — OneLake Access Framework — are recorded here using
cannot see, the five exceptions that defeat it, and what `rebuild=True` costs.
- `docs/api/Deployment.md` was headed `generate(rebuild=False)` and never said what the parameter
did. It does now.
- `docs/runbook.md` had no entry for the operational case an estate using a table glob will meet
first — a new table matching `sales.*` that is not granted, after a run that reported success.
It now has 3i, covering why the skip holds, what `rebuild=True` re-resolves and drops, the
`exclude_tables` hold-back, the stale converse when a table is deleted, and the unrelated
conditions that force a re-resolution as a side effect. The word `rebuild` previously appeared
in the runbook **only** in 3e — `setup(rebuild=True)`, which drops a control table and loses its
data — so an operator searching for it found the destructive one and nothing else. 3e now says
so and points at 3i.
- `notebooks/olaf_cookbook.ipynb` told readers that re-running an unchanged config is a no-op
without saying that a changed **catalog** is also a no-op, which is the half that surprises
people. It now says so and points at RUNBOOK 3i.
- `docs/config-examples.md`'s CLS whitelist example said the difference "only shows up on a later
generation" — the same ambiguity 7a had just lost two sections earlier.

## [1.1.0] - 2026-08-27

Expand Down
7 changes: 4 additions & 3 deletions docs/config-examples.md
Original file line number Diff line number Diff line change
Expand Up @@ -247,9 +247,10 @@ lists them.

**Takeaway:** `generate` computes `visible_columns` as `include_columns` directly (the columns that
exist on the table), so today's result matches 7a's resolved allow-list exactly. The difference
only shows up on a later generation: whitelist mode does not add a new column until
the author explicitly includes it (unlike 7a's blacklist, where regeneration adds a
new non-excluded column to OLAF's allow-list). Setting both `include_columns` and
only shows up on a later generation that actually rebuilds: whitelist mode does not add
a new column until the author explicitly includes it (unlike 7a's blacklist, where a real
regeneration adds a new non-excluded column to OLAF's allow-list — and, like every other
catalog change, an unchanged config skips instead). Setting both `include_columns` and
`exclude_columns` on the same row is an error (E12 below) — pick one CLS mode per row.

### 7c. RLS + CLS for one member — one role, never two (rule C5)
Expand Down
44 changes: 44 additions & 0 deletions docs/runbook.md
Original file line number Diff line number Diff line change
Expand Up @@ -220,6 +220,11 @@ capture a recovery point, run health and the external access review, supply a fr
attestation, then review the rebuild result. Never put a real table sample in a public
issue or pull request.

**Not the same `rebuild` as the one in 3i.** `setup(rebuild=True)` DROPS and recreates a
control table and loses its data; `generate(rebuild=True)` only re-resolves the config
and writes a new mapping. They share a parameter name and nothing else. If you arrived
here searching for `rebuild` because a new table is missing from a role, you want 3i.

## 3f. `Files/security/` grows without bound — what to prune, and what not to

Define an organization-owned retention policy for imported workbooks, review CSVs,
Expand Down Expand Up @@ -256,6 +261,45 @@ containment. It preserves an operation/incident sentinel and reports remaining
tables/files. It must return that exposure remediation is not proved. It cannot erase
prior readers, Delta history, caches, copies, exports, or external logs.

## 3i. A new table matches an existing glob but is not granted — `generate(rebuild=True)`

A role scoped `sales.*` covers the tables that existed when its mapping was generated.
A table created since then is absent from the mapping, so no grant on it is submitted,
even though every run since has reported success.

This is the idempotency skip, not a fault. `generate(rebuild=False)` — the default —
compares `config_hash`, a fingerprint of the config rows, and rebuilds nothing when it
matches. That fingerprint cannot see the catalog, so a table appearing in a lakehouse
never changes it. Treat the behaviour as intended: a table arriving in storage is not
by itself a decision to share it, and the run that shares it should be one somebody
asked for.

To take it in, re-run generate with `rebuild=True`, then plan and review as normal
before any apply. **`rebuild=True` re-resolves every pattern in the config**, so it also
takes in every other table that has appeared since the last real generate, and drops any
table that has since been deleted. Read the plan before approving it; the set that
arrives is rarely only the table you had in mind. To hold one back deliberately, name it
in `exclude_tables` — the table must already exist in the catalog, because a zero-match
exclude is an error (rule A2).

Distinguish this from `setup(rebuild=True)` in 3e, which drops and recreates a control
table and loses data. This one writes a mapping and nothing else.

A table OLAF has not granted is not a table OLAF has denied. Whether anyone can read it
still depends on workspace and item permissions and the access path, so do not treat a
missing grant as containment. If it must be unreadable, confirm that separately.

The same skip leaves the converse stale: a table deleted from the lakehouse remains in
the mapping until a real generate, and apply will keep submitting a role that names a
path which no longer exists.

Several unrelated conditions also force a full re-resolution with no parameter — a
member pattern in the config, drifted member objectIds, a mapping stamped for another
target, a member table in an error state, and a framework-version change. Any of them
brings in new catalog matches as a side effect, so a run made for one of those reasons
can widen scope without an accompanying config edit. This is another reason to read the
plan rather than the config diff.

## 4. Test / CI gate

Local CI uses synthetic fixtures and no live Fabric tenant:
Expand Down
5 changes: 4 additions & 1 deletion notebooks/olaf_cookbook.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -173,7 +173,10 @@
"#\n",
"# When:\n",
"# after every config edit — always before plan/apply. rebuild=False is idempotent, so\n",
"# re-running an unchanged config is a no-op (status \"skipped\").\n",
"# re-running an unchanged config is a no-op (status \"skipped\"). The skip compares the\n",
"# CONFIG only, so a table added to the catalog since the last generate does not change\n",
"# it: a new table matching an existing glob stays out of the mapping, and ungranted,\n",
"# until rebuild=True. See RUNBOOK 3i.\n",
"# Expect:\n",
"# status \"success\" with the grants/roles/warnings counts (or \"skipped\" if unchanged),\n",
"# plus a new versioned CSV directly under mapping_history_dir (default Files/security/mapping-history).\n",
Expand Down