Skip to content

Repository files navigation

OLAF — OneLake Access Framework. Arctic owl shield mark and wordmark.

Review every access change. Apply only what you approved.

tests Coverage: 100% against in-memory fixtures Latest release tag License: MIT Status: community Preview

Python 3.11, 3.12 and 3.13 Microsoft Fabric Runtime 1.3 / Spark 3.5 or newer Dependencies installed with --require-hashes Linted and formatted with ruff

Important

An independent community Preview for evaluation and development, not a production-ready security product. Its mutating path uses Microsoft's bulk Data Access Roles PUT, which Microsoft documents as Preview and not recommended for production.

OLAF — OneLake Access Framework is a plan → review → apply workflow for Microsoft Fabric OneLake data access roles. It is one self-contained Fabric notebook with an Excel-authored configuration, a generated mapping, a saved plan, and an audit trail.

OLAF is not affiliated with, endorsed by, sponsored by, or supported by Microsoft. The product names above are used only to describe interoperability. See Microsoft's Trademark and Brand Guidelines and the project's platform contract.

What is OneLake security?

OneLake security is Microsoft Fabric's native, data-plane access control: you define security roles on the data itself (a lakehouse's tables, folders, rows, columns) and OneLake enforces them everywhere — instead of re-implementing security per engine with T-SQL views, workspace boundaries, or report-side filters.

  • Role-based access — grant Read per table (/Tables/schema/table) or folder (/Files/...); users with no role see nothing (deny by default). Create and manage roles →
  • Row-level security (RLS) — SQL-like predicates per table per role (static values; the OneLake-supported operator subset — = <> > >= < <= IN NOT AND OR IS BLANK NULL TRUE FALSE, rule C9). A member of several roles gets the union of what they allow. Row-level security →
  • Column-level security (CLS) — hide specific columns per role. Column-level security →
  • Entra-native membership — role members are Entra users, security groups, or service principals; membership changes happen in Entra, no sync tables to maintain.
  • One definition, every engine — enforced across the SQL analytics endpoint, Spark, and Direct Lake semantic models (Power BI), so reports and notebooks see the same filtered data.
  • Granular write — separate ReadWrite roles for scoped write access (RLS/CLS don't apply there).

Worth knowing: roles carrying RLS/CLS are read-only; workspace Admins/Members/Contributors bypass OneLake security (it governs consumers); role definitions are per lakehouse.

Why OLAF?

OneLake security provides the enforcement layer. OLAF makes operating that layer reviewable, repeatable, and easier to govern when deployments grow beyond a handful of roles and tables.

What you need What OLAF does
Know what will change plan shows every role and grant that will be created, updated, or omitted from the submitted payload.
Prevent an unreviewed write apply requires a saved plan and refuses stale or drifted state.
Catch risky configuration early Validation blocks common RLS/CLS, predicate, casing, and multi-role exposure mistakes before deployment.
Keep the runtime simple One unchanged notebook serves every project; configuration and runtime parameters carry project intent.
Explain what happened Plan and apply are linked in the audit log with who, what, when, and the relevant configuration state.

Full rule-by-rule detail → docs/architecture.md.

What OLAF does

Need OLAF behavior
Review intended access generate resolves authored scopes and members; plan presents the desired-versus-live diff.
Refuse stale approval apply requires a saved plan for the same config and mapping generation and rechecks observable drift.
Fail closed around sensitive control data Every sensitive write requires the DAR snapshot gate and an operation sentinel; the per-run operator attestation is optional and only recorded, as attested or unknown.
Preserve recovery evidence Destructive paths record prepared intent and recovery pointers before a real DAR write.
Explain current state Read-only audit methods expose lineage, drift, coverage, and engine-explicit policy calculations.

OLAF validates configuration and request construction. Microsoft Fabric remains the enforcement system, and enforcement depends on the engine, access mode, workspace role, item permissions, and shortcuts. See the official engine and user access model and SQL endpoint enforcement guidance.

Before you start

Caution

OLAF stores principal identifiers and authorization/recovery state in control tables and under Files/security. Sensitive modes are disabled by default and require a clean DAR snapshot with an ETag. The per-run workspace isolation attestation is optional — OLAF records it as attested or unknown and never gates on it. Complete the external access review before uploading a real workbook. Start with Protecting OLAF control data.

  • Use Microsoft Fabric Runtime 1.3 / Spark 3.5 or newer, then verify the selected runtime and bundled package versions in your Fabric environment. Microsoft publishes the runtime lifecycle and OneLake security limitations.
  • Use a workspace Admin or Member identity for DAR edits and ensure the REST caller has OneLake.ReadWrite.All. Contributor does not edit DAR definitions. See workspace permissions and the bulk DAR authorization contract.
  • Treat the same-lakehouse layout as a trusted-administrator boundary, not cryptographic or transactional isolation. If that threat model is unacceptable, do not import real principal data or run sensitive modes in this Preview.

Quick start

OLAF runs inside a Fabric notebook; it is not a CLI.

  1. Import notebooks/olaf.ipynb and optionally the olaf_master_workflow.ipynb driver. See Getting the notebooks into Fabric.

  2. Attach the intended lakehouse and run the read-only health diagnostic before setup:

    OLAF.configure(lakehouse_name="SampleLakehouse")
    OLAF.health()
  3. Review and remove reserved-path DAR overlap, broad ReadAll access, dynamic default-reader access, unauthorized elevated workspace roles, shares, shortcuts, and automation that can reopen them. Record the review in your change system. Microsoft documents these access paths in the OneLake access model.

  4. Supply the review reference for this run, then create the control tables:

    OLAF.configure(
        lakehouse_name="SampleLakehouse",
        control_data_isolation_attestation="change-review-123",
    )
    OLAF.setup()
  5. Only now upload a sanitized copy of configs/onelake_security.xlsx under Files/security, replace every synthetic row, verify every Entra object ID, and import both sheets:

    OLAF.load_config("member", "Files/security/onelake_security.xlsx", "member")
    OLAF.load_config("config", "Files/security/onelake_security.xlsx", "config")
  6. Use a new attestation reference for each sensitive operation and review every result:

    OLAF.generate()
    OLAF.plan()
    OLAF.apply()

Do not treat a successful request or dry run as proof of propagation or enforcement. The release has fixture-based CI evidence only; it does not claim live verification for the release SHA. See Evidence status.

Modes at a glance

Mode Purpose Writes
setup Create or migrate the four control tables after the privacy gate. tables and audit
validate Run configuration validation without a mapping, log, file, backup, or DAR write. none
generate Resolve config and member data into a versioned mapping and review artifact. sensitive artifacts and audit
plan Compare desired and live DAR state and save the reviewed plan. sensitive audit
apply Submit the reviewed payload through the Preview bulk DAR endpoint. backup, audit, and DAR
rollback Restore a selected config version, then run the guarded deployment chain. config, artifacts, audit, and DAR
show Read-only live pivot. none
trace Read-only operational and drift summary. none

reset() and cleanup() are interactive destructive utilities. Reset is subject to the full sensitive-write gate. Cleanup is an emergency containment path that may run without attestation, preserves an incident sentinel, and cannot prove that prior disclosure was erased. See the mode manual and runbook.

Every mode returns one result envelope:

mode · status · changed · message · params · data · error
batch_id · run_id · config_hash

changed may be null when a real write outcome is ambiguous. A confirmed or ambiguous write is never reported as changed=false. See Error handling.

Configuration model

The workbook has two authored sheets:

  • config: roles, table/folder scopes, permission, RLS/CLS, and member names;
  • member: the synthetic-name-to-Entra-object-ID preload used by OLAF.

The workbook is an untrusted input. Verify every object ID against Entra before loading. OLAF deliberately does not call Microsoft Graph for directory resolution; that is a project design choice, not a claim that Graph tokens are universally impossible in Fabric notebooks. NotebookUtils documents an evolving audience list: Get a token.

The generated mapping carries explicit scopes and provenance. It is a review artifact, not an authorization boundary by itself. Full references:

OLAF's predicate parser is intentionally conservative and is not an exhaustive statement of what the platform accepts. Use Microsoft's current RLS syntax reference. Role, member, path, and latency values are volatile platform limits; consult the current limitations and latencies pages at deployment time.

Concurrency and recovery

OLAF requires an ETag-bearing DAR snapshot for sensitive writes. Conditional DAR requests use the exact captured ETag; a missing or changed snapshot blocks instead of silently refreshing the approved state. Microsoft documents the collection ETag, optional If-Match, and 412 response in the official list and bulk PUT references.

That ETag covers only the DAR collection. It does not lock workspace sharing, privileged access, shortcuts, or local Delta/file writes. A REST response, control table change, audit append, backup write, and Delta RESTORE are not one transaction. Preserve the incident sentinel, prepared record, and recovery pointers after a partial or ambiguous operation. Follow Protecting OLAF control data and the recovery runbook.

Documentation

Roadmap

Options, not promises or release dates. Each is blocked on a specific, checkable condition, and none may weaken the control-data boundary or turn an observed service behavior into a platform contract. docs/roadmap.md carries the detail and the official sources behind each one.

  • Differential apply — per-role writes instead of one bulk PUT, so an apply touches only what changed. Blocked on a stable, concurrency-safe official contract: the single-role endpoints are themselves Preview, and the design needs documented precondition behavior plus a recovery model for a partially completed role set.
  • RLS + CLS cross-grant detection at generate — closing rule C5's blind spot, where a user arrives through a group on one side and directly on the other. The conservative rule stays until evidence justifies otherwise; OLAF deliberately does not call Microsoft Graph.
  • Scheduled drift detection — a gate an adopter schedules rather than wires themselves. Must stay read-only, separate identity drift from policy drift, and keep real principal values out of CI logs.
  • Local authoring tool — validate and build a config from a developer machine against a real workspace, so table names, member resolution and rule violations surface while the config is still being written instead of at generate. Blocked on where real identifiers and a Fabric token are allowed to live once they leave the Fabric boundary; validation only, never a second write path.
  • Separate control store — a separately secured store or independently managed keys, for organizations that cannot accept v1's trusted-administrator boundary. Not a minimal v1 change.

Not planned: replacing Fabric enforcement with an OLAF query layer · workspace and item RBAC · calling Microsoft Graph in v1 · claiming production readiness while the required endpoint is Preview — the full list and why.

Repository layout

notebooks/   runtime and optional workflow/example notebooks
configs/     synthetic starter workbook
tests/       fixture-based pytest suite
scripts/     repository checks
docs/        operator and API documentation
assets/      OLAF-owned artwork

Security and support

Read SECURITY.md before using real principal data. GitHub private vulnerability reporting is the confidential route when maintainers have enabled it; the release gate verifies that setting before publication. GitHub documents the reporting flow and configuration requirement.

General questions and sanitized bug reports belong in GitHub issues. See SUPPORT.md and CONTRIBUTING.md.

The OLAF Arctic Owl Sentinel — a crystalline owl holding a notebook that carries the OLAF shield crest and its padlock

The Arctic Owl keeps watch over your lake — every change is seen, reviewed,
and approved before it is submitted.

License and identity

Code, documentation, and project artwork are licensed under MIT, with OLAF contributors as the collective notice. Brand guidance is a request, not an additional license restriction: OLAF brand guidelines.


Independent-project disclaimer. OLAF is an independent community project and is not affiliated with, endorsed by, sponsored by, or supported by Microsoft. Microsoft, Microsoft Fabric, and OneLake are trademarks of the Microsoft group of companies. No Microsoft logo or product icon is used by this project.

Releases

Packages

Contributors

Languages