Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 35 additions & 3 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,16 +10,48 @@ stored in `VERSION`.

## [Unreleased]

No unreleased entries yet.
### Changed

- retired the removed AzureFox `all-checks` path from the lab validator and current operator docs
so live runs no longer assume a grouped command that AzureFox `main` rejects as of April 8, 2026
- documented an explicit live-run strategy for known slow paths, including Key Vault soft-delete
waits during OpenTofu apply and `role-trusts` latency during AzureFox validation
- added a lab-side Azure Activity Log bundle script and doc so SOC-style local log pulls can be
packaged with run-window phase markers without introducing a separate Azure logging backend
- added validator-emitted `command-timeline.json` artifacts plus bundle-export support so SOC
analysts can line AzureFox command start and finish times up with raw Azure Activity Log windows

## [1.2.0] - 2026-04-05

### Added

- one Azure Automation account with a system-assigned identity so the lab now owns a deterministic
Phase 4 `automation` proof surface instead of only validating zero-account execution

### Changed

- promoted `automation`, `devops`, `lighthouse`, and `cross-tenant` into the standalone validator
path with truth-preserving checks that distinguish deterministic proof from tenant-shaped or
external-config-shaped command behavior
- promoted `vmss` into the standalone validator path because the current lab already deploys a
stable internal VM scale set that AzureFox can read deterministically
- expanded `all-checks` validator coverage to include the current `storage` section and the newer
`lighthouse`, `automation`, `devops`, `vmss`, and `snapshots-disks` command membership reflected
in the main AzureFox repo
- updated repo docs and release language to target AzureFox `1.2.0` / Phase 4 parity instead of
leaving Phase 4 described as mostly discovery-only work
- tightened the documented truth boundary for external or tenant-shaped surfaces:
`cross-tenant` and `lighthouse` stay evidence-led, while `devops` requires a real Azure DevOps
organization for pipeline proof and otherwise should surface the expected configuration issue

## [1.1.0] - 2026-04-05

### Added

- Phase 3.5 checkpoint note for the AzureFox `1.1.0` release boundary:
`docs/phase3-compute-apps-network-checkpoint.md`
[phase3-compute-apps-network-checkpoint-2026-04-08.md](/Users/cfarley/Documents/AzureFox-reference/terraform-labs/phase3-compute-apps-network-checkpoint-2026-04-08.md)
- Phase 4 live-capture note for the AzureFox `1.2.0` command lane:
`docs/phase4-command-discovery-checkpoint.md`
[phase4-command-discovery-checkpoint-2026-04-08.md](/Users/cfarley/Documents/AzureFox-reference/terraform-labs/phase4-command-discovery-checkpoint-2026-04-08.md)

### Changed

Expand Down
76 changes: 50 additions & 26 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,14 +30,16 @@ command implementation, and release source of truth.

## AzureFox Coverage

The lab is built to exercise this release-gated subset of AzureFox commands and sections. AzureFox
The lab is built to exercise this release-gated subset of AzureFox standalone commands. AzureFox
may have additional commands on `main` that are still discovery-only or not yet backed by
deterministic lab proof objects.

Current validator coverage:

- `whoami`
- `inventory`
- `automation`
- `devops`
- `arm-deployments`
- `env-vars`
- `tokens-credentials`
Expand All @@ -46,12 +48,15 @@ Current validator coverage:
- `permissions`
- `privesc`
- `role-trusts`
- `lighthouse`
- `cross-tenant`
- `resource-trusts`
- `auth-policies`
- `managed-identities`
- `keyvault`
- `storage`
- `vms`
- `vmss`
- `nics`
- `dns`
- `endpoints`
Expand All @@ -64,28 +69,33 @@ Current validator coverage:
- `acr`
- `databases`
- `snapshots-disks`
- `all-checks --section identity`
- `all-checks --section network`
- `all-checks --section compute`
- `all-checks --section config`
- `all-checks --section secrets`
- `all-checks --section resource`

Optional grouped follow-up:

- `chains credential-path`
- `chains deployment-path`
- `chains escalation-path`

The project is OpenTofu-first, but the HCL stays close to standard Terraform style so it feels
familiar to most operators.

Current checkpoint notes:

- `docs/phase3-compute-apps-network-checkpoint.md`
- `docs/phase4-command-discovery-checkpoint.md`
- `docs/activity-log-bundles.md`
- `docs/live-run-strategy.md`
- historical checkpoint notes now live under:
[phase2-secrets-config-resource-checkpoint-2026-04-08.md](/Users/cfarley/Documents/AzureFox-reference/terraform-labs/phase2-secrets-config-resource-checkpoint-2026-04-08.md)
- historical checkpoint notes now live under:
[phase3-compute-apps-network-checkpoint-2026-04-08.md](/Users/cfarley/Documents/AzureFox-reference/terraform-labs/phase3-compute-apps-network-checkpoint-2026-04-08.md)
- historical checkpoint notes now live under:
[phase4-command-discovery-checkpoint-2026-04-08.md](/Users/cfarley/Documents/AzureFox-reference/terraform-labs/phase4-command-discovery-checkpoint-2026-04-08.md)

Current release boundary:

- this repo is now aligned to AzureFox `1.1.0` / Phase 3.5 for release-gated validation
- Phase 4 / `1.2.0` remains discovery-first here, except for `snapshots-disks`, which is now a
validator-backed proof surface
- broader PostgreSQL relational parity is still tracked as an AzureFox main-repo follow-up rather
than being overstated in this lab release
- this repo now targets AzureFox `1.2.0` / Phase 4 as the current parity boundary
- deterministic lab-backed proof now includes `snapshots-disks`, `vmss`, and one Automation account
- `lighthouse` and `cross-tenant` are validated as evidence-led tenant surfaces rather than fixed row-count proof
- `devops` is validated conditionally: without `AZUREFOX_DEVOPS_ORG`, the validator expects the truthful missing-organization issue instead of pretending pipeline coverage exists

## Lab Shape

Expand Down Expand Up @@ -115,6 +125,7 @@ Current release boundary:
- One AKS cluster with a public control-plane endpoint and system-assigned identity
- One Azure Container Registry with public network access and admin user enabled
- One Azure SQL server with one user database
- One Azure Automation account with a system-assigned identity
- One public DNS zone plus one private DNS zone with a registration-enabled VNet link
- Three deployment-history objects:
one succeeded subscription deployment with linked template URI
Expand All @@ -139,6 +150,7 @@ With this setup, AzureFox should surface:
- Key Vault public-network, private-endpoint, and purge-protection posture from `keyvault`
- public storage and open firewall findings from `storage`
- a public VM with an attached identity from `vms`
- one internal VM scale set footprint from `vmss`
- NIC attachment and public-IP references from `nics`
- public IP and Azure-managed hostname visibility from `endpoints`
- NIC-backed public ingress evidence from `network-ports`
Expand All @@ -149,14 +161,13 @@ With this setup, AzureFox should surface:
- AKS control-plane endpoint, agent-pool count, OIDC posture, and addon visibility from `aks`
- ACR login-server, admin-user, webhook, replication, and policy posture from `acr`
- Azure SQL endpoint, visible user-database inventory, and minimal TLS posture from `databases`
- Azure Automation account identity and zero-object execution posture from `automation`
- managed-disk attachment, network-access, and encryption posture from `snapshots-disks`
- delegated-management evidence from `lighthouse` when the subscription exposes it
- outside-tenant trust evidence from `cross-tenant` without turning ambient tenant posture into a deterministic row-count claim
- Azure DevOps pipeline evidence from `devops` only when a real organization is configured
- DNS zone inventory and private-endpoint-backed namespace usage from `dns`
- identity checkpoint orchestration artifacts from `all-checks --section identity`
- network checkpoint orchestration artifacts from `all-checks --section network`
- compute checkpoint orchestration artifacts from `all-checks --section compute`
- config checkpoint orchestration artifacts from `all-checks --section config`
- secrets checkpoint orchestration artifacts from `all-checks --section secrets`
- resource checkpoint orchestration artifacts from `all-checks --section resource`
- optional grouped follow-up through AzureFox `chains` families when you want a higher-level review path in addition to the standalone proof artifacts

`auth-policies` is handled a little differently in this repo:

Expand Down Expand Up @@ -301,8 +312,15 @@ By default the validator:
- executes AzureFox from `--azurefox-dir`
- runs in `--mode full`, which executes the current release-gated standalone AzureFox command set
- prints progress lines before and after each AzureFox step, including elapsed time and target artifact directories
- records per-command UTC start and finish timestamps plus elapsed duration in `command-timeline.json`
- stores proof artifacts under `proof-artifacts/latest`

For richer `devops` proof, point AzureFox at a real Azure DevOps organization before you run the validator:

```bash
export AZUREFOX_DEVOPS_ORG=<org-name>
```

Optional flags:

```bash
Expand All @@ -315,39 +333,44 @@ Useful scoped reruns:

```bash
python3 scripts/validate_azurefox_lab.py --mode commands-only
python3 scripts/validate_azurefox_lab.py --mode all-checks-only
python3 scripts/validate_azurefox_lab.py --mode full
python3 scripts/validate_azurefox_lab.py --mode full --skip-command role-trusts
```

Runtime notes:

- use `--mode full` as the single end-to-end validation run
- `--mode full` no longer bundles `all-checks`; run `--mode all-checks-only` separately only when you intentionally want wrapper coverage
- `commands-only` is now just an explicit standalone-only rerun alias for the same command family as `full`
- if the live lab is already up and you only changed outputs or validator expectations, refresh the
OpenTofu state before rerunning validation so stale `validation_manifest` data does not cause a
false mismatch
- use `--mode commands-only` when you want the individual command outputs without the orchestration pass
- use `--mode all-checks-only` only when you are specifically validating the section wrapper and artifact emission path in isolation
- do not treat `all-checks-only` as part of the default release-validation sequence unless we explicitly decide the wrapper coverage is required
- `role-trusts` can take several minutes because the Azure API path is slow; the validator now emits periodic wait lines during that step instead of appearing hung
- after `role-trusts` has been validated once for the current phase, reruns can use `--skip-command role-trusts` unless you changed that slice or hit a blocker that points back to it
- Key Vault replacement during `tofu apply` can spend several minutes in Azure soft-delete before recreate completes; treat that as a known slow path rather than a surprise hang
- more generally, do not rerun a known slow validation path by default; only pay that cost again
when the changed slice touches it, a live blocker points back to it, or the team explicitly wants
the extra proof
- use [docs/live-run-strategy.md](/Users/cfarley/Documents/Terraform Labs for AzureFox/docs/live-run-strategy.md) as the standing rule set for full passes versus fast reruns

Artifacts include:

- one JSON payload per AzureFox command
- copied loot files emitted by AzureFox
- `all-checks --section <section>` output plus `run-summary.json` for `identity`, `network`, `compute`, `config`, `secrets`, and `resource`
- `command-timeline.json`
- `summary.json`
- `summary.txt`
- `azurefox-mismatch-report.md`
- `identity-mismatch-report.md`
- `azurefox-follow-up-items.md`

Optional SOC / detection artifact flow:

- `command-timeline.json` now records UTC start and finish markers plus duration for each AzureFox validation command so analysts can correlate those markers against Azure control-plane activity
- this timestamp artifact only covers the validator command lane; keep recording `apply` and `destroy` timestamps separately when you want the full lab window in the bundle
- use [docs/activity-log-bundles.md](/Users/cfarley/Documents/Terraform Labs for AzureFox/docs/activity-log-bundles.md) to pull Azure Activity Log locally for the full lab window and package it with phase markers plus validator command markers
- the bundle script is [export_activity_log_bundle.py](/Users/cfarley/Documents/Terraform Labs for AzureFox/scripts/export_activity_log_bundle.py)

## Evidence Boundaries

This lab is here to validate AzureFox output against real Azure objects. It is not a substitute for
Expand All @@ -364,8 +387,9 @@ What AzureFox can verify directly from read-only control-plane and Graph data:
- that managed-identity token surfaces correlate across web workloads, VMs, and deployment history
- that Azure-managed App Service and Function App hostnames are visible control-plane endpoint paths, not proven live ingress
- that NIC-backed public ingress evidence comes from visible NSG allow rules rather than guessed reachability
- that storage, API Management, AKS, ACR, and Azure SQL depth stays evidence-based when only management metadata is visible
- that storage, VMSS, Automation, API Management, AKS, ACR, and Azure SQL depth stays evidence-based when only management metadata is visible
- that the current DNS boundary stays at zone inventory and private-endpoint-backed namespace usage rather than record export or live resolution proof
- that `lighthouse`, `cross-tenant`, and `devops` stay honest about external prerequisites, tenant shape, or partial-read boundaries

What only the lab can confirm once infrastructure exists and the validator has been run:

Expand Down
2 changes: 1 addition & 1 deletion VERSION
Original file line number Diff line number Diff line change
@@ -1 +1 @@
1.1.0
1.2.0
74 changes: 74 additions & 0 deletions docs/activity-log-bundles.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
# Activity Log Bundles

Use this when you want a local Azure Activity Log bundle for SOC or detection work without sending
anything into a separate Azure logging backend.

The bundle script lives at:

- `scripts/export_activity_log_bundle.py`

It writes a local bundle directory with:

- `run-window.json`
- `metadata.json`
- `timeline.md`
- `azure-activity-log.json`
- `command-timeline.json` if you pass the validator artifact into the exporter
- `<run-id>.zip` unless you pass `--no-zip`

## Minimal Flow

Record UTC timestamps for the infrastructure phases as you move through the lab run:

```bash
START_UTC="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
APPLY_START_UTC="$START_UTC"
tofu apply
APPLY_END_UTC="$(date -u +%Y-%m-%dT%H:%M:%SZ)"

VALIDATE_START_UTC="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
python3 scripts/validate_azurefox_lab.py --mode full
VALIDATE_END_UTC="$(date -u +%Y-%m-%dT%H:%M:%SZ)"

DESTROY_START_UTC="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
tofu destroy
DESTROY_END_UTC="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
```

The validator now writes `proof-artifacts/latest/command-timeline.json` automatically. That file
captures per-command UTC start and finish markers plus elapsed duration for the AzureFox command
lane, so you no longer need to hand-build those validator markers yourself.

Then export the bundle:

```bash
python3 scripts/export_activity_log_bundle.py \
--run-id live-20260408 \
--start-time "$START_UTC" \
--end-time "$DESTROY_END_UTC" \
--command-timeline-file proof-artifacts/latest/command-timeline.json \
--phase apply_start="$APPLY_START_UTC" \
--phase apply_end="$APPLY_END_UTC" \
--phase validate_start="$VALIDATE_START_UTC" \
--phase validate_end="$VALIDATE_END_UTC" \
--phase destroy_start="$DESTROY_START_UTC" \
--phase destroy_end="$DESTROY_END_UTC"
```

By default this writes to:

- `proof-artifacts/activity-log/<run-id>/`
- `proof-artifacts/activity-log/<run-id>.zip`

## Notes

- This is local pull-only. It does not require Log Analytics, Event Hub, or Storage export.
- The script uses `az monitor activity-log list`, so it needs a working Azure CLI login.
- The default `--max-events` is `5000`. Raise it if you expect a busier subscription window.
- `scripts/validate_azurefox_lab.py` now writes `proof-artifacts/latest/command-timeline.json`
automatically so you can hand the exporter a per-command start and finish timeline from the same run.
- `command-timeline.json` only covers the AzureFox validator commands. You still need manual
`date -u` markers or another wrapper for `tofu apply`, `tofu destroy`, and any other external
steps you want represented in the bundle window.
- The timeline file is there to help analysts line phase markers and AzureFox command markers up
with Azure event timestamps and operation names.
73 changes: 73 additions & 0 deletions docs/live-run-strategy.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
# Live Run Strategy

Use this when you are running the lab against a real Azure subscription and want a practical rule
set instead of rediscovering the same slow paths each time.

## Known Slow Paths

- `azurerm_key_vault.open` replacement is a normal Azure slow path.
- Azure Key Vault deletes move through soft-delete before the name is fully reusable, so OpenTofu
can spend several minutes waiting even after the vault disappears from the active subscription
view.
- `role-trusts` is a known slow Azure API path during AzureFox validation and can take several
minutes before JSON returns.

## Release-Candidate Pass

Pay the full slow-path cost once when the goal is a fresh release-candidate baseline:

```bash
tofu init
tofu plan
tofu apply
python3 scripts/validate_azurefox_lab.py --mode full
tofu destroy
```

Keep `proof-artifacts/latest/command-timeline.json` from that validator pass so the same release
candidate run has per-command UTC markers if the SOC later wants the Azure Activity Log window.

Use this full pass when:

- the lab shape changed
- the validator expectations changed in a way that touches the slow slices
- AzureFox changed its output for `keyvault` or `role-trusts`
- you need a new clean baseline for release notes or proof artifacts

## Fast Rerun Strategy

Do not pay every slow-path cost on every rerun.

If the infrastructure is already up and the change did not touch the lab shape:

```bash
tofu apply -refresh-only
python3 scripts/validate_azurefox_lab.py --mode full --skip-command role-trusts
```

Keep the refreshed run's `command-timeline.json` too. It is the quickest way to tell the difference
between a known slow validator command and a run that truly drifted or hung.

Use this faster rerun when:

- you changed validator wording, docs, or artifact handling
- you are rechecking a live mismatch that does not point back to `role-trusts`
- the current state only needs a manifest refresh before validation

## Operational Rules

- Treat Key Vault replacement waits as expected unless they exceed the normal Azure delete/recreate
window by a wide margin.
- Treat `role-trusts` as skippable on reruns once it has already been validated for the current
phase.
- Keep the validator's `command-timeline.json` with the same proof artifacts as `summary.json` so
later SOC bundle exports can show when each AzureFox command actually ran.
- Do not keep stale `all-checks` assumptions in the live-run workflow. Current AzureFox `main`
removed that grouped command on April 8, 2026; use standalone command validation here and treat
`chains` as optional grouped follow-up instead.
- If a rerun does not need new Azure infrastructure truth, prefer refresh-only plus validator rerun
over a full destroy/reapply cycle.
- If the SOC wants a local starter pack for detections, use
[activity-log-bundles.md](/Users/cfarley/Documents/Terraform Labs for AzureFox/docs/activity-log-bundles.md)
to package Azure Activity Log plus phase markers and per-command AzureFox markers after the run
window closes.
Loading
Loading