Skip to content

docs: validate n8n import dry run #13

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Jdelg718 merged 2 commits into from
May 18, 2026
Merged

docs: validate n8n import dry run #13

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
ca05d4d
docs: validate n8n import dry run
May 15, 2026
2d1f9d3
docs: add tomorrow restart notes
May 15, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
98 changes: 59 additions & 39 deletions HANDOFF.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,11 @@

- GitHub: `https://github.com/Jdelg718/BotFucker`
- Default branch: `main`
- Latest merged milestone: Phase 10 optional LLM classifier (`feat: add optional llm classifier hook`, PR #11)
- Current working branch: `phase-11-guarded-yolo-guardrails`
- Current PR target: Phase 11 guarded YOLO guardrails
- Current local demo target: demonstrate deterministic local review, optional mocked LLM classifier fallback/validation, approved-action export, dry-run n8n bridge contract, and fail-closed YOLO policy checks
- Latest merged milestone: Phase 11 guarded YOLO policy (`feat: add guarded yolo policy`, PR #12)
- Current working branch: `phase-12-n8n-import-dry-run-validation`
- Current PR target: Phase 12 real n8n import/dry-run validation — PR #13 open and CI green: `https://github.com/Jdelg718/BotFucker/pull/13`
- Tomorrow start: merge PR #13 if still green, then branch from updated `main` for Phase 13 Reviewed Action Bridge Promotion Plan
- Current local demo target: demonstrate deterministic local review, optional mocked LLM classifier fallback/validation, approved-action export, dry-run n8n bridge contract, fail-closed YOLO policy checks, and real n8n import validation results
- Current promo artifact: `promo/botfucker-animated-explainer/renders/botfucker-animated-explainer_narrated-final.mp4`

## What BotFucker Is
Expand All @@ -26,7 +27,8 @@ Important files:

```text
DESIGN.md # v2 architecture and principles
ROADMAP.md # phased product roadmap, current through Phase 11 guarded YOLO guardrails
ROADMAP.md # phased product roadmap, current through Phase 12 n8n import validation
docs/n8n-import-validation.md # Phase 12 real n8n import/dry-run procedure and results
docs/webhook-contract.md # normalized n8n/webhook JSON contract
docs/n8n-workflow.json # importable n8n import starter workflow
docs/n8n-workflow.md # n8n import operator guide and safety checklist
Expand Down Expand Up @@ -68,7 +70,8 @@ These are non-negotiable:
Run from repo root:

```bash
python3 -m py_compile outreach_filter.py botfucker/*.py
python3 scripts/validate_n8n_workflow_exports.py
python3 -m py_compile outreach_filter.py botfucker/*.py scripts/validate_n8n_workflow_exports.py
python3 -m unittest discover -s tests -v
```

Expand Down Expand Up @@ -115,15 +118,16 @@ git log --oneline -5
Expected current top commit:

```text
e0443b7 feat: add optional llm classifier hook (#11)
713bc7f feat: add guarded yolo policy (#12)
```

If this handoff update has been merged after that, the top commit will be newer. The important part is that PR #11 content is present.
If this handoff update has been merged after that, the top commit will be newer. The important part is that PR #12 content is present.

### Verify locally

```bash
python3 -m py_compile outreach_filter.py botfucker/*.py
python3 scripts/validate_n8n_workflow_exports.py
python3 -m py_compile outreach_filter.py botfucker/*.py scripts/validate_n8n_workflow_exports.py
python3 -m unittest discover -s tests -v
```

Expand Down Expand Up @@ -167,6 +171,7 @@ Say explicitly:
- current bridge work is dry-run/log-only, not live provider mutation
- optional LLM classifier is provider-hooked and validated, with deterministic fallback
- legacy live automation now requires explicit YOLO guardrails before provider actions
- real n8n import validation passed sample-only dry-run on n8n-vps; validation rows were cleaned up

## Animated Explainer Demo

Expand Down Expand Up @@ -234,76 +239,91 @@ Delivered optional `llm_provider` hook to `classify_message`, untrusted/bounded

### Phase 11 — Guarded YOLO Mode

Implemented on branch `phase-11-guarded-yolo-guardrails`.
Delivered `YoloPolicy`, fail-closed YOLO decision evaluator, default-off exact confirmation phrase, emergency stop, action/classification allowlists, confidence/daily-limit/tone gates, legacy live-auto-approve guardrail requirement, and tests proving live action attempts fail closed.

### Phase 12 — Real n8n Import/Dry-Run Validation

Implemented on branch `phase-12-n8n-import-dry-run-validation`.

Delivered:

1. Added `botfucker.yolo_policy.YoloPolicy` and `evaluate_yolo_decision`.
2. YOLO is disabled by default.
3. Requires exact confirmation phrase: `I ACCEPT BOTFUCKER YOLO RISK`.
4. Supports emergency stop override.
5. Gates provider actions by action allowlist, classification allowlist, confidence threshold, daily action limit, and reply tone allowlist.
6. Legacy `--live --auto-approve` path now requires a YOLO policy before live provider actions.
7. Live warning path checks `send_warning`, `write_blacklist`, and `move_to_sales`; blacklist match delete checks `delete_message`.
8. Added tests proving default denial, confirmation failure, valid pass, gate failures, emergency stop, and live-auto-approve guardrail requirement.
1. Added `scripts/validate_n8n_workflow_exports.py` static workflow export preflight.
2. Added `samples/approved-actions.sample.json` safe fake approved-action bundle.
3. Added `docs/n8n-import-validation.md` with exact procedure and actual n8n-vps results.
4. Added explicit workflow IDs required by n8n 2.18.5 CLI import.
5. Updated n8n file paths to `/home/node/.n8n-files` because Read/Write Files rejects arbitrary paths on the target.
6. Updated approved-action bridge to parse `readWriteFile` binary JSON with `getBinaryDataBuffer` before schema validation.
7. Imported both workflows into n8n-vps as inactive/manual workflows.
8. Executed approved-action bridge sample-only dry-run; final node emitted `would_execute: approve_warning`, `provider_execution: not_performed`, and `bridge_status: dry_run_logged_only`.
9. Cleaned validation workflow rows, sample execution rows, related n8n workflow metadata rows, and temp files.

Verification:

```bash
python3 -m py_compile outreach_filter.py botfucker/*.py
python3 scripts/validate_n8n_workflow_exports.py
python3 -m py_compile outreach_filter.py botfucker/*.py scripts/validate_n8n_workflow_exports.py
python3 -m unittest discover -s tests -v
```

## Next PR Recommendation

Build **Phase 12: Real n8n Import/Dry-Run Validation**, not OAuth.
Build **Phase 13: Reviewed Action Bridge Promotion Plan**, not OAuth.

Suggested scope:

1. Import `docs/n8n-workflow.json` into Kent's n8n test target or local matching n8n.
2. Import `docs/n8n-approved-action-bridge.json` inactive/dry-run.
3. Confirm node compatibility, file paths, and environment variable assumptions.
4. Run sample-only dry-run payloads end-to-end.
5. Document import/export fixes.
6. Attach no Gmail/Microsoft/IMAP/SMTP mutation credentials.
1. Define how one provider action type graduates from dry-run to live review.
2. Keep credentials in n8n only.
3. Design persistent processed-`audit_id` state.
4. Document rollback and emergency stop.
5. Require provider-specific sandbox/manual tests.
6. Require Rex/Gus security/ops review before any live mutation node is connected.

## Suggested Prompt for Kodex/Codex

```text
You are working on BotFucker, an AI-era inbox defense app.

Read DESIGN.md, ROADMAP.md, HANDOFF.md, README.md, docs/webhook-contract.md, docs/n8n-workflow.md, docs/n8n-approved-action-bridge.md, and docs/provider-auth-plan.md.
Read DESIGN.md, ROADMAP.md, HANDOFF.md, README.md, docs/webhook-contract.md, docs/n8n-workflow.md, docs/n8n-approved-action-bridge.md, docs/n8n-import-validation.md, and docs/provider-auth-plan.md.

First, verify the current Phase 11 branch without changing behavior:
- run python3 -m py_compile outreach_filter.py botfucker/*.py
First, verify the current Phase 12 branch without changing behavior:
- run python3 scripts/validate_n8n_workflow_exports.py
- run python3 -m py_compile outreach_filter.py botfucker/*.py scripts/validate_n8n_workflow_exports.py
- run python3 -m unittest discover -s tests -v
- inspect botfucker/yolo_policy.py and tests/test_yolo_guardrails.py
- confirm live provider actions require explicit YOLO guardrails
- inspect docs/n8n-import-validation.md and samples/approved-actions.sample.json
- confirm n8n workflows include explicit ids, are inactive, and use /home/node/.n8n-files for Read/Write Files paths

Then review Phase 11 only: Guarded YOLO Mode.
Then review Phase 12 only: Real n8n Import/Dry-Run Validation.

Check that YOLO is disabled by default, requires the exact confirmation phrase, supports emergency stop, gates provider actions by allowlist/classification/confidence/daily limit/tone, and that legacy --live --auto-approve cannot silently mutate mail without a YOLO policy.
Check that both workflows imported into n8n-vps as inactive/manual, approved-action bridge executed sample-only dry-run, final output was provider_execution:not_performed, and cleanup removed validation rows/temp files.

Do not add real OAuth. Do not add provider credentials. Do not attach Gmail/Microsoft/IMAP/SMTP mutation credentials. Do not enable live n8n provider actions. Preserve the provider boundary: live provider execution remains separately reviewed and guarded.
```

## Team Plan

- **Amy**: orchestration and scope control. She keeps the product from wandering into OAuth swamp country before n8n dry-run validation is real.
- **Chip**: owns n8n import/export compatibility fixes if Phase 11 review passes.
- **Rex**: security veto on YOLO guardrails, live-action safety gates, credential absence, provider-boundary isolation, and XSS regressions.
- **Gus**: local demo verification, n8n import/dry-run validation, CLI ergonomics, CI, and operator docs.
- **Fred**: n8n version/node compatibility research only; no direct OAuth implementation yet.
- **Amy**: orchestration and scope control. She keeps the product from wandering into OAuth swamp country before bridge promotion is reviewed.
- **Chip**: owns reviewed action bridge promotion docs/tests if Phase 12 review passes.
- **Rex**: security veto on processed-audit dedupe, credential absence, live-action safety gates, provider-boundary isolation, and XSS regressions.
- **Gus**: n8n operator verification, dry-run bridge observability, cleanup steps, CI, and operator docs.
- **Fred**: provider sandbox/action-limit research only; no direct OAuth implementation yet.

## Known Follow-Up Issues

- Deterministic classifier still needs real-world tuning.
- Optional LLM classifier exists only as a provider hook; no real provider wiring or credentials are implemented.
- No production OAuth yet.
- n8n workflow package needs real local import testing against Kent's actual n8n instance before activation.
- Real n8n import validation passed on n8n-vps with sample-only dry-run and cleanup; do not activate those workflows without a separate reviewed bridge-promotion plan.
- n8n approved action bridge is dry-run only; live provider actions still need a separate explicit reviewed workflow.
- YOLO guardrails exist but live provider actions still require explicit operator configuration and must not be casually enabled.

## Tomorrow Restart

- PR #13 is open and CI green: `https://github.com/Jdelg718/BotFucker/pull/13`.
- First move tomorrow: re-check PR #13, merge if green, pull `main`, branch Phase 13.
- Phase 13 target: **Reviewed Action Bridge Promotion Plan**.
- Do **not** add OAuth, provider credentials, or live n8n provider mutation nodes.
- Keep live provider execution separate, reviewed, audited, deduped by processed `audit_id`, rollback-ready, and security/ops-reviewed.

## Product Voice

BotFucker should be funny and hostile to spam, but serious about safety.
Expand Down
23 changes: 23 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -454,6 +454,29 @@ export BF_YOLO_EMERGENCY_STOP=false

If any gate fails, BotFucker raises before the live provider action. Subtle? No. That is the point.

## Phase 12 n8n Import Validation

Phase 12 proved the n8n exports import and dry-run on Kent's real n8n target (`n8n-vps`, n8n 2.18.5) without attaching provider mutation credentials.

Artifacts:

- `scripts/validate_n8n_workflow_exports.py` — static preflight for workflow exports
- `samples/approved-actions.sample.json` — fake approved-action bundle
- `docs/n8n-import-validation.md` — exact import/dry-run/cleanup procedure and result

Actual result:

- both workflows imported inactive/manual
- approved-action bridge executed sample-only dry-run
- output stayed `provider_execution: not_performed`
- cleanup removed validation workflows and sample execution rows

Compatibility fixes shipped:

- workflow exports include explicit `id` fields for n8n CLI import
- file paths use `/home/node/.n8n-files`, because n8n 2.18.5 blocks arbitrary local file paths
- approved-action bridge parses `readWriteFile` binary JSON via `getBinaryDataBuffer`

## Test Before Going Live

Compile-check the script and package:
Expand Down
73 changes: 53 additions & 20 deletions ROADMAP.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -269,18 +269,23 @@ Acceptance criteria:
- Never silently enables aggressive/legal-ish replies.
- Blocks provider actions that fail allowlist/confidence/daily-limit/emergency-stop gates.

### Phase 12 — Real n8n Import/Dry-Run Validation
### Phase 12 — Real n8n Import/Dry-Run Validation ✅

Status: implemented on `phase-12-n8n-import-dry-run-validation`.

Goal: test the packaged n8n workflows against Kent's actual n8n instance without activating live provider mutations.

Deliverables:
Delivered:

- import `docs/n8n-workflow.json` into n8n-vps or local n8n test instance
- import `docs/n8n-approved-action-bridge.json` inactive/dry-run
- confirm workflow node compatibility with the installed n8n version
- confirm environment variable/file-path assumptions
- run sample-only dry-run payloads end-to-end
- document any n8n import/export fixes
- added `scripts/validate_n8n_workflow_exports.py` static preflight validator
- added `samples/approved-actions.sample.json` safe fake approved-action bundle
- added `docs/n8n-import-validation.md` with exact procedure and actual results
- imported both workflows into `n8n-vps` running n8n 2.18.5
- confirmed both imported as inactive/manual workflows
- executed the approved-action bridge dry-run on sample data only
- confirmed dry-run output: `would_execute: approve_warning`, `provider_execution: not_performed`
- cleaned up validation workflows, related sample execution rows, and temp files
- fixed n8n compatibility issues: workflow `id` required, files must live under `/home/node/.n8n-files`, readWriteFile JSON arrives as binary and must be parsed with `getBinaryDataBuffer`

Acceptance criteria:

Expand All @@ -289,6 +294,26 @@ Acceptance criteria:
- No Gmail/Microsoft/IMAP/SMTP mutation credentials are attached.
- Activation remains manual and reviewed.

### Phase 13 — Review Action Bridge Promotion Plan

Goal: define the reviewed path from dry-run logs to a live provider bridge without adding OAuth/provider mutation directly to BotFucker core.

Deliverables:

- operator checklist for promoting one provider action type at a time
- explicit credential ownership in n8n only
- audit/dedupe persistence design for processed `audit_id` values
- rollback/emergency-stop steps
- provider-specific sandbox/manual test plan
- security review checklist before any live provider action node is connected

Acceptance criteria:

- Live bridge remains separate from BotFucker core.
- Only reviewed approved-action records are eligible.
- Every provider mutation is idempotent, audited, and reversible where possible.
- No live provider credentials are committed or exported.

## Local Kodex/Codex Demo Plan

Kent is pulling this locally onto Kodex/Codex next. The demo should show what exists now, not pretend Phase 8 is already done. Revolutionary concept, apparently.
Expand Down Expand Up @@ -335,23 +360,31 @@ Use fake or sanitized JSON only. Real mailbox payloads stay out of the repo.

## Near-Term Recommendation

Next PR should be **Phase 12: Real n8n Import/Dry-Run Validation**, not OAuth implementation.
Next PR should be **Phase 13: Reviewed Action Bridge Promotion Plan**, not OAuth implementation.

Recommended scope:

- import both n8n JSON workflows into Kent's actual n8n test instance or a local matching n8n version
- keep workflows inactive/manual and sample-data-only
- validate node compatibility, environment variables, and file paths
- run the dry-run path end-to-end with fake/sample payloads
- document any import/export fixes
- attach no Gmail/Microsoft/IMAP/SMTP mutation credentials
- define how one provider action type graduates from dry-run to live review
- keep credentials in n8n only
- design persistent processed-`audit_id` state
- document rollback and emergency stop
- require provider-specific sandbox/manual tests
- require Rex/Gus security/ops review before any live mutation node is connected

OAuth can still wait. We proved the n8n package imports and dry-runs in the real beast. Next is a promotion plan for one action type at a time, because wiring live mail mutation without a checklist is how you manufacture regret at scale.

### Tomorrow restart checklist

OAuth can still wait. We now have import, review, export, bridge, optional LLM classification, and YOLO guardrails. Next: prove the n8n package actually imports and dry-runs in the real beast instead of admiring the JSON like it’s a museum artifact.
1. Re-check PR #13 CI and mergeability.
2. Squash-merge PR #13 into `main` if still green.
3. Pull updated `main` and create Phase 13 branch.
4. Build **Reviewed Action Bridge Promotion Plan** only — still no OAuth, no provider credentials, no live action node activation.
5. Use tests/docs to prove the promotion plan keeps credentials in n8n, persists processed `audit_id` state, and requires rollback/security/operator review.

## Team Utilization

- **Amy**: orchestrates scope, keeps phases honest, and blocks shiny-object OAuth detours.
- **Chip**: owns n8n workflow import/export fixes if real n8n exposes compatibility issues.
- **Rex**: reviews live-action safety gates, credential absence, and provider-boundary isolation.
- **Gus**: verifies n8n import, environment assumptions, local demo steps, CI, and operator docs.
- **Fred**: researches n8n node compatibility/version quirks only; no direct OAuth implementation yet.
- **Chip**: owns bridge promotion docs/tests and any dry-run-to-live scaffolding after review.
- **Rex**: reviews credential absence, processed-audit dedupe, live-action safety gates, and rollback.
- **Gus**: verifies n8n operator steps, imports, cleanup, and bridge observability.
- **Fred**: researches provider-specific sandbox/action constraints only; no direct OAuth implementation yet.
Loading
Loading