From ca05d4d9651b5101f5843031b80efebff9fdfb37 Mon Sep 17 00:00:00 2001 From: Amy Hermes Date: Thu, 14 May 2026 20:08:02 -0400 Subject: [PATCH 1/2] docs: validate n8n import dry run --- HANDOFF.md | 81 +++++++++------- README.md | 23 +++++ ROADMAP.md | 65 +++++++++---- docs/n8n-approved-action-bridge.json | 72 +++++++++++--- docs/n8n-import-validation.md | 107 +++++++++++++++++++++ docs/n8n-workflow.json | 5 +- docs/provider-auth-plan.md | 4 +- samples/approved-actions.sample.json | 20 ++++ scripts/validate_n8n_workflow_exports.py | 117 +++++++++++++++++++++++ tests/test_n8n_import_validation.py | 63 ++++++++++++ tests/test_n8n_workflow_docs.py | 4 + 11 files changed, 488 insertions(+), 73 deletions(-) create mode 100644 docs/n8n-import-validation.md create mode 100644 samples/approved-actions.sample.json create mode 100644 scripts/validate_n8n_workflow_exports.py create mode 100644 tests/test_n8n_import_validation.py diff --git a/HANDOFF.md b/HANDOFF.md index 12b29c4..bdd147a 100644 --- a/HANDOFF.md +++ b/HANDOFF.md @@ -4,10 +4,10 @@ - GitHub: `https://github.com/Jdelg718/BotFucker` - Default branch: `main` -- Latest merged milestone: Phase 10 optional LLM classifier (`feat: add optional llm classifier hook`, PR #11) -- Current working branch: `phase-11-guarded-yolo-guardrails` -- Current PR target: Phase 11 guarded YOLO guardrails -- Current local demo target: demonstrate deterministic local review, optional mocked LLM classifier fallback/validation, approved-action export, dry-run n8n bridge contract, and fail-closed YOLO policy checks +- Latest merged milestone: Phase 11 guarded YOLO policy (`feat: add guarded yolo policy`, PR #12) +- Current working branch: `phase-12-n8n-import-dry-run-validation` +- Current PR target: Phase 12 real n8n import/dry-run validation +- Current local demo target: demonstrate deterministic local review, optional mocked LLM classifier fallback/validation, approved-action export, dry-run n8n bridge contract, fail-closed YOLO policy checks, and real n8n import validation results - Current promo artifact: `promo/botfucker-animated-explainer/renders/botfucker-animated-explainer_narrated-final.mp4` ## What BotFucker Is @@ -26,7 +26,8 @@ Important files: ```text DESIGN.md # v2 architecture and principles -ROADMAP.md # phased product roadmap, current through Phase 11 guarded YOLO guardrails +ROADMAP.md # phased product roadmap, current through Phase 12 n8n import validation +docs/n8n-import-validation.md # Phase 12 real n8n import/dry-run procedure and results docs/webhook-contract.md # normalized n8n/webhook JSON contract docs/n8n-workflow.json # importable n8n import starter workflow docs/n8n-workflow.md # n8n import operator guide and safety checklist @@ -115,10 +116,10 @@ git log --oneline -5 Expected current top commit: ```text -e0443b7 feat: add optional llm classifier hook (#11) +713bc7f feat: add guarded yolo policy (#12) ``` -If this handoff update has been merged after that, the top commit will be newer. The important part is that PR #11 content is present. +If this handoff update has been merged after that, the top commit will be newer. The important part is that PR #12 content is present. ### Verify locally @@ -167,6 +168,7 @@ Say explicitly: - current bridge work is dry-run/log-only, not live provider mutation - optional LLM classifier is provider-hooked and validated, with deterministic fallback - legacy live automation now requires explicit YOLO guardrails before provider actions +- real n8n import validation passed sample-only dry-run on n8n-vps; validation rows were cleaned up ## Animated Explainer Demo @@ -234,66 +236,73 @@ Delivered optional `llm_provider` hook to `classify_message`, untrusted/bounded ### Phase 11 — Guarded YOLO Mode -Implemented on branch `phase-11-guarded-yolo-guardrails`. +Delivered `YoloPolicy`, fail-closed YOLO decision evaluator, default-off exact confirmation phrase, emergency stop, action/classification allowlists, confidence/daily-limit/tone gates, legacy live-auto-approve guardrail requirement, and tests proving live action attempts fail closed. + +### Phase 12 — Real n8n Import/Dry-Run Validation + +Implemented on branch `phase-12-n8n-import-dry-run-validation`. Delivered: -1. Added `botfucker.yolo_policy.YoloPolicy` and `evaluate_yolo_decision`. -2. YOLO is disabled by default. -3. Requires exact confirmation phrase: `I ACCEPT BOTFUCKER YOLO RISK`. -4. Supports emergency stop override. -5. Gates provider actions by action allowlist, classification allowlist, confidence threshold, daily action limit, and reply tone allowlist. -6. Legacy `--live --auto-approve` path now requires a YOLO policy before live provider actions. -7. Live warning path checks `send_warning`, `write_blacklist`, and `move_to_sales`; blacklist match delete checks `delete_message`. -8. Added tests proving default denial, confirmation failure, valid pass, gate failures, emergency stop, and live-auto-approve guardrail requirement. +1. Added `scripts/validate_n8n_workflow_exports.py` static workflow export preflight. +2. Added `samples/approved-actions.sample.json` safe fake approved-action bundle. +3. Added `docs/n8n-import-validation.md` with exact procedure and actual n8n-vps results. +4. Added explicit workflow IDs required by n8n 2.18.5 CLI import. +5. Updated n8n file paths to `/home/node/.n8n-files` because Read/Write Files rejects arbitrary paths on the target. +6. Updated approved-action bridge to parse `readWriteFile` binary JSON with `getBinaryDataBuffer` before schema validation. +7. Imported both workflows into n8n-vps as inactive/manual workflows. +8. Executed approved-action bridge sample-only dry-run; final node emitted `would_execute: approve_warning`, `provider_execution: not_performed`, and `bridge_status: dry_run_logged_only`. +9. Cleaned validation workflow rows, sample execution rows, related n8n workflow metadata rows, and temp files. Verification: ```bash -python3 -m py_compile outreach_filter.py botfucker/*.py +python3 scripts/validate_n8n_workflow_exports.py +python3 -m py_compile outreach_filter.py botfucker/*.py scripts/validate_n8n_workflow_exports.py python3 -m unittest discover -s tests -v ``` ## Next PR Recommendation -Build **Phase 12: Real n8n Import/Dry-Run Validation**, not OAuth. +Build **Phase 13: Reviewed Action Bridge Promotion Plan**, not OAuth. Suggested scope: -1. Import `docs/n8n-workflow.json` into Kent's n8n test target or local matching n8n. -2. Import `docs/n8n-approved-action-bridge.json` inactive/dry-run. -3. Confirm node compatibility, file paths, and environment variable assumptions. -4. Run sample-only dry-run payloads end-to-end. -5. Document import/export fixes. -6. Attach no Gmail/Microsoft/IMAP/SMTP mutation credentials. +1. Define how one provider action type graduates from dry-run to live review. +2. Keep credentials in n8n only. +3. Design persistent processed-`audit_id` state. +4. Document rollback and emergency stop. +5. Require provider-specific sandbox/manual tests. +6. Require Rex/Gus security/ops review before any live mutation node is connected. ## Suggested Prompt for Kodex/Codex ```text You are working on BotFucker, an AI-era inbox defense app. -Read DESIGN.md, ROADMAP.md, HANDOFF.md, README.md, docs/webhook-contract.md, docs/n8n-workflow.md, docs/n8n-approved-action-bridge.md, and docs/provider-auth-plan.md. +Read DESIGN.md, ROADMAP.md, HANDOFF.md, README.md, docs/webhook-contract.md, docs/n8n-workflow.md, docs/n8n-approved-action-bridge.md, docs/n8n-import-validation.md, and docs/provider-auth-plan.md. -First, verify the current Phase 11 branch without changing behavior: -- run python3 -m py_compile outreach_filter.py botfucker/*.py +First, verify the current Phase 12 branch without changing behavior: +- run python3 scripts/validate_n8n_workflow_exports.py +- run python3 -m py_compile outreach_filter.py botfucker/*.py scripts/validate_n8n_workflow_exports.py - run python3 -m unittest discover -s tests -v -- inspect botfucker/yolo_policy.py and tests/test_yolo_guardrails.py -- confirm live provider actions require explicit YOLO guardrails +- inspect docs/n8n-import-validation.md and samples/approved-actions.sample.json +- confirm n8n workflows include explicit ids, are inactive, and use /home/node/.n8n-files for Read/Write Files paths -Then review Phase 11 only: Guarded YOLO Mode. +Then review Phase 12 only: Real n8n Import/Dry-Run Validation. -Check that YOLO is disabled by default, requires the exact confirmation phrase, supports emergency stop, gates provider actions by allowlist/classification/confidence/daily limit/tone, and that legacy --live --auto-approve cannot silently mutate mail without a YOLO policy. +Check that both workflows imported into n8n-vps as inactive/manual, approved-action bridge executed sample-only dry-run, final output was provider_execution:not_performed, and cleanup removed validation rows/temp files. Do not add real OAuth. Do not add provider credentials. Do not attach Gmail/Microsoft/IMAP/SMTP mutation credentials. Do not enable live n8n provider actions. Preserve the provider boundary: live provider execution remains separately reviewed and guarded. ``` ## Team Plan -- **Amy**: orchestration and scope control. She keeps the product from wandering into OAuth swamp country before n8n dry-run validation is real. -- **Chip**: owns n8n import/export compatibility fixes if Phase 11 review passes. -- **Rex**: security veto on YOLO guardrails, live-action safety gates, credential absence, provider-boundary isolation, and XSS regressions. -- **Gus**: local demo verification, n8n import/dry-run validation, CLI ergonomics, CI, and operator docs. -- **Fred**: n8n version/node compatibility research only; no direct OAuth implementation yet. +- **Amy**: orchestration and scope control. She keeps the product from wandering into OAuth swamp country before bridge promotion is reviewed. +- **Chip**: owns reviewed action bridge promotion docs/tests if Phase 12 review passes. +- **Rex**: security veto on processed-audit dedupe, credential absence, live-action safety gates, provider-boundary isolation, and XSS regressions. +- **Gus**: n8n operator verification, dry-run bridge observability, cleanup steps, CI, and operator docs. +- **Fred**: provider sandbox/action-limit research only; no direct OAuth implementation yet. ## Known Follow-Up Issues diff --git a/README.md b/README.md index f7ee3eb..a23ce96 100644 --- a/README.md +++ b/README.md @@ -454,6 +454,29 @@ export BF_YOLO_EMERGENCY_STOP=false If any gate fails, BotFucker raises before the live provider action. Subtle? No. That is the point. +## Phase 12 n8n Import Validation + +Phase 12 proved the n8n exports import and dry-run on Kent's real n8n target (`n8n-vps`, n8n 2.18.5) without attaching provider mutation credentials. + +Artifacts: + +- `scripts/validate_n8n_workflow_exports.py` — static preflight for workflow exports +- `samples/approved-actions.sample.json` — fake approved-action bundle +- `docs/n8n-import-validation.md` — exact import/dry-run/cleanup procedure and result + +Actual result: + +- both workflows imported inactive/manual +- approved-action bridge executed sample-only dry-run +- output stayed `provider_execution: not_performed` +- cleanup removed validation workflows and sample execution rows + +Compatibility fixes shipped: + +- workflow exports include explicit `id` fields for n8n CLI import +- file paths use `/home/node/.n8n-files`, because n8n 2.18.5 blocks arbitrary local file paths +- approved-action bridge parses `readWriteFile` binary JSON via `getBinaryDataBuffer` + ## Test Before Going Live Compile-check the script and package: diff --git a/ROADMAP.md b/ROADMAP.md index 930e187..6e0883f 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -269,18 +269,23 @@ Acceptance criteria: - Never silently enables aggressive/legal-ish replies. - Blocks provider actions that fail allowlist/confidence/daily-limit/emergency-stop gates. -### Phase 12 — Real n8n Import/Dry-Run Validation +### Phase 12 — Real n8n Import/Dry-Run Validation ✅ + +Status: implemented on `phase-12-n8n-import-dry-run-validation`. Goal: test the packaged n8n workflows against Kent's actual n8n instance without activating live provider mutations. -Deliverables: +Delivered: -- import `docs/n8n-workflow.json` into n8n-vps or local n8n test instance -- import `docs/n8n-approved-action-bridge.json` inactive/dry-run -- confirm workflow node compatibility with the installed n8n version -- confirm environment variable/file-path assumptions -- run sample-only dry-run payloads end-to-end -- document any n8n import/export fixes +- added `scripts/validate_n8n_workflow_exports.py` static preflight validator +- added `samples/approved-actions.sample.json` safe fake approved-action bundle +- added `docs/n8n-import-validation.md` with exact procedure and actual results +- imported both workflows into `n8n-vps` running n8n 2.18.5 +- confirmed both imported as inactive/manual workflows +- executed the approved-action bridge dry-run on sample data only +- confirmed dry-run output: `would_execute: approve_warning`, `provider_execution: not_performed` +- cleaned up validation workflows, related sample execution rows, and temp files +- fixed n8n compatibility issues: workflow `id` required, files must live under `/home/node/.n8n-files`, readWriteFile JSON arrives as binary and must be parsed with `getBinaryDataBuffer` Acceptance criteria: @@ -289,6 +294,26 @@ Acceptance criteria: - No Gmail/Microsoft/IMAP/SMTP mutation credentials are attached. - Activation remains manual and reviewed. +### Phase 13 — Review Action Bridge Promotion Plan + +Goal: define the reviewed path from dry-run logs to a live provider bridge without adding OAuth/provider mutation directly to BotFucker core. + +Deliverables: + +- operator checklist for promoting one provider action type at a time +- explicit credential ownership in n8n only +- audit/dedupe persistence design for processed `audit_id` values +- rollback/emergency-stop steps +- provider-specific sandbox/manual test plan +- security review checklist before any live provider action node is connected + +Acceptance criteria: + +- Live bridge remains separate from BotFucker core. +- Only reviewed approved-action records are eligible. +- Every provider mutation is idempotent, audited, and reversible where possible. +- No live provider credentials are committed or exported. + ## Local Kodex/Codex Demo Plan Kent is pulling this locally onto Kodex/Codex next. The demo should show what exists now, not pretend Phase 8 is already done. Revolutionary concept, apparently. @@ -335,23 +360,23 @@ Use fake or sanitized JSON only. Real mailbox payloads stay out of the repo. ## Near-Term Recommendation -Next PR should be **Phase 12: Real n8n Import/Dry-Run Validation**, not OAuth implementation. +Next PR should be **Phase 13: Reviewed Action Bridge Promotion Plan**, not OAuth implementation. Recommended scope: -- import both n8n JSON workflows into Kent's actual n8n test instance or a local matching n8n version -- keep workflows inactive/manual and sample-data-only -- validate node compatibility, environment variables, and file paths -- run the dry-run path end-to-end with fake/sample payloads -- document any import/export fixes -- attach no Gmail/Microsoft/IMAP/SMTP mutation credentials +- define how one provider action type graduates from dry-run to live review +- keep credentials in n8n only +- design persistent processed-`audit_id` state +- document rollback and emergency stop +- require provider-specific sandbox/manual tests +- require Rex/Gus security/ops review before any live mutation node is connected -OAuth can still wait. We now have import, review, export, bridge, optional LLM classification, and YOLO guardrails. Next: prove the n8n package actually imports and dry-runs in the real beast instead of admiring the JSON like it’s a museum artifact. +OAuth can still wait. We proved the n8n package imports and dry-runs in the real beast. Next is a promotion plan for one action type at a time, because wiring live mail mutation without a checklist is how you manufacture regret at scale. ## Team Utilization - **Amy**: orchestrates scope, keeps phases honest, and blocks shiny-object OAuth detours. -- **Chip**: owns n8n workflow import/export fixes if real n8n exposes compatibility issues. -- **Rex**: reviews live-action safety gates, credential absence, and provider-boundary isolation. -- **Gus**: verifies n8n import, environment assumptions, local demo steps, CI, and operator docs. -- **Fred**: researches n8n node compatibility/version quirks only; no direct OAuth implementation yet. +- **Chip**: owns bridge promotion docs/tests and any dry-run-to-live scaffolding after review. +- **Rex**: reviews credential absence, processed-audit dedupe, live-action safety gates, and rollback. +- **Gus**: verifies n8n operator steps, imports, cleanup, and bridge observability. +- **Fred**: researches provider-specific sandbox/action constraints only; no direct OAuth implementation yet. diff --git a/docs/n8n-approved-action-bridge.json b/docs/n8n-approved-action-bridge.json index 9f7641d..205a2fb 100644 --- a/docs/n8n-approved-action-bridge.json +++ b/docs/n8n-approved-action-bridge.json @@ -1,4 +1,5 @@ { + "id": "botfucker-approved-action-bridge-dry-run-v1", "name": "BotFucker Approved Action Bridge Dry Run", "nodes": [ { @@ -7,30 +8,39 @@ "name": "Manual Trigger", "type": "n8n-nodes-base.manualTrigger", "typeVersion": 1, - "position": [-840, 0] + "position": [ + -840, + 0 + ] }, { "parameters": { "operation": "read", - "fileSelector": "={{ $env.BOTFUCKER_APPROVED_ACTIONS || 'approved-actions.json' }}", + "fileSelector": "={{ $env.BOTFUCKER_APPROVED_ACTIONS || '/home/node/.n8n-files/approved-actions.sample.json' }}", "options": {} }, "id": "read-approved-actions-json", "name": "Read approved-actions.json", "type": "n8n-nodes-base.readWriteFile", "typeVersion": 1, - "position": [-600, 0], + "position": [ + -600, + 0 + ], "notes": "Reads the approved-action export created by BotFucker review_cli. Dry-run bridge only." }, { "parameters": { - "jsCode": "const rows = $input.all();\nconst first = rows[0] || {};\nlet bundle = first.json || {};\nif (Array.isArray(bundle)) {\n bundle = bundle[0] || {};\n}\nif (bundle.schema !== 'botfucker.approved_actions.v1') {\n throw new Error(`Unexpected BotFucker approved action schema: ${bundle.schema || 'missing'}`);\n}\nif (bundle.safety_scope !== 'provider_action_export_only') {\n throw new Error('Approved action bundle safety_scope must be provider_action_export_only');\n}\nconst processedRaw = $env.BOTFUCKER_PROCESSED_AUDIT_IDS || '';\nconst processedAuditIds = new Set(processedRaw.split(',').map((value) => value.trim()).filter(Boolean));\nconst actions = Array.isArray(bundle.actions) ? bundle.actions : [];\nconst validActions = actions.filter((action) => {\n if (!action.audit_id || processedAuditIds.has(action.audit_id)) return false;\n if (action.provider_execution !== 'not_performed') return false;\n if (action.safety_scope !== 'provider_action_export_only') return false;\n if (action.approved_action !== 'approve_warning') return false;\n return true;\n});\nreturn validActions.map((action) => ({\n json: {\n ...action,\n dry_run: true,\n bridge_status: 'validated_not_executed',\n dedupe_key: action.audit_id\n }\n}));" + "jsCode": "const rows = $input.all();\nconst first = rows[0] || {};\nlet bundle = first.json || {};\nif (first.binary && first.binary.data) {\n const buffer = await this.helpers.getBinaryDataBuffer(0, 'data');\n bundle = JSON.parse(buffer.toString('utf8'));\n}\nif (Array.isArray(bundle)) {\n bundle = bundle[0] || {};\n}\nif (bundle.schema !== 'botfucker.approved_actions.v1') {\n throw new Error(`Unexpected BotFucker approved action schema: ${bundle.schema || 'missing'}`);\n}\nif (bundle.safety_scope !== 'provider_action_export_only') {\n throw new Error('Approved action bundle safety_scope must be provider_action_export_only');\n}\nconst processedRaw = $env.BOTFUCKER_PROCESSED_AUDIT_IDS || '';\nconst processedAuditIds = new Set(processedRaw.split(',').map((value) => value.trim()).filter(Boolean));\nconst actions = Array.isArray(bundle.actions) ? bundle.actions : [];\nconst validActions = actions.filter((action) => {\n if (!action.audit_id || processedAuditIds.has(action.audit_id)) return false;\n if (action.provider_execution !== 'not_performed') return false;\n if (action.safety_scope !== 'provider_action_export_only') return false;\n if (action.approved_action !== 'approve_warning') return false;\n return true;\n});\nreturn validActions.map((action) => ({\n json: {\n ...action,\n dry_run: true,\n bridge_status: 'validated_not_executed',\n dedupe_key: action.audit_id\n }\n}));" }, "id": "validate-and-dedupe-approved-actions", "name": "Validate And Dedupe Approved Actions", "type": "n8n-nodes-base.code", "typeVersion": 2, - "position": [-360, 0], + "position": [ + -360, + 0 + ], "notes": "Validates botfucker.approved_actions.v1, filters already processed audit_id values, and keeps execution dry-run only." }, { @@ -41,7 +51,10 @@ "name": "Dry Run Provider Action Log", "type": "n8n-nodes-base.code", "typeVersion": 2, - "position": [-120, 0], + "position": [ + -120, + 0 + ], "notes": "Logs what the future provider bridge would attempt. It does not execute mailbox changes." }, { @@ -54,18 +67,45 @@ "name": "Safety Boundary Notes", "type": "n8n-nodes-base.stickyNote", "typeVersion": 1, - "position": [-600, -360] + "position": [ + -600, + -360 + ] } ], "connections": { "Manual Trigger": { - "main": [[{"node": "Read approved-actions.json", "type": "main", "index": 0}]] + "main": [ + [ + { + "node": "Read approved-actions.json", + "type": "main", + "index": 0 + } + ] + ] }, "Read approved-actions.json": { - "main": [[{"node": "Validate And Dedupe Approved Actions", "type": "main", "index": 0}]] + "main": [ + [ + { + "node": "Validate And Dedupe Approved Actions", + "type": "main", + "index": 0 + } + ] + ] }, "Validate And Dedupe Approved Actions": { - "main": [[{"node": "Dry Run Provider Action Log", "type": "main", "index": 0}]] + "main": [ + [ + { + "node": "Dry Run Provider Action Log", + "type": "main", + "index": 0 + } + ] + ] } }, "settings": { @@ -74,9 +114,15 @@ "pinData": {}, "staticData": null, "tags": [ - {"name": "botfucker"}, - {"name": "approved-action-bridge"}, - {"name": "dry-run"} + { + "name": "botfucker" + }, + { + "name": "approved-action-bridge" + }, + { + "name": "dry-run" + } ], "active": false, "versionId": "botfucker-approved-action-bridge-dry-run-v1" diff --git a/docs/n8n-import-validation.md b/docs/n8n-import-validation.md new file mode 100644 index 0000000..16dd7eb --- /dev/null +++ b/docs/n8n-import-validation.md @@ -0,0 +1,107 @@ +# n8n Import Validation — Phase 12 + +Phase 12 validates the packaged BotFucker n8n workflows against a real n8n target while keeping everything inactive, dry-run, and sample-only. The validation target used for this phase is `n8n-vps`, running n8n 2.18.5 in Docker. + +This is still not OAuth work. no Gmail, no Microsoft, no IMAP, no SMTP, and no provider mutation credentials are attached. We are proving that the workflow exports import and that the approved-action bridge can exercise a dry-run path with sample data. Very glamorous. Like QA, but with fewer party hats. + +## Artifacts under validation + +- `docs/n8n-workflow.json` — local review import starter workflow. +- `docs/n8n-approved-action-bridge.json` — approved-action bridge dry-run workflow. +- `samples/approved-actions.sample.json` under n8n's allowed local-files directory, `/home/node/.n8n-files`. n8n 2.18.5 blocks Read/Write Files access outside that directory unless configured otherwise. +- `scripts/validate_n8n_workflow_exports.py` — static preflight check before n8n import. + +## Static preflight + +Run locally before copying anything to n8n: + +```bash +python3 scripts/validate_n8n_workflow_exports.py +``` + +The script verifies both workflow exports are inactive, manual-triggered, credential-free, and free of obvious provider mutation nodes/terms. It also verifies the approved-action bridge includes `botfucker.approved_actions.v1`, `audit_id`, and `dry_run` markers. + +## Real n8n target + +Target checked during Phase 12: + +```bash +ssh n8n-vps 'docker exec n8n-n8n-1 n8n --version' +# 2.18.5 +``` + +Container: + +```text +n8n-n8n-1 +``` + +## Sample-only import procedure + +Copy workflow exports and sample bundle to the n8n host/container: + +```bash +scp docs/n8n-workflow.json docs/n8n-approved-action-bridge.json samples/approved-actions.sample.json n8n-vps:/tmp/ +ssh n8n-vps 'docker exec n8n-n8n-1 mkdir -p /tmp/botfucker-phase12' +ssh n8n-vps 'docker cp /tmp/n8n-workflow.json n8n-n8n-1:/tmp/botfucker-phase12/n8n-workflow.json' +ssh n8n-vps 'docker cp /tmp/n8n-approved-action-bridge.json n8n-n8n-1:/tmp/botfucker-phase12/n8n-approved-action-bridge.json' +ssh n8n-vps 'docker cp /tmp/approved-actions.sample.json n8n-n8n-1:/home/node/.n8n-files/approved-actions.sample.json' +``` + +Import as inactive/manual workflows: + +```bash +ssh n8n-vps 'docker exec n8n-n8n-1 n8n import:workflow --input=/tmp/botfucker-phase12/n8n-workflow.json' +ssh n8n-vps 'docker exec n8n-n8n-1 n8n import:workflow --input=/tmp/botfucker-phase12/n8n-approved-action-bridge.json' +``` + +After import, verify both workflow rows are `active = 0` before any execution attempt. + +## Dry-run execution target + +Only the approved-action bridge is suitable for Phase 12 dry-run execution because it reads a local sample bundle and logs `would_execute` records with `provider_execution: not_performed`. + +Use the sample bundle only: + +```bash +ssh n8n-vps 'docker exec -e BOTFUCKER_APPROVED_ACTIONS=/home/node/.n8n-files/approved-actions.sample.json n8n-n8n-1 n8n execute --id=' +``` + +Expected behavior: + +- schema `botfucker.approved_actions.v1` is accepted +- `audit_id` is used as the dedupe key +- output is dry-run / `would_execute` +- `provider_execution` remains `not_performed` +- no provider mailbox mutation occurs + +## Cleanup + +Delete validation workflows after the test unless Kent explicitly wants them left in n8n. + +Deletion can be done through the n8n UI or by removing only the validation rows from n8n's SQLite tables by exact workflow name. If doing SQL cleanup, delete validation workflows from `workflow_entity` and related rows such as `workflow_history`, `shared_workflow`, `workflow_statistics`, and `workflow_dependency` by exact workflow ID. Do not touch unrelated workflows. Yes, the obvious warning is necessary, because databases are where confidence goes to die. + +## Validation result + +Actual Phase 12 run on `n8n-vps` / n8n 2.18.5: + +- Static preflight passed with `python3 scripts/validate_n8n_workflow_exports.py`. +- Both workflows imported successfully as inactive/manual workflows after adding explicit workflow IDs: + - `botfucker-local-review-import-v1` + - `botfucker-approved-action-bridge-dry-run-v1` +- Import verification showed both `active = 0`. +- Approved-action bridge dry-run executed successfully against `samples/approved-actions.sample.json` copied to `/home/node/.n8n-files/approved-actions.sample.json`. +- Final dry-run node emitted: + - `dry_run: true` + - `would_execute: approve_warning` + - `audit_id: audit-sample-0001` + - `provider_execution: not_performed` + - `bridge_status: dry_run_logged_only` +- No Gmail, Microsoft, IMAP, SMTP, or provider mutation credentials were attached. +- Cleanup completed: validation workflow rows, related workflow history/dependency/statistic/share rows, sample execution rows, and temp sample files were removed. + +Compatibility fixes discovered the hard way, because naturally n8n had opinions: + +1. n8n 2.18.5 CLI import requires workflow exports to include non-null `id` values. +2. Read/Write Files access is restricted to `/home/node/.n8n-files` on this target. +3. `readWriteFile` returns file contents as binary data; the Code node must parse with `await this.helpers.getBinaryDataBuffer(0, 'data')` before validating the approved-action schema. diff --git a/docs/n8n-workflow.json b/docs/n8n-workflow.json index 23e7789..3da1048 100644 --- a/docs/n8n-workflow.json +++ b/docs/n8n-workflow.json @@ -1,4 +1,5 @@ { + "id": "botfucker-local-review-import-v1", "name": "BotFucker Local Review Import", "nodes": [ { @@ -63,7 +64,7 @@ { "parameters": { "operation": "write", - "fileName": "={{ $env.BOTFUCKER_N8N_MESSAGES || 'n8n-messages.json' }}", + "fileName": "={{ $env.BOTFUCKER_N8N_MESSAGES || '/home/node/.n8n-files/n8n-messages.json' }}", "dataPropertyName": "data", "options": {} }, @@ -79,7 +80,7 @@ }, { "parameters": { - "command": "cd \"${BOTFUCKER_REPO:-/path/to/BotFucker}\" && python3 -m botfucker.review_cli --db \"${BOTFUCKER_REVIEW_DB:-botfucker_review.sqlite3}\" import-webhook-json \"${BOTFUCKER_N8N_MESSAGES:-n8n-messages.json}\"" + "command": "cd \"${BOTFUCKER_REPO:-/path/to/BotFucker}\" && python3 -m botfucker.review_cli --db \"${BOTFUCKER_REVIEW_DB:-botfucker_review.sqlite3}\" import-webhook-json \"${BOTFUCKER_N8N_MESSAGES:-/home/node/.n8n-files/n8n-messages.json}\"" }, "id": "import-into-local-review-queue", "name": "Import Into Local Review Queue", diff --git a/docs/provider-auth-plan.md b/docs/provider-auth-plan.md index 55b16ac..7f96518 100644 --- a/docs/provider-auth-plan.md +++ b/docs/provider-auth-plan.md @@ -159,9 +159,9 @@ Do not add live provider whitelist/blacklist mutations. ## Next engineering step -The next engineering step should be **Real n8n Import/Dry-Run Validation**, not OAuth. +The next engineering step should be **Reviewed Action Bridge Promotion Plan**, not OAuth. -Approved action export, the dry-run bridge contract, the optional LLM classifier hook, and YOLO guardrails now exist. Next, prove the packaged n8n workflows import and execute dry-run/sample-only paths against a real n8n target without attaching provider mutation credentials. +Approved action export, the dry-run bridge contract, the optional LLM classifier hook, YOLO guardrails, and real n8n import/dry-run validation now exist. Next, define how one reviewed provider action type can graduate from dry-run to live bridge while keeping credentials in n8n only, persisting processed `audit_id` dedupe state, and requiring rollback/security/operator review before any live mutation node is connected. ## Acceptance criteria for Phase 7 diff --git a/samples/approved-actions.sample.json b/samples/approved-actions.sample.json new file mode 100644 index 0000000..d3794d1 --- /dev/null +++ b/samples/approved-actions.sample.json @@ -0,0 +1,20 @@ +{ + "schema": "botfucker.approved_actions.v1", + "generated_at": "2026-05-14T00:00:00Z", + "safety_scope": "provider_action_export_only", + "actions": [ + { + "audit_id": "audit-sample-0001", + "item_id": "sample-review-0001", + "message_id": "sample-message-0001@example.invalid", + "thread_id": "sample-thread-0001", + "provider": "sample", + "approved_action": "approve_warning", + "approved_actor": "local-reviewer@example.invalid", + "approved_at": "2026-05-14T00:00:00Z", + "draft_reply": "Sample warning reply for dry-run validation only.", + "provider_execution": "not_performed", + "safety_scope": "provider_action_export_only" + } + ] +} diff --git a/scripts/validate_n8n_workflow_exports.py b/scripts/validate_n8n_workflow_exports.py new file mode 100644 index 0000000..7aab73c --- /dev/null +++ b/scripts/validate_n8n_workflow_exports.py @@ -0,0 +1,117 @@ +#!/usr/bin/env python3 +"""Validate BotFucker n8n workflow exports before importing them. + +This is intentionally local/static. It checks the repo artifacts for the +properties that matter before an operator copies them into n8n: inactive, +manual-triggered, dry-run where required, and free of credential/provider +mutation nodes. It does not call n8n, Gmail, Microsoft, IMAP, SMTP, or any +provider API. +""" + +from __future__ import annotations + +import json +import sys +from pathlib import Path +from typing import Any + +REPO_ROOT = Path(__file__).resolve().parents[1] +WORKFLOW_PATHS = [ + REPO_ROOT / "docs" / "n8n-workflow.json", + REPO_ROOT / "docs" / "n8n-approved-action-bridge.json", +] + +FORBIDDEN_STRINGS = [ + "authorization", + "access_token", + "refresh_token", + "api_key", + "password", + "credential", +] + +FORBIDDEN_NODE_TERMS = [ + "gmail", + "microsoft", + "imap", + "smtp", + "emailSend", + "sendMail", + "deleteMessage", + "moveMessage", + "archive", + "label", +] + + +def load_json(path: Path) -> dict[str, Any]: + try: + payload = json.loads(path.read_text(encoding="utf-8")) + except json.JSONDecodeError as exc: + raise AssertionError(f"{path}: invalid JSON: {exc}") from exc + if not isinstance(payload, dict): + raise AssertionError(f"{path}: workflow export must be a JSON object") + return payload + + +def node_names(workflow: dict[str, Any]) -> set[str]: + return {str(node.get("name", "")) for node in workflow.get("nodes", [])} + + +def validate_workflow(path: Path) -> list[str]: + workflow = load_json(path) + serialized = json.dumps(workflow, sort_keys=True) + serialized_lower = serialized.lower() + errors: list[str] = [] + + if workflow.get("active") is not False: + errors.append(f"{path.name}: active must be false for import validation") + if "nodes" not in workflow or not isinstance(workflow["nodes"], list): + errors.append(f"{path.name}: nodes list missing") + if "connections" not in workflow or not isinstance(workflow["connections"], dict): + errors.append(f"{path.name}: connections object missing") + if "Manual Trigger" not in node_names(workflow): + errors.append(f"{path.name}: Manual Trigger node missing") + if "manualTrigger" not in serialized: + errors.append(f"{path.name}: manualTrigger node type missing") + + for forbidden in FORBIDDEN_STRINGS: + if forbidden in serialized_lower: + errors.append(f"{path.name}: forbidden credential-like string found: {forbidden}") + + node_types = "\n".join(str(node.get("type", "")) for node in workflow.get("nodes", [])) + node_type_lower = node_types.lower() + for term in FORBIDDEN_NODE_TERMS: + if term.lower() in node_type_lower: + errors.append(f"{path.name}: forbidden provider mutation node type found: {term}") + + if path.name == "n8n-approved-action-bridge.json": + if "botfucker.approved_actions.v1" not in serialized: + errors.append(f"{path.name}: approved-actions schema marker missing") + if "dry_run" not in serialized: + errors.append(f"{path.name}: dry_run marker missing") + if "audit_id" not in serialized: + errors.append(f"{path.name}: audit_id dedupe marker missing") + + return errors + + +def main() -> int: + all_errors: list[str] = [] + for path in WORKFLOW_PATHS: + all_errors.extend(validate_workflow(path)) + + if all_errors: + print("n8n workflow export validation failed:") + for error in all_errors: + print(f"- {error}") + return 1 + + print("n8n workflow export validation passed:") + for path in WORKFLOW_PATHS: + print(f"- {path.relative_to(REPO_ROOT)}") + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/tests/test_n8n_import_validation.py b/tests/test_n8n_import_validation.py new file mode 100644 index 0000000..080e698 --- /dev/null +++ b/tests/test_n8n_import_validation.py @@ -0,0 +1,63 @@ +import json +import unittest +from pathlib import Path + + +REPO_ROOT = Path(__file__).resolve().parents[1] +VALIDATION_GUIDE = REPO_ROOT / "docs" / "n8n-import-validation.md" +VALIDATOR_SCRIPT = REPO_ROOT / "scripts" / "validate_n8n_workflow_exports.py" +SAMPLE_ACTIONS = REPO_ROOT / "samples" / "approved-actions.sample.json" + + +class N8nImportValidationDocsTests(unittest.TestCase): + def test_validation_guide_documents_real_n8n_dry_run_and_cleanup(self): + guide = VALIDATION_GUIDE.read_text(encoding="utf-8") + + self.assertIn("n8n 2.18.5", guide) + self.assertIn("n8n-vps", guide) + self.assertIn("inactive", guide.lower()) + self.assertIn("dry-run", guide.lower()) + self.assertIn("sample-only", guide.lower()) + self.assertIn("/home/node/.n8n-files", guide) + self.assertIn("approved-actions.sample.json", guide) + self.assertIn("delete validation workflows", guide.lower()) + self.assertIn("no Gmail", guide) + self.assertIn("no Microsoft", guide) + self.assertIn("no IMAP", guide) + self.assertIn("no SMTP", guide) + self.assertIn("Cleanup completed", guide) + self.assertIn("getBinaryDataBuffer", guide) + self.assertIn("non-null `id`", guide) + self.assertIn("would_execute: approve_warning", guide) + self.assertIn("provider_execution: not_performed", guide) + self.assertIn("no provider mutation credentials", guide.lower()) + + def test_validator_script_checks_both_workflow_exports_and_forbidden_mutations(self): + script = VALIDATOR_SCRIPT.read_text(encoding="utf-8") + + self.assertIn("n8n-workflow.json", script) + self.assertIn("n8n-approved-action-bridge.json", script) + self.assertIn("active", script) + self.assertIn("manualTrigger", script) + self.assertIn("forbidden", script.lower()) + for phrase in ("gmail", "microsoft", "imap", "smtp", "emailSend", "access_token", "password"): + self.assertIn(phrase, script) + + def test_sample_approved_actions_bundle_is_safe_and_matches_schema(self): + bundle = json.loads(SAMPLE_ACTIONS.read_text(encoding="utf-8")) + serialized = json.dumps(bundle, sort_keys=True).lower() + + self.assertEqual(bundle["schema"], "botfucker.approved_actions.v1") + self.assertEqual(bundle["safety_scope"], "provider_action_export_only") + self.assertEqual(len(bundle["actions"]), 1) + action = bundle["actions"][0] + self.assertEqual(action["audit_id"], "audit-sample-0001") + self.assertEqual(action["approved_action"], "approve_warning") + self.assertEqual(action["provider_execution"], "not_performed") + self.assertIn("example.invalid", serialized) + for forbidden in ("authorization", "access_token", "refresh_token", "api_key", "password", "credential"): + self.assertNotIn(forbidden, serialized) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/test_n8n_workflow_docs.py b/tests/test_n8n_workflow_docs.py index af7c26b..889bfa4 100644 --- a/tests/test_n8n_workflow_docs.py +++ b/tests/test_n8n_workflow_docs.py @@ -15,6 +15,7 @@ def test_workflow_export_is_valid_json_without_credentials(self): workflow = json.loads(WORKFLOW_PATH.read_text(encoding="utf-8")) serialized = json.dumps(workflow, sort_keys=True).lower() + self.assertEqual(workflow["id"], "botfucker-local-review-import-v1") self.assertEqual(workflow["name"], "BotFucker Local Review Import") self.assertIn("nodes", workflow) self.assertIn("connections", workflow) @@ -82,6 +83,7 @@ def test_action_bridge_workflow_is_valid_dry_run_json_without_credentials(self): workflow = json.loads(ACTION_BRIDGE_WORKFLOW_PATH.read_text(encoding="utf-8")) serialized = json.dumps(workflow, sort_keys=True).lower() + self.assertEqual(workflow["id"], "botfucker-approved-action-bridge-dry-run-v1") self.assertEqual(workflow["name"], "BotFucker Approved Action Bridge Dry Run") self.assertFalse(workflow["active"]) self.assertIn("nodes", workflow) @@ -117,6 +119,8 @@ def test_action_bridge_workflow_connections_keep_provider_actions_disabled(self) validation_code = nodes["Validate And Dedupe Approved Actions"]["parameters"]["jsCode"] dry_run_code = nodes["Dry Run Provider Action Log"]["parameters"]["jsCode"] + self.assertIn("getBinaryDataBuffer", validation_code) + self.assertIn("JSON.parse", validation_code) self.assertIn("botfucker.approved_actions.v1", validation_code) self.assertIn("audit_id", validation_code) self.assertIn("processedAuditIds", validation_code) From 2d1f9d38b208c4008c5d9279dd27bc40e4f2e37b Mon Sep 17 00:00:00 2001 From: Amy Hermes Date: Thu, 14 May 2026 20:31:09 -0400 Subject: [PATCH 2/2] docs: add tomorrow restart notes --- HANDOFF.md | 19 +++++++++++++++---- ROADMAP.md | 8 ++++++++ 2 files changed, 23 insertions(+), 4 deletions(-) diff --git a/HANDOFF.md b/HANDOFF.md index bdd147a..cc2d206 100644 --- a/HANDOFF.md +++ b/HANDOFF.md @@ -6,7 +6,8 @@ - Default branch: `main` - Latest merged milestone: Phase 11 guarded YOLO policy (`feat: add guarded yolo policy`, PR #12) - Current working branch: `phase-12-n8n-import-dry-run-validation` -- Current PR target: Phase 12 real n8n import/dry-run validation +- Current PR target: Phase 12 real n8n import/dry-run validation — PR #13 open and CI green: `https://github.com/Jdelg718/BotFucker/pull/13` +- Tomorrow start: merge PR #13 if still green, then branch from updated `main` for Phase 13 Reviewed Action Bridge Promotion Plan - Current local demo target: demonstrate deterministic local review, optional mocked LLM classifier fallback/validation, approved-action export, dry-run n8n bridge contract, fail-closed YOLO policy checks, and real n8n import validation results - Current promo artifact: `promo/botfucker-animated-explainer/renders/botfucker-animated-explainer_narrated-final.mp4` @@ -69,7 +70,8 @@ These are non-negotiable: Run from repo root: ```bash -python3 -m py_compile outreach_filter.py botfucker/*.py +python3 scripts/validate_n8n_workflow_exports.py +python3 -m py_compile outreach_filter.py botfucker/*.py scripts/validate_n8n_workflow_exports.py python3 -m unittest discover -s tests -v ``` @@ -124,7 +126,8 @@ If this handoff update has been merged after that, the top commit will be newer. ### Verify locally ```bash -python3 -m py_compile outreach_filter.py botfucker/*.py +python3 scripts/validate_n8n_workflow_exports.py +python3 -m py_compile outreach_filter.py botfucker/*.py scripts/validate_n8n_workflow_exports.py python3 -m unittest discover -s tests -v ``` @@ -309,10 +312,18 @@ Do not add real OAuth. Do not add provider credentials. Do not attach Gmail/Micr - Deterministic classifier still needs real-world tuning. - Optional LLM classifier exists only as a provider hook; no real provider wiring or credentials are implemented. - No production OAuth yet. -- n8n workflow package needs real local import testing against Kent's actual n8n instance before activation. +- Real n8n import validation passed on n8n-vps with sample-only dry-run and cleanup; do not activate those workflows without a separate reviewed bridge-promotion plan. - n8n approved action bridge is dry-run only; live provider actions still need a separate explicit reviewed workflow. - YOLO guardrails exist but live provider actions still require explicit operator configuration and must not be casually enabled. +## Tomorrow Restart + +- PR #13 is open and CI green: `https://github.com/Jdelg718/BotFucker/pull/13`. +- First move tomorrow: re-check PR #13, merge if green, pull `main`, branch Phase 13. +- Phase 13 target: **Reviewed Action Bridge Promotion Plan**. +- Do **not** add OAuth, provider credentials, or live n8n provider mutation nodes. +- Keep live provider execution separate, reviewed, audited, deduped by processed `audit_id`, rollback-ready, and security/ops-reviewed. + ## Product Voice BotFucker should be funny and hostile to spam, but serious about safety. diff --git a/ROADMAP.md b/ROADMAP.md index 6e0883f..c1a3326 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -373,6 +373,14 @@ Recommended scope: OAuth can still wait. We proved the n8n package imports and dry-runs in the real beast. Next is a promotion plan for one action type at a time, because wiring live mail mutation without a checklist is how you manufacture regret at scale. +### Tomorrow restart checklist + +1. Re-check PR #13 CI and mergeability. +2. Squash-merge PR #13 into `main` if still green. +3. Pull updated `main` and create Phase 13 branch. +4. Build **Reviewed Action Bridge Promotion Plan** only — still no OAuth, no provider credentials, no live action node activation. +5. Use tests/docs to prove the promotion plan keeps credentials in n8n, persists processed `audit_id` state, and requires rollback/security/operator review. + ## Team Utilization - **Amy**: orchestrates scope, keeps phases honest, and blocks shiny-object OAuth detours.