diff --git a/AGENTS.md b/AGENTS.md index a5a5420..a27948b 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -55,6 +55,8 @@ This repository is the `skill2workflow` open-source harness. - Control snapshot: `PYTHONPATH=src python3 -m skill2workflow.cli control-snapshot --state-dir /tmp/skill2workflow-control -o /tmp/skill2workflow-control-snapshot.json` - First-run demo: `python3 scripts/demo_bootstrap.py --work-dir /tmp/skill2workflow-demo` - Pilot smoke: `python3 scripts/pilot_playbook_smoke.py --work-dir /tmp/skill2workflow-pilot` +- Controlled Lark pilot: `python3 scripts/controlled_lark_pilot.py --help` +- Controlled Lark preflight: `python3 scripts/controlled_lark_pilot.py preflight --input /tmp/skill2workflow-private-case.json` - Schedule smoke: `python3 scripts/schedule_smoke.py --work-dir /tmp/skill2workflow-schedule-loop29` - Package smoke: `python3 scripts/package_smoke.py --work-dir /tmp/skill2workflow-package-smoke` - Secret hygiene: `python3 scripts/secret_hygiene.py examples/workflows` diff --git a/README.md b/README.md index 42c348b..e441c89 100644 --- a/README.md +++ b/README.md @@ -498,9 +498,9 @@ ROADMAP.md # Open-source delivery roadmap ## Roadmap -Current maturity: Local Evaluation. The local-first harness covers all five approved architecture layers, and Delivery Loops 1-38 are complete. +Current maturity: Local Evaluation. The local-first harness covers all five approved architecture layers, and Delivery Loops 1-39 are complete. -The active priority is Loop 39: implement only the readiness-approved Lark/Feishu `create_task` live action behind explicit opt-in while keeping dry-run behavior as the default. +Loop 40 is deferred after a normalized real Pilot failure; no further live calls are authorized under that Pilot. A future controlled real-team Pilot requires fresh authorization and the local no-network preflight before its separate human approval. The production direction is a self-hosted, single-tenant runtime for one team. See `ROADMAP.md` for the production-readiness gates, rolling Loop queue, acceptance evidence, and deferred boundaries. @@ -510,6 +510,8 @@ See: - `ROADMAP.md` - `docs/authoring.md` - `docs/connectors.md` +- `docs/controlled-pilot-deferral-review.md` +- `docs/controlled-live-pilot.md` - `docs/credential-boundary.md` - `docs/examples.md` - `docs/pilot-playbook.md` diff --git a/ROADMAP.md b/ROADMAP.md index d28d5f1..5e26b91 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -12,11 +12,11 @@ Workflow DSL remains the authoritative execution source of truth. LiteGraph and - Published release: `v0.1.0` - Workflow DSL compatibility line: `0.1.x` artifacts using `schema_version: "0.1.0"` -- Completed delivery loops: 1-38 +- Completed delivery loops: 1-39 - Current maturity: Local Evaluation -- Active loop: Loop 39, Scoped Live Lark Task Connector +- Active loop: None; Loop 40 is deferred pending a new partner-approved pilot - Next maturity gate: Controlled Live Pilot -- Next decision: validate the scoped live action in a controlled pilot or explicitly defer broader live behavior +- Next decision: select and authorize a new controlled Pilot only after post-incident review ## Production Readiness Path @@ -28,9 +28,9 @@ The repository can compile, validate, publish, trigger, execute, pause, resume, ### Controlled Live Pilot -**Target loops:** 39-40. +**Target loops:** 40. -This gate requires one explicitly enabled live connector action plus controlled pilot evidence. It does not imply general live SaaS readiness. +This gate requires the completed scoped live connector action plus controlled pilot evidence. It does not imply general live SaaS readiness. ### Self-hosted Beta @@ -48,78 +48,41 @@ Candidate evidence includes backup and restore, upgrade and migration policy, ca ## Active Loop -### Loop 39: Scoped Live Lark Task Connector +### Loop 40: Controlled Live Connector Pilot -**Status:** Next engineering loop. +**Status:** Deferred after a normalized provider validation failure in the controlled real-team Pilot. -**Goal:** Implement only the Loop 38-approved Lark/Feishu `create_task` live action behind explicit opt-in while keeping dry-run mode as the default. +**Prior basis:** The Lark/Feishu task connector has package-level and pilot-workflow dry-run evidence, including the sales renewal risk workflow after a manual control gate. Loop 39 also produced the redacted connector-validation note at `docs/lark-live-connector-validation.md`. Live behavior remains limited to the fixed `create_task` action. The one scoped live connector validation is not the controlled real-team business-workflow pilot required for Loop 40. -**Why now:** Loop 36 proved the out-of-core package boundary, Loop 37 proved the connector in a sales renewal risk workflow after a manual control gate, and Loop 38 approved implementation after package-level and pilot-workflow dry-run evidence. The remaining risk is disciplined live execution without weakening credential isolation, duplicate prevention, audit redaction, rollback, or Workflow DSL compatibility. +**Outcome:** The Pilot retained its history, completed the required human-rejection and safety exercises, and recorded a private partner/operator `defer` decision. One approved live run completed, but a later approved attempt failed with normalized `validation_failed`; therefore the five-day acceptance threshold was not met. No finalization or repository evidence export occurred. -**Decision boundary:** Loop 38 approved only scoped live `create_task` work. Any broader Lark/Feishu API behavior requires another readiness review. The full decision is recorded in `docs/lark-live-connector-readiness.md`. +**Safety outcome:** The failed run remains in owner-only Pilot state. No retry was issued, no subsequent live run was approved, and no raw provider message, task data, identifier, or credential was exported. A recorded Pilot decision now closes that workspace to subsequent starts and decisions, and initialization never replaces its Charter. The connector also provides a no-Vault, no-network `preflight` check that constructs the fixed request shape before a future human-gated run. -Approved scope: +**Deferral review:** [`docs/controlled-pilot-deferral-review.md`](docs/controlled-pilot-deferral-review.md) records the supported facts, the intentionally unconfirmed provider root cause, and the fresh-authorization re-entry gate. -- Connector id and kind: `lark_task` -- Operation: `create_task` -- Live mode: `live` -- Default mode: `dry_run` -- Credential handle: `lark_bot_access_token` -- Idempotency key: derived from `workflow_id + version + run_id + node_id` -- Test transport: fake Lark HTTP receiver or injected fake transport; no live network in CI -- Evidence: compact connector and audit metadata only +**Re-entry boundary:** A new Pilot requires fresh partner and operator authorization, a new valid charter, a new private case, a successful local `preflight`, and a separate explicit approval for each real create. The old failed run must never be retried or replaced. Live behavior remains limited to the fixed `create_task` action; any broader Lark/Feishu API behavior requires another readiness review. -Implementation order: - -1. Add failing tests for opt-in, credential resolution, success, API failures, timeout, malformed responses, and redaction. -2. Add idempotency and duplicate-prevention tests before outbound request code. -3. Implement the minimal live `create_task` action. -4. Prove existing dry-run tests and smokes remain unchanged. -5. Update connector documentation without changing Workflow DSL compatibility. - -Acceptance criteria: - -- The project can create one live Lark/Feishu task through an explicitly enabled local connector path. -- Live behavior requires a feature flag or equivalent explicit opt-in. -- Dry-run remains the default for examples, CI, and contributor onboarding. -- Credential handling and audit redaction rules are explicit in code, tests, and docs. -- Resolved credentials, authorization headers, raw task values, raw request bodies, and raw response payloads never enter run state, audit, snapshots, or connector summaries. -- Duplicate task creation is blocked for the same derived idempotency key. -- `401 or 403`, rate limits, network timeouts, validation failures, and malformed responses become normalized failures where possible. -- The live path can be disabled or reverted without changing Workflow DSL compatibility. - -Required verification: +The dry-run behavioral baseline remains available through: ```bash -PYTHONPATH=src python3 -m unittest discover -s tests -v -python3 -m py_compile src/skill2workflow/*.py -python3 scripts/secret_hygiene.py examples/workflows python3 scripts/lark_task_pilot_smoke.py --work-dir /tmp/skill2workflow-lark-task-pilot -git diff --check ``` -Explicitly excluded from Loop 39: - -- OAuth and token refresh -- Hosted callbacks or ingress -- Automatic connector discovery or package installation -- Marketplace indexing -- Queues or production scheduling -- Other Lark/Feishu APIs or operations +Loop 40 is not complete and does not advance maturity. Its private decision is not a substitute for the completed five-day evidence gate. ## Rolling Loop Queue -This rolling queue is ordered, but only Loop 39 is committed. Select the next loop after reviewing evidence from the previous one; candidate loop numbers may change when that evidence changes the plan. +This rolling queue is ordered. Loop 40 is deferred and there is no active delivery loop; select the next loop only after the post-incident review. | Loop | Status | Goal | Exit artifact | | --- | --- | --- | --- | -| Loop 39: Scoped Live Lark Task Connector | Next | Implement the approved live `create_task` path behind explicit opt-in | Tested live path, fake-transport evidence, and updated docs | -| Loop 40: Controlled Live Connector Pilot | Candidate | Exercise Loop 39 through a controlled real-team pilot | Controlled live-pilot runbook, redacted run and audit evidence, failure and rollback exercises, and a continue/harden/defer decision | +| Loop 39: Scoped Live Lark Task Connector | Complete | Explicit live `create_task` opt-in, fake-transport coverage, native provider idempotency, redaction and rollback boundaries, and one redacted real-validation evidence note | +| Loop 40: Controlled Live Connector Pilot | Deferred | Controlled real-team Pilot stopped after a normalized provider validation failure and a private partner/operator `defer` decision | Retained private history, safe failure/rollback evidence, and a no-network request preflight; no maturity advance | | Loop 41: Self-hosted Runtime Service Boundary | Candidate | Add one long-running service entry point with validated configuration | Health/readiness checks, graceful shutdown, and restart continuity evidence | | Loop 42: Authenticated Ingress And Production Credentials | Candidate | Require authentication by default for the production service path and resolve credential handles at execution time | Compact security audit evidence and a documented external TLS termination boundary | | Loop 43: Durable Recurring Scheduling And Safe Dispatch | Candidate | Persist recurring schedules with restart recovery and a defined missed-run policy | Durable dispatch records and lease or locking semantics for one SQLite-backed service instance | -Loop 40 must produce a reproducible controlled live-pilot runbook, redacted evidence, explicit failure and rollback exercises, and a decision to continue, harden, or defer broader live integration work. The repository must not commit live credentials or raw live payload evidence. +Loop 40 is explicitly deferred, not complete. Any future Pilot must begin under a new authorization boundary and must still produce a reproducible controlled live-pilot runbook, redacted evidence, explicit failure and rollback exercises, and a decision to continue, harden, or defer broader live integration work. The repository must not commit live credentials or raw live payload evidence. Loop 41 keeps the runtime scope single-instance and single-tenant. It does not introduce worker coordination or a multi-tenant service boundary. @@ -199,6 +162,7 @@ The detailed implementation plans under `docs/superpowers/plans/` are the histor | Loop 36: First Product Connector Package Smoke | Complete | Lark/Feishu task connector dry-run package fixture, explicit-loading smoke, credential-handle evidence, and compact connector metadata | | Loop 37: Product Connector Pilot Scenario | Complete | Sales renewal risk workflow using the Lark/Feishu task dry-run connector after a manual gate, with webhook trigger, audit, snapshot, and LiteGraph overlay artifacts | | Loop 38: Live Connector Readiness Review | Complete | Decision note approving only scoped live Lark/Feishu `create_task` follow-up, with credential, idempotency, failure, audit, test, and rollback boundaries | +| Loop 39: Scoped Live Lark Task Connector | Complete | Explicit live `create_task` opt-in, fake-transport coverage, native provider idempotency, redaction and rollback boundaries, and one redacted real-validation evidence note | ## Release Direction diff --git a/docs/connectors.md b/docs/connectors.md index 16ed2f2..0b52153 100644 --- a/docs/connectors.md +++ b/docs/connectors.md @@ -1,7 +1,7 @@ # Connector Runtime `skill2workflow` currently ships a minimal local connector runtime. It is designed to make connector-bound workflow nodes testable and auditable without adding external services, SDK dependencies, secret storage, or a connector marketplace. -Loop 33 adds one explicitly loaded local external connector fixture to prove the extension boundary. Loop 36 adds the first product-shaped connector package fixture, a Lark/Feishu task `create_task` dry-run connector. Loop 37 proves that connector inside a sales renewal risk pilot workflow. Loop 38 readiness review approved only a scoped live `create_task` follow-up, documented in `docs/lark-live-connector-readiness.md`. These loops do not add automatic discovery, live SaaS calls, OAuth, or marketplace behavior. +Loop 33 adds one explicitly loaded local external connector fixture to prove the extension boundary. Loop 36 adds the first product-shaped connector package fixture, a Lark/Feishu task `create_task` dry-run connector. Loop 37 proves that connector inside a sales renewal risk pilot workflow. Loop 38 readiness review approved only a scoped live `create_task` follow-up, documented in `docs/lark-live-connector-readiness.md`. Loop 39 implements that one opt-in live action while preserving explicit loading and the dry-run default; it does not add automatic discovery, OAuth, token refresh, or marketplace behavior. Workflow DSL remains the execution truth source. Connector bindings live on workflow nodes, and the local executor records connector lifecycle events in run state and control-plane audit logs. @@ -245,7 +245,7 @@ Connector package smoke contract: Package conventions intentionally exclude automatic connector discovery, package installation, marketplace indexing, OAuth, hosted callbacks, queues, production schedulers, and product-specific SaaS connector behavior. -## Lark/Feishu Task Connector Dry-Run Package +## Lark/Feishu Task Connector: Dry-Run Default And Scoped Live Mode `examples/connectors/lark_task_connector.py` is the first product-shaped connector package fixture. It stays outside the built-in connector registry and must be explicitly loaded with `load_external_connector(...)`. @@ -253,7 +253,8 @@ Supported scope: - connector id and kind: `lark_task` - operation: `create_task` -- mode: `dry_run` +- default mode: `dry_run` +- opt-in mode: live - node type: `tool_call` - credential handle: `lark_bot_access_token` - input mapping: body-only values from `/input/title`, `/input/description`, `/input/assignee_open_id`, and `/input/due_at` @@ -265,9 +266,34 @@ The connector validates the request shape, resolves the local credential handle, - input mapping status and input key names - booleans indicating whether title, description, assignee, and due date were present -It does not call the live Lark/Feishu API, create tasks, perform OAuth, refresh tokens, host callbacks, install packages, auto-discover connectors, or enqueue background jobs. Raw mapped task values and resolved credential values must not appear in connector output or audit metadata. +Dry-run remains the default when `mode` is missing or is `dry_run`. It validates and summarizes the request without a provider call. Raw mapped task values and resolved credential values must not appear in connector output or audit metadata. -The live connector readiness decision is documented in `docs/lark-live-connector-readiness.md`. That decision approves only a future opt-in `create_task` live mode behind explicit credential, idempotency, failure-handling, audit-redaction, local-test, and rollback boundaries. The current package remains dry-run-only until that follow-up implementation is merged. +The live connector readiness decision is documented in `docs/lark-live-connector-readiness.md`. The package now supports only the approved opt-in `create_task` action. Live network activity requires both an explicitly loaded binding with `mode: live` and the exact environment switch `SKILL2WORKFLOW_LARK_TASK_LIVE=1`; removing the switch immediately rolls the connector back to no live calls. No other truthy environment values enable it. + +The connector posts only to the fixed Feishu domestic Task API v2 endpoint: + +```text +https://open.feishu.cn/open-apis/task/v2/tasks?user_id_type=open_id +``` + +It uses a fixed 10-second timeout. The required provider scope is either `task:task:write` or `task:task:writeonly`, and the documented limit is 10 create requests per second. The connector derives the native Feishu `client_token` from runtime-owned workflow id, workflow version, run id, and node id. Retries may still invoke transport; reusing the stable token with unchanged request parameters lets Feishu perform provider-side deduplication. The connector does not locally block retry transport calls. + +Normal execution resolves the `lark_bot_access_token` handle through the configured credential provider. `LARK_BOT_ACCESS_TOKEN` is reserved for the guarded validation helper and is not a Workflow DSL or connector-binding field. The helper must be run outside CI with explicit confirmation: + +```bash +vibe vault run --env LARK_BOT_ACCESS_TOKEN -- env SKILL2WORKFLOW_LARK_TASK_LIVE=1 python3 scripts/lark_task_live_validation.py \ + --confirm-live-create \ + --validation-run-id '' \ + --assignee-open-id '' \ + --title '' \ + --description '' +``` + +Use Avibe Vault as shown, or an equivalent secret manager that injects `LARK_BOT_ACCESS_TOKEN` only into the child process. Never paste the token into the command or shell history. + +CI injects a fake transport and never accesses the live network. Recognized Feishu provider codes take precedence over generic HTTP status classification. Normalized `provider_status` values are exactly: `live_disabled`, `validation_failed`, `credential_failed`, `authorization_failed`, `permission_denied`, `rate_limited`, `resource_not_found`, `idempotency_conflict`, `provider_unavailable`, `timeout`, `malformed_response`, and `completed`. + +Connector-produced output, audit, events, snapshots, and summaries contain presence flags and compact statuses only. They never contain raw provider messages, task values, the task guid, resolved token, `client_token`, raw request, or raw response. Durable user-supplied task input remains unchanged under `run.context.input`; it is not connector-produced state. Run the dry-run smoke from a source checkout: @@ -285,6 +311,8 @@ python3 scripts/lark_task_pilot_smoke.py --work-dir /tmp/skill2workflow-lark-tas The pilot uses the same explicitly loaded package inside a workflow that starts through the local webhook trigger boundary, waits at a manual gate, resumes with approval, and then invokes the connector. It proves business handoff and operator evidence, not live Lark/Feishu task creation. +For the one approved controlled real-team pilot, follow `docs/controlled-live-pilot.md`. The dry-run remains the default; that runbook permits only the fixed Feishu domestic `create_task` action behind the existing explicit live guards. + HTTP connector bindings may also reference local credential handles: ```json diff --git a/docs/controlled-live-pilot.md b/docs/controlled-live-pilot.md new file mode 100644 index 0000000..6f2c968 --- /dev/null +++ b/docs/controlled-live-pilot.md @@ -0,0 +1,236 @@ +# Controlled Lark/Feishu Live Pilot Runbook + +This runbook operates Loop 40 as a paid assisted engagement for one consenting real team. It is not a general live-connector guide. Team consent, the designated assignee's consent, and a paid or contractually committed engagement must all be confirmed before initialization. Customer identity, pricing, payment, and contract details remain outside this repository. + +The dry-run remains the default. Controlled live behavior is limited to one `create_task` action through the explicitly loaded `lark_task` connector and the fixed Feishu domestic Task API host. Do not adapt these commands to another action, host, API, connector, or provider. + +The operator phases are: init case-template preflight start decide evidence exercise-failure exercise-rollback verify finalize . Each successful phase prints one compact redacted JSON line. Keep run ids and all private working material in the owner-controlled operating environment. + +## 1. Prerequisites And Private Workspace + +Use an owner-controlled directory outside the source repository. This runbook uses: + +```bash +export PRIVATE_PILOT="$HOME/.local/share/skill2workflow/pilots/loop-40-new-engagement" +``` + +Do not place the private workspace, case files, decision draft, credentials, provider diagnostics, or terminal captures in the repository. Use a new opaque directory for every separately authorized Pilot; never reuse a deferred or finalized workspace. `init` never replaces an existing Charter, and `start` or `decide` fail closed once the workspace records a decision or finalization marker; create a new work directory for the separately authorized Pilot. The tool rejects a work directory inside the repository and creates pilot directories with owner-only permissions where the platform supports them. + +Before proceeding, the operator must verify all three facts directly with the partner: + +- the team consents to the assisted pilot; +- the real task assignee consents to task creation; +- the engagement is paid or contractually committed. + +Only confirmation booleans are recorded. Do not enter a customer name, price, payment detail, contract text, or other business detail in the init command. + +## 2. Initialize The Fixed Charter + +From the source checkout, run the exact init command for the approved engagement window: + +```bash +python3 scripts/controlled_lark_pilot.py init \ + --work-dir "$PRIVATE_PILOT" \ + --starts-on 2026-07-18 \ + --expires-on 2026-08-15 \ + --confirm-team-consent \ + --confirm-assignee-consent \ + --confirm-commercial-engagement +``` + +The command constructs the fixed charter in code. Its only operator-selected values are the two dates and the three required consent/commercial-confirmation booleans. Inspect `$PRIVATE_PILOT/private/charter.json` locally and confirm the scenario, workflow version, `assisted` support model, `Asia/Shanghai` timezone, and thresholds of five approved runs, five days, and two cases. Do not copy the private path or local inspection output into repository evidence. + +Run the unchanged dry-run rehearsal before live work: + +```bash +python3 scripts/lark_task_pilot_smoke.py \ + --work-dir "$PRIVATE_PILOT/private/rehearsal" +``` + +The rehearsal must complete in `dry_run` mode. A dry-run failure stops the pilot. + +## 3. Prepare One Private Case + +Create the owner-only empty template without placing business values on the command line: + +```bash +python3 scripts/controlled_lark_pilot.py case-template \ + --work-dir "$PRIVATE_PILOT" \ + --name day-1 \ + --case-id case-001 +``` + +It creates `$PRIVATE_PILOT/private/cases/day-1.json` once with the exact five +keys and blank business fields. It never overwrites an existing file and is +blocked for a deferred or finalized workspace. Open that owner-only file +locally, replace only the blank values with partner-approved content, and do +not put those values in shell arguments. + +Each case file must then conform to this exact schema: + +```json +{ + "type": "object", + "additionalProperties": false, + "required": ["pilot_case_id", "account_name", "renewal_risk", "owner_open_id", "due_at"], + "properties": { + "pilot_case_id": {"type": "string", "const": "case-001"}, + "account_name": {"type": "string", "minLength": 1}, + "renewal_risk": {"type": "string", "minLength": 1}, + "owner_open_id": {"type": "string", "minLength": 1}, + "due_at": {"type": "string", "format": "date-time"} + } +} +``` + +`case-001` is an opaque pilot identifier, not an account name. For the fourth approved calendar date, use the Day 4 exact schema below. It differs only in the required opaque id: + +```json +{ + "type": "object", + "additionalProperties": false, + "required": ["pilot_case_id", "account_name", "renewal_risk", "owner_open_id", "due_at"], + "properties": { + "pilot_case_id": {"type": "string", "const": "case-002"}, + "account_name": {"type": "string", "minLength": 1}, + "renewal_risk": {"type": "string", "minLength": 1}, + "owner_open_id": {"type": "string", "minLength": 1}, + "due_at": {"type": "string", "format": "date-time"} + } +} +``` + +This exact second schema ensures the acceptance evidence represents at least two private cases. Put real partner-approved values only in the private file; do not put them in shell arguments, documentation, tickets, or repository files. + +Protect the case before start: + +```bash +chmod 600 "$PRIVATE_PILOT/private/cases/day-1.json" +python3 scripts/controlled_lark_pilot.py preflight \ + --input "$PRIVATE_PILOT/private/cases/day-1.json" +python3 scripts/controlled_lark_pilot.py start \ + --work-dir "$PRIVATE_PILOT" \ + --input "$PRIVATE_PILOT/private/cases/day-1.json" +``` + +`preflight` constructs the exact fixed Task v2 request body locally and returns only compact presence and readiness fields. It checks the locally verifiable Task contract, including non-empty title, RFC 3339 deadline, and the 3,000 UTF-8-character limits for title and description. It does not resolve Vault credentials, enable live mode, does not make a network request, or create a Feishu task. A `ready` result is a local contract check only; it does not replace the human review, the explicit approval, or a real provider result. An `invalid` result stops the case before `start`; correct the owner-only case file and run preflight again. + +`start` runs the same no-network preflight again and fails before it accesses the control plane if it is not `ready`; it cannot be used to bypass the check. The start result must show `preflight_ready: true`, `run_status: waiting`, and `current_node: review_renewal_risk`. Record the opaque run id privately. Before any decision, the designated operator must inspect the compact waiting summary and the owner-only case file, confirm the intended assignee and task contents, and verify that the run is still waiting at that exact human gate. + +## 4. Make The Explicit Human Decision + +### Approval: Vault-issued short-lived tenant token + +Only an approved, inspected waiting run may receive the live switch and App Secret. Export the non-secret App ID for the approved Feishu app, and let Vault inject only `LARK_APP_SECRET`. The runner exchanges it in memory for one short-lived tenant access token only after every live guard passes; it never writes the App Secret, issued token, or token-exchange response to private state, audit, evidence, or command output. Use this exact approve-only command shape: + +```bash +export LARK_APP_ID="your-approved-feishu-app-id" +vibe vault run --env LARK_APP_SECRET -- \ + env SKILL2WORKFLOW_LARK_TASK_LIVE=1 \ + python3 scripts/controlled_lark_pilot.py decide \ + --work-dir "$PRIVATE_PILOT" \ + --run-id "$APPROVED_RUN_ID" \ + --approve \ + --confirm-live-create +``` + +Never paste an App Secret or token into the command, a file, or shell history. Approval must fail closed unless the run is waiting, the explicit confirmation is present, the live switch is exactly `1`, `LARK_APP_ID` and the Vault-injected `LARK_APP_SECRET` exist, the token exchange succeeds, and the fixed live workflow binding is unchanged. `LARK_BOT_ACCESS_TOKEN` remains a legacy compatibility fallback only when no App Secret is injected; it is not the recommended Pilot path. Success reports normalized status and presence booleans only; it must not print task values, provider messages, provider task ids, App Secret or token material, or request/response bodies. + +### Rejection: no Vault + +Rejection neither needs nor permits live confirmation. After inspecting a separate waiting run, run without Vault and without either live environment variable: + +```bash +env -u SKILL2WORKFLOW_LARK_TASK_LIVE -u LARK_BOT_ACCESS_TOKEN -u LARK_APP_ID -u LARK_APP_SECRET \ + python3 scripts/controlled_lark_pilot.py decide \ + --work-dir "$PRIVATE_PILOT" \ + --run-id "$REJECTED_RUN_ID" \ + --reject +``` + +The result must report `gate_decision: rejected` and `connector_invoked: false`. `--reject --confirm-live-create` is an operator error. + +## 5. Regenerate Evidence After Every Run + +After every approved, rejected, or failed run, regenerate the complete private evidence pack: + +```bash +python3 scripts/controlled_lark_pilot.py evidence \ + --work-dir "$PRIVATE_PILOT" +``` + +Inspect `$PRIVATE_PILOT/evidence` for compact statuses and presence flags only. Retain failed historical runs; never replace them with clean runs. Do not export or commit an incomplete pack. + +Complete at least five approved live runs across five distinct calendar days in `Asia/Shanghai`. Use `case-001` for the first three dates, `case-002` on the fourth date, and either opaque id on the fifth. This threshold is not an SLA or general reliability claim. The pack must also contain at least one rejected human-gate run with no connector invocation. + +## 6. Exercise Failure And Rollback + +Run both safe exercises with live variables removed: + +```bash +env -u SKILL2WORKFLOW_LARK_TASK_LIVE -u LARK_BOT_ACCESS_TOKEN -u LARK_APP_ID -u LARK_APP_SECRET \ + python3 scripts/controlled_lark_pilot.py exercise-failure \ + --work-dir "$PRIVATE_PILOT" + +env -u SKILL2WORKFLOW_LARK_TASK_LIVE -u LARK_BOT_ACCESS_TOKEN -u LARK_APP_ID -u LARK_APP_SECRET \ + python3 scripts/controlled_lark_pilot.py exercise-rollback \ + --work-dir "$PRIVATE_PILOT" +``` + +The disabled-live exercise must report `provider_status: live_disabled` with no credential resolution and no provider transport. Rollback must prove that live approval is blocked while the unchanged dry-run pilot still completes. Regenerate private evidence after both exercises. + +## 7. Run The Fixed Sanitized Verification + +After the five-day gate and exercises are complete, run: + +```bash +env -u SKILL2WORKFLOW_LARK_TASK_LIVE -u LARK_BOT_ACCESS_TOKEN \ + python3 scripts/controlled_lark_pilot.py verify \ + --work-dir "$PRIVATE_PILOT" +``` + +The fixed verify phase runs exactly seven checks: focused controlled-pilot tests, the full suite, Python compilation, secret hygiene, connector smoke, dry-run pilot smoke, and `git diff --check`. All seven must pass. Verification output contains command ids, exit status, pass/fail state, and duration only; it excludes captured command output. + +## 8. Prepare The Redacted Decision + +Partner and operator choose `continue`, `harden`, or `defer`. Create `$PRIVATE_PILOT/private/decision.json` using this exact allowlisted schema and a short rationale with no customer, account, user, task, token, provider, price, or contract detail: + +```json +{ + "schema_version": "controlled-lark-pilot-decision-0.1.0", + "decision": "defer", + "partner_acknowledged": true, + "operator_acknowledged": true, + "commercial_engagement_confirmed": true, + "rationale": "The controlled evidence supports the recorded next-step decision within the approved boundary." +} +``` + +Protect the file before finalization: + +```bash +chmod 600 "$PRIVATE_PILOT/private/decision.json" +``` + +The decision file must be an owner-only regular file outside the repository, not a symbolic link. Rationale is accepted only through this file; it is never a command-line argument. + +## 9. Finalize And Export Only After Every Gate Passes + +Run the exact finalize command from the repository root: + +```bash +python3 scripts/controlled_lark_pilot.py finalize \ + --work-dir "$PRIVATE_PILOT" \ + --decision-file "$PRIVATE_PILOT/private/decision.json" \ + --output-dir docs/pilot-evidence/loop-40 +``` + +Finalization must fail while any threshold, rejection, exercise, verification, commercial confirmation, acknowledgement, or decision condition is missing. The only permitted repository export target is exactly `docs/pilot-evidence/loop-40`. Review the generated allowlisted JSON before committing; never commit private state, raw payloads, provider values, or credentials. + +After the engagement, rotate or delete the Pilot App Secret (or legacy static token, if used) according to the partner's credential policy, remove the live switch, and retain only the agreed private retention set and validated redacted repository evidence. + +## Incident Stop And Deferral + +Stop immediately if a run exposes a forbidden value, bypasses the human gate, creates an unexpected duplicate, targets the wrong assignee, encounters a permission or redaction anomaly, uses a non-normalized provider result, or deviates from the fixed domestic endpoint and action. Remove the live switch, do not approve another run, retain authoritative private state, record a `defer` candidate decision, and return with a failing regression test. The runner also rejects every later `--approve` in that workspace before it reads a credential or calls a transport; a fresh work directory and authorization boundary are required. A recorded decision closes that workspace to further starts and decisions; never hide, replace, or retry the failed run. + +Offline tests, fake transport, an empty evidence skeleton, and implementation readiness must not advance Loop 40. A deferred Pilot remains at Local Evaluation; any replacement Pilot requires a fresh authorization boundary and must successfully finalize the paid five-day real-team evidence gate before the separate Roadmap completion task may run. diff --git a/docs/controlled-pilot-deferral-review.md b/docs/controlled-pilot-deferral-review.md new file mode 100644 index 0000000..f02f400 --- /dev/null +++ b/docs/controlled-pilot-deferral-review.md @@ -0,0 +1,60 @@ +# Controlled Pilot Deferral Review + +## Scope And Decision + +This review records the Loop 40 controlled live Pilot after the partner and +operator both confirmed `defer`. It is an operational boundary record, not +completion evidence and not authorization for another live request. + +The original private workspace is closed. There is no retry, replacement, or +additional live decision under that workspace. Its owner-only history remains +authoritative; no raw task values, identifiers, credentials, request bodies, +response bodies, or provider diagnostics are committed here. + +## Supported Facts + +- The scoped Lark task connector previously completed one separately approved + live `create_task` validation; the compact redacted record is in + [`lark-live-connector-validation.md`](lark-live-connector-validation.md). +- The controlled Pilot recorded one approved live completion, a human + rejection with no connector invocation, and passing disabled-live and + rollback exercises. +- A later explicitly approved controlled attempt returned the normalized + status `validation_failed`. It did not reach the five approved live runs + across five `Asia/Shanghai` calendar days required by Loop 40. +- The connector intentionally retained no raw provider message. Therefore the + review cannot attribute that `validation_failed` result to a specific + provider field, permission, task member, deadline, or task value. + +The current local `preflight` reconstructs the same fixed Task v2 payload +without credentials or network access. It verifies the locally knowable +request shape: a non-empty summary, optional description, open-id assignee +member, timezone-aware millisecond deadline, and deterministic client token. +This gives local contract evidence only; it does not prove future provider +acceptance and does not change the historical result. + +## Re-entry Gate For A Separately Authorized Pilot + +A future Pilot may be considered only after fresh partner and operator +authorization. It must use a new private work directory, a new valid Charter, +new owner-only case files, and the no-network `preflight` before each `start`. +Every real create remains a separate inspected human approval with the live +switch and Vault injection present only for that one command. + +The replacement engagement must independently complete five approved live +runs across five distinct `Asia/Shanghai` dates, use at least two opaque case +ids, contain a human rejection, pass the failure and rollback exercises, and +pass fixed verification before it can be finalized. It cannot count or repair +the deferred workspace's history. + +If an equivalent normalized failure occurs, stop the new Pilot, retain its +private facts, and obtain a new decision instead of widening the API action, +capturing raw provider content, or issuing a retry. Any request to broaden the +connector beyond the fixed domestic `create_task` boundary requires a separate +readiness review. + +## Current Status + +Loop 40 remains deferred at Local Evaluation. The next external prerequisite +is a separately authorized paid assisted engagement; repository tests and +dry-runs do not satisfy it. diff --git a/docs/examples.md b/docs/examples.md index 77a9299..520b335 100644 --- a/docs/examples.md +++ b/docs/examples.md @@ -112,6 +112,8 @@ python3 scripts/lark_task_pilot_smoke.py --work-dir /tmp/skill2workflow-lark-tas The pilot writes workflow, trigger response, run state, audit, connector list, control-plane snapshot, and LiteGraph overlay artifacts. Use it to inspect whether a product connector package remains understandable in a business workflow with a control point. It is still dry-run only and does not create live Lark/Feishu tasks. +This local example is the dry-run baseline, not evidence from the controlled real-team pilot. Operators running the separately approved paid engagement must use `docs/controlled-live-pilot.md`; dry-run remains the default and controlled live execution remains limited to one fixed action. + ### Local Pilot Scenario Pack The pilot scenario pack is generated at runtime rather than committed as static fixtures. It runs customer support escalation, sales renewal follow-up, and risk exception review through local-only workflows and HTTP receivers: diff --git a/docs/lark-live-connector-readiness.md b/docs/lark-live-connector-readiness.md index b8333be..74dd40e 100644 --- a/docs/lark-live-connector-readiness.md +++ b/docs/lark-live-connector-readiness.md @@ -1,6 +1,6 @@ # Lark/Feishu Live Connector Readiness Review -Decision: proceed to a scoped live `create_task` implementation in Loop 39. +Decision: proceed to a scoped live `create_task` implementation in Loop 39. That implementation is now available as an explicitly loaded, opt-in connector path while dry-run remains the default. This decision approves one narrow follow-up implementation path. It does not make live SaaS connector behavior part of the default runtime, and it does not change the Workflow DSL authority model. @@ -14,7 +14,7 @@ Loop 37 pilot-workflow dry-run smoke proved the same connector inside a business python3 scripts/lark_task_pilot_smoke.py --work-dir /tmp/skill2workflow-lark-task-pilot ``` -The remaining risk is no longer whether the local connector package shape works. The remaining risk is whether a single live Lark/Feishu API action can be executed with explicit credentials, idempotency, failure handling, audit redaction, and rollback boundaries. +Loop 39 closes the connector-level implementation risk with fake-transport coverage for explicit credentials, provider-native idempotency, failure handling, audit redaction, and rollback boundaries. A real provider call remains a separately guarded validation action and is never part of CI. ## Approved Live Action Surface @@ -39,6 +39,16 @@ The live request body may use the same business fields already validated by dry- `examples/connectors/lark_task_connector.py remains dry-run-only in Loop 38`. Loop 39 must introduce live behavior behind an explicit opt-in path, not as an implicit change to existing dry-run examples. +Loop 39 implements that scoped path against one fixed provider boundary: + +```text +POST https://open.feishu.cn/open-apis/task/v2/tasks?user_id_type=open_id +``` + +The connector constructs the method, Feishu domestic host, Task API v2 path, `user_id_type=open_id` query, and headers internally. Workflow input cannot override them. The required provider scope is either `task:task:write` or `task:task:writeonly`, the documented limit is 10 create requests per second, and the connector uses a fixed 10-second timeout. + +Live activation requires both `mode: live` in the explicitly loaded connector binding and the exact environment switch `SKILL2WORKFLOW_LARK_TASK_LIVE=1`. Missing mode and explicit `mode: dry_run` both retain dry-run behavior. Any other environment value returns compact `provider_status: live_disabled` metadata before credential resolution or transport invocation. + Explicit non-goals: - No OAuth. @@ -65,26 +75,34 @@ The token is resolved only through the credential provider at connector executio Workflow DSL may reference the handle name, but the resolved credential value must remain outside immutable workflow artifacts and persisted run evidence. Missing credentials, unsupported credential targets, or provider resolution errors must become failed connector results with compact error metadata. -## Idempotency And Duplicate Prevention +Normal connector execution resolves only that approved handle through the configured credential provider. Unrelated header-target handles are not materialized. `LARK_BOT_ACCESS_TOKEN` is not a general connector configuration surface; it is read only by the guarded live-validation helper, which immediately wraps it in the existing credential provider and never accepts or prints the token as a command-line value. -Live `create_task` must require an idempotency key before making a task creation request. +## Idempotency And Duplicate Prevention -The local runtime should derive the default key from `workflow_id + version + run_id + node_id` unless the future API contract exposes a better first-class idempotency field. If Lark/Feishu does not support a native idempotency header for task creation, Loop 39 must still record the derived key in local connector metadata and reject unsafe re-execution for the same key before attempting duplicate task creation. +Live `create_task` requires all four runtime-owned identity values from `workflow_id + version + run_id + node_id` before making a request. The connector canonicalizes those values into a deterministic UUID v5 for the provider's native `client_token`. This native token is the provider idempotency key: it is stable for retries of the same execution identity and changes for a different version, run, or node. -Duplicate task creation is the primary live-operation hazard. The connector should prefer a safe failure over guessing whether a previous call succeeded when the idempotency record is ambiguous. +Retries may still invoke transport; the stable native `client_token` and unchanged request parameters let Feishu perform provider-side deduplication. This provider mechanism controls duplicate task creation: the connector does not locally block retry transport calls and Loop 39 adds no local idempotency database. Connector-produced state records only `idempotency_key_present`, never the token digest. A provider-reported idempotency conflict becomes a safe failure with compact metadata instead of a guessed retry. ## Failure Modes -Loop 39 must map expected live failures into normalized connector results rather than leaking provider-specific details through exceptions: +Loop 39 maps expected live failures into normalized connector results rather than leaking provider-specific details through exceptions. The compact `provider_status` categories are: -| Failure | Required behavior | -| --- | --- | -| `401 or 403` | Failed connector result with compact authorization status and no token echo. | -| Permission denied for task creation | Failed connector result with compact permission status. | -| `rate limit` | Failed connector result that allows existing retry policy to decide whether to retry. | -| `network timeout` | Failed connector result or existing connector execution error path, with no raw request body in audit. | -| Provider validation error | Failed connector result with field-level category only. | -| Unexpected provider response | Failed connector result without raw response payload leakage. | +| Failure | `provider_status` | Required behavior | +| --- | --- | --- | +| Live switch absent | `live_disabled` | Fail before credential or transport access. | +| Invalid local input or execution identity | `validation_failed` | Fail before transport access. | +| Missing credential | `credential_failed` | Preserve only the handle and compact status. | +| HTTP `401` or provider code `99991663` | `authorization_failed` | Fixed error text with no token echo. | +| HTTP `403` | `permission_denied` | Fixed error text with no token echo. | +| HTTP 429 / `rate limit` | `rate_limited` | Existing retry policy decides whether to retry. | +| Provider validation, missing resource, or idempotency conflict | `validation_failed`, `resource_not_found`, or `idempotency_conflict` | Do not retain provider messages or identifiers. | +| HTTP 5xx/provider unavailable | `provider_unavailable` | A retry reuses the same `client_token`. | +| `network timeout` | `timeout` | No raw request body or token in the error. | +| Unexpected provider response | `malformed_response` | Failed connector result without raw response payload leakage. | + +Recognized Feishu provider codes take precedence over generic HTTP status classification. If no recognized code is available, the connector falls back to the HTTP status category. + +Normalized `provider_status` values are exactly: `live_disabled`, `validation_failed`, `credential_failed`, `authorization_failed`, `permission_denied`, `rate_limited`, `resource_not_found`, `idempotency_conflict`, `provider_unavailable`, `timeout`, `malformed_response`, and `completed`. Audit and run output should preserve enough state for operators to understand whether the live call was attempted, completed, failed, or skipped. It should not preserve raw provider responses unless a future redaction contract explicitly allows safe structured fields. @@ -116,11 +134,13 @@ Disallowed audit and run-state fields: - raw request bodies - raw response payloads -The live result may expose a Lark task id only through a compact presence flag by default. If Loop 39 needs the task id for operator diagnostics, the PR must justify why the id is non-secret and must still keep the default audit path compact. +The live connector must not copy raw task values into connector-produced state. It also must not copy raw provider messages, the provider task guid, the resolved token, the idempotency digest, request bodies, or response bodies into connector output, audit, events, snapshots, or summaries. A successful call exposes the task guid only as `lark_task_id_present: true`. + +This connector boundary does not rewrite the existing durable-input contract. Values explicitly supplied by a user remain unchanged in durable `run.context.input`; those values may include the task input used by mapping. Connector redaction applies to connector-produced and promoted state, while credential values remain prohibited from all persistent state. ## Local Test Strategy -Loop 39 should test live-mode behavior with a fake Lark HTTP receiver or injected fake transport. There must be no live network in CI. +Loop 39 tests live-mode behavior with a fake Lark HTTP receiver implemented as an injected fake transport. There is no live network in CI. Required tests before any live API implementation can merge: @@ -130,17 +150,28 @@ Required tests before any live API implementation can merge: - live mode sends only the approved `create_task` request shape - the fake receiver can simulate success, `401 or 403`, rate limit, network timeout, validation error, and malformed response cases - raw task values and resolved credentials do not appear in result summaries, run state, audit logs, or snapshot artifacts -- the idempotency key blocks duplicate task creation attempts for the same `workflow_id + version + run_id + node_id` +- retries for the same `workflow_id + version + run_id + node_id` reuse the stable native `client_token` and unchanged parameters so Feishu performs provider-side deduplication; the connector does not locally block retry transport calls + +The existing dry-run tests and smoke commands continue to pass unchanged. After CI-safe fake-transport tests pass, an operator may run the separately guarded validation helper outside CI: + +```bash +vibe vault run --env LARK_BOT_ACCESS_TOKEN -- env SKILL2WORKFLOW_LARK_TASK_LIVE=1 python3 scripts/lark_task_live_validation.py \ + --confirm-live-create \ + --validation-run-id '' \ + --assignee-open-id '' \ + --title '' \ + --description '' +``` -The existing dry-run tests and smoke commands must continue to pass unchanged. +Use Avibe Vault as shown, or an equivalent secret manager that injects `LARK_BOT_ACCESS_TOKEN` only into the child process. Never paste the token into the command or shell history. The helper is inert without the confirmation flag, the exact environment switch, a token injected through the environment, a stable validation run id, and an assignee open id. It prints compact presence/status metadata only and writes no run state. ## Rollback Boundaries -Loop 39 must keep live behavior behind a feature flag or equivalent explicit opt-in setting. The default remains dry-run. +Loop 39 keeps live behavior behind a feature flag: the exact environment switch. The default remains dry-run. Rollback requirements: -- A maintainer can disable live mode without removing the dry-run connector package. +- A maintainer can disable live mode without removing the dry-run connector package by removing `SKILL2WORKFLOW_LARK_TASK_LIVE=1` (or setting any value other than the exact string `1`). - Reverting Loop 39 must not require changing Workflow DSL compatibility. - A maintainer must be able to revert Loop 39 without changing Workflow DSL compatibility. - Reverting Loop 39 must not invalidate existing dry-run smokes, pilot artifacts, or connector package documentation. diff --git a/docs/lark-live-connector-validation.md b/docs/lark-live-connector-validation.md new file mode 100644 index 0000000..81a3c6c --- /dev/null +++ b/docs/lark-live-connector-validation.md @@ -0,0 +1,31 @@ +# Lark/Feishu Live Connector Validation + +Validation timestamp (UTC): `2026-07-16T07:08:40Z` + +Observed compact result: + +- connector_id: `lark_task` +- operation: `create_task` +- mode: `live` +- credential_status: `resolved` +- idempotency_key_present: `true` +- provider_status: `completed` +- lark_task_id_present: `true` +- assignee_present: `true` + +The validation task was assigned to the consenting current user and retained as visible evidence. + +The user explicitly approved a one-time standard-tier Vault fallback after protected-tier passkey/WebAuthn provisioning failed. The credential had `open.feishu.cn` allowed-host metadata, was short-lived, and was deleted immediately after success. + +Raw task values, user ids, credentials, request bodies, response bodies, and task ids are intentionally omitted. + +Offline verification commands run before the live write: + +```bash +PYTHONPATH=src python3 -m unittest discover -s tests -v +python3 -m py_compile src/skill2workflow/*.py examples/connectors/lark_task_connector.py +python3 scripts/secret_hygiene.py examples/workflows +python3 scripts/lark_task_connector_smoke.py --work-dir /tmp/skill2workflow-lark-task-connector +python3 scripts/lark_task_pilot_smoke.py --work-dir /tmp/skill2workflow-lark-task-pilot +git diff --check +``` diff --git a/docs/superpowers/plans/2026-07-11-scoped-live-lark-task.md b/docs/superpowers/plans/2026-07-11-scoped-live-lark-task.md new file mode 100644 index 0000000..cd99c48 --- /dev/null +++ b/docs/superpowers/plans/2026-07-11-scoped-live-lark-task.md @@ -0,0 +1,1583 @@ +# Scoped Live Lark Task Connector Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Complete Loop 39 by adding one explicitly enabled, idempotent, redacted Feishu Task v2 `create_task` path and validating it once with a Vault-injected bot token. + +**Architecture:** Keep live behavior inside the explicitly loaded `lark_task` external connector. `LocalExecutor` supplies ephemeral execution identity, the connector uses a fixed Feishu endpoint plus native `client_token`, and all CI coverage uses an injected standard-library-compatible transport; one separate guarded command performs the approved real write after offline verification. + +**Tech Stack:** Python 3.9 standard library, `unittest`, `urllib.request`, existing external connector and credential-provider contracts, Avibe Vault. + +## Global Constraints + +- Workflow DSL remains the execution source of truth and stays at schema version `0.1.0`. +- Live scope is only `lark_task` / `create_task` / Feishu domestic `https://open.feishu.cn/open-apis/task/v2/tasks?user_id_type=open_id`. +- Dry-run remains the default when `mode` is missing or equals `dry_run`. +- Live network activity requires both `mode: live` and `SKILL2WORKFLOW_LARK_TASK_LIVE=1`. +- Only the exact environment value `1` enables live behavior. +- Live credentials use handle `lark_bot_access_token`; resolved values never enter workflow artifacts, state, events, audit, snapshots, validation artifacts, errors, or terminal output. +- Provider idempotency uses a SHA-256 hex `client_token` derived from canonical JSON for `[workflow_id, workflow_version, run_id, node_id]`. +- The connector may not accept a configurable API base, path, method, query, or arbitrary headers. +- Raw task values may remain only in the pre-existing durable `run.context.input`; the connector must not copy them into node results, events, audit, snapshots, output, or summaries. +- Runtime-owned `_execution` metadata is ephemeral and overwrites any user-provided value before connector invocation. +- No CI command may use live network access. +- The real validation write must use the already approved task content and current-user assignment from conversation context, without committing those raw values or user id. +- Keep Python runtime dependencies limited to the standard library. + +--- + +## File Map + +- `src/skill2workflow/executor.py`: create ephemeral connector context containing runtime-owned execution identity. +- `examples/connectors/lark_task_connector.py`: preserve dry-run and implement the fixed live create-task request, native idempotency, transport, failure mapping, and redaction. +- `src/skill2workflow/lark_task_live_validation.py`: guarded, compact, no-state live validation orchestration. +- `scripts/lark_task_live_validation.py`: source-checkout command wrapper. +- `tests/test_executor.py`: prove `_execution` delivery and non-persistence. +- `tests/test_lark_task_connector.py`: live activation, request shape, idempotency, failure matrix, and leakage tests. +- `tests/test_lark_task_live_validation.py`: validation command guard and compact-output tests. +- `docs/connectors.md`: public live connector contract. +- `docs/lark-live-connector-readiness.md`: native idempotency decision and durable-input clarification. +- `docs/lark-live-connector-validation.md`: redacted evidence from the approved real write. +- `tests/test_live_connector_readiness.py`: documentation contract for the implemented live boundary. +- `ROADMAP.md`, `README.md`, `tests/test_production_roadmap.py`, `tests/test_product_connector_pilot_roadmap.py`, `tests/test_first_product_connector_candidate_docs.py`: Loop 39 completion and Loop 40 transition. +- `docs/superpowers/plans/2026-07-11-scoped-live-lark-task.md`: execution checklist and verification record. + +### Task 1: Ephemeral Connector Execution Identity + +**Files:** +- Modify: `tests/test_executor.py` +- Modify: `src/skill2workflow/executor.py` + +**Interfaces:** +- Consumes: `LocalExecutor._execute_connector_node(...)` and `ConnectorRuntime.execute_connector(node, credential_provider=None, context=None)` +- Produces: `_connector_context(state: RunState, node_id: str) -> Dict[str, object]`, consumed by every connector call and by Task 2 idempotency derivation + +- [x] **Step 1: Write the failing executor test** + +Add this test to `ExecutorTests`: + +```python + def test_connector_receives_ephemeral_execution_identity_without_persisting_it(self): + runtime = _CapturingConnectorRuntime() + original_context = { + "input": {"title": "Durable title"}, + "_execution": {"workflow_id": "forged"}, + } + + with TemporaryDirectory() as tmp: + executor = LocalExecutor(Path(tmp), connector_runtime=runtime) + state = executor.run(_http_connector_workflow("https://unused.invalid"), context=original_context) + persisted = executor.get_run(state["run_id"]) + + self.assertEqual(len(runtime.contexts), 1) + self.assertEqual( + runtime.contexts[0]["_execution"], + { + "workflow_id": "workflow_connector", + "workflow_version": "0.1.0", + "run_id": state["run_id"], + "node_id": "call_api", + }, + ) + self.assertEqual(runtime.contexts[0]["input"], {"title": "Durable title"}) + self.assertEqual(state["context"], original_context) + self.assertEqual(persisted["context"], original_context) +``` + +Add this helper after the test class: + +```python +class _CapturingConnectorRuntime: + def __init__(self): + self.contexts = [] + + def execute_connector(self, node, credential_provider=None, context=None): + self.contexts.append(context) + return { + "status": "completed", + "connector": {"id": "http", "kind": "http"}, + "output": {}, + } +``` + +- [x] **Step 2: Run the focused test and verify RED** + +Run: + +```bash +PYTHONPATH=src python3 -m unittest tests.test_executor.ExecutorTests.test_connector_receives_ephemeral_execution_identity_without_persisting_it -v +``` + +Expected: FAIL because the connector receives only the durable context and the forged `_execution` object is not replaced. + +- [x] **Step 3: Implement the runtime-owned connector context** + +Replace the connector call's current `context=state.get("context", {})` argument with: + +```python + context=_connector_context(state, current_id), +``` + +Add this helper before `_now()`: + +```python +def _connector_context(state: RunState, node_id: str) -> Dict[str, object]: + durable = state.get("context", {}) + context = copy.deepcopy(durable) if isinstance(durable, dict) else {} + context["_execution"] = { + "workflow_id": str(state.get("workflow_id", "")), + "workflow_version": str(state.get("workflow_version", "")), + "run_id": str(state.get("run_id", "")), + "node_id": str(node_id), + } + return context +``` + +- [x] **Step 4: Run focused and executor tests** + +Run: + +```bash +PYTHONPATH=src python3 -m unittest tests.test_executor -v +``` + +Expected: all executor tests PASS. + +- [x] **Step 5: Commit Task 1** + +Run: + +```bash +git add src/skill2workflow/executor.py tests/test_executor.py +git commit -m "feat: add connector execution identity" +``` + +Expected: one commit containing only the executor context change and its test. + +### Task 2: Live Activation, Request Transformation, And Idempotency + +**Files:** +- Modify: `examples/connectors/lark_task_connector.py` +- Modify: `tests/test_lark_task_connector.py` + +**Interfaces:** +- Consumes: Task 1 `_execution` context and existing `StaticCredentialProvider` +- Produces: `execute(binding, credential_provider=None, context=None, transport=None)`, `_provider_request_body(...)`, `_client_token(...)`, and the injectable transport contract used by Tasks 3 and 4 + +- [x] **Step 1: Add deterministic test transport helpers** + +Retain the existing `json` import and add: + +```python +import os +from datetime import datetime +from unittest.mock import patch + +from skill2workflow.connectors import ExternalConnector +``` + +Replace `_load_lark_task_connector()` with: + +```python +def _load_lark_task_connector(transport=None): + connector = load_external_connector(ROOT / "examples" / "connectors" / "lark_task_connector.py") + if transport is None: + return connector + + def execute_with_transport(binding, credential_provider=None, context=None): + return connector.executor( + binding, + credential_provider=credential_provider, + context=context, + transport=transport, + ) + + return ExternalConnector(manifest=connector.manifest, executor=execute_with_transport) + + +class _FakeResponse: + def __init__(self, status, payload): + self.status = status + self._payload = payload + + def read(self): + if isinstance(self._payload, bytes): + return self._payload + return json.dumps(self._payload).encode("utf-8") + + def close(self): + return None + + +class _FakeTransport: + def __init__(self, status=200, payload=None, error=None): + self.status = status + self.payload = payload if payload is not None else { + "code": 0, + "msg": "success", + "data": {"task": {"guid": "task-guid-must-not-leak"}}, + } + self.error = error + self.calls = [] + + def __call__(self, request, timeout): + self.calls.append({"request": request, "timeout": timeout}) + if self.error is not None: + raise self.error + return _FakeResponse(self.status, self.payload) + + +class _FailIfResolvedCredentialProvider: + def resolve(self, handle): + raise AssertionError(f"credential resolution must not run: {handle}") + + +def _execution_context(run_id="run_live", node_id="create_lark_task"): + return { + "input": { + "title": "Renewal risk follow-up", + "description": "Customer ACME needs executive review", + "assignee_open_id": "ou_123456", + "due_at": "2026-07-09T09:00:00Z", + }, + "_execution": { + "workflow_id": "workflow_lark_live", + "workflow_version": "0.1.0", + "run_id": run_id, + "node_id": node_id, + }, + } +``` + +- [x] **Step 2: Write live-disabled and successful-request tests** + +Add these tests to `LarkTaskConnectorTests`: + +```python + def test_lark_task_live_mode_requires_exact_environment_opt_in(self): + transport = _FakeTransport() + runtime = ConnectorRuntime([_load_lark_task_connector(transport)]) + + for value in (None, "", "true", "yes", "0"): + environment = {} if value is None else {"SKILL2WORKFLOW_LARK_TASK_LIVE": value} + with self.subTest(value=value), patch.dict(os.environ, environment, clear=True): + result = runtime.execute_connector( + _lark_task_node(mode="live"), + credential_provider=_FailIfResolvedCredentialProvider(), + context=_execution_context(), + ) + + self.assertEqual(result["status"], "failed") + self.assertEqual(result["audit"]["provider_status"], "live_disabled") + + self.assertEqual(transport.calls, []) + + def test_lark_task_live_mode_sends_fixed_redacted_idempotent_request(self): + transport = _FakeTransport() + runtime = ConnectorRuntime([_load_lark_task_connector(transport)]) + + with patch.dict(os.environ, {"SKILL2WORKFLOW_LARK_TASK_LIVE": "1"}, clear=True): + result = runtime.execute_connector( + _lark_task_node(mode="live"), + credential_provider=StaticCredentialProvider( + {"lark_bot_access_token": "local-lark-secret"} + ), + context=_execution_context(), + ) + + self.assertEqual(result["status"], "completed") + self.assertEqual(len(transport.calls), 1) + call = transport.calls[0] + request = call["request"] + request_body = json.loads(request.data.decode("utf-8")) + expected_due = str(int(datetime.fromisoformat("2026-07-09T09:00:00+00:00").timestamp() * 1000)) + + self.assertEqual(request.method, "POST") + self.assertEqual( + request.full_url, + "https://open.feishu.cn/open-apis/task/v2/tasks?user_id_type=open_id", + ) + self.assertEqual(request.get_header("Authorization"), "Bearer local-lark-secret") + self.assertEqual(request.get_header("Content-type"), "application/json; charset=utf-8") + self.assertEqual(request_body["summary"], "Renewal risk follow-up") + self.assertEqual(request_body["description"], "Customer ACME needs executive review") + self.assertEqual( + request_body["members"], + [{"id": "ou_123456", "type": "user", "role": "assignee"}], + ) + self.assertEqual(request_body["due"], {"timestamp": expected_due, "is_all_day": False}) + self.assertEqual(len(request_body["client_token"]), 64) + self.assertNotIn("source", request_body) + self.assertEqual(call["timeout"], 10.0) + self.assertEqual(result["audit"]["provider_status"], "completed") + self.assertTrue(result["audit"]["idempotency_key_present"]) + self.assertTrue(result["audit"]["lark_task_id_present"]) + + encoded = json.dumps(result, ensure_ascii=False) + for forbidden in ( + "local-lark-secret", + "Renewal risk follow-up", + "Customer ACME needs executive review", + "ou_123456", + "2026-07-09T09:00:00Z", + "task-guid-must-not-leak", + request_body["client_token"], + ): + self.assertNotIn(forbidden, encoded) + + def test_lark_task_client_token_is_stable_per_execution_identity(self): + transport = _FakeTransport() + runtime = ConnectorRuntime([_load_lark_task_connector(transport)]) + + with patch.dict(os.environ, {"SKILL2WORKFLOW_LARK_TASK_LIVE": "1"}, clear=True): + runtime.execute_connector( + _lark_task_node(mode="live"), + credential_provider=StaticCredentialProvider( + {"lark_bot_access_token": "local-lark-secret"} + ), + context=_execution_context(), + ) + runtime.execute_connector( + _lark_task_node(mode="live"), + credential_provider=StaticCredentialProvider( + {"lark_bot_access_token": "local-lark-secret"} + ), + context=_execution_context(), + ) + runtime.execute_connector( + _lark_task_node(mode="live"), + credential_provider=StaticCredentialProvider( + {"lark_bot_access_token": "local-lark-secret"} + ), + context=_execution_context(run_id="run_other"), + ) + runtime.execute_connector( + _lark_task_node(mode="live"), + credential_provider=StaticCredentialProvider( + {"lark_bot_access_token": "local-lark-secret"} + ), + context=_execution_context(node_id="create_other_lark_task"), + ) + + tokens = [json.loads(call["request"].data.decode("utf-8"))["client_token"] for call in transport.calls] + self.assertEqual(tokens[0], tokens[1]) + self.assertNotEqual(tokens[0], tokens[2]) + self.assertNotEqual(tokens[0], tokens[3]) + + def test_lark_task_missing_mode_remains_dry_run(self): + node = _lark_task_node() + del node["connector"]["mode"] + runtime = ConnectorRuntime([_load_lark_task_connector()]) + + result = runtime.execute_connector( + node, + credential_provider=StaticCredentialProvider( + {"lark_bot_access_token": "local-lark-secret"} + ), + context={"input": {"title": "Task"}}, + ) + + self.assertEqual(result["status"], "completed") + self.assertEqual(result["audit"]["mode"], "dry_run") +``` + +Extend `test_lark_task_manifest_is_explicit_external_connector` with: + +```python + self.assertEqual( + connector.manifest["config_schema"]["properties"]["mode"]["enum"], + ["dry_run", "live"], + ) + self.assertIn("dry-run-default", connector.manifest["description"]) +``` + +- [x] **Step 3: Run live tests and verify RED** + +Run: + +```bash +PYTHONPATH=src python3 -m unittest tests.test_lark_task_connector -v +``` + +Expected: FAIL because `execute` rejects live mode and has no transport parameter. + +- [x] **Step 4: Implement live activation and successful request construction** + +Update the module docstring to describe a dry-run-default connector with scoped live support. Add these imports and constants: + +```python +import hashlib +import json +import os +from datetime import datetime +from urllib import request as urllib_request +from typing import Dict, List, Tuple + + +LIVE_ENVIRONMENT_SWITCH = "SKILL2WORKFLOW_LARK_TASK_LIVE" +LIVE_URL = "https://open.feishu.cn/open-apis/task/v2/tasks?user_id_type=open_id" +LIVE_TIMEOUT_SECONDS = 10.0 +REQUIRED_CREDENTIAL_HANDLE = "lark_bot_access_token" +``` + +Change the entrypoint signature to: + +```python +def execute(binding: Dict[str, object], credential_provider=None, context=None, transport=None) -> Dict[str, object]: +``` + +Update only these manifest values; keep the connector id, kind, external entrypoint, credential policy, and audit event list unchanged: + +```python +"description": "Explicit dry-run-default connector with opt-in scoped Feishu task creation.", +"mode": {"type": "string", "enum": ["dry_run", "live"]}, +``` + +Preserve the complete current dry-run branch. For live mode, use these helpers and call order: + +```python +def _live_enabled() -> bool: + return os.environ.get(LIVE_ENVIRONMENT_SWITCH) == "1" + + +def _execution_identity(context: object) -> List[str]: + context_root = context if isinstance(context, dict) else {} + execution = context_root.get("_execution", {}) + if not isinstance(execution, dict): + return [] + values = [ + str(execution.get("workflow_id") or ""), + str(execution.get("workflow_version") or ""), + str(execution.get("run_id") or ""), + str(execution.get("node_id") or ""), + ] + return values if all(values) else [] + + +def _client_token(context: object) -> str: + identity = _execution_identity(context) + if not identity: + raise ConnectorExecutionError("lark_task live execution identity is required") + canonical = json.dumps(identity, ensure_ascii=False, separators=(",", ":")) + return hashlib.sha256(canonical.encode("utf-8")).hexdigest() + + +def _provider_request_body(body: Dict[str, object], context: object) -> Dict[str, object]: + title = body.get("title") + if not isinstance(title, str) or not title.strip(): + raise ConnectorExecutionError("lark_task connector task title is required") + + payload: Dict[str, object] = { + "summary": title, + "client_token": _client_token(context), + } + description = body.get("description") + if description is not None: + if not isinstance(description, str): + raise ConnectorExecutionError("lark_task connector description must be a string") + payload["description"] = description + + assignee = body.get("assignee_open_id") + if assignee is not None: + if not isinstance(assignee, str) or not assignee.strip(): + raise ConnectorExecutionError("lark_task connector assignee_open_id must be a non-empty string") + payload["members"] = [{"id": assignee, "type": "user", "role": "assignee"}] + + due_at = body.get("due_at") + if due_at is not None: + payload["due"] = {"timestamp": _due_timestamp(due_at), "is_all_day": False} + return payload + + +def _due_timestamp(value: object) -> str: + if not isinstance(value, str) or not value.strip(): + raise ConnectorExecutionError("lark_task connector due_at must be an RFC 3339 string") + normalized = value[:-1] + "+00:00" if value.endswith("Z") else value + try: + parsed = datetime.fromisoformat(normalized) + except ValueError: + raise ConnectorExecutionError("lark_task connector due_at must be an RFC 3339 string") + if parsed.tzinfo is None: + raise ConnectorExecutionError("lark_task connector due_at must include a timezone") + return str(int(parsed.timestamp() * 1000)) + + +def _request(payload: Dict[str, object], token: str) -> urllib_request.Request: + return urllib_request.Request( + LIVE_URL, + data=json.dumps(payload, ensure_ascii=False, separators=(",", ":")).encode("utf-8"), + headers={ + "Authorization": f"Bearer {token}", + "Content-Type": "application/json; charset=utf-8", + }, + method="POST", + ) + + +def _default_transport(request: urllib_request.Request, timeout: float): + return urllib_request.urlopen(request, timeout=timeout) +``` + +Change credential resolution to return both compact metadata and an internal handle-to-value map: + +```python +def _resolve_credentials(credentials: object, credential_provider) -> Tuple[Dict[str, object], Dict[str, str]]: + if credentials in (None, []): + return {"status": "skipped", "handles": []}, {} + if not isinstance(credentials, list): + raise ConnectorExecutionError("connector.credentials must be a list") + + handles: List[str] = [] + values: Dict[str, str] = {} + for index, credential in enumerate(credentials): + if not isinstance(credential, dict): + raise ConnectorExecutionError(f"connector.credentials[{index}] must be an object") + target = str(credential.get("target") or "") + if target != "header": + raise ConnectorExecutionError(f"connector.credentials[{index}].target must be header") + handle = str(credential.get("handle") or "") + if not handle: + raise ConnectorExecutionError(f"connector.credentials[{index}].handle is required") + if credential_provider is None: + raise ConnectorExecutionError(f"credential handle not found: {handle}") + try: + values[handle] = credential_provider.resolve(handle) + except CredentialResolutionError as error: + raise ConnectorExecutionError(str(error)) + handles.append(handle) + + return {"status": "resolved", "handles": sorted(handles)}, values +``` + +After live resolution, enforce the approved handle exactly: + +```python +if REQUIRED_CREDENTIAL_HANDLE not in credential_values: + raise ConnectorExecutionError( + f"credential handle not found: {REQUIRED_CREDENTIAL_HANDLE}" + ) +``` + +Dry-run uses only the summary. Live mode requires `REQUIRED_CREDENTIAL_HANDLE` in the internal values and passes it to `_request`. + +Add this Task 2 success parser and compact result builder; Task 3 will extend the parser for failures without changing the result shape: + +```python +def _successful_task_present(response) -> bool: + try: + status = int(getattr(response, "status", 0)) + raw = response.read() + finally: + close = getattr(response, "close", None) + if callable(close): + close() + if status < 200 or status >= 300: + raise ConnectorExecutionError("lark_task live provider request failed") + try: + payload = json.loads(raw.decode("utf-8")) + except (UnicodeDecodeError, json.JSONDecodeError): + raise ConnectorExecutionError("lark_task live provider response is invalid") + if not isinstance(payload, dict) or payload.get("code") != 0: + raise ConnectorExecutionError("lark_task live provider response is invalid") + data = payload.get("data", {}) + task = data.get("task", {}) if isinstance(data, dict) else {} + guid = task.get("guid") if isinstance(task, dict) else "" + if not isinstance(guid, str) or not guid: + raise ConnectorExecutionError("lark_task live provider response is invalid") + return True + + +def _live_result( + status: str, + audit: Dict[str, object], + provider_status: str, + mapping_summary: Dict[str, object], + credential_summary: Dict[str, object] = None, + idempotency_key_present: bool = False, + task_id_present: bool = False, +) -> Dict[str, object]: + compact = dict(audit) + compact.update( + { + "credential_status": str((credential_summary or {}).get("status") or "skipped"), + "idempotency_key_present": idempotency_key_present, + "provider_status": provider_status, + "lark_task_id_present": task_id_present, + } + ) + result = { + "status": status, + "connector": {"id": "lark_task", "kind": "lark_task"}, + "output": dict(compact), + "audit": compact, + "input_mapping": mapping_summary, + } + if credential_summary: + result["credentials"] = credential_summary + if status == "failed": + result["error"] = f"lark_task live request failed: {provider_status}" + return result +``` + +Implement the live branch in this exact order: + +1. map the existing request body and build presence-only audit metadata; +2. if `_live_enabled()` is false, return `_live_result("failed", ..., "live_disabled", ...)` without resolving credentials or calling transport; +3. build the provider payload and `client_token`; +4. resolve credentials and require `REQUIRED_CREDENTIAL_HANDLE`; +5. call `(transport or _default_transport)(_request(payload, token), LIVE_TIMEOUT_SECONDS)`; +6. require `_successful_task_present(response)`; +7. return `_live_result("completed", ..., "completed", ..., credential_summary, True, True)`. + +Do not add live-only keys to the base `_task_audit_metadata`; `_live_result` adds them so existing dry-run audit equality remains unchanged. + +- [x] **Step 5: Run the connector tests and verify GREEN** + +Run: + +```bash +PYTHONPATH=src python3 -m unittest tests.test_lark_task_connector -v +``` + +Expected: all dry-run and new successful live tests PASS. + +- [x] **Step 6: Commit Task 2** + +Run: + +```bash +git add examples/connectors/lark_task_connector.py tests/test_lark_task_connector.py +git commit -m "feat: add scoped live lark task request" +``` + +Expected: one commit containing the live happy path and its focused tests. + +### Task 3: Failure Normalization And Leakage Coverage + +**Files:** +- Modify: `examples/connectors/lark_task_connector.py` +- Modify: `tests/test_lark_task_connector.py` + +**Interfaces:** +- Consumes: Task 2 transport, request builder, credential values, and compact audit metadata +- Produces: `_provider_outcome(status: int, raw: bytes)`, `_failed_live_result(...)`, and stable `provider_status` behavior consumed by runtime audit and Task 4 + +- [x] **Step 1: Add the table-driven provider failure tests** + +Add this test: + +```python + def test_lark_task_live_mode_normalizes_provider_failures_without_leakage(self): + cases = [ + (401, {"code": 999, "msg": "raw-auth-detail"}, "authorization_failed"), + (403, {"code": 1470403, "msg": "raw-permission-detail"}, "permission_denied"), + (429, {"code": 999, "msg": "raw-rate-detail"}, "rate_limited"), + (400, {"code": 1470400, "msg": "raw-validation-detail"}, "validation_failed"), + (404, {"code": 1470404, "msg": "raw-resource-detail"}, "resource_not_found"), + (500, {"code": 1470422, "msg": "raw-idempotency-detail"}, "idempotency_conflict"), + (500, {"code": 1470500, "msg": "raw-provider-detail"}, "provider_unavailable"), + (500, b"not-json-provider-body", "provider_unavailable"), + ] + + for status, payload, expected_status in cases: + with self.subTest(status=status, expected_status=expected_status): + transport = _FakeTransport(status=status, payload=payload) + runtime = ConnectorRuntime([_load_lark_task_connector(transport)]) + with patch.dict(os.environ, {"SKILL2WORKFLOW_LARK_TASK_LIVE": "1"}, clear=True): + result = runtime.execute_connector( + _lark_task_node(mode="live"), + credential_provider=StaticCredentialProvider( + {"lark_bot_access_token": "local-lark-secret"} + ), + context=_execution_context(), + ) + + self.assertEqual(result["status"], "failed") + self.assertEqual(result["audit"]["provider_status"], expected_status) + encoded = json.dumps(result, ensure_ascii=False) + self.assertNotIn("local-lark-secret", encoded) + self.assertNotIn("raw-", encoded) + self.assertNotIn("not-json-provider-body", encoded) +``` + +Add imports for provider exception coverage: + +```python +import io +import socket +from urllib import error as urllib_error +``` + +Add malformed, timeout, and URL-error coverage: + +```python + def test_lark_task_live_mode_normalizes_timeout_and_malformed_success(self): + cases = [ + (_FakeTransport(error=TimeoutError("raw timeout body")), "timeout"), + (_FakeTransport(error=urllib_error.URLError(socket.timeout("raw socket timeout"))), "timeout"), + (_FakeTransport(error=urllib_error.URLError("raw network failure")), "provider_unavailable"), + ( + _FakeTransport( + error=urllib_error.HTTPError( + "https://open.feishu.cn/open-apis/task/v2/tasks", + 403, + "raw http reason", + {}, + io.BytesIO(b'{"code":1470403,"msg":"raw-http-body"}'), + ) + ), + "permission_denied", + ), + (_FakeTransport(payload=b"not-json-success-body"), "malformed_response"), + (_FakeTransport(payload={"code": 0, "data": {}}), "malformed_response"), + (_FakeTransport(payload={"code": 0, "data": {"task": {"guid": ""}}}), "malformed_response"), + ] + + for transport, expected_status in cases: + with self.subTest(expected_status=expected_status): + runtime = ConnectorRuntime([_load_lark_task_connector(transport)]) + with patch.dict(os.environ, {"SKILL2WORKFLOW_LARK_TASK_LIVE": "1"}, clear=True): + result = runtime.execute_connector( + _lark_task_node(mode="live"), + credential_provider=StaticCredentialProvider( + {"lark_bot_access_token": "local-lark-secret"} + ), + context=_execution_context(), + ) + + self.assertEqual(result["status"], "failed") + self.assertEqual(result["audit"]["provider_status"], expected_status) + encoded = json.dumps(result, ensure_ascii=False) + self.assertNotIn("raw timeout body", encoded) + self.assertNotIn("raw socket timeout", encoded) + self.assertNotIn("raw network failure", encoded) + self.assertNotIn("raw http reason", encoded) + self.assertNotIn("raw-http-body", encoded) + self.assertNotIn("not-json-success-body", encoded) +``` + +Add this preflight test proving the transport is not called for missing execution identity, invalid due time, or missing credential: + +```python + def test_lark_task_live_preflight_failures_never_call_transport(self): + missing_execution = {"input": dict(_execution_context()["input"])} + invalid_due = json.loads(json.dumps(_execution_context())) + invalid_due["input"]["due_at"] = "2026-07-09T09:00:00" + cases = [ + (missing_execution, StaticCredentialProvider({"lark_bot_access_token": "secret"}), "validation_failed"), + (invalid_due, StaticCredentialProvider({"lark_bot_access_token": "secret"}), "validation_failed"), + (_execution_context(), StaticCredentialProvider({}), "credential_failed"), + ] + + for context, provider, expected_status in cases: + with self.subTest(expected_status=expected_status): + transport = _FakeTransport() + runtime = ConnectorRuntime([_load_lark_task_connector(transport)]) + with patch.dict(os.environ, {"SKILL2WORKFLOW_LARK_TASK_LIVE": "1"}, clear=True): + result = runtime.execute_connector( + _lark_task_node(mode="live"), + credential_provider=provider, + context=context, + ) + + self.assertEqual(result["status"], "failed") + self.assertEqual(result["audit"]["provider_status"], expected_status) + self.assertEqual(transport.calls, []) +``` + +- [x] **Step 2: Run the new tests and verify RED** + +Run: + +```bash +PYTHONPATH=src python3 -m unittest tests.test_lark_task_connector -v +``` + +Expected: FAIL because provider and transport failures are not yet returned as compact failed results. + +- [x] **Step 3: Implement fixed failure classification** + +Add: + +```python +PROVIDER_CODE_STATUS = { + 1470400: "validation_failed", + 1470403: "permission_denied", + 1470404: "resource_not_found", + 1470422: "idempotency_conflict", + 1470500: "provider_unavailable", +} + + +def _http_status(status: int) -> str: + if status == 400: + return "validation_failed" + if status == 401: + return "authorization_failed" + if status == 403: + return "permission_denied" + if status == 404: + return "resource_not_found" + if status == 429: + return "rate_limited" + if status >= 500: + return "provider_unavailable" + return "malformed_response" + + +def _decode_provider(raw: bytes): + try: + payload = json.loads(raw.decode("utf-8")) + except (UnicodeDecodeError, json.JSONDecodeError): + return None + return payload if isinstance(payload, dict) else None + + +def _provider_outcome(status: int, raw: bytes) -> Tuple[str, bool]: + payload = _decode_provider(raw) + if payload is not None: + code = payload.get("code") + if code == 0: + data = payload.get("data", {}) + task = data.get("task", {}) if isinstance(data, dict) else {} + guid = task.get("guid") if isinstance(task, dict) else "" + if isinstance(guid, str) and guid: + return "completed", True + return "malformed_response", False + if isinstance(code, int) and code in PROVIDER_CODE_STATUS: + return PROVIDER_CODE_STATUS[code], False + return _http_status(status), False + + +def _failed_live_result( + audit: Dict[str, object], + provider_status: str, + mapping_summary: Dict[str, object], + credential_summary: Dict[str, object] = None, + idempotency_key_present: bool = False, +) -> Dict[str, object]: + compact = dict(audit) + compact.update( + { + "credential_status": str((credential_summary or {}).get("status") or "skipped"), + "idempotency_key_present": idempotency_key_present, + "provider_status": provider_status, + "lark_task_id_present": False, + } + ) + result = { + "status": "failed", + "connector": {"id": "lark_task", "kind": "lark_task"}, + "output": dict(compact), + "error": f"lark_task live request failed: {provider_status}", + "audit": compact, + "input_mapping": mapping_summary, + } + if credential_summary: + result["credentials"] = credential_summary + return result +``` + +Add imports: + +```python +import socket +from urllib import error as urllib_error +``` + +Add the complete transport wrapper: + +```python +def _transport_outcome(request: urllib_request.Request, transport) -> Tuple[str, bool]: + try: + response = transport(request, LIVE_TIMEOUT_SECONDS) + except urllib_error.HTTPError as error: + try: + raw = error.read() + finally: + error.close() + return _provider_outcome(int(error.code), raw) + except (TimeoutError, socket.timeout): + return "timeout", False + except urllib_error.URLError as error: + if isinstance(error.reason, (TimeoutError, socket.timeout)): + return "timeout", False + return "provider_unavailable", False + + try: + status = int(getattr(response, "status", 0)) + raw = response.read() + finally: + close = getattr(response, "close", None) + if callable(close): + close() + return _provider_outcome(status, raw) +``` + +Replace the Task 2 live sender's direct response parsing with `_transport_outcome(request, transport or _default_transport)`. Build a completed result only for `("completed", True)`; return `_failed_live_result(..., idempotency_key_present=True)` for every post-transport failure. + +Provider code classification must precede generic HTTP classification. Never include `msg`, response bytes, exception text, request JSON, token, or task guid in result or error text. + +In the live branch, wrap `_provider_request_body(...)` in `except ConnectorExecutionError` and return `_failed_live_result(..., "validation_failed", ..., idempotency_key_present=False)`. Wrap `_resolve_credentials(...)` in a separate `except ConnectorExecutionError` and return `_failed_live_result(..., "credential_failed", ..., credential_summary={"status": "failed", "handles": [REQUIRED_CREDENTIAL_HANDLE]}, idempotency_key_present=True)` because the client token has already been derived. Preserve existing dry-run exceptions exactly. + +- [x] **Step 4: Run focused connector and dry-run smoke tests** + +Run: + +```bash +PYTHONPATH=src python3 -m unittest tests.test_lark_task_connector tests.test_lark_task_connector_smoke tests.test_lark_task_pilot -v +python3 scripts/lark_task_connector_smoke.py --work-dir /tmp/skill2workflow-lark-task-connector +python3 scripts/lark_task_pilot_smoke.py --work-dir /tmp/skill2workflow-lark-task-pilot +``` + +Expected: all tests and both dry-run smokes PASS without live network access. + +- [x] **Step 5: Commit Task 3** + +Run: + +```bash +git add examples/connectors/lark_task_connector.py tests/test_lark_task_connector.py +git commit -m "feat: normalize live lark task failures" +``` + +Expected: one commit containing failure and leakage hardening. + +### Task 4: Guarded Live Validation Command + +**Files:** +- Create: `src/skill2workflow/lark_task_live_validation.py` +- Create: `scripts/lark_task_live_validation.py` +- Create: `tests/test_lark_task_live_validation.py` + +**Interfaces:** +- Consumes: Task 2/3 `lark_task` entrypoint, `ConnectorRuntime`, `ExternalConnector`, `StaticCredentialProvider` +- Produces: `run_lark_task_live_validation(repo_root, title, description, assignee_open_id, validation_run_id, confirmed, transport=None) -> Dict[str, object]` and `main(argv=None) -> int` + +- [x] **Step 1: Write validation guard and compact-output tests** + +Create `tests/test_lark_task_live_validation.py`: + +```python +import json +import os +from pathlib import Path +from unittest import TestCase +from unittest.mock import patch + +from skill2workflow.lark_task_live_validation import run_lark_task_live_validation + + +ROOT = Path(__file__).resolve().parents[1] + + +class LarkTaskLiveValidationTests(TestCase): + def test_live_validation_requires_confirmation_switch_token_and_identity(self): + cases = [ + ({}, False, "run_validation", "ou_test", "live validation requires --confirm-live-create"), + ({"SKILL2WORKFLOW_LARK_TASK_LIVE": "1"}, True, "run_validation", "ou_test", "LARK_BOT_ACCESS_TOKEN is required"), + ({"LARK_BOT_ACCESS_TOKEN": "secret"}, True, "run_validation", "ou_test", "SKILL2WORKFLOW_LARK_TASK_LIVE=1 is required"), + ({"SKILL2WORKFLOW_LARK_TASK_LIVE": "1", "LARK_BOT_ACCESS_TOKEN": "secret"}, True, "", "ou_test", "validation run id is required"), + ({"SKILL2WORKFLOW_LARK_TASK_LIVE": "1", "LARK_BOT_ACCESS_TOKEN": "secret"}, True, "run_validation", "", "assignee open id is required"), + ] + + for environment, confirmed, run_id, assignee, expected in cases: + with self.subTest(expected=expected), patch.dict(os.environ, environment, clear=True): + with self.assertRaisesRegex(ValueError, expected): + run_lark_task_live_validation( + ROOT, + title="Validation title", + description="Validation description", + assignee_open_id=assignee, + validation_run_id=run_id, + confirmed=confirmed, + transport=lambda request, timeout: None, + ) + + def test_live_validation_returns_only_compact_metadata(self): + transport = _FakeTransport() + environment = { + "SKILL2WORKFLOW_LARK_TASK_LIVE": "1", + "LARK_BOT_ACCESS_TOKEN": "live-validation-secret", + } + with patch.dict(os.environ, environment, clear=True): + result = run_lark_task_live_validation( + ROOT, + title="Validation title", + description="Validation description", + assignee_open_id="ou_validation", + validation_run_id="run_validation", + confirmed=True, + transport=transport, + ) + + self.assertEqual( + result, + { + "ok": True, + "connector_id": "lark_task", + "operation": "create_task", + "mode": "live", + "credential_status": "resolved", + "idempotency_key_present": True, + "provider_status": "completed", + "lark_task_id_present": True, + "assignee_present": True, + }, + ) + encoded = json.dumps(result) + for forbidden in ( + "live-validation-secret", + "Validation title", + "Validation description", + "ou_validation", + "task-guid-must-not-leak", + ): + self.assertNotIn(forbidden, encoded) + + +class _FakeResponse: + status = 200 + + def read(self): + return json.dumps( + {"code": 0, "msg": "success", "data": {"task": {"guid": "task-guid-must-not-leak"}}} + ).encode("utf-8") + + def close(self): + return None + + +class _FakeTransport: + def __init__(self): + self.calls = [] + + def __call__(self, request, timeout): + self.calls.append((request, timeout)) + return _FakeResponse() +``` + +- [x] **Step 2: Run the new test and verify RED** + +Run: + +```bash +PYTHONPATH=src python3 -m unittest tests.test_lark_task_live_validation -v +``` + +Expected: FAIL because the validation module does not exist. + +- [x] **Step 3: Implement the validation module** + +Create `src/skill2workflow/lark_task_live_validation.py` with: + +```python +"""Guarded one-shot validation for the scoped live Lark task connector.""" + +from __future__ import annotations + +import argparse +import json +import os +from pathlib import Path +from typing import Dict + +from .connectors import ConnectorRuntime, ExternalConnector +from .credentials import StaticCredentialProvider +from .external_connectors import load_external_connector + + +LIVE_SWITCH = "SKILL2WORKFLOW_LARK_TASK_LIVE" +TOKEN_ENVIRONMENT = "LARK_BOT_ACCESS_TOKEN" + + +def run_lark_task_live_validation( + repo_root: Path, + title: str, + description: str, + assignee_open_id: str, + validation_run_id: str, + confirmed: bool, + transport=None, +) -> Dict[str, object]: + if not confirmed: + raise ValueError("live validation requires --confirm-live-create") + if os.environ.get(LIVE_SWITCH) != "1": + raise ValueError("SKILL2WORKFLOW_LARK_TASK_LIVE=1 is required") + token = os.environ.get(TOKEN_ENVIRONMENT, "") + if not token: + raise ValueError("LARK_BOT_ACCESS_TOKEN is required") + if not validation_run_id: + raise ValueError("validation run id is required") + if not assignee_open_id: + raise ValueError("assignee open id is required") + + connector = load_external_connector(Path(repo_root) / "examples" / "connectors" / "lark_task_connector.py") + if transport is not None: + original = connector + + def execute_with_transport(binding, credential_provider=None, context=None): + return original.executor( + binding, + credential_provider=credential_provider, + context=context, + transport=transport, + ) + + connector = ExternalConnector(manifest=original.manifest, executor=execute_with_transport) + + runtime = ConnectorRuntime([connector]) + result = runtime.execute_connector( + _validation_node(title, description, assignee_open_id), + credential_provider=StaticCredentialProvider({"lark_bot_access_token": token}), + context={ + "_execution": { + "workflow_id": "workflow_lark_task_live_validation", + "workflow_version": "0.1.0", + "run_id": validation_run_id, + "node_id": "create_lark_task", + } + }, + ) + audit = result.get("audit", {}) if isinstance(result.get("audit"), dict) else {} + return { + "ok": result.get("status") == "completed", + "connector_id": str(result.get("connector", {}).get("id", "")), + "operation": str(audit.get("operation", "")), + "mode": str(audit.get("mode", "")), + "credential_status": str(audit.get("credential_status", "")), + "idempotency_key_present": bool(audit.get("idempotency_key_present")), + "provider_status": str(audit.get("provider_status", "")), + "lark_task_id_present": bool(audit.get("lark_task_id_present")), + "assignee_present": bool(audit.get("assignee_present")), + } + + +def _validation_node(title: str, description: str, assignee_open_id: str) -> Dict[str, object]: + return { + "id": "create_lark_task", + "type": "tool_call", + "connector": { + "id": "lark_task", + "kind": "lark_task", + "operation": "create_task", + "mode": "live", + "request": { + "body": { + "title": title, + "description": description, + "assignee_open_id": assignee_open_id, + } + }, + "credentials": [ + { + "target": "header", + "name": "Authorization", + "handle": "lark_bot_access_token", + "prefix": "Bearer ", + } + ], + }, + } + + +def main(argv=None) -> int: + parser = argparse.ArgumentParser(prog="lark_task_live_validation") + parser.add_argument("--repo-root", type=Path, default=Path(__file__).resolve().parents[2]) + parser.add_argument("--title", required=True) + parser.add_argument("--description", required=True) + parser.add_argument("--assignee-open-id", required=True) + parser.add_argument("--validation-run-id", required=True) + parser.add_argument("--confirm-live-create", action="store_true") + args = parser.parse_args(argv) + result = run_lark_task_live_validation( + args.repo_root, + title=args.title, + description=args.description, + assignee_open_id=args.assignee_open_id, + validation_run_id=args.validation_run_id, + confirmed=args.confirm_live_create, + ) + print(json.dumps(result, ensure_ascii=False, indent=2)) + return 0 if result["ok"] else 1 +``` + +Create `scripts/lark_task_live_validation.py` using the same source-checkout wrapper pattern as `scripts/lark_task_connector_smoke.py`, importing `main` from `skill2workflow.lark_task_live_validation`. + +- [x] **Step 4: Run validation and focused connector tests** + +Run: + +```bash +PYTHONPATH=src python3 -m unittest tests.test_lark_task_live_validation tests.test_lark_task_connector -v +``` + +Expected: all tests PASS with injected transport only. + +- [x] **Step 5: Commit Task 4** + +Run: + +```bash +git add src/skill2workflow/lark_task_live_validation.py scripts/lark_task_live_validation.py tests/test_lark_task_live_validation.py +git commit -m "feat: add guarded lark live validation" +``` + +Expected: one commit containing the validation command and tests. + +### Task 5: Public Connector Contract Documentation + +**Files:** +- Modify: `docs/connectors.md` +- Modify: `docs/lark-live-connector-readiness.md` +- Modify: `tests/test_live_connector_readiness.py` + +**Interfaces:** +- Consumes: implemented behavior from Tasks 1-4 and the approved design +- Produces: tested public documentation for activation, endpoint, native idempotency, durable-input interpretation, failure categories, and validation command + +- [x] **Step 1: Extend the failing documentation contract** + +Add assertions to `test_lark_live_connector_readiness_decision_is_documented` for these exact strings across the readiness and connector guides: + +```python + self.assertIn("https://open.feishu.cn/open-apis/task/v2/tasks?user_id_type=open_id", decision) + self.assertIn("native `client_token`", decision) + self.assertIn("SKILL2WORKFLOW_LARK_TASK_LIVE=1", decision) + self.assertIn("run.context.input", decision) + self.assertIn("must not copy raw task values into connector-produced state", decision) + self.assertIn("python3 scripts/lark_task_live_validation.py", decision) + + self.assertIn("mode: live", connectors) + self.assertIn("SKILL2WORKFLOW_LARK_TASK_LIVE=1", connectors) + self.assertIn("provider_status", connectors) + self.assertIn("LARK_BOT_ACCESS_TOKEN", connectors) +``` + +- [x] **Step 2: Run the docs contract and verify RED** + +Run: + +```bash +PYTHONPATH=src python3 -m unittest tests.test_live_connector_readiness -v +``` + +Expected: FAIL because the docs still describe live mode as future work. + +- [x] **Step 3: Update connector and readiness documentation** + +Document these exact boundaries: + +- fixed Task API v2 endpoint and `user_id_type=open_id`; +- `mode: live` plus exact environment switch; +- dry-run default and explicit loading; +- native `client_token` derived from runtime identity; +- `lark_bot_access_token` handle and `LARK_BOT_ACCESS_TOKEN` only for the guarded validation helper; +- required provider scope is either `task:task:write` or `task:task:writeonly`; +- documented provider limit is 10 create requests per second; +- 10-second timeout; +- compact `provider_status` categories from the design; +- no raw provider messages, task values, task guid, token, request, or response in connector-produced state; +- durable `run.context.input` remains unchanged and may contain user-supplied task input; +- fake transport in CI and guarded validation command outside CI; +- rollback by removing the environment switch. + +Preserve the historical sentence `examples/connectors/lark_task_connector.py remains dry-run-only in Loop 38` because existing tests use it as Loop 38 evidence. Replace only current-state language saying the package remains dry-run-only with language saying dry-run remains the default and live is opt-in. + +- [x] **Step 4: Run documentation and connector tests** + +Run: + +```bash +PYTHONPATH=src python3 -m unittest tests.test_live_connector_readiness tests.test_lark_task_connector tests.test_lark_task_live_validation -v +``` + +Expected: all tests PASS. + +- [x] **Step 5: Commit Task 5** + +Run: + +```bash +git add docs/connectors.md docs/lark-live-connector-readiness.md tests/test_live_connector_readiness.py +git commit -m "docs: document scoped live lark connector" +``` + +Expected: one docs-contract commit. + +### Task 6: Offline Verification And Approved Real Validation + +**Files:** +- Create after successful call: `docs/lark-live-connector-validation.md` +- Create: `tests/test_lark_live_connector_validation_docs.py` + +**Interfaces:** +- Consumes: guarded validation command from Task 4, Vault secret `LARK_BOT_ACCESS_TOKEN`, and the current Lark user's authenticated `open_id` from conversation metadata +- Produces: one retained Feishu validation task and one committed redacted evidence note + +- [x] **Step 1: Run all offline verification before any live write** + +Run: + +```bash +PYTHONPATH=src python3 -m unittest discover -s tests -v +python3 -m py_compile src/skill2workflow/*.py examples/connectors/lark_task_connector.py +python3 scripts/secret_hygiene.py examples/workflows +python3 scripts/lark_task_connector_smoke.py --work-dir /tmp/skill2workflow-lark-task-connector +python3 scripts/lark_task_pilot_smoke.py --work-dir /tmp/skill2workflow-lark-task-pilot +git diff --check +``` + +Expected: all tests, compilation, hygiene, smokes, and diff checks PASS. Stop before live validation if any command fails. + +- [x] **Step 2: Locate or request the protected Vault secret** + +Run: + +```bash +vibe vault find LARK_BOT_ACCESS_TOKEN +``` + +If no static secret exists, run exactly: + +```bash +vibe vault request LARK_BOT_ACCESS_TOKEN --reason "Validate the explicitly approved Loop 39 Feishu create_task live path once" --spec-json '{"kind":"static","protection":"protected","description":"Short-lived Feishu bot access token for the approved Loop 39 validation","tags":["feishu","loop39","skill:lark-live-validation"],"policy":{"allowed_hosts":["open.feishu.cn"],"auth":{"type":"bearer"}}}' +``` + +Expected: the Vault either reports an existing static secret or asks the user to add/approve it in the browser. Never ask the user to paste the token into chat. + +- [x] **Step 3: Prepare ephemeral approved task parameters** + +In the execution shell, set these three variables from the already approved conversation values and authenticated current-message metadata: + +```bash +test -n "$LARK_VALIDATION_TITLE" +test -n "$LARK_VALIDATION_DESCRIPTION" +test -n "$LARK_VALIDATION_ASSIGNEE_OPEN_ID" +``` + +Expected: all three checks exit `0`. Do not write their values to a repository file, report, audit note, or terminal output. + +- [x] **Step 4: Execute the approved live write exactly once** + +Run: + +```bash +vibe vault run --env LARK_BOT_ACCESS_TOKEN -- env \ + SKILL2WORKFLOW_LARK_TASK_LIVE=1 \ + PYTHONPATH=src \ + python3 scripts/lark_task_live_validation.py \ + --confirm-live-create \ + --validation-run-id skill2workflow-loop39-live-validation-20260711 \ + --title "$LARK_VALIDATION_TITLE" \ + --description "$LARK_VALIDATION_DESCRIPTION" \ + --assignee-open-id "$LARK_VALIDATION_ASSIGNEE_OPEN_ID" +``` + +Expected compact output: + +```json +{ + "ok": true, + "connector_id": "lark_task", + "operation": "create_task", + "mode": "live", + "credential_status": "resolved", + "idempotency_key_present": true, + "provider_status": "completed", + "lark_task_id_present": true, + "assignee_present": true +} +``` + +If protected Vault use requests approval, wait for the user to approve and rerun the same `vibe vault run` command once. If the provider response is not completed, stop and diagnose without changing the idempotency run id or task parameters. + +- [x] **Step 5: Write the failing evidence-note contract test** + +Create `tests/test_lark_live_connector_validation_docs.py`: + +```python +from pathlib import Path +from unittest import TestCase + + +ROOT = Path(__file__).resolve().parents[1] + + +class LarkLiveConnectorValidationDocsTests(TestCase): + def test_live_validation_evidence_is_compact_and_redacted(self): + evidence = (ROOT / "docs" / "lark-live-connector-validation.md").read_text(encoding="utf-8") + + self.assertIn("# Lark/Feishu Live Connector Validation", evidence) + self.assertIn("- connector_id: `lark_task`", evidence) + self.assertIn("- operation: `create_task`", evidence) + self.assertIn("- mode: `live`", evidence) + self.assertIn("- credential_status: `resolved`", evidence) + self.assertIn("- idempotency_key_present: `true`", evidence) + self.assertIn("- provider_status: `completed`", evidence) + self.assertIn("- lark_task_id_present: `true`", evidence) + self.assertIn("- assignee_present: `true`", evidence) + self.assertIn("Raw task values, user ids, credentials, request bodies, response bodies, and task ids are intentionally omitted.", evidence) +``` + +- [x] **Step 6: Run the evidence test and verify RED** + +Run: + +```bash +PYTHONPATH=src python3 -m unittest tests.test_lark_live_connector_validation_docs -v +``` + +Expected: FAIL because the evidence file does not exist. + +- [x] **Step 7: Create the redacted evidence note from actual compact output** + +Create `docs/lark-live-connector-validation.md` with: + +- heading `# Lark/Feishu Live Connector Validation`; +- the actual UTC validation timestamp returned or recorded immediately after the call; +- the eight exact compact fields asserted by the test; +- the exact omission sentence asserted by the test; +- the offline verification commands used before the write; +- a statement that the task was assigned to the consenting current user and retained as visible evidence, without the user id or task content. + +Do not include the task title, description, assignee id, task guid, token, request body, response body, provider message, or idempotency digest. + +- [x] **Step 8: Run evidence test and secret checks** + +Run: + +```bash +PYTHONPATH=src python3 -m unittest tests.test_lark_live_connector_validation_docs -v +python3 scripts/secret_hygiene.py examples/workflows +git diff --check +``` + +Expected: test PASS, no secret findings, no whitespace errors. + +- [x] **Step 9: Commit Task 6 evidence** + +Run: + +```bash +git add docs/lark-live-connector-validation.md tests/test_lark_live_connector_validation_docs.py +git commit -m "docs: record live lark connector validation" +``` + +Expected: one commit containing only redacted evidence and its contract test. + +### Task 7: Loop 39 Roadmap Transition And Final Verification + +**Files:** +- Modify: `ROADMAP.md` +- Modify: `README.md` +- Modify: `tests/test_production_roadmap.py` +- Modify: `tests/test_product_connector_pilot_roadmap.py` +- Modify: `tests/test_first_product_connector_candidate_docs.py` +- Modify: `tests/test_live_connector_readiness.py` +- Modify: `docs/superpowers/plans/2026-07-11-scoped-live-lark-task.md` + +**Interfaces:** +- Consumes: all implementation, offline evidence, and real validation from Tasks 1-6 +- Produces: completed Loop 39 history, active Loop 40 planning state, matching README summary, and a checked execution record + +- [x] **Step 1: Update Roadmap contract tests to the completed state** + +Update exact assertions so they require: + +```text +Completed delivery loops: 1-39 +Active loop: Loop 40, Controlled Live Connector Pilot +| Loop 39: Scoped Live Lark Task Connector | Complete | +| Loop 40: Controlled Live Connector Pilot | Next | +Delivery Loops 1-39 are complete +``` + +Keep Loops 41-43 as Candidate. Preserve the four maturity gates and keep current maturity as Local Evaluation. + +Add assertions that Roadmap links `docs/lark-live-connector-validation.md`, keeps live behavior limited to the fixed `create_task` action, and distinguishes the one connector validation from the Loop 40 controlled business-workflow pilot. + +- [x] **Step 2: Run Roadmap tests and verify RED** + +Run: + +```bash +PYTHONPATH=src python3 -m unittest tests.test_production_roadmap tests.test_product_connector_pilot_roadmap tests.test_first_product_connector_candidate_docs tests.test_live_connector_readiness -v +``` + +Expected: FAIL because Roadmap and README still show Loop 39 active. + +- [x] **Step 3: Transition Roadmap and README** + +In `ROADMAP.md`: + +- set completed loops to 1-39; +- set active loop to Loop 40; +- move Loop 39 into Delivery History with fake-transport, native-idempotency, redaction, rollback, and real-validation evidence; +- make Loop 40 `Next` and expand its active-loop section with goal, why-now evidence, scope, exclusions, acceptance evidence, and verification direction; +- retain Loops 41-43 as candidates; +- link the redacted validation note; +- state that the single connector validation is not the controlled real-team workflow pilot; +- keep current maturity at Local Evaluation until Loop 40 completes. + +In `README.md`, change only the compact Roadmap summary to Loops 1-39 complete and Loop 40 active. Do not copy Loop 41-43 titles or acceptance criteria. + +- [x] **Step 4: Run Roadmap and all focused Loop 39 tests** + +Run: + +```bash +PYTHONPATH=src python3 -m unittest \ + tests.test_production_roadmap \ + tests.test_product_connector_pilot_roadmap \ + tests.test_first_product_connector_candidate_docs \ + tests.test_live_connector_readiness \ + tests.test_lark_live_connector_validation_docs \ + tests.test_lark_task_connector \ + tests.test_lark_task_live_validation \ + tests.test_executor -v +``` + +Expected: all focused tests PASS. + +- [x] **Step 5: Commit the Roadmap transition** + +Run: + +```bash +git add ROADMAP.md README.md tests/test_production_roadmap.py tests/test_product_connector_pilot_roadmap.py tests/test_first_product_connector_candidate_docs.py tests/test_live_connector_readiness.py +git commit -m "docs: complete loop 39 live lark connector" +``` + +Expected: one Roadmap/README contract commit. + +- [x] **Step 6: Run final full verification** + +Run: + +```bash +PYTHONPATH=src python3 -m unittest discover -s tests -v +python3 -m py_compile src/skill2workflow/*.py examples/connectors/lark_task_connector.py +python3 scripts/secret_hygiene.py examples/workflows +python3 scripts/lark_task_connector_smoke.py --work-dir /tmp/skill2workflow-lark-task-connector +python3 scripts/lark_task_pilot_smoke.py --work-dir /tmp/skill2workflow-lark-task-pilot +git diff --check +git status --short +``` + +Expected: all tests and checks PASS; only this plan is modified for checkbox completion. + +- [x] **Step 7: Mark every completed plan step** + +Change each successfully executed `- [ ]` checkbox in this file to `- [x]`. Do not mark a step whose expected command or live-write outcome was not achieved. + +- [x] **Step 8: Commit the completed execution record** + +Run: + +```bash +git add docs/superpowers/plans/2026-07-11-scoped-live-lark-task.md +git commit -m "docs: complete scoped live lark task plan" +``` + +Expected: the final commit contains only the checked implementation plan. diff --git a/docs/superpowers/plans/2026-07-17-controlled-live-pilot.md b/docs/superpowers/plans/2026-07-17-controlled-live-pilot.md new file mode 100644 index 0000000..1a42cda --- /dev/null +++ b/docs/superpowers/plans/2026-07-17-controlled-live-pilot.md @@ -0,0 +1,1997 @@ +# Controlled Live Connector Pilot Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Deliver and operate Loop 40 as a paid, assisted, controlled real-team sales-renewal pilot with explicit human approval, five approved live runs across five calendar days, redacted evidence, failure and rollback exercises, and a final `continue`, `harden`, or `defer` decision. + +**Architecture:** Reuse the existing published sales-renewal workflow and out-of-core Lark connector, but add a separate controlled-live orchestration path backed by SQLite private state. Keep raw business input and the Vault-injected token outside the repository; derive a strict allowlisted evidence pack through a separate evidence module. Do not advance the Roadmap until the multi-day real pilot, exercises, commercial acknowledgement, partner acknowledgement, and complete verification all pass. + +**Tech Stack:** Python 3.9 standard library, `unittest`, existing `LocalControlPlane`, SQLite storage, `ConnectorRuntime`, `StaticCredentialProvider`, explicit external connector loading, Avibe Vault process injection, JSON evidence, Markdown documentation. + +## Global Constraints + +- Workflow DSL remains the execution source of truth. +- Python 3.9 standard library remains sufficient; add no runtime dependency. +- The production direction remains self-hosted and single-tenant for one team. +- The only live operation is `lark_task.create_task` against the fixed Feishu domestic boundary. +- The Lark connector remains out of core and must be explicitly loaded. +- `dry_run` remains the connector and committed-example default. +- Existing Workflow DSL `0.1.0` compatibility remains unchanged. +- Private runtime state uses SQLite and must live outside the repository. +- The token may enter only as `LARK_BOT_ACCESS_TOKEN` injected into the approval process; never accept it as a CLI argument or file field. +- Real provider calls never run in unit tests or CI. +- Evidence must omit customer identity, account id/name, renewal-risk text, assignee open id, due time, credentials, authorization headers, request/response bodies, provider messages, task ids, and idempotency digests. +- At least five approved live workflow runs must span five distinct dates in the charter timezone `Asia/Shanghai` and represent at least two opaque private case ids. +- At least one human rejection, one disabled-live failure exercise, and one rollback exercise are mandatory. +- A paid or contractually committed assisted engagement is represented only as `commercial_engagement_confirmed: true`. +- Fake-transport success is implementation evidence, not real-pilot completion evidence. +- Roadmap completion occurs only in the final task after all real evidence passes. +- Parser, compiler, validator, executor, connector, storage, or CLI behavior changes begin with a failing test. + +--- + +## File Map + +- Modify `src/skill2workflow/lark_task_pilot.py`: expose the existing sales-renewal Workflow DSL builder without changing dry-run behavior. +- Create `src/skill2workflow/controlled_lark_pilot.py`: private workspace, charter, case validation, workflow start, explicit decision, exercises, verification, and finalization orchestration. +- Create `src/skill2workflow/controlled_lark_pilot_evidence.py`: pure redaction, evidence extraction, acceptance aggregation, validation, and atomic JSON writes. +- Create `scripts/controlled_lark_pilot.py`: thin source-checkout CLI wrapper. +- Modify `tests/test_lark_task_pilot.py`: reusable-template and dry-run regression tests. +- Create `tests/test_controlled_lark_pilot.py`: charter, private-state, start, decision, exercise, verification, and finalization tests. +- Create `tests/test_controlled_lark_pilot_evidence.py`: allowlist, leakage, aggregation, and deterministic-write tests. +- Create `tests/test_controlled_lark_pilot_docs.py`: runbook, command, evidence, and Roadmap-boundary contracts. +- Create `docs/controlled-live-pilot.md`: operator runbook for the assisted paid pilot. +- Modify `docs/connectors.md`: link the controlled pilot while retaining the narrow live boundary. +- Modify `docs/examples.md`: distinguish dry-run scenario evidence from the controlled live pilot. +- Modify `ROADMAP.md`, `README.md`, and existing Roadmap tests only after the real pilot finalizes. +- Generate `docs/pilot-evidence/loop-40/` only from a validated finalized private evidence pack. + +--- + +### Task 1: Reusable Sales-Renewal Workflow Template + +**Files:** +- Modify: `tests/test_lark_task_pilot.py` +- Modify: `src/skill2workflow/lark_task_pilot.py` + +**Interfaces:** +- Consumes: existing `_lark_task_pilot_workflow()` implementation. +- Produces: `build_lark_task_pilot_workflow(mode: str = "dry_run", workflow_id: str = "workflow_lark_task_pilot", workflow_version: str = "0.1.0", workflow_name: str = "lark-task-sales-renewal-pilot") -> Dict[str, object]`. + +- [x] **Step 1: Write failing template and regression tests** + +Add the import and tests: + +```python +from skill2workflow.lark_task_pilot import ( + build_lark_task_pilot_workflow, + run_lark_task_pilot, +) + + +def test_lark_task_pilot_workflow_builder_keeps_dry_run_default(self): + workflow = build_lark_task_pilot_workflow() + node = next(item for item in workflow["nodes"] if item["id"] == "create_lark_task") + + self.assertEqual(workflow["workflow"]["id"], "workflow_lark_task_pilot") + self.assertEqual(workflow["workflow"]["version"], "0.1.0") + self.assertEqual(node["connector"]["mode"], "dry_run") + self.assertIn("dry-run", node["description"]) + + +def test_lark_task_pilot_workflow_builder_can_create_separate_live_artifact(self): + workflow = build_lark_task_pilot_workflow( + mode="live", + workflow_id="workflow_controlled_lark_pilot", + workflow_version="0.1.0", + workflow_name="controlled-lark-task-sales-renewal-pilot", + ) + node = next(item for item in workflow["nodes"] if item["id"] == "create_lark_task") + + self.assertEqual(workflow["workflow"]["id"], "workflow_controlled_lark_pilot") + self.assertEqual(workflow["workflow"]["name"], "controlled-lark-task-sales-renewal-pilot") + self.assertEqual(node["connector"]["mode"], "live") + self.assertNotIn("dry-run", node["description"]) + + +def test_lark_task_pilot_workflow_builder_rejects_unknown_mode(self): + with self.assertRaisesRegex(ValueError, "mode must be dry_run or live"): + build_lark_task_pilot_workflow(mode="other") +``` + +- [x] **Step 2: Run the focused test and verify RED** + +Run: + +```bash +PYTHONPATH=src python3 -m unittest tests.test_lark_task_pilot -v +``` + +Expected: import failure because `build_lark_task_pilot_workflow` does not exist. + +- [x] **Step 3: Expose and parameterize the existing builder** + +Apply these exact edits to the existing dictionary so every unchanged node and edge remains in place: + +```diff +- workflow = _lark_task_pilot_workflow() ++ workflow = build_lark_task_pilot_workflow() +@@ +-def _lark_task_pilot_workflow() -> Dict[str, object]: ++def build_lark_task_pilot_workflow( ++ mode: str = "dry_run", ++ workflow_id: str = "workflow_lark_task_pilot", ++ workflow_version: str = "0.1.0", ++ workflow_name: str = "lark-task-sales-renewal-pilot", ++) -> Dict[str, object]: ++ if mode not in ("dry_run", "live"): ++ raise ValueError("mode must be dry_run or live") ++ live = mode == "live" + return { +@@ +- "id": "workflow_lark_task_pilot", +- "name": "lark-task-sales-renewal-pilot", +- "description": "Local sales renewal risk pilot using the Lark/Feishu task dry-run connector.", +- "version": "0.1.0", ++ "id": workflow_id, ++ "name": workflow_name, ++ "description": ( ++ "Controlled sales renewal risk pilot using the scoped live Lark/Feishu task connector." ++ if live ++ else "Local sales renewal risk pilot using the Lark/Feishu task dry-run connector." ++ ), ++ "version": workflow_version, +@@ +- "description": "Validate a Lark/Feishu owner follow-up task request without calling the live API.", ++ "description": ( ++ "Create the approved Lark/Feishu owner follow-up task through the scoped live connector." ++ if live ++ else "Validate a Lark/Feishu owner follow-up task request without calling the live API." ++ ), +@@ +- "instruction": "Create a dry-run Lark/Feishu task for the account owner.", ++ "instruction": ( ++ "Create the approved live Lark/Feishu task for the account owner." ++ if live ++ else "Create a dry-run Lark/Feishu task for the account owner." ++ ), +@@ +- "mode": "dry_run", ++ "mode": mode, +``` + +- [x] **Step 4: Run focused and fixture tests and verify GREEN** + +Run: + +```bash +PYTHONPATH=src python3 -m unittest tests.test_lark_task_pilot tests.test_examples -v +``` + +Expected: all tests pass; the existing smoke still emits `mode: dry_run`. + +- [x] **Step 5: Commit the reusable template** + +```bash +git add src/skill2workflow/lark_task_pilot.py tests/test_lark_task_pilot.py +git commit -m "refactor: expose lark pilot workflow template" +``` + +--- + +### Task 2: Private Workspace And Paid Pilot Charter + +**Files:** +- Create: `tests/test_controlled_lark_pilot.py` +- Create: `src/skill2workflow/controlled_lark_pilot.py` + +**Interfaces:** +- Consumes: `Path`, JSON, filesystem permissions, the approved design constants. +- Produces: + - `initialize_pilot(repo_root: Path, work_dir: Path, charter: Dict[str, object], now: datetime = None) -> Dict[str, object]` + - `load_pilot_charter(work_dir: Path, now: datetime = None) -> Dict[str, object]` + - `load_private_case(repo_root: Path, input_path: Path) -> Dict[str, object]` + - constants `PILOT_SCHEMA_VERSION`, `WORKFLOW_ID`, `WORKFLOW_VERSION`, `PILOT_TIMEZONE`. + +- [x] **Step 1: Write failing charter, path, permission, and case tests** + +Create `tests/test_controlled_lark_pilot.py` with: + +```python +import json +import os +from datetime import datetime, timezone +from pathlib import Path +from tempfile import TemporaryDirectory +from unittest import TestCase + +from skill2workflow.controlled_lark_pilot import ( + initialize_pilot, + load_pilot_charter, + load_private_case, +) + + +ROOT = Path(__file__).resolve().parents[1] +NOW = datetime(2026, 7, 18, 9, 0, tzinfo=timezone.utc) + + +def _valid_charter(): + return { + "schema_version": "controlled-lark-pilot-0.1.0", + "scenario_id": "sales_renewal_risk_followup", + "workflow_id": "workflow_controlled_lark_pilot", + "workflow_version": "0.1.0", + "support_model": "assisted", + "timezone": "Asia/Shanghai", + "starts_on": "2026-07-18", + "expires_on": "2026-08-15", + "team_consent_confirmed": True, + "assignee_consent_confirmed": True, + "commercial_engagement_confirmed": True, + "required_approved_runs": 5, + "required_distinct_days": 5, + "required_distinct_cases": 2, + } + + +class ControlledLarkPilotTests(TestCase): + def test_initialize_pilot_creates_owner_only_private_workspace(self): + with TemporaryDirectory() as tmp: + work_dir = Path(tmp) / "controlled-pilot" + result = initialize_pilot(ROOT, work_dir, _valid_charter(), now=NOW) + + self.assertEqual(result["status"], "initialized") + self.assertEqual(result["workflow_id"], "workflow_controlled_lark_pilot") + self.assertEqual(work_dir.stat().st_mode & 0o077, 0) + self.assertEqual((work_dir / "private" / "charter.json").stat().st_mode & 0o077, 0) + self.assertTrue((work_dir / "state").is_dir()) + self.assertTrue((work_dir / "evidence").is_dir()) + + def test_initialize_pilot_rejects_repository_work_dir(self): + with self.assertRaisesRegex(ValueError, "outside the repository"): + initialize_pilot(ROOT, ROOT / ".pilot-private", _valid_charter(), now=NOW) + + def test_charter_requires_consent_commercial_status_thresholds_and_active_dates(self): + invalid_values = [ + ("team_consent_confirmed", False), + ("assignee_consent_confirmed", False), + ("commercial_engagement_confirmed", False), + ("required_approved_runs", 4), + ("required_distinct_days", 4), + ("required_distinct_cases", 1), + ("timezone", "UTC"), + ] + for key, value in invalid_values: + charter = _valid_charter() + charter[key] = value + with self.subTest(key=key), TemporaryDirectory() as tmp: + with self.assertRaises(ValueError): + initialize_pilot(ROOT, Path(tmp) / "pilot", charter, now=NOW) + + def test_load_pilot_charter_rejects_expired_charter(self): + with TemporaryDirectory() as tmp: + work_dir = Path(tmp) / "pilot" + initialize_pilot(ROOT, work_dir, _valid_charter(), now=NOW) + with self.assertRaisesRegex(ValueError, "expired"): + load_pilot_charter( + work_dir, + now=datetime(2026, 8, 16, 0, 0, tzinfo=timezone.utc), + ) + + def test_load_private_case_requires_external_owner_only_exact_shape(self): + payload = { + "pilot_case_id": "case-001", + "account_name": "Private Account", + "renewal_risk": "Private Risk", + "owner_open_id": "ou_private", + "due_at": "2026-08-15T09:00:00Z", + } + with TemporaryDirectory() as tmp: + path = Path(tmp) / "case.json" + path.write_text(json.dumps(payload), encoding="utf-8") + os.chmod(path, 0o600) + self.assertEqual(load_private_case(ROOT, path), payload) + + os.chmod(path, 0o644) + with self.assertRaisesRegex(ValueError, "owner-only"): + load_private_case(ROOT, path) +``` + +- [x] **Step 2: Run the new test and verify RED** + +Run: + +```bash +PYTHONPATH=src python3 -m unittest tests.test_controlled_lark_pilot -v +``` + +Expected: import failure because `skill2workflow.controlled_lark_pilot` does not exist. + +- [x] **Step 3: Implement charter validation and secure workspace helpers** + +Create `src/skill2workflow/controlled_lark_pilot.py` with these constants and core helpers: + +```python +from __future__ import annotations + +import json +import os +from datetime import date, datetime, timezone +from pathlib import Path +from typing import Dict +from zoneinfo import ZoneInfo + + +PILOT_SCHEMA_VERSION = "controlled-lark-pilot-0.1.0" +WORKFLOW_ID = "workflow_controlled_lark_pilot" +WORKFLOW_VERSION = "0.1.0" +SCENARIO_ID = "sales_renewal_risk_followup" +PILOT_TIMEZONE = "Asia/Shanghai" +REQUIRED_CASE_KEYS = { + "pilot_case_id", + "account_name", + "renewal_risk", + "owner_open_id", + "due_at", +} + + +def initialize_pilot( + repo_root: Path, + work_dir: Path, + charter: Dict[str, object], + now: datetime = None, +) -> Dict[str, object]: + repo_root = Path(repo_root).resolve() + work_dir = Path(work_dir).resolve() + _require_outside_repository(repo_root, work_dir, "pilot work directory") + normalized = _validate_charter(charter, now=now) + + _mkdir_private(work_dir) + _mkdir_private(work_dir / "private") + _mkdir_private(work_dir / "state") + _mkdir_private(work_dir / "evidence") + _write_private_json(work_dir / "private" / "charter.json", normalized) + return { + "status": "initialized", + "scenario_id": SCENARIO_ID, + "workflow_id": WORKFLOW_ID, + "workflow_version": WORKFLOW_VERSION, + "commercial_engagement_confirmed": True, + } + + +def load_pilot_charter(work_dir: Path, now: datetime = None) -> Dict[str, object]: + path = Path(work_dir).resolve() / "private" / "charter.json" + payload = json.loads(path.read_text(encoding="utf-8")) + return _validate_charter(payload, now=now) + + +def load_private_case(repo_root: Path, input_path: Path) -> Dict[str, object]: + repo_root = Path(repo_root).resolve() + input_path = Path(input_path).resolve() + _require_outside_repository(repo_root, input_path, "private case input") + _require_owner_only(input_path) + payload = json.loads(input_path.read_text(encoding="utf-8")) + if not isinstance(payload, dict) or set(payload) != REQUIRED_CASE_KEYS: + raise ValueError("private case input must contain only the approved fields") + normalized = {key: str(payload.get(key) or "").strip() for key in sorted(REQUIRED_CASE_KEYS)} + if not all(normalized.values()): + raise ValueError("private case input fields must be non-empty strings") + if any(token in normalized["pilot_case_id"].lower() for token in ("account", "customer", "@", " ")): + raise ValueError("pilot_case_id must be an opaque identifier") + return normalized + + +def _validate_charter(charter: object, now: datetime = None) -> Dict[str, object]: + if not isinstance(charter, dict): + raise ValueError("pilot charter must be a JSON object") + normalized = json.loads(json.dumps(charter, ensure_ascii=False)) + required_exact = { + "schema_version": PILOT_SCHEMA_VERSION, + "scenario_id": SCENARIO_ID, + "workflow_id": WORKFLOW_ID, + "workflow_version": WORKFLOW_VERSION, + "support_model": "assisted", + "timezone": PILOT_TIMEZONE, + "required_approved_runs": 5, + "required_distinct_days": 5, + "required_distinct_cases": 2, + } + for key, expected in required_exact.items(): + if normalized.get(key) != expected: + raise ValueError(f"pilot charter {key} must be {expected}") + for key in ( + "team_consent_confirmed", + "assignee_consent_confirmed", + "commercial_engagement_confirmed", + ): + if normalized.get(key) is not True: + raise ValueError(f"pilot charter {key} must be true") + starts_on = date.fromisoformat(str(normalized.get("starts_on", ""))) + expires_on = date.fromisoformat(str(normalized.get("expires_on", ""))) + current = (now or datetime.now(timezone.utc)).astimezone( + ZoneInfo(PILOT_TIMEZONE) + ).date() + if current < starts_on: + raise ValueError("pilot charter has not started") + if current > expires_on: + raise ValueError("pilot charter expired") + return normalized + + +def _require_outside_repository(repo_root: Path, path: Path, label: str) -> None: + if path == repo_root or repo_root in path.parents: + raise ValueError(f"{label} must be outside the repository") + + +def _mkdir_private(path: Path) -> None: + path.mkdir(parents=True, exist_ok=True, mode=0o700) + os.chmod(path, 0o700) + + +def _write_private_json(path: Path, value: object) -> None: + path.write_text(json.dumps(value, ensure_ascii=False, indent=2), encoding="utf-8") + os.chmod(path, 0o600) + + +def _require_owner_only(path: Path) -> None: + if os.name == "posix" and path.stat().st_mode & 0o077: + raise ValueError("private case input must use owner-only permissions") +``` + +- [x] **Step 4: Run charter tests and verify GREEN** + +Run: + +```bash +PYTHONPATH=src python3 -m unittest tests.test_controlled_lark_pilot -v +``` + +Expected: all charter, path, permission, and input-shape tests pass. + +- [x] **Step 5: Commit the private workspace boundary** + +```bash +git add src/skill2workflow/controlled_lark_pilot.py tests/test_controlled_lark_pilot.py +git commit -m "feat: add controlled pilot charter boundary" +``` + +--- + +### Task 3: Durable Start At The Human Gate + +**Files:** +- Modify: `tests/test_controlled_lark_pilot.py` +- Modify: `src/skill2workflow/controlled_lark_pilot.py` + +**Interfaces:** +- Consumes: `build_lark_task_pilot_workflow`, `LocalControlPlane`, `ConnectorRuntime`, `load_external_connector`, `StaticCredentialProvider`, charter and private-case helpers. +- Produces: + - `start_pilot_run(repo_root: Path, work_dir: Path, input_path: Path, now: datetime = None, transport=None) -> Dict[str, object]` + - `_pilot_control_plane(repo_root: Path, work_dir: Path, credential_provider, transport=None) -> LocalControlPlane`. + +- [x] **Step 1: Write the failing durable-start test** + +Add: + +```python +def _write_private_case(path: Path, case_id: str = "case-001") -> None: + path.write_text( + json.dumps( + { + "pilot_case_id": case_id, + "account_name": "Private Account", + "renewal_risk": "Private Risk", + "owner_open_id": "ou_private", + "due_at": "2026-08-15T09:00:00Z", + } + ), + encoding="utf-8", + ) + os.chmod(path, 0o600) + + +def _start_waiting_pilot(tmp: str, case_id: str = "case-001"): + root = Path(tmp) + work_dir = root / "pilot" + input_path = root / "case.json" + initialize_pilot(ROOT, work_dir, _valid_charter(), now=NOW) + _write_private_case(input_path, case_id=case_id) + started = start_pilot_run(ROOT, work_dir, input_path, now=NOW) + return work_dir, started + + +def test_start_pilot_run_publishes_live_workflow_and_stops_at_gate(self): + with TemporaryDirectory() as tmp: + work_dir, result = _start_waiting_pilot(tmp) + control = LocalControlPlane(work_dir / "state", storage="sqlite") + run = control.get_run(result["run_id"]) + workflow = control.get_workflow("workflow_controlled_lark_pilot", "0.1.0") + node = next(item for item in workflow["nodes"] if item["id"] == "create_lark_task") + + self.assertEqual(result["run_status"], "waiting") + self.assertEqual(result["current_node"], "review_renewal_risk") + self.assertEqual(result["input_keys"], [ + "account_name", "due_at", "owner_open_id", "pilot_case_id", "renewal_risk" + ]) + self.assertEqual(run["status"], "waiting") + self.assertEqual(node["connector"]["mode"], "live") + self.assertNotIn("Private Account", json.dumps(result)) +``` + +Add imports for `LocalControlPlane` and `start_pilot_run`. + +- [x] **Step 2: Run the focused test and verify RED** + +Run: + +```bash +PYTHONPATH=src python3 -m unittest tests.test_controlled_lark_pilot.ControlledLarkPilotTests.test_start_pilot_run_publishes_live_workflow_and_stops_at_gate -v +``` + +Expected: import failure because `start_pilot_run` does not exist. + +- [x] **Step 3: Implement fixed workflow publication and trigger** + +Add: + +```python +from .connectors import ConnectorRuntime, ExternalConnector +from .control_plane import LocalControlPlane +from .credentials import StaticCredentialProvider +from .external_connectors import load_external_connector +from .lark_task_pilot import build_lark_task_pilot_workflow + + +def start_pilot_run( + repo_root: Path, + work_dir: Path, + input_path: Path, + now: datetime = None, + transport=None, +) -> Dict[str, object]: + repo_root = Path(repo_root).resolve() + work_dir = Path(work_dir).resolve() + load_pilot_charter(work_dir, now=now) + pilot_input = load_private_case(repo_root, input_path) + control = _pilot_control_plane( + repo_root, + work_dir, + credential_provider=StaticCredentialProvider({}), + transport=transport, + ) + workflow = build_lark_task_pilot_workflow( + mode="live", + workflow_id=WORKFLOW_ID, + workflow_version=WORKFLOW_VERSION, + workflow_name="controlled-lark-task-sales-renewal-pilot", + ) + control.publish_workflow(workflow) + response = control.trigger_workflow( + { + "workflow_id": WORKFLOW_ID, + "version": WORKFLOW_VERSION, + "source": "controlled-live-pilot", + "idempotency_key": "", + "input": pilot_input, + } + ) + run = control.get_run(str(response["run_id"])) + if run.get("status") != "waiting" or run.get("current_node") != "review_renewal_risk": + raise ValueError("controlled pilot run did not stop at the expected human gate") + return { + "run_id": str(response["run_id"]), + "workflow_id": WORKFLOW_ID, + "workflow_version": WORKFLOW_VERSION, + "run_status": "waiting", + "current_node": "review_renewal_risk", + "input_keys": sorted(pilot_input), + } + + +def _pilot_control_plane( + repo_root: Path, + work_dir: Path, + credential_provider, + transport=None, +) -> LocalControlPlane: + connector = load_external_connector(repo_root / "examples" / "connectors" / "lark_task_connector.py") + if transport is not None: + original = connector + + def execute_with_transport(binding, credential_provider=None, context=None): + return original.executor( + binding, + credential_provider=credential_provider, + context=context, + transport=transport, + ) + + connector = ExternalConnector(manifest=original.manifest, executor=execute_with_transport) + runtime = ConnectorRuntime([connector]) + return LocalControlPlane( + work_dir / "state", + storage="sqlite", + credential_provider=credential_provider, + connector_runtime=runtime, + ) +``` + +Do not print or return `pilot_input`. + +- [x] **Step 4: Run start and existing pilot tests and verify GREEN** + +Run: + +```bash +PYTHONPATH=src python3 -m unittest tests.test_controlled_lark_pilot tests.test_lark_task_pilot -v +``` + +Expected: all tests pass and no transport is called during `start`. + +- [x] **Step 5: Commit durable start** + +```bash +git add src/skill2workflow/controlled_lark_pilot.py tests/test_controlled_lark_pilot.py +git commit -m "feat: start controlled lark pilot runs" +``` + +--- + +### Task 4: Explicit Approve Or Reject Decision + +**Files:** +- Modify: `tests/test_controlled_lark_pilot.py` +- Modify: `src/skill2workflow/controlled_lark_pilot.py` + +**Interfaces:** +- Consumes: waiting SQLite run, environment switch, Vault-injected token, optional fake transport. +- Produces: + - `decide_pilot_run(repo_root: Path, work_dir: Path, run_id: str, approved: bool, confirmed_live: bool = False, now: datetime = None, transport=None) -> Dict[str, object]`. + - `_validate_controlled_live_binding(workflow: Dict[str, object], run: Dict[str, object]) -> None`. + +- [x] **Step 1: Write failing approval, rejection, guard, repeat, and redaction tests** + +Add a fake transport that records calls and returns a compact successful provider response: + +```python +class _FakeResponse: + status = 200 + + def read(self): + return json.dumps( + {"code": 0, "msg": "success", "data": {"task": {"guid": "private-task-guid"}}} + ).encode("utf-8") + + def close(self): + return None + + +class _FakeTransport: + def __init__(self): + self.calls = [] + + def __call__(self, request, timeout): + self.calls.append((request, timeout)) + return _FakeResponse() +``` + +Add complete tests that start a fresh waiting run for each decision path: + +```python +def test_decide_approve_requires_all_live_guards_and_returns_redacted_summary(self): + with TemporaryDirectory() as tmp: + work_dir, started = _start_waiting_pilot(tmp) + transport = _FakeTransport() + environment = { + "SKILL2WORKFLOW_LARK_TASK_LIVE": "1", + "LARK_BOT_ACCESS_TOKEN": "private-token", + } + with patch.dict(os.environ, environment, clear=True): + result = decide_pilot_run( + ROOT, + work_dir, + started["run_id"], + approved=True, + confirmed_live=True, + now=NOW, + transport=transport, + ) + + self.assertEqual(result["run_status"], "completed") + self.assertEqual(result["gate_decision"], "approved") + self.assertEqual(result["provider_status"], "completed") + self.assertTrue(result["lark_task_id_present"]) + self.assertEqual(len(transport.calls), 1) + encoded = json.dumps(result) + for forbidden in ( + "private-token", "Private Account", "Private Risk", "ou_private", + "private-task-guid", + ): + self.assertNotIn(forbidden, encoded) + + +def test_decide_reject_needs_no_token_and_never_calls_transport(self): + with TemporaryDirectory() as tmp: + work_dir, started = _start_waiting_pilot(tmp) + transport = _FakeTransport() + with patch.dict(os.environ, {}, clear=True): + result = decide_pilot_run( + ROOT, work_dir, started["run_id"], approved=False, now=NOW, + transport=transport, + ) + self.assertEqual(result["run_status"], "failed") + self.assertEqual(result["gate_decision"], "rejected") + self.assertFalse(result["connector_invoked"]) + self.assertEqual(transport.calls, []) + + +def test_decide_approve_fails_before_resume_when_confirmation_switch_or_token_is_missing(self): + with TemporaryDirectory() as tmp: + work_dir, started = _start_waiting_pilot(tmp) + transport = _FakeTransport() + cases = [ + ({"SKILL2WORKFLOW_LARK_TASK_LIVE": "1", "LARK_BOT_ACCESS_TOKEN": "token"}, False, "confirmation"), + ({"LARK_BOT_ACCESS_TOKEN": "token"}, True, "SKILL2WORKFLOW_LARK_TASK_LIVE=1"), + ({"SKILL2WORKFLOW_LARK_TASK_LIVE": "1"}, True, "LARK_BOT_ACCESS_TOKEN"), + ] + for environment, confirmed, expected in cases: + with self.subTest(expected=expected), patch.dict(os.environ, environment, clear=True): + with self.assertRaisesRegex(ValueError, expected): + decide_pilot_run( + ROOT, + work_dir, + started["run_id"], + approved=True, + confirmed_live=confirmed, + now=NOW, + transport=transport, + ) + self.assertEqual(transport.calls, []) + + +def test_decide_rejects_second_decision_for_terminal_run_without_transport(self): + with TemporaryDirectory() as tmp: + work_dir, started = _start_waiting_pilot(tmp) + transport = _FakeTransport() + environment = { + "SKILL2WORKFLOW_LARK_TASK_LIVE": "1", + "LARK_BOT_ACCESS_TOKEN": "private-token", + } + with patch.dict(os.environ, environment, clear=True): + decide_pilot_run( + ROOT, work_dir, started["run_id"], approved=True, + confirmed_live=True, now=NOW, transport=transport, + ) + with self.assertRaisesRegex(ValueError, "not waiting"): + decide_pilot_run( + ROOT, work_dir, started["run_id"], approved=True, + confirmed_live=True, now=NOW, transport=transport, + ) + self.assertEqual(len(transport.calls), 1) +``` + +Import `patch` from `unittest.mock` and `decide_pilot_run` from the new module. + +Add a table-driven unit test for `_validate_controlled_live_binding` that mutates each fixed property in turn—workflow id/version, current gate, connector id, operation, mode, credential handle, run id, and node id—and asserts a stable `ValueError` before any credential provider or transport can be constructed. + +- [x] **Step 2: Run decision tests and verify RED** + +Run: + +```bash +PYTHONPATH=src python3 -m unittest tests.test_controlled_lark_pilot -v +``` + +Expected: failure because `decide_pilot_run` does not exist. + +- [x] **Step 3: Implement explicit decision guards and compact summaries** + +Add: + +```python +LIVE_SWITCH = "SKILL2WORKFLOW_LARK_TASK_LIVE" +TOKEN_ENVIRONMENT = "LARK_BOT_ACCESS_TOKEN" + + +def decide_pilot_run( + repo_root: Path, + work_dir: Path, + run_id: str, + approved: bool, + confirmed_live: bool = False, + now: datetime = None, + transport=None, +) -> Dict[str, object]: + repo_root = Path(repo_root).resolve() + work_dir = Path(work_dir).resolve() + load_pilot_charter(work_dir, now=now) + + preflight = _pilot_control_plane( + repo_root, + work_dir, + credential_provider=StaticCredentialProvider({}), + ) + current = preflight.get_run(str(run_id)) + workflow = preflight.get_workflow(WORKFLOW_ID, WORKFLOW_VERSION) + _validate_controlled_live_binding(workflow, current) + + token = "" + if approved: + if not confirmed_live: + raise ValueError("live approval requires explicit confirmation") + if os.environ.get(LIVE_SWITCH) != "1": + raise ValueError("SKILL2WORKFLOW_LARK_TASK_LIVE=1 is required") + token = os.environ.get(TOKEN_ENVIRONMENT, "") + if not token: + raise ValueError("LARK_BOT_ACCESS_TOKEN is required") + + credentials = {"lark_bot_access_token": token} if approved else {} + control = _pilot_control_plane( + repo_root, + work_dir, + credential_provider=StaticCredentialProvider(credentials), + transport=transport, + ) + state = control.resume_published_run(str(run_id), approved=approved) + events = control.list_audit_events(run_id=str(run_id)) + connector_events = [ + event for event in events + if event.get("type") in ("connector_started", "connector_completed", "connector_failed") + and event.get("node_id") == "create_lark_task" + ] + connector_metadata = {} + for event in reversed(connector_events): + metadata = event.get("connector_metadata") + if isinstance(metadata, dict): + connector_metadata = metadata + break + return { + "run_id": str(run_id), + "workflow_id": WORKFLOW_ID, + "workflow_version": WORKFLOW_VERSION, + "run_status": str(state.get("status", "")), + "gate_decision": "approved" if approved else "rejected", + "connector_invoked": bool(connector_events), + "connector_status": str(connector_events[-1].get("connector_status", "")) if connector_events else "", + "credential_status": str(connector_events[-1].get("credential_status", "")) if connector_events else "", + "provider_status": str(connector_metadata.get("provider_status", "")), + "idempotency_key_present": bool(connector_metadata.get("idempotency_key_present")), + "lark_task_id_present": bool(connector_metadata.get("lark_task_id_present")), + } +``` + +`_validate_controlled_live_binding` checks exact workflow id/version, run id presence, `status == "waiting"`, `current_node == "review_renewal_risk"`, and the `create_lark_task` node's id, connector id `lark_task`, operation `create_task`, mode `live`, and sole credential handle `lark_bot_access_token`. It also requires all execution-identity fields used by provider idempotency to be non-empty. It raises only fixed redacted messages. + +Do not include `token`, context, task values, provider payloads, or provider messages in raised errors or summaries. + +- [x] **Step 4: Run controlled and connector tests and verify GREEN** + +Run: + +```bash +PYTHONPATH=src python3 -m unittest tests.test_controlled_lark_pilot tests.test_lark_task_connector tests.test_executor tests.test_control_plane -v +``` + +Expected: all tests pass; fake approval makes exactly one transport call; rejection makes none. + +- [x] **Step 5: Commit explicit decision control** + +```bash +git add src/skill2workflow/controlled_lark_pilot.py tests/test_controlled_lark_pilot.py +git commit -m "feat: add controlled lark pilot decisions" +``` + +--- + +### Task 5: Strict Redacted Evidence Pack + +**Files:** +- Create: `tests/test_controlled_lark_pilot_evidence.py` +- Create: `src/skill2workflow/controlled_lark_pilot_evidence.py` +- Modify: `src/skill2workflow/controlled_lark_pilot.py` + +**Interfaces:** +- Consumes: full private run states, control-plane audit events, validated charter, exercise/verification/decision state. +- Produces: + - `build_run_evidence(run: Dict[str, object], audit_events: List[Dict[str, object]]) -> Dict[str, object]` + - `build_acceptance_summary(charter: Dict[str, object], runs: List[Dict[str, object]], distinct_private_cases: int, exercises: Dict[str, object], verification: Dict[str, object], decision: Dict[str, object]) -> Dict[str, object]` + - `validate_evidence_pack(pack: Dict[str, object], forbidden_values: List[str]) -> None` + - `write_evidence_pack(output_dir: Path, pack: Dict[str, object]) -> Dict[str, object]` + - `generate_pilot_evidence(repo_root: Path, work_dir: Path, output_dir: Path = None, now: datetime = None) -> Dict[str, object]`. + +- [x] **Step 1: Write failing run-evidence allowlist and leakage tests** + +Create `tests/test_controlled_lark_pilot_evidence.py` with synthetic full state that includes forbidden raw values and audit that includes only compact connector metadata: + +```python +import json +from pathlib import Path +from tempfile import TemporaryDirectory +from unittest import TestCase + +from skill2workflow.controlled_lark_pilot_evidence import ( + build_acceptance_summary, + build_run_evidence, + validate_evidence_pack, + write_evidence_pack, +) + + +class ControlledLarkPilotEvidenceTests(TestCase): + def test_build_run_evidence_uses_exact_allowlist(self): + run = { + "run_id": "run_001", + "workflow_id": "workflow_controlled_lark_pilot", + "workflow_version": "0.1.0", + "status": "completed", + "context": { + "input": { + "pilot_case_id": "case-001", + "account_name": "Private Account", + "renewal_risk": "Private Risk", + "owner_open_id": "ou_private", + "due_at": "2026-08-15T09:00:00Z", + } + }, + "node_results": {"create_lark_task": {"output": {}}}, + } + audit = [ + { + "type": "run_started", + "run_id": "run_001", + "timestamp": "2026-07-18T01:00:00+00:00", + }, + { + "type": "run_resumed", + "run_id": "run_001", + "approved": True, + "timestamp": "2026-07-18T01:01:00+00:00", + }, + { + "type": "connector_completed", + "run_id": "run_001", + "node_id": "create_lark_task", + "connector_id": "lark_task", + "connector_status": "completed", + "credential_status": "resolved", + "credential_handles": ["lark_bot_access_token"], + "connector_metadata": { + "operation": "create_task", + "mode": "live", + "provider_status": "completed", + "task_title_present": True, + "task_description_present": True, + "assignee_present": True, + "due_at_present": True, + "idempotency_key_present": True, + "lark_task_id_present": True, + }, + "timestamp": "2026-07-18T01:01:01+00:00", + }, + { + "type": "run_completed", + "run_id": "run_001", + "timestamp": "2026-07-18T01:01:02+00:00", + }, + ] + + evidence = build_run_evidence(run, audit) + + self.assertEqual(evidence["run_id"], "run_001") + self.assertEqual(evidence["gate_decision"], "approved") + self.assertEqual(evidence["provider_status"], "completed") + self.assertTrue(evidence["case_id_present"]) + encoded = json.dumps(evidence) + for forbidden in ( + "case-001", "Private Account", "Private Risk", "ou_private", + "2026-08-15T09:00:00Z" + ): + self.assertNotIn(forbidden, encoded) +``` + +Add rejection coverage that expects `gate_decision: rejected`, `connector_invoked: false`, and empty connector status fields. + +- [x] **Step 2: Write failing five-day aggregation and atomic-write tests** + +Use five synthetic redacted records with `completed_at` values on July 18–22 in `Asia/Shanghai`; supply only the already-counted private-case cardinality, and assert: + +```python +summary = build_acceptance_summary( + charter=_valid_charter(), + runs=approved_runs + [rejected_run], + distinct_private_cases=2, + exercises={"failure": {"passed": True}, "rollback": {"passed": True}}, + verification={"all_passed": True}, + decision={ + "decision": "continue", + "partner_acknowledged": True, + "operator_acknowledged": True, + "commercial_engagement_confirmed": True, + "rationale": "The controlled workflow delivered the agreed result.", + }, +) +self.assertTrue(summary["ready_to_finalize"]) +self.assertEqual(summary["approved_live_runs"], 5) +self.assertEqual(summary["distinct_calendar_days"], 5) +self.assertEqual(summary["distinct_private_cases"], 2) +self.assertEqual(summary["rejected_runs"], 1) +self.assertEqual(summary["unmet_conditions"], []) +``` + +For atomic writing: + +```python +with TemporaryDirectory() as tmp: + output = Path(tmp) / "evidence" + first = write_evidence_pack(output, pack) + second = write_evidence_pack(output, pack) + self.assertEqual(first["file_count"], second["file_count"]) + self.assertFalse(any(path.name.endswith(".tmp") for path in output.rglob("*"))) +``` + +- [x] **Step 3: Run evidence tests and verify RED** + +Run: + +```bash +PYTHONPATH=src python3 -m unittest tests.test_controlled_lark_pilot_evidence -v +``` + +Expected: import failure because the evidence module does not exist. + +- [x] **Step 4: Implement pure run evidence and acceptance aggregation** + +Create the new module with exact allowlisted keys: + +```python +RUN_EVIDENCE_KEYS = { + "schema_version", + "run_id", + "workflow_id", + "workflow_version", + "started_at", + "completed_at", + "run_status", + "gate_decision", + "case_id_present", + "connector_invoked", + "connector_id", + "connector_status", + "credential_status", + "credential_handles", + "operation", + "mode", + "provider_status", + "task_title_present", + "task_description_present", + "assignee_present", + "due_at_present", + "idempotency_key_present", + "lark_task_id_present", +} + + +def build_run_evidence(run, audit_events): + resumed = _last_event(audit_events, "run_resumed") + connector = _last_connector_event(audit_events) + metadata = connector.get("connector_metadata", {}) if connector else {} + if not isinstance(metadata, dict): + metadata = {} + trigger_input = run.get("context", {}).get("input", {}) + if not isinstance(trigger_input, dict): + trigger_input = {} + evidence = { + "schema_version": "controlled-lark-pilot-evidence-0.1.0", + "run_id": str(run.get("run_id", "")), + "workflow_id": str(run.get("workflow_id", "")), + "workflow_version": str(run.get("workflow_version", "")), + "started_at": str(_first_event(audit_events, "run_started").get("timestamp", "")), + "completed_at": str(_terminal_event(audit_events).get("timestamp", "")), + "run_status": str(run.get("status", "")), + "gate_decision": ( + "approved" if resumed.get("approved") is True + else "rejected" if resumed.get("approved") is False + else "pending" + ), + "case_id_present": bool(str(trigger_input.get("pilot_case_id", "")).strip()), + "connector_invoked": bool(connector), + "connector_id": str(connector.get("connector_id", "")) if connector else "", + "connector_status": str(connector.get("connector_status", "")) if connector else "", + "credential_status": str(connector.get("credential_status", "")) if connector else "", + "credential_handles": list(connector.get("credential_handles", [])) if connector else [], + "operation": str(metadata.get("operation", "")), + "mode": str(metadata.get("mode", "")), + "provider_status": str(metadata.get("provider_status", "")), + "task_title_present": bool(metadata.get("task_title_present")), + "task_description_present": bool(metadata.get("task_description_present")), + "assignee_present": bool(metadata.get("assignee_present")), + "due_at_present": bool(metadata.get("due_at_present")), + "idempotency_key_present": bool(metadata.get("idempotency_key_present")), + "lark_task_id_present": bool(metadata.get("lark_task_id_present")), + } + if set(evidence) != RUN_EVIDENCE_KEYS: + raise ValueError("run evidence keys do not match the allowlist") + return evidence +``` + +Implement `_first_event`, `_last_event`, `_last_connector_event`, and `_terminal_event` as small deterministic scans over dictionaries only. `build_acceptance_summary` must: + +- count an approved live run only when workflow id/version, `gate_decision`, terminal run status, case-id presence, connector invocation/id/status, credential status/sole handle, operation, mode, provider status, all mapped-input presence flags, idempotency presence, and task-id presence exactly match the controlled success contract; +- convert `completed_at` into `Asia/Shanghai` dates with `zoneinfo.ZoneInfo`; +- accept only an integer distinct-case count derived by the private orchestrator; never accept or retain raw case ids in the evidence module; +- require at least one terminal rejected/failed run with `gate_decision == "rejected"` and `connector_invoked == false`; +- require both exercises, complete verification, valid decision, both acknowledgements, and commercial confirmation; +- return every failed predicate as a stable string in `unmet_conditions`. + +- [x] **Step 5: Implement evidence validation and atomic per-file replacement** + +Use a fixed output map and `os.replace`: + +```python +def _write_json_atomic(path: Path, value: object) -> None: + path.parent.mkdir(parents=True, exist_ok=True) + temporary = path.with_name(f".{path.name}.tmp") + temporary.write_text(json.dumps(value, ensure_ascii=False, indent=2), encoding="utf-8") + os.replace(str(temporary), str(path)) + + +def validate_evidence_pack(pack, forbidden_values): + encoded = json.dumps(pack, ensure_ascii=False, sort_keys=True) + leaf_strings = _all_string_leaves(pack) + for value in forbidden_values: + if not isinstance(value, str) or not value: + continue + if (len(value) >= 4 and value in encoded) or value in leaf_strings: + raise ValueError("evidence pack contains a forbidden private value") + for run in pack.get("runs", []): + if set(run) != RUN_EVIDENCE_KEYS: + raise ValueError("run evidence keys do not match the allowlist") + + +def write_evidence_pack(output_dir, pack): + output_dir = Path(output_dir) + files = { + output_dir / "pilot-charter.json": pack["charter"], + output_dir / "evidence-index.json": pack["index"], + } + for sequence, run in enumerate(pack["runs"], start=1): + files[output_dir / "runs" / f"{sequence:03d}.json"] = run + for name, exercise in sorted(pack.get("exercises", {}).items()): + files[output_dir / "exercises" / f"{name}.json"] = exercise + if pack.get("verification"): + files[output_dir / "verification.json"] = pack["verification"] + if pack.get("decision"): + files[output_dir / "decision.json"] = pack["decision"] + for path, value in files.items(): + _write_json_atomic(path, value) + _remove_stale_json_files(output_dir, set(files)) + return {"status": "written", "file_count": len(files), "output_dir": str(output_dir)} +``` + +Define `_all_string_leaves` as a recursive read-only traversal of dictionaries and lists. Define exact key allowlists for the charter, evidence index, each exercise kind, verification command/result, decision, and top-level pack as well as for runs. `validate_evidence_pack` must reject an unknown or missing key, a wrong primitive/container type, a non-allowlisted credential handle, a non-fixed workflow/connector/operation/mode identity, or a forbidden value anywhere in the encoded pack. Add one negative test per artifact category that inserts an extra raw-looking field and expects fail-closed validation. + +`_remove_stale_json_files` may delete only `.json` descendants of the exact `output_dir` passed to it that are absent from the new fixed map. Before scanning, it must reject a filesystem root, an existing symlink, or any path containing a symlink component. This writer never decides whether a repository export path is authorized. + +- [x] **Step 6: Implement orchestration from private SQLite state** + +In `controlled_lark_pilot.py`, add the exact public signature and one private in-memory builder shared with finalization: + +```python +def generate_pilot_evidence( + repo_root: Path, + work_dir: Path, + output_dir: Path = None, + now: datetime = None, +) -> Dict[str, object]: +``` + +```python +def _build_pilot_evidence( + repo_root: Path, + work_dir: Path, + decision_override: Dict[str, object] = None, + now: datetime = None, +) -> Dict[str, object]: +``` + +Implement it so that it: + +1. loads the valid charter; +2. constructs the SQLite control plane with an empty credential provider; +3. lists runs, loads each full run, filters only the controlled workflow id/version, and sorts them deterministically by authoritative start timestamp then run id; +4. builds redacted run evidence and gathers private case ids in memory only for runs that qualify as approved live completions; +5. discards the raw case-id set immediately after calculating its cardinality; +6. derives `exercises/rejection.json` from the first qualifying human-rejected run in that stable order, with only exercise name, passed boolean, run id, gate decision, and connector-invoked boolean; +7. loads optional private failure, rollback, verification, and decision JSON; +8. gathers every string value from private run `context.input` plus any currently injected token into `forbidden_values`; +9. passes only the distinct-case integer to `build_acceptance_summary` and records an explicit `generated_at` derived from `now` in the evidence index; +10. builds and validates the pack; +11. defaults output exactly to `work_dir/evidence`; +12. permits an explicit repository-contained output only after `private/finalization.json` records a successful finalization and only when the resolved path equals `repo_root/docs/pilot-evidence/loop-40`; rejects every other repository-contained output; +13. writes the validated pack and returns only status, counts, unmet conditions, and the output directory. + +- [x] **Step 7: Run evidence and controlled tests and verify GREEN** + +Run: + +```bash +PYTHONPATH=src python3 -m unittest tests.test_controlled_lark_pilot_evidence tests.test_controlled_lark_pilot -v +``` + +Expected: all tests pass; encoded evidence contains no synthetic private values. + +- [x] **Step 8: Commit the evidence boundary** + +```bash +git add src/skill2workflow/controlled_lark_pilot.py src/skill2workflow/controlled_lark_pilot_evidence.py tests/test_controlled_lark_pilot.py tests/test_controlled_lark_pilot_evidence.py +git commit -m "feat: add redacted controlled pilot evidence" +``` + +--- + +### Task 6: Safe Exercises, Fixed Verification, And Finalization + +**Files:** +- Modify: `tests/test_controlled_lark_pilot.py` +- Modify: `tests/test_controlled_lark_pilot_evidence.py` +- Modify: `src/skill2workflow/controlled_lark_pilot.py` +- Modify: `src/skill2workflow/controlled_lark_pilot_evidence.py` + +**Interfaces:** +- Produces: + - `exercise_disabled_live(repo_root: Path, work_dir: Path, now: datetime = None) -> Dict[str, object]` + - `exercise_rollback(repo_root: Path, work_dir: Path, now: datetime = None) -> Dict[str, object]` + - `verify_pilot(repo_root: Path, work_dir: Path, command_runner=None) -> Dict[str, object]` + - `finalize_pilot(repo_root: Path, work_dir: Path, decision: Dict[str, object], output_dir: Path = None, now: datetime = None) -> Dict[str, object]`. + +- [x] **Step 1: Write failing disabled-live and rollback exercise tests** + +Use a credential spy whose `resolve()` records calls and a transport spy. Assert that `exercise_disabled_live` returns exactly: + +```python +{ + "exercise": "disabled_live", + "passed": True, + "provider_status": "live_disabled", + "credential_resolution_attempted": False, + "transport_attempted": False, +} +``` + +Assert the private exercise file exists at `private/exercises/failure.json` and contains no synthetic input values. + +For rollback, clear both live environment variables, run the helper, and assert: + +```python +self.assertEqual(result["exercise"], "rollback") +self.assertTrue(result["passed"]) +self.assertEqual(result["live_switch_enabled"], False) +self.assertEqual(result["live_approval_blocked"], True) +self.assertEqual(result["dry_run_status"], "completed") +``` + +- [x] **Step 2: Write failing fixed-verification tests** + +Inject a fake runner that records argument arrays. Assert: + +```python +result = verify_pilot(ROOT, work_dir, command_runner=fake_runner) +self.assertTrue(result["all_passed"]) +self.assertEqual( + [item["id"] for item in result["commands"]], + [ + "focused-tests", + "full-tests", + "compile", + "secret-hygiene", + "connector-smoke", + "dry-run-pilot-smoke", + "diff-check", + ], +) +self.assertNotIn("LARK_BOT_ACCESS_TOKEN", fake_runner.environments[0]) +self.assertNotIn("SKILL2WORKFLOW_LARK_TASK_LIVE", fake_runner.environments[0]) +``` + +The verification result may contain command ids, exit codes, and durations only; it must not persist stdout/stderr. + +- [x] **Step 3: Write failing finalization tests** + +Build an incomplete pack and assert `finalize_pilot` raises with the stable unmet condition list. Build a complete synthetic pack and decision: + +```python +decision = { + "schema_version": "controlled-lark-pilot-decision-0.1.0", + "decision": "continue", + "partner_acknowledged": True, + "operator_acknowledged": True, + "commercial_engagement_confirmed": True, + "rationale": "The controlled workflow delivered the agreed business result.", +} +``` + +Use a fully temporary synthetic repository root and work directory for export tests; never write test evidence into the checkout. Assert invalid decision values, false acknowledgements, false commercial status, empty rationale, and rationale containing any private case value fail before export. For every failure, assert `private/finalization.json` and the requested repository export are absent. For a complete case, assert both the private evidence pack and requested exact repository export are byte-for-byte equivalent JSON maps, the owner-only finalization marker exists, and a subsequent `generate_pilot_evidence` may replace only that exact repository target. Also assert any other repository-contained output path fails closed. + +- [x] **Step 4: Run exercise/finalization tests and verify RED** + +Run: + +```bash +PYTHONPATH=src python3 -m unittest tests.test_controlled_lark_pilot tests.test_controlled_lark_pilot_evidence -v +``` + +Expected: failures because exercise, verification, and finalization functions do not exist. + +- [x] **Step 5: Implement disabled-live exercise with spies** + +Build a synthetic private case inside the private work directory, start the normal controlled workflow, and ensure the live switch is absent for the exercise. Construct the control plane with a connector wrapper that records transport access and a credential provider that records `resolve()` calls, then call `LocalControlPlane.resume_published_run` directly; do not call `decide_pilot_run`, whose operator guard intentionally rejects the missing switch before the connector boundary is exercised. Persist only the exact compact result above. The exercise passes only when: + +```python +passed = ( + provider_status == "live_disabled" + and credential_resolution_attempted is False + and transport_attempted is False +) +``` + +The exercise must restore the caller's environment after execution. + +- [x] **Step 6: Implement rollback through a blocked live approval and the unchanged dry-run helper** + +With both live environment variables removed, start a fresh synthetic private controlled case and run this exact guard probe with a transport spy: + +```python +live_approval_blocked = False +try: + decide_pilot_run( + repo_root, + work_dir, + started["run_id"], + approved=True, + confirmed_live=True, + now=now, + transport=transport_spy, + ) +except ValueError as error: + live_approval_blocked = str(error) == "SKILL2WORKFLOW_LARK_TASK_LIVE=1 is required" +if not live_approval_blocked or transport_spy.calls: + raise ValueError("rollback did not prove the disabled live boundary") +``` + +Leave that run waiting as historical proof. Then call: + +```python +from .lark_task_pilot import run_lark_task_pilot + +dry_run = run_lark_task_pilot( + repo_root=repo_root, + work_dir=work_dir / "private" / "rollback-dry-run", + reset=True, +) +result = { + "exercise": "rollback", + "passed": live_approval_blocked and dry_run.get("run_status") == "completed", + "live_switch_enabled": os.environ.get(LIVE_SWITCH) == "1", + "live_approval_blocked": live_approval_blocked, + "dry_run_status": str(dry_run.get("run_status", "")), +} +``` + +The public rollback command must fail before this helper if the live switch is still exactly `1`; the operator removes it, then runs the exercise. + +- [x] **Step 7: Implement the fixed offline verification command set** + +Use exact argument arrays, `cwd=repo_root`, a sanitized environment with both live variables removed, and `capture_output=True`. The fixed commands are: + +```python +commands = [ + ("focused-tests", [python, "-m", "unittest", "tests.test_controlled_lark_pilot", "tests.test_controlled_lark_pilot_evidence", "tests.test_controlled_lark_pilot_docs", "-v"]), + ("full-tests", [python, "-m", "unittest", "discover", "-s", "tests", "-v"]), + ("compile", [python, "-m", "py_compile", *sorted_source_files, connector_file]), + ("secret-hygiene", [python, "scripts/secret_hygiene.py", "examples/workflows"]), + ("connector-smoke", [python, "scripts/lark_task_connector_smoke.py", "--work-dir", str(work_dir / "private" / "connector-smoke")]), + ("dry-run-pilot-smoke", [python, "scripts/lark_task_pilot_smoke.py", "--work-dir", str(work_dir / "private" / "dry-run-smoke")]), + ("diff-check", ["git", "diff", "--check"]), +] +``` + +Set `PYTHONPATH=src` in the sanitized environment for Python commands. Persist one compact record per command with `id`, `exit_code`, and `passed`. Set `all_passed` only when every exit code is zero. + +- [x] **Step 8: Implement fail-closed finalization and safe export** + +Validate the decision exact schema and allowlisted keys. Pass it as `decision_override` to `_build_pilot_evidence` without first persisting it, require `ready_to_finalize: true`, and validate the complete candidate pack against all private values. Always atomically replace the private derived pack in `work_dir/evidence`; when `output_dir` is provided, also require that any repository-contained destination resolves exactly to `repo_root/docs/pilot-evidence/loop-40` before writing the same validated pack there. Only after every requested write succeeds, atomically persist `private/decision.json` and an owner-only `private/finalization.json` marker with exact keys `schema_version`, `finalized`, `decision`, and `finalized_at`. Ordinary `generate_pilot_evidence` may subsequently regenerate the repository export only when that marker is valid. A failed candidate or write leaves no finalization marker and never advances the Roadmap. + +The return value is: + +```python +{ + "status": "finalized", + "decision": decision["decision"], + "approved_live_runs": index["approved_live_runs"], + "distinct_calendar_days": index["distinct_calendar_days"], + "distinct_private_cases": index["distinct_private_cases"], + "rejected_runs": index["rejected_runs"], + "output_dir": str(output_dir or (work_dir / "evidence")), +} +``` + +- [x] **Step 9: Run focused tests and verify GREEN** + +Run: + +```bash +PYTHONPATH=src python3 -m unittest tests.test_controlled_lark_pilot tests.test_controlled_lark_pilot_evidence -v +``` + +Expected: all exercise, verification, finalization, and leakage tests pass. + +- [x] **Step 10: Commit exercises and finalization** + +```bash +git add src/skill2workflow/controlled_lark_pilot.py src/skill2workflow/controlled_lark_pilot_evidence.py tests/test_controlled_lark_pilot.py tests/test_controlled_lark_pilot_evidence.py +git commit -m "feat: finalize controlled lark pilot evidence" +``` + +--- + +### Task 7: Operator CLI And Controlled Pilot Runbook + +**Files:** +- Create: `scripts/controlled_lark_pilot.py` +- Create: `docs/controlled-live-pilot.md` +- Create: `tests/test_controlled_lark_pilot_docs.py` +- Modify: `docs/connectors.md` +- Modify: `docs/examples.md` +- Modify: `README.md` +- Modify: `AGENTS.md` + +**Interfaces:** +- Consumes: all Task 2–6 public functions. +- Produces: `main(argv=None) -> int` with `init`, `start`, `decide`, `evidence`, `exercise-failure`, `exercise-rollback`, `verify`, and `finalize` subcommands; one complete operator runbook. + +- [x] **Step 1: Write failing CLI and documentation contract tests** + +Create tests that assert: + +```python +def test_controlled_pilot_runbook_documents_every_safe_phase(self): + runbook = (ROOT / "docs" / "controlled-live-pilot.md").read_text(encoding="utf-8") + for command in ( + " init ", + " start ", + " decide ", + " evidence ", + " exercise-failure ", + " exercise-rollback ", + " verify ", + " finalize ", + ): + self.assertIn(command, runbook) + self.assertIn("vibe vault run --env LARK_BOT_ACCESS_TOKEN", runbook) + self.assertIn("chmod 600", runbook) + self.assertIn("five distinct calendar days", runbook) + self.assertIn("must not advance Loop 40", runbook) + + +def test_docs_preserve_dry_run_and_narrow_live_boundaries(self): + connectors = (ROOT / "docs" / "connectors.md").read_text(encoding="utf-8") + examples = (ROOT / "docs" / "examples.md").read_text(encoding="utf-8") + readme = (ROOT / "README.md").read_text(encoding="utf-8") + self.assertIn("docs/controlled-live-pilot.md", connectors) + self.assertIn("dry-run remains the default", connectors) + self.assertIn("controlled real-team pilot", examples) + self.assertIn("Loop 40", readme) + self.assertIn("Current maturity: Local Evaluation", readme) +``` + +Add CLI parser tests that call the complete safe argument lists below, capture stdout, and assert it is compact JSON without raw values: + +```python +init_args = [ + "init", + "--work-dir", str(work_dir), + "--starts-on", "2026-07-18", + "--expires-on", "2026-08-15", + "--confirm-team-consent", + "--confirm-assignee-consent", + "--confirm-commercial-engagement", +] +reject_args = [ + "decide", + "--work-dir", str(work_dir), + "--run-id", started["run_id"], + "--reject", +] +self.assertEqual(main(init_args), 0) +self.assertEqual(main(reject_args), 0) +``` + +- [x] **Step 2: Run CLI/docs tests and verify RED** + +Run: + +```bash +PYTHONPATH=src python3 -m unittest tests.test_controlled_lark_pilot_docs -v +``` + +Expected: failure because the script and runbook do not exist. + +- [x] **Step 3: Implement the thin CLI wrapper** + +The wrapper adds `src` to `sys.path` exactly like existing scripts and imports `main` from `skill2workflow.controlled_lark_pilot`. In the module, build subparsers with these safe inputs: + +```text +init --work-dir PATH --starts-on YYYY-MM-DD --expires-on YYYY-MM-DD + --confirm-team-consent --confirm-assignee-consent --confirm-commercial-engagement +start --work-dir PATH --input PATH +decide --work-dir PATH --run-id RUN_ID (--approve | --reject) [--confirm-live-create] +evidence --work-dir PATH [--output-dir PATH] +exercise-failure --work-dir PATH +exercise-rollback --work-dir PATH +verify --work-dir PATH +finalize --work-dir PATH --decision-file PATH [--output-dir PATH] +``` + +`init` constructs the exact fixed charter in code; it accepts no customer or commercial details. `decide --approve` is the only phase that reads the injected token. `--reject` must reject `--confirm-live-create` as unnecessary. `finalize` reads a redacted decision JSON file; it does not accept the rationale on the command line. + +Every successful command prints one compact JSON summary. Every expected operator error exits nonzero with fixed text that does not contain private input. + +- [x] **Step 4: Write the full operator runbook** + +Document: + +- the paid assisted engagement and consent prerequisite; +- private directory creation outside the repository; +- exact charter initialization command; +- the approved private case JSON schema; +- `chmod 600` before `start`; +- inspection of the waiting run before a decision; +- Vault injection for approval only; +- rejection without Vault; +- evidence regeneration after every run; +- five approved runs across five `Asia/Shanghai` dates and two opaque case ids; +- disabled-live failure and rollback exercises; +- sanitized fixed verification; +- redacted decision-file schema; +- finalization and export to `docs/pilot-evidence/loop-40` only after every gate passes; +- token rotation/deletion after the pilot; +- incident stop conditions and `defer` behavior; +- the explicit statement that implementation readiness must not advance Loop 40. + +- [x] **Step 5: Link the runbook without advancing maturity** + +Add the script command to `AGENTS.md`, a narrow link in `docs/connectors.md`, a dry-run/live distinction in `docs/examples.md`, and one runbook link in `README.md`. Keep these exact status statements unchanged: + +```text +Current maturity: Local Evaluation +Completed delivery loops: 1-39 +Active loop: Loop 40, Controlled Live Connector Pilot +``` + +- [x] **Step 6: Run CLI/docs and Roadmap tests and verify GREEN** + +Run: + +```bash +PYTHONPATH=src python3 -m unittest tests.test_controlled_lark_pilot_docs tests.test_production_roadmap tests.test_product_connector_pilot_roadmap tests.test_first_product_connector_candidate_docs -v +``` + +Expected: all pass; Loop 40 remains active and incomplete. + +- [x] **Step 7: Commit CLI and runbook** + +```bash +git add AGENTS.md README.md docs/connectors.md docs/examples.md docs/controlled-live-pilot.md scripts/controlled_lark_pilot.py src/skill2workflow/controlled_lark_pilot.py tests/test_controlled_lark_pilot_docs.py +git commit -m "docs: add controlled lark pilot runbook" +``` + +--- + +### Task 8: Offline Implementation Verification And Review Gate + +**Files:** +- Modify: `docs/superpowers/plans/2026-07-17-controlled-live-pilot.md` + +**Interfaces:** +- Consumes: Tasks 1–7. +- Produces: implementation-ready controlled Pilot tooling with no claim of real-pilot completion. + +- [x] **Step 1: Run all focused controlled-pilot tests** + +```bash +PYTHONPATH=src python3 -m unittest tests.test_controlled_lark_pilot tests.test_controlled_lark_pilot_evidence tests.test_controlled_lark_pilot_docs tests.test_lark_task_pilot tests.test_lark_task_connector tests.test_lark_task_live_validation -v +``` + +Expected: all pass. + +- [x] **Step 2: Run the full test suite in an environment that permits local loopback ports** + +```bash +PYTHONPATH=src python3 -m unittest discover -s tests -v +``` + +Expected: all tests pass. If the sandbox blocks `127.0.0.1` binding, rerun the identical command with the required sandbox escalation; do not reinterpret permission errors as code failures. + +- [x] **Step 3: Run complete offline safety verification** + +```bash +python3 -m py_compile src/skill2workflow/*.py examples/connectors/lark_task_connector.py +python3 scripts/secret_hygiene.py examples/workflows +python3 scripts/lark_task_connector_smoke.py --work-dir /tmp/skill2workflow-lark-task-connector +python3 scripts/lark_task_pilot_smoke.py --work-dir /tmp/skill2workflow-lark-task-pilot +git diff --check +``` + +Expected: every command exits 0; no live provider call occurs. + +- [x] **Step 4: Run final code review** + +Use `superpowers:requesting-code-review`. Review against the approved design, with special attention to: + +- approval never auto-runs; +- token boundary and error redaction; +- private directory rejection and permissions; +- evidence exact allowlist; +- repeated terminal decision suppression; +- exercise truthfulness; +- incomplete evidence failing finalization; +- Roadmap remaining at Local Evaluation. + +Expected: no unresolved critical, important, or minor finding. + +- [x] **Step 5: Record implementation readiness without advancing Loop 40** + +Check Tasks 1–8 in this plan and add a dated verification note containing only command names, pass/fail status, and test count. State explicitly: + +```text +Controlled Pilot tooling is implementation-ready. Loop 40 remains incomplete until the five-day paid real-team evidence gate finalizes. +``` + +- [x] **Step 6: Commit the implementation verification record** + +```bash +git add docs/superpowers/plans/2026-07-17-controlled-live-pilot.md +git commit -m "docs: verify controlled pilot tooling" +``` + +#### Verification note — 2026-07-20 + +| Command | Status | Test count | +|---|---|---:| +| `expanded-focused-tests` | PASS | 174/174 | +| `full-tests` | PASS | 319/319 | +| `compile` | PASS | 33 files | +| `secret-hygiene` | PASS | 12 fixtures, 0 findings | +| `connector-smoke` | PASS | 1 | +| `dry-run-pilot-smoke` | PASS | 1 | +| `diff-check` | PASS | — | +| `controlled-pilot-verify` | PASS | 7/7 | + +Controlled Pilot tooling is implementation-ready. Loop 40 remains incomplete until the five-day paid real-team evidence gate finalizes. + +--- + +### Task 9: Initialize The Paid Pilot And Complete Day 1 Controls + +**Files:** +- Private only: `$HOME/.local/share/skill2workflow/pilots/loop-40/` +- No repository evidence is committed in this task. + +**Interfaces:** +- Consumes: one consenting paid/contracted partner, Vault credential, assignee consent, real private case file. +- Produces: valid charter, dry-run rehearsal, one approved live run, one rejected run, failure exercise, rollback exercise, and private redacted evidence. + +- [ ] **Step 1: Initialize the private charter** + +Run with the approved engagement window of 2026-07-18 through 2026-08-15: + +```bash +python3 scripts/controlled_lark_pilot.py init \ + --work-dir "$HOME/.local/share/skill2workflow/pilots/loop-40" \ + --starts-on 2026-07-18 \ + --expires-on 2026-08-15 \ + --confirm-team-consent \ + --confirm-assignee-consent \ + --confirm-commercial-engagement +``` + +Expected: `status: initialized`, three confirmation booleans true, and no partner identity or commercial terms in stdout or the charter. + +- [ ] **Step 2: Run the unchanged dry-run rehearsal** + +```bash +python3 scripts/lark_task_pilot_smoke.py \ + --work-dir "$HOME/.local/share/skill2workflow/pilots/loop-40/private/rehearsal" +``` + +Expected: `run_status: completed`, `mode: dry_run`. + +- [ ] **Step 3: Prepare the first private real case file** + +Create `$HOME/.local/share/skill2workflow/pilots/loop-40/private/cases/day-1.json` with real partner-approved values that validate against this exact schema: + +```json +{ + "type": "object", + "additionalProperties": false, + "required": ["pilot_case_id", "account_name", "renewal_risk", "owner_open_id", "due_at"], + "properties": { + "pilot_case_id": {"type": "string", "const": "case-001"}, + "account_name": {"type": "string", "minLength": 1}, + "renewal_risk": {"type": "string", "minLength": 1}, + "owner_open_id": {"type": "string", "minLength": 1}, + "due_at": {"type": "string", "format": "date-time"} + } +} +``` + +Enter the four private business values only in that owner-controlled file, never in the repository or terminal command. Then run: + +```bash +chmod 600 "$HOME/.local/share/skill2workflow/pilots/loop-40/private/cases/day-1.json" +python3 scripts/controlled_lark_pilot.py start \ + --work-dir "$HOME/.local/share/skill2workflow/pilots/loop-40" \ + --input "$HOME/.local/share/skill2workflow/pilots/loop-40/private/cases/day-1.json" +``` + +Expected: a compact waiting summary. Record the returned run id privately as `DAY1_RUN_ID`. + +- [ ] **Step 4: Inspect and explicitly approve the first waiting run** + +Only after the designated operator confirms the waiting run: + +```bash +vibe vault run --env LARK_BOT_ACCESS_TOKEN -- \ + env SKILL2WORKFLOW_LARK_TASK_LIVE=1 \ + python3 scripts/controlled_lark_pilot.py decide \ + --work-dir "$HOME/.local/share/skill2workflow/pilots/loop-40" \ + --run-id "$DAY1_RUN_ID" \ + --approve \ + --confirm-live-create +``` + +Expected: `run_status: completed`, `provider_status: completed`, and `lark_task_id_present: true`, with no task value or id. + +- [ ] **Step 5: Complete the human rejection evidence** + +Create `$HOME/.local/share/skill2workflow/pilots/loop-40/private/cases/rejection.json` with the same five-field shape as Step 3, changing only `pilot_case_id` to the opaque id `case-rejection-001`, and use partner-approved private values. Protect it, start it, inspect the waiting summary, and record the returned run id privately as `REJECTION_RUN_ID`: + +```bash +chmod 600 "$HOME/.local/share/skill2workflow/pilots/loop-40/private/cases/rejection.json" +python3 scripts/controlled_lark_pilot.py start \ + --work-dir "$HOME/.local/share/skill2workflow/pilots/loop-40" \ + --input "$HOME/.local/share/skill2workflow/pilots/loop-40/private/cases/rejection.json" +``` + +Reject that waiting run without Vault: + +```bash +python3 scripts/controlled_lark_pilot.py decide \ + --work-dir "$HOME/.local/share/skill2workflow/pilots/loop-40" \ + --run-id "$REJECTION_RUN_ID" \ + --reject +``` + +Expected: `gate_decision: rejected`, `connector_invoked: false`, and no Lark connector event. + +- [ ] **Step 6: Complete disabled-live failure and rollback exercises** + +Run without either live environment variable: + +```bash +env -u SKILL2WORKFLOW_LARK_TASK_LIVE -u LARK_BOT_ACCESS_TOKEN \ + python3 scripts/controlled_lark_pilot.py exercise-failure \ + --work-dir "$HOME/.local/share/skill2workflow/pilots/loop-40" + +env -u SKILL2WORKFLOW_LARK_TASK_LIVE -u LARK_BOT_ACCESS_TOKEN \ + python3 scripts/controlled_lark_pilot.py exercise-rollback \ + --work-dir "$HOME/.local/share/skill2workflow/pilots/loop-40" +``` + +Expected: both exercises report `passed: true`; failure reports no credential or transport access; rollback reports live approval blocked and dry-run completed. + +- [ ] **Step 7: Regenerate and inspect private redacted evidence** + +```bash +python3 scripts/controlled_lark_pilot.py evidence \ + --work-dir "$HOME/.local/share/skill2workflow/pilots/loop-40" +``` + +Expected: one approved live run, one rejection, both exercises passing, and unmet conditions for the remaining dates/runs, verification, and final decision. Do not commit the incomplete pack. + +--- + +### Task 10: Complete Approved Runs On Days 2–5 + +**Files:** +- Private only: `$HOME/.local/share/skill2workflow/pilots/loop-40/` +- No repository evidence is committed until Task 11 finalization. + +**Interfaces:** +- Consumes: Task 9 private state, the same approved workflow version, continuing paid engagement, and real partner-approved inputs. +- Produces: five total approved live runs across five `Asia/Shanghai` dates and at least two opaque case ids. + +- [ ] **Step 1: Complete Day 2 approved run** + +Create an owner-only private file with opaque id `case-001`, start it, inspect the waiting run, approve it through Vault, and regenerate evidence using the exact Task 9 commands with `day-2.json` and the returned Day 2 run id. + +Expected evidence: `approved_live_runs: 2`, `distinct_calendar_days: 2`, `distinct_private_cases: 1`. + +- [ ] **Step 2: Complete Day 3 approved run** + +Repeat on a third `Asia/Shanghai` calendar date with opaque id `case-001`. + +Expected evidence: `approved_live_runs: 3`, `distinct_calendar_days: 3`, `distinct_private_cases: 1`. + +- [ ] **Step 3: Complete Day 4 approved run with the second private case** + +Repeat on a fourth date using opaque id `case-002` and real partner-approved values. + +Expected evidence: `approved_live_runs: 4`, `distinct_calendar_days: 4`, `distinct_private_cases: 2`. + +- [ ] **Step 4: Complete Day 5 approved run** + +Repeat on a fifth date using either `case-001` or `case-002`. + +Expected evidence: `approved_live_runs: 5`, `distinct_calendar_days: 5`, `distinct_private_cases: 2`, `rejected_runs >= 1`. + +- [ ] **Step 5: Stop immediately on a redaction, duplicate, permission, or provider anomaly** + +If any run exposes a forbidden value, creates an unexpected duplicate, targets the wrong user, bypasses the gate, or produces non-normalized provider output: + +1. remove the exact live switch; +2. do not run another approval; +3. retain private state; +4. record a `defer` candidate decision; +5. return to implementation with a failing regression test. + +Do not replace failed historical runs with clean runs. + +--- + +### Task 11: Verify, Finalize, Export Evidence, And Advance The Roadmap + +**Files:** +- Generate: `docs/pilot-evidence/loop-40/pilot-charter.json` +- Generate: one sequential JSON file below `docs/pilot-evidence/loop-40/runs/` for every recorded controlled run +- Generate: `docs/pilot-evidence/loop-40/exercises/rejection.json` +- Generate: `docs/pilot-evidence/loop-40/exercises/failure.json` +- Generate: `docs/pilot-evidence/loop-40/exercises/rollback.json` +- Generate: `docs/pilot-evidence/loop-40/verification.json` +- Generate: `docs/pilot-evidence/loop-40/evidence-index.json` +- Generate: `docs/pilot-evidence/loop-40/decision.json` +- Modify: `ROADMAP.md` +- Modify: `README.md` +- Modify: `tests/test_production_roadmap.py` +- Modify: `tests/test_product_connector_pilot_roadmap.py` +- Modify: `tests/test_first_product_connector_candidate_docs.py` +- Modify: `tests/test_controlled_lark_pilot_docs.py` +- Modify: `docs/superpowers/plans/2026-07-17-controlled-live-pilot.md` + +**Interfaces:** +- Consumes: complete private multi-day evidence and partner/operator decision. +- Produces: validated commit-safe Loop 40 evidence, Controlled Live Pilot maturity, and the next Roadmap decision. + +- [ ] **Step 1: Run fixed sanitized verification** + +```bash +env -u SKILL2WORKFLOW_LARK_TASK_LIVE -u LARK_BOT_ACCESS_TOKEN \ + python3 scripts/controlled_lark_pilot.py verify \ + --work-dir "$HOME/.local/share/skill2workflow/pilots/loop-40" +``` + +Expected: `all_passed: true` and all seven fixed command ids pass. + +- [ ] **Step 2: Prepare the redacted decision file** + +Create the owner-only private decision file with the chosen decision and a rationale containing no customer, account, user, task, token, price, or contract detail: + +```json +{ + "schema_version": "controlled-lark-pilot-decision-0.1.0", + "decision": "continue", + "partner_acknowledged": true, + "operator_acknowledged": true, + "commercial_engagement_confirmed": true, + "rationale": "The controlled workflow delivered the agreed business result within the approved safety boundary." +} +``` + +`decision` may be `continue`, `harden`, or `defer`; do not force `continue` if evidence supports another outcome. + +- [ ] **Step 3: Finalize and export the validated pack** + +```bash +chmod 600 "$HOME/.local/share/skill2workflow/pilots/loop-40/private/decision.json" +python3 scripts/controlled_lark_pilot.py finalize \ + --work-dir "$HOME/.local/share/skill2workflow/pilots/loop-40" \ + --decision-file "$HOME/.local/share/skill2workflow/pilots/loop-40/private/decision.json" \ + --output-dir docs/pilot-evidence/loop-40 +``` + +Expected: `status: finalized`, five approved live runs, five dates, at least two cases, at least one rejection, all exercises and verification passed, and no unmet condition. + +- [ ] **Step 4: Run repository secret and forbidden-value checks on exported evidence** + +Run the evidence validator plus targeted searches using only known schema keys, never raw private values: + +```bash +python3 scripts/controlled_lark_pilot.py evidence \ + --work-dir "$HOME/.local/share/skill2workflow/pilots/loop-40" \ + --output-dir docs/pilot-evidence/loop-40 +python3 scripts/secret_hygiene.py examples/workflows +rg -n '"(owner_open_id|account_name|renewal_risk|due_at|client_token|guid|request|response)"[[:space:]]*:' docs/pilot-evidence/loop-40 +rg -n 'Authorization|Bearer |LARK_BOT_ACCESS_TOKEN' docs/pilot-evidence/loop-40 +``` + +Expected: validator passes; both `rg` commands return no matches (exit 1). Allowed boolean fields such as `due_at_present` do not match the exact JSON-key check. Schema documentation outside the evidence directory may contain these terms and is not part of this check. + +- [ ] **Step 5: Write failing Roadmap completion tests** + +Update exact assertions to require: + +```text +Completed delivery loops: 1-40 +Current maturity: Controlled Live Pilot +Loop 40: Controlled Live Connector Pilot | Complete +docs/pilot-evidence/loop-40/evidence-index.json +docs/pilot-evidence/loop-40/decision.json +``` + +Require Loop 41 to become `Next` only when the final decision is `continue` or `harden`; if the decision is `defer`, require no active implementation loop and document the defer decision instead. + +- [ ] **Step 6: Run Roadmap tests and verify RED** + +```bash +PYTHONPATH=src python3 -m unittest tests.test_production_roadmap tests.test_product_connector_pilot_roadmap tests.test_first_product_connector_candidate_docs tests.test_controlled_lark_pilot_docs -v +``` + +Expected: failures because Roadmap and README still describe Loop 40 as active and Local Evaluation. + +- [ ] **Step 7: Advance Roadmap and README from validated evidence only** + +In `ROADMAP.md`: + +- set completed loops to 1-40; +- set current maturity to Controlled Live Pilot; +- move Loop 40 to Delivery History with links to the runbook, evidence index, exercises, verification, and decision; +- state the recorded final decision; +- select Loop 41 only if the decision permits it; +- keep the fixed live action boundary and all Loop 41–43 exclusions truthful. + +In `README.md`, update only the compact maturity and active-loop summary. + +- [ ] **Step 8: Run complete final verification** + +```bash +PYTHONPATH=src python3 -m unittest discover -s tests -v +python3 -m py_compile src/skill2workflow/*.py examples/connectors/lark_task_connector.py +python3 scripts/secret_hygiene.py examples/workflows +python3 scripts/lark_task_connector_smoke.py --work-dir /tmp/skill2workflow-lark-task-connector-final +python3 scripts/lark_task_pilot_smoke.py --work-dir /tmp/skill2workflow-lark-task-pilot-final +git diff --check +``` + +Expected: every command exits 0. + +- [ ] **Step 9: Request final code and evidence review** + +Use `superpowers:requesting-code-review` to verify the complete spec and evidence map. The reviewer must independently confirm every acceptance condition from committed evidence and must treat missing or indirect evidence as incomplete. + +Expected: no unresolved finding and an explicit statement that the Controlled Live Pilot gate is proven. + +- [ ] **Step 10: Update this plan's execution record and commit Loop 40 completion** + +Check every completed step, add the final test count and review result, then commit: + +```bash +git add ROADMAP.md README.md docs/pilot-evidence/loop-40 docs/superpowers/plans/2026-07-17-controlled-live-pilot.md tests/test_production_roadmap.py tests/test_product_connector_pilot_roadmap.py tests/test_first_product_connector_candidate_docs.py tests/test_controlled_lark_pilot_docs.py +git commit -m "docs: complete controlled live pilot" +``` + +- [ ] **Step 11: Remove or rotate the live credential outside the repository** + +Delete the short-lived Vault credential or rotate the bot token according to the partner's security policy. Verify the credential is absent without printing its value. This external action is required for pilot closeout but produces only a boolean operator acknowledgement, never a committed secret record. diff --git a/docs/superpowers/specs/2026-07-11-scoped-live-lark-task-design.md b/docs/superpowers/specs/2026-07-11-scoped-live-lark-task-design.md new file mode 100644 index 0000000..35e24b2 --- /dev/null +++ b/docs/superpowers/specs/2026-07-11-scoped-live-lark-task-design.md @@ -0,0 +1,418 @@ +# Scoped Live Lark Task Connector Design + +**Date:** 2026-07-11 + +**Status:** Approved for implementation planning + +## Purpose + +Complete Loop 39 by adding one explicitly enabled live action to the out-of-core `lark_task` connector: create one Feishu task through Task API v2. + +The live path must preserve the existing dry-run default, Workflow DSL compatibility, credential-handle boundary, durable trigger-input contract, compact audit contract, and explicit connector-loading model. It must also support one manually confirmed live validation after local fake-transport evidence passes. + +## Decision Summary + +- Product endpoint: Feishu domestic service only. +- Connector id and kind: `lark_task`. +- Operation: `create_task`. +- Live activation: `mode: live` plus `SKILL2WORKFLOW_LARK_TASK_LIVE=1`. +- Default mode: `dry_run`. +- Credential handle: `lark_bot_access_token`. +- HTTP implementation: Python standard library. +- Test transport: injectable callable; no live network in CI. +- Idempotency: native Feishu `client_token`, derived from execution identity. +- Real validation: one explicitly confirmed task assigned to the consenting current user. +- Runtime scope: single action, single fixed domain, no OAuth or token refresh. + +## Verified Provider Contract + +The implementation targets the documented Feishu Task API v2 create endpoint: + +```text +POST https://open.feishu.cn/open-apis/task/v2/tasks?user_id_type=open_id +``` + +Official documentation: + +- Create task: `https://open.feishu.cn/document/uAjLw4CM/ukTMukTMukTM/task-v2/task/create` +- Task v2 overview and idempotency: `https://open.feishu.cn/document/task-v2/overview?lang=zh-CN` + +Relevant provider facts: + +- `summary` is required. +- `description` is optional. +- `members` can contain an assignee identified by `open_id`. +- `due.timestamp` is an epoch-millisecond string. +- `client_token` activates provider-native idempotency. +- Repeated calls using the same `client_token` must keep request parameters unchanged. +- The documented endpoint limit is 10 requests per second. +- Either `task:task:write` or `task:task:writeonly` is sufficient for create access. +- Both bot and user access tokens are supported; Loop 39 uses the existing bot-token handle. + +Loop 39 does not target `open.larksuite.com`, accept a custom base URL, or support Task API v1. + +## Architecture + +### 1. Ephemeral Execution Identity + +`LocalExecutor` already stores these fields outside `state.context`: + +- `workflow_id` +- `workflow_version` +- `run_id` +- current node id + +Before each connector invocation, the executor will create a deep copy of the durable run context and overwrite a reserved `_execution` object: + +```json +{ + "_execution": { + "workflow_id": "workflow_example", + "workflow_version": "0.1.0", + "run_id": "run_example", + "node_id": "create_lark_task" + } +} +``` + +This enriched object is passed only to `ConnectorRuntime.execute_connector(...)`. It is not written back to `state.context`, run state, audit, snapshots, or workflow artifacts. User-provided `_execution` data is never trusted; runtime-owned values replace it. + +The existing `input` and `trigger` context objects remain unchanged. + +### 2. External Connector Boundary + +The implementation remains in `examples/connectors/lark_task_connector.py`. The connector stays out of the built-in registry and must still be loaded explicitly with `load_external_connector(...)`. + +The public entrypoint remains backward compatible: + +```python +execute(binding, credential_provider=None, context=None, transport=None) +``` + +The optional `transport` parameter exists only to inject deterministic fake HTTP behavior in tests and the validation helper. Normal runtime calls omit it and use the standard-library sender. + +No live behavior is added to the built-in HTTP connector. + +### 3. Dual Live Activation + +Live network activity requires both: + +1. connector binding `mode` equals `live`; and +2. environment variable `SKILL2WORKFLOW_LARK_TASK_LIVE` equals the exact string `1`. + +If `mode` is missing, the connector continues in `dry_run`. If `mode` is `live` but the environment switch is absent or has any other value, the connector returns a compact failed result with `provider_status: live_disabled` before credential resolution or transport invocation. + +No other truthy spellings are accepted. This makes rollback a one-variable operation and prevents accidental activation. + +### 4. Fixed Network Boundary + +The live connector constructs the method, URL, query string, and headers internally: + +- method: `POST` +- scheme and host: `https://open.feishu.cn` +- path: `/open-apis/task/v2/tasks` +- query: `user_id_type=open_id` +- headers: `Authorization: Bearer ` and `Content-Type: application/json; charset=utf-8` + +Workflow DSL and connector bindings cannot override the endpoint, method, query, authorization header, content type, or arbitrary extra headers. + +This prevents the live connector from becoming a general outbound HTTP or SSRF surface. + +## Request Transformation + +The connector continues to map the existing local business fields: + +| Local field | Feishu request field | Rule | +| --- | --- | --- | +| `title` | `summary` | Required, non-empty string | +| `description` | `description` | Optional string | +| `assignee_open_id` | `members[0]` | Optional member `{id, type: "user", role: "assignee"}` | +| `due_at` | `due` | Optional timezone-aware RFC 3339 value converted to epoch milliseconds | + +`due_at` accepts `Z` or an explicit UTC offset. Naive timestamps are rejected before network activity. The request uses `is_all_day: false`; all-day date support is outside Loop 39. + +Unknown local body keys are ignored by the provider request builder. Raw `source` or other dry-run fixture metadata is not forwarded. + +## Idempotency + +Live mode requires all four runtime-owned execution fields: + +- workflow id +- workflow version +- run id +- node id + +The connector serializes those four strings as a canonical JSON array and derives a deterministic UUID v5. The canonical UUID becomes Feishu `client_token`. + +Properties: + +- stable across retry attempts within one run and node; +- different for different workflow versions, runs, or nodes; +- safe to record only as `idempotency_key_present: true`, not as the digest itself; +- compatible with provider-native retry behavior after timeout or server error. + +Live mode fails before transport invocation if execution identity is absent or incomplete. + +Because Feishu supports native `client_token`, Loop 39 does not add a local idempotency database. Fake-transport tests prove stable token derivation and unchanged request parameters across retries. The one real validation call uses a stable validation run id; an accidental repeat with identical parameters therefore uses the same provider token instead of creating a new task. + +## Credential Handling + +The connector resolves `lark_bot_access_token` only at live execution time. The resolved value is held in a local variable long enough to construct the in-memory Authorization header. + +The resolved token must never appear in: + +- Workflow DSL +- connector binding copies returned to callers +- trigger input +- persisted run state +- node results +- events or audit +- snapshots +- smoke or validation artifacts +- exception text + +The result retains only the existing credential summary: + +```json +{ + "status": "resolved", + "handles": ["lark_bot_access_token"] +} +``` + +The manual validation helper obtains the token from `LARK_BOT_ACCESS_TOKEN`, injected through Avibe Vault, and immediately wraps it in the existing `StaticCredentialProvider`. The command never accepts the token as a CLI argument and never prints it. + +## Transport Contract + +The default transport uses `urllib.request` with a fixed 10-second timeout. The transport contract is: + +```python +transport(request: urllib.request.Request, timeout: float) -> response +``` + +The response exposes an HTTP status and a `read() -> bytes` method. Fake transports capture the `Request` for assertions and return deterministic response objects or raise deterministic exceptions. + +Tests must not use live network access. They inject deterministic responses or exceptions for every required path. + +The transport layer may read a provider response into memory for parsing, but raw response bytes and provider messages are never returned, logged, persisted, or embedded in an exception. + +## Response Normalization + +A successful provider response must satisfy all of these conditions: + +- HTTP status is successful; +- JSON decodes to an object; +- top-level `code` equals `0`; +- `data.task.guid` is a non-empty string. + +The raw guid is not returned. The compact result contains only: + +```json +{ + "operation": "create_task", + "mode": "live", + "task_title_present": true, + "task_description_present": true, + "assignee_present": true, + "due_at_present": false, + "credential_status": "resolved", + "idempotency_key_present": true, + "provider_status": "completed", + "lark_task_id_present": true +} +``` + +The `output` and `audit` objects can share compact metadata. Neither contains raw task values, the idempotency digest, or the provider task guid. + +## Failure Normalization + +Expected failures become connector failed results with fixed, provider-independent error text and compact `provider_status` values: + +| Failure | `provider_status` | Required behavior | +| --- | --- | --- | +| Live environment switch absent | `live_disabled` | Fail before credential or transport access | +| Missing/invalid local input or execution identity | `validation_failed` | Fail before transport access | +| Missing credential | `credential_failed` | Existing compact credential error path; no token | +| HTTP 401 or code `99991663` | `authorization_failed` | Fixed error text | +| HTTP 403 or code `1470403` | `permission_denied` | Fixed error text | +| HTTP 429 | `rate_limited` | Failed result compatible with existing retry policy | +| code `1470400` | `validation_failed` | Do not retain provider message | +| code `1470404` | `resource_not_found` | Do not retain provider identifiers | +| code `1470422` | `idempotency_conflict` | Safe failure; no concurrent call behavior added | +| HTTP 5xx or code `1470500` | `provider_unavailable` | Retry reuses the same `client_token` | +| Timeout | `timeout` | No request body or token in error | +| Non-JSON or structurally invalid response | `malformed_response` | No raw response leakage | + +When an HTTP error carries a valid Feishu JSON error object, provider `code` classification takes precedence over the generic HTTP status. Otherwise the connector falls back to the HTTP status class; for example, an unparseable HTTP 400 response becomes `validation_failed` and an unparseable HTTP 500 response becomes `provider_unavailable`. + +The current executor retry policy remains unchanged. Loop 39 does not add backoff, retry classification, queues, or worker coordination. + +## Redaction And Durable Input Clarification + +Loop 24 intentionally persists the normalized trigger input under `run.context.input`. Loop 39 preserves that contract. + +The live connector redaction rule therefore means: + +- the connector must not copy raw task values into `node_results`, connector `output`, connector `audit`, runtime events, control-plane audit, snapshots, or connector summaries; +- values explicitly supplied as durable trigger input remain in `run.context.input` under the existing contract; +- credential values are never trigger input and remain prohibited everywhere persistent. + +Tests scan every connector-produced and promoted surface for token, title, description, assignee, due value, raw request JSON, and raw response content. A separate assertion confirms that the runtime-owned `_execution` object is not persisted. + +Adding sensitive-input declarations, encryption, field-level retention, or trigger redaction would require a separate approved loop. + +## Manual Live Validation + +Loop 39 adds: + +- `src/skill2workflow/lark_task_live_validation.py` +- `scripts/lark_task_live_validation.py` + +The command is inert unless all of these are present: + +- `--confirm-live-create` +- `--validation-run-id ` +- `--assignee-open-id ` +- `SKILL2WORKFLOW_LARK_TASK_LIVE=1` +- `LARK_BOT_ACCESS_TOKEN` injected in the environment + +The task title and description are supplied at runtime. The validation performed for this loop uses the exact task content and current-user assignment already approved in the conversation, but those raw values and the user's open id are not committed to repository artifacts. + +The helper directly exercises the explicitly loaded connector through `ConnectorRuntime`. It does not write run state. It prints only compact status and presence metadata. + +After fake-transport tests and the full verification suite pass, the agent will request or locate `LARK_BOT_ACCESS_TOKEN` through Avibe Vault and execute the confirmed live write once. If Vault approval is required, the normal protected-secret approval flow applies. + +The created task is retained as visible validation evidence. The repository receives only a redacted evidence note containing: + +- validation timestamp; +- connector id and operation; +- live opt-in present; +- credential resolved flag; +- idempotency key present flag; +- provider completed status; +- task id present flag; +- assignee present flag. + +It does not contain the token, task guid, task title, description, assignee id, raw request, or raw response. + +## Testing Strategy + +### Connector Tests + +Extend `tests/test_lark_task_connector.py` using injected transports for: + +- missing mode remains dry-run; +- explicit dry-run remains unchanged; +- live mode without the environment switch invokes no credential provider or transport; +- live mode requires `lark_bot_access_token`; +- exact method, fixed URL, query, headers, and provider request shape; +- title, description, assignee, and due transformations; +- unknown body fields are not forwarded; +- stable `client_token` for the same execution identity; +- different client tokens for different run or node identities; +- HTTP 401, 403, 429, and 5xx; +- provider codes `1470400`, `1470403`, `1470404`, `1470422`, and `1470500`; +- timeout; +- non-JSON, missing code, nonzero code, missing task, and missing guid responses; +- success metadata; +- recursive leakage scans over result data. + +### Executor Tests + +Extend `tests/test_executor.py` to prove: + +- the connector receives the runtime-owned `_execution` object; +- a user-provided `_execution` object is overwritten; +- persisted `state.context` does not acquire `_execution`; +- trigger input remains durable and unchanged. + +### Validation Helper Tests + +Create `tests/test_lark_task_live_validation.py` for: + +- missing confirmation flag; +- missing environment switch; +- missing token environment variable; +- missing validation run id or assignee; +- fake successful live validation; +- compact output only; +- no state-directory writes; +- token and task-value leakage scan. + +### Regression Verification + +Required verification includes: + +```bash +PYTHONPATH=src python3 -m unittest tests.test_lark_task_connector tests.test_executor tests.test_lark_task_live_validation -v +PYTHONPATH=src python3 -m unittest discover -s tests -v +python3 -m py_compile src/skill2workflow/*.py examples/connectors/lark_task_connector.py +python3 scripts/secret_hygiene.py examples/workflows +python3 scripts/lark_task_connector_smoke.py --work-dir /tmp/skill2workflow-lark-task-connector +python3 scripts/lark_task_pilot_smoke.py --work-dir /tmp/skill2workflow-lark-task-pilot +git diff --check +``` + +No CI command performs live network access. + +## Documentation And Roadmap Transition + +Implementation updates: + +- `docs/connectors.md` with the live activation, fixed endpoint, credential, idempotency, error, and redaction contracts; +- `docs/lark-live-connector-readiness.md` with the confirmed native `client_token` decision and durable-input clarification; +- a redacted live-validation evidence note after the successful manual call; +- `ROADMAP.md` after all evidence is complete. + +When Loop 39 is complete: + +- completed delivery loops become 1-39; +- Loop 39 moves to Delivery History; +- current maturity remains Local Evaluation until the complete controlled-pilot gate is met; +- Loop 40 becomes the next loop: Controlled Live Connector Pilot; +- the one connector-level validation call is not presented as the Loop 40 business-workflow pilot. + +`README.md` receives only the matching active-loop summary update. + +## Rollback + +Immediate rollback is setting or leaving `SKILL2WORKFLOW_LARK_TASK_LIVE` to anything other than `1`. + +Code rollback removes the live branch and validation helper while retaining the existing dry-run connector package, examples, smokes, credential handle, manifest, and Workflow DSL `0.1.0` compatibility. + +No rollback requires schema migration or state conversion. + +## Explicit Non-goals + +- Lark international domain +- Custom API base URLs +- OAuth or token refresh +- Tenant token acquisition +- Hosted secret management +- Other Feishu task operations +- Task API v1 +- All-day due dates +- General outbound HTTP behavior +- Backoff or retry-policy redesign +- Local idempotency database +- Sensitive trigger-input declarations or persistence redaction +- Connector discovery, installer, or marketplace +- Queue, worker pool, or production scheduler +- Committed live credentials, task ids, user ids, raw request bodies, or raw responses + +## Acceptance Criteria + +Loop 39 is complete only when: + +- dry-run remains the default and all existing dry-run tests and smokes pass unchanged; +- live mode requires both binding intent and the exact environment switch; +- only the fixed Feishu Task API v2 create endpoint can be called; +- the token is resolved by handle and absent from every persistent or returned surface; +- native `client_token` is stable for one execution identity and changes across distinct identities; +- success and all documented failure categories produce compact normalized results; +- raw task and provider values are absent from connector-produced run, audit, snapshot, and summary surfaces; +- `_execution` metadata is ephemeral; +- one confirmed live validation task is created for the consenting user through Vault-injected credentials; +- the committed validation evidence is redacted; +- full tests, compilation, secret hygiene, dry-run smokes, and diff checks pass; +- Loop 39 documentation and Roadmap state are updated without changing Workflow DSL compatibility. diff --git a/docs/superpowers/specs/2026-07-17-controlled-live-pilot-design.md b/docs/superpowers/specs/2026-07-17-controlled-live-pilot-design.md new file mode 100644 index 0000000..d1daf68 --- /dev/null +++ b/docs/superpowers/specs/2026-07-17-controlled-live-pilot-design.md @@ -0,0 +1,281 @@ +# Controlled Live Connector Pilot Design + +## Status + +Approved on 2026-07-17 for Loop 40 implementation. + +This design advances `skill2workflow` from Local Evaluation to the Controlled Live Pilot gate. It does not claim Self-hosted Beta or general live SaaS readiness. + +## Goal + +Run the existing sales-renewal-risk workflow as an assisted paid pilot for one consenting real team. The workflow must preserve an explicit human decision before the fixed Lark/Feishu `create_task` action, produce reproducible redacted evidence, exercise failure and rollback boundaries, and end with a documented `continue`, `harden`, or `defer` decision. + +The pilot is commercially real only when the partner has agreed to a paid or contractually committed assisted engagement. Repository evidence records only `commercial_engagement_confirmed: true`; customer identity, pricing, contract text, and payment details remain outside the repository. + +## Approved Scenario + +The only business scenario in Loop 40 is: + +1. A sales operator submits a real renewal-risk case. +2. The workflow persists the non-secret business input in a private runtime state directory. +3. A designated operator reviews the case at a human gate. +4. Rejection terminates the run without invoking the Lark connector. +5. Approval resumes the same durable run and invokes the fixed Lark/Feishu `create_task` action. +6. The resulting task is assigned to a consenting real user. +7. A redacted evidence generator derives compact proof from run and audit state without copying raw business values, credentials, provider messages, or task identifiers. + +The existing Loop 37 sales-renewal dry-run workflow is the behavioral baseline. Loop 40 introduces a separate controlled-live pilot path rather than changing the dry-run default. + +## Options Considered + +### Extend the existing pilot runner — selected + +Add a controlled-live runner and evidence pack around the current workflow, control plane, SQLite storage, credential provider, external connector, and audit surfaces. + +This option reuses the strongest verified path, keeps the scope at one action, and produces workflow-level evidence instead of another connector-only validation. + +### Wrap the one-shot validation helper + +The existing validation helper can prove a real provider write, but it bypasses workflow publication, trigger input, the human gate, durable resume, and control-plane audit. It cannot satisfy Loop 40 by itself. + +### Build the long-running production service first + +A service boundary would be useful for Self-hosted Beta, but it belongs to Loop 41. Pulling it into Loop 40 would expand scope without improving the immediate real-team evidence requirement. + +## Acceptance Contract + +The pilot gate is complete only when all of the following are true: + +- A consenting real team has an approved pilot charter. +- The charter records `commercial_engagement_confirmed: true` and the assisted support model without customer identity, price, payment, or contract details. +- At least five approved live workflow runs complete across at least five distinct calendar days. +- The approved runs represent at least two distinct private renewal-risk cases. +- At least one real human-gate rejection completes without any connector invocation. +- One safe failure exercise proves that a disabled live switch prevents credential resolution and provider transport. +- One rollback exercise proves that live behavior can be disabled while the existing dry-run pilot remains operational. +- Every live run uses the fixed Feishu domestic host and fixed `create_task` operation. +- Every live retry for the same execution identity reuses the provider-native idempotency token. +- The generated evidence pack contains no resolved credential, authorization header, raw task value, user id, provider payload, provider message, task id, or idempotency digest. +- The complete automated test suite, focused pilot checks, dry-run smoke, compilation, secret hygiene, and `git diff --check` pass. +- The partner and operator record a final `continue`, `harden`, or `defer` decision with a short redacted rationale. + +Five approved runs across five calendar days are a minimum evidence threshold, not a reliability or SLA claim. Failed business runs remain part of the evidence and do not get silently discarded or replaced. + +## Architecture + +### Controlled pilot command + +Introduce one operator-facing command wrapper backed by a focused Python module. The command has explicit phases rather than an auto-approved one-shot flow: + +- `init`: validate and persist a redacted pilot charter plus private pilot configuration. +- `start`: read one private business-input JSON file, publish or reuse the immutable workflow version, trigger a run, and stop at the human gate. +- `decide --approve`: resume an existing waiting run and allow the live connector action only when every live guard is present. +- `decide --reject`: resume the run as rejected and prove no connector action occurred. +- `evidence`: regenerate the complete redacted evidence pack from authoritative private state. +- `exercise-failure`: prove the disabled-live preflight boundary without resolving credentials or calling transport. +- `exercise-rollback`: disable live execution and prove the existing dry-run smoke still completes. +- `finalize`: validate all acceptance conditions and write the final decision record. + +Command names may be implemented as subcommands or equivalent explicit flags, but the phase separation and safety properties are contract requirements. + +### Private runtime state + +All raw business input and resolved secrets stay outside the repository in an operator-selected private work directory. + +The private directory contains: + +- SQLite control-plane and run state; +- private business-input files; +- local execution configuration; +- any operator-only diagnostic material that contains raw values. + +The runner must reject a private work directory inside the repository. Newly created private directories and files use owner-only permissions where the platform supports them. The tool must not accept a credential as a command-line argument. The preferred live path injects `LARK_APP_SECRET` through Vault or an equivalent secret manager and accepts the non-secret `LARK_APP_ID` from the process environment. Only after every live guard passes, it exchanges those values in memory for one short-lived tenant access token and immediately wraps that token in the existing credential-provider interface. The App Secret, issued token, and token-exchange response must not be persisted or printed. `LARK_BOT_ACCESS_TOKEN` remains a legacy fallback only when no App Secret is injected. + +The current durable run-context contract permits user-supplied business values in private run state. It does not permit credentials in trigger input or persisted state. + +### Redacted evidence pack + +The evidence directory is safe to inspect, review, and commit after validation. It is derived from authoritative runtime and audit state and never serves as execution input. + +The pack contains: + +- `pilot-charter.json`: scenario id, workflow id/version, support model, consent flags, commercial-confirmation boolean, planned date range, and acceptance thresholds; +- `runs/.json`: compact run identity, timestamps, gate decision, terminal status, connector status, approved metadata-presence flags, credential handle names, credential status, provider status, and idempotency-presence boolean; +- `exercises/rejection.json`: proof that a rejected gate produced no connector event; +- `exercises/failure.json`: proof that the disabled live switch stopped before credentials or transport; +- `exercises/rollback.json`: proof that live mode was disabled and the dry-run pilot still completed; +- `evidence-index.json`: aggregate counts, distinct calendar days, distinct private-case count represented only as a count, exercise status, test status, and unmet acceptance conditions; +- `decision.json`: `continue`, `harden`, or `defer`, partner acknowledgement boolean, operator acknowledgement boolean, commercial-confirmation boolean, and a compact redacted rationale. + +The evidence pack must not contain the private input path, customer name, account id, renewal-risk text, assignee open id, due date, token, authorization header, request body, response body, provider task guid, provider message, client-token digest, or raw run context. + +Evidence generation is deterministic for a given authoritative state except for an explicit generation timestamp. Re-running it replaces derived files atomically rather than appending contradictory summaries. + +### Live safety gates + +An approved live decision requires all of these conditions in the same process: + +- the run is currently waiting at the expected human gate; +- the operator passes an explicit live-confirmation flag; +- `SKILL2WORKFLOW_LARK_TASK_LIVE=1` is exact; +- `LARK_APP_ID` and Vault-injected `LARK_APP_SECRET` are present and the in-memory tenant-token exchange succeeds, or (legacy fallback only) `LARK_BOT_ACCESS_TOKEN` is present through process injection; +- the workflow connector is `lark_task` with operation `create_task` and mode `live`; +- the credential handle is exactly `lark_bot_access_token`; +- the workflow id, version, run id, and node id needed for provider idempotency are present; +- the pilot charter is valid and has not expired; +- the run has not already reached a terminal state. +- no previous approved live `create_task` connector attempt in the same private work directory has failed; such a failure requires a fresh work directory and authorization boundary. + +Missing gates fail closed. A rejection never requires the live switch or credential. Re-running approval for a completed run must not create another provider request. + +### Workflow and data flow + +`start` builds the controlled-live workflow from a fixed template, validates it, publishes the immutable version, and triggers it with one private case. It must not mutate the committed Loop 37 dry-run fixture or make live mode the connector default. + +The workflow sequence remains: + +```text +start -> review_renewal_risk -> create_lark_task -> end + \-> failure +review_renewal_risk --reject---------------------> failure +``` + +Input mapping remains limited to title, description, assignee, and due time. The connector continues to construct the Feishu domestic URL, HTTP method, headers, timeout, and provider body internally. + +Approval resumes the same run. The executor supplies `workflow_id + workflow_version + run_id + node_id` ephemerally, and the connector derives the provider-native client token from that identity. Retrying the same node keeps the identity stable; a distinct run gets a distinct token. + +## Failure And Rollback Exercises + +### Human rejection + +Start a valid case, reject it at the gate, and record a terminal rejected/failed run. Audit evidence must include the gate decision and must not include `connector_started`, `connector_completed`, or `connector_failed` for the Lark task node. + +### Disabled-live failure + +Use a controlled test run with the live connector binding but without the exact live switch. The result must be `provider_status: live_disabled`. Credential resolution and transport must not occur. The evidence records presence/status fields only. + +This exercise is safe for routine execution because it cannot write to Feishu. + +### Rollback + +Remove the live switch, verify that live approval fails closed, and then run the existing dry-run sales-renewal pilot unchanged. The rollback is successful only when the dry-run completes and its redaction checks still pass. + +No rollback step edits Workflow DSL compatibility, removes the connector package, or deletes historical evidence. + +## Error Handling + +- Invalid charter, private input, directory location, file permissions, workflow binding, or run state fails before live execution. +- Missing or expired charter fails before trigger or resume. +- Missing live confirmation, environment switch, credential, or execution identity fails closed with compact status. +- Provider failures retain only the normalized status already approved in Loop 39. +- Evidence validation failures never delete private runtime state; they report the exact evidence contract violation without echoing the sensitive value. +- Finalization fails while any threshold, exercise, acknowledgement, or verification item is missing. +- A failed run remains inspectable and countable; the operator starts a new run only for a new business attempt, not to rewrite history. + +## Testing Strategy + +Implementation follows test-driven development. + +Focused tests cover: + +- charter validation, expiry, consent, and commercial-confirmation requirements; +- rejection of repository-contained private work directories; +- owner-only file and directory creation where supported; +- fixed controlled-live workflow shape and unchanged dry-run default; +- start stopping at the expected human gate; +- approval and rejection as separate operations; +- approval guard failures before credential or transport access; +- repeated approval of a terminal run making no connector call; +- fake-transport live success through the complete published workflow; +- fake provider failure and stable retry identity; +- rejection producing no connector events; +- failure and rollback exercise evidence; +- deterministic evidence regeneration and atomic replacement; +- exact evidence allowlist and forbidden-value leakage tests; +- acceptance aggregation for five approved runs across five days and two private cases; +- finalization rejecting incomplete evidence; +- `continue`, `harden`, and `defer` decision validation; +- CLI summaries containing compact metadata only. + +Verification commands include: + +```bash +PYTHONPATH=src python3 -m unittest tests.test_controlled_lark_pilot -v +PYTHONPATH=src python3 -m unittest discover -s tests -v +python3 -m py_compile src/skill2workflow/*.py examples/connectors/lark_task_connector.py +python3 scripts/secret_hygiene.py examples/workflows +python3 scripts/lark_task_connector_smoke.py --work-dir /tmp/skill2workflow-lark-task-connector +python3 scripts/lark_task_pilot_smoke.py --work-dir /tmp/skill2workflow-lark-task-pilot +git diff --check +``` + +Real provider calls are never part of automated tests or CI. + +## Pilot Operating Sequence + +1. Create a private pilot directory outside the repository. +2. Initialize and review the redacted charter. +3. Run the unchanged dry-run rehearsal. +4. Inject the token through Vault and start the first private case. +5. Inspect the waiting run and explicitly approve or reject it. +6. Regenerate and inspect the redacted evidence pack. +7. Repeat until at least five approved runs span five calendar days and two private cases. +8. Complete the explicit rejection exercise. +9. Complete the disabled-live failure and rollback exercises. +10. Run the complete verification suite. +11. Record partner and operator acknowledgement plus the final decision. +12. Run finalization; only a successful finalization may advance the Roadmap gate. + +## Roadmap Completion + +After finalization proves every acceptance condition: + +- add the controlled live-pilot runbook; +- commit only validated redacted evidence; +- record the final decision; +- move Loop 40 to complete in `ROADMAP.md`; +- update the completed-loop count to 1-40; +- set current maturity to Controlled Live Pilot; +- select or defer the next loop based on the evidence; +- update the compact `README.md` status without copying the rolling queue; +- preserve Loops 41-43 as candidates unless the final decision explicitly selects one under the Roadmap rules. + +Roadmap completion is a separate final task after the multi-day real pilot. Passing fake-transport tests and generating an empty evidence skeleton must not advance the maturity gate. + +## Out Of Scope + +- Additional Lark/Feishu actions or APIs. +- OAuth, token refresh, hosted callbacks, or public ingress. +- Automatic connector discovery, installation, or marketplace behavior. +- Background workers, queues, distributed scheduling, or production scheduling. +- Multi-tenant control plane, tenant isolation, RBAC, or IAM. +- Hosted secret management or a general credential product. +- Loop 41 long-running service behavior. +- Exactly-once claims, SLA claims, or general production-readiness claims. +- Committing customer identity, commercial terms, raw pilot payloads, or live credentials. + +## Implementation Boundaries + +- Python 3.9 standard library remains sufficient. +- Workflow DSL remains the execution source of truth. +- SQLite is used for the controlled pilot's private durable state. +- The Lark connector remains out of core and explicitly loaded. +- Dry-run remains the connector and example default. +- Existing Workflow DSL `0.1.0` compatibility is unchanged. +- No new runtime dependency is introduced. +- Parser, compiler, validator, executor, connector, storage, or CLI behavior changes begin with failing tests. + +## Completion Evidence Map + +| Requirement | Authoritative evidence | +| --- | --- | +| Paid assisted engagement | Validated charter and final decision with `commercial_engagement_confirmed: true` | +| Real-team use | Five redacted approved-run records across five dates plus partner acknowledgement | +| Complete workflow | Published workflow identity, waiting gate evidence, resume evidence, and connector terminal status | +| Human control | Approved-run gate decisions and one rejection with no connector event | +| Live action | `mode: live`, `provider_status: completed`, and task-id-presence boolean | +| Idempotency | Stable execution identity tests and idempotency-presence evidence | +| Failure boundary | Disabled-live exercise with no credential or transport access | +| Rollback boundary | Live-disabled proof plus unchanged successful dry-run smoke | +| Redaction | Evidence allowlist validator, forbidden-value tests, and secret-hygiene verification | +| Business decision | Final validated `continue`, `harden`, or `defer` record | diff --git a/examples/connectors/lark_task_connector.py b/examples/connectors/lark_task_connector.py index d6d14b3..69dcb97 100644 --- a/examples/connectors/lark_task_connector.py +++ b/examples/connectors/lark_task_connector.py @@ -1,9 +1,16 @@ -"""Dry-run Lark/Feishu task connector package fixture for Loop 36.""" +"""Dry-run-default Lark/Feishu task connector with scoped live support.""" from __future__ import annotations import copy -from typing import Dict, List +import json +import os +import socket +import uuid +from datetime import datetime +from typing import Dict, List, Tuple +from urllib import error as urllib_error +from urllib import request as urllib_request from skill2workflow.connectors import ( CONNECTOR_EXECUTION_CONTRACT_VERSION, @@ -13,6 +20,20 @@ from skill2workflow.credentials import CredentialResolutionError +LIVE_ENVIRONMENT_SWITCH = "SKILL2WORKFLOW_LARK_TASK_LIVE" +LIVE_URL = "https://open.feishu.cn/open-apis/task/v2/tasks?user_id_type=open_id" +LIVE_TIMEOUT_SECONDS = 10.0 +REQUIRED_CREDENTIAL_HANDLE = "lark_bot_access_token" +PROVIDER_CODE_STATUS = { + 99991663: "authorization_failed", + 1470400: "validation_failed", + 1470403: "permission_denied", + 1470404: "resource_not_found", + 1470422: "idempotency_conflict", + 1470500: "provider_unavailable", +} + + MANIFEST = { "manifest_version": CONNECTOR_MANIFEST_VERSION, "id": "lark_task", @@ -20,12 +41,12 @@ "kind": "lark_task", "status": "active", "node_types": ["tool_call"], - "description": "External dry-run connector package for creating Lark/Feishu task requests locally.", + "description": "Explicit dry-run-default connector with opt-in scoped Feishu task creation.", "config_schema": { "type": "object", "properties": { "operation": {"type": "string"}, - "mode": {"type": "string"}, + "mode": {"type": "string", "enum": ["dry_run", "live"]}, "request": { "type": "object", "properties": { @@ -54,8 +75,8 @@ } -def execute(binding: Dict[str, object], credential_provider=None, context=None) -> Dict[str, object]: - """Validate a Lark task create request without calling the live Lark API.""" +def execute(binding: Dict[str, object], credential_provider=None, context=None, transport=None) -> Dict[str, object]: + """Validate or execute a scoped Lark task create request.""" if not isinstance(binding, dict): raise ConnectorExecutionError("lark_task connector binding must be an object") @@ -64,8 +85,8 @@ def execute(binding: Dict[str, object], credential_provider=None, context=None) raise ConnectorExecutionError("lark_task connector only supports operation create_task") mode = str(binding.get("mode") or "dry_run") - if mode != "dry_run": - raise ConnectorExecutionError("lark_task connector only supports mode dry_run") + if mode not in ("dry_run", "live"): + raise ConnectorExecutionError("lark_task connector only supports modes dry_run and live") request = binding.get("request", {}) if request is None: @@ -74,8 +95,75 @@ def execute(binding: Dict[str, object], credential_provider=None, context=None) raise ConnectorExecutionError("lark_task connector.request must be an object") body, mapping_summary = _mapped_body(request, context) - credential_summary = _resolve_credentials(binding.get("credentials", []), credential_provider) audit = _task_audit_metadata(operation, mode, body) + + if mode == "live": + if not _live_enabled(): + return _live_result("failed", audit, "live_disabled", mapping_summary) + + try: + payload = _provider_request_body(body, context) + except ConnectorExecutionError: + return _failed_live_result( + audit, + "validation_failed", + mapping_summary, + idempotency_key_present=False, + ) + + try: + credential_summary, credential_values = _resolve_credentials( + binding.get("credentials", []), credential_provider + ) + if REQUIRED_CREDENTIAL_HANDLE not in credential_values: + raise ConnectorExecutionError( + f"credential handle not found: {REQUIRED_CREDENTIAL_HANDLE}" + ) + except ConnectorExecutionError: + return _failed_live_result( + audit, + "credential_failed", + mapping_summary, + credential_summary={ + "status": "failed", + "handles": [REQUIRED_CREDENTIAL_HANDLE], + }, + idempotency_key_present=True, + ) + + try: + live_request = _request(payload, credential_values[REQUIRED_CREDENTIAL_HANDLE]) + except Exception: + return _failed_live_result( + audit, + "credential_failed", + mapping_summary, + credential_summary, + idempotency_key_present=True, + ) + + provider_status, task_id_present = _transport_outcome(live_request, transport or _default_transport) + if provider_status != "completed" or not task_id_present: + return _failed_live_result( + audit, + provider_status, + mapping_summary, + credential_summary, + idempotency_key_present=True, + ) + return _live_result( + "completed", + audit, + "completed", + mapping_summary, + credential_summary, + True, + True, + ) + + credential_summary, _credential_values = _resolve_credentials( + binding.get("credentials", []), credential_provider + ) if not audit["task_title_present"]: raise ConnectorExecutionError("lark_task connector task title is required") @@ -93,6 +181,53 @@ def execute(binding: Dict[str, object], credential_provider=None, context=None) } +def preflight(binding: Dict[str, object], context=None) -> Dict[str, object]: + """Construct the fixed live payload without resolving a credential or using transport.""" + audit = { + "operation": "", + "mode": "", + "task_title_present": False, + "task_description_present": False, + "assignee_present": False, + "due_at_present": False, + } + mapping_summary = {"status": "not_applied", "input_keys": []} + ready = False + try: + if not isinstance(binding, dict): + raise ConnectorExecutionError("lark_task connector binding must be an object") + operation = str(binding.get("operation") or "") + if operation != "create_task": + raise ConnectorExecutionError("lark_task connector only supports operation create_task") + mode = str(binding.get("mode") or "") + if mode != "live": + raise ConnectorExecutionError("lark_task preflight requires mode live") + request = binding.get("request", {}) + if request is None: + request = {} + if not isinstance(request, dict): + raise ConnectorExecutionError("lark_task connector.request must be an object") + body, mapping_summary = _mapped_body(request, context) + audit = _task_audit_metadata(operation, mode, body) + _provider_request_body(body, context) + ready = True + except ConnectorExecutionError: + pass + return { + "status": "ready" if ready else "invalid", + "connector": {"id": "lark_task", "kind": "lark_task"}, + "output": { + "operation": audit["operation"], + "mode": audit["mode"], + "provider_payload_constructed": ready, + "credential_resolution_attempted": False, + "network_called": False, + }, + "audit": audit, + "input_mapping": mapping_summary, + } + + def _task_audit_metadata(operation: str, mode: str, body: Dict[str, object]) -> Dict[str, object]: return { "operation": operation, @@ -112,13 +247,198 @@ def _present(value: object) -> bool: return True -def _resolve_credentials(credentials: object, credential_provider) -> Dict[str, object]: +def _live_enabled() -> bool: + return os.environ.get(LIVE_ENVIRONMENT_SWITCH) == "1" + + +def _execution_identity(context: object) -> List[str]: + context_root = context if isinstance(context, dict) else {} + execution = context_root.get("_execution", {}) + if not isinstance(execution, dict): + return [] + values = [ + str(execution.get("workflow_id") or ""), + str(execution.get("workflow_version") or ""), + str(execution.get("run_id") or ""), + str(execution.get("node_id") or ""), + ] + return values if all(values) else [] + + +def _client_token(context: object) -> str: + identity = _execution_identity(context) + if not identity: + raise ConnectorExecutionError("lark_task live execution identity is required") + canonical = json.dumps(identity, ensure_ascii=False, separators=(",", ":")) + return str(uuid.uuid5(uuid.NAMESPACE_URL, "skill2workflow:lark_task:" + canonical)) + + +def _provider_request_body(body: Dict[str, object], context: object) -> Dict[str, object]: + title = body.get("title") + if not isinstance(title, str) or not title.strip(): + raise ConnectorExecutionError("lark_task connector task title is required") + if len(title) > 3000: + raise ConnectorExecutionError("lark_task connector task title exceeds the provider limit") + + payload: Dict[str, object] = { + "summary": title, + "client_token": _client_token(context), + } + description = body.get("description") + if description is not None: + if not isinstance(description, str): + raise ConnectorExecutionError("lark_task connector description must be a string") + if len(description) > 3000: + raise ConnectorExecutionError( + "lark_task connector description exceeds the provider limit" + ) + payload["description"] = description + + assignee = body.get("assignee_open_id") + if assignee is not None: + if not isinstance(assignee, str) or not assignee.strip(): + raise ConnectorExecutionError("lark_task connector assignee_open_id must be a non-empty string") + payload["members"] = [{"id": assignee, "type": "user", "role": "assignee"}] + + due_at = body.get("due_at") + if due_at is not None: + payload["due"] = {"timestamp": _due_timestamp(due_at), "is_all_day": False} + return payload + + +def _due_timestamp(value: object) -> str: + if not isinstance(value, str) or not value.strip(): + raise ConnectorExecutionError("lark_task connector due_at must be an RFC 3339 string") + normalized = value[:-1] + "+00:00" if value.endswith("Z") else value + try: + parsed = datetime.fromisoformat(normalized) + except ValueError: + raise ConnectorExecutionError("lark_task connector due_at must be an RFC 3339 string") + if parsed.tzinfo is None: + raise ConnectorExecutionError("lark_task connector due_at must include a timezone") + return str(int(parsed.timestamp() * 1000)) + + +def _request(payload: Dict[str, object], token: str) -> urllib_request.Request: + return urllib_request.Request( + LIVE_URL, + data=json.dumps(payload, ensure_ascii=False, separators=(",", ":")).encode("utf-8"), + headers={ + "Authorization": f"Bearer {token}", + "Content-Type": "application/json; charset=utf-8", + }, + method="POST", + ) + + +def _default_transport(request: urllib_request.Request, timeout: float): + return urllib_request.urlopen(request, timeout=timeout) + + +def _http_status(status: int) -> str: + if status == 400: + return "validation_failed" + if status == 401: + return "authorization_failed" + if status == 403: + return "permission_denied" + if status == 404: + return "resource_not_found" + if status == 429: + return "rate_limited" + if status >= 500: + return "provider_unavailable" + return "malformed_response" + + +def _decode_provider(raw: bytes): + try: + payload = json.loads(raw.decode("utf-8")) + except (UnicodeDecodeError, json.JSONDecodeError): + return None + return payload if isinstance(payload, dict) else None + + +def _provider_outcome(status: int, raw: bytes) -> Tuple[str, bool]: + payload = _decode_provider(raw) + if payload is not None: + code = payload.get("code") + if isinstance(code, int) and code in PROVIDER_CODE_STATUS: + return PROVIDER_CODE_STATUS[code], False + if code == 0: + if status < 200 or status >= 300: + return _http_status(status), False + data = payload.get("data", {}) + task = data.get("task", {}) if isinstance(data, dict) else {} + guid = task.get("guid") if isinstance(task, dict) else "" + if isinstance(guid, str) and guid: + return "completed", True + return "malformed_response", False + return _http_status(status), False + + +def _safe_close(response) -> None: + try: + close = getattr(response, "close", None) + if callable(close): + close() + except Exception: + return + + +def _read_provider_outcome(response, status: int) -> Tuple[str, bool]: + try: + raw = response.read() + except (TimeoutError, socket.timeout): + return "timeout", False + except urllib_error.URLError as error: + if isinstance(error.reason, (TimeoutError, socket.timeout)): + return "timeout", False + return "provider_unavailable", False + except Exception: + return "provider_unavailable", False + finally: + _safe_close(response) + return _provider_outcome(status, raw) + + +def _transport_outcome(request: urllib_request.Request, transport) -> Tuple[str, bool]: + try: + response = transport(request, LIVE_TIMEOUT_SECONDS) + except urllib_error.HTTPError as error: + try: + status = int(error.code) + except Exception: + _safe_close(error) + return "malformed_response", False + return _read_provider_outcome(error, status) + except (TimeoutError, socket.timeout): + return "timeout", False + except urllib_error.URLError as error: + if isinstance(error.reason, (TimeoutError, socket.timeout)): + return "timeout", False + return "provider_unavailable", False + except Exception: + return "provider_unavailable", False + + try: + status = int(getattr(response, "status", 0)) + except Exception: + _safe_close(response) + return "malformed_response", False + return _read_provider_outcome(response, status) + + +def _resolve_credentials( + credentials: object, credential_provider +) -> Tuple[Dict[str, object], Dict[str, str]]: if credentials in (None, []): - return {"status": "skipped", "handles": []} + return {"status": "skipped", "handles": []}, {} if not isinstance(credentials, list): raise ConnectorExecutionError("connector.credentials must be a list") handles: List[str] = [] + values: Dict[str, str] = {} for index, credential in enumerate(credentials): if not isinstance(credential, dict): raise ConnectorExecutionError(f"connector.credentials[{index}] must be an object") @@ -128,15 +448,78 @@ def _resolve_credentials(credentials: object, credential_provider) -> Dict[str, handle = str(credential.get("handle") or "") if not handle: raise ConnectorExecutionError(f"connector.credentials[{index}].handle is required") + if handle != REQUIRED_CREDENTIAL_HANDLE: + continue if credential_provider is None: raise ConnectorExecutionError(f"credential handle not found: {handle}") try: - credential_provider.resolve(handle) + values[handle] = credential_provider.resolve(handle) except CredentialResolutionError as error: raise ConnectorExecutionError(str(error)) handles.append(handle) - return {"status": "resolved", "handles": sorted(handles)} + return {"status": "resolved", "handles": sorted(handles)}, values + + +def _live_result( + status: str, + audit: Dict[str, object], + provider_status: str, + mapping_summary: Dict[str, object], + credential_summary: Dict[str, object] = None, + idempotency_key_present: bool = False, + task_id_present: bool = False, +) -> Dict[str, object]: + compact = dict(audit) + compact.update( + { + "credential_status": str((credential_summary or {}).get("status") or "skipped"), + "idempotency_key_present": idempotency_key_present, + "provider_status": provider_status, + "lark_task_id_present": task_id_present, + } + ) + result = { + "status": status, + "connector": {"id": "lark_task", "kind": "lark_task"}, + "output": dict(compact), + "audit": compact, + "input_mapping": mapping_summary, + } + if credential_summary: + result["credentials"] = credential_summary + if status == "failed": + result["error"] = f"lark_task live request failed: {provider_status}" + return result + + +def _failed_live_result( + audit: Dict[str, object], + provider_status: str, + mapping_summary: Dict[str, object], + credential_summary: Dict[str, object] = None, + idempotency_key_present: bool = False, +) -> Dict[str, object]: + compact = dict(audit) + compact.update( + { + "credential_status": str((credential_summary or {}).get("status") or "skipped"), + "idempotency_key_present": idempotency_key_present, + "provider_status": provider_status, + "lark_task_id_present": False, + } + ) + result = { + "status": "failed", + "connector": {"id": "lark_task", "kind": "lark_task"}, + "output": dict(compact), + "error": f"lark_task live request failed: {provider_status}", + "audit": compact, + "input_mapping": mapping_summary, + } + if credential_summary: + result["credentials"] = credential_summary + return result def _mapped_body(request: Dict[str, object], context: object): diff --git a/scripts/controlled_lark_pilot.py b/scripts/controlled_lark_pilot.py new file mode 100644 index 0000000..60b2d8c --- /dev/null +++ b/scripts/controlled_lark_pilot.py @@ -0,0 +1,17 @@ +#!/usr/bin/env python3 +"""Operate the controlled Lark/Feishu pilot from a source checkout.""" + +from pathlib import Path +import sys + + +REPO_ROOT = Path(__file__).resolve().parents[1] +SRC_ROOT = REPO_ROOT / "src" +if str(SRC_ROOT) not in sys.path: + sys.path.insert(0, str(SRC_ROOT)) + +from skill2workflow.controlled_lark_pilot import main + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/scripts/lark_task_live_validation.py b/scripts/lark_task_live_validation.py new file mode 100644 index 0000000..02fc82d --- /dev/null +++ b/scripts/lark_task_live_validation.py @@ -0,0 +1,17 @@ +#!/usr/bin/env python3 +"""Run guarded Lark/Feishu task live validation from a source checkout.""" + +from pathlib import Path +import sys + + +REPO_ROOT = Path(__file__).resolve().parents[1] +SRC_ROOT = REPO_ROOT / "src" +if str(SRC_ROOT) not in sys.path: + sys.path.insert(0, str(SRC_ROOT)) + +from skill2workflow.lark_task_live_validation import main + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/src/skill2workflow/_controlled_lark_pilot_evidence_validation.py b/src/skill2workflow/_controlled_lark_pilot_evidence_validation.py new file mode 100644 index 0000000..161c9a1 --- /dev/null +++ b/src/skill2workflow/_controlled_lark_pilot_evidence_validation.py @@ -0,0 +1,413 @@ +"""Exact schema and cross-pack validation for controlled pilot evidence.""" + +from __future__ import annotations + +from datetime import date, datetime +from typing import Callable, Dict, List, Set + + +EVIDENCE_SCHEMA_VERSION = "controlled-lark-pilot-evidence-0.1.0" +EXERCISE_SCHEMA_VERSION = "controlled-lark-pilot-exercise-0.1.0" +VERIFICATION_SCHEMA_VERSION = "controlled-lark-pilot-verification-0.1.0" +DECISION_SCHEMA_VERSION = "controlled-lark-pilot-decision-0.1.0" +INDEX_SCHEMA_VERSION = "controlled-lark-pilot-index-0.1.0" +WORKFLOW_ID = "workflow_controlled_lark_pilot" +WORKFLOW_VERSION = "0.1.0" +CONNECTOR_ID = "lark_task" +CREDENTIAL_HANDLE = "lark_bot_access_token" +OPERATION = "create_task" +MODE = "live" +TIMEZONE = "Asia/Shanghai" + + +def _keys(names: str) -> Set[str]: + return set(names.split()) + + +RUN_EVIDENCE_KEYS = _keys( + "schema_version run_id workflow_id workflow_version started_at completed_at " + "run_status gate_decision case_id_present connector_invoked connector_id " + "connector_status credential_status credential_handles operation mode " + "provider_status task_title_present task_description_present assignee_present " + "due_at_present idempotency_key_present lark_task_id_present" +) +CHARTER_KEYS = _keys( + "schema_version scenario_id workflow_id workflow_version support_model timezone " + "starts_on expires_on team_consent_confirmed assignee_consent_confirmed " + "commercial_engagement_confirmed required_approved_runs required_distinct_days " + "required_distinct_cases" +) +TOP_LEVEL_KEYS = _keys("charter runs exercises verification decision index") +EXERCISE_SLOT_KEYS = _keys("rejection failure rollback") +REJECTION_EXERCISE_KEYS = _keys( + "schema_version exercise passed run_id gate_decision connector_invoked" +) +FAILURE_EXERCISE_KEYS = _keys( + "schema_version exercise passed provider_status credential_resolution_attempted " + "transport_attempted" +) +ROLLBACK_EXERCISE_KEYS = _keys( + "schema_version exercise passed live_switch_enabled live_approval_blocked " + "dry_run_status" +) +VERIFICATION_KEYS = _keys("schema_version all_passed commands") +VERIFICATION_COMMAND_KEYS = _keys("id exit_code passed duration_ms") +DECISION_KEYS = _keys( + "schema_version decision partner_acknowledged operator_acknowledged " + "commercial_engagement_confirmed rationale" +) +INDEX_KEYS = _keys( + "schema_version generated_at workflow_id workflow_version timezone " + "approved_live_runs required_approved_runs distinct_calendar_days " + "required_distinct_days distinct_private_cases required_distinct_cases " + "rejected_runs rejection_passed failure_passed rollback_passed " + "verification_passed decision_recorded decision partner_acknowledged " + "operator_acknowledged commercial_engagement_confirmed ready_to_finalize " + "unmet_conditions" +) +VERIFICATION_COMMAND_IDS = ( + "focused-tests", + "full-tests", + "compile", + "secret-hygiene", + "connector-smoke", + "dry-run-pilot-smoke", + "diff-check", +) +UNMET_CONDITIONS = ( + "approved_live_runs_threshold", + "distinct_calendar_days_threshold", + "distinct_private_cases_threshold", + "human_rejection", + "disabled_live_exercise", + "rollback_exercise", + "verification", + "decision", + "partner_acknowledgement", + "operator_acknowledgement", + "commercial_engagement_confirmation", +) +PROVIDER_STATUSES = { + "", + "authorization_failed", + "completed", + "credential_failed", + "idempotency_conflict", + "live_disabled", + "malformed_response", + "permission_denied", + "provider_unavailable", + "rate_limited", + "resource_not_found", + "timeout", + "validation_failed", +} +PRESENCE_FIELDS = ( + "task_title_present", + "task_description_present", + "assignee_present", + "due_at_present", + "idempotency_key_present", + "lark_task_id_present", +) + + +def aware_datetime(value: object, label: str, allow_empty: bool = False): + if allow_empty and value == "": + return None + if type(value) is not str: + raise ValueError(f"{label} must be a timestamp string") + try: + parsed = datetime.fromisoformat(value.replace("Z", "+00:00")) + except ValueError as error: + raise ValueError(f"{label} must be an ISO timestamp") from error + if parsed.tzinfo is None or parsed.utcoffset() is None: + raise ValueError(f"{label} must include a timezone") + return parsed + + +def _require_keys(value: object, keys: Set[str], label: str) -> Dict[str, object]: + if not isinstance(value, dict) or set(value) != keys: + raise ValueError(f"{label} keys do not match the allowlist") + return value + + +def _require_bool(value: object, label: str) -> None: + if type(value) is not bool: + raise ValueError(f"{label} must be a boolean") + + +def _require_nonnegative_int(value: object, label: str) -> None: + if type(value) is not int or value < 0: + raise ValueError(f"{label} must be a nonnegative integer") + + +def _validate_charter(charter: object) -> None: + value = _require_keys(charter, CHARTER_KEYS, "charter") + exact = { + "schema_version": "controlled-lark-pilot-0.1.0", + "scenario_id": "sales_renewal_risk_followup", + "workflow_id": WORKFLOW_ID, + "workflow_version": WORKFLOW_VERSION, + "support_model": "assisted", + "timezone": TIMEZONE, + "required_approved_runs": 5, + "required_distinct_days": 5, + "required_distinct_cases": 2, + } + if any(value.get(key) != expected for key, expected in exact.items()): + raise ValueError("charter fixed values are invalid") + for key in ( + "required_approved_runs required_distinct_days required_distinct_cases" + ).split(): + _require_nonnegative_int(value.get(key), f"charter {key}") + for key in ( + "team_consent_confirmed assignee_consent_confirmed " + "commercial_engagement_confirmed" + ).split(): + if value.get(key) is not True: + raise ValueError(f"charter {key} must be true") + if type(value.get("starts_on")) is not str or type(value.get("expires_on")) is not str: + raise ValueError("charter dates must be strings") + try: + starts = date.fromisoformat(value["starts_on"]) + expires = date.fromisoformat(value["expires_on"]) + except ValueError as error: + raise ValueError("charter dates must be ISO dates") from error + if starts > expires: + raise ValueError("charter date range is invalid") + + +def validate_run(run: object) -> None: + value = _require_keys(run, RUN_EVIDENCE_KEYS, "run evidence") + if value.get("schema_version") != EVIDENCE_SCHEMA_VERSION: + raise ValueError("run evidence schema is invalid") + for key in ("run_id", "workflow_id", "workflow_version", "run_status", "gate_decision"): + if type(value.get(key)) is not str: + raise ValueError(f"run evidence {key} must be a string") + if not value["run_id"].strip(): + raise ValueError("run evidence run_id must be nonempty") + if value["workflow_id"] != WORKFLOW_ID or value["workflow_version"] != WORKFLOW_VERSION: + raise ValueError("run evidence workflow identity is invalid") + if value["run_status"] not in ("waiting", "completed", "failed", "rejected"): + raise ValueError("run evidence status is invalid") + if value["gate_decision"] not in ("pending", "approved", "rejected"): + raise ValueError("run evidence gate decision is invalid") + started = aware_datetime(value.get("started_at"), "started_at") + completed = aware_datetime(value.get("completed_at"), "completed_at", allow_empty=True) + if value["run_status"] in ("completed", "failed", "rejected"): + if completed is None: + raise ValueError("completed_at is required for terminal run evidence") + if completed < started: + raise ValueError("completed_at must not precede started_at") + elif completed is not None: + raise ValueError("completed_at must be empty for nonterminal run evidence") + for key in ( + "case_id_present connector_invoked task_title_present " + "task_description_present assignee_present due_at_present " + "idempotency_key_present lark_task_id_present" + ).split(): + _require_bool(value.get(key), f"run evidence {key}") + if value.get("connector_id") not in ("", CONNECTOR_ID): + raise ValueError("run evidence connector identity is invalid") + if value.get("connector_status") not in ("", "running", "completed", "failed"): + raise ValueError("run evidence connector status is invalid") + if value.get("credential_status") not in ("", "resolved", "failed", "skipped"): + raise ValueError("run evidence credential status is invalid") + if value.get("credential_handles") not in ([], [CREDENTIAL_HANDLE]): + raise ValueError("run evidence credential handles are invalid") + if value.get("operation") not in ("", OPERATION) or value.get("mode") not in ("", MODE): + raise ValueError("run evidence connector binding is invalid") + if value.get("provider_status") not in PROVIDER_STATUSES: + raise ValueError("run evidence provider status is invalid") + if value["connector_invoked"] is False: + empty = all( + value[key] in ("", []) + for key in ( + "connector_id connector_status credential_status credential_handles " + "operation mode provider_status" + ).split() + ) and all(value[field] is False for field in PRESENCE_FIELDS) + if not empty: + raise ValueError("uninvoked connector evidence must be empty") + + +def _validate_exercise(name: str, exercise: object) -> None: + if exercise is None: + return + schemas = { + "rejection": REJECTION_EXERCISE_KEYS, + "failure": FAILURE_EXERCISE_KEYS, + "rollback": ROLLBACK_EXERCISE_KEYS, + } + value = _require_keys(exercise, schemas[name], f"{name} exercise") + if value.get("schema_version") != EXERCISE_SCHEMA_VERSION: + raise ValueError(f"{name} exercise schema is invalid") + _require_bool(value.get("passed"), f"{name} exercise passed") + if name == "rejection": + if ( + value.get("exercise") != "rejection" + or type(value.get("run_id")) is not str + or not value["run_id"].strip() + or value.get("gate_decision") != "rejected" + or value.get("connector_invoked") is not False + ): + raise ValueError("rejection exercise values are invalid") + if value["passed"] is not True: + raise ValueError("rejection exercise passed contradicts its facts") + elif name == "failure": + if value.get("exercise") != "disabled_live" or value.get("provider_status") not in PROVIDER_STATUSES: + raise ValueError("failure exercise values are invalid") + _require_bool(value.get("credential_resolution_attempted"), "failure credential attempt") + _require_bool(value.get("transport_attempted"), "failure transport attempt") + fact = bool( + value["provider_status"] == "live_disabled" + and value["credential_resolution_attempted"] is False + and value["transport_attempted"] is False + ) + if value["passed"] is not fact: + raise ValueError("failure exercise passed contradicts its facts") + else: + if value.get("exercise") != "rollback" or value.get("dry_run_status") not in ("", "completed", "failed"): + raise ValueError("rollback exercise values are invalid") + _require_bool(value.get("live_switch_enabled"), "rollback live switch") + _require_bool(value.get("live_approval_blocked"), "rollback approval") + fact = bool( + value["live_switch_enabled"] is False + and value["live_approval_blocked"] is True + and value["dry_run_status"] == "completed" + ) + if value["passed"] is not fact: + raise ValueError("rollback exercise passed contradicts its facts") + + +def _validate_verification(verification: object) -> None: + if verification is None: + return + value = _require_keys(verification, VERIFICATION_KEYS, "verification") + if value.get("schema_version") != VERIFICATION_SCHEMA_VERSION: + raise ValueError("verification schema is invalid") + _require_bool(value.get("all_passed"), "verification all_passed") + commands = value.get("commands") + if not isinstance(commands, list): + raise ValueError("verification commands must be a list") + seen = [] + for command in commands: + item = _require_keys(command, VERIFICATION_COMMAND_KEYS, "verification command") + if item.get("id") not in VERIFICATION_COMMAND_IDS: + raise ValueError("verification command identity is invalid") + seen.append(item["id"]) + _require_nonnegative_int(item.get("exit_code"), "verification exit code") + _require_nonnegative_int(item.get("duration_ms"), "verification duration") + _require_bool(item.get("passed"), "verification command passed") + if item["passed"] != (item["exit_code"] == 0): + raise ValueError("verification command result is inconsistent") + if tuple(seen) != VERIFICATION_COMMAND_IDS: + raise ValueError("verification must contain the exact seven commands in order") + if value["all_passed"] != all(command["passed"] for command in commands): + raise ValueError("verification aggregate is inconsistent") + + +def _validate_decision(decision: object) -> None: + if decision is None: + return + value = _require_keys(decision, DECISION_KEYS, "decision") + if value.get("schema_version") != DECISION_SCHEMA_VERSION: + raise ValueError("decision schema is invalid") + if value.get("decision") not in ("continue", "harden", "defer"): + raise ValueError("decision value is invalid") + for key in ( + "partner_acknowledged operator_acknowledged commercial_engagement_confirmed" + ).split(): + _require_bool(value.get(key), f"decision {key}") + if type(value.get("rationale")) is not str or not value["rationale"].strip(): + raise ValueError("decision rationale must be a nonempty string") + + +def _validate_index(index: object) -> None: + value = _require_keys(index, INDEX_KEYS, "evidence index") + if ( + value.get("schema_version") != INDEX_SCHEMA_VERSION + or value.get("workflow_id") != WORKFLOW_ID + or value.get("workflow_version") != WORKFLOW_VERSION + or value.get("timezone") != TIMEZONE + ): + raise ValueError("evidence index identity is invalid") + aware_datetime(value.get("generated_at"), "generated_at") + for key in ( + "approved_live_runs required_approved_runs distinct_calendar_days " + "required_distinct_days distinct_private_cases required_distinct_cases " + "rejected_runs" + ).split(): + _require_nonnegative_int(value.get(key), f"evidence index {key}") + for key in ( + "rejection_passed failure_passed rollback_passed verification_passed " + "decision_recorded partner_acknowledged operator_acknowledged " + "commercial_engagement_confirmed ready_to_finalize" + ).split(): + _require_bool(value.get(key), f"evidence index {key}") + if value.get("decision") not in ("", "continue", "harden", "defer"): + raise ValueError("evidence index decision is invalid") + unmet = value.get("unmet_conditions") + if not isinstance(unmet, list) or any(type(item) is not str for item in unmet): + raise ValueError("evidence index unmet conditions must be strings") + if unmet != [item for item in UNMET_CONDITIONS if item in unmet]: + raise ValueError("evidence index unmet conditions are invalid") + if value["ready_to_finalize"] != (unmet == []): + raise ValueError("evidence index readiness is inconsistent") + + +def _all_string_leaves(value: object) -> Set[str]: + leaves: Set[str] = set() + if isinstance(value, dict): + for item in value.values(): + leaves.update(_all_string_leaves(item)) + elif isinstance(value, list): + for item in value: + leaves.update(_all_string_leaves(item)) + elif isinstance(value, str): + leaves.add(value) + return leaves + + +def validate_evidence_pack( + pack: Dict[str, object], + forbidden_values: List[str], + run_sort_key: Callable[[Dict[str, object]], tuple], + rejection_exercise: Callable[[List[Dict[str, object]]], object], + build_summary: Callable[..., Dict[str, object]], +) -> None: + value = _require_keys(pack, TOP_LEVEL_KEYS, "evidence pack") + _validate_charter(value["charter"]) + if not isinstance(value["runs"], list): + raise ValueError("evidence pack runs must be a list") + for run in value["runs"]: + validate_run(run) + if value["runs"] != sorted(value["runs"], key=run_sort_key): + raise ValueError("evidence pack runs are not in stable order") + exercises = _require_keys(value["exercises"], EXERCISE_SLOT_KEYS, "exercises") + for name in ("rejection", "failure", "rollback"): + _validate_exercise(name, exercises[name]) + _validate_verification(value["verification"]) + _validate_decision(value["decision"]) + _validate_index(value["index"]) + if exercises["rejection"] != rejection_exercise(value["runs"]): + raise ValueError("rejection exercise does not match the first rejected run") + expected_summary = build_summary( + value["charter"], + value["runs"], + value["index"]["distinct_private_cases"], + exercises, + value["verification"], + value["decision"], + ) + if any(value["index"].get(key) != item for key, item in expected_summary.items()): + raise ValueError("evidence index does not match the acceptance summary") + if not isinstance(forbidden_values, list): + raise ValueError("forbidden values must be a list") + leaf_strings = _all_string_leaves(value) + for forbidden in forbidden_values: + if not isinstance(forbidden, str) or not forbidden: + continue + if any(forbidden in leaf for leaf in leaf_strings): + raise ValueError("evidence pack contains a forbidden private value") diff --git a/src/skill2workflow/_controlled_lark_pilot_evidence_writer.py b/src/skill2workflow/_controlled_lark_pilot_evidence_writer.py new file mode 100644 index 0000000..150f653 --- /dev/null +++ b/src/skill2workflow/_controlled_lark_pilot_evidence_writer.py @@ -0,0 +1,615 @@ +"""Secure low-level I/O and stable evidence-writer facade.""" + +from __future__ import annotations + +import inspect +import json +import os +import secrets +import stat +from pathlib import Path +from typing import Dict + +from ._controlled_lark_pilot_pack_transaction import ( + EvidencePackTransaction, + PackTransactionIO, +) + + +def _replace_supports_dir_fd() -> bool: + try: + parameters = inspect.signature(os.replace).parameters + except (TypeError, ValueError): + return False + return "src_dir_fd" in parameters and "dst_dir_fd" in parameters + + +_DIR_FD_SUPPORTED = bool( + os.name == "posix" + and hasattr(os, "O_DIRECTORY") + and hasattr(os, "O_NOFOLLOW") + and hasattr(os, "O_NONBLOCK") + and all( + function in os.supports_dir_fd + for function in ( + os.open, + os.mkdir, + os.stat, + os.unlink, + os.link, + os.rename, + os.rmdir, + ) + ) + and all( + function in os.supports_follow_symlinks + for function in (os.stat, os.link) + ) + and os.listdir in os.supports_fd + and _replace_supports_dir_fd() +) + + +def _require_dir_fd_support() -> None: + if not _DIR_FD_SUPPORTED: + raise ValueError("secure directory-fd evidence writes are not supported") + + +def _directory_flags() -> int: + return os.O_RDONLY | os.O_DIRECTORY | os.O_NOFOLLOW + + +def _canonicalize_root_alias(path: Path) -> Path: + """Resolve only a root-owned top-level alias such as macOS /var.""" + if len(path.parts) < 2: + return path + top_level = Path(path.anchor) / path.parts[1] + try: + top_level_item = os.stat(top_level, follow_symlinks=False) + except OSError: + return path + if not stat.S_ISLNK(top_level_item.st_mode): + return path + if hasattr(top_level_item, "st_uid") and top_level_item.st_uid != 0: + return path + resolved = top_level.resolve(strict=True) + return resolved.joinpath(*path.parts[2:]) + + +def _close_descriptors(*descriptors) -> None: + first_error = None + for descriptor in descriptors: + if descriptor is None: + continue + try: + os.close(descriptor) + except BaseException as error: + if first_error is None: + first_error = error + if first_error is not None: + raise first_error + + +def _open_child_directory(parent_fd: int, name: str, create: bool) -> int: + try: + return os.open(name, _directory_flags(), dir_fd=parent_fd) + except FileNotFoundError: + if not create: + raise + try: + os.mkdir(name, 0o700, dir_fd=parent_fd) + except FileExistsError: + pass + try: + return os.open(name, _directory_flags(), dir_fd=parent_fd) + except OSError as error: + raise ValueError( + "evidence output component must not be a symbolic link or non-directory" + ) from error + except OSError as error: + raise ValueError( + "evidence output component must not be a symbolic link or non-directory" + ) from error + + +def _open_relative_directory(root_fd: int, components: tuple, create: bool) -> int: + descriptor = os.dup(root_fd) + try: + for component in components: + child = _open_child_directory(descriptor, component, create=create) + os.close(descriptor) + descriptor = child + return descriptor + except BaseException: + os.close(descriptor) + raise + + +def _read_json_at(parent_fd: int, name: str, *, owner_only: bool = False): + file_descriptor = None + try: + try: + file_descriptor = os.open( + name, + os.O_RDONLY | os.O_NONBLOCK | os.O_NOFOLLOW, + dir_fd=parent_fd, + ) + except FileNotFoundError: + raise + except OSError as error: + raise ValueError( + "anchored JSON file must not be a symbolic link or non-regular file" + ) from error + item = os.fstat(file_descriptor) + if not stat.S_ISREG(item.st_mode): + raise ValueError("anchored JSON file must be a regular file") + if owner_only and os.name == "posix" and item.st_mode & 0o077: + raise ValueError("private authorization JSON must use owner-only permissions") + handle = os.fdopen(file_descriptor, "r", encoding="utf-8") + file_descriptor = None + with handle: + return json.load(handle) + finally: + _close_descriptors(file_descriptor) + + +def read_json_anchored(path: Path, *, owner_only: bool = False): + _require_dir_fd_support() + absolute = _canonicalize_root_alias( + Path(os.path.abspath(os.fspath(path))) + ) + if absolute == Path(absolute.anchor): + raise ValueError("anchored JSON path must not be a filesystem root") + root_fd = os.open(absolute.anchor, _directory_flags()) + parent_fd = None + try: + parent_fd = _open_relative_directory( + root_fd, + absolute.parts[1:-1], + create=False, + ) + return _read_json_at( + parent_fd, + absolute.name, + owner_only=owner_only, + ) + finally: + _close_descriptors(parent_fd, root_fd) + + +def _require_declared_directory_identity( + root_fd: int, + output: Path, + anchored_output_fd: int, + label: str, +) -> None: + observed = None + try: + try: + observed = _open_relative_directory( + root_fd, + output.parts[1:], + create=False, + ) + except (FileNotFoundError, OSError, ValueError) as error: + raise ValueError(f"declared {label} path changed during write") from error + expected_stat = os.fstat(anchored_output_fd) + observed_stat = os.fstat(observed) + if (expected_stat.st_dev, expected_stat.st_ino) != ( + observed_stat.st_dev, + observed_stat.st_ino, + ): + raise ValueError(f"declared {label} path changed during write") + finally: + if observed is not None: + os.close(observed) + + +def _private_target_stat(parent_fd: int, name: str): + try: + item = os.stat(name, dir_fd=parent_fd, follow_symlinks=False) + except FileNotFoundError: + return None + if stat.S_ISLNK(item.st_mode): + raise ValueError("private JSON target must not be a symbolic link") + if not stat.S_ISREG(item.st_mode): + raise ValueError("private JSON target must be a regular file") + return item + + +def _same_entry(first, second) -> bool: + if first is None or second is None: + return first is second + return (first.st_dev, first.st_ino, first.st_mode) == ( + second.st_dev, + second.st_ino, + second.st_mode, + ) + + +def _open_private_parent(path: Path, create: bool) -> tuple: + _require_dir_fd_support() + absolute = _canonicalize_root_alias( + Path(os.path.abspath(os.fspath(path))) + ) + if absolute == Path(absolute.anchor) or not absolute.name: + raise ValueError("private JSON path must not be a filesystem root") + root_fd = os.open(absolute.anchor, _directory_flags()) + parent_fd = None + try: + parent_fd = _open_relative_directory( + root_fd, + absolute.parts[1:-1], + create=create, + ) + parent_mode = os.fstat(parent_fd).st_mode + if not stat.S_ISDIR(parent_mode): + raise ValueError("private JSON parent must be a directory") + if os.name == "posix" and parent_mode & 0o077: + raise ValueError("private JSON parent must use owner-only permissions") + return absolute, root_fd, parent_fd + except BaseException: + _close_descriptors(parent_fd, root_fd) + raise + + +def ensure_private_directory_anchored(path: Path) -> None: + """Create one owner-only directory through no-follow directory descriptors.""" + _require_dir_fd_support() + absolute = _canonicalize_root_alias( + Path(os.path.abspath(os.fspath(path))) + ) + if absolute == Path(absolute.anchor): + raise ValueError("private directory must not be a filesystem root") + root_fd = os.open(absolute.anchor, _directory_flags()) + directory_fd = None + try: + directory_fd = _open_relative_directory( + root_fd, + absolute.parts[1:], + create=True, + ) + item = os.fstat(directory_fd) + if not stat.S_ISDIR(item.st_mode): + raise ValueError("private workspace node must be a directory") + if os.name == "posix": + os.fchmod(directory_fd, 0o700) + _require_declared_directory_identity( + root_fd, + absolute, + directory_fd, + "private", + ) + finally: + _close_descriptors(directory_fd, root_fd) + + +def require_private_json_target(path: Path) -> None: + """Fail closed unless a private JSON target is missing or a regular file.""" + absolute, root_fd, parent_fd = _open_private_parent(path, create=False) + try: + _private_target_stat(parent_fd, absolute.name) + _require_declared_directory_identity( + root_fd, + absolute.parent, + parent_fd, + "private", + ) + finally: + _close_descriptors(parent_fd, root_fd) + + +def invalidate_private_json_anchored(path: Path) -> None: + """Atomically remove a stale private JSON result through anchored descriptors.""" + absolute, root_fd, parent_fd = _open_private_parent(path, create=True) + transaction_name = "" + transaction_fd = None + try: + initial = _private_target_stat(parent_fd, absolute.name) + if initial is not None: + transaction_name, transaction_fd = _allocate_transaction_directory( + parent_fd, + f"{absolute.name}-invalidation", + ) + try: + os.rename( + absolute.name, + "stale", + src_dir_fd=parent_fd, + dst_dir_fd=transaction_fd, + ) + except FileNotFoundError as error: + raise ValueError( + "private JSON target changed during invalidation" + ) from error + moved = os.stat( + "stale", + dir_fd=transaction_fd, + follow_symlinks=False, + ) + if not _same_entry(initial, moved): + raise ValueError("private JSON target changed during invalidation") + os.fsync(parent_fd) + if _private_target_stat(parent_fd, absolute.name) is not None: + raise ValueError("private JSON target changed during invalidation") + _require_declared_directory_identity( + root_fd, + absolute.parent, + parent_fd, + "private", + ) + finally: + first_error = None + try: + _close_descriptors(transaction_fd) + except BaseException as error: + first_error = error + if transaction_name: + try: + _remove_tree_at(parent_fd, transaction_name) + os.fsync(parent_fd) + except BaseException as error: + if first_error is None: + first_error = error + try: + _close_descriptors(parent_fd, root_fd) + except BaseException as error: + if first_error is None: + first_error = error + if first_error is not None: + raise first_error + + +def _write_private_json_at( + parent_fd: int, + name: str, + value: object, + *, + require_missing: bool = False, +) -> None: + descriptor = None + temporary = "" + linked = False + completed = False + try: + initial = _private_target_stat(parent_fd, name) + if require_missing and initial is not None: + raise ValueError("private JSON target must not already exist") + flags = os.O_WRONLY | os.O_CREAT | os.O_EXCL | os.O_NOFOLLOW + for _attempt in range(16): + temporary = f".{name}.{secrets.token_hex(8)}.tmp" + try: + descriptor = os.open( + temporary, + flags, + 0o600, + dir_fd=parent_fd, + ) + break + except FileExistsError: + continue + if descriptor is None: + raise FileExistsError("could not allocate a private JSON temporary file") + os.fchmod(descriptor, 0o600) + with os.fdopen(descriptor, "w", encoding="utf-8") as handle: + descriptor = None + json.dump(value, handle, ensure_ascii=False, indent=2) + handle.flush() + os.fsync(handle.fileno()) + + current = _private_target_stat(parent_fd, name) + if not _same_entry(initial, current): + raise ValueError("private JSON target changed during write") + if require_missing: + os.link( + temporary, + name, + src_dir_fd=parent_fd, + dst_dir_fd=parent_fd, + follow_symlinks=False, + ) + linked = True + os.unlink(temporary, dir_fd=parent_fd) + else: + os.replace( + temporary, + name, + src_dir_fd=parent_fd, + dst_dir_fd=parent_fd, + ) + temporary = "" + final = _private_target_stat(parent_fd, name) + if final is None or (os.name == "posix" and final.st_mode & 0o077): + raise ValueError("private JSON target must use owner-only permissions") + os.fsync(parent_fd) + completed = True + finally: + try: + if descriptor is not None: + os.close(descriptor) + finally: + if temporary: + try: + os.unlink(temporary, dir_fd=parent_fd) + except FileNotFoundError: + pass + if require_missing and linked and not completed: + try: + os.unlink(name, dir_fd=parent_fd) + os.fsync(parent_fd) + except FileNotFoundError: + pass + + +def write_private_json_anchored( + path: Path, + value: object, + *, + require_missing: bool = False, +) -> None: + """Atomically replace owner-only JSON through anchored no-follow descriptors.""" + absolute, root_fd, parent_fd = _open_private_parent(path, create=True) + published = False + completed = False + try: + _write_private_json_at( + parent_fd, + absolute.name, + value, + require_missing=require_missing, + ) + published = True + _require_declared_directory_identity( + root_fd, + absolute.parent, + parent_fd, + "private", + ) + completed = True + finally: + try: + if require_missing and published and not completed: + try: + os.unlink(absolute.name, dir_fd=parent_fd) + os.fsync(parent_fd) + except FileNotFoundError: + pass + finally: + _close_descriptors(parent_fd, root_fd) + + +def _write_json_atomic(parent_fd: int, name: str, value: object) -> None: + flags = os.O_WRONLY | os.O_CREAT | os.O_EXCL | os.O_NOFOLLOW + descriptor = None + temporary = "" + try: + for _attempt in range(16): + temporary = f".{name}.{secrets.token_hex(8)}.tmp" + try: + descriptor = os.open( + temporary, + flags, + 0o600, + dir_fd=parent_fd, + ) + break + except FileExistsError: + continue + if descriptor is None: + raise FileExistsError("could not allocate an evidence temporary file") + with os.fdopen(descriptor, "w", encoding="utf-8") as handle: + descriptor = None + json.dump(value, handle, ensure_ascii=False, indent=2) + handle.flush() + os.fsync(handle.fileno()) + os.replace( + temporary, + name, + src_dir_fd=parent_fd, + dst_dir_fd=parent_fd, + ) + temporary = "" + os.fsync(parent_fd) + finally: + try: + if descriptor is not None: + os.close(descriptor) + finally: + if temporary: + try: + os.unlink(temporary, dir_fd=parent_fd) + except FileNotFoundError: + pass + + +def _remove_tree_at(parent_fd: int, name: str) -> None: + try: + item = os.stat(name, dir_fd=parent_fd, follow_symlinks=False) + except FileNotFoundError: + return + if stat.S_ISDIR(item.st_mode) and not stat.S_ISLNK(item.st_mode): + directory_fd = os.open(name, _directory_flags(), dir_fd=parent_fd) + try: + for child in os.listdir(directory_fd): + _remove_tree_at(directory_fd, child) + finally: + os.close(directory_fd) + os.rmdir(name, dir_fd=parent_fd) + else: + os.unlink(name, dir_fd=parent_fd) + + +def _allocate_transaction_directory(parent_fd: int, label: str) -> tuple: + for _attempt in range(16): + name = f".{label}.{secrets.token_hex(8)}.txn" + try: + os.mkdir(name, 0o700, dir_fd=parent_fd) + except FileExistsError: + continue + descriptor = os.open(name, _directory_flags(), dir_fd=parent_fd) + if os.fstat(descriptor).st_mode & 0o077: + _close_descriptors(descriptor) + _remove_tree_at(parent_fd, name) + raise ValueError("transaction directory must be owner-only") + return name, descriptor + raise FileExistsError("could not allocate a transaction directory") + + +def _pack_io() -> PackTransactionIO: + return PackTransactionIO( + require_dir_fd_support=_require_dir_fd_support, + directory_flags=_directory_flags, + close_descriptors=_close_descriptors, + open_relative_directory=_open_relative_directory, + require_declared_directory_identity=_require_declared_directory_identity, + same_entry=_same_entry, + write_json_atomic=_write_json_atomic, + remove_tree_at=_remove_tree_at, + allocate_transaction_directory=_allocate_transaction_directory, + ) + + +def prepare_evidence_pack( + output_dir: Path, + pack: Dict[str, object], +) -> EvidencePackTransaction: + return EvidencePackTransaction(_pack_io(), output_dir, pack) + + +def finish_durable_resources(*resources) -> None: + """Best-effort cleanup after the caller's durable commit point.""" + pending = list(resources) + for _attempt in range(2): + retry = [] + for resource, method_name in pending: + try: + getattr(resource, method_name)() + except Exception: + retry.append((resource, method_name)) + pending = retry + if not pending: + return + for resource, _method_name in pending: + isolate = getattr(resource, "isolate_cleanup_failure", None) + if isolate is None: + continue + try: + isolate() + except Exception: + pass + + +def write_evidence_pack(output_dir: Path, pack: Dict[str, object]) -> Dict[str, object]: + transaction = prepare_evidence_pack(output_dir, pack) + try: + transaction.commit() + except BaseException: + transaction.abort() + raise + result = { + "status": "written", + "file_count": transaction.file_count, + "output_dir": str(transaction.output), + } + finish_durable_resources((transaction, "finish")) + return result diff --git a/src/skill2workflow/_controlled_lark_pilot_operations.py b/src/skill2workflow/_controlled_lark_pilot_operations.py new file mode 100644 index 0000000..0c36323 --- /dev/null +++ b/src/skill2workflow/_controlled_lark_pilot_operations.py @@ -0,0 +1,567 @@ +"""Fail-closed local operations for the controlled Lark pilot facade.""" + +from __future__ import annotations + +import json +import os +import subprocess +import sys +import tempfile +import time +from collections.abc import Mapping +from contextlib import contextmanager +from dataclasses import dataclass +from datetime import datetime, timezone +from pathlib import Path +from typing import Callable, Dict +from zoneinfo import ZoneInfo + +from ._controlled_lark_pilot_evidence_validation import ( + DECISION_KEYS, + DECISION_SCHEMA_VERSION, + EXERCISE_SCHEMA_VERSION, + VERIFICATION_COMMAND_IDS, + VERIFICATION_SCHEMA_VERSION, +) +from ._controlled_lark_pilot_evidence_writer import ( + finish_durable_resources, + invalidate_private_json_anchored, + write_private_json_anchored, +) + + +LIVE_SWITCH = "SKILL2WORKFLOW_LARK_TASK_LIVE" +TOKEN_ENVIRONMENT = "LARK_BOT_ACCESS_TOKEN" +APP_ID_ENVIRONMENT = "LARK_APP_ID" +APP_SECRET_ENVIRONMENT = "LARK_APP_SECRET" +FINALIZATION_SCHEMA_VERSION = "controlled-lark-pilot-finalization-0.1.0" +FINALIZATION_KEYS = {"schema_version", "finalized", "decision", "finalized_at"} + + +@dataclass(frozen=True) +class Task6Dependencies: + require_outside_repository: Callable + load_charter: Callable + initialize: Callable + start: Callable + decide: Callable + control_plane: Callable + build_evidence: Callable + evidence_output: Callable + prepare_evidence_pack: Callable + open_private_session: Callable + finalization_bundle: Callable + pilot_timezone: str + + +@contextmanager +def live_environment_removed(): + """Remove only the live switch without accessing the injected token.""" + existed = LIVE_SWITCH in os.environ + previous = os.environ.get(LIVE_SWITCH, "") + os.environ.pop(LIVE_SWITCH, None) + try: + yield + finally: + if existed: + os.environ[LIVE_SWITCH] = previous + else: + os.environ.pop(LIVE_SWITCH, None) + + +class _SanitizedEnvironment(Mapping): + """Read safe environment values lazily without touching excluded values.""" + + def __init__(self, source, *, excluded, overrides): + excluded_keys = frozenset(excluded) + self._source = source + self._overrides = dict(overrides) + self._source_keys = tuple( + key + for key in source + if key not in excluded_keys and key not in self._overrides + ) + self._source_key_set = frozenset(self._source_keys) + + def __getitem__(self, key): + if key in self._overrides: + return self._overrides[key] + if key not in self._source_key_set: + raise KeyError(key) + return self._source[key] + + def __iter__(self): + yield from self._source_keys + yield from self._overrides + + def __len__(self): + return len(self._source_keys) + len(self._overrides) + + +class CredentialResolutionSpy: + def __init__(self): + self.calls = [] + + def resolve(self, handle: str) -> str: + self.calls.append(handle) + raise AssertionError("disabled-live exercise attempted credential resolution") + + +class TransportSpy: + def __init__(self): + self.calls = [] + + def __call__(self, request, timeout): + self.calls.append((request, timeout)) + raise AssertionError("safe exercise attempted provider transport") + + +def persist_exercise(work_dir: Path, name: str, result: Dict[str, object]) -> None: + artifact = { + "schema_version": EXERCISE_SCHEMA_VERSION, + **result, + } + write_private_json_anchored( + Path(work_dir) / "private" / "exercises" / f"{name}.json", + artifact, + ) + + +def exercise_disabled_live_operation( + repo_root: Path, + work_dir: Path, + now: datetime, + dependencies: Task6Dependencies, +) -> Dict[str, object]: + repo_root = Path(repo_root).resolve() + work_dir = Path(work_dir).resolve() + dependencies.require_outside_repository( + repo_root, + work_dir, + "pilot work directory", + ) + charter = dependencies.load_charter(work_dir, now=now) + provider_status = "" + credentials = CredentialResolutionSpy() + transport = TransportSpy() + with live_environment_removed(): + with tempfile.TemporaryDirectory( + dir=str(work_dir / "private"), + prefix=".disabled-live-exercise-", + ) as temporary: + exercise_work = Path(temporary) / "pilot" + case_path = Path(temporary) / "case.json" + dependencies.initialize(repo_root, exercise_work, charter, now=now) + write_private_json_anchored( + case_path, + { + "pilot_case_id": "exercise-disabled-001", + "account_name": "Disabled Live Exercise Account", + "renewal_risk": "Disabled Live Exercise Risk", + "owner_open_id": "ou_disabled_live_exercise", + "due_at": "2026-08-15T09:00:00Z", + }, + ) + started = dependencies.start( + repo_root, + exercise_work, + case_path, + now=now, + ) + control = dependencies.control_plane( + repo_root, + exercise_work, + credential_provider=credentials, + transport=transport, + ) + control.resume_published_run(started["run_id"], approved=True) + events = control.list_audit_events(run_id=started["run_id"]) + for event in reversed(events): + if ( + isinstance(event, dict) + and event.get("node_id") == "create_lark_task" + and isinstance(event.get("connector_metadata"), dict) + ): + provider_status = str( + event["connector_metadata"].get("provider_status", "") + ) + break + result = { + "exercise": "disabled_live", + "passed": ( + provider_status == "live_disabled" + and not credentials.calls + and not transport.calls + ), + "provider_status": provider_status, + "credential_resolution_attempted": bool(credentials.calls), + "transport_attempted": bool(transport.calls), + } + persist_exercise(work_dir, "failure", result) + return result + + +def exercise_rollback_operation( + repo_root: Path, + work_dir: Path, + now: datetime, + dependencies: Task6Dependencies, +) -> Dict[str, object]: + repo_root = Path(repo_root).resolve() + work_dir = Path(work_dir).resolve() + dependencies.require_outside_repository( + repo_root, + work_dir, + "pilot work directory", + ) + if os.environ.get(LIVE_SWITCH) == "1": + raise ValueError("remove the live switch before running rollback exercise") + charter = dependencies.load_charter(work_dir, now=now) + transport = TransportSpy() + with live_environment_removed(): + with tempfile.TemporaryDirectory( + dir=str(work_dir / "private"), + prefix=".rollback-live-probe-", + ) as temporary: + probe_work = Path(temporary) / "pilot" + dependencies.initialize(repo_root, probe_work, charter, now=now) + case_path = probe_work / "private" / "rollback-case.json" + write_private_json_anchored( + case_path, + { + "pilot_case_id": "rollback-proof-001", + "account_name": "Rollback Exercise Account", + "renewal_risk": "Rollback Exercise Risk", + "owner_open_id": "ou_rollback_exercise", + "due_at": "2026-08-15T09:00:00Z", + }, + ) + started = dependencies.start( + repo_root, + probe_work, + case_path, + now=now, + ) + live_approval_blocked = False + try: + dependencies.decide( + repo_root, + probe_work, + started["run_id"], + approved=True, + confirmed_live=True, + now=now, + transport=transport, + ) + except ValueError as error: + live_approval_blocked = ( + str(error) == "SKILL2WORKFLOW_LARK_TASK_LIVE=1 is required" + ) + if not live_approval_blocked or transport.calls: + raise ValueError("rollback did not prove the disabled live boundary") + + from .lark_task_pilot import run_lark_task_pilot + + dry_run = run_lark_task_pilot( + repo_root=repo_root, + work_dir=work_dir / "private" / "rollback-dry-run", + reset=True, + ) + result = { + "exercise": "rollback", + "passed": ( + live_approval_blocked + and dry_run.get("run_status") == "completed" + ), + "live_switch_enabled": os.environ.get(LIVE_SWITCH) == "1", + "live_approval_blocked": live_approval_blocked, + "dry_run_status": str(dry_run.get("run_status", "")), + } + persist_exercise(work_dir, "rollback", result) + return result + + +def verify_pilot_operation( + repo_root: Path, + work_dir: Path, + command_runner, + dependencies: Task6Dependencies, +) -> Dict[str, object]: + repo_root = Path(repo_root).resolve() + work_dir = Path(work_dir).resolve() + dependencies.require_outside_repository( + repo_root, + work_dir, + "pilot work directory", + ) + return run_fixed_verification(repo_root, work_dir, command_runner) + + +def finalize_pilot_operation( + repo_root: Path, + work_dir: Path, + decision: Dict[str, object], + output_dir: Path, + now: datetime, + dependencies: Task6Dependencies, +) -> Dict[str, object]: + repo_root = Path(repo_root).resolve() + work_dir = Path(work_dir).resolve() + dependencies.require_outside_repository( + repo_root, + work_dir, + "pilot work directory", + ) + output, _repository_export = dependencies.evidence_output( + repo_root, + work_dir, + output_dir, + ) + normalized_decision = validate_final_decision(decision) + finalized = now or datetime.now(timezone.utc) + if finalized.tzinfo is None or finalized.utcoffset() is None: + raise ValueError("pilot finalization time must include a timezone") + + private_output = work_dir / "evidence" + marker = { + "schema_version": FINALIZATION_SCHEMA_VERSION, + "finalized": True, + "decision": normalized_decision["decision"], + "finalized_at": finalized.astimezone( + ZoneInfo(dependencies.pilot_timezone) + ).isoformat(), + } + transactions = [] + private_session = dependencies.open_private_session(work_dir / "private") + bundle = None + authorization_snapshot = None + durable_success = False + try: + bundle = dependencies.finalization_bundle(private_session) + pack = dependencies.build_evidence( + repo_root, + work_dir, + decision_override=normalized_decision, + now=now, + private_session=private_session, + ) + private_session.check_identity() + index = pack.get("index") + if not isinstance(index, dict): + raise ValueError("pilot evidence index is invalid") + if index.get("ready_to_finalize") is not True: + unmet = index.get("unmet_conditions") + if not isinstance(unmet, list) or any( + type(item) is not str for item in unmet + ): + raise ValueError("pilot evidence unmet conditions are invalid") + raise ValueError( + "pilot evidence is not ready to finalize: " + ", ".join(unmet) + ) + + transactions.append( + dependencies.prepare_evidence_pack(private_output, pack) + ) + if output != private_output: + transactions.append( + dependencies.prepare_evidence_pack(output, pack) + ) + private_session.check_identity() + for transaction in transactions: + transaction.commit() + private_session.check_identity() + + bundle.publish_decision(normalized_decision) + authorization_snapshot = bundle.publish_marker(marker) + if ( + authorization_snapshot.decision != normalized_decision + or authorization_snapshot.marker != marker + ): + raise ValueError("private finalization bundle verification failed") + authorization_snapshot.validate() + for transaction in transactions: + transaction.validate_durable_commit() + private_session.check_identity() + authorization_snapshot.validate() + + # Irreversible durable-success commit point: authorization, packs, + # private directory, and lock identities have all been revalidated. + durable_success = True + except BaseException: + rollback_errors = [] + if authorization_snapshot is not None: + try: + authorization_snapshot.close() + except BaseException as rollback_error: + rollback_errors.append(rollback_error) + if bundle is not None: + try: + bundle.rollback() + except BaseException as rollback_error: + rollback_errors.append(rollback_error) + for transaction in reversed(transactions): + try: + transaction.abort() + except BaseException as rollback_error: + rollback_errors.append(rollback_error) + try: + private_session.close() + except BaseException as rollback_error: + rollback_errors.append(rollback_error) + if rollback_errors: + raise RuntimeError( + "controlled pilot finalization rollback failed" + ) from rollback_errors[0] + raise + if durable_success: + resources = [] + if authorization_snapshot is not None: + resources.append((authorization_snapshot, "close")) + if bundle is not None: + resources.append((bundle, "finish")) + resources.extend( + (transaction, "finish") for transaction in transactions + ) + resources.append((private_session, "close")) + finish_durable_resources(*resources) + return { + "status": "finalized", + "decision": normalized_decision["decision"], + "approved_live_runs": index["approved_live_runs"], + "distinct_calendar_days": index["distinct_calendar_days"], + "distinct_private_cases": index["distinct_private_cases"], + "rejected_runs": index["rejected_runs"], + "output_dir": str(output), + } + + +def validate_final_decision(decision: Dict[str, object]) -> Dict[str, object]: + if type(decision) is not dict or set(decision) != DECISION_KEYS: + raise ValueError("pilot decision keys do not match the allowlist") + if decision.get("schema_version") != DECISION_SCHEMA_VERSION: + raise ValueError("pilot decision schema is invalid") + if decision.get("decision") not in ("continue", "harden", "defer"): + raise ValueError("pilot decision value is invalid") + for key in ( + "partner_acknowledged", + "operator_acknowledged", + "commercial_engagement_confirmed", + ): + if decision.get(key) is not True: + raise ValueError(f"pilot decision {key} must be true") + rationale = decision.get("rationale") + if type(rationale) is not str or not rationale.strip(): + raise ValueError("pilot decision rationale must be a nonempty string") + return json.loads(json.dumps(decision, ensure_ascii=False)) + + +def run_fixed_verification( + repo_root: Path, + work_dir: Path, + command_runner=None, +) -> Dict[str, object]: + python = sys.executable + source_files = sorted( + str(path.relative_to(repo_root)) + for path in (repo_root / "src" / "skill2workflow").glob("*.py") + ) + commands = [ + ( + "focused-tests", + [ + python, + "-m", + "unittest", + "tests.test_controlled_lark_pilot", + "tests.test_controlled_lark_pilot_evidence", + "tests.test_controlled_lark_pilot_docs", + "-v", + ], + ), + ( + "full-tests", + [python, "-m", "unittest", "discover", "-s", "tests", "-v"], + ), + ( + "compile", + [ + python, + "-m", + "py_compile", + *source_files, + "examples/connectors/lark_task_connector.py", + ], + ), + ( + "secret-hygiene", + [python, "scripts/secret_hygiene.py", "examples/workflows"], + ), + ( + "connector-smoke", + [ + python, + "scripts/lark_task_connector_smoke.py", + "--work-dir", + str(work_dir / "private" / "connector-smoke"), + ], + ), + ( + "dry-run-pilot-smoke", + [ + python, + "scripts/lark_task_pilot_smoke.py", + "--work-dir", + str(work_dir / "private" / "dry-run-smoke"), + ], + ), + ("diff-check", ["git", "diff", "--check"]), + ] + if tuple(command_id for command_id, _arguments in commands) != VERIFICATION_COMMAND_IDS: + raise ValueError("fixed verification command identity is invalid") + + environment = _SanitizedEnvironment( + os.environ, + excluded=( + LIVE_SWITCH, + TOKEN_ENVIRONMENT, + APP_ID_ENVIRONMENT, + APP_SECRET_ENVIRONMENT, + ), + overrides={"PYTHONPATH": "src"}, + ) + runner = command_runner or subprocess.run + records = [] + verification_path = work_dir / "private" / "verification.json" + invalidate_private_json_anchored(verification_path) + for command_id, arguments in commands: + started = time.monotonic_ns() + completed = runner( + arguments, + cwd=repo_root, + env=environment, + capture_output=True, + ) + duration_ms = max(0, (time.monotonic_ns() - started) // 1_000_000) + exit_code = getattr(completed, "returncode", None) + if type(exit_code) is not int: + raise ValueError("verification runner returned an invalid exit code") + if exit_code < 0: + exit_code = 128 + abs(exit_code) + records.append( + { + "id": command_id, + "exit_code": exit_code, + "passed": exit_code == 0, + "duration_ms": duration_ms, + } + ) + result = { + "schema_version": VERIFICATION_SCHEMA_VERSION, + "all_passed": all(item["passed"] for item in records), + "commands": records, + } + write_private_json_anchored( + verification_path, + result, + ) + return result diff --git a/src/skill2workflow/_controlled_lark_pilot_pack_transaction.py b/src/skill2workflow/_controlled_lark_pilot_pack_transaction.py new file mode 100644 index 0000000..651bd58 --- /dev/null +++ b/src/skill2workflow/_controlled_lark_pilot_pack_transaction.py @@ -0,0 +1,409 @@ +"""Whole-directory evidence pack staging, commit, and rollback.""" + +from __future__ import annotations + +import os +import stat +from dataclasses import dataclass +from pathlib import Path +from typing import Callable, Dict + + +@dataclass(frozen=True) +class PackTransactionIO: + require_dir_fd_support: Callable + directory_flags: Callable + close_descriptors: Callable + open_relative_directory: Callable + require_declared_directory_identity: Callable + same_entry: Callable + write_json_atomic: Callable + remove_tree_at: Callable + allocate_transaction_directory: Callable + + +def _validate_directory_tree( + io: PackTransactionIO, + directory_fd: int, +) -> None: + for name in os.listdir(directory_fd): + item = os.stat(name, dir_fd=directory_fd, follow_symlinks=False) + if stat.S_ISLNK(item.st_mode): + raise ValueError("evidence output descendants must not be symbolic links") + if stat.S_ISDIR(item.st_mode): + child = _open_directory_at(io, directory_fd, name) + try: + _validate_directory_tree(io, child) + finally: + os.close(child) + continue + if not stat.S_ISREG(item.st_mode): + raise ValueError("evidence output descendants must be regular files") + + +def _pack_files(pack: Dict[str, object]) -> Dict[tuple, object]: + files = { + ((), "pilot-charter.json"): pack["charter"], + ((), "evidence-index.json"): pack["index"], + } + for sequence, run in enumerate(pack["runs"], start=1): + files[(("runs",), f"{sequence:03d}.json")] = run + for name in ("rejection", "failure", "rollback"): + exercise = pack["exercises"][name] + if exercise is not None: + files[(("exercises",), f"{name}.json")] = exercise + if pack["verification"] is not None: + files[((), "verification.json")] = pack["verification"] + if pack["decision"] is not None: + files[((), "decision.json")] = pack["decision"] + return files + + +def _open_output_parent(io: PackTransactionIO, output_dir: Path) -> tuple: + io.require_dir_fd_support() + output = Path(os.path.abspath(os.fspath(output_dir))) + if output == Path(output.anchor) or not output.name: + raise ValueError("evidence output must not be a filesystem root") + root_fd = os.open(output.anchor, io.directory_flags()) + parent_fd = None + try: + parent_fd = io.open_relative_directory( + root_fd, + output.parts[1:-1], + create=True, + ) + return output, root_fd, parent_fd + except BaseException: + io.close_descriptors(parent_fd, root_fd) + raise + + +def _output_target_stat(parent_fd: int, name: str): + try: + item = os.stat(name, dir_fd=parent_fd, follow_symlinks=False) + except FileNotFoundError: + return None + if stat.S_ISLNK(item.st_mode): + raise ValueError("evidence output must not be a symbolic link") + if not stat.S_ISDIR(item.st_mode): + raise ValueError("evidence output must be a directory") + return item + + +def _open_directory_at( + io: PackTransactionIO, + parent_fd: int, + name: str, +) -> int: + try: + return os.open(name, io.directory_flags(), dir_fd=parent_fd) + except OSError as error: + raise ValueError("evidence output must remain a directory") from error + + +class EvidencePackTransaction: + """Prepare and atomically exchange one complete evidence directory.""" + + def __init__( + self, + io: PackTransactionIO, + output_dir: Path, + pack: Dict[str, object], + ): + self.io = io + self.output, self.root_fd, self.parent_fd = _open_output_parent( + io, + output_dir, + ) + self.output_fd = None + self.transaction_fd = None + self.stage_fd = None + self.reservation_fd = None + self.transaction_name = "" + self.initial = None + self.published = None + self.committed = False + self.closed = False + self.file_count = 0 + try: + self.initial = _output_target_stat(self.parent_fd, self.output.name) + if self.initial is not None: + self.output_fd = _open_directory_at( + io, + self.parent_fd, + self.output.name, + ) + if not io.same_entry(self.initial, os.fstat(self.output_fd)): + raise ValueError("evidence output changed during transaction setup") + _validate_directory_tree(io, self.output_fd) + self.transaction_name, self.transaction_fd = ( + io.allocate_transaction_directory( + self.parent_fd, + self.output.name, + ) + ) + os.mkdir("stage", 0o700, dir_fd=self.transaction_fd) + self.stage_fd = _open_directory_at(io, self.transaction_fd, "stage") + files = _pack_files(pack) + for (components, name), item in files.items(): + parent_fd = io.open_relative_directory( + self.stage_fd, + components, + create=True, + ) + try: + io.write_json_atomic(parent_fd, name, item) + finally: + os.close(parent_fd) + os.fsync(self.stage_fd) + self.file_count = len(files) + self._require_initial_identity() + except BaseException: + self._cleanup_uncommitted() + raise + + def _require_parent_identity(self) -> None: + self.io.require_declared_directory_identity( + self.root_fd, + self.output.parent, + self.parent_fd, + "output parent", + ) + + def _require_initial_identity(self) -> None: + self._require_parent_identity() + current = _output_target_stat(self.parent_fd, self.output.name) + if not self.io.same_entry(self.initial, current): + raise ValueError("declared output path changed during transaction") + if self.output_fd is not None and not self.io.same_entry( + self.initial, + os.fstat(self.output_fd), + ): + raise ValueError("declared output path changed during transaction") + + def _require_published_identity(self, *, require_declared_parent: bool = True) -> None: + if require_declared_parent: + self._require_parent_identity() + published = _open_directory_at(self.io, self.parent_fd, self.output.name) + try: + expected = ( + self.published + if self.published is not None + else os.fstat(self.stage_fd) + ) + observed = os.fstat(published) + if not self.io.same_entry(expected, observed): + raise ValueError("published evidence output identity is invalid") + if self.published is None: + self.published = observed + finally: + os.close(published) + + def commit(self) -> None: + if self.closed or self.committed: + raise ValueError("evidence transaction cannot be committed") + self._require_initial_identity() + moved_old = False + reserved = False + published = False + try: + if self.initial is not None: + os.rename( + self.output.name, + "backup", + src_dir_fd=self.parent_fd, + dst_dir_fd=self.transaction_fd, + ) + moved_old = True + os.mkdir(self.output.name, 0o700, dir_fd=self.parent_fd) + reserved = True + self.reservation_fd = _open_directory_at( + self.io, + self.parent_fd, + self.output.name, + ) + reservation = _output_target_stat( + self.parent_fd, + self.output.name, + ) + if not self.io.same_entry( + reservation, + os.fstat(self.reservation_fd), + ): + raise ValueError("evidence output reservation changed before publish") + os.replace( + "stage", + self.output.name, + src_dir_fd=self.transaction_fd, + dst_dir_fd=self.parent_fd, + ) + published = True + os.fsync(self.parent_fd) + self.committed = True + self._require_published_identity() + except BaseException as error: + try: + if published: + self.committed = True + self.rollback() + elif moved_old: + os.rename( + "backup", + self.output.name, + src_dir_fd=self.transaction_fd, + dst_dir_fd=self.parent_fd, + ) + os.fsync(self.parent_fd) + elif reserved: + current = _output_target_stat( + self.parent_fd, + self.output.name, + ) + if ( + current is not None + and self.reservation_fd is not None + and self.io.same_entry( + current, + os.fstat(self.reservation_fd), + ) + ): + os.rmdir(self.output.name, dir_fd=self.parent_fd) + os.fsync(self.parent_fd) + except BaseException as rollback_error: + raise RuntimeError( + "evidence transaction commit rollback failed" + ) from rollback_error + raise error + + def rollback(self) -> None: + if self.closed or not self.committed: + return + self._require_published_identity(require_declared_parent=False) + os.rename( + self.output.name, + "failed", + src_dir_fd=self.parent_fd, + dst_dir_fd=self.transaction_fd, + ) + if self.initial is not None: + os.rename( + "backup", + self.output.name, + src_dir_fd=self.transaction_fd, + dst_dir_fd=self.parent_fd, + ) + os.fsync(self.parent_fd) + self.committed = False + self._require_parent_identity() + + def finish(self) -> None: + if self.closed: + return + self.validate_durable_commit() + self._cleanup_uncommitted() + + def validate_durable_commit(self) -> None: + if self.closed or not self.committed: + raise ValueError("evidence transaction is not durably committed") + self._require_published_identity() + + def abort(self) -> None: + if self.closed: + return + rollback_error = None + try: + if self.committed: + self.rollback() + except BaseException as error: + rollback_error = error + try: + self._cleanup_uncommitted() + except BaseException as error: + if rollback_error is None: + rollback_error = error + if rollback_error is not None: + raise RuntimeError("evidence transaction rollback failed") from rollback_error + + def _cleanup_uncommitted(self) -> None: + if self.closed: + return + first_error = None + for attribute in ( + "reservation_fd", + "stage_fd", + "output_fd", + "transaction_fd", + ): + try: + self._close_descriptor_attribute(attribute) + except BaseException as error: + if first_error is None: + first_error = error + transaction_descriptors_closed = all( + getattr(self, attribute) is None + for attribute in ( + "reservation_fd", + "stage_fd", + "output_fd", + "transaction_fd", + ) + ) + if transaction_descriptors_closed and self.transaction_name: + try: + self.io.remove_tree_at(self.parent_fd, self.transaction_name) + os.fsync(self.parent_fd) + self.transaction_name = "" + except BaseException as error: + if first_error is None: + first_error = error + if not self.transaction_name: + for attribute in ("parent_fd", "root_fd"): + try: + self._close_descriptor_attribute(attribute) + except BaseException as error: + if first_error is None: + first_error = error + self.closed = self.parent_fd is None and self.root_fd is None + if first_error is not None: + raise first_error + + def _close_descriptor_attribute(self, attribute: str) -> None: + descriptor = getattr(self, attribute) + if descriptor is None: + return + try: + self.io.close_descriptors(descriptor) + except BaseException: + try: + os.fstat(descriptor) + except OSError: + setattr(self, attribute, None) + raise + setattr(self, attribute, None) + + def isolate_cleanup_failure(self) -> None: + """Close descriptors while preserving an owner-only hidden residual.""" + if self.closed: + return + for attribute in ( + "reservation_fd", + "stage_fd", + "output_fd", + "transaction_fd", + "parent_fd", + "root_fd", + ): + descriptor = getattr(self, attribute) + if descriptor is None: + continue + for _attempt in range(2): + try: + os.close(descriptor) + break + except OSError: + try: + os.fstat(descriptor) + except OSError: + break + setattr(self, attribute, None) + self.closed = True diff --git a/src/skill2workflow/_controlled_lark_pilot_private_authorization.py b/src/skill2workflow/_controlled_lark_pilot_private_authorization.py new file mode 100644 index 0000000..5d3620b --- /dev/null +++ b/src/skill2workflow/_controlled_lark_pilot_private_authorization.py @@ -0,0 +1,632 @@ +"""Anchored private authorization sessions and finalization bundles.""" + +from __future__ import annotations + +import json +import os +import stat +import fcntl +from dataclasses import dataclass +from pathlib import Path + +from ._controlled_lark_pilot_evidence_writer import ( + _allocate_transaction_directory, + _close_descriptors, + _open_private_parent, + _open_relative_directory, + _private_target_stat, + _read_json_at, + _remove_tree_at, + _require_declared_directory_identity, + _same_entry, + _write_private_json_at, +) + + +def _require_owner_only_regular(item, label: str) -> None: + if item is None or not stat.S_ISREG(item.st_mode): + raise ValueError(f"{label} must be a regular file") + if os.name == "posix" and item.st_mode & 0o077: + raise ValueError(f"{label} must use owner-only permissions") + + +def _entry_fingerprint(item) -> tuple: + return ( + item.st_dev, + item.st_ino, + item.st_mode, + item.st_size, + item.st_mtime_ns, + item.st_ctime_ns, + ) + + +def _read_descriptor_bytes(descriptor: int) -> bytes: + os.lseek(descriptor, 0, os.SEEK_SET) + chunks = [] + while True: + chunk = os.read(descriptor, 65536) + if not chunk: + break + chunks.append(chunk) + os.lseek(descriptor, 0, os.SEEK_SET) + return b"".join(chunks) + + +@dataclass +class AuthorizationEntrySnapshot: + session: "AnchoredPrivateSession" + name: str + label: str + descriptor: int + fingerprint: tuple + raw_content: bytes + value: object + closed: bool = False + + @classmethod + def capture( + cls, + session: "AnchoredPrivateSession", + name: str, + label: str, + ) -> "AuthorizationEntrySnapshot": + descriptor = None + try: + try: + descriptor = os.open( + name, + os.O_RDONLY | os.O_NONBLOCK | os.O_NOFOLLOW, + dir_fd=session.parent_fd, + ) + except FileNotFoundError: + raise + except OSError as error: + raise ValueError( + f"{label} must not be a symbolic link or non-regular file" + ) from error + opened = os.fstat(descriptor) + _require_owner_only_regular(opened, label) + named = os.stat( + name, + dir_fd=session.parent_fd, + follow_symlinks=False, + ) + _require_owner_only_regular(named, label) + fingerprint = _entry_fingerprint(opened) + if fingerprint != _entry_fingerprint(named): + raise ValueError(f"{label} changed during authorization snapshot") + raw_content = _read_descriptor_bytes(descriptor) + value = json.loads(raw_content.decode("utf-8")) + after = os.fstat(descriptor) + named_after = os.stat( + name, + dir_fd=session.parent_fd, + follow_symlinks=False, + ) + if ( + fingerprint != _entry_fingerprint(after) + or fingerprint != _entry_fingerprint(named_after) + or raw_content != _read_descriptor_bytes(descriptor) + ): + raise ValueError(f"{label} changed during authorization snapshot") + return cls( + session=session, + name=name, + label=label, + descriptor=descriptor, + fingerprint=fingerprint, + raw_content=raw_content, + value=value, + ) + except BaseException: + _close_descriptors(descriptor) + raise + + def validate(self) -> None: + if self.closed or self.descriptor is None: + raise ValueError(f"{self.label} authorization snapshot is closed") + opened = os.fstat(self.descriptor) + _require_owner_only_regular(opened, self.label) + try: + named = os.stat( + self.name, + dir_fd=self.session.parent_fd, + follow_symlinks=False, + ) + except (FileNotFoundError, OSError) as error: + raise ValueError(f"{self.label} changed after authorization snapshot") from error + _require_owner_only_regular(named, self.label) + if ( + self.fingerprint != _entry_fingerprint(opened) + or self.fingerprint != _entry_fingerprint(named) + or self.raw_content != _read_descriptor_bytes(self.descriptor) + ): + raise ValueError(f"{self.label} changed after authorization snapshot") + + def close(self) -> None: + if self.closed: + return + descriptor = self.descriptor + try: + _close_descriptors(descriptor) + except BaseException: + try: + os.fstat(descriptor) + except OSError: + self.descriptor = None + self.closed = True + raise + self.descriptor = None + self.closed = True + + def isolate_cleanup_failure(self) -> None: + descriptor = self.descriptor + self.descriptor = None + self.closed = True + if descriptor is not None: + try: + os.close(descriptor) + except OSError: + pass + + +class AuthorizationBundleSnapshot: + """Retain one immutable decision/marker authorization view.""" + + def __init__(self, session: "AnchoredPrivateSession"): + self.session = session + self.decision_entry = None + self.marker_entry = None + self.closed = False + try: + session.check_identity() + self.decision_entry = AuthorizationEntrySnapshot.capture( + session, + "decision.json", + "private decision", + ) + self.marker_entry = AuthorizationEntrySnapshot.capture( + session, + "finalization.json", + "private finalization marker", + ) + self.validate() + except BaseException: + self.close() + raise + + @property + def decision(self): + return self.decision_entry.value + + @property + def marker(self): + return self.marker_entry.value + + def validate(self) -> None: + if self.closed: + raise ValueError("private authorization bundle snapshot is closed") + self.session.check_identity() + self.decision_entry.validate() + self.marker_entry.validate() + self.session.check_identity() + + def close(self) -> None: + if self.closed: + return + first_error = None + for entry in (self.marker_entry, self.decision_entry): + if entry is None: + continue + try: + entry.close() + except BaseException as error: + if first_error is None: + first_error = error + self.closed = all( + entry is None or entry.closed + for entry in (self.marker_entry, self.decision_entry) + ) + if first_error is not None: + raise first_error + + def isolate_cleanup_failure(self) -> None: + for entry in (self.marker_entry, self.decision_entry): + if entry is not None: + entry.isolate_cleanup_failure() + self.closed = True + + +def _remove_authorization_target(parent_fd: int, name: str) -> None: + try: + item = os.stat(name, dir_fd=parent_fd, follow_symlinks=False) + except FileNotFoundError: + return + if stat.S_ISDIR(item.st_mode) and not stat.S_ISLNK(item.st_mode): + os.rmdir(name, dir_fd=parent_fd) + else: + os.unlink(name, dir_fd=parent_fd) + + +def _invalidate_declared_marker( + private_dir: Path, + original_parent_fd: int, +) -> None: + try: + _absolute, root_fd, parent_fd = _open_private_parent( + Path(private_dir) / ".rollback-placeholder", + create=False, + ) + except (FileNotFoundError, OSError, ValueError): + return + try: + if not _same_entry(os.fstat(original_parent_fd), os.fstat(parent_fd)): + _remove_authorization_target(parent_fd, "finalization.json") + os.fsync(parent_fd) + if _private_target_stat(parent_fd, "finalization.json") is not None: + raise RuntimeError("declared private path retained finalization authorization") + finally: + _close_descriptors(parent_fd, root_fd) + + +class AnchoredPrivateSession: + """Hold one owner-only private directory identity and exclusive auth lock.""" + + LOCK_NAME = ".pilot-authorization.lock" + + def __init__(self, private_dir: Path): + placeholder = Path(private_dir) / ".session-placeholder" + absolute, self.root_fd, self.parent_fd = _open_private_parent( + placeholder, + create=False, + ) + self.private_dir = absolute.parent + self.lock_fd = None + self.closed = False + try: + self.lock_fd = self._open_lock() + self.check_identity() + except BaseException: + _close_descriptors(self.lock_fd, self.parent_fd, self.root_fd) + self.lock_fd = None + self.parent_fd = None + self.root_fd = None + self.closed = True + raise + + def _open_lock(self) -> int: + flags = os.O_RDWR | os.O_NONBLOCK | os.O_NOFOLLOW + created = False + descriptor = None + locked = False + completed = False + try: + try: + descriptor = os.open( + self.LOCK_NAME, + flags | os.O_CREAT | os.O_EXCL, + 0o600, + dir_fd=self.parent_fd, + ) + created = True + except FileExistsError: + try: + descriptor = os.open( + self.LOCK_NAME, + flags, + dir_fd=self.parent_fd, + ) + except OSError as error: + raise ValueError("private authorization lock is invalid") from error + item = os.fstat(descriptor) + if created: + os.fchmod(descriptor, 0o600) + item = os.fstat(descriptor) + _require_owner_only_regular(item, "private authorization lock") + fcntl.flock(descriptor, fcntl.LOCK_EX | fcntl.LOCK_NB) + locked = True + observed = os.stat( + self.LOCK_NAME, + dir_fd=self.parent_fd, + follow_symlinks=False, + ) + _require_owner_only_regular(observed, "private authorization lock") + if not _same_entry(item, observed): + raise ValueError("private authorization lock changed during open") + os.fsync(self.parent_fd) + completed = True + return descriptor + except BlockingIOError as error: + raise ValueError("private authorization session is busy") from error + except BaseException: + raise + finally: + if descriptor is not None and not completed: + if locked: + try: + fcntl.flock(descriptor, fcntl.LOCK_UN) + except OSError: + pass + try: + os.close(descriptor) + except OSError: + pass + + def check_identity(self) -> None: + if self.closed: + raise ValueError("private authorization session is closed") + _require_declared_directory_identity( + self.root_fd, + self.private_dir, + self.parent_fd, + "private authorization", + ) + parent_mode = os.fstat(self.parent_fd).st_mode + if parent_mode & 0o077: + raise ValueError("private authorization parent must use owner-only permissions") + if self.lock_fd is None: + raise ValueError("private authorization lock changed") + opened_lock = os.fstat(self.lock_fd) + _require_owner_only_regular(opened_lock, "private authorization lock") + try: + named_lock = os.stat( + self.LOCK_NAME, + dir_fd=self.parent_fd, + follow_symlinks=False, + ) + except (FileNotFoundError, OSError) as error: + raise ValueError("private authorization lock changed") from error + _require_owner_only_regular(named_lock, "private authorization lock") + if not _same_entry(opened_lock, named_lock): + raise ValueError("private authorization lock changed") + + def authorization_bundle_snapshot(self) -> AuthorizationBundleSnapshot: + return AuthorizationBundleSnapshot(self) + + def read_json(self, relative_path: Path, *, required: bool = True): + if self.closed: + raise ValueError("private authorization session is closed") + relative = Path(relative_path) + if relative.is_absolute() or not relative.name or ".." in relative.parts: + raise ValueError("private authorization path must be relative") + try: + parent_fd = _open_relative_directory( + self.parent_fd, + relative.parts[:-1], + create=False, + ) + except FileNotFoundError: + if required: + raise + return None + try: + try: + return _read_json_at(parent_fd, relative.name, owner_only=True) + except FileNotFoundError: + if required: + raise + return None + finally: + os.close(parent_fd) + + def close(self) -> None: + if self.closed: + return + first_error = None + if self.lock_fd is not None: + try: + fcntl.flock(self.lock_fd, fcntl.LOCK_UN) + except BaseException as error: + first_error = error + for attribute in ("lock_fd", "parent_fd", "root_fd"): + descriptor = getattr(self, attribute) + if descriptor is None: + continue + try: + _close_descriptors(descriptor) + except BaseException as error: + try: + os.fstat(descriptor) + except OSError: + setattr(self, attribute, None) + if first_error is None: + first_error = error + else: + setattr(self, attribute, None) + self.closed = all( + getattr(self, attribute) is None + for attribute in ("lock_fd", "parent_fd", "root_fd") + ) + if first_error is not None: + raise first_error + + def isolate_cleanup_failure(self) -> None: + if self.lock_fd is not None: + try: + fcntl.flock(self.lock_fd, fcntl.LOCK_UN) + except OSError: + pass + for attribute in ("lock_fd", "parent_fd", "root_fd"): + descriptor = getattr(self, attribute) + if descriptor is not None: + try: + os.close(descriptor) + except OSError: + pass + setattr(self, attribute, None) + self.closed = True + + def __enter__(self): + return self + + def __exit__(self, _error_type, _error, _traceback): + self.close() + + +def open_private_session(private_dir: Path) -> AnchoredPrivateSession: + return AnchoredPrivateSession(private_dir) + + +class PrivateFinalizationBundle: + """Transactionally publish decision then marker within one private session.""" + + def __init__(self, session: AnchoredPrivateSession): + self.session = session + self.transaction_name = "" + self.transaction_fd = None + self.initial_decision = None + self.has_backup = False + self.decision_published = False + self.marker_published = False + self.closed = False + try: + session.check_identity() + if _private_target_stat(session.parent_fd, "finalization.json") is not None: + raise ValueError("controlled pilot is already finalized") + self.initial_decision = _private_target_stat( + session.parent_fd, + "decision.json", + ) + if self.initial_decision is not None: + _require_owner_only_regular( + self.initial_decision, + "private decision", + ) + self.transaction_name, self.transaction_fd = ( + _allocate_transaction_directory( + session.parent_fd, + "finalization", + ) + ) + if self.initial_decision is not None: + os.link( + "decision.json", + "decision-backup", + src_dir_fd=session.parent_fd, + dst_dir_fd=self.transaction_fd, + follow_symlinks=False, + ) + backup = os.stat( + "decision-backup", + dir_fd=self.transaction_fd, + follow_symlinks=False, + ) + if not _same_entry(self.initial_decision, backup): + raise ValueError("private decision changed during backup") + self.has_backup = True + session.check_identity() + except BaseException: + self._cleanup_transaction() + raise + + def publish_decision(self, decision: object) -> None: + if self.closed or self.decision_published: + raise ValueError("private decision cannot be published") + _write_private_json_at( + self.session.parent_fd, + "decision.json", + decision, + ) + self.decision_published = True + self.session.check_identity() + + def publish_marker(self, marker: object) -> AuthorizationBundleSnapshot: + if self.closed or not self.decision_published or self.marker_published: + raise ValueError("private finalization marker cannot be published") + self.session.check_identity() + _write_private_json_at( + self.session.parent_fd, + "finalization.json", + marker, + require_missing=True, + ) + self.marker_published = True + self.session.check_identity() + return self.session.authorization_bundle_snapshot() + + def rollback(self) -> None: + if self.closed: + return + first_error = None + try: + _remove_authorization_target( + self.session.parent_fd, + "finalization.json", + ) + if self.has_backup: + os.replace( + "decision-backup", + "decision.json", + src_dir_fd=self.transaction_fd, + dst_dir_fd=self.session.parent_fd, + ) + self.has_backup = False + else: + _remove_authorization_target( + self.session.parent_fd, + "decision.json", + ) + os.fsync(self.session.parent_fd) + except BaseException as error: + first_error = error + try: + _invalidate_declared_marker( + self.session.private_dir, + self.session.parent_fd, + ) + except BaseException as error: + if first_error is None: + first_error = error + try: + self._cleanup_transaction() + except BaseException as error: + if first_error is None: + first_error = error + if first_error is not None: + raise RuntimeError("private finalization rollback failed") from first_error + + def finish(self) -> None: + if self.closed: + return + if not self.marker_published: + raise ValueError("private finalization bundle is incomplete") + self.session.check_identity() + self._cleanup_transaction() + + def _cleanup_transaction(self) -> None: + if self.closed: + return + first_error = None + if self.transaction_fd is not None: + descriptor = self.transaction_fd + try: + _close_descriptors(descriptor) + except BaseException as error: + try: + os.fstat(descriptor) + except OSError: + self.transaction_fd = None + first_error = error + else: + self.transaction_fd = None + if self.transaction_fd is None and self.transaction_name: + try: + _remove_tree_at(self.session.parent_fd, self.transaction_name) + os.fsync(self.session.parent_fd) + self.transaction_name = "" + except BaseException as error: + if first_error is None: + first_error = error + self.closed = self.transaction_fd is None and not self.transaction_name + if first_error is not None: + raise first_error + + def isolate_cleanup_failure(self) -> None: + if self.transaction_fd is not None: + try: + os.close(self.transaction_fd) + except OSError: + pass + self.transaction_fd = None + self.closed = True diff --git a/src/skill2workflow/connectors.py b/src/skill2workflow/connectors.py index e7d569b..7dfd4c9 100644 --- a/src/skill2workflow/connectors.py +++ b/src/skill2workflow/connectors.py @@ -16,6 +16,7 @@ ConnectorBinding = Dict[str, object] ConnectorResult = Dict[str, object] ExternalConnectorExecutor = Callable[..., ConnectorResult] +ExternalConnectorPreflight = Callable[..., ConnectorResult] CONNECTOR_MANIFEST_VERSION = "skill2workflow-connector-0.1.0" CONNECTOR_EXECUTION_CONTRACT_VERSION = "skill2workflow-connector-execution-0.1.0" @@ -103,6 +104,7 @@ class ExternalConnector: manifest: Dict[str, object] executor: ExternalConnectorExecutor + preflight: ExternalConnectorPreflight = None class ConnectorRuntime: @@ -132,6 +134,7 @@ def register_external_connector(self, connector: ExternalConnector) -> None: self._external_connectors[connector_id] = ExternalConnector( manifest=copy.deepcopy(connector.manifest), executor=connector.executor, + preflight=connector.preflight, ) def list_connectors(self) -> List[Dict[str, object]]: diff --git a/src/skill2workflow/controlled_lark_pilot.py b/src/skill2workflow/controlled_lark_pilot.py new file mode 100644 index 0000000..ae52f56 --- /dev/null +++ b/src/skill2workflow/controlled_lark_pilot.py @@ -0,0 +1,1514 @@ +from __future__ import annotations + +import argparse +import json +import os +import re +import sys +import urllib.request +from contextlib import contextmanager +from datetime import date, datetime, timezone +from pathlib import Path +from typing import Dict, List, Tuple +from zoneinfo import ZoneInfo + +from .connectors import ConnectorRuntime, ExternalConnector +from .control_plane import LocalControlPlane +from .credentials import StaticCredentialProvider +from ._controlled_lark_pilot_operations import ( + APP_ID_ENVIRONMENT, + APP_SECRET_ENVIRONMENT, + FINALIZATION_KEYS, + FINALIZATION_SCHEMA_VERSION, + LIVE_SWITCH, + TOKEN_ENVIRONMENT, + Task6Dependencies, + exercise_disabled_live_operation, + exercise_rollback_operation, + finalize_pilot_operation, + validate_final_decision, + verify_pilot_operation, +) +from .controlled_lark_pilot_evidence import ( + INDEX_SCHEMA_VERSION, + _is_approved_live_run, + _rejection_exercise, + _run_sort_key, + build_acceptance_summary, + build_run_evidence, + validate_evidence_pack, + prepare_evidence_pack_transaction, + write_evidence_pack, +) +from ._controlled_lark_pilot_private_authorization import ( + PrivateFinalizationBundle, + open_private_session, +) +from ._controlled_lark_pilot_evidence_writer import ( + ensure_private_directory_anchored, + finish_durable_resources, + read_json_anchored, + write_private_json_anchored, +) +from .external_connectors import load_external_connector +from .lark_task_pilot import build_lark_task_pilot_workflow + + +PILOT_SCHEMA_VERSION = "controlled-lark-pilot-0.1.0" +WORKFLOW_ID = "workflow_controlled_lark_pilot" +WORKFLOW_VERSION = "0.1.0" +SCENARIO_ID = "sales_renewal_risk_followup" +PILOT_TIMEZONE = "Asia/Shanghai" +REQUIRED_CHARTER_KEYS = { + "schema_version", + "scenario_id", + "workflow_id", + "workflow_version", + "support_model", + "timezone", + "starts_on", + "expires_on", + "team_consent_confirmed", + "assignee_consent_confirmed", + "commercial_engagement_confirmed", + "required_approved_runs", + "required_distinct_days", + "required_distinct_cases", +} +REQUIRED_CASE_KEYS = { + "pilot_case_id", + "account_name", + "renewal_risk", + "owner_open_id", + "due_at", +} +_CASE_TEMPLATE_NAME = re.compile(r"[a-z0-9][a-z0-9-]{0,63}") + +SOURCE_REPOSITORY_ROOT = Path(__file__).resolve().parents[2] +_OPERATOR_ERROR = "controlled pilot command failed" +_REJECT_CONFIRMATION_ERROR = ( + "controlled pilot rejection does not use live confirmation" +) +_LARK_TENANT_TOKEN_URL = ( + "https://open.feishu.cn/open-apis/auth/v3/tenant_access_token/internal" +) +_LARK_TENANT_TOKEN_TIMEOUT = 20 + + +class _OperatorCLIError(Exception): + def __init__(self, message: str = _OPERATOR_ERROR): + super().__init__(message) + self.message = message + + +class _RedactedArgumentParser(argparse.ArgumentParser): + def __init__(self, *args, **kwargs): + kwargs["allow_abbrev"] = False + super().__init__(*args, **kwargs) + + def error(self, message): + del message + raise _OperatorCLIError() + + +def _build_controlled_pilot_parser() -> argparse.ArgumentParser: + parser = _RedactedArgumentParser( + prog="controlled_lark_pilot.py", + description="Operate the fixed controlled Lark/Feishu pilot.", + ) + commands = parser.add_subparsers( + dest="command", + required=True, + parser_class=_RedactedArgumentParser, + ) + + initialize = commands.add_parser("init") + initialize.add_argument("--work-dir", type=Path, required=True) + initialize.add_argument("--starts-on", required=True) + initialize.add_argument("--expires-on", required=True) + initialize.add_argument( + "--confirm-team-consent", + action="store_true", + required=True, + ) + initialize.add_argument( + "--confirm-assignee-consent", + action="store_true", + required=True, + ) + initialize.add_argument( + "--confirm-commercial-engagement", + action="store_true", + required=True, + ) + + case_template = commands.add_parser("case-template") + case_template.add_argument("--work-dir", type=Path, required=True) + case_template.add_argument("--name", required=True) + case_template.add_argument("--case-id", required=True) + + start = commands.add_parser("start") + start.add_argument("--work-dir", type=Path, required=True) + start.add_argument("--input", type=Path, required=True) + + preflight = commands.add_parser("preflight") + preflight.add_argument("--input", type=Path, required=True) + + decide = commands.add_parser("decide") + decide.add_argument("--work-dir", type=Path, required=True) + decide.add_argument("--run-id", required=True) + decision = decide.add_mutually_exclusive_group(required=True) + decision.add_argument("--approve", action="store_true") + decision.add_argument("--reject", action="store_true") + decide.add_argument("--confirm-live-create", action="store_true") + + evidence = commands.add_parser("evidence") + evidence.add_argument("--work-dir", type=Path, required=True) + evidence.add_argument("--output-dir", type=Path) + + failure = commands.add_parser("exercise-failure") + failure.add_argument("--work-dir", type=Path, required=True) + + rollback = commands.add_parser("exercise-rollback") + rollback.add_argument("--work-dir", type=Path, required=True) + + verify = commands.add_parser("verify") + verify.add_argument("--work-dir", type=Path, required=True) + + finalize = commands.add_parser("finalize") + finalize.add_argument("--work-dir", type=Path, required=True) + finalize.add_argument("--decision-file", type=Path, required=True) + finalize.add_argument("--output-dir", type=Path) + return parser + + +def _fixed_charter(arguments) -> Dict[str, object]: + return { + "schema_version": PILOT_SCHEMA_VERSION, + "scenario_id": SCENARIO_ID, + "workflow_id": WORKFLOW_ID, + "workflow_version": WORKFLOW_VERSION, + "support_model": "assisted", + "timezone": PILOT_TIMEZONE, + "starts_on": arguments.starts_on, + "expires_on": arguments.expires_on, + "team_consent_confirmed": arguments.confirm_team_consent, + "assignee_consent_confirmed": arguments.confirm_assignee_consent, + "commercial_engagement_confirmed": ( + arguments.confirm_commercial_engagement + ), + "required_approved_runs": 5, + "required_distinct_days": 5, + "required_distinct_cases": 2, + } + + +def _select_summary_fields( + result: object, + fields: Tuple[str, ...], +) -> Dict[str, object]: + if not isinstance(result, dict): + raise ValueError("controlled pilot operation returned an invalid summary") + return {key: result[key] for key in fields if key in result} + + +def _command_summary( + command: str, + result: object, + charter: Dict[str, object] = None, +) -> Dict[str, object]: + if command == "init": + summary = _select_summary_fields( + result, + ( + "status", + "scenario_id", + "workflow_id", + "workflow_version", + ), + ) + summary.update( + { + "team_consent_confirmed": charter["team_consent_confirmed"], + "assignee_consent_confirmed": charter[ + "assignee_consent_confirmed" + ], + "commercial_engagement_confirmed": charter[ + "commercial_engagement_confirmed" + ], + } + ) + return summary + if command == "start": + return _select_summary_fields( + result, + ( + "run_id", + "workflow_id", + "workflow_version", + "run_status", + "current_node", + "input_keys", + "preflight_ready", + ), + ) + if command == "case-template": + return _select_summary_fields(result, ("status", "field_count")) + if command == "preflight": + return _select_summary_fields( + result, + ( + "status", + "connector_id", + "operation", + "mode", + "task_title_present", + "task_description_present", + "assignee_present", + "due_at_present", + "provider_payload_constructed", + "credential_resolution_attempted", + "network_called", + ), + ) + if command == "decide": + return _select_summary_fields( + result, + ( + "run_id", + "workflow_id", + "workflow_version", + "run_status", + "gate_decision", + "connector_invoked", + "connector_status", + "credential_status", + "provider_status", + "idempotency_key_present", + "lark_task_id_present", + ), + ) + if command == "evidence": + return _select_summary_fields( + result, + ( + "status", + "file_count", + "run_count", + "approved_live_runs", + "distinct_calendar_days", + "distinct_private_cases", + "rejected_runs", + "unmet_conditions", + ), + ) + if command == "exercise-failure": + return _select_summary_fields( + result, + ( + "exercise", + "passed", + "provider_status", + "credential_resolution_attempted", + "transport_attempted", + ), + ) + if command == "exercise-rollback": + return _select_summary_fields( + result, + ( + "exercise", + "passed", + "live_switch_enabled", + "live_approval_blocked", + "dry_run_status", + ), + ) + if command == "verify": + summary = _select_summary_fields(result, ("all_passed",)) + commands = result.get("commands", []) + if not isinstance(commands, list): + raise ValueError("controlled pilot verification summary is invalid") + summary["commands"] = [ + _select_summary_fields( + item, + ("id", "exit_code", "passed", "duration_ms"), + ) + for item in commands + ] + return summary + if command == "finalize": + return _select_summary_fields( + result, + ( + "status", + "decision", + "approved_live_runs", + "distinct_calendar_days", + "distinct_private_cases", + "rejected_runs", + ), + ) + raise ValueError("controlled pilot command is invalid") + + +def _load_private_decision(decision_file: Path) -> Dict[str, object]: + declared = Path(os.path.abspath(os.fspath(decision_file))) + _require_outside_repository( + SOURCE_REPOSITORY_ROOT, + declared, + "private decision file", + ) + _require_outside_repository( + SOURCE_REPOSITORY_ROOT, + declared.resolve(), + "private decision file", + ) + value = read_json_anchored(declared, owner_only=True) + if not isinstance(value, dict): + raise ValueError("private decision file must contain a JSON object") + return value + + +def _dispatch_controlled_pilot(arguments) -> Tuple[Dict[str, object], object]: + command = arguments.command + if command == "init": + charter = _fixed_charter(arguments) + result = initialize_pilot( + SOURCE_REPOSITORY_ROOT, + arguments.work_dir, + charter, + ) + return _command_summary(command, result, charter=charter), result + if command == "case-template": + result = create_private_case_template( + SOURCE_REPOSITORY_ROOT, + arguments.work_dir, + arguments.name, + arguments.case_id, + ) + elif command == "start": + result = start_pilot_run( + SOURCE_REPOSITORY_ROOT, + arguments.work_dir, + arguments.input, + ) + elif command == "preflight": + result = preflight_pilot_case( + SOURCE_REPOSITORY_ROOT, + arguments.input, + ) + elif command == "decide": + if arguments.reject and arguments.confirm_live_create: + raise _OperatorCLIError(_REJECT_CONFIRMATION_ERROR) + result = decide_pilot_run( + SOURCE_REPOSITORY_ROOT, + arguments.work_dir, + arguments.run_id, + approved=bool(arguments.approve), + confirmed_live=bool(arguments.confirm_live_create), + ) + elif command == "evidence": + result = generate_pilot_evidence( + SOURCE_REPOSITORY_ROOT, + arguments.work_dir, + output_dir=arguments.output_dir, + ) + elif command == "exercise-failure": + result = exercise_disabled_live( + SOURCE_REPOSITORY_ROOT, + arguments.work_dir, + ) + elif command == "exercise-rollback": + result = exercise_rollback( + SOURCE_REPOSITORY_ROOT, + arguments.work_dir, + ) + elif command == "verify": + result = verify_pilot( + SOURCE_REPOSITORY_ROOT, + arguments.work_dir, + ) + elif command == "finalize": + decision = _load_private_decision(arguments.decision_file) + result = finalize_pilot( + SOURCE_REPOSITORY_ROOT, + arguments.work_dir, + decision, + output_dir=arguments.output_dir, + ) + else: + raise _OperatorCLIError() + return _command_summary(command, result), result + + +def main(argv=None) -> int: + try: + arguments = _build_controlled_pilot_parser().parse_args(argv) + summary, _result = _dispatch_controlled_pilot(arguments) + except _OperatorCLIError as error: + print(error.message, file=sys.stderr) + return 2 + except (OSError, RuntimeError, ValueError): + print(_OPERATOR_ERROR, file=sys.stderr) + return 1 + print( + json.dumps(summary, ensure_ascii=False, separators=(",", ":")), + flush=True, + ) + return 0 + + +def _task6_dependencies() -> Task6Dependencies: + return Task6Dependencies( + require_outside_repository=_require_outside_repository, + load_charter=load_pilot_charter, + initialize=initialize_pilot, + start=start_pilot_run, + decide=decide_pilot_run, + control_plane=_pilot_control_plane, + build_evidence=_build_pilot_evidence, + evidence_output=_evidence_output, + prepare_evidence_pack=prepare_evidence_pack_transaction, + open_private_session=open_private_session, + finalization_bundle=PrivateFinalizationBundle, + pilot_timezone=PILOT_TIMEZONE, + ) + + +def exercise_disabled_live( + repo_root: Path, + work_dir: Path, + now: datetime = None, +) -> Dict[str, object]: + return exercise_disabled_live_operation( + repo_root, + work_dir, + now, + _task6_dependencies(), + ) + + +def exercise_rollback( + repo_root: Path, + work_dir: Path, + now: datetime = None, +) -> Dict[str, object]: + return exercise_rollback_operation( + repo_root, + work_dir, + now, + _task6_dependencies(), + ) + + +def verify_pilot( + repo_root: Path, + work_dir: Path, + command_runner=None, +) -> Dict[str, object]: + return verify_pilot_operation( + repo_root, + work_dir, + command_runner, + _task6_dependencies(), + ) + + +def finalize_pilot( + repo_root: Path, + work_dir: Path, + decision: Dict[str, object], + output_dir: Path = None, + now: datetime = None, +) -> Dict[str, object]: + return finalize_pilot_operation( + repo_root, + work_dir, + decision, + output_dir, + now, + _task6_dependencies(), + ) + + +def generate_pilot_evidence( + repo_root: Path, + work_dir: Path, + output_dir: Path = None, + now: datetime = None, +) -> Dict[str, object]: + repo_root = Path(repo_root).resolve() + work_dir = Path(work_dir).resolve() + _require_outside_repository(repo_root, work_dir, "pilot work directory") + output, repository_export = _evidence_output(repo_root, work_dir, output_dir) + if repository_export: + private_session = open_private_session(work_dir / "private") + authorization_snapshot = None + transaction = None + durable_success = False + try: + try: + authorization_snapshot = ( + private_session.authorization_bundle_snapshot() + ) + except FileNotFoundError as error: + raise ValueError( + "successful private finalization is required for repository export" + ) from error + decision = authorization_snapshot.decision + marker = authorization_snapshot.marker + authorization_snapshot.validate() + private_session.check_identity() + pack = _build_pilot_evidence( + repo_root, + work_dir, + decision_override=decision, + now=now, + private_session=private_session, + historical_authorization=(decision, marker), + ) + _require_finalized_export(pack, marker, decision) + authorization_snapshot.validate() + transaction = prepare_evidence_pack_transaction(output, pack) + authorization_snapshot.validate() + transaction.commit() + transaction.validate_durable_commit() + private_session.check_identity() + authorization_snapshot.validate() + + # Irreversible durable-success commit point for repository export. + durable_success = True + written = { + "status": "written", + "file_count": transaction.file_count, + "output_dir": str(transaction.output), + } + except BaseException: + cleanup_errors = [] + if transaction is not None: + try: + transaction.abort() + except BaseException as cleanup_error: + cleanup_errors.append(cleanup_error) + if authorization_snapshot is not None: + try: + authorization_snapshot.close() + except BaseException as cleanup_error: + cleanup_errors.append(cleanup_error) + try: + private_session.close() + except BaseException as cleanup_error: + cleanup_errors.append(cleanup_error) + if cleanup_errors: + raise RuntimeError( + "repository evidence export rollback failed" + ) from cleanup_errors[0] + raise + if durable_success: + finish_durable_resources( + (authorization_snapshot, "close"), + (transaction, "finish"), + (private_session, "close"), + ) + else: + pack = _build_pilot_evidence(repo_root, work_dir, now=now) + written = write_evidence_pack(output, pack) + index = pack["index"] + return { + "status": written["status"], + "file_count": written["file_count"], + "run_count": len(pack["runs"]), + "approved_live_runs": index["approved_live_runs"], + "distinct_calendar_days": index["distinct_calendar_days"], + "distinct_private_cases": index["distinct_private_cases"], + "rejected_runs": index["rejected_runs"], + "unmet_conditions": list(index["unmet_conditions"]), + "output_dir": written["output_dir"], + } + + +def _build_pilot_evidence( + repo_root: Path, + work_dir: Path, + decision_override: Dict[str, object] = None, + now: datetime = None, + private_session=None, + historical_authorization=None, +) -> Dict[str, object]: + repo_root = Path(repo_root).resolve() + work_dir = Path(work_dir).resolve() + _require_outside_repository(repo_root, work_dir, "pilot work directory") + if historical_authorization is None: + charter = load_pilot_charter(work_dir, now=now) + else: + if private_session is None: + raise ValueError("historical charter requires private authorization") + authorization_decision, authorization_marker = historical_authorization + private_session.check_identity() + charter = _validate_historical_repository_charter( + private_session.read_json(Path("charter.json")), + authorization_decision, + authorization_marker, + now=now, + ) + private_session.check_identity() + control = _pilot_control_plane( + repo_root, + work_dir, + credential_provider=StaticCredentialProvider({}), + ) + private_runs = [] + forbidden_values: List[str] = [] + for summary in control.list_runs(): + if ( + not isinstance(summary, dict) + or summary.get("workflow_id") != WORKFLOW_ID + or summary.get("workflow_version") != WORKFLOW_VERSION + ): + continue + requested_run_id = str(summary.get("run_id", "")) + run = control.get_run(requested_run_id) + if not isinstance(run, dict) or run.get("run_id") != requested_run_id: + raise ValueError("controlled pilot run identity is invalid") + audit = control.list_audit_events(run_id=requested_run_id) + evidence = build_run_evidence(run, audit) + private_runs.append((evidence, run)) + context = run.get("context", {}) + trigger_input = context.get("input", {}) if isinstance(context, dict) else {} + forbidden_values.extend(_private_string_values(trigger_input)) + private_runs.sort(key=lambda item: _run_sort_key(item[0])) + runs = [item[0] for item in private_runs] + private_case_ids = set() + for evidence, run in private_runs: + if not _is_approved_live_run(evidence): + continue + context = run.get("context", {}) + trigger_input = context.get("input", {}) if isinstance(context, dict) else {} + case_id = trigger_input.get("pilot_case_id", "") if isinstance(trigger_input, dict) else "" + if isinstance(case_id, str) and case_id.strip(): + private_case_ids.add(case_id) + distinct_private_cases = len(private_case_ids) + del private_case_ids + del private_runs + + private_dir = work_dir / "private" + exercise_dir = private_dir / "exercises" + if private_session is not None: + private_session.check_identity() + exercises = { + "rejection": _rejection_exercise(runs), + "failure": ( + private_session.read_json(Path("exercises/failure.json"), required=False) + if private_session is not None + else _load_optional_private_json(exercise_dir / "failure.json") + ), + "rollback": ( + private_session.read_json(Path("exercises/rollback.json"), required=False) + if private_session is not None + else _load_optional_private_json(exercise_dir / "rollback.json") + ), + } + verification = ( + private_session.read_json(Path("verification.json"), required=False) + if private_session is not None + else _load_optional_private_json(private_dir / "verification.json") + ) + decision = ( + json.loads(json.dumps(decision_override, ensure_ascii=False)) + if decision_override is not None + else ( + private_session.read_json(Path("decision.json"), required=False) + if private_session is not None + else _load_optional_private_json(private_dir / "decision.json") + ) + ) + if private_session is not None: + private_session.check_identity() + summary = build_acceptance_summary( + charter, + runs, + distinct_private_cases, + exercises, + verification, + decision, + ) + generated = now or datetime.now(timezone.utc) + if generated.tzinfo is None or generated.utcoffset() is None: + raise ValueError("evidence generation time must include a timezone") + index = { + "schema_version": INDEX_SCHEMA_VERSION, + "generated_at": generated.astimezone(ZoneInfo(PILOT_TIMEZONE)).isoformat(), + "workflow_id": WORKFLOW_ID, + "workflow_version": WORKFLOW_VERSION, + "timezone": PILOT_TIMEZONE, + **summary, + } + pack = { + "charter": charter, + "runs": runs, + "exercises": exercises, + "verification": verification, + "decision": decision, + "index": index, + } + validate_evidence_pack(pack, forbidden_values) + return pack + + +def _private_string_values(value: object) -> List[str]: + values: List[str] = [] + if isinstance(value, dict): + for item in value.values(): + values.extend(_private_string_values(item)) + elif isinstance(value, list): + for item in value: + values.extend(_private_string_values(item)) + elif isinstance(value, str) and value: + values.append(value) + return values + + +def _load_optional_private_json(path: Path): + try: + value = read_json_anchored(path) + except FileNotFoundError: + return None + if not isinstance(value, dict): + raise ValueError(f"private pilot artifact {path.name} must be an object") + return value + + +def _evidence_output( + repo_root: Path, work_dir: Path, output_dir: Path +) -> Tuple[Path, bool]: + if output_dir is None: + return work_dir / "evidence", False + output = Path(os.path.abspath(os.fspath(output_dir))) + resolved = output.resolve() + declared_in_repository = output == repo_root or repo_root in output.parents + resolved_in_repository = resolved == repo_root or repo_root in resolved.parents + if not declared_in_repository and not resolved_in_repository: + return output, False + allowed = repo_root / "docs" / "pilot-evidence" / "loop-40" + if output != allowed or resolved != allowed: + raise ValueError("repository evidence output must equal docs/pilot-evidence/loop-40") + return output, True + + +def _require_finalized_export( + pack: Dict[str, object], + marker: object, + decision: object, +) -> None: + _validate_finalization_marker(marker) + pack_decision = pack.get("decision") + index = pack.get("index") + if ( + not isinstance(decision, dict) + or marker["decision"] != decision.get("decision") + or decision != pack_decision + or not isinstance(index, dict) + or index.get("ready_to_finalize") is not True + ): + raise ValueError("private finalization does not authorize this evidence pack") + + +def _validate_finalization_marker(marker: object) -> datetime: + if not isinstance(marker, dict) or set(marker) != FINALIZATION_KEYS: + raise ValueError("private finalization marker is invalid") + if ( + marker.get("schema_version") != FINALIZATION_SCHEMA_VERSION + or marker.get("finalized") is not True + or marker.get("decision") not in ("continue", "harden", "defer") + ): + raise ValueError("private finalization marker is invalid") + return _require_aware_iso( + marker.get("finalized_at"), + "private finalization timestamp", + ) + + +def _require_aware_iso(value: object, label: str) -> datetime: + if type(value) is not str: + raise ValueError(f"{label} must be an ISO timestamp with timezone") + try: + parsed = datetime.fromisoformat(value.replace("Z", "+00:00")) + except ValueError as error: + raise ValueError(f"{label} must be an ISO timestamp with timezone") from error + if parsed.tzinfo is None or parsed.utcoffset() is None: + raise ValueError(f"{label} must be an ISO timestamp with timezone") + return parsed + + +def _issue_lark_tenant_access_token( + app_id: str, + app_secret: str, + *, + token_transport=None, +) -> str: + """Issue one short-lived tenant token without persisting provider output.""" + + payload = json.dumps( + {"app_id": app_id, "app_secret": app_secret}, + ensure_ascii=False, + separators=(",", ":"), + ).encode("utf-8") + request = urllib.request.Request( + _LARK_TENANT_TOKEN_URL, + data=payload, + headers={"Content-Type": "application/json; charset=utf-8"}, + method="POST", + ) + requester = token_transport or urllib.request.urlopen + try: + response = requester(request, timeout=_LARK_TENANT_TOKEN_TIMEOUT) + try: + raw_response = response.read() + finally: + close = getattr(response, "close", None) + if callable(close): + close() + except Exception: + raise ValueError("Lark tenant token exchange failed") from None + + try: + decoded = json.loads(raw_response.decode("utf-8")) + token = decoded.get("tenant_access_token") + except (AttributeError, TypeError, UnicodeDecodeError, ValueError): + raise ValueError("Lark tenant token exchange failed") from None + if ( + type(decoded) is not dict + or type(decoded.get("code")) is not int + or decoded.get("code") != 0 + or type(token) is not str + or not token + ): + raise ValueError("Lark tenant token exchange failed") + return token + + +def _resolve_live_lark_token(*, token_transport=None) -> str: + """Resolve the approved live credential only after every live guard passes.""" + + if APP_SECRET_ENVIRONMENT in os.environ: + app_secret = os.environ[APP_SECRET_ENVIRONMENT] + app_id = os.environ.get(APP_ID_ENVIRONMENT, "") + if not app_id: + raise ValueError("LARK_APP_ID is required with LARK_APP_SECRET") + if not app_secret: + raise ValueError("LARK_APP_SECRET is required with LARK_APP_ID") + return _issue_lark_tenant_access_token( + app_id, + app_secret, + token_transport=token_transport, + ) + token = os.environ.get(TOKEN_ENVIRONMENT, "") + if not token: + raise ValueError("LARK_BOT_ACCESS_TOKEN is required") + return token + + +def _has_prior_live_connector_failure(control, requested_run_id: str) -> bool: + """Fail closed when this Pilot has a prior approved live connector failure.""" + + for event in control.list_audit_events(): + if ( + not isinstance(event, dict) + or event.get("run_id") == requested_run_id + or event.get("type") != "node_failed" + or event.get("node_id") != "create_lark_task" + ): + continue + metadata = event.get("connector_metadata") + if ( + isinstance(metadata, dict) + and metadata.get("operation") == "create_task" + and metadata.get("mode") == "live" + and metadata.get("provider_status") != "completed" + ): + return True + return False + + +def decide_pilot_run( + repo_root: Path, + work_dir: Path, + run_id: str, + approved: bool, + confirmed_live: bool = False, + now: datetime = None, + transport=None, + token_transport=None, +) -> Dict[str, object]: + repo_root = Path(repo_root).resolve() + work_dir = Path(work_dir).resolve() + _require_outside_repository(repo_root, work_dir, "pilot work directory") + if type(approved) is not bool: + raise ValueError("approved must be a boolean") + load_pilot_charter(work_dir, now=now) + with _open_unclosed_pilot_session(work_dir): + return _decide_pilot_run_open( + repo_root, + work_dir, + run_id, + approved=approved, + confirmed_live=confirmed_live, + transport=transport, + token_transport=token_transport, + ) + + +def _decide_pilot_run_open( + repo_root: Path, + work_dir: Path, + run_id: str, + *, + approved: bool, + confirmed_live: bool, + transport=None, + token_transport=None, +) -> Dict[str, object]: + """Resume one waiting run while the private Pilot decision lock is held.""" + + preflight = _pilot_control_plane( + repo_root, + work_dir, + credential_provider=StaticCredentialProvider({}), + ) + requested_run_id = str(run_id) + current = preflight.get_run(requested_run_id) + if ( + not isinstance(current, dict) + or current.get("run_id") != requested_run_id + ): + raise ValueError("controlled pilot run identity is invalid") + workflow = preflight.get_workflow(WORKFLOW_ID, WORKFLOW_VERSION) + _validate_controlled_live_binding(workflow, current) + + token = "" + if approved: + if _has_prior_live_connector_failure(preflight, requested_run_id): + raise ValueError( + "controlled pilot requires a new work directory after a live connector failure" + ) + if type(confirmed_live) is not bool or confirmed_live is not True: + raise ValueError("live approval requires explicit boolean confirmation") + if os.environ.get(LIVE_SWITCH) != "1": + raise ValueError("SKILL2WORKFLOW_LARK_TASK_LIVE=1 is required") + token = _resolve_live_lark_token(token_transport=token_transport) + + credentials = {"lark_bot_access_token": token} if approved else {} + control = _pilot_control_plane( + repo_root, + work_dir, + credential_provider=StaticCredentialProvider(credentials), + transport=transport, + ) + state = control.resume_published_run(requested_run_id, approved=approved) + events = control.list_audit_events(run_id=requested_run_id) + connector_events = [ + event + for event in events + if event.get("type") + in ("connector_started", "connector_completed", "connector_failed") + and event.get("node_id") == "create_lark_task" + ] + connector_metadata = {} + terminal_connector_event = connector_events[-1] if connector_events else {} + failure_facts_event = terminal_connector_event + for event in reversed(connector_events): + metadata = event.get("connector_metadata") + if isinstance(metadata, dict): + connector_metadata = metadata + break + if terminal_connector_event.get("type") == "connector_failed": + for index, event in enumerate(events): + if event is not terminal_connector_event: + continue + if index + 1 >= len(events): + break + failure = events[index + 1] + if ( + failure.get("type") == "node_failed" + and failure.get("node_id") == "create_lark_task" + ): + metadata = failure.get("connector_metadata") + if isinstance(metadata, dict): + connector_metadata = metadata + failure_facts_event = failure + break + return { + "run_id": requested_run_id, + "workflow_id": WORKFLOW_ID, + "workflow_version": WORKFLOW_VERSION, + "run_status": str(state.get("status", "")), + "gate_decision": "approved" if approved else "rejected", + "connector_invoked": bool(connector_events), + "connector_status": ( + str(terminal_connector_event.get("connector_status", "")) + if connector_events + else "" + ), + "credential_status": ( + str(failure_facts_event.get("credential_status", "")) + if connector_events + else "" + ), + "provider_status": str(connector_metadata.get("provider_status", "")), + "idempotency_key_present": bool( + connector_metadata.get("idempotency_key_present") + ), + "lark_task_id_present": bool( + connector_metadata.get("lark_task_id_present") + ), + } + + +def _validate_controlled_live_binding( + workflow: Dict[str, object], + run: Dict[str, object], +) -> None: + invalid = "controlled pilot live binding is invalid" + if not isinstance(workflow, dict) or not isinstance(run, dict): + raise ValueError(invalid) + if run.get("status") != "waiting": + raise ValueError("controlled pilot run is not waiting") + + metadata = workflow.get("workflow") + if not isinstance(metadata, dict): + raise ValueError(invalid) + if ( + metadata.get("id") != WORKFLOW_ID + or metadata.get("version") != WORKFLOW_VERSION + or run.get("workflow_id") != WORKFLOW_ID + or run.get("workflow_version") != WORKFLOW_VERSION + ): + raise ValueError(invalid) + if not isinstance(run.get("run_id"), str) or not str(run["run_id"]).strip(): + raise ValueError(invalid) + if run.get("current_node") != "review_renewal_risk": + raise ValueError(invalid) + + expected_workflow = build_lark_task_pilot_workflow( + mode="live", + workflow_id=WORKFLOW_ID, + workflow_version=WORKFLOW_VERSION, + workflow_name="controlled-lark-task-sales-renewal-pilot", + ) + expected_workflow["workflow"]["status"] = "published" + if workflow != expected_workflow: + raise ValueError(invalid) + durable_workflow = run.get("workflow") + if not isinstance(durable_workflow, dict) or durable_workflow != workflow: + raise ValueError(invalid) + + +def start_pilot_run( + repo_root: Path, + work_dir: Path, + input_path: Path, + now: datetime = None, + transport=None, +) -> Dict[str, object]: + repo_root = Path(repo_root).resolve() + work_dir = Path(work_dir).resolve() + _require_outside_repository(repo_root, work_dir, "pilot work directory") + load_pilot_charter(work_dir, now=now) + with _open_unclosed_pilot_session(work_dir): + return _start_pilot_run_open( + repo_root, + work_dir, + input_path, + transport=transport, + ) + + +def _start_pilot_run_open( + repo_root: Path, + work_dir: Path, + input_path: Path, + *, + transport=None, +) -> Dict[str, object]: + """Create one waiting run while the private Pilot decision lock is held.""" + pilot_input = load_private_case(repo_root, input_path) + preflight = _preflight_pilot_input(repo_root, pilot_input) + if preflight["status"] != "ready": + raise ValueError("controlled pilot preflight did not pass") + control = _pilot_control_plane( + repo_root, + work_dir, + credential_provider=StaticCredentialProvider({}), + transport=transport, + ) + workflow = build_lark_task_pilot_workflow( + mode="live", + workflow_id=WORKFLOW_ID, + workflow_version=WORKFLOW_VERSION, + workflow_name="controlled-lark-task-sales-renewal-pilot", + ) + control.publish_workflow(workflow) + response = control.trigger_workflow( + { + "workflow_id": WORKFLOW_ID, + "version": WORKFLOW_VERSION, + "source": "controlled-live-pilot", + "idempotency_key": "", + "input": pilot_input, + } + ) + run = control.get_run(str(response["run_id"])) + if ( + run.get("status") != "waiting" + or run.get("current_node") != "review_renewal_risk" + ): + raise ValueError("controlled pilot run did not stop at the expected human gate") + return { + "run_id": str(response["run_id"]), + "workflow_id": WORKFLOW_ID, + "workflow_version": WORKFLOW_VERSION, + "run_status": "waiting", + "current_node": "review_renewal_risk", + "input_keys": sorted(pilot_input), + "preflight_ready": True, + } + + +def _pilot_control_plane( + repo_root: Path, + work_dir: Path, + credential_provider, + transport=None, +) -> LocalControlPlane: + connector = load_external_connector( + repo_root / "examples" / "connectors" / "lark_task_connector.py" + ) + if transport is not None: + original = connector + + def execute_with_transport(binding, credential_provider=None, context=None): + return original.executor( + binding, + credential_provider=credential_provider, + context=context, + transport=transport, + ) + + connector = ExternalConnector( + manifest=original.manifest, + executor=execute_with_transport, + ) + runtime = ConnectorRuntime([connector]) + return LocalControlPlane( + work_dir / "state", + storage="sqlite", + credential_provider=credential_provider, + connector_runtime=runtime, + ) + + +def initialize_pilot( + repo_root: Path, + work_dir: Path, + charter: Dict[str, object], + now: datetime = None, +) -> Dict[str, object]: + repo_root = Path(repo_root).resolve() + work_dir = Path(os.path.abspath(os.fspath(work_dir))) + _require_outside_repository( + repo_root, + work_dir.resolve(), + "pilot work directory", + ) + normalized = _validate_charter(charter, now=now) + + private_dir = work_dir / "private" + state_dir = work_dir / "state" + evidence_dir = work_dir / "evidence" + charter_path = private_dir / "charter.json" + for directory in (work_dir, private_dir, state_dir, evidence_dir): + ensure_private_directory_anchored(directory) + write_private_json_anchored(charter_path, normalized, require_missing=True) + return { + "status": "initialized", + "scenario_id": SCENARIO_ID, + "workflow_id": WORKFLOW_ID, + "workflow_version": WORKFLOW_VERSION, + "commercial_engagement_confirmed": True, + } + + +def create_private_case_template( + repo_root: Path, + work_dir: Path, + name: str, + case_id: str, + now: datetime = None, +) -> Dict[str, object]: + """Create one blank, owner-only Pilot case without shell-supplied business data.""" + repo_root = Path(repo_root).resolve() + work_dir = Path(os.path.abspath(os.fspath(work_dir))) + _require_outside_repository(repo_root, work_dir.resolve(), "pilot work directory") + normalized_name = _validate_case_template_name(name) + normalized_case_id = _validate_opaque_case_id(case_id) + load_pilot_charter(work_dir, now=now) + with _open_unclosed_pilot_session(work_dir): + cases_dir = work_dir / "private" / "cases" + ensure_private_directory_anchored(cases_dir) + write_private_json_anchored( + cases_dir / f"{normalized_name}.json", + { + "pilot_case_id": normalized_case_id, + "account_name": "", + "renewal_risk": "", + "owner_open_id": "", + "due_at": "", + }, + require_missing=True, + ) + return {"status": "template_written", "field_count": len(REQUIRED_CASE_KEYS)} + + +def load_pilot_charter(work_dir: Path, now: datetime = None) -> Dict[str, object]: + work_dir = Path(os.path.abspath(os.fspath(work_dir))) + path = work_dir / "private" / "charter.json" + payload = read_json_anchored(path) + return _validate_charter(payload, now=now) + + +@contextmanager +def _open_unclosed_pilot_session(work_dir: Path): + """Hold the private decision lock and reject any closed Pilot workspace.""" + with open_private_session(Path(work_dir) / "private") as session: + decision = session.read_json(Path("decision.json"), required=False) + marker = session.read_json(Path("finalization.json"), required=False) + if decision is not None or marker is not None: + raise ValueError("controlled pilot is closed; use a new work directory") + session.check_identity() + yield + session.check_identity() + + +def _validate_case_template_name(value: object) -> str: + if type(value) is not str or not _CASE_TEMPLATE_NAME.fullmatch(value): + raise ValueError("private case template name is invalid") + return value + + +def _validate_opaque_case_id(value: object) -> str: + if type(value) is not str or not value.strip(): + raise ValueError("pilot_case_id must be an opaque identifier") + normalized = value.strip() + if any( + token in normalized.lower() + for token in ("account", "customer", "@", " ") + ): + raise ValueError("pilot_case_id must be an opaque identifier") + return normalized + + +def load_private_case(repo_root: Path, input_path: Path) -> Dict[str, object]: + repo_root = Path(repo_root).resolve() + input_path = Path(os.path.abspath(os.fspath(input_path))) + resolved_input = input_path.resolve() + _require_outside_repository(repo_root, input_path, "private case input") + _require_outside_repository(repo_root, resolved_input, "private case input") + payload = read_json_anchored(input_path, owner_only=True) + if not isinstance(payload, dict) or set(payload) != REQUIRED_CASE_KEYS: + raise ValueError("private case input must contain only the approved fields") + if any( + type(payload.get(key)) is not str or not payload[key].strip() + for key in REQUIRED_CASE_KEYS + ): + raise ValueError("private case input fields must be non-empty strings") + normalized = { + key: payload[key].strip() for key in sorted(REQUIRED_CASE_KEYS) + } + try: + due_at = datetime.fromisoformat( + normalized["due_at"].replace("Z", "+00:00") + ) + except ValueError as error: + raise ValueError( + "private case due_at must be an ISO timestamp with timezone" + ) from error + if due_at.tzinfo is None or due_at.utcoffset() is None: + raise ValueError( + "private case due_at must be an ISO timestamp with timezone" + ) + normalized["pilot_case_id"] = _validate_opaque_case_id( + normalized["pilot_case_id"] + ) + return normalized + + +def preflight_pilot_case(repo_root: Path, input_path: Path) -> Dict[str, object]: + """Validate one private case through the exact live payload builder only.""" + repo_root = Path(repo_root).resolve() + pilot_input = load_private_case(repo_root, input_path) + return _preflight_pilot_input(repo_root, pilot_input) + + +def _preflight_pilot_input( + repo_root: Path, + pilot_input: Dict[str, object], +) -> Dict[str, object]: + """Construct the exact live payload from an already validated private input.""" + workflow = build_lark_task_pilot_workflow( + mode="live", + workflow_id=WORKFLOW_ID, + workflow_version=WORKFLOW_VERSION, + workflow_name="controlled-lark-task-sales-renewal-pilot", + ) + workflow["workflow"]["status"] = "published" + _validate_controlled_live_binding( + workflow, + { + "status": "waiting", + "workflow_id": WORKFLOW_ID, + "workflow_version": WORKFLOW_VERSION, + "run_id": "preflight", + "current_node": "review_renewal_risk", + "workflow": workflow, + }, + ) + node = next( + item + for item in workflow.get("nodes", []) + if isinstance(item, dict) and item.get("id") == "create_lark_task" + ) + binding = node.get("connector") + connector = load_external_connector( + repo_root / "examples" / "connectors" / "lark_task_connector.py" + ) + if not callable(connector.preflight): + raise ValueError("controlled pilot connector does not provide a local preflight") + raw_result = connector.preflight( + binding, + context={ + "input": pilot_input, + "_execution": { + "workflow_id": WORKFLOW_ID, + "workflow_version": WORKFLOW_VERSION, + "run_id": "preflight", + "node_id": "create_lark_task", + }, + }, + ) + if not isinstance(raw_result, dict): + raise ValueError("controlled pilot connector preflight result is invalid") + output = raw_result.get("output", {}) + audit = raw_result.get("audit", {}) + if ( + not isinstance(output, dict) + or not isinstance(audit, dict) + or raw_result.get("status") not in ("ready", "invalid") + ): + raise ValueError("controlled pilot connector preflight result is invalid") + return { + "status": str(raw_result["status"]), + "connector_id": "lark_task", + "operation": str(output.get("operation", "")), + "mode": str(output.get("mode", "")), + "task_title_present": bool(audit.get("task_title_present")), + "task_description_present": bool(audit.get("task_description_present")), + "assignee_present": bool(audit.get("assignee_present")), + "due_at_present": bool(audit.get("due_at_present")), + "provider_payload_constructed": bool( + output.get("provider_payload_constructed") + ), + "credential_resolution_attempted": bool( + output.get("credential_resolution_attempted") + ), + "network_called": bool(output.get("network_called")), + } + + +def _validate_charter(charter: object, now: datetime = None) -> Dict[str, object]: + normalized = _validate_charter_schema_and_range(charter) + _require_active_charter_window(normalized, now=now) + return normalized + + +def _validate_charter_schema_and_range(charter: object) -> Dict[str, object]: + if not isinstance(charter, dict): + raise ValueError("pilot charter must be a JSON object") + if set(charter) != REQUIRED_CHARTER_KEYS: + raise ValueError("pilot charter must contain only the approved fields") + normalized = json.loads(json.dumps(charter, ensure_ascii=False)) + required_exact = { + "schema_version": PILOT_SCHEMA_VERSION, + "scenario_id": SCENARIO_ID, + "workflow_id": WORKFLOW_ID, + "workflow_version": WORKFLOW_VERSION, + "support_model": "assisted", + "timezone": PILOT_TIMEZONE, + "required_approved_runs": 5, + "required_distinct_days": 5, + "required_distinct_cases": 2, + } + for key, expected in required_exact.items(): + if normalized.get(key) != expected: + raise ValueError(f"pilot charter {key} must be {expected}") + for key in ( + "team_consent_confirmed", + "assignee_consent_confirmed", + "commercial_engagement_confirmed", + ): + if normalized.get(key) is not True: + raise ValueError(f"pilot charter {key} must be true") + starts_on = date.fromisoformat(str(normalized.get("starts_on", ""))) + expires_on = date.fromisoformat(str(normalized.get("expires_on", ""))) + if starts_on > expires_on: + raise ValueError("pilot charter date range is invalid") + return normalized + + +def _require_active_charter_window( + charter: Dict[str, object], + now: datetime = None, +) -> None: + starts_on = date.fromisoformat(charter["starts_on"]) + expires_on = date.fromisoformat(charter["expires_on"]) + current = (now or datetime.now(timezone.utc)).astimezone( + ZoneInfo(PILOT_TIMEZONE) + ).date() + if current < starts_on: + raise ValueError("pilot charter has not started") + if current > expires_on: + raise ValueError("pilot charter expired") + + +def _validate_historical_repository_charter( + charter: object, + decision: object, + marker: object, + now: datetime = None, +) -> Dict[str, object]: + normalized = _validate_charter_schema_and_range(charter) + starts_on = date.fromisoformat(normalized["starts_on"]) + expires_on = date.fromisoformat(normalized["expires_on"]) + current = (now or datetime.now(timezone.utc)).astimezone( + ZoneInfo(PILOT_TIMEZONE) + ).date() + if current < starts_on: + raise ValueError("pilot charter has not started") + if current <= expires_on: + return normalized + + normalized_decision = validate_final_decision(decision) + finalized_at = _validate_finalization_marker(marker) + if marker["decision"] != normalized_decision["decision"]: + raise ValueError("private finalization marker is invalid") + finalized_on = finalized_at.astimezone(ZoneInfo(PILOT_TIMEZONE)).date() + if not starts_on <= finalized_on <= expires_on: + raise ValueError( + "private finalization does not authorize an expired pilot charter" + ) + return normalized + + +def _require_outside_repository(repo_root: Path, path: Path, label: str) -> None: + if path == repo_root or repo_root in path.parents: + raise ValueError(f"{label} must be outside the repository") diff --git a/src/skill2workflow/controlled_lark_pilot_evidence.py b/src/skill2workflow/controlled_lark_pilot_evidence.py new file mode 100644 index 0000000..4ba871d --- /dev/null +++ b/src/skill2workflow/controlled_lark_pilot_evidence.py @@ -0,0 +1,642 @@ +"""Public facade and pure builders for controlled Lark pilot evidence.""" + +from __future__ import annotations + +from datetime import date, datetime +from pathlib import Path +from typing import Dict, List, Set +from zoneinfo import ZoneInfo + +from ._controlled_lark_pilot_evidence_validation import ( + CHARTER_KEYS, + CONNECTOR_ID, + CREDENTIAL_HANDLE, + DECISION_KEYS, + DECISION_SCHEMA_VERSION, + EVIDENCE_SCHEMA_VERSION, + EXERCISE_SCHEMA_VERSION, + EXERCISE_SLOT_KEYS, + FAILURE_EXERCISE_KEYS, + INDEX_KEYS, + INDEX_SCHEMA_VERSION, + MODE, + OPERATION, + PRESENCE_FIELDS, + PROVIDER_STATUSES, + REJECTION_EXERCISE_KEYS, + ROLLBACK_EXERCISE_KEYS, + RUN_EVIDENCE_KEYS, + TIMEZONE, + TOP_LEVEL_KEYS, + UNMET_CONDITIONS, + VERIFICATION_COMMAND_IDS, + VERIFICATION_COMMAND_KEYS, + VERIFICATION_KEYS, + VERIFICATION_SCHEMA_VERSION, + WORKFLOW_ID, + WORKFLOW_VERSION, + aware_datetime as _aware_datetime, + validate_evidence_pack as _validate_evidence_pack, + validate_run as _validate_run, +) +from ._controlled_lark_pilot_evidence_writer import ( + prepare_evidence_pack as _prepare_evidence_pack, + write_evidence_pack as _write_evidence_pack, +) + + +CONNECTOR_EVENT_TYPES = ( + "connector_started", + "connector_failed", + "connector_completed", +) +TERMINAL_EVENT_TYPES = ("run_completed", "run_failed", "run_rejected") +FAILED_METADATA_KEYS = set(PRESENCE_FIELDS) | { + "operation", + "mode", + "provider_status", + "credential_status", +} + + +def _indexed_events(events: List[Dict[str, object]], event_types: tuple) -> list: + return [ + (index, event) + for index, event in enumerate(events) + if isinstance(event, dict) and event.get("type") in event_types + ] + + +def _raw_string(value: object, label: str, nonempty: bool = False) -> str: + if type(value) is not str or (nonempty and not value.strip()): + suffix = " nonempty" if nonempty else "" + raise ValueError(f"{label} must be a{suffix} string") + return value + + +def _bound_event(event: object, run_id: str, label: str) -> datetime: + if not isinstance(event, dict) or event.get("run_id") != run_id: + raise ValueError(f"{label} must be bound to the controlled run") + return _aware_datetime(event.get("timestamp"), f"{label} timestamp") + + +def _bound_event_in_window( + event: object, + run_id: str, + label: str, + started_at: datetime, + completed_at: datetime, +) -> datetime: + timestamp = _bound_event(event, run_id, label) + if timestamp < started_at or timestamp > completed_at: + raise ValueError(f"{label} timestamp must be within the run interval") + return timestamp + + +def _validate_connector_sequence( + candidates: list, + retrying: list, + run_id: str, + resume_index: int, + terminal_index: int, + started_at: datetime, + completed_at: datetime, +) -> Dict[str, object]: + completed = {} + attempt_open = False + for index, event in candidates: + if event.get("node_id") != "create_lark_task": + raise ValueError("connector event must target the controlled node") + if not resume_index < index < terminal_index: + raise ValueError("connector event is out of semantic audit order") + event_type = event["type"] + _bound_event_in_window( + event, + run_id, + event_type, + started_at, + completed_at, + ) + connector_id = _raw_string(event.get("connector_id"), "connector id") + if connector_id != CONNECTOR_ID: + raise ValueError("connector event identity is invalid") + connector_status = _raw_string( + event.get("connector_status"), "connector status" + ) + expected_status = { + "connector_started": "running", + "connector_failed": "failed", + "connector_completed": "completed", + }[event_type] + if connector_status != expected_status: + raise ValueError(f"{event_type} has an invalid connector status") + if completed: + raise ValueError("connector events must not follow connector completion") + if event_type == "connector_started": + if attempt_open: + raise ValueError("connector attempt cannot start twice") + attempt_open = True + elif event_type == "connector_failed": + if not attempt_open: + raise ValueError("connector failure must follow a started attempt") + attempt_open = False + else: + if not attempt_open: + raise ValueError("connector completion must follow a started attempt") + attempt_open = False + completed = event + if attempt_open: + raise ValueError("connector attempt is missing a terminal connector event") + + expected_retry_pairs = { + (previous[0], following[0]) + for previous, following in zip(candidates, candidates[1:]) + if previous[1].get("type") == "connector_failed" + and following[1].get("type") == "connector_started" + } + retry_pairs = set() + for index, event in retrying: + if event.get("node_id") != "create_lark_task": + raise ValueError("retry event must target the controlled node") + if not resume_index < index < terminal_index: + raise ValueError("retry event is out of semantic audit order") + _bound_event_in_window( + event, + run_id, + "node_retrying", + started_at, + completed_at, + ) + previous = [item for item in candidates if item[0] < index] + following = [item for item in candidates if item[0] > index] + if ( + not previous + or previous[-1][1].get("type") != "connector_failed" + or not following + or following[0][1].get("type") != "connector_started" + ): + raise ValueError("node_retrying must separate failed and started attempts") + pair = (previous[-1][0], following[0][0]) + if pair in retry_pairs: + raise ValueError("connector retry transition must not be duplicated") + retry_pairs.add(pair) + if retry_pairs != expected_retry_pairs: + raise ValueError("connector retry events must exactly match retry transitions") + return completed + + +def _validate_failed_node_facts( + candidates: list, + connector_events: list, + run_id: str, + terminal_index: int, + started_at: datetime, + completed_at: datetime, +) -> Dict[str, object]: + if len(candidates) != 1: + raise ValueError("approved connector failure requires one node_failed event") + index, event = candidates[0] + if event.get("node_id") != "create_lark_task": + raise ValueError("node_failed must target the controlled node") + timestamp = _bound_event_in_window( + event, + run_id, + "node_failed", + started_at, + completed_at, + ) + if not connector_events: + raise ValueError("node_failed requires a connector failure") + connector_index, connector_failure = connector_events[-1] + if ( + connector_failure.get("type") != "connector_failed" + or index != connector_index + 1 + or not index < terminal_index + ): + raise ValueError("node_failed must follow the final connector failure") + connector_timestamp = _bound_event_in_window( + connector_failure, + run_id, + "connector_failed", + started_at, + completed_at, + ) + if timestamp < connector_timestamp: + raise ValueError("node_failed must not precede connector failure") + + metadata = event.get("connector_metadata") + if not isinstance(metadata, dict) or set(metadata) != FAILED_METADATA_KEYS: + raise ValueError("failed connector metadata keys do not match the allowlist") + for field in PRESENCE_FIELDS: + if type(metadata.get(field)) is not bool: + raise ValueError(f"connector presence field {field} must be a boolean") + for key in ("operation", "mode", "provider_status", "credential_status"): + _raw_string(metadata.get(key), f"failed connector {key}") + + credential_status = _raw_string( + event.get("credential_status"), + "failed connector credential status", + ) + if credential_status != metadata["credential_status"]: + raise ValueError("failed connector credential status is inconsistent") + handles = event.get("credential_handles") + if type(handles) is not list or any(type(item) is not str for item in handles): + raise ValueError("credential handles must be a list of strings") + return event + + +def build_run_evidence( + run: Dict[str, object], audit_events: List[Dict[str, object]] +) -> Dict[str, object]: + if not isinstance(run, dict): + raise ValueError("run must be an object") + if not isinstance(audit_events, list): + raise ValueError("audit events must be a list") + run_id = _raw_string(run.get("run_id"), "run_id", nonempty=True) + workflow_id = _raw_string(run.get("workflow_id"), "workflow_id") + workflow_version = _raw_string(run.get("workflow_version"), "workflow_version") + run_status = _raw_string(run.get("status"), "run status") + + started_events = _indexed_events(audit_events, ("run_started",)) + if len(started_events) != 1: + raise ValueError("exactly one run_started event is required") + started_index, started_event = started_events[0] + started_at = _bound_event(started_event, run_id, "run_started") + + terminal_events = _indexed_events(audit_events, TERMINAL_EVENT_TYPES) + expected_terminal = ( + "run_completed" + if run_status == "completed" + else "run_failed" + if run_status in ("failed", "rejected") + else "" + ) + if expected_terminal: + if ( + len(terminal_events) != 1 + or terminal_events[0][1].get("type") != expected_terminal + ): + raise ValueError("terminal event does not match the run status") + terminal_index, terminal = terminal_events[0] + completed_at = _bound_event(terminal, run_id, expected_terminal) + if terminal_index <= started_index or completed_at < started_at: + raise ValueError("terminal timestamp must not precede run_started") + else: + if terminal_events: + raise ValueError("nonterminal run must not have a terminal event") + terminal_index = len(audit_events) + completed_at = started_at + terminal = {} + + resumed_events = _indexed_events(audit_events, ("run_resumed",)) + connector_events = _indexed_events(audit_events, CONNECTOR_EVENT_TYPES) + retrying_events = _indexed_events(audit_events, ("node_retrying",)) + node_failed_events = _indexed_events(audit_events, ("node_failed",)) + if expected_terminal: + if len(resumed_events) != 1: + raise ValueError("exactly one run_resumed event is required") + resume_index, resumed = resumed_events[0] + _bound_event_in_window( + resumed, + run_id, + "run_resumed", + started_at, + completed_at, + ) + if not started_index < resume_index < terminal_index: + raise ValueError("run_resumed is out of semantic audit order") + if type(resumed.get("approved")) is not bool: + raise ValueError("run_resumed approved must be a boolean") + else: + if resumed_events or connector_events or retrying_events or node_failed_events: + raise ValueError("waiting run must not contain decision or connector events") + resume_index = terminal_index + resumed = {} + + completed_connector = ( + _validate_connector_sequence( + connector_events, + retrying_events, + run_id, + resume_index, + terminal_index, + started_at, + completed_at, + ) + if connector_events or retrying_events + else {} + ) + if resumed.get("approved") is False: + if ( + run_status not in ("failed", "rejected") + or connector_events + or node_failed_events + ): + raise ValueError("rejected run must fail without connector events") + elif resumed.get("approved") is True: + if run_status == "completed": + if not completed_connector or node_failed_events: + raise ValueError("completed approved run requires connector completion") + elif run_status == "failed": + if not connector_events or completed_connector: + raise ValueError("failed approved run requires connector failure") + else: + raise ValueError("approved run status is invalid") + + connector = completed_connector or ( + connector_events[-1][1] if connector_events else {} + ) + facts_event = completed_connector + if resumed.get("approved") is True and run_status == "failed": + facts_event = _validate_failed_node_facts( + node_failed_events, + connector_events, + run_id, + terminal_index, + started_at, + completed_at, + ) + metadata = facts_event.get("connector_metadata", {}) if facts_event else {} + context = run.get("context", {}) + if not isinstance(context, dict): + context = {} + trigger_input = context.get("input", {}) + if not isinstance(trigger_input, dict): + trigger_input = {} + case_id = trigger_input.get("pilot_case_id") + if type(case_id) is not str or not case_id.strip(): + raise ValueError("pilot_case_id must be a nonempty string") + + raw_handles = facts_event.get("credential_handles", []) if facts_event else [] + if type(raw_handles) is not list or any( + type(item) is not str for item in raw_handles + ): + raise ValueError("credential handles must be a list of strings") + handles = list(raw_handles) + if facts_event: + if not isinstance(metadata, dict): + raise ValueError("connector metadata must be an object") + for field in PRESENCE_FIELDS: + if type(metadata.get(field)) is not bool: + raise ValueError(f"connector presence field {field} must be a boolean") + operation = _raw_string(metadata.get("operation"), "connector operation") + mode = _raw_string(metadata.get("mode"), "connector mode") + provider_status = _raw_string( + metadata.get("provider_status"), "provider status" + ) + connector_id = _raw_string(connector.get("connector_id"), "connector id") + connector_status = _raw_string( + connector.get("connector_status"), "connector status" + ) + credential_status = _raw_string( + facts_event.get("credential_status"), "credential status" + ) + elif connector: + operation = mode = provider_status = "" + connector_id = _raw_string(connector.get("connector_id"), "connector id") + connector_status = _raw_string( + connector.get("connector_status"), "connector status" + ) + credential_status = ( + _raw_string(connector.get("credential_status"), "credential status") + if "credential_status" in connector + else "" + ) + else: + operation = mode = provider_status = "" + connector_id = connector_status = credential_status = "" + + evidence = { + "schema_version": EVIDENCE_SCHEMA_VERSION, + "run_id": run_id, + "workflow_id": workflow_id, + "workflow_version": workflow_version, + "started_at": started_event["timestamp"], + "completed_at": terminal.get("timestamp", ""), + "run_status": run_status, + "gate_decision": ( + "approved" + if resumed.get("approved") is True + else "rejected" + if resumed.get("approved") is False + else "pending" + ), + "case_id_present": True, + "connector_invoked": bool(connector), + "connector_id": connector_id, + "connector_status": connector_status, + "credential_status": credential_status, + "credential_handles": handles, + "operation": operation, + "mode": mode, + "provider_status": provider_status, + **{ + field: metadata[field] if facts_event else False + for field in PRESENCE_FIELDS + }, + } + _validate_run(evidence) + if run_status == "completed" and not _is_approved_live_run(evidence): + raise ValueError("completed approved run has invalid success facts") + return evidence + + +def _is_approved_live_run(run: object) -> bool: + if not isinstance(run, dict): + return False + expected = { + "schema_version": EVIDENCE_SCHEMA_VERSION, + "workflow_id": WORKFLOW_ID, + "workflow_version": WORKFLOW_VERSION, + "run_status": "completed", + "gate_decision": "approved", + "case_id_present": True, + "connector_invoked": True, + "connector_id": CONNECTOR_ID, + "connector_status": "completed", + "credential_status": "resolved", + "credential_handles": [CREDENTIAL_HANDLE], + "operation": OPERATION, + "mode": MODE, + "provider_status": "completed", + "task_title_present": True, + "task_description_present": True, + "assignee_present": True, + "due_at_present": True, + "idempotency_key_present": True, + "lark_task_id_present": True, + } + if any(run.get(key) != value for key, value in expected.items()): + return False + try: + _aware_datetime(run.get("completed_at"), "completed_at") + except ValueError: + return False + return True + + +def _is_human_rejection(run: object) -> bool: + if not ( + isinstance(run, dict) + and run.get("workflow_id") == WORKFLOW_ID + and run.get("workflow_version") == WORKFLOW_VERSION + and run.get("run_status") in ("failed", "rejected") + and run.get("gate_decision") == "rejected" + and run.get("connector_invoked") is False + ): + return False + try: + started = _aware_datetime(run.get("started_at"), "started_at") + completed = _aware_datetime(run.get("completed_at"), "completed_at") + except ValueError: + return False + return completed >= started + + +def _rejection_exercise(runs: List[Dict[str, object]]): + for run in runs: + if _is_human_rejection(run): + return { + "schema_version": EXERCISE_SCHEMA_VERSION, + "exercise": "rejection", + "passed": True, + "run_id": run["run_id"], + "gate_decision": "rejected", + "connector_invoked": False, + } + return None + + +def _run_sort_key(run: Dict[str, object]) -> tuple: + return ( + _aware_datetime(run.get("started_at"), "started_at"), + run.get("run_id", ""), + ) + + +def _qualified_exercise(exercises: object, name: str) -> bool: + if not isinstance(exercises, dict): + return False + exercise = exercises.get(name) + if not isinstance(exercise, dict) or exercise.get("passed") is not True: + return False + if name == "failure" and len(exercise) > 1: + return bool( + exercise.get("provider_status") == "live_disabled" + and exercise.get("credential_resolution_attempted") is False + and exercise.get("transport_attempted") is False + ) + if name == "rollback" and len(exercise) > 1: + return bool( + exercise.get("live_switch_enabled") is False + and exercise.get("live_approval_blocked") is True + and exercise.get("dry_run_status") == "completed" + ) + return True + + +def build_acceptance_summary( + charter: Dict[str, object], + runs: List[Dict[str, object]], + distinct_private_cases: int, + exercises: Dict[str, object], + verification: Dict[str, object], + decision: Dict[str, object], +) -> Dict[str, object]: + if type(distinct_private_cases) is not int or distinct_private_cases < 0: + raise ValueError("distinct private cases must be a nonnegative integer") + if not isinstance(charter, dict) or not isinstance(runs, list): + raise ValueError("charter and runs must use the expected containers") + approved = [run for run in runs if _is_approved_live_run(run)] + days: Set[date] = set() + for run in approved: + completed = _aware_datetime(run["completed_at"], "completed_at") + days.add(completed.astimezone(ZoneInfo(TIMEZONE)).date()) + rejected_runs = sum(1 for run in runs if _is_human_rejection(run)) + failure_passed = _qualified_exercise(exercises, "failure") + rollback_passed = _qualified_exercise(exercises, "rollback") + verification_passed = bool( + isinstance(verification, dict) and verification.get("all_passed") is True + ) + decision_recorded = bool( + isinstance(decision, dict) + and decision.get("decision") in ("continue", "harden", "defer") + and type(decision.get("rationale")) is str + and bool(decision["rationale"].strip()) + ) + partner_acknowledged = bool( + decision_recorded and decision.get("partner_acknowledged") is True + ) + operator_acknowledged = bool( + decision_recorded and decision.get("operator_acknowledged") is True + ) + charter_commercial = charter.get("commercial_engagement_confirmed") is True + commercial_confirmed = bool( + charter_commercial + and ( + decision is None + or ( + decision_recorded + and decision.get("commercial_engagement_confirmed") is True + ) + ) + ) + predicates = ( + len(approved) >= charter.get("required_approved_runs", 0), + len(days) >= charter.get("required_distinct_days", 0), + distinct_private_cases >= charter.get("required_distinct_cases", 0), + rejected_runs >= 1, + failure_passed, + rollback_passed, + verification_passed, + decision_recorded, + partner_acknowledged, + operator_acknowledged, + commercial_confirmed, + ) + unmet = [name for name, passed in zip(UNMET_CONDITIONS, predicates) if not passed] + return { + "approved_live_runs": len(approved), + "required_approved_runs": charter.get("required_approved_runs", 0), + "distinct_calendar_days": len(days), + "required_distinct_days": charter.get("required_distinct_days", 0), + "distinct_private_cases": distinct_private_cases, + "required_distinct_cases": charter.get("required_distinct_cases", 0), + "rejected_runs": rejected_runs, + "rejection_passed": rejected_runs >= 1, + "failure_passed": failure_passed, + "rollback_passed": rollback_passed, + "verification_passed": verification_passed, + "decision_recorded": decision_recorded, + "decision": str(decision.get("decision", "")) if decision_recorded else "", + "partner_acknowledged": partner_acknowledged, + "operator_acknowledged": operator_acknowledged, + "commercial_engagement_confirmed": commercial_confirmed, + "ready_to_finalize": not unmet, + "unmet_conditions": unmet, + } + + +def validate_evidence_pack( + pack: Dict[str, object], forbidden_values: List[str] +) -> None: + _validate_evidence_pack( + pack, + forbidden_values, + _run_sort_key, + _rejection_exercise, + build_acceptance_summary, + ) + + +def write_evidence_pack(output_dir: Path, pack: Dict[str, object]) -> Dict[str, object]: + validate_evidence_pack(pack, []) + return _write_evidence_pack(output_dir, pack) + + +def prepare_evidence_pack_transaction( + output_dir: Path, + pack: Dict[str, object], +): + validate_evidence_pack(pack, []) + return _prepare_evidence_pack(output_dir, pack) diff --git a/src/skill2workflow/executor.py b/src/skill2workflow/executor.py index 601e505..801b8ea 100644 --- a/src/skill2workflow/executor.py +++ b/src/skill2workflow/executor.py @@ -209,7 +209,7 @@ def _execute_connector_node( connector_result = self.connector_runtime.execute_connector( node, credential_provider=self.credential_provider, - context=state.get("context", {}), + context=_connector_context(state, current_id), ) except ConnectorExecutionError as error: connector_result = { @@ -353,6 +353,18 @@ def _node_map(workflow: Dict[str, object]) -> Dict[str, Dict[str, object]]: return {node["id"]: node for node in workflow.get("nodes", [])} +def _connector_context(state: RunState, node_id: str) -> Dict[str, object]: + durable = state.get("context", {}) + context = copy.deepcopy(durable) if isinstance(durable, dict) else {} + context["_execution"] = { + "workflow_id": str(state.get("workflow_id", "")), + "workflow_version": str(state.get("workflow_version", "")), + "run_id": str(state.get("run_id", "")), + "node_id": str(node_id), + } + return context + + def _now() -> str: return datetime.now(timezone.utc).isoformat() diff --git a/src/skill2workflow/external_connectors.py b/src/skill2workflow/external_connectors.py index bea816f..af6b237 100644 --- a/src/skill2workflow/external_connectors.py +++ b/src/skill2workflow/external_connectors.py @@ -23,9 +23,12 @@ def load_external_connector(path: Path) -> ExternalConnector: manifest = getattr(module, "MANIFEST", None) executor = getattr(module, "execute", None) + preflight = getattr(module, "preflight", None) if manifest is None: raise ValueError(f"external connector fixture must define MANIFEST: {path}") if executor is None: raise ValueError(f"external connector fixture must define execute: {path}") + if preflight is not None and not callable(preflight): + raise ValueError(f"external connector fixture preflight must be callable: {path}") - return ExternalConnector(manifest=manifest, executor=executor) + return ExternalConnector(manifest=manifest, executor=executor, preflight=preflight) diff --git a/src/skill2workflow/lark_task_live_validation.py b/src/skill2workflow/lark_task_live_validation.py new file mode 100644 index 0000000..988af86 --- /dev/null +++ b/src/skill2workflow/lark_task_live_validation.py @@ -0,0 +1,128 @@ +"""Guarded one-shot validation for the scoped live Lark task connector.""" + +from __future__ import annotations + +import argparse +import json +import os +from pathlib import Path +from typing import Dict + +from .connectors import ConnectorRuntime, ExternalConnector +from .credentials import StaticCredentialProvider +from .external_connectors import load_external_connector + + +LIVE_SWITCH = "SKILL2WORKFLOW_LARK_TASK_LIVE" +TOKEN_ENVIRONMENT = "LARK_BOT_ACCESS_TOKEN" + + +def run_lark_task_live_validation( + repo_root: Path, + title: str, + description: str, + assignee_open_id: str, + validation_run_id: str, + confirmed: bool, + transport=None, +) -> Dict[str, object]: + if not confirmed: + raise ValueError("live validation requires --confirm-live-create") + if os.environ.get(LIVE_SWITCH) != "1": + raise ValueError("SKILL2WORKFLOW_LARK_TASK_LIVE=1 is required") + token = os.environ.get(TOKEN_ENVIRONMENT, "") + if not token: + raise ValueError("LARK_BOT_ACCESS_TOKEN is required") + if not validation_run_id: + raise ValueError("validation run id is required") + if not assignee_open_id: + raise ValueError("assignee open id is required") + + connector = load_external_connector(Path(repo_root) / "examples" / "connectors" / "lark_task_connector.py") + if transport is not None: + original = connector + + def execute_with_transport(binding, credential_provider=None, context=None): + return original.executor( + binding, + credential_provider=credential_provider, + context=context, + transport=transport, + ) + + connector = ExternalConnector(manifest=original.manifest, executor=execute_with_transport) + + runtime = ConnectorRuntime([connector]) + result = runtime.execute_connector( + _validation_node(title, description, assignee_open_id), + credential_provider=StaticCredentialProvider({"lark_bot_access_token": token}), + context={ + "_execution": { + "workflow_id": "workflow_lark_task_live_validation", + "workflow_version": "0.1.0", + "run_id": validation_run_id, + "node_id": "create_lark_task", + } + }, + ) + audit = result.get("audit", {}) if isinstance(result.get("audit"), dict) else {} + return { + "ok": result.get("status") == "completed", + "connector_id": str(result.get("connector", {}).get("id", "")), + "operation": str(audit.get("operation", "")), + "mode": str(audit.get("mode", "")), + "credential_status": str(audit.get("credential_status", "")), + "idempotency_key_present": bool(audit.get("idempotency_key_present")), + "provider_status": str(audit.get("provider_status", "")), + "lark_task_id_present": bool(audit.get("lark_task_id_present")), + "assignee_present": bool(audit.get("assignee_present")), + } + + +def _validation_node(title: str, description: str, assignee_open_id: str) -> Dict[str, object]: + return { + "id": "create_lark_task", + "type": "tool_call", + "connector": { + "id": "lark_task", + "kind": "lark_task", + "operation": "create_task", + "mode": "live", + "request": { + "body": { + "title": title, + "description": description, + "assignee_open_id": assignee_open_id, + } + }, + "credentials": [ + { + "target": "header", + "name": "Authorization", + "handle": "lark_bot_access_token", + "prefix": "Bearer ", + } + ], + }, + } + + +def main(argv=None) -> int: + parser = argparse.ArgumentParser(prog="lark_task_live_validation") + parser.add_argument("--repo-root", type=Path, default=Path(__file__).resolve().parents[2]) + parser.add_argument("--title", required=True) + parser.add_argument("--description", required=True) + parser.add_argument("--assignee-open-id", required=True) + parser.add_argument("--validation-run-id", required=True) + parser.add_argument("--confirm-live-create", action="store_true") + args = parser.parse_args(argv) + result = run_lark_task_live_validation( + args.repo_root, + title=args.title, + description=args.description, + assignee_open_id=args.assignee_open_id, + validation_run_id=args.validation_run_id, + confirmed=args.confirm_live_create, + ) + print(json.dumps(result, ensure_ascii=False, indent=2)) + return 0 if result["ok"] else 1 diff --git a/src/skill2workflow/lark_task_pilot.py b/src/skill2workflow/lark_task_pilot.py index b4efa88..20443e2 100644 --- a/src/skill2workflow/lark_task_pilot.py +++ b/src/skill2workflow/lark_task_pilot.py @@ -45,7 +45,7 @@ def run_lark_task_pilot( runtime = ConnectorRuntime([external_connector]) connector_ids = [manifest["id"] for manifest in runtime.list_connectors()] - workflow = _lark_task_pilot_workflow() + workflow = build_lark_task_pilot_workflow() errors = validate_workflow(workflow) if errors: raise ValueError("; ".join(errors)) @@ -158,14 +158,26 @@ def _pilot_trigger_input() -> Dict[str, object]: } -def _lark_task_pilot_workflow() -> Dict[str, object]: +def build_lark_task_pilot_workflow( + mode: str = "dry_run", + workflow_id: str = "workflow_lark_task_pilot", + workflow_version: str = "0.1.0", + workflow_name: str = "lark-task-sales-renewal-pilot", +) -> Dict[str, object]: + if mode not in ("dry_run", "live"): + raise ValueError("mode must be dry_run or live") + live = mode == "live" return { "schema_version": "0.1.0", "workflow": { - "id": "workflow_lark_task_pilot", - "name": "lark-task-sales-renewal-pilot", - "description": "Local sales renewal risk pilot using the Lark/Feishu task dry-run connector.", - "version": "0.1.0", + "id": workflow_id, + "name": workflow_name, + "description": ( + "Controlled sales renewal risk pilot using the scoped live Lark/Feishu task connector." + if live + else "Local sales renewal risk pilot using the Lark/Feishu task dry-run connector." + ), + "version": workflow_version, "status": "draft", }, "entry": "start", @@ -194,17 +206,25 @@ def _lark_task_pilot_workflow() -> Dict[str, object]: "id": "create_lark_task", "type": "tool_call", "title": "Create owner follow-up task", - "description": "Validate a Lark/Feishu owner follow-up task request without calling the live API.", + "description": ( + "Create the approved Lark/Feishu owner follow-up task through the scoped live connector." + if live + else "Validate a Lark/Feishu owner follow-up task request without calling the live API." + ), "action": { "kind": "tool_call", - "instruction": "Create a dry-run Lark/Feishu task for the account owner.", + "instruction": ( + "Create the approved live Lark/Feishu task for the account owner." + if live + else "Create a dry-run Lark/Feishu task for the account owner." + ), }, "retry": {"max_attempts": 0}, "connector": { "id": "lark_task", "kind": "lark_task", "operation": "create_task", - "mode": "dry_run", + "mode": mode, "request": { "body": { "source": "skill2workflow-lark-task-pilot", diff --git a/tests/test_controlled_lark_pilot.py b/tests/test_controlled_lark_pilot.py new file mode 100644 index 0000000..3cf60d5 --- /dev/null +++ b/tests/test_controlled_lark_pilot.py @@ -0,0 +1,2175 @@ +import json +import os +import subprocess +import sys +from collections.abc import MutableMapping +from copy import deepcopy +from datetime import datetime, timezone +from pathlib import Path +from tempfile import TemporaryDirectory +from types import SimpleNamespace +from unittest import TestCase, skipUnless +from unittest.mock import patch + +from skill2workflow.compiler import validate_workflow +from skill2workflow.control_plane import LocalControlPlane +from skill2workflow.controlled_lark_pilot import ( + _validate_controlled_live_binding, + create_private_case_template, + decide_pilot_run, + exercise_disabled_live, + exercise_rollback, + generate_pilot_evidence, + initialize_pilot, + load_pilot_charter, + load_private_case, + preflight_pilot_case, + start_pilot_run, + verify_pilot, +) +from skill2workflow.credentials import StaticCredentialProvider +from skill2workflow.lark_task_pilot import build_lark_task_pilot_workflow + + +ROOT = Path(__file__).resolve().parents[1] +NOW = datetime(2026, 7, 18, 9, 0, tzinfo=timezone.utc) + + +class _FakeResponse: + status = 200 + + def read(self): + return json.dumps( + { + "code": 0, + "msg": "private-provider-message", + "data": {"task": {"guid": "private-task-guid"}}, + } + ).encode("utf-8") + + def close(self): + return None + + +class _FakeTransport: + def __init__(self): + self.calls = [] + + def __call__(self, request, timeout): + self.calls.append((request, timeout)) + return _FakeResponse() + + +class _ValidationFailureResponse: + status = 400 + + def read(self): + return b"{}" + + def close(self): + return None + + +class _ValidationFailureTransport: + def __init__(self): + self.calls = [] + + def __call__(self, request, timeout): + self.calls.append((request, timeout)) + return _ValidationFailureResponse() + + +class _FakeTokenResponse: + status = 200 + + def __init__(self, payload): + self._payload = payload + self.closed = False + + def read(self): + return json.dumps(self._payload).encode("utf-8") + + def close(self): + self.closed = True + + +class _FakeTokenTransport: + def __init__(self, payload=None): + self.payload = payload or { + "code": 0, + "tenant_access_token": "fresh-private-tenant-token", + "expire": 7200, + } + self.calls = [] + self.responses = [] + + def __call__(self, request, timeout): + self.calls.append((request, timeout)) + response = _FakeTokenResponse(self.payload) + self.responses.append(response) + return response + + +class _FakeCommandResult: + def __init__(self, returncode=0): + self.returncode = returncode + self.stdout = "private-command-stdout" + self.stderr = "private-command-stderr" + + +class _FakeCommandRunner: + def __init__(self, exit_codes=None): + self.exit_codes = list(exit_codes or [0] * 7) + self.arguments = [] + self.environments = [] + self.working_directories = [] + self.capture_output = [] + + def __call__(self, arguments, *, cwd, env, capture_output): + self.arguments.append(list(arguments)) + self.environments.append(dict(env)) + self.working_directories.append(cwd) + self.capture_output.append(capture_output) + return _FakeCommandResult(self.exit_codes[len(self.arguments) - 1]) + + +class _TokenValueReadSpy(MutableMapping): + def __init__(self, values, allow_token_read=False): + self._values = dict(values) + self.allow_token_read = allow_token_read + self.token_value_reads = 0 + self.token_mutations = 0 + self.app_secret_value_reads = 0 + self.app_secret_mutations = 0 + + def __getitem__(self, key): + if key == "LARK_BOT_ACCESS_TOKEN": + self.token_value_reads += 1 + if not self.allow_token_read: + raise AssertionError("non-approval path read the injected token value") + if key == "LARK_APP_SECRET": + self.app_secret_value_reads += 1 + if not self.allow_token_read: + raise AssertionError("non-approval path read the injected app secret") + return self._values[key] + + def __setitem__(self, key, value): + if key == "LARK_BOT_ACCESS_TOKEN": + self.token_mutations += 1 + if key == "LARK_APP_SECRET": + self.app_secret_mutations += 1 + self._values[key] = value + + def __delitem__(self, key): + if key == "LARK_BOT_ACCESS_TOKEN": + self.token_mutations += 1 + if key == "LARK_APP_SECRET": + self.app_secret_mutations += 1 + del self._values[key] + + def __iter__(self): + return iter(self._values) + + def __len__(self): + return len(self._values) + + def __contains__(self, key): + return key in self._values + + def peek(self, key): + return self._values[key] + + +def _valid_charter(): + return { + "schema_version": "controlled-lark-pilot-0.1.0", + "scenario_id": "sales_renewal_risk_followup", + "workflow_id": "workflow_controlled_lark_pilot", + "workflow_version": "0.1.0", + "support_model": "assisted", + "timezone": "Asia/Shanghai", + "starts_on": "2026-07-18", + "expires_on": "2026-08-15", + "team_consent_confirmed": True, + "assignee_consent_confirmed": True, + "commercial_engagement_confirmed": True, + "required_approved_runs": 5, + "required_distinct_days": 5, + "required_distinct_cases": 2, + } + + +def _valid_case(): + return { + "pilot_case_id": "case-001", + "account_name": "Private Account", + "renewal_risk": "Private Risk", + "owner_open_id": "ou_private", + "due_at": "2026-08-15T09:00:00Z", + } + + +def _recorded_defer_decision(): + return { + "schema_version": "controlled-lark-pilot-decision-0.1.0", + "decision": "defer", + "partner_acknowledged": True, + "operator_acknowledged": True, + "commercial_engagement_confirmed": True, + "rationale": "Recorded private defer decision.", + } + + +def _write_private_case(path: Path, case_id: str = "case-001") -> None: + path.write_text( + json.dumps( + { + "pilot_case_id": case_id, + "account_name": "Private Account", + "renewal_risk": "Private Risk", + "owner_open_id": "ou_private", + "due_at": "2026-08-15T09:00:00Z", + } + ), + encoding="utf-8", + ) + os.chmod(path, 0o600) + + +def _start_waiting_pilot(tmp: str, case_id: str = "case-001"): + root = Path(tmp) + work_dir = root / "pilot" + input_path = root / "case.json" + initialize_pilot(ROOT, work_dir, _valid_charter(), now=NOW) + _write_private_case(input_path, case_id=case_id) + started = start_pilot_run(ROOT, work_dir, input_path, now=NOW) + return work_dir, started + + +def _published_controlled_workflow(): + workflow = build_lark_task_pilot_workflow( + mode="live", + workflow_id="workflow_controlled_lark_pilot", + workflow_version="0.1.0", + workflow_name="controlled-lark-task-sales-renewal-pilot", + ) + workflow["workflow"]["status"] = "published" + return workflow + + +class ControlledLarkPilotTests(TestCase): + def test_case_template_creates_owner_only_empty_private_case_once(self): + with TemporaryDirectory() as temporary: + work_dir = Path(temporary) / "pilot" + initialize_pilot(ROOT, work_dir, _valid_charter(), now=NOW) + + result = create_private_case_template( + ROOT, + work_dir, + "day-1", + "case-001", + now=NOW, + ) + + case_path = work_dir / "private" / "cases" / "day-1.json" + payload = json.loads(case_path.read_text(encoding="utf-8")) + original = case_path.read_bytes() + mode = case_path.stat().st_mode & 0o077 + with self.assertRaisesRegex( + ValueError, + "private JSON target must not already exist", + ): + create_private_case_template( + ROOT, + work_dir, + "day-1", + "case-001", + now=NOW, + ) + + unchanged = case_path.read_bytes() + + self.assertEqual(result, {"status": "template_written", "field_count": 5}) + self.assertEqual( + payload, + { + "pilot_case_id": "case-001", + "account_name": "", + "renewal_risk": "", + "owner_open_id": "", + "due_at": "", + }, + ) + self.assertEqual(mode, 0) + self.assertEqual(unchanged, original) + + def test_case_template_rejects_closed_workspaces_and_unsafe_identifiers(self): + with TemporaryDirectory() as temporary: + work_dir = Path(temporary) / "pilot" + initialize_pilot(ROOT, work_dir, _valid_charter(), now=NOW) + decision_path = work_dir / "private" / "decision.json" + decision_path.write_text( + json.dumps(_recorded_defer_decision()), + encoding="utf-8", + ) + os.chmod(decision_path, 0o600) + + with self.assertRaisesRegex( + ValueError, + "controlled pilot is closed; use a new work directory", + ): + create_private_case_template( + ROOT, + work_dir, + "day-1", + "case-001", + now=NOW, + ) + for name, case_id in ( + ("../day-1", "case-001"), + ("day-1", "account-001"), + ): + with self.subTest(name=name, case_id=case_id), self.assertRaisesRegex( + ValueError, + "template name|opaque identifier", + ): + create_private_case_template( + ROOT, + work_dir, + name, + case_id, + now=NOW, + ) + + self.assertFalse((work_dir / "private" / "cases").exists()) + + def test_case_template_rejects_a_cases_directory_symlink_without_writing_target(self): + with TemporaryDirectory() as temporary: + root = Path(temporary) + work_dir = root / "pilot" + initialize_pilot(ROOT, work_dir, _valid_charter(), now=NOW) + target = root / "cases-target" + target.mkdir() + (work_dir / "private" / "cases").symlink_to( + target, + target_is_directory=True, + ) + + with self.assertRaisesRegex(ValueError, "symbolic link|directory"): + create_private_case_template( + ROOT, + work_dir, + "day-1", + "case-001", + now=NOW, + ) + + self.assertEqual(list(target.iterdir()), []) + + def test_preflight_uses_private_case_without_vault_or_network_access(self): + with TemporaryDirectory() as temporary: + root = Path(temporary) + case_path = root / "case.json" + _write_private_case(case_path) + + result = preflight_pilot_case(ROOT, case_path) + + self.assertEqual(result["status"], "ready") + self.assertEqual(result["connector_id"], "lark_task") + self.assertEqual(result["operation"], "create_task") + self.assertEqual(result["mode"], "live") + self.assertTrue(result["provider_payload_constructed"]) + self.assertFalse(result["credential_resolution_attempted"]) + self.assertFalse(result["network_called"]) + self.assertTrue(result["task_title_present"]) + self.assertTrue(result["task_description_present"]) + self.assertTrue(result["assignee_present"]) + self.assertTrue(result["due_at_present"]) + encoded = json.dumps(result, ensure_ascii=False) + self.assertNotIn("Private Account", encoded) + self.assertNotIn("Private Risk", encoded) + self.assertNotIn("ou_private", encoded) + self.assertNotIn("2026-08-15T09:00:00Z", encoded) + + def test_preflight_rejects_a_non_object_connector_result(self): + with TemporaryDirectory() as temporary: + root = Path(temporary) + case_path = root / "case.json" + _write_private_case(case_path) + malformed_connector = SimpleNamespace(preflight=lambda *_args, **_kwargs: None) + + with patch( + "skill2workflow.controlled_lark_pilot.load_external_connector", + return_value=malformed_connector, + ): + with self.assertRaisesRegex( + ValueError, + "controlled pilot connector preflight result is invalid", + ): + preflight_pilot_case(ROOT, case_path) + + def test_non_approval_operations_never_read_or_mutate_injected_token(self): + with TemporaryDirectory() as tmp: + root = Path(tmp).resolve() + work_dir = root / "pilot" + input_path = root / "case.json" + spy = _TokenValueReadSpy( + { + "LARK_BOT_ACCESS_TOKEN": "injected-private-token", + "LARK_APP_ID": "cli_test", + "LARK_APP_SECRET": "injected-private-app-secret", + "KEEP_ME": "yes", + } + ) + runner = _FakeCommandRunner() + + def without_token_access(label, operation): + before = ( + spy.token_value_reads, + spy.token_mutations, + spy.app_secret_value_reads, + spy.app_secret_mutations, + ) + with self.subTest(operation=label): + result = operation() + self.assertEqual( + ( + spy.token_value_reads, + spy.token_mutations, + spy.app_secret_value_reads, + spy.app_secret_mutations, + ), + before, + ) + self.assertEqual( + spy.peek("LARK_BOT_ACCESS_TOKEN"), + "injected-private-token", + ) + self.assertEqual( + spy.peek("LARK_APP_SECRET"), + "injected-private-app-secret", + ) + return result + + with patch.object(os, "environ", spy): + without_token_access( + "init", + lambda: initialize_pilot( + ROOT, + work_dir, + _valid_charter(), + now=NOW, + ), + ) + _write_private_case(input_path) + without_token_access( + "preflight", + lambda: preflight_pilot_case(ROOT, input_path), + ) + started = without_token_access( + "start", + lambda: start_pilot_run( + ROOT, + work_dir, + input_path, + now=NOW, + ), + ) + without_token_access( + "reject", + lambda: decide_pilot_run( + ROOT, + work_dir, + started["run_id"], + approved=False, + now=NOW, + ), + ) + without_token_access( + "evidence", + lambda: generate_pilot_evidence(ROOT, work_dir, now=NOW), + ) + without_token_access( + "exercise-failure", + lambda: exercise_disabled_live(ROOT, work_dir, now=NOW), + ) + without_token_access( + "exercise-rollback", + lambda: exercise_rollback(ROOT, work_dir, now=NOW), + ) + without_token_access( + "verify", + lambda: verify_pilot( + ROOT, + work_dir, + command_runner=runner, + ), + ) + + for environment in runner.environments: + self.assertNotIn("LARK_BOT_ACCESS_TOKEN", environment) + self.assertNotIn("LARK_APP_ID", environment) + self.assertNotIn("LARK_APP_SECRET", environment) + self.assertNotIn("SKILL2WORKFLOW_LARK_TASK_LIVE", environment) + self.assertEqual(environment["KEEP_ME"], "yes") + + def test_approval_remains_the_only_path_that_reads_injected_token(self): + with TemporaryDirectory() as tmp: + work_dir, started = _start_waiting_pilot(tmp) + spy = _TokenValueReadSpy( + { + "SKILL2WORKFLOW_LARK_TASK_LIVE": "1", + "LARK_BOT_ACCESS_TOKEN": "injected-private-token", + }, + allow_token_read=True, + ) + with patch.object(os, "environ", spy): + result = decide_pilot_run( + ROOT, + work_dir, + started["run_id"], + approved=True, + confirmed_live=True, + now=NOW, + transport=_FakeTransport(), + ) + + self.assertEqual(result["run_status"], "completed") + self.assertEqual(spy.token_value_reads, 1) + self.assertEqual(spy.token_mutations, 0) + self.assertEqual(spy.app_secret_value_reads, 0) + self.assertEqual(spy.app_secret_mutations, 0) + + def test_approved_live_decision_exchanges_vault_app_secret_in_memory(self): + with TemporaryDirectory() as tmp: + work_dir, started = _start_waiting_pilot(tmp) + task_transport = _FakeTransport() + token_transport = _FakeTokenTransport() + environment = { + "SKILL2WORKFLOW_LARK_TASK_LIVE": "1", + "LARK_APP_ID": "cli_test", + "LARK_APP_SECRET": "vault-private-app-secret", + } + with patch.dict(os.environ, environment, clear=True): + result = decide_pilot_run( + ROOT, + work_dir, + started["run_id"], + approved=True, + confirmed_live=True, + now=NOW, + transport=task_transport, + token_transport=token_transport, + ) + control = LocalControlPlane(work_dir / "state", storage="sqlite") + run = control.get_run(started["run_id"]) + events = control.list_audit_events(run_id=started["run_id"]) + + self.assertEqual(result["run_status"], "completed") + self.assertEqual(len(task_transport.calls), 1) + self.assertEqual(len(token_transport.calls), 1) + request, timeout = token_transport.calls[0] + self.assertEqual(timeout, 20) + self.assertEqual( + request.full_url, + "https://open.feishu.cn/open-apis/auth/v3/tenant_access_token/internal", + ) + self.assertEqual(request.get_method(), "POST") + self.assertEqual( + json.loads(request.data.decode("utf-8")), + {"app_id": "cli_test", "app_secret": "vault-private-app-secret"}, + ) + self.assertTrue(token_transport.responses[0].closed) + connector_request, _connector_timeout = task_transport.calls[0] + self.assertIsInstance(connector_request.get_header("Authorization"), str) + encoded = json.dumps({"result": result, "events": events, "run": run}) + self.assertNotIn("vault-private-app-secret", encoded) + self.assertNotIn("fresh-private-tenant-token", encoded) + + def test_approved_live_decision_fails_closed_when_token_exchange_fails(self): + with TemporaryDirectory() as tmp: + work_dir, started = _start_waiting_pilot(tmp) + task_transport = _FakeTransport() + token_transport = _FakeTokenTransport({"code": 99991663}) + environment = { + "SKILL2WORKFLOW_LARK_TASK_LIVE": "1", + "LARK_APP_ID": "cli_test", + "LARK_APP_SECRET": "vault-private-app-secret", + } + with patch.dict(os.environ, environment, clear=True): + with self.assertRaisesRegex( + ValueError, + "Lark tenant token exchange failed", + ): + decide_pilot_run( + ROOT, + work_dir, + started["run_id"], + approved=True, + confirmed_live=True, + now=NOW, + transport=task_transport, + token_transport=token_transport, + ) + control = LocalControlPlane(work_dir / "state", storage="sqlite") + run = control.get_run(started["run_id"]) + + self.assertEqual(run["status"], "waiting") + self.assertEqual(task_transport.calls, []) + self.assertEqual(len(token_transport.calls), 1) + self.assertTrue(token_transport.responses[0].closed) + + def test_exercise_disabled_live_uses_real_boundary_without_credentials_or_transport(self): + expected = { + "exercise": "disabled_live", + "passed": True, + "provider_status": "live_disabled", + "credential_resolution_attempted": False, + "transport_attempted": False, + } + with TemporaryDirectory() as tmp: + work_dir = Path(tmp) / "pilot" + initialize_pilot(ROOT, work_dir, _valid_charter(), now=NOW) + environment = { + "SKILL2WORKFLOW_LARK_TASK_LIVE": "1", + "LARK_BOT_ACCESS_TOKEN": "private-token", + } + with patch.dict(os.environ, environment, clear=True): + result = exercise_disabled_live(ROOT, work_dir, now=NOW) + restored = { + key: os.environ.get(key) + for key in environment + } + + exercise_path = work_dir / "private" / "exercises" / "failure.json" + persisted = json.loads(exercise_path.read_text(encoding="utf-8")) + exercise_mode = exercise_path.stat().st_mode & 0o077 + encoded = json.dumps({"result": result, "persisted": persisted}) + remaining_private_bytes = b"".join( + path.read_bytes() + for path in work_dir.rglob("*") + if path.is_file() + ) + + self.assertEqual(result, expected) + self.assertEqual(restored, environment) + self.assertEqual( + set(persisted), + { + "schema_version", + "exercise", + "passed", + "provider_status", + "credential_resolution_attempted", + "transport_attempted", + }, + ) + self.assertEqual({key: persisted[key] for key in expected}, expected) + self.assertEqual(exercise_mode, 0) + for forbidden in ( + "exercise-disabled-001", + "Disabled Live Exercise Account", + "Disabled Live Exercise Risk", + "ou_disabled_live_exercise", + "private-token", + ): + self.assertNotIn(forbidden, encoded) + self.assertNotIn(forbidden.encode("utf-8"), remaining_private_bytes) + + def test_exercise_rollback_rejects_enabled_live_before_writes(self): + with TemporaryDirectory() as tmp: + work_dir = Path(tmp) / "pilot" + initialize_pilot(ROOT, work_dir, _valid_charter(), now=NOW) + with patch.dict( + os.environ, + { + "SKILL2WORKFLOW_LARK_TASK_LIVE": "1", + "LARK_BOT_ACCESS_TOKEN": "private-token", + }, + clear=True, + ), patch( + "skill2workflow.lark_task_pilot.run_lark_task_pilot", + side_effect=AssertionError("dry run must not start"), + ) as dry_run: + with self.assertRaisesRegex(ValueError, "remove.*live switch"): + exercise_rollback(ROOT, work_dir, now=NOW) + + dry_run.assert_not_called() + self.assertFalse( + (work_dir / "private" / "exercises" / "rollback.json").exists() + ) + self.assertFalse((work_dir / "private" / "rollback-live-probe").exists()) + + def test_task6_operations_restore_live_environment_on_every_exception_path(self): + environment = { + "SKILL2WORKFLOW_LARK_TASK_LIVE": "0", + "LARK_BOT_ACCESS_TOKEN": "private-token", + } + with TemporaryDirectory() as tmp: + work_dir = Path(tmp) / "pilot" + initialize_pilot(ROOT, work_dir, _valid_charter(), now=NOW) + with patch.dict(os.environ, environment, clear=True): + with patch( + "skill2workflow.controlled_lark_pilot.start_pilot_run", + side_effect=RuntimeError("disabled exercise failed"), + ), self.assertRaisesRegex(RuntimeError, "disabled exercise failed"): + exercise_disabled_live(ROOT, work_dir, now=NOW) + disabled_restored = { + key: os.environ.get(key) for key in environment + } + + with patch.dict(os.environ, environment, clear=True): + with patch( + "skill2workflow.lark_task_pilot.run_lark_task_pilot", + side_effect=RuntimeError("rollback dry run failed"), + ), self.assertRaisesRegex(RuntimeError, "rollback dry run failed"): + exercise_rollback(ROOT, work_dir, now=NOW) + rollback_restored = { + key: os.environ.get(key) for key in environment + } + + observed_environment = {} + + def failing_runner(arguments, *, cwd, env, capture_output): + observed_environment.update(env) + raise RuntimeError("verification runner failed") + + with patch.dict(os.environ, environment, clear=True): + with self.assertRaisesRegex( + RuntimeError, + "verification runner failed", + ): + verify_pilot(ROOT, work_dir, command_runner=failing_runner) + verification_restored = { + key: os.environ.get(key) for key in environment + } + + self.assertFalse( + (work_dir / "private" / "exercises" / "failure.json").exists() + ) + self.assertFalse( + (work_dir / "private" / "exercises" / "rollback.json").exists() + ) + self.assertFalse((work_dir / "private" / "verification.json").exists()) + + self.assertEqual(disabled_restored, environment) + self.assertEqual(rollback_restored, environment) + self.assertEqual(verification_restored, environment) + self.assertNotIn("SKILL2WORKFLOW_LARK_TASK_LIVE", observed_environment) + self.assertNotIn("LARK_BOT_ACCESS_TOKEN", observed_environment) + + def test_task6_operations_reject_repository_work_dir_before_side_effects(self): + runner = _FakeCommandRunner() + with patch( + "skill2workflow.controlled_lark_pilot.load_pilot_charter", + side_effect=AssertionError("charter must not be loaded"), + ) as load_charter, patch( + "skill2workflow.controlled_lark_pilot._pilot_control_plane", + side_effect=AssertionError("control plane must not be created"), + ) as control_plane: + for operation in ( + lambda: exercise_disabled_live(ROOT, ROOT / "private", now=NOW), + lambda: exercise_rollback(ROOT, ROOT / "private", now=NOW), + lambda: verify_pilot( + ROOT, + ROOT / "private", + command_runner=runner, + ), + ): + with self.subTest(operation=operation), self.assertRaisesRegex( + ValueError, + "outside the repository", + ): + operation() + + load_charter.assert_not_called() + control_plane.assert_not_called() + self.assertEqual(runner.arguments, []) + + def test_exercise_rollback_proves_guard_preserves_waiting_run_and_dry_run(self): + with TemporaryDirectory() as tmp: + work_dir = Path(tmp) / "pilot" + initialize_pilot(ROOT, work_dir, _valid_charter(), now=NOW) + with patch.dict( + os.environ, + {"LARK_BOT_ACCESS_TOKEN": "private-token"}, + clear=True, + ): + result = exercise_rollback(ROOT, work_dir, now=NOW) + restored_token = os.environ.get("LARK_BOT_ACCESS_TOKEN") + + persisted_path = ( + work_dir / "private" / "exercises" / "rollback.json" + ) + persisted = json.loads(persisted_path.read_text(encoding="utf-8")) + persisted_mode = persisted_path.stat().st_mode & 0o077 + dry_run_artifact_exists = ( + work_dir + / "private" + / "rollback-dry-run" + / "artifacts" + / "run.json" + ).is_file() + + self.assertEqual( + result, + { + "exercise": "rollback", + "passed": True, + "live_switch_enabled": False, + "live_approval_blocked": True, + "dry_run_status": "completed", + }, + ) + self.assertEqual(restored_token, "private-token") + self.assertEqual({key: persisted[key] for key in result}, result) + self.assertEqual(persisted_mode, 0) + self.assertFalse((work_dir / "private" / "rollback-live-probe").exists()) + self.assertTrue(dry_run_artifact_exists) + + def test_exercise_rollback_can_be_repeated_without_reinitializing_probe(self): + with TemporaryDirectory() as tmp: + work_dir = Path(tmp) / "pilot" + initialize_pilot(ROOT, work_dir, _valid_charter(), now=NOW) + with patch.dict( + os.environ, + {"LARK_BOT_ACCESS_TOKEN": "private-token"}, + clear=True, + ): + first = exercise_rollback(ROOT, work_dir, now=NOW) + second = exercise_rollback(ROOT, work_dir, now=NOW) + + self.assertEqual(first["passed"], True) + self.assertEqual(second["passed"], True) + self.assertFalse( + any((work_dir / "private").glob(".rollback-live-probe-*")) + ) + + def test_verify_pilot_runs_exact_offline_commands_and_persists_compact_results(self): + with TemporaryDirectory() as tmp: + work_dir = Path(tmp) / "pilot" + initialize_pilot(ROOT, work_dir, _valid_charter(), now=NOW) + runner = _FakeCommandRunner() + with patch.dict( + os.environ, + { + "SKILL2WORKFLOW_LARK_TASK_LIVE": "1", + "LARK_BOT_ACCESS_TOKEN": "private-token", + "KEEP_ME": "yes", + }, + clear=True, + ): + result = verify_pilot(ROOT, work_dir, command_runner=runner) + resolved_work_dir = work_dir.resolve() + + verification_path = work_dir / "private" / "verification.json" + persisted = json.loads(verification_path.read_text(encoding="utf-8")) + verification_mode = verification_path.stat().st_mode & 0o077 + + python = sys.executable + sorted_source_files = sorted( + str(path.relative_to(ROOT)) + for path in (ROOT / "src" / "skill2workflow").glob("*.py") + ) + expected_arguments = [ + [ + python, + "-m", + "unittest", + "tests.test_controlled_lark_pilot", + "tests.test_controlled_lark_pilot_evidence", + "tests.test_controlled_lark_pilot_docs", + "-v", + ], + [python, "-m", "unittest", "discover", "-s", "tests", "-v"], + [ + python, + "-m", + "py_compile", + *sorted_source_files, + "examples/connectors/lark_task_connector.py", + ], + [python, "scripts/secret_hygiene.py", "examples/workflows"], + [ + python, + "scripts/lark_task_connector_smoke.py", + "--work-dir", + str(resolved_work_dir / "private" / "connector-smoke"), + ], + [ + python, + "scripts/lark_task_pilot_smoke.py", + "--work-dir", + str(resolved_work_dir / "private" / "dry-run-smoke"), + ], + ["git", "diff", "--check"], + ] + self.assertEqual(runner.arguments, expected_arguments) + self.assertEqual(runner.working_directories, [ROOT] * 7) + self.assertEqual(runner.capture_output, [True] * 7) + for environment in runner.environments: + self.assertNotIn("LARK_BOT_ACCESS_TOKEN", environment) + self.assertNotIn("SKILL2WORKFLOW_LARK_TASK_LIVE", environment) + self.assertEqual(environment["PYTHONPATH"], "src") + self.assertEqual(environment["KEEP_ME"], "yes") + self.assertTrue(result["all_passed"]) + self.assertEqual( + [item["id"] for item in result["commands"]], + [ + "focused-tests", + "full-tests", + "compile", + "secret-hygiene", + "connector-smoke", + "dry-run-pilot-smoke", + "diff-check", + ], + ) + self.assertEqual(result, persisted) + self.assertEqual( + set(result), {"schema_version", "all_passed", "commands"} + ) + for item in result["commands"]: + self.assertEqual( + set(item), {"id", "exit_code", "passed", "duration_ms"} + ) + self.assertIs(type(item["duration_ms"]), int) + self.assertGreaterEqual(item["duration_ms"], 0) + encoded = json.dumps(result) + self.assertNotIn("private-command-stdout", encoded) + self.assertNotIn("private-command-stderr", encoded) + self.assertNotIn("private-token", encoded) + self.assertEqual(verification_mode, 0) + + def test_verify_pilot_records_nonzero_command_without_short_circuiting(self): + with TemporaryDirectory() as tmp: + work_dir = Path(tmp) / "pilot" + initialize_pilot(ROOT, work_dir, _valid_charter(), now=NOW) + runner = _FakeCommandRunner([1, 0, 0, 0, 0, 0, 0]) + + result = verify_pilot(ROOT, work_dir, command_runner=runner) + + self.assertFalse(result["all_passed"]) + self.assertEqual(len(runner.arguments), 7) + self.assertEqual(result["commands"][0]["exit_code"], 1) + self.assertFalse(result["commands"][0]["passed"]) + + def test_verify_pilot_normalizes_negative_exit_and_runs_all_commands(self): + with TemporaryDirectory() as tmp: + work_dir = Path(tmp) / "pilot" + initialize_pilot(ROOT, work_dir, _valid_charter(), now=NOW) + runner = _FakeCommandRunner([-9, 0, 0, 0, 0, 0, 0]) + + result = verify_pilot(ROOT, work_dir, command_runner=runner) + persisted = json.loads( + (work_dir / "private" / "verification.json").read_text( + encoding="utf-8" + ) + ) + + self.assertEqual(len(runner.arguments), 7) + self.assertEqual(result, persisted) + self.assertFalse(result["all_passed"]) + self.assertEqual(result["commands"][0]["exit_code"], 137) + self.assertFalse(result["commands"][0]["passed"]) + + def test_verify_pilot_invalidates_old_success_before_runner_exception(self): + with TemporaryDirectory() as tmp: + work_dir = Path(tmp) / "pilot" + initialize_pilot(ROOT, work_dir, _valid_charter(), now=NOW) + verify_pilot( + ROOT, + work_dir, + command_runner=_FakeCommandRunner(), + ) + verification = work_dir / "private" / "verification.json" + self.assertTrue(verification.exists()) + + calls = [] + + def interrupted_runner(arguments, *, cwd, env, capture_output): + calls.append(list(arguments)) + raise RuntimeError("verification interrupted") + + with self.assertRaisesRegex(RuntimeError, "verification interrupted"): + verify_pilot( + ROOT, + work_dir, + command_runner=interrupted_runner, + ) + + self.assertEqual(len(calls), 1) + self.assertFalse(verification.exists()) + generated = generate_pilot_evidence(ROOT, work_dir, now=NOW) + persisted_evidence = json.loads( + (work_dir / "evidence" / "evidence-index.json").read_text( + encoding="utf-8" + ) + ) + self.assertFalse( + (work_dir / "evidence" / "verification.json").exists() + ) + self.assertFalse(persisted_evidence["verification_passed"]) + self.assertFalse(persisted_evidence["ready_to_finalize"]) + self.assertIn("verification", generated["unmet_conditions"]) + + def test_verify_pilot_rejects_symlink_or_changed_stale_result_before_commands(self): + with TemporaryDirectory() as tmp: + root = Path(tmp).resolve() + work_dir = root / "pilot" + initialize_pilot(ROOT, work_dir, _valid_charter(), now=NOW) + verification = work_dir / "private" / "verification.json" + outside = root / "outside-verification.json" + outside.write_text("sentinel", encoding="utf-8") + os.chmod(outside, 0o600) + verification.symlink_to(outside) + runner = _FakeCommandRunner() + + with self.assertRaisesRegex(ValueError, "symbolic link"): + verify_pilot(ROOT, work_dir, command_runner=runner) + + self.assertEqual(runner.arguments, []) + self.assertEqual(outside.read_text(encoding="utf-8"), "sentinel") + self.assertTrue(verification.is_symlink()) + verification.unlink() + + verify_pilot(ROOT, work_dir, command_runner=_FakeCommandRunner()) + replacement = work_dir / "private" / "replacement.json" + replacement.write_text("{}", encoding="utf-8") + os.chmod(replacement, 0o600) + real_rename = os.rename + changed = [] + + def replace_before_invalidation( + source, + target, + *, + src_dir_fd=None, + dst_dir_fd=None, + ): + if ( + os.fspath(source) == "verification.json" + and os.fspath(target) == "stale" + and not changed + ): + os.replace( + replacement, + verification, + ) + changed.append(True) + return real_rename( + source, + target, + src_dir_fd=src_dir_fd, + dst_dir_fd=dst_dir_fd, + ) + + runner = _FakeCommandRunner() + with patch.object( + os, + "rename", + side_effect=replace_before_invalidation, + ), self.assertRaisesRegex(ValueError, "changed during invalidation"): + verify_pilot(ROOT, work_dir, command_runner=runner) + + self.assertEqual(changed, [True]) + self.assertEqual(runner.arguments, []) + self.assertFalse(verification.exists()) + + def test_decide_approve_requires_all_live_guards_and_returns_redacted_summary(self): + with TemporaryDirectory() as tmp: + work_dir, started = _start_waiting_pilot(tmp) + transport = _FakeTransport() + environment = { + "SKILL2WORKFLOW_LARK_TASK_LIVE": "1", + "LARK_BOT_ACCESS_TOKEN": "private-token", + } + with patch.dict(os.environ, environment, clear=True): + result = decide_pilot_run( + ROOT, + work_dir, + started["run_id"], + approved=True, + confirmed_live=True, + now=NOW, + transport=transport, + ) + control = LocalControlPlane(work_dir / "state", storage="sqlite") + run = control.get_run(started["run_id"]) + events = control.list_audit_events(run_id=started["run_id"]) + + self.assertEqual(result["run_status"], "completed") + self.assertEqual(result["gate_decision"], "approved") + self.assertTrue(result["connector_invoked"]) + self.assertEqual(result["connector_status"], "completed") + self.assertEqual(result["credential_status"], "resolved") + self.assertEqual(result["provider_status"], "completed") + self.assertTrue(result["idempotency_key_present"]) + self.assertTrue(result["lark_task_id_present"]) + self.assertEqual( + set(result), + { + "connector_invoked", + "connector_status", + "credential_status", + "gate_decision", + "idempotency_key_present", + "lark_task_id_present", + "provider_status", + "run_id", + "run_status", + "workflow_id", + "workflow_version", + }, + ) + self.assertEqual(run["status"], "completed") + self.assertEqual(run["context"]["input"], _valid_case()) + self.assertTrue( + any(event.get("type") == "run_resumed" for event in events) + ) + self.assertTrue( + any(event.get("type") == "connector_completed" for event in events) + ) + self.assertEqual(len(transport.calls), 1) + encoded = json.dumps({"result": result, "events": events, "run": run}) + for forbidden in ( + "private-token", + "private-task-guid", + "private-provider-message", + ): + self.assertNotIn(forbidden, encoded) + encoded_audit = json.dumps(events) + for private_value in _valid_case().values(): + self.assertNotIn(private_value, encoded_audit) + summary = json.dumps(result) + for private_value in _valid_case().values(): + self.assertNotIn(private_value, summary) + + def test_decide_reject_needs_no_token_and_never_calls_transport(self): + with TemporaryDirectory() as tmp: + work_dir, started = _start_waiting_pilot(tmp) + transport = _FakeTransport() + with patch.dict(os.environ, {}, clear=True): + result = decide_pilot_run( + ROOT, + work_dir, + started["run_id"], + approved=False, + now=NOW, + transport=transport, + ) + control = LocalControlPlane(work_dir / "state", storage="sqlite") + run = control.get_run(started["run_id"]) + events = control.list_audit_events(run_id=started["run_id"]) + + self.assertEqual(result["run_status"], "failed") + self.assertEqual(result["gate_decision"], "rejected") + self.assertFalse(result["connector_invoked"]) + self.assertEqual(run["status"], "failed") + self.assertTrue( + any( + event.get("type") == "run_resumed" + and event.get("approved") is False + for event in events + ) + ) + self.assertFalse( + any(str(event.get("type", "")).startswith("connector_") for event in events) + ) + self.assertEqual(transport.calls, []) + + def test_live_connector_failure_requires_a_new_work_directory_for_approval(self): + with TemporaryDirectory() as tmp: + work_dir, first = _start_waiting_pilot(tmp) + failed_transport = _ValidationFailureTransport() + environment = { + "SKILL2WORKFLOW_LARK_TASK_LIVE": "1", + "LARK_BOT_ACCESS_TOKEN": "private-token", + } + with patch.dict(os.environ, environment, clear=True): + first_result = decide_pilot_run( + ROOT, + work_dir, + first["run_id"], + approved=True, + confirmed_live=True, + now=NOW, + transport=failed_transport, + ) + + second_case = Path(tmp) / "second-case.json" + _write_private_case(second_case, case_id="case-002") + second = start_pilot_run(ROOT, work_dir, second_case, now=NOW) + spy = _TokenValueReadSpy( + { + "SKILL2WORKFLOW_LARK_TASK_LIVE": "1", + "LARK_APP_ID": "cli_test", + "LARK_APP_SECRET": "private-app-secret", + } + ) + blocked_transport = _FakeTransport() + blocked_token_transport = _FakeTokenTransport() + with patch.object(os, "environ", spy): + with self.assertRaisesRegex( + ValueError, + "new work directory after a live connector failure", + ): + decide_pilot_run( + ROOT, + work_dir, + second["run_id"], + approved=True, + confirmed_live=True, + now=NOW, + transport=blocked_transport, + token_transport=blocked_token_transport, + ) + rejected = decide_pilot_run( + ROOT, + work_dir, + second["run_id"], + approved=False, + now=NOW, + transport=blocked_transport, + ) + + self.assertEqual(first_result["provider_status"], "validation_failed") + self.assertEqual(len(failed_transport.calls), 1) + self.assertEqual(spy.token_value_reads, 0) + self.assertEqual(spy.app_secret_value_reads, 0) + self.assertEqual(blocked_transport.calls, []) + self.assertEqual(blocked_token_transport.calls, []) + self.assertEqual(rejected["gate_decision"], "rejected") + self.assertFalse(rejected["connector_invoked"]) + + def test_decide_approve_fails_before_resume_when_confirmation_switch_or_token_is_missing(self): + with TemporaryDirectory() as tmp: + work_dir, started = _start_waiting_pilot(tmp) + transport = _FakeTransport() + cases = [ + ( + { + "SKILL2WORKFLOW_LARK_TASK_LIVE": "1", + "LARK_BOT_ACCESS_TOKEN": "token", + }, + False, + "confirmation", + ), + ( + {"LARK_BOT_ACCESS_TOKEN": "token"}, + True, + "SKILL2WORKFLOW_LARK_TASK_LIVE=1", + ), + ( + {"SKILL2WORKFLOW_LARK_TASK_LIVE": "1"}, + True, + "LARK_BOT_ACCESS_TOKEN", + ), + ( + { + "SKILL2WORKFLOW_LARK_TASK_LIVE": "1", + "LARK_APP_SECRET": "private-app-secret", + }, + True, + "LARK_APP_ID", + ), + ( + { + "SKILL2WORKFLOW_LARK_TASK_LIVE": "1", + "LARK_APP_ID": "cli_test", + "LARK_APP_SECRET": "", + "LARK_BOT_ACCESS_TOKEN": "legacy-token", + }, + True, + "LARK_APP_SECRET", + ), + ( + { + "SKILL2WORKFLOW_LARK_TASK_LIVE": "true", + "LARK_BOT_ACCESS_TOKEN": "token", + }, + True, + "SKILL2WORKFLOW_LARK_TASK_LIVE=1", + ), + ( + { + "SKILL2WORKFLOW_LARK_TASK_LIVE": "1 ", + "LARK_BOT_ACCESS_TOKEN": "token", + }, + True, + "SKILL2WORKFLOW_LARK_TASK_LIVE=1", + ), + ] + for environment, confirmed, expected in cases: + with self.subTest(expected=expected), patch.dict( + os.environ, + environment, + clear=True, + ): + with self.assertRaisesRegex(ValueError, expected): + decide_pilot_run( + ROOT, + work_dir, + started["run_id"], + approved=True, + confirmed_live=confirmed, + now=NOW, + transport=transport, + ) + control = LocalControlPlane(work_dir / "state", storage="sqlite") + run = control.get_run(started["run_id"]) + + self.assertEqual(run["status"], "waiting") + self.assertEqual(transport.calls, []) + + def test_decide_requires_approved_to_be_an_exact_boolean(self): + for approved in ("false", 1, 0, None): + with self.subTest(approved=approved), TemporaryDirectory() as tmp: + work_dir, started = _start_waiting_pilot(tmp) + transport = _FakeTransport() + environment = { + "SKILL2WORKFLOW_LARK_TASK_LIVE": "1", + "LARK_BOT_ACCESS_TOKEN": "private-token", + } + with patch.dict(os.environ, environment, clear=True): + with self.assertRaisesRegex( + ValueError, + "approved must be a boolean", + ): + decide_pilot_run( + ROOT, + work_dir, + started["run_id"], + approved=approved, + confirmed_live=True, + now=NOW, + transport=transport, + ) + control = LocalControlPlane(work_dir / "state", storage="sqlite") + run = control.get_run(started["run_id"]) + self.assertEqual(run["status"], "waiting") + self.assertEqual(transport.calls, []) + + def test_decide_approve_requires_confirmation_to_be_exact_boolean_true(self): + for confirmed_live in ("true", 1): + case = self.subTest(confirmed_live=confirmed_live) + with case, TemporaryDirectory() as tmp: + work_dir, started = _start_waiting_pilot(tmp) + transport = _FakeTransport() + environment = { + "SKILL2WORKFLOW_LARK_TASK_LIVE": "1", + "LARK_BOT_ACCESS_TOKEN": "private-token", + } + with patch.dict(os.environ, environment, clear=True): + with self.assertRaisesRegex( + ValueError, + "explicit boolean confirmation", + ): + decide_pilot_run( + ROOT, + work_dir, + started["run_id"], + approved=True, + confirmed_live=confirmed_live, + now=NOW, + transport=transport, + ) + control = LocalControlPlane(work_dir / "state", storage="sqlite") + run = control.get_run(started["run_id"]) + self.assertEqual(run["status"], "waiting") + self.assertEqual(transport.calls, []) + + def test_decide_rejects_second_decision_for_terminal_run_without_transport(self): + with TemporaryDirectory() as tmp: + work_dir, started = _start_waiting_pilot(tmp) + transport = _FakeTransport() + environment = { + "SKILL2WORKFLOW_LARK_TASK_LIVE": "1", + "LARK_BOT_ACCESS_TOKEN": "private-token", + } + with patch.dict(os.environ, environment, clear=True): + decide_pilot_run( + ROOT, + work_dir, + started["run_id"], + approved=True, + confirmed_live=True, + now=NOW, + transport=transport, + ) + with patch.dict(os.environ, {}, clear=True): + with self.assertRaisesRegex(ValueError, "not waiting"): + decide_pilot_run( + ROOT, + work_dir, + started["run_id"], + approved=True, + confirmed_live=True, + now=NOW, + transport=transport, + ) + + self.assertEqual(len(transport.calls), 1) + + def test_validate_controlled_live_binding_rejects_each_fixed_property(self): + workflow = _published_controlled_workflow() + run = { + "run_id": "run_controlled", + "workflow_id": "workflow_controlled_lark_pilot", + "workflow_version": "0.1.0", + "status": "waiting", + "current_node": "review_renewal_risk", + "workflow": deepcopy(workflow), + } + _validate_controlled_live_binding(workflow, run) + + def workflow_meta(target, key, value): + target["workflow"][key] = value + + def run_field(target, key, value): + target[key] = value + + def connector_field(target, key, value): + node = next( + item for item in target["nodes"] if item["id"] == "create_lark_task" + ) + node["connector"][key] = value + + def credential_handle(target, _key, value): + node = next( + item for item in target["nodes"] if item["id"] == "create_lark_task" + ) + node["connector"]["credentials"][0]["handle"] = value + + def connector_node_id(target, _key, value): + node = next( + item for item in target["nodes"] if item["id"] == "create_lark_task" + ) + node["id"] = value + + cases = [ + ("workflow id", "workflow", workflow_meta, "id", "workflow_other"), + ("workflow version", "workflow", workflow_meta, "version", "9.9.9"), + ("run workflow id", "run", run_field, "workflow_id", "workflow_other"), + ("run workflow version", "run", run_field, "workflow_version", "9.9.9"), + ("current gate", "run", run_field, "current_node", "other_gate"), + ("connector id", "workflow", connector_field, "id", "http"), + ("operation", "workflow", connector_field, "operation", "delete_task"), + ("mode", "workflow", connector_field, "mode", "dry_run"), + ( + "credential handle", + "workflow", + credential_handle, + "handle", + "other_secret", + ), + ("run id", "run", run_field, "run_id", ""), + ("node id", "workflow", connector_node_id, "id", "other_node"), + ] + for label, target_name, mutate, key, value in cases: + candidate_workflow = deepcopy(workflow) + candidate_run = deepcopy(run) + target = candidate_workflow if target_name == "workflow" else candidate_run + mutate(target, key, value) + if target_name == "workflow": + mutate(candidate_run["workflow"], key, value) + with self.subTest(property=label), self.assertRaisesRegex( + ValueError, + "controlled pilot live binding is invalid", + ): + _validate_controlled_live_binding(candidate_workflow, candidate_run) + + def test_validate_controlled_live_binding_rejects_synchronized_extra_live_action(self): + workflow = _published_controlled_workflow() + first_live_action = next( + node for node in workflow["nodes"] if node["id"] == "create_lark_task" + ) + extra_live_action = deepcopy(first_live_action) + extra_live_action["id"] = "create_second_lark_task" + first_live_action["on_success"] = "create_second_lark_task" + workflow["nodes"].append(extra_live_action) + first_success_edge = next( + edge for edge in workflow["edges"] if edge["id"] == "edge_task_end" + ) + first_success_edge["id"] = "edge_task_second" + first_success_edge["to"] = "create_second_lark_task" + workflow["edges"].append( + { + "id": "edge_second_task_end", + "from": "create_second_lark_task", + "to": "end", + "label": "next", + } + ) + workflow["edges"].append( + { + "id": "edge_second_task_failure", + "from": "create_second_lark_task", + "to": "failure", + "label": "failure", + } + ) + run = { + "run_id": "run_controlled", + "workflow_id": "workflow_controlled_lark_pilot", + "workflow_version": "0.1.0", + "status": "waiting", + "current_node": "review_renewal_risk", + "workflow": deepcopy(workflow), + } + + self.assertEqual(validate_workflow(workflow), []) + with self.assertRaisesRegex( + ValueError, + "controlled pilot live binding is invalid", + ): + _validate_controlled_live_binding(workflow, run) + + def test_decide_rechecks_external_work_dir_before_charter_or_state_access(self): + with TemporaryDirectory() as tmp: + root = Path(tmp) + repo_root = root / "repo" + work_dir = repo_root / "forged-pilot" + transport = _FakeTransport() + with patch( + "skill2workflow.controlled_lark_pilot.load_pilot_charter", + side_effect=AssertionError("charter must not be loaded"), + ) as load_charter, patch( + "skill2workflow.controlled_lark_pilot._pilot_control_plane", + side_effect=AssertionError("state must not be accessed"), + ) as control_plane: + with self.assertRaisesRegex(ValueError, "outside the repository"): + decide_pilot_run( + repo_root, + work_dir, + "run_private", + approved=False, + now=NOW, + transport=transport, + ) + + load_charter.assert_not_called() + control_plane.assert_not_called() + self.assertEqual(transport.calls, []) + + def test_decide_rejects_run_state_that_does_not_match_requested_identity(self): + workflow = _published_controlled_workflow() + current = { + "run_id": "run_other", + "workflow_id": "workflow_controlled_lark_pilot", + "workflow_version": "0.1.0", + "status": "waiting", + "current_node": "review_renewal_risk", + "workflow": deepcopy(workflow), + } + + class MismatchedRunControl: + def __init__(self): + self.resume_calls = [] + + def get_run(self, run_id): + return deepcopy(current) + + def get_workflow(self, workflow_id, workflow_version): + return deepcopy(workflow) + + def resume_published_run(self, run_id, approved=True): + self.resume_calls.append((run_id, approved)) + return {"status": "failed"} + + def list_audit_events(self, run_id=""): + return [] + + with TemporaryDirectory() as tmp: + work_dir = Path(tmp) / "pilot" + initialize_pilot(ROOT, work_dir, _valid_charter(), now=NOW) + control = MismatchedRunControl() + with patch( + "skill2workflow.controlled_lark_pilot._pilot_control_plane", + return_value=control, + ): + with self.assertRaisesRegex( + ValueError, + "controlled pilot run identity is invalid", + ): + decide_pilot_run( + ROOT, + work_dir, + "run_requested", + approved=False, + now=NOW, + ) + + self.assertEqual(control.resume_calls, []) + + def test_start_pilot_run_publishes_live_workflow_and_stops_at_gate(self): + with TemporaryDirectory() as tmp: + work_dir, result = _start_waiting_pilot(tmp) + control = LocalControlPlane(work_dir / "state", storage="sqlite") + run = control.get_run(result["run_id"]) + workflow = control.get_workflow( + "workflow_controlled_lark_pilot", + "0.1.0", + ) + node = next( + item for item in workflow["nodes"] if item["id"] == "create_lark_task" + ) + non_sqlite_state = b"".join( + path.read_bytes() + for path in (work_dir / "state").rglob("*") + if path.is_file() and path.suffix != ".sqlite3" + ) + runs_sqlite_exists = (work_dir / "state" / "runs.sqlite3").is_file() + runs_dir_exists = (work_dir / "state" / "runs").exists() + + self.assertEqual( + set(result), + { + "current_node", + "input_keys", + "preflight_ready", + "run_id", + "run_status", + "workflow_id", + "workflow_version", + }, + ) + self.assertEqual(result["run_status"], "waiting") + self.assertEqual(result["current_node"], "review_renewal_risk") + self.assertTrue(result["preflight_ready"]) + self.assertEqual( + result["input_keys"], + [ + "account_name", + "due_at", + "owner_open_id", + "pilot_case_id", + "renewal_risk", + ], + ) + self.assertEqual(run["status"], "waiting") + self.assertEqual(run["context"]["input"], _valid_case()) + self.assertTrue(runs_sqlite_exists) + self.assertFalse(runs_dir_exists) + self.assertEqual(node["connector"]["mode"], "live") + self.assertNotIn("Private Account", json.dumps(result)) + self.assertNotIn(b"Private Account", non_sqlite_state) + + def test_start_pilot_run_does_not_resolve_credentials_or_call_transport(self): + transport_calls = [] + + def forbidden_transport(*args, **kwargs): + transport_calls.append((args, kwargs)) + raise AssertionError("start must not call live transport") + + with TemporaryDirectory() as tmp: + root = Path(tmp) + work_dir = root / "pilot" + input_path = root / "case.json" + initialize_pilot(ROOT, work_dir, _valid_charter(), now=NOW) + _write_private_case(input_path) + with patch.dict( + os.environ, + {"SKILL2WORKFLOW_LARK_TASK_LIVE": "1"}, + ), patch.object( + StaticCredentialProvider, + "resolve", + side_effect=AssertionError("start must not resolve credentials"), + ) as resolve_credential: + result = start_pilot_run( + ROOT, + work_dir, + input_path, + now=NOW, + transport=forbidden_transport, + ) + + self.assertEqual(result["run_status"], "waiting") + resolve_credential.assert_not_called() + self.assertEqual(transport_calls, []) + + def test_start_pilot_run_requires_local_preflight_before_control_plane_access(self): + with TemporaryDirectory() as tmp: + root = Path(tmp) + work_dir = root / "pilot" + input_path = root / "case.json" + invalid_case = _valid_case() + invalid_case["renewal_risk"] = "private-risk-" + ("x" * 3000) + input_path.write_text(json.dumps(invalid_case), encoding="utf-8") + os.chmod(input_path, 0o600) + initialize_pilot(ROOT, work_dir, _valid_charter(), now=NOW) + + with patch( + "skill2workflow.controlled_lark_pilot._pilot_control_plane" + ) as control_plane: + with self.assertRaisesRegex( + ValueError, + "controlled pilot preflight did not pass", + ): + start_pilot_run(ROOT, work_dir, input_path, now=NOW) + + control_plane.assert_not_called() + + def test_recorded_decision_blocks_new_runs_and_live_decisions(self): + with TemporaryDirectory() as tmp: + work_dir, started = _start_waiting_pilot(tmp) + case_path = Path(tmp) / "next-case.json" + _write_private_case(case_path, case_id="case-002") + decision_path = work_dir / "private" / "decision.json" + decision_path.write_text( + json.dumps(_recorded_defer_decision()), + encoding="utf-8", + ) + os.chmod(decision_path, 0o600) + original_get = os.environ.get + + def prohibit_token_read(key, default=None): + if key == "LARK_BOT_ACCESS_TOKEN": + raise AssertionError("closed pilot must not read the token") + return original_get(key, default) + + with patch( + "skill2workflow.controlled_lark_pilot._pilot_control_plane", + side_effect=AssertionError("closed pilot must not access the control plane"), + ) as control_plane, patch.object( + os.environ, + "get", + side_effect=prohibit_token_read, + ): + with self.assertRaisesRegex( + ValueError, + "controlled pilot is closed; use a new work directory", + ): + start_pilot_run(ROOT, work_dir, case_path, now=NOW) + with self.assertRaisesRegex( + ValueError, + "controlled pilot is closed; use a new work directory", + ): + decide_pilot_run( + ROOT, + work_dir, + started["run_id"], + approved=True, + confirmed_live=True, + now=NOW, + ) + + control_plane.assert_not_called() + + def test_start_pilot_run_rechecks_external_work_dir_before_creating_state(self): + with TemporaryDirectory() as tmp: + root = Path(tmp) + repo_root = root / "repo" + work_dir = repo_root / "forged-pilot" + private_dir = work_dir / "private" + private_dir.mkdir(parents=True) + (private_dir / "charter.json").write_text( + json.dumps(_valid_charter()), + encoding="utf-8", + ) + input_path = root / "case.json" + _write_private_case(input_path) + + with self.assertRaisesRegex(ValueError, "outside the repository"): + start_pilot_run(repo_root, work_dir, input_path, now=NOW) + + self.assertFalse((work_dir / "state").exists()) + + def test_initialize_pilot_creates_owner_only_private_workspace(self): + with TemporaryDirectory() as tmp: + work_dir = Path(tmp) / "controlled-pilot" + result = initialize_pilot(ROOT, work_dir, _valid_charter(), now=NOW) + + self.assertEqual(result["status"], "initialized") + self.assertEqual(result["workflow_id"], "workflow_controlled_lark_pilot") + self.assertEqual(work_dir.stat().st_mode & 0o077, 0) + self.assertEqual( + (work_dir / "private" / "charter.json").stat().st_mode & 0o077, + 0, + ) + self.assertTrue((work_dir / "state").is_dir()) + self.assertTrue((work_dir / "evidence").is_dir()) + + def test_initialize_pilot_refuses_to_replace_an_existing_charter(self): + with TemporaryDirectory() as tmp: + work_dir = Path(tmp) / "controlled-pilot" + initialize_pilot(ROOT, work_dir, _valid_charter(), now=NOW) + charter_path = work_dir / "private" / "charter.json" + original = charter_path.read_bytes() + + with self.assertRaisesRegex( + ValueError, + "private JSON target must not already exist", + ): + initialize_pilot(ROOT, work_dir, _valid_charter(), now=NOW) + + self.assertEqual(charter_path.read_bytes(), original) + + def test_initialize_pilot_rejects_repository_work_dir(self): + with self.assertRaisesRegex(ValueError, "outside the repository"): + initialize_pilot(ROOT, ROOT / ".pilot-private", _valid_charter(), now=NOW) + + def test_initialize_pilot_rejects_private_subdirectory_symlinks(self): + for child_name in ("private", "state", "evidence"): + with self.subTest(child_name=child_name), TemporaryDirectory() as tmp: + work_dir = Path(tmp) / "pilot" + work_dir.mkdir() + target = Path(tmp) / "symlink-target" + target.mkdir() + os.chmod(target, 0o755) + (work_dir / child_name).symlink_to( + target, + target_is_directory=True, + ) + + with self.assertRaisesRegex(ValueError, "symbolic link"): + initialize_pilot(ROOT, work_dir, _valid_charter(), now=NOW) + + self.assertEqual(target.stat().st_mode & 0o777, 0o755) + self.assertEqual(list(target.iterdir()), []) + + @skipUnless(os.name == "posix", "symlink checks require POSIX") + def test_initialize_pilot_rejects_intermediate_symlink_without_writing_target(self): + with TemporaryDirectory() as tmp: + root = Path(tmp) + declared_parent = root / "declared" + target_parent = root / "target" + declared_parent.mkdir() + target_parent.mkdir() + alias = declared_parent / "alias" + alias.symlink_to(target_parent, target_is_directory=True) + work_dir = alias / "pilot" + + with self.assertRaisesRegex(ValueError, "symbolic link|non-directory"): + initialize_pilot(ROOT, work_dir, _valid_charter(), now=NOW) + + self.assertEqual(list(target_parent.iterdir()), []) + + def test_initialize_pilot_rejects_charter_symlink_without_changing_target(self): + with TemporaryDirectory() as tmp: + work_dir = Path(tmp) / "pilot" + private_dir = work_dir / "private" + private_dir.mkdir(parents=True) + target = Path(tmp) / "charter-target.json" + target.write_text("sentinel", encoding="utf-8") + os.chmod(target, 0o640) + (private_dir / "charter.json").symlink_to(target) + + with self.assertRaisesRegex(ValueError, "symbolic link"): + initialize_pilot(ROOT, work_dir, _valid_charter(), now=NOW) + + self.assertEqual(target.read_text(encoding="utf-8"), "sentinel") + self.assertEqual(target.stat().st_mode & 0o777, 0o640) + + def test_initialize_pilot_rejects_non_directory_workspace_nodes(self): + with TemporaryDirectory() as tmp: + work_dir = Path(tmp) / "pilot" + work_dir.mkdir() + state_path = work_dir / "state" + state_path.write_text("sentinel", encoding="utf-8") + + with self.assertRaisesRegex(ValueError, "directory"): + initialize_pilot(ROOT, work_dir, _valid_charter(), now=NOW) + + self.assertEqual(state_path.read_text(encoding="utf-8"), "sentinel") + + def test_initialize_pilot_uses_private_temp_and_cleans_up_publish_failure(self): + observed = {} + + def fail_link( + source, + destination, + *, + src_dir_fd=None, + dst_dir_fd=None, + follow_symlinks=True, + ): + del dst_dir_fd, follow_symlinks + observed["mode"] = ( + os.stat(source, dir_fd=src_dir_fd, follow_symlinks=False).st_mode + & 0o777 + ) + observed["destination"] = Path(destination).name + raise OSError("publish failed") + + with TemporaryDirectory() as tmp: + work_dir = Path(tmp) / "pilot" + with patch( + "skill2workflow._controlled_lark_pilot_evidence_writer.os.link", + side_effect=fail_link, + ): + with self.assertRaisesRegex(OSError, "publish failed"): + initialize_pilot(ROOT, work_dir, _valid_charter(), now=NOW) + + self.assertEqual( + observed, + {"mode": 0o600, "destination": "charter.json"}, + ) + self.assertEqual(list((work_dir / "private").iterdir()), []) + + def test_charter_requires_consent_commercial_status_thresholds_and_active_dates(self): + invalid_values = [ + ("team_consent_confirmed", False), + ("assignee_consent_confirmed", False), + ("commercial_engagement_confirmed", False), + ("required_approved_runs", 4), + ("required_distinct_days", 4), + ("required_distinct_cases", 1), + ("timezone", "UTC"), + ] + for key, value in invalid_values: + charter = _valid_charter() + charter[key] = value + with self.subTest(key=key), TemporaryDirectory() as tmp: + with self.assertRaises(ValueError): + initialize_pilot(ROOT, Path(tmp) / "pilot", charter, now=NOW) + + def test_charter_rejects_unknown_fields(self): + charter = _valid_charter() + charter["account_name"] = "must stay private" + with TemporaryDirectory() as tmp: + with self.assertRaisesRegex(ValueError, "only the approved fields"): + initialize_pilot(ROOT, Path(tmp) / "pilot", charter, now=NOW) + + def test_load_pilot_charter_rejects_expired_charter(self): + with TemporaryDirectory() as tmp: + work_dir = Path(tmp) / "pilot" + initialize_pilot(ROOT, work_dir, _valid_charter(), now=NOW) + with self.assertRaisesRegex(ValueError, "expired"): + load_pilot_charter( + work_dir, + now=datetime(2026, 8, 16, 0, 0, tzinfo=timezone.utc), + ) + + def test_task6_entrypoints_reject_fifo_charter_without_blocking_or_side_effects(self): + for operation in ("generate", "finalize", "disabled", "rollback"): + with self.subTest(operation=operation), TemporaryDirectory() as tmp: + root = Path(tmp).resolve() + work_dir = root / "pilot" + initialize_pilot(ROOT, work_dir, _valid_charter(), now=NOW) + charter = work_dir / "private" / "charter.json" + charter.unlink() + os.mkfifo(charter, 0o600) + output = root / "output" + script = f""" +from pathlib import Path +from skill2workflow.controlled_lark_pilot import exercise_disabled_live, exercise_rollback, finalize_pilot, generate_pilot_evidence +from tests.test_controlled_lark_pilot import NOW +from tests.test_controlled_lark_pilot_evidence import _valid_decision +repo = Path({str(ROOT)!r}) +work = Path({str(work_dir)!r}) +output = Path({str(output)!r}) +operation = {operation!r} +try: + if operation == "generate": + generate_pilot_evidence(repo, work, output_dir=output, now=NOW) + elif operation == "finalize": + finalize_pilot(repo, work, _valid_decision(), output_dir=output, now=NOW) + elif operation == "disabled": + exercise_disabled_live(repo, work, now=NOW) + else: + exercise_rollback(repo, work, now=NOW) +except ValueError as error: + print(str(error)) + raise SystemExit(0) +raise SystemExit(2) +""" + process = subprocess.Popen( + [sys.executable, "-c", script], + cwd=ROOT, + env={ + key: value + for key, value in {**os.environ, "PYTHONPATH": "src"}.items() + if key != "SKILL2WORKFLOW_LARK_TASK_LIVE" + }, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + text=True, + ) + try: + stdout, stderr = process.communicate(timeout=1.0) + except subprocess.TimeoutExpired: + process.kill() + process.communicate() + self.fail(f"{operation} blocked on a FIFO charter") + + self.assertEqual(process.returncode, 0, stderr) + self.assertIn("regular file", stdout) + self.assertFalse(output.exists()) + self.assertFalse( + (work_dir / "private" / "finalization.json").exists() + ) + self.assertFalse( + (work_dir / "private" / "exercises" / "failure.json").exists() + ) + self.assertFalse( + (work_dir / "private" / "exercises" / "rollback.json").exists() + ) + + def test_case_entrypoints_reject_fifo_without_blocking_or_side_effects(self): + for operation in ("load", "start"): + with self.subTest(operation=operation), TemporaryDirectory() as tmp: + root = Path(tmp).resolve() + work_dir = root / "pilot" + initialize_pilot(ROOT, work_dir, _valid_charter(), now=NOW) + case_path = root / "case.json" + os.mkfifo(case_path, 0o600) + script = f""" +from pathlib import Path +from skill2workflow.controlled_lark_pilot import load_private_case, start_pilot_run +from tests.test_controlled_lark_pilot import NOW +repo = Path({str(ROOT)!r}) +work = Path({str(work_dir)!r}) +case_path = Path({str(case_path)!r}) +operation = {operation!r} +try: + if operation == "load": + load_private_case(repo, case_path) + else: + start_pilot_run(repo, work, case_path, now=NOW) +except ValueError as error: + print(str(error)) + raise SystemExit(0) +raise SystemExit(2) +""" + process = subprocess.Popen( + [sys.executable, "-c", script], + cwd=ROOT, + env={**os.environ, "PYTHONPATH": "src"}, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + text=True, + ) + try: + stdout, stderr = process.communicate(timeout=1.0) + except subprocess.TimeoutExpired: + process.kill() + process.communicate() + self.fail(f"{operation} blocked on a FIFO private case") + + self.assertEqual(process.returncode, 0, stderr) + self.assertIn("regular file", stdout) + self.assertEqual( + list((work_dir / "state").glob("registry/*.json")), + [], + ) + + def test_load_private_case_rejects_symlink_parent_and_final_component(self): + for symlink in ("parent", "final"): + with self.subTest(symlink=symlink), TemporaryDirectory() as tmp: + root = Path(tmp).resolve() + actual = root / "actual" + actual.mkdir() + os.chmod(actual, 0o700) + target = actual / "case.json" + _write_private_case(target) + if symlink == "parent": + alias = root / "alias" + alias.symlink_to(actual, target_is_directory=True) + case_path = alias / "case.json" + else: + case_path = root / "case.json" + case_path.symlink_to(target) + + with self.assertRaisesRegex(ValueError, "symbolic link|non-directory"): + load_private_case(ROOT, case_path) + + def test_load_private_case_requires_external_owner_only_exact_shape(self): + payload = _valid_case() + with TemporaryDirectory() as tmp: + path = Path(tmp) / "case.json" + path.write_text(json.dumps(payload), encoding="utf-8") + os.chmod(path, 0o600) + self.assertEqual(load_private_case(ROOT, path), payload) + + os.chmod(path, 0o644) + with self.assertRaisesRegex(ValueError, "owner-only"): + load_private_case(ROOT, path) + + def test_load_private_case_rejects_extra_fields(self): + payload = _valid_case() + payload["lark_token"] = "must-not-enter-case" + self._assert_private_case_rejected(payload, "only the approved fields") + + def test_load_private_case_rejects_missing_fields(self): + for key in _valid_case(): + payload = _valid_case() + del payload[key] + with self.subTest(key=key): + self._assert_private_case_rejected( + payload, + "only the approved fields", + ) + + def test_load_private_case_rejects_empty_fields(self): + for key in _valid_case(): + payload = _valid_case() + payload[key] = " " + with self.subTest(key=key): + self._assert_private_case_rejected(payload, "non-empty strings") + + def test_load_private_case_rejects_non_string_fields_before_control_plane(self): + invalid_values = (1, 1.5, True, None, [], {}) + for key in _valid_case(): + for value in invalid_values: + payload = _valid_case() + payload[key] = value + with self.subTest(key=key, value=value), patch( + "skill2workflow.controlled_lark_pilot._pilot_control_plane", + side_effect=AssertionError("control plane must not be created"), + ) as control_plane, TemporaryDirectory() as tmp: + root = Path(tmp) + work_dir = root / "pilot" + input_path = root / "case.json" + initialize_pilot(ROOT, work_dir, _valid_charter(), now=NOW) + input_path.write_text(json.dumps(payload), encoding="utf-8") + os.chmod(input_path, 0o600) + + with self.assertRaisesRegex(ValueError, "non-empty strings"): + start_pilot_run(ROOT, work_dir, input_path, now=NOW) + + control_plane.assert_not_called() + self.assertFalse((work_dir / "state" / "runs.sqlite3").exists()) + + def test_load_private_case_requires_timezone_aware_due_at(self): + for due_at in ( + "not-a-time", + "2026-08-15T09:00:00", + "2026-08-15", + ): + payload = _valid_case() + payload["due_at"] = due_at + with self.subTest(due_at=due_at): + self._assert_private_case_rejected(payload, "due_at") + + for due_at in ( + "2026-08-15T09:00:00Z", + "2026-08-15T09:00:00+08:00", + ): + payload = _valid_case() + payload["due_at"] = due_at + with self.subTest(due_at=due_at), TemporaryDirectory() as tmp: + path = Path(tmp) / "case.json" + path.write_text(json.dumps(payload), encoding="utf-8") + os.chmod(path, 0o600) + self.assertEqual(load_private_case(ROOT, path), payload) + + def test_load_private_case_rejects_non_opaque_case_ids(self): + for pilot_case_id in ( + "account-001", + "customer-001", + "owner@example.com", + "case 001", + ): + payload = _valid_case() + payload["pilot_case_id"] = pilot_case_id + with self.subTest(pilot_case_id=pilot_case_id): + self._assert_private_case_rejected(payload, "opaque identifier") + + def _assert_private_case_rejected(self, payload, message): + with TemporaryDirectory() as tmp: + path = Path(tmp) / "case.json" + path.write_text(json.dumps(payload), encoding="utf-8") + os.chmod(path, 0o600) + with self.assertRaisesRegex(ValueError, message): + load_private_case(ROOT, path) diff --git a/tests/test_controlled_lark_pilot_docs.py b/tests/test_controlled_lark_pilot_docs.py new file mode 100644 index 0000000..6efa105 --- /dev/null +++ b/tests/test_controlled_lark_pilot_docs.py @@ -0,0 +1,706 @@ +import io +import json +import os +import tempfile +import unittest +import warnings +from contextlib import redirect_stderr, redirect_stdout +from datetime import datetime, timedelta +from pathlib import Path +from unittest.mock import patch +from zoneinfo import ZoneInfo + + +ROOT = Path(__file__).resolve().parents[1] + +from skill2workflow.controlled_lark_pilot import main + + +class ControlledLarkPilotCLITests(unittest.TestCase): + def _invoke(self, arguments): + stdout = io.StringIO() + stderr = io.StringIO() + with warnings.catch_warnings(), redirect_stdout( + stdout + ), redirect_stderr(stderr): + warnings.simplefilter("ignore", ResourceWarning) + result = main(arguments) + return result, stdout.getvalue(), stderr.getvalue() + + def _assert_compact_json_line(self, stdout): + self.assertTrue(stdout.endswith("\n")) + self.assertEqual(stdout.count("\n"), 1) + payload = json.loads(stdout) + self.assertEqual( + stdout, + json.dumps(payload, ensure_ascii=False, separators=(",", ":")) + "\n", + ) + return payload + + def _active_dates(self): + today = datetime.now(ZoneInfo("Asia/Shanghai")).date() + return today.isoformat(), (today + timedelta(days=30)).isoformat() + + def _init(self, work_dir): + starts_on, expires_on = self._active_dates() + return self._invoke( + [ + "init", + "--work-dir", + str(work_dir), + "--starts-on", + starts_on, + "--expires-on", + expires_on, + "--confirm-team-consent", + "--confirm-assignee-consent", + "--confirm-commercial-engagement", + ] + ) + + def _write_case(self, path): + path.write_text( + json.dumps( + { + "pilot_case_id": "case-opaque-001", + "account_name": "PRIVATE ACCOUNT VALUE", + "renewal_risk": "PRIVATE RISK VALUE", + "owner_open_id": "ou_private_assignee", + "due_at": "2026-08-15T09:00:00+08:00", + } + ), + encoding="utf-8", + ) + os.chmod(path, 0o600) + + def _write_decision(self, path, mode=0o600): + path.write_text( + json.dumps( + { + "schema_version": "controlled-lark-pilot-decision-0.1.0", + "decision": "defer", + "partner_acknowledged": True, + "operator_acknowledged": True, + "commercial_engagement_confirmed": True, + "rationale": "REDACTED PRIVATE DECISION RATIONALE", + } + ), + encoding="utf-8", + ) + os.chmod(path, mode) + + def test_parser_dispatches_all_ten_commands_and_prints_only_compact_summaries(self): + secret = "SHOULD-NOT-REACH-STDOUT" + common = { + "status": "ok", + "run_id": "run_safe", + "workflow_id": "workflow_controlled_lark_pilot", + "workflow_version": "0.1.0", + "run_status": "waiting", + "current_node": "review_renewal_risk", + "input_keys": ["pilot_case_id"], + "preflight_ready": True, + "connector_id": "lark_task", + "operation": "create_task", + "mode": "live", + "task_title_present": True, + "task_description_present": True, + "assignee_present": True, + "due_at_present": True, + "provider_payload_constructed": True, + "network_called": False, + "gate_decision": "rejected", + "connector_invoked": False, + "connector_status": "", + "credential_status": "", + "provider_status": "", + "idempotency_key_present": False, + "lark_task_id_present": False, + "file_count": 1, + "run_count": 1, + "approved_live_runs": 0, + "distinct_calendar_days": 0, + "distinct_private_cases": 0, + "rejected_runs": 1, + "unmet_conditions": ["approved_live_runs_threshold"], + "exercise": "safe", + "passed": True, + "credential_resolution_attempted": False, + "transport_attempted": False, + "live_switch_enabled": False, + "live_approval_blocked": True, + "dry_run_status": "completed", + "all_passed": True, + "commands": [], + "decision": "defer", + "field_count": 5, + "raw_private_value": secret, + "rationale": secret, + "token": secret, + "provider_message": secret, + } + with tempfile.TemporaryDirectory() as temporary: + root = Path(temporary) + work_dir = root / "pilot" + case_path = root / "case.json" + case_path.write_text("{}", encoding="utf-8") + decision_path = root / "decision.json" + self._write_decision(decision_path) + starts_on, expires_on = self._active_dates() + commands = ( + [ + "init", + "--work-dir", + str(work_dir), + "--starts-on", + starts_on, + "--expires-on", + expires_on, + "--confirm-team-consent", + "--confirm-assignee-consent", + "--confirm-commercial-engagement", + ], + [ + "case-template", + "--work-dir", + str(work_dir), + "--name", + "day-1", + "--case-id", + "case-opaque-001", + ], + ["start", "--work-dir", str(work_dir), "--input", str(case_path)], + ["preflight", "--input", str(case_path)], + [ + "decide", + "--work-dir", + str(work_dir), + "--run-id", + "run_safe", + "--reject", + ], + ["evidence", "--work-dir", str(work_dir)], + ["exercise-failure", "--work-dir", str(work_dir)], + ["exercise-rollback", "--work-dir", str(work_dir)], + ["verify", "--work-dir", str(work_dir)], + [ + "finalize", + "--work-dir", + str(work_dir), + "--decision-file", + str(decision_path), + ], + ) + targets = ( + "initialize_pilot", + "create_private_case_template", + "start_pilot_run", + "preflight_pilot_case", + "decide_pilot_run", + "generate_pilot_evidence", + "exercise_disabled_live", + "exercise_rollback", + "verify_pilot", + "finalize_pilot", + ) + for arguments, target in zip(commands, targets): + with self.subTest(command=arguments[0]), patch( + "skill2workflow.controlled_lark_pilot." + target, + return_value=dict(common), + ): + result, stdout, stderr = self._invoke(arguments) + self.assertEqual(result, 0) + self.assertEqual(stderr, "") + self._assert_compact_json_line(stdout) + self.assertNotIn(secret, stdout) + + def test_real_init_start_and_reject_flow_never_requires_vault_or_prints_private_values(self): + with tempfile.TemporaryDirectory() as temporary: + root = Path(temporary) + work_dir = root / "pilot" + case_path = root / "case.json" + self._write_case(case_path) + + result, initialized_stdout, initialized_stderr = self._init(work_dir) + self.assertEqual(result, 0) + self.assertEqual(initialized_stderr, "") + initialized = self._assert_compact_json_line(initialized_stdout) + self.assertTrue(initialized["team_consent_confirmed"]) + self.assertTrue(initialized["assignee_consent_confirmed"]) + self.assertTrue(initialized["commercial_engagement_confirmed"]) + + result, started_stdout, started_stderr = self._invoke( + ["start", "--work-dir", str(work_dir), "--input", str(case_path)] + ) + self.assertEqual(result, 0) + self.assertEqual(started_stderr, "") + started = self._assert_compact_json_line(started_stdout) + self.assertEqual(started["run_status"], "waiting") + self.assertTrue(started["preflight_ready"]) + + original_get = os.environ.get + + def reject_without_token_access(key, default=None): + if key == "LARK_BOT_ACCESS_TOKEN": + raise AssertionError("rejection must not read the token") + return original_get(key, default) + + with patch.object(os.environ, "get", side_effect=reject_without_token_access): + result, rejected_stdout, rejected_stderr = self._invoke( + [ + "decide", + "--work-dir", + str(work_dir), + "--run-id", + started["run_id"], + "--reject", + ] + ) + self.assertEqual(result, 0) + self.assertEqual(rejected_stderr, "") + rejected = self._assert_compact_json_line(rejected_stdout) + self.assertEqual(rejected["gate_decision"], "rejected") + self.assertFalse(rejected["connector_invoked"]) + combined = initialized_stdout + started_stdout + rejected_stdout + for private_value in ( + "PRIVATE ACCOUNT VALUE", + "PRIVATE RISK VALUE", + "ou_private_assignee", + ): + self.assertNotIn(private_value, combined) + + charter = json.loads( + (work_dir / "private" / "charter.json").read_text(encoding="utf-8") + ) + self.assertEqual( + set(charter), + { + "schema_version", + "scenario_id", + "workflow_id", + "workflow_version", + "support_model", + "timezone", + "starts_on", + "expires_on", + "team_consent_confirmed", + "assignee_consent_confirmed", + "commercial_engagement_confirmed", + "required_approved_runs", + "required_distinct_days", + "required_distinct_cases", + }, + ) + + def test_case_template_never_reads_vault_and_refuses_overwrite(self): + with tempfile.TemporaryDirectory() as temporary: + root = Path(temporary) + work_dir = root / "pilot" + result, stdout, stderr = self._init(work_dir) + self.assertEqual(result, 0) + self.assertEqual(stderr, "") + self._assert_compact_json_line(stdout) + + original_get = os.environ.get + + def reject_token_read(key, default=None): + if key == "LARK_BOT_ACCESS_TOKEN": + raise AssertionError("case template must not read the token") + return original_get(key, default) + + arguments = [ + "case-template", + "--work-dir", + str(work_dir), + "--name", + "day-1", + "--case-id", + "case-opaque-001", + ] + with patch.object(os.environ, "get", side_effect=reject_token_read): + result, stdout, stderr = self._invoke(arguments) + self.assertEqual(result, 0) + self.assertEqual(stderr, "") + summary = self._assert_compact_json_line(stdout) + self.assertEqual(summary, {"status": "template_written", "field_count": 5}) + + case_path = work_dir / "private" / "cases" / "day-1.json" + original = case_path.read_bytes() + self.assertEqual(case_path.stat().st_mode & 0o077, 0) + self.assertNotIn("LARK_BOT_ACCESS_TOKEN", stdout) + + result, stdout, stderr = self._invoke(arguments) + self.assertEqual(result, 1) + self.assertEqual(stdout, "") + self.assertEqual(stderr, "controlled pilot command failed\n") + self.assertEqual(case_path.read_bytes(), original) + + def test_start_rejects_non_string_or_naive_case_values_with_fixed_error(self): + invalid_cases = ( + ("account_name", 42), + ("renewal_risk", ["private"]), + ("owner_open_id", {"private": True}), + ("pilot_case_id", False), + ("due_at", None), + ("due_at", "2026-08-15T09:00:00"), + ) + for key, value in invalid_cases: + with self.subTest(key=key, value=value), tempfile.TemporaryDirectory() as temporary: + root = Path(temporary) + work_dir = root / "pilot" + case_path = root / "case.json" + case = { + "pilot_case_id": "case-opaque-001", + "account_name": "PRIVATE ACCOUNT VALUE", + "renewal_risk": "PRIVATE RISK VALUE", + "owner_open_id": "ou_private_assignee", + "due_at": "2026-08-15T09:00:00+08:00", + } + case[key] = value + case_path.write_text(json.dumps(case), encoding="utf-8") + os.chmod(case_path, 0o600) + self.assertEqual(self._init(work_dir)[0], 0) + + result, stdout, stderr = self._invoke( + ["start", "--work-dir", str(work_dir), "--input", str(case_path)] + ) + + self.assertNotEqual(result, 0) + self.assertEqual(stdout, "") + self.assertEqual(stderr, "controlled pilot command failed\n") + self.assertFalse((work_dir / "state" / "runs.sqlite3").exists()) + + def test_approve_summary_and_parser_errors_do_not_echo_token_or_unknown_input(self): + token = "vault-injected-secret-token" + raw_provider_value = "raw-provider-task-value" + response = { + "run_id": "run_safe", + "workflow_id": "workflow_controlled_lark_pilot", + "workflow_version": "0.1.0", + "run_status": "completed", + "gate_decision": "approved", + "connector_invoked": True, + "connector_status": "completed", + "credential_status": "resolved", + "provider_status": "completed", + "idempotency_key_present": True, + "lark_task_id_present": True, + "token": token, + "provider_message": raw_provider_value, + } + with tempfile.TemporaryDirectory() as temporary, patch.dict( + os.environ, + {"LARK_BOT_ACCESS_TOKEN": token}, + clear=True, + ), patch( + "skill2workflow.controlled_lark_pilot.decide_pilot_run", + return_value=response, + ) as decide: + result, stdout, stderr = self._invoke( + [ + "decide", + "--work-dir", + str(Path(temporary) / "pilot"), + "--run-id", + "run_safe", + "--approve", + "--confirm-live-create", + ] + ) + self.assertEqual(result, 0) + self.assertEqual(stderr, "") + self._assert_compact_json_line(stdout) + self.assertNotIn(token, stdout + stderr) + self.assertNotIn(raw_provider_value, stdout + stderr) + self.assertTrue(decide.call_args.kwargs["approved"]) + self.assertTrue(decide.call_args.kwargs["confirmed_live"]) + + result, stdout, stderr = self._invoke(["unknown-command", token]) + self.assertNotEqual(result, 0) + self.assertEqual(stdout, "") + self.assertNotIn(token, stderr) + + def test_reject_with_live_confirmation_is_a_fixed_error_before_dispatch(self): + with tempfile.TemporaryDirectory() as temporary, patch( + "skill2workflow.controlled_lark_pilot.decide_pilot_run" + ) as decide: + result, stdout, stderr = self._invoke( + [ + "decide", + "--work-dir", + str(Path(temporary) / "pilot"), + "--run-id", + "run_safe", + "--reject", + "--confirm-live-create", + ] + ) + self.assertNotEqual(result, 0) + self.assertEqual(stdout, "") + self.assertEqual( + stderr, + "controlled pilot rejection does not use live confirmation\n", + ) + decide.assert_not_called() + + def test_real_approve_path_fails_closed_when_each_live_guard_is_missing(self): + with tempfile.TemporaryDirectory() as temporary: + root = Path(temporary) + work_dir = root / "pilot" + case_path = root / "case.json" + self._write_case(case_path) + self.assertEqual(self._init(work_dir)[0], 0) + result, stdout, stderr = self._invoke( + ["start", "--work-dir", str(work_dir), "--input", str(case_path)] + ) + self.assertEqual(result, 0) + self.assertEqual(stderr, "") + run_id = self._assert_compact_json_line(stdout)["run_id"] + base = [ + "decide", + "--work-dir", + str(work_dir), + "--run-id", + run_id, + "--approve", + ] + cases = ( + ({}, base), + ({}, base + ["--confirm-live-create"]), + ( + {"SKILL2WORKFLOW_LARK_TASK_LIVE": "1"}, + base + ["--confirm-live-create"], + ), + ) + for environment, arguments in cases: + with self.subTest(environment=environment), patch.dict( + os.environ, + environment, + clear=True, + ): + result, stdout, stderr = self._invoke(arguments) + self.assertNotEqual(result, 0) + self.assertEqual(stdout, "") + self.assertEqual(stderr, "controlled pilot command failed\n") + + @unittest.skipUnless(os.name == "posix", "owner-only permissions require POSIX") + def test_finalize_rejects_non_owner_only_and_symlink_decision_files(self): + with tempfile.TemporaryDirectory() as temporary: + root = Path(temporary) + work_dir = root / "pilot" + decision = root / "decision.json" + self._write_decision(decision, mode=0o644) + arguments = [ + "finalize", + "--work-dir", + str(work_dir), + "--decision-file", + str(decision), + ] + with patch("skill2workflow.controlled_lark_pilot.finalize_pilot") as finalize: + result, stdout, stderr = self._invoke(arguments) + self.assertNotEqual(result, 0) + self.assertEqual(stdout, "") + self.assertEqual(stderr, "controlled pilot command failed\n") + finalize.assert_not_called() + + os.chmod(decision, 0o600) + link = root / "decision-link.json" + link.symlink_to(decision) + arguments[-1] = str(link) + result, stdout, stderr = self._invoke(arguments) + self.assertNotEqual(result, 0) + self.assertEqual(stdout, "") + self.assertEqual(stderr, "controlled pilot command failed\n") + finalize.assert_not_called() + + def test_finalize_rejects_decision_file_inside_repository(self): + with tempfile.TemporaryDirectory(dir=ROOT) as temporary, \ + tempfile.TemporaryDirectory() as outside: + decision = Path(temporary) / "decision.json" + self._write_decision(decision) + with patch("skill2workflow.controlled_lark_pilot.finalize_pilot") as finalize: + result, stdout, stderr = self._invoke( + [ + "finalize", + "--work-dir", + str(Path(outside) / "pilot"), + "--decision-file", + str(decision), + ] + ) + self.assertNotEqual(result, 0) + self.assertEqual(stdout, "") + self.assertEqual(stderr, "controlled pilot command failed\n") + finalize.assert_not_called() + + def test_expected_operator_error_uses_fixed_redacted_stderr(self): + private_error = "PRIVATE INPUT /private/path/provider-message" + with tempfile.TemporaryDirectory() as temporary, patch( + "skill2workflow.controlled_lark_pilot.start_pilot_run", + side_effect=ValueError(private_error), + ): + result, stdout, stderr = self._invoke( + [ + "start", + "--work-dir", + str(Path(temporary) / "pilot"), + "--input", + private_error, + ] + ) + self.assertNotEqual(result, 0) + self.assertEqual(stdout, "") + self.assertEqual(stderr, "controlled pilot command failed\n") + self.assertNotIn(private_error, stderr) + + def test_start_invalid_utf8_uses_fixed_redacted_stderr_without_traceback(self): + with tempfile.TemporaryDirectory() as temporary: + root = Path(temporary) + work_dir = root / "pilot" + invalid_input = root / "invalid-case.json" + invalid_input.write_bytes(b'\xff{"account_name":"private"}') + os.chmod(invalid_input, 0o600) + self.assertEqual(self._init(work_dir)[0], 0) + + result, stdout, stderr = self._invoke( + [ + "start", + "--work-dir", + str(work_dir), + "--input", + str(invalid_input), + ] + ) + + self.assertNotEqual(result, 0) + self.assertEqual(stdout, "") + self.assertEqual(stderr, "controlled pilot command failed\n") + self.assertNotIn("Traceback", stderr) + self.assertNotIn("account_name", stderr) + + def test_finalize_invalid_utf8_uses_fixed_redacted_stderr_without_traceback(self): + with tempfile.TemporaryDirectory() as temporary: + root = Path(temporary) + decision = root / "invalid-decision.json" + decision.write_bytes(b'\xff{"rationale":"private"}') + os.chmod(decision, 0o600) + + with patch( + "skill2workflow.controlled_lark_pilot.finalize_pilot" + ) as finalize: + result, stdout, stderr = self._invoke( + [ + "finalize", + "--work-dir", + str(root / "pilot"), + "--decision-file", + str(decision), + ] + ) + + self.assertNotEqual(result, 0) + self.assertEqual(stdout, "") + self.assertEqual(stderr, "controlled pilot command failed\n") + self.assertNotIn("Traceback", stderr) + self.assertNotIn("rationale", stderr) + finalize.assert_not_called() + + +class ControlledLarkPilotDocumentationTests(unittest.TestCase): + def test_controlled_pilot_runbook_documents_every_safe_phase(self): + runbook = (ROOT / "docs" / "controlled-live-pilot.md").read_text( + encoding="utf-8" + ) + for command in ( + " init ", + " case-template ", + " preflight ", + " start ", + " decide ", + " evidence ", + " exercise-failure ", + " exercise-rollback ", + " verify ", + " finalize ", + ): + self.assertIn(command, runbook) + self.assertIn("paid assisted engagement", runbook) + self.assertIn("vibe vault run --env LARK_APP_SECRET", runbook) + self.assertIn("LARK_APP_ID", runbook) + self.assertIn("chmod 600", runbook) + self.assertIn("five distinct calendar days", runbook) + self.assertIn("Asia/Shanghai", runbook) + self.assertIn("case-001", runbook) + self.assertIn("case-002", runbook) + self.assertIn('"const": "case-002"', runbook) + self.assertIn("Day 4 exact schema", runbook) + self.assertIn("docs/pilot-evidence/loop-40", runbook) + self.assertIn("continue", runbook) + self.assertIn("harden", runbook) + self.assertIn("defer", runbook) + self.assertIn("must not advance Loop 40", runbook) + + def test_runbook_documents_exact_private_schemas_and_safety_boundaries(self): + runbook = (ROOT / "docs" / "controlled-live-pilot.md").read_text( + encoding="utf-8" + ) + for field in ( + '"pilot_case_id"', + '"account_name"', + '"renewal_risk"', + '"owner_open_id"', + '"due_at"', + '"schema_version"', + '"partner_acknowledged"', + '"operator_acknowledged"', + '"commercial_engagement_confirmed"', + '"rationale"', + ): + self.assertIn(field, runbook) + self.assertIn("repo", runbook.lower()) + self.assertIn("owner-only", runbook) + self.assertIn("fixed Feishu domestic", runbook) + self.assertIn("one `create_task` action", runbook) + self.assertIn("dry-run remains the default", runbook) + self.assertIn("does not resolve Vault credentials", runbook) + self.assertIn("does not make a network request", runbook) + self.assertIn("runs the same no-network preflight", runbook) + self.assertIn("new work directory", runbook) + self.assertIn("rejects every later `--approve`", runbook) + self.assertIn("rotate or delete", runbook) + self.assertIn("stop", runbook.lower()) + + def test_docs_preserve_dry_run_and_narrow_live_boundaries(self): + connectors = (ROOT / "docs" / "connectors.md").read_text(encoding="utf-8") + examples = (ROOT / "docs" / "examples.md").read_text(encoding="utf-8") + readme = (ROOT / "README.md").read_text(encoding="utf-8") + self.assertIn("docs/controlled-live-pilot.md", connectors) + self.assertIn("dry-run remains the default", connectors) + self.assertIn("controlled real-team pilot", examples) + self.assertIn("docs/controlled-pilot-deferral-review.md", readme) + self.assertIn("docs/controlled-live-pilot.md", readme) + self.assertIn("Loop 40", readme) + self.assertIn("Current maturity: Local Evaluation", readme) + self.assertIn("Delivery Loops 1-39 are complete", readme) + + def test_deferral_review_records_only_supported_findings_and_reentry_gate(self): + review = (ROOT / "docs" / "controlled-pilot-deferral-review.md").read_text( + encoding="utf-8" + ) + normalized = " ".join(review.split()) + + self.assertIn("# Controlled Pilot Deferral Review", normalized) + self.assertIn("`validation_failed`", normalized) + self.assertIn("cannot attribute", normalized) + self.assertIn("raw provider message", normalized) + self.assertIn("no retry", normalized) + self.assertIn("fresh partner and operator authorization", normalized) + self.assertIn("new private work directory", normalized) + self.assertIn("no-network `preflight`", normalized) + self.assertIn("five approved live runs", normalized) + self.assertNotIn("root cause was", normalized.lower()) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/test_controlled_lark_pilot_evidence.py b/tests/test_controlled_lark_pilot_evidence.py new file mode 100644 index 0000000..02a1599 --- /dev/null +++ b/tests/test_controlled_lark_pilot_evidence.py @@ -0,0 +1,1461 @@ +import json +import os +import shutil +from copy import deepcopy +from datetime import datetime, timezone +from pathlib import Path +from tempfile import TemporaryDirectory +from unittest import TestCase +from unittest.mock import patch + +from skill2workflow.controlled_lark_pilot import ( + decide_pilot_run, + initialize_pilot, + start_pilot_run, +) +from skill2workflow.controlled_lark_pilot_evidence import ( + RUN_EVIDENCE_KEYS, + build_acceptance_summary, + build_run_evidence, + validate_evidence_pack, +) +from skill2workflow.control_plane import LocalControlPlane + + +ROOT = Path(__file__).resolve().parents[1] +NOW = datetime(2026, 7, 23, 9, 0, tzinfo=timezone.utc) + + +class _FakeResponse: + status = 200 + + def read(self): + return json.dumps( + { + "code": 0, + "msg": "private-provider-message", + "data": {"task": {"guid": "private-task-guid"}}, + } + ).encode("utf-8") + + def close(self): + return None + + +class _FakeTransport: + def __init__(self): + self.calls = [] + + def __call__(self, request, timeout): + self.calls.append((request, timeout)) + return _FakeResponse() + + +class _PermissionDeniedResponse: + status = 403 + + def read(self): + return json.dumps( + {"code": 1470403, "msg": "private-provider-message"} + ).encode("utf-8") + + def close(self): + return None + + +def _valid_charter(): + return { + "schema_version": "controlled-lark-pilot-0.1.0", + "scenario_id": "sales_renewal_risk_followup", + "workflow_id": "workflow_controlled_lark_pilot", + "workflow_version": "0.1.0", + "support_model": "assisted", + "timezone": "Asia/Shanghai", + "starts_on": "2026-07-18", + "expires_on": "2026-08-15", + "team_consent_confirmed": True, + "assignee_consent_confirmed": True, + "commercial_engagement_confirmed": True, + "required_approved_runs": 5, + "required_distinct_days": 5, + "required_distinct_cases": 2, + } + + +def _approved_run(sequence=1, completed_at=None): + return { + "schema_version": "controlled-lark-pilot-evidence-0.1.0", + "run_id": f"run_{sequence:03d}", + "workflow_id": "workflow_controlled_lark_pilot", + "workflow_version": "0.1.0", + "started_at": f"2026-07-{17 + sequence:02d}T00:59:00+00:00", + "completed_at": completed_at + or f"2026-07-{17 + sequence:02d}T01:00:00+00:00", + "run_status": "completed", + "gate_decision": "approved", + "case_id_present": True, + "connector_invoked": True, + "connector_id": "lark_task", + "connector_status": "completed", + "credential_status": "resolved", + "credential_handles": ["lark_bot_access_token"], + "operation": "create_task", + "mode": "live", + "provider_status": "completed", + "task_title_present": True, + "task_description_present": True, + "assignee_present": True, + "due_at_present": True, + "idempotency_key_present": True, + "lark_task_id_present": True, + } + + +def _rejected_run(sequence=6): + return { + "schema_version": "controlled-lark-pilot-evidence-0.1.0", + "run_id": f"run_{sequence:03d}", + "workflow_id": "workflow_controlled_lark_pilot", + "workflow_version": "0.1.0", + "started_at": "2026-07-23T00:59:00+00:00", + "completed_at": "2026-07-23T01:00:00+00:00", + "run_status": "failed", + "gate_decision": "rejected", + "case_id_present": True, + "connector_invoked": False, + "connector_id": "", + "connector_status": "", + "credential_status": "", + "credential_handles": [], + "operation": "", + "mode": "", + "provider_status": "", + "task_title_present": False, + "task_description_present": False, + "assignee_present": False, + "due_at_present": False, + "idempotency_key_present": False, + "lark_task_id_present": False, + } + + +def _valid_exercises(): + return { + "rejection": { + "schema_version": "controlled-lark-pilot-exercise-0.1.0", + "exercise": "rejection", + "passed": True, + "run_id": "run_006", + "gate_decision": "rejected", + "connector_invoked": False, + }, + "failure": { + "schema_version": "controlled-lark-pilot-exercise-0.1.0", + "exercise": "disabled_live", + "passed": True, + "provider_status": "live_disabled", + "credential_resolution_attempted": False, + "transport_attempted": False, + }, + "rollback": { + "schema_version": "controlled-lark-pilot-exercise-0.1.0", + "exercise": "rollback", + "passed": True, + "live_switch_enabled": False, + "live_approval_blocked": True, + "dry_run_status": "completed", + }, + } + + +def _valid_verification(): + command_ids = ( + "focused-tests", + "full-tests", + "compile", + "secret-hygiene", + "connector-smoke", + "dry-run-pilot-smoke", + "diff-check", + ) + return { + "schema_version": "controlled-lark-pilot-verification-0.1.0", + "all_passed": True, + "commands": [ + {"id": command_id, "exit_code": 0, "passed": True, "duration_ms": 1} + for command_id in command_ids + ], + } + + +def _valid_decision(): + return { + "schema_version": "controlled-lark-pilot-decision-0.1.0", + "decision": "continue", + "partner_acknowledged": True, + "operator_acknowledged": True, + "commercial_engagement_confirmed": True, + "rationale": "The controlled workflow delivered the agreed result.", + } + + +def _valid_index(): + return { + "schema_version": "controlled-lark-pilot-index-0.1.0", + "generated_at": "2026-07-23T09:00:00+08:00", + "workflow_id": "workflow_controlled_lark_pilot", + "workflow_version": "0.1.0", + "timezone": "Asia/Shanghai", + "approved_live_runs": 5, + "required_approved_runs": 5, + "distinct_calendar_days": 5, + "required_distinct_days": 5, + "distinct_private_cases": 2, + "required_distinct_cases": 2, + "rejected_runs": 1, + "rejection_passed": True, + "failure_passed": True, + "rollback_passed": True, + "verification_passed": True, + "decision_recorded": True, + "decision": "continue", + "partner_acknowledged": True, + "operator_acknowledged": True, + "commercial_engagement_confirmed": True, + "ready_to_finalize": True, + "unmet_conditions": [], + } + + +def _valid_pack(): + return { + "charter": _valid_charter(), + "runs": [_approved_run(index) for index in range(1, 6)] + + [_rejected_run()], + "exercises": _valid_exercises(), + "verification": _valid_verification(), + "decision": _valid_decision(), + "index": _valid_index(), + } + + +def _raw_completed_run_and_audit(): + run = { + "run_id": "run_raw", + "workflow_id": "workflow_controlled_lark_pilot", + "workflow_version": "0.1.0", + "status": "completed", + "context": {"input": {"pilot_case_id": "case-raw"}}, + } + audit = [ + { + "type": "run_started", + "run_id": "run_raw", + "timestamp": "2026-07-18T01:00:00+00:00", + }, + { + "type": "run_resumed", + "run_id": "run_raw", + "approved": True, + "timestamp": "2026-07-18T01:01:00+00:00", + }, + { + "type": "connector_started", + "run_id": "run_raw", + "node_id": "create_lark_task", + "connector_id": "lark_task", + "connector_status": "running", + "timestamp": "2026-07-18T01:01:00+00:00", + }, + { + "type": "connector_completed", + "run_id": "run_raw", + "node_id": "create_lark_task", + "connector_id": "lark_task", + "connector_status": "completed", + "credential_status": "resolved", + "credential_handles": ["lark_bot_access_token"], + "connector_metadata": { + "operation": "create_task", + "mode": "live", + "provider_status": "completed", + "task_title_present": True, + "task_description_present": True, + "assignee_present": True, + "due_at_present": True, + "idempotency_key_present": True, + "lark_task_id_present": True, + }, + "timestamp": "2026-07-18T01:01:01+00:00", + }, + { + "type": "run_completed", + "run_id": "run_raw", + "timestamp": "2026-07-18T01:01:02+00:00", + }, + ] + return run, audit + + +def _raw_retry_run_and_audit(): + run, audit = _raw_completed_run_and_audit() + completed = audit.pop(3) + audit[3:3] = [ + { + "type": "connector_failed", + "run_id": "run_raw", + "node_id": "create_lark_task", + "connector_id": "lark_task", + "connector_status": "failed", + "timestamp": "2026-07-18T01:01:00+00:00", + }, + { + "type": "node_retrying", + "run_id": "run_raw", + "node_id": "create_lark_task", + "timestamp": "2026-07-18T01:01:00+00:00", + }, + { + "type": "connector_started", + "run_id": "run_raw", + "node_id": "create_lark_task", + "connector_id": "lark_task", + "connector_status": "running", + "timestamp": "2026-07-18T01:01:00+00:00", + }, + completed, + ] + return run, audit + + +def _raw_failed_run_and_audit(provider_status="permission_denied"): + run = { + "run_id": "run_failed_raw", + "workflow_id": "workflow_controlled_lark_pilot", + "workflow_version": "0.1.0", + "status": "failed", + "context": {"input": {"pilot_case_id": "case-failed-raw"}}, + } + metadata = { + "operation": "create_task", + "mode": "live", + "provider_status": provider_status, + "credential_status": "resolved", + "task_title_present": True, + "task_description_present": True, + "assignee_present": True, + "due_at_present": True, + "idempotency_key_present": True, + "lark_task_id_present": False, + } + audit = [ + { + "type": "run_started", + "run_id": "run_failed_raw", + "timestamp": "2026-07-18T01:00:00+00:00", + }, + { + "type": "run_resumed", + "run_id": "run_failed_raw", + "approved": True, + "timestamp": "2026-07-18T01:01:00+00:00", + }, + { + "type": "connector_started", + "run_id": "run_failed_raw", + "node_id": "create_lark_task", + "connector_id": "lark_task", + "connector_status": "running", + "timestamp": "2026-07-18T01:01:00+00:00", + }, + { + "type": "connector_failed", + "run_id": "run_failed_raw", + "node_id": "create_lark_task", + "connector_id": "lark_task", + "connector_status": "failed", + "timestamp": "2026-07-18T01:01:01+00:00", + }, + { + "type": "node_failed", + "run_id": "run_failed_raw", + "node_id": "create_lark_task", + "credential_status": "resolved", + "credential_handles": ["lark_bot_access_token"], + "connector_metadata": metadata, + "timestamp": "2026-07-18T01:01:01+00:00", + }, + { + "type": "run_failed", + "run_id": "run_failed_raw", + "timestamp": "2026-07-18T01:01:02+00:00", + }, + ] + return run, audit + + +def _prepare_repo(root): + repo_root = root / "repo" + connector_dir = repo_root / "examples" / "connectors" + connector_dir.mkdir(parents=True) + shutil.copyfile( + ROOT / "examples" / "connectors" / "lark_task_connector.py", + connector_dir / "lark_task_connector.py", + ) + return repo_root + + +def _write_case(path, case_id): + path.write_text( + json.dumps( + { + "pilot_case_id": case_id, + "account_name": f"Private Account {case_id}", + "renewal_risk": f"Private Risk {case_id}", + "owner_open_id": f"ou_private_{case_id}", + "due_at": "2026-08-15T09:00:00Z", + } + ), + encoding="utf-8", + ) + os.chmod(path, 0o600) + + +def _write_private_artifacts(work_dir): + private = work_dir / "private" + exercises = private / "exercises" + exercises.mkdir(mode=0o700) + exercise_values = { + "failure.json": _valid_exercises()["failure"], + "rollback.json": _valid_exercises()["rollback"], + } + values = { + "verification.json": _valid_verification(), + "decision.json": _valid_decision(), + } + for name, value in exercise_values.items(): + path = exercises / name + path.write_text(json.dumps(value), encoding="utf-8") + os.chmod(path, 0o600) + for name, value in values.items(): + path = private / name + path.write_text(json.dumps(value), encoding="utf-8") + os.chmod(path, 0o600) + + +def _build_ready_state(root): + repo_root = _prepare_repo(root) + work_dir = root / "pilot" + initialize_pilot(repo_root, work_dir, _valid_charter(), now=NOW) + transport = _FakeTransport() + environment = { + "SKILL2WORKFLOW_LARK_TASK_LIVE": "1", + "LARK_BOT_ACCESS_TOKEN": "private-token", + } + for sequence in range(5): + case_id = f"case-{1 + sequence % 2:03d}" + input_path = root / f"approved-{sequence}.json" + _write_case(input_path, case_id) + started_at = f"2026-07-{18 + sequence:02d}T00:59:00+00:00" + completed_at = f"2026-07-{18 + sequence:02d}T01:00:00+00:00" + with patch( + "skill2workflow.control_plane._now", return_value=started_at + ), patch("skill2workflow.executor._now", return_value=started_at): + started = start_pilot_run( + repo_root, work_dir, input_path, now=NOW, transport=transport + ) + with patch.dict(os.environ, environment, clear=True), patch( + "skill2workflow.control_plane._now", return_value=completed_at + ), patch( + "skill2workflow.executor._now", return_value=completed_at + ): + decide_pilot_run( + repo_root, + work_dir, + started["run_id"], + approved=True, + confirmed_live=True, + now=NOW, + transport=transport, + ) + rejected_path = root / "rejected.json" + _write_case(rejected_path, "case-rejected") + with patch( + "skill2workflow.control_plane._now", + return_value="2026-07-23T00:59:00+00:00", + ), patch( + "skill2workflow.executor._now", + return_value="2026-07-23T00:59:00+00:00", + ): + rejected = start_pilot_run( + repo_root, work_dir, rejected_path, now=NOW, transport=transport + ) + with patch.dict(os.environ, {}, clear=True), patch( + "skill2workflow.control_plane._now", + return_value="2026-07-23T01:00:00+00:00", + ), patch( + "skill2workflow.executor._now", + return_value="2026-07-23T01:00:00+00:00", + ): + decide_pilot_run( + repo_root, + work_dir, + rejected["run_id"], + approved=False, + now=NOW, + transport=transport, + ) + _write_private_artifacts(work_dir) + return repo_root, work_dir, transport + + +class ControlledLarkPilotEvidenceTests(TestCase): + def test_forbidden_scan_checks_raw_substrings_in_each_unescaped_leaf(self): + forbidden_values = ("C1", 'line\n"quote"\\tail') + for forbidden in forbidden_values: + pack = _valid_pack() + pack["decision"]["rationale"] = f"prefix {forbidden} suffix" + with self.subTest(forbidden=repr(forbidden)), self.assertRaisesRegex( + ValueError, + "forbidden private value", + ): + validate_evidence_pack(pack, [forbidden]) + + def test_build_run_evidence_requires_nonempty_exact_string_case_id(self): + for invalid in ("", " ", 1, [], {}): + run, audit = _raw_completed_run_and_audit() + run["context"]["input"]["pilot_case_id"] = invalid + with self.subTest(value=repr(invalid)), self.assertRaisesRegex( + ValueError, + "pilot_case_id", + ): + build_run_evidence(run, audit) + + def test_build_run_evidence_requires_exact_boolean_presence_metadata(self): + fields = ( + "task_title_present", + "task_description_present", + "assignee_present", + "due_at_present", + "idempotency_key_present", + "lark_task_id_present", + ) + invalid_values = ("false", 1, {}) + for field in fields: + for invalid in invalid_values: + run, audit = _raw_completed_run_and_audit() + audit[3]["connector_metadata"][field] = invalid + with self.subTest(field=field, value=repr(invalid)), self.assertRaisesRegex( + ValueError, + "presence", + ): + build_run_evidence(run, audit) + + def test_build_run_evidence_preserves_strict_failed_node_metadata(self): + normalized_statuses = ( + "permission_denied", + "resource_not_found", + "idempotency_conflict", + "rate_limited", + "timeout", + ) + for provider_status in normalized_statuses: + run, audit = _raw_failed_run_and_audit(provider_status) + + evidence = build_run_evidence(run, audit) + + with self.subTest(provider_status=provider_status): + self.assertEqual(evidence["run_status"], "failed") + self.assertEqual(evidence["gate_decision"], "approved") + self.assertEqual(evidence["connector_id"], "lark_task") + self.assertEqual(evidence["connector_status"], "failed") + self.assertEqual(evidence["credential_status"], "resolved") + self.assertEqual( + evidence["credential_handles"], + ["lark_bot_access_token"], + ) + self.assertEqual(evidence["operation"], "create_task") + self.assertEqual(evidence["mode"], "live") + self.assertEqual(evidence["provider_status"], provider_status) + self.assertTrue(evidence["idempotency_key_present"]) + self.assertFalse(evidence["lark_task_id_present"]) + + def test_build_run_evidence_rejects_unbound_or_ambiguous_failed_node_facts(self): + mutations = [] + + run, audit = _raw_failed_run_and_audit() + audit.pop(4) + mutations.append(("missing node_failed", run, audit)) + + run, audit = _raw_failed_run_and_audit() + audit.insert(5, deepcopy(audit[4])) + mutations.append(("duplicate node_failed", run, audit)) + + run, audit = _raw_failed_run_and_audit() + node_failed = audit.pop(4) + audit.insert(3, node_failed) + mutations.append(("node_failed before connector_failed", run, audit)) + + run, audit = _raw_failed_run_and_audit() + audit[4]["run_id"] = "run_other" + mutations.append(("wrong run", run, audit)) + + run, audit = _raw_failed_run_and_audit() + audit[4]["node_id"] = "other_node" + mutations.append(("wrong node", run, audit)) + + run, audit = _raw_failed_run_and_audit() + audit[4]["timestamp"] = "2026-07-18T01:00:59+00:00" + mutations.append(("timestamp before connector failure", run, audit)) + + run, audit = _raw_failed_run_and_audit() + audit[4]["connector_metadata"]["raw_provider_message"] = "private" + mutations.append(("unknown metadata", run, audit)) + + run, audit = _raw_failed_run_and_audit("arbitrary_status") + mutations.append(("unknown provider status", run, audit)) + + run, audit = _raw_failed_run_and_audit() + run["status"] = "rejected" + mutations.append(("approved rejected status", run, audit)) + + for label, candidate_run, candidate_audit in mutations: + with self.subTest(case=label), self.assertRaises(ValueError): + build_run_evidence(candidate_run, candidate_audit) + + def test_real_sqlite_failed_live_run_retains_redacted_failure_facts(self): + with TemporaryDirectory() as tmp: + root = Path(tmp) + work_dir = root / "pilot" + input_path = root / "case.json" + initialize_pilot(ROOT, work_dir, _valid_charter(), now=NOW) + _write_case(input_path, "case-failed-integration") + started = start_pilot_run(ROOT, work_dir, input_path, now=NOW) + + transport_calls = [] + + def permission_denied_transport(request, timeout): + transport_calls.append((request, timeout)) + return _PermissionDeniedResponse() + + with patch.dict( + os.environ, + { + "SKILL2WORKFLOW_LARK_TASK_LIVE": "1", + "LARK_BOT_ACCESS_TOKEN": "private-token", + }, + clear=True, + ): + decision = decide_pilot_run( + ROOT, + work_dir, + started["run_id"], + approved=True, + confirmed_live=True, + now=NOW, + transport=permission_denied_transport, + ) + + control = LocalControlPlane(work_dir / "state", storage="sqlite") + run = control.get_run(started["run_id"]) + events = control.list_audit_events(run_id=started["run_id"]) + evidence = build_run_evidence(run, events) + + connector_failure = next( + event for event in events if event["type"] == "connector_failed" + ) + node_failure = next( + event for event in events if event["type"] == "node_failed" + ) + self.assertNotIn("connector_metadata", connector_failure) + self.assertIn("connector_metadata", node_failure) + self.assertEqual(decision["run_status"], "failed") + self.assertEqual(decision["connector_status"], "failed") + self.assertEqual(decision["credential_status"], "resolved") + self.assertEqual(decision["provider_status"], "permission_denied") + self.assertTrue(decision["idempotency_key_present"]) + self.assertFalse(decision["lark_task_id_present"]) + self.assertEqual(len(transport_calls), 1) + self.assertEqual(evidence["run_status"], "failed") + self.assertEqual(evidence["gate_decision"], "approved") + self.assertEqual(evidence["connector_status"], "failed") + self.assertEqual(evidence["provider_status"], "permission_denied") + self.assertEqual(evidence["credential_status"], "resolved") + self.assertEqual( + evidence["credential_handles"], ["lark_bot_access_token"] + ) + self.assertTrue(evidence["idempotency_key_present"]) + self.assertFalse(evidence["lark_task_id_present"]) + self.assertEqual( + build_acceptance_summary( + _valid_charter(), + [evidence], + 1, + {}, + None, + None, + )["approved_live_runs"], + 0, + ) + + def test_build_run_evidence_binds_terminal_events_to_run_status_and_time_order(self): + mutations = [] + + run, audit = _raw_completed_run_and_audit() + audit[0]["run_id"] = "run_other" + mutations.append(("mismatched start", run, audit)) + + run, audit = _raw_completed_run_and_audit() + audit[-1]["run_id"] = "run_other" + mutations.append(("mismatched terminal", run, audit)) + + run, audit = _raw_completed_run_and_audit() + audit.pop() + mutations.append(("missing terminal", run, audit)) + + run, audit = _raw_completed_run_and_audit() + audit[-1]["type"] = "run_failed" + mutations.append(("wrong terminal type", run, audit)) + + run, audit = _raw_completed_run_and_audit() + audit[-1]["timestamp"] = "2026-07-18T00:59:59+00:00" + mutations.append(("terminal before start", run, audit)) + + for label, run, audit in mutations: + with self.subTest(case=label), self.assertRaises(ValueError): + build_run_evidence(run, audit) + + def test_build_run_evidence_rejects_any_conflicting_start_or_terminal_event(self): + mutations = [] + + run, audit = _raw_completed_run_and_audit() + audit.insert( + 1, + { + "type": "run_started", + "run_id": "run_other", + "timestamp": "2026-07-18T01:00:01+00:00", + }, + ) + mutations.append(("extra mismatched start", run, audit)) + + run, audit = _raw_completed_run_and_audit() + audit.insert( + -1, + { + "type": "run_completed", + "run_id": "run_other", + "timestamp": "2026-07-18T01:01:01+00:00", + }, + ) + mutations.append(("extra mismatched terminal", run, audit)) + + run, audit = _raw_completed_run_and_audit() + audit.insert( + -1, + { + "type": "run_completed", + "run_id": "run_raw", + "timestamp": "not-a-time", + }, + ) + mutations.append(("extra invalid terminal time", run, audit)) + + for label, run, audit in mutations: + with self.subTest(case=label), self.assertRaises(ValueError): + build_run_evidence(run, audit) + + def test_build_run_evidence_requires_success_metadata_on_completed_event(self): + run, audit = _raw_completed_run_and_audit() + audit[3]["type"] = "connector_started" + started_spoof = (run, audit) + + run, audit = _raw_completed_run_and_audit() + metadata = audit[3].pop("connector_metadata") + audit.insert( + -1, + { + "type": "node_recovered", + "run_id": "run_raw", + "node_id": "create_lark_task", + "connector_metadata": metadata, + "timestamp": "2026-07-18T01:01:01+00:00", + }, + ) + unrelated_metadata = (run, audit) + + for label, (run, audit) in ( + ("connector_started spoof", started_spoof), + ("unrelated metadata event", unrelated_metadata), + ): + with self.subTest(case=label), self.assertRaises(ValueError): + build_run_evidence(run, audit) + + def test_build_run_evidence_requires_exact_success_facts_for_completed_run(self): + mutations = [] + + run, audit = _raw_completed_run_and_audit() + audit[3]["connector_metadata"]["provider_status"] = "provider_unavailable" + mutations.append(("provider status", run, audit)) + + run, audit = _raw_completed_run_and_audit() + audit[3]["connector_metadata"]["lark_task_id_present"] = False + mutations.append(("task id presence", run, audit)) + + run, audit = _raw_completed_run_and_audit() + audit[3]["credential_status"] = "skipped" + mutations.append(("credential status", run, audit)) + + run, audit = _raw_completed_run_and_audit() + audit.insert( + 2, + { + "type": "connector_started", + "run_id": "run_raw", + "node_id": "create_lark_task", + "connector_id": "other_connector", + "connector_status": "running", + "timestamp": "2026-07-18T01:01:00+00:00", + }, + ) + mutations.append(("preceding connector identity", run, audit)) + + for label, run, audit in mutations: + with self.subTest(case=label), self.assertRaises(ValueError): + build_run_evidence(run, audit) + + def test_build_run_evidence_requires_semantic_audit_list_order(self): + mutations = [] + + run, audit = _raw_completed_run_and_audit() + audit[0], audit[1] = audit[1], audit[0] + mutations.append(("resume before start", run, audit)) + + run, audit = _raw_completed_run_and_audit() + connector = audit.pop(3) + audit.append(connector) + mutations.append(("connector after terminal", run, audit)) + + run, audit = _raw_completed_run_and_audit() + connector = audit.pop(3) + audit.insert(1, connector) + mutations.append(("connector before resume", run, audit)) + + for label, run, audit in mutations: + with self.subTest(case=label), self.assertRaises(ValueError): + build_run_evidence(run, audit) + + def test_build_run_evidence_rejects_duplicate_or_mismatched_decision_events(self): + mutations = [] + + run, audit = _raw_completed_run_and_audit() + audit.insert(2, deepcopy(audit[1])) + mutations.append(("duplicate resume", run, audit)) + + run, audit = _raw_completed_run_and_audit() + audit.append(deepcopy(audit[-1])) + mutations.append(("duplicate terminal", run, audit)) + + run, audit = _raw_completed_run_and_audit() + mismatched = deepcopy(audit[3]) + mismatched["run_id"] = "run_other" + audit.insert(3, mismatched) + mutations.append(("mismatched connector run", run, audit)) + + run, audit = _raw_completed_run_and_audit() + audit[3]["type"] = "connector_failed" + mutations.append(("failed event with completed status", run, audit)) + + for label, run, audit in mutations: + with self.subTest(case=label), self.assertRaises(ValueError): + build_run_evidence(run, audit) + + def test_build_run_evidence_requires_started_attempt_before_each_terminal(self): + run, completion_without_start = _raw_completed_run_and_audit() + completion_without_start.pop(2) + + failed_run, failure_without_start = _raw_completed_run_and_audit() + failed_run["status"] = "failed" + failure_without_start[-1]["type"] = "run_failed" + failure_without_start[3] = { + "type": "connector_failed", + "run_id": "run_raw", + "node_id": "create_lark_task", + "connector_id": "lark_task", + "connector_status": "failed", + "timestamp": "2026-07-18T01:01:01+00:00", + } + failure_without_start.pop(2) + + for label, candidate_run, audit in ( + ("completion without start", run, completion_without_start), + ("failure without start", failed_run, failure_without_start), + ): + with self.subTest(case=label), self.assertRaises(ValueError): + build_run_evidence(candidate_run, audit) + + def test_build_run_evidence_requires_exact_retry_pair_mapping(self): + run, missing_retry = _raw_retry_run_and_audit() + missing_retry.pop(4) + + duplicate_run, duplicate_retry = _raw_retry_run_and_audit() + duplicate_retry.insert(5, deepcopy(duplicate_retry[4])) + + extra_run, extra_retry = _raw_completed_run_and_audit() + extra_retry.insert( + 3, + { + "type": "node_retrying", + "run_id": "run_raw", + "node_id": "create_lark_task", + "timestamp": "2026-07-18T01:01:00+00:00", + }, + ) + + for label, candidate_run, audit in ( + ("missing retry", run, missing_retry), + ("duplicate retry", duplicate_run, duplicate_retry), + ("extra retry", extra_run, extra_retry), + ): + with self.subTest(case=label), self.assertRaises(ValueError): + build_run_evidence(candidate_run, audit) + + def test_build_run_evidence_accepts_no_retry_sequence(self): + run, audit = _raw_completed_run_and_audit() + + evidence = build_run_evidence(run, audit) + + self.assertEqual(evidence["connector_status"], "completed") + self.assertEqual(evidence["provider_status"], "completed") + + def test_build_run_evidence_accepts_normal_retry_sequence(self): + run, audit = _raw_retry_run_and_audit() + + evidence = build_run_evidence(run, audit) + + self.assertEqual(evidence["connector_status"], "completed") + self.assertEqual(evidence["provider_status"], "completed") + + def test_build_run_evidence_requires_exact_rejection_sequence(self): + def rejection(): + run = { + "run_id": "run_rejected", + "workflow_id": "workflow_controlled_lark_pilot", + "workflow_version": "0.1.0", + "status": "failed", + "context": {"input": {"pilot_case_id": "case-rejected"}}, + } + audit = [ + { + "type": "run_started", + "run_id": "run_rejected", + "timestamp": "2026-07-18T01:00:00+00:00", + }, + { + "type": "run_resumed", + "run_id": "run_rejected", + "approved": False, + "timestamp": "2026-07-18T01:01:00+00:00", + }, + { + "type": "run_failed", + "run_id": "run_rejected", + "timestamp": "2026-07-18T01:02:00+00:00", + }, + ] + return run, audit + + mutations = [] + + run, audit = rejection() + audit.pop(1) + mutations.append(("missing resume", run, audit)) + + run, audit = rejection() + audit[1]["approved"] = True + mutations.append(("approved without connector", run, audit)) + + run, audit = rejection() + audit.insert(2, deepcopy(audit[1])) + mutations.append(("duplicate rejection resume", run, audit)) + + run, audit = rejection() + _success_run, success_audit = _raw_completed_run_and_audit() + connectors = deepcopy(success_audit[2:4]) + for connector in connectors: + connector["run_id"] = "run_rejected" + connector["timestamp"] = "2026-07-18T01:01:30+00:00" + audit[2:2] = connectors + mutations.append(("rejection with connector", run, audit)) + + for label, run, audit in mutations: + with self.subTest(case=label), self.assertRaises(ValueError): + build_run_evidence(run, audit) + + def test_build_run_evidence_allows_waiting_run_without_decision(self): + run = { + "run_id": "run_waiting", + "workflow_id": "workflow_controlled_lark_pilot", + "workflow_version": "0.1.0", + "status": "waiting", + "context": {"input": {"pilot_case_id": "case-waiting"}}, + } + audit = [ + { + "type": "run_started", + "run_id": "run_waiting", + "timestamp": "2026-07-18T01:00:00+00:00", + } + ] + + evidence = build_run_evidence(run, audit) + + self.assertEqual(evidence["run_status"], "waiting") + self.assertEqual(evidence["gate_decision"], "pending") + self.assertFalse(evidence["connector_invoked"]) + + def test_build_run_evidence_rejects_invalid_or_naive_event_timestamps(self): + for index in (0, -1): + for invalid in ("not-a-time", "2026-07-18T01:00:00"): + run, audit = _raw_completed_run_and_audit() + audit[index]["timestamp"] = invalid + with self.subTest(index=index, value=invalid), self.assertRaises(ValueError): + build_run_evidence(run, audit) + + def test_build_run_evidence_bounds_resume_and_connector_timestamps(self): + for index, timestamp in ( + (1, "2026-07-18T00:59:59+00:00"), + (2, "2026-07-18T01:01:03+00:00"), + ): + run, audit = _raw_completed_run_and_audit() + audit[index]["timestamp"] = timestamp + with self.subTest(index=index), self.assertRaisesRegex( + ValueError, + "run interval", + ): + build_run_evidence(run, audit) + + def test_validate_pack_requires_terminal_timestamp_for_qualifying_rejection(self): + pack = _valid_pack() + pack["runs"][-1]["completed_at"] = "" + + with self.assertRaisesRegex(ValueError, "completed_at"): + validate_evidence_pack(pack, []) + + def test_build_run_evidence_uses_exact_allowlist_without_private_values(self): + run = { + "run_id": "run_001", + "workflow_id": "workflow_controlled_lark_pilot", + "workflow_version": "0.1.0", + "status": "completed", + "context": { + "input": { + "pilot_case_id": "case-001", + "account_name": "Private Account", + "renewal_risk": "Private Risk", + "owner_open_id": "ou_private", + "due_at": "2026-08-15T09:00:00Z", + } + }, + "node_results": { + "create_lark_task": { + "output": { + "message": "private-provider-message", + "task_id": "private-task-id", + } + } + }, + } + audit = [ + { + "type": "run_started", + "run_id": "run_001", + "timestamp": "2026-07-18T01:00:00+00:00", + }, + { + "type": "run_resumed", + "run_id": "run_001", + "approved": True, + "timestamp": "2026-07-18T01:01:00+00:00", + }, + { + "type": "connector_started", + "run_id": "run_001", + "node_id": "create_lark_task", + "connector_id": "lark_task", + "connector_status": "running", + "timestamp": "2026-07-18T01:01:00+00:00", + }, + { + "type": "connector_completed", + "run_id": "run_001", + "node_id": "create_lark_task", + "connector_id": "lark_task", + "connector_status": "completed", + "credential_status": "resolved", + "credential_handles": ["lark_bot_access_token"], + "connector_metadata": { + "operation": "create_task", + "mode": "live", + "provider_status": "completed", + "task_title_present": True, + "task_description_present": True, + "assignee_present": True, + "due_at_present": True, + "idempotency_key_present": True, + "lark_task_id_present": True, + }, + "timestamp": "2026-07-18T01:01:01+00:00", + }, + { + "type": "run_completed", + "run_id": "run_001", + "timestamp": "2026-07-18T01:01:02+00:00", + }, + ] + + evidence = build_run_evidence(run, audit) + + self.assertEqual(set(evidence), RUN_EVIDENCE_KEYS) + self.assertEqual(evidence["run_id"], "run_001") + self.assertEqual(evidence["gate_decision"], "approved") + self.assertEqual(evidence["provider_status"], "completed") + self.assertTrue(evidence["case_id_present"]) + encoded = json.dumps(evidence) + for forbidden in ( + "case-001", + "Private Account", + "Private Risk", + "ou_private", + "2026-08-15T09:00:00Z", + "private-provider-message", + "private-task-id", + ): + self.assertNotIn(forbidden, encoded) + + def test_build_run_evidence_reports_human_rejection_without_connector_data(self): + run = { + "run_id": "run_rejected", + "workflow_id": "workflow_controlled_lark_pilot", + "workflow_version": "0.1.0", + "status": "failed", + "context": {"input": {"pilot_case_id": "case-rejected"}}, + } + audit = [ + { + "type": "run_started", + "run_id": "run_rejected", + "timestamp": "2026-07-18T01:00:00+00:00", + }, + { + "type": "run_resumed", + "run_id": "run_rejected", + "approved": False, + "timestamp": "2026-07-18T01:01:00+00:00", + }, + { + "type": "run_failed", + "run_id": "run_rejected", + "timestamp": "2026-07-18T01:02:00+00:00", + }, + ] + + evidence = build_run_evidence(run, audit) + + self.assertEqual(evidence["gate_decision"], "rejected") + self.assertFalse(evidence["connector_invoked"]) + for key in ( + "connector_id", + "connector_status", + "credential_status", + "operation", + "mode", + "provider_status", + ): + self.assertEqual(evidence[key], "") + self.assertEqual(evidence["credential_handles"], []) + + def test_acceptance_summary_requires_complete_contract_over_five_shanghai_days(self): + approved_runs = [_approved_run(index) for index in range(1, 6)] + rejected_run = _rejected_run() + + summary = build_acceptance_summary( + charter=_valid_charter(), + runs=approved_runs + [rejected_run], + distinct_private_cases=2, + exercises={"failure": {"passed": True}, "rollback": {"passed": True}}, + verification={"all_passed": True}, + decision={ + "decision": "continue", + "partner_acknowledged": True, + "operator_acknowledged": True, + "commercial_engagement_confirmed": True, + "rationale": "The controlled workflow delivered the agreed result.", + }, + ) + + self.assertTrue(summary["ready_to_finalize"]) + self.assertEqual(summary["approved_live_runs"], 5) + self.assertEqual(summary["distinct_calendar_days"], 5) + self.assertEqual(summary["distinct_private_cases"], 2) + self.assertEqual(summary["rejected_runs"], 1) + self.assertEqual(summary["unmet_conditions"], []) + + def test_acceptance_summary_lists_every_unmet_condition_in_stable_order(self): + summary = build_acceptance_summary( + charter=_valid_charter(), + runs=[], + distinct_private_cases=0, + exercises={"rejection": None, "failure": None, "rollback": None}, + verification=None, + decision=None, + ) + + self.assertFalse(summary["ready_to_finalize"]) + self.assertEqual( + summary["unmet_conditions"], + [ + "approved_live_runs_threshold", + "distinct_calendar_days_threshold", + "distinct_private_cases_threshold", + "human_rejection", + "disabled_live_exercise", + "rollback_exercise", + "verification", + "decision", + "partner_acknowledgement", + "operator_acknowledgement", + ], + ) + self.assertTrue(summary["commercial_engagement_confirmed"]) + + def test_acceptance_summary_does_not_count_partial_provider_success(self): + fields = ( + ("workflow_id", "workflow_other"), + ("workflow_version", "9.9.9"), + ("gate_decision", "pending"), + ("run_status", "failed"), + ("case_id_present", False), + ("connector_invoked", False), + ("connector_id", ""), + ("connector_status", "failed"), + ("credential_status", "failed"), + ("credential_handles", []), + ("operation", ""), + ("mode", ""), + ("provider_status", "provider_unavailable"), + ("task_title_present", False), + ("task_description_present", False), + ("assignee_present", False), + ("due_at_present", False), + ("idempotency_key_present", False), + ("lark_task_id_present", False), + ) + for field, value in fields: + candidate = _approved_run() + candidate[field] = value + with self.subTest(field=field): + summary = build_acceptance_summary( + _valid_charter(), + [candidate], + 1, + {"failure": {"passed": True}, "rollback": {"passed": True}}, + {"all_passed": True}, + _valid_decision(), + ) + self.assertEqual(summary["approved_live_runs"], 0) + + def test_acceptance_summary_rejects_non_integer_private_case_cardinality(self): + for value in (True, 1.0, "2", -1): + with self.subTest(value=value), self.assertRaisesRegex( + ValueError, + "distinct private cases", + ): + build_acceptance_summary( + _valid_charter(), + [], + value, + {}, + None, + None, + ) + + def test_validate_pack_accepts_exact_schema_and_optional_none_slots(self): + validate_evidence_pack(_valid_pack(), []) + pack = _valid_pack() + pack["runs"] = [] + pack["exercises"] = {"rejection": None, "failure": None, "rollback": None} + pack["verification"] = None + pack["decision"] = None + pack["index"].update( + build_acceptance_summary( + pack["charter"], + pack["runs"], + 0, + pack["exercises"], + pack["verification"], + pack["decision"], + ) + ) + validate_evidence_pack(pack, []) + + def test_validate_pack_rejects_equal_but_wrong_charter_threshold_type(self): + pack = _valid_pack() + pack["charter"]["required_approved_runs"] = 5.0 + + with self.assertRaisesRegex(ValueError, "nonnegative integer"): + validate_evidence_pack(pack, []) + + def test_validate_pack_rejects_index_or_rejection_artifact_contradictions(self): + pack = _valid_pack() + pack["index"]["approved_live_runs"] = 4 + with self.assertRaisesRegex(ValueError, "acceptance summary"): + validate_evidence_pack(pack, []) + + pack = _valid_pack() + pack["exercises"]["rejection"] = None + with self.assertRaisesRegex(ValueError, "rejection exercise"): + validate_evidence_pack(pack, []) + + def test_validate_pack_requires_all_seven_verification_commands_even_on_failure(self): + pack = _valid_pack() + pack["verification"]["commands"].pop() + pack["verification"]["all_passed"] = False + pack["index"].update( + build_acceptance_summary( + pack["charter"], + pack["runs"], + 2, + pack["exercises"], + pack["verification"], + pack["decision"], + ) + ) + + with self.assertRaisesRegex(ValueError, "seven commands"): + validate_evidence_pack(pack, []) + + def test_validate_pack_requires_exercise_passed_to_equal_fact_predicate(self): + mutations = ( + ("rejection", "passed", False), + ("failure", "passed", False), + ("failure", "provider_status", "provider_unavailable"), + ("rollback", "passed", False), + ("rollback", "live_switch_enabled", True), + ) + for name, field, value in mutations: + pack = _valid_pack() + pack["exercises"][name][field] = value + pack["index"].update( + build_acceptance_summary( + pack["charter"], + pack["runs"], + 2, + pack["exercises"], + pack["verification"], + pack["decision"], + ) + ) + with self.subTest(exercise=name, field=field), self.assertRaisesRegex( + ValueError, + "passed", + ): + validate_evidence_pack(pack, []) + + def test_validate_pack_requires_all_connector_presence_false_when_not_invoked(self): + fields = ( + "task_title_present", + "task_description_present", + "assignee_present", + "due_at_present", + "idempotency_key_present", + "lark_task_id_present", + ) + for field in fields: + pack = _valid_pack() + pack["runs"][-1][field] = True + with self.subTest(field=field), self.assertRaisesRegex( + ValueError, + "uninvoked connector evidence", + ): + validate_evidence_pack(pack, []) + + def test_validate_pack_rejects_unknown_keys_in_every_artifact_category(self): + paths = ( + ("top-level", ()), + ("charter", ("charter",)), + ("run", ("runs", 0)), + ("index", ("index",)), + ("rejection", ("exercises", "rejection")), + ("failure", ("exercises", "failure")), + ("rollback", ("exercises", "rollback")), + ("verification", ("verification",)), + ("verification-command", ("verification", "commands", 0)), + ("decision", ("decision",)), + ) + for label, path in paths: + candidate = _valid_pack() + target = candidate + for component in path: + target = target[component] + target["private_raw_field"] = "private-value" + with self.subTest(category=label), self.assertRaises(ValueError): + validate_evidence_pack(candidate, []) + + def test_validate_pack_rejects_wrong_types_fixed_identities_and_credentials(self): + cases = ( + ("boolean integer", ("index", "approved_live_runs"), True), + ("workflow", ("runs", 0, "workflow_id"), "workflow_other"), + ("connector", ("runs", 0, "connector_id"), "http"), + ("operation", ("runs", 0, "operation"), "delete_task"), + ("mode", ("runs", 0, "mode"), "dry_run"), + ("credential", ("runs", 0, "credential_handles"), ["other_secret"]), + ("decision boolean", ("decision", "partner_acknowledged"), 1), + ) + for label, path, value in cases: + pack = _valid_pack() + target = pack + for component in path[:-1]: + target = target[component] + target[path[-1]] = value + with self.subTest(case=label), self.assertRaises(ValueError): + validate_evidence_pack(pack, []) + + def test_validate_pack_orders_runs_by_timestamp_instant_then_run_id(self): + first = _approved_run(1) + first["started_at"] = "2026-07-18T09:00:00+08:00" + first["completed_at"] = "2026-07-18T09:01:00+08:00" + second = _approved_run(2) + second["started_at"] = "2026-07-18T02:00:00+00:00" + second["completed_at"] = "2026-07-18T02:01:00+00:00" + pack = _valid_pack() + pack["runs"] = [first, second] + pack["exercises"]["rejection"] = None + pack["index"].update( + build_acceptance_summary( + pack["charter"], + pack["runs"], + 2, + pack["exercises"], + pack["verification"], + pack["decision"], + ) + ) + + validate_evidence_pack(pack, []) + + pack["runs"].reverse() + with self.assertRaisesRegex(ValueError, "stable order"): + validate_evidence_pack(pack, []) + + def test_validate_pack_rejects_forbidden_private_value_anywhere(self): + pack = _valid_pack() + pack["decision"]["rationale"] = "Private Account was reviewed." + + with self.assertRaisesRegex(ValueError, "forbidden private value"): + validate_evidence_pack(pack, ["Private Account"]) + +if __name__ == "__main__": + import unittest + + unittest.main() diff --git a/tests/test_controlled_lark_pilot_evidence_integration.py b/tests/test_controlled_lark_pilot_evidence_integration.py new file mode 100644 index 0000000..b438166 --- /dev/null +++ b/tests/test_controlled_lark_pilot_evidence_integration.py @@ -0,0 +1,1450 @@ +import json +import os +import shutil +from datetime import timedelta +from pathlib import Path +from tempfile import TemporaryDirectory +from unittest import TestCase +from unittest.mock import patch + +from skill2workflow.controlled_lark_pilot import ( + finalize_pilot, + generate_pilot_evidence, + initialize_pilot, +) +from skill2workflow.controlled_lark_pilot_evidence import ( + prepare_evidence_pack_transaction, +) +from skill2workflow._controlled_lark_pilot_private_authorization import ( + AnchoredPrivateSession, + PrivateFinalizationBundle, + open_private_session, +) +from skill2workflow._controlled_lark_pilot_pack_transaction import ( + EvidencePackTransaction, +) + +from tests.test_controlled_lark_pilot_evidence import ( + NOW, + _build_ready_state, + _prepare_repo, + _valid_charter, + _valid_decision, +) +from tests.test_controlled_lark_pilot import _TokenValueReadSpy + + +def _json_bytes_map(directory): + return { + str(path.relative_to(directory)): path.read_bytes() + for path in sorted(directory.rglob("*.json")) + } + + +def _all_bytes_map(directory): + if not directory.exists(): + return {} + return { + str(path.relative_to(directory)): path.read_bytes() + for path in sorted(directory.rglob("*")) + if path.is_file() + } + + +def _valid_finalization_marker(): + return { + "schema_version": "controlled-lark-pilot-finalization-0.1.0", + "finalized": True, + "decision": "continue", + "finalized_at": "2026-07-23T17:00:00+08:00", + } + + +def _write_owner_only_json(path, value): + path.write_text(json.dumps(value), encoding="utf-8") + os.chmod(path, 0o600) + + +class ControlledLarkPilotEvidenceIntegrationTests(TestCase): + def test_finalize_never_reads_or_mutates_injected_token_value(self): + with TemporaryDirectory() as tmp: + root = Path(tmp).resolve() + repo_root, work_dir, _transport = _build_ready_state(root) + (work_dir / "private" / "decision.json").unlink() + spy = _TokenValueReadSpy( + { + "LARK_BOT_ACCESS_TOKEN": "injected-private-token", + "KEEP_ME": "yes", + } + ) + with patch.object(os, "environ", spy): + result = finalize_pilot( + repo_root, + work_dir, + _valid_decision(), + output_dir=root / "requested", + now=NOW, + ) + + self.assertEqual(result["status"], "finalized") + self.assertEqual(spy.token_value_reads, 0) + self.assertEqual(spy.token_mutations, 0) + self.assertEqual( + spy.peek("LARK_BOT_ACCESS_TOKEN"), + "injected-private-token", + ) + + def test_finalize_retries_every_transient_post_commit_cleanup(self): + with TemporaryDirectory() as tmp: + root = Path(tmp).resolve() + repo_root, work_dir, _transport = _build_ready_state(root) + (work_dir / "private" / "decision.json").unlink() + requested = repo_root / "docs" / "pilot-evidence" / "loop-40" + real_bundle_finish = PrivateFinalizationBundle.finish + real_pack_finish = EvidencePackTransaction.finish + real_session_close = AnchoredPrivateSession.close + calls = {"bundle": 0, "session": 0} + pack_calls = {} + + def flaky_bundle_finish(bundle): + calls["bundle"] += 1 + if calls["bundle"] == 1: + raise OSError("transient bundle finish failure") + return real_bundle_finish(bundle) + + def flaky_pack_finish(transaction): + name = str(transaction.output) + pack_calls[name] = pack_calls.get(name, 0) + 1 + if pack_calls[name] == 1: + raise OSError("transient pack finish failure") + return real_pack_finish(transaction) + + def flaky_session_close(session): + calls["session"] += 1 + if calls["session"] == 1: + raise OSError("transient session close failure") + return real_session_close(session) + + with patch.object( + PrivateFinalizationBundle, + "finish", + new=flaky_bundle_finish, + ), patch.object( + EvidencePackTransaction, + "finish", + new=flaky_pack_finish, + ), patch.object( + AnchoredPrivateSession, + "close", + new=flaky_session_close, + ): + result = finalize_pilot( + repo_root, + work_dir, + _valid_decision(), + output_dir=requested, + now=NOW, + ) + + self.assertEqual( + result, + { + "status": "finalized", + "decision": "continue", + "approved_live_runs": 5, + "distinct_calendar_days": 5, + "distinct_private_cases": 2, + "rejected_runs": 1, + "output_dir": str(requested), + }, + ) + self.assertEqual(calls, {"bundle": 2, "session": 2}) + self.assertEqual( + pack_calls, + {str(work_dir / "evidence"): 2, str(requested): 2}, + ) + self.assertEqual( + [ + path + for base in (work_dir, requested.parent) + for path in base.rglob("*") + if path.name.endswith((".tmp", ".txn")) + ], + [], + ) + + def test_finalize_ignores_persistent_post_commit_cleanup_errors(self): + with TemporaryDirectory() as tmp: + root = Path(tmp).resolve() + repo_root, work_dir, _transport = _build_ready_state(root) + (work_dir / "private" / "decision.json").unlink() + requested = repo_root / "docs" / "pilot-evidence" / "loop-40" + real_bundle_finish = PrivateFinalizationBundle.finish + real_pack_finish = EvidencePackTransaction.finish + real_session_close = AnchoredPrivateSession.close + calls = {"bundle": 0, "session": 0} + pack_calls = {} + + def noisy_bundle_finish(bundle): + calls["bundle"] += 1 + real_bundle_finish(bundle) + raise OSError("persistent bundle finish failure") + + def noisy_pack_finish(transaction): + name = str(transaction.output) + pack_calls[name] = pack_calls.get(name, 0) + 1 + real_pack_finish(transaction) + raise OSError("persistent pack finish failure") + + def noisy_session_close(session): + calls["session"] += 1 + real_session_close(session) + raise OSError("persistent session close failure") + + with patch.object( + PrivateFinalizationBundle, + "finish", + new=noisy_bundle_finish, + ), patch.object( + EvidencePackTransaction, + "finish", + new=noisy_pack_finish, + ), patch.object( + AnchoredPrivateSession, + "close", + new=noisy_session_close, + ): + result = finalize_pilot( + repo_root, + work_dir, + _valid_decision(), + output_dir=requested, + now=NOW, + ) + + self.assertEqual(result["status"], "finalized") + self.assertEqual(result["output_dir"], str(requested)) + self.assertGreaterEqual(calls["bundle"], 2) + self.assertGreaterEqual(calls["session"], 2) + self.assertGreaterEqual(pack_calls[str(work_dir / "evidence")], 2) + self.assertGreaterEqual(pack_calls[str(requested)], 2) + self.assertEqual( + _json_bytes_map(work_dir / "evidence"), + _json_bytes_map(requested), + ) + self.assertEqual( + json.loads( + (work_dir / "private" / "finalization.json").read_text( + encoding="utf-8" + ) + ), + _valid_finalization_marker(), + ) + + def test_repository_export_rolls_back_when_authorization_entries_change_after_commit(self): + mutations = ("decision_replace", "marker_replace", "marker_mode", "marker_content") + for mutation in mutations: + with self.subTest(mutation=mutation), TemporaryDirectory() as tmp: + root = Path(tmp).resolve() + repo_root, work_dir, _transport = _build_ready_state(root) + private = work_dir / "private" + decision_path = private / "decision.json" + marker_path = private / "finalization.json" + _write_owner_only_json(marker_path, _valid_finalization_marker()) + requested = repo_root / "docs" / "pilot-evidence" / "loop-40" + real_commit = EvidencePackTransaction.commit + mutated = [] + + def commit_then_mutate(transaction): + result = real_commit(transaction) + if mutated: + return result + if mutation == "decision_replace": + replacement = _valid_decision() + replacement["rationale"] = "A different public rationale." + temporary = private / "replacement-decision.json" + _write_owner_only_json(temporary, replacement) + os.replace(temporary, decision_path) + elif mutation == "marker_replace": + temporary = private / "replacement-finalization.json" + _write_owner_only_json( + temporary, + _valid_finalization_marker(), + ) + os.replace(temporary, marker_path) + elif mutation == "marker_mode": + os.chmod(marker_path, 0o644) + else: + changed = _valid_finalization_marker() + changed["finalized_at"] = "2026-07-23T17:00:01+08:00" + _write_owner_only_json(marker_path, changed) + mutated.append(True) + return result + + with patch.object( + EvidencePackTransaction, + "commit", + new=commit_then_mutate, + ), self.assertRaisesRegex(ValueError, "authorization|changed|owner-only"): + generate_pilot_evidence( + repo_root, + work_dir, + output_dir=requested, + now=NOW, + ) + + self.assertEqual(mutated, [True]) + self.assertFalse(requested.exists()) + + def test_finalize_rolls_back_when_published_authorization_snapshot_changes(self): + with TemporaryDirectory() as tmp: + root = Path(tmp).resolve() + repo_root, work_dir, _transport = _build_ready_state(root) + private = work_dir / "private" + requested = repo_root / "docs" / "pilot-evidence" / "loop-40" + private_before = _json_bytes_map(work_dir / "evidence") + real_publish = PrivateFinalizationBundle.publish_marker + changed = [] + + def publish_then_replace_decision(bundle, marker): + observed = real_publish(bundle, marker) + replacement = _valid_decision() + replacement["rationale"] = "A different public rationale." + temporary = private / "replacement-decision.json" + _write_owner_only_json(temporary, replacement) + os.replace(temporary, private / "decision.json") + changed.append(True) + return observed + + with patch.object( + PrivateFinalizationBundle, + "publish_marker", + new=publish_then_replace_decision, + ), self.assertRaisesRegex(ValueError, "authorization|changed"): + finalize_pilot( + repo_root, + work_dir, + _valid_decision(), + output_dir=requested, + now=NOW, + ) + + self.assertEqual(changed, [True]) + self.assertEqual(_json_bytes_map(work_dir / "evidence"), private_before) + self.assertFalse(requested.exists()) + self.assertFalse((private / "finalization.json").exists()) + + def test_invalid_lock_rejections_do_not_leak_file_descriptors(self): + for invalid in ("fifo", "non_owner"): + with self.subTest(invalid=invalid), TemporaryDirectory() as tmp: + root = Path(tmp).resolve() + repo_root = _prepare_repo(root) + work_dir = root / "pilot" + initialize_pilot( + repo_root, + work_dir, + _valid_charter(), + now=NOW, + ) + lock = work_dir / "private" / AnchoredPrivateSession.LOCK_NAME + if invalid == "fifo": + os.mkfifo(lock, 0o600) + else: + lock.write_text("invalid", encoding="utf-8") + os.chmod(lock, 0o644) + before = len(os.listdir("/dev/fd")) + + for _attempt in range(24): + with self.assertRaisesRegex(ValueError, "lock|owner-only|regular"): + open_private_session(work_dir / "private") + + self.assertEqual(len(os.listdir("/dev/fd")), before) + + def test_lock_open_post_flock_failure_closes_descriptor_and_unlocks(self): + with TemporaryDirectory() as tmp: + root = Path(tmp).resolve() + repo_root = _prepare_repo(root) + work_dir = root / "pilot" + initialize_pilot(repo_root, work_dir, _valid_charter(), now=NOW) + before = len(os.listdir("/dev/fd")) + + with patch( + "skill2workflow._controlled_lark_pilot_private_authorization.os.fsync", + side_effect=OSError("lock fsync failed"), + ): + for _attempt in range(12): + with self.assertRaisesRegex(OSError, "lock fsync failed"): + open_private_session(work_dir / "private") + + self.assertEqual(len(os.listdir("/dev/fd")), before) + session = open_private_session(work_dir / "private") + session.close() + + def test_authorization_snapshot_close_retries_without_fd_leak(self): + with TemporaryDirectory() as tmp: + root = Path(tmp).resolve() + repo_root, work_dir, _transport = _build_ready_state(root) + private = work_dir / "private" + _write_owner_only_json( + private / "finalization.json", + _valid_finalization_marker(), + ) + before = len(os.listdir("/dev/fd")) + session = open_private_session(private) + snapshot = session.authorization_bundle_snapshot() + authorization = __import__( + "skill2workflow._controlled_lark_pilot_private_authorization", + fromlist=["_close_descriptors"], + ) + real_close = authorization._close_descriptors + failures = [] + + def fail_once(*descriptors): + if not failures: + failures.append(descriptors) + raise OSError("transient snapshot close failure") + return real_close(*descriptors) + + with patch.object( + authorization, + "_close_descriptors", + side_effect=fail_once, + ): + with self.assertRaisesRegex(OSError, "snapshot close failure"): + snapshot.close() + snapshot.close() + session.close() + + self.assertEqual(len(failures), 1) + self.assertTrue(snapshot.closed) + self.assertEqual(len(os.listdir("/dev/fd")), before) + + def test_lock_unlink_or_replace_invalidates_existing_session(self): + for mutation in ("unlink", "replace"): + with self.subTest(mutation=mutation), TemporaryDirectory() as tmp: + root = Path(tmp).resolve() + repo_root = _prepare_repo(root) + work_dir = root / "pilot" + initialize_pilot( + repo_root, + work_dir, + _valid_charter(), + now=NOW, + ) + private = work_dir / "private" + first = open_private_session(private) + second = None + try: + lock = private / AnchoredPrivateSession.LOCK_NAME + lock.unlink() + if mutation == "replace": + lock.write_text("replacement", encoding="utf-8") + os.chmod(lock, 0o600) + + with self.assertRaisesRegex(ValueError, "lock.*changed"): + first.check_identity() + second = open_private_session(private) + second.check_identity() + with self.assertRaisesRegex(ValueError, "lock.*changed"): + first.check_identity() + finally: + if second is not None: + second.close() + first.close() + + def test_finalize_rechecks_lock_identity_before_authorization_commit(self): + with TemporaryDirectory() as tmp: + root = Path(tmp).resolve() + repo_root, work_dir, _transport = _build_ready_state(root) + private = work_dir / "private" + requested = repo_root / "docs" / "pilot-evidence" / "loop-40" + private_before = _json_bytes_map(work_dir / "evidence") + real_commit = EvidencePackTransaction.commit + changed = [] + + def commit_then_replace_lock(transaction): + result = real_commit(transaction) + if not changed: + lock = private / AnchoredPrivateSession.LOCK_NAME + lock.unlink() + lock.write_text("replacement", encoding="utf-8") + os.chmod(lock, 0o600) + changed.append(True) + return result + + with patch.object( + EvidencePackTransaction, + "commit", + new=commit_then_replace_lock, + ), self.assertRaisesRegex(ValueError, "lock.*changed"): + finalize_pilot( + repo_root, + work_dir, + _valid_decision(), + output_dir=requested, + now=NOW, + ) + + self.assertEqual(changed, [True]) + self.assertEqual(_json_bytes_map(work_dir / "evidence"), private_before) + self.assertFalse(requested.exists()) + self.assertFalse((private / "finalization.json").exists()) + + def test_repository_export_fails_closed_while_private_authorization_is_locked(self): + with TemporaryDirectory() as tmp: + root = Path(tmp).resolve() + repo_root, work_dir, _transport = _build_ready_state(root) + _write_owner_only_json( + work_dir / "private" / "finalization.json", + _valid_finalization_marker(), + ) + requested = repo_root / "docs" / "pilot-evidence" / "loop-40" + + with open_private_session(work_dir / "private"): + with self.assertRaisesRegex(ValueError, "busy"): + generate_pilot_evidence( + repo_root, + work_dir, + output_dir=requested, + now=NOW, + ) + + self.assertFalse(requested.exists()) + + def test_repeated_finalization_fails_before_changing_committed_bundle_or_packs(self): + with TemporaryDirectory() as tmp: + root = Path(tmp).resolve() + repo_root, work_dir, _transport = _build_ready_state(root) + (work_dir / "private" / "decision.json").unlink() + requested = repo_root / "docs" / "pilot-evidence" / "loop-40" + decision = _valid_decision() + finalize_pilot( + repo_root, + work_dir, + decision, + output_dir=requested, + now=NOW, + ) + private_pack = _json_bytes_map(work_dir / "evidence") + requested_pack = _json_bytes_map(requested) + decision_bytes = (work_dir / "private" / "decision.json").read_bytes() + marker_bytes = (work_dir / "private" / "finalization.json").read_bytes() + + with self.assertRaisesRegex(ValueError, "already finalized"): + finalize_pilot( + repo_root, + work_dir, + decision, + output_dir=requested, + now=NOW + timedelta(minutes=1), + ) + + self.assertEqual(_json_bytes_map(work_dir / "evidence"), private_pack) + self.assertEqual(_json_bytes_map(requested), requested_pack) + self.assertEqual( + (work_dir / "private" / "decision.json").read_bytes(), + decision_bytes, + ) + self.assertEqual( + (work_dir / "private" / "finalization.json").read_bytes(), + marker_bytes, + ) + transient = [ + path + for base in (work_dir, requested.parent) + for path in base.rglob("*") + if path.name.endswith((".tmp", ".txn")) + ] + self.assertEqual(transient, []) + + def test_finalize_removes_marker_injected_after_decision_and_rolls_back_all_outputs(self): + with TemporaryDirectory() as tmp: + root = Path(tmp).resolve() + repo_root, work_dir, _transport = _build_ready_state(root) + decision_path = work_dir / "private" / "decision.json" + decision_path.unlink() + marker_path = work_dir / "private" / "finalization.json" + requested = repo_root / "docs" / "pilot-evidence" / "loop-40" + (work_dir / "evidence" / "old.json").write_text( + '{"pack":"old-private"}', encoding="utf-8" + ) + requested.mkdir(parents=True) + (requested / "old.json").write_text( + '{"pack":"old-requested"}', encoding="utf-8" + ) + private_before = _json_bytes_map(work_dir / "evidence") + requested_before = _json_bytes_map(requested) + real_replace = os.replace + injected = [] + + def inject_marker_after_decision( + source, + target, + *, + src_dir_fd=None, + dst_dir_fd=None, + ): + result = real_replace( + source, + target, + src_dir_fd=src_dir_fd, + dst_dir_fd=dst_dir_fd, + ) + if os.fspath(target) == "decision.json" and not injected: + _write_owner_only_json(marker_path, _valid_finalization_marker()) + injected.append(True) + return result + + with patch.object( + os, + "replace", + side_effect=inject_marker_after_decision, + ), self.assertRaises(ValueError): + finalize_pilot( + repo_root, + work_dir, + _valid_decision(), + output_dir=requested, + now=NOW, + ) + + self.assertEqual(injected, [True]) + with self.assertRaisesRegex(ValueError, "finalization"): + generate_pilot_evidence( + repo_root, + work_dir, + output_dir=requested, + now=NOW, + ) + self.assertEqual(_json_bytes_map(work_dir / "evidence"), private_before) + self.assertEqual(_json_bytes_map(requested), requested_before) + self.assertFalse(decision_path.exists()) + self.assertFalse(marker_path.exists()) + + def test_finalize_parent_swap_after_decision_cannot_leave_repository_authorization(self): + with TemporaryDirectory() as tmp: + root = Path(tmp).resolve() + repo_root, work_dir, _transport = _build_ready_state(root) + decision_path = work_dir / "private" / "decision.json" + decision_path.unlink() + private = work_dir / "private" + original_private = root / "original-private" + requested = repo_root / "docs" / "pilot-evidence" / "loop-40" + real_replace = os.replace + swapped = [] + + def swap_private_after_decision( + source, + target, + *, + src_dir_fd=None, + dst_dir_fd=None, + ): + result = real_replace( + source, + target, + src_dir_fd=src_dir_fd, + dst_dir_fd=dst_dir_fd, + ) + if os.fspath(target) == "decision.json" and not swapped: + private.rename(original_private) + shutil.copytree(original_private, private) + os.chmod(private, 0o700) + _write_owner_only_json( + private / "finalization.json", + _valid_finalization_marker(), + ) + swapped.append(True) + return result + + with patch.object( + os, + "replace", + side_effect=swap_private_after_decision, + ), self.assertRaises(ValueError): + finalize_pilot( + repo_root, + work_dir, + _valid_decision(), + output_dir=requested, + now=NOW, + ) + + self.assertEqual(swapped, [True]) + with self.assertRaisesRegex(ValueError, "finalization"): + generate_pilot_evidence( + repo_root, + work_dir, + output_dir=requested, + now=NOW, + ) + self.assertFalse((private / "finalization.json").exists()) + self.assertFalse(requested.exists()) + + def test_repository_export_rejects_private_swap_after_decision_open(self): + with TemporaryDirectory() as tmp: + root = Path(tmp).resolve() + repo_root, work_dir, _transport = _build_ready_state(root) + private = work_dir / "private" + original_private = root / "original-private" + _write_owner_only_json( + private / "finalization.json", + _valid_finalization_marker(), + ) + requested = repo_root / "docs" / "pilot-evidence" / "loop-40" + real_open = os.open + swapped = [] + + def swap_after_decision_open(path, flags, mode=0o777, *, dir_fd=None): + descriptor = ( + real_open(path, flags, mode) + if dir_fd is None + else real_open(path, flags, mode, dir_fd=dir_fd) + ) + if ( + dir_fd is not None + and os.fspath(path) == "decision.json" + and not swapped + ): + private.rename(original_private) + private.mkdir(mode=0o700) + _write_owner_only_json( + private / "finalization.json", + _valid_finalization_marker(), + ) + swapped.append(True) + return descriptor + + with patch.object( + os, + "open", + side_effect=swap_after_decision_open, + ), self.assertRaisesRegex(ValueError, "private|authorization|changed"): + generate_pilot_evidence( + repo_root, + work_dir, + output_dir=requested, + now=NOW, + ) + + self.assertEqual(swapped, [True]) + self.assertFalse(requested.exists()) + + def test_repository_authorization_requires_owner_only_parent_decision_and_marker(self): + for permissive in ("parent", "decision", "marker"): + with self.subTest(permissive=permissive), TemporaryDirectory() as tmp: + root = Path(tmp).resolve() + repo_root, work_dir, _transport = _build_ready_state(root) + private = work_dir / "private" + marker = private / "finalization.json" + _write_owner_only_json(marker, _valid_finalization_marker()) + target = { + "parent": private, + "decision": private / "decision.json", + "marker": marker, + }[permissive] + os.chmod(target, 0o755 if permissive == "parent" else 0o644) + requested = repo_root / "docs" / "pilot-evidence" / "loop-40" + + with self.assertRaisesRegex(ValueError, "owner-only"): + generate_pilot_evidence( + repo_root, + work_dir, + output_dir=requested, + now=NOW, + ) + + self.assertFalse(requested.exists()) + + def test_finalize_decision_publication_failure_restores_both_old_packs(self): + with TemporaryDirectory() as tmp: + root = Path(tmp).resolve() + repo_root, work_dir, _transport = _build_ready_state(root) + decision_path = work_dir / "private" / "decision.json" + decision_path.unlink() + requested = repo_root / "docs" / "pilot-evidence" / "loop-40" + (work_dir / "evidence" / "old.json").write_text( + '{"pack":"old-private"}', encoding="utf-8" + ) + requested.mkdir(parents=True) + (requested / "old.json").write_text( + '{"pack":"old-requested"}', encoding="utf-8" + ) + private_before = _json_bytes_map(work_dir / "evidence") + requested_before = _json_bytes_map(requested) + real_replace = os.replace + + def fail_decision_publish( + source, + target, + *, + src_dir_fd=None, + dst_dir_fd=None, + ): + if os.fspath(target) == "decision.json": + raise OSError("decision publication failed") + return real_replace( + source, + target, + src_dir_fd=src_dir_fd, + dst_dir_fd=dst_dir_fd, + ) + + with patch.object( + os, + "replace", + side_effect=fail_decision_publish, + ), self.assertRaisesRegex(OSError, "decision publication failed"): + finalize_pilot( + repo_root, + work_dir, + _valid_decision(), + output_dir=requested, + now=NOW, + ) + + self.assertEqual(_json_bytes_map(work_dir / "evidence"), private_before) + self.assertEqual(_json_bytes_map(requested), requested_before) + self.assertFalse(decision_path.exists()) + self.assertFalse((work_dir / "private" / "finalization.json").exists()) + + def test_finalize_rejects_incomplete_evidence_before_any_write(self): + expected_unmet = ( + "approved_live_runs_threshold, distinct_calendar_days_threshold, " + "distinct_private_cases_threshold, human_rejection, " + "disabled_live_exercise, rollback_exercise, verification" + ) + with TemporaryDirectory() as tmp: + root = Path(tmp).resolve() + repo_root = _prepare_repo(root) + work_dir = root / "pilot" + initialize_pilot(repo_root, work_dir, _valid_charter(), now=NOW) + requested = repo_root / "docs" / "pilot-evidence" / "loop-40" + + with self.assertRaisesRegex( + ValueError, + expected_unmet, + ): + finalize_pilot( + repo_root, + work_dir, + _valid_decision(), + output_dir=requested, + now=NOW, + ) + + self.assertEqual(list((work_dir / "evidence").iterdir()), []) + self.assertFalse((work_dir / "private" / "decision.json").exists()) + self.assertFalse( + (work_dir / "private" / "finalization.json").exists() + ) + self.assertFalse(requested.exists()) + + def test_finalize_rejects_invalid_or_private_decisions_before_any_write(self): + with TemporaryDirectory() as tmp: + root = Path(tmp).resolve() + repo_root, work_dir, _transport = _build_ready_state(root) + (work_dir / "private" / "decision.json").unlink() + requested = repo_root / "docs" / "pilot-evidence" / "loop-40" + invalid_decisions = [] + for key, value in ( + ("decision", "ship"), + ("partner_acknowledged", False), + ("operator_acknowledged", False), + ("commercial_engagement_confirmed", False), + ("partner_acknowledged", 1), + ("rationale", ""), + ("rationale", " "), + ("rationale", "Private Account case-001 was successful."), + ): + candidate = _valid_decision() + candidate[key] = value + invalid_decisions.append(candidate) + missing = _valid_decision() + del missing["schema_version"] + invalid_decisions.append(missing) + extra = _valid_decision() + extra["customer_name"] = "Private Account" + invalid_decisions.append(extra) + + for candidate in invalid_decisions: + with self.subTest(candidate=candidate), self.assertRaises(ValueError): + finalize_pilot( + repo_root, + work_dir, + candidate, + output_dir=requested, + now=NOW, + ) + self.assertEqual(list((work_dir / "evidence").iterdir()), []) + self.assertFalse( + (work_dir / "private" / "decision.json").exists() + ) + self.assertFalse( + (work_dir / "private" / "finalization.json").exists() + ) + self.assertFalse(requested.exists()) + + def test_finalize_writes_equivalent_private_and_requested_packs_then_marker(self): + with TemporaryDirectory() as tmp: + root = Path(tmp).resolve() + repo_root, work_dir, _transport = _build_ready_state(root) + (work_dir / "private" / "decision.json").unlink() + requested = repo_root / "docs" / "pilot-evidence" / "loop-40" + (work_dir / "evidence" / "notes.txt").write_text( + "old private note", encoding="utf-8" + ) + requested.mkdir(parents=True) + (requested / ".old-note").write_text( + "old repository note", encoding="utf-8" + ) + nested_note = requested / "archive" / "notes.txt" + nested_note.parent.mkdir() + nested_note.write_text("old nested note", encoding="utf-8") + decision = _valid_decision() + + result = finalize_pilot( + repo_root, + work_dir, + decision, + output_dir=requested, + now=NOW, + ) + + private_pack = _json_bytes_map(work_dir / "evidence") + requested_pack = _json_bytes_map(requested) + private_all = _all_bytes_map(work_dir / "evidence") + requested_all = _all_bytes_map(requested) + persisted_decision = json.loads( + (work_dir / "private" / "decision.json").read_text( + encoding="utf-8" + ) + ) + marker_path = work_dir / "private" / "finalization.json" + marker = json.loads(marker_path.read_text(encoding="utf-8")) + decision_mode = ( + work_dir / "private" / "decision.json" + ).stat().st_mode & 0o077 + marker_mode = marker_path.stat().st_mode & 0o077 + encoded = b"".join(requested_pack.values()) + json.dumps(result).encode() + + self.assertEqual(private_pack, requested_pack) + self.assertEqual(private_all, private_pack) + self.assertEqual(requested_all, requested_pack) + self.assertEqual(persisted_decision, decision) + self.assertEqual( + marker, + { + "schema_version": "controlled-lark-pilot-finalization-0.1.0", + "finalized": True, + "decision": "continue", + "finalized_at": "2026-07-23T17:00:00+08:00", + }, + ) + self.assertEqual(decision_mode, 0) + self.assertEqual(marker_mode, 0) + self.assertEqual( + result, + { + "status": "finalized", + "decision": "continue", + "approved_live_runs": 5, + "distinct_calendar_days": 5, + "distinct_private_cases": 2, + "rejected_runs": 1, + "output_dir": str(requested), + }, + ) + for forbidden in ( + "case-001", + "case-002", + "case-rejected", + "Private Account", + "Private Risk", + "ou_private", + "private-token", + "private-provider-message", + "private-task-guid", + ): + self.assertNotIn(forbidden.encode("utf-8"), encoded) + + stale = requested / "stale.json" + stale.write_text("{}", encoding="utf-8") + regenerated = generate_pilot_evidence( + repo_root, + work_dir, + output_dir=requested, + now=NOW + timedelta(seconds=1), + ) + self.assertEqual(regenerated["output_dir"], str(requested)) + self.assertFalse(stale.exists()) + self.assertEqual( + json.loads( + (requested / "decision.json").read_text(encoding="utf-8") + ), + decision, + ) + + def test_repository_regeneration_allows_only_successfully_finalized_expired_charter(self): + expired_now = NOW + timedelta(days=25) + with TemporaryDirectory() as tmp: + root = Path(tmp).resolve() + repo_root, work_dir, _transport = _build_ready_state(root) + (work_dir / "private" / "decision.json").unlink() + requested = repo_root / "docs" / "pilot-evidence" / "loop-40" + finalize_pilot( + repo_root, + work_dir, + _valid_decision(), + output_dir=requested, + now=NOW, + ) + (requested / "notes.txt").write_text( + "must not survive regeneration", encoding="utf-8" + ) + + regenerated = generate_pilot_evidence( + repo_root, + work_dir, + output_dir=requested, + now=expired_now, + ) + + self.assertEqual(regenerated["output_dir"], str(requested)) + self.assertEqual(regenerated["unmet_conditions"], []) + self.assertFalse((requested / "notes.txt").exists()) + with self.assertRaisesRegex(ValueError, "expired"): + generate_pilot_evidence( + repo_root, + work_dir, + now=expired_now, + ) + with self.assertRaisesRegex(ValueError, "expired"): + generate_pilot_evidence( + repo_root, + work_dir, + output_dir=root / "external-evidence", + now=expired_now, + ) + + before = _all_bytes_map(requested) + marker = work_dir / "private" / "finalization.json" + valid_marker = json.loads(marker.read_text(encoding="utf-8")) + marker.unlink() + with self.assertRaisesRegex(ValueError, "finalization"): + generate_pilot_evidence( + repo_root, + work_dir, + output_dir=requested, + now=expired_now, + ) + self.assertEqual(_all_bytes_map(requested), before) + + invalid_marker = dict(valid_marker) + invalid_marker["finalized_at"] = "2026-08-16T09:00:00+08:00" + _write_owner_only_json(marker, invalid_marker) + with self.assertRaisesRegex(ValueError, "finalization|expired"): + generate_pilot_evidence( + repo_root, + work_dir, + output_dir=requested, + now=expired_now, + ) + self.assertEqual(_all_bytes_map(requested), before) + + def test_expired_incomplete_pack_cannot_borrow_repository_authorization(self): + with TemporaryDirectory() as tmp: + root = Path(tmp).resolve() + repo_root = _prepare_repo(root) + work_dir = root / "pilot" + initialize_pilot(repo_root, work_dir, _valid_charter(), now=NOW) + private = work_dir / "private" + _write_owner_only_json(private / "decision.json", _valid_decision()) + _write_owner_only_json( + private / "finalization.json", + _valid_finalization_marker(), + ) + requested = repo_root / "docs" / "pilot-evidence" / "loop-40" + + with self.assertRaisesRegex(ValueError, "does not authorize"): + generate_pilot_evidence( + repo_root, + work_dir, + output_dir=requested, + now=NOW + timedelta(days=25), + ) + + self.assertFalse(requested.exists()) + + def test_finalize_allows_external_export_but_rejects_other_repository_path(self): + with TemporaryDirectory() as tmp: + root = Path(tmp).resolve() + repo_root, work_dir, _transport = _build_ready_state(root) + (work_dir / "private" / "decision.json").unlink() + wrong = repo_root / "docs" / "pilot-evidence" / "other" + + with self.assertRaisesRegex(ValueError, "repository evidence output"): + finalize_pilot( + repo_root, + work_dir, + _valid_decision(), + output_dir=wrong, + now=NOW, + ) + + self.assertEqual(list((work_dir / "evidence").iterdir()), []) + self.assertFalse(wrong.exists()) + self.assertFalse((work_dir / "private" / "decision.json").exists()) + self.assertFalse( + (work_dir / "private" / "finalization.json").exists() + ) + + external = root / "shared-redacted-evidence" + finalized = finalize_pilot( + repo_root, + work_dir, + _valid_decision(), + output_dir=external, + now=NOW, + ) + self.assertEqual(finalized["output_dir"], str(external)) + self.assertEqual( + _json_bytes_map(work_dir / "evidence"), + _json_bytes_map(external), + ) + + def test_finalize_rejects_declared_repo_symlink_before_private_pack_write(self): + with TemporaryDirectory() as tmp: + root = Path(tmp).resolve() + repo_root, work_dir, _transport = _build_ready_state(root) + (work_dir / "private" / "decision.json").unlink() + outside = root / "outside-output" + outside.mkdir() + sentinel = outside / "sentinel.txt" + sentinel.write_text("sentinel", encoding="utf-8") + linked_parent = repo_root / "docs" / "pilot-evidence" + linked_parent.mkdir(parents=True) + linked = linked_parent / "wrong" + linked.symlink_to(outside, target_is_directory=True) + + with self.assertRaisesRegex(ValueError, "repository evidence output"): + finalize_pilot( + repo_root, + work_dir, + _valid_decision(), + output_dir=linked, + now=NOW, + ) + + self.assertEqual(list((work_dir / "evidence").iterdir()), []) + self.assertEqual(sentinel.read_text(encoding="utf-8"), "sentinel") + self.assertFalse((work_dir / "private" / "decision.json").exists()) + self.assertFalse( + (work_dir / "private" / "finalization.json").exists() + ) + + def test_finalize_write_failure_never_leaves_decision_or_marker(self): + with TemporaryDirectory() as tmp: + root = Path(tmp).resolve() + repo_root, work_dir, _transport = _build_ready_state(root) + (work_dir / "private" / "decision.json").unlink() + requested = root / "requested-evidence" + private_note = work_dir / "evidence" / "notes.txt" + private_note.write_text("old private note", encoding="utf-8") + requested.mkdir() + requested_note = requested / "notes.txt" + requested_note.write_text("old requested note", encoding="utf-8") + private_before = _all_bytes_map(work_dir / "evidence") + requested_before = _all_bytes_map(requested) + writes = [] + + def failing_second_write(output_dir, pack): + writes.append(Path(output_dir)) + transaction = prepare_evidence_pack_transaction(output_dir, pack) + if len(writes) == 2: + transaction.commit = lambda: (_ for _ in ()).throw( + OSError("requested export failed") + ) + return transaction + + with patch( + "skill2workflow.controlled_lark_pilot.prepare_evidence_pack_transaction", + side_effect=failing_second_write, + ), self.assertRaisesRegex(OSError, "requested export failed"): + finalize_pilot( + repo_root, + work_dir, + _valid_decision(), + output_dir=requested, + now=NOW, + ) + + self.assertEqual(writes, [work_dir / "evidence", requested]) + self.assertEqual(_all_bytes_map(work_dir / "evidence"), private_before) + self.assertEqual(_all_bytes_map(requested), requested_before) + self.assertFalse((work_dir / "private" / "decision.json").exists()) + self.assertFalse( + (work_dir / "private" / "finalization.json").exists() + ) + self.assertEqual( + [ + path + for path in work_dir.rglob("*") + if path.name.endswith((".tmp", ".txn")) + ], + [], + ) + + def test_finalize_rejects_static_private_decision_or_marker_symlink_before_pack_write(self): + for artifact in ("decision.json", "finalization.json"): + with self.subTest(artifact=artifact), TemporaryDirectory() as tmp: + root = Path(tmp).resolve() + repo_root, work_dir, _transport = _build_ready_state(root) + decision_path = work_dir / "private" / "decision.json" + decision_path.unlink() + target = root / f"outside-{artifact}" + target.write_text("sentinel", encoding="utf-8") + linked = work_dir / "private" / artifact + linked.symlink_to(target) + + with self.assertRaisesRegex( + ValueError, + "symbolic link|regular file", + ): + finalize_pilot( + repo_root, + work_dir, + _valid_decision(), + output_dir=root / "requested", + now=NOW, + ) + + self.assertEqual(list((work_dir / "evidence").iterdir()), []) + self.assertEqual(target.read_text(encoding="utf-8"), "sentinel") + self.assertTrue(linked.is_symlink()) + + def test_generate_evidence_from_real_sqlite_state_is_redacted_stable_and_ready(self): + with TemporaryDirectory() as tmp: + root = Path(tmp).resolve() + repo_root, work_dir, transport = _build_ready_state(root) + with patch.dict( + os.environ, + {"LARK_BOT_ACCESS_TOKEN": "private-token"}, + clear=True, + ): + result = generate_pilot_evidence(repo_root, work_dir, now=NOW) + + evidence_dir = work_dir / "evidence" + index = json.loads( + (evidence_dir / "evidence-index.json").read_text(encoding="utf-8") + ) + run_files = sorted((evidence_dir / "runs").glob("*.json")) + runs = [json.loads(path.read_text(encoding="utf-8")) for path in run_files] + encoded = "".join( + path.read_text(encoding="utf-8") + for path in evidence_dir.rglob("*.json") + ) + native_tokens = [ + json.loads(request.data.decode("utf-8"))["client_token"] + for request, _timeout in transport.calls + ] + + self.assertEqual(result["output_dir"], str(evidence_dir)) + self.assertEqual(result["run_count"], 6) + self.assertEqual(result["distinct_private_cases"], 2) + self.assertEqual(result["unmet_conditions"], []) + self.assertTrue(index["ready_to_finalize"]) + self.assertEqual(index["approved_live_runs"], 5) + self.assertEqual(index["distinct_calendar_days"], 5) + self.assertEqual(index["rejected_runs"], 1) + self.assertEqual(index["generated_at"], "2026-07-23T17:00:00+08:00") + self.assertEqual(len(run_files), 6) + self.assertEqual( + [(run["started_at"], run["run_id"]) for run in runs], + sorted((run["started_at"], run["run_id"]) for run in runs), + ) + self.assertEqual(len(transport.calls), 5) + for forbidden in ( + "case-001", + "case-002", + "case-rejected", + "Private Account", + "Private Risk", + "ou_private", + "private-token", + "private-provider-message", + "private-task-guid", + *native_tokens, + ): + self.assertNotIn(forbidden, encoded) + self.assertNotIn(forbidden, json.dumps(result)) + + finalization = work_dir / "private" / "finalization.json" + repo_output = repo_root / "docs" / "pilot-evidence" / "loop-40" + valid_marker = { + "schema_version": "controlled-lark-pilot-finalization-0.1.0", + "finalized": True, + "decision": "continue", + "finalized_at": "2026-07-23T17:00:00+08:00", + } + marker_target = root / "outside-finalization.json" + marker_target.write_text(json.dumps(valid_marker), encoding="utf-8") + finalization.symlink_to(marker_target) + with self.assertRaisesRegex(ValueError, "symbolic link"): + generate_pilot_evidence( + repo_root, + work_dir, + output_dir=repo_output, + now=NOW, + ) + self.assertEqual( + json.loads(marker_target.read_text(encoding="utf-8")), valid_marker + ) + self.assertFalse(repo_output.exists()) + finalization.unlink() + + finalization.mkdir() + with self.assertRaisesRegex(ValueError, "regular file"): + generate_pilot_evidence( + repo_root, + work_dir, + output_dir=repo_output, + now=NOW, + ) + self.assertFalse(repo_output.exists()) + finalization.rmdir() + + invalid_markers = [ + {**valid_marker, "finalized": 1}, + {**valid_marker, "decision": "defer"}, + {**valid_marker, "finalized_at": "2026-07-23T17:00:00"}, + ] + for marker in invalid_markers: + _write_owner_only_json(finalization, marker) + with self.subTest(marker=marker), self.assertRaises(ValueError): + generate_pilot_evidence( + repo_root, + work_dir, + output_dir=repo_output, + now=NOW, + ) + self.assertFalse(repo_output.exists()) + + _write_owner_only_json(finalization, valid_marker) + exported = generate_pilot_evidence( + repo_root, + work_dir, + output_dir=repo_output, + now=NOW, + ) + self.assertEqual(exported["output_dir"], str(repo_output)) + self.assertTrue((repo_output / "evidence-index.json").is_file()) + + def test_generate_evidence_allows_external_explicit_output_without_finalization(self): + with TemporaryDirectory() as tmp: + root = Path(tmp).resolve() + repo_root = _prepare_repo(root) + work_dir = root / "pilot" + initialize_pilot(repo_root, work_dir, _valid_charter(), now=NOW) + output = root / "external-evidence" + + result = generate_pilot_evidence( + repo_root, + work_dir, + output_dir=output, + now=NOW, + ) + + self.assertEqual(result["output_dir"], str(output)) + self.assertEqual(result["run_count"], 0) + self.assertTrue((output / "pilot-charter.json").is_file()) + self.assertFalse((work_dir / "private" / "finalization.json").exists()) + + def test_generate_evidence_rejects_unfinalized_or_wrong_repository_output(self): + with TemporaryDirectory() as tmp: + root = Path(tmp).resolve() + repo_root = _prepare_repo(root) + work_dir = root / "pilot" + initialize_pilot(repo_root, work_dir, _valid_charter(), now=NOW) + exact = repo_root / "docs" / "pilot-evidence" / "loop-40" + + with self.assertRaisesRegex(ValueError, "finalization"): + generate_pilot_evidence( + repo_root, + work_dir, + output_dir=exact, + now=NOW, + ) + with self.assertRaisesRegex(ValueError, "repository evidence output"): + generate_pilot_evidence( + repo_root, + work_dir, + output_dir=repo_root / "docs" / "pilot-evidence" / "other", + now=NOW, + ) + + self.assertFalse(exact.exists()) + + def test_repository_export_rejects_marker_swapped_to_symlink_before_open(self): + with TemporaryDirectory() as tmp: + root = Path(tmp).resolve() + repo_root, work_dir, _transport = _build_ready_state(root) + finalization = work_dir / "private" / "finalization.json" + valid_marker = { + "schema_version": "controlled-lark-pilot-finalization-0.1.0", + "finalized": True, + "decision": "continue", + "finalized_at": "2026-07-23T17:00:00+08:00", + } + finalization.write_text(json.dumps(valid_marker), encoding="utf-8") + original_marker = root / "original-finalization.json" + outside_marker = root / "outside-finalization.json" + outside_marker.write_text(json.dumps(valid_marker), encoding="utf-8") + output = repo_root / "docs" / "pilot-evidence" / "loop-40" + real_exists = Path.exists + real_open = os.open + swapped = [] + + def racing_exists(path): + if Path(path) == finalization and not swapped: + finalization.rename(original_marker) + finalization.symlink_to(outside_marker) + swapped.append(True) + return real_exists(path) + + def racing_open(path, flags, mode=0o777, *, dir_fd=None): + if os.fspath(path) == "finalization.json" and not swapped: + finalization.rename(original_marker) + finalization.symlink_to(outside_marker) + swapped.append(True) + if dir_fd is None: + return real_open(path, flags, mode) + return real_open(path, flags, mode, dir_fd=dir_fd) + + with patch.object( + Path, + "exists", + autospec=True, + side_effect=racing_exists, + ), patch.object(os, "open", side_effect=racing_open), self.assertRaisesRegex( + ValueError, + "symbolic link|regular file", + ): + generate_pilot_evidence( + repo_root, + work_dir, + output_dir=output, + now=NOW, + ) + + self.assertEqual(swapped, [True]) + self.assertEqual( + json.loads(outside_marker.read_text(encoding="utf-8")), valid_marker + ) + self.assertFalse(output.exists()) + + +if __name__ == "__main__": + import unittest + + unittest.main() diff --git a/tests/test_controlled_lark_pilot_evidence_writer.py b/tests/test_controlled_lark_pilot_evidence_writer.py new file mode 100644 index 0000000..01076c5 --- /dev/null +++ b/tests/test_controlled_lark_pilot_evidence_writer.py @@ -0,0 +1,676 @@ +import json +import os +import subprocess +import sys +from copy import deepcopy +from pathlib import Path +from tempfile import TemporaryDirectory +from unittest import TestCase +from unittest.mock import patch + +from skill2workflow.controlled_lark_pilot_evidence import ( + build_acceptance_summary, + write_evidence_pack, +) +from skill2workflow._controlled_lark_pilot_evidence_writer import ( + read_json_anchored, + write_private_json_anchored, +) + +from tests.test_controlled_lark_pilot_evidence import _valid_pack + + +class ControlledLarkPilotEvidenceWriterTests(TestCase): + def test_write_pack_retries_transient_finish_cleanup_after_durable_commit(self): + with TemporaryDirectory() as tmp: + root = Path(tmp).resolve() + output = root / "evidence" + writer = __import__( + "skill2workflow._controlled_lark_pilot_evidence_writer", + fromlist=["_remove_tree_at"], + ) + real_remove = writer._remove_tree_at + failures = [] + + def fail_once(parent_fd, name): + if name.endswith(".txn") and not failures: + failures.append(name) + raise OSError("transient finish cleanup failure") + return real_remove(parent_fd, name) + + with patch.object(writer, "_remove_tree_at", side_effect=fail_once): + result = write_evidence_pack(output, _valid_pack()) + + self.assertEqual( + result, + { + "status": "written", + "file_count": 13, + "output_dir": str(output), + }, + ) + self.assertEqual(len(failures), 1) + self.assertEqual( + [path for path in root.iterdir() if path.name.endswith(".txn")], + [], + ) + self.assertTrue((output / "evidence-index.json").is_file()) + + def test_write_pack_persistent_finish_cleanup_keeps_durable_success(self): + with TemporaryDirectory() as tmp: + root = Path(tmp).resolve() + output = root / "evidence" + writer = __import__( + "skill2workflow._controlled_lark_pilot_evidence_writer", + fromlist=["_remove_tree_at"], + ) + real_remove = writer._remove_tree_at + failures = [] + + def fail_transaction_cleanup(parent_fd, name): + if name.endswith(".txn"): + failures.append(name) + raise OSError("persistent finish cleanup failure") + return real_remove(parent_fd, name) + + with patch.object( + writer, + "_remove_tree_at", + side_effect=fail_transaction_cleanup, + ): + result = write_evidence_pack(output, _valid_pack()) + + residual = [ + path for path in root.iterdir() if path.name.endswith(".txn") + ] + self.assertEqual(result["status"], "written") + self.assertEqual(result["output_dir"], str(output)) + self.assertGreaterEqual(len(failures), 2) + self.assertEqual(len(residual), 1) + self.assertEqual(residual[0].stat().st_mode & 0o077, 0) + self.assertTrue((output / "evidence-index.json").is_file()) + + def test_anchored_json_reader_rejects_fifo_without_blocking(self): + with TemporaryDirectory() as tmp: + fifo = Path(tmp).resolve() / "private.json" + os.mkfifo(fifo, 0o600) + script = ( + "from pathlib import Path; " + "from skill2workflow._controlled_lark_pilot_evidence_writer " + "import read_json_anchored; " + f"path = Path({str(fifo)!r}); " + "\ntry:\n read_json_anchored(path)" + "\nexcept ValueError as error:\n print(str(error)); raise SystemExit(0)" + "\nraise SystemExit(2)" + ) + process = subprocess.Popen( + [sys.executable, "-c", script], + cwd=Path(__file__).resolve().parents[1], + env={**os.environ, "PYTHONPATH": "src"}, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + text=True, + ) + try: + stdout, stderr = process.communicate(timeout=1.0) + except subprocess.TimeoutExpired: + process.kill() + process.communicate() + self.fail("anchored JSON reader blocked on a FIFO") + + self.assertEqual(process.returncode, 0, stderr) + self.assertIn("regular file", stdout) + + def test_require_missing_private_json_publish_never_overwrites_racing_target(self): + with TemporaryDirectory() as tmp: + private = Path(tmp).resolve() / "pilot" / "private" + private.mkdir(parents=True) + os.chmod(private, 0o700) + marker = private / "finalization.json" + racing_value = {"sentinel": "must-survive"} + real_stat = os.stat + observations = [] + + def racing_stat(target, *, dir_fd=None, follow_symlinks=True): + if ( + dir_fd is not None + and os.fspath(target) == "finalization.json" + and follow_symlinks is False + ): + observations.append(True) + try: + return real_stat( + target, + dir_fd=dir_fd, + follow_symlinks=follow_symlinks, + ) + except FileNotFoundError: + if len(observations) == 2: + descriptor = os.open( + target, + os.O_WRONLY | os.O_CREAT | os.O_EXCL, + 0o600, + dir_fd=dir_fd, + ) + with os.fdopen(descriptor, "w", encoding="utf-8") as handle: + json.dump(racing_value, handle) + raise + if dir_fd is None: + return real_stat(target, follow_symlinks=follow_symlinks) + return real_stat( + target, + dir_fd=dir_fd, + follow_symlinks=follow_symlinks, + ) + + with patch.object(os, "stat", side_effect=racing_stat), self.assertRaises( + FileExistsError + ): + write_private_json_anchored( + marker, + {"finalized": True}, + require_missing=True, + ) + + self.assertGreaterEqual(len(observations), 2) + self.assertEqual(json.loads(marker.read_text(encoding="utf-8")), racing_value) + + def test_private_json_writer_is_owner_only_atomic_and_rejects_static_links(self): + with TemporaryDirectory() as tmp: + root = Path(tmp).resolve() + private = root / "pilot" / "private" / "exercises" + path = private / "failure.json" + + write_private_json_anchored(path, {"passed": True}) + + self.assertEqual( + json.loads(path.read_text(encoding="utf-8")), {"passed": True} + ) + self.assertEqual(path.stat().st_mode & 0o077, 0) + self.assertEqual(private.stat().st_mode & 0o077, 0) + self.assertFalse( + any(item.name.endswith(".tmp") for item in private.iterdir()) + ) + + target = root / "outside.json" + target.write_text("sentinel", encoding="utf-8") + path.unlink() + path.symlink_to(target) + with self.assertRaisesRegex(ValueError, "symbolic link"): + write_private_json_anchored(path, {"passed": False}) + self.assertEqual(target.read_text(encoding="utf-8"), "sentinel") + + path.unlink() + path.mkdir() + with self.assertRaisesRegex(ValueError, "regular file"): + write_private_json_anchored(path, {"passed": False}) + + def test_private_json_writer_rejects_parent_path_swap_without_redirecting(self): + with TemporaryDirectory() as tmp: + root = Path(tmp).resolve() + private = root / "pilot" / "private" + private.mkdir(parents=True) + os.chmod(private, 0o700) + anchored = root / "anchored-private" + outside = root / "outside" + outside.mkdir() + sentinel = outside / "sentinel.json" + sentinel.write_text("sentinel", encoding="utf-8") + real_open = os.open + swapped = [] + + def racing_open(path, flags, mode=0o777, *, dir_fd=None): + name = os.fspath(path) + if not swapped and name.endswith(".tmp"): + private.rename(anchored) + private.symlink_to(outside, target_is_directory=True) + swapped.append(True) + if dir_fd is None: + return real_open(path, flags, mode) + return real_open(path, flags, mode, dir_fd=dir_fd) + + with patch.object(os, "open", side_effect=racing_open), self.assertRaisesRegex( + ValueError, + "declared private path", + ): + write_private_json_anchored( + private / "decision.json", {"decision": "continue"} + ) + + self.assertEqual(swapped, [True]) + self.assertEqual(sentinel.read_text(encoding="utf-8"), "sentinel") + self.assertEqual(list(outside.glob("*.json")), [sentinel]) + self.assertTrue((anchored / "decision.json").is_file()) + + def test_private_json_writer_removes_new_marker_when_parent_swap_fails(self): + with TemporaryDirectory() as tmp: + root = Path(tmp).resolve() + private = root / "pilot" / "private" + private.mkdir(parents=True) + os.chmod(private, 0o700) + anchored = root / "anchored-private" + outside = root / "outside" + outside.mkdir() + sentinel = outside / "sentinel.json" + sentinel.write_text("sentinel", encoding="utf-8") + real_open = os.open + swapped = [] + + def racing_open(path, flags, mode=0o777, *, dir_fd=None): + name = os.fspath(path) + if not swapped and name.endswith(".tmp"): + private.rename(anchored) + private.symlink_to(outside, target_is_directory=True) + swapped.append(True) + if dir_fd is None: + return real_open(path, flags, mode) + return real_open(path, flags, mode, dir_fd=dir_fd) + + with patch.object(os, "open", side_effect=racing_open), self.assertRaisesRegex( + ValueError, + "declared private path", + ): + write_private_json_anchored( + private / "finalization.json", + {"finalized": True}, + require_missing=True, + ) + + self.assertEqual(swapped, [True]) + self.assertFalse((anchored / "finalization.json").exists()) + self.assertEqual(sentinel.read_text(encoding="utf-8"), "sentinel") + self.assertEqual(list(outside.glob("*.json")), [sentinel]) + + def test_private_json_writer_rejects_final_path_swap_before_replace(self): + with TemporaryDirectory() as tmp: + root = Path(tmp).resolve() + private = root / "pilot" / "private" + private.mkdir(parents=True) + os.chmod(private, 0o700) + path = private / "decision.json" + path.write_text("{}", encoding="utf-8") + backup = private / "original-decision.json" + outside = root / "outside.json" + outside.write_text("sentinel", encoding="utf-8") + real_stat = os.stat + observed = [] + + def racing_stat(target, *, dir_fd=None, follow_symlinks=True): + if ( + dir_fd is not None + and os.fspath(target) == "decision.json" + and follow_symlinks is False + ): + observed.append(True) + if len(observed) == 2: + path.rename(backup) + path.symlink_to(outside) + if dir_fd is None: + return real_stat(target, follow_symlinks=follow_symlinks) + return real_stat( + target, + dir_fd=dir_fd, + follow_symlinks=follow_symlinks, + ) + + with patch.object(os, "stat", side_effect=racing_stat), self.assertRaisesRegex( + ValueError, + "changed|symbolic link", + ): + write_private_json_anchored(path, {"decision": "continue"}) + + self.assertEqual(len(observed), 2) + self.assertEqual(outside.read_text(encoding="utf-8"), "sentinel") + self.assertTrue(path.is_symlink()) + self.assertFalse( + any(item.name.endswith(".tmp") for item in private.iterdir()) + ) + + def test_write_pack_is_atomic_idempotent_and_replaces_with_exact_allowlist(self): + pack = _valid_pack() + with TemporaryDirectory() as tmp: + output = Path(tmp).resolve() / "evidence" + stale = output / "runs" / "999.json" + stale.parent.mkdir(parents=True) + stale.write_text("{}", encoding="utf-8") + keep = output / "notes.txt" + keep.write_text("keep", encoding="utf-8") + hidden = output / ".private-note" + hidden.write_text("hidden", encoding="utf-8") + nested = output / "archive" / "notes.txt" + nested.parent.mkdir() + nested.write_text("nested", encoding="utf-8") + + first = write_evidence_pack(output, pack) + second = write_evidence_pack(output, pack) + + self.assertEqual(first["file_count"], second["file_count"]) + self.assertFalse(stale.exists()) + self.assertFalse(keep.exists()) + self.assertFalse(hidden.exists()) + self.assertFalse(nested.exists()) + self.assertFalse((output / "archive").exists()) + self.assertFalse( + any(path.name.endswith(".tmp") for path in output.rglob("*")) + ) + self.assertEqual( + json.loads((output / "runs" / "001.json").read_text(encoding="utf-8")), + pack["runs"][0], + ) + + def test_write_pack_second_file_failure_leaves_missing_target_absent(self): + with TemporaryDirectory() as tmp: + output = Path(tmp).resolve() / "evidence" + real_write = __import__( + "skill2workflow._controlled_lark_pilot_evidence_writer", + fromlist=["_write_json_atomic"], + )._write_json_atomic + calls = [] + + def fail_second(parent_fd, name, value): + calls.append(name) + if len(calls) == 2: + raise OSError("second staged file failed") + return real_write(parent_fd, name, value) + + with patch( + "skill2workflow._controlled_lark_pilot_evidence_writer._write_json_atomic", + side_effect=fail_second, + ), self.assertRaisesRegex(OSError, "second staged file failed"): + write_evidence_pack(output, _valid_pack()) + + self.assertGreaterEqual(len(calls), 2) + self.assertFalse(output.exists()) + + def test_write_pack_second_file_failure_restores_existing_complete_pack(self): + with TemporaryDirectory() as tmp: + output = Path(tmp).resolve() / "evidence" + write_evidence_pack(output, _valid_pack()) + notes = output / "notes.txt" + notes.write_text("pre-call private note", encoding="utf-8") + hidden = output / ".sentinel" + hidden.write_text("pre-call hidden note", encoding="utf-8") + nested = output / "archive" / "notes.txt" + nested.parent.mkdir() + nested.write_text("pre-call nested note", encoding="utf-8") + before = { + str(path.relative_to(output)): path.read_bytes() + for path in sorted(output.rglob("*")) + if path.is_file() + } + replacement = deepcopy(_valid_pack()) + replacement["charter"]["expires_on"] = "2026-08-16" + real_write = __import__( + "skill2workflow._controlled_lark_pilot_evidence_writer", + fromlist=["_write_json_atomic"], + )._write_json_atomic + calls = [] + + def fail_second(parent_fd, name, value): + calls.append(name) + if len(calls) == 2: + raise OSError("second staged file failed") + return real_write(parent_fd, name, value) + + with patch( + "skill2workflow._controlled_lark_pilot_evidence_writer._write_json_atomic", + side_effect=fail_second, + ), self.assertRaisesRegex(OSError, "second staged file failed"): + write_evidence_pack(output, replacement) + + after = { + str(path.relative_to(output)): path.read_bytes() + for path in sorted(output.rglob("*")) + if path.is_file() + } + self.assertGreaterEqual(len(calls), 2) + self.assertEqual(after, before) + + def test_write_pack_omits_optional_none_and_removes_old_optional_json(self): + with TemporaryDirectory() as tmp: + output = Path(tmp).resolve() / "evidence" + write_evidence_pack(output, _valid_pack()) + pack = _valid_pack() + pack["exercises"] = { + "rejection": None, + "failure": None, + "rollback": None, + } + pack["runs"] = [] + pack["verification"] = None + pack["decision"] = None + pack["index"].update( + build_acceptance_summary( + pack["charter"], + pack["runs"], + 0, + pack["exercises"], + pack["verification"], + pack["decision"], + ) + ) + + result = write_evidence_pack(output, pack) + + self.assertEqual(result["file_count"], 2 + len(pack["runs"])) + self.assertFalse((output / "verification.json").exists()) + self.assertFalse((output / "decision.json").exists()) + self.assertEqual(list((output / "exercises").glob("*.json")), []) + + def test_write_pack_fails_if_declared_output_is_swapped_after_anchoring(self): + with TemporaryDirectory() as tmp: + root = Path(tmp).resolve() + output = root / "evidence" + output.mkdir() + anchored = root / "anchored-evidence" + outside = root / "outside" + outside.mkdir() + sentinel = outside / "sentinel.json" + sentinel.write_text("sentinel", encoding="utf-8") + real_open = os.open + swapped = [] + + def racing_open(path, flags, mode=0o777, *, dir_fd=None): + name = os.fspath(path) + if not swapped and name.endswith(".tmp"): + output.rename(anchored) + output.symlink_to(outside, target_is_directory=True) + swapped.append(True) + if dir_fd is None: + return real_open(path, flags, mode) + return real_open(path, flags, mode, dir_fd=dir_fd) + + with patch.object(os, "open", side_effect=racing_open), self.assertRaisesRegex( + ValueError, + "declared output|symbolic link", + ): + write_evidence_pack(output, _valid_pack()) + + self.assertEqual(swapped, [True]) + self.assertEqual(sentinel.read_text(encoding="utf-8"), "sentinel") + self.assertEqual(list(outside.glob("*.json")), [sentinel]) + self.assertEqual(list(anchored.iterdir()), []) + + def test_write_pack_fails_if_declared_output_becomes_missing_or_different(self): + for replacement in ("missing", "different-directory"): + with self.subTest(replacement=replacement), TemporaryDirectory() as tmp: + root = Path(tmp).resolve() + output = root / "evidence" + output.mkdir() + anchored = root / "anchored-evidence" + real_open = os.open + swapped = [] + + def racing_open(path, flags, mode=0o777, *, dir_fd=None): + name = os.fspath(path) + if not swapped and name.endswith(".tmp"): + output.rename(anchored) + if replacement == "different-directory": + output.mkdir() + swapped.append(True) + if dir_fd is None: + return real_open(path, flags, mode) + return real_open(path, flags, mode, dir_fd=dir_fd) + + with patch.object( + os, + "open", + side_effect=racing_open, + ), self.assertRaisesRegex(ValueError, "declared output"): + write_evidence_pack(output, _valid_pack()) + + self.assertEqual(swapped, [True]) + self.assertEqual(list(anchored.iterdir()), []) + if output.exists(): + self.assertEqual(list(output.iterdir()), []) + + def test_write_pack_parent_swap_after_publish_restores_anchored_old_pack(self): + with TemporaryDirectory() as tmp: + root = Path(tmp).resolve() + declared_parent = root / "declared" + output = declared_parent / "evidence" + write_evidence_pack(output, _valid_pack()) + before = { + str(path.relative_to(output)): path.read_bytes() + for path in sorted(output.rglob("*")) + if path.is_file() + } + replacement = deepcopy(_valid_pack()) + replacement["charter"]["expires_on"] = "2026-08-16" + anchored_parent = root / "anchored-declared" + real_replace = os.replace + swapped = [] + + def swap_parent_after_directory_publish( + source, + target, + *, + src_dir_fd=None, + dst_dir_fd=None, + ): + result = real_replace( + source, + target, + src_dir_fd=src_dir_fd, + dst_dir_fd=dst_dir_fd, + ) + if source == "stage" and target == "evidence" and not swapped: + declared_parent.rename(anchored_parent) + declared_parent.mkdir() + swapped.append(True) + return result + + with patch.object( + os, + "replace", + side_effect=swap_parent_after_directory_publish, + ), self.assertRaisesRegex(RuntimeError, "rollback"): + write_evidence_pack(output, replacement) + + restored_output = anchored_parent / "evidence" + after = { + str(path.relative_to(restored_output)): path.read_bytes() + for path in sorted(restored_output.rglob("*")) + if path.is_file() + } + self.assertEqual(swapped, [True]) + self.assertEqual(after, before) + self.assertEqual(list(declared_parent.iterdir()), []) + self.assertEqual( + [path for path in anchored_parent.iterdir() if path.name.endswith(".txn")], + [], + ) + + def test_write_pack_removes_crash_leftover_from_exact_new_pack(self): + with TemporaryDirectory() as tmp: + output = Path(tmp).resolve() / "evidence" + output.mkdir() + leftover = output / ".pilot-charter.json.tmp" + leftover.write_text("crash-leftover", encoding="utf-8") + + result = write_evidence_pack(output, _valid_pack()) + + self.assertEqual(result["status"], "written") + self.assertFalse(leftover.exists()) + self.assertTrue((output / "pilot-charter.json").is_file()) + + def test_write_pack_cleans_random_temp_when_atomic_replace_fails(self): + with TemporaryDirectory() as tmp: + output = Path(tmp).resolve() / "evidence" + + with patch( + "skill2workflow._controlled_lark_pilot_evidence_writer.os.replace", + side_effect=OSError("replace failed"), + ), self.assertRaisesRegex(OSError, "replace failed"): + write_evidence_pack(output, _valid_pack()) + + self.assertEqual( + [path for path in output.rglob("*") if path.name.endswith(".tmp")], + [], + ) + + def test_write_pack_fails_closed_without_secure_directory_fd_support(self): + with TemporaryDirectory() as tmp: + output = Path(tmp).resolve() / "evidence" + + with patch( + "skill2workflow._controlled_lark_pilot_evidence_writer._DIR_FD_SUPPORTED", + False, + ), self.assertRaisesRegex(ValueError, "directory-fd"): + write_evidence_pack(output, _valid_pack()) + + self.assertFalse(output.exists()) + + def test_write_pack_rejects_stale_json_symlink_without_touching_target(self): + with TemporaryDirectory() as tmp: + root = Path(tmp).resolve() + output = root / "evidence" + output.mkdir() + target = root / "outside.json" + target.write_text("sentinel", encoding="utf-8") + (output / "stale.json").symlink_to(target) + + with self.assertRaisesRegex(ValueError, "symbolic link"): + write_evidence_pack(output, _valid_pack()) + + self.assertEqual(target.read_text(encoding="utf-8"), "sentinel") + + def test_write_pack_rejects_root_and_symlink_components_without_touching_target(self): + with self.assertRaisesRegex(ValueError, "root"): + write_evidence_pack(Path("/"), _valid_pack()) + + with TemporaryDirectory() as tmp: + root = Path(tmp).resolve() + target = root / "target" + target.mkdir() + sentinel = target / "stale.json" + sentinel.write_text("sentinel", encoding="utf-8") + linked = root / "linked" + linked.symlink_to(target, target_is_directory=True) + + with self.assertRaisesRegex(ValueError, "symbolic link"): + write_evidence_pack(linked, _valid_pack()) + + self.assertEqual(sentinel.read_text(encoding="utf-8"), "sentinel") + + with TemporaryDirectory() as tmp: + root = Path(tmp).resolve() + output = root / "evidence" + output.mkdir() + target = root / "target" + target.mkdir() + sentinel = target / "sentinel.txt" + sentinel.write_text("sentinel", encoding="utf-8") + (output / "runs").symlink_to(target, target_is_directory=True) + + with self.assertRaisesRegex(ValueError, "symbolic link"): + write_evidence_pack(output, _valid_pack()) + + self.assertEqual(sentinel.read_text(encoding="utf-8"), "sentinel") + self.assertEqual(list(target.glob("*.json")), []) + + +if __name__ == "__main__": + import unittest + + unittest.main() diff --git a/tests/test_executor.py b/tests/test_executor.py index 7602a60..5137af6 100644 --- a/tests/test_executor.py +++ b/tests/test_executor.py @@ -256,6 +256,45 @@ def test_retry_policy_retries_failed_connector_and_records_recovery(self): self.assertIn("node_retrying", event_types) self.assertIn("node_recovered", event_types) + def test_connector_receives_ephemeral_execution_identity_without_persisting_it(self): + runtime = _CapturingConnectorRuntime() + original_context = { + "input": {"title": "Durable title"}, + "_execution": {"workflow_id": "forged"}, + } + + with TemporaryDirectory() as tmp: + executor = LocalExecutor(Path(tmp), connector_runtime=runtime) + state = executor.run(_http_connector_workflow("https://unused.invalid"), context=original_context) + persisted = executor.get_run(state["run_id"]) + + self.assertEqual(len(runtime.contexts), 1) + self.assertEqual( + runtime.contexts[0]["_execution"], + { + "workflow_id": "workflow_connector", + "workflow_version": "0.1.0", + "run_id": state["run_id"], + "node_id": "call_api", + }, + ) + self.assertEqual(runtime.contexts[0]["input"], {"title": "Durable title"}) + self.assertEqual(state["context"], original_context) + self.assertEqual(persisted["context"], original_context) + + +class _CapturingConnectorRuntime: + def __init__(self): + self.contexts = [] + + def execute_connector(self, node, credential_provider=None, context=None): + self.contexts.append(context) + return { + "status": "completed", + "connector": {"id": "http", "kind": "http"}, + "output": {}, + } + def _approval_workflow(): return { diff --git a/tests/test_first_product_connector_candidate_docs.py b/tests/test_first_product_connector_candidate_docs.py index 867f64c..097ebc8 100644 --- a/tests/test_first_product_connector_candidate_docs.py +++ b/tests/test_first_product_connector_candidate_docs.py @@ -47,7 +47,8 @@ def test_loop_35_lark_task_candidate_decision_is_documented(self): self.assertIn("| Loop 36: First Product Connector Package Smoke | Complete |", roadmap) self.assertIn("| Loop 37: Product Connector Pilot Scenario | Complete |", roadmap) self.assertIn("| Loop 38: Live Connector Readiness Review | Complete |", roadmap) - self.assertIn("| Loop 39: Scoped Live Lark Task Connector | Next |", roadmap) + self.assertIn("| Loop 39: Scoped Live Lark Task Connector | Complete |", roadmap) + self.assertIn("| Loop 40: Controlled Live Connector Pilot | Deferred |", roadmap) self.assertIn("Lark/Feishu task connector", roadmap) self.assertIn("package-level and pilot-workflow dry-run evidence", roadmap) diff --git a/tests/test_lark_live_connector_validation_docs.py b/tests/test_lark_live_connector_validation_docs.py new file mode 100644 index 0000000..fcd5af9 --- /dev/null +++ b/tests/test_lark_live_connector_validation_docs.py @@ -0,0 +1,35 @@ +from pathlib import Path +from unittest import TestCase + + +ROOT = Path(__file__).resolve().parents[1] + + +class LarkLiveConnectorValidationDocsTests(TestCase): + def test_live_validation_evidence_is_compact_and_redacted(self): + evidence = (ROOT / "docs" / "lark-live-connector-validation.md").read_text(encoding="utf-8") + + self.assertIn("# Lark/Feishu Live Connector Validation", evidence) + self.assertIn("- connector_id: `lark_task`", evidence) + self.assertIn("- operation: `create_task`", evidence) + self.assertIn("- mode: `live`", evidence) + self.assertIn("- credential_status: `resolved`", evidence) + self.assertIn("- idempotency_key_present: `true`", evidence) + self.assertIn("- provider_status: `completed`", evidence) + self.assertIn("- lark_task_id_present: `true`", evidence) + self.assertIn("- assignee_present: `true`", evidence) + self.assertIn( + "Raw task values, user ids, credentials, request bodies, response bodies, and task ids are intentionally omitted.", + evidence, + ) + self.assertIn( + "The user explicitly approved a one-time standard-tier Vault fallback after protected-tier " + "passkey/WebAuthn provisioning failed.", + evidence, + ) + self.assertIn( + "The credential had `open.feishu.cn` allowed-host metadata, was short-lived, and was deleted " + "immediately after success.", + evidence, + ) + self.assertNotIn("constrained to the fixed connector endpoint", evidence) diff --git a/tests/test_lark_task_connector.py b/tests/test_lark_task_connector.py index ed95b9e..00c5584 100644 --- a/tests/test_lark_task_connector.py +++ b/tests/test_lark_task_connector.py @@ -1,8 +1,20 @@ +import io import json +import os +import socket +import uuid +from datetime import datetime from pathlib import Path from unittest import TestCase +from unittest.mock import patch +from urllib import error as urllib_error -from skill2workflow.connectors import ConnectorExecutionError, ConnectorRuntime, validate_connector_manifest +from skill2workflow.connectors import ( + ConnectorExecutionError, + ConnectorRuntime, + ExternalConnector, + validate_connector_manifest, +) from skill2workflow.credentials import StaticCredentialProvider from skill2workflow.external_connectors import load_external_connector @@ -27,6 +39,11 @@ def test_lark_task_manifest_is_explicit_external_connector(self): [manifest["id"] for manifest in ConnectorRuntime([connector]).list_connectors()], ["manual", "http", "lark_task"], ) + self.assertEqual( + connector.manifest["config_schema"]["properties"]["mode"]["enum"], + ["dry_run", "live"], + ) + self.assertIn("dry-run-default", connector.manifest["description"]) def test_lark_task_dry_run_returns_compact_metadata_without_payload_values(self): runtime = ConnectorRuntime([_load_lark_task_connector()]) @@ -76,15 +93,776 @@ def test_lark_task_dry_run_returns_compact_metadata_without_payload_values(self) def test_lark_task_rejects_live_mode_and_missing_credentials(self): runtime = ConnectorRuntime([_load_lark_task_connector()]) - with self.assertRaisesRegex(ConnectorExecutionError, "lark_task connector only supports mode dry_run"): - runtime.execute_connector(_lark_task_node(mode="live")) - with self.assertRaisesRegex(ConnectorExecutionError, "credential handle not found: lark_bot_access_token"): runtime.execute_connector(_lark_task_node(), context={"input": {"title": "Task"}}) + def test_lark_task_live_mode_requires_exact_environment_opt_in(self): + transport = _FakeTransport() + runtime = ConnectorRuntime([_load_lark_task_connector(transport)]) + + for value in (None, "", "true", "yes", "0"): + environment = {} if value is None else {"SKILL2WORKFLOW_LARK_TASK_LIVE": value} + with self.subTest(value=value), patch.dict(os.environ, environment, clear=True): + result = runtime.execute_connector( + _lark_task_node(mode="live"), + credential_provider=_FailIfResolvedCredentialProvider(), + context=_execution_context(), + ) + + self.assertEqual(result["status"], "failed") + self.assertEqual(result["audit"]["provider_status"], "live_disabled") + + self.assertEqual(transport.calls, []) + + def test_lark_task_preflight_constructs_the_live_payload_without_credentials_or_transport(self): + connector = _load_lark_task_connector() + preflight = connector.preflight + + result = preflight( + _lark_task_node(mode="live")["connector"], + context=_execution_context(), + ) + + self.assertEqual(result["status"], "ready") + self.assertEqual(result["connector"], {"id": "lark_task", "kind": "lark_task"}) + self.assertEqual( + result["output"], + { + "operation": "create_task", + "mode": "live", + "provider_payload_constructed": True, + "credential_resolution_attempted": False, + "network_called": False, + }, + ) + self.assertEqual( + result["audit"], + { + "operation": "create_task", + "mode": "live", + "task_title_present": True, + "task_description_present": True, + "assignee_present": True, + "due_at_present": True, + }, + ) + + encoded = json.dumps(result, ensure_ascii=False) + for forbidden in ( + "Renewal risk follow-up", + "Customer ACME needs executive review", + "ou_123456", + "2026-07-09T09:00:00Z", + ): + self.assertNotIn(forbidden, encoded) + + def test_lark_task_preflight_rejects_provider_invalid_text_without_leaking_it(self): + connector = _load_lark_task_connector() + for field in ("title", "description"): + with self.subTest(field=field): + context = _execution_context() + invalid_text = "private-text-" + ("x" * 3000) + context["input"][field] = invalid_text + + result = connector.preflight( + _lark_task_node(mode="live")["connector"], + context=context, + ) + + self.assertEqual(result["status"], "invalid") + self.assertFalse(result["output"]["provider_payload_constructed"]) + self.assertFalse(result["output"]["credential_resolution_attempted"]) + self.assertFalse(result["output"]["network_called"]) + self.assertNotIn(invalid_text, json.dumps(result, ensure_ascii=False)) + + def test_lark_task_live_mode_sends_fixed_redacted_idempotent_request(self): + transport = _FakeTransport() + runtime = ConnectorRuntime([_load_lark_task_connector(transport)]) + + with patch.dict(os.environ, {"SKILL2WORKFLOW_LARK_TASK_LIVE": "1"}, clear=True): + result = runtime.execute_connector( + _lark_task_node(mode="live"), + credential_provider=StaticCredentialProvider( + {"lark_bot_access_token": "local-lark-secret"} + ), + context=_execution_context(), + ) -def _load_lark_task_connector(): - return load_external_connector(ROOT / "examples" / "connectors" / "lark_task_connector.py") + self.assertEqual(result["status"], "completed") + self.assertEqual(len(transport.calls), 1) + call = transport.calls[0] + request = call["request"] + request_body = json.loads(request.data.decode("utf-8")) + expected_due = str(int(datetime.fromisoformat("2026-07-09T09:00:00+00:00").timestamp() * 1000)) + + self.assertEqual(request.method, "POST") + self.assertEqual( + request.full_url, + "https://open.feishu.cn/open-apis/task/v2/tasks?user_id_type=open_id", + ) + self.assertEqual(request.get_header("Authorization"), "Bearer local-lark-secret") + self.assertEqual(request.get_header("Content-type"), "application/json; charset=utf-8") + self.assertEqual(request_body["summary"], "Renewal risk follow-up") + self.assertEqual(request_body["description"], "Customer ACME needs executive review") + self.assertEqual( + request_body["members"], + [{"id": "ou_123456", "type": "user", "role": "assignee"}], + ) + self.assertEqual(request_body["due"], {"timestamp": expected_due, "is_all_day": False}) + self.assertEqual(len(request_body["client_token"]), 36) + self.assertEqual(str(uuid.UUID(request_body["client_token"])), request_body["client_token"]) + self.assertNotIn("source", request_body) + self.assertEqual(call["timeout"], 10.0) + self.assertEqual(result["audit"]["provider_status"], "completed") + self.assertTrue(result["audit"]["idempotency_key_present"]) + self.assertTrue(result["audit"]["lark_task_id_present"]) + + encoded = json.dumps(result, ensure_ascii=False) + for forbidden in ( + "local-lark-secret", + "Renewal risk follow-up", + "Customer ACME needs executive review", + "ou_123456", + "2026-07-09T09:00:00Z", + "task-guid-must-not-leak", + request_body["client_token"], + ): + self.assertNotIn(forbidden, encoded) + + def test_lark_task_live_mode_resolves_only_approved_credential_handle(self): + transport = _FakeTransport() + runtime = ConnectorRuntime([_load_lark_task_connector(transport)]) + node = _lark_task_node(mode="live") + node["connector"]["credentials"].append( + { + "target": "header", + "name": "X-Unrelated-Secret", + "handle": "unrelated_header_secret", + } + ) + provider = _RecordingCredentialProvider( + { + "lark_bot_access_token": "local-lark-secret", + "unrelated_header_secret": "must-not-be-materialized", + } + ) + + with patch.dict(os.environ, {"SKILL2WORKFLOW_LARK_TASK_LIVE": "1"}, clear=True): + result = runtime.execute_connector( + node, + credential_provider=provider, + context=_execution_context(), + ) + + self.assertEqual(result["status"], "completed") + self.assertEqual(provider.calls, ["lark_bot_access_token"]) + self.assertEqual( + result["credentials"], + {"status": "resolved", "handles": ["lark_bot_access_token"]}, + ) + self.assertNotIn("must-not-be-materialized", json.dumps(result)) + + def test_lark_task_live_mode_normalizes_provider_failures_without_leakage(self): + cases = [ + (401, {"code": 999, "msg": "raw-auth-detail"}, "authorization_failed"), + (400, {"code": 99991663, "msg": "raw-token-detail"}, "authorization_failed"), + (403, {"code": 1470403, "msg": "raw-permission-detail"}, "permission_denied"), + (429, {"code": 999, "msg": "raw-rate-detail"}, "rate_limited"), + (400, {"code": 1470400, "msg": "raw-validation-detail"}, "validation_failed"), + (404, {"code": 1470404, "msg": "raw-resource-detail"}, "resource_not_found"), + (500, {"code": 1470422, "msg": "raw-idempotency-detail"}, "idempotency_conflict"), + (500, {"code": 1470500, "msg": "raw-provider-detail"}, "provider_unavailable"), + (500, b"not-json-provider-body", "provider_unavailable"), + ] + + for status, payload, expected_status in cases: + with self.subTest(status=status, expected_status=expected_status): + transport = _FakeTransport(status=status, payload=payload) + runtime = ConnectorRuntime([_load_lark_task_connector(transport)]) + with patch.dict(os.environ, {"SKILL2WORKFLOW_LARK_TASK_LIVE": "1"}, clear=True): + result = runtime.execute_connector( + _lark_task_node(mode="live"), + credential_provider=StaticCredentialProvider( + {"lark_bot_access_token": "local-lark-secret"} + ), + context=_execution_context(), + ) + + self.assertEqual(result["status"], "failed") + self.assertEqual(result["audit"]["provider_status"], expected_status) + self.assertTrue(result["audit"]["idempotency_key_present"]) + self.assertFalse(result["audit"]["lark_task_id_present"]) + self.assertEqual( + result["error"], + f"lark_task live request failed: {expected_status}", + ) + self.assertEqual(result["output"], result["audit"]) + encoded = json.dumps(result, ensure_ascii=False) + self.assertNotIn("local-lark-secret", encoded) + self.assertNotIn("raw-", encoded) + self.assertNotIn("not-json-provider-body", encoded) + + def test_lark_task_live_mode_normalizes_timeout_and_malformed_success(self): + cases = [ + (_FakeTransport(error=TimeoutError("raw timeout body")), "timeout"), + (_FakeTransport(error=urllib_error.URLError(socket.timeout("raw socket timeout"))), "timeout"), + (_FakeTransport(error=urllib_error.URLError("raw network failure")), "provider_unavailable"), + ( + _FakeTransport( + error=urllib_error.HTTPError( + "https://open.feishu.cn/open-apis/task/v2/tasks", + 403, + "raw http reason", + {}, + io.BytesIO(b'{"code":1470403,"msg":"raw-http-body"}'), + ) + ), + "permission_denied", + ), + (_FakeTransport(payload=b"not-json-success-body"), "malformed_response"), + (_FakeTransport(payload={"code": 0, "data": {}}), "malformed_response"), + (_FakeTransport(payload={"code": 0, "data": {"task": {"guid": ""}}}), "malformed_response"), + ] + + for transport, expected_status in cases: + with self.subTest(expected_status=expected_status): + runtime = ConnectorRuntime([_load_lark_task_connector(transport)]) + with patch.dict(os.environ, {"SKILL2WORKFLOW_LARK_TASK_LIVE": "1"}, clear=True): + result = runtime.execute_connector( + _lark_task_node(mode="live"), + credential_provider=StaticCredentialProvider( + {"lark_bot_access_token": "local-lark-secret"} + ), + context=_execution_context(), + ) + + self.assertEqual(result["status"], "failed") + self.assertEqual(result["audit"]["provider_status"], expected_status) + self.assertTrue(result["audit"]["idempotency_key_present"]) + self.assertFalse(result["audit"]["lark_task_id_present"]) + self.assertEqual( + result["error"], + f"lark_task live request failed: {expected_status}", + ) + encoded = json.dumps(result, ensure_ascii=False) + self.assertNotIn("local-lark-secret", encoded) + self.assertNotIn("raw timeout body", encoded) + self.assertNotIn("raw socket timeout", encoded) + self.assertNotIn("raw network failure", encoded) + self.assertNotIn("raw http reason", encoded) + self.assertNotIn("raw-http-body", encoded) + self.assertNotIn("not-json-success-body", encoded) + + def test_lark_task_live_mode_normalizes_response_read_failures_and_closes(self): + timeout_response = _FakeResponse( + 200, + {}, + error=TimeoutError("raw response read timeout"), + ) + http_error_body = _ReadErrorBody( + urllib_error.URLError(socket.timeout("raw http error read timeout")) + ) + http_error = urllib_error.HTTPError( + "https://open.feishu.cn/open-apis/task/v2/tasks", + 503, + "raw http error reason", + {}, + http_error_body, + ) + url_error_response = _FakeResponse( + 200, + {}, + error=urllib_error.URLError("raw response read network failure"), + ) + cases = [ + (_FakeTransport(response=timeout_response), timeout_response, "timeout"), + (_FakeTransport(error=http_error), http_error_body, "timeout"), + ( + _FakeTransport(response=url_error_response), + url_error_response, + "provider_unavailable", + ), + ] + + for transport, close_target, expected_status in cases: + with self.subTest(expected_status=expected_status): + runtime = ConnectorRuntime([_load_lark_task_connector(transport)]) + with patch.dict(os.environ, {"SKILL2WORKFLOW_LARK_TASK_LIVE": "1"}, clear=True): + result = runtime.execute_connector( + _lark_task_node(mode="live"), + credential_provider=StaticCredentialProvider( + {"lark_bot_access_token": "local-lark-secret"} + ), + context=_execution_context(), + ) + + self.assertEqual(result["status"], "failed") + self.assertEqual(result["audit"]["provider_status"], expected_status) + self.assertTrue(result["audit"]["idempotency_key_present"]) + self.assertFalse(result["audit"]["lark_task_id_present"]) + self.assertEqual( + result["error"], + f"lark_task live request failed: {expected_status}", + ) + self.assertTrue(close_target.closed) + encoded = json.dumps(result, ensure_ascii=False) + self.assertNotIn("local-lark-secret", encoded) + self.assertNotIn("raw response read timeout", encoded) + self.assertNotIn("raw http error read timeout", encoded) + self.assertNotIn("raw http error reason", encoded) + self.assertNotIn("raw response read network failure", encoded) + + def test_lark_task_live_mode_normalizes_ordinary_transport_failures_without_leakage(self): + token = "dummy-transport-token" + + def transport(request, timeout): + raise ValueError(f"raw transport failure: {request.get_header('Authorization')}") + + runtime = ConnectorRuntime([_load_lark_task_connector(transport)]) + with patch.dict(os.environ, {"SKILL2WORKFLOW_LARK_TASK_LIVE": "1"}, clear=True): + result = runtime.execute_connector( + _lark_task_node(mode="live"), + credential_provider=StaticCredentialProvider({"lark_bot_access_token": token}), + context=_execution_context(), + ) + + self.assertEqual(result["status"], "failed") + self.assertEqual(result["audit"]["provider_status"], "provider_unavailable") + self.assertEqual(result["error"], "lark_task live request failed: provider_unavailable") + encoded = json.dumps(result, ensure_ascii=False) + self.assertNotIn(token, encoded) + self.assertNotIn("raw transport failure", encoded) + + def test_lark_task_live_mode_normalizes_ordinary_response_read_failures_and_closes(self): + token = "dummy-read-token" + response = _FakeResponse(200, {}, error=ValueError(f"raw response read failure: {token}")) + runtime = ConnectorRuntime([_load_lark_task_connector(_FakeTransport(response=response))]) + + with patch.dict(os.environ, {"SKILL2WORKFLOW_LARK_TASK_LIVE": "1"}, clear=True): + result = runtime.execute_connector( + _lark_task_node(mode="live"), + credential_provider=StaticCredentialProvider({"lark_bot_access_token": token}), + context=_execution_context(), + ) + + self.assertEqual(result["status"], "failed") + self.assertEqual(result["audit"]["provider_status"], "provider_unavailable") + self.assertEqual(result["error"], "lark_task live request failed: provider_unavailable") + self.assertTrue(response.closed) + encoded = json.dumps(result, ensure_ascii=False) + self.assertNotIn(token, encoded) + self.assertNotIn("raw response read failure", encoded) + + def test_lark_task_live_mode_normalizes_ordinary_status_failures_and_closes(self): + token = "dummy-status-token" + response = _StatusErrorResponse(ValueError(f"raw status failure: {token}")) + runtime = ConnectorRuntime([_load_lark_task_connector(_FakeTransport(response=response))]) + + with patch.dict(os.environ, {"SKILL2WORKFLOW_LARK_TASK_LIVE": "1"}, clear=True): + result = runtime.execute_connector( + _lark_task_node(mode="live"), + credential_provider=StaticCredentialProvider({"lark_bot_access_token": token}), + context=_execution_context(), + ) + + self.assertEqual(result["status"], "failed") + self.assertEqual(result["audit"]["provider_status"], "malformed_response") + self.assertEqual(result["error"], "lark_task live request failed: malformed_response") + self.assertTrue(response.closed) + encoded = json.dumps(result, ensure_ascii=False) + self.assertNotIn(token, encoded) + self.assertNotIn("raw status failure", encoded) + + def test_lark_task_live_mode_normalizes_request_construction_failures_without_leakage(self): + token = "dummy-construction-token" + connector = _load_lark_task_connector() + request_module = connector.executor.__globals__["urllib_request"] + runtime = ConnectorRuntime([connector]) + + with patch.object( + request_module, + "Request", + side_effect=ValueError(f"raw request construction failure: {token}"), + ), patch.dict(os.environ, {"SKILL2WORKFLOW_LARK_TASK_LIVE": "1"}, clear=True): + result = runtime.execute_connector( + _lark_task_node(mode="live"), + credential_provider=StaticCredentialProvider({"lark_bot_access_token": token}), + context=_execution_context(), + ) + + self.assertEqual(result["status"], "failed") + self.assertEqual(result["audit"]["provider_status"], "credential_failed") + self.assertEqual(result["error"], "lark_task live request failed: credential_failed") + encoded = json.dumps(result, ensure_ascii=False) + self.assertNotIn(token, encoded) + self.assertNotIn("raw request construction failure", encoded) + + def test_lark_task_live_mode_normalizes_read_and_close_failures_without_leakage(self): + token = "dummy-read-close-token" + response = _CloseErrorResponse( + read_error=ValueError(f"raw response read failure: {token}"), + close_error=ValueError(f"raw close failure: {token}"), + ) + runtime = ConnectorRuntime([_load_lark_task_connector(_FakeTransport(response=response))]) + + with patch.dict(os.environ, {"SKILL2WORKFLOW_LARK_TASK_LIVE": "1"}, clear=True): + result = runtime.execute_connector( + _lark_task_node(mode="live"), + credential_provider=StaticCredentialProvider({"lark_bot_access_token": token}), + context=_execution_context(), + ) + + self.assertEqual(result["status"], "failed") + self.assertEqual(result["audit"]["provider_status"], "provider_unavailable") + self.assertTrue(response.close_attempted) + encoded = json.dumps(result, ensure_ascii=False) + self.assertNotIn(token, encoded) + self.assertNotIn("raw response read failure", encoded) + self.assertNotIn("raw close failure", encoded) + + def test_lark_task_live_mode_normalizes_status_and_close_failures_without_leakage(self): + token = "dummy-status-close-token" + response = _CloseErrorResponse( + status_error=ValueError(f"raw status failure: {token}"), + close_error=ValueError(f"raw close failure: {token}"), + ) + runtime = ConnectorRuntime([_load_lark_task_connector(_FakeTransport(response=response))]) + + with patch.dict(os.environ, {"SKILL2WORKFLOW_LARK_TASK_LIVE": "1"}, clear=True): + result = runtime.execute_connector( + _lark_task_node(mode="live"), + credential_provider=StaticCredentialProvider({"lark_bot_access_token": token}), + context=_execution_context(), + ) + + self.assertEqual(result["status"], "failed") + self.assertEqual(result["audit"]["provider_status"], "malformed_response") + self.assertTrue(response.close_attempted) + encoded = json.dumps(result, ensure_ascii=False) + self.assertNotIn(token, encoded) + self.assertNotIn("raw status failure", encoded) + self.assertNotIn("raw close failure", encoded) + + def test_lark_task_live_mode_normalizes_http_error_code_and_close_failures_without_leakage(self): + token = "dummy-http-close-token" + body = _CloseErrorBody(ValueError(f"raw close failure: {token}")) + error = urllib_error.HTTPError( + "https://open.feishu.cn/open-apis/task/v2/tasks", + _IntErrorCode(ValueError(f"raw http status failure: {token}")), + "raw http error reason", + {}, + body, + ) + runtime = ConnectorRuntime([_load_lark_task_connector(_FakeTransport(error=error))]) + + with patch.dict(os.environ, {"SKILL2WORKFLOW_LARK_TASK_LIVE": "1"}, clear=True): + result = runtime.execute_connector( + _lark_task_node(mode="live"), + credential_provider=StaticCredentialProvider({"lark_bot_access_token": token}), + context=_execution_context(), + ) + + self.assertEqual(result["status"], "failed") + self.assertEqual(result["audit"]["provider_status"], "malformed_response") + self.assertTrue(body.close_attempted) + encoded = json.dumps(result, ensure_ascii=False) + self.assertNotIn(token, encoded) + self.assertNotIn("raw http status failure", encoded) + self.assertNotIn("raw close failure", encoded) + + def test_lark_task_live_mode_requires_2xx_for_provider_success(self): + cases = [ + (401, "authorization_failed"), + (403, "permission_denied"), + (429, "rate_limited"), + (500, "provider_unavailable"), + ] + + for status, expected_status in cases: + with self.subTest(status=status, expected_status=expected_status): + transport = _FakeTransport( + status=status, + payload={ + "code": 0, + "msg": "raw false success detail", + "data": {"task": {"guid": "raw-false-success-guid"}}, + }, + ) + runtime = ConnectorRuntime([_load_lark_task_connector(transport)]) + with patch.dict(os.environ, {"SKILL2WORKFLOW_LARK_TASK_LIVE": "1"}, clear=True): + result = runtime.execute_connector( + _lark_task_node(mode="live"), + credential_provider=StaticCredentialProvider( + {"lark_bot_access_token": "local-lark-secret"} + ), + context=_execution_context(), + ) + + self.assertEqual(result["status"], "failed") + self.assertEqual(result["audit"]["provider_status"], expected_status) + self.assertTrue(result["audit"]["idempotency_key_present"]) + self.assertFalse(result["audit"]["lark_task_id_present"]) + encoded = json.dumps(result, ensure_ascii=False) + self.assertNotIn("raw false success detail", encoded) + self.assertNotIn("raw-false-success-guid", encoded) + + def test_lark_task_live_preflight_failures_never_call_transport(self): + missing_execution = {"input": dict(_execution_context()["input"])} + invalid_due = json.loads(json.dumps(_execution_context())) + invalid_due["input"]["due_at"] = "2026-07-09T09:00:00" + cases = [ + ( + missing_execution, + StaticCredentialProvider({"lark_bot_access_token": "secret"}), + "validation_failed", + False, + ), + ( + invalid_due, + StaticCredentialProvider({"lark_bot_access_token": "secret"}), + "validation_failed", + False, + ), + (_execution_context(), StaticCredentialProvider({}), "credential_failed", True), + ] + + for context, provider, expected_status, idempotency_key_present in cases: + with self.subTest(expected_status=expected_status): + transport = _FakeTransport() + runtime = ConnectorRuntime([_load_lark_task_connector(transport)]) + with patch.dict(os.environ, {"SKILL2WORKFLOW_LARK_TASK_LIVE": "1"}, clear=True): + result = runtime.execute_connector( + _lark_task_node(mode="live"), + credential_provider=provider, + context=context, + ) + + self.assertEqual(result["status"], "failed") + self.assertEqual(result["audit"]["provider_status"], expected_status) + self.assertEqual( + result["audit"]["idempotency_key_present"], + idempotency_key_present, + ) + self.assertFalse(result["audit"]["lark_task_id_present"]) + self.assertEqual( + result["error"], + f"lark_task live request failed: {expected_status}", + ) + self.assertEqual(transport.calls, []) + encoded = json.dumps(result, ensure_ascii=False) + self.assertNotIn("secret", encoded) + + def test_lark_task_client_token_is_stable_per_execution_identity(self): + transport = _FakeTransport() + runtime = ConnectorRuntime([_load_lark_task_connector(transport)]) + + with patch.dict(os.environ, {"SKILL2WORKFLOW_LARK_TASK_LIVE": "1"}, clear=True): + runtime.execute_connector( + _lark_task_node(mode="live"), + credential_provider=StaticCredentialProvider( + {"lark_bot_access_token": "local-lark-secret"} + ), + context=_execution_context(), + ) + runtime.execute_connector( + _lark_task_node(mode="live"), + credential_provider=StaticCredentialProvider( + {"lark_bot_access_token": "local-lark-secret"} + ), + context=_execution_context(), + ) + runtime.execute_connector( + _lark_task_node(mode="live"), + credential_provider=StaticCredentialProvider( + {"lark_bot_access_token": "local-lark-secret"} + ), + context=_execution_context(run_id="run_other"), + ) + runtime.execute_connector( + _lark_task_node(mode="live"), + credential_provider=StaticCredentialProvider( + {"lark_bot_access_token": "local-lark-secret"} + ), + context=_execution_context(node_id="create_other_lark_task"), + ) + + tokens = [json.loads(call["request"].data.decode("utf-8"))["client_token"] for call in transport.calls] + self.assertEqual(tokens[0], tokens[1]) + self.assertNotEqual(tokens[0], tokens[2]) + self.assertNotEqual(tokens[0], tokens[3]) + for token in tokens: + self.assertEqual(str(uuid.UUID(token)), token) + + def test_lark_task_missing_mode_remains_dry_run(self): + node = _lark_task_node() + del node["connector"]["mode"] + runtime = ConnectorRuntime([_load_lark_task_connector()]) + + result = runtime.execute_connector( + node, + credential_provider=StaticCredentialProvider( + {"lark_bot_access_token": "local-lark-secret"} + ), + context={"input": {"title": "Task"}}, + ) + + self.assertEqual(result["status"], "completed") + self.assertEqual(result["audit"]["mode"], "dry_run") + + +def _load_lark_task_connector(transport=None): + connector = load_external_connector(ROOT / "examples" / "connectors" / "lark_task_connector.py") + if transport is None: + return connector + + def execute_with_transport(binding, credential_provider=None, context=None): + return connector.executor( + binding, + credential_provider=credential_provider, + context=context, + transport=transport, + ) + + return ExternalConnector(manifest=connector.manifest, executor=execute_with_transport) + + +class _FakeResponse: + def __init__(self, status, payload, error=None): + self.status = status + self._payload = payload + self.error = error + self.closed = False + + def read(self): + if self.error is not None: + raise self.error + if isinstance(self._payload, bytes): + return self._payload + return json.dumps(self._payload).encode("utf-8") + + def close(self): + self.closed = True + + +class _ReadErrorBody: + def __init__(self, error): + self.error = error + self.closed = False + + def read(self): + raise self.error + + def close(self): + self.closed = True + + +class _StatusErrorResponse: + def __init__(self, error): + self.error = error + self.closed = False + + @property + def status(self): + raise self.error + + def close(self): + self.closed = True + + +class _CloseErrorResponse: + def __init__(self, status=200, read_error=None, status_error=None, close_error=None): + self._status = status + self.read_error = read_error + self.status_error = status_error + self.close_error = close_error + self.close_attempted = False + + @property + def status(self): + if self.status_error is not None: + raise self.status_error + return self._status + + def read(self): + if self.read_error is not None: + raise self.read_error + return b"{}" + + def close(self): + self.close_attempted = True + if self.close_error is not None: + raise self.close_error + + +class _CloseErrorBody: + def __init__(self, close_error): + self.close_error = close_error + self.close_attempted = False + + def close(self): + self.close_attempted = True + raise self.close_error + + +class _IntErrorCode: + def __init__(self, error): + self.error = error + + def __int__(self): + raise self.error + + +class _FakeTransport: + def __init__(self, status=200, payload=None, error=None, response=None): + self.status = status + self.payload = payload if payload is not None else { + "code": 0, + "msg": "success", + "data": {"task": {"guid": "task-guid-must-not-leak"}}, + } + self.error = error + self.response = response + self.calls = [] + + def __call__(self, request, timeout): + self.calls.append({"request": request, "timeout": timeout}) + if self.error is not None: + raise self.error + if self.response is not None: + return self.response + return _FakeResponse(self.status, self.payload) + + +class _FailIfResolvedCredentialProvider: + def resolve(self, handle): + raise AssertionError(f"credential resolution must not run: {handle}") + + +class _RecordingCredentialProvider: + def __init__(self, values): + self.values = values + self.calls = [] + + def resolve(self, handle): + self.calls.append(handle) + return self.values[handle] + + +def _execution_context(run_id="run_live", node_id="create_lark_task"): + return { + "input": { + "title": "Renewal risk follow-up", + "description": "Customer ACME needs executive review", + "assignee_open_id": "ou_123456", + "due_at": "2026-07-09T09:00:00Z", + }, + "_execution": { + "workflow_id": "workflow_lark_live", + "workflow_version": "0.1.0", + "run_id": run_id, + "node_id": node_id, + }, + } def _lark_task_node(operation="create_task", mode="dry_run"): diff --git a/tests/test_lark_task_live_validation.py b/tests/test_lark_task_live_validation.py new file mode 100644 index 0000000..d000489 --- /dev/null +++ b/tests/test_lark_task_live_validation.py @@ -0,0 +1,96 @@ +import json +import os +from pathlib import Path +from unittest import TestCase +from unittest.mock import patch + +from skill2workflow.lark_task_live_validation import run_lark_task_live_validation + + +ROOT = Path(__file__).resolve().parents[1] + + +class LarkTaskLiveValidationTests(TestCase): + def test_live_validation_requires_confirmation_switch_token_and_identity(self): + cases = [ + ({}, False, "run_validation", "ou_test", "live validation requires --confirm-live-create"), + ({"SKILL2WORKFLOW_LARK_TASK_LIVE": "1"}, True, "run_validation", "ou_test", "LARK_BOT_ACCESS_TOKEN is required"), + ({"LARK_BOT_ACCESS_TOKEN": "secret"}, True, "run_validation", "ou_test", "SKILL2WORKFLOW_LARK_TASK_LIVE=1 is required"), + ({"SKILL2WORKFLOW_LARK_TASK_LIVE": "1", "LARK_BOT_ACCESS_TOKEN": "secret"}, True, "", "ou_test", "validation run id is required"), + ({"SKILL2WORKFLOW_LARK_TASK_LIVE": "1", "LARK_BOT_ACCESS_TOKEN": "secret"}, True, "run_validation", "", "assignee open id is required"), + ] + + for environment, confirmed, run_id, assignee, expected in cases: + with self.subTest(expected=expected), patch.dict(os.environ, environment, clear=True): + with self.assertRaisesRegex(ValueError, expected): + run_lark_task_live_validation( + ROOT, + title="Validation title", + description="Validation description", + assignee_open_id=assignee, + validation_run_id=run_id, + confirmed=confirmed, + transport=lambda request, timeout: None, + ) + + def test_live_validation_returns_only_compact_metadata(self): + transport = _FakeTransport() + environment = { + "SKILL2WORKFLOW_LARK_TASK_LIVE": "1", + "LARK_BOT_ACCESS_TOKEN": "live-validation-secret", + } + with patch.dict(os.environ, environment, clear=True): + result = run_lark_task_live_validation( + ROOT, + title="Validation title", + description="Validation description", + assignee_open_id="ou_validation", + validation_run_id="run_validation", + confirmed=True, + transport=transport, + ) + + self.assertEqual( + result, + { + "ok": True, + "connector_id": "lark_task", + "operation": "create_task", + "mode": "live", + "credential_status": "resolved", + "idempotency_key_present": True, + "provider_status": "completed", + "lark_task_id_present": True, + "assignee_present": True, + }, + ) + encoded = json.dumps(result) + for forbidden in ( + "live-validation-secret", + "Validation title", + "Validation description", + "ou_validation", + "task-guid-must-not-leak", + ): + self.assertNotIn(forbidden, encoded) + + +class _FakeResponse: + status = 200 + + def read(self): + return json.dumps( + {"code": 0, "msg": "success", "data": {"task": {"guid": "task-guid-must-not-leak"}}} + ).encode("utf-8") + + def close(self): + return None + + +class _FakeTransport: + def __init__(self): + self.calls = [] + + def __call__(self, request, timeout): + self.calls.append((request, timeout)) + return _FakeResponse() diff --git a/tests/test_lark_task_pilot.py b/tests/test_lark_task_pilot.py index aa99412..e4fc265 100644 --- a/tests/test_lark_task_pilot.py +++ b/tests/test_lark_task_pilot.py @@ -3,10 +3,48 @@ from tempfile import TemporaryDirectory from unittest import TestCase -from skill2workflow.lark_task_pilot import run_lark_task_pilot +from skill2workflow.lark_task_pilot import ( + build_lark_task_pilot_workflow, + run_lark_task_pilot, +) class LarkTaskPilotTests(TestCase): + def test_lark_task_pilot_workflow_builder_keeps_dry_run_default(self): + workflow = build_lark_task_pilot_workflow() + node = next(item for item in workflow["nodes"] if item["id"] == "create_lark_task") + + self.assertEqual(workflow["workflow"]["id"], "workflow_lark_task_pilot") + self.assertEqual(workflow["workflow"]["version"], "0.1.0") + self.assertEqual(node["connector"]["mode"], "dry_run") + self.assertEqual( + node["description"], + "Validate a Lark/Feishu owner follow-up task request without calling the live API.", + ) + + def test_lark_task_pilot_workflow_builder_can_create_separate_live_artifact(self): + workflow = build_lark_task_pilot_workflow( + mode="live", + workflow_id="workflow_controlled_lark_pilot", + workflow_version="0.1.0", + workflow_name="controlled-lark-task-sales-renewal-pilot", + ) + node = next(item for item in workflow["nodes"] if item["id"] == "create_lark_task") + + self.assertEqual(workflow["workflow"]["id"], "workflow_controlled_lark_pilot") + self.assertEqual(workflow["workflow"]["name"], "controlled-lark-task-sales-renewal-pilot") + self.assertEqual(node["connector"]["mode"], "live") + self.assertNotIn("dry-run", node["description"]) + + def test_lark_task_pilot_workflow_builder_propagates_non_default_version(self): + workflow = build_lark_task_pilot_workflow(workflow_version="9.8.7") + + self.assertEqual(workflow["workflow"]["version"], "9.8.7") + + def test_lark_task_pilot_workflow_builder_rejects_unknown_mode(self): + with self.assertRaisesRegex(ValueError, "mode must be dry_run or live"): + build_lark_task_pilot_workflow(mode="other") + def test_lark_task_pilot_runs_sales_renewal_flow_with_control_gate(self): repo_root = Path(__file__).resolve().parents[1] diff --git a/tests/test_live_connector_readiness.py b/tests/test_live_connector_readiness.py index 9b34c6b..09d4033 100644 --- a/tests/test_live_connector_readiness.py +++ b/tests/test_live_connector_readiness.py @@ -72,13 +72,54 @@ def test_lark_live_connector_readiness_decision_is_documented(self): self.assertIn("default remains dry-run", decision) self.assertIn("revert Loop 39 without changing Workflow DSL compatibility", decision) + self.assertIn("https://open.feishu.cn/open-apis/task/v2/tasks?user_id_type=open_id", decision) + self.assertIn("native `client_token`", decision) + self.assertIn("SKILL2WORKFLOW_LARK_TASK_LIVE=1", decision) + self.assertIn("run.context.input", decision) + self.assertIn("must not copy raw task values into connector-produced state", decision) + self.assertIn("python3 scripts/lark_task_live_validation.py", decision) + self.assertIn("fixed 10-second timeout", decision) + self.assertIn("either `task:task:write` or `task:task:writeonly`", decision) + self.assertIn("10 create requests per second", decision) + self.assertIn( + "Retries may still invoke transport; the stable native `client_token` and unchanged request parameters let Feishu perform provider-side deduplication", + decision, + ) + self.assertNotIn("idempotency key blocks duplicate task creation attempts", decision) + self.assertIn( + "Recognized Feishu provider codes take precedence over generic HTTP status classification", + decision, + ) + self.assertIn( + "Normalized `provider_status` values are exactly: `live_disabled`, `validation_failed`, `credential_failed`, `authorization_failed`, `permission_denied`, `rate_limited`, `resource_not_found`, `idempotency_conflict`, `provider_unavailable`, `timeout`, `malformed_response`, and `completed`.", + decision, + ) + self.assertIn("docs/lark-live-connector-readiness.md", connectors) self.assertIn("Loop 38 readiness review approved only a scoped live `create_task` follow-up", connectors) + self.assertIn("mode: live", connectors) + self.assertIn("SKILL2WORKFLOW_LARK_TASK_LIVE=1", connectors) + self.assertIn("provider_status", connectors) + self.assertIn("LARK_BOT_ACCESS_TOKEN", connectors) + vault_command = ( + "vibe vault run --env LARK_BOT_ACCESS_TOKEN -- env " + "SKILL2WORKFLOW_LARK_TASK_LIVE=1 python3 scripts/lark_task_live_validation.py" + ) + self.assertIn(vault_command, decision) + self.assertIn(vault_command, connectors) + self.assertIn("Never paste the token into the command or shell history", decision) + self.assertIn("Never paste the token into the command or shell history", connectors) self.assertIn("| Loop 38: Live Connector Readiness Review | Complete |", roadmap) - self.assertIn("Active loop: Loop 39, Scoped Live Lark Task Connector", roadmap) - self.assertIn("| Loop 39: Scoped Live Lark Task Connector | Next |", roadmap) - self.assertIn("Loop 38 approved only scoped live `create_task` work", roadmap) + self.assertIn("Active loop: None; Loop 40 is deferred pending a new partner-approved pilot", roadmap) + self.assertIn("| Loop 39: Scoped Live Lark Task Connector | Complete |", roadmap) + self.assertIn("| Loop 40: Controlled Live Connector Pilot | Deferred |", roadmap) + self.assertIn("docs/lark-live-connector-validation.md", roadmap) + self.assertIn("Live behavior remains limited to the fixed `create_task` action.", roadmap) + self.assertIn( + "The one scoped live connector validation is not the controlled real-team business-workflow pilot required for Loop 40.", + roadmap, + ) def _read(path: str) -> str: diff --git a/tests/test_product_connector_pilot_roadmap.py b/tests/test_product_connector_pilot_roadmap.py index 27e0919..1184bcf 100644 --- a/tests/test_product_connector_pilot_roadmap.py +++ b/tests/test_product_connector_pilot_roadmap.py @@ -9,11 +9,12 @@ class ProductConnectorPilotRoadmapTests(TestCase): def test_loop_37_product_connector_pilot_is_scoped(self): roadmap = (ROOT / "ROADMAP.md").read_text(encoding="utf-8") - self.assertIn("Active loop: Loop 39, Scoped Live Lark Task Connector", roadmap) + self.assertIn("Active loop: None; Loop 40 is deferred pending a new partner-approved pilot", roadmap) self.assertIn("| Loop 36: First Product Connector Package Smoke | Complete |", roadmap) self.assertIn("| Loop 37: Product Connector Pilot Scenario | Complete |", roadmap) self.assertIn("| Loop 38: Live Connector Readiness Review | Complete |", roadmap) - self.assertIn("| Loop 39: Scoped Live Lark Task Connector | Next |", roadmap) + self.assertIn("| Loop 39: Scoped Live Lark Task Connector | Complete |", roadmap) + self.assertIn("| Loop 40: Controlled Live Connector Pilot | Deferred |", roadmap) self.assertIn("sales renewal risk workflow", roadmap) self.assertIn("manual control gate", roadmap) @@ -21,6 +22,6 @@ def test_loop_37_product_connector_pilot_is_scoped(self): "python3 scripts/lark_task_pilot_smoke.py --work-dir /tmp/skill2workflow-lark-task-pilot", roadmap, ) - self.assertIn("Loop 38 approved only scoped live `create_task` work", roadmap) - self.assertIn("Credential handling and audit redaction rules are explicit", roadmap) - self.assertIn("feature flag or equivalent explicit opt-in", roadmap) + self.assertIn("Live behavior remains limited to the fixed `create_task` action.", roadmap) + self.assertIn("one scoped live connector validation", roadmap) + self.assertIn("not the controlled real-team business-workflow pilot required for Loop 40", roadmap) diff --git a/tests/test_production_roadmap.py b/tests/test_production_roadmap.py index 8a8fda7..64a5125 100644 --- a/tests/test_production_roadmap.py +++ b/tests/test_production_roadmap.py @@ -26,20 +26,26 @@ def test_roadmap_uses_a_rolling_production_readiness_path(self): self.assertIn("self-hosted, single-tenant workflow runtime for one team", roadmap) self.assertIn("- Current maturity: Local Evaluation", roadmap) - self.assertIn("- Active loop: Loop 39, Scoped Live Lark Task Connector", roadmap) + self.assertIn("- Completed delivery loops: 1-39", roadmap) + self.assertIn("- Active loop: None; Loop 40 is deferred pending a new partner-approved pilot", roadmap) self.assertIn("- Next maturity gate: Controlled Live Pilot", roadmap) + self.assertIn("docs/controlled-pilot-deferral-review.md", roadmap) self.assertIn("### Local Evaluation", roadmap) self.assertIn("**Status:** Achieved.", roadmap) self.assertIn("### Controlled Live Pilot", roadmap) - self.assertIn("**Target loops:** 39-40.", roadmap) + self.assertIn("**Target loops:** 40.", roadmap) self.assertIn("### Self-hosted Beta", roadmap) self.assertIn("**Target loops:** 41-43.", roadmap) self.assertIn("### Production Baseline", roadmap) self.assertIn("**Status:** Directional; no loop numbers assigned.", roadmap) self.assertIn( - "| Loop 40: Controlled Live Connector Pilot | Candidate |", + "| Loop 39: Scoped Live Lark Task Connector | Complete |", + roadmap, + ) + self.assertIn( + "| Loop 40: Controlled Live Connector Pilot | Deferred |", roadmap, ) self.assertIn( @@ -114,6 +120,7 @@ def test_roadmap_preserves_complete_delivery_history(self): "| Loop 36: First Product Connector Package Smoke | Complete | Lark/Feishu task connector dry-run package fixture, explicit-loading smoke, credential-handle evidence, and compact connector metadata |", "| Loop 37: Product Connector Pilot Scenario | Complete | Sales renewal risk workflow using the Lark/Feishu task dry-run connector after a manual gate, with webhook trigger, audit, snapshot, and LiteGraph overlay artifacts |", "| Loop 38: Live Connector Readiness Review | Complete | Decision note approving only scoped live Lark/Feishu `create_task` follow-up, with credential, idempotency, failure, audit, test, and rollback boundaries |", + "| Loop 39: Scoped Live Lark Task Connector | Complete | Explicit live `create_task` opt-in, fake-transport coverage, native provider idempotency, redaction and rollback boundaries, and one redacted real-validation evidence note |", ] for row in history_rows: @@ -124,7 +131,15 @@ def test_loop_39_completion_requires_one_explicitly_enabled_live_task(self): roadmap = _read("ROADMAP.md") self.assertIn( - "The project can create one live Lark/Feishu task through an explicitly enabled local connector path.", + "docs/lark-live-connector-validation.md", + roadmap, + ) + self.assertIn( + "Live behavior remains limited to the fixed `create_task` action.", + roadmap, + ) + self.assertIn( + "The one scoped live connector validation is not the controlled real-team business-workflow pilot required for Loop 40.", roadmap, ) @@ -132,12 +147,12 @@ def test_readme_summarizes_without_copying_the_rolling_queue(self): readme = _read("README.md") self.assertIn("Current maturity: Local Evaluation", readme) - self.assertIn("Delivery Loops 1-38 are complete", readme) - self.assertIn("Loop 39", readme) + self.assertIn("Delivery Loops 1-39 are complete", readme) + self.assertIn("Loop 40", readme) self.assertIn("self-hosted, single-tenant runtime for one team", readme) self.assertIn("`ROADMAP.md`", readme) + self.assertIn("Loop 40 is deferred", readme) candidate_loop_titles = [ - "Loop 40: Controlled Live Connector Pilot", "Loop 41: Self-hosted Runtime Service Boundary", "Loop 42: Authenticated Ingress And Production Credentials", "Loop 43: Durable Recurring Scheduling And Safe Dispatch",