Harden controlled Lark pilot credentials

AGENTS.md

@@ -55,6 +55,8 @@ This repository is the `skill2workflow` open-source harness.
 - Control snapshot: `PYTHONPATH=src python3 -m skill2workflow.cli control-snapshot --state-dir /tmp/skill2workflow-control -o /tmp/skill2workflow-control-snapshot.json`
 - First-run demo: `python3 scripts/demo_bootstrap.py --work-dir /tmp/skill2workflow-demo`
 - Pilot smoke: `python3 scripts/pilot_playbook_smoke.py --work-dir /tmp/skill2workflow-pilot`
+- Controlled Lark pilot: `python3 scripts/controlled_lark_pilot.py --help`
+- Controlled Lark preflight: `python3 scripts/controlled_lark_pilot.py preflight --input /tmp/skill2workflow-private-case.json`
 - Schedule smoke: `python3 scripts/schedule_smoke.py --work-dir /tmp/skill2workflow-schedule-loop29`
 - Package smoke: `python3 scripts/package_smoke.py --work-dir /tmp/skill2workflow-package-smoke`
 - Secret hygiene: `python3 scripts/secret_hygiene.py examples/workflows`

README.md

@@ -498,9 +498,9 @@ ROADMAP.md        # Open-source delivery roadmap
 
 ## Roadmap
 
-Current maturity: Local Evaluation. The local-first harness covers all five approved architecture layers, and Delivery Loops 1-38 are complete.
+Current maturity: Local Evaluation. The local-first harness covers all five approved architecture layers, and Delivery Loops 1-39 are complete.
 
-The active priority is Loop 39: implement only the readiness-approved Lark/Feishu `create_task` live action behind explicit opt-in while keeping dry-run behavior as the default.
+Loop 40 is deferred after a normalized real Pilot failure; no further live calls are authorized under that Pilot. A future controlled real-team Pilot requires fresh authorization and the local no-network preflight before its separate human approval.
 
 The production direction is a self-hosted, single-tenant runtime for one team. See `ROADMAP.md` for the production-readiness gates, rolling Loop queue, acceptance evidence, and deferred boundaries.
 
@@ -510,6 +510,8 @@ See:
 - `ROADMAP.md`
 - `docs/authoring.md`
 - `docs/connectors.md`
+- `docs/controlled-pilot-deferral-review.md`
+- `docs/controlled-live-pilot.md`
 - `docs/credential-boundary.md`
 - `docs/examples.md`
 - `docs/pilot-playbook.md`

ROADMAP.md

@@ -12,11 +12,11 @@ Workflow DSL remains the authoritative execution source of truth. LiteGraph and
 
 - Published release: `v0.1.0`
 - Workflow DSL compatibility line: `0.1.x` artifacts using `schema_version: "0.1.0"`
-- Completed delivery loops: 1-38
+- Completed delivery loops: 1-39
 - Current maturity: Local Evaluation
-- Active loop: Loop 39, Scoped Live Lark Task Connector
+- Active loop: None; Loop 40 is deferred pending a new partner-approved pilot
 - Next maturity gate: Controlled Live Pilot
-- Next decision: validate the scoped live action in a controlled pilot or explicitly defer broader live behavior
+- Next decision: select and authorize a new controlled Pilot only after post-incident review
 
 ## Production Readiness Path
 
@@ -28,9 +28,9 @@ The repository can compile, validate, publish, trigger, execute, pause, resume,
 
 ### Controlled Live Pilot
 
-**Target loops:** 39-40.
+**Target loops:** 40.
 
-This gate requires one explicitly enabled live connector action plus controlled pilot evidence. It does not imply general live SaaS readiness.
+This gate requires the completed scoped live connector action plus controlled pilot evidence. It does not imply general live SaaS readiness.
 
 ### Self-hosted Beta
 
@@ -48,78 +48,41 @@ Candidate evidence includes backup and restore, upgrade and migration policy, ca
 
 ## Active Loop
 
-### Loop 39: Scoped Live Lark Task Connector
+### Loop 40: Controlled Live Connector Pilot
 
-**Status:** Next engineering loop.
+**Status:** Deferred after a normalized provider validation failure in the controlled real-team Pilot.
 
-**Goal:** Implement only the Loop 38-approved Lark/Feishu `create_task` live action behind explicit opt-in while keeping dry-run mode as the default.
+**Prior basis:** The Lark/Feishu task connector has package-level and pilot-workflow dry-run evidence, including the sales renewal risk workflow after a manual control gate. Loop 39 also produced the redacted connector-validation note at `docs/lark-live-connector-validation.md`. Live behavior remains limited to the fixed `create_task` action. The one scoped live connector validation is not the controlled real-team business-workflow pilot required for Loop 40.
 
-**Why now:** Loop 36 proved the out-of-core package boundary, Loop 37 proved the connector in a sales renewal risk workflow after a manual control gate, and Loop 38 approved implementation after package-level and pilot-workflow dry-run evidence. The remaining risk is disciplined live execution without weakening credential isolation, duplicate prevention, audit redaction, rollback, or Workflow DSL compatibility.
+**Outcome:** The Pilot retained its history, completed the required human-rejection and safety exercises, and recorded a private partner/operator `defer` decision. One approved live run completed, but a later approved attempt failed with normalized `validation_failed`; therefore the five-day acceptance threshold was not met. No finalization or repository evidence export occurred.
 
-**Decision boundary:** Loop 38 approved only scoped live `create_task` work. Any broader Lark/Feishu API behavior requires another readiness review. The full decision is recorded in `docs/lark-live-connector-readiness.md`.
+**Safety outcome:** The failed run remains in owner-only Pilot state. No retry was issued, no subsequent live run was approved, and no raw provider message, task data, identifier, or credential was exported. A recorded Pilot decision now closes that workspace to subsequent starts and decisions, and initialization never replaces its Charter. The connector also provides a no-Vault, no-network `preflight` check that constructs the fixed request shape before a future human-gated run.
 
-Approved scope:
+**Deferral review:** [`docs/controlled-pilot-deferral-review.md`](docs/controlled-pilot-deferral-review.md) records the supported facts, the intentionally unconfirmed provider root cause, and the fresh-authorization re-entry gate.
 
-- Connector id and kind: `lark_task`
-- Operation: `create_task`
-- Live mode: `live`
-- Default mode: `dry_run`
-- Credential handle: `lark_bot_access_token`
-- Idempotency key: derived from `workflow_id + version + run_id + node_id`
-- Test transport: fake Lark HTTP receiver or injected fake transport; no live network in CI
-- Evidence: compact connector and audit metadata only
+**Re-entry boundary:** A new Pilot requires fresh partner and operator authorization, a new valid charter, a new private case, a successful local `preflight`, and a separate explicit approval for each real create. The old failed run must never be retried or replaced. Live behavior remains limited to the fixed `create_task` action; any broader Lark/Feishu API behavior requires another readiness review.
 
-Implementation order:
-
-1. Add failing tests for opt-in, credential resolution, success, API failures, timeout, malformed responses, and redaction.
-2. Add idempotency and duplicate-prevention tests before outbound request code.
-3. Implement the minimal live `create_task` action.
-4. Prove existing dry-run tests and smokes remain unchanged.
-5. Update connector documentation without changing Workflow DSL compatibility.
-
-Acceptance criteria:
-
-- The project can create one live Lark/Feishu task through an explicitly enabled local connector path.
-- Live behavior requires a feature flag or equivalent explicit opt-in.
-- Dry-run remains the default for examples, CI, and contributor onboarding.
-- Credential handling and audit redaction rules are explicit in code, tests, and docs.
-- Resolved credentials, authorization headers, raw task values, raw request bodies, and raw response payloads never enter run state, audit, snapshots, or connector summaries.
-- Duplicate task creation is blocked for the same derived idempotency key.
-- `401 or 403`, rate limits, network timeouts, validation failures, and malformed responses become normalized failures where possible.
-- The live path can be disabled or reverted without changing Workflow DSL compatibility.
-
-Required verification:
+The dry-run behavioral baseline remains available through:
 
 ```bash
-PYTHONPATH=src python3 -m unittest discover -s tests -v
-python3 -m py_compile src/skill2workflow/*.py
-python3 scripts/secret_hygiene.py examples/workflows
 python3 scripts/lark_task_pilot_smoke.py --work-dir /tmp/skill2workflow-lark-task-pilot
-git diff --check
 ```
 
-Explicitly excluded from Loop 39:
-
-- OAuth and token refresh
-- Hosted callbacks or ingress
-- Automatic connector discovery or package installation
-- Marketplace indexing
-- Queues or production scheduling
-- Other Lark/Feishu APIs or operations
+Loop 40 is not complete and does not advance maturity. Its private decision is not a substitute for the completed five-day evidence gate.
 
 ## Rolling Loop Queue
 
-This rolling queue is ordered, but only Loop 39 is committed. Select the next loop after reviewing evidence from the previous one; candidate loop numbers may change when that evidence changes the plan.
+This rolling queue is ordered. Loop 40 is deferred and there is no active delivery loop; select the next loop only after the post-incident review.
 
 | Loop | Status | Goal | Exit artifact |
 | --- | --- | --- | --- |
-| Loop 39: Scoped Live Lark Task Connector | Next | Implement the approved live `create_task` path behind explicit opt-in | Tested live path, fake-transport evidence, and updated docs |
-| Loop 40: Controlled Live Connector Pilot | Candidate | Exercise Loop 39 through a controlled real-team pilot | Controlled live-pilot runbook, redacted run and audit evidence, failure and rollback exercises, and a continue/harden/defer decision |
+| Loop 39: Scoped Live Lark Task Connector | Complete | Explicit live `create_task` opt-in, fake-transport coverage, native provider idempotency, redaction and rollback boundaries, and one redacted real-validation evidence note |
+| Loop 40: Controlled Live Connector Pilot | Deferred | Controlled real-team Pilot stopped after a normalized provider validation failure and a private partner/operator `defer` decision | Retained private history, safe failure/rollback evidence, and a no-network request preflight; no maturity advance |
 | Loop 41: Self-hosted Runtime Service Boundary | Candidate | Add one long-running service entry point with validated configuration | Health/readiness checks, graceful shutdown, and restart continuity evidence |
 | Loop 42: Authenticated Ingress And Production Credentials | Candidate | Require authentication by default for the production service path and resolve credential handles at execution time | Compact security audit evidence and a documented external TLS termination boundary |
 | Loop 43: Durable Recurring Scheduling And Safe Dispatch | Candidate | Persist recurring schedules with restart recovery and a defined missed-run policy | Durable dispatch records and lease or locking semantics for one SQLite-backed service instance |
 
-Loop 40 must produce a reproducible controlled live-pilot runbook, redacted evidence, explicit failure and rollback exercises, and a decision to continue, harden, or defer broader live integration work. The repository must not commit live credentials or raw live payload evidence.
+Loop 40 is explicitly deferred, not complete. Any future Pilot must begin under a new authorization boundary and must still produce a reproducible controlled live-pilot runbook, redacted evidence, explicit failure and rollback exercises, and a decision to continue, harden, or defer broader live integration work. The repository must not commit live credentials or raw live payload evidence.
 
 Loop 41 keeps the runtime scope single-instance and single-tenant. It does not introduce worker coordination or a multi-tenant service boundary.
 
@@ -199,6 +162,7 @@ The detailed implementation plans under `docs/superpowers/plans/` are the histor
 | Loop 36: First Product Connector Package Smoke | Complete | Lark/Feishu task connector dry-run package fixture, explicit-loading smoke, credential-handle evidence, and compact connector metadata |
 | Loop 37: Product Connector Pilot Scenario | Complete | Sales renewal risk workflow using the Lark/Feishu task dry-run connector after a manual gate, with webhook trigger, audit, snapshot, and LiteGraph overlay artifacts |
 | Loop 38: Live Connector Readiness Review | Complete | Decision note approving only scoped live Lark/Feishu `create_task` follow-up, with credential, idempotency, failure, audit, test, and rollback boundaries |
+| Loop 39: Scoped Live Lark Task Connector | Complete | Explicit live `create_task` opt-in, fake-transport coverage, native provider idempotency, redaction and rollback boundaries, and one redacted real-validation evidence note |
 
 ## Release Direction
 

docs/connectors.md

@@ -1,7 +1,7 @@
 # Connector Runtime
 
 `skill2workflow` currently ships a minimal local connector runtime. It is designed to make connector-bound workflow nodes testable and auditable without adding external services, SDK dependencies, secret storage, or a connector marketplace.
-Loop 33 adds one explicitly loaded local external connector fixture to prove the extension boundary. Loop 36 adds the first product-shaped connector package fixture, a Lark/Feishu task `create_task` dry-run connector. Loop 37 proves that connector inside a sales renewal risk pilot workflow. Loop 38 readiness review approved only a scoped live `create_task` follow-up, documented in `docs/lark-live-connector-readiness.md`. These loops do not add automatic discovery, live SaaS calls, OAuth, or marketplace behavior.
+Loop 33 adds one explicitly loaded local external connector fixture to prove the extension boundary. Loop 36 adds the first product-shaped connector package fixture, a Lark/Feishu task `create_task` dry-run connector. Loop 37 proves that connector inside a sales renewal risk pilot workflow. Loop 38 readiness review approved only a scoped live `create_task` follow-up, documented in `docs/lark-live-connector-readiness.md`. Loop 39 implements that one opt-in live action while preserving explicit loading and the dry-run default; it does not add automatic discovery, OAuth, token refresh, or marketplace behavior.
 
 Workflow DSL remains the execution truth source. Connector bindings live on workflow nodes, and the local executor records connector lifecycle events in run state and control-plane audit logs.
 
@@ -245,15 +245,16 @@ Connector package smoke contract:
 
 Package conventions intentionally exclude automatic connector discovery, package installation, marketplace indexing, OAuth, hosted callbacks, queues, production schedulers, and product-specific SaaS connector behavior.
 
-## Lark/Feishu Task Connector Dry-Run Package
+## Lark/Feishu Task Connector: Dry-Run Default And Scoped Live Mode
 
 `examples/connectors/lark_task_connector.py` is the first product-shaped connector package fixture. It stays outside the built-in connector registry and must be explicitly loaded with `load_external_connector(...)`.
 
 Supported scope:
 
 - connector id and kind: `lark_task`
 - operation: `create_task`
-- mode: `dry_run`
+- default mode: `dry_run`
+- opt-in mode: live
 - node type: `tool_call`
 - credential handle: `lark_bot_access_token`
 - input mapping: body-only values from `/input/title`, `/input/description`, `/input/assignee_open_id`, and `/input/due_at`
@@ -265,9 +266,34 @@ The connector validates the request shape, resolves the local credential handle,
 - input mapping status and input key names
 - booleans indicating whether title, description, assignee, and due date were present
 
-It does not call the live Lark/Feishu API, create tasks, perform OAuth, refresh tokens, host callbacks, install packages, auto-discover connectors, or enqueue background jobs. Raw mapped task values and resolved credential values must not appear in connector output or audit metadata.
+Dry-run remains the default when `mode` is missing or is `dry_run`. It validates and summarizes the request without a provider call. Raw mapped task values and resolved credential values must not appear in connector output or audit metadata.
 
-The live connector readiness decision is documented in `docs/lark-live-connector-readiness.md`. That decision approves only a future opt-in `create_task` live mode behind explicit credential, idempotency, failure-handling, audit-redaction, local-test, and rollback boundaries. The current package remains dry-run-only until that follow-up implementation is merged.
+The live connector readiness decision is documented in `docs/lark-live-connector-readiness.md`. The package now supports only the approved opt-in `create_task` action. Live network activity requires both an explicitly loaded binding with `mode: live` and the exact environment switch `SKILL2WORKFLOW_LARK_TASK_LIVE=1`; removing the switch immediately rolls the connector back to no live calls. No other truthy environment values enable it.
+
+The connector posts only to the fixed Feishu domestic Task API v2 endpoint:
+
+```text
+https://open.feishu.cn/open-apis/task/v2/tasks?user_id_type=open_id
+```
+
+It uses a fixed 10-second timeout. The required provider scope is either `task:task:write` or `task:task:writeonly`, and the documented limit is 10 create requests per second. The connector derives the native Feishu `client_token` from runtime-owned workflow id, workflow version, run id, and node id. Retries may still invoke transport; reusing the stable token with unchanged request parameters lets Feishu perform provider-side deduplication. The connector does not locally block retry transport calls.
+
+Normal execution resolves the `lark_bot_access_token` handle through the configured credential provider. `LARK_BOT_ACCESS_TOKEN` is reserved for the guarded validation helper and is not a Workflow DSL or connector-binding field. The helper must be run outside CI with explicit confirmation:
+
+```bash
+vibe vault run --env LARK_BOT_ACCESS_TOKEN -- env SKILL2WORKFLOW_LARK_TASK_LIVE=1 python3 scripts/lark_task_live_validation.py \
+  --confirm-live-create \
+  --validation-run-id '<stable-run-id>' \
+  --assignee-open-id '<open_id>' \
+  --title '<task-title>' \
+  --description '<task-description>'
+```
+
+Use Avibe Vault as shown, or an equivalent secret manager that injects `LARK_BOT_ACCESS_TOKEN` only into the child process. Never paste the token into the command or shell history.
+
+CI injects a fake transport and never accesses the live network. Recognized Feishu provider codes take precedence over generic HTTP status classification. Normalized `provider_status` values are exactly: `live_disabled`, `validation_failed`, `credential_failed`, `authorization_failed`, `permission_denied`, `rate_limited`, `resource_not_found`, `idempotency_conflict`, `provider_unavailable`, `timeout`, `malformed_response`, and `completed`.
+
+Connector-produced output, audit, events, snapshots, and summaries contain presence flags and compact statuses only. They never contain raw provider messages, task values, the task guid, resolved token, `client_token`, raw request, or raw response. Durable user-supplied task input remains unchanged under `run.context.input`; it is not connector-produced state.
 
 Run the dry-run smoke from a source checkout:
 
@@ -285,6 +311,8 @@ python3 scripts/lark_task_pilot_smoke.py --work-dir /tmp/skill2workflow-lark-tas
 
 The pilot uses the same explicitly loaded package inside a workflow that starts through the local webhook trigger boundary, waits at a manual gate, resumes with approval, and then invokes the connector. It proves business handoff and operator evidence, not live Lark/Feishu task creation.
 
+For the one approved controlled real-team pilot, follow `docs/controlled-live-pilot.md`. The dry-run remains the default; that runbook permits only the fixed Feishu domestic `create_task` action behind the existing explicit live guards.
+
 HTTP connector bindings may also reference local credential handles:
 
 ```json