fix(runtime.coordinator): losing a worker fails the threads it hosted

[laithalsaadoon]

Summary

InMemoryCoordinator.deregister_worker now fails every non-terminal thread the departing worker still hosted. Before this change a thread whose host worker was gone stayed idle in the registry: list_threads kept offering it, get_thread_status kept answering idle, and every routed operation on it (submit, notify, cancel, terminate, fork) raised ThreadNotFoundError for a thread that was registered. CoordinatorEndpoint._serve_connection calls deregister_worker in its finally when a client socket drops, so this is the only signal the coordinator gets when a worker process dies.

Addresses #50 (the worker-death half). WorkerLostError and its error_kind = "worker_lost" classification already landed in #51; this PR adds only the coordinator and worker behavior that raises it.

What changed

• runtime/coordinator.py: deregister_worker appends a FailedEvent naming the lost worker to each non-terminal thread of that worker and moves the thread to ThreadStatus.FAILED. The orphaned thread ids are remembered per worker so _adapter_for can raise WorkerLostError(worker_id, thread_ids) with the whole set for a registered-but-unroutable thread, in place of ThreadNotFoundError. _apply_lifecycle returns early when the thread is already in a terminal status, so a COMPLETED or FAILED from a cycle that was in flight when the worker died cannot move the thread back to idle.
• runtime/worker.py: _teardown only schedules each thread's deregister_thread; LocalWorker.close now drains those tasks before it calls deregister_worker. Without that, an orderly close would hand deregister_worker a registry still holding the worker's threads and publish all of them as failed.
• runtime/errors.py: WorkerLostError docstring states when the coordinator raises it and how it reaches a remote caller.
• protocols.py: Coordinator.deregister_worker documents the cascade and its postcondition; every routed method documents WorkerLostError in Raises.
• spec/ stubs mirror the docstring changes.

Invariants

• No thread hosted by a deregistered worker is left in a non-terminal status (deregister_worker postcondition).
• A terminal ThreadStatus is never downgraded by a later lifecycle event.
• A dispatcher-emitted FAILED still means "one cycle raised, the thread is idle and runnable"; only the coordinator writes ThreadStatus.FAILED, and only from deregister_worker.
• deregister_worker stays idempotent: a second call finds no non-terminal thread and appends no second FAILED event.
• An orderly LocalWorker.close produces no FAILED events and leaves list_threads empty.

Wire compatibility

The FAILED event is an existing kind every peer decodes. A remote caller sees the loss as a RemoteError with kind == "worker_lost" and remote_type == "WorkerLostError". WorkerLostError is deliberately not added to channel._KNOWN_EXCEPTIONS: its worker_id and thread_ids cannot be rebuilt from an ErrorFrame's message, so rehydrating it would mean inventing them. No signature changes.

Tests

• tests/test_worker_death.py (new): a real CoordinatorEndpoint, a client hosting a LocalWorker, and a second observer client. After the host client closes its socket, the observer sees the thread FAILED in list_threads, a terminal FailedEvent naming the worker id, and RemoteError(kind="worker_lost") from submit and notify.
• tests/test_lifecycle_invariants.py (six new cases): deregister_worker fails the hosted threads and the log ends with a FailedEvent; routed operations on an orphaned thread raise WorkerLostError carrying every orphaned id; a second deregister_worker appends nothing; a late CompletedEvent does not downgrade FAILED; a failed cycle leaves the thread IDLE; an orderly close fails nothing.

Gate

Run on this branch (f56e1e37, base 1125f0c4), mcp 2.1.1:

ruff format --check src tests   1 file would be reformatted (pre-existing on main: src/ai_functions/memory/base.py), 108 already formatted
hatch run lint                  All checks passed!
hatch run check-spec            Success: no issues found in 80 modules
hatch run test                  481 passed, 2 skipped