[upstream-sync] feat(live-component): rework controller internals; add drain registry (upstream #1957)

[github-actions]

Upstream Change Summary

Type: architectural-change
Difficulty: Hard
Recommendation: Adapt

Upstream significantly reworked the live-component subsystem in rust/core/src/engine/live_component.rs. The existing RwLock + inflight_counter + next_seq op-serialization was replaced with a pending-map + per-subpath drain-task model. Key additions: LiveComponentState<Prof> rewrite with coalescing pending map and per-subpath drain tasks, App::drop_app now walks a live_components drain registry with a 30s per-component timeout (join_all), atomic cancel-and-snapshot under the registry lock to close a concurrent-mount race, mount_live_async 30s timeout on cancel_and_drain of prior incarnations, and nested live mount support via operator.update(LiveCompClass) with a cancellable update_full_lock. A new Error::cancelled() + CancelledError HostError pattern was also introduced (PyO3 boundary maps to Python asyncio.CancelledError — irrelevant for recoco). This builds on the LiveComponent foundation tracked in issue #148.

Upstream References

• PR #1957: feat(live-component): rework controller internals; add drain registry cocoindex-io/cocoindex#1957 — rework controller internals; add drain registry
• Upstream repo: https://github.com/cocoindex-io/cocoindex

Relevant Upstream Files / Areas

• rust/core/src/engine/live_component.rs — main controller rewrite
• rust/core/src/engine/app.rs — drop_app drain integration
• rust/core/src/engine/component.rs — parent backref, active_children with Weak keys
• rust/core/src/engine/context.rs, execution.rs
• rust/utils/src/error.rs — new Error::cancelled() / CancelledError

Maps to recoco: crates/recoco-core/src/engine/

Recoco Considerations

• Relation to [upstream-sync] feat: LiveComponent support in Rust core engine (upstream #1795) #148: Issue [upstream-sync] feat: LiveComponent support in Rust core engine (upstream #1795) #148 tracks the initial LiveComponent support (upstream #1795). This PR is a substantial further evolution — the drain registry and per-subpath model are significant reliability improvements and should be tracked separately.
• PyO3 exclusions: rust/py/src/live_component.rs changes (PyO3 boundary for Python asyncio.CancelledError) are not needed in recoco. Only the pure Rust core changes in rust/core/ matter.
• CancelledError pattern: The is_cancelled() on HostError trait and Error::cancelled() constructor in rust/utils/ map to crates/recoco-utils/. This is relevant and clean to adopt.
• Timeout constants: The 30s cancel_and_drain / drop_app timeouts are hardcoded. Recoco may want to expose these via Settings / LmdbSettings equivalent for library users.
• Shutdown correctness: The drop_app drain registry is a correctness fix (prevent resource leaks on shutdown). This should be a high-priority adoption.
• Active children GC: Weak-keyed active_children + identity-checked Drop is a clean memory management pattern to adopt.

Integration Notes

Adapt rather than adopt directly: port the Rust core changes from rust/core/src/engine/live_component.rs, app.rs, and component.rs while excluding the Python binding cascade. The CancelledError / is_cancelled() utility additions in rust/utils/ are straightforward to port to crates/recoco-utils/. Verify against the existing LiveComponent foundation in recoco before starting — some of the intermediate state from #148 and related issues may already handle parts of this.