[upstream-sync] cocoindex v1.0.6–v1.0.7 Rust changes (2026-05-18 to 2026-06-01)

[github-actions]

Upstream Sync Summary: v1.0.6–v1.0.7 (2026-05-18 to 2026-06-01)

This issue tracks all upstream Rust-relevant changes from cocoindex v1.0.6 and v1.0.7 not yet covered by existing recoco issues. Python-only changes, already-tracked PRs, and trivial changes are excluded.

---

Already Tracked (do not re-open)

The following upstream changes are covered by existing recoco issues:

• #2022 → recoco [upstream-sync] refactor(engine,state_store): parallel-write shape, borrow-only reconcile (upstream #2022) #178 (parallel-write shape, borrow-only reconcile)
• #2014 → recoco [upstream-sync] fix(engine): submit parent target states after child readiness (upstream #2014) #177 (submit after child readiness)
• #2004 → recoco [upstream-sync] fix(engine): scope GC-sweep read transaction to list_tombstones only (upstream #2004) #175 (GC-sweep rtxn scope)
• #1994, #1996 → recoco [upstream-sync] fix(engine): concurrent preempt race + pending_process_token refactor (upstream #1994, #1996) #174 (concurrent preempt race + pending_process_token)
• #1984 → recoco [upstream-sync] feat(splitter): add Astro tree-sitter language support (upstream #1984) #172 (Astro tree-sitter)

---

New Relevant Changes

🔴 High Priority — Bug Fixes in Rust Core

#2038 — fix(engine): don't force prev_may_be_missing on multi-state items

• Type: bug-fix | Difficulty: Medium | Recommendation: Adopt
• pre_commit incorrectly set prev_may_be_missing = ... || item.is_pending() for multi-state items, causing a permanent divergence where a failed DROP COLUMN would never retry correctly.
• Files: rust/core/src/engine/execution.rs, rust/core/src/state/db_schema.rs
• Recoco: crates/recoco-core/src/execution/ — adopt fix directly; affects the postgres target column-drop retry path.

#1999 — fix(engine): cascade on_error through Build-mode GC sweep

• Type: bug-fix | Difficulty: Medium | Recommendation: Adopt
• Orphan-delete failures during commit-phase GC sweep were silently swallowed. Now routes through the parent's exception handler chain via ComponentBuildContext::on_error.
• Files: rust/core/src/engine/app.rs, component.rs, context.rs, execution.rs, live_component.rs
• Recoco: crates/recoco-core/src/execution/ — adopt; improves error surfacing in the Rust API.

---

🔴 High Priority — Architectural Changes (Engine/State Store)

#2037 — refactor(engine,state_store): unified-precommit + PrecommitSession

• Type: architectural-change | Difficulty: Hard | Recommendation: Adopt
• Replaces two-phase precommit_read → precommit_write split with a single-txn AppStore::precommit(component_path, callback). Adds PrecommitSession type that provides type-level proof the txn is open. Engine's submit() now builds paths_to_claim as declared_paths − existing_paths_in_tracking_info to skip redundant reads.
• Files: rust/core/src/engine/execution.rs, rust/core/src/state_store/app_store.rs, rust/core/src/state_store/mod.rs, rust/core/src/state_store/submit_session.rs
• Recoco: crates/recoco-core/src/ — major refactor; closely related to already-tracked #2030. Must be ported as a unit with #2030.

#2030 — refactor(engine,state_store): submit() rewrite + Committer::commit shape

• Type: architectural-change | Difficulty: Hard | Recommendation: Adopt
• Introduces per-component SubmitSession through begin_submit → precommit_read → reconcile → precommit_write → commit/cleanup. Committer::commit_in_txn replaced by SubmitSession::commit(plan, existence_reconciler). Adds cleanup_tombstone_standalone, finalize_memoization_standalone, and ensure_existence_chain_standalone as coalescing helpers on AppStore.
• Files: rust/core/src/engine/app.rs, context.rs, environment.rs, execution.rs, id_sequencer.rs, live_component.rs, rust/core/src/state_store/
• Recoco: crates/recoco-core/ — port together with #2037; these two form a unit.

#2010 — refactor(state_store): remove ReadTxn from API; standalone reads per call

• Type: architectural-change | Difficulty: Medium | Recommendation: Adopt
• AppStore read methods now have *_in_txn(wtxn, ...) variants (for reads inside write txn) and standalone xxx(...) that open their own snapshot. Removes ReadTxn, AnyTxn, read_txn_for_inspect types. spawn_iter_stable_paths_with_node_type moved to tokio::task::spawn_blocking.
• Files: rust/core/src/engine/context.rs, environment.rs, execution.rs, id_sequencer.rs, rust/core/src/inspect/db_inspect.rs, rust/core/src/state_store/
• Recoco: crates/recoco-core/ — adopt; simplifies the state store API and eliminates caller-managed snapshot lifetimes.

---

🟡 Medium Priority — New Rust Features / API Changes

#1954 — feat(splitter): add Bash, CMake, and HCL tree-sitter support

• Type: new-feature | Difficulty: Easy | Recommendation: Adopt
• Adds syntax-aware splitting for .sh/.bash, .cmake/.cmake.in, and .hcl/.tf via tree-sitter-bash 0.25.1, tree-sitter-cmake 0.7.1, tree-sitter-hcl 1.1.0 (all targeting tree-sitter ^0.25).
• Files: rust/ops_text/ (maps to crates/recoco-splitters)
• Recoco: Add to crates/recoco-splitters; feature-gate under existing splitter feature. Check if tree-sitter version matches recoco's pinned version.

#2031 — refactor(error): add Error::replica + HostError::try_clone

• Type: api-change | Difficulty: Medium | Recommendation: Adopt
• Adds HostError::try_clone(&self) -> Option<Box<dyn HostError>> (default None) and Error::replica() so batch/shared error consumers get proper error clones rather than flattened ResidualError. Fixes cancellation flavor loss in batch residuals.
• Files: rust/utils/src/error.rs, rust/utils/src/batching.rs
• Recoco: crates/recoco-utils/src/error.rs and batching.rs — adopt; pure Rust improvement, no Python components. Recoco has no HostedPyErr so the Python impl is excluded.

#2042 — feat(engine): add stats_group() for scoped stats reports

• Type: new-feature | Difficulty: Hard | Recommendation: Adapt
• Adds coco.stats_group(title, ...) for splitting stats aggregation into named sub-groups. Rust side moves processing_stats + components_readiness to per-view field on ComponentProcessorContext, adds detached lifecycle tasks for group readiness.
• Files: rust/core/src/engine/component.rs, context.rs, progress_display.rs
• Recoco: crates/recoco-core/ — adapt for Rust-only API (no Python bindings needed); expose via recoco's public API if stats/progress watching is exposed.

#1945 — Add preview mode for update actions

• Type: new-feature | Difficulty: Medium | Recommendation: Adapt
• Adds preview mode to the engine: reconciliation runs through pre_commit, planning runs in a standalone write txn that is dropped without commit, sink.apply(...) is skipped. Rust engine changes in app.rs, component.rs, context.rs, execution.rs, live_component.rs.
• Files: rust/core/src/engine/ (app, component, context, execution, live_component)
• Recoco: crates/recoco-core/ — adapt; expose preview parameter in recoco's public Rust API for App::update().

#1995 — feat(live_component): auto_refresh + unified error-handling channel

• Type: new-feature | Difficulty: Hard | Recommendation: Adapt
• Unifies error-handling around OnError = Fn(Error) → Future<Result<()>>. Adds "process_live" to MountKind. Adds report_exception() on LiveComponentOperator. Rust core engine changes across all major engine files.
• Files: rust/core/src/engine/app.rs, component.rs, context.rs, execution.rs, live_component.rs
• Recoco: crates/recoco-core/ — adapt; the OnError type and MountKind extension are relevant to the Rust SDK. Exclude auto_refresh (Python-specific decorator pattern).

---

🟡 Medium Priority — Progress Display Fixes (Rust Core)

#2033 + #2036 — fix(progress_display): PTY reader stability

• Type: bug-fix | Difficulty: Easy | Recommendation: Adopt
• #2033: Moves PTY master reader from tokio::spawn + AsyncFd to std::thread::spawn with blocking read(), preventing Tokio runtime starvation under heavy trace load.
• #2036: Switches master fd to non-blocking with poll(100ms) + Arc<AtomicBool> shutdown flag, fixing macOS hang when inherited slave fds prevent EOF.
• Files: rust/core/src/engine/progress_display.rs
• Recoco: crates/recoco-core/src/execution/ — adopt both as a pair; pure Rust correctness fixes.

#2040 — fix(progress_display): no-TTY progress on refresh interval only

• Type: bug-fix | Difficulty: Easy | Recommendation: Adopt
• In non-TTY path, renders progress only on refresh interval (was every stats change → flood). Adds dedicated single-shot terminated channel to ProcessingStats; adds AppOpHandle::wait_terminated.
• Files: rust/core/src/engine/progress_display.rs, rust/core/src/engine/stats.rs, rust/core/src/engine/app.rs
• Recoco: crates/recoco-core/ — adopt; improves behavior when redirecting output to files.

---

🟢 Lower Priority

#1992 — feat(telemetry): add optional application label via env var

• Type: new-feature | Difficulty: Easy | Recommendation: Adopt
• Adds COCOINDEX_APPLICATION_FOR_TRACKING env var; stores as Option<String> on TelemetryContext, serialized with skip_serializing_if.
• Files: rust/core/src/telemetry/mod.rs
• Recoco: crates/recoco-core/ — adopt; trivial additive change to telemetry (rename env var to RECOCO_APPLICATION_FOR_TRACKING).

#2044 — feat(engine): report_to_stdout configures refresh interval

• Type: new-feature | Difficulty: Easy | Recommendation: Adapt
• Adds ProgressDisplayOptions::from_refresh_secs(Option<f64>) on the Rust side. Python API surface is bool | timedelta.
• Files: Primarily Python; Rust ProgressDisplayOptions struct
• Recoco: crates/recoco-core/ — adapt; expose refresh interval configuration in the Rust ProgressDisplayOptions struct.

---

Notable Non-Action: #2032 — Do NOT delete str_sanitize.rs

Upstream deleted rust/utils/src/str_sanitize.rs because their Postgres target moved to Python. Recoco must NOT follow this deletion — crates/recoco-utils/src/str_sanitize.rs is actively used by crates/recoco-core/src/ops/targets/postgres.rs (the Rust Postgres target). This is a divergence point.

---

Skipped (Python-only or trivial)

#2050 (example review), #2045 (Mermaid docs), #2035 (Python logger), #2029 (Python halfvec ops), #2028 (CI CRLF), #1951 (Python LanceDB), #2026 (agent instructions), #2012 (NumPy serde), #2013 (Python LanceDB optimize), #2008 (Python LanceDB optimize), #2006 (Python entity resolution), #2003 (Python live component), #2000 (memo docs), #1998 (Python live component), #1997 (iggy/mypy), #1969 (Python Iggy connector)

Upstream References

• Releases: https://github.com/cocoindex-io/cocoindex/releases/tag/v1.0.6 · https://github.com/cocoindex-io/cocoindex/releases/tag/v1.0.7
• Upstream repo: https://github.com/cocoindex-io/cocoindex
• Period: 2026-05-18 to 2026-06-01