Skip to content

feat(taskmill): add hierarchical child tasks with two-phase execution#2

Merged
deepjoy merged 2 commits into
mainfrom
child-tasks
Mar 14, 2026
Merged

feat(taskmill): add hierarchical child tasks with two-phase execution#2
deepjoy merged 2 commits into
mainfrom
child-tasks

Conversation

@deepjoy

@deepjoy deepjoy commented Mar 14, 2026

Copy link
Copy Markdown
Owner
  • Add parent-child task hierarchy: executors can spawn child tasks via
    TaskContext::spawn_child() / spawn_children() during execution
  • Introduce two-phase execution model — after a parent's execute() returns
    and all children complete, the scheduler calls finalize() on the parent
    executor (e.g. for assembly work like CompleteMultipartUpload)
  • Add Waiting task status for parents blocked on active children
  • Support fail_fast mode (default): first child failure cancels siblings
    and fails the parent immediately. When disabled, the parent waits for all
    children before resolving
  • Cascading cancellation: cancelling a parent cancels all its children
  • Crash recovery preserves waiting parents; their running children are
    reset to pending and re-trigger finalization on completion
  • Add module-level rustdoc across all modules and a crate-level quick-start
    example
  • Tighten public API: make StateMap, ChildSpawner, SmoothedReader, and
    run_sampler pub(crate)

Schema changes

  • tasks and task_history tables gain parent_id INTEGER and
    fail_fast INTEGER NOT NULL DEFAULT 1 columns
  • Partial indexes on parent_id for both tables

deepjoy added 2 commits March 13, 2026 18:51
Parent tasks can spawn children during execution via ChildSpawner.
Parents enter a Waiting state until all children finish, then a
finalize() phase runs before the parent completes. Configurable
fail_fast flag controls whether first child failure cancels siblings.
Cancelling a parent cascades to all children.

Also fixes a pool self-deadlock in store.complete/fail where the
connection was held across maybe_prune(), and changes the default
retention policy to MaxCount(10,000).
- Add module-level doc comments (`//!`) to all public and internal modules
  (backpressure, priority, registry, resource, sampler, sysinfo_monitor,
  scheduler, dispatch, gate, progress, store, task)
- Add a `# Quick start` code example to the crate root (`lib.rs`)
- Add doc examples for `PressureSource`, `ResourceSampler`, and
  `ProgressReporter`
- Fix broken intra-doc links by qualifying cross-module references
  (e.g. `[TaskContext::state](crate::TaskContext::state)`)
- Tighten visibility: make `StateMap`, `ChildSpawner`, `SmoothedReader`,
  and `run_sampler` `pub(crate)` instead of `pub`
- Remove `ChildSpawner` and `SmoothedReader` from the public re-exports
@deepjoy deepjoy closed this Mar 14, 2026
@deepjoy deepjoy reopened this Mar 14, 2026
@deepjoy deepjoy enabled auto-merge (squash) March 14, 2026 01:58
@deepjoy deepjoy disabled auto-merge March 14, 2026 01:59
@deepjoy deepjoy merged commit e13a623 into main Mar 14, 2026
2 checks passed
@github-actions github-actions Bot mentioned this pull request Mar 14, 2026
deepjoy pushed a commit that referenced this pull request Mar 14, 2026
## 🤖 New release

* `taskmill`: 0.1.1 -> 0.2.0 (⚠ API breaking changes)

### ⚠ `taskmill` breaking changes

```text
--- failure constructible_struct_adds_field: externally-constructible struct adds field ---

Description:
A pub struct constructible with a struct literal has a new pub field. Existing struct literals must be updated to include the new field.
        ref: https://doc.rust-lang.org/reference/expressions/struct-expr.html
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.46.0/src/lints/constructible_struct_adds_field.ron

Failed in:
  field SchedulerSnapshot.waiting_count in /tmp/.tmpcxDyU1/taskmill/src/scheduler/mod.rs:48
  field SchedulerSnapshot.waiting_count in /tmp/.tmpcxDyU1/taskmill/src/scheduler/mod.rs:48
  field TaskRecord.parent_id in /tmp/.tmpcxDyU1/taskmill/src/task.rs:103
  field TaskRecord.fail_fast in /tmp/.tmpcxDyU1/taskmill/src/task.rs:107
  field TaskRecord.parent_id in /tmp/.tmpcxDyU1/taskmill/src/task.rs:103
  field TaskRecord.fail_fast in /tmp/.tmpcxDyU1/taskmill/src/task.rs:107
  field TaskSubmission.parent_id in /tmp/.tmpcxDyU1/taskmill/src/task.rs:229
  field TaskSubmission.fail_fast in /tmp/.tmpcxDyU1/taskmill/src/task.rs:234
  field TaskSubmission.parent_id in /tmp/.tmpcxDyU1/taskmill/src/task.rs:229
  field TaskSubmission.fail_fast in /tmp/.tmpcxDyU1/taskmill/src/task.rs:234
  field TaskHistoryRecord.parent_id in /tmp/.tmpcxDyU1/taskmill/src/task.rs:144
  field TaskHistoryRecord.fail_fast in /tmp/.tmpcxDyU1/taskmill/src/task.rs:146
  field TaskHistoryRecord.parent_id in /tmp/.tmpcxDyU1/taskmill/src/task.rs:144
  field TaskHistoryRecord.fail_fast in /tmp/.tmpcxDyU1/taskmill/src/task.rs:146

--- failure enum_no_repr_variant_discriminant_changed: enum variant had its discriminant change value ---

Description:
The enum's variant had its discriminant value change. This breaks downstream code that used its value via a numeric cast like `as isize`.
        ref: https://doc.rust-lang.org/reference/items/enumerations.html#assigning-discriminant-values
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.46.0/src/lints/enum_no_repr_variant_discriminant_changed.ron

Failed in:
  variant SchedulerEvent::Paused 6 -> 7 in /tmp/.tmpcxDyU1/taskmill/src/scheduler/mod.rs:116
  variant SchedulerEvent::Resumed 7 -> 8 in /tmp/.tmpcxDyU1/taskmill/src/scheduler/mod.rs:118
  variant SchedulerEvent::Paused 6 -> 7 in /tmp/.tmpcxDyU1/taskmill/src/scheduler/mod.rs:116
  variant SchedulerEvent::Resumed 7 -> 8 in /tmp/.tmpcxDyU1/taskmill/src/scheduler/mod.rs:118

--- failure enum_variant_added: enum variant added on exhaustive enum ---

Description:
A publicly-visible enum without #[non_exhaustive] has a new variant.
        ref: https://doc.rust-lang.org/cargo/reference/semver.html#enum-variant-new
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.46.0/src/lints/enum_variant_added.ron

Failed in:
  variant TaskStatus:Waiting in /tmp/.tmpcxDyU1/taskmill/src/task.rs:28
  variant TaskStatus:Waiting in /tmp/.tmpcxDyU1/taskmill/src/task.rs:28
  variant SchedulerEvent:Waiting in /tmp/.tmpcxDyU1/taskmill/src/scheduler/mod.rs:114
  variant SchedulerEvent:Waiting in /tmp/.tmpcxDyU1/taskmill/src/scheduler/mod.rs:114

--- failure function_missing: pub fn removed or renamed ---

Description:
A publicly-visible function cannot be imported by its prior path. A `pub use` may have been removed, or the function itself may have been renamed or removed entirely.
        ref: https://doc.rust-lang.org/cargo/reference/semver.html#item-remove
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.46.0/src/lints/function_missing.ron

Failed in:
  function taskmill::resource::sampler::run_sampler, previously in file /tmp/.tmpv1zNi5/taskmill/src/resource/sampler.rs:80

--- failure struct_missing: pub struct removed or renamed ---

Description:
A publicly-visible struct cannot be imported by its prior path. A `pub use` may have been removed, or the struct itself may have been renamed or removed entirely.
        ref: https://doc.rust-lang.org/cargo/reference/semver.html#item-remove
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.46.0/src/lints/struct_missing.ron

Failed in:
  struct taskmill::resource::sampler::SmoothedReader, previously in file /tmp/.tmpv1zNi5/taskmill/src/resource/sampler.rs:41
  struct taskmill::SmoothedReader, previously in file /tmp/.tmpv1zNi5/taskmill/src/resource/sampler.rs:41
  struct taskmill::registry::StateMap, previously in file /tmp/.tmpv1zNi5/taskmill/src/registry.rs:24
  struct taskmill::StateMap, previously in file /tmp/.tmpv1zNi5/taskmill/src/registry.rs:24
```

<details><summary><i><b>Changelog</b></i></summary><p>

<blockquote>

## [0.2.0](v0.1.1...v0.2.0)
- 2026-03-14

### Added

- *(taskmill)* add hierarchical child tasks with two-phase execution
([#2](#2))
- initial release (migrated from deepjoy/shoebox)
([#1](#1))

### Other

- Initial commit
</blockquote>


</p></details>

---
This PR was generated with
[release-plz](https://github.com/release-plz/release-plz/).

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant