Feat/event driven simulation

AGENTS.md

@@ -0,0 +1,64 @@
+# TopoQueue engineering guidance
+
+## Product invariants
+
+- Determinism is a product feature. Identical inputs must produce byte-for-byte stable text and JSON output.
+- TopoQueue is an educational simulator. Never claim Kubernetes scheduler, Kueue, or production-scheduler compatibility.
+- Preserve the behavior and stable output shapes of the existing `schedule` and `compare` commands.
+- Use logical integer ticks for simulations. Do not use wall-clock time, sleeps, randomness, or background timers.
+- Keep state ownership explicit. Independent policy runs must not share mutable scheduling state.
+- Prefer the Go standard library. Do not add a dependency unless the feature cannot reasonably be implemented without it.
+- Reject invalid or ambiguous input early, with contextual errors.
+- Check arithmetic that can overflow.
+- Keep unrelated refactors out of feature commits.
+
+## Repository map
+
+- `cmd/topoqueue/`: CLI parsing and command wiring
+- `internal/model/`: strict YAML models, loading, cloning, and validation
+- `internal/scheduler/`: placement, policies, comparisons, and simulation logic
+- `internal/output/`: deterministic text and JSON renderers
+- `examples/`: runnable scenarios
+- `docs/`: design documents and executable plans
+
+## Verification
+
+After each milestone:
+
+1. Run `gofmt` on changed Go files.
+2. Run focused package tests.
+3. Run `go test ./...` before committing.
+
+Before declaring the branch complete, run:
+
+```sh
+make fmt-check
+make vet
+make test
+make build
+make demo
+make demo-simulate
+make benchmark
+git diff --check main...HEAD
+```
+
+Fix failures before proceeding. Do not weaken, skip, or delete tests merely to make a check pass.
+
+## Git discipline
+
+- Work only on the current feature branch; never commit directly to `main`.
+- Use the repository's existing commit-signing configuration. Do not change Git configuration or disable signing.
+- Create coherent, reviewable commits. Target 8–12 commits for a large milestone, but never split changes merely to increase the count.
+- Every commit must compile and pass `go test ./...`.
+- Do not create empty commits.
+- Do not amend or rewrite commits that existed before this feature branch.
+- Do not push, force-push, tag, create a release, or open/merge a pull request.
+- Finish with a clean worktree.
+
+## ExecPlan
+
+For the event-driven simulation milestone, maintain the living plan at:
+
+`docs/exec-plans/v0.2-event-driven-simulation.md`
+
+Update its progress, decisions, validation evidence, and known limitations as work proceeds.

Makefile

@@ -1,4 +1,4 @@
-.PHONY: build test vet fmt-check demo benchmark
+.PHONY: build test vet fmt-check demo demo-simulate benchmark
 
 BINARY := bin/topoqueue
 
@@ -26,5 +26,12 @@ demo:
 		--jobs examples/jobs.yaml \
 		--output text
 
+demo-simulate:
+	go run ./cmd/topoqueue simulate \
+		--cluster examples/cluster.yaml \
+		--jobs examples/timed-jobs.yaml \
+		--policy all \
+		--output text
+
 benchmark:
 	go test -run '^$$' -bench . ./internal/scheduler

README.md

@@ -5,7 +5,7 @@
 TopoQueue is a deterministic Go CLI for exploring how queue policy and
 topology constraints affect accelerator-job admission.
 
-> TopoQueue is an educational snapshot simulator. It is not a Kubernetes
+> TopoQueue is an educational scheduling simulator. It is not a Kubernetes
 > scheduler and does not implement Kueue semantics.
 
 ## Features
@@ -18,6 +18,8 @@ topology constraints affect accelerator-job admission.
 - Human-readable text and stable, indented JSON output.
 - Concurrent comparison of policies with isolated mutable scheduling state.
 - Strict YAML decoding that rejects unknown fields.
+- Discrete-event workload simulation with logical arrivals, fixed durations,
+  completion-driven resource release, and terminal lifecycle results.
 
 ## Installation
 
@@ -64,6 +66,16 @@ go run ./cmd/topoqueue schedule \
   --output json
 ```
 
+Simulate both policies over the checked-in timed workload:
+
+```sh
+./bin/topoqueue simulate \
+  --cluster examples/cluster.yaml \
+  --jobs examples/timed-jobs.yaml \
+  --policy all \
+  --output text
+```
+
 ## Example comparison
 
 This is the output of `make demo` using the checked-in example files:
@@ -88,6 +100,26 @@ batch     pending   -       -           0/16   0/4   head_of_line_blocked: head_
 
 The `CPU` and `GPU` columns in job decisions show allocated/requested units.
 
+## Event-driven simulation
+
+The `simulate` command uses a separate timed-jobs schema with required
+`arrivalTick` and `durationTicks` fields. It advances directly between logical
+event ticks; it does not sleep or use wall-clock time. At each tick it completes
+and releases all finishing jobs, enqueues all arrivals, and then runs one
+admission cycle.
+
+The checked-in example demonstrates the difference over time:
+
+| Policy | Completed | Unscheduled | Makespan | Total wait | Average wait | Maximum wait |
+|---|---:|---:|---:|---:|---:|---:|
+| backfill | 3 | 0 | 12 | 7 | 2.33 | 7 |
+| strict-fifo | 3 | 0 | 14 | 17 | 5.67 | 10 |
+
+Backfill starts `batch` at tick 2 while `train-xl` waits for the whole cluster.
+Strict FIFO retains `train-xl` at the queue head, so `batch` starts only after
+`train-xl` completes. See [Event-driven simulation](docs/event-driven-simulation.md)
+for the input contract, event ordering, lifecycle fields, and metric definitions.
+
 ## Algorithm
 
 Jobs are considered in YAML order. For each attempted job, TopoQueue computes
@@ -106,6 +138,13 @@ head-of-line blocked. Backfill records an unplaceable job and continues trying
 later jobs. A comparison runs both simulations concurrently, but each policy
 run is single-threaded and owns its scheduling state.
 
+Timed simulation reuses the same placement algorithm. Arrivals are ordered by
+logical tick and input position, and running jobs are ordered by completion tick
+and input position. Completion releases the exact recorded per-node allocation.
+Pending work is retried only at arrival and completion ticks. When neither can
+occur again, remaining jobs become terminally unscheduled with final placement
+reasons.
+
 ## Complexity
 
 Let `N` be the number of nodes, `D` the number of topology domains, and `J` the
@@ -114,17 +153,21 @@ sorting nodes and domain candidates, for a worst-case bound of
 `O(N log N + D log D)` time and `O(N + D)` temporary space. A full backfill run
 is `O(J * (N log N + D log D))`; strict FIFO may stop before all `J` jobs.
 Policy comparison performs two independent runs with the same per-run bounds.
+For a timed workload, heap operations add `O(log J)` per admission/completion.
+One admission cycle may inspect every pending job, and up to `O(J)` event ticks
+can retry pending work, so the deliberately simple worst case is
+`O(J^2 * (N log N + D log D) + J log J)` time and `O(N + D + J)` state.
 
 ## Limitations
 
-- The input is a static snapshot; there is no API server, watch loop, or
-  cluster controller.
+- Cluster capacity is still an input snapshot; there is no API server, watch
+  loop, or cluster controller.
 - Resources are whole, non-negative CPU and GPU units; cluster and per-job
   totals must fit in a signed 64-bit integer.
 - Replicas are identical, cannot be split across nodes, and support at most one
   required topology key per job.
-- There is no preemption, priority, fairness, retry, reservation, or job
-  lifecycle model.
+- Timed jobs have fixed successful durations. There are no failures, failure
+  retries, preemption, priorities, fairness, reservations, or elastic jobs.
 - TopoQueue does not implement Kubernetes scheduling behavior or exact Kueue
   semantics.
 - The simulator is educational and is not intended for production scheduling.
@@ -137,6 +180,7 @@ make vet
 make test
 make build
 make demo
+make demo-simulate
 make benchmark
 ```
 
@@ -149,6 +193,7 @@ internal/scheduler/   policies, placement, comparison, tests, and benchmark
 internal/output/      deterministic text and JSON renderers
 examples/             example cluster and ordered jobs
 docs/design.md        design decisions and deliberate non-goals
+docs/event-driven-simulation.md  timed input, lifecycle, and metric semantics
 .github/workflows/    formatting, vet, race-test, and build CI
 ```
 

cmd/topoqueue/main.go

@@ -18,6 +18,7 @@ import (
 const (
 	outputText = "text"
 	outputJSON = "json"
+	policyAll  = "all"
 )
 
 func main() {
@@ -48,6 +49,8 @@ func run(ctx context.Context, args []string, stdout, stderr io.Writer) error {
 		return runSchedule(ctx, args[1:], stdout, stderr)
 	case "compare":
 		return runCompare(ctx, args[1:], stdout, stderr)
+	case "simulate":
+		return runSimulate(ctx, args[1:], stdout, stderr)
 	case "help", "-h", "--help":
 		return writeRootUsage(stdout)
 	default:
@@ -58,6 +61,75 @@ func run(ctx context.Context, args []string, stdout, stderr io.Writer) error {
 	}
 }
 
+func runSimulate(ctx context.Context, args []string, stdout, stderr io.Writer) error {
+	flags := flag.NewFlagSet("simulate", flag.ContinueOnError)
+	flags.SetOutput(stderr)
+	clusterPath := flags.String("cluster", "", "path to the cluster YAML file")
+	jobsPath := flags.String("jobs", "", "path to the timed jobs YAML file")
+	policyName := flags.String("policy", string(scheduler.PolicyStrictFIFO), "queue policy: strict-fifo, backfill, or all")
+	outputName := flags.String("output", outputText, "output format: text or json")
+	if err := flags.Parse(args); err != nil {
+		if errors.Is(err, flag.ErrHelp) {
+			return nil
+		}
+		return fmt.Errorf("parse simulate flags: %w", err)
+	}
+	if flags.NArg() != 0 {
+		return fmt.Errorf("simulate: unexpected positional arguments: %v", flags.Args())
+	}
+	if err := requireInputPaths(*clusterPath, *jobsPath); err != nil {
+		return fmt.Errorf("simulate: %w", err)
+	}
+	if err := validateSimulationPolicy(*policyName); err != nil {
+		return fmt.Errorf("simulate: %w", err)
+	}
+	if err := validateOutputFormat(*outputName); err != nil {
+		return fmt.Errorf("simulate: %w", err)
+	}
+
+	cluster, jobs, err := loadSimulationInputs(*clusterPath, *jobsPath)
+	if err != nil {
+		return err
+	}
+	if *policyName == policyAll {
+		results, err := scheduler.CompareSimulations(ctx, cluster, jobs)
+		if err != nil {
+			return fmt.Errorf("simulate all policies: %w", err)
+		}
+		switch *outputName {
+		case outputText:
+			if err := output.WriteSimulationComparisonText(stdout, results); err != nil {
+				return fmt.Errorf("render simulation comparison text output: %w", err)
+			}
+		case outputJSON:
+			if err := output.WriteSimulationComparisonJSON(stdout, results); err != nil {
+				return fmt.Errorf("render simulation comparison JSON output: %w", err)
+			}
+		}
+		return nil
+	}
+
+	policy, err := scheduler.ParsePolicy(*policyName)
+	if err != nil {
+		return fmt.Errorf("simulate: %w", err)
+	}
+	result, err := scheduler.Simulate(ctx, cluster, jobs, policy)
+	if err != nil {
+		return fmt.Errorf("simulate policy %q: %w", policy, err)
+	}
+	switch *outputName {
+	case outputText:
+		if err := output.WriteSimulationText(stdout, result); err != nil {
+			return fmt.Errorf("render simulation text output: %w", err)
+		}
+	case outputJSON:
+		if err := output.WriteSimulationJSON(stdout, result); err != nil {
+			return fmt.Errorf("render simulation JSON output: %w", err)
+		}
+	}
+	return nil
+}
+
 func runSchedule(ctx context.Context, args []string, stdout, stderr io.Writer) error {
 	flags := flag.NewFlagSet("schedule", flag.ContinueOnError)
 	flags.SetOutput(stderr)
@@ -171,6 +243,26 @@ func loadInputs(clusterPath, jobsPath string) (model.Cluster, model.JobSet, erro
 	return cluster, jobs, nil
 }
 
+func loadSimulationInputs(clusterPath, jobsPath string) (model.Cluster, model.TimedJobSet, error) {
+	cluster, err := model.LoadCluster(clusterPath)
+	if err != nil {
+		return model.Cluster{}, model.TimedJobSet{}, err
+	}
+	jobs, err := model.LoadTimedJobs(jobsPath)
+	if err != nil {
+		return model.Cluster{}, model.TimedJobSet{}, err
+	}
+	if err := model.ValidateTopologyRequirements(cluster, jobs.StaticJobs()); err != nil {
+		return model.Cluster{}, model.TimedJobSet{}, fmt.Errorf(
+			"validate cluster file %q against timed jobs file %q: %w",
+			clusterPath,
+			jobsPath,
+			err,
+		)
+	}
+	return cluster, jobs, nil
+}
+
 func requireInputPaths(clusterPath, jobsPath string) error {
 	if clusterPath == "" {
 		return errors.New("--cluster is required")
@@ -190,8 +282,18 @@ func validateOutputFormat(value string) error {
 	}
 }
 
+func validateSimulationPolicy(value string) error {
+	if value == policyAll {
+		return nil
+	}
+	if _, err := scheduler.ParsePolicy(value); err != nil {
+		return fmt.Errorf("unsupported policy %q (supported: %s, %s, %s)", value, scheduler.PolicyStrictFIFO, scheduler.PolicyBackfill, policyAll)
+	}
+	return nil
+}
+
 func writeRootUsage(w io.Writer) error {
-	if _, err := fmt.Fprintln(w, "Usage: topoqueue <schedule|compare> [flags]"); err != nil {
+	if _, err := fmt.Fprintln(w, "Usage: topoqueue <schedule|compare|simulate> [flags]"); err != nil {
 		return err
 	}
 	if _, err := fmt.Fprintln(w, "Run 'topoqueue <command> -h' for command flags."); err != nil {